Assessing the Validity and Reliability of a Questionnaire on Dietary Fibre-related Knowledge in a Turkish Student Population

This study aimed to validate a questionnaire on dietary fibre (DF)-related knowledge in a Turkish student population. Participants (n=360) were either undergraduate students who have taken a nutrition course for 14 weeks (n=174) or those in another group who have not taken such a nutrition course (n=186). Test-retest reliability, internal reliability, and construct validity of the questionnaire were determined. Overall internal reliability (Cronbach's alpha=0.90) and test-retest reliability (0.90) were high. Significant differences (p<0.001) between the scores of the two groups of students indicated that the questionnaire had satisfactory construct validity. It was found that one-fifth of the students were unsure of the correct answer for any item, and 52.5% of them were not aware that DF had to be consumed on a daily basis. Only 36.4 to 44.2% of the students were able to correctly identify the food sources of DF.


INTRODUCTION
Dietary fibre (DF) is deemed to be a key component in a healthful diet (1). DF consumption may be important in the prevention or control of cancer, diabetes mellitus, hypertension, hyperlipidaemia, and other health problems (2). There are no accurate data on the daily intake of DF in Turkey. Various studies have reported that the intake of DF by adolescents (3) and university students (4-6) fell below the daily recommended level. The fruit and vegetable consumption by Turkish adults was also found to be inadequate (1.6 portions of fruit and 1.6 portions of vegetables per day) (7).
Consumers need to enhance their knowledge and competence as informed consumers are able to perform their food choices in a complex environment with a wide variety of foods available. They need to be aware of not only the nutritional recommendations and basic food-based guidelines but also how to apply this knowledge in their food choices, diet, Turkey. The population in this study included two groups of students: one group had taken a general nutrition course for 14 weeks as a requirement of their curriculum (n=180), and the other group was composed of engineering students (studying computer, electrical-electronics, civil and mechanical engineering) who had not taken such a nutrition course or any other courses relevant to nutrition (n=212). Of them, 174 students who took nutrition courses and 186 engineering students who did not take such courses (total of 360 students) gave written informed consent to take part in the study. The general nutrition course included the function, structure, and sources of macro-and micronutrients, and general principles of healthy eating. Engineering students did not have any specialization in knowledge on nutrition. This ensured that one group had greater knowledge on nutrition.

Settings
The questionnaire was developed in 2012 in Istanbul, Turkey. Istanbul is one of the largest cities in the world (with a population of approximately 17 million), and it serves as Turkey's economical, social and cultural centre. The university that the study was carried out in is a foundation university and ranks among the most prestigious universities in Turkey.

Ethical approval
The Research Evaluation Committee of the Department of Nutrition and Dietetics, Yeditepe University, approved the study.

Questionnaire development and testing
On the basis of review of the current material containing dietary advice and the literature linking DF and health, it was decided to divide the questionnaire into two sections. The first section (Section A) included statements about the relationship of DF and health, and the second section (Section B) included statements on the knowledge of DF content in some foods. Each statement had three possible responses (True/False/Not sure). A pool of 46 statements that belonged to these two sections was generated. Using this pool of statements, two reviews were carried out by a panel of three dieticians to select the best in terms of clarity of the questions, accuracy of the knowledge measured, and interpretability. This process reduced the number of statements to 20. After item analysis, the total number of statements in the final questionnaire was 18 (i.e. 9 in each section). Demographic questions on gender and age were included at the end of the questionnaire. The responses to the questionnaire took 3 to 5 minutes.
The questionnaire was self-administered in groups at the end of the lectures under the supervision of the authors. Some students completed the questionnaire on two separate occasions that were two weeks apart. The period of two weeks was considered long enough for participants to have forgotten their responses but not long enough for a real change to occur in their knowledge on nutrition (15)(16)(17). Participants were not informed of the second administration of the questionnaire on the first occasion.
The responses in the first administration were used in assessing construct validity and internal consistency reliability. Two sets of responses (i.e. the first and the second administration) were used in measuring test-retest reliability.

Item analysis
Item analysis involves statistical analysis of the results of a test administration to identify which items can be retained and which need to be discarded (18). The results were analyzed for difficulty with and discrimination of the items, and only those items meeting the analysis criteria were retained.

Difficulty with item
The item difficulty index indicates the percentage of respondents who answer an item correctly. Since difficulty refers to the percentage of getting the item right, the smaller the percentage figure, the more difficult the item is (18). According to Kline (19), items are not useful if they are answered correctly by more than 80% or fewer than 20% of the respondents. Items, therefore, were rejected if over 80% or under 20% of the respondents answered them correctly.

Item discrimination
Item discrimination is the ability of an individual item to discriminate between those who do well on the measure and those who do not (16). An itemto-total score correlation of 0.20 has been given as the cutoff point below which items should be discarded (16,19,20), and, therefore, only the items that met the correlation of 0.20 or higher were retained.

Final knowledge test
Those items that met the item analysis criteria formed the final knowledge questionnaire.
Construct validity, internal consistency reliability, and test-retest reliability of the final questionnaire were assessed.

Construct validity
Validity is the extent to which a test measures what it is intended to measure (8). One of the easiest ways to assess construct validity is to give the measure to two groups, one of which is known to have higher knowledge on nutrition than the other group (16).

Internal consistency reliability
The tendency towards consistency found in repeated measurements of the same phenomenon is referred to as reliability (21). Internal consistency refers to the extent to which all of the items in a scale measure the different aspects of the same attribute (22). Cronbach's alpha is often used in assessing the reliability of tests for knowledge on nutrition, with questions that have more than two possible responses (8). Cronbach's alpha ranges from r=0 to 1, with r=0.7 or greater considered as sufficiently reliable (18).

Test-retest reliability
Test-retest reliability involves administering the same measure to the same group of test-takers under the same conditions on two different occasions and correlating the scores (23). The reliability coefficient is simply the correlation (usually a Pearson's correlation) between the scores on the first and the second testing (17). The value for a Pearson's coefficient can fall between 0.00 (no correlation) and 1.00 (perfect correlation).

Data collection and analysis
The raw data from responses of each participant were coded numerically. The responses were converted to 1 and 0 for correct and incorrect answers respectively ('not sure' responses were also coded as incorrect). Data were entered and analyzed using the Statistical Package for Social Sciences (version 17.0) (SPSS Inc., Chicago, IL, USA). A number of statistical tests were performed to assess the validity and reliability of the questionnaire. Student's t-test was employed to determine significant differences between the group means (p<0.001). Chisquare test was used in establishing whether the distribution of age and gender differed significantly between the two groups (p<0.05). The effect of the differences in age and gender on the results was checked by an analysis of covariance (ANCOVA), with significance level set at p<0.05.

RESULTS
Compliance was high (90%) for the first test, with almost all students completing the questionnaire. Three hundred and sixty students completed the questionnaire at least once (174 students who took nutrition courses and 186 students who did not). Of them, 202 students (88 students who took nutrition courses and 114 students who did not) completed the questionnaire twice. There was a significant gender difference between the two groups, with 97.1% of the students who took nutrition courses and 67.2% of the students who did not take nutrition courses being female (p=0.000) ( Table 1). This was due to the fact that majority of the students who took nutrition courses came from the departments that were dominated by females (such as nutrition and dietetics and nursing and health services). Majority of the students in both groups (70.7% of the students who took nutrition courses and 75.3% of the other group) were aged between 21 and 24 years. Difference in age between the two groups was significant (p=0.003). Table 2 shows that the students who studied nutrition scored consistently higher than the stu- dents in the other group on both sections of the questionnaire (p<0.001). The students who studied nutrition had an average score (mean±SD) of 12.7±7.4, and the other group had an average score (mean±SD) of 7.4±4.5 out of a maximum score of 18. The scores ranged from 2 to 18 in the group of students who studied nutrition and 0 to 16 in the other group. The students who studied nutrition answered more questions correctly in both Section A (6.5±1.8) and Section B (6.1±1.9) compared to the other group (3.5±2.4 and 4.0±2.5 for Section A and B respectively).

Construct validity
Given the different apparent gender balance of the two groups, gender was controlled for in an analysis of covariance but this did not have a significant effect on the results. When the same analysis was carried out for age, it was found that the difference in age did not affect the results significantly. The questionnaire, therefore, met the criterion for construct validity.

Internal consistency reliability
Cronbach's alpha values in the two groups were different ( Table 3). The values (individual sections and as cumulative) for the students who did not take nutrition courses were higher than for the other group. When all students were considered, the alpha values for both Section A and B were 0.83. The alpha value for the overall questionnaire was 0.90.

Test-retest reliability
Pearson's correlation coefficient was calculated on the scores of the participants (88 students who studied nutrition and 114 students who did not study nutrition) who completed the questionnaire twice. The correlation coefficients varied across the sections and the two student groups (ranging from 0.69 to 0.87). The overall reliability was high (r=0.90, p<0.001) ( Table 3).

Responses
The most correctly-answered questions (66.4 to 67.2%) in Section A (DF and health) were about the effects of DF on health [i.e. prevention of colon cancer (A3), helping maintain body-weight (A4) and bowel movements (A7)] (Table 4). Four questions [(i.e. DF is a carbohydrate in foods (A1), DF has high energy content (A2), the intake of DF by

DISCUSSION
Significant differences between the scores of the students who had taken a nutrition course for 14 weeks and the ones who had not taken this course indicated that the questionnaire had satisfactory construct validity, even when taking into account the skewed gender and age characteristics of the two groups. The students who had taken the nutrition course scored higher on both sections of the questionnaire. These students gave more correct answers for each statement (ranging from 51 to 72%) (data not shown). Majority of the "Not sure" responses were given by the group who had not taken the course (ranging from 61 to 90%) (data not shown). Therefore, it could be concluded that the nutrition course improved the knowledge level of the students who had taken the course.
The overall test-retest reliability value (0.90) was high, and this suggested that the questionnaire measured the knowledge of DF consistently over time. This value was slightly higher than in the studies that employed a general nutrition knowledge questionnaire in Australia (0.87) and Turkey (0.86) and lower than the value obtained for the same questionnaire in the UK (0.98) (15,24,25).
Nutrition scholars prefer internal reliability values of 0.70 or greater as an indication that a test is sufficiently reliable for measuring knowledge and knowledge structures (26). Cronbach's alpha values ranging from 0.5 to 0.8 (mostly between 0.6 and 0.7) were reported for studies that measured knowledge on nutrition (8,27,28). The internal reliability value of the questionnaire (0.90) was higher than these values and, therefore, the questionnaire could be considered a reliable tool to measure the knowledge on DF. This value is very similar to the values obtained from validation studies of a general nutrition knowledge questionnaire in Australia (25) and Turkey (24) (0.92 and 0.89, respectively) and lower than the UK study (0.97) (15).
Searching for studies that assessed knowledge on DF in Turkey produced only one result. However, a modified questionnaire was used in that study (13), and no statistical data on reliability and validity were presented. Therefore, it was not possible to compare the current results with those of that study.
One implication of this research is that a high number of students were unaware of the dietary recommendation about DF (item A5), and many students had a poor knowledge of the food sources of DF (item B4, B5, B6, and B7). It is strongly believed that these students were not able to make healthful food choices and meet the daily recommendation for DF due to their lack of knowledge. Without sufficient knowledge, individuals are not able to translate knowledge into the adoption of healthier food habits (29). While there are no data to compare our findings, studies have shown that university students can increase their knowledge on nutrition by taking relevant courses (30)(31)(32). However, further studies are necessary to verify this possible relationship between the knowledge and consumption of DF. This could be achieved by asking the participants to keep 3-day food diaries in addition to the employment of the DF questionnaire.

Limitations
There are limitations in this research, and the main one concerns the sample selection. The sample was based on convenience. Due to the nature of the group that the sample was selected from (i.e. students studying dietetics and nursing), majority of the students were female. However, gender (as a potential confounding variable) was controlled for and did not have any significant effect on the results.
It is generally agreed that educated people demonstrate significantly better knowledge on nutrition (22,33). It is tempting to suggest that if this questionnaire was used in exploring the knowledge level of the general public on DF, more questions would be incorrectly answered by the participants.

Conclusions
The questionnaire assessed in this study proved to be a valid and reliable tool to measure the knowledge on DF. Since it is easy to understand and it can be completed by the participants in a short time, it can be used in determining the knowledge level of other sub-populations or the general public more readily and consistently. Implementing the questionnaire would probably not increase knowledge on DF in the Turkish community but it could be used in identifying areas where people are the most deficient. This information could be used in developing publichealth nutrition efforts to improve dietary habits and, in turn, influence nutrition-related disease risk.