Validation Information: Scottish Collaborative Group FFQ
Masson 2003
Statistical approaches for assessing the relative validity of a food-frequency questionnaire: use of correlation coefficients and the kappa statistic
Objective: To compare different statistical methods for assessing the relative validity of a self-administered, 150-item, semi-quantitative food-frequency questionnaire (FFQ) with 4-day weighed diet records (WR).
Design: Subjects completed the Scottish Collaborative Group FFQ and carried out a 4-day WR. Relative agreement between the FFQ and WR for energy-adjusted nutrient intakes was assessed by Pearson and Spearman rank correlation coefficients, the percentages of subjects classified into the same and opposite thirds of intake, and Cohen's weighted kappa.
Subjects: Forty-one men, mean age 36 (range 21-56) years, and 40 women, mean age 33 (range 19-58) years, recruited from different locations in Aberdeen, Scotland.
Results: Spearman correlation coefficients tended to be lower than Pearson correlation coefficients, and were above 0.5 for 10 of the 27 nutrients in men and 17 of the 27 nutrients in women. For nutrients with Spearman correlation coefficients above 0.5, the percentage of subjects correctly classified into thirds ranged from 39 to 78%, and weighted kappa values ranged from 0.23 to 0.66.
Conclusions: Both Spearman correlation coefficients and weighted kappa values are useful in assessing the relative validity of estimates of nutrient intake by FFQs. Spearman correlation coefficients above 0.5, more than 50% of subjects correctly classified and less than 10% of subjects grossly misclassified into thirds, and weighted kappa values above 0.4 are recommended for nutrients of interest in epidemiological studies.
Total number of nutrients validated: 25
Not all of the nutrients validated in the validation studies are included in the table below, as statistical data was only selected to be displayed for a number of nutrients, this included:
- Energy
- Fat
- Saturated Fat
- Mono-unsaturated Fat
- Poly-unsaturated Fat
- Carbohydrates
- Protein
- Sugar
- Non‐starch polysaccharides(NSP)
- Sodium
- Calcium
- Iron
- Zinc
- Retinol
- Folate
- Folic Acid
- Vitamin B12
- Vitamin C
- Fruit & Vegetables
- Urinary Nitrogen
To find information on the other validated nutrients please read the validation study.
- Energy
- Macronutrients: 9
- Micronutrients: 15
| Comparator | Lifestage | Sex | Nutrient Measured | Mean Difference | Standard Deviation | Correlation Coefficient | Cohen's Kappa Coefficient | Percentage Agreement | Percentage Agreement Categories | Lower Limits of Agreement | Upper Limits of Agreement |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Weighed Food Diary | Adults | Male Only | Energy (kcal) | 511 (Median) | 0.24 (S); 0.35 (P) | 0.12 (w) | 34 | 3 | |||
| Energy (kJ) | 2137 (Median) | 0.24 (S); 0.35 (P) | 0.12 (w) | 34 | 3 | ||||||
| Protein (g) | 21 (Median) | 0.25† (S); 0.53‡ (P) | 0.08 (w) | 34 | 3 | ||||||
| Fat (g) | 19 (Median) | 0.42† (S); 0.54‡ (P) | 0.21 (w) | 44 | 3 | ||||||
| Saturated Fat (g) | 12 (Median) | 0.59† (S); 0.55‡ (P) | 0.38 (w) | 51 | 3 | ||||||
| MUFA (g) | 9 (Median) | 0.36† (S); 0.33‡ (P) | 0.16 (w) | 37 | 3 | ||||||
| PUFA (g) | 0 (Median) | -0.07† (S); 0.08‡ (P) | -0.14 (w) | 22 | 3 | ||||||
| Total Sugars (g) | 53 (Median) | 0.17† (S); 0.41‡ (P) | 0.08 (w) | 37 | 3 | ||||||
| Fibre (NSP) (g) | 6 (Median) | 0.65† (S); 0.64‡ (P) | 0.43 (w) | 54 | 3 | ||||||
| Calcium (mg) | 384 (Median) | 0.49† (S); 0.52‡ (P) | 0.39 (w) | 54 | 3 | ||||||
| Iron (mg) | 2.1 (Median) | 0.60† (S); 0.63‡ (P) | 0.43 (w) | 56 | 3 | ||||||
| Zinc (mg) | 5.3 (Median) | 0.54† (S); 0.54‡ (P) | 0.23 (w) | 39 | 3 | ||||||
| Retinol (µg) | 237 (Median) | -0.13† (S); -0.49‡ (P) | -0.03 (w) | 29 | 3 | ||||||
| Folate (µg) | 74 (Median) | 0.55† (S); 0.56‡ (P) | 0.29 (w) | 46 | 3 | ||||||
| Vitamin B12 (µg) | 2.9 (Median) | 0.25† (S); 0.24‡ (P) | 0.23 (w) | 51 | 3 | ||||||
| Vitamin C (mg) | 46.6 (Median) | 0.48† (S); 0.64‡ (P) | 0.15 (w) | 37 | 3 | ||||||
| Weighed Food Diary | Adults | Female Only | Energy (kcal) | 310 (Median) | 0.39 (S); 0.40 (P) | 0.37 (w) | 58 | 3 | |||
| Energy (kJ) | 1298 (Median) | 0.39 (S); 0.40 (P) | 0.37 (w) | 58 | 3 | ||||||
| Protein (g) | 26 (Median) | 0.43† (S); 0.51‡ (P) | 0.37 (w) | 50 | 3 | ||||||
| Fat (g) | 12 (Median) | 0.64† (S); 0.83‡ (P) | 0.37 (w) | 53 | 3 | ||||||
| Saturated Fat (g) | 8 (Median) | 0.71† (S); 0.81‡ (P) | 0.54 (w) | 65 | 3 | ||||||
| MUFA (g) | 7 (Median) | 0.66† (S); 0.86‡ (P) | 0.26 (w) | 43 | 3 | ||||||
| PUFA (g) | 2 (Median) | 0.58† (S); 0.68‡ (P) | 0.43 (w) | 55 | 3 | ||||||
| Total Sugars (g) | 44 (Median) | 0.72† (S); 0.72‡ (P) | 0.54 (w) | 63 | 3 | ||||||
| Fibre (NSP) (g) | 7 (Median) | 0.76† (S); 0.73‡ (P) | 0.54 (w) | 60 | 3 | ||||||
| Calcium (mg) | 267 (Median) | 0.75† (S); 0.78‡ (P) | 0.60 (w) | 70 | 3 | ||||||
| Iron (mg) | 2.2 (Median) | 0.54† (S); 0.64‡ (P) | 0.43 (w) | 58 | 3 | ||||||
| Zinc (mg) | 3.9 (Median) | 0.57† (S); 0.61‡ (P) | 0.43 (w) | 55 | 3 | ||||||
| Retinol (µg) | 123 (Median) | 0.34† (S); 0.37‡ (P) | 0.20 (w) | 45 | 3 | ||||||
| Folate (µg) | 82 (Median) | 0.73† (S); 0.78‡ (P) | 0.60 (w) | 68 | 3 | ||||||
| Vitamin B12 (µg) | 1.15 (Median) | 0.31† (S); 0.39‡ (P) | 0.20 (w) | 43 | 3 | ||||||
| Vitamin C (mg) | 22.1 (Median) | 0.59† (S); 0.68‡ (P) | 0.43 (w) | 60 | 3 |
Some results have been calculated using statistical techniques based on the published data.
For further information on statistical terms click on Statistical tests used in validation studies
All correlations coefficients in the table are unadjusted unless stated otherwise. For adjusted correlation coefficients and other statistical methods used in the study e.g. paired t-tests, please read the validation articles.
- # Adjusted
- † Energy adjusted.
- ‡ For loge-transformed, energy-adjusted nutrient intakes.
- ^ Adjacent included.
- ᵟ Participants provided identical responses.
- (w) = Weighted.
Masson LF, McNeill G, Tomany JO, Simpson JA, Peace HS, Wei L et al. Statistical approaches for assessing the relative validity of a food-frequency questionnaire: use of correlation coefficients and the kappa statistic. Public health nutrition. 2003 Jun 1;6(03):313-21.