Statistical approaches for assessing the relative validity of a food-frequency questionnaire: use of correlation coefficients and the kappa statistic

By : LF Masson1,G MCNeill2,*, JO Tomany2, JA Simpson3, HS Peace2,L Wei2, DA Grubb4 and C Bolton-Smith5

Abstract

Objective: To compare different statistical methods for assessing the relative validity of a self-administered,  150-item, semi-quantitative  food-frequency questionnaire (FFQ) with 4-day weighed  diet records  (WR). Design: Subjects completed the Scottish Collaborative Group FFQ and carried out a 4- day WR. Relative agreement  between the FFQ and WR for energy-adjusted nutrient takes was assessed by Pearson and Spearman rank correlation coefficients, the percentages of subjects  classified into the same and  opposite thirds of intake,  and Cohen’s weighted  kappa. Subjects: Forty-one men, mean age 36 (range 21 – 56) years, and 40 women, mean age 33 (range 19 – 58) years, recruited  from different locations in Aberdeen,  Scotland. Results:  Spearman   correlation   coefficients   tended   to   be   lower   than   Pearson correlation  coefficients,  and  were  above  0.5 for 10 of the 27 nutrients  in men  and 17 of the 27 nutrients in women.  For nutrients with Spearman correlation  coefficients above 0.5, the percentage of subjects correctly classified into thirds ranged from 39 to  78%, and weighted  kappa  values ranged  from 0.23 to 0.66. 

Conclusions: Both Spearman correlation  coefficients and weighted  kappa  values are useful  in  assessing  the  relative  validity  of  estimates  of  nutrient  intake  by  FFQs. Spearman  correlation  coefficients  above  0.5, more  than  50% of subjects  correctly classified and less than 10% of subjects grossly misclassified into thirds, and weighted kappa  values above 0.4 are recommended for nutrients of interest in epidemiological studies.