reliability and validity of digital imaging as a measure of schoolchildren's fruit and...

ª 2014 by the Academy of Nutrition and Dietetics.

RESEARCH
Original Research
Reliability and Validity of Digital Imaging as a Measureof Schoolchildren’s Fruit and Vegetable ConsumptionJennifer C. Taylor, MS; Bethany A. Yon, PhD; Rachel K. Johnson, PhD, MPH, RD

ARTICLE INFORMATION

Article history:Accepted 27 February 2014

Keywords:School nutritionDietary assessmentFruits and vegetablesReliabilityValidity

Copyright ª 2014 by the Academy of Nutritionand Dietetics.2212-2672/$36.00http://dx.doi.org/10.1016/j.jand.2014.02.029

ABSTRACTBackground As more and more interventions aim to increase schoolchildren’s fruit andvegetable (FV) consumption, less resource-intensive yet valid alternatives to weighedplate waste (WPW) are needed for assessing dietary intake.Objectives We aimed to test the reliability and validity of digital imaging (DI) anddigital imaging with observation (DIþO) in assessing children’s FV consumption duringschool lunch.Design FV consumption (in grams) was assessed on lunch trays from third- to fifth-grade children over eight visits (31 to 68 trays collected per visit) to compare WPWwith DI and DIþO.Setting Two elementary schools (327 and 631 students enrolled, respectively).Main outcome measures Interobserver reliability of DI. Validity of DI and DIþOcompared against WPW.Statistical analyses Reliability was assessed by percent agreement and intraclasscorrelation coefficients (ICCs). Validity was assessed by Pearson correlations, paired ttests, and Bland-Altman plots.Results Reliability was acceptable for DI; percent agreement was 96% and the ICC was0.92. FV consumption assessments by DI and WPW (n¼159) were highly correlated(r¼0.96; P<0.001). Mean FV consumption using DI (96.7 g) was within 1.0 g of WPWand not significantly different from WPW (P¼0.56), and Bland-Altman limits of agree-ment for individual-tray FV consumption were e32.9 to 31.3 g. FV consumption as-sessments by DIþO and WPW were highly correlated (r¼0.98; P<0.001). Mean FVconsumption using DIþO (99.3 g) was within 1.0 g of WPW and not significantlydifferent from WPW (P¼0.38), and limits of agreement for individual-tray FV con-sumption were e25.0 to 26.8 g.Conclusions DI was reliable for assessing children’s FV consumption during schoollunch. DI and DIþO were valid for assessing mean consumption but less precise forestimating individual-tray consumption. Valid estimations of mean FV consumptionwere achieved using DI without cafeteria observations, thereby reducing labor and time.Thus, DI is especially promising for assessing children’s mean FV consumption duringschool lunch.J Acad Nutr Diet. 2014;-:---.

MOST US CHILDREN FAIL TO MEET NATIONALrecommendations for fruit and vegetable (FV)consumption.1,2 Low FV consumption is con-cerning because FV consumption may reduce

the risk of some cancers3,4 and cardiovascular disease,5,6

and plays an important role in achieving or maintaining ahealthy weight.7,8 In an effort to address childhood obesityand improve childhood nutrition, the US Department of Agri-culture released new school meal standards in 2012 for theNational School Lunch and School Breakfast Programs.9 Theseregulations increase the number of servings of FV and the va-riety of vegetable items offered and require children to selectat least one fruit (breakfast) or one fruit or vegetable (lunch)as a component of the meal.9 However, requiring children toselect FV does not guarantee consumption. To determine theinfluence of these school meal standards, as well as other

interventions targeting FV consumption, practical yet reliableand valid methods for assessing children’s consumption ofthese foods are needed.Various dietary assessment methods are used to estimate

children’s dietary intake in the school environment. Methodsrelying on self-report are commonly used, although theirlimitations are well documented.10-12 Many of these limita-tions may be overcome using an objective measure based ontrained researchers’ meal observations. Foods may beweighed individually using weighed plate waste (WPW), amethod that provides reliable estimates of intake by physi-cally weighing food selections and plate waste13 and isrecognized as the gold standard for meal observationsbecause it is the most accurate and precise approach.14

However, this approach is used infrequently because it istime- and labor-intensive.14

JOURNAL OF THE ACADEMY OF NUTRITION AND DIETETICS 1

http://dx.doi.org/10.1016/j.jand.2014.02.029

RESEARCH

Alternatively, food consumption can be visually estimatedusing digital imaging (DI); DI may be advantageous because itrequires less in-school data collection time than WPW anddirect observation. DI was validated against WPW in labora-tory15,16 and restaurant settings.17 Although DI has been usedin recent studies to estimate children’s dietary intake in schoolcafeterias,18-20 the method has not been validated againstWPW in this setting. In addition, whereas some DI proceduresused in school cafeterias have reported incorporating obser-vations of children’s second servings of foods,20,21 earlierstudies have not reported collecting observations of tradingandsharingof foods. Theextent towhich thesebehaviors affectchildren’s true FV consumption needs to be explored todeterminewhether the absence of cafeteria observations in DIprotocols affects the validity of dietary intake estimations.The purpose of this study was to develop a less resource-

intensive, valid method for objectively assessing children’sFV consumption during school lunch. The aims of this studywere to evaluate the reliability of DI in two unique schoolenvironments and determine the validity of DI as well asdigital imaging with cafeteria observations (DIþO) comparedagainst WPW.

METHODSStudy DesignTwo Vermont elementary schools participated in the studyand were selected because they served a wide variety of FVitems with varied preparation and serving styles. The twoschool cafeteria environments differed in several ways thatwere expected to affect data collection, including layout ofthe foodservice area, FV menu offerings, and lunch disposal(Table 1). Written permission was obtained from the schoolnutrition directors and principals. The study was approved asexempt research by the University of Vermont InstitutionalReview Board.Nineteen undergraduate students completed a 12-hour

training program developed and facilitated by two graduatenutrition students during fall 2011. Students were trained asresearch associates to assess FV consumption using mealobservation methods. Research associates practiced thesemethods using a variety of school lunch FV items presentedin variable portion sizes. During the final training session, 10volunteers were observed eating lunch in the laboratory andFV consumption was estimated in real time. Interobserverreliability was evaluated during this school lunch simulation;percent agreement was 94% and the intraclass correlationcoefficient (ICC) was 0.89 (95% CI 0.78 to 0.97).After completing the training, the research team collected

school lunch FV consumption data in two elementary schoolsover eight school visits (four visits per school) during spring2012. Reliability was evaluated during all school visits inwhich DI was used. Methods were initially tested separatelyin each school; DI was the only method used during the firstvisit to each school, followed by WPW during the secondvisits. To test validity, trays were assessed using DI, DIþO, andWPW simultaneously during the remaining two visits to eachschool.

Data CollectionData were collected on the contents of a random sample oflunch trays from third- to fifth-grade students with no

2 JOURNAL OF THE ACADEMY OF NUTRITION AND DIETETICS

identifying information collected about students. During eachschool visit, stickers in a variety of colors, each with a uniquecode, were adhered to all lunch trays. One color was observedacross all methods used at that visit. The code enabledtracking of each tray throughout the observation period.Average weights of preportioned FV items were determinedbased on a random sample of five servings of each item.When items could be self-served in variable portion sizes (eg,leafy greens from a salad bar), weight was determined for astandard serving size. The research team collected data on FVselections as children exited the lunch line and on FV con-sumption after trays were collected at the end of the meal.Cafeteria observations of trading, sharing, or seconds werecollected while children ate lunch in the cafeteria.Cafeteria observations were collected by three to six

research associates circulating the cafeteria observing mul-tiple tables. Research associates were familiarized with FVitems offered and average serving sizes before collectingobservations. When FV items were added or removed fromdesignated trays, the tray number, type, and quantity of FVitem(s) were recorded. This included instances where chil-dren with designated trays left a table during the meal todispose of items or obtain second servings. These observa-tions were applied to data collected using both WPW andDIþO, but not DI, to update consumption estimates.Procedures for collecting data on FV selections and con-

sumption are described below for each method. For allmethods, if the selected FV item had an inedible component(eg, apple cores, orange peels, and soup containers), theinedible component was expected to be present on thecollected tray. If the inedible component was missing, a codewas used to indicate that FV consumption could not bedetermined because of missing data.

Weighed Plate Waste. Children’s FV selections wererecorded at the beginning of the meal before children exitedthe lunch line. FV items served in preportioned sizes werecounted and FV items served in variable portion sizes wereweighed to the nearest gram. With the exception of self-served leafy greens available in the school with a salad bar,all FV items in each school were available as preportionedservings or countable pieces (eg, cucumber wheels and babycarrots). At the end of the meal, trays were collected andplate waste weighed for each item. FV consumption for eachitem was determined by subtracting grams plate waste fromgrams selected.

Digital Imaging. All digital images were taken with Pow-erShot ELPH 300 HS digital cameras (Canon) held atapproximately 75� and 18 to 24 in above the lunch tray,without the assistance of tripods. Images of food selectionswere taken while children held their trays. Post-meal imagesof plate waste were taken on a flat surface with obstructionssuch as napkins and utensils removed.Two research associates independently completed visual

estimations for each school visit where DI was used bycomparing images of selection and plate waste side-by-sideon a computer monitor and were blinded to the other’s es-timates (Figure 1). Reference images of the FV items offeredduring each school visit were available. FV items served inpreportioned sizes were counted. The volumes of FV itemsserved in variable portion sizes were estimated to the nearest

-- 2014 Volume - Number -

Table 1. Characteristics of two Vermont elementary schools participating in a study comparing weighed plate waste to digitalimaging and digital imaging with observation for the assessment of children’s fruit and vegetable consumption during schoollunch

Characteristic School A School B

Total enrollment 631 327

Grades enrolled 3-6 K-5

Race/ethnicity (%)

White, non-Hispanic 91 84

Asian/black/Hispanic/othera 9 16

Free/reduced meal eligibility (%) 35 52

NSLPb participation rate (%) 56 68

Cafeteria environment

Fruit and vegetable serving style All items preportioned by staff Some items preportioned by staff;self-serve items available from salad bar

Layout of foodservice checkoutareas

6 registers within 1 centralizedcheckout station

1 register at each of 2 checkoutstations

Layout of disposal areas Trash barrels dispersed in dining areanear tables

1 centralized disposal area

aOther ethnicities were American Indian/Alaskan Native.bNSLP¼National School Lunch Program.

RESEARCH

1/4 cup. At the end of the meal, the FV percentage consumedfor each item was estimated using Comstock’s 6-point scale(none¼0%, taste¼10%, some¼25%, half¼50%, most¼75%, andall¼100%).14 To allow DI estimations of grams consumed to becompared with WPW measurements, FV selections wereconverted to grams (based on average weights of prepor-tioned items and grams per cup of FV items served in variableportion sizes) and multiplied by the percentage consumed.The two research associates’ estimations of grams consumedwere averaged for each FV item. Adapting methods used bySwanson,22 any substantial disagreement between associates’estimations of the FV percentage consumed (defined inthis study as differences >25%) were assessed by a third

Figure 1. Digital images of schoolchildren’s lunch tray selections (fruit and vegetable consumption during lunch.


associate; DI estimations were averaged for the two associ-ates with the closest ratings.

Digital Imaging with Observation. DIþO was estimatedby combining the FV consumption data collected using DIwith the cafeteria observations of trading, sharing, and sec-onds of FV items.

Outcome MeasuresA tray was included in the final sample collected if it wasobserved both at the beginning of the meal with data on FVselection and at the end of the meal with data on FV plate

left) and plate waste (right) for assessment of schoolchildren’s


RESEARCH

waste. Trays observed at both of these time points wereincluded regardless of if any FV items were selected. Interob-server reliability was tested for DI over six school visits bycomparing two research associates’ visual estimations for thepercentage consumedof FV items. Twoundergraduate andonegraduate research associate completed all the visual estima-tions. The pairs of research associates’ estimates used to testinterobserver reliability varied among school visits. The val-idity of DI and DIþO were tested during four school visits bycomparing estimations of FV consumption by each methodagainst actual grams consumed as measured by WPW.

Data AnalysesInterobserver reliability was assessed for estimations of foodconsumption by percentage agreement and ICCs. Percentageagreement was assessed using procedures adapted fromBaglio and colleagues.23 For each FV item, two observers werein agreement if their consumption estimations were within25% of each other. For example, if 1 cup leafy greens wasselected, observers were in agreement if their estimationswere within 1/4 cup. Percentage agreement was calculated bydividing the number of agreements by the number ofagreements plus disagreements. Acceptable percentageagreement was defined as �85%.23 ICCs assessed observers’estimations for the percentage consumed of all FV items on atray. The values for percentage consumed for all FV itemsobserved on the tray were summed. ICC (1,1) was usedbecause a different set of observers rated each lunch tray(one-way analysis), and because the unit of reliability wasbased on individual, rather than mean, ratings.24 ICCs wereconsidered fair if 0.41 to 0.60, moderate if 0.61 to 0.80, andsubstantial if 0.81 to 1.00.25

Validity was evaluated by comparing DI and DIþO esti-mations of grams consumed to actual grams measured usingWPW. Pearson correlations compared the methods forassessing fruit, vegetable, and total FV consumption per tray,and for assessing consumption of items within each FV itemcategory. FV item categories included juice and entrée itemssuch as lasagna and soup. Other categories consolidateditems with similar visual characteristics or serving styles suchas leafy greens (self-served from the salad bar and prepor-tioned side and entrée salads); preportioned FV sides(steamed broccoli and cauliflower, bagged carrots, fruitcocktail, and sliced vegetables with dip); salad bar vegetables(beans, carrots, celery, and cucumbers); and whole fruit(apples, bananas, kiwis, oranges, and pears). Correlationswere considered acceptable if �0.80. Paired t tests deter-mined the degree of under- or overestimation by DI andDIþO for mean fruit, vegetable, and FV consumption per tray.Bland-Altman plots were used to plot the mean of each dig-ital method and WPW against the difference between thosetwo methods. Limits of agreement were considered accept-able if they were no wider than 20 g. Twenty grams wasdeemed appropriate because it is equal to one quarter of astandard 80 g (1/2 cup) FV serving, or 1/8 cup.4 School-basedinterventions have reported increases in FV consumption assmall as this one quarter serving (1/8 cup).26 Data analyseswere conducted using the Statistical Package for the SocialSciences (version 20.0.0, 2011, SPSS Inc) and the StatisticalAnalysis Software (version 9.3, 2010, SAS Institute Inc) withP<0.05 (two-tailed) required for statistical significance.


RESULTSThe characteristics of the two participating schools are pre-sented in Table 1.Of the total 410 trays distributed across visits using DI, 276

(67%) were collected, with a minimum of 31 trays collectedper school visit. Among the total collected trays, 27 (10%)were missing FV consumption for some, but not all, FV itemson the tray. Among the remaining 134 trays not collected, 68(51%) were not observed at the beginning of lunch to collectFV selection data, 45 (34%) were not physically collected atthe end of lunch to observe plate waste, 10 (7%) were missingFV consumption data for all items on the tray, 10 (7%) con-tained visual obstructions in the digital image, and 1 (1%) wasnot coded for other reasons. In the school with a salad barand in the school offering only preportioned FV, 75% and 60%of distributed trays, respectively, yielded data available foranalyses.Among the 276 collected trays, no FV were selected on 44

(16%). Reliability was evaluated among these remaining trays(n¼232) and validity evaluated among those with data forboth DI and WPW (n¼162 trays, 269 items) or DIþO andWPW (n¼164 trays, 274 items). The sample size was largerfor DIþO because DIþO yielded data for five FV items on fivetrays that had previously been omitted when using DI due tomissing evidence. DIþO data were modified for an additionalnine FV items on seven trays; these were items previouslyestimated using DI but adjusted for DIþO after adding cafe-teria observations (eg, selection of a second serving of soupmid-meal).

ReliabilityThe reliability of DI estimations of FV consumption duringschool observations was acceptable. Two research associatesrated all trays observed by DI (n¼232 trays; 436 FV items).Agreement was 96% for DI, and the ICC was 0.92 (95% CI 0.90to 0.94). Reliability was similar across all school visits;percent agreement was �90% and the ICC was �0.90.

ValidityPearson correlations for fruit, vegetable, total FV, and FVitem categories are presented in Table 2. DI (r¼0.91 to0.96) and DIþO (r¼0.95 to 0.98) were strongly correlatedwith WPW for consumption of fruit, vegetable, and FVcombined per tray (P<0.001). Correlations were similarwithin each school using DI (r¼0.88 to 0.98) and DIþO(r¼0.93 to 0.98). Correlations for most FV item categorieswere above 0.80 with the lowest correlations for lasagna(r¼0.62) and soup (r¼0.65) using DI and for leafy greensusing DI and DIþO (r¼0.59). After adding cafeteria obser-vations to DI (ie, DIþO), all correlations were above 0.80except leafy greens (r¼0.59).DI and DIþO were valid methods for estimating mean FV

consumption, and DI was not significantly different fromWPW for estimating both mean fruit and mean vegetableconsumption. Actual mean FV consumption measured byWPW was 97.5�59.1 g among trays compared with DI(96.7�61.3 g) and 98.4�59.5 g among trays compared withDIþO (99.3�61.4 g). DI was not significantly different fromWPW for assessing mean fruit (P¼0.15), vegetable (P¼0.68),and combined FV (P¼0.56) consumption (Table 3). Cafeteriaobservations did not provide substantial improvements to


Table 2. Pearson correlations between grams of fruits and vegetables consumed during school lunch as measured by weighedplate waste and digital imaging estimation methods among schoolchildren in two Vermont elementary schools

Food item

Pearson Correlation

Digital imaging Digital imagingDobservation

Fruit 0.95 (n¼113)*** 0.97 (n¼117)***

Vegetable 0.91 (n¼106)*** 0.95 (n¼106)***

Combined fruits and vegetables 0.96 (n¼159)*** 0.98 (n¼164)***

Juice 0.98 (n¼82)*** 0.98 (n¼84)***

Lasagna 0.62 (n¼39)*** 0.84 (n¼39)***

Leafy greens 0.59 (n¼15)* 0.59 (n¼15)*

Pizza 0.94 (n¼25)*** 0.94 (n¼25)***

Preportioned fruit and vegetable sides 0.98 (n¼28)*** 0.98 (n¼28)***

Salad bar vegetablesa 0.84 (n¼20)*** 0.85 (n¼21)***

Soup 0.65 (n¼31)*** 0.99 (n¼31)***

Whole fruit 0.76 (n¼29)*** 0.87 (n¼31)***

aExcludes leafy greens.*P<0.05.***P<0.001.

RESEARCH

the validity of DI estimations of mean fruit and/or vegetableconsumption. DIþO overestimated mean vegetable con-sumption by 2.8 g (P¼0.01), whereas mean fruit (P¼0.29) andmean combined FV (P¼0.38) consumption were not signifi-cantly different from WPW.In analyses by school, DIþO overestimated mean vegetable

consumption by 4.1 g (P¼0.02) in the school with a self-servesalad bar, whereas differences were not significant in theschool serving only preportioned FV (P¼0.50). There were no

Table 3. Comparison of mean grams consumed of fruits and vegobservation against weighed plate waste to estimate schoolchildrin two Vermont elementary schools

Food type Trays (n)a Mean g

)WPW

Fruit 113 99.5�4

Vegetable 106 44.7�3

Total FVe 159 97.5�4

)WPW

Fruit 117 99.0�4

Vegetable 107 45.0�3

Total FV 164 98.4�4

aSample sizes vary between methods because some trays were not observed by all methods.bSE¼standard error.cWPW¼weighed plate waste.dDI¼digital imaging.eFV¼fruit and vegetable.fDIþO¼digital imaging with observation.*Differences were significant at P<0.05.


other significant differences for assessing mean consumptionbetween WPW and DIþO nor DI within schools.The 95% limits of agreement for individual-tray FV con-

sumption using DI were e32.9 to 31.3 g (Figure 2). The 95%limits of agreement for individual-tray FV consumption usingDIþO were e25.0 to 26.8 g. Thus, although accurate forassessing mean intake and highly correlated with WPW,there was variability in estimates for individual-tray con-sumption that could exceed 1/8 cup FV, or 20 g.4

etables estimated by digital imaging and digital imaging withen’s consumption of fruits and vegetables during school lunch

rams consumed–SEb P value

c/ )DId/

.1 97.4�4.4 0.15

.2 45.3�3.2 0.68

.7 96.7�4.9 0.56c/ )DIþOf/

.1 97.9�4.2 0.29

.2* 47.8�3.4* 0.01

.6 99.3�4.8 0.38


Figure 2. Bland-Altman plots comparing weighed plate waste to digital estimation methods for assessment of schoolchildren’s fruitand vegetable consumption per tray at lunch. Difference between methods was plotted as digital estimation method minusweighed plate waste. The center line represents the mean difference between weighed plate waste and the visual estimationmethod. The lower and upper boundaries represent the 95% limits of agreement between methods. Visual estimation methods are(A) digital imaging and (B) digital imaging with observation.

RESEARCH

DISCUSSIONThe major findings of this study were that DI was reliable forestimating children’s FV consumption, DI and DIþO producedvalid estimates of mean FV consumption when comparedwith the gold standard WPW, and DI and DIþO were lessprecise for assessing FV consumption for individual trays.Several challenges affected the collection of FV consump-

tion data. Visual obstructions in digital images (children’shands, milk containers, and soup bowls) prevented theidentification of some tray stickers or FV items. Still, visualobstructions affected a relatively small number of trays(n¼10), especially considering that tripods or separate im-aging stations were not used as in previous studies.21,22,27,28

In addition, it was difficult to collect all trays within thedesignated sample without disrupting normal cafeteria rou-tines when multiple children exited the lunch line ordisposed of trays at once. At the school offering only pre-portioned FV, lunch aides collected and consolidated trayswhile children were still seated at tables, rendering sometrays unusable, which may help explain why a lower per-centage of trays were collected in this school. Cafeteria ob-servations collected during the meal explained someinstances of missing plate waste, such as FV items that weredisposed of separately from other contents of the tray. Similarincidents were observed in a study using DI, where 24% to29% of whole fruit could not be estimated because of missingplate waste evidence at the end of lunch.27

DI was reliable for estimating children’s FV consumption.Reliability for DI was high and similar to previousstudies.22,28,29 In a study using DI in an elementary school,97% of observers’ estimations of food consumption werewithin 20% of each other and 99% were within 30% of each


other.22 In other studies, the ICC was 0.71 for observers’ es-timations of FV consumption from digital images collected ina university cafeteria28 and 0.92 for DI estimations of vege-table consumption of hospital meals.29

Estimations of mean fruit and FV consumption by DI andDIþO were not significantly different from WPW. DIþO over-estimatedmeanvegetable consumption. In comparing the twoschools, this difference was significant only in the schoolwhere some FV were self-served from a salad bar, not theschool where all FV were preportioned, suggesting that foodsserved in variable portion sizes may be more challenging toestimate using digital methods. However, DI or DIþO estima-tions of mean consumption were within 3 g of actual con-sumption (roughly the equivalent of a small sugar snap pea),suggesting the degree of inaccuracy is very small. This degreeof error is similar to a laboratory-basedmeal simulationwhereDI overestimated mean FV consumption by 5 g.15

Correlations were below 0.80 for four FV item categories invisual estimations using DI (lasagna, leafy greens, soup, andwhole fruit), and correlations for FV item categoriesimproved for all except leafy greens when cafeteria obser-vations were used to adjust estimations (ie, DIþO). However,fruit, vegetable, and combined FV consumption per tray werestrongly correlated with WPW using DI regardless of thelower correlations observed in some FV categories. Thefindings suggest that, although DIþOmay be more precise forestimating certain FV items, cafeteria observations do notprovide substantial improvements to the precision of themethod for general estimations of fruit and/or vegetableconsumption.Although DI and DIþO were both strongly correlated with

WPW for fruit, vegetable, and FV consumption per tray


RESEARCH

(r¼0.91 to 0.98), there was a large degree of variability in theextent to which consumption was under- or overestimatedusing digital methods, which could result in some individualtrays being misestimated by more than 1/8 cup FV. The limitsof agreement indicate that, among 95% of all trays,individual-tray FV consumption could be misestimated by 25to 33 g using DI or DIþO. Because a typical half-cup FVserving weighs 80 g, this discrepancy could result in under-or overestimation by more than 1/8 cup, making the methodless sensitive to small changes in individual-tray FV con-sumption. This may reflect limitations of an ordinal scale,such as the 6-point Comstock scale used in this study, incomparison to a continuous scale used when collecting actualweights consumed. Future studies could investigate theextent to which scales with smaller increments19,22 mightimprove the precision of estimations for individual-trayconsumption. At the same time, the findings from thisstudy demonstrate that a 6-point scale is sufficient foraccurately estimating group-level (mean) FV consumption.Notable strengths of this study include the collection of a

large sample of FV items and lunch trays from two uniqueschool cafeterias, and its evaluation of how cafeteria obser-vations influence the validity of DI. There were several limi-tations. This study took place in only two schools and maynot be generalizable to others with different FV offerings orcafeteria settings. The study was limited to observations ofchildren participating in the National School Lunch Programand did not assess FV consumption for items in home-packedlunches or from other school venues. Both schools servedchildren where the majority of children participated in theschool lunch program, with few or no à la carte foods avail-able. The majority of the FV items were served in prepor-tioned servings or countable pieces, whereas only one item—leafy greens from a salad bar in one school—required portionsize estimation. Because children in this sample selected leafygreens infrequently, future research should continue to studythe validity of visual estimations of leafy greens and other FVserved in variable portion sizes such as self-served apple-sauce and grated carrots. In addition, although a randomsample of five servings was used to determine averageweights of preportioned items, some FV weights were morevariable, which may have limited the precision of selectionestimates. Future research on DI should consider proceduresthat address this variability, such as collecting a larger samplesize to determine average weight.Another limitation was that designated trays were not

observed continuously throughout the lunch period toidentify all instances where FV were added or removed. Inprevious studies using direct observation, staff observed asmall number of children seated at one table for the durationof lunch30-33 or asked children directly if any foods wereshared.34,35 Although this study collected observations oftrading, sharing, and seconds of FV items during lunch, not alldesignated trays could be observed simultaneously becausethey were dispersed throughout the cafeteria, requiringresearch associates to instead observe multiple tables.Consequently, some instances of trading, sharing, and sec-onds may not have been observed. Other studies observedthat inaccurate or incomplete data resulted when cafeteriaobservations were excluded from assessments of schoollunch consumption.27,36 Further research could test the val-idity of DI with and without cafeteria observations using a


more meticulous process to verify the findings of this study.The collection of more detailed observations for furthervalidation testing is labor-intensive, requiring the training ofa larger research team or collecting fewer lunch trays.More research is warranted on DI as a method for

measuring school children’s FV consumption because themethod demonstrated similar validity to DIþO and offers theadded advantage of a less time- and labor-intensive datacollection procedure. Future research could refine DI tech-niques to further reduce the time and labor required for datacollection during school visits. For instance, minor adjust-ments were made to trays to improve visibility of FV items,because all trays were collected at the end of lunch. Addi-tional research could explore whether the validity of DI issubstantially affected when tray contents are not adjustedbefore collecting postmeal images. Future research could alsoinvestigate the feasibility of recruiting school staff or parentvolunteers to collect tray images, potentially reducing datacollection costs.

CONCLUSIONSDI is a less costly and less labor-intensive yet valid alternativeto WPW for assessing schoolchildren’s mean FV consumptionduring lunch. In addition to demonstrating that DI is a reli-able method, this study found the validity of DI and DIþO forFV consumption estimations, compared with WPW, wassimilar. This suggests that the exclusion of mealtime obser-vations does not greatly influence the accuracy of estimationsof children’s mean fruit, vegetable, and FV consumption.More research on DI is needed to replicate these findings inother school environments with other FV offerings andfurther investigate the validity of DI with and without theaddition of cafeteria observations. DI appears especially ad-vantageous because it requires less data collection duringschool visits than DIþO, WPW, or direct observation. DI is apromising method for use in the evaluation of school-basedinterventions designed to increase children’s mean FV con-sumption during school meals.

References1. Krebs-Smith SM, Guenther PM, Subar AF, Kirkpatrick SI, Dodd KW.

Americans do not meet federal dietary recommendations. J Nutr.2010;140(10):1832-1838.

2. Guenther PM, Dodd KW, Reedy J, Krebs-Smith SM. Most Americanseat much less than recommended amounts of fruits and vegetables.J Am Diet Assoc. 2006;106(9):1371-1379.

3. Riboli E, Norat T. Epidemiologic evidence of the protective effect offruit and vegetables on cancer risk. Am J Clin Nutr. 2003;78(3 suppl):559S-569S.

4. World Cancer Research Fund/American Institute for CancerResearch. Food, Nutrition, Physical Activity, and the Prevention ofCancer: A Global Perspective. Washington, DC: American Institute forCancer Research; 2007.

5. He FJ, Nowson CA, Lucas M, MacGregor GA. Increased consumptionof fruit and vegetables is related to a reduced risk of coronary heartdisease: Meta-analysis of cohort studies. J Hum Hypertens.2007;21(9):717-728.

6. He FJ, Nowson CA, MacGregor GA. Fruit and vegetable consumptionand stroke: Meta-analysis of cohort studies. Lancet. 2006;367(9507):320-326.

7. Ledoux TA, Hingle MD, Baranowski T. Relationship of fruit andvegetable intake with adiposity: A systematic review. Obes Rev.2011;12(5):e143-e150.

8. Nutrition and Your Health: Dietary Guidelines for Americans, 2010. 7thed. Washington, DC: Government Printing Office; 2010.


http://refhub.elsevier.com/S2212-2672(14)00231-7/sref1

























RESEARCH

9. US Department of Agriculture. Nutrition standards in the NationalSchool Lunch and School Breakfast Programs. Final rule. FederalRegister. 2012;77:4088-4167.

10. Thompson FE, Subar AF. Dietary assessment methodology. In:Coulston AM, Boushey CJ, Ferruzzi MG, eds. Nutrition in the Pre-vention and Treatment of Disease. 3rd ed. London, UK: Elsevier;2013:5-46.

11. Van Staveren WA, Ocké MC, de Vries JHM. Estimation of dietaryintake. In: Erdman JW, Macdonald IA, Zeisel SH, eds. PresentKnowledge in Nutrition. 10th ed. Oxford, UK: Wiley-Blackwell; 2012:1012-1026.

12. Livingstone MBE, Robson PJ, Wallace JMW. Issues in dietary intakeassessment of children and adolescents. Br J Nutr. 2004;92(suppl 2):S213-S222.

13. Adams MA, Pelletier RL, Zive MM, Sallis JF. Salad bars and fruit andvegetable consumption in elementary schools: A plate waste study.J Am Diet Assoc. 2005;105(11):1789-1792.

14. Comstock EM, St Pierre RG, Mackiernan YD. Measuring individualplate waste in school lunches. Visual estimation and children’s rat-ings vs. actual weighing of plate waste. J Am Diet Assoc. 1981;79(3):290-296.

15. Williamson DA, Allen HR, Martin PD, Alfonso AJ, Gerald B, Hunt A.Comparison of digital photography to weighed and visual estimationof portion sizes. J Am Diet Assoc. 2003;103(9):1139-1145.

16. Nicklas TA, O’Neil CE, Stuff J, Goodell LS, Liu Y, Martin CK. Validityand feasibility of a digital diet estimation method for use withpreschool children: A pilot study. J Nutr Educ Behav. 2012;44(6):618-623.

17. Hinton EC, Brunstrom JM, Fay SH, et al. Using photography in “TheRestaurant of the Future”: A useful way to assess portion selectionand plate cleaning? Appetite. 2013;63:31-35.

18. Wengreen HJ, Madden GJ, Aguilar SS, Smits RR, Jones BA. Incentiv-izing children’s fruit and vegetable consumption: Results of a UnitedStates pilot study of the Food Dudes program. J Nutr Educ Behav.2013;45(1):54-59.

19. Williamson DA, Han H, Johnson WD, Martin CK, Newton RL. Modi-fication of the school cafeteria environment can impact childhoodnutrition. Results from the Wise Mind and LA Health studies.Appetite. 2013;61(1):77-84.

20. Martin CK, Thomson JL, Leblanc MM, et al. Children in school cafe-terias select foods containing more saturated fat and energy than theInstitute of Medicine recommendations. J Nutr. 2010:1653-1660.

21. Martin CK, Newton RL, Anton SD, et al. Measurement of children’sfood intake with digital photography and the effects of secondservings upon food intake. Eat Behav. 2007;8(2):148-156.

22. Swanson M. Digital photography as a tool to measure school cafe-teria consumption. J Sch Health. 2008;78(8):432-437.


23. Baglio ML, Baxter SD, Guinn CH, Thompson WO, Shaffer NM,Frye FHA. Assessment of interobserver reliability in nutrition studiesthat use direct observation of school meals. J Am Diet Assoc.2004;104(9):1385-1392.

24. Shrout PE, Fleiss JL. Intraclass correlations: Uses in assessing raterreliability. Psychol Bull. 1979;86(2):420-428.

25. Shrout PE. Measurement reliability and agreement in psychiatry. StatMethods Med Res. 1998;7(3):301-317.

26. Knai C, Pomerleau J, Lock K, McKee M. Getting children to eatmore fruit and vegetables: A systematic review. Prev Med.2006;42(2):85-95.

27. Swanson M, Branscum A, Nakayima PJ. Promoting consumption offruit in elementary school cafeterias. The effects of slicing apples andoranges. Appetite. 2009;53(2):264-267.

28. Williamson DA, Allen HR, Martin PD, Alfonso A, Gerald B, Hunt A.Digital photography: A new method for estimating food intake incafeteria settings. Eat Weight Disord. 2004;9(1):24-28.

29. Parent M, Niezgoda H, Keller HH, Chambers LW, Daly S. Comparisonof visual estimation methods and modified textures: Real-time vsdigital imaging. J Acad Nutr Diet. 2012;112(10):1636-1641.

30. Baxter SD, Hardin JW, Guinn CH, Royer JA, Mackelprang AJ, Smith AF.Fourth-grade children’s dietary recall accuracy is influenced byretention interval (target period and interview time). J Am Diet Assoc.2009;109(5):846-856.

31. Smith AF, Baxter SD, Hardin JW, Guinn CH. Validation-study con-clusions from dietary reports by fourth-grade children observedeating school meals are generalizable to dietary reports by compa-rable children not observed. Public Heal. Nutr. 2007;10(10):1057-1066.

32. Baranowski T, Islam N, Baranowski J, et al. The food intake recordingsoftware system is valid among fourth-grade children. J Am DietAssoc. 2002;102(3):380-385.

33. Weber JL, Lytle L, Gittelsohn J, et al. Validity of self-reported dietaryintake at school meals by American Indian children: The PathwaysStudy. J Am Diet Assoc. 2004;104(5):746-752.

34. Templeton SB, Marlette MA, Panemangalore M. Competitive foodsincrease the intake of energy and decrease the intake of certainnutrients by adolescents consuming school lunch. J Am Diet Assoc.2005;105(2):215-220.

35. Biltoft-Jensen A, Bysted A, Trolle E, et al. Evaluation of Web-baseddietary assessment software for children: Comparing reportedfruit, juice and vegetable intakes with plasma carotenoid concen-tration and school lunch observations. Br J Nutr. 2013;110(1):186-195.

36. Baxter SD, Thompson WO, Davis HC. Trading of food duringschool lunch by first- and fourth-grade children. Nutr Res. 2001;21:499-503.

AUTHOR INFORMATIONJ. C. Taylor is a graduate student, Department of Nutrition, University of California, Davis; at the time of the study, she was a graduate student inthe Department of Nutrition and Food Sciences, University of Vermont, Burlington. B. A. Yon is a research associate and R. K. Johnson is Robert L.Bickford Jr, Green and Gold Professor of Nutrition, Department of Nutrition and Food Sciences, University of Vermont, Burlington.

Address correspondence to: Rachel K. Johnson, PhD, MPH, RD, 225B MLS Carrigan Wing, Department of Nutrition and Food Sciences, Universityof Vermont, Burlington, VT 05405. E-mail: [email protected]

STATEMENT OF POTENTIAL CONFLICT OF INTERESTNo potential conflict of interest was reported by the authors.

FUNDING/SUPPORTThis research was funded by the US Department of Agriculture/Vermont Agricultural Experiment Station and the University of Vermont BickfordScholar Fund.

ACKNOWLEDGEMENTSThe authors thank Alan Howard and Leah Conchieri for their support with data and statistical analyses, the undergraduate students on theresearch team at the University of Vermont for their assistance with data collection, and the school nutrition directors at the two school sites fortheir cooperation.
































































































mailto:[email protected]

reliability and validity of digital imaging as a measure of schoolchildren's fruit and...

Documents