adequate (or adipose?) yearly progress ......patricia m. anderson, kristin f. butcher, and diane...

ADEQUATE (OR ADIPOSE?) YEARLY

PROGRESS: ASSESSING THE EFFECT

OF “NO CHILD LEFT BEHIND” ON

CHILDREN’S OBESITY

AbstractThis paper investigates how accountability pressures under NoChild Left Behind (NCLB) may have affected students’ rate ofoverweight. Schools facing pressure to improve academic out-comes may reallocate their efforts in ways that have unintendedconsequences for children’s health. To examine the impact ofschool accountability, we create a unique panel dataset contain-ing school-level data on test scores and students’ weight outcomesfrom schools in Arkansas. We code schools as facing account-ability pressures if they are on the margin of making AdequateYearly Progress, measured by whether the school’s minimum-scoring subgroup had a passing rate within 5 percentage pointsof the threshold. We find evidence of small effects of account-ability pressures on the percent of students at a school who areoverweight. This finding is little changed if we controlled for theschool’s lagged rate of overweight, or use alternative ways to iden-tify schools facing NCLB pressure.

Patricia M. Anderson

Department of Economics

Dartmouth College

Hanover, NH 03755-3514

patricia.m.anderson

@dartmouth.edu

Kristin F. Butcher

Department of Economics

Wellesley College

Wellesley, MA 02481

[email protected]

Diane Whitmore Schanzenbach

(corresponding author)

School of Education and

Social Policy

Northwestern University

Evanston, IL 60208

[email protected]

doi:10.1162/EDFP_a_00201

C© 2017 Association for Education Finance and Policy

54

Patricia M. Anderson, Kristin F. Butcher, and Diane Whitmore Schanzenbach

1. INTRODUCTIONThe federal No Child Left Behind (NCLB) legislation was passed in 2002, ushering ina national era of test-based school accountability programs. Although states have somelatitude in defining specific requirements (Davidson et al. 2013), all states were requiredto define and implement stringent accountability standards and to penalize schools thatfailed to meet these standards.1 A substantial amount of research into NCLB and thestate-level accountability movement that preceded it has documented many ways thatschools respond to accountability pressures (see the review chapter by Figlio and Ladd2015; also Rouse et al. 2013; and Reback, Rockoff, and Schwartz 2014). Overall, testscores improve—sometimes quite substantially—after accountability is enacted (Deeand Jacob 2010).2 Some of the gains have been driven by schools’ changes in policiesand practices that devote more time and/or effort to instruction in tested subjects (see,e.g., Chiang 2009; Reback, Rockoff, and Schwartz 2014; Chakrabarti 2014). Others aredue to more strategic behavioral responses to the incentives, such as shifting efforttoward students on the cusp of passing (Reback 2008; Neal and Schanzenbach 2009),strategically assigning students to special education or English Language Learner status(Cullen and Reback 2006; Chakrabarti 2013a), suspending low-performing students(Figlio 2006), or outright cheating (Jacob and Levitt 2003; Sass, Apperson, and Bueno2015).

As described in Figlio and Ladd (2015), policy makers must make tradeoffs indetermining how broad-based to make their school accountability systems. Educationalstakeholders typically value not only math and reading achievement, but also a broaderarray of outcomes, including higher-order learning that cannot be easily measuredon standardized tests, noncognitive skills, achievement in other subject matters, andbroader life outcomes (such as citizenship and health). Of course, it is difficult to reliablymeasure a broad set of outcomes, however, and even if it were possible to measureall of the outcomes it would be difficult to assign optimal weights to each outcome inan accountability framework. As such, most accountability systems under NCLB arenarrowly focused on math and reading achievement, elevating the importance of theseoutcomes and providing incentives to narrow the scope of instruction.

In particular, because schools are not directly held accountable for student health,there is scope for NCLB incentives to inadvertently harm outcomes in this area. In thispaper, we measure whether the pressures of test-based accountability inadvertently ledto increases in students’ rates of obesity and overweight. Many policies and practices ofschools have potential impacts on students’ calorie expenditures and food intake, andthus ultimately their body weight. For example, recess and gym classes offer childrenopportunities to burn calories during the day. Foods served during school meals, aswell as extra foods sold as fundraisers, at snack bars and vending machines, and treatsgiven as rewards or during celebrations, all have potential to affect dietary intake andhabit formation. Schools can also influence (with homework assignments) the amountof free time outside of school that children have available. The incentives from test-based accountability may induce schools to alter their practices along any or all of these

1. In 2011, the U.S. Department of Education began allowing waivers for states to implement alternate plans. Wefocus only on the years when NCLB was in full effect.

2. See also Carnoy and Loeb 2002; Jacob 2005; Figlio and Rouse 2006; Rockoff and Turner 2010; Dee and Jacob2011; and Wong, Steiner, and Cook 2013.

55

Adequate (or Adipose?) Yearly Progress

dimensions. To increase time in instruction of tested subjects, for example, schoolsmay reduce time spent in recess and/or physical education (PE). They may increase thelength of the school day, require after-school tutoring, or assign more homework. Newfinancial pressures may induce school administrators to try to raise new funds throughfood-based fundraisers or outside food and beverage contracts. Schools may use foodas rewards to motivate students.3 As described later in this paper, there is evidence thatschools use these types of strategies, and they have potential to impact student healthas measured by the rate of overweight and obesity.

To examine the potential spillover of school accountability programs onto the rate ofoverweight, we create a unique dataset that combines NCLB rules, test scores, and thepercent of students whose body mass index (BMI) is above the cutoff for “overweight,”using data from Arkansas. NCLB required states to set thresholds, based on the percentof students achieving a “passing” mark on standardized tests, that determine whetherschools in that state are making Adequate Yearly Progress (AYP) toward the ultimategoal of having all students achieve a passing mark on the tests. As described in thispaper, we argue that schools near the passing threshold are most likely to face higherlevels of accountability “pressure” under NCLB and are most likely to adopt new policiesand practices that may spill over to students’ rate of overweight. Empirically, we findthat schools in this pressured group see moderate increases in their rates of overweight,and these findings are robust to a variety of specifications, such as including the school’slagged rate of overweight. The results broaden our understanding of the many waysthat schools have responded to NCLB, and add to the scant literature on the impacts oftesting-based school accountability on child health.

2. BACKGROUNDImpacts of Accountability

A large literature on NCLB and the state accountability programs that preceded it hasdocumented various ways that test-based accountability has changed aspects of howschools operate (see Figlio and Ladd 2015 for a recent review). The literature is built ona variety of research designs, some of which are more well-identified than others.

Much of what we know about the impacts of accountability has been drawn fromcleanly identified, quasi-experimental approaches. For example, one approach com-monly used in the literature is to use a “program introduction” design to compare out-comes before and after accountability was introduced. Some of these also use variationacross states prior to NCLB, when various states were adopting their own accountabil-ity programs (e.g., Jacob 2005; Ballou and Springer 2008; Neal and Schanzenbach2009). Such an approach is typically limited to the short-run impacts of accountabil-ity in the few years after its introduction. Another well-identified, quasi-experimentalapproach is to use a regression discontinuity (RD) to isolate the response to failing tomeet an annual goal (Chiang 2009; Chakrabarti 2013b, 2014; Rouse et al. 2013). Inthese approaches, schools with passing rates just below the AYP threshold are com-pared to schools with passing rates just above, under the assumption that there are no

3. Although there is dispute in the literature as to whether cortisol contributes to obesity, if it does, there is thepossibility that testing pressures cause stress in children, which may increase cortisol secretions (and thusperhaps lead to weight gain). Reback, Rockoff, and Schwartz (2014) find that attending a school on the marginof passing does not cause students to report being more anxious about standardized tests.

56


important differences between schools on either side of that threshold (other than onemakes AYP and the other does not). Both the program introduction and RD approachesin the literature have had additional successes combining their basic approach withstrategies that leverage differential incentives under the accountability policies. For ex-ample, studies have made comparisons across high- versus low-stakes grade levels andsubject matter, and between so-called “bubble” students (those on the cusp of passingthe test while their classmates are more solidly in passing or failing territory). To besure, the literature based on the quasi-experimental evidence is extremely important,and much of what we know about the impacts of NCLB is from these studies.

A series of thoughtful papers by Dee and Jacob (2010, 2011) documents the overallimpact of NCLB. Because the policy was adopted nationwide at one point in time, ithas been a challenge to find a suitable comparison group that was left untreated. Aftershowing that private schools not subject to NCLB do not turn out to be an appropriatecomparison group because of changes in student-level selection into these schools,researchers use a strategy comparing states that adopted accountability prior to NCLBwith those that did not. They find that NCLB led to improvements in math achievementas measured by the low-stakes National Assessment of Educational Progress (NAEP)test, especially among students who are young or disadvantaged. On the other hand,they do not find systematic improvements in reading achievement on the NAEP. Theyalso document increases in nonfederal education spending, teacher compensation, thepercentage of teachers with graduate degrees, and instructional time in both mathand reading. The Dee and Jacob results estimating the impact of NCLB as a wholeare consistent with a broader literature studying impacts of accountability policies invarious states.

NCLB Pressure

There are reasons to believe that the impacts of NCLB go beyond what can be cleanlyidentified in research studies such as those described in the previous section. Considerthe important RD literature on NCLB that compares schools on either side of thepassing threshold, leveraging the different levels of pressure on either side of a strictcutoff. Although the RD literature has shown that there are different levels of pressureon either side, in addition, even the “untreated” schools that have just barely passedthe NCLB threshold are likely to face accountability pressure despite the fact they havenot (yet) failed. In other words, even though the schools on the failing side of themargin may be facing more pressure than those above the cutoff, the schools thathave just managed to pass have also faced accountability pressure.4 Our approachhere is different—we estimate the effect of schools coming under NCLB pressure bycomparing schools near the passing threshold to those further away from it.

To help fix ideas about which schools will be “pressured” to change under NCLB, itis useful to consider some details of NCLB in Arkansas. In Arkansas, and most otherstates, schools were held accountable for the fraction of children in a school who earn a

4. We investigated the RD strategy in this paper but abandoned it when we found that, in the case of Arkansas,there were discontinuities at the passing threshold in background characteristics such as percent black andpercent low-income, thus invalidating the required assumptions for RD.

57


passing score on the state standardized tests in math and literacy.5 A feature of NCLBis that states determine the details of their accountability program within parametersdefined by the federal government, including determining what test would be usedfor accountability, what would be the passing threshold, and what fraction of studentsmust pass each year for a school to meet the AYP standard. For an elementary school tobe deemed passing in Arkansas in 2002, approximately 30 percent of students in theschool had to pass each test.6 The percent passing goal increased by about 7 percentagepoints each year, in order to reach the federally-mandated goal of 100 percent proficiencyby 2014.7 A hallmark of NCLB is that in addition to the overall percentage passing in theschool, each student subgroup—as defined by race, socioeconomic status, and othereducational categories—was required to meet the same percent passing rate.8 In orderfor a school to make AYP in a given year, not only does the overall passing rate of itsstudent population for each tested grade have to be above the threshold set for thatyear, but the passing rate of all designated subgroups in the school with a large enoughenrollment must also meet or exceed the goal threshold.9 If any one of the studentsubgroups fails to attain AYP, then the entire school would be designated as failingto meet AYP.10 Consistent with NCLB principles nationwide, if schools in Arkansasfail to meet the AYP goals for two consecutive years, they are required to implementcorrective actions that increase in severity over time.11

According to the theory of change underlying accountability policies, the threat ofpunishment should induce schools to alter their practices in a manner that wouldimprove student achievement. Because the threat of punishment may have been

5. The passing threshold on the Arkansas state test is lower than the threshold on the National Assessment ofEducational Progress (NAEP) test. In particular, 62 (61) percent of students passed the fourth grade state testin literacy (math), and 28 (26) percent of fourth graders passed the NAEP test. This 34–35 percentage-pointdifference in pass rates across tests is in line with the U.S. average of 32–37 points (Education Week 2006).

6. It is common to refer to “failing” schools or “passing” schools. However, the official nomenclature is thatschools that are “failing” schools are in “School Improvement Status.”

7. Annual AYP percent passing goals by grade and subject are listed in Appendix table A.1 (available in a separateonline appendix that can be accessed on Education Finance and Policy’s Web site at http://www.mitpressjournals.org/doi/suppl/10.1162/EDFP_a_00201). The starting points were slightly lower for higher grades, and theannual increase in the goals were thereby slightly higher in order to reach 100 percent proficiency by 2014.Note that in 2006, Arkansas revised the AYP goals downward, requiring larger annual increases going forward.Like most other states, Arkansas received a waiver in 2011 (after our data conclude) and was no longer requiredto hold schools accountable to these goals.

8. Additional details important for the data to work but add little to the intuition of the program, such as minimumsubgroup size rules, the safe harbor provision, and the ability of schools to use a three-year average percentpassing instead of their current pass rate, are described in more detail in Appendix B in the online appendix.

9. “Large enough” is defined in Arkansas as 40 students or 5 percent of enrollment (whichever is larger). Studentsubgroups for accountability under NCLB are defined by race (whites, African Americans, Hispanics, etc.), andfor low socioeconomic status students. We omit the English language learners and students with disabilitiessubgroups because of lack of consistent data on group size.

10. Although the basic AYP rules are straightforward enough, in practice a school could be deemed to meet orfail to meet AYP for several other reasons. For example, even if a school (or subgroup) has a lower fraction ofstudents meeting AYP than the passing standard required, it still might meet AYP through the “Safe Harbor”provision, which allows a school to be deemed as passing if the percentage of failing students (within subjectand subgroup) declined by 10 percent relative to the prior year. On the other hand, a school would be deemedas failing despite its passing rate if too low a fraction of its students participated in the test, or if attendance orgraduation rates were below the target thresholds.

11. These corrective actions ranged from allowing students to transfer to a different non-failing school in thedistrict in year 1, to being required to offer supplemental instruction to students in year 2, to more extrememeasures such as school restructuring in year 5.

58

http://www.mitpressjournals.org/doi/suppl/10.1162/EDFP_a_00201


particularly salient for schools on the margin of passing, they may be expected tohave made the largest changes in response to accountability pressures.12

We expect schools facing accountability pressure—as measured by those close to,but on both sides of, the margin of passing—to respond to these pressures by adoptingnew practices. Although there is not a perfectly clean identification strategy to measurewhether a school is exposed to accountability pressure, researchers have attempted toisolate these impacts in a variety of ways. Our approach is most similar to that usedby Reback, Rockoff, and Schwartz (2014), in which they predict the likelihood thateach subgroup within a school will meet the AYP requirement. They then categorizesubgroups into whether they have a “high” chance of meeting AYP, a “low” chance, orif they are on the margin with only a “moderate” chance. Further aggregating this tothe school level, they designate schools as being on the AYP margin and compare theseschools with those that were more clearly in passing or failing territory, to measure howinputs and outcomes differ in the marginal schools. They find that scores on low-stakesexams improve in schools on the margin, suggesting real improvements in learningand not merely “teaching to the test.” In addition, they find increases in teachinghours by specialty teachers, reductions in time spent in whole-class instruction, lessteaching of nontested subjects, and lower rates of teacher satisfaction in schools onthe margin. Their results are robust to using various approaches to operationalizewhich schools are “pressured” under NCLB, and also to adding cross-state differencesthat make use of different accountability thresholds across states. As described in thefollowing, our approach is broadly similar to Reback, Rockoff, and Schwartz (2014)in that we identify schools on the margin of passing, and thus “pressured by NCLB,”and test whether they have differential outcomes compared with schools farther awayfrom the margin. The intuition is that those schools “close” to the passing thresholdare those where administrators will be compelled to make changes that may affectstudents’ rate of overweight. In order to identify these schools, we follow the statute,which uses (along with other factors) the passing rate among specified subgroups ofstudents to determine whether the school is designated as making AYP. We categorizeschools as on the margin of passing if the lowest scoring subgroup of students isplus or minus 5 percentage points of the passing threshold. As any designation of aschool as “pressured by NCLB” will necessarily be ad hoc, we also examine a numberof reasonable alternative definitions.

In theory, accountability policies work to raise academic performance in part be-cause they induce schools to exert more effort with a given level of resources. Ineconomic parlance, we would say that prior to accountability, some schools were notoperating at their production possibilities frontier, and the realignment of incentiveswas intended to move schools toward that frontier. Of course, we do not directly observeeffort and must infer changes in effort from factors that we do observe, including testscores and other elements, such as time use and, as we argue in this paper, the percentof students who are overweight. There are several potential mechanisms through whichaccountability pressure could lead to increased rates of overweight. For example, in re-sponse to pressure, schools may decide to reallocate time away from physical activity

12. In the first few years of NCLB, approximately 25 percent of Arkansas schools were out of compliance with AYP(Blankenship and Barnet 2006); 46 percent were failing in 2009.

59


and toward instructional activities (including classwork, homework, extra tutoring, andso on). Of course, some reallocations of effort may not be expected to have any impacton students’ BMI. For example, if schools adopt new curricula or instructional methodsthat replace old approaches but do not change the time allocated to instruction, theremay be no impact on calorie balance despite a great deal of effort on the part of schools.

It is tempting to try to more precisely parameterize NCLB pressure. For example, arethose schools having two subgroups on the margin of passing facing more pressure thanschools that have only one? We argue that with the data available here such an endeavoris unlikely to be fruitful—the answer will depend on the underlying distribution ofstudent ability within each of the subgroups.13 Our measure will, of course, produce anestimate of NCLB pressure that is an average across different schools, some of whichwill face more pressure than others, and some of which will make different types ofchanges than others to try to make AYP. It is important to emphasize that we do notinterpret these estimates as elasticities or impulse response functions. Nevertheless,as long as our categorization of schools correctly identifies those schools more likely tobe making changes in response to NCLB, one can interpret the coefficient we estimateas the average effect of all the different types of changes schools might make in aneffort to comply with AYP rules on students’ rates of overweight. If our measure isprimarily noise and has no relationship to being pressured by NCLB we would expectno statistically meaningful relationship between our measure of pressure and students’weight outcomes. If our measure of pressure is systematically picking up something elsethat is correlated with students’ weight, such as overall quality of the school or trendsin student demographic characteristics, then that, indeed, would be a problem forinterpreting the coefficient we estimate. As we describe in this paper, our methodologyis intended to control for these confounding factors and relies on variation in whethera school’s lowest-scoring subgroup, among otherwise similar schools, is on the marginof passing. The results indicate that NCLB pressure as conceived in our measure stablypredicts higher rates of overweight, and indicate an unintended spillover from academicaccountability to children’s health.

Research on Schools and Obesity

In addition to the literature on NCLB, another related literature is on schools’ abilityto impact children’s calorie balance through the food or physical activity environment.There is evidence that obesity is affected by the school food environment, includ-ing school lunches, vending machines, and other competitive foods (see Anderson,Butcher, and Schanzenbach 2010, and Hoynes and Schanzenbach 2015 for literaturereviews). Cawley, Frisvold, and Meyerhoefer (2013) find that increased time in PE

13. For example, if two groups are within 5 points, and the students who are failing are very close to the passingscore, it may require less effort to move the school to passing than in a school with one group on the margin,but the highest-scoring student who has failed the test has to improve by 30 or more points. We do not havethe underlying data that allow us to categorize schools according to the improvement that individual studentsneed to make to achieve a passing score. However, even this detailed underlying data would not necessarilysolve the problem. A school might be on the margin of passing because of a negative shock to some or all ofits students (a barking dog or bad cold on the test day), and schools might expect mean reversion to improvescores with little or no effort on the part of the schools.

60


classes because of state mandates reduces obesity among elementary school children,though the impact is concentrated among boys.14

A smaller number of studies has focused specifically on the role of accountability onobesity-related outcomes, finding results that support the hypothesis that accountabilitypressure affects school environments in ways that would be expected to increase obesity.Figlio and Winicki (2005) find that schools facing accountability sanctions increase thenumber of calories offered in their school lunches during the testing period. Andersonand Butcher (2006b) find that schools in states with accountability measures are morelikely to give students access to junk food (and schools more likely to give studentsaccess to junk food have students with higher BMI).15 Yin (2009) uses cross-statedifferences in the implementation of accountability laws (pre-NCLB) to explore theeffects of accountability on obesity, finding that high school students in states withaccountability laws show a significant increase in BMI and obesity rates. Exploringpotential mechanisms, she finds evidence that female adolescents’ participation in PEclasses declines with the introduction of accountability.16

As discussed in more detail subsequently, to explore the possibility that NCLBmight have increased children’s rate of overweight, we use data on test performanceas well as rates of overweight and obesity for schools in Arkansas. Although ouranalysis is reduced-form and cannot isolate the particular mechanisms through whichaccountability pressure contributes to overweight, there is some evidence that principalsreport undertaking behaviors that could affect students’ calorie balance. To betterunderstand the mechanisms connecting accountability and obesity rates, we conductedan online survey of Arkansas school principals, inquiring about school practices andhow they changed after the adoption of NCLB.17 Almost 22 percent of respondingelementary school principals in Arkansas reported cutting time from recess after NCLBwas adopted.18 This is consistent with findings from the Center on Education Policy(2007) that 20 percent of school districts nationwide have decreased recess time sinceNCLB was enacted, with an average decrease of 50 minutes per week. In addition,a quarter of survey respondents in Arkansas report using food as a reward for goodacademic performance, and 10 percent reported holding fundraisers that involved food.

3. METHODOLOGYAs discussed in section 2, past research has used a variety of methods to identify aschool’s exposure to accountability pressures. Similar to Reback, Rockoff, and Schwartz(2014), our main approach focuses on schools most likely to be marginal, which we

14. On the other hand, Cawley, Meyerhoefer, and Newhouse (2007) find no evidence among high school studentsthat an increase in time in PE class reduces students’ body weight or likelihood of being overweight.

15. This paper uses a two sample two-stage least squares estimation strategy, and whether or not the school is ina state that has an accountability rule is one of the factors used in the first stage which predicts the fraction ofschools in a county that gives students access to junk food.

16. Indicating the potential for school accountability to spill over to children’s health more generally, Bokhari andSchneider (2011) find that state accountability policies lead to more children being diagnosed with attentiondeficit hyperactivity disorders and prescribed psychostimulant drugs for its treatment.

17. Sample size of the principal survey is 191, or approximately a 20 percent response rate. Responding schoolswere positively selected based on test scores, socioeconomic status, and whether they were passing underNCLB. See Anderson, Butcher, and Schanzenbach (2011) for more details.

18. Only 1 percent of the elementary school sample reported reducing time spent in PE class, though 4 percentreported pulling poor-performing students in from recess or PE class to work further on tested subjects.

61


define as whether the school’s minimum-scoring grade-test-subgroup was within 5percentage points of meeting the AYP proficiency rate in a given year.19 It is importantto note, though, that we are not trying to estimate a precise reaction to accountabil-ity pressure—that is, to obtain estimates implying that a school missing AYP by xpercentage points would be predicted to increase the rate of student overweight by ypercentage points. Rather, given past findings on the effects of accountability pressureson school behaviors, we simply want to determine if there is any evidence for thesebehaviors having unintended spillover effects on student rates of overweight. Thus, toinvestigate the role of NCLB on students’ overweight status, we build up to estimatingthe following model:

overwgts t = α + βpressureds t−1 + πoverwgts t−1 +4∑

j=1γ j pctnw

js t +

4∑

j=1δ j pctpoor

js t

+4∑

j=1ϕ j mathprof

js t−1 +

4∑

j=1θ j litprof

js t−1 + ωyeart + εs t , (1)

where overwgt is the percentage of students in school s at time t who are overweight.The variables mathprof and litprof are measures of the school’s overall proficiency rate(relative to the AYP goal) in math and literacy, respectively, pctnw is the percent of thestudent enrollment that is nonwhite, pctpoor is the percent of student enrollment whoare economically disadvantaged, and year is a linear time trend. The variable pressuredis an indicator for whether, as of the prior year, the school is likely to be pressured byNCLB. In our main results this is an indicator variable equal to 1 if the minimum-scoringgrade-test subgroup was within 5 percentage points of the AYP proficiency thresholdin any previous year.20 Note that we also include a lagged dependent variable—thatis, the prior year’s rate of overweight—that serves to control for a host of fixed andslow-moving unobserved determinants of student health.

This specification is designed to eliminate (or at least greatly reduce) any correlationbetween pressuredst-1 and est. First, we include year to account for any joint trends in therate of overweight and the probability of being pressured.21 Second, we flexibly controlfor a school’s poverty rate and racial composition, because these factors are known topredict both test scores and the rate of overweight. To allow for a nonlinear relationship,we include polynomials to the fourth degree in each factor. We control for a measureof the school’s overall proficiency rates in math and reading in a similarly flexiblemanner. Recall that the accountability rules assign AYP status based on the school’slowest-scoring subgroup that meets the minimum group size. As a result, two schoolswith similar overall performance can find themselves in different NCLB statuses basedon differences in proficiency rates among their lowest-scoring subgroup. In fact, they

19. As described earlier, there is no theoretically “clean” cutoff for where in the distribution NCLB pressure occurs,and 5 percentage points is an ad hoc cutoff. Using plus or minus 1, 3, . . . , 15 percentage points as the cutoffsresults in coefficients that are within the 95 percent confidence interval of our main result (see Appendixfigure B.1 in the online appendix).

20. In a series of robustness checks, we define “pressured” in a few different ways, as described later in this paper.21. We also experiment with allowing the linear trend to differ by the school’s demographics or past rate of

overweight. This will control for differential trends in overweight for schools that differ by demographics orinitial weight. Including these controls for differential trends has no impact on the role of pressure.

62


may even have similar subgroup scores, but in one school the subgroup may not be nu-merically large enough to count toward the rating. Thus, by controlling generally for theschool’s proficiency, the coefficient of interest is driven by differences in achievementacross subgroups. In other words, two schools with the same overall achievement ratesmay be very similar, but one school was pressured by NCLB because of a strugglingsubgroup, whereas the other school did not face that risk.22 Our regressions essentiallycompare the rate of overweight across these two schools.

Finally, we also control for the prior year’s rate of overweight at the school. Theprior year’s rate of overweight is determined in part by factors such as the school’slocation, the students’ genetic propensity for overweight, and demographics that wedo not observe.23 To the extent that these unobserved determinants of overweight arefixed (such as school location) or moving only slowly over time (such as the compositionof the student population), then directly including the lagged rate of overweight willcontrol for these unobserved determinants. As a result, the remaining potential sourceof bias would have to be unobserved—but transitory—determinants of a school’s rateof overweight that are also correlated with our measure of NCLB pressure. Ultimately,although we cannot claim to rule out all potential sources of bias in the estimatedimpact of NCLB pressure on a school’s rate of overweight, we can rule out the mostreasonable sources of correlation between pressuredst-1 and est. In particular, our method-ology controls for differences in observed demographics and overall school quality, andunobserved differences that would be captured in last year’s rate of overweight.

4. DATA FROM ARKANSASIn order to determine whether there is a relationship between NCLB pressure andchildren’s overweight status, we construct a unique dataset from different publiclyavailable sources that merges school-level information on the percentage of studentsin a grade in a school (overall and by subgroup) who achieve a passing score onstandardized math and literacy tests, the rate of overweight, and other demographiccharacteristics. Details of the final dataset creation can be found in Appendix A in theonline appendix.

Arkansas Assessment of Childhood and Adolescent Obesity

In 2003, the state of Arkansas passed an act intended to help combat childhood andadolescent obesity.24 Although obesity has been increasing nationwide, obesity levelswere particularly high in Arkansas. In 2003, about 21 percent of school-aged childrenin Arkansas were obese or overweight, and this figure was about 17 percent for thenation as a whole (Ogden et al. 2006). A central component of the initiative was thereporting of health risk information to parents (ACHI 2004).

22. One might be interested in measuring whether there is a particularly strong impact on the rate of overweightof students in the actual subgroups that were marginal. Unfortunately, to estimate this, we do not have accessto sufficiently disaggregated data on weight outcomes.

23. Location could be important for multiple reasons. For example, Currie et al. (2010) find that being located closeto fast food restaurants increases a school’s students’ probability of being obese, and Anderson and Butcher(2006a) point to the potential importance of being able to walk or bike to school.

24. This section draws heavily from the yearly reports on the Arkansas Assessment of Childhood and AdolescentObesity released by the Arkansas Center for Health Improvement (ACHI). Reports are available online at:www.achi.net.

63


The Arkansas Center for Health Improvement spearheaded the effort to collectheight and weight information for each school child in the state of Arkansas. Thiseffort included ensuring that each school had the equipment and trained personnelnecessary to accurately weigh and measure each child.25 After children were weighedand measured, a personalized letter then went home to each parent describing thechild’s BMI, where this fit in the BMI distribution (whether the child was obese,overweight, healthy weight, or underweight), and the type of health risks that mightbe associated with an unhealthy BMI.26 Parents of children with an unhealthy weightwere urged to consult a physician.

Importantly, schools were not held accountable for their students’ BMI, but weremerely the conduit through which the state could measure the information and provideit to parents. The implicit assumption of this effort was that if the state provides betterinformation to parents, then parents could make—or help their children make—betterinformed, more healthful choices that could improve children’s weight outcomes. Anannual public report is available on the Arkansas Center for Health Improvement Website with the percent of students who are underweight, normal weight, overweight, andobese at each public school in Arkansas.27 Thus, thanks to the Arkansas Assessmentof Childhood and Adolescent Obesity, we have panel data on school-level rates ofoverweight from 2004 to 2010. Note that we do not have access to the information atthe individual student level.

School Academic Performance Reports

One of the requirements of NCLB was to make publicly available school-level infor-mation on the passing rates, both overall and for student subgroups. The ArkansasDepartment of Education provided us with school report card data for 2002–10, whichprovide information on the percent of students in each tested grade who scored “pro-ficient” on the literacy and math tests. These percentages are reported both overall forthe grade and for subgroups as well.28

We coded schools as being “pressured” by NCLB according to the relationshipbetween the pass rate for the minimum passing subgroup relative to the threshold passrate required to make AYP for that year (Appendix table A.1 in the online appendix liststhe thresholds for test by grade and year). Our preferred definition designates a schoolas pressured if the minimum passing subgroup was within plus or minus 5 percentagepoints of the threshold at any point in the past. There are many details involved indetermining AYP status and it is impossible for us to perfectly predict which schoolsin Arkansas make adequate yearly progress with the publicly available data aggregatedto the grade-level, subject, and subgroup by school level. Our approach, however, onlyrequires that we have correctly identified the marginal schools—those close enough to

25. Training included taking each measure a number of times to ensure accuracy.26. Weight categories are based on age-by-gender percentiles from a fixed population, where underweight is below

the 5th percentile, overweight is above the 85th percentile, and obese is above the 95th percentile.27. Note that for some schools, overweight and obesity rates are not broken out separately, but combined, so we

use this measure of overweight or obese in our models.28. The subgroups are white, African American, Hispanic, and low socioeconomic status students. Pass rates are

listed as long as there are ten in a subgroup but only count toward AYP if there are forty individuals or anumber equaling at least 5 percent of school enrollments, whichever is greater. We omit the English languagelearners and students with disabilities subgroups because of lack of consistent data on group size.

64


Table 1. Summary Statistics: Analysis Sample

Full Pressured NeverSample in Past Pressured

(1) (2) (3)

Pressured in past 0.553 1 0(0.497)

Below pressured margin 0.239 0 0.535(0.427) (0.499)

Above pressured margin 0.207 0 0.465(0.406) (0.499)

Overweight rate 38.44 38.92 37.85(6.597) (6.281) (6.925)

Percent nonwhite 29.99 24.45 36.85(28.74) (25.24) (31.24)

Percent economically disadvantaged 57.15 56.48 57.98(19.78) (17.57) (22.20)

English proficiency rate relative to AYP 9.319 9.513 9.079threshold (previous year) (17.27) (15.10) (19.63)

Math proficiency rate relative to AYP 11.48 12.38 10.36Threshold (previous year) (18.14) (14.70) (21.60)

Overweight rate (previous year) 38.27 38.69 37.76(6.264) (5.979) (6.565)

Observations 4,588 2,539 2,049

Notes: Pressured in past means that the minimum-scoring subgroup hada proficiency rate within 5 points of the AYP target for some year in thepast. Above AYP margin implies that the minimum-scoring subgroup hasnever had a proficiency rate within 5 points of the AYP target, and had aproficiency rate more than 5 points above the AYP target last year. BelowAYP margin implies that the minimum-scoring subgroup has never had aproficiency rate within 5 points of the AYP target, and had a proficiency ratemore than 5 points below the AYP target last year. Overweight rate includesall weights above normal weight. Standard deviations in parentheses.

the threshold that they feel it is warranted to make changes they believe will affect testscores. We check the robustness of our results to alternate definitions of “pressure,”but the definition is always based on the AYP designated passing threshold for a schoolby test, grade, and year.

School Demographics

We also control for measures of a school’s demographic characteristics. These includethe percent nonwhite (black or Hispanic) and percent economically disadvantaged.Beginning in 2008, the school report card data directly report the overall and subgroupsizes, which allows us to create these measures of demographics. Prior to that, we usethe Common Core Data for each school to construct these measures, as described inmore detail in Appendix A in the online appendix.

5. RESULTSSummary Statistics

We start by examining some basic descriptive statistics in table 1, where column 1presents the overall sample, column 2 presents the pressured sample, and column 3presents the never pressured sample. The majority of observations—about 55 percentof school-by-year cells in our data—have faced our definition of NCLB pressure in

65


the past because their minimum-scoring subgroup has scored within 5 points of theAYP threshold. This measure of pressured implies splitting the full sample into thestock of schools that have been pressured, and the stock of schools that have neverbeen pressured. To get a better understanding of the characteristics of this stock ofnonpressured schools, we additionally create a flow measure of whether the lowestscoring grade-test-subgroup last year was more than 5 points below the threshold(below the margin) or more than 5 points above the threshold (above the margin).29 Inthe average school-by-year observation for the full sample in column 1, about 24 percentare coded below the margin by this definition, and another 20 percent are above theAYP margin. Additionally in column 1, we see that the mean for percent of overweightstudents is 38.4; 57.2 for percent of students economically disadvantaged; and 30 forpercent nonwhite. Finally, on average, the measure of schools’ overall passing rates(that is, not disaggregating by subgroup) exceeded the target English proficiency rateby 9.3 percentage points and the target math proficiency rate by 11.5 percentage pointsin the prior year.

Comparing column 2 with column 3 shows that pressured schools have an averagerate of overweight that is slightly higher than never pressured schools (38.92 versus37.85), but this does not appear to be driven by demographic differences at theseschools. Pressured schools have a very slightly lower average rate of economicallydisadvantaged students, and a much lower average percentage of nonwhite students.In terms of overall school performance, both pressured and nonpressured schools havesimilar average overall passing rates in English, and pressured schools score higheron average in math. Given the generally higher rates of obesity for poor and minoritychildren in the United States, all else equal, we would expect the schools in column 3 tohave higher rates of overweight. In fact, though, in the means we find a higher averagerate of overweight for the pressured schools in column 2.30

Table 2 presents the results of building up model 1 by sequentially adding controlvariables. Column 1 is a basic regression that controls only for a time trend, column 2adds controls for the fourth-order polynomials in the measures of overall math andliteracy proficiency rate, and column 3 additionally adds fourth-order polynomials inpercent nonwhite and economically disadvantaged in the school-year cell.31 Standarderrors are clustered by school throughout. The results are stable across these specifi-cations, with the rate of overweight about 1 percentage point higher for those schoolslabeled as pressured by NCLB. Although adding controls does add to the explana-tory power of the model, there is very little impact on the key coefficient. Our in-terpretation of this result is that although the school’s overall proficiency rates anddemographics are important correlates of the overall rate of overweight, none of these

29. It is important to realize that although the control group of never pressured schools includes both schoolsclearly failing last year and clearly passing last year, this should not be interpreted as representing schools thatalways fail or always pass. Of schools clearly failing in one year, 10 percent are clearly passing the next, and ofschools clearly passing in one year, 13 percent are clearly failing the next.

30. Appendix table A.2 in the online appendix provides summary statistics separately for schools above and belowthe pressured margin.

31. We experimented with allowing the trend to vary across demographic groups and/or past overweight rate. Inall cases the coefficient on pressured was significant and between 0.50 and 0.52, so for simplicity we use onlya simple trend. We also investigated allowing the impact of pressure to vary by whether the school just missed(versus just passed) the threshold, but the interaction term was close to zero and insignificant.

66


Table 2. Effects of Accountability Pressures on School Rates of Over-weight Students

(1) (2) (3) (4)

Pressured in past 1.033∗∗∗ 1.220∗∗∗ 1.205∗∗∗ 0.522∗∗∗(0.351) (0.313) (0.286) (0.151)

Overweight rate 0.608∗∗∗(previous year) (0.0153)

Overall proficiency rate NO YES YES YES

Demographic controls NO NO YES YES

Observations 4,588 4,588 4,588 4,588

R2 0.007 0.125 0.248 0.496

Notes: Pressured in past is defined as whether a school’s minimum-scoringsubgroup had a proficiency rate within 5 points of the AYP target for someyear in the past. Overweight rate includes all weights above normal weight.Demographic controls are a quartic in percent nonwhite and a quartic inpercent economically disadvantaged. Overall proficiency rate controls are aquartic in the standardized overall literature proficiency rate and a quarticin the standardized overall math proficiency rate. All models include anannual trend. Standard errors which are robust to heteroskedasticity andwithin-school correlation are in parentheses.∗p < 0.1; ∗∗p < 0.05; ∗∗∗p < 0.01.

variables is strongly correlated with our measure of NCLB pressure. In other words,two schools that are otherwise very similar in terms of overall achievement and de-mographic makeup may face starkly different pressures under NCLB. That differentialpressure may lead to policies and practices being adopted that increase students’ rate ofoverweight, as suggested by the positive estimated coefficient on our measure of NCLBpressure.

The possibility remains, of course, that there are unobserved determinants of aschool’s obesity rate that are correlated with our measure of pressure. In addition toschool policies, the rate of overweight will be determined by factors such as the school’slocation, students’ genetic predispositions to being overweight, local norms around foodconsumption and exercise, and other characteristics that we cannot observe. To the ex-tent that these unobserved determinants are either fixed or change relatively slowly,though, we can control for them by including the school’s lagged rate of overweight,because the lagged rate was itself determined by these unobserved characteristics.32

Column 4 incorporates the lagged rate of overweight along with a trend and polyno-mials in the school’s overall proficiency rate and demographic characteristics, and isour preferred specification. After controlling for the lagged rate of overweight, the onlypotential source of omitted variables bias would be due to unobserved, but transitory,determinants of a school’s rate of overweight that are also correlated with our measureof NCLB pressure. Note also that this saturated specification leaves the model essen-tially trying to explain a one-year change in the rate of overweight, a potentially much

32. An alternative approach is to include school fixed effects to control for unobserved (and observed) determinantsof the rate of overweight that are fixed over time. When we control for these other potential determinants ofthe rate of overweight using fixed effects, we estimate a coefficient on pressured that is positive, statisticallysignificantly different from zero, and not statistically significantly different from the estimate in our preferredspecification (see Appendix table A.3 in the online appendix).

67


more difficult task than explaining its level. As the results in column 4 indicate, thereis a strong positive relationship across years in the rate of overweight, with a coefficientof 0.61 on the lagged rate. Nonetheless, even after accounting for the lagged rate ofoverweight and thus all of the otherwise unobservable factors for which that controls,there is still an effect of NCLB pressure that is positive and statistically significantlydifferent from zero. The estimated coefficient is 0.522, indicating that schools facingNCLB pressure have about a one-half of one percentage point increase in the rate ofoverweight since the previous year. Simulations in Anderson, Butcher, and Schanzen-bach (2010) suggest that an increase of this magnitude could be obtained with justover a twelve-minute per week reduction in moderate activity (such as recess) over theschool year, or well fewer than an additional 150 calories consumed per week.33 Thus,these results from our preferred model that controls for unobserved (but slow-movingor fixed) determinants of overweight are well within the range of what is plausible foran annual change in the rate of overweight from the types of policy changes schoolsadopt in response to NCLB pressure.

Robustness Checks

The results of table 2 are consistent with the idea that NCLB pressures lead schools tomake changes to policies and practices that are harmful to students’ weight outcomes.In that table, we define NCLB pressure according to whether a school’s minimum-scoring subgroup had a passing rate within 5 percentage points of the AYP thresholdin any past year. This is our preferred specification because, by statute, a school’s AYPstatus is determined by the minimum-scoring subgroup. If all subgroups but one wereeasily meeting the proficiency targets, and the minimum-scoring subgroup is close tothe passing threshold, then the school would achieve AYP status if it could modestlyimprove only one subgroup’s performance. Schools in this category may have beenespecially likely to make the types of marginal changes that could affect students’ ratesof overweight. We believe these schools are unlikely to immediately undo these changes,even if they meet AYP the next year. Recall that the AYP thresholds are increasing overtime, making it important for schools to maintain their effort levels. Thus, we preferthis measure of pressure that turns on once, and remains on. However, by focusingon the minimum-scoring subgroup having been close to the target at any time inthe past, our treatment group will grow over time, with our control group potentiallybecoming more extreme. That is, over time, schools that have never been close to theAYP threshold may be more likely to have been consistently missing or making AYPby more than 5 points. Table 3 investigates the importance of the changing stock ofpressured schools by showing our preferred specification for a rolling sample, startingwith just the first two years of data in column 1, up to our main sample in column5, which simply reproduces column 4 of table 2. The estimates are always statisticallysignificant, and quite stable, remaining in the range of 0.4 to 0.5.

Although this stock measure of being pressured based on the minimum-scoringsubgroup is our preferred measure, there are other ways to operationalize the concept

33. Although the published tables only present calorie changes in 150-calorie increments, unpublished resultsindicate that it would actually take under 50 extra calories per week to increase a school’s rate of overweight by0.522 percentage points.

68


Table 3. Effects of Accountability Pressures on School Rates of Overweight Students: AlternativeSample Periods

2005—06 2005—07 2005—08 2005—09 2005—10(1) (2) (3) (4) (5)

Min. subgroup pressured in past 0.427∗∗ 0.481∗∗∗ 0.375∗∗ 0.493∗∗∗ 0.522∗∗∗(0.214) (0.165) (0.153) (0.155) (0.151)

Observations 1,388 2,127 2,941 3,766 4,588

R2 0.646 0.644 0.582 0.520 0.496

Notes: Minimum subgroup pressured in past is defined as whether the lowest scoring subgroupin a school had a proficiency rate within 5 points of the AYP target for some year in the past. Allmodels include the lagged rate of overweight, a quartic in percent nonwhite, a quartic in percenteconomically disadvantaged, a quartic in the standardized overall literature proficiency rate, aquartic in the standardized overall math proficiency rate, and an annual trend. Standard errorswhich are robust to heteroskedasticity and within-school correlation are in parentheses.∗∗p < 0.05; ∗∗∗p < 0.01.

of NCLB pressure. First, the assumption that schools would not immediately reversethe changes in practices that they made in response to NCLB pressure may be incorrect,meaning schools’ behaviors may only be influenced by the prior year’s proficiency re-sults. Additionally, although overall AYP status is determined by the lowest-performingsubgroup, the passing rate of each group is publicly disclosed. This gives schools theincentive to get as many subgroups as possible up to the proficiency targets. As a result,having any subgroup on the margin of the passing threshold could trigger the changesin policies and practices that increase students’ rate of overweight. Finally, it may be thecase that all schools that are not meeting AYP will implement these types of policies,even if on their own such marginal changes are unlikely to increase scores sufficientlyto reach the thresholds. In such a case, some of our control schools are, in fact, be-ing pressured by NCLB, and only schools clearly meeting AYP should be consideredunpressured. Recall, however, that only when using a flow measure of pressure (i.e.,pressured just last year) is there an obvious way to assign schools as being above orbelow the pressure point. For our preferred stock measure of pressure (i.e., pressuredat any time in the past), we simply assign nonpressured schools a flow measure ofbeing below the margin if the lowest-scoring subgroup last year missed AYP by morethan 5 points.

Table 4, then, implements a range of alternative definitions of NCLB pressure.Each column represents a different definition of NCLB pressure, with the bottompanel restricting the control group to schools coded as above the AYP margin. Thefirst column uses our preferred definition of pressured, so the first column of the toppanel reproduces column 4 from table 2. The second column continues to define NCLBpressure by the minimum-scoring subgroup, but only codes a school as facing pressureif it was on the AYP margin the prior year. In this column, the effect is positive but notsignificantly different from zero.34 A potential explanation for this lack of significanceis that our initial assumption is correct, and schools do not reverse their policy changesif they move out of marginal status, and thus the comparison group is increasingly

34. Using this measure with different samples (as in table 3) reveals that the impact of “pressured last year”declines monotonically across the samples but is significant, around p = 0.05 for the first two samples.

69


Table 4. Effects of Accountability Pressures on School Rates of Overweight Stu-dents: Alternative Definitions of Pressure and Comparison Groups

Pressure Definition

Minimum Minimum Any AnySubgroup, Subgroup, Subgroup, Subgroup,Any Past Pressure Any Past PressurePressure Last Year Pressure Last Year

(1) (2) (3) (4)

Panel A: Comparing Marginal with Non-marginal Schools

Pressure indicator 0.522∗∗∗ 0.170 1.129∗∗∗ 0.650∗∗∗(0.151) (0.189) (0.195) (0.179)

R2 0.496 0.495 0.498 0.496

Panel B: Comparing High, Marginal and Low-Scoring Schools

Pressure indicator 1.052∗∗∗ 0.550∗∗ 1.169∗∗∗ 0.692∗∗∗(0.202) (0.231) (0.210) (0.220)

Below pressure margin 1.043∗∗∗ 0.749∗∗∗ 0.349 0.102(0.268) (0.267) (0.418) (0.318)

R2 0.497 0.495 0.498 0.496

Notes: Sample size is 4,588. The columns differ in how NCLB pressure is defined.Minimum subgroup, any past pressure (column 1) defines the pressured group bywhether a school’s lowest scoring subgroup had a proficiency rate within 5 points of theAYP target for some year in the past. Minimum subgroup, pressure last year (column2) defines it based on the school’s lowest-scoring subgroup score the previous yearonly. Any subgroup, any past pressure (column 3) is an indicator for whether anyaccountability subgroup in the school had a proficiency rate within 5 points of theAYP target in any prior year. Any subgroup, pressure last year (column 4) indicateswhether any subgroup had a marginal proficiency rate the prior year. All modelsinclude the lagged rate of overweight, a quartic in percent nonwhite, a quartic inpercent economically disadvantaged, a quartic in the standardized overall literatureproficiency rate, a quartic in the standardized overall math proficiency rate, and anannual trend. Panel A compares the pressured group to nonpressured groups, andPanel B includes controls for the pressured group and below the pressured margin(omitted group is above the pressured margin—see text for definition). Standarderrors which are robust to heteroskedasticity and within-school correlation are inparentheses.∗∗p < 0.05; ∗∗∗p < 0.01.

populated by schools that were pressured in the past. In other words, over time theschools in the control group undertake the same behaviors as those in the treatmentgroup, having implemented them in the past when they were first pressured.

The third and fourth columns allow for the possibility that schools are focused ongetting as many subgroups to the target proficiency level as possible. Thus, a school isdefined as pressured if any subgroup is in the margin of passing, either at any time inthe past (column 3) or just last year (column 4). Note that in columns 1 and 2, in schoolslabeled as facing NCLB pressure, all other subgroups for these pressured schools wereeither easily meeting their proficiency targets or were just slightly above the minimum-scoring group and also within the 5-point region. By contrast, in columns 3 and 4 someof the school’s other subgroups may be more than 5 points below the target threshold,but the school is defined as facing NCLB pressure as long as at least one subgroupis close to passing. Turning to column 3, the estimated coefficient on “pressured” is1.13—about twice as large as the coefficient from our main specification. Finally, incolumn 4 we find an estimate of 0.65, which is well within the 95 percent confidenceinterval of the coefficient from our main specification.

70


Turning to the first column in the bottom panel, we see that using our preferreddefinition of “pressured” at some time in the past, but comparing this group only tothose schools where the minimum-scoring subgroup was more than 5 points abovethe AYP threshold last year, results in a much larger estimate of 1.05. Note that this isvery similar to the coefficient in the following row, which compares schools for whichthe minimum-scoring subgroup last year was more than 5 points below the thresholdwith those for which it was more than 5 points above.35 In the second column, we usea consistent flow measure, comparing schools whose minimum-scoring subgroup lastyear was within 5 points of AYP with those that were greater than 5 points. In this case,the contemporaneous pressure measure is significantly different from zero, at 0.55.At the same time, the coefficient for schools missing by more than 5 points is evenlarger (but not significantly different) at 0.75. These results seem consistent with theidea described earlier that the “control” group using the contemporaneous measureis contaminated with schools that have also felt pressure in the past, and have madepolicy changes that affect rates of overweight. In columns 3 and 4 the comparisongroup is made up of schools for which no subgroup scored less than 5 points aboveAYP last year, and the second row is schools for which some subgroup scored morethan 5 points below last year.36 Both of these alternative definitions find positive andsignificant coefficients on pressure, but positive and insignificant coefficients on beingbelow the margin. The estimated coefficients of 1.17 and 0.69 on the pressure indicatorare very similar to those from the top panel.

In sum, using alternate definitions of pressure and a variety of time periods doeslittle to change our overall finding that schools facing NCLB pressure see increasesin their students’ growth rate of overweight. Using our preferred specification, whichincludes the school’s lagged rate of overweight and defines NCLB pressure by whetherthe lowest-performing subgroup has ever been on the margin of the passing threshold,we find that facing NCLB pressure implies about a 0.5 percentage-point increase ina pressured school’s growth rate of overweight. This result is fairly stable over time.Only when we change our definition to a contemporaneous measure, where a schoolis only categorized as pressured if the minimum-scoring subgroup was within 5 pointsin the prior year, do we find a statistically insignificant, but still positive, effect. If thismeasure of pressure is compared to only schools whose minimum-scoring subgroupscored more than 5 points above the threshold last year (column 2, panel B), however,significance is obtained. This suggests that schools whose minimum-scoring subgroupscored more than 5 points below the threshold last year are also likely to have beenpressured by NCLB in past years. Overall, our alternate approaches find significantestimates of between about 0.4 and 1.2, all of which are consistent with the idea thatpolicy changes due to NCLB have contributed to increasing rates of overweight amongpressured schools’ students.

35. It is important to remember that these “tails” are defined on a flow basis (the performance of the lowest-scoring subgroup last year), and the “pressure” measure is defined on a stock basis (the performance of thelowest-scoring subgroup any time in the past).

36. Defining the “tails” here is even more complicated because of the use of any subgroup to define pressured.Even when focused on just last year, although some subgroup is within 5 points, another subgroup may bebelow 5 points and another subgroup above 5 points. Thus, if any subgroup missed AYP by more than 5 pointslast year, we assign the school to that group, making the control group schools where no subgroup was lessthan 5 points above AYP last year.

71


Notes: Pressured implies the minimum-scoring subgroup had a proficiency rate within 5 points of the AYP target. Dashed linesrepresent a heteroskedasticity-robust 95% confidence interval.

Figure 1. Estimated Impact of Being Pressured under NCLB on Overweight Rate, by Year.

Event Study

The timing of increases in the rate of overweight can be estimated directly using anevent study analysis. Specifically, we fit the following equation to our analysis sample:

overwgts t = α +6∑

i=−5βi 1 (τs t = i ) +

4∑

j=1γ j pctnw

js t +

4∑

j=1δ j pctpoor

js t

+4∑

j=1ϕ j mathprof

js t−1 +

4∑

j=1θ j litprof

js t−1 + ωyeart + εs t , (2)

where overwgt is the percentage of students in school s at time t who are overweightor obese, and mathprof, litprof, pctnw, pctpoor, and year are as above. Note that theevent study does not include the lagged dependent variable. The key set of explanatoryvariables are the event-year dummies, τ st, defined so that τ = 0 in the first year that aschool’s minimum subgroup scores within 5 points of the passing threshold (i.e., thefirst year a school is pressured by our preferred measure), and τ = 1 denotes the firstyear after a school is declared to be under pressure, and so on. In the years in whichτ < 0, the school was not yet under pressure. The coefficients are measured relative tothe first year of pressure, that is, τ = 0. Schools that never have pressured status arestill in the dataset, but they only help identify the relationship between covariates andoverweight because the vectors of τ ’s are all zero for such schools. Because we havea relatively short panel, the event study analysis is unbalanced, meaning that not allschools facing NCLB pressure are observed in all time periods.

Figure 1 plots the event-year coefficients from estimating equation 2 on our sample.In the period prior to the school’s minimum subgroup first scoring within 5 points ofthe AYP threshold, the school’s rate of overweight is not significantly different from

72


zero (that is, the rate for the omitted time period, or the first year that the school facesNCLB pressure). The year after first becoming pressured, however, the school’s rateof overweight increases by a statistically significant 1 percentage point. Simulationsin Anderson, Butcher, and Schanzenbach (2010) suggest that a one percentage-pointincrease in the rate of overweight could occur after a school year in which moderateactivity levels were cut by twenty-four minutes per week. Alternatively, a slightly largerchange in the rate of overweight is predicted if children consumed 150 additionalcalories on average per week. To the extent that schools responded to NCLB pressureby cutting as little as five minutes per day of recess or adding vending machines thatresulted in students eating an extra pack of chips per week, the results from this eventstudy conform well to the hypothesis that accountability pressures under NCLB mayhave caused increases in the rate of overweight.

6. CONCLUSIONSThrough the No Child Left Behind Act, schools faced increasing incentives to improveperformance on standardized tests, and past research clearly documents that a widevariety of schools’ policies and practices are affected by accountability pressures. Be-cause schools are graded based primarily on standardized test scores (but not on otherstudent outcomes such as children’s health), schools facing accountability pressuremay make decisions designed to increase test scores but that have unintended negativeconsequences for children’s weight.

This paper adds to the small amount of evidence on the effect of school account-ability on child health. We find schools on the margin of making AYP under NCLBrules, and thus presumably under strong pressure to improve test scores, have about a0.5 percentage point higher growth rate of overweight among their students. This resultis not identified by comparing poorly performing schools to better performing schools.In fact, the schools we identify as facing NCLB pressure are generally in the middle ofthe socioeconomic spectrum, with schools that perform both generally very well andvery poorly. Note that this effect is estimated in models that include flexible measures ofoverall school test performance and demographic characteristics, as well as the schools’lagged rate of overweight. This specification helps ensure that the coefficient of interestis not merely picking up other differences across schools on the margin of making AYPthat are correlated with student overweight. Finally, the result is robust in a variety ofdifferent specifications.

These results present the first direct evidence that the NCLB accountability rulesmay have unintended adverse consequences for students’ weight outcomes. As weattempt to improve schools along a given dimension, it seems logical that other areasfor which schools are not explicitly held accountable may suffer. As a result, parents,school administrators, and policy makers should keep in mind the potential for impactson children’s health as they consider how to devise incentives and reallocate schoolresources in pursuit of test score gains.

ACKNOWLEDGMENTWe thank Jannine Riggs, Denise Airola, and Jim Boardman of the Arkansas Department ofEducation for helpful discussions about the Arkansas education data and accountability rules.We received generous financial support from the Robert Wood Johnson Foundation (grant

73


57922), a Rockefeller Center at Dartmouth College Reiss Family Faculty Research Grant, anda Wellesley College Faculty Award. Elora Ditton, Brian Dunne, A. J. Felkey, Brenna Jenny, andAlan Kwan provided excellent research assistance. We thank Eric Edmonds; Jonathan Guryan;seminar participants at UC-Davis, Boston College, Louisiana State University, the Universityof Toulouse; and conference participants at the American Economic Association 2010 annualmeetings and the Rockefeller Center Health Policy Workshop for helpful comments.

REFERENCESAnderson, Patricia M., and Kristin F. Butcher. 2006a. Childhood obesity: Trends and potentialcauses. Future of Children: Child Overweight and Obesity 16(1):19–45. doi:10.1353/foc.2006.0001.

Anderson, Patricia M., and Kristin F. Butcher. 2006b. Reading, writing, and refreshments: Doschool finances contribute to childhood obesity? Journal of Human Resources 41(3):467–494.doi:10.3368/jhr.XLI.3.467.

Anderson, Patricia M., Kristin F. Butcher, and Diane Whitmore Schanzenbach. 2010. Schoolpolicies and children’s obesity. In Current issues in health economics, edited by Daniel Slottjeand Rusty Tchernis, pp. 1–16. Bingley, UK: Emerald Group Publishing Limited. doi:10.1108/S0573-8555(2010)0000290004.

Anderson, Patricia M., Kristin F. Butcher, and Diane Whitmore Schanzenbach. 2011. Adequate(or adipose?) yearly progress: Assessing the effect of No Child Left Behind on children’s obesity.NBER Working Paper No. 16873.

Arkansas Center for Health Improvement (ACHI). 2004. The Arkansas assessment of childhoodand adolescent obesity. Little Rock: ACHI.

Ballou, Dale, and Matthew G. Springer. 2008. Achievement trade-offs and No Child Left Behind.Unpublished paper, Vanderbilt University.

Blankenship, Virginia H., and Joshua H. Barnet. 2006. AYP in Arkansas: Who’s on track?Arkansas Education Report 3(2):1–13.

Bokhari, Farasat A. S., and Helen Schneider. 2011. School accountability laws and the con-sumption of psychostimulants. Journal of Health Economics 30(2):355–372. doi:10.1016/j.jhealeco.2011.01.007.

Carnoy, Martin, and Susanna Loeb. 2002. Does external accountability affect student outcomes?A cross-state analysis. Educational Evaluation and Policy Analysis 24(4):305–331. doi:10.3102/01623737024004305.

Cawley, John, Chad Meyerhoefer, and David Newhouse. 2007. The impact of state physical educa-tion requirements on youth physical activity and overweight. Health Economics 16(12):1287–1301.doi:10.1002/hec.1218.

Cawley, John, David Frisvold, and Chad Meyerhoefer. 2013. The impact of physical educa-tion on obesity among elementary school children. Journal of Health Economics 32(4):743–755.doi:10.1016/j.jhealeco.2013.04.006.

Center on Education Policy. 2007. Choices, changes, and challenges: Curriculum and instruction inthe NCLB era. Washington, DC: Center on Education Policy.

Chakrabarti, Rajashri. 2013a. Accountability with voucher threats, responses, and the test-taking population: Regression discontinuity evidence from Florida. Education Finance and Policy8(2):121–167. doi:10.1162/EDFP_a_00088.

74

http://dx.doi.org/10.1353/foc.2006.0001http://dx.doi.org/10.3368/jhr.XLI.3.467http://dx.doi.org/10.1108/S0573-8555(2010)0000290004http://dx.doi.org/10.1016/j.jhealeco.2011.01.007http://dx.doi.org/10.3102/01623737024004305http://dx.doi.org/10.1002/hec.1218http://dx.doi.org/10.1016/j.jhealeco.2013.04.006http://dx.doi.org/10.1162/EDFP_a_00088


Chakrabarti, Rajashri. 2013b. Vouchers, public school response, and the role of incentives: Evi-dence from Florida. Economic Inquiry 51(1):500–526. doi:10.1111/j.1465-7295.2012.00455.x.

Chakrabarti, Rajashri. 2014. Incentives and responses under No Child Left Behind: Crediblethreats and the role of competition. Journal of Public Economics 110:124–146. doi:10.1016/j.jpubeco.2013.08.005.

Chiang, Hanley. 2009. How accountability pressure on failing schools affects student achieve-ment. Journal of Public Economics 93(9):1045–1057. doi:10.1016/j.jpubeco.2009.06.002.

Cullen, Julie Berry, and Randall Reback. 2006. Tinkering toward accolades: School gamingunder performance accountability system. In Improving school accountability: Check-ups or choice,advances in applied microeconomics, vol. 14, edited by Timothy J. Gronberg and Dennis W. Jansen,pp. 1–34. Amsterdam: Elsevier Science. doi:10.1016/S0278-0984(06)14001-8.

Currie, Janet, Stefano Della Vigna, Enrico Moretti, and Vikram Pathania. 2010. The effect offast food restaurants on obesity and weight gain. American Economic Journal: Economic Policy2(3):32–63. doi:10.1257/pol.2.3.32.

Davidson, Elizabeth, Randall Reback, Jonah E. Rockoff, and Heather L. Schwartz. 2013. Fiftyways to leave a child behind: Idiosyncrasies and discrepancies in states’ implementation ofNCLB. NBER Working Paper No. 18988.

Dee, Thomas S., and Brian A. Jacob. 2011. The impact of No Child Left Behind on studentachievement. Journal of Policy Analysis and Management 30(3):418–446. doi:10.1002/pam.20586.

Dee, Thomas S., and Brian A. Jacob. 2010. The impact of No Child Left Behind on students, teach-ers, and schools. Brookings Papers on Economic Activity (Fall):149–207. doi:10.1353/eca.2010.0014.

Education Week. 2006. Quality counts at 10: A decade of standard-based education. Availablewww.edweek.org/ew/toc/2006/01/05/index.html. Accessed 13 November 2012.

Figlio, David. 2006. Testing, crime and punishment. Journal of Public Economics 90(4–5):837–851.doi:10.1016/j.jpubeco.2005.01.003.

Figlio, David, and Helen Ladd. 2015. School accountability and student achievement. In Handbookof research in education finance and policy, edited by Helen F. Ladd and Margaret E. Goertz,pp. 166–182. New York: Routledge.

Figlio, David, and Cecilia Elena Rouse. 2006. Do accountability and voucher threats improvelow-performing schools? Journal of Public Economics 90(1–2):239–255. doi:10.1016/j.jpubeco.2005.08.005.

Figlio, David, and Joshua Winicki. 2005. Food for thought: The effects of school accountabilityplans on school nutrition. Journal of Public Economics 89(2–3):381–394. doi:10.1016/j.jpubeco.2003.10.007.

Hoynes, Hilary W., and Diane Whitmore Schanzenbach. 2015. U.S. food and nutrition programs.NBER Working Paper No. 21057.

Jacob, Brian. 2005. Accountability, incentives and behavior: The impact of high-stakes testingin Chicago public schools. Journal of Public Economics 89(5–6):761–796. doi:10.1016/j.jpubeco.2004.08.004.

Jacob, Brian A., and Steven Levitt. 2003. Rotten apples: An investigation of the prevalenceand predictors of teacher cheating. Quarterly Journal of Economics 118(3):843–877. doi:10.1162/00335530360698441.

75

http://dx.doi.org/10.1111/j.1465-7295.2012.00455.xhttp://dx.doi.org/10.1016/j.jpubeco.2013.08.005http://dx.doi.org/10.1016/j.jpubeco.2009.06.002http://dx.doi.org/10.1016/S0278-0984(06)14001-8http://dx.doi.org/10.1257/pol.2.3.32http://dx.doi.org/10.1002/pam.20586http://dx.doi.org/10.1353/eca.2010.0014file:www.edweek.org/ew/toc/2006/01/05/index.htmlhttp://dx.doi.org/10.1016/j.jpubeco.2005.01.003http://dx.doi.org/10.1016/j.jpubeco.2005.08.005http://dx.doi.org/10.1016/j.jpubeco.2003.10.007http://dx.doi.org/10.1016/j.jpubeco.2004.08.004http://dx.doi.org/10.1162/00335530360698441


Neal, Derek, and Diane Whitmore Schanzenbach. 2009. Left behind by design: Profi-ciency counts and test-based accountability. Review of Economics and Statistics 92(2):263–283.doi:10.1162/rest.2010.12318.

Ogden, Cynthia L., Margaret D. Carroll, Lester R. Curtin, Margaret A. McDowell, Carolyn J.Tabak, and Katherine M. Flegal. 2006. Prevalence of overweight and obesity in the United States,1999–2004. Journal of the American Medical Association 295(13):1549–1555. doi:10.1001/jama.295.13.1549.

Reback, Randall. 2008. Teaching to the rating: School accountability and distribution of studentachievement. Journal of Public Economics 92(5):1394–1415. doi:10.1016/j.jpubeco.2007.05.003.

Reback, Randall, Jonah Rockoff, and Heather L. Schwartz. 2014. Under pressure: Job security,resource allocation, and productivity in schools under No Child Left Behind. American EconomicJournal: Economic Policy 6(3):207–241. doi:10.1257/pol.6.3.207.

Rockoff, Jonah, and Lesley J. Turner. 2010. Short-run impacts of accountability on school quality.American Economic Journal: Economic Policy 2(4):119–147. doi:10.1257/pol.2.4.119.

Rouse, Cecelia Elena, Jane Hannaway, Dan Goldhaber, and David Figlio. 2013. Feeling the Floridaheat? How low-performing schools respond to voucher and accountability pressure. AmericanEconomic Journal: Economic Policy 5(2):251–281. doi:10.1257/pol.5.2.251.

Sass, Tim R., Jarod Apperson, and Carycruz Bueno. 2015. The long-run effects of teacher cheatingon student outcomes: A report for the Atlanta public schools. Available www.atlantapublicschools.us/crctreport. Accessed 1 October 2015.

Wong, Vivian C., Peter M. Steiner, and Thomas D. Cook. 2013. Analyzing regression-discontinuity designs with multiple assignment variables: A comparative study of four es-timation methods. Journal of Educational and Behavioral Statistics 38(2):107–141. doi:10.3102/1076998611432172.

Yin, Lu. 2009. Are school accountability systems contributing to adolescent obesity? Unpub-lished paper, University of Florida.

76
http://dx.doi.org/10.1162/rest.2010.12318http://dx.doi.org/10.1001/jama.295.13.1549http://dx.doi.org/10.1016/j.jpubeco.2007.05.003http://dx.doi.org/10.1257/pol.6.3.207http://dx.doi.org/10.1257/pol.2.4.119http://dx.doi.org/10.1257/pol.5.2.251http://www.atlantapublicschools.us/crctreporthttp://dx.doi.org/10.3102/1076998611432172

adequate (or adipose?) yearly progress ......patricia m. anderson, kristin f. butcher, and diane...

Documents