imberman 2011 - achievement and behavior in charter schools

ACHIEVEMENT AND BEHAVIOR IN CHARTER SCHOOLS: DRAWINGA MORE COMPLETE PICTURE

Scott A. Imberman*

Abstract—I use a long panel with broad grade coverage to establish whethercharter schools affect cognitive and noncognitive skill formation. Schoolsthat begin as charters generate large improvements in discipline and atten-dance but not test scores, with the exception of math in middle schools. Thissuggests improvements in noncognitive but not cognitive skills, althoughthese improvements do not persist if students return to regular publicschools. Charters that convert from regular public schools have little impacton either skill type. These results are robust to potential biases from selectionoff of precharter trends, attrition, and persistence.

I. Introduction

THE charter school movement is one of the fastest-growing education reforms in the United States today.

Charter schools operate under a contract, called a charter,with a government agency. These schools are provided adegree of autonomy from local school boards and freedomfrom some regulations in return for additional accountabilityrequirements. Despite often being managed by private orga-nizations, charters are public schools and receive almost allof their funding from government sources. Since 1997 thenumber of charter schools in the United States has increasedalmost sixfold, and the number of charter students has morethan doubled since 1999, as is shown in figure 1. As of 2006,1.15 million students nationwide attended charter schools.

One of the largest questions in the literature is how char-ter schools affect the outcomes of students who attend them.It is unclear whether charters are beneficial or detrimentalto students. On the one hand, charters have fewer regu-latory burdens and are at higher risk of being shut downif they underperform. This provides incentives to increaseeffort. On the other hand, charters have high levels of studentturnover, and eliminating certain regulations may be detri-mental to students. In addition to this theoretical ambiguity,

Received for publication October 30, 2007. Revision accepted forpublication November 12, 2009.

* University of HoustonI thank the Maryland Population Research Center for financial support.

I extend my sincerest gratitude to the employees and administrators of ananonymous school district for providing me with data and assistance andfor making this project possible. I am especially grateful for the guidanceand assistance provided by my dissertation advisor, Mark Duggan, and theimmeasurable help from Steven Craig. In addition, I give special thanksto Judy Hellerstein, Bill Evans, Jeff Smith, two anonymous referees, andthe editor, Michael Greenstone, for helpful suggestions and advice. I alsothank Rajashri Chakrabarti, Ken Chay, Aimee Chin, Jose Galdo, JonahGelbach, Ginger Jin, Beom-Soo Kim, Melissa Kearney, Adriana Kugler,Jordan Matsudaira, Jennifer King Rice, John Rust, Seth Sanders, JohnShea, Barbara Sianesi, Alex Whalley, Ye Zhang, Ron Zimmer, and seminarparticipants at Georgia State University, Stanford Institute for EconomicPolicy Research, Mathematica, RAND, University of Houston, Universityof Maryland, UNC-Chapel Hill, Urban Institute, Virginia Tech, APPAM,the North American Summer Meetings of the Econometric Society, andSEA. This work was done as part of my dissertation at the University ofMaryland. All errors remain my own.

The online appendix referred to throughout the article is available athttp://www.mitpressjournals.org/doi/suppl/10.1162/REST_a_00077.

the empirical evidence has been mixed. Of the papers that usemore advanced econometric techniques, some researchersfind insignificant or negative impacts of attending a charterschool (Betts et al., 2006; Bifulco & Ladd, 2006; Buddin &Zimmer, 2005; Hanushek et al., 2007; Sass, 2006; Zimmeret al., 2008; Zimmer & Buddin, 2003, 2006), while others findpositive impacts (Booker et al., 2007; Clark, 2009; Hoxby &Murarka, 2009; Hoxby & Rockoff, 2004; McClure et al.,2005; Solmon & Goldschmidt, 2004; Solmon, Paark, & Gar-cia, 2001).1 Therefore, the effect of charter schools on studentoutcomes is unclear.

One of the potential reasons for the wide variation in resultsis that some charter schools may not see test scores as theirprimary output. Many charters focus on students with spe-cial needs, such as those who are over age for their grade,new immigrants, or students who have difficulty behavingwell in a normal school environment. These students oftenneed instruction not just in their academic ability—cognitiveskills—but in motivation, self-esteem, and self-discipline,which are types of noncognitive skills.

The distinction between how cognitive and noncognitiveskills are affected by education interventions has becomeincreasingly important in the light of recent research showingthat noncognitive skills have substantial influence on labormarket outcomes and degree attainment (Heckman, Stixrud,& Urzua, 2006; Heckman & Rubinstein, 2001). Thus, ifcharters improve noncognitive skills while having a limitedeffect on cognitive skills, they could still be effective toolsfor improving students’ outcomes in later life.

Unfortunately, it is difficult to measure noncognitive skillsdirectly. While testing of achievement is common in schools,other skills are rarely, if ever, tested. However, Heckman et al.(2006) establish that both cognitive and noncognitive skillsimprove behavioral outcomes. Hence, if charters improvebehavior and attendance rates, that could be indicative ofimprovement in noncognitive skills even if cognitive skills—measured by achievement—do not improve. In particular,attendance is a logical proxy for self-discipline and motiva-tion, and using student behavior as a proxy for noncognitiveskills has been done in Segal (2009a).

In order to answer these questions, I use a unique dataset from a large urban school district in the Southwest(LUSD-SW) to provide a broad look at achievement, stu-dent discipline, and attendance. Using these data, I establishwhether there is evidence that charter schools affect cognitiveor noncognitive skills. To my knowledge, no other study has

1 Most of these papers use fixed effects or some similar identificationstrategy. Hoxby and Muraka (2009), Hoxby and Rockoff (2004), andMcClure et al. (2005) instead use admission lotteries into charters as naturalexperiments.

The Review of Economics and Statistics, May 2011, 93(2): 416–435© 2011 by the President and Fellows of Harvard College and the Massachusetts Institute of Technology

ACHIEVEMENT AND BEHAVIOR IN CHARTER SCHOOLS 417

Figure 1.—Charter Growth In the United States

Sources: 1997–1998, U.S. Department of Education National Charter School Reports. 1999–2003: U.S. Department of Education Common Core of Data. 2005: National Alliance for Public Charter Schools. 2006:Center for Education Reform. 2004 data are unavailable, so a linear interpolation is provided.

considered how charters affect both of these skill sets. Thispanel is also useful in that it provides wider coverage overtime and grades than any previous study of charter schools.In total, I have test score data for grades 1 through 11 overnine years, during which charter schools were operating ineach year. Attendance and discipline data covers grades 1through 12 over thirteen years. Using test scores with widegrade coverage appears particularly important, as limiting tothe grade spans covered in Hanushek et al. (2007) and Bookeret al. (2007) provides different results. In addition, I am ableto look in detail at long-term impacts of charter schools whilestudents are enrolled in charters and after they return to reg-ular public schools. Both of these have rarely been studiedin prior work and only on samples that did not include highschool students.2 Whether charter schools generate lastingimpacts is particularly important. For the foreseeable future,the stock of charter schools in the United States will be smallrelative to noncharters. As a consequence, most students whoenter charters in elementary and middle school will return tononcharter schools before leaving the public school system.If charters provide short-term benefits but no long-term ben-efits, the usefulness of these schools for generating humancapital will be limited. Hence, through studying multipleoutcomes, using a broad base of students over a long timeframe, and analyzing long-term effects, I provide a compre-hensive and wide-ranging analysis of charter school impactson charter students.

I separate my analysis by two types of charters: start-upsand conversions. Previous work has shown these schools

2 Booker et al. (2007) find that student performance improves as timein charters increases and also when they leave charters. Bifulco and Ladd(2006) also look at how students perform in their first year in a charterand later years separately and find that first-year results were considerablyworse. My strategy also differs from these two studies as I am able toinstrument for students’ charter exit decision, which affects both the timein charter and persistence results.

differ in their impacts (Sass, 2006; Bifulco & Ladd, 2006;Zimmer and Buddin, 2003). As a result, considering themtogether could lead to aggregation bias.3 In addition, iden-tifying whether these schools provide different impacts mayhave policy implications, since states and districts could allowonly one type when starting a charter program, as is the case inIowa, Mississippi, and Nevada.4 Start-up charters are schoolswith voluntary enrollment that begin as charters. Conversionsin LUSD are schools that were previously regular publicschools that convert to charter status. They keep the samestaff, location, and attendance zones; thus, most of their stu-dents are assigned based on location of residence like anyregular school. Both types of charters benefit from exemp-tions from some regulations. Nonetheless, we would expectlittle impact from conversions since the change in the struc-ture of the school is minimal. My results show this generallyto be true.

Start-up charters, however, generate impacts on studentoutcomes. While I find no statistically significant effect over-all from attending a start-up charter on test scores, there is animprovement in math for students in middle school gradesof 0.07 to 0.18 standard deviations. Nonetheless, despitethese test score findings, students garner large and statisti-cally significant improvements in attendance and disciplinefrom attending start-up charters. On average, attendance ratesincrease by 2.4 percentage points, or 23% of the absencerate in the year prior to charter entry. Start-ups also reduceannual disciplinary infractions by 0.5 to 0.8 instances, a verylarge impact relative to the average of 1.1 infractions in theyear prior to entry. However, these results do not persist after

3 Hanushek et al. (2007) and Booker et al. (2007) do not disaggregate bythese charter types. However, in Texas, only a few charters are conversions.

4 According to the Center for Education Reform, Iowa and Mississippipermit only conversions. Nevada permits only start-ups. The other 37 statesthat permit charters and Washington, D.C., allow both.

http://www.mitpressjournals.org/action/showImage?doi=10.1162/REST_a_00077&iName=master.img-000.jpg&w=394&h=193

418 THE REVIEW OF ECONOMICS AND STATISTICS

students return to noncharter schools. In particular, atten-dance and discipline impacts disappear immediately afterstudents return to regular public schools. Therefore, while theimpact of start-up charters on cognitive skills is small, theyappear to generate a substantial improvement in noncognitiveskills; however, the drop-off after students return to regu-lar public schools shows that these skills require continualreinforcement in students.

Nonetheless, one should be cautious in this interpretationas there are other potential explanations for the behaviorresults. One possibility is that charters differ from regularpublic schools in how they enforce or report discipline. Thisis an important concern, and although it is not possible tocompletely rule out this explanation, I provide a series oftests and arguments that show that it is unlikely this is driv-ing the results. Of particular importance is that the strongattendance results, which are much harder for the charters tomanipulate than discipline, serve to reinforce the disciplineresults, and the fact that both measures show large and signif-icant improvements provides evidence of noncognitive skillimprovements.

Another possibility is that the results may reflect institu-tional differences between charters and noncharters ratherthan skill formation. The results for persistence are con-sistent with this theory. In addition, while Heckman et al.(2006) show that noncognitive skills themselves improvelabor market outcomes and Segal (2009b) establishes a linkbetween teacher-reported student behavior and later-life earn-ings, a causal impact of behavior and school attendanceon wages has yet to be established. Nevertheless, even ifthe discipline and attendance impacts do not translate intoimprovements later in life, they are potentially important out-comes in their own right since parents consider behavior tobe an important factor in the decision on whether to sendtheir children to charters. In a survey of Texas charter par-ents, Weiher and Tedin (2002) show that only one-quarter listtest scores as the primary reason for sending their children tocharters, while more than two-thirds cite moral values, dis-cipline, or safety. Thus, if the student exhibits improvementsin behavior and increased school attendance, parents’ util-ity would likely increase even if those improvements are notpermanent.

In addition to having multiple outcomes and the ability toassess long-term charter impacts, I also address some econo-metric issues. One potential econometric problem is that theassumptions underlying fixed effects are invalid if studentschoose to attend charter schools based on changes in out-comes. If this occurs, then the estimates of charter impactsmay be contaminated by mean reversion. This phenomenonhas been widely noted in the job-training literature (Ashen-felter, 1978; Heckman & Smith, 1999) while in education,mean reversion has been shown to occur in standardizedexams (Chay, McEwan, & Urquiola, 2005). Hanushek et al.(2007) use interrupted panel estimates to argue that this selec-tion does not pose a problem. While I find some graphicalevidence of this type of selection in start-up charters, my

interrupted panel estimates show little change from the mainregression estimates. Consequently, although this selectiondoes appear to exist, it does not substantially affect the impactestimates.

Another potential problem is nonrandom attrition. Thiscould create bias if the charter students leave the district ata rate that differs from that of public school students. WhileLUSD is a central city school district, it is bordered by tenschool districts. It also has many state charter schools and pri-vate schools within its boundaries. As such, there are a lot ofeducational options for parents, so the attrition bias is a con-cern. Other work has also shown that attrition bias in charterresearch is a potential problem. For example, Hanushek et al.(2007) find that charter students leave Texas public schoolsat more than 2.5 times the rate of noncharter students. Toaddress this, in addition to tests for differential attrition, Iuse a unique semiparametric attrition adjustment procedurefor fixed-effects analyses proposed by Kyriazidou (1997).This procedure has not previously been used to assess charterimpacts. These results suggest that nonrandom attrition doesnot have a substantial effect on the charter impact estimates.

II. LUSD Characteristics and Data Description

In this paper, I use a unique panel of student-level admin-istrative records from a large urban school district in theSouthwest. LUSD-SW was one of the first school districtsin the United States to institute a charter program. The pro-gram, which began in 1996 with 2 schools, expanded to halfits current size in 1997 and 1998. By 2006 there were 32 char-ter schools, 23 start-ups and 9 conversions.5 Students fromnearly 300 noncharter schools are also observed in the data.During the time period studied, approximately 50 state char-ter schools were also in operation for which I do not havedata. Figure 2 shows the evolution of the charter program inLUSD by examining the fraction of enrollment in start-upand conversion charters. In 1997 and 1998, all of the con-versions obtained charter status, after which their enrollmentshrank relative to total growth. Most of the start-up char-ters opened in 1998 and 2001, but their population steadilyincreased over time. As of the 2006–2007 school year, 4%of students in LUSD attended a charter school. Table 1 pro-vides some summary information about charter schools andstudents. Start-up students are more likely to be minority,poorer, and more at risk than noncharter students. The schoolsare smaller on average and spend more per student on instruc-tion but less on other expenses. Conversions are also more

5 One charter existed under contract with LUSD prior to the enactmentof the state’s charter laws and then promptly switched. Since enrollment isvoluntary, I define it as a start-up. Some start-ups reside on the campus of anexisting school but are considered independent schools and have voluntaryenrollment. One conversion charter maintains a large gifted and talentedmagnet program. In order to prevent the impact of this program from influ-encing the charter impact estimates, I drop any student who attends thatschool from the analysis, leaving eight conversions in the final sample.


Figure 2.—LUSD-SW Charter Enrollment by Year, Grades 1–12

heavily minority and poorer than regular public schools butare otherwise similar.6

The LUSD-SW data set includes information on scoresfrom the Stanford Achievement Test, disciplinary records,attendance rates, and a number of student characteristics. Bycombining these outcomes I assess to what extent charterschools affect both cognitive and noncognitive skill develop-ment. The data cover the 1994–95 to 2006–2007 academicyears, and I am able to follow individual students for as longas they attend school in LUSD, providing a long time serieson many students.7 Hence, I am also able to look at how theseskills develop in the long term both during and after charterexposure.

Stanford Achievement Test scores are available startingwith the 1998–99 school year, providing nine years of testscore data. The exam is norm referenced, so it reflectsachievement relative to a national sample of students. Ianalyze three exam subjects—math, reading, and language—given in grades 1 through 11. For each subject, I standardizethe scaled scores to be mean 0 and standard deviation 1 withingrade and year across LUSD. Hence, impacts are measuredin standard deviation units relative to the district average. Inorder to ensure that the impacts are analyzed on the same setsof students across all tests, after standardizing the scores, Ilimit the sample to students who have scores for all three

6 Start-up charters tend to be spread across all grade levels. There arethirteen start-ups covering at least one elementary grade, twelve coveringat least one middle school grade, and eight covering at least one high schoolgrade. Since charters often diverge from the standard grade structure, thesenumbers refer to only 23 schools. Seven of the eight conversions I studycover at least one elementary grade. Three cover at least one middle schoolgrade, but only one of these includes grades 7 or 8. There are no high schoolconversions.

7 After dropping observations for early education, prekindergarten, andkindergarten, 56% of students who are first observed in the data prior toninth grade have at least four observations. In addition, only 27% of start-up charter and 21% of conversion charter students have neither pre- norpost-charter observations.

exams.8 The final sample for test scores contains approx-imately 1.14 million student-year observations, including15,000 start-up and 20,000 conversion charter observations.In table 1, I show that for both types of charters, the test scoremeans are not statistically significantly different from thoseof regular public schools.

Discipline and attendance records of students cover 1994–1995 through 2006–2007 for all students in grades 1 to 12.The attendance rate provides the percentage of days the stu-dent attends school while enrolled. Discipline records provideinformation on the type and length of punishment for anyinfraction that results in an in-school suspension or moresevere punishment.9 I use the total number of disciplinaryinfractions per year in most of the analyses. Across LUSD,17% of student-year observations have at least one infrac-tion, and 9% have multiple infractions. For start-up charters,those figures are 8% and 3%, respectively, and for conver-sions, they are 14% and 6%. In total, the sample for disciplineand attendance contains 2.23 million observations, of which20,000 are students in start-ups and 40,000 are students inconversions. Table 1 shows that on average, start-up charters

8 Some students who are not proficient enough in English in grades 1–8took a separate Spanish language exam called Aprenda. While I have dataon these exam results, the scores are not directly comparable to those ofstudents taking the English exam so I do not include them in the analysis.Almost all students who take Aprenda are in grades 1–5 and account for 24%of all test takers in those grades. One concern is that since start-ups includefewer limited English proficient (LEP) or special education students, andhence only a handful of Aprenda takers, the Stanford Achievement Testresults may be biased. However, test score regressions limited to studentswho are not classified as LEP or special education for the duration of thetest sample show similar results to the baseline regressions. Hence, this isnot a substantial problem.

9 Unfortunately, infractions that generate less harsh punishments, such asdetention, are not in the data. In addition, the records provide only limitedinformation on the type of infraction, since 80% of infractions are unspec-ified student code violations. Nonetheless, a few severe infractions such assubstance abuse, criminal behavior, and, for 2002–2003 and later, fightingare identified in the data.



Table 1.—Summary Statistics by Charter Status

Demographics (Student-Level Observations) Outcomes (Student-Level Observations)

Variable Noncharter Start-Up Conversion Variable Noncharter Start-Up Conversion

Female 0.49 0.47 0.48 Math −0.03 −0.02 −0.11(0.50) (0.50) (0.50) (0.97) (0.93) (0.95)

[0.7] [1.1] [0.1] [1.0]White 0.10 0.03 0.01 Reading −0.03 −0.06 −0.11

(0.30) (0.16) (0.11) (0.98) (0.90) (0.92)

[4.5] [7.8] [0.5] [0.9]Black 0.33 0.31 0.50 Language −0.02 −0.04 −0.08

(0.47) (0.46) (0.50) (0.98) (0.91) (0.94)

[0.3] [1.2] [0.2] [0.6]Hispanic 0.54 0.66 0.48 Disciplinary 0.44 0.16 0.35

(0.50) (0.47) (0.50) infractions (1.37) (0.73) (1.29)

[1.5] [0.5] [4.3] [0.4]Grade level 5.85 6.53 3.74 Substance abuse 0.0074 0.0073 0.0007

(3.33) (3.50) (2.07) infractions (0.13) (0.12) (0.03)

[0.5] [3.0] [0.0] [5.6]Free lunch 0.60 0.58 0.79 Violent crime 0.0047 0.0024 0.0019

(0.49) (0.49) (0.41) infractions (0.09) (0.07) (0.06)

[0.5] [9.4] [1.4] [2.7]Reduced-price 0.07 0.11 0.08 Fighting 0.0359 0.0163 0.0498

lunch (0.25) (0.31) (0.27) infractions (0.22) (0.15) (0.27)

[6.1] [1.6] [2.1] [0.7]Other economic 0.05 0.11 0.07 Attendance rate 94.4 93.2 95.7

disadvantage (0.23) (0.32) (0.25) (8.89) (11.98) (5.97)

[2.9] [1.2] [0.6] [1.5]Limited English 0.25 0.18 0.29

proficiency (0.43) (0.39) (0.46)[1.1] [0.5]

At-risk status 0.58 0.65 0.59(0.49) (0.48) (0.49)

[0.9] [0.2]Special education 0.11 0.05 0.11

(0.32) (0.21) (0.32)[6.0] [0.1]

Gifted and 0.10 0.01 0.05talented (0.30) (0.12) (0.23)

[6.3] [3.6]Immigrant 0.13 0.13 0.09

(0.33) (0.34) (0.29)[0.2] [1.7]

School characteristics (School-level observations)Instructional 4,054 5,511 3,688

spending per (5,668) (5,341) (743)student [1.8] [1.7]

Adminstrative 519 164 455spending per (1,670) (350) (158)student [4.6] [1.0]

Other spending 936 254 918per student (1,363) (322) (389)

[8.1] [0.1]Enrollment 739 225 599

(509) (263) (277)[7.9] [1.7]

Standard deviations in parentheses. Absolute T -statistics from a regression of the variable on start-up and conversion status in brackets. Test scores are standard deviation units from scale scores normalized withingrade and year. Spending figures are per pupil. Demographics, attendance, and general infractions include 2.45 million student-year observations. Due to limited years of availability substance abuse and violenceinfractions include 1.79 million observations, while fighting includes approximately 700,000 observations. Test scores include 1.2 million observations. School characteristics include approximately 3,900 school-yearobservations.

have significantly lower overall infraction rates and fewerfighting infractions than noncharters. Conversions have fewersubstance abuse and violent crime infractions. Means forattendance are not statistically significantly different fromregular public schools for either charter type. Thus, the sum-mary statistics provide suggestive evidence of there beinglittle impact on test scores for either charter type but someimprovement in behavior for start-up charters. This is con-sistent with charters’ improving noncognitive skills but not

cognitive skills. To confirm this result, I turn to regressionanalysis.

III. Empirical Strategy

A. Baseline Model

In order to identify whether charters improve cognitiveand noncognitive skills, I use the following individual fixed-effects model:


yit = α + θCConversionit + θSStartupit + DemogitΓ

+ SwitchitΦ + GradeyearitΨ + φi + εit , (1)

where yit is some outcome measure for student i at timet, Conversionit and Startupit are indicators for the type ofcharter the student is enrolled in Demogit is a vector of time-variant observable demographic characteristics, Gradeyearit

is a set of grade-by-year indicator variables that account forchanges in outcomes over time and grade level, φi is an indi-vidual fixed effect, and εit is an independent and identicallydistributed (i.i.d.) error.10 Switchit is a set of variables thatdefine whether a student changes schools in year t due to astructural change from normal grade progression, a nonstruc-tural change for some other reason, or if it is the student’s firstyear in the district. All of these variables are interacted withindicators for the student’s grade level.11

A problem with this strategy is that prior test scores playa role in current achievement. To address this, researchersoften use a value-added (or gains) version of the fixed-effects model where the dependent variable is the annualchange in outcomes.12 This is equivalent to assuming that iflagged achievement is an explanatory variable, then its coef-ficient equals 1. Since the role of lagged achievement likelydecays, it may be preferable to have a model that explicitlyincludes lagged achievement as an explanatory variable, asin Hanushek et al. (2007) and Sass (2006):

yit = α + βyi,t−1 + θCConversionit + θSStartupit

+ DemogitΓ + SwitchitΦ + GradeyearitΨ + φi + εit .13

(2)

Since lagged scores are endogenous, these papers instru-ment with twice-lagged scores. However, recent research has

10 Demogit includes free-lunch status, reduced-price lunch status, othereconomic disadvantage, whether the student is a recent immigrant, andwhether a parent is a migrant worker. Other economic disadvantage indi-cates the student does not receive free or reduced-price lunch but doesreceive other poverty assistance. More detailed definitions are in the onlineappendix.

11 Hanushek et al. (2007), Booker et al. (2007), and Bifulco and Ladd(2006) suggest that properly controlling for student switches is importantfor separating charter impacts from the effects of switching schools. I followBifulco and Ladd and define a nonstructural switch as switching into aschool where less than 10% of a student’s previous class switches intothe same school. Conversely, a student undergoes a structural switch whenmore than 10% of his or her previous class switches into the same school.Twelve percent of student-years undergo nonstructural switches, 11% ofstudent-years undergo structural switches, and 12% are students movingfrom outside the district.

12 Alternatively once could use a random trend model where both sidesof the estimating equation are differenced and then demeaned. While thisallows for individual time trends, it substantially reduces precision andcould exacerbate bias if the trends are nonlinear. Nonetheless, estimatesfrom this model are similar to the baseline levels models.

13 Sass (2006) uses an Arellano-Bond model. I conducted regressionsusing this procedure and found results that are similar to my baseline lev-els model. Hanushek et al. (2007) first-difference their data to remove thefixed effects rather than demean and then instrument the change in laggedachievement with the twice-lagged level.

suggested that factors in children’s distant youth play impor-tant roles in later achievement (Todd & Wolpin, 2007), sug-gesting that twice-lagged scores are unlikely to be exogenous,and thus the estimates in these papers may be biased.

As an alternative, I use both levels and value-added models.In expectation, these two models bound the true estimate,which would be identified by equation (2). I provide the proofof this in the appendix.14 Therefore, I am able to identifycharter impacts within a range of values while avoiding biasesthat are introduced by endogenous instruments.

B. Potential Biases from Selection

While attending any school is a choice, parents are notrestricted by having to live in a specific attendance zoneor meet some transfer qualification for most charters. As aresult, selection into charters may be a more substantial prob-lem than selection into regular public schools. The studentfixed-effects analysis used in this paper corrects for selec-tion into charters based on unobserved characteristics thatdo not change over time, such as innate ability. Nonethe-less, one may be concerned of selection due to time-varyingfactors. While I cannot eliminate all types of selection thatcould play a role, I am able to check for a few specific typesthat would be particularly important in the charter context.15

My main findings, that there is little improvement in cogni-tive skills but substantial improvement in noncognitive skillsfrom attending a start-up charter, are robust to these potentialbiases.

The first issue I consider is whether entry into charterschools is based on pre-entry trends in the dependent variable(Booker et al., 2007; Hanushek et al., 2007; Bifulco & Ladd,2006; Sass, 2006). It is possible that students enter charterschools due to changes in test scores or behavior or a changein some strong correlate with these outcomes. Such a situa-tion has been widely noted in the job-training literature andis commonly called “Ashenfelter’s dip” (Ashenfelter, 1978;Heckman & Smith, 1999). Since a parent may see a drop inperformance as an indicator that the current school does notmeet his or her child’s needs, it is reasonable to believe thatstudents change schooling environments in response to poor

14 This framework is similar to that proposed by Guryan (2001), whichuses difference-in-differences and lagged-dependent variable models tobound a true estimate where it is not clear whether selection is based onfixed characteristics or previous outcomes.

15 Another solution is to use oversubscription lotteries as natural experi-ments (Hoxby & Murarka, 2009; McClure et al., 2005; Hoxby & Rockoff,2004). While a lottery-based strategy has substantial advantages over fixedeffects, there are three potentially undesirable aspects. First, oversubscribedschools are likely of higher quality than schools with spaces available;hence, a comparison of lottery winners and losers will identify the impactsfor only the best charter schools. Second, lottery studies may be subject tomore attrition bias than panel studies, since parents who lose lotteries maybe more likely to send their children to private schools or other districts thanthose who win. Third, often lotteries are limited to a very small number ofschools—four in the case of Hoxby and Rockoff (2004) and one in the caseof McClure et al. (2006). Nonetheless, Hoxby and Muraka (2009) are ableto address all three of these concerns well, as their sample of New YorkCity charter schools has forty schools with lotteries and they appear to havelittle attrition bias.


Figure 3.—Transitions between School Types, 1998–2005, Grades 1–11

performance. This violates the strict exogeneity assumptionfor fixed effects, which requires that future and past outcomesare not affected by current charter status. While much of thisproblem is dealt with by appropriately controlling for stu-dents’ switching status, I provide evidence that some smalldips remain for start-up charters. Hence, I follow Hanusheket al. (2007) in the use of interrupted panel estimates to checkfor bias from endogenous charter entry. In addition, regres-sions that look at how charter impacts vary with time spent ina charter are used to check for mean reversion, which wouldbe a sign that the estimates suffer from this bias.

Another problem is that some parents may choose to leaveLUSD altogether if their student performs poorly in the char-ter school. If this occurs at a different rate from that in regularpublic schools conditioning on observables and student fixedeffects, then there are time periods when these students shouldbe observed but are not. This could lead to attrition bias. ForLUSD, this is particularly important because of the manyother options available to students, including private schools,state charters, and more than ten suburban school districts.Figure 3 provides some suggestive evidence that endoge-nous attrition is a potential problem. While about 16% ofnoncharter students exit LUSD each year, 26% of start-upcharter students attrit. Though not shown in the figure, thedifferences are more dramatic over longer time periods. Forexample, 39% of noncharter third graders between 1998 and2000 are no longer in LUSD five years later, while that num-ber is 43% for conversion students and 63% for start-upstudents. These statistics may simply reflect different charac-teristics of the schools, such as different grade levels coveredor the types of students who attend. Indeed, regressions ofattrition propensity on charter status, including all of thecovariates in equation (1), show no statistically significantrelationship between charter status and attrition propensity

at all grade levels.16 However, a model that interacts char-ter status with test scores shows that start-up students withhigher reading scores and conversion students with highermath scores are less likely to attrit than comparable non-charter students. These results can be found in the onlineappendix.

While differential attrition appears to be only a minor issue,the evidence is not definitive. As such, I use a nonparamet-ric procedure for correcting sample selection in individualfixed-effects models proposed by Kyriazidou (1997) to fur-ther check for attrition bias. Her insight is that since fixedeffects correct for attrition based on time-invariant factors,one can correct for endogenous attrition due to time-variantfactors by weighting toward observations where there is nochange in attrition propensity.

To produce Kyriazidou’s (1997) estimator, one must firstdefine the selection equation,

sit = WitΩ + ζi + μit , (3)

where sit equals 1 if the student is no longer in the sampleand 0 otherwise, Wit is a set of variables that are observedwhether or not the individual has attrited, ζi is an individ-ual specific fixed effect, and μit is random i.i.d. error. Wit

need not contain all (or any) of the variables in the outcomeequation, but it does need to contain an exclusion restriction.In this paper, I use a model that includes the student’s lastobserved start-up or conversion charter status, free lunch sta-tus interacted with grade level, reduced-price lunch interactedwith grade level, other economic disadvantage interacted withgrade level, recent immigration status, whether a parent is a

16 In the model with a pooled estimate across all grade levels, the coef-ficient on attrition from start-ups is 0.037 (s.e. 0.025) and conversions is0.003 (s.e. 0.010).



migrant worker, grade-by-year indicators, and whether thestudent is ineligible to attend his or her previous school dueto his or her predicted grade not being offered, which servesas the exclusion restriction.17 The assumption underlying thisrestriction is that being in a grade beyond the student’s lastobserved school’s highest grade is correlated with attritionbut not student outcomes.18 While one may be concernedthat these students are invariably older and thus this may cor-relate with outcomes, particularly discipline and attendance,the inclusion of grade-by-year fixed effects mitigates this. Themodel is estimated using a conditional fixed-effects logit. Theoutcome equation in my model is the baseline charter impactequation.

After removing the fixed effect in the outcome equationthrough first-differencing, Kyriazidou (1997) argues that inobservations where (Wit−Wis)Ω = 0 for s < t, the individualhas not had a change in circumstances that affects attrition.Since a student’s innate tendency to switch schools is cap-tured by fixed effects, I can generate consistent estimates ofθ, the charter effect, by using only those observations wherethis holds true.

Since limiting to observations where (Wit − Wis)Ω = 0would reduce power substantially, Kyriazidou (1997) pro-poses using kernel weights to focus the analysis on observa-tions that are close to (Wit − Wis)Ω = 0. Therefore, in thesecond stage, I run a first-differenced version of equation (1)weighted by

ψ̂it,n = 1

hnK

((Wi,t − Wi,t−1)Ω̂

hn

), (4)

where K is a kernel function with bandwidth hn and (Wit −Wi,t−1)Ω̂ is the first-differenced linear prediction from theselection model estimation.19

A third problem that could arise is bias from students leav-ing the charters and returning to the regular public schools.Both the possibility that students leave because they performpoorly in charters and that charters have long-term impactson outcomes can contribute to the bias. In the first case,if students leave charters prematurely due to poor perfor-mance, then they will reduce the number of charter periodobservations and the influence of bad charters. In the sec-ond case, charters that have long-term impacts that persistwhen students return to regular public schools will also affectperiods when students are not in charters and bias impacts

17 For attrited observations, I use the grade the student would have beenin assuming normal grade progression.

18 The idea behind this exclusion restriction is that a student would bemore likely to leave the district if she has to switch schools anyway; that is,the relative costs of leaving the district fall if students are forced to switchschools.

19 The appropriate bandwidth is found using the mean squared error (MSE)minimization procedure described in Kyriazidou (1997). Since the MSEminimizing bandwidth is sensitive to the initial value, I follow Dustmannand Rochina-Barrachina (2000) in using the initial value that is as close aspossible to the constant for the MSE minimizing bandwidth as asymptotictheory says the two should converge. I use the gaussian kernel to generatethe weights.

toward 0.20 To solve both problems, I conduct analyses thatinclude indicators for whether a student is in a “post-charter”period, which allows us to compare charter impacts directly topre-charter periods while identifying whether charter impactspersist after students return to regular public schools:21

yit = α + θS,0Startupit +4∑

k=1

θS,kPostStartupk

+ θC,0Conversionit +4∑

k=1

θC,kPostConversionk

+ DemogitΓ + SwitchitΦ + GradeyearitΨ + φi + εit ,(5)

where PostStartupk and PostConversionk denote the student’sbeing in year k after leaving the charter. For k = 4, I includeany time period after the third year. To account for students’endogenously exiting charters, I instrument for being in apost-charter period with whether the student has the listednumber of predicted grades past his or her previous charter’shighest grade covered. Predicted grades are calculated usingnormal grade progression starting from the year of charterentry.22 This avoids potential complications from the charter’sretention policies. For example, the estimator for being oneyear after start-up is instrumented using whether the studentis one predicted grade level higher than the highest gradecovered in the charter school he or she attended. Hence, I amable to isolate the effects from students who are forced outof charters due to being grade ineligible instead of those wholeave voluntarily, possibly because they do not perform wellin the charter.

IV. Results

A. Charter School Impacts on Student Outcomes

I now turn to my main results. First, I consider test scores tosee how charter schools affect cognitive skill development.In figure 4, I provide graphs that trace out the residuals ofstudent outcomes from a fixed-effects regression, includingall of the covariates in model (1) except charter status. Thisallows me to look at how outcomes change as students entercharters net of fixed student characteristics, economic status,immigration status, grade level, and time. The top row showsstart-up charters, which are the type of charter in which onewould most expect to see charter impacts. While math scoresseem to increase after entry, reading and language show farless improvement. The graphs also show some evidence of

20 This is a violation of the strict exogeneity assumption for fixed effects,which implies that outcomes in future and past periods cannot be correlatedwith current charter status conditioning on observables and the fixed effect.

21 It is possible to be in a conversion charter and also in a post-start-upperiod at the same time (and conversely for start-up and postconversion);however, only 2% of charter students ever attend both types.

22 While I do not include kindergarten in the analysis, as there are notest scores for this grade and enrollment is optional in LUSD, I am able toidentify whether a student enters a charter in kindergarten.


Figure 4.—Test Scores before and after Charter Entry

selection into the charter off precharter trends since test scoresdrop in the year immediately prior to entry (−1), which couldpotentially bias the estimates. For conversions, test scoresappear to fall off slightly after entry. The graphs suggest thatwith the possible exception of math in start-ups, cognitiveskill improvement is small at best.

Table 2 provides the main test score results for this paper,along with the interrupted panel and Kyriazidou (1997) esti-mates. The standard errors for each regression are robust toheteroskedasticity and clustered by school. Although I focuson models that consider both conversion and start-up char-ters, it is useful to start by looking at a model where start-upsand conversions are constrained to have the same coefficient.In this case, there is a statistically significant effect on mathin levels of 0.07 standard deviations, but no statistically sig-nificant impacts for any other estimate. When I disaggregatethe estimate, the results for start-ups confirm the graphicalanalysis as there is no statistically significant impact of start-up charters on test scores in levels or value-added models.23

With the exception of math in levels, all of the point estimatesare relatively close to 0. Consequently, these results suggestlittle improvement in cognitive skills in start-up charters onaverage. For conversions while math in levels is statisticallysignificant at 0.07 standard deviations, the value-added esti-mate is insignificant and thus I cannot say that conversions

23 Levels models restricted to the value-added sample provide estimatesthat are similar for all outcomes and are available in the online appendix.

have a math impact.24 In addition, this impact will disappearwhen I consider the role of persistence.

A concern with using fixed effects to study charter impactsis that since many students enter charters in kindergarten,by excluding earlier years the estimates are identified offstudents who may disproportionately benefit from charters.I test the extent of this problem by conducting regressionsrestricted to students in grades 3 to 11, 3 to 8, and 4 to 8.25

The results for these are provided in the online appendix. Ifthe fixed-effects estimates in prior research are biased fromexcluding early charter students, then adding first and sec-ond graders should change the estimates.26 When restrictinggrade spans to grades 3 to 11, there is little change from thebaseline models in the test score impact estimates for start-ups or conversions. Hence, it appears that excluding grades1 and 2 from the analysis does not bias the results. However,when I restrict to grades 3 to 8 or 4 to 8, the estimates forstart-ups are statistically significant and positive in both lev-els and value-added models for math and language and in

24 At-risk, LEP, gifted and talented, and special education status could beuseful covariates, but they are also potentially influenced by school qual-ity, or charters may not classify students the same way as noncharters. Assuch, these variables would also be inappropriate as outcomes. Nonethelessadding LEP, at-risk status, gifted and talented status, and special educationstatus to the baseline model has very little effect on the estimates.

25 Hanushek et al. (2007) and Booker et al. (2007) use, respectively, thegrade spans 3–8 and 4–8.

26 One complication with this strategy is that, as noted by Hoxby andMurarka (2009), the benefit of adding these grades to analyses may be smallsince test scores in these grades are less accurate measures of achievementthan tests in later grades.



Table 2.—Effect of Attending a Charter School on Test Scores.

A: Levels B: Value-AddedMath Reading Language Number of Math Reading Language Number of(1) (2) (3) Observations (4) (5) (6) Observations

OLS with student Baseline Any charter 0.071∗∗ 0.023 0.033 1,141,480 0.026 −0.025 0.004 779,343fixed effects (0.031) (0.023) (0.020) (0.022) (0.016) (0.016)

OLS with student Baseline Start-up 0.072 0.015 0.022 1,141,480 0.026 −0.033 −0.007 779,343fixed effects (0.052) (0.038) (0.033) (0.025) (0.024) (0.023)

Conversion 0.070∗∗ 0.029 0.041∗ 0.026 −0.019 0.014(0.033) (0.026) (0.024) (0.034) (0.021) (0.021)

Interrupted panel Drop year prior Start-up 0.060 0.010 0.013 1,135,119 0.062 0.002 0.015 772,568to charter (0.055) (0.041) (0.036) (0.051) (0.038) (0.031)

Conversion 0.081∗∗ 0.040 0.048∗ 0.070∗ 0.030 0.042(0.039) (0.032) (0.028) (0.041) (0.035) (0.027)

Drop 2 years prior Start-up 0.056 0.009 0.006 1,131,123 0.041 −0.008 0.001 767,916to charter (0.054) (0.041) (0.036) (0.051) (0.038) (0.032)

Conversion 0.084∗∗ 0.042 0.043 0.070∗ 0.029 0.038(0.041) (0.033) (0.029) (0.042) (0.036) (0.029)

Kyriazidou (1997) First differences Start-up 0.049 0.003 0.011 618,050 0.059 −0.009 0.006 526,203attrition model (unweighted) (0.040) (0.028) (0.022) (0.036) (0.033) (0.024)

Conversion 0.073∗∗∗ 0.029 0.054∗∗∗ 0.063 0.001 0.028(0.022) (0.022) (0.017) (0.043) (0.028) (0.022)

MSE minimizing Start-up 0.053 0.004 0.012 618,050 0.075∗∗ −0.005 0.007 526,203bandwidth (0.039) (0.028) (0.022) (0.036) (0.034) (0.026)

Conversion 0.071∗∗∗ 0.028 0.054∗∗∗ 0.051 −0.008 0.031(0.022) (0.022) (0.017) (0.041) (0.028) (0.021)

1/4 × MSE minimizing Start-up 0.065 0.005 0.018 618,050 0.100∗∗ 0.014 0.043 526,203bandwidth (0.042) (0.026) (0.024) (0.046) (0.032) (0.037)

Conversion 0.044∗ 0.018 0.063∗∗∗ 0.015 −0.037 0.066∗∗(0.023) (0.023) (0.021) (0.044) (0.037) (0.031)

All regressions include an individual fixed effect, free lunch status, reduced-price lunch status, other economic disadvantage, recent immigration status, whether a parent is a migrant worker, and grade-by-yearindicators. Standard errors clustered by school in parentheses. ∗ , ∗∗ , ∗∗∗ denote significance at the 10%, 5%, and 1% levels, respectively. Levels models cover 1998–2006 and grades 1–11. Value-added models cover1999–2006 and grades 2–11. Because students are defined to be in the sample in the first grade and year of each sample, these observations are dropped from the Kyriazidou attrition analysis.

levels for reading. This is indicative of heterogeneous effectsacross grade levels, an issue I explore further below.

The next two rows show the interrupted panel estimates.27

These estimates account for the preentry dips seen in figure4. These estimates are close to the baseline results regardlessof whether one or two years prior are dropped. A possi-ble exception is math for conversions in the value-addedmodels where the impact increases to 0.07 standard devia-tions but is significant only at the 10% level. The last threerows provide estimates using the Kyriazidou (1997) attritioncorrection procedure. Since this relies on a first-differencedestimator rather than a fixed-effects estimator, I use theseto examine how much estimates change when the attritioncorrections are applied rather than as true impact estimates.The unweighted model is the first-differences corollary tothe baseline model in the first row. If attrition is a problem,then one would expect there to be large changes in the esti-mates as the weights are added. I show both models usingmean-squared error minimizing weights and weights wherethe bandwidths are one-fourth of the MSE minimizing band-widths. While start-up math impacts increase and become

27 In value-added models, Hanushek et al. (2007) keep the gain in the firstcharter year as the difference between year t and t − 1. While this reducesthe bias from the precharter gain measures, it does not reduce bias fromthe excessive gain in the first charter year. I modify the procedure suchthat the dependent variable in the value-added models is the average gainover the dropped years. That is, when year t − 1 is dropped, I use thedifference in test scores in t and t − 2 divided by 2. Both strategies providesimilar results for all outcomes.

statistically significant in value-added models, levels modelsare still statistically insignificant and change little. There isalso some increase for language in conversions. Nonetheless,these results at worst suggest that I may be slightly under-estimating math scores for start-ups and language scoresfor conversions, but in general the baseline results hold upwell.

As such, I am unable to detect a statistically significanteffect of start-up charter schools on measures of cogni-tive skills. However, it is still unclear how charters affectnoncognitive skills. Since many start-up charters in LUSDtarget students with behavioral problems and those whoare classified as at risk of dropping out of school, theseschools may focus more on noncognitive than cognitiveskills. While some cognitive skill improvement spills into dis-cipline and attendance, the lack of test score impacts impliesthat improvements in these outcomes would likely be due tononcognitive skill enhancement. Figure 5 provides the samegraphs for discipline and attendance as figure 4 provides fortest scores. While there is little change in either of these mea-sures after charter entry for conversions, the figures showdrops in disciplinary infractions and increases in attendancerates immediately after entry. In both cases, there is somereversion after the first year, but both outcomes remain atlevels that improve on preentry performance. As in the casewith test scores, however, there is evidence of selection intocharters off worsening attendance, so once again I need touse interrupted panel estimates to address this potential bias.


Figure 5.—Discipline and Attendance, Before and After Charter Entry

Panel A of table 3 shows the baseline, interrupted panel,and attrition-adjusted results for discipline and attendance.I begin by providing results for an overall charter effect.The results show statistically significant improvements indiscipline of 0.2 to 0.4 infractions for charter attendees,while attendance improves by 1 percentage point. However,the combined estimate hides substantial differences betweenstart-up and conversion charters. While there are no sta-tistically significant estimates in any model for conversioncharters, start-up charters have large and statistically signif-icant improvements in discipline and attendance. Using thelevels and value-added model estimates as bounds, baselineestimates in the first row show annual disciplinary infrac-tions falling by between 0.5 and 0.8 incidences when studentsattend start-up charters. These estimates are statistically sig-nificant at the 1% level. Average infractions in the year prior tocharter entry are 1.1, so the start-up charter impact is between45% and 73% of the precharter mean. Start-ups also have sta-tistically significant impacts at the 1% level on attendance.Baseline estimates show an increase of 2.3 percentage points.In the year prior to entry, attendance rates average 91.0%, sothe attendance impact accounts for 26% of the precharterabsence rate.28 In addition, despite the graphical evidence forthe Ashenfelter dips, discipline and attendance results hold in

28 Regressions limiting the discipline and attendance analysis to studentswho took all three Stanford exams are similar for discipline. Attendanceimpacts are lower but remain statistically significant at the 1% level. Resultsare provided in the online appendix.

the interrupted panel and attrition-adjusted estimates. There-fore, it appears that start-up charters improve noncognitiveskill formation.

Since LUSD audits attendance by checking teachers’ logswith reported attendance, any systematic misreporting wouldrequire participation of administrators and teachers, whichwould be very difficult. Therefore, we can be confident thatthe attendance results reflect actual behavioral improvements.Nonetheless, it is possible that the discipline results are dueto differences in enforcement or reporting rather than actualbehavior improvements. To address this concern, I provide afew pieces of evidence that at least part of the impact reflectsbehavioral improvements. First, the large impact from atten-dance reinforces the discipline results since they are highlycorrelated and both reflect noncognitive skill improvement.29

Second, regressions that use severe infractions—substanceabuse, violent criminal activity, and fighting—as outcomesshow statistically significant drops from attending a start-upcharter. In levels models, substance abuse infractions fall by0.015 (s.e. 0.006), violent crimes by 0.008 (s.e. 0.002), and

29 An OLS regression of attendance rate on the number of infractions,free-lunch, reduced-price lunch, other economic disadvantage, recent immi-grant, parents’ migrant status, gender, race, and grade-year interactionsprovides a point estimate on infractions of −1.12 and a standard error (clus-tered by school) of 0.05. It is possible that higher attendance could generatemore disciplinary problems in charter schools, as the marginal attendeeswould tend to misbehave. Nonetheless, this would cause me to underes-timate the impact of charters on discipline. Since I find large impacts onboth attendance and discipline, it remains most likely that both result froma common underlying skill enhancement.



Table 3.—Effect of Attending a Charter School on Discipline, Attendance, and Retention

Disciplinary Levels Disciplinary Value-AddedInfractions Attendance Number of Infractions Attendance Number of

(1) (2) Observations (3) (4) Observations

A: Disciplinary Infractions and AttendanceOLS with student Baseline Any charter −0.356∗∗∗ 1.029∗∗ 2,233,050 −0.233∗∗ 1.132∗∗ 1,777,994

fixed effects (0.127) (0.470) (0.109) (0.544)

OLS with student Baseline Start-up −0.795∗∗∗ 2.268∗∗∗ 2,233,050 −0.498∗∗∗ 2.338∗∗∗ 1,777,994fixed effects (0.113) (0.607) (0.151) (0.757)

Conversion 0.023 −0.044 0.005 0.047(0.070) (0.186) (0.058) (0.167)

Interrupted panel Drop year prior to Start-up −0.789∗∗∗ 2.149∗∗∗ 2,221,517 −0.789∗∗∗ 1.852∗∗∗ 1,758,480charter (0.101) (0.545) (0.085) (0.364)

Conversion 0.028 −0.045 0.024 0.199(0.074) (0.182) (0.088) (0.174)

Drop 2 years prior Start-up −0.775∗∗∗ 2.058∗∗∗ 2,211,570 −0.797∗∗∗ 1.873∗∗∗ 1,756,982to charter (0.088) (0.491) (0.084) (0.367)

Conversion 0.026 0.025 0.024 0.181(0.078) (0.177) (0.089) (0.178)

Kyriazidou (1997) First differences Start-up −0.809∗∗∗ 2.467∗∗ 1,457,716 −0.849∗∗∗ 3.071∗∗∗ 1,067,566attrition model (unweighted) (0.171) (1.027) (0.208) (1.129)

Conversion 0.029 0.017 0.034 0.032(0.073) (0.194) (0.094) (0.171)

MSE minimizing Start-up −0.818∗∗∗ 2.500∗∗ 1,457,716 −0.842∗∗∗ 3.242∗∗∗ 1,067,566bandwidth (0.174) (1.015) (0.216) (1.203)

Conversion 0.034 0.017 0.062 0.061(0.075) (0.194) (0.102) (0.172)

1/4 × MSE Start-up −0.879∗∗∗ 3.125∗∗ 1,457,716 −0.865∗∗∗ 4.721∗∗∗ 1,067,566minimizing (0.190) (1.218) (0.302) (1.646)

bandwidth Conversion 0.109 0.013 0.271∗ 0.143(0.095) (0.218) (0.161) (0.212)

B: Additional Outcomes (Levels Only)Alternative

Any EducationInfraction Expulsion Placement Retention

Start-up −0.250∗∗∗ −0.0036∗∗∗ −0.020∗∗∗ 0.0024∗(0.036) (0.0013) (0.004) (0.0014)

Convert 0.002 −0.0016∗∗ −0.005∗∗ −0.0001(0.015) (0.0007) (0.002) (0.0001)

Observations 2,233,050 1,740,282 1,740,282 1,777,994

All regressions include an individual fixed effect, free lunch status, reduced-price lunch status, other economic disadvantage, recent immigration status, whether a parent is a migrant worker, and grade-by-yearindicators. Standard errors clustered by school in parentheses. ∗ , ∗∗ , ∗∗∗ denote significance at the 10%, 5%, and 1% levels, respectively. Levels models cover 1994–2006 and grades 1–12 and value-added models1994–2006 and grades 2–12 for discipline and attendance. Because students are defined to be in the sample in the first grade and year of each sample, these observations are dropped from the Kyriazidou attritionanalysis. Additional outcomes cover all grades 1–12 but retention, expulsion, and alternative education program placement are not available in all years.

fighting by 0.024 (s.e. 0.009).30 It is unlikely that principalspunish students for these infractions with punishments thatare less severe than in-school suspensions; hence, improve-ments in these outcomes are most likely due to behavioralimprovements.31

Third, at four to seven times the standard error, the enforce-ment or reporting bias would need to be very large to makethe discipline estimates statistically insignificant. Fourth, amultinomial logit regression of type of punishment on char-ter status shows that start-up charter students were more likelyto receive out-of-school suspensions than in-school suspen-sions, suggesting that if anything, punishments in start-upcharters are harsher than in noncharters. It is possible that

30 All standard errors are clustered by school. Value-added models werenot statistically significant. However, since these are low-frequency events,it is very rare to get more than one infraction in subsequent years; thus,it would be difficult for value-added models to identify impacts on theseoutcomes.

31 In addition, for substance abuse and criminal infractions, the principalis legally obligated to notify the police department.

this result reflects a drop in minor infractions without thecommensurate drop in major infractions. However, the dropin severe infractions suggests this is unlikely to be the case.32

Finally, in panel B of table 3, the first three columns pro-vide results for some alternative measures of discipline. Sincethese are all binary outcomes, I report only levels models.All regressions include the same covariates as in the baselineregressions. I consider the effect of attending a charter onwhether the student has any disciplinary infractions duringthe year, whether the student is expelled, and whether thestudent is placed in an alternative school for students withdisciplinary problems. Start-up charters provide statisticallysignificant drops in likelihood of all three of these outcomes.

32 The unit of observation is at the student infraction level, and the left-out category is in-school suspension. The regression includes covariatesfor gender, race, free lunch, reduced-price lunch, other economic disadvan-tage, immigration status, whether student is a recent immigrant, whetherthe parent is a migrant worker, infraction type—substance abuse, violentcrime, nonviolent crime, truancy, other—and grade-by-year dummies. Theestimate of attending a start-up charter on out-of-school suspensions isstatistically significant at the 5% level. These results are available on request.


Table 4.—Charter Impacts by Grade Level and Age of Charter

Math Reading Language Discipline Attendance(1) (2) (3) (4) (5)

A: Impacts by Grade LevelLevelsStartup × Grades 1–5 −0.010 −0.005 −0.011 −0.182∗∗∗ −0.029

(0.042) (0.033) (0.035) (0.035) (0.103)

Startup × Grades 6–8 0.182∗∗∗ 0.064∗∗ 0.076∗∗∗ −0.933∗∗∗ 1.607∗∗∗(0.064) (0.028) (0.022) (0.122) (0.262)

Startup × Grades 9–12 −0.021 −0.040 −0.029 −0.916∗∗∗ 3.562∗∗∗(0.095) (0.096) (0.077) (0.156) (0.623)

Observations 1,141,480 1,141,480 1,141,480 2,233,050 2,233,050Value addedStartup × Grades 1–5 −0.011 −0.058 −0.045 −0.333∗∗∗ 0.231

(0.056) (0.036) (0.038) (0.102) (0.224)

Startup × Grades 6–8 0.071∗∗∗ 0.005 0.010 −0.677∗∗∗ 1.229∗∗∗(0.023) (0.023) (0.021) (0.238) (0.302)

Startup × Grades 9–12 −0.024 −0.076 −0.014 −0.401∗∗ 3.734∗∗∗(0.049) (0.053) (0.034) (0.198) (0.868)

Observations 779,343 779,343 779,343 1,777,994 1,777,994B: Impacts by Age of Charter (Levels)

Startup × 1 Year 0.046 0.040 0.031 −0.653∗∗∗ 5.898∗∗∗(0.045) (0.044) (0.027) (0.098) (2.150)

Startup × 2 Years 0.073 0.013 0.057 −0.621∗∗∗ 1.820(0.059) (0.060) (0.057) (0.123) (1.829)

Startup × 3 Years 0.024 −0.078∗ −0.023 −0.691∗∗∗ 1.646∗∗∗(0.065) (0.045) (0.030) (0.122) (0.438)

Startup × 4 Years 0.185∗∗ 0.045 0.061∗ −0.858∗∗∗ 3.320∗∗∗(0.081) (0.037) (0.034) (0.150) (1.020)

Startup × 5+ Years 0.056 0.031 0.012 −0.921∗∗∗ 1.148(0.060) (0.050) (0.044) (0.127) (0.735)

Convert × 1 Year 0.093 0.038 0.017 0.033 −0.025(0.071) (0.034) (0.043) (0.062) (0.209)

Convert × 2 Years 0.086 0.055 0.034 −0.097∗∗ 0.114(0.055) (0.041) (0.040) (0.042) (0.171)

Convert × 3 Years 0.149∗∗ 0.077∗∗ 0.117∗∗ 0.070 −0.211(0.059) (0.038) (0.057) (0.113) (0.274)

Convert × 4 Years 0.123 0.086 0.082 −0.015 −0.063(0.099) (0.094) (0.056) (0.061) (0.246)

Convert × 5+ Years −0.033 −0.051 −0.005 0.093 −0.170(0.054) (0.049) (0.036) (0.119) (0.238)

Observations 1,141,480 1,141,480 1,141,480 2,232,727 2,232,727

Each column in each panel is a separate regression. All regressions include an individual fixed effect, free lunch status, reduced-price lunch status, other economic disadvantage, recent immigration status, whether aparent is a migrant worker, and grade-by-year indicators. Grade levels refer to the grade of the student. Standard errors clustered by school in parentheses. ∗ , ∗∗ , ∗∗∗ denote significance at the 10%, 5%, and 1% levels,respectively. Levels models cover 1998–2006 and grades 1–11 for test scores and 1994–2006 and grades 1–12 for other outcomes. Value-added models cover 1999–2006 and grades 2–11 for test scores and 1994–2006and grades 2–12 for other outcomes. Columns in each panel show separate regressions. Grade-level regressions also contain an indicator for conversion charters, but since only one conversion covers grades 7–8 and noconversions cover grades 9–12, I do not separate that estimate by grade level. Value-added results for age of charter are similar to levels results and are available in the online appendix. Observation counts by schooltype are also available in the online appendix.

Conversion charters also show drops in expulsion rates andalternative education placement, but these are substantiallysmaller than for start-ups. Thus, the discipline results for start-ups are robust across multiple margins. The last column ofpanel B looks at retention. The results show a slight increasein retentions for start-ups, but this is significant only at the10% level. Whether students are retained more in charters ispotentially interesting but provides an unclear interpretation.Higher retention could indicate that students perform worse,but it could also indicate a policy difference where chartersare more likely to hold marginal students back or that char-ters are better at identifying students who need to be heldback.

So far I have established that on average, start-up chartersprovide little improvement in cognitive skills while gener-ating large improvements in noncognitive skills. Since thisis an average result, these impacts may vary by school andstudent characteristics. Table 4 provides some results on how

charter impacts differ by grade level and age of the charter.The table shows total effects for each type of school. Modelswith main effects and interactions are available in the onlineappendix. Panel A looks at variation by grade level. Sincethe there is only one conversion charter in the sample withgrades 7 and 8 and none with grades 9 through 12, a pooledestimate for conversion charters is included in the regressionbut not shown.33 In columns 1 through 3, I show that whileelementary and high school start-ups have no statisticallysignificant impact on any test score, start-ups with grades6 through 8 (middle school grades) fare quite well. All threeof the estimates in levels models show that students in thesegrades perform statistically significantly better than nonchar-ter students, and models with main effects and interactions

33 Full results, along with observation counts by school type, are providedin the online appendix.


Table 5.—Start-up Charter Impacts by Student Characteristics


A: LevelsStart-up 0.286 0.086 −0.024 −0.948∗∗∗ 2.601∗∗∗

(0.185) (0.086) (0.074) (0.120) (0.956)

Start-up × Black −0.238 −0.087 0.034 0.228∗ −1.094(0.166) (0.076) (0.058) (0.119) (0.740)

Start-up × Hispanic −0.305∗∗ −0.133∗∗∗ −0.006 −0.098 0.434(0.154) (0.045) (0.027) (0.093) (0.467)

Start-up × Female 0.056∗∗ 0.005 0.013 0.238∗∗∗ 0.055(0.027) (0.022) (0.019) (0.074) (0.244)

Start-up × Economically Disadvantaged −0.028 −0.024∗ −0.020 −0.039 −0.090(0.025) (0.013) (0.017) (0.043) (0.268)

Start-up × Immigrant 0.034 0.030 0.018 0.078∗∗ −1.358∗∗(0.030) (0.019) (0.025) (0.033) (0.565)

Observations 1,141,480 1,141,480 1,141,480 2,233,050 2,233,050B: Value added

Start-up 0.027 −0.102∗ −0.158∗∗∗ −0.745∗∗∗ 3.381∗∗(0.061) (0.058) (0.048) (0.219) (1.404)

Start-up × Black 0.054 0.093 0.196∗∗∗ 0.075 −1.908(0.061) (0.060) (0.044) (0.161) (1.506)

Start-up × Hispanic −0.024 0.011 0.141∗∗∗ −0.052 −0.060(0.038) (0.056) (0.044) (0.072) (1.150)

Start-up × Female 0.034∗ −0.034∗ −0.042∗ 0.381∗∗ 0.329(0.019) (0.019) (0.024) (0.151) (0.429)

Start-up × Economically Disadvantaged −0.041 0.014 −0.003 −0.071 0.021(0.029) (0.040) (0.022) (0.093) (0.540)

Start-up × Immigrant 0.053∗ −0.034 −0.033 0.058 −0.409∗∗(0.027) (0.027) (0.026) (0.042) (0.163)

Observations 779,343 779,343 779,343 1,777,994 1,777,994

Each column in each panel is a separate regression. All regressions include an individual fixed effect, free lunch status, reduced-price lunch status, other economic disadvantage, recent immigration status, whethera parent is a migrant worker, and grade-by-year indicators. Regressions also include the main effect and the same set of interactions for conversion charters, along with start-up and conversion interactions with othernonwhite students. These students make up less than 1% of start-up and 4% of conversion charter students. Full results are available in the online appendix. Standard errors clustered by school in parentheses. ∗ , ∗∗ ,∗∗∗ denote significance at the 10%, 5%, and 1% levels, respectively. Levels models cover 1998–2006 and grades 1–11 for test scores and 1994–2006 and grades 1–12 for other outcomes. Value-added models cover1999–2006 and grade 2–11 for test scores and 1994–2006 and grades 2–12 for other outcomes.

show they perform better than both their elementary and high-school counterparts. However, for value-added models, onlymath has a statistically significant effect. These results implythat middle school start-ups increase math scores by 0.7 to0.18 standard deviations. Reading and language increase byat most 0.07 and 0.08 standard deviations, but I cannot rule outa zero effect. Hence, while on average there is little evidenceof cognitive skill improvements from start-up charters, thereis some weak evidence of improvements for middle schools.Discipline and attendance results show evidence of noncog-nitive skill improvements at all grade levels. Elementaryschools show improvement in discipline but not attendance,while other grade levels show improvements in both mea-sures. Nonetheless, average attendance rates are higher inelementary grades so there is less room for improvement.The district-wide average is 96.4% for grades 1 through 5and 92.6% for grades 6 and higher.

Panel B of table 4 looks at how charter impacts vary bythe age of the charter in levels models. Value-added modelsprovided similar results and are available in the online appen-dix. Previous work on charter schools has generally foundthat as charters age, their test score impacts improve (Bookeret al., 2007; Hanushek et al., 2007; Sass, 2006; Bifulco &Ladd, 2006). While I caution that since I have 31 charters,there is less variation in this school-level characteristic than inprevious papers, I nonetheless find little evidence of improve-ment in test scores for start-ups or conversions as the schools

age. I do find some evidence of improvement in disciplinein start-ups, as those that are five or more years old have 0.3fewer annual disciplinary infractions per student than first-year start-ups. However, this is offset by lower attendanceimprovements after the first year. It is unclear why atten-dance rates would drop after the first year. One possibilityis that schools that are new need to maintain a closely knitcommunity in order to succeed, so they try harder to inducestudents to attend.

In table 5, I consider how charter school impacts vary bystudent characteristics. Since I do not find substantial evi-dence of impacts for conversion charters, I report only theestimates for start-up charters.34 Each column in panel A orB is a separate regression with a main effect and interactioneffects. No estimate for race is significantly different fromthe main effect in both levels and value-added models, soI cannot say for sure if there is a racial difference. Thereare, however some notable differences by gender. Girls instart-ups appear to perform better on math tests than boys by0.03 to 0.06 standard deviations. While somewhat tenuous—the value-added estimate is significant only at 10%—this isan intriguing result as there has been considerable focus ineducation policy on how to improve math scores for girls. Dis-ciplinary infraction impacts are smaller for girls, though they

34 Full results, including conversion estimates, are provided in the onlineappendix.


Table 6.—Mechanisms of Discipline and Attendance Impacts in Start-up Charters

A: Levels B: Value-AddedDisciplinary DisciplinaryInfractions Attendance Infractions Attendance

(1) Baseline (from tables 2 and 3) Coefficient −0.795∗∗∗ 2.268∗∗∗ −0.498∗∗∗ 2.338∗∗∗(0.113) (0.607) (0.151) (0.757)

Observations 2,233,050 2,233,050 1,777,994 1,777,994(2) Controlling for per student instructional expenditures Coefficient −0.799∗∗∗ 2.318∗∗∗ −0.536∗∗∗ 2.406∗∗∗

(0.121) (0.582) (0.139) (0.791)Observations 2,221,995 2,221,995 1,769,243 1,769,243

(3) Controlling per student school leadership expenditures Coefficient −0.852∗∗∗ 1.908∗∗∗ −0.586∗∗∗ 2.276∗∗∗(0.131) (0.583) (0.183) (0.759)

Observations 2,223,425 2,223,425 1,770,359 1,770,359(4) Controlling per student other expenditures Coefficient −0.849∗∗∗ 1.716∗∗∗ −0.520∗∗ 2.073∗∗∗

(0.129) (0.613) (0.205) (0.756)Observations 2,221,995 2,221,995 1,769,243 1,769,243

(5) Controlling for per student instructional, school leadership, Coefficient −0.755∗∗∗ 1.733∗∗∗ −0.354∗∗ 1.823∗∗and other expenditures (0.115) (0.641) (0.144) (0.733)

Observations 2,221,995 2,221,995 1,769,243 1,769,243(6) Controlling for student body composition Coefficient −0.798∗∗∗ 2.343∗∗∗ −0.341∗ 2.099∗∗∗

(0.113) (0.582) (0.184) (0.680)Observations 2,232,700 2,232,700 1,777,666 1,777,666

(7) Controlling for enrollment Coefficient −0.550∗∗∗ 2.899∗∗∗ −0.125 2.277∗∗∗(0.129) (0.706) (0.200) (0.837)

Observations 2,232,700 2,232,700 1,777,666 1,777,666

Each coefficient is from a separate regression. All regressions include an individual fixed effect, free lunch status, reduced-price lunch status, other economic disadvantage, recent immigration status, whether a parentis a migrant worker, and grade-by-year indicators. Regressions also include a conversion charter estimate that is not shown here. Additional controls for potential charter mechanisms are added as described above.Each row/column combination is a separate regression with both conversion and start-up charter estimates. Full results are provided in the online appendix. Standard errors clustered by school in parentheses. ∗ , ∗∗ , ∗∗∗denote significance at the 10%, 5%, and 1% levels, respectively.

still show large improvements. This is not surprising, as boystend to have more discipline problems on average than girlsdo. For economic status, there is no statistically significantdifference for any outcome. Immigrants have lower improve-ments in attendance than nonimmigrants, but the combinedmain effect and interaction is still positive.35

In table 6 I investigate some mechanisms the charterimpacts may work through. I showed in table 1 that start-up charters have more instructional expenditures and fewerexpenditures on other functions, and they are generallysmaller than noncharter schools. Hence, seeing how theimpact estimates change as we control for these characteris-tics can provide insight into the paths through which chartersaffect students. Nonetheless, these covariates are potentiallyendogenous, so one should be careful of interpreting anychange in the charter impact estimate as being causally deter-mined by the added covariates. Tests score estimates for allcharters and discipline and attendance estimates for conver-sions are generally unaffected by adding these covariates, soI leave those results in the online appendix.

The first row of table 6 repeats the baseline estimates fromtables 2 and 3. The second through fifth rows show whathappens when we add different categories of per student

35 An additional element of heterogeneity that one may expect to see isthat students with precharter behavioral problems improve more in start-up charters than other students do. Indeed, regressions that interact charterstatus with infractions in the year prior to charter entry show that the impactson infractions increase as precharter infractions increase. Nonetheless, thisdoes not appear to spill over into test scores, as improvements in thesemeasures fall with the number of infractions, though not to the point wherea large number of students would have significantly negative test scoreimpacts. These results are available on request.

expenditures. Adding these factors does not generate sub-stantial changes in the impact estimates. In row 6 I controlfor the percentage of each school that is white, Hispanic,black, limited English proficient, special education, gifted,and economically disadvantaged. In this case, there is a dropin the discipline estimate in value-added models, to the pointwhere it is significant only at the 10% level. However, all ofthe other estimates change little.

Row 7 controls for a quadratic in total school enrollment.Since start-ups tend to be smaller than noncharters, admin-istrators may find it easier to maintain control over studentsand spend more time dealing with discipline problems. Theresults in row 7 are consistent with this theory. When enroll-ment is added, the disciplinary infraction estimate rises by0.145 in levels and .373 in value-added models. While thelevels estimate remains statistically significant, the value-added estimates becomes insignificant. Therefore, it appearsthat school size and peer characteristics may play roles inthe discipline improvements found in the start-up charters. Inparticular, part of the discipline improvement may be due toadministrators’ having smaller schools that they can closelymonitor. This could be an important avenue for further studyas there is much interest in the benefits of small schools.

B. Evolution of Charter Impacts over Time

One of the key advantages of the LUSD-SW data set isthat the length and breadth of the panel allows me to inves-tigate some long-term impacts of charters on cognitive andnoncognitive skills that most of the previous research is notable to do. First, in table 7, I provide results from regressionswhere charter impacts are allowed to vary by time spent in


Table 7.—Charter Impacts by Time in Charter


A: LevelsStart-up—1 year 0.013 −0.016 −0.004 −0.837∗∗∗ 3.270∗∗∗

(0.060) (0.045) (0.042) (0.115) (1.009)

Start-up—2 years 0.008 −0.062 −0.008 −1.042∗∗∗ 2.197∗∗∗(0.078) (0.075) (0.059) (0.200) (0.412)

Start-up—3 years −0.114 −0.128 −0.122 −0.981∗∗∗ 3.756∗∗∗(0.098) (0.106) (0.082) (0.168) (0.660)

Start-up—4 or more years 0.151 −0.122 −0.042 −1.577∗∗∗ 11.774(0.270) (0.150) (0.130) (0.354) (10.629)

Conversion—1 year 0.095∗ 0.056∗ 0.064∗ 0.006 0.029(0.050) (0.029) (0.039) (0.048) (0.179)

Conversion—2 years −0.013 −0.074 −0.007 0.075 −0.175(0.138) (0.100) (0.091) (0.082) (0.360)

Conversion—3 years 0.062 0.031 −0.020 −0.182 0.564(0.139) (0.093) (0.061) (0.111) (0.421)

Conversion—4 or more years 0.019 0.000 0.095 −0.207∗ 0.984∗(0.137) (0.142) (0.086) (0.118) (0.535)

Observations 1,141,480 1,141,480 1,141,480 2,233,050 2,233,050B: Value added

Start-up—1 year 0.053 −0.006 0.046 −0.792∗∗∗ 3.800∗∗∗(0.052) (0.039) (0.047) (0.222) (1.408)

Start-up—2 years −0.005 −0.084∗ 0.017 −0.226∗ 1.700∗∗∗(0.068) (0.050) (0.048) (0.122) (0.442)

Start-up—3 years 0.159 0.103 0.121 −0.232 3.645∗∗∗(0.163) (0.112) (0.089) (0.359) (0.636)

Start-up—4 or more years 0.446∗∗ 0.202∗ 0.375∗∗ −0.035 7.327(0.183) (0.120) (0.148) (0.731) (5.546)

Conversion—1 year 0.069 −0.021 0.053 0.004 0.148(0.108) (0.075) (0.055) (0.063) (0.203)

Conversion—2 years −0.163 −0.179 0.018 −0.186∗∗ 0.477(0.155) (0.112) (0.087) (0.093) (0.329)

Conversion—3 years 0.335∗∗ 0.165 0.253 −0.208∗∗ 0.740(0.165) (0.139) (0.223) (0.097) (0.501)

Conversion—4 or more years 0.012 0.001 0.255 −0.150 0.992(0.332) (0.218) (0.171) (0.094) (0.604)

Observations 779,343 779,343 779,343 1,777,994 1,777,994

Each column in each panel is a separate regression. All regressions include an individual fixed effect, free lunch status, reduced-price lunch status, other economic disadvantage, recent immigration status, whethera parent is a migrant worker, and grade-by-year indicators. All post-first-year periods are instrumented by the potential number of years the student could have been in the charter school based on grade at charter entry.Standard errors clustered by school in parentheses. ∗ , ∗∗ , ∗∗∗ denote significance at the 10%, 5%, and 1% levels, respectively. Levels models cover 1998–2006 and grades 1–11 for test scores and 1994–2006 and grades 1–12for other outcomes. Value-added models cover 1999–2006 and grades 2–11 for test scores and 1994–2006 and grades 2–12 for other outcomes. Observation counts by time in charter are provided in the online appendix.

a charter. It is possible that initially a student may not per-form well in a charter due to the shock of having to changeeducational environments, but over time, the student couldacclimate to the new environment and improve. Or it is pos-sible that the new environment of attending a charter couldtemporarily motivate the student and improve performance,but as time goes on, the student may return to old habits.

In order to address the potential for endogenous exit, Iinstrument for charter attendance in any year after the first.In this case, I use the number of potential years a studentcould have been in the charter. That is, I subtract the firstgrade offered in the charter from the grade the student is pre-dicted to be in. The predicted grade is calculated by assumingnormal grade progression from the grade the student was inwhen he or she entered the charter. For test scores (columns1–3), start-up charters show some evidence of improvementwhen students have been in a charter for four or more years,but only in value-added models. Thus, there appears to besome weak evidence that students who spend long peri-ods of time in start-ups improve, but those who spend onlyshort periods of time in start-ups do no better or worse. For

conversion charters, it appears that test score impacts remainroughly constant as time in the charter increases.36

Looking at columns 4 and 5, for discipline and attendancethere appears to be improvement as time in start-up chartersincrease. For example, in the first year, annual disciplinaryinfractions are down by 0.8 in levels models. After the thirdyear, they are down by 1.6. An F-test of this difference is sta-tistically significant at the 5% level. Attendance also showsa pattern of improvement. The pattern for discipline does nothold for start-ups in value-added models. This suggests thatdiscipline improves substantially in the first year, but furtherimprovements in subsequent years are small. Nonetheless,discipline impacts do increase as time spent in the charterincreases. These results also show that there is no mean rever-sion in charter impacts and thus provide further evidence thatthe preentry dips in test scores and attendance do not imposesubstantial bias.

36 The 2SLS models are somewhat different from OLS models, whichshow improvement in start-up charters over time. These results are availablein the online appendix.


Figure 6.—Test Scores before and after Charter Exit

Figure 7.—Discipline and Attendance by before and after Charter Exit




Table 8.—Persistence of Charter Impacts


A: Levels ModelsStart-up 0.038 −0.734∗∗∗ −0.734∗∗∗ 2.027∗∗∗ 2.509∗∗∗

(0.046) (0.109) (0.123) (0.554) (0.725)1 to 3 years after start-up 0.048 0.167∗∗∗ 0.244 −0.712∗∗∗ 0.655

(0.041) (0.047) (0.151) (0.209) (0.521)

4 or more years after start-up −0.040 0.161∗∗∗ −0.104 −0.829∗∗∗ −0.033(0.165) (0.051) (0.165) (0.224) (1.253)

Conversion 0.007 0.015 0.110 −0.322∗ −0.771∗∗∗(0.039) (0.066) (0.120) (0.189) (0.289)

1 to 3 years after conversion −0.050 −0.025 0.173 −0.522∗∗∗ −1.295∗∗∗(0.058) (0.052) (0.198) (0.191) (0.429)

4 or more years after conversion −0.065 0.027 0.053 −0.358∗ −1.302∗∗∗(0.063) (0.052) (0.166) (0.209) (0.432)

F-test of post-start-up estimates 2.11 0.10 0.96 2.93∗ 1.3F-test of post-conversion estimates 5.69∗∗∗ 2.65∗ 0.57 0.96 6.42∗∗∗Observations 1,141,480 1,141,480 1,141,480 2,233,050 2,233,050

B: Value added ModelsStart-up 0.002 −0.446∗∗∗ −0.396∗∗∗ 2.319∗∗∗ 2.414∗∗∗

(0.033) (0.154) (0.148) (0.776) (0.853)1 to 3 years after start-up 0.033 0.192∗∗∗ 0.561∗∗∗ −0.001 −0.027

(0.041) (0.052) (0.154) (0.152) (0.438)

4 or more years after start-up −0.073 0.025 −0.237 −0.383∗∗ 0.602(0.187) (0.046) (0.193) (0.181) (0.910)

Conversion −0.072 −0.092 0.092 −0.062 −0.708∗∗∗(0.063) (0.058) (0.115) (0.170) (0.231)

1 to 3 Years after conversion −0.109 −0.185∗∗∗ 0.155 −0.144 −1.255∗∗∗(0.072) (0.051) (0.173) (0.150) (0.278)

4 or more years after conversion −0.132∗ −0.056 0.030 −0.316∗ −1.099∗∗∗(0.070) (0.041) (0.119) (0.170) (0.311)

F-test of post-start-up estimates 0.52 3.29∗∗ 0.51 13.04∗∗∗ 0.30F-test of post-conversion estimates 0.74 2.04 2.18 0.79 11.59∗∗∗Observations 779,343 779,343 779,343 1,777,994 1,777,994

Each column in each panel is a separate regression. Two- and three-year postcharter estimates are similar to the one- and four-year estimates, so I do not show them here, though the full set of results is provided inthe online appendix. All regressions include an individual fixed effect, free lunch status, reduced-price lunch status, other economic disadvantage, recent immigration status, whether a parent is a migrant worker, andgrade-by-year indicators. All post-charter periods are instrumented by whether the student is the listed number of years beyond the last grade level covered by his or her prior charter school. Standard errors clusteredby school in parentheses. ∗ , ∗∗ , ∗∗∗ denote significance at the 10%, 5%, and 1% levels, respectively. Levels models cover 1998–2006 and grades 1–11 for test scores and 1994–2006 and grades 1–12 for other outcomes.Value-added models cover 1999–2006 and grades 2–11 for test scores and 1994–2006 and grades 2–12 for other outcomes. Observation counts by post-charter period are provided in the online appendix.

I now turn to how charter impacts persist after stu-dents return to regular public schools. Analyzing persistenceinforms us as to whether the skills students learn in char-ters remain without the need for reinforcement. Persistencecan also generate bias in the impact estimates since someof the impacts would be attributed to noncharter periods.Figures 6 and 7 provide an initial look at whether charterimpacts persist by graphing residuals from the regressionsused for figures 4 and 5 for students before and after theyleave charters. Figure 6 shows that after charter exit, mathtest scores drop for start-ups and all subjects drop for con-versions. Figure 7 shows that the discipline and attendancegains achieved while students are enrolled in start-ups dropoff after they leave. These measures are relatively stable afterstudents leave conversions. Hence, these figures suggest thatcharter impacts do not persist after students leave.

In table 8, I provide regression results that account forthe persistence of charter impacts when students return toregular public schools. The table shows 2SLS results usingthe instrument for post-charter periods described at the endof section IIIB. To check persistence, I use indicators forhaving attended a charter one to three years prior and for

having attended a charter more than four years prior.37 Testscores are provided in columns 1 to 3, discipline in column4, and attendance in column 5. Each column in panels A andB refers to a single regression.

First, for conversion charters, any significant positiveimpacts found in the baseline models become statisticallyinsignificant. This provides additional confirmation that con-version charters are ineffective at improving student out-comes. Turning to the results for start-ups, columns 1, 2 and3 show that after leaving the charter, students perform wellon test scores in the first three years, with statistically sig-nificant and positive impacts in the levels models for math.Nonetheless, all of the other estimates for post-start-up testscores are either negative or statistically insignificant. Thus,the evidence for persistence in test scores for start-ups isweak. As such, the estimates for the start-up impact on testscores remain statistically insignificant at the 5% or lowerlevel in all models.

37 OLS results are available in the online appendix. Analyses using indica-tors for one, two, three, and four or more years post-charter showed similarresults and are also available in the online appendix.


Columns 4 and 5 show that after start-up students return toregular public schools, disciplinary infractions increase andattendance falls to the point where they do not differ fromnoncharter students. While it is true that in the post-charterperiods, the students are older and thus we would expect theseoutcomes to worsen, the inclusion of grade-by-year indicatorscorrects for this problem. Hence, there is no persistent impactof start-up charters on noncognitive skills.

One possible explanation for this result is that the noncog-nitive skill improvements from start-up charters need contin-ual reinforcement or else they dissipate. Another possibleexplanation is that the improvements in behavior are dueto institutional factors rather than skill formation. Hence,while the start-ups provide an environment that helps stu-dents improve behavior and attendance, once they return toregular public schools, they lose the benefit of that environ-ment. A third alternative is that the estimated impacts reflectreporting and enforcement bias in charters. However, as dis-cussed in section IVA, it is highly unlikely that attendancerates are manipulated while the discipline results appear tobe, at least partially, due to real behavioral changes.

V. Conclusion

In this paper, I investigate the effect of charter schools onstudents who attend them using data from a large urban schooldistrict in the Southwest (LUSD-SW) with an extensive char-ter program. I provide a broad outlook on charter schoolsby focusing on multiple outcomes and using a panel withwide grade coverage and long time frames to provide anal-yses of long-term impacts both while in and after attendingcharters. By combining analyses of test scores with disci-pline and attendance impacts, I am able to identify whethercharter schools affect both cognitive and noncognitive skillformation.

I find that while charters that convert from regular pub-lic schools to charter status have little effect on cognitive ornoncognitive skills, schools that begin as charters, that is,start-up charters, generate large improvements in disciplineand attendance. Disciplinary infractions drop by 0.5 to 0.8instances per year compared to a precharter mean of 1.1,and attendance rates rise by 2.3 percentage points, whichis equivalent to 26% of the precharter mean. These impactsare statistically significant at the 1% level across many mod-els and specification checks. While discipline and attendanceimpacts could also be caused by cognitive skill enhance-ments, I can establish improvements in test scores only formath in middle school start-ups, whereas estimates for ele-mentary and high schools are close to zero on all test scoremeasures. Hence, I interpret the discipline and attendanceresults as showing that start-up charters generate noncogni-tive skill improvements. These individual fixed-effects resultsare robust to potential biases from selection into charters offtrends in outcomes and attrition.

The long and broad nature of my panel also allow meto follow students for a number of years while they are

enrolled in charters and after they leave charters. Whileimpacts for either type of charter are relatively stable astime in the charter increases, the discipline and attendanceimpacts found in start-ups do not persist after students returnto regular public schools, so there do not appear to be long-term behavioral improvements. This result suggests that anynoncognitive skills that have improved in start-up chartersrequire constant reinforcement. I cannot rule out, however,that the improvements in attendance and discipline mayinstead reflect institutional differences between charters andnoncharter public schools.

I also caution that while the discipline and attendanceresults are intriguing, they may reflect reporting or enforce-ment differences. The lack of persistence increases thisconcern as such a result is consistent with reporting bias.Since attendance is difficult to manipulate or misreport, thisis mainly a concern for discipline. To address this, I pro-vide multiple pieces of indirect evidence suggesting thatthe discipline results, at least in part, reflect real behavioralchanges in the start-up charter students. In particular, the largeimpacts on attendance buttress the discipline improvements,and combined, they provide substantial evidence that start-ups generate noncognitive skill improvements in spite of thelack of cognitive skill improvement.

Finally, I should note that charter schools vary consider-ably in their mission and the rules they are subject to; eachis a unique entity. Hence, it is possible that the results forthe charters in LUSD do not extend to other school districtsand states. Therefore, it is important that future research oncharter schools consider alternative outcomes like disciplineand attendance so that we can determine whether noncogni-tive skill improvement is a general characteristic of charterschools.

REFERENCES

Ashenfelter, Orley, “Estimating the Effect of Training Programs on Earn-ings,” this review 60:1 (1978), 47–57.

Betts, Julian R., Lorien A. Rice, Andrew C. Zau, Y. Emily Tang, and CoryR. Koedel, “Does School Choice Work? Effects on Student Integra-tion and Achievement,” Public Policy Institute of California report(2006).

Bifulco, Robert, and Helen F. Ladd, “The Impacts of Charter Schools onStudent Achievement: Evidence from North Carolina,” EducationFinance and Policy 1:1 (2006), 123–138.

Booker, Kevin, Scott M. Gilpatric, Timothy Gronberg, and Dennis Jansen,“The Impact of Charter School Attendance on Student Perfor-mance,” Journal of Public Economics 91:5/6 (2007), 849–876.

Buddin, Richard, and Ron Zimmer, “Student Achievement in CharterSchools: A Complex Picture,” Journal of Policy Analysis andManagement 24:2 (2005), 351–371.

Chay, Kenneth Y., Patrick J. McEwan, and Miguel Urquiola, “The Cen-tral Role of Noise in Evaluating Interventions That Use TestScores to Rank Schools,” American Economic Review 95:4 (2005),1237–1258.

Clark, Damon, “The Performance and Competitive Effects of SchoolAutonomy,” Journal of Political Economy 117:4 (2009), 745–783.

Dustmann, Christian, and Maria E. Rochina-Barrachina, “Selection Cor-rection in Panel Data Models: An Application to Labour Supply andWages,” IZA discussion paper no. 162 (2000).

Guryan, John, “Desegregation and Black Dropout Rates,” NBER workingpaper no. 8345 (2001).


Hanushek, Eric A., John F. Kain, Steven G. Rivkin, and Gregory F. Branch,“Charter School Quality and Parental Decision Making with SchoolChoice,” Journal of Public Economics 91:5/6 (2007), 823–848.

Heckman, James J., and Jeffrey A. Smith, “The Pre-Programme EarningsDip and the Determinants of Participation in a Social Programme:Implications for Simple Programme Evaluation Strategies,” Eco-nomic Journal 109:457 (1999), 313–348.

Heckman, James J., and Yona Rubinstein, “The Importance of Noncog-nitive Skills: Lessons from the GED Testing Program,” AmericanEconomic Review: Papers and Proceedings 91:2 (2001), 145–149.

Heckman, James J., Jora Stixrud, and Sergio Urzua, “The Effects ofCognitive and Noncognitive Abilities on Labor Market Outcomesand Social Behavior,” Journal of Labor Economics 24:3 (2006),411–482.

Hoxby, Caroline M., and Sonali Murarka, “Charter Schools in NewYork City: Who Enrolls and How They Affect Their Students’Achievement,” NBER working paper no. 14852 (2009).

Hoxby, Caroline M., and Jonah E. Rockoff, “The Impact of Charter Schoolson Student Achievement,” Department of Economics, HarvardUniversity, mimeo. (2004).

Kyriazidou, Ekaterini, “Estimation of a Panel Data Sample SelectionModel,” Econometrica 65:6 (1997), 1335–1364.

McClure, Larry, Betsy Strick, Rachel Jacob-Almeida, and ChristopherReicher, “The Preuss School at UCSD: School Characteristics andStudents’ Achievement,” CREATE-UCSD report (2005).

Sass, Tim R., “Charter Schools and Student Achievement in Florida,”Education Finance and Policy 1:1 (2006), 123–138.

Segal, Carmit, “Classroom Behavior,” Journal of Human Resources 43:4(2009a), 783–814.

——— “Misbehavior, Education, and Labor Market Outcomes,” Depart-ment of Economics, Universitat Pompeu Fabra, mimeo. (2009b).

Solmon, Lewis, and Pete Goldschmidt, “Comparison of Traditional PublicSchools and Charter Schools on Retention, School Switching, andAchievement Growth,” Goldwater Institute policy report (2004).

Solmon, Lewis, Kern Paark, and David Garcia, “Does Charter SchoolAttendance Improve Test Scores? The Arizona Results,” GoldwaterInstitute occasional report (2001).

Todd, Petra E., and Kenneth I. Wolpin, “The Production of CognitiveAchievement in Children: Home, School and Racial Test ScoreGaps,” Journal of Human Capital 1:1 (2007), 91–136.

Weiher, Gregory, R., and Kent L. Tedin, “Does Choice Lead to RaciallyDistinctive Schools? Charter Schools and Household Prefer-ences,” Journal of Policy Analysis and Management 21 (2000),79–92.

Zimmer, Ron, and Richard Buddin, “Academic Outcomes” (pp. 37–62),in Charter School Operations and Performance: Evidence fromCalifornia (Santa Monica, CA: RAND, 2003).

——— “Charter School Performance in Two Large Urban Districts,”Journal of Urban Economics 60:2 (2006), 307–326.

Zimmer, Ron, Suzanne Blanc, Brian Gill, and Jolley Christman, “Evaluatingthe Performance of Philadelphia’s Charter Schools,” Rand workingpaper (2008).

APPENDIX

Proof of Expected Value of Level and Value-Added Fixed Effects EstimatesBounding the Lagged-Dependent Variable Model with Fixed Effects

Let us first simplify notation and denote X as a k×nt vector of demeanedcovariates while Y is a 1 × nt vector of the demeaned student outcomevariable and Yt−1 is the 1 × nt vector of demeaned once-lagged outcomevariables. Our true model becomes

Yt = Xβ + Yt−1γ + ε. (A1)

In a levels framework, the lagged outcomes enter into the error term suchthat there is composite error:

μ = γYt−1 + ε. (A2)

This provides us with

E(β̂L) = β + γ[X′X]−1[X′E(Yt−1)]. (A3)

For a value-added model, we subtract Yt−1 from each side of equation (A1)to get

Yt − Yt−1 = Xβ + (γ − 1)Yt−1 + ε, (A4)

which will provide us with an estimate of b such that

E(β̂VA) = β + (γ − 1)[X′X]−1[X′E(Yt−1)]. (A5)

Let us further define the matrix A = [X′X]−1[X′E(Yt−1)] and the kth rowof A as Ak; hence,

E(β̂Lk ) = βk + γAk (A6)

E(β̂VAk ) = βk + (γ − 1)Ak. (A7)

Thus, assuming that 0 ≤ γ ≤ 1, if Ak > 0, then E(β̂Lk ) > β > E(β̂VA

k ),while if Ak < 0, then E(β̂L

k ) < β < E(β̂VAk ). In either case, the levels model

and value-added models bound β.

This article has been cited by:

1. C. Kirabo Jackson. 2012. School competition and teacher labor markets: Evidence from charter school entry in North Carolina.Journal of Public Economics 96:5-6, 431-448. [CrossRef]

2. Elisabetta Gentile, Scott A. Imberman. 2011. Dressed for Success? The Effect of School Uniforms on Student Achievement andBehavior. Journal of Urban Economics . [CrossRef]

3. Celeste K. Carruthers. 2011. New schools, new students, new teachers: Evaluating the effectiveness of charter schools. Economicsof Education Review . [CrossRef]

4. Ron Zimmer, Brian Gill, Kevin Booker, Stéphane Lavertu, John Witte. 2011. Examining charter student achievement effectsacross seven states. Economics of Education Review . [CrossRef]

http://dx.doi.org/10.1016/j.jpubeco.2011.12.006

http://dx.doi.org/10.1016/j.jue.2011.10.002

http://dx.doi.org/10.1016/j.econedurev.2011.06.001

http://dx.doi.org/10.1016/j.econedurev.2011.05.005

imberman 2011 - achievement and behavior in charter schools

Documents