report executive summary: what we have learned about ......executive summary: what we have learned...

ClarifyingComplex

EducationIssues

September 2002

R E P O R T

This summary was preparedby EdSource on behalf of the CSR Research Consortium.It is an excerpt from the2002 publication, What We Have Learned AboutClass Size Reduction inCalifornia, prepared for the California Department of Education as the capstonereport of a four-year study of class size reduction in California. The full report isavailable at: www.classize.org

© Copyright 2002 by EdSource, Inc.

alifornia made education history of a sort in 1996 when itspent $1 billion to reduce class sizes in kindergarten

through third grade classes statewide. This vast ClassSize Reduction (CSR) program has now been in operationthrough six full school years and in many ways has be-come institutionalized in the state’s public schools.

During that time, a state-commissionedevaluation conducted by the CSR ResearchConsortium has been examining the program, its implementation, and its impact on school operations and student achievement. In June 2002the Consortium—made up ofAmerican Institutes for Research(AIR), RAND, Policy Analysisfor California Education(PACE), EdSource, andWestEd—issued its capstone report on this extended evaluation, entitled What We HaveLearned About ClassSize Reduction inCalifornia. This is the executive summaryof that Consortium report. It briefly recapsthe history of K–3class size reduction in California, highlightsthe major findingsfrom the Consortium’swork, and summarizesits key policy recom-mendations. The full report as well as previous research and technical docu-ments are available at www.classize.org.

C

Executive Summary:

What We Have Learned About K–3 Class Size Reduction in California

An Overview of K–3 CSR in CaliforniaThe mid-1990s found California worried aboutthe education its students were receiving.Standardized tests provided evidence that thestate’s students were losing ground compared totheir counterparts across the country. The re-sults of the 1994 National Assessment of Edu-cational Progress (NAEP) released in 1995only reinforced the concern—California’sfourth graders had tied for last place in readingamong the 39 states that participated in NAEP.

A task force assembled by the CaliforniaDepartment of Education called for variousreforms, among which smaller classes wasstrongly favored not only by the teachers’unions, but also by parents and teachers. Cali-fornia elementary schools had the largest classsize in the country—averaging 29 students.Evidence from the Tennessee STAR experi-ment (see To Learn More box on page 8) hadshown rather clearly that elementary studentsin the primary grades did better academicallywhen in smaller versus larger classes in K–3,and the difference was greatest for low-income and minority students.

All that was missing to put class size reduc-tion into place was the political will and themoney to do so. The dot-com boom of the1990s solved the latter problem by providing awindfall of tax revenues, most of which wererequired by law to be spent on elementary andsecondary education. Republican GovernorPete Wilson and the Democratically controlledLegislature seized the moment and passed Sen-ate Bill 1777 (O’Connell) in July 1996. Thelaw provided districts with $650 per student foreach K–3 classroom with 20 or fewer students,provided that they first reduced all first-gradeclasses in a school, followed by all second-grade classes, and finally by either kindergartenor third-grade classes. The cost to the state inthe first year was roughly $1 billion dollars, andin the current year roughly $1.6 billion.

The California Department of Educationand a group of California foundations awardedcontracts to the American Institutes for Research (AIR), which along with RANDheaded a consortium to evaluate the effects of

class size reduction on achievement, on thequality of the state’s teaching corps, on specialneeds students, and on other practices. The Consortium—which also included PolicyAnalysis for California Education (PACE),EdSource, and WestEd—has produced threeevaluation reports thus far. This is the fourthand final report. In it, we summarize previousfindings and discuss new research done in thefinal year of the contract; we also include a setof policy recommendations and conclude withpolicy lessons learned.

What did we learn?The major findings from this four-year evaluation can be summarized as follows:

1. Implementation of CSR occurred rapidly, although it lagged in schools serving minorityand low-income students. Districts reactedquickly to the opportunity presented by theCSR program when it was enacted in July1996. CSR implementation was virtuallycomplete for first and second grades by thesecond year of the program, and for kinder-garten and third-grade students by thefourth year of the program. Implementationwas slower, however, for schools with higherpercentages of minority students and oflow-income students, partially becauseschools in urban districts had more diffi-culty acquiring the needed space to expandthe number of classrooms.

2. Our analyses of the relationship of CSR tostudent achievement were inconclusive.Student achievement has been increasingsince the first administration of the Stan-ford Achievement Test (SAT-9) in 1998,but we could find only limited evidencelinking these gains to CSR. We found apositive association in 1998 between third-grade class size and SAT-9 scores after controlling for differences in student andschool characteristics. However, the size of this CSR effect was small. (See figureon page 3.) In the following year (1998–99),these positive differences persisted whenstudents who had been in reduced sizethird-grade classes moved to the fourthgrade and regular size classes. The spring1999 SAT-9 results showed that fourth-

Executive Summary: CSR ● September 2002

Trish WilliamsEdSource Executive Director

2002–03 EdSource Board of DirectorsKelvin K. Lee, PresidentSuperintendent, Dry Creek Joint Elementary School District

Peter Schrag, Vice PresidentContributing Editor, Sacramento Bee

Lawrence O. Picus, Fiscal OfficerProfessor, Rossier School of Educa-tion, University of Southern California

Robert G. Haskell, SecretarySenior Vice President, Public Affairs,Pacific Life Insurance Company

Ray BacchettiScholar in Residence, CarnegieFoundation for the Advancement of Teaching

Pam BradyVice President for Education, California State PTA

Davis CampbellPresident, CSBAGovernance Institute

Susan J. CochranPresident, American Associationof University Women–California

Judy DavisChair, League of Women Votersof California Education Fund

Jan DomenePresident, California State PTA

Joni E. FinneyVice President, National Center forPublic Policy and Higher Education

Gerald C. HaywardDirector, Policy Analysis for California Education

Ann Nkiruka IfekwunigweNational Board Certified Teacher,Carthay Center Elementary School

Barbara InatsuguPresident, League of Women Voters of California

Maggie Carrillo MejiaSuperintendent, Montebello Unified School District

John B. MocklerPresident,John Mockler & Associates

Ted OlssonManager, Corporate Community Relations, Western United States,IBM Corporation

Amado M. PadillaProfessor of Education,Stanford University

Krys WulffPast President, American Associationof University Women–California

2

grade students who had been in reducedsize third-grade classes scored higher thanthose who had not been in such classes. By 2001, CSR implementation was nearlycomplete, and as a result we could not ex-amine differences in SAT-9 scores betweenstudents who were and were not in reducedsize classes. Instead, we tracked achieve-ment gains between cohorts of studentswith slightly different patterns of exposureto CSR from kindergarten through thirdgrade. Although both overall exposure toCSR and statewide average test scores in-creased across cohorts, the magnitude ofthe changes in test scores did not trackwith the incremental changes in CSR.Thus, attribution of gains in scores to CSR is not warranted. More refinedschool-level analyses also failed to findmeaningful differences in second- or third-grade scores of students with an additionalyear of exposure to CSR in first grade com-pared to students who participated only in grades 2 and 3. We could not determinewhether our inability to link CSR toachievement was due to weakness of the ef-fect of incremental differences in CSR orto design limitations (or a combination ofboth). We were also limited in our ability todetermine how much of the recent gain inachievement was attributable to CSR andhow much was linked to other initiatives.

3. CSR was associated with declines in teacherqualifications and a more inequitable distribution of credentialed teachers. CSR im-plementation along with increasing enroll-ments required an enormous increase in thenumber of K–3 teachers in California. Be-tween 1995–96 (the year before CSR imple-mentation) and 1998–99 (the third year ofthe program), the total number of K–3teachers increased 46 percent, from 62,226to 91,112. To meet the increased demand forteachers, many districts hired teachers with-out full credentials. As a result, the propor-tion of K–3 teachers who were not fullycredentialed (e.g., teachers with intern oremergency credentials) increased from 1.8percent before the program started to 12.5percent in the second year of the program.Most of the uncredentialed teachers were

hired by schools serving the most disadvan-taged students, in part because these schoolswere slower to implement CSR, and certifi-cated teachers had already been hired else-where. (See figure on page 4.) In 2000–01,more than one in five K–3 teachers were not fully credentialed in schools with high percentages of low-income, English learners(EL), minority, and Hispanic students (primarily large and urban).

4. CSR had only a modest effect on teacher mobility. One of the fears was that class sizereduction would result in two types ofteacher mobility—teachers from urbanschools moving into suburban schools, andupper grade elementary teachers movinginto K–3. While there was some initial increase, the effect was small and soon disappeared. Approximately 7 percent offirst-grade teachers in 1995–96 (the yearprior to CSR) had been teaching in a dif-ferent school the previous year. That per-centage rose to 11 percent in 1996–97 and had dropped down to 5 percent by1999–2000. The same pattern was true inthe other elementary grades. The schooltransfer rate was small, especially whencompared to the 46 percent increase in thenumber of K–3 teachers during this period.


3

0

10

20

30

40

50

60

70

80

90

100

CSR Increment

Third Grade Non-CSR

SpellingLanguageMathematicsReading

Subject

Per

cen

tag

e

Percentage of Students Scoring Above National Median,by CSR Status and by Subject, on SAT-9 in 1998

5. CSR implementation did not affect special education identification or placement. Therewas some concern that reducing class sizemight affect the number of students referredfor special education assessment, the num-ber of students identified as needing specialeducation services, or the number of specialeducation students placed in special dayclasses (instead of in general educationclasses). Interviews with special educationdirectors indicated that the referrals for special education assessments increased.Our analyses of statewide enrollment datashowed no evidence that CSR affected the rates for identification or placement.

6. Students in reduced size third-grade classes received more individual attention, but similarinstruction and curriculum. Compared toteachers with larger classes, teachers of re-duced size classes were more likely to saythat they know what each student knowsand can do, that they provide feedback onwriting assignments within one day, thatthey give more individual attention to stu-dents, and that they are able to meet theinstructional needs of all students. Teachersin reduced size classes also reported fewerbehavior problems and reported that stu-dents were more likely to complete the lesson for the day and less likely to be “off

task” for more than five minutes. Butteachers in both reduced and non-reducedsize third-grade classes reported spendingsimilar amounts of time and covering similar amounts of curriculum in both lan-guage arts and mathematics.

7. Parents liked reduced size classes. Based onsurvey results, parents of third-grade stu-dents in reduced size classes rated selectedfeatures of their child’s education higherthan did parents of children in non-reduced size classes. The differences in rating of classroom size were particularly pronounced, with parents of children in re-duced classes reporting satisfaction levels farhigher than those parents of children innon-reduced classes. However, parents ofchildren in both reduced and non-reducedsize classes expressed equal satisfaction withthe qualifications of their children’s teachers.

8. Classroom space and dollars were taken fromother programs to support CSR. Most districtsin our statewide sample reported incurringoperating costs for CSR that exceeded statepayments for it, and these funding problemspersisted or even worsened in subsequentyears. Districts attempted to overcome bud-get shortfalls created by CSR by reducingfunds for facility maintenance and adminis-trative services. About one-third of thesedistricts also reduced resources for profes-sional development, computer programs, or libraries. To be able to implement the program, many schools reported having toreallocate classrooms that had been desig-nated for special education back to K–3classrooms, thereby forcing special educationclasses to use alternative spaces. CSR imple-mentation also preempted space from suchprograms as music and arts, athletics, andchild care.

9. In spite of budget shortfalls, most districts arenot projecting CSR cutbacks for 2002–03.In spite of the fact that the state of Califor-nia is projecting a significant budget deficitfor 2002–03, and that many districts areforecasting total revenues that will not meetprojected expenses, none of the 38 districtswe surveyed in 2002 indicated that they arecontemplating elimination of CSR in the

4


0

5

10

15

20

25

30% or more

17.5% to 30%

7.5% to 17.5%

Less than 7.5%

2000–011999–001998–991997–981996–971995–96

Academic Year

Per

cen

tag

e o

f K

–3 W

ork

forc

e

Percentage of K–3 Teachers Not Fully Credentialed,by Low-income Students in School

immediate future. Some did indicate, how-ever, that such a reduction was a possibilityand would continue to be discussed as theirbudgets were developed. However, it wouldbe a “last resort” given the popularity ofCSR with parents and teachers.

Policy Recommendations CSR is an enormously popular program in California among parents and teachers. It isalso clear that local educators and parentsvalue reduced class sizes for many reasonsother than improved achievement as measuredby statewide test scores. Therefore, maintain-ing small K–3 classes in California is likely toremain a priority. Nonetheless, based on ourevaluation and on research done on CSR inother states, we believe that some changes tothe program should be considered. In addition,the state policy environment of 2002 ismarkedly different from that which existed in1996 when CSR was introduced and implemen-tation began. The state has started moving to-ward a systemic standards-based system, with astrong emphasis on high expectations, account-ability, and accompanying rewards and sanctionsbased on growth in student achievement.

1. Improve the effectiveness of the current CSRprogram by integrating and aligning it withother reforms.

The Consortium is impressed with theneed to link CSR to the state’s overallstrategic direction—i.e., to end its currentstatus as a stand-alone categorical programby integrating and aligning it with thestate’s standards-based policies. Such ashift, we believe, would allow CSR to better support the state’s standards-basedreform strategy and might prompt better results from the CSR investment. Schoolsmay be able to use other elements of standards-based reform—e.g., additionalfunding to turn around low-performingschools—in ways that allow them to takefuller advantage of the opportunities smallclasses have been shown to present in somestates, especially for low-income and mi-nority students. In short, integrating CSRwith the state’s evolving standards-based reform policies could significantly bolster

California’s ability to meet its objective of improving student achievement.

2. Be explicit about the assumptions underlyingstate reimbursement of CSR and take steps todetermine the real costs and cost-effectivenessof CSR.

There exists a fundamental difference in theway CSR is viewed by state policymakers andschool district personnel. State officials de-scribe the program as an incentive program,not a state mandate. As such, they argue,districts have the option to participate ornot. They further argue that the state pro-vides adequate resources through the combi-nation of CSR funding and general purposefunding. Many districts, however, feel thatthe state indicated an intent to fully fund theprogram when it adjusted the funding up-ward in 1997. State support has not kept upwith costs since then, and districts believethat the state should once again provide ade-quate resources for full funding. Regardless ofwhether or not costs are fully reimbursed, therules regarding appropriate cost attributionshould be explicit, and districts ought to havereasonably predictable revenue streams sothat they can make informed choices aboutimplementing CSR. While determiningcosts attributable to specific programs is not asimple matter, a careful cost review could il-luminate this issue and result in a single setof rules relating to cost attribution. Evenmore importantly, having solid cost datawould assist state and local policymakers indetermining the cost-effectiveness of CSRcompared with other possible reforms.

3. Provide more local flexibility within the current CSR program by allowing a school-wide average of 20 students in grades K–3.

Along the lines suggested by the LegislativeAnalyst’s Office—and consistent with therecommendations made by the CSR Consortium in previous evaluation re-ports—local districts should be given theflexibility to vary class size by up to twostudents per class as long as the class sizeaverage within a school remains at 20 stu-dents or fewer. Having the class size cap of20 apply to a school rather than to eachclass within a school would give schools a

5


The Consortium is

impressed with the

need to link CSR to

the state’s overall

strategic direction—

i.e., to end its

current status as

a stand-alone

categorical program

by integrating

and aligning it

with the state’s

standards-based

policies.

6

modicum of additional flexibility whileonly modestly affecting the way the limit is applied.

4. Further test CSR’s potential to improve theachievement of low-income/minority studentsby providing additional resources to create and evaluate even smaller class sizes in selected schools.

Based on the evidence from the TennesseeSTAR experiment, it appears that class sizereduction can be an especially effectivestrategy for raising the achievement of themost at-risk students if the classes for thisgroup are small enough and are staffed byskilled and qualified teachers. The stateshould consider conducting carefully con-trolled experiments to examine the differ-ence moving to a class size of 15 or fewerwould make, beginning with those schoolsthat serve the largest number of low-income and minority students.

5. Provide incentives to a small number of districts to experiment so that cost-neutral alternative class size reduction strategies canbe tested and evaluated.

If the state were to allow a relatively smallnumber of school districts to use their CSRfunding to create randomized trials of othersmall class size arrangements, it could com-pare the effectiveness of the current CSRprogram with alternative class size reduc-tion designs. Participating districts wouldbe required to randomly assign schools, orclasses within schools, to the current CSRprogram structure or an alternative model.Researchers could then track the changesin student achievement for the alternatives.The state should also consider allowing dis-tricts to compare one or more non-CSRuses of the funds against the current CSRmodel, again with the requirement thatthis be done using randomized trials. Bothof these options have the virtue of provid-ing information about the cost-effective-ness of the alternatives (since all wouldhave the same cost), something that couldnot be done as part of our evaluation. Themajor incentive to districts to participate

would be the ability to design their classsize reduction or other programs to meetlocal needs. A second incentive would be additional state funds for participating districts for technical assistance in puttingtogether the research design and for evaluating the effects of alternative uses of CSR money.

6. Further explore why and how class size reduction works by identifying best instructional practices in small classes.

Not enough is known yet about the condi-tions under which class size reduction ismost effective in improving studentachievement, which means we can offer little guidance about how to make it workbetter. More research is needed to under-stand which classroom practices are themost effective in small classes and whetherthese differ from best practices in largerclasses. We know that reduced size classeshad some effects on instructional practicein California, but we do not know whattype of changes in classroom teachingwould be needed to maximize the benefitsof the reform.

7. Before undertaking any statewide effort to expand CSR to additional grades, ensure that there are sufficient facilities and qualified teachers.

The state has taken substantive action toremedy the facility and teacher shortageproblems originally created by K–3 CSRand to address these issues more generally.There has been a thoughtful, concerted ef-fort in recent years to establish new policiesrelated to teacher preparation, credential-ing, recruitment, and retention. As to California’s school facilities crisis, signifi-cant progress has been made since CSRwas signed into law. Meanwhile, however,some school districts in California con-tinue to be severely constrained by the capacity of their facilities. It is unclearwhether the state’s efforts will fully eliminate the current problems, much less provide the kind of capacity that would be needed for CSR expansion.


The state should

consider conducting

carefully controlled

experiments to

examine the

difference moving

to a class size of

15 or fewer would

make, beginning

with those schools

that serve the

largest number

of low-income and

minority students.


7

Lessons learned that apply to future education reform efforts

Beyond the specific recommendations wemake related to California’s current CSR pro-gram, we think that the state’s experience pro-vides some broader lessons for policymakers.Whether California embarks on additionalclass size reduction initiatives or undertakesother large-scale reforms, we believe the fol-lowing serve as important guides for develop-ing effective education reform policy:

■ Be clear about where a given initiative fits withinthe state’s overall educational policy plan. It isimportant that state leaders are clear aboutwhat educational problem an initiative is try-ing to solve, and what system of supportsmight have to be in place for it to work.

■ Start small before creating a new, expensivestatewide program. If the state wants to as-sure that its education reform dollars arewell spent, it should consider the use of con-trolled randomized field trials that can beevaluated prior to implementing dramaticreforms statewide. Once we know whatworks and in what situations, and what supports are required, then the reform canbe taken statewide with greater chances of effectiveness.

■ Examine the context carefully prior to imple-mentation. The rapid, massive introduction ofthe CSR initiative had deleterious effects onthe types of teachers hired, classroom space,and financial resources in a number of dis-tricts and schools. Some of those effectscould have been foreseen—and perhaps mit-igated—if state leaders had taken a morestudied approach.

■ Carefully assess the capacity of the state’s edu-cational system to implement change. TheCSR reform strained the capacity of thestate’s educational system at all levels. Forexample, the number of less-than-fully-

qualified teachers went up more than tenfold in the first two years of CSR in California. Further, the districts with thehighest percentage of at-risk students werealso the most likely to get the newest andleast qualified teachers. When initiating anew program, careful consideration is nec-essary to ensure that it does not exacer-bate existing inequities; instead, it shouldhelp decrease them.

■ Match financial resources with student needs.California districts serving the highest percent-ages of urban, low-income, minority, and ELstudents were most likely to take funds thathad been targeted for other educational needsin order to implement CSR. In this sense, someof the better-off districts gained, while the lesswell-off districts were further burdened byCSR. One way to avoid this would be to use afunding formula based on need.

■ Allow some flexibility to accommodate local dif-ferences. Any educational intervention is unlikely to succeed to its full capacity if it does not allow for adaptations to local needsand context.

■ Allow sufficient time to implement large-scale ini-tiatives. The California class size reductionprogram was announced in July 1996 for im-plementation statewide in September 1996.This set off a mad scramble to convert space,obtain equipment, and hire teachers. Therapid start gave a dramatic advantage to dis-tricts that had excess capacity, had a readypool of qualified applicants, were not strug-gling with other administrative issues, andcould redirect resources to this effortquickly. In the future, any large initiativeshould have ample time for districts andschools to plan between passing it into lawand implementing it.

If the state wants

to assure that its

education reform

dollars are well spent,

it should consider

the use of controlled

randomized field

trials that can be

evaluated prior

to implementing

dramatic reforms

statewide.

8. Invest in an enhanced education data system so that the effectiveness of the state’seducation reforms can be better determined.

California’s education data system mustbe redesigned to allow researchers to linkteachers and children with their achieve-ment scores over time in order to bettermeasure student gains from year to year.In this way, the state can more accuratelymeasure the effects of specific reforms onstudent achievement. The creation ofsuch a data system requires an adequateinvestment of time, money, and politicalwill by the state and by local school districts; but ultimately the investmentpromises important returns related to the effectiveness of education reforms.

ConclusionThe results of our evaluation, a changingstate policy context, and new class size reduction research in other states—all of these provide justification for reexamin-ing California’s current class size reductionpolicy. As suggested in our full report, thestate can change some aspects of the CSRprogram without abandoning its commit-ment to smaller K–3 classes. Carefully con-trolled pilot programs can help determinewhat is working and why.


8

What We Have Learned About Class Size Reduction in California

CSR Research Consortium Members and Report Authors

American Institutes for Research (AIR)George W. Bohrnstedt, lead author Jamie Shkolnik

Lawrence W. Gallagher Ed Wiley

Hiroyuki Hikawa

RANDBrian M. Stecher, lead author J. R. Lockwood

Catherine H.Augustine Daniel F. McCaffrey

Dionne Barnes Marika J. Suttorp

Delia Bugliari Georges Vernez

Policy Analysis for California Education (PACE)Gerald C. Hayward, Sacramento

Michael W. Kirst, Stanford University

EdSourceTrish Williams Miriam Rokeach

Mary Perry Jackie Teague

WestEdJoan McRobbie

The CSR Consortium thanks the following funders for their support of this four-year project:

California Department of Education The Walter S. Johnson Foundation

The Walter and Elise Haas Fund The San Francisco Foundation

The William and Flora Hewlett Foundation The Stuart Foundation

EdSource is a not-for-profit 501(c)(3) organization established in California in 1977.

Independent and impartial, EdSource strives to advance the common good by developing and widely distributing trustworthy, useful information that clarifies complex K–12 education issues and promotes thoughtful decisions about California’s public school system.

To learn more about our organization, please contact:

EdSource4151 Middlefield Road Suite 100 Palo Alto, CA 94303-4743650/857-9604 Fax: 650/857-9618

E-mail:[email protected] visit us on the web: www.edsource.org

For data about every school and district in California, visit the Education Data Partnership: www.ed-data.k12.ca.us

To Learn MoreFor a discussion of the latest results fromthe Tennessee STAR experiment, see:“Class Size Reduction in Grades K–3.”Finn, J. In: School Reform Proposals: The Research Evidence. Tempe, AZ: Education Policy Research Unit, Arizona State University, 2002.

For a more extensive bibliography of re-search and readings related to class size,see the references section of the CSR Research Consortium’s capstone reportavailable at: www.classize.org

California Department of Education’sClass Size Reduction web page:www.cde.ca.gov/classsize

Federal Class Size Reduction web page:www.ed.gov/offices/OESE/ClassSize/index.html

report executive summary: what we have learned about ......executive summary: what we have learned...

Documents