ericed 419 275 title institution pub date note available from. pub type edrs price descriptors....

ED 419 275

TITLE

INSTITUTIONPUB DATENOTEAVAILABLE FROM

PUB TYPEEDRS PRICEDESCRIPTORS

ABSTRACT

DOCUMENT RESUME

EA 029 020

Designing and Implementing Standards-Based AccountabilitySystems.Education Commission of the States, Denver, CO.1998-03-0052p.

ECS Distribution Center, 707 17th Street, Suite 2700,Denver, CO 80202-3427 (Stock No. AN-98-1; $10 plus $3postage and handling).Reports Evaluative (142)MF01/PC03 Plus Postage.*Accountability; *Educational Assessment; EducationalChange; Educational Environment; *Educational Innovation;Elementary Secondary Education; Program Development; SchoolAdministration; School Effectiveness; *Standards

To help educators with accountability concerns, this guideprovides a method for establishing and evaluating accountability systems instates, districts, and schools. Standards-based accountability refers tocollecting and reporting information based on students' progress towardachieving established standards. To be useful, an accountability system mustmeet the varying information needs of parents, educators, policy-makers andthe public. Ten policy issues can help establish this information baseincluding: defining purposes and goals; establishing a design andimplementing a process; deciding who will be held accountable and howprogress will be measured; how performance and progress will be compared; atwhat levels data will be collected and reported; administrators should weighthe costs of implementing such a system; creating assessment with rewards,sanctions, and other incentives; helping the public understand results;supporting teachers, schools, and districts; and fine-tuning the system. Twoappendices provide indicators of state accountability systems and a glossaryof terms. (RJM)

********************************************************************************* Reproductions supplied by EDRS are the best that can be made *

* from the original document. *

********************************************************************************

ECS

EDUCATION

COMMISSION

OF THE STATES

U.S. DEPARTMENT OF EDUCATIONOffic of Educational Research and Improvement

ED ATIONAL RESOURCES INFORMATIONCENTER (ERIC)

This document has been reproduced asreceived from the person or organizationoriginating it.

Minor changes have been made toimprove reproduction quality.

Points of view or opinions stated in thisdocument do not necessarily representofficial OERI position or policy.

PERMISSION TO REPRODUCE ANDDISSEMINATE THIS MATERIAL HAS

BEEN GRANTED BY

AA/TO THE EDUCATIONAL RESOURCES

INFORMATION CENTER (ERIC)1

Designing and Implementing

Standards-BasedAccountability Systems

Education Commission of the States707 17th Street, Suite 2700

Denver, Colorado 80202-3427303-299-3600

FAX: 303-296-8332email: [email protected]

www.ecs.org

March 1998

EC S

EDUCATION

COMMISSION

OF THE STATES

3

Designing and Implementing Standards-Based Accountability Systems

© Copyright 1998 by the Education Commission of the States (ECS). All rights reserved.

The Education Commission of the States is a nonprofit, nationwide interstate compact formed in 1965 to help gov-ernors, state legislators, state education officials and others develop policies to improve the quality of education. TheECS office is located in Denver, Colorado.

The New American Schools Development Corporation is a bipartisan, nonprofit organization formed by Americancorporate and foundation leaders. Its mission is to help communities throughout the United States improve theirpublic schools by implementing school improvement designs that the New American Schools' design teams havedeveloped.

It is ECS and New American Schools policy to take affirmative action to prevent discrimination in their policies, pro-grams and employment practices.

ECS is pleased to have other organizations or individuals share its materials with their constituents. To request per-mission to excerpt part of this publication either in print or electronically, please write or fax Josie Cana les, EducationCommission of the States, 707 17th St., Suite 2700, Denver, CO 80202-3427; fax: 303-296-8332.

Copies of this book are available for $10.00 plus postage and handling from the ECS Distribution Center, 707 17thStreet, Suite 2700, Denver, Colorado 80202-3427, 303-299-3692. Ask for No. AN-98-1. ECS accepts prepaid orders,MasterCard, American Express and Visa. All sales are final.

Postage and handling charges if your order totals: Up to $10.00, $3.00; $10.01-$25.00, $4.25; $25.01-$50.00, $5.75;$50.01-$75.00, $8.50; $75.01-$100.00, $10.00; over $100.00, $12.00.

Generous discounts are available for bulk orders of single publications. They are: 10-24 copies, 10% discount; 25-49copies, 20% discount; 50+ copies, 30% discount.

Education Commission of the States/Page iiit


TABLE OF CONTENTS

ACKNOWLEDGMENTS iv

EXECUTIVE SUMMARY

INTRODUCTION 1

UNDERSTANDING STANDARDS-BASED ACCOUNTABILITY 3

POLICY ISSUES 10

Policy Issue 1: Defining Purposes and Goals 10

Policy Issue 2: Establishing a Design and Implementation Process 12

Policy Issue 3: Deciding Who Will Be Held Accountable and How Performance andProgress Will Be Measured 13

Policy Issue 4: Deciding How Performance and Progress Will Be Compared 15

Policy Issue 5: Deciding at What Levels To Collect and Report Data 20

Policy Issue 6: Weighing the Costs 21

Policy Issue 7: Creating Rewards, Sanctions and Other Incentives 22

Policy Issue 8: Helping the Public Understand Results 25

Policy Issue 9: Supporting Teachers, Schools and Districts 27

Policy Issue 10: Fine-Tuning the System 31

CONCLUSION 33

APPENDIX A: Frequently Used Indicators in State Accountability Systems 34

APPENDIX B: Glossary of Terms 36

ENDNOTES 38

ADDITIONAL RESOURCES FROM ECS 40

Education Commission of the States/Page iii


ACKNOWLEDGMENTS

This report, as part of the Education Commission of theStates' work in the dissemination and scale-up of theNew American Schools effort, is funded by a generousgrant from the Annenberg Foundation.

The report was written by Jane Armstrong, director ofpolicy studies for ECS; Michael C. Biance, associateprofessor of educational leadership at Florida StateUniversity; and Edward Roeber, director of studentassessment programs for the Council of Chief StateSchool Officers. The section on rewards and sanctionsrelies heavily on the excellent research of Richard A.King and Judith K. Mathers at the University ofNorthern Colorado.

Education Commission of the States/Page iv

A special thanks to Nancy M. Sanders, ECS commis-sioner and assistant professor of educational adminis-tration at the University of Colorado at Denver; andEllen Still, director of research, South Carolina SenateEducation Committee, who reviewed preliminary andfinal drafts of this report.

Thanks also to the following ECS staff members fortheir assistance in the preparation of this report: JudyBray, state systems coordinator; Josie Canales, commu-nications assistant; Kathy Christie, information coordi-nator; Julie O'Brian, state systems coordinator; RobertPalaich, director of constituent services; Chris Pipho,director of state relations and clearinghouse; SherryFreeland Walker, director of publications; SuzanneWeiss, writer/editor; and Anna West, productioncoordinator.

6

Designing and 'Implementing Standards-Based Accountability Systems

EXECUTIVE SUMMARY

The challenge facing American education is to help allstudents learn at higher levels than ever before. Settinghigh academic standards clearly defined statementsof what students are expected to know and be able to doat various points in their schooling is a crucial partof meeting this challenge. Standards provide policy-makers, educators, parents and the public with themeans to monitor, measure and continuously improvestudent achievement and school quality.

With the use of content standards as the foundation foreducation reform, many state leaders are rethinking thestructure, purpose and emphasis of their states' account-ability systems. Among the major questions policy-makers must address are the following:

Who should be held accountable? For what?

How should student and/or school progress be mea-sured, compared and reported?

At what levels of the system will accountability databe collected, analyzed and reported?

What consequences will be attached to the results?

How can accountability systems be structured to sup-port greater flexibility?

How can public support and understanding of theaccountability system be encouraged?

What is the state role in creating support structuresfor districts?

What Ifs Accountability?Accountability refers to the systematic collection,analysis and use of information to hold schools, educa-tors and others responsible for student performance.Standards-based accountability refers to collecting andreporting information based on student progress onachieving established standards. Elements of anaccountability system include input measures (such asqualification of teachers or dollars spent on education),process measures (such as descriptions of how mathe-matics or science classes are taught) and outcome

7

measures (such as student achievement measures orgraduation rates).

Develloping Coordinated AccountabilitySystemsTo be useful, an accountability system must meet thevarying information needs of parents, educators, policy-makers and the public. To address these needs, state,district and school accountability systems should bedeveloped in a coordinated fashion. The state account-ability system should monitor the progress of schoolsand districts and hold them accountable for helping stu-dents achieve high standards. District accountabilitysystems should collect information related to state anddistrict goals and standards and hold schools account-able for their performance targets. School accountabil-ity systems should focus on collecting achievement datarelated to school, district and state goals while provid-ing diagnostic information for improving student learn-ing and school improvement.

Policy Essues in Designing andIlmpllementing Accountability SystemsPOLICY ISSUE 1: Defining Purposes and Goals

Accountability systems must be designed to achievespecific purposes. These purposes can include:

Monitoring and reporting school and studentprogress on achieving standards

Evaluating the performance of the education system

Holding schools and others accountable

Allocating resources

Certifying or promoting students

Improving student and school performance

Selecting or placing students

Planning staff development.

Education Commission of the States/Page v


The challenge is to design a coordinated accountabilitysystem that emphasizes different purposes at differentlevels state, district, school and classroom basedon the feasibility of data collection, data needs andinformation needs.

POLICY ISSUE 2: Establishing a Design andImplementation Process

States may choose to appoint a working group or createan independent panel charged with the design, imple-mentation and maintenance of the accountability sys-tem. This panel makes recommendations related toissues, such as how performance will be assessed andprogress reported; how accountability results will beused to foster change and improvement; and what struc-tures will be put in place to build local capacity and sup-port school-level accountability. The design processusually begins by determining a set of principles andgoals for the accountability system.

POLICY ISSUE 3: Deciding Who Will Be HeldAccountable and How Performance and ProgressWill Be Measured

Policymakers can choose from a number of differenttypes of student achievement measures, includingnorm-referenced tests, criterion-referenced tests, cus-tomized norm-referenced tests, performance assess-ments or portfolios. Each has distinct advantages anddisadvantages, and costs of administering each can varywidely. In a standards-based accountability system, stu-dent achievement measures that provide accurateassessments of student progress on standards should bethe highest priority. Other indicators also should be col-lected to provide an accurate picture of the system andidentify areas in need of improvement.

POLICY ISSUE 4: Deciding How Performance andProgress Will Be Compared

At the heart of any accountability system is the deter-mination of what constitutes satisfactory performance.Comparing the scores of students, schools or districts isa common way to make this determination. There areseveral approaches to the use of comparisons,including:

Education Commission of the States/Page vi\-

Comparing students or schools to absolute perfor-mance on achieving standards using proficiencylevels

Comparing students' or schools' gain scores orprogress toward performance targets

Comparing schools or students to "expected" scores.

Most state accountability systems report studentprogress using proficiency levels on standards and onhow well schools meet their performance targets.

POLICY ISSUE 5: Deciding at What Levels ToCollect and Report Data

Accountability results can be collected and reported atthe state, district, school or individual student level.The levels of reporting usually are determined by thepurposes of the accountability system as well as costconsiderations. Results most often are reported at theschool level because most state accountability systemshold schools accountable for student results. School-level results, however, can be aggregated to create dis-trict- and state-level results. Some states reportindividual student scores, usually on measures that cer-tify or promote students, and other states only reportdistrict results.

POLICY ISSUE 6: Weighing the Costs

A number of design elements affect the costs ofaccountability systems and the student assessments onwhich they rely. These accountability system featurestend to increase costs: custom-developed tests;, tasks oritems that are "hand scored"; assessing a large numberof items, grades or subject-matter areas; high stakesassociated with test results; rewards for high-perform-ing schools; and the extensiveness of the set of indica-tors collected.

POLICY ISSUE 7: Creating Rewards, Sanctions andOther Incentives

Rewards and sanctions offer the potential for focusingteachers' work, motivating school improvement effortsand improving teaching and learning. Six states havecreated accountability systems that use rewards and


sanctions to encourage schools and districts to increasestudent performance on standards. Rewards can be inthe form of cash, recognition or increased flexibility.Sanctions usually are implemented on a continuum,starting with assistance and possibly ending with theclosing of a school or a "state takeover" if adequateprogress is not made.

POLICY ISSUE 8: Helping the Public UnderstandResults

How the public, policymakers and educators viewand whether they understand the accountability sys-tem has much to do with its success. For parents, teach-ers and the public to see the accountability system asfair, they need to understand how the accountabilitysystem works, what led to specific rewards and sanc-tions being used, and why such actions are likely to beproductive. Finally, educators and policymakers mustwork together to help parents and the public understandnew reporting formats and how a standards-basedaccountability system differs from previous systemsthat may have used norm-referenced tests.

POLICY ISSUE 9: Supporting Teachers, Schoolsand Districts

State efforts to ensure accountability need to focus asmuch on creating the conditions for effective teaching

as they do on holding schools accountable for results.For teachers to be effective, they need to have theknowledge and skills to teach effectively, helping allstudents meet high standards. States can play severaldifferent roles in providing support for educators,including creating standards for licensure/certificationand professional development, organizing professionaldevelopment centers, providing additional resources,developing reform networks or providing direct assis-tance to schools or districts.

POLICY ISSUE 10: Fine-Tuning the System

The success of an accountability system is determinednot only by its design but also by the level of commit-ment policymakers demonstrate to its ongoing imple-mentation and refinement. The task force or panelcharged with implementation of the program shouldconduct an annual survey of schools and/or districts toget feedback on the status of implementation and thesystem's effectiveness; prepare an annual report to thelegislative education committees and state board of edu-cation on the status of program implementation withrecommendations for changes, if needed; and, conducta longitudinal study of the impact the accountabilitysystem has on student performance.

Education Commission of the States/Page vii


INTRODUCTION

"Improving student performance requires a clear picture of what you want to accomplish, acomprehensive measurement system to gauge progress and a commitment to act on theresults to make appropriate changes."

Colorado Governor Roy Romer

The challenge facing American education is to help allstudents learn at higher levels than ever before. Settinghigh academic standards clearly defined statementsof what students are expected to know and be able to doat various points in their schooling is a crucial partof meeting this challenge. Standards provide policy-makers, educators, parents and the public with themeans to monitor, measure and continuously improvestudent achievement and school quality.

States began to promote the use of standards in the late1980s in response to mounting public concern overdeclines in student achievement, as measured bynational and international assessments.' This trendraised doubts about America's ability to maintain thequality of its workforce and compete in the global mar-ketplace. The quality of America's schools came to beseen as more important and less acceptable than everbefore. And it was student performance not simply"seat time" that mattered. The consensus was thatschools should be judged more by "outcomes" (studentachievement) than by "inputs" (resources, facilities,number of advanced degrees among teachers and soforth). Standards would allow the education system tobe judged by, and to be held more accountable for,results.

With the use of content standards as the foundation foreducation reform, many state leaders are rethinking thestructure, purpose and emphasis of their states'accountability systems. Making the shift from a con-ventional, input-focused accountability system to onedriven by standards entails a number of complex designand implementation issues. Among the major questionspolicymakers must address are the following:

Who should be held accountable? For what?

At what levels of the system (state, district, school,classroom) will accountability data be collected, ana-lyzed and reported?

40

How will student and/or school progress be mea-sured, compared, judged and reported?

Will and, if so, to what extent accountabilitydata be used as a basis for rewarding or sanctioningschools or certifying students?

What are the costs and trade-offs of accountabilitydesign features?

How will the accountability system be structured topromote and support greater flexibility and authorityat the local level?

How can understanding of and support for standards-based accountability be strengthened among educa-tors, parents and the public at large?

How will the appropriate state role in supportingschool and district improvement be determined?

How should a coordinated accountability system thatprovides information for policy decisions at all levelsbe designed?

Using This GuideThis guide is designed to heighten policymakers' under-standing of these choices and challenges. It describesthe various ways standards-based accountability sys-tems can be designed and used, and how they can con-tribute to state and local education improvement. Theunderlying premises of accountability systems used inthis guide are:

Academic standards should provide the basis forevaluating student performance and progress.

Schools should be held accountable for studentachievement and student progress toward standards.

Education Commission of the States/Page 1


Standards-based accountability

systems . . . are a vital tool in state

and local education improvement

efforts.


Schools must be provided local authority and flexi-bility to use various instructional approaches to helpstudents achieve standards.

The most useful accountability system is one focusedon motivating and supporting higher performanceand continuous improvement at all levels of the edu-cation system state, district, school and class-room.

The first section of this guide examines in a question-and-answer format the basic definitions, elements andlimitations of standards-based accountability systems.The second section outlines 10 policy issues in thedesign and implementation of such systems, and pro-vides a look at various approaches, design options andtrade-offs, and support structures available to policy-makers. A glossary (see Appendix B) defines termsused in the report.

Developing standards-based accountability systems is adifficult and complex process, but such systems are avital tool in state and local education improvementefforts. When designed thoughtfully and implementedsuccessfully, accountability systems can help more stu-dents achieve at higher levels and help improve theoverall efficiency and effectiveness of public education.


UNDERSTANDING STANDARDS-BASEDACCOUNTABILITY

What fs Accountability?Accountability refers to the systematic collection,analysis and use of information to hold schools, educa-tors and others responsible for the performance of stu-dents and the education system.

Accountability is most effective when it does thefollowing:

Links authority and adequate resources to responsi-bility

Defines clear lines of responsibility and mutualobligations

Involves fair and adequate assessment againstagreed-upon goals

Involves appropriate consequences (both rewardsand sanctions) for observed performance.

How Ifs Accountability Differentfrom Assessment?The concepts of assessment and accountability some-times are confused, since student test results are usuallythe primary element of state indicator systems used tohold schools, or students, accountable. In this docu-ment, accountability systems are defined as: the system-atic collection of input, process and outcome data, aswell as the use of these data, to make decisions aboutthe effectiveness of schools, districts or states.Therefore, accountability systems differ from assess-ment both in terms of the breadth of information col-lected and the manner in which the information is usedin making decisions.

What Standards- asedAccountability?The focus of a standards-based accountability system isto measure success against clearly defined standards. Astandards-based accountability system focuses attentionon student learning rather than on compliance with rulesand regulations. It allows states to move away from reg-ulating schools based on inputs (the number of books inthe library and the proportion of certified staff, for

42

example) and move toward a model of steering byresults using rewards, sanctions and assistance tomove schools toward higher levels of performance.'

To assess student learning, state or local leaders firstestablish standards that define what students shouldknow and be able to do at various points in their school-ing. In some states, such as Delaware, standards weredetermined by a joint effort of state and local leadersand are expected to be used by every school district. Inother states, such as Colorado, the state developedmodel content standards. Districts are expected either toadopt the state standards or develop their own standardsthat "meet or exceed" the state standards. In anothermodel exemplified by Iowa, standards are determinedby each school district.

Performance standards which provide concreteexamples and definitions of how well students mustlearn the material represented by content standardsthen are used to create performance levels that definestudents' demonstrated proficiency_at various_points_asthey progress toward meeting a standard.

In a standards-based system, states focus not so muchon how, but on whether, schools are helping studentsachieve the standards. The emphasis is on holdingschools accountable for student performance rather thanon the methods schools use to achieve results.

What Are the ask Eliementsof a Standards- ased AccountabilitySystem?Accountability systems typically collect a wide varietyof information about districts or schools that can becharacterized as input, process and outcome measures.

Input measures include information such as the qual-ifications of teachers, the dollars spent on education,and the numbers and types of students being served.In a standards-based accountability system, inputscan be used as background information to evaluategains in student achievement and better understandthe environment in which schools are operating.



Process measures describe the types of programsoffered in schools. For example, they might includedescriptions of how science or mathematics classesare taught, what professional growth opportunitiesare offered to teachers and what types of special-edu-cation programs are available for students with dis-abilities.

Outcome measures are used to describe the achieve-ment and accomplishments of students, includinggrades, test scores and the number of students gradu-ating from high school, entering college or becomingemployed.

Standards-based accountability systems focus on stu-dent performance (outcomes), with input and processmeasures used to help explain or interpret the outcomes.States are beginning to use and collect data on multiplemeasures and even accept alternative measureswhen the results are used to make important decisionsrelated to students. Since a variety of types of informa-tion are available about schools, the challenge is forpolicymakers and educators to select from among thenumerous available data elements those that are mostuseful in creating a clear, coherent picture of the out-comes of a school, district or state.

How Can Standards-11 asedAccountability e Used To PromoteEmprovement at Various Level Isof the Education System?To be useful, an accountability system must meet thevarying information needs of parents, educators, stateand local policymakers, and the public.'

Parents want to know what their child knows and cando, while a teacher is more concerned with identify-ing a student's strengths and what instructional chal-lenges are needed.

A school principal needs to know if student achieve-ment is comparable to that in similar schools and howwell students are achieving state and local standards.


District leaders may be more concerned aboutwhether the achievement needs are greater in mathe-matics than reading, so that additional resources canbe allocated for mathematics instruction. Or theymay be concerned that several schools are not meet-ing their performance targets.

At the state level, concern is often focused onwhether there is equity in school programs, whetherdifferences in student performance are due to the lackof resources in some schools, and whether studentshave the knowledge and skills to help the stateremain economically competitive.

To address these various information needs, account-ability systems should be developed in a coordinatedfashion across the multiple levels of the education sys-tem. Using assessments as an example, Figure 1 (page5) shows how the need for information on specific stu-dents expands as assessments move from the state levelto schools and classrooms.

The implications of these information needs for thedesign of state, district, school and classroom assess-ments include the following:

The state accountability system should focus on mon-itoring student progress on achieving state standards,and holding schools and districts accountable forthat progress. State assessments should be alignedwith state content standards to provide targets forinstruction. States may promote the administrationand scoring of performance tasks at the local levelthat are reported to the state and used as part of thestate accountability data.

The district accountability system should focus oncollecting information related to district goals andstandards while holding schools accountable fortheir performance targets. Districts may choose toassess students in grades or academic areas not cov-ered by the state assessment. More performancetasks can be administered and scored by teachers toguide their instruction and provide information onindividual students. District-level data should givelocal policymakers guidance on which school pro-

3


grams are most effective, identify areas whereresources need to be reallocated to support improve-ments, and provide comparisons among schools thatare shared with parents and community members.

School-level accountability should focus on the col-lection of achievement data related to school goalswhile providing diagnostic information for improv-ing student learning and facilitating studentplacement.

At the classroom level, more direct and complexassessments, such as performance tasks, portfolios ormastery exhibitions should be used for student diag-nosis and program improvement.

Table 1 (pages 6 and 7) suggests some purposesand design features of a coordinated, multi-levelaccountability system.

Figure 1

RELATIONSHIP BETWEEN LEVEL OF ASSESSMENTS AND KNOWLEDGEOF SPECIFIC STUDENTS

,*\edge\n_dePIP_I\G swde`"

ok sPe

v4 edgeNo

\oo- Aosspek,

stou

National State District ClassroomAssessments Assessments Assessments Assessments

Source: A Comprehensive Guide to Designing Standards-Based Districts, Schools, and Classrooms by RobertJ. Marzano and John S. Kendall, p. 89. Reprinted with permission from the Mid-Continent Regional EducationalLaboratory (Aurora, CO).



Table 1PURPOSES, DESIGN FEATURES AND USES OF ACCOUNTABILITY SYSTEMS

AT THE STATE, DISTRICT, SCHOOL AND CLASSROOM LEVELS

LevelPurposes of AccountabilitySystem

Primary Design Featuresof an Accountability SystemBased on Purposes

Accomplishing the Purposes:Uses of AccountabilityInformation

State Monitoring system progress

Identifying the relative perfor-mance of one system to another

Holding districts and schoolsaccountable

Providing targets for instruction

Providing information for programimprovement


Certifying students

Indicators could include informa-tion on expenditures, district orschool demographics, programquality

Assessment is aligned to statestandards and assesses only coreacademic standards

At least three grades are assessedone in elementary school, one

in middle school and one in highschool

Assessment item formats are pri-marily selected-response withsome short-answer, extended-response and performance tasksto assess more complex standards

If accountability system is "highstakes," then assessment meetshigh technical and legal criteria

Providing rewards and sanctionsfor high- and low-performing dis-tricts or schools

Revising state policy to improveadequacy and equity

Reallocating resources to wherethey are most needed

Targeting assistance to low-per-forming schools

Providing information to the publicon the status of schools and theeducation system

Endorsing diplomas or providingcertificates of mastery

Focusing teachers' attention onstandards

District Monitoring district progressImproving student performance

Holding schools accountable



Evaluating program effectiveness


Indicators collect information onschool demographics, school-levelexpenditures, school program,teacher quality and parentsatisfaction

Assessment is aligned to districtstandards and may be adminis-tered at every grade or at gradesnot assessed by the state account-ability system

District assessment results arereported to the community andmay be included in the stateassessment results

Assessment item formats includeselected-response and can includemore short-answer, extendedresponse, performance tasks andportfolios to assess more complexskills and variations in schooldesigns while meeting technicalaccuracy

Providing rewards and sanctionsfor high- and low-performingschools (including determining ifschools meet their performancecontract agreements)

Reallocating resources to schoolsthat need additional assistance

Identifying effective programs suchas whole-school designs

Providing comparisons with other(similar) schools in the district

Reporting school-level data to thepublic to promote conversationsrelated to school designs andschool improvement; creating aconsumer guide to district schools

15



Table 1 (Continued)PURPOSES, DESIGN FEATURES AND USES OF ACCOUNTABILITY SYSTEMS

AT THE STATE, DISTRICT, SCHOOL AND CLASSROOM LEVELS

LevelPurposes of AccountabilitySystem

Primary Design Featuresof an Accountability SystemBased on Purposes


School Evaluating program effectiveness

Placing students

Certifying students

Improving student performance

Indicators could include commu-nity characteristics, student demo-graphics, attendance rates,graduation/completion rates, track-ing post-school student perfor-mance, school-level inputs suchas resources allocated to profes-sional development, instructionalmaterials, and degree of imple-mentation of school program

Assessments are aligned withschool goals and school program

Assessment item formats caninclude selected-response, short-answer, extended-response, per-formance tasks, portfolios andmastery exhibitions

Assessment results are availablein time to make improvements

Input for school's strategic plan-ning process

Evaluating how well schools meettheir goalsEvaluating effectiveness of schooldesign for student population andcommunity needs

Input for reallocating resourceswithin the school

Basis for communitywide conver-sations on school improvementGauging parent satisfaction withthe school program

Providing information for district'sconsumer guide to schools

Classroom Student diagnosis


Assessments are aligned withschool program and are appropri-ate for the grade or student group

Assessment results are immedi-ately available to use for studentdiagnosis to improve studentperformance

Providing information on individualstudents for communication withparents

Providing information to place orselect students

Providing information on effective-ness of classroom teaching andlearning



A flexible system recognizes that,

when it comes to educating students,

one size does not fit all.


Why Ifs Fllexibillity a Criticalf Componentof Standards- I ased AccountabillitySystems?If educators and others in the system are to be heldaccountable for student performance, they must havethe authority and responsibility to meet those goals.Because students have different backgrounds and learn-ing styles, schools need to take different approaches tomeet their learning needs. To provide this needed flexi-bility, many educators and policymakers are turning to"whole-school designs." Exemplified by the NewAmerican Schools (NAS) designs, whole-schooldesigns provide different blueprints, approaches orphilosophies to help schools and communities worktogether to help all students meet high standards.

Flexibility, by promoting choices, gives schools moreautonomy to be responsive to parents' wishes and stu-dent needs, gives teachers and administrators a strongersense of purpose and responsibility, creates models ofinnovation and encourages schools to use theirresources more efficiently. A flexible system recog-nizes that, when it comes to educating students, onesize does not fit all.

A flexible system requires an operating environmentthat maximizes schools' ability to help all studentsreach high levels of achievement. First, all schoolsshould have clearly defined standards that reflect highexpectations for students. Multiple forms of assessmentshould be used to measure student progress toward thestandards. Teachers should be provided assistance tohelp them strengthen instruction, align curriculum withhigher standards, build in a variety of ongoing assess-ments to develop a rich picture of student learning andadjust their practice to meet individual students' needs.Finally, schools need the authority to decide how todeploy all available resources money, staff and time

to help students reach high standards.' If schools areto be held responsible for student learning, they needthe authority to make decisions to accomplish this.

When well-designed accountability systems are cou-pled with other policy changes, such as standards-basedreform, site-based management and budgeting, publicschool choice, charter schools and reform networks,


schools are given more authority to produce results, yetare held accountable by parents and community mem-bers. Schools also are held accountable by local schoolboards, which may close schools that do not meetagreed-upon performance goals.

What, Are Limitations of AccountabilitySystems?The biggest limitations of accountability systems arethe quality and quantity of the information reported, andthe alignment of the data to the particular school anddistrict. Are the data current? If the data are one or moreyears old, they may not be pertinent to the school or dis-trict at the current time. Are the data elements related tothe school? For example, a 7th-grade score may reflectmore about the elementary schools feeding the middleschool than the middle school itself.

The data elements also may be limited. For example,student performance is more important in an account-ability system than data about inputs (e.g., expendituresor staffing). It is not uncommon, however, for the latterto be more prevalent than the former types of informa-tion.

There are other limitations to accountability systemsworth noting.

First, much of the information reported as indicatorstypically is obtained from external sources. Forexample, college-entrance test scores, state assess-ment scores, and financial and staffing data may bereported by the state or by national organizations.Hence, they may not fit well with the local schoolsystem and may contain errors as the data are col-lected, edited and reported by others.

Other information that may be available but is notreported might change some of the observationsreported or decisions made. For example, informationthat classroom teachers collect, such as portfolio dataor their own tests, might add another dimension to thereporting of student accountability information. Sincethese data are rarely collected and reported in stateaccountability systems, however, these potentiallyvaluable insights are not reported, and the picture of

18

student performance may be somewhat misleading.Are there other sources of information that can orshould be used?

Second, some important outcomes of schooling arenot easily measured by the indicators selected forreporting. The most valued outcomes of schooling(e.g., gainful employment, successful parenting, sat-isfying life) are not easily measured while studentsare in school or even later in life. Hence, things suchas grades and coursetaking are substituted for thesemore important longer-term outcomes.

Third, achievement data and other indicators aremost often cross-sectional in nature. This means thedata presented at different grades for a particular yearcome from different groups of students. Are changesobserved due to differences in students or differencesin the education system?

Fourth, none of the explanatory factors that may beassociated or "correlated" with student achievementnecessarily account for student performance. Forexample, socioeconomic status and achievement areoften correlated, yet poverty does not cause lowachievement. Since the effective schools researchabounds with high-poverty schools with highachievement, there are clearly other factors, such asteacher expectations or parental involvement in edu-cation, that are at work. The fact that two variablesare statistically correlated does not mean that onecauses the other, and such causal inferences shouldbe avoided.

Users of accountability system information should becautious before putting forth explanations for thecauses for student performance. Is one variable thatis correlated with another actually caused by thatvariable?

Finally, most indicators provided in accountabilityreports contain only one to three years of data. Wherepossible, more data should be provided; trend analy-sis should not be carried out with fewer than threeyears of data. If more data were available, would thesame trends in performance be observed?



POLICY ISSUES

There is no single "right" way to

design and develop standards-based

accountability systems.


There is no single "right" way to design and developstandards-based accountability systems. A state'saccountability system will, and must, reflect the socialand political environment within the state, as well as thestate's goals and objectives.' The experience of severalstates that have developed or are in the process ofdesigning standards-based accountability systems,however, suggests a number of critical steps in the suc-cessful design and implementation of such systems.

POLICY ISSUE 1:Defining Purposes and GoallsAccountability systems can serve several different pur-poses. An accountability system that attempts to per-form too many functions, however, inevitably will donone of them well. Therefore, it is important to identifydesired purposes and to design an accountability systemto serve those purposes effectively. One way to servemultiple purposes is to create a coordinated account-ability system where different purposes are identifiedand met at different levels of the education system.

In some cases, an accountability system designed forone purpose cannot be used effectively for another pur-pose. For example, using a traditional "off-the-shelf"norm-referenced test in an accountability system,although providing student percentile rankings, maynot be effective in assessing student progress towardstate or local content standards. Likewise, limiting datacollection in an accountability system to district-levelsamples may not provide enough information forrewards and sanctions to be applied to individualschools. The worst-case scenario is for policymakers toestablish the structure of a system without first deter-mining the purposes. To do so may mean that multiple,sometimes conflicting, purposes (and messages to edu-cators) may emerge, and that users of the accountabilitysystem may be unclear about how the informationresulting from the system is to be used.

In a standards-based accountability system, the mostobvious purpose is to monitor, evaluate and publiclyreport the progress of students, schools and districtstoward achievement of content standards and otherestablished goals.

19


Other purposes can include:

To provide information for policy decisions.Accountability data can be used to evaluate or for-mulate policy, especially policies focused on equityand resource allocation.

To provide information for program improvement.An accountability system can include the collectionof data on multiple, related indicators and report theresults for classrooms and schools. The results canhelp identify areas of strength and weakness, eitheron an absolute or relative basis, so schools will knowwhich programs or services need improvement.

To evaluate the performance of the education systemrelative to other systems. Sometimes, accountabilitysystems are designed to permit external comparisonsof state, district or school performance with the per-formance of systems in other states or countries.

To hold schools and/or districts accountable.Standards-based accountability systems collectinformation on students' proficiency on standards,allowing school and district scores and reports to beanalyzed and compared.

To allocate resources. Standards-based accountabil-ity systems can be used to help allocate or targetresources, such as providing rewards for high-per-forming schools and assistance for low-performingschools.

To certify or promote students. Some states useaccountability systems to retain students or promotethem to the next level (such as from elementaryschool to middle school) or to certify students forgraduation. When accountability systems are usedfor high-stakes purposes, the measures relied onmust be legally defensible and of high technicalquality.

To improve individual student performance. Theassessment portion of the accountability system canbe used to provide feedback to individual students ontheir strengths and weaknesses and provide informa-tion on how learning might be improved.

An accountability system that

attempts to perform too many

functions, however, inevitably will

do none of them well.



The challenge is to design a coordi-

nated accountability system that

emphasizes different purposes at

different levels.


To select or place students. The assessment portionof the accountability system can be used to select orplace students in various programs based on theirneeds and level of achievement.

To plan staff development. School accountabilityresults can be used to identify curricular or instruc-tional weaknesses for targeted professional develop-ment.

The challenge is to design a coordinated accountabilitysystem that emphasizes different purposes at differentlevels state, district, school and classroom basedon the feasibility of data collection, data needs andinformation uses.

POLICY ISSUE 2:Establishing a Design andImplementation ProcessOnce the central focus and purpose of the accountabil-ity system has been defined, a number of crucial issues

some involving policy choices, others involvingtechnical issues and questions must be addressed.These issues include the following:

What will the unit of accountability be (student,school, district or all three)?

How will performance and progress be measured,compared and reported?

How will accountability data be used to fosterchange and improvement?

What role will local school districts, the state depart-ment of education and other agencies play in theaccountability process?

Addressing these and other important questions in acoordinated, coherent fashion can require the creationof an independent panel charged with the design, imple-mentation and maintenance of the accountability sys-tem. The establishment of such a panel, whosecomposition and responsibilities may be set in statute orby state board of education rule, sends a message ofobjectivity, fairness and importance.

21


Resources for the design and development processshould be identified at the outset. External expertisemay be required, although a great deal of support can beprovided by the state department of education, univer-sities and practitioners within the state, providing addi-tional opportunities for buy-in on the part of variousgroups and individuals. Task forces and ad hoccommittees in such areas as assessment, professionaldevelopment, collective bargaining, technology, post-secondary articulation and parent involvement can beused to extend the accountability panel's resources.

One of the first tasks a panel has is to create principlesof an effective accountability system. This establishes aframe of reference for policy discussions and providesdirection in establishing a knowledge base among thepanel members. The next task is to translate these prin-ciples into purposes, goals and the design of anaccountability system.

In addition, a public information process is needed toinclude the media, as well as the legislative and execu-tive branches, schools, districts and the general public.Regular reports on the progress of design and imple-mentation can help minimize misinformation. Hearingsand other opportunities for public comment during thedesign phase also can help add to the credibility andcoherence of the process.

POLI[CY I[SSUE 3:Decidfing Who WEI II,e Heild Account-abile and How Performance andProgress WEI 11:e MeasuredMost accountability systems focus on holding studentsand schools accountable. But, states are beginning toexpand accountability systems to articulate expecta-tions for more participants in the education process,assess their performance and hold them accountable.

For example, Delaware's new accountability system(see sidebar) plans to hold teachers and administratorsaccountable by requiring them to meet professionalteaching and administrator standards, respectively.These standards include program approval, initial licen-sure, induction process, evaluation and recertification.

4Z

DELAWARE'S PRINCIPLES OF ANEFFECTIVE ACCOUNTABILITY SYSTEM

Accountability should be integrated with profes-sional development so that all students have theopportunity to succeed.

Accountability should be tied to progress towardthe state's academic standards, with students andschools expected to perform at agreed-upon highlevels.

Accountability should be linked to authority.

To the extent possible and practical, all system par-ticipants students, parents, educators, schools,business and the community should be account-able in some way.

Performance should be linked to consequences forindividuals as well as schools, districts and thestate. Schools whose students meet or exceed thestandards should be rewarded. Those that need helpshould receive it. Those that persistently or dramat-ically fail should be penalized.

Progress toward student and professional standardsshould be measured with technically adequate andfair assessment tools.

The accountability system should be easily under-stood by, perceived as fair by and make good senseto the public and educators.

An accountability system must include help for stu-dents having difficulty meeting the requirements.

Accountability must be supported by an informationsystem and analytic capacity that ensure improvedpolicy, programs and practices.

Source: The Missing Link: Connecting ProfessionalDevelopment with Accountability To ImproveStudent Learning in Delaware, 1997.



Other states, such as Tennessee, hold teachers account-able by measuring the gain in student achievement dur-ing the academic year. Delaware also plans to holddepartment of education staff accountable based onemployee performance, program evaluations, customersatisfaction surveys and annual overall gains of the edu-cation system. See Table 2 on page 16 for state account-ability examples of who is being held accountable andfor what.

Student Achievement Measures

Nearly every state uses one or more assessments to col-lect information about student achievement.6 Amongthe types of tests used in state accountability systemsare the following:

Norm-referenced tests. These tests are commerciallyavailable instruments that provide comparisonsbetween individual student or school scores andthose of a nationally representative norm group.Norm-referenced tests are inexpensive to purchaseand score, provide national comparisons and are eas-ier for parents to understand. The disadvantages arethat they rarely provide an exact match to a state ordistrict's content standards, and historically have notincluded performance tasks to drive more complexlearning and instruction.

Criterion-referenced tests. These are typically cus-tom-developed assessments, based on a state's con-tent standards (although credible "off-the-shelf' testsaligned to standards developed at the national levelare beginning to emerge). Selected-response and con-structed-response items are used, and scores arereported relative to a state's performance standards.The advantage is that custom-developed tests arealigned closely with state and district standards; thedisadvantage is that there are more costs associatedwith custom development than an off-the-shelf test.

Customized norm-referenced tests. Occasionally,assessment programs use a version of a norm-refer-enced test to maintain the national comparisonadvantages of the off-the-shelf test, but customize it


to fit the particular state or local content standards.This approach shares the disadvantage of higher test-development costs with criterion-referenced tests, butallows the instruments to be aligned more closelywith state and local standards than conventionalnorm-referenced tests. Another disadvantage is thefrequent lack of "direct" or performance measures.

Performance assessments. Performance assessments,in which students actually produce work (an essay, alaboratory experiment, a painting), have the advan-tage of being closely linked to curriculum and con-tent standards. By connecting what is taught andwhat is tested, these assessments also may influencewhat teachers teach. They are, however, more expen-sive to develop, administer and score.

Portfolios. Portfolio assessment, in which examplesof student work are compiled and evaluated overtime, is another way to collect information about stu-dents. Some states, including Vermont and Kentucky,are using student portfolios as part of their stateassessment system. Portfolios have the advantage ofproviding longitudinal data showing student growthover time. Their primary disadvantages are the timeand resources needed to collect the information andto evaluate it fairly and reliably. If portfolios are usedin high-stakes situations, care must be taken to ensurethat scorers have been well-trained.

Although states may use any of the tests describedabove, it should be noted that standards-based assess-ments (all of the types of tests mentioned above exceptnorm-referenced tests) have several advantages overtheir traditional counterparts.' These advantages includethe following:

Standards-based assessments are closely linked tocontent and curriculum standards. By connectingwhat is taught and what is tested, assessments influ-ence what teachers teach. Standards-based assess-ments compare student accomplishment topreestablished goals, rather than to the performanceof other students.

23


o They can incorporate new forms of assessment. Suchassessments can require students to undertake"authentic" tasks, such as writing an essay or con-ducting a hands-on science experiment. Such assess-ments capture a broader range of complex thinkingand problem-solving skills.

o They model tasks for teachers' professional develop-ment. Performance tasks serve as examples teacherscan use in their classrooms. Teachers' scoring of per-formance assessments also provides an opportunityto discuss standards and performance expectationsfor students.

It is possible for states to include multiple types ofassessments in an accountability system. Maryland, forexample, uses criterion-referenced tests and perfor-mance tasks in its primary state assessment, but offersdistricts the option of using norm-referenced tests on alimited basis to provide comparisons to national norms.

In addition to student assessments, accountability sys-tems typically include a broader set of indicators. Theseindicators are usually reported in school report cards orschool profiles. Indicators used by the states are catego-rized in Appendix A.8

Some indicators, such as student and family character-istics, may explain differences in performance betweenschools or districts. Other indicators, such as curriculumor advanced courses offered, are considered to be underthe direct control of teachers, administrators or localschool boards and may indicate the aspirational level ofstudents or the school. Both types of indicators shouldbe included in an accountability system.

The challenge for policymakers is to identify and mea-sure the outputs that are most valued (rather than justthose most available), and to collect information on theinputs and processes needed to provide an accurate pic-ture of the system and identify areas in need ofimprovement.

POLICY IISSUE 4:Deciding How Performance andProgress Win Iue ComparedAt the heart of any accountability system is the deter-mination of what constitutes satisfactory performance.Comparing the scores of students, schools or districts isa common way to make this determination. There areseveral approaches to the use of comparisons:

Comparing students or schools to absolute perfor-mance. States frequently report student- or school-level data in terms of the percentage of students whomeet or exceed a proficiency level on each standard.For example, the scores of 8th graders on a mathe-matics computational test might be reported as 25%"partially proficient," 65% "proficient" and 10%"advanced." Reporting absolute scores sends a mes-sage that the state's goal is to have all students,regardless of background characteristics, perform athigh levels.

o Comparing students' or schools gain scores. Gainscores can be determined in several ways. Using alongitudinal data-collection approach, differencesbetween scores of the same individual students arecompared as they move through different grade lev-els. For example, a student's score on a mathematicstest taken in 7th grade is compared with the same stu-dent's score on an equivalent test taken in 8th grade.School scores are the average gain made by matchedstudents in a school.

Longitudinal data collection is more expensive becausethe same students must be identified and tested, andresults matched and compared. The advantage, how-ever, is that longitudinal data indicate the "value added"of the schooling experience. Tennessee's assessmentprogram uses this approach.

Gain scores also can be computed using cross-sectionaldata-collection approaches. For example, the percent ofproficient 8th-grade students in a given school one year



Table 2WHO SHOULD BE HELD ACCOUNTABLE? FOR WHAT? USING WHAT MEASURES?

WITH WHAT SUPPORT? SOME POLICY OPTIONS

Who ShouldBe HeldAccountable? For What?

Using WhatMeasures?

With WhatConsequences?

Needing WhatSupports?

Legislature Adequate and equitableschool fundingPolicies aligned tostandardsLegislation for account-ability system

Report card of votingrecordsPolicy review

Constitutionality of fundingformulaRe-election

Adequate state resourcesfor educationPolicy options to supportstandards

State Boardand StateDepartmentof Education

Creation of anaccountability system anddata collectionStandardsPublic reporting of districtand/or school-levelaccountability resultsStatewide studentachievement gainsIdentification of lowperforming schoolsEffective technical assis-tance to schools/districts

Student achievementmeasuresOther indicatorsCustomer satisfactionsurveysProgram evaluationsEmployee performance

Election/appointmentprocessBoard reconstitutionLegislative actionPerformance-based com-pensation of employeesProbation, reassignmentor dismissal of staff

Adequate funding ofaccountability programStudent achievementmeasures that are of hightechnical quality andaligned to standardsAdequate resources andstaff expertise for techni-cal assistance to schoolsand districtsMechanism to useaccountability results forsystem improvementAn engaged public

TeacherEducation

Programs meet teacherand administrator programapproval guidelinesPrograms meetNCATE/NASDTECstandardsTeacher education appli-cants meet K-12 profi-ciency standardsTeacher education gradu-ates meet professionalteaching standards

Student competencymeasuresNCATE/NASDTECaccreditationMeasures of professionalteaching standards

Loss of teacher educationprogram accreditation orapproval

Redesign of teacher edu-cation courses to supportstandards-based systemCourse instruction thatmodels diverse instruc-tional strategies

Local SchoolBoards

Meeting professional stan-dards for school boardsCreating local policies thatsupport a standards-based systemCreating districtwideimprovement planMeeting district perfor-mance targetsConducting public hear-ings on accountabilityresults

Measures of board pro-fessional standardsLocal policy reviewMeasures of meeting per-formance targetsDistrict quality reviews

Election/appointmentprocessBoard reconstitution

Adequate state policyframeworkAdequate state and localresourcesCentral office staff able toimplement programs andpoliciesQualified teachers



Table 2 (Continued)WHO SHOULD BE HELD ACCOUNTABLE? FOR WHAT? USING WHAT MEASURES?

Who ShouldBe HeldAccountable?

WITH WHAT SUPPORT? SOME POLICY OPTIONS

Needing WhatSupports?For What?

Using WhatMeasures?

With WhatConsequences?

Districts/Schools

Student proficiency onacademic standardsMeeting performancetargetsAdequate inputsAdequate processesMeeting rules andregulations of stateboardDistrict and schoolimprovement plans

Student achievementmeasuresMeasures of indicatorssuch as dropout, atten-dance and graduationratesAccreditation measuresProgram review

Public reporting of resultsRewardsDeregulationSanctionsRequired assistance

Adequate resources andauthorityLocal policy frameworkthat supports standardsimplementationCurricula aligned to stan-dardsAccess to technical assis-tance and school designsSchool/communityprocess to set goals,review, act on results

Principals/Administrators

Student proficiency onacademic standardsMeeting professionaladministrator standardsAdministrator evaluation

Student achievementmeasuresMeasures of other indica-tors such as school anddistrict goalsAdministrator certificationmeasuresProfessional review

Initial certification andevaluationLicensure or certificationRecertificationSalary bonusesRemediation and supportProbation, reassignmentor dismissal

Professional developmentopportunitiesAdequate resources andauthorityAccess to technical assis-tance and school designs

Teachers Student proficiency onacademic standardsProfessional teachingstandardsTeacher attendance

Student achievementmeasuresMeasures of other indi-catorsTeacher certificationmeasures such as Praxisor INTASC measures

Teacher evaluation tiedto student achievementRecertificationDifferentiated staffingrolesPay for knowledge/skillsSalary bonusesRemediation and supportProbation, reassignmentor dismissal

Professional developmentopportunitiesAdequate resourcesAligned curriculumAccess to technical assis-tance and school designs

Students Proficiency on academicstandardsMeeting course orgraduation requirements

State and local assess-mentsExit/end-of-courseexamsTranscriptsGPA, other indicators

Promotion to next gradeor levelTest scores placed onpermanent transcriptsHigh school graduationDiploma endorsementSpecial recognitionParticipation in extracur-ricular activitiesRequired remediation

Opportunity to learnQualified teachersHigh quality curriculaRemediation programs

Parents Parent participation andvolunteerismCommunication betweenhome and schoolParenting skills andstudent support

Indicator measuringparent involvement inschools

Parent recognitionSchool choice

Parent engagement andinvolvement in schools

Community/BusinessLeaders

Hiring based on studenttranscriptAllow parents time off toattend school conferencesSupport education-relatedeventsPartner with schools

Survey to identify busi-nesses that use highschool transcripts inhiring processIndicator measuringbusiness involvement inschools

Legislation to requireparents release time forschool eventsIncentives for hiringpractices

Business engagementand interest in education

Source: Many ideas in this chart are drawn from Accountability: Blueprint for Delaware, October 1997.



DEFINITIONSDERST DING

TERMSCOMP

FORARISONS

Absolute performance. The percent of studentsreported by the proficiency level they meet or exceed

regardless of their starting point, past performanceor demographics.

Cross-sectional data. Comparison scores of studentsat a given grade level with different students at thatsame grade the prior year.

Expected score. The anticipated score for a student,school or district based on the correlation of back-ground factors with student achievement to derive ascore. This derived score then can be compared withactual student, school or district performance.

Longitudinal data. Data collected from the same stu-dents (or a cohort) over time. Longitudinal data cantake into account statistical determinants of achieve-ment such as socioeconomic status and family back-ground and are suited to determining the "valueadded" over time.


can be compared with the percent of proficient 8th-grade students from the same school the, prior year.Although cross-sectional comparisons are lessexpensive to assess, student mobility and differences inthe student cohorts from year to year can make theresults misleading.'

"Gain scores" reward schools that make progress, evenlow-performing schools, by recognizing and encourag-ing continued growth and improvement. Using gainscores instead of absolute performance gives everyschool a chance to earn a reward.

Another way to use gain scores is to create an "expectedscore" based on prior performance, taking into accountthe impact of socioeconomic or other background fac-tors. For example, anticipated scores of this year's 4thgraders, calculated from last year's 3rd-grade test, arecompared with actual scores. This method is a statisti-cal compromise of using absolute and relative stan-dards, in that each group of students serves as its owncomparison group.

Comparing students using percentile rankings onstandardized tests. Comparing students using per-centile ranks allows districts to use different norm-referenced tests for accountability purposes. InArkansas, for example, schools are allowed tochoose a state-approved norm-referenced test. If40% of students are at or below the 25th percentileon the approved test, the school is placed in the firstphase of sanctions. One problem of schools or dis-tricts using different tests is that it is nearly impossi-ble to aggregate data upward to provide a state-levelsnapshot of student progress toward standards.

Pros and Cons of Comparison Methods

Using absolute comparisons gives preference to histor-ically advantaged, high-achieving schools. Otherschools that serve students in low socioeconomic com-munities may make large gains in student achievement,yet remain below performance standards. But, someeducators and policymakers believe schools should notbe rewarded or penalized for factors they cannot con-trol. They believe it is fairer either to compare improve-ment against schools or school districts with similar

Designing .and. Implementing Standards-Based Accountability Systems

socioeconomic and demographic characteristics, or toadjust student or school scores to account for the typesof students served.

Controlling for student background or prior achieve-ment, however, can institutionalize low expectations forpoor, minority and/or low-achieving students. In effect,this practice holds schools with large proportions ofsuch students to a lower standard of performance. '° Acompromise solution is to use both approaches, reward-ing schools for a high level of absolute performance, aswell as rewarding schools making significant progress.This approach would focus attention and effort on com-petition among all schools, while still permittingschools to strive for rewards relative to others in asimilar situation."

Setting Performance Standards, Cut Scores andTargets for Improvement

Performance standards usually are set by panelsappointed at the state or district level (depending onwhere the standards are set). These panels review stu-dent work reflecting a range of abilities and determinethe level at which that work should be classified.'Panels typically make judgments individually, discusstheir rationales and then, aided by statistical programsthat convert their judgments into proposed scores, con-sider the implications in light of actual performancedata. Then a series of "cut scores" is established, allow-ing student performance on the assessments to be con-verted into proficiency levels. These levels, such as"proficient," "advanced" and "partially proficient," areused for reporting purposes to identify the percentageof students in each category.

States or districts may want to set targets for improvedperformance. Kentucky has one approach to settingperformance targets for schools. A score is assigned toeach level of proficiency on standards. The state's fourcategories of performance "distinguished," "profi-cient," "apprentice" and "novice" carry scores of140, 100, 40 and 0, respectively. Kentucky's goal is tohelp each school average a score of 100 possible ifall students performed at the proficient level or, forexample, if 50% were proficient, 30% were

A compromise solution is to use

both approaches, rewarding schools

for a high level of absolute perfor-

mance, as well as rewarding schools

making significant progress.



distinguished and 20% were apprentice (.5x100 +.3x140 + .2x40 = 100).

Under this formula, schools can achieve progress byincreasing the percentage of "distinguished" students orby reducing the share of "novices." So, the performancetarget for each school in Kentucky is determined by thegap between the school's score and the state goal of100. Kentucky wants all schools to reach the goalwithin 20 years, so a school is expected to reduce thegap between where it is and where it is expected to beby 1/20th each year. Most states and districts set annualtargets for gains between 2% and 5%, but the number ofyears allowed to close the gap varies considerablyamong states.

OLICY ISSUE 5:eciding at What Levels To Collect

and sport ataAccountability results can be collected and reported toallow comparisons at the international, national, state,district, school or individual student level. The levels ofreporting usually are determined by the purposes of theaccountability system, as well as by cost considerations.

International or National. Recent state-by-state com-parisons conducted by the National Assessment ofEducational Progress (NAEP) have allowed states tocompare student achievement in mathematics andreading with other states based on NAEP-developedcontent objectives. Similarly, results from the ThirdInternational Mathematics and Science Study(TIMSS) compare U.S. student achievement withresults from other countries. States wanting to maketheir own comparisons to other countries in mathe-matics and science could have the TIMSS test admin-istered with a state sample. Another option is forstates with similar standards to establish consortia todevelop some common accountability measuresallowing cross-state comparisons.

State. In the 1980s, a number of states reported stu-dent achievement only at the state level. The intent ofthis kind of system was to provide low-stakes moni-toring information on how well the education system


as a whole was doing. Patterned after NAEP, it wasan inexpensive way to provide monitoring informa-tion because of the relatively small number of stu-dents that had to be tested. Many of these states,however, eventually added data collection at the dis-trict and school levels so they could provide reportsfor parents, teachers, administrators and students.

District. Collecting and reporting accountability dataat the district level can identify district-level progresstoward standards and other established district orstate goals. States such as Arkansas and SouthCarolina use the district as the unit of accountabilityand provide rewards and sanctions based on district-level results. One advantage of using the district asthe unit of accountability is that it can reduce costs bykeeping data collection relatively limited (more stu-dents have to be assessed on more variables forschool-level reporting). Another advantage is thatthis approach provides districts flexibility to chooseassessments aligned to their standards and curricu-lum. The disadvantage of using districts as the report-ing unit is that the performance of schools within adistrict differs, and few districts have put into placeaccountability structures or can describe school-improvement targets for a given year.

School. Most states use the school as the reportingunit for their accountability systems. In any district,some schools will be performing better than others.Reporting only district-level results masks these dif-ferences. School-level reporting holds each schoolaccountable for results, encouraging school staffs andtheir communities to work together to improve stu-dent learning. It also allows the district or state to tar-get remedial assistance, as well as rewards andsanctions.

Classroom. Most district reporting mechanismsinclude the classroom as one level of reporting.Classroom data can provide information on the rela-tive effectiveness of a teacher over time and of thecurriculum or program used.

Student. One reason for testing all students at one ormore grade levels is to certify students for promotion

2


or graduation. In most states, this data-collectionstrategy is used only at certain grades, although fre-quently all students in that grade are tested. Abouthalf of the states use individual student data to pro-mote students from grade to grade or level to level,to award a high school diploma or to endorse a highschool diploma. The remaining states assess studentsprimarily as a means of guiding individual studentremediation or reviewing and improving the school'sinstructional program:

A coordinated accountability system can ensure thatinformation for all levels of the system is available foruse by policymakers, educators and communitymembers.

POLICY ISSUE 6:Weighing the Costs,A number of design elements affect the costs ofaccountability systems and the student assessments onwhich they rely. Attention should be given to the poten-tial costs of various design elements, including thefollowing:

Custom-developed tests vs. "off -the- shelf" or com-mercially available tests. A custom-made test thatmatches a state's standards is more expensive todevelop than an 'off-the-shelf' norm-referenced test.

Selected-response-only vs. "mixed" assessments.Assessments that require a mix of short-answer orextended written responses with selected-response(e.g., multiple choice) exercises can cost two or threetimes as much as machine-scored tests to score. Theyalso may take more time to administer than selected-response-only assessments.

Number of items assessed. The more items tested, thegreater the administration and scoring costs. Somestates deal with this issue by "matrix sampling" testitems. In a matrix sample, each student takes a por-tion of the item set, and statistical techniques areused to create school or district scores.

`4(

A coordinated accountability system

can ensure that information for all

levels of- the system is available for

use by policymakers, educators and

community members.



Number of grades assessed. The greater the numberof grade levels tested, the higher the cost of adminis-tration, scoring and reporting. One way to reducecosts is to test fewer grade levels, particularly if morethan one type of test is used. For example, a stan-dards-referenced assessment could be used at grades4, 7 and 10 in mathematics and reading, while a com-parable measure in science and social studies couldbe administered at grades 5, 8 and 11. This approachis less costly than assessing students in all gradesfrom 4-11.

High stakes associated with test results. High-stakestests must be of higher technical quality in order to belegally defensible. In addition, for security reasons,new forms of the assessment will need to be createdannually, since the stakes are too high toreuse past tests or test items.

Extensiveness of the set of indicators to be collected.While there may be a desire to be as complete as pos-sible in selecting the indicators to be reported alongwith student achievement data, the larger the set to becollected and reported, the greater the cost. Someelements may already be collected by the state or bydistricts, so the costs of including these data elementsmay be minimal. Other desirable data elements maynot be currently available and will require additionalresources to collect, analyze and report. Some dataelements (for example, the employment of studentsafter high school) may be deemed virtually impossi-ble to collect on a statewide basis.

Size of rewards provided to schools or districts.Indiana most recently awarded over $3 million toschools; South Carolina awards about $5 million. In1995, Kentucky had $26 million available for teacherbonuses. The resources associated with providingrewards should be considered part of the costs of anaccountability system.

The costs of collecting and reporting on each desireddata element will need to be weighed, including thestaff time and effort needed to obtain information fromschools and districts. Either the direct or indirect costsassociated with data collection may limit the extent to


which various data elements are included in theaccountability system. In general, more expensiveapproaches will produce sounder accountability sys-tems. But, with finite resources for education, the trade-offs between spending resources on an accountabilitysystem versus putting the resources into classroominstruction must be weighed.

POLICY ISSUE 7:Creating Rewards, Sanctionsand Other IncentivesRewards and sanctions offer the potential for motivat-ing school improvement efforts and improving teachingand learning (see the ECS publication, A Policymaker'sGuide to Incentives for Students, Teachers and Schools,for more information about incentives). Progresstoward achieving standards, along with other indicators,can be the basis for incentives for high performance anddisincentives for poor performance. And, includingother incentives in the system, they also can serve toimprove motivation and performance:3

Six states Georgia, Indiana, Kentucky, NorthCarolina, South Carolina and Texas have createdrewards and sanctions to encourage schools and dis-tricts to increase student performance on standards."How progress is measured and reported has significantimplications for how rewards are allocated and sanc-tions imposed, and how the public views the fairness ofthe accountability system. Among the questions policy-makers need to address are the following:

o Will there be absolute standards that must be metregardless of the school's baseline level of perfor-mance and factors specific to that school?

How should low-scoring but improving schools berewarded?

Should consistently high-scoring schools berewarded for their high level of performance in thesame way as low-scoring but improving schools?

® What assistance will the state provide to help low-performing schools improve?


o Who will impose sanctions when schools fail toimprove? What is the scope of the sanctions? Willthere be a due-process procedure? What will be thecost of imposing the sanctions?

o What will the rewards be and who will receive them?Will the rewards encourage teamwork or will theycause dissension and low morale?

If a state chooses to attach rewards or sanctions toaccountability results, the accountability system is con-sidered "high stakes" (see the ECS publication,Education Accountability Systems in 50 States for moreinformation about state accountability systems, includ-ing the use of incentives and sanctions). If conse-quences are to be attached to the results, theaccountability system needs to be technically andlegally defensible: it must measure what it says it mea-sures and must do so accurately and with multiple mea-sures. This step is especially important whenaccountability results are used to make decisions aboutindividual students, such as promotion, graduation oran endorsed diploma.

Rewards

Rewards can come in the form of cash, recognition orincreased flexibility. They may include cash awards forschool improvement or to be distributed as staffbonuses, freedom from regulations, or statewide recog-nition and status. States that provide cash rewards maylimit how schools spend the money. Some states allowthe reward money to be used for staff bonuses, althoughKentucky, for one, is reconsidering this approach.'5Other states ban the use of reward money for staffbonuses and athletics. Still others require rewardbonuses to be used only for school improvementefforts.

Nonmonetary awards can be given in the form of relieffrom regulations and thus increased flexibility. InTexas, a school rated "exemplary" is exempt fromrequirements and prohibitions specified in the stateschool code. The school is still held accountable, how-ever, for a lengthy list of code sections, including cur-riculum and graduation requirements, elementary

324. LLD

How progress is measured and

reported has significant implications

for how rewards are allocated and

sanctions imposed, and how the

public views the fairness of the

accountability system.



class-size limits and programs for students who are dis-abled, bilingual or at risk.'

South Carolina grants broad exemptions to qualifyingschools. A school must have received an incentiveaward twice over a three-year period in addition to otherrequirements. Flexibility status earns release fromrequirements of the defined minimum program.Qualifying schools also are freed from regulations gov-erning class scheduling, class structure and staffing."Exempt schools must continue to participate in the stateassessment program and score well to retain deregu-lated status.

In some states, such as Indiana, monetary awards aresupplemented with noncash awards or public recogni-tion. The use of such awards (called "Four Star" schoolsin Indiana) originated with the desire to recognize high-performing schools that did not earn awards based ongrowth scores. Four Star schools are those that performin the top quartile of schools in each of four indicatorsand satisfy all expected performance levels defined inIndiana's school accreditation program. The Four Stardesignation recognizes high-capacity districts, while atthe same time allowing a larger amount of money to goto poorer school districts that improve.

Sanctions

Sanctions usually are implemented on a continuum,starting with assistance and possibly ending with clos-ing a school or the state taking it over if adequateprogress is not being made. Twenty states have enacted"academic bankruptcy" laws that allow state interven-tion in severely troubled districts.' Early responses toacademic bankruptcy can include such things as publichearings or the assignment of onsite technical assis-tance, as is available under Kentucky's DistinguishedEducator Program. If progress is not made, the possibil-ity exists of a state takeover of the district and otherforms of intervention. Other options include changingmanagement of the district (school board or schoolsuperintendent) or school (principal and teachers),allowing parents to choose another school or providingadditional resources. Most states have accountabilitymechanisms focused on identifying low-performing


schools. Some states, however, also impose sanctionson districts with a large number of low-performingschools.

States are exercising considerable caution as theyimplement sanction programs. The problem with high-stakes accountability systems is they must be backed upwith technically accurate and legally defensible assess-ments and procedures. To be conservative, many states,such as Maryland, are identifying only a handful ofschools generally agreed to be at the lowest level of stu-dent performance.

Florida's accountability program was developed overseveral years before 1996 legislative action gave thestate superintendent authority to intervene in the opera-tion of districts that failed to make adequate progress:9Delaware may allow the use of an alternative assess-ment and other indicators instead of the state assess-ment results to report student results using proficiencylevels.

Do Rewards and Sanctions Work?

With the exception of South Carolina, state reward andsanction programs have been in effect for only a fewyears. Early evidence suggests, however, that sanctionsdo motivate schools to do better. But the results aremixed regarding the impact of rewards.

Richard A. King and Judith K. Mathers,2° in an analysisof four states with high-stakes accountability systems,made the following conclusions regarding the effect ofrewards on teachers' motivation:

Extrinsic rewards may contribute to reform imple-mentation but may not motivate change.

Extrinsic factors pale in comparison with the intrin-sic aspects of teachers' work.

State and local attention to improvement efforts hasas great an incentive effect as the money associatedwith such status.

Regarding the effect of rewards on schools and class-rooms, these same researchers concluded the following:

A


Changes in classroom practice are present in bothreward and nonreward classrooms in the four states.

Leadership and other factors may be more criticalthan monetary rewards in change efforts.

Rewards and sanctions may stimulate conversationswith professionals and parents on school improve-ment.

More attention is given to racial/ethnic minoritygroup students in states in which rewards and sanc-tions take such students' performance into account.

The impact of rewards on improving student test scoresis more difficult to ascertain. Few studies are availablethat include findings on the effects of rewards on stu-dent achievement.

Some state leaders believe the mere threat of sanctionsis a more powerful motivator than the distribution offinancial rewards. Local school leaders want to avoidthe negative publicity associated with low performanceor the stigma attached to being designated a "school indecline." But sanctions do seem to work. Kentuckyreports that most of the schools targeted as being "indecline" in 1994 appear to be making enough progressto ensure that more extreme sanctions are not imposed.2'In more extreme cases, however, such as state takeoversor the reconstitution of districts, the long-term effects ofsanctions are less clear.

Cautions for High-Stakes Testing

High-stakes accountability systems can produce someunintended, negative consequences. These can includethe narrowing of instruction, de-emphasis on untestedsubject areas, test-score inflation through inappropriateor questionable test preparation practices, and staffmorale problems."

Research shows that in a high-stakes system" what getstested gets taught, and that the converse is also truewhat is not tested is not taught. High-stakes tests thatassess student performance on only one or two contentareas reading and mathematics, for example willhave an adverse affect on how much science or social

studies is taught. Similarly, accountability systems thatconsistently assess only some standards within a con-tent area likely will result in limited instruction on thestandards not assessed.

Inflation of test scores was first noted in a 1987 studythat showed that many states had an implausible pro-portion of districts reporting themselves to be "abovethe national average."" Test-score inflation can be dueto many factors, including making comparisons usingoutdated norms, and teaching to the test.

Staff morale problems can occur when rewards areselectively awarded, such as one state's recent policy ofproviding cash rewards only to principals of high-performing schools.

POLICY IISSUE 8:Elellping the Puha Understand ResuiltsHow the public, policymakers and educators viewand whether they understand the accountability sys-tem has a lot to do with its success. One- challenge iscommunicating clearly the results of the accountabilitysystem to policymakers both in terms of the reportcards and the system evaluation. A second challenge isto ensure that parents, teachers and the public see theaccountability system as fair. To achieve this, thesegroups need to understand how the accountability sys-tem works, what led to specific rewards and sanctionsbeing used, and why such actions are likely to beproductive.

State leaders report that parents are confused about thedifference between the new tests that report studentprogress toward standards and the familiar norm-refer-enced tests in which a single score gives a percentilerank against a national norm." Standards-basedaccountability systems place students in categories suchas "partially proficient," "proficient" or "advanced"based on their assessment scores. Parents may notknow what it means to be "partially proficient" and maybe more comfortable with percentile-rank comparisonsthan with assessment of learning progress over time.



COMMUNICATING NEW ASSESSMENTRESULTS TO PARENTS

To assist educators in explaining the new state assess-ment results to parents and the public, CONNECTColorado's Statewide Systemic Initiative, and a busi-ness coalition, Teaming for Results, created ateacher's guide to the Colorado Student AssessmentProgram.

To explain the difference between criterion-referencedtests and norm-referenced tests, Colorado uses the fol-lowing example: Picture a group of students climbinga mountain. A norm-referenced test tells whether agiven student is in the lead, near the middle of thegroup or lagging behind. It will not show where onthe mountainside (near the top, middle or bottom)each student is only their relative position to oneanother. The state assessment, on the other hand, isbased on the state model content standards (a crite-rion-referenced test) and provides student achieve-ment information based on proficiency levels.Proficiency levels show how well a student is meetingthe state standards, or, in this case, where the studentis located on the mountain.

Source: How Are Colorado Students and SchoolsMeasuring Up?: A Teacher's Guide to the ColoradoStudent Assessment Program. CONNECT andTeaming for Results, 1997.


Issues such as low scores, the consequences of testresults and costs sometimes arise and will need to beexplained."

Many of the "lessons learned" about involving the pub-lic in establishing standards and developing new assess-ments also apply to the design and implementation ofstandards-based accountability systems. They includethe following:

Public involvement is inseparable from policy devel-opment and implementation.

Listen to people first, talk later.

If you do not involve teachers, expect to fail.

Show how new approaches enhance rather thanreplace old methods.

Expect criticism and respond to it, but do not becomepreoccupied with it.

Understand that it takes time to get results, and setexpectations accordingly.

Demystify reforms. Do not just tell people about newstandards, assessments and accountability processes;show them examples.

Communicate the big picture to the public whilereserving technical discussions for teachers and oth-ers directly involved in implementation.

Be willing to adjust plans based on the needs andconcerns of educators, parents and the public.

Build a strategic communications plan designedaround people's questions and needs, and use itthroughout the design and implementation process(see the ECS publications, Building CommunitySupport for Schools: A Practical Guide to StrategicCommunications, Do-It-Yourself Focus Groups: ALow-Cost Way To Listen to Your Community, So YouHave Standards . . . Now What? and Listen, Discussand Act: Parents' and Teachers' Views on Education

1:,,,7


Reform for more information on communicationsplans and efforts).

POLICY ISSUE 9:Supporting Teachers, Schoollsand DistrictsStates can play several different roles in providing sup-port for educators, including creating standards forlicensure or certification and professional development,organizing professional development centers, providingadditional resources, developing reform networks, orproviding direct assistance to schools or districts.

Ensuring Professional Accountability

State efforts to ensure accountability need to focus asmuch on creating the conditions for effective teachingas they do on holding schools accountable. Raising aca-demic expectations for students will work only if teach-ers have the knowledge and skills to make gooddecisions and teach in ways that help all students meethigher levels of achievement. Increasing accountabilityshould be coupled with investments in improvedteaching.

The recent report of the National Commission onTeaching and America's Future" noted the failures ofrecent reforms are due not to schools' unwillingness toimprove, but to the facts that most educators do notknow how to implement the reforms, and their workenvironments are not structured to help them do so. Ifincreased student achievement is the goal, it will needto be coupled with accountability policies that ensureteachers and other educators have the knowledge andskills they need to teach effectively to the new stan-dards and help schools evaluate and reshape theirpractices."

To ensure teachers have the knowledge and skills theyneed to teach to the new standards, many states areputting in place teacher licensure or certification stan-dards. These can include what constitutes effectiveteaching and what teachers should know and be able todo (such as knowing core content, adapting teaching fordiverse learners, using multiple assessment strategies,using a variety of instructional strategies, etc.).

3

State efforts to ensure accountability

need to focus as much on creating

the conditions for effective teaching

as they do on holding schools

accountable.



The impact of professional teaching standards can beextended by state support of ongoing teacher profes-sional development. This could include licensing teach-ers based on performance tests of subject-matterknowledge, teaching knowledge and teaching skill tiedto standards developed by the Interstate New TeacherAssessment and Support Consortium. Another option isto create mentoring programs for beginning teachersthat include assessment of teaching skills as the basisfor a continuing license. Standards and assessments ofthe National Board for Professional Teaching Standardscould be the standard for accomplished teaching and thebasis for rewarding high levels of knowledge and skill.

Supporting Professional Development

States play a number of roles in supporting professionaldevelopment.29 In addition to taking the lead in devel-oping professional standards for teacher preparation,certification or licensure, and professional develop-ment, states are providing assistance and resources forprofessional development opportunities. States such asDelaware and Ohio have developed regional profes-sional development centers providing training and sup-port for teachers. Other states, such as Utah, appropriate2% of the Minimal School Program for state develop-ment activities at the local level and $1 million forstatewide preparation activities.

Utah encourages districts to create a staff developmentpolicy, designate a staff development coordinator, alignstaff development activities with improvement goalsand evaluate the programs. The Utah Department ofEducation also publishes an annual listing of summerinservice activities conducted by the state and otherproviders such as higher education institutions and localdistricts. Vermont involves teachers in the scoring of itsstate assessment program as yet another approach toprofessional development. Regional networks of teach-ers are organized to provide training on portfolio scor-ing and instructional training to help students createportfolios. Many states also support mentor teacher pro-grams ranging from providing competitive grants to dis-tricts to create mentor programs to developingacademies that train experienced teachers and adminis-trators to become in-school consultants.


States also are providing professional development toadministrators, usually through administrator acade-mies. To lead instruction that supports students achiev-ing high standards, administrators also must understandhow to help teachers with instructional approaches, howto use data for student learning and school improve-ment, and how to share decisionmaking with teachersand school community members.

Promoting Local Accountability

School-level accountability can include a number ofdifferent strategies. First, schools need student achieve-ment and other information to evaluate their effective-ness. District accountability systems should collect dataon important goals and in grades and subject areaswhere the state accountability system does not collectinformation.

States can support the collection of additional datathrough their state assessment programs by requiringand supporting the administration and scoring of localassessment data, with a sample of the results reported tothe state and included in the state accountability data.For example, North Carolina plans to place numerousperformance tasks and scoring guides on a World WideWeb site which all teachers can access. Teams of edu-cators from each district will score student portfolioseach fall; the results will inform classroom instructionand be part of a local accountability system.

Schools may want to implement their own accountabil-ity systems such as requiring a student exhibition forgraduation and setting up school-quality review com-mittees. Mastery exhibitions require students to demon-strate their knowledge in front of review committees.This public demonstration of knowledge allows teach-ers and the community to talk about the meaning ofachieving standards and the quality of student work.Constructive debates can help teachers interpret schoolgoals and develop effective strategies for achievingthem, which can lead to staff seeking new knowledgeand skills.

School-quality reviews can be another strategy to fosterlocal accountability. Teams of practitioners, possibly

3-7


including state department of education staff, spend aweek in individual schools to examine teaching andlearning practices. Feedback is provided to fosterimprovement, and the school-quality reviews also canresult in descriptive evidence of how well the school isdoing. This evidence can become a baseline for improv-ing teaching and learning practices over the next severalyears.

States can support local accountability by creatingexpectations that individual schools establish their ownstandards for performance and reporting systems. It cansupport staff-development opportunities to help teach-ers within a school formulate performance goals andways to implement them. Districts and states can estab-lish and support reform networks to assist in sharingbest practices and procedures for evaluating studentlearning and school goals and, most important, actingon the results.

Professional Development Centers

A number of states have taken a more active role in sup-porting or providing professional development. In addi-tion to rethinking certification or licensure standards forteachers, several states are creating professional devel-opment centers."

Ohio has 12 regional professional development centersfor teacher training and development. These centerscreate and coordinate professional development oppor-tunities and assist district staff with school improve-ment strategies. These centers also serve as a regionalstructure for many state programs such as school-to-work and tech prep. The centers build collaborationamong school districts, institutions of higher educationand private providers, and link resources and expertiseto schools and districts.

Delaware also has a state-supported teacher centerwhich grew out of collaborative efforts among severalschool districts. The center now manages eight regionalprofessional development centers which provide inser-vice workshops, free instructional materials, on-loanequipment and a place and opportunity for teachers toshare ideas about best practices.

38

Providing Additional Resources

Several states provide additional resources for schoolsor students who are achieving below grade level. Forexample, Alabama provides funds to schools for tutor-ial assistance for students one grade level or morebehind the norm. Indiana provides grants to helpschools provide remediation for students who scorebelow state proficiency standards and prevent futureremediation of students at risk of falling below the stan-dards.

In 1996, Delaware authorized $200,000 for studenttutoring and mentoring activities, distributed competi-tively to K-5 schools to establish programs for studentsacademically at risk. Programs must address one-on-one tutoring, early-childhood preventive interventionstrategies, parental involvement, and program and stu-dent performance evaluation.

Mississippi has established' a Support Our Students pro-gram within the Department of Education to awardgrants to community and neighborhood organizationsthat provide high-quality after-school programs for chil-dren in grades K-9 and comprehensive delivery of ser-vices to those children.

New Jersey runs nontraditional high schools in each ofits 21 districts, most on college campuses. At-risk stu-dents get extra support services and one-on-one coun-seling to help with academic and behavior problems.Each student's program is tailored to his or her individ-ual needs and may even include enrolling in collegecourses.

Ohio has taken an innovative approach to supportingschool improvement. The Venture Capital program pro-vides "risk capital" of up to $25,000 for each of fiveyears to help schools improve. Venture Capital grantsare designed to be long-term, evolving efforts focusedon a particular dimension of change, e.g., curriculumdevelopment, professional development, assessment.After the five-year time frame, schools are expected tohave made significant progress in institutionalizingtheir commitment to professional development andtransforming the culture in which school renewal isimplemented.



Reform networks can help educa-

tors, schools and districts learn

about approaches to instruction,

curriculum design, assessment,

professional development and

management strategies that have

proved effective in other settings,

and assist them in adapting these

approaches to their own

circumstances.


Reform Networks

Reform networks are interconnected groups of educa-tors, schools or districts with a common interest in aspecific reform approach.3' Some networks focus on aspecific subject, such as reading or mathematics, whileothers target specific topics, such as assessment, liter-acy or minority student achievement. Still other net-works offer a focused approach to reform andrestructuring through the use of a specific "whole-school" design that links all aspects of the school or dis-trict curriculum, instruction, assessment, stafftraining and school management. Other networks relyon more formal structures and processes, including acentral staff that provides teachers and school leaderswith the information and expertise they need toimprove their knowledge and skills in specific ways.

Reform networks can help educators, schools and dis-tricts learn about approaches to instruction, curriculumdesign, assessment, professional development and man-agement strategies that have proved effective in othersettings, and assist them in adapting these approaches totheir own circumstances. Networks provide a widerange of opportunities for professional growth andenrichment by engaging members in varied activities

curriculum workshops, leadership institutes, intern-ships, conferences, study groups, electronic bulletinboards designed around participants' needs andinterests.

Some states that have enacted academic bankruptcylaws have found networks to be a useful tool forimproving performance of chronically low-achieving ormismanaged schools. In several states, includingCalifornia, Maryland and New Jersey, a school identi-fied as failing may be required to affiliate with a reformnetwork as part of its turn-around plan. States also maytake an active role in serving as a clearinghouse or bro-ker for networks.

Providing Direct Assistance

Kentucky's Distinguished Educator program is perhapsthe best known example of a state-directed programproviding assistance to low-performing schools.Created as part of the Kentucky Education Reform Act,

3.9


Distinguished Educators are selected from a pool ofapplicants to serve as teaching ambassadors to schoolsdesignated "in decline" or those that request the extraassistance. At the same time, Kentucky adopted theSchool Transformation Assistance and Renewal(STAR) project for assisting schools "in decline."STAR provides resources to schools and defines thechange process Distinguished Educators use to helpimprove low-performing schools. The majority ofschools with Distinguished Educators have improvedstudent achievement such that they were no longer des-ignated as "in decline."

POUCY ISSUE 1©Fine-Taming the SystemThe success or failure of an accountability system isdetermined not only by its design but also by the levelof commitment policymakers demonstrate to its ongo-ing implementation and refinement. Without such sus-tained commitment, an accountability system may beperceived as just another program imposed by the state.

It is important, for example, that proposed legislationbe carefully monitored to ensure consistency with theaccountability design. The state accountability panel ortask force should be alert to initiatives that not onlywould be inconsistent and detrimental, but also wouldsend a message to the education community that com-mitment to the accountability system is waning.Similarly, state education department and other agen-cies' policies must be reviewed for potential obstaclesto implementing the accountability system.

Other Key Issues To Consider

Standards and assessments should be updated andrefined. As skill needs change, content standardsshould be adjusted to reflect these, as well as theassessments that measure them.

The state's budget must be aligned to the account-ability design, not only within the education portionof the budget, but also in other areas such as humanservices and health (for example, pre-K programs,juvenile justice and safe-school programs).

40)

The success or failure of an

accountability system is determined

not only by its design but also by the

level of commitment policymakers

demonstrate to its ongoing

implementation and refinement.



The state education department's relationship withthe accountability panel, as well as its responsibili-ties for implementation, must be clearly defined. Thedepartment's implementation plan must have a deliv-ery system capable of providing the necessary tech-nical assistance. It also must have the ability todisseminate information in a consistent, clear andeffective manner.

Training should be a high priority. Beyond the initialtraining in how the accountability system will work,there will be a need for ongoing training of teachers,principals, superintendents, school board members,university professors and school advisory councilmembers.

Attention must be paid to the role of postsecondaryinstitutions. Not only will they be the recipients ofstudents held to the state's standards, they also willproduce the teachers who must have the skills toteach to the standards.

The business community needs to be involved.Business community involvement may be in the formof technical expertise at the school or district level;influence in shaping state policy; and active, ongoingsupport of the state's standards.

Newspaper editorial boards should be kept up-to-date on the development and implementation of the


accountability system. If they are not kept within theloop, visibility of the accountability efforts will belessened, and the potential for misinformation will beincreased.

The state accountability panel should have responsi-bility for ongoing evaluation and refinement of theaccountability system, including the following:

Conducting an annual survey of schools and dis-tricts (this could be a random sample) to providefeedback on the status of implementation, theextent and effectiveness of the supporting infra-structure, and the identification of obstacles toimplementation

Preparing an annual report to the legislative edu-cation committees and state board of educationon the status of implementation

Annually reviewing the design and structure ofthe accountability system and making recom-mendations to the legislature on statutorychanges, to the state board on rule changes and tothe state department of education on administra-tive policy changes

Conducting a longitudinal study of the impact ofthe accountability system on student perfor-mance.

41


CONCLUSION

"A clear picture of what you want to accomplish, acomprehensive measurement system to gauge progressand a commitment to act on the results to make appro-priate changes" those were Colorado Governor RoyRomer's requirements for improving student perfor-mance. Content and performance standards areintended to provide the "clear picture" of what needs tobe accomplished. A standards-based accountability sys-tem is the comprehensive measurement system togauge progress. Demonstrating the "commitment toact," by supporting schools and districts with profes-sional development opportunities, by engaging all stu-dents, and by securing broad public support andinvolvement that is the challenge that remains.

42Education Commission of the States/Page 33


APPENDIX A

Level

FREQUENTLY USED INDICATORS IN STATE ACCOUNTABILITY SYSTEMS

Purposes of AccountabilitySystem

Primary Design Features ofan Accountability SystemBased on Purposes


State Monitoring system progress

Identifying the relative perfor-mance of one system toanother

Holding districts and schoolsaccountable

Providing targets forinstruction

Providing information for pro-gram improvement


Certifying students

Indicators could includeinformation on expenditures,district or school demographics,program quality.

Assessment is aligned to statestandards and assesses onlycore academic standards.

At least three grades areassessed one in elementaryschool, one in middle schooland one in high school.

Assessment item formats areprimarily selected-response withsome short-answer, extended-response and performancetasks to assess more complexstandards.

If accountability system is "highstakes," then assessment meetshigh technical and legal criteria.

Providing rewards and sanctionsfor high- and low-performingdistricts or schools

Revising state policy to improveadequacy and equity

Reallocating resources to wherethey are most needed

Targeting assistance to low-per-forming schools

Providing information to thepublic on the status of schoolsand the education system

Endorsing diplomas or providingcertificates of mastery

Focusing teachers' attention onstandards

District Monitoring district progress


Holding schools accountable




Indicators collect informationon school demographics,school-level expenditures,school program, teacher qualityand parent satisfaction.

Assessment is aligned todistrict standards and may beadministered at every grade orat grades not assessed by thestate accountability system.

District assessment results arereported to the community andmay be included in the stateassessment results.

Assessment item formatsinclude selected-response andcan include more short-answer,extended-response, perfor-mance tasks and portfolios toassess more complex skillsand variations in school designswhile meeting technicalaccuracy.

Providing rewards and sanctionsfor high- and low-performingschools (including determining ifschools meet their performancecontract agreements)

Reallocating resources toschools that need additionalassistance

Identifying effective programssuch as whole-school designs

Providing comparisons withother (similar) schools in thedistrict

Reporting school-level data tothe public to promote conversa-tions related to school designsand school improvement; creat-ing a consumer guide to districtschools

Education Commission of the States/Page 34 43

Designing' and Implementing Standards-Based Accountability Systems

APPENDIX A (Continued)

Level

FREQUENTLY USED INDICATORS IN STATE ACCOUNTABILITY SYSTEMS

Purposes of AccountabilitySystem

Primary Design Features ofan Accountability SystemBased on Purposes


School Providing targets for instruction


Placing students

Certifying students


Indicators could include com-munity characteristics, studentdemographics, attendance rates,graduation/completion rates,tracking post-school studentperformance, school-level inputssuch as resources allocated toprofessional development,instructional materials anddegree of implementation ofschool program.

Assessments are aligned withschool goals and school pro-gram.

Assessment item formats caninclude selected-response, short-answer, extended-response, per-formance tasks, portfolios andmastery exhibitions.

Assessment results are availablein time to make improvements.

Input for school's strategic plan-ning process

Evaluating how well schoolsmeet their goals

Evaluating effectiveness ofschool design for student popula-tion and community needs

Input for reallocating resourceswithin the school

Basis for communitywide conver-sations on school improvement

Gauge parent satisfaction withthe school program

Provide information for district'sconsumer guide to schools

Classroom Student diagnosis


Assessments are aligned withschool program and are appro-priate for the grade or studentgroup.

Assessment results are immedi-ately available to use for studentdiagnosis to improve studentperformance.

Provide information on individualstudents for communication withparents

Provide information to place orselect students

Provide information on effective-ness of classroom teaching andlearning

44 Education Commission of the States/Page 35


APPENDIX B: GLOSSARY OF TERMS

Accountability. Accountability systems provide infor-mation that tells policymakers, the public and othershow well the education system classrooms, schoolsand districts is doing. Information typically includesstudent assessment data and indicators such as dropoutand graduation rates. Accountability information can beused in different ways: to provide information to thepublic, to help all the groups involved reach agreementon how to improve the system, or to provide rewards orsanctions for success or failure.

Accreditation. Accreditation is the review process aschool or district undergoes periodically to ensure thatit meets state requirements and quality measures. Theprocess usually involves the review of a district orschool's evaluation procedures and improvement plans;the effectiveness of education programs and services;and other policies, practices and managementprocesses. Recently, some states began using accredita-tion as a performance-based process focused on studentachievement and school-improvement planning.

Assessment. Assessment is the measurement of what astudent knows and is able to do, usually expressed interms of progress toward a standard or mastery of astandard. Assessment can include diverse measures,such as multiple-choice tests, constructed-responseexercises, performance measures and portfolios.

Content standards. Content standards are statementsthat define what students should know and be able to doin various subject areas and at different points in theireducation.

Criterion-referenced test. This test is designed to pro-vide a clear picture of what a student knows and can do.It measures performance against an established criterionor standard, rather than in comparison to a norm group.

Cross-sectional data. Cross-sectional data comparescores of students at a given grade level with differentstudents at that same grade the prior year.

Expected score. The anticipated score for a student,school or district based on the correlation of back-ground factors with student achievement to derive a


score. This derived score then can be compared withactual student, school or district performance.

Indicator system. The system of measures of inputs,processes and outcomes used to describe the perfor-mance of a school, district or state.

Longitudinal data. Data collected from the same cohortof students over time. Longitudinal data can take intoaccount statistical determinants of achievement such associoeconomic status and family background and aresuited to determining "value added" by a teacher orschool.

Norm-referenced test. This test is designed to comparea student's test score with those of students in a norm orcomparison group, such as a nationally representativesample. More generally, it has come to refer to compar-isons of performance among students, schools, districtsor states.

Performance assessments. Performance assessments,also called "authentic" or "direct" assessments, measureactual, demonstrated skills Many times, these are con-structed or written responses to questions on an assess-ment.

Performance standards. Performance standards pro-vide explicit definitions and concrete examples of howwell students are expected to learn the material repre-sented by content standards. Performance "levels" alsomay be used to define students' demonstrated profi-ciency at various points as they progress toward a stan-dard.

Portfolios. Portfolios are collections of student workdesigned to show progress over time, level of accom-plishment and/or the student's best work.

Stakes. Stakes are the consequences tied to perfor-mance on an assessment or test. A "low-stakes" test hasfew or no consequences tied to results; a "high-stakes"test has consequences related to performance. Stakescan include rewards for high performance, sanctions forlow performance, promotion or graduation. The

45


rank-ordering of schools or districts when test resultsare publicly reported can be considered "high stakes."

Value added. The amount of student achievement "con-tributed" by a teacher or school during a school year orother set period of time. A school's contribution to

students' achievement is the "value added" by theschool. How much schools add to students' learning canbe calculated by assessing the differences in fall-to-spring (or fall-to-fall or spring-to-spring) testing of thesame students.



ENDNOTES

1. Diane Ravitch, National Standards in AmericanEducation: A Citizens' Guide (Washington, DC: TheBrookings Institution, 1995).

2. Richard F. Elmore, Charles H. Abelman and SusanFuhrman, "The New Accountability in StateEducation Reform: From Process to Performance,"in Holding Schools Accountable, Helen F. Ladd,Editor (Washington, DC: The Brookings Institution,1996).

3. Edward D. Roeber, Designing CoordinatedAssessment Systems for IASA Title I (Washington,DC: Council of Chief State School Officers, 1997).

4. New American Schools, Accountability for ResultsWorking Group, Shared Responsibility for Results:A Statement of Purpose (Washington, DC: NAS,1996).

5. Education Commission of the States, EducationAccountability Systems in 50 States (Denver: ECS,1997).

6. See, for example, American Federation of Teachers,Standards Matter 1997 (Washington, DC: AFT,1997); Council of Chief State School Officers andNorth Central Regional Educational Laboratory,State Student Assessment Program Database(Washington, DC: CCSSO, 1996).

7. Education Commission of the States, A Policy-maker's Guide to Standards-Based Assessment(Denver, CO: ECS/CRESST, 1997).

8. Mary Sinclair and Ron Erickson, "Identification ofPotential Educational Indicators," in MinnesotaEducational Accountability Reporting System,Volume II (University of Minnesota, 1997).

9. Daniel Koretz, "Using Student Assessments forEducational Accountability," in ImprovingAmerica's Schools: The Role of Incentives, Eric A.Hanushek and Dale W. Jorgenson, Editors(Washington, DC: National Research CouncilNational Academy Press, 1996).


10. Elmore, Abelman and Fuhrman.

11. David Cohen, "Standards-Based School Reform:Policy, Practice and Performance," in HoldingSchools Accountable, Helen F. Ladd, Editor(Washington, DC: The Brookings Institution,1996).

12. ECS, A Policymaker's Guide to Standards-BasedAssessment.

13. Education Commission of the States, Clearing-house Brief "Academic Bankruptcy" (Denver,CO: ECS, 1997).

14. Richard A. King and Judith K. Mathers, ThePromise and Reality of Rewards for SchoolImprovement (Greeley, CO: University of NorthernColorado, 1996).

15. "Kentucky's Teachers Get Bonuses, But Some AreCaught Cheating," Wall Street Journal (September2, 1997).

16. King and Mathers.


18. ECS, Clearinghouse Notes "AcademicBankruptcy."

19. See Florida Commission on Education Reform andAccountability, Florida's System of SchoolImprovement and Accountability, 1996(Tallahassee, FL: Florida's Commission onEducation Reform and Accountability, 1996, andHB1009 enacted by the 1996 Florida legislature).



22. Koretz.

23. L.B. Resnick and D.P. Resnick, "Assessing theThinking Curriculum: New Tools for EducationalReform," in Changing Assessments: Alternative

4 7


Views of Aptitude, Achievement and Instruction, B.R. Gifford and M.C. O'Connor, Editors (Boston,MA: Kluwer Academic Publishers, 1992).

24. Koretz.

25. Education Commission of the States, So You HaveStandards . . . Now What? (Denver, CO: ECS,1997).

26. ECS, So You Have Standards . . . Now What?

27. National Commission on Teaching and America'sFuture, Doing What Matters Most: Investing inQuality Teaching (New York, NY: NationalCommission on Teaching and America's Future,1997).

28. Linda Darling-Hammond, Creating Policy forAccountable Schools (Unpublished manuscript).

29. Consortium for Policy Research in Education,Policies and Programs for Professional Develop-ment of Teachers: A 50 State Profile (University ofPennsylvania: CPRE, 1997).

30. CPRE.

31. For more information on reform networks, seeEducation Commission of the States, A Policy-makers' Guide to Education Reform Networks,Denver, CO: ECS, 1997).



ADDITIONAL RESOURCES FROM ECS

Here are some other related publications from ECS thatyou'll find useful for getting citizen input and makingdecisions about education reform:

Accouintabffity

Education Accountability in 50 StatesHow does your state's accountability system comparewith other states? This publication defines the compo-nents of a performance-based education accountabilitysystem and then shows what each state's system lookslike. Tables illustrate which states have standards andassessments, multiple indicators, rewards and sanc-tions; whether they are based in statute or regulation;how decentralized states compare to centralized states;and other information. 1997 (SI-97-12), 23 pp., $7.50

A Policymaker's Guide to Incentives for Students,Teachers and Schools

The goal of dramatically raising student achievementcannot be accomplished without changing the incen-tives for students, teachers and schools. This guide isdesigned to heighten policymakers' understanding ofthe choices and challenges related to creating incentivesfor these groups. It identifies the elements for improv-ing performance in a standards-based system; describesincentives for students, including using standards forpromotion and admissions to higher education or entryinto apprenticeship programs; highlights incentives toincrease teachers' knowledge and skills; describesincentives for schools, including reallocating resources,boosting achievement though school awards and imple-menting extracurricular programs; and finally, looks atimplications for state policymakers in creating incen-tives. 1997 (AN-97-5), 44 pp., $10.00

Accountability State and CommunityResponsibility

A decade ago, the idea that states could, or would, stepin and intervene in local school districts because stu-dents were not achieving enough was almost unthink-able. Now, however, state intervention not only isacceptable among a majority of the states, but is becom-ing a major policy tool to deal with seriously under-achieving schools or districts. Policymakers shared"lessons learned" at a recent ECS conference in Ohio;


this paper summarizes that discussion. 1998 (SI-98-2),20 pp., $6.50

CommunicationsBuilding Community Support for Schools: A PracticalGuide to Strategic Communications

Learn how to build credibility, trust and stronger ties tothe community by involving the community as changeis considered not after the fact. This guide focuses onthe practical tools you need to develop strategic com-munications with education colleagues, parents and thegeneral public. A step-by-step plan will help you setmore thoughtful priorities, spend scarce resources morewisely, listen and respond to the needs and concerns ofpeople inside and outside the education community, tar-get activities to those who are most affected by and con-cerned about education improvement efforts, andregularly measure results. 1997 (AN-97-3), 32 pp.,$10.00

Do-It-Yourself Focus Groups: A Low-Cost Way ToListen to Your Community

What is important to people in the school community?What changes do they want to see in their schools, andhow do they want to participate in decisionmaking?Learn how to use the valuable information that informalconversation often yields. This guide helps you gatherinformation in a systematic way, organize what youhear, report back what you've learned to the communityand, most important, use that information to guideschool change. 1997 (AN-97-2), 31 pp., $10.00

America's Public Schools Must Change . . . But CanThey?

Half the voters in the last national election say their votewas affected by a candidate's stand on public education.How important is education in selecting political candi-dates? What role do people expect national leaders toplay in improving education? And what kinds of changedo people want to see in public schools? These andother questions are answered in a detailed report of anational survey ECS conducted following theNovember 1996 general election. Includes charts anddemographic breakdowns. 1997 (AN-97-1), 26 pp.$10.00

49


Let's Talk About Education Improvement

Everyone has an opinion about education and they wantto talk about it. You can help foster this dialogue andinvolve all segments of your community in discussionsabout how public schools can do a better job of educat-ing students. This conversation guide offers tools forplanning and holding a meeting, discussion activities,follow-up and take-home materials, and lists additionalresources. 1996 (AN-96-3), 27 pp., $10.00

Listen, Discuss and Act: Parents' and Teachers' Viewson Education Reform

What do the people closest to students think of educa-tion improvement efforts? Any communication aboutchanges must start by listening to the people mostaffected by the changes. This report details the results ofparent surveys and parent and teacher focus groups inseven cities and states. Includes recommendations forbuilding public and political support for educationimprovement. 1996 (AN-96-2), 36 pp., $10.00

Listen, -Discuss and Act: Focus Groups AboutImproving Education

A powerful video companion to the report Listen,Discuss and Act: Parents' and Teachers' Views onEducation Reform. Hear what parents and teachers sayabout standards, parent involvement, assessment andaccountability, and what they think gets in the way ofefforts to improve schools. 1996 (AN-96-V2), 19 min.,$10.00

Jargon `AWARD -WINNER

Winner of a Telly Award for out-standing video, this video provides alight-hearted look at a serious subject:

how the language used by educators and educa-tion policymakers often obscures meaning and under-mines support for reform. 1995 (SM-95-V4), 5:14 min.,$8.00

Education ReformThe Progress of Education Reform: 1997

This year's report examines research data on studentachievement, public attitudes and demographic trends

with a brief interpretation of what the data means.Major education reform efforts and state actions underway are analyzed as to their effectiveness in improvingstudent achievement. 1998 (SI-98-1), 51 pp., $12.50

A Policymaker's Guide to Education ReformNetworks

Get the jump on the new federal money for educatorsinterested in affiliating with a reform network! Thispublication tells you the kinds of networks available,how they work, and the benefits and services they offer.An easy-to-read-and-compare chart provides a thumb-nail sketch of some major education reform networkmodels. 1997 (SI-97-11), 40 pp., $10.00

The State Education SystemA Continuum of Systemic Change

What constitutes systemic change? Where are you as astate in the change process? This chart describes differ-ent stages of systemic change in five key policy areas:vision, policymaking and alignment, teaching andlearning, public and political support, and networks andcollaboration. An effective dialogue tool for policy-makers and lay audiences. 1993 (SI-93-1), wall chart(18 x 24 inches), $2.00

Creating a Flexible, High-Performance Education System

This set of charts provides descriptions of developmen-tal stages for an education system that is moving from aregulatory system to one that encourages and supports adiverse set of schools. This tool describes the state pol-icy changes necessary to create the flexibility neededfor a high-performance system. Five key areasaddressed include: standards, curriculum, assessmentand accountability; decentralizing decisionmaking tothe school level; professional development; resourcereallocation; and public engagement. This tool can pro-vide a basis for dialogue, self-assessment and actionplanning to move the system toward desired goals. 1996(SI-96-10), 13 pp., $5.00

SERIES



StandardsA Policymaker's Guide to Standards-Led AssessmentLearn how standards-led assessments closely link whatis taught to what is tested. The helpful guide examinesthe important challenges of building consensus, assur-ing accurate measures, estimating costs, definingprogress, addressing legal challenges and building pub-lic support. (This guide is jointly published with theNational Center for Research on Evaluation, Standardsand Student Testing.) 1997 (SI-97-3), 27 pp., $10.00

So You Have Standards . . . Now What?

With standards in place or being developed in nearlyevery state, next on the horizon are new assessmentsthat measure progress toward achieving the standardsand make it possible to target improvement effortseffectively. This guide provides tips and strategies onhow to involve educators, the public and parents indeciding on new forms of assessment. Includes a list ofconcerns the public is raising about assessments andtips for meeting the policy and communicationschallenges ahead. 1997 (SI-97-2), 29 pp., $10.00

Stages of Implementationof Standards-Led EducationThis wall chart describes the developmentalstages of an education system implementing standards-based reforms. It describes how the system will shift inkey areas such as public engagement, standards, assess-ment, curriculum and instruction, professional develop-ment, colleges and universities, and equity. This tool isespecially effective in promoting dialogue and self-assessment among educators, administrators, policy-makers and the public. (It is the shortened version of thepolicy tool described below.) 1997 (SI-97-9), wall chart(24 x 22 inches), $5.00

SERIES

Stages of Implementationof Standards-Led EducationThis is similar to the wall chart described abovebut includes more in-depth information and three addi-tional policy areas technology, system diagnosis andplanning, and roles and responsibilities. This tool also iseffectively used by groups to promote dialogue onwhere a school, district or state is on a continuum ofchange and provides a description of what the next levelof implementation will look like. 1996 (SI-96-8), 16pp., $5.00

To order one of the above publications, please call 303-299-3692.

5 IEducation Commission of the States/Page 42

Education

4ECS

EDUCATION

COMMISSION

OF THE STATES

Commission cong StatesWCW IMG] auce12, ad:RD E4K0

Denver Colorado 80202-3427303-299-3600

GM.3 303-296-8332Guaen ecs.orghttp://www.ees.org

52

(9/92)

U.S. DEPARTMENT OF EDUCATIONOffice of Educational Research and Improvement (OEM)

Educational Resources Information Center (ERIC)

NOTICE

REPRODUCTION BASIS

IC

This document is covered by a signed "Reproduction Release(Blanket)" form (on file within the ERIC system), encompassing allor classes of documents from its source organization and, therefore,does not require a "Specific Document" Release form.

This document is Federally-funded, or carries its own permission toreproduce, or is otherwise in the public domain and, therefore, maybe reproduced by ERIC without a signed Reproduction Releaseform (either "Specific Document" or "Blanket").

ericed 419 275 title institution pub date note available from. pub type edrs price descriptors....

Documents