document resume ed 367 664 author bond, linda; and others · 2014-05-07 · document resume. tm 021...

ED 367 664

AUTHORTITLE

INSTITUTION

SPONS AGENCY

PUB DATECONTRACTNOTEAVAILABLE FROM

PUB TYPE

EDRS PRICEDESCRIPTORS

IDENTIFIERS

ABSTRACT

DOCUMENT RESUME

TM 021 101

Bond, Linda; And OthersSurveying the Landscape of State EducationalAssessment Programs. EdTalk.Council for Educational Development and Research,Washington, D.C.; National Education Association,Washington, D.C.

Office of Educational Research and Improvement (ED),Washington, DC.[93]

RP91002001-1058p.

The Council for Educational Development and Research,2000 L Street, N.W., Suite 601, Washington, DC 20036($5) .

Reports Evaluative/Feasibility (142)

MFOI/PC03 Plus Postage.Academic Achievement; Databases; Data Collection;*Educational Assessment; Educational Change;Educational Improvement; Educational Objectives;Elementary Secondary Education; Evaluation Methods;Interviews; Models; National Surveys; SchoolDistricts; Standards; State Legislation; *StatePrograms; *Student Evaluation; *TeacherParticipation; *Test Construction; *TestingPrograms

Performance Based Evaluation; *Reform Efforts

This publication explores the relationship betweenstate systemic reform initiatives and state student assessmentprograms using a database established by thle North Central RegionalEducational Laboratory and interviews with state assessment directorsin California, Indiana, Maryland, Michigan, Texas, and Vermont.States throughout the country are actively pursuing systemiceducational reform. The survey and interviews reveal that statesapproach the development of new or revised assessments in thefollowing ways: (I) developing totally new assessments to accompanynew goals; (2) reconfiguring or supplementing existing assessments;(3) revising traditional assessment instruments and raisingstandards; and (4) leaving assessment development to local schooldistricts. The first three approaches are highlighted in thisdiscussion. It appears that there is not a single best model forstate assessment and that innovation should be supported andencouraged. Assessment and reform efforts should foster the samegoals, and teacher involvement is essential. Quick fixes will not bethe answer. If states change assessment without changinginstructional practices, there is little likelihood of improvedstudent performance. Appendixes contain a glossary and an overview ofassessment practices. Two tables present survey information.(Contains 14 references.) (SLD)

4:21N.,

Tr\A

11"\riC) U.S. DEPARTMENT OF EDUCATION

Office of Educational Research and Improvement

It- EIDU TIONAL RESOURCES INFORMATIONCENTER (ERIC)

en This document has been reproduced asreceived I rom the person or organization

Zoriginating it

k.)0 Minor changes have been made to improve

reproduction quality

Points of view or opinions slated in this dorument do not necessarily represent officialOERI position or policy

of state educational assessment programs

BEST COPY AVAILABLE

of state educational assessment programs

Linda Bond

with Lawrence Friedman & Arie van der Ploeg

Council for Educational Development and ResearchNational Education Association

3

Ed Talk is published by the Council for Educational Development and Research2000 L Street, NW Suite 601 Washington, D.C. 20036 (202) 223-1593

Dena G. Stoner, Executive DirectorUllik Rouk, EditorJulie Srodes and Dan Shore, Production Assistants

This project has been funded at least in part with Federal funds from the U.S.Department of Education under contract numbers RP91002001 through

RP91002010. The content of this publication does not necessarily reflect the viezvsor policies of the U.S. Department of Education nor does mention of trade names,

commercial products, or organizations imply endorsement by the U.S. Government.

AK

"4.4v

NWREL fNonhwasl Rogunal

Educational Laboratory IPortland. OR /

MeRELMod-con:men! Rag:on&Educational Laboratory

Aurora. CO

NCRELNOIth Control PinionedEducational Laboratory

Oak Brook. IL

-1"---,i/ ----1----FWL l. NvFor Wast Laboratory ?t, \

lorEducational Research \ \

end Development \Sart Francrsco.CA

\'i / A.' Treame

ND

SD I

NE&IRecacant Laboratory tor

Edur.atrooal Improvementol rho Noetheast and Islands

tfauerlo Rico and Virgin Islands)Andovafi MA

vT

/-1; NIT

# MA

NJ

uy

CT

RBSRosoarch

DEPalter Schools

Phrlarlolphia. PA

.2)

rArnerican Sarno.rComrnonwealth cl the

Northern Mariana Islandseedaroter7 Slates el Korona.

Kosraepohnpro.Chuuk and Yap*GuamRopLtolic of the Marshall istaridsRepublc ol Palau

T'Nere,

het.a

NE

KS

PRELPacrfic Region

Educehonet LaboralorYHonolulute

TX

ok I

7 Nc ;31

GA \'Ff' SERVE

.' LA (---,. ..ri Southaastarn Regional Vision

4.),,,,,....... Education

ra/ k Greensboro. NC

r SEIDLTT \

I

DC

AELAppalachia Educational

LaboratoryCharleston. V/V

Southwest EducetionalDaveroprnsre Laboratory

AllStan. TX

Regional Educational Laboratories

Northeast RegionThe Regional Laboratory for EducationalImprovement of the Northeast and Islands300 Brickstone Square, Suite 900Andover, Massachusetts 0181(1508/470-0098Director: David P. Crandall

Mid-Atlantic RegionResearch for Better Schools444 North Third StreetPhiladelphia, Pennsylvania 19123215/574-9300Director: John E. Hopkins

Appalachian RegionAppalachia Educational LaboratoryP.O. Box 1348Charleston, West Virginia 25325304/347-0400Director: Terry L. Eidell

Southeastern RegionSouthEastern Regional Vision for EducationP.O. Box 5367Greensboro, North Carolina 27412919/334-3211Director: Roy H. Forbes

Southwestern RegionSouthwest Educational Development Laboratory211 East Seventh StreetAustin, Texas 78701512/476-6861Director: Preston C. Kronkosky

Central RegionMid-continent Regional Educational Laboratory2550 South Parker Road, Suite 500Aurora, Colorado 80014303/337-0990Director: C. L. Hutchins

Midwestern RegionNorth Central Regional Educational Laboratory1900 Spring Road, Suite 300Oak Brook, Illinois 60521708/571-4700Director: Jeri Nowakowski

Northwestern RegionNorthwest Regional Educational Laboratory101 S.W. Main Avenue, Suite 500Portland, Oregon 97204503/275-9500Director: Robert R. Rath

Western RegionFar West Laboratory for Educational Research andDevelopment730 Harrison StreetSan Francisco, California 94107415/565-3(10(1Director: Dean H. Nafziger

Pacific RegionPacific Region Educational Laboratory828 Fort Street Mall, Suite 500Honolulu, HI 96813808/533-6000Director: 'ohn W. Kofel

5

pf ;Imp e 1, r Fti.4? "4 4.

iri-

.>

Table of Contents

Preface

A New Era for Student Assessment 1

States as Incubators of Assessment Innovation 3

Profiles of State Assessment Programs 7

California and Vermont Start Fresh 9

Maryland and Michigan Add to What Is Already There 17

Indiana and Texas Explore New Approaches Cautiously 24

A Cavalcade of Change 31

Findings and Recommendations 33

References 37

Appendix 1: Glossary of Terms 39

Appendix 2: An Overview of State Assessment Programs 43

Acknowledgments

Preface

rr he Council for Educational Development and Research established theEdTalk publication series as a way to inform a variety of audiences about

nationally significant topics in education. This publication explores therelationship between state systemic reform initiatives and state studentassessment programs. It stems from a common interest in systemic educationreform by the Council, its regional educational laboratory membership, andthe National Education Association (NEA).

In systemic reform, changes in one part of the educational system are inte-grated with changes in another part. Expectations for students, school cur-riculurn, instruction and instructional materials, teacher preparation, certifica-tion and inservice, and school organization and governance all work togetherso that the entire system moves toward the same goals. The Council, anassociation that provides the latest information from educational research anddevelopment to educators and policymakers, wanted to know how statestudent assessment programs are leveraging state systemic reform initiatives.With all the evidence pointing to sta te student assessment programs affectinghow teachers teach and what students learn, not surprisingly, NEA, with its2.5 million teacher members, had a keen interest in the same question.

The North Central Regional Educational Laboratory (NCREL) houses a statestudent assessment program database, developed in collaboration with theCouncil of Chief State School Officers." NCREL used this database, whichcontains up-to-date information from the annual Association of StateAssessment Programs' survey of the nation's state assessment directors, toconduct this study. In addition to analyzing the findings in the database,NCREL interviewed state assessment directors on the relationship betweenassessment and reform in the six states it profiled California, Indiana,Maryland, Michigan, Texas, and Vermont.

By publishing this paper, we aim to give educators, policymakers, and othersinterested in the topic of educational assessment a look at the range of differ-ent state assessment programs being implemented across this country toleverage systemic reform.

Ed Roeber of the Council of Chief State School Officers has single-handedly collected state assessment informationfor over fifteen years. He worked closely with NCREL on the conversion of the survey to database format, andconsulted on the design of this study. His thorough editing of this paper is also gratefully acknowledged.

A New Era for Student Assessment

dvocates of educational improvement are pressing for "systemic"reform. Systemic reform strategies aim to transform all parts of the

educational system in a coordinated way. Curriculum; instruction andinstructional materials; local, state, and national assessment; school organi-zation and governance; and teacher preparation and inservice are all syn-chronized to work together toward a common set of agreed-upon standardsfor student achievement.

Of all these areas, student assessment* is increasingly being used to leveragesystemic reform. How student assessment is organized and documented,and what it means for students and schools, has taken center stage in thedebate about systemic reform. Thepurpose of this paper is to provide aquick review of the issues that are ... student assessment isemerging in this debate. It illustrates

I

increasingly being used towhy decisions about student leverage systemic reform.assessment are often made the waythey are.

I

Calls for a nationwide assessment system that holds all students, schools,local districts, and states accountable for student achievement are echoingthrough Congress. Assessment plays a pivotal role in most state reformstrategies. Furthermore, Secretary of Education Richard Riley brought withhim from his South Carolina governorship a staunch belief that testing canstimulate educational reform. Similarly, as governor of Arkansas, PresidentClinton advocated testing to measure students' academic progress. Manyothdr governors are relying on testing to raise school performance byholding educators accountable for their students' test results.

Nearly a decade of test-driven educational reform has shown that educatorsdo indeed match their teaching to the skills and knowledge tested on stateassessments. This is particularly true when test scores are the decidingfactor in determining sanctions or rewards.' One consequence of this

Student assessment refers to any in-depth method of determining what students know and can do, includingtesting, teacher observations, collections of students' work (portfolios), projects, and performances. It is consideredto be more comprehensive than testing whiLh typically employs a one-time, paper-and-pencil measure.

practice is that a tension has developed between what is known to be top-notchinstruction and what is actually on the test. On the one hand, there is a growingconsensus that schools should instruct students in complex, problem solvingskills; on the other hand, most present-day assessment programs still revolvearound multiple-choice tests of basic skills.

Many states are trying tc .csolve the conflict by promoting a systemic reformagenda based on new standards for student performance. As the MathematicalSciences Education Board of the National Research Council put it:

Calls for standards in education have been echoing across thenation for several years, especially since political leaders of bothparties decided to adopt bipartisan national goals for education.Standards without appropriate means of measuring progress,however, amount to little more than empty rhetoric. To staythe course, we must measure the things that really count.2

Surveying the Landscape of State Educational Assessment Programs describes thevariety of state assessment programs being planned or already operating in thiscountry.3 It also tells how six states used assessment to foster their own uniquestate approaches to systemic reform. While, other states are significantly

changing their assessmentprograms as well, these six states

... states are adjusting their were chosen because they repre-assessment programs to meet sent a range of state assessment

their individual needs. strategies. The small number ofstate assessment programsstudied limits this report as

does the fact that state assessment is changing rapidly and new ideas are beinginvented every day. Nonetheless, the triumphs and challenges in these six statesbring new policy-relevant information to the educational assessment debate.

This report does not suggest that there is a right way or a wrong way to conductstate assessment. It does demonstrate that states are adjusting their assessmentprograms to meet their individual needs. It is clear that no one model fits all theassessment needs in education reform.

A glossary has been included in Appendix 1 to ensure a common understandingof assessment terms, although some key terms have been defined on the pageswhere they first appear. Appendix 2 presents a summary of state assessmentprograms nationally.

2

States as Incubatorsof Assessment Innovation

Systemic school reform is having a dramatic impact on state assessmentstrategies. The number of states with mandated student assessment

programs has grown from 29 in 1980 to 46 in 1992.4 Each state assesses threeor four grade levels, on average. This means that at least one-third of allpublic elementary and secondary students participate in state-conductedassessments each year.5

States are embracing new and exciting approaches to student assessment.These assessment programs reflect new expectations for students, new pur-poses and methodologies, and new links between assessment and instruction.All across the country, state assessment programs have become incubators ofassessment innovation.

The state assessment landscape is as varied as it is dynamic. Some states aredeveloping totally new assessments to measure new learner outcomesthose things students should knowand be able to do as a result of their States are embracing new andeducational experience. Other states exciting approaches toare revising their old assessment student assessment.instruments and raising standardsfor student performance. Somestates are supplementing existing student assessments with newer assess-ments. A number of states are encouraging local school districts to take thelead in assessment reform, and to involve teachers in developing assessments,setting standards, ar..d scoring.

Much of the current activity was brought about because as states identifiednew learning outcomes for their students, they realized that many of the skillsthey sought (for example, creative problem-solving and teamwork skills)could not be assessed with traditional, multiple-choice tests. This led them toexplore innovative student assessment strategies. The two most commonapproaches to innovative student assessment are performance assessment,which assesses a student's actual performance of a complex skill or task, andthe creation of portfolios, in which a collection of a student's work is scored asa whole.

1 0

Seventeen states report using performance assessments and six states reportusing portfolios. Of these, four states Arizona, California, Kentucky, andNew Mexico use both. Most states have extended their testing programswith these procedures; that is, they are implementing the newer assessments

but still administeringeither their criterion-referenced test (CRT)*, anorm-referenced test(NRT)**, or both. The threeexceptions are Massachu-setts, which reports using

only performance and writing assessment; Vermont, which relies primarily onportfolio assessment but also administers a short criterion-referenced test inmathematics and a writing sample; and Kentucky, which plans to use perfor-mance assessments and portfolios but currently uses a "transitional"multiple-choice and short answer test.

Seventeen states report usingperformance assessments and six

states report using portfolios.

Many other states report that although they would like tc expand alternativeassessment, so far their changes have not been dramatic. They give thefollowing reasons for not changing more rapidly:

1. Accountability constraints. Assessments that are used topromote students or to award high school diplomas must begrounded in what students are taught now. Assessing studentson curricula not yet in place and using the results to makepromotion and graduation decisions would leave schooldistricts open to court suits.

2. Technical constraints. The technical quality of new assess-ments is still being researched. States are hesitant to embraceassessments that may not yield reliable and valid results:**

3. Professional development constraints. The new curriculaand assessments require an approach to education that is verydifferent from present practice. The lack of professional devel-opment for teachers and principals is hampering states' abilityto change instruction.

Crittrion-referenced tests measure student performance on an agreed-upon set of skills and against anagreed-upon standard. They are traditionally multiple-choice tests.

Norm-referenced tests compare student test performance against that of other students, usually a national sample.

Reliable assessment results remain the same regardless of when or by whom the assessment is administered, or bywhom the results are scored. Assessments are considered valid when they measure what is supposed to bemeasured. For example, if a student receives a score of 4 on an assessment of problem solving, that score should bean accurate reflection of the student's problem-solving ability (validity) no matter who is rating that student'sresponse or when the response is being rated (reliability).

4 1 1

4. Financial constraints. Newer assessments are usually moreexpensive to develop, administer, and score than traditionalones. Many states simply do not have sufficient resources.

Despit, difficulties, many states are pioneering new performance-basedassessment instruments and improving the content of existing assessments inorder to promote systemic reform.

Profiles of State Assessment Programs

Calls to improve the match between assessment and the new learneroutcomes demanded by educational reform first sounded in the late

1980s. By the 1990s, reformers clamored for a complete change from conven-tional practice, including changes in expectations for students; changes incurriculum and instructional practices; changes in how student learning isassessed; and changes in the roles, rules, and responsibilities of those whoparticipate in the educational system.

New expectations call for students to become active and engaged learnerswho are more responsible for their own learning. In the industrial model ofschooling, still in operation in many places, students are passive recipients ofknowledge. They areexpected simply to The goal of education reform is tomemorize and recallinformation. Grades provide students with a better balancereflect accuracy and between the old core of memorizingindependent work. and repeating and the new core ofThese are the same inquiry, thinking, communicating,characteristics that an and problem solving.industrial society oncerequired of workers. Buttoday, technology dominates the workplace. Information quickly becomesobsolete. Memorizing and recalling information is not enough. Studentsmust use their minds to apply knowledge to complex problems, communicatetheir ideas clearly to others, work cooperatively in groups, and find solutionsto real-life problems. Thus, students need to acquire the tools for in-depthinquiry and problem solving, as well as to learn important definitions, facts,and skills. The goal of education reform is to provide students with a betterbalance between the old core of memorizing and repeating and the new coreof inquiry, thinking, communicating, and problem solving. Achieving thisgoal requires changing the many parts of the educational system.

Because reform and assessment do not fall into the "one size fits all" category,each state approaches them differently. This section illustrates how stateshave designed their reform and assessment programs to fit their needs. Itdiscusses how states tailor their assessment programs according to the

7

1 3

purposes to which they put the results. Over and over again, it shows howthe national focus on higher standards and new learner outcomes is drivingassessment reform in states.

The six state assessment programs described here demonstrate the range ofassessment programs being implemented across the country. These statesdiffer in the speed with which they are changing their assessment programsand the factors that are influencing their ability to change.

The first two, California and Vermont, are developing totally new assess-ments to accompany new educational goals and learner outcomes. Financialconstraints, however, are placing great stress on each state's ability to provide

accompanying high-qualityprofessional development. In

... states differ in the speed with addition, Vermont, which giveswhich they are changing their local schools a lot of flexibility inassessment programs and the assessment, is learning that thefactors that are influencing lack of centralized quality control

their ability to change. can limit the usefulness of assess-ment results for makingdecisions.

Maryland and Michigan are supplementing their existing student assessmentprograms with newer assessments. Both are balancing their desire to imple-ment newer forms of assessment with financial, legal, technical, and profes-sional development constraints. They are using traditional assessments forstudent accountability purposes and newer assessments for school andinstructional improvement.

Finally, Indiana and Texas are retaining their traditional assessment instru-ments but revising them to include new learner outcomes, as well as raisingtheir standards for student performance. They are adopting newerapproaches to assessment slowly. The reason is that these states use theresults of student assessment programs to make decisions about individualstudents. Should a student sue the state because the test unfairly denied himor her a high school diploma or promotion to the next grade, the state mustbe able to prove that the assessment is technically sound and measures onlywhat is taught in school.

8 1 4

California and Vermont Start Fresh

Some states have decided that only a "revolutionary" approach toassessment can change school practice. These states are abandoning their

old assessment methods and replacing them with new ones, or they aredesigning first-time assessmentprograms that rely primarily on Some states have decidedalternative assessment. California, that only a "revolutionary"for example, is responding to two approach to assessment canrecent legislative mandates onethat requires performance assess- change school practice.ment and the other that requiresreporting individual scores instead of school scores. Vermont has a majorchallenge as a state that only recently adopted state assessments, to ensure thetechnical quality of its assessment results while keeping teachers highlyinvolved in developing and scoring the state assessment.

California

In response to state legislation, California is changing its assessment programin two major ways. First, it is moving toward performance assessment as ameans to improve instruction for students. Second, whereas California usedto report out only school scores on assessments, it is now beginning to reportout individual student scores. Making both these changes at once has provento be an extremely challenging task.

California's Learning Assessment System (CLAS) has three components:

Elementary, middle grades, and high school performanceassessments in reading, writing, and mathematics have beenadopted for grades 4, 8, and 10. These yielded school-levelresults in 1992-93, and will yield individual results at grades 4and 8 in 1993-94. Performance assessments in science andhistory/social studies were field tested in grades 5, 8, and 10 in1992-93 and will yield fifth-grade school-level results in 1993-94and individual results in 1994-95.

9 1 5

Golden State Examination (GSE) is a series of voluntaryexams that students take at the end of their coursework in asubject. The GSE was first administered in 1987 in high schoolalgebra, geometry, biology, chemistiy, economics, and U.S.history. Additional subject exams are being developed inEnglish composition and in science. The exams includealternative assessment strategies such as essays, laboratoryexperiments, and portfolios.

Career-Technical Assessment Program (C-TAP) consists ofvoluntary, end-of-program exams covering such fields as animalscience, computer technology, construction technology, healthsystems, and home economics/child care. C-TAP is still underdevelopment at Far West Laboratory in San Francisco.

The California Department of Education has made the development of avariety of assessments to measure higher-level thinking skills a major priority.These assessments will include more sophisticated multiple-choice questions

(i.e., questions that... the primary purpose of California's require students to

new assessment system is to support the make connections

instruction called for in the state's among severalconcepts and/or usecurriculum frameworks. more than onestrategy to arrive at

the correct answer), short answer questions, essay questions, and perfor-mance assessment.6 When California's assessment system is fully imple-mented, student assessment will be based on a combination of assessments,some of which will be embedded in curricula and portfolios.

Purpose of Assessment

State legislation specifies that the primary purpose of California's new assess-ment system is to support the instruction called for in the state's curriculumframeworks. These curriculum frameworks are the structure on whichschools build their curricula. They typically outline the knowledge and skillsstudents are expected to learn. The new components of the assessmentsystem challenge students to perform tasks that demonstrate their acquisitionof these skills and knowledge. The curriculum frameworks have been thefocus of a great deal of professional development in California's schools.

In addition to supporting its curriculum frameworks, California expects itsassessment program to provide information on how well schools and school

1610

districts are performing. Both school and school district results will bereported to the public. School results will indicate the percentage of studentsin the school at each of six performance levels. District results will show thepercentage of students who met statewide performance standards in eachsubject area.

Impact of Assessment on Students

Until this latest legislation, students in California participated in the state'sassessment program but did not receive individual scores. Student scoreswere combined into school and school district scores, which became part ofan annual accountability report to the public on overall student achievement.

Beginning in the 1993-94 school year, students in grades 4, 8, and 10 willreceive individual scores in reading, writing, and mathematics. Secondaryschool students also take the Golden State Examination end-of-course andend-of-program assessment series if their school chooses to administer them.Assessments in science andhistory/social science will be Regardless of the choices statesphased in as funding permits. ; make in reform and assessment,The new legislation requires trade-offs are inevitable.that, by the end of grade 12,each student will have an"Individual Record of Accomplishment." This is a portfolio that includesstudent scores on state assessments, voluntary Golden State Exam results inacademic subjects, any Career-Technical Assessment Exams that the studenthas taken, and other indicators of achievement.

In addition to receiving individual test results, students may notice otherdifferences: first, the kind of assessment being used and, second, a muchcloser link between assessment and instruction. Reformers' intent is that,with appropriate staff development, teachers will change their instructionalpractices and ways of evaluating student work.

Assessment and Reform

Regardless of the choices states make in reform and assessment, trade-offs areinevitable. Strengthening an assessment's utility for one purpose weakens itfor another. California is a state that, because of its size and tradition ofinnova.tion, has for some years been moving toward fundamental changes incurriculum, instruction, and assessment. The state now faces two majorhurdles. The first is technical. The second hurdle is posed by a shortage oftime and money.

111 7

Technical challenges face every state assessment program, but they areparticularly great for states that are switching to performance assessments.These states must ensure that their performance assessments are technicallysound that is, they must ensure the reliability, lack of bias, and comparabil-ity of their assessments for all the purposes for which they are being used.California has an active research effort underway, both in the state educationdepartment and by numerous outside contractors, and a policy of adoptinginnovative assessments only as the research warrants their use. However,because California does not use assessments to make decisions about indi-vidual students, it is less constrained in its approach to assessment than someother states. It also balances performance assessments and portfolios withmore traditional forms of assessments when these are appropriate.

California's second major hurdle is to overcome the constraints of time andmoney so that it can fulfill its new mandate to provide individual scores. Toobtain a school district or school building score, fewer activities are generally

necessary than to obtain a... because California does not use reliable mdividual score.

assessments to make decisions The shift to performanceabout individual students, it is less assessment complicates

constrained in its approach to this issue because perfor-

assessment than some other states. mance assessment takesmore time and is morecostly than traditional

testing. So questions remain ... Can enough examples of a student's work becollected to adequately assess that student's knowledge and understanding ofthe material? Can examples be collected that cover enough of the material toconsider the assessment results a good overall description of the curriculumgoals? To relieve some of these concerns, California is supplementing its stateassessment with portfolio assessments that are embedded in students' every-day work in order to ensure that sufficient student information is availablefor a good estimate of achievement.

Vermont

Vermont is one of only a handful of states that has never had a statewideassessment program. Its attempt to implement a traditional, standardizedtest in 1989 was so criticized by educators that the state dropped the plan.Although its present assessment program is voluntary, most school districtsdo participate.

1 812

Vermont's state assessment program has two components:

Writing and mathematics portfolios are developed andscored in grades 4 and 8. All students in these grades developportfolios, but only a sample of the portfolios are included inthe official state assessment. A grade 10 portfolio in mathemat-ics is being planned. Discussions are also underway to establisha fine arts portfolio.

Uniform test results in mathematics and writing in grades 4and 8 are placed in the portfolio and used, in part, to confirmportfolio ratings.

Two features of Vermont's assessment program make it popular among manyof the state's educators according to recent study conducted by the RANDCorporation? First, Vermont's teachers and students are actively involved inselecting student work to beplaced in the portfolios. Teachers ... Vermont's teachers andactually grade their own students' students are actively involvedportfolios. Second, there is no in selecting student work to beaccountability tied to the program, placed in the portfolios.either for students or schools.

During the 1992-93 school year, fourth and eighth graders in participatingschool districts began compiling portfolios in mathematics and writing.These portfolios hold work samples that reflect the student's mastery of sevenstate mathematics criteria and five writing criteria. Portfolios also contain theresults of the state's uniform assessment in mathematics and writing.

Purpose of Assessment

Vermont's director of assessment describes the assessment program as havingtwo primary purposes. It is "a means of supporting effective instructionwhich benefits all students," and "a means of providing accurate informationfor program evaluation." At the present time, information on studentachievement from portfolios is used to improve instruction and to letteachers, parents, and students know how well students have mastered statecriteria in writing and mathematics. The state clearly intends to hold schoolsaccountable for portfolio results, but only after it is sure the results are reliableenough to do so. State education agency officials say, however, that it is nottheir intent to use portfolios to determine if students have fulfilled highschool graduation requirements or to assign students grades.


Vermont is the only state in which portfolios are the backbone of assessment.The RAND study found that student attitudes toward portfolios are predomi-nantly positive, mainly because students enjoy producing the projects that areplaced in them.8

Unlike most states that collect a single sample of student work byadministering a standardized test, both revisions and samples of students'best work are placed in portfolios. The learner outcomes considered mostimportant are those that students achieve by solving problems, writing acrossthe curriculum, producing original work, and using higher-order thinkingskills, such as considering contradictory information and deciding on acourse of action even though uncertainty exists.

Vermont's uniform test in writing contains standard writing exercises; theuniform mathematics test contains multiple-choice items and open-ended

problems. All students in grades 4 and

Vermont is the only state in 8 take these uniform tests, which are

which portfolios are the intended to supplement the portfolios.These tests, however, are not intended

backbone of assessment. to be the primary instruments inVermont's assessment system.


The state's goal is to integrate its assessment system into everyday instructionand let teachers tailor assessment to classroom instruction. Because of this,changes in curriculum, instruction, and assessment are expected to occur inconcert.

A major challenge for Vermont has been making its assessment reliable sothat scores are comparable across the state and reflect differences in studentperformance, not teachers' confusion about what the instructional goals meanor how to teach them. In the first and second years of implementation,student achievement could be reported on a statewide basis only becauseresults for any group smaller than that were not reliable. One reason is thatbecause teachers interpreted the scoring criteria differently, they did notalways give the same score to similar quality work. Consequently, a score inone part of the state did not necessarily mean the same thing as the samescore in another part of the state.

20

14

Insufficient teacher training in the scoring criteria was one cause of theproblem. Vermont has developed general criteria to guide teachers as theyscore student work in writing and problem solving. As a rule, these criteriacan be applied to many samples of writing and problem solving; they are nottied to a specific task. This allows teachers to place samples of student workthat are representative of classroom instruction in the portfolios, but it al qomeans that teachers must be able to recognize good writing and/or probiemsolving.

According to the director of Vermont's assessment program, while teachersand administrators did attend training sessions in selecting and scoringportfolio content, the training itself was inadequate and uneven across thestate. Variations in training made it difficult to compare portfolio scoresacross schools and classrooms. Reliable results on the uniform test wereavailable, however, and Vermont intends to release these results at the schooland district level.

Vermont has improved the reliability of its assessment results (at least inmathematics) by establishing regional scoring centers where scorers rate asample of portfolios from each region. This use of regional scoring centers isa temporary measure, however, and Vermont still plans for teachers to do allof their own scoring once they have had sufficient professional development.

The RAND Corporation evaluated the first two years of Vermont's assess-ment program. In a survey of teachers, it found other, more positive changesemerging from Vermont's portfolio assessments. Many teachers reporthaving changed their instructional approach as a result of the portfolioproject. For example, theyreport that students are Many teachers report havingwriting more in the class- changed their instructional approachroom and are spending more as a result of the portfolio project.time on problem-solvingtasks. There is also evidenceof positive change in mathematics instruction. Eighty-two percent of theteachers who responded to the RAND survey reported an increase in theamount of instructional time they devoted to mathematics problem solving,while 49 percent reported spending less time on computation than before.9 Itappears that a substantial number of teachers recognize that the changes inassessment support the goals of reform. Many schools have shown theirconfidence in the project by expanding the use of portfolios beyond the twogrades in the state's assessmcnt program.

While Vermont has been severely criticized for its assessment results notbeing reliable, it has also been applauded for seeking a remedy to theproblem that stays true to the state's original goal of extensively involving

2115

teachers in state assessment. State officials, meanwhile, cite the positivechanges in instructional practice that have been reported as a result of thenew assessment less time on rote memory work, more time on solvingproblems and understanding concepts and are trying to further eithancethis positive effect while improving the reliability of the scoring.

Maryland and MichiganAdd to What Is Already There

Maryland and Michigan are two of the many states that are reconfiguringtheir student assessment programs or supplementing them with newer

assessments in order to support systemic reform. Both states have legislativemandates to move toward performance assessment. However, the difficultiesin trying to collect reliable data on student achievement for high schoolgraduation decisions have caused one to keep its multiple-choice graduationtest and the other to consider implementing such a test.

Maryland has added an extensive new performance assessment that yieldsschool-level information for instructional improvement purposes, yet itretains its traditional, basic skills high school graduation test. Michigan,which has a history of designing innovative assessments, is adding a highschool graduation test that could end up being less innovative than the state'score assessment program simply because it must be legally defensible.

Maryland

Maryland's School Performance Program was established in 1989. The firstassessments were administered in 1991. However, because the performance-based, criterion-referenced test cannot, in its present form, yield results thatare reliable enough for making decisions about whether or not students havemet the requirements for high school graduation, the state has retained amultiple-choice graduation test implemented in the 1970s "for the timebeing." Some confusion created by one state assessment program containingtwo very different approaches to assessment, even though they are for verydifferent purposes, appears to be unavoidable at present.

The components of Maryland's state assessment program are:

Criterion-referenced assessments are administered in math-ematics, reading, science, social studies, and writing to samplesof students in grades 3, 5, and 8. These assessments consist ofopen-ended questions, essays, and performance assessments

17 2 3

based on Maryland's new curriculum frameworks. Approxi-mately 350 teachers have helped to develop the performanceassessments. An additional 600 teachers are paid to score theassessments over the summer.

The High School Functional Graduation Test is a minimumcompetency test that all students must pass to graduate. Thistest, made up of traditional, multiple-choice questions, is firstgiven in grade 7. Most students pass it by grade 9. A school'sannual performance report shows both the percentage ofstudents who meet satisfactory or excellent standards of perfor-mance the first time they take the test and the percentage whodo not meet.standards.

The Comprehensive Test of Basic Skills/4 is a norm-referenced, shelf* test in basic-skills. It is given to a sample ofstudents in each district for use in national comparisons.

Beginning in 1994, the state will administer a new criterion-referenced schoolperformance assessment to students in grade 11 as well. This assessment willinclude both a state component, made up of assessments in specific subjectsand interdisciplinary performance tasks, and a local component consisting ofportfolios and projects.

Purposes of Assessment

According to the state director of assessment, Maryland's assessmentprogram sets out "to insure that all students in Maryland learn rigorouscontent during their educational experiences." Different components of theprogram, however, serve different purposes.

The purpose of Maryland's criterion-referenced assessments is to hold theschools accountabl- for student achievement and to help them improve theirprograms. Each school's performance is rated using a number of indicators,including student assessment results, promotion rates, graduation completionrates, and student attendance and dropout rates. Each fall schools receive theresults of their previous year's criterion-referenced assessments. Shortly afterthat, the results are made public. School scores tell what percentage ofstudents in grades 3, 5, and 8 meet excellent or satisfactory proficiency levelson the assessments. Schools must develop and implement a schoolimprovement plan based on the analysis of their strengths and weaknesses.

--------_-____ --A shelf test is one that can be bought "as is" from a test publisher. It is typically based on common elements of

most schools' curricula as determined by standard textbooks.

1824

Even though the state is moving away from norm-referenced assessment andusing fewer multiple-choice items, the high school graduation test and thenorm-referenced Comprehensive Test of Basic Skills/4 remain key compo-nents of Maryland's state >issessment program. The state has several reasonsfor retaining these tests. First, multiple-choice graduation tests produceindividual student scores. They aredefensible in court. Second, the norm- While the newer assessmentsreferenced test yields national corn- take more time, their designparative information and is the only resembles good instructionform of assessment that is currentlyaccepted for Chapter 1 evaluation, and students enjoy them.Nonetheless, these two assessments,along with their very differentcontent, give teachers conflicting messages about what is important forstudents to learn. In the future, the state intends to ahow schools to waive thehigh school graduation test for students who perform satisfactorily on theeighth-grade criterion-referenced assessment.


The high school graduation test is the only assessment in which all studentsparticipate and receive individual scores. The CTBS/ 4 yields individualscores but is taken by samples of students. In all, each student takes about athird of the test. Similarly, the criterion-referenced assessments are taken byonly a sample of students, with some students taking some parts of the testand other students taking other parts. While the newer assessments takemore time, their design resembles good instruction and students enjoy them.Since only a sample of students participate in the criterion-referenced assess-ments, the time schools must set aside for assessment is more manageablethan it would be if the assessment were administered to everyone.


Since Maryland's new performance assessment program holds schoolsaccountable and is aimed at helping them improve, it does not face the sameconstraints as assessment programs that lead to decisions about students.Since each participating student is tested on only one portion of the assess-ment, the state is able to administer the entire assessment to arrive at a schoolscore. Individual student performance results are returned to schools, but thestate urges schools not to over-interpret them because they may not reflect thestudent's proficiency on the entire curriculum. Each student is assessed ineach content area, however, and the results do provide an additional source ofinformation about student achievement if used cautiously.

19 25

The state is committed to performance assessment and plans to make it thecentral feature of its student assessment program. Unlike Vermont, whichinvolved a great many '-achers in developing its performance assessmentprogram and has teachers scoring their own portfolios, Maryland workedclosely with a national test publisher. This has limited the local flexibility thatis both the strength and the Achilles' heel of the Vermont program. On theother hand, Maryland's program is more centrally controlled than Vermont's,and so yields more reliable assessment results.1° However, central controlmay impede the kind of local ownership and program buy-in that character-izes the Vermont program. Trade-offs are inevitable and states must decidewhich trade-offs they are most comfortable with, and design their assessmentprograms accordingly.

Michigan

Michigan, which has had a state assessment program for approximately 25years, is considered a leader in the development of innovative multiple-

choice and performance assess-Michigan ... is considered a ments. Right now, the stateleader ir the development of assessment program has two

innovative multiple-choice and major components. The state

performance assessments. legislature recently required that athird component be added in theform of a high school graduation

test that 'you ld ensure that students graduate from high school with neces-sary skills for success. Legal constraints imposed by this high school gradua-tion requirement may slow the development of innovative assessments inthat state.

When this third component is fully developed, Michigan's EducationalAssessment Program will consist of:

The Michigan Educational Assessment of Progress (MEAP)is a multiple-choice test administered at grades 4, 7, and 10 inreading and mathematics, and at grades 5, 8, and 11 in science.It is nationally regarded as among the most innovativemultiple-choice tests in the country.

Michigan Employability Skills Portfolio was designedprimarily as a resume-building exercise for high school students.This portfolio demonstrates that the student has masteredMichigan employability skills. Students share it with prospectiveemployers or post-secondary educational institutions.

20 2 6

Michigan High School Graduation Test (under develop-ment) will be administered for the first time in 1994-95 and willbecome a graduation requirement for the class of 1997.


Until recently, the sole purpose of Michigan's assessment program was toidentify students who needed remedial assistance to meet the state's educa-tional standards. Legislative action in March, 1989, increased school account-ability by requiring schools to submit Annual Educational Reports to the StateBoard of Education and to the public.

With these changes plus the addition of the High School Graduation Test, theprimary purpose of state assessment shifted from e: znosing student andschool weaknesses to holdingstudents and schools account- ... the primary purpose of stateable for their performance. assessment shifted from diagnosingMichigan is promoting

Il student and school weaknesses tochanges in the educational

!I holding students and schoolssystem that match the out-; ! accountable for their performance.comes that will be measured II

on the state assessment.


Although schools have always been under pressure to perform well on theMEAP because test scores are reported in the newspapers, the addition of ahigh school graduation test will increase pressure on individual students.The extent of that pressure will depend largely on how teachers andprincipals react to the new accountability requirements.

Schools are accustomed to receiving MEAP results during the winter andproviding extra help to students who do not satisfactorily meet statestandards. The state distributes compensatory education funds to schoolsaccording to the number of students who need assistance. Prior to its require-ment that students pass a high school graduation test, Michigan set itsstandards intentionally high to reflect "world-class" levels. As a result, manystudents scored less than satisfactorily.

Implementation of a high school graduation test changes this. Because theresults of the high school graduation test will be used to award high schooldiplomas and the test must be able to meet legal challenges, it can measureonly what is taught, not what the state may consider "world-class" standards.

21 27

The assessments must also meet rigorous technical quality standards. Thedanger, though, is that other parts of the assessment system may continue tomeasure and encourage the teaching of these higher standards, sendingteachers a double message about what is important to teach.


Michigan faces two major challenges. First, the move to a strong studentaccountability system limits the content of the assessment to what is actually

taught. Newer content basedNewer content based on revised on revised learner outcomes

learner outcomes will need to bephased into the assessment only

after it has become a part ofthe school's curriculum.

will need to be phased into theassessment only after it hasbecome a part of the school'scurriculum. Second, the moveto greater accountability hasbeen coupled with a legislated

mandate to use performance-based assessment strategies. Since any assess-ment that is used for student accountability requires evidence of technicalquality, and since "technical quality" for performance assessment is still beingdefined and researched, implementation of the newer assessments may beslowed down considerably.

The state's graduation test has already had an impact on student standards.The legislature adopted a lower standard on the high school MEAP test inorder to use the assessment program to endorse high school graduation.Until the graduation test is in place, the state is affixing an official seal to thediplomas of students who have passed the MEAP test. The MichiganDepartment of Education plans to address this dilemma of lower standardsby releasing two sets of standards for the High School Graduation Test, onethat matches current curricula and student performance levels, and anotherfor the year 2000 that encourages schools and students to reach beyondpresent expectations.

Michigan's legislation created financial and technical dilemmas as well. Itrequires the state education agency to develop a graduation test that includesopen-ended, performance tasks. However, performance assessment costsmore than traditional assessment. During its 1993 sesson, the legislatureeliminated property taxes as a source of school funding but did not replacethem with another funding source (the legislature has a year to find alterna-tive funding). Consequently, the funding of performance assessment is "iffy"at best.

22 28

In addition, an expert panel commissioned by the State Board of Educationand funded by North Central Regional Educational Laboratory has warnedrTainst using such an assessment for graduation until the technical qualitycan be assured. This will also cost money and time, two commoditiesMichigan education is lacking.

The Michigan Department of Education is fully cognizant of these challengesand has been proactive in addressing them. First, it has sought legislation tochange the high school graduation test mandate to a special seal which isaffixed to a student's diploma if he or she has passed the test. Second, it hascalled together a panel of educational, legal, and psychometric leaders to helpimprove the accountability focus of the test. Finally, it has commissioned agroup of experts in curriculum and assessment to plan a new assessmentsystem.

2329

Indiana and TexasExplore New Approaches Cautiously

I ndiana and Texas look to accountability as their major approach toimproving the achievement of schools and students. To measure the knowl-

edge and skills that are the goals of their educational reforms, both states areraising their standards for student performance and revising their traditionalassessment instruments to reflect new learner outcomes. They are exploringother, newer approaches to assessment cautiously.

Texas, which has several different assessment programs in place, is beginningits exploration of alternative assessment by adding voluntary end-of-courseexams for students. The assessment used for high school accountabilitypurposes remains the same for now. In Indiana where a single assessmentmeasures both state and student performance, constraints on changing theprogram are even greater.

Indiana

Indiana, like many states, is investigating ways to incorporate innovativeforms of assessment into its assessment program, but real change is beingmade slowly. The state's assessment program, Indiana Statewide Testing forEducational Progress (ISTEP), has been used for purposes of holding studentsand schools accountable since 1988. This largely basic skills test is a custom-ized version of an off-the-shelf*, norm-referenced exam. In other words, thestate has added items to the shelf test to align it more closely with outcomesIndiana expects from its students. In the past, a writing sample was alsocollected from students at several grade levels each year, but funds for thiswere cut in 1992.

The 1992 Workforce Development Legislation requires a new assessmentsystem based on new learner outcomes (essential skills) that will also be usedto measure school and student accountability. In addition, the legislationmandates a high school graduation test.

Off-the-shelf assessments are pre-developed by test publishers and purchased "as is", although somecustomization, through the addition or deletion of sections of the test, is possible.

24 30


When asked if Indiana's state assessment program was an informationprogram, an accountability program, or a curricular and instructionalimprovement program, the director of student assessment replied, "All threeand seven others." ISTEP is used both to diagnose school and student weak-nesses, and to measure school and student accountability It has been used toidentify which students need remediation, to accredit schools, and to bestow

monetary and other awards to schools.... Indiana's testing Because of this, Indiana's testing program is

program is considered considered very "high stakes," meaning that

very "high stakes"... test results are used to provide or withholdsomething of value to students (promotion tothe next grade) and schools (accreditation

and awards). The Commission for Higher Education also uses ninth-gradeISTEP results to counsel high school students on college preparation. BecauseIndiana uses a single assessment for several purposes, it has not been able tomake dramatic changes to the assessment program. Changing the assessmentprogram for one purpose would weaken it for another. For example, when astate's assessment program is used largely for accountability purposes, theinformation that it provides about students' strengths and weaknesses isusually inadequate for teachers to use to improve instruction. Indiana isplanning to move to a system of assessments, each of which will serve a givenpurpose. This, clearly, will help.


The state funds remedial service over the summer for students who fail tomeet state standards on ISTEP. If, at the end of summer school, students failto meet the state's standards on a retest, they repeat the grade unlessprincipals and teachers have reason to believe that a student's test result isinaccurate or that staying in the same grade is not in the student's bestinterest.

ISTEP's emphasis on identifying low-achieving students is the reason for theassessment program's focus on basic skills. Such a focus, however, providesonly limited information about middle and upper-achieving students. Thenew legislation and the essential skills that it mandates will change thecontent of the assessment system so that it assesses more of the skills calledfor in state reforms. Still, the speed with which these changes can be madewill be restricted by the need to ensure that these new skills have time to findtheir way into the schools, and that the assessments' technical quality canwithstand challenges in court.

25 31


Indiana's assessment program faces all of the challenges inherent in anaccountability system. Although Indiana's assessment program has neverbeen legally challenged, ISTEP must be technically sound so that it can with-stand a legal challenge should one arise. It must also have staying power overtime. Since the state awards money to schools that demonstrate improvementon ISTEP and other outcome measures, the content of assessment must bestable from one year to the next. This further constrains the state's ability toradically change its assessment programs.

Despite such constraints and the challenges, Indiana is shifting the content ofits state assessment program to reflect new state learner outcomes. In 1992,the Indiana General Assembly mandated two changes in the state's assess-ment system: it required that the amount of assessment in early grades bedecreased and thata high school Indiana, along with many other states, isgraduation exam exploring ways to minimize the "teach onebe added. A State way, assess another" message the stateStandards Task has been sending to educators.Force is selectingessential skills thatall students must acquire before they can graduate. These standards will beaccompanied by new criterion-referenced exams at grades 4, 8, and 10beginning in the 1993-94 school year. Students who pass the high school exitexam, called the Grade 10 Gateway Exam, will be given a Certificate ofMastery which allows them to select either a predominantly academic ortechnical program for their last two years of high school.

To match the content of assessment more closely with important learneroutcomes, Indiana's new criterion-referenced assessment system will includeessential skills such as complex problem solving. The state intends to accom-plish this by improving the multiple-choice, short-answer, and essay items onthe present state-level assessment. Some open-ended mathematics problemsare likely to be added to the assessment program and the writing assessmentmay be restored. A matrix-sampled performance assessment (where somestudents do some items, while others do the rest) is also being considered.Locally, diagnostic tests and portfolios are being planned for individualstudents.

Like many states, Indiana must reconcile its reforms with the content of itspresent state exam. Making curriculum and instruction innovative has beendifficult because ISTEP measures traditional skills. Indiana, along with manyother states, is exploring ways to minimize the "teach one way, assessanother" message the state has been sending to educators.

26 32

Texas

Texas is another state that has a history of using assessment to measure howwell students and schools are meeting state requirements. Because high

school graduation is tiedHow Texas integrates performance to state assessment, the

assessment into its high-stakes state has had to defend itsaccountability system will be assessment in court. In

interesting for other states to watch. 1992, the Texas legislaturemandated that the stateassessment be based

primarily on students' performance on a variety of complex problem-solvingtasks. How Texas integrates performance assessment into its high-stakesaccountability system will be interesting for other states to watch.

Texas administers two assessments, the recently implemented TexasAssessment of Academic Skills and the older Norm-Referenced AssessmentProgram. Both programs are undergoing changes.

Texas Assessment of Academic Skills (TAAS) is a criterion-referenced exam that is aligned with the essential elements inTexas' state curriculum in reading, writing, and mathematics ingrades 4, 8, and 10. In 1994, science and social studies will beadded to the TAAS in grades 4 and 8. A student must pass thetenth-grade exit test in order to receive a diploma.

The Norm-Referenced Assessment Program is a standard-ized test administered in grades 3 through 11 in language,mathematics, reading, science, and social studies. However,only the reading and mathematics tests are state-mandated.


Texas uses its assessment program to diagnose student weaknesses and toidentify which students need remediation, as well as to confer high schooldiplomas, to accredit schools (as part of a published school performancereport), and to present awards to schools. The state provides compensatoryeducation funds to help cover the cost of remedial instruction for studentswho fail to meet state standards on TAAS. It also plans to provide technicalassistance for low-performing schools and staff development for teachers inthose subject areas covered by the state assessments.

3327


As in most states where accountability is the major purpose for state assess-ment, the content of the assessment influences students' educational experi-ence. In Texas, students get remedial help if they do not do well on the stateassessment, and do not graduate from high school if they fail to meet statestandards on the exit test. In addition, because school accreditation andschool awards are based on student performance, it is likely that studentsspend at least sometime preparing for the Texas now requires its assessmentstate assessment. As system to cover the curriculum in

changes to cover morethe state assessment more depth, but at fewer grade levels

and with higher student expectations.complex skills,students undoubtedly I_will find that theirinstructional experiences will include more of these skills. Otherwise, theycan claim that they have not had the opportunity to learn the skills on thestate assessment. Many states that are moving to performance assessment arecounting on their assessment program to have this classroom effect.However, accountability constraints limit what Texas can do.


The latest wave of reform has not changed Texas' long history of state testingfor accountability. In fact, it spurred Texas to upgrade its expectations forstudents and the content and design of its state assessments. Texas nowrequires its assessment system to cover the curriculum in more depth, but atfewer grade levels and with higher student expectations. As a result, TAAStests more complex thinking skills.

Because teachers are known to teach to the items on a high-stakes assessment,the hope is that the shift to performance assessment will change both howteachers teach and the outcomes they expect of students. The TexasEducation Agency is counting on the student assessment program to promotethe state reform agenda, produce higher standards for all, and impiovecurriculum and instruction to help students achieve these standards.

Although TAAS will remain in place until 1995, a considerable amount ofdevelopment and piloting of performance assessments is going on in algebra,biology, computer-based technology, and oral language proficiency, as well asin science and social studies. While TAAS is still a fairly traditional program,made up of criterion-referenced assessment plus a writing sample, it will not

28

34

always be so because of its mandated move toward performance assessment.From watching Texas, we can gain an understanding of how a high-accountability, high-innovation state designs performance assessment tomeasure important new skills, while ensuring that its program can stand upin court.

3529

A Cavalcade of Change

Not surprisingly, states other than the six profiled here are changing theirassessment programs in response to systemic reform initiatives. The

following examples illustrate the range of changes taking place across thecountiy.

Planning

Alaska is planning new assess .tent strategies in reading, mathematics, andscience that will replace the nationally norm-referenced Iowa Test of BasicSkills (a norm-referenced, shelf test).

Florida is preparing to implement a statewide, school-level accountabilityplan. Portfolio and performance assessments are expected to be criticalelements.

Idaho intends to convert to performance assessment first in language arts,mathematics, social studies, and science, and then in the arts, the humanities,and health and physical education.

Nebraska, one of the few states that has no statewide assessment program,is contemplating a new accountability system.

The state education agency in South Carolina plans to recommend that thelegislature eliminate all state-mandated student assessments prior to grade 4.It also wants to create a new assessment program featuring alternativeassessments based on the state's new curriculum frameworks.

Wisconsin is working with the Wisconsin Center for Educational Researchat the University of Wisconsin-Madison to develop performance assessmentsfor its state assessment program.

3136

Starting Small

The Arkansas state education agency design team is working with sixschools to develop curriculum and portfolio assessments in mathematics, andwith six other schools to do the same in language arts.

Kansas is modifying its reading and mathematics assessments and refiningits pilot writing assessment in grades 5 and 8.

Washington developed and is field-testing performance measures tosupplement its existing multiple-choice, paper-and-pencil exams.

Moving More Rapidly

Arizona has introduced statewide performance assessments in grades 3, 8,and 12 in reading, mathematics, and writing.

By 1995, Delaware expects to assess four grades using new nontraditionalassessments matched to the state's new curriculum framework.

In the 1993-94 school year, North Carolina will replace the CaliforniaAchievement Tests with new curriculum-referenced, state-developed studentassessments and performance measures.

The state of Minnesota is eliminating Carnegie units as the measure of highschool completion and is changing to a system of assessments based ontwenty-three learner outcomes. Local schools are developing theseassessments with the support of consultants.

3732

Findings and Recommendations

FT hroughout the United States, states are actively pursuing systemiceduca tional reform. Although none is using exactly the same approach as

another, most are seeking ways to tie all parts of the educational systemstudent expectations, school curriculum, instruction and instructional materi-als, teacher preparation, certification and inservice, school organization andgovernance to a set of standards that better match students' needs forsuccess in the twenty-first century. Because state student assessment pro-grams have been shown to affect how teachers teach and what students learn,great attention is being given to making sure that assessment programs matchthe goals of systemic reform.

Surveying the 50 state education agencies and conducting in-depth inter-views with six of them has revealed that states are approaching the de ,lop-ment of new or revised assessment systems with one of four general strate-gies. The first three strategies were highlighted in this paper: (1) developingtotally new assessments to accompany new goals for learner outcomes; (2)reconfiguring or supplementing existing student assessment programs withnewer assessments; and (3) revising traditional assessment instruments toaccount for new learner outcomes and raising standards for student perfor-mance. The fourth strategy in state assessment is to leave the developmentand implementation of assessment programs to local school districts.

A number of factors influence approaches to assessment in the states. Theyare related to the level of accountability tied to assessment, technical qualityissues and the need to withstand legal challenges, the professional develop-ment that is available to educators who must implement these changes, andfinancial resources.

Our study suggests the following conclusions and recommendations:

1. There is no single best model for state assessment; innovation in thestates should be supported and encouraged. States have different goalsfor their assessment programs, and different approaches are neces-sary to meet those goals. The federal government should notimpose a single model, but should support the innovation that istaking place and should encourage the states to network and share

33 38

ideas. States need help and support from research and develop-ment in the area of innovative assessment; they do not need a newnational agenda that does not take their needs into account.

2. If assessment emphasizes basic skills while systemic reform initia-tives consider other student outcomes to be more appropriate, teacherswill be torn between teaching to the assessment and teaching to thereform. If high stakes are attached to the basic skills assessment,teachers will focus instruction and reform on basic skills and noton students' application of knowledge and skills. Moreover,changes aimed at restructuring and refocusing the instructionalprocess will not take place.

3. Teacher involvement, buy-in, and professional development/technical assistance are needed to sustain reform efforts. States whereteacher buy-in and involvement in the development of theassessment system are pronounced tend to be more successful atinfluencing change in the classroom. Without teachers beinginvolved and understanding the changes that need to be made,the best assessment in the world will not produce change in theclassroom.

4. Quick fixes are not the answer. In difficult financial times, statesand the national government may be tempted to turn to tela-tively inexpensive, quick-fix reform interventions, such as usingassessment for accountability purposes in lieu of the necessarybut more expensive reforms in professional development andinstructional practice. They should avoid doing this.

5. If states change their assessment practices without changinginstructional practices, there is little likelihood of improved studentperformance. In fact, the newer assessments may do an evenbetter job of documenting low performance simply because theyare designed to assess higher-level skills. This will furthershake the public's confidence in schools and make investing inschools even less likely in tough financial times.

The states are laboratories for assessment policy. Learning from them may bethe best way to collect information about optimal assessment options.Investing in the research and development necessary to make these newforms of assessment work, and determining their impact on students andschools, is a critical federal role.

3034

States are committed to systemic reform that is geared to what students needto know and be able to do to succeed in the twenty-first century. In order forthem to be successful, however, all parts of the educational system must bematched to their goals. If state assessments are measuring different goalsbecause of accountability demands, technical quality issues, lack of profes-sional development, and financial constraints systemic reform is unlikelyto occur.

Too often, curriculum, instruction, and assessment have worked at crosspurposes, with high-stakes state assessment results taking priority over whatchildren need to learn. Today, states are working to bring curriculum, instruc-tion, and assessment into better alignment in order to promote the educa-tional changes that reformers have supported for years. The state assessmentlandscape is filled with purposeful change.

40-35

References

1. Bond, L. A. & Cohen, D. A. (1991). In R. Stake (Ed.), Advances inProgram Evaluation: Effects of Mandated Assessment on Teaching, (Vol.1B, pp. 75-100. Greenwich, Connecticut: JAI Press, Inc.).

Corbett, H. D. & Wilson, B. L. (1991). Testing, Reform and Rebellion,(Norwood, New Jersey: Ablex Publishing Corporation).

O'Sullivan, R. G. (1991). "Teachers' Perceptions of the Effects ofTesting on Classroom Practice" in R. Stake (Ed.), Advances in ProgramEvaluation: Effects of Mandated Assessment on Teaching (Vol. 1B, pp. 145-162. Greenwich, Connecticut: JAI Press, Inc.).

Stake, R. & Theobald, P. (1991). "Teacher's Views of Testing's Impacton Classrooms" in R. Stake (Ed.), Advances in Program Evaluation:Effects of Mandated Assessment on Teaching, (Volume 1B, pp. 189-201.Greenwich, Connecticut: JAI Press, Inc.).

2. Mathematical Sciences Education Board, National Research Council(1993). Measuring What Counts (Washington, D.C.: NationalAcademy Press).

3. The information in this report is drawn from two sources. Informationabout state assessment programs comes from the annual Association ofState Assessment Programs survey of the nation's state assessmentdirectors. This information is available in the State StudentAssessment Programs 1992-1993, a database developed by the Councilof Chief State School Officers and the North Central RegionalEducational Laboratory. The second source of information is a series ofsemi-structured interviews with the assessment directors in the sixstates.

4. Office of Technology Assessment (1992). Testing in American Schools,Asking the Right Questions (Washington, D.C.: Congress of the UnitedStates, 297-934 QL 3).

4137

5. North Central Regional Educational Laboratory (1993). A PolicyMaker's Guide to High School Graduation Testing (Oak Brook, IL:Author).

6. This information is taken directly from a California Department ofEducation Report to the Governor and Legislature, SB 662: "A NewStatewide Student Assessment System." Five Year Cost andImplementation Plan, 1992.

7. Koretz, D., McCaffrey, D., Klein, S., Bell, R., & Stecher, B. (1992). TheReliability of Scores from the 1992 Vermont Portfolio Assessment Program(Washington, D.C.: RAND Institute on Education and Teaching).

8. Koretz, D., Klein, S., McCaffrey, D., & Stecher, B. (1993a). InterimReport: The Reliability of Vermont Portfolio Scores in the 1992-93 SchoolYear (October 18, 1994, Washington, D.C.: RAND Institute onEducation and Teaching).

9. Koretz, D., Stecher, B., Klein, S., McCaffrey, D., & Deibert, E. (1993b).Can Portfolios Assess Student Performance and Influence Instruction? The1992 Vermont Experience (December, 1993, Washington, D.C.: RANDInstitute on Education and Teaching).

10. Reidy, E., Yen, W., Gabrys, R., Hill, R., & Haertel, E. (1993). The Use ofPerformance Assessment in High-Stakes Environments: Is There SufficientTechnical Quality for High-Stakes Usage? A presentation at the Councilof Chief State School Officers' National Conference on Large ScaleAssessment, June 8-10, 1993 (Albuquerque, New Mexico).

11. Hart, D. (1994). Authentic Assessment: A Handbook for Educators(Redding, MA: Addison-Wesley).

38

Appendix 1:Glossary of Terms

T o help readers understand the assessment issues discussed in this report,we offer the following definitions of common assessment terms. A

number of these definitions are adapted from D. Hart's Authentic Assessment:A Handbook for Educators.11

Alternative assessment: any type of assessment in which students create aresponse to a question or task. In traditional assessments, students choose aresponse from a given list, such as multiple-choice, true/false, or matching.Alternative assessments can include short answer questions, essays, perfor-mance assessments, oral presentations, demonstrations, exhibitions, andportfolios.

Analytical trait scoring: judging a performance several times along severaldifferent important dimensions. An example might be the judging of a pieceof persuasive writing for the author's attention to audience, correct use ofgrammar and punctuation, focus on the topic, and persuasiveness ofargument.

Anchor papers or benchmark performances: examples of performance thatserve as a standard against which other papers or performances may bejudged. In writing, anchor papers are selected from actual student essays thatare considered to exemplify the quality of a pe:formance level of "1", "2","3", and so forth. If used with analytical scoring, there may be anchor papersor benchmark performances for each trait being assessed.

Authentic assessment: assessment that both mirrors and measures studentperformance in "real-world" tasks and situations. For example, to assessauthentically a student's ability to problem solve, the student is given areal-world problem and assessed on how he or she goes about solving it.

Context (of a performance assessment): the circumstances within which theperformance is embedded. For example, problem solving can be assessed inthe context of a specific subject (for example, mathematics) or in the context ofa real-life laboratory problem requiring the use of mathematics, science, andcommunication skills. Or, science process skills can be assessed in the contextof a large-scale, high-stakes assessment or a classroom grading context.

39 43

Criterion-referenced assessment: an assessment designed to reveal what astudent knows, understands, or can do in relation to specific performanceobjectives. Criterion-referenced assessments are used to identify studentstrengths and weaknesses with regard to specific knowledge or skills that aregoals of the instructional program.

Curriculum frameworks: describe what should be taught in order forstudents to acquire certain skills.

Dispositions: affective outcomes such as flexibility, perseverance, self-confidence, and a positive attitude toward science and mathematics. Somenew assessments attempt to measure these outcomes.

Holistic scoring: using a scoring guide and/or anchor papers to assign asingle, overall score to a performance.

Matrix-sampling: giving a portion of the assessment to different, representa-tive samples of students so that no student need take the entire assessment.The scores that are obtained are group rather than individual scores, and areoften used to look at the performance of a school or school district.

Norm-referenced assessment: an assessment designed to reveal how anindividual student's performance or test result ranks or compares to that ofan appropriate peer group.

On-demand assessment: assessment that takes place at a predeterminedtime and place. State tests, SATs, and most final exams are examples ofon-demand assessment.

Open-ended tasks: the kind of performance required of students when theymust generate a solution to a problem or perform a task when there is nosingle, right answer.

Open-response tasks: the kind of performance required of students whenthey are required to generate an answer, rather than select it from amongseveral possible answers, but there is a single, correct response. An examplemight be: There are four pieces of wood, each measuring seven feet. If youused them as a fence around your square yard, how large an area would youcreate?

Performance assessment: direct, systematic observation of an actual studentperformance or examples of student performances and rating of thatperformance according to pre-established performance criteria.

4044

Performance criteria: a description of the characteristics that will be judgedfor a task. Performance criteria may be holistic, analytical trait, general, orspecific. Performance criteria are expressed as a rubric or scoring guide.Anchor papers or benchmark performances may be used to identify eachlevel of competency in the rubric or scoring guide.

Reliability: an indication of the consistency of scores across evaluators orover time. An assessment is considered reliable when the same answersreceive the same score no matter when or how or who does the scoring.

Rubric: an established set of criteria for scoring or rating students'performance on tests, portfolios, writing samples, or other performance tasks.

Scale: the range of scores possible for the student to achieve on a test or anassessment. Performance assessments typically use a 4- to 6-point scale,compared to a scale of 100 or more with traditional multiple-choice tests.

Standard: defines what level of skill students must demonstrate on thelearner outcomes.

Standardized assessments: assessments that are administered and scored inexactly the same way for all students. Traditional standardized tests aretypically mass-produced and machine-scored and are designed to measureskills and knowledge that are thought to be taught to all students in a fairlystandardized way. However, performance assessments can also be standard-ized if they are administered and scored in the same way for all students.Standardization is an important consideration if comparisons are to be madebetween scores.

Task (as in a "performance task"): a goal-directed assessment exercise. If thetask is authentic, it is designed to elicit from students their application of abroad range of knowledge and skills to solve a complex problem.

Validity: an indication of how well an assessment actually measures what itis supposed to measure. A valid assessment measures what it is supposed tomeasure and not extraneous features. For example, a valid assessment ofmathematics problem solving would be to measure the student's ability tosolve a problem, and not the ability to read the problem.

41 45

Appendix 2:An Overview of State Assessment Programs

T able 1 summarizes state assessment activity in each of the 50 states. Foreach state, the number of separate testing programs within each state, the

grade level and subjects tested, the type of assessment used, and the purposesof assessment are included. The totals at the bottom of the chart are columntotals.

Table 2 reports the grade levels and subjects tested, and the sampling methodused for each of the separate testing components within each state. The totalsalong the bottom of the chart are column totals.

4 643

TA

BLE

1.

SU

MM

AR

Y O

F S

TA

TE

AS

SE

SS

ME

NT

PR

OG

RA

MS

ST

AK

AL

AR

AZ CA

CO

0 M 2 6 3 3 3 3

Gra

des

11

1

K1

23

45

67

89

01

2

--

AV

A -

A-

V-

AA

AA

A A

AA

- A

SA

A S

A A

AS

A A

-A

A-

- A

A-

AA

- A

--

-AE

AE

E-

S-

-S

-S

-

Sub

ject

s

SS

ALM

WS

cVpR

AA

VA

AA

AA

AA

AS

AA

AA

AA

A A

EE

E -

SS

S -

S

Tes

t Typ

eN

C

RR

PP

WT

Trf

SA

VA

A-

-A

AA

--

S

AA

AA

A-

VA

AA

SS

S -

SC

T2

--A

-A-A

A-

AA

AA

A-

- A

A-A

DE

2A

- A

A-

AA

AA

AA

AA

-A

--

-A

FL

3-

S-

--

SA

A-

AA

AA

AA

GA

5A

-A

-A

--

A-

AA

AA

AA

AA

AA

-A

H I

3A

--

A-

AV

AV

VA

A -

VV

AV

--

-IA

0ID

2A

AA

AA

AA

AA

AI L

1A

A-A

AA

-A

A-

AA

AA

AA

A-

-A

I N1

AA

- -A

- A

A-

--

AA

- A

AA

A-

--

KS

3A

A-

-A

--

A-

-A

AA

AA

A-

AK

Y3

-A

--

-A

--

-A

AA

AA

AS

-S

ALA

4A

AA

AA

A -

AA

A -

AA

A A

AA

AA

--

AM

A1

-A

--

-A

--

-A

AA

- A

A-

-A

-A

MD

2A

- A

- A

AA

A -

-A

AA

AA

- A

A-

AM

E1

-A

--

-A

--

A-

AA

A A

AA

--

AA

MI

1-

AA

- A

A-

AA

-A

A-

-A

-A

--

-M

N1

--S

SS

-SS

-S-

SS

S S

SM

O2

S_

-S-

-S-S

S-

SS

SS

S-

S-

-S

MS

4-

AA

A-

AE

EA

EA

AA

A-

AA

--

AM

T1

A-

--

-A

--

A-

AA

-A

AA

NC

1A

AA

A A

A A

EE

EA

AA

AA

-A

--

-A

ND

2-

A-

-A

-A

--

A-

AA

-A

AA

-A

NE

0N

H0

NJ

3A

AR

AR

AA

AA

A-

AN

M4

- A

A A

AA

A-

A-

A-

-A

AA

AA

AA

AA

AN

V3

-A

- -A

- -A

- A

RA

AA

A-

-A

NY

7-

AA

AA

-A

AA

AE

AA

AA

AE

- A

A-A

OH

2-

- A

- A

- A

AA

RR

AA

AA

A-

-A

OK

3-

A-

A-

A-

AA

A -

AA

AA

AA

AA

-A

OR

2-

A-

A-

-A

--

A-

AA

AA

AA

PA

1A

AA

AA

RA

AA

AA

AR

I4

--

AA

-A-A

-A-

-A

AA

AA

A -

AS

C2

AA

AA

AA

A A

AA

AR

AA

A -

A-

AA

A -

AS

D2

--

-1i

--

- A

A-

A-

AA

- A

A A

-A

A-

--

TN

3-

AA

AA

AA

AA

AV

-A

AV

AA

-A

A-

-V

TX

2-

- A

A A

AA

AA

A A

RA

A A

AA

-A

A-

-A

UT

2V

V V

VV

AV

V A

V V

AV

AA

-A

A-

AA

--

AV

A2

--

-A-A

- A

--

A-

AA

AA

AA

A-

-A

VT

3-

--

A-

--

A-

--

-A

AA

AA

WA

1-

--

A-

--

A-

-A

-A

A-

AA

--

A-

-A

WI

2-

-A

--

--

A-

A-

-A

AA

-A

--

AA

--

AW

V3

AA

AA

AA

-A

AA

A-

AA

AA

A-

-A

A-

-A

WY

3=

Ass

essm

ent P

urpo

se

PA

Sc

KD

PG

SIE

AcA

A0

YY

YY

YY

YY

-Y

YY

YY

-Y

YY

YY

YY

YY

Y-

Y

YY

YY

YY

YY

YY

YY

YY

YY

YY

YY

YY

YY

YY Y

YY

YY

YY

YY

YY

YY

Y Y

YY

YY

YY

YY

Y

YY

YY

YY

YY

YY

YY

Y

YY

YY

YY

Y-

--

--

YY

YY

--

Y

YY

YY

YY

Y Y

YY

YY

YY

YY

-Y

YY

YY

YY

Y-

YY

YY

YY

YY

YV

Y-

YY

YY

YY

- Y

YY

-Y

-Y

YY

- Y

YY

YY

- Y

YY

YY

-Y

YV

Y-

YY

YY

-r

YY

Y-

YY

-YY

YY

-Y

Y

Y

YY

YY

TO

TA

L3

57

26 3

1 22

26

15 4

1 22

27

35 1

6 45

46

36 3

1 34

42

3 .3

4 34

17

6 38

4 23

9 18

8 39

30

41 1

08

5 14

47

LEG

EN

D:

CE

LL C

ON

TE

NT

SA

= A

ll ar

e te

sted

S =

Sam

ple

is te

sted

E =

End

-of-

cour

se o

r -p

rogr

am te

stin

gV

= V

olun

tary

= R

epea

t tes

ting

Y =

Pur

pose

app

lies

at le

ast o

neco

mpo

nent

#CO

M =

Num

ber

of a

sses

smen

tco

mpo

nent

s

SU

BJE

CT

S

L =

Lan

guag

e, in

clud

ing

read

ing

M =

Mat

hem

atic

sW

= W

ritin

gS

S =

Soc

ial s

tudi

esS

c =

Sci

ence

V =

Voc

atio

nal

Ap

= A

ptitu

deR

= R

eadi

ness

TE

ST

TY

PE

NR

T =

Nor

m-r

efer

ence

d te

stC

RT

= C

riter

ion-

refe

renc

ed te

stP

r =

Per

form

ance

test

ing

Pf =

Por

tfolio

ass

essm

ent

WS

= W

ritin

g sa

mpl

es

AS

SE

SS

ME

NT

PU

RP

OS

EK

= K

inde

rgar

ten

or g

rade

one

rea

dine

ssD

= S

tude

nt d

iagn

osis

or

plac

emen

tP

= S

tude

nt p

rom

otio

nG

= H

igh

scho

ol g

radu

atio

nS

= H

igh

scho

ol s

kills

gua

rant

eeI

= Im

prov

emen

t of i

nstr

uctio

nP

E =

Pro

gram

eva

luat

ion

A =

Acc

ount

abili

ty, s

choo

l per

form

ance

mon

itorin

gA

c =

Sch

ool a

ccre

dita

tion

SA

= S

tude

nt a

war

ds o

r re

cogn

ition

ScA

= S

choo

l aw

ards

or

reco

gniti

on0

= O

ther

EX

PLA

NA

TO

RY

NO

TE

: Mos

t sta

tes

cond

uct s

ever

al a

sses

smen

t pro

gram

s S

ide-

by-s

ide

(see

the

#CO

M c

olum

n). T

his

tabl

e ag

greg

atos

acr

oss

thes

e as

sess

men

t "co

mpo

nent

s" to

the

stat

e le

vel.

It m

ust b

e in

terp

rete

d ca

utio

usly

, em

phas

izin

g th

e te

rrr.

"at

leas

t" in

the

follo

win

g se

nse:

Ala

ska

cond

ucts

at l

east

one

ass

essm

ent p

rogr

am a

sses

sing

at l

east

SO

ME

four

thgr

ader

s in

lang

uage

art

s, m

athe

mat

ics,

oi w

ritin

g; it

als

o co

nduc

ts a

t lea

st o

ne a

sses

smen

t of S

OM

E fi

fth g

rade

rs in

lang

uage

art

s, m

athe

mat

ics,

OR

writ

ing;

Ala

ska

mak

es u

se o

f an

NR

T a

nd a

writ

ing

sam

ple;

it te

sts

to im

prov

e in

stru

ctio

n, fo

r pr

ogra

m e

valu

atio

n, a

nd fo

r ac

coun

tabi

lity

purp

oses

. The

tota

ls a

t the

bot

tom

of t

he ta

ble

are

read

sim

ilarly

: the

se s

tate

sea

ch c

ondu

ct a

t lea

st o

ne o

f the

se a

sses

smen

t "co

mpo

nent

s."

48

TABLE 2

SUMMARY OF STATE ASSESSMENT PROGRAM COMPONENTS

ST

AKAKAKAK

ALALALAL

ALALALALALAL

ALALALALALAL

ARARARARARARARARARARAR

AZAZAZAZAZ

AZAZAZAZ

CACACACACACA

CA

CACA

CACACACACACA

CA

1 LEGEND: A . All are testedS = Sample is testedE = End-of-course or program testing for all participantsV = Testing is voluntaryR = Repeat testing till pass

---- Grades Assessed ----1 1 1

K 1 2 3 4 5 6 7 8 9 0 1 2 Program Component -- Subject Assessed

A A

Norm-Referenced Testing -- Language Arts- Norm-Referenced Testing Math- Norm-Referenced Testing Reading

V Writing Writing

A

AA

A

AA

AA

A- A- A- A

- A- -

A

A

A

AAA

AA

A

- AA

AA

AAAA -

A -- -

E EE E

E EE EE E

EEEEEEEEEEEEEEEEEEEEEEEE

A A- - A - A -

A A -s A A

- A - A - -

Alabama Basic Competency Tests -- Language ArtsAlabama Basic Competency Tests -- MathematicsAlabama Basic Competency Tests -- ReadingAlabama Direct Assessment of Writing -- WritingAlgebra I End-of-Course Test -- MathematicsDifferential Aptitude Test w/Career Interest -- AptitudesHigh School Graduation Examination -- LanguageHigh School Graduation Examination -- MathematicsHigh School Graduation Examination -- ReadingIntegrated Reading/Writing Assessment, Gr. 2 Reading ComprehensionIntegrated Reading/Writing Assessment, Gr. 2 -- WritingStanford Achievement Test -- LanguageStanford Achievement Test -- MathematicsStanford Achievement Test -- ReadingStanford Achievement Test -- ScienceStanford Achievement Test -- Social Studies

ArkansasArkansasArkansasArkansasArkansasStanfordStanfordStanfordStanfordStanford

Minimum Performance TestMinimum Performance TestMinimum Performance TestMinimum Performance TestMinimum Performance TestAchievement Test -Achievement Test -Achievement TestAchievement TestAchievement Test -

-- Language Arts-- Math-- Reading-- Science-- Social Studies

- Language- Math- Reading- Science- Social Studies

Writing Assessment -- Writing

District Assessment Plans -- MathDistrict Assessment Plans -- ReadingDistrict Assessment Plans -- WritingNorm-Referenced Tests -- Language ArtsNorm-Referenced Tests -- ':athNorm-Referenced Tests -- ReadingStatewide Performance-Based AssessmentStatewide Performance-Based AssessmentStatewide Performance-Based Assessment

Math-- Reading-- Writing

Career-Technical Assessment Project (C-TAP)Career-Technical Assessment Project (C-TAP)Career-Technical Assessment Project (C-TAP)Career-Technical Assessment Project (C-TAP)Career-Technical Assessment Project (C-TAP)Golden State Exams (HS end-of-course)Golden State Exams (HS end-of-course)Golden State Exams (HS end-of-course)Golden State Exams (HS end-of-course)Golden State Exams (HS end-of-course)Golden State Exams (HS end-of-course)Performance Assessment -- HistoryPerformance Assessment -- MathPerformance Assessment -- ReadingPerformance Assessment -- SciencePerformance Assessment -- Writing

454 9

-- Animal Science-- Computer Sci/Info Systems-- Construction Tech-- Health Info Services-- Home Ec./Child Care

-- AlgebraBiologyChemistryEconomicsGeometryU.S. History


i

LEGEND: A = All are testedS = Sample is testedE = End-of-course or program testing for all participantsV = Testing is voluntaryR = Repeat testing till pass


ST K 1 2 3 4 5 6 7 8 9 0 1 2 Program Component -- Subject Assessed

CO - - - S - - S - S - ITBS/TAP -- Math

CO - -S- -S- -S- - ITBS/TAP -- Reading

CO - -S- -S- -S- - Math and Science Assessment Math

CO - - -S- -S- -S- - Math and Science Assessment Science

CO - - -S- -S- -S- - Writing -- Writing

CT A Connecticut

CT A Connecticut

CT A Connecticut

CT A ConnecticutCT -A-A-A- - Connecticut

CT -A-A-A- - ConnecticutCT -A-A-A- - Connecticut

CT -A-A-A- - - - Connecticut

Academic PerformanceAcademic PerformanceAcademic PerformanceAcademic PerformanceMastery Test (CMT) --Mastery Test (CMT) --Mastery Test (CMT) -Mastery Test (CMT)

Test (CAPT) -- Language ArtsTest (CAPT) MathTest (CAPT) Science

Test (CAPT) Social StudiesLanguage ArtsMathReading

- Writing

DE -A- -A-A- -A- Stanford Achievement Test -- Language ArtsDE - -A- -A-A- -A- Stanford Achievement Test -- MathDE - - -A- -A-A- -A- Stanford Achieveme, Test -- Reading

DE -A- -A-A- -A- Stanford Achievem , Test -- Science

DE - - -A- -A-A- -A- Stanford Achievement Test -- Social StudiesDE -A-A- -A-A- Writing -- Writing

FL -S- - -S-S- Florida Writing Assessment Program -- WritingA Grade Ten Assessment Test (GTAT) -- Math

FL A Grade Ten Assessment Test (GTAT) -- Reading

FL A High School Competency Test (HSCT) -- Math

FL A High School Competency Test (HSCT) -- Reading

FL A High School Competency Test (HSCT) -- Writing

GA A A Curriculum-Based Assessments (CBA) -- Health Education

GA - -A-A- -A- - - Curriculum-Based Assessments (CBA) -- Math

GA - -A-A- -A- - - - Curriculum-Based Assessments (CBA) -- Reading

GA - - -A-A- -A- - - Curriculum-Based Assessments (CBA) Science

GA -A-A- -A- - Curriculum-Based Assessments (CBA) -- Social Studies

GA A High School Graduation Tests (GHSGT) -- English Language Art

GA A High School Graduation Tests (GHSGT) -- Health EducationGA A High School Graduation Tests (GHSGT) Math

GA A High School Graduation Tests (GHSGT) -- ScienceGA A High School Graduation Tests (GHSGT) -- Social StudiesGA A High School Graduation Tests (GHSGT) Writing

GA A Kindergarten Assessment Program (GKAP) -- Communication

GA A Kindergarten Assessment Program (GKAP) Logical/Mathematical

GA A Kindergarten Assessment Program (GKAP) Personal/Physical

GA A Kindergarten Assessment Program (GKAP) -- Social

GA - -A-A- -A- Norm-Referenced Testing Math

GA - -A-A- -A- - - - Norm-Referenced Testing -- ReadingGA A Norm-Referenced Testing (HS) Math

GA A Norm-Referenced Testing (HS) Reading

GA A Norm-Referenced Testing (HS) -- Science

GA A Norm-Referenced Testing (HS) -- Social StudiesGA A Norm-Referenced Testing (HS) Writing

GA - - -A-A- -A- - Writing Assessment -- Writing

HI VVVVV Credit by Examination (CbyE) Content Areas (8)

HI A R R Test of Essential Competencies (HSTEC) 15 competencies

HI - - -A- -A-A-A- - Stanford Achievement Test -- LanguageHI -A- -A-A-A- - Stanford Achievement Test -- Math

HI - -A- -A-A-A- - Stanford Achievement Test -- Reading

IA NO STATE ASSESSMENT PROGRAM

465 0


I LEGEND: A = All are testedS = Sample is testedE = End-of-course or program testing for all participantsV = Testing is voluntaryR = Repeat testing till pass

Grades Assessed ----1 1 1


ID

ID

ID

ID

ID

ID

ID

ID

ID

A A Direct Writing Assessment (DWA) -- WritingA A Norm-ReferencedA A A Norm-ReferencedA A A Norm-ReferencedA A A Norm-ReferencedA A A Norm-ReferencedA A Norm-ReferencedA A Norm-Referenced

A Norm-Referenced

IL - A - AIL - A - AIL A - - AIL A A

IL - - A - - A

IN - - A A A -IN - A A AIN - - A - AIN A - A -

KSKSKS

KY

KY

KYKY

KY

KY

KYKY

KY

KY

KY

KY

A AA AA -

A -

- - -A- -A- A- -A- -A- A

LA ALA A A ALA A A ALA ALA

LA

LA

LALA - - -A-A- - ALA - -A-A- ALA - -A-A- - ALA - -A-A- ALA - -A-A- - A

MAMAMAMA

IllinoisIllinoisIllinoisIllinoisIllinois

Testing -- Language ArtsTesting MathTesting -- ReadingTesting -- ScienceTesting -- Social StudiesTesting -- Study SkillsTesting -- VocabularyTesting -- Written Expression

Goals Assessment Program -Goals Assessment Program -Goals Assessment Program -Goals Assessment Program -Goals Assessment Program -

Statewide AssessmentStatewide AssessmentStatewide AssessmentStatewide Assessment

Mathematics -- MathReading ReadingWriting -- Writing

A KIRIS transitionalA KIRIS transitional

- A KIRIS transitionalA KIRIS transitional

- A KIR1S transitionalS Performance eventsS Performance events

- S Performance eventsS Performance events

- S Performance eventsA Portfolio

- A Portfolio

Math- Reading- Science- Social StudiesWriting

MathLanguage ArtsScienceSocial Studies

assessment -- Mathassessment -- Readingassessment -- Scienceassessment -- Social Studiesassessment -- Writing-- Arts 8 Humanities

Career/Voc. EducatioNMath

-- Science-- Social Studies

assessment -- Mathassessment -- Writing

Kindergarten Develop. Readiness Screen. Prog. -- Developmental AreasLA Education Assesment Program Language ArtsLA Education Assesment Program -- MathLA Education Assesment Program -- Writing

A LA Graduation Exit Examination -- MathA LA Graduation Exit Examination -- ScienceA LA Graduation Exit Examination -- Social Studies

A LA Graduation Exit Examination -- Writing- LA Statewide Norm-Referenced Testing Program -- Language Arts

LA Statewide Norm-Referenced Testing Program -- MathLA Statewide Norm-Referenced Testing Program -- ReadingLA Statewide Norm-Referenced Testing Program -- Science

- LA Statewide Norm-Referenced Testing Progran -- Social Studies

- A

AA

- A

Massachusetts Education Assessment Program -- MathMassachusetts Education Assessment Program -- ReadingMassachusetts Education Assessment Program -- ScienceMassachusetts Education Assessment Program -- Social Studies

4751


1 LEGEND: A = All are testedS = Sample is testedE = End-of-course or program testing for all participantsV = Testing is voluntaryR = Repeat testing till pass


ST K 1 2 3 4 5 6 7 8 9 0 1 2 Program Component

MDMOMDMDMDMDMDMDMD

ME

MEMEMEMEME

MI

MI

MI

MNMNMN

MN

MNMNMNMNMN

- A - A - - -

- A - - A - -

A - - A - - - -

- A - - A - -

A - - A -

AA A

A VV A

A -

A -AA - -

A - -

A - -

A -

A - -

A

MO S

MO S

MOMOMO

MSMS

MSMS

MS

MS

MSMS

MSMS

MSMS

MT

MT

MT

MT

MT

- A

A

A

- A- A

A

A - - AA - A

A

AAA

-- Subject Assessed

CRTs -- MathCRTs -- ReadingCRTs -- ScienceCRTs SociaL StudiesCRTs -- WritingHigh School GraduationHigh School GraduationHigh School GraduationHigh School Graduation

State TestsState TestsState TestsState TestsState TestsState Tests

RequirementRequirementRequirementRequirement

-- Humanities-- Math-- Reading-- Science-- Social Studies-- Writing

Citizenship-- Math-- Reading-- Writing

Michigan Educational Assessment Program-MEAP --Michigan Educational Assessment Program-MEAP

A Michigan Educational Assessment Program-MEAP

Minnesota TestingMinnesota TestingMinnesota TestingMinnesota TestingMinnesota TestingMinnesota TestingMinnesota TestingMinnesota TestingMinnesota Testing

ProgramProgramProgramProgramProgramProgramProgramProgramProgram

English ConventionsHealth EducationMathMusicReadingScienceSocial StudiesVisual ArtsWriting

S S Missouri Mastery and Achievement Test --S S Missouri Mastery and Achievement T:st --S S Missouri Mastery and Achievement Test --S S Missouri Mastery and Achievement Test --S Writing Assessment -- Writing

A - A A -A - A - AA - A - A - -

A A - AA - A A

E E E

A RA RA R

-

E E

A -A -A

A -

A

Basic Skills Assessment Program, Grade 5Basic skills Assessment Program, Grade 5Basic Skills Assessment Program, Grade 5Functional Literacy Examination (FLE) --Functional Literacy Examination (FLE) --Functional Literacy Examination (FLE)Stanford Achievement Test -- LanguageStanford Achievement Test -- MathStanford Achievement Test -- ReadingStanford Achievement Test -- ScienceStanford Achievement Test Social StudiesSubject Area Testing Program (SATP) -- Algebra

MathReadingScience

MathReadingScienceSocial Studies

-- Math-- Reading-- WritingMathReadingWriting

Student Assessment RequirementStudent Assessment Requirement -Student Assessment Requirement -Student Assessment RequirementStudent Assessment Requirement -

48

- Language Arts- Math- ReadingScience

- Social Studies

52


I LEGEND: A = All are testedS = Sample is testedE = End-of-course or program testing for all participantsV = Testing is voluntaryR = Repeat testing till pass

Grades AsFessed1 1 1


NC EEEE North Carolina Testing Program -- Algebra/GeometryNC EEEE North Carolina Testing Program -- H.S. ScienceNC EEEE North Carolina Testing Program -- H.S. Social StudiesNC - -AAAAAA- - - North Carolina Testing Program -- MathNC - -AAAAAA- - - - North Carolina Testing Program -- ReadingNC - -AAAAAA- - - - North Carolina Testing Program -- ScienceNC - - -AAAAAA- - North Carolina Testing Program -- Social StudiesNC A - A A North Carolina Testing Program -- Writing

ND -A-A- -A- State-Wide Standardized Testing -- LanguageND -A-A- -A- State-Wide Standardized Testing -- MathND -A-A- -A- State-Wide Standardized Testing -- ReadingND -A-A- -A- State-Wide Standardized Testing -- ScienceND -A-A- -A- State-Wide Standardized Testing -- Social StudiesND -A-A- -A- State-Wide Standardized Testing -- SpellingND A A A State-Wide Standardized Testing -- Study SkillsND A State-Wide Standardized Testing -- Word AnalysisND - -A-A- -A- State-Wide Standardized Testing (Cognitive) -- AnalogiesND - -A-A- -A- State-Wide Standardized Testing (Cognitive) MemoryND - -A-A- -A- State-Wide Standardized Testing (Cognitive) SequencesND -A-A- -A- State-Wide Standardized Testing (Cognitive) Verbal Reasoning

NE NO STATE ASSESSMENT PROGRAM

NH NO STATE ASSESSMENT PROGRAM

NJ A Grade 8 Early Warning Test -- MathNJ A Grade 8 Early Warning Test -- ReadingNJ A Grade 8 Early Warning Test -- WritingNJ A R R R* Grade 9 High School Proficiency Test MathNJ A R R R* Grade 9 High School Proficiency Test -- ReadingNJ A R R R* Grade 9 High School Proficiency Test -- WritingNJ A R* Grade 11 High School Proficiency Test -- MathNJ A R* Grade 11 High School Proficiency Test -- ReadingNJ A R* Grade 11 High School Proficiency Test -- Writing

NM A High School Competency Examination -- Language ArtsNM A High School Competency Examination -- MathNM A High School Competency Examination -- ReadingNM A High School Competency Examination -- ScienceNM A High School Competency Examination -- Social StudiesNM A High School Competency Examination WritingNM A - Norm-Referenced Standardized Testing Program -- MathNM A - Norm-Referenced Standardized Testing Program -- ReadingNM - A Norm-Referenced Standardized Testing Piogram -- SpellingNM A Norm-Referenced Standardized Tasting Program -- VocabularyNM - A A Reading Assessment for Grades 1 and 2 -- ReadingNM - - -A-A-V- - - Writing Assessment -- Writing

NV A A Analytic-Trait Scored Writing -- WritingNV A A CTBS -- LanguageNV A - - A - - CTBS -- MathNV A - - A - - CTBS -- ReadingNV A R High School Proficiency Examination Program MathNV A R High School Proficiency Examination Program ReadingNV A R High School Proficiency Examination Program Writing

495 3


1 LEGEND: A = All are testedS = Sample is testedE = End-of-course or program testing for all participantsV = Testing is voluntaryR = Repeat testing till pass



NY

NY

NYNY

NY

NYNY

NYNY

NY

NYNY

NY

NY

NY

NY

NY

NY

NY

A

E E E -

A AA AA A

AA

A

A

A AA

V V

Occupational Education Proficiency Exams -- Career/Voc. EducationPreliminary Competency Tests -- ReadingPreliminary Competency Tests -- WritingProgram Evaluation Tests -- ScienceProgram Evaluation Tests Social StudiesPupil Evaluation Program (PEP) -- MathPupil Evaluation Program (PEP) -- ReadingPupil Evaluation Program (PEP) -- WritingRegents Competency Tests -- MathRegents Competency Tests ReadingRegents Competency Tests ScienceRegents Competency Tests Social StudiesRegents Competency Tests Writing

-- English-- Foreign Languages-- Math-- Science-- Social StudiesExaminations -- Foreign

Regents Examination ProgramsRegents Examination ProgramsRegents Examination Programs

E Regents Examination ProgramsRegents Examination ProgramsSecond Language Proficiency

OH ARRROH ARRROH ARRROH ARRROH --A-A-A-A--OH --A-A-A-A--OH --A-A-A-A--

Ninth-Grade Proficiency TestNinth-Grade Proficiency TestNinth-Grade Proficiency TestNinth-Grade Proficiency TestNorm-Referenced AchievementNorm-Referenced AchievementNorm-Referenced Achievement

OK A A A CRT -- MathOK A CRT -- ReadingOK A A A CRT -- ScienceOK A CRT -- WritingOK -A-A-A A- Norm-ReferencedOK -A-A-A A- Norm-ReferencedOK -A-A-A A- Norm-ReferencedOK -A-A-A A- Norm-ReferencedOK -A-A-A A- Norm-ReferencedOK A A Norm-Referenced

OR

OR

OR

OR

PAPA

PA

RI

RI

RI

RI

RI

- A- A

A- A

AA

A

A -- -

A -A

AA -

A

A -

A -A

A A RA A R

A A

A - A - A -

A A -

- A - A

A - A - A -

A

ing -- Citizenshiping -- Mathing -- Readinging -- WritingTests -- LanguageTests -- MathTests Reading

Achievement TestingAchievement TestingAchievement TestingAchievement TestingAchievement TestingWriting Assessment

Languages

LanguageMathReading

-- Science-- Social Studies-- Writing

Reading, Mathematics and Health -- Health EducationReading, Mathematics and Health -- MathReading, Mathematics and Health -- ReadingWriting -- Writing

Reading/Writing/Math State Assessment -- MathReading/Writing/Math State Assessment -- ReadingReading/Writing/Math State Assessment -- Writing

Health -- Health EducationNorm-Referenced Reading and Math -- MathNorm-Referenced Reading and Math -- ReadingPhysical Fitness -- Physical EducationWriting -- Writing

5054


i LEGEND: A = All are testedS = Sample is testedE = End-of-course or programV = Testing is voluntaryR = Repeat testing till pass


K 1 2 3 4 5 6 7 8 9 0 1 2 Program Component

A A A -AA A A

testing for all participants

-- Subject Assessed

-A-A-ARR Basic Skills AssessmentBasic Skills Assessment

-A-A-ARR Basic Skills Assessment- A Basic Skills Assessment

A A ARRBasic Skills AssessmentA -A-A-A- Norm-Referenced ProgramA -A-A-A- Norm-Referenced ProgramA -A-A-A- Norm-Referenced Program

A - AA - A

A - AA A

A

-AAAAAAA-AAAAAAA-AAAAAAA-AAAAAAA-AAAAAAA

A

-AAAAAAAAA--AAAAAAAAA--AAAAAAAAA--AAAAAAAAA--AAAAAAAAA-

A -A-A-ARA -A-A-ARA -A-A-AR

VVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVV

A A AA A AA A A

A A A

A A A

AA

A

Program MathProgram -- ReadinessProgram -- ReadingProgram -- ScienceProgram -- Writing

Language/English-- Math-- Reeding

Achievement and Ability Testing -- MathAchievement and Ability Testing -- ReadingAchievement and Ability Testing -- ScienceAchievement and Ability Testing -- Social StudiesCareer Assessment Program -- Career/Voc. Educatio

Achievement Test --Achievement Test -Achievement Test -Achievement TestAchievement Test -Competency Test --Writing Assessment

LanguageMath

- ReadingScience

- Social StudiesMinimum Competency-- Writing

Norm-Referenced Assessment Program for TexasNorm-Referenced Assessment Program for TexasNorm-Referenced Assessment Program for TexasNorm-Referenced Assessment Program for TexasNorm-Referenced Assessment Program for TexasTexas Assessment of Academic Skills (TAAS)Texas Assessment of Academic Skills (TAAS) --Texas Assessment of Academic Skills (TAAS)

Core Curriculum AssessmentCore Curric.alum AssessmentCore Curriculum AssessmentStatewide Testing ProgramStatewide Testing ProgramStatewide Testing ProgramStatewide Testing ProgramStatewide Testing Program

Competency TestCompetency TestCompetency TestNorm-ReferencedNorm-ReferencedNorm-ReferencedNorm-Referenced

Program -- MathProgram -- ReadingProgram -- Science-- Language Arts-- Math-- Reading-- Science-- Social Studies

(Literacy Passport)(Literacy Passport) --(Literacy Passport) --Test -- MathTest -- ReadingTest -- ScienceTest -- Social Studies

LanguageMath

-- Reading-- Science-- Social StudiesMathReadingWriting

MathReadingWriting

Portfolio: Writing and Mathematics MathPortfoLio: Writing and Mathematics WritingUniform Test: Mathematics MathUniform Test: Writing -- Writing

Basic Assessment Program -- MathB2sic Assessment Program -- ReadingBasic Assessment Program -- ScienceBasic Assessment Program -- Social Studies

515 5


i LEGEND: A = All are testedS = SampLe is tested

i

IE = End-of-course or program testing for all participants

1

V = Testing is voluntary1

iR = Repeat testing till pass


ST K 1 2 3 4 5 6 7 8 9 0 1 2 Program Component Subject Assessed

WI A

WI

WI

WI

WI

WI

Third Grade Reading Test (TGRT) -- ReadingA A Wisconsin Student Assessment Program (WSAS) -- EnglishA A Wisconsin Student Assessment Program (WSAS) -- MathA A Wisconsin Student Assessment Program (WSAS) -- ReadingA A Wisconsin Student Assessment Program (WSAS) -- ScienceA A Wisconsin Student Assessment Program (WSAS) -- Writing

WV -A- -A- -A-A-WV -A- -A- -A-A-WV - -A- -A- -A-A-WV -A- -A- -A-A-WV -A- -A- -A-A-WV - -A- -A- -A-A-WV -A- -A- -A-A-WV - AAAAAAWV - AAAAAAWV - -A- - -A-A

WY

Comprehensive Tests of Basic Skills --Comprehensive Tests of Basic Skills --Comprehensive Tests of Basic Skills --Comprehensive Tests of Basic Skills --Comprehensive Tests of Basic Skills --Comprehensive Tests of Basic Skills --Comprehensive Tests of Basic Skills --Criterion-Referenced Tests -- MathCriterion-Referenced Tests -- ReadingWriting Assessment -- Writing

NO STATE ASSESSMENT PROGRAM

LanguageMathReadingScienceSocial StudiesSpellingStudy Skills

Notes: AZNJ

NY

Districts must report at grades 3, 8, and 11. They may test more grades.Grade 9 being phased out; Grade 11 will replace.Schools choose whether to test in grade 8 or 9.

Acknowledgments

A special thank you goes to Jeri Nowakowski, Executive Director of North Central RegionalEducational Laboratory, for the initial conception of this project ani for the generousprovision of staff to prepare this paper. Another special thanks goes to Rod Riffel whoguided this project for the NEA and remained patient and helpful through the revisions ofnumerous drafts. He and others at NEA, as well as Dean Arrasmith of Northwest RegionalEducational Laboratory, Julie Bell of the National Council of State Legislators, PatriciaCeperley of Appalachia Educational Laboratory, Gary Estes of the Far West Laboratory, JoanHerman of the National Center for Research on Evaluation, Standards, and Student Testing,Wes Hoover of the Southwest Educational Development Laboratory, Diane Kell of TheRegional Laboratory for Educational Improvement of the Northeast and Islands/TheNetwork, Inc., and Andrew Porter of the Wisconsin Center for Education Research, allreviewed drafts of the report and offered very helpful suggestions for revision. We areespecially grateful to the state testing directors and their staff who were interviewed andreviewed drafts of the document Sue Bennett, California; Richard Peters and Donna Long,Indiana; Bob Gabrys, Steve Ferrara and Barbara Kapinus, Maryland; Diane Smolen andPeggy Dutcher, Michigan; Phyllis Stolpe, Texas; and Susan Rigney, Vermont. Shannon Cahilland Felicia Bohannon of NCREL conducted telephone interviews and summarized them, andCraig Sauter assisted with the re-writing of the opening sections of the report. DeannaDurrett, Director of the Regional Policy Information Center at NCREL, and Dena Stoner,Director of CEDaR, provided wise counsel concerning the recommendations and conclusions.And finally, most sincere thanks to Ullik Rouk of CEDaR who spent countless hoursreviewing each draft of the report, providing considerable feedback and support, and editingthe report into its final form.

The Authors

5 7

neaNADONAL EDUCATION ASSOCIATION

CEDEIRCouncil for Educational Development and Research

BEST COPY AVAILABLE

document resume ed 367 664 author bond, linda; and others · 2014-05-07 · document resume. tm 021...

Documents