Making Numbers Work…

NHSAA: Living with the NCLB Act

Ann RemusSeptember 21, 2004

To Improve Instruction

Where to Begin?…one analysis at a time

Examples:

I. Program evaluation

II. Authentic assessment that fosters good teaching

III. Diagnosis to allow differentiated instruction/value-added assessment

I. Program Evaluation

Growth by Grade on a Norm-referenced test: Gates-MacGinite Reading Tests

We used to disaggregate by hand for Title I and SPED – led to program improvements

Difficulty keeping consistency of early skills program when grades were split among three schools in 99-00.

Gates MacGinitie Total Battery Extended Scale ScoreBy Grade Over Time

1999-2003

416 419424

432 432

474 477 475481 484

501 500 501505 505

535 538533 536 536

400

420

440

460

480

500

520

540

1999 2000 2001 2002 2003 1999 2000 2001 2002 2003 1999 2000 2001 2002 2003 1999 2000 2001 2002 2003

Grade 1 Grade 2 Grade 3 Grade 5

Grade Level Over Time

To

tal

Ba

tte

ry E

SS

Growth By Grade, Gates-MacGinite

1st Grade+16

2nd Grade+9

3rd Grade+4

5th Grade+1

Note: Black lines indicate national median Total Battery ESS for each grade level. 1st Grade: 391, 2nd Grade: 446, 3rd, Grade: 472, 5th Grade: 507.

Growth By Grade: Observations

District focus/professional development on early skills may account for growth in grades 1 and 2

The data: identifies the need to investigate upper

level elementary grades’ potential for increased growth

Identifies the need to disaggregate data By cohort By NCLB categories

Gates-MacGinite Average Total Battery ESS Grade 2

455464

468 472

498 502507 508

400

420

440

460

480

500

520

540

Going into 6thGrade

Going into 5thGrade

Going into 4thGrade

Going into 3rdGrade

Cohort

To

tal

Bat

tery

ES

S

Middle

Higher

Disaggregation: Stanine Ranges of Cohorts

National Median ESS: 446

Growth for Middle Stanine Range: 17 ESS

Growth for Higher Stanine Range: 10 ESS

Growth by Cohort, Disaggregated by Stanine

Ranges : Conclusions

Disaggregating cohorts helps us see which groups are growing more or less quickly

The middle stanine range improved more than did the higher stanine range

Progress of high and middle ranges against national median score has improved across time

Instructional implication: To “catch up,” the lower and middle groups must

continue to grow more quickly Need focused attention on best practices for the

upper group as well

Introduction of Open Court Reading Program

Prior series no longer available; wanted to make use of the research from the National Reading Panel in our programming

2002-2003: pilot, with one teacher in each grade, grades 1-4, in all three schools

2003-2004: first year of full district-wide implementation, grade 1

Average Total Score Gates MacGinitie Reading Test

(1999-2004)

416419

424

432 432

450

390

400

410

420

430

440

450

460

1999 2000 2001 2002 2003 2004

School Year

Ext

end

ed S

cal

e S

core

+16

+18

Comparison of Disaggregated Groups Grade 1 Gates Reading Tests Results

79

50

39

29

52

78

87

50

5962 60

97

0

10

20

30

40

50

60

70

80

90

100

BedfordAverage Grade

1

National Normfor Grade 1

SpecialEducation

ESL ReadingSupport

Math Apps

Groups

Per

cen

tile

Eq

uiv

alen

t o

f M

ean

Sco

re

2002-2003

2003-2004

Disaggregating grade 1 by NCLB categories after our first year of

Open Court reading

II. Authentic Assessment to Encourage Good Teaching

In the mid-1990’s, the district looked closely at its curriculum, deciding on essential questions and grade level expectations.

That work was continued with detailed curriculum maps.

One of the outcomes expected of grade 2 science, is understanding mass and volume.

measuring liquid volume and mass: following procedures and making

predictions

7484

5060708090

100

1999 2004

Year

Per

cen

t

Average Score

Source: Market ResearchSource: Market Research

Grade 2 Authentic Assessment

Leadership Seminar

Presented by Barbara GondekMay 4, 2004

Impact of Process Writing on Writing Assessment

Another Example of Authentic Assessment

Hypothesis

Students assigned to strong writing instructors in grades 7 and 8 will be shown to perform better on the Bedford Formal Writing Assessment than those who had average or weak writing instructors.

Parameters

Grades Seven and Eight Two years of scores (2002 and 2003) Sorted by Language Arts teachers Teachers were classified on the basis of

the extent to which they used all the steps of the Writing Process in their classrooms during 2002 and 2003.

1. Find an idea or topic. 2. Formulate a thesis - narrow the topic. 3. Generate details - facts, examples, and support. 4. Outline if it seems helpful. 5. Write your first draft. 6. Let things rest for a day or two. 7. Revise - look at the paper from the reader's point of view;

reorganize and add explanations where necessary. 8. Let things rest again. 9. Revise again and edit for conventions.10. Type or write the final draft.11. Proofread at least three times - once aloud, once for any

errors you habitually make, and once backwards.

Writing Process Steps

Classification of Teachers

Strong Average Weak

•Used Process Writing steps regularly•Required students to write every day

• Used Process Writing Steps occasionally •Required students to write 2-3 times per week

• Seldom used Process Writing•Most writing was “quick writes”

Extent of Process Writing vs. Student Writing Skills

67(263)

67(251)

73 (254)

0

20

40

60

80

100

Percent

Strong (392) Average (373) Weak (350)

Use of Writing Process

High Basic and Above Basic and Below

Extent of Process Writing vs. Special Education Student

Writing Skills

56(33)

59(30)

72(44)

0102030405060708090

100

Percent

Strong (59) Average (51) Weak (61)

Use of Writing Process

High Basic and Above Basic and Below

Current Writing Assessment Writing prompt is given at the

beginning of the class. Students are given a limited time

to organize ideas, brainstorm, and draft an essay.

Final copy is completed by the end of the testing session.

Writing Assessment Team

Representatives from Grades 1 – 8 worked together over the summer

Developed an assessment format and rubric that lines up with process writing.

1. Find an idea or topic. 2. Formulate a thesis - narrow the topic. 3. Generate details - facts, examples, and support. 4. Outline if it seems helpful. 5. Write your first draft. 6. Let things rest for a day or two. 7. Revise - look at the paper from the reader's point of view;

reorganize and add explanations where necessary. 8. Let things rest again. 9. Revise again and edit for conventions.10. Type or write the final draft.11. Proofread at least three times - once aloud, once for any

errors you habitually make, and once backwards.

Writing Process Steps

Assessment Plan for Fall 2004 Pre-assessment in September Post assessment in May-June Analytical scoring as opposed to

holistic Assessment given in stages similar

to process writing, over a period of days

Assessment is not the end of the writing process.

It is the bridge to revision.

III. Diagnostic/Value-Added Assessment

NWEA - given in grades 3 through 8 each fall and spring: piloted in 02-03, full implementation in 03-04.

Fall administration gives information necessary for immediate differentiation of instruction

Yearly growth by RIT ranges shows effectiveness of differentiation

7th Grade NWEA Fall to Spring RIT Growth

0.0

2.0

4.0

6.0

8.0

10.0

12.0

14.0

16.0

18.0

190-199 200-210 210-219 220-229 230-239 240-249 250-259

RIT Ranges

Fall t

o S

pri

ng

RIT

Gro

wth

McKelvie Average Fall to Spring Growth NWEA Growth Norms

n= 5 4 23 67 111 67 20

Grade 7 Math NWEA Fall to Spring Growth

2003-2004

Grade 7 Language UsageNWEA Fall to Spring Growth

0

1

2

3

4

5

6

7

200-209 210-219 220-229 230-239RIT Ranges

Fal

l to

Sp

rin

g R

IT g

row

th

McKelvieaverage fall tospring growth

NWEA nationalnorms

16 80 143 60

2003-2004

Gates-MacGinite total battery ESS vs. NWEA reading RIT total

Gates v NWEA

y = 0.32x + 46.04

R2 = 0.79

100

120

140

160

180

200

220

240

260

280

200 300 400 500 600 700 800

Gates (ESS)

NW

EA

(R

IT)

What to do? Collect data, backwards and

forwards Zero in on groups, smaller groups,

and then individuals Talk about what you see in the

data: different people may see different things

Keep at it…