datag annual testing accountabilty discussion 10 081
DESCRIPTION
David's Abrams presentation on October 5th - audio afterTRANSCRIPT
![Page 1: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/1.jpg)
DATAG: Annual State Assessment/Accountability Discussion
David AbramsAssistant Commissioner for Standards,
Assessment, and ReportingDATAG Fall Conference
October 3, 2008
![Page 2: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/2.jpg)
Achievement is up in English statewide, except in Grade 8.
69.0
%
68.6
%
67.1
%
60.4
%
56.4
%
49.3
% 61.5
%
67.1
%
68.0
%
68.1
%
63.2
%
57.8
%
57.0
%
63.4
%
70.1
%
71.1
%
77.6
%
66.9
%
70.0
%
56.1
% 68.5
%
Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Grades 3-8
2006 2007 2008
Percentage of Students Scoring at Levels 3 and 4
Grade 3 185,603 198,457 195,777Grade 4 190,951 197,499 197,016Grade 5 201,262 202,133 198,022Grade 6 204,249 204,463 200,505Grade 7 210,735 211,839 207,278Grade 8 212,320 213,971 209,180Grades 3-8 1,205,120 1,228,362 1,207,778
Number Tested 2006 2007 2008
2008 3-8 ELA results
![Page 3: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/3.jpg)
669
666
663
656
650667
665
661
655
655669
666
667
661
662
657
652665
Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8
2006 2007 2008
At all grade levels, the mean scale score was above 650, the score that denotes meeting the ELA
standards.
2008 3-8 ELA results
650
![Page 4: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/4.jpg)
More Students with Disabilities are meeting the English standards at every grade level,
except Grade 8.
26.6
%
26.5
%
26.6
%
16.8
%
16.1
%
10.5
%
20.2
%
28.0
%
27.6
%
29.1
%
19.7
%
17.3
%
15.6
%
22.8
%
30.8
%
29.8
%
40.7
%
23.9
%
29.4
%
13.3
% 27.9
%
Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Grades 3-8
2006 2007 2008
Percentage of Students with Disabilities Scoring at
Levels 3 and 4
Grade 3 23,811 26,692 27,285Grade 4 26,474 28,281 29,983Grade 5 28,987 29,985 30,661Grade 6 28,883 29,055 31,195Grade 7 29,237 29,842 31,180Grade 8 29,119 29,514 31,077Grades 3-8 166,511 173,369 181,381
Number Tested 2006 2007 2008
2008 3-8 ELA results
![Page 5: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/5.jpg)
The achievement gap in English is narrowing for Black and Hispanic students.
77.6
%
42.4
%
46.1
%
46.5
%
71.8
%
61.5
%75.6
%
45.2
%
45.6
%
50.8
%
75.4
%
63.4
%79.6
%
52.9
%
52.6
%
57.3
%
79.0
%
68.5
%
Asian/PacificIslander
Black Hispanic AmericanIndian/Alaskan
Native
White Total Public
2006 2007 2008
Percentage of Students Scoring at Levels 3 and 4
2008 3-8 ELA results
![Page 6: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/6.jpg)
The performance of ELL students on the English test improved overall.
34.3
%
26.6
%
20.6
%
11.2
%
8.5
%
4.9
%
16.2
%
30.5
%
23.0
%
18.9
%
10.4
%
7.0
%
5.9
%
18.0
%
33.1
%
31.9
%
34.0
%
14.9
%
17.4
%
6.3
%
25.1
%
Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Grades 3-8
2006 2007 2008
Percentage of ELL Students Scoring at Levels 3 and 4
2006 2007 2008Grade 3 3,684 17,093 17,433Grade 4 4,379 14,200 14,683Grade 5 6,686 11,480 11,916Grade 6 5,585 9,934 10,323Grade 7 6,234 9,299 9,798Grade 8 5,852 10,076 9,046Grades 3-8 32,420 72,082 73,199
Number of ELL Students Tested
2008 3-8 ELA results
![Page 7: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/7.jpg)
Achievement in math is up statewide. 80.5
%
77.9
%
68.4
%
60.4
%
55.6
%
53.9
% 65.8
%
85.2
%
79.9
%
76.1
%
71.2
%
66.4
%
58.8
% 72.7
%89.9
%
83.8
%
83.2
%
79.4
%
78.9
%
69.8
%
80.7
%
Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Grades 3-8
2006 2007 2008
Percentage of Students Scoring at Levels 3 and 4
Number of Students Tested 2006 2007 2008Grade 3 201,956 200,217 197,500Grade 4 202,791 199,391 198,730Grade 5 209,242 203,956 199,746Grade 6 211,428 206,220 202,058Grade 7 217,308 213,436 209,039Grade 8 219,414 215,415 210,716Grades 3-8 1,262,139 1,238,635 1,217,789
2008 3-8 math results
![Page 8: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/8.jpg)
677
676
666
656
652
685
674
668
663
657
688
683
680
675
674
666
651
680
Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8
2006 2007 2008
At all grade levels, the mean scale score was above 650, the score that denotes meeting the math
standard.
650
2008 3-8 math results
![Page 9: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/9.jpg)
More Students with Disabilities met the math standards at every grade level.
50.0
%
44.8
%
31.6
%
21.6
%
18.0
%
17.1
% 30.2
%
57.1
%
47.2
%
41.7
%
31.9
%
26.8
%
20.7
% 37.2
%
66.6
%
53.4
%
52.8
%
42.4
%
42.9
%
31.0
% 47.8
%
Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Grades 3-8
2006 2007 2008
Percentage of Students with Disabilities Scoring at Levels 3
and 4
Grade 3 27,045 26,780 27,325Grade 4 29,043 28,327 30,072Grade 5 30,290 29,960 30,662Grade 6 30,077 29,040 31,119Grade 7 29,791 29,659 31,037Grade 8 29,539 29,305 30,899Grades 3-8 175,785 173,071 181,114
Number Tested 2006 2007 2008
2008 3-8 math results
![Page 10: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/10.jpg)
The achievement gap in math is narrowing for Black and Hispanic students.
85.2
%
45.7
%
51.5
%
53.9
%
76.3
%
65.8
%
89.0
%
54.6
%
60.5
%
61.8
%
82.0
%
72.7
%
92.9
%
65.9
%
71.1
%
73.0
% 88.3
%
80.7
%
Asian/PacificIslander
Black Hispanic AmericanIndian/Alaskan
Native
White Total Public
2006 2007 2008
Percentage of Students Scoring at Levels 3 and 4
2008 3-8 math results
![Page 11: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/11.jpg)
Regents EnglishStudents Tested and Scores Received: 1996-2007
114 123135
176 166177 175 183
196208
219
153 150
91102 105
123 118136 130
140152 151
163 171
191192
182173171157
120113
159 152
0
50
100
150
200
250
1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007
Num
ber
in T
hous
ands
Tested 55-100 65-100
All Students
![Page 12: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/12.jpg)
Regents Mathematics Students Tested and Scores Received: 1996-2007
158 158171
192 192
166
212 217 227 235
120 130 139 146133
107
158
114 106 116 119 124105
81
131
176 177 185196
253
204225213204201
0
50
100
150
200
250
1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007
Num
ber
in T
hous
ands
Tested 55-100 65-100
Data for 1999–2002 include both Mathematics A and Sequential Mathematics, Course I.
Data for 2003 forward are for Mathematics A only.
All Students
![Page 13: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/13.jpg)
Regents Global History and Geography Students Tested and Scores Received: 1996-2007
122 131146
157174
187206
220232 243
181 183 188
92 86104
116
150136
148 152 153 154 153
206192174167
161149134
121103
173
128
0
50
100
150
200
250
1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007
Num
ber i
n Th
ousa
nds
Tested 55-100 65-100
The data for 2001 forward are for the Regents Global History and Geography examination only. The data for 2000 are for both the Regents Global History and Geography and Global Studies examinations. The data for previous years are for Regents Global Studies only.
All Students
![Page 14: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/14.jpg)
Regents U.S. History & Government Students Tested and Scores Received: 1996-2007
108119 126
139 144
176 179191
203 209
180 185
87 8696
104 108121
135150
142 145160 164
173
164 164156
165158
141
101111
127122
50
75
100
125
150
175
200
225
1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007
Num
ber
in T
hous
ands
Tested 55-100 65-100
All Students
![Page 15: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/15.jpg)
Regents Living Environment/Biology Students Tested and Scores Received: 1996-2007
110 106 114123 129
184 178188 185
207216
226
188199
75 80 81 88 91
144154 151 146
156 163 170
179165168167164
92 95105 109
50
75
100
125
150
175
200
225
250
1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007
Num
ber
in T
hous
ands
Tested 55-100 65-100
All Students
Data for 1996 through 2000 are for the Regents Biology examination. Data for 2001 are for both the Regents Biology and the Regents Living Environment examinations. Data for 2002 through 2005 are for the Regents Living Environment examination.
![Page 16: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/16.jpg)
16
New York City Large City Urban-Suburban
Rural Average Low Total Public
2001 Cohort 2002 Cohort 2003 Cohort
Percentage of Students Graduating with Regents or Local Diploma After 4 Years
Results Through June, All Students
By Need/Resource Capacity Category
Total Cohort Graduation RatePublic Schools
2001, 2002, 2003 Cohorts
![Page 17: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/17.jpg)
17
New York City Large City Urban-Suburban
Rural Average Low Total Public
2003 Cohort through June 2003 Cohort through August
Percentage of Students Graduating with Regents or Local Diploma After 4 Years
Results Through June 2007 and Through August 2007
By Need/Resource Capacity Category
2003 Total Cohort Graduation RatePublic Schools, All Students
![Page 18: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/18.jpg)
18
IEP Diploma, 1.6%
Still Enrolled, 17.0%
Transferred to GED Program, 1.2%
Dropped Out, 11.5%
Graduated, 68.6%
Students Who Started 9th Grade in 2003, Results After Four Years
69% of students statewide in the 2003 Total Cohort graduated by June 2007; 17% were still enrolled.
2003 Total Cohort Students = 220,332
All Students in Public Schools
![Page 19: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/19.jpg)
19
IEP Diploma1.6%
Still Enrolled14.6%
Transferred to GED Program
1.2%
Dropped Out11.5%
Graduated71.1%
Students Who Started 9th Grade in 2003, Results After Four Years
71% of students statewide in the 2003 Total Cohort graduated by August 2007; 15% were still enrolled.
2003 Total Cohort Students = 220,332All Students in
Public Schools
![Page 20: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/20.jpg)
20
Total Cohort Graduation RateAll Students
Four Years Five Years Six Years
2001 Cohort
2002 Cohort
2003 Cohort
Percentage of Students Graduating with Regents or Local Diploma After 4, 5 and 6 YearsResults Through June
Cohort Membership
2001 212,272
2002 214,729
2003 220,332
![Page 21: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/21.jpg)
21
2003 Total Cohort Graduation Rate
By Racial/Ethnic Group, By Gender
Percentage of Students Graduating with Regents or Local Diploma After 4 YearsThrough June 2007
Asian
American Indian / Alaska Native
HispanicBlack White
Solid colors: Females
Stripes: Males
![Page 22: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/22.jpg)
22
Total Cohort Graduation Rate for
By Racial/Ethnic Group2001, 2002, 2003 Total Cohorts
55.2
%
72.1
%
45.3
%
42.2
%
78.9
%
65.8
%
51.6
%
73.8
%
47.5
%
45.1
%
79.9
%
67.2
%
53.5
%
76.1
%
50.8
%
47.4
%
80.7
%
68.6
%
American Indian /Alaska native
Asian Black Hispanic White All Students
2001 Cohort 2002 Cohort 2003 Cohort
Percentage of Students Graduating with Regents or Local Diploma After 4 Years
Through June
![Page 23: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/23.jpg)
23
Total Cohort Graduation Rate for By Racial/Ethnic Group, By Gender
2001, 2002, 2003 Total Cohorts
64.0
%
77.4
%
53.3
%
48.8
%
83.2
%
71.2
%
56.8
%
79.3
%
54.6
%
51.2
%
83.8
%
72.0
%
61.2
%
81.9
%
57.4
%
53.9
%
84.6
%
73.4
%
46.9
%
67.2
%
37.5
%
36.0
%
74.8
%
60.6
%
46.6
%
68.7
%
40.4
%
39.1
%
76.2
%
62.6
%
46.2
%
70.7
%
44.0
%
41.0
%
77.0
%
63.9
%
American Indian /Alaska native
Asian Black Hispanic White All Students
Females 2001 Cohort Females 2002 Cohort Females 2003 Cohort Males 2001 Cohort Males 2002 Cohort Males 2003 Cohort
Percentage of Students Graduating with Regents or Local Diploma After 4 YearsThrough June
Solid colors: FemalesStripes: Males
![Page 24: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/24.jpg)
Total Public High School Graduates Each Year
YearNumber of Students Earning Regents or Local Diplomas
1995-96 136,754
1996-97 138,990
1997-98 139,531
1998-99 140,365
1999-00 141,510
2000-01 141,634
2001-02 143,070
2002-03 143,818
2003-04 153,202
2004-05 153,202
2005-06 161,732
2006-07 164,790
![Page 25: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/25.jpg)
Integrated Algebra
Pre-standard setting measurement review committee
Standard setting committee Post-standard setting measurement
review committee
![Page 26: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/26.jpg)
Standard Setting
Use operational sample: N=142,286 Calibration using the operational
sample Rasch and Partial Credit Model Produce ordered item book, RP 0.67:
39 operational items 22 anchor items
![Page 27: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/27.jpg)
Standard Setting
Two committees, 30 and 31 people, completely independent meetings
Item mapping methodology Process:
Achievement Level Descriptors Total of three rounds Use of p value and impact data
![Page 28: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/28.jpg)
Integrated Algebra: Standard-Setting
Relative Frequency Distributions by Grade Levels for Integrated Algebra, June 2008
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
4.0
0 10 20 30 40 50 60 70 80 90 100
Raw Score
Rel
ati
ve
Per
cen
t
GRADE 8
GRADE 9
GRADE 10
![Page 29: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/29.jpg)
Integrated Algebra Year II Identify representative sample
Use enrollment data Use Need/Resource Categories
Validity study: Reliability analysis Classification accuracy Operational and field test results contrast IRT model fit and dimensionality analysis Summary statistics by NCLB subgroups
![Page 30: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/30.jpg)
Public Forum on New York State’s NCLB Growth Model Proposal
David AbramsAssistant Commissioner for Standards, Assessment, and
Reporting
![Page 31: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/31.jpg)
“…to ensure that all children have a fair, equal, and significant opportunity to obtain
a high-quality education and reach, at a minimum, proficiency on challenging state academic achievement standards and state
academic assessments.”
Purpose of No Child Left Behind
![Page 32: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/32.jpg)
Chapter 57 of the Laws of 2007: Growth Model
“By the start of the 2008-2009 school year, the Regents shall establish, using existing state assessments, an interim, modified accountability system for schools and districts that is based on a growth model, subject to approval of the United States department of education where required under federal law.”
![Page 33: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/33.jpg)
Key Questions: How do we design accountability models that move
students from low performance to proficient as well as from proficient to distinction?
How can we ensure that improved scores represent improved learning?
How do we take data and turn it into actionable information that improves teaching and learning?
How do we move from beating the odds to changing the odds?
Next Generation Accountability System Design
![Page 34: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/34.jpg)
Accountability: Status vs. Growth
Status Models: take a snapshot of a subgroup’s or school’s level of student proficiency at one point in time and often compare that proficiency level with an established target.
Growth Models: measure progress by tracking the achievement scores of the same students from one year to the next to determine student progress.
![Page 35: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/35.jpg)
Why Growth?
Types of Performance
Improve-ment
Status
Low Growth High Growth
Low Status
High Status
Growth AccelerationEffective-ness
Achieve-ment
Status Change
Low/Low Low/High
High/Low High/High
Status/Growth Combinations
![Page 36: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/36.jpg)
Betebenner, Jan. 2008, for RI project
![Page 37: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/37.jpg)
Two Types of Growth Targets
Policy driven targets start with a policy goal (what should be) and then establish the targets for performance that are necessary to achieve this goal.
Data driven targets start with historical performance (what has been) and use that as a basis to project what should be expected of the units to be measured.
![Page 38: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/38.jpg)
New York State’s Proposal
Use growth in two different ways: To make more refined AYP
determinations (must be approved by USED)
To supplement AYP and make a more comprehensive system, attending in particular to growth of students who are proficient or higher (not necessary to be approved by USED)
![Page 39: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/39.jpg)
Constraints
Using growth in AYP is highly constrained: Focused on reaching proficiency and reducing the
achievement gap Specifics dictated by USED Proposal must be submitted by Oct. 15 to USED; if
approved, would apply to 2008-09 data Using growth outside of AYP is less constrained, but there
is less agreement on approaches: NYSED will work with partners to create this growth
proposal Regents would like proposal by end of this school year
![Page 40: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/40.jpg)
New York State: Local Initiatives
A number of NYS districts have developed local growth and value-added models. The two most prominent are: NYC’s Progress Report initiative. Capital Region BOCES initiative.
These initiatives are neither endorsed by SED nor require SED’s endorsement.
These initiatives are not constrained by USED’s growth model guidelines and were not designed for use in making AYP decisions.
These models can inform the development of State growth and/or value-added models for which USED’s permission is not required.
![Page 41: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/41.jpg)
SED’s Interim Growth Model Design Principles
Interim Growth Model shall be implemented in 08-09 school year (with Regents and USED approval).
Model shall meet core principles of Spellings 11/21/05 correspondence.
Model shall be based upon NY’s current State assessment program & shall not require the implementation of new assessments.
Model shall utilize such data as is currently collected through State data collection processes and shall not require the collection of new data elements.
Model’s purpose shall be to make more refined determinations of student progress, identify with greater precision high performing schools and districts, and support greater differentiation in support and services to schools and districts in need of improvement.
![Page 42: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/42.jpg)
SED’s Interim Growth Model Design Principles
The model shall be based upon measuring whether students are proficient or on track towards proficiency within a prescribed time period.
Model shall use an “open architecture.” All calculations should be transparent.
The interim growth model shall be a stage in a process leading, by 2010-11, to the development of an enhanced system that includes a value-added model.
The NCLB model should be combined with a State model that includes consequences and/or incentives for promoting growth for all students, while placing no school or district at risk of failing to make AYP, if it would make AYP under the current status model.
![Page 43: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/43.jpg)
USED Seven Core Principles
1. All students proficient or on track to proficiency by 2014; set annual goals to close subgroup gaps.
2. Expectations for annual achievement based on meeting grade level proficiency, not based on demographic characteristics or school characteristics.
3. Produce separate decisions for math and ELA/reading.
4. Include students, subgroups, schools, and districts in accountability.
![Page 44: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/44.jpg)
USED Core Principles – cont.
5. Assessment data: annual, 3-8 & high school, operational for more than one year (i.e., at least two years’ of data), produce comparable results grade-to-grade and year-to-year; approved in Peer Review.
6. Track student progress (longitudinal).7. Include participation rates and additional
academic indicator for accountability.
![Page 45: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/45.jpg)
USED Peer Review Additional Specifications
• Fully approved assessment system• Uniform minimum-n for all groups• No confidence intervals for growth• Very limited recalculation of student growth
target• Cannot use with multiple other non-Status
approaches, such as Safe Harbor and Index• Apply growth to all students for reporting
and accountability (preference)• Report student growth results (preference)• State has vertical scale (preference?)
![Page 46: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/46.jpg)
SED Draft Proposal
For grades 3-8, utilize a “proficiency plus” growth model for grades 3-8 similar to North Carolina’s approved model.
For high school, utilize a “value tables” model similar to Delaware’s approved grades 3-8 model.
Include an enhanced middle level and high school component in the proposal.
Build a “growth for all” State component that sets growth targets for all students, including those who are already proficient.
![Page 47: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/47.jpg)
NCLB Growth Model: General Approach
If a student scores proficient or above (Level 3/4) in the current year, include that student’s results in the Performance Index as is done under the present status model.
Use growth to check whether students who did not yet score proficient (Level 2) have grown enough that it is likely they will become proficient within a designated amount of time.
For purposes of calculating the Performance Index, give schools and districts “full credit” for any student who either scores proficient or above or who is deemed to be on track for proficiency.
![Page 48: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/48.jpg)
“On-Track” Growth to ProficiencyExample
3 4 5 6 7 8
Prof. 6
Prof. 5
Prof. 4
Prof. 3
Observed growth Gr. 3-4 projected to Gr. 8 Proficient
On track to be proficient
Proficient or
Prof. 8
Prof. 7
![Page 49: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/49.jpg)
3-8 Growth Model: Simplified Example
Level 3 Scale Score = 650. Billy scores a 614 in Grade 3 ELA. Billy is 36 points below proficiency (650- 614). Billy has four years to become proficient. Billy must close the gap by ¼ (9 points) in Grade 4. Billy’s proficiency target in Grade 4 is 623 (614 + 9). Billy scores 635 in Grade 4. Billy now has three years to become proficient. Billy must close the gap by 1/3 (5 points) in Grade 5. Billy’s proficiency target in Grade 5 is 640 (635 + 5).
![Page 50: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/50.jpg)
Growth Model: Middle School Extension
Students in middle school would be evaluated on whether they made sufficient growth to become proficient by the designated high school Regents examination.
The designated high school target is proficient on the Regents Examination in Integrated Algebra and proficient on the Regents Comprehensive Examination in English.
Students in middle school would have until the target assessment to be projected proficient; the number of years permitted would be based upon the grade the student entered middle school.
This middle school extension will only apply to schools (and their subgroups), not to district AYP decisions in instances where students transfer among schools within a district.
![Page 51: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/51.jpg)
Middle Level Extension: Simplified Example
Level 3 Math Regents Exam is equated to scale score of 663*.
Billy scores a 623 in Grade 5 Math. Billy enrolls in a new middle school in Grade 6. Billy is 40 points below proficiency (663-623). Billy has four years to become proficient. Billy must close the gap by 1/4 (10 points) in Grade 6. Billy’s proficiency target in Grade 6 is 633 (623 + 10). If Billy remained in his original school in Grade 6, then his
proficiency target would have been 637. (650**-623= 27/2 years = 13.5 + 623 = 637.)
*Actual equated score not yet determined**Represents Grade 7 Level 3 Scale Score
![Page 52: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/52.jpg)
3-8 Growth Model: Implications for Schools
Level 1 Level 2 Level 3 Level 4 On Track Towards Proficiency
Performance Index
(0 index points) (1 Index Point per
Student)
(2 Index Points per Student)
(2 Index Points per Student)
(2 Index Points per Student)
Number of Students 10 50 35 5 NA 130
Less Number of Students On Track Towards Proficiency
1 4 NA NA
Totals 9 46 35 5 5 136
![Page 53: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/53.jpg)
Growth Model: High School Extension
Students who enter high school having scored Level 1 or at low-Level 2 on the Grade 8 ELA or Mathematics tests are considered on track towards proficiency if they score between 55-64 on the designated Regents examinations prior to Grade 12.
Schools have five years for certain English Language Learners, certain students with disabilities, and students who enter high school far below standards to demonstrate proficiency in English language arts and mathematics.
Value table is interim model to be used for cohorts prior to the 2008-2009 school year cohort (i.e. next three years).
![Page 54: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/54.jpg)
Growth Model: High School Values Table
Score on High School Regents Exam
Initial Grade 8 level Less than 55 55-64 65 and higher
Level 1 0 200 200
2-minus 0 200 200
2-plus 0 100 200
3 0 100 200
4 0 100 200
![Page 55: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/55.jpg)
Timeline for NCLB Growth Models
July/September 2008 : Proposed model submitted to
Board of Regents for Review.
September/October, 2008: Discussion with the Field of
the Model.
October, 2008: Submission to USED.
Fall 2008: Approval of model by USED.
September 2009: Use of model to make AYP decisions
based on 2008-09 school year data, subject to availability
of resources.
![Page 56: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/56.jpg)
Building a “Growth for All” Model
Regents have directed SED to provide recommendations
for how to hold schools and districts accountable for
growth of students beyond proficiency as part of the
process of moving towards creation of a value-added
accountability model.
This “growth for all” model can be separate from NY’s
NCLB accountability system and need not be constrained
by NCLB growth model rules.
The Regents will need to decide what rewards and/or
consequences should be based upon a “growth for all”
model.
![Page 57: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/57.jpg)
Building a “Growth for All” Model
One possibility would be to modify the current
process for designation of High Performing and
Rapidly Improving schools to include a “growth for
all” component: other possibilities include rewards,
regulatory relief, and differentiated consequences.
![Page 58: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/58.jpg)
Approaches to “Growth for All” Models
- Student growth in terms of what other reference
schools or reference groups have achieved (e.g.
“peer schools,” “low-income Hispanic students”).
-Growth of students compared to other students who
started with similar growth histories.
-Student growth in relation to statistical expectations
for what the student would have learned with a
typical teacher/school.
![Page 59: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/59.jpg)
Define “Value-added”
Increase over previous score or performance
Increase over what was expected Attribution of performance changes
(increases/decreases) to agents/conditions
![Page 60: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/60.jpg)
What do we value?
“A Year’s Worth of Growth” More than “A Year’s Worth of Growth” Not Going Backwards Relative Growth Absolute Growth Growth to Proficiency Growth to a Point Beyond Proficiency
![Page 61: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/61.jpg)
What happens next?
- Questions and Answers
- Small group discussions and
completion of surveys
- Summary and next steps
![Page 62: Datag Annual Testing Accountabilty Discussion 10 081](https://reader035.vdocuments.net/reader035/viewer/2022062513/555f1eecd8b42abe748b4757/html5/thumbnails/62.jpg)
More Information
To submit questions or requests for more information, please e-mail: