understandin g the api & the ayp aplus+ annual conference october 2010 del mar, california diane...

Understanding the API & the AYP

APLUS+ Annual ConferenceOctober 2010

Del Mar, CaliforniaDiane Grotjohn

[email protected]

California And the Feds

California Academic Performance Index

Established by PSAA in 1999Measures growth

Moving Target

FederalAnnual Yearly Progress

Established by NCLB Act in 2001Measures progress towards a set goalAll schools have same goal

APLUS+ Conference October 2010 2

Uses of the API

• Ranks schools• Measures progress of state

intervention programs• Is part of Adequate Yearly Progress• Provides public information on

school improvement• Determines eligibility for awards• Is only valid as growth measure

from one year to the next (can not be compared long-range)

APLUS+ Conference October 2010

3

Growth API

• For school and district: 5% of difference between API and ‘interim’ growth target of 800. – Minimum growth target for any school is 5 until

school has 796 API– PSAA law says that those who do not make

growth are underperforming

• For subgroups: – Until 2007: 80% of school or district’s growth– Now: 5% of subgroup’s difference between API

and 800– PSAA law says that lack of sufficient growth in

ANY subgroup means the whole school is underperforming.


4

Growth Targets for API

Growth Target:

Schoolwide or Subgroup Base API:

200 – 690 691-795 796-799 800-1000

Schoolwide orSubgroup

5% difference between base API and 800

5-point gain

796 +4-pts 797 +3-pts798 +2-pts799 + 1-pt

Maintain 800 or more


5

API Target Calculation


6

Maximum 1000

Target 800

Example School 535

Minimum 200

0

5% x (800-535) = 13

Growth Target

5% of the Distance to the Statewide Performance Target

Required Components of API*

• CAPA – California Alternate Performance Assessment

• CMA – California Modified Assessment

• CST – California Standards Test

• CAHSEE – California High School Exit Exam


7

*Include when valid and reliable

API for Schools, Districts, and Subgroups

• Two APIs each year: Base and Growth• Base API is used for ranking deciles.(Feb-

Mar)• Growth API shows changes in most

recent testing (Fall)• Since 2001, a new API calculation

formula each year. Year to year comparisons are not valid. Can only compare growth rates, not API score

• District API (base and growth) since 2003APLUS+ Conference October 2010

8

Example: Assessment Changes


9

API Performance LevelWeighting Factors


10

CST Performance

Levels

NRT Performance

BandsCAHSEE

Score

API Performance Level Weighting

Factors

Advanced 80-99th NPR Pass 1000

Proficient 60-79th NPR N/A 875

Basic 40-59th NPR N/A 700

Below Basic 20-39th NPR N/A 500

Far Below Basic 1-19th NPR No Pass 200

Note: A Weighting Level of 200 is “assigned” to “untested” students in 8 -11 grade Math and 9 – 11 grade Science

2007/2008 and 2008/2009API Test Weights

Grades 2-8


11

Note: Test weights are not shown as percentages and do not total 1.00.

Content Area 2009–10 API Test Weights

CST/CMA/CAPA in ELA, Grades 2–8 0.48

CST/CMA/CAPA in Mathematics, Grades 2–8 0.32

CST/CMA/CAPA in Science, Grades 5 and 8 0.20

CST in History–Social Science, Grade 8 0.20

Assignment of 200, CST in Mathematics, Grade 8

0.10

2008/2009 API Test WeightsGrades 9-12


12

Content Area 2009-10 API Test Weights

CST/CAPA in ELA, Grades 9-11 0.30

CST/CAPA in Mathematics, Grades 9-11 0.20

CST in Science, Grades 9-11 0.22

CST/CAPA in Life Science, Grade 10 0.10

CST in History-Social Science, Grades 9-11 0.23

CAHSEE ELA, Grades 10-12 0.30

CAHSEE Mathematics, Grades 10-12 0.30

Assignment of 200, CST in Mathematics, Grades 9-11 0.10

Assignment of 200, CST in Science, Grades 9-11 0.05

Note: For CAHSEE, grades eleven and twelve are counted only if the student passed. Test weights are not shown as percentages and do not total 1.00.

Note: For CAHSEE, grades eleven and twelve are counted only if the student passed. Test weights are not shown as percentages and do not total 1.00.


13


14


15


16

2009/2010 API Overview, 9 – 12 Schools

“Assignment of 200”What is it?

• A score of 200 is assigned to every student who does not have a valid score in– CST-Math (grades 8-11)

• A test weight of 0.10 is used in the API calculation instead of 0.32 (grade 8) or 0.20 (grades 9-11)

– CST-Science (grades 9-11)• a test weight of 0.05 is used in the API calculation

instead of 0.22


17

Math Rules

Rules for Grades 8-9 CST in General Math:


18

For grade 8, the performance level of the student record is lowered by one performance level

For grade 9, the performance level of the student record is lowered by two performance levels

Scale Calibration Factor

• Designed to “maintain consistency” in statewide average API from one reporting cycle to the next

• The “SCF” is calculated for each grade span• Difference between statewide average Growth and

Base API’s for the same year’s test data• Added to school, LEA, subgroup API• Does NOT preserve comparability across reporting

cycles• Can not accurately compare year-to-year scores

04/21/23 19

Subgroups

• Numerically significant subgroups ALSO have Base and Growth APIs.– “Numerically significant” if either:

• 100 students in group• 50 students in group, which is at least 15% of

total student population

• Numerically significant subgroups all must grow 5% of the difference of the subgroups previous year’s API and 800.

• “Comparable Improvement” or “CI” is now included in API reports.

APLUS+ Conference October 2010 20


21

Subgroup Determination

English Learners

• Who counts, and when?– If in US less than one year, doesn’t count

for proficiency on AYP•March 15 is cut off date

– Does count in API and in participation rates for AYP

– Is included in API calculations, but NOT in EL subgroup API


22

Reclassified Fluent-English-Proficient (RFEP)

• Included in EL subgroup UNLESS– Scored Proficient or above on ELA CST for 3 years– If not marked, ETS assumes “yes”

• Not counted to determine if subgroup is numerically significant

• NOTE: Verify the accuracy of demographics with ETS – if not identified, they will assume not in subgroup


23

So, Exactly, WHO counts?

• All students enrolled on first day of testing– With some exceptions . . .


24

How is “Enrollment First Day of Testing” Determined?

•Is there a test document for the student?

(See flow chart page 31)


25

How is “Number Tested” Determined?

• Was the student “enrolled on the first day of testing”?

• Did the student take any portion of any test?

(Flow chart p. 32)


26

How is “Valid Scores” Determined?

• Is the student included in “number students tested”?

• Was the student continuously enrolled for the academic year?


27

Including the CAHSEE

• All 10th graders enrolled at “census” count

• All 11th and 12th grade passers count

(Flow Chart p. 35)


28

Inclusion/Exclusion Rules

Mobility

Completely Blank Test

Irregularity

Unmatched Score

Accommodations/Modifications

Special Testing Conditions


29


Mobility If a student has been continuously

enrolled in a school from the October CBEDS date to the testing date, the student is counted in the school API.

English learners enrolled in a U.S. school for less than a year are not counted in the school or LEA API (matches AYP calculation rules).


30


Completely Blank Test If a student record shows no scores or items

attempted on any part of the CST, and CAPA used in API, the entire STAR Program student record is not included in API.

Exceptions:

If a student record shows “Tested but Marked No Answers” (Code Z) for any content area, the content area is included and assigned a weight of 200


31


Completely Blank Test

Exceptions (continued):

CAHSEE grade 10 student census (February or March) record showing “Blank/ Not Attempted” for one or both content areas is included and assigned a weight of 200 for content area(s). Blank records for grades 11 and 12 are excluded.


32


Accommodations

CST or CAHSEE:

Accommodations do not effect score


33

Note: This rule is the same for AYP calculations.


Modifications for APICST or CAHSEE:

The score is included for the content area and assigned a weight of 200.

The “Matrix of Test Variations, Accommodations, and Modifications for Administration of California Statewide Assessments” can be found at http://www.cde.ca.gov/ta/tg/sr.


34


Modifications for AYP

CST or CAHSEE:

Counted in AYP participation rate as “not tested

Not counted in AYP percent proficient


35

Statewide Ranking

• Schools ranked 1 to 10; 10 is best.

• Half of all schools will always be underperforming. Always.

• Greater API growth is needed to move out of decile1 or 2 than deciles 5, 6, 7.

• If you stand still, others will pass you by.


36

API Ranks

• Ranks are established by deciles.• Base API is used to establish ranks• A school’s rank may, or may not,

improve if API improves depending on what other schools did.

• LEAs and ASAM schools receive API scores, but do not receive ranks

• Small schools (<99 valid scores) do not receive similar school ranks


37


38

“School Type” for API

• Ranks are calculated separately by school type

• Schools are compared by “type”• 3 “types”:

– Elementary, Middle, and High– Based on lowest and highest grade in

which a student was reported at CBEDs


39

My school doesn’t “fit” a “type”

• School type is based on enrollment in “core grade spans”– Elementary: K-5– Middle: 7 – 8– High: 9 - 12

• Some (very few) schools are assigned “type” based on name or characteristics

• If Base API school type is different from Growth API school type, districts are given opportunity to change


40

Similar School Rank Variables


41

Similar School Ranks• Determine school type: Elementary, Middle,

High• Calculate School Characteristics Index (SCI)• Sort schools by SCI. Find 50 schools above

and 50 below target school.• Using identified similar schools, sort schools

by API. Identify deciles.


42

AYP


43

And for AYP….

•Four requirements:–Participation Rates–Percent Proficient–Graduation Rate–API as “additional indicator”


44

Schools are “underperforming” if

• School or any subgroup misses any target

• If miss same target for 2 years, go into Program Improvement (Unless not a Title 1 school)

• Get out of PI if meet all targets for 2 years

• Number of schools in PI in California is growing each year


45

AYP Targets for 2010

• Percent Proficient for K-8 – ELA: 56.8– Math: 58.0

• Percent Proficient for High Schools– ELA: 55.6– Math: 54.8

• Minimum API of 680 – Or growth of at least 1 point

• Graduation Rate– 90% or– Fixed growth rate or– Variable growth rate


46

AYP Proficiency Targets Over Time


47

Four-Year Graduation Rate Formula for ESEAGraduation Rate for 2010

Number of Graduates (2008–09)

divided by

Number of Graduates (2008–09)+ Grade 9 Dropouts (2005–06) + Grade 10 Dropouts (2006–07)

+ Grade 11 Dropouts (2007–08) + Grade 12 Dropouts (2008–09)


48


49

Assessments Used in API/AYP

Thank you!


50

understandin g the api & the ayp aplus+ annual conference october 2010 del mar, california diane...

Documents

api scoredistrict api

growthbase api

margrowth api

api test weightsgrades

api test weightscstcmacapa

subgroup base api

growth apifor school

apiaplus conference