2011 Top to Bottom and Persistently Lowest Achieving Schools List

Overview Briefing: MDEAugust 23, 2011

Top to Bottom (TTB) versus Persistently Lowest Achieving (PLA) •Top to Bottom Ranking: Ranking of all

schools in the state, based on proficiency, improvement and achievement gap in all five tested subjects.

•PLA List: List of the schools identified as Persistently Low Achieving Schools (PLA schools) following a set of federal guidelines

Why are the lists different?Top to Bottom PLA

Subjects included MathReadingWritingScienceSocial Studies

MathReading

Graduation rate? Yes No

Components Achievement (1/2)Improvement (1/4)Achievement gap (1/4)

Proficiency (2/3)Improvement (1/3)

Proficiency? Uses standardized measure of student performance (z-score)

Uses proficiency levels

High achieving schools?

Calculation adjustments to avoid “ceiling effects”

No adjustment

Tiers? No tiers; all schools included

Tiers; Title I, AYP and school level considered

Why are we publishing both?•TTB Ranking

▫Includes ALL schools▫Represents MDE’s preferred methodology,

developed in collaboration with stakeholders Three rounds of public hearings Three State Board of Education presentations Referent group feedback Multiple presentations to stakeholder groups

(30+ presentations)▫Provides increased light of day for a larger

number of low-performing schools ▫Changes made to reflect feedback and

recommendations from stakeholders

Why are we publishing both?

•PLA list▫State statute ties our identification of PLA

schools to the approved business rules for the School Improvement Grant program.

▫Attempted to obtain waiver from USED; waiver denied.

▫Would not approve new rules because of USED timelines

▫Must use currently approved federal rules for identification of PLA schools

Information to be published on 8.26.11

Top to Bottom Ranking

PLA List

List Yes Yes

Ranking (number) Yes No

Full data file Yes Upon request from PLA schools

“Brackets” display Yes Upon request from PLA schools

Data dictionary Yes Upon request

Business rules Yes Yes

FAQ Yes Yes

Explanatory Powerpoint

Yes Yes

Z-Score “Cheat Sheet”

•Z-scores are a standardized measure that helps you compare individual student (or school) data to state average data.

Student z-score = (Student Scale Score) – (Statewide average of scale scores)

Standard Deviation of Scale ScoreSchool z-score=(School Value) – (Statewide average of that value)

Standard deviation of that value

Z-Score “Cheat Sheet”•Z-scores are centered around zero•Positive numbers mean the student or

school is above the state average•Negative numbers mean the student or

school is below the state average

0 1 2 3-1

-2

-3

State Average Better than state

average….…Worse than state

average

Z-Score Examples•Your school has a z-score of 1.5. You are

better than the state average.

0 1 2 3-1

-2

-3

State Average Better than state

average….…Worse than state

average

Z-score of 1.5

Z-Score Examples•Your school has a z-score of .2. You are

better than the state average, but not by a lot.

0 1 2 3-1

-2

-3

State Average Better than state

average….…Worse than state

average

Z-score of 1.5

Z-score of 0.2

Z-Score Examples•Your school has a z-score of -2.0. You are

very far below state average.

0 1 2 3-1

-2

-3

State Average Better than state

average….…Worse than state

average

Z-score of 1.5

Z-score of 0.2

Z-score of -2.0

TTB Overview

•Prezi presentation•Will contain voiceover

•Interactive Overview of TTB Ranking

Using the “Brackets” Tool

•Created a tool to help schools walk through their own data.

•Will post this on the website.•Will send this out to all of you after this

presentation so you can try it out ahead of time.

•TTB Brackets Display

Comparing PLA and TTB

Number of Schools Identified

List Number of

Schools2011 PLA List 982010 PLA list 92Lowest 5% of Top to Bottom

151

Comparison of 2010 and 2011 PLA Lists

Comparison NumberOn both PLA 2010 and PLA 2011

58

New to 2011 PLA list 40 (4 of these not ranked in 2010)

On 2010 list; OFF 2011 list

34

2011 PLA List: DistrictsDistrict Information Number

Districts represented on 2011 PLA list

53

Districts represented on 2010 PLA list

48

Number of Detroit schools (2011)

38

Number of Detroit schools (2010)

40

Comparison: PLA and Lowest 5%

Comparison NumberPLA 2011 AND Lowest 5% TTB

48

Schools on 2011 PLA list who would NOT be identified in Lowest 5%

50

Schools in lowest 5% who are not in 2011 PLA list

103

On 2011 PLA; Not on Lowest 5%•Highest rank of these schools: 64th

percentile (University High)•Lowest rank: 5th percentile

Understanding University High2011 PLA Lowest 5%

Subjects included Reading and math only

Reading, math, science, social studies, writing

High performance subjects

Math MathSocial studiesGraduation rate

Lower performing subjects

Reading ScienceSocial StudiesWriting

Achievement gaps -- Small gaps; helps ranking

Tiers? Tier 2 (high schools only, Title I eligible, fail AYP)

No tiers

Calculations Percent proficient and improvement disadvantages UP

Z-scores help UP, as do achievement gaps

Common Concerns

Why are some schools on the PLA list but ranked higher than 5th percentile in TTB?

•Differences in ranking methodology; most significantly:▫Only math and reading in PLA rules; all five

subjects in TTB.▫Graduation rate in TTB ranking

•Tiers used in PLA▫Tier 1: Receiving Title I, failing AYP▫Tier 2: Eligible for Title I, secondary school

Are you comparing “apples to apples?”•Improvements in TTB to ensure we are

comparing students and schools more equally▫Translating student scale scores into z-

scores instead of into performance levels; compares students to like students.

▫Z-scores on school measures compare elementary/middle schools to other elementary/middle schools, and high schools to other high schools.

High-performing schools are disadvantaged by the ranking•Included “ceiling effects” provisions

▫Schools with over 90% of students proficient are ranked on achievement and gap only

▫Students who are previously proficient who maintain are counted as improving

▫Graduation rate over 90%; do not look at improvement, only rate.

The inclusion of achievement gap hurts high performing schools•High performing schools who do not

ensure all students are high-performing will have their ranking impacted

•Only ¼ of final ranking•Don’t want to decrease proficiency to

improve gap because achievement is ½ of ranking

•Need to get serious about making sure all students are learning

This system is too complex and hard to understand•Capturing school performance requires a

nuanced system.• Complexity does not decrease transparency

▫ Transparency does not equal simplicity▫ Transparency does equal the ability for external verification▫ Transparency includes providing details on the system’s

complexity

• Complexity of the model has been added at the request of the field and experts to more appropriately capture school performance

• MDE will support the transparency through professional learning, technical assistance, and open access to data

Questions? Concerns?What are we missing for the release plans? What additional information would you like to have available for internal usage?