september 11, 2009 everyman state: using delivery to achieve ambitious graduation goals american...

September 11, 2009

Everyman State: Using Delivery to Achieve Ambitious Graduation Goals

American Diploma Project NetworkAnnual Leadership Team Meeting

Plenary session

CONFIDENTIAL AND PROPRIETARYAny use of this material without specific permission of McKinsey & Company is strictly prohibited

2

Contents

▪ Introduction: Delivery, ADP and Everyman State

▪ Setting out: Establishing goals, trajectories and routines

▪ Falling behind: How do we get back on track?

▪ Success: Impact of increased CCR graduates

▪ Conclusion: What can EDI do for you?

3

What is “Deliverology?”

“ Deliverology” (n.) is a systematic process for driving progress and delivering results.

What are you trying to do?

How are you planning to do it?

At any given moment, how will you know whether you are on track?

If not, what are you going to do about it?

It will enable you to answer rigorously the following questions:

1

2

3

4

4

Like all of you, Everyman State has established ambitious college- and career-ready graduation requirements in line with the ADP mission

▪ In order to meet the demands of today’s workforce, all students must graduate with at least:

– 4 years of Math, including Algebra I, Geometry, and Algebra II

– 4 years of rigorous English– 3 years of Science– 3 years of Social Studies

▪ Graduating college- and career-ready will be a default curriculum

▪ In order to receive course credit, students must pass both the course and the end-of-course assessment

NEW college- and career-ready requirements

Previous requirements

All students must graduate with at least these courses:

▪ 2 years of Math, including Algebra I▪ 3 years of English▪ 2 years of Science

5

Everyman State has established ambitious goals for implementation

6

Like many of you, Everyman State faces demographic and performance challenges in meeting graduation goals

Graduation rates, Class of 2008Percent of each group by graduation type

9th grade entry cohort demographics, Class of 2008Percent of all students

3

3333

100

LEPSPED

11

Low-income

URMTotal

Number of

students

99,200 2,60010,80032,60032,900

LEP

57

25

SPED

58

23

Low-income

57

25

URM

55

24

All students

70

49

Cohort graduates

CCR graduates

69,800

51,600

1,500

800

6,300

2,500

18,700

10,400

18,200

9,700

College- and career-ready graduation rate

Cohort graduation rate

7

In order to reach its goals, Everyman State will need 91,900 on-time graduates in 2017, and 81,700 college- and career-ready grads

SOURCE: Team analysis

Required graduates

81,700

91,900

Class of 2017 goal

80% CCR grads

90% cohort grads

Projected 9th grade cohort

102,100

102,100

8

21,000

7,000

3,200

10,400

2,500

800

Low-income

Special Education

Limited English Proficiency

0.75

0.74

0.710.42

0.49

0.51

0.700.40

In order to halve the achievement gap in CCR graduation rates by 2017, Everyman State must focus on struggling populations


23,1009,700Under-represented minorities

Student subgroup

Ratio to non-disadvantaged student rate

2017 goal ratio (50% less gap)

Class of 2017 grads required

Class of 2008 CCR grads

College- and career-ready graduation rates

9

First new cohort

To stay on track to its 2017 goal, Everyman State plots the graduation targets it must reach each year on its trajectory


40,000

45,000

50,000

55,000

60,000

65,000

70,000

75,000

80,000

85,000

90,000

95,000

100,000

Number of graduates

Graduating class

Cohort goal = 91,900 grads

201720162015201420132008

49% graduate CCR today

70% graduate on time today

CCR goal = 81,700 grads

10

At its current performance levels, Everyman State will fall 32,200 graduates short of its CCR goal


Gap to goal = 32,200

First new cohort

Number of graduates

Graduating class

201720162015201420132008

Baseline = 49,500

CCR goal = 81,700

40,000

45,000

50,000

55,000

60,000

65,000

70,000

75,000

80,000

85,000

11

Currently planned interventions only yield 8,700 additional CCR graduates


Impact on leading indicatorIntervention

# of additional CCR grads

Early warning indicators (reduced cumulative dropouts by focusing on 9th and 10th grades)

▪ 45% decrease in cumulative dropout rate

3,900

Saturday Academy (Saturday school for students at risk of passing course but failing EOC test)

▪ 10% increase in pass rates for courses taken during academic year

4,800

All existing interventions 8,700

12

First new cohort

With these planned initiatives, Everyman State will fall 23,500 graduates short of the necessary trajectory to reach the CCR goal


Number of graduates

Graduating class

201720162015201420132008

Gap to goal = 23,500

Projected CCR withplanned interventions= 58,200

Baseline = 49,500

CCR goal = 81,700

40,000

45,000

50,000

55,000

60,000

65,000

70,000

75,000

80,000

85,000

13

Everyman State uses five types of benchmarking to learn about other ways to build a trajectory that will reach its goal

Againsthistory

Within your systems

Againstother similar

systems

Againstorganizations with similar processes

Againstthe world

14

To close the gap, Everyman State looked at its system and realized that several demographic groups were struggling to pass core CCR courses


25-30▪ URM, LI, LEP, SPED

Math EOC tests

▪ Not part of a subgroup

75-80

Algebra I Geometry Algebra IIFourth-Year Math

CCR graduates

Course passage ratesPercent

15

Everyman State looked at other states and learned that using ELT to target support at these students could add 3,600 CCR graduates in 2017

1 Overall impact on system; rates based on 50% improvement in select Math, 15% in English, 33% of failing students; affects year passage rates only.SOURCE: Team analysis

30-35▪ URM, LI, LEP, SPED

With extended learning time, Everyman

State gains 3,600 additional CCR graduates by 20171

Math EOC tests

Algebra I Geometry Algebra IIFourth-Year Math

CCR graduates

Course passage ratesPercent

▪ Not part of a subgroup

75-80

16

Through similar analyses, Everyman State identified severalother interventions to close the gap to its goalIntervention

Number ofadditionalCCR gradsImpact on leading indicator

Extended time for bottom 33% of schools (ELT)

▪ 50% increase in math year course rates▪ 15% increase in English year course rates

3,600

Literacy remediation ▪ 75% decrease in failures of English year courses 5,100

90% summer school enrollment

▪ 90% enrollment in summer CCR courses when year course is failed

5,100

9th grade readiness (Alg 1 and writing)

▪ Increase in Alg 1 and Eng 1 pass rates by 9th grade class of 2017– 40% year course, 25% summer (SPED)– 30% year course, 15% summer (URM, LI, LEP)– 10% year course, 5% summer (All others)

700

Early college high school ▪ 20% decrease in dropout rate▪ 30% decrease in off-track/continuing rate

6,000

Mid-career math recruitment (50% lowest-performing schools)

▪ 15% improvement in math year course pass rates 100

Total 24,500

8th grade planning (guidance counselor initiative)

▪ 100% on-track 9th grade enrollment by 2013▪ 20% decrease in off-track/continuing rate

3,900


17

First new cohort

With these new interventions, Everyman State has a series of plans that it believes will get carry it to its college- and career-ready goal for 2017

1 Impact masked by students’ failing English sequence2 Includes impact of other initiatives

Number of graduates

Graduating class

201720162015201420132008


40,000

45,000

50,000

55,000

60,000

65,000

70,000

75,000

80,000

85,000

Extended time = 3,6002

Mid-career math recruitment = 1001

Early college high school = 6,000

9th-grade readiness = 700

90% summer school = 5,100

8th grade planning = 3,900

82,700Literacy remediation = 5,1002

67,400

72,600

62,000

77,700

Planned interventions = 8,700

Baseline = 49,500

CCR goal = 81,700

19

Trajectories are not necessarily straight lines, because the timing and size of your interventions may vary

Low trajectory (policy has a lagged impact)

High trajectory (policy has an immediate impact)Sample policy trajectories

Long-term strategic goal

Historical performance

Starting point

Performance

Time

20

Everyman State also develops a schedule of regular routines to monitor progress

Definition

▪ Monthly note to the CSSO with a briefing on delivery progressMonthly notes

SEA stocktakes

▪ Quarterly meeting between CSSO and relevant SEA leaders to regularly assess each planned intervention

District stocktakes

▪ Quarterly meeting between CSSO and leaders of major districts to regularly assess progress

Delivery reports

▪ Semi-annual assessment of the status of all of the system’s targets and underlying interventions

21

Rationale summary

Quality of planning, implementation and performance management

Recent performance

against trajectory and

milestones

Highly problematic – requires urgent and decisive actionRed

Mixed – aspect(s) require substantial attention, some goodYellow/GreenGood – requires refinement and systematic implementationGreen

Problematic – requires substantial attention, some aspects need urgent attentionYellow/Red

Everyman State uses this assessment framework to routinely assess progress on each intervention

Likelihood of delivery

Capacity to drive progress

Judgement

Degree of challenge

Stage of delivery

Rating

L / M / H / VH

1 / 2 /3 / 4

22

Everyman State uses a summary table of assessments to track progress on all of its interventions

Program Rank Likelihoodof delivery

Stage ofdelivery

Capacity todrive progress

Quality of planning,implementation andperformance mgmt.

Degree of challenge

Assessment criteria Overall judgment

Early Warning Indicators

Saturday Academy

8th-grade planning

90% Summer school enrollment

9th-grade readiness

Early college high school

Mid-career math recruiting

Extended time

Literacy remediation

23

Discussion 1: Goals, trajectories and routines

Q1. How useful would this process of trajectory setting and creating routines be in reaching your college- and career-readiness goals?

Have you conducted a similar process of goal-setting and trajectory-setting? Why or why not?

Do you have access to the necessary information to draw useful conclusions from this type of exercise?

Q2. How challenging would trajectory building and establishing similar routines be in your state?

▪ What would be the biggest barriers to building a trajectory and establishing routines in your system?

▪ Do you have the capacity or capability in your system to perform the required data analysis? To oversee progress?

▪ Do system leaders have the time and ability to prioritize these activities given competing demands?

24

Goals, trajectories and routines: Time to vote!

How useful would this process of trajectory setting and creating routines be in reaching your college- and career-readiness goals?

A. “Not useful at all"

B. “Possibly useful – need to know more”

C. “Probably useful – want to know more”

D. “Definitely useful”

25

Goals, trajectories and routines: Time to vote!

How challenging would trajectory building and establishing similar routines be in your state?

A. “Impossible”

B. “Very challenging”

C. “Somewhat challenging”

D. “Not challenging at all”

26

Contents






27

First new cohort

When the Class of 2013 graduates, Everyman State realizes it is already off-track to its CCR goal for 2017

Number of graduates

Graduating class

201720162015201420132008

Current trajectory

Required trajectory


Gap to goal = 6,100

Projected off-track = 75,600

40,000

45,000

50,000

55,000

60,000

65,000

70,000

75,000

80,000

85,000CCR goal = 81,700

Actual 2013 graduates

28

Mid-career math recruitment

Medium 1Successful as expected in early stages of recruitment

Early college high school Medium 1Well-received and some decrease in dropouts; need to benchmark rigor of programs

90% summer school enrollment

Low 2Results as expected; some improvements needed in enforcement/performance mgmt.

8th-grade planning High 2Extremely effective where implemented; some need for additional counselor capacity

9th-grade readiness Med-Low 2Significant gains in student achievement in all classes by 9th grade measures

Saturday Academy Med-Low 3Enrollment used as enrichment; need to communicated intended beneficiaries

Extended time Med-High 1Insufficient teacher capacity; impact is only 25% increase in math scores

Early Warning Indicators High 3Dropout rate has not changed at all; needs further attention

Literacy remediation High 1Results as expected in early stages of implementation, especially in English II scores

A review of progress shows mixed results amongst the interventions

Program

Planning, perform. mgmt.

Degree of challenge Rank

Stage ofdelivery

Capacityto driveprogress Justification

Likelihoodof delivery

Assessment criteria Overall judgment

29

The reduced impact of Saturday Academies is due to miscommunication of the program as an enrichment service


Recent performance


milestones





Judgement

Degree of challenge

Rating


▪ Program effective; only requires accurate targeting of students

▪ At all levels, ineffective communication has led to targeting wrong students for Saturday program. Program used as enrichment, not intervention

▪ Educator and administrative capacity is sufficient

▪ Need to develop and emphasize process for identifying at-risk students who should be placed in this program

Med-Low

Rationale summary

Stage of delivery ▪ Advanced; in place for over 3 years3

Solution:

Communication plan and diagnostic testing

30

The reduced impact of extended time is due to absence of excellent educator capacity to teach in ELT schools


Recent performance


milestones





Judgement

Degree of challenge

Rating


▪ Extended time costly and success depends on quality of teachers

▪ Structure and funding exist▪ Need for improved performance

management to identify lower-than-projected gains earlier in cohort progression through CCR sequences

▪ Quality of teaching, certification and tenure of teachers at ELT schools are below peer schools, leading to reduced student gains

Med-High

Rationale summary

Stage of delivery▪ Beginning; implemented beginning in

20111

Solution:

Recruit better teachers and incentivize to teach

at ELT schools

31

The reason for the consistently high dropout rate is unclear and merits further investigation


Recent performance


milestones





Judgement

Degree of challenge

Rating


▪ Cause of dropout rate is unknown

▪ Dropout rate of traditionally at-risk students has actually declined but overall dropout rate has not

▪ Performance management system offers no explanation

▪ Educator capacity to identify Early Warning indicators and support at-risk students is above levels of peer systems

High

Rationale summary

Stage of delivery ▪ Advanced; in place for over 3 years3

Solution:

?

32

Q3. How well-equipped is your state to resolve significant problems where you cannot identify the root of the issue?

Do you have processes or systems in place for problem-solving?

What information do you need? Which people are critical to resolving the problem?

Do you face challenges in gathering and analyzing information?

Do you face challenges in prioritizing difficult issues over competing demands?

Q4. What is the biggest barrier to problem-solving?

Discussion 2: Problem-solving and resolution

33

Problem-solving and resolution: Time to vote!

How well-equipped is your state to resolve such problems?

A. “Not at all – it would be impossible”

B. “Poorly equipped”

C. “Somewhat equipped”

D. “Well-equipped”

34

Problem-solving and resolution: Time to vote!

What is the biggest barrier to problem-solving?

A. “Lack of capacity”

B. “Limited access to and/or poor quality of data”

C. “Competing priorities”

D. “Inadequate systems and processes”

36

For maximum impact, a problem-solving process should include these key steps:

Conducting a priority review

Identify problem

Initial analysis of dataand evidence

Identify delivery chain and structure issues

Generate initial hypotheses

Generate interview questions

Field work

Analyze evidence and structure solutions

Agree on key actions

37

After conducting a priority review, Everyman State finds existing programs do not address “new dropouts,” who need support for CCR classes


Interviews and front-line visits support data that “new dropouts” need support for CCR courses

Surveys identify that primary reason for dropping out is lack of academic support

▪ Students with traditional drop-out profiles (disinterested, unprepared) have decreased 25% since the Early Warning indicator intervention

▪ Students dropping out today do not match profiles of traditional dropouts (better prepared for high school, find classes interesting)

▪ “New” dropouts report feeling sense of failure and desire for increased support

▪ Most “new” dropouts tend to leave after passing 1-2 years of CCR curriculum and struggling with later courses

44

2630

Primary reason for dropoutPercent of dropouts

Can’t get academic support

Classes aren’t interesting

Family/personal reasons

38

Everyman State decides to pilot three academic support initiatives it believes will prevent “new dropouts” in large districts 2013-2014


Class of 2013

Pilot program

Leading indicators impacted

Expected decrease in cumulative dropoutsPercent

Graduation coaches

▪ 11th-12th grade dropouts, URM

20

Skills preparatory camp

▪ 9-10th grade dropouts, URM

15

AVID ▪ All dropouts, URM 25AVIDLane District

Skills preparatory campAllegheny District

Graduation coachesMontgomery District

Dropout ratePercent of entry cohort

2122

20

Class of 2014

-10%-21%

17

2018

-11%

39

The following year (2014-2015), Everyman State rolls out the highly successful AVID program system-wide

0

5

10

15

20

25

30

35

0 10 20 30 40

Dropout rate, Class of 2013Percent of cohort

Decrease in dropout rate, 2013-2015Percent

In the subsequent school year (2015-2016) Everyman State uses, “leader districts” to improve “laggard districts,” further increasing the impact of the new dropout intervention:

▪ Teacher, principal and superintendent exchanges

▪ Best practice workshops

▪ Identification of key implementation barriers

▪ Peer sharing

“Leader districts”

“Laggard districts”

Individual district performance


40

With its new dropout intervention, Everyman State truly believes it can get back on track to its 2017 goal

Graduating class

Number of graduates

Projected off-track =79,500

201720162015201420132008

CCR goal = 81,700

Projected with interventions = 82,600

First new cohort


40,000

45,000

50,000

55,000

60,000

65,000

70,000

75,000

80,000

85,000

41

Contents






42

First new cohort

By 2017, with continued monitoring and problem solving, Everyman State reaches its goal, and manages to halve the achievement gap


40,000

45,000

50,000

55,000

60,000

65,000

70,000

75,000

80,000

85,000

90,000

95,000

100,000

Graduating class

Cohort grads = 92,000 grads

20172016201520142013

Number of graduates

2008

CCR grads = 82,600

43

The increase in Cohort and CCR graduation rates in Everyman State increases college enrollment and completion

SOURCE: Strong American Schools, Diploma to Nowhere, 2008; BLS 2008; Census 2007; Chicago Public Schools 2008; Indiana Core 40; Role of Community Colleges in Florida’s Economy, 2004 (see Appendix for further details)

Total

73,800

12,300

Increased CCR graduates (enrollment of increased CCR grads)

Increased graduates (enrollment of additional cohort grads)

9,500

Baseline (enrollment from baseline grads)

52,000

College enrollment by Class of 2017Number of students

Total

50,400

Increased CCR graduates(completion by increased CCR grads)

3,800

Increased graduates (completion by additional cohort grads)

33,300

Baseline (completion by baseline grads)

13,300

6-year degree attainment by Class of 2017Number of graduates

This increase in college graduates

in the workforce increases GDP by $7 bn

over the lifetime of the Class of 2017high school graduates

44

After 5 years of relentlessly striving and problem-solving toward its high-school

graduation goals, Everyman State added 43,000 high school graduates,

17,000 college graduates and significantly boosted its economy.

Imagine

in your state.what this would look like

45

Contents






46

The U.S. Education Delivery Institute (USEDI) will build the “capacity to deliver” of state-level K-12 and higher education systems

▪ The mission of the USEDI is to develop the capacity of K-12 and higher education leaders to define and deliver on their key education priorities, so that intent at the system level translates to impact at the student level

Mission

Major functions

▪ Provide direct support to participating systems to build their capacity to deliver

▪ Connect participating systems in a collaborative learning network

▪ Build a knowledge base of best practices and “case stories” of delivery

Partners ▪ The Education Trust

▪ Achieve

Support ▪ Michael Barber (leadership)

▪ McKinsey & Company

september 11, 2009 everyman state: using delivery to achieve ambitious graduation goals american...

Documents

everyman state

careerready graduation

ccr graduation rates

years of science slide

graduation type

cohort grads

ccr grads college

cohort graduates ccr