An Evaluation of the Early Progress of The Pittsburgh Promise®

and New Haven Promise

Gabriella C. Gonzalez and Robert Bozick

2

In 2010, The Promise Asked RAND to Evaluate Early Progress Toward Its First Two Goals

1. Mitigate and reverse the population declines in the city of Pittsburgh and the enrollment declines in PPS

2. Grow the high school completion rates, college readiness, and post-high school success of all PPS students

3. Deploy a well-prepared and energized workforce and an eager core of community volunteers

3

RAND Examined Baseline Trends and Patterns in Four Areas

Enrollment in

PPS

Influence on parents’

decision to enroll child in

PPS

Influence on students’

attitudes and behaviors

College enrollment

and persistence

rates

4

PPS Enrollment Has Begun to Stabilize Since the Program’s Inception

2000 2001 2002 2003 2004 2005 2006 2007 2008 20090

5,000

10,000

15,000

20,000

25,000

30,000

35,000

40,000

45,000

2007-08: In-ception of The Promise

Numberof K-12

students

5

Percent of Students Enrolling and Continuing in PPS Has Remained Steady

Percentage

Continuation

Enrollment

Pre-Promise School Years (2005-06, 2006-07) Early Promise School Years (2007-08, 2008-09, 2009-10)

6

There Is No Clear Pattern In Enrollment of New Transfers into PPS Schools

6th 7th 8th 9th 10th 11th 12th0

50

100

150

200

250

300

350

400

450

Pre-Promise Years (2005-06, 2006-07)Promise Years (2007-08, 2008-09, 2009-10)

Number

7

The Promise Was an Important Factor in Parents’ Enrollment Decisions

3.9 “We wanted to take advantage of the Pittsburgh Promise scholarship program”

5

4

3

2

1

Not at all important

Somewhat important

Very important

8

5

4

3

2

1

Not at all important

Somewhat important

Very important

Non-Whites and Parents with Lower Education and Economic Status Assigned The Promise

Higher Importance than Other Parents Parents with a high school degree or lower (4.4)

Other (3.5)Non-white (4.2)

Parents with lower income (4.3)

9

Students Reported Being Motivated by the Program

• The opportunity for college funds motivated them to strive to meet eligibility requirements– Obtain at least a 2.5 GPA– Maintain 90 percent attendance

• The Promise factored into their decisions to attend college

• However, many students did not understand key program elements

10

Students reported being

motivated by the program

Summary of Key Findings

Enrollment in

PPS

Influence on parents’

decision to enroll child in

PPS

Influence on students’

attitudes and behaviors

College enrollment

and persistence

rates

PPS enrollments have stabilized,

rather than continuing to

decline; numbers of students new to the district are

inconsistent across the years

Parents considered the

program an important factor

in enrollment decisions

Enrollment rates for scholarship-

eligible graduates have

increased; persistence rates

have remained about the same

11

RAND Recommended Some Ways to Strengthen the Program

• Standardize efforts to provide information to students about the college and federal financial aid application process

• Implement practices to improve students’ knowledge about characteristics of The Promise scholarship program

• Implement online reporting to help students ascertain their eligibility status

• Institute a mentoring system in which Promise Scholars mentor high school students

12

Looking Ahead• These early findings provide a solid baseline for

future evaluations

• To fully assess the program’s impact, there must be enough time for a full cohort to go through high school, complete college, and enter the work force

• Future research should examine other important components of the program, such as

– High school graduation rates– Community engagement– Workforce characteristics

13

In July 2013, New Haven Promise Board asked RAND to Analyze Progress to Date of New Haven District

Reforms and New Haven Promise1. Measure the progress of New Haven Public School

(NHPS) students’ educational outcomes– Examine variations in educational outcomes since 2010 and

associations among School Change components– Compare district’s educational outcomes:

• Before 2010• With students in other CT districts

2. Evaluate the implementation of New Haven Promise– Analyze students’ and parents’ attitudes about Promise and NHPS– Review Promise’s performance to date:

• Compare program design to “promising practices” in field– Examine patterns of community-level indicators

• Since 2010• Compared to other cities in CT

3. Develop a tool for New Haven Promise to report education indicators each year

14

Some Research Ideas• How well have “Promise”-like programs (writ large)

met their intended goals?– More (needy) students attending PSEs?– Supported students in persisting and graduating from PSEs?– Encouraged students to return to region? – Motivated parents to enroll children in District?

• Are these programs helping students most in need to find life trajectories that best meet their desired goals or needs?

– PSE?– Find a job? Of what kind?

• Are these programs improving community outcomes?

15