Kolajo Paul Afolabikolajo_afolabi@mail.harvard.edu

Harvard Graduate School of EducationNational Partnership for Educational Access Conference

A il 28 2011April 28, 2011

1

Hourglass labor market (Carnevale Smith &

Enrollment rising, but completion rate falling(Carnevale, Smith, &

Strohl, 2010)BA recipients earn 66% more over their

completion rate falling (Turner, 2004)At 4-yr colleges, 46%

more over their lifetimes (Baum, Ma, & Payea, 2010)Ci i t d

obtain a BA in 5 yrs◦ Proportion lower for

students of color andCivic engagement and health benefits (Curie & Moretti, 2003; Dee, 2004)

students of color and low-income students (NCES, 2003)

In Boston 35% of2004)Increased tax revenues (NCPPHE, 2004)

In Boston, 35% of students obtain a BA in 6yrs (BPIC, 2008)

Economic & Social Benefits

College Completion Realities 2

Lots of research few apples to applesLots of research, few apples-to-apples comparisons◦ Myers (2003); Turner (2004)Recent rigorous research has focused on 2yr and nontradtional students◦ Scrivener & Weiss (2009); Richburg Hayes et al◦ Scrivener & Weiss (2009); Richburg-Hayes et al.

(2009); Bettinger & Baker (2011)Integration to college and access to information and ◦ Tinto (1993); Cushman (2006)Information key for low income and firstInformation key for low-income and first generation students◦ Kane & Avery (2004)◦ Person, Rosenbaum, & Deil-Amen (2006)

3

Q1: Who participates in Bottom Line’s Access and Success programs?Q2: How do students describe their reasons for leaving college?for leaving college?Q3: When do Access-Only students leave college?college?Q4: What is the effect of the Success program on the probability on degree attainment? p y g

4

Boston-based college access and success

Counseling starts summer before collegeaccess and success

programAccess program serves

summer before college enrollmentDEAL framework: p g

roughly 500 students at 38 high schoolsSuccess Program

Degree, Employment, Aid, and LifeTwo one-on-oneSuccess Program

serves 750 students at 18 target schools

Two one on one campus visits a monthAlso connect over phone email

Program Overview Student Success Counseling

phone, email

g O Counseling

5

All participants in B Li ’

N=2068P i i hBottom Line’s

programs 2002-2008

Participants have a GPA>2.5 and are either low-income or2008

Data from program databases, National

either low income or first generationAll start at 4yr

llStudent Clearinghouse, IPEDSNotes from Success

colleges3 cohorts with 6 yrs of data (n=606)Notes from Success

counselors on students

of data (n 606)5 cohorts with 4 yrs of data (n=1355)

Data Sample 6

Q1: Described who Bottom Line’s participants d h h llwere and where they went to college

Q2: Explored how students describe leaving college by analyzing counselor logs fromcollege by analyzing counselor logs from Bottom Line’s database for the Success participants who left collegeQ3: Looked at when Access-Only students left college by analyzing enrollment data for t d t h l ti i t d i th Astudents who only participated in the Access

Program

7

V i bl A St d t S St d tVariable Access Students Success StudentsHS GPA (out of 4) 2.89 3.03SAT (02-05, max 1600) 892 852SAT (02 05, max 1600) 892 852SAT (06, max 2400) 1296 1293Percent low-income 57% 77%Percent first gen 75% 87%Percent black 55% 52%Percent female 70% 74%Percent Hispanic/Latino 25% 28%P t i i t ll 72% 51%Percent in private college 72% 51%Percent in suburban college 46% 39%Percent in urban college 45% 61%Percent in urban college 45% 61%

8

Twenty-six percent of students who participate only in the Access Program obtain a college degree within four years, and 45 percent obtain a degree within six yearspercent obtain a degree within six years. Of the students who leave college, only about half leave by the end of their first year.half leave by the end of their first year.Enrollment data show that students are leaving college at all points in their post-secondary years.

9

Students described their reasons for leaving college as being related to:◦ Issues external to their campus life

A d i i i◦ Academic transition◦ Economic frustration

T d i i t ti dbl k◦ Temporary administrative roadblocks.

10

Enrollment in Bottom Line’s Success program l il bl t t d t h ll d twas only available to students who enrolled at

a “target” institutionStudents who participated in the AccessStudents who participated in the Access program but did not attend a target institution make an arguably good control groupgroupSuccess program participants were matched to their Access-only counterparts using y p gnearest neighbor propensity score matching (with replacement)

11

Outcomes◦ GRAD6YRi: obtained a degree within 6yrs of

enrollment◦ GRAD4YRi: obtained a degree within 4yrs ofGRAD4YRi: obtained a degree within 4yrs of

enrollmentTreatment◦ SUCCESSi: Participated in the Success programMatching Variables◦ BACKGROUND◦ BACKGROUNDi◦ ACADEMICi◦ COLLEGEjj

12

Propensity Score Estimation

[ ] ( )0 1 2 3

1Pr 1

1 i i ji BACKGROUND ACADEMICS COLLEGE

SUCCESSe

β β β β− + + += =

+◦ For the ith student at the jth collegeAverage Treatment Effect

( )1 e+

Average Treatment Effect

( ) ( )1ˆ 1 0match i jt

Y Y SUCCESS Y SUCCESSn

⎡ ⎤= = − =⎣ ⎦∑

◦ Where Y is obtaining a degree, i indexes the treated cases, and j indexes the control cases

tn

, j13

Ana Maya

Participated in Access

Participated in Access

Participated in Access

Participated in AccessAccessAccess

-First-generation

AccessAccess

-First-generation

-Latina

-HS GPA = 2.9

-Latina

-HS GPA = 3.1Non-targetTarget

-Enrolls in 4-yr, mid-size, public, selective college

-Enrolls in 4-yr, large, public, selective college

Non target school = no Success counseling

Target school = Success counseling selective college

Propensity to be

selective college

Propensity to be

gg

p yin Success = .75

p yin Success = .73 1

4

Success Access Difference

Unmatched

6yr grad rate 73% 45% 28%

4yrgrad rate 45% 27% 18%4yrgrad rate 45% 27% 18%

M h d

6yr grad rate 81% 39% 43%

Matched4yrgrad rate 42% 23% 19%

6yr grad rate 82% 48% 34%Parametric Estimates

y g

4yrgrad rate 57% 37% 20%

15

Limitations of propensity score matchingPSM l d bi d ti t h◦ PSM only produces unbiased estimates when assignment to treatment conditional on the covariates is independent of the outcome (Dehejia & Wahba, 1999; Diaz & Handa, 2006 )◦ Self-selection into target schools

Would negatively bias my estimatesWould negatively bias my estimatesGeneralizability◦ Already self-selected groupy g p◦ Mixed evidence on effectiveness of counseling

with other populations (Bettinger & Baker, 2011; Scrivener & Weiss 2009)Scrivener & Weiss, 2009)

16

Evidence continues to gather that information and li b k i h l i dcounseling can be key in helping students to

access and succeed in higher education◦ Bettinger & Baker (2011)g ( )◦ Bettinger, Long, Oreopoulos, & Sanbanmatsu

(2010) S i & W i (2009)◦ Scrivener & Weiss (2009)◦ Richburg-Hayes et al. (2009)Need to establish importance of well-identifiedNeed to establish importance of well identified assignment mechanisms in demonstrating program effectiveness

17

Avery, C. & Kane, T. (2004). Student perceptions of college opportunities: The Boston COACH program. In Caroline Hoxby (Ed.), College choices: The economics of where to go, when to go, and how to pay for it (pp. 355-391).

h h f hChicago: The University of Chicago Press.Baum, S., Ma, J., Payea, K. (2010). Education pays 2010; The benefits of higher education for individuals and society. New York: The College Board.Bettinger E & Baker R (2011) The effects of student coaching in college:Bettinger, E. & Baker, R. (2011). The effects of student coaching in college: An evaluation of a randomized experiment in student mentoring. NBER Working Paper No. 16881. National Bureau of Economic Research.Bettinger, E. P., Long, B. T., Oreopoulos, P., & Sanbonmatsu, L. (2009). TheBettinger, E. P., Long, B. T., Oreopoulos, P., & Sanbonmatsu, L. (2009). The role of simplification and information in college decisions: Results from the H&R Block FAFSA experiment. NBER Working Paper No. 15361. National Bureau of Economic Research.Boston Private Industry Council. (2008). Getting to the finish line: College enrollment and graduation; A seven year longitudinal study of the Boston Public Schools class of 2000. Boston, Massachusetts: Author.

18

Carnevale, A.P., Smith, N., & Strohl, J. (2010). Help wanted: Projections of jobs and education requirements though 2018. Washington, DC: Georgetown University Center on Education and the Workforce.Currie, J. & Moretti, E. (2003). Mother’s education and the intergenerational transmission of human capital: Evidence from college openings. Quarterly Journal of Economics, 118(4), 1495-1532.Cushman K (2006) First in the family: Your college years Providence RI:Cushman, K. (2006). First in the family: Your college years. Providence, RI: Next Generation Press.Dee, T. S. (2004). Are there civic returns to education? Journal of Public Economics, 88(9-10), 1697-1720.Economics, 88(9 10), 1697 1720.Dehejia, R. H., & Wahba, S. (1999). Causal effects in nonexperimental studies: Reevaluating the evaluation of training programs. Journal of the American Statistical Association, 94(448), 1053–1062.Diaz, J. J., & S. Handa. (2006). An assessment of propensity score matching as a nonexperimental impact estimator: Evidence from Mexico's PROGRESA program. Journal of Human Resources, 41(2), 319-345

19

M R D (2003) C ll S P W hi t DC P thMeyers, R. D. (2003) College Success Programs. Washington, DC: Pathways to College Network.National Center for Education Statistics (NCES). (2004). College Persistence on the Rise? Changes in 5-Year Degree Completion and Postsecondaryon the Rise? Changes in 5 Year Degree Completion and Postsecondary Persistence Rates Between 1994 and 2000. Washington, DC: U.S. Department of Education. National Center for Public Policy and Higher Education (NCPPHE). (2004. Policy alert: The educational pipeline: Big investment, big returns. Washington, DC: Author.Person, A., Rosenbaum, J., & Deil-Amen, R. (2006). Student planning and information problems in different college structures Teachers Collegeinformation problems in different college structures. Teachers College Record, 108, 374–396.Richburg-Hayes, L., Brock, T., LeBlanc, A., Paxson, C., Rouse, C.E., & Barrow, L. (2009) Rewarding persistence: Effects of a performance-based scholarship ( ) g p p pprogram for low-income parents. New York, NY: MDRC.Scrivener, S., & Weiss, M. J. (2009). More Guidance, Better Results? New York: MDRC.

20

Ti t V (1993) L i ll R thi ki th d f t d tTinto, V. (1993). Leaving college: Rethinking the causes and cures of student attrition (2nd ed.). Chicago; London: University of Chicago Press.Turner, S. E. (2004). Going to college and finishing college: Explaining different educational outcomes. In Caroline Hoxby (Ed.), College choices: Thedifferent educational outcomes. In Caroline Hoxby (Ed.), College choices: The economics of where to go, when to go, and how to pay for it (pp. 13-56). Chicago: The University of Chicago Press.

21

Roughly 22% of students who attend target schools don’t participate in SuccessDoesn’t really effect 6yr estimate, but 4yr attenuates significantlyattenuates significantly

Exclude –PSM Exclude - PM ITT - PSM6yr 32% 41% 20%4yr 5% 22% 0%4yr 5% 22% 0%

22

Bentley College Salem State CollegeBoston College Smith CollegeBoston University Suffolk UniversityBridgewater State College Tufts UniversityClark University (MA) U Mass-AmherstCollege of The Holy Cross U Mass-BostonMassachusetts College of Liberal Arts U Mass-DartmouthLiberal Arts U Mass DartmouthNortheastern Foundation Year

Worcester Polytechnic Institute

Northeastern University Worcester State College23

Class Access Only Success Total2002 118 90 2082003 118 68 1862004 145 67 2122005 293 98 3912005 293 98 3912006 215 143 3582007 209 143 3522007 209 143 3522008 190 171 361

24Total 1288 780 2068