is it worthwhile to finish? youth job-training programmes (yjtp) … · 2009. 5. 15. · projoven...

Is it worthwhile to finish?

Youth job-training programmes (YJTP) and low completion rates: the case of Projoven-Peru

Denis de Crombrugghe, Henry Espinoza and Hans Heijke

4th IZA/WB Conference: Employment and Development

Maastricht Graduate School of Governance

Outline

• Introduction / Motivation

• Country background

• Projoven description

• Data discussion

• Empirical model

• Results

• Conclusions


Introduction (1)

• YJTPs impact => mixed evidence

– Wages

• little or no effects in the U.S. and Europe (Heckman et al., 1999)

• positive and statistically significant in LAC (Betcherman et al., 2004)

– Employment

• no significant effects in the U.S. (Heckman et al., 1999)

• small positive returns in Europe and LAC, particularly for women(Betcherman et al., 2004, and Irribarán and Rosas, 2008)


Introduction (2)

• … but YJTP still implemented because of:

– Rising inequalities

• Wages skilled workers relative to unskilled workers

• Access to education and vocational training


Introduction (3)

• Low completion rates phenomenon in YJTPProgramme Country Year Completion

rate Source

Projoven1 Peru 1996-2005 60% de Crombrugghe et al. (2008)

Projoven2 Uruguay 2004-2005 51% Projoven Survey (2006)

ChileJoven3 Chile 1996-1999 74% Santiago Consultores Asociados (1999)

Proyecto Joven Argentina 1996-1997 90% Aedo and Nuñez (2004)

Procajoven Panama 2005 77% Ibarrarán and Rosas (2007)

Juventud y Empleo Dominican Republic

2004 60% Card et al. (2007)

Training programmes4 Germany 2000-2002 69% Kluve et al. (2007)

National Job Training Partnership Act (JTPA)5

USA 1987-1989 58% Heckman et al. (2000)

Notes:1 The figure corresponds to the average completion rate of the first 13 public calls.

2 10th public call.

3 Phase 2. The figures belong to the “training and job experience” component (CEL).

4 Only men sample. Includes "occupation specific training programmes" and "general training programmes".

5 Only classroom training. In this case dropouts are individuals who enrolled in the programme but did not show up.


Introduction (4)

• … Nevertheless, drop out rates in YJTPshave been neglected in estimations of treatment effects…

– Exceptions:

• Theoretical: Heckman et. al. (1998); Angelucci and Attanasio (2006); and…

• Analysis of training intensity: Mealli et al. (1996); Kluve et al. (2007); and Flores-Lagunes et al. (2007).


Introduction (5)

• Aim of the paper:

– Is it worthwhile to finish?

• Better understanding of YJTP by estimating the Projoven’s effects (on wages and employment) in the

presence of dropouts with partial treatment. Controlling for:

– Ashenfelter’s dip

– Partial/endogenous/heterogeneous treatment

– Selection into work

• It’s not about why trainees dropout, but about the

consequences of that decision.


Country background

• Projoven is implemented in a period of vigorous economic growth. Av. real GDP growth (1996-2007): 5%

• Youth unemployment rate (16-25 years old) : 13% (1996-2007)

• The average real monthly wages for youngsters fell between 1996 and 2007: -26%

• In addition youngsters work in precarious conditions: 60% working in informal labour market.

• Youths represent 22% of total population (around 5 millions individuals)


Projoven description (1)

• Projoven was conceived as a policy intervention to

mitigate the impact of economic reforms on

economically disadvantaged youngsters.

• It’s a training programme which has trained more

than 40,000 youngsters in 15 public calls since

1996.

• Projoven targets youngsters (16-24) with low

educational attainment (max. secondary level) in

poverty.


Projoven description (2)

• Projoven selects and finances training institutions (ECAPs) to provide low-skilled jobs training.

• ECAPs provide three-month classroom training and subsequently (in principle) place trainees in a three-month internships (on-the-job training paid by firms). Afterwards the training firm may or may not hire the trainee on permanent basis– 20% of the trainees are not placed in internships.– of those placed in internships only half complete the

training.

• ECAPs are heterogeneous regarding quality of training. It may be a factor explaining the probability of completion.


Data discussion (1)

• Projoven-Peru 6th public call

– Non-experimental design

– Voluntary programme (possibility of self-selection)

– 1014 participants and 1014 control group

– Base line survey: November 1999 (2-3 months before

the programme)

– Three follow-up comparison surveys: 6 months after the programme (May 2001), 12 months after the programme

(November 2001), and 18 months after the programme (May 2002).

Data discussion (2) : Selection into treatment Official

Control

Group

Treated Group

New Control

Group

Total 1,014 992 992

City

Arequipa 205 205 205

Chiclayo 122 110 110

Cusco 117 117 117

Lima 368 361 361

Trujillo 202 199 199

Age (years) 19.8 19.6 19.7

Sex

Male 493 474 443

Female 521 518 549

Poverty Score 18.95 16.71 18.8

Household income per capita (S/.) 150.3 127.1 129.8

Lowest quartile hh income per capita (%) 19.9 29.6 31.9

Child (%) 24.5 14.2 15.3

Married (%) 21.9 10.3 11.9

Education Level (%)

No Education 0.0 0.2 0.0

Uncompleted Primary 0.6 0.5 0.6

Completed Primary 2.5 2.4 2.5

1st grade secondary 3.0 2.6 4.1

2nd grade secondary 4.2 3.7 3.4

3rd grade secondary 4.6 4.7 4.6

4th grade secondary 4.5 5.7 5.2

5th grade secondary 80.6 80.1 79.4

Official Control

Group

Treated Group

New Control

Group

Total 1,014 992 992

Labour market status(%)

Employed 60.4 60.3 60.28

Unemployed 29.5 26.8 26.41

Inactive 10.2 12.9 13.31

Transitions (%) (from October 1999 to November 1999)

Unemployed/OLF -> Employed 8.2 8.2 9.6

Employed -> Unemployed/OLF 4.0 4.1 3.1

Unemployed/OLF -> Unemployed/OLF 40.7 45.4 42.2

Employed -> Employed 47.0 42.3 45.1

No work experience (%) 27.6 37.1 34.6

Non-salaried family worker 6.7 13.5 16.8

Labour Income (S/. 2001)

Monthly1 238.71 158.54 173.65

Hourly 1.49 1.12 1.23

Working hours (week) 55.21 46.80 42.30


Data discussion (3)

• Dealing with non experimental design

– Control group formation

• 3 potential control group individuals per very trainee

• Search starts in the neighbourhood and district as limit

• Criteria

– For sex, they should have the same sex.

– For age, the control should be +/- 2 years with respect to the

participant.

– For poverty score, +/- 5 points.

– Education level, the same as the trainee.

• Programme operator choose as a control one individual among the three at baseline (2-3 months before the programme)


Data discussion (4)

• Dealing with non experimental design– Ashenfelter’s dip

• Although similar in observables, trainees are poorer than control group

• DiD misleading if not controlled for that.

– Propensity score matching with replacement• More comparable control group, using labour market

status change as regressors. (Heckman and Smith, 1999)

• Drawback: correlated observations which is controlled with cluster option (STATA)


Data discussion (5)

• Trainees distribution by completion level. Projoven 6th public call

2.2

57.0

15.9

16.1

8.9

0.0

0

10

20

30

40

50

60

1 2 3 4 5 6

Completed months in training

Pa

rtic

ipan

ts (

%)

Classroom training On-the-job training


Empirical framework: Diff-in-Diff (1)

• Effects on Wages

– Two-step GMM fixed effects (Mundlak’s terms) with sample selection correction [Semykina and Wooldridge, 2006]

• Selection into work:

• Introducing Mills’ ratios into wage equation

( ) ( )itiititit w+ζZ+δZΦ=Z|=S 1Pr ( )0,1 Normal~iit Z|w t= 1, .. . ,T

ittititititititit e+κλ +ζZ+η+H+βX+πfp+γp=Y ∑ˆ1 eit =ai +u it

it :ratio Mills'

:error term .di.-non

:effect time

:incomehh Ln child

:vector var.relatedlabour and cdemographi

:completionojt Projoven

: trainingclassroomion participat Projoven

λ

ei

H

Z++X

X)(

f

p)(

it

t

itit

it

it

it


Empirical framework: Diff-in-Diff (2)

• Effects on Employment likelihood

– Two-step GMM fixed effects (Mundlak’s terms) Linear probability model

• No Selection into work:

it

t

itit

it

it

it

ei

H

Z++X

X)(

f

p)(

:error term .di.-non

:effect time

:incomehh Ln child

:vector var.relatedlabour and cdemographi

:completionojt Projoven

: trainingclassroomion participatProjoven

( ) ittiitititittititiit e+H+ζZ+η+πpf+βX+γp=H,f,p,α|=L 11Pr


Empirical framework (3)

• Previous results

– Flores-Lagunes et al. (2007) [the U.S.] (2007)

and Chong and Galdo (2006) [Peru]: returns

classroom phase higher than OJT phase.

– Mealli et al. (1996) [the U.K.] completing YJTP

higher returns. Chacaltana et al. (2003) [Peru]

Projoven is only effective for those completing

OJT in the short run.

Results: Projoven’s effects on wages (1)

T=4: Nov-99, May-01, Nov-01 and May-02. This represents, baseline before Projoven, 6 months after Projoven, 12 months after Projoven and 18

months after Projoven, respectively.

Naive model (A) 2S GMM w/ SSC (B) 2S GMM w/ SSC - only trainees sample (C)

Coef. Std. Err. Coef. Std. Err. Coef. Std. Err.

Projoven 0.007 0.053 -0.068 0.099

Finish Projoven 0.092** 0.041 0.207 0.163 0.333* 0.172

Mill's ratio T=1 -0.613*** 0.156 -0.588*** 0.155

Mill's ratio T=2 -0.237* 0.133 -0.309** 0.127

Mill's ratio T=3 -0.174 0.121 -0.290*** 0.103

Mill's ratio T=4 -0.399** 0.178 -0.374** 0.183

Constant 5.338*** 0.204 5.578*** 0.221 5.800*** 0.219

Control variables yes yes yes

City dummies2 yes yes Yes

Time dummies yes yes yes

Mundlak's terms yes yes yes

Number of observations 4802 4802 2453

Wald test (overall) F(25, 1315) = 38.25 F(29, 1315) = 34.74 F(28, 894) = 41.91

R2 0.3022 0.3064 0.3727

Wald test significance of Mill's ratios ( )2

4χ = 16.52 ( )2

4χ = 19.07

p-value 0.0024 0.0008

Test relevance of Instrument (Anderson)

( )2

4χ = 315.303 ( )2

4χ = 143.984

p-value 0.000 0.000

Hansen J-test ( )2

3χ = 2.859 ( )2

3χ = 1.408

p-value 0.4138 0.7036

Results: Projoven’s effects on wages (2) -

heterogeneity Heterogeneity regarding

individuals characteristics

Heterogeneity across time

Finish*Projoven 0.507*

(0.275)

Heterogeneity of treatment

Finish*Female -0.218

(0.650)

Finish*No work experience -0.088

(0.083)

Finish*Low hh income pc -0.350***

(0.103)

Finish*T=2 0.479***

(0.141)

Finish*T=3 0.270

(0.207)

Finish*T=4 0.282

(0.209)

Number of obs 2453 2453

Wald test (overall) F( 31, 894) = 37.18 F( 28, 894) = 1.98

R2 0.360 0.374

Wald test significance of Mill's ratios ( )2

4χ 9.23 17.99

p-value 0.056 0.001

Test relevance of Instrument (Anderson) ( )2

3χ = 27.655 ( )2

4χ = 138.776

p-value 0.000 0.000

Hansen J-test ( )2

2χ = 0.536 ( )2

3χ = 1.394

p-value 0.7648 0.7069

Results: Projoven’s effects on employment (1) Naive LPM (A) LPM 2s GMM (B) LPM 2s GMM -

only trainees

sample (C)

Coef. Std.

Err.

Coef. Std.

Err.

Coef. Std.

Err.

Projoven -0.032 0.022 -0.048 0.049

Finish Projoven 0.059*** 0.020 0.084 0.079 0.125* 0.070

Constant 0.620*** 0.098 0.627*** 0.097 0.713*** 0.094

Control variables yes yes yes

City dummies1 yes yes yes

Time dummies yes yes yes

Mundlak's terms yes yes yes

Number of obs 7910 7910 3961

Wald test F(22, 1479) = 28.56 F(22, 1479) = 29.35 F(21, 991) = 26.19

R2 0.1151 0.1148 0.1094

Test relevance of Instrument

(Anderson)

( )2

4χ = 502.148 ( )2

4χ = 217.394

p-value 0.000 0.000

Hansen J-test ( )2

3χ = 3.247 ( )2

3χ = 2.612

p-value 0.355 0.4554

% of predicted val;ues out of range [0,1]

3.67 3.68 3.38

Results: Projoven’s effects on employment (2) Heterogeneity

regarding individuals

characteristics

Heterogeneity across time

Finish*Projoven 0.311**

0.130

Heterogeneity of treatment

Finish*Female -0.081

0.279

Finish*No work experience -0.480***

0.046

Finish*Low hh income pc -0.100**

0.045

Finish*T=2 0.037

0.053

Finish*T=3 0.034

0.098

Finish*T=4 0.303***

0.098

Number of observations 3961 3961

Wald test (overall) F(24, 991) = 26.43 F(21, 991) = 25.22

R2 0.018 0.091

Test relevance of Instrument

(Anderson) ( )2

3χ = 48.838 ( )2

4χ = 198.723

p-value 0.000 0.000

Hansen J-test ( )2

2χ = 3.022 ( )2

3χ = 2.647

p-value 0.221 0.4493


Conclusions (1)

• Completing classroom training yields no returns [wages and employment].

• Completing on-the-job training is positive [wages and employment] – Nevertheless, Projoven fails to produce income and

employment gains on the poorest trainees and on those with no work experience prior to the training.

• Why?– It might be related to training content:

• work habits (low-skilled jobs training)

• signalling effect (only observed once completed)

• Quality of training providers (ECAPs) – main determinant of completion


Conclusions (2)

• Further remarks

– Dropping out as a rational decision

• Can better results be achieved if we improved the content of the on-the-job training phase?

– Should job-training programmes target the

poorest ones and those without working

experience?

– Dropping out because of the economic context

• Should job-training programmes be countercyclical?


AppendixOfficial sample

0

10

20

30

40

50

60

70

80

ago-99

oct-99

dic-99

feb-00

abr-00

jun-00

ago-00

oct-00

dic-00

feb-01

abr-01

jun-01

ago-01

oct-01

dic-01

feb-02

abr-02

Employment share (%)

Control Group Treatment Group (all)

Contol Group (lowest quartile) Treatment Group (lowest quartile)

Training PeriodBase Period

0

50

100

150

200

250

300

350

400

aug

-99

oct

-99

dec

-99

feb

-00

apr-

00

jun

-00

aug

-00

oct

-00

dec

-00

feb

-01

apr-

01

jun

-01

aug

-01

oct

-01

dec

-01

feb

-02

apr-

02

Mo

nth

ly w

ag

es

(S/

20

01

)

Control Group Treatment Group (all)

Control Group (lowest quartile) Treatment Group (lowest quartile)

Training PeriodBase Period

Appendix

Probit regression – Selection into treatment ( to estimate propensity score) /

conditional on eligibility. Dep. Var. [Projoven = 1] Marginal

effects Standard error

Ln Family Income per capita -0.050* 0.027

Lowest quartile hh income per capita 0.063* 0.038

Age 0.014*** 0.005

Houshold size -0.004 0.005

Female 0.000 0.024

Married -0.171*** 0.039

Having Child -0.060 0.043

Years schooling -0.013 0.009

Father tertiary education 0.071* 0.040

Mather tertiary education 0.165*** 0.060

Additional training course 0.098*** 0.027

Unemployed/OLF -> Employed 0.008 0.043

Employed -> Unemployed/OLF 0.023 0.058

Unemployed/OLF -> Unemployed/OLF

-0.074** 0.036

No work experience (%) 0.175*** 0.037

Non-wage family worker 0.203*** 0.038

Num obs 2006

Wald X2(16) 151.97

Log Likelihood -1307.72

Pseudo R2 0.0594

Predicted outcomes

0 (%) 34.42

1 (%) 59.37

Appendix

Selection into work. Probit models. Dep. Variable: [Employed = 1]

T=1 T=2 T=3 T=4

Marginal

effects

Std.

Err.

Marginal

effects

Std.

Err.

Marginal

effects

Std.

Err.

Marginal

effects

Std.

Err.

Projoven 0.062** 0.034 -0.029 0.035 -0.030 0.033

Years schooling 0.023 0.017 0.039 0.033 0.018 0.019 0.014 0.017

Potential experience

0.019 0.033 0.025*** 0.008 -0.026 0.041 0.035 0.036

Female -0.199*** 0.037 -0.045 0.038 -0.063* 0.038 -0.014 0.036

Child 0.237** 0.125 0.070 0.144 0.018 0.154 0.100 0.107

Child*Female -0.209** 0.122 -0.399*** 0.068 -0.423*** 0.063 -0.314** 0.126

Married -0.066 0.097 -0.242*** 0.082 -0.163 0.102 -0.027 0.087

Ln hh income per capita

0.069*** 0.023 -0.047*** 0.018 -0.102*** 0.020 -0.075*** 0.016

Another job-

training course

-0.011 0.045 -0.036 0.054 -0.138** 0.062 -0.172*** 0.063

City dummies Yes Yes Yes Yes

Mundlak's terms Yes Yes Yes Yes

Number observations

1984 1984 1970 1972

Wald tests (overall) ( )2

17χ = 124.20 ( )2

18χ = 112.09 ( )2

18χ = 133.54 ( )2

18χ = 79.65

Pseudo R2 0.1041 0.099 0.135 0.0919

Log likelihood -1194.24 -1200.97 -1096.46 -875.05


References (1)

• Aedo, C. and S. Núñez. (2001) “The Impact of Training Policies in Latin America and the Caribbean: The Case of Programa Joven”. ILADES/Georgetown University. Graduate Program in Economics.

• Angelucci, M.; and O. Attanasio. (2006) “Estimating ATT effects with non-experimental data and low compliance”. IZA Discussion Paper No. 2368.Becker, G. (1964) Human Capital. New York: National Bureau of Economic Research.

• Betcherman, G., K. Olivas, and A. Dar. (2004) “Impacts of active labour market programs: new evidence from evaluations with particular attention to developing and transition countries.” World Bank Social Protection Discussion Paper 0402, 2004.

• Card, D., P. Ibarraran, F. Regalia, D. Rosas and Y. Soares (2007) “The Labor Market Impacts of Youth Training in the Dominican Republic: Evidence from a Randomized Evaluation”. NBER Working Paper No. 12883.

• Chacaltana, J., G. Guerrero, H. Espinoza, and O. Pain. (2003) “¿Qué funciona y qué no funciona en Projoven: una evaluación de los procesos de capacitación y medición de impacto”. IADB Report.

• Flores-Lagunes, A., A. Gonzalez and T. C. Neumann. (2007) “Estimating the effects of length of exposure to a training program: the case of Job Corps”.IZA Discussion Paper No. 2846.

• Heckman, J., R. LaLonde and J. Smith (1999), “The economics and econometrics of active labor market programs,” in O. Ashenfelter and D. Card (eds.), Chapter 31, Handbook of Labor Economics, Vol. IV, 1865-2073.


References (2)

• Heckman, J. J., N. Hohmann, J. Smith and M. Khoo. (2000) “Substitution and Dropout Bias in Social Experiments: A Study of an Influential Social Experiment”, Quarterly Journal of Economics, Vol. 115, No. 2, pp. 651-694.

• Heckman, J. J. and J. A. Smith (1999) “The pre-program earnings dip and the determinants of participation in a social program: implications for simple program evaluation strategies”. NBER Working Paper 6983.

• Heckman, J. J., J. A. Smith and C. Taber (1998) “Accounting for dropouts in evaluation of social programs”. The Review of Economics and Statistics, Vol. 80, No 1, pp. 1-14.

• Kluve, J., H. Schneider, A. Uhlendorff and Z. Zhao. (2007) “Evaluating continuous training programs using the generalized propensity score”. IZA Discussion Paper No. 3255.

• Mealli, F., S. Pudney and J. Thomas. (1996) “Training duration and post-training outcome: a duration-limited competing risks model”. The Economic Journal, Vol. 106, No. 435, pp. 422-433.

• Mincer, J. (1958) “Investment in human capital and personal income distribution”. Journal of Political Economy, Vol. 66, No. 3, pp. 281-302.

• Semykina, A. and Wooldridge, J. (2006) “Estimating panel data models in presence of endogeneity and selection: theory and application”, Discussion paper, Department of Economics, Michigan State University, East Lansing, MI.

• Santiago Consultores Asociados (1999): "Evaluación Ex-Post Chile Joven Fase II”, Mimeo

is it worthwhile to finish? youth job-training programmes (yjtp) … · 2009. 5. 15. · projoven...

Documents