ifs impact assessment of the lone parent pilots project team: ifs: mike brewer, james browne, claire...
Post on 19-Dec-2015
218 views
TRANSCRIPT
IFS
Impact assessment of the lone parent pilots
Project team: IFS: Mike Brewer, James Browne, Claire Crawford & Lorraine Dearden. PSI: Genevieve Knight
© Institute for Fiscal Studies, 2006
Main findings & outline of talk
• Participation– after 12 months of pilots, 6% of eligible LPs received IWC– But participation still rising in Phase 1 areas after 20 months, so impact may grow
• Impact– Small impacts on moves off benefit and into work, particularly for lone parents recently
on NDLP
• Rest of talk – Methods & data– What might we expect?– Results– Concluding thoughts
• All based on DWP Research Report 415 (http://www.dwp.gov.uk/asd/asd5/rports2007-2008/rrep415.pdf).
© Institute for Fiscal Studies, 2006
What are we trying to achieve?
• Estimate impact of “lone parent pilots” on labour market outcomes of lone parents previously on IS
• Use administrative data– benefit receipt from DWP, employment spells from
HMRC (WPLS)
• Evaluation will tell us “how much”, not “why”
© Institute for Fiscal Studies, 2006
Why use time-limited in-work benefits?
• Wage growth (Connolly & Gottshalk, 2006; Walker and Lydon, 2005)
• One-off costs of starting work?• Very high discount rate?• Habits?
• Related policies:– SSP (Canada). Reduced welfare claims, increased
employment, increased earnings, increased income BUT no long-run effects (Card and Hyslop, 2005; Connolly and Gottshalk, 2002)
– UK experience: “Employment credit” for older workers; ERA & Pathways to Work pilots
© Institute for Fiscal Studies, 2006
Work and Pensions Longitudinal Study (WPLS)
• Administrative data-set combining– DWP: all benefit claims and participation in NDs from June 1999– HMRC: data from all P45/P46s (filled in when employers start/stop
paying someone)
• Matched on NINO plus name, DOB, gender, postcode• Personal information: age, gender, ethnicity and postcode
– but can merge other characteristics from other DWP databases.
• Outcomes– Which benefits/programmes– Whether in “work”– Doesn’t tell us earnings (yet), hours worked, or hourly wage
© Institute for Fiscal Studies, 2006
WPLS: problems with “work” measure • Lots of noise
– Some entries correspond receipt of taxable state benefit– Multiple entries for (apparent) same job– Multiple entries with same start date, different end date– Jobs where start or end known approximately (year, but not day)– Jobs where only end-date known– Jobs at times inconsistent with benefit receipt
• Need not include jobs paying < tax threshold– Tax threshold: £91. Min wage * 16 hours: £72 (April 2004).
• Does not capture self-employment nor informal employment (but neither would be eligible for IWC)
© Institute for Fiscal Studies, 2006
What outcomes are we measuring, and for whom?
• Measure impact on all eligible for IWC– Future work will measure impact on job retention for IWC recipients
• Divide eligible lone parents into “stock” and “flow”– Stock: eligible for IWC when pilot starts (large sample)– Flow: become eligible after pilot starts (more interesting in long-run)
• Outcomes measured in WPLS for people in WPLS (!)– “whether off IS/JSA/IB X days after first potentially eligible for IWC”– “whether in work X days after first potentially eligible for IWC”– Benefit outcomes until 31/3/06, work outcomes until 30/9/05
• “Work” measure in WPLS based on employers telling HMRC when they start/stop paying an employee
– Lots of noise– Does not capture informal employment– Need not include jobs paying < tax threshold (16 hours @ min wage)
© Institute for Fiscal Studies, 2006
Method
• “Difference-in-differences”– Compare outcomes in LPP areas with other areas after LPPs
started– Compare outcomes in LPP areas with other areas before LPPs
started– Attribute any differences to LPP
• No sensible control group within the pilot areas– People without children on JSA ?
• So use lone parents in all other parts of England as “control areas”, and estimate impact with difference in differences– Don’t identify matched control areas (Blundell et al (2005))
• [Differences between pilot and control areas]
© Institute for Fiscal Studies, 2006
Empirical specification
• Outcomes: – off benefit / in work X days after eligibility.
• Explanatory variables– History of benefit receipt and work (30 months before eligibility)– Whether claimed disability benefit, JSA, or been on NDLP in 30 months before
eligibility, entitlement to IS (at start of claim)– Personal characteristics (when first eligible)
• Age, number of children, age of youngest children, ethnicity, gender, month first eligible.
– Area characteristics (based on postcode when first eligible)• Indicators for JC+ district, supply of formal childcare (ward, 2003/4), unemployment (TTWA, 2002/3), deprivation
quintile (SOA, 2002/3), qualifications of non-working lone parents (SOA, 2001), employment rate (SOA, 2001), % of lone parents who are owner-occupiers (SOA, 2001)
– No time trend, but indicators for month• Linear probability model (ie OLS on binary outcome)
• Estimate impact – Across all districts (flow only; for stock, separate regression for each phase)– For each phase– For each district– By individual characteristics
© Institute for Fiscal Studies, 2006
In Work Credit: detail
• Gradual roll-out:Phase 1 (Apr 2004): Bradford, N London, SE London Phase 2 (Oct 2004): Leicestershire, Dudley and Sandwell, W
London, Lancashire W, Staffs, Leeds (+ Cardiff & Edinburgh) Phase 3 (Apr 2005): Brent, City & E London, S London, LambethPhase 4 (Oct 2005): Surrey, Sussex, Essex, Kent, Hampshire,
IoW, Berks, Bucks, Beds, Herts (not covered)• Five districts also have extra spending for “personal
advisers” (ND+fLP)• Affects around a third of LPs
• All of London and south-east in pilot areas. Argh!
© Institute for Fiscal Studies, 2006
Timeline and sample
Apr 02 – Mar 04
Apr 04 – Oct 04
Oct 04 – Mar 05
Apr 05 – Mar 06
Phase 1 Unaffected Affected
Phase 2 Unaffected Affected
Phase 3 Unaffected Affected
Phase 4 Ignored
Other areas Control
© Institute for Fiscal Studies, 2006
Sample size: how many are potentially eligible?
Stock Flow Flow
(outcomes after 6 months)
(outcomes after 12 months)
Phase 1 42,374 14,480 10,316
Phase 2 77,290 16,928 8,281
Phase 3 71,705 7,152 211
Total 191,369 38,560 18,808
Eligible on day pilot started
First eligible after pilot started
© Institute for Fiscal Studies, 2006
Typical profile: flow
0%
10%
20%
30%
40%
50%
60%
70%
-24 -18 -12 -6 0 6 12 18 24
Months since eligibility to fictitious pilot
Off
ben
efit
/in
wo
rk
Off benefit
In work
© Institute for Fiscal Studies, 2006
Typical profile: stock
0%
10%
20%
30%
40%
50%
60%
70%
-24 -18 -12 -6 0 6 12 18 24
Months since eligibility to fictitious pilot
Off
ben
efit
/in
wo
rk
Off benefit
In work
© Institute for Fiscal Studies, 2006
Typical profile: lessons
• Eligible population is prone to long spells on benefit– After 12 months, 15-20% of flow (10-15% of stock)
are off benefit
• “Work” measure looks too high, but changes are more plausible. – After 12 months, 5-10 ppt more are in work
• Pilot areas have worse outcomes than control areas, particularly Phases 1 & 3 (London)
© Institute for Fiscal Studies, 2006
Take-up (1): ever received IWC as % ever potentially eligible
0%
2%
4%
6%
8%
10%
0 5 10 15 20
Months since IWC began
Nu
mb
er e
ver
rece
ived
/
Nu
mb
er e
ver
elig
ible
Phase 1
Phase 2
Phase 3
Numerator: DWP financial data (stops Nov 2005). Denominator: WPLS
Corrected, 25/10/06
© Institute for Fiscal Studies, 2006
Take-up (2): new IWC claims as % of benefit exits and % of job starts
0%
50%
100%
150%
200%
0 5 10 15 20
Months since IWC began
Ne
w I
WC
cla
ims
/
Flo
ws
off
b
en
efi
t O
R m
ov
es
in
to w
ork
Phase 1,workPhase 2,workPhase 3,workPhase 1,benPhase 2,benPhase 3,ben
Numerator: DWP financial data. Denominator: WPLS
© Institute for Fiscal Studies, 2006
Results: flow
-10%
-5%
0%
5%
10%
15%
20%
Control
Phase 1
Phase 2
Phase 3
Differences
Off benefit after 9 months
Date first (potentially) eligible to LPPs
© Institute for Fiscal Studies, 2006
Results: flow
Impact of LPP (ppts)
3 months 6 months 9 months 12 months
1 Benefit
Work
-0.7
0.2
-0.3
0.4
-0.4
0.8
-0.3
2 Benefit
Work
0.2
0.3
0.5
0.8
0.9
0.7
0.8
3 Benefit
Work
-0.4
-.0.1
-0.4 -0.3
All areas -0.2
0.2
0.1
0.6
0.3
0.7
0.3
Bold and italicised means statistically different from zero
© Institute for Fiscal Studies, 2006
Results: stock (phase 1)
0%
5%
10%
15%
20%
25%
0 91 182 273 364 455 546 637 728 819
Control,before
Phase 1,before
Control,after
Phase 1,after
Days since eligibility to LPPs
Off benefit
© Institute for Fiscal Studies, 2006
Results: stock (phase 2)
0%
5%
10%
15%
20%
25%
0 91 182 273 364 455 546 637 728 819
Control,before
Phase 2,before
Control,after
Phase 2,after
Days since eligibility to programme
Off benefit
© Institute for Fiscal Studies, 2006
Results: stock (phase 3)
0%
5%
10%
15%
20%
25%
0 91 182 273 364 455 546 637 728 819
Control,after
Phase 3,before
Control,after
Phase 3,after
Days since eligibility to programme
Off benefit
© Institute for Fiscal Studies, 2006
Results: stock
Impact of LPP (ppts)
6 months 12 months 18 months 24 months
1 Benefit
Work0.0
0.2
0.5
0.2
0.8
0.9
1.2
2 Benefit
Work0.0
0.2
0.3
0.6
0.6
3 Benefit
Work0.7
1.1
1.2
Bold and italicised means statistically different from zero
© Institute for Fiscal Studies, 2006
Impact by subgroups
• Number of children, gender– No consistent pattern & few significant differences
• Age of youngest child– Weak (statistically insignificant) evidence that response
greater where youngest child age 3 or more• Recent participation in NDLP
– Evidence that impact greater for LPs who have recently been on NDLP, but estimate not consistent/stable across districts
• By district– Considerable variation
• IWC vs ND+fLP areas– Evidence that ND+fLP areas have WORSE outcomes
© Institute for Fiscal Studies, 2006
Results: flow, by age of youngest child
-2%
-1%
0%
1%
2%
3%
4%
5%
Under 1s Aged 1 to 3 Aged 3 to 5 Aged 5 to 8 Aged 8 to 11 Aged 11+
Off benefit after 9 months
(Results similar amongst stock)
© Institute for Fiscal Studies, 2006
District-level impacts: flow
-3.0
-2.0
-1.0
0.0
1.0
2.0
3.0
Off
ben
efit
aft
er 2
73 d
ays
(pp
ts)
Sig diff from 0
Sig diff from 0
Average (not sig diff from 0)
© Institute for Fiscal Studies, 2006
District-level impacts: flow
-3.0
-2.0
-1.0
0.0
1.0
2.0
3.0
Off
ben
efit
aft
er 2
73 d
ays
(pp
ts)
Sig diff from 0
Sig diff from 0
Average ND+fLP
Average IWC (sig diff from 0)
© Institute for Fiscal Studies, 2006
District-level impacts: stock
-3.0
-2.0
-1.0
0.0
1.0
2.0
3.0
Off
ben
efit
aft
er 3
64 d
ays
(pp
ts)
Sig diff from 0
Average
© Institute for Fiscal Studies, 2006
District-level impacts: stock
-3.0
-2.0
-1.0
0.0
1.0
2.0
3.0
Off
ben
efit
aft
er 3
64 d
ays
(pp
ts)
Sig diff from 0
Average IWC (sig diff from 0)
Average ND+fLP
© Institute for Fiscal Studies, 2006
Impact by recent NDLP participation
-1
-0.5
0
0.5
1
1.5
2
2.5
3
3.5
All flow Stock, phase 1 Stock, phase 2 Stock, phase 3
Pp
ts
Not on NDLP
Recent NDLP
Past NDLP
Sig diff from 0
“Recent” = On NDLP 6-12 months before eligible to LPP“Past” = On NDLP 13-30 months before eligible to LPP
Off benefit after 273/364 days
© Institute for Fiscal Studies, 2006
Summary of results
• Impact– Small impacts on flows off benefit, particularly for lone
parents recently on NDLP, and easier to detect in stock sample than flow
• Participation– After 12 months, 6% of LPs have received IWC– But participation still rising after 20 months, so impact may
grow
• Why impact so small?– Either LPs don’t hear about IWC, or they aren’t responding
to it yet– Is 0-2 ppts small? Level without treatment is 15-20%
© Institute for Fiscal Studies, 2006
Problems/extensions
• “Common trends”• So far, estimated many unrelated regressions
– What gain would there be from estimating a duration model with time-varying treatment?
• Pilot and control areas are different– Matched difference-in-differences (Blundell et al NDYP)
• Not yet used data on receipt of IWC– Joint model of NDLP & IWC & flows off benefit– Impact on retention
• Technical report due early 2008, future reports on more data late 2008 & 2010.