from evidence to action
TRANSCRIPT
FROM EVIDENCE TO ACTION: The Story of Cash Transfers & Impact Evaluation in Sub-Saharan Africa
Ms. Jenn Yablonski, UNICEFDr. Sudhanshu Handa, University of North
CarolinaOn behalf of all the editors & authors
8th SPIAC-B MeetingSeptember 22, 2016
I. Introduction to the Transfer Project
II. From Evidence to Action: Addressing myths, findings and impacts
Outline
SDG 1.3: Implement nationally appropriate social protection systems & measures for all, including floors, and by 2030 achieve substantial coverage of the poor & the vulnerable.
Wide range of social & economic
outcomes
Universal coverage & access to
social protection
Key features of the African ‘Model’ of cash transfers
• Programs tend to be unconditional (or with ‘soft’ conditions)
• Targeting tends to be based on poverty & vulnerability OVC, labor-constrained, elderly
• Important community involvement in targeting process• Payments tend to be manual, ‘pulling’ participants to
pay-points Opportunity to deliver complementary services
Transfer Project countries
Ethiopia, Ghana, Kenya, Lesotho, Malawi, Madagascar, South Africa, Tanzania, Zambia and Zimbabwe
Transfer Project approach
Stage I. Design of Impact Evaluation
Stage II. Implementation & Analysis
Stage III. Use of Results & Dissemination
• Focus on supporting impact evaluations of national programmes & research-policy interface• Close relationship between all national stakeholders
• Impact evaluations as part of broader evidence/learning agendas & policy processes at national & regional level
Transfer Project approach
• Innovative research components: Mixed methods: quantitative, qualitative & local
economy impacts simulation Youth transitions to adulthood & HIV risk (UNICEF &
UNC) Productive impacts, local economy effects FAO (PtoP)
Country Quantitative Qualitative Lewie Other analysis
Ethiopia Non-experimental X X Targeting, payment process
Ghana Non-experimental X X Transfer payments
Kenya Experimental X X Operational effectiveness
Lesotho Experimental X X Rapid appraisal, targeting, costing & fiscal sustainability
Malawi (incl. Mchinji pilot)
Experimental X Not on Mchinji pilot
Targeting, operational effectiveness, transfer payments
South Africa Non-experimental X No Take up rate, targeting
Zambia (CG & MCTG) Experimental Not on
MCTGNot on MCTG
Impact comparisons across programme, targeting
Zimbabwe Non-experimental X XInstitutional capacity assessment rapid assessment, MIS analysis,
process evaluation
Methods used by the Transfer Project
Contribution to strengthening the evidence-based case for promoting social protection
as a poverty reduction instrument
Impact of Transfer Project:Global level
Generation of evidence on the broad range impacts of social cash transfers• Poverty impacts: child & household level• Social impacts: education, access to health, nutrition-sensitive
indicators, food security• Addressing economic & social determinants of HIV risk:
adolescent wellbeing• Building the economic case: economic & productive impacts
at household level; Impacts to beneficiaries & to local economy
Social cash transfers can work in low-income contexts, including SSA; can be affordable; are a worthwhile investment
Impact of Transfer Project:Regional level
• Strong evidence base on impact of cash transfers now available in SSA
• Context-specific design and implementation (home grown models, community participation, unconditional transfers, etc)
• Strengthen evidence base to feed to important regional processes (AU commitments, etc)
• Contribution to changing the discourse: SP as an investment, not a cost
Results from impact evaluations have influenced design of programs and
contributed to strategic policy decisions
Impact of Transfer Project:Country level
• Adjustment to program design & implementation (targeting, transfer size)
• Moving from cash to cash+ (specifically in terms of nutrition, agriculture & HIV/AIDS)- cash is important, but not sufficient
• Contribute to build & strengthen the case for scale-up & expansion:
Impact evaluations instrumental in strengthening reputation of social cash transfer programs, & confidence with which policymakers decide scale up
Economic & productive impacts: addressing concerns regarding dependency & contribution of the poor to inclusive growth
• Ghana – Gov triples transfer size after baseline simulations show level too low
• Kenya – Increase transfer size based on 4-year follow-up; Gov able to respond to criticisms w/ rigorous data
• Lesotho – Scale up after secondary analysis/ large impacts seen on the ground• Malawi – Ghana & Zambia lessons on predictability ensured payments not
skipped• Zimbabwe – Revised tageting system after positive comparison to more mature
programs• Zambia – Massive scale up: Gov contribution jumped from $4m to $35m (2014)
Continues to increase; campaign issue in recent elections
Research informing policy: Examples of scale-up
The highlights
Domain of impact Evidence Food securityAlcohol & tobaccoSubjective well-beingProductive activitySecondary school enrollmentSpending on school inputs (uniforms, shoes, clothes)Health, reduced morbidityHealth, seeking careSpending on healthNutritional statusIncreased fertility
Common criticisms or doubts that we hear on the ground
• Cash will be wasted: Will be spent on alcohol and other bads’• Its just a ‘hand-out’, not used for productive activities,
cannot contribute to development• Causes dependency, laziness• Leads to inflation, disrupts local economy• For child focused grants, increases fertility
Wasted? Across-the-board impacts on food security
Ethiopia SCTP
Ghana LEAP
Kenya CT-OVC
Lesotho CGP
Malawi SCTP
Zambia MCTG
Zambia CGP
ZIM HSCT
Spending on food & quantities consumed
Per capita food expenditures ü ü ü ü ü ü ü üPer capita expenditure, food items ü ü ü ü ü üKilocalories per capita ü üFrequency & diversity of food consumption
Number of meals per day ü ü üDietary diversity/Nutrient rich food ü ü ü ü ü üFood consumption behaviours
Coping strategies adults/children ü ü ü üFood insecurity access scale ü ü ü
Green check marks represent significant impact, black are insignificant and empty is indicator not collected
Wasted? No evidence cash is ‘wasted’ on alcohol & tobacco• Alcohol/tobacco represent 1% of budget share• Across 7 countries, no positive impacts found on
alcohol/tobacco• Data comes from detailed consumption modules covering
over 250 individual items• In Lesotho negative impacts on alcohol consumption
(possible decrease through decrease in poverty-related stress?)• Alternative measurement approaches yield same result:• “Has alcohol consumption increased in this community over the last
year?”• “Is alcohol consumption a problem in your community?”
Claim: Its just a ‘hand-out’, not used for productive activities, cannot contribute to development
School enrollment impacts (secondary age children): Same range as those from CCTs in Latin America
02468
101214161820
8
3
7 8
15
8 9
12
6
9
6
10
Primary enrollment already high, impacts at secondary level. Ethiopia is all children age 6-16.Bars represent percentage point impacts
Significant increase in share of households who spend on school-age children’s uniforms, shoes and other clothing
Ghana (LEAP) Lesotho (CGP) Malawi (SCTP) Zambia (MCTG) Zambia (CGP) Zim (HSCT) small hh
Zim (HSCT) large hh
0
5
10
15
20
25
30
35
11
26
30
23
32
11
5
Solid bars represent significant impact, shaded not significant. Lesotho includes shoes and school uniforms only, Ghana is schooling expenditures for ages 13-17. Other countries are shoes, change of clothes, blanket
ages 5-17.
Percentage point increase
Grade 3 math test – Serenje District, ZambiaMore kids in school but school quality still a challenge
Productive impacts positive but vary by transfer size, other operational features
Zambia Ethiopia Malawi ZIM Lesotho Kenya Ghana
Agricultural inputs +++ +++ +++ NS ++ NS +++
Agricultural tools +++ +++ +++ +++ NS NS NS
Agricultural production
+++ +++ +++ NS ++ NS NS
Livestock ownership
+++ +++ +++ +++ ++ Small hhlds
NS
Non farm enterprise +++ NS +++ +++ NS +FHH NS
Stronger impact Mixed impact Less impactNS=not significant+++=positive, significant---=negative, significant
Claim: Leads to lazinessWe find reduction in casual wage labor, shift to on farm and more productive activities
Zambia Kenya Malawi Ethiopia ZIM Lesotho Ghana
Agricultural/casual wage labor
- - - - - - - - - --- --- -- NS
Family farm +++ +++ NS +++ NS NS +++
Non farm business (NFE)
+++ +++ ++ NS NS NS NS
Non agricultural wage labor
+++ NS +++ NS NS NS
Shift from casual wage labor to family business consistently reported in qualitative fieldwork
Claim: Leads to laziness
“I used to be a slave to ganyu (labour) but now I’m a bit free.” -elderly beneficiary, Malawi
0.0
1.0
2.0
3.0
4D
ensi
ty
0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100age
Claim: Lead to inflation, disrupts local economy
• In six countries, tested for inflation in intervention versus control communities using basket of goods• No inflationary effects found
• Why not?• Beneficiaries small share of community, typically 15-20 percent• Poorest households, low purchasing power, don’t buy enough to affect
market prices• Sufficient supply response to meet demand
In fact, cash transfers lead to positive multiplier effects in local economy!!
Kenya (Nyanza)
Ethiopia (Abi_adi)
ZIM Zambia Kenya (Garissa)
Lesotho Ghana Ethiopia (Hintalo)
0
0.5
1
1.5
2
2.5
3
Multiplier: Amount generated in local economy by every $1 transferred
Claim: Cash transfers cannot contribute to developmentMultiplier effects of cash transfers in Zambia & Malawi Zambia (ZMK) Malawi (MK)
MCP CGP SCTPAnnual value of transfer (A) 720 720 26,169Savings 33 41 381Loan repayment 3 2 916Consumption 1021 767 41,520Livestock & productive assets 138 66 124Non agricultural assets 163Total spending (consumption + spending) (B) 1195 876 44,282
Estimated multiplier (B/A) 1.66 1.22 1.69
Impacts are based on econometric results and averaged across all follow-up surveys.Estimates for productive tools and livestock derived by multiplying average increase (numbers) by market price. Only statistically significant impacts are considered.
Claim: In child focused programs, increases fertilityEvidence suggests the opposite, if anything • Zambia Child Grant Programme
No impacts on total fertility or whether currently pregnant Some indication of improved birth outcomes (fewer pregnancy
complications)
Kenya Cash Transfer for Orphans & Vulnerable ChildrenReduction in early pregnancy among young women age 15-24 by 6 ppNo increase in number of children living in household
• South Africa Child Support Grant Reduction in early pregnancy by 11 percentage points
Where is evidence the weakest in terms of impact?Young child health and morbidityRegular impacts on morbidity, but less consistency on care seeking
Ghana LEAP
Kenya CT-OVC
Lesotho CGP
Malawi SCTP
Zambia CGP
Zimbabwe HSCT
Proportion of children who suffered from an illness/Frequency of illnesses ü ü ü ü ü üPreventive care ü ü üCurative care ü ü ü üEnrollment into the National Health Insurance Scheme üVitamin A supplementation ü
Supply of services typically much lower than for education sector.More consistent impacts on health expenditure (increases)
Green check marks represent positive protective impacts, black are insignificant and red is risk factor impact. Empty is indicator not collected
Where is evidence the weakest in terms of impact?No impacts on young child nutritional status (anthropometry)• Evidence based on Kenya CT-OVC, South Africa CSG, Zambia CGP,
Malawi SCTP, Zimbabwe HSCT However, Zambia CGP 13pp increase in IYCF 6-24 months
• Some heterogeneous impacts If mother has higher education (Zambia CGP and South Africa CSG) or if protected water source in home (Zambia CGP)
• Possible explanations…Determinants of nutrition complex - involve care, sanitation, water, disease environment & food
Weak health infrastructure in deep rural areasFew children 0-59 months in typical OVC or labor-constrained household
Meanwhile, emerging evidence that transfers enable safe-transition of adolescents into adulthood: Impacts on sexual debut among youth
Kenya (N=1,443) Malawi (N=1684) Zimbabwe (N=787) South Africa, girls (N = 440)
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
50%
36%
27%
17%
11%
44%
32%28%
Treat Control
-6 pp impact**
-7 pp impact**
-13 pp impact***
Kenya and Zimbabwe impacts driven by girls, Malawi driven by boys. Zambia no impacts.
-11 pp impact***
How to make cash work better? Impacts depend on transfer size‘Rule of thumb’ of 20 percent of consumption
Ghana 2010
Kenya CT-OVC
(big)
Burkina TASAF 2012
Kenya CT-OVC
RSA CSG
Malawi 2014
Lesotho CGP
(2010)
Ghana 2015
Kenya CT-OVC (small)
Zim (HSCT)
Zambia CGP
Zambia MCP
Malawi 2007
0
5
10
15
20
25
30
35
40Widespread impact
Selective impact
% o
r per
cap
ita c
onsu
mpti
on
How to make cash work better? Transfers must be predictable and regular!
Regular and predictable transfers facilitate planning, consumption smoothing and investment
Sep-10
Nov-10
Jan-1
1
Mar-11
May-11
Jul-1
1
Sep-11
Nov-11
Jan-1
2
Mar-12
May-12
Jul-1
2
Sep-12
0
1
Zambia CGP
# of
pay
men
ts
Apr-10
Jun-1
0
Aug-10
Oct-10
Dec-10
Feb-11
Apr-11
Jun-1
1
Aug-11
Oct-11
Dec-11
Feb-12
Apr-12
0
1
2
3
4
5
6
Ghana LEAP
# of
pay
men
ts
Regular and predictableLumpy and irregular
From Evidence to Action: Key messages
• Social cash transfers can be transformative for children, families & communities
Wide range of impacts across many social & economic domains — but depends on implementation & other factors (cash transfers are not a magic bullet)
• Impact evaluations have helped build credibility of social protection sector in SSA
• Evidence debunks myths, e.g. cash transfers do not create dependency
• Transfer Project utilizes an innovative approach: the ‘how’ matters• Evaluations & learning not only to ‘assess’ results, but to inform national policy &
progressively strengthen program design & implementation
THANK YOU!Website: www.cpc.unc.edu/projects/transfer
Facebook: https://www.facebook.com/TransferProject
Twitter: @TransferProjct
0.0
1.0
2.0
3.0
4D
ensi
ty
0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100age
0.0
1.0
2.0
3.0
4D
ensi
ty0 20 40 60 80 100
age
0.0
1.0
2.0
3.0
4D
ensi
ty
0 20 40 60 80age at baseline
Zambia SCT (Monze Evaluation)
0.0
2.0
4.0
6D
ensi
ty
0 20 40 60 80 100Age in Wave 1
Kenya CT-OVC
Malawi SCT Zimbabwe HSCT
Unique demographic structure of recipient householdsin OVC and labor-constrained models (missing prime-ages)
How much do programs pay? Benefit structure and level in selected programs (US$)
# members Ghana LEAP
Malawi SCT
MOZ PSA
Zimbabwe HSCT
Kenya CT-OVC
Zambia SCT
1 person 8 2.83 7 10 15 flat 12 flat
2 9.50 3.66 9 15
3 11 4.83 11 20
4+ 13.25 6.17 13.50 25
Beneficiary consumption pp per day
0.62 0.34 0.50 0.85 0.70 0.30
ZAM: $24 if disabled memberMLW: 0.83 and 1.67 top-up per child in primary and secondary school respectively