the future of monitoring and evaluation: lessons from a decade of impact evaluations
DESCRIPTION
The Future of Monitoring and Evaluation: Lessons from a Decade of Impact Evaluations. Eric Foster-Moore The World Bank | Africa Water Resources [email protected]. Our project will directly benefit 400,000 people … Our project will benefit the people of Mozambique … - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: The Future of Monitoring and Evaluation: Lessons from a Decade of Impact Evaluations](https://reader036.vdocuments.net/reader036/viewer/2022062501/56816663550346895dd9f313/html5/thumbnails/1.jpg)
The Future of Monitoring and Evaluation: Lessons from a Decade of Impact Evaluations
Eric Foster-MooreThe World Bank | Africa Water Resources
![Page 2: The Future of Monitoring and Evaluation: Lessons from a Decade of Impact Evaluations](https://reader036.vdocuments.net/reader036/viewer/2022062501/56816663550346895dd9f313/html5/thumbnails/2.jpg)
Development works
Our project will directly benefit 400,000 people…
Our project will benefit the people of Mozambique…
Number of women benefitted
Growth rate in country X increased by 2 percentage points
![Page 3: The Future of Monitoring and Evaluation: Lessons from a Decade of Impact Evaluations](https://reader036.vdocuments.net/reader036/viewer/2022062501/56816663550346895dd9f313/html5/thumbnails/3.jpg)
Art credit: @ChrisPlanicka
“No it won’t.”
“$195 billion in aid per year for 20 years would
eliminate poverty.”
Jeffrey SachsBig Push, Millennium Villages Project, The End of Poverty
William Easterly
The White Man’s BurdenDead Aid by Dambisa MoyoCan the West Save Africa?The white band’s burden…
![Page 4: The Future of Monitoring and Evaluation: Lessons from a Decade of Impact Evaluations](https://reader036.vdocuments.net/reader036/viewer/2022062501/56816663550346895dd9f313/html5/thumbnails/4.jpg)
1. Is aid effective?2. How do we know? 3. How do we apply this to our own work?
Three questions
![Page 5: The Future of Monitoring and Evaluation: Lessons from a Decade of Impact Evaluations](https://reader036.vdocuments.net/reader036/viewer/2022062501/56816663550346895dd9f313/html5/thumbnails/5.jpg)
Boone (1996): Nope.
Does aid increase growth? A macro perspective
Burnside and Dollar (2002). It depends. Effective when combined with “good policy.”
Easterly, Levine, and Roodman (2003). Not really.
![Page 6: The Future of Monitoring and Evaluation: Lessons from a Decade of Impact Evaluations](https://reader036.vdocuments.net/reader036/viewer/2022062501/56816663550346895dd9f313/html5/thumbnails/6.jpg)
Original Burnside-Dollar results
![Page 7: The Future of Monitoring and Evaluation: Lessons from a Decade of Impact Evaluations](https://reader036.vdocuments.net/reader036/viewer/2022062501/56816663550346895dd9f313/html5/thumbnails/7.jpg)
Easterly, Levine, and Roodman
![Page 8: The Future of Monitoring and Evaluation: Lessons from a Decade of Impact Evaluations](https://reader036.vdocuments.net/reader036/viewer/2022062501/56816663550346895dd9f313/html5/thumbnails/8.jpg)
Boone (1996): Nope.
Does aid increase growth? A macro perspective
Burnside and Dollar (2002). It depends. Effective when combined with “good policy.”
Easterly, Levine, and Roodman (2003). Not really.
Clemens, Radelet, Bhavnani, and Bazzi (2012). It still depends.
(Chris Blattman)
![Page 9: The Future of Monitoring and Evaluation: Lessons from a Decade of Impact Evaluations](https://reader036.vdocuments.net/reader036/viewer/2022062501/56816663550346895dd9f313/html5/thumbnails/9.jpg)
What about at the micro level?
Often just make them up: “This project will directly benefit 300,000 people.”
Move beyond correlations. Establish causality. Do an experiment. Some are natural, some are contrived
Natural experiment: half of a community displaced by a reservoir
Contrived experiment: the Fiala (2013) paper
![Page 10: The Future of Monitoring and Evaluation: Lessons from a Decade of Impact Evaluations](https://reader036.vdocuments.net/reader036/viewer/2022062501/56816663550346895dd9f313/html5/thumbnails/10.jpg)
Grants, loans, training, or a combo?
Fiala (2013):
Stimulating Microenterprise Growth: Results from a Loans, Grants, and Training Experiment in Uganda
![Page 11: The Future of Monitoring and Evaluation: Lessons from a Decade of Impact Evaluations](https://reader036.vdocuments.net/reader036/viewer/2022062501/56816663550346895dd9f313/html5/thumbnails/11.jpg)
Loans Loans plus training Cash Cash plus training Control
Treatment groups:
Follow up surveys after 6 and 9 months.
![Page 12: The Future of Monitoring and Evaluation: Lessons from a Decade of Impact Evaluations](https://reader036.vdocuments.net/reader036/viewer/2022062501/56816663550346895dd9f313/html5/thumbnails/12.jpg)
Loan only: effect is gone after nine months
Loans and training: 54 percent increase in profits for men Grant only: no effect whatsoever. For men, Fiala speculates, “Knowing that
the loan had to be repaid appears to have led them to use the money more effectively in the business.”
Grants and training: no effect
Women: no positive effects. “Family pressure on women appears to have significant negative effects on business investment decisions: married women with family nearby perform worse than the control group in a number of the interventions.”
Results
![Page 13: The Future of Monitoring and Evaluation: Lessons from a Decade of Impact Evaluations](https://reader036.vdocuments.net/reader036/viewer/2022062501/56816663550346895dd9f313/html5/thumbnails/13.jpg)
What is the effect of institutions on economic performance?
Acemoglu, Johnson, and Robinson (2000): The Colonial Origins of Comparative Development
![Page 14: The Future of Monitoring and Evaluation: Lessons from a Decade of Impact Evaluations](https://reader036.vdocuments.net/reader036/viewer/2022062501/56816663550346895dd9f313/html5/thumbnails/14.jpg)
Methods
Stronginstitutions
Strong economic performance
Other stuff
Instrumental variable
![Page 15: The Future of Monitoring and Evaluation: Lessons from a Decade of Impact Evaluations](https://reader036.vdocuments.net/reader036/viewer/2022062501/56816663550346895dd9f313/html5/thumbnails/15.jpg)
Hypothesis
![Page 16: The Future of Monitoring and Evaluation: Lessons from a Decade of Impact Evaluations](https://reader036.vdocuments.net/reader036/viewer/2022062501/56816663550346895dd9f313/html5/thumbnails/16.jpg)
“Differences in institutions account for roughly three quarters of the differences in income per capita.”
Not driven by outliers such as the United States, Canada, Australia, etc.
Not driven by simply being on the African continent.
Not driven by distance from the equator (i.e. geography).
Results
![Page 17: The Future of Monitoring and Evaluation: Lessons from a Decade of Impact Evaluations](https://reader036.vdocuments.net/reader036/viewer/2022062501/56816663550346895dd9f313/html5/thumbnails/17.jpg)
Compare program alternatives Quantify effects of program X in place Z at time T Force you to identify and test assumptions Microfinance, cash transfers, education, health, job
training, etc.
What RCTs do really well:
![Page 18: The Future of Monitoring and Evaluation: Lessons from a Decade of Impact Evaluations](https://reader036.vdocuments.net/reader036/viewer/2022062501/56816663550346895dd9f313/html5/thumbnails/18.jpg)
What you see: Policy and legal frameworks
strengthened Cash collection ratio; debts repaid Number of people in project area with
access to “Improved Water Sources” Percent of biological samples failing Number of direct beneficiaries, percent
of which are female Number of staff trained
Monitoring and evaluation: some examples
What you don’t see: Household income Business profitability Governance Strength of political parties Comparison to other
interventions Opportunity cost Articulation of assumptions
![Page 19: The Future of Monitoring and Evaluation: Lessons from a Decade of Impact Evaluations](https://reader036.vdocuments.net/reader036/viewer/2022062501/56816663550346895dd9f313/html5/thumbnails/19.jpg)
1. There’s an entire sector of projects that don’t get tested and where assumptions that are probably decades old dominate. Let’s bring the lessons of impact evaluation to these projects.
2. Need to be strategic about their application in order to learn something about human behavior.
Two problems with RCTs
![Page 20: The Future of Monitoring and Evaluation: Lessons from a Decade of Impact Evaluations](https://reader036.vdocuments.net/reader036/viewer/2022062501/56816663550346895dd9f313/html5/thumbnails/20.jpg)
1. Is aid effective? 2. How do we know? 3. How do we apply this to our own work?
Three questions
![Page 21: The Future of Monitoring and Evaluation: Lessons from a Decade of Impact Evaluations](https://reader036.vdocuments.net/reader036/viewer/2022062501/56816663550346895dd9f313/html5/thumbnails/21.jpg)
Backup
![Page 22: The Future of Monitoring and Evaluation: Lessons from a Decade of Impact Evaluations](https://reader036.vdocuments.net/reader036/viewer/2022062501/56816663550346895dd9f313/html5/thumbnails/22.jpg)
Solow model of growth
![Page 23: The Future of Monitoring and Evaluation: Lessons from a Decade of Impact Evaluations](https://reader036.vdocuments.net/reader036/viewer/2022062501/56816663550346895dd9f313/html5/thumbnails/23.jpg)
From E-L-R (2003)
![Page 24: The Future of Monitoring and Evaluation: Lessons from a Decade of Impact Evaluations](https://reader036.vdocuments.net/reader036/viewer/2022062501/56816663550346895dd9f313/html5/thumbnails/24.jpg)
From AJR