quasi experimental methods: difference in differences -...
TRANSCRIPT
![Page 1: Quasi experimental methods: Difference in differences - CEGAcega.berkeley.edu/assets/cega_events/36/Quasi-Experimental_Methods.pdf · Quasi experimental methods: Difference in differences](https://reader030.vdocuments.net/reader030/viewer/2022040310/5d4ce4e488c993821c8b6afd/html5/thumbnails/1.jpg)
Quasi experimental methods: Difference in differences
Quasi experimental methods: Difference in differences
Prashant Bharadwaj
University of California, San Diego March 24th, 2010 Material in this presentation developed from CEGA and World Bank materials.
![Page 2: Quasi experimental methods: Difference in differences - CEGAcega.berkeley.edu/assets/cega_events/36/Quasi-Experimental_Methods.pdf · Quasi experimental methods: Difference in differences](https://reader030.vdocuments.net/reader030/viewer/2022040310/5d4ce4e488c993821c8b6afd/html5/thumbnails/2.jpg)
Quick re‐capQuick re‐cap
1.
While using non experimental data to infer causal relationships, we must think through sample
selection and omitted variables bias2.
Comparing just pre‐post or participant vs
non‐
participant is not enough3.
This lecture is about differencing out the potential
omitted variables bias
![Page 3: Quasi experimental methods: Difference in differences - CEGAcega.berkeley.edu/assets/cega_events/36/Quasi-Experimental_Methods.pdf · Quasi experimental methods: Difference in differences](https://reader030.vdocuments.net/reader030/viewer/2022040310/5d4ce4e488c993821c8b6afd/html5/thumbnails/3.jpg)
Difference‐in‐Differences, GraphicallyDifference‐in‐Differences, Graphically
Pre Post
Treatment
Control
![Page 4: Quasi experimental methods: Difference in differences - CEGAcega.berkeley.edu/assets/cega_events/36/Quasi-Experimental_Methods.pdf · Quasi experimental methods: Difference in differences](https://reader030.vdocuments.net/reader030/viewer/2022040310/5d4ce4e488c993821c8b6afd/html5/thumbnails/4.jpg)
Difference‐in‐Differences, GraphicallyDifference‐in‐Differences, Graphically
Pre Post
Effect of program using
only pre‐
& post‐
data
from T group (ignoring
general time trend).
![Page 5: Quasi experimental methods: Difference in differences - CEGAcega.berkeley.edu/assets/cega_events/36/Quasi-Experimental_Methods.pdf · Quasi experimental methods: Difference in differences](https://reader030.vdocuments.net/reader030/viewer/2022040310/5d4ce4e488c993821c8b6afd/html5/thumbnails/5.jpg)
Difference‐in‐Differences, GraphicallyDifference‐in‐Differences, Graphically
Pre Post
Effect of program using
only T & C comparison
from post‐intervention
(ignoring pre‐existing
differences between T & C
groups).
![Page 6: Quasi experimental methods: Difference in differences - CEGAcega.berkeley.edu/assets/cega_events/36/Quasi-Experimental_Methods.pdf · Quasi experimental methods: Difference in differences](https://reader030.vdocuments.net/reader030/viewer/2022040310/5d4ce4e488c993821c8b6afd/html5/thumbnails/6.jpg)
Difference‐in‐Differences, GraphicallyDifference‐in‐Differences, Graphically
Pre Post
![Page 7: Quasi experimental methods: Difference in differences - CEGAcega.berkeley.edu/assets/cega_events/36/Quasi-Experimental_Methods.pdf · Quasi experimental methods: Difference in differences](https://reader030.vdocuments.net/reader030/viewer/2022040310/5d4ce4e488c993821c8b6afd/html5/thumbnails/7.jpg)
Difference‐in‐Differences, Graphically
Difference‐in‐Differences, Graphically
Pre Post
Effect of program
difference‐in‐difference
(taking into account pre‐
existing differences
between T & C and general
time trend).
![Page 8: Quasi experimental methods: Difference in differences - CEGAcega.berkeley.edu/assets/cega_events/36/Quasi-Experimental_Methods.pdf · Quasi experimental methods: Difference in differences](https://reader030.vdocuments.net/reader030/viewer/2022040310/5d4ce4e488c993821c8b6afd/html5/thumbnails/8.jpg)
Identifying AssumptionIdentifying Assumption
• Whatever happened to the control group over time is what would have happened to the
treatment group in the absence of the program.
Pre Post
Effect of program
difference‐in‐difference
(taking into account pre‐
existing differences
between T & C and general
time trend).
![Page 9: Quasi experimental methods: Difference in differences - CEGAcega.berkeley.edu/assets/cega_events/36/Quasi-Experimental_Methods.pdf · Quasi experimental methods: Difference in differences](https://reader030.vdocuments.net/reader030/viewer/2022040310/5d4ce4e488c993821c8b6afd/html5/thumbnails/9.jpg)
DD: Simple ExampleDD: Simple Example
Pre Post Difference
Treatment 100 170 70
Control 65 100 35
Difference in Difference 35
![Page 10: Quasi experimental methods: Difference in differences - CEGAcega.berkeley.edu/assets/cega_events/36/Quasi-Experimental_Methods.pdf · Quasi experimental methods: Difference in differences](https://reader030.vdocuments.net/reader030/viewer/2022040310/5d4ce4e488c993821c8b6afd/html5/thumbnails/10.jpg)
Example: Change in marriage lawsExample: Change in marriage laws
1.
In 1957 Mississippi amended its marriage law
1.
Raised minimum age for men and women
2.
Introduced parental consent laws
3.
Proof of age, blood tests, other restrictions
![Page 11: Quasi experimental methods: Difference in differences - CEGAcega.berkeley.edu/assets/cega_events/36/Quasi-Experimental_Methods.pdf · Quasi experimental methods: Difference in differences](https://reader030.vdocuments.net/reader030/viewer/2022040310/5d4ce4e488c993821c8b6afd/html5/thumbnails/11.jpg)
Impact of changes in marriage lawImpact of changes in marriage law
1.
How can we figure out the impact of this marriage law on
outcomes such as
1.
Marriages
2.
Fertility
3.
Education
2.
Relevant for our context in terms of empowering women –
does postponement of marriage empower women by
delaying fertility and increasing education?
![Page 12: Quasi experimental methods: Difference in differences - CEGAcega.berkeley.edu/assets/cega_events/36/Quasi-Experimental_Methods.pdf · Quasi experimental methods: Difference in differences](https://reader030.vdocuments.net/reader030/viewer/2022040310/5d4ce4e488c993821c8b6afd/html5/thumbnails/12.jpg)
What happened after 1957?What happened after 1957?
![Page 13: Quasi experimental methods: Difference in differences - CEGAcega.berkeley.edu/assets/cega_events/36/Quasi-Experimental_Methods.pdf · Quasi experimental methods: Difference in differences](https://reader030.vdocuments.net/reader030/viewer/2022040310/5d4ce4e488c993821c8b6afd/html5/thumbnails/13.jpg)
DD in this contextDD in this context
1.
Marriage law affected Mississippi residents and immediate neighbors (treated
group)
2.
But did other changes happen during this time that would affect marriage rates, fertility and education?
1.
Changes in other laws –
compulsory schooling etc
2.
Changes in work opportunities for women
3.
Migration
4.
Pre 1957 trends in marriage?
3.
Nearby states, counties are a potential control
group
![Page 14: Quasi experimental methods: Difference in differences - CEGAcega.berkeley.edu/assets/cega_events/36/Quasi-Experimental_Methods.pdf · Quasi experimental methods: Difference in differences](https://reader030.vdocuments.net/reader030/viewer/2022040310/5d4ce4e488c993821c8b6afd/html5/thumbnails/14.jpg)
Pre and post marriage lawPre and post marriage law
![Page 15: Quasi experimental methods: Difference in differences - CEGAcega.berkeley.edu/assets/cega_events/36/Quasi-Experimental_Methods.pdf · Quasi experimental methods: Difference in differences](https://reader030.vdocuments.net/reader030/viewer/2022040310/5d4ce4e488c993821c8b6afd/html5/thumbnails/15.jpg)
Appropriate control groupsAppropriate control groups
1.
Counties next to Mississippi
2.
Age groups not affected by the law
3.
Triple difference:
DD estimates for younger vs older age groups
1.
In the event of treatment specific trends affecting all age groups
![Page 16: Quasi experimental methods: Difference in differences - CEGAcega.berkeley.edu/assets/cega_events/36/Quasi-Experimental_Methods.pdf · Quasi experimental methods: Difference in differences](https://reader030.vdocuments.net/reader030/viewer/2022040310/5d4ce4e488c993821c8b6afd/html5/thumbnails/16.jpg)
Trends in treatment and controlTrends in treatment and control
![Page 17: Quasi experimental methods: Difference in differences - CEGAcega.berkeley.edu/assets/cega_events/36/Quasi-Experimental_Methods.pdf · Quasi experimental methods: Difference in differences](https://reader030.vdocuments.net/reader030/viewer/2022040310/5d4ce4e488c993821c8b6afd/html5/thumbnails/17.jpg)
Distance to MississippiDistance to Mississippi
![Page 18: Quasi experimental methods: Difference in differences - CEGAcega.berkeley.edu/assets/cega_events/36/Quasi-Experimental_Methods.pdf · Quasi experimental methods: Difference in differences](https://reader030.vdocuments.net/reader030/viewer/2022040310/5d4ce4e488c993821c8b6afd/html5/thumbnails/18.jpg)
Did other things change differentially
Did other things change differentially
![Page 19: Quasi experimental methods: Difference in differences - CEGAcega.berkeley.edu/assets/cega_events/36/Quasi-Experimental_Methods.pdf · Quasi experimental methods: Difference in differences](https://reader030.vdocuments.net/reader030/viewer/2022040310/5d4ce4e488c993821c8b6afd/html5/thumbnails/19.jpg)
Placebo assignment of lawPlacebo assignment of law
![Page 20: Quasi experimental methods: Difference in differences - CEGAcega.berkeley.edu/assets/cega_events/36/Quasi-Experimental_Methods.pdf · Quasi experimental methods: Difference in differences](https://reader030.vdocuments.net/reader030/viewer/2022040310/5d4ce4e488c993821c8b6afd/html5/thumbnails/20.jpg)
Impact of marriage lawImpact of marriage law
1.
Marriage decline2.
Immediate rates of fertility decline1.
Unintended effect: Some evidence of increases in
illegitimate childbearing2.
No effect on completed fertility
3.
Enrollment in school increases1.
Overall high school completion rates higher
![Page 21: Quasi experimental methods: Difference in differences - CEGAcega.berkeley.edu/assets/cega_events/36/Quasi-Experimental_Methods.pdf · Quasi experimental methods: Difference in differences](https://reader030.vdocuments.net/reader030/viewer/2022040310/5d4ce4e488c993821c8b6afd/html5/thumbnails/21.jpg)
Indonesia Schooling (Duflo)Indonesia Schooling (Duflo)
Duflo 2001
![Page 22: Quasi experimental methods: Difference in differences - CEGAcega.berkeley.edu/assets/cega_events/36/Quasi-Experimental_Methods.pdf · Quasi experimental methods: Difference in differences](https://reader030.vdocuments.net/reader030/viewer/2022040310/5d4ce4e488c993821c8b6afd/html5/thumbnails/22.jpg)
ConclusionsConclusions
Difference in differences is a good non‐experimental method for impact evaluation if:
1.Randomization is difficult or expensive2.Control group is well defined3.Assumption that trends in treatment and control in
absence of treatment would have been the same is true