2014 04 28 and 30
DESCRIPTION
lecture slides econometrics easy berkeley 141TRANSCRIPT
-
Ch. 13: Experiments and
Quasi-Experiments
Econ 141 Spring 2014
Lecture: April 28 & 30, 2014
Bart Hobijn
4/28&30/2014 Econ 141, Spring 2014 1
The views expressed in these lecture notes are solely those of the instructor and do not necessarily
reflect those of the UC Berkeley, or other institutions with which he is affiliated.
-
Sources of exogenous variation
Chapter 12 showed that successful identification of causation requires exogenous variation in the
explanatory variable in the regression.
Ch.13 introduces two types of such variation
1. Experiments
Situation in which exogenous variation is explicitly
generated by the researcher.
2. Quasi-experiments
Situation in which particular circumstances generated
exogenous variation in the data.
Practical examples of experiments and quasi experiments.
4/28&30/2014 Econ 141, Spring 2014 2
-
Experiment setup
Estimate causal effect of treatment on
outcome (Jargon comes from medical sciences)
Potential outcome Outcome, , for an individual under potential treatment, . Causal effect is difference in outcome if treatment is
received and when it is not received.
Average treatment (causal) effect
Average causal effect across population of interest 4/28&30/2014 Econ 141, Spring 2014 3
-
Difference estimator
Regression model
= 0 + 1 + +
outcome
treatment
1 average treatment effect Parameter that we want to consistently estimate
control variables
effect of control variables not necessarily consistently estimated
residual of the regression 4/28&30/2014 Econ 141, Spring 2014 4
-
Randomized controlled experiment
Researcher controls whether an individual is subjected to treatment.
Treatment can be randomly assigned to participants in experiment.
Fully random or
Based on observables to better approximate population.
Treatment assigned to assure exogenous variation in .
4/28&30/2014 Econ 141, Spring 2014 5
-
Quasi controlled experiment
Whether an individual is subjected to treatment or not is not determined by the
researchers.
Particular circumstances resulted in random assignment of treatment to participants in
experiment.
(Part of) variation in treatment across individuals exogenous.
4/28&30/2014 Econ 141, Spring 2014 6
-
Five studies as examples
Experiment
Duration Dependence and Labor Market Conditions: Evidence from a Field Experiment
Kroft, Lange, and Notowidigdo (QJE, 2013)
Quasi-experiment
Household Expenditure and the Income Tax Rebates of 2001 Johnson, Parker, and Souleles (AER, 2006)
Using Electoral Cycles in Police Hiring to Estimate the Effect of Police on Crime
Levitt (AER, 1996), comment by McCrary (AER, 2002)
Minimum Wages and Employment: A Case Study of the Fast Food Industry in New Jersey and Pennsylvania
Card and Krueger (AER, 1994)
Did Securitization Lead to Lax Screening? Evidence from Subprime Loans.
Keys, Mukherjee, Seru, and Vig (QJE, 2010)
4/28&30/2014 Econ 141, Spring 2014 7
-
Focus on same points across papers
1. Research question
2. Regression specification
3. Potential threats to internal validation
4. Instrument/experiment to resolve threats
5. OLS and IV results
6. Hypothesis tests
7. Interpretation
4/28&30/2014 Econ 141, Spring 2014 8
-
Experiment
Duration Dependence and Labor Market Conditions: Evidence from a Field Experiment Kroft, Lange, and Notowidigdo (QJE, 2013)
4/28&30/2014 Econ 141, Spring 2014 9
-
Duration dependence of finding job
4/28&30/2014 Econ 141, Spring 2014 10
-
Causes of duration dependence
Sample selection / unobserved heterogeneity The best workers get hired first and contribute to high job-
finding rate of unemployed at short durations. The pool of
unemployed workers at longer durations just consist of ones
that are less likely to get hired.
Pure duration dependence The longer an individual worker remains unemployed the
less likely she or he is to find a job in subsequent months.
4/28&30/2014 Econ 141, Spring 2014 11
-
Research question
Sample selection / unobserved heterogeneity The best workers get hired first and contribute to high job-
finding rate of unemployed at short durations. The pool of
unemployed workers at longer durations just consist of ones
that are less likely to get hired.
Pure duration dependence The longer an individual worker remains unemployed the
less likely she or he is to find a job in subsequent months.
Do potential employers hold the time someone has been
out of a job against them and are they less likely to call
back those who have been unemployed longer for an
interview when they apply for a job?
4/28&30/2014 Econ 141, Spring 2014 12
-
Regression specification
Linear probability model
, = 0 + 1, + 2 log , + , + ,
individual, city (MSA) person lives in
, = 1 if gets called back for interview after applying for a job. Is 0 otherwise
, Dummy if employed at other job at time of application.
, Duration of unemployment if not employed at other job.
, Set of conditioning variables
4/28&30/2014 Econ 141, Spring 2014 13
-
Regression specification
Linear probability model
, = 0 + 1, + 2 log , + , + ,
individual, city (MSA) person lives in
, = 1 if gets called back for interview after applying for a job. Is 0 otherwise
, Dummy if employed at other job at time of application.
, Duration of unemployment if not employed at other job.
, Set of conditioning variables
4/28&30/2014 Econ 141, Spring 2014 14
Research question
boils down to asking
whether < or not.
-
Why not apply OLS?
Linear probability model
, = 0 + 1, + 2 log , + , + ,
individual, city (MSA) person lives in
, = 1 if gets called back for interview after applying for a job. Is 0 otherwise
, Dummy if employed at other job at time of application.
, Duration of unemployment if not employed at other job.
, Set of conditioning variables
4/28&30/2014 Econ 141, Spring 2014 15
Unobserved Heterogeneity:
Workers with undesirable unobserved
characteristics will be unemployed
longer and be less likely to be called
back for an interview when they apply
for a job.
cov , , , < 0
Threat to internal validation
-
Experiment is a solution
Linear probability model
, = 0 + 1, + 2 log , + , + ,
individual, city (MSA) person lives in
, = 1 if gets called back for interview after applying for a job. Is 0 otherwise
, Dummy if employed at other job at time of application.
, Duration of unemployment if not employed at other job.
, Set of conditioning variables
4/28&30/2014 Econ 141, Spring 2014 16
Send out a set of, otherwise identical,
resumes to apply for job openings where the
only difference is the treatments , and log .
Estimate whether there is a significant
difference in call back rates as a function of
the duration of unemployment, .
-
Description of treatments
4/28&30/2014 Econ 141, Spring 2014 17
-
Description of sample
4/28&30/2014 Econ 141, Spring 2014 18
Table describes the
composition of the
sample of 12,054
resumes that
researchers sent
out, the call back
rate they got, and
the types of jobs
they applied for.
-
Show treatment is randomly assigned
4/28&30/2014 Econ 141, Spring 2014 19
-
Experiment is a solution
Linear probability model
, = 0 + 1, + 2 log , + , + ,
individual, city (MSA) person lives in
, = 1 if gets called back for interview after applying for a job. Is 0 otherwise
, Dummy if employed at other job at time of application.
, Duration of unemployment if not employed at other job.
, Set of conditioning variables
4/28&30/2014 Econ 141, Spring 2014 20
Because this is an experiment we have
generated all variation in , and , as exogenously.
Result is that we can estimate equation using
OLS for the data we obtain from the
experiment.
-
Table with main result
4/28&30/2014 Econ 141, Spring 2014 21
-
Table with main result
4/28&30/2014 Econ 141, Spring 2014 22
-
Figure illustrates the point
4/28&30/2014 Econ 141, Spring 2014 23
-
Interpretation
Very convincing evidence that employers are less likely to call back persons who have
been unemployed for more than 6 moths for
an interview when they apply for a job.
Call back rate drops by about 3 percentage points.
4/28&30/2014 Econ 141, Spring 2014 24
-
Quasi-Experiment
Household Expenditure and the Income Tax Rebates of 2001
Johnson, Parker, and Souleles (AER, 2006)
4/28&30/2014 Econ 141, Spring 2014 25
-
Research question
Permanent Income Hypothesis (PIH) the hypothesis states that a change in permanent income, rather than a change in temporary income, affects the choices
that determine a consumer's consumption patterns. The key
conclusion of this theory is that transitory, temporary changes
in income have little effect on consumer spending behavior,
whereas permanent changes can have large effects on
consumer spending behavior.
4/28&30/2014 Econ 141, Spring 2014 26
-
Research question
Permanent Income Hypothesis (PIH) the hypothesis states that a change in permanent income, rather than a change in temporary income, affects the choices
that determine a consumer's consumption patterns. The key
conclusion of this theory is that transitory, temporary changes
in income have little effect on consumer spending behavior,
whereas permanent changes can have large effects on
consumer spending behavior.
Did consumers change their consumption level upon receipt of
the one-time 2001 federal income tax rebates?
4/28&30/2014 Econ 141, Spring 2014 27
-
Regression equation
,+1 , = 0,,
12
=1
+ 1 + 2,+1 + ,+1
, Household s consumption in month
, Month dummy for which month time falls in
Set of conditioning variables
,+1 Size of the one-time tax rebate a household r receives at time + 1.
4/28&30/2014 Econ 141, Spring 2014 28
-
Permanent Income Hypothesis
,+1 , = 0,,
12
=1
+ 1 + 2,+1 + ,+1
, Household s consumption in month
, Month dummy for which month time falls in
Set of conditioning variables
,+1 Size of the one-time tax rebate a household r receives at time + 1.
4/28&30/2014 Econ 141, Spring 2014 29
Permanent Income Hypothesis
For households that satisfy the PIH the one-time
tax rebate which affects their current income but
barely their permanent income it should be the
case that they do not change their consumption
in response to the tax rebate. That is, the PIH
implies the null hypothesis : = .
-
Why OLS might be inconsistent
,+1 , = 0,,
12
=1
+ 1 + 2,+1 + ,+1
, Household s consumption in month
, Month dummy for which month time falls in
Set of conditioning variables
,+1 Size of the one-time tax rebate a household r receives at time + 1.
4/28&30/2014 Econ 141, Spring 2014 30
Observed and unobserved characteristics that affect the
level of consumption, like the level of different types of
income, a households preference for leisure, etc., might not only affect the level of consumption (and the size of
its change) but also the size of the tax rebate a household
receives.
If that is the case then cov ,+1, ,+1 0
and OLS not consistent.
-
Timing of Rebate Instrumental Variable
Timing of month in which households received the tax rebate depended on their social security
number.
This choice of timing meant random timing of rebate across households.
Let ,+1 = 1 if household received tax
rebate in month + 1 and 0 otherwise.
,+1 is positively correlated with ,+1 (valid)
and random across households (exogenous).
4/28&30/2014 Econ 141, Spring 2014 31
-
Table with main results from OLS
4/28&30/2014 Econ 141, Spring 2014 32
-
Reduced form regression
4/28&30/2014 Econ 141, Spring 2014 33
Putting instrument
in as regressor in IV regression rather
than fitted values of
endogenous
variables is called reduced form
regression.
It always has a
better fit than the IV
regression
-
Table with main results from OLS
4/28&30/2014 Econ 141, Spring 2014 34
-
Note the footnote to this table
4/28&30/2014 Econ 141, Spring 2014 35
-
PIH rejected
4/28&30/2014 Econ 141, Spring 2014 36
-
OLS and 2SLS give similar results
4/28&30/2014 Econ 141, Spring 2014 37
Test whether OLS
estimates are valid
using Hausman test
Problem set
-
Additional results in paper
Estimate the timing of the spending response with respect to the rebate.
Show that it is mainly low income households and those with a low level of liquid assets
whose consumption responds most to the tax
rebates.
Replication files with STATA do file to generate the results are on bSpace.
Problem 3 on Problem set on Chapters 12 and 13.
4/28&30/2014 Econ 141, Spring 2014 38
-
Interpretation
Spending of a large fraction of U.S. households responds to temporary changes
in income.
Important for understanding
Permanent income hypothesis
Importance of liquidity constraints
Aggregate demand effect of temporary fiscal stimulus.
4/28&30/2014 Econ 141, Spring 2014 39
-
Quasi-Experiment
Using Electoral Cycles in Police Hiring to Estimate the Effect of Police
on Crime
Levitt (AER, 1996), comment by McCrary (AER, 2002)
4/28&30/2014 Econ 141, Spring 2014 40
-
Crime rates peaked in early 90s
4/28&30/2014 Econ 141, Spring 2014 41
-
Research question
Does putting more cops on the street reduce
the crime rate in a city?
4/28&30/2014 Econ 141, Spring 2014 42
-
Regression equation
4/28&30/2014 Econ 141, Spring 2014 43
-
Why OLS might be a problem
4/28&30/2014 Econ 141, Spring 2014 44
Cities are likely to increase the size of their police force
as a direct response to an increase in crime.
So cov ln , > .
OLS inconsistent and likely to understate the degree to
which increasing the police force reduces crime.
-
Election cycles also drive changes
in police force Timing of Mayoral and Gubernatorial
elections is determined by law and not
affected by changes in crime rates.
Use election-driven changes in size of police forces as exogenous variation in ln to identify causal effect of size of police force on
crime.
4/28&30/2014 Econ 141, Spring 2014 45
-
1st stage regression to show relevance
4/28&30/2014 Econ 141, Spring 2014 46
First-stage regression
Regress endogenous variable on instruments
and all the explanatory variables that are also
in the second-stage regression.
-
1st stage regression to show relevance
4/28&30/2014 Econ 141, Spring 2014 47
-
1st stage regression to show relevance
4/28&30/2014 Econ 141, Spring 2014 48
-
1st stage regression to show relevance
4/28&30/2014 Econ 141, Spring 2014 49
-
Columns (4) and (5) reduced forms
4/28&30/2014 Econ 141, Spring 2014 50
-
Columns (4) and (5) reduced forms
4/28&30/2014 Econ 141, Spring 2014 51
-
2nd stage regression:
police reduces crime
4/28&30/2014 Econ 141, Spring 2014 52
-
2nd stage regression:
police reduces crime
4/28&30/2014 Econ 141, Spring 2014 53
-
Levitt did weighted least squares
4/28&30/2014 Econ 141, Spring 2014 54
-
But he reversed the weights
4/28&30/2014 Econ 141, Spring 2014 55
McCrary (2002) points out that Levitt made a programming error and gave unreliable
observations more weight rather than less.
Main results not robust to correction of this error.
-
McCrarys correction table
4/28&30/2014 Econ 141, Spring 2014 56
-
Heteroskedasticity matters!
4/28&30/2014 Econ 141, Spring 2014 57
-
Quasi-experiment
Minimum Wages and Employment: A Case Study of the Fast Food Industry in New Jersey and
Pennsylvania
Card and Krueger (AER, 1994)
4/28&30/2014 Econ 141, Spring 2014 58
-
Minimum wage debate continues
WASHINGTONPresident Barack Obama's proposal Tuesday to raise the federal minimum wage is likely to
rekindle debates over whether the measure helps or hurts
low-income workers.
White House officials say the move to boost the wage to
$9 an hour, from $7.25, is aimed at addressing poverty
and helping low-income Americans.
But the proposal likely will be opposed by Republicans
and business groups, which have traditionally said raising
the minimum wage discourages companies from hiring
low-skilled workers.
Wall Street Journal
4/28&30/2014 Econ 141, Spring 2014 59
-
Research question
How does raising the minimum wage affect
labor demand (employment levels)?
Quasi experiment On April 1, 1992, New Jersey's minimum wage rose from $4.25 to $5.05 per hour. To evaluate the impact of the law we surveyed 410fast-food
restaurants in New Jersey and eastern Pennsylvania before and after the
rise. Comparison of employment growth at stores in New Jersey and
Pennsylvania (where the minimum wage was constant) provide simple
estimates of the effect of the higher minimum wage.
4/28&30/2014 Econ 141, Spring 2014 60
-
Regression equation
4/28&30/2014 Econ 141, Spring 2014 61
-
Regression equation
4/28&30/2014 Econ 141, Spring 2014 62
-
Sample of fast food restaurants
4/28&30/2014 Econ 141, Spring 2014 63
-
Fast food restaurants in study
4/28&30/2014 Econ 141, Spring 2014 64
-
NJ & PA restaurants similar
4/28&30/2014 Econ 141, Spring 2014 65
-
NJ & PA restaurants similar
4/28&30/2014 Econ 141, Spring 2014 66
-
Pre-treatment wages similar
4/28&30/2014 Econ 141, Spring 2014 67
-
Post-treatment wages different
4/28&30/2014 Econ 141, Spring 2014 68
-
Raising minimum wage
increased employment?
4/28&30/2014 Econ 141, Spring 2014 69
-
Puzzling result questioned
Internal validity:
Did NJ restaurants already adjust their hiring before the first wave of survey?
Was there another NJ vs PA state-specific shock/policy that affected relative demand for fast
food?
Were restaurants sampled in both states really similar?
External validity:
How does NJ/PA fast food to U.S. labor market?
4/28&30/2014 Econ 141, Spring 2014 70
-
Quasi-experiment
Did Securitization Lead to Lax Screening? Evidence from Subprime Loans.
Keys, Mukherjee, Seru, and Vig (QJE, 2010)
4/28&30/2014 Econ 141, Spring 2014 71
-
Mortgage crisis
2008 financial crisis was largely due to default
(delinquency) rates on mortgages that were
part of mortgage backed securities increasing.
Research question:
Did financial institutions lower their standards
on the mortgages they issued when they knew
they could offload them into mortgage backed
securities?
4/28&30/2014 Econ 141, Spring 2014 72
-
Regression discontinuity
Discontinuity is treatment:
Loans with a credit score above 620 were
much easier securitized and offloaded.
Research question
Was there more scrutiny on loans with a score
just below 620 than on loans just above?
Regression discontinuity analysis
4/28&30/2014 Econ 141, Spring 2014 73
-
Regression discontinuity
Discontinuity is treatment:
Loans with a credit score above 620 were
much easier securitized and offloaded.
Research question
Was there more scrutiny on loans with a score
just below 620 than on loans just above?
Regression discontinuity analysis
4/28&30/2014 Econ 141, Spring 2014 74
Degree of scrutiny determined by amount of
background documentation on loan applicant
and on loan
Low documentation
vs
Full documentation
-
4/28&30/2014 Econ 141, Spring 2014 75
-
Regression discontinuity equation
4/28&30/2014 Econ 141, Spring 2014 76
-
Picture says more than regression
4/28&30/2014 Econ 141, Spring 2014 77
-
Regression confirms picture
4/28&30/2014 Econ 141, Spring 2014 78
-
Interpretation
Size of discontinuity suggests that financial firms were less diligent with loans they could
more easily offload and securitize.
For loans they were more likely to hold on their own book they required more
documentation.
4/28&30/2014 Econ 141, Spring 2014 79
-
Where to go after this class
This class taught main statistical theory behind
regression analysis. Logical topics beyond what
was covered are
Alternative estimation methods Method of Moments, Generalized Method of Moments, Maximum Likelihood, Bayesian
Estimation.
Time series analysis Observations correlated over time
Extended panel data techniques
Find applications Labor Economics, Industrial Organization, Macroeconomics, Development Economics, ect.
4/28&30/2014 Econ 141, Spring 2014 80