The MCC Incentive EffectQuantifying Incentives for Policy Change in

an Ex-post Rewards System

Prepared for the Millennium Challenge Corporation by :

Ingrid Aune

Yanyan Chen

Christina Miller

Joshua Williams

-- Spring 2013 --

The Team and the Outline

Ingrid AuneYanyan ChenChristina Miller Joshua Williams

1. The MCC Incentive Effect and Our Data

2. Preliminary Analysis

3. Models and Analysis

4. Conclusion and Recommendations

Previous Quantitative Studies

• Johnson and Zajonc (2006)• Öhler, Nunnenkamp and Dreher (2010)

Previous Studies

• Data: Third-party sources• Indicators used• Quasi-experiment

Our Study

Data

• Economic Freedom• Investing in People• Ruling Justly

Indicator Categories

• Qualitative MCC Incentive Effect• Close to Passing or Failing the Indicator Threshold• The Control of Corruption Standard• Threshold Partnership• Middle Range of Government Expenditures

Treatment Groups

Indicator Mobility

Top 10 Rank Change

Source: Authors’ Calculations

Credit Depth

Legal Rights

Cost to Start Business

Time to Start Business

Fiscal Policy

Inflation

Time to Register Property

Cost to Register Property

Regulatory Quality

Trade Freedom

Girls Primary Education

Health Expenditure

Immunization Rate

Education Expenditures

Civil Liberties

Corruption

Government Effectiveness

Political Rights

Rule of Law

Voice and Accountability

Top 10 Rank Change

Investing in People

Economic Freedom

Ruling Justly

Source: Authors’ Calculations Top 10 Rank Change

Correlation between Indicators

Size is proportional to Correlation

Cost to Start

Business

0 0 0.095 0.61 0

0.094

Time to Start

Business

0.15 0 0 0.15

0 0

Fiscal Policy

0 0 0.1

0 0 0 Inflation 0.14 0.11

0.46 0.43 0 0

Time to Register Property

0.25

0.43 0.22 0.12 0.16 0.4 Regulation Critical Value = 0.3

Correlation between Indicators (continued)

Size is proportional to Correlation

Civil Liberty 0.19 0.12 0.65 0.14 0.33

0.45 Corruption 0.29 0.13 0.21 0.24

0.35 0.68 Government Effectiveness

0.13 0.38 0.1

0.93 0.38 0.29 Political Rights

0.01 0.28

0.53 0.8 0.73 0.41 Rule of Law 0.26

0.01 0.01 0.11 0.01 0.14Voice and

Accountability Critical Value = 0.3

Plausible Conjectures of Treatment Groups

Treatment Covariates SdClose to Passing or Failing -0.09 (0.14)Control of Corruption Standard 0.49*** (0.14)Threshold Partnership 0.72*** (0.16)Countries in the 2nd or 3rd Quartile of Government Expenditure 0.54*** (0.14)Standard errors in parentheses，*** p<0.01, ** p<0.05, * p<0.1

MCC Quanlitative Evidence

Preliminary Analysis: Locally weighted scatter plot smooth (LOWESS)

Positive MCC Incentive Effect

Treatment: Close to Passing or Failing

-0.10

-0.05

0.00

0.05

0.10

2002~2004 2004~2006 2006~2008 2008~2010

ControlTreatmentControlTreatment

Treatment: Close to Passing or Failing

Treatment: Qualitative MCC Incentive Effect

-0.1

0.0

0.1

2002~2004 2004~2006 2006~2008 2008~2010

ControlTreatmentControlTreatment

Neutral/Mixed MCC Incentive Effect

Preliminary Analysis: LOWESS (continued)

Treatment: Qualitative MCC Incentive Effect

Treatment: Middle Range Government Expenditures

-0.10

-0.05

0.00

0.05

0.10

2002~2004 2004~2006 2006~2008 2008~2010

ControlTreatmentControlTreatment

NegativeMCC Incentive Effect

Preliminary Analysis: LOWESS (continued)

Treatment: Middle Range Government Expenditures

Difference-in-Differences

Differences

• Difference in scores between two years• Difference between two time periods• Difference between control and treatment

Models

• Rate of Reform• Likelihood of Improvement• Proportion of Indicators Improved

Overview of General Model Results

All Results: 38 (+) and 31 (-)

Model 1 Rate of Reform

Model 2 Likelihood to Improve

Model 3 Proportion of

Indicators Improved

Treatment

Economic Freedom

Investing in People

Ruling Justly Economic Freedom

Investing in People

Ruling Justly Economic Freedom

Investing in People

Ruling Justly

+ - + - + - + - + - + - + - + - + -

Qualitative MCC Incentive Effect 2 1 1 1 2 3 2 0 3 0 2 3 0 0 0 0 0 1

Close to Passing or Failing the Indicator Threshold 1 0 0 0 2 0 0 0 0 0 3 0 0 0 0 0 0 0

The Control of Corruption Standard 2 3 0 0 2 0 1 2 0 0 0 0 0 0 0 0 0 0

Threshold Partnership 0 2 1 3 2 1 0 0 0 2 2 1 0 1 0 1 0 0

Government Expenditure 3 2 2 1 0 2 1 0 3 0 0 1 0 0 1 0 0 0

Totals of Positive and Negative Results

8 8 4 5 8 6 4 2 6 2 7 5 0 1 1 1 0 1

All results: 38+ / 31-

Select Statistically Significant Results:

Treatment Model IndicatorTime Period

2004-2006 2006-2008 2008-2010

Qualitative MCC Incentive Effect

Rate of Reform

Cost to Start a Business 53.3**(26.94)

Cost to Register Property 1.205**(0.601)

Trade Freedom -5.690*(3.188)

Likelihood of Improvement Regulatory Quality 0.429***

(0.146)0.456***(0.144)

Threshold Partnership Rate of Reform

Time to Register Property -24.41*(14.45)

Depth of Credit Information -0.657**(0.330)

Economic Freedom

Treatment Model IndicatorTime Period

2004-2006 2006-2008 2008-2010

Qualitative MCC Incentive Effect

Rate of Reform

Girls' Primary Education Completion

7.215***(2.762)

Immunization Rate -4.504**(2.224)

Likelihood of Improvement

Girls' Primary Education Completion

0.353**(0.166)

0.487***(0.164)

0.366**(0.178)

Threshold Partnership

Rate of Reform

Primary Education Expenditures 0.911**

(0.373)

Health Expenditures -4.678**(2.127)

-5.614***(2.134)

Immunization Rate -5.453**(2.540)

Likelihood of Improvement Health Expenditures -0.459***

(0.165) -0.435**(0.166)

Investing in People

Treatment Model Indicator Time Period2004-2006 2006-2008 2008-2010

Qualitative MCC Incentive Effect

Rate of Reform

Civil Liberties 1.566*(0.878)

Political Rights 2.398**(1.146)

Voice and Accountability -0.117**(0.0575)

-0.213***(0.0579)

-0.181***(0.0579)

Likelihood of Improvement

Political Rights 0.366**(0.139)

Control of Corruption 0.278*(0.148)

Voice and Accountability -0.295**(0.145)

-0.442***(0.142)

-0.324**(0.145)

Threshold Partnership

Rate of Reform

Civil Liberties 2.307**(1.006)

Political Rights 2.443*(1.335)

Rule of Law -0.124**(0.0611)

Likelihood of Improvement

Control of Corruption 0.427**(0.170)

Political Rights 0.284*(0.168)

Rule of Law -0.357**(0.177)

Ruling Justly

Number of Results in Economic Freedom (Breakdown)

Number of Results in Investing in People

Number of Results in Ruling Justly

Number of Positive and Negative ResultsDifferent Time Periods

Economic Freedom Investing in People Ruling Justly

Conclusions and Recommendations

Lack of quantitative evidence for the MCC incentive effect

Newer and stronger documentation of the effects in the data

A new road-map:

• Improved treatment groups• Insight on indicator categories• Separate study of lower-income and lower-middle-income countries

Recommendation to the MCC

Create Aspirations

Provide Knowledge and Information• Publish the ranking each country has for each indicator and overall.

• Cut the number of indicators, particularly in the Ruling Justly category.

• Increasing transparency by setting three or five year plans for indicators. Separate Countries by Income Groups

Questions

Acknowledgements

We would like to thank the following people for help on this presentation:

• Professor Melanie Manion, University of Wisconsin-Madison• Andria Hayes-Birchler and the people at the Millennium Challenge

Corporation• Andrew Lambert and the staff at the La Follette School of Public Affairs• Daisy Bui Chung and the students at the La Follette School of Public

Affairs