economic evaluation of public policy and programs course overview econ 4140 fall 2010
TRANSCRIPT
Page 2
• This course will apply economic theory and statistics to the evaluation of economic policy and programs.
• Economic policy/programs include– Spending on public infrastructure (roads, bridges, hospitals,
etc.)– Spending on programs (environmental protection, training
unemployed, vaccinations for H1NI, )– Regulations– Social marketing (health warnings on cigarettes,
temperature set-backs on thermostats
• The analytical structure comprises what is known as cost-benefit analysis.
Objectives of the course
Page 3
Course components
1. Cost-benefit/cost-effectiveness analysis
2. Measuring and valuing program impacts
3. The political economy of cost-benefit analysis
Page 4
• Overview of economic programs and policy
• Time value of money• Risk and uncertainty• Option and existence values
Part 1: Theoretical foundations of cost benefit analysis and cost effectiveness analysis
Page 5
– Experimental and quasi experimental design
– Net impact evaluation– Review of net impact evaluations– Key evaluation methods (qualitative and
quantitative)– Performance measurement and monitoring
Part 2: Measuring and valuing impacts
Page 7
Discussion questions
Imagine you had been asked to provide advice to the government on the following policies.
1. First, what is your opinion? Why?
2. Now how you would frame the problem and what evidence is needed to support your conclusion?
3. What are arguments for an opposite policy? What evidence would support that view?
4. What issues of fairness exist, if any?
Page 8
Policy proposals• Making university free
– requiring repayment of costs through an income tax surcharge
– Requiring no repayment• Offering to pay everyone in the workforce a $50
subsidy to take a flu shot every year.• Increasing the price of gasoline by 1¢/liter each
month for the next 2 years and:– Placing the proceeds in general revenue or– Devoting the proceeds to energy conservation and
pollution control technologies• Subsidizing those who purchase fuel efficient
vehicles
Page 10
… divide half a sheet of paper by a line into two columns; writing over one Pro and over the other Con. Then during the three of four days consideration I put down under the different heads, hints of the different motives that at different times occurred to me, for and against the measure. When I have thus got them all together, I endeavour to estimate the respective weights; and where I find two or three equal, I strike them both. If I find a reasons pro equals two reasons con, I strike out the three. If I find two reasons con equal to some three reasons pro, I strike out the five; and thus proceeding I find at length where the balance lies; and if after a day or two or farther consideration, nothing new that is of importance occurs on either side, I come to a determination accordingly. And although the weight or reasons cannot be take with the precision of algebraic quantities, and yet when each is thus considered, separately and comparatively, and the whole lies before me, I think I am able to judge better and am less liable to make a rash step.
Benjamin Franklin, 1772
Page 11
CEA and CBA: a quick tour
CEA CBA
Scope Local – single output/outcome
Global – multiple outcomes valued
Unit of measure for outcome
Natural Money
Time frame Better in the Immediate/short-term
Extended over many years is common.
Main Decision Purpose
Retrospective Prospective (ex-ante)
Value for money (ex-post)
Application Activities-outputs-outcomes (more on this later)
Outcomes or impacts only
Reference At least one program alternative
No reference needed since when benefits exceed costs, the project can rationally proceed.
Page 12
Types of CBA
• Ex-ante CBA attempts to project costs and benefits forward to support decision-making…it is most useful for planning and ranking alternative spending
• Ex-post CBA provides insights after the project and may determine value for money, servng as a guide for future policy.
Page 13
Example 1 :Vaccination programs for employees
• Three vaccine programs are being evaluated for cost effectiveness.
• Outcomes include:– Sick days – Total number of employees affected
• The program with the lowest number of sick days per dollar cost is the most cost effective in terms of outcomes.
• The key assumption is that the three programs are essentially similar. No program has a markedly different profile in terms of adverse impact; the only difference is efficacy.
Page 14
Example 2: CEA Training interventions
A common goal for many training interventions is the return to work.
CEA compares the costs of reaching the same outcome by alternative methods.
Typical examples of outcomes include:• Return to work for six months• Hours of work after the intervention• Number of trainees who become employed• Wages after training• Post intervention Employment Insurance benefit
Page 15
A fast tour of CBA
• CBA compares investments – especially those in the public domain - using economic concepts of social welfare.
• CBA is rooted in economic theory.
• The welfare of stakeholders emerges from the comparisons of utility for individual stakeholders as revealed by their preferences and valuations.
• Stakeholder choices in the market reveal their preferences and become the guide for how to increase (maximize) their welfare.
Page 16
Key assumptions of CBA
1. Society is composed of individuals who are free to make choices (such as how much they work), and social welfare is the simple sum of individual welfare.
2. People are the best judges of their welfare (consumer/producer sovereignty).
3. Changes in individual welfare are measurable by the willingness to pay for something.
Page 17
The main features of cost-benefit analysis
• Cost-benefit analysis includes all social benefits and costs.
• It sums the values of an investment across all users• It must translate change in welfare into a dollar
amount. This can include benefits such as:– Avoided harms (deaths from disease)– Lost wages due to inability to work– Value of time saved.
• This creates an inevitable value (political) perspective for CBA.
Page 18
CBA uses money to measure everything
The main advantage of using money to value outcome is that all outcomes may be included. This is a major advantage over CEA Many non-economists are uncomfortable with the main steps in CBA:
• Translating all benefits and costs into money.• Comparing the changes in welfare among different groups.• Decisions favour majority rule and those with the greatest
willingness to pay, regardless of personal circumstances.• Participants in the economy appear to count more than those who
do not (the benefits for those earning a wage may appear to count more than the benefits for those who are unemployed, such as children, retired persons, students, etc.)
Page 19
Example 1 – Traffic congestion
Overpass: A new suburban subdivision is created beyond a main east/west transcontinental rail line. With 2,000 new households, new retail malls, and a main road linking north and south Winnipeg, traffic delays caused by rail traffic cause substantial delays.
Option 1: Create an overpass at a cost of $30 millionOption 2: Impose restrictions on rail traffic
CBA compares the ratio of benefits to the costs for each option, as well as the “hidden” option of doing nothing.
Page 20
Translating benefits into $
• Because CBA needs to express all benefits and costs as a dollar value, simplifying assumptions must be used.
• These assumptions can dramatically alter a CBA.
• Careful peer review requires a team approach to create a valid CBA.
Page 21
Example 2 – Vaccination programs
Benefit• Reduced short-term cost due to
illness• Reduced long-term cost for
caring for the small number of catastrophic incidents
• Averted loss of incomes for those who are disabled/dying
• Averted costs of lost time at work and play.
Cost• Vaccination program• Economic loss for the small
number who experience adverse reactions to vaccine.
Page 23
Steps in a CBA
Enumerate costs and benefits
Value the costs and benefits
Comparing costs and benefits
Page 25
Taking Charge! – A training program for single parents on social assistanceTaking Charge! was a pilot program jointly funded by HRDC and the Manitoba government. It focused on offering a range of supports for single parents on income assistance.Key features included:
• High level of support (daycare, counseling, basic education, volunteer experience, job placement, etc.)
• The program recruited IA clients, performed employability assessments, developed tailored training programs, contracted with service providers to delivery the training, and supported job placement.
http://www.hrsdc.gc.ca/en/cs/sp/hrsdc/edd/reports/1999-000448/sp-ah109-e.pdf
Page 26
Cost-benefit model
Basic for comparison:• “Control” group (Income assistance –
welfare clients who have never taken a training program)
• “Comparison” group (Income assistance participants in four other training interventions)
• “Program” group (Taking Charge!)
Page 27
Nature of the CBA
The outcome of interest is whether TC! participants return to work faster than those on Income Assistance and whether a TC! “grad” has a “better” employment outcome.
Better is defined as– Higher wages– Longer employment stints– Fewer stints on income assistance
We addressed this question using regression analysis and survival analysis.
Page 29
Costs summary for Taking Charge! Overall costs and activity for the main interventions (Fiscal 1997/98)
TC!
94/95 - 97/98
Employment Connections
97/98
OFE 96/97
Clients served 3237 937 569 Total program cost $10,053,735 $1,256,200 $156,000 Average cost per client $3,106 $1,341 $274 Special needs payments $6 $12 $33 Average cost per client served $3,112 $1,353 $307 % clients employed** 0.45 0.55 0.07 Clients employed 1456 515 39 Average cost per client employed $6,905 $2,439 $4,000 * For TC!, this is clients served from program inception to March 31, 1998. For Employment Connections, this is clients served in 1997/98 and for OFE clients served in 1996/98. **After nine months of intervention termination, based on the follow-up survey. Sources: TC! annual reports, Employment Connections annual reports, OFE annual reports.
Page 30
Summary of post-intervention experience
Percent remaining on assistance
Group 3 months 6 months 9 months
Average time to come off
assistance (months)
Treatment (TC!) 79% 71% 65% 20 Employment Connections 76% 63% 55% 14
Comparison* 79% 67% 58% 16 Control 87% 82% 74% 23 * Includes EC, OFE, Community Partners and Pathways. The average time to come off assistance just for the three groups net of Employment Connections is 17.5 months.
Page 31
The benefit-cost framework
Costs Benefits A. Income assistance
payments during training E. Increased benefit reductions
because of earned income + B. Training allowances
(books, special needs) + F. Reduced income assistance
due to lower time on welfare + C. Cost of training + G. Increased taxes from
employment earnings = D. Total Costs = H. Total Benefits
Page 32
Average training time (months) and average income assistance payments (Item A in Table 76)
Adjusted average
duration of training
(months)
Average monthly IA
during training
Total IA received during training
(Adjusted duration of
training multiplied by IA during training)
Average monthly IA
after training
Group
a b c a b c d Treatment (TC!)
4.4 3.7 0.7 $904 $3,978 $3,345 $633 $682
Comparison 2.7 2.4 0.3 $891 $2,406 $2,138 $267 $650 Control* 0.0 0.0 0.0 $897 $0 $0 $0 $693 * To support the statistical analysis, we defined the intervention for the Controls as 1 month, but in reality the intervention period for these IA participants is 0.