economic evaluation apa oh v. 2(2)
DESCRIPTION
Slides from session given by Peter Mallow and Della Rucker on methods for evaluating economic impacts of policy choices at Ohio American Planning Association Conference, October 2011.TRANSCRIPT
![Page 1: Economic Evaluation Apa Oh V. 2(2)](https://reader036.vdocuments.net/reader036/viewer/2022062320/5590ca461a28ab4d538b4681/html5/thumbnails/1.jpg)
A more useful tool: Using economic evaluation to make
better decisions
APA Ohio State Planning ConferenceOctober 20 & 21, 2011
Dayton, Ohio
![Page 2: Economic Evaluation Apa Oh V. 2(2)](https://reader036.vdocuments.net/reader036/viewer/2022062320/5590ca461a28ab4d538b4681/html5/thumbnails/2.jpg)
Agenda
• Introductions
• Definitions and goals of economic evaluations• Economic evaluation vs. economic impact• How to do an economic evaluation• Discussion
![Page 3: Economic Evaluation Apa Oh V. 2(2)](https://reader036.vdocuments.net/reader036/viewer/2022062320/5590ca461a28ab4d538b4681/html5/thumbnails/3.jpg)
3
What are the Goals of Economic Evaluations?
• Because all resources are scarce, economic evaluations help us…..– Identify which initiatives will offer the greatest
value• “Value” means providing the most benefit per
unit of cost• This is the so-called “efficiency criterion.”
Guarantees that the greatest aggregate benefit will be provided for that cost.
– But efficiency has little to say about equity; that is, about how the total benefits get distributed
![Page 4: Economic Evaluation Apa Oh V. 2(2)](https://reader036.vdocuments.net/reader036/viewer/2022062320/5590ca461a28ab4d538b4681/html5/thumbnails/4.jpg)
4
Why do we need to do economic evaluations of our planning and policy choices?
• We know different choices have different costs and impacts, but we don’t know what those are automatically. – As a result, we can either figure it out or
“guesstimate”– Too often, we guesstimate – and get it
wrong.
• We can use economic evaluation methods as a tool kit to make our real-world decision making better.
![Page 5: Economic Evaluation Apa Oh V. 2(2)](https://reader036.vdocuments.net/reader036/viewer/2022062320/5590ca461a28ab4d538b4681/html5/thumbnails/5.jpg)
5
How does this differ from an economic impact study?
• Economic IMPACT: essentially a political tool– Problem of counting indirect
impacts: who’s right?– Not always balanced against cost– Most useful for persuasion– Losing effectiveness due to overuse?
![Page 6: Economic Evaluation Apa Oh V. 2(2)](https://reader036.vdocuments.net/reader036/viewer/2022062320/5590ca461a28ab4d538b4681/html5/thumbnails/6.jpg)
6
Therefore….
• Use an economic impact study if are trying to persuade people to do something.
• Use an economic evaluation if you are trying to find answers and make decisions.
![Page 7: Economic Evaluation Apa Oh V. 2(2)](https://reader036.vdocuments.net/reader036/viewer/2022062320/5590ca461a28ab4d538b4681/html5/thumbnails/7.jpg)
Types of economic evaluation
![Page 8: Economic Evaluation Apa Oh V. 2(2)](https://reader036.vdocuments.net/reader036/viewer/2022062320/5590ca461a28ab4d538b4681/html5/thumbnails/8.jpg)
8
The Groundwork for Economic Evaluations
• Burden of Cost – What impact will changing a policy have on people?– Cost may be economic, social, or psychological cost to
individual, organization, or community.
– Example: changing how a city service is delivered.– Who will be directly impacted?– Who will be impacted as a result of the first set of
impacts?– What will it require for the impacted people/orgs to make
up the difference, or avoid a loss of quality?
![Page 9: Economic Evaluation Apa Oh V. 2(2)](https://reader036.vdocuments.net/reader036/viewer/2022062320/5590ca461a28ab4d538b4681/html5/thumbnails/9.jpg)
9
What Makes a Decision Hard?
• Uncertainty – Do we know what’s really
going on?– Do we know what the best
approach will be?– Do we know if this has
worked in the past?– What happens if we get it
wrong?
• Urgency
– How soon will it happen?– How soon do we have to
deal with it?
• Conflicting objectives
– Cars vs. pedestrians?– Commercial or residential?– Quality of service vs. range
of access?
![Page 10: Economic Evaluation Apa Oh V. 2(2)](https://reader036.vdocuments.net/reader036/viewer/2022062320/5590ca461a28ab4d538b4681/html5/thumbnails/10.jpg)
10
Cost-Effectiveness Analysis
• Compares the relative costs and outcomes (effects) of two or more courses of action– Costs are measured in monetary units and outcomes are
measured in effectiveness, such as deaths averted , park use increased or VMT improved.
– Typically the CEA is expressed in terms of a ratio where the denominator (bottom) is the impact from the potential action, and the numerator (top) is the cost.
_COST_OUTCOME
COST:OUTCOME
![Page 11: Economic Evaluation Apa Oh V. 2(2)](https://reader036.vdocuments.net/reader036/viewer/2022062320/5590ca461a28ab4d538b4681/html5/thumbnails/11.jpg)
11
Cost-Effectiveness Analysis Example
Building a new park
Cost: expenditure required to build the park according to design.
Benefit factors: number of people expected to use the park, impact on health of users (health costs, weight loss, health care costs).
You’ll note that this is the tricky part….can be done, but make assumptions clear.
![Page 12: Economic Evaluation Apa Oh V. 2(2)](https://reader036.vdocuments.net/reader036/viewer/2022062320/5590ca461a28ab4d538b4681/html5/thumbnails/12.jpg)
12
Cost-Benefit Analysis
• Cost-benefit analysis (CBA) differs from CEA and CUA in that benefits are measured in monetary units, just like costs
• This feature allows one to compare widely different types of interventions using CBA, since all benefits and costs are expressed in monetary units.
• But with public policy issues, converting outcomes to cash isn’t always a good idea.
![Page 13: Economic Evaluation Apa Oh V. 2(2)](https://reader036.vdocuments.net/reader036/viewer/2022062320/5590ca461a28ab4d538b4681/html5/thumbnails/13.jpg)
13
Conceptual Underpinnings of CEA
• Unlimited wants
• Finite resources
• Inevitability of choices
• Balance of benefits and costs
• Need for formal analysis
![Page 14: Economic Evaluation Apa Oh V. 2(2)](https://reader036.vdocuments.net/reader036/viewer/2022062320/5590ca461a28ab4d538b4681/html5/thumbnails/14.jpg)
14
Solving the Resource Allocation Problem
• The approach: constrained optimization• Two ways of framing the problem:
What allocation of resources will achieve the greatest possible gains without exceeding the volume of available resources?
What allocation of resources will minimize our expenditures, subject to the requirement that we achieve at least some target level of benefit?
![Page 15: Economic Evaluation Apa Oh V. 2(2)](https://reader036.vdocuments.net/reader036/viewer/2022062320/5590ca461a28ab4d538b4681/html5/thumbnails/15.jpg)
15
Finding the Cost-Effective Solution
• Identify mutually-exclusive interventions– Eliminate strongly and weakly dominated interventions
– Select the most effective program with a CE ratio less than or equal to some threshold value
threshold value = decision maker’s willingness to pay for a unit of benefit
More costly, less effective than some competing alternative
![Page 16: Economic Evaluation Apa Oh V. 2(2)](https://reader036.vdocuments.net/reader036/viewer/2022062320/5590ca461a28ab4d538b4681/html5/thumbnails/16.jpg)
+
+
-
Calculateincremental
CE ratio
Calculateincremental
CE ratio
Dominant
Dominated
-
Incremental Cost-Effectiveness Ratios
Incr
emen
tal
Effec
tiven
ess
Incremental Cost
16
![Page 17: Economic Evaluation Apa Oh V. 2(2)](https://reader036.vdocuments.net/reader036/viewer/2022062320/5590ca461a28ab4d538b4681/html5/thumbnails/17.jpg)
17
Incremental CE Ratios
• The incremental CE ratio comparing P2 to P1:
Cost of P2 – Cost of P1
Effect of P2 –Effect of P1
This tells you how much more you would have to spend for each additional unit of benefit you would gain if you choose a more costly, more effective
program rather than a less costly, less effective program
![Page 18: Economic Evaluation Apa Oh V. 2(2)](https://reader036.vdocuments.net/reader036/viewer/2022062320/5590ca461a28ab4d538b4681/html5/thumbnails/18.jpg)
18
Using an Incremental CE Ratio to Guide Choice
Incremental Effect
Incr
emen
tal C
ost
slope = l
“BAD”[CE ratio >l]
“GOOD”[CE ratio <l]
![Page 19: Economic Evaluation Apa Oh V. 2(2)](https://reader036.vdocuments.net/reader036/viewer/2022062320/5590ca461a28ab4d538b4681/html5/thumbnails/19.jpg)
19
Importance of Incremental Analysis
Tx Cost Effect
A $100,000 10 LM
B $110,000 10.001 LM
![Page 20: Economic Evaluation Apa Oh V. 2(2)](https://reader036.vdocuments.net/reader036/viewer/2022062320/5590ca461a28ab4d538b4681/html5/thumbnails/20.jpg)
20
Importance of Incremental Analysis
Tx Cost Effect CE
A $100,000 10 LM $10,000/LM
B $110,000 10.001 LM $10,999/LM
![Page 21: Economic Evaluation Apa Oh V. 2(2)](https://reader036.vdocuments.net/reader036/viewer/2022062320/5590ca461a28ab4d538b4681/html5/thumbnails/21.jpg)
Importance of Incremental Analysis
Tx Cost Effect CE Incremental CE
A $100,000 10 LM $10,000/LM —
B $110,000 10.001 LM $10,999/LM $10,000,000/LM
![Page 22: Economic Evaluation Apa Oh V. 2(2)](https://reader036.vdocuments.net/reader036/viewer/2022062320/5590ca461a28ab4d538b4681/html5/thumbnails/22.jpg)
Importance of Incremental Analysis
Tx Cost Effect CE Incremental CE
A $100,000 10 LM $10,000/LM —
B $110,000 10.001 LM $10,999/LM $10,000,000/LM
Right way to
calculate CE ratios!
Wrong way to
calculate CE ratios!
![Page 23: Economic Evaluation Apa Oh V. 2(2)](https://reader036.vdocuments.net/reader036/viewer/2022062320/5590ca461a28ab4d538b4681/html5/thumbnails/23.jpg)
23
Incremental Analysis With Multiple Options
• Example: Image three different street segments, and three different paving options:– Option 1: Pot Hole Repair– Option 2: Walnut Street, chip and seal– Option 3: Walnut Street, grind and repave– Option 4: Main Street, chip and seal– Option 5: Main Street, grind and repave– Option 6: George Parkway, chip and seal – Option 7: George Parkway, grind and repave– Option 8: Vine Street, chip and seal– Option 9: Vine Street, grind and repave
• Since each road carries a different volume of traffic, and since each paving option has a different lifespan, each has different costs and different length of benefit.
![Page 24: Economic Evaluation Apa Oh V. 2(2)](https://reader036.vdocuments.net/reader036/viewer/2022062320/5590ca461a28ab4d538b4681/html5/thumbnails/24.jpg)
24
Costs and Effects
Strategy Cost ($) Life span/traffic volume ratio
1 40,288 39.08
2 44,786 42.56
3 45,944 43.04
4 47,046 43.01
5 48,596 43.60
6 54,628 43.20
7 56,812 43.83
8 58,082 43.80
9 61,119 44.62
![Page 25: Economic Evaluation Apa Oh V. 2(2)](https://reader036.vdocuments.net/reader036/viewer/2022062320/5590ca461a28ab4d538b4681/html5/thumbnails/25.jpg)
25
Costs and Effects
9
87
6
543
2
140,000
45,000
50,000
55,000
60,000
65,000
39 40 41 42 43 44 45
Life span/traffic volume ratio
Cost
($)
![Page 26: Economic Evaluation Apa Oh V. 2(2)](https://reader036.vdocuments.net/reader036/viewer/2022062320/5590ca461a28ab4d538b4681/html5/thumbnails/26.jpg)
26
9
7
5
32
140,000
45,000
50,000
55,000
60,000
65,000
39 40 41 42 43 44 45
Life span/traffic volume ratio
Cost
($)
Eliminate Strongly Dominated Strategies
![Page 27: Economic Evaluation Apa Oh V. 2(2)](https://reader036.vdocuments.net/reader036/viewer/2022062320/5590ca461a28ab4d538b4681/html5/thumbnails/27.jpg)
27
Calculate CE Ratios
Strategy Cost ($) Effect Incremental CE ratio
1 40,288 39.08 —
2 44,786 42.56 (44,786-40,288)/(42.56-39.08) = 1,293
3 45,944 43.04 (45,944-44,786)/(43.04-42.56) = 2,413
5 48,596 43.60 (48,596-45,944)/(43.60-43.04) = 4,736
7 56,812 43.83 (56,812-48,596)/(43.83-43.60) = 35,722
9 61,119 44.62 (61,119-56,812)/(44.62-43.83) = 5,452
![Page 28: Economic Evaluation Apa Oh V. 2(2)](https://reader036.vdocuments.net/reader036/viewer/2022062320/5590ca461a28ab4d538b4681/html5/thumbnails/28.jpg)
28
Identify and Eliminate Weakly Dominated Strategies
Strat. Cost ($) Effect Incremental CE ratio
1 40,288 39.08 —
2 44,786 42.56 (44,786-40,288)/(42.56-39.08) = 1,293
3 45,944 43.04 (45,944-44,786)/(43.04-42.56) = 2,413
5 48,596 43.60 (48,596-45,944)/(43.60-43.04) = 4,736
7 56,812 43.83 (56,812-48,596)/(43.83-43.60) = 35,722
9 61,119 44.62 (61,119-56,812)/(44.62-43.83) = 5,452
![Page 29: Economic Evaluation Apa Oh V. 2(2)](https://reader036.vdocuments.net/reader036/viewer/2022062320/5590ca461a28ab4d538b4681/html5/thumbnails/29.jpg)
29
Identify and Eliminate Weakly Dominated Strategies
9
7
5
32
140,000
45,000
50,000
55,000
60,000
65,000
39 40 41 42 43 44 45
Life span/traffic volume ratio
Cost
($)
![Page 30: Economic Evaluation Apa Oh V. 2(2)](https://reader036.vdocuments.net/reader036/viewer/2022062320/5590ca461a28ab4d538b4681/html5/thumbnails/30.jpg)
30
Recalculate CE Ratios
Strategy Cost ($) Effect Incremental CE ratio
1 40,288 39.08 —
2 44,786 42.56 (44,786-40,288)/(42.56-39.08) = 1,293
3 45,944 43.04 (45,944-44,786)/(43.04-42.56) = 2,413
5 48,596 43.60 (48,596-45,944)/(43.60-43.04) = 4,736
9 61,119 44.62 (61,119- 48,596)/(44.62-43.60) = 12,277
![Page 31: Economic Evaluation Apa Oh V. 2(2)](https://reader036.vdocuments.net/reader036/viewer/2022062320/5590ca461a28ab4d538b4681/html5/thumbnails/31.jpg)
31
Recalculate CE Ratios
9
5
3
140,000
45,000
50,000
55,000
60,000
65,000
39 40 41 42 43 44 45
Effect
Cost
($)
2
![Page 32: Economic Evaluation Apa Oh V. 2(2)](https://reader036.vdocuments.net/reader036/viewer/2022062320/5590ca461a28ab4d538b4681/html5/thumbnails/32.jpg)
32
Which Strategy to Use?
• Move down the list of undominated programs until the CE ratio of the next program is “too high”
• What is threshold CE ratio?– No correct answer
![Page 33: Economic Evaluation Apa Oh V. 2(2)](https://reader036.vdocuments.net/reader036/viewer/2022062320/5590ca461a28ab4d538b4681/html5/thumbnails/33.jpg)
How to set up an economic evaluation
![Page 34: Economic Evaluation Apa Oh V. 2(2)](https://reader036.vdocuments.net/reader036/viewer/2022062320/5590ca461a28ab4d538b4681/html5/thumbnails/34.jpg)
34
Basic Steps in an Economic Evaluation
1. Define the problem
2. Identify the relevant alternatives
3. Estimate expected costs
4. Estimate expected effects
5. Compare expected costs and effects
6. Time preferences & discounting
7. Analyze uncertainty
8. Sources of data
![Page 35: Economic Evaluation Apa Oh V. 2(2)](https://reader036.vdocuments.net/reader036/viewer/2022062320/5590ca461a28ab4d538b4681/html5/thumbnails/35.jpg)
35
Define the Problem
• Choice of perspective is particularly important– Whose costs are considered relevant?
• Possible perspectives– Local government– Departments– Nonprofits– Subsets of residents– Businesses – Residents
• Whatever you choose, make it clear
![Page 36: Economic Evaluation Apa Oh V. 2(2)](https://reader036.vdocuments.net/reader036/viewer/2022062320/5590ca461a28ab4d538b4681/html5/thumbnails/36.jpg)
36
Identify the Relevant Alternatives
• Possible comparators– “Best” alternative– Most widely used alternative– “Do nothing”
• Appropriate comparator may vary by location
• Potential for bias
![Page 37: Economic Evaluation Apa Oh V. 2(2)](https://reader036.vdocuments.net/reader036/viewer/2022062320/5590ca461a28ab4d538b4681/html5/thumbnails/37.jpg)
37
Sources of Data
• Big challenge in public policy– We don’t measure well– Think through what you need – Gather local data if possible– Make clear what you are assuming
![Page 38: Economic Evaluation Apa Oh V. 2(2)](https://reader036.vdocuments.net/reader036/viewer/2022062320/5590ca461a28ab4d538b4681/html5/thumbnails/38.jpg)
38
• Associate resource utilization with outcomes and events
• Associate costs with resource utilization
Calculate Expected Costs
![Page 39: Economic Evaluation Apa Oh V. 2(2)](https://reader036.vdocuments.net/reader036/viewer/2022062320/5590ca461a28ab4d538b4681/html5/thumbnails/39.jpg)
39
Direct Costs
• Construction & Development•Operating Expenses• Staff/staff time•Advising/Consulting
![Page 40: Economic Evaluation Apa Oh V. 2(2)](https://reader036.vdocuments.net/reader036/viewer/2022062320/5590ca461a28ab4d538b4681/html5/thumbnails/40.jpg)
40
Indirect Costs
• Impact on others– Cash costs for service
replacement/augmentation
– Time costs to users– Impacts on other
programs’ capacity to meet objectives
![Page 41: Economic Evaluation Apa Oh V. 2(2)](https://reader036.vdocuments.net/reader036/viewer/2022062320/5590ca461a28ab4d538b4681/html5/thumbnails/41.jpg)
41
Compare Costs and Effects
• Identify strongly and weakly dominated strategies
• Calculate incremental cost-effectiveness ratios
![Page 42: Economic Evaluation Apa Oh V. 2(2)](https://reader036.vdocuments.net/reader036/viewer/2022062320/5590ca461a28ab4d538b4681/html5/thumbnails/42.jpg)
42
Time Preference and Discounting
• Prefer to receive benefits now, defer costs to future
• Present value of X occurring T years from now
= X/(1+r)T
Time preference
• Discount costs and benefits at same rate
• Most widely used rate: 3% (typically in range 0%-10%)
Common practice
![Page 43: Economic Evaluation Apa Oh V. 2(2)](https://reader036.vdocuments.net/reader036/viewer/2022062320/5590ca461a28ab4d538b4681/html5/thumbnails/43.jpg)
43
Analyze Uncertainty
• Perform sensitivity analysis– Helps to gauge the
reliability and robustness of the results
– Particularly important for public policy, given data problems
• Identify ranges of values for model inputs
• Recalculate economic evaluation using these different values
![Page 44: Economic Evaluation Apa Oh V. 2(2)](https://reader036.vdocuments.net/reader036/viewer/2022062320/5590ca461a28ab4d538b4681/html5/thumbnails/44.jpg)
44
![Page 45: Economic Evaluation Apa Oh V. 2(2)](https://reader036.vdocuments.net/reader036/viewer/2022062320/5590ca461a28ab4d538b4681/html5/thumbnails/45.jpg)
45
Thank you !
Peter Mallow, AICPResearch Associate – S2 Statistical SolutionsPhD Student – University of [email protected]
Della Rucker, AICP, CEcDPrincipal – Wise Economy Workshopwww.wiseeconomy.comDella.rucker@wiseeconomy.com513.288-6613Twitter: @dellaruckerFacebook: Della Rucker Aicp CecdAlso on Google + and LinkedIn