decisions, decisions: what policymakers need to know about cost-benefit analysis
DESCRIPTION
TRANSCRIPT
Slide 1
Decisions, Decisions: What Policymakers Need to Know About Cost-Benefit Analysis
October 25, 2012
Craig Prins, Executive Director, Oregon Criminal Justice Commission Sarah Galgano, Policy Analyst, Vera Institute of Justice
Slide 2
Decisions, Decisions: What Policymakers Need to Know About Cost-Benefit Analysis
Craig Prins Executive Director
Oregon Criminal Justice Commission
Sarah Galgano Policy Analyst
Vera Institute of Justice
Slide 3
The Cost-Benefit Knowledge Bank for Criminal Justice (CBKB) is a project of the Vera Institute of Justice funded by the U.S. Department of Justice’s Bureau of Justice Assistance.
• Website (cbkb.org) • Cost-benefit analysis toolkit • Snapshots of CBA literature • Podcasts, videocasts, and webinars • Roundtable discussions • Community of practice • Technical assistance
Slide 4
Agenda
Introductions & housekeeping
Cost-benefit analysis (CBA) basics
Case study: The use of CBA in Oregon
CBA and decision making
How CBA can answer questions about policy options
Q&A
5 minutes
5 minutes
10 minutes
15 minutes
10 minutes
15 minutes
Slide 5
Housekeeping items
Questions:
Use the Chat feature to send us questions during the webinar.
Slide 6
Housekeeping items (continued)
• If you need webinar support or for troubleshooting: Type questions into the chat box. Call 800-843-9166. Send e-mail to [email protected].
• This webinar is being recorded.
• The recording and PowerPoint slides will be posted
on cbkb.org.
Slide 7
Preview
This webinar will discuss:
The role of CBA in decision making, even when data is imperfect
Questions frequently asked about CBA results
The ways a CBA can address questions relevant to decision making
Slide 8
CBA Basics
Slide 9
What is cost-benefit analysis?
A type of economic analysis that compares the costs and benefits of policies and programs, and that:
• includes the perspectives of multiple stakeholders;
• presents a long-term picture;
• uses dollars as a common measurement; and
• allows you to compare programs and policies that have different outcomes.
Slide 10
The CBA process
1. Determine the impact of the initiative.
2. Determine which perspectives to include.
3. Measure costs and benefits (in dollars).
4. Compare costs and benefits.
5. Assess the reliability of the results.
For a glossary of CBA terms, refer to cbkb.org/basics/glossary
Slide 11
Case Study: The Use of CBA in Oregon
Slide 12
Justice policy is multifaceted
• Justice policy often has these goals:
Deter new criminal activity.
Reduce recidivism.
Improve public safety cost-effectively.
• A new policy has resource costs, but can also save government resources and benefit society.
• CBA monetizes savings, benefits, and resource costs.
Slide 13
Building a capacity for CBA in Oregon
• In 2006, Oregon Criminal Justice Commission began the development of statewide cost-benefit model for the criminal justice system.
• Oregon leveraged the cost-benefit work of the Washington State Institute for Public Policy (WSIPP).
• The Oregon model was designed to provide information to policymakers and the public about the relative costs and effectiveness of justice investments. Oregon’s cost-benefit methodology is available at
http://www.oregon.gov/cjc/docs/cost_benefit_methodology_090106.pdf
Slide 14
Oregon invests in reentry programs
• Oregon made an investment decision based on credible CBA results from another state.
• WSIPP had estimates of reentry program success in Washington.
• Would the results translate to Oregon? The state has 36 diverse counties.
WSIPP cost estimates may not apply to rural areas.
The program’s effect in Oregon may differ from the WSIPP estimate.
Slide 15
Example of WSIPP estimates
Slide 16
Oregon’s reentry program
• Continues prison treatment services for people once they are in the community
• Coordinates ancillary, community-based services for recently released individuals: Mental health services Employment counseling Educational attainment programs
• Increases public safety and reduces recidivism
Slide 17
CBA of Oregon’s reentry program
• Researchers at the Oregon Criminal Justice Commission (CJC) conducted an outcome evaluation and a cost-benefit analysis.
• Researchers used local data to:
Determine program-specific costs for the reentry center.
Track individuals who use services: Did they reoffend?
Measure the benefits specific to Oregon.
Slide 18
CBA results: Oregon’s reentry program
The statewide reentry program:
Reduces recidivism by 27%
Results in an estimated 3.5 fewer felony convictions for every 10 participants over a 10-year period
Costs about $3,400 per participant
Generates benefits of $8,600 to taxpayers and $14,000 in avoided victimizations per participant
Results in benefit-cost ratio of $6.73 to every $1 spent
Program evaluation and cost-benefit analysis available at: http://www.oregon.gov/CJC/docs/reentry_eval_final.pdf
Slide 19
Lessons learned from Oregon CBA
• Local data adds value when using a CBA to inform policy decisions. Statewide averages can be misleading.
• Success is not a sure thing; program fidelity is key.
• CBA adds important information to the policy dialogue.
Slide 20
Questions?
Craig Prins Sarah Galgano [email protected] [email protected]
Please use the Chat feature to send us your questions.
Slide 21
CBA and Decision Making
Slide 22
CBA is a decision-making tool
• CBA is a way to consider the pros and cons systematically.
• Justice policy is a long-term investment decision.
• Beyond good intentions: Is a given policy the most effective way to improve public safety?
• CBA is a business-like approach to increasing public safety and reducing cost as our “nonprofit’s” goal.
Slide 23
Frequently asked questions about CBA
Decision makers (e.g., legislators, legislative staff, budget planners) often have questions about CBA:
1. Can I believe the results?
2. Are the results guaranteed?
3. Who gets the benefits?
4. What is the alternative?
Slide 24
Can I believe the results?
• Confidence in the results is related to the quality of the CBA’s inputs.
Are the predicted policy effects based on a rigorous analysis? Are the program’s costs measured accurately?
• Sensitivity analysis should be used to “kick the tires” on the results.
• Transparency with the details will increase readers’ confidence in the study.
Slide 25
Are the results guaranteed?
• No. No one has a crystal ball.
• “Past performance does not guarantee future results.”
• The policy effect can vary from the CBA results for these reasons: Measurement of the predicted policy effect
Fidelity of the program to the policy that was evaluated
• The costs and benefits of the policy effect may be hard to monetize.
Slide 26
Who gets the benefits?
• The return on investment is both to the government budget (taxpayer benefits) and to society (such as victim benefits).
• These benefits can be presented separately to provide clarity as to how they accrued.
• Some benefits may accrue in the long term.
• Budget savings are realized when there is less crime and therefore less spending on courts, policing, and corrections.
Slide 27
Who gets the benefits? (continued)
• CJC estimate: A 1% drop in recidivism results in $4.3 million avoided in annual victim and taxpayer costs due to crime.
• Who gets these savings?
Savings may be reallocated to other sectors of corrections/state budgets.
Taxpayer resources may be used more efficiently.
Slide 28
What is the alternative?
• CBA is based on programmatic/initiative investments.
• A single cost-benefit analysis reports the net benefit versus “business as usual.”
• Several CBAs can provide a menu of policy choices (similar to the ratings in Consumer Reports).
Slide 29
Questions?
Craig Prins Sarah Galgano [email protected] [email protected]
Please use the Chat feature to send us your questions.
Slide 30
How CBA Can Answer Questions About Policy Options
Slide 31
Frequently asked questions about CBA
1. Can I believe the results?
2. Are the results guaranteed?
3. Who gets the benefits?
4. What is the alternative?
Slide 32
How CBA can answer these questions
Question: How CBA can answer:
1. Can I believe the results? Base-case scenario Partial sensitivity analysis
2. Are the results guaranteed? Break-even analysis Best- and worse-case scenarios analysis Monte Carlo analysis
3. Who gets the benefits? Analysis by perspective
4. What is the alternative? Menu of CBA results
Slide 33
Hypothetical prison education program
• This example will be used to illustrate how CBA can address questions 1-3.
• Hypothetical program:
Aims to reduce number of people who commit new crimes after their release
Uses estimates based on evaluations of similar programs because information on program impact is not available Has reduced recidivism by 15 percent in other states Costs an estimated $10,000
Slide 34
1. Can I believe the results?
A CBA can earn the trust of readers by:
Documenting why “base-case” scenario assumptions were chosen
Using partial sensitivity analysis to illustrate how the results vary if the assumptions are inaccurate
Slide 35
Base-case scenario
Total Cost Total Benefit Net Benefit Prison education $10,000 $17,000 $7,000
Table 1: Hypothetical Prison Education Program: Base Case
Note: The program is assumed to reduce recidivism by 15 percent over three years, based on studies of other similar programs.
Slide 36
• Provides information on the best estimate of a policy’s net benefits
• Is based on direct evaluation of the initiative or evaluations of similar programs
• Explains what assumptions were used
Base-case scenario
Slide 37
Partial sensitivity analysis
Change in recidivism +5% 0% -5% -10% -15% -20% Program costs $10,000 $10,000 $10,000 $10,000 $10,000 $10,000 Program benefits -$2,000 $5,000 $8,000 $10,000 $17,000 $22,000 Net benefits (Benefit minus cost) -$12,000 -$5,000 -$2,000 $0 $7,000 $12,000
Table 2: Hypothetical Prison Education Program: Partial Sensitivity Analysis
Slide 38
Partial sensitivity analysis
• Provides information on how variation from the base-case scenario will affect the CBA results
• Determines how sensitive the results are to changes in a single variable
• Involves selecting a single variable in the analysis and changing its value (holding other variables constant)
Slide 39
2. Are the results guaranteed?
• A CBA cannot provide guaranteed results, but can address uncertainty by:
Reporting the policy effect necessary for the costs and benefits to break even
Reporting the best-case and worse-case scenarios
Modeling the probability of potential net benefit using Monte Carlo analysis
Slide 40
Break-even analysis
Change in recidivism +5% 0% -5% -10% -15% -20% Program costs $10,000 $10,000 $10,000 $10,000 $10,000 $10,000 Program benefits -$2,000 $5,000 $8,000 $10,000 $17,000 $22,000 Net benefits (Benefit minus cost) -$12,000 -$5,000 -$2,000 $0 $7,000 $12,000
Break-even point
Table 3: Hypothetical Prison Education Program: Break-Even Analysis
Slide 41
Break-even analysis
• Provides information on the policy effect necessary for the program benefits to equal the program costs
• Useful when information about program impact is unavailable
Slide 42
Best-case/worse-case scenario analysis
Change in recidivism +5% 0% -5% -10% -15% -20% Program costs $10,000 $10,000 $10,000 $10,000 $10,000 $10,000 Program benefits -$2,000 $5,000 $8,000 $10,000 $17,000 $22,000 Net benefits (Benefit minus cost) -$12,000 -$5,000 -$2,000 $0 $7,000 $12,000
Best case
Worst case
Table 4: Hypothetical Prison Education Program: Best-case and Worse-case Scenarios
Slide 43
Best-case/worse-case scenario analysis
• Provides policymakers with information on how a broad range of a policy’s possible outcomes affect the bottom line.
• Establish upper and lower boundaries of a cost-benefit study’s results.
• Worst-case scenario analysis is based on using all the least favorable assumptions.
• Best-case scenario analysis is based on all the most favorable assumptions.
Slide 44
Example of Monte Carlo results
Slide 45
Monte Carlo analysis
• Provides information about the probability that outcomes will occur
• Examines multiple variables simultaneously and simulates thousands of scenarios
• Produces a range of possible results and their associated probabilities
For more on Monte Carlo analysis, watch the recording of the CBKB webinar “Sensitivity Analysis for Cost-Benefit Studies of Justice Policies” at cbkb.org.
Slide 46
3. Who gets the benefits?
Taxpayer Victim Participant Total Costs $10,000 $0 $0 $10,000
Benefits $12,000 $2,000 $3,000 $17,000
Net benefit $2,000 $2,000 $3,000 $7,000
Table 5: Hypothetical Prison Education Program, by Perspective
Slide 47
Reporting costs and benefits by perspective • CBA can answer this question by reporting the costs
and benefits for each perspective (or stakeholder) in the analysis.
• Outcomes related to perspectives that were not measured quantitatively should be discussed qualitatively.
For more on perspectives, refer to the CBKB Perspectives tool at cbkb.org/toolkit/perspectives.
Slide 48
4. What is the alternative?
CBA can be performed on more than one policy option to provide a menu of options.
Slide 49
Policy options in Washington State
Slide 50
Wrap-up
Question: How CBA can answer:
1. Can I believe the results? Base-case scenario Partial sensitivity analysis
2. Are the results guaranteed? Break-even analysis Best- and worse-case scenarios analysis Monte Carlo analysis
3. Who gets the benefits? Analysis by perspective
4. What is the alternative? Menu of CBA results
Slide 51
Questions and Answers
Slide 52
Questions?
Craig Prins Sarah Galgano [email protected] [email protected]
Please use the Chat feature to send us your questions.
Slide 53
Wrap-up
Slide 54
Review
• We covered:
• How CBA adds to policy dialogue
• What questions arise about CBA results
• How CBA can address frequently asked questions
Slide 55
Takeaways
• CBA is a useful decision-making tool.
• CBAs is not just about the results. The analysis adds important information to the policy dialogue.
Slide 56
Follow-up
Before you log out of the webinar, please complete the evaluation form.
To receive information and notifications about upcoming webinars and other events:
• Visit the Cost-Benefit Knowledge Bank for Criminal Justice at cbkb.org.
• Subscribe to receive updates from CBKB.
• Follow us on Twitter at twitter.com/CBKBank.
Slide 57
Contact information Craig Prins
Executive Director Oregon Criminal Justice
Commission
Sarah Galgano Policy Analyst
Vera Institute of Justice
Slide 58
The Cost-Benefit Knowledge Bank for Criminal Justice (CBKB) is a project of the Vera Institute of Justice funded by the U.S. Department of Justice’s Bureau of Justice Assistance.
• Website (cbkb.org) • Cost-benefit analysis toolkit • Snapshots of CBA literature • Podcasts, videocasts, and webinars • Roundtable discussions • Community of practice • Technical assistance
Slide 59
Thank you!
Slide 60
This project is supported by Grant No. 2009-MU-BX K029 awarded by the Bureau of Justice Assistance. The Bureau of Justice Assistance is a component of the Office of Justice Programs, which also includes the Bureau of Justice Statistics, the National Institute of Justice, the Office of Juvenile Justice and Delinquency Prevention, and the Office of Sex Offender Sentencing, Monitoring, Apprehending, Registering, and Tracking. Points of view or opinions in this document are those of the author and do not represent the official position or policies of the United States Department of Justice.