A BRIEF REVIEW OF BENEFIT-COST TESTING FOR ENERGY EFFICIENCY PROGRAMS:

OVERVIEW OF STATE APPROACHESAND SOME KEY ISSUES

Martin Kushler, Ph.D.Senior Fellow

American Council for an Energy-Efficient Economy (ACEEE)July 26, 2016

222

The American Council for an Energy-Efficient Economy (ACEEE)

• Nonprofit 501(c)(3) dedicated to advancing energy efficiency through research, communications, and conferences. Founded in 1980.

• ~40 staff in Washington DC, + field offices in DE, MI, and WI.• Focus on End-Use Efficiency in Industry, Buildings, Utilities,

and Transportation; and State & National Policy • Funding: Foundations (34%), Federal & State Grants (7%),

Contract research work (21%) Conferences and Publications (34%), Contributions and Other (4%)

Martin Kushler, Ph.D. (Senior Fellow, ACEEE)• 30 years conducting research in the utility industry, including:• 10 years as Director of the ACEEE Utilities Program• 10 years as Director of Evaluation at the Michigan PSC• Have assisted over a dozen states with utility EE policies

TOPICS• Background and purposes of benefit-cost testing• Overview of the 5 traditional cost-effectiveness

tests• National survey results on state approaches to

benefit-cost tests• How the Minnesota B/C approach compares• Some current key issues and concerns in the

industry regarding cost-effectiveness testing • Conclusions

3

SOME HISTORY BEHIND B/C TESTING FOR EE• Opposition to energy efficiency requirements by utilities and other

powerful parties (e.g., industrial customers)• High “burden of proof” placed on energy efficiency as a resource• Scrutiny disproportionate to any other utility expenditure• The legacy continues today

e.g., comparison of energy efficiency to other “alternative resources”100% of states require benefit-cost testing for energy efficiencyOnly 67% of states require B/C testing for load management

programsOnly 28% of states require B/C testing for renewable energy

“programs” (for RPS, virtually nothing)

4

PURPOSES OF BENEFIT-COST TESTINGFOR UTILITY ENERGY EFFICIENCY PROGRAMS

• To help ensure that ratepayer dollars are prudently spent (in this case, defined as the “benefits” being equal to or greater than the “costs”….. i.e., a B/C ratio of 1.0 or greater)

• To help prioritize amongst resource/program options (i.e., larger B/C ratios deliver more benefits per dollar)*

____________ * Of course, other factors also influence selection of

programs

5

THE 5 “CALIFORNIA” TESTS

• The Participant test [PART]• The Utility or Program Administrator test [UCT or PACT]• The Total Resource Cost test [TRC]• The Societal cost test [SCT]• The Ratepayer Impact Measure test [RIM]

6

7

THE ACEEE STUDY

• In 2012 ACEEE published the results of a national survey to identify and document energy efficiency program evaluation requirements and methods in each of the 50 states

• One of the key focus areas was state approaches to cost-effectiveness testing

Detailed results available in the full report:A National Survey of State Policies and Practices for the

Evaluation of Ratepayer-Funded Energy Efficiency Programs http://www.aceee.org/research-report/u122

8

SUMMARY OF STATE APPROACHESREGARDING BENEFIT-COST TESTS

• 44 states have ratepayer funded energy efficiency programs• All 44 states surveyed indicated that they use some type of

benefit-cost test. 41 have a “primary” test.

9

10

LEVEL AT WHICH B/C TESTS ARE REQUIRED

70% Overall portfolio 70% Total program* 40% Customer project 30% Individual Measure**________________

* Of these states, nearly half noted that they have some exceptions, such as low-income programs, pilot programs, etc.

** A majority of those states have some exceptions or flexibility in the application of B/C tests at the measure level, such as allowing ‘bundling’ of measures, or exceptions for certain types of programs (e.g., ‘whole house’ programs, low-income programs, etc.)

11

MINNESOTA APPROACH TO B/C TESTING

• Minn. Rules 7690.0550 requires utilities to report cost-effectiveness from the Societal, Utility, Participant, and Ratepayer perspectives

• Generally follows the methodology outlined in the California Standard Practice Manual for these tests

• Approve cost-effectiveness at the utility segment level, rather than the individual program level.

• “The Department has generally focused on ensuring that programs are cost-effective from the Utility and Societal perspectives, as they provide a relatively balanced comparison of CIP program benefits and costs.” (source: DOC memo, 6/24/16)

12

BENEFIT AND COST ELEMENTS IN MINNESOTA B/C TESTS

Component Societal Cost Test

Utility Cost Test

Participant Cost Test

Ratepayer Impact Measure

Avoided Energy and Capacity Benefit Benefit -- Benefit

Natural Gas Environmental Damage Factor Benefit -- -- --

Non-Natural Gas Fuel Environmental Damage Factor Benefit -- -- --

Utility Administrative Costs Cost Cost -- Cost

Utility Measure Incentive Payments -- Cost Benefit Cost

Direct Participant Costs Cost -- Cost --

Participant Bill Savings -- -- Benefit Cost

13

 

MINNESOTA APPROACH TO ‘AVOIDED COSTS’• Minnesota is very noteworthy on the natural gas side,

for having the state establish the avoided cost assumptions and values for all utilities to use

• On the electric side, each utility can establish their own avoided costs (recognizing diversity among electric utility resource situations)

• Although in early 2016, the Dept. reviewed the avoided cost assumptions for the big 3 electrics (MP, OTP,Xcel) Ok’d avoided capacity & energy costs for 2017-2019 Called for study of avoided T&D costs for 2018-2019 Called for Staff to continue to assess whether standardization of

avoided energy and capacity costs is warranted for 2020 plans

14

SOME KEY CONCERNS IN B/C TESTING

1. Imbalance in types of costs and benefits considered

2. Incomplete consideration of utility system benefits

3. The rate impact measure (RIM) test

15

1616

WHY ARE THESE ISSUES IMPORTANT?• Energy Efficiency is by far the cheapest source of new

supply for utilities (true for electricity and gas) • If flawed or incorrect B/C tests are resulting in exclusion

of EE programs that are actually cost-effective… then utility system costs (paid by ratepayers) will be higher than they need to be

• From a broader state public policy perspective, if EE programs are excluded, additional benefits will also be lost Reduced environmental emissions Local economic benefits ($ retained in-state, local employment, etc.) Improved housing & building stock More competitive businesses (reduced operating costs) Inter-class and intra-class equity (residential customers, low-income)

KEY CONCERN #1: IMBALANCE IN THE TYPES OF COSTS AND BENEFITS

QUANTIFIED IN THE COST-EFFECTIVENESS TEST

The core problem:

As currently implemented, the TRC test* is fundamentally imbalanced…. it includes all participant costs for an energy efficiency project, but ignores all of the participant ‘non-energy’ benefits from the project.

___________________

* Note: a Societal Test can have the same imbalance, if not properly calculated

17

18

DATA ON COSTS AND BENEFITS INCLUDEDBenefits Included in the Primary Test used by States• Avoided utility system costs: All States• Environmental: 14 (32%) [8 calculate, 6 use general ‘adder’]

[10 states include CO2 as part of rationale]

• Other fuels and water 7 (17%)• Participant non-energy benefits: 2 (5%)

Reduced maintenance: 2 (5%) Health: 0 Comfort: 0 Improved productivity: 0 Increased property value: 0

[Only 1 out of 29 states using the TRC as a primary test included a specific customer NEB as a quantified benefit.]

Costs Included in the Primary Test used by States• Utility Program costs: All States• Participant costs: 36 (88%) (including all the TRC states)

19

REASONS FOR CONCERNS ABOUT THE ‘IMBALANCE’TRC= (utility costs + participant costs) vs. utility benefits only• Not conceptually logical – customers invest their money

in EE projects for a variety of benefits - - not solely to save energy. Why include all costs they incur but exclude many benefits in a B/C calculation?

• Systematically biased against EE – these extra ‘customer’ costs are not considered when selecting supply-side options (e.g., purchased power, distributed generation, customer-sited renewables, etc.)

• Out-of-step with common practice in program design and marketing (which often emphasizes NEBs)

• Will result in ‘screening out’ programs that would be cost-effective from a utility resource perspective

DOES THIS PROBLEM WITH PARTICIPANT COSTS VS. PARTICIPANT NEBs REALLY MATTER?

• Maybe not that important in the past Simpler programs Smaller EE budgets and savings goals Lots of EE ‘passed’ TRC, so not an issue of concern

• Increasingly important today Much more aggressive EE goals…will require “deeper”

savings, bigger ‘projects’ Program strategies that emphasize NEBs in persuading

customers to participate Expect to see larger participant cost contributions,

driven at least in part by NEBs associated with a project

20

EXAMPLE: TRC AND HOME PERFORMANCEScreening without NEBs (courtesy of Chris Neme)

CostsMeasures $7,500Administration $1,500Total $9,000

BenefitsTherms kWh kW

Energy Savings 300 750 0.6Savings Life -Yrs 20 10 10Avoided Cost/Unit $1.35 $0.14 $115Value 4,645$ 1,020$ 682$ 6,347$

Net Benefits (2,653)$

Benefit-Cost Ratio 0.71

21

22

REMEDIATION OPTIONSTRC= (utility costs + participant costs) vs. utility benefits only

To address the “imbalance”:1. Adjust participant cost to “energy portion only”TRC= (utility costs + participant energy portion of costs) vs. utility benefits only2. Add NEBs to “benefits”TRC= (utility costs + participant costs) vs. utility benefits + participant benefits3. Switch tests – to the UCT/PACT TRC UCT= (utility costs + participant costs) vs. utility benefits only

23

1. Cost Adjustments (‘Energy’ portion only)

Advantages“apples to apples”Fewer cost-effective

programs fail screening

DisadvantagesMore $ on evaluationRelies on subjective

judgmentDifficult to be prospectiveNot economically optimal

Cost reduction can be less than value of NEBs

Summary: better than nothing; help for selected programs.

24

2. Add NEBs to ScreeningAdvantagesMost accurate choiceAll societally cost-

effective programs pass

DisadvantagesLots more $ on

evaluation(esp. if addressing all key

NEBs Complex, can be

controversialRegulators may feel

they lack authority

Summary: theoretically ideal, but difficult in practice.

25

3. Switch to UCT/PACTAdvantages“apples to apples”Simplest choiceLeast expensive optionSymmetry w/supply

sideIs optimal from a utility

system perspective

Disadvantages• Doesn’t consider all

societal costs and benefits

Summary: may be easiest solution in the short term.

26

Application of Fixes Home Performance Example(courtesy C. Neme)

TRC TodayTRC Cost Adjusted

TRC w/NEBs PACT

CostsMeasure Costs $7,500

Rebate 33% $2,500 $2,500 $2,500 $2,500 $2,500Participant 67% $5,000 $5,000 $5,000 $5,000

Administration $1,500 $1,500 $1,500 $1,500 $1,500

Customer Attribution of CostsEnergy Reasons 50%Non-Energy Reasons 50%Cost Adjustment (3,750)$ -$3,750

Total Costs $9,000 $5,250 $9,000 $4,000

BenefitsEnergy - Avoided Costs 6,000$ $6,000 $6,000 $6,000 $6,000

Non-Energy 6,000$ $6,000

Total Benefts $6,000 $6,000 $12,000 $6,000

Net Benefits -$3,000 $750 $3,000 $2,000FAIL PASS PASS PASS

Scenario

2727

CONCERN #2:UTILITY SYSTEM BENEFITS OFTEN UNDER-VALUED

27 All costs typically included (administration, rebates, eval., etc.) Energy & capacity benefits typically included (albeit

inconsistently) But many other benefits often not included or under-valued

Avoided T&D costs often excluded or under-valued Reserve margin benefits often omitted Avoided environmental compliance costs often excluded Wholesale price suppression effects not commonly captured Risk mitigation benefits rarely included Lower credit/collection costs rarely included Line loss reductions commonly understated

Higher at peak than rest of year – only sometimes addressed Should use marginal loss rates, but average losses used instead Result: Efficiency under-valued in all

screening tests.

28

LOW-INCOME PROGRAMS ARE A SPECIAL CATEGORY• Benefits included in a typical comprehensive low-

income weatherization program:

o Reduced utility credit and collection costso Home repair (housing preservation & resident

stability)o increased comforto Indoor air quality (mold, allergens, radon, CO, etc)o Poisons (house cleaners, lead, etc)o Safety (furnace, housing structure, living

conditions, etc)The value of these “non-energy” benefits typically exceeds the value of the “energy” benefits

Nearly all states have special B/C provisions for low-income energy efficiency programs

29Source: Regulatory Assistance Project

CONCERN #3:

THE RATEPAYER IMPACT MEASURE (RIM)IS INAPPROPRIATE AS A TEST OF COST-EFFECTIVENESS

• It is an indicator of distributional impacts of cost re-allocation… NOT “cost-effectiveness”

• “lost revenues” are the major “cost” component of the RIM test, but

• Lost revenues represent “sunk costs” of other aspects of utility system operation…. They are NOT a “cost” of the energy efficiency program

• The RIM test tells us nothing about the cost-effectiveness of energy efficiency compared to other resources

• The RIM test is never applied to any other utility system resource (they would all fail)

30

THE RIM TEST HAS PERVERSE IMPACTS ON RESOURCE SELECTION

• In most typical conditions, virtually no EE program will pass the RIM test - - even if it is by far the cheapest new resource

• In fact, a program to go and rip out efficiency measures from buildings might well pass the RIM testo Retail rate: 12 cents/kWho Avoided cost: 8 cents/kWho Program cost: 3 cents/kWh[RIM = +12 -8 -3 = +1 ]

The RIM test has been widely abandoned as a decision-making factor. Essentially none of the 45+ states with utility EE programs use the RIM test as its primary test

31

REGULATOR CONCERN ABOUT ‘RATES’ IS LEGITIMATE, BUT… • The RIM test does not really tell you anything about the

magnitude of any rate impact (for that you need full IRP system modeling)

• While logically, rates may rise a bit due to spreading utility system costs over fewer sales (although total bills will go down)…..over time, rates may be reduced by energy efficiency as well…due to large avoided system costs.

See:Counting the Capacity that Didn’t Hatch” The Rate Mitigation Effect of DSM Programs, Jennifer Edwards & Nancy Lange, MNCEE. Presented at the 2013 ACEEE National Conference on Energy Efficiency as a Resource, Nashville, TN.

32

THE CONCERN FOR “NON-PARTICIPANTS” IS MISGUIDED• That concern is never voiced for other resources (new

power plant, new substation, etc.)Moreover, it’s time to flip the logic around:• Nearly all customers still have opportunities for energy

efficiency improvement• If energy efficiency programs are widely available for all

customers, and a customer chooses to not participate, and thereby stay “inefficient”….

….that customer is creating a need for additional utility system resources…and raising utility system costs for all customers The RIM test (aka “non-participant test”) protects wasteful

customers who are raising costs for all other customers.

33

34

CONCLUSIONS• Benefit-cost testing can be informative, and useful for decision-

making….if properly done• Use of benefit-cost tests for assessing ratepayer-funded energy

efficiency programs is essentially universal in the states• Reliance upon TRC for cost-effectiveness screening is very

widespread (much due to legacy and entrenched practice)• The TRC test (as commonly applied) has serious shortcomings

that are likely to impede the full acquisition of cost-effective energy efficiency as a utility resource.

• The RIM test should never be used for EE program screening• Utility system benefits are often under-valued in all of the B/C

tests• These concerns about B/C testing are leading to a re-examination

of this issue in the industry and the regulatory community

35

KEY RECOMMENDATIONS

• Fix the “imbalance” problem with the currently dominant test (TRC), or switch B/C tests

• Some combination of a Utility Cost Test and Societal Cost Test is likely a more practical approach

• Be sure to fully account for all utility system benefits• Apply B/C screen at the program and portfolio level,

not at the individual measure level• Don’t use RIM test as a screen

[Minnesota is in pretty good shape on most of these.]

PRO’S AND CON’S OF MINNESOTA’S B/C APPROACHPro’s• Prioritizes theoretically ‘balanced’ tests (Societal and Utility)• Focuses on approval at the segment level (allows for

flexibility for individual programs)• Includes quantification of environmental benefits• State specification of natural gas B/C inputs ensures

objectivity and consistencyCon’s• Does not include participant NEBs in societal test, even

though participant costs are included• Doesn’t quantify other state policy objectives beyond enviro• May not fully quantify utility system benefits • Does not include utility shareholder incentives as a cost

36

MICHIGAN’S DEFINITION OF COST-EFFECTIVENESS

“Utility system resource cost test” means a standard that is met for an investment in energy optimization if, on

a life cycle basis, the total avoided supply-side costs to the provider, including representative values for electricity or natural gas supply, transmission, distribution, and other associated costs, are greater than the total costs to the provider of administering and delivering the energy optimization program, including net costs for any provider incentives paid by customers and capitalized costs recovered under section 89.” [Emphasis added]

[PA 295, Section 13(d)]

37

MK “PUBLIC INTEREST TEST”Costs• All utility program costs (including shareholder incentives)• Any government program costs (including tax credits)• [Participant costs (only if participant NEBs are included) ]Benefits• All utility system avoided costs (see prior full list)• Other fuel and water savings (valued at “avoided cost”)• Environmental benefits (including greenhouse gases)• Any other specified state policy objective (may or may not

be quantified)• [Participant NEBs (only if participant costs are included) ]

Use ‘societal’ discount rate

38

EPILOGUE: RECENT DEVELOPMENTS• In the fall of 2013 the National Home Performance Council

and Conservation Services Group convened a working group of national experts in energy efficiency program evaluation, to review current B/C testing practices and recommend improvements

• That group developed a proposed new “Resource Value Framework”….

and formed an “Energy Efficiency Screening Coalition” to advocate for an improved B/C testing protocol

• Now an expanded effort is underway, to develop a new “Standard Practice Manual”…to replace the old California SPM– The National Efficiency Screening Project

See http://www.nationalefficiencyscreening.org

39

40

KEY ELEMENTS OF THE NEW B/C FRAMEWORK’1. Clarifies the objective of efficiency screening: to identify energy

efficiency programs that are in the public interest, as defined by that state.

2. Allows flexibility for each state to determine an efficiency screening test that accounts for the energy policy goals of that state.

3. Builds off of the existing screening tests, especially the Utility Cost and the Societal Cost tests.

4. Ensures a symmetrical approach to incorporating costs and benefits

5. Allows for consideration of relevant hard-to-quantify benefits.

6. Provides an explicit, transparent process to identify the appropriate screening test for each state.

41

Table 4.1 Examples of Cost-Effectiveness Tests

Utility System

Only

Utility System +

Participant

Utility System +

Low Income

All Quantified

Impacts Energy Efficiency Program Costs:

Utility System

Program Administration Costs Yes Yes Yes Yes

Financial Incentive to Participant Yes Yes Yes Yes

Non-Utility System

Participant Costs --- Yes For LI Yes

Participant Non-Energy Costs --- Yes For LI Yes

Other... --- --- --- Yes

Energy Efficiency Program Benefits:

Utility System

Energy Impacts (net) Yes Yes Yes Yes

Generation Capacity Impacts (net) Yes Yes Yes Yes

Transmission and Distribution Impacts (net) Yes Yes Yes Yes

Wholesale Market Price Suppression Effects Yes Yes Yes Yes

Environmental Compliance Impacts (net) Yes Yes Yes Yes

Risk Impacts (net) Yes Yes Yes Yes

Utility Non-Energy Impacts (net) Yes Yes Yes Yes

Non-Utility System

Low-Income Participant Non-Energy Impacts (net) --- Yes Yes Yes

Other Participant Non-Energy Impacts (net) --- Yes --- Yes

Other Energy Savings (oil, propane, gas) (net) --- Yes For LI Yes

Other Resource Savings (water, sewer) (net) --- Yes For LI Yes

Reduce Low-Income Energy Burden --- --- Yes Yes

Reduce Reliance on Fossil Fuels --- --- --- Yes

Reduced Water Consumption --- --- --- Yes

Other Energy Policy Benefits --- --- --- Yes

Environmental Benefits (net) --- --- --- Yes

Jobs and Economic Development (net) --- --- --- Yes

Societal Health Care Impacts (net) --- --- --- Yes

Other Non-Utility Benefits... --- --- --- Yes

SOME REFERENCESIs it Time to Ditch the TRC? Examining Concerns with Current Practice in Benefit-Cost AnalysisChris Neme & Marty Kushler, 2010 ACEEE Summer Study on Energy Efficiency in Buildings  Energy Efficiency Cost-Effectiveness Screening: How to Properly Account for ‘Other Program Impacts’ and Environmental Compliance CostsTim Woolf, William Steinhurst, Erin Malone, Kenji TakahashiSynapse Energy Economics, November 2012 Recognizing the Full Value of Energy EfficiencyJim Lazar & Ken Colburn, Regulatory Assistance Project, Sept. 2013

42