valuation of non-energy benefits to determine cost ... · valuation of non-energy benefits to...

Valuation of Non-Energy Benefits to Determine Cost-Effectiveness of Whole-House Retrofits Programs:

A Literature Review

Jennifer Thorne Amann

May 2006

Report Number A061

© American Council for an Energy-Efficient Economy 1001 Connecticut Avenue, NW, Suite 801, Washington, DC 20036

(202) 429-8873 phone, (202) 429-2248 fax, http://aceee.org

http://aceee.org/

Non-Energy Benefits, ACEEE

CONTENTS Acknowledgements................................................................................................................... ii Executive Summary ................................................................................................................. iii 1. Introduction............................................................................................................................1 2. Studies Included in the Literature Review.............................................................................2 3. Cost-Effectiveness Tests........................................................................................................3

Tests That Do NOT Incorporate NEBs................................................................................. 3 Tests that DO Incorporate NEBs .......................................................................................... 5

4. Non-Energy Benefits from Whole-House Retrofits: Identifying NEBs, Valuation

Methodologies, and Results .................................................................................................6 Identification of Non-Energy Benefits.................................................................................. 7 Valuation of Non-Energy Benefits ....................................................................................... 8

5. Proposed Methodology ........................................................................................................12

How Can NEBs Best Be Addressed?.................................................................................. 12 Which Test Is Appropriate? ................................................................................................ 13 Other Issues......................................................................................................................... 14 Data Needs .......................................................................................................................... 14

6. Summary and Conclusions ..................................................................................................14 Full Reference List (including Supplementary References .....................................................15 Appendix: State-by-State List of Cost-Effectiveness Tests Used............................................17

i


ACKNOWLEDGEMENTS Funding for this research was provided by the U.S. Environmental Protection Agency (EPA). I would like to thank David Lee and Carol May of EPA for their interest in this work, and particularly Carol for her careful review and thoughtful comments on an earlier draft. Thanks also to Dick Spellman for providing useful research data for the report. My appreciation to Oscar Bloch, Bob Knight, Bill Saxonis, and Lisa Skumatz for their thoughts on the topic and review of an earlier draft. Several ACEEE staff also provided input and comments; I would like to thank Steve Nadel, Bill Prindle, and Dan York for their efforts. Finally, many thanks to Katie Ackerly for stepping in to provide last-minute fact-checking and pulling together the final draft and to Renee Nida for her expert editorial review and report production.

ii


EXECUTIVE SUMMARY Growing anecdotal evidence and a limited amount of research suggest that many homeowners purchase whole-house retrofit services for the associated non-energy benefits (NEBs) including improved comfort, aesthetic enhancements, and better indoor air quality. Despite the value of NEBs to consumers, the cost-effectiveness tests used by many regulators to evaluate whole-house initiatives do not include the value of NEBs. This undervalues the benefits of these programs, making it harder for them to meet a target cost-effectiveness threshold. In general, regulatory agencies have been reluctant to use cost-effectiveness tests that incorporate NEBs because of the lack of consensus regarding methodologies for establishing NEB values and incorporating those values into the tests. This report presents a review of the literature on cost-benefit tests and non-energy benefits from residential retrofits. Specifically, the review draws from information on the common cost-benefit tests used by utilities and regulatory agencies; surveys of the NEBs common to whole-house retrofits; and valuation studies that have been conducted for these programs. Resources on these topics were developed through discussions with researchers and program managers and a search of literature from relevant organizations and conferences. The report represents the first phase of a three-phase project ACEEE is conducting with support from the U.S. Environmental Protection Agency and the New York State Energy Research and Development Authority (NYSERDA). A consumer survey, final analysis, and workshop regarding recommendations for incorporating NEBs into cost-effectiveness tests will build upon the findings presented here. A number of cost-benefit tests are used by utilities and regulatory agencies in every state to determine program cost-effectiveness. Tests that do not incorporate comfort and other non-energy benefits include the Utility Cost (or Program Administrator) Test, the Participant Test, the Total Resource Cost Test, the Ratepayer Impact Measure, and the Societal Test. Tests that do incorporate NEBs include the Public Purpose Test, the Total Market Effects Test, the Program Efficiency Test, and the Initial Cost-Benefit Test. With the exception of the Public Purpose Test, which is used to evaluate low-income weatherization programs in some states, the tests that do take NEBs into account are used for evaluation purposes only. For the tests that are widely used, it is common for program administrators and regulatory agencies to consider the results of multiple tests when evaluating program cost-effectiveness. Studies of residential retrofit programs (including low-income weatherization, HPwES, and other home retrofit programs) have catalogued an array of benefits beyond energy savings. Benefits often relate to financial savings other than energy bill relief, comfort, aesthetics, noise reduction, health and safety, and convenience. The data and methodologies for quantifying participant NEBs are less well developed, but have been estimated at 50% to 300% of annual household energy bill savings. Much of the data available on NEB valuation comes from studies of low-income weatherization programs. These studies provide a valuable framework for understanding valuation methodologies but their results do not adequately apply to all types of home retrofit programs. The literature suggests that it is important not only to develop a way to quantify

iii


NEBs, but to understand the benefits that consumers value most. These motivations are shown to differ according to region and economic status. More research on benefit valuation for non-low-income retrofit programs is needed. Studies attempting to quantify non-energy benefits typically involve a combination of survey techniques and computational or statistical analysis. Findings demonstrate that consumer surveys in which participants are asked to compare or categorically rate the importance of various benefits give much more consistent and conservative results than surveys that ask participants to estimate dollar values for the non-energy benefits. These findings, in conjunction with an understanding of the cost-effectiveness tests in use today, inform a proposed methodology for incorporating NEBs into cost-benefit tests. Possible strategies include assigning a dollar value to NEBs and either subtracting this value from participant costs or adding this value to program benefits. A third option is to discount total costs by the percentage that participants are paying for NEBs in order to isolate the true cost of energy savings. This approach may be less rigorous than attempts to establish specific NEB values, but it could provide a useful solution and minimize the time and expense of data collection. Each of these methodologies has advantages and drawbacks, and additional data is needed to aid a thorough evaluation of these approaches.

iv


1. INTRODUCTION A growing body of anecdotal evidence suggests that many consumers purchase whole-house retrofit services for the associated non-energy benefits (NEBs) including improved comfort, aesthetic enhancements, and better indoor air quality, among others. Limited research conducted with consumers that have purchased these services lends support to the anecdotal reports and provides initial estimates of the value consumers place on the energy and non-energy benefits of whole-house retrofit services. Despite the value of NEBs to consumers, the cost-effectiveness tests used by many regulators to evaluate whole-house initiatives do not include the value of NEBs. This undervalues the benefits of these programs, making it harder for them to meet a target cost-effectiveness threshold. Many regulatory agencies have been reluctant to use cost-effectiveness tests that incorporate NEBs because of the lack of consensus on methodologies for establishing NEB value and incorporating those values into the tests, and the lack of data on NEB values to program participants. With support from the U.S. Environmental Protection Agency and the New York State Energy Research and Development Authority (NYSERDA), ACEEE is conducting a three-phase project to address the needs for better information and valuation of the non-energy benefits associated with residential retrofits.1 Of particular interest are whole-house retrofit programs such as Home Performance with ENERGY STAR® (HPwES) and similar independent programs. Through the project, ACEEE will explore the feasibility of incorporating (or deducting) comfort and other NEBs and their value to consumers into cost-benefit methods commonly used by utilities and regulators. As a starting point for the project, this report presents a review of the literature on cost-benefit tests and non-energy benefits from residential retrofits. It also summarizes discussions with individuals who have grappled with the question of how to treat home retrofit NEBs in cost-benefit analyses. The report concentrates on three broad areas: identifying NEBs, the valuation methodologies used, and results of the valuations; review of common cost-effectiveness tests and their treatment of NEBs; and issues for development of a proposed methodology for including NEBs in cost-effectiveness tests. Specifically, Section 2 provides an overview of the studies included in the review, Section 3 summarizes several cost-effectiveness tests including those most commonly used, and Section 4 describes typical NEBs from whole-house retrofits and methods for valuing these NEBs. Section 5 discusses key elements of a proposed methodology for incorporating NEBs as well as data needs and other issues for consideration and Section 6 offers a summary and preliminary conclusions. In the second phase of the project, ACEEE will survey participants in the NYSERDA HPwES program about the NEBs realized in their home retrofits and the value they place on the NEBs. Survey design and methodology will draw on the findings of this literature review and advice from an advisory group of program managers, researchers, and others with expertise in this area. Survey data will be analyzed in the third phase of the project and used to help develop recommendations for improved cost-effectiveness tests that recognize the value and costs of NEBs to consumers. ACEEE will convene a workshop of interested parties from the energy efficiency, utility, and regulatory communities to review the results 1 For the purposes of this study, “non-energy benefits” refers to those that are experienced by the individual participant, as opposed to broad-scale environmental benefits and other externalities.

1


of the project and develop a strategy for working with regulators to encourage adoption of the recommended cost-effectiveness tests. 2. STUDIES INCLUDED IN THE LITERATURE REVIEW This review draws from the literature on cost-benefit tests used by regulatory agencies and, to a lesser extent, program evaluators to determine the cost-effectiveness of energy efficiency programs; surveys of the NEBs common to whole-house retrofits; and valuation studies conducted for whole-house retrofit and weatherization programs. The list of sources was developed through discussions with researchers and program managers and a search of the literature from relevant organizations and conferences. The studies reviewed are listed in Table 1. The literature on weatherization programs provides useful insights on the types of NEBs expected in whole-house retrofit programs that address many of the same building envelope and equipment measures, as well as practical examples of the valuation methodologies of interest. The data on weatherization programs is particularly helpful since whole-house retrofit programs for middle and upper-income households have a relatively short history,2 the number of programs in operation remains small, and several programs have a limited scope. As these programs mature and the number of programs grows, additional data will be available. Table 1: Studies Included in the Review

Cost-Effectiveness Tests Reference Full Citation

Swisher, de Martino Jannuzzi & Redlinger (1997)

Swisher, Joel N., Gilberto de Martino Jannuzzi, and Robert Y. Redlinger. 1997. Tools and Methods for Integrated Resource Planning: Improving Energy Efficiency and Protecting the Environment. November. Prepared for the UNEP Collaborating Centre on Energy and Environment. Roskilde, Denmark: RISØ National Laboratory.

CPUC (2001) [CPUC] California Public Utilities Commission. 2001. California Standard Practice Manual: Economic Analysis of Demand-Side Programs and Projects. October. Sacramento, Calif.: California Public Utilities Commission.

TecMarket Works, Inc. (2001)

TecMarket Works, Inc. 2001. The Low-Income Public Purpose Test (LIPPT): Updated for Version 2.0. May 25. Prepared for the RRM Working Group Cost Effectiveness Committee with Skumatz Economic Research Associates, Inc. and Megdal and Associates. Oregon, Wisc.: TecMarket Works, Inc.

Wisconsin Division of Energy (2003)

Wisconsin Division of Energy. 2003. Focus on Energy Statewide Evaluation: Initial Cost-Benefit Analysis. Final Report. March 31. Madison, Wisc.: Wisconsin Department of Administration, Division of Energy.

2 There were quite a few programs in the 1980s and early 1990s, but NEBs were rarely assessed. Furthermore, whole house retrofit techniques have improved since then, making data on these old programs not particularly relevant for evaluating today’s programs.

2


Identification of NEBs, Valuation Methodologies and Results Reference Full Citation

Skumatz & Dickerson (1998)

Skumatz, Lisa and Chris Ann Dickerson. 1998. “Extra! Extra! Non-Energy Benefits of Residential Programs Swamp Load Impacts!” In Proceedings of the 1998 ACEEE Summer Study on Energy Efficiency in Buildings, 8.301–8.312. Washington, D.C.: American Council for an Energy-Efficient Economy.

Skumatz, Dickerson & Coates (2000)

Skumatz, Lisa, Chris Ann Dickerson, and Brian Coates. 2000. “Non-Energy Benefits in the Residential and Non-Residential Sectors: Innovative Measurements and Results for Participant Benefits.” In Proceedings of the 2000 ACEEE Summer Study on Energy Efficiency in Buildings, 8.353–8.364. Washington, D.C.: American Council for an Energy-Efficient Economy.

Skumatz (2001)

Skumatz, Lisa. 2001. “Non-Energy Benefits (NEBs)—A Comprehensive Analysis and Modeling of NEBs for Commercial & Residential Programs.” In AESP 12th National Energy Services Conference Proceedings, 459–471. Jupiter, Fla.: Association of Energy Services Professionals International.

Skumatz (2002)

Skumatz, Lisa. 2002. “Comparing Participant Valuation Results Using Three Advance Survey Measurement Techniques: New Non-Energy Benefits (NEB) Computations of Participant Value.” In Proceedings of the 2002 ACEEE Summer Study on Energy Efficiency in Buildings, 8.307–8.320. Washington, D.C.: American Council for an Energy-Efficient Economy.

Fuchs, Skumatz & Ellefsen (2004)

Fuchs, Leah, Lisa Skumatz, and Jennifer Ellefsen. 2004. “Non-Energy Benefits (NEBs) from ENERGY STAR: Comprehensive Analysis of Appliance, Outreach, and Homes Programs.” In Proceedings of the 2004 ACEEE Summer Study on Energy Efficiency in Buildings, 2.79–2.89. Washington, D.C.: American Council for an Energy-Efficient Economy.

Lutzenhiser Associates (2004)

Lutzenhiser Associates. 2004. Final Evaluation Report: California Building Performance Contractors Association Comprehensive Whole House Residential Retrofit Program. CPUC-172-02. August 15. Portland, Oreg.: Lutzenhiser Associates.

NYSERDA (2005)

[NYSERDA] New York State Energy & Research Development Authority. 2005. New York Energy $mart Program Evaluation and Status Report: Report to the System Benefits Charge Advisory Group. May. Albany, N.Y.: New York State Energy & Research Development Authority.

3. COST-EFFECTIVENESS TESTS A review of the literature reveals a number of cost-benefit tests used by utilities and regulatory agencies to determine program cost-effectiveness. The discussion below draws largely from the definition of tests in the California Standard Practice Manual (CPUC 2001) and the discussion of these tests found in Swisher, de Martino Jannuzzi, and Redlinger (1997) with other sources cited as used. For the purposes of this review, the tests have been split into two categories: those that do not include NEBs and those that do. Table 2 summarizes the costs and benefits included in each test and which states use the test. A full state-by-state list of the cost-effectiveness tests used is included in the appendix. Review of current state practices shows that the most widely used tests today do not include NEBs. Tests That Do NOT Incorporate NEBs

Utility Cost Test (or Program Administrator Test): Focused on program costs and energy benefits to the utility or other program administrator, the Utility Cost Test compares the costs of the program to the utility or other program administrator (e.g., incentives and administrative costs) to the benefits accrued to the utility (e.g., avoided costs of fuel,

3


operations, and capacity). Results of the tests may be expressed as a net present value, cost-benefit ratio, or levelized cost. The Utility Cost Test is used in Oregon, Connecticut, Washington, Illinois, New York, and the Bonneville Power Authority territory, among others. In many cases, the Utility Cost Test is used in conjunction with one or more other tests.

Table 2: Summary of Cost-Benefit Tests

Test Benefits Costs States Using

Utility Cost (Program Administrator)

• Avoided supply costs for transmission, distribution, and generation (TD&G)

• Avoided gas and water supply costs

• Program administration. • Participant incentives • Increased supply cost

CA, CT, HI, IA, IL, IN, MI, MN, MO, NY, OR, RI, TX, VA, WA, BPA

Ratepayer Impact Measure (RIM)

Same as above, plus • Increased revenue

Same as above, plus • Decreased revenue

AR, CO, FL, GA, HI, IA, IN, MI, MN, NC, ND, NV, SC, VA, WI

Participant Cost • Utility bill reductions • Participant incentives • Participant direct costs

AR, CA, FL, HI, IA, IN, MI, MN, NY, VA

Total Resource Cost

• Avoided supply costs for TD&G

• Avoided gas and water supply costs

• Utility bill reductions

• Program administration • Participant incentives • Participant direct costs • Increased supply costs • Decreased revenue

AR, CA, CT, CO, GA, HI, IA, ID, IN, MA, ME, MI, MO, MT, NH, NJ, NV, NY, RI, SC, UT, VA, WA

Societal

Same as above, plus • Externality benefits (reduced

pollution, improved reliability, etc.)

Same as above

AZ, IA, ME, MN, MO, MT, NJ, OR, VT, WI

Public Purpose (includes NEBs)

Same as above, plus • Participant incentives • Quantifiable participant

NEBs

Same as above

CA, KY, WI (low-income)

Total Market Effects (includes NEBs)

Same as above plus additional participant NEBs (for program and spillover participants) and broader macroeconomic effects

Same as above

For evaluation purposes only

Program Efficiency (includes NEBs) Same as above Same as above, excluding

participant direct costs For evaluation purposes only

Initial BCA (Simple BC) (includes NEBs)

Same as Public Purpose Test, plus participant direct costs (as negative benefit)

Same as above For evaluation purposes only

Ratepayer Impact Measure Test (RIM; also known as the “No Losers Test”): Designed to

measure the impact of a program on customer rates, the RIM test gives an indication of the direction and magnitude of rate changes expected in response to an efficiency program. Benefits included in RIM are avoided supply costs from reductions in transmission, distribution, generation, and capacity costs for periods of reduced load as well as increased revenues for periods of increased load. On the cost side, RIM includes the cost of the program (incentives and administrative costs) to the utility or other entities as well as decreased revenues for periods of decreased loads and increased supply costs for periods of increased load. The RIM is generally the most restrictive of the tests in use

4


as it tends to favor load management programs and penalize energy-saving programs. Results of the RIM test may be expressed as the lifecycle revenue impact (in dollars or cents) per kWh, kW, therm, or customer; annual or first-year revenue impacts per kWh, kW, therm, or customer; a cost-benefit ratio; or a net present value. Florida, Georgia, and North Carolina have used the RIM test since the mid-1990s. In 2003, Colorado switched from the Total Resource Cost test to the RIM test.

Participant Cost Test: Focused on the costs and benefits of the program to program

participants, this test compares participant costs (including initial equipment costs and ongoing operations and maintenance) to benefits. Non-energy benefits are not included in the Participant Test; benefits are limited to energy bill savings, incentives (from the utility or other entity), and tax credits. Gross energy savings are included in the benefits at the customer’s actual retail rates. Results are expressed as net present value per average participant, net present value for the total program, a cost-benefit ratio, or discounted payback. New York and California use the Participant Cost Test in conjunction with other tests to determine program cost-effectiveness.

Total Resource Cost Test (TRC): Intended to assess the overall program costs and

benefits to the utility, participants, and society at large, the TRC compares total program costs (utility incentive and administrative costs and participant costs) to program benefits in terms of avoided supply costs (based on net program energy savings). Tax credits are considered a reduction to costs under the TRC. Results of the TRC are expressed as a net present value, cost-benefit ratio, or levelized cost. The TRC is the most widely used cost-effectiveness test and is currently used by California, Connecticut, Massachusetts, New Jersey, Washington, New York, Wisconsin, and other states.

Societal Test: A variation on the TRC, the Societal Test expands the TRC to include the

effects of externalities (e.g., environmental, national security, etc.) as program costs, excludes tax credits from the program benefits, and uses a societal discount rate rather than market discount rates. Like the TRC, results may be expressed as a net present value, cost-benefit ratio, or levelized cost. Oregon, Vermont, Minnesota, Maine, and Wisconsin all use some variation of the Societal Test alone or in conjunction with other tests.

Tests that DO Incorporate NEBs

Public Purpose Test (PPT): Like the TRC, the PPT is intended to assess program costs and benefits to utilities, participants, and all parts of society. The PPT seeks to incorporate a broad range of non-energy benefits, but in reality their inclusion has been limited by difficulties in establishing a methodology for valuing non-energy benefits that is acceptable to all parties involved. California has developed a variation on the PPT specifically for low-income energy efficiency programs. The LIPPT is designed to incorporate as many NEBs as possible from the utility, societal, and participant perspectives including those that have been excluded from the standard PPT because of the challenges in determining agreed-upon values. In an initial test of the LIPPT methodology using data from an imaginary program, the inclusion of NEBs boosted the program cost-benefit ratio from 0.7 to 1.5 (TecMarket Works 2001).

5


Total Market Effects Test (TMET): Unlike most of the other tests described here, the TMET compares program benefits (electricity and fuel savings and, optionally, NEBs) for both program participants and spillover participants to the costs incurred by the program implementer and program participants. Because it includes the benefits to spillover participants, the TMET is particularly useful for assessing the cost-effectiveness of market transformation programs. In the case of home performance programs, spillover is a key component of program design as contractors are expected to apply the techniques and tools they acquire through the program to projects outside of the program. Results of the TMET are reported as a cost-benefit ratio.

Program-Efficiency Test (PET): This test compares the same benefits as the TMET, but

only includes costs incurred by the program implementer. Essentially, it is an expanded version of the Utility Cost Test. The PET is intended to help program administrators determine the cost-effectiveness of the program to their own organization. By excluding participant costs, the PET may yield more realistic estimates of cost-effectiveness for programs where participants are investing significant sums of their own for a broad range of benefits beyond energy savings. Results of the PET are expressed as a cost-benefit ratio—a value greater than 1.0 indicates that program benefits exceed the administrator’s costs.

Initial Cost-benefit Analysis: The Initial BCA has been used to assess the cost-

effectiveness of the portfolio of programs operated by Wisconsin Focus on Energy (Wisconsin Division of Energy 2003). The Initial BCA Analysis uses the “simple BCA test,” a modification of the Public Purpose Test, to compare program costs and benefits. The simple BCA test includes all customer costs as negative benefits, keeping customer costs and benefits on the same side of the equation so as not to confuse them with program spending, which is included on the cost side of the comparison. The Initial BCA Analysis also uses the “economic development BCA test,” which includes the same costs and benefits as the simple BCA test, but also models the program’s effect on the Wisconsin economy.

It is common for program administrators and regulatory agencies to consider the results of multiple tests when evaluating program cost-effectiveness. Looking at multiple tests can help determine how the program affects different stakeholders. For example, California uses the TRC and the Participant Test to gauge the cost-effectiveness of the California Building Performance Contractors Association (CBPCA) program. The TRC is used to determine the cost-benefit ratio of the program and the Participants Test is used to determine the program’s attractiveness to customers. 4. NON-ENERGY BENEFITS FROM WHOLE-HOUSE RETROFITS: IDENTIFYING NEBS, VALUATION METHODOLOGIES, AND RESULTS A broad range of NEBs from home retrofit projects have been identified in studies over the past 20 years. In a 1983 study of consumer decision-making in California, in-depth interviews demonstrated that comfort and aesthetic benefits far outweighed energy concerns for most homeowners and that very few homeowners took time to assess the economic

6


benefits of their investments by monitoring energy bills or calculating payback times for their home retrofit projects (Wilhite 1994). Since that time, researchers have catalogued the array of NEBs associated with residential retrofit projects, including benefits identified directly by the researchers themselves and those identified through surveys of retrofit customers, contractors, utilities, and other interested parties. Many of the studies identified for this literature review deal with low-income weatherization programs, which have a long history of providing comprehensive retrofit services to residential customers. While these studies identify some NEBs that are common to residential programs regardless of participant income, many of the benefits of interest in these studies are not relevant for other retrofit programs.3 For those benefits common to both types of programs, valuation of NEBs may be quite different—these issues are discussed in greater detail below. Fortunately, data are now becoming available from newer whole-house retrofit programs, such as the HPwES programs operating in New York, Wisconsin, California, and other areas. Identification of Non-Energy Benefits The broad range of benefits from home retrofits have been identified and classified according to the perspective of their chief beneficiary: the utility, society, or program participants (TecMarket Works 2001, Skumatz 2002).4 Various cost-effectiveness tests take into account each of these perspectives in different ways. Table 3 lists some of the benefits identified from the utility and the societal perspectives. For the most part, economic data and other mechanisms exist for quantifying utility and societal NEBs. According to Skumatz (2001), estimated value of NEBs from the utility perspective range from 10% to 50% of annual household energy bill savings—savings for non-low income programs are at the lower end of the range and values for all programs increase if gas measures are included; from the societal perspective, NEB values range from 100% to 300% of annual household bill savings. Table 3: Utility and Society Non-Energy Benefits

Utility Perspective Societal Perspective Transmission and/or distribution savings Peak load reductions

Economic impacts (job creation, tax revenue) Improved housing stock/preservation Emissions/environmental impacts Health and safety benefits Water and wastewater savings

The remainder of this section will focus on the identification and valuation of benefits from the program participant perspective. As noted above, studies of residential retrofit programs (including low-income weatherization, HPwES, and other home retrofit programs) have identified a diverse group of benefits beyond energy savings. The data and methodologies for

3 For purposes of this report, benefits that are relevant to low-income programs but not to other residential programs are excluded from the discussion. The list includes benefits to utilities and participants of fewer billing arrearages; fewer service shutoffs, reconnects, and notices; and reduced moving costs. See TecMarket Works (2001) for a detailed discussion. 4 The TecMarket Works (2001) and Skumatz (2002) work included review of more than 300 relevant studies and papers dealing with non-energy benefits, valuation methodology, etc.

7


quantifying participant NEBs are less well developed, as discussed below, but have been estimated at 50% to 300% of annual household energy bill savings (Skumatz 2001). Table 4 summarizes participant NEBs identified in the literature and grouped into categories by ACEEE. Table 4: Program Participant Non-Energy Benefits

General Benefit Specifics/Examples Financial benefits (other than energy cost savings)

Program incentives: rebates, low interest financing, subsidized home assessment/diagnosis Water and wastewater bill savings Reduced equipment repair and maintenance Increased home resale value Improved home durability

Comfort benefits Improved airflow Reduced drafts and temperature swings Better humidity control

Aesthetic benefits More attractive windows, appliances, etc. Less dust Reduced/eliminated mold and/or water damage Protection of furnishings Dimmable lighting

Health and safety benefits Improved respiratory health Reduced allergic reactions Lower fire/accident risk (from gas equipment)

Noise reduction benefits Quieter HVAC and other equipment Less external noise intrusion

Education-related benefits Reduced transaction costs (knowing what to look for when purchasing equipment; ease in locating appropriate products) Persistence of savings Greater understanding of home operation

Convenience benefits Automatic thermostat controls Easier filter changes Faster hot water delivery Less dusting and vacuuming

Other benefits Greater control over energy use/energy bill Reduced sick days from school and work Ease of selling home Enhanced pride/prestige Improved sense of environmental responsibility Enhanced peace of mind/responsibility for family well-being

Sources: TecMarket Works 2001; Skumatz 2002; Fuchs, Skumatz, and Ellefsen 2004; Knight 2005a

Valuation of Non-Energy Benefits While identification and categorization of NEBs have been the subject of numerous papers, efforts to establish robust estimates of the value of NEBs to program participants are limited. Again, much of the data available comes from studies of low-income weatherization programs. This data offers useful information on NEB valuation and insights regarding various valuation methodologies, but data specific to whole-house retrofit programs that target a broader audience are needed. It is likely that participants in these programs will value NEBs quite differently given differences in income and motivations for seeking home performance upgrades as well as the type of retrofit activities performed and the cost of those

8


services to the participant. Methods for calculating the monetary value of benefits must also be adjusted—for example, the value of lost work time (e.g., sick days) is computed using minimum wage rates for low-income households and would significantly undervalue the lost time for participants earning higher wages. The remainder of this section describes the methods that have been used in participant NEB valuation studies and summarizes the results of studies conducted to date. Valuation Methods NEB studies have employed different methods of estimating the value of NEBs to residential program participants, each with their own advantages and disadvantages. These methods include:

Computational methods: Estimates of value can be calculated for some benefits. For example, reduced transaction costs and lost work time can be calculated using wage rates or other monetary values for time. Increases in property value can be derived from the annual reductions in fuel costs and the cost of structural improvements or other repairs made, and water/sewer savings can be calculated using data on actual water and sewer rates. Unfortunately, there are no estimates or proxy values available for less tangible participant benefits including comfort, aesthetics, convenience, etc. and these values may vary widely among individual participants or groups of participants.

Participant surveys:. A variety of valuation methods have been used by economists,

social scientists, and others to establish the monetary value of intangible goods. These methods have been broadly applied in the fields of health care, natural resources, and product marketing. Methods applied to the valuation of NEBs from residential retrofits include:

o Willingness to pay (WTP) surveys: WTP surveys are widely used in the environment

and natural resources fields to estimate the value of intangible or hard-to-measure benefits, such as recreation benefits, parks and green space, and pollution abatement. Survey respondents are asked the dollar amount that they would be willing to pay for each specific benefit from a given program and for the full range of benefits from the program. An advantage of WTP surveys is that they provide specific dollar values for benefits that can be compared to each other and to the value given for the comprehensive set of program benefits. Disadvantages include the difficulty that many respondents have in answering the questions, the volatility of the responses, and significant variations in responses based on socioeconomic, demographic, and attitudinal variables.

o Comparative valuations (CV): In this approach (also called relative valuation),

researchers ask respondents to compare the value of each NEB relative to the energy savings from the program. First, respondents are asked whether the benefit has higher or lower value than the energy savings, then they are asked how much higher or lower the value. The value of NEBs is expressed as a percentage of energy savings. Using program data on energy savings, a dollar value for the NEBs can be calculated.

9


Skumatz developed this approach for use in studies of residential appliance and low-income weatherization programs (Skumatz and Dickerson 1998; Skumatz, Dickerson and Coates 2000) and has since applied it in studies of ENERGY STAR home performance, new homes, and appliance programs (Fuchs, Skumatz, and Ellefsen 2004). In these studies, respondents found the CV questions much easier to answer than WTP questions and the responses were more consistent than those from WTP surveys.

o Labeled Magnitude Scaling Approach (LMS): This method was originally developed

to gauge differences in the intensity of taste and pain sensations. Skumatz (2002) adapted this approach for a study of the Northeast Utilities Weatherization Residential Assistance Partnership program. Respondents were asked to use specifically worded scale responses (much more valuable, more valuable, similar value, less valuable, and much less valuable) to rate the importance of program NEBs relative to a numerical ranking of energy savings benefits. The responses were assigned numeric valuations and translated into dollar amounts. This approach yielded similar results to comparative valuation. This method was also used in conjunction with CV in the study of NEBs in the ENERGY STAR programs cited above (Fuchs, Skumatz, and Ellefsen 2004).

o Reported Motivations and Factor-Importance Judgments: Customer-reported

motivations for pursuing home performance projects and the relative weighting of those motivations can also be used to determine the value of the energy and non-energy benefits resulting from the project. In an evaluation of the first phase of the California Building Performance Contractors Association program (CBPCA),5 customers were asked about their motivations for buying comprehensive home performance retrofits. Customers reported multiple motivations among these six categories (in order of frequency): specific system/building concern; environmental health and energy costs (tied); comfort; resource conservation; and other (Lutzenhiser Associates 2004). Another survey is under development for evaluation of the second phase of the program and a pre-test pilot survey has been conducted. In this survey, reported motivations will be weighted by importance. This relative weighting could be used to determine the relative portion of project costs paid for energy and non-energy benefits.

Statistical Analysis of Revealed Preferences (Revealed Willingness to Pay): A

combination of program data and survey results can be used to derive estimates of NEB value using a revealed preferences model. Revealed preferences (or revealed WTP) models are used to determine how reported intent translates into action. Such a model would incorporate data on the measures installed through a home performance program, the cost of the installed measures, the NEBs reported by participants, and the value of those NEBs as determined through a CV survey to derive estimates of the actual costs participants paid for the energy and NEBs associated with common projects or measures (Carroll 2005). While this approach has not been applied to home performance retrofits, it has been used in other fields and could prove useful for providing more robust results

5 The CBPCA program is implementing Home Performance with ENERGY STAR.

10


than CV studies alone. One drawback of this approach is the time and expense associated with data collection and analysis.

Skumatz (2002) compared the NEB value estimates derived using WTP, CV, and/or LMS methods in three studies of low-income programs. One study utilized all three methods, another used two methods (WTP and CV), and the third used only CV. Data from these studies allowed for comparison of the results from different methods for the same program as well as comparison of results for the same method across different programs. Findings demonstrate that CV and LMS give much more consistent results than WTP and the questions are generally easier for participants to answer. These methods are also more conservative than the WTP approach, which yielded estimated NEB values four to ten times greater than CV and LMS. More recently, an approach using a combination of CV and LMS was used in a study of participant NEBs from the New York HPwES (Fuchs, Skumatz, and Ellefsen 2004). The results of this study imply that it would be useful to conduct multi-method studies with participants of whole-house retrofit programs. Such studies would yield more robust and credible results while providing useful insight into the most reliable valuation techniques. If resources do not allow the use of multiple methods, findings suggest that CV and LMS are easier for respondents and yield more reasonable value estimates. Valuation Study Results

This section summarizes the results of NEB valuation estimates developed for weatherization (Table 5) and HPwES (Table 6) programs. Table 5: NEB Valuation Estimates for Weatherization Programs

Program Name

Sample Size

Valuation Method

NEB Multiplier

Participant Value

($/year)

Northeast Utilities Weatherization Residential Assistance Partnership 425

WTP CV

LMS

2–10 80%–100%

0.989

$220 to $1000 $65 to $110 $70 to $110

CA Utilities Low Income Weatherization Program 321 WTP

CV 4.25** 98%

$170 $33 to $47

PGE Venture Partners Pilot Program* 150 CV 60% $50 * The VPP program was a low-income weatherization program. The sample of 150 also included interviews with participants in appliance rebate and multi-family lighting programs, but the majority of interviews were with VPP participants. The results reported here are for weatherization only. ** This value represents the WTP multiplier and reported WTP dollar value for the total of all the reported NEBs. When the values given for each individual NEB are summed, the total value is $687 (an implied multiplier of 17). The total value of the package of NEBs as reported by participants is 25% of the sum of values reported for each individual benefit. Sources: Skumatz (2002); TecMarket Works (2001)

11


Table 6: NEB Valuation Estimates for Home Performance with ENERGY STAR Programs

Program Sample Size Valuation Method NEB Multiplier

Participant Value

($/year) New York 81 Combined CV and LMS 1.0 $477

Wisconsin 169 Calculated or derived from other studies N/A $400

Sources: NYSERDA (2005); Wisconsin Division of Energy (2003) Beyond attempts to quantify the value of NEBs, it is useful to understand the NEBs that participants consider most important and the primary motivators for customer investment in home retrofits. New York HPwES participants revealed that the most important NEBs associated with their home retrofits were environmental benefits, personal satisfaction, and comfort; other important benefits included ease of selling home, improved ability to stay in the home, and equipment performance (NYSERDA 2005). In California, the recent pre-test of the 2005 participant survey showed respondents weight resource conservation and improved comfort among the most important motivations for purchasing home performance retrofits, followed by reduced energy bills, home air quality, and health issues (Knight 2005b). Data from the small number of studies summarized in Tables 3 and 4 show a substantial difference in the value of NEBs from low-income weatherization and HPwES programs. Additional studies are needed to see if these results are consistent or can be replicated. Given the extent of the retrofits and type of measures installed through the HPwES programs and the amount of the customer investment (averaging more than $7,000 per project in New York and $15,000 per project in California), greater energy savings and higher NEB values are expected for HPwES programs. Further research and data is needed to test this hypothesis. 5. PROPOSED METHODOLOGY The results of this literature review will be used to guide development of alternative cost-benefit approaches—modifications to existing cost-benefit tests or new methods—for home performance programs.6 This section explores some preliminary ideas that would result in more realistic assessment of program cost-effectiveness. How Can NEBs Best Be Addressed? Results of this literature review point to several options for incorporating NEBs into cost-benefit tests. Possible strategies include assigning a dollar value to NEBs and either subtracting this value from participant costs or adding this value to program benefits. A third option is to discount the percentage of total costs that participants are paying for NEBs in order to isolate the true cost of energy savings.

6 Although the focus here is on home retrofits, the proposed methods may also be relevant for other programs where participants seek and receive extensive NEBs.

12


Since NEBs are not of specific interest to program administrators and regulators (beyond their role in program marketing and participant decision-making), it can be argued that the value of the NEBs should be taken out of the cost-benefit equation for efficiency programs. This could be accomplished by subtracting the value of the NEBs from the participant costs. A significant drawback of this approach is the difficulty in establishing a specific dollar value for NEBs. Each of the valuation methodologies discussed earlier has its advantages and disadvantages when applied to cost-benefit tests; values vary depending on the participant and specifics of the project, and it is hard to determine the value of NEBs across all participating projects. The flip-side of this approach is to add the value of the NEBs to the benefit side of the cost-benefit equation to recognize their value as program benefits to participants. This approach faces the same challenges as subtracting the NEBs from participant costs. Furthermore, it seems more appropriate to take NEB values out of the deliberations of energy efficiency programs than to add in values for goods that are outside the scope of the regulatory agencies mandate and which yield benefits to individual consumers rather than broader societal benefits. Another approach is to discount participant costs to reflect the true cost of energy benefits by themselves. Using survey data to determine the average value of NEBs relative to energy savings, participant costs could be reduced by a percentage that would reflect the NEB value. For example, if participants value NEBs equally to energy savings, participant costs included in the cost-benefit test would be half of the overall project costs. This approach may be less rigorous than attempts to establish specific NEB values, but it could provide a useful short-term solution and minimize the time and expense of data collection and extensive participant surveys. Which Test Is Appropriate? Not only is it important to consider credible ways to quantify the value of NEBs from residential retrofits and how these values can be incorporated into existing or modified cost-benefit tests, we must also consider whether the appropriate tests are being used to measure the cost-effectiveness of home performance programs. The TRC, Participant Test, and Societal Test include all participant direct costs without considering the value of NEBs to these participants or the broader market effects of these programs. Market effects, including spillover, are a critical component of market transformation program design. These programs are designed to alter market infrastructure to increase adoption of energy efficiency technologies and services by a larger segment of the population than could be served in the program. Program implementers are interested in including the value of energy and non-energy benefits enjoyed by non-participants as a result of program activity in cost-benefit tests evaluating their programs. While TMET and PET attempt to include these benefits, there are many difficulties in estimating spillover effects for these programs and these tests have not been adopted outside of the efficiency program evaluation community.

13


Other Issues At the same time that we are examining the treatment of NEBs in cost-benefit tests and the appropriate tests for home performance programs, it makes sense to take a look at other issues that affect the results of the tests. For example, many states use market discount rates rather than lower societal rates in calculating lifecycle costs and benefits for efficiency programs. These higher discount rates set a higher hurdle for cost-effective investments with real benefits to participants and society at large. Another issue is the cost projections used in the cost-benefit calculations. If projected fuel costs are outdated or unreasonably low, efficiency investments will not compare favorably. These and other issues should be explored as part of any effort to encourage the regulatory community to adopt new cost-benefit methods. Data Needs In order to further flesh out modified cost-benefit methodologies, additional research is needed on the value of NEBs to home performance program participants and the types of cost-benefit tests used by regulatory agencies and their views of these tests. More information on how the tests are applied (including assumptions for discount rates and future fuel costs) would also be useful. Modeling of the various modifications proposed to the tests using NEB values obtained from studies cited here and new data to be collected in New York State would also be needed. TecMarket Works, Inc.7 and Energy and Environmental Economics, Inc.8 have developed spreadsheet tools that could be useful for this purpose. 6. SUMMARY AND CONCLUSIONS Our review of the literature suggests options for including the value that consumers place on NEBs into cost-effectiveness tests for whole-house retrofit initiatives; however, no clear methodology emerges from the studies included in our review. Each of the methodologies proposed above have their advantages and drawbacks. Additional data will aid an evaluation of these approaches. For example, the results of broader valuation studies can be used to test the proposed methodologies to see if a discounting approach (whereby participant costs are reduced by a set percentage that reflects the value consumers place on NEBs) yields robust results that can substitute for the more detailed and extensive data collection required to establish specific NEB values for each program under evaluation. ACEEE has assembled an advisory panel to appraise this literature review, suggest the best design for further data collection efforts, and work with us to develop a proper methodology for incorporating NEB values in cost-effectiveness tests. The results of the advisory panel review will support more concrete conclusions and a plan for moving forward with further research toward final recommendations.

7 TecMarket Works, Inc. Web site: http://www.tecmarket.net/index.html8 Energy and Environmental Economics, Inc. Web site: http://www.ethree.com

14

http://www.tecmarket.net/index.html

http://www.ethree.com/


FULL REFERENCE LIST (INCLUDING SUPPLEMENTARY REFERENCES) [California] State of California. 2004. Draft Energy Efficiency Policy Rules, Terms and

Definitions for Post-2005 Programs. R.01-08-028 MEG/tcg 12/30/04. Sacramento, Calif.: State of California.

Carroll, David. 2005. “Rethinking Cost-Benefit Tests: Practical Examples.” Presentation to

2005 Affordable Comfort Conference. May 17. Indianapolis. [CPUC] California Public Utilities Commission. 2001. California Standard Practice

Manual: Economic Analysis of Demand-Side Programs and Projects. October. Sacramento, Calif.: California Public Utilities Commission.

Fuchs, Leah., Lisa Skumatz, and Jennifer Ellefsen. 2004. “Non-Energy Benefits (NEBs)

from ENERGY STAR: Comprehensive Analysis of Appliance, Outreach, and Homes Programs.” In Proceedings of the 2004 ACEEE Summer Study on Energy Efficiency in Buildings, 2.79–2.89. Washington, D.C.: American Council for an Energy-Efficient Economy.

GDS & Associates. 2004. Summary of Benefit/Cost Tests for Energy Efficiency Programs.

Marietta, Ga.: GDS Associates. Gillmore, Bill (North Carolina Utilities Commission). 2006. Personal communication.

February 6. Knight, Robert (Bevilacqua-Knight, Inc.). 2005a. Personal communication with J.T. Amann.

March. ———. 2005b. “Evaluation Issues for Home Performance Programs.” Presentation to the

2005 Affordable Comfort Conference. May 17. Indianapolis, Ind. Lutzenhiser Associates. 2004. Final Evaluation Report: California Building Performance

Contractors Association Comprehensive Whole House Residential Retrofit Program. CPUC-172-02. August 15. Portland, Oreg.: Lutzenhiser Associates.

[NYSERDA] New York State Energy & Research Development Authority. 2005. New York

Energy $mart Program Evaluation and Status Report: Report to the System Benefits Charge Advisory Group. May. Albany, N.Y.: New York State Energy & Research Development Authority.

[RAP] The Regulatory Assistance Project. 2006. The Energy Efficiency Policy Toolkit.

http://www.raponline.org. Montpelier, Vt.: The Regulatory Assistance Project. Skumatz, Lisa. 2001. “Non-Energy Benefits (NEBs)—A Comprehensive Analysis and

Modeling of NEBs for Commercial & Residential Programs.” In AESP 12th National

15

http://www.raponline.org/


Energy Services Conference Proceedings. 459–471. Jupiter, Fla.: Association of Energy Services Professionals International.

———. 2002. “Comparing Participant Valuation Results Using Three Advance Survey

Measurement Techniques: New Non-Energy Benefits (NEB) Computations of Participant Value.” In Proceedings of the 2002 ACEEE Summer Study on Energy Efficiency in Buildings, 8.307–8.320. Washington, D.C.: American Council for an Energy-Efficient Economy.

Skumatz, Lisa and Chris Ann Dickerson. 1998. “Extra! Extra! Non-Energy Benefits of Residential Programs Swamp Load Impacts!” In Proceedings of the 1998 ACEEE Summer Study on Energy Efficiency in Buildings, 8.301–8.312. Washington, D.C.: American Council for an Energy-Efficient Economy.

Skumatz, Lisa, Chris Ann Dickerson, and Brian Coates. 2000. “Non-Energy Benefits in the

Residential and Non-Residential Sectors: Innovative Measurements and Results for Participant Benefits.” In Proceedings of the 2000 ACEEE Summer Study on Energy Efficiency in Buildings, 8.353–8.364. Washington, D.C.: American Council for an Energy-Efficient Economy.

Swisher, Joel N., Gilberto de Martino Jannuzzi, and Robert Y. Redlinger. 1997. Tools and

Methods for Integrated Resource Planning: Improving Energy Efficiency and Protecting the Environment. November. Prepared for the UNEP Collaborating Centre on Energy and Environment. Roskilde, Denmark: RISØ National Laboratory.

TecMarket Works, Inc. 2001. The Low-Income Public Purpose Test (LIPPT): Updated for

Version 2.0. May 25. Prepared for the RRM Working Group Cost Effectiveness Committee with Skumatz Economic Research Associates, Inc. and Megdal and Associates. Oregon, Wisc.: TecMarket Works, Inc.

Titus, E., M. Nevius, and J. Michals. 2004. “How Do We Measure Market Effects? Counting

the Ways, and Why It Matters.” In Proceedings of the 2004 ACEEE Summer Study on Energy Efficiency in Buildings, 6.117–6.128. Washington, D.C.: American Council for an Energy-Efficient Economy.

Wilhite, H. 1994. “Market Signals Fall Short as Policy Instruments to Encourage Energy

Savings in the Home.” In Proceedings of the 1994 ACEEE Summer Study on Energy Efficiency in Buildings, 1.193–1.200. Washington, D.C.: American Council for an Energy-Efficient Economy.

Wisconsin Division of Energy. 2003. Focus on Energy Statewide Evaluation: Initial Cost-

Benefit Analysis. Final Report. March 31. Madison, Wisc.: Wisconsin Department of Administration, Division of Energy.

16


APPENDIX: STATE-BY-STATE LIST OF COST-EFFECTIVENESS TESTS USED Source except where noted: GDS & Associates (2004)

State Cost-Benefit Test Alabama NA Alaska NA Arizona Societal Arkansas Participant, RIM, TRC California TRC, Utility (Program Administrator)†, Participant*, Societalx, PPT Colorado RIM, TRCx

Connecticut TRC, Utilityx

Delaware NA Florida RIM, Participant Georgia TRC, RIM Hawaii Utility, TRC, Participant, RIM Idaho TRC Illinois Utilityx

Indiana Participant, RIM, Utility, and TRC Iowa RIM, Participant*, Utility, TRC, Societal Kansas NA Kentucky California PPT Louisiana NA Maine TRC, Societalx

Maryland NA. The Public Service Commission has recommended a move toward a societal perspective to the General Assembly.

Massachusetts TRC Michigan NA Minnesota Societal Test, Utility, Participant, RIM

Mississippi Participant, TRC, Utility, RIM

Missouri Utility, TRC, Societal Montana TRC, Societal Nebraska NA Nevada RIM, TRC New Hampshire Modified TRC (including 15% adder for environment) New Jersey TRC, Societalx

New Mexico NA New York TRC, Utility, Participant, Total Market Effects North Carolina RIM◊

North Dakota RIM, IRP (not mandatory) Ohio NA Oklahoma NA Oregon Societal, Utility

Pennsylvania Low-income only; 7- or 12-year payback depending on measure life Rhode Island TRC, Utilityx

17


State Cost-Benefit Test South Carolina RIM, TRC South Dakota NA Tennessee NA Texas Utilityx

Utah TRC Vermont Societal Virginia Participant, Utility, RIM, TRC Washington TRC, Utility West Virginia NA Wisconsin ROI (resembles TRC), RIM, California PPT, Societalx

Wyoming NA NA = No test mandated or commonly used or no information available. Tests that include NEBs are indicated with italics. * RAP (2006) x Titus, Nevius, and Michals (2004) † California (2004) ◊ Gillmore (2006)

18

valuation of non-energy benefits to determine cost ... · valuation of non-energy benefits to...

Documents