Guidelines for the Development and Maintenance of RTF-Approved Measure Savings Estimates

December 7, 2010Regional Technical Forum

Presented by: Michael Baker

Why Do We Need Guidelines

Define a traceable and transparent decision-making process

Achieve reliability commensurate with measure savings and benefits

Base decisions on consistent and transparent savings estimates

212/7/2010

Types of savings estimation methods

Deemed Unit Energy Savings (UES) Program must reliably capture measure identification

data, e.g., single family forced-air electric heat Savings = Delivered units X appropriate UES

Standard Protocols Standardized data collection and analysis Site-specific savings estimate

Custom Protocol Site-specific data collection, analysis and savings

estimate

312/7/2010

Deemed Unit Energy Savings

412/7/2010

Measure Specification

Eligible Applications Measure Identifiers Savings Baseline

Upgrade New - recent typical choices of eligible end users in purchasing new equipment and services

Upgrade, Replacement and Conversion - typical existing conditions found among eligible end users

Implementation Standards Product Standards

512/7/2010

UES Estimation Procedure

Statistical Experimental (treatment and control) Quasi – Experimental (pre / post)

Meta-Statistical A number of small studies Central tendency / lower bound deemed reliable

Calibrated Engineering Required for highly interactive measures Calibration to baseline and efficient case measure

effected end use consumption

612/7/2010

Calibrated Engineering Procedures

Aspects to consider in judging reliability: Input Parameters Model Calibration UES Components Interactions between Measures

Last-In Rule Heating/Cooling Interactions

712/7/2010

Provisional - Required Quality of Savings Estimate

Reliable baseline consumption for measure-affected end uses

Models calibrated to baseline end use consumption

Sound engineering and statistical analyses Programs commit to collect efficient case

consumption of the measure-affected end uses

812/7/2010

Deemed - Required Quality of Savings Estimate

Statistical or meta-statistical Findings are determined to be sufficiently

reliable Calibrated Engineering

Reliable baseline and efficient-case energy consumption for measure-affected end uses

Sound engineering and statistical analyses Models calibrated to both baseline and

efficient-case end use energy consumption

912/7/2010

Value-Based Standard for Reliability

Many factors contribute to reliability: Sample design and size measurement procedure Validity of the model specification and model calibration

RTF collective judgment determines sufficient reliability RTF may require that up to 3% of measure life cycle

benefits be spent to achieve sufficiency This is an upper limit on the spending Proposer and RTF should always seek the least cost

strategy

1012/7/2010

Development and Maintenance Process

Proposed Provisional

Until adequate data for deemed or de-activated Deemed

Sunset criteria Under Review

UES can still be used De-Activated

UES no longer approved

1112/7/2010

Special Status

Pilot Upon request, the RTF will provide a review and

comment on pilot projects intended to collect data needed to satisfy the criteria for either provisional or deemed measures

Out-Of-Compliance - Measures approved prior to these guidelines UES estimates continue to be approved if a plan for

bringing the measure into compliance is adopted within one year following adoption of these guidelines

1212/7/2010

Transparent Estimates of Savings

Summary Sheet – Four Tables Measure Classification and Properties Measure Identifiers Constant Parameters Unit Energy Savings (UES) Estimation Method,

Parameters and Sources See Appendix A for an example

1312/7/2010

Standard Protocols

1412/7/2010

What is a Standard Protocol

Document describing precisely the data collection and analysis required to estimate savings.

Transparent calculator that implements exactly the savings estimation procedure

See Appendix B for an example.

1512/7/2010

Purpose of Standard Protocols

Standardization of data collection and analysis for selected measures or closely related groups of measures Eliminates the need for site-specific

measurement planning Allows for uniform quality control tests Reduces required skill levels Reduces costs of savings estimation

1612/7/2010

Requirements for Standard Protocols

Precise specification of the measure including eligibility rules

Simplest reliable savings estimation procedure

Entirely prescriptive data collection and analysis procedure that will work for all eligible projects

Independent of program design and delivery method

1712/7/2010

More Requirements for Standard Protocols

Skills required are common among the region’s program implementation workforce

Protocol is cost-effective across a wide range of project savings

Yield savings estimates that are additive, i.e., independent of the savings from other measures implemented at the same site

Protocol documentation complies with section 3.5

1812/7/2010

Best Practice Savings Estimate

Obtain best practice examples from regional or national sources Power measurements for affected systems Measurements of primary determinants Measurements of less expensive and simpler

surrogates for primary determinants Pre and Post measurements

Develop best practice savings estimation model

1912/7/2010

Simplest Reliable Savings Estimation Method

Goal is +/- 20% estimation error for each site and unbiased estimate

Systematically explore simplification strategies Trending current Estimating primary determinants from surrogates Using performance curves to estimate pre Minimizing measurement duration

Select simplest reliable method

2012/7/2010

Protocol Template

Purpose Definition of Key Terms Eligible Projects Required Knowledge and Skills of

Practitioner Required Commissioning Data Collection Requirements Savings Estimation Steps

2112/7/2010

More Protocol Template

Sampling Procedure Relationship to Other Protocols and

Guidelines Estimate of Typical Cost Provisional Data Collection User’s Guide to Savings Calculator

2212/7/2010

Transparent and Accessible Savings Calculator

Calculator must accept exactly the data that is required by the protocol

Implemented using computer software that is generally accessible and reasonable priced for all practitioners

Either inherently transparent, e.g. Excel, or fully documented

2312/7/2010

Provisional Status and Data Requirements

Number and quality of best practice examples may not be adequate to definitively support the selection of a simplified savings estimation procedure

Provisional version of the protocol with a sunset criteria

Additional data collection requirements All data made available to RTF

2412/7/2010

Custom Protocol

Outside the scope of SBW’s current contract

Prior work by IPMVP, US DOE FEMP, the RTF, BPA, regional utilities and other agencies should be considered in developing guidance for the custom protocol

2512/7/2010