New-Construction Manufactured Homes:Update on the Path to Proven Savings

Regional Technical ForumNovember 10th, 2015

Josh Rushton & Mohit Singh-Chhabra

2

Today’s Objective

• Review recent discussions related MH calibration and new construction measures

• Get RTF advice and consent on general direction for MH new construction measures– Staff/CAT recommends moving NEEM (ENERGYSTAR)

and EcoRated Homes measures to Planning category – Research needed to directly compare average energy

consumption in efficient-case homes and in standard (HUD+) new-construction homes

– No change to measures for existing MHs

3

Measure Background

• Two measures for new-construction Manufactured Homes (MH)– MH ENERGY STAR (NEEM 1.1): Only shell specs– MH Eco-Rated: Shell specs + lighting + appliances– Confused about specs? See Additional Slides!

• Both measures currently– Category “Proven”– Status “Under Review”– Sunset November 30, 2015

4

Calibration History (Existing MH)• August 2014: RTF approved MH Calibration for existing homes (Phases I & II)

– Calibration captures adjustment factors that align modeled heating energy consumption (SEEM.69) with observed heating energy consumption (VBDD)

– Adjustments correct an unknown mix of misalignments related to heating load, system performance, behavioral, and other factors

• December 2014: CAT presented Wx savings for existing MH based on calibration– Savings values much lower than expected– RTF directed CAT to review calibration, compare with pre/post program data

• June 2015: CAT showed how calibration results compare to pre/post data from Idaho Power program – Reasonable agreement for Wx savings in Electric Resistance (ER) homes; RTF

approved calibration curve and Wx savings for existing MHs with ER heat – Poor agreement for Heat Pump (HP) conversions and Wx in HP homes; HP-related

MH measures moved to planning; Research Strategy approved at August meeting

5

Calibration History (New MH)• September RTF Meeting

– CAT proposed extending calibration curve for new construction homes by adding 89 NEEM homes to the RBSA dataset

– Baylon observed that VBDD naturally over-estimates space heating • Algorithm picks up HDD-correlated energy consumption from other end-

uses (e.g., winter-time increases in lighting and water heating)• Worrisome because calibration objective was to align SEEM’s space

heating estimates with VBDD • May be especially important for NC homes with low heat loads

• Since September– NEEA contracted Ecotope and Energy RM to look into the VBDD

question and assess whether calibration can reliably estimate savings for new construction measures

– CAT worked with NEEA, Ecotope, and Energy RM to resolve easy issues and identify difficult issues

6

Recent Discussions

7

NEEA Working Group Concern 1CAT Response in RED

• Back-of-envelope calculations suggest VBDD picks up about 1000 kWh of HDD-correlated consumption from end-uses other than space heating

• Makes sense. Easy to modify calculations– Subtract 1000 kWh from VBDD heating energy variable

at the outset – Notation: Adjusted VBDD = VBDD.ht.kWh - 1000– Leads to ≈1000 kWh reduction in baseline and efficient-

case heating energy estimates• No significant effect on savings estimates

8

NEEA Working Group Concern 2CAT Response in RED

• The relationship between SEEM and VBDD is distorted when the calibration curve is defined in terms of percent differences (SEEM – VBDD)/SEEM– A linear relationship or simple off-set between SEEM and VBDD will appear

curved when expressed in percent terms– This leads to more extreme adjustments for efficient homes than standard

homes, and this will bias savings estimates

• CAT disagrees. Single relationship can be expressed in more than one way– Trend may look different under different transformations, but this doesn’t

change the basic relationship (more on this next)– Can change how different points influence regression fit

• This can affect final fitted trend, especially when data is sliced thin• Effect is not systematic – not a bias threat

– No systematic effect on savings (see additional slides)

How do predictions based on percent-scale regression look when we convert them to predicted kWh and add them to the absolute-scale plot (left)?

Concern 2: Percent differences vs absolute kWhData filters: No heat pumps, R2 > 0.45, SEEM & VBDD < 20,000 kWh

Concern 2: Percent differences vs absolute kWhData filters: No heat pumps, R2 > 0.45, SEEM & VBDD < 20,000 kWh

Predictions from percent-difference regression roughly agree with trend in absolute kWh plot

Concern 2: Percent differences vs absolute kWh…a closer look at that lower plot

Linear fit

Predictions from percent- differences plot

Loess smoother fit

12

NEEA Working Group Concern 3CAT Response in RED

• Calibration may be capturing behavioral or other issues. Doesn’t seem consistent with pure-physics account of heat loss.

• Right – main concern is how much slower the meter spins

13

Concern 4.1: Comparing RBSA to NEEMCAT Response in RED

• RBSA not a sufficient comparison group• Only includes 6 non-NEEM homes

built since 2004 • Other RBSA U-values may not be

consistent with NEEM U-values • ∆kWh / ∆SEEM.69 (slope) differs

between RBSA and NEEM samples• CAT agrees data sets may not be similar

enough for “fair” comparison• Today’s recommendation based on

this concern• Probably shouldn’t read much into

apparent difference in slopes• NEEM homes all have the same U-

values in SEEM input• Data sets look more similar when x-

axis scaled as kWh/sq.ft. (see additional slides)

14

Concern 4.2: Comparing NEEM to NEEM?CAT Response in RED

• ∆kWh / ∆SEEM.69 (the slope) near 1 for NEEM homes

• Makes physical sense – maybe we should use this trend for calibration adjustments

• CAT disagrees. In NEEM sample, SEEM.69 only varies because of climate and sq. ft.

• Slope ≈ 1 only says SEEM.69 treats sq. ft. & climate about right in NEEM homes

• Not enough to generalize ∆kWh / ∆SEEM.69 to address changes in Uo

15

Proposed Path Forward

16

Motivation

• Need data that provides direct insight into difference between energy consumption in efficient new MHs and standard new MHs

• Ideally, data will include billing kWh and building specs for samples of efficient and standard new homes

17

Staff Recommendation• New-Construction MH (ENERGYSTAR and EcoRated Homes): Move both

measures to Planning. Research strategy to emphasize that savings estimate(s) should be closely tied to observed differences in kWh between efficient and baseline groups. – Final UES values likely to include adjustments based on SEEM models (plus

observed ∆kWh / ∆SEEM.69) • Examples: Savings by heating zone; cross-walk between NEEM and EcoRated savings

estimates – Area of special emphasis: For basic savings estimates to be accepted, need fairly

“clean” comparison across groups. • Confounding variables (heat pumps, wood heat, etc.) tracked where possible to avoid

major imbalances between the groups • “Fair” comparison probably more important than wide regional representation or highly

detailed audit data

• Existing MH measures: No change. We’ve already compared savings against IDP pre/post program data – More data always welcome; no need to move back to planning in the mean time

18

Discussion and RTF Direction

Is this the right path?

Can feasible research really get us to more solid ground?

Should Staff/CAT develop a Research Strategy in order to move the ENERGYSTAR and EcoRated manufactured homes measures to the Planning category?

19

Decision needed

“I, _______, move that the RTF extends the sunset date for both manufactured homes measures, ENERGYSTAR and EcoRated homes, to March 31, 2016”

20

Additional Slides

21

Alphabet Soup• HUD. Minimum federal standard, some homes still being built to this spec

– HUD update working its way through the system, likely to take effect within a few years – New HUD requirements expected to come close to the current NEEM specs (NEEM 1.1).

• HUD+. Estimated average spec for non-NEEM/EcoRated home since about 2010. • NEEM 1.0 was SuperGoodSense after that program ended. NEEM 1.0 lasted until 2004.• NEEM 1.1 refers to the NEEM program as updated in 2004 (current spec at time of this presentation)

– Relative to NEEM 1.0, big change is that NEEM 1.1 locks down window efficiency at U-0.35 (prior to this, efficient windows could be traded off against other shell components).

– NEEM 1.1 specs cover shell components, duct sealing and ventilation (exhaust fan, max 32 Watts, running continuously, no heat exchanger), plus a tiny DHW thing (0.93).

– NEEM 1.1 meets current ENERGY STAR requirements, but the reverse does not always hold. It might not make sense for RTF to use the ENERGY STAR label in its measure workbooks because NEEM 1.1 is really the requirement, and the national ENERGY STAR spec would not always meet NEEM. (Marketers would still be able to use ENERGY STAR label for all NEEM homes)

• EcoRated is NEEM 1.1, plus window U-0.32, 80% efficient lighting, ENERGYSTAR fridge and dishwasher, and low-flow showerheads and faucets.

• HPMH is a very efficient spec that has been studied in small demonstration projects but currently has very little traction in the region.

• NEEM 2.0 doesn’t yet exist. It is currently a placeholder for the program that will replace NEEM 1.1 after the coming HUD update takes effect

22

Savings estimates according to three different calibration regressions (1)

Estimated savings for existing construction measures. For demonstration only (figures have not undergone thorough QC)

Represented calibrations:• Percent-difference regression

with original VBDD estimates – Essentially the calibration from

September• Percent difference regression

with VBDD -1000 – Upper-left graph in four-plot

slide above • Absolute-scale regression

– Upper-right graph in four-plot slide

-

0.10

0.20

0.30

0.40

0.50

0.60

0.70

Pct. diff vs SEEM.sq.ft (raw vbdd.ht) Pct. diff vs SEEM.sq.ft VBDD vs SEEM kWh

23

Savings estimates according to three different calibration regressions (2)

Estimated savings for existing construction measures. For demonstration only (figures have not undergone thorough QC)

Represented calibrations:• Percent-difference regression

with original VBDD estimates – Essentially the calibration from

September• Percent difference regression

with VBDD -1000 – Upper-left graph in four-plot

slide above • Absolute-scale regression

– Upper-right graph in four-plot slide

-

0.10

0.20

0.30

0.40

0.50

0.60

0.70

Pct. diff vs SEEM.sq.ft (raw vbdd.ht) Pct. diff vs SEEM.sq.ft VBDD vs SEEM kWh

-

1.00

2.00

3.00

4.00

5.00

6.00

7.00

8.00

9.00

10.00

Pct. diff vs SEEM.sq.ft (raw vbdd.ht) Pct. diff vs SEEM.sq.ft VBDD vs SEEM kWh

24

• Both graphs use y = percent difference • Only difference is x-variable: SEEM kWh/ft2 (left) vs SEEM kWh (right)

• Left: As SEEM kWh/ft2 varies, trend in percent differences similar between NEEM and RBSA

• Right: As SEEM kWh varies, trend in percent differences looks different between NEEM and RBSA

• This highlights the risk of drawing conclusions from a comparison of different data sets

• Note: x-variable choice (kWh versus kWh/ft2) has little net effect on estimated savings in RBSA-only calibration

Note on X-axis scaleData filters: No heat pumps, R2 > 0.45, SEEM & VBDD < 20,000 kWh