seem 94 calibration to single family rbsa data analysis and proposed actions

87
SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions RTF SEEM Calibration Subcommittee May 7, 2013

Upload: enid

Post on 24-Feb-2016

24 views

Category:

Documents


0 download

DESCRIPTION

SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions. RTF SEEM Calibration Subcommittee May 7, 2013. Goals for today’s Subcommittee Meeting. Review the following presentation in detail Consensus on next steps Are there any needed changes to the analysis? - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

SEEM 94 Calibration to Single Family RBSA Data

Analysis and proposed actions

RTF SEEM Calibration SubcommitteeMay 7, 2013

Page 2: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

2

Goals for today’s Subcommittee Meeting

• Review the following presentation in detail– Consensus on next steps• Are there any needed changes to the analysis?• Is there subcommittee consensus that the RTF should

make a decision stating SEEM94 is calibrated? – Receive suggestions from the subcommittee for

improvements in the presentation• Does it adequately tell the full story?• Is it the appropriate tool present to the RTF (assuming

previously covered sections will be skimmed over)?

Page 3: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

SEEM 94 Calibration to Single Family RBSA Data

Analysis and proposed actions

Regional Technical ForumMay 21, 2013

Page 4: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

Overview

• Background– Purpose– History

• Methodology– Data– Regression– Calibration

• Discussion• Proposal

Overview - 4

Page 5: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

Overview

• Background– Purpose– History

• Methodology– Data– Regression– Calibration

• Discussion• Proposal

Overview - 5

Page 6: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

Purpose of Calibration

6

Page 7: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

Background - Purpose 7

Align SEEM with Measured Energy Use• The SEEM model is used to estimate energy savings for

most space-heating-affected residential UES measures using the “calibrated engineering” estimation procedure (see section 2.3.3 of guidelines)– Heat Pumps and Central AC (ASHP, GSHP, DHP)– Weatherization– New Homes– Duct Sealing– Space Conditioning Interaction Factor

• Goal: Ensure SEEM94’s results are grounded in measured space heating energy use of single family homes. Use RBSA as source of measured data.

Page 8: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

8

RTF Savings Guidelines2.3.3.2. Model Calibration

In most cases, calibrated engineering procedures will involve at least one stage of modeling in which baseline and efficient case energy consumption are estimated for the measure-affected end use. For example, the heating load for single-family homes is estimated as part of the derivation of UES for ductless heat pump conversion. A simulation model is used to derive the heating end use for typical homes in different climate zones. Ideally, the model would be calibrated to measured heating end use for a sample of homes. If end use data are not available, the model should at least be calibrated to metered total use for the sample. Calibration should also be performed for samples that have adopted the measure, i.e., the efficient case. For measures that affect new buildings the calibration may be limited to the efficient case or to comparable buildings of recent vintage.

Background - Purpose

Page 9: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

Overview

• Background– Purpose– History

• Methodology– Data– Regression– Calibration

• Discussion• Proposal

Overview - 9

Page 10: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

10

RTF Decision History

Background - History

Date RTF Decision Summary Housing Type T-stat Results Data Sources Used in Calibration

Nov-2009 SEEM 92 model is calibrated. Single Family

HP & Gas FAF70°F Day ; 64°F NightElectric FAF and Zonal

66°F Day & Night

1. Res New Const. Billing Analysis (RLW 2007) 2. SGC Metered Data 3. NEEA Heat Pump Study (2005) Note: Very limited representation of Zones 2 & 3

Apr-2011SEEM 93 model is

calibrated. (implicit decision)

Single Family with GSHP 70°F Day ; 64°F Night 1. Missoula GSHP Study (1996)

Dec-2011 Use updated SEEM94 model

Single Family,Manufactured

Homen/a

Ecotope updated SEEM code to model the physics of the house infiltration, rather than rely on a constant stipulated infiltration rate input in previous versions of SEEM.

Dec-2011 SEEM 94 model is calibrated

Manufactured Home

69.4°F Day61.6°F Night

1. NEEM 2006 2. NEEA Heat Pump Study (2005) 3. MAP 1995 4. RCDP (manufactured homes)

Sep-2012 SEEM 94 model is calibrated Multifamily

Walk-up and Corridor68°F Day& Night

Townhouses66°F Day & Night

1. Multifamily MCS (SBW 1994) 2. MF Wx Impact Evaluation for PSE (SBW 2011) 3. New Multifamly Building Analysis (Ecotope 2009) 4. ARRA Verification for King County (Ecotope 2010)

Page 11: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

Background - History 11

RTF Decision History (Continued)For “model is calibrated” decisions…Calibration Methodology:1. Use available house and operation characteristics data from

billing/metering studies to develop inputs to SEEM runs;2. Adjust SEEM thermostat setting input to achieve a good

match (on average) between SEEM output (annual heating energy use) and billing/metering study results.

Note: The data sources used were free of (or mostly free of) supplemental fuel usage (wood, propane, oil, etc.) • Collection of reliable electric and gas usage data for space

heat consumption is relatively easy compared to other fuels.

Page 12: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

Date Forum Topic Outcome Links

1/23/13 Full RTF

Proposal to adopt calibration:

Send staff back to assess calibration needs related to

climate and measure parameters; and

engage subcommittee.

PresentationMinutes

3/20/13 Sub-committee Status update and check in. Presentation

Minutes

5/7/13 Sub-committee

Review staff’s proposal in detail. Decide whether to recommend RTF adoption.

Today PresentationMinutes

SF Calibration to RBSA - Recent History

Heating System Type

HeatingHigh °F(day)

HeatingLow °F(night)

Electric Zonal64 64

Electric FAF

Gas FAF 68.6 63.9

Heat Pump 69.6 65.4

History - 12

Page 13: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

Overview

• Background– Purpose– History

• Methodology– Data– Regression– Calibration

• Discussion• Proposal

Overview - 13

Page 14: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

Methodology – Overview

14

Page 15: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

15

Two Sources of Heating Energy Estimates

RBSA-PRISM. Estimates of annual “space heating use” for each house determine by using PRISM

– PRISM is a “change-point” regression model that uses billing data to estimate temperature-sensitive use

– PRISM analysis based on monthly billing data (at least 2years)

SEEM. Estimated annual space heating energy use for each house based on SEEM engineering model

– RBSA individual home characteristics (e.g., thermal envelop, heating system type, duct tightness) used as model inputs;

– Initial model runs use thermostat set to 68°F day & night• SEEM is a one-zone model, so t-stat setting input represents the average

setting for the entire house• Actual t-stat settings are not well documented (occupant reported settings are

unreliable, especially for “zonal systems”)• Thermostat setting will be used (step 2, below) as the “calibration knob”

Methodology - Overview

Page 16: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

16

Step 1 (Regression)

Use regression techniques to identify building characteristics that drive systematic differences between SEEM(68°F) and PRISM space heating energy use estimates.

Methodology - Overview

Page 17: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

17

Step 2 (Calibration)Use regression results to determine thermostat set-point that will align (i.e., “calibrate”) SEEM with PRISM annual space heating use– Calibration based on comparing average of all SEEM (68)

annual estimates to average of all PRISM annual estimates– Calibration is based on building characteristics identified in

regression.– SEEM run for each house at varying “day-time” thermostat

settings, with “night-time” thermostat settings based on occupant reported setbacks

Methodology - Overview

Page 18: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

Overview

• Background– Purpose– History

• Methodology– Data– Regression– Calibration

• Discussion• Proposal

Overview - 18

Page 19: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

19

Data Sources

Data Source used in this calibration:Underlying database* for the Single Family Residential Building Stock Assessment (2012)– RBSA study’s database offers recent billing analyses results

and detailed house characteristics on 1404 single family houses in the Region.

– RBSA data allows inputs for SEEM runs to be well defined for individual homes.

Methodology - Data

* Using a pre-release version of the database for this analysis .

Page 20: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

20

Key Model Input Parameters

RBSA Data Availability

UA Available for each house.

Weather Zip code (available for each house) linked to nearest TMY3 weather station.

Gas Heating Efficiency Available for some houses; used average for remaining houses.

HP Operation & Efficiency Not readily available. Used ARI control & 7.9 HSPF.

Duct System Leakage and Surface Area

Available for some houses; used average for remaining houses with ducts.

Duct System Insulation and Location

Available for each house.

Infiltration Available for some houses; used a floor area-scaled average (by foundation type) for remaining houses

Mechanical Ventilation Not available. Assumed 2 hours /day at 50 cfm.

Non-Lighting Internal Gains Not available. See next slide for details.

Lighting Internal Gains LPD available for each house; assumed 1.5 hours/day.

T-stat Setting Available based on interviews, but used this as the “calibration knob”.

Methodology - Data

Page 21: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

21

Detail: Non-Lighting Internal Gains• Equation:

• Based loosely on Building America Benchmark*– Used the original equation and values (averaged) to determine average internal

gains for RBSA homes.• Original equation also includes Number of Bedroom and Finished Floor Area terms

– Set Number of Bedrooms and Finished Floor Area terms to zero and adjusted Number of People term to achieve same average internal gains for RBSA homes.

• Building America Benchmark based on– “The appliance loads were derived by NREL from EnergyGuide labels, a Navigant

analysis of typical models available on the market that meet current NAECA appliance standards, and several other studies. ”

– “The general relationship between appliance loads, number of bedrooms, and house size, was derived empirically from the 2001 RECS. ”

*Hendron, Robert. "Building America Research Benchmark Definition, Updated December 20, 2007." NREL/USDOE EERE. January 2008. NREL/TP-550-42662

Methodology - Data

Page 22: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

Methodology - Data 22

Realistic SEEM Simulations Not Feasible/Possible for All Homes in RBSA

Issue Count

More than one foundation type 331

25% > Ceiling Area to Floor Area > 200%, or Missing Ceiling U-value 36

Footprint Area to Floor Area < 20% 36

30% > Wall Area to Floor Area > 200%, or Missing Wall U-value 24

Missing Floor U-value for Crawlspace Foundation 5

Window Area = 0 3

Window u-value = 0 3

• Resulting House Count: 1011– These issues overlap on some houses, so the sum of

the counts cannot be subtracted from 1404 to get 1011.

Page 23: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

23

Data Filters Excluded Some RBSA HomesVariable Filter

Value(s)Notes Count

(filtered out)SEEM Run Valid SEEM run must be valid (> 0 kWh/yr). 4

Billing Energy Use > 1,500 kWh/yr

Intends to screen out partially used or unused houses

38

eRsq and gRsq = 0 or ≥0.45

Screens out houses with poor billing analysis results (0.45 per David Baylon)

398

Non-natural-gas & non-electric Fuel Use

0 Screens out houses with wood, oil, propane, etc. consumption because billing analysis not performed.

352

Primary Heating System

eZonal, eFAF, gFAF, HP

Removes gas boilers, wood stoves, etc. 216

Secondary Heating System Fuel

Electric or Gas

Removes wood stoves, propane heaters, etc. 274

• Gas Billing converted to kWh/year using reported AFUE• Resulting House Count: 293• (The counts for each item overlap here, too)

Methodology - Data

Page 24: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

Methodology - Data 24

Additional Data Filter for PRISM Excluded Additional Homes

Exclude any home that had an out-of-range PRISM T-balance for one or more components.

– The PRISM analysis restricted balance point temperatures to be between 48 and 70 ⁰F.• T-balance below 48⁰ is plausible.• T-balance above 70⁰ is not physically plausible. We filter these out since such

values are evidence of a poor PRISM fit.

• Our 293 sites’ T-bal values include…– 10 that defaulted to 70⁰ when PRISM’s initial fit exceeded the max.

(Excluded from analysis.) – 16 that defaulted to 48⁰ when PRISM’s initial fit was below the minimum.

(Kept)– 267 whose PRISM balance points were within the acceptable range.

(Kept)• This leaves 283 sites for the present analysis.

Page 25: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

Methodology - Data 25

Final Data SetSEEM values calculated with t-stat = 68°F (constant)

Seem (68) Heating Energy (kWh)

Page 26: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

Overview

• Background– Purpose– History

• Methodology– Data– Regression– Calibration

• Discussion• Proposal

Overview - 26

Page 27: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

27

Methodology - Regression

Page 28: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

Methodology – Regression 28

Regression Overview (1)• Analysis

Identify and quantify any systematic patterns (trends) in the differences between SEEM(68°F) and PRISM savings estimates ( ∆ kWh = SEEM(68°F) kWh ‒ PRISM kWh.

• Systematic means “explained by known variables.” (Example: SEEM(68°F) kWh tends to exceed PRISM kWh in cooler climates.)

• Tacit assumption: PRISM estimates roughly unbiased.• Definitions

• “PRISM kWH” = Heating energy use via billing analysis; from RBSA SF dataset.

• “SEEM(68°F) kWh” = Heating energy use via SEEM runs using house-specific characteristics data from the RBSA SF dataset with thermostat set to 68°F

Page 29: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

Methodology – Regression 29

Regression Overview (2)• Problem is multivariate… – A single underlying trend (example: ∆ increasing with

heating use) may appear in multiple guises (∆ increasing with HDD, or with U-value, or with building heat loss)

• Approach is multiple regression…– Compare PRISM kWh with SEEM kWh when SEEM is run

with a constant T-stat setting (68°F day, 68°F night.)– Y-variable is the percent difference between SEEM kWh

and PRISM kWh (when SEEM uses T-Stat=68°F).– X-variables are physical characteristics known through

RBSA. (Specifying the x-variables is a large part of the work.)

Page 30: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

30

Setting up the Regression (1)Primary interest is in differences between SEEM(68) kWh and PRISM kWh—the Y-variable must capture these differences. – Heteroskedasticity. The SEEM(68) /PRISM differences

generally increase in magnitude in proportion to SEEM(68) kWh (or PRISM kWh). (See earlier graph.)

– Measurement error (random noise). As estimates of heating kWh, SEEM(68) and PRISM both have substantial standard errors.

Methodology – Regression

Page 31: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

31

Setting up the Regression (2)

• Note choice of signs: means SEEM > PRISM.• What goes in the denominator? “??” =

“Actual kWh” would be ideal. – Using SEEM kWh or PRISM kWh would skew y-

values. (Next slide.)

– Log-transforms (closely related) not quite right either.

– Instead, divide by midpoint: “??” = (SEEM + PRISM)/2. (Two slides down.)Methodology – Regression

Page 32: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

Dividing by PRISM kWh magnifies differences where PRISM’s random error happens to be negative (since these values get artificially small denominators). This biases the percent differences upwards.

32Methodology – Regression

Page 33: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

Upward bias (mostly) goes away when we divide by the value halfway between SEEM(68°F) and PRISM.

33Methodology – Regression

Page 34: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

Upward bias (mostly) goes away when we divide by the value halfway between SEEM(68°F) and PRISM.

34Methodology – Regression

Page 35: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

Upward bias (mostly) goes away when we divide by the value halfway between SEEM(68°F) and PRISM.

35Methodology – Regression

Page 36: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

Upward bias (mostly) goes away when we divide by the value halfway between SEEM(68°F) and PRISM.

36Methodology – Regression

Page 37: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

Methodology – Regression 37

Building the Regression Model• Goal is to identify variables that lead to systematic

differences between SEEM(68°F) and PRISM.– “Lead to” is only seen in rough trends (think: correlation).

• Looking to capture unknown effects – not a physical model.

• Model development is iterative. – A variable may be weakly correlated with raw y-values

but strongly correlated with y’s that have been adjusted to account for some other variable’s influence.

Page 38: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

Important Limitations• Avoiding Colinearity - When a potential x-variable closely

tracks some combination of variables that are already included.– Example – Including both heat loss rate and vintage– This redundancy leads to unstable model fits.– Threshold for “tracks too closely” gets low when the usual suspects

are around:

High noise / faint signal / small sample.

• Pursuing Parsimony. General principle: Don’t over-fit the data (by including too many explanatory variables).

• Incomplete data variables. Some variables (e.g., duct tightness and infiltration) aren’t known for very many houses.

Methodology (Regression) - 38Methodology – Regression

Page 39: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

39

Prominent x-variable candidates• Characteristics that likely influence differences

between SEEM(68°F) and PRISM estimates of use– Thermal efficiency drivers (U-values, duct tightness,

infiltration, …)– Heating system type – Climate (i.e., HDDs)

• Following graphs illustrate “influence” of several variables (separately) on percent difference between SEEM(68°F) and PRISM

Methodology – Regression

Page 40: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

40Methodology – Regression

SEEM (68 ) - PRISMMidpoint

SEEM (68 ) - PRISMMidpoint

Page 41: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

41Methodology – Regression

SEEM (68 ) - PRISMMidpoint

SEEM (68 ) - PRISMMidpoint

Page 42: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

42Methodology – Regression

Page 43: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

43

Insulation Variables

The big surfaces: Wall, ceiling, and floor. • Express in terms of heat loss (U-values,

weighted by surface area as appropriate)• We separate out Floor U because of different

foundation types. – One variable accounts for ceiling and wall heat

loss.– Another variable accounts for floor heat loss in

crawlspace homes.

Methodology – Regression

Page 44: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

44

Variable for Wall/Ceiling U

Applies to all homes (regardless of foundation type).A simple indicator variable:

“Wall/Ceiling Insulation is Poor” ifWall u-value > 0.25, ORCeiling u-value > 0.25, OR Both u-values > 0.25.

This variable captures the main effect of the weighted average. (See next slide)

Methodology – Regression

Page 45: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

45Methodology – Regression

Page 46: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

46

Variable for Floor U

Particularly interested in crawlspace heat loss since crawlspace insulation is a common measure.

Variable definition: “Yes/No” indicator for uninsulated crawlspace.

Note: Sites with basements, slabs, and insulated crawlspaces all have Uninsulated Crawl = “No”

Methodology – Regression

Page 47: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

47

Do these indicator variables really capture the insulation effects?

The next two slides compare various u-values’ relationships with– Unadjusted (raw) percent differences;– Percent differences that have been adjusted for

the two insulation variables included in the regression.

Methodology – Regression

Page 48: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

48Methodology – Regression

Tom Eckman
Show Unadjusted and Adjusted as top and bottom on each slide. Split Uo & Ceiling U from Wall and Floor
Page 49: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

49Methodology – Regression

Page 50: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

50

Heating System Variable

Four distinct heating systems in the sample:Electric zonal Electric FAF Gas FAF Heat pump

After controlling for insulation, heating system effect appears to be captured with just two groups:

“Electric Resistance” = Electric zonal / Electric FAF “Gas/HP” = Gas FAF / Heat Pump

Parsimony: two is better than four!

Methodology – Regression

Page 51: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

51Methodology – Regression

Page 52: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

52Methodology – Regression

Page 53: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

Methodology – Regression 53

Model 1 fit summary: Est. s.e. p-value (Intercept) -0.01 (0.04) 0.80 elec. resistance 0.27 (0.05) 0.00 poor.ins.ceil.wall 0.42 (0.08) 0.00 uninsulated.crawl 0.15 (0.07) 0.04 Adjusted R-square = 0.212

…and with an interaction term for insulation: Est. s.e. p-value Intercept -0.18 (0.04) 0.62 elec. resistance 0.27 (0.05) 0.00 poor.ins. ceil.wall 0.49 (0.09) 0.00 uninsulated.crawl 0.21 (0.08) 0.01 poor.ins.c.w*unins.crawl -0.21 (0.16) 0.19 Adjusted R-square = 0.214

No strong recommendation either way because of low p-value, but proposal is to drop the interaction term.

Tom Eckman
Tells us what "P-value" means and why we should car. Need to have tutorial on why R^2 of .21 is at all explanatory?
Page 54: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

54

Climate variable

HDD effect is not very pronounced. Next slide shows percent differences (adjusted for effects in the previous regression), versus HDDs• Standard HDDs with constant (65⁰) base.• Plot shows (slight) positive correlation

between HDDs and adjusted y-values. • Group means (x-mean, y-mean) lie very near

the overall trend line.

Methodology – Regression

Page 55: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

55

(Black line indicates OLS linear regression fit.)

Methodology – Regression

Page 56: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

56

Climate Variable

• A modest linear trend is clear from the plot.

• Could either use indicator variables, or the actual HDDs values (a single continuous variable).– Group means agree with overall linear trend

almost perfectly, so little practical difference.

• We use the continuous variable, x = HDDs.

Methodology – Regression

Page 57: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

57

Previous fit: Estimate s.e. p-value (Intercept) -0.01 (0.04) 0.80 elec. resistance 0.27 (0.05) 0.00 poor.ins.ceil.wall 0.42 (0.08) 0.00 uninsulated.crawl 0.15 (0.07) 0.04 Adjusted R-square = 0.212 And now with HDDs: Estimate s.e. p-value Intercept -0.40 (0.15) 0.01elec. resistance 0.27 (0.05) 0.00poor.ins.ceil.wall 0.44 (0.07) 0.00 uninsulated.crawl 0.13 (0.07) 0.07Base-65 HDDs 7.3e-5 (2.7e-5) 0.01 Adjusted R-square = 0.230

Methodology – Regression

Page 58: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

Interpreting the HDD Coefficient

Our fitted coefficient for HDDs was . What does this mean in practical terms?

In our sample, the HDDs averages differ by about 1500 HDDS from one climate zone to the next.

Since , the climate zone effect corresponds to about an 11% difference.

58Methodology – Regression

Page 59: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

59

So far, so good…• The next two slides compare four variables’

relationships with– Unadjusted (raw) percent differences; and– Percent differences that have been adjusted for all

four variables included in the regression.• HDDs and heat source show zero relationship with

adjusted differences.• Square footage and internal gains relationships

went from weak to weaker (even though they are not included in the model – that’s good!).

Methodology – Regression

Page 60: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

60Methodology – Regression

Page 61: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

61Methodology – Regression

Page 62: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

62

Percent difference versus midpoint has also improved…(And midpoint isn’t in the model either)

Methodology – Regression

Page 63: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

63

And the insulation variables’ plots still look good…

Methodology – Regression

Page 64: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

64Methodology – Regression

Page 65: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

65Methodology – Regression

Page 66: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

Methodology – Regression 66

What else should we consider?

Next slide indicates several variables’ correlation with adjusted percent differences.

Page 67: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

67

Observations• Duct leakage has the largest apparent

correlation, but this variable is sparsely populated. (We’ll look at it next.)

• PRISM HDDs have moved up – these had almost no correlation with unadjusted differences. (We discuss at the end.)

• RBSA (reported) t-stat values have a slight negative correlation with % differences. (This sign makes sense, but we’d expect more.)

Methodology – Regression

Page 68: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

68

Duct Leakage

• Have direct RLF and SLF measurements for 33 homes;

• Also, 87 homes have no ducts (zero leakage);• Another 38 (excluded from analysis) have

ducts entirely inside of conditioned spaces.– Some of these spaces are basements designated

“conditioned” simply because they contain ducts.– 26 of the 38 have “heated basements”

• Not much basis for calibration here…Methodology – Regression

Page 69: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

69Methodology – Regression

Page 70: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

Methodology – Regression 70

Duct Leakage (continued)• Visually, there’s not much correlation in the

range containing most of the data. • Numerically… – Correlation is 53% when only the 33 measured values

are included;– Drops to 15% when 4 right-most points are omitted;– Values drop to 31.5% and 5.3% when we include

homes without ducts (zero leakage).• Weak (and ambiguous) basis for recommending

adjustment specific to duct-leakage.

Page 71: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

71

Infiltration

• Have direct infiltration measurements for 95 homes;

• But even less reason for calibration here…

Methodology – Regression

Page 72: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

72Methodology – Regression

Page 73: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

Methodology – Regression 73

Infiltration (continued)

• Visually, the relationship is null.• Numerically… – Correlation is -11.8% when all points are included;– Changes to 2.3% when single right-most point is

omitted.• No basis for adjustment for infiltration once

other variables are included.

Page 74: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

74

PRISM Balance Point

• A question was raised at the May 20 subcommittee meeting regarding whether the regression should take into account the house balance point determined by PRISM.

Methodology – Regression

Page 75: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

75Methodology – Regression

Page 76: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

76

PRISM HDDs

• Definitely a trend, but is it unexpected? Does it require action?– We know that unobserved variables drive a portion of SEEM-PRISM

deviations that we treat as noise. – Consider a home where an unobserved variable yields an effective

balance point that is lower than we would expect based only on observed variables. This home will tend to satisfy both:• SEEM(68) kWh > PRISM kWh and • TMY HDD > PRISM HDD

– In other words, the presence of unobserved variables causes a positive correlation between kWh differences and HDD differences.

• Conclusion: The presence of unobserved variables should yield a trend like the one seen on the previous slide.

• So the trend is what we would expect.

Methodology – Regression

Page 77: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

77

Proposed Final Model

Variable Estimated Standard p-value coefficient errorIntercept -0.40 (0.15) 0.01elec. resistance 0.27 (0.05) 0.00poor.ins.ceil.wall 0.44 (0.07) 0.00 uninsulated.crawl 0.13 (0.07) 0.07HDDs (Base 65) 7.3e-5 (2.7e-5) 0.01

Adjusted R-square = 0.230

𝑦=𝑖𝑛𝑡𝑒𝑟𝑐𝑒𝑝𝑡+𝛽𝑒𝑙𝑒𝑐 .𝑟𝑒𝑠𝑖𝑠× 𝐼𝑒𝑙𝑒𝑐 .𝑟𝑒𝑠𝑖𝑠+𝛽𝑝𝑜𝑜𝑟 .𝑖𝑛𝑠 .𝑐𝑒𝑖𝑙 .𝑤𝑎𝑙𝑙× 𝐼𝑝𝑜𝑜𝑟 .𝑖𝑛𝑠 .𝑐𝑒𝑖𝑙 .𝑤𝑎𝑙𝑙+𝛽𝑢𝑛𝑖𝑛𝑠 . 𝑐𝑟𝑎𝑤𝑙× 𝐼𝑢𝑛𝑖𝑛𝑠 .𝑐𝑟𝑎𝑤𝑙+𝛽𝐻𝐷𝐷×𝐻𝐷𝐷

Methodology – Regression

Page 78: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

Overview

• Background– Purpose– History

• Methodology– Data– Regression– Calibration

• Discussion• Proposal

Overview - 78

Page 79: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

Methodology – Calibration 79

Final Step: Interpreting Results• From the fitted model, we obtain adjustment

factors that apply to SEEM output to align SEEM with the RBSA-PRISM data.

• A given site’s adjustment factor depends on the values of the explanatory variables for that site.

• Group HDDs by climate zone. Then for each zone, there are 8 possible configurations of the three other variables.

• This yields 24 distinct adjustment factors in all.

Page 80: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

80

Calibration FactorsSEEM(68) differs from PRISM by these factors (on average)

0%

20%

40%

60%

80%

100%

No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes

No Yes No Yes No Yes No Yes No Yes No Yes

Gas FAF / HP Elec. Res. Gas FAF / HP Elec. Res. Gas FAF / HP Elec. Res.

Climate Zone 1 Climate Zone 2 Climate Zone 3

Methodology – Calibration

Page 81: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

81

T-Stat Calibration

• Translate percent kWh adjustments into adjustments in daytime t-stat setting (from 68 °F).

• No data limitations here: we can directly observe SEEM’s sensitivity to t-stat settings.

• Method: 1. Run SEEM for each house at multiple temperature settings in 2 degree increments

– Daytime Settings: … 58, 60, 62, … – Nighttime Setback: Daytime setting - setback

» Setback: Use average difference between reported daytime and nighttime t-stat settings in RBSA dataset; by heating system type:

2. Determine relationship of calibration factors to temperature settings for each of the 24 scenarios.3. Interpolate to determine “calibrated” t-stat settings.

(need to add a graph to help explain this)

Heating System Type Avg Setback (°F)

Electric FAF 6.0 Electric Zonal 4.8 Heat Pump 4.3 Gas FAF 4.8

Methodology – Calibration

Page 82: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

82

Calibrated Thermostat Settings

50

55

60

65

70

No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes

No Yes No Yes No Yes No Yes No Yes No Yes

Gas FAF / HP Elec. Res. Gas FAF / HP Elec. Res. Gas FAF / HP Elec. Res.

Climate Zone 1 Climate Zone 2 Climate Zone 3

Heating System Type Avg Setback (°F)

Electric FAF 6.0 Electric Zonal 4.8 Heat Pump 4.3 Gas FAF 4.8

Methodology – Calibration

Page 83: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

Overview

• Background– Purpose– History

• Methodology– Data– Regression– Calibration

• Discussion• Proposal

Overview - 83

Page 84: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

Discussion 84

Discussion• Are we done?• Decision: SEEM94 is “calibrated”; it will give reliable

heating energy consumption results – for single family houses with the following characteristics:

• Heating System is one or more of the following: Gas FAF, Electric FAF, HP, zonal electric (no other heating system type);

• Occupied/normal houses (PRISM worked); – if the following inputs are used:

• Calibrated Thermostat Settings (see slide above); and• Internal Gains:

Page 85: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

Discussion 85

Next Steps• If the RTF agrees it’s calibrated, the RTF will be able to use SEEM94 to help

estimate energy savings for residential single family– Heat Pump

• Conversions• Upgrades• Commissioning, Controls, and Sizing

– Weatherization• Insulation• Windows• Infiltration reduction

– Duct Sealing– New Home Efficiency Upgrades

• “Help” is used here because we will still need to deal with “non-electric benefits” for these measures.– This topic is out of scope for today’s discussion. The goal today is simply to

determine whether SEEM has been calibrated to provide reliable results.

Page 86: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

Overview

• Background– Purpose– History

• Methodology– Data– Regression– Calibration

• Discussion• Proposal

Overview - 86

Page 87: SEEM 94 Calibration to  Single Family RBSA Data Analysis and proposed actions

Proposal 87

Proposed Motion

“I _______ move that the RTF consider SEEM94 calibrated for single family houses.”