meea technical webinar: a new approach to estimating free ridership in upstream lighting programs

MEEA Technical Webinar Series:

A New Approach to Estimating

Free Ridership for Upstream

Lighting Programs

Presenters: Stan Mertz - Applied Proactive Technologies &

Tami Buhr - Opinions Dynamics

Thursday, October 18th, 2012

MEEA’s Role in the Midwest

• Nonprofit serving 13 Midwest states

• 10+ years serving states, energy offices, utilities and communities

• Staff of 25 in Chicago

• Actions

– Designing & Administering Energy Efficiency Programs

– Evaluating & Promoting Emerging Technologies

– Regional Voice for DOE/EPA & ENERGY STAR

– Coordinating Utility Program Efforts

– Delivering Training & Workshops

– Advancing Energy Efficiency Policy

– Promoting Best Practices

REVENUE NEUTRAL MODELA New Approach to Estimating Free

Ridership for Upstream Lighting Programs

October 2012

Agenda

1. Challenges associated with estimating lighting program NTG

2. Theoretical background underlying Revenue Neutral Sales Model

3. Example of how the model works

4. Questions

Revenue Neutral Model 2

Lighting Program Evaluation

Challenges and Traditional Methods


Estimating Lighting Program Free Ridership is Challenging

• Programs usually delivered through an upstream markdown method

• Customers purchase discount lighting, walk out the store and disappear• Often unaware of the discount

• Note that this does not mean customer would have purchased bulbs at full price

• Retailers are fully aware of their participation

• Traditional methods use data from both customers and retailers to estimate free ridership


What Evaluators Really Want: Full Sales Data

• Data that tracks sales of EE lighting with and without program pricing would provide the best estimate of program impact

• Could see the actual lift in sales when program in effect

• Depending on how long program has been running, measurement approaches could include:

• Pre-program sales, sales when programs turn pricing on and off, sales of like products that are not discounted, sales in comparison areas that do not have programs

• Unfortunately, retailers will not provide sales of EE lighting at regular price. Will only provide sales of products lighting programs discount.


Sales Data Alternatives for Estimating Free Ridership

• Participant self-reports• Common method for estimating program free ridership for rebate programs• Difficult with upstream programs where customers purchase program incented product • Difficult with upstream programs where customers purchase program incented product

and disappear

• Two survey methods commonly used:• General population telephone surveys

• Call utility customers and ask detailed questions about past lighting purchases• Results are of questionable validity due to timing of survey , small nature of purchase, and difficulty

identifying program purchasers

• In-store customer interviews• Interview customers in store immediately after they make purchase decision• Greater confidence in self-report results• Usually make use of non-probability samples• Expensive and challenging to conduct


Sales Data Alternatives for Estimating Free Ridership (2)

• Retailer Interviews• Conduct interviews with corporate or store level retailers and ask for

ti t f g i t lestimate of program impact on sales• Are no more likely to give up the numbers in an interview than a request for

data. • Store level staff often do not know sales• Store level staff often do not know sales• Corporate level do not know for a specific utility territory• At best, get rough estimates• May have vested interest in seeing programs continue • May have vested interest in seeing programs continue

• Modeling Techniques• Many require use of self-report data in addition to other data (e.g. multi-

state model revealed preference models)state model, revealed preference models)• Suffer from the same validity problems


Another Look at Sales Data

• Existing FR methods are challenging, expensive and produce questionable results

• We keep going back and asking for complete sales data. • Maybe if we ask again, or ask nicer, or ask a different person, we’ll get it.

• We have asked everyone for sales data including program implementersI l t l h l l t g t t t t d t i t t • Implementers also help evaluators get access to stores to conduct intercept interviews

• We started talking about the challenges associated with our existing free ridership methods and alternativesp

• Is there something we can do with the sales data we do have?• Have program sales data. Also have program prices, regular prices, and sales goals

for each retailer for each product

• Estimate non-program sales using data we do have and a model of retailer behavior


Revenue Neutral Model

The Alternative Sales Data Method


Retailer Behavior

• Retailers will only participate in utility lighting programs if their participation is revenue neutral• Their “top line sales” remain the same or increase; cannot decrease

• But why are “top line sales” so important to a retailer?

• How exactly do retailers factor topline sales into their decision to participate in utility programs?


What is Topline Sales?

• “Top Line Sales” is a reference to the gross sales or revenues of a company.

• The "top" reference relates to the fact that on a company's income statement, the first line at the top of the page is generally reserved for gross sales or revenue. • Program reimbursements for sales are not included in revenue

• A company that increases its revenues is said to be "growing its top line", or "generating top-line growth".

• This contrasts with net income (or net earnings per share), which is usually the bottom line of the company's income statement.


Topline Sales Impact Example

• Retailer must sell a minimum of 254 additional units just to get back to $697 total sales dollars generated before program

$697.00 $697.00

Top Line Sales $

Incremental sales lift+254 units

R l R il $6 97

Sales $ Generated $197.00

+254 units

Regular Retail $6.97

Promo Retail $1.97Discount $5.00 100 Units

sold100 Units

sold354 Units

sold


Gross Margin Impact Example

• Retailer is made whole on the Gross Margin line after discounts have been reimbursed on first 100 units and all additional units provide incremental GM $ G th Growth.

• Unfortunately, the rebate dollars are not able to be credited to Top Line Sales dollars.

IncrementalGross

Margin $ Generated

Incremental Gross

Margin lift on

+254 units

100 Units sold before

If only 100 Units sold

354 Units sold


Program during Program

Retailer Decision Tree: Year 1 (SKU Level)

Less than i t

Decline to participate ORRevise Strategy of Incentives to

How many units can be sold in the

Promotional time period at the

prior to Program

gymeet Necessary Revenue Dollar

level

preduced price? Same or Above

prior to Program

Same Above

Participate atAgreed Upon Incentive level and Allocation

Amount

Participate atAgreed Upon Incentive level and Negotiate Additional

Allocation Amount


Retailer Decision Tree: Additional Years (SKU Level)

Was the Previous Year

promotion revenue

Discontinue OR

No

promotion revenue neutral or better? Participate with Revised Incentive

Discontinue OR

How many units can be sold in the

Yes Less than Previous Year

Revise Strategy of Incentives to meet Previous Year Revenue or

Additional Promotion Opportunity

Next Year? Same or Above Previous Year

Continue atCurrent Incentives

d

Continue at Current Incentives

d

Same Above


and Same Allocation

and Negotiate Additional Allocation

Model Implementation


Required Data

• Need program tracking data at sku level for:• Sales goals

• MSRP

• Incentive amount (and any changes in incentive amount over the promotional timeframe)

• Program sales• Program sales

• Use the first three to estimate sales without program discount pricing

• Use program sales to calculate free ridership


Model Implementation

• Can estimate the program’s likely free ridership before the program year

• Likely free ridership is what the program will receive if each sku meets its sales goal and the program does not allow sales to exceed goals

• Actual free ridership is based on final program sales• Will be higher for skus that do not meet sales goal

• Will be lower for those that exceed goals


Use in Actual Evaluations

• Have used the model in two evaluations so far

• Results are not yet publicly available

• Used multiple methods for one evaluation and all came up with similar p pnumbers

• We are pursuing a patent on the model. We are pursuing a patent on the model.


Model Benefits

• Can calculate free ridership by• Bulb type (standard, specialty, fixtures, CFLs, LEDs)

• Retailer

• During special promotions compared to regular program pricing

• None of the existing lighting free ridership methods provide this level of detail


Program Design

• Can use results to identify changes to program implementation to minimize free ridership within existing program budgets.

• Tailor the product mix by bulb type

• Vary incentive levels by retailer and bulb type• Some retailer and bulb types have higher FR rates

• Need greater incentives to encourage purchase by non-free riders

• Maximize incentive dollars by matching incentive amount to bulb and retailer types


Questions or Comments?

Tami Buhr

Director of Survey Research

Opinion Dynamics

[email protected]

617-301-4654

Stan Mertz

Director of Retail OperationsDirector of Retail Operations

Applied Proactive Technologies

[email protected]

413 731 6546413-731-6546


The EE Story (Future)

• Future: Finding a new portfolio

– Lighting savings going down

– Some program saturation

– Need ‘new’ programs

• Whole home (HPwES, air sealing, etc)

• Systems work (HVAC systems, smart homes, etc)

• Behavior programs (changing the customer habit)

• Education

• Building Energy Codes (adoption, training and

compliance)

– Challenges

• Cost effectiveness (non-energy benefits not counted)

• More complex (contractors, systems, etc)

Presenter Contact Information

Stan Martz, Applied Proactive Technology –

[email protected]

Tami Buhr, Opinion Dynamics –

[email protected]