iso new england 2002 demand response program evaluation presentation of “major findings” to the...

ISO New England ISO New England 2002 Demand 2002 Demand

Response Program Response Program EvaluationEvaluation

Presentation of “Major Findings”Presentation of “Major Findings”ToTo

The ISO Demand Response The ISO Demand Response Working GroupWorking Group

Townsley Consulting GroupTownsley Consulting GroupFebruary 7, 2003February 7, 2003

2TOWNSLEY CONSULTING GROUP

Evaluation FocusEvaluation Focus

Benefit Cost AnalysisBenefit Cost Analysis Total Resource TestTotal Resource Test Sensitivity AnalysisSensitivity Analysis Congestion Cost AssessmentCongestion Cost Assessment Impact of Class 2 DR on ECPImpact of Class 2 DR on ECP

Retail Participant SurveyRetail Participant Survey Stakeholder Surveys i.e. PUCs, LSEsStakeholder Surveys i.e. PUCs, LSEs Review of Key ProcessesReview of Key Processes


Cost Benefit Analysis-Cost Benefit Analysis-Class 1Class 1

Total Resource Benefit Cost Ratios:Total Resource Benefit Cost Ratios:All New England (NE) 0.85All New England (NE) 0.85SWCT 0.94SWCT 0.94NE minus SWCT 0.32NE minus SWCT 0.32• The B/C ratio could be improved by The B/C ratio could be improved by

assuming more than a 1 year amortization assuming more than a 1 year amortization year of infrastructure costs.year of infrastructure costs.

• Significantly increasing the reliability benefit Significantly increasing the reliability benefit as represented by TMOR would be required as represented by TMOR would be required to achieve program cost effectiveness in to achieve program cost effectiveness in areas beyond SWCT.areas beyond SWCT.


Benefit Cost Analysis-Benefit Cost Analysis-Class 2Class 2

Total Resource Benefit Cost Ratio = Total Resource Benefit Cost Ratio = 0.530.53

The B/C cost ratio could be improved The B/C cost ratio could be improved substantially by increasing the substantially by increasing the average percentage of enrolled load average percentage of enrolled load reduced per called event hour from reduced per called event hour from 9% to in excess of 20%.9% to in excess of 20%.


Benefit Cost Sensitivity Benefit Cost Sensitivity Analysis Class 1 (SWCT)Analysis Class 1 (SWCT)

Reliability Benefit Value Benefit/Cost Reliability Benefit Value Benefit/Cost RatioRatio

$20 KW-month* 0.43$20 KW-month* 0.43

$63 KW-month ** 1.25$63 KW-month ** 1.25** Incremental value approved by the CT DPUC.Incremental value approved by the CT DPUC.

** Marginal value of ISO’s CT Supplemental RFP** Marginal value of ISO’s CT Supplemental RFP

The base case Value of Load Loss for retail The base case Value of Load Loss for retail customers is $ 11,151/ hr. Which is well customers is $ 11,151/ hr. Which is well within the generally accepted range of within the generally accepted range of $5K to $15K/hr.$5K to $15K/hr.


Benefit Cost Sensitivity Benefit Cost Sensitivity Analysis-Class 2Analysis-Class 2

• Extending the number of called event Extending the number of called event hours to include all hours where the ECP hours to include all hours where the ECP exceeds exceeds

$ 50/MWh would increase the total $ 50/MWh would increase the total resource cost benefit ratio to 1.13.resource cost benefit ratio to 1.13.

• Additionally, increasing the percentage of Additionally, increasing the percentage of the enrolled load responding on average the enrolled load responding on average each hour from 9% to 27% would further each hour from 9% to 27% would further increase the total resource cost benefit increase the total resource cost benefit ratio to 2.82.ratio to 2.82.


Congestion Price Duration Congestion Price Duration Impact on Class 2Impact on Class 2

# of hours Congestion** # of hours Congestion** AverageAverage

at Hours Costat Hours Cost >$200 >$100 >$50 $/MWh>$200 >$100 >$50 $/MWhBoston* 28 493 7,671 7,975 68.3Boston* 28 493 7,671 7,975 68.3SWCT * 136 642 2,010 3,028 47.5SWCT * 136 642 2,010 3,028 47.5NE ECP 8 53 1,045 35.6***NE ECP 8 53 1,045 35.6***

* NE ECP + Congestion Costs* NE ECP + Congestion Costs** Based on Jan-Nov 2002 data** Based on Jan-Nov 2002 data*** Based on Jan-Dec 2002 data*** Based on Jan-Dec 2002 data


Congestion Price Duration Congestion Price Duration Impact on Class 2Impact on Class 2

Hours Required to Achieve Program Cost Hours Required to Achieve Program Cost Effectiveness Effectiveness

as a Percent of Enrolled Loadas a Percent of Enrolled Load ___________Hours__________ ___________Hours__________ % MW % MW NE-ECP Boston CT SWCT NE-ECP Boston CT SWCT 5 4 5 4 1999 993 1020 593 1999 993 1020 593 10 7 816 406 257 21710 7 816 406 257 217 15 11 468 235 124 11115 11 468 235 124 111 20 15 306 156 66 6420 15 306 156 66 64 30 22 158 79 29 2830 22 158 79 29 28 40 29 92 45 21 2040 29 92 45 21 20 50 37 58 29 16 1550 37 58 29 16 15


Impact of Class 2 DR on Impact of Class 2 DR on ECPECP

Analysis of Supply Curve Data for a 200MW Load Analysis of Supply Curve Data for a 200MW Load Reduction.Reduction.

LowerLowerEvent Bid Price PriceEvent Bid Price PriceDay Hr. ECP Price Impact SavingsDay Hr. ECP Price Impact Savings6/3 14 $195 $179 $16 $101,4006/3 14 $195 $179 $16 $101,4006/3 15 $194 $179 $15 $ 94,4006/3 15 $194 $179 $15 $ 94,4006/3 16 $115 $102 $13 $ 83,3006/3 16 $115 $102 $13 $ 83,3008/14 14 $1,000 $1,000 --- ---8/14 14 $1,000 $1,000 --- ---8/14 15 $1,000 $1,000 --- ---8/14 15 $1,000 $1,000 --- ---8/14 16 $1,000 $1,000 --- ---8/14 16 $1,000 $1,000 --- ---Issue:Issue:• Price caps would seem to diminish the DR price Price caps would seem to diminish the DR price

impact savingsimpact savings


Retail Participant Survey Retail Participant Survey ResultsResults

All retail DR Program participants were All retail DR Program participants were e-mailed an comprehensive, automated e-mailed an comprehensive, automated Microsoft Word survey to ascertain their Microsoft Word survey to ascertain their opinions and thoughts regarding opinions and thoughts regarding program participation and to develop program participation and to develop information useful to future program information useful to future program planning.planning.

Approximately 17% of the participants Approximately 17% of the participants returned a completed survey document.returned a completed survey document.


Key Marketing Key Marketing InfluencerInfluencer

From whom do you prefer to receive information about the program(s)?

(check all that apply)

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Newspaper

Other please specify:

Other service provider

Trade Association

Through an information meeting withmultiple sponsors

Your state public utility commission

Competitive energy supplier(s)

Your local electric company

ISO New England

%

From whom


Information MeetingsInformation MeetingsDid you attend an informational meeting?

0%

10%

20%

30%

40%

50%

60%

70%

Yes I attended a meeting I chose not to attend I did not hear about an informational meeting

Percentage

Response


Key Communication Key Communication MethodsMethods

Percent Rating Communication Methods as Extremely or Very Desirable

(multiple answers)

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

In p

ers

on

co

nta

ct/

ac

co

un

tre

p E-m

ail

s

Te

ch

nic

al

Info

Me

eti

ng

s

Ge

ne

ral

Info

Me

eti

ng

s

Le

tte

r

Bro

ch

ure

s

Ne

ws

lett

ers

We

b s

ite

s

TV

Ra

dio

%

Communciation Method


Satisfaction with EnrollerSatisfaction with EnrollerWho led you through the sign-up/enrollment process? How satisfied were you

with the support they provided?

Tota

l Sele

ctin

g E

ntit

y

Tota

l Sele

ctin

g E

ntit

y

Tota

l Sele

ctin

g E

ntit

y

0%

10%

20%

30%

40%

50%

60%

70%

80%

Local electric company Competitive Energy Supplier Other Service Provider

Extremely Satisfied

Very Satisfied

Satisfied

Marginally Satisfied

Not Satisfied

no response

Total Selecting Entity

Entity who led through sign-up process

Level of satisfaction and total


Program Rules/Procedures – how Program Rules/Procedures – how receivedreceived

How did you receive the program rules and procedures?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

Both verbally and in writing Electronically (Email orWebsite)

In writing only Only verbally Not at all

Percentage

How program rules and procedures were received


Rating of Payment/Settlement Rating of Payment/Settlement ProcessProcess

How would you rate the program payment/settlement process?

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

No Comment

Don't know

Poor

Fair

Average

Good

Excellent

%

Rating


Event Notification Event Notification MethodMethod

Comparison of Notification Method Used vs. Preferred

e-mail

e-mail

automated phone call


pager

pager

fax

fax

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Used during 2002 Preference

e-mail


pager

fax

2002 vs. Prefered

Notification Method


Event Non-ParticipationEvent Non-ParticipationReason for not participating in an event after enrolling in the program (check all

that apply)

0% 10% 20% 30% 40% 50% 60% 70%

Expected paymentamount insufficient

Had not received paymentfor previous event(s)

Insufficient personnel

Plant already closed

Could not reduce load asplanned

Did not receive notice intime

%

Reason


Major Load Curtailment Major Load Curtailment ActionsActions

Major Action Taken to Reduce Load During an Event(multiple responses)

0% 5% 10% 15% 20% 25% 30% 35% 40% 45%

Other please specify

Adjusted temperature or shut offwater heating

Shed load via the EMS system

Shut down office/plant by

Adjusted temperature or shut offrefrigeration

Operated on-site generator

Adjusted temperature or shut offair conditioning

Reduced or shut off officeequipment

Halted or reduced productionprocess(es)

Turned off lights

%

Major Actions


Financial ImpactFinancial ImpactFinancial Impacts of Program

Yes Y

es

Yes

Yes

Yes

No

No

No

No

Don’t k

now

Don’t k

now

Don’t k

now

Don’t k

now

No c

om

ment

No c

om

ment

Not

applic

able

Not

applic

able

No

Don’t k

now

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

Incrementaldecrease -

operating revenue

Incrementalincrease - operatingexpense

Add't adverseimpact ofprogram

participation

Loss of amenities(e.g. comfort,convenience)

Add't benefits ofprogram

participation

Yes

No

Don’t know

No comment

Not applicable

Financial Impacts

Answer


Simple Payback Period Simple Payback Period RequiredRequired

What simple pay back period would your company require for an investment to improve your facility’s capability to manage load?

0%

5%

10%

15%

20%

25%

30%

35%

40%

Less than 1year

Less than 2years

Less than 3years

Less than 5years

Less than 7years

Less than 10years

No specificinvestment

criteria

Otherinvestment

criteria

Would notmake such an

investment

Percentage

Simple pay back period


Estimated Economic LossEstimated Economic Loss- Unplanned Power Outage- Unplanned Power Outage

Estimted Economic Loss From Unplanned Power Outage - 2 hrs during weekday afternoon

0%

2%

4%

6%

8%

10%

12%

14%

16%

18%

20%

$0 to $499 $500 to $999 $1,000 to$4,999

$5,000 to$9,999

$10,000 to$19,999

$20,000 to$39,999

$40,000 to$59,999

$60,000 to$79,999

$80,000 to$999,999

$100,000 ormore

Percentage

Range of Economic Loss


Installed EquipmentInstalled EquipmentEquipment Installed to Participate in DRP

0% 10% 20% 30% 40% 50% 60%

Load control devices forequipment

Load control devices forlighting

Interval sub-meters at majorloads

New on-site generators

Energy management system

None

Comm. Equip./Notificationtechnologies

Interval meter and/or pulseinitiator at service entrance

Internet-based meter-reading

Percentage

Equipment Installed


Future Program FeaturesFuture Program FeaturesDesirability of Possible Future Program Features

0%

10%

20%

30%

40%

50%

60%

Financing orincentives for

loadmanagementinfrastructure

tech.investments

Free facility loadmanagment

potentialassessments

Air emissionspermitting

assistance -standby

generators

Multi-yr paym’tguarantees

independent ofcurtailmentfrequency

Ability to tradeor sell monthlyDR capacity

credits

Reducedstandby

generator retailtariff back-up

reqs.

Ability toreselect new

programtype/format

monthly

Essential

Highly desirable

Desirable

Indifferent

Don't know

Undesirable

Program Feature

Desirabilty:


Overall SatisfactionOverall SatisfactionOverall Satisfaction With the Program

0%

5%

10%

15%

20%

25%

30%

35%

very satisfied somewhat satisfied satisfied somewhatdissatisfied

very dissatisfied

Percentage

Satisfaction


Satisfaction with ISO – NE Satisfaction with ISO – NE SupportSupport

Overall Satisfaction With ISO-NE Program Support

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

very satisfied somewhat satisfied satisfied somewhatdissatisfied

very dissatisfied

Percentage

Satisfaction


Stakeholder Survey-PUC Stakeholder Survey-PUC staffsstaffs

In person interviews were held with In person interviews were held with most PUC staffs in NE.most PUC staffs in NE.

The following feedback was received:The following feedback was received:• All felt that power supply issues All felt that power supply issues

were a high priority for their state, were a high priority for their state, but not all thought that DR was a but not all thought that DR was a high priority.high priority.



• Most PUC staffs did not see the 2002 Most PUC staffs did not see the 2002 DRP as successful based on: DRP as successful based on: The amount of enrolled load.The amount of enrolled load. Start-up problems (i.e., late rollout, Start-up problems (i.e., late rollout,

customer notification, and settlement customer notification, and settlement problems, etc.) problems, etc.)

Class 1 resource not tested.Class 1 resource not tested.• All felt the ISO was primarily All felt the ISO was primarily

responsible for reliability within the responsible for reliability within the Standard Market Zones.Standard Market Zones.



• Satisfaction with the LSE’s efforts in Satisfaction with the LSE’s efforts in marketing and communicating the marketing and communicating the benefits of DR range from excellent to benefits of DR range from excellent to unreliable.unreliable.

• Regional Barriers to DR deployment Regional Barriers to DR deployment identified:identified: Not easily understood by customers.Not easily understood by customers. Lack of technical support for customers.Lack of technical support for customers. Incentive payments too low.Incentive payments too low. Absence of residential and small C/I Absence of residential and small C/I

customer aggregation.customer aggregation.



• Comments were not favorable regarding the Comments were not favorable regarding the potential for pricing reforms to more potential for pricing reforms to more directly align wholesale and retail prices. directly align wholesale and retail prices. Some support exists for voluntary TOU Some support exists for voluntary TOU rates and load shedding program rates and load shedding program participation. Political and legislative participation. Political and legislative barriers to reforms were perceived as barriers to reforms were perceived as substantial.substantial.

• Most PUCs did not anticipate a significant Most PUCs did not anticipate a significant DR communication role, citing financial and DR communication role, citing financial and people resource constraints.people resource constraints.



• Recommendations included:Recommendations included: More specific goal setting for DR programs More specific goal setting for DR programs

i.e. the amount of load to be enrolled i.e. the amount of load to be enrolled reflecting sharp supply cost increases at reflecting sharp supply cost increases at various load levels.various load levels.

Achieve greater parity in market pricing Achieve greater parity in market pricing between supply and demand resources.between supply and demand resources.

Create an ISO/NEPOOL standing Create an ISO/NEPOOL standing committee on DR and related customer committee on DR and related customer issues which is equivalent in stature and issues which is equivalent in stature and resources to the Reliability and Markets resources to the Reliability and Markets committees. committees.


Stakeholder Survey-LSEsStakeholder Survey-LSEs

LSE’s (or distribution companies) were LSE’s (or distribution companies) were given the opportunity to participate in a given the opportunity to participate in a group discussion or respond to an e-mail group discussion or respond to an e-mail survey instrument.survey instrument.

The following feedback was received:The following feedback was received:• All respondents indicated power supply All respondents indicated power supply

issues were a high priority for them. issues were a high priority for them. One mentioned the targeting of Demand One mentioned the targeting of Demand Side Management Resources to reduce Side Management Resources to reduce load near substations with reliability load near substations with reliability concerns.concerns.



• Comments were quite divided regarding Comments were quite divided regarding the perceived success of the 2002 the perceived success of the 2002 Program. Concerns expressed included:Program. Concerns expressed included: Low level of load signed up.Low level of load signed up. Low payments to customers and settlement Low payments to customers and settlement

problems.problems. The low value of ICAP created via the The low value of ICAP created via the

program.program. Problems with establishing the baseline Problems with establishing the baseline

measurement levels and confusion measurement levels and confusion regarding customer activation date. regarding customer activation date.



• All respondents felt the ISO has the All respondents felt the ISO has the overall responsibility for zonal reliability.overall responsibility for zonal reliability.

• Marketing and Communication initiatives Marketing and Communication initiatives reported by respondents:reported by respondents: Letters sent to customers 200 KW and larger.Letters sent to customers 200 KW and larger. Account representatives called on customers.Account representatives called on customers. Hosted public forums and posted information Hosted public forums and posted information

on a website. on a website.

No respondent indicated they planned to do No respondent indicated they planned to do anything different in 2003. anything different in 2003.



• Problems encountered in marketing Problems encountered in marketing the program:the program: Customers were confused about whether Customers were confused about whether

to sign-up for Class 1 or Class 2 program.to sign-up for Class 1 or Class 2 program. Explaining ICAP and TMOR to customers Explaining ICAP and TMOR to customers

was difficult.was difficult. Customers seemed to be making modest Customers seemed to be making modest

enrollment commitments as a learning enrollment commitments as a learning step and to gain access to incentives for step and to gain access to incentives for infrastructure upgrades.infrastructure upgrades.



• Respondents were divided on whether Respondents were divided on whether or not program goals and objectives or not program goals and objectives were communicated clearly.were communicated clearly.

• Recommendations for the future:Recommendations for the future: Use the ISO’s website to communicate on Use the ISO’s website to communicate on

FAQs and other relevant program FAQs and other relevant program information.information.

Enhance ISO support to provide faster Enhance ISO support to provide faster turnaround on inquiries and settlement turnaround on inquiries and settlement questions.questions.

Improve event calling process.Improve event calling process.



• On the issue of the ISO’s involvement in On the issue of the ISO’s involvement in program design, administration, and marketing, program design, administration, and marketing, respondents offered the following comments:respondents offered the following comments: Reactivate the DR working group to resolve and Reactivate the DR working group to resolve and

address issues and provide input into planning.address issues and provide input into planning. ISO should be involved in planning and ISO should be involved in planning and

administration; marketing should be shared with administration; marketing should be shared with participants. participants.

Alternatively some felt that the ISO provided Alternatively some felt that the ISO provided insufficient administrative resources in 2002, and insufficient administrative resources in 2002, and therefore should outsource future program therefore should outsource future program administration.administration.


Review of Key ProcessesReview of Key Processes

During the course of the evaluation study, information During the course of the evaluation study, information emerged from survey data and interviews that emerged from survey data and interviews that indicated some administrative difficulties may have indicated some administrative difficulties may have arisen during the start-up and implementation of the arisen during the start-up and implementation of the 2002 DR Program.2002 DR Program.

Issues that surfaced as possible concerns included:Issues that surfaced as possible concerns included:• The quality of the information residing in the primary The quality of the information residing in the primary

customer information data base.customer information data base.• The validity of data utilized in the settlement process.The validity of data utilized in the settlement process.• The ability of the ISO to respond to and resolve in a The ability of the ISO to respond to and resolve in a

timely manner issues emanating from the enrollment, timely manner issues emanating from the enrollment, event notification and settlement processes.event notification and settlement processes.



To ascertain if problems in some To ascertain if problems in some administrative processes were administrative processes were contributing to stakeholder and retail contributing to stakeholder and retail customer dissatisfaction, the customer dissatisfaction, the information flow for the enrollment and information flow for the enrollment and settlement processes was reviewed settlement processes was reviewed with ISO staff and appropriate with ISO staff and appropriate stakeholders to understand the stakeholders to understand the information flow and the relevant information flow and the relevant history.history.



The review of the information flow for the The review of the information flow for the enrollment and settlement processes yielded enrollment and settlement processes yielded the following findings:the following findings:

• The staff assigned to handle initial processing The staff assigned to handle initial processing of the NX11c data was probably insufficient of the NX11c data was probably insufficient to handle the volume and initially lacked the to handle the volume and initially lacked the training necessary to evaluate and properly training necessary to evaluate and properly interpret the information. This resulted in interpret the information. This resulted in improper information being passed along in improper information being passed along in the enrollment process and to the primary the enrollment process and to the primary customer data base.customer data base.


Review of Key ProcessesReview of Key Processes• In late spring of 2002 the ISO implemented a In late spring of 2002 the ISO implemented a

web based solution to automate and web based solution to automate and streamline the NX11c enrollment process. As streamline the NX11c enrollment process. As the program enrollment process was well the program enrollment process was well underway the new customer information underway the new customer information database was being populated as time database was being populated as time permitted from the original database. This permitted from the original database. This required staff to rely on two separate required staff to rely on two separate databases for primary customer information databases for primary customer information including activation status. Also the web including activation status. Also the web based solution initially had data formatting based solution initially had data formatting incompatibilities with the settlement data incompatibilities with the settlement data base which went undetected for a short time. base which went undetected for a short time.



• The information management problems The information management problems and some incongruence in baseline and some incongruence in baseline calculations between the RETX system calculations between the RETX system and the ISO’s settlement system and the ISO’s settlement system resulted in several questions from resulted in several questions from customers and LSE’s on settlement customers and LSE’s on settlement matters. The absence of well understood matters. The absence of well understood protocols for handling inquiries of this protocols for handling inquiries of this nature and a shortage of available staff nature and a shortage of available staff resources led to long delays (months) in resources led to long delays (months) in resolving settlement questions.resolving settlement questions.



The ISO is currently undertaking the following The ISO is currently undertaking the following measures to address the process issues measures to address the process issues identified:identified:

• Increasing DR professional staff resources from Increasing DR professional staff resources from one part-time individual last spring to a full one part-time individual last spring to a full time manager and two professional support time manager and two professional support staff.staff.

• Implementing an “open solutions” software Implementing an “open solutions” software system, which will among other objectives, system, which will among other objectives, standardize baseline calculations for settlement standardize baseline calculations for settlement and customer information purposes.and customer information purposes.

• Substantially upgrade and formalize its Substantially upgrade and formalize its business procedures and protocols for DR business procedures and protocols for DR administration. administration.


AddendumAddendum

Additional Survey Additional Survey ResultsResults


Program Rules – easiness to Program Rules – easiness to understandunderstand

How Easy were the program rules and procedures to understand?

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

Extremely easy Very easy Easy Marginally easy Not easy

Percent

How Easy


Program Rules and Procedures Program Rules and Procedures –supplier–supplier

If you received written program rules and procedures, who supplied them?

0%

10%

20%

30%

40%

50%

60%

70%

Your local electriccompany

ISO - New England Competitive energysupplier(s)

Other Your state publicutility commission

I did not receivethem

Other serviceprovider

Percentage

From whom written program rules and procedures were received


Economic DRP GoalsEconomic DRP GoalsDoes your company have a specific economic goal or requirement for

participation in a demand response program event?

0%

10%

20%

30%

40%

50%

60%

70%

80%

Yes No Don’t know No comment

Percentage

Response


Preference for Economic vs. Preference for Economic vs. Emergency ProgramsEmergency Programs

Based on the generic demand response programs described , would you be more likely to:

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

Participate in either type ofprograms, depending onpotential for an economic

gain

Participate only in anEmergency type program

Participate only in anEconomic type program

skip to Question 7

Don’t know Not participate at all ineither program type, please

explain skip to nextsection

Sum of Percent

More like to:

iso new england 2002 demand response program evaluation presentation of “major findings” to the...

Documents

hours cost

program cost effectiveness

bc cost ratio

hours congestion

ecp price impact savings63

bc ratio

event hours

ne ecp congestion costs