event driven index - nerc analysis subcommitte… · event driven index 5 event driven index –...

3353 Peachtree Road NE Suite 600 North Tower

Atlanta GA 30326 404-446-2560 | wwwnerccom

Event Driven Index NERC Performance Analysis Subcommittee Approved by NERC Planning Committee March 6 2012 Approved by NERC Operating Committee March 7 2012

ii Event Driven Index ndash February 2012

This page intentionally left blank

Event Driven Index ndash February 2012 iii

Table of Contents

TABLE OF CONTENTS III

INTRODUCTION 1

SEVERITY RISK INDEX 2

EVENT DRIVEN INDEX 4

NEXT STEPS 6

APPENDIX I ndash SRI CALCULATION FUNDAMENTALS 7

Introduction

1 Event Driven Index ndash February 2012

Figure 1 Universe of Risk Concepts

Introduction The NERC Performance Analysis Subcommittee (former Reliability Metrics Working Group ndash RMWG) work has progressed since its formation and following the release of the initial reliability metric whitepaper in December 20071 In August 2010 the RMWG released its Integrated Bulk Power System Risk Assessment Concepts paper2 introducing new concepts such as the ldquouniverse of riskrdquo of the bulk power system shown in the Figure 1 In the concepts paper a method to assess ldquoevent-drivenrdquo risks was introduced and the Severity Risk Index (SRI) was established to quantify the impact of various events of the bulk power system The concepts paper and SRI refinement calculation were endorsed by NERCrsquos Operating (OC) and Planning Committees (PC) in September 2010 Subsequently a companion whitepaper3 Integrated Risk Assessment Approach ndash Refinement to Severity Risk Index was developed and approved by OC and PC in March 2011

This paper builds on previous SRI work towards establishing a single Event Driven Index (EDI) as shown inside the top circle of Figure 1

1 The initial reliability metrics whitepaper is located at httpwwwnerccomdocspcrmwgReliability_Metrics_white_paperpdf

2 httpwwwnerccomdocspcrmwgIntegrated_Bulk_Power_System_Risk_Assessment_Concepts_Finalpdf 3

httpwwwnerccomdocspcrmwgSRI_Equation_Refinement_May6_2011pdf

Event Driven Index (EDI)

Indicates Risk from Major System Events

Standards Statute Driven

Index (SDI)

Indicates Risks from Severe

Impact Standard Violations

Condition Driven Index (CDI)

Indicates Risk from Key Reliability

Indicators

Severity Risk Index

Event Driven Index ndash February 2012 2

Severity Risk Index As defined in the second approved whitepaper the SRI is a daily blended metric where transmission loss generation loss and load loss events are aggregated into a single value that represents risk to the system Each element (transmission generation and load loss) is weighted by the inventory for that element to rate significant events appropriately SRI values range from zero (a theoretical condition in which virtually no elements out of service) to 1000 (a theoretical condition in which every transmission line all generation units and all load lost (for more than 12 hours) across the system in a single day) The SRI was designed to be fungible and usable for the entirety of NERC as well as more granularly

Figure 2 captures the daily severity risk index value from 2008 to 2010 including the historic significant events used to pilot the calculation On a yearly basis these daily performance measurements are sorted in descending order to evaluate the year-on-year performance of the system Since there is significant disparity between normal days and events in terms of SRI values the curve is depicted using a logarithmic scale

Each yearrsquos data is sorted in descending order At the left-side of the curve the days that the system is severely stressed are plotted The central more linear portion of the curve identifies the routine daily performance Finally the far right side of the curve shows the values plotted for days when very little is impacting the system and substantially all lines generating units and load are in service nominally exceptionally good days Based on the SRI 2009 and 2010 had fewer extreme days than 2008 also 2010 had more exceptionally good days Routine daily performance appears to remain consistent across all three years

Figure 2 NERC Annual Daily Severity Risk Index (SRI) Sorted Descending with Historic Benchmark Days

Severity Risk Index


Also depicted in Figure 2 are the historically significant event day scores It also roughly characterizes the relationship between NERC event categories4 and SRI scores As the SRI increases the category or severity of the event increases

The event analysis process5 benefits from the SRI as it enables a numerical comparison of events By using this measure an event can be ranked by its severity By using the SRI the event analysis process can see which events to learn from and reduce which events to avoid and when resilience needs to be increased under high impact low frequency events As noted on the graph a different strategy ldquoLearn and Reducerdquo or ldquoAvoid or Increase Resiliencerdquo can be recommended

The event severity risk index (SRI) was developed to measure relative severity ranking of events based on event occurrence rate and their impact to the bulk power system As work progressed with this measure it was determined that the data sources best supported a metric which measures the impact of various components of the systems for a given day rather than specific events These components include load (as a proxy for the impact to customers) or loss of facilities (such as generators or transmission lines) These measures provide a quantitative approach to determine which events have more impact on bulk power system reliability In other words the metric is an integrated measurement system which classifies an eventrsquos impact on each of the components that are critical to the holistic performance of the bulk power system

4 httpwwwnerccomfilesERO_Event_Analysis_Process_Document_Version_1_Feb_2012pdf

5 httpwwwnerccompagephpcid=5|365

Event Driven Index


Event Driven Index Conceptually the Event Driven Index (EDI) can be considered as ldquosuccess ratesrdquo for daily performance experienced over a specific time period of interest at this time it is believed that quarterly or annually provides a stable indication of the success of the system in performing against the conditions it experiences It is the inverse of the SRI and EDI captures the average system availability (in‐service) rate under these various conditions It measures the state of being whole entire or in service over time The events include the combinations of transmission outages generation outages and duration‐factored load loss

The EDI is intended to be used as a historical measure neither a real‐time nor forward‐looking performance score As such the index is calculated using actual SRI data6 and reflects past experiences of the power system The existing data sources include Transmission Availability Data System (TADS) Generating Availability Data System (GADS) Events Reports (including OE‐417 Forms) and Event Analysis database

The value of the EDI can be determined based on the assessed performance from daily SRI information The EDI calculation and metric aggregation are described in Equation (1)

100 sum

(1)

where the SRI is a daily Severity Risk Index and the duration is a specific time period of interest such as quarterly or annually The value of EDI will range from 0 and 100 At this time neither SRI nor EDI incorporate any ldquoriskrdquo parameters As experience with both measures increases there may be opportunity to transition into a risk‐based view

Based on the SRI values in Figure 2 and Equation 1 quarterly EDI trending for the past three years is presented in Figure 3 No significant EDI trend changes are observed from 2008 to 2010

The EDI provides a basis for prioritization of events based on bulk power system integrity equipment performance andor engineering judgment Stakeholders can use the tool to measure the severity of these days compare against event analysis reports and evaluate disturbance history The relative severity ranking of events considers both the occurrence of an event and its impact in order to quantify the eventrsquos impact to reliability The index EDI increases if the accumulation of significant events (both in count and in magnitude) is reduced over a trending period

6 From a real time or forward‐looking point of view one can also create other indices linking real‐time or forecast variables and factoring in

probabilities As the industry gains experience with this backward‐looking metric it will continue to consider whether certain real‐time or forecast metrics are valuable indicators of the reliability of the bulk power system

Event Driven Index


The development of an Event Driven Index (EDI) aims to inform increase transparency and quantify the performance of the system It can serve also to identify the effectiveness of risk reduction andor mitigation actions For this work to be relevant and useful to stakeholders an open development process is used incorporating continuous improvement through leveraging industry expertise and technical judgment Other factors that impact SRI and EDI to be considered in the future include whether equipment operated as designed and resulted in loss of load from a reliability perspective (intentional and controlled load-shedding) and weather-related (or weather-exacerbated) events be grouped into a separate category The goal is to provide the industry meaningful trends of the bulk system performance and guidance on how they can improve reliability and support risk-informed decision making

Figure 3 NERC 2008-2010 Event Driven Index by Quarter

Figure 4 shows the change in the Event Driven Index from quarter to quarter in percent Although there appears to be a seasonal trend the percent change from quarter to quarter varies from less than 010 and there are no significant trends from 2008 to 2010

Figure 4 NERC ∆Event Driven Index by Quarter

9970

9975

9980

9985

9990

9995

10000

EDI

Quarter

-025

-015

-005

005

015

025

∆ED

I

Quarter

Next Steps


Next Steps NERC requested input from the Sandia National Laboratories and US Energy Information Administration (EIA) to identify and monitor areas for improving the value of indices such as SRI and EDI7 Perhaps in the future by linking the reliability data (transmissiongeneration outages disturbance event reports etc) and applying risk cluster and other statistical analysis reliability causal-effect relationships may merge The resulting model can be established to characterize and monitor the current state of the BPS reliability

This is an ambitious undertaking and it will continue to evolve as an understanding of what factors contribute to or indicate the level of reliability develops In the coming years analysis of the SRI and EDI will expand to build the causal-effect model with deeper analysis of initiating events and their impact

7 Review of BES Reliability Metrics (item 2f) and IRI Next Steps (3ai)

httpwwwnerccomdocspcPC_Meeting_Presentations_December_2011_Atlantazip NERC Operating Committee and Planning Committee Meeting December 13-14 2011 Atlanta GA

Appendix I ndash SRI Calculation Fundamentals


Appendix I ndash SRI Calculation Fundamentals8 With the Operating CommitteePlanning Committee OCPC approval of a risk assessment framework and concepts whitepaper9 at the September 2010 meeting Reliability Metrics Working Group RMWG applied the concepts and conducted sensitivity analyses to evaluate the impact of specific variables and coefficients on calculating a severity risk index This helped establish importance of each variable in calculating the blended metric

In Equation 2 the value of the severity is calculated based on impact of risk-significant events and the relative weightings

SRIevent = wL times (MWL) + wT times (NT) + wG times (NG) + wD times (HD) + wE times (NE) (2)

Where

SRIevent = severity risk index for specified event wL = weighting of load loss MWL = normalized MW of Load Loss in percent wT = weighting of transmission lines lost NT = normalized number of transmission lines lost in percent wG = weighting of generators lost NG = normalized number of generators lost in percent wD = weighting of duration of event HD = normalized duration of the event in percent wE = weighting of equipment damage and NE = normalized number of equipment damaged in percent

The RMWG recommends focusing transmission generation and load losses (NT NG and MWL) to develop severity risk curves initially it enhanced load loss as a method of incorporating consequential damage with the expectation that damaged equipment would impact the promptness of restoration The following risk factors identified in the concepts document will be reviewed and updated periodically as a part of ongoing open improvement process

1 Equipment damage (NE) ndash Due to infrequent occurrences this factor will be considered later once more data becomes available

2 Duration of transmission loss and generation loss ndash The duration of transmission facility loss and generation facility loss will be considered in the future since their reliability impact is relatively small compared with the facility loss itself

8 Material used from httpwwwnerccomdocspcrmwgSRI_Equation_Refinement_May6_2011pdf

9 httpwwwnerccomdocspcrmwgIntegrated_Bulk_Power_System_Risk_Assessment_Concepts_Finalpdf



However the RMWG believes the duration of load loss should be included in the Severity Risk Index (SRI) since it directly impacts end-user customers and may act as an indicator of consequential damage

Therefore the refined SRI equation is

SRIevent = (RPL) times wL times (MWL) + wT times (NT) + wG times (NG) (3)

Where

SRIevent = severity risk index for specified event (assumed to span one day) wL = 60 weighting of load loss MWL = normalized MW of Load Loss in percent wT = 30 weighting of transmission lines lost NT = normalized number of transmission lines lost in percent wG = 10 weighting of generators lost NG = normalized number of generators lost in percent RPL = load Restoration Promptness Level RPL = ⅓ if restoration lt 4 hours RPL = ⅔ if 4 le restoration lt 12 hours RPL = 1 if restoration ge 12 hours

The value of SRIevent will range from 0 to 1 The loss of load due to transmission-related events is weighted the highest (60) since it directly indicates an inability to deliver load to delivery points per ALR610 The transmission outages are ranked second (30) which reveals inability to meet ALR1 ALR2 and ALR3 characteristics Generation outages are placed third (10) because the generation outages have less impact to the grid since operating reserves are allocated to preserve load and generation balance

The normalized MW of Load Loss in percent MWL is equal to the total MW loss divided by coincident daily peak load The daily peak load is aggregated at NERC interconnection or regional level

10 httpwwwnerccomdocspcDefinition-of-ALR-approved-at-Dec-07-OC-PC-mtgspdf



The normalized number of transmission lines lost in percent NT is equal to the number of AC transmission lines lost weighted by their average line MVA ratings and the total of number of AC circuits at each voltage class obtained from TADS inventory report The following average line MVA ratings are used to measure different impact in terms of voltage classes

bull 230 kV ndashgt 700 MVA bull 345 kV ndashgt 1300 MVA bull 500 kV ndashgt 2000 MVA bull 765 kV ndashgt 3000 MVA

For example if an event in the Eastern Interconnection resulted in the loss of 100 AC lines rated at 230 kV 40 lines at 345 kV 10 lines at 500 kV and 3 lines at 765 kV for this event

Where 2708 is the total number of AC circuits at 230 kV in the Eastern Interconnection 1078 at 345 kV and 344 at 500 kV and 33 at 765 kV

For presentation purposes the SRI values were multiplied by 1000 and called SRIindex or the indexed SRI scorersquos equation is

SRIindex = 1000 times SRIevent (3)

Table of Contents

Introduction

Severity Risk Index

Event Driven Index

Next Steps

Appendix I ndash SRI Calculation Fundamentals7F

ii Event Driven Index ndash February 2012

This page intentionally left blank


Table of Contents


INTRODUCTION 1



NEXT STEPS 6


Introduction











Index (SDI)





Indicators

Severity Risk Index






Severity Risk Index







Event Driven Index





100 sum

(1)






Event Driven Index






9970

9975

9980

9985

9990

9995

10000

EDI

Quarter

-025

-015

-005

005

015

025

∆ED

I

Quarter

Next Steps











Where












Where













Table of Contents

Introduction

Severity Risk Index

Event Driven Index

Next Steps



Table of Contents


INTRODUCTION 1



NEXT STEPS 6


Introduction











Index (SDI)





Indicators

Severity Risk Index






Severity Risk Index







Event Driven Index





100 sum

(1)






Event Driven Index






9970

9975

9980

9985

9990

9995

10000

EDI

Quarter

-025

-015

-005

005

015

025

∆ED

I

Quarter

Next Steps











Where












Where













Table of Contents

Introduction

Severity Risk Index

Event Driven Index

Next Steps


Introduction











Index (SDI)





Indicators

Severity Risk Index






Severity Risk Index







Event Driven Index





100 sum

(1)






Event Driven Index






9970

9975

9980

9985

9990

9995

10000

EDI

Quarter

-025

-015

-005

005

015

025

∆ED

I

Quarter

Next Steps











Where












Where













Table of Contents

Introduction

Severity Risk Index

Event Driven Index

Next Steps


Severity Risk Index






Severity Risk Index







Event Driven Index





100 sum

(1)






Event Driven Index






9970

9975

9980

9985

9990

9995

10000

EDI

Quarter

-025

-015

-005

005

015

025

∆ED

I

Quarter

Next Steps











Where












Where













Table of Contents

Introduction

Severity Risk Index

Event Driven Index

Next Steps


Severity Risk Index







Event Driven Index





100 sum

(1)






Event Driven Index






9970

9975

9980

9985

9990

9995

10000

EDI

Quarter

-025

-015

-005

005

015

025

∆ED

I

Quarter

Next Steps











Where












Where













Table of Contents

Introduction

Severity Risk Index

Event Driven Index

Next Steps


Event Driven Index





100 sum

(1)






Event Driven Index






9970

9975

9980

9985

9990

9995

10000

EDI

Quarter

-025

-015

-005

005

015

025

∆ED

I

Quarter

Next Steps











Where












Where













Table of Contents

Introduction

Severity Risk Index

Event Driven Index

Next Steps


Event Driven Index






9970

9975

9980

9985

9990

9995

10000

EDI

Quarter

-025

-015

-005

005

015

025

∆ED

I

Quarter

Next Steps











Where












Where













Table of Contents

Introduction

Severity Risk Index

Event Driven Index

Next Steps


Next Steps











Where












Where













Table of Contents

Introduction

Severity Risk Index

Event Driven Index

Next Steps







Where












Where













Table of Contents

Introduction

Severity Risk Index

Event Driven Index

Next Steps







Where













Table of Contents

Introduction

Severity Risk Index

Event Driven Index

Next Steps










Table of Contents

Introduction

Severity Risk Index

Event Driven Index

Next Steps


event driven index - nerc analysis subcommitte… · event driven index 5 event driven index –...

Documents