measuring the resilience of energy distribution …...1 chapter one measuring the resilience of...

C O R P O R A T I O N

Measuring the Resilience of Energy Distribution Systems

Henry H. Willis and Kathleen Loa

Limited Print and Electronic Distribution Rights

This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited. Permission is given to duplicate this document for personal use only, as long as it is unaltered and complete. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial use. For information on reprint and linking permissions, please visit www.rand.org/pubs/permissions.html.

The RAND Corporation is a research organization that develops solutions to public policy challenges to help make communities throughout the world safer and more secure, healthier and more prosperous. RAND is nonprofit, nonpartisan, and committed to the public interest.

RAND’s publications do not necessarily reflect the opinions of its research clients and sponsors.

Support RANDMake a tax-deductible charitable contribution at

www.rand.org/giving/contribute

www.rand.org

For more information on this publication, visit www.rand.org/t/RR883

Published by the RAND Corporation, Santa Monica, Calif.

© Copyright 2015 RAND Corporation

R® is a registered trademark.

iii

Preface

As policymakers continue to consider the complex risks from natural disasters, terrorism, aging infrastructure, and climate change, resilience is a topic that is likely to receive increasing atten-tion. If energy infrastructure’s resilience is to improve, better use of metrics will be crucial to guiding planning and evaluating progress. This report reviews literature on metrics for energy system resilience to help the U.S. Department of Energy develop a framework for evaluating and improving the resilience of energy systems for use in its ongoing efforts to draft the first Quadrennial Energy Review.

The report was sponsored by the U.S. Department of Energy, Office of Energy Policy and Systems Analysis. It should be of interest to policymakers, energy industry professionals, and researchers interested in understanding how to manage resilience of infrastructure systems.

The RAND Environment, Energy, and Economic Development Program

The research reported here was conducted in the RAND Environment, Energy, and Economic Development Program, which addresses topics relating to environmental quality and regula-tion, water and energy resources and systems, climate, natural hazards and disasters, and eco-nomic development, both domestically and internationally. Program research is supported by government agencies, foundations, and the private sector.

This program is part of RAND Justice, Infrastructure, and Environment, a division of the RAND Corporation dedicated to improving policy and decisionmaking in a wide range of policy domains, including civil and criminal justice, infrastructure protection and homeland security, transportation and energy policy, and environmental and natural resource policy.

Questions or comments about this report should be sent to the project leader, Henry H. Willis ([email protected]). For more information about the Environment, Energy, and Economic Development Program, see http://www.rand.org/energy or contact the director at [email protected].

v

Acknowledgments

The authors wish to thank Debra Knopman (RAND Corporation), Keith Crane (RAND Cor-poration), Eric Vugrin (Sandia National Laboratory), and Ross Guttromson (Sandia National Laboratory) for the discussions and suggestions they provided while this work was completed.

vii

Contents

Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iiiAcknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vFigures and Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix

CHAPTER ONE

Measuring the Resilience of Energy Distribution Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

CHAPTER TWO

Defining Resilience for Infrastructure Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

CHAPTER THREE

A Framework for Organizing Resilience Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

CHAPTER FOUR

Existing Metrics for Resilience of Electricity, Refined Oil, and Natural Gas Systems . . . . . . . . . . . . 9

CHAPTER FIVE

Developing Metrics for Energy Resilience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23Improve Collection and Management of Data on Inputs and Capacities at the Facility and

System Levels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23Develop Better Measures of Capabilities at the System and Regional Levels . . . . . . . . . . . . . . . . . . . . . . . . . . . 23Improve Understanding of How Capabilities and Performance Translate to Outcomes at

the Regional and National Levels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

ix

Figures and Tables

Figures 2.1. Resilience Describes the State of Service from a System in Response to a Disruption . . . . . 4 2.2. Different Systems Will Have Different Resilience to the Same Disruption . . . . . . . . . . . . . . . . . 5 2.3. Different Responses Will Lead to Different Resilience at Different Costs . . . . . . . . . . . . . . . . . . . 5 2.4. Resilience of a System Also Depends on the Timescale Considered . . . . . . . . . . . . . . . . . . . . . . . . . 6 3.1. Metrics Can Serve Both Operational and Strategic Decisionmaking . . . . . . . . . . . . . . . . . . . . . . . . 8 4.1. Summary of Reviewed Literature on Energy Resilience Metrics, by Sector . . . . . . . . . . . . . . . 10 4.2. Summary of Metrics Identified, by Resolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 4.3. Summary of Facility/System Metrics Identified, by Type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 4.4. Summary of System/Region/Nation Metrics Identified, by Type . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

Tables 4.1. Overview of Literature Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 4.2. Energy Resilience Metrics for Electricity Systems at the Facility/System Level . . . . . . . . . . . . 14 4.3. Energy Resilience Metrics for Electricity Systems at the System/Region Level . . . . . . . . . . . . 17 4.4. Energy Resilience Metrics for Electricity Systems at the Region/Nation Level . . . . . . . . . . . . 19 4.5. Energy Resilience Metrics for Oil and Natural Gas Systems at the Facility/System

Level . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 4.6. Energy Resilience Metrics for Oil and Natural Gas Systems at the System/

Region Level . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 4.7. Energy Resilience Metrics for Oil and Natural Gas Systems at the Region/Nation

Level . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

1

CHAPTER ONE

Measuring the Resilience of Energy Distribution Systems

The U.S. economy depends on reliable and affordable distribution of energy. However, energy distribution systems are vulnerable to a diverse and dynamic set of disruptions. Sev-eral events over the past few years highlight the range of challenges that energy distribution systems need to address and how those challenges are evolving due to climate change and new technologies.

In 2012, Superstorm Sandy left more than 8.5 million customers without power, with out-ages persisting more than one week (U.S. Department of Energy, 2012a). While communities recovered, residents faced shortages of gasoline that persisted during the same period (National Association of Convenience Stores, 2013). Extreme cold spells in the Northeast during the winter of 2014 sharply increased demand for natural gas, leading the prices of gas and elec-tricity to rise (U.S. Energy Information Association, 2014). Furthermore, in 2013, Pacific Gas and Electric Company’s Metcalf transmission substation in San Jose, California, suffered a complex attack involving the physical damage of transformers and the systematic disabling of both emergency response communications and supervisory control and data acquisition systems, highlighting the vulnerability of the power grid to both physical and cyber sabotage ( Memmott, 2014).

Events like these have motivated the federal government to develop a coordinated strat-egy to improve infrastructure performance, security, and resilience. Specifically, President Barack Obama has called upon the federal government to (1) advance a national unity of effort to strengthen and maintain secure, functioning, and resilient critical infrastructure (White House, 2013) and (2) strengthen U.S. energy policy by addressing challenges faced by the nation’s energy infrastructure (White House, 2014).

As part of the response to these mandates and as part of the 2014 Quadrennial Energy Review, the Department of Energy asked the RAND Corporation to develop a framework for measuring the resilience of energy distribution systems and to summarize the state of metrics for resilience of the electric power, refined oil, and natural gas distribution systems.

This report summarizes the concepts addressed by measures of resilience (Chapter Two), describes a framework for organizing alternative metrics used to measure resilience of energy distribution systems (Chapter Three), and reviews the state of metrics for resilience of such systems (Chapter Four).

3

CHAPTER TWO

Defining Resilience for Infrastructure Systems

Resilience has been defined many ways. Consider, for example, the following definitions from engineering literature, policy directives, and the academic community:

Resilience is the ability of the system to withstand a major disruption within acceptable degradation parameters and to recover within an acceptable time and composite costs and risks. (Haimes, 2009)

Resilience is the ability to prepare for and adapt to changing conditions and withstand and recover rapidly from disruptions. . . . [It] includes the ability to withstand and recover from deliberate attacks, accidents, or naturally occurring threats or incidents. (White House, 2013)

Resilience is the ability to prepare and plan for, absorb, recover from, and more success-fully adapt to adverse events. (Committee on Increasing National Resilience to Hazards and Disasters; Committee on Science, Engineering, and Public Policy; and The National Academies, 2012)

While distinctions exist among definitions, comparing them reveals four aspects of the system being addressed.

First, resilience describes the state of service being provided by a system in response to a disruption. When assessing resilience, key questions would be whether the service has been degraded, how much of the service has been degraded, how quickly the service has been restored, and how completely the service has been restored. Therefore, resilience does not describe a dichotomous state of whether or not a disruption has occurred. Rather, resilience describes the degree of disruption across multiple dimensions, which could include type, qual-ity, time, and geography of service provision. Figure 2.1 illustrates a notional response of an energy system to a disruption. Percentage of service provided, reflected along the y-axis, could be measured in terms of electricity delivered, gallons of fuel shipped, economic output gener-ated by energy, or other metrics that will be discussed in subsequent chapters. The disruption could be a natural disaster, industrial accident, or terrorist attack. The time of disruption and rate of service decline would depend on the nature of the event, design of the system, and mode with which the system is operated. The duration of disruption (measured in time and reflected along the x-axis), along with the rate and extent of recovery, would depend on these same fac-tors. Recovery may not be complete, as illustrated in the Figure 2.1 example, in which service delivery only recovers to 90 percent of its original level.

4 Measuring the Resilience of Energy Distribution Systems

Second, the state of a system depends on how it was designed and how it is operated. These choices influence whether and how service is degraded during a disruption, how quickly it recovers, and how completely it recovers. For example, an electricity grid system that is designed with more redundancy, operated with more contingencies for backup, and designed with recovery in mind (Figure 2.2, System B) might experience a lesser and briefer disruption and, if so, would be more resilient than a system that has less redundancy, has fewer backups, and is more difficult to rebuild (Figure 2.2, System A).

Third, different responses will lead to different resilience at different costs. For example, with additional resources, it may be possible to rebuild an electricity grid after a disaster with more-efficient equipment, and as a result, the quality of service provided after recovery could exceed the original level of service provided (Figure 2.3, System B, Response 2).

Finally, resilience of a system also depends on the timescale. If recovery of a grid places equipment where it was and as it was designed, over a period of years, the system may expe-rience repeated disruptions if climate change leads to greater frequency of intense flooding (Figure 2.4, Response 1). If the system is continually maintained and upgraded, the service provided could improve, but at a cost (Figure 2.4, Response 2). However, if maintenance and upgrades are not made, operations might be cheaper but service could be expected to decline in the future (Figure 2.4, Response 3).

Figure 2.1Resilience Describes the State of Service from a System in Response to a Disruption

RAND RR883-2.1

0

20

40

60

80

100

120

Time (days)

Perc

enta

ge

of

serv

ice

pro

vid

ed

Defining Resilience for Infrastructure Systems 5

Figure 2.2Different Systems Will Have Different Resilience to the Same Disruption

RAND RR883-2.2

0

20

40

60

80

100

120

Time (days)

Perc

enta

ge

of

serv

ice

pro

vid

ed System A

System B

Figure 2.3Different Responses Will Lead to Different Resilience at Different Costs

RAND RR883-2.3

0

20

40

60

80

100

120

Time (days)

Perc

enta

ge

of

serv

ice

pro

vid

ed System A, Response 1

System B, Response 1

System B, Response 2


Our review of definitions finds additional concepts that are sometimes included in definitions of resilience. Some of these are redundant; others distinguish important system characteristics. Examples are reliability, robustness, recoverability, sustainability, hardness, vulnerability, fault tolerance, and redundancy. While relevant, these additional terms are not used consistently. Reconciling the competing definitions of resilience in the literature is a dif-ficult and not terribly productive task. Instead, when attempting to define metrics of resilience in the context of the Quadrennial Energy Review, it is more important to capture the relevant aspects of service delivery, system design, system operations, disruptions, costs, and timescale.

Figure 2.4Resilience of a System Also Depends on the Timescale Considered

RAND RR883-2.4

0

20

40

60

80

100

120

Time (years)

Perc

enta

ge

of

serv

ice

pro

vid

ed System A, Response 1

System A, Response 3

System A, Response 2

7

CHAPTER THREE

A Framework for Organizing Resilience Metrics

We track metrics to be able to keep score, to tell whether goals have been met or whether success has been achieved. We track metrics to improve quality, to tell where improvements are possible and whether progress is being made. We also track metrics simply to account for resources, to tell whether budgets are being met and to know where assets reside.

Metrics of resilience are used for many purposes and at many levels. Some of the reasons for metrics are more relevant to a federal perspective and others to a local or facility perspec-tive. For example, at a national or regional level, it may be important to know how resilience affects economic output or economic damage stemming from disasters. For a refinery operator, it may be more important to know how many spare parts are in stock and what options exist for backup power generation.

There is no single set of metrics that supports all decisionmaking needs; rather, each purpose may need a unique set of metrics. But across levels of decisionmaking, the metrics ought to be organized within a consistent measurement framework. Logic models provide a consistent framework for organizing metrics in the fields of program evaluation and qual-ity improvement (Rogers et al., 2000; Greenfield, Williams, and Eiseman, 2006). From an operational perspective, a logic model explains how activities, budgets, and people (i.e., inputs) ultimately contribute to desired outcomes. From a strategic perspective, a logic model explains which inputs are needed to support strategy. From either perspective, a hierarchy of metrics exists to connect inputs to outcomes and improve understanding about how to achieve out-comes more effectively and efficiently (Figure 3.1).

The building blocks of resilience are inputs, which define what is available to support resil-ience. In the context of energy systems, examples of inputs include budgets, equipment, spare parts, and personnel to support recovery operations. On their own, these inputs do not provide resilience unless they are organized to support functions or tasks.

In a logic model framework, the ways in which inputs are organized to support resil-ience are called capacities. Examples of capacities for energy systems include response teams capable of repairing equipment, recovery plans that can be implemented following a disaster, or advanced technologies that can be used to reroute power and reconstitute portions of a grid during disruptions. Having these capacities in place is not the same thing as being able to use them, however.

Capability metrics reflect how well capacities can serve a system when they are needed. Ultimately, capability metrics describe how proficiently tasks can be performed. Examples include the ability to detect leaks or outages, to repair damaged power lines or pipelines, or to restore power outages.


Capabilities are ultimately desired because they improve system performance. Performance metrics describe what is produced by an engineered system. In the context of energy systems, examples of metrics include the amount of energy delivered or operating characteristics of the system, such as efficiency, reliability, fault tolerance, sustainability, or robustness.

In the end, the performance of energy systems depends on how the systems generate the outcomes that society is seeking to achieve. Resilience of energy systems can be measured by many outcomes, such as reduced damage from disasters, increased economic activity, or reduced deaths and injuries from disasters.

The literature on measuring resilience contains examples of metrics from each of these categories: inputs, capacities, capabilities, performance, and outcomes. When selecting metrics for resilience, it is useful to understand the availability and maturity of metrics that exist across these categories.

Figure 3.1Metrics Can Serve Both Operational and Strategic Decisionmaking

RAND RR883-3.1

Operational Perspective

Strategic Perspective

PerformanceCapabilitiesCapacitiesInputs Outcomes

What is produced?

What tasks can be performed?

How are inputs organized?

What is available?

What is achieved?

Examples• Energy delivery• Ef�ciency• Reliability• Hardness• Robustness• Sustainabiity

Examples• Outage

detection• Line repair• Backup delivery• Outage

restoration

Examples• Response teams• Plans• Aid agreements• Smat-grid tech

Examples• Budgets• Equipment• Number of

spare parts• Number of

generators• Number of

workers

Examples• Increased

economic activity

• Reduced costs and damage

• Improved human welfare

9

CHAPTER FOUR

Existing Metrics for Resilience of Electricity, Refined Oil, and Natural Gas Systems

To better understand how industry, governments, and communities assess the resilience of energy systems, we reviewed published reports and peer-reviewed journal articles to find exam-ples of metrics that are being discussed and used. We analyzed studies from 1997 to 2014, identified by searching several commonly used databases for terms related to measurement, resilience, and energy (see Table 4.1).

The literature search identified 58 papers and reports related to measuring energy system resilience (see Figure 4.1), and we reviewed each paper to identify resilience metrics. Tables 4.2, 4.3, and 4.4 list examples of metrics that were discussed in these papers.1 The 154 metrics that we identified illustrate the types of resilience metrics that are used across the energy industry. The units used and concepts tracked by metrics vary at each level of the hierarchy. At the input level, metrics tend to describe the amount of energy produced, transmitted, or stored or the number of people, facilities, or equipment available to support these activities. At the capacity level, metrics describe the existence and extent of systems, policies, and organizations in place to support energy capabilities. At the capability level, metrics describe the potential for energy systems to provide sources or factors that determine that capability. At the performance level, metrics describe the quality, amount, and efficiency of the services being provided by energy systems. Finally, the outcome level describes how energy influences aspects of societal welfare through health, safety, and the economy. Examples and descriptions of specific metrics can be found in the literature cited for each exemplary metric.

1 The citations listed in Tables 4.2, 4.3, and 4.4 are instances where the metric can be found in the literature. In some cases, the same metric was listed in several papers. We did not attempt to catalog all citations for each metric. As a result, we have also not cited in these tables every paper that was reviewed as part of this study. The full list of papers reviewed is included in the bibliography.

Table 4.1Overview of Literature Review

Criteria Details of Literature Review

Databases searched Web of Science, GreenFile, Science.gov, Google, Google Scholar

Years included 1997–2014

Search terms used distribution, electrical power, electricity, energy, fuel, gasoline, generation, grid, indicator, index, measure, metric, natural gas, nuclear, petrochemical, petroleum, pipelines, power, production, redundancy, reliability, resilience, transmission, vulnerability


The review of metrics provides examples of the many approaches that are being used to measure the resilience of energy systems. To better understand the range of metrics that exist for energy resilience, we categorized metrics by three dimensions: resolution, type, and maturity.

Resolution refers to the scale of the system being described. Metrics are collected at differ-ent resolutions based on who is monitoring them and why. The literature we reviewed focused on metrics for individual facilities, for systems owned by single firms,2 and for regions in which several systems exist, as well as from a national perspective. In practice, there is some semantic ambiguity about whether a metric provides perspective for a facility or a system, a system or a region, or a region or the nation. To reflect this ambiguity, our categorizations in Tables 4.2, 4.3, and 4.4 present metrics for electricity systems at a facility/system level, system/region level, and region/nation level, respectively. Tables 4.5, 4.6, and 4.7 present metrics for oil or gas sys-tems at the same levels.

The second dimension, type, refers to where the metric fits within the organizing frame-work described in Chapter Three. The extent to which measures exist for inputs, capacities, capabilities, performance, and outcomes also depends on how measures are being used. While planning and evaluation efforts are most interested in capabilities, performance, and out-comes, resource allocation and accountability efforts also need to track the status of inputs and capacities.

2 In this context, we are using the word system to refer to a collection of infrastructure that operates together to provide a coordinated service. Examples of systems are portions of the electricity grid, pipeline networks, or railway networks. Depending on the context, a system could be operated by a single company or authority. In other contexts, systems could be operated through coordination by multiple firms.

Figure 4.1Summary of Reviewed Literature on Energy Resilience Metrics, by Sector

RAND RR883-4.1

0

5

10

15

30

35

40

Sector described

Oil and natural gas Electricity General energy, or both

Nu

mb

er o

f p

aper

s

25

20

Existing Metrics for Resilience of Electricity, Refined Oil, and Natural Gas Systems 11

Maturity refers to how well defined the metrics are, how systematically they are collected, and how well they are organized. Our assessment of maturity reflects the practical challenges that extend beyond defining metrics and that can arise when collecting and managing data. We categorized metrics into three levels of maturity:

• Low—Metrics are mentioned in the literature but not consistently defined, and, if col-lected, only sporadically or inconsistently.

• Medium—Metrics are collected using well-defined methods but collected sporadically.• High—Metrics are collected using well-defined methods and collected at the appropriate

timescale.

By categorizing the metrics in this way, we drew three observations about the state of measur-ing resilience of energy systems.

First, there are more metrics for electricity systems (105 in the reviewed literature) than for systems for oil or natural gas (67 metrics).3 The difference in numbers of metrics is a result of two factors. First, as Figure 4.1 illustrates, we identified fewer papers about oil and gas systems than about electricity systems. Second, the papers about electricity systems covered more aspects of resilience than the papers about oil and gas systems. Greater attention to the resilience of electricity systems is not surprising, given that the electricity grid infrastructure (e.g., transmission and distribution lines) is more exposed and vulnerable to disruptions than oil and gas infrastructure (e.g., pipelines and storage tanks).

Second, the literature pays more attention to metrics for the more detailed levels of facili-ties and systems. Almost half of the metrics identified (70 of 154) describe aspects of systems at the facility and system levels (see Figure 4.2). Many more types of inputs and capacities are measured as part of inventory and accounting processes across the energy industry than were identified in the literature.

While there is greater attention to metrics at the facility/system level, these metrics pre-dominantly described inputs, capabilities, and performance (63 of 70 shown in Figure 4.3). Only one outcome metric was identified for the facility/system level, a component of a loss damage index describing damage caused by fire. Furthermore, most of these metrics (53 of 70) were judged to be of low or medium maturity. One possible explanation for this is that metrics at the facility and system levels are collected and managed by different entities that are more directly responsible for managing resources to improve performance than for support-ing broader societal outcomes. In addition, each entity may have its own reasons for collecting metrics and may have crafted definitions of metrics that most suit its own needs. Therefore, there may be opportunities to standardize definitions of metrics collected at the facility and system levels and potentially create a systematic means of collecting and aggregating data across facilities and systems if efficient means of doing so can be implemented.

3 Although only 154 metrics were identified, some metrics are relevant to more than one sector. Thus, the total mentioned here is 172.


Figure 4.2Summary of Metrics Identified, by Resolution

RAND RR883-4.2

0

10

20

30

60

70

80

Resolution of metrics

Facility/system System/region Region/nation

Res

olu

tio

n

50

40

Figure 4.3Summary of Facility/System Metrics Identified, by Type

RAND RR883-4.3

0

5

10

15

30

35

40

Type of metric

Typ

e

25

20



Third—and in contrast to the facility/system level—regional and national metrics focus more on aspects of performance and outcomes. We identified 50 performance and outcome metrics at these levels, and 22 of the 23 outcome metrics were at the system/region/nation level (see Figure 4.4). Generally, performance measures existed at the system/region level and were of medium to high maturity. In comparison, outcome measures were identified at both the system/region and region/nation levels, but they were judged to be of low or medium maturity. Furthermore, the literature did not clearly describe how the performance of systems contributed to changes in outcomes. This raises questions about whether it is possible to effec-tively manage energy systems to achieve desired outcomes by tracking performance metrics. As a result, understanding how infrastructure performance contributes to societal outcomes remains an active area of research. This suggests that opportunities may exist to improve track-ing and use of outcome data.

In summary, the metrics from our literature review, presented in Tables 4.2 through 4.7, present a complex picture of how resilience is managed and measured in energy systems. While many metrics exist, there is no single metric or set of metrics for each purpose. Different met-rics are needed to understand resilience at different levels of energy systems, and opportunities exist to improve metrics for each purpose.

Figure 4.4Summary of System/Region/Nation Metrics Identified, by Type

RAND RR883-4.4

0

5

10

15

25

30

Type of metric

Typ

e

20



Tab

le 4

.2En

erg

y R

esili

ence

Met

rics

fo

r El

ectr

icit

y Sy

stem

s at

th

e Fa

cilit

y/Sy

stem

Lev

el

Inp

uts

Cap

acit

ies

Cap

abili

ties

Perf

orm

ance

Ou

tco

mes

Ener

gy

feed

sto

ck (

H)

(McC

arth

y, O

gd

en, a

nd

Sp

erlin

g, 2

007

)

Ener

gy

no

t su

pp

lied

(L)

(B

ran

cucc

i Mar

tín

ez-

An

ido

et

al.,

2012

)

Ener

gy

sto

rag

e (L

) (B

hat

nag

ar e

t al

., 20

13)

Gen

erat

ors

ava

ilab

le (

#)

(H)

(Ro

e an

d S

chu

lman

, 20

12)

Hyd

rop

ho

bic

co

atin

g o

n

equ

ipm

ent

(L)

(Keo

gh

an

d C

od

y, 2

013)

Key

rep

lace

men

t eq

uip

men

t st

ock

pile

(L)

(K

eog

h a

nd

Co

dy,

201

3)

Red

un

dan

t p

ow

er li

nes

(L

) (K

eog

h a

nd

Co

dy,

20

13)

Rei

nfo

rced

co

ncr

ete

vers

us

wo

od

en

dis

trib

uti

on

po

les

(L)

(Keo

gh

an

d C

od

y, 2

013)

Siti

ng

infr

astr

uct

ure

(L)

(K

eog

h a

nd

Co

dy,

201

3)

Un

der

gro

un

d, o

verh

ead

, u

nd

erse

a d

istr

ibu

tio

n/

cab

le li

nes

(M

) (D

ou

kas

et

al.,

2011

; Ro

use

an

d K

elly

, 20

11)

Un

iqu

e en

cryp

ted

p

assw

ord

s fo

r u

tilit

y “s

mar

t” d

istr

ibu

tio

n (

L)

(Keo

gh

an

d C

od

y, 2

013)

Wo

rker

s em

plo

yed

(#)

(H

) (M

cCar

thy,

Og

den

, an

d

Sper

ling

, 20

07; K

eog

h

and

Co

dy,

201

3)

Co

mm

un

icat

ion

/co

ntr

ol s

yste

ms/

con

tro

l cen

ters

(L)

(W

ard

, 201

3)

Elec

tric

al p

rote

ctio

n a

nd

met

erin

g

(L)

(War

d, 2

013)

Equ

ipm

ent

po

siti

on

ing

(L)

(K

eog

h

and

Co

dy,

201

3)

Flo

w p

ath

s, li

ne

flo

w li

mit

s (L

) (B

om

par

d, N

apo

li, a

nd

Xu

e, 2

010)

Gen

/lo

ad b

us

dis

trib

uti

on

(L)

(B

om

par

d, N

apo

li, a

nd

Xu

e, 2

010)

Res

erve

/sp

are

cap

acit

y (M

) (W

illis

an

d G

arro

d, 1

997;

Mo

lyn

eau

x et

al

., 20

12)

Sub

stat

ion

s (s

wit

chya

rds)

—o

verh

ead

lin

es a

nd

un

der

gro

un

d

cab

les

are

inte

rco

nn

ecte

d (

L)

(War

d, 2

013)

An

cilla

ry s

ervi

ce (

L) (

Bh

atn

agar

et

al.,

201

3)

Haz

ard

rat

e re

lati

ng

fu

nct

ion

—al

tere

d h

azar

d r

ate

of

com

po

nen

t af

ter

a ce

rtai

n

mai

nte

nan

ce (

M)

(Wan

g a

nd

G

uo

, 201

3)

Lin

e m

itig

atio

n—

rero

ute

el

ectr

ical

flo

ws

du

e to

lin

e o

verl

oad

ing

or

“co

ng

esti

on

” (L

) (R

oe

and

Sch

ulm

an, 2

012)

Load

bia

sin

g—

maj

or

adju

stm

ents

in a

uto

mat

ed

dis

pat

chin

g s

oft

war

e (L

) (R

oe

and

Sch

ulm

an, 2

012)

Net

-ab

ility

—m

easu

res

the

apti

tud

e o

f th

e g

rid

in

tran

smit

tin

g p

ow

er f

rom

g

ener

atio

n t

o lo

ad b

use

s ef

fici

entl

y (L

) (B

om

par

d, N

apo

li,

and

Xu

e, 2

010)

Path

red

un

dan

cy—

asse

sses

th

e av

aila

ble

red

un

dan

cy in

ter

ms

of

pat

hs

in t

ran

smit

tin

g p

ow

er f

rom

a

gen

erat

ion

to

a lo

ad b

us

bas

ed

on

en

tro

py

(L)

(Bo

mp

ard

, Nap

oli,

an

d X

ue,

201

0)

Pro

tect

ive

and

sw

itch

ing

d

evic

es—

mea

n t

ime

to r

epai

r (M

) (Y

edd

anap

ud

i, 20

12)

Pro

tect

ive

and

sw

itch

ing

d

evic

es—

swit

chin

g r

elia

bili

ty (

M)

(Yed

dan

apu

di,

2012

)

Pro

tect

ive

and

sw

itch

ing

d

evic

es—

mea

n t

ime

to s

wit

ch

(M)

(Yed

dan

apu

di,

2012

)

Via

bili

ty o

f in

vest

men

ts (

L)

(McC

arth

y, O

gd

en, a

nd

Sp

erlin

g,

2007

)

Co

effi

cien

t o

f va

riat

ion

of

the

freq

uen

cy in

dex

of

sag

s (M

) (S

hu

n e

t al

., 20

12)

Co

ntr

ol P

erfo

rman

ce S

tan

dar

d 2

vi

ola

tio

ns—

on

e o

f th

e C

alif

orn

ia

Ind

epen

den

t Sy

stem

Op

erat

or

CII

SO

pri

nci

pal

rel

iab

ility

sta

nd

ard

s (H

) (R

oe

and

Sch

ulm

an, 2

012)

Bu

lk e

lect

ric

syst

em r

elia

bili

ty

per

form

ance

ind

ices

(M

) (B

illin

ton

an

d

Wan

gd

ee, 2

006

)

Der

ated

po

wer

—ra

ted

po

wer

mu

ltip

le

wit

h t

he

relia

bili

ty o

f th

e p

lan

t (M

) (V

oo

rsp

oo

ls a

nd

D’H

aese

leer

, 20

04)

Dro

pp

ed/l

ost

ph

ase

—p

ow

er q

ual

ity

met

ric

(M)

(Ro

use

an

d K

elly

, 201

1)

Edg

e re

silie

nce

tra

ject

ory

—re

lati

on

ship

bet

wee

n r

elia

bili

ty a

nd

re

silie

nce

tra

ckin

g a

mo

vin

g r

ang

e o

f R

2 fo

r th

e C

on

tro

l Per

form

ance

St

and

ard

2 (

H)

(Ro

e an

d S

chu

lman

, 20

12)

Ener

gy

effi

cien

cy/i

nte

nsi

ty (

H)

(Gn

anso

un

ou

, 20

08; M

oly

nea

ux

et a

l.,

2012

; Wan

g e

t al

., 20

12)

Failu

re r

ate

(M)

(Wan

g a

nd

Gu

o, 2

013)

Flic

ker—

po

wer

qu

alit

y m

etri

c (M

) (R

ou

se a

nd

Kel

ly, 2

011)

Har

mo

nic

dis

tort

ion

s—p

ow

er q

ual

ity

met

ric

(M)

(Ro

use

an

d K

elly

, 201

1)

Ove

rhea

d a

nd

un

der

gro

un

d li

ne

seg

men

ts—

mea

n t

ime

to r

epai

r (L

) (Y

edd

anap

ud

i, 20

12)

Ove

rhea

d a

nd

un

der

gro

un

d li

ne

seg

men

ts—

per

man

ent

failu

re r

ate

(L)

(Yed

dan

apu

di,

2012

)

Load

loss

dam

age

ind

ex—

dam

age

cau

sed

by

fire

to

th

e el

ectr

ical

sys

tem

(M

) (L

uci

a,

2012

; Bag

chi,

Spri

nts

on

, an

d

Sin

gh

, 201

3)


Inp

uts

Cap

acit

ies

Cap

abili

ties

Perf

orm

ance

Ou

tco

mes

Ove

rhea

d a

nd

un

der

gro

un

d li

ne

seg

men

ts—

tem

po

rary

fai

lure

rat

e (L

) (Y

edd

anap

ud

i, 20

12)

Peak

-to

-pea

k vo

ltag

e—

po

wer

qu

alit

y m

etri

c (M

) (R

ou

se a

nd

Kel

ly, 2

011)

Phas

e im

bal

ance

—p

ow

er q

ual

ity

met

ric

(M)

(Ro

use

an

d K

elly

, 201

1)

Pro

tect

ive

and

sw

itch

ing

d

evic

es—

pro

bab

ility

of

failu

re (

M)

(Yed

dan

apu

di,

2012

)

Pro

tect

ive

and

sw

itch

ing

d

evic

es—

pro

tect

ion

rel

iab

ility

(M

) (Y

edd

anap

ud

i, 20

12)

Pro

tect

ive

and

sw

itch

ing

dev

ices

—R

eclo

se r

elia

bili

ty (

M)

(Yed

dan

apu

di,

2012

)

Rap

id v

olt

age

chan

ges

—p

ow

er

qu

alit

y m

etri

c (M

) (R

ou

se a

nd

Kel

ly,

2011

)

Res

ilien

ce in

dex

(1)—

par

amet

er t

hat

q

uan

tifi

es t

he

po

ten

tial

pro

bab

ility

fo

r th

e m

alfu

nct

ion

of

the

syst

em (

M)

(Afg

an a

nd

Cve

tin

ovi

c, 2

010)

Res

ilien

ce in

dex

(2)

—d

eriv

ed f

rom

ro

bu

stn

ess,

res

ou

rcef

uln

ess,

an

d

reco

very

; ran

ge

fro

m 0

(lo

w r

esili

ence

) to

10

0 (h

igh

res

ilien

ce)

(M)

(Fis

her

et

al.,

2010

; Car

lso

n e

t al

., 20

12; A

fgan

an

d C

veti

no

vic,

201

3)

Surv

ivab

ility

—ev

alu

ate

the

apti

tud

e o

f th

e n

etw

ork

in a

ssu

rin

g t

he

po

ssib

ility

to

mat

ch g

ener

atio

n a

nd

d

eman

d in

cas

e o

f fa

ilure

s o

r at

tack

s (L

) (B

om

par

d, N

apo

li, a

nd

Xu

e, 2

010)

Syst

em a

vera

ge

inte

rru

pti

on

du

rati

on

in

dex

—su

stai

ned

ou

tag

e m

etri

c;

mea

sure

s an

nu

al s

yste

mw

ide

ou

tag

e d

ura

tio

n f

or

sust

ain

ed o

uta

ges

(H

) (L

ayto

n, 2

00

4; E

to a

nd

LaC

om

mar

e,

2008

; Ro

use

an

d K

elly

, 201

1)

Tab

le 4

.2—

Co

nti

nu

ed


Inp

uts

Cap

acit

ies

Cap

abili

ties

Perf

orm

ance

Ou

tco

mes

Syst

em a

vera

ge

inte

rru

pti

on

fr

equ

ency

ind

ex—

sust

ain

ed o

uta

ge

met

ric;

mea

sure

s sy

stem

wid

e o

uta

ge

freq

uen

cy f

or

sust

ain

ed o

uta

ges

(H

) (L

ayto

n, 2

00

4; E

to a

nd

LaC

om

mar

e,

2008

; Ro

use

an

d K

elly

, 201

1)

Un

sch

edu

led

gen

erat

or

ou

tag

es (

L)

(Ro

e an

d S

chu

lman

, 201

2)

Vo

ltag

e d

ips—

po

wer

qu

alit

y m

etri

c (M

) (R

ou

se a

nd

Kel

ly, 2

011)

Vo

ltag

e le

vel/

sup

ply

vo

ltag

e va

riat

ion

s—p

ow

er q

ual

ity

met

ric

(M)

(Ro

use

an

d K

elly

, 201

1)

Vo

ltag

e sa

gs/

swel

ls—

po

wer

qu

alit

y m

etri

c (M

) (R

ou

se a

nd

Kel

ly, 2

011)

Vo

ltag

e u

nb

alan

ce—

po

wer

qu

alit

y m

etri

c (M

) (R

ou

se a

nd

Kel

ly, 2

011)

NO

TE: M

etri

c ap

pea

rs in

bo

ld. M

atu

rity

leve

l ap

pea

rs in

par

enth

eses

: H =

hig

h; M

= m

ediu

m; a

nd

L =

low

. Th

e ci

tati

on

s in

clu

ded

in t

his

tab

le a

re in

stan

ces

wh

ere

the

met

ric

can

be

fou

nd

in t

he

liter

atu

re, b

ut

they

are

no

t n

eces

sari

ly e

xhau

stiv

e.

Tab

le 4

.2—

Co

nti

nu

ed


Tab

le 4

.3En

erg

y R

esili

ence

Met

rics

fo

r El

ectr

icit

y Sy

stem

s at

th

e Sy

stem

/Reg

ion

Lev

el

Inp

uts

Cap

acit

ies

Cap

abili

ties

Perf

orm

ance

Ou

tco

mes

Tran

smis

sio

n li

nes

av

aila

ble

(#)

(M

) (R

oe

and

Sch

ulm

an, 2

012)

Fun

ctio

nal

zo

nes

—g

ener

atio

n, t

ran

smis

sio

n, a

nd

d

istr

ibu

tio

n (

H)

(McC

arth

y,

Og

den

, an

d S

per

ling

, 20

07)

Hie

rarc

hic

al le

vels

(H

LI,

HLI

I, H

LIII

)—H

LI c

on

sid

ers

on

ly g

ener

atin

g f

acili

ties

, H

LII a

dd

s tr

ansm

issi

on

fa

cilit

ies,

an

d H

LIII

incl

ud

es

all t

hre

e fu

nct

ion

al z

on

es

(H)

(McC

arth

y, O

gd

en, a

nd

Sp

erlin

g, 2

007

)

Op

erat

or

trai

nin

g (

L) (

Keo

gh

an

d C

od

y, 2

013)

Mu

tual

ass

ista

nt

agre

emen

ts

(L)

(Keo

gh

an

d C

od

y, 2

013)

Tran

sfo

rmer

s—co

nn

ecti

ng

p

arts

of

the

net

wo

rk

op

erat

ing

at

dif

fere

nt

volt

ages

(L)

(W

ard

, 201

3)

Tree

tri

mm

ing

met

rics

(L)

(K

eog

h a

nd

Co

dy,

201

3)

Ad

equ

acy—

the

abili

ty o

f th

e sy

stem

to

su

pp

ly c

ust

om

er

req

uir

emen

ts u

nd

er n

orm

al

op

erat

ing

co

nd

itio

ns

(H)

(McC

arth

y, O

gd

en, a

nd

Sp

erlin

g,

2007

)

Co

ng

esti

on

co

ntr

ol (

L) (

Car

valh

o

et a

l., 2

014)

Ave

rag

e Se

rvic

e A

vaila

bili

ty In

dex

(A

NSI

) (H

) (L

ayto

n, 2

00

4)

Ave

rag

e Se

rvic

e In

terr

up

tio

n D

ura

tio

n

Ind

ex (

H)

(Yed

dan

apu

di,

2012

)

Cu

sto

mer

Ave

rag

e In

terr

up

tio

n D

ura

tio

n

Ind

ex—

sust

ain

ed o

uta

ge

met

ric;

mea

sure

s av

erag

e d

ura

tio

n o

f su

stai

ned

ou

tag

e p

er c

ust

om

er (

H)

(Lay

ton

, 20

04;

Eto

an

d

LaC

om

mar

e, 2

008

; Ro

use

an

d K

elly

, 201

1)

Cu

sto

mer

To

tal A

vera

ge

Inte

rru

pti

on

D

ura

tio

n In

dex

(H

) (Y

edd

anap

ud

i, 20

12)

Cu

sto

mer

Ave

rag

e In

terr

up

tio

n F

req

uen

cy

Ind

ex—

mea

sure

s cu

sto

mer

ave

rag

e in

terr

up

tio

n f

req

uen

cy (

H)

(Lay

ton

, 20

04;

R

ou

se a

nd

Kel

ly, 2

011)

Cu

sto

mer

s ex

per

ien

cin

g lo

ng

est

inte

rru

pti

on

du

rati

on

s (C

ELID

-X;

CEL

ID-8

)—su

stai

ned

ou

tag

e m

etri

c;

mea

sure

s th

e p

erce

nta

ge

of

cust

om

ers

exp

erie

nci

ng

ext

end

ed o

uta

ges

last

ing

m

ore

th

an X

ho

urs

(H

) (R

ou

se a

nd

Kel

ly,

2011

)

Cu

sto

mer

s ex

per

ien

cin

g m

ult

iple

in

terr

up

tio

ns

(CEM

I-X

)—su

stai

ned

ou

tag

e m

etri

c; m

easu

res

the

per

cen

tag

e o

f cu

sto

mer

s w

ith

mu

ltip

le o

uta

ges

. Th

is

met

ric

hel

ps

to m

easu

re r

elia

bili

ty a

t a

cust

om

er le

vel a

nd

can

iden

tify

pro

ble

ms

no

t m

ade

app

aren

t b

y sy

stem

wid

e av

erag

es

(H)

(Ro

use

an

d K

elly

, 201

1)

Cu

sto

mer

s ex

per

ien

cin

g m

ult

iple

m

om

enta

ry in

terr

up

tio

ns

(CEM

MI-

X;

CEM

MI-

4)—

mea

sure

s th

e p

erce

nta

ge

of

cust

om

ers

wh

o e

xper

ien

ced

X m

om

enta

ry

inte

rru

pti

on

s (H

) (R

ou

se a

nd

Kel

ly, 2

011)

Cu

sto

mer

s in

terr

up

ted

per

inte

rru

pti

on

in

dex

(H

) (L

ayto

n, 2

00

4)

An

nu

al p

rice

cap

(H

) (B

illin

ton

an

d W

ang

dee

, 20

06)

An

nu

al a

llow

ed r

even

ue

(H)

(Bill

into

n a

nd

Wan

gd

ee, 2

006

)

Co

st o

f in

terr

up

tio

n—

soci

al,

com

mer

cial

, in

du

stri

al, e

tc. (

L)

(Do

uka

s et

al.,

201

1)

Imp

act

fact

or

on

th

e p

op

ula

tio

n—

shar

e o

f th

e p

op

ula

tio

n a

ffec

ted

by

the

po

wer

loss

(M

) (P

olja

nse

k,

Bo

no

, an

d G

uti

erre

z, 2

012)

Lon

g-d

ista

nce

tra

nsm

issi

on

co

sts

(M)

(Do

uka

s et

al.,

201

1)

No

ise

(L)

(Do

uka

s et

al.,

201

1)

Perf

orm

ance

-bas

ed r

egu

lati

on

re

war

d/p

enal

ty s

tru

ctu

re (

L)

(Bill

into

n a

nd

Wan

gd

ee, 2

006

)

Pric

e o

f el

ectr

icit

y (M

) (D

ou

kas

et a

l., 2

011)

Val

ue

of

lost

load

—va

lue

of

un

serv

ed e

ner

gy;

cu

sto

mer

s’

valu

e o

f th

e o

pp

ort

un

ity

cost

of

ou

tag

es o

r b

enefi

ts

forg

on

e th

rou

gh

inte

rru

pti

on

s in

ele

ctri

city

su

pp

ly (

L) (

Will

is

and

Gar

rod

, 199

7; L

uci

a, 2

012)


Inp

uts

Cap

acit

ies

Cap

abili

ties

Perf

orm

ance

Ou

tco

mes

Eco

no

my—

ach

ieve

th

e b

est

pro

fits

by

adju

stin

g t

he

po

wer

sys

tem

op

erat

ion

m

od

e to

min

imiz

e lin

e lo

sses

, mak

ing

fu

ll u

se o

f eq

uip

men

t, e

nsu

rin

g t

he

secu

rity

o

f th

e p

ow

er s

yste

m, a

nd

mee

tin

g u

tilit

y u

sers

’ dem

and

(M

) (W

ang

, 201

2)

Fair

nes

s—co

nsi

sts

of

the

fulfi

llmen

t ra

te o

f co

ntr

act

and

sta

nd

ard

dev

iati

on

ind

exes

(L)

(W

ang

, 201

2)

Inte

rru

pte

d e

ner

gy

asse

ssm

ent

rate

(M

) (B

illin

ton

an

d W

ang

dee

, 20

06)

Load

po

int

ind

ices

per

cu

sto

mer

—n

um

ber

o

f o

uta

ges

per

yea

r; d

ura

tio

n o

f o

uta

ges

p

er y

ear;

un

avai

lab

le/a

vaila

ble

ser

vice

(M

) (Y

edd

anap

ud

i, 20

12)

Loss

of

off

site

po

wer

(M

) (I

nte

rnat

ion

al

Ato

mic

En

erg

y A

gen

cy, 2

012)

Min

imu

m le

vel o

f se

rvic

e/ta

rget

s (M

) (R

ou

se a

nd

Kel

ly, 2

011)

Mo

men

tary

ave

rag

e in

terr

up

tio

n

freq

uen

cy in

dex

—m

om

enta

ry o

uta

ge

met

ric;

mea

sure

s fr

equ

ency

of

mo

men

tary

o

uta

ges

. Mo

men

tary

ou

tag

es a

nd

th

e p

ow

er s

urg

es a

sso

ciat

ed w

ith

th

em c

an

dam

age

con

sum

er p

rod

uct

s an

d h

urt

ce

rtai

n b

usi

nes

s se

cto

rs (

M)

(Lay

ton

, 20

04;

R

ou

se a

nd

Kel

ly, 2

011)

Secu

rity

—d

ynam

ic r

esp

on

se o

f th

e sy

stem

to

un

exp

ecte

d in

terr

up

tio

ns;

rel

ates

th

e sy

stem

’s a

bili

ty t

o e

nd

ure

th

em (

H)

(McC

arth

y, O

gd

en, a

nd

Sp

erlin

g, 2

007

)

Tran

smis

sio

n lo

sses

(M

) (D

ou

kas

et a

l.,

2011

)

NO

TE: M

etri

c ap

pea

rs in

bo

ld. M

atu

rity

leve

l ap

pea

rs in

par

enth

eses

: H =

hig

h; M

= m

ediu

m; a

nd

L =

low

. Th

e ci

tati

on

s in

clu

ded

in t

his

tab

le a

re in

stan

ces

wh

ere

the

met

ric

can

be

fou

nd

in t

he

liter

atu

re, b

ut

they

are

no

t n

eces

sari

ly e

xhau

stiv

e.

Tab

le 4

.3—

Co

nti

nu

ed


Tab

le 4

.4En

erg

y R

esili

ence

Met

rics

fo

r El

ectr

icit

y Sy

stem

s at

th

e R

egio

n/N

atio

n L

evel

Inp

uts

Cap

acit

ies

Cap

abili

ties

Perf

orm

ance

Ou

tco

mes

Sto

rm r

eser

ve f

un

ds

(L)

(Keo

gh

an

d C

od

y, 2

013)

Co

nce

ntr

atio

n o

f m

arke

t su

pp

liers

(M

) (B

lyth

an

d L

efev

re, 2

00

4)

Her

fin

dah

l-H

ersc

hm

ann

ind

ex—

use

d t

o

mea

sure

mar

ket

con

cen

trat

ion

ris

k; s

qu

are

of

each

par

tici

pan

t’s

mar

ket

shar

e ad

ded

to

get

her

acr

oss

all

par

tici

pan

ts w

ith

th

e la

rges

t sh

ares

(M

) (B

lyth

an

d L

efev

re, 2

00

4;

Rey

mo

nd

, 20

07)

Geo

po

litic

al m

arke

t co

nce

ntr

atio

n r

isk

(M)

(Bly

th a

nd

Lef

evre

, 20

04)

CO

2 em

issi

on

s (M

) (D

ou

kas

et a

l., 2

011)

Der

egu

late

d e

lect

rici

ty m

arke

ts—

allo

cati

on

o

f lo

sses

(L)

(D

ou

kas

et a

l., 2

011)

Pub

lic d

eath

s/in

juri

es (

du

e to

po

wer

in

terr

up

tio

ns)

(M

) (A

ust

ralia

n E

lect

rica

l R

egu

lato

ry A

uth

ori

ties

Co

un

cil,

2005

–20

06;

Ro

use

an

d K

elly

, 201

1)

Pub

lic d

eath

s/in

juri

es (

du

e to

inte

ract

ion

s w

ith

th

e d

istr

ibu

tio

n s

yste

m)

(M)

(Au

stra

lian

El

ectr

ical

Reg

ula

tory

Au

tho

riti

es C

ou

nci

l, 20

05–2

006

; Ro

use

an

d K

elly

, 201

1)


Tab

le 4

.5En

erg

y R

esili

ence

Met

rics

fo

r O

il an

d N

atu

ral G

as S

yste

ms

at t

he

Faci

lity/

Syst

em L

evel

Inp

uts

Cap

acit

ies

Cap

abili

ties

Perf

orm

ance

Ou

tco

mes

Hu

bs—

no

des

wit

h t

he

mo

st li

nks

are

th

e m

ost

inte

rco

nn

ecte

d a

nd

ser

ve a

s h

ub

s (H

) (N

adea

u, 2

007

)

Lin

ks—

flo

w b

etw

een

no

des

tak

es

pla

ce o

n li

nks

(ro

ads,

ele

ctri

c p

ow

er

tran

smis

sio

n li

nes

, wat

er m

ain

s,

etc.

) (H

) (N

adea

u, 2

007

; Vu

gri

n a

nd

Tu

rnq

uis

t, 2

012;

Elli

son

, Co

rbet

, an

d

Bro

oks

, 201

3)

No

des

—el

emen

t o

f th

e n

etw

ork

th

at c

an r

ecei

ve g

as f

rom

sto

rag

e fa

cilit

ies,

pip

elin

e in

terc

on

nec

tio

ns,

o

r p

rod

uct

ion

are

as (

H)

(An

der

son

20

01; N

adea

u, 2

007

; Elli

son

, Co

rbet

, an

d B

roo

ks, 2

013)

Prim

ary

ener

gy

sup

ply

—in

clu

des

th

e sy

stem

s an

d p

roce

sses

use

d t

o

sup

ply

a p

rim

ary

ener

gy

reso

urc

e to

its

po

int

of

con

vers

ion

into

th

e fi

nal

en

erg

y p

rod

uct

of

inte

rest

(H

) (M

cCar

thy,

Og

den

, an

d S

per

ling

, 20

07)

Sto

rag

e fa

cilit

ies/

no

des

, in

term

edia

te s

tora

ge

(#)

(H)

(Vu

gri

n

and

Tu

rnq

uis

t, 2

012;

Elli

son

, Co

rbet

, an

d B

roo

ks, 2

013)

Emer

gen

cy p

roce

du

res/

emer

gen

cy s

hu

tdo

wn

sys

tem

(M

) (H

su, S

hu

, an

d T

sao

, 201

0)

Max

imu

m/m

inim

um

flo

w (

H)

(Elli

son

, Co

rbet

, an

d B

roo

ks, 2

013)

Co

st p

er u

nit

of

flo

w (

H)

(Elli

son

, Co

rbet

, an

d B

roo

ks, 2

013)

Effi

cien

cy o

f fl

ow

—o

ne

min

us

the

frac

tio

n

of

gas

bu

rned

as

com

pre

sso

r fu

el (

H)

(Nad

eau

, 20

07; E

lliso

n, C

orb

et, a

nd

Bro

oks

, 20

13)

Res

po

nse

to

eq

uip

men

t o

uta

ges

—d

egre

e to

wh

ich

th

e sy

stem

is a

ble

to

co

nti

nu

e to

re

liab

ly o

per

ate

in t

he

even

t o

f eq

uip

men

t d

ow

nti

me

(L)

(McC

arth

y, O

gd

en, a

nd

Sp

erlin

g, 2

007

)

NO

TE: M

etri

c ap

pea

rs in

bo

ld. M

atu

rity

leve

l ap

pea

rs in

par

enth

eses

: H =

hig

h; M

= m

ediu

m; a

nd

L =

low

. Th

e ci

tati

on

s in

clu

ded

in t

his

tab

le a

re in

stan

ces

wh

ere

the

met

ric

can

be

fou

nd

in t

he

liter

atu

re, b

ut

they

are

no

t n

eces

sari

ly e

xhau

stiv

e.


Tab

le 4

.6En

erg

y R

esili

ence

Met

rics

fo

r O

il an

d N

atu

ral G

as S

yste

ms

at t

he

Syst

em/R

egio

n L

evel

Inp

uts

Cap

acit

ies

Cap

abili

ties

Perf

orm

ance

Ou

tco

mes

Fun

ctio

nal

zo

nes

—p

rim

ary

ener

gy

sup

ply

, en

erg

y p

roce

ssin

g, a

nd

co

nve

rsio

n a

nd

tra

nsp

ort

(M

) (M

cCar

thy,

Og

den

, an

d

Sper

ling

, 20

07)

Ad

apti

ve c

apac

ity—

deg

ree

to

wh

ich

th

e sy

stem

is c

apab

le o

f se

lf-

org

aniz

atio

n f

or

reco

very

of

syst

em

per

form

ance

leve

ls (

L) (

Vu

gri

n,

War

ren

, an

d E

hle

n, 2

011)

Cap

acit

y—ab

ility

of

the

syst

em

to p

rovi

de

suffi

cien

t th

rou

gh

pu

t to

su

pp

ly fi

nal

dem

and

(M

) (M

cCar

thy,

Og

den

, an

d S

per

ling

, 20

07)

Info

rmat

ion

sec

uri

ty—

the

deg

ree

to w

hic

h in

form

atio

n a

sset

s in

th

e sy

stem

are

sec

ure

ag

ain

st

thre

ats

(L)

(McC

arth

y, O

gd

en, a

nd

Sp

erlin

g, 2

007

)

Inte

rdep

end

enci

es—

the

deg

ree

to w

hic

h t

he

syst

em r

elie

s o

n

oth

er in

fras

tru

ctu

re f

or

its

relia

ble

o

per

atio

n a

nd

is v

uln

erab

le t

o

that

infr

astr

uct

ure

’s d

isru

pti

on

(L)

(M

cCar

thy,

Og

den

, an

d S

per

ling

, 20

07)

Phys

ical

sec

uri

ty—

the

deg

ree

to

wh

ich

ph

ysic

al a

sset

s in

th

e sy

stem

ar

e se

curi

ty a

gai

nst

th

reat

s (L

) (M

cCar

thy,

Og

den

, an

d S

per

ling

, 20

07)

Ab

sorp

tive

cap

acit

y—d

egre

e to

wh

ich

a

syst

em c

an a

uto

mat

ical

ly a

bso

rb t

he

imp

acts

o

f a

syst

em’s

per

turb

atio

ns

and

min

imiz

e co

nse

qu

ence

s w

ith

litt

le e

ffo

rt (

L) (

Vu

gri

n,

War

ren

, an

d E

hle

n, 2

011)

Co

nn

ecti

vity

loss

—th

e av

erag

e re

du

ctio

n in

th

e ab

ility

of

sin

ks t

o r

ecei

ve fl

ow

fro

m s

ou

rces

(M

) (P

olja

nse

k, B

on

o, a

nd

Gu

tier

rez,

201

2)

Ener

gy

pro

cess

ing

an

d c

on

vers

ion

—re

late

s to

pro

du

ctio

n o

f th

e fi

nal

en

erg

y p

rod

uct

(H

) (M

cCar

thy,

Og

den

, an

d S

per

ling

, 20

07)

Flex

ibili

ty—

the

deg

ree

to w

hic

h t

he

syst

em

can

ad

apt

to c

han

gin

g c

on

dit

ion

s (L

) (M

cCar

thy,

Og

den

, an

d S

per

ling

, 20

07)

His

tory

—th

e d

egre

e to

wh

ich

th

e sy

stem

h

as b

een

pro

ne

to d

isru

pti

on

in t

he

pas

t (M

) (M

cCar

thy,

Og

den

, an

d S

per

ling

, 20

07)

Inte

rmit

ten

cy—

the

deg

ree

to w

hic

h t

he

syst

em la

cks

con

stan

t le

vels

of

pro

du

ctiv

ity

(M)

(McC

arth

y, O

gd

en, a

nd

Sp

erlin

g, 2

007

)

Net

wo

rk r

esili

ency

—m

easu

red

by

its

abili

ty t

o

keep

su

pp

lyin

g a

nd

dis

trib

uti

ng

nat

ura

l gas

in

spit

e o

f d

amag

e to

pip

elin

es, l

iqu

efied

nat

ura

l g

as im

po

rt t

erm

inal

s, s

tora

ge,

an

d o

ther

gas

so

urc

es (

M)

(Nad

eau

, 20

07)

Res

po

nse

to

dem

and

flu

ctu

atio

ns—

the

exte

nt

to w

hic

h t

he

syst

em is

ab

le t

o a

dap

t to

ch

ang

es in

th

e q

uan

tity

of

ener

gy

dem

and

ed

or

loca

tio

n o

f d

eman

d (

L) (

McC

arth

y, O

gd

en,

and

Sp

erlin

g, 2

007

)

Syst

emic

imp

act—

imp

act

that

a d

isru

pti

on

h

as o

n s

yste

m p

rod

uct

ivit

y; m

easu

red

by

eval

uat

ing

th

e d

iffe

ren

ce b

etw

een

a t

arg

etin

g

syst

em p

erfo

rman

ce le

vel a

nd

th

e ac

tual

sy

stem

per

form

ance

(L)

(V

ug

rin

, War

ren

, an

d

Ehle

n, 2

011;

Vu

gri

n a

nd

Tu

rnq

uis

t, 2

012)

Dem

and

sat

isfi

ed (%

) (H

) (N

adea

u, 2

007

)

Imp

acts

on

inte

rdep

end

ent

syst

ems—

the

deg

ree

to w

hic

h

a d

isru

pti

on

in t

he

syst

em

mig

ht

feas

ibly

cau

se d

amag

e to

inte

rdep

end

ent

syst

ems

(L)

(McC

arth

y, O

gd

en, a

nd

Sp

erlin

g, 2

007

)

Op

tim

al r

esili

ence

co

sts—

resi

lien

ce c

ost

s fo

r a

syst

em

wh

en t

he

op

tim

al r

eco

very

st

rate

gy—

min

imiz

ing

th

e co

mb

ined

sys

tem

imp

act

and

to

tal r

eco

very

eff

ort

co

sts—

is

emp

loye

d (

L) (

Vu

gri

n, W

arre

n,

and

Eh

len

, 201

1)

Rec

ove

ry-d

epen

den

t re

silie

nce

co

sts—

resi

lien

ce

cost

s o

f a

syst

em u

nd

er a

p

arti

cula

r re

cove

ry s

trat

egy

(L)

(Vu

gri

n, W

arre

n, a

nd

Eh

len

, 20

11)

NO

TE: M

etri

c ap

pea

rs in

bo

ld. M

atu

rity

leve

l ap

pea

rs in

par

enth

eses

: H =

hig

h; M

= m

ediu

m; a

nd

L =

low

. Th

e ci

tati

on

s in

clu

ded

in t

his

tab

le a

re in

stan

ces

wh

ere

the

met

ric

can

be

fou

nd

in t

he

liter

atu

re, b

ut

they

are

no

t n

eces

sari

ly e

xhau

stiv

e.


Tab

le 4

.7En

erg

y R

esili

ence

Met

rics

fo

r O

il an

d N

atu

ral G

as S

yste

ms

at t

he

Reg

ion

/Nat

ion

Lev

el

Inp

uts

Cap

acit

ies

Cap

abili

ties

Perf

orm

ance

Ou

tco

mes

Div

ersi

ty o

f im

po

rt f

uel

s (H

) (G

nan

sou

no

u, 2

009

)

Nat

ura

l gas

str

ateg

ic

rese

rve

(M)

(Elli

son

, Kel

ic,

and

Co

rbet

, 20

06)

Her

fin

dah

l-H

ersc

hm

ann

in

dex

—u

sed

to

mea

sure

mar

ket

con

cen

trat

ion

ris

k; s

qu

are

of

each

par

tici

pan

t’s

mar

ket

shar

e ad

ded

to

get

her

acr

oss

all

par

tici

pan

ts w

ith

th

e la

rges

t sh

ares

(M

) (B

lyth

an

d L

efev

re,

200

4; R

eym

on

d, 2

007

)

Imp

ort

leve

ls—

the

deg

ree

to

wh

ich

pri

mar

y en

erg

y su

pp

ly

relie

s o

n r

eso

urc

es o

rig

inat

ing

o

uts

ide

of

the

cou

ntr

y (H

) (M

cCar

thy,

Og

den

, an

d S

per

ling

, 20

07)

Imp

ort

co

nce

ntr

atio

n—

the

deg

ree

to w

hic

h im

po

rts

are

con

cen

trat

ed a

mo

ng

a s

mal

l g

rou

p o

f su

pp

lyin

g c

ou

ntr

ies

(H)

(McC

arth

y, O

gd

en, a

nd

Sp

erlin

g,

2007

)

Imp

ort

dep

end

ency

—d

ivid

ed

into

so

urc

e d

epen

den

ce, t

ran

sit

dep

end

ence

, an

d f

acili

ty

dep

end

ence

(M

) (R

eym

on

d,

2007

)

Ind

ust

rial

asp

ects

—vu

lner

abili

ty

ind

icat

or

(M)

(Rey

mo

nd

, 20

07)

Rat

e o

f d

epen

den

cy—

mea

sure

d

by

the

net

en

erg

y im

po

rts/

tota

l p

rim

ary

ener

gy

con

sum

pti

on

ra

tio

(M

) (R

eym

on

d, 2

007

)

Vu

lner

abili

ty—

pro

po

rtio

nal

to

th

e re

lian

ce o

n im

po

rted

gas

fr

om

co

un

trie

s in

geo

po

litic

al

con

flic

t (M

) (R

eym

on

d, 2

007

)

Ab

ility

to

exp

and

fac

iliti

es—

the

deg

ree

to w

hic

h t

he

syst

em c

an

be

easi

ly a

nd

co

st-e

ffec

tive

ly

exp

and

ed (

L) (

McC

arth

y, O

gd

en,

and

Sp

erlin

g, 2

007

)

Imp

ort

cap

acit

y (M

) (E

lliso

n,

Co

rbet

, an

d B

roo

ks, 2

013)

Pip

elin

e ca

pac

ity

use

d (%

) (H

) (N

adea

u, 2

007

)

Res

ilien

cy—

abili

ty t

o s

up

ply

gas

to

cu

sto

mer

s w

illin

g t

o p

ay t

he

clea

rin

g p

rice

, eve

n in

th

e fa

ce

of

sup

ply

co

nst

rain

ts (

L) (

Ellis

on

, K

elic

, an

d C

orb

et,2

006

)

Res

tora

tive

cap

acit

y—ab

ility

of

a sy

stem

to

be

rep

aire

d e

asily

; th

ese

rep

airs

are

co

nsi

der

ed t

o b

e d

ynam

ic (

L) (

Vu

gri

n, W

arre

n, a

nd

Eh

len

, 201

1)

Tota

l rec

ove

ry e

ffo

rt—

effi

cien

cy

wit

h w

hic

h t

he

syst

em r

eco

vers

fr

om

a d

isru

pti

on

, mea

sure

d b

y an

alyz

ing

th

e am

ou

nt

of

reso

urc

es

exp

end

ed d

uri

ng

th

e re

cove

ry

pro

cess

(L)

(V

ug

rin

, War

ren

, an

d

Ehle

n, 2

011)

Sect

or

coo

rdin

atio

n—

the

deg

ree

to w

hic

h c

oo

rdin

atio

n

bet

wee

n s

take

ho

lder

s w

ith

in

the

sect

or

resu

lts

in a

n e

ffec

tive

ex

chan

ge

of

info

rmat

ion

ale

rtin

g

stak

eho

lder

s o

f em

erg

ing

th

reat

s an

d m

itig

atio

n s

trat

egie

s (L

) (M

cCar

thy,

Og

den

, an

d S

per

ling

, 20

07)

Tran

spo

rtat

ion

—en

com

pas

ses

the

tran

smis

sio

n a

nd

dis

trib

uti

on

of

the

fin

al e

ner

gy

pro

du

ct t

o it

s p

oin

t o

f en

d u

se (

H)

(McC

arth

y, O

gd

en, a

nd

Sp

erlin

g, 2

007

)

Eco

no

mic

imp

act—

the

deg

ree

to w

hic

h a

dis

rup

tio

n in

th

e sy

stem

mig

ht

feas

ibly

cau

se

eco

no

mic

dam

age

to in

du

stry

st

akeh

old

ers,

th

e g

ove

rnm

ent,

o

r th

e p

ub

lic (

L) (

McC

arth

y,

Og

den

, an

d S

per

ling

, 20

07)

Envi

ron

men

tal i

mp

act—

the

deg

ree

to w

hic

h a

dis

rup

tio

n

in t

he

syst

em m

igh

t fe

asib

ly

cau

se e

nvi

ron

men

tal d

amag

e (M

) (M

cCar

thy,

Og

den

, an

d

Sper

ling

, 20

07)

Hu

man

hea

lth

imp

act—

the

deg

ree

to w

hic

h a

dis

rup

tio

n in

th

e sy

stem

mig

ht

feas

ibly

har

m

the

hea

lth

of

emp

loye

es o

r th

e p

ub

lic (

M)

(McC

arth

y, O

gd

en,

and

Sp

erlin

g, 2

007

)

Phys

ical

sh

ort

age

(M)

(Elli

son

, K

elic

, an

d C

orb

et, 2

006

)

Pric

e/p

rice

vo

lati

lity

(H)

(Elli

son

, Kel

ic, a

nd

Co

rbet

, 20

06; M

cCar

thy,

Og

den

, an

d

Sper

ling

, 20

07)

NO

TE: M

etri

c ap

pea

rs in

bo

ld. M

atu

rity

leve

l ap

pea

rs in

par

enth

eses

: H =

hig

h; M

= m

ediu

m; a

nd

L =

low

. Th

e ci

tati

on

s in

clu

ded

in t

his

tab

le a

re in

stan

ces

wh

ere

the

met

ric

can

be

fou

nd

in t

he

liter

atu

re, b

ut

they

are

no

t n

eces

sari

ly e

xhau

stiv

e.

23

CHAPTER FIVE

Developing Metrics for Energy Resilience

Resilience is a topic of interest for policymakers as they consider the complex risks from natu-ral disasters, terrorism, aging infrastructure, and climate change faced by the transmission, storage, and distribution systems for U.S. energy. If energy infrastructure is to become more resilient, better use of metrics will be crucial to guiding planning and evaluating progress. The literature review presented in this report suggests three recommendations that could improve the metrics available to support energy policy.

Improve Collection and Management of Data on Inputs and Capacities at the Facility and System Levels

The literature on energy resilience includes many metrics that describe the state of inputs and capacities for energy facilities or localized energy systems. Because these metrics are col-lected by different organizations and entities for specific purposes, they are frequently not standardized and are rarely collected and managed in a manner that facilitates analysis to support policy. Doing so could provide an evidence base upon which cost-effective strate-gies to improve energy resilience could be developed. Examples of alternative standardization approaches vary, from mandatory regulatory reporting requirements to voluntary adoption of consensus-based management best practices. However, these types of knowledge management alternatives could also be costly. Thus, it is prudent to consider alternative approaches to imple-menting data collection and management, how much they might cost, and how they might serve the interests of different stakeholder groups.

Develop Better Measures of Capabilities at the System and Regional Levels

Improvements in capability are the principal levers through which energy system owners and operators can improve performance to achieve desired societal outcomes. However, this litera-ture review suggests that at the system and regional levels, metrics have not been well defined and performance data on capabilities are not regularly collected.

Measuring capabilities to respond to and recover from extreme events is difficult. For complex energy systems, there is not yet a consensus about what the core required capabili-ties for a system or region should be. Without opportunities to observe performance of capa-bilities regularly, because extreme events are rare, it is difficult to demonstrate proficiency in capabilities. Approaches used to measure and improve quality of services in other contexts—


such as health care quality, community resilience, and public health preparedness—illustrate approaches that may be useful. Examples from these fields include extracting insights from exercises and after-action reviews, developing drills that test components of a system, and developing community recommendations through consensus processes that collect experience across communities or sectors.

Defining and measuring capabilities are useful steps in developing a framework for improving the resilience of energy systems. Developing a stronger understanding of what are key capabilities and how to measure a system or region’s proficiency requires public- and private-sector cooperation to identify which capabilities to measure, practical and valid exer-cises or tests of proficiency in those capabilities, and efficient means of collecting and manag-ing data on capability metrics.

Improve Understanding of How Capabilities and Performance Translate to Outcomes at the Regional and National Levels

The goal of improving energy system resilience is to make communities safer and more pro-ductive. The literature on outcomes of energy system resilience reflects these goals and includes many potential outcome metrics. The literature does not, however, provide clarity about how to adjust capabilities and system performance to most effectively achieve desired outcomes. Building this understanding requires a coordinated effort to establish an evidence base in terms of metrics for inputs, capacities, capabilities, and performance described earlier in this chapter. This empirical foundation can then serve as a basis for modeling and understanding the complex technical and social interactions through which energy systems support societal goals for safety and prosperity.

25

Bibliography

Afgan, N., & Cvetinovic, D. (2010). Wind power plant resilience. Thermal Science, 14(2), 533–540. doi: 10.2298/tsci1002533a

———. (2013). Resilience evaluation of the South East European Natural Gas Routes system catastrophe. International Journal of Engineering and Innovative Technology (IJEIT), 3(5), 175–180.

Akhavein, A., & Fotuhi Firuzabad, M. (2011). A heuristic-based approach for reliability importance assessment of energy producers. Energy Policy, 39(3), 1562–1568. doi: 10.1016/j.enpol.2010.12.030

Amirat, A., Mohamed-Chateauneuf, A., & Chaoui, K. (2006). Reliability assessment of underground pipelines under the combined effect of active corrosion and residual stress. International Journal of Pressure Vessels and Piping, 83(2), 107–117.

Anderson, R. J. (2001, November). Natural gas infrastructure reliability (presentation). National Energy Technology Laboratory. Retrieved June 16, 2015, from http://wpweb2.tepper.cmu.edu/ceic/presentations/Anderson.pdf

Anghel, M., Werley, K. A., & Motter, A. E. (2007, January). Stochastic model for power grid dynamics. Paper presented at the 40th Annual Hawaii International Conference on System Sciences.

Australian Electrical Regulatory Authorities Council. (2005–2006). Electrical incident data: Australia and New Zealand. Retrieved June 16, 2015, from http://www.erac.gov.au/downloads/Erac%202005-2006.pdf

Bagchi, A., Sprintson, A., & Singh, C. (2013). Modeling the impact of fire spread on an electrical distribution network. Electric Power Systems Research, 100, 15–24. doi: 10.1016/j.epsr.2013.01.009

Bhatnagar, D., Currier, A. B., Hernandez, J., Ma, O., & Kirby, B. (2013). Market and policy barriers to energy storage deployment: A study for the energy storage systems program. Albuquerque, NM: Sandia National Laboratories.

Bhattacharyya, S. C. (2009). Fossil-fuel dependence and vulnerability of electricity generation: Case of selected European countries. Energy Policy, 37(6), 2411–2420. doi: 10.1016/j.enpol.2009.02.031

Billinton, R., & Wangdee, W. (2006, June 11–15). Utilizing bulk electric system reliability performance index probability distributions in a performance based regulation framework. Paper presented at the 9th International Conference on Probabilistic Methods Applied to Power Systems, Stockholm, Sweden.

Blyth, W., & Lefevre, N. (2004). Energy security and climate change policy interactions: An assessment framework. Paris: International Energy Agency.

Bompard, E., Napoli, R., & Xue, F. (2010). Extended topological approach for the assessment of structural vulnerability in transmission networks. Generation, Transmission & Distribution, IET, 4(6), 716–724. doi: 10.1049/iet-gtd.2009.0452

Brancucci Martínez-Anido, C., Bolado, R., De Vries, L., Fulli, G., Vandenbergh, M., & Masera, M. (2012). European power grid reliability indicators, what do they really tell? Electric Power Systems Research, 90, 79–84.

Carlson, L., Bassett, G., Buehring, W., Collins, M., Folga, S., Haffenden, B., Petit, F., Phillips, J., Verner, D., & Whitfield, R. (2012). Resilience: Theory and application. Argonne, IL: Argonne National Laboratory.


Carvalho, R., Buzna, L., Bono, F., Masera, M., Arrowsmith, D. K., & Helbing, D. (2014). Resilience of natural gas networks during conflicts, crises and disruptions. PLoS ONE, 9(3), e90265. doi: 10.1371/journal.pone.0090265

Committee on Increasing National Resilience to Hazards and Disasters; Committee on Science, Engineering, and Public Policy; and The National Academies. (2012). Disaster resilience: A national imperative. Washington, DC: National Academies Press.

D’Amico, G., Petroni, F., & Prattico, F. (2013). Wind speed modeled as an indexed semi‐Markov process. Environmetrics, 24(6), 367–376.

Doukas, H., Karakosta, C., Flamos, A., & Psarras, J. (2011). Electric power transmission: An overview of associated burdens. International Journal of Energy Research, 35(11), 979–988. doi: 10.1002/er.1745

Ellison, J., Kelic, A., & Corbet, T. (2006). Is a natural gas strategic reserve for the US necessary? A system dynamics approach. Paper presented at the 25th International Conference of the System Dynamics Society, Boston, MA. Retrieved March 1, 2013, from http://www.systemdynamics.org/conferences/2007/proceed/papers/ELLIS216.pdf

Ellison, J. F., Corbet, T. F., & Brooks, R. E. (2013). Natural gas network resiliency to a “shakeout scenario” earthquake. Albuquerque, NM: Sandia National Laboratories.

El-Salmawy, H., Youssef, K., Abdulla, S., Gamea, M., & Hamdy, K. (2010). Tracking the reliability indicators in the Egyptian electric power system. Paper presented at the 14th International Middle East Power Systems Conference, Cairo University, Egypt.

Eto, J. H., & LaCommare, K. H. (2008). Tracking the reliability of the U.S. electric power system: An assessment of publicly available information reported to state public utility commissions. Berkeley, CA: Ernest Orlando Lawrence Berkeley National Laboratory.

Fisher, R. E., Bassett, G. W., Buehring, W. A., Collins, M. J., Dickinson, D. C., Eaton, L. K., Haffenden, R. A., Hussar, N. E., Klett, M. S., Lawlor, M. A., Miller, D. J., Petit, F. D., Peyton, S. M., Wallace, K. E., Whitfield, R. G., & Peerenboom, J. P. (2010, August). Constructing a resilience index for the Enhanced Critical Infrastructure Protection Program. Argonne, IL: Argonne National Laboratory.

Gnansounou, E. (2008). Assessing the energy vulnerability: Case of industrialised countries. Energy Policy, 36(10), 3734-3744.

———. (2009, November 18–20). Indicators of energy vulnerability (presentation). World Energy Council.

Greenfield, V. A., Williams, V. L., & Eiseman, E. (2006). Using logic models for strategic planning and evaluation: Application to the National Center for Injury Prevention and Control. Santa Monica, CA: RAND Corporation, TR-370-NCIPC. Retrieved May 21, 2014, from http://www.rand.org/pubs/technical_reports/TR370.html

Haimes, Y. Y. (2009). On the definition of resilience in systems, Risk Analysis, 29(4), 498–501.

Harto, C. B., Yan, Y. E., Demissie, Y. K., Elcock, D., Tidwell, V. C., Hallett, K., Macknick, J., Wigmosta, M. S., & Tesfa, T. K. (2012). Analysis of drought impacts on electricity production in the Western and Texas interconnections of the United States. Argonne, IL: Argonne National Laboratory.

Hines, P., Blumsack, S., Cotilla Sanchez, E., & Barrows, C. (2010, January 5–8). The topological and electrical structure of power grids. Paper presented at the 43rd Hawaii International Conference on System Sciences.

Holmgren, Å. (2007). A framework for vulnerability assessment of electric power systems. In A. Murray & T. Grubesic (Eds.), Critical Infrastructure (pp. 31–55): Springer Berlin Heidelberg.

Hsu, B., Shu, M., & Tsao, M. (2010). Reliability measures for liquefied natural gas receiving terminal based on the failure information of emergency shutdown system. Shanghai: InTech China.

International Atomic Energy Agency. (2012). Electric grid reliability and interface with nuclear power plants. Nuclear Energy Series No. NG-T-3.8. Vienna: IAEA. Retrieved May 21, 2015, from http://www-pub.iaea.org/MTCD/publications/PDF/Pub1542_web.pdf

Keogh, M., & Cody, C. (2013, November). Resilience in regulated utilities. The National Association of Regulatory Utility Commissioners.

Bibliography 27

Layton, L. (2004). Electric system reliability indices. Retrieved May 11, 2015, from http://www.l2eng.com/Reliability_Indices_for_Utilities.pdf

Linkov, I., Bridges, T., Creutzig, F., Decker, J., Fox-Lent, C., Kröger, W., Lambert, J., Levermann, A., Montreuil, B., Nathwani, J., Nyer, R., Renn, O., Scharte, B., Scheffler, A., Schreurs, M., & Thiel-Clemen, T. (2014). Changing the resilience paradigm. Nature Climate Change, 4(6), 407–409.

Lucia, L. (2012). Review of the value of lost load (VoLL). Energy Market Company, Paper No. EMC/RCP/60/2012/ CP38.

Makarov, Y. V., & Moharari, N. S. (1999). A generalized power system reliability and security index. Paper presented at the IEEE Power Tech Conference, Budapest.

McCarthy, R. W., Ogden, J. M., & Sperling, D. (2007). Assessing reliability in energy supply systems. Energy Policy, 35(4), 2151–2162. doi: 10.1016/j.enpol.2006.06.016

Memmott, M. (2014, February 5). Sniper attack on Calif. power station raises terrorism fears. National Public Radio. Retrieved July 18, 2014, from http://www.npr.org/blogs/thetwo-way/2014/02/05/272015606/sniper-attack-on-calif-power-station-raises-terrorism-fears

Molyneaux, L., Wagner, L., Froome, C., & Foster, J. (2012). Resilience and electricity systems: A comparative analysis. Energy Policy, 47, 188–201.

Murkowski, L. (2014). Powering the future: Ensuring that federal policy fully supports electric reliability—An Energy 20/20 White Paper. Washington, DC: U.S. Senate Committee on Energy.

Nadeau, J. (2007). Improving the resiliency of the natural gas supply and distribution network (thesis). Naval Postgraduate School.

National Association of Convenience Stores. (2013). 2013 NACS retail fuels report: How Hurricane Sandy affected the fuels industry. Retrieved July 18, 2014, from http://www.nacsonline.com/YourBusiness/FuelsReports/GasPrices_2013/Pages/default.aspx

North American Electric Reliability Corporation. (2013, December). 2013 long-term reliability assessment. Atlanta, GA.

Northeastern University. (2013, December 3). After Hurricane Sandy: Lessons learned for bolstering energy resilience (conference). Hosted by Kostas Research Institute for Homeland Security, Northeastern University & Columbia University’s National Center for Disaster Preparedness. Retrieved from http://www.northeastern.edu/kostas/wp-content/uploads/2013/ 08/Health-Systems-and-Services-Resilience.pdf

Petit, F. D., Bassett, G. W., Black, R., Buehring, W. A., Collins, M. J., Dickinson, D. C., Fisher, R. E., Haffenden, R. A., Huttenga, A. A., Klett, M. S., Phillips, J. A., Thomas, M., Veselka, S. N., Wallace, K. E., Whitfield, R. G., & Peerenboom, J. P. (2013). Resilience measurement index: An indicator of critical infrastructure resilience. Argonne, IL: Argonne National Laboratory.

Poljansek, K., Bono, F., & Gutierrez, E. (2012). Seismic risk assessment of interdependent critical infrastructure systems: The case of European gas and electricity networks. Earthquake Engineering & Structural Dynamics, 41(1), 61–79. doi: 10.1002/eqe.1118

Reymond, M. (2007). European key issues concerning natural gas: Dependence and vulnerability. Energy Policy, 35(8), 4169–4176. doi: 10.1016/j.enpol.2007.02.030

Roe, E., & Schulman, P. R. (2012). Toward a comparative framework for measuring resilience in critical infrastructure systems. Journal of Comparative Policy Analysis, 14(2), 114–125. doi: 10.1080/13876988.2012.664687

Rogers, P. J., Petrosino, A., Huebner, T. A., & Hacsi, T. A. (2000). Program theory evaluation: Practice, promise, and problems. New Directions for Evaluation, 87(Fall), 5–13.

Rouse, G., & Kelly, J. (2011). Electricity reliability: Problems, progress and policy solutions. Galvin Electricity Initiative.


Shun, T., Hadjsaid, N., Xiangning, X., & Kieny, C. (2012, June 17–20). Power quality & reliability assessment of distribution system considering voltage interruptions and sags. Paper presented at the 15th International Conference on Harmonics and Quality of Power.

Song, H., & Kezunovic, M. (2006). Static analysis of vulnerability and security margin of the power system. Paper presented at the Transmission and Distribution Conference and Exhibition, 2005/2006 IEEE PES.

Sumper, A., Sudrià, A., & Ferrer, F. (2004). International reliability analysis in distribution networks. Paper presented at the International Conference on Renewable Energies and Power Quality.

U.S. Department of Energy (2012a). Hurricane Sandy situation report #19. Office of Electricity Delivery & Energy Reliability.

———. (2012b). Reliability improvements from the application of distribution automation technologies—Initial results. Washington, DC.

U.S. Energy Information Association (2014). Northeast and Mid-Atlantic power prices react to winter freeze and natural gas constraints. Retrieved July 18, 2014, from http://www.eia.gov/todayinenergy/detail.cfm?id=14671

Van der Vleuten, E., & Lagendijk, V. (2010). Interpreting transnational infrastructure vulnerability: European blackout and the historical dynamics of transnational electricity governance. Energy Policy, 38(4), 2053–2062. doi: 10.1016/j.enpol.2009.11.030

Voorspools, K. R., & D’Haeseleer, W. D. (2004). Reliability of power stations: Stochastic versus derated power approach. International Journal of Energy Research, 28(2), 117–129. doi: 10.1002/er.954

Vugrin, E. D., Brown, N. J. K., & Turnquist, M. A. (2010). Optimal recovery sequencing for critical infrastructure resilience assessment. Albuquerque, NM: Sandia National Laboratories.

Vugrin, E. D., & Turnquist, M. A. (2012). Design for resilience in infrastructure distribution networks. Environment, Systems, and Decisions, 33, 104–120.

Vugrin, E. D, Warren, D. E, & Ehlen, M. A. (2011). A resilience assessment framework for infrastructure and economic systems: Quantitative and qualitative resilience analysis of petrochemical supply chains to a hurricane. Process Safety Progress, 30(3), 280–290.

Wang, H., Lin, Z., Wen, F., & Huang, J. (2012, September 8–9). A comprehensive evaluation index system for power system operation. Paper presented at the International Conference on Sustainable Power Generation and Supply.

Wang, Y., & Guo, C. (2013). The study on effect of random maintenance quality of components on composite power system reliability. Paper presented at the Power and Energy Engineering Conference. doi: 10.1109/APPEEC.2013.6837263

Ward, D. (2013). The effect of weather on grid systems and the reliability of electricity supply. Climatic Change, 121(1), 103–113. doi: 10.1007/s10584-013-0916-z

White House. (2013, February 12). Presidential policy directive 21: Critical infrastructure security and resilience.

———. (2014, January 9). Presidential memorandum: Establishing a Quadrennial Energy Review.

Willis, K. G., & Garrod, G. D. (1997). Electricity supply reliability—Estimating the value of lost load. Energy Policy, 25(1), 97–103. doi: 10.1016/S0301-4215(96)00123-1

Yeddanapudi, S. R. K. (2012). Distribution system reliability evaluation (presentation). Iowa State University. Retrieved June 16, 2015, from http://www.slideserve.com/oshin/distribution-system-reliability-evaluation

measuring the resilience of energy distribution …...1 chapter one measuring the resilience of...

Documents