entity risk assessment - nerc.com assurance initiative... · april 3, 2013. clarity assurance ......
TRANSCRIPT
C L A R I T Y ▪ A S S U R A N C E ▪ R E S U LT S
MIDWESTRELIABIL ITYORGANIZATION
Improving RELIABILITY and mitigating RISKS to the Bulk Power System
Entity Risk Assessment
NERC WorkshopReliability Assurance Initiative
Dan Skaar, President and CEOApril 3, 2013
CLARITY ▪ ASSURANCE ▪ RESULTS
Hey Wait a MinuteMoment 1 (2008)
If Registered Entities proactively fix and self report violations, why fine them? …. aren’t they just doing the right thing?
Result – don’t fine Registered Entities for doing the right things…Zero Dollar Penalty, ACP/FFT, Compliance Exceptions (RAI)…?
3
“Anybody in there? Think McFly, Think”
CLARITY ▪ ASSURANCE ▪ RESULTS
Our Philosophy:Risk and Reliability
The hallmark of reliability in complex, real-time, interdependent systems is not that errors won’t
occur...
it’s that errors and operating anomalies won’t create an uncontrolled cascading event outside of the design
criteria - High Reliability Organizational Theory.
4
CLARITY ▪ ASSURANCE ▪ RESULTS
Using Highly Reliable Organization Theory to Manage RISK
1. Preoccupation with failure • Attention on close calls and near misses (“being lucky vs.
being good”); focus more on failures rather than successes
2. Reluctance to simplify interpretations• Solid “root cause” analysis practices
3. Sensitivity to operations• Situational awareness and carefully designed change
management processes
4. Commitment to resilience• Resources are continually devoted to corrective action plans
and training
5. Deference to expertise• Listen to your experts on the front lines (ex. authorities
follows expertise)
5
Five characteristics of highly reliable organizations
CLARITY ▪ ASSURANCE ▪ RESULTS
BTW, Small Stuff Matters
6
The small stuff matters.
Small stuff should be detected, reported, corrected, and prevented under normal management practices – its good utility practice.
If Registered Entities are proactively taking care of the small stuff before it escalates, it’s not prosecuted as a federal case. They are simply doing their job and what the public expects!
CLARITY ▪ ASSURANCE ▪ RESULTS
Hey Wait a MinuteMoment 2 (2009/2010)
Region had a lot of violations with PRC-005/PRC-008…• …there seemed to be wide
variations in the technical application of these standards….
• …it’s costing everyone a lot of money…let’s have a dialogue with the industry…they are the experts...they can solve problems…
Result – Engaged industry producing Application Guides
7
“Houston, we have a problem”
CLARITY ▪ ASSURANCE ▪ RESULTS
05
10152025303540
2007/2008 2009 2010 2011 2012
PRC Application Guide Roll Out
Increase in self reported violations resulted
from strengthening protection programs
Stabilizing trend
Note: A violation includes multiple elements
Positive Impacts from Engagement with IndustryImproving Protection Systems Maintenance and Testing
Num
ber o
f Vio
latio
ns
8
PRC‐005/008 Violation Trends
2.42.1
1.8 1.8
0.00
0.5
1
1.5
2
2.5
3
2007/2008 2009 2010 2011 2012
Decline in VSLs due to stronger programs
Aver
age
Viol
atio
n Se
verit
y Le
vel
(VSL
)
CLARITY ▪ ASSURANCE ▪ RESULTS
Hey Wait a MinuteMoment 3 (2010/2011)
…appeared to MRO staff that Registered Entities were only assessing compliance with Reliability Standards prior to an audit…”point in time compliance”……are Registered Entities systematically managing risk with Reliability Standards?…lets open a dialogue with industry on our concern…Result – Model Controls around Reliability Standards
9
You get what you inspect, not what you expect.
CLARITY ▪ ASSURANCE ▪ RESULTS
Example of Internal Control Program Scheme
ProgramDocuments (Procedures)
StandardWork Order
Supervisory Review
Management Oversight
Checklist followed and completed, exceptions noted, follow‐up notes signed
Review for completeness and accuracy, follow‐up actions closed or scheduled to be completed, signed
Periodic sampling of work orders to determine program is being completed and properly reviewed
Procedure/Process Control Control Activity Control Type
Primary Control
Secondary Control
Tertiary Control
10
CLARITY ▪ ASSURANCE ▪ RESULTS
Capability Maturity Model
Ad Hoc
Repeatable
Defined
Managed
Optimized
Emer
gent
Matu
re
Internal Control Design
Reliability Stan
dards R
egulation
Less More
HIGH
LOW
Risk
$$$
$
Internal Con
trols arou
nd Reliability
Stan
dards
AD‐HOC
OPTIMIZED
Relationship Between Size/Function and Internal Controls
Pareto Principle80% of inherent risk lies with 20% of Registered Entities
Large
Small
CLARITY ▪ ASSURANCE ▪ RESULTS
Risk Relationship with Internal Controls
12
Internal controls should be
commensurate with the level of inherent
risk.
CLARITY ▪ ASSURANCE ▪ RESULTS
What’s the Risk to Reliability?
Uncontrolled, cascading event outside the design criteria (Section 215 of FPA)Causes identified in blackout reports“Patterns” • Series of smaller matters• Documentation doesn’t match performance
Others?
13
CLARITY ▪ ASSURANCE ▪ RESULTS
Framework for Considering Risk
Inherent Risks
Control Risks
Detection Risks
Entity Specific
System Wide
Quality and rigor of internal controls to address risks to
BES reliability
Evaluated in terms of industry adopted framework or criteria
(ex. corrective action programs, elements of
operational excellence, HRO principles, etc.)
Quality and rigor of oversight, depth of procedures, etc.
• Size and interconnections• Geography/Topology• Technology• Past history, events, other
• Assessments/Studies• Past/Current Performance• Emerging Threats• Cyber (Connectivity)
Applicable Standards
“Hard”
“Soft”
Two Dimensions
Procedures, systems, training, etc.
Governance, culture, etc.
Procedures and Risk Controls
around Applicable Standards
Criteria or Principles
IDENTIFY UNDERSTAND/ADDRESS MITIGATE/PREVENT
• Governance/Training• Procedures/Checklists• Systems/”Flags”
CLARITY ▪ ASSURANCE ▪ RESULTS
Our Collective Job:Managing RISK
Strategic and Tactical
Identify Risk
UnderstandRisk
Address Risk
Prevent Risk
15
Successful organizations … have learned that the higher the risk, the more necessary it is to engage everyone's commitment and intelligence.
Margaret J. Wheatley