evolving toward a high assurance smart gridcontrol room sends command to close grid segments are out...
TRANSCRIPT
BOEING is a trademark of Boeing Management Company.Copyright © 2011 Boeing. All rights reserved. 1 of 20
Evolving Toward a High Assurance Smart Grid Through a Distributed Control Architecture
Brad [email protected]
Smart Grid Cyber Security is more than just applying IT security
to grid control linksIt is a total System Design approach
Copyright © 2011 Boeing. All rights reserved. 2 of 20
AgendaHigh Assurance
A broad term encompassing dimensions
of high security, high availability, high reliability
Review of threat examples, with Lessons Learned
Grid integration
Threat Response:
An Architectural Approach
to achieve a High Assurance Smart Grid
Copyright © 2010 Boeing. All rights reserved.
Copyright © 2011 Boeing. All rights reserved. 3 of 20
A definition
High Assurance Smart Grid (HASG) ≠
Evaluation Assurance Level (EAL) for Smart Grid
High Assurance Smart Grid (HASG) ≈
“integrated approaches for assuring reliability,
availability, integrity, privacy, confidentiality, safety, and real-time performance of complex systems…”
See High Assurance Systems Engineering (HASE) 2010 conference web site at: http://web.mst.edu/~hase/hase2010/
Copyright © 2011 Boeing. All rights reserved. 4 of 20
The ThreatLessons Learned
(IT Solutions Approach)
Apply appropriate security to remote access
Critical patch installation needs to drive trusted agent status
Data/command integrity
Defense-in-depth strategies, Firewalls & IDS
Delete user accounts after terminations
Don’t perform database updates on live systems
Don’t use administrative controls to solve system anomalies
Identify controls to critical assets
Integrated physical security
Investigate anomalous system behavior
Role based access
Secure remote (trusted) access channels
Trusted agents
Use secure radio transmissions
All necessary, but not sufficientthese do not address grid control architecture
Copyright © 2010 Boeing. All rights reserved.
Copyright © 2011 Boeing. All rights reserved. 5 of 20
NIST Smart Grid Conceptual Reference Diagram
System Details
Copyright © 2010 Boeing. All rights reserved.
Copyright © 2011 Boeing. All rights reserved. 6 of 20
High Assurance Smart Grid Attributes
Integrated Energy Management, Cyber Security and Physical Security with Defense in Depth
–
Including strong Role Based Access Control (RBAC) for people and devices
Secure distributed architecture enables autonomy and eliminates single point of failure
Assume compromise in the system
(through accident, malice or system failure) and engineer energy control systems accordingly
Auto-Responsive (AR) Loads–
if you can’t remotely control it, the remote control can’t be compromised
Creating a High Assurance Smart Gridrequires utilizing the best attributes from multiple disciplines
Firewall/External Gateway
Physical Security
PKI
Authentication, Authorization,
and Accounting
Encryption- Data in Transit
- Data at Rest
SIEM
HIDS
Host Firewall
Detection/Response
Layer
Data Protection
Layer
Service Protection
Layer
Boundary Protection
Layer
Honeypots
Protocol Access Lists
Routing Protocols
Authentication
SwitchportSecurity
NIDS Sensors Secure
NMS protocol
Firewall/External Gateway
Physical Security
PKI
Authentication, Authorization,
and Accounting
Encryption- Data in Transit
- Data at Rest
S IEM
HIDS
Host Firewall
Detection/Response
Layer
Data Protection
Layer
Service Protection
Layer
Boundary Protection
Layer
Honeypots
Protocol Access
Lists
Routing Protocols
Authentication
SwitchportSecurity
NIDS Sensors Secure
NMS protocol
Firewall/External Gateway Physical
Security (e.g. tamper protection)
PKI
Authentication, Authorization,
and Accounting
Protocol Access Lists
Encryption- Data in Transit
- Data at Rest
Autonomous Sensors
Actuators
Secure NMS
protocolIDS
Sensors
Autonomous Protection
Layer
Data Protection
Layer
Service Protection
Layer
Boundary Protection
Layer
Firewall/External Gateway
Physical Security
PKI
Authentication, Authorization,
and Accounting
Encryption- Data in Transit
- Data at Rest
SIEM
HIDS
Host Firewall
Detection/Response
Layer
Data Protection
Layer
Service Protection
Layer
Boundary Protection
Layer
Honeypots
Protocol Access
Lists
Routing Protocols
Authentication
SwitchportSecurity
NIDS Sensors Secure
NMS protocol
BulkPower
Trans-mission
Distri-bution
AMIHANBMSControl
Room DG
DSFRR
Copyright © 2010 Boeing. All rights reserved.
Copyright © 2011 Boeing. All rights reserved. 7 of 20
Defense in Depth Model See NERC Smart Grid Task Force Report Reliability Considerations from the Integration of Smart Grid
http://www.nerc.com/files/SGTF_Report_Final_posted.pdf
Copyright © 2010 Boeing. All rights reserved.
Copyright © 2011 Boeing. All rights reserved. 8 of 20
Risk = Likelihood x Impact
Risk = Threat x Vulnerability x Impact
Threats
Vulnera
bilitie
s
Power System Impact
L
Activitie
s to driv
e down unacceptable Risk
H
H
L
L
H
Risk Management Approach to Selecting Security Controls
H
H
L
Like
lihoo
d
Activitie
s to driv
e down unacceptable Risk
Power System Impact
Copyright © 2010 Boeing. All rights reserved.
Copyright © 2011 Boeing. All rights reserved. 9 of 20
Cyber Security Defense in Depth & Risk Management
Defense in Depth & Risk Management Assessment determine which controls are needed at each node or type of node
H
H
LLi
kelih
ood
Activitie
s to driv
e down unacceptable Risk
Power System Impact
SIEM
Encryption- Data in Transit
- Data at Rest Secure
NMS protocol
Routing Protocol
Authentication
Authentication,Authorization,& Accounting
Network Firewall &
External Gateway
SwitchportSecurity
PKI
NetworkIDS
Honeypots
HostIDS
HostFirewall
ProtocolAccess
Lists
Grid Controls
Cross-DomainGuard
HASH Algorithms
Physical Security
System
Boundary Protection
Layer
Correlation&Response
Layer
Data ProtectionLayer
Service ProtectionLayer
Copyright © 2010 Boeing. All rights reserved.
Copyright © 2011 Boeing. All rights reserved. 10 of 20
Ener
gy F
low
Mesh with Distributed Intelligence
Mesh with Distributed GenerationQ: Why have a distributed control architecture?
A2: It reduces risk of the Control Room as a point of failure
EnergyDistributed
Network
ControlDistributed
NetworkFu
ture
Past
Pres
ent
HierarchicalPr
esen
t
Hierarchical
Grid
Con
trol
Flo
w
A1: The Grid and Grid Control Architecture must match
G1
S1
S2
L1
L2
L3
L4
L5
L6
L8
L9
L7
G2
G3
S3
S0
AB
C
D
EF
G
H
L
Energy Flow
Original grid energy flow shown in blackInterties for increased reliability shown in blueArrows show unidirectional vs. bidirectional energy flowDoes not reflect the added complexity of small scale distributed generation
G1
S1
S2
F1
F2
F3
F4
F5
F6
F8
F9
F7
G2
G3
S3
S0
A
B
C
D
E
F
G
H
L
Energy Flow
Original grid energy flow shown in blackInterties for increased reliability shown in blueArrows show unidirectional vs. bidirectional energy flowDoes not reflect the added complexity of small scale distributed generation
Control Data Flow
ControlCenter
Copyright © 2010 Boeing. All rights reserved.
Copyright © 2011 Boeing. All rights reserved. 11 of 20
Distributed Energy Distributed Control
G1
S1
S2
F1
F2
F3
F4
F5
F6
F8
F9
F7
G2
G3
S3
S0
A
B
C
D
E
F
G
H
L
Energy Flow
Original grid energy flow shown in blackInterties for increased reliability shown in blueArrows show unidirectional vs. bidirectional energy flowDoes not reflect the added complexity of small scale distributed generation
Control Data Flow
ControlCenter
Redundancy and diversity improve reliability and security for the electricity flow
The same is true for the control architecture
Copyright © 2011 Boeing. All rights reserved. 12 of 20
High Assurance Smart Grid Substation Example
In both examples,
Control Room sends command to close
Grid segments are out of phase, which will cause damage if actuator closes
High Assurance Smart Grid comes only from integratingCyber Security, Physical Security, and Distributed Energy Management
In Substation 1, Actuator 1 trusts the command, activates, resulting in damage
In Substation 2, Actuator 2 receives a command to close, directly validates of local sensor status and Substation 3 status,
and refuses the command
Copyright © 2010 Boeing. All rights reserved.
Copyright © 2011 Boeing. All rights reserved. 13 of 20
Strong Distributed Cyber Security Enables Trusted Distributed Intelligence for Energy Control
Not just “Distributed Agents”
but Distributed Intelligence
Many Agents are just “Rules-Based”
Autonomy requires Distributed Intelligence
Software Control Agents for:
Grid Management
Cyber Security
Physical Security
Smart Grid Control Node
EMI HardenedSingle Board Computer
Real Time Operating System
Control Agents
Pw
rFlo
wC
trl
Dem
and
/R
e spo
nse
I/F
FiberNIC
Real Time Cyber Security Agent
Se n
sorI
nte g
r at io
n
Distributed Control Agents assure no action is taken based on a single input HASG leverages distributed cyber agents developed for DOD
Com
mun
icat
ions
Copyright © 2010 Boeing. All rights reserved.
Copyright © 2011 Boeing. All rights reserved. 14 of 20
Why have Load Control?
Because Loads are not smart enough to manage themselves
Potential Solutions
Increase automation and security to
achieve load device control over individual devices or groups of devices
Increase the ability of loads to manage
themselves in response to grid conditions
Copyright © 2010 Boeing. All rights reserved.
Copyright © 2011 Boeing. All rights reserved. 15 of 20
An Ethernet Comparison to the Electricity Network
Shared Attributes
Shared media: many nodes, one set of ‘network’
wires per ‘subnet’
Congestion: overloading impacts quality of service
Non-Deterministic: highly reliable when “over-engineered”, but no guarantee of service
Peak loads: impacted by predictable but random patterns (predictable random distributions)
Non-shared Attribute
Ethernet has randomness built into controls to increase reliability of data throughput
Copyright © 2010 Boeing. All rights reserved.
Copyright © 2011 Boeing. All rights reserved. 16 of 20
Auto-Responsive (AR) Load Control
PNNL Study –
load senses grid frequency–
Reduce Load when frequency is below 59.95Hz (for example)
HOWEVER, use a random function (say, 5-20 minutes) for when to resume load
Avoids creating rapid grid load oscillations
Same concept (but different time constraints) as Ethernet Collision Detection & Retry Timing
Provides a gradual increase in load after underspeed
–
Increase load when frequency is above 60.05Hz (for example)
And drop load immediately when frequency goes back below 60.05Hz
Provides a clipping function for overspeed
AMI Meter Connect/Disconnect Functions–
Function is service connect/disconnect, not life safety of workforce–
Use a random function (say, 5-20 minutes, rectangular distribution) for when meter responds
Avoids potential for rapid shocks to system if AMI control network is compromised
Overall, avoids complexity of having to build an expanded control network with hundreds of millions of control nodes on a national basis
Copyright © 2011 Boeing. All rights reserved. 17 of 20
High Assurance Smart Grid Attributes
High Assurance Smart Grid Solutionsutilizing the best attributes from multiple disciplines
Firewall/External Gateway
Physical Security
PKI
Authentication, Authorization,
and Accounting
Encryption- Data in Transit
- Data at Rest
SIEM
HIDS
Host Firewall
Detection/Response
Layer
Data Protection
Layer
Service Protection
Layer
Boundary Protection
Layer
Honeypots
Protocol Access Lists
Routing Protocols
Authentication
SwitchportSecurity
NIDS Sensors Secure
NMS protocol
Firewall/External Gateway
Physical Security
PKI
Authentication, Authoriz ation,
and Accounting
Encryption- Data in Transit
- Data at Rest
S IEM
HIDS
Host Firewall
Detection/Response
Layer
Data Protection
Layer
Service Protection
Layer
Boundary Protection
Layer
Honeypots
Protocol Access
Lists
Routing Protocols
Authentication
SwitchportSecurity
NIDS Sensors Secure
NMS protocol
Firewall/External Gateway Physical
Security (e.g. tamper protection)
PKI
Authentication, Authorization,
and Accounting
Protocol Access Lists
Encryption- Data in Transit
- Data at Rest
Autonomous Sensors
Actuators
Secure NMS
protocolIDS
Sensors
Autonomous Protection
Layer
Data Protection
Layer
Service Protection
Layer
Boundary Protection
Layer
Firewall/External Gateway
Physical Security
PKI
Authentication, Authorization,
and Accounting
Encryption- Data in Transit
- Data at Rest
SIEM
HIDS
Host Firewall
Detection/Response
Layer
Data Protection
Layer
Service Protection
Layer
Boundary Protection
Layer
Honeypots
Protocol Access
Lists
Routing Protocols
Authentication
SwitchportSecurity
NIDS Sensors Secure
NMS protocol
BulkPower
Trans-mission
Distri-bution
AMIHANBMSControl
Room DG
DSFRR
Integrated Energy Management, Cyber Security and Physical Security with Defense in Depth
–
Including strong Role Based Access Control (RBAC) for people and devices
Secure distributed architecture enables autonomy and eliminates single point of failure
Assume compromise in the system
(through accident, malice or system failure) and engineer energy control systems accordingly
Auto-Responsive (AR) Loads–
if you can’t remotely control it, the remote control can’t be compromised
Copyright © 2011 Boeing. All rights reserved. 18 of 20
The Solution –
A System Design Approach
Smart Grid Cyber Security is more than just applying IT security to grid control links – It is a total System design approach
High Assurance Architectural Requirements
Firewall/External Gateway
Physical Security
PKI
Authentication, Authorization,
and Accounting
Encryption- Data in Transit
- Data at Rest
SIEM
HIDS
Host Firewall
Detection/Response
Layer
Data Protection
Layer
Service Protection
Layer
Boundary Protection
Layer
Honeypots
Protocol Access Lists
Routing Protocols
Authentication
SwitchportSecurity
NIDS Sensors Secure
NMS protocol
Firewall/External Gateway
Physical Security
PKI
Authentication, Authorization,
and Accounting
Encryption- Data in Transit
- Data at Rest
S IEM
HIDS
Host Firewall
Detection/Response
Layer
Data Protection
Layer
Service Protection
Layer
Boundary Protection
Layer
Honeypots
Protocol Access
Lists
Routing Protocols
Authentication
SwitchportSecurity
NIDS Sensors Secure
NMS protocol
Firewall/External Gateway Physical
Security (e.g. tamper protection)
PKI
Authentication, Authorization,
and Accounting
Protocol Access Lists
Encryption- Data in Transit
- Data at Rest
Autonomous Sensors
Actuators
Secure NMS
protocolIDS
Sensors
Autonomous Protection
Layer
Data Protection
Layer
Service Protection
Layer
Boundary Protection
Layer
Firewall/External Gateway
Physical Security
PKI
Authentication, Authorization,
and Accounting
Encryption- Data in Transit
-Data at Rest
SIEM
HIDS
Host Firewall
Detection/Response
Layer
Data Protection
Layer
Service Protection
Layer
Boundary Protection
Layer
Honeypots
Protocol Access
Lists
Routing Protocols
Authentication
SwitchportSecurity
NIDS Sensors Secure
NMS protocol
BulkPower
Trans-mission
Distri-bution
AMIHANBMSControl
Room DG
DSFRR
Apply appropriate security to remote accessCritical patch installation needs
to drive trusted agent statusData/command integrityDefense-in-depth strategies,
Firewalls & IDSDelete user accounts after
terminationsDon’t perform database updates
on live systemsDon’t use administrative controls
to solve system anomalies Identify controls to critical assets Integrated physical security Investigate anomalous system
behaviorRole based accessSecure remote (trusted) access
channelsTrusted agentsUse secure radio transmissions
IT Lessons Learned
Copyright © 2011 Boeing. All rights reserved. 19 of 20
The Integrated Solution for a High Assurance Smart Grid: Energy Management, Cyber Security and Physical Security
1.
Engineer energy control systems using High Assurance principles
–
From utility, aviation, space and government systems
2.
Defense in depth
–
Best attributes of cyber security in a layered approach
3.
Risk management approach to selecting cyber security controls
–
Select from Defense in Depth controls based on Risk Assessment
4.
Distributed intelligence
–
For Cyber Security, Physical Security and Grid Management
5.
Increase use of Auto-Responsive (AR) load management
–
Enhance grid stability without expansive Command & Control systems
–
If you can’t remotely control it, the remote control can’t be compromised
High Assurance Smart Grid Solutions –An integrated approach across multiple disciplines