© 2015 Electric Power Research Institute, Inc. All rights reserved.

Integrating IT, OT, and Physical

Security Events for Enterprise-Wide

Situational Awareness

Centralized Threat

Management

Ralph E. King

Principal Project Leader

European Engagement Summit

April 29, 2015

2© 2015 Electric Power Research Institute, Inc. All rights reserved.

Protective Measures

Network Management Systems

DNP3 Secure Authentication v5

EPRI’s Cyber Security and Privacy Program:

Cyber Security Technology Projects for 2015:

Managing Cyber Incidents

Integrated Threat Analysis Framework

Security Incident Management Task Force

Integrated Security Operations Center

3© 2015 Electric Power Research Institute, Inc. All rights reserved.

Centralized Threat Management

Integrated Security Operations Center

20142015

2013

4© 2015 Electric Power Research Institute, Inc. All rights reserved.

Integrated Security Operations Center

2015 Project Plan (Base Program)

Report: Guidelines for Integration of

Substations and Field Devices into an ISOC

2013 Report: Guidelines for Planning an Integrated Operations

Center

2014 Report: Guidelines for Integration of

Control Center Systems into an

ISOC

ISOC Architecture & Lab Testbed

for SubstationsUse Cases for

Substation Domain

Technology Transfer

Workshop

5© 2015 Electric Power Research Institute, Inc. All rights reserved.

ISOC Conceptual Architecture

6© 2015 Electric Power Research Institute, Inc. All rights reserved.

Security Event Sources

Network

DevicesFirewalls Intrusion

Prevention

VPN /

Remote AccessServers

Smart MetersSubstation

DevicesSCADA

Operational Security Events

Personnel

MonitoringCameras Fire Alarms Badge

Readers

Building

Automation

IT Security Events

Physical Security Events

ISOC Security

Events

?

Grid Operations Events

7© 2015 Electric Power Research Institute, Inc. All rights reserved.

Examples of Correlated Alarms in an ISOC

An employee has logged

into a SCADA workstation

in the Control Center.

The Physical Access Control

System badge credentials to

access the Control Center do

not match the login credentials

from the Identity Management

System.

An IT employee logs in

remotely to perform

maintenance on the

Historian database.

No work order exists in the

Work Management System for

this work to be performed.

The ISOC is alerted that a

USB drive being plugged

into a field device at the

substation.

The physical security

monitoring center is alerted of a

cut fence and an unauthorized

person on the grounds of a

substation.

8© 2015 Electric Power Research Institute, Inc. All rights reserved.

High-Level Data Flow for Substation and Field Systems

Locations

Regions

ISOC SIEM

Region 1

Region 2

Region 3

Location 3a

Location 3b

Region (n)

Operational

Devices

IT Devices

Physical Security

Devices

Mediation

Device

• Logging,

normalization, &

aggregation

• Forward data to

SIEM

• Store logs

• Buffering

capability

9© 2015 Electric Power Research Institute, Inc. All rights reserved.

EPRI Cyber Security Lab/Smart Grid Substation Lab

Example Test bed: Architecture for FirstEnergy TSOC Project

CSRL ArcSight

Security Information & Event Management (SIEM)

LogRythm Splunk

Region 1Radiflow

SGSL

FLIR

A310pt

Honeywell

HD4MWI

Honeywell

HDZ20H

Honeywell

H4D1FR

FLIR

064Y2

Honeywell

H4D2F

Location

1A

TrendNet

TPE-1020

Badge Readers

PW-6000

Cisco

302-08MP

Cisco

CGS-2520

Cisco

CGR-2010

SEL 3620

Ruggedcom

2100

GE D60

Cooper

Cybectec

Door

Contacts

SEL 3622

Boomerang

Ballistic SW

Honeywell

HDZ20H

10© 2015 Electric Power Research Institute, Inc. All rights reserved.

Mediation Device Example

Radiflow 3180 Gateway

– Single source from the location (Substation) to SIEM

– Single aggregation point for (IT, OT, PS) devices

– Cost reduction for SIEM tool (one interface for gateway vs. each device)

– Store and forward real time data from devices

– Support NERC CIP substation device monitoring requirement

– Ability to forward messages in common format

– Multiple Interface to support IP, Serial connectivity

– Made minimal device modifications to meet FE requirements

Function as a syslog server

Log and store inbound and out bound data

Ability to apply pattern matching and filtering to inbound messages

Support (syslog, auditlog, ascii txt) messages

11© 2015 Electric Power Research Institute, Inc. All rights reserved.

Radiflow Message Handling

12© 2015 Electric Power Research Institute, Inc. All rights reserved.

Radiflow Message Handling

13© 2015 Electric Power Research Institute, Inc. All rights reserved.

Security Cameras

14© 2015 Electric Power Research Institute, Inc. All rights reserved.

Ballistic Detection Simulator

15© 2015 Electric Power Research Institute, Inc. All rights reserved.

PG&E Metcalf Substation Attack– April 2013Wall Street Journal Article (February 4, 2014)

■ Shooting occurred for 19 Minutes

■ Telephone cables were cut

■ Surgically knocked out 17 giant

transformers

■ Rerouted power to avoid blackout

■ Police arrived 1 minute after

shooters disappeared

■ 27 days to repair and restore

system

16© 2015 Electric Power Research Institute, Inc. All rights reserved.

Integrated Threat Analysis Framework (ITAF)

New EPRI Supplemental Project

Addresses the barriers in correlating security and grid

operations events

Integration between Cyber Security, Physical Security, and

Grid Operations

Currently in Project Initiation Phase

Plan full launch by May 4

17© 2015 Electric Power Research Institute, Inc. All rights reserved.

ISOC + Grid Operations Events = ITAF

Behavioral Learning

Appliances

Industrial Security

Appliances

Physical Security

Systems

Threat and

Vulnerability

Information

Sources

Grid

Operations

Events

OT Security Events

Field Devices

Substation Gateways

Control Center Systems

IT Security Events

Network Device Logs

IT System Logs

Business Systems

Field Network

Operations Center

Reporting

Log and Event Aggregation

Correlation Engine

Security Information and Event Management (SIEM)

18© 2015 Electric Power Research Institute, Inc. All rights reserved.

Data & Information Flow for Power Systems DataU

tilit

y

Sta

ffC

om

mS

ub

sta

tion

s

Sub 1 Sub 2 Sub 3 Sub 4

• Integration happens at the Utility Staff level

• Validated data is sent to Security Operations (SO) and security confirmation is sent back to GO and GM

GO SO GM

One or More Data Buses

GO’ GM’Grid

Operations

Grid

MaintenanceAlertAlert

Security

Operations

19© 2015 Electric Power Research Institute, Inc. All rights reserved.

Integrated Event Analysis Framework (ITAF)

Objectives and Scope Address the barriers to integrating power

system operations events into a security operations center by:

– Developing security event scenarios

– Identifying operational and asset condition data sources to support event detection

– Developing an event analysis framework

– Testing scenario detection in EPRI’s lab as well as utility host sites

Value Centralizes threat analysis for security

and grid operations for significantly improved threat response

Details and Contact Price:

- Level 1: $50,000- Level 2: $75,000

Project will start in 2015

Ralph King, Principle Technical Leader

reking@epri.com, (865) 218-8160

To Join, contact ICCS Technical Advisor:

Scott Sternfeld

ssternfeld@epri.com, (843) 813-4593

SPN Number: 3002005065

Address the barriers in correlating security and operations events

20© 2015 Electric Power Research Institute, Inc. All rights reserved.

2014 Cyber Security Technologies Reports

Report Title Product ID

DNP3 (IEEE Std 1815TM) Secure Authentication:

Implementation and Migration Guide and Demonstration Report3002003736

Network System Management: Implementations and Applications

of the IEC 62351-7 Standard3002003738

Guidelines for Integrating Control Center Systems Into an

Integrated Security Operations Center3002003739

How to download EPRI Reports:

1. Go to www.epri.com

2. Type the Product ID in the Search Bar

21© 2015 Electric Power Research Institute, Inc. All rights reserved.

Together…Shaping the Future of Electricity