Presented to:

Presented by:

The Perfect Storm – Cyber RDT&E

ITEA Cyber Workshop

John Ross

25 February 2015

NAVAIR 5.4H – Cyberwarfare Lead

NAVAIR Public Release 2015-87

Approved for public release; distribution unlimited

BLUF

• Weaponization of the Cyber domain has created a "perfect

storm" of new requirements and challenges for the Research

Development Test and Evaluation (RDT&E) community

• Programs and Platforms must now consider and test for:

– Offensive and defensive Cyber effects in realistic battle-

space scenarios

– Operational resilience against Cyber threats

– Mission impact of Cyber effect in System of Systems (SoS)

• These new requirements necessitate innovative Modeling and

Simulation (M&S) solutions and creation of RDT&E

environments that include effects of both offensive and

defensive Cyber Warfare

1

RDT&E Cyber STORM

2

cc

• Proliferation of software systems into weapons

systems creates new vulnerabilities

• Integrated SoS drives increased use of networking

• Dynamic and complex Battle-space environment

• Interoperability and integration

• Advanced Persistent threat

• Commercial Software

– Widely Exploited

– Obsolescence

• Common Weaknesses

• Zero Day Exploits

• Cybersecurity Patch Management

– Develop and test mitigations and patches

before deployment

• Multiple Versions and configurations

• Legacy Systems

– Keep interoperable with newer fielded systems

– Sustainment

• Non-proprietary and open architecture

– Increased Cyber vulnerabilities and exposure

based on open concept

Attack Surface: A system’s exposure to reachable and exploitable cyber vulnerabilitiesSource: SANS Attack Surface Problem: http://www.sans.edu/research/security-laboratory/article/did-attack-surface

Battlespace Integration

Joint – Complex – Integrated - Collaborative

SYSTEMS

OF

SYSTEMS

System Design and Development

Acquisition Process System Focused

Each System Developed to Program Requirements

without assurance of System Interoperability at

Fleet Introduction

Systems of Systems Environment Requires

New and Innovative Approaches throughout

Development to ensure Systems are

Interoperable – Interoperability Must be Built In

Fleet Introduction

Simulations (DoD, Ctr)

Systems Integration Labs (DoD, Ctr)

Installed Systems Test Facilities (DoD, Ctr)

Open Air Ranges (DoD)

Live Assets (DoD, coalition, other agency)

Fleet Interoperable

Multiple Cyber Attack Surfaces

3

Cybersecurity Testing

– Categorize system and operational

environment

– Select required Cybersecurity

controls

– Implement Cybersecurity

– Assess Cybersecurity controls

• Perform Security scans to verify

compliance with mitigations

– Authorize system

• Develop supporting

documentation for accreditation

– Monitor Cybersecurity controls

• Security scans and patches to

address vulnerabilities

4

cc

Weapon systems are NOT adequately addressed

• Traditional Cybersecurity framework must be applied differently for

weapon systems

Cyber Hardening Strategy • Enable Testing of Cybersecurity defensive measures during concept, design,

development and deployment

– Perform vulnerability assessments against defined standards early in

development cycle

• Assess the Cyber defense capability of the System Under Test (SUT) in a

realistic combat environment enabled by Live, Virtual, Constructive (LVC)

• Perform Cybersecurity assessment of Operational Test (OT) readiness in

context of Protect, Detect, React and Restore (PDRR)

• Emulate the system or critical components that are susceptible to Cyber

threat

– Not practical to expose fielded weapon systems to malicious code

• Develop capability to simulate Cyber threat against systems and platforms

• Develop instrumentation capable of distinguishing between performance

issues and malicious code

– Collect data to assess operator, system, and malicious activity

5

Cybersecurity T&E • Test early and often to prevent proliferation of vulnerable designs

• Cybersecurity T&E of real-time systems with unique bus interfaces

• Cybersecurity hardening and non-traditional testing methods

– Non-networked and intermittent connectivity

– Unique domain issues

• Holistic approach to Cybersecurity testing to requirements, influences areas to be tested

based on potential attack surfaces and impacts to mission from a Cyber attack

• Concurrent Cyber System Engineering

– System lifecycle Cybersecurity T&E

– Developers must design in cybersecurity measures

– Perform risk reduction events

– Identify mission effectiveness measures

6

MS B

Understand

Cybersecurity

Requirements

Characterize

Cyber Attack

Surface

Cooperative

Vulnerability

Identification

Adversarial

Cybersecurity

DT&E

Full Rate

Production

Decision Review

MS CMS A

Technology

Maturation &

Risk Reduction

Engineering &

Manufacturing

Development

Production and

Deployment O&S

SRR SFR CDR TRR SVRASR

Materiel

Solution

Analysis

MDDDRAFT

CDDAOA CDD CPD

IATT

Vulnerability and

Penetration

Assessment

IOT&E

T&E

Phases

OTRR

DT&E

Event

ATO

Adversarial

Assessment

Req

DecisionPre-

EMD

DT&E

Assess-

ment

DT&E

Assess-

ment

PDR

Cyber Initiatives Supporting RDT&E

• NAVAIR Cyber Warfare Detachment (CWD)

• Federated (Red) Penetration Team Partnerships

– Navy Information Operations Command

(NIOC) Norfolk

– Threat Systems Management Office (TSMO) -

Army

• Cyber Test Analysis and Simulation Environment

(CyberTASE)

• National Cyber Range (NCR)

• Regional Service Delivery Points (RSDP)

7

NAVAIR Cyber Warfare Detachment

• NAVAIR Cyber Warfare Detachment established to address Cyber requirements and gaps

– Create a Cyber-aware workforce with right mix of Cyber and domain system expertise

– Create integrated Cyber policies, processes, best practices and standards

– Smart make/buy Cyber infrastructure decisions to support our weapons systems and business systems

– Deliver Cyber-resilient integrated warfighting capabilities

– Partner - leverage external Cyber expertise

8

Cyber Infrastructure /

R&D Investments

NAVAIR Cyber Warfare Detachment

9

Efforts

• Conduct prioritized risk assessments

of deployed weapon systems

• Cross-competency teams

• Identify access points

• Maintenance connections,

removable media, intermittent

connections, apertures, supply

chain

• Influence Cyber Security System

Engineering

• Cyber resiliency…

• Mission Analysis

• Kill chain mission effects

• Field Response

• System Design

• Build adequate Systems-of-Systems

(SoS) architecture / system

documentation

Cyber hygiene does not fully mitigate sophisticated attacks

CyberTASECyber Test Analysis and Simulation Environment

• Development of testing instrumentation to assess how defensive

mechanisms perform against an ongoing cyber attack and the correlation of

data gathered across Cyber stacks

• Live-Virtual-Constructive (LVC) environment capable of mimicking large

scale operational scenarios with Cyber instrumentation

• Support evaluation of operational resilience against Cyber threats utilizing

instrumentation, models, and simulations that perform data collection,

monitoring, near real-time and post-test analysis, storage, and visualization

of test data

10

SUT

Red Team - Portray

Advanced Persistent

Threat

Operators Exercise

SUT, Mission Threads -

Protect, Detect, React,

Restore

National Cyber Range (NCR)

Encapsulation Architecture &

Operational ProceduresComputing Assets/Facility

Integrated Cyber Event Tool Suite Cyber Test Team

NCR provides secure facilities, innovative technologies, repeatable processes, and the skilled workforce necessary to rapidly create hi-fidelity, mission representative Cyberspace environments…

11

RSDP• Regional Service Delivery Points (RSDPs):

– Provide enterprise resources to generate virtualized representative cyber

environments

– Provide increased capacity and scalability to create persistent,

representative cyber-threat environments

– Provide common range services (i.e., traffic generation, simulation,

instrumentation, visualization, and integrated event management)

– Flexible and adaptable to evolving users requirements

– Leverage the latest technology to deliver cost and performance efficiencies

– Key component of the JMETC MILS Network (JMN)

12

Address Cyber T&E Capacity & Capability Gaps

M&S and LVC for Cyber Testing

13

cc

• Linking system-of-systems and families-of-systems in

distributed test environment to assess cyber resilience

– Deliver Cyber resilient integrated warfighting capabilities

– Determine mission critical components necessary to achieve

objective

– Assessment of kill chain impact and Cyber effects on mission

– Performance of SUT subjected to Cyber effects to inform

further detailed Cyber testing

– Adding Cyber components to the existing modeling and

simulation that model and capture the Cyber effects and

capture mission impact

– Ability to operate system and in presence of Cyber attack

Live Virtual Constructive

Kill Chain

Assessment

System of

Systems

Evaluation

Survivability

and

Resiliency

Design and

Development

Reconstruction

and Regression

Analysis

Installed Systems Cyber Testing

14

• Key Elements of Cyber M&S for Mission

Level Testing

– Authoritative cyber data model to

emulate in M&S realistic environment

– Assess Cyber effect propagation

throughout the system or system of

systems

– Perform analysis of Cyber effects and

ability of systems and operators to

detect and mitigate

– Repeatable methodology for

evaluation of the Cyber test results

– Effect of Cyber attack on mission

outcome

Installed Systems Cyber testing supports identifying susceptibilities of

attack surfaces within the system or system of systems

Cyber Needs and Gaps

• Ability to assess kill chain and mission impact

• Realistic models for constructive T&E

– Threat vectors and behaviors

– Engineering level models of SUT feeding

higher level models

– Authoritative data sources for cyber threats

• Development of autonomous defensive

measures to mitigate Cyber effects

• Warning indications that the system is under

attack

15

Key Take Away

• Conduct M&S to assess Cyber effects

• Perform Cyber risk assessments

• Develop Cyber laboratories and tools for

offensive and defensive techniques and

measures

• Increase investments in Cyber workforce,

processes, and infrastructure

16

Questions

17

Think like a Hacker…

Insights - Ideas