The Rare Glitch Project:Scenario Graph Generation andMDP-Based Analysis

The Rare Glitch Project:Scenario Graph Generation andMDP-Based Analysis

Computer Science DepartmentCarnegie Mellon University

Pittsburgh, PA

Jeannette M. Wing

The work on survivability analysis is funded by DARPA. It is done jointly with Somesh Jha (University of Wisconsin) and Oleg Sheyner (CMU graduate student).

Carnegie Mellon: The Rare Glitch Project 2 Jeannette M. Wing

Overview of Our MethodOverview of Our Method

Checker

(Network) Model

(Survivability) Property

Phase 1

Scenario Graph

Analyzer

Reliability Query,Cost Query, etc.Phase

2

Scenario Set

Carnegie Mellon: The Rare Glitch Project 3 Jeannette M. Wing

Relation to ARO ProposalRelation to ARO Proposal

Hypothesis: Our two-phase method and tool suite are applicable to analyzing embedded systems.

Rationale Network model is simpler (bus-based, not end-to-end). Reliability and cost are important factors in embedded

systems design.

Our plan Enrich and make current tool suite robust. Apply method to embedded systems examples. Pursue foundational issues wrt models for reliability

and cost analyses.

Carnegie Mellon: The Rare Glitch Project 4 Jeannette M. Wing

Survivable SystemsSurvivable Systems

What if a terrorist hacker brings down the nation’s power grid? an act of Mother Nature causes the international

financial network to fail?

Critical infrastructures Utilities: gas, electricity, nuclear, water, … Communications: telephone, networks, … Transportation: airlines, railways, highways, … Medical: emergency services, hospitals, … Financial: banking, trading, …

Carnegie Mellon: The Rare Glitch Project 5 Jeannette M. Wing

SurvivabilitySurvivability

A system is survivable if it can continue to provide end services despite the presence of faults.

Carnegie Mellon: The Rare Glitch Project 6 Jeannette M. Wing

Modeling for Survivability AnalysisModeling for Survivability Analysis

Our starting point Handle both benign and malicious faults.

Throw out independence assumption. Incorporate semantics of end service in model.

Do not necessarily treat nodes and links the same. Include cost in the model from the start.

Steps in our approach Model general network topology (nodes and links) Analyze in two phases for

Functional behavior Reliability, cost, etc.

Carnegie Mellon: The Rare Glitch Project 7 Jeannette M. Wing

Phase 1Phase 1

Network Model =

Survivability Property =

Scenario Graph =

Model Checker = (modified) NuSMV

A set of concurrently executing Finite State Machines.

A predicate in CTL.

A set of related examples.

Carnegie Mellon: The Rare Glitch Project 8 Jeannette M. Wing

Simple Example: A Banking SystemSimple Example: A Banking System

FRB 1

FRB 3FRB 2

MC 1 MC 3MC 2

Bank A Bank CBank B

a1a2

b1 b2 c1

Carnegie Mellon: The Rare Glitch Project 9 Jeannette M. Wing

Network ModelNetwork Model

Processes Nodes and links are processes (i.e., FSMs)

banks, money centers, federal reserve banks, and links Communication via shared variables (i.e., finite

queues) representing channels, and hence interconnections.

Failures Faults represented by special state variable

fault:{normal, failed, intruded} Links and banks can fail at any time

Failed link blocks all traffic. Failed bank routes all checks to an arbitrarily chosen

money center. Money centers and federal reserve banks do not fail.

Carnegie Mellon: The Rare Glitch Project 10 Jeannette M. Wing

Survivability PropertiesSurvivability Properties

Fault-related Money never deposited into wrong account.

AG(error)

Service-related A check issued eventually clears.

AG(checkIssued AF(checkCleared))

Carnegie Mellon: The Rare Glitch Project 12 Jeannette M. Wing

Scenario GraphsScenario Graphs

Given a state machine, M, and a property, P, a scenario graph is a concise representation of the set of traces of M with respect to P. P = fault property

A fault scenario graph represents all system traces that end in a state that does not satisfy P.

P = service property A service success (fail) scenario graph represents all

system traces in which an issued service successfully finishes (fails to finish).

Carnegie Mellon: The Rare Glitch Project 13 Jeannette M. Wing

Fault Scenario GraphFault Scenario Graph

Intuition:

• Each “counterexample” spit out by the model checker is a scenario.

• Survivability property gives a slice of the model.

Each path is a scenario of how a fault can occur.

Carnegie Mellon: The Rare Glitch Project 14 Jeannette M. Wing

Survivability PropertiesSurvivability Properties

Fault-related Money never deposited into wrong account.

AG(error)

Service-related A check issued eventually clears.

AG(checkIssued AF(checkCleared))

Carnegie Mellon: The Rare Glitch Project 15 Jeannette M. Wing

A Service Success Scenario GraphA Service Success Scenario Graph

issueCheck(A, C)

send(A, MC-2)

send(MC-2, FRB-1)

send(FRB-1, FRB-3)

send(FRB-3, MC-3)

send(MC-3, C)

debitAccount

send(FRB-2, FRB-3)

send(MC-1, FRB-2)

send(A, MC-1)

up(a2)

up(c1)

down(a2) & up(a1)

Carnegie Mellon: The Rare Glitch Project 16 Jeannette M. Wing

A Service Fail Scenario GraphA Service Fail Scenario Graph

issueCheck(A, C)

FAIL

down(A)

up(a2)

up(A)

pick(MC-2)

down(c1)

down(a2)

pick(MC-1)

down(a1)

down(c1)

down(c1)

up(a1)

send(A, MC-2) send(A, MC-1)

FAIL

FAIL

FAIL

Carnegie Mellon: The Rare Glitch Project 17 Jeannette M. Wing

Overview of MethodOverview of Method

Network Model

Reliability Query,Cost Query, etc.Analyzer

Scenario Set

Survivability Property

Phase 2

Phase 1

Scenario Graph

Checker

Annotations(e.g., probabilities, cost)

Carnegie Mellon: The Rare Glitch Project 18 Jeannette M. Wing

Phase 2: Reliability Analysis (in a Nutshell)Phase 2: Reliability Analysis (in a Nutshell)

Annotations = Probabilities Use Bayesian Networks to model dependence of

events.

Symbolic Use symbolic probabilities

high, medium, low Use NDFA theory to compute scenario set.

Continuous Use numeric probabilities

[0.0, 1.0] Use Markov Decision Processes to model both

nondeterministic and probabilistic transitions.

Carnegie Mellon: The Rare Glitch Project 23 Jeannette M. Wing

Phase 2: Continuous AnalysisPhase 2: Continuous Analysis

Use real values for probabilities.

May leave probabilities of some events unspecified.

Markov Decision Processes Mix of nondeterministic and probabilistic transitions

Why? System is not closed. Hard to assign probabilities to some faults (e.g.,

intrusions). Environment makes choice (i.e., decision) and can be

demonic!

Carnegie Mellon: The Rare Glitch Project 24 Jeannette M. Wing

Reliability AnalysisReliability Analysis

Goal of (malicious) environment: Devise an optimal policy to minimize reliability.

Assign to each state, s, a value, V(s), computed using a standard policy iteration algorithm from MDP literature.

Let V* be the value function after convergence. Then, for initial state of scenario graph, s0, V*(s0) computes worst-case probability of service eventually finishing.

Carnegie Mellon: The Rare Glitch Project 25 Jeannette M. Wing

A Typical ExampleA Typical Example

V(Bad) = 0.0V(Good) = 1.0

0.6

0.7

0.6

BadGood

0.40.

60.3

0.7

0.65

Carnegie Mellon: The Rare Glitch Project 26 Jeannette M. Wing

Bayesian Network for Bank ExampleBayesian Network for Bank Example

P(a2 | a1) = 1/2

P(a2 | a1) = 1/4

P(a1) = 1/2 a1

a2

c1

P(c1) = 1/2

b2

P(b2) = 1/2

b1

P(b1) = 1/2

Carnegie Mellon: The Rare Glitch Project 27 Jeannette M. Wing

A Service Success Scenario GraphA Service Success Scenario Graph

issueCheck(A, C)

send(A, MC-2)

send(MC-2, FRB-1)

send(FRB-1, FRB-3)

send(FRB-3, MC-3)

send(MC-3, C)

debitAccount

send(FRB-2, FRB-3)

send(MC-1, FRB-2)

send(A, MC-1)

up(a2)

up(c1)

down(a2) & up(a1)3/8 1/4

1/2The worst case probability that a check issued by Bank A on Bank C is

(1/2 * 3/8) + (1/2 * 1/4) = 5/16

Carnegie Mellon: The Rare Glitch Project 28 Jeannette M. Wing

Cost-Benefit AnalysisCost-Benefit Analysis

Goal: Choose a set of links to upgrade to achieve higher reliability, given my cost constraints (e.g., fixed budget).

Identify new actions that correspond to decisions an architect needs to make (e.g., upgrade a1).

Associate a cost with each action.

Define constraints on costs.

Carnegie Mellon: The Rare Glitch Project 29 Jeannette M. Wing

Upgrade Links in Banking SystemUpgrade Links in Banking System

FRB 1

FRB 3FRB 2

MC 1 MC 3MC 2

Bank A Bank CBank B

a1a2

b1 b2 c1??

?

Carnegie Mellon: The Rare Glitch Project 30 Jeannette M. Wing

Cost Constraint ExampleCost Constraint Example Assume:

If we upgrade a1 and c1 then P(a1) and P(c1) both increase to 3/4 .

If a2 is upgraded, then P(a2) is: P(a2|a1) = 3/4 P(a2| a1) = 3/8

Aim: Maximize the worst-case reliability subject to the constraint that at most two links can be upgraded. Solve this non-linear integer programming problem:

xa1 + xa2 + xc1 < 2

Best option: Upgrade a1 and c1. xa1 = 1 and xa2 = 1 7/16

xa1 = 1 and xc1 = 1 39/64xa2 = 1 and xc3 = 1 9/16

Carnegie Mellon: The Rare Glitch Project 31 Jeannette M. Wing

Constrained Markov Decision ProcessesConstrained Markov Decision Processes

<S, A, P, c, d>

S is a finite state space.

A is a finite set of actions.

P are transition probabilities. Psas’ is the probability of moving from state s to s’ if action a is chosen.

c: (S x A) is the immediate cost. c(s, a) is the cost of choosing action a at state s.

d: (S x A) k is a k-dimensional vector of immediate costs, captures additional cost constraints.

Carnegie Mellon: The Rare Glitch Project 32 Jeannette M. Wing

Survivability Case StudiesSurvivability Case Studies

Somesh Jha Trading floor model of major investment bank (being

“sanitized”) 10K lines of NuSMV half-million nodes in scenario graph 50 threat scenarios 45 found by system 5 new threat scenarios found With independence assumption, too many misses.

B2B e-commerce NYC start-up (Jha) 50K lines of Statecharts 2 million NuSMV beyond capability of tool

Oleg Sheyner Intrusion detection (ongoing)

Carnegie Mellon: The Rare Glitch Project 33 Jeannette M. Wing

Intrusion Detection System Case StudyIntrusion Detection System Case Study Done by Oleg Sheyner and Lincoln Labs.

Motivated by hand drawn poster of attack scenarios.

Illustrates only first part of method.

Carnegie Mellon: The Rare Glitch Project 35 Jeannette M. Wing

Example of Attack Tree Developed by a Professional Red TeamExample of Attack Tree Developed by a Professional Red Team

Sandia Red Team “White Board” attack tree from DARPA CC20008 Information battle space preparation experiment

Carnegie Mellon: The Rare Glitch Project 36 Jeannette M. Wing

Phase 1 Example:Multistage Network Penetration

database

adversary

ftp

sshd

firewall router

IDSip1

ip2

ipaftp

Goal: Root access to host ip2

Attack Arsenal

• Sshd buffer overflow - remotely get root• Ftp .rhosts file - establish trust between hosts• Remote login - exploit trust between hosts• Local buffer overflow - locally get root

Detected

Carnegie Mellon: The Rare Glitch Project 38 Jeannette M. Wing

NuSMV EncodingNuSMV Encoding

Network 1 attack host, 2 target hosts

with services 3x3 connectivity matrix

existence of routing path ability to connect to ftp and

ssh services 3x3 trust matrix

Adversary Privilege levels for each

host

Attacks 4 attacks some have multiple flavors

NuSMV Statistics 82 bits of state (282 states)

<40K representation nodes~7000 reachable states

2 sec runtime on 1GHz Pentium III

8MB of memory used

Carnegie Mellon: The Rare Glitch Project 39 Jeannette M. Wing

Scenario-Generating PropertiesScenario-Generating Properties

Don’t care about detection

AG (adversary.privilege[2] < root)

Want stealth

AG ((adversary.privilege[2] < root) or (IDS.detected))

Carnegie Mellon: The Rare Glitch Project 43 Jeannette M. Wing

The Rare Glitch Tool SuiteThe Rare Glitch Tool Suite

…

Prism

Checkmate

PVS

Symp

nuSMV

Specification

Model

Counter-examples

Checkers and Provers

Specification and Modeling Languages

Analysis Engines

Reliability and Cost Analyzers

Explanation Generator

Abstraction/Refinement Processor

…

XMLScenario graphXML2nuSMV

Carnegie Mellon: The Rare Glitch Project 44 Jeannette M. Wing

Plan in Relationship to The Rare Glitch ProjectPlan in Relationship to The Rare Glitch Project Enrich and make current tool suite robust.

Integrate with other existing project tools.

Apply method to embedded systems examples. Work with Clarke to add reliability analysis to

automotive example.

Pursue foundational issues wrt models for reliability and cost analyses. Understand relationship to other probabilistic models,

hybrid models, etc.