NCSD-ADS-DOC-3810-2.0-20070412

Wagner Associates

ARO Workshop on Cyber Situation Awareness

RPD-inspired Hypothesis Reasoning for Cyber Situation Awareness

November 14, 2007John Yen, Mike McNeese, and Peng Liu

COLLEGE OF INFORMATIONSCIENCES AND TECHNOLOGY

2

COLLEGE OF INFORMATIONSCIENCES AND TECHNOLOGYOverview

• Cognitive Foundation: RPD Model• RPD-enabled Collaborative Agents: R-

CAST• Hypothesis Reasoning in R-CAST• Similarity-based Activation of

Hypothesis• Gathering Missing Relevant Information

3

COLLEGE OF INFORMATIONSCIENCES AND TECHNOLOGYRecognition-Primed Decision

• A cognitive model of human decision-making under time pressure.

• A naturalistic decision-making model• A holistic decision-making model

– Includes gathering relevant information– Captures the entire decision making process,

not just the “decision point”.

• An adaptive decision-making process– Includes detecting changes in the environment

so that decisions can be adapted.

4

COLLEGE OF INFORMATIONSCIENCES AND TECHNOLOGY

Three Types of Relevant Informationin RPD Model

– Missing Cues

– Criteria for Evaluating

Options

– Expectancy

Adapted from G.A. Klein 1989

start

end

missinformation complete

information

workable

not workable

Investigation Feature matching

Expectancy monitor Evaluate option

Implementoption

Situation analysis

anomalies detected

Learning

5

COLLEGE OF INFORMATIONSCIENCES AND TECHNOLOGYRPD-enabled Agents: R-CAST

Manage Information Requirements

Anticipate Information Requirements

Knowledge base

Inference Rules

Relate high-level info needsto lower-level information

New/missing information

RPD Decision Model

Experiences

Decisions

Deliberated decisions:What to do?

Evaluation Criteria

Recommender Option

How to evaluation options?

Process manager

PlanKnowledge

Execute/Monitor

How to implement it?

How to seek/share information?

Information manager

InvestigationStrategies

Information Requirements

How to communicate?

Communication manager

Directory & protocol

Conversations

What cues are needed? What expectancies are monitored?Who needs it?Deadline?

start

end

missinformation complete

information

workable

not workable

Investigation Feature matching

Expectancy monitor Evaluate COA

ImplementCOA

Situation analysis

anomalies detected

Learning

RPD Model

R-CAST

Investigation in RPD

Information Manager in R-CAST

6

COLLEGE OF INFORMATIONSCIENCES AND TECHNOLOGYHypothesis Reasoning

• Hypothesis guides the seeking of relevant information.

C o llab o r a tiv eD ec is io nM ak in g

C o llab o r a tiv eE v id en c eC h ain in g

Ho m e P ag e

Pag e 1

Pag e 2

Pag e 3Pag e 4

Pag e 5

Pag e 6

Op tio n1

Op tio n2

Op tio n1

Op tio n2

Op tio n1

Op tio n2

Op tio n1

Op tio n2

Op tio n2

Op tio n1

Su b 1Su b 1

Su b 1

Su b 1

Su b 1

Su b 1Su b 1

Su b 2

Su b 1

Su b 1

Su b 2

Su b 2

Su b 2

Su b 2

Su b 1

Su b 2

Su b 2

Su b 2

Su b 2

Su b 2

Su b 3

Su b 4

Su b 3

Su b 3

Su b 3

Su b 3

Su b 3

Op tio n1

Su b 1Su b 2

Op tio n2

Su b 1Su b 2

Evidence S

pace O O

Hypothesis S

pace

R C A S T

R C A S T

R C A S T

f o r m in g /r e f in in g

ev o lv in gs u p p o r tin g

tr ig g er in g

7

COLLEGE OF INFORMATIONSCIENCES AND TECHNOLOGYHypothesis Reasoning in R-CAST

H y po th e s isM a n a g e r

K n o wle dg e -ba s eM a n a g e r

C o m m u n ica t io nM a n a g e r

I n fo rm a t io nM a n a g e r

D e cis io nM a n a g e r

G o a l /S itu a t io n

A ct io n

K n o wle dg eB a s e

A g e n tD ire cto ry

M u lti -Laye rB aye s ianN e twor k

8

COLLEGE OF INFORMATIONSCIENCES AND TECHNOLOGY

9

COLLEGE OF INFORMATIONSCIENCES AND TECHNOLOGY

Similarity-based Activation of Hypotheses

• Based on similarity-based matching with cues of “Experience”

• Allows for partial matching• Cues can be associated

with weights• Variable bindings of

hypotheses are established by the matching process.

Experience e1

Cue:C1C3C5

Hypothesize B

10

COLLEGE OF INFORMATIONSCIENCES AND TECHNOLOGY

Closest ExperiencesFor Alternative Hypotheses

RecommendedHypothesis

Current Situation

Similarity-based Matching for Hypothesis Activation

e1

e12

e14

Hypothesis Type D

e10

e5e6

Hypothesis Type C e4

e3

Hypothesis Type A

e7

e8

e9

e2

Hypothesis Type B

X

11

COLLEGE OF INFORMATIONSCIENCES AND TECHNOLOGYHypothesis Activation

Experience C1 C2 C3 C4 C5 Hypothesis

e1 Large - Yes - ? B

e3 - - - - A

e8 - - Violated - C

e14 - - - - D

• Shows the hypothesis that matches the current situation best• Presents option analysis for alternative hypotheses

Matching cues of the recommended hypothesis

Matching cues of alternative hypothesis

Cues not applicable for a hypothesis

Unknown cues relevant for a hypothesis

12

COLLEGE OF INFORMATIONSCIENCES AND TECHNOLOGYOption Analysis for Alternative

Hypotheses

C1 C2 C3 C4 C5 Hypothesis

Large - Yes - ? B

- No - - A

- - Violated - C

- - - - D

• Shows what conditions would have resulted in alternative hypothese

• Blue cells indicate conditions identical to the current situations

• Example:– If C3 did not occur,

the recommended hypothesis would have been A

13

COLLEGE OF INFORMATIONSCIENCES AND TECHNOLOGYOverview

• Cognitive Foundation: RPD Model• RPD-enabled Collaborative Agents: R-

CAST• Hypothesis Reasoning in R-CAST• Similarity-based Activation of

HypothesisGathering Missing Relevant Information• Automated Update/Refine of Hypothesis

14

COLLEGE OF INFORMATIONSCIENCES AND TECHNOLOGY

R-CAST Automates Gathering Relevant Information

Four sources of information for matching with experiences1. Facts in knowledge base2. Inference rules in knowledge base3. External services4. Hypothesis

Experience

C1 C3C5

B

Cues

Hypothesis

Inference Rules

C9 ?C3 ?

InformationManager

RPD DecisionModel

KnowledgeBase

C3C9

CommunicationManager

C9Service

C1

Facts

HypothesisManager

C5?

15

COLLEGE OF INFORMATIONSCIENCES AND TECHNOLOGY

Gather Missing InformationThrough Backward Reasoning and Hypothesis

E

C3

D

F

G

H

Missing Information

Known

Known

Experience

C3

Hypothesize B

Cues

Decision

Missing Information

Information Requirement

Inference Rules

InformationManager

RPD DecisionModel

AgentHypothesize F

Request: E

16

COLLEGE OF INFORMATIONSCIENCES AND TECHNOLOGYSummary

• RPD-based agents enable similarity-based activation of hypotheses– Allow for incomplete information– Enable comparison with alternative hypotheses

• Reasoning about missing relevant information– Through backward inference

• Potential for Cyber Situation Awareness– Using hypothesis reasoning to infer missing information– Using hypothesis reasoning to reduce false positive alerts.

Current Efforts• A novel integration of Bayes Net with predicate logic for

missing information reasoning.• Refinement of hypotheses through reasoning about their

variable bindings.