1

Whitaker– 5 minute madnessGeorgia Tech Research Institute

Elizabeth T. Whitaker, Ph.D.Betty.Whitaker@GTRI.GATech.edu

2

Case-Based Reasoning for Knowledge Discovery

GTRI investigated analytic strategies used in the process of discovering new knowledge, as part of the ARDA/DTO Novel Intelligence from Massive Data (NIMD) program. We designed & prototyped a software tool for intelligence analysts that uses case-based reasoning and case-based planning to plan and execute complex interdependent Internet searches to aid analysts in discovering information relevant to a tasking. Our case-based reasoning approach represents best-practice analytic strategies as domain-specific search plans which are stored in a case library. The prototype matches an analyst’s current problem with the most similar problem in the case library and adapts the associated search plan to solve the current problem.

Future

Present

Implicit Search Strategies

Discovered Knowledge

Discovered Knowledge

Knowledge Discovery

Plans

Case-Based Reasoning for

Knowledge Discovery

Tasking

Analyst

Analyst

Transform

2006

3

Case-Based Reasoning for Knowledge Discovery

Case-Based Reasoning for Knowledge Discovery (CBR for KD) Capabilities Cases• Does the organization possess the technical capabilities?• Does the organization have access to the raw materials?• What manufacturing resources are available?• Who are the experts in this area?• Who have the experts collaborated with and what are their capabilities?• What publications and education exist in this area?

Case Based Reasoning for Knowledge Discovery

Knowledge Discovery Process Models

Analyst Model

Knowledge Discovery

Plans

Discovered Knowledge

Retrieval

Adaptation

KnowledgeRequest

Feedback

Internet

Analyst

Case Library

Plan Execution

2006

4

IRAD: Brain-Based Cognitive Architecture for Training

Immersive Interaction Environment

Dialog Manager

Learning Objective

Student Model

World Model

ScenarioCase Library

Scenario Generation

Scenario Adaptation

Sequence of Events

Text StoryCase Library

Performance Evaluation

fMRI Analysis

Traditional Testing Brain Scans

Interaction History

Domain Expert

Reflection (Self-Improvement)

Student

Brain-Based Model of Learning

Task Decomposition

2011

5

IARPA ICArUS: Integrated Cognitive-Neuroscience Architectures for Understanding Sensemaking

GTRI’s role

• Provide machine learning and case-based reasoning expertise, input and guidance for development of ICArUS requirements and architecture. (Whitaker, Isbell, Hale)

• Define the theoretical basis for models of the Parietal Cortex and Anterior Cingulate Cortex , suitable for forming the bases for development of computational models. (Eric Schumacher)

• Support design and development of ICARUS systems to process challenge problem data

6

DARPA Deep Green Conflict AnalysisSystem Dynamics models for COA analysis in support of a commander’s decision-making

PFG Input• Contains lightly-

processed scenario information

Extract Units and Locations• Paths of each unit

over time• Locations are

averaged across the 75 data rows in each sample

Determine Potential Conflicts• Iterate across all

path segments and find intersections in 2D

• Only uses location data, not time

Create System Dynamics Model• Model will be

graphically difficult to read but structurally accurate

Execute Model Over Various Input Ranges• Potential variables

to range: movement delays, speed, minor changes in latitude/longitude

Determine Actual Entanglements• Compare output

values to input ranges to determine sensitivity

• Show locations with a conflict intensity that exceeds a predetermined threshold

B1

B2

B3

B6

B5

B8

B7

R5

R1

R2 R4

R6

B4

R3

Translation into System DynamicsConceptual Entanglement Discovery

2009

Entanglement could be caused by unplanned delays or changes in speed

7

DARPA Integrated Learning Project

GTRI collaborated with a large team of researchers on the DARPA Integrated Learning project, which had as its goal to research the integration of multiple machine learning paradigms to learn to solve a problem by observing an expert in a single problem-solving session. GTRI, collaborating with the Georgia Tech College of Computing, developed a case-based learner & reasoner to perform as part of the integrated learning activity.

2008

Performance

DataBB

Constraint Learner

Hierarchical Case Learner Case Learner

Human Expert Trace

Constraints Hierarchical Case Base Case Base

Constraints

GoalsBB

Performance Goals (PG)

Hierarchical Retrieval

Adaptation

Retrieved Cases

PG

Adapted Cases

PG

Proposed Solutions

Adaptation Failure

Learning

8

DARPA BLADE: Behavioral Learning for Adaptive Electronic Warfare

• Learning and Reusing Cases for Classifying EW Signals and Generating Countermeasures• Using Case-based learning for to supply support for EW planners by learning cases from observation of the application of countermeasures• Integrating case-based reasoning and planning techniques with other reasoning components and approaches to provide more robust EW

countermeasure synthesis through meta-reasoning• Development of an intelligent similarity metric for active case-learning of EW countermeasures•

Performance

Learning

Adaptation Failure

Orchestrator

Constraint Learner

Protocol Case Learner

Countermeasure Case Learner

Constraints ProtocolCase Base

Countermeasure Case Base

Target Problem

Two-Stage Retrieval

Adaptation

Retrieved Cases

Problem

Countermeasure Solutions

Training Trace

Constraints

Proposed Solutions

Human Feedback

BDA

Content SelectorDomain Expert Module

Student Model

SeriousGame

Environment

Student Modeler

Cognitive Biases Curriculum Model

Learning Content Library

CBR Adaptati

on

Teaching Theory

Learning Theory

Critic Module

Student Log

IARPA SIRIUS Project

Working Memory

Narrative Composition Case

Data, History, Strategies

Draft Narrative

Case Retrieval Algorithm

with Similarity Metric

Narrative Template

Instantiated PlanPlan Instantiationwith Adaptation

Plan Execution System

Agent Agent Agent

Population Characteristics

IntrospectionAgent

Operator

O O

Narrative Composition Plan

Library

CaseLibrary

Desired Influence Subproblems

Narrative Components

Narrative Templates

Draft Narrative

DARPA Narrative Networks --PARABLE Architecture

11

Questions and Discussion