Brigade and Below Battlefield Awareness A BORG Integration Project

ARL Knowledge Fusion COE Program

Joseph Lewthwaite

2

Overview

Demonstration of the B3AS, a BORG attack answer tool, implemented as a web service.

Overview of the BORG architecture Overview of the B3AS architecture

3

Scenario: Civil War and Peninsular Conflict

Kim Jong-il dies suddenly.

Over next several months:

Small group of more moderate leaders announce their assumption of control of the central government.

Moderate’s View: Detrimental to survival of the country to continue hardened stance against the West and South.

Moderate’s Intentions: Open a more active dialogue. Status: Unable to gain complete control or vows of loyalty from

all Party members loyal to Kim Jong-il’s legacy, and ominously, from some of the more hard-line elements within the military.

Hardliners oppose moderates. See their positions of relative privilege and power being placed

in jeopardy. Supporters: Include the Navy Chief of Staff, who has gained

the backing of several senior officers in Pyongyang military headquarters, and two corps commanders, one near the DMZ and one north of Pyongyang.

4

Scenario Attack Questions

Company from 2nd Infantry are on the edge of a small hamlet they have been tasked to secure.

Who is defending the hamlet? Are they hostile? How are they deployed? What assets do they possess? What are they doing?

5

B3AS Demonstration

What you will see – Both the operators and warriors point of view.

Web Service implementation Look for:

Frame loss Image quality Video navigation – pan, zoom, rewind,

forward Enhancements through visualizations

6

B3AS

Demo

7

B3AS and the BORG

The BORG program strives to prove that Attack Answer tools can be automated using fusion methodologies and techniques.

The B3AS is an instance of an Attack Answer tool.

8

BORG Architecture

Server Server Server

Workstation

ClientAgent

ServerAgent

ServerAgent

Workstation

ClientAgent

Distributed Computing Base

Collective (Ontologies, Protocols ….)

SecureConfigurationManagementRepository

Pen computer

ClientAgent

SIPRNET

Gateway

Internet

ServerResource

ServerResource

ServerResource

ServerAgent

ServerResource

DataData

ServerAgent

Trusted Code Base

Communal distributed,shared processing power

Platform integrationbetween individual warriorsthrough whole installations

Secure runtimeenvironment over both

wired and wirelessnetworks

AccessControlRoles &

PrivilegesIntelligent, mobile agents

Access to multipleheterogeneous data sets

and sensors

Cell phone

TelephoneMultiple warrior-machine

interfaces

Dynamic answers vs staticconnections

Adaptive, intelligent agents

9

BORG R&D Environment at KIMCOE

B3AS

OO TechnologiesCognitive Deficiencies

(Dr.Scott-Johnson,MSU)(Dr.Bronner, MSU)(Dr.Kattel, MSU)

Visualization, Fusion andMotion Tracking(Dr. Scott, MSU )

Integration of Multimediaand GIS Data

(Dr.George, Clark-Atlanta)

ACT-RCarnegie Mellon

ARL

KLT Motion Tracking(Dr.Birchfield, Clemson)

B3AS Web Service(Mr.Lewthwaite, G.D.)

Autonomous Fusionand NavigationDistinct Sources

Advanced Data Mining

Real-Time Data Streams(Dr.Cowley, PNNL)

Advanced Data Mining(Dr.Russell, MSU)

StarlightPNNLARL

Advanced Interactions

Knowledge Eng,Cognitive Models,

HCI(Dr.Scott-Johnson,MSU)

(Dr.Bronner, MSU)

High Level Language forMilitary Fusion

(Dr.Scherl, Monmouth)

Story (Dr. Subrahmanian,

UMD)

ACT-RCarnegie Mellon

ARL

Interactive Fusion andNavigationSummary of Source

BORG Battlefield and Homeland Security Vignettes

Java, C++, Lisp, .NET, HTML, XML, SQLServer, Oracle

IMPACT (Dr. Subrahmanian, UMD)

NATO Ontology (Dr. Hendler, UMD)

Agent Based Platforms (Dr. Thompson, MSU; Dr. Watkins, MSU)

Tactical Operation Center, Command and Control

B3AS Video Feeder(Mr.Lewthwaite, G.D.)

Architecture, Design, Integration (Mr.Lewthwaite, G.D.)

10

B3AS

Objective - Deliver a real-time fusion stream to answer the warriors’ attack question(s) that is informative, sharp and focused.

What – Heterogeneous distributed database that fuses data continually in real-time using a visual medium as the delivery mechanism.

Why – Provide Army context and focus to the research of the KIMCOE.

11

B3AS Project

Dr Craig ScottMotion TrackingMorgan State

Dr. BirchfieldKLT

Clemson

Dr. ThompsonDr. Watkins

AgentsMorgan State

Dr. SubrahmanianIMPACT

UMD

Dr BronnerDr. Scott-Jonson

HCIMorgan State

Dr. Simons, ChabrisGorillas in our Midst

Harvard

Take

+

+

Dr. GeorgeHeterogeneous DB

Clark-Atlanta

12

B3AS Architecture

Cluster

Local StorageFrame + Metadata

Local StorageFrame + Metadata

Intelligence

UAV Server

Operators’Workstation

Frame Extractor FilterVideo Stream

Frame AnnotatorMeta-data Stream

B3AS ServerAgent

B3AS ClientAgent

Navigation Commands

Analysts’Workstation

Notes...

Video Stream

Highlights

GIS

JCDB

Analysis Highlights

Observations

IndexMetadata

13

Status and Plans

Built the B3AS Video Feeder, a utility used to easily test real-time video/image processing algorithms against a video stream.

Built the Video Streamer, a web service allowing the warrior to interact with a fusion stream from a browser.

Enhance the Video Streamer with collaboration and annotation capabilities.

Develop mechanisms for insertion of custom processing algorithms.

Develop mechanisms for fusing multiple visual streams.

14

Questions???