198- advanced decision architectures for the warfighter.pdf

7/27/2019 198- ADVANCED DECISION ARCHITECTURES FOR THE WARFIGHTER.pdf

1/557

E D I T E D B Y PAT RI C I A M C D E RM O T T AN D LAU RE L ALLE N D E R

ADVANCED DECISI ON ARCHITEC TURES

FOR THE WARFI GHTE R:F O U N D AT I O N S A N D T E C H N O L O G Y


2/557

i

A D VA N C E D D E C I S I O N A R C H I T E C T U R E S

F O R T H E WA R F I G H T E R :

F O U N D AT I O N S A N D T E C H N O L O G Y

E D I T E D B Y PAT R I C I A M C D E R M O T T

A N D L A U R E L A L L E N D E R

This book, produced by the Partners of the

Army Research Laboratory Advanced Decision Architectures

Collaborative Technology Alliance, also serves as the

Final Report for DAAD19-01-2-0009.


3/557


4/557

SECTIONI.COLLECTING,

PROCESSING,

AND

DISTRIBUTING

BATTLEFIELD

INFORMATION

SECTIONI

COLLECTING,PROCESSING,ANDDISTRIBUTINGBATTLEFIELD

INFORMATION


5/557

14 SECTIONI


6/557

15

Chapter1

1.

DYNAMICNETWORKANALYSISAPPLIEDTOEXPERIMENTS

FROMTHEDECISIONARCHITECTURESRESEARCHENVIRONMENT

KathleenM.Carley,Ph.D.

CarnegieMellonUniversity,Pittsburgh,PAMichaelK.Martin,Ph.D.

CarnegieMellonUniversity,Pittsburgh,PAJohnP.Hancock

ArtisTech,Inc.,Fairfax,VA

INTRODUCTION

ADACTAresearchisproducingexperimentalresultsfromsimulations

ofintermeshednetworksofwarfightersandbattlefieldsurveillance

assets.Thesenetworksformacomplexsystemwithbehaviorsthat

emergefrompatternsofinteractionamongconstituententities.Thesim

ulatedinteractionsarespatiallysituated,temporallydistributedcommu

nicationsamongpeople,robots,andsoftwareagents.Ingeneralterms,

thecomplexsystempartsofwhichweaddressinthispapercanbe

conceptualizedasatwolevelmetanetworkthatincludesinteractions

amonghumanagentsatonelevel,interactionsamongartificialagentsat

anotherlevel,andcrosslevelinteractionsbetweenhumanandartificial

agents.

TheissueaddressedinthispaperisDynamicNetworkAnalysis(DNA)

ofsystembehavior.Forthepurposesofthischapter,wearenotinter

estedinexaminingtheperformanceofonesystemrelativetoanother.

Althoughthis

type

of

comparative

analysis

can

be

useful

for

researchers

orsystemdesigners,itisofquestionableusetowarfighters.Instead,we

areinterestedinanalysesthatproducetacticallyrelevant,actionable

resultsthathighlightthestrengthsandweaknessesinthesystembeing


7/557

16 CARLEY,MARTIN,ANDHANCOCK

observed.Tobeuseful,theanalyticalresultsmustfostertacticalinsight

andstimulatebattlefielddecisionsthatprudentlyinfluencefuturesys

tembehavior.

Tothisend,wedescribetwocasestudiesthatapplyDNAtothesim

ulatedbattlefield

data

being

generated

by

experiments

in

the

Decision

ArchitecturesResearchEnvironment(DARE).Thefirstcaseinvolvesinter

ceptsofsimulatedcommunicationsamonghumanagents,whichwe

frameasanexerciseinadversarialreasoning.Thesecondcaseinvolves

simulatedcommunicationsamongsurveillanceassets(i.e.,software

agentsandrobots),whichweframeasanexerciseinunderstandingthe

automatedcontrolofaPersistentCoordinatedVideoSurveillance(PCVS)

system.Together,webelievethecasestudiesdemonstratehowDNA

(Carley,2002)

can

foster

tactical

insight

in

complex

multi

entity

scenar

ios.TheyalsodemonstratethatthecombinationofDNAtechniques

requiredfortacticalinsightmayvaryaccordingtothetypeofnetwork

beinganalyzed.Finally,theyshowhowDNAassistsinthedevelopment,

understandingandtuningofsoftwareagentsystems.

ANALYTICALAPPROACHOneapproachtoanalyzingsystembehavioremploysmainstream

statisticaltechniquesonsummarymeasuresofperformance(e.g.,percentoftargetstracked,percentofprioritytargetstrackedwithaccept

ableaccuracy,trackcorrelation,etc.).Thistypeofanalysis,however,

doesnotfullyexploittheinformationgeneratedinDAREexperiments

orbybattlefieldsurveillanceingeneral.Moretothepoint,itprovideslit

tleinsightintotheidentificationofleversinthenetworksunderlyingsys

tembehavior.Onceidentified,leverscanbeusedtodetermineactionsto

taketoinfluencefuturesystembehavior.

DNAprovides

an

alternative

analytical

approach

that

compliments

mainstreamstatistics.WithDNA,thefocusshiftsfromaggregatemea

suresofperformanceforacollectionofbattlefieldentitiestotheperfor

manceimpliedbythestructureofrelationsamongbattlefieldentities.

Thisshiftistheessenceofwhatitmeanstoviewthebattlefieldfroma

networkscienceperspective.Thatis,fromanetworkscienceperspective

wearelessconcernedwithwhatisnormal(e.g.,averages,dispersions)

aboutentitiesinthebattlefield(e.g.,people,places)andmorecon

cernedabout

detecting

substantive

patterns

in

the

observed

relations

amongentities.TheemphasisonthestructureofrelationsinDNAmakes

itparticularlywellsuitedtothedetectionofanomaliesandexceptions

(e.g.,centralities,exclusivities)theleverswithpotentiallylargeinflu


8/557

DynamicNetworkAnalysis 17encesonsystembehavior.DNA,therefore,fostersscrutinyofstrengths

andvulnerabilitiesintherelationsamongbattlefieldentities(i.e.,the

observedsystem).WithaDNAmodel,wecanidentify(amongother

things)implicitgroupsofentities,keypeopleandlocations,andopera

tionallysignificant

time

frames.

We

can

even

begin

to

infer

relations

amongentitieswherenonehavebeenobserved.

Networksciencehasbeenhamperedhistoricallybyadominantly

socialperspectivefocusingonwhointeractswithwhom.However,Carley

(2002)arguedthatthesesocialnetworksexistwithinanecologyofnet

worksthatcanusefullybecharacterizedintermsofthedynamicsofthe

relationsamongthewho,what,where,how,andwhy.Thisisknownas

themetanetworkperspective.*ORA(e.g.,Carley,Columbus,DeReno,

Reminga&

Moon,

2008)

is

adynamic

network

analysis

package

that

can

beusedtoassessmultimode,multiplexnetworks;identifykeyplayers,

groupsandvulnerabilities;enablecomparisonoftwoormorenetworks;

andfacilitatereasoningaboutspatiotemporalnetworks.

*ORAsupportsanalysisofdynamicnetworksinmanyways: (1)com

parisonsoftemporallyorderedsnapshotsofstaticnetworks,(2)statisti

calchangedetectiononsequencesofnetworks,(3)trailanalysisfortrail

dataandconversionoftraildatatonetworks,(4)simulationofchangein

networks,and(5)comparativestaticsforimmediateimpactassessment.

Herein,wemakeuseofstatisticalchangedetectionandtrailanalysis,

alongwith*ORAsvisualizationcapabilities.

CASESTUDY1:TERRORISTSINADELPHIThescenarioforcasestudy1wasframedasinterceptsofsimulated

communicationsamongagentsrepresentingterroristsandnoncomba

tants(e.g.,pizzadeliveryguys).Theagentscommunicatedviaphoneand

emailas

they

moved

about

the

Adelphi

region.

DataGenerationThedataweregeneratedbyArtisTechsAlgoLinksimulator.The

AlgoLinksimulatorwasoriginallydevelopedtotestthecapabilitiesof

messageanalysistoolstosupportintelligenceanalysisrequirementsin

battlefieldcommunications.AlgoLink(seeFigure1.1)facilitatescustom

constructionof

entity

networks

that

follow

specified

communication

structures,times,places,durations,andbehaviors.Itgeneratesboththe

foreground(networkofinterest)andthebackgroundcommunicationas

specifiedandstitchesthemtogetherintoasinglecommunicationrecord.


9/557


Backgroundcommunicationisbothstructured(e.g.intermingledhierar

chicalandsocialorganizations)andrandomasspecifiedbythehuman

simulationoperator.AlgoLinkusesrealisticcommunicationandorganiza

tionaldata,timing,andmorphologiesbutcontainsnoinformationabout

anyreal

person

or

organization.

Figure1.1TheAlgoLinkmessagesimulatorinterfaceThedatasetcreatedforthisexperimentwasgeneratedbyArtisTech

stafflookingforwardtosystembasedIntelligentAgentcommunication

behavioranalysis.Theentitiesofinterestwereorganizedintoasmall

numberofcellsthatwereuniquelyconnectedandstitchedintoa

largerbackgroundcommunity. Thedatasetwasgeographicallycen

teredontheARLAdelphicampusbecausetheARTEMISprojectiscoordi

natedwiththeARLComputationandInformationSciencesDirectorateresearchsystems.Thesimulatedsocialstructurewasmorphologically

similartoamediumsizedcommunityofintelligentagentsactingwitha

specificpurposeinabattlefieldCommand,Control,andCommunications

(C3)system.Tosimulateaneventthatstimulatesthecommunication

network,aspikeincommunicationvolumewasinsertedataselected

time.TheAlgoLinkgenerateddatasetwasarapidwaytoassessthefea

sibilityofcollaborationbetweentheARTEMISPCVSandCASOSteams.


10/557

DynamicNetworkAnalysis 19Analysis

TheAlgoLinkoutputwasdeliveredasanXMLfilecontaininga

sequenceofcommunicationsrecords.Eachsimulatedcommunication

recordidentified

the

sender,

the

receiver,

the

time

the

communication

occurred,itsduration,andwhetheritscontentwasoperationallyrele

vant,irrelevant,orambiguous.Italsocontainedlatitudeandlongitude

forthepositionofthemobilesendersandreceiversduringeachcommu

nication.

Ouranalysisstrategyinthiscasecanbegenerallydescribedasan

overviewandzoom.Thatis,wefirstexaminedthegeneralcontextof

communicationsactivities,andthendrilleddowntodetermineimpor

tantagents,

time

frames,

and

locations.

The

subset

of

*ORA

capabilities

thatprovedparticularlyusefulhereincludedgeospatialvisualization,key

playeridentification,changedetectionanalysis,andthecorrelationof

standardnetworkandgeospatialvisualizations.

Using*ORAsgeospatialvisualizationcapabilities(e.g.,Davis,Olson,

&Carley,2008)weexploitedthepresenceoftimevarying,geolocated

attributesoftheinterceptedcommunicationstodiscoverthescenario

involvedsuspiciousentitiesfleeingtheAdelphiarea(seeFigure1.2).

Figure1.2AgentsfleeingAdelphioverthetimecourseofthescenario

Usingafuzzygroupclusteringtechnique,FOG(e.g.,Davis&Carley,

2008),wefoundthatthesuspiciousentitieswereorganizedintofive

groupswithsharedmembers(seeFigure1.3).Theinterstitialmembers

arelikelytocontainthecoordinatorsandleaders.


11/557


Figure1.3Agentsorganizedin5groupsthatsharemembers

Todrilldown,wefirstused*ORAsKeyEntityReporttoidentifythe

threeagentsmostcriticaltooperations(seeFigure1.4).Becausethe

datawereaboutcommunications,twodifferentcentralitymeasures

wereuseddegreecentralityandbetweennesscentrality.Degreecen

tralitymeasureswhoisconnectedtomostothers(i.e.,theactormost

likelytobeintheknow).Betweennesscentralitymeasureswhois

mostlikelytobeonallthepathsbywhichinformationflows(i.e.,the

actormostlikelytobeinfluential).Thisenablednarrowingourfocustoa

smallgroupofleadersinsteadoffocusingonthesetofinterstitialmembers.


12/557

DynamicNetworkAnalysis 21

Figure1.4KeyEntityReportidentifies3importantagents

Wenext

asked,

is

something

happening?

One

way

of

answering

this

istoseewhetherthereisachangeinstandardbehavior.Usingthe

ChangeDetectionReport(seeFigure1.5),weidentifiedperiod3asthe

timeframeinwhichoperationsmostlikelyoccurred.

Figure1.5ChangeDetectionReportsignalstimeperiod3isdifferent


13/557


Thenetworkchangedetectionanalysis(McCulloh&Carley,2008;

2009;McCulloh,Webb,Graham,Carley&Horn,2008)extendschange

detectionfromoperationsresearch,whereithasbeenusedonvariable

leveldata,

to

relational

data.

This

is

astatistical

approach

for

detecting

smallpersistentchangesinorganizationalbehaviorovertimeusingsta

tisticalprocesscontroltechniquesappliedtonetworksummarystatistics.

Period2wasthepointatwhichorganizationalbehaviorchanged,leading

toradicaldifferencebyperiod3.Thisappearstohavebeenaplanning

executionphaseshift.

Examinationofindividuallevelmetricsforthethreekeyplayersand

networklevelmetrics(e.g.,centrality,betweenness,efficiency,connect

edness)corroborated

our

interpretation

that

period

3was

operationally

significant.Individuallevelmetricsindicatedthatactor286engagedin

extensivecoordinationatperiod2,passingthereignsofcontroltoactor

652atperiod4(seeFigure1.6).

Figure1.6Individuallevelmetricsconvergeonperiod3

Examinationofnetworklevelmetrics(seeFigure1.7)showedthatthe

groupwasgenerallyaverydistributedstructurethatcoordinatedintoa

centrallycontrolled,moreefficient,unitatperiod3.Then,itwentback

toits


14/557


Figure1.7Networklevelmetricsconvergeonperiod3

standardformandbecamemorehiddeninthesociocommunication

environment.

Havingidentifiedthekeyplayersandtimeperiod,wefocusedour

analysesondiscoveringwhatmayhavehappened.Examinationofthe

agentxlocationnetworkforperiod3(seeFigure1.8)indicatedalarge

clusterofsuspiciousentitiesintheAdelphiarea(includingkeyplayer,

Agent286),afairlylargeclusterofsuspiciousentities(includinganother

Figure1.8Agent

xLocation

Network

for

period

3

keyplayer,Agent97)inwhatappearstobeastagingarea,anapparent

waypointbetweenthestagingareaandtheclusterofsuspiciousentities


15/557


inAdelphi(alsovisitedbyAgent97,whoseemstobealiaison),anda

runner(keyplayer,Agent652)whovisitsmanylocationswithfewsuspi

ciousentitiespresent.

Anotherkeyadvance,developedaspartoftheCTA,wasthecapabil

ityto

move

between

trail

data

who

was

where

when

and

networks.

Whenweexaminedthetrailsvisualizationforperiod4(i.e.,theperiod

immediatelyfollowingtheapparentoperation,perhapsaperiodofinitial

surveillance),wesawthatthethreekeyplayerswereneverinthesame

placeatthetime;Agent652wasagainrunning,whereastheactivitiesof

Agents97and286wererestrictedtooneortwoareas.Theapparent

coordinationhandofffromAgent286toAgent652isrelatedto652s

increaseinspatialmovementandcoordinationneededduetoincreased

movement.In

the

trails

visualization

(see

Figure

1.9),

time

progresses

downtheyaxis.Geographicregionsformverticalbinsalongthexaxis.

Arrowsareplottedasagentsmovefromregiontoregion(orwithin

regions),andinthiscasearecolorcodedtothethreekeyplayers.

Figure1.9*ORALoomvisualizationoftrailsfor3keyplayersduringperiod4

Finally,correlatingastandardagentxlocationnetworkvisualization

withageospatialvisualizationfortheendofthescenariowefoundthat

Agent286wasalonewithasinglemovementbetweentwolocationsin

Adelphi,Agent97washoledupwithasizablegroupofsuspiciousenti

tiesnorth

of

Adelphi,

and

Agent

652

was

alone

but

on

the

run

(see

Figure

1.10).


16/557


Figure1.10Correlationofagentxlocationnetworkandgeographiclocationatsce

narioend

Giventhepatternofcommunicationsandmovementduringthesce

nario,twocoursesofactionsappearreasonable: (1)scourAdelphifora

bomb,IED,etc.plantedduringoperationsinperiod3,or(2)goafterdis

persedsuspiciousentities.Withrespecttoaction2,Agent286maybean

easytarget

with

direct

knowledge

of

the

operations

that

occurred

during

period3.TargetingthelocationwhereAgent97ishiding,however,will

yieldmoresuspiciousentities.Notethatthesimulateddatadoesnot

includecommunicationscontentsospecificationoftheeventisnotpos

sible,buttheappliedDNAanalysisaccuratelyidentifiedthetime,place,

andleadentitiesinthesimulation.

CASESTUDY2:MOVEMENTINTHEPERIMETERThescenarioforthiscasecenteredonanautomatedsurveillance

system(i.e.,theARTEMISPCVSsystem)thatisresponsibleforidentifying

movingentitieswithintheperimeterofaBlueForce researchcom

pound.Thesystemdividestheperimeterintofourareasofresponsibility,

whereeachareaisassignedtoaTaskingAgentthatisresponsiblefor

surveillance(seeFigure1.11).TaskingAgentsgetsimulatedmovement

reportsfromsimplevideoanalysisalgorithms.TaskingAgentsthenprior

itize

targets

and

assign

mobile

robotic

assets

to

pursue

and

identify

the

targets.


17/557


Figure1.11Tasking

Agent

Areas

DataGenerationThedataweregeneratedbyArtisTechsARTEMISPCVSsystemproto

type(seeFigure1.12).Thisprototypeuseshundredsofsmallreasoning

Figure1.123DViewofARTEMISPCVSscenarioontheGISARLAdelphitestbed

model


18/557

DynamicNetworkAnalysis 27algorithmsencapsulatedinsoftwareagents.Theagentscommunicate

withhumans,agents,andothersystemelementsandevencreateand

deleteotheragentswithfrequenciesthataredependentontheirreac

tionstothesensedenvironment(simulation).

Theparticular

experimental

run

analyzed

here

was

conducted

to

determinewhetherTaskingAgentreasoningandcommunicationabout

sharingmobileroboticassetswasfunctioningasexpected.Thescenario

includedashorttimeofquiescenceinthecompound,followedbythe

injectionofarelativelylargenumberofmovingtargetsthatmoved

aboutthecompoundusingreasonablepathsbutfreelycrossingareasof

surveillanceresponsibility.Whentargetscrossareasofresponsibilitythe

complexityofreasoningincreasesandrequiresthatTaskingAgentstrans

fer(handoff)

tracking

and

even

possibly

lend

robotic

assets

to

other

TaskingAgents.Theactoflendingtheassetinvolvescommunicationto

notifyanadjacentTaskingAgentofanincomingunidentifiedentity,

andamultimessageTaskerGlobalTaskerhandshaketotransfercon

trol.Thereweretwoofthesehandoffeventsinthescenario.Thissce

nariowasasimpleonetofacilitateearlycollaborationbetweenARTEMIS

andCASOSstaff.

Analysis

Aswithcasestudy1,thedataweredeliveredasanXMLfilecom

prisedofasequenceofcommunications.Thecommunicationswere

betweensoftwareagentsorsoftwareagentsandrobots.

Althoughthedatasetsweresuperficiallysimilar(i.e.,logsofcommu

nicationsrecords),casestudy2presentedseveralchallengesnotpresent

incasestudy1.Thesechallengesaroseprimarilybecausethedatafor

thiscasewererelativelyimpoverished.DNArequireslargedatasetsand

wedid

receive

more

data

for

case

study

2than

for

case

study

1.

However,

theextradatawereoflittlebenefitbecausetheyprovidedlittleinforma

tionregardingthestructureofthesystemwewereanalyzing.Intermsof

structuralinformation,theextradataweremostlyredundant.

Theanalysiswascomplicatedbythefactthatthescenarioincluded

onlytwoinstancesofthetargethandoffeventthesignalwewereto

detect.Instructuralterms,thismeansthatwewerelookingforachange

innetworkstructurethatinvolved(perArtisTechsdescriptionofthe

handshake)three

links

at

most.

Thus,

the

statistical

change

detection

analysisusedincasestudy1wasofnouse.Suchasmallchangeinstruc

turewouldnotbedetectedasbeingstatisticallysignificant.


19/557


Theanalysiswasfurthercomplicatedbytheabsenceofgeocoordi

natesforthemobilerobots.Therefore,*ORAsgeospatialvisualization

capabilitiescouldnotbeemployed.Theinsightthatcanbegleanedfrom

visualizationsofagentsxlocations,asusedincasestudy1,wasalsolost.

Withoutlocation,

we

also

lacked

any

means

for

constructing

trails

data

toexaminewhowaswherewhen.

Finally,thegoalofanalysisincasestudy2differedfromcasestudy1.

Incasestudy1,weemployedDNAtechniquesdesignedtoidentify

importantentities,locations,andtimes.Thepurposeoftheanalysis,

therefore,wastoidentifycentralities.Incontrast,thepurposeofthe

analysisforcasestudy2wastoidentifyexclusivities(i.e.,thetwoTasker

GlobalTaskerhandshakes).

Giventhe

impoverished

data

set

and

our

goal

of

detecting

only

two

instancesofthehandoffamongasmallsetofagentsandrobots,the

analysisstrategyweadoptedforthiscasewasoneofconvergingopera

tions(tokickstartaDNAofricherdatainthefuture).ArtisTechper

sonnel(withtheirknowledgeofthesystem)manuallyanalyzedthedata.

CASOSpersonnelappliedDNAtechniquestothedata.Thefollowingdis

cussesonlytwooftheissuesweaddressed.

TofindthehandoffwhereoneTaskingAgentloanedarobotto

anotherTaskingAgent,wereliedonArtisTechsidentificationofmes

sagetypesthatindicatesuchahandoffoccurred.Threemessagetypes

wererelatedtothehandoff:RemoveIdentity,ReportPositionToSelected

Tasker,andReportPositionToSelectedTaskerReturn.Findingthehandoff

wasthensimplyamatterofusing*ORAsSphereofInfluencecapabilities

tovisualizewhichagentswereinvolved.Figure1.13showsthethree

importantmessages,alongwiththeagentsthatsentandreceivedthem.

ItcanbeseenthatTaskingAgent3ispositionedinthenetworkdiffer

entlythanTaskingAgents1,2,and4.Furthermore,weseethatTasking

Agent3was

the

only

agent

to

send

the

ReportPositionToSelectedTasker

Returnmessage(asindicatedbythearrowonthelinkbetweentheagent

nodeandthemessagenode).GlobalTaskingAgent7receivedthismes

sageandsentaRemoveIdentitymessagetoTaskingAgent3.Thesphere

ofinfluencealsoindicatesthattheReportPositionToSelectedTaskermes

sagewasprobablynotuniquelyrelatedtothehandshake.


20/557


Figure1.13SphereofInfluencevisualizationshowingrobothandoff

Inanticipationofricherdatasets,thesecondissueweexaminedwas

whetherwecoulduseDNAtechniquestopartitionagentsintofore

groundandbackgroundagents.Tothisend,theArtisTechteamproduced

ameticulous

message

trace

analysis

and

event

identification

as

ground

truthfortheCASOSteam.Evenasfullyknowledgeabledesignersofthe

communicationlogicthisanalysisanddocumentationtookmorethan4

hoursusingsimpletextsearchandanumericmessagefrequencyanalysis

providedbyCASOS.Thecomplexityofthisanalysisunderscorestheneed

fornetworkanalysistechniquesforamorecompletesystemexperiment

analysis.

PerArtisTechsanalysis,agentscouldbedividedintoforeground

agentsthatweresubstantivelyrelatedtothescenarioandbackground

agentsthatexistedsimplytomakethesimulationrun.Ourtaskat

CASOS,therefore,wastoemployoneoranotherDNAtechniquetosepa

rateforegroundfrombackgroundagents.WefoundthattheNewman

groupingalgorithmworkedwell,separatingforegroundandbackground

agentsintogroupsthatcloselyapproximatedthemanualanalysis.Seven

oftenbackgroundagentswerecorrectlyclassified.Buttherewasdis

agreementbetweenArtisTechjudgesregardingwhetheroneofthethree

misclassifiedagentswasabackgroundoraforegroundagent.Onefore

groundagent

out

of

29

undisputed

foreground

agents

was

misclassified.

ThattheNewmangroupingalgorithmcorrespondsfairlywellwiththe

judgmentsofdomainexpertsispromising,andsuggestsalineof

researchconcerningthepsychologicalvalidityofNewmangrouping.


21/557


SYSTEMUNDERSTANDINGANDTUNINGACTIVITIESArtisTechpostulatedtheadditionalbenefitofperformingDNAon

theoutputlogsfromtheinternalsystemcommunications;theanalysis

supportedunderstanding

of

how

the

team

and

system

achieved

the

mea

suredperformance.TheARTEMISresearchteamwasintheprocessof

settingupsystemmodelexperimentstostudytheadvantagesthatauto

matedreasoningcouldaddtowidelyusednumericimageandvideopro

cessingforthepurposeofPCVS.Aswemodeledthereasoningandsetit

intothesimulationwehaveconductedmanytestruns.Wecaneasilysee

whenthemacrosystembehaviorisasdesignedandwhenitdeviates

fromintentions.However,todeterminewhythedeviationsoccurand

howthe

reactive

agent

networks

achieve

intended

system

performance

requiresmessagelevelanalysis.ArtisTechsharedmixedresulttestlogs

withCASOSspecificallytofacilitatetheidentificationofexpectedand

unexpectedresults,andhowtheyarise.

Resultsfrominitialanalysesinbothcasestudiesdivergedfrom

expectedresults.Inhindsightthisishardlysurprising.Designofacom

plex,multiagentsystemwithemergentreactivesystembehaviormay

notbepossiblewithoutthesupportofnetworkanalytics.

Inbothcaseswefoundevidenceofpragmatic,technicallycorrect,

programmingpracticesthatinterferedwithsubstantiveDNAofthecommunicationslogs.ArtisTechconfirmedthattheexperimentrunsthat

wereanalyzedwerenotconsideredfinalorexpectedtobeentirelycor

rect,merelytypicalincontentandform.Whetherweviewthisasaveri

ficationissue(i.e.,buildingthethingright)oravalidationissue(i.e.,

buildingtherightthing)dependsonperspective.Giventhatverification

isprimarilyaninternalactivity;thedegreetowhichprogrammers

achievedtheintentbehindspecificationsisamatterofinterpretation.

Fromthe

perspective

of

network

analysis

(i.e.,

users

of

the

output

logs),

however,wecannoteaminorshortfallinvalidation.Specifically,com

municationrecordsthatarenotrelevanttothescenariobeinganalyzed

shouldbefilteredfromthelogtoincreaseusability.

Intheinitialanalysis,weexperimentallyillustratedthatDNA

approachesprovideacapabilitytoanalyzecomplexmessagesetstofind

particularbehaviorpatternsofinterest.Inthesecondanalysiswebegan

forgingacollaborativeresearchapproachdesignedtounearththeana

lyticsteps

the

combinations

of

DNA

techniques

required

to

analyze

differenttypesofmessagingbehavior.


22/557

DynamicNetworkAnalysis 31LESSONSLEARNEDANDFUTUREDIRECTIONS

Opencollaborationbetweendataprovidersandnetworkanalysts

createdabeneficialgapbetweenexpectedandobservedsystembehav

ior.As

data

provider,

ArtisTech

developed

the

multi

agent

system

and

environmentsimulations,designedthescenarios,andconductedthe

simulationbasedexperimentswithexpectedsystembehaviorsinmind.

Thenonlinearitiesinherenttocomplexsystemscomprisedofinteracting

agents,however,makeitnotoriouslydifficulttopredictemergentand

reactivebehavior,andareindeedthereasoncomputersimulationisnec

essary.Asnetworkanalyst,CASOSreceivedmilitarilyrelevantbattlefield

simulationdatawithoutpriorknowledgeofexpectedsystembehavior.

Thechallenge

to

CASOS,

therefore,

was

to

use

DNA

to

characterize

what

happenedinthemysteriousscenariosreceivedandCASOSobserva

tionsinitiallydivergedfromArtisTechdefaultexpectations.Toresolvethe

discrepancy,weusedDNAtoexaminewhythesimulationsdidnot

behaveasexpected.Thus,thepostulatedbenefitoftheArtisTechCASOS

collaborationwastheuseofnetworkanalyticstogaininsightintothe

performanceofmultiagentsystems.ArtisTech,withCASOS,nowplans

tousethisapproachandDNAtoolstobuildasetofreusablesystem

experimentanalysismethodsthatwillbeappliedtounderstandhow

variantHuman/AutomationPCVSexperimentsachieveobservedresultsusingnetworksofcommunicationandbehaviors.

WithregardtodemonstratingthetacticalrelevanceofDNA,the

availabilityofmilitaryscenariodataisinvaluable.Itprovidesopportuni

tiestocombineextantDNAtechniquesintoanalyticstrategiesthatpro

duceresultswarfighterscanuse.Generally,wefoundthatthefamilyof

DNAmetricsandvisualizationsthathavebeenimplementedin*ORA

overthepast10orsoyearsprovidesanamplebasisforconductingtacti

callyrelevant,

actionable

analyses

on

battlefield

data.

Two*ORAcapabilitieswereparticularlyhelpfulduringthiseffort:

geovisualizationandnetworkchangedetection.The(notsosimple)act

ofplacingnetworksonamapputsabstractsocialnetworksintoacon

crete,spatialcontext.Itprovidesexplicitinformationaboutwherethe

actionistakingplace.Thecapacitytodetectstructuralchangesamong

temporallyorderednetworksprovidesexplicitinformationaboutwhen

theactionistakingplace.

Thecase

studies

also

helped

to

identify

several

areas

where

future

developmentcouldimprove*ORAstacticalrelevance.Theimprove

mentswouldgenerallysupportdeliveryofoneormoreTacticalInsight

Reports.Asenvisioned,thesereportswouldcontaintheoutputsofall


23/557


DNAtechniquesthatcontributetoaparticularanalyticstrategy.From

thetwocasestudiesdescribedabove,itappearsthatcorrespondences

amongnetworksoftherelationsamongentities,networksofgeospa

tiallyanchoredentities,andnetworksdistributedovertimewillplaya

centralrole

in

such

reports.

ForArtisTechthegoalofDNAapplicationstosystemanalysisand

monitoringistoinvestigatethecreationofageneralizeddatastructure

andDNAmethodcombinationapproachthatwillallowtheencodingof

expectedordetectedmessageandbehaviorpatterns.Theinitialapplica

tionofthisconcernspostexperimentreview.Themorefarreaching

implicationsofsuchDNAmechanismsextendtorealtimeembedded

systemmonitoringtoincreasedistributedsystemsecurityandusertrust.

ACKNOWLEDGEMENTS

ThisworkispartoftheDynamicsNetworksprojectatthecenterfor

ComputationalAnalysisofSocialandOrganizationalSystems(CASOS)of

theSchoolofComputerScience(SCS)atCarnegieMellonUniversity

(CMU).For*ORA,forthepartsusedinthisanalysis,thegeospatialcom

ponentsweresupportedbyAROandERDCTECW911NF0710317,the

*ORALoombyONRN000140610104,theovertimechangedetection

basicanalysisbyARIW91WAW07C0063andtheORAimplementationofchangedetectionbyARLDAAD190120009,thefourieranalysisbyONR

N000140610104,thekeyentityanalysiswassupportedbyONR

N000140610104,extensionstohandlecommunicationsnetworksby

ARLthroughthe CommunicationsandNetworks(CN)Collaborative

TechnologyAlliance(CTA)20002504andtheimmediateimpactassess

mentbyONRN000140610104.Thebasicresearchconductedhereto

explorehowtousethesetechnologiesforadversarialbehaviorandto

linkto

the

ArtisTech

ARTEMIS

PCVS

system,

AlgoLink

simulator,

DARE

wassupportedbytheArmyResearchLabthroughtheAdvancedDecision

Architecture(ADA)CollaborativeTechnologyAlliance(CTA) DAAD1901

20009.Theviewsandproposalscontainedinthisdocumentarethoseof

theauthorandshouldnotbeinterpretedasrepresentingtheofficialpol

icies,eitherexpressedorimplied,oftheOfficeofNavalResearch,the

ArmyResearchOffice,theArmyResearchInstitute,theArmyResearch

LabortheU.S.government.


24/557

DynamicNetworkAnalysis 33REFERENCES

Carley,K.M.(2002).Smartagentsandorganizationsofthefuture.InL.Lievrouw&

S.Livingstone(Eds.),Thehandbookofnewmedia(pp.206220).ThousandOaks,CA:Sage.Carley,K.M.,Columbus,D.,DeReno,M.,Reminga,J.,&Moon,I.C.(2008).ORAUsersGuide2008(Tech.Rep.No.CMUISR08125).Pittsburgh,PA:InstituteforSoftwareResearch,SchoolofComputerScience,CarnegieMellonUniversity.

Davis,G.B.,&Carley,K.M.(2008).ClearingtheFOG:Fuzzy,overlappinggroupsfor

socialnetworks.SocialNetworks,30(3),201 212.Davis,G.B.,Olson,J.,&Carley,K.M.(2008).OraGISandloom:Spatialandtemporal

extensionstotheORAanalysisplatform(Tech.Rep.No.CMUISR08121).Pittsburgh,PA:InstituteforSoftwareResearch,SchoolofComputerScience,CarnegieMellon

University.

McCulloh,I.,

&

Carley,

K.M.

(2008).

Detectingchangeinhumansocialbehaviorsimulation(Tech.Rep.No.CMUISR08135).Pittsburgh,PA:InstituteforSoftware

Research,SchoolofComputerScience,CarnegieMellonUniversity.


25/557



26/557


27/557

E D I T E D B Y PAT RI C I A M C D E RM O T T AN D LAU RE L ALLE N D E R

ADVANCED DECISI ON ARCHITEC TURES

FOR THE WARFI GHTE R:F O U N D AT I O N S A N D T E C H N O L O G Y


28/557

i

A D VA N C E D D E C I S I O N A R C H I T E C T U R E S

F O R T H E WA R F I G H T E R :

F O U N D AT I O N S A N D T E C H N O L O G Y

E D I T E D B Y PAT R I C I A M C D E R M O T T

A N D L A U R E L A L L E N D E R

This book, produced by the Partners of the

Army Research Laboratory Advanced Decision Architectures

Collaborative Technology Alliance, also serves as the

Final Report for DAAD19-01-2-0009.


29/557


30/557

SECTIONI.COLLECTING,

PROCESSING,

AND

DISTRIBUTING

BATTLEFIELD

INFORMATION

SECTIONI

COLLECTING,PROCESSING,ANDDISTRIBUTINGBATTLEFIELD

INFORMATION


31/557

14 SECTIONI


32/557

35

Chapter2

2.

FROMBACKGROUNDSUBTRACTIONTOTHREATDETECTIONIN

AUTOMATEDVIDEOSURVEILLANCE

JoshuaEckroth

TheOhio

State

University,

Columbus,

OH

DikpalReddy

UniversityofMaryland,UMIACS,CollegePark,MD

JohnR.Josephson,Ph.D.

TheOhioStateUniversity,Columbus,OH

RamaChellappa,

Ph.D.

UniversityofMaryland,UMIACS,CollegePark,MD

TimothyN.Miller

TheOhioStateUniversity,Columbus,OH

INTRODUCTION:PERSISTENTVIDEOSURVEILLANCEAsvideocamerasandothersensorsbecomecheaperandeasierto

network,sensornetworksbecomeincreasinglyattractivemeansto

acquireusefulinformationformilitaryoperations,suchasvideosurveil

lanceforfacilitiesprotection,andpersistentsurveillancetomaintain

sensorycontactwithtargetsofinterest(Pendal,2005).However,without

assistancefromautomation,humanswillbeoverloadedbyinformation,

andunabletouseiteffectively.

Whatinformationconsumesisratherobvious:itconsumesthe

attentionof

its

recipients.

Hence

awealth

of

information

creates

apov

ertyofattention,andaneedtoallocatethatattentionefficientlyamong

theoverabundanceofinformationsourcesthatmightconsumeit.

(Simon,1971)


33/557

36 ECKROTHETAL.

Itisdifficultforhumanstovigilantlymonitoralargenumberofvideo

feedsforextendedperiodswithoutfatigue,orcomplacency,especiallyif

theyareaskedtorecognizethesignificanceofrareeventsinacomplex

streamofevents(Warm,et.al.,1996;Grier,et.al.2003;Pattyn,et.al.,

2008).Moreover,

sometimes

events

of

interest

cannot

be

recognized

simplyfromthevideowithoutknowingwhereaspecificcameraispoint

ing,atarestrictedarea,forexample.Keepingtrackofthelocationand

significanceofacamerasfieldofviewimposesanadditionalcognitive

burden.Thecognitiveburdenisevengreaterifrecognizingeventsof

interestrequirestrackingentitiesastheybecomevisibleindifferentcam

eras,andaccessingmentalmapstounderstandthesignificanceof

motionpaths.

Automationcan

help.

It

can

potentially

provide

users

with

alerts

basedonrecognizingindicatorsofthreateningbehavior,includingbehav

iorthatcannotbedetectedwithasinglecamera.Forexample,themove

mentofanentityfromplacetoplaceinrelationtothemapmightshowa

patternindicativeofscoutingtheperimeterofafacility,wherenosingle

cameraviewshowsanythingsuspicious.Manyunsolvedtechnicalprob

lemsremain,however,includingproblemsabouthowtoextractneeded

informationfromvideoimagery,andproblemsofhowtoprocess

acquiredinformationtoclimbthelevelsofabstractionfromindividual

cameracenteredframeworkstoaworldcenteredframework,andfrom

motiondescription(ObjectO5604movedalongpathP52fromlocation

L12attimeT55toL13atT66.)tobehaviordescription(O5604proceeded

slowlynorthonroadR3 totheintersectionwithR7,turnedEastonR7.)torecognitionofindicatorsofthreat(O5604slowlycircledthefacil

ity.).

Abundantvideoalsothreatenstooverloadcommunicationsand

storagesystems.

Typically,

the

sensed

bits

are

highly

redundant

for

the

tasktobeperformed.Forexample,whenitisimportanttotrackamov

ingobject,theimportantinformationiswhichpartoftheimageischang

ingcontinuously.Tounderstandwhichregionisrelevant,subtractionof

theunchangingbackgroundisneeded,andifitcanbeperformedatthe

sensoritself,significantlyfewerbitsneedtobetransmitted.Accordingly,

anewframeworkcanbeenvisagedinwhichthetraditionalwaysofsens

inganimage(inrectangularpixels)arereplacedbysensingtheimage

information

in

a

compact

form

(compressed

sensing),

and

then

recon

structingtherequiredimage,asneeded,fromthiscompactinformation.

Thispaperdescribessomeelementsofrecentprogressintechnology

forvideosurveillancethatspecificallyaddressmethodsforbackground

subtractiontodetectchangesfromframetoframeinvideo,trackingof


34/557

AutomatedVideoSurveillance 37

viewedobjectsincameracenteredimagespace,trackingofobjectsin

worldorientedobjectspaceusinginformationfrommultiplecameras,

andmethodsforclimbinglevelsofabstractionindescriptionsof

behavior.

BACKGROUNDSUBTRACTIONANDTRACKINGUSINGCOMPRESSIVECAMERAS

Introduction

Mostcamerasinstalledforsurveillanceapplicationsareusedforsim

plevisiontaskssuchasdetectionandtracking,poseestimation,and3D

reconstruction

from

silhouettes,

and

very

few

for

higher

level

tasks

such

asactivityrecognition.Inmanyofthesetasks,theamountofdatacol

lectedishugeforthepurposeoftheapplication.Forexample,atypical

backgroundsubtractionalgorithmcomparesasingleframe,fullresolu

tionimageobtainedfromacameratoanotherfullresolutionimagethat

hasbeenlabeledthebackground.Pixelsthatdifferbetweenthetwo

imagesareconsideredforeground.Theresultisblobsinthefore

groundthatrepresentobjectsthatarebelievedtohavemovedbetween

thetimethebackgroundimagewasacquiredandthetimethefore

groundimage

was

acquired

(usually,

the

background

image

is

updated

everytimeanewframeisobtainedfromthecamera,sothatmovements

fromoneframetothenextaredetectedandplacedintheforeground).

Theresultingforegroundimageisoftenrepresentedasablackandwhite

fullresolutionimage,whereblackrepresentsbackgroundandwhiterep

resentsforeground;iftheseimagesareobtainedfrom,say,parkinglot

surveillance,thentheforegroundblobswilllikelyresemblesilhouettes

ofmovingcarsandwalkingpeople.Yet,theseblobsoccupyonlysmall

regionsof

the

image,

since,

in

most

surveillance

video,

there

are

few

movingobjectsatanyonetime.Similarly,whentracking,weareinter

estedinonlyasmallregionoftheimagethatismoving,anddonotcare

abouttheotherpartsoftheimage.Neverthelesstheentireimageis

commonlysensedandtransmittedtoacentrallocationforprocessing.

Thismeansthatahugeamountofdata,whichisultimatelyuseless,is

collectedandtransmitted,thuswastingsensingandbandwidth

resources.Inthispaper,wepresentanapproachtoalleviatingthisprob

lem.We

show

our

approach

on

two

vision

applications:

background

sub

tractionandtracking.Inbothoftheseapplications,weusea

compressivecameratoobservetheimagesandthenprocessingis

doneonthesemeasurements.


35/557

38 ECKROTHETAL.

Acompressivecameraisadevicethathasbeenbuiltontheprinci

plesofcompressedsensing(Candes,2006).Suchacamerameasures,not

theimagepixelslikeaconventionalcamera,butasmallnumberofran

domlinearprojectionsoftheimage.Thenumberofrequiredmeasure

mentsis

significantly

fewer

than

the

number

of

pixels

in

aconventional

camera,andthisinherentlyreducesthespaceneededtostorethe

image.Thefullimage(intheformofpixels)canbereconstructedfrom

theselinearcombinations.

Wepresentaninnovativeapproachwhereinvisiontasksareper

formed,notonthefullimages,butonthecompressedmeasurements

directly,withouttheneedtoreconstructthefullimages.Becausethefull

imageisnotneededforthetasks,asignificantreductionisachievedin

theamount

of

data

that

needs

to

be

sensed.

In

the

next

section,

we

pro

videabriefintroductiontocompressedsensingtheoryandcompressive

cameras.Wethenpresentourapproachtobackgroundsubtractionand

trackingoncompressedmeasurements.

CompressiveCamerasandComputerVisionApplicationsCompressedSensing.Supposewehaveanimage ofsize

(i.e.,vectorized),

then

we

can

represent

the

image

in

some

basis

as

(1)

where isasparsecoefficientvector( sparse).Thevector has

veryfewlargenonzerocomponentsindicatingthattheimagecanbe

compressed.Waveletsareanexampleofsuchabasis.

IntheCSframework(Candes,2006)wedonotmeasurethe larg

estelementsof ,butweinsteadmeasure linearprojectionsof

theimage ontoanotherbasis .

(2)

where arethecompressedmeasurements.Since ,the

systemofequationsareunderdetermined,bututilizingthesparsityofwecanrecoverthesignalbysolvingthefollowing optimization

problemcalledBasisPursuit.

(3)

Compressivecameras.Basedoncompressivesensing(CS)theory,a

prototypesinglepixelcamera(SPC)wasproposedin(Wakin,2006).The

x 1N

x =

M N

198- advanced decision architectures for the warfighter.pdf

Documents