organic persistent intelligence, surveillance, and ... · rganic persistent intelligence,...
TRANSCRIPT
JOHNS HOPKINS APL TECHNICAL DIGEST, VOLUME 31, NUMBER 2 (2012) 167
rganic Persistent Intelligence, Surveillance, and Reconnais-sance (OPISR) is a visionary, game-changing approach to
intelligence, surveillance, and reconnaissance. OPISR is able to significantly reduce the time required to obtain and distribute rel-
evant intelligence to the frontline war fighter. OPISR is a novel combination of distributed image processing, information management, and control algorithms that enable real-time, autonomous coordination between ad hoc coalitions of autonomous unmanned vehicles, unattended ground sensors, and frontline users. In September 2011, a proto-type OPISR system was demonstrated with more than 12 unmanned vehicles and unattended ground sensors supporting three users.
Organic Persistent Intelligence, Surveillance, and Reconnaissance
David H. Scheidt
warfighters todirectlycoordinateon tacticaldecisionsby using modern communications equipment (duringWorldWarII,thiswasradio).Alliedforceswereforbid-dentouseradiobecause it“wasn’t secure,”andAlliedmaneuverdecisionsweremadebygeneralsatheadquar-tersandbasedonhand-courieredreports.Germandeci-sions were made on-the-fly by Panzer III commandersand Ju87 (Stuka)pilots conversingover the radio.BythetimetheFrenchcommandersmettodecidewhattodoabouttheGermanadvance,RommelandGuderian’sPanzershadtraveledmorethan200milesandreachedtheEnglishChannel.
As demonstrated repeatedly in military history,including the German advance in 1940, the speed at
INTRODUCTIONInthespringof1940,thecombinedFrench,British,
Dutch,andBelgian forcesoutnumbered theirGermancounterparts in troops, mechanized equipment, tanks,fighter planes, and bombers. The Me 109E Germanfighter aircraft was roughly equivalent to the BritishSpitfire,andtheFrenchCharB1tankwassuperiortotheGermanPanzerIII.Inaddition,theAllieswerefight-ing on their home soil, which greatly simplified theirlogistics.Yetinlessthan6weeks,theBelgians,Dutch,andFrenchsurrenderedtotheGermans,andtheBritishretreatedacrosstheEnglishChannel.EventhoughtheAllieshadsuperiorequipmentandlargerforces,theyweredefeatedbytheGermans,whoemployedAuftragstaktik,acommandandcontroltechniquethatenabled“edge”
D. H. SCHEIDT
JOHNS HOPKINS APL TECHNICAL DIGEST, VOLUME 31, NUMBER 2 (2012)168
such as the iRobot Packbot and the AeroVironmentRaven are essential equipment for the modern war-fighter.Theagilityprovidedbyfield-deployablevehiclescomes at a cost—the use of field-deployable unitsincreases logistics and workload demands on frontlineforces.Whencomparedwithlargerunmannedvehicles,fielddeployableunitsofferlimitedsensingandtime-on-target capabilities. Medium-sized vehicles, such as theInsitu ScanEagle and AAI Shadow, offer longer timeon station and more capable payloads. Medium-sizedvehiclesalsodonotmakelogisticsorworkloaddemandsontheedgewarfighter.LargeunmannedvehiclessuchastheGeneralAtomicsPredatorandNorthropGrummanGlobal Hawk offer still more capable payloads andincreasedtimeonstationandalsodonotincreaseedgewarfighterlogisticsorworkload.However,providingtheedge warfighter timely access to intelligence productsproducedbymediumand largevehicles is a challengebecause medium- and large-sized unmanned vehiclesproduce massive amounts of data that are difficult toprocess and disseminate from centralized commandposts.Infact,ArielBleicherreported:
In2009alone,theU.S.AirForceshot24years’worthofvideo over Iraq and Afghanistan using spy drones. Thetroubleis,therearen’tenoughhumaneyestowatchitall.Thedelugeofvideodatafromtheseunmannedaerialvehi-cles,orUAVs,islikelytogetworse.Bynextyear,asinglenewReaperdronewillrecord10videofeedsatonce,andtheAirForceplanstoeventuallyupgradethatnumberto65.JohnRush,chiefoftheIntelligence,SurveillanceandReconnaissanceDivisionoftheU.S.NationalGeospatial-IntelligenceAgency,projectsthatitwouldtakeanunten-able16,000analyststostudythevideofootagefromUAVsandotherairbornesurveillancesystems.”3
Ifintelligence,surveillance,andreconnaissance(ISR)information from medium- and large-scale unmannedvehicles could be processed and distributed in time,thesevehiclescouldpotentiallyprovidesignificantISRcapabilitytotheedgewarfighter.Fortheedgewarfighterto takeadvantageof the ISRcapability representedbythese assets, information relevant to that specific war-fighter must be gleaned from the mass of informationavailableandthenmustbepresentedtothewarfighterina timelymanner.ThispresentsachallengebecausecrewsanalyzingUAVpayloaddata(farfewerthanRush’s16,000analysts)arenotapprisedofthechangingtacticalneedsofallwarfighters,nordothewarfightershavetheaccessorthetimerequiredtoselectandaccessdatafromUAVsources.Currently,operationscentersareusedtogatheranddisseminateinformationfrompersistentISRassets. This centralized information management pro-cess introduces a delay between the observation andtransmissiontothewarfighter,whichreducesforceagil-ityandoperationaleffectiveness.AlthoughU.S.soldiersareempoweredtooperateoncommandbyintent,theirISR systems are all too frequently centralized systemsreminiscentoftheFrenchcommandstructure.ForU.S.
which battlefield decisions are made can be a decid-ingfactorinthebattle.Aprocessmodelthatdescribesmilitary command and control is the observe, orient,decide, act (OODA) loop described by Boyd (Fig. 1).1Boyd shows that, in military engagements, the sidethat can “get inside the opponent’s OODA loop” bymorerapidlycompletingtheOODAcyclehasadistinctadvantage.Intheirinfluentialstudy“PowertotheEdge:Command . . . Control . . . in the Information Age,”AlbertsandHayes2use the termagility todescribeanorganization’sabilitytorapidlyrespondtochangingbat-tlefield conditions. Modern warfare case studies, suchas studies of the Chechens against the Russians, andstudiesofnot-so-modernwarfare, suchasNapoleonatUlm, indicate thatagileorganizationsenjoyadecisivemilitaryadvantage.AlbertsandHayespointoutthatacommon featureof agileorganizations is anempower-mentoffrontlineforces,referredtoasedgewarfighters.Commandersfacilitateorganizationalagilitybyexercis-ing“commandbyintent,”throughwhichcommandersprovideabstractgoalsandobjectivestoedgewarfighterswho thenmake independentdecisionsbasedon thesegoalsandtheirownbattlefieldawareness.Thisempow-ermentofedgewarfightersreducestheOODAloopatthepointofattack,providingthedesiredagility.
A distinguishing characteristic of the conflicts inAfghanistanandIraqistheexplosivegrowthintheuseof unmanned vehicles. Between the first and secondGulf wars, unmanned vehicles transitioned from anoveltyitemtoanindispensablecomponentoftheU.S.military. Field-deployable organic unmanned vehicles
Observe
OrientAct
Decide
Figure 1. Boyd’s OODA cycle models the military decision-making process. Military organizations that perform their OODA cycle more rapidly than opponents gain a substantial competitive advantage.
ORGANIC PERSISTENT INTELLIGENCE, SURVEILLANCE, AND RECONNAISSANCE
JOHNS HOPKINS APL TECHNICAL DIGEST, VOLUME 31, NUMBER 2 (2012) 169
to impacted soldiers in real time.OPISR is capableofrapidly managing large, complex, dynamic situationsbecause it uses a decentralized, ad hoc organizationalstructure.SystemsthatusedecentralizedstructuressuchasOPISRareknowntobemoreeffectiveatthetimelycoordinationofcomplexsystems.Anexplanationofwhydecentralizedcommandandcontrolsystemscanprovidemorerapidresponsethancentralizedsystemsisfoundin“OnOptimizingCommandandControlStructures”4byScheidtandSchultz.OPISRtracksthelocationandISRneedsofallBlueforces,maintainingacontextualaware-nessofthewarfighter’scurrenttacticalneeds.
As relevant tactical informationbecomes available,OPISRpresentsitdirectlytothewarfighterthroughanintuitive handheld device. The information require-mentsthatareusedtodetermineinformationrelevancearedefinedby thewarfighter through the samehand-held interface.This interface supports abstractqueriessuchas (i)patrol these roads; (ii) search this area; (iii)provideimageryofaspecificlocation;(iv)trackalltar-getsofaspecificclassonaspecificrouteorlocation;or(v)alertmewheneverathreatisidentifiedwithinacer-taindistanceofmylocation.Informationthatmatchesthese queries is sent by the system to the handheld
forcestobecomefullyagile,theISRsystemssupportingU.S.soldiersmustbeasagileasthesoldierstheysupport,and APL has developed a system to do just this. Thissystem is Organic Persistent Intelligence, Surveillance,andReconnaissance(OPISR),anISRinfrastructurethatprovidestherapidresponseoforganicISRsystemswiththebreadthandstayingpowerofpersistentISRsystems.
THE OPISR SYSTEMOPISRisasoftwareandcommunicationssubsystem
that,whenaddedtoanISRassetsuchasanunmannedvehicleorunattendedsensor,supportstherapid,autono-mousmovementof informationacrossa tactical force.AsshowninFig.2,warfightersinteractwithOPISRasasystem.WhenusingOPISR,warfightersconnectintotheOPISR“cloud,”taskOPISRwithmission-levelISRneeds, and are subsequently provided with the intelli-gence they need. This capability provides intelligencedirectly to the warfighter without requiring the war-fighter to personally direct, or even know about, theOPISR assets gathering the information. OPISR isautonomous. As a system, OPISR seeks out relevantinformation, pushing key tactical information directly
Figure 2. OPISR’s concept of operation allows UAVs of various sizes to communicate with each other, with users, and with commanders through an ad hoc, asynchronous cloud. The cloud communicates goals from users to vehicles and sensor observations from users to vehicles. Comms, communications.
D. H. SCHEIDT
JOHNS HOPKINS APL TECHNICAL DIGEST, VOLUME 31, NUMBER 2 (2012)170
pilot capable of providing stable flight can be modi-fiedtobecomeautonomousvehiclesbyconnectingtheautopilottotheOPISRprocessor.Whenthevehicleisoperating autonomously, the autopilot sends guidanceandcontrol(GNC)telemetry to theOPISRprocessor.TheprocessorusestheGNCdatatodeviseacontinualstream of waypoints that are sent to the autopilot tofollow.TheOPISRprocessoralsousestheGNCtelem-etry to produce metadata that are associated with thesensordata.ThecombinedsensordataandmetadataarethenusedbytheOPISRsystemasawhole.
In service, unmanned vehicles frequently use sepa-ratecommunicationschannelsforcontrolandimagery.BecauseOPISRdevicesperformbothimageprocessingandcontrolonboardthedevice,thesecommunicationschannels, and the traditionalpilot ground station, arenolongerrequired.Ineffect,OPISRdevicesarecapableofoperatingfullyindependentlyofdirecthumansuper-vision.NotethatOPISRdevicesarestillrespondingtowarfighter requests;however, thesedevicesaccomplishthiswithoutrequiringcontinuouscommunicationswiththe warfighter being serviced. Although OPISR doesnot require traditional control and payload communi-cations, OPISR devices do support these legacy capa-bilities.BecausetheOPISRnodescommunicateoveraseparatechannel,OPISRfunctionalitymaybeprovidedin tandemwith traditionalcontrol.This is inkeepingwiththeOPISRdictumthatOPISRbeanentirelyaddi-tivecapability;unmannedvehicleownerslosenofunc-tionality by adding OPISR. However, OPISR vehiclesareresponsivetocommandsfromhumanoperatorsandwill, atany time,allowanauthorizedhumanoperatortooverrideOPISRprocessordecisions.Likewise,legacyconsumersofinformationwillstillreceivetheiranalogdatastreams.NotethatevenwhentheOPISRprocessorisdeniedcontrolbytheUAVpilot,theOPISRsystemwill continue to share information directly with edgewarfightersasappropriate.
device. The handheld interfaceprovides a map of the surround-ing area that displays real-timetracks and detections and imag-erymetadata.The imagerymeta-data describes, at a glance, theimagery available from the sur-rounding area. OPISR-enabledvehicles are autonomous—if theinformation required by the war-fighter is not available at thetime the query is made, OPISRunmannedvehiclesautonomouslyrelocate so that their sensorscanobtain the required information.OPISR-enabled unmanned vehi-cles support multiple warfighterssimultaneously,withvehiclesself-organizingtodefinejointcoursesofactionthatsatisfytheinformationrequirementsofallwarfighters.
Becausewarfightersarerequiredtooperateinharsh,failure-prone conditions, OPISR was designed to beextremely robust and fault tolerant. OPISR’s designersviewed communications opportunistically, designingthesystemtotakeadvantageofcommunicationschan-nels when available but making sure to avoid any/alldependenciesoncontinuoushigh-quality servicecom-munications.Accordingly,allOPISRdevicesarecapa-bleofoperatingindependentlyasstandalonesystemsorasadhoccoalitionsofdevices.WhenanOPISRdeviceiscapableofcommunicatingwithotherdevices,itwillexchangeinformationthroughnetworkedcommunica-tionsandtherebyimprovetheeffectivenessofthesystemasawhole.However,ifcommunicationsareunavailable,eachdevicewillcontinuetoperformpreviouslyidenti-fiedtasks.Whenmultipledevicesareoperatinginthesamearea,theywillself-organizetoefficientlyperformwhatever taskswarfightershave requested (devices arecapable of coordinating their efforts even when theycannotcommunicateoveranetworkiftheyarecapableofobservingeachotherthroughtheirorganicsensors).Taskprioritizationisbasedonwarfighterauthorityandwarfighter-assignedimportance.
OPISR HardwareOff-the-shelf unmanned vehicles and unattended
sensors can be incorporated into the OPISR systembyaddingtheOPISRpayload.AsshowninFig.3,theOPISRpayloadconsistsofthreehardwarecomponents:an OPISR processor that executes the OPISR soft-ware,anOPISRradiothatprovidescommunicationstootherOPISRnodes includingOPISR’shandheldinter-face devices, and an analog-to-digital converter thatis used to convert payload sensor signals into digitalform.Unmannedvehicles thathaveanonboardauto-
OPISRpayloadAnalog-to-digital
converterPayload sensors
GNC sensors
Actuators
Payload comms
Traditionalground station
Control comms
Autopilot
OPISRground station
Other unmannedvehicle
OPISRprocessor
OPISRmodem
Stockunmannedvehicle
Legacy(not required)
Figure 3. OPISR’s hardware architecture is based on a modular payload that can be fitted onto different types of unmanned vehicles.
ORGANIC PERSISTENT INTELLIGENCE, SURVEILLANCE, AND RECONNAISSANCE
JOHNS HOPKINS APL TECHNICAL DIGEST, VOLUME 31, NUMBER 2 (2012) 171
sensorsonboardtheagent’sdevice,fromotheragents,or from pattern recognition/data fusion software con-tainedwithintheagent.Theintegrityofthedatastoredontheblackboardismaintainedbyatruthmaintenancesystem(TMS).TheTMSperformstwofunctions.First,theTMSresolvesconflictsbetweenbeliefs.Thesimplestform of conflict resolution is accomplished by storingthebeliefwiththemorerecenttimestamp.Forexample,onebeliefmightpositthatthereisatargetatgrid[x,y]attimet0,andasecondbeliefmightpositthatthereisnotargetatgrid[x,y]attimet1.Moresophisticatedcon-flict-resolutionalgorithmsarescheduledtobeintegratedintoOPISRin2012.ThesecondTMSfunctionis theefficient storageof informationwithintheblackboard.Whenperformingthistask,theTMScachesthemostrelevant,timelyinformationforrapidaccessand,whenlong-livedsystemsgeneratemoredatathancanbeman-agedwithin the system, theTMS removes less impor-tantinformationfromtheblackboard.Forcachingandremoval,theimportanceofinformationisdefinedbytheage,proximity,uniqueness,andoperationalrelevance.
Agent Communications ManagerCoordination between agents is asynchronous,
unscheduled, and completely decentralized, as it hasto be, because any centralized arbiter or scheduledcommunications would introduce dependenciesthat reduce the robustness and fault tolerance thatis paramount in the OPISR design. Because agentcommunication is asynchronous and unscheduled,there is no guarantee that any two agents will havematching beliefs at an instance of time. Fortunately,
OPISR SoftwareOPISRisbasedonadistributedmultiagentsoftware
architecture.Eachsoftwareagent servesasaproxy forthedeviceonwhichitislocated,andalldeviceswithinOPISR have their own agents including unmannedvehicles, unattended sensors, and user interfaces. AsshowninFig.4,eachagent iscomposedof fourmajorsoftwarecomponents: apayloadmanager,whichman-ages the sensor information from the device’s organicsensors; a distributed blackboard, which serves as arepository for the shared situational awareness withinthe agent system; an agent communications manager,whichmanagestheflowofinformationbetweenagents;andacSwarmcontroller,whichdeterminesacourseofactionforthosedevicesthatarecapableofautonomousmovement.Alldeviceswithinthesystem,includingthewarfighter’shandhelddevice,arepeerswithinOPISR.
Distributed BlackboardIn the 1980s, Nii5,6 described a method for multi-
agent systems to communicate with each other in anasynchronousmannercalledablackboardsystem.Liketheirnamesakeinthephysicalworld,blackboardsystemsallowagentstopostmessagesforpeeragentconsumptionatanindeterminatetime.EachOPISRagentcontainsapersonalblackboard system thatmaintains amodeloftheagent’senvironment.Threetypesofinformationarestoredoneachagent’sblackboard:beliefs,metadata,andrawdata.RawdataareunprocessedsensordatafromasensorwithintheOPISRsystem.Metadataisinforma-tionthatprovidescontexttoasetofrawdataincludingsensorposition,pose,andtimeofcollection.Beliefsareabstract “facts” about thecurrent situation. Beliefsinclude geospatial arti-facts suchas targets,Blueforce locations, or searchareas. Beliefs can bedeveloped autonomouslyfrom onboard patternrecognition software ordata fusion algorithms orasserted by humans. Mis-sion-level objectives, thegoals that drive OPISR,areaspecialclassofbeliefthatmustbeproducedbyahuman.Thestoringandretrieval of informationto and from agent black-boards is performed bythe blackboard manager.The blackboard man-ager accepts, stores, andretrievesinformationfrom
Agent communicationsmanager Other agents
Network
Metadataobject
Imageryobject
Payload manager
GNC data
Sensor data
BlackboardmanagerBlackboard
Beliefs
Metadata
Raw data
Truth maintenancesystem
Distributed blackboard cSwarm controller
Fielddatabase
Subbehaviors
Traf�c monitor
Interface controlcomponent Autopilot
DCF
Figure 4. OPISR software architecture contains four major elements: the payload manager gener-ates new “beliefs” from sensor observations, the distributed blackboard maintains situational aware-ness by aggregating beliefs from the on- and offboard observations, the agent communications manager exchanges beliefs between other vehicles and users, and the cSwarm controller devises a course of action based on beliefs. DCF, Dynamic co-fields.
D. H. SCHEIDT
JOHNS HOPKINS APL TECHNICAL DIGEST, VOLUME 31, NUMBER 2 (2012)172
• Searchingcontiguousareasdefinedbywarfighters
• Searchinglinearnetworkssuchasroads
• Transitingtoawaypoint
• Blueforceover-watch
• Targettracking
• Perimeterpatrol
• Information exchange infrastructure, in whichunmanned vehicles maneuver to form a networkconnectionbetweenaninformationsource,suchasanunattended sensor, andwarfighters that requireinformationonthesource.Notethatthewarfighteris not required to specify this behavior; the war-fighterneedsonlytospecifytheinformationneed,andthevehicle(s)utilize(s)thisbehaviorasameanstosatisfytheneed.
• Activediagnosis,inwhichvehiclesreduceuncertainor incomplete observations through their organicsensing capabilities. For example, a UAV with asensingcapabilitythatisabletoclassifytargetswillautomaticallymovetoandclassifyunclassifiedtar-getsbeingtrackedbyacooperatingradar.
Inadditiontothemission-levelbehaviorsdescribedabove,OPISRvehiclesexhibitcertainattributeswithinallbehaviors.Theseuniversalattributesare:
• Avoiding obstacles or user-defined out-of-boundsareas
• Responding to direct human commands. OPISRunmannedvehiclesaredesignedtofunctionauton-omously in response to mission-level objectives;however, when operators provide explicit flightinstructions,OPISRvehiclesalwaysrespondtothehuman commands instead of to the autonomouscommands.
OPISR EXPERIMENTATIONThe current OPISR system is the culmination of a
decade-longAPLexplorationofautonomousunmannedvehicles. Key technical elements of the OPISR systemweredemonstratedinhardware-in-the-loopexperimentsas early as 2003. More than 20 hardware demonstra-tions have included the following: DCF,9 the distrib-uted blackboard,10 delay-tolerant communications,11and simultaneous support for multiple end users.12 Assuccessful as theseexperimentshavebeen,APL,priorto 2011, had not integrated the full suite of OPISRcapabilities described in this article on a large, dispa-rate set of vehicles. In September 2011, APL demon-
the control algorithms used by OPISR are robust tobelief inconsistencies. Cross-agent truth maintenanceis designed to the same criteria as agent-to-agentcommunications: Information exchanges betweenagents seek to maximize the consistency of the mostimportant information but do not require absoluteconsistency between agent belief systems. Informationexchange between agents is performed by the agentcommunications manager. When communicationsare established between agents, the respective agentcommunications managers facilitate an exchange ofinformation between their respective blackboards.When limitedbandwidthand/orbrief exchanges limittheamountofinformationexchangedbetweenagents,eachagentcommunicationsmanagerusesaninterfacecontrol component toprioritize the information tobetransmitted. Information is transmitted in priorityorder, with priority being determined by informationclass (beliefs being the most important, followed bymetadata), goal association (e.g., if a warfighter hasrequestedspecificinformation,thenthatinformationisgivenpriority),timeliness,anduniqueness.
Autonomous Control: cSwarmOPISR’s autonomous unmanned vehicles use an
approach called dynamic co-fields (DCF), also knownasstigmergicpotentialfields,togeneratemovementandcontrolactions.DCFisaformofpotentialfieldcontrol.Potential field control techniques generate movementortriggeractionsbyassociatinganartificialfieldfunc-tionwithgeospatialobjects.InOPISR,theobjectsthatareusedtoderivefieldsarebeliefs.Fieldsrepresentsomecombinationofattractionand/orrepulsion.Byevaluatingthefieldsforallknownbeliefsatavehicle’scurrentloca-tion,agradientvectorisproduced.Thisgradientvectoristhenusedtodictateamovementdecision.DevelopedatAPLin2003,7DCFextendsanearlierpotentialfieldapproachcalledco-fields8bymakingthepotentialfieldsdynamicwithrespecttotimeandalsomakingvehiclefieldsself-referential.Self-referentialfieldsarefieldsthatinducevehicledecisionsthataregeneratedbythevehi-cle’s own presence. Adding these dynamic qualities iskeytomanagingtwowell-knownproblemswithpoten-tial field approaches: namely, the tendency of vehiclestobecomestuckinlocalminimaandthepropensitytoexhibitundesiredoscillatorybehavior.AsimplementedinOPISR,DCFisusedtocausespecificbehaviorssuchas search, transit,or track,aswellasbehavioral selec-tion. The DCF algorithm is encoded in the cSwarmsoftwaremodule.AllunmannedvehiclesinOPISRexe-cutecSwarm.DCFbehaviorsspecifictouniqueclassesof vehicle are produced by tailoring the field formula,which is stored in a database within cSwarm. OPISRautonomousunmannedvehiclesarecapableofavarietyofbehaviorsincluding:
ORGANIC PERSISTENT INTELLIGENCE, SURVEILLANCE, AND RECONNAISSANCE
JOHNS HOPKINS APL TECHNICAL DIGEST, VOLUME 31, NUMBER 2 (2012) 173
(Fig.5b),aSegwaygroundvehicle(Fig.6),customAPL-developedsurfacevehicles(Fig.5c),andanOceanServerIver2underseavehicle (Fig.5d).Thesevehiclesusedawide rangeofpayload sensors—including electro-opti-cal,passiveacoustic,side-scansonar,andlidarsensors—todetect, classify, and trackwaterbornevehicles, landvehicles,dismounts,andmine-likeobjects.ISRtaskingwasgeneratedbythreeproxywarfighters,twoofwhichwereonlandandoneofwhichwasonthewater.TherequestedISRtasksrequiredtheuseofallofthevehiclebehaviorspreviouslydescribed.
FUTURE WORKInFY2012,APLisplanningadditionalimprovements
to the OPISR system. Specific OPISR improvementsinclude (i) the integration of more sophisticated datafusiontechniques intothedistributedblackboard,spe-cificallyupstreamdata fusionandclosed-loop ISR, (ii)flighttestingofautonomousbehaviorsthatallowUAVsto form network bridges between remote sensors andusers, (iii) introduction of advanced simulation-basedtestandevaluationtechniques,and(iv)theintegrationofExec-SpecintotheOPISRframeworkandExec-Specflight testing.Exec-Spec isanautonomysystemdevel-opedbyAPLforspacecraftcontrol.IntheOPISR–Exec-Spec integration, Exec-Spec will manage vehicle faultmanagementandsafetyoverridefunctions.
CONCLUSIONThe OPISR system is a framework that provides
a capability through which numerous unmanned
strated10OPISR-enabledvehiclesworkingwith threeunattended ground sensors in support of three users.Thedemonstration employed air, ground, surface, andunderseaISRneedswithsurveillancebeingconductedunder the water, on the water, and on and over land.The autonomous unmanned vehicles included threeBoeing ScanEagles (Fig. 5a), three Procerus Unicorns
Figure 5. OPISR vehicles featuring (a) ScanEagle, (b) Unicorn, (c) surface vehicles, and (d) Iver2 undersea vehicle.
Figure 6. OPISR’s ground vehicles were Segway RMP 200s that were fitted with lidar, electro-optic, and passive acoustic sensors.
D. H. SCHEIDT
JOHNS HOPKINS APL TECHNICAL DIGEST, VOLUME 31, NUMBER 2 (2012)174
3Bleicher,A.,“TheUAVDataGlut,”IEEE Spectrum,http://spectrum.ieee.org/robotics/military-robots/the-uav-data-glut(Oct2010).
4Scheidt,D.,andSchultz,K.,“OnOptimizingCommandandControlStructures,”inProc. 16thInternational Command and Control Research and Technology Symp. (ICCRTS),QuebecCity,Quebec,Canada,pp.1–12(2011).
5Nii,H.,“TheBlackboardModelofProblemSolvingandtheEvolu-tionofBlackboardArchitectures,”AI Magazine7(2),38–53(1986).
6Nii,H.,“BlackboardApplicationSystems,BlackboardSystemsandaKnowledgeEngineeringPerspective,”AI Magazine7(3),82–107(1986).
7Scheidt, D., Stipes, J., and Neighoff, T., “Cooperating UnmannedVehicles,”IEEE International Conf. on Networking, Sensing and Con-trol,Tucson,AZ,pp.326–331(2005).
8Mamei, M., Zambonelli, F., and Leonardi, L.,“Co-Fields: Towards aUnifying Approach to the Engineering of Swarm Intelligent Sys-tems,”inThird International Workshop on Engineering Societies in the Agents World III,Madrid,Spain,pp.68–81(2002).
9Chalmers,R.W.,Scheidt,D.H.,Neighoff,T.M.,Witwicki,S.J.,andBamberger,R.J.,“CooperatingUnmannedVehicles,”Proc. AIAA 3rd “Unmanned Unlimited” Technical Conf.,Chicago,IL,paperno.AIAA2004-6252(2004).
10Hawthorne,R.C.,Neighoff,T.M.,Patrone,D.S.,andScheidt,D.H.,“Dynamic World Modeling in a Swarm of Heterogeneous Autono-mousVehicles,”Proc. Association of Unmanned Vehicle Systems Inter-national,Anaheim,CA(Aug2004).
11Bamberger,R.,Hawthorne,R.,andFarrag,O.,“ACommunicationsArchitecture for a Swarm of Small Unmanned, Autonomous AirVehicles,”inProc. AUVSI’s Unmanned Systems North America Symp.,Anaheim,CA(2004).
12Stipes,J.,Scheidt,D.,andHawthorne,C.,“CooperatingUnmannedVehicles,” in Proc. International Conf. on Robotics and Automation,Rome,Italy(2007).
13U.S. Department of Defense, Quadrennial Defense Review, http://www.defense.gov/qdr/(2010).
platforms simultaneously provide real-time actionableintelligence to tactical units; abstract, manageablesituationalawarenesstotheatercommanders;andhigh-qualityforensicdatatoanalysts.APLhasdemonstratedan OPISR system that includes a distributed, self-localizing camera payload that provides imagery andpositional metadata necessary to stitch informationfrommultiplesources;adistributedcollaborationsystemthatisbasedonrobustadhocwirelesscommunicationsandagent-baseddatamanagement;andauserinterfacethat allows users to receive real-time stitched imageryfromunmannedvehiclesandthatdoesnotrequireuserstodirectly control (or evenexpresslybe awareof) theunmannedvehiclesproducingtheimagery.OPISRisaboldvisionthatpresentsaninnovativeapproachtoISR,an important enabler emphasized in the Quadrennial Defense Review13andotherkeypolicydocuments,andgivestheLaboratoryanenhancedabilitytohelpsponsorsaddressfuturecapabilitygapsinthiscriticalarea.
REFERENCES 1Boyd, J., The Essence of Winning and Losing, http://www.belisarius.
com/modern_business_strategy/boyd/essence/eowl_frameset.htm. 2Alberts,D.,andHayes,R.,“PowertotheEdge:Command...Con-
trol . . . in the Information Age,” Information Age Transformation Series,Washington,DC:CommandandControlResearchProgramPress(2003).
David H. ScheidtisamemberoftheAPLPrincipalProfessionalStaffintheResearchandExploratoryDevelopmentDepartment.Hiscurrentworkconcentratesontheresearchanddevelopmentofdistributedintelligentcontrolsystems.BeforecomingtoAPL,hespent14yearsinindustryperformingandsupervisingthedevelopmentofinformationandcontrolsystemsincludingametadatabasecontainingapproximately1000heterogeneousdatabases,statewideimmunol-ogytracking,locomotivecontrol,railroaddispatching,anddistributedmultilevelsecureinformationsystems.Hise-mailaddressis[email protected].
The Author
TheJohns Hopkins APL Technical Digestcanbeaccessedelectronicallyatwww.jhuapl.edu/techdigest.