1

Wright State UniversityBiomedical, Industrial & Human Factors Eng.

Bay of Biscay, Agent Modeling Study

Raymond HillResearch sponsored by:

2

Purpose

Update project with DMSO/AFRL presented at last year’s conference AFIT Operational Sciences Department WSU BIE Department

Two pieces of work accomplished to date that I will discuss today

Some future plans Suggestions and comments? Sorry, I made minor changes last night

3

Quick Background on Project

Lots of interest in agent models Project Albert work Brawler modeling work Next Generation Mission Model

Other agent model work as well Adaptive interface agents Intelligent software agents Internet agents

Challenge is how to bring agent models into the higher level models?

4

Why Higher Level Modeling?

Need to better capture command and control effects

Need to capture “intangibles” Need to model learning based on battlefield

information Need better representation of actual

information use versus perfect use Agents and agent models hold promise but

bring along many issues

5

Agent Modeling Challenges

Output analysis Particularly with more complex models and models that are

not necessarily replicable

Accurate human behavior modeling In particular, command behavior modeling

Level of fidelity in model Beyond that of bouncing dots

Interaction of agents and legacy modeling approaches Brawler extensions into theater and campaign level modeling

6

Agent Modeling Challenges (cont).

Human interaction with the models The visual impact of interactions

among the agents “What if” analyses when human

behavior is being modeled Verification and Validation

7

The Project

Need a “use case” for agent models

Dr McCue’s book great example of operational analysis

Bay of Biscay scenario amenable to agent modeling Lots of information available

Forms a basis for subsequent research

8

Efforts Completed

Capt Ron “Greg” Carl (masters thesis) Search theory focus - finished

Capt Joe Price (masters thesis) Game theory focus - finished

Subhashini Ganapathy Optimization study - finished Entering PhD candidacy

Lance Champagne Dissertation defense in early Fall Same time twins are due!

9

Efforts Completed

Capt Ron “Greg” Carl (masters thesis) Search theory focus - finished

Capt Joe Price (masters thesis) Game theory focus - finished

Subhashini GanapathySubhashini Ganapathy Optimization study - finishedOptimization study - finished Entering PhD candidacyEntering PhD candidacy

Lance ChampagneLance Champagne Dissertation defense in early FallDissertation defense in early Fall Same time twins are due!Same time twins are due!

10

Snapshot of AFIT Model

11

Methodology - Game Portion

Allied search strategies When to search? Day versus night?

German U-boat surfacing strategies When to surface? Day versus night?

Two-person zero-sum game Players: Allied search aircraft and German U-boats Met rationality assumption

Non-perfect information Neither side knows the exact strategy the other uses

Objective is number of U-boat detections Allied goal: maximize German goal: minimize

Zero-sum game

12

Game Formulation

Allies: two pure search strategies Only day and only night

Germans: two pure surfacing strategies Only day and only night

Next step to include mixed strategies Let parameter range from 0 to 1 as strategy More interesting than simple pure strategy Still more interesting with adaptation

Simple adaptation algorithm Agents allowed to adapt strategy each month

13

Results – No Adaptation

Response Surface Methodology model Adjusted R2 = 0.947

1U

-Boa

t Day

Str

ateg

y0

U-Boat Detections

600

500

400

300

200

100

0

0 Aircraft Day Strategy 1

0

Aircraft Day Strategy

U-Boat Day Strategy

U-Boat Detections

Equilibrium Point, 0.7, 0.54

14

Adaptation Experiment

Design Point

Allied Search Strategy - Start

Allied Search Strategy - End

U-Boat Surfacing Strategy - Start

U-Boat Surfacing

Strategy - End

Average Number U-Boat

Detections1 (1, 0) (0.542, 0.458) (0, 1) (0.164, 0.836) 183.752 (1, 0) (0.625, 0.375) (1, 0) (0.327, 0.673) 180.453 (0.5, 0.5) (0.522, 0.478) (0.5, 0.5) (0.259, 0.741) 182.6

Both sides can adapt strategies (simple model) Three design points chosen: Adaptation occurs every month Investigate results 20 replications; 12-month warm-up; 12 months of

statistics collection (April 1943 – February 1944)

15

Adaptation ConvergenceTwo-Player Adaptation

Design Point 1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Start 1 2 3 4 5 6 7 8 9 10 11 12

Update (Months)

Da

y S

tra

teg

y

Aircraft Day Strategy U-Boat Day Strategy

Aircraft Starting Strategy: (1, 0)U-Boat Starting Strategy: (0, 1)

16

Adaptation ConvergenceTwo-Player Adaptation

Design Point 3

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Start 1 2 3 4 5 6 7 8 9 10 11 12

Update (Months)

Da

y S

trat

egy

Aircraft Day Strategy U-Boat Day Strategy

Aircraft Starting Strategy: (0.5, 0.5)U-Boat Starting Strategy: (0.5, 0.5)

17

Methodology Search Portion

Design data compiled according to hierarchy Historical fact Published studies Data derived from raw numbers Good judgment

MOE is number of U-boat sightings U-boat density constant between replications Aircraft flight hours same between replications Therefore, sightings = search efficiency

Two cases; search regions don’t overlap, do overlap

18

350

NM

2200 NM2

Non-overlapping Search Regions

19

100

NM

2100 NM2

Overlapping Search Regions

20

Non-overlapping Search Regions

Means Comparison—All Pairs (20 Iterations)(Similar Letters Indicate Statistical Equivalence)

Search Pattern

Mean Sightings

Square A 106.9Creeping Line A B 98.3Barrier Patrol B 96.4Sector B 91.9Parallel B 91.7

21

Non-overlapping Search Regions

Means Comparison—All Pairs (30 Iterations)(Similar Letters Indicate Statistical Equivalence)

Search Pattern

Mean Sightings

Square A 105.9Creeping Line B 97.3Barrier Patrol B 91.4

22

Overlapping Search Regions

Means Comparison—All Pairs (30 Iterations)(Similar Letters Indicate Statistical Equivalence)

Search Pattern

Mean Sightings

Square A 122.1Parallel A 121.0Barrier Patrol A 118.0Sector A 115.6Creeping Line A 115.6

23

Future Applications

Generalized architecture promotes re-use Coast Guard Deep-water efforts Air Force UAV search in rugged terrain or

urban environments

Human-in-the-loop issues permeate Search and rescue using UAVs Reconnaissance using UAVs Combat missions using UCAVs

24

Future Efforts

Champagne completing dissertation Ganapathy starting candidacy

Looked at simulation-based optimization Examining human-mediated optimization techniques Application to search and rescue or operational routing

Extensions planned Extend game theory aspects Further refinement of search results and optimization

use

25

Questions?