s ystems analysis laboratory helsinki university of technology games and bayesian networks in air...

S ystemsAnalysis LaboratoryHelsinki University of Technology

Games and Bayesian Networks Games and Bayesian Networks in Air Combat Simulation Analysisin Air Combat Simulation Analysis

M.Sc. Jirka Poropudas and Dr.Tech. Kai Virtanen

Systems Analysis LaboratoryHelsinki University of Technology

[email protected], [email protected]


OutlineOutline

• Air combat (AC) simulation

• Games in validation and optimization– Estimation of games from simulation data

– Analysis of estimated games

• Dynamic Bayesian networks (DBNs)– Estimation of DBNs from simulation data

– Analysis of estimated DBNs

• Conclusions


Air Combat SimulationAir Combat Simulation

• Commonly used models based on discrete event simulation

• Most cost-efficient and flexible method

Objectives for AC simulation studies: • Acquire information on systems performance• Compare tactics and hardware configurations• Increase understanding of AC and its progress


Discrete Event Simulation ModelDiscrete Event Simulation Model

Simulation input• Aircraft and

hardware configurations

• Tactics

• Decision making parameters

Simulation output• Number of kills and

losses

• Aircraft trajectories

• AC events

• etc.Decision making logic

Aircraft, weapons and hardware models

Stochastic elements

Validation of the model?Optimization of output?Evolution of simulation?


Existing Approaches to Simulation AnalysisExisting Approaches to Simulation Analysis

• Simulation metamodels– Mappings from simulation input to output

- Response surface methods, regression models, neural networks

• Validation methods– Real data, expert knowledge, statistical methods, sensitivity

analysis

• Simulation-optimization methods– Ranking and selection, stochastic gradient approximation,

metaheuristics, sample path optimization


Limitations of Existing ApproachesLimitations of Existing Approaches

• Existing approaches are one-sided– Action of the adversary is not taken into account– Two-sided setting studied with games

• Existing approaches are static– AC is turned into a static event– Time evolution studied with dynamic Bayesian networks


Games from Simulation DataGames from Simulation Data• Definition of scenario

– Aircraft, weapons, sensory and other systems– Initial geometry– Objectives = Measures of effectiveness (MOEs)– Available tactics and systems = Tactical alternatives

• Simulation of the scenario– Input: tactical alternatives– Output: MOE estimates

• Games estimated from the simulation data• Games used for validation and/or

optimization


Estimation of GamesEstimation of Games

RED

0.0040.036-0.833x3

0.0230.013-0.811x2

0.8850.855-0.077 x1

y3y2y1

BL

UE

IIIIIIIx3

IIIIIIIx2

IVIVII x1

y3y2y1

RED, min

BL

UE

, max

MOE estimates Payoff

Discrete tactical alternatives x and y

Analysis of variance

Simulation

Discrete decision variables x and y

Game


Estimation of GamesEstimation of Games

MOE estimates Payoff

Continuous tactical alternatives x and y

Simulation

Continuous decision variables x and y

Game

0

5

10

15

0

5

10

15

0

0.1

0.2

0.3

0.4

0.5

0

5

10

15

0

5

10

15

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

Blue xRed y

MO

E

estim

ate

Blue xRed y

Pay

off

Regression analysis

Experimental design


Analysis of GamesAnalysis of Games• Validation: Confirming that the simulation model

performs as intended

– Comparison of the scenario and properties of the game

– Symmetry, dependence between decision variables and payoffs, best responses and Nash equilibria

• Optimization: Comparison of effectiveness of tactical alternatives

– Different payoffs, best responses and Nash equilibria, dominance between alternatives, max-min solutions


Example: Missile Support Time GameExample: Missile Support Time Game

yx

x

y

Phase 1: SupportRelay radar information on the adversary to the missile

Phase 2: Extrapolation

Phase 3: Locked

• Symmetric one-on-one scenario• Tactical alternatives: Support times x and y• Objective => MOE: combination of kill probabilities• Simulation using X-Brawler


Game PayoffsGame PayoffsRegression models for kill probabilities:

Probability of Blue kill Probability of Red kill

0

5

10

15

0

5

10

15

0

0.2

0.4

0.6

0.8

0

5

10

15

0

5

10

15

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

Blue’s support time x

Blue’s support time x Red’s support time y

Red’s support time y

Payoff: Weighted sum of kill probabilities

• Blue: wB*Blue kill prob. + (1-wB)*Red kill prob.

• Red: wR*Red kill prob. + (1-wR)*Blue kill prob.

• Weights = Measure of aggressiveness


0 5 10 150

5

10

15

Best ResponsesBest ResponsesBest response =

Optimal support time against a given support time of the adversary

Best responses with different weights

Nash equilibria:Intersections of the best responses

WR=0

WR=0.5

WR=0.25

WR=0.75

WB=0.75WB=0.5WB=0.25WB=0

Blue’s support time x

Re

d’s

su

ppo

rt ti

me

y


Analysis of GameAnalysis of Game

• Symmetry– Symmetric kill probabilities and best responses

• Dependency– Increasing support times => Increase of kill probabililties

• Different payoffs– Increasing aggressiveness (higher values of wB and wR)

=> Longer support times• Best responses & Nash equilibria

– Increasing aggressiveness (higher values of wB and wR) => Longer support times


DBNs from Simulation DataDBNs from Simulation Data• Definition of simulation state

– Aircraft, weapons, sensory and other systems

• Simulation of the scenario– Input: tactical alternatives– Output: simulation state at all times

• DBNs estimated from the simulation data– Network structure– Network parameters

• DBNs used to analyze evolution of AC– Probabilities of AC states at time t– What if -analysis


Definition of State of ACDefinition of State of AC

• 1 vs. 1 AC

• Blue and Red

• Bt and Rt = AC state at time t

• State variable values

• “Phases” of simulated pilots

– Part of the decision making model

– Determine behavior and phase transitions for individual pilots

– Answer the question ”What is the pilot doing at time t?” Example of AC phases in X-Brawler

simulation model


Dynamic Bayesian Network for ACDynamic Bayesian Network for AC

• Dynamic Bayesian network– Nodes = variables

– Arcs = dependencies

• Dependence between variables described by– Network structure

– Conditional probability tables

• Time instant t presented by single time slice

• Outcome Ot depends on Bt and Rt

time slice


Dynamic Bayesian NetworkDynamic Bayesian NetworkFitted to Simulation DataFitted to Simulation Data

• Basic structure of DBN is assumed

• Additional arcs added to improve fit

• Probability tables estimated from simulation data


• Continuous probability curves estimated from simulation data

• DBN model re-produces probabilities at discrete times

• DBN gives compact and efficient model for the progress of AC

Evolution of ACEvolution of AC


What If -AnalysisWhat If -Analysis

• Evidence on state of AC fed to DBN

• For example, blue is engaged within visual range combat at time 125 s

– How does this affect the progress of AC?

– Or AC outcome?

• DBN allows fast and efficient updating of probability distributions

– More efficient what-if analysis

• No need for repeated re-screening simulation data


ConclusionsConclusions• New approaches for AC simulation analysis

– Two-sided and dynamic setting

– Simulation data represented in informative and compact form

• Game models used for validation and optimization

• Dynamic Bayesian networks used for analyzing the evolution of AC

• Future research:

– Combination of the approaches => Influence diagram games


References References » Anon. 2002. The X-Brawler air combat simulator management summary. Vienna,

VA, USA: L-3 Communications Analytics Corporation.

» Gibbons, R. 1992. A Primer in Game Theory. Financial Times Prenctice Hall.

» Feuchter, C.A. 2000. Air force analyst’s handbook: on understanding the nature of analysis. Kirtland, NM. USA: Office of Aerospace Studies, Air Force Material Command.

» Jensen, F.V. 2001. Bayesian networks and decision graphs (Information Science and Statistics). Secaucus, NJ, USA: Springer-Verlag New York, Inc.

» Law, A.M. and W.D. Kelton. 2000. Simulation modelling and analysis. New York, NY, USA: McGraw-Hill Higher Education.

» Poropudas, J. and K. Virtanen. 2007. Analyzing Air Combat Simulation Results with Dynamic Bayesian Networks. Proceedings of the 2007 Winter Simulation Conference.

» Poropudas, J. and K. Virtanen. 2008. Game Theoretic Approach to Air Combat Simulation Model. Submitted for publication.

» Virtanen, K., T. Raivio, and R.P. Hämäläinen. 1999. Decision theoretical approach to pilot simulation. Journal of Aircraft 26 (4):632-641.

s ystems analysis laboratory helsinki university of technology games and bayesian networks in air...

Documents

simulation data games

evolution of simulation

air combat simulation

ac simulation studies

y simulation continuous

games existing approaches

moe estimates games

progress slide