![Page 1: Game Theory for Security: Lessons Learned from Deployed Applications Milind Tambe Chris Kiekintveld Richard John & teamcore.usc.edu](https://reader038.vdocuments.net/reader038/viewer/2022110209/56649e0d5503460f94af5ed9/html5/thumbnails/1.jpg)
Game Theory for Security:Lessons Learned from Deployed Applications
Milind TambeChris KiekintveldRichard John &teamcore.usc.edu
![Page 2: Game Theory for Security: Lessons Learned from Deployed Applications Milind Tambe Chris Kiekintveld Richard John & teamcore.usc.edu](https://reader038.vdocuments.net/reader038/viewer/2022110209/56649e0d5503460f94af5ed9/html5/thumbnails/2.jpg)
Motivation: Infrastructure Security
Limited security resources: Selective checking
Adversary monitors defenses, exploits patterns
LAX
50 miles
![Page 3: Game Theory for Security: Lessons Learned from Deployed Applications Milind Tambe Chris Kiekintveld Richard John & teamcore.usc.edu](https://reader038.vdocuments.net/reader038/viewer/2022110209/56649e0d5503460f94af5ed9/html5/thumbnails/3.jpg)
Game Theory: Bayesian Stackelberg Games
Security allocation: (i) Target weights; (ii) Opponent reaction
Stackelberg: Security forces commit first
Bayesian: Uncertain adversary types
Optimal security allocation: Weighted random
Strong Stackelberg Equilibrium (Bayesian)
NP-hard
Terminal#1
Terminal#2
Terminal #1 5, -3 -1, 1
Terminal #2 -5, 5 2, -1Police
Adversary
![Page 4: Game Theory for Security: Lessons Learned from Deployed Applications Milind Tambe Chris Kiekintveld Richard John & teamcore.usc.edu](https://reader038.vdocuments.net/reader038/viewer/2022110209/56649e0d5503460f94af5ed9/html5/thumbnails/4.jpg)
Game Theory for Infrastructure Security: Outline
Deployed real world applications
Research challenges
Efficient algorithms: Scale-up to real-world problems
Human adversary: Bounded rationality & observations
Observability: Adversary surveillance uncertainty
Payoff uncertainty: ..
Evaluation
Algorithms: AAMAS(06,07,08,09); AAAI (08,10),...under review…
Algorithmic & experimental GT: AAMAS’09, AI Journal (2010), …
Observability: AAMAS’10,…
Applications: AAMAS Industry track (08,09), AI Magazine (09), Journal ITM (09), Interfaces (10), Informatica (10),…
![Page 5: Game Theory for Security: Lessons Learned from Deployed Applications Milind Tambe Chris Kiekintveld Richard John & teamcore.usc.edu](https://reader038.vdocuments.net/reader038/viewer/2022110209/56649e0d5503460f94af5ed9/html5/thumbnails/5.jpg)
Deployed Applications: ARMOR & IRIS
ARMOR: Randomized checkpoints & canine at LAX (August 2007)
IRIS: Randomized allocation of air marshals to flights (October 2009)
![Page 6: Game Theory for Security: Lessons Learned from Deployed Applications Milind Tambe Chris Kiekintveld Richard John & teamcore.usc.edu](https://reader038.vdocuments.net/reader038/viewer/2022110209/56649e0d5503460f94af5ed9/html5/thumbnails/6.jpg)
In Progress
TSA: Towards national deployment Spring’2011
GUARDS: Currently evaluation, testing
Coast Guard: Pilot demonstration Boston, March 2011
Los Angeles Sheriff’s Dept: Ticketless travelers…
GUARDS PROTECT Acronym
![Page 7: Game Theory for Security: Lessons Learned from Deployed Applications Milind Tambe Chris Kiekintveld Richard John & teamcore.usc.edu](https://reader038.vdocuments.net/reader038/viewer/2022110209/56649e0d5503460f94af5ed9/html5/thumbnails/7.jpg)
Outline of Presentation
Deployed real world applications
Research challenges
Efficient algorithms: Scale-up to real-world problems
Observability: Adversary surveillance uncertainty
Human adversary: Bounded rationality, observation power
Payoff uncertainty: Payoffs not point values; distributions
Evaluation
![Page 8: Game Theory for Security: Lessons Learned from Deployed Applications Milind Tambe Chris Kiekintveld Richard John & teamcore.usc.edu](https://reader038.vdocuments.net/reader038/viewer/2022110209/56649e0d5503460f94af5ed9/html5/thumbnails/8.jpg)
Efficient Algorithms
Challenges: Combinatorial explosions due to
Defender strategies: Allocations of resources to targets
E.g. 100 flights, 10 FAMS
Attacker strategies: Attack paths
E.g. Multiple attack paths to targets in a city
Adversary types: Adversary strategy combination
![Page 9: Game Theory for Security: Lessons Learned from Deployed Applications Milind Tambe Chris Kiekintveld Richard John & teamcore.usc.edu](https://reader038.vdocuments.net/reader038/viewer/2022110209/56649e0d5503460f94af5ed9/html5/thumbnails/9.jpg)
Scale-up:Defender actions
Scale-up:Attacker actions
Scale-up:Attacker types
Domain structure exploited
Exact orApprox
Type of equilibrium
Algorithm
Low Low Medium None Approx SSE ASAP 2007
Low Low Medium None Exact SSE DOBSS 2008
Low Low Medium None Exact rationality, observation
COBRA2009
Medium Low Low High (Security
game, 1 target)Exact SSE ORIGAMI
2009
Medium Low Low High (Security game, 2 targets)
Approx SSE ERASER2009
Medium Low Low Med (Security game, N targets)
Exact SSE ASPEN2010
Medium Medium Low High (zero-sum, graph)
Approx SSE RANGER2010
Review2010
ARMOR
ARMOR
IRIS-I
IRIS-II
IRIS-III
SCALE-UP
![Page 10: Game Theory for Security: Lessons Learned from Deployed Applications Milind Tambe Chris Kiekintveld Richard John & teamcore.usc.edu](https://reader038.vdocuments.net/reader038/viewer/2022110209/56649e0d5503460f94af5ed9/html5/thumbnails/10.jpg)
ARMOR: Multiple Adversary Types
Term #1 Term #2
Term#1 5, -3 -1, 1
Term#2 -5, 5 2, -1
Term #1 Term #2
Term#1 2, -1 -3, 4
Term#2 -3, 1 3, -3
Term #1 Term #2
Term#1 4, -2 -1,0.5
Term#2 -4, 3 1.5, -0.5
P=0.3 P=0.5 P=0.2
NP-hard
Previous work: Linear programs using Harsanyi transformation
111 121 112 211 … … … 222
Terminal #1
3.3,-2.2 2.3,…
Terminal #2
-3.8,2.6 …,…
![Page 11: Game Theory for Security: Lessons Learned from Deployed Applications Milind Tambe Chris Kiekintveld Richard John & teamcore.usc.edu](https://reader038.vdocuments.net/reader038/viewer/2022110209/56649e0d5503460f94af5ed9/html5/thumbnails/11.jpg)
Mixed-integer programs
No Harsanyi transformation
Significantly more efficient than previous approaches at the time
Multiple Adversary Types:Decomposition for Bayesian Stackelberg Games
}1,0{],1...0[
)1()(0
1,1..
max ,
lji
lji
Xi
lij
l
Qj
lj
ii
lji
lij
Xi Ll Qj
lqx
qx
MqxCa
qxts
qxRp
![Page 12: Game Theory for Security: Lessons Learned from Deployed Applications Milind Tambe Chris Kiekintveld Richard John & teamcore.usc.edu](https://reader038.vdocuments.net/reader038/viewer/2022110209/56649e0d5503460f94af5ed9/html5/thumbnails/12.jpg)
Scale-up:Defender actions
Scale-up:Attacker actions
Scale-up:Attacker types
Domain structure exploited
Exact orApprox
Type of equilibrium
Algorithm
Low Low Medium None Approx SSE ASAP 2007
Low Low Medium None Exact SSE DOBSS 2008
Low Low Medium None Exact rationality, observation
COBRA2009
Medium Low Low High (Security
game, 1 target)Exact SSE ORIGAMI
2009
Medium Low Low High (Security game, 2 targets)
Approx SSE ERASER2009
Medium Low Low Med (Security game, N targets)
Exact SSE ASPEN2010
Medium Medium Low High (zero-sum, graph)
Approx SSE RANGER2010
Medium Medium Low High (zero-sum, graph)
Exact SSE Submitted2010
ARMOR
ARMOR
IRIS-I
IRIS-II
IRIS-III
SCALE-UP
![Page 13: Game Theory for Security: Lessons Learned from Deployed Applications Milind Tambe Chris Kiekintveld Richard John & teamcore.usc.edu](https://reader038.vdocuments.net/reader038/viewer/2022110209/56649e0d5503460f94af5ed9/html5/thumbnails/13.jpg)
Federal Air Marshals Service
Flights (each day)~27,000 domestic flights~2,000 international flights
International Flights from Chicago O’Hare
Estimated 3,000-4,000 air marshals
Massive scheduling problem:How to assign marshals to flights?
![Page 14: Game Theory for Security: Lessons Learned from Deployed Applications Milind Tambe Chris Kiekintveld Richard John & teamcore.usc.edu](https://reader038.vdocuments.net/reader038/viewer/2022110209/56649e0d5503460f94af5ed9/html5/thumbnails/14.jpg)
IRIS Scheduling Tool
![Page 15: Game Theory for Security: Lessons Learned from Deployed Applications Milind Tambe Chris Kiekintveld Richard John & teamcore.usc.edu](https://reader038.vdocuments.net/reader038/viewer/2022110209/56649e0d5503460f94af5ed9/html5/thumbnails/15.jpg)
IRIS Scheduling Tool
Flight Information
Resources
Risk Information
GameModel
SolutionAlgorithm
RandomizedDeploymentSchedule
![Page 16: Game Theory for Security: Lessons Learned from Deployed Applications Milind Tambe Chris Kiekintveld Richard John & teamcore.usc.edu](https://reader038.vdocuments.net/reader038/viewer/2022110209/56649e0d5503460f94af5ed9/html5/thumbnails/16.jpg)
IRIS: Large Numbers of Defender Strategies
Strategy 1 Strategy 2 Strategy 3
Strategy 1
Strategy 2
Strategy 3
Strategy 4
Strategy 5
Strategy 6
Strategy 1 Strategy 2 Strategy 3
Strategy 1
Strategy 2
Strategy 3
Strategy 4
Strategy 5
Strategy 6
784 x 512
100,000,000…x
1000…
4 Flight tours2 Air Marshals
100 Flight tours10 Air Marshals
6 Schedules
1.73 x 1013
Schedules:ARMORout of memory
FAMS: Joint Strategies
![Page 17: Game Theory for Security: Lessons Learned from Deployed Applications Milind Tambe Chris Kiekintveld Richard John & teamcore.usc.edu](https://reader038.vdocuments.net/reader038/viewer/2022110209/56649e0d5503460f94af5ed9/html5/thumbnails/17.jpg)
Addressing Scale-up in Defender Strategies
Security game:Payoffs depend on attacked target covered or not
Target independence
Avoid enumeration of all joint strategies:
Marginals: Probabilities for individual strategies/schedules
Sample required joint strategies: IRIS I and IRIS II
But: Sampling may be difficult if schedule conflicts IRIS I (single target/flight), IRIS II (pairs of targets)
Branch & Price: Probabilities on joint strategies
Enumerates required joint strategies, handles conflicts IRIS III (arbitrary schedules over targets)
![Page 18: Game Theory for Security: Lessons Learned from Deployed Applications Milind Tambe Chris Kiekintveld Richard John & teamcore.usc.edu](https://reader038.vdocuments.net/reader038/viewer/2022110209/56649e0d5503460f94af5ed9/html5/thumbnails/18.jpg)
Explosion in Defender Strategies:Marginals for Compact Representation
ARMORActions
Tour combos
Prob
1 1,2,3 x1
2 1,2,4 x2
3 1,2,5 x3
… … …
120 8,9,10 x120
CompactAction
Tour Prob
1 1 y1
2 2 y2
3 3 y3
… … …
10 10 y10
Attack 1
Attack 2
Attack …
Attack 6
1,2,3 5,-10 4,-8 … -20,91,2,4 5,-10 4,-8 … -20,91,3,5 5,-10 -9,5 … -20,9… … … … …
ARMOR: 10 tours, 3 air marshals Payoff duplicates: Depends on target covered
IRIS MILP similar to ARMOR 10 instead of 120 variables y1+y2+y3…+y10 = 3 Construct samples over tour combos
}1,0{],1...0[
)1()(0
1,1..
max ,
lji
lji
Xi
lij
l
Qj
lj
ii
lji
lij
Xi Ll Qj
lqx
qx
MqxCa
qxts
qxRp
![Page 19: Game Theory for Security: Lessons Learned from Deployed Applications Milind Tambe Chris Kiekintveld Richard John & teamcore.usc.edu](https://reader038.vdocuments.net/reader038/viewer/2022110209/56649e0d5503460f94af5ed9/html5/thumbnails/19.jpg)
IRIS Speedups: Efficient Algorithms II
FAMSIreland
FAMSLondon
ARMORActions
ARMORRuntime
IRIS Runtime
6,048 4.74s 0.09s
85,275 ---- 1.57s
10 11 12 13 14 15 16 17 18 19 200
0.51
1.52
2.53
3.54
4.55
Scaling with Targets: CompactARMOR IRIS I IRIS II
Targets
Run
tim
es (
min
)
![Page 20: Game Theory for Security: Lessons Learned from Deployed Applications Milind Tambe Chris Kiekintveld Richard John & teamcore.usc.edu](https://reader038.vdocuments.net/reader038/viewer/2022110209/56649e0d5503460f94af5ed9/html5/thumbnails/20.jpg)
IRIS III
Next generation of IRIS
General scheduling constraints
Schedules can be any subset of targets
Resource can be constrained to any subset of schedules
Branch and Price Framework
Techniques for large-scale optimization
Not an “out of the box” solution
![Page 21: Game Theory for Security: Lessons Learned from Deployed Applications Milind Tambe Chris Kiekintveld Richard John & teamcore.usc.edu](https://reader038.vdocuments.net/reader038/viewer/2022110209/56649e0d5503460f94af5ed9/html5/thumbnails/21.jpg)
IRIS III: Branch and Price:Branch & Bound + Column Generation
First Node: all ai [0,1]
Lower bound 1: a1= 1, arest= 0
Second node: a1= 0, arest [0,1]
Lower bound 2: a1= 0, a2= 1, arest= 0
Third node: a1,a2= 0,
arest [0,1]
LB last: ak= 1, arest= 0
Not “out of the box”• Bounds and branching: IRIS I• Column generation leaf nodes: Network flow
![Page 22: Game Theory for Security: Lessons Learned from Deployed Applications Milind Tambe Chris Kiekintveld Richard John & teamcore.usc.edu](https://reader038.vdocuments.net/reader038/viewer/2022110209/56649e0d5503460f94af5ed9/html5/thumbnails/22.jpg)
Column Generation
“Master”Problem(mixed integer program)
Restricted set of joint schedules
“Slave”Problem
Return the “best”joint schedule to add
Target 3 Target 7
… …
Resource Sink
Capacity 1 on all links
(N+1)th joint schedule
Solution with N joint schedules
Minimum cost network flow: Identifies joint schedule to add
![Page 23: Game Theory for Security: Lessons Learned from Deployed Applications Milind Tambe Chris Kiekintveld Richard John & teamcore.usc.edu](https://reader038.vdocuments.net/reader038/viewer/2022110209/56649e0d5503460f94af5ed9/html5/thumbnails/23.jpg)
Results: IRIS III
200 400 600 800 10001
10
100
1000
Comparison (200 Targets, 10 Resources)
ERASER-C
BnP
ASPEN
Number of Schedules
Run
tim
e (i
n se
cs)
[log
-sca
le]
IRIS II
IRIS III
B&P
5 10 15 200
1000
2000
3000
4000
5000
6000
7000
8000
Scale-up (200 Targets, 1000 schedules)
2 Targets/Schedule
3 Targets/Schedule
4 Targets/Schedule
5 Targets/Schedule
Number of Resources
Run
time
(in
seco
nds)
![Page 24: Game Theory for Security: Lessons Learned from Deployed Applications Milind Tambe Chris Kiekintveld Richard John & teamcore.usc.edu](https://reader038.vdocuments.net/reader038/viewer/2022110209/56649e0d5503460f94af5ed9/html5/thumbnails/24.jpg)
Modeling Complex Security Games
Approach: domain experts supply the model
Experts must understand necessary game inputs
What information is available? Sensitive?
Number of inputs must be reasonable (tens, not thousands)
What models can we solve computationally?
![Page 25: Game Theory for Security: Lessons Learned from Deployed Applications Milind Tambe Chris Kiekintveld Richard John & teamcore.usc.edu](https://reader038.vdocuments.net/reader038/viewer/2022110209/56649e0d5503460f94af5ed9/html5/thumbnails/25.jpg)
Robustness
We do not know exactly
Strategies/capabilities
Payoffs/preferences
Observation capabilities
…
How can we take this into account?
Richer models
Faster algorithms to solve these models
Sensitivity analysis
![Page 26: Game Theory for Security: Lessons Learned from Deployed Applications Milind Tambe Chris Kiekintveld Richard John & teamcore.usc.edu](https://reader038.vdocuments.net/reader038/viewer/2022110209/56649e0d5503460f94af5ed9/html5/thumbnails/26.jpg)
Approximating Infinite Bayesian Games
Idea: replace point estimates for attacker payoffs with Gaussian distributions
![Page 27: Game Theory for Security: Lessons Learned from Deployed Applications Milind Tambe Chris Kiekintveld Richard John & teamcore.usc.edu](https://reader038.vdocuments.net/reader038/viewer/2022110209/56649e0d5503460f94af5ed9/html5/thumbnails/27.jpg)
Attacker Surveillance: Stackelberg vs Nash
Defender commits first:
Attacker conducts surveillance
Stackelberg (SSE)
Simultaneous move game:
Attacker conducts no surveillance
Mixed strategy Nash (NE)
SSE
NE = Minimax
Set of defender strategies
How should a defender compute her strategy?
For security games with SSAS:
![Page 28: Game Theory for Security: Lessons Learned from Deployed Applications Milind Tambe Chris Kiekintveld Richard John & teamcore.usc.edu](https://reader038.vdocuments.net/reader038/viewer/2022110209/56649e0d5503460f94af5ed9/html5/thumbnails/28.jpg)
Evaluation of Real-World Applications
“Application track” papers:
Beyond run-time and optimality
Difficult and not “Solved”!Reviewer questions Operational perspective
Do your algorithms work: are we safe?
No 100% protection; only increase cost/uncertainty of attacker
Turn off security apparatus for a year; compare
?? adversaries will cooperate..??
Send in a red team NOT the right test
Give us all the before/after data Security sensitivities
![Page 29: Game Theory for Security: Lessons Learned from Deployed Applications Milind Tambe Chris Kiekintveld Richard John & teamcore.usc.edu](https://reader038.vdocuments.net/reader038/viewer/2022110209/56649e0d5503460f94af5ed9/html5/thumbnails/29.jpg)
So how can we evaluate?...
No 100% security; are we better off than previous approaches?
Models and simulations
Human adversaries in the lab
Expert evaluation
Systems in use for a number of years: internal evaluations
Future:
Newer domains that allow validation in the real-world
Criminologists (TSA)
![Page 30: Game Theory for Security: Lessons Learned from Deployed Applications Milind Tambe Chris Kiekintveld Richard John & teamcore.usc.edu](https://reader038.vdocuments.net/reader038/viewer/2022110209/56649e0d5503460f94af5ed9/html5/thumbnails/30.jpg)
Models & Simulations I
-2-10123456
Rew
ard
Days
ARMOR v/ s Non-weighted (uniformed) Random for Canines
ARMOR: 6 canines
ARMOR: 5 canines
ARMOR: 3 canines
Non-weighted: 6 canines
![Page 31: Game Theory for Security: Lessons Learned from Deployed Applications Milind Tambe Chris Kiekintveld Richard John & teamcore.usc.edu](https://reader038.vdocuments.net/reader038/viewer/2022110209/56649e0d5503460f94af5ed9/html5/thumbnails/31.jpg)
Human Adversaries In the Lab
![Page 32: Game Theory for Security: Lessons Learned from Deployed Applications Milind Tambe Chris Kiekintveld Richard John & teamcore.usc.edu](https://reader038.vdocuments.net/reader038/viewer/2022110209/56649e0d5503460f94af5ed9/html5/thumbnails/32.jpg)
Human Adversaries in the Lab
Unobserved 5 Observations 20 Observations Unlimited
-4-3.5
-3-2.5
-2-1.5
-1-0.5
00.5
1
Average expected reward
DOBSSMAXIMINUniformCOBRACOBRA-C
ARMOR
ARMOR: Outperforms uninformed random, not Maximin COBRA: Anchoring bias, “epsilon-optimal”
MqxCaq
Xxxts
qxRp
lji
Xi
lij
llj
lji
lij
Xi Ll Qj
lqx
)1()'()1(
|)|/1()1('..
max ,
![Page 33: Game Theory for Security: Lessons Learned from Deployed Applications Milind Tambe Chris Kiekintveld Richard John & teamcore.usc.edu](https://reader038.vdocuments.net/reader038/viewer/2022110209/56649e0d5503460f94af5ed9/html5/thumbnails/33.jpg)
Expert Evaluation I
April 2008 February 2009
LAX Spokesperson, CNN.com, July 14, 2010: "Randomization and unpredictability is a key factor in keeping the terrorists unbalanced….It is so effective that airports across the United States are adopting this method."
![Page 34: Game Theory for Security: Lessons Learned from Deployed Applications Milind Tambe Chris Kiekintveld Richard John & teamcore.usc.edu](https://reader038.vdocuments.net/reader038/viewer/2022110209/56649e0d5503460f94af5ed9/html5/thumbnails/34.jpg)
Expert Evaluation II
Federal Air Marshals Service (May 2010):
We…have continued to expand the number of flights scheduled using IRIS….we are satisfied with IRIS and confident in using this scheduling approach.
James B. Curren
Special Assistant, Office of Flight Operations,
Federal Air Marshals Service
![Page 35: Game Theory for Security: Lessons Learned from Deployed Applications Milind Tambe Chris Kiekintveld Richard John & teamcore.usc.edu](https://reader038.vdocuments.net/reader038/viewer/2022110209/56649e0d5503460f94af5ed9/html5/thumbnails/35.jpg)
What Happened at Checkpoints before and after ARMOR-- Not a Scientific Result!
January 2009• January 3rd Loaded 9/mm pistol• January 9th 16-handguns, 4-rifles,1-assault rifle; 1000 rounds of ammo• January 10th Two unloaded
shotguns • January 12th Loaded 22/cal rifle• January 17th Loaded 9/mm pistol• January 22nd Unloaded 9/mm pistol
Arrest data:
![Page 36: Game Theory for Security: Lessons Learned from Deployed Applications Milind Tambe Chris Kiekintveld Richard John & teamcore.usc.edu](https://reader038.vdocuments.net/reader038/viewer/2022110209/56649e0d5503460f94af5ed9/html5/thumbnails/36.jpg)
Deployed Applications: ARMOR, IRIS, GUARDS
Research challenges
Efficient algorithms: Scale-up to real-world problems
Observability: Adversary surveillance uncertainty
Human adversary: Bounded rationality, observation power
…
![Page 37: Game Theory for Security: Lessons Learned from Deployed Applications Milind Tambe Chris Kiekintveld Richard John & teamcore.usc.edu](https://reader038.vdocuments.net/reader038/viewer/2022110209/56649e0d5503460f94af5ed9/html5/thumbnails/37.jpg)
Future Work I
Protect networks (AAAI’10
under review)
Adversary uncertainty
(under review)
Payoffs/types
Surveillance
…
A B
A ? ?
B ?
![Page 38: Game Theory for Security: Lessons Learned from Deployed Applications Milind Tambe Chris Kiekintveld Richard John & teamcore.usc.edu](https://reader038.vdocuments.net/reader038/viewer/2022110209/56649e0d5503460f94af5ed9/html5/thumbnails/38.jpg)
Future Work II:Game Theory and Human Behavior
![Page 39: Game Theory for Security: Lessons Learned from Deployed Applications Milind Tambe Chris Kiekintveld Richard John & teamcore.usc.edu](https://reader038.vdocuments.net/reader038/viewer/2022110209/56649e0d5503460f94af5ed9/html5/thumbnails/39.jpg)
Experiment result
PT = Prospect theoryQRE = Quantal Response Equilibrium