structured models for multi-agent interactions daphne koller stanford university joint work with...
Post on 21-Dec-2015
218 views
TRANSCRIPT
![Page 1: Structured Models for Multi-Agent Interactions Daphne Koller Stanford University Joint work with Brian Milch, U.C. Berkeley](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d585503460f94a37f66/html5/thumbnails/1.jpg)
Structured Models forMulti-Agent Interactions
Daphne KollerStanford University
Joint work with Brian Milch, U.C. Berkeley
![Page 2: Structured Models for Multi-Agent Interactions Daphne Koller Stanford University Joint work with Brian Milch, U.C. Berkeley](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d585503460f94a37f66/html5/thumbnails/2.jpg)
Scaling Up• Question:
– Modeling and solving small games is already hard– How can we scale up to larger ones?
• Answer:– Real-world situations have a lot of structure– Otherwise people wouldn’t be able to handle them
• Goal: construct– languages based on structured representations,
allowing compact models of complex situations– algorithms that exploit this structure to support
effective reasoning
![Page 3: Structured Models for Multi-Agent Interactions Daphne Koller Stanford University Joint work with Brian Milch, U.C. Berkeley](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d585503460f94a37f66/html5/thumbnails/3.jpg)
Representations of Games
• Normal form– basic units: strategies– game representation loses
all structure– matrix size exponentially
larger than game tree
• Extensive form– basic units: events– game structure explicitly
encodes time, information– game tree size can still be
very large
strategies of player II
stra
tegi
es o
f pl
ayer
I
![Page 4: Structured Models for Multi-Agent Interactions Daphne Koller Stanford University Joint work with Brian Milch, U.C. Berkeley](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d585503460f94a37f66/html5/thumbnails/4.jpg)
Representation & Inferenceso
lutio
n tim
e (s
ec)
size of tree
0
5
10
15
20
25
30
0 5000 10000 15000 20000
Normal form
Sequence form
Minimax linear program for two-player zero-sum gamesApplied to abstract 2-player poker [Koller + Pfeffer]
[Romanovskii, 1962; Koller, Megiddo & von Stengel, 1994]
![Page 5: Structured Models for Multi-Agent Interactions Daphne Koller Stanford University Joint work with Brian Milch, U.C. Berkeley](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d585503460f94a37f66/html5/thumbnails/5.jpg)
MAID Representation
• MAID form– basic units: variables & dependencies between them– game structure explicitly encodes time, information, independence– can be exponentially smaller than game tree– game structure supports new forms of decomposition & backward
inductions• solving can be exponentially more efficient than extensive form
Sales-A
B SalesStrategy
Cost
Commission
ResourceAllocation
Sales-B
A SalesStrategy
Commission
Revenue
![Page 6: Structured Models for Multi-Agent Interactions Daphne Koller Stanford University Joint work with Brian Milch, U.C. Berkeley](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d585503460f94a37f66/html5/thumbnails/6.jpg)
Outline
Probabilistic Reasoning: Bayesian networks[Pearl, Jensen, …]
Influence Diagrams
Strategic Relevance
Exploiting Structure for Solving Games
![Page 7: Structured Models for Multi-Agent Interactions Daphne Koller Stanford University Joint work with Brian Milch, U.C. Berkeley](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d585503460f94a37f66/html5/thumbnails/7.jpg)
Probability Distributions
• Probabilistic model (e.g., a la Savage):– set of possible states in which the world can be;– probability distribution over this space.
• State: assignment of values to variables– diseases, symptoms, predisposing factors, …
• Problem:– n variables 2n states (or more);– representing the joint distribution is infeasible.
![Page 8: Structured Models for Multi-Agent Interactions Daphne Koller Stanford University Joint work with Brian Milch, U.C. Berkeley](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d585503460f94a37f66/html5/thumbnails/8.jpg)
P(A | B,E)a function
Val(B,E) (Val(A))
Bayesian Network
nodes = random variablesedges = direct probabilistic influence
Network structure encodes conditional independencies: Phone-Call is independent of Burglary given Alarm
PhoneCall Newscast
EarthquakeBurglary0.8 0.2
b
e
b
0.6 0.4
0.01 0.99
0.2 0.8
eb
e
e
b
EB P(A | B, E)
Alarm
![Page 9: Structured Models for Multi-Agent Interactions Daphne Koller Stanford University Joint work with Brian Milch, U.C. Berkeley](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d585503460f94a37f66/html5/thumbnails/9.jpg)
BN Semantics: Probability Model
• Compact & natural representation:– nodes have k parents 2kn vs. 2n parameters– parameters natural and easy to elicit.
qualitativeBN structure
+local
probabilitymodels
full jointdistribution
over domain=
e)|nP(a)|P(ce),b|P(aP(e))bP()nc,a,e,,bP(
C
A
N
EB
![Page 10: Structured Models for Multi-Agent Interactions Daphne Koller Stanford University Joint work with Brian Milch, U.C. Berkeley](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d585503460f94a37f66/html5/thumbnails/10.jpg)
BN Semantics: Independencies• The graph structure of the BN implies a set of
conditional independence assumptions– satisfied by every distribution over this graph
C
A
N
EB
Burglary and Call independent given AlarmNewscast and Alarm independent given
Earthquake
Burglary and Earthquake independent
C
A
N
EB
C
A
N
EB
C
A
N
EB
![Page 11: Structured Models for Multi-Agent Interactions Daphne Koller Stanford University Joint work with Brian Milch, U.C. Berkeley](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d585503460f94a37f66/html5/thumbnails/11.jpg)
BN Semantics: Dependencies• BN structure also specifies potential dependencies
– those that might hold for some distribution over graph
C
A
N
EB
• Burglary and Earthquake dependent given Alarm
C
A
N
EB
![Page 12: Structured Models for Multi-Agent Interactions Daphne Koller Stanford University Joint work with Brian Milch, U.C. Berkeley](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d585503460f94a37f66/html5/thumbnails/12.jpg)
Active paths
C
A
D
B
• Probabilistic influence “flows” along “active” paths• “d-separation” if there is no active path
B, C can be dependent
B, C are independentgiven A
B, C can be dependentgiven A,D
A
D
Simple linear-time algorithm for testing conditionalSimple linear-time algorithm for testing conditionalindependence using only graphical structure:independence using only graphical structure:
• Sound: d-separation independence for all P
• Complete: no d-separation dependence for almost all P
![Page 13: Structured Models for Multi-Agent Interactions Daphne Koller Stanford University Joint work with Brian Milch, U.C. Berkeley](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d585503460f94a37f66/html5/thumbnails/13.jpg)
CPCS
21000 states
![Page 14: Structured Models for Multi-Agent Interactions Daphne Koller Stanford University Joint work with Brian Milch, U.C. Berkeley](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d585503460f94a37f66/html5/thumbnails/14.jpg)
Bayesian Networks• Explicit representation of domain structure
• Cognitively intuitive compact models of complex domains
• Same model allows relevant probabilities to be computed in any evidence state
• Algorithms that exploit structure for effective inference even in very large models
![Page 15: Structured Models for Multi-Agent Interactions Daphne Koller Stanford University Joint work with Brian Milch, U.C. Berkeley](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d585503460f94a37f66/html5/thumbnails/15.jpg)
Outline
Probabilistic Reasoning: Bayesian networks
Influence Diagrams[Howard, Shachter, Jensen, …]
Strategic Relevance
Exploiting Structure for Solving Games
![Page 16: Structured Models for Multi-Agent Interactions Daphne Koller Stanford University Joint work with Brian Milch, U.C. Berkeley](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d585503460f94a37f66/html5/thumbnails/16.jpg)
Example: The Tree Killer
• Alice wants a patio, but the benefit outweighs the cost only if she gets an ocean view
• Bob’s tree blocks her view• Alice chooses whether to poison the tree• Tree may become sick• Bob chooses whether to call a tree doctor
– Alice can see whether tree doctor comes
• Alice chooses whether to build her patio• Tree may die when winter comes
![Page 17: Structured Models for Multi-Agent Interactions Daphne Koller Stanford University Joint work with Brian Milch, U.C. Berkeley](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d585503460f94a37f66/html5/thumbnails/17.jpg)
Standard Representation: Game Tree
Poison Tree?
Tree Sick?
Call Tree Doctor?
Build Patio?
Tree Dead?
5 levels; 25 = 32 terminal nodes
![Page 18: Structured Models for Multi-Agent Interactions Daphne Koller Stanford University Joint work with Brian Milch, U.C. Berkeley](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d585503460f94a37f66/html5/thumbnails/18.jpg)
Multi-Agent Influence Diagrams (MAIDs)
TreeSick
TreeDoctor
View
Cost
SpikeTree
BuildPatio
TreeDead
Tree
Influence diagram representation easily extended to multiple agents
“Tree killer”example
![Page 19: Structured Models for Multi-Agent Interactions Daphne Koller Stanford University Joint work with Brian Milch, U.C. Berkeley](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d585503460f94a37f66/html5/thumbnails/19.jpg)
Decision Nodes
• Incoming edges are information edges– variables whose values the agent knows when deciding– agent’s strategy can depend on values of parents
• Each parent instantiation– u Val(Parents(D))
is an information set• Perfect recall: if D1 precedes D2
– at D2 agent remembers:• his decision at D1
• everything he knew at D1
– formally: {D1,Parents(D1)} Parents(D2)– usually perfect recall edges are implicit, not drawn
TreeSick
TreeDoctor
SpikeTree
BuildPatio
![Page 20: Structured Models for Multi-Agent Interactions Daphne Koller Stanford University Joint work with Brian Milch, U.C. Berkeley](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d585503460f94a37f66/html5/thumbnails/20.jpg)
Strategies
• Strategy at D:– A pure (deterministic) strategy specifies an action at D for
every information set u – A behavior strategy specifies a distribution over actions
for every u
• Strategy specifies distribution P(D | Parents(D))– turns a decision node into a chance node– information parents play exactly the same role as parents
of chance node
![Page 21: Structured Models for Multi-Agent Interactions Daphne Koller Stanford University Joint work with Brian Milch, U.C. Berkeley](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d585503460f94a37f66/html5/thumbnails/21.jpg)
MAID Semantics• MAID M defines a set of possible strategy profiles• M plus any strategy profile defines a BN M[]
– Each decision node D becomes a chance node, with [D] as its CPD
• M[] defines a probability distribution, from which we can derive an expected utility for each agent:
• Thus, a MAID defines a mapping from strategy profiles to expected utility vectors
aU UValu
Ma uuUPEUU )(
][ )()(
![Page 22: Structured Models for Multi-Agent Interactions Daphne Koller Stanford University Joint work with Brian Milch, U.C. Berkeley](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d585503460f94a37f66/html5/thumbnails/22.jpg)
ReadabilityP1 Hand P2 Hand
Bet
Flop Cards
BetBet
BetBet
Card 4
Bet
Bet
Bet
U
Bet
![Page 23: Structured Models for Multi-Agent Interactions Daphne Koller Stanford University Joint work with Brian Milch, U.C. Berkeley](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d585503460f94a37f66/html5/thumbnails/23.jpg)
CompactnessSuitability
1W
Building1E
Building1W
Suitability 1E
Suitability 2W
Building2E
Building2W
Suitability 2E
Suitability 3W
Building3E
Building3W
Suitability 3E
Util 1E
Util 2W
Util 2E
Util 1W
Util 3W
Util 3E“Road”
example
![Page 24: Structured Models for Multi-Agent Interactions Daphne Koller Stanford University Joint work with Brian Milch, U.C. Berkeley](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d585503460f94a37f66/html5/thumbnails/24.jpg)
Compactness• Assume all variables have three values• Each decision node observes three variables• Number of information sets per agent: 33 = 27
• Size of MAID: – n chance nodes of “size” 3– n decision nodes of “size” 27·3
• Size of game tree: – 2n splits, each over three values
• Size of normal (matrix) form:– n players, each with 327 pure strategies
54n
32n
(327)n
![Page 25: Structured Models for Multi-Agent Interactions Daphne Koller Stanford University Joint work with Brian Milch, U.C. Berkeley](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d585503460f94a37f66/html5/thumbnails/25.jpg)
Outline
Probabilistic Reasoning: Bayesian networks
Influence Diagrams
Strategic Relevance
Exploiting Structure for Solving Games
![Page 26: Structured Models for Multi-Agent Interactions Daphne Koller Stanford University Joint work with Brian Milch, U.C. Berkeley](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d585503460f94a37f66/html5/thumbnails/26.jpg)
Optimality and Equilibrium
• Let E be a subset of Da, and let be a partial strategy over E
• Is the best partial strategy for agent a to adopt?– Depends on decision rules for other decision nodes
is optimal for a strategy profile if for all partial strategies ’ over E :
• A strategy profile is a Nash equilibrium if for every agent a, Da
is optimal for
)',(),( EE aa EUEU
![Page 27: Structured Models for Multi-Agent Interactions Daphne Koller Stanford University Joint work with Brian Milch, U.C. Berkeley](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d585503460f94a37f66/html5/thumbnails/27.jpg)
MAIDs and Games
• A MAID is equivalent to a game tree: it defines a mapping from strategy profiles to payoff vectors
• Finding equilibria in the MAID is equivalent to finding equilibria in the game tree
• One way to find equilibrium in MAID: – construct the game tree– solve the game
Incurs exponential blowup in representation size
• Question: can we find equilibria in a MAID directly?
![Page 28: Structured Models for Multi-Agent Interactions Daphne Koller Stanford University Joint work with Brian Milch, U.C. Berkeley](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d585503460f94a37f66/html5/thumbnails/28.jpg)
Local Optimization
• Consider finding a decision rule for a single decision node D that is optimal for
• For each instantiation pa of Pa(D), must find P* that maximizes:
• Some decision rules in may not affect this maximization problem
)( )(
][* ),|()|(
DVald U UValuM
a
upaduUPpadPU
![Page 29: Structured Models for Multi-Agent Interactions Daphne Koller Stanford University Joint work with Brian Milch, U.C. Berkeley](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d585503460f94a37f66/html5/thumbnails/29.jpg)
Strategic Relevance
• Intuitively, D relies on D’ if we need to know the decision rule for D’ in order to determine the optimal decision rule for D.
• We define a relevance graph, with:– a node for each decision– an edge from D to D’ if D relies on D’
D
D’
![Page 30: Structured Models for Multi-Agent Interactions Daphne Koller Stanford University Joint work with Brian Milch, U.C. Berkeley](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d585503460f94a37f66/html5/thumbnails/30.jpg)
Examples I: Information
D
D’
D
D’
U
don’tcare
U
D
D’
D
D’
perfectenough
U
D
D’
D
D’
perfectinfo
U
D
D’
D
D’
simultaneousmove
U
![Page 31: Structured Models for Multi-Agent Interactions Daphne Koller Stanford University Joint work with Brian Milch, U.C. Berkeley](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d585503460f94a37f66/html5/thumbnails/31.jpg)
Examples II: Simple Card Game
Bet2 relies on Bet1 even though Bet2 observes Bet1
– Bet2 can depend on Deal
– Deal influences U
– Need probability model of Bet2 to derive posterior on Deal and compute expectation over U
Bet1
Bet2
Decision D can require D’ even if D’ is observed at D !
Bet1
Bet2
Deal
U
![Page 32: Structured Models for Multi-Agent Interactions Daphne Koller Stanford University Joint work with Brian Milch, U.C. Berkeley](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d585503460f94a37f66/html5/thumbnails/32.jpg)
Examples III: Decoupled Utilities
Bet2 relies on Bet1 even without influence on utility
– Bet2 can depend on Deal
– Deal influences U
– Need probability model of Bet2 to derive posterior on Deal and compute expectation over U
Bet1
Bet2
Bet1
Bet2
Deal
U U
![Page 33: Structured Models for Multi-Agent Interactions Daphne Koller Stanford University Joint work with Brian Milch, U.C. Berkeley](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d585503460f94a37f66/html5/thumbnails/33.jpg)
Examples IV: Tree Killer
PoisonTree
BuildPatio
TreeDoctor
TreeSick
TreeDoctor
View
Cost
PoisonTree
BuildPatio
TreeDead
Tree
![Page 34: Structured Models for Multi-Agent Interactions Daphne Koller Stanford University Joint work with Brian Milch, U.C. Berkeley](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d585503460f94a37f66/html5/thumbnails/34.jpg)
s-Reachability
• D’ is s-reachable from D if there is some among the descendants of D, such that if a new parent were added to D, there would be an active path from to U given D and Pa(D).
D
U
D
U
given
existsCPD of D’ influences P(U | D,Pa(D))
D relies on D’(D’ relevant to D)
D’
'̂D
'̂D'̂D
![Page 35: Structured Models for Multi-Agent Interactions Daphne Koller Stanford University Joint work with Brian Milch, U.C. Berkeley](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d585503460f94a37f66/html5/thumbnails/35.jpg)
s-Reachability
Theorem: s-reachability is sound & complete for strategic relevance
Nodes that D relies on are the nodes that are s-reachable from D.
• Sound: no s-reachability strategic irrelevance P,U• Complete: s-reachability relevance for some P,U
Theorem: Can build the relevance graph in quadratic time using only structure of MAID
![Page 36: Structured Models for Multi-Agent Interactions Daphne Koller Stanford University Joint work with Brian Milch, U.C. Berkeley](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d585503460f94a37f66/html5/thumbnails/36.jpg)
Outline
Probabilistic Reasoning: Bayesian networks
Influence Diagrams
Strategic Relevance
Exploiting Structure for Solving Games
![Page 37: Structured Models for Multi-Agent Interactions Daphne Koller Stanford University Joint work with Brian Milch, U.C. Berkeley](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d585503460f94a37f66/html5/thumbnails/37.jpg)
Intuition: Backward Induction• D’ observes D• Can optimize decision rule at D’
without knowing decision rule at D• Having optimized D, can optimize D’
• D doesn’t care about D’ • Can optimize decision rule at D
without knowing decision rule at D’ • Having optimized D’ , can optimize
D
D
D’
D
D’
U
U
D
D’
D
D’
U
![Page 38: Structured Models for Multi-Agent Interactions Daphne Koller Stanford University Joint work with Brian Milch, U.C. Berkeley](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d585503460f94a37f66/html5/thumbnails/38.jpg)
Generalized Backward InductionIdea:Idea: Solve decisions by order of relevance graph Solve decisions by order of relevance graph
Generalized Backward Induction:• Choose decision node D that relies on no other• Find optimal strategy for D by maximizing its
local expected utility• Replace D by chance node
D
D’
D
D’
U
UD
D’
D
D’
U
![Page 39: Structured Models for Multi-Agent Interactions Daphne Koller Stanford University Joint work with Brian Milch, U.C. Berkeley](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d585503460f94a37f66/html5/thumbnails/39.jpg)
Finding Equilibria: Acyclic Relevance Graphs
• Choose any strategy profile for D1,…,Dn-1
• Derive decision rule for Dn that is optimal for • Node Dn does not rely on preceding ones
is optimal for any other strategy profile as well!
D1 D2 DnDn-1…D1 D2 Dn-1Dn-1
• We can now set as CPD for Dn
• And continue by optimizing Dn-1
Dn
![Page 40: Structured Models for Multi-Agent Interactions Daphne Koller Stanford University Joint work with Brian Milch, U.C. Berkeley](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d585503460f94a37f66/html5/thumbnails/40.jpg)
Generalized Backward Induction
Theorem: If the relevance graph of a MAID is acyclic, it can be solved by generalized backward induction, and the result is a pure-strategy Nash equilibrium
• Given topological sort D1,…,Dn of relevance graph:
• Begin with arbitrary fully mixed strategy profile • For i = n down to 1:
– Find decision rule for Di that is optimal for • Decision rules at previous decisions fixed earlier
• Decision rules at subsequent decisions irrelevant
– Let (Di) =
![Page 41: Structured Models for Multi-Agent Interactions Daphne Koller Stanford University Joint work with Brian Milch, U.C. Berkeley](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d585503460f94a37f66/html5/thumbnails/41.jpg)
When is the Relevance Graph Acyclic?
• Single-agent influence diagrams with perfect recall
• Multi-agent games with perfect information
• Some games with imperfect information– e.g., Tree Killer example
But in many MAIDs the relevance graph has cycles…
![Page 42: Structured Models for Multi-Agent Interactions Daphne Koller Stanford University Joint work with Brian Milch, U.C. Berkeley](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d585503460f94a37f66/html5/thumbnails/42.jpg)
Cyclic Relevance GraphsQuestion:Question: What if the relevance graph is cyclic? What if the relevance graph is cyclic?
• Strongly connected component (SCC): – maximal subgraph s.t. directed path between every
pair of nodes
• The decisions in each SCC require each other– They must be optimized together
• Different SCCs can be solved separately
![Page 43: Structured Models for Multi-Agent Interactions Daphne Koller Stanford University Joint work with Brian Milch, U.C. Berkeley](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d585503460f94a37f66/html5/thumbnails/43.jpg)
Generalized Backward InductionGiven topological sort C1,…,Cm of SCCs in relevance
graph:• Begin with arbitrary fully mixed strategy profile • For i = m down to 1:
– Construct reduced MAID M[-Ci]
• Strategies for previous SCCs selected before• Strategies for subsequent SCCs irrelevant
– Create game tree for M[-Ci]
– Use game solver to find equilibrium strategy profile for Ci in this reduced game
– Let (Ci) = Theorem: If find equilibrium for each SCC, the result
is equilibrium for whole game
![Page 44: Structured Models for Multi-Agent Interactions Daphne Koller Stanford University Joint work with Brian Milch, U.C. Berkeley](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d585503460f94a37f66/html5/thumbnails/44.jpg)
“Road” Relevance Graph
1W 1E
2W 2E
3W 3E
Note: Reduced games over SCCs are not subgames!
![Page 45: Structured Models for Multi-Agent Interactions Daphne Koller Stanford University Joint work with Brian Milch, U.C. Berkeley](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d585503460f94a37f66/html5/thumbnails/45.jpg)
Experiment: “Road” Example
Reminder, for n=4: Tree size: 6561 nodes Matrix size: 4.71027
Running Time of Backward Induction Algorithm
0
100
200
300
400
500
600
0 10 20 30 40
Number of Plots of Land
Tim
e (s
eco
nd
s)
For n=40: Tree size: 1.47 1038 nodes
![Page 46: Structured Models for Multi-Agent Interactions Daphne Koller Stanford University Joint work with Brian Milch, U.C. Berkeley](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d585503460f94a37f66/html5/thumbnails/46.jpg)
Cutting Cycles
• Idea: enumerate possible values d for some decision D– Once we determine D, residual MAID has acyclic relevance graph– Solve residual MAID using generalized backward induction– Check whether combined strategy with d is an equilibrium
Theorem: Can find all pure strategy equilibria in time linear in # of SCCs, exponential in max # of decisions required to cut all loops in component
• May need to instantiate several decision nodes to cut cycle• Can deal with each SCC separately
D
![Page 47: Structured Models for Multi-Agent Interactions Daphne Koller Stanford University Joint work with Brian Milch, U.C. Berkeley](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d585503460f94a37f66/html5/thumbnails/47.jpg)
Irrelevant Information
Sales-A
B Sales
Cost
Commission
Resource
Sales-B
A Sales
Commission
Revenue
What if B can observe A’s decisioncompletely irrelevant to him
• We can automatically– analyze relevance based on graph structure – eliminate irrelevant information edges
• In associated tree, safe merging of information sets• Leads to exponential decrease in # of decisions to
optimize in influence diagram!
![Page 48: Structured Models for Multi-Agent Interactions Daphne Koller Stanford University Joint work with Brian Milch, U.C. Berkeley](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d585503460f94a37f66/html5/thumbnails/48.jpg)
Related Work
• Suryadi and Gmytrasiewicz (1999) use multi-agent influence diagrams, but with recursive modeling
• Milch and Koller (2000) use the MAID representation described here, but have no algorithm for finding equilibria
• Nilsson and Lauritzen (2000) discuss limited memory influence diagrams (LIMIDs) and derive s-reachability, but do not apply it to multi-agent case
• La Mura (2000) proposes game networks, with an undirected notion of strategic dependence
![Page 49: Structured Models for Multi-Agent Interactions Daphne Koller Stanford University Joint work with Brian Milch, U.C. Berkeley](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d585503460f94a37f66/html5/thumbnails/49.jpg)
Future Work
• Take advantage of structure within SCCs
• Represent asymmetric scenarios compactly
• Detect irrelevant observations
![Page 50: Structured Models for Multi-Agent Interactions Daphne Koller Stanford University Joint work with Brian Milch, U.C. Berkeley](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d585503460f94a37f66/html5/thumbnails/50.jpg)
Computational Game Theory
• Expert analysis of:– “Prototypical” examples
that highlight key issues – Abstracted problems for
big organizations
• Autonomous agents interacting economically
• Decision support systems for consumers
• Complex problems:– many relevant variables– interacting decisions
• Simplified examples– small enough to be
analyzed by hand
Game theory: Past Game theory: Future
Goals: Make game theory• a broadly usable tool even for lay people• a formal basis for interacting autonomous agentsby allowing real-world games to be easily representedand solved.
![Page 51: Structured Models for Multi-Agent Interactions Daphne Koller Stanford University Joint work with Brian Milch, U.C. Berkeley](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d585503460f94a37f66/html5/thumbnails/51.jpg)
Conclusions• Multi-agent influence diagrams:
– compact intuitive language for multi-agent interactions– basic units: variables rather than strategies or events
• MAIDs make explicit structure that is lost in game trees
• Can exploit structure to find equilibria efficiently – sometimes exponentially faster than existing algorithms
• Exciting question: What else does structure buy us?
![Page 52: Structured Models for Multi-Agent Interactions Daphne Koller Stanford University Joint work with Brian Milch, U.C. Berkeley](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d585503460f94a37f66/html5/thumbnails/52.jpg)
http://robotics.stanford.edu/~koller