price of anarchy georgios piliouras. games (i.e. multi-body interactions) interacting entities...
TRANSCRIPT
Price of Anarchy
Georgios Piliouras
Games (i.e. Multi-Body Interactions)
• Interacting entities• Pursuing their own goals• Lack of centralized control
Prediction?
Games
n players Set of strategies Si for each player i Possible states (strategy profiles) S=×Si Utility ui:S→R Social Welfare Q:S→R Extend to allow probabilities Δ(Si), Δ(S) ui(Δ(S))=E(ui(S)) Q(Δ(S))=E(Q(S))
(review)
Zero-Sum Games & Equilibria
0, 0 -1, 1 1, -11, -1 0, 0 -1, 1-1, 1 1, -1 0, 0
RockRock
Paper
Paper
Scissors
Scissors
Nash: A product of mixed strategies s.t. no player has a profitable deviating strategy.
1/3 1/3 1/3
1/31/31/3
Existence, Uniqueness of Payoffs [von Neumann 1928](review)
General Games & Equilibria?
1, 0 -1, 1 1, -11, -1 0, 0 -1, 1-1, 1 1, -1 0, 0
RockRock
Paper
Paper
Scissors
Scissors
Nash: A product of mixed strategies s.t. no player has a profitable deviating strategy.
Borel conjectured the non-existence of eq. in general
Prediction in GamesIdea 1
Nash Equilibrium (1950): A strategy tuple (i.e. one for each agent) s.t. no agent can deviate profitably.
For finite games, it always exists when we allow agents to randomize
Proof on the board
Equilibria & Prediction
0, 0 -1, 1 1, -11, -1 0, 0 -1, 1-1, 1 1, -1 0, 0
NE 0, 0 -1, 1 1, -11, -1 0, 0 -1, 1-1, 1 1, -1 0, 0
Implicit assumptions The players all will do their utmost to maximize
their expected payoff as described by the game. The players are flawless in execution. The players
have sufficient intelligence to deduce the solution. The players know (can compute) the planned
equilibrium strategy of all of the other players. The players believe that a deviation in their own
strategy will not cause deviations by any other players. There is common knowledge that all players meet these conditions, including this one.
Uniqueness
Games & Equilibria
0, 0 -1, 1 1, -11, -1 0, 0 -1, 1-1, 1 1, -1 0, 0
RockRock
Paper
Paper
Scissors
Scissors
Nash: A product of mixed strategies s.t. no player has a profitable deviating strategy.
1/3 1/3 1/3
1/31/31/3
Equilibria & Prediction
20, 20 0, 11, 0 1, 1
StagStag
Hare
Hare
Multiple Nash:Which one to choose?
Prediction in GamesIdea 2a
Koutsoupias and Papadimitriou (1999) If there exist several Nash Equilibria, then be
pessimistic and output the worst possible one. (worst case analysis)
Worst in terms of what? Social Welfare Examples of Social Welfare: Sum of utilities, maxmin utility, median utility
Metrics of Social WelfareExamples
Sum of latencies (sec)maxmin utility ($)
Throughput bottleneck (bit/sec)
Prediction in GamesIdea 2b
Koutsoupias and Papadimitriou (1999) If there exist several Nash Equilibria,
then be pessimistic and output the worst possible one. (worst case analysis)
Normalization
Social Cost (worst Equilibrium)
Social Cost (OPT)Price of Anarchy =
≥ 1
PoA = ≥ 1
Social Cost (worst Equilibrium)
Social Cost (OPT)
x10
10
0A
B
C
D
10 agentsA D
PoA = 4/3
PoA ≤ 5/2, for all networks
delay (x) = x
[Koutouspias, Christodoulou 05]
Price of Anarchy
Equilibria & Prediction
0, 0 -1, 1 1, -11, -1 0, 0 -1, 1-1, 1 1, -1 0, 0
NE 0, 0 -1, 1 1, -11, -1 0, 0 -1, 1-1, 1 1, -1 0, 0
NE PoA
Advantages of PoA Approach
Simplicity Widely Applicable (conditions?) Allows for cross-domain comparisons (e.g.
routing game vs facility location game) Analytically tractable?
Several variants: Price of Stability, Price of Total Anarchy, Price of X,…
YES, 1000+ citations
BREAK
Q: Any other ways to make predictions in multi-body problems? How do you do it in real life situations?
Recap + Plan• Games + Worst Case Analysis +
Normalization
• PoA =
• To do: – PoA Analysis (when welfare = sum utility)– Beyond Nash equilibria
Social Cost (worst Equilibrium)
Social Cost (OPT)
PoA
Congestion Games• n players and m resources (“edges”)• Each strategy corresponds to a set of
resources (“paths”)• Each edge has a cost function ce(x) that
determines the cost as a function on the # of players using it.
• Cost experienced by a player = sum of edge costs x xx x
2x 2xx x
Cost(red)=6
Cost(green)=8
Potential Games• A potential game is a game that exhibits a
function Φ: S→R s.t. for every s ∈ S and every agent i,
ui (si,s-i) - ui (s) = Φ (si,s-i) - Φ (s) • Every congestion game is a potential game:
Why?• This implies that any such game has a pure
NE. Why?
PoA ≤ 5/2 for linear latencies[Koutouspias, Christodoulou 05], [Roughgarden 09]
Definition: A game is (λ,μ)-smooth if i Ci(s*
i,s-i) ≤ λcost(s*) + μ cost(s) for all s,s*
Then: POA (of pure Nash eq) ≤ λ/(1-μ)
Proof: Let s arbitrary Nash eq.cost(s) = i Ci(s) [definition of social cost]
≤ i Ci(s*i,s-i) [s a Nash eq]
≤ λcost(s*) + μ cost(s) [(λ,μ)-smooth]
PoA ≤ 5/2 for linear latencies[Koutouspias, Christodoulou 05], [Roughgarden 09]
Technical lemma: A linear congestion game is (5/3,1/3)-smooth.
Proof :Step 0: Matlab simulations to get a hint about
what is the best possible (λ,μ) s.t. game is (λ,μ)-smooth.
Step 1: Verify hypothesis (On the board).
Tight Example
• N agents, • 2N elements (x1, x2,…, xN) (y1, y2,…, yN)
c(x)=x for all of them• Each agent i has 2 strategies : (xi ,yi) or
(xi, yi-1, yi+1)
…
x1
…
y1xN yNx2 y2
BREAK 2
Q: What about PoA of mixed NE?
Recap + Plan• Games + Worst Case Analysis +
Normalization
• PoA =
• To do: – PoA Analysis (when welfare = sum utility)– Beyond Nash equilibria
Social Cost (worst Equilibrium)
Social Cost (OPT)
PoA
0, 0 -1, 1 1, -11, -1 0, 0 -1, 1-1, 1 1, -1 0, 0
Rock Paper Scissors
RockPaperScissors
1/3
1/3
1/3
1/3 1/3 1/3
Other Equilibrium Notions
Nash: A product of mixed strategies s.t. no player has a profitable deviating strategy.
0, 0 -1, 1 1, -11, -1 0, 0 -1, 1-1, 1 1, -1 0, 0
Nash: A probability distribution over outcomes, that is a product of mixed strategiess.t. no player has a profitable deviating strategy.
Choose any of the green outcomes uniformly (prob. 1/9)
Rock Paper Scissors
RockPaperScissors
1/3
1/3
1/3
1/3 1/3 1/3
Other Equilibrium Notions
0, 0 -1, 1 1, -11, -1 0, 0 -1, 1-1, 1 1, -1 0, 0
Nash: A probability distribution over outcomes,
s.t. no player has a profitable deviating strategy.
Rock Paper Scissors
RockPaperScissors
1/3
1/3
1/3
1/3 1/3 1/3
Coarse Correlated Equilibria (CCE):
Other Equilibrium Notions
A probability distribution over outcomes, s.t. no player has a profitable deviating strategy.
Rock Paper Scissors
RockPaperScissors
Coarse Correlated Equilibria (CCE):
0, 0 -1, 1 1, -11, -1 0, 0 -1, 1-1, 1 1, -1 0, 0
Other Equilibrium Notions
A probability distribution over outcomes, s.t. no player has a profitable deviating strategy.
Rock Paper Scissors
RockPaperScissors
Coarse Correlated Equilibria (CCE):
0, 0 -1, 1 1, -11, -1 0, 0 -1, 1-1, 1 1, -1 0, 0
Choose any of the green outcomes uniformly (prob. 1/6)
Other Equilibrium Notions
A probability distribution over outcomes, s.t. no player has a profitable deviating strategyeven if he can condition the advice from the dist..
Rock Paper Scissors
RockPaperScissors
Correlated Equilibria (CE):
0, 0 -1, 1 1, -11, -1 0, 0 -1, 1-1, 1 1, -1 0, 0
Choose any of the green outcomes uniformly (prob. 1/6)
Other Equilibrium Notions
Is this a CE? NO
Other Equilibrium Notions
Pure NE NE CE CCE
Smoothness bounds extend to CCE
Definition: A game is (λ,μ)-smooth if i Ci(s*
i,s-i) ≤ λcost(s*) + μ cost(s)for all s,s*
Then: POA (of pure Nash eq) ≤ λ/(1-μ)
Proof: Let s arbitrary Nash eq.cost(s) = i Ci(s) [definition of social cost]
≤ i Ci(s*i,s-i) [s a Nash eq]
≤ λcost(s*) + μ cost(s) [(λ,μ)-smooth]
Smoothness bounds extend to CCE
Definition: A game is (λ,μ)-smooth ifi Ci(s*
i,s-i) ≤ λcost(s*) + μ cost(s)for all s,s*
Then: POA (of pure Nash eq) ≤ λ/(1-μ)
Proof: Let s arbitrary CCE.E[cost(s)] = E[i Ci(s)] [definition of social cost]
≤ E[ i Ci(s*i,s-i)] [s a CCE]
≤ λ E[cost(s*)] + μ E[cost(s)]
Criticism of PoA Analysis
• What happens in we add 10^10 to the utilities of each agent?
• Tightness is achieved over classes.
• Holds only for sum of utilities
• Sensitive to noise
Open Questions
• Choose your favorite class of games. Attempt (λ,μ)-smoothness analysis.–Possible problems• Technique gives trivial upper bounds• Still need to identify lower bounds
• What about uncertainty?
• Other (hidden) assumptions?
[Balcan,Blum,Mansur’09] [Balcan,Constantin,Ehrlich‘11]
Recap
• Nash always exists (fixed point) but not unique
• PoA addresses non-uniqueness
• (λ,μ)-smoothness general technique for proving PoA bounds, extends to other notions, and can provide tight bounds
Thank You