game theory and its applications to networks · game theory and its applications to networks - part...

Game Theory and its Applications toNetworks -

Part II: General Games

Corinne Touati

Master ENS Lyon, Fall 2010

Game in Extensive Form

An n-person extensive form game is a finite tree structure with:

I A vertex indicating the starting point of the game,

I A pay-off function assigning a n-vector to each terminalvertex of the tree,

I A partition of the nodes of the tree into n + 1 sets (with N i

the set of player i),

I A probability distribution, defined at each vertex of S0, amongthe immediate followers of this vertex,

I A subpartition of each player set N i into information sets ηijsuch that all nodes of a information set has the same numberof children and that no node follows another node of the sameinformation set.

Corinne Touati (INRIA) Strict Competition Games in Extensive Form 2 / 17










Equilibrium

Every finite n-person game with perfectinformation has an equilibrium n-tuple ofstrategies










Be careful

A B

Player 2

RL

Player 1

(2, 1)(0, 0)

(1, 2)

L R

A (0, 0) (2, 1)

B (1, 2) (1, 2)

There are 2 Nash equilibria! ((A,R) and (B,L))BUT, only (A,R) is a subgame perfect equilibrium

Surprising Examples


Forward Induction

ConcertBook

1, 30, 0

0, 03, 12, 2

Strategic form:B S

Book (2, 2) (2, 2)B (3, 1) (0, 0)S (0, 0) (1, 3)

Chain-Store Game: short-termversus long term

F

CS k

C

OutIn

k

CS

(5, 1)

(2, 2)(0, 0)

Centripede Game

SSSSS

C C C C C C

S(6, 5)

(3, 1) (4, 6)(5, 3)(0, 2)(1, 0) (2, 4)

Surprising Examples


Forward Induction

Book Concert

1, 30, 0

0, 03, 12, 2

Strategic form:B S

Book (2, 2) (2, 2)B (3, 1) (0, 0)S (0, 0) (1, 3)

Chain-Store Game: short-termversus long term

F

CS k

C

OutIn

k

CS

(5, 1)

(2, 2)(0, 0)

Centripede Game

SSSSS

C C C C C C

S(6, 5)

(3, 1) (4, 6)(5, 3)(0, 2)(1, 0) (2, 4)

Stackelberg Equilibrium

Definition.

A Stackelberg game is a two-player extensive game with perfectinformation in which a ”leader” chooses an action from a set A1and a ”follower”, informed of the leader’s choice, chooses anaction from a set A2. The Stackelberg equilibria are solutions ofthe problem:

max(a1,a2)∈A1×A2

u1(a1, a2) s.t. a2 ∈ argmaxa′2∈A2u2(a1, a′2)

Example:Level 1

Level 2

a b c

y n y n y n

(0, 0) (0, 0) (0, 2) (0, 0)(1, 1)(2, 0)


Game in Normal Form

Definition: Nash Equilibrium.

In a Nash equilibrium, no player has incentive to unilaterallymodify his strategy.

strategy utility

s∗ is a Nash equilibrium iff:

∀p,∀ sp , up(s∗1, . . . , s∗p , . . . s∗n) ≥ up (s∗1, . . . , sp , . . . , s∗n)

In a compact form:∀p,∀sp, up(s∗−p, s∗p) ≥ up(s∗−p, sp)

Corinne Touati (INRIA) Strict Competition Games in Normal Form 5 / 17

Nash Equilibrium: Examples

Find the Nash equilibria of these games (with pure strategies)

The prisoner dilemma

collaborate deny

collaborate (1, 1) (3, 0)deny (0, 3) (2, 2)

⇒ not efficient

Battle of the sexes

Paul / Claire Opera Foot

Opera (2, 1) (0, 0)Foot (0, 0) (1, 2)

⇒ not unique

Rock-Scisor-Paper

1/2 P R S

P (0, 0) (1,−1)(−1, 1)R (−1, 1) (0, 0) (1,−1)S (1,−1)(−1, 1) (0, 0)

⇒ No equilibrium





collaborate deny

collaborate (1, 1) (3, 0)deny (0, 3) (2, 2)⇒ not efficient

Battle of the sexes


Opera (2, 1) (0, 0)Foot (0, 0) (1, 2)

⇒ not unique

Rock-Scisor-Paper

1/2 P R S

P (0, 0) (1,−1)(−1, 1)R (−1, 1) (0, 0) (1,−1)S (1,−1)(−1, 1) (0, 0)

⇒ No equilibrium





collaborate deny


Battle of the sexes


Opera (2, 1) (0, 0)Foot (0, 0) (1, 2)⇒ not unique

Rock-Scisor-Paper

1/2 P R S

P (0, 0) (1,−1)(−1, 1)R (−1, 1) (0, 0) (1,−1)S (1,−1)(−1, 1) (0, 0)

⇒ No equilibrium





collaborate deny


Battle of the sexes


Opera (2, 1) (0, 0)Foot (0, 0) (1, 2)⇒ not unique

Rock-Scisor-Paper

1/2 P R S

P (0, 0) (1,−1)(−1, 1)R (−1, 1) (0, 0) (1,−1)S (1,−1)(−1, 1) (0, 0)⇒ No equilibrium


Mixed Nash Equilibria


Definition: Mixed Strategy Nash Equilibria.

A mixed strategy for player i is a probability distribution over the set ofpure strategies of player i.An equilibrium in mixed strategies is a strategy profile σ∗ of mixedstrategies such that: ∀p,∀σi, up(σ∗−p, σ∗p) ≥ up(σ∗−p, σp).

Theorem 1.

Any finite n-person noncooperative game has at least one equilibriumn-tuple of mixed strategies.

Proof.

Kakutani fixed point theorem: Let S be a non-empty, compact andconvex subset of a Euclidean space. Let f : S → P(S) (the power set ofS) with a closed graph and such that ∀x ∈ S, f(x) is non-empty andconvex. Then f has a fixed point.

Apply Kakutani to f :

{[0, 1]N → P([0, 1]N )σ 7→ ⊗i∈{1,N}Bi(σi)

with Bi(σi)

Mixed Nash Equilibria


Definition: Mixed Strategy Nash Equilibria.

A mixed strategy for player i is a probability distribution over the set ofpure strategies of player i.An equilibrium in mixed strategies is a strategy profile σ∗ of mixedstrategies such that: ∀p,∀σi, up(σ∗−p, σ∗p) ≥ up(σ∗−p, σp).

Theorem 1.

Any finite n-person noncooperative game has at least one equilibriumn-tuple of mixed strategies.

Proof.

Kakutani fixed point theorem: Let S be a non-empty, compact andconvex subset of a Euclidean space. Let f : S → P(S) (the power set ofS) with a closed graph and such that ∀x ∈ S, f(x) is non-empty andconvex. Then f has a fixed point.

Apply Kakutani to f :

{[0, 1]N → P([0, 1]N )σ 7→ ⊗i∈{1,N}Bi(σi)

with Bi(σi)

Consequence:

I The players mixed strategies are independant randomizations.

I In a finite game, up(σ) =∑a

∏p′

σp′(ap′)

ui(a).I Function ui is multilinear

I In a finite game, σ∗ is a Nash equilibrium iff ∀ai in the supportof σi, ai is a best response to σ

∗−i

Mixed Nash Equilibria: Examples

Find the Nash equilibria of these games (with mixed strategies)


collaborate deny


⇒ No strictly mixed equilibria

Battle of the sexes


Opera (2, 1) (0, 0)Foot (0, 0) (1, 2)

σ1 = (2/3, 1/3), σ2 = (1/3, 2/3)

Rock-Scisor-Paper

1/2 P R S

P (0, 0) (1,−1)(−1, 1)R (−1, 1) (0, 0) (1,−1)S (1,−1)(−1, 1) (0, 0)

σ1 = σ2 = (1/3, 1/3, 1/3)


Mixed Nash Equilibria: Examples

Find the Nash equilibria of these games (with mixed strategies)


collaborate deny


⇒ No strictly mixed equilibria

Battle of the sexes


Opera (2, 1) (0, 0)Foot (0, 0) (1, 2)

σ1 = (2/3, 1/3), σ2 = (1/3, 2/3)

Rock-Scisor-Paper

1/2 P R S

P (0, 0) (1,−1)(−1, 1)R (−1, 1) (0, 0) (1,−1)S (1,−1)(−1, 1) (0, 0)

σ1 = σ2 = (1/3, 1/3, 1/3)Corinne Touati (INRIA) Strict Competition Games in Normal Form 8 / 17

Nash Equilibrium with Chance Move

Corinne Touati (INRIA) Strict Competition Chance Moves 9 / 17

Nature take decision w1 or w2 with probability 1/2.(w1) a b

a (0, 0) (6,−3)b (−3, 6) (5, 5)

(w2) a b

a (−20,−20) (−7,−16)b (−16,−7) (−5,−5)

Nash equilibrium if:

I aucun des joueurs ne connâıtl’état:

EN: (b, b), utilité :(0, 0)

I les deux joueurs sont informés:

EN: ((a, a)|w1), ((b, b)|w2),utilité: (−2.5,−2.5)

I Seul le joueur 1 est au courant:

EN: ((a, a)|w1) , ((b, a)|w2),utilité (−8,−3, 5)

Nature

(0, 0) (−5,−5)(−16, 7)(−7,−16)(−20,−20)(5, 5)(−3, 6)(6,−3)

⇒Information can bedetrimental




a (0, 0) (6,−3)b (−3, 6) (5, 5)

(w2) a b

a (−20,−20) (−7,−16)b (−16,−7) (−5,−5)


I aucun des joueurs ne connâıtl’état:EN: (b, b), utilité :(0, 0)





Nature

(0, 0) (−5,−5)(−16, 7)(−7,−16)(−20,−20)(5, 5)(−3, 6)(6,−3)





a (0, 0) (6,−3)b (−3, 6) (5, 5)

(w2) a b

a (−20,−20) (−7,−16)b (−16,−7) (−5,−5)







Nature

(−5,−5)(0, 0) (6,−3) (−3, 6) (5, 5) (−20,−20)(−7,−16)(−16, 7)





a (0, 0) (6,−3)b (−3, 6) (5, 5)

(w2) a b

a (−20,−20) (−7,−16)b (−16,−7) (−5,−5)



I les deux joueurs sont informés:EN: ((a, a)|w1), ((b, b)|w2),utilité: (−2.5,−2.5)



Nature

(−5,−5)(0, 0) (6,−3) (−3, 6) (5, 5) (−20,−20)(−7,−16)(−16, 7)





a (0, 0) (6,−3)b (−3, 6) (5, 5)

(w2) a b

a (−20,−20) (−7,−16)b (−16,−7) (−5,−5)



I les deux joueurs sont informés:EN: ((a, a)|w1), ((b, b)|w2),utilité: (−2.5,−2.5)

I Seul le joueur 1 est au courant:EN: ((a, a)|w1) , ((b, a)|w2),utilité (−8,−3, 5)

Nature

(−5,−5)(0, 0) (6,−3) (−3, 6) (5, 5) (−20,−20)(−7,−16)(−16, 7)





a (0, 0) (6,−3)b (−3, 6) (5, 5)

(w2) a b

a (−20,−20) (−7,−16)b (−16,−7) (−5,−5)

Definition: Bayesian Games.

A Bayesian game consists of:

I (N,A,U), the sets of players, actions and associated utilities

I Ω a set states of natureI For each player:

I A probability measure pi on Ω: a priori belief about the state of nature.I A set of signals TiI A function τi : Ω→ Ti (partial observation)

Then, the posterior belief of player i is

0 if ω 6∈ τ−1i (ti)pi(ω)

pi(τ−1i (ti))

else.

Correlated Equilibria

Definition: Correlated Equilibria.

A correlated equilibrium for the bimatrix game A is a correlatedstrategy P s.t.

∀i, ∀k,∑j

aijpij ≥∑j

akjpij

∀i, ∀`,∑i

bijpij ≥∑i

bi`pij

Proposition:

Let (aij , bij){1..N},{1..M} a game. Consider a zero-sum matrixgame C with entries:

c(i,j),(h,k) =

{aij − akj if i = h ∈M,k ∈Mbij − bik if j = h ∈ N, k ∈ N

Then, an optimal mixed strategy for player I of this game is acorrelated equilibrium for the original game.

Corinne Touati (INRIA) Strict Competition Correlated Equilibria 10 / 17




∀i, ∀k,∑j

aijpij ≥∑j

akjpij

∀i, ∀`,∑i

bijpij ≥∑i

bi`pij

Proposition:


c(i,j),(h,k) =


Then, an optimal mixed strategy for player I of this game is acorrelated equilibrium for the original game.

Corinne Touati (INRIA) Strict Competition Correlated Equilibria 10 / 17

Example

Game

((−10,−10) (5, 0)

(0, 5) (−1,−1)

).

Two pure Nash equilibria (1, 0), (0, 1) and (0, 1), (1, 0).One mixed equilibria (1/16, 5/16), (1/16, 5/16) with an expectedpayoff −5/8.

The correlated equilibria are or the form P =

(p11 p12p21 p22

). with

p11 + p12 + p21 + p22 = 1, max(2.5p11, 0.4p22) ≥ min(p12, p21).

For instance,

(0 0.50.5 0

)gives an expected payoff 2.5.




∀i, ∀k,∑j

aijpij ≥∑j

akjpij

∀i, ∀`,∑i

bijpij ≥∑i

bi`pij

Proposition:


c(i,j),(h,k) =


Then, an optimal mixed strategy for player I of this game is acorrelated equilibrium for the original game.Corinne Touati (INRIA) Strict Competition Correlated Equilibria 10 / 17




∀i, ∀k,∑j

aijpij ≥∑j

akjpij

∀i, ∀`,∑i

bijpij ≥∑i

bi`pij

Proposition:


c(i,j),(h,k) =


Then, an optimal mixed strategy for player I of this game is acorrelated equilibrium for the original game.Corinne Touati (INRIA) Strict Competition Correlated Equilibria 10 / 17

Example

Game

((−10,−10) (5, 0)

(0, 5) (−1,−1)

).

Two pure Nash equilibria (1, 0), (0, 1) and (0, 1), (1, 0).One mixed equilibria (1/16, 5/16), (1/16, 5/16) with an expectedpayoff −5/8.

The correlated equilibria are or the form P =

(p11 p12p21 p22

). with

p11 + p12 + p21 + p22 = 1, max(2.5p11, 0.4p22) ≥ min(p12, p21).

For instance,

(0 0.50.5 0

)gives an expected payoff 2.5.

Matrix C =

−10 0 0 −104 0 10 00 10 0 40 −4 −4 0

Fictitious Plays Fail

Example

1/2 P R S

P (0, 0) (1,−1)(−1, 1)R (−1, 1) (0, 0) (1,−1)S (1,−1)(−1, 1) (0, 0)

1

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

0 0.2 0.4 0.6 0.8 0

Corinne Touati (INRIA) Strict Competition Computing Nash Equilibria 11 / 17

Is learning Nash Equilibria simple?

⇒ A natural approach is using Best Response dynamics

This does not always converge to a Nash equilibrium:(17, 8) (4, 9) (3, 4)

(10, 10) (6, 5) (2, 4)

(8, 0) (1, 2) (5,5)

Known to converge in some class of games (for instance, potentialgames)

Corinne Touati (INRIA) Strict Competition Computing Nash Equilibria 12 / 17

The Flow Control Problem

Imagine a system with:

I n individual users aiming at optimizing their throughput xnI A routing matrix A giving the set of paths followed by each

connection: Ai,j =

{1 if connection iuses link j0 otherwise

I Capacity constraints on each link C`

I What is the Nash equilibrium of the game? What protocoldoes it corresponds to?

I How can we implement fairness in a distributed way?

Corinne Touati (INRIA) Strict CompetitionThe General Flow Control

Problem 13 / 17

The Flow Control Problem:The Non Cooperative Game

Example: A simple network with 3 links

(namSimple.mpeg)

n0→2 = 2, n1→2 = 3, n2→3 = 4Throughput of flow i:

λi.capa

λ1 + λ2


Problem 14 / 17

namSimple.mpegMedia File (video/mpeg)


Linksequations:

λ′2 =λ2C

λ2 + λ′1λ′′2 =

λ′2C

λ3 + λ′2= C−λ′3

Throughputreceived byflow 2:

λ′′2 = C−λ3C

λ3 +C

1+λ′1/λ2

=C2

C + λ3(1 + λ′1/λ′2)

1

4

3

2

λ3λ′′1

λ2

λ′1λ1

λ′′2

λ′′2

λ′3

λ′′1

λ′2


Problem 15 / 17


(namUDP.mpeg)

Ring topology network, Nidentical links with capacity CSource i uses links i and i+ 1(mod N), hypothesis C >> λ

Exit throughput of flow i:

λ′′ =C2

C + λ(1 + λ′/λ′), and

λ′

λ=

1

2

(√1 +

4C

λ− 1

)∼ Cλ

Then λ′′ ∼ C2

λ


Problem 16 / 17

namCercleUDP.mpegMedia File (video/mpeg)


This is network collapse:I The network is fullI Little or no useful information is going through (here

λ′′ ∼ C2

λ→λ→∞ 0)

Observed in 1984 (cf RFC 896) with TCP flows: the protocoldetects a loss, so it retransmits the packet, hence increasing itsincoming throughput...

Since then a flow control mechanism has been impremented inTCP ,

Why hadn’t we observe this kind of phenomena before withtelephony?Corinne Touati (INRIA) Strict Competition

The General Flow ControlProblem 17 / 17

Games in Extensive FormGames in Normal FormChance MovesCorrelated EquilibriaComputing Nash EquilibriaThe General Flow Control Problem

game theory and its applications to networks · game theory and its applications to networks - part...

Documents