Nash Equilibria In Graphical Games On Trees

Edith Elkind

Leslie Ann Goldberg

Paul Goldberg

Games and Strategies

• Games: strategic interactions between rational entities

• Solution concepts: what’s going to happen?– dominant strategies– Nash equilibrium– ….

• Can it be computed?– if your computer cannot find it, the market

probably cannot either

Matrix (normal form) Games

2 0

0 1

1 0

0 3

Row player:

Column player:

0

1

0 1 0 1

0

1

• finite set of players {1, …, n}

• each player has k actions

(pure strategies): 1, …, k

• payoffs of the ith player: Pi: {1, …, k}n → R

Nash Equilibrium

2 0

0 1

1 0

0 3

Row player:

Column player:

0

1

0 1 0 1

0

1

• Nash equilibrium: a strategy profile such that

noone wants to deviate given other players’ strategies, i.e., each player’s strategy is a best response to others’ strategies:– (0, 0) and (1, 1) are both NE

Pure vs. Mixed Strategies

1 -1

-1 1

-1 1

1 -1

Row player:

Column player:

H

T

H T H T

H

T

• NE in pure strategies may not exist!– “matching pennies”

• Mixed strategy: a probability distribution over actions– 50% tail, 50% head

Existence of NE

• Theorem (Nash 1951): any n-player k-action game in normal form has an equilibrium in mixed strategies

can we find one in poly-time?

2 players, n actions

• Representation: two n x n matrices• Computation:

– all known methods are exptime– can it be NP-hard? no: NE always exists– PPAD-hardness: notion of hardness for total search

problems– DGP’06: finding NE in 4-player games is PPAD-hard– CD’06: finding NE in 2-player games is PPAD-hard– DGP reduction uses graphical games

(the topic of this talk!)

n-player 2-action games

• representation: payoffs to each player for every action profile (vector in {0, 1}n): n2n numbers

• graphical games:– players are vertices of a graph– V’s payoff depends on actions of W in N(V) U V– n players, max degree d => n2d+1 numbers

TU

V

W t=0, u=0, v=0, w=0: 12t=1, u=0, v=0, w=0: 31 ….t=1, u=1, v=1, w=1: -6

W’s payoffs(16 cases):

Complexity: what is known

• Bounded-degree trees:– Exp-time algorithm/poly-time approximation

algorithm to find all NE (Kearns, Littman, Singh, UAI 2001)

– ??? poly-time algorithm to find a single NE (Kearns, Littman, Singh, NIPS’2001)

• Heuristics for graphs with cycles

• General graphs:– PPAD-complete (DGP’06) even if max deg=3

Our Results (1)

• Algorithm in NIPS’01 paper is incorrect (does not always output a NE)

• We fix the NIPS’01 algorithm, but…– our algorithm runs in poly-time on paths– with a trick, also on cycles– can be used to find

• (a representation for) all NE in n3 time, or• a single NE in n2 time

Our Results (2)

• There is a graph of pathwidth 2 on which our algorithm runs in exp time– true for all algorithms that use the basic

approach of the UAI’01 paper

• The problem is PPAD-complete for bounded pathwidth graphs

• Open question: what if pathwidth = 1?– generalizes a cool geometry problem (talk to

me if you like those, or see the paper)

Warm-up: 2-player 2-action games

2 0

0 1

1 0

0 3

Row player:

Column player:

0

1

0 1 0 1

0

1

Suppose R plays 1 w.p. r

EP(C) from playing 0: (1-r)*1

EP(C) from playing 1: r*3

1-r > 3r iff r < ¼

Suppose C plays 1 w.p. c

EP(R) from playing 0: (1-c)*2

EP(R) from playing 1: c*1

(1-c)*2 > c iff c < 2/3

1/4 1

r

BR(C)c1

2/3BR(R)

mixed NE: r=1/4, c=2/3

• Potential best response: v is a PBR to w iff when W plays w, there is a NE for TV in which V plays v.

• upstream pass: construct PBRV(w) from PBRU1(v), PBRU2(v) and PBRU3(v)

• downstream pass: root selects its strategy based on the children’s PBR’s; propagates to leaves

Algorithm for Trees (KLS’01)

TV

W

V

U1U2

U3

v

w

KLS algorithm: running time

• For bounded-degree trees, constructs all PBR (and then find a NE) in exp time

• FPTAS for an -NE:– superimpose PBR with a -grid– there exists a grid point -close to PBR– -NE ( = poly() ):

no one can gain more than by deviating

Computing PBR: Example

• Payoffs to V: – P000 = 1, P001 = -9, P100 = 9, P101 = -1, Pu1w = 0 for u, w =0, 1

• E0 = EP(V) from playing 0: (1-u)(1-w)*1+(1-u)w*(-9)+u(1-w)*9+uw*(-1) = 1+8u-10w

• E1 = EP(V) from playing 1: 0• E0 = E1 iff w = (8u+1)/10 = f(u)

U V W

.5 1

1u

v

v

1

1

.5

.1 .9 w

(v, u) → (f(u), v)

PBRU(v) PBRV(w)

Trees: too many segments

v v w

ut v

v1 v2 v1 v2

v1

v2

KLS (NIPS’01): can “trim” PBR

Incorrect!

W

V

T U

(v,t), (v,u) → (f(u,t), v)

u2

u1

t2

t1

Solutions?

• Solution 1 (for paths): algorithm of UAI’01 paper, careful analysis– the number of segments/rectangles in each

PBR is O(n2)– running time O(n3)

• Solution 2 (for paths): can pick a subset of each PBR consisting of O(n) segments– O(n2) running time

O(n3) algorithm• f(u) =(au+b)/(cu+d) u*: cu*+d = 0 [v1, v2] x {u} => {f(u)} x [v1, v2]

{v} x [u1, u2] => [f(u1), f(u2)] x {v} if u* not in [u1, u2]

[0, f(u2)] U [f(u1), 1] x {v} if u1≤ u*≤ u2

• PBRV(w) vs PBRU(v): new segments at v=1 and v=0,

some segments break into two --- double in size?

• no: count the event points

u

v

v

w

(v, u) → (f(u), v)

PBRU(v) PBRV(w)

u*

Extension to trees? V0 V1 V2 Vn-1 Vn

U1

T1

Un-1

T2

U2

Tn-1 Tn

Un

Hardness results

• pathwidth 2: our algorithm is not poly-time– and neither is any two-pass algorithm that

stores subsets of PBR

• pathwidth > k: (probably) all algorithms are not poly-time– finding NE in this case is PPAD-hard– idea: modify the construction in DGP’06

Good Nash Equilibria

2 0

0 1

1 0

0 3

Row player:

Column player:

0

1

0 1 0 1

0

1

Nash equilibria: • (0, 0): total payoff is 3• (1, 1): total payoff is 4• (1/4, 2/3): total payoff is 17/12

not all NE are created equal…

What is a good NE?

• maximize sum of player’s payoffs

• guarantee to each player a payoff of at least ti

• (almost) equal payoffs

• any combination of those….

Can we use PBR data structure to compute those?

Can we represent it?

• any GG with integer payoffs on a tree has a rational NE

• Any PBR consists of segments and rectangles with rational coordinates

• Yet, total payoff-maximizing NE may be irrational

Our result (EGG’07): for any algebraic , deg() = n, there is a GG with int payoffs on a path of length O(n) in whichin the best NE player 1 plays

Approximation

• Can we use the FPTAS of KLS’01? – superimpose PBR with a -grid

• Observation: there is a grid point -close to best NE– look for best point on the grid close to PBR

• dynamic programming

– -NE ( = poly() ): no one can gain more than by deviating

True Nash

• -NE is not always appropriate– what if players are not willing to lose ?

• Can we find a (true) NE that is -close to the best (true) NE?

• Idea: – add borders of rectangles

in PBR to the grid– only consider grid points in PBR

Bounded Payoff Nash

• Similar algorithm works --- FPTAS– Also for other kinds of “good” NE

• If all payment bounds are rational, there is a BP NE that is “almost” rational (deg ≤ 2)

• Open question: can we compactly represent all bounded payoff NE?– perhaps by incorporating payoff bounds into

PBR?

Conclusions

Nash equilibria in graphical games on trees

complexity still unknown…