The Computational Complexityof Finding Nash Equilibria

Edith ElkindIntelligence, Agents, Multimedia group (IAM)

School of Electronics and CS

U. of Southampton

Games and Strategies

• Games: strategic interactions between rational entities

• Solution concepts: what’s going to happen?– dominant strategies– Nash equilibrium– ….

• Can it be computed?– if your computer cannot find it, the market

probably cannot either

Matrix (Normal Form) Games

2 0

0 1

1 0

0 3

Row player:

Column player:

0

1

0 1 0 1

0

1

• finite set of players {1, …, n}

• each player has k actions

(pure strategies): 1, …, k

• payoffs of the ith player: Pi: {1, …, k}n → R

Nash Equilibrium

2 0

0 1

1 0

0 3

Row player:

Column player:

0

1

0 1 0 1

0

1

• Nash equilibrium: a strategy profile such that

noone wants to deviate given other players’ strategies, i.e., each player’s strategy is a best response to others’ strategies:– (0, 0) and (1, 1) are both NE

Pure vs. Mixed Strategies

1 -1

-1 1

-1 1

1 -1

Row player:

Column player:

H

T

H T H T

H

T

• NE in pure strategies may not exist!– “matching pennies”

• Mixed strategy: a probability distribution over actions– 50% tail, 50% head

Existence of NE

• Theorem (Nash 1951): any n-player k-action game in normal form has an equilibrium in mixed strategies

can we find one in poly-time?

Plan of the Talk

2 players, k actions

n players, 2 actions

other cool stuff

2 (r=const) players, k actions

• Input representation:– 2 players: two k x k matrices– r players: r k x k x … x k matrices

• poly-size for constant r

• Output representation: – for 2 players all NE are in Q– but not for 3 and more players…

• Checking for pure NE: easy – at most k2 strategy profiles

2 players, k actions: mixed NE

• Naïve approaches: exp(k)

• Simplex-like approach (Lemke-Howson algorithm):– works well in practice– exp(k) in the worst case (2004)

• Is it time to give up?– maybe the problem is NP-hard?

Is Finding NE NP-hard?

• Reminder: a problem P is NP-hard if you can reduce 3-SAT to it:– “yes”-instance 3-SAT → “yes”-instance of P– “no”-instance 3-SAT → “no”-instance of P

• Problem: each instance of NASH is a “yes”-instance!– every game has a NE

• Formally: if NASH is NP-hard then NP = coNP• Need: complexity theory for

total search problems

Reducibility Among Search Problems

• S associates x in X with a solution set S(x)• Total search problem: for any x, S(x) is not empty

S: X Y

T: X’ Y’

If T is easy, so is S• S is reducible to T if:

– f, g easy to compute– g(T(f(x))) is in S(x)

f g

Completeness Results?

• Can we prove that any total search problem is reducible to r-NASH?

• Not really: the class T of all total search problems is a semantic class– not known how to find complete problems for these

• Want to pick a large subclass S of T s.t.– S includes some natural problems– there are problems that are complete for S– in particular, r-NASH is complete for S

• Input: Boolean circuits S (Successor), P (Predecessor):– n inputs, n outputs– S(0n) ≠ 0n, P(0n) = 0n

• Output: x ≠ 0n s.t. – S(P(x)) ≠ x or P(S(x)) ≠ x

Intuition: G=(V, E): – V = n; – E = {(x,y) | y=S(x), x=P(y)}

END OF THE LINE

00000

01011

11001

01011

PPAD

• PPAD: Polynomial Parity Argument, Directed version

• PPAD is the class of all search problems that are reducible to END OF THE LINE

search problem solution

circuits S, T “end of the line”f g

r-NASH is in PPAD

• Proof on Nash’s theorem:– existence of NE reduces to Brouwer’s fixpoint

theorem– Brouwer’s fixpoint theorem reduces to

Sperner’s lemma– Sperner’s lemma is proven by a parity

argument (similar to END OF THE LINE)

• Reduction of r-NASH to END OF THE LINE can be extracted from these proofs (Papadimitriou 94)

Brouwer’s Fixpoint Theorem

• Brouwer’s Theorem: Any continuous mapping from the simplex to itself has a fixpoint.

• Nash Brouwer proof sketch:– set of all strategy profiles → simplex– mapping: (s1, …, sn) → (s1+1, …, sn+n),

where i is a shift in the direction of best response to (s1, …, si-1, si+1, …, sn)

– NE is a point where noone wants to deviate, i.e., a fixpoint

• Proper coloring:– vertices on BC are not blue– vertices on AC are not green– vertices on AB are not yellow

• Sperner’s Lemma: there exists a trichromatic triangle

• Brouwer’s theorem Sperner’s Lemma:– x is blue if the grad(F) at x points away from A, etc.– trichromatic triangle “has no direction” – repeat at increased resolution…

Sperner’s Lemma

A

B

C

Reductions (Papadimitriou 1994)

END OF THE LINE is PPAD-complete

TRICHROMATIC TRIANGLE is PPAD-complete

3D-BROUWER is PPAD-complete

r-NASH is in PPAD

r-NASH vs 3D BROUWER

• Existence of NE follows from Brouwer’s fixpoint theorem

• NE are special cases of Brouwer’s fixpoints– just how special?

• Can any fixpoint be represented as a NE of a game?– Is there a reduction

from 3D BROUWER to r-NASH?

Hardness Reductions: the Timeline

• 3D-BROUWER is PPAD-complete (Papadimitriou, 1994)

• 4-NASH is PPAD-complete (Daskalakis,Goldberg, Papadimitriou, Sep 2005)

• 3-NASH is PPAD-complete (Daskalakis, Papadimitriou, Oct 2005, Chen, Deng, Nov 2005)

• 2-NASH is PPAD-complete !!! (Chen, Deng, Dec 2005)

n players, 2 actions

• representation: payoffs to each player for every action profile (vector in {0, 1}n): n2n numbers

• graphical games:– players are vertices of a graph– V’s payoff depends on actions of W in N(V) U V– n players, max degree d => n2d+1 numbers

TU

V

W t=0, u=0, v=0, w=0: 12t=1, u=0, v=0, w=0: 31 ….t=1, u=1, v=1, w=1: -6

W’s payoffs(16 cases):

Algorithms: What Was Known

• Bounded-degree trees:– Exp-time algorithm/poly-time approximation

algorithm to find all NE (Kearns, Littman, Singh, UAI 2001)

– ??? poly-time algorithm to find a single NE (Kearns, Littman, Singh, NIPS’2001)

• Heuristics for graphs with cycles

Our Results (E., Goldberg, Goldberg’06)

• Algorithm in NIPS’01 paper is incorrect (does not always output a NE)

• We fix the NIPS’01 algorithm, but…– our algorithm runs in poly-time on paths– with a trick, also on cycles

• There is a graph of pathwidth 2 on which our algorithm runs in exp time– true for all algorithms that use the basic

approach of the UAI’01 paper

Warm-up: 2-player 2-action games

2 0

0 1

1 0

0 3

Row player:

Column player:

0

1

0 1 0 1

0

1

Suppose R plays 1 w.p. r

EP(C) from playing 0: (1-r)*1

EP(C) from playing 1: r*3

1-r > 3r iff r < ¼

Suppose C plays 1 w.p. c

EP(R) from playing 0: (1-c)*2

EP(R) from playing 1: c*1

(1-c)*2 > c iff c < 2/3

1/4 1

r

BR(C)c1

2/3BR(R)

mixed NE: r=1/4, c=2/3

• Potential best response: v is a PBR to w iff when W plays w, there is a NE for TV in which V plays v.

• upstream pass: construct PBRV(w) from PBRU1(v), PBRU2(v) and PBRU3(v)

• downstream pass: root selects its strategy based on the children’s PBR’s; propagates to leaves

Algorithm for Trees (KLS’01)

TV

W

V

U1U2

U3

v

w

Computing PBR on a Path

• E0 = EP(V) from playing 0: (1-u)(1-w)*v000+(1-u)w*v001+u(1-w)*v100+uw*v101 = auw+bu+cw+d

• E1 = EP(V) from playing 1: (1-u)(1-w)*v010+(1-u)w*v011+u(1-w)*v110+uw*v111 = a’uw+b’u+c’w+d’

• E0 = E1 iff w = (Au+B)/(Cu+D) = f(u)

U V W

.5 1

1u

v

v

1

1

.5

.1 .9 w

(v, u) → (f(u), v)

PBRU(v) PBRV(w)

Trees: too many segments

v v w

ut v

v1 v2 v1 v2

v1

v2

KLS (NIPS’01): can “trim” PBR

Incorrect!

W

V

T U

(v,t), (v,u) → (f(u,t), v)

u2

u1

t2

t1

Solutions?

• Solution 1 (for paths): algorithm of UAI’01 paper, careful analysis– the number of segments/rectangles in each

PBR is O(n2)– running time O(n3)

• Solution 2 (for paths): can pick a subset of each PBR consisting of O(n) segments– O(n2) running time

Extension to trees? V0 V1 V2 Vn-1 Vn

U1

T1

Un-1

T2

U2

Tn-1 Tn

Un

Graphical games: hardness results

• NP-hard?– no: total search problem

• PPAD-hard?– yes!– in fact, this is how the hardness result for

4-player games was obtained (Goldberg, Papadimitriou, Aug 2005)

Equivalences: GP’05r-player game G NE of G

deg 3 graphical game G’ NE of G’

f g

d2-player game G’ NE of G’

deg d graphical game G NE of G

f g

Combining Reductions: GP’05

r-player game G NE of G

9-player game G’ NE of G’

f g

Finding NE in a 4-player game is as hard as

finding NE in a r-player game for any constant r

X4

PPAD-hardness: missing details

• 3D-Brouwer is PPAD-complete (Papadimitriou, 1994)

• 4-NASH is as hard as deg 3-GG (Goldberg, Papadimitriou, Aug 2005)

• deg 3-GG is PPAD-complete and hence 4-NASH is PPAD-complete

(Daskalakis,Goldberg, Papadimitriou, Sep 2005) • 3-NASH is PPAD-complete

(Daskalakis, Papadimitriou, Oct 2005, Chen, Deng, Nov 2005)

• 2-NASH is PPAD-complete !!! (Chen, Deng, Dec 2005)

NE with special properties

Pure NE:

• easy for constant number of players

• NP-hard for general graphical games – even if max degree = 3– NP vs. PPAD: pure NE may not exist!

• poly-time on trees (KLS algorithm)– also on graphs with bounded treewidth

Welfare-Maximizing NE

2 0

0 1

1 0

0 3

Row player:

Column player:

0

1

0 1 0 1

0

1

Nash equilibria: • (0, 0): total payoff is 3• (1, 1): total payoff is 4• (1/4, 2/3): total payoff is 17/12

not all NE are created equal…

Algorithms for Good NE

• 2-player games: checking for NE with total payoff > T is NP-hard (Gilboa Zemel 89, Conitzer, Sandholm 03)

• Graphical games: for any algebraic , deg() = n, there is a GG with int payoffs on a path of length O(n) in which in the best NE player 1 plays approximation algorithms for any (E., Goldberg, Goldberg 07)

Approximate NE

• -Nash equilibrium: a strategy profile such that noone can gain > by deviating

• Graphical games on trees: poly-time algorithms for any (KLS’01)

• 2-player games ( utilities in [0, 1] ): – PPAD-complete for =O(1/n)– Approximation for constant :

• 0.5 WINE’06 (Dec 2006)• 0.382 ( =1-1/) ACM EC’07 (June 2007)• 0.364 WINE’07 (Dec 2007)• 0.339 WINE’07 (Dec 2007)

Conclusions

• Computational aspects of game-theoretic questions are crucial

• Lots of cool open problems– computing NE in graphical games on trees– finding -Nash in 2-player games for small

• A rich set of techniques

• Talk to me if you want to know more…

Mixed strategies and payoffs

• Payoff matrices:

• the row player plays a = (a1, …, an)• the column player plays b = (b1, …, bn)• expected payoff of R when playing i: (Ri, *, b)• expected payoff of C when playing i: (C*, j, a)

R11 R12 … R1n

R21 R22 … R2n

…Rn1 Rn2 … Rnn

C11 C12 … C1n

C21 C22 … C2n

…Cn1 Cn2 … Cnn

R: C:

• if 1st player’s strategy a supported on I N ai ≠ 0 iff i I

2nd player’s strategy b supported on J N bj ≠ 0 iff j J

• then I BR(b): (b, Ri, *) ≥ (b, Rk, *) for all i I, k N

J BR(a): (a, C*, j) ≥ (a, C*, k) for all j J, k N– LP on variables a1, …, an, b1, …, bn

– solutions to LP ↔ Nash equilibria

• running time: 22kpoly(k)

2 players, k actions: support guessing

linear inequalities!

Reminder: 2-player 2-action games

2 0

0 1

1 0

0 3

Row player:

Column player:

0

1

0 1 0 1

0

1

Suppose R plays 1 w.p. r

EP(C) from playing 0: (1-r)*1

EP(C) from playing 1: r*3

1-r > 3r iff r < ¼

Suppose C plays 1 w.p. c

EP(R) from playing 0: (1-c)*2

EP(R) from playing 1: c*1

(1-c)*2 > c iff c < 2/3

1/4 1

r

BR(C)c1

2/3BR(R)

mixed NE: r=1/4, c=2/3

Computing PBR on a path

• f(u) = (au+b)/(cu+d)• a, b, c, d are determined by V’s payoffs

U V W

.5 1

1u

v

v

1

1

.5

.1 .9 w

(v, u) → (f(u), v) + “tails”

PBRU(v) PBRV(w)