EC3224 Autumn Lecture #05Extensive Form Games with Perfect Information

• Reading– Osborne, Chapters 5, 6, 7.1., 7.2, 7.7

• Learning outcomes– construct simple games in extensive form– understand the concept of a subgame-perfect

equilibrium for such games– find the subgame-perfect equilibrium through

backward induction

Extensive Form Games

• The strategic form of a game does not represent the timing of moves

• Hence plans of actions are fixed and cannot be changed

• In contrast, extensive form games capture the sequential structure of a game

• This captures that players can change their plans• For now, we consider extensive form games with

perfect information, i.e. when choosing an action a player knows the actions chosen by players moving before her

Histories and Terminal Histories

• A sequence of actions in an extensive form game for which no actions follow is called a terminal history

• if (a1, a2 , …, ak) is a terminal history, than any (a1, a2 , …, am) with m ≤ k is a subhistory and for m < k a proper subhistory (including empty sequence Ø)

• A history is any subhistory of any terminal history

Extensive Form Games

An extensive form game is characterized by• players• a set of terminal histories (that is all possible

complete sets of actions in the game)• a player function that assigns to any proper

subhistory of any terminal history the player who moves after this history

• preferences over the set of terminal histories

Game Trees

• An extensive form game can be represented in a game tree

• This shows – who moves when (at the nodes, representing

the (non-terminal) histories) – their available actions (the branches) – and the payoffs representing preferences over

terminal histories (at the terminal nodes)

A Game Tree

Player 1

Player 1

Left

A Game Tree

A Game Tree

Player 1

Left Right

A Game Tree

Player 1

Left Right

Player 2

A Game Tree

Player 1

Left Right

Player 2

l

A Game Tree

Player 1

Left Right

Player 2

l r

A Game Tree

Player 1

Left Right

Player 2 Player 2

l lr r

A Game Tree

Player 1

Left Right

Player 2 Player 2

l lr r

U1(L,l)

U2(L,l)

• The preferences can be represented by a payoff function over the terminal histories

A Game Tree

Player 1

Left Right

Player 2 Player 2

l lr r

U1(L,l)

U2(L,l)

U1(L,r)

U2(L,r)

• The preferences can be represented by a payoff function over the terminal histories

A Game Tree

Player 1

Left Right

Player 2 Player 2

l lr r

U1(L,l)

U2(L,l)

U1(L,r)

U2(L,r)

U1(R,l)

U2(R,l)

U1(R,r)

U2(R,r)

• The preferences can be represented by a payoff function over the terminal histories

Strategies

• A strategy is a complete description of a player’s actions at all the nodes when it’s his turn to move, e.g. for player 2 to choose r after L and l after R. Player 2 has 4 strategies: {(l,l),(l,r),(r,l),(r,r)}

• Mixed strategies are defined as usual

Player 1

Left Right

Player 2 Player 2

l lr r

U1(L,l)

U2(L,l)

U1(L,r)

U2(L,r)

U1(R,l)

U2(R,l)

U1(R,r)

U2(R,r)

Nash Equilibrium

• The Nash equilibrium of an extensive form game is defined as usual:– s* is a (mixed-strategy) Nash equilibrium if for every

player i and every mixed strategy si:

Ui(s*) Ui (si, s-i*)where Ui(s) is i’s expected payoff for the terminal histories that are induced by the players following strategies s

• Nash equilibrium is not a satisfactory concept for extensive form games:– moves after histories that would not be reached are

irrelevant for the outcome, so in Nash equilibrium, the actions do not have to be payoff maximizing at these points

– but what if a player makes a mistake and the history is reached?

Example: Mini Ultimatum Game• Proposer (Player 1) can suggest one of two splits of £10: (5,5)

and (9,1).• Responder (Player 2) can decide whether to accept or reject

(9,1), but has to accept (5,5). Reject leads to 0 for both

Player 1

(9,1)

a r

9 1

0 0

(5,5)

5 5

Player 2

Mini Ultimatum Game in Strategic Form

Player 2

accept (9,1) reject (9,1)

Player 1 propose (5,5) 5,5 5,5

propose (9,1) 9,1 0,0

• There are two equilibria: 1. (propose (9,1), accept (9,1))2. (propose (5,5), reject (9,1)).

• Equilibrium 2 is in weakly dominated strategies (reject (9,1) is weakly dominated)

• In strategic form, we assume players have correct beliefs due to experience

• But if 1 always proposes (5,5), she does not gain experience after propose (9,1)

• Experience could result from occasional mistakes• But if 1 occasionally mistakenly chooses (9,1), then

reject (9,1) is not optimal any more• In extensive form it is seen that equilibrium 2 is not

convincing because it relies on a non-credible threat: if the 1 proposes (9,1) player 2 has an incentive to deviate (i.e. to accept)

Example: Mini Ultimatum Game

Sequential Rationality• Think about equilibria in a game in extensive form• What should we do after a history that will not occur

in equilibrium?• Following the definition of Nash-equilibrium, the

choice is irrelevant after this history• However, an equilibrium appears to be more

convincing, if we require that each player chooses optimally in any decision node and takes into account that all players will do so in the future

• We call this sequential rationality • This notion is captured by subgame perfect

equilibrium (Selten, 1965)

Nobel Prize in Economics, 1994

QuickTime™ and a decompressor

are needed to see this picture.QuickTime™ and a

decompressorare needed to see this picture.

• Reinhard Selten (left), John Harsanyi (right) and…

QuickTime™ and a decompressor

are needed to see this picture.

John F. Nash

SubgamesA subgame is a part of an extensive form game

following some history, with the player function and preferences as for the whole game

• A subgame is a game in itself:– it starts at a single node

– it contains all moves of the whole game following the history

– once we are in a subgame, we do not leave it

• By convention, we consider the entire game to be a subgame of itself

Subgame Perfect equilibrium• A strategy s for a game induces a strategy s(g) for any

subgame g• A subgame perfect Nash equilibrium (SPNE) is a set

of strategies {si, i=1,…,n} such that for each subgame g, the set of induced strategies {si(g), i=1,…,n} forms a Nash equilibrium for this subgame, that is, in no subgame can a player increase her payoff by deviating to another strategy.

• In particular, since a game is a subgame of itself, a SPNE is always a Nash equilibrium

Subgame Perfect equilibrium

• In a SPNE the past is irrelevant, i.e. however we got to the subgame, we have to play an equilibrium, even if the strategies imply that we do not reach this subgame

• This is quite a strong requirement, because even if we got to the subgame by clearly non-rational behavior, we still require rationality for the future

• Furthermore, in a SPNE, all players must have the same expectation concerning the equilibrium to be played in a subgame, which is not necessarily always reasonable

Example: Mini Ultimatum Game• There are 2 subgames: whole game and following (9,1)• If 1 proposes (9,1), 2 is better off accepting• Given that, 1 is better off proposing (9,1)• Thus the only subgame perfect equilibrium is: (propose

(9,1), accept (9,1))Player 1

(9,1)

a r

9 1

0 0

(5,5)

5 5

Player 2

Backward Induction• Length of a subgame: number of moves in longest

terminal history• A game has a finite horizon if the length of the

longest terminal history is finite• Subgame perfect equilibria of games with finite

horizon can be found by backward induction• Backward induction:

– first find the optimal actions in the subgames of length 1– then taking these actions as given, find the optimal actions

of players who move at beginning of subgames of length 2– continue working backwards, ends after finitely many steps

• If in each subgame there is only one optimal action, this procedure leads to a unique subgame perfect equilibrium

Example: Mini Ultimatum Game• There is one subgame of length 1, following (9,1)

Player 1

(9,1)

a r

9 1

0 0

(5,5)

5 5

Player 2

Example: Mini Ultimatum Game• There is one subgame of length 1, following (9,1)• The optimal action is accept

Player 1

(9,1)

a r

9 1

0 0

(5,5)

5 5

Player 2

Example: Mini Ultimatum Game• There is one subgame of length 1, following (9,1)• The optimal action is accept• There is one subgame of length 2, the whole game

Player 1

(9,1)

a r

9 1

0 0

(5,5)

5 5

Player 2

Example: Mini Ultimatum Game• There is one subgame of length 1, following (9,1)• The optimal action is accept• There is one subgame of length 2, the whole game• Taking “accept” in the subgame of length 1 as given, we see

that (9,1) is optimalPlayer 1

(9,1)

a r

9 1

0 0

(5,5)

5 5

Player 2

The Ultimatum Game• Proposer (Player 1) suggest (integer) split of a fixed pie, say £10. • Responder (Player 2) accepts (proposal is implemented) or

rejects (both receive 0)• There is no unique solution for the subgame following (10,0)

Player 1

Player 2

(10,0)

a r

10 0

0 0

(9,1)

a r

9 1

0 0

a r

(5,5)…

5 5

0 0

(0,10)…

ra

0 10

0 0

If at one step, there is no unique best action of a player, backward

induction as above does not workWe can generalize:• at each step, consider the set of optimal actions• then follow each of them as aboveThis procedure allows us to find all subgame perfect equilibria of a

game with finite horizonApply this to ultimatum game For subgames of length 1, for each offer (10 - x, x) with x > 0, the

responder strictly prefers to acceptfor (10,0) responder is indifferent, so there are two optimal actions,

accept and rejectSo there are two subgame perfect equilibria:1. The proposer offers (10,0) and the responder accepts all offers,

including (10,0)2. The proposer offers (9,1) and the responder rejects (10,0) but

accepts all positive offers, i.e. accepts (9,1), (8,2) etc

Generalizing backward induction

Existence of subgame perfect equilibrium• A game is finite if it has a finite horizon and finitely

many terminal histories (i.e. each time a player moves, he has finitely many available actions)

• If a player has finitely many actions, we can find (at least) one that is optimal

• Thus there is always a solution at each step of backward induction and thus

Proposition: Every finite extensive game with perfect information has a subgame perfect equilibrium

Further generalizing backward induction• Assume after player 1 chooses R, both players move once more, but

simultaneously (1 chooses row, 2 column)• Thus we have a subgame after R and subgame perfect equilibrium

requires equilibrium play in this subgame• By starting at the end, we find SPNE ((A,U),(a,L)) and ((B,D),(a,R)

Player 1

A

a b

3 1

0 0

B

2,2 3,1

1,3 4,4

L R

D

U

Player 2

Backward Induction and Iterated Elimination of Dominated Strategies

• Backward Induction corresponds to iterated elimination of dominated strategies in the strategic form of the game:– a strategy that is not payoff maximizing in a subgame of

length 1 is dominated by the strategy that yields a higher payoff in this subgame but is otherwise identical

– picking the payoff-maximizing choice in this subgame corresponds to eliminating the dominated strategies

– having eliminated non-maximizing strategies in subgames of length 1, a strategy that is not payoff maximizing in a subgame of length 2 is dominated by one that is

– continuing in this fashion leads exactly to the same solution as backward induction

Commitment• In some situations a player can profit from reducing his options• Here, Player 2 would like to eliminate option l after Right• This would lead Player 1 to choose Left• Hence Player 2 would profit from being able to commit to

choosing r after Right• But the ability to commit would be part of the game

Player 1

Left Right

Player 2 Player 2

l lr r

5

5

0

0

8

2

0

0

Commitment

Player 1

Left Right

Player 2 Player 2

l r r

5

5

0

0

0

0

• In some situations a player can profit from reducing his options• Here, Player 2 would like to eliminate option l after Right• This would lead Player 1 to choose Left• Hence Player 2 would profit from being able to commit to

choosing r after Right• But the ability to commit would be part of the game

Problems with Backward Induction

• If there are many players, the demands on the players’ rationality become very strong.

• Example: 100 players

1

99

0

0

…

0

2

T T

P P

0

99

0

…

0

……………...100 P

T

0

0

…

0

99

100

100

…

100

100

Problems with Backward Induction

• If a player moves repeatedly and has already violated the backward induction solution once, what shall the others think?

• Example: centipede game

1

4

1

2

T T

P P

T

2

8

1

T

16

4

P 2

T

8

32

P 1

T

2P P

64

16

32

128

256

64

Application: Agenda Control and Strategic Voting• 3 players are voting upon three proposals, x,y,z• Preferences:

– P1: u(x) > u(y) > u(z)– P2: u(y) > u(z) > u(x)– P3: u(z) > u(x) > u(y)

• It is agreed that they will vote in the following way:– first two proposals are voted upon– then the winning proposal and the remaining proposal are voted upon

• Assume P1 can determine the sequence of votes (the agenda)– then if all votes are truthful, she will choose first y against z and then

the winner (y) against x, then x wins– But truthful voting is not SPNE: in the last stage, voting will be

truthful, but knowing that, P2 should vote for z in the first stage (P2 will vote strategically)

– in the SPNE, P1 will choose for the first round x and z, then P3 will vote strategically for x (because otherwise y will win in the end) and then x will win against y in the second stage

– So by being able to determine the agenda, P1 can get preferred result

Problem set #051. Consider the centipede game

a) Find the subgame perfect equilibrium through backward inductionb) What is the problem for player 2 if player 1 chooses I on his first

move?c) How would you play the game?

2. Do question 1 (a) and (c) for the “100-players game”.3. Osborne 163.24. Osborne 173.35. (Osborne 173.4)6. (Osborne 176.1)7. (Osborne 177.1)ESSAY TOPIC (max 1500 words): Choose a recent story from

the news. Describe the situation as a game (in strategic or extensive form). Find the equilibrium. Discuss whether actual events correspond to the equilibrium (if not discuss possible reasons why not). Discuss your assumptions.