3.2.2. dynamic games of complete information: backward induction and subgame perfection

3.2.2. Dynamic Games of complete information: Backward Induction and Subgame perfection

- Repeated Games -

Strategic Behavior in Business and Econ

Outline

3.1. What is a Game ?3.1.1. The elements of a Game3.1.2 The Rules of the Game: Example3.1.3. Examples of Game Situations3.1.4 Types of Games

3.2. Solution Concepts3.2.1. Static Games of complete information: Dominant

Strategies and Nash Equilibrium in pure and mixed strategies

3.2.2. Dynamic Games of complete information: Backward Induction and Subgame perfection


Repeated Games

A Repeated Game is a special case of a dynamic (sequentialmoves) game that consists of a (usually) static game beingplayed several times, one after the other

The game that is repeated is called the “stage game”

The (stage) game can be played a given number of times(known to all the players) or an indefinite number of times.


Repeated Games

Thus, we can have a:

Finitely Repeated Game.When the stage game is player a number T of

rounds (1, 2, 3, . . ., T). T is known to all the players

Infinitely Repeated GameWhen either

After each round the game continues to the next round with probability p and ends with probability (1 – p)

The game is played forever but at each round the value of the payoffs decreases by a factor of “p”


Finitely Repeated Games

Recall the Prisoners' DilemmaA “generic” version of the game is represented in the tableBelow

C D

C 3,3 0 ,5

D 5 ,0 1, 1

Where,C stands for “cooperate”D stands for “defect”



We saw that both players defecting is the unique equilibriumof the game. In fact, D is a Dominant Strategy for each player

C D

C 3,3 0 ,5

D 5 ,0 1, 1

The apparently paradoxical behavior is that, although bothPlayers would mutually benefit from Cooperation, selfInterests leads to the worse outcome by Defecting




C D

C 3,3 0 ,5

D 5 ,0 1, 1

Repeating the game opens interesting possibilitiesTo “punish” egoistic (defect) behaviorsTo “reward” the right (cooperative) behavior




C D

C 3,3 0 ,5

D 5 ,0 1, 1

Examples: “Stick and Carrot Strategies” (Trigger Strategies)1. I will start with cooperation, and will mimic your behavior

afterwards2. I will start with cooperation and will keep doing so unless

you defect. In such case I will defect forever


The 2-times Repeated Prisoners' Dilemma

The tree representation of the 2-times Repeated Prisoners'Dilemma is shown in the next slide:

Notice:The “dotted” lines representing the simultaneous choice in

each stage of the gameThe payoffs at the end of the game correspond to the

sum of the payoffs in each stage Try to imagine the tree in a 3-times Repeated Prisoners'

Dilemma


C

C

C

C

C

C

C

C

C

C

C

C

C

C

CD

D

D

D

D

D

D

D

DD

D

D

D

D

D

6, 6

3, 8

8, 3

4, 43, 8

0, 105, 5

1, 68, 3

5, 510, 0

6, 14, 4

1, 66, 1

2, 2

Stage 1 Stage 2



The game must be solved by Backward Induction using theSubgame Perfection technique (since there are “linked” nodesthat indicate that the game is of Imperfect Information)

Notice that his is always the case when we repeat a static game

We must, therefore, “solve” each of the 4 “subgames” in the second stage of the game and then move backwards


C

C

C

C

C

C

C

C

C

C

C

C

C

C

CD

D

D

D

D

D

D

D

DD

D

D

D

D

D

6, 6

3, 8

8, 3

4, 43, 8

0, 105, 5

1, 68, 3

5, 510, 0

6, 14, 4

1, 66, 1

2, 2

Stage 1 Stage 2

Subgame 1

Subgame 2

Subgame 3

Subgame 4


Subgame 1

C D

C 6, 6 3, 8

D 8, 3 4 , 4

C D

C 3, 8 0 , 10

D 5, 5 1, 6

C D

C 4, 4 1, 6

D 6, 1 2, 2

C D

C 8, 3 5 , 5

D 10 , 0 6, 1

Subgame 2

Subgame 4Subgame 3



Notice that the solution in each subgame is always the same:

Player 1: DefectPlayer 2: Defect

And, again, Defect is a Dominant Strategy for each playerin each subgame

This is not a coincidence (as we will see shortly)

Thus, proceeding backwards in the tree we get . . .


C

C

C

C

C

C

C

C

C

C

C

C

C

C

CD

D

D

D

D

D

D

D

DD

D

D

D

D

D

6, 6

3, 8

8, 3

4, 43, 8

0, 105, 5

1, 68, 3

5, 510, 0

6, 14, 4

1, 66, 1

2, 2

Stage 1 Stage 2

Subgame 1

Subgame 2

Subgame 3

Subgame 4


C

C

CD

D

D

4, 4

1, 6

6, 1

2, 2

Stage 1 Stage 2

Again, what remains afterwe move backwards in thethree is another “simultaneousmove” game, the one that corresponds to the firststage of the game.

We must “solve” in looking atthe table representation


C

C

CD

D

D

4, 4

1, 6

6, 1

2, 2

Stage 1 Stage 2

C D

C 4, 4 1, 6

D 6, 1 2, 2


C

C

CD

D

D

4, 4

1, 6

6, 1

2, 2

Stage 1 Stage 2

Thus, knowing what will bethe outcome in the secondstage of the game . . .

C D

C 4, 4 1, 6

D 6, 1 2, 2


C

C

CD

D

D

4, 4

1, 6

6, 1

2, 2

Stage 1 Stage 2

Both players will also defectin the first round. (It's againa Dominant Strategy !)

C D

C 4, 4 1, 6

D 6, 1 2, 2


The T-times Repeated Prisoners' Dilemma

Stage 1 (D, D)

T=1

Stage 1 (D, D)

T=2

Stage 2 (D, D)

Stage 1 (D, D)

T=3

Stage 2 (D, D)

Stage 3 (D, D)

Stage 1 (D, D)

Any T

Stage 2 (D, D)

Stage 3 (D, D)

Stage T (D, D)· · · ·


The T-times Repeated Prisoners' Dilemma

No matter how many times the Prisoners' Dilemma is repeated, theequilibrium is always the same: Defect in every round.Why don't punishments (rewards) work ?

Intuition:At the last repetition, since the players know that there will not

be a “new chance” (no punishment-reward is possible), the best thing to do is to Defect

Knowing that, in the next-to-last round players know that in the next round the opponent will not cooperate. Then, why should I cooperate today if tomorrow my opponent is going to defect ? Again, the best thing to do is to Defect

We can apply this argument “backwards” to conclude that the best thing to do is to Defect all the time.


Finitely Repeated Games: General Facts

Any Finitely Repeated Game that consists of the repetition of a stage game that is a static game produces a “Dynamic Game of Imperfect Information

The tree representing such game is (usually) very large It should be solved by Backward Induction The following statements are always true in such games:

If the stage game has a unique Nash Equilibrium, then the Finitely Repeated Game has unique Subgame Perfect Equilibrium consisting of the repetition of that Nash Equilibrium in every round of the game

If the stage game has more than one Nash Equilibria, then any “reasonable” combination of those equilibria is a Subgame Perfect Equilibrium of the Finitely Repeated Game


Infinitely Repeated Games

We have seen that both players defecting is the unique subgame perfect equilibrium of the Finitely RepeatedPrisoners' Dilemma

C D

C 3,3 0 ,5

D 5 ,0 1, 1

Trigger Strategies do not lead to cooperation1. I will start with cooperation, and will mimic your behavior

afterwards (Tit-for-Tat)2. I will start with cooperation and will keep doing so unless

you defect. In such case I will defect forever (Grimm Trigger)


There are two possible interpretations of games that are repeated but not a fixed number of rounds

After each round the game continues to the next round with probability p and ends with probability (1 – p)

Example: Two firms compete day after day, but there is certain probability that one of them goesbankrupt and then the game is over

The game is played forever (an indefinite number of times) but at each round the value of the payoffs decreases by “p”

Example: Two people negotiate with offers and counteroffers over an item. As time goes by, the item loses

value. The game is over when they reach an agreement



The two different interpretations of a Infinitely Repeated Game are technically equivalent.

Since there is no “last round”, there is no possibility of thinking backwards. This opens real opportunities to achieve cooperation in the Prisoners' Dilemma !

In general, Infinitely Repeated Games are very complex



Let x be any positive number (for instance, money) and p any positive number smaller than 1 (for instance, a probability).Then,

x·p + x·p2 + x·p3 + x·p4 + · · · = x

x·p2 + x·p3 + x·p4 + x·p5· · · = x

and so on ...

Mathematical aside (infinite sums)

p

(1 - p)

p2

(1 - p)


The Infinitely Repeated Prisoners' DilemmaHow cooperation can be sustained in equilibrium ?

Payoff Computation: Imagine that Player 2 plays Grimm TriggerWhat is the (expected) payoff for Player 1 if after each round thegame continues with probability p (and ends with probability (1-p))

If Player 1 plays “cooperate” all the time Stage 1 Stage 2 Stage 3 Stage 4 · · ·

3 3·p + 0·(1-p) 3·p2 + 0·(1-p2) · · · · = 3 +3p + 3p2 + 3p3 + · · ·

• If Player 1 plays “defect” all the time Stage 1 Stage 2 Stage 3 Stage 4 · · ·

5 1·p + 0·(1-p) 1·p2 + 0·(1-p2) · · · · = 5 + 1p + 1p2 + 1p3 + · · ·



Payoff Computation: Imagine that Player 2 plays Grimm TriggerWhat is the payoff for Player 1 if after each round thevalue of the money decreases by a factor of p (for instance, if themoney decreases a 10% then p=0.9)

Playing “cooperate” all the time Stage 1 Stage 2 Stage 3 Stage 4 · · ·

3 3·p 3·p2 · · · · = 3 + 3p + 3p2 + · · ·3 3·(0.9) 3·(0.9)·(0.9) = 3 + 3·(0.9) + 3·(0.9)2 + · · . = 3 + 2.7 * 2.43 + · · ·

• Playing “defect” all the time Stage 1 Stage 2 Stage 3 Stage 4 · · ·

5 1·p 1·p2 · · · · = 5 + 1p + 1p2 + · · ·5 1·(0.9) 1·(0.9)·(0.9) = 5 + 1·(0.9) + 1·(0.9)2 + · ·

= 5 + 0.9 + 0.81 + · · ··



Imagine that Player 2 plays Grimm TriggerWhat is the best for Player 1, Cooperate or Defect ?

Expected Payoff from “cooperate” all the time Stage 1 Stage 2 Stage 3 Stage 4 · · ·

3 3·p + 0·(1-p) 3·p2 + 0·(1-p2) · · · · = 3 + 3p + 3p2 + · · ·

• Expected Payoff from “defect” all the time Stage 1 Stage 2 Stage 3 Stage 4 · · ·

5 1·p + 0·(1-p) 1·p2 + 0·(1-p2) · · · · = 5 + 1p + 1p2 + · · ·



Imagine that Player 2 plays Grimm TriggerWhat is the best for Player 1, Cooperate or Defect ?

Expected Payoff from “cooperate” all the time

E(Cooperate) = 3 + 3p + 3p2 + 3p3 + · · · = 3 + 3

• Expected Payoff from “defect” all the time

E(Defect) = 5 + 1p + 1p2 + 1p3 + · · · = 5 + 1

p

(1 - p)

p

(1 - p)



Imagine that Player 2 plays Grimm Trigger What is the best for Player 1, Cooperate or Defect ?

E(Cooperate) = 3 + 3

• E(Defect) = 5 + 1

Cooperate will be better if E(Cooperate) > E(Defect), that is, if

3 + 3 > 5 + 1 p > ½

p

(1 - p)

p

(1 - p)

p

(1 - p)

p

(1 - p)



Thus, cooperation can be sustained in equilibrium in the Infinitely Repeated Prisoners' Dilemma thanks to Trigger Strategies

Depends on “p” With Tit-for-Tat it is also possible to sustain cooperation, but

then p > 2/3 But, “cooperation” is not the unique equilibrium. There are equilibria

with “defection” as well


Infinitely Repeated Games: General Facts

Any Infinitely Repeated Game that consists of the repetition of a stage game that is a static game produces a “Dynamic Game of Imperfect Information”

The tree representing such game is (usually) very large It can not be solved by Backward Induction The following statement is always true in such games:

No matter how many equilibria the stage game has, any “reasonable” combination of strategies can be a Subgame Perfect Equilibrium in such games (Folk Theorem)


C D

C 3,3 0 ,5

D 5 ,0 1, 1

Infinitely Repeated Games: General Facts

What does “reasonable” mean ?

Notice that by playing Defect all the time any player can guarantee himself a payoff of at least 1 per each round. Thus, any “reasonable” outcome of the game should pay each playerat least 1 per round


Summary

Any Repeated Game that consists of the repetition of a stage game that is a static game produces a “Dynamic Game of Imperfect Information”

The tree representing such game is very large If the game is Finitely Repeated, it must be solved by

Backward Induction If the game is Infinitely Repeated, it can not be solved by

Backward Induction


The following statements are always true in Finitely Repeated Games:: If the stage game has a unique Nash Equilibrium, then the

Finitely Repeated Game has unique Subgame Perfect Equilibrium consisting of the repetition of that Nash Equilibrium in every round of the game

If the stage game has more than one Nash Equilibria, then any “reasonable” combination of those equilibria is a Subgame Perfect Equilibrium of the Finitely Repeated Game

The following statement is always true in Infinitely Repeated Games: No matter how many equilibria the stage game has, any “reasonable” combination of strategies can be a Subgame Perfect Equilibrium in such games (Folk Theorem)

Summary


Axelrod's Simulation

R. Axelrod, The Evolution of Cooperation

Prisoner’s Dilemma repeated 200 times

Economists submitted strategies

Pairs of strategies competed

Winner: Tit-for-Tat

Reasons:

Forgiving, Nice, Clear


Not necessarily tit-for-tat

Doesn’t always work

Don’t be envious

Don’t be the first to cheat

Reciprocate opponent’s behavior

Cooperation and defection

Don’t be too clever

To be credible, incorporate a clear policy of punishment

Lessons from Axelrod’s Simulation

3.2.2. dynamic games of complete information: backward induction and subgame perfection

Documents