backward induction in finitely repeated prisoner's dilemma...

INTRODUCTION DESIGN RESULTS CONCLUSION

Backward Induction in Finitely RepeatedPrisoner’s Dilemma: Experimental Evidence

Matthew Embrey Guillaume Fréchette Sevgi YukselMaastricht NYU NYU

Zurich 2013

1 / 22


MOTIVATION

I The prisoner’s dilemma is a canonical game in social sciences.I We know little about how people behave in finitely repeated PDs.I Results are inconclusive on whether or not subjects learn

backward induction:I some studies find cooperation to decline with experience,I others find that first defection occurs later on in the interaction.

I Folk wisdom: As the horizon of the repeated game increasescooperation rates increase.

I Backward induction is more difficult as there are more steps ofreasoning required.

2 / 26


COOPERATION DECREASES WITH EXPERIENCE0

.2.4

.6.8

1C

oop. ra

te

0 2 4 6 8 10Match

Mean cooperation

12

3R

ound

0 2 4 6 8 10Match

Mean time to first defection

PD1, Horizon = 4

Dal Bo (2005)

3 / 26


COOPERATION INCREASES WITH EXPERIENCE0

.2.4

.6.8

1C

oop. ra

te

0 5 10 15 20Match

Mean cooperation

12

34

56

7R

ound

0 5 10 15 20Match


Horizon = 10

Andreoni & Miller (1993)

4 / 26


PRIOR WORK

I Studies differ in the parameters of the game: number of matches,the horizon, stage game payoffs; but rarely within studies.

I Data is analyzed differentlyI Round 1 behaviorI Last round behaviorI Mean cooperation rateI Mean time to first defection

5 / 26


WHAT WE DO

I Meta-analysis of the prior experimental research.I Conduct a new experiment to understand how the ability to

perform backward induction varies with the environment.I Document how this varies with the parameters.

6 / 26


OUR CONTRIBUTIONS

Statistical:I Measures used in the literature (mean cooperation rate, mean

time to first defection) can be misleading.

Design:I Establish that having many matches is important.I Conduct the first study varying payoffs and horizon in a between

subject design.

Conceptual:I Propose that the impact of the horizon confounds other factors.I Horizon affects play:

I not because it requires more steps of backward induction,I but because it increases the value of cooperation which can be

captured by the basin of attraction.

7 / 26


META: PAPERS AND PARAMETERS

1. Selten and Stoecker (1986)I H = 10, 25 matches

2. Andreoni and Miller (1993)I H = 10, 20 matches, 1 session

3. Cooper, DeJong, Forsythe, and Ross (1996)I H = 10, 2 matches, 3 sessions

4. Hauk and Nagel (2001)I H = 10, 10 (×6) matches

5. Dal Bó (2005)I H = {2, 4}, {5,8,9,10} matches, 4 sessions, within design

6. Bereby-Meyer and Roth (2006)I H = 10, 20 matches, 4 sessions

7. Friedman and Oprea (2012)I H = 8, 8 matches, 3 sessions, within design

8 / 26


META: STAGE GAME PARAMETERS

Original NormalizedC D

C R SD T P

C DC R−P

R−P = 1 S−PR−P = −`

D T−PR−P = 1 + g P−P

R−P = 0

T > R > P > S

9 / 26


META: STAGE GAME PARAMETERSBETWEEN SUBJECTS DESIGNS ONLY

01

23

4g

0 1 2 3 4 5l

DB2005 AM1993

CDFR1996 FO2012

BMR2006 SS1986

HN2001

Normalized payoffs

Finitely repeated PD experiments

10 / 26


META: STAGE GAME PARAMETERSBETWEEN SUBJECTS DESIGNS ONLY

01

23

4g

0 1 2 3 4 5l

AM1993 CDFR1996

BMR2006 SS1986

HN2001

Normalized payoffs

Finitely repeated PD experiments

11 / 26


META: STANDARD ANALYSIS

I Payoffs matter: high g and high ` decrease cooperation.I With experience: last round cooperation rate close to zero.I Average and round 1 cooperation, round to first defection:

I when H is small, they decrease;I when H is large, they increase.

12 / 26



Cooperation rate (%) Mean roundAverage Round 1 Last Round to first def.

Experiment H g ` 1 L 1 L 1 L 1 L

DB2005 2 1.17 0.83 14 13 18 14 10 11 1.21 1.20within sub. 2 0.83 1.17 25 9 32 13 17 5 1.42 1.14

4 1.17 0.83 33 20 44 32 25 8 1.99 1.584 0.83 1.17 31 22 37 34 20 12 1.76 1.61

FO212 8 4.00 4.00 33 33 43 67 23 3 2.27 3.53within sub. 8 2.00 4.00 38 34 43 63 30 3 2.77 3.67

8 1.33 0.67 40 48 43 73 37 3 2.83 4.438 0.67 0.67 44 69 50 87 30 23 3.1 6.07

BMR2006 10 2.33 2.33 38 66 61 93 22 7 3.19 7.39AM1993 10 1.67 1.33 17 47 36 86 14 0 1.50 5.50

CDFR1996 10 0.44 0.78 52 57 60 67 20 27 4.63 5.53

First defection is set to Horizon + 1 if there are no defection.

Table : Cooperation rates and mean round to first defection13 / 26





DB2005 2 1.17 0.83 14 13 18 14 10 11 1.21 1.20within sub. 2 0.83 1.17 25 9 32 13 17 5 1.42 1.14

4 1.17 0.83 33 20 44 32 25 8 1.99 1.584 0.83 1.17 31 22 37 34 20 12 1.76 1.61

FO212 8 4.00 4.00 33 33 43 67 23 3 2.27 3.53within sub. 8 2.00 4.00 38 34 43 63 30 3 2.77 3.67

8 1.33 0.67 40 48 43 73 37 3 2.83 4.438 0.67 0.67 44 69 50 87 30 23 3.1 6.07

BMR2006 10 2.33 2.33 38 66 61 93 22 7 3.19 7.39AM1993 10 1.67 1.33 17 47 36 86 14 0 1.50 5.50

CDFR1996 10 0.44 0.78 52 57 60 67 20 27 4.63 5.53




VALUE OF COOPERATION VS. DEFECTIONBASINS OF ATTRACTION

Normalized PD

C DC 1, 1 -`, 1 + gD 1 + g, -` 0, 0

sizeBAD = `(H−1)+`−g

Reduced Game

Grim ADGrim H, H -`, 1 + gAD 1 + g, -` 0, 0

sizeBAD = `(H−1)+`−g

17 / 26


META: DETERMINANTS OF COOPERATION

Table : Probit regression (Marginal Effects): round 1.

(1) (2)

basin of att. -0.359∗∗∗ (0.093)g -0.063∗∗∗ (0.020) -0.034∗ (0.018)l -0.006 (0.023) -0.020 (0.025)

horizon 0.034∗∗∗ (0.009) 0.003 (0.014)

observations 5398 5398

Clustered (session level) standard errors in parentheses.∗∗∗1%, ∗∗5%, ∗10% significance.Note: Regressors not reported here include cooperation in round 1 offirst match, opponent cooperation in round 1 of previous match, andnumber of matches played.

18 / 26


DESIGN

2 × 2 factorial design (between subjects)

Summary of treatmentsPD/Horizon 4 8Easy E4 E8Difficult D4 D8

I Horizon: 4 or 8 roundsI Payoffs of the PD: Easy vs. DifficultI In each session subjects played 20-30 finitely repeated PDs

Easy PD

C DC 51, 51 22, 63D 63, 22 39, 39

Difficult PD

C DC 51, 51 5, 81D 81, 5 39, 39

19 / 26


DESIGNBASIN OF ATTRACTION OF GRIM

0.2

.4.6

.81

0 2 4 6 8 10Horizon

DB2005 AM1993

CDFR1996 FO2012

BR2006 EFY

Basin of Attraction

Potential Gains to Cooperation

20 / 26


FIRST DEFECTION1

23

0 5 10 15 20 25 30 0 5 10 15 20 25 30

E4 D4

Round

12

34

56

7

0 5 10 15 20 25 30 0 5 10 15 20 25 30

D8 E8

Round

Match


21 / 26


COOPERATION BY ROUND0

.2.4

.6.8

1

0 5 10 15 20 25 30 0 5 10 15 20 25 30

E4 D4

0.2

.4.6

.81

0 5 10 15 20 25 30 0 5 10 15 20 25 30

D8 E8

Match

First round

Cooperation Rate

22 / 26


UNRAVELLING OF COOPERATION0

.2.4

.6.8

1C

oo

pe

ratio

n R

ate

0 5 10 15 20 25 30Match

Round 1 Round 8

Round 7 Round 6

Treatment: E8

Mean cooperation rate by round

23 / 26


BREAKDOWN OF COOPERATION

0.1

.2.3

.4.5

1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9

D8 E8

Matches 1−10 Matches 11−20 Matches 21−30

Pro

ba

bili

ty

Breakdown of cooperation

Round

24 / 26


FIRST DEFECTION AND LAST COOPERATION3

45

67

Ro

un

d

0 10 20 30Match

Last cooperation First defection

Treatment: E8

Evolution of first defection

25 / 26


CONCLUSION

I Parameters of the game have a significant impact on round 1cooperation rates.

I The majority of this effect can be captured by the basin ofattraction.

I Despite differences in round 1 behavior, subjects behavior isconsistent with the logic of backward induction.

26 / 26

backward induction in finitely repeated prisoner's dilemma...

Documents