Economics Letters 79 (2003) 239–245www.elsevier.com/ locate/econbase

O n asynchronously repeated gamesa b ,*Satoru Takahashi , Quan Wen

aDepartment of Economics, Harvard University, Cambridge, MA 02138,USAbDepartment of Economics, Vanderbilt University, VU Station B [351819, 2301Vanderbilt Place, Nashville,

TN 37235-1819,USA

Received 18 September 2002; accepted 10 October 2002

Abstract

We demonstrate that the classical Folk theorem may not fully characterize the set of subgame perfectequilibrium payoffs in an asynchronously repeated game. The limiting set of equilibrium payoffs cruciallydepends on players’ asynchronous move structure in the repeated game. 2002 Elsevier Science B.V. All rights reserved.

Keywords: Folk theorem; Repeated games; Asynchronous moves; Minimax; Maximin

JEL classification: C72; C73

In classical repeated games, players are assumed to revise their actions simultaneously andsynchronously in every period. In a seminal paper, Fudenberg and Maskin (1986) establish theclassical Folk theorem: under certain conditions, every feasible and strictly individually rationalpayoff of a normal-form stage game can be supported by a subgame perfect equilibrium in thecorresponding repeated game when the discounting is low enough. Fudenberg and Tirole (1991) firstraise the issues on simultaneous and synchronous treatments of players’ moves in repeated games.Part of recent research on repeated games has been dealing with these two closely related issues.Rubinstein and Wolinsky (1995) and Sorin (1995) show that if players do not move simultaneously instage games, the classical Folk theorem could be invalid. Takahashi (2002) and Wen (2002a) provideplayers’ relevant reservation values (called effective minimax) in general extensive-form games andrepeated extensive-form games, respectively. These studies show that the limiting set of subgameperfect equilibrium payoffs crucially depends on players’ non-simultaneous move structure in therepeated game.

Regarding the issue of players’ synchronous moves in repeated games, Lagunoff and Matsui (1997)first demonstrate an anti-Folk theorem. More specifically, when two players move asynchronously in a

*Corresponding author. Tel.:11-615-322-0174; fax:11-615-343-8495.E-mail address: quan.wen@vanderbilt.edu(Q. Wen).

0165-1765/02/$ – see front matter 2002 Elsevier Science B.V. All rights reserved.doi:10.1016/S0165-1765(02)00325-7

240 S. Takahashi, Q. Wen / Economics Letters 79 (2003) 239–245

repeated pure coordination game, the only subgame perfect equilibrium in the repeated game is therepetition of the Pareto dominant Nash equilibrium of the stage game. Under the NEU (NonEquivalent Utility, see Abreu et al., 1994) and FPI (Finite Periods of Inaction) conditions, and theassumptions that past mixed actions are observable and players who do not move in a period will haveto play the samemixed actions taken in the previous period, Yoon (2001) obtains the classical Folktheorem in asynchronously repeated games. Yoon (2002) reaches a similar conclusion when the stagegame may not satisfy the NEU condition. Wen (2002b) studies repeated games with asynchronousmoves where players who do not move have to play therealizations of their mixed actions taken inthe previous period, and completely characterizes the limiting set of subgame perfect equilibriumpayoffs, which crucially depends on players’ asynchronous move structure in the repeated game.Repeated games with asynchronous moves belong to the class of stochastic games in Dutta (1995),where players’ minimax values are calculated from the entire stochastic games. Most of those studiessuggest that the limiting set of subgame perfect equilibrium payoffs in a repeated game cruciallydepends on players’ move structure in the repeated game.

The key issue here is what the other players can do to punish a deviator given players’asynchronous move structure in a repeated game. Suppose a deviator does not move in a period, givenhis past action (even mixed action), the other players would be able to enforce deviator’s payoffsufficiently close to his maximin value, rather than his minimax value in the stage game. In this paper,we provide a simple example to demonstrate that the classical Folk theorem may not fully characterizethe limiting set of subgame perfect equilibrium payoffs as the discount factor goes to one. There mayexist a subgame perfect equilibrium in a repeated game where a player’s payoff is strictly lower thanhis minimax value in the stage game.

Consider a repeated game where players 1 and 2 may revise their actions in every period, but player3 may revise his action once every two periods. All the three players choose their actions in period 1of the repeated game. This repeated game satisfies the FPI condition. Players’ strategic interaction intwo consecutive periods, an odd period and the following even period, can be considered as asequential game in Wen (2002a), where all the three players move in the first step and players 1 and 2move in the second step. Thus this asynchronously repeated game could be modeled as a repeatedsequential game. The sequential ‘stage’ game depends not only on the stage game but also on theplayers’ discount factor in the repeated game. Wen (2002a) shows that a player’s minimax value inthis two-step sequential game could be the player’s lowest possible payoff among subgame perfectequilibrium payoffs. However, the average value of a player’s minimax value in this sequential gameover the two periods could be strictly less than his minimax value in the stage game. In the followingexample, we will show that player 3’s average payoff in a subgame perfect equilibrium could bestrictly less than his minimax value in the stage game.

The stage game is a three-player normal-form game where each playeri has two pure actions,Ai

and B for i 51, 2 and 3. Players’ payoffs are given in the following two 232 matrices:i

Payoff matrices in the three-player game

A B A B2 2 2 2

A 0, 0, 21 1, 0, 0 0, 0, 0 1, 0, 01

B 0, 1, 0 1, 1, 0 0, 1, 0 1, 1, 211

A B3 3

S. Takahashi, Q. Wen / Economics Letters 79 (2003) 239–245 241

Player 1 is the row player, 2 is the column player and 3 is the matrix player. Assume every playermay play mixed actions. Denote playeri’s mixed action bys [ [0, 1], wheres represents thei i

probability that playeri playsA , for i 5 1, 2 and 3. It is easy to see that this normal-form stage gamei

satisfies the NEU condition, and players 1 and 2 have the same minimax value of 0. For a mixed3action profile (s , s , s )[ [0, 1] , player 3’s expected payoff is given by1 2 3

u (s , s , s )5 2s s s 2 (12s )(12s )(12s ). (1)3 1 2 3 1 2 3 1 2 3

Player 3’s minimax value in the stage game is

1]m ;min maxu (s , s , s )5 2 , (2)3 3 1 2 3s ,s s 41 2 3

where players 1 and 2 equally mix their two pure actions so thats 5s 5 1/2. Any mixed action1 2

s [ [0, 1] is one of player 3’s best responses. On the other hand, player 3’s maximin value in this3

stage game is

1]M ;max minu (s , s , s )5 2 , (3)3 3 1 2 3s s ,s 23 1 2

where players 1 and 2 minimize player 3’s payoff by choosing the pure action on which player 3 putshigher probability. Whens 51/2, players 1 and 2 can choose either their first or second pure actions.3

In other words, in order to minimize player 3’s payoff, players 1 and 2 choose the followingstrategies:

(0, 0) if s [ [0, 1 /2],3(s , s )5 (4)H1 2 (1, 1) if s [ [1 /2, 1].3

Given players 1 and 2’s strategies, player 3 maximizes his payoff by mixing his two pure actions withequal probability, i.e.,s 51/2. Player 3’s maximin value is useful to determine player 3’s lowest3

possible payoff in the period where player 3 cannot revise his action.Now consider player 3’s lowest possible subgame perfect equilibrium payoff, i.e., the infimum of

player 3’s equilibrium payoffs, in the repeated game. Assume that every mixed action is observableand a player who does not move in a period will have to play the same mixed action taken in theprevious period. Let the discount factor bed [ (0, 1). Since the continuation game in period 3 is thesame as the repeated game itself, we can adopt Shaked and Sutton (1984)’s backward induction forbargaining games to derive the infimum of player 3’s equilibrium payoffs in the repeated game.Denote byL (d ) the infimum of player 3’s discounted average equilibrium payoffs. Then player 3’s3

lowest possible equilibrium payoff in period 3 is alsoL (d ). Given player 3’s mixed actions in3 3

period 1, player 3’s lowest possible subgame perfect equilibrium payoff in period 2 cannot be lowerthan

9 9(12d )min u (s , s , s )1dL (d ). (5)3 1 2 3 3s ,s9 91 2

The solution to (5) is that players 1 and 2 adopt the strategies described by (4) in period 2. Note thatif player 3 played either pure action in period 1, his payoff in period 2 would be2 1. In order tomaximize his payoff in period 2, player 3 should equally mix his two pure actions in period 1 so that(5) is maximized to be2 (12d ) /21dL (d ). By (5), L (d ) satisfies the following inequality:3 3

242 S. Takahashi, Q. Wen / Economics Letters 79 (2003) 239–245

29 9L (d )$min max (12d )u (s , s , s )1 (12d )d min u (s , s , s )1d L (d ) ,3 3 1 2 3 3 1 2 3 3F Gs ,s s s ,s1 2 3 9 91 2

which yieldsL (d )$m (d ), where3 3

1]] 9 9m (d ); min max u (s , s , s )1d min u (s , s , s ) . (6)3 3 1 2 3 3 1 2 3F Gs ,s s11d s , s1 2 3 9 91 2

As a matter of fact,m (d ) is the average value of player 3’s minimax payoff in the two-step sequential3

9game, with an adjustment factor of 1/(11d ). It is not hard to see thatm (d )[ [M , m ]. Taking (s ,3 3 3 1

9s ) from (4), we obtain the objective function of (6):2

9 9u (s , s , s )1d min u (s , s , s )3 1 2 3 3 1 2 3s ,s9 91 2

(7)2s s s 2 (12s )(12s )(12s )2d(12s ) if s [ [0, 1 /2],1 2 3 1 2 3 3 35H

2s s s 2 (12s )(12s )(12s )2ds if s [ [1 /2, 1].1 2 3 1 2 3 3 3

Note that (7) is continuous with respect tos [ [0, 1], and differentiable everywhere except at3

s 5 1/2. To maximize (7), consider the derivative of (7) with respect tos :3 3

≠]] 9 9u (s , s , s )1d min u (s , s , s )3 1 2 3 3 1 2 3F G≠s s ,s9 93 1 2

(8)2s s 1 (12s )(12s )1d if s [ [0, 1 /2),1 2 1 2 3

5H2s s 1 (12s )(12s )2d if s [ (1 /2, 1].1 2 1 2 3

Depending on the value of2s s 1 (12s )(12s ), we need to discuss the following five cases:1 2 1 2

1. If 2s s 1 (12s )(12s ), 2d, (8) implies that (7) is decreasing ins on [0, 1]. Player 3’s1 2 1 2 3

best response is thens 5 0.3

2. If 2s s 1 (12s )(12s )5 2d, (8) implies that (7) is constant ins on [0, 1/2] and1 2 1 2 3

decreasing on [1/2, 1]. Player 3’s best response is thens [ [0, 1 /2].3

3. If 2d , 2s s 1 (12s )(12s ),d, (8) implies that (7) is increasing ins on [0, 1/2] and1 2 1 2 3

decreasing on [1/2, 1]. Player 3’s best response is thens 5 1/2.3

4. If 2s s 1 (12s )(12s )5d, (8) implies that (7) is increasing ins on [0, 1/2] and constant1 2 1 2 3

on [1/2, 1]. Player 3’s best response is thens [ [1 /2, 1].3

5. If 2s s 1 (12s )(12s ).d, (8) implies that (7) is increasing ins on [0, 1]. Player 3’s best1 2 1 2 3

response is thens 5 1.3

Substituting player 3’s best response to (6), we have

9 9max u (s , s , s )1d min u (s , s , s )3 1 2 3 3 1 2 3F Gs s , s3 9 91 2

2 (12s )(12s )2d if 2s s 1 (12s )(12s ), 2d,1 2 1 2 1 2

2 [s s 1 (12s )(12s )1d ] /2 if 2d # 2s s 1 (12s )(12s )#d,5 1 2 1 2 1 2 1 252s s 2d if 2s s 1 (12s )(12s ).d,1 2 1 2 1 2

S. Takahashi, Q. Wen / Economics Letters 79 (2003) 239–245 243

2which attains its minimum2 (11d ) /4 at either

11d 12d]] ]]s 5s 5 or s 5s 5 .1 2 1 22 2

Thus (6) becomes

2(11d )1 11dF G]] ]]] ]]]m (d )5 2 5 2 , (9)3 11d 4 4

which converges toM 5 21/2, player 3’s maximin value in the stage game, asd → 1. m (d ) is3 3

player 3’s minimax value in the entire repeated game or player 3’s adjusted minimax value in thesequential game. It is strictly less than player 3’s minimax valuem 5 21/4 in the stage game when3

d .0.Now the question is whether player 3’s payoff could be sufficiently close tom (d ) in a subgame3

perfect equilibrium of the repeated game. The answer is positive. Consider a strategy profile whereplayers 1 and 2 chooses 5s 5 (12d ) /2 in every odd period and follow (4) to minimize player 31 2

in every even period, while player 3 playss 5 1/2 in every odd period (note that player 3 cannot3

revise his action in any even period). Neither player 1 nor 2 has any incentive to deviate since each ofthem has the same expected payoff given the other two players’ strategies. Our calculation showsplayer 3 has no incentive to deviate either. Player 3’s payoff in this subgame perfect equilibrium isexactly equal tom (d ). In fact, Wen (2002a) implies that every feasible payoff vectorv 5 (v , v , v )3 1 2 3

such thatv 4 (0, 0, 21/2) can be supported by a subgame perfect equilibrium whend is largeenough. In general, a player’s lowest possible equilibrium payoff depends not only on his minimaxvalue in the entire repeated game, but also on the set of feasible payoffs in the stage game and theother players’ minimax values.

This example also demonstrates that when we find a player’s lowest possible equilibrium payoff,we need to consider the dynamic effect of players’ actions. For example, if players 1 and 2 simplyminimaximize player 3’s payoff in every odd period and minimize it in every even period, player 3’saverage payoff will be

1 1 d 1 11 2d]] S ]D ]] S ]D ]]]2 1 2 5 2 .m (d ),311d 4 11d 2 4(11d )

which is not optimal in terms of punishing player 3. On the other hand, if player 3 simply adopts abest response to players 1 and 2’s actions in every odd period, his average payoff may be lower thanm (d ) since player 3 needs to consider the consequence of his action in the following period as well3

given the strategies by the other two players.Take one step further and consider that players 1 and 2 revise their actions in every period but

player 3 revises his action once everyT periods. The same backward induction overT consecutiveperiods implies that player 3’s minimax value in the entire repeated game, or player 3’s adjustedminimax value in aT-step sequential game, is

1 T]] 9 9m (d ) ; min max (12d )u (s , s , s )1 (d 2d ) min u (s , s , s )3 T 3 1 2 3 3 1 2 3F Gs ,s s s ,s1 2 312d 9 91 2

T T2 (12d ) / [4(12d )] if 1 2d $d 2d ,

5T5

21/2 if 12d #d 2d ,

244 S. Takahashi, Q. Wen / Economics Letters 79 (2003) 239–245

which is non-increasing inT. This result suggests that a player’s minimax value in the entire repeatedgame, which is the key in characterizing the limiting set of subgame perfect equilibrium payoffs,crucially depends on players’ asynchronous move structure in the repeated game.

In this class of repeated games under the NEU condition, a player’s reservation value in the entirerepeated game should be between his maximin and minimax values in the stage game. In two-playerfinite games in normal-form, a player’s mixed minimax value coincides with his mixed maximinvalue, hence coincides with his reservation value in the corresponding asynchronously repeatedgames. In games with more than two players, a player’s maximin value is generally less than hisminimax value. Thus, the classical Folk theorem could be misleading in asynchronously repeatedgames. As the example demonstrates, player 3’s minimax value in the repeated game varies betweenthese two values, depending on the frequency of player 3’s moves in the repeated game. Generallyspeaking, the less frequently a player moves, the lower his minimax value in the repeated game willbe. A player’s reservation value in the repeated game depends on players’ asynchronous movestructure, and so does the Folk theorem. A player’s minimax strategy in the stage game may not bethe most effective way to enforce his behavior in the repeated game since it ignores the dynamiceffect of players’ fixed actions. Consequently, the classical Folk theorem does not characterize thelimiting set of subgame perfect equilibrium payoffs in asynchronously repeated games when theyhave more than two players. For the general analysis for this class of dynamic games, we need toadopt the state-dependent backward induction to derive a player’s effective minimax value (see Wen,1994, 2002b) for the entire repeated game, similar to those developed by Takahashi (2002) and Wen(2002a).

A cknowledgements

We would like to thank Michihiro Kandori, Aki Matsui, and Kiho Yoon for comments andsuggestions. Taiji Furusawa provided us with many helpful comments on the preliminary draft. Theusual disclaimer applies.

R eferences

Abreu, D., Dutta, P.K., Smith, L., 1994. The Folk theorem for repeated games: a NEU condition. Econometrica 62, 939–948.Dutta, P.K., 1995. A Folk theorem for stochastic games. Journal of Economic Theory 66, 1–32.Fudenberg, D., Maskin, E., 1986. The Folk theorem in repeated games with discounting or with incomplete information.

Econometrica 54, 533–554.Fudenberg, D., Tirole, J., 1991. Games Theory. The MIT Press, Cambridge.Lagunoff, R., Matsui, A., 1997. Asynchronous choice in repeated coordination games. Econometrica 65, 1467–1477.Rubinstein, A., Wolinsky, A., 1995. Remarks on infinitely repeated extensive-form games. Games and Economic Behavior 9,

110–115.Shaked, A., Sutton, J., 1984. Involuntary unemployment as a perfect equilibrium in a bargaining model. Econometrica 52,

1351–1364.Sorin, S., 1995. A note on repeated extensive games. Games and Economic Behavior 9, 116–123.Takahashi, S., 2002. Effective minimax values in extensive form games with almost perfect information. Mimeo, Harvard

University.Wen, Q., 1994. The Folk theorem for repeated games with complete information. Econometrica 62, 949–954.

S. Takahashi, Q. Wen / Economics Letters 79 (2003) 239–245 245

Wen, Q., 2002a. A Folk theorem for repeated sequential games. Review of Economic Studies 69, 493–512.Wen, Q., 2002b. Repeated games with asynchronous moves. Mimeo, Vanderbilt University.Yoon, K., 2001. A Folk theorem for asynchronously repeated games. Econometrica 69, 191–200.Yoon, K., 2002. The effective minimax value of asynchronously repeated games. Mimeo, Korea University.