repeated games with intervention: theory and applications in communications

IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 60, NO. 10, OCTOBER 2012 3123

Repeated Games with Intervention:Theory and Applications in Communications

Yuanzhang Xiao, Jaeok Park, and Mihaela van der Schaar, Fellow, IEEE

Abstract—In communication systems where users share com-mon resources, selfish behavior usually results in suboptimalresource utilization. There have been extensive works that modelcommunication systems with selfish users as one-shot games andpropose incentive schemes to achieve Pareto-optimal outcomes.However, in many communication systems, due to strong negativeexternalities among users, the sets of feasible payoffs in one-shotgames are nonconvex. Thus, it is possible to expand the set offeasible payoffs by having users choose different action profilesin an alternating manner. In this paper, we formulate a model ofrepeated games with intervention. First, by using repeated gameswe can convexify the set of feasible payoffs in one-shot games.Second, by using intervention in repeated games we can achievea larger set of equilibrium payoffs and loosen requirements forusers’ patience to achieve a target payoff. We study the problemof maximizing a welfare function defined on users’ payoffs.We characterize the limit set of equilibrium payoffs. Given theoptimal equilibrium payoff, we derive the sufficient condition onthe discount factor and the intervention capability to achieve it,and design corresponding equilibrium strategies. We illustrateour analytical results with power control and flow control.

Index Terms—Repeated games, intervention, power control,flow control.

I. INTRODUCTION

GAME theory is a formal framework to model andanalyze the interaction of selfish users. It has been used

in the literature to study communication networks with selfishusers [1][2]. Most works modeled communication systems asone-shot games, studied the inefficiency of noncooperativeoutcomes, and proposed incentive schemes, such as pricing[3]–[12], to improve the inefficient outcomes towards thePareto boundary. Recently, a new incentive scheme, called“intervention”, has been proposed in [13] in the one-shotgame framework, with applications to medium access control(MAC) games [14][15] and power control games [16].1 Inan intervention scheme, the designer places an interventiondevice, which can monitor the user behavior and intervenein their interaction, in the system. The intervention deviceobserves a signal about the users’ actions and then chooses anintervention action depending on the observed signal. In this

Paper approved by T. T. Lee, the Editor for Switching Architecture Perfor-mance of the IEEE Communications Society. Manuscript received September17, 2011; revised February 27, 2012.

Y. Xiao and M. van der Schaar are with the Electrical EngineeringDepartment, University of California, Los Angeles, CA 90095 USA (e-mail:{yxiao, mihaela}@ee.ucla.edu).

J. Park is with the School of Economics, Yonsei University, Seoul 120-749, Korea (e-mail: [email protected]). He was previously with theElectrical Engineering Department, University of California, Los Angeles,CA 90095 USA.

Digital Object Identifier 10.1109/TCOMM.2012.071612.1106261With the same philosophy as intervention, a packet-dropping incentive

scheme was proposed for flow control games in [17].

Payoff of user 2

Payoff of user 1

Nash equilibrium payoff,one-shot games w/o

intervention

Pareto boundary,repeated games

Pareto boundary,one-shot games

Incentive provision byintervention, one-shot games

Incentive provision byintervention, repeated

games

Boundary of the set of NEpayoffs, one-shot gameswith intervention

Boundary of the limit setof equilibrium payoffs,repeated games w/ointervention

Boundary of the limit setof equilibrium payoffs,repeated games with

intervention

Fig. 1. Sets of equilibrium payoffs in a two-user game under differentincentive schemes. We use the same intervention device (thus the sameintervention capability) in the games with intervention. The (limit) sets ofequilibrium payoffs in repeated games are obtained with the discount factorarbitrarily close to 1. We can see that the set of equilibrium payoffs of therepeated game with intervention includes the set of equilibrium payoffs inthe one-shot game with intervention and that in the repeated game withoutintervention.

way, it can deter misbehavior of a user by exerting punishmentfollowing a signal that suggests a deviation.

In communication systems where users create severe con-gestion or interference, increasing a user’s payoff requires asignificant sacrifice of others’ payoffs. This feature is reflectedby a nonconvex set of feasible payoffs in some systems studiedin the aforementioned works [3]–[17] using one-shot gamemodels. For example, in one-shot power control games, theset of feasible payoffs is nonconvex when the cross channelgains are large [18][19]. In one-shot MAC games basedon the collision model, the set of feasible payoffs is alsononconvex, because transmissions from multiple users causepacket loss [10][20]. Moreover, the sets of feasible payoffsof some one-shot flow control games are also nonconvex.To sum up, the sets of feasible payoffs are nonconvex inmany communication scenarios, and when the set of feasiblepayoffs is nonconvex, its Pareto boundary can be dominatedby a convex combination of different payoffs (see Fig. 1 forillustration). In one-shot games, such convex combinationscannot be achieved unless a public correlation device is used.2

When users interact in a system for a long period of

2Public correlation devices are used in game theory literature as a simplemethod to achieve payoffs that are convex combinations of pure-actionpayoffs. Such devices may not be available in communication systems. Even ifsuch devices are available, there are additional costs on broadcasting randomsignals generated by a public correlation device.

0090-6778/12$31.00 © 2012 IEEE

3124 IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 60, NO. 10, OCTOBER 2012

time, we can model their interaction as a repeated game, inwhich users play a stage game repeatedly over time obtaininginformation about the actions of other users. In a repeatedgame, users can choose different actions in an alternatingmanner, obtaining a convex combination of different stage-game payoffs as a repeated-game payoff. A repeated-gamestrategy specifies actions to be taken given a history ofpast observations and thus can be interpreted as a protocol.If a repeated game strategy constitutes a subgame perfectequilibrium (SPE), then no user can gain from deviation atany occasion. Hence, a SPE can be considered as a deviation-proof protocol.

In this paper, we consider the problem of designing aprotocol to maximize some welfare function defined on users’payoffs among (subgame perfect) equilibrium protocols. First,we characterize the set of equilibrium payoffs when users aresufficiently patient. This set determines the feasible set of thewelfare maximization problem. Consequently, a larger set ofequilibrium payoffs can result in higher social welfare. In thispaper, we will characterize the set of equilibrium payoffs inrepeated games with intervention, and show that using inter-vention can yield a larger set of equilibrium payoffs than thecorresponding set without intervention (Fig. 1 illustrates thepromising gain by using repeated games with intervention.).

Given the feasible set determined by the set of equilibriumpayoffs, we can obtain the target payoff that is in the fea-sible set and maximizes the welfare function. Then we willanalytically determine the range of the intervention capabilityand the discount factors under which the target payoff canbe achieved at an SPE. The intervention capability reflectsthe extent to which the intervention device can intervene inthe users’ interaction. The discount factor is affected by theuser patience and the network dynamics. It represents the rateat which users discount future payoffs; a less patient userhas a smaller discount factor. The discount factor can alsomodel the probability of users remaining in the network ineach period; a more dynamic network results in a smallerdiscount factor. A lower discount factor is desirable in thesense that with a lower requirement, a protocol is effectivein a wider variety of users and more dynamic networks. Wewill show that using intervention can lower the requirementon the discount factor compared to the case without interven-tion. Moreover, we obtain a trade-off between the discountfactor and the intervention capability. Hence, given a discountfactor, we can calculate the minimum intervention capabilityrequired to support the target payoff, and thus determine whichintervention device to use. Conversely, given the interventioncapability available, we can calculate the minimum discountfactor required, and thus determine the types of users andnetworks that can be supported. Finally, given a target payoffand the discount factor, we show how to construct an equilib-rium strategy, namely the deviation-proof protocol. Simulateresults validate our theoretical analysis and demonstrate theperformance gain over existing incentive schemes.

The rest of this paper is organized as follows. Section IIprovides a summary of related works. Section III describesa repeated game model generalized by intervention and for-mulates the protocol design problem. In Section IV, wecharacterize the set of equilibrium payoffs when the discount

factor is arbitrarily close to one, and specify the structure ofequilibrium strategies. Then we solve the design problem inSection V. Simulation results are presented in Section VI.Finally, Section VII concludes the paper.

II. RELATED WORKS

To improve the inefficient noncooperative outcomes, mostexisting works proposed incentive schemes based on pricing[3]–[12] under one-shot game models. As discussed before,these schemes are inefficient when the multi-user interferenceis so strong that the set of feasible payoffs in one-shot games isnonconvex. In addition, one problem in pricing is to determinehow a user’s payoff changes with the amount of payment.Although it is usually assumed that the payoff is linearlydecreasing in the amount of payment, this assumption maynot be true. For example, a user’s payoff may be a concavedecreasing function of the amount of payment, reflecting thefact that a user’s payoff decreases faster when the paymentis larger. One advantage of intervention is that the designerknows how the users’ payoffs are affected by the actionsof the intervention device according to some physical laws.For example, in power control, the designer knows that bytransmitting at certain power level, the intervention devicecan cause some interference to the users, and the amount ofinterference will be added to the denominator of each user’ssignal-to-interference-and-noise ratio (SINR). Another advan-tage of intervention is that the intervention device directlyinteracts with the users in the system, instead of using outsideinstruments such as monetary payments in pricing. As a result,intervention can provide more robust incentives in the sensethat users cannot avoid punishment.

Intervention was originally proposed in [13] under one-shot game models, and was applied to one-shot power controlgames [16] and one-shot MAC games [14][15]. Although ithas several advantages over pricing, intervention in one-shotgames is inefficient in the scenarios with strong multi-userinterference. This motivates us to study intervention underrepeated game models.

In economics, the study of repeated games focuses onproving folk theorems [21]. Specifically, they characterizethe set of equilibrium payoffs when users are sufficientlypatient and the network is static (i.e., the discount factorcan be arbitrarily close to 1). For most games, they provethat any equilibrium payoff can be sustained under somesufficiently large discount factor, but they do not know howlarge the discount factor has to be to sustain a particularequilibrium payoff. In contrast, we are more interested in thecase where the users are impatient or the network is dynamicwith users leaving with certain probability. Hence, given anequilibrium payoff, we want to analytically determine howlarge a discount factor we need in order to achieve the givenequilibrium payoff. A computational method was proposed in[22]. Analytical characterization is available for a few specialcases, such as games with transferable payoffs in [23] andthe repeated prisoners’ dilemma game in [24]. For the classof games we consider, our result (Theorem 1) is the first thatanalytically determines how large the discount factor needs tobe to achieve a given Pareto-optimal equilibrium payoff.

XIAO et al.: REPEATED GAMES WITH INTERVENTION: THEORY AND APPLICATIONS IN COMMUNICATIONS 3125

Following the game theory literature, most works applyingrepeated games in communication systems assume that the dis-count factor can be arbitrarily close to 1 [25]–[32]. Moreover,almost all the works use strategies in which the users playthe same action profile at the equilibrium [25]–[35]. Sincethe users play the same action profile, they can only achievethe feasible payoffs in one-shot games. Hence, applying suchstrategies to the scenarios with strong multi-user interferencewill still result in inefficient outcomes. In contrast, we allowthe users to choose different action profiles in an alternatingmanner to achieve Pareto optimality, although such repeated-game strategies are much harder to analyze.

III. MODEL OF REPEATED GAMES WITH INTERVENTION

A. The Stage Game With Intervention

We consider a system with N users. The set of the users isdenoted by N � {1, 2, . . . , N}. Each user i chooses an actionai from the set Ai ⊂ R

ki for some integer ki > 0. An actionprofile of the users is denoted by a = (a1, . . . , aN ), and the setof action profiles of the users is denoted by A =

∏Ni=1 Ai. We

use a−i to denote the action profile of the users except for useri. In addition to the N users, there exists an intervention devicein the system, indexed by 0. The intervention device choosesan action a0 from the set A0 ⊂ R

k0 for some integer k0 > 0.We refer to (a0, a) ∈ A0 × A as an action profile (of theintervention device and the users). The payoffs of the users aredetermined by the action profile, and user i’s payoff function isdenoted by ui : A0×A → R. Given the payoff functions of theusers, the set A0 determines the intervention capability of theintervention device, since it reflects the extent to which the in-tervention device can affect the users’ payoffs. We assume thatthere exists a null intervention action, denoted by a0 ∈ A0,which corresponds to the case where there is no interventiondevice. We further assume that an intervention action can onlydecrease the payoffs of the users, i.e., ui(a0, a) ≤ ui(a0, a)for all a0 ∈ A0, all a ∈ A, and all i ∈ N . In other words,intervention can provide only punishment to users, not reward.A (strategic-form) game with intervention is summarized bythe tuple 〈N , (A0, (Ai)i∈N ), (ui)i∈N 〉. For technical reasons,we assume that each of A0 and Ai is a compact and convexset and that ui is continuous for all i.

An important feature of the intervention device is that itdoes not have its own objective and can be programmed in theway that the protocol designer desires. Hence, when discussingequilibrium, we need to take only the users’ incentives intoaccount. An action profile (a∗0, a

∗) is a Nash equilibrium (NE)of the game 〈N , (A0, (Ai)i∈N ), (ui)i∈N 〉 if

ui(a∗0, a

∗) ≥ ui(a∗0, ai, a

∗−i), ∀ai ∈ Ai, ∀i ∈ N . (1)

We call an action profile (a0, a∗) a Nash equilibrium without

intervention if (1) is satisfied with a∗0 = a0.

B. The Repeated Game With Intervention

In a repeated game with intervention, the users play thesame game 〈N , (A0, (Ai)i∈N ), (ui)i∈N 〉 in every period t =0, 1, 2, . . .. The game played in each period is called the stagegame of the repeated game. At the end of period t, all the usersand the intervention device observe the action profile chosen

in period t, which is denoted by (at0, at). That is, we assume

perfect monitoring. As a result, the users and the interventiondevice share the same history at each period, and the historyat the beginning of period t is the collection of all the actionprofiles before period t. We denote the history in period t ≥1 by ht = (a00, a

0; a10, a1; . . . ; at−1

0 , at−1), while we set theinitial history, i.e., the history in period 0, as h0 = ∅. The setof possible histories in period t is denoted by H t, and theset of all possible histories by H =

⋃∞t=0 H t.

The (pure) strategy of user i in the repeated game is amapping from the set of all possible histories to its action set,written as σi : H → Ai. When user i uses strategy σi, itsaction at history ht is determined by ati = σi(h

t). The strategyprofile of the users is denoted by σ = (σ1, . . . , σN ), and thestrategy profile of the users except for user i by σ−i. The setof all strategies of user i is denoted by Σi, while the set of allstrategy profiles is denoted by Σ =

∏Ni=1 Σi. The intervention

device chooses its actions following an intervention rule,which is represented by a mapping σ0 : H → A0.3 When theintervention device uses the intervention rule σ0, its actionat history ht is determined by at0 = σ0(h

t). The set of allintervention rules is denoted by Σ0. When the interventionrule is constant at a0, namely σ0(h) = a0 for all h ∈ H , therepeated game with intervention reduces to the conventionalrepeated game without intervention.

To compute the payoff of a user in the repeated game, weuse the discounted average payoff of the user. We assumethat all the users have the same discount factor δ ∈ [0, 1),as commonly assumed in the literature [25]–[35]. Then thepayoff function of user i in the repeated game is

Ui(σ0, σ) = (1 − δ)∑∞

t=0 δtui(a

t0(σ0, σ), a

t(σ0, σ)), (2)

where (at0(σ0, σ), at(σ0, σ)) is the action profile in period

t induced by (σ0, σ). The path of plays under (σ0, σ),{(at0(σ0, σ), a

t(σ0, σ))}∞t=0, can be calculated recursively by(a00(σ0, σ), a

0(σ0, σ)) = (σ0(∅), σ(∅)) and for all t ≥ 1,

(at0(σ0, σ), at(σ0, σ)) (3)

=(σ0(a

00(σ0, σ), a

0(σ0, σ); . . . ; at−10 (σ0, σ), a

t−1(σ0, σ)),

σ(a00(σ0, σ), a0(σ0, σ); . . . ; a

t−10 (σ0, σ), a

t−1(σ0, σ))).

User i’s continuation strategy induced by any history ht ∈H , denoted σi|ht , is defined by σi|ht(hτ ) = σi(h

thτ ), ∀hτ ∈H , where hthτ is the concatenation of the history ht followedby the history hτ . Similarly, we can define the continuationintervention rule, σ0|ht . By convention, we use σ|ht andσ−i|ht to denote the continuation strategy profile of the usersand the continuation strategy profile of the users except foruser i, respectively. Then a pair of the intervention rule andthe strategy profile (σ0, σ) is a subgame perfect equilibrium(SPE) of the repeated game with intervention if for all ht ∈ H

Ui(σ0|ht , σ|ht) ≥ Ui(σ0|ht , σ′i, σ−i|ht), ∀σ′

i ∈ Σi, ∀i ∈ N .

If the users use a strategy profile that constitutes a SPEtogether with an intervention rule, no user can gain fromchoosing a different strategy starting at any history.

3We call σ0 an “intervention rule” instead of an “intervention strategy” be-cause the intervention device is not a strategic player but can be programmedto follow a rule prescribed by the designer.


C. Problem Formulation

We use the term “protocol” to refer to (σ0, σ), whichprescribes the behavior of the intervention device and theusers in the repeated game. There is a protocol designer whochooses an intervention rule σ0 and recommends a strategyprofile σ to the users, having complete information about therepeated game described above. In order to make the protocol“deviation-proof” following any history, the designer shouldchoose a protocol that constitutes a SPE. The designer has anobjective function, which is a welfare function defined over therepeated-game payoffs of the users, denoted by W : RN → R.Then the protocol design problem can be formally written as

maxσ0,σ

W (U1(σ0, σ), . . . , UN (σ0, σ)) (4)

subject to (σ0, σ) is subgame perfect equilibrium.

Two widely-used welfare functions are sum payoff∑N

i=1 Ui

(the utilitarian social welfare function) and max-min fair-ness mini∈N Ui (the egalitarian social welfare function) [36].Sometimes, the designer may want to guarantee a minimumpayoff γi to each user i. This minimum payoff guarantee canbe incorporated into the welfare function.

IV. CHARACTERIZING THE LIMIT SET OF EQUILIBRIUM

PAYOFFS

A. Folk Theorem for Repeated Games With Intervention

In order for the designer to solve the problem (4), itis important to identify the set of payoffs achievable at aSPE. In general, finding the set of equilibrium payoffs fora given discount factor is a challenging task. However, folktheorems characterize the limit set of equilibrium payoffs asthe discount factor goes to 1. We will adapt conventionalfolk theorems for repeated games with intervention. Our proofis constructive and thus provide an equilibrium protocol thatachieves a desired equilibrium payoff. Our results show thatintervention can enlarge the limit set of equilibrium payoffsby strengthening punishment for deviation.

We first provide some preliminaries for our folk theorem.Definition 1 (Minmax Payoff With Intervention): User i’s

(pure-action) minmax payoff with intervention is defined as

vwi � min(a0,a−i)∈A0×A−i

maxai∈Ai

ui(a0, ai, a−i). (5)

User i’s minmax payoff is the payoff it can guaranteeby playing a best response to others’ actions at every his-tory. We say that a payoff vector v = (v1, . . . , vN ) isstrictly individually rational if vi > vwi for all i ∈ N .Let u(a0, a) = (u1(a0, a), . . . , uN(a0, a)). Then the set offeasible and strictly individually rational payoffs is

V †w = co{v ∈ R

N : ∃(a0, a) ∈ A0 ×A s.t. v = u(a0, a)}∩ {v ∈ R

N : vi > vwi , ∀i ∈ N},where coX denotes the convex hull of a set X . When theaction profile to achieve a user’s minmax payoff does notdepend on the minmaxed user, we obtain a mutual minmaxprofile, as formally defined below.

…

…

Deviation Deviation Deviation

…

Whodeviated?

User 1

User N

… … …

0ew 1ew 1ew T

1 0pw1 1pw 1 1pw L

0 00 ,a a00 ,a a0a 1 1

0 ,a a10 ,a a1a

1 10 ,T Ta a10 ,T 1a 1 a 1a

*0 1 1ˆˆ , ,a a a *

0 1 1ˆˆ , ,a a a

*0 1 1ˆˆ , ,a a a

*0 ˆˆ , ,N Na a a *

0 ˆˆ , ,N Na a a

*0 ˆˆ , ,N Na a a

0Npw 1N

pw 1Npw L

Fig. 2. The automaton of the equilibrium strategy of the game with a mutualminmax profile (a0, a). Circles are states, where {we(τ)}T−1

τ=0 is the set ofstates in the equilibrium path, and

{wi

p(�)}L−1

�=0is the set of states in the

punishment phase for user i. The initial state is we(0). Solid arrows arethe prescribed state transitions labeled by the action profiles leading to thetransitions. Dashed arrows are the state transitions when deviation happens.a∗i is user i’s best response to a0 and a−i.

Definition 2 (Mutual Minmax Profile): An action profile(a0, a) is a mutual minmax profile if for all i ∈ N ,

vwi = maxai

ui(a0, ai, a−i).

Now we state the folk theorem for repeated games withintervention.

Proposition 1 (Minmax Folk Theorem): Suppose that thereexists a mutual minmax profile (a0, a). Then for every feasibleand strictly individually rational payoff v ∈ V †

w , there existsδ < 1 such that for all δ ∈ (δ, 1), there exists a subgameperfect equilibrium with payoffs v, of the repeated game withintervention.

Proof: See [37, Appendix A].We briefly describe the structure of equilibrium protocols

that are used to prove the folk theorem. Suppose that we wantto achieve a payoff vector v. By [21, Lemma 3.7.2], whenthe discount factor is sufficiently close to 1, we can find asequence of action profiles {(at0, at)}∞t=0 whose discountedaverage payoffs are v and whose continuation payoffs at anytime t are close to v. In the equilibrium protocol constructedin the proof of Proposition 1, the intervention device and theusers start from following the sequence {(at0, at)}∞t=0 and keepfollowing it as long as there is no unilateral deviation. If useri deviates unilaterally, the intervention device chooses a0 andthe other users choose a−i for L periods as a punishmenton user i, where L is chosen sufficiently large to make anydeviation from the sequence unprofitable. After the L periodsof punishment, the intervention device and the users returnto the sequence starting from the beginning. Any unilateraldeviation during the L periods of punishment will trigger


another punishment on the deviating user for L periods. Fig. 2shows an automaton representation of the equilibrium protocoldescribed above assuming that the sequence is a repetition ofa cycle of finite length T .

B. Roles of Intervention in the Folk Theorem

We discuss two roles of intervention in obtaining the folktheorem and provide two examples to illustrate these roles.First, intervention lowers the minmax payoff of each user andthus enlarges the limit set of equilibrium payoffs. In otherwords, by utilizing intervention, we can provide a strongerthreat to deter a deviation, and thus we can achieve moreoutcomes as a SPE. Second, intervention may turn the mutualminmax profile into a NE of the stage game when the mutualminmax profile without intervention is not a NE. Having amutual minmax profile that is a NE simplifies the structureof equilibrium protocols. When we have a mutual minmaxprofile that is a NE, permanent reversion to the mutual minmaxprofile provides the most severe punishment on every user.Thus, we can focus on grim trigger protocols, where anydeviation from a prescribed sequence of action profiles resultsin permanent reversion to the mutual minmax profile. On thecontrary, when the mutual minmax profile is not a NE, there isa user that can increase its stage-game payoff by deviation, andwe use the promise of returning to the equilibrium path aftera finite number of periods to deter users from deviating in apunishment phase. As a result, having a mutual minmax profilethat is a NE makes it simpler for the designer to construct anequilibrium protocol and for the users to execute the protocol.

Example 1 (Power Control): Consider a network where Nusers and the intervention device transmit power in a wirelesschannel. Each user i’s action is its transmit power ai ∈ Ai =[0, ai], and the action of the intervention device is its transmitspower a0 ∈ A0 = [0, a0]. We assume that ai > 0 for all iand a0 > 0. User i’s SINR is calculated by hiiai/(hi0a0 +∑

j �=i hijaj + ni), where hij is the channel gain from userj’s transmitter to user i’s receiver, and ni is the noise powerat user i’s receiver. Each user i’s stage-game payoff is itsthroughput [4][19][39], namely

ui(a0, ai, a−i) = log2

(1 +

hiiaihi0a0 +

∑j �=i hijaj + ni

). (6)

Using any other increasing function of the SINR as the payofffunction does not change the analysis here.

In this power control game, the null intervention action isa0 = 0. Without intervention (i.e., a0 is fixed at a0), the onlyNE of the stage game is (a0, a) = (0, a1, . . . , aN ), whereevery user transmits at its maximum power, and the NE isthe mutual minmax profile with payoff vo = (vo1, . . . , v

oN ) =

u(0, a). With intervention, the mutual minmax profile is(a0, a), which is also a NE, with payoff vw = (vw1 , . . . , v

wN ) =

u(a0, a) < u(0, a). Note that vwi < voi ∀i, and that each vwireduces as a0 increases. Hence, in this example, interventionplays only the first role, expanding the limit set of equilibriumpayoffs as the maximum intervention power a0 increases.

Example 2: Consider a network where N users and theintervention device transmit packets through a single server,which can be modeled as an M/M/1 queue [41]–[43]. User

i’s action is its transmission rate ai ∈ Ai = [0, ai]. Theintervention device also transmits packets at the rate of a0 ∈A0 = [0, a0]. We assume that ai > 0 for all i and a0 > 0.User i’s payoff is a function of its transmission rate ai and itsdelay 1/(μ− a0 −

∑Nj=1 aj) [40]–[43], defined by

ui(a0, ai, a−i) = aβi

i max{0, μ− a0 −

∑Nj=1 aj

}, (7)

where μ > 0 is the server’s service rate, and βi > 0 is theparameter reflecting the trade-off between the transmissionrate and the delay. Here the “max” function indicates the factthat the payoff is zero when the total arrival rate is larger thanthe service rate. We assume that the service rate is no smallerthan the maximum total arrival rate without intervention, i.e.,μ ≥∑N

j=1 aj .Without intervention, the mutual minmax profile is

(0, a1, . . . , aN), where every user transmits at its maximumrate. When there is no intervention, user i’s best response toa−i is given by a∗i = min

{βi

1+βi

(μ−∑j �=i aj

), ai

}. Thus,

the mutual minmax profile (0, a1, . . . , aN ) is a NE if and onlyif

ai ≤ βi

1+βi

(μ−∑j �=i aj

), ∀i ∈ N . (8)

Hence, without intervention, the mutual minmax profile is nota NE if ai/βi > μ −∑N

j=1 aj for some i. However, withintervention, the mutual minmax profile (a0, a1, . . . , aN ) is aNE, as long as the maximum rate a0 of the intervention deviceis high enough to yield

μ−∑j �=i aj − a0 ≤ 0, ∀i ∈ N . (9)

Now we analyze, in the context of the above flow controlgame, the conditions for a protocol to support a constant actionprofile (0, a) as a SPE outcome using punishments of playingthe mutual minmax profile for L periods. As derived in theproof of Proposition 1, the protocol using L as the punishmentlength is a SPE if the discount factor δ satisfies the followingtwo sets of inequalities: first, for all i ∈ N ,

δ + · · ·+ δL (10)

≥(a∗i )

βi

(μ− a∗i −

∑j �=i aj

)− aβi

i

(μ−∑N

j=1 aj

)aβi

i

(μ−∑N

j=1 aj

)− aβi

i ·(μ− a0 −

∑Nj=1 aj

)+ ,

where a∗i = min{ai,

βi

1+βi

(μ−∑j �=i aj

)}and (x)+ �

max{0, x}; second, for all i ∈ N ,

δL (11)

≥(a∗i )

βi ·(μ− a0 − a∗i −

∑j �=i aj

)+aβi

i

(μ−∑N

j=1 aj

)− aβi

i ·(μ− a0 −

∑Nj=1 aj

)+

−aβi

i ·(μ− a0 −

∑Nj=1 aj

)+aβi

i

(μ−∑N

j=1 aj

)− aβi

i ·(μ− a0 −

∑Nj=1 aj

)+ ,

where a∗i = min{ai,

βi

1+βi

(μ− a0 −

∑j �=i aj

)}. Equation

(10) guarantees that no user can gain from deviating fromthe equilibrium path, while equation (11) ensures that no usercan gain from deviating in a punishment phase. If the mutual


0 10 20 30 40 50 60 70 80 90 1000.4

0.5

0.6

0.7

0.8

0.9

1

Length of the punishment phase

Min

imum

dis

coun

t fac

tor

No interventionIntervention with max. rate 0.5Intervention with max. rate 1.0Intervention with max. rate 2.5

Fig. 3. Minimum discount factors to support the pure-action profile thatachieves maximum sum payoff, under different punishment lengths andmaximum intervention flow rates. N = 4. The service rate is μ = 10 bits/s.The maximum flow rates for all the users are 2.5 bits/s. The trade-off factorsare β1 = β2 = 2 and β3 = β4 = 3.

minmax profile is a NE, we can use perpetual reversion to themutual minmax profile as the most severe punishment. In thiscase, executing punishment is self-enforcing, and we requireonly (10) with the left-hand side replaced by δ/(1− δ).

Fig. 3 plots the minimum discount factor required for theprotocol to be a SPE under different punishment lengths Land maximum intervention flow rates a0 in a scenario whereN = 4, μ = 10 bits/s, ai = 2.5 bits/s for all i, β1 = β2 = 2and β3 = β4 = 3. The target payoff is chosen to maximize theproportional fairness (defined as the sum of the logarithms ofpayoffs), which is achieved at the action profile (0, a) wherea1 = a2 = 1.43 bits/s and a3 = a4 = 2.14 bits/s. Whenthe maximum intervention flow rate is 2.5 bits/s, the mutualminmax profile is a NE, and by using permanent reversion toit we obtain the minimum discount factor 0.48. When there isno intervention or when the maximum intervention rate is 0.5bits/s or 1.0 bits/s, the mutual minmax profile is not a NE. Inthis case, given a punishment length, the minimum discountfactor decreases as the the maximum intervention flow rateincreases. In other words, intervention make the protocol aSPE for a wider range of the discount factors by makingpunishment stronger. Thus, this example exhibits both rolesof intervention. We can see that the minimum discount factorincreases with the punishment length beyond a threshold. Thisis because the users need to be more patient to carry out alonger punishment, reflected in (11).

V. DESIGN OF EQUILIBRIUM PROTOCOLS

In Section IV, we have characterized the limit set of equi-librium payoffs V †

w . With this set, we can obtain the optimalequilibrium payoff v� that maximizes the welfare function bysolving v� = argmaxv∈V †

wW (v). Now we will derive the

minimum discount factor under which v� can be achieved byan equilibrium protocol of the structure described in Fig. 2,and construct the corresponding equilibrium protocol.

For analytical tractability, we impose the following threeassumptions on the stage game.

Assumption 1: With intervention, the stage game has amutual minmax profile (a0, a), which is also a NE of thestage game.

Assumption 2: For each user i, there exists an action profileai of the users such that for any j �= i,

vi � maxa ui(a0, a) = ui(a0, ai) and uj(a0, a

i) = 0. (12)

Assumption 3: The set of feasible payoffs is Vw =co{(0, . . . , 0),u(a0, a

1), . . . ,u(a0, aN )}

.Assumption 1 holds in many resource sharing scenarios.

Suppose that each user’s action is a scalar representing itsusage level and that increasing a user’s usage level increasesits own payoff while decreasing the payoffs of all the otherusers. Then each user’s choosing its maximum usage levelis a NE, while it is a mutual minmax profile when theintervention device also chooses its maximum interventionlevel. The power control game discussed in Example 1 satisfiesAssumption 1. The flow control game in Example 2 alsosatisfies Assumption 1 when the intervention flow rate is largeenough to make the mutual minmax profile a NE. The actionprofile ai in Assumption 2 is the one at which user i takes themost advantage of resources. Suppose that there is interferenceamong users and that each user has an action not to utilizethe resources at all. Then full utilization by a user can beachieved only when all the other users do not use the resourcesat all, leading to uj(a0, a

i) = 0 for all j �= i, for all i. Whenthe minimum usage level of some user is positive, we mayhave uj(a0, a) > 0 for some i and j. Our analysis can becarried over to this scenario, but for notational simplicity wechoose to assume uj(a0, a

i) = 0 for all j �= i, for all i.Assumption 2 holds for the power control and flow controlgames in Examples 1 and 2. Lastly, Assumption 3 is likely tohold when there is strong interference among the users.

When the welfare function W is increasing in the payoffsof the users, a solution to the protocol design problem (4)occurs on the Pareto boundary of the limit set of equilibriumpayoffs V †

w . Due to Assumption 3, we can write the Pareto

boundary of V †w by P =

{v ∈ V †

w :∑N

i=1(vi/vi) = 1}

. Fora target payoff v� ∈ P , we derive the sufficient conditionon the discount factor under which v� can be achieved by anequilibrium protocol.

Theorem 1: Given a target payoff v� ∈ P , there existsa discount factor δ(v�), such that for any discount factorδ ≥ δ(v�), there exists an equilibrium protocol that achievesthe target payoff v�. In particular, δ(v�) can be calculatedanalytically by

δ(v�) = max

{maxj �=i

wj − v�jwj − vwj

, (13)

2(N − 1)

(N −R) +√(N −R)2 + 4(R− S)(N − 1)

},

where wj = maxi�=j maxaj uj(a0, aj , ai−j) is the maximum

stage-game payoff that user j can obtain by deviating fromany of the profiles

{(a0, a

i)}i�=j

, R =∑N

i=1(wi/vi), and

S =∑N

i=1(vwi /vi).

Proof: See [37, Appendix C].


TABLE IDISTRIBUTED ALGORITHM RUN BY USER i

Require: The target payoff v� and the discount factor δ

Initialization: τ = 0, v(0) = v�, νi = wi − (wi − vi) · δ,∀i.Repeat

calculates vi(τ)/νi, ∀icalculates i∗ � argmaxi∈N vi(τ)/νiaτ = ai∗ , v(τ + 1) = 1

δv(τ) − 1−δ

δ· u(a0, ai∗)

τ ← τ + 1

Until v(τ) = v�

Note that δ(v�) is increasing with respect to the minmaxpayoffs vwi for all i ∈ N . Hence, a larger intervention capabil-ity decreases the minmax payoffs, which results in a smallerδ(v�), and thus allows a wider range of discount factors.By Theorem 1, we can determine the minimum interventioncapability required to achieve a target payoff v� under agiven discount factor δ, by finding the minimum interventioncapability such that δ = δ(v�). When A0 is one-dimensional(k0 = 1), the intervention capability is determined by themaximum intervention action a0 = argmaxa0∈A0 a0. Sinceδ(v�) is decreasing in a0, we can determine the minimum a0such that δ = δ(v�) efficiently by bisection methods.

Next, for a given target payoff v� ∈ P and a discount factorδ ≥ δ(v�), we construct an equilibrium protocol that achievesv�. Due to Assumption 1, we can use a trigger protocol inwhich the punishment phase lasts forever (L = ∞). Dueto Assumptions 2 and 3, every action profile (at0, a

t) in theequilibrium path {(at0, at)}∞t=0 should be one of the actionprofiles

{(a0, a

i)}Ni=1

.Theorem 2: Given a target payoff v� ∈ P , let {at}∞t=0 be

the sequence of action profiles generated by the algorithm inTable I, and let (σ0, σ) be the trigger protocol in which theintervention device and the users follow {(a0, at)}∞t=0, andany deviation leads to a perpetual play of (a0, a). Then forany discount factor δ ≥ δ(v�), the protocol (σ0, σ) is a SPEof the repeated game with intervention and achieves v�.

Proof: See [37, Appendix D].The algorithm in Table I can be run by each user in a

distributed fashion. In each period, each user does N divisionsand finds the largest one among the results of the N divisions.The overall complexity is O(N), which is fairly small.

Remark 1: Note that any action profile a(t) in the sequenceis from the set of {ai}Ni=1. In other words, only one user takesnonzero action in each period. This greatly simplifies the mon-itoring burden of the intervention device and the users, whenthe users are not strategic in reporting deviation. Actually, ateach period, the inactive users, who take zero actions, do notneed to monitor, because the active user, who takes nonzeroaction at that period, can sense the interference and report tothe intervention device the detection of deviation. Then theintervention device can broadcast the detection of deviation totrigger the punishment. In addition, no inactive user can gainfrom sending false report to the intervention device, becausethe intervention device only trusts the report from the activeuser. Note that if the users are strategic in reporting deviation,an active user may not report deviation voluntarily, if the lossfrom low future payoffs in the punishment phase is larger than

0 5 10 15 200

5

10

15

20

Number of users

Sum

Pay

off

Service rate = Number of users

0 5 10 15 200

0.5

1

Number of users

Abs

olut

e Fa

irnes

s

Service rate = Number of users

0 5 10 15 200

5

10

Number of users

Sum

Pay

off

Service rate limited at 10 bits/s

0 5 10 15 200

0.5

1

Number of users

Abs

olut

e Fa

irnes

s

Service rate limited at 10 bits/s

Fig. 4. Performance comparison among the four schemes with the numberof users increasing. Lines with asterisks: repeated games with intervention,lines with circles: repeated games without intervention, lines with squares:one-shot games with incentive schemes, lines with crosses: Nash equilibriumof the one-shot game.

the loss from low current payoffs due to interference from thedeviating user. In this case, having reports from multiple usersmay be helpful. However, the strategic behavior in reportingdeviation is out of the scope of this paper.

VI. SIMULATION RESULTS

In this section, we consider the flow control game in Exam-ple 2 to illustrate the performance gain of using interventionin repeated games and the analytical result on the trade-offbetween the discount factor and the intervention capability.

A. Performance Gain

First, we compare the performance when the protocoldesigner solves the protocol design problem (4) using fourdifferent schemes, namely greedy algorithms [41]–[43], one-shot games with incentive schemes [13][17][40], repeatedgames without intervention, and repeated games with inter-vention. The two welfare functions we use are the sum payoff∑

i∈N Ui and the max-min fairness mini∈N Ui. When we usethe sum payoff as the welfare function, we impose a minimumpayoff guarantee γi for each user i to prevent the users fromreceiving extremely low payoffs at the solution to the sumpayoff maximization problem. Greedy algorithms achieve theNE, which may not satisfy the minimum payoff guarantees.For one-shot games with incentive schemes, we assume thatthe entire Pareto boundary of the set of pure-action payoffs canbe achieved as NE by using appropriate incentive schemes, inorder to get the best performance achievable by this scheme.

1) Impact of the number of users: We compare how theperformance scales with the number of users under the fourschemes. We study the symmetric case, where all the usershave the same throughput-delay trade-off β = 3 and thesame maximum flow rate normalized to 1 bits/s, such thatthe performance is affected only by the number of users, notby the heterogeneity of the users. We consider two scenarios:


first, the server’s capacity (service rate) increases linearly withthe number of users, i.e. μ = N bits/s, and second, the server’scapacity is limited at 10 bits/s, i.e. μ = min{N, 10} bits/s. Themaximum intervention flow rate is a0 = max{μ−(N−1), 0}bits/s to ensure the mutual minmax profile is an NE. For eachuser i, we set the minimum payoff guarantee as 10% of themaximum stage-game payoff vi, namely γi = 10% · vi bits/s.

Fig. 4 shows the sum payoff and the fairness achieved bydifferent schemes. When the capacity increases linearly withthe number of users N , both the sum payoff and the fairnessincrease with N by using repeated games. In contrast, whenusing one-shot games with incentive schemes, the sum payoffand fairness increase initially when the number of users issmall, and decrease when N > 4. A sum payoff or fairnessof the value 0 means that the minimum payoff cannot beguaranteed. This happens in one shot games with incentiveschemes when N > 5. The NE payoff does not satisfy theminimum payoff guarantee with any number of users.

When the capacity is limited at 10 bits/s, for repeated gameswith intervention, the sum payoff reaches the bottleneck of 10,and due to congestion, the fairness decreases when N > 10.For repeated games without intervention, the sum payoff andthe fairness decrease more rapidly, and the minimum payoffguarantee cannot be met when N ≥ 15. For one-shot games,the trend of the sum payoff and fairness is similar to the casewith increasing capacity.

In conclusion, using repeated games with intervention has alarge performance gain over the other three schemes, in termsof both the sum payoff and the fairness.

2) Impact of minimum payoff guarantees: We compare theperformance of the four schemes under different minimumpayoff guarantees. The system parameters are the same asthose in Fig. 3. The maximum intervention flow rate is a0 =2.5 bits/s to ensure the mutual minmax profile is an NE. We setthe same minimum payoff guarantee for all the users, namelyγi = γj , ∀i, j ∈ N .

Table II shows the sum payoff and the fairness achieved bythe four schemes. We write “N/A” when the minimum payoffguarantee cannot be satisfied. For repeated games, we showthe minimum discount factors allowed to achieve the optimalperformance in the parenthesis next to the performance metric.An immediate observation is the inefficiency of the NE, asexpected.

When using one-shot game model with incentive schemes,the performance loss compared to using repeated games issmall when the minimum payoff guarantee is small (γi = 1).However, when the minimum payoff guarantee increases,using one-shot games is far from optimality in terms of boththe sum payoff and the fairness. Note that using one-shotgames fails to satisfy the large minimum payoff guarantee(γi = 14). In summary, using repeated games has largeperformance gain over using one-shot games in most cases,and is necessary when the minimum payoff guarantee is large.

Under the specific parameters in this simulation, if weallow the discount factor to be sufficiently close to 1, usingintervention in repeated games has little performance gainover repeated games without intervention, especially whenthe minimum payoff guarantee is large. This is because theminmax payoff without intervention is already small, such

0 2 4 6 8 10 12 14 16 180.82

0.84

0.86

0.88

0.9

0.92

0.94

0.96

0.98

1

1.02

Minimum Payoff Gaurantee

Min

imum

Dis

coun

t Fac

tor

No interventionIntervention with max. rate 0.5Intervention with max. rate 1.0Intervention with max. rate 2.5

Min. payoff: 0.5Min. payoff: 1.7

Min. payoff: 4.1

Fig. 5. The trade-offs between the minimum discount factor and theminimum payoff guarantee under different maximum intervention flow rates.At the beginning of each curve, we mark the smallest minimum payoffguarantees we can impose, which indicates the largest feasible set of theprotocol design problem with each maximum intervention flow rate.

0.82 0.84 0.86 0.88 0.9 0.92 0.94 0.96 0.98 1 1.020

0.5

1

1.5

2

2.5

Discount Factor

Min

imum

Inte

rven

tion

Flow

Rat

e R

equi

red

Min. payoff: 3Min. payoff: 5Min. payoff: 7Min. payoff: 10Min. payoff: 14

Fig. 6. The trade-offs between the required maximum intervention flow rateand the discount factor under different minimum payoff guarantees.

that its is Pareto dominated by some large minimum payoffguarantees. In this case, the advantage of using interventionin repeated games is that it allows smaller discount factors toachieve the same or better performance, compared to usingrepeated games without intervention. This is also confirmedby Fig. 5, which shows the minimum discount factor allowedto achieve the target payoff that maximizes the sum payoff,under different minimum payoffs γ and different maximumintervention flow rates.

B. Trade-off among δ, γ, and a0

Consider the same system as that in Fig. 3 and Fig. 5.Suppose that the target payoff is the one that maximizes thesum payoff. In Fig. 6, we plot the trade-off between the re-quired maximum intervention flow rate and the discount factorunder different minimum payoff guarantees. In Fig. 7, we plotthe trade-off between the required maximum intervention rateand the minimum payoff guarantees under different discount


TABLE IIPERFORMANCE COMPARISON AMONG DIFFERENT SCHEMES UNDER DIFFERENT MINIMUM PAYOFFS GUARANTEES

(DISCOUNT FACTORS FOR REPEATED GAMES SHOWN IN THE PARENTHESIS)

Min. payoff Metrics NE [41]–[43] One-shot [13][17][40] Repeated w/o intervention Repeated with intervention

γi = 1Sum Payoff 39.3 110.4 110.2 (1.000) 114.2 (0.987)

Absolute Fairness 4.0 9.6 16.7 (0.861) 16.7 (0.840)

γi = 3Sum Payoff 39.3 85.8 108.2 (1.000) 108.2 (0.962)

Absolute Fairness 4.0 10.6 16.7 (0.861) 16.7 (0.840)

γi = 7Sum Payoff N/A 64.4 96.2 (0.960) 96.2 (0.910)

Absolute Fairness N/A 10.3 16.7 (0.861) 16.7 (0.840)

γi = 14Sum Payoff N/A N/A 75.2 (0.861) 75.2 (0.840)

Absolute Fairness N/A N/A 16.7 (0.861) 16.7 (0.840)

0 2 4 6 8 10 12 14 16 180

0.5

1

1.5

2

2.5

Minimum Payoff

Min

imum

Inte

rven

tion

Flow

Rat

e R

equi

red

Discount factor: 0.85Discount factor: 0.90Discount factor: 0.95

Fig. 7. The trade-offs between the required maximum intervention flow rateand the minimum payoff guarantee under different discount factors.

factors. The protocol designer can use these trade-off curvesas guidelines for determining the maximum intervention flowrate required under different discount factors and minimumpayoff guarantees.

VII. CONCLUSION

In this paper, we proposed a repeated game model gen-eralized by intervention, which can be applied to a largevariety of communication systems. We use repeated gamesto achieve the equilibrium payoffs that Pareto dominate somepayoffs on the Pareto boundary of the nonconvex set offeasible payoffs in one-shot games. In addition, we applyintervention in repeated games, and show that interventionenlarges the limit set of equilibrium payoffs and simplifies theequilibrium protocol. Then we consider the protocol designproblem of maximizing the welfare function. For any targetpayoff, we derive sufficient conditions on the discount factorand the intervention capability under which the target payoffcan be achieved at a SPE. Under the discount factor and theintervention capability satisfying the sufficient condition, weconstruct an equilibrium protocol to achieve the target payoff.Simulation results show the great performance gain, in termsof sum payoff and max-min fairness, by using intervention inrepeated games.

REFERENCES

[1] A. MacKenzie and S. Wicker, “Game theory and the design of self-configuring, adaptive wireless networks,” IEEE Commun. Mag., vol. 39,no. 11, pp. 126–131, Nov. 2001.

[2] E. Altman, T. Boulogne, R. El-Azouzi, T. Jimenez, and L. Wynter, “Asurvey on networking games in telecommunications,” Computers andOperations Research, vol. 33, no. 2, pp. 286–311, Feb. 2006.

[3] J. Huang, R. A. Berry, and M. L. Honig, “Distributed interferencecompensation for wireless networks,” IEEE J. Sel. Areas Commun., vol.24, no. 5, pp. 1074–1084, May 2006.

[4] T. Alpcan, T. Basar, R. Srikant, and E. Altman, “CDMA uplink powercontrol as a noncooperative game,” Wireless Networks, vol. 8, pp. 659–670, 2002.

[5] C. U. Saraydar, N. B. Mandayam, and D. J. Goodman, “Efficient powercontrol via pricing in wireless data networks,” IEEE Trans. Commun.,vol. 50, no. 2, pp. 291–303, Feb. 2002.

[6] J. Huang, R. A. Berry, and M. L. Honig, “Auction-based spectrumsharing,” Mobile Networks and Applications, vol. 11, pp. 405–418, 2006.

[7] C. Long, Q. Zhang, B. Li, H. Yang, X. Guan, “Non-cooperative powercontrol for wireless ad hoc networks with repeated games,” IEEE J. Sel.Areas Commun., vol. 25, no. 6, pp. 1101–1112, Aug. 2007.

[8] A. Schmeink, “On fair rate adaption in interference-limited systems,”European Trans. Telecommun., vol. 22, no. 5, pp. 200–210, Aug. 2011.

[9] D. Wang, C. Comaniciu, and U. Tureli, “Cooperation and fairness forslotted Aloha,” Wireless Personal Commun., vol. 43, no. 1, pp. 13–27,2007.

[10] L. Yang, H. Kim, J. Zhang, M. Chiang, and C. W. Tan, “Pricing-based spectrum access control in cognitive radio networks with randomaccess,” in Proc. 2011 IEEE INFOCOM, pp. 2228–2236.

[11] T. Basar and R. Srikant, “Revenue-maximizing pricing and capacityexpansion in a many-users regime,” in Proc. 2002 IEEE INFOCOM,pp. 1556–1563.

[12] H. Shen and T. Basar, “Optimal nonlinear pricing for a monopolisticnetwork service provider with complete and incomplete information,”IEEE J. Sel. Areas Commun., vol. 25, pp. 1216–1223, Aug. 2007.

[13] J. Park and M. van der Schaar, “The theory of intervention gamesfor resource sharing in wireless communications,” IEEE J. Sel. AreasCommun., vol. 30, no. 1, pp. 165–175, Jan. 2012.

[14] J. Park and M. van der Schaar, “Stackelberg contention games inmultiuser networks,” EURASIP J. Advances in Signal Process., vol.2009, pp. 1–15, 2009.

[15] J. Park and M. van der Schaar, “Designing incentive schemes basedon intervention: the case of imperfect monitoring,” in Proc. 2011GameNets.

[16] Y. Xiao, J. Park, and M. van der Schaar, “Intervention in power controlgames with selfish users,” IEEE J. Sel. Topics Signal Process., vol. 6,no. 2, pp. 165–179, Apr. 2012.

[17] Y. Gai, H. Liu, and B. Krishnamachari, “A packet dropping-basedincentive mechanism for M/M/1 queues with selfish users,” in Proc.2011 IEEE INFOCOM, pp. 2687–2695.

[18] S. Stanczak and H. Boche, “On the convexity of feasible QoS regions,”IEEE Trans. Inf. Theory, vol. 53, no. 2, pp. 779–783, Feb. 2007.

[19] E. G. Larsson, E. A. Jorswieck, J. Lindblom, and R. Mochaourab,“Game theory and the flat-fading Gaussian interference channel,” IEEESignal Process. Mag., vol. 26, no. 5, pp. 18–27, Sep. 2009.

[20] J. Park and M. van der Schaar, “Medium access control protocols withmemory,” IEEE/ACM Trans. Netw., vol. 18, no. 6, pp. 1921–1934, Dec.2010.

[21] G. J. Mailath and L. Samuelson, Repeated Games and Reputations:Long-run Relationships. Oxford University Press, 2006.


[22] K. L. Judd, S. Yeltekin, and J. Conklin, “Computing supergame equi-libria,” Econometrica, vol. 71, no. 4, pp. 1239–1254, July 2003.

[23] S. Goldlucke and S. Kranz, “Infinitely repeated games with publicmonitoring and monetary transfers,” J. Economic Theory, forthcoming.

[24] G. J. Mailath, I. Obara, and T. Sekiguchi, “The maximum efficientequilibrium payoff in the repeated prisoners’ dilemma,” Games andEconomic Behavior, vol. 40, pp. 99–122, 2002.

[25] R. J. La and V. Anantharam, “Optimal routing control: repeated gameapproach,” IEEE Trans. Autom. Control, vol. 47, no. 3, pp. 437–450,Mar. 2002.

[26] R. Etkin, A. Parekh, and D. Tse, “Spectrum sharing for unlicensedbands,” IEEE J. Sel. Areas Commun., vol. 25, no. 3, pp. 517–528, Apr.2007.

[27] G. Theodorakopoulos and J. S. Baras, “Game theoretic modeling ofmalicious users in collaborative networks,” IEEE J. Sel. Areas Commun.,vol. 26, no. 7, pp. 1317–1327, Sep. 2008.

[28] C. Pandana, Z. Han, and K. J. R. Liu, “Cooperation enforcementand learning for optimizing packet forwarding in autonomous wirelessnetworks,” vol. 7, no. 8, pp. 3150–3163, Aug. 2008.

[29] Y. Wu, B. Wang, K. J. R. Liu, and T. C. Clancy, “Repeated open spec-trum sharing game with cheat-proof strategies,” IEEE Trans. WirelessCommun., vol. 8, no. 4, pp. 1922–1933, Apr. 2009.

[30] Z. Ji, W. Yu, and K. J. R. Liu, “A belief evaluation framework inautonomous MANETs under noisy and imperfect observation: vul-nerability analysis and cooperation enforcement,” IEEE Trans. MobileComput., vol. 9, no. 9, pp. 1242–1254, Sep. 2010.

[31] L. Jiang, V. Anantharam, and J. Walrand, “How bad are selfish invest-ments in network security?” IEEE/ACM Trans. Netw., vol. 19, no. 2, pp.549–560, Apr. 2011.

[32] B. Niu, H. V. Zhao, and H. Jiang, “A cooperation stimulation strategyin wireless multicast networks,” IEEE Trans. Signal Process., vol. 59,no. 5, pp. 2355–2369, May 2011.

[33] M. Felegyhazi, J. P. Hubaux, and L. Buttyan, “Nash equilibria of packetforwarding strategies in wireless ad hoc networks,” IEEE Trans. MobileComput., vol. 5, no. 5, pp. 463–476, May 2006.

[34] Z. Han, Z. Ji, and K. J. R. Liu, “A cartel maintenance frameworkto enforce cooperation in wireless networks with selfish users,” IEEETrans. Wireless Commun., vol. 7, no. 5, pp. 1889–1899, May 2008.

[35] M. LeTreust and S. Lasaulce, “A repeated game formulation of energy-efficient decentralized power control,” IEEE Trans. Wireless Commun.,vol. 9, no. 9, pp. 2860–2869, Sep. 2010.

[36] R. B. Myerson, “Utilitarianism, egalitarianism, and the timing effect insocial choice problems,” Econometrica, vol. 49, no. 4, pp. 883-897, June1981.

[37] Y. Xiao, J. Park, and M. van der Schaar, “Repeated games withintervention: theory and applications in communications,” Tech. Report.Available: http://arxiv.org/abs/1111.2456.

[38] J. E. Hopcroft, R. Motwani, and J. D. Ullman, Introduction to AutomataTheory, Languages, and Computation. Addison Wesley, 2006.

[39] M. Chiang, P. Hande, T. Lan, and C. W. Tan, “Power control in wireless

cellular networks,” Foundations and Trends in Networking, vol. 2, no.4, pp. 381–533, Apr. 2008.

[40] Y. Su and M. van der Schaar, “Linearly coupled communication games,”IEEE Trans. Commun., vol. 59, no. 9, pp. 2543–2553, Sep. 2011.

[41] K. Bharath-Kumar and J. M. Jaffe, “A new approach to performanceoriented flow control,” IEEE Trans. Commun., vol. 29, pp. 427–435,1981.

[42] C. Douligeris and R. Mazumdar, “A game theoretic perspective to flowcontrol in telecommunication networks,” J. Franklin Institute, vol. 329,no. 2, pp. 383–402, 1992.

[43] Z. Zhang and C. Douligeris, “Convergence of synchronous and asyn-chronous greedy algorithm in a multiclass telecommunications environ-ment,” IEEE Trans. Commun., vol. 40, no. 8, pp. 1277–1281, Aug. 1992.

Yuanzhang Xiao received the B.E. and M.E. degree in electrical engineeringfrom Tsinghua University, Beijing, China, in 2006 and 2009, respectively.He is currently pursuing the Ph.D. degree in the Electrical EngineeringDepartment at University of California, Los Angeles. His research interestsinclude game theory, optimization, communication networks, and networkeconomics.

Jaeok Park received the B.A. degree in economicsfrom Yonsei University, Seoul, Korea, in 2003, andthe M.A. and Ph.D. degrees in economics fromthe University of California, Los Angeles, in 2005and 2009, respectively. He is currently an AssistantProfessor in the School of Economics at YonseiUniversity, Seoul, Korea. From 2009 to 2011, he wasa Postdoctoral Scholar in the Electrical EngineeringDepartment at the University of California, Los An-geles. From 2006 to 2008, he served in the Republicof Korea Army. His research interests include game

theory, mechanism design, network economics, and wireless communication.

Mihaela van der Schaar is Chancellor’s Professor of Electrical Engineeringat University of California, Los Angeles. She is an IEEE Fellow, a Distin-guished Lecturer of the Communications Society for 2011-2012 and the Editorin Chief of IEEE Transactions on Multimedia. She received an NSF CAREERAward (2004), the Best Paper Award from IEEE Transactions on Circuits andSystems for Video Technology (2005), the Okawa Foundation Award (2006),the IBM Faculty Award (2005, 2007, 2008), the Most Cited Paper Award fromEURASIP: Image Communications Journal (2006), the Gamenets ConferenceBest Paper Award (2011) and the 2011 IEEE Circuits and Systems SocietyDarlington Award Best Paper Award. She holds 33 granted US patents. Formore information about her research visit: http://medianetlab.ee.ucla.edu/

repeated games with intervention: theory and applications in communications

Documents