non-inferior nash strategies for routing control in parallel-link communication networks

INTERNATIONAL JOURNAL OF COMMUNICATION SYSTEMSInt. J. Commun. Syst. 2005; 18:347–361Published online 15 March 2005 in Wiley InterScience (www.interscience.wiley.com). DOI: 10.1002/dac.708

Non-inferior Nash strategies for routing control in parallel-linkcommunication networks

Yong Liu1,z and Marwan A. Simaan2,n,y

1Department of Electrical and Computer Engineering, The Ohio State University, Columbus, OH 43210, U.S.A.2Department of Electrical Engineering, University of Pittsburgh, Pittsburgh, PA 15261, U.S.A.

SUMMARY

We consider a routing control problem of two-node parallel-link communication network shared bycompetitive teams of users. Each team has various types of entities (traffics or jobs) to be routed on thenetwork. The users in each team cooperate for the benefit of their team so as to achieve optimal routingover network links. The teams, on the other hand, compete among themselves for the network resourcesand each has an objective function that relates to the overall performance of the network. For each team,there is a centralized decision-maker, called the team leader or manager, who coordinates the routingstrategies among all entities in his team. A game theoretic approach to deal with both cooperation withineach team and competition among the teams, called the Non-inferior Nash strategy, is introduced.Considering the roles of a group manager in this context, the concept of a Non-inferior Nash strategy witha team leader is introduced. This multi-team solution provides a new framework for analysinghierarchically controlled systems so as to address complicated coordination problems among the varioususers. This strategy is applied to derive the optimal routing policies for all users in the network. It is shownthat Non-inferior Nash strategies with a team leader is effective in improving the overall networkperformance. Various types of other strategies such as team optimization and Nash strategies are alsodiscussed for the purpose of comparison. Copyright # 2005 John Wiley & Sons, Ltd.

KEY WORDS: communication networks; parallel link networks; routing control; game theory; Nashstrategies; non-inferior strategies

1. INTRODUCTION

The problem of routing is encountered in all and every communication network shared by alarge number of users. In many cases, it is necessary to multiplex the resources of informationflow in order to assign dedicated links of sufficient capacity to all the users to meet their needs.Traditionally, the network was designed and built as a single entity with a single objective under

Contract/grant sponsor: DARPA; contract/grant number: F33615-01-C3151

Received 4 May 2004Revised 4 November 2004

Accepted 18 November 2004Copyright # 2005 John Wiley & Sons, Ltd.

yE-mail: [email protected]: [email protected]

nCorrespondence to: Marwan A. Simaan, Department of Electrical Engineering, University of Pittsburgh, Pittsburgh,PA 15261, U.S.A.

the assumption that users were passive and would cooperate for the overall performance ofrouting in the entire network. In modern communication networks, however, this assumption ofa single administration is no longer valid since the users typically have various, evencontradictory, performance measures and demands. One possible way of managing such anetwork is to let the individual users compete with each other in a way that allows each of themto reach an optimal working state. In such a situation, users change their control strategiesbased on the state of the network. The change of strategy taken by one user is likely to causechanges in other users’ strategies, resulting in a dynamic, continuously changing network. Theoutcome of such a system is therefore heavily dependent on the actions taken by all the users,and thus its overall optimization can best be analysed within the framework of game theory.

The literature on the analysis of competitive routing control problems using game theory isvery rich. The routing problems in communication networks shared by selfish users aremodelled as non-cooperative games in References [1–5] or non-cooperative repeated games inReference [6]. The Nash equilibrium, a main concern in References [1–6], ensures that no userfinds it beneficial to change his behaviour unilaterally. Conditions for existence and uniquenessof Nash equilibria are presented in their work based on various types of cost functions for theusers, such as polynomial link holding functions in Reference [1], utility functions in the form of‘throughput/delay’ in Reference [2], utility function in the form of ‘throughput–delay’ inReference [3], communication quality functions in Reference [4], and average delay functions inReferences [5, 6]. However, Nash equilibria are generically inefficient and exhibit suboptimalnetwork performance. This deficiency can be overcome with the intervention of a network agentsuch as a service provider. Stackelberg strategies have been applied to address this issue inReferences [7, 8]. Considering a network manager who acts as a Stackelberg leader and controlsa portion of the network flow, Korilis et al. [7] derived necessary and sufficient conditions for theexistence of a maximally efficient strategy for a manager to drive the network into a globaloptimum. Note that the leader considered in Reference [7] is a special user in the system, andhence the problem is not formulated in a hierarchical structure. Basar et al. [8] introduced ahierarchical structure in the network by assuming that there is a Stackelberg leader, who sets theprice per unit of bandwidth and multiple Nash followers, who decide on their flow rates. Inother words, the leader wishes to maximize the total revenue the network collects and thefollowers (other users) choose their levels of flow by maximizing an objective function thatrepresents a tradeoff between the disutility of payment to the leader and congestion costs on thelink they use. The main observation in this work is that the revenue-incentive for the networkincreases the available capacity (or decreases the delay) in proportion to the number of users inthe network. In References [7, 8], however, only one service provider is considered.

In this paper, we consider a more complicated organization of the network where severalcompetitive teams of users share the network resources. We assume that each team has adecision-maker, usually called the team leader or manager, who centralizes all decisions for thatteam. A practical example of such an organization is a set of companies, each with differentclasses of traffic requirements, such as data, audio, image or video, using wireless local areanetworks and all sharing the same internet resource to send their data. The network manager foreach company attempts to optimize the performance of all traffic sent from his company. At thesame time, the team leaders may have no choice but to compete with each other so as to bestserve their own users over the network. One natural way of managing such a network is to allowthe users belonging to the same team to cooperate with each other and to let the team leaderscompete with each other and settle to an equilibrium where each team reaches optimum

Copyright # 2005 John Wiley & Sons, Ltd. Int. J. Commun. Syst. 2005; 18:347–361

Y. LIU AND M. A. SIMAAN348

operating point. We note that a similar structure is considered in Reference [9]; however, eachteam leader has an objective which is the average of the objectives of all users in his team. Theproblem of interest to us in this paper is when each team leader has his own objective function,which is completely independent from those of the users in his team. This problem can bemodelled as a multi-team game [10] with an extended hierarchical structure as shown inFigure 1. A Non-inferior Nash strategy with a leader (NNSL) [10] has been developed to tacklethe optimization of such a system.

In this paper, we will apply NNSL to a simple network consisting of a common source and acommon destination node interconnected by a number of parallel links. Our main goal is todevise a control scheme for the network and investigate the effectiveness of the NNSL in theproblem of splitting the data flow among the link resource. The outline of the paper is asfollows. In Section 2, we present a generic parallel-link network model and formulate therouting problem within the framework of a multi-team hierarchical structure. In Section 3, weintroduce the concepts of Non-inferior Nash strategy (NNS) and the NNSL, and derive anecessary and sufficient condition that guarantees the existence of a solution to the routingproblem. In Section 4, we consider a network with only one team of users. In Section 5, weconsider a more general network with two teams of users. Finally, in Section 6, we present someconcluding remarks.

2. MODEL AND PROBLEM FORMULATION

We consider a set N ¼ f1; . . . ;Ng of team leaders, each managing several types of users thatshare a set L ¼ f1; . . . ;Sg of communication links, interconnecting a common source and acommon destination node. Let cj be the capacity (or service rate) of link j; c ¼ ðc1; . . . ; cSÞ thecapacity configuration, and C ¼

PSj¼1 cj total capacity of the system of parallel links. Without

loss of generality, suppose that c15c25 � � �5cS: Let MX denote team leader X ðX ¼ 1; . . . ;NÞwho serves nX users and the ith ði ¼ 1; . . . ; nXÞ user has a throughput demand that is a Poissonprocess with average rate lXi > 0: Let l ¼

PNX¼1

PnXi¼1 lXi be the total throughput demand of all

users in the networks. Furthermore, for stability reasons it is supposed that the total throughputdemand is less than the total capacity of parallel links, i.e. l5C: The ith user using the servicemanaged by MX ships its flow by splitting its demand lXi over the set of parallel links. Let fXi ðjÞ

Routing in Communication

Network

Users Cooperation

Organizationalrelationship

Interaction

Team

Competition

Figure 1. Diagram of hierarchical structure in network routing.


NON-INFERIOR NASH STRATEGIES 349

denote the expected fraction of flow that user i from MX sends on link j: The user flow fractionconfiguration

fXi ¼ ðfXi ð1Þ; . . . ; fXi ðSÞÞ ð1Þ

is called a routing strategy of user i from MX and the set

FXi ¼

fXi 2 RS : 04lXi fXi ðjÞ4cj;

PSj¼1 fXi ðjÞ ¼ 1; 04fXi ðjÞ41; j 2 L

8><>:

9>=>; ð2Þ

of strategies that satisfy the user’s demand is called the strategy space of user i from MX: Therouting control profile for the users from MX is denoted by

fX ¼ ðfX1 ; . . . ; fXnXÞ ð3Þ

and takes values in the product strategy space

FX ¼ �nXi¼1F

Xi ð4Þ

The system routing control profile is given by

f ¼ ðf1; . . . ; fNÞ ð5Þ

and takes values in the overall product strategy space

F ¼ �NX¼1F

X ð6Þ

The details of such a system are shown in Figure 2.The user i from MX has a certain routing decision fXi ¼ ðfXi ð1Þ; . . . ; f

Xi ðSÞÞ to make for the

purpose of, for example, minimizing average delay time. Consider the average delay for eachuser as a cost function. Without loss of generality, we let the service requirement of each user be

Figure 2. Two-node parallel-link communication network with multiple teams of users.



exponentially distributed with mean 1. We concentrate on the M=M=1 delay function [11] dðjÞon link 1 ðj 2 LÞ

dðjÞ ¼

1

cj �PN

X¼1

PnXi¼1 lXi f

Xi ðjÞ

PNX¼1

PnXi¼1 lXi f

Xi ðjÞ5cj

1PN

X¼1

PnXi¼1 lXi f

Xi ðjÞ5cj

8>><>>: ð7Þ

Thus, the total delay for user i from MX is

dXi ¼XSj¼1

lXi fXi ðjÞdðjÞ ð8Þ

The average delay, i.e. the cost function, for user i from MX the control strategy profile fij to beminimized is given by

JXi ðfÞ ¼dXilXi

¼XSj¼1

fXi ðjÞdðjÞ ð9Þ

where JXi : F ! R and, obviously, this cost function depends on the control strategies of otherusers as well. The team leaders may have various forms of objective functions. Let these bedenoted by PX : FX ! R: In this paper, we consider two types of objective functions for teamleaders: efficiency objective function (type 1) and flow cost objective function (type 2). A teamleader with an objective function of type 1 wants to maximize the efficient utilization of thehighest capacity link, which is given by

type 1: PXðfXÞ ¼XnXi¼1

lXi fXi ðSÞ ð10aÞ

A team leader with an objective function of type 2 wants to minimize the total cost of flow forhis users. Let pXðjÞ be the cost paid by users from MX for their flow on link j and that MX wishesto minimize the total cost of the flow for MX given by

type 2: PXðfXÞ ¼XSj¼1

pXðjÞXnXi¼1

lXi fXi ðjÞ

!ð10bÞ

The optimal routing problem is formulated as

maxfX

PXðfXÞ for type 1 fX 2 FX;X 2 N

or minfX

PXðfXÞ for type 2 fX 2 FX;X 2 N

ð11aÞ

for each team leader in the system;

s:t: minfXi

JXi ðfÞ f 2 F; fXi 2 FXi ; X 2 N; i ¼ 1; . . . ; nX ð11bÞ

for each user from MX:Note that when N ¼ 1; the above problem is a team optimization problem.



3. NON-INFERIOR NASH STRATEGIES FOR ROUTING PROBLEMS

The two-node parallel-link communication network discussed in the previous section is a typicalexample of multi-team systems [10] that are controlled by several competing teams of decision-makers, with each team consisting of several cooperating decision-makers. The optimization ofa multi-team system must be done within a framework that combines team theory [12] withgame theory [13]. Focusing on routing problems, we refer to this framework as non-zero-summulti-team routing games. NNS [10] have been developed for cooperation among all memberswithin each team and ensure a non-cooperative Nash equilibrium among all teams. We refer tothis strategy as the Non-inferior Nash solution.

Without loss of generality, and for the sake of simplicity of notation, in this paper we willconsider network systems where there are only two managers: M1 and M2: Systems with morethan two teams can be treated in a very similar manner.

Definition 3.1The pair of routing control vectors f #ff 1; #ff 2g 2 F1 � F 2 is an NNS in a two-team routing game if,for any other f 1 2 F1 and f 2 2 F 2;

fJ1i ðf1; #ff 2Þ4J1i ð #ff

1; #ff 2Þ; i ¼ 1; . . . ; n1g only iffJ1i ðf1; #ff 2Þ ¼ J1i ð #ff

1; #ff 2Þ; i ¼ 1; . . . ; n1g ð12Þ

and

fJ2i ð #ff1; f 2Þ4J2i ð #ff

1; #ff 2Þ; i ¼ 1; . . . ; n2g only iffJ2i ð #ff1; f 2Þ ¼ Ji

2ð #ff 1; #ff 2Þ; i ¼ 1; . . . ; n2g ð13Þ

Each condition in the above definition requires that the routing control vector chosen byone team (say #ff 1 for users from M1) be a non-inferior solution against the control vector chosenby the other team ( #ff 2 for users from M2). Additionally, a pair of routing control vectors f #ff 1; #ff 2gthat satisfy conditions (12) and (13) simultaneously will also represent a Nash equilibriumsolution between the two teams of users. In other words, the NNS satisfies the following twoproperties:

Property 3.1Within each manager MX; the control vector #ff X is a non-inferior (or Pareto) strategy for usersfrom MX; and

Property 3.2Between the two teams of users, the pair of control vectors f #ff 1; #ff 2g is a Nash equilibriumstrategy.

Thus, with this pair of strategies f #ff 1; #ff 2g there is no incentive for the users in one team tocollectively deviate, since this will not improve the objective functions of all users of that teamsimultaneously, but instead will cause a deterioration in the overall team’s performance.

The counterpart of the traditional reaction set of game theory when figuring out a NNS iscalled the Non-inferior reaction set (NRS) and is defined as follows:

Definition 3.2The map R2

NRS½ f1� : F 1 ! F 2 is defined as the Non-inferior reaction set for the users from M2:

If given any arbitrary routing control vector f 1 2 F1 for the users from M1; the routing control



vector f 2 2 R2NRS½ f

1� satisfies

fJ2i ðf1; f 2#Þ4J2i ðf

1; f 2Þ; i ¼ 1; . . . ; n2g only if fJ2i ðf1; f 2#Þ ¼ Ji

2ðf 1; f 2Þ; i ¼ 1; . . . ; n2g ð14Þ

for all f 2# 2 F2:In a similar way, we can define R1

NRS½ f2� : F2 ! F1 as the NRS for the users from M1: Thus,

the NRS for the users from M2 is equivalent to the collection of all non-inferior control sets forthe users from M2 for all possible choices of control vectors by the members of M1:

For the purpose of simplifying the notation, when one manager is denoted by X; we will use %XXto denote the other manager, and vice versa, i.e.

%XX ¼2 when X ¼ 1

1 when X ¼ 2

8<: ð15Þ

Let us assume that manager M %XX has chosen a team control f%XX; and the corresponding NRS

for the users from MX can be determined by minimizing the function [10]

JX;xX

ðf 1; f 2Þ ¼XnXi¼1

xXi JXi ðf

1; f 2Þ ð16Þ

with respect to f X for every vector of parameters xX ¼ ðxX1 ; xX2 ; . . . ; x

XnXÞ0 2 WX where WX is given

by

WX ¼xX 2 RnX :

PnXi¼1 xXi ¼ 1; 04xXi 41;

xX ¼ ðxX1 ; xX2 ; . . . ; x

XnXÞ0

8<:

9=; ð17Þ

Let CXNRSðx

X; f%XXÞ denote the set of solutions f X;x

X

to the optimization problem given in (16) and

parameterized by xX: We now give a definition of the NNS in terms of the vector x ¼ x1

x2

� �;

followed by a theorem that provides necessary conditions for its existence.

Definition 3.3For a given vector x; the pair of team control vectors f #ff 1;x; #ff 2;xg is a NNS to a two-team routinggame if

#ff 1;x 2 C1NRSðx

1; #ff 2;x2

Þ and #ff 2;x 2 C2NRSðx

2; #ff 1;x1

Þ ð18Þ

Theorem 3.1 (Liu and Simaan [10] (Existence of a NNS in two-team routing games))For each team X 2 f1; 2g; F 1 � F 2 is a compact and convex subset of RLðn1 þ n2Þ: If the costfunctionals JXi ðf

1; f 2Þ : F 1 � F 2 ! R for i ¼ 1; . . . ; nX are jointly continuous in f 1 andf 2 2 F1 � F2; and strictly convex in fX for every f

%XX 2 F%XX; then for every vector of weights x ¼

x1

x2

� �there exists a Non-inferior Nash solution.

As discussed before, for every vector of weights x ¼ x1

x2

� �there exists a Non-inferior Nash

solution to a two-team routing game without considering the intervention of managers. We theninvolve the manager’s cost functions as a mechanism for the managers to select a strategy fromthis set. We call this strategy the Non-inferior Nash solution with a leader.



As can be seen from the analysis above, how each team chooses a specific solution from itsnon-inferior set (or how it chooses the weight vector xX) is critical in determining the resultingNNS. While this choice can be done by a mutual (enforceable) agreement among all teammembers, in some cases there may exist a team leader whose responsibility is to make thatchoice. Furthermore, the manager may use a different cost function as a criterion for makingthis choice. If all the managers’ cost functions depend on the control variables of all users, then agame situation will also exist among the managers and the specific choices of non-inferiorsolutions will have to be made based on a game theoretic approach. This situation actuallyoccurs in many real applications such as in cooperative teaming of autonomous entities (UAVs,robots, etc.), in the control of ancillary services in future energy distribution grids, as well as inthe management of computer communication networks. In this section, we will address thisissue in the context of the two-team routing game where each manager has the task of choosingthe team’s weight vector xX: As before, we will consider only the case of two teams of users. Wewill assume that the manager MX chooses xX 2 SX � WX so as to minimize his cost functionPXð #ff X;xÞ ¼ #PPXðx1; x2Þ: The subset SX corresponds to the values of parameters xX 2 WX for whicha NNS solution exists. Within this new structure, we give the following definition of the NNSLto a two-team routing game.

Definition 3.4The pair of strategies f #ff 1;#xx1; #ff 2;#xx2g is a NNSL to a two-team routing game if there exists a pairf#xx1; #xx2g such that

#PP1ð#xx1; #xx2Þ4 #PP1ðx1; #xx2Þ for all x1 2 S1

#PP2ð#xx1; #xx2Þ4 #PP2ð#xx1; x2Þ for all x2 2 S2

ð19Þ

In other words, the pair f #ff 1;#xx1

; #ff 2;#xx2g is an NNSL if the pair of weight vectors f#xx1; #xx2g results in a

Nash equilibrium between the objective functions of the two leaders.In some special cases, each leader will be faced with a simple optimization problem rather

than a game with the other leader. This situation will occur in a two-team system where oneteam has only one decision-maker as illustrated in the example from microeconomics [10]. Ingeneral, as we can see from the definition above, when each leader’s objective function isexpressed in terms of the weight vectors, the resulting functions may end up depending on theweight vector of that leader’s team only. In other words, on the higher level, a non-cooperativegame exists between two leaders who select the appropriate control variables xX 2 SX in order toimprove their own objectives. Nash strategy is a reasonable solution to such a game.

4. TEAM OPTIMIZATION FOR SINGLE-TEAM ROUTING CONTROL PROBLEMS

Before applying NNSL, let’s consider the team optimization problem [12] in routing control, i.e.N ¼ 1: For simplicity, we consider two users with the throughput demand of l1 and l2;respectively, and two parallel links in the system with capacities of c1 and c2; respectively. Let xand y denote the fraction of flow demand of user 1 and user 2, respectively, that will be assignedto link 1. According to constraints in (2), 1� x (or 1� y) is the fraction of flow demand of theuser 1 (or user 2) that will be assigned to link 2. The system is illustrated in Figure 3.



As expressed in (9) with delay function calculated from expressions (7) and (8), and withN ¼ 1; nX ¼ 2; f 11ð1Þ ¼ x; f 12ð1Þ ¼ y; f 11ð2Þ ¼ 1� x; f 12ð2Þ ¼ 1� y; l11 ¼ l1; l12 ¼ l2; the costfunction Ji for user i is given by

J1ðx; yÞ ¼x

c1 � l1x� l2yþ

1� x

c2 � l1ð1� xÞ � l2ð1� yÞð20Þ

and

J2ðx; yÞ ¼y

c1 � l1x� l2yþ

1� y

c2 � l1ð1� xÞ � l2ð1� yÞð21Þ

In the team optimization problem, both users can cooperate with each other and there is amanager for the system, whose objective is to maximize the efficient usage of the link with highcapacity (objective function of type 1). The objective function for the team leader is given by

JMðx; yÞ ¼ l1ð1� xÞ þ l2ð1� yÞ ð22Þ

The team optimization problem can be formulated as

maxx;y

JMðx; yÞ ð23Þ

s:t: minx

J1ðx; yÞ and miny

J2ðx; yÞ ð24Þ

c1 � l1x� l2y > 0 and c2 � l1ð1� xÞ � l2ð1� yÞ > 0 ð25Þ

04x; y41 ð26Þ

The cost functions J1ðx; yÞ and J2ðx; yÞ are convex with respect to x and y over the convex spacegiven by (25) and (26). Thus, the optimal solution for (24) can be determined by minimizing aweighted scalar-valued cost function Jðx; y; aÞ [14] as follows:

minx;y

Jðx; y; aÞ ¼ aJ1ðx; yÞ þ ð1� aÞJ2ðx; yÞ ð27Þ

where a is a weight factor satisfying 04a41:As we know, for each a; there exists an optimal solution ðxnðaÞ; ynðaÞÞ: Since the cost function

of the manager on the higher level is also determined by the optimal controls of x by user 1 andof y by user 2, JMðx; yÞ becomes a function of weight factor a: In other words, the objectivefunction of the manager is used to decide the optimal choice of a:

As a numerical example, let p1 ¼ 400; p2 ¼ 100; l1 ¼ 1; l2 ¼ 3; c1 ¼ 3 and c2 ¼ 6; where pi isthe cost paid by user i ði ¼ 1; 2Þ: The convex set given by (25) and (26) is expressed as the blue-

1λ

2λ

1 2x yλ λ+Link 1

1 2(1 ) (1 )x yλ λ+ −−Link 2

DS

Figure 3. Single-team routing problem.



shaded area shown in Figure 4. The cost function of user 1, J1ðx; yÞ; is given in Figure 5. Weobserve that J1ðx; yÞ is convex with respect to the convex set given by (25). However, theobjective function for user 1, (i.e. the average delay), is extremely large with respect to thedecisions around the boundaries c1 � l1x� l2y ¼ 0 and c2 � l1ð1� xÞ � l2ð1� yÞ ¼ 0: There-fore, in practice, user 1 has to avoid the use of those decision choices. The objective functions

Figure 4. Convex set of the given example.

Figure 5. Convex cost function J1ðx; yÞ:



J1ðx; yÞ and J2ðx; yÞ in reasonable areas are given in Figure 6. After figuring out all the possiblecooperative controls for both users, i.e. ðxnðaÞ; ynðaÞÞ for all a’s, we substitute these solutions to(22) to calculate the optimal value of JM: The result is shown in Figure 7: an ¼ 0:25;xnðanÞ ¼ 0:03; ynðanÞ ¼ 0:3; Jn1 ¼ 0:3456; Jn2 ¼ 0:3838 and JnM ¼ 3:07:

Figure 6. Cost functions J1ðx; yÞ and J2ðx; yÞ in reasonable areas.

Figure 7. Objective function for the manager w.r.t. different values of weight factor.



5. NNSL FOR MULTI-TEAM ROUTING CONTROL PROBLEMS

In this section, we consider the multi-team routing problem. For simplicity, assume there aretwo teams, called HET and TELE, respectively, and each team has two users as well. Weconsider a two-node parallel-link communication network as before. The total system capacityis C ¼

P2X¼1 cX: Let the throughput demand of user i from HET arrive at the system with rate

lHi ði ¼ 1; 2Þ: The total throughput demand for the users from HET is lH ¼P2

i¼1 lHi : Thefractions of flow of user 1 and user 2 from HET assigned to link 1 are x ð2 ½0; 1�Þ and y ð2 ½0; 1�Þ;respectively. Let the throughput demand of user j served by TELE arrive at the system with ratelTj ðj ¼ 1; 2Þ: The total throughput demand for TELE customers is lT ¼

P2j¼1 lTj : The fractions

of flow of user 1 and user 2 from TELE assigned to link 1 are u ð2 ½0; 1�Þ and v ð2 ½0; 1�Þ;respectively. Furthermore, we only consider the total capacity that can accommodate the totaluser demand, i.e. lN þ lT4C: The whole system is illustrated in Figure 8.

As before, each user wants to minimize the average delay in the system. It can be formulatedas the following optimal problem.

For the users from HET

minx

JH1 ðx; y; u; vÞ ¼x

gðx; y; u; vÞþ

1� x

hðx; y; u; vÞð28Þ

miny

JH2 ðx; y; u; vÞ ¼y

gðx; y; u; vÞþ

1� y


and, for the users from TELE,

minu

JT1 ðx; y; u; vÞ ¼u

gðx; y; u; vÞþ

1� u


minv

JT2 ðx; y; u; vÞ ¼v

gðx; y; u; vÞþ

1� v


s:t: gðx; y; u; vÞ > 0 and hðx; y; u; vÞ > 0 ð32Þ

04x; y; u; v41 ð33Þ

where

gðx; y; u; vÞ ¼ C1 � lH1 x� lH2 y� lT1 u� lT1 v

and

hðx; y; u; vÞ ¼ C2 � lH1 ð1� xÞ � lH2 ð1� yÞ � lT1 ð1� uÞ � lT1 ð1� vÞ

1Hλ

1Tλ

2Tλ

HET

TELE

D 2Hλ

S

Link 1

Link 2

Figure 8. Two-team routing problem.



Clearly, this optimal problem can be formulated as a multi-team game with N ¼ 2 andn1 ¼ n2 ¼ 2: The solution to this problem is a NNS. The average delay objective functions JHiand JTj ði; j ¼ 1; 2Þ are strictly convex over the convex space given by (32) and (33). According toTheorem 3.1, there exists a NNS under a given weight vector x ¼ ½xN ¼ ða; 1� aÞ;xT ¼ ðb; 1� bÞ� to the routing problem for the users served by the two managers. The weightedscalar-valued objective functions for the users from the two managers are given by

JHðaÞ ¼ aJH1 þ ð1� aÞJH2 ð34Þ

JTðbÞ ¼ bJT1 þ ð1� bÞJT2 ð35Þ

Note that the NNS are the functions of a and b; i.e. xn ¼ xnða;b; yn ¼ ynða;bÞ; un ¼ unða;bÞ andvn ¼ vnða;bÞ: Since there are infinite combinations of a and b; we still need to decide the optimalweight vector xn: We introduce different types of objective function for the two managers

ðtype 1Þ maxðxn;ynÞ

JHMðxn; ynÞ ¼ lH1 ð1� xnÞ þ lH2 ð1� ynÞ ð36Þ

and

ðtype 2Þ minðun ;vnÞ

JTMðun; vnÞ ¼ pT1 ðlT1 u

n þ lT2 vnÞ þ pT2 ðl

T1 ð1� unÞ þ lT2 ð1� vnÞÞ ð37Þ

The manager from HET wants to maximize the throughput on the link with high capacityðC2 > C1Þ; and the manager from TELE wishes to minimize the total cost of usage of differentlinks. Let pT1 and pT2 be the price per flow for link 1 and link 2, respectively. It is clear that JHMð�Þand JTMð�Þ are the functions of a and b as well. The parameters a and b are weighting factors forthe first team members of team HET and team TELE, respectively. For example, the value of arepresents the relative importance of the objective of user 1 from team HET. Clearly, 1� a and1� b are weighting factors for the second team members of team HET and team TELE,respectively. Usually, the values of a and b can be chosen according to some other criterion,subjective or otherwise. In our problem formulation, the optimal choices of a and b can bedetermined by figuring out a Nash solution to a non-cooperative game between the twomanagers with respect to the objective functions JHMða; bÞ and JTMða;bÞ: Since it is not easy toobtain the analytical expression of NNSL to such a complicated hierarchical decision-makingsystem, we use a numerical example to illustrate the properties and effectiveness of NNSL.

Let c1 ¼ 3; c2 ¼ 6; lH1 ¼ 1; lH2 ¼ 3; lT1 ¼ 0:5; lT2 ¼ 1; pT1 ¼ 10 and pT2 ¼ 30: The correspondingNNSL (optimal routing fractions) under the managers’ objective functions are as follows: an ¼

0:25; bn ¼ 0:8 (or 0.85,0.9,0.95,1), xn ¼ 0:7; yn ¼ 0; un ¼ 0; vn ¼ 1; JH*1 ¼ 0:6748; J

H*2 ¼ 0:4545;

JT*1 ¼ 0:4545; J

T*2 ¼ 0:7692; J

H*M ¼ 3:3; and J

T*M ¼ 25:

For the purpose of comparison, we consider the situation where each class chooses the beststrategy for its users given the decision of other classes and they do not consider thecorresponding manager’s objective function. Clearly, these strategies among four classes ofusers compose a Nash equilibrium strategy, which is given in Table I with xn ¼ 0:2; yn ¼ 0:3;

Table I. Nash strategy.

JH*1 J

H*2 J

T*1 J

T*2 J

H*M J

T*M

0.5424 0.5574 0.5194 0.5424 2.78 40



un ¼ 0:02 and vn ¼ 0:2: In other words, no user has a rational motive to unilaterally deviatefrom its equilibrium strategy.

In comparison, we observe that, using Nash strategy, some, but not all, users may gain betterin reducing average delay time. However, with considering the managers’ cost functions, usingNNS with the leader’s objective function, the total flow through link 2 for NM from HET, (i.e.3.3), is greater than 2.78 resulting from Nash strategy, and cost paid by the manager from TELEis 25, which is less than that when implementing Nash solution. In other words, the objectivesfor both managers are improved by using NNSL.

6. CONCLUSION

In this paper, we formulate the routing problem in a parallel-link network, shared by multipleteams of users with a hierarchical structure, as a multi-team game with Nmanagers, each havingseveral users. We applied a new game theoretic control strategy, called the NNSL to this routingcontrol problem. This strategy is extended from NNS, which is used for selecting a particularsolution from the set of NNS if each team has a leader that optimizes an objective functiondifferent from those of the team members. In the general case, obtaining this solution may alsoinvolve a game among the team leaders. We illustrated the case where the Non-inferior NashSolution with a leader represents a Nash equilibrium strategy among the leaders. Other types ofsolution concepts, such as the Stackelberg strategy, can also be easily implemented among theteam leaders. We use a simple example with N ¼ 2; and n1 ¼ n2 ¼ 2 to illustrate the fact thatNNSL is effective in improving the overall system performance. We also show that using Nashstrategies among the users only is generally inefficient.

ACKNOWLEDGEMENTS

The views and conclusions contained herein are those of the authors and should not be interpreted asnecessarily the official policies of or endorsements, either expressed or implied, of DARPA or AFRL.

REFERENCES

1. Altman E, Basar T, Jimenez T, Shimkin N. Competitive routing in networks with polynomial costs. IEEETransactions on Automatic Control 2002; 47(1):92–96.

2. Altman E, Basar T, Srikant R. Nash equilibria for combined flow control and routing in networks: Asymptoticbehavior for a large number of users. IEEE Transactions on Automatic Control 2002; 47(6):917–930.

3. Sahin I, Simaan MA. Routing and flow control for parallel links communication networks and multiple competingusers. Proceedings of the 15th IEEE International Symposium on Personal, Indoor, and Mobile RadioCommunications, Barcelona, Spain, 5–8 September 2004.

4. Yamaoka K, Sugawara S, Sakai Y. Connection oriented packet communication control method based on gametheory. IEEE International Conference on Communications, Vancouver, Canada, vol. 2, 1999; 1346–1351.

5. Economides AA, Silvester JA. A game theory approach to cooperative and non-cooperative routing problems. IEEEInternational Telecommunication Symposium, Rio de Janeiro, Brazil, 1990; 597–601.

6. La RJ, Anantharam V. Optimal routing control: Repeated game approach. IEEE Transactions on Automatic Control2003; 47(3):437–450.

7. Korilis YA, Lazar AA, Orda A. Achieving network optima using Stackelberg routing strategies. IEEE/ACMTransactions on Networking 1997; 5(1):161–173.

8. Basar T, Srikant R. A Stackelberg network fame with a large number of followers. Journal of Optimization Theoryand Applications 2002; 115(3):479–490.



9. Boulogne T, Altman E, Kameda H, Pourtallier O. Mixed equilibrium (ME) for multiclass routing games. IEEETransactions on Automatic Control 2002; 47(6):903–916.

10. Liu Y, Simaan MA. Noninferior Nash strategies for multi-team systems. Journal of Optimization Theory andApplications 2004; 120(1):29–51.

11. Daigle NJ. Queueing Theory for Telecommunications. Addison-Wesley: Reading, MA, 1992.12. Kim KH, Roush FW. Team Theory. Halsted Press: New York, 1987.13. Basar T, Olsder GJ. Dynamic Non-cooperative Game Theory. Academic Press: New York, 1995.14. Dacunha NO, Polak E. Constrained minimization under vector-valued criteria in finite dimensional space. Journal of

Mathematical Analysis and Applications 1967; 19:103–124.

AUTHORS’ BIOGRAPHIES

Yong Liu received the BS degree in Information Science and MS degree in SystemEngineering from XiDian University, PR China, in 1996 and 1999, respectively. Shereceived the PhD degree in Electrical Engineering with specialty in AutomaticControl from the University of Pittsburgh in 2003. She worked as a post-doctoralresearcher in the Department of Electrical and Computer Engineering at the OhioState University for one year since she graduated, and worked on the development ofcooperative control strategies for networked unmanned air vehicles. She is a memberof IEEE. Her areas of expertise include game theory and its applications in large-scale systems, game-theoretic dynamic system modelling, optimal control andestimation theory.

Marwan A. Simaan received the PhD degree in Electrical Engineering from theUniversity of Illinois at Urbana-Champaign in 1972 and did postdoctoral work atthe Coordinated Science Laboratory at the University of Illinois until 1974. In 1976he joined the Department of Electrical Engineering at the University of Pittsburghwhere he is currently the Bell of PA/Bell Atlantic Professor. He served as chair of thedepartment from 1991 to 1998. He has held a variety of research and consultingpositions in industry including the English Electric Leo-Marconi Computers Ltd.;Bell Telephone Laboratories; Shell Development Company; Gulf R&D Company;and ALCOA Laboratories. His research interests are in the broad areas of control,signal processing and telecommunication. He has edited four books and writtenmore than 300 articles in journals, books, conference proceedings and technicalreports. His recent research has focused on the cooperative control of teams of

semi-autonomous systems in a competitive environment.Dr Simaan is coeditor of the Journal of Multidimensional Systems and Signal Processing (Kluwer) and

currently serves or has served on the editorial boards of a number of journals including the Proceedings ofthe IEEE, the IEEE Press, the IEEE Transactions on Circuits and Systems (Part II), the IEEETransactions on Geoscience and Remote Sensing, the Journal of Optimization Theory and Applications,and Integrated Computer-Aided Engineering.Dr Simaan received three Best Paper Awards (1985, 1988 and 1999) and a Distinguished Alumnus

Award from the Department of Electrical and Computer Engineering at the University of Illinois atUrbana-Champaign (1995). He is a member of the U.S. National Academy of Engineering, a Fellow ofIEEE and AAAS, and a member of AAAI and ASEE. He served on numerous professional committeesand boards including the IEEE Fellow Committee, the AACC Awards Committee, which he chaired from1997 to 1999, and the Engineering Section of AAAS for which he currently serves as Secretary. He is aregistered Professional Engineer in Pennsylvania and he serves as an Electrical Engineering programEvaluator for the Accreditation Board for Engineering and Technology (ABET).



non-inferior nash strategies for routing control in parallel-link communication networks

Documents