Network Routing Problem
Input: network topology, link metrics, and traffic matrix
Output: set of routes to carry traffic
AB
C
D
E
S1
R1
R3
R2
S3
S2
Network Routing: Classical Approach
Routing as optimization problem e.g., minimum total delay in network focus on global network performance (social
optimal) performance of individual user not important
Centralized or distributed algorithms e.g., link state or distance vector
Passive users users are oblivious to routing decisions
Network Routing: Game-Theoretic Approach
Routing as game between users users determine route decision based solely on individual performance (selfish routing) strongly dependent on other users’ decisions
Non-cooperative game (non-zero sum) users compete for network resources
Equilibrium point of operation Nash equilibrium point (NEP)
Selfish Network Routing
Advantages no need of centralized control or global
agreement on routing algorithm individual user’s performance considered greater adaptability
• changes in user demands or changes in network conditions
Disadvantages multiple equilibria (eq. selection problem) convergence to equilibrium no network-wide optimality at equilibrium
• cost of “selfish routing” user’s must have detailed knowledge of network
Applications of Game Theory to Network Routing
Competitive routing in multiuser communication networksA. Orda, R. Rom and N. ShimkinIEEE/ACM Transactions on Networking, 1 (5) 1993
How bad is selfish routing?T. Roughgarden and E. TardosJournal of the ACM, 49 (2) 2002
Selfish routing with atomic playersT. RoughgardenACM/SIAM Symp. on Discrete Algorithms (SODA) 2005
Simple Model: Network of Parallel Links
set of users share a set of parallel links each user has fixed demand (data rate) users decide how to split demand across links
minimize individual cost link has a load dependent cost (e.g., delay)
A
S1
B
R1
S2
S3
S4
R2
R3
R4
set of parallel links: set of users: each user has a fixed demand (data rate): user splits its demand across links
flow of user i on link l: flow configuration of user i:
system flow configuration:
feasible configurations satisfy nonnegative and demand constraints
Network of Parallel Links
),,( I1 fff
}L,,1{ L}I,,1{ I
ir
ilf
),,( L1iii fff
User’s Cost Function Cost function of user i:
cost depends on flow configuration of all users
Assumptions on cost function sum of user-link cost function: can be infinite convex in when finite, continuously differentiable in at least one user with infinite cost can change its flow
configuration to have finite cost• aggregate demand must be less than aggregate link capacity
Ll
lil
i fCfC )()(
ilC
ilf
)( fC i
Very mild conditions on cost function
The Game Users individually decide their flow configuration
goal is to minimize its own cost
Nash Equilibrium Point (NEP) system flow configuration such that no user can
reduce their cost by changing its flow allocation is a NEP if for all i, the following
holds:Ffff I )
~,...,
~(
~ 1
)}~
,...,,...,~({)
~( 1min Iii
Ff
i fffCfCii
No user can reduce their cost by rerouting their own flow
The Issues
Existence of NEP is at least one NEP guaranteed to always exist
Uniqueness of NEP under which conditions (if any) do we have a
single NEP Convergence to (and stability of) NEP
play dynamics that lead to a NEP
System properties at the NEP e.g., how does users divide allocate their flows
Existence of NEP
N-person convex game [Rosen65] joint strategy set is convex, closed and
bounded each player’s payoff function is convex in their
own strategy existence of NEP proven by Katutani fixed point
theorem Can also show using Kuhn-Tucker
conditions necessary and sufficient for system flow
configuration to be a NEP
Uniqueness of NEP
Uniqueness of NEP only under a type of cost functions (type-A functions) cost function has two parameters: user’s i and
aggregate of all others monotonically increasing in each parameter still very general (e.g., M/M/1 delay function)
Proof by contradiction using Kuhn-Tucker conditions
System has a single operating point
System Properties at NEP
Assume all users share same type-A cost function but users can have different demands
Monotonicity of link usage user with higher demand uses more of each
and every link used a user with higher demand uses more links
Higher capacity links receive more users does not hold in general, only under yet
another type of cost function (which still captures M/M/1)
Intuitive but assuring properties
Simple case study two-users sharing two parallel links
Dynamical model: Elementary Stepwise System Users take turns in updating their flow
configuration• measure load on links, adjust its flow to minimize cost
flow of user i on link l at step n
Dynamical System
A
S1
BR1
S2 R2
)(nf il
Convergence to NEP
Let denote unique NEP of gameInitialize system with any feasible
flow configuration: f(0)Convergence to NEP guaranteed
*f
*)(lim fnfn
Framework used in proof not aplicable in general limited to two link, two user structure
General Topology
Users decide how to split their demands over possible paths users know network topology (directed graph)
AB
C
D
E
S1
R1
R3
R2
S3
S2
Existence and Uniqueness of NEP
Existence of NEP same argument as before (N-person convex
game) No unique NEP for type-A cost functions
shown by counterexample Uniqueness shown only under very strict
conditions for cost function not very interesting networking scenarios
Analysis of general network in this modeling framework is much harder
The “Price of Anarchy” Equilibria of non-cooperative games usually
inefficient e.g., prisoner’s dilemma Pareto optimal usually not a NEP
Quantify inefficiency in terms of a global objective “price of anarchy” (coordination versus
competition)
Price of Anarchy of a Game
objective function value at NEP
optimal objective function value=
if multiple NEP exists, take sup (or inf) over NEP set
Cost of Selfish Routing
How does total cost compare? flow allocation at a NEP optimal flow allocation
Total cost of flow configuration:
where is load dependent link cost function e.g., link delay
lLl
ll ffcfC
)()(
(.)lc
Example (1/4)
flow configuration cost
optimal flow allocation
can be realized with
A
S1
BR1
S2 R2
r1 = 0.5
r2 = 0.5
xxcl )(1
1)(2
xcl
)25.0,25.0(,)25.0,25.0( 21 ff
21222111
2)()()( llllllll ffffcffcfC
)5.0,5.0(),( ***
21 ll fff
4/375.0)( * fC
Example (2/4)
But this is not NEP… Cost of a flow configuration to user i
A
S1
BR1
S2 R2
r1 = 0.5
r2 = 0.5
xxcl )(1
1)(2
xcl
il
ill
illl
illl
i fffffcffcfC211222111
)()()(
375.025.025.0*5.0)( * fC i
By rerouting traffic user 1 (or 2) can reduce its cost: )2.0,3.0('1f
365.02.03.0*55.0)'(1 fClower cost!
higher cost
Example (3/4)
NEP given by
link 1 is a dominant strategy (link 2 never used) Cost to user i at NEP
Total cost of NEP configuration
A
S1
BR1
S2 R2
r1 = 0.5
r2 = 0.5
xxcl )(1
1)(2
xcl
)0,5.0(,)0,5.0( 21 ff
5.0)( fC i
1)( fC
higher cost!
Example (4/4)
Optimal cost: NEP cost:
Price of Anarchy:
A
S1
BR1
S2 R2
r1 = 0.5
r2 = 0.5
xxcl )(1
1)(2
xcl
1)( fC4/3)( * fC
3
4
4/3
1
)(
)(*
fC
fC
Thm:[Roughgarden/Tardos00] POA of selfish routing w/affine cost functions is at most 4/3 for any network topology and traffic matrix!
Another example (non-linear cost)…
NEP: both users only use link 1 cost is 1
Optimal: 1-ε for link 1 and ε for link 2 ε depends on d, but is small for large d cost ≈ 0
Price of anarchy can be arbitrarily large goes to infinity as d goes to infinity
A
S1
BR1
S2 R2
r1 = 0.5
r2 = 0.5
dl xxc )(1
1)(2
xcl
So how bad is selfish routing? It depends...
cost functions, network topology, traffic matrix, user demands, etc.
In reality, not so bad achieves close to optimal cost in Internet-like
environments (simulation study) Another positive (and nice) result:
Thm:[Roughgarden/Tardos00] selfish routing is no worst than the optimal routing of twice as much traffic for any cost function, network topology and traffic matrix!
Congestion Control Problem
Input: network topology, routes, link characteristics, traffic matrix
Output: set of data rates to be used
AB
C
D
E
S1
R1
R3
R2
S3
S2
Congestion Control: Classical Approach
Congestion control as optimization problem match user’s demand to network capacity and
achieve some fairness among users focus on global network performance (social
optimal) performance of individual user not important
Centralized or distributed algorithms e.g., TCP, max-min fairness
Passive users users are oblivious to congestion decisions
Congestion Control: Game-Theoretic Approach Congestion control as game between users
users determine their own data rates decision based solely on individual performance
Non-cooperative game (non-zero sum) users compete for network resources
Equilibrium point of operation Nash equilibrium point (NEP)
Key Assumption: A higher sending rate do not necessarily yields better performance for user
Routing Games vs Congestion Control Games
Routing games users determine
network routes multi-path routing
and traffic splitting is possible
users’ data rates are given and must be routed
Congestion games users determine
their data rate network routes
are given (single path)
Applications of Game Theory to Congestion Control
Making greed work in networks: a game-theoretic analysis of switch service disciplinesS. ShenkerIEEE/ACM Transactions on Networking, 3 (6) 1995
An evolutionary game-theoretic approach to congestion controlD. Menasché, D. Figueiredo, E. de Souza e Silva Performance Evaluation, 62 (1-4) 2005
Simple Model: Single Bottleneck Link
S1R1
S2
S3
S4
R2
R3
R4
set of users share a bottleneck link users decide their data rates
maximize individual performance user’s performance depends on link load
e.g., quality of service provided by link
Single Bottleneck Link Users determine sending rate: Link modeled as M/M/1 queue
unit capacity packet scheduling policy
Scheduling policy induces average queue length for each user : avg. queue length of user i
User’s utility function
strictly increasing in strictly decreasing in convex and derivable everywhere
ir
ic
),( iii crU
icir
Scheduling Policy Determined by system operator Allocation function
scheduling policy P induces an avg. queue length for each user given all user’s data rate
FIFO example
Must satisfy some constraints aggregate average queue size same as M/M/1
Allocation function can be realized by different service disciplines
)(rC Pi
j j
iFIFOi r
rrC
1)(
Fair Share Allocation Allocate service capacity fairly among user’s
demand user’s requesting less obtain higher priority
Implemented through a priority queueing algorithm
Assume: r1 < … < rN
UserPriority Level
A B C D
1 r1 - - -
2 r1 r2 - r1 - -
3 r1 r2 - r1 r3 - r2 -
4 r1 r2 - r1 r3 - r2 r4 - r3
fraction of traffic gets lower priority
MAC: Set of Monotonic Allocation Functions
Consider a set of possible allocation functions
: increases, increases
: increases, does not decrease
Includes all typical service disciplines FIFO, LIFO, PS, fair share allocation
0
i
i
r
C
0
j
i
r
C
or
0
j
i
r
Cat for all with rk ro
k0
j
i
r
C r
ir ic
jr ic
The Problem Investigated
Relationship between NEP and service disciplines (MAC functions)
Which service disciplines yield good NEP? Properties of NEP of a given MAC
efficiency fairness convergence to equilibrium user protection
System designer can select service discipline that yields good equilibrium
Efficiency of NEP
Efficiency in terms of Pareto optimal no global objective function of system outcome
Pareto optimal outcome: no other outcome is preferred by all users
Thm:[Shenker95] There is no allocation function in MAC such that every NEP is Pareto optimal
Under some additional constraints fair share is always efficient constrained users’ utility function symmetric rate vector
Uniqueness of NEP
Allocation functions can induce multiple NEP undesirable since users cannot coordinate
Thm:[Shenker95] Fair share mechanism always has a unique NEP Fair share is the only allocation function that always yields a unique NEP
Convergence to Equilibrium Dynamics through a generalized hill
climbing algorithm users eliminate strategies that always perform
worst system converges to a reduced set of strategies
Different from best-response dynamics
Thm:[Shenker95] With the fair share mechanism, all generalized hill climbing
algorithm converges to the NEP
Convergence is also fast (superlinear) and stable
Application to Multimedia Traffic
Users share common bottleneck link User’s choose data rate to be sent by source
only few data rates available Utility given by “perceived” quality
S1
R1
R2
R3
R4
Investigate dynamics and convergence using evolutionary game theory
1r2r3r
Why Evolutionary Game Theory
Model how users change their strategy
Users are not perfect: stochastic dynamics, myopic, etc
Which NEP will be achieved (if more than one exists)
Efficiency of selected NEP
Evolutionary Game Theory
Entities of Model and Interactions
Users(strategy set)
QoS Model(E-model or other)
Link model(M/M/1/k or other)
choice of strategiescauses impact on
performancemetric feeds
yields perceivedquality to
Two-layer Markovian Model
users’actions 0, 3 1, 2 2, 1 3, 0
link perf.
layer 1
layer 2
QoS ofeach user
QoS ofeach user
QoS ofeach user
QoS ofeach user
States and Users Utility
)()()()(1 ,,,,, i
Mi
mi
li nnnn
: number of users selecting strategy l in state)(i
lnM : number of data rates available to users
is
: state is
),( islU : utility function of strategy l in state is
No constraints on users’ utility function should be defined for every state
Transition Matrix
)()()()(1 ,,,,, i
Mi
mi
li nnnn
)()()()(1 ,1,,1,, j
Mj
mj
lj nnnn
otherwisen
slUslUifslUsmUni
l
ijiji
l
)(
)( ),(),()),(),((
Transitions determined by QoS in each state rate of change proportional to gain transitions can reduce QoS (users make errors) Markov chain is ergodic
Main Problem Investigated
System in steady state Users make no mistakes
Assume: 0, t
States that have non-negligible steady state probability
States that correspond to
NEP
What is the relationship?
Proposition 1
under the condition that this state is contained in a quasi-absorbing set
If a state has non-negligible steady state probability
This state is also a NEP
Proposition 2
proof via simple counter-example
This state also has non-
negligible steady state probability
If a state is a NEP
Summary of ResultsStates with non-negligible SS probability are NEP
correspond to “stable” statesSome NEP are not stable
system dynamics cannot converge on themStill possible to have multiple stable NEP
not clear where system will convergestate with highest probability?