game theory for wireless networks€¦ · pareto optimality; subgame perfection; ... game theory...

Game theory for wireless networks Georg-August University Göttingen

Game theory for wireless networks

static games;

dynamic games;

repeated games;

strict and weak dominance;

Nash equilibrium;

Pareto optimality;

Subgame perfection;

…

Georg-August University Göttingen Game theory for wireless networks

Outline

1 Introduction

2 Static games

3 Dynamic games

4 Repeated games

2


Brief introduction to Game Theory

Proper operation of wireless networks requires appropriate rule enforcement mechanisms to prevent or discourage malicious or selfish behavior

– Discouraging malicious or selfish behavior can benefit from game theoretic modeling

Classical applications: economics, but also politics and biology

Example: should a company invest in a new plan, or enter a new market, considering that the competition may make similar moves?

Most widespread kind of game: non-cooperative (meaning that the players do not attempt to find an agreement about their possible moves)

3


Classification of games

Non-cooperative Cooperative

Static Dynamic (repeated)

Strategic-form Extensive-form

Perfect information Imperfect information

Complete information Incomplete information

4

Cooperative

Imperfect information

Incomplete information


Classification of games

5

non-cooperative games: individuals cannot make binding agreements and the unit of

analysis is the individual. In cooperative game theory, binding agreements between

players are allowed and the unit of analysis is the group or coalition.

Static games: each player makes a single move and all moves are made simultaneously.

Perfect info: each player has a perfect knowledge of the whole previous actions of

other players at any moment it has to make a new move.

Complete info: each player knows who the other players are, what are their possible

strategies and what payoff will result of each player for any combination of moves.


Cooperation in self-organized wireless networks

6

S1

S2

D1 D2

Usually, the devices are assumed to be cooperative.

But what if they are not?


Assumptions and terminology

7

We will consider two players and non-cooperative games in four simple

game examples related to wireless network operations:

• Forwarder’s dilemma

• Joint packet forwarding game

• Multiple access game

• Jamming game

The players wish to transmit or receive data packets coping with limited

transmission resources which can make them to have selfish or malicious

behavior.

Si: Strategy of player i (e.g. forward the packet, drop the packet, etc.)

S-i: Strategy of the opponents of player i (as we will have only two players i

and j, S-i will be Sj)

Having any strategy Si the players would get some benefit and have to pay

some cost

Ui: Utility or payoff of player i expresses the benefit of him given a strategy

minus the cost it has to incur: Ui = bi - ci

Users controlling the devices are rational, i.e. try to maximize their benefit.


Outline

1 Introduction

2 Static games

3 Dynamic games

4 Repeated games

8


Example 1: The Forwarder’s Dilemma

9

?

?

Blue Green

Forwarder’s Dilemma:

• Two players: Blue and Green

• Green has one data packet to send to its receiver (R1) via Blue and also

blue has one data packet to send to its receiver (R2) via Green.

• If Blue forwards the packet for Green to R1, it must pay the transmission

cost of c and Green will get the benefit of 1.

• The dilemma:

• each player is tempted to drop the other player’s packet to save its

energy

• but the other player may reason in the same way -- > they could do

better by forwarding for each other

R1 R2


From a problem to a game

game formulation: G = (P,S,U)

– P: set of players

– S: set of strategy functions

– U: set of payoff functions

strategic-form representation of the game

(each cell corresponds to one possible combination of strategies of the players and is presented as (Ublue , Ugreen))

10

• Benefit for the source of a packet reaching the destination: 1

• Cost of packet forwarding for the forwarder: c (0 < c << 1)

(1-c, 1-c) (-c, 1)

(1, -c) (0, 0)

Blue

Green

Forward

Drop

Forward Drop


Solving the Forwarder’s Dilemma (1/2)

' '( , ) ( , ), ,i i i i i i i i i iu s s u s s s S s S

iu U

i is S

is

11

Strict dominance: strictly best strategy, for any possible strategy of the other

player(s)

where: payoff function of player i

strategies of all players except player i

In this game, Drop strictly dominates Forward from the point of view of both

players:

Because 1-c<1 and –c<0

Therefore the solution of the game is (D,D), i.e. both players choose the

strategy drop

Forward

Drop

Drop

Strategy strictly dominates strategy S’i if

(1-c, 1-c) (-c, 1)

(1, -c) (0, 0)

Blue Green

Forward


Solving the Forwarder’s Dilemma (2/2)

12

Solving the game by iterative strict dominance:

iteratively eliminate strictly dominated strategies

(1-c, 1-c) (-c, 1)

(1, -c) (0, 0)

Blue Green

Forward

Drop

Forward Drop

This is the lack of trust between the players that leads

to this suboptimal solution (green is never sure that Blue will

forward for it, if it forwards for blue (as they play simultaneously) and vice

versa.)

Drop strictly dominates Forward Dilemma

(F,F) would result in a better outcome for both players BUT }


Example 2: The Joint Packet Forwarding Game

13

?

Blue Green Source Dest

?

For this game there is no strictly dominated strategies !

• A sender sends its packets to its receiver and two players (Blue and Green)

should forward the packets for it.

• Reward for packet reaching the destination: 1 for both players

• Cost of packet forwarding: c (0 < c << 1)

(1-c, 1-c) (-c, 0)

(0, 0) (0, 0)

Blue

Green

Forward

Drop

Forward Drop


Weak dominance

14

? Blue Green Source Dest

?

'( , ) ( , ),i i i i i i i iu s s u s s s S

with strict inequality for at least one s-i

Solving the game by Iterative weak dominance:

iteratively eliminate weakly dominated strategies

(1-c, 1-c) (-c, 0)

(0, 0) (0, 0)

Blue

Green

Forward

Drop

Forward Drop For Green, Drop is weakly dominated

by Forward (so the second column is

eliminated); then in the remaining parts

of the table Blue would do better to

Forward.

The result of the iterative weak

dominance is not unique in general.

Weak dominance: Strategy s’i is weakly dominated by strategy si if


Nash equilibrium (1/2)

15

Nash Equilibrium: a game strategy such that no player can increase its

payoff by deviating unilaterally

(1-c, 1-c) (-c, 1)

(1, -c) (0, 0)

Blue

Green

Forward

Drop

Forward Drop

E1: The Forwarder’s

Dilemma: (Drop, Drop) is a Nash

Equilibrium; if Green deviates to

(D,F) its payoff decreases (from

0 to –c) and similarly if blue

deviates to (F,D) its payoff

decreases (from 0 to –c)

E2: The Joint Packet

Forwarding game: There are 2 Nash Equilibria for

this game: (F,F) and (D,D)

(1-c, 1-c) (-c, 0)

(0, 0) (0, 0)

Blue

Green

Forward

Drop

Forward Drop


Nash equilibrium (2/2)

16

* * *( , ) ( , ),i i i i i i i iu s s u s s s S

iu U

i is S

where: payoff function of player i

strategy of player i

( ) arg max ( , )i i

i i i i is S

b s u s s

The best response of player i to the profile of strategies s-i is

a strategy si such that:

Nash Equilibrium = Mutual best responses

Caution! Many games have more than one Nash equilibrium

Strategy profile s* constitutes a Nash equilibrium if, for each player i,


Example 3: The Multiple Access game

17

Benefit for the source of a

successful transmission: 1

Cost of transmission: c

(0 < c << 1)

• For this game:

o There is no strictly dominating strategy

o There are two Nash equilibria

(0, 0) (0, 1-c)

(1-c, 0) (-c, -c)

Blue

Green

Quiet

Transmit

Quiet Transmit

Time-division channel

Two players have packets to send to their receivers but

through the same channel; if in a given time slot both

transmit packets there will be a collision and no packet

will be delivered to its destination

If one player transmits and the other stays quiet in that

time slot, the packet will be received by its receiver and

its source will get some benefit


Mixed strategy Nash equilibrium

(1 )(1 ) (1 )blueu p q c pqc p c q

18

objectives

– Blue: choose p to maximize ublue

– Green: choose q to maximize ugreen

(1 )greenu q c p

1 , 1p c q c

p: probability of transmit for Blue

q: probability of transmit for Green

is a mixed strategy Nash equilibrium

Green: If p<1-c --> 1-c-p is positive; Green chooses q=1

If p>1-c --> 1-c-p is negative; Green chooses q=0

If p=1-c 1-c-p=0; Green’s benefit not affected by its move

Blue: If q<1-c --> 1-c-q is positive; Blue chooses p=1

If q>1-c --> 1-c-q is negative; Blue chooses p=0

If q=1-c 1-c-q=0; Blue’s benefit not affected by its move


Example 4: The Jamming game

19

Transmitter: has a data packet to send to

its receiver at each time slot

Jammer: is interested in preventing the

transmitter from successful transmission

by sending packets on the same channel

Transmitter:

benefit of 1 for successful transmission

loss of -1 for jammed transmission

Jammer:

benefit of 1 for successful jamming

loss of -1 for missed jamming

There is no pure-strategy Nash equilibrium

two channels:

C1 and C2

(-1, 1) (1, -1)

(1, -1) (-1, 1)

Blue

Green

C1

C2

C1 C2

transmitter

jammer

1 1,

2 2p q is a mixed strategy Nash equilibrium

p: probability of transmit on

C1 for Blue

q: probability of transmit on

C1 for Green


Theorem by Nash, 1950

20

Theorem:

Every finite strategic-form game has a mixed-

strategy Nash equilibrium.


Efficiency of Nash equilibria

21

E2: The Joint

Packet Forwarding

game (1-c, 1-c) (-c, 0)

(0, 0) (0, 0)

Blue

Green

Forward

Drop

Forward Drop

• How to choose between several Nash equilibria ?

• One way is to compare strategy profiles using the concept of

Pareto-optimality.

Pareto-optimality: A strategy profile is Pareto-optimal if it is not possible to

increase the payoff of any player without decreasing the payoff of another

player.

• Strategy profile s is Pareto-superior to strategy profile s’ if for any player i:

With strict inequality for at least one player.


Back to the examples

In Forwarder’s dilemma game: – The Nash Equilibrium (D,D) is not Pareto-Optimal

– Strategy profiles (F,F), (F,D) and (D,F) are Pareto-Optimal but not Nash Equilibria.

In Joint Packet forwarding game: – (F,F) and (D,D) are Nash Equilibria; out of them only (F,F) is Pareto-Optimal.

In Multiple Access game: – (T,Q) and (Q,T) are Nash Equilibria and Pareto-Optimal.

In Jamming game: – There exists no pure-strategy Nash Equilibrium (D,D) but all pure-strategy profiles are

Pareto-Optimal.

22


Back to the examples

In Multiple Access game with mixed-strategy: – The mixed-strategy Nash Equilibrium (p=1-c, q=1-c) results in the payoff (0,0);

– Hence, both pure-strategy Nash Equilibria are Pareto-superior to it.

It can be shown in general that there does not exist a mixed strategy profile that is Pareto-superior to all pure strategy profiles. Because each mixed strategy of a player I is a linear combination of her pure strategies with positive coefficients that sum up to one.

23


Outline

1 Introduction

2 Static games

3 Dynamic games

4 Repeated games

24


Extensive-form games

usually to model sequential decisions

game represented by a tree

Example 3 modified: the Sequential Multiple Access game: – Blue plays first, then Green plays.

– Blue: leader; Green: follower

25

Green

Blue

T Q

T Q T Q

(-c,-c) (1-c,0) (0,1-c) (0,0)

Reward for successful

transmission: 1

Cost of transmission: c

(0 < c << 1)

Green

Time-division channel


Strategies in dynamic games

The strategy defines the moves for a player for every node in the game, even for those nodes that are not reached if the strategy is played.

26

Green

Blue

T Q

T Q T Q

(-c,-c) (1-c,0) (0,1-c) (0,0)

Green strategies for Blue:

T, Q

strategies for Green:

TT, TQ, QT and QQ

TQ means that player p2 transmits if p1 transmits and remains quiet if p1 remains quiet.

There are three pure-strategy Nash Equilibria: (T,QT), (T,QQ) and (Q,TT)

Blue: T Green’s best move Q; Blue: Q Green’s best move T


Backward induction

Solve the game by reducing from the final stage

Green’s best moves: Q if Blue moves T and T if Blue moves Q

Blue can calculate its best move knowing this argument:

– Blue’s best move is T (comparing (1-c,0) to (0,1-c))

So backward induction starts from the last stage and ends at the root

– The continuous line from a leaf of the tree to the root would be the solution

27

Green

Blue

T Q

T Q T Q

(-c,-c) (1-c,0) (0,1-c) (0,0)

Green

Backward induction solution: h={T, Q}


Subgame perfection

In fact extends the notion of Nash equilibrium

28

Green

Blue

T Q

T Q T Q

(-c,-c) (1-c,0) (0,1-c) (0,0)

Green Stackelberg equilibria using

backward induction :

(T, QT) and (T, QQ)

Stackelberg equilibrium (subgame perfect equilibrium ): A strategy

profile s is a Stackelberg equilibrium for a game with p1 as the leader

and p2 as the follower if p1 maximizes her payoff subject to the

constraint that player p2 chooses according to her best response

function.


Outline

1 Introduction

2 Static games

3 Dynamic games

4 Repeated games

29


Repeated games

repeated interaction between the players (in stages)

move: decision in one interaction

strategy: defines how to choose the next move, given the previous moves

history: the ordered set of moves in previous stages

– most prominent games are history-1 games (players consider only the previous stage)

initial move: the first move with no history

finite-horizon vs. infinite-horizon games

stages denoted by t

30


Utilities: Objectives in the repeated game

finite-horizon (finite no. of stages ) vs. infinite-horizon games

myopic vs. long-sighted repeated games

1i iu u t

0

T

i i

t

u u t

31

0

i i

t

u u t

Myopic games : players maximize their next stage payoff

long-sighted finite games: players maximize their payoff

for the whole finite duration T

long-sighted infinite: the period of maximizing the

payoff is not finite

payoff with discounting: 0

t

i i

t

u u t

0 1 is the discounting factor


Strategies in the repeated game

usually, history-1 strategies (strategy depend only on the previous stage), based on different inputs:

– others’ behavior:

– others’ and own behavior:

– payoff:

1i i im t s m t

1 ,i i i im t s m t m t

32

1i i im t s u t

• Example strategies in the Forwarder’s Dilemma:

Blue (t) initial move

F D strategy name

Green (t+1) F F F AllC (always cooperate)

F F D Tit-For-Tat (TFT) (repeat the opponent’s move)

D D D AllD(always defect)

D D F Anti-TFT(opposite to the opponent’s move)


Analysis of the Repeated Forwarder’s Dilemma (1/3)

Blue strategy Green strategy

AllD AllD

AllD TFT

AllD AllC

AllC AllC

AllC TFT

TFT TFT

Blue payoff Green payoff

0 0

1 -c

1/(1-ω) -c/(1-ω)

(1-c)/(1-ω) (1-c)/(1-ω)

(1-c)/(1-ω) (1-c)/(1-ω)

(1-c)/(1-ω) (1-c)/(1-ω)

33

infinite game with discounting: usually used to model finite games

in which the players are not aware of the duration of the game

• By decreasing values of future payoffs:

0

t

i i

t

u u t

0< ω <1



AllC receives a high payoff with itself and TFT, but

AllD exploits AllC

AllD performs poor with itself

TFT performs well with AllC and itself, and

TFT retaliates the defection of AllD

34

Blue strategy Green strategy

AllD AllD

AllD TFT

AllD AllC

AllC AllC

AllC TFT

TFT TFT

Blue payoff Green payoff

0 0

1 -c

1/(1-ω) -c/(1-ω)

(1-c)/(1-ω) (1-c)/(1-ω)

(1-c)/(1-ω) (1-c)/(1-ω)

(1-c)/(1-ω) (1-c)/(1-ω)

TFT is the best strategy if ω is high (ω ≈ 1)!



Blue strategy Green strategy Blue payoff Green payoff

AllD AllD 0 0

TFT TFT (1-c)/(1-ω) (1-c)/(1-ω)

35

Theorem: In the Repeated Forwarder’s Dilemma, if both

players play AllD, it is a Nash equilibrium.

Theorem: In the Repeated Forwarder’s Dilemma, both

players playing TFT is a Nash equilibrium as well.

The Nash equilibrium sBlue = TFT and sGreen = TFT is

Pareto-optimal (but sBlue = AllD and sGreen = AllD is not) !


(1, 1) (-1, 2)

(2, -1) (0, 0)

Country 1

Country 2

Reduce

military

investment

Increase

military

investment

Reduce military

investment

Increase military

investment

Payoffs:

2: I have weaponry superior to the one of the opponent

1: We have equivalent weaponry and managed to reduce it on both sides

0: We have equivalent weaponry and did not managed to reduce it on both sides

-1: My opponent has weaponry that is superior to mine

An Example beyond Engineering

36


Who is malicious? Who is selfish?

37

Both security and game theory backgrounds are useful in many cases !!

Harm everyone: viruses,…

Selective harm: DoS,… Spammer

Cyber-gangster:

phishing attacks,

trojan horses,…

Big brother

Greedy operator

Selfish mobile station


Conclusion

Game theory can help modeling greedy behavior in wireless networks

Four simple examples were used to explain how to capture the wireless networking problem in a corresponding game

Game theory has been recently applied to wireless communication, but discipline still is in its infancy

38

game theory for wireless networks€¦ · pareto optimality; subgame perfection; ... game theory...

Documents