Introductory Microeconomics
More Formal Concepts of Game Theory and Evolutionary Game Theory
Prof. Wolfram Elsner Faculty of Business Studies and Economics
iino โ Institute of Institutional and Innovation Economics
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
06.05.2014
2
Readings for this lecture
Mandatory reading this time:
More Formal Concepts of Game Theory and Evolutionary Game Theory, in: Elsner/Heinrich/Schwardt (2014): The Microeconomics of Complex Economies, Academic Press, pp. 193-226.
The lecture and the slides are complements, not substitutes
An additional reading list can be found at the companion website
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
Basic games of classical GT was introduced in Chapter 2
Now: Understand the formal structure of classical decision theory and GT
Classical GT relies on (boundedly) rational agents
Evolutionary GT allows to relax such assumptions and focuses on the dynamic performance of strategies
First: Understand most important formal concepts of classical GT
Then: Move to evolutionary GT
06.05.2014
3
Outline
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
A strategic game is described by
The rules of the game
The agents of the game (here: finite number)
The strategies of the agents (here: finite number)
The information available to the agents
Normal form game
Agents make decisions simultaneously
They do not know about the decision of the others
06.05.2014
4
Basic Concepts of Game Theory
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
Symbol Explanation
๐ ๐ A pure strategy of the ith agent
๐๐ Set of all pure strategies of agent i
๐ = ๐๐ , ๐ = 1, โฆ , ๐ Set of strategies of all players
๐ โ๐ = ๐ ๐ ๐โ ๐ Strategies of all other agents than agent i
๐ = ๐ ๐ , ๐ โ๐ Feasible configuration of strategies
06.05.2014
5
Notation I
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
Symbol Explanation
ฮ ๐ ๐ = ฮ ๐ ๐ โ๐ = ฮ ๐ ๐ ๐ , ๐ โ๐ โ๐ ๐โ๐ โ๐ Set of payoffs for all possible combinations of strategies
๐บ = ๐๐; ฮ ๐ ๐ ; ๐ผ๐ , ๐ = 1, โฆ , ๐ General description for a normal form game
๐บ = ๐1, ๐2; ฮ 1 ๐ , ฮ 2 ๐ ; ๐ผ1, ๐ผ2 Description for a normal form game with two players
06.05.2014
6
Notation II
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
06.05.2014
7
Matrix notation
Strategy 1 Strategy 2
Strategy 1
ฮ ๐ต(๐ ๐ด1 , ๐ ๐ต1)
ฮ ๐ด(๐ ๐ด1 , ๐ ๐ต1)
ฮ ๐ต(๐ ๐ด1 , ๐ ๐ต2)
ฮ ๐ด(๐ ๐ด1 , ๐ ๐ต2)
Strategy 2
ฮ ๐ต(๐ ๐ด2 , ๐ ๐ต1)
ฮ ๐ด(๐ ๐ด2 , ๐ ๐ต1)
ฮ ๐ต(๐ ๐ด2 , ๐ ๐ต2)
ฮ ๐ด(๐ ๐ด2 , ๐ ๐ต2)
๐บ = ๐1, ๐2; ฮ 1 ๐ , ฮ 2 ๐ ; ๐ผ1, ๐ผ2
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
In order to predict the outcome of an interaction, assumptions regarding the agentsโ behavior must be made
Utility maximizing agents with well-defined preference orderings, i.e. for any outcomes ๐ and ๐ the following holds:
Completeness: ๐ โป ๐ ๐๐ ๐ โป ๐ ๐๐ ๐ ~ ๐
Reflexivity: ๐ โป ๐ โบ ๐ โบ ๐
Transitivity: ๐ โป ๐ ๐ โป ๐ โน ๐ โป ๐
Common Knowledge of Rationality
Every agent knows that all agents are rational, that all all other agents also know that all agents are rational, that they also are aware that all agents know that all are rational, etc.
06.05.2014
8
Assumptions regarding the agents
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
Consider a non-interactive decision situation (to bring or not to bring an umbrella) under risk (the state of the world is unknown, it may or may not rain).
06.05.2014
9
A Preliminary Consideration: Non-Interactive Concepts, Decision Theory
Rain No Rain
Bring Umbrella
4
5
No Umbrella
-10
10
P
l
a
y
e
r
S t a t e of the W o r l d
How will the player decide?
The obvious difference to game theory is that the State of the World is unknown and unconstrained by rationality
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
06.05.2014
10
Non-Interactive Concepts, Decision Theory
R No R
Max.
U 4
5
5
No U
-10
10
10
P
l
a
y
e
r
S t a t e of the W o r l d
An optimistic concept: Maximax
Find the best possible payoff for every strategy and maximize
R No R
Min.
U 4
5
4
No U
-10
10
-10
P
l
a
y
e
r
S t a t e of the W o r l d
An pessimistic concept: Minimax
Find the worst possible payoff for every strategy and maximize
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
06.05.2014
11
Non-Interactive Concepts, Decision Theory contd.
R No R
U 4
5
No U
-10
10
P
l
a
y
e
r
S t a t e of the W o r l d
An opportunity cost based concept: Savageโs Minimax Regret
Construct the regret matrix (how much would the player regret this decision in this state of the world compared to the other decision she could have taken), find the highest regret for every strategy and minimize
R No R
Max.
U 0
5
5
No U
6
0
6
S
t
r
a
t
.
R e g r e t
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
There are also parametric decision criteria (i.e. criteria that assign a priori probabilities to the states of the world) including, e.g. the Laplace and the Hurwicz criterion
Of course, all decision theory concepts may also be used to make predictions for strategic games (these predictions would be valid even without CKR, i.e. even if agents assume their opponents to be irrational)
However, for many problem structures, these concepts fail
GT allows for more advanced prediction methods by taking the agentsโ capability to make consistent, systematic, and rational decisions into account
06.05.2014
12
Non-Interactive Concepts, Decision Theory contd.
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
Players are rational, i.e. payoff maximizing and neither benevolent nor envious
They know that all other players think the same way
Expectations about the behavior of the other players can be formed
The players then play the strategy giving them the best outcome (highest payoff) given their expectations about the reasoning of the other players
06.05.2014
13
How to solve the game
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
06.05.2014
14
Dominance of strategies
and
The latter strategy ๐ ๐~ is said to be dominated by the first
If a strategy gives strictly higher payoffs regardless of the choice of the opponent, it is said to strictly dominate the other strategy
Rational players never play a strictly dominated strategy
ฮ ๐ ๐ ๐โ, ๐ โ๐ โฅ ฮ ๐ ๐ ๐
~, ๐ โ๐ โ๐ โ๐
โ๐ โ๐: ฮ ๐ ๐ ๐โ, ๐ โ๐ > ฮ ๐(๐ ๐
~, ๐ โ๐)
A strategy ๐ ๐โ โ ๐๐ dominates a strategy ๐ ๐
~ โ ๐๐ iff
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
One way to predict the outcome of a game is therefore the successive elimination of strictly dominated strategies (SESDS)
SESDS does not require any assumptions about opponentโs behavior
06.05.2014
15
SESDS
Strat. 1 Strat. 2
Stra-tegy
1
4 4
2 2
Stra-tegy
2
2 2
0 0
Strat. 1
Stra-tegy
1
4 4
Stra-tegy
2
2 2
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
SESDS does not always yield a solution:
06.05.2014
16
SESDS
P
l
a
y
e
r
A
P l a y e r B
Strategy 1 Strategy 2
Strategy 1 2
2
0 0
Strategy 2
0 0
2 2
There are no strictly dominated strategies to remove
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
SESDS does not always yield a solution:
But: Thanks to CKR, agents can form expectations about the choices made by other agents
They can choose the best possible response to the expected choice of their opponents
Due to CKR, the others will do so as well
The resulting situation is a combination of mutual best responses
This situation is called Nash Equilibrium (NE)
06.05.2014
17
Nash Equilibria
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
The formal definition is as follows:
06.05.2014
18
Nash Equilibria
If a game is solvable via SESDS the solution is also a NE
But: Not all NE can be explored via SESDS
Also, There are games that do not have a NE in pure strategies at all
Therefore: Introduce the distinction between pure and mixed strategies
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
How do you play Rock-Paper-Scissors?
To always play the same strategy is a bad idea
Mixed strategies capture the idea of playing different pure strategies with some probability
06.05.2014
19
Mixed Strategies
Rock Paper Scissors
Rock 0
0
1 -1
-1
1
Paper -1
1
0 0
1
-1
Scissors 1
-1
-1
1
0
0
P
l
a
y
e
r
A
P l a y e r B
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
A mixed strategy ฯi for player i is a vector in which every pure strategy is associated with a probability
For the two strategy case: ๐๐ =๐1 โ ๐
The Nash Equilibrium is defined as a configuration of mixed strategies for the n players such that
06.05.2014
20
Nash Equilibrium and Mixed Strategies
ฮ ๐ ๐๐โ, ๐โ๐โ โฅ ฮ ๐ ๐๐ , ๐โ๐
โ โ๐๐โ๐
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
Every finite n-person normal-form game with a finite number of strategies for each player has at least one NE in pure or mixed strategies (proven by John Nash in 1950)
Thanks to the NE and the concept of mixed strategies, all finite n-person normal-form games with a finite number of strategies can be solved in theory
How can we compute the NE in mixed strategies?
06.05.2014
21
Nash Equilibrium and Mixed Strategies
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
Define the ๐ฅ ร ๐ฆโmatrix ๐ with the payoffs for the first player:
๐ =2 13 0
Expected payoffs are given by:
ฮ 1 = ๐1๐ก ร๐ ร ๐2
06.05.2014
22
Computation of NE in Mixed Strategies
Dove Hawk
Dove 2
2
3 1
Hawk 1
3
0 0
P
l
a
y
e
r
A
P l a y e r B
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
06.05.2014
23
Computation of NE in Mixed Strategies
ฮ 1 = ๐ 1 โ ๐2 13 0
๐1 โ ๐
ฮ 1 = ๐1๐ก ร๐ ร ๐2
ฮ 1 = โ2๐๐ + 3๐ + ๐
๐ฮ 1
๐๐= โ2๐ +1
๐ฮ 1๐๐= โ2๐ + 1 = 0
q = 0.5 โ ๐ = 0.5
This procedure can be employed for every (symmetric) game
Always take the first derivative of the expected payoff function with respect to the strategy parameter of the same player (๐ for 1, ๐ for 2) and solve it for 0
For symmetric games, the equations are identical for both (with ๐ and ๐ exchanged) and thus have to be solved only once
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
06.05.2014
24
Reaction Functions
๐ and ๐ can also be shown as the reaction functions p(๐) and ๐(๐) of the playersโ reaction to the opponentโs choice
Intersections denote NE (the current Hawk-Dove game has 3)
Note: The caption in Figure 8.12 in Chapter 8 is incorrect (same Figure as on this slide; the Figure depicts Hawk-Dove, not Matching Pennies)
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
Mixed strategy NE are where players are indifferent between
both pure (and mixed) strategies (i.e. where ๐ฮ 1
๐๐=0, ๐ฮ 2
๐๐= 0)
Mixed strategy NE may seem oddly unstable but this is not necessarily the case (see Evolutionary Game Theory below).
Also, under CKR it is rational to deliberately choose mixed NE strategies in order to facilitate the emergence of the equilibrium and to avoid being exploited (Aumannโs defense)
This is especially true if there is no pure strategy NE; consider the Matching Pennies game (or the Rock-Paper-Scissors game above)
06.05.2014
25
Reaction Functions
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
06.05.2014
26
Mixed Strategy NE
Heads Tails
Heads 1
-1
-1 1
Tails -1
1
1 -1
P
l
a
y
e
r
1
P l a y e r 2
In zero-sum games (like this Matching Pennies game), unequal payoff denotes one player being exploited by the other
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
Many extensions to the Nash Equilibrium have been developed
Some of these, the ones for extensive or repeated (subgame perfect Nash Equilibrium) as well as evolutionary games (ESS, โฆ ), will be presented below
There are also refinements that ensure that the equilibrium is still valid under stochastic perturbations (the โtrembling handโ), e.g. Seltenโs Trembling Hand Perfect Equilibrium, Myersonโs Proper Equilibrium (see textbook), or Harsanyiโs Bayesian Nash Equilibrium.
06.05.2014
27
Extensions of the Nash Equilibrium
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
In normal-form games, the players make their decisions simultaneously but this is not the case for other types of games
For games in sequential form we use a new notation, the extensive-form notation
Note that this is necessary only if the agents have full information about the decisions made by the previous agents
Otherwise the game is equivalent to a normal-form game
06.05.2014
28
Extensive Form Games
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
Depending on the choice of the first player, the second player faces a different decision situation
It is therefore convenient to define complete strategies for the players
A complete strategy gives each player an instruction for all possible situations
06.05.2014
29
Extensive-Form Notation
Player 1
Player 2 Player 2
Result 1 Result 2 Result 3 Result 4
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
For an extensive game ๐บ๐ธ let ๐ be a set containing all possible states of ๐บ๐ธ
๐๐ด will contain all possible situations in which player A can possibly make a decision
A complete strategy for player A gives a an instruction for any element in ๐๐ด
06.05.2014
30
Complete Strategies
Player 1
Player 2
๐1
๐2 ๐3
๐4 ๐5 ๐6 ๐7
C
C C
D
D D
(0,0) (2,โ1) (โ1,2) (1,1)
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
Analytical derivation of NE cumbersome
Thanks to CKR we can rely on backward induction
06.05.2014
31
How to solve extensive games
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
06.05.2014
32
Backward Induction
Player 1
Player 2
๐1
๐2 ๐3
๐4 ๐5 ๐6 ๐7
C
C C
D
D D
The reasoning for player 1 is as follows:
What would player 2 do in situation ๐2?
Choose D
Result is would be ๐5
What would player 2 do in situation ๐3?
Choose D
Result would be ๐7
Since 1 prefers ๐7 (via ๐3) to ๐5 (via ๐2), she chooses D (which leads to ๐3 and then ๐7)
(0,0) (2,โ1) (โ1,2) (1,1)
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
Analytical derivation of NE cumbersome
Thanks to CKR we can rely on backward induction
Advantage: All NE found with backward induction are also subgame perfect
A NE is subgame perfect if it is also a NE of all the subgames that contain the NE
06.05.2014
33
How to solve extensive games
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
Up to this point every game we have investigated was played only once
The solutions we have obtained are called one-shot solutions
Now we consider repeated games
A supergame ๐ข is a sequence of repetitions of the normal-form game ๐บ that is infinite from the playersโ point of view, i.e. either really infinite or indefinite (with stochastic probability to end in each period)
06.05.2014
34
Repeated Games and Supergames
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
If ๐ข is finite it can be expressed as an extensive game and solved via backward induction
If ๐ข is infinite or indefinite, there is no last period, so backward induction will not work
Still, strategies can be characterized as complete (if they contain instructions for every possible situation) or not complete and equilibria can be characterized as subgame perfect or not
If both players follow complete strategies, all choices are predictable and the payoffs are thus known in advance (to rational players and observers); this allows to calculate a present value to be used in deciding between strategies in advance
06.05.2014
35
Solving Supergames
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
To obtain the present value for a future payoff ๐ one uses a discount parameter ๐ฟ (with 0 โค ๐ฟ โค 1) which denotes the playerโs valuation of future payoffs (the present value in time 0 of a payoff ๐ in time 0 is ๐, โฆ of the same payoff in time 1 it is ๐ฟ๐, in time 2, ๐ฟ2๐, etc.)
The present value for an infinite sequence of payoff ๐ is
therefore given as ฮ = ๐ + ๐ฟ๐ + ๐ฟ2๐ + ๐ฟ3๐ +... =๐
1โ๐ฟ
Agents make the decisions about their strategy plan based on the present values of the expected payoffs from the different strategy plans
06.05.2014
36
Present Values
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
Strategy plans can be very simple, e.g. always to play one strategy
In the PD, ALL-D is a strategy plan according to which the player always defects
Strategy plans can also be more complex, esp. when the choice of the player depends on the decisions made by other players in the past
In supergames, the players usually have a memory that allows expectation formation (which is known to the player so that the strategy plan can make use of the fact)
06.05.2014
37
Strategy Plans
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
Consider a supergame based on a social dilemma (a generalized PD), i.e.
a cooperative solution exists which constitutes the social optimum and a Pareto optimum of the underlying game
but the cooperative strategy is exploitable (i.e. the cooperative solution is not a NE)
The folk theorem states that it is possible to reach the cooperative solution in the supergame
This is achieved by using trigger strategies, positing a credible thread of punishment if the opponent should deviate from the cooperative strategy (Rubinsteinโs proof).
06.05.2014
38
Folk Theorem and Trigger Strategies
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
Define a strategy that guarantees a minimum of payoff for the opponent as the minmax strategy (analogous to maximin, only that her payoff is minimized by her opponents choice not maximized by her own choice)
If the opponentโs payoff resulting from using the minmax strategy on her is less then her payoff from cooperation, the minmax strategy can serve as a threat of punishment for deviating from cooperation
The threat is said to be credible if the punishing player receives a higher or equal payoff from playing the minmax strategy than from allowing herself to be exploited (more generally: if the minmax strategy is not strictly dominated)
Credible threats can be used to construct a trigger strategy ๐ ๐ก๐๐๐๐๐๐๐ข
06.05.2014
39
Trigger Strategies and Rubinsteinโs Proof of the Folk Theorem
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
A strategy profile can include the instruction to employ ๐ ๐๐๐๐๐๐ฅ always if a specific expectation is not met
By doing so, mutually beneficial agreements that do not constitute a NE in the one-shot case can be enforced
Example: The Prisonerโs Dilemma and tit-for-tat
๐ ๐ก๐๐๐๐๐๐๐ข
= ๐ ๐๐น๐๐ข= ๐๐๐๐๐๐๐๐ก๐, ๐๐ ๐กโ๐ ๐๐กโ๐๐ ๐๐๐๐ฆ๐๐ โ๐๐ ๐๐๐๐๐๐๐๐ก๐๐ ๐๐๐ ๐ก ๐๐๐ข๐๐ ๐๐๐๐๐๐ก, ๐๐ ๐กโ๐ ๐๐กโ๐๐ ๐๐๐๐ฆ๐๐ โ๐๐ ๐๐๐ก ๐๐๐๐๐๐๐๐ก๐๐ ๐๐๐ ๐ก ๐๐๐ข๐๐
To defect is the minmax strategy that serves as credible threat, all players playing this trigger strategy is a NE in the repeated PD
Trigger strategies can be constructed in different ways; they could for instance also โ less forgivingly โ punish forever after one defection
06.05.2014
40
Trigger Strategies and Rubinsteinโs Proof of the Folk Theorem
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
Allows to conveniently construct microfounded economic models, thus closing gaps between micro, meso, and macro level
In Evolutionary Game Theory (EGT) we consider populations of agents, all continuously playing one and the same strategy according to their type
Agents are matched randomly to play again and again the same underlying game (which must be symmetric so that the positions of row player and column player are exchangable)
The population shares of these types (strategies) develop according to their performance in those games
06.05.2014
41
Evolutionary Game Theory
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
The changing share of types may be seen as reproduction with the offspring continuing the same strategies or as poorly performing agents consciously changing their strategy (type)
The most important solution techniques are
1. Analysis of evolutionary stability (evolutionary stable strategies, ESS)
2. Replicator dynamics
3. Simulation (see textbook Chapter 9)
The players are matched randomly
When they meet, they play the underlying game using their predetermined strategy
Composition of the population changes according to the agentโs performance in playing the underlying game
06.05.2014
42
Evolutionary Game Theory
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
NE: Combination of mutual best answers
In the population context we consider only strategies:
A strategy is evolutionary stable if a population dominated by it is not invadable by any other strategy
If a population is dominated by an ESS, the situation will remain stable
What does โnot invadableโ mean?
06.05.2014
43
Evolutionary Stability
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
06.05.2014
44
The principle of evolutionary stability
Symbol Explanation
๐ซ๐ข๐ฑ An evolutionary population setting
๐บ๐ข๐ฑ The underlying (symmetric) one-shot normal form game in this setting
The payoff matrix
ฮ ๐1/๐2 = ๐1๐๐ ๐2
Expected payoff of the first strategy against the second
๐
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
Consider a population of agents playing ๐โ
Now consider a very small group of players entering the group and playing ๐~ โ ๐โ
If the new strategy yields better payoffs than the old one, the
share of players playing ๐~ will increase
In this case the ๐~ has invaded ๐โ and ๐โcannot be said to be an
ESS
We will now formalize the concept of evolutionary stability
06.05.2014
45
The principle of evolutionary stability
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
Let ํ be the (arbitrarily small) share of the invading group playing ๐~
A share of (1 โ ํ) is therefore playing strategy ๐โ
๐โ is an ESS if it yields a higher expected payoff than ๐~
Formally:
๐โ๐๐(1 โ ํ)๐โ + ๐โ๐๐ํ๐~ > ๐~๐๐(1 โ ํ)๐โ+ ๐~๐๐ํ๐~
This rather complicated formula can conveniently be tested using two simple conditions,
The first is constructed from letting ํ=0 for which the above inequality must hold at least weakly, with โฅ (1st condition)
The second results additionally for the case that the first condition holds with equality (2nd condition).
06.05.2014
46
How to test for evolutionary stability
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
06.05.2014
47
How to test for evolutionary stability
(Weak) First con-dition of evolu-tionary stability:
Holds
Holds
๐โ is evolutionary stable
Does not hold
๐โis not evolutionary stable
Does not hold
Holds
Second condition of evolutionary stability
๐โ๐๐๐โ โฅ ๐~๐๐๐โ
๐โ๐๐๐โ > ๐~๐๐๐โ ๐โ๐๐๐~ > ๐~๐๐๐~
Strict first condition of evolutionary stability:
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
06.05.2014
48
How to test for evolutionary stability
Test considering which candidate strategies: pure and mixed NE strategies, since non-NE-strategies are by definition not best answers to themselves (and this is required by the 1st condition of ESS).
Test against which invading strategies: Against all competing pure strategies (in the 2-strategy case โ since mixed strategies are then only linear combinations combining properties of only two pure strategies)
In cases with more than 2 strategies, evolutionary stability must be tested against mixed strategies as well.
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
Consider as an example the Hawk Dove game above.
We have game matrix ๐ =2 13 0
and three NE strategies,
H =01, D =
10, and M =
0.50.5
Test H against D, 1st condition:
0 12 13 0
01โฅ 1 0
2 13 0
01
0 โฑ 1
The condition does not hold, H is therefore not an ESS because it can be invaded by D-players
06.05.2014
49
Evolutionary Stability: Example
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
Test D against H, 1st condition:
1 02 13 0
10โฅ 0 1
2 13 0
10
2 โฑ 3
The condition does not hold, D is therefore not an ESS because it can be invaded by H-players
M must be tested against both D and H; Test M against H, 1st condition:
0.5 0.52 13 0
0.50.5โฅ 0 1
2 13 0
0.50.5
1.5 โฅ 1.5
Condition holds with equality, test of 2nd condition necessary
06.05.2014
50
Evolutionary Stability: Example contd.
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
Test M against H, 2nd condition:
0.5 0.52 13 0
01> 0 1
2 13 0
01
0.5 > 0
Condition holds, H cannot invade M
Test M against D, 1st condition:
0.5 0.52 13 0
0.50.5โฅ 1 0
2 13 0
0.50.5
1.5 โฅ 1.5
Condition holds with equality, test of 2nd condition necessary
06.05.2014
51
Evolutionary Stability: Example contd.
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
Test M against D, 2nd condition:
0.5 0.52 13 0
10> 1 0
2 13 0
10
2.5 > 2
Condition holds, D cannot invade M
Since all mixed strategies are linear combinations of D and H, none of those (except M itself) can be able to invade M
M is therefore this gameโs only ESS
06.05.2014
52
Evolutionary Stability: Example contd.
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
The focus is not on single strategies but on stable compositions of populations
The population is modeled as a dynamical system
A dynamical system describes how state variables change over time
Consider a system with z state variables:
06.05.2014
53
Replicator Dynamics
The development path depends on the initial values and the development equations
๐๐ก =
๐1,๐ก๐2,๐กโฎ๐๐ง,๐ก
= ๐๐,๐ก ๐=1,โฆ,๐ง
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
Here the state variables represent the shares of specific agents playing a certain strategy (except for the last share which is the difference to 100%, thus determined by the others)
The shares might change according to a difference or differential equation:
06.05.2014
54
Replicator Dynamics
๐๐ก+1 = ๐น๐ท~(๐๐ก)
๐๐(๐ก)
๐๐ก= ๐น๐~(๐๐ก)
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
06.05.2014
55
Replicator Dynamics
Symbol Explanation
๐๐,๐ก The evolutionary fitness of agent i at time t
๐ = ๐๐,๐ก๐๐,๐ก
๐
Average fitness in the population
For the evolutionary performance of the agents it is important how their fitness compares to the average fitness
One can now analyze the dynamical system and search for stable equilibria
๐๐,๐ก+1 = ๐น๐ท~(๐๐,๐ก , ๐๐,๐ก , ๐๐ก)
๐๐๐(๐ก)
๐๐ก= ๐น๐~(๐๐ ๐ก , ๐๐ ๐ก , ๐๐(๐ก))
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
Replicator Dynamics requires an explicit assumption on the form and speed of the replicator (this is implicit in ESS)
Typical forms set the dynamic as proportional to the relation of individual and average fitness with the most common canonical forms being
06.05.2014
56
Replicator Dynamics
๐๐,๐ก+1 = ๐๐,๐ก(๐๐,๐ก/๐๐ก) ๐๐๐(๐ก)
๐๐ก= ๐๐ ๐ก (๐๐ ๐ก โ ๐(๐ก))
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
The conditions for equilibria are:
In order to test for the stability of the equilibria one calculates the eigenvalues of the development equation for the equilibria (see Chapters 10 and 11 of the textbook for details)
06.05.2014
57
How to determine (stable) equilibria
For difference equations, the equilibrium is stable if all eigenvalues have an absolute value smaller than unity
For differential equations, the equilibrium is stable if all eigenvalues are negative
๐๐,๐ก+1โ = ๐น๐ท
~ ๐๐๐ก, ๐๐,๐ก, ๐๐ก = ๐๐กโ
๐๐๐(๐ก)
๐๐ก= ๐น๐~ ๐๐ ๐ก , ๐๐ ๐ก , ๐๐ ๐ก = 0
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
Consider again the Hawk-Dove game above with the
canonical replicator function ๐๐๐(๐ก)
๐๐ก= ๐๐ ๐ก (๐๐ ๐ก โ ๐(๐ก)), call
the pure strategies ๐ฅ๐ and assume the ๐๐ = ฮ ๐ = ๐ฅ๐๐๐ ๐ (at any
point in time, leaving out the (๐ก) for convenience)
We further have ๐ =2 13 0
, ๐ฅ1 = ๐ท =10
, ๐ฅ2 = ๐ป =01,
and therefore also
๐1 = 1 02 13 0
๐11 โ ๐1
= 1 + ๐1
๐2 = 0 12 13 0
๐11 โ ๐1
= 3๐1
06.05.2014
58
Replicator Dynamics: Example
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
Since ๐ = ๐๐๐๐ ๐ =๐1๐1+(1 โ ๐1)๐2 , we can rearrange the replicator equation
๐๐1
๐๐ก= ๐1 (๐1 โ ๐)
๐๐1
๐๐ก= ๐1 (๐1 โ ๐1๐1 โ (1 โ ๐1)๐2)=๐1 ((1 โ ๐1)๐1 โ (1 โ ๐1)๐2)
๐๐1
๐๐ก= ๐1 (1 โ ๐1)(๐1 โ ๐2)
Substituting ๐1 and ๐2 yields ๐๐1๐๐ก= ๐1 1 โ ๐1 1 + ๐1 โ 3๐1 = ๐1 1 โ ๐1 1 โ 2๐1
06.05.2014
59
Replicator Dynamics: Example contd.
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
Applying the equilibrium condition ๐๐1
๐๐ก= 0 yields three fixed
points ๐1,1 = 0, ๐1,2 = 1, and ๐1,3 = 0.5
To assess the stability, we bring the replicator equation into polynomial form
๐๐1๐๐ก= 2๐1
3 โ 3๐12 + ๐1
and obtain the only element of the systemโs Jacobian ๐(๐๐๐/๐๐ก)
๐๐๐= 6๐1
2 โ 6๐1 + 1
the linearization of which is the (dominant) eigenvalue.
06.05.2014
60
Replicator Dynamics: Example contd.
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
We obtain the three eigenvalues for the three equilibria
๐ ๐1,1 = 1 (i.e. ๐1,1 is unstable),
๐ ๐1,2 = 1 (i.e. ๐1,2 is also unstable),
๐ ๐1,3 = โ0.5 (i.e. ๐1,3 is stable)
The result is identical to that obtained from analysis of ESS: a composition of the population (or equivalently of each individual strategy) of both strategies with equal shares is the only stable equilibrium or fixed point. The pure strategy equilibria / fixed points exist but are unstable.
06.05.2014
61
Replicator Dynamics: Example contd.
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
This chapter
gave a comprehensive introduction to formal game theory including different notations (matrix, formal, extensive)
introduced advanced solution concepts of decision theory and game theory (including SESDS and Nash Equilibria in pure and mixed strategies, backward induction, โฆ)
covered non-normal-form games (including extensive games, repeated games)
gave a formal introduction to Evolutionary Game Theory (ESS and replicator dynamics)
06.05.2014
62
Summary
Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies
Chapter 08: More Formal Concepts of Game Theory and Evolutionary Game Theory
06.05.2014
63
Readings for the next lecture
Compulsory reading:
Introduction to Simulation and Agent-Based Modeling, in: Elsner/Heinrich/Schwardt: Microeconomics of Complex Economies, pp. 227-247.
For further readings visit the companion website