mathematical theory of games

PART 2: MORE GAMES, NASH EQUILIBRIUM, BARGAINING AND DILEMMAS

MATHEMATICAL THEORY OF GAMES

MATH ALIVE, TUES. APRIL 10, 2014 A SUPPLEMENTARY LECTURE BY PHILIPPE H. TRINH • MATHEMATICAL INSTITUTE, OXFORD UNIVERSITY

WHAT IS WRONG WITH NASH’S (RUSSELL CROWE’S) LOGIC IN “A BEAUTIFUL MIND”?

LECTURE OUTLINE

2

Non-zero-sum games

equilibrium solutions that

require coordination

non-optimal equilibrium

solutions

cooperative vs. non-cooperative

REVIEW OF LAST CLASS

3

• Proposed the simplest type of game: zero-sum games between two players, Rose and Colin.

• Proposed two idea of an equilibrium solution (also called saddle point, Nash equilibrium)

• If (X,Y) is a Nash equilibrium, then neither player will want to diverge from their strategy.

A B

A 3 -3

B 2 1

COLIN

ROSE

THE BAR SCENE IN “A BEAUTIFUL MIND”

Brunette Blonde

Brunette

Blonde

COLIN

ROSE

• The movie scene is wrong! • The brunette/brunette solution is not a Nash equilibrium • Other mixed strategies exist (for n players)

A REVIEW OF LAST CLASS: MIXED STRATEGIES

• Lisa demonstrates the importance of mixed strategies in a zero-sum game with no pure equilibrium

A REVIEW OF LAST CLASS: MIXED STRATEGIES

• For the rock-paper-scissors game, the optimal (Nash equilibrium) strategy is a mixed strategy of (1/3, 1/3, 1/3)

• In order to prove this, calculate the expected earnings for Lisa/Bart using (1/3, 1/3, 1/3) and show if both players play this, there is no incentive to diverge.

Rock Paper Scissors

Rock

Paper

Scissors

BART

LISA

SOME MORE TIPS ON HOW TO MASTER RPS

• In 2005, wealthy Japanese art collector asked two auction houses (Christie vs. Sotheby) to play RPS where the winner would auction his collection of paintings worth millions.

• Christie consulted expert advice by 11 year-old daughter of employee: “everybody expects you to play rock”

• Christie (scissors) won versus Sotheby (paper)

Rock Paper Scissors

Percentage 36% 30% 34%

FOR STATISTICS AND MORE ON THE CHRISTIE STORY, SEE “THAT SETTLES IT” IN NEW SCIENTIST, 22/29 DEC. 2007

BASED ON 1.8 MILLION THROWS ON FACEBOOK (SIMILAR STATISTICS FOR OTHER STUDIES)

NASH’S EQUILIBRIUM PAPER IN 1950• Following a meeting with von

Neumann, Nash published his ideas on equilibrium solutions of non-zero-sum non-cooperative games.

• Paper is only one page long(!!) and is found in PNAS January 1, 1950, vol. 36 no. 1

• Revolutionary perhaps not for its complexity, but rather its simplicity and clarity

• Demonstrates the existence (non-constructive) of “optimal” solutions in non-zero-sum games

THE GAME OF CHICKEN

• Iconic scene from “Rebel Without a Cause” (1955) • Two opponents drive towards each other (or a cliff ). Do you

stay the course or do you back down?

THE GAME OF CHICKEN (MATRIX VERSION)

• Two Nash equilibria • The question is how are these equilibrium solutions decided? • We must open up the rules of the game…can we allow for

commitment strategies?

Swerve Straight

Swerve

Straight

COLIN

ROSE

PRISONER’S DILEMMA

• Proposed in 1950 by Melvin Dresher and Merrill Flood (RAND Corp), then given the prisoner’s backstory by Albert Tucker (Princeton) during a seminar

• Classic example: two prisoners are given choice to confess or not confess (alternatively, cooperate vs. defect)

• Optimal solution is not “optimal”…so what do we mean by optimal?

Don’t confess Confess

Don’t confess

Confess

COLIN

ROSE

PARETO OPTIMAL 1

• An outcome is non-Pareto optimal if there exists another outcome which would give both players higher payoffs, or would give one player the same payoff, and the other, a higher payoff.

• Otherwise, the outcome is Pareto optimal.


Don’t confess

Confess

COLIN

ROSE

NASH EQUILIBRIUM BUT NOT PARETO OPTIMAL!

PARETO OPTIMAL BUT NOT NASH EQUILIBRIUM

PARETO OPTIMAL 2

• Every outcome of a zero-sum game is Pareto optimal. Why?


Don’t confess

Confess

COLIN

ROSE

Pareto Principle To be acceptable as a solution, an outcome

should be Pareto optimal

PAYOFF DIAGRAM

Cooperate Defect

Cooperate (1,1) (-1,2)Defect (2,-1) (0,0)

COLIN

ROSE

CD

DDDC

CC

ROSE PAYOFF

COLIN PAYOFF • Pure strategies can be visualized as points in the Rose-Colin payoff plane.

• Mixed strategies lead to expected payoffs that lie within the polygon.

APPLICABILITY OF PRISONER’S DILEMMA

• Very fertile testing bed for studies in social psychology in relation research on cooperation in experimental subjects and societies

• In 1980, Robert Axelrod wrote that >350 articles on Prisoner’s Dilemma had been published in the last 10 years in a single journal!

• Example of application: “Self-sacrifice, cooperation and aggression in women of varying sex-role orientations” by Baefsky and Berger (1974) in Personality Social Psychology Bull. 1: 296-298.

FROM RAPOPORT & CHAMMAH (1970)

RESOLVING PRISONER’S DILEMMA

• Many ways to ‘resolve’ Prisoner’s dilemma • Some ideas: introduce metagames, introduce mathematical

description of cooperation, introduce iterative strategies… • How you resolve depends on what you want and the exact

circumstance that led you to Prisoner’s dilemma

NASH’S BARGAINING SCHEME I

• Nash proposed a theory for how players could bargain with one another to determine a cooperative solution.

• Players agree on status quo (SQ) point. If bargaining fails, players receive SQ payoff (handled by arbitrar).

• Four axioms for cooperation:

1. Rationality Outcome should lie inside the payoff polygon.

2. Linear invariance Outcome does not depend on how contestants

‘price’ the rewards.

3. Symmetry Outcome is the same if contestants change

places.

4. Independence of irrelevant alternatives If the outcomes they would not have chosen become unfeasible,

nothing changes.

CD

DDDC

CC

ROSE PAYOFF

COLIN PAYOFF

NASH’S BARGAINING SCHEME II

Nash Bargaining Theorem There is one and only one bargaining scheme satisfying Axioms 1-4. If the status quo point, SQ, is (0,0), then the bargaining solution is the point in

the polygon that maximizes the product of the payoffs, x*y

CD

DDDC

CC

ROSE PAYOFF

COLIN PAYOFF

by Nash’s axioms, this is the unique “optimal”

solution with cooperation

CHALLENGES TO NASH’S BARGAINING SCHEME

• Unclear how the SQ point is determined in practice: Nash offered the idea of threats to determine SQ point.

• The Axiom 4 (Independence of irrelevant alternatives) is contentious—removable of some outcomes may change weighting of others.

• Consider pricing of subscriptions to the Economist journal…

SPLIT THE DOLLAR GAME

• Players are asked to split $100 according to the following rules: • Each player simultaneously calls out an amount from $0 to $100. • If the sum exceeds $100, players receive nothing. • Otherwise, players receive what they called out.

STATUS QUOROSE PAYOFF

COLIN PAYOFF

• The dilemma is that every single point along the dashed line is a Nash equilibrium (and also Pareto optimal)

• But now there is a unique bargaining solution

$100 TO ROSE

$100 TO COLIN

COOP’ SOL’N

REPEATED PLAY OF PRISONER’S DILEMMA

• Another ‘resolution’ of prisoner’s dilemma involves extending the game to multiple iterations (one argues this is how real life works)

• Under repeated play, cooperation over the long run is clearly optimal (and a Nash equilibrium)

Cooperate-Cooperate Cooperate-Defect

Defect-Defect

Defect-Defect

Cooperate-Cooperate

Cooperate-Cooperate

NASH EQUILIBRIUM IN THE LONG RUN SUBOPTIMAL IN THE LONG RUN

DOMINO DILEMMA IN REPEATED PLAY

• However, the dilemma is that all games are finite in practice.

Cooperate-Cooperate

Cooperate-Defect

Defect-Defect

Cooperate-Cooperate

Cooperate-Cooperate

Cooperate-Cooperate

• Knowing there are N = 100 games, it is better to defect at 99th game.

• …but knowing your opponent defects at 99 means defect occurs at 98th…

• This can be resolved by proposing that the games end with some non-zero probability

THE METAGAME SOLUTION• Basic idea: form a game-within-game whereby Rose/Colin’s actions

are determined through their predictions of their opponent’s moves

Cooperate regardless

Choose same as Rose

Choose opposite from Rose Defect regardless

Cooperate (0,0) (0,0) (-2,1) (-2,1)Defect (1,-2) (-1,-1) (1,-2) (-1,-1)

COLIN

ROSE

Cooperate Defect

Cooperate (0,0) (-2,1)Defect (1,-2) (-1,-1)

COLIN

ROSE

ORIGINAL GAME:

FIRST METAGAME:

THE SECOND METAGAME

Cooperate regardless

CC

Choose same as Rose CD

Choose opposite from Rose

DC

Defect regardless DD

CCCC (0,0)

CCCD (0,0)

CCDC (0,0)

CCDD (0,0) (0,0) (1,-2) (-1,-1)

CDCC (-1,-1)

CDCC (-1,-1)

CDDC (-1,-1)

CDDD (-1,-1)

DCCC (0,0)

DCCD (0,0)

DCDC (0,0)

DCDD (1,-2) (0,0) (1,-2) (-1,-1)

DDCC (-1,-1)

DDCD (-1,-1)

DDDC (-1,-1)

DDDD (1,-2) (-1,-1) (1,-2) (-1,-1)

COLIN

ROSE

DCDD means Rose only cooperates if Colin plays his CD strategy (i.e. Rose cooperates if Colin believes Rose cooperates and hence cooperates himself!)

ROBERT AXELROD AND HIS 1980 GAME

• We should not present an oversimplified representation of Prisoner’s Dilemma, and we can ask what happens in practice.

• Axelrod invited game theorists to submit programs to play iterated Prisoner’s Dilemma

• 14 programs in first competition • Rules: (i) round robin between contestants;

(ii) contestants also faced off with their twin and RANDOM; (iii) N = 200 moves; 3 points for mutual cooperation, 1 point for mutual defection; 5 points for defecting if opponent cooperates.

• Tournament was repeated five times.

SEE “EFFECTIVE CHOICE IN THE PRISONER’S DILEMMA” BY AXELROD IN J. CONFLICT RESOLUTION (1980)

RESULT’S OF AXELROD’S FIRST TOURNAMENT• Winner (504.5 pts) was TIT FOR TAT (by Rapoport), coded in only 4 lines


IF N = 1, THEN COOPERATE IF N > 1, THEN DO WHAT OPPONENT DID LAST ROUND

• Second place (500.3 pts) was 41 lines in lengthIF N = 1, THEN COOPERATE THE MORE OFTEN OPPONENT DEFECTS, THE MORE A “PUNUSHMENT” COUNTER INCREASES VARIOUS BLOCKS IF OPPONENT’S MOVEMENTS ARE ERRATIC DEFECT ON 2 LAST TURNS

• Fourth place (481.9 pts) 8 lines in lengthIF OPPONENT DID A DIFFERENT MOVE LAST ITERATION, COOPERATE WITH PROBABILITY 2/7 OTHERWISE, ALWAYS COOPERATE

• Fifteenth place (276.3 pts) 5 lines

COOPERATE AND DEFECT WITH RANDOM PROBABILITY

AXELROD’S FOUR PROPOSED STRATEGIES


1. Be niceNever defect first

2. RetaliatePunish defection

3. ForgiveOnce punished, allow future cooperation

4. Be clear Your strategy should be consistent and easy-to-predict

Mathematical modeling !

LESSONS FROM GAME THEORY I: HOW IS MATHS USED

Given a complicated real-life problem, analysis begins by simplifying the problem to the bare essentials: !a) Propose axioms and principles b) Reduce the dimensionality (n-player to 2-player) c) Reduce the available strategies

Nash equilibrium and its caveats !

LESSONS FROM GAME THEORY II: WHAT OPTIMAL MEANS

A set of strategies is a Nash equilibrium if no player can do better by unilaterally changing his or her strategy. !Dilemmas and paradoxes arise when the Nash equilibrium is not unique, is not (Pareto) optimal, or clashes with another principle (e.g. Newcomb’s problem)

Reduction of games to a unified theory !

LESSONS FROM GAME THEORY III: HOW TO VIEW GAMES

Len Fisher wrote a book “Rock, Paper, Scissors—Game Theory in Everyday Life” proposing that there are 7 dilemmas in life, represented by: 1) Prisoner’s Dilemma 2) Tragedy of the Commons (basically n-player PD) 3) Free Rider problem 4) Chicken 5) Volunteer’s Dilemma 6) Battle of the Sexes 7) Stag Hunt !Using mathematics, we see the problem much more clearly. These dilemmas all stem from (i) multiple Nash equilibria and/or (ii) non-Pareto equilibria

Ball game Movie

Ball game

Movie

COLIN

ROSE

Act Don’t Act

Act

Don’t Act

COLIN

ROSE

BATTLE OF THE SEXES VOLUNTEER’S DILEMMA

DARWIN MUSES MARRY VS. NOT MARRY

These notes record Darwin's speculations about the prospect of marriage and his future life and work. They were written before his engagement and marriage to his cousin Emma Wedgwood in January 1839. The note has been conjecturally dated to July 1838.

mathematical theory of games

Documents