Heterogeneous Payoffs and Social Diversity in the Spatial

Prisoner’s Dilemma game

Dept Computer Science and Software Engineering

Golriz RezaeiDr. Michael Kirley

SEAL08 Conference8 Dec 2008

Evolution of cooperation

• Open ended question in many areas• Evolutionary Computation (IEEE Trans, CEC, GECCO)• Autonomous agents and multi agent systems (AAMAS)• Distributed Artificial Intelligence (DAI)• Physics (Statistical Physics)• Biology (Theoretical biology, Nature)

• Prisoner’s Dilemma (PD game)

• Different individual conditions (Heterogeneity) have impact

• In this paper we investigate this idea on a version of the Spatial Prisoner’s Dilemma (SPD) game.

Good abstract Game theoretic approach Mathematical model Applied in many areas (biology, economics, and sociology)

Today’s Agenda

• Brief overview of Prisoner’s Dilemma game and different variations

• The challenge and related works• Proposed model• Evaluation by experiments• Conclusion• Questions

Prisoner’s DilemmaC

cooperateD

Defect

Ccooperate

R=3 R=3

T=5 S=0

DDefect

S=0T=5

P=1P=1

• 2 players / agents • 2 choices (C or D)• Actual values order

• Order change game change

• (D,D) Nash Equilibrium But

i) T > R > P > Sii) 2R >= (T + S)

Iteration reciprocal interaction Spatial local neighbourhood

Spatial Prisoner’s Dilemma• Limited to local neighbourhood interaction only• Accumulates received payoffs from games fitness• At the end of each round selection process

imitation of the most successful neighbour (MSN)• Clusters of cooperators

outweigh losses against defectors

1 2 3

4 5

6 7 8

The Challenge

• Typically

“Universal fixed payoff matrix”

• Hypothesis

Introducing “social diversity” alters trajectory of the population.

Related work

• Few studies investigated the impact of varying the magnitude of the payoff matrix values

1. Tomochi and Kono [Physical Review E 2002]:• Payoff matrix evolved based on the ratio of defectors

(considered R and P only) - Universal payoff matrix

2. Perc and Szolnoki [Physical Review E 2008]:• Random noise added to the individual payoff matrix at the

beginning of the game - Fixed matrix till the end

3. Fort [Physica A 2007]:• The payoff matrix was correlated with a spatial and temporal

zones (considered only T) - The Prisoner’s Dilemma inequality was relaxed.

Proposed model

• Idea Associated payoffs evolve based on individual experience.

• Each agent

• Dynamic payoffs each agent has its own version of payoff matrix and it gets updated at each time step based on the level of the agent’s experience

Age increases at each time step αi(t+1) = αi(t) + 1

Life-span expected life time (λi) randomly drawn from a uniform distribution αi(t) == λi die and replaced by a new random agent

Proposed model

Update

Where is the payoff values for agent i at time t is the default payoff matrix values T, R, P, S is the magnitude of the rescaled values is the age of agent i at time t is the expected life time of agent i is limiting factor and characterises the uncertainty related to

the environment

1)

2)

Three scenarios

1. Standard PD universal fixed Payoffs no Age

2. Homogeneous model universal fixed Payoffs Age

3. Heterogeneous model individual Payoffs Age

• What is the equilibrium state?

Experimental Setup

• Implemented in Netlogo4.0 [Wilensky 1999]• Underlying framework Standard Spatial

Iterated Prisoner’s Dilemma. Agents mapped on 2-D regular lattice (32*32 torus)

• Population initialized 20% cooperators• Each trial 1000 iterations• All configurations 30 times• Statistical results are reported

Experiment 1 sensitivity to the base payoff values

• Two different base level payoff values

T, R, P, S and K = 0.2

a) Big 5, 3, 1, 0

b) Small 1, 1, 0, 0

Experiment 2 sensitivity to the magnitude of K

• base level payoff values T, R, P, S 5, 3, 1, 0

• K was changed systematically

• K represents environmental constraint on social diversity

Snapshots

• Evolving population for homogeneous and heterogeneous model

• K = 0.1 and initial cooperation 20%• Varying size clusters of cooperators (black)

Homogeneous

Heterogeneous

Conclusion

• Results heterogeneous social diversity, promotes cooperation.

• Differences to previous work each agent is equipped with their own evolving payoff matrix. The evolving payoff matrix agents’ age or experience level.

• More realistic approach real world scenarios.

• Future work extend the model to distributed multiagent systems (P2P, MANET)

Questions?

Thank you

References

• H. Fort, On evolutionary spatial heterogeneous games, Physica A (2007).

• M. Perc and A. Szolnoki, Social diversity and promotion of cooperation in spatial prisoner's dilemma game, Physical Review E 77 (2008).

• M. Tomochi and M. Kono, Spatial prisoner's dilemma games with dynamic payoff matrices, Physical Review E 65 (2002), no. 026112.

• Wilensky, U.: NetLogo is a cross-platform multi-agent programmable modeling environment. In: Modeling Nature’s Emergent Patterns with Multi-agent Languages. Proceedings of EuroLogo 2002 (2002),http://ccl.northwestern.edu/netlogo/

Experiment 3 sensitivity to the life span (λ)

• base level payoff values T, R, P, S 5, 3, 1, 0• K = 0.2• λ from different range

Experiment 4 sensitivity to the replacement strategy

• base level payoff values T, R, P, S 5, 3, 1, 0• K = 0.2• Replacement with

random generated agent

and defector agent

Related work

• Few studies have examined the impact of varying the magnitude of the payoff matrix values in PD

• Tomochi and Kono:• Payoff matrix was designed to evolve based on the ratio of

defectors (cooperators) to the whole population. (considered R and P only)

• Universal payoff matrix applicable to all agents at time t. • The level of cooperation within population was directly related to

the payoff matrix values

Related work …

• Perc and Szolnoki:• Random noise drawn from alternative statistical

distributions was added to the payoff matrix at the beginning of the game. (fixed matrix till the end)

• They concluded that this correlated “social diversity mechanism” promoted higher-levels of cooperation in the spatial game examined.

• It was suggested that variable social status might play a crucial role in the evolution of cooperation.

Related work …

• Fort:• The payoff matrix was correlated with a spatial and temporal

zones. (considered only T) • It was possible that the payoffs for an agent and their opponent

were not equal – reminiscent of what happens in general in real life.

• The results reported suggested that the effect of asymmetries in the interactions between agents, which takes into account the effect of asymmetries in the costs and benefits on the evolution of cooperation, had a direct impact on the proportion of agents cooperating in the population.

• The Prisoner’s Dilemma inequality was relaxed, and when the payoff matrix values changed, the game oscillated between the Prisoner’s Dilemma game and Chicken game or the game becomes Stag Hunt game.

What is the idea?

• Ex./ You and your friend, colleague• Ex./ 2 countries punishment system for the same crime.

• Different individual conditions (Heterogeneity) have impact on the behaviour of two people/agents and may alter their interaction and their cooperation.

• In this paper we investigate this idea on a version of the Spatial Prisoner’s Dilemma (SPD) game.

• Why?

Good abstract many real world scenarios. Famous game theoretic approach capture agents interaction Mathematical model study the evolution of cooperation Applied in many areas biology, economics, and sociology