towards realistic models for evolution of cooperation
DESCRIPTION
Towards Realistic Models for Evolution of Cooperation. LIK MUI. … about procedure …. Briefly go over the paper Clarify major points Describe simulations (not in paper). RoadMap. Introduction Cooperation Models Simulations Conclusion. . Evolution of Cooperation. Animals cooperate - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Towards Realistic Models for Evolution of Cooperation](https://reader033.vdocuments.net/reader033/viewer/2022052510/56812cb5550346895d916b3f/html5/thumbnails/1.jpg)
Towards Realistic Models for Evolution of Cooperation
LIK MUI
![Page 2: Towards Realistic Models for Evolution of Cooperation](https://reader033.vdocuments.net/reader033/viewer/2022052510/56812cb5550346895d916b3f/html5/thumbnails/2.jpg)
… about procedure …
• Briefly go over the paper– Clarify major points
• Describe simulations (not in paper)
![Page 3: Towards Realistic Models for Evolution of Cooperation](https://reader033.vdocuments.net/reader033/viewer/2022052510/56812cb5550346895d916b3f/html5/thumbnails/3.jpg)
RoadMap
• Introduction
• Cooperation Models
• Simulations
• Conclusion
![Page 4: Towards Realistic Models for Evolution of Cooperation](https://reader033.vdocuments.net/reader033/viewer/2022052510/56812cb5550346895d916b3f/html5/thumbnails/4.jpg)
Evolution of Cooperation
• Animals cooperate
• Two questions:
– How does cooperation as a strategy becomes stable evolutionarily?
– How does cooperation arise in the first place?
![Page 5: Towards Realistic Models for Evolution of Cooperation](https://reader033.vdocuments.net/reader033/viewer/2022052510/56812cb5550346895d916b3f/html5/thumbnails/5.jpg)
Darwinian Natural Selection
“Survival of the fittest”
• If evolution is all about individual survival, how can cooperation be explained?
• Fittest what?
![Page 6: Towards Realistic Models for Evolution of Cooperation](https://reader033.vdocuments.net/reader033/viewer/2022052510/56812cb5550346895d916b3f/html5/thumbnails/6.jpg)
Fittest what ?
• Individual– Rational agency theory (Kreps, 1990)
• Group– Group selection theory (Wilson, 1980)
• Gene– Selfish gene hypothesis (Dawkins, 1979)
• Organization– Classic organizational theory (Simon,
1969)
![Page 7: Towards Realistic Models for Evolution of Cooperation](https://reader033.vdocuments.net/reader033/viewer/2022052510/56812cb5550346895d916b3f/html5/thumbnails/7.jpg)
RoadMap
• Introduction
• Cooperation Models• Group Selection• Kinship Theory• Direct Reciprocity• Indirect Reciprocity• Social Learning
• Simulations
• Conclusion
![Page 8: Towards Realistic Models for Evolution of Cooperation](https://reader033.vdocuments.net/reader033/viewer/2022052510/56812cb5550346895d916b3f/html5/thumbnails/8.jpg)
Group Selection
• Intuition: we ban cannibalism but not carnivorousness
• Population/species: basic unit of natural selection
• Problem: explain war, family feud, competition, etc.
![Page 9: Towards Realistic Models for Evolution of Cooperation](https://reader033.vdocuments.net/reader033/viewer/2022052510/56812cb5550346895d916b3f/html5/thumbnails/9.jpg)
Kinship Theory I
• Intuition: nepotism
• Hamilton’s Rule:
– Individuals show less aggression and more cooperation towards closer kin if rule is satisfied
– Basis for most work on kinship theory
• Wright’s Coefficient of Related: r– Self: r=1– Siblings: r=0.5– Grandparent-grandchild: r=0.25
cr
b
![Page 10: Towards Realistic Models for Evolution of Cooperation](https://reader033.vdocuments.net/reader033/viewer/2022052510/56812cb5550346895d916b3f/html5/thumbnails/10.jpg)
Kinship Theory II
• Cannot explain:– Competition in viscuous population– Symbioses– Dynamics of cooperation
![Page 11: Towards Realistic Models for Evolution of Cooperation](https://reader033.vdocuments.net/reader033/viewer/2022052510/56812cb5550346895d916b3f/html5/thumbnails/11.jpg)
Direct Reciprocity
• Intuition: being nice to others who are nice
• “Reciprocal Altruism”– Trivers (1971)
• Tit-for-tat and PD tournament– Axelrod and Hamilton (1981)
• Cannot explain:– We cooperate not only with people who cooperate
with us
![Page 12: Towards Realistic Models for Evolution of Cooperation](https://reader033.vdocuments.net/reader033/viewer/2022052510/56812cb5550346895d916b3f/html5/thumbnails/12.jpg)
Indirect Reciprocity
• Intuition: respect one who is famous
• Social-biological justifications– Biology: generalized altruism (Trivers, 1971, 1985)– Sociobiology: Alexandar (1986)– Sociology: Ostrom (1998)
• 3 types of indirect reciprocity:– Looped– Observer-based– Image-based
![Page 13: Towards Realistic Models for Evolution of Cooperation](https://reader033.vdocuments.net/reader033/viewer/2022052510/56812cb5550346895d916b3f/html5/thumbnails/13.jpg)
Indirect Reciprocity: Looped
• Looped Indirect Reciprocity– Boyd and Richerson (1989)
![Page 14: Towards Realistic Models for Evolution of Cooperation](https://reader033.vdocuments.net/reader033/viewer/2022052510/56812cb5550346895d916b3f/html5/thumbnails/14.jpg)
Indirect Reciprocity: Observers
• Observer-based Reciprocity– Pollock and Dugatkin (1992)
![Page 15: Towards Realistic Models for Evolution of Cooperation](https://reader033.vdocuments.net/reader033/viewer/2022052510/56812cb5550346895d916b3f/html5/thumbnails/15.jpg)
Indirect Reciprocity: Image
• Image (reputation) based Reciprocity– Nowak and Sigmund (1998, 2000)
![Page 16: Towards Realistic Models for Evolution of Cooperation](https://reader033.vdocuments.net/reader033/viewer/2022052510/56812cb5550346895d916b3f/html5/thumbnails/16.jpg)
Social Learning
• Intuition: imitate those who are successful
• Cultural transmission– Boyd and Richerson (1982)
• Docility– Simon (1990, 1991)
![Page 17: Towards Realistic Models for Evolution of Cooperation](https://reader033.vdocuments.net/reader033/viewer/2022052510/56812cb5550346895d916b3f/html5/thumbnails/17.jpg)
Critiques of Existing Models
• Many theories each explaining one or a few aspects of cooperation
• Unrealism of model assumptions
![Page 18: Towards Realistic Models for Evolution of Cooperation](https://reader033.vdocuments.net/reader033/viewer/2022052510/56812cb5550346895d916b3f/html5/thumbnails/18.jpg)
Unrealism for Existing Models
• asexual, non-overlapping generations • simultaneous play for every interaction
– c.f., Abell and Reyniers, 2000
• dyadic interactions• mostly predetermined behavior
– c.f., May, 1987 (lack of modeling stochasticity)
• discrete actions (cooperate or defect)• social structure and cooperation
– c.f., Simon, 1991; Cohen, et al., 2001
• extend social learning– c.f., Simon, 1990
![Page 19: Towards Realistic Models for Evolution of Cooperation](https://reader033.vdocuments.net/reader033/viewer/2022052510/56812cb5550346895d916b3f/html5/thumbnails/19.jpg)
RoadMap
• Introduction
• Cooperation Models
• Simulations• Nowak and Sigmund Game• Prisoner’s Dilemma Game• Simon’s Docility Hypothesis
• Conclusion
![Page 20: Towards Realistic Models for Evolution of Cooperation](https://reader033.vdocuments.net/reader033/viewer/2022052510/56812cb5550346895d916b3f/html5/thumbnails/20.jpg)
Nowak and Sigmund Game
• Payoff Matrix
C = 0.1
B = 1.0
• Image Adjustment
A = 1
Interact
Not interact
Donor -C 0
Recipient
B 0
Interact
Not interact
Donor A -A
Recipient
0 0
![Page 21: Towards Realistic Models for Evolution of Cooperation](https://reader033.vdocuments.net/reader033/viewer/2022052510/56812cb5550346895d916b3f/html5/thumbnails/21.jpg)
Using Global Image: 1 Run
t=0
0102030405060708090
100
-5 -4 -3 -2 -1 0 1 2 3 4 5 6
Ab
un
dan
ce
t=9
0
10
20
30
40
50
60
70
80
90
100
-5 -4 -3 -2 -1 0 1 2 3 4 5 6
t=149
0
10
20
30
40
50
60
70
80
90
100
-5 -4 -3 -2 -1 0 1 2 3 4 5 6
t=19
0
10
20
30
40
50
60
70
80
90
100
-5 -4 -3 -2 -1 0 1 2 3 4 5 6
![Page 22: Towards Realistic Models for Evolution of Cooperation](https://reader033.vdocuments.net/reader033/viewer/2022052510/56812cb5550346895d916b3f/html5/thumbnails/22.jpg)
Using Global Image: 100 Runs
average over 50 sim ulations t=0
0
10
20
30
40
50
60
70
80
90
100
-5 -4 -3 -2 -1 0 1 2 3 4 5 6
average over 50 sim ulations t=0
0
10
20
30
40
50
60
70
80
90
100
-5 -4 -3 -2 -1 0 1 2 3 4 5 6
average over 50 sim ulations t=19
0
10
20
30
40
50
60
70
80
90
100
-5 -4 -3 -2 -1 0 1 2 3 4 5 6
average over 50 sim ulations t=999
0
10
20
30
40
50
60
70
80
90
100
-5 -4 -3 -2 -1 0 1 2 3 4 5 6
![Page 23: Towards Realistic Models for Evolution of Cooperation](https://reader033.vdocuments.net/reader033/viewer/2022052510/56812cb5550346895d916b3f/html5/thumbnails/23.jpg)
Dynamics using Global Reputation
50,000
-6
-4
-2
0
2
4
6
8
Number of Generations
Str
ateg
y, K
50,000
0
0.5
1
1.5
2
2.5
Number of Generations
Pay
off
0
0.05
0.1
0.15
0.2
0.25
0.3
1 2 3 4 5 6 7 8 9 10 11 12
Strategy, K
Fre
qu
ency
![Page 24: Towards Realistic Models for Evolution of Cooperation](https://reader033.vdocuments.net/reader033/viewer/2022052510/56812cb5550346895d916b3f/html5/thumbnails/24.jpg)
Using 10 Observers/Interactions
n=20
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
-5 -4 -3 -2 -1 0 1 2 3 4 5 6
Strategy, K
Fre
qu
ency
n=50
0
0.05
0.1
0.15
0.2
0.25
0.3
-5 -4 -3 -2 -1 0 1 2 3 4 5 6
Strategy, K
Fre
qu
ency
n=100
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
-5 -4 -3 -2 -1 0 1 2 3 4 5 6
Strategy, K
Fre
qu
ency
n=200
0
0.05
0.1
0.15
0.2
0.25
0.3
-5 -4 -3 -2 -1 0 1 2 3 4 5 6
Strategy, K
Fre
qu
ency
![Page 25: Towards Realistic Models for Evolution of Cooperation](https://reader033.vdocuments.net/reader033/viewer/2022052510/56812cb5550346895d916b3f/html5/thumbnails/25.jpg)
Evolutionary PD Game
• Repeated Prisoners’ Dilemma Game
• Agent Actions:Action = { cooperate, defect }
• Payoff Matrix:C D
C 3/3 0/5
D 5/0 1/1
![Page 26: Towards Realistic Models for Evolution of Cooperation](https://reader033.vdocuments.net/reader033/viewer/2022052510/56812cb5550346895d916b3f/html5/thumbnails/26.jpg)
PD Game Agent Strategies
• All defecting (AllD)
• Tit-for-tat (TFT)
• Reputational Tit-for-tat (RTFT): using various notions of reputation
![Page 27: Towards Realistic Models for Evolution of Cooperation](https://reader033.vdocuments.net/reader033/viewer/2022052510/56812cb5550346895d916b3f/html5/thumbnails/27.jpg)
Base Case: PD GameGroup Reputation (base: min_gr >= 0)
0
20
40
60
80
100
120
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49
Generation
TF
T C
ou
nt10000 12000 12100 12200 12500
![Page 28: Towards Realistic Models for Evolution of Cooperation](https://reader033.vdocuments.net/reader033/viewer/2022052510/56812cb5550346895d916b3f/html5/thumbnails/28.jpg)
Simple Groups: social structures
• Group structure affects members– Interactions, observations, and knowledge– Persistent structure
• Groups actions– Observed indirectly through member's
actions
![Page 29: Towards Realistic Models for Evolution of Cooperation](https://reader033.vdocuments.net/reader033/viewer/2022052510/56812cb5550346895d916b3f/html5/thumbnails/29.jpg)
Group Membership
• Member agents– Have public group identity– Directly associated with one environment
• Group Structure is a Tree– Least common ancestral node (LCAN)– Events occur with respect to a shared
environment
![Page 30: Towards Realistic Models for Evolution of Cooperation](https://reader033.vdocuments.net/reader033/viewer/2022052510/56812cb5550346895d916b3f/html5/thumbnails/30.jpg)
Shared Environment Example
Agents Group
A1,A2 G1
A3,A4 G2
A5,A2 G1
A1,A3 G0
A5,A3 G0
A1
A2 A4A3
A5
A0
G0
G1 G2
G3
![Page 31: Towards Realistic Models for Evolution of Cooperation](https://reader033.vdocuments.net/reader033/viewer/2022052510/56812cb5550346895d916b3f/html5/thumbnails/31.jpg)
PD Game with Group Reputation(varying encounters per generation EPG)
Group Reputation (min_gr >= 0.5)
0
20
40
60
80
100
120
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49
Generation
TF
T C
ou
nt
100 200 500 1000 1200
![Page 32: Towards Realistic Models for Evolution of Cooperation](https://reader033.vdocuments.net/reader033/viewer/2022052510/56812cb5550346895d916b3f/html5/thumbnails/32.jpg)
PD Game with Group Reputation (100 EPG; varying Inter-group interaction probability)
Group Reputation (min_gr >= 0.5, varying ip)
0
20
40
60
80
100
120
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49
Generation
TF
T C
ou
nt
0.1 0.3 0.325 0.35 1.0
![Page 33: Towards Realistic Models for Evolution of Cooperation](https://reader033.vdocuments.net/reader033/viewer/2022052510/56812cb5550346895d916b3f/html5/thumbnails/33.jpg)
Groups/Organizations: bounded rationality explanation
• Docility– Cooperation (altruism) as an explanation for the
formation of groups/organizations
• Why individuals “identify” with a group?– boundedly rational individuals– increase their survival fitness
(Simon, 1969, 1990, 1991)
![Page 34: Towards Realistic Models for Evolution of Cooperation](https://reader033.vdocuments.net/reader033/viewer/2022052510/56812cb5550346895d916b3f/html5/thumbnails/34.jpg)
PD Game with Docility(50 cooperators and 50 defectors; 100 EPG; 1.0 IP)
Varying intergroup docility, intragroup docility = 1.0
0
20
40
60
80
100
120
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49
Generation
Co
op
erat
or
Co
un
t
0.0 0.4 0.41 0.425 1.0
![Page 35: Towards Realistic Models for Evolution of Cooperation](https://reader033.vdocuments.net/reader033/viewer/2022052510/56812cb5550346895d916b3f/html5/thumbnails/35.jpg)
Conclusion
• Reviewed 5 major approaches to study evolution of cooperation
• Provided 2 main critiques for existing models
• Constructed model extensions addressing the critiques
![Page 36: Towards Realistic Models for Evolution of Cooperation](https://reader033.vdocuments.net/reader033/viewer/2022052510/56812cb5550346895d916b3f/html5/thumbnails/36.jpg)
Implications for Computer Science
• Artificial intelligence– Benevolent agents are not good enough
(c.f., multi-agents systems)– Learning theory can be used to study evolution of
cooperation
• Systems– Improve system design by understanding the
dynamics of agents– Accountability substrate needed for distributed
systems
![Page 37: Towards Realistic Models for Evolution of Cooperation](https://reader033.vdocuments.net/reader033/viewer/2022052510/56812cb5550346895d916b3f/html5/thumbnails/37.jpg)
Future Plan
• Extend the simple group social structure
• Overlapping generations
• Sexual reproduction
• Extend social learning using realistic/robust learning model
![Page 38: Towards Realistic Models for Evolution of Cooperation](https://reader033.vdocuments.net/reader033/viewer/2022052510/56812cb5550346895d916b3f/html5/thumbnails/38.jpg)
Modeling Diploid Organisms
![Page 39: Towards Realistic Models for Evolution of Cooperation](https://reader033.vdocuments.net/reader033/viewer/2022052510/56812cb5550346895d916b3f/html5/thumbnails/39.jpg)
Modeling Diploid Organisms
![Page 40: Towards Realistic Models for Evolution of Cooperation](https://reader033.vdocuments.net/reader033/viewer/2022052510/56812cb5550346895d916b3f/html5/thumbnails/40.jpg)
Modeling Diploid Organisms
Parental Chromosomes One of 2 Child Chromosomes
![Page 41: Towards Realistic Models for Evolution of Cooperation](https://reader033.vdocuments.net/reader033/viewer/2022052510/56812cb5550346895d916b3f/html5/thumbnails/41.jpg)
Simulation Demo
• Recall PD payoff matrix: C D
C R/R S/T
D T/S P/P
• PD strategies: viewed as a probability vectors– Strategy: <PI, PT, PR, PP, PS>
– TFT: < 1, 1, 1, 0, 0 >– AllD: < 0, 0, 0, 0, 0 >– AllC: < 1, 1, 1, 1, 1 >– STFT: < 0, 1, 1, 0, 0 >
![Page 42: Towards Realistic Models for Evolution of Cooperation](https://reader033.vdocuments.net/reader033/viewer/2022052510/56812cb5550346895d916b3f/html5/thumbnails/42.jpg)
Simulation: a search problem
• Search Optimal PD Strategy– Search space: I, T, R, P, S probabilities