coevolutionary learning with genetic algorithms. problem for learning algorithms: how to select...
Post on 19-Dec-2015
219 views
TRANSCRIPT
![Page 1: Coevolutionary Learning with Genetic Algorithms. Problem for learning algorithms: How to select “training environments” appropriate to different stages](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d2d5503460f94a04ad2/html5/thumbnails/1.jpg)
Coevolutionary Learning with Genetic Algorithms
![Page 2: Coevolutionary Learning with Genetic Algorithms. Problem for learning algorithms: How to select “training environments” appropriate to different stages](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d2d5503460f94a04ad2/html5/thumbnails/2.jpg)
Problem for learning algorithms:
How to select “training environments” appropriate to different stages of learning?
One solution: Co-evolve training examples, using inspiration from host-parasite coevolution in nature.
![Page 3: Coevolutionary Learning with Genetic Algorithms. Problem for learning algorithms: How to select “training environments” appropriate to different stages](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d2d5503460f94a04ad2/html5/thumbnails/3.jpg)
Host-parasite coevolution in nature
• Hosts evolve defenses against parasites
• Parasites find ways to overcome defenses
• Hosts evolve new defenses
• Continual “biological arms race”
![Page 4: Coevolutionary Learning with Genetic Algorithms. Problem for learning algorithms: How to select “training environments” appropriate to different stages](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d2d5503460f94a04ad2/html5/thumbnails/4.jpg)
Heliconius-egg mimicry in Passiflora
http://www.ucl.ac.uk/~ucbhdjm/courses/b242/Coevol/Coevol.html
![Page 5: Coevolutionary Learning with Genetic Algorithms. Problem for learning algorithms: How to select “training environments” appropriate to different stages](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d2d5503460f94a04ad2/html5/thumbnails/5.jpg)
• Darwin recognized the importance of coevolution in driving evolution
![Page 6: Coevolutionary Learning with Genetic Algorithms. Problem for learning algorithms: How to select “training environments” appropriate to different stages](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d2d5503460f94a04ad2/html5/thumbnails/6.jpg)
• Darwin recognized the importance of coevolution in driving evolution
• Coevolution was later hypothesized to be major factor in evolution of sexual reproduction
![Page 7: Coevolutionary Learning with Genetic Algorithms. Problem for learning algorithms: How to select “training environments” appropriate to different stages](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d2d5503460f94a04ad2/html5/thumbnails/7.jpg)
Coevolutionary Learning
![Page 8: Coevolutionary Learning with Genetic Algorithms. Problem for learning algorithms: How to select “training environments” appropriate to different stages](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d2d5503460f94a04ad2/html5/thumbnails/8.jpg)
Coevolutionary Learning
Candidate solutions and training environments coevolve.
![Page 9: Coevolutionary Learning with Genetic Algorithms. Problem for learning algorithms: How to select “training environments” appropriate to different stages](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d2d5503460f94a04ad2/html5/thumbnails/9.jpg)
Coevolutionary Learning
Candidate solutions and training environments coevolve.
– Fitness of candidate solution (host): how well it performs on training examples.
![Page 10: Coevolutionary Learning with Genetic Algorithms. Problem for learning algorithms: How to select “training environments” appropriate to different stages](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d2d5503460f94a04ad2/html5/thumbnails/10.jpg)
Coevolutionary Learning
Candidate solutions and training environments coevolve.
– Fitness of candidate solution (host): how well it performs on training examples.
– Fitness of training example (parasite): how well it defeats candidate solutions.
![Page 11: Coevolutionary Learning with Genetic Algorithms. Problem for learning algorithms: How to select “training environments” appropriate to different stages](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d2d5503460f94a04ad2/html5/thumbnails/11.jpg)
Sample Applications of Coevolutionary Learning
![Page 12: Coevolutionary Learning with Genetic Algorithms. Problem for learning algorithms: How to select “training environments” appropriate to different stages](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d2d5503460f94a04ad2/html5/thumbnails/12.jpg)
Sample Applications of Coevolutionary Learning
– Coevolving minimal sorting algorithms (Hillis)
• Hosts: Candidate sorting algorithms• Parasites: Lists of items to sort
![Page 13: Coevolutionary Learning with Genetic Algorithms. Problem for learning algorithms: How to select “training environments” appropriate to different stages](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d2d5503460f94a04ad2/html5/thumbnails/13.jpg)
Sample Applications of Coevolutionary Learning
– Game playing strategies (e.g., Rosin & Belew; Fogel; Juillé & Pollack)
• Hosts: Candidate strategies for Nim, 3D Tic Tac Toe, backgammon, etc.
• Parasites: Another population of candidate strategies
![Page 14: Coevolutionary Learning with Genetic Algorithms. Problem for learning algorithms: How to select “training environments” appropriate to different stages](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d2d5503460f94a04ad2/html5/thumbnails/14.jpg)
Sample Applications of Coevolutionary Learning
– HIV drug design (e.g., Rosin)
• Hosts: Candidate protease inhibitors to match HIV protease enzymes
• Parasites: Evolving protease enzymes
![Page 15: Coevolutionary Learning with Genetic Algorithms. Problem for learning algorithms: How to select “training environments” appropriate to different stages](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d2d5503460f94a04ad2/html5/thumbnails/15.jpg)
Sample Applications of Coevolutionary Learning
– Robot behavior (e.g., Sims; Nolfi & Floreano)
• Hosts: Robot control programs
• Parasites: Obstacles; mazes; competing robot control programs;
![Page 16: Coevolutionary Learning with Genetic Algorithms. Problem for learning algorithms: How to select “training environments” appropriate to different stages](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d2d5503460f94a04ad2/html5/thumbnails/16.jpg)
Why should we expect coevolutionary learning to speed-up and/or improve evolution?
![Page 17: Coevolutionary Learning with Genetic Algorithms. Problem for learning algorithms: How to select “training environments” appropriate to different stages](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d2d5503460f94a04ad2/html5/thumbnails/17.jpg)
Why should we expect coevolutionary learning to speed-up and/or improve evolution?
“Biological” arms races
Increased “biodiversity”?
![Page 18: Coevolutionary Learning with Genetic Algorithms. Problem for learning algorithms: How to select “training environments” appropriate to different stages](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d2d5503460f94a04ad2/html5/thumbnails/18.jpg)
Practical problems observed in coevolutionary learning
![Page 19: Coevolutionary Learning with Genetic Algorithms. Problem for learning algorithms: How to select “training environments” appropriate to different stages](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d2d5503460f94a04ad2/html5/thumbnails/19.jpg)
Practical problems observed in coevolutionary learning
• Cycling:
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
0 5 10 15 20 25 30
fitn
ess
generation
hosts
parasites
low “true” fitness
![Page 20: Coevolutionary Learning with Genetic Algorithms. Problem for learning algorithms: How to select “training environments” appropriate to different stages](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d2d5503460f94a04ad2/html5/thumbnails/20.jpg)
• Loss of gradient for hosts
0
5
10
15
20
0 1 2 3 4 5 6 7 8
fitn
ess
generation
hosts
parasites
![Page 21: Coevolutionary Learning with Genetic Algorithms. Problem for learning algorithms: How to select “training environments” appropriate to different stages](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d2d5503460f94a04ad2/html5/thumbnails/21.jpg)
• Over-“virulence” of parasites
0
5
10
15
20
0 1 2 3 4 5 6 7 8
fitn
ess
generation
hosts
parasites
![Page 22: Coevolutionary Learning with Genetic Algorithms. Problem for learning algorithms: How to select “training environments” appropriate to different stages](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d2d5503460f94a04ad2/html5/thumbnails/22.jpg)
Hypothesis
Distributing host and parasite populations in space will overcome these impediments by:
– Preserving diversity in the populations
– Fostering arms races between hosts and parasites
![Page 23: Coevolutionary Learning with Genetic Algorithms. Problem for learning algorithms: How to select “training environments” appropriate to different stages](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d2d5503460f94a04ad2/html5/thumbnails/23.jpg)
Our Experiments
(Mitchell, Thomure, & Williams, 2006)
Spatial Non-spatial
Coevolution
Evolution
![Page 24: Coevolutionary Learning with Genetic Algorithms. Problem for learning algorithms: How to select “training environments” appropriate to different stages](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d2d5503460f94a04ad2/html5/thumbnails/24.jpg)
Problem domains used in experiments
1. Function induction from data
2. Cellular automata
I.e., given a set of points generated bya “secret” function, find the function.
• Hosts: Candidate functions
• Parasites: Points generated bythe secret function
http://webscripts.softpedia.com/scriptScreenshots/Polynomial-curve-fitting-Screenshots-62898.html
![Page 25: Coevolutionary Learning with Genetic Algorithms. Problem for learning algorithms: How to select “training environments” appropriate to different stages](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d2d5503460f94a04ad2/html5/thumbnails/25.jpg)
• Host and parasite populations live on a two-dimensional toroidal (i.e., donut-shaped) grid with one host (h) and one parasite (p) per site
Spatial Coevolution
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
![Page 26: Coevolutionary Learning with Genetic Algorithms. Problem for learning algorithms: How to select “training environments” appropriate to different stages](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d2d5503460f94a04ad2/html5/thumbnails/26.jpg)
• Host and parasite populations live on a two-dimensional toroidal (i.e., donut-shaped) grid with one host (h) and one parasite (p) per site
Spatial Coevolution
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
€
Fitness of host h = fraction of
9 neighboring parasites (p)
dealt with correctly
€
Fitness of parasite p = fraction of
9 neighboring hosts (h)
which get p wrong
![Page 27: Coevolutionary Learning with Genetic Algorithms. Problem for learning algorithms: How to select “training environments” appropriate to different stages](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d2d5503460f94a04ad2/html5/thumbnails/27.jpg)
• Host and parasite populations live on a two-dimensional toroidal (i.e., donut-shaped) grid with one host (h) and one parasite (p) per site
Spatial Coevolution
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
hp
At each generation, each h is replaced by mutated copy of winner of tournament among itself and 8 neighboring hosts.
At each generation, each p is replaced by mutated copy of winner of tournament among itself and 8 neighboring parasites.
![Page 28: Coevolutionary Learning with Genetic Algorithms. Problem for learning algorithms: How to select “training environments” appropriate to different stages](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d2d5503460f94a04ad2/html5/thumbnails/28.jpg)
• Host and parasite populations live in a non-spatial world (no notion of “neighbor” or “distance)
Non-Spatial Coevolution
h
p
h
hh
h
h
p
p
p
p
p
p
p
p
hh
h hp
p
p
h
h
€
Fitness of host h = fraction of
9 random parasites (p)
dealt with correctly
€
Fitness of parasite p = fraction of
9 random hosts (h)
which get p wrong
![Page 29: Coevolutionary Learning with Genetic Algorithms. Problem for learning algorithms: How to select “training environments” appropriate to different stages](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d2d5503460f94a04ad2/html5/thumbnails/29.jpg)
• Host and parasite populations live in a non-spatial world (no notion of “neighbor” or “distance)
Non-Spatial Coevolution
h
p
h
hh
h
h
p
p
p
p
p
p
p
p
hh
h hp
p
p
h
h
At each generation, each h is replaced by mutated copy of winner of tournament among itself and 8 random hosts.
At each generation, each p is replaced by mutated copy of winner of tournament among itself and 8 random parasites.
![Page 30: Coevolutionary Learning with Genetic Algorithms. Problem for learning algorithms: How to select “training environments” appropriate to different stages](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d2d5503460f94a04ad2/html5/thumbnails/30.jpg)
• Spatial Evolution:
– Same as spatial coevolution, except parasites don’t evolve.
– A new population of random parasites is generated at each generation.
![Page 31: Coevolutionary Learning with Genetic Algorithms. Problem for learning algorithms: How to select “training environments” appropriate to different stages](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d2d5503460f94a04ad2/html5/thumbnails/31.jpg)
• Non-Spatial Evolution:
– Same as non-spatial coevolution, except parasites don’t evolve.
– A new sample of random parasites is generated at each generation.
![Page 32: Coevolutionary Learning with Genetic Algorithms. Problem for learning algorithms: How to select “training environments” appropriate to different stages](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d2d5503460f94a04ad2/html5/thumbnails/32.jpg)
Results
Function Induction
Cellular Automata
Spatial Coev. 78% (39/50) 67% (20/30)
Non-Spatial Coev.
0% (0/50) 0% (0/20)
Spatial Evol. 14% (7/50) 0% (0/30)
Non-Spatial Evol.
6% (3/50) 0% (0/20)
Percentage of successful runs
![Page 33: Coevolutionary Learning with Genetic Algorithms. Problem for learning algorithms: How to select “training environments” appropriate to different stages](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d2d5503460f94a04ad2/html5/thumbnails/33.jpg)
In short: Spatial coevolution significantly out-performs other methods on both problems
![Page 34: Coevolutionary Learning with Genetic Algorithms. Problem for learning algorithms: How to select “training environments” appropriate to different stages](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d2d5503460f94a04ad2/html5/thumbnails/34.jpg)
Analysis
Why was spatial coevolution successful?
Hypotheses:
1. Maintains diversity over long period of time
2. Creates extended “arms race” between hosts and parasite populations
Here we examine these hypotheses for the cellular automaton task.
[That is, we will do so after the lecture on cellular automata. To be continued…]