evolution and computation - simons institute for the ... · • [stern an 1958] arbitrary boolean...
TRANSCRIPT
![Page 1: Evolution and Computation - Simons Institute for the ... · • [Stern AN 1958] Arbitrary Boolean Functions • What if we have an arbitrary function of genes (no environmental variable](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67815092fe173e9c6bf38e/html5/thumbnails/1.jpg)
Evolution and Computation
Christos H. Papadimitriou The Simons Institute
![Page 2: Evolution and Computation - Simons Institute for the ... · • [Stern AN 1958] Arbitrary Boolean Functions • What if we have an arbitrary function of genes (no environmental variable](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67815092fe173e9c6bf38e/html5/thumbnails/2.jpg)
The Algorithm as a Lens
• It started with Alan Turing, 60 years ago • Algorithmic thinking as a novel and
productive point of view for understanding and transforming the Sciences
• Major theme of the Simons Institute • …and of this symposium • This talk: Evolution
![Page 3: Evolution and Computation - Simons Institute for the ... · • [Stern AN 1958] Arbitrary Boolean Functions • What if we have an arbitrary function of genes (no environmental variable](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67815092fe173e9c6bf38e/html5/thumbnails/3.jpg)
Evolution before Darwin
• Erasmus Darwin
![Page 4: Evolution and Computation - Simons Institute for the ... · • [Stern AN 1958] Arbitrary Boolean Functions • What if we have an arbitrary function of genes (no environmental variable](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67815092fe173e9c6bf38e/html5/thumbnails/4.jpg)
Before Darwin
• J.-B. Lamarck
![Page 5: Evolution and Computation - Simons Institute for the ... · • [Stern AN 1958] Arbitrary Boolean Functions • What if we have an arbitrary function of genes (no environmental variable](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67815092fe173e9c6bf38e/html5/thumbnails/5.jpg)
Before Darwin
• Charles Babbage
[ca. 1820, paraphrased] “God created not species, but the Algorithm for creating species”
![Page 6: Evolution and Computation - Simons Institute for the ... · • [Stern AN 1958] Arbitrary Boolean Functions • What if we have an arbitrary function of genes (no environmental variable](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67815092fe173e9c6bf38e/html5/thumbnails/6.jpg)
The Origin of Species • Natural Selection • Common Ancestry • Possibly the world’s most masterfully
compelling scientific argument • The six editions:1859, 1860, 1861,
1866, 1869, 1872
![Page 7: Evolution and Computation - Simons Institute for the ... · • [Stern AN 1958] Arbitrary Boolean Functions • What if we have an arbitrary function of genes (no environmental variable](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67815092fe173e9c6bf38e/html5/thumbnails/7.jpg)
The Wallace-Darwin papers: Exponential Growth
![Page 8: Evolution and Computation - Simons Institute for the ... · • [Stern AN 1958] Arbitrary Boolean Functions • What if we have an arbitrary function of genes (no environmental variable](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67815092fe173e9c6bf38e/html5/thumbnails/8.jpg)
Brilliant argument, and yet many questions left unasked, e.g.:
• How does novelty arise?
• What is the role of sex?
![Page 9: Evolution and Computation - Simons Institute for the ... · • [Stern AN 1958] Arbitrary Boolean Functions • What if we have an arbitrary function of genes (no environmental variable](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67815092fe173e9c6bf38e/html5/thumbnails/9.jpg)
Cryptography against Lamarck
• A. Weismann
[ca. 1880, paraphrased] “The mapping from genotype to phenotype is one-way”
![Page 10: Evolution and Computation - Simons Institute for the ... · • [Stern AN 1958] Arbitrary Boolean Functions • What if we have an arbitrary function of genes (no environmental variable](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67815092fe173e9c6bf38e/html5/thumbnails/10.jpg)
Genetics
• Gregor Mendel [1866] • Number of citations between 1866 and 1901:
3
![Page 11: Evolution and Computation - Simons Institute for the ... · • [Stern AN 1958] Arbitrary Boolean Functions • What if we have an arbitrary function of genes (no environmental variable](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67815092fe173e9c6bf38e/html5/thumbnails/11.jpg)
The crisis in Evolution 1900 - 1920
• Mendelians vs. Darwinians • Geneticists vs. Biometricists/
Gradualists
![Page 12: Evolution and Computation - Simons Institute for the ... · • [Stern AN 1958] Arbitrary Boolean Functions • What if we have an arbitrary function of genes (no environmental variable](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67815092fe173e9c6bf38e/html5/thumbnails/12.jpg)
The “Modern Synthesis” 1920 - 1950
Fisher – Wright - Haldane
![Page 13: Evolution and Computation - Simons Institute for the ... · • [Stern AN 1958] Arbitrary Boolean Functions • What if we have an arbitrary function of genes (no environmental variable](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67815092fe173e9c6bf38e/html5/thumbnails/13.jpg)
Big questions remain e.g.:
• How does novelty arise?
• What is the role of sex?
![Page 14: Evolution and Computation - Simons Institute for the ... · • [Stern AN 1958] Arbitrary Boolean Functions • What if we have an arbitrary function of genes (no environmental variable](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67815092fe173e9c6bf38e/html5/thumbnails/14.jpg)
Disbelief
“Our thesis is that Neo-Darwinism cannot explain the basic phaenomena of evolution on the basis of physico-chemistry” Schützenberger, 1966
![Page 15: Evolution and Computation - Simons Institute for the ... · • [Stern AN 1958] Arbitrary Boolean Functions • What if we have an arbitrary function of genes (no environmental variable](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67815092fe173e9c6bf38e/html5/thumbnails/15.jpg)
Disbelief at the top
“The eye to this day gives me a cold shudder.”
![Page 16: Evolution and Computation - Simons Institute for the ... · • [Stern AN 1958] Arbitrary Boolean Functions • What if we have an arbitrary function of genes (no environmental variable](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67815092fe173e9c6bf38e/html5/thumbnails/16.jpg)
Disbelief, algorithmic version
“What algorithm could have achieved
all this in a mere 1012 steps?”
(surprise: we have an answer…)
![Page 17: Evolution and Computation - Simons Institute for the ... · • [Stern AN 1958] Arbitrary Boolean Functions • What if we have an arbitrary function of genes (no environmental variable](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67815092fe173e9c6bf38e/html5/thumbnails/17.jpg)
Valiant’s Evolvability
• “ How do you find a 3-billion long string in 3 billion years?” L. G.Valiant
Computationally-inspired model of Evolution shown to amount to a weak form of learning
![Page 18: Evolution and Computation - Simons Institute for the ... · • [Stern AN 1958] Arbitrary Boolean Functions • What if we have an arbitrary function of genes (no environmental variable](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67815092fe173e9c6bf38e/html5/thumbnails/18.jpg)
Evolution and CS Practice: Genetic Algorithms [ca. 1980s]
• To solve an optimization problem… • …create a population of solutions/genotypes • …who evolve through mutations and sex… • …and procreate with success proportional to
their objective function value • Eventually, some very good solutions are
bound to arise in the soup
![Page 19: Evolution and Computation - Simons Institute for the ... · • [Stern AN 1958] Arbitrary Boolean Functions • What if we have an arbitrary function of genes (no environmental variable](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67815092fe173e9c6bf38e/html5/thumbnails/19.jpg)
And in this Corner… Simulated Annealing
• Inspired by asexual reproduction • Mutations are adopted with probability
increasing with fitness/objective differential
![Page 20: Evolution and Computation - Simons Institute for the ... · • [Stern AN 1958] Arbitrary Boolean Functions • What if we have an arbitrary function of genes (no environmental variable](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67815092fe173e9c6bf38e/html5/thumbnails/20.jpg)
The Mystery of Sex Deepens
• Simulated annealing (asexual reproduction) works fine
• Genetic algorithms (sexual reproduction) don’t work
• In Nature, the opposite happens: Sex is successful and ubiquitous
![Page 21: Evolution and Computation - Simons Institute for the ... · • [Stern AN 1958] Arbitrary Boolean Functions • What if we have an arbitrary function of genes (no environmental variable](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67815092fe173e9c6bf38e/html5/thumbnails/21.jpg)
?
![Page 22: Evolution and Computation - Simons Institute for the ... · • [Stern AN 1958] Arbitrary Boolean Functions • What if we have an arbitrary function of genes (no environmental variable](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67815092fe173e9c6bf38e/html5/thumbnails/22.jpg)
A Radical Thought
• What if sex is a mediocre optimizer of fitness (= expectation of offspring)?
• What if sex optimizes something else? • And what if this something else is its
raison d’ être?
![Page 23: Evolution and Computation - Simons Institute for the ... · • [Stern AN 1958] Arbitrary Boolean Functions • What if we have an arbitrary function of genes (no environmental variable](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67815092fe173e9c6bf38e/html5/thumbnails/23.jpg)
Mixability!
• [Livnat et al, PNAS 2008] • Simulations show that natural selection
under asex optimizes fitness • But under sex it optimizes mixability: • = The ability of alleles (gene variants) to
perform well with a broad spectrum of other alleles
![Page 24: Evolution and Computation - Simons Institute for the ... · • [Stern AN 1958] Arbitrary Boolean Functions • What if we have an arbitrary function of genes (no environmental variable](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67815092fe173e9c6bf38e/html5/thumbnails/24.jpg)
Explaining Mixability: The Fisher-Wright model
• Fitness landscape of a 2-gene organism
3" 2" 4" 5" 4"1" 0" 0" 7" 2"2" 1" 0" 4" 3"1" 8" 1" 3" 2"
""
3" 2" 4" 5" 4"1" 0" 0" 7" 2"2" 1" 0" 4" 3"1" 8" 1" 3" 2"
""
!!!!
3! 2! 4! 5! 4!1! 0! 0! 7! 2!2! 1! 0! 4! 3!1! 8! 1! 3! 2!
Rows: alleles of gene A
Columns: alleles of gene B
Entries: fitness of the combination
![Page 25: Evolution and Computation - Simons Institute for the ... · • [Stern AN 1958] Arbitrary Boolean Functions • What if we have an arbitrary function of genes (no environmental variable](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67815092fe173e9c6bf38e/html5/thumbnails/25.jpg)
Explaining Mixability (cont)
• Asex will select the largest numbers
3" 2" 4" 5" 4"1" 0" 0" 7" 2"2" 1" 0" 4" 3"1" 8" 1" 3" 2"
""
3" 2" 4" 5" 4"1" 0" 0" 7" 2"2" 1" 0" 4" 3"1" 8" 1" 3" 2"
""
!!!!
3! 2! 4! 5! 4!1! 0! 0! 7! 2!2! 1! 0! 4! 3!1! 8! 1! 3! 2!
![Page 26: Evolution and Computation - Simons Institute for the ... · • [Stern AN 1958] Arbitrary Boolean Functions • What if we have an arbitrary function of genes (no environmental variable](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67815092fe173e9c6bf38e/html5/thumbnails/26.jpg)
Explaining Mixability (cont)
• But sex will select the rows and columns with the largest average
3" 2" 4" 5" 4"1" 0" 0" 7" 2"2" 1" 0" 4" 3"1" 8" 1" 3" 2"
""
3" 2" 4" 5" 4"1" 0" 0" 7" 2"2" 1" 0" 4" 3"1" 8" 1" 3" 2"
""
!!!!
3! 2! 4! 5! 4!1! 0! 0! 7! 2!2! 1! 0! 4! 3!1! 8! 1! 3! 2!
![Page 27: Evolution and Computation - Simons Institute for the ... · • [Stern AN 1958] Arbitrary Boolean Functions • What if we have an arbitrary function of genes (no environmental variable](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67815092fe173e9c6bf38e/html5/thumbnails/27.jpg)
Pointer Dogs
![Page 28: Evolution and Computation - Simons Institute for the ... · • [Stern AN 1958] Arbitrary Boolean Functions • What if we have an arbitrary function of genes (no environmental variable](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67815092fe173e9c6bf38e/html5/thumbnails/28.jpg)
Pointer Dogs
C. H. Waddington
![Page 29: Evolution and Computation - Simons Institute for the ... · • [Stern AN 1958] Arbitrary Boolean Functions • What if we have an arbitrary function of genes (no environmental variable](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67815092fe173e9c6bf38e/html5/thumbnails/29.jpg)
Waddington’s Experiment (1952)
Generation 1 Temp: 20o C
![Page 30: Evolution and Computation - Simons Institute for the ... · • [Stern AN 1958] Arbitrary Boolean Functions • What if we have an arbitrary function of genes (no environmental variable](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67815092fe173e9c6bf38e/html5/thumbnails/30.jpg)
Waddington’s Experiment (1952)
Generation 2-4 Temp: 40o C ~15% changed Select and breed those
![Page 31: Evolution and Computation - Simons Institute for the ... · • [Stern AN 1958] Arbitrary Boolean Functions • What if we have an arbitrary function of genes (no environmental variable](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67815092fe173e9c6bf38e/html5/thumbnails/31.jpg)
Waddington’s Experiment (1952)
Generation 5 Temp: 40o C ~60% changed Select and breed those
![Page 32: Evolution and Computation - Simons Institute for the ... · • [Stern AN 1958] Arbitrary Boolean Functions • What if we have an arbitrary function of genes (no environmental variable](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67815092fe173e9c6bf38e/html5/thumbnails/32.jpg)
Waddington’s Experiment (1952)
Generation 6 Temp: 40o C ~63% changed Select and breed those
![Page 33: Evolution and Computation - Simons Institute for the ... · • [Stern AN 1958] Arbitrary Boolean Functions • What if we have an arbitrary function of genes (no environmental variable](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67815092fe173e9c6bf38e/html5/thumbnails/33.jpg)
Waddington’s Experiment (1952)
(…) Generation 20 Temp: 40o C ~99% changed
![Page 34: Evolution and Computation - Simons Institute for the ... · • [Stern AN 1958] Arbitrary Boolean Functions • What if we have an arbitrary function of genes (no environmental variable](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67815092fe173e9c6bf38e/html5/thumbnails/34.jpg)
Surprise!
Generation 20 Temp: 20o C ~25% stay changed!!
![Page 35: Evolution and Computation - Simons Institute for the ... · • [Stern AN 1958] Arbitrary Boolean Functions • What if we have an arbitrary function of genes (no environmental variable](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67815092fe173e9c6bf38e/html5/thumbnails/35.jpg)
Genetic Assimilation
• Adaptations to the environment become genetic!
![Page 36: Evolution and Computation - Simons Institute for the ... · • [Stern AN 1958] Arbitrary Boolean Functions • What if we have an arbitrary function of genes (no environmental variable](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67815092fe173e9c6bf38e/html5/thumbnails/36.jpg)
Is There a Genetic Explanation?
Function f ( x, h ) with these properties: • Initially, Prob x ~ p[0] [f ( x, h = 0)] ≈ 0% • Then Probp[0][f ( x, 1)] ≈ 15% • After breeding Probp[1][f ( x, 1)] ≈ 60% • Successive breedings, Probp[20][f ( x,1)] ≈ 99% • Finally, Probp[20][f ( x, 0)] ≈ 25%
![Page 37: Evolution and Computation - Simons Institute for the ... · • [Stern AN 1958] Arbitrary Boolean Functions • What if we have an arbitrary function of genes (no environmental variable](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67815092fe173e9c6bf38e/html5/thumbnails/37.jpg)
A Genetic Explanation
• Suppose that “red head” is this Boolean function of 10 genes and “high temperature”
“red head” = “x1 + x2 + … + x10 + 3h ≥ 10” • Suppose also that the genes are independent
random variables, with pi initially half, say • All properties of the Waddington
experiment satisfied • [Stern AN 1958]
![Page 38: Evolution and Computation - Simons Institute for the ... · • [Stern AN 1958] Arbitrary Boolean Functions • What if we have an arbitrary function of genes (no environmental variable](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67815092fe173e9c6bf38e/html5/thumbnails/38.jpg)
Arbitrary Boolean Functions
• What if we have an arbitrary function of genes (no environmental variable h)
• Suppose the satisfying genotypes have a fitness advantage (1 + ε vs. 1, say)
• Will this trait be fixed eventually?
![Page 39: Evolution and Computation - Simons Institute for the ... · • [Stern AN 1958] Arbitrary Boolean Functions • What if we have an arbitrary function of genes (no environmental variable](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67815092fe173e9c6bf38e/html5/thumbnails/39.jpg)
Arbitrary Functions: Yes!
Theorem: Any Boolean function of genes which confers an evolutionary advantage will be eventually fixed (with high probability) (2013; with Adi Livnat, Aviad Rubinstein, Greg Valiant, Andrew Won)
![Page 40: Evolution and Computation - Simons Institute for the ... · • [Stern AN 1958] Arbitrary Boolean Functions • What if we have an arbitrary function of genes (no environmental variable](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67815092fe173e9c6bf38e/html5/thumbnails/40.jpg)
Which means that… • “With sex, all moderate-sized Boolean
functions are evolvable.” • “Look, Ma, no mutations!” • Novel complex traits can emerge, through
sex, in the whole population, without “Fisherian propagation”
![Page 41: Evolution and Computation - Simons Institute for the ... · • [Stern AN 1958] Arbitrary Boolean Functions • What if we have an arbitrary function of genes (no environmental variable](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67815092fe173e9c6bf38e/html5/thumbnails/41.jpg)
Neutral Theory and Weak Selection
• Kimura 1970: Evolution proceeds not by leaps upwards, but mostly “horizontally,” through statistical drift
• Weak selection: the values in the fitness matrix are very close, say in [1 – ε, 1 + ε]
![Page 42: Evolution and Computation - Simons Institute for the ... · • [Stern AN 1958] Arbitrary Boolean Functions • What if we have an arbitrary function of genes (no environmental variable](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67815092fe173e9c6bf38e/html5/thumbnails/42.jpg)
Changing the subject: The experts problem
• Every day you must choose one of n experts • The advice of expert i on day t results in a
gain G[i, t] in [-1, 1] • Challenge: Do as well as the best expert in
retrospect • Surprise: It can be done! • [Hannan 1958, Cover 1980, Winnow,
Boosting, no-regret learning, MWUA, …]
![Page 43: Evolution and Computation - Simons Institute for the ... · • [Stern AN 1958] Arbitrary Boolean Functions • What if we have an arbitrary function of genes (no environmental variable](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67815092fe173e9c6bf38e/html5/thumbnails/43.jpg)
Multiplicative weights update
• Initially, assign all experts same weight/probability
• At each step, increase the weight of each by (1 + ε G[i, t]) (and then normalize)
• Theorem: Does as well as the best expert • MWUA solves: zero-sum games, linear
programming, convex programming, network congestion,…
![Page 44: Evolution and Computation - Simons Institute for the ... · • [Stern AN 1958] Arbitrary Boolean Functions • What if we have an arbitrary function of genes (no environmental variable](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67815092fe173e9c6bf38e/html5/thumbnails/44.jpg)
Disbelief
Computer scientists find it hard to believe that such a crude technique solves all these sophisticated problems
(cf: the other disbelievers)
![Page 45: Evolution and Computation - Simons Institute for the ... · • [Stern AN 1958] Arbitrary Boolean Functions • What if we have an arbitrary function of genes (no environmental variable](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67815092fe173e9c6bf38e/html5/thumbnails/45.jpg)
Theorem: Under weak selection, evolution of a species is a game • the players are the genes • the strategies are the alleles • the common utility is the fitness of the
organism (coordination game) • the probabilities are the allele frequencies • game is played through multiplicative updates
(2013, with E. Chastain, A. Livnat, U. Vazirani)
![Page 46: Evolution and Computation - Simons Institute for the ... · • [Stern AN 1958] Arbitrary Boolean Functions • What if we have an arbitrary function of genes (no environmental variable](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67815092fe173e9c6bf38e/html5/thumbnails/46.jpg)
Finally…
• Variance preservation: MWUA is known to maximize entropy
• The curious successes of Evolution and MWUA: Two mysteries united
• This is the role of sex in Evolution
![Page 47: Evolution and Computation - Simons Institute for the ... · • [Stern AN 1958] Arbitrary Boolean Functions • What if we have an arbitrary function of genes (no environmental variable](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67815092fe173e9c6bf38e/html5/thumbnails/47.jpg)
Sooooo…
• The theory of life is deep and fascinating • And rife with interesting technical problems
that are unexpectedly computational • How robust? (Strong selection? Complex
landscapes?) • Environment? Mutations? • Test?
![Page 48: Evolution and Computation - Simons Institute for the ... · • [Stern AN 1958] Arbitrary Boolean Functions • What if we have an arbitrary function of genes (no environmental variable](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67815092fe173e9c6bf38e/html5/thumbnails/48.jpg)
Thanks!