Statistical Mechanics for GAsA gentle introduction

Presented by :

Yann SEMET

Universite de Technologie de Compiegne

Overview

Motivations and general idea

Definitions

Selection analysis (pb. independent)

Problem specific analysis : mutation, crossover

Results on OneMax

Beyond simple problems

2 documents

An analysis of Genetic Algorithms Using Statistical Mechanics

Prugel-Bennett, Shapiro. 1994

Modeling the dynamics of Gas using Statistical Mechanics

Rattray. 1996

Motivations

Markov chains :Exact model

Gets intractable with size

Statistical mechanicsProbabilistic model

More compact

Macroscopic description

Macroscopics

Cumulants

Mean correlation

Evaluates the evolution of fitness distribution

Modelling the dynamics

Each genetic operator :A set of difference equations (on macroscopics)

Iteration

Non trivial terms : Maximum entropy ansatz

Finite population effectsA finite sample from an infinite population

Selection

Infinite population again

Definitions

Genotype, phenotype and fitness :

Fitness distribution :

Cumulants

Definition :

First two :

Cumulants (cont.)

Infinite population :

Finite sample corrections :

Gram-Charlier expansion

A convenient approximation

Correlation

A measure of genotype similarity

Mean value :

Best population member

Our goal after all

Modelling selection

Problem independent

A general scheme :

2 stages :Random sampling from infinite population

Generating a new infinite one

Selection (cont.)

Generating the cumulants :

Selection (cont.)

Expansion :

Finally :

Correlation after selection

2 terms :

Duplication

Natural change (problem specific)

Final approximation :

Selection schemes

Particular schemes :Boltzmann

Truncation

Ranking

Tournament

Tackling problems

Problem specific operators

A convenient class of problems : Functions of an additive genotype

Cumulants :

Mutation (1)

Mutation (2)

Cumulants :

Notice non trivial terms

Correlation :

Crossover

A generalized form of uniform crossover :

Cumulants :

Mean correlation unchanged

Maximum entropy ansatz

Calculate terms non trivially related to known macroscopics

Assumptions on allele distribution

2 constraintsMean phenotype

Correlation

Results

Onemax problem

2nd class of problems

Fitness=stochastic function of phenotype

Test problem : perceptron with binary weights

Competent model :Size population accurately to remove noise

Size training batch consequently

A NP hard problem

Storing random pattern in a binary perceptron

Insight gained

Half failure :Technical difficulties

Inconsistencies

Incomplete model

Extensions of the model

Two tests :1 simple diploid GA

1 temporally varying fitness

Successful description under :Bit-simulated crossover

Extra constraints for MEA

Summary

Motivation of macroscopic models

Cumulants

Mean correlation

Dynamics modeling

Simple problems

Extensions

Conclusions

StrengthsCompactness and accuracy

Finite population effects

WeaknessesLimitations of MAE

NP hard problem inaccurately modeled

Technical limitation

Fundamental limitations ?

Punctuated equilibria