Prof. Pier Luca Lanzi

Real-Coded Extended Compact Genetic Algorithm based on Mixtures of ModelsPier Luca Lanzi, Luigi Nichetti, Kumara Sastry,Davide Voltini, and David E. Goldberg

OBUPM-2008, July 12, 2008, Atlanta, GA

Estimation of Distribution Algorithms

Population

Probabilistic

ModelSelectionNew

Population

Extended Compact Genetic Algorithm

Motivations

OBUPM-2008, July 12, 2008, Atlanta, GA

What are the Motivations? 6

Real-Coded ECGA

Adaptive Discretization

Ying Ping Chen & c.

Real-Coded BOA

Chang Wook Ahn

Real-Coded ECGA

Basic Discretization

Fossati & al.

Is there something simpler?What if we downgrade RBOA to ECGA models?

This Talk:

From RBOA to RECGA

Real-Coded ECGA

OBUPM-2008, July 12, 2008, Atlanta, GA

8What is the Approach (This Time)

No discretization

Model Building

Model Scoring

Model Fitting

Model Sampling

What are the differences?

The model is simpler than the one used in RBOA

Therefore it should be faster

The analysis should be easier

More complex than the other two RECGAs

Real Domain

OBUPM-2008, July 12, 2008, Atlanta, GA

Model Building 10

A

Selected Population

Clustering Distributions for structure A B C D

A B C D

A B C D

A B C D

A B C D

B C D

A B C D

OBUPM-2008, July 12, 2008, Atlanta, GA

Model Building 11

[A B] C DA B C D

Accuracy + Complexity Accuracy + Complexityvs

OBUPM-2008, July 12, 2008, Atlanta, GA

12

A B C D

A C [B D]

[A C] B D

[A B] C D

A [B C] D

A B [C D]

[A D] B C

[A B] [C D]

[A B C] D

[A B D] C

[A B C D]

Searching for the Model Structure

Candidate Structures from A B C D

Candidate Structures from [A B] C D

Candidate Structures from [A B] [C D]

Final Structure:

[A B] [C D]

OBUPM-2008, July 12, 2008, Atlanta, GA

14

Selected population partitioned according to the model structure

Clustering(again)

A B C D

C D

A B

A B A B

A B

Model Building: Parameters Fitting

A B

Compute model parameters

Structure [A B] [C D]

OBUPM-2008, July 12, 2008, Atlanta, GA

15

Selected population partitioned according to the model structure

Clustering(again)

A B C D

C D

A B

Model Building: Parameters Fitting

A B

Compute model parameters

Structure [A B] [C D]

OBUPM-2008, July 12, 2008, Atlanta, GA

16

Selected population partitioned according to the model structure

Clustering(again)

A B C D

C D

A B

Model Building: Parameters Fitting

A B

Compute model parameters

Structure [A B] [C D]

OBUPM-2008, July 12, 2008, Atlanta, GA

17

Selected population partitioned according to the model structure

Clustering(again)

A B C D

C D

A B

Model Building: Parameters Fitting

A B

Compute model parameters

Structure [A B] [C D]

OBUPM-2008, July 12, 2008, Atlanta, GA

18

New Individual

Probabilistic model from the previous step

Random selection of the model components

Model Sampling

B C DA B C DA

[A B]

[C D]

OBUPM-2008, July 12, 2008, Atlanta, GA

19

New Individual

Probabilistic model from the previous step

Random selection of the model components

Model Sampling

B C DA B C DA

[A B]

[C D]

OBUPM-2008, July 12, 2008, Atlanta, GA

20

New Individual

Probabilistic model from the previous step

Random selection of the model components

Model Sampling

B C DA B C DA

[A B]

[C D]B C DA

OBUPM-2008, July 12, 2008, Atlanta, GA

What Experiments?

Scalability Analysis

Number of evaluations vs number of variables

Population size vs number of variables

Influence of the clustering algorithm

Model Selection (typically, k-means)

Model Sampling (typically, adaptive, randomized leader algorithm)

Requirement for convergence during bisection

In the 97% of the runs,

the 99.9% of the population is an optimum

22

OBUPM-2008, July 12, 2008, Atlanta, GA

First a Simple Function 23

Sphere Function

Analyze scalability when an univariate model is enough

OBUPM-2008, July 12, 2008, Atlanta, GA

First a Simple Function 24

Model Selection

O(n1.75)-O(0.5)

Model Fitting

OBUPM-2008, July 12, 2008, Atlanta, GA

Adapting Everything...

RLA+RLA scales as O(n1.1)

25

OBUPM-2008, July 12, 2008, Atlanta, GA

RDP

26Then a Deceptive One...

Analyze scalability when competent recombination is needed

+ + +….

+ +….

OBUPM-2008, July 12, 2008, Atlanta, GA

27Then a Deceptive One...

RECGA scales

subquadratically

OBUPM-2008, July 12, 2008, Atlanta, GA

More Results ... 29

Model Selection

(subquadratic)

Model Fitting

(subquadratic)

OBUPM-2008, July 12, 2008, Atlanta, GA

Adapting Everything... 30

Conclusions

OBUPM-2008, July 12, 2008, Atlanta, GA

Conclusions

Yet, another, real-coded ECGA

Behaves similarly to the simpler real-coded ECGA

Rules of thumb?

Sphere function

Scalability similar to univariate methods

The number of clusters using for selection, matters.

Real Deceptive function

Number of generations scale subquadratically

Increasing the number of clusters in sampling matters

Overall, the full adaptive version provides a good trade-off

34

OBUPM-2008, July 12, 2008, Atlanta, GA

What Next?

Please, not another real-coded ECGA ...

How do we go from simple discretization (Fossati et al.),Adaptive Discretization (Chen et al.), to this model?

Virtual alphabets (Goldberg 1991)

Theory of convergence for real-coded GAs

Selection restricts subsequent search to intervals with above average fitness, dimension by dimension

Intervals form the characters of a virtual alphabet, searched during recombination

35

Virtual alphabets, discretizations, models?

OBUPM-2008, July 12, 2008, Atlanta, GA

Thank You!

Any Question?