Population Methods

Metaheuristics

Byung-Hyun Ha

R1

2

Outline

Introduction

Evolution Strategies

Genetic Algorithm

Exploitative Variations

Differential Evolution

Particle Swarm Optimization

Summary

3

Introduction

Population-based methods Keeping around a sample of candidate solutions rather than a single

candidate solution Candidate solutions affect how other candidates will hill-climb

• Poor solutions to be rejected and new ones created, by good solutions• Solutions to be Tweaked in the direction of the better solutions• c.f., different from a parallel hill-climber

Stealing concepts from biology• Evolutionary Computation, broader concept

Evolutionary Algorithm (EA)• Generational algorithms

• e.g., Genetic Algorithm (GA)• Steady-state algorithms

• e.g., Evolution Strategies (ES)

4

Introduction

Common terms used in Evolutionary Computation Individual and population

• A candidate solution and set of candidate solutions

Genotype or genome• An individual’s data structure, as used during breeding• Chromosome -- a genotype in the form of a fixed-length vector• Gene -- particular slot position in a chromosome

Phenotype• How the individual operates during fitness assessment

1 0 0 1

U' = {1, 4} (include item 1 and 4)

genotype

phenotype

knapsack problem artificial ant (Ch. 4)

5

Introduction

Common terms used in Evolutionary Computation (cont’d) Child and parent

• A child is the Tweaked copy of a candidate solution (its parent)

Mutation• Plain Tweaking (asexual breeding)

Crossover or recombination• A Tweak which takes two parents (sexual breeding)

Fitness and fitness landscape• Quality and quality function

Selection• Picking individuals based on their fitness

Breeding• Producing children from a population of parents by selection and Tweaking

Generation• One cycle of fitness assessment, breeding, and population reassembly; or• The population produced each such cycle

6

Introduction

Common terms used in Evolutionary Computation (cont’d)

0 0 1 1

0 0 0 1

0 1 1 0

1 0 0 1

1 0 0 0

selection 0 1 0 1

0 1 0 1

0 0 1 1

1 0 0 1

mutation

1 1 0 1

crossover

0 0 0 1

1 0 1 1

parents

children

ith population

(i + 1)th population1100 1100

1011 1011

0000 0000

1001 1001

1101 1101

1010 1010

ith generation

(i + 1)th generation

7

Introduction

Abstract form of EA

Population initialization Biased to good ones vs. uniform randomness?

8

Evolution Strategies

Truncation selection and mutation only (, ) and ( + ) algorithm

9

Evolution Strategies

(, ) algorithm (/ children) for each of selected parents ( in population) e.g., (5, 20) Evolution Strategy

Effect of parameters Size of , size of , degree to which Mutation is performed

10

Evolution Strategies

( + ) algorithm ( selected parents) + (/ children of each selection) next population More exploitative than (, ), because parents compete with children

11

Evolution Strategies

Mutation rate, 2

Mutation of vector of real numbers• Gaussian Convolution with 2

Controlling exploration and exploitation Specifying mutation rate

• Static• Adaptive

• Good starting point -- one-fifth rule

if more than 1/5 children are fitter than parents: increase 2

else: decrease 2

c.f., degeneracy of Evolution Strategy (1 + 1) -- plain Hill-Climbing (1 + ) -- Steepest Ascent Hill-Climbing (1, ) -- Steepest Ascent Hill-Climbing with Replacement

12

Genetic Algorithm

John Holland, 1970s Originally for fixed-length vectors of boolean values, e.g., 001101001

13

Genetic Algorithm

Mutation (Tweaking) Examples

• For binary-valued vectors• Bit-flip mutation

• For permutations• (Adjaceint) pairwise interchange, insertion, inversion

• For tour of TSP• 2-opt or 3-opt

• For tree• For real-valued vectors

Requirements (Talbi, 2009)• Ergodicity• Validity• Locality

2-opt

14

Genetic Algorithm

Crossover Examples

• One-point

• Weakness -- Pr(v1 and vl to be broken) > Pr(v1 and v2 to be broken)

If v1 and vl simultaneously work for high fitness?

• Two-point -- treating v1 and vl fairly

• Uniform -- treating vi and vj fairly

15

Genetic Algorithm

Crossover (cont’d) Convergence by successive crossover of individuals

• Possibly premature

Then, why we use it? Premise of building blocks for high fitness

• e.g., 1011 of 10110101• c.f., schema theory

i.e., linkage between genes for high fitness• One- and two-point crossover

• assuming highly-linked genes are locatednear to one another

Moral• Check your problem. ;-)

16

Genetic Algorithm

More crossover For real-valued vectors

• Line recombination

• Intermediate recombination

17

Genetic Algorithm

More crossover (cont’d) (Talbi, 2009) (convergence?) For permutations

• Order crossover (OX)

• Partially mapped crossover (PMX)

For tour of TSP• Cycle crossover (CX)

1 2 3 4 5 6 7 8 9

8 4 1 5 9 3 6 2 71 9 3 4 5 6 2 7 8

1 2 3 4 5 6 7 8 9

8 4 1 5 9 3 6 2 7

8 4 3 4 5 6 6 2 7

8 5 3 4 5 6 3 2 7

8 9 3 4 5 6 1 2 7

1 2 3 4 5 6 7 8 9

9 3 7 8 2 6 5 1 4

1 2 3 4 5 6 7 8 9

9 3 7 8 2 6 5 1 4

1 3 7 4 2 6 5 8 9

9 2 3 8 5 6 7 1 4

18

Genetic Algorithm

Selection c.f., Evolution Strategy Truncation Selection Roulette Selection (Fitness-Proportionate Selection)

• Scheme

• Let s = i fi , where is the fitness of ith individual.

• Select ith individual with probability (fi / s)

• Selection pressure problem• How about minimization problems?

Just to use fi' = 1/ fi ?

• Or, if difference of fi and fj is too large?

19

Genetic Algorithm

Selection (cont’d) Stochastic Universal Sampling

20

Genetic Algorithm

Selection (cont’d) Tournament Selection

• Primary selection technique used for GA• Recall selection pressure problem of previous methods.

• How to scale fitness appropriately?• A solution -- non-parametric selection algorithm

• Tournament size t• 1 (random search), very large (Truncation Selection with = 1)• 2 (most popular), ...

21

Exploitative Variations

Trend in new algorithms -- more exploitative!

Elitism Similar to ( + )

• Injecting directly fittest individuals (elites) of previous population into the next

Preventing possible premature convergence• Increasing mutation and crossover noise• Weakening selection pressure• Reducing number of elites

Steady-State Genetic Algorithm Updating population, incrementally

• Introducing a few children (possibly one or two)• Give death to the same number of selected individuals

Preventing premature convergence• Selecting individuals randomly for death

+ Approaches in Elitism

c.f., Generation Gap Algorithms -- using large value of new children

22

Exploitative Variations

Tree-Style Genetic Programming Pipeline

Hybrid Evolutionary and Hill-Climbing Algorithms Revising and replacing each individual with some hill-climbing

Adjusting degree of exploitation -- t A.k.a. memetic algorithms

23

Exploitative Variations

Scatter Search with Path Relinking Combining

• exploitative mechanisms (hybrid methods, steady-state evolution)• explicit attempt to force diversity (exploration)

Schema• Initial population

• Seeds provided + random individuals that are very different from one another and from the seeds

• Applying hill-climbing to each seed• Loop

• Selecting some very fit individuals and some very diverse individuals

(much less than population)• Generating new seeds by crossover

Path Relinking, here• Add new ones to population after applying hill-climb

Required -- measure of diversity• Then, how to generate diverse individuals?

24

Exploitative Variations

Scatter Search with Path Relinking (cont’d) Path Relinking

• Original path (solid) and one possible relinked path (dotted)

25

Differential Evolution

Mutation by vector addition and subtraction Working in metric vector spaces (booleans, metric integer spaces, reals)

Generating children For each parent i,

• Selecting randomly individuals, a, b, c• Generating child by crossover of i and a + (b – c)

Survival selection Child is thrown away if it is worse than parent

Adaptive but not much global Determining the size of mutation largely based on

current variance in the population Children are entirely based on the existing parents

26

Particle Swarm Optimization

Idea based on swarming and flocking behaviors in animals maintaining a single static population Tweaking in response to new discoveries about the space

Particle representation Location x, velocity v = x(t) – x(t–1)

• where x(t) is location at time t

c.f., working in multidimensional metric (usually real-valued) spaces

Update of location x, x+, x! -- best of x, best of any of informant of x, best of anyone, so far v v + b(x* – x) + c(x+ – x) + d(x! – x) x x + v c.f., candidate is mutated towards the best discovered solutions so far.

Effect of parameters

27

Summary

Population methods Resampling techniques (EA) and directed mutation (PSO) Generational algorithms and steady-state algorithms

Mutation and crossover Chromosomes in real- and boolean-valued vectors, permutations, tours

Selection Roulette-wheel with fitness scaling and tournament

Adaptive mutation algorithm

Particle Swarm Optimization