Parameter Control for Evolutionary Algorithms,Constrained Problems and Constraint-Handling

Techniques

FIT4012 Advanced topics in computational science

September 1, 2014

Parameter control for Evolutionary Algorithms

I Parameters of Evolutionary Algorithms

I Parameter Control

I Adaptive Parameter Control

I Adaptive Range Parameter Control

Parameters of Evolutionary Algorithms

I Solution representation: the search-spaceI Strategy parameters: how to search

I Crossover operator: uniform, single-point, etc.I Crossover rate: [0.6, 1.0]I Mutation operator: uniform, single-point, etc.I Mutation rate: [0.001, 0.5]I Population size: [1, 200] etc.

Problem

I Setting algorithm parameters is computationally expensive.

I Different parameter values are required for different stages ofthe optimisation process.

Parameter Control

1. DeterministicI mr0 = 0.5, mrt = mrt−1 − 0.01, t = {10, 20, ..., n}

2. Self-adaptive

I 1 1 0 1 0 1 0 1 0 0 0.3

3. AdaptiveI Feedback mechanism: pt(υij), qt(υij)

Adaptive Parameter Control

A genetic algorithm

Stopping criterionTrue

False

Evolve solution(s)

Evaluate solution(s) Final solutionsInitial population

A genetic algorithm with adaptive parameter control

Stopping criterionTrue

False

Quality attribution

Effect assessment

Feedback collection

Selection Evolve solution(s)

Evaluate solution(s) Final solutionsInitial population

State-of-the-art Adaptive Parameter Control Strategies

I Probability Matching (PM),

pt(υij) = pmin + (1− n ∗ pmin)qt(υij)∑ns=1 qt(υis)

I Adaptive Pursuit (AP),

pt(υij) =

pt−1(υij) + β(pmax − pt−1(υij)) if j = j∗pt−1(υij) + β(pmin − pt−1(υij)) otherwise

I Dynamic Multi-Armed Bandits (DMAB).

Issues

I The choice of parameter assignment is made based on staticpredefined ranges.

I E.g. crossover rate: [0.6, 0.8], [0.8, 1.0]

I Defining narrow ranges leads to more accuracy but increasedcombinatorial complexity.

I Wider ranges entail a sampling inaccuracy.

Research question and approach

I What is an effective method for configuring real-valuedparameters during the optimisation process?

I Ideally, the ranges should be optimised by the parametercontrol process.

Research question and approach

I What is an effective method for configuring real-valuedparameters during the optimisation process?

I Ideally, the ranges should be optimised by the parametercontrol process.

Adaptive Range Parameter Selection

υi2υi1 p(e+|υi2)p(e+|υi1) { }−−−−−−−︸ ︷︷ ︸[υi1(min),υi1(max)]

−−−−−−−︸ ︷︷ ︸[υi2(min),υi2(max)]

υi2υi1

−−−−−−−︸ ︷︷ ︸[υi1(min),υi1(max)]

−−−−−−−︸ ︷︷ ︸[υi2(min),υi2(max)]

} p(e+|υi2){p(e+|υi1)

υi2υi11

−−−−︸ ︷︷ ︸[υi11 (min),υi11 (max)]

−−−−−−−︸ ︷︷ ︸[υi2(min),υi2(max)]

} p(e+|υi2){p(e+|υi11) υi12[υi12 (min),υi12 (max)]︷ ︸︸ ︷

−−−−

p(e+|υi12)

Adaptive Range Parameter Control

υi2υi11

−−−−︸ ︷︷ ︸[υi11 (min),υi11 (max)]

−−−−−−−︸ ︷︷ ︸[υi2(min),υi2(max)]

} p(e+|υi2){p(e+|υi11) υi,12[υi12 (min),υi12 (max)]︷ ︸︸ ︷

−−−

p(e+|υi12)

}

υ′i2υi1

−−−−︸ ︷︷ ︸[υi1(min),υi1(max)]

−−−−−−−−−−︸ ︷︷ ︸[υ′

i2(min),υ′

i2(max)]

}p(e+|υ′i2){p(e+|υi1)

Experimental Design

I Controlled parameters:I Mutation rate,I Crossover rate.

I Benchmark problems:I Royal Road,I Quadratic Assignment Problem (QAP),I Multiobjective Quadratic Assignment Problem (mQAP),I Component Deployment Problem (Scheduling length and

communication overhead).

I Benchmark adaptive parameter control methods:I Probability Matching (PM),I Adaptive Pursuit (AP),I Dynamic Multi-Armed Bandits (DMAB): statistical tests to

restart parameter rewards

I 30 trials.

Results

Royal Road QAP

0.486

0.4865

0.487

0.4875

0.488

0.4885

0.489

PM AP DMAB ARPS

Nor

mal

ised

fitn

ess

0.2655

0.266

0.2665

0.267

0.2675

0.268

PM AP DMAB ARPS

Nor

mal

ised

fitn

ess

mQAP Component deployment

0.84

0.86

0.88

0.9

0.92

0.94

0.96

PM AP DMAB ARPS

Hyp

ervo

lum

e

0.9

0.91

0.92

0.93

0.94

0.95

0.96

0.97

0.98

PM AP DMAB ARPS

Hyp

ervo

lum

e

Kolmogorov-Smirnov test

Royal Road QAP mQAP

d p d p d p

ARPS vs. DMAB 0.3414 0.000 0.5172 0.000 0.5172 0.000ARPS vs. AP 0.3793 0.000 0.5172 0.000 0.6207 0.000ARPS vs. PM 0.4759 0.000 0.5517 0.000 0.5172 0.000

Parameter ranges

Crossover rate

0.6

0.65

0.7

0.75

0.8

0.85

0.9

0.95

1

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

Cros

sove

r ra

te

Iterations

0.6

0.65

0.7

0.75

0.8

0.85

0.9

0.95

1

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

Cros

sove

r ra

te

Iterations

Mutation Rate

0

0.1

0.2

0.3

0.4

0.5

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

Mut

atio

n ra

te

Iterations

0

0.1

0.2

0.3

0.4

0.5

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

Mut

atio

n ra

te

Iterations

Predictive parameter control

The past effectiveness of parameter values provides the informationnecessary to project their chance of leading to good results atiteration t + 1

0 10 20 30 40 50

0.45

0.50

0.55

0.60

0.65

Iterations

Suc

cess

rate

PPC

I The effect of a parameter value is calculated as:

et(vij) =ns(υij)

n(υij)(1)

I The effect of a parameter value over time is the time series

e1(υij), e2(υij), ..., et(υij). (2)

I A predictive model in adaptive parameter control has thegeneral form:

qt(υij) = f (e1(υij), e2(υij), ..., et(υij), �) (3)

Predictive models

I Linear regression

qt(υij)− et(υij)σ(et(υij))

= r · t − tσ(t)

(4)

I Simple moving average

qt(υij) =et−1(υij) + ...+ et−k(υij)

k(5)

I Exponentially weighted moving average

qt(υij) =t∑

k=0

α(1− α)ket−(k+1)(υij) (6)

I Autoregressive Integrated Moving Average

qt(υij) = φ1et−1(υij) + �t (7)

Results

Assumptions of the forecasting models

I Linearity: can you fit a linear model?

I Normality: are the errors normally distributed?

I Independence: are the error terms independent?

I Homoscedasticity: is the variance in errors similar over time?

I Stationarity: does the mean and standard deviation remainthe same over time?

Linearity Normality Independence Homoscedasticity Stationarity

LR + + + + -

SMA - - - + +

EWMA - - - + +

ARIMA - + + + +

Statistical tests

I Linearity: fit a linear model and report the p − valueI Normality: Kolmogorov-Smirnov (KS) test

I Independence: Durbin-Watson (DW) statistical test

I Homoscedasticity: Breusch-Pagan test

I Stationarity: Kwiatkowski-Phillips-Schmidt-Shin (KPSS) test

Ranges/values of parameters analysed

Parameter Ranges/values

Mutation rate - 2 ranges [0.001,0.249], [0.25,0.49]

Mutation rate - 4 ranges [0.001,0.1249], [0.125,0.249], [0.25,0.3749], [0.375,0.49]

Crossover rate - 2 ranges [0.6,0.79], [0.8,0.99]

Crossover rate - 4 ranges [0.6,0.69], [0.7,0.79], [0.8,0.89], [0.9,0.99]

Population size - 2 ranges [20,60], [61,100]

Population size - 4 ranges [20,40], [41,60], [61,80], [81,100]

Mating pool size - 2 ranges [0.1,0.39], [0.4,0.69]

Mating pool size - 4 ranges [0.1,0.249], [0.25,0.39], [0.4,0.549], [0.55,0.69]

Mutation operator Single-point, Uniform

Crossover operator Single-point, Uniform

Results

Characteristics of the effects of the discrete parameter values. Thepercentages represent the percentage of times (of a total 30) H0was not rejected.

Parameter Problem Linear Normal Ind.t Hom. Stat.Single-point QAP 100% 100% 70% 93% 97%mutation MQAP 100% 97% 70% 83% 83%

Royal Road 90% 90% 73% 87% 87%Uniform QAP 100% 100% 77% 93% 93%mutation MQAP 100% 97% 80% 83% 90%

Royal Road 97% 87% 73% 87% 90%Single-point QAP 100% 100% 77% 87% 93%crossover MQAP 100% 97% 80% 83% 90%

Royal Road 97% 93% 73% 87% 87%Uniform QAP 100% 100% 77% 97% 93%crossover MQAP 100% 97% 77% 83% 93%

Royal Road 100% 90% 73% 87% 83%

Recommendations on the suitability of forecasting methods

Parameter LR SMA EWMA ARIMA

Mutation rate + + + +

Crossover rate + + + +

Population size ∼ ∼ ∼ ∼Mating pool size + + + +

Mutation operator + + + +

Crossover operator + + + +

Performance comparison

Average Quality Attribution (AQA), Extreme Quality Attribution(EQA) and Predictive Quality Attribution (PQA)Royal Road QAP

0.236

0.238

0.24

0.242

0.244

0.246

0.248

AQA EQA PQA

Nor

mal

ised

fitn

ess

0.804

0.8045

0.805

0.8055

0.806

0.8065

0.807

0.8075

0.808

AQA EQA PQA

Hyp

ervo

lum

emQAP Component deployment

0.19

0.195

0.2

0.205

AQA EQA PQA

Hyp

ervo

lum

e

0.78

0.79

0.8

0.81

0.82

0.83

0.84

0.85

AQA EQA PQA

Nor

mal

ised

fitn

ess

Means and standard deviations

The means and standard deviations over 30 runs of AverageQuality Attribution (AQA), Extreme Quality Attribution (EQA)and Predictive Quality Attribution (PQA).

Mean Standard Deviation

Problem AQA EQA PQA AQA EQA PQA

CHR20A 0.4909 0.4845 0.4972 1.627E-02 1.306E-02 1.920E-02

BUR26E 0.2416 0.2395 0.2422 2.049E-03 1.893E-03 2.365E-03

TAI30A 0.4909 0.4845 0.4982 1.627E-02 1.306E-02 1.870E-02

STE36B 0.7978 0.7984 0.8308 7.823E-03 9.038E-03 8.704E-03

Royal Road 0.0064 0.0063 0.0072 1.537E-04 6.3723E-03 1.473E-03

KC10-2fl-5rl 0.8062 0.8061 0.8067 1.040E-03 8.284E-04 9.555E-04

KC30-3fl-1rl 0.1984 0.1961 0.1995 3.255E-03 3.974E-03 3.806E-03

KC30-3fl-2rl 0.5199 0.5175 0.5223 7.312E-03 5.439E-03 4.821E-03

Statistical analysis

The Kolmogorov-Smirnov test values for the 30 runs of AverageQuality Attribution (AQA), Extreme Quality Attribution (EQA)and Predictive Quality Attribution (PQA)

PQA vs. AQA PQA vs. EQA

Problem d p d p

CHR20A 0.2333 0.042 0.3333 0.045

BUR26E 0.3000 0.048 0.3000 0.049

TAI30A 0.3294 0.046 0.3517 0.039

STE36B 0.9667 0.000 0.9667 0.000

Royal Road 0.6333 0.000 0.7333 0.000

KC10-2fl-5rl 0.3333 0.045 0.4000 0.011

KC30-3fl-1rl 0.2897 0.016 0.4333 0.005

KC30-3fl-2rl 0.3433 0.035 0.4333 0.005

Future research

Search Space

Global optimum

Local optima

Variable 1 Variable 2

FunctionvalueCharacterisation

Modality

Uniformity

Diversity

Learning

Bayesian Networks

Forecasting

Control Theory

Adaptation

Local search

EA

ACO

Constrained Problems

The component deployment problem

Example: Reliability Optimisation in Embedded Systems

ECU3

ECU1

ECU2

ECU5

ECU6

ECU7

ECU8Bus0 (CAN)

Bus2(Rear LIN)

ECU0

Bus1(Front LIN)

ECU4

EmergencyStopDetector

1

HMIOutputs

12DistanceCalc

13

14SpeedCalc

ObjectRecogn-ition

10

SpeedLimitter

8

ModeSwitch

9ABSMainUnit

0

LoadCompen-sator

3

5WSR-F

6WAC-R

7WAC-F

BrakePedalSensor

2

4WSR-R

ACCMainUnit

11

WAC : Wheel Actuator Controllers (Front and Rear) WSR : Wheel Sensor Readers (Front and Rear)

Reliability function

R ≈∏

i∈IRvc (ci )i ·

∏

i ,j∈IRvl (lij )ij (8)

I Reliability of a component: Ri = e−fr(d(ci ))·

wl(ci )

ps(d(ci ))

I Reliability of a link: Rij = e−fr(d(ci ),d(cj ))·

ds(ci ,cj )

dr(d(ci ),d(cj ))

I Expected number of visits for a component:vc(ci ) = q0(ci ) +

∑j∈I(vc(cj) · p(cj , ci ))

I Expected number of visits for a link:vl(lij) = 0 +

∑x∈{i}(vc(cx) · p(cx , lij))

Constraints

I Memory

mem(d) = ∀h ∈ H :∑

Ch∈d−1(h)

mc(Ch) ≤ mh(h) (9)

I Colocation constraints

coloc(d) = ∀c ∈ C : (h ∈ cr(ci , cj) ⇒ d(ci ) 6= d(cj) (10)

I Communication

com(d) = ∀c ∈ C : (h ∈ cm(ci , cj) ⇒ ld(ci ),d(cj ) ≥ 0) (11)

Constraint optimisation problem formulation

maximise R ≈ ∏i∈I Rvc (ci )i ·

∏i ,j∈I

subject to ∀h ∈ H : ∑Ch∈d−1(h)

mc(Ch) ≤ mh(h)

∀c ∈ C : (h ∈ cr(ci , cj) ⇒ d(ci ) 6= d(cj)

∀c ∈ C : (h ∈ cm(ci , cj) ⇒ ld(ci ),d(cj ) ≥ 0)

Constraint handling

I Eliminating infeasible candidates

I Penalizing functions

I Repairing infeasible candidates

Eliminating infeasible candidates: death penalty

Advantages

I The algorithm does not spend a significant amount of timeevaluating illegal individuals.

Disadvantages

I For some problems the probability of generating a feasiblesolution is relatively small

I Infeasible regions may serve as a bridge between feasibleregions.

I In this approach non-feasible solutions do not contribute tothe gene-pool of any population

Penalising functions

Generating potential solutions without considering the constraintsand then to penalize them by decreasing the ‘goodness’ of theevaluation function.

I assign a constant as a penalty measure

I assign a penalty measure depend on the degree of violation:the larger violation is, the greater penalty is imposed

I the growth of the penalty can be logarithmic, linear,quadratic, exponential, etc.

Example:

R ≈∏

i∈IRvc (ci )i ·

∏

i ,j∈IRvl (lij )ij − w (12)

Repair algorithms

‘Correct’ any infeasible solutions so generated.

Disadvantages

I Such repair algorithms might be computationally intensive torun and the resulting algorithm must be tailored to theparticular application.

I Moreover, for some problems the process of correcting asolution may be as difficult as solving the original problem.

Example:

I Change the allocation of a software component to a differenthardware unit.

I Swap the allocations of two software components.

Are constrained problems more difficult than unconstrainedproblems?

10% 25% 50% 75% 100%

5.0e-08

1.5e-07

Interactions

Pre

dict

or e

rror

Constrained problems can be easy

10% 25% 50% 75% 100%

0.999980

0.999990

Interactions

Fitness

The end