umdas for dynamic optimization problems
DESCRIPTION
Genetic and Evolutionary Computation Congress 2008. Atlanta, USA.TRANSCRIPT
GECCO08 - AtlantaFernandes, Lima and Rosa – “UMDAs for DOPs”
UMDAs for Dynamic Optimization Problems
Carlos Fernandes1,2
Claudio Lima3
Agostinho C. Rosa1
1LaSEEB-ISR-IST, Technical Univ. of Lisbon (IST), Portugal2 Department of Architecture and Computer Technology, University of Granada, Spain3 Informatics Laboratory, University of Algarve, Portugal
GECCO08 - AtlantaFernandes, Lima and Rosa – “UMDAs for DOPs”
Motivation and Objectives
•Design an update strategy for UMDAs to deal with Dynamic Optimization Problems (DOPs), based on ACO.
Problem: full convergence.When solving DOPs, finding the optima is not the main task; the algorithm must track the optima.Full convergence (without ways to escape it) is not suitable for DOPs (even if the population converges to the global optimum)
Solution: delay or avoid full convergence.Change probability distribution.Maintain probability distribution except when close to 0 and 1
Combination of strategies
GECCO08 - AtlantaFernandes, Lima and Rosa – “UMDAs for DOPs”
Motivation and Objectives
•Proposal: Update strategy for UMDA based on ACO equations.
•Effects: Change probability distribution.
•Why? Reinforcement/evaporation equations allow us to control the convergence speed of the algorithm
ACO algorithms build solutions by travelling trough the nodes of a (combinatorial) problem. (TSP, for instance…)
After the evaluation of the solutions, edges that belong to good solutions are reinforced (pheromone)
Each generation, pheromone in all edges is evaporated – diversity, mutation.
In the following generation, solutions/paths are chosen according to the amount of pheromone in each connection between nodes.
GECCO08 - AtlantaFernandes, Lima and Rosa – “UMDAs for DOPs”
Motivation and Objectives
ACO Univariate EDA
initialize pheromone in all edgesrepeat sample N solutions evaluate solutions update pheromoneuntil stop criterion
initialize probability modelrepeat sample N solutions evaluate solutions update model parametersuntil stop criterion
GECCO08 - AtlantaFernandes, Lima and Rosa – “UMDAs for DOPs”
Motivation and Objectives
The Binary Ant Algorithm (presented in GECCO07). Based on Ant Colony Optimization (ACO), BAA builds pheromone trails between binary variables.
•BAA is a kind of Estimation of Distribution Algorithm (EDA)
There are similarities between EDAs and ACO. The pheromone trails are similar to the probability models, and reinforcement/evaporation is ACO’s update strategy.
0 0 0
1 1 1
0
1
0
1
Solution Dimension
BAA
GECCO08 - AtlantaFernandes, Lima and Rosa – “UMDAs for DOPs”
Reducing Diversity LossBranke, J., Lode, C., and Shapiro, J. 2007. Adressing sampling errors and diversity loss in UMDA. Proceedings of the 2007 Genetic and Evolutionary Computation Conference, ACM, 508-515.
•Permutation Sampling: reduces loss due to sampling; used in all the experiments
•Loss Correction (LC)
•Laplace Correction
•Iterated Laplace Correction (iLaplace)
•Boundary Correction: changes probability distribution near 0 and 1
Changes probability distribution
GECCO08 - AtlantaFernandes, Lima and Rosa – “UMDAs for DOPs”
Reducing Diversity Loss
GECCO08 - AtlantaFernandes, Lima and Rosa – “UMDAs for DOPs”
UMDASet γi ← 1/2 for all i = 1 . . .L;repeat Sample N strings according to make a population D. Generate a new population Ds from D by selecting the f×N fittest strings. for i = 1 to L do update model:
end foruntil stop criterion
Replace by ACO-like equations
GECCO08 - AtlantaFernandes, Lima and Rosa – “UMDAs for DOPs”
Reinforcement/Evaporation (RE) Update
Set γi ← 1/2 for all i = 1 . . .L; Set ← 0 for all i = 1 . . .L; Set α and βrepeat sample N strings according to make a population D. generate new population Ds by selecting the f×N fittest strings. update pheromone
evaporate for i = 1 to L do update model end foruntil stop criterion met
β = 1
+
+
If α = 1 and β = 1, we have the standard update strategy
1 0 1 1…
GECCO08 - AtlantaFernandes, Lima and Rosa – “UMDAs for DOPs”
Results – Diversity Loss in Flat Landscape
β = 1; N = 20; f = 0.5; L = 100 α = 1; N = 20; f = 0.5; L = 100
GECCO08 - AtlantaFernandes, Lima and Rosa – “UMDAs for DOPs”
Test Set Severity of change: This criterion establishes how strongly the problem is changing
Speed of change: This criterion establishes how often the environment changes
Yang and Yao’s dynamic problems generator*
•By using a binary mask, dynamic environments are created by applying the mask to each solution before its evaluation.
•Severity of change is controlled by setting the number of 1’s in the mask.
•Speed of change is controlled by defining the number of generations between the application of a different mask.
*Yang, S. and Yao, X. 2005. Experimental study on PBIL algorithms for dynamic optimization problems. Soft Computing 9(11), 815-834.
GECCO08 - AtlantaFernandes, Lima and Rosa – “UMDAs for DOPs”
Test Set
•Functions
•Onemax
•Royal Road
Speed was set to = 10, 100 (generations)𝜏Severity was set to ρ = 0.05, 0.6 and 0.95
6 different scenarios
Performance is measured by the mean best-of-generation values, i.e., best fitness averaged over all generations, and then over all runs
30 runs for each configuration
Compared RE with Loss, Laplace, iterated Laplace and Boundary Correction
GECCO08 - AtlantaFernandes, Lima and Rosa – “UMDAs for DOPs”
Results – RE Parameters
β = 1 β = 0.5
Royal Road
•Decreasing β improves performance when speed is high and severity is low
•β = 1 and α = 1 is standard UMDA’s update strategy
GECCO08 - AtlantaFernandes, Lima and Rosa – “UMDAs for DOPs”
Results
Onemax Royal Road
τρ
100.05
100.6
100.95
1000.05
1000.6
1000.95
100.05
100.6
100.95
1000.05
1000.6
1000.95
1 2
RE1 iLap − + + − − − − + ~ ~ ~ ~
RE1 LC + + + − − − + + + + + +
RE2 iLap − + + ~ ~ + + + + + ~ ~
RE2 LC ~ + + ~ ~ + + + + + + +
iLap LC + + + ~ ~ ~ + + + + + +
Comparing strategies that do not avoid full convergence
RE1: α = 0.6, β = 1RE2: α = 1, β = 0.5
GECCO08 - AtlantaFernandes, Lima and Rosa – “UMDAs for DOPs”
Results
Table 2. Statistical analysis of the results in table 1.
Onemax Royal Road
τρ
100.05
100.6
100.95
1000.05
1000.6
1000.95
100.05
100.6
100.95
1000.05
1000.6
1000.95
1 2
RE1+BC iLap+BC ~ + + ~ + + + + + + + +
RE1+BC Laplace − + + − + + + + ~ + + +
RE1+BC LC+BC − + + ~ + + + + ~ + + +
RE1+BC BC − + + ~ + + + + + + + +
RE2+BC iLap+BC − + + − − − + + ~ + + +
RE2+BC Laplace − + + − − − ~ + ~ + + +
RE2+BC LC+BC − + + − − − ~ ~ ~ ~ − ~
RE2+BC BC − + + − − − + + ~ + + +
Avoiding full convergence
RE1: α = 0.8, β = 1RE2: α = 0.9, β = 0.5
*Laplace performs well when compared to iL+BC, LC+BC and BC
GECCO08 - AtlantaFernandes, Lima and Rosa – “UMDAs for DOPs”
Results
Dynamic Royal Road with = 100. 𝜏 β = 1 (α = 1 curves correspond to the standard UMDA update strategy)
ρ = 0.05 ρ = 0.6 ρ = 0.9
RE
GECCO08 - AtlantaFernandes, Lima and Rosa – “UMDAs for DOPs”
Results
Table 2. Statistical analysis of the results in table 1.
Dynamic Royal Road with = 100. 𝜏 α = 0.8, β = 1
RE Laplace
GECCO08 - AtlantaFernandes, Lima and Rosa – “UMDAs for DOPs”
ConclusionsRE is capable of outperforming other diversity loss correction techniques
•RE performs well when compared to Loss Correction and Iterated Laplace Correction
•RE with boundary correction outperforms other strategies in a wide range of scenarios
Laplace Correction attains better results than other techniques (except RE)
Diversity of the UMDA with RE may be controlled by α and β parameters
RE seems to work well with α between 0.6 and 0.9, depending on β and depending if we hybridize it with Boundary Correction.
GECCO08 - AtlantaFernandes, Lima and Rosa – “UMDAs for DOPs”
Conclusions
RE strategy works well without needing to know when the fitness changes. There DOPs with changes that are not detectable (or that are too costly to detect).
GECCO08 - AtlantaFernandes, Lima and Rosa – “UMDAs for DOPs”
Future work
In-depth study of the effects of parameter values on performance.
Understand how α and β affect UMDA’s behavior.
Tests on dynamic trap functions.
Extend the strategy to other EDAs.