evolutionary optimization algorithms
DESCRIPTION
A brief presentation on optimization techniques.TRANSCRIPT
EVOLUTIONARY OPTIMIZATION ALGORITHMSBy
Sumanth (EC8435)
2
by S
um
anth
Kum
ar
OVERVIEW
What is Optimization? How to Optimize? Deterministic Techniques Vs
Evolutionary Techniques Genetic Algorithm Particle Swarm Optimization Ant Colony Optimization Differential Evolution Conclusion ( Most efficient algorithm) References
3/2
6/2
01
2
3
by S
um
anth
Kum
ar
WHAT IS OPTIMIZATION ?
Effective selection of optimal solution. Maximum or Minimum. Cost function.
Applications Signal Processing. Communications. Computer Networks. Economics etc..
3/2
6/2
01
2
4
by S
um
anth
Kum
ar
HOW TO OPTIMIZE ??
Calculus tell us to differentiate and find optimum value.
To differentiate cost function must be known. Numerical methods can be used, if cost
function is not available. Newton’s forward and backward differences
can be used if sample data is known.
3/2
6/2
01
2
5
by S
um
anth
Kum
ar
MAJOR PROBLEM
As complexity of cost function increases.
3/2
6/2
01
2
Rastrigin’s Function
6
by S
um
anth
Kum
ar
PROBLEMS WITH DETERMINISTIC TECHNIQUES
Struck at local minima or maxima. Computationally complex.
3/2
6/2
01
2
local
global
7
by S
um
anth
Kum
ar
PROBABILISTIC OPTIMIZATION TECHNIQUES
Searches N dimensional search space randomly and lands on a approximate solution.
Doesn’t provide ideal optimum value but gives approximate value.
Evolutionary algorithms got their basic ideas from nature.
Examples Survival of the fittest. Motion of birds in search of food. Ants in search of food. Bees etc..
3/2
6/2
01
2
8
by S
um
anth
Kum
ar
DETERMINISTIC VS PROBABILISTIC TECHNIQUES
Results when Rastrigin's function is optimized in both methods.
3/2
6/2
01
2
Compared value
Deterministic Technique
Genetic Algorithm Particle Swarm Optimization
Maximum 60.3002 76.5207 72.042
Minimum 3.1198 0.0095 1.3590
Time (in s) 74.84 0.1307 0.0564
9
by S
um
anth
Kum
ar
GENETIC ALGORITHM (GA)
Proposed by John Holland in 1975. Based on natural selection process Survival
of the fittest. Every possible solution is called
chromosome. Chromosomes are crossed over and mutated
to get better offsprings. Works iteratively. Each iteration is called as Generation. Best chromosomes are passed on to next
generations.
3/2
6/2
01
2
10
by S
um
anth
Kum
ar
GENETIC ALGORITHM (GA) CONTD.. 3
/26
/20
12
Evaluation of Individuals
11
by S
um
anth
Kum
ar
GENETIC ALGORITHM (GA) CONTD..
Chromosome Binary coded - 1 0 1 0 0 1 1 1 0 1 Real encoded - 1.236 2.36 4.23
Crossover Single point Two point
3/2
6/2
01
2
Chromosome 1
1 1 0 1 0 0 1 1 1 0 0
Chromosome 2
0 1 1 0 0 0 1 0 0 1 0
Offspring 1 1 1 0 1 0 0 1 0 0 1 0
Offspring 2 0 1 1 0 0 0 1 1 1 0 0
12
by S
um
anth
Kum
ar
GENETIC ALGORITHM (GA) CONTD..
Mutation
3/2
6/2
01
2
0 5 10 15 20 2568
70
72
74
76
78
80
82
No of iterations
Valu
e o
f th
e c
ost
function
Convergence plot for Rastrigin’s Funciton
GA
chromosome
1 0 1 1 0 0 0 1 1 0
offspring 1 0 1 1 0 1 0 1 1 0
13
by S
um
anth
Kum
ar
PARTICLE SWARM OPTIMIZATION
Inspired by the swarm of birds in search of food.
Considers both self and group knowledge while searching.
Initial population are called as particles. Each particle has a position and velocity
randomly assigned. Lesser computational complexity than GA. Positions and velocities are updated in every
iteration. No crossover and mutation operators.
3/2
6/2
01
2
14
by S
um
anth
Kum
ar
PARTICLE SWARM OPTIMIZATION CONT.. 3
/26
/20
12
15
by S
um
anth
Kum
ar
PARTICLE SWARM OPTIMIZATION CONT..
pbest is particle’s best position in previous generations.
gbest is best position of all particles in a generation.
Velocity update rule
where c1,c2 are learning factors [0 4]
Position update rule
3/2
6/2
01
2
])[][(**2])[][(**1][]1[ ixigbestrandcixipbestrandciViV
]1[][]1[ iVixix
16
by S
um
anth
Kum
ar
PARTICLE SWARM OPTIMIZATION CONT.. 3
/26
/20
12
0 5 10 15 20 2570
72
74
76
78
80
82
No of iterations
Valu
e o
f th
e c
ost
function
Convergence plot for Rastrigin’s Funciton
PSO
17
by S
um
anth
Kum
ar
ANT COLONY OPTIMIZATION
Inspired by behavior of ants in search of food.
Majorly used for finding optimal paths in complex graphs.
Ants lays a trail of pheromone along their path.
3/2
6/2
01
2
18
by S
um
anth
Kum
ar
ANT COLONY OPTIMIZATION CONT..
Consider a Travelling Sales Man (TSP) problem.
N cities and M ants. Condition for transition from a city i to j :
Whether it is already visited or not. Distance between the cities (dij ) . Amount of artificial pheromone.
Path with maximum pheromone is the shortest of all paths.
3/2
6/2
01
2
19
by S
um
anth
Kum
ar
DIFFERENTIAL EVOLUTION (DE)
3/2
6/2
01
2
Initial population candidate solutions within search space called Agents.
Optimizes a problem by iteratively improving the candidate solutions.
Similar to GA , DE uses recombination and mutation operators.
Phases of DE : Initialization Mutation Recombination Selection
20
by S
um
anth
Kum
ar
DIFFERENTIAL EVOLUTION (DE) CONT..
Initialization XiG = [ X1,I,G , X2,i,G , X3,i,G, …. XD,I,G ,] i= 1,2,3 …
N Mutation Vi,G+1 =Xr1,G + F * (Xr2,G - Xr3,G)
where F is differential weight € [0 2] Recombination
Selection Selected agents are passed on to next
generations.
3/2
6/2
01
2
Gij
GijGij X
Vu
,,
1,,1,,
If randji ≤ CR or j = Irand
If randji > CR or j ≠ Irand
21
by S
um
anth
Kum
ar
CONCLUSION
Which Optimization algorithm is best? Which algorithm should be used for my
problem?
NO FREE LUNCH theorem for optimization. Every optimization algorithm is best for one
or other problem. On an average all algorithms efficiency is
almost equal. Simulate and find which algorithm best suites
your problem.
3/2
6/2
01
2
22
by S
um
anth
Kum
ar
REFERENCES Global Optimization Algorithms Theory and Application: Thomas Weise,
Version: 2009-06-26. Marcin Molga, Czesław Smutnicki Test functions for optimization needs :
kwietnia 2005. Yi-Tung Kao, Erwie Zahara. "A hybrid genetic algorithm and particle
swarm optimization for multimodal functions" Science Direct, Applied Soft Computing 8 (2008) 849–857.
Y. Shi, R. Eberhart."Empirical study of particle swarm optimization ".Proc. of Congress on Evolutionary Computation, 1999: 1945-1950.
Tutorial on Particle Swarm Optimization - http://www.swarmintelligence.org/tutorials.php
Hozefa M Botee, Eric Bonabeau: Evolving Ant Colony Optimization: Adv Computer Systems (1998) 1, 149_159.
An Introduction to Differential Evolution: Kelly Fleetwood.
3/2
6/2
01
2
23
by S
um
anth
Kum
ar
3/2
6/2
01
2