advanced algorithms and applicationslabsticc.univ-brest.fr/~lemarch/eng/cours/algop2avaneng.pdf ·...
TRANSCRIPT
![Page 1: Advanced algorithms and applicationslabsticc.univ-brest.fr/~lemarch/ENG/Cours/algoP2avanENG.pdf · Sometimes only available method (e.g program optimization) Or exact methods for](https://reader036.vdocuments.net/reader036/viewer/2022070918/5fb8247024aa510ccf007c51/html5/thumbnails/1.jpg)
1
Advanced algorithms and applications
Laurent LemarchandLaurent LemarchandLISyC/UBOLISyC/UBO
[email protected]@univ-brest.fr
![Page 2: Advanced algorithms and applicationslabsticc.univ-brest.fr/~lemarch/ENG/Cours/algoP2avanENG.pdf · Sometimes only available method (e.g program optimization) Or exact methods for](https://reader036.vdocuments.net/reader036/viewer/2022070918/5fb8247024aa510ccf007c51/html5/thumbnails/2.jpg)
2
Dynamic Programmingprinciple (Bellman)
Decompose problem P into sub-problemsP
1, P
2, ..., P
n
Solve the sub problems to find the solutions v
1, v
2, ..., v
n
Combine these solutions to solve P :V* = f(v
1, v
2, ..., v
n)
Recursive mechanism
![Page 3: Advanced algorithms and applicationslabsticc.univ-brest.fr/~lemarch/ENG/Cours/algoP2avanENG.pdf · Sometimes only available method (e.g program optimization) Or exact methods for](https://reader036.vdocuments.net/reader036/viewer/2022070918/5fb8247024aa510ccf007c51/html5/thumbnails/3.jpg)
3
Dynamic programmingusage conditions
Optimality principle
Example : shortest paths : (A,B) is optimal iif (A,C) and (B,C) are also optimal.
A solution is optimal iff its subproblemsare also optimal
A B
C
![Page 4: Advanced algorithms and applicationslabsticc.univ-brest.fr/~lemarch/ENG/Cours/algoP2avanENG.pdf · Sometimes only available method (e.g program optimization) Or exact methods for](https://reader036.vdocuments.net/reader036/viewer/2022070918/5fb8247024aa510ccf007c51/html5/thumbnails/4.jpg)
4
Optimality principlecounterexample
Search of the shortest path of a given depth In this case you need to evaluate all intermediate
solutions
depth 1 depth 2
![Page 5: Advanced algorithms and applicationslabsticc.univ-brest.fr/~lemarch/ENG/Cours/algoP2avanENG.pdf · Sometimes only available method (e.g program optimization) Or exact methods for](https://reader036.vdocuments.net/reader036/viewer/2022070918/5fb8247024aa510ccf007c51/html5/thumbnails/5.jpg)
5
Dynamic programmingShortest path : Ford-Bellman
a g
f
e
d
c
b
Path finding : shortest path from a to one of its successors
Recursively : (s) = min s' preds(s)
(s') + ws' s
Other examples : Chortle-crf, resource allocation, inventory management ...
3
2
32
2
-2-11
245
2
![Page 6: Advanced algorithms and applicationslabsticc.univ-brest.fr/~lemarch/ENG/Cours/algoP2avanENG.pdf · Sometimes only available method (e.g program optimization) Or exact methods for](https://reader036.vdocuments.net/reader036/viewer/2022070918/5fb8247024aa510ccf007c51/html5/thumbnails/6.jpg)
6
Dynamic programmingEdition distance : Levenshtein metric
String metric for computing differences between 2 strings Number of edition operations to do for rewriting string A into B
Insertion: poe > pome Deletion: pome > ome Substitution: ome > ame
diff and agrep commands
poe to ame distanceis 3 – minimal ?
![Page 7: Advanced algorithms and applicationslabsticc.univ-brest.fr/~lemarch/ENG/Cours/algoP2avanENG.pdf · Sometimes only available method (e.g program optimization) Or exact methods for](https://reader036.vdocuments.net/reader036/viewer/2022070918/5fb8247024aa510ccf007c51/html5/thumbnails/7.jpg)
7
Dynamic programmingLevenshtein algorithm
algo Levenshtein
for i = 1 to n1 for j = 1 to n2 if str1[i-1] == str2[j-1] then cost = 0 else cost = 1 endif
d[i, j] := minimum( d[i-1, j ] + 1, // delete d[i, j-1] + 1, // insert d[i-1, j-1] + cost // substitute )
endfor endfor return d[n1, n2]end Levenshtein
input string str1[1..n1]string str2[1..n2]table d[n1+1,n2+1]
init d[0,0] = 0// if one string is null// dist = other's length i [1.. n1] d[i,0] = i j [1.. n2] d[0,j] = j
![Page 8: Advanced algorithms and applicationslabsticc.univ-brest.fr/~lemarch/ENG/Cours/algoP2avanENG.pdf · Sometimes only available method (e.g program optimization) Or exact methods for](https://reader036.vdocuments.net/reader036/viewer/2022070918/5fb8247024aa510ccf007c51/html5/thumbnails/8.jpg)
8
Dynamic programmingLevenshtein algorithm example
Firststep
…...............
Example : wikipedia
dist(NICHE to CHIEN) = 5
Computing d[i,j] Left + 1: d[i-1, j] + 1. (insert) Above +1: d[i, j-1] + 1 (delete) Diagonal : d[i-1,j-1]+ (0 or 1) substitution (if needed)
![Page 9: Advanced algorithms and applicationslabsticc.univ-brest.fr/~lemarch/ENG/Cours/algoP2avanENG.pdf · Sometimes only available method (e.g program optimization) Or exact methods for](https://reader036.vdocuments.net/reader036/viewer/2022070918/5fb8247024aa510ccf007c51/html5/thumbnails/9.jpg)
9
Branch and Bound methodsprinciple
All solutions implicit enumeration Search sub spaces S
1, S
2, ..., S
n
Quality of each sub space v(S1), v(S
2), ..., v(S
n)
S2S3
S1
S1 S2 S3
v S1
v S2
v S3
f s
![Page 10: Advanced algorithms and applicationslabsticc.univ-brest.fr/~lemarch/ENG/Cours/algoP2avanENG.pdf · Sometimes only available method (e.g program optimization) Or exact methods for](https://reader036.vdocuments.net/reader036/viewer/2022070918/5fb8247024aa510ccf007c51/html5/thumbnails/10.jpg)
10
Branch and Bound methodsprinciple
Let f(s) to be maximized An ever known feasible solution a
1, with v(a
1) = 12
v(S3) = 11 implies that we don't need to explore S
3
v(Si) is an upper bound on the quality of the feasible
solutions contained in Si
v S1
v S2
v S3
f s
a1
![Page 11: Advanced algorithms and applicationslabsticc.univ-brest.fr/~lemarch/ENG/Cours/algoP2avanENG.pdf · Sometimes only available method (e.g program optimization) Or exact methods for](https://reader036.vdocuments.net/reader036/viewer/2022070918/5fb8247024aa510ccf007c51/html5/thumbnails/11.jpg)
11
Branch and Bound methodsB&B algorithm
We also have an objective function, f(s) (max)
A partitionning function that splits a search space S into sub spaces :
S1, S
2, ..., S
n : S = U
n S
i
A quality evaluation function thats gives for a space an upper bound on the quality of its feasible solutions :
f : s S f(s*) v(S)
A search strategy to choose which is a next suub space Si to be
explorated at each step.
branch
bound
![Page 12: Advanced algorithms and applicationslabsticc.univ-brest.fr/~lemarch/ENG/Cours/algoP2avanENG.pdf · Sometimes only available method (e.g program optimization) Or exact methods for](https://reader036.vdocuments.net/reader036/viewer/2022070918/5fb8247024aa510ccf007c51/html5/thumbnails/12.jpg)
12
Branch and Bound methodssearch tree
The processing of the algorithm corresponds to the traversal of a search tree :
Each node represents a sub space
Leaves are feasible solutions or empty (without any feasible solution) sub spaces
Children S1, S
2, ..., S
n de S result of S
partitioning
![Page 13: Advanced algorithms and applicationslabsticc.univ-brest.fr/~lemarch/ENG/Cours/algoP2avanENG.pdf · Sometimes only available method (e.g program optimization) Or exact methods for](https://reader036.vdocuments.net/reader036/viewer/2022070918/5fb8247024aa510ccf007c51/html5/thumbnails/13.jpg)
13
Branch and Bound methodsalgorithm
Space S, Evaluation v(S)(min),split() next()node_list L = {S}Initial bound U = Best solution
Sbest
=
Algo B&Bwhile L do
S = next(L)S
1, S
2, ..., S
n= split(S)
foreach Si do
if v(Si) > U or S
i unfeasible
discard Si
else if Si feasible
Sbest
= Si
U = v(Si) (= f(S
i))
else L = L U {S
i}
end
![Page 14: Advanced algorithms and applicationslabsticc.univ-brest.fr/~lemarch/ENG/Cours/algoP2avanENG.pdf · Sometimes only available method (e.g program optimization) Or exact methods for](https://reader036.vdocuments.net/reader036/viewer/2022070918/5fb8247024aa510ccf007c51/html5/thumbnails/14.jpg)
14
Cuts within the search tree if v(Si) > U (if min)
Implicit enumeration
Branch and Bound methodscuts
1
2 5
43
16
18
= 25(U=25)
= 19(U=19)
20
Implicitlyenumerated
sub tree
![Page 15: Advanced algorithms and applicationslabsticc.univ-brest.fr/~lemarch/ENG/Cours/algoP2avanENG.pdf · Sometimes only available method (e.g program optimization) Or exact methods for](https://reader036.vdocuments.net/reader036/viewer/2022070918/5fb8247024aa510ccf007c51/html5/thumbnails/15.jpg)
15
Parallel algorithm Master/slave
// evaluation of nodes Centralized and distributed versions
Speedup results Positive or negative anomalies depending on the evaluation function and the search strategy
Bound update problem
Branch and Bound methodsparallelism
Slave 1
Slave 2
Slave 3
master
![Page 16: Advanced algorithms and applicationslabsticc.univ-brest.fr/~lemarch/ENG/Cours/algoP2avanENG.pdf · Sometimes only available method (e.g program optimization) Or exact methods for](https://reader036.vdocuments.net/reader036/viewer/2022070918/5fb8247024aa510ccf007c51/html5/thumbnails/16.jpg)
16
Example, centralized implemantation 1 master, 2 slaves, comms 1 time unit // ... Search tree traversal DFSsequential runtime of 7 time units
Branch and Bound methodsparallelism
1
2 7
63
16
18
22 = 19(U=19)
= 17(U=17)
18
54 = 23(U=23)
24
![Page 17: Advanced algorithms and applicationslabsticc.univ-brest.fr/~lemarch/ENG/Cours/algoP2avanENG.pdf · Sometimes only available method (e.g program optimization) Or exact methods for](https://reader036.vdocuments.net/reader036/viewer/2022070918/5fb8247024aa510ccf007c51/html5/thumbnails/17.jpg)
17
Branch and Bound methodsparallelism
1
2 2
16
18
= 17(U=17)
Speedup issuperlinear !
1
2 7
63
16
18
22 = 19(U=19)
= 17(U=17)
18
54 = 23(U=23)
24
+1 comm
Example, centralized implemantation 1 master, 2 slaves, comms 1 time unit // ... Search tree traversal DFSsequential runtime of 7 time units2 time units with 2 // procs
![Page 18: Advanced algorithms and applicationslabsticc.univ-brest.fr/~lemarch/ENG/Cours/algoP2avanENG.pdf · Sometimes only available method (e.g program optimization) Or exact methods for](https://reader036.vdocuments.net/reader036/viewer/2022070918/5fb8247024aa510ccf007c51/html5/thumbnails/18.jpg)
18
Branch and Bound methodsInteger programming example
Simplex method solves problems in ℝ How to solve it in ℕ ?
C1
C2 C3
S
x1
x2 x1+3x2⩽15−2x1−x2⩽−12−3x1−11x2⩽−66min 2x1+4x2
x1,x2∈ℝ
x1,x2 ∈ℕ
![Page 19: Advanced algorithms and applicationslabsticc.univ-brest.fr/~lemarch/ENG/Cours/algoP2avanENG.pdf · Sometimes only available method (e.g program optimization) Or exact methods for](https://reader036.vdocuments.net/reader036/viewer/2022070918/5fb8247024aa510ccf007c51/html5/thumbnails/19.jpg)
19
v(Sℝ) v(S
ℕ) since ℕ ℝ
Simplex provides a bound on the solution value in ℕ
We separate on variables bivalents (0-1 IP) Or in some integer interval (IP)
x1
x2
x3
0 1
1
1111
10 0
0000
Branch and Bound methodsInteger programming example
![Page 20: Advanced algorithms and applicationslabsticc.univ-brest.fr/~lemarch/ENG/Cours/algoP2avanENG.pdf · Sometimes only available method (e.g program optimization) Or exact methods for](https://reader036.vdocuments.net/reader036/viewer/2022070918/5fb8247024aa510ccf007c51/html5/thumbnails/20.jpg)
20
It was the first use of the B&B principle (Little algorithm)
Asymetric TSPproblem P
Branch and Bound applicationsTSP example
Exemple : Algorithme et Complexité #4 – c 2002 – Michel Van Caneghem
![Page 21: Advanced algorithms and applicationslabsticc.univ-brest.fr/~lemarch/ENG/Cours/algoP2avanENG.pdf · Sometimes only available method (e.g program optimization) Or exact methods for](https://reader036.vdocuments.net/reader036/viewer/2022070918/5fb8247024aa510ccf007c51/html5/thumbnails/21.jpg)
21
We substract each row its own minimum, and then apply the same operation to columnsproblem P2
Constants Don't change s
olutions ordering
z*(P) = z*(P2) + d
d = rows min(l) +colonms min(c)d = 16 + 3
Branch and Bound applicationsevaluation function
d is a lower bound
![Page 22: Advanced algorithms and applicationslabsticc.univ-brest.fr/~lemarch/ENG/Cours/algoP2avanENG.pdf · Sometimes only available method (e.g program optimization) Or exact methods for](https://reader036.vdocuments.net/reader036/viewer/2022070918/5fb8247024aa510ccf007c51/html5/thumbnails/22.jpg)
22
Branching : flavour on cheapest tours : 0 value edges.
Overcost if edge is excuded : starting from its initial node and going to its terminal … choose the most costly, expecting to cut it later ...
Branch and Bound applicationsbranching function
![Page 23: Advanced algorithms and applicationslabsticc.univ-brest.fr/~lemarch/ENG/Cours/algoP2avanENG.pdf · Sometimes only available method (e.g program optimization) Or exact methods for](https://reader036.vdocuments.net/reader036/viewer/2022070918/5fb8247024aa510ccf007c51/html5/thumbnails/23.jpg)
23
Delete row and column for positive branch, for backward edge (sub tour)
on the edge for negative branch
Branch and Bound applicationsone step
Final search tree
![Page 24: Advanced algorithms and applicationslabsticc.univ-brest.fr/~lemarch/ENG/Cours/algoP2avanENG.pdf · Sometimes only available method (e.g program optimization) Or exact methods for](https://reader036.vdocuments.net/reader036/viewer/2022070918/5fb8247024aa510ccf007c51/html5/thumbnails/24.jpg)
24
Find a tour t T of minimal length into some graph G = (V, E) Find a minimal spanning tree a A within graph G' = (V', E)
V' = V \ {V0}
Connect V0 to a in order to obtain a*' A'
Every tour t where an edge is deleted is a tree, so : T A' and t T, v(a*') v(t)
The algorithm that computes a*' gives a lower bound on the subspace T
Branch and Bound applicationsanother bound (MST)
![Page 25: Advanced algorithms and applicationslabsticc.univ-brest.fr/~lemarch/ENG/Cours/algoP2avanENG.pdf · Sometimes only available method (e.g program optimization) Or exact methods for](https://reader036.vdocuments.net/reader036/viewer/2022070918/5fb8247024aa510ccf007c51/html5/thumbnails/25.jpg)
25
Approximative result, but Sometimes only available method (e.g program optimization) Or exact methods for approximative model only (e.g circuit
testing) Usefullness
Combinatorial explosion Multiple or fuzzy objectives Variability (robustness) Fast runtimes more important than performance
Combinatorial OptimizationHeuristic methods
![Page 26: Advanced algorithms and applicationslabsticc.univ-brest.fr/~lemarch/ENG/Cours/algoP2avanENG.pdf · Sometimes only available method (e.g program optimization) Or exact methods for](https://reader036.vdocuments.net/reader036/viewer/2022070918/5fb8247024aa510ccf007c51/html5/thumbnails/26.jpg)
26
Greedy : build a solution step by step, they never come back on partial choices Often far from optimality
Very fast Local search improvement possible
Examples Knapsack Covering Maximum stable TSP
Combinatorial OptimizationGreedy methods
![Page 27: Advanced algorithms and applicationslabsticc.univ-brest.fr/~lemarch/ENG/Cours/algoP2avanENG.pdf · Sometimes only available method (e.g program optimization) Or exact methods for](https://reader036.vdocuments.net/reader036/viewer/2022070918/5fb8247024aa510ccf007c51/html5/thumbnails/27.jpg)
27
knapsack : fill a knapsack of limited weight, choosing the most interesting objects within a list
Linear program What mean variables
a1x1+ a2x2 ... + anxn bmax c1x1+ c2x2 ... + cnxn
xj j Xj ℕ
Order by profit c1 / a1 c2 / a2 ... cn / an
let x1 = min (1, b/a1) ; b = b - a1x1
Iterate on x2, x3, ..., xn
Greedy algorithmsKnapsack
![Page 28: Advanced algorithms and applicationslabsticc.univ-brest.fr/~lemarch/ENG/Cours/algoP2avanENG.pdf · Sometimes only available method (e.g program optimization) Or exact methods for](https://reader036.vdocuments.net/reader036/viewer/2022070918/5fb8247024aa510ccf007c51/html5/thumbnails/28.jpg)
28
Cover n elements by at least one object, each object has its own cost. Minimize total cost of chosen objects
A, covering matrix. What mean variables Aij
Aij 1
min c1x1 + c2x2 + … + cmxm
xj 1 and xj ℕ
Find k s.t ck/aj = min j 1..n cj/aj with aj = i=1..m Aij
Delete column k and rows such that Aik = 1
Iterate on reduced problem
Greedy algorithmsCovering
![Page 29: Advanced algorithms and applicationslabsticc.univ-brest.fr/~lemarch/ENG/Cours/algoP2avanENG.pdf · Sometimes only available method (e.g program optimization) Or exact methods for](https://reader036.vdocuments.net/reader036/viewer/2022070918/5fb8247024aa510ccf007c51/html5/thumbnails/29.jpg)
29
Closest Neighbourgh algorithm Random starting town
Go to next closest unvisited town
Loop from last to first town
Greedy algorithmsTravelling salesman probllem
![Page 30: Advanced algorithms and applicationslabsticc.univ-brest.fr/~lemarch/ENG/Cours/algoP2avanENG.pdf · Sometimes only available method (e.g program optimization) Or exact methods for](https://reader036.vdocuments.net/reader036/viewer/2022070918/5fb8247024aa510ccf007c51/html5/thumbnails/30.jpg)
30
Problem to face : go out local extrema Random exploration, even costly We have to focus on one solution
(convergency) Single solution evolution
Descent method Simulated annealing Tabu search
Or multiple solutions Genetic algorithm Ants
Neighborhood methods (local search)principle
z min
ok???
![Page 31: Advanced algorithms and applicationslabsticc.univ-brest.fr/~lemarch/ENG/Cours/algoP2avanENG.pdf · Sometimes only available method (e.g program optimization) Or exact methods for](https://reader036.vdocuments.net/reader036/viewer/2022070918/5fb8247024aa510ccf007c51/html5/thumbnails/31.jpg)
31
Define a neighborhood gunction V : S Sn
Provide a set of n solutions similar (close to) s S Explore these n solutions in order to find one that is better
than s
Sometimes non polynomial algorithm Optimality ?
Alternatively, search from a set of seed solutions
Local searchneighborhood
![Page 32: Advanced algorithms and applicationslabsticc.univ-brest.fr/~lemarch/ENG/Cours/algoP2avanENG.pdf · Sometimes only available method (e.g program optimization) Or exact methods for](https://reader036.vdocuments.net/reader036/viewer/2022070918/5fb8247024aa510ccf007c51/html5/thumbnails/32.jpg)
32
Objective function min f : S ℝ
Local search method V : S Sn
Provide a set of n solutions similar (close to) s S
generate an initial solution s0
s = s0
While end not reacheds' = min s V(s) f(s)
Local searchdescent
![Page 33: Advanced algorithms and applicationslabsticc.univ-brest.fr/~lemarch/ENG/Cours/algoP2avanENG.pdf · Sometimes only available method (e.g program optimization) Or exact methods for](https://reader036.vdocuments.net/reader036/viewer/2022070918/5fb8247024aa510ccf007c51/html5/thumbnails/33.jpg)
33
Example : 2-opt for TSP local search(Lin, 1965, n(n - 3)/2) Neighborhood of n2 tours
T' = T U { ik, j l } \ { i j, k l }
3-opt possible, but very large neighborhood n(n – 3)(n - 2)
Local searchTSP Problem
j
i l
kj
il
k
![Page 34: Advanced algorithms and applicationslabsticc.univ-brest.fr/~lemarch/ENG/Cours/algoP2avanENG.pdf · Sometimes only available method (e.g program optimization) Or exact methods for](https://reader036.vdocuments.net/reader036/viewer/2022070918/5fb8247024aa510ccf007c51/html5/thumbnails/34.jpg)
34
Graph G = (V, E) avec |X| = 2n find a partition X = V1 U V2 t.q |V1| = |V2| = n which minimizes the number of edges crossing the 2 parts Pairwise exchange neighborhood
Local search2-way Partitioning problem
a
b
c=7
a
b
c=5 = 7-3+1
![Page 35: Advanced algorithms and applicationslabsticc.univ-brest.fr/~lemarch/ENG/Cours/algoP2avanENG.pdf · Sometimes only available method (e.g program optimization) Or exact methods for](https://reader036.vdocuments.net/reader036/viewer/2022070918/5fb8247024aa510ccf007c51/html5/thumbnails/35.jpg)
35
At each step, choice the exchange maximizing the cut number gain Constraint : a node can be swapped only one time N/2 steps at most
2-way partitionningKernighan-Lin heuristic
![Page 36: Advanced algorithms and applicationslabsticc.univ-brest.fr/~lemarch/ENG/Cours/algoP2avanENG.pdf · Sometimes only available method (e.g program optimization) Or exact methods for](https://reader036.vdocuments.net/reader036/viewer/2022070918/5fb8247024aa510ccf007c51/html5/thumbnails/36.jpg)
36
Recall : trade off to find between Improving current solution Performing an efficient search of all the search space
Always the local extrema problem
Tradeoff
Improving descent methodspoints
z min
ok
???
Exploitation Exploration
![Page 37: Advanced algorithms and applicationslabsticc.univ-brest.fr/~lemarch/ENG/Cours/algoP2avanENG.pdf · Sometimes only available method (e.g program optimization) Or exact methods for](https://reader036.vdocuments.net/reader036/viewer/2022070918/5fb8247024aa510ccf007c51/html5/thumbnails/37.jpg)
37
Applicable to a lot of problems Dynamic evolution of the tradeoff exploit./explor. Based on crystal metal cooking model
Evolution of energy levels within the metal
Meta heuristicssimulated annealing (SA)
E(X)
X
temper annealingFast temperature
descent
Heating again
![Page 38: Advanced algorithms and applicationslabsticc.univ-brest.fr/~lemarch/ENG/Cours/algoP2avanENG.pdf · Sometimes only available method (e.g program optimization) Or exact methods for](https://reader036.vdocuments.net/reader036/viewer/2022070918/5fb8247024aa510ccf007c51/html5/thumbnails/38.jpg)
38
Simulated annealinganalogy physics/optimization
optimization
physics
![Page 39: Advanced algorithms and applicationslabsticc.univ-brest.fr/~lemarch/ENG/Cours/algoP2avanENG.pdf · Sometimes only available method (e.g program optimization) Or exact methods for](https://reader036.vdocuments.net/reader036/viewer/2022070918/5fb8247024aa510ccf007c51/html5/thumbnails/39.jpg)
39
First Explore randomly the search space
Vary the degree of non determism (temperature)
Start high level : exploration, with a very random behavior
End low level : exploitation as with descent methods
Simulated annealingprinciple
![Page 40: Advanced algorithms and applicationslabsticc.univ-brest.fr/~lemarch/ENG/Cours/algoP2avanENG.pdf · Sometimes only available method (e.g program optimization) Or exact methods for](https://reader036.vdocuments.net/reader036/viewer/2022070918/5fb8247024aa510ccf007c51/html5/thumbnails/40.jpg)
40
From current solution si, explore its neighborhood to obtain si+1
If f(si+1) – f(si) < 0, accept si+1 (minimisation)
Else, accept si+1 based on probality :
Simulated annealingone step
p sisi1=e− f si i−f si
T
Gibbs-Bolzmanndistribution
≈
![Page 41: Advanced algorithms and applicationslabsticc.univ-brest.fr/~lemarch/ENG/Cours/algoP2avanENG.pdf · Sometimes only available method (e.g program optimization) Or exact methods for](https://reader036.vdocuments.net/reader036/viewer/2022070918/5fb8247024aa510ccf007c51/html5/thumbnails/41.jpg)
41
Space S, Evaluation f(s)(min),temperature(T)accept(f,T)Initial temperature T = T0
Best solutions = sbest = greedy()
Algo SAwhile criteria1 do
while criteria2 do s2 = neighbor(s, S) f = f(s2) - f(s)
if f(s2) < f(sbest) sbest= s2
if f < 0 or accept(f, T) s = s2
endw
T = temperature(T)Endw End SA
Simulated annealingalgorithm
![Page 42: Advanced algorithms and applicationslabsticc.univ-brest.fr/~lemarch/ENG/Cours/algoP2avanENG.pdf · Sometimes only available method (e.g program optimization) Or exact methods for](https://reader036.vdocuments.net/reader036/viewer/2022070918/5fb8247024aa510ccf007c51/html5/thumbnails/42.jpg)
42
Population(Chromosomes)
Evaluation(fitness)
GeneticOperators
Manipulation
Newgeneration
Selection(groupement)
Parents
Reproduction
Genetic algorithmsOverview
![Page 43: Advanced algorithms and applicationslabsticc.univ-brest.fr/~lemarch/ENG/Cours/algoP2avanENG.pdf · Sometimes only available method (e.g program optimization) Or exact methods for](https://reader036.vdocuments.net/reader036/viewer/2022070918/5fb8247024aa510ccf007c51/html5/thumbnails/43.jpg)
43
Population
Set of individuals
Individuall
Encode a solution
Chromosome based representation
Chromosomes
Allele based
Genetic algorithmsData
00
1 a
cb
![Page 44: Advanced algorithms and applicationslabsticc.univ-brest.fr/~lemarch/ENG/Cours/algoP2avanENG.pdf · Sometimes only available method (e.g program optimization) Or exact methods for](https://reader036.vdocuments.net/reader036/viewer/2022070918/5fb8247024aa510ccf007c51/html5/thumbnails/44.jpg)
44
Population
Reproduction
duplication
Individual
Mutation
Crossing over
Genetic processingbased on chromosomes
a b
+
![Page 45: Advanced algorithms and applicationslabsticc.univ-brest.fr/~lemarch/ENG/Cours/algoP2avanENG.pdf · Sometimes only available method (e.g program optimization) Or exact methods for](https://reader036.vdocuments.net/reader036/viewer/2022070918/5fb8247024aa510ccf007c51/html5/thumbnails/45.jpg)
45
Space S, Evaluation f(s)(min),crossover(P)mutation(P)
Mutation and crossoverrates
Algo GAgenerate initial population P in Swhile end not reached do
reproduction(P) according to f() crossover(P)
mutation(P)endwBest solution within population end GA
Genetic algorithmsalgorithm
![Page 46: Advanced algorithms and applicationslabsticc.univ-brest.fr/~lemarch/ENG/Cours/algoP2avanENG.pdf · Sometimes only available method (e.g program optimization) Or exact methods for](https://reader036.vdocuments.net/reader036/viewer/2022070918/5fb8247024aa510ccf007c51/html5/thumbnails/46.jpg)
46
Population
For reproduction (roulette wheel)
Chances to be selected proportional to fitness Operations
Mutation rate example: 0,5%
Crossing over% of populationRandom cutting sites
Genetic processingramdom choices
i1
i2 i3
i4
Explorationof
the search space
Exploitationof
the search space
![Page 47: Advanced algorithms and applicationslabsticc.univ-brest.fr/~lemarch/ENG/Cours/algoP2avanENG.pdf · Sometimes only available method (e.g program optimization) Or exact methods for](https://reader036.vdocuments.net/reader036/viewer/2022070918/5fb8247024aa510ccf007c51/html5/thumbnails/47.jpg)
47
Empirical technique
Lot of freedom for Chromosome coding (building blocks)
Probabilities
Operators
Coupling witch others approaches
Parallelism
Genetic algorithmsimplementation
![Page 48: Advanced algorithms and applicationslabsticc.univ-brest.fr/~lemarch/ENG/Cours/algoP2avanENG.pdf · Sometimes only available method (e.g program optimization) Or exact methods for](https://reader036.vdocuments.net/reader036/viewer/2022070918/5fb8247024aa510ccf007c51/html5/thumbnails/48.jpg)
48
Different alternatives
Master/slave oriented, fine-grain model (// based on individual evaluation)
+ Local search applicable
Large grain approach (// based on the evolution of multiple independant populations)
Island model (with some individuals migrating from populations)
Genetic algorithmsparallelism
![Page 49: Advanced algorithms and applicationslabsticc.univ-brest.fr/~lemarch/ENG/Cours/algoP2avanENG.pdf · Sometimes only available method (e.g program optimization) Or exact methods for](https://reader036.vdocuments.net/reader036/viewer/2022070918/5fb8247024aa510ccf007c51/html5/thumbnails/49.jpg)
49
Benchmark TSPLIBhttp://www.iwr.uni-heidelberg.de/groups/comopt/software/TSPLIB95/index.html
2D instances (euclidian distances)
Island model, 10 stations, ethernet 100Mb/s, MPI (// interprocs)
1000 generations 1000 individuals Mutation 30%, (cross over ?) Circular migration, 100 iterations, 20% of population. 1 point Crossover, roulette whell selection
Genetic algorithmsparallelism - example
![Page 50: Advanced algorithms and applicationslabsticc.univ-brest.fr/~lemarch/ENG/Cours/algoP2avanENG.pdf · Sometimes only available method (e.g program optimization) Or exact methods for](https://reader036.vdocuments.net/reader036/viewer/2022070918/5fb8247024aa510ccf007c51/html5/thumbnails/50.jpg)
50
Genetic algorithmsparallelism - example
Comparison of Parallel Metaheuristics for Solving the TSPM. Lazarova, P. Borovska CompSysTech’08