simulated annealing. difficulty in searching global optima starting point descend direction local...
TRANSCRIPT
![Page 1: Simulated Annealing. Difficulty in Searching Global Optima starting point descend direction local minima global minima barrier to local search](https://reader036.vdocuments.net/reader036/viewer/2022081811/5697bf811a28abf838c85220/html5/thumbnails/1.jpg)
Simulated Annealing
![Page 2: Simulated Annealing. Difficulty in Searching Global Optima starting point descend direction local minima global minima barrier to local search](https://reader036.vdocuments.net/reader036/viewer/2022081811/5697bf811a28abf838c85220/html5/thumbnails/2.jpg)
Difficulty in Searching Global Optima
startingpoint
descenddirection
local minima
global minima
barrier to local search
![Page 3: Simulated Annealing. Difficulty in Searching Global Optima starting point descend direction local minima global minima barrier to local search](https://reader036.vdocuments.net/reader036/viewer/2022081811/5697bf811a28abf838c85220/html5/thumbnails/3.jpg)
Simulated Annealing
Origin:
The annealing process of heated solids.
Intuition:
By allowing occasional ascent in the search process, we might be able to escape the trap of local minima.
![Page 4: Simulated Annealing. Difficulty in Searching Global Optima starting point descend direction local minima global minima barrier to local search](https://reader036.vdocuments.net/reader036/viewer/2022081811/5697bf811a28abf838c85220/html5/thumbnails/4.jpg)
Consequences of the Occasional Ascents
Help escaping the local optima.
desired effect
Might pass global optima after reaching it
adverse effect
![Page 5: Simulated Annealing. Difficulty in Searching Global Optima starting point descend direction local minima global minima barrier to local search](https://reader036.vdocuments.net/reader036/viewer/2022081811/5697bf811a28abf838c85220/html5/thumbnails/5.jpg)
Simulated Annealing Inspired from ancient process of forging iron Annealing refers to the fast heating of a metal and then
cooling it slowly The method was first proposed by Metropolis (1953) Kirkpatrick et al. (1982) later improved the SA method
and applied it to optimization problems
![Page 6: Simulated Annealing. Difficulty in Searching Global Optima starting point descend direction local minima global minima barrier to local search](https://reader036.vdocuments.net/reader036/viewer/2022081811/5697bf811a28abf838c85220/html5/thumbnails/6.jpg)
Heat the solid state metal to a high temperature.
Cool it down very slowly according to a specific schedule.
If the heating temperature is sufficiently high to ensure random state and the cooling process is slow enough to ensure thermal equilibrium, then the atoms will place themselves in a pattern that corresponds to the global energy minimum of a perfect crystal.
Simulated Annealing
![Page 7: Simulated Annealing. Difficulty in Searching Global Optima starting point descend direction local minima global minima barrier to local search](https://reader036.vdocuments.net/reader036/viewer/2022081811/5697bf811a28abf838c85220/html5/thumbnails/7.jpg)
Simulated Annealing SA differs from hill climbing in such a way
that a move is selected at random and then it is decided whether to accept it.
In SA, better moves are always accepted. Worse moves are accepted probabilistically.
![Page 8: Simulated Annealing. Difficulty in Searching Global Optima starting point descend direction local minima global minima barrier to local search](https://reader036.vdocuments.net/reader036/viewer/2022081811/5697bf811a28abf838c85220/html5/thumbnails/8.jpg)
Simulated Annealing The probability of accepting a worse state
is a function of both the temperature of the system and the change in the cost function
As the temperature decreases, the probability of accepting worse moves decreases
If T=0, no worse moves are accepted (i.e. hill climbing)
![Page 9: Simulated Annealing. Difficulty in Searching Global Optima starting point descend direction local minima global minima barrier to local search](https://reader036.vdocuments.net/reader036/viewer/2022081811/5697bf811a28abf838c85220/html5/thumbnails/9.jpg)
Control of Annealing Process
Acceptance of a search step (Metropolis Criterion):
Assume the performance change in the search direction is .
Accept a ascending step only if it pass a random test,
Always accept a descending step, i.e. 0
1,0exp randomT
![Page 10: Simulated Annealing. Difficulty in Searching Global Optima starting point descend direction local minima global minima barrier to local search](https://reader036.vdocuments.net/reader036/viewer/2022081811/5697bf811a28abf838c85220/html5/thumbnails/10.jpg)
At each temperature, search is allowed to proceed for a certain number of steps, L(k).
Control of Annealing Process
Cooling Schedule:
T, the annealing temperature, is the parameter that control the frequency of acceptance of ascending steps.
We gradually reduce temperature T(k).
The choice of parametersis called the cooling schedule.
kLkT ,
![Page 11: Simulated Annealing. Difficulty in Searching Global Optima starting point descend direction local minima global minima barrier to local search](https://reader036.vdocuments.net/reader036/viewer/2022081811/5697bf811a28abf838c85220/html5/thumbnails/11.jpg)
SA Glossary Solution: An answer to a problem without
respect to its apparent “value” Neighbor: A solution which is “next” to a given
solution in the solution space (or neighborhood) Fitness: The “value” of a solution Temperature: The current average rate at
which less fit solutions are accepted Annealing Schedule: The function which
lowers (or raises) the temperature the algorithm uses during its search
![Page 12: Simulated Annealing. Difficulty in Searching Global Optima starting point descend direction local minima global minima barrier to local search](https://reader036.vdocuments.net/reader036/viewer/2022081811/5697bf811a28abf838c85220/html5/thumbnails/12.jpg)
SA (Pseudo Code)
Algorithm SIMULATED-ANNEALINGBegin
temp = INIT-TEMP;place = INIT-PLACEMENT;while (temp > FINAL-TEMP) do
while (inner_loop_criterion = FALSE) donew_place = PERTURB(place);ΔC = COST(new_place) - COST(place);if (ΔC < 0) then
place = new_place;else if (RANDOM(0,1) < e-(ΔC/temp)) then
place = new_place;temp = SCHEDULE(temp);
End.
![Page 13: Simulated Annealing. Difficulty in Searching Global Optima starting point descend direction local minima global minima barrier to local search](https://reader036.vdocuments.net/reader036/viewer/2022081811/5697bf811a28abf838c85220/html5/thumbnails/13.jpg)
What Annealing Schedule? This is a complicated question!
There are linear schedules, geometric schedules, oscillating schedules, schedules based on the rate of improvement
Some SA algorithms use a GA to design their annealing schedule!
A very basic, yet effective, annealing schedule is one that is geometric, but stepped T = T * 0.9996 However, this is only done every M iterations, which is
either determined by the rate of improvement or by a hard figure (say every 1000 iterations)
Quantitative analysis is the only way to determine the most effective schedule
![Page 14: Simulated Annealing. Difficulty in Searching Global Optima starting point descend direction local minima global minima barrier to local search](https://reader036.vdocuments.net/reader036/viewer/2022081811/5697bf811a28abf838c85220/html5/thumbnails/14.jpg)
Cooling Schedule
Speed at which temperature is reduced has an effect
Too fast and the optima are not found
Too slow and time is wasted
![Page 15: Simulated Annealing. Difficulty in Searching Global Optima starting point descend direction local minima global minima barrier to local search](https://reader036.vdocuments.net/reader036/viewer/2022081811/5697bf811a28abf838c85220/html5/thumbnails/15.jpg)
1) A beginning temperature.2) An ending temperature.3) An assumption of the number of iterations per temperature.4) A cooling rate.
SA Parameters
![Page 16: Simulated Annealing. Difficulty in Searching Global Optima starting point descend direction local minima global minima barrier to local search](https://reader036.vdocuments.net/reader036/viewer/2022081811/5697bf811a28abf838c85220/html5/thumbnails/16.jpg)
Starting Temperature Must be hot enough to allow moves to
almost every neighborhood state Must not be so hot that we conduct a
random search for a long period of time Problem is finding a suitable starting
temperature
![Page 17: Simulated Annealing. Difficulty in Searching Global Optima starting point descend direction local minima global minima barrier to local search](https://reader036.vdocuments.net/reader036/viewer/2022081811/5697bf811a28abf838c85220/html5/thumbnails/17.jpg)
Implementation of Simulated Annealing
Select a local search scheme
Determine the cooling schedule
For example:
• Set L = n, the number of variables in the problem.
• Set T(0) such that exp(-/T(0)) 1.• Set T(k+1) = T(k), where is a
constant smaller but close to 1.