topic 6: stigmergy, swarm intelligence and ant algorithms
DESCRIPTION
Topic 6: Stigmergy, Swarm Intelligence and Ant Algorithms. swarm intelligence stigmergy ant algorithms AntNet: routing AntSystem: TSP. Swarm Intelligence. - PowerPoint PPT PresentationTRANSCRIPT
Topic 6: Stigmergy, Swarm Intelligence and Ant Algorithms
swarm intelligence stigmergy ant algorithms
AntNet: routing AntSystem: TSP
Swarm Intelligence
“Swarm Intelligence (SI) is the property of a system whereby the collective behaviors of (unsophisticated) agents interacting locally with their environment cause coherent functional global patterns to emerge.”
characteristics of a swarm: distributed, no central control or data source no (explicit) model of the environment perception of environment, i.e. sensing ability to change environment
problem solving is emergent behaviour
Some natural Swarm Intelligence systems Ants
find the shortest path to food make cemeteries sort their brood by size etc.
Termites build nests with complex features like
fortified chambers spiral air vents fungus gardens etc.
Bees gather pollen with high efficiency, exploiting the nearest richest food source first.
Geese coordinate takeoff and landing flight patterns etc.
Fish / Birds /… swarms
How Do Social Insects Coordinate Their Behaviour?
communication is necessary
two types of communication direct
antennation, trophallaxis (food or liquid exchange),mandibular contact, visual contact, chemical contact, etc.
indirect two individuals interact indirectly when one of them modifies the
environment and the other responds to the new environment at a later time
called stigmergy
Stigmergy
“La coordination des taches, la regulation des constructions ne dependent pas directement des oeuvriers, mais des constructions elles-memes. L’ouvrier ne dirige pas son travail, il est guidé par lui. C’est à cette stimulation d’un type particulier que nous donnons le nom du STIGMERGIE (stigma, piqure; ergon, travail, oeuvre = oeuvre stimulante).” Grassé P. P., 1959
[“The coordination of tasks and the regulation of constructions does not depend directly on the workers, but on the constructions themselves. The worker does not direct his work, but is guided by it. It is to this special form of stimulation that we give the name STIGMERGY (stigma, sting; ergon, work, product of labour = stimulating product of labour).”]
Stigmergy
Communication through marks in the environment … Marks serve as a shared memory for the agents Promotes loose coupling Robust
Stigmergy ~ action + sign: Agents put marks in the environment (inform other agents about issues of
interest) other agents perceive these marks (influence their behavior) manipulate marks other agents perceive the marks etc.
Marks can be static (environment does not manipulate marks over time)
e.g., flags dynamic (environment manipulates marks over time)
e.g., pheromones, gradient fields
Example of Stigmergy:Ant cemeteries in nature
simple behaviour rules for ants wander around
if you find a dead antpick it up with probability inversely proportional
to the number of other dead ants nearby if you are carrying a dead ant
put it down with probability directly proportionalto the number of other dead ants nearby
Example of Stigmergy: Trail Following and Ants Foraging Behaviour
while walking, ants and termites may deposit a pheromone on the ground follow with high probability pheromone trails they sense on the
ground
Nest
Food
Ants Foraging BehaviourExample: The Double Bridge Experiment
15 cm
Nest Food
A
B0
25
50
75
100
0 5 10 15 20 25 30
Time (minutes)
% of passages
% Passages A % Passages B
Simple bridge % of ant passages on the two branches
Goss et al., 1989, Deneubourg et al., 1990
Asymmetric Binary Bridge Experiment
Food foraging in nature
Some Results
r is the length ratio among the two bridges
0
50
100
0-20 20-40 40-60 60-80 80-100
% of traffic on the short branch
r = 1
0
50
100
0-20 20-40 40-60 60-80 80-100
% of traffic on the short branch
r = 2
0
50
100
0-20 20-40 40-60 60-80 80-100
% of traffic on the short branch
r = 2
Short branch added later
Nest
Food
Evaporation
and
Refresh
“Artificial” Stigmergy
Indirect communication mediated by modifications of environmental states which are only locally accessible by the communicating agents
Dorigo & Di Caro, 1999
Characteristics of artificial stigmergy: Indirect communication Local accessibility
What Are Ant Algorithms?
Ant algorithms are multi-agent systems that exploit artificial stigmergy as a means for coordinating artificial ants for the solution of computational problems
examples1. AntNet: network routing
2. AntSystem: TSP
AntNet: An Ant Algorithm for Routing in Packet-Switched Networks (e.g. Internet)
the routing problem build routing table at each node
costs are dynamic adaptive routing is hard
routing table for node kdestination 5 2 1 …next node 3 3 8
AntNet data structure
routing table probabilities of choosing each neighbor nodes for each possible
final destination
trips vector contains statistics about ants’ trip times from current node k to
each destination node d (means and variances)
destination nodeneighbour node d
…n
…
P(k,n,d)
Artificial Ants in a Network
?Probabilistic rule to
choose the path
Pheromone traildepositing
Source
Destination
Memory
Simple AntNet Algorithm
ants are launched regularlyfrom each node to randomly chosen destination
ants build their paths probabilistically with probability function of artificial pheromone values heuristic values
ants memorize visited nodes and incurred costs
once destination is reached, ants deterministically retrace their path backwards, updating pheromone trails that is a function of the quality of the solution they generated
refresh + evaporation !!
AntNet Routing - Agents
Two kinds of Agents Forward Ant
explores the network and collects information when reaches the destination, changes into backward ant
Backward Ant goes back in the same path as forward ant update routing tables for all the nodes in the path
AntNet: F-Ants and B-Ants
F-ants collect implicit and explicit information on paths and traffic load
implicit through arrival rate at destination explicit by storing experienced trip times
share queues with data packets B-ants
backpropagate fast use higher priority queues
Backward Ant
1 2 3 4
Forward Ant
(N1,T1)
(N2,T2)
(N1,T1)Forward Ant
(N1,T1)
(N2,T2)
(N3,T3)Forward Ant
(N1,T1)
(N2,T2)
(N3,T3)
(N4, T4)Forward Ant
Backward AntBackward AntBackward Ant
Using Pheromone and Memoryto Choose the Next Node
Memory of visited nodes
i
ijd
ird
j
k
ikd
r
Ants’ Probabilistic Transition Rule
ijd is the amount of pheromone trail on edge (i,j,d)
Jik is the set of feasible nodes ant k positioned on
node i can move to
Using Pheromone and Heuristicto Choose the Next Node
Memory of visited nodes
ij
k
r
ikd ; ikd
ijd ; ijd
ird ; ird
stored in pheromone table
a heuristic evaluation of link (i,j,d)which introduces problem specific information
Using Pheromone and Heuristicto Choose the Next Node
Memory of visited nodes
ij
k
r
ikd ; ikd
ijd ; ijd
ird ; ird
stored in pheromone table
a heuristic evaluation of link (i,j,d)which introduces problem specific information:for AntNet: proportional to the inverse of link (i,j) queue length
B-ants update routing tables: if B-ant has source node d and goes from node n to k
Increase the probability of the channel that backward ant comes fromP(k,n,d) = P(k,n,d) + R.(1 – P(k,n,d))
Decrease the probability of the other channels P(k,i,d) = P(k,i,d) - R.(1 – P(k,i,d)) for all i != n
R (0 < R <= 1), function of- T: time experienced by F-ant- m: avg time for same destination memorized in trips table- sigma: std. deviation for same destination memorized in trips table
destination nodeneighbour node d
…n
…
P(k,n,d)
AntNet evaluation
extensive tests and experiments American NSF net Japanese NTT net artificial networks
simple graphs grid networks
in short: good results similar throughput compared to other routing algorithms remarkably smaller avg. packet delay
Ant Colony Optimization (ACO)
TSP has been a popular problem for the ACO models.
Key concepts: Positive feedback
build a solution using local solutions, by keeping good solutions in memory
Negative feedback want to avoid premature convergence, evaporate the pheromone
Time scale number of runs are also critical
Traveling Salesman Problem (TSP)
Ant Optimization of TSP
1. seed all paths with initial pheromone
2. ants make a tour of all cities ant probabilistically selects next city to visit considering distance and
pheromone strength on each path
3. ants deposit pheromone on their path with an intensity inversely proportional to the length of their tour
4. pheromone decays with each time step
5. repeat steps 2 - 4 until some threshold is met
A simple TSP example
A
ED
C
B
1
[]
4
[]
3
[]
2
[]
5
[]
dAB =100;dBC = 60…;dDE =150
Iteration 1
A
ED
C
B1
[A]
5
[E]
3
[C]
2
[B]
4
[D]
How to build next sub-solution?
A
ED
C
B1
[A]
1
[A]
1
[A]1
[A]
1
[A,D]
otherwise 0
allowed j if k
⎪⎪⎩
⎪⎪⎨
⎧∈
∑=∈ kallowedk
ikik
ijij
kij
][)]t([][)]t([
)t(pβα
βα
otherwise 0
allowed j if k
⎪⎪⎩
⎪⎪⎨
⎧∈
∑=∈ kallowedk
ikik
ijij
kij
][)]t([][)]t([
)t(pβα
βα
Iteration 2
A
ED
C
B3
[C,B]
5
[E,A]
1
[A,D]
2
[B,C]
4
[D,E]
Iteration 3
A
ED
C
B
4
[D,E,A]
5
[E,A,B]
3
[C,B,E]
2
[B,C,D]
1
[A,D,C]
Iteration 4
A
ED
C
B4
[D,E,A,B]
2
[B,C,D,A]
5
[E,A,B,C]
1
[A,D,C,E]
3
[C,B,E,D]
Iteration 5
A
ED
C
B
1
[A,D,C,E,B]
3
[C,B,E,D,A]
4
[D,E,A,B,C]
2
[B,C,D,A,E]
5
[E,A,B,C,D]
Path and Pheromone Evaluation
1
[A,D,C,E,B]
5
[E,A,B,C,D]
L1 =300⎪⎩
⎪⎨
⎧ ∈=
otherwise 0
tour )j,i(ifL
Q
kkj,iτΔ
L2 =450
L3 =260
L4 =280
L5 =420
2
[B,C,D,A,E]
3
[C,B,E,D,A]
4
[D,E,A,B,C]
5B,A
4B,A
3B,A
2B,A
1B,A
totalB,A ΔΔΔΔΔΔ ++++= 5
B,A4
B,A3
B,A2
B,A1
B,Atotal
B,A ΔΔΔΔΔΔ ++++=
End of First Run
All ants die
New ants are born
Save Best Tour (Sequence and length)
stopping criteriastagnationmax.runs
Ant cycle for TSP
1. tabu list
2. random walk
3. priorities
4. backtracing and considering edge length
55
3322
11 1144
11
Tabu List
n : number of cities = one city tourm : number of antsk : ant indexs : member in tabu list (current city)
for each iteration in the ant algorithm cycle:insert town in „visited node list“
for (int k = 1 ; k <= m ; k++) { insert town of ant k in Tabuk(s) }
Random Walk & Priorities
i,j : edge between nodes i,jnij : 1/distance(i,j)α : weight for markingβ : weight for node „neighbourhoodness“
0,][)]([
][)]([)( elseallowedjif
nt
nttp
kkij
kallowedlilil
ijij ∈⋅
⋅=∑
∈
βα
βα
probability of ant k for going from city i to city j
from ant routing table:
allowedk : unvisited cities for ant k
Backtracing
∑==
ΔΔm
k
k
ijij1ττ
ijijij tnt ρ Δ+⋅=+ )()(
i,j : edge between nodes i,jij (t): marking at time tρ : evaporation rate (t,t+n)
marking after tour = evaporation_rate * marking_old + marking_delta
marking_delta = sum of markings of all ants k who passed (i,j)
0,),( elseTourjiifLQ k
k
k
ij ∈=Δ
Q/Lk : const/tour-length of ant kWith Tabu List, ant routing table:
Ant System (Ant Cycle) Dorigo [1] 1991
t = 0; NC = 0; τij(t)=c for ∆τij=0Place the m ants on the n nodes
Update tabuk(s)
Compute the length Lk of every antUpdate the shortest tour found
=For every edge (i,j)Compute
For k:=1 to m do
Initialize
Choose the city j to move to. Use probability
Tabu list management
Move k-th ant to town j. Insert town j in tabuk(s)
Set t = t + n; NC=NC+1; ∆τij=0 NC<NCmax
&& not stagn.
Yes
End
No
Yes
ijijij )t()nt( Δρ +=+
⎪⎩
⎪⎨
⎧ ∈=
otherwise 0
by tabu describedtour k)j,i(ifL
Q
kkj,iτΔ
kijijij : ΔΔΔ +=
otherwise 0
allowed j if k
⎪⎪⎩
⎪⎪⎨
⎧∈
∑=∈ kallowedk
ikik
ijij
kij
][)]t([][)]t([
)t(pβα
βα
10-Cities-Problem CCA0
AntSystem marking distribution at beginning and after 100 cycles
Oliver30 Problem
cycles: 342length : 420
α = 1β = 5ρ = 0.5
Some inherent advantages
positive feedback accounts for rapid discovery of good solutions
distributed computation avoids premature convergence
the greedy heuristic helps find acceptable solution in the early stages of the search process
the collective interaction of a population of agents
Disadvantages in Ant Systems
slower convergence than other heuristics performed poorly for TSP problems larger than 75
cities
Improvements to AS: Ant Colony System (ACS)
ACS (pheromone update)
)t()t()1()1t( bestijijij Δρρ +−=+
update pheromone trail while building the solution ants eat pheromone on the trail local search added before pheromone update
Research Summary
Ant System (AS) [Dorigo, Maniezzo & Colorni] Original implementation of ant-inspired optimization
Ant Colony System (ACS) [Dorigo & Gambardella] Ants probabilistically choose exploitation or biased exploration
Max-Min Ant System (MMAS) [Stutzle & Hoos] Only ant with best tour deposits pheromone Range of pheromone limited to given interval Pheromone levels initialized to max levels
Research Summary
refinements
Pheromone Trail Smoothing (PTS) increases pheromone on all segments in proportion to the difference from
the max value
Ranked Ants n ants with best paths deposit pheromone based on rank
Elite Ants best path per cycle receives additional pheromone
Modified 3-Opt iteratively swap order of three-city sub-paths such that total path length
decreases
General ACO
a stochastic construction procedure probabilistically build a solution iteratively adding solution components to partial
solutions heuristic information pheromone trail
reinforcement modify the problem representation at each iteration
ants work concurrently and independently collective interaction via indirect communication leads
to good solutions
Conclusion
general ACO work well on static problems like TSP, but hard to beat specialist algorithms
but most of all …
ants are “dynamic” optimizers inherently capable of dealing with dynamism