Topic 7: Stigmergy, Swarm Intelligence and Ant Algorithms stigmergy swarm intelligence ant algorithms AntNet: routing AntSystem: TSP

Some natural Swarm Intelligence systemsAntsfind the shortest path to foodgo on raiding parties to gather itcarry it cooperativelymake cemeteriessort their brood by sizeetc. Termitesbuild nests with complex features likefortified chambersspiral air ventsfungus gardensetc.Beesgather pollen with high efficiency, exploiting the nearest richest food source first.Geesecoordinate takeoff and landingflight patternsetc.Fish / Birds /swarmsHuman societieseconomies ?

Swarm IntelligenceSwarm Intelligence (SI) is the property of a system whereby the collective behaviors of (unsophisticated) agents interacting locally with their environment cause coherent functional global patterns to emerge.

characteristics of a swarm:distributed, no central control or data sourceno (explicit) model of the environmentperception of environment, i.e. sensingability to change environment

problem solving is emergent behaviour

How Do Social Insects Coordinate Their Behaviour?

communication is necessary

two types of communicationdirectantennation, trophallaxis (food or liquid exchange), mandibular contact, visual contact, chemical contact, etc.indirecttwo individuals interact indirectly when one of them modifies the environment and the other responds to the new environment at a later time called stigmergy

Stigmergy La coordination des taches, la regulation des constructions ne dependent pas directement des oeuvriers, mais des constructions elles-memes. Louvrier ne dirige pas son travail, il est guid par lui. Cest cette stimulation dun type particulier que nous donnons le nom du STIGMERGIE (stigma, piqure; ergon, travail, oeuvre = oeuvre stimulante). Grass P. P., 1959

[The coordination of tasks and the regulation of constructions does not depend directly on the workers, but on the constructions themselves. The worker does not direct his work, but is guided by it. It is to this special form of stimulation that we give the name STIGMERGY (stigma, sting; ergon, work, product of labour = stimulating product of labour).]

Example of Stigmergy: Trail Following and Ants Foraging Behaviour

while walking, ants and termitesmay deposit a pheromone on the groundfollow with high probability pheromone trails they sense on the ground

Example of Stigmergy:Ant cemeteries in naturesimple behaviour rules for antswander around

if you find a dead ant pick it up with probability inversely proportional to the number of other dead ants nearbyif you are carrying a dead ant put it downwith probability directly proportional to the number of other dead ants nearby

StigmergyStimulation of workers by the performance they have achieved Grass P. P., 1959

Stigmergy Communication through marks in the environment Marks serve as a shared memory for the agentsPromotes loose coupling Robust

Stigmergy ~ action + sign: Agents put marks in the environment (inform other agents about issues of interest) other agents perceive these marks (influence their behavior) manipulate marks other agents perceive the marks .etc.

Marks can be static (environment does not manipulate marks over time) e.g., flagsdynamic (environment manipulates marks over time)e.g., pheromones, gradient fields

Ants Foraging BehaviourExample: The Double Bridge Experiment15 cmNestFoodABSimple bridge % of ant passages on the two branchesGoss et al., 1989, Deneubourg et al., 1990

Food foraging in nature

Asymmetric Binary Bridge Experiment

Some Resultsr is the length ratio among the two bridgesr = 2r = 2Short branch added later

Chart4

46

34

10

10

0

% of traffic on the short branch

% of experiments

Chart1

15

0

0

0

85

% of traffic on the short branch

% of experiments

Sheet1

Equal BranchesShort/long branchesAdaptation

0-2050046

20-40034

40-608010

60-800510

80-10042950

100100100

Sheet1

% of traffic on the short branch

% of experiments

Sheet2

% of traffic on the short branch

% of experiments

Sheet3

% of traffic on the short branch

% of experiments

Chart3

0

0

0

5

95

% of traffic on the short branch

% of experiments

Chart1

15

0

0

0

85

% of traffic on the short branch

% of experiments

Sheet1

Equal BranchesShort/long branchesAdaptation

0-2050046

20-40034

40-608010

60-800510

80-10042950

100100100

Sheet1

% of traffic on the short branch

% of experiments

Sheet2

% of traffic on the short branch

% of experiments

Sheet3

% of traffic on the short branch

% of experiments

Artificial StigmergyIndirect communication mediated by modifications of environmental states which are only locally accessible by the communicating agentsDorigo & Di Caro, 1999

Characteristics of artificial stigmergy: Indirect communicationLocal accessibility

What Are Ant Algorithms?Ant algorithms are multi-agent systems that exploit artificial stigmergy as a means for coordinating artificial ants for the solution of computational problems

examplesAntNet: network routingAntSystem: TSP

AntNet: An Ant Algorithm for Routing in Packet-Switched Networks (e.g. Internet)the routing problembuild routing table at each node

costs are dynamicadaptive routing is hard

Sheet1

routing table for node k

destination521

next node338

AntNet data structurerouting tableprobabilities of choosing each neighbor nodes for each possible final destination

trips vectorcontains statistics about ants trip times from current node k to each destination node d (means and variances)P(k,n,d)

Sheet1

destination node

neighbour noded

n

Artificial Ants in a Network?Pheromone traildepositingSourceDestinationMemory

Simple AntNet Algorithm

ants are launched regularly from each node to randomly chosen destination

ants build their paths probabilistically with probability function ofartificial pheromone valuesheuristic values

ants memorize visited nodes and incurred costs

once destination is reached, ants deterministically retrace their path backwards, updating pheromone trails that is a function of the quality of the solution they generated

refresh + evaporation !!

AntNet Routing - AgentsTwo kinds of AgentsForward Ant explores the network and collects informationwhen reaches the destination, changes into backward antBackward Antgoes back in the same path as forward antupdate routing tables for all the nodes in the path

AntNet: F-Ants and B-AntsF-antscollect implicit and explicit information on paths and traffic loadimplicit through arrival rate at destinationexplicit by storing experienced trip timesshare queues with data packetsB-antsbackpropagate fastuse higher priority queuesBackward Ant1234Backward AntBackward AntBackward Ant

Using Pheromone and Memoryto Choose the Next NodeMemory of visited nodesitijdtirdjktikdr

Ants Probabilistic Transition Rule tijd is the amount of pheromone trail on edge (i,j,d) Jik is the set of feasible nodes ant k positioned on node i can move to

Using Pheromone and Heuristicto Choose the Next NodeMemory of visited nodesijkrtikd ; ikdtijd ; ijdtird ; irdstored in pheromone tablea heuristic evaluation of link (i,j,d)which introduces problem specific information

Using Pheromone and Heuristicto Choose the Next NodeMemory of visited nodesijkrtikd ; ikdtijd ; ijdtird ; irdstored in pheromone tablea heuristic evaluation of link (i,j,d)which introduces problem specific information:for AntNet: proportional to the inverse of link (i,j) queue length

B-ants update routing tables:if B-ant has source node d and goes from node n to k

Increase the probability of the channel that backward ant comes fromP(k,n,d) = P(k,n,d) + R.(1 P(k,n,d)) Decrease the probability of the other channels P(k,i,d) = P(k,i,d) - R.(1 P(k,i,d))for all i != n

R (0 < R

AntNet evaluationextensive tests and experimentsAmerican NSF netJapanese NTT netartificial networkssimple graphsgrid networks

in short: good resultssimilar throughput compared to other routing algorithmsremarkably smaller avg. packet delay

Ant Colony Optimization (ACO)TSP has been a popular problem for the ACO models. - several reasons why TSP is chosen

Key concepts: Positive feedbackbuild a solution using local solutions, by keeping good solutions in memory.Negative feedbackwant to avoid premature convergence, evaporate the pheromone.Time scalenumber of runs are also critical.

Traveling Salesman Problem (TSP)

Ant Optimization of TSPseed all paths with initial pheromone

ants make a tour of all citiesant probabilistically selects next city to visit considering distance and pheromone strength on each path

ants deposit pheromone on their path with an intensity inversely proportional to the length of their tour

pheromone decays with each time step

repeat steps 2 - 4 until some threshold is met

A simple TSP exampleAEDCBdAB =100;dBC = 60;dDE =150

Iteration 1AEDCB

How to build next sub-solution?AEDCB

Iteration 2AEDCB

Iteration 3AEDCB

Iteration 4AEDCB

Iteration 5AEDCB

Path and Pheromone EvaluationL1 =300L2 =450L3 =260L4 =280L5 =420

End of First RunAll ants dieNew ants are bornSave Best Tour (Sequence and length) stopping criteriastagnationmax.runs

Ant cycle for TSP tabu list random walk priorities backtracing and considering edge length5321141

Tabu Listn : number of cities = one city tour m : number of ants k : ant index s : member in tabu list (current city)for each iteration in the ant algorithm cycle: insert town in visited node listfor (int k = 1 ; k

Random Walk & Prioritiesi,j : edge between nodes i,j nij : 1/distance(i,j) a : weight for marking b : weight for node neighbourhoodnessprobability of ant k for going from city i to city jfrom ant routing table:allowedk : unvisited cities for ant k

Backtracingi,j : edge between nodes i,j tij (t): marking at time t r : evaporation rate (t,t+n)marking after tour = evaporation_rate * marking_old + marking_deltamarking_delta = sum of markings of all ants k who passed (i,j)Q/Lk : const/tour-length of ant kWith Tabu List, ant routing table:

Ant System (Ant Cycle) Dorigo [1] 1991t = 0; NC = 0; ij(t)=c for ij=0Place the m ants on the n nodes

Update tabuk(s)

Compute the length Lk of every antUpdate the shortest tour found

=For every edge (i,j)Compute For k:=1 to m do

InitializeChoose the city j to move to. Use probabilityTabu list management

Move k-th ant to town j. Insert town j in tabuk(s)Set t = t + n; NC=NC+1; ij=0 NC

10-Cities-Problem CCA0AntSystem marking distribution at beginning and after 100 cycles

Oliver30 Problemcycles: 342 length : 420a = 1 b = 5 r = 0.5

Advantages:positive feedback accounts for rapid discovery of good solutionsdistributed computation avoids premature convergencethe greedy heuristic helps find acceptable solution in the early solution in the early stages of the search processthe collective interaction of a population of agents

Disadvantages:slower convergence than other heuristicsperformed poorly for TSP problems larger than 75 cities

Improvements to AS: Ant Colony System (ACS)ACS (pheromone update)update pheromone trail while building the solutionants eat pheromone on the traillocal search added before pheromone update

Research Summary

Ant System (AS) [Dorigo, Maniezzo & Colorni]Original implementation of ant-inspired optimization

Ant Colony System (ACS) [Dorigo & Gambardella]Ants probabilistically choose exploitation or biased exploration

Max-Min Ant System (MMAS) [Stutzle & Hoos]Only ant with best tour deposits pheromoneRange of pheromone limited to given intervalPheromone levels initialized to max levels

Research Summaryrefinements

Pheromone Trail Smoothing (PTS)increases pheromone on all segments in proportion to the difference from the max value

Ranked Ants n ants with best paths deposit pheromone based on rank

Elite Antsbest path per cycle receives additional pheromone

Modified 3-Optiteratively swap order of three-city sub-paths such that total path length decreases

Result ComparisonPTS = Pheromone Trail SmoothingMMAS = MaxMin AntSystemACS = Ant Colony SystemAS = Ant System10,000 cycles per run[Stutzle & Hoos, 1999]

Instance

Optimal

MMAS+pts

MMAS

ACS

eil51

426

427.1

427.6

428.1

kroA100

21282

21291.6

21320.3

21420.0

d198

15780

15956.8

15972.5

16054.0

Instance

AS-Rank

AS-Rank +pts

AS-Elite

AS-Elite +pts

AS

eil51

434.5

428.8

428.3

427.4

437.3

kroA100

21746.0

21394.9

21522.8

21431.9

22471.4

d198

16199.1

16025.2

16205.0

16140.8

16702.1

General ACOa stochastic construction procedureprobabilistically build a solutioniteratively adding solution components to partial solutions heuristic informationpheromone trailreinforcementmodify the problem representation at each iteration

ants work concurrently and independentlycollective interaction via indirect communication leads to good solutions

Conclusiongeneral ACO work well on static problems like TSP, but hard to beat specialist algorithmsbut most of all ants are dynamic optimizersinherently capable of dealing with dynamism

Other Application Domains for Antsmanufacturing controlpeer-2-peeractive networkingsimilar principlesartificial stigmergy / feedback through pheromonesinformation decay (pheromone evaporation)probabilistic choice in path following