the ant colony optimization (aco) metaheuristic: a swarm...

The Ant Colony Optimization (ACO) Metaheuristic:

a Swarm Intelligence Frameworkfor Complex Optimization Tasks

Gianni Di Caro

[email protected]

IDSIA, USI/SUPSI, Lugano (CH)

Road map (1)

Part I: An introduction to Swarm Intelligence (SI) Generalities about SI The role of Nature as source of inspiration Characteristics of SI design Quick overview of the main frameworks based on SI

Part II: The Ant Colony Optimization (ACO) metaheuristic General background: combinatorial optimization,

algorithmic complexity, (meta)heuristics, shortest paths ACO’s biological background: Stigmergy and the shortest

path behavior displayed by ant colonies Description of the metaheuristic How to design new instances of ACO algorithms Quick review of applications

1

Road map (2)

Part III: ACO for traveling salesman and vehicle routingproblems, and their successful application to real-worldproblems (Prof. L. Gambardella)

Part IV: ACO for adaptive network routing problems AntNet: ACO for best-effort routing in wired IP networks,

description of the algorithm and experimental results AntHocNet: ACO for best-effort routing in wireless mobile

ad hoc networks, description of the algorithm and results ofsimulation experiments in realistic settings

Afternoon: General discussions about ACO, and computerexperiments on the effect of different choices regardingparameters and sub-components of the algorithm (consideringthe Traveling Salesman Problem as test case)

2

Part I:

Introduction to Swarm Intelligence

3

Swarm Intelligence: what’s this? (1)

A computational and behavioral metaphor for problem solving thatoriginally took its inspiration from the Nature’s examples of collectivebehaviors (from the end of ’90s):

Social insects (ants, termites, bees, wasps): nest building,foraging, assembly, sorting,. . .

Vertebrates: swarming, flocking, herding, schooling

Any attempt to design algorithms or distributed problem-solvingdevices inspired by the collective behavior of social insects and otheranimal societies [Bonabeau, Dorigo and Theraulaz, 1999]

4

Swarm Intelligence: what’s this? (2)

. . . however, we don’t really need to “stick” on animal societies:collective and distributed intelligence can be seen at almostany level in the biological world (cells, organs, immune andnervous systems, a living body . . . ) or in human organizations

Nowadays swarm intelligence more generically refers to thebottom-up design of distributed control systems that displayforms of “intelligence” (useful behaviors) at the global level asthe result of local interactions among a number of (minimalist)units

In other words: design of complex adaptive systems that dothe job

. . . let’s try to explain this more in detail starting from some ofthe Nature’s examples that gave impetus to this new field . . .

5

Nature’s examples of SI

Fish schooling ( c©CORO, CalTech) 6

Nature’s examples of SI (2)

Birds flocking in V-formation ( c©CORO, Caltech)7


Termites’ nest ( c©Masson)

8


Ant chain ( c©S. Camazine) Ant wall ( c©S. Camazine)9


Ants: leaf-cutting, breeding, chaining

Ants: Food catering

Bees: scout dance

10

What do all these behaviors have in common?

Distributed society of autonomous individuals/agents

Control is fully distributed among the agents

Communications among the individuals are localized

Stochastic agent decisions (partial/noisy view)

System-level behaviors appear to transcend the behavioralrepertoire of the single (minimalist) agent

Interaction rules seem to be simple

The overall response of the system features: Robustness Adaptivity Scalability

11

Swarm Intelligence design means . . . ?

Allocating computing resources to a (large?) number ofminimalist units (swarm?)

No centralized control (not at all?)

Units interact in a simple and localized way

Units do not need a representation of the global task

Stochastic components are important

. . . and let the system generate useful global behaviors byself-organization

Modular design shifting complexity from modules to protocols

12

Self-organization = no explicit programming?

Self-organization consists of set of dynamical mechanisms whereby structureappears at the global level as the result of interactions among lower-levelcomponents. The rules specifying the interactions among the system’sconstituent units are executed on the basis of purely local information,without reference to the global pattern, which is an emergent property of thesystem rather than a property imposed upon the system by an externalordering influence [Bonabeau et al., 1997]

More general: any dynamic system from which order emerges entirelyas a result of the properties of individual elements in the system, andnot from external pressures (e.g., Benard cellular convection,Belousov-Zhabotinski reactions)

In more abstract terms: self-organization is related to an increase ofthe statistical complexity of the causal states of the process [Shalizi,2001]: when a number of units have reached organized coordination,it is necessary to retain more information about the inputs in order tomake a statistically correct prediction

13

I had a dream . . .

. . . I can generate complexity out of simplicity: I can put all theprevious ingredients in a pot, boil them down and get good, robust,adaptive, scalable algorithms for my problems!

. . . it reminds me of alchemists . . .

Task complexity is a conserved variable

14

The dark side of SI design

Predictability is a problem in distributed bottom-up approaches

Efficiency is another issue (BTW, are ants efficient?)

What’s the overall cost? (self-organization is dissipative)

Loads of parameters to assign (e.g., how many agents?)

Nature had millions of years to “design” effective systems byontogenetic and phylogenetic evolution driven by selection,genetic recombination and random mutations,but we have less time. . .

15

Algorithmic frameworks: taxonomy

Agent characteristics: simple/complex,learns/adapts/pre-compiled, reactive/planning, internalstate/stateless

Agent task: provide a component of a solution or provide acomplete solution

Other components: stochastic/deterministic, fully decentralizedor able to exploit some form of centralized control

Common to all frameworks is the fact that the agentsinteract/communicate . . .

16

Algorithmic frameworks: communication models

Point-to-point: antennation, trophallaxis (food or liquidexchange), mandibular contact, direct visual contact, chemicalcontact, . . . unicast radio contact!

Broadcast-like: the signal propagates to some limited extentthroughout the environment and/or is made available for arather short time (e.g., use of lateral line in fishes to detectwater waves, generic visual detection, actual radio broadcast

Indirect: two individuals interact indirectly when one of themmodifies the environment and the other respondsasynchronously to the modified environment at a later time.This is called stigmergy [Grassé, 1959] (e.g., pheromonelaying/following, blackboard systems, (wiki)web)

17

Algorithmic frameworks: examples (1)

Ant Colony Optimization (ACO) and Particle Swarm Optimization(PSO) [Kennedy & Eberhart, 2001], are the most popular(optimization) frameworks based on the original notion of SI

They are both based on a natural metaphor: Shortest path behavior in ant colonies→ ACO (Ant algorithms) Group interaction in bird flocking→ PSO

Both the frameworks are based on the repeated sampling of solutionsto the problem at hand (each agent provides a solution)

They make use of two different ways of distributing, accessing andusing information in the environment:

Stigmergy→ ACO Broadcast-like→ PSO

Both in ACO and PSO agents are supposed to be rather simple, anddo not learn at individual level

Applications: continuous function optimization for PSO, discreteoptimization for ACO 18

Algorithmic frameworks: PSO, some facts (2)

Population-based stochastic optimization technique

Purpose: optimization of continuous nonlinear functions

Background: bird flocking [Reynolds, 1984] and roosting[Heppner & Grenader, 1990], artificial life, social systems

First work: [Eberhart and Kennedy, 1995]

Popularity: A book [Kennedy and Eberhart, 2001], a recentspecial issue on IEEE Transaction on EvolutionaryComputation [Vol. 8, June 2004], topic of interest in severalconferences and workshops

It’s actually a sort of generalization of Cellular Automata[Wolfram, 1984]

19

Algorithmic frameworks: PSO algorithm (3)

Each agents is a particle-like data structure containing: the coordinatesof the current location in the optimization landscape, the best solutionpoint visited so far, the subset of other agents seen as neighbors

procedure Particle Swarm Optimization()foreach particle ∈ ParticleSet do

init at random positions and velocity;select at random the neighbor set;

end foreachwhile (¬ stopping criterion)

foreach particle ∈ ParticleSet docalculate current fitness and update memory;get neighbor with best fitness;calculate individual deviation between current and best so far fitness;calculate social deviation between current and best neighbor fitness;calculate velocity vector variation as weighted sum between deviations;update velocity vector;

end foreachend while

return best solution generated;20

Algorithmic frameworks: other examples (4)

Cellular Automata [Ulam, 1942; Wolfram, 1984]

Artificial Neural Networks [McCulloch & Pitts, 1943]

Artificial Immune Systems [from mid ’90]

Parallel Evolutionary Computation [from mid ’80]

COllective INtelligence [Wolpert & Tumer, 2000]

Cultural Algorithms [Reynolds, 1994]

But also: Multi-Agent Systems, Distributed Artificial Intelligence,Artificial Life, Computational Economics, . . . , Statistical Physics,. . .

21

Applications of SI systems

Optimization problems which are unsuitable or unfeasible foranalytical or exact approaches: nasty multimodal functions, NP-hardproblems, non-stationary environments hard to model, distributedsystems with hidden states, problems with many variables andsources of uncertainty,. . .

Dynamic problems requiring distributed / multi-agent modeling:collective robotics, sensor networks, telecommunication networks, . . .

Problems suitable for distributed / multi-agent modeling: militaryapplications (surveillance with UAVs), simulation of large scalesystems (economics, social interactions, biological systems, virtualcrowds)

Entertainment: video games, collective artistic performance

22

Collective robotics: the power of the swarm!

Collective robotics is attracting a lot of interest: groups ofrobots play soccer (RoboCup), unload cargo, patrol vast areas,cluster objects, self-assemble (Swarm-bots), and will maybe“soon” participate to war :-( . . .

Robot assembly to cross a gap

Robot assembly to transport a prey

Look at RoboCup (www.robocup.org) and Swarm-bots(www.swarm-bots.org)!

23

Part II:

The Ant Colony Optimization Metaheuristic

24

Background: Combinatorial problems

Instance of a combinatorial optimization problem:

Finite set S of feasible solutions Each with an associated real-valued cost J

Find the solution s∗ ∈ S with minimal cost

A combinatorial problem is a set of instances usually sharing somecore properties

In practice, a compact formulation 〈C,Ω, J〉 is used: C is a finite set of elements (decisions) to select→ solution

components Ω is a set of relations among C ’s elements→ constraints The feasible solutions are subsets of components that satisfy the

constraints Ω : S = P(C)|Ω

Costs can change over time, can be probabilistic, the solution set canbe dynamic, problem characteristics might require a specific solutionapproach (centralized, offline, distributed, online),. . . 25

Background: Problem complexity

The time complexity function of an algorithm for a given problemindicates, for each input size n, the maximum time the algorithmneeds to find the optimal solution to an instance of that size(worst-case time complexity)

A polynomial time algorithm has time complexity O(g(n)) where g is apolynomial function. If the time complexity cannot be polynomiallybounded the algorithm is said exponential [Garey & Johnson, 1979]

A problem is said intractable if there is no polynomial time algorithm

Exact algorithms are guaranteed to find the optimal solution and toprove the optimality for every finite size instance of the combinatorialproblem in bounded, instance-dependent time

For the so-called NP-hard class of optimization problems exactalgorithms need, in the worst case, exponential time to find theoptimum (e.g., traveling salesman, quadratic assignment, vehiclerouting, graph coloring, satisfiability,. . . )

26

Background: Heuristics and metaheuristics

For most NP-hard problems of interest, the performance ofexact algorithms is not satisfactory due to the hugecomputational time which is required even for small instances

Heuristic: the aim is to provide good solutions at relatively lowcomputational cost but no guarantee on the absolute quality ofthe generated solution is made available (at most, theasymptotic convergence is guaranteed)

Metaheuristic: a high-level strategy which guides other(possibly problem-specific) heuristics to search for solutions ina possibly wide set of problem domains (e.g., see [Glover &Kochenber, 2002])

Metaheuristics are usually effective and flexible, and areincreasingly gaining popularity. Notable examples are: TabuSearch, Simulated Annealing, Iterated Local Search,Evolutionary Computation, Rollout, GRASP, ACO . . . 27

Background: Problem representation and graphs

Any combinatorial problem can be represented in terms of acapacited directed graph, the component graph: the nodes are the solution components C

the arcs express the relationship between pairs ofcomponents: the presence of an arc 〈ci, cj〉, ci, cj ∈ C,indicates that the ordered pair (sequence) 〈ci, cj〉 is feasiblein a solution

the weight associated to an arc 〈ci, cj〉 is the cost ofincluding in the solution the ordered pair (sequence) 〈ci, cj〉

The optimal solution for the combinatorial problem isassociated to the minimum cost (shortest) path on thecomponent graph

28

Background: Example of component graph (1)

Shortest path (SPP): C = graph nodes

Ex. Sequential decision processes (capacited graph)

Source

Destination

1

4

3

8

9

6

5

7

2

29


Traveling salesman problem (TSP): C = cities to visit

Ex. Goods delivery

Constrained shortest path

1

4

3

8

9

6

5

7

2

30


Data routing: C = network nodes

Shortest path + Multiple traffic flows to route simultaneously

Telecommunication networks

Data traffic

Source

Destination

1

4

3

8

9

6

5

7

2

31

Background: How do we solve these SP problems?

SPP: very efficient (polynomial) algorithms are available (labelsetting / correcting methods)

TSP: it’s NP-hard, optimal algorithms are computationallyinefficient or totally unfeasible→ Heuristic algorithms for good solutions in practice

Routing: there are optimal distributed algorithms for shortestpaths and traffic flows allocation, but:

Non-stationarities in traffic and topology, uncertainties, QoSconstraints, are the norm not the exception!

→ Optimized solutions require full adaptivity and fullydistributed behaviors (Traffic engineering / Autonomic. . . )

32

Background: Construction processes

The component graph is also called construction graph: the evolutionof a generic construction process (sequential decision process) canbe represented as a walk on the component graph

procedure Generic construction algorithm()t← 0;xt ← ∅, J ← 0;while (xt /∈ S ∨ ¬stopping criterion)

ct ← select component(C| xt);J ← J + get component cost(ct| xt);xt+1 ← add component(xt, ct);t← t + 1;

end whilereturn xt, J ;

xj is a partial solution, made of a subset of solution components:xj = c1, c2, . . . , cj → xj+1 = c1, c2, . . . , cj , cj+1, ci ∈ C

Examples: Greedy algorithms, Rollout algorithms [Bertsekas et al.,1997], Dynamic Programming [Bellman, 1957], ACO . . .

33

What did we learn from the background notions?

Some classes of combinatorial problems (NP-hard anddynamic/distributed) are very hard to deal with, and ask for heuristics,or, better, for metaheuristics, which are more flexible

A combinatorial problem can be reduced to a shortest path problem

Therefore, if we have a metaheuristic which can effectively deal withconstrained/distributed/dynamic shortest path problems we have got apowerful and very general problem-solving device for a vast class ofproblems of theoretical and practical interest

But this is precisely what ACO is: a multi-agent metaheuristic withadaptive and learning components whose characteristics areextremely competitive for constrained/distributed/dynamic shortestpath problems . . . since ACO is designed after ant colonies displayinga distributed and adaptive shortest path behavior

34

From Stigmergy to Ant-inspired algorithms

Stigmergy is at the core of most of all the amazing collectivebehaviors exhibited by the ant/termite colonies (nest building,division of labor, structure formation, cooperative transport)

Grassé (1959) introduced this term to explain nest building intermite societies

Goss, Aron, Deneubourg, and Pasteels (1989) showed howstigmergy allows ant colonies to find shortest paths betweentheir nest and sources of food

These mechanisms have been reverse engineered to giveraise to a multitude of ant colony inspired algorithms based onstigmergic communication and control

The Ant Colony Optimization metaheuristic (ACO) [Dorigo, DiCaro & Gambardella, 1999] is the most popular, general, andeffective SI framework based on these principles 35

A short history of ACO (1990 - 2005)

Ant System [Dorigo et al., 1991], for TSP, the first algorithmreverse engineering the ant colony behavior

Application of the same ideas to different combinatorialproblems (e.g., TSP, QAP, SOP, Routing. . . )

Several state-of-the-art algorithms (e.g, ACS [Gambardella &Dorigo, 1996])

Definition of the Ant Colony Optimization (ACO)metaheuristic [Dorigo, Di Caro & Gambardella, 1999]:abstraction of mechanisms and synthesis a posteriori

Convergence proofs, links with other frameworks (e.g.,reinforcement learning, Monte Carlo methods, SwarmIntelligence), application to new problems (e.g., scheduling,subset), workshops, books . . . (1999-2005)

36

Stigmergy and stigmergic variables

Stigmergy means any form of indirect communication among aset of possibly concurrent and distributed agents whichhappens through acts of local modification of the environmentand local sensing of the outcomes of these modifications

The local environment’s variables whose value determine inturn the characteristics of the agents’ response, are calledstigmergic variables

Stigmergic communication and the presence of stigmergicvariables is expected (depending on parameter setting) to giveraise to a self-organized global behaviors

Blackboard/post-it, style of asynchronous communication37

Examples of stigmergic variables

Leading to diverging behavior at the group level:

The height of a pile of dirty dishes floating in the sink

Nest energy level in foraging robot activation [Krieger andBilleter, 1998]

Level of customer demand in adaptive allocation of pick-uppostmen [Bonabeau et al., 1997]

Leading to converging behavior at the group level:

Intensity of pheromone trails in ant foraging: convergence ofthe colony on the shortest path (SP)!

38

SP behavior: pheromone laying

While walking or touching objects, theants lay a volatile chemical substance,called pheromone

Pheromone distribution modifies the environment (the way it isperceived) creating a sort of attractive potential field for the ants

This is useful for retracing the way back, for mass recruitment, forlabor division and coordination, to find shortest paths. . .

39

SP behavior: pheromone-biased decisions

While walking, at each step a routing decision is issued.Directions locally marked by higher pheromone intensity arepreferred according to some probabilistic rule:

π ( τ η),π

τ

η

Decision RuleStochastic

MorphologyTerrain

Pheromone

???

This basic pheromone laying-following behavior is the mainingredient to allow the colony converge on the shortest pathbetween the nest and a source of food

40

SP behavior: a simple example. . .

Nest

Food

t = 0 t = 1

Nest

Food

Pheromone Intensity Scale

Food

Nest

t = 2 t = 3

Nest

Food41

. . . and a more complex situation

FoodNest


Multiple decision nodes

A path is constructed through a sequence of decisions

Decisions must be taken on the basis of local information only

A traveling cost is associated to node transitions

Pheromone intensity locally encodes decision goodness ascollectively estimated by the repeated path sampling

42

Ant colonies: Ingredients for shortest paths

A number of concurrent autonomous (simple?) agents (ants)

Forward-backward path following/sampling

Local laying and sensing of pheromone

Step-by-step stochastic decisions biased by local pheromone intensityand by other local aspects (e.g., terrain)

Multiple paths are concurrently tried out and implicitly evaluated

Positive feedback effect (local reinforcement of good decisions)

Iteration over time of the path sampling actions

Persistence (exploitation) and evaporation (exploration) of pheromone

. . . Convergence onto the shortest path?

43

What pheromone represents in abstract terms?

Distributed, dynamic, and collective memory of the colony

Learned goodness of a local move (routing choice)

Circular relationship: pheromone trails modify environment→locally bias ants decisions→ modify environment

Outcomes of path construction are used to modify pheromone distribution

Paths

Pheromone distribution biases path constructionπ

τ

44

ACO: general architecture

procedure ACO metaheuristic()while (¬ stopping criterion)

schedule activitiesant agents construct solutions using pheromone();pheromone updating();daemon actions(); /∗ OPTIONAL ∗/

end schedule activitiesend while

return best solution generated;

45

ACO: From natural to artificial ant colonies(1)

Destination

Source1

4

3

8

9

5

7

2

τ ;η 14

13τ ;η 13

12τ ;η 12

τ ;η

τ ;η 58

14 59 59

58

6


Each decision node i holds an array of pheromone variables:~τi = [τij ] ∈ IR,∀j ∈ N (i) → Learned through path sampling

τij = q(j|i): learning estimate of the quality/goodness/utility of movingto next node j conditionally to the fact of being in i

Each decision node i also holds an array of heuristics variables:~ηi = [ηij ] ∈ IR,∀j ∈ N (i) → Resulting from other sources ↑

ηij is also an estimate of q(j|i) but it comes from a process or a prioriknowledge not related to the ant actions (e.g., node-to-node distance)

46

ACO: From natural to artificial ant colonies(2)

Destination

Source

1

4

3

8

9

5

7

2

6

Each ant is an autonomous agent that constructs a path P1→9

→ proposes a solution to the problem

There might be one or more ants concurrently active at thesame time. Ants do not need synchronization

A stochastic decision policy selects node transitions:

πε(i, j; ~τi, ~ηi)47

ACO: Combining τ and η in the Ant-routing table

The values of τi and ηi at each node i must be combined inorder to assign a precise goodness value to each locallyavailable next hop j ∈ N (i):

Ai(j) = fτ (τ i, j) fη(ηi, j)

Ai(j) is called the (Ant-routing table): it summarizes all theinformation locally available to make next hop selection

Examples:

ταij · η

βij

ατij + (1− α)ηij

What’s the right balance between τ and η ?

48

ACO: Stochastic decision policy

At node i, πε(i, j) first assigns a selection probability to eachfeasible next node j ∈ N (i) according to its goodness asstored in the ant-routing table, and then makes a probabilisticselection

Example - Random-proportional: πε(i, j) =Ai(j)

∑

k∈N (i)Ai(k)

Example - ε-greedy:

if pb > pu : πε(i, j) = 1 if j = arg maxAi(j), 0 otherwiseelse : πε(i, j) = 1/|N (i)|, ∀j ∈ N (i)

Example - Soft-max: πε(i, j) =eAi(j)/T

∑

k∈N (i) eAi(k)/T, T → 0

49

ACO’s logical diagram can help to understand

Generation of solutions

Pheromone

Schedule Activities

Problem Representation

Manager

Construction of solutions

using pheromone variables

to bias the construction steps

Ant-like Agents

Daemon Actions

without the direct

use of pheromone

Decision Points

Pheromone

Combinatorial

Problem

Optimization

50

ACO: Some important issues still to clarify. . .

What the decision nodes / pheromone variables represent? How do we map the original combinatorial problem into a

learning problem?

When and how pheromone variables are updated (evaluation)? How do we use the sampled data to learn good decisions?

51

Decisions are based on state features

ACO’s philosophy consists in making use of the memory ofpast experience (solution generation) with the aim of learningthe values of a small parameter set, the pheromone set, thatare used by the decision policy to construct solutions

Pheromone variables represent the real-valued decisionvariables which are the learning target

In principle we should try to learn good state transitions: ↑τij = q(xj |xi), or, equivalently τij = q(cj |xi)

This “right” MDP is computationally unfeasible:number of decision variables > number of states(neuro-dynamic programming?)

It’s hard to learn. . .

The alternative is to trade optimality for efficiency using statefeatures instead of the full states: τij = q(cj |ρ(xi))The available state information can be used for feasibility ↑ 52

State diagram for a 4-cities TSP (DP & ants)

53

Pheromone updating

Ants update pheromone online step-by-step→ Implicit pathevaluation based on on traveling time and rate of updates

Ant’s way is inefficient and risky

The right way is online delayed + pheromone manager filter: Complete the path Evaluate and Select “Retrace” and assign credit / reinforce the goodness value

of the decision (pheromone variables) that built the path Total path cost can be safely used as reinforcement signal

Example TSP: s = (c1, c3, c5, c7, c9), J(s) = Lτ13 ← τ13 + 1/L, τ35 ← τ35 + 1/L, . . .

Online step-by-step decrease for exploration (e.g., ACS)

Offline: daemon, evaporation: τij ← ρτij , ρ ∈ [0, 1],54

Designing ACO algorithms

Representation of the problem (pheromone model ~τ ): what are themost effective state features to consider to learn good decisions?

Heuristic variables ~η: what are they and what’s their relative weightwith respect to the pheromone in the decisions?

Ant-routing table A(τ, η): how pheromone and heuristic variables arecombined together to define the goodness of a decision?

Stochastic decision policy πε: how to make good exploration withoutsacrifying exploitation of promising/good actions?

Policies for pheromone updating: online, offline?

Scheduling of the ants: how and/or how many per iteration?

What about restarting after stagnation?

Is it useful to put limits in pheromone ranges?

Pheromone initialization and evaporation, other constants, . . . ?

Daemon components: what’s the effect of local search? 55

Hints: Best design choices from experience

State feature (pheromone model): Last component (forassignment problems)

Heuristic variables: Problem’s costs or lower bounds

Ant-routing functions: Multiplicative or additive

Decision policy: ε-greedy and random proportional

Pheromone updating: Elitist strategies

Number of ants: few ants per a large number of iterations

Pheromone values: Bounded ranges

Daemon procedures: Problem-specific local search

56

Theoretical results ?

Few proofs of asymptotic probabilistic convergence:

in value: ants will find the optimal solution

in solution: all the ants will converge on the optimal solution

Only mild mathematical assumptions

Convergence results are mainly valid for TSP-like problems

It’s important to put finite bounds on the pheromone values

57

Construction vs. Modification methods

Construction algorithms work on the solution components set

Modification strategies act upon the search set of the completesolutions: starting with a complete solution and proceeding bymodifications of it (e.g., Local Search)

Construction methods make use of an incremental local viewof a solution, while modification approaches are based on aglobal view of the solution

58

Generic Local Search (Modification) algorithm

procedure Modification heuristic()define neighborhood structure();s← get initial solution(S);sbest ← s;while (¬ stopping criterion)

s′ ← select solution from neighborhood(N (s));if (accept solution(s′))

s← s′;if (s < sbest)

sbest ← s;end if

end ifend while

return sbest;

59

ACO + Local Search as Daemon component

As a matter of fact, the best instances of ACO algorithms for(static/centralized) combinatorial problems are those makinguse of a problem-specific local search daemon procedure

It is conjectured that ACO’s ants can provide good startingpoints for local search. More in general, a constructionheuristic can be used to quickly build up a complete solution ofgood quality, and then a modification procedure can take thissolution as a starting point, trying to further improve it bymodifying some of its parts

This hybrid two-phases search can be iterated and can be veryeffective if each phase can produce a solution which is locallyoptimal within a different class of feasible solutions. With theintersection between the two classes being negligible.

60

Applications and performance

Traveling salesman: state-of-the-art / good performance Quadratic assignment: good / state-of-the-art Scheduling: state-of-the-art / good performance Vehicle routing: state-of-the-art / good performance Sequential ordering: state-of-the-art performance Shortest common supersequence: good results Graph coloring and frequency assignment: good results Bin packing: state-of-the-art performance Constraint satisfaction: good performance Multi-knapsack: poor performance Timetabling: good performance Optical network routing: promising performance Set covering and partitioning: good performance Parallel implementations and models: good parallelization efficiency

Routing in telecommunications networks: state-of-the-art performance61

Basic references on SI, ACO, and Metaheuristics

M. Dorigo, V. Maniezzo and A. Colorni, "The Ant System: Optimization by a colony ofcooperating agents", IEEE Transactions on Systems, Man, and Cybernetics-Part B, Vol. 26(1),pages 29-41, 1996

M. Dorigo and L. M. Gambardella, "Ant Colony System: A Cooperative Learning Approach tothe Traveling Salesman Problem", IEEE Transactions on Evolutionary Computation", Vol. 1(1),pages 53-66, 1997

G. Di Caro and M. Dorigo, "AntNet: Distributed Stigmergetic Control for CommunicationsNetworks", Journal of Artificial Intelligence Research (JAIR), Vol. 9, pages 317-365, 1998

M. Dorigo and G. Di Caro, "The Ant Colony Optimization Meta-Heuristic", in "New Ideas inOptimization", D. Corne, M. Dorigo and F. Glover editors, McGraw-Hill, pages 11–32, 1999

M. Dorigo, G. Di Caro and L. M. Gambardella, "Ant Algorithms for Discrete Optimization",Artificial Life, Vol. 5(2), pages 137-172, 1999

E. Bonabeau, M. Dorigo and G. Theraulaz, "Swarm Intelligence: From Natural to ArtificialSystems", Oxford University Press, 1999

J. Kennedy and R.C. Eberhart, "Swarm Intelligence", Morgan Kaufmann, 2001 S. Camazine, J.-L. Deneubourg, N. R. Franks, J. Sneyd, G. Theraulaz and E. Bonabeau,

"Self-Organization in Biological Systems", Princeton University Press, 2001 D.H. Wolpert and K. Tumer, "An introduction to collective intelligence", in Handbook of Agent

Technology, ed. J. Bradshaw, AAAI/MIT Press, 2001 F. Glover and G. Kochenberger, "Handbook of Metaheuristics", Kluwer Academic Publishers,

International Series in Operations Research and Management Science, Vol. 57, 2003 G. Di Caro, "Ant Colony Optimization and its Application to Adaptive Routing in

Telecommunication Networks", Ph.D. thesis, Faculté des Sciences Appliquées, Université Librede Bruxelles, Brussels, Belgium, 2004

M. Dorigo and T. Stuetzle, "Ant Colony Optimization", MIT Press, Cambridge, MA, 2004 62

Next . . .

ACO application to traveling salesman and vehicle routingproblems of theoretical and practical interest

ACO application to problems of adaptive routing intelecommunication networks

63

the ant colony optimization (aco) metaheuristic: a swarm...

Documents