shortest paths through a reinforced random walk417959/fulltext01.pdf · 2011. 5. 19. · describe a...
TRANSCRIPT
U.U.D.M. Project Report 2011:8
Examensarbete i matematik, 30 hpHandledare och examinator: David J.T. Sumpter
Maj 2011
Department of MathematicsUppsala University
Shortest paths through a reinforced random walk
Dan Li
Acknowledgements
I would first like to thank Professor David Sumpter for his patient guidance, giving
demeanor, the many valuable lessons/opportunities, and friendship during my study. I would
like to thank Professor Sumpter, Anders Johansson, Qi Ma and Alma Kirlic at UU that have
helped me with many aspects of my study including intellectual discussions, advise, sharing
expertise, and friendship. I would like to thank the support of my family for their constant
source of encouragement, understanding, and love, without of which I could not have
completed this work.
II
Abstract
In this article, we first study random walks on special electric networks with unit
resistors and use four methods to solve Dirichlet problems in two dimensions. Then we study
random walks on general resistor networks and give probabilistic interpretations to the
qualities in electric network. In order to understand the movements of ants in an electric
network, we describe a random walk model where ants moving through the network do not
modify their path. After making a modification to this model by involving conductivity, we
build current reinforced random walk model since ants are made to respond the current they
generate. Here we allow ants to iteratively improve the path they take so that they can find the
shortest path. Two algorithms based on exponential distribution and Poisson distribution are
built and compared. We will look at the results about shortest path finding by random walks
and implement simulation of two algorithms for shortest path problems.
Keywords: shortest path problem, random walk, electric network, reinforced random
walk, exponential distribution, Poisson distribution
III
Contents
1. Introduction ............................................................................................ 1
2. Random walks on networks[12] ............................................................... 3
2.1 Random walks in two dimensions.................................................................... 3
2.1.1 Define problem in terms of particles walking in a network.............................................. 3
2.1.2 The payoff function: Probability of winning ...................................................................... 4
2.1.3 Correspondence with an electric network.......................................................................... 4
2.1.4 Harmonic functions in two dimensions; the uniqueness and maximum principle ......... 6
2.1.5 The Dirichlet problem .......................................................................................................... 7
2.2 The Monte Carlo solution................................................................................. 8
2.2.1 Monte Carlo Method ............................................................................................................ 8
2.2.2 Implementation on examples using method of Monte Carlo............................................ 9
2.3 The method of relaxations.............................................................................. 10
2.3.1 The method of relaxations ................................................................................................. 10
2.3.2 Implementation on examples using method of relaxations ............................................. 11
2.4 Exact solution by linear algebra ..................................................................... 12
2.5 Solution by the method of Markov chains ..................................................... 15
2.5.1 Some definitions[14][15]......................................................................................................... 15
2.5.2 Markov chain methods[14] .................................................................................................. 15
2.5.3 Implementation on examples using method of Markov chain........................................ 17
2.5.4 Summary ............................................................................................................................. 21
2.6 Random walks on more general networks ..................................................... 21
2.6.1 Random walks on a graph[12]............................................................................................. 21
2.6.2 Some language from graph theory[17] ............................................................................... 22
2.6.3 Probabilistic interpretation in electrical network............................................................ 22
2.6.4 Random walk model........................................................................................................... 24
3. Shortest path problems ......................................................................... 28
3.1 Ant colony optimization ................................................................................. 28
3.1.1 Ants’ foraging behavior and optimization ........................................................................ 29
3.1.2 Double bridge experiments ................................................................................................ 29
3.1.3 ACO for the Travelling Salesman Problem ...................................................................... 30
3.1.4 The Ant Colony Optimization Metaheuristic................................................................... 31
3.1.5 Ant System (AS) [3][20][21] ..................................................................................................... 32
3.1.6 Simple ant colony optimization (S-ACO) ......................................................................... 34
3.2 Current reinforced random walks................................................................... 35
3.2.1 Description of current reinforced random walks ............................................................ 35
3.2.2 Pheromone Rule and Ant Moving Rule ............................................................................ 36
3.2.3 A Simple Explanation ......................................................................................................... 37
3.2.4 Exponential distributed simulation................................................................................... 39
IV
V
3.2.5 Implementation in Matlab ................................................................................................. 40
3.3 An efficient algorithm based on Poisson distribution .................................... 46
3.3.1 Some knowledge of the Poisson process[14] ....................................................................... 46
3.3.2 Event-driven approaches ................................................................................................... 47
3.3.3 Poisson distributed simulation .......................................................................................... 48
3.3.4 Implementation in Matlab ................................................................................................. 49
4. Physarum solver ................................................................................... 56
4.1 Introduction to Physarum............................................................................... 56
4.2 Mathematical modeling of maze-solving by Physarum................................. 57
4.3 Biologically inspired adaptive network[27] ..................................................... 59
5. Appendix .............................................................................................. 62
References ................................................................................................ 76
Shortest paths through a reinforced random walk 1
1. Introduction
In our world, the emergence of complex behavior in a system consisting of simple,
interacting elements is among the most fascinating phenomena. We can find many
examples in every field of today’s scientific interest, ranging from the motion of animal
swarms[1] of biology, to the behavior of social groups[2]. Ant colonies, and more generally
social insect societies, are distributed systems that, in spite of the simplicity of their
individuals, present a highly structured social organization[3]. As a result of this
organization, ant colonies can accomplish complex tasks that in some cases far exceed the
individual capabilities of a single ant.
The field of “ant algorithms” [4] [5]studies models derived from the observation of real
ants’ behavior, and uses these models as a source of inspiration for the design of novel
algorithms for the solution of optimization and distributed control problems. There is a
problem faced by ants in an ant colony. How can a large number of individuals, none of
which have global information about the problem build a short connecting path between
food and the nest? Ants usually find their food sources by chance, for example, they reach
their destination in a stochastic way. In our model, the ants merely count on the local
information provided by chemical trail in order to guide themselves. They do not have
additional navigation or information processing capacities and are not subject to long-range
attracting forces to the food sources or to the nest. Ants coordinate their activities via
stigmergy[1] [6], a form of indirect communication mediated by modifications of the
environment. For example, a foraging ant[1] deposits a chemical on the ground which
increases the probability that other ants will follow the same path. We draw attention to this
specific collective phenomenon of trail formation[2]. Trails are adapted to the requirements
of their users. In the course of time, frequently used trails become more developed, making
them more attractive, whereas rarely used trails vanish again. These dynamical processes[7]
occur without any common planning or direct communication among the users.
How almost blind animals, like ants can manage to build the shortest paths from the
colony to the feeding sources and back? The collective behavior[8] is a form of behavior
where the more ants following a trail, the more attractive that trail becomes for being
followed. Pheromone trails[9] are the medium that used to communicate information among
individuals regarding paths, and to decide where to go. A moving ant lays pheromone in
varying quantities on the path. When an isolated ant moves at random, an ant encountering
a previously laid trail can detect it and decide to follow the path with high probability, thus
reinforcing the path with its own pheromone[11] . The probability with which an ant chooses
Shortest paths through a reinforced random walk 2
a path increases with the number of ants that chose the same path previously. The process is
characterized by a positive feedback loop. The trail formation model includes an equation
of motion, an equation for environmental changes, and an orientation relation.
The trails are used to connect the food sources with the nest to allow for a collective
exploitation of the food. In the case of ants, the markings named pheromones, which also
provide the basic orientation for foraging and homing of the animals[2]. Ants can find the
shortest way through a maze and even amoeba can minimize the length of their connections
between food sources. One way to see such problem solving is in terms of individuals
performing a random walk on a graph. As the individuals walk they reinforce the path they
have taken so others coming after them are more likely to take the same path[10]. Those
taking the shortest path will reinforce this most strongly. Under certain conditions, the path
created by such a process will converge to the shortest path. This project will look at results
about shortest path finding by random walks[12] and implement simulations of algorithm for
shortest path problems.
Ants are not unique in such shortest path problem. There are parallels between the
creation of pheromone trails in ants, the growth of tubes in Physarum and the construction
of trails by human pedestrians[2]. There is a sense in which nutrients, ants and humans all
perform a reinforced random walk within the network they create. Indeed, the algorithm
works in all three systems at some level. It is simple, being based on a reinforcement of
path (pheromone trail and slime mold’s tube) in response to traffic flow (ant traffic and
protoplasmic flow).
The article is structured as follows. First, describe random walks in networks[12] and
use different methods to find solution to the Dirichlet problem in two dimensions. Monte
Carlo method[12] is studied carefully. Random walks on general resistor networks are
studied and probabilistic interpretations to the qualities in electric network are given. Then,
describe a random walk model which can not be said to be solving the shortest path
problem. Based on this random walk model, the reinforced random walk model is built
after making a modification so that it is possible to make the random walkers moving
through the network modify their path. Discuss a mathematical result which shows that the
algorithm of reinforced random walk will, provided the algorithm is structured properly,
always converge to our optimal solution — the shortest path. In addition, I also study the
Ant Colony Optimization (ACO) [3] [6] and the Physarum Solver [13]. The algorithms of
reinforced random walk model based on exponential distribution and Poisson distribution
are stated and the simulations of algorithms are implemented using Matlab.
Shortest paths through a reinforced random walk 3
2. Random walks on networks[12]
In this chapter, random walks on finite networks will be studied. We follow the
monograph of Doyle and Snell [12] in what follows, giving different examples than
provided by them. We know that the connection between random walks and electrical
networks has been recognized for some time. We will establish the connection between the
electrical concepts of current and voltage and corresponding descriptive quantities of
random walks regarded as finite state Markov chains. Note that we call the random walker
as “particle” in electrical network. However, I would like to call the random walker as “ant”
in the random walk model of solving shortest paths problems.
2.1 Random walks in two dimensions
2.1.1 Define problem in terms of particles walking in a network
Let 21 2 1 2, : ,z z z z R
2 D
2 denote two-dimensional integer array (lattice).
Let , then the boundary of is D : 1cD y D x D x y . Note that
and are disjoint necessarily. A simple random walk starting at
D
D 2x is a stochastic
process, that is, a simple random walk is a random process where a walker starts at point x
and chooses one of its neighbors with equal probability and moves there at each time unit.
In the figures 2-1-1 and 2-1-2 below, we illustrate two such arrays. We consider the
case with boundary values 1 or 0 in the figures. The large dots represent boundary points;
those marked 0 indicate that the walker receive no payoff and those marked 1 indicate that
the walker gets payoff 1. We study a simple random walk on a two dimensional array. The
walker, starting at the interior point ,x a b , moves randomly to one of the four
neighboring points: , 1 1,y a b b2 1,y a , 3 , 1y a b or 4 , 1y a b . Each
alternative is chosen equally likely, i.e. with probability1
4. This random moving continues
until the walker reaches a boundary point, in which case he/she remains at this point, i.e.
the random walk is absorbed at the boundary points and obtain the payoff indicated.
Shortest paths through a reinforced random walk 4
Figure 2-1-1: Small arrays for a random walk Figure 2-1-2: Another arrays for a random walk
2.1.2 The payoff function: Probability of winning
Consider a random walk on one of the arrays in figures 2-1-1 and 2-1-2. Let p x be
the probability, starting at x , that the random walk is absorbed at a boundary point with
value 1. It is also the expected final payoff for starting point x . We regard p x as a
function defined on the points of the array. Using conditioning with respect to the next
point visited, it is easy to see that the function p x has the following fundamental
properties:
1. or 0p x 1p x if x is a boundary point with value 0 or 1, respectively.
2. For an interior point x , the value p x is the average of p y at the neighbours,
i.e.
. 1 2 3
1
4 4p x p y p y p y p y (2.1)
We can express (2.1) by stating that the function p is discrete-harmonic (at each interior
point). The function p x , will in other words be a solution the discrete Dirichlet problem
for the array: It is harmonic in the interior with specified values at the boundary.
2.1.3 Correspondence with an electric network
We consider an electric network depicted in figure 2-1-2. We assume that all resistors
have the same resistance. Voltage 1v x is established at the boundary points
1, 2, 4,5,9,13x and voltage 0v x , for 8,12,14x . We obtain this by shorting
the two set of points and applying a voltage source (battery) between them.
Shortest paths through a reinforced random walk 5
Figure 2-1-3: An electric network
If two points x and are connected by a resistance of magnitude , then the
current through the resistor equals
y r
,i x y v x v y r / by Ohm’s law. By
Kirchhoff’s current laws, the current flowing into a point x must be equal to the current
flowing out. For an interior point x , the current flowing into x equals
1 2 3 4.
y N x
v y v x
r
v y v x v y v x v y v x v y v x
r r r r
where is a shorthand for +u max ,0u and where 1 2 3 4: , , ,N x y y y y denotes the
four neighbors of x . The flow out of node x is similarly obtained as
y N x
v x v y
r
and equating both sides gives Kirchhoff’s current law in the form
y N x y N x
v y v x v x v y
r r
which is equivalent to
0
y N x
v x v y
r
(2.2)
since . Multiplying through by , and solving for gives + +a b b a a b r v x
1 2 3 4
1.
4v x v y v y v y v y
That is, the voltage function satisfies the same equation (2.1) as the payoff
function
v x
p x constructed before. It is also a fact that v x will have the same values at
Shortest paths through a reinforced random walk 6
the boundary points as p x if we identify the electrical network. It is natural to guess
that they are equal, and in order to deduce this, we will need a uniqueness principle for
discrete harmonic functions.
2.1.4 Harmonic functions in two dimensions; the uniqueness and maximum principle
We define harmonic functions for sets of lattice points in the plane (a lattice point is a
point in the plane with integer coordinates). Let be a finite set of lattice points
partitioned into two disjoint set
S
B and such that: (a) every vertex in has four
neighbours in and (b) every pair of points and in can be connected by a p
consisting of vertices in S , i.e. a sequence P x
D D
S u v
lx
S ath
1 2...x , 1x u and lx v , of points ix
in S , where, for all i , ix is a neighbour of 1ix . We refer to points in B as boundary
points and points in D as inte r points. rio
A function f is harmonic on S B D if it satisfies (2.1) for all points x in D , i.e.
the average equation
1.
| | y N x
f xN x
f y (2.3)
There is no restriction on the values of f x at the boundary points x in B .
As we have seen, the payoff function p x , defined in section 2.1.2, is harmonic
which follows by conditioning on the first steps. The voltage function v x is harmonic,
which follows by Kirchhoff’s Current Laws and Ohm’s Law.
The Uniqueness Principle for the Dirichlet problem asserts that there cannot be two
different harmonic functions having the same boundary values: If p x and are
two functions that are harmonic on
v x
S B D with boundary B and if
all z en
v z p z for
thB p x v r all x fo x S . This is a general fact about harmonic
functions. We will approach the Uniqueness Principle by way of the Maximum Principle
for harmonic functions. The Maximum Principle states that a harmonic function defined on
takes on its maximum value S max :M f x x S and its minimum value
minm f :x x S on the boundary B , i.e. points where the averaging equation (2.3)
Shortest paths through a reinforced random walk 7
does not hold. The argument for it is natural but we do not give the details: At interior
points f x equals the average of its neighbours so f x must either be smaller than
one of its neighbours or else all neighbours have the value f x . If f x M at one
interior point x then f z M for some z S , since we can connect x by a path to a
boundary vertex . The argument for the minimum value is symmetric. z
p x v x and To deduce the uniqueness principle, we note that if are harmonic in
then is also harmonic. This is due to equation (2.3) being a
homogenous linear equation and is readily verified by direct application. By assumption,
we have , for all in
S r x
r z
v
v
x
z
p x
0p z z B . But then the Maximum Principle states
that the maximum and minimum value of r x is zero which means that for all r x 0
x S and this means that p x v x x S for all .
2.1.5 The Dirichlet problem
In the following sections 2.2~ 2.5, we will show how the simple random walks in
figures 2-1-1 and 2-1-2 could be used to construct the solutions to a discrete version of the
classical Dirichlet problem, say discrete Dirichlet problem. It is necessary to define
harmonic function and Laplace operator in a discrete setting.
Suppose that is bounded. The Dirichlet problem is to find the unique
harmonic function : such that i) Laplace’s equation
2
D R
D
u 0u x x D , where
is the Laplace operator; ii) p xu x x D .
0u x If , is harmonic on if for every D2D u x D , we have .
Suppose that . The Laplace operator of is defined as 2u R u:
1 1
1 1
4 4e e
u x u x e
1e 1
4u xu x e u x
. This definition gives us
another view of Laplace operator, as the difference between the mean value of over the
neighbors of point
u
x and the value of at u x .
Shortest paths through a reinforced random walk 8
2.2 The Monte Carlo solution
We will generate approximate solution to a Dirichlet problem in two dimensions
before finding the exact solution. In this section, we present a Monte Carlo method that
using random walks for finding approximate solutions.
2.2.1 Monte Carlo Method
The Monte Carlo method can be used to calculate the scalar field satisfying Laplace’s
equation over a solution region with specified boundary conditions. It is basically an
application of the law of large numbers, since (an extension of) the argument above shows
that an absorbing random walk starting at x obtains the harmonic solution as the expected
final payoff. A given random walk is terminated when the boundary B of the solution
region is reached. Assume p z f z for boundary points z B are prescribed
boundary values or final payoffs. We want to compute or approximate the expected payoff
p x starting at the interior point x D . Assume that we can make random
experiments starting at
N
x and compute the empirical averages of the payoff
1, ,
z B
p x N N z p zN
where is the number of times the walk was absorbed at N z z B . By the Law of
Large Numbers we obtain that , p x N approach p x with probability one as
. N
Figure 2-2-1: The Matlab code in Appendix 2-2-1 calculates the scalar field at the centre of a square solution region
with boundary conditions on the bottom, right, top, and left boundaries, respectively.
Let us give an example and make the implementation using Monte Carlo method. In
figure 2-2-2, we consider the case with boundary values 1 or 0 as in our example. In this
case, the expected final payoff is the probability that the walker ends up at a boundary point
Shortest paths through a reinforced random walk 9
with value 1 (“success”) rather than a point of value 0 (“failure”). We start
random walks from each of the interior points and, for each
10,000N
x , count the number of
successes and estimate
n x
/p x n x N . The Matlab code is stated in Appendix 2-2-2.
Figure 2-2-2 The network and boundary values Figure 2-2-3 The nodes are denoted with integers
We get the outcome in Matlab:
We reorder the outcome and rewrite the solution in the form as figure 2-2-4:
1 1
0.837 0.837 1
1 0.838 0.516 0.372 0
1 0 0
Figure 2-2-4: Empirical frequency of success
2.2.2 Implementation on examples using method of Monte Carlo
We implement Monte Carlo simulation on our two examples which are shown in
figures 2-1-1 and 2-1-2 in section 2.1.1 and get the Monte Carlo results shown in figures
2-2-5 and 2-2-6. The Matlab code can be found in Appendix 2-2-3 (a) and 2-2-3 (b).
(a) For figure 2-1-1:
Shortest paths through a reinforced random walk 10
0 0
1 0.290 0.085 0
0 0.080 0.043 0
0 0
Figure 2-1-1: The network and boundary values Figure 2-2-5: Empirical frequency of success
(b) For figure 2-1-2:
1 1
1 0.830 0.789
1 0.877 0.502 0.326 0
1 0 0
1
Figure 2-1-2: The network and boundary values Figure 2-2-6: Empirical frequency of success
2.3 The method of relaxations
In this section, we will describe the method of relaxation. It is a more effective
method for finding approximate solutions to a discrete Dirichlet problem. This method was
originally developed as a way to get approximate solutions to continuous Dirichlet problem.
But we would like to replace the continuous problem with a discrete problem.
2.3.1 The method of relaxations
The method of relaxations[12] is a simple numerical scheme to get approximate
solutions to a discrete Dirichlet problem. We are looking for a function which has specified
boundary values, for which the value at any interior point is the average of the values at its
neighbours. The method goes like this. Pick up an interior point and adjust the value of the
function to be the average of the values at its neighbours. Then continue through the rest of
the interior points, updating every interior point by the average of its neighbours. After
running though all the interior points, the function will not be harmonic after only one
iteration of this process, although the function we get will be more harmonic than the
function we started with. If we iterate this averaging process, running all of the interior
points again and again, the resulting function will better and closely approximate the
solution to our Dirichlet problem.
Shortest paths through a reinforced random walk 11
2.3.2 Implementation on examples using method of relaxations
We will illustrate the method of relaxations in the case of the examples introduced in
section 2.1.1.
(a) For figure 2-1-1:
In figure 2-3-1, the computations are illustrated on our simple example (example in
figure 2-1-1). We first assign the value 0 to all interior points and keep the boundary points
fixed. Each iteration traverses all interior points in order. Note that we go from left to right
moving up each column replacing each value by the average of the four neighbouring
values. We print the results to three decimal places. The computations for this first iteration
of the method of relaxation are
0.250 1/ 4 1 0 0 0 ; 0.063 1/ 4 0.25 0 0 0 ;
0.063 1/ 4 0.25 0 0 0 ; 0.031 1/ 4 0.063 0.063 0 0 .
We continue the iteration and at iteration 5 and 6, we obtain the same results to three
decimal places. It is obvious that the values for the interior points have converged to the
required harmonic function.
Thus, the solution to the Dirichlet problem is precisely as (h) in figure 2-3-1.
0 0
1 0 0 0
0 0 0 0
0 0
0 0
1 0.250 0.063 0
0 0.063 0.031 0
0 0
(a) figure 2-1-1: The Dirichlet problem (b) Initial configuration (c) After one iteration
0 0
1 0.281 0.078 0
0 0.078 0.039 0
0 0
0 0
1 0.289 0.082 0
0 0.082 0.041 0
0 0
0 0
1 0.291 0.083 0
0 0.083 0.042 0
0 0
(d) After two iterations (e) After three iterations (f) After four iterations
0 0
1 0.292 0.084 0
0 0.084 0.042 0
0 0
0 0
1 0.292 0.084 0
0 0.084 0.042 0
0 0
(g) After five iterations (h) After six iterations
Figure 2-3-1 The relaxation method (for figure 2-1-1)
Shortest paths through a reinforced random walk 12
(b) For figure 2-1-2:
In figure 2-3-2, the computations are illustrated on a slightly larger example (example
in figure 2-1-2).
1 1
1 0 0 1
1 0 0 0 0
1 0 0
1 1
1 0.547 0.648 1
1 0.750 0.188 0.047 0
1 0 0
(a) figure 2-1-2: The Dirichlet problem (b) Initial configuration (c) After one iteration
1 1
1 0.749 0.749 1
1 0.797 0.348 0.249 0
1 0 0
1 1
1 0.802 0.776 1
1 0.837 0.459 0.302 0
1 0 0
1 1
1 0.817 0.782 1
1 0.865 0.492 0.310 0
1 0 0
(d) After two iterations (e) After three iterations (f) After four iterations
1 1
1 0.821 0.786 1
1 0.873 0.5 0.321 0
1 0 0
1 1
1 0.823 0.787 1
1 0.875 0.504 0.323 0
1 0 0
1 1
1 0.823 0.787 1
1 0.875 0.506 0.323 0
1 0 0
(g) After five iterations (h) After six iterations (i) After seven iterations
1 1
1 0.823 0.787 1
1 0.876 0.506 0.323 0
1 0 0
1 1
1 0.823 0.787 1
1 0.876 0.506 0.323 0
1 0 0
(j) After eight iterations (k) After nine iterations
Figure 2-3-2 The relaxation method (for figure 2-1-2)
After only nine iterations, we obtain the same result to three decimal places accuracy.
This only took a fraction of a second of computing time. We shall say that these results are
correct to three place accuracy. The Monte Carlo method took several seconds of
computing time and did not even give three place accuracy.
2.4 Exact solution by linear algebra
In this section, we will find an exact solution to a two dimensional Dirichlet problem.
We introduce a method by solving a system of linear equations.
Shortest paths through a reinforced random walk 13
(a) For figure 2-1-1,
Figure 2-1-1: The interior points are labelled a, b, c, d
By averaging property, we have
1
4
4
4
4
b ca
a db
a dc
b cd
x xx
x xx
x xx
x xx
After rewriting these equations in the matrix form, then we will get
1 1/ 4 1/ 4 0 1/ 4
1/ 4 1 0 1/ 4 0
1/ 4 0 1 1/ 4 0
0 1/ 4 1/ 4 1 0
a
b
c
d
x
x
x
x
For simplicity, we write it as Ax u . must have an inverse since there is a unique
solution. And we have
A
1x A u .
Carrying out this calculation, we will have
0.292
0.083
0.083
0.042
Calculated x
We compare this calculated solution with the approximate solutions found earlier:
0.290
0.085
0.080
0.043
Monte Carlo x
. .
0.292
0.084Re
0.084
0.042
laxed x
Shortest paths through a reinforced random walk 14
We can find that Monte Carlo approximations were fairly good because no error is
greater than 0.01. The relaxed approximations were good indeed since two values do not
show error and the errors of the other two values are only 0.001 (that is far less than 0.01).
(b) For figure 2-1-2,
Figure 2-1-2: The interior points have been labelled a, b, c, d, e
After labelling each interior point as , we represent the average property of
each interior point in terms of the following matrix form
, , , ,a b c d e
1 1/ 4 0 1/ 4 0 1/ 2
1/ 4 1 0 0 1/ 4 1/ 2
0 0 1 1/ 4 0 3/
1/ 4 0 1/ 4 1 1/ 4 0
0 1/ 4 0 1/ 4 1 0
a
b
c
d
e
x
x
x
x
x
4
.
There is a unique solution:
0.823
0.787
0.876
0.506
0.323
Calculated x
For comparison, the approximate solutions found earlier using two different method
are shown again
0.830
0.789
0.877
0.502
0.326
Monte Carlo x
. .
0.823
0.787
Re 0.876
0.506
0.323
laxed x
We see that the Monte Carlo approximations were fairly good because no error of the
simulation is greater than 0.01. The relaxed approximations were good indeed, because no
error occurs at all
Shortest paths through a reinforced random walk 15
2.5 Solution by the method of Markov chains
Solving an equation like can be interpreted as finding the fundamental matrix of
a Markov chain. In this section we will show how to find exact solution to the Dirichlet
problem by the method of Markov chains. It can be seen as a more complicated version of
the method of linear equations which has been stated above.
2.5.1 Some definitions[14][15]
We have a set of states 1 2, , , rS s s s
r
and a chance process that moves around
through these states. The process moves from the state to the state with the
probability . This process is called finite Markov chain. We can represent the transition
probabilities as an matrix P . This matrix is called transition matrix.
is
P
js
ijP
ijP r by
If we raise the transition matrix to its power as , the elements can be
stated as the probability that the chain that started in state will be in state after
steps. A Markov chain is a regular chain if some power of the transition matrix has no
zeros.
P nth nP
is
nijP
js n
A state with is called closed state or absorbing state. Once entered, cannot
be left. If a Markov chain has at least one absorbing state and if it is possible, not
necessarily in one step, to reach at least one absorbing state from any state, it is called an
Absorbing Markov chain. The states of an absorbing Markov chain that are not traps can be
called non-absorbing states. An absorbing Markov chain will finally end up in an absorbing
state if it is started in a non-absorbing state. We can denote by the matrix
i 1iip
B with elements
ijB which can be stated as the probability, starting at a non-absorbing state , that the
process ends at an absorbing state .
is
js
2.5.2 Markov chain[14] methods
We can find a way to solve our Dirichlet problem if we could find out the matrix B
by Markov chain techniques. In fact, the Dirichlet problem has a generalization in the
context of absorbing Markov chains so that it can be solved by Markov chain methods. We
will show this later.
Shortest paths through a reinforced random walk 16
We assume is an absorbing Markov chain which has absorbing states and
non-absorbing states. In order to let the absorbing states come first and the non-absorbing
states come last, we reorder the states then rewrite our transition matrix of dimension
as the canonical form
P
u v
P
u v by u v
0IP
R Q
,
where I is a identity matrix; is matrix with all elements 0. u by u 0 u by v
The matrix is called the fundamental matrix, it can be interpreted as
(Note that
1N I Q
2 ...I Q Q 1N I Q
I is not same as the identity matrix we stated
above, it is dimension here). The elements of the matrix represent the
expected number of times that the chain, starting at , will reach state before
absorption. In order to get the probability of ending up at a given absorbing state, we add
up the probabilities of going that given absorbing state from all the non-absorbing states
weighted by the number of times we expected to be in those non-absorbing states. So the
absorption probabilities
v by v ijN
iS
N
S j
B can be calculated by matrix formula 1B NR I Q R
.
Let us see a random walk example in figure 2-5-1 (a). We find the matrix B by the
Markov chain techniques which are shown in figure 2-5-1 (b) ~ (e):
(a) Random walks starts at 2, ends at 0 or 4 (b) The canonical form of the transition matrix
0 1/ 2 0
1/ 2 0 1/ 2
0 1/ 2 0
Q
1
3 / 2 1 1/ 2
1 2 1
1/ 2 1 3/ 2
N I Q
(c) Random walks reach non-absorbing states (d) The fundamental matrix
Shortest paths through a reinforced random walk 17
(e) The absorption probabilities matrix B
Figure 2-5-1 Markov chain methods
The absorbing Markov chain can be made to correspond to a Dirichlet problem in the
following way. The absorbing states can be seen as boundary states and the non-absorbing
states can be seen as interior states. Let B be the set of boundary states and the set of
interior states as we have stated in section 2.1.4. Let
D
f be a function which domain is the
state space of a Markov chain . For in set D , we have ijj
f i P fP i j . Then f
is a harmonic function for . If we represent P f as a column vector , f f is harmonic
if and only if . f=fP
We can write as , where represents the values of ff
ff
B
D
fB f at the boundary
points and fD represents the values of f at the interior points. We have
f f
f fB
D D
I
0
B Q
B
. If we let f fD BNR , then we see that
f1f f f f f fD B B B B B BR QN R
f =Bf
I QN R N NR Q I Q Q NR R N . Thus
D B gives a solution to . In Markov chain theory 1B NR I R
Q . Thus
1f fD BI Q
R . From this equation, we can see that the values of a harmonic function at
the boundary points determine the values of the function.
2.5.3 Implementation on examples using method of Markov chain
We will illustrate the method of Markov chain in the case of our two examples.
(a) For figure 2-1-1,
Shortest paths through a reinforced random walk 18
Figure 2-1-1 Figure 2-5-2
1) We regard this random walk as a Markov chain with states 1,2,3,…,12 in figure
2-5-2. The transition matrix is given by:
1 0 0 0 0 0 0 0 0 0 0 0
0 1 0 0 0 0 0 0 0 0 0 0
0 0 1 0 0 0 0 0 0 0 0 0
1/ 4 0 1/ 4 0 1/ 4 0 0 1/ 4 0 0 0 0
0 1/ 4 0 1/ 4 0 1/ 4 0 0 1/ 4 0 0 0
0 0 0 0 0 1 0 0 0 0 0 0
0 0 0 0 0 0 1 0 0 0 0 0
0 0 0 1/ 4 0 0 1/ 4 0 1/ 4 0 1/ 4 0
0 0 0 0 1/ 4 0 0 1/ 4 0 1/ 4 0 1/ 4
0 0 0 0 0 0 0 0 0 1 0 0
0 0 0 0 0 0 0 0 0 0 1 0
0 0 0 0 0 0 0 0 0 0 0 1
P
2) We reorder the states so that the absorbing states come first and the non-absorbing
states come last. Then our transition matrix has the canonical form:
0IP
R Q
,
where R is the matrix of random walks reach absorbing states; is the matrix of
random walks reach non-absorbing states.
Q
Shortest paths through a reinforced random walk 19
1/ 4 0 1/ 4 0 0 0 0 0
0 1/ 4 0 1/ 4 0 0 0 0
0 0 0 1/ 4 0 0 1/ 4 0
0 0 0 0 0 1/ 4 0 1/ 4
R
, ,
0 1/ 4 1/ 4 0
1/ 4 0 0 1/ 4
1/ 4 0 0 1/ 4
0 1/ 4 1/ 4 0
Q
1
0.2917 0.0833 0.2917 0.1667 0 0.0417 0.0833 0.0417
0.0833 0.2917 0.0833 0.3333 0 0.0833 0.0417 0.0833
0.0833 0.0417 0.0833 0.3333 0 0.0833 0.2917 0.0833
0.0417 0.0833 0.0417 0.1667 0 0.2917 0.0833 0.2917
B I Q R
3) , , f
ff
B
D
1 2 3 6 7 10 11 12f f f f f f f f f 10 0 0 0 0 0 0T T
B
4
5
8
9
f 0.2917
f 0.0833f f
f 0.0833
f 0.0417
D BB
0.292
0.083.
0.083
0.042
Markov chains x
When starting at node 4, the expected payoff of the game is 0.292; when starting at
node 5 and node 8, the expected payoff is 0.083; when starting at node 9, the expected
payoff is 0.042.
(b) For figure 2-1-2,
Shortest paths through a reinforced random walk 20
Figure 2-1-2 Figure 2-5-3
The canonical form of the transition matrix is
1 0 0 0 0 0 0 0 0 0 0 0 0 0
0 1 0 0 0 0 0 0 0 0 0 0 0 0
0 0 1 0 0 0 0 0 0 0 0 0 0 0
0 0 0 1 0 0 0 0 0 0 0 0 0 0
0 0 0 0 1 0 0 0 0 0 0 0 0 0
0 0 0 0 0 1 0 0 0 0 0 0 0 0
0 0 0 0 0 0 1 0 0 0 0 0 0 0
0 0 0 0 0 0 0 1 0 0 0 0 0 0
0 0 0 0 0 0 0 0 1 0 0 0 0 0
1/ 4 1/ 4 1/ 4 0 0 0 0 0 0 0 0 1/ 4 0 0
0 1/ 4 0 1/ 4 0 0 0 0 0 0 0 1/ 4 1/ 4 0
0 0 0 0 1/ 4 0 0 0 0 1/ 4 1/ 4 0 0 1/ 4
0 0 0 0 0 1
P
/ 4 1/ 4 1/ 4 0 0 1/ 4 0 0 1/ 4
0 0 0 0 0 0 1/ 4 0 1/ 4 0 0 1/ 4 1/ 4 0
1/ 4 1/ 4 1/ 4 0 0 0 0 0 0
0 1/ 4 0 1/ 4 0 0 0 0 0
0 0 0 0 1/ 4 0 0 0 0
0 0 0 0 0 1/ 4 1/ 4 1/ 4 0
0 0 0 0 0 0 1/ 4 0 1/ 4
R
,
0 0 1/ 4 0 0
0 0 1/ 4 1/ 4 0
1/ 4 1/ 4 0 0 1/ 4
0 1/ 4 0 0 1/ 4
0 0 1/ 4 1/ 4 0
Q
1
0.2697 0.2921 0.2697 0.0225 0.0787 0.0112 0.0337 0.0112 0.0225
0.0225 0.3160 0.0225 0.2935 0.0899 0.0843 0.1278 0.0843 0.0435
0.0787 0.1685 0.0787 0.0899 0.3146 0.0449 0.1348 0.0449 0.0899
0.0112 0.0955 0.0112 0.0843 0.044
B I Q R
9 0.2921 0.3764 0.2921 0.0843
0.0225 0.0660 0.0225 0.0435 0.0899 0.0843 0.3778 0.0843 0.2935
1 2 4 5 8 9 12 13 14f f f f f f f f f f 111101010T T
B ,
Shortest paths through a reinforced random walk 21
3
6
7
10
11
f 0.8764
f 0.8230
f ff 0.5056
f 0.7865
f 0.3230
D BB
.
0.823
0.787
0.876
0.506
0.323
Markov chains x
2.5.4 Summary
In order to find the approximate solution to a Dirichlet problem in two dimensions, we
use two methods — Monte Carlo method in section 2.2 and relaxation method in section
2.3. In order to find an exact solution to two dimensional Dirichlet problem, we use the
method of linear equations in section 2.4 and method of Markov chains which may be
viewed as a more complicated version of the method of linear equations.
Since we consider the case in our example with boundary values 1 or 0, the expected
final payoff is the probability p x that the walker ends up at a boundary point with value
1. In addition, we have proved that p x v x in section 2.1.4. All the above four
alternative methods can determine the voltage and current in our networks.
2.6 Random walks on more general networks
So far, the networks we studied are special networks since they have unit resistors. In
this chapter we will introduce general resistor networks, and consider what it means to
carry out a random walk on such general network. In the random walk model, we will study
our electric network in terms of the random walkers, ants, performing a random walk on a
graph. We sometimes refer to the random walkers as “ants” or “particles” in electric
network.
2.6.1 Random walks on a graph[12]
Given a graph and a starting point, we select a neighbor of it at random, and move it to
this neighbor; then we select a neighbor of this new point at random, and move to it, etc.
The random sequence of points selected this way is a random walk on the graph.
There is not much difference between the theory of random walks on graphs and the
theory of finite Markov chains; every Markov chain can be viewed as random walk on a
directed graph, if we allow weighted edges. Similarly, time-reversible Markov chains can
be viewed as random walks on undirected graphs.
Shortest paths through a reinforced random walk 22
Random walks on more general, but finite graphs have received much attention. The
theory of random walks is very closely related to a number of other branches of graph
theory. Basic properties of a random walk are determined by the spectrum of the graph, and
also by electrical resistance of the electric network naturally associated with graphs.
2.6.2 Some language from graph theory[17]
A graph[14][16] is a finite collection of nodes with pairs of nodes connected by edges. It
is convenient to label the nodes using the integers. If it is possible to go between any two
nodes by moving along the edges, the graph can be called connected.
Let be a connected graph, where is the set of nodes (or vertices) and
is the set of edges. We assign to each edge
,G V E V
E xy a resistance xyr . The conductance of an
edge xy is 1/xyC
P
xyr . We define a random walk on to be a Markov chain with
transition matrix given by
G
xyxy
x
CP
C , where x xyC C
y .
2.6.3 Probabilistic interpretation in electrical network
Assume that a network of resistors are assigned to the edges of a connected graph. A
one-volt battery is applied across two points and , establishing a voltage and
. In this section, we will find the voltages
a b 1av
0bv xv and the currents xyi in the circuit. We
use the following probabilistic interpretations of electrical network theory.
a. The probabilistic interpretation of voltage:
As above we interpret the voltage as a “hitting probability”. If two points x and
are connected by a resistance of magnitude
y
xyr
, then by Ohm’s law, the current through the
resistor equals /xy x y xy xi v v r v v y Cxy . Note that we have xyi iyx . By
Kirchhoff’s Current Laws, the total current flowing into any point other than or is 0.
So that for
a b
0 ,xyy
i x a b . This will be true if x yy
v v C 0xy . And this equation
can also be written as /x xy y xyy y
v C v C x xy x yy
C C v xy yy
P vv . Thus the
voltages have the property that x xyy
v yP v for ,x a b . It means that voltage xv is
harmonic at all points except points and b . a
Shortest paths through a reinforced random walk 23
Let xp be the hitting probability that the walker starting at x will reach point
before point . Then
a
b xp is also harmonic at all points ,x a b . In addition, and v p
have the same boundary values and 1a av p 0b bv p . If we make and b
absorbing states, can be modified as an absorbing Markov chain
a
P P . And and v p
are both solutions to the Dirichlet problem for the Markov chain with the same boundary
values. Thus, . In conclusion, the voltage v p xv at any point x represents the
probability that a walker starting from x will return to before reaching b when a
unit voltage is applied between a and b , mak 1a
a
ing v and 0bv .
We could assume an arbitrage voltage instead of a unit voltage between points
and . We can replace the “hitting probability”
av a
b xp by the expected payoff in the game in
which the walker starting at x , is paid if is reached before . av a b
b. Probabilistic interpretation of current:
We first look at the process of electrical conduction before we interpret the current①:
positively charged particles enter the network at a source and walk randomly from
point to point until they reach a sink , where they leave the network. We hypothesize
that the current
a
b
xyi is proportional to the expected number of movements along the edge
from x to , where movements from to y y x are counted as negative.
Let xu be the expected number of visits to state x before reaching the sink . Then b
x y yu xy
u P for ,x a b and . We will have 0bu /x y xy x yy
u u P C C since
x xy xC P y yC P . It means /x x xCv u is harmonic for ,x a b , where xu can be
understood as “charge” and xC as “capacitance” in physics. We also have and
.
0bv
/a av u aC y x xy y xyxxy x yi v v xy
uC xy x
x y y
u u CC u
C C
xyP y yxu P
x
C u
C C represents
the current flows from x to y . Here, x xyu P is the expected number of times the walker
will go from x to y and is the expected number of times he/she will go from y yxu P y
Shortest paths through a reinforced random walk 24
to x . In this case, /x xv u Cx solves a discrete Neumann problem, that is,
. Thus x y xyy
v v C b
e
1
1
0
if x a
if x
otherwis
x v is harmonic on \ ,D V a b , i. e. outside
sources and sinks. Note that the uniqueness principle in this case holds “up to a constant”.
In conclusion, if there is a unit current flows into point and flows out of point ,
the current
a b
xyi is equal to the expected net number of times the walker, starting at and
wandering until he/she arrives at , will pass along the edge from
a
b to . yx
Note ①: Actually, the current interpretation makes most sense in relation to a Neumann
problem, instead of the Dirichlet problem so far we discussed.
c. Effective resistance and the escape probability:
a axxi iImpose a voltage at and av v a 0bv . A current will flow into the
circuit from the outside source. The current depends on the overall resistance. So we define
the effective resistance between and by effr a /a ar v ib eff . The effective
conductance is . When 1/f eff aC r /ef ai v 1av , we will have eff aC i , which means the
effective conductance equals the total current flowing into . This current is
, where
ai
y aC
a
ay y ay
P v C p
a a yi v v v a aC / 1y av Cay ay y
C C esc escp is
the probability, starting at , that the walk reaches before returning to . Hence,
, so that
a b a
ef a escC C pf eff
esca
Cp
C
a
s
.
2.6.4 Random walk model
In an electrical network we have studied above, the positively charged particles enter
the network at point and wander around until they arrived at point , where they leave
the network. The walker will keep on walking if he/she returns to before reaching b .
b
a
We can picture the random walker in a electrical network as ants foraging. Ants prefer
to take paths with short length and keep on walking until it reaches the target. Ants enter a
network at a source and they want to find the path through a network to a sink . We t
Shortest paths through a reinforced random walk 25
constantly inject ants into the source and take the ants away from the sink. The ants walk
independently throughout the network until they find the sink where they disappear.
In order to make the notations clear, I make a comparison table below. The table 2-6-1
shows the notations we will use in our ants’ random walk model in this section and the
following reinforced random walk model in section 3.2.
Table 2-6-1 Comparison between two random walkers
Random
walker
Interpretation
Electrical particles Ants
Voltage
A unit voltage applied between a
and b : 1av and 0bv
xv : The probability that particle
starting from x will return to a
before reaching b .
/x xv u C x . (Dual interpretation)
xu : Charge
xC : Capacitance
For the source s , 1sv and
for the sink t , 0tv .
iv : The probability that an ant
at node i arrives back at the
source s before arriving at
the sink t .
What the voltage tells us is
how far an ant has to go to get
to the sink t .
/i iv u Ci
: The expected number of iu
visits to state before i
reaching sink . t
Current
xyi : The expected net number of
times that a particle, starting at a and walking until it reaches b , will move along the edge from x
to y .
xy xy x yi C v v . (Dual
interpretation)
ijI : The expected rate at
which ants pass from i jto
minus the expected rate ants
pass from j to i .
What the current tells us is how many ants are moving in a particular direction.
Shortest paths through a reinforced random walk 26
Resistance
The effective resistance between
a and is b /eff a ar v i .
The effective conductance is
1/ /eff eff a aC r i v .
Each edge between and i j has
a resistance proportional to the
length of the connection . ij
Probability
Escape probability is the probability,
starting at , that the walker reaches a
b before returning to . a
1/
1/a
eff effesc
a ax E
C rp
C r
x
,
where is the set of nodes which aE
are connected to node . a
When an ant arrives at a node , i
the probability that it moves next
to node j , is
1/
1/i
ij
ikk E
P go fromi j
,
where is the set of nodes iE
which are connected to . i
We make some assumptions for the model: 1) In the graph, each edge between two
nodes and i j has a resistance, which is proportional to the length of the connection
edge (In the random walk model, we use to denote resistance); 2) Ants prefer to
travel down the edges with lower resistance; 3) Ants are constantly injected into the source,
and they are taken away once reach the sink. Thus, source is always occupied and sink
is always empty.
ij ij
s
t
When an ant arrives at a node , the probability that it moves next to node i j , is
1/
1/i
ij
ikk E
P go fromi j
(3.1)
where is the set of nodes which are connected to . An alternative but macroscopic
equivalent description: If we choose one edge of the network at each time step, the
probability that choosing edge is given as:
iE i
ij
1/
1/ij
ee E
(3.2)
where is the set of all edges in the graph. E
Shortest paths through a reinforced random walk 27
We swap the ants on the nodes of the chosen edge. If is empty and i j is occupied
then j becomes empty and becomes occupied. If both nodes are occupied or empty
then their status is unaltered.
i
Let’s construct a graph (figure 2-6-2) and let a number of ants move on this graph:
Figure 2-6-2: Flow of stacked ants between edge in a network ij
a) The node can be seen as a building which has n floors (levels). Each floor
(level) can only contain one ant. Each node contains a maximum of n ants;
b) We denote the number of ants on node i and j as iN and jN . Let the
ants place on top of each other from the first level, then /iN n can be seen as
the level to which the site i is filled, i.e. / 6 / 8iN n
in figure 2-6-2;
c) We pick one edge ij at random on each time step as in equation (3.2), then
pick a level at random and swap any of the ants at that level;
d) The probability that there is a change in the levels of ants on edge ij is
/i jN N n . If both nodes are occupied or empty on the level we pick, no flow
exists on that level;
Thus, the probability that there is a flow of ants between and i j on a given time
step is proportional to
i
ijij
N NI
j (3.3)
Shortest paths through a reinforced random walk 28
where iN N j can be think of as the voltage or potential difference. The current is equal
to the potential difference divided by the length of path of fixed resistance per unit length.
In this random walk model, the ants can not be said to be solving a problem because
they are not aware of the potential difference and current they are generating. However,
based on this model, we can make some modification to make it solve a problem. If we
make our ants respond to the current they generate, we can allow them to iteratively
improve the path they take through the network. The ants which take local information
about the flow of and number of ants around them can build solutions to problems such as
finding the shortest path between a source and a sink. We will describe the advanced model
— reinforced random walk model in section 3.2.
3. Shortest path problems
In section 2.6.4, we described random walk model where ants moving around the
network do not modify their path. But we can make a modification based on this model by
allowing pheromone conductivity. We call this new revised model current reinforced
random walk model. In new model, ants can update the conductivity in response to the
current they generate so that we can allow them to iteratively improve the path they take
through the network. The path that ants reinforce most strongly is the shortest path.
In this chapter, I first study Ant Colony Optimization. Although ant algorithms and
ACO can be applied to solve combinatorial optimization problems such as travelling
salesman problem, they can not solve our shortest path problem. In the second and third
part of this chapter, we will study current reinforced random walk on the graph. In chapter 4,
we will introduce Physarum solver which is the “mean-field” version of current reinforced
random walk.
3.1 Ant colony optimization
Ants have inspired a number of methods and techniques. The most studied and the
most successful among these techniques is the general purpose optimization technique
known as ant colony optimization (ACO) [6]. ACO is inspired by the foraging behavior of
ant colonies. The ants lay pheromone on the path they have taken in order to mark some
favorable path that should be followed by members coming after them. ACO exploits a
similar mechanism for solving optimization problems.
Shortest paths through a reinforced random walk 29
3.1.1 Ants’ foraging behavior and optimization
The visual perspective faculty of many ant species is only rudimentarily developed and
there are ant species that are completely blind. One of the problem studied was to
understand how these almost blind ants can establish the shortest paths leading from the
colony to the food sources and come back. An early research on ants’ behavior gave us an
important insight that most of the communication among individuals, or between
individuals and the environment, is based on the use of chemicals called pheromones
produced by the ants. The trail pheromone is a specific type of pheromone that plays a
particularly important role in social life of ants. A moving ant lays this kind of pheromone
on the ground thus marking the paths from food to the nest by a trail of this substance. By
sensing pheromone trails foragers can follow the path to food discovered by other ants. This
collective trail left by other is the inspiring source of ACO.
3.1.2 Double bridge experiments
The foraging behavior of ants is based on indirect communication mediated by
pheromones[3]. Ants deposit pheromones on the ground so that they form this way a
pheromone trail between food sources and the nest. Other ants can detect the pheromone
and tend to follow, probabilistically, paths where pheromone concentration is higher. Ants
are able to transport food to the nest in an effective way through this mechanism.
The pheromone laying and following behavior of ants has been investigated. One well
known experiment named “double bridge experiment[1] [6]”, using a double bridge
connecting a nest of a colony of ants to a food source. Experiments were ran by varying the
ration between the length of the two branches of the double bridge, where
was the length of the longer branch and
/lr s l
s the length of the shorter one.
In the first experiment the bridge had two branches of equal length (figure 3-1-1).
Initially, there is no pheromone on two branches so that the ants do not have a preference
and they randomly select any of the branches. Due to random fluctuation, one of the two
bridges presents a stronger pheromone concentration than the other after some time,
therefore, attracts more ants. This will lead to a further amount of pheromone on that bridge
so that the whole colony converges towards the use of the same single path after some time.
1r
In the second experiment, the length ration was set to 2r (figure 3-1-2), so that the
long bridge was twice as long as the short one. The stochastic fluctuations in the initial
choice of a bridge are much reduced. This mechanism plays an important role: the ants
choosing the short bridge by chance are the first to reach the nest. The short bridge receives
Shortest paths through a reinforced random walk 30
pheromone earlier than the long one so that this fact increases the probability that further
ants select it rather than the long one.
Figure 3-1-1: Branches have equal lengths Figure 3-1-2: Branches have different lengths
Pheromone evaporation is too slow which could favor the exploration of new paths:
the lifetime of pheromone is comparable to the duration of a trail, which means that the
pheromone evaporates too slowly to allow the ant colony to “forget” the suboptimal path to
which they converged so that the new and shorter one can be discovered and “learned”. [6]
3.1.3 ACO for the Travelling Salesman Problem
The TSP[3] [6][11][19] is the problem of a salesman who starting from his hometown,
wants to find a shortest tour that takes him through a given set of cities and then go back
home, visiting each city only once. The TSP can be represented by a complete weighted
graph with being the set of nodes representing the cities, and being
the set of edges representing connections between any two cities. The set fully connects
the components . The goal in the TSP is to find a minimum length Hamiltonian circuit of
the graph. Note that a Hamiltonian circuit is a closed path visiting each of the
,G V E
V
V E
E
n V
nodes of only once. G
ACO is an iterative algorithm. We simulate a number of ants moving on a graph. The
pheromone trail[3] is associated with every edge so that it can be detected and modified by
ants. Each ant builds a solution by walking from node to node on the graph. The ant will
not visit any node that it has visited. At each step of solution construction, an ant selects the
following node according to a stochastic mechanism based on the pheromone. In figure
3-1-3, the ant stands in node , the following node will be stochastically selected among
the unvisited ones. If
i
j has not been visited before, it can be selected with a probability
that is proportional to the pheromone associated with edge ,i j . We modify the pheromone
values at the end of an iteration based on the quality of solution constructed by ants so that
they could construct solutions similar to the best ones previously constructed.
Shortest paths through a reinforced random walk 31
Figure 3-1-3: An ant in city chooses the next city to visit as a function of the pheromone values iijD and of the
heuristic values on the edges connecting city to the cities 1/ ij i j the ant has not visited yet.
3.1.4 The Ant Colony Optimization Metaheuristic
A metaheuristic[3] is a general algorithm that could be used to different optimization
problems with relatively few modifications to make them adapted to a specific problem.
ACO is a metaheuristic in which a colony of ants cooperate in finding good solutions to
discrete optimization problems. Ant builds a solution by walking on the fully connected
construction graph ,CG V E . The set of all possible solution components is denoted by
. The graph can be obtained from in two ways: components may be represented by
nodes or by edges. Ants move from node to node along the edges and lay a certain amount
of pheromone on the components. The pheromone amount
C C
D depends on the quality of
the solution found by ants. The following ants use the pheromone information as a guide.
The ACO metaheuristic[3] is shown in Algorithm below. After initialization, the
metaheuristic iterates over three procedures. At each iteration, a number of solutions are
constructed by the ants; then these solutions are improved through a local search (optional);
finally the pheromone is updated.
Algorithm[3] [6]: The ACO Metaheuristic
Set parameters, initialize pheromone trails
while (termination condition not met)
Construct Ant Solutions
Apply Local Search % optional
Update Pheromones
End
(1) Construct Ant Solutions: The process of constructing ant solutions can be
considered as a walk on . Ants move by applying a stochastic local decision that ,CG V E
Shortest paths through a reinforced random walk 32
makes use of pheromone trails and heuristic information so that they build solutions to
optimization problem. Once an ant has built a solution, it evaluates the solution that will be
used in the Update Pheromones procedure to decide how much pheromone to lay.
(2) Apply Local Search: Once solutions have been constructed, it is common to
improve the solutions through a local search before updating the pheromone. However, this
procedure is optional.
(3) Update Pheromones: Updating the pheromone aims at increasing the pheromone
values associated with good solutions and decreasing those associated with bad solutions.[6]
This process can be achieved by two steps: (i) Increasing the pheromone levels: The deposit
of new pheromone increases the probability that those components which produced a good
solution will be used again by future ants. (ii) Decreasing pheromone values by pheromone
evaporation: Evaporation is a useful form of forgetting. It avoids a too rapid convergence of
the algorithm toward a suboptimal region.
3.1.5 Ant System (AS) [3][20][21]
On one tour, each ant leaves pheromone quantity given by , where is a
constant, is the length of its tour. So that more pheromone left on shorter tours and less
pheromone left on longer tours per unit length. In addition, the existing trails are reduced
by a factor named evaporation rate
/ kQ Q
k
before new pheromone is laid since pheromone
evaporation. At each iteration, the values of pheromone are updated by all the ants that
have built a solution itself
m
[1] [5]. The updating of the trail level associated with the edge
connecting to
ijD
i j , is updated as follows:
1
1m
kij ij ij
k
D D
D (3.4)
where
/ ,
0 ,k kij
Q if ant k used edge i j in its tourD
otherwise
, (3.5)
where
t iteration counter
0,1 evaporation rate, parameter of the reduction of ijD
m number of ants
Shortest paths through a reinforced random walk 33
kijD increase of trail level (the quantity of pheromone laid) on edge
caused by ant
,i j
k
ijD total increase of pheromone trail on edge ,i j
Q quantity of pheromone per tour laid by an ant, is a constant Q
k tour length of ant k
In the construction of a solution, ants select the following city to be visited through a
stochastic mechanism[11]. The higher the probability that a city is selected is, the more
intense the trail level leading to this city is[6]. The adapted behavior is based on the trail
levels and is regulated by a parameter ; the greedy behavior is based on the visibility and
is regulated by a parameter . When ant stays in city and it has so far constructed
the partial solution , the probability of going to city
k i
ps j is given by:
,
0 ,
pil
ij ij pijk
il ilc N sij
Dif c N s
Dp
otherwise
(3.6)
where
1
ijij
(3.7)
intensity of pheromone trail between cities and ijD i j
from city i visibility of city jij
parameter to regulate the influence of ijD
parameter to regulate the influence of ij
pN s set of cities that have not been visited yet; the set of feasible components
distance between cities and ij i j
Th ts have completed a tour. [5] The set of
citie ne
is selection process is repeated until all an
s to be visited is reduced by one city in each of iteration t . By keeping on visiting, o
city is left finally and it is selected with probability 1ijp . For each ant the length of the
Shortest paths through a reinforced random walk 34
tour generated is calculated and the best tour found so far is updated. [3] The next iteration
1t can be started based on updated pheromone levels.
Algorithm: The Ant System
Initialization
for 1t : number of iterations
for 1k : m
repeat until ant k has completed a tour
select the c ty i j to be visited next
with probability ij p given by equation (3.6)
calculate k of the tour generated by ant k
update ijD on l edges according to equation ( .4al 3 )
End
3.1.6 Simple ant colony optimization (S-ACO)
lks down an edge with probability According to equation (3.6), ant at node i wa ij
1/,
ij ijD
1/ij ijk
D
wher omone in the ed
(3.8)
e D is the concentration of pher ge connecting i to j , is the ij ij
length of the edge ij . Here we consider the simplest case 1 where the ants
employ the same resistance rule as in equation (3.1), i.e. the ity that an ant at n
i travelling to
probabil ode
j is
/
/ij ij
ij iji
D
D
(3.9)
rease in pr D t in the ant system is assumed to inc oportion to the number of ant on that ij
edge and to decrease with evaporation rate . The pheromone update rule is
1ij ij ij jiD t t t t t t D (3.10)
next ant travels from to where =1ij t if the i j . The probabi
determined by equation (3.9).
lity that an ant moves is
Shortest paths through a reinforced random walk 35
Though ants algorithms and Ant colony ptim zation can be applied to solve the
solve our shortest path problem
O i
combinatorial optimization problems like the travelling salesman problem, they can not
. It is necessary to introduce reinforced random walk model
whic
3.2.1 Description of current reinforced random walks
nt as in the description of ant stacks on each
node in section 2.6.4. Now, we allow the ants to modify the conductivity of the network
they
h will always converge to the optimal solution of the shortest path problem.
3.2 Current reinforced random walks
We keep the description of ant moveme
move in. Let ijD t be the conductivity of an edge at time t .
Figure 3-2-1: Ants are allowed to modify pheromone conductivity
The probability that an ant is moved from to i j is proportional to
i j ij
ij
N N D t
. Hence, an edge is chosen at each e step with probability proportional
to
tim
| |i j ij
ij
N N D t
.
from
We assume that every edge is independent and that the time until the next ant moves
to i j is exponentially distributed with rate:
i j ij
ijij
N N D t
I t (3.12)
Shortest paths through a reinforced random walk 36
The input and output currents are fixed to constants 0sI and 0tI . The time interval
t until the next edge of the network is updated is taken from an exponential distribution
with the mean value equals
1 (3.13)
i j e
e Ee
N N D t
We update the conductivity in response to current between and i j so that on the
chosen edge
ij :
1ij ij ijD t t t D t t (3.14)
where is , here, is the expected value of ij t q q ij t .
e exp v Th ected alue of ij t equals ij i j
ijE ij
D N Nq t q
.
This gives a “mean-field” equation as follows:
ij i j
ij i j
ijij
Nq t
D N ND q
dt t t
. (3.15)
On all other edges , we have
D N
ij ij ij ijij
dD D t t D tD
e 1eD t t t D t e . On each time step, the
conductivity decreases slightly, but the conductivity is increased if an ant crosses. This is a
curre m
3.2.2
nt reinforced rando walk.
I will implement the simulations based on Poisson process: Exponential distribution
and Poisson distribution.
Pheromone Rule and Ant Moving Rule
Figure 3-2-2
Shortest paths through a reinforced random walk 37
iN : Ant number
: Pheromone conductivity
: Resistance
: Chosen edge rate
To each path of the graph
ijD
ijl
ijq
,V E , we ij G associate a variable called
ph omone trails are read and written by the ants. Pheromone evaporation
can anism that avoids quick convergence of all the ants
toward a suboptimal path. The decrease in pheromone intensity favors the exploration of
di g the whole search process.
ach time step:
ijD
eromone trail. Pher
be seen as an exploration mech
fferent paths durin
Pheromone change at e
1 ,ij ij ijD t t t D t t 0 1
, 1 1 , 0D t D t t 1when t 1ij ij ij
1
0ij
Ant walks on this wayt
Ant walks on another way
Only one ant walks at each time step in this model.
Chosen edge rate:
0.001D ij
i j
ij ijij
N Nq D
ij
ij
q
qOne edge is chosen at each step with probability
j
. An ant moves from more ants
node to less ants node. Ant returns to start if it arrives at goal.
1, 1i i j j iN N N N N N j
1, 1i i j j iN N N N N N j
3.2.3 A Simple Explanation
Shortest paths through a reinforced random walk 38
Given:
1ij ijD
1 10N , 2 8N , 3 3N , 4 0N
Solution:
According to the edge choosing rate: i j
ij ijij
N Nq D
1 212 12
12
10 8 2N N
q D
1 313 13 10 3 7
N Nq D
13
2 424 24
24
8 0 8N N
q D
3 434 34
34
3 0 3N N
q D
2 323 23
23
8 3 5N N
q D
2 7 8 3 5 25ijq
5
25, 2
3
8 7
3 4
N
Edge 2-3 is chosen withN
2
4
8 7
10 11
N
N
8
25,Edge 2-4 is chosen with (node 4 is goal)
Shortest paths through a reinforced random walk 39
3.2.4 Exponential distributed simulation
ma ix shows the connected edges and lengths in the graph
% initial pheromone conductivity
tor f initial ant numbers on each node
% the expected value of
...G % the tr
...D
...N % vec o
...q ij
...inRate % the inflow rate in the source
outRat ...e % the outflow rate in the sink
% evaporation rate ...
……
— for
pick e
1:k S % iterate from 1 to S
ij i j
ij
Ddge at random with probabilistic rate
t N N
ck t ion 1
expij i j
ij
D t N N
pi from exponential distribut
% for each flip, transport one ant along edge ij from high to low
— if , an ant moves from to i jN N i j
1i iN t t N t
else
1j jN t t N t
ijD t t D t t ij ij
1i iN t t N t
1j jN t t N t
ji ji ijD t t D t t
—
,
— end
end
1 ,ij ijD t t t i j D t
t t t
Shortest paths through a reinforced random walk 40
— end
mentation in Matlab
In Monte Carlo methods, repeated experiments with stochastic transition components
are run to collect data about the statistics of interest. Based on Monte Carlo methods, we
will firs plement the Exponential distributed simulation in an simple graph, then in a
much more complicated graph. Matlab code could be found in the appendix 3-2(A) and
e graph with 4 nodes in figure 3-2-2, see the code in
App
3.2.5 Imple
t im
3-2(B).
(A) First, we implement on th
endix 3-2(A).
Figure 3-2-2
Show connection and length in a matrix:
1) Outcome:
0 1 6 0
1 0 2 4
6 2 0 1
0 4 1 0
G
2) We plot the number of ants on each node:
Shortest paths through a reinforced random walk 41
0 0.5 1 1.5 2 2.5
x 104
0
5
10
15
20
25
30
35
t
n
number of ant on each node,(number of node is4)
The deep blue line represents the number of ants at node 1; the green line represents
the n umber of ants at node 2; the red line represents the number of ants at node 3; the light
blue represents the number of ants at node 4. The number of ants on each node tends to be
equilibrium after iterations. It fluctuates near a value in an interval.
3) We plot the total amount of pheromone:
0 0.5 1 1.5 2 2.5
x 104
0
5
10
15
20
25
30
t
tota
lD
total amount of pheromone
Shortest paths through a reinforced random walk 42
The blue line represents the total quantity of pheromone. It tends to be equilibrium
after iterations. It fluctuates near a value in an interval.
4) Conclusion:
In the outcome matrix of conductivity , we find that elements and
(the elements in the red boxes) reinforce the pheromone. So there is
, as the green path in the graph below.
D 12D ,
The length of the shortest path is
23D
the shortest path
34D
1 2 3 4
12 23 34 1 2 1 4 .
(B) Now we implement on a much complicated graph with 10 nodes in figure 3-2-3.
See the code in Appendix 3-2(B).
Figure 3-2-3
Show connection and length in a matrix:
Shortest paths through a reinforced random walk 43
0 2 0 0 0 0 1 0 0 0
2
0 2 0 3 0 0 0 0 0
0 2 0 4 0 0 0 0 0 0
0 0 4 0 0 1 0 0 0 6
0 3 0 0 0 1 5 0 0 0
0 0 0 1 1 0 0 0 1 2
1 0 0 0 5 0 0 4 0 0
0 0 0 0 0 0 4 0 3 0
0 0 0 0 0 1 0 3 0 2
0 0 0 6 0 2 0 0 2 0
G
1) Outcome:
2) We plot the number of ants on each node:
Th axis represents the iteration time; the y e x axis represents the number of ants.
In the figure below, the traces with different colors represent the numbers of ants on
Shortest paths through a reinforced random walk 44
different nodes. The number of ants on each node tends to be equilibrium after a large
number of iterations. It fluctuates near a value in an interval.
0 0.5 1 1.5 2 2.5
x 104
0
10
20
30
40
50
60
t
n
number of ant on each node,(number of node is10)
3) We plot the total amount of pheromone:
0 0.5 1 1.5 2 2.5
x 104
0
5
10
15
20
25
40
30
35
t
tota
lD
total amount of pheromone
Shortest paths through a reinforced random walk 45
The blue line represents the total quantity of pheromone. It tends to be equilibrium
after iterations. It fluctuates near a value in an interval.
4) Conclusion:
In the outcome matrix of conductivity , we find that elements and
reinforce the pheromone. The shortest path is ath
raph below. The length of the shortest path is
D 12D ,
10
6,10
25D ,
, as the green p
2 3
56D
1
6,10D
in the g
1 2 5 6
12 25 56 2 8 .
Shortest paths through a reinforced random walk 46
3.3
istributed simulation we have studied in section 3.2.4 and 3.2.5, we
picked up which follows exponential distribution[14]. In order to build a new algorithm,
we ma ber of ants going on the edge connecting node and
An efficient algorithm based on Poisson distribution
In exponential d
t
ke the num i j during a fixed
time interval follow Poisson distribution. It is desirable to know how these two algorithms
come and what the difference is between them, even it is interesting to study which one is
better. First, we introduce some knowledge of Poisson process in section 3.3.1. Then in
section 3.3.2, we will introduce event-driven approaches that gave inspiration in our two
algorithms. Finally, in the last two sections, we show our algorithm of Poisson simulation
and the implementation of it.
3.3.1 Some knowledge of the Poisson process[14]
The Poisson process is a collection : 0N t t
e t
of random variables, where
is the number of events that have occurred up to tim The number of events between
time and time is given as
N t
.
a b N a
e process. Consequences of
is a Poisson dist
N b
tim
N t
and has a Poisson distribution. The
Poisson process is a continuous- this definition include: i)
The probability di tion of ribution; ii) The probability
distribution of the waiting time until the next occurrence is an exponential distribution.
The exponential distribution is a family of continuous probability distributions. It
describes the time between events in a Poisson process, i.e. a process in which events occur
continuously and independently at a constant average rate. It describes the time for a
continuous process to change state. In the exponential distributed simulation of our CRRW
model, the edge is chosen by ants independe tic rate.
sen to update is taken from an
exponential distribution.
The Poisson distribution is a discrete probability distribution that expresses the
probability of a number of events occurring in a fixed period of time if these events occur
with a known average rate and independently of the time since the last event. The Poisson
distribution can also be used for the number of events in other specified intervals such as
distance, area or volume. One example that may be modeled as a Poisson distribution is the
number of phone calls arriving at a call centre per minute. Of course, in our CRRW model,
during a fixed time interval, the number of ants walking along the edge connecting to
stribu
ntly and with a probabilis The time
interval t until the next edge of the network is cho
i j
Shortest paths through a reinforced random walk 47
Pois models as a Poisson distribution. The notation of Poisson distribution is , where
the
If
is a positive real number, equal to the expected number of occurr r during
g
the number of arrivals in a given time interval
ences that occu
iven interval.
0, t follows the Poisson distribution,
twith mean , then the lengths of the inter-arrival times follow the exponential distribution,
with mean 1/ .
3.3.2 Event-driven approaches
Event-driven methods require explicit consideration of events. We need to consider
numbers of events that can occur, each causing the numbers in the relative classes to
increase or decrease by one, i.e. in the network ant chooses the edge at random with rate
| |i j ijN N D t. Result: 1i iN N and N
ij
j j i j
the next event, based on the cumu The time and numbers
to iterate the model through time (box 3-3-1).
Event-driven approaches: Gillespie’s Direct Algorithm
( if
( if This schem
lative rates of all possible events.
in each class are then updated according to which event is chosen.
1j jN
one updates.
i jN N
e estim
) or
ates the tim
This process is repeated
1i iN N
e until 1N N N N ); pheromand
[28]
1. Label all possible events as 1,..., nE E .
2. For each event determine the rate at which it occurs, 1,..., nR R .
3. The rate at which any event occurs is 1
n
total mmR R
.
4. The time until the next event is 1
1log
total
t randR
.
5. Generate a new random number . Set 2 2 totalrand P rand R .
6. Event p occurs if 1
1 1
p p
m mm m
R P R
7. The time is updated now, t t t , and event p is performed.
8. Return to Step 2.
Box 3-3-1: Gillespie’s direct algorithm
Shortest paths through a reinforced random walk 48
However, this method has low simulation times for large population ter
mber of different event types. We will use the simple SIS model to explain the algorithm
(in box 3-3-2).
sizes. Af
modification, a new algorithm comes up that is significantly faster when there are a large
nu
ve AEvent-driven approaches: Effecti lgorithm[28]
1. Let the time interval t , be “small” and fixed.
2. Let RM t TM t and represent the number of transmission and recovery events by
3. D
time t .
efine ,i i iM M t t M t i T R , then we will have
1| ,T
XYP M X Y t o t
N
, 1|RP M Y Y t o t .
t , the incr i4. For small ements M are a y Poiss hat: pproximatel on, so t
T
XYM Poisson
N
t
, RM Poisson Y t .
5. Then we can update:
T RX t t X t M M , T RY t t Y t M M .
6. Update the tim
. Return to step 4.
e:
t t t
Box 3-3-2: an lgo effective a rithm
From b tion of a orithm based
on Poisson distribution is much more efficient than that based on exponential distribution.
So that I wo ted algorithm of our reinforced random
walk model and implement the simulation.
3.3.3 Poisso
% the matrix shows the connected edges and lengths in the graph
ity
% vector of initial number of ants on each node
ox 3-3-1 and box 3-3-2, we get to know that the simula lg
uld like to introduce the Poisson distribu
n distributed simulation
...G
...D % initial pheromone conductiv
...N
... a , it should be less an zero
article, it is a fixed parameter which can be changed based
% on the sim
% ev poration rate th
...t
...t % fix the time interval
...Q % charge per p
ulation
Shortest paths through a reinforced random walk 49
—fo
outflow, ants go out of the sink
% for each flip, tra
r 1:ts S
...sN % inflow, ants come into the source
...tN %
nsport ants along edge ij from high to low
—for each ij
( , , ) , ,ijij
DP o
ij
P i t N ts i N ts j sign N ts i N ts jG
hat if )
% the number of ants going from node i to j during a fixed time interval follow
t % update the pheromone conductivity
—end
(Note t 0, 0ij ijG then P
t
% Poisson distribution
1ij ij ijD t D Q P
—for 1:k length G
% update the numbers of ants after all the ants’ currents are known
, ,: :,ts k sum P k sum P k 1,N ts k N
—end
—en
ulation in an simple graph, then in a
ould be found in the appendix 3-3 (C) and 3-3
(D). W
than the im ulation.
(C) First we im
Appendix 3-3(C).
t t t
d
3.3.4 Implementation in Matlab
We first implement the Poisson distributed sim
much more complicated graph. Matlab code c
e find that the implementation of Poisson distributed simulation is more efficient
plementation of exponential distributed sim
plement on a graph with 4 nodes in figure 3-2-2. See the code in
Shortest paths through a reinforced random walk 50
Figure 3-2-2
Show connection and length in a matrix:
1) Outcome:
0 1 6 0
1 0 2 4
0 4 1 0
G
6 2 0 1
2) W ber of ants on each node:
e
e plot the num
axis represents the iteration time; the y x Th axis represents the number of ants.
umber of ants at node 1; the green line represents the
num ue
The deep blue line represents the n
ber of ants at node 2; the red line represents the number of ants at node 3; the light bl
represents the number of ants at node 4. The number of ants on each node tends to be
equilibrium after iterations. It fluctuates near a value in an interval.
Shortest paths through a reinforced random walk 51
0 0.5 1 1.5 2 2.5 3 3.5
x 104
0
1000
2000
3000
4000
10000
5000
6000
7000
8000
9000
3) We plot the total amount of pheromone:
0 0.5 1 1.5 2 2.5 3
x 104
0
1
2
3
4
10
5
6
7
8
9
Shortest paths through a reinforced random walk 52
The blue line represents the total quantity of pheromone. It tends to be equilibrium after iterations.
4) Conclusion:
In the outcome matrix of conductivity , we find that elements and
reinforce the pheromone. So the shortest path is
graph below. The length of the shortest path is
D 12D ,
1 4
23D
, as the green path in the
34D
1 2 3 4
12 23 34 1 2 . It has the same
outcome as in exponential distributed simulation (A).
(D) We implement on a much complicated graph with 10 nodes in figure 3-2-3. See
the code in Appendix 3-3(C).
Figure 3-2-3
Show connection and length in a matrix:
Shortest paths through a reinforced random walk 53
0 2 0 0 0 0 1 0 0 0 2 0 2 0 3 0 0 0 0 0
0 2 0 4 0 0 0 0 0 0
0 0 4 0 0 1 0 0 0 6
0 3 0 0 0 1 5 0 0 0
0 0 0 1 1 0 0 0 1 2
1 0 0 0 5 0 0 4 0 0
0 0 0 0 0 0 4 0 3 0
0
2 0 0 2 0
G
0 0 0 0 1 0 3 0 2
0 0 0 6 0
1) Outcome:
2) We plot the number of ants on each node:
y The x axis represents the iteration time; the axis represents the number of ants.
In the figure below, the traces with different colors represent the numbers of ants on
different nodes. The number of ants on each node tends to be equilibrium after a large
Shortest paths through a reinforced random walk 54
number of iterations. It fluctuates near a value in an interval.
0 0.5 1 1.5 2 2.5 3 3.5
x 104
0
2000
4000
16000
6000
8000
10000
12000
14000
3) We plot the total amount of pheromone:
30
0 0.5 1 1.5 2 2.5 3
x 104
0
5
10
15
20
25
Shortest paths through a reinforced random walk 55
The blue line represents the total quantity of pheromone. It tends to be equilibrium after iterations. 4) Conclusion:
In the outcome matrix of conductivity , we find that elements and
reinforce the pheromone. So the shortest path is
test path is
D
2 3
12D ,
10
25D ,
.
56D
The length of 6,10D
the shor
1 2 5 6
812 25 56 6,10 1 2 . It has the same outcome as in
exponential distributed simulation (B).
We can find that the implementation of Poisson simulation is much more efficient than
that of exponential simulation. Among the four implementations, (C) is efficient than (A)
and (D) is efficient than (B).
Shortest paths through a reinforced random walk 56
4. Physarum solver
Some organisms grow in the form of an interconnected network as part of their
foraging strategy to discover and exploit new resources. Such systems can adapt to their
environment continuously and balance the cost of producing an efficient network with the
consequences of limited failure in a competitive world. The slime mode Physarum
polycephalum is such an organism[27].
4.1 Introduction to Physarum
The Physarum Polycephalum[13][22] is a large, single-celled amoeboid organism[23][24]
with an intracellular structure consisting of a network of tubes[25], see the graph 4-1-1. This
organism forages for food resources in a woodland environment. During exploration, it
spreads with a relatively continuous foraging margin to maximize the area searched. The
large amoeboid plasmodium of Physarum is able to solve a maze and some other types of
geometrical puzzle, and is also able to connect multiple food locations via a smart network,
then successfully optimize survival tasks. The challenge is to extract a mathematical
algorithm for this natural computation. We could propose a mathematical model, and
consider how this primitive organism solves a shortest path finding problem[25] on the graph
by Physarum-like way[13][26].
4-1-1 Plasmodium
The Physarum solver is a simple, intuitive and highly localized mechanism for
solving shortest path problems which works as an adapting electrical network[17]. The
Physarum consists of a network of tubes. This network acts as an information highway
transporting chemical and physical signals and it also enables nutrients to circulate
throughout the organism. When food sources are presented to a starved plasmodium which
Shortest paths through a reinforced random walk 57
has spread over the whole surf
and concentrate on the food sources.
ace, parts of the organism are connected by only a few tubes
The path connecting these parts is the shortest path.
e
ze-solving by Physarum
d the Physarum solver[13] as a model of how Physarum
builds a minimal link between two food sources. We assume this slime mode Physarum
cons
Past experiments have shown two rules: first, open-ended tubes are likely to
disappear; second, the longer tube is likely to disappear when more tubes connect the sam
two food sources.
4.2 Mathematical modeling of ma
Tero et al. (2006) first describe
ists of a network of tubes. We illustrate this model using maze in Figure 4-2-1. Each
segment in the figure represents a section of tube.
In the graph, we make two nodes 1N and 2N be the food sources. Other nodes are
denoted as 3N , 4N , 5N and so on. There is also a rule: if there are more than one tubes
connect the same pair of nodes, we place the nodes in the center of the tubes.
4-2-1 Expression of the maze by graph( Figure from [13])
Ordinary nodes are indicated by squares and the nodes of the food sources are given by circles.
We have:
1) : the source 1N
2N : the sink
2) ijM : the tube section between nodes iN and jN
3) ijQ : the flux[26] go through the tube ijM
Shortest paths through a reinforced random walk 58
,ijij i j
ij
DQ p p
(4.1 )
: the length of the segment
where
ij ijM
p ip , j : the pressure at nodes and
4)
i jN N
ijI : the flow rate of protoplasm, it is given by Ohm’s law (equation (3.12)).
In equation (3.12), i j ij
ijij
n n D tI t
, note that we use ,i jn n here. in is the
number of ants at node iN and jn is the number of ants at nod . The e jN pressure ip is
proportional to n , i.e. iin
p .
5) D e ductivity of the pipe
i n
: th con
ij
| |
ij iji j
ij
ij ij
dD Dp ijp D
dt
Q D
(4.2)
This gives a deterministic ‘mean-field’ version of equation (3.14). We can replace
in equation (3.14) with its expected value, i.e. ij t ijQ , then obtain equation (4.2).
In equation (3.14), 1ij ij ijD t t t D t t
ijij
D t t D ij ijt tD t
t t ij ijdD
Dijdt dt
Here, we replace with its expected value ij t ij i j
ijij
D n nq dt q
.
ij i jijij
ij
D n ndDD q
d
□ We have
t
According to Kirchhoff’s Law, at each node j 1, 2j , the inflow and outflow must
be balanced as:
0 1,iji
j Q 2 (4.3)
Shortest paths through a reinforced random walk 59
for t The nodes of foo
network. At the food sources, equation (4.3) is modified by a source (or sink) term.
he source 1N and the sink 2N d sources drive the flow through the .
2ii
Q
1 0
0
0,
0,
iiQ Q
Q
(4.4)
is e f rc oing into the sink. Because the
time le f
shuttle streaming, the sources are taken to be constant. Note that is a constant.
)
and (4.4) as follows:
where 0Q th lux coming from the sou e and g
sca of network adaptation is an order of magnitude larger than the time scale o
0Q
The network Poisson equation for the pressure is derived from the equation (4.1), (4.3
0
0
1,
2,
0
iji j
i ij
Q jD
p p Q j
otherwise
(4.5)
as a basic pressure level. All2 0p ip ’s can be obtained by solving (4.5), and each
ijij i j
Dp p is also obtained.
ij
be
the sta
through the shortest path between the sink and the source remains, the other pipes are died
out then.
4.3 Biologically inspired adaptive network
al and social systems. They are a
critical part of the infrastructure needed to facilitate efficient movement of people, energy,
ation. However, m
design principles because the main motivation was to achieve high transport efficiency at
reasonable cost but with less em s tolerant to failure or
interruption. It is desirable to develop networks with greater intrinsic resilience.
e Physarum to develop a biologically inspired model for adaptive
netw grows in the form o
4-3-1) which has been subjected to successive rounds of evolutionary selection. And the
Q
In order to apply this dynamics to path-finding problem in a network, two nodes to
rt and the goal is to be the sink and the source of current 0Q , and only the pipes
[27]
Transport networks are ubiquitous in biologic
resources and inform ost networks have emerged without clear global
phasis on making system
We study th
ork development. Physarum f an interconnected network (figure
Shortest paths through a reinforced random walk 60
network is likely to have reached a point at which cost, efficiency and resilience are
combinatorial optimization problems. The Physarum networks showed charateristic
balanced appropriately. Such network is likely to yield reasonable solutions to
s
simi terms of cost, tra
tolerance. However, the Physarum networks self-organized without centralized control or
expl al
and simultaneous removal of redundant connections.
t the
cond
lar to those of the rail network in nsport efficiency, and fault
icit glob information by a process of selective reinforcement of preferred routes
We developed a mathematical model for adaptive network construction to emulate
this behavior. In order to describe the adaptation of tubular thickness, we assume tha
uctivity ijD changes in time according to the flux ijQ as follows:
ijij ij
dDf Q D
dt
f Q is an increasing function with 0 0f where . This equation implied that
conduc
segment. In the equation, the first term on the right side describes the expansion of tubes
in response to the flux; the second term represents the rate of the tube constriction, so
that in the absence of flow the tubes will gradually disappear.
tivity tends to decline exponentially, but is enhanced by the flux through the
f Q is given by / 1r r
f Q Q Q . r is a parameter controls the
0
( 0Q in eq
nonlinearity of feedback ( ). The selection of specific parameter combinations
uation (4.5)) and can give a network with features similar to those of both
ics of
network adaptability through ite
0r
r
Q
the Physarum system and the rail network.
The biological inspired mathematical model can capture the basic dynam
ration of local rules and produces solutions with
properties comparable to or better than those of real-world infrastructure networks. The
model has a number of parameters that allow adjustment of the benefit/cost ratio to
increase specific features such as fault tolerance or transport efficiency, while keeping
costs low.
Shortest paths through a reinforced random walk 61
4-3-1 Network formation in Physarum polycephalum (Figure from [27])
(A) At 0t , a small plasmodium of Physarum was placed at the location of Tokyo in a
experimental arena bounded by the Pacific coastline (white border) and supplemented w
additional food sources at each of the major cities in the region (white dots). The horizontal
of each panel is 17cm. (B -F) The plasmodium grow out from
n
ith
width
the initial food source with a
contiguous mar essively colonized each of the food s growing
margin, the s lium resolved into a netw od sources.
gin and progr ources. Behind the
preading myce ork of tubes interconnecting the fo
(This figure is from Tero’s paper [27])
Shortest paths through a reinforced random walk 62
5. Appendix
2-2-1 Page 8
imax=101; % number of x grid divisions
jmax=101; % number of y grid divisions
nmax=10000; % number of random walks
% boundary conditions
bottom_bc=1;
right_bc=2;
top_bc=3;
left_bc=4;
s=0; % initialize the scalar potential
for n=1:nmax
% begin the random walk at the centre of the grid
i=
t
elseif (r>=0.25)&&(r<0.5)
i=i-1; % step left
elseif (r>=0.5)&&(r<0.75)
j=j+1; % step up
else
j=j-1; % step down
end
end
% random walker reaches a side
if (j==1)&&(i~=1)&&(i~=imax)
s=s+bottom_bc;
elseif (j==jmax)&&(i~=1)&&(i~=imax)
s=s+top_bc;
elseif (i==1)&&(j~=1)&&(j~=jmax)
51; j=51;
% perform random walk until a boundary is reached
while(i~=1)&&(j~=1)&&(i~=imax)&&(j~=jmax)
r=rand;
if r<0.25
i=i+1; % step righ
Shortest paths through a reinforced random walk 63
s=s+left_b
elseif (i==imax)&&(j~
c;
=1)&&(j~=jmax)
ht_bc;
end
end
ndom walks
2-2-2
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Monte Carlo Solution
% describe the graph in a matrix, the internal points are 2,3,6,7,11; the boundary points are
% 1,4,5,8,9,10,12,13
%%% %%%%%%%%%%%%%%%%%%%%%%
G=z % initialize the matrix with all zero elements
G(1, ed
G(2,3)=1;G node 2 is connected with node 3 and node 6
G(3, cted with node 1, 2, 4, 7
G(4,
G(5,6)=1 e 5 is connected with node 6
G(6,2)=1;G(6,5)=1;G(6,7)=1;G(6,10)=1; % node 6 is connected with node 2, 5, 7, 10
G(7,3)=1 ; % node 7 is connected with node 3, 6, 8, 11
G(8,7)=1 ith node 7
G(9,10)= ith node 10
G(10,6)= node 10 is connected with node 6, 9, 11
G(11,7)=1 (11,13)=1; % node 11 is connected with node 7, 10,
% 12, 13
G(12,11)= with 11
G(13,11)=1; % node 13 is connected with 11
B=[1,4,5,8,9,10,12,13] % points with value 0
B1= value 1
%
I=zeros(1
v=ze
T=10000 n times
for t
s=s+rig
s=s/nmax % divide by the total number of ra
Page 9
%%%%%%%%%%%%%%%%%%%
eros(13);
3)=1; % node 1 and node 3 are connect
(2,6)=1; %
1)=1;G(3,2)=1;G(3,4)=1;G(3,7)=1; % node 3 is conne
3)=1; % node 4 is connected with node 3
; % nod
;G(7,6)=1;G(7,8)=1;G(7,11)=1
; % node 8 is connected w
1; % node 9 is connected w
1;G(10,9)=1;G(10,11)=1; %
;G(11,10)=1;G(11,12)=1;G
1; % node 12 is connected
[1,4,5,9,10] % points with
current
3);
ros(1,13);
; % iteratio
=1:T
Shortest paths through a reinforced random walk 64
for i
3);
while (sum(pos==B)==0) % check if hit boundary
r=rand; % choose a random number r
es are connected with i
p_choose_v=G(pos,:)/sum(G(pos,:)); % the probability of choosing
% random walk starting from node i
p_choose_v(p_choose_v>0);
= find(cumsum(p_choose)>r,1,'first');
5
)=1;G(8,9)=1;G(8,11)=1;
9,12)=1;
;
=1:13
pos=i;
I_step=zeros(1
num_connect_v=sum(G(pos,:)); % how many nod
p_ind=find(p_choose_v>0); % find the elements with positive probability
p_choose=
% the positive probability of choosing random walk starting from node i
chosen
chosen_v = p_ind(chosen);
I_step(pos,chosen_v)=I_step(pos,chosen_v)+1;
pos=chosen_v;
end
v(i)=v(i)+(sum(pos==B1));
I_step;
I=I+abs(I_step-I_step');
end
end
v=v/T
2-2-3(a) Page 10
G=zeros(12); % initialize the matrix with all zero elements
G(1,4)=1; % node 1 and node 4 are connected
G(2,5)=1; % node 2 is connected with node
G(3,4)=1;
G(4,1)=1;G(4,3)=1;G(4,5)=1;G(4,8)=1;
G(5,2)=1;G(5,4)=1;G(5,6)=1;G(5,9)=1;
G(6,5)=1;
G(7,8)=1;
G(8,4)=1;G(8,7
G(9,5)=1;G(9,8)=1;G(9,10)=1;G(
G(10,9)=1
Shortest paths through a reinforced random walk 65
G(11,8)=1;
G(12,9)=1;
% the internal points are 4,5.8,9.
v=zeros(1,12);
for i=1:12
% choose a random number r
um_connect_v=sum(G(pos,:)); % how many nodes are connected with i
um(G(pos,:)); % the probability of choosing random
% walks starting from node i
e_v>0); % find the element with positive probability
p_choose=p_choose_v(p_choose_v>0);
% the positive probability of choosing random walk starting from node i
chosen = find(cumsum(p_choose)>r,1,'first');
chosen_v = p_ind(chosen);
(pos,chosen_v)=I_step(pos,chosen_v)+1;
step;
;
% the boundary points are 1,2,3,6,7,10,11,12.
B=[1,2,3,6,7,10,11,12]
B1=[1] % point with score 1.
I=zeros(12);
T=10000; % starting T times random walks
for t=1:T
pos=i;
I_step=zeros(12);
while (sum(pos==B)==0) % check if hit boundary
r=rand;
n
p_choose_v = G(pos,:)/s
p_ind=find(p_choos
I_step
pos=chosen_v;
end
v(i)=v(i)+(sum(pos==B1));
I_
I=I+abs(I_step-I_step');
end
end
v=v/T
2-2-3(b) Page 10
G=zeros(14)
Shortest paths through a reinforced random walk 66
G(1,3)=1;
(2,6)=1;
)=1;
1;
,6)=1;G(7,8)=1;G(7,11)=1;
;G(10,9)=1;G(10,11)=1;G(10,13)=1;
1,12)=1;G(11,14)=1;
B1=[1,2,4,5,9,13]; % points with score 1.
T=10000; %
(sum(pos==B)==0)
_connect_v=sum(G(pos,:));
(G(pos,:));
p_ind=find(p_choose_v>0);
p_choose=p_choose_v(p_choose_v>0);
chosen = find(cumsum(p_choose)>r,1,'first');
chosen_v = p_ind(chosen);
(pos,chosen_v)=I_step(pos,chosen_v)+1;
s=chosen_v;
G(2,3)=1; G
G(3,1)=1;G(3,2)=1;G(3,4)=1;G(3,7
G(4,1)=1;G(4,3)=1;G(4,5)=1;G(4,8)=1;
G(5,6)=1;
G(6,2)=1;G(6,5)=1;G(6,7)=1;G(6,10)=
G(7,3)=1;G(7
G(8,7)=1;
G(9,10)=1;
G(10,6)=1
G(11,7)=1;G(11,10)=1;G(1
G(12,11)=1;
G(13,10)=1;
G(14,11)=1;
% the internal points are 3,6,7,10,11.
% the boundary points are 1,2,4,5,8,9,12,13,14.
B=[1,2,4,5,8,9,12,13,14];
I=zeros(14);
v=zeros(1,14);
iteration times
for t=1:T
for i=1:14
pos=i;
I_step=zeros(14);
while
r=rand;
num
p_choose_v = G(pos,:)/sum
I_step
po
Shortest paths through a reinforced random walk 67
end
v(i)=v(i)+(sum(pos==B1));
ge 40
%%%%%%%%%%%%%%%%%%%
ph G=(V,G), we first take an easy case. We have 4 nodes 1\2\3\4 in the graph.
the source and node 4 is the sink.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% show connection and length in a matrix
% the number of nodes
;
% set initial pheromone conductivity matrix
ial numbers of ants on each node
eromone that an ant deposits on this way during a
interval
1, avoiding a too rapid
I_step;
I=I+abs(I_step-I_step');
end
end
v=v/T
3-2 (A) Pa
%%%%%%%%%%%%%%%%%%%%%%%%
% Exponential distributed Simulation:
% Input gra
% Node 1 is
%%%%%%%
clear;
G=[0 1 6 0;
1 0 2 4;
6 2 0 1;
0 4 1 0];
N=length(G);
D=[0 1 1 0;
1 0 1 1
1 1 0 1;
0 1 1 0];
n(1,:)=[10,5,5,0]; % set init
q=0.1; % the amount of ph
% time
inRate=25; % the inflow for the source node
outRate=25; % the outflow for the sink node
lamda=0.5; % evaporation rate, should be less than
% convergence
t=0;
ts=1;
Shortest paths through a reinforced random walk 68
S=10000*floor(1/lamda); % iteration times
r v=1:N
T(u,v)=abs(n(ts,u)-n(ts,v)); % T is the matrix with the elements |
end
Gs=max(reshape(D.*T./G,1,N^2),0); % D.*T./G is the matrix with the elements
%
for s=1:S
T=zeros(N);
for u=1:N
fo
i jn n |
end
| |i j ijn n D
ij
=nansum(nansum(Gs)); % sG=
% ants coming in to the system
Gs=[Gs inRate outRate];
| |i j i
ij
n n D j
sG
Dt=(-1/sG)*log(rand); % exponentially distributed
G;
cumsum(Gs)<=r)+1;
n(ts,:);
n(ts+1,
else
j=ind-(i
flip an ant
if n(ts,i)>n(ts,j)
n(ts+1,i)=n(ts,i)-1;
t=t+Dt;
storet(ts)=t;
% choose one edge at random
r=rand*s
ind=sum(
n(ts+1,:)=
if (ind==N^2+1)
% ant coming in
1)=n(ts,1)+1;
elseif (ind==N^2+2)
n(ts+1,N)=max(0,n(ts,N)-1);
i=ceil(ind/N);
-1)*N;
%
Shortest paths through a reinforced random walk 69
n(ts+1,j)=n(ts,j)+1;
D(i,j)=D(i,j)+q; % pheromone laid by an ant
ts+1,i)=n(ts,i)+1;
j)=n(ts,j)-1;
end
-Dt*lamda)*D % quantity of pheromone after evaporation
=max(0.001,D).*(G>0); % the elements should not be less than zero
totalD(ts)=sum(sum(D));
plot(n)
nt on each node,(number of node is',num2str(N),')'])
l('t')
omone')
%%%%%%%%%%%%%%%%%%%%%%%
onential distributed Simulation:
There are ten nodes in the graph.
e and node 10 is the sink.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
10 is the sink.
else
n(
n(ts+1,
D(j,i)=D(j,i)+q;
D=(1
D
end
ts=ts+1;
end
figure(1)
grid on,box on
xlabel('t')
ylabel('n')
title(['number of a
figure(2)
plot(totalD)
grid on, box on
xlabe
ylabel('totalD')
title('total amount of pher
3-2 (B) Page 42
%%%%%%%%%%%%%%%%%%%
% Exp
% Input graph G=(V,G).
% Node 1 is the sourc
%%%%%%%%%%
clear;
% node 1 is the source and node
Shortest paths through a reinforced random walk 70
G=[0 2 0 0 0 0 1 0 0 0;
2 0 2 0 3 0 0 0 0 0;
0 2 0 4 0 0 0 0 0 0;
0 0 2 0]; % show connection and length in a matrix
f nodes
1 0 0 0 0 1 0 0 0;
1 0 0 0 0 0;
1 0 0 0 0 0 0;
0 0 1
0 1 1 0 0 0
1 0 0 0 1 1
0 0 0 0 0 0 1 0 1 0
0 1 0 1 0 1
1 0 0 1 0]; % set initial pheromone conductivity matrix
5,5,15,10,5,15,0]; % set initial numbers of ants on each node
% the amount of pheromone that an ant deposits on this way
% during a time interval
source node
outRate=25; % the outflow for the sink node
aporation rate, should be less than 1, avoiding a too rapid convergence
u=1:N
0 0 4 0 0 1 0 0 0 6
0 3 0 0 0 1 5 0 0 0
0 0 0 1 1 0 0 0 1 2
1 0 0 0 5 0 0 4 0 0
0 0 0 0 0 0 4 0 3 0
0 0 0 0 0 1 0 3 0 2
0 0 0 6 0 2
N=length(G); % the number o
D=[0
1 0 1 0
0 1 0
0 0 1 0 0 1 0
0 1 0 0
0 0 0 1
1 0 0 0 1 0 0 1 0 0
0 0 0 0
0 0 0 1 0
n(1,:)=[20,10,5,1
q=0.1;
inRate=25; % the inflow for the
lamda=0.5; % ev
t=0;
ts=1;
S=10000*floor(1/lamda); % iteration times
for s=1:S
T=zeros(N);
for
for v=1:N
Shortest paths through a reinforced random walk 71
T(u,v)=abs(n(ts,u)-n(ts,v));
.*T./G,1,N^2),0);
to the system
ate];
(Gs));
d); % exponentially distributed.
storet(ts)=t;
s)<=r)+1;
,1)+1;
)
x(0,n(ts,N)-1);
if n(ts,i)>n(ts,j)
else
n(ts+1,i)=n(ts,i)+1;
D(j,i)=D(j,i)+q;
t*lamda)*D % quantity of pheromone after evaporation
1,D).*(G>0); % the elements should not be less than zero
end
end
Gs=max(reshape(D
% ants coming in
Gs=[Gs inRate outR
sG=nansum(nansum
Dt=(-1/sG)*log(ran
t=t+Dt;
% choose one edge at random
r=rand*sG;
ind=sum(cumsum(G
n(ts+1,:)=n(ts,:);
if (ind==N^2+1)
% ant coming in
n(ts+1,1)=n(ts
elseif (ind==N^2+2
n(ts+1,N)=ma
else
i=ceil(ind/N);
j=ind-(i-1)*N;
% flip an ant
n(ts+1,i)=n(ts,i)-1;
n(ts+1,j)=n(ts,j)+1;
D(i,j)=D(i,j)+q; % pheromone laid by an ant
n(ts+1,j)=n(ts,j)-1;
end
D=(1-D
D=max(0.00
Shortest paths through a reinforced random walk 72
end
ts=ts+1;
(ts)=sum(sum(D));
t on each node,(number of node is',num2str(N),')'])
omone')
%%%%%%%%%%%%%%%%%%%%%%%
son distributed simulation in 4 nodes graph
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
urce and node 4 is the sink
% the number of nodes
totalD
end
figure(1)
plot(n)
grid on,box on
xlabel('t')
ylabel('n')
title(['number of an
figure(2)
plot(totalD)
grid on, box on
xlabel('t')
ylabel('totalD')
title('total amount of pher
3-3 (C) Page 49
%%%%%%%%%%%%%%%%%%%
% Pois
%%%%%%%%%%%%
clear;
% node 1 is the so
G=[0 1 6 0;
1 0 2 4;
6 2 0 1;
0 4 1 0]; % show connection and length in a matrix
N=length(G);
D=[0 1 1 0;
1 0 1 1;
1 1 0 1;
0 1 1 0]; % initial pheromone conductivity
Shortest paths through a reinforced random walk 73
n(1,:)=[10,7,3,0]; % set initial numbers of ants on each node
% evaporation rate, should be less than 1
% fix the time interval
% charge/particle
floor(1/lamda); % number of iteration
e(G)); % initial current matrix
r ts=1:S
t;
n(ts,1)+poissrnd(b);
ber of ants run in the sink is a Poisson distributed number
)=max(0,n(ts,N)-poissrnd(b));
ber of ants run away from the sink is also a Poisson distributed number
for j=1:N
i,j)==0)
end
(i,j)*Dt; % update the pheromone
ated after all the current is known
n(ts+1,k) = n(ts,k) - sum(P(k,:)) + sum(P(:,k));
0.001,D).*(G>0)
)=sum(sum(D));
lamda=0.3;
t=0;
ts=1;
Dt=0.02;
Q=0.01;
b=20;
S=10000*
P=zeros(siz
fo
t=t+D
n(ts,1)=
% the num
n(ts,N
% the num
for i=1:N
if (G(
P(i,j)=0;
else
P(i,j)=poissrnd(Dt*abs(n(ts,i)-n(ts,j))*D(i,j)/G(i,j))*sign(n(ts,i)-n(ts,j));
D(i,j)=D(i,j)*(1-Dt*lamda)+Q*P
end
end
for k=1:N
% the number of ants should be upd
end
D=max(
totalD(ts
end
Shortest paths through a reinforced random walk 74
3-3 (D) Page 52
clear;
e take 10 nodes, node 1 is the source and node 10 is the sink
0 4 0 0 0 0 0 0;
0 0 0 1 1 0 0 0 1 2
5 0 0 4 0 0
4 0 3 0
0
0
N=le nodes
D=[0
0 0 0;
;
1
0
0 1 0]; % set initial pheromone matrix
,10,5,15,5,15,10,5,15,0]; % the number of ants
mda=0.3; % evaporation rate, should be less than 1
t=0;
ts=1
% charge/particle
000*floor(1/lamda); % number of iteration
P=zeros(size(G)); % initial current matrix
% w
G=[0 2 0 0 0 0 1 0 0 0;
2 0 2 0 3 0 0 0 0 0;
0 2
0 0 4 0 0 1 0 0 0 6
0 3 0 0 0 1 5 0 0 0
1 0 0 0
0 0 0 0 0 0
0 0 0 0 1 0 3 0 2
0 0 6 0 2 0 0 2 0]; % show connection and length in a matrix
ngth(G); % the number of
1 0 0 0 0 1 0 0 0;
1 0 1 0 1 0 0
0 1 0 1 0 0 0 0 0 0
0 0 1 0 0 1 0 0 0 1
0 1 0 0 0 1 1 0 0 0
0 0 0 1 1 0 0 0 1
1 0 0 0 1 0 0 1 0 0
0 0 0 0 0 0 1 0 1
0 0 0 0 0 1 0 1 0 1
0 0 0 1 0 1 0
n(1,:)=[20
la
;
Dt=0.02; % fix the time interval
Q=0.01;
b=20;
S=10
Shortest paths through a reinforced random walk 75
for ts=1:S
t=t+Dt;
ts,1)=n(ts,1)+poissrnd(b);
number
,N)-poissrnd(b));
ants run away from the sink is also a Poisson distributed number
==0)
,j)=0;
,j)=poissrnd(Dt*abs(n(ts,i)-n(ts,j))*D(i,j)/G(i,j))*sign(n(ts,i)-n(ts,j));
t; % update the pheromone
f ants should be updated after all the current is known
(ts,k) - sum(P(k,:)) + sum(P(:,k));
*(G>0)
m(D));
n(
% the number of ants run in the sink is a Poisson distributed
n(ts,N)=max(0,n(ts
% the number of
for i=1:N
for j=1:N
if (G(i,j)
P(i
else
P(i
end
D(i,j)=D(i,j)*(1-Dt*lamda)+Q*P(i,j)*D
end
end
for k=1:N
% the number o
n(ts+1,k) = n
end
D=max(0.001,D).
totalD(ts)=sum(su
end
Shortest paths through a reinforced random walk 76
References
[1] o, and Guy Theraulaz. Swarm Intelligence: From Natural
39-46, 1999.
[2] Joachim Keltsch. Active walker model for the
,
997.
nd Thomas Stutzle. Ant Colony Optimization. The MIT Press, pages
76, 2004.
Maria Gambardella. Ant Colony System: A cooperative
Traveling Salesman Problem. IEEE TRANSACTIONS ON
RY, Vol. 1, No.1, 53-66, 1997.
putational study. University of Vienna, 1997.
ts as a computational intelligence technique. IEEE COMPUTATIONAL
LLIGENCE MAGAZINE, 1556-603X, 28-39, 2006.
] Daniel Merkle and Martin Middendorf. Modeling the dynamics of ant colony
volutionary Computation 10(3): 235-262, 2002.
[8] thnology,
Denebourg, S. Aron, S. Goss, and J. M. Pasteels. The self-organizing exploratory
havior, Vol. 3, 159, 1990.
enebourg, S. Aron and J. M. Pasteels. “How trail laying
and trial following can solve foraging problems for ant colonies,’’ in Behavioral
Mechanisms of Food Selection, R. N. Hughes Ed., NATO-ASI Series. Berlin:
Springer-Verlag, vol. G 20, 1990.
1] Marco Dorigo, Vittorio Maniezzo, and Alberto Colorni. The ant system: Optimization
by a colony of cooperating agents. IEEE TRANSACTION ON SYSTEMS, MAN,
AND CYBERNETICS-PART B, 26(1): 29-41, 1996.
2] Peter G. Doyle and J. Laurie Snell. Random walks and electric networks. Mathematical
Association of America, Washington, DC, 1984.
3] Atsushi Tero, Ryo Kobayashi, and Toshiyuki Nakagaki. Physarum solver: A biologically inspired method of road-network navigation. Physica A: Statistical Mechanics and its Applications, 363(1): 115–119, 2006.
[14] David Stirzaker. Stochastic Processes &Models[M]. Oxford University Press, pages 50,
Eric Bonabeau, Marco Dorig
to Artificial Systems. Oxford University Press, pages 1-8, 14-16, 25-36,
Dirk Helbing, Frank Schweitzer, and
formation of human and animal trail systems. Physical Review E, vol. 56, 2527-2539
1
[3] Marco Dorigo a
1-34, 65-69, 70-
[4] Marco Dorigo and Luca
Learning Approach to the
EVOLUTIONA
[5] Bernd Bullnheimer and Richard F. Hartl. A new rank based version of the ant
system—a com
[6] Marco Dorigo, Mauro Birattari and Thomas Stutzle. Ant colony optimization:
artificial an
INTE
[7
optimization. E
J. L. Denebourg and S. Goss. Collective patterns and decision-making. E
Ecology and Evolution, vol. I, 295-311, 1989. [9] J. L.
pattern of the Argentine ant. Journal of Insect Be
[10] S. Goss, R. Beckers, J. L. D
[1
[1
[1
Shortest paths through a reinforced random walk 77
8, 2005.
. Reversibility and stochastic networks. John Wiley & Sons Ltd.,
[16] Laszlo Lovasz. Random
[18] Colorni, Marco Dorigo and Vittorio Maniezzo. Distributed Optimization by
Ant Colonies. In proceedings of ECAL91-
42, Paris.
[20] Marco Dorigo. Optimnico di Milano, Italy, 1992.
. . Rep. 91-016, 1991.
lasmic move
[23] oeboid organism. Nature
[24] T.. Flow-network
, 47–52,2001.
l., 244(4):
[28] d
s, pages 201~204, 2008.
71-76, 107-12
[15] Frank P. Kelly
Chichester, 1979. Wiley Series in Probability and Mathematical Statistics.
walks on graphs: a survey. YALEU/DCS/TR-1029, 1994.
[17] Kentaro Ito, Anders Johansson, Toshiyuki Nakagaki and Atsushi Tero. Convergence
Properties for the Physarum Solver. Arxiv preprint arXiv:1101.5249, 2011.
Alberto
European conference on artificial life,
Elsevier Publishing, 134-1
[19] E. L. Lawler, J. K. Lenstra, A.H.G. Rinnooy-Kan and D. B. Shmoys Eds.. The
travelling Salesman Problem. New York: Wiley, 1985.
ization, learning and natural algorithms (in Italian), Ph.D. dissertation. Dipartimento di Elettronica, Politec
[21] M. Dorigo, V. Maniezzo, and A. Colorni. Positive feedback as a search strategyDipartimento di Elettronica, Politecnico di Milano, Italy, Tech
[22] Kobayashi, R., Tero, A., Nakagaki, T.: Mathematical model for rhythmic protopment in the true slime mold. Journal of Mathematical Biology 53, 273–286,
2006. Nakagaki, T., Yamada, H., Toth, A.. Maze-solving by an am407, 470–470, 2000. Tero, A., Yumiki, K., Kobayashi, R., Saigusa, T., and Nakagaki,adaptation in physarum amoebae. Theory in Biosciences 127, 89–94, 2008.
[25] Nakagaki, T., Yamada, H., Toth, A.. Path finding by tube morphogenesis in an amoeboid organism. Biophysical Chemistry 92
[26] Toshiyuki Nakagaki, Atsushi Tero, and Ryo Kobayashi. A mathematical model for adaptive transport network in path finding by true slime mold. J. Theor. Bio553-564, 2007.
[27] Atsushi Tero, Seiji Takagi, Tetsu Saigusa, and Kentaro Ito. Rules for Biologically Inspired Adaptive Network Design. Science Vol. 327, 439-442, 2010.
Matt J. Keeling and Pejman Rohani. Modeling infectious diseases in humans an
animals. Princeton University Pres