KohonAnts: A Self-Organizing Ant Algorithm for Clustering and

Pattern ClassificationAntonio M. Mora, C.M. Fernandes, J.J. Merelo, J.L.J. LaredoDpto. ATC. UNIVERSITY OF GRANADA (Spain)

V. Ramos, A.C. RosaLASEEB-ISR/IST. UNIVERSITY OF LISBON (Portugal)

INDEX

Ant Algorithms

Kohonen’s Self Organizing Maps

KohonAnts

Experiments and Results

Conclusions and Future Work

Ant Algorithms Natural Ants

Ants live in colonies and cooperate searching for the benefit of the colony instead of their own. Ants move following (smelling) and depositing a chemical substance known as pheromone (which is also evaporated). This ‘comunication’ method is known as stigmergy.

They can find the shortest path between their nest and the food location.

They build clusters with their larvae and also with their dead bodies.

Ant Algorithms Main Features

There are many agents called (artificial) ants.

All of them move in a graph following and depositing (artificial) pheromone.

They cooperate to find a solution (usually every ant yields a complete solution).

There are some formulae applied in the running: state transition rule decides the next step for each ant

pheromone updating contribution and evaporation

evaluation function assigns the cost to every solution

Ant Algorithms Ant System Model

Introduced by Chialvo and Millonas in 1995. Ants move in a grid. Each one is represented by its position r, and orientation to move .

They decide where to move without consider their previous movements.

They apply the so-called response function, which depends on 3 parameters:

pheromone density of the cell

degree of randomness with which an ant follow the gradient of pheromone

factor of ant’s ability to sense pheromone

Kohonen’s SOM Main Features

It is an unsupervised Artificial Neural Network.

There is only an input and an output layer.

There is a Competitive learning (only one neuron is activated for an input pattern).

It Maps high-dimensional data into a two-dimensional representation space (usually rectangular or hexagonal grid).

After training phase, neurons with similar weights tend to cluster on the map.

Kohonen’s SOM General Structure

• N neurons in the input layer, M neurons in the output layer. Each input neuron is connected with all the output ones. Every input sample is associated to an input neuron.

• Each neuron of the output layer has associated a weight vector Wj, as the same dimension as the input data.

Kohonen’s SOM Working

• For each input pattern/sample, the most similar neuron (considering its weight vector), and its neighbours, are activated.

• The weight vector of every one of these neurons is updated to be closer to the input values.

• The SOM evolves, ‘moving’ the neurons in order to fit to the input space (of samples).

KohonAnts Presentation

It merges Ant System with Kohonen’s SOM concepts. It is a clustering and classification algorithm.

**

INPUTSAMPLES

Each input sample (vector of variables) is associated to an ant.

They move in a toroidal grid which has a vector (of the same dimension) associated to each cell (pheromone).

KohonAnts Working

Initialization Decide Where to Go

For each Ant:

Pheromone Update

Pheromone EvaporationEND

Stop criteria

is TRUE

No more Ants

Stop criteria

is FALSE

KohonAnts Main Functions (I)

Decide Where to Go every ant evaluates its neighbourhood (similar to SOM) to find the cell where move to. A probability is associated to every cell.

the probability of each cell depends on the euclidean distance between the ant vector and the centroid of a zone surrounding the cell.

o

o

o

o

• yellow circle is the neighbourhood (all possible cells to move) with radius = 2.

• green circles are areas to calculate centroid (marked with ’o’).

• the centroid is the arithmetic mean of the vectors inside the circle.

So, every ant moves with higher probability to cells surrounded by vectors similar to its own.

KohonAnts Main Functions (II)

Pheromone Update once an ant chooses the cell to move to (inside its neighbourhood), it updates the vector of that cell and makes it closer to its own vector.

it applies a formula similar to the learning function of SOM, so it depends on a learning rate .

Pheromone Evaporation once all the ants have moved, the pheromone tends to disappear (the vectors revert to its initial values).

it uses a formula similar to the evaporation function of Ant Algorithms, which depends on an evaporation rate .

KohonAnts Key Features

The neighbourhood radius and the learning rate decreases with the iterations (as in SOM). It means big changes at the beginning (clustering) and small changes the rest of the running (refinement).

An ant can ‘jump’ or ‘fly’, since it can move to cells far than one hop.

The pheromone changes the grid (environment) and makes it closer to the ant’s vector (input pattern/sample).

The ants tend to form clusters near other similar ants (similar vectors).

The environment (the grid) also ‘evolves’ and can be used as a classification tool.

Experiments Datasets

IRIS data of 3 species of Iris plant, 50 samples of each class, and 4 variables per sample (flower attributes).

GLASS data of 6 classes of glass, 214 samples in total, and 9 variables per sample (chemical composition).

PIMA data of 2 classes of diabetes patients, 768 samples, and 8 variables per sample (medical data).

Experiments Clustering (Study)

performed over IRIS plants dataset (150 samples, 3 classes)

it shows ants’ final position in the grid after 100 iterations

a clear line of behaviour can be marked depending on and

it can be seen that clear clusters are formed for some values of these parameters

Experiments Clustering (Results)

performed over IRIS plants dataset (150 samples, 3 classes)

it shows the evolution of the clusters of ants

in a few iterations, some clusters appear

then, they are improved/refined in the following iterations

Initial situation Iteration 10

Iteration 40 Iteration 100

Experiments Classification

example performed over IRIS plants dataset (150 samples, 3 classes)

this is the final situation of the grid pheromone. Each cell is associated to a class

we apply the KNN method to classify, so the class associated to a sample is the correspondent to the (majority) of the K closest vectors (of the grid)

Experiments Classification (Results)

Every dataset has been transformed into 6:

• 3 of 50% training/50% test

• 3 of 90% training/10% test

The results beat the classical KNN method

In recent works also beat other methods such as Neural Networks, Fuzzy Logic, Linear Discriminant Analysis…

Conclusions And Future Wok We have presented a new algorithm which combines ideas of two soft-computing methods such as Kohonen’s Self-Organizing Maps and Ant System Model.

It can be used as a clustering and as a pattern classification method.

The preliminary results yielded are very promising and beat some other techniques.

FUTURE WORK:

Make a comparison with some clustering algorithms and with other classification ones. Try more difficult problems.

Implement and test some other ideas used in SOMs and AS which may be improve the algorithm. Such a neighbourhood updating, a stop criteria or weights in the decide where to go function.

Thank You!

The sources, bin and data can be found in:

https://forja.rediris.es/websvn/wsvn/geneura/KohonAnts/