cooperative behavior of artificial neural agents based on ......supervised learning : parameters...

WETICE 2008 - Workshop on Coordination Models and Applications (CoMA)

Cooperative behavior of artificial

neural agents based on evolutionary

architecturesAlessandro Londei, Piero Savastano, Marta Olivetti Belardinelli

Interuniversity Center for Research on Cognitive Processing

In Natural and Artificial Systems - ECONA, “Sapienza” University of Rome

Email: [email protected]


Empiricism vs. Innatism

Constructivism: Knowledge cannot be instructed by a teacher, it can only be constructed

by a learner (learning by environment)

Instructivism: Knowledge is instructed by a teacher (learning by a supervisor)

Empiricism

Nativism: Knowledge (skills or abilities) is hard-wired into the brain at birth.

Innatism

Certain cognitive modules are built-in at birth that allow to learn and acquire certain

skills as language (N. Chomsky, J. Fodor) and other cognitive functions

The human mind of a newborn is not a tabula rasa


Empiricism in Neural Networks

Supervised learning: parameters (synaptic weights) are modified until the response is correct

At the end of the learning phase, knowledge representation is given by weights distribution…

…and this is only true for the chosen neural architecture!

Stim

ulu

s

Response

Supervisor

Architecture plays a crucial role in the ability of a network to learn a specific behavior


Nativism in Neural Networks?

Evolutionary robotics: agents have a neural brain (fixed feed-forward architecture)

whose weights are described by an evolutionary process (genetic algorithm) [Nolfi 1994,

Belew 1992]S

tim

ulu

s

Response

Genetic coding

Fitness to the environment

• architecture is not considered as an evolutionary parameter

• phenotype is only given by synaptic weights

Low plausibility

No internal dynamical evolution

Absent learning phase (almost…)


Nativism in Neural Networks!

Topology and Weight Evolving Artificial Neural Networks (TWEANNs): optimization of

neural systems through augmenting topologies [Stanley 2002]

Drawback

One gene for each connection

Minimal network means weak robustness to failure


An improved nativist approach

Genetic

populationfitness crossing-over mutations

New

population

Plasticity

selection

Fixed feed-forward

architecture

Architecture

selectionDynamical recurrent

architecture

Food


Genetic Coding for a Neural Architecture

Phenotype:

- 8 sensorial inputs for: directional food detection (3), directional obstacle detection (2),

food proximity (2), hunger proprioceptor (1)

- 2 motor outputs

Neural Network:

- 40 max excitatory/inhibitory neurons with

sigmoidal activation function

- Hebb (through time) learning rule

- Neural transmission delay

Genotype:

- 7-length slot per neuron

- labeling for connection structure

- redundancy (forbidden codes)


Neuron and Network Model

yn= ! x

n( ) =1

1" exp # $ xn"T( )( )

Neuron:

- Sigmoidal activation function

- Beta and offset described in genetic code

- Excitatory or inhibitory (from label)

!!

y

1t ! "

1( )

y

kt ! "

k( )

y

Nt ! "

N( )

Network:

- Connections given by labels similarity (Hamming distance)

- Distance defines time-delay

- PSP given by synaptic weight and delayed inputs

- Hebbian learning rule !w

nk= e"#t

$ yn

t( ) $ ykt " %

k( )

x

nt( ) = w

kn! y

kt " #

k( )k$S

%


Boolean Gates - AND

01

11

10

00

01

11

01

00

01

10

11 11 11

0010


Boolean Gates - XOR

10

11

00

01

10

11

10

01

10

110000

11 10


Artificial Neural Agentslif

etim

e

generations

Best agent fitness

Worst agent fitness

Average fitness

Environment: a square arena containing 4 agents and 10 incremental food (max 80).

Arena sides are sticky and dangerous for the agents. Food increases lifetime by 200

steps and sides wounds the agents (lifetime decreases by 4 at each step)

Fitness: is given by the sum of all agents lifetime.

fitness = LTk

k

!


Agents Behavior

Algorithm demonstration

Gen #1: casual behavior

Gen #200: sensitivity to food and obstacle

Gen #400: emergence of a strategy for area exploration

Gen #1000: global optimization of social behavior


Features - Small World

Characteristic Path Length L(p)

typical separation between 2 vertices in the graph

Clustering Coefficient C(p)

cliquishness of a typical neighborhood

example C.Elegans: Lnorm=0.85, Cnorm=0.20

Watts Nature, 1998

Averaged over the first 10 best agents of each generation


Features - Chaos

Rabbit Olfactory Bulb - Freeman IEEE

Trans. on Circ. and Syst., 1988

3D evolution of 3 inner neurons -

autonomous case


Lyapunov Exponent

Features - Chaos

! = limT"#

1

Tlog

!

$ T( )!

$ 0( )


Features - Experimentum Crucis


Features - Cooperation

Fitness takes into account the “global ability to survive”

fitness = LTk

k

! "1

2LT

k" LT

jk , j

!



Agents behavior increases global lifetime by means of a greater modulation of the speed:

agents tend to stop after feeding and begin to search for food again after a “refractory”

transient

Difference between speed modulations is significant: F1,99=13.43, p<4·10-4

Improved agent models may be implemented in order to evidence put in evidence more

realistic cooperative behavior (communication and language emergence)


Conclusion

“A multi-agent system is one composed of multiple interacting software

components known as agents, which are typically capable of co-operating to

solve problems that are beyond the abilities of any individual member”

• Cooperation and social abilities depend on the agent phenotype and the

environment the agents are embedded in

• This neural model allows an improvement of the neural networks world in order to

be a candidate for MA systems, in respect of a native and unsupervised approach

• A way to reveal and analyze the complex aspects of natural neural dynamics

cooperative behavior of artificial neural agents based on ......supervised learning : parameters...

Documents