evolutionary algorithms as a complex dynamical systems

NAVY Research GroupDepartment of Computer Science

Faculty of Electrical Engineering and Computer Science VŠB-TUO17. listopadu 15

708 33 Ostrava-PorubaCzech Republic

Evolutionary Algorithms as a Complex Dynamical Systems

Ivan Zelinka

MBCS CIPT, www.bcs.org/http://www.springer.com/series/10624

Department of Computer ScienceFaculty of Electrical Engineering and Computer Science, VŠB-TUO

17. listopadu 15 , 708 33 Ostrava-PorubaCzech Republic

www.ivanzelinka.eu

navy.cs.vsb.cz2

Topics

• Evolutionary algorithms and mutual relations between its dynamics, complex networks and its control.

• Complex systems and behavior

• EA and feedback loop control system

navy.cs.vsb.cz3

Topics

navy.cs.vsb.cz4

Objectives

What do you can expect from this tutorial:• An introduction into complex behavior, such as:

– Chaotic– Stochastic– Catastrophe

• Sketch why EAs can be considered as a complex systems • Relations between EA dynamics and complex networks• EA as a feedback loop cybernetic system

What do you cannot expect from this tutorial:• To be expert on any of above mentioned fields

navy.cs.vsb.cz5

Tutorial structure

The tutorial structure is:• Complex system overview• Complex system behavior (chaos, randomness,

determinism)• Evolutionary algorithms (EA) dynamics

– Dynamics of EAs– Dynamics of EA strategies

• Complex networks• EA and complex networks (CN)

– CN as a part of Eas strategies– EAs dynamics as a complex network

• Control EAs dynamics

navy.cs.vsb.cz6

FEI VŠB-TU

http://www.vsb.cz/en/

http://www.vsb.cz/en/

navy.cs.vsb.cz7

NAVYhttp://navy.cs.vsb.cz

navy.cs.vsb.cz8

Presentation phase

navy.cs.vsb.cz9

Evolutionary and genetic dynamics, processes and laws

Fitness

Population dynamic

Hereditability

Hereditability

DNA coding (Schrodinger,

Watson and Creek)

Feature

Fitness

MEndelDar win

Tur ing Bar iccel i Moder n comput er evol ucionist s: Hol l and, Schwef el , Rechenber g, Fogel , Baeck, Pr ice, Koza, O'Nei l l , Ryan, ...

Gregor Johann Mendel July 20, 1822

–January 6, 1884.

Gregor Charles Darwin 12 February 1809 –19 April 1882.

navy.cs.vsb.cz10

Initial population setting

Control parameters definition of the selected evolutionary algorithm

Fitness evaluation of each individual

(parent)

Parent selection based on their

fitness

Offspring creation

Mutation of a new offsprings

Fitness evaluation

Best individual selection from

parents and offsprings

New empty population

occupation by selected individuals

Old population is replaced by new

one

Evolutionary loop

Evolution – the central dogma

From the above mentioned main ideas of Darwin and Mendel theory of evolution, ECT uses some building blocks, see the diagram.

The evolutionary principles are transferred into computational methods in a simplified form that will be outlined now.

11 navy.cs.vsb.cz

Evolutionary dynamics Animations

12 navy.cs.vsb.cz

EA dynamics Video

navy.cs.vsb.cz13

Presentation phase

navy.cs.vsb.cz14

Complex networks and EAsTwo ways of use

• CN as a controlling structure of EA dynamics

• EAs dynamics as a social interactions creating complex network

navy.cs.vsb.cz15

Complex networks inside EAsCN as a controlling structure of EA dynamics

• CN and internal dynamics of PSO

– Star

– Ring

– Wheel

– Pyramid

navy.cs.vsb.cz16

Evolution as a complex networkHow to convert

navy.cs.vsb.cz17


• Mathematica example

navy.cs.vsb.cz18

12

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20


12

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

navy.cs.vsb.cz19

Evolution as a complex network

An example of activated leaders with moment when evolution has found global extreme. In such a moment the best individual is repeatedly selected (see line after 230 migrations) and become to be extremely attractive node of all.

20 Ivan Zelinka, Donald Davendra and Vaclav Snasel

F ig. 18. and it s histogram of the vert ices

connect ions (note that winning vertex has

with almost 900 connect ions).

F ig. 19. An example of ” rich become to

be richer” , see also Figure 20 and 21

0 50 100 150 200 250 3000

20

40

60

80

100

Migration No.

Indiv

idu

alN

o.

F ig. 20. An example of act ivated leaders

with moment when evolut ion has found

global ext reme. In such a moment the best

individual is repeatedly selected (see line

after 230 migrat ions) and become to be

ext remely at t ract ive node of all.



global ext reme.

rather than randomly remoted algorithms. We think that this is quite

logical and close to the idea of prefered linking in the complex networks

modelling social behavior (citat ion networks, etc)

6. Evaluat ion and vizualizat ion: Evaluat ion has been done so that histogram

has been vizualized for each evolut ionary process (e.g. Figure 16), and ac-

t ivated vert ices e.g. Figure 25. To vizualize all experiments we have joined

all those figures (i.e. with act ivated vert ices) into one figure to vizualize

behavior of EAs from CNS point of view. As an example one can see Fig-

ures 30, 31,32, 36,37. Most of this figures show two phases; phase of ” free”

compet it ion, when each vertex (individual) has a chance to win and phase

of ” winner take all” , i.e. when no bet ter solut ion are generated (we are in

global extreme or EAs has got into stagnat ion) and st ill the same vertex

is selected like a winner. This ” e↵ect ” has been observable especially for

20 Ivan Zelinka, Donald Davendra and Vaclav Snasel

F ig. 18. and it s histogram of the vert ices

connect ions (note that winning vertex has

with almost 900 connect ions).

0 50 100 150 200 250 3000

20

40

60

80

Migration No.

Ind

ivid

ual

No

.

F ig. 19. An example of ” rich become to

be richer” , see also Figure 20 and 21



global ext reme. In such a moment the best

individual is repeatedly selected (see line

after 230 migrat ions) and become to be

ext remely at t ract ive node of all.



global ext reme.

rather than randomly remoted algorithms. We think that this is quite

logical and close to the idea of prefered linking in the complex networks

modelling social behavior (citat ion networks, etc)

6. Evaluat ion and vizualizat ion: Evaluat ion has been done so that histogram

has been vizualized for each evolut ionary process (e.g. Figure 16), and ac-

t ivated vert ices e.g. Figure 25. To vizualize all experiments we have joined

all those figures (i.e. with act ivated vert ices) into one figure to vizualize

behavior of EAs from CNS point of view. As an example one can see Fig-

ures 30, 31,32, 36,37. Most of this figures show two phases; phase of ” free”

compet it ion, when each vertex (individual) has a chance to win and phase

of ” winner take all” , i.e. when no bet ter solut ion are generated (we are in

global ext reme or EAs has got into stagnat ion) and st ill the same vertex

is selected like a winner. This ” e↵ect ” has been observable especially for

An example of ”rich become to be richer”

navy.cs.vsb.cz20


KCoreComponents

FindGraphPartition

PageRankCentrality

CommunityGraphPlot

navy.cs.vsb.cz21

Evolution as a complex networkNetwork attributes interpretation

• Adjacency graph

• Graph partition

• Degree centrality

• Community

• …

navy.cs.vsb.cz22

Adjacency graph

• Meaning: Graph with vertices and oriented edges.

• Interpretation: Visualization of evolutionary dynamics in the form of so called graph. Each vertex represent one individual in the population and each edge (oriented of course) represent successful offspring creation (i.e. fitness improvement of active parent in this philosophy) between parents connected by that edge.

navy.cs.vsb.cz23

Adjacency graph

Adjacency graph

navy.cs.vsb.cz24

Graph partition

• Meaning: Graph partition finds a partition of vertices such that the number of edges having endpoints in different parts is minimized. For a weighted graph, graph partition finds a partition such that the sum of edge weights for edges having endpoints in different parts is minimized.

• Interpretation: Individuals in population are separated into ”groups” according to their interactions with another individuals, based on their success in active individual fitness improvements. ”Endpoints” can be understood like successful participation of selected individuals in active individual fitness. On Figure 2 is partition visualized by colors. This analysis gives view on population structure and shows the set of individuals that got or donate oriented edges (support from / to) the same group of individuals. Based on number of connections or weights (if multiple edges are understood like integer weights) of edge, it can be analyzed what part of population was the most important in the evolutionary dynamics for given case.

navy.cs.vsb.cz25

Graph partition

navy.cs.vsb.cz26

Degree centrality

• Meaning: Degree centrality of g gives a list of vertex degrees for the vertices in the underlying simple graph of g. Degree centrality will give high centralities to vertices that have high vertex degrees. The vertex degree for a vertex v is the number of edges incident to v. For a directed graph, the in-degree is the number of incoming edges and the out-degree is the number of outgoing edges. For an undirected graph, in-degree and out-degree coincide.

• Interpretation: Degree centrality shows how many in-coming (support from individuals) or out-coming (support to individuals) edges vertex individual under study has. This quantity can be related to progress of the evolutionary search and used to made conclusion of what set of individuals has maximally contribute to that. On Figure 4 are individuals sized according to that degree.

navy.cs.vsb.cz27

Degree centrality

navy.cs.vsb.cz28

Community

• Meaning: Community graph plot attempts to draw the vertices grouped into communities.

• Interpretation: Community graph plot showing the individuals grouped into communities. Communities (with border are individuals that communicate amongst themselves (higher density of edges in community, multi edges are not visualized here, rather than between communities) and community are then joined by connections that are ”one-way” and shows flow

navy.cs.vsb.cz29

Community

navy.cs.vsb.cz30


• M. Soleimani-Pouri et al. dealt with solving of the

maximum clique problem in social networks, In: Soleimani-pouri M, Rezvanian A, Meybodi MR (2014) An Ant Based Particle

Swarm Optimization Algorithm for Maximum Clique Problem in Social

Networks, Springer International Publishing, pp 455–462.,

where ant based particle swarm algorithm is used.

• Q. Wu and J-K. Hao made a review on algorithms for maximum clique problem , In: Wua Q, Hao JK (2014) A review

on algorithms for maximum clique problems. European Journal of Operational Research.

navy.cs.vsb.cz31


• M. Gong et al. described complex network clustering by multiobjective discrete particle swarm optimization based on decomposition , In: Gong M, CaiQ, Chen X, Ma L (2014) Complex network clustering by multiobjectivediscrete particle swarm optimization based on decomposition. IEEE Transactions on Evolutionary Computation 18:82–97 .

• In: Li Y, Liu J, Liu C (2014) A comparative analysis of evolutionary and memetic algorithms for community detection from signed social networks. Soft Computing 18:329–348 , a comparative analysis of evolutionary and memetic algorithms for community detection from signed social networks is described etc.

navy.cs.vsb.cz32


• Since 1999, D. Ashlock et al. described graph-based genetic algorithm using a combinatorial graph to limit choice of crossover partner , In: Ashlock D, Smucker M,

Walker J (1999) Graph based genetic algorithms. In: Evolutionary Computation, IEEE, pp 1362–1368.

navy.cs.vsb.cz33


• In 2007, S. Mabu et al. described a graph-based evolutionary algorithm and graph-based evolutionary algorithm with reinforcement learning , In: Mabu S,

Hirasawa K, Hu J (2007) A graph-based evolutionary algorithm: Genetic network programming (gnp) and its extension using reinforcement

learning. Evolutionary Computation 15:369–398 . These algorithms deal with dynamic environment by using the higher expression ability of graph structure and inherently equipped functions in it.

navy.cs.vsb.cz34


• In the years 2010 and 2011, I. Zelinka et al. described the method for visualizing the dynamics of EA by the CNs and investigated the analogy between the individuals in the population and nodes of the CN and the relationships between individuals in the EA and edges between the nodes in CN, In: – Zelinka I, Davendra DD, Snasel V, Senkerik R, Jasek R (2010) Preliminary investigation on

relations between complex networks and evolutionary algorithms dynamics. In: Computer Information Systems and Industrial Management Applications (CISIM) 31.

– Zelinka I, Davendra DD, Senkerik R, Jasek R (2011) Do evolutionary algorithm dynamics create complex network structures? In: Complex Systems, Vol. 20, Issue 2.

– Zelinka, Ivan, Snasel, Vaclav, Abraham, Ajith (Eds.) , Handbook of Optimization, Springer, 2013

– Zelinka I. et. al, Evolutionary algorithms dynamics and its hidden complex network structures. CEC 2014, p. 3246 – 3251, Beijing, China

navy.cs.vsb.cz35


• In 2014, D. D. Davendra et al. analyzed the development of the CN in the discrete self-organizing migrating algorithm in Davendra D, Zelinka I, Senkerik R, Pluhacek M (2014) Complex network analysis of discrete self-organising migrating algorithm. In: Nostradamus 2014: Prediction, Modeling and Analysis of Complex Systems, pp 161–174.

• As we will show in the next sections, our approach extends the ideas already mentioned in the previous slide. The way how we construct the CN reflects the behavior of individuals during the generations of the DE.

navy.cs.vsb.cz36

ExamplesDE driven by CN centralities

navy.cs.vsb.cz37


Skanderova L., Fabian T., Differential Evolution Dynamic Analysis by Complex Network, submitted for publication.

navy.cs.vsb.cz38



navy.cs.vsb.cz39



navy.cs.vsb.cz40



navy.cs.vsb.cz41



navy.cs.vsb.cz42


navy.cs.vsb.cz43


navy.cs.vsb.cz44


navy.cs.vsb.cz45

ExamplesABC driven by CN centralities

Davendra D., Metlicka M., Ensemble Centralities based Adaptive Artificial Bee Algorithm , CEC 2015, Sendai, Japan

navy.cs.vsb.cz46



navy.cs.vsb.cz47



navy.cs.vsb.cz48



navy.cs.vsb.cz49



navy.cs.vsb.cz50

EA as complex network


navy.cs.vsb.cz51

Complex system overview

• Is evolutionary dynamics complex?

• What is it complex system?

• Can be complex system controlled?

navy.cs.vsb.cz52


Growth

Models

Cel

lula

r A

uto

mat

a

Emergence

Hierarchical ModelsFractals

ChaosComplex systems

Adaptive behavior

Computation

Coadap

tation Recursion

Stra

nge

att

ract

ors

Phase transition

Self organ

ization

Inco

mputab

ilityArtificial life, artificial intelligence, swarm intelligence, adaptation in natural processes ...

Complex networks, economical systems, social networks and systems, nervous system, cells and living things, including human beings, modern energy or telecommunication infrastructures, ...

Systems exhibiting deterministic chaos dynamics: economy, climate, physics, biology, astrophysics, numerical calculations, ...

Complexity of fractals in visualization, systems with chaotic behavior, static structures in physics and modeling of dynamic processes of growth, hierarchical models and information processing and communication,...

Numerical simulation, Kolmogorov algorithm complexity, P and NP problems, incomputability and physical limits of computation,...

navy.cs.vsb.cz53


• One of many possible definition of a complex system is like:

• A complex system is a system composed of interconnected parts that as a whole exhibit one or more properties (behavior among the possible properties) not obvious from the properties of the individual parts. This characteristic of every system is called emergence.

• A systems complexity may be of one of two forms: disorganized complexity and organized complexity. In essence, disorganized complexity is a matter of a very large number of parts, and organized complexity is a matter of the subject system (quite possibly with only a limited number of parts) exhibiting emergent properties.

navy.cs.vsb.cz54


Today a lot of various informal descriptions of complex systems have been put forward, and these may give some insight into their, very often interesting, properties. As mentioned in the special edition of Science journal(Vol. 284. No. 5411, 1999)) about complex systems highlighted several of these:

• A complex system is a highly structured system, which shows structure with variations.

• A complex system is one whose evolution is very sensitive to initial conditions or to small perturbations, one in which the number of independent interacting components is large, or one in which there are multiple pathways by which the system can evolve. This is very typical for deterministic chaos system.

• A complex system is one that by design or function or both is difficult to understand and verify.

• A complex system is one in which there are multiple interactions between many different components Complex systems are systems in process that constantly evolve and unfold over time.

• ...

navy.cs.vsb.cz55


A complex adaptive system has some or all of the following attributes*:• The number of parts (and types of parts) in the system and the

number of relations between the parts is non-trivial – however, there is no general rule to separate "trivial" from "non-trivial";

• The system has memory or includes feedback;• The system can adapt itself according to its history or feedback;• The relations between the system and its environment are non-

trivial or non-linear;• The system can be influenced by, or can adapt itself to, its

environment; and• The system is highly sensitive to initial conditions. • ...

* Johnson, Neil F. (2007). Two's Company, Three is Complexity: A simple guide to the science of all sciences. Oxford: Oneworld. ISBN 978-1-85168-488-5.

http://en.wikipedia.org/wiki/International_Standard_Book_Number

http://en.wikipedia.org/wiki/Special:BookSources/978-1-85168-488-5

navy.cs.vsb.cz56


Examples of complex systems include natural as well as artificial:

• ant colonies• human economies• social structures• climate• nervous systems• cells and living things• including human beings• modern energy or telecommunication infrastructures

and much more. Many systems of research and technology interest to humans are complex systems. Complex systems are studied by many areas of natural science, mathematics, physics, biology and social science. Fields that specialize in the interdisciplinary study of complex systems include systems theory, complexity theory, systems ecology and mainly cybernetics.

navy.cs.vsb.cz57


navy.cs.vsb.cz58

Complex system behavior

navy.cs.vsb.cz59


Biological self-organized systems

navy.cs.vsb.cz60

Complex system behavior Is animal behavior chaotic?

navy.cs.vsb.cz61


200 400 600 800 1000

0.2

0.4

0.6

0.8

1.0

200 400 600 800 1000

0.2

0.4

0.6

0.8

1.0

200 400 600 800 1000

0.2

0.4

0.6

0.8

1.0

200 400 600 800 1000

0.2

0.4

0.6

0.8

1.0

navy.cs.vsb.cz62


200 400 600 800 1000

0.2

0.4

0.6

0.8

1.0

200 400 600 800 1000

0.2

0.4

0.6

0.8

1.0

0.2 0.4 0.6 0.8 1.0

0.2

0.4

0.6

0.8

1.0

Chaos Randomness0.2 0.4 0.6 0.8 1.0

0.2

0.4

0.6

0.8

1.0

xk+1

xk

xk+1

xk

navy.cs.vsb.cz63

Complex system behavior Some Class 1 Behaviour

navy.cs.vsb.cz64


navy.cs.vsb.cz65


navy.cs.vsb.cz66


navy.cs.vsb.cz67

Presentation phase

navy.cs.vsb.cz68

Complex networks to CML

navy.cs.vsb.cz69


navy.cs.vsb.cz70


0 10 20 30 40 500

10

20

30

40

50

1

23

4

5

1

2

3

45

1

2

3

4

5

Dynamic of CML is then

understand like changes in

connections between

vertices

navy.cs.vsb.cz71



navy.cs.vsb.cz72


Figure 30 Example 6: Dynamics of the complex network with edges deactivation by real number

coefficient. Different colors including white one represent different vertices (sites) activation.

Figure 31 Example 7: Dynamics of the complex network with edges deactivation by real number

coefficient. Different colors including white one represent different vertices (sites) activation.

Figure 32 Example 8: Zoom of the network with 20 vertices in 500 iterations.

CONCLUSION

In this paper we have proposed method how to visualize and simulate (model) complex network

dynamics by means of the CML systems. CML modeling is well known in the domain of so-

called spatiotemporal deterministic chaos. For analysis and control of the CML systems has been

developed numerous techniques, based on classical mathematics as well as on the heuristic

methods. Based on this fact we have used and tested method, how to visualize CN dynamics as a

CML system. As a summarization and conclusion can be state this:

The main model of complex network, used for simulations here, has been based on

randomly initialized network with different number of vertices.

Edges between vertices have been also initialized randomly, however, roulette method

from genetic algorithms, have been used to add the new edge to the vertex, and/or

increase or decrease importance of existing edge. Roulette method has been used in order

to prefer richer vertices (i.e. vertices with more incoming edges) and thus support in this

way the main idea of complex networks with small world phenomenon.

Two different dynamics of CN has been used. The first one was based on fact that a

simple small CN has been initialized and then, after each iteration, a new vertex has been

added and selected edges adjusted. This is visualized in Figure 20. The second one has

been based on preliminary fact, that we already have developed network, with constant

number of vertices and dynamic itself depends only on the edge weights evolution.

All substantial and typical results are visualized via Figure 17 - Figure 32 and it can be

stated that our technique of CNCML is usable and complex behavior of CN can be

visualized in this way. It is also obvious, that methods of analysis and control of CMLs

can be also used for CNCML, because conversion classical CN to the CNCML basically

create specific, more complex version, of the CML and there are no restriction of use of

Dynamics of the complex network with edges deactivation by real number coefficient. Different colors including white one represent different vertices (sites) activation.

Zoom of the network with 20 vertices in 500 iterations.

navy.cs.vsb.cz73

Presentation phase

navy.cs.vsb.cz74

• Deterministic?

• Stochastic?

• Chaotic?

• Emergent?

What kind of behavior can we expect?

navy.cs.vsb.cz75

• Deterministic?


navy.cs.vsb.cz76


• Stochastic?

77 navy.cs.vsb.cz

Chaos - behavior

Wright A., Agapie A.: Cyclic and Chaotic Behavior in Genetic Algorithms. In: Proc. of Genetic and Evolutionary Computation Conference (GECCO),San Francisco, July 7-11 (2001)

78 navy.cs.vsb.cz

Chaos - behavior

navy.cs.vsb.cz79

Chaos - hamiltonian systems

• The study of Hamiltonian systems has its roots in the 19th century when it was introduced by Irish mathematician William Hamilton.

• For mechanical systems, typical feature of Hamiltonian systems is that no dissipation of energy occurs in them, so that mechanical Hamiltonian system is also the so-called conservative one.

• In general dynamical system theory the term “conservative” means that certain scalar function, having typical properties of energy, is preserved along system trajectories.

• The creation of chaos theory for Hamiltonian systems was contributed to by scientists such as Boltzman (who established the foundations of ergodic theory and discovered the contradiction between the reversibility of a system and irreversibility of its behavior) and Poincare.

• Hamiltonian systems included application in many areas of physics, such as plasma physics, quantum mechanics and others.

navy.cs.vsb.cz80

Chaos - dissipative systems

• Dissipative dynamic systems are systems where energy

escapes into the surroundings and state space volume is

reduced.

• Typical examples include weight on spring (dissipation

being caused by friction between the body and air and

energy losses inside the material), motion of a wheel,

electronic resonance circuits.

• Since the topics of dissipative dynamic systems are the

subject of a whole presentation, demonstration of a

exact real system will be given here.

• Well-known classical example of a dynamic system is

defined by the Lorenz system.

navy.cs.vsb.cz81

Chaos - universal features

• Deterministic chaos possesses many features that are common to chaotic

behavior irrespective of the physical system which is the cause of this

behavior.

• This common nature is expressed by the term universality so as to stress

the universal nature of the phenomena.

• The quantity and properties of the features as well as the complexity of links

between them are so extensive that they could make up a topic for a

separate publication.

• These include, in particular, Feigenbaum’s constants α and δ , the U-

sequence, Lyapunov exponents, self-similarity and processes by which a

system usually passes from deterministic behavior to chaotic behavior:

intermittence, period doubling, metastable chaos and crises.

• Another property which is, curiously, not included in the pantheon of

universalities will be mentioned at the beginning: the deterministic nature

and non-predictability of deterministic chaos.

navy.cs.vsb.cz82

Chaos - universal featuresDeterminism and Unpredictability of the Behavior of Deterministic Chaos - Sensitivity

to Initial Conditions

• The deterministic structure of systems which generate chaos and their unpredictability constitute another typical feature of the universal properties of deterministic chaos.

• It is actually irrelevant what type the chaotic system is (chemical, biological, electronic, economic, ...): it holds invariably that their mathematical models are fully deterministic (there is no room for randomness as such in them) and are unpredictable in their behavior.

navy.cs.vsb.cz83

Selected examplesMechanical System - Billiard

navy.cs.vsb.cz84

Selected examplesMechanical System – Mad Pendulum

Video

navy.cs.vsb.cz85

Selected examplesMechanical System – Mad Pendulum

navy.cs.vsb.cz86

Selected examplesAstronomy - the Three-Body Problem

α Centauri A a B … 23AU, 80 yearsα Centauri C 13000 AU

Artistic vision system of 3 stellar HD 188753 A, in the constellation Cygnus. In this

triple star system orbiting planets like Jupiter around the brightest star in about 3.3

day! .

navy.cs.vsb.cz87


navy.cs.vsb.cz88


Video

navy.cs.vsb.cz89

Selected examplesElectronic System - Chua’s Circuit

• Electronic circuits are among the most popular systems used to demonstrate deterministic chaos.

• Their popularity comes from the fact that electronic circuits are easy to set up and provide fast response to impulse.

• Typical representatives of electronic circuits with deterministic chaos include Chua’s circuit, whose hardware design and behavior are shown in next slide.

• Chua’s circuit can be described mathematically, which can be used to simulate the behavior of the circuit.

navy.cs.vsb.cz90


• The core of Chua’s circuit is a nonlinear resistor, sometimes called Chua’s diode.

• Chua’s attractor visualized by the program Mathematica (left) and on the oscilloscope connected to its hardware implementation.

• In

the nonlinear resistor g(x) is represented by

navy.cs.vsb.cz91


navy.cs.vsb.cz92


Video

navy.cs.vsb.cz93

Selected examplesSpatiotemporal Chaos

• This is a spatiotemporally coupled system with the development of n equations that affect each other via a coupling constant, usually denoted ε.

• CML can be regarded as a field of kind of “oscillators” which affect each other.

• Mathematical description of a CML using an iteration equation for its activity consists in where the function which is denoted f(...) represents the iteration equation.

navy.cs.vsb.cz94

Selected examplesSpatiotemporal Chaos

navy.cs.vsb.cz95

Selected examplesHidden attractors

• An oscillation in a dynamical system can be easily localized numerically if initial conditions from its open neighborhood lead to long-time behavior that approaches the oscillation.

• Such oscillation (or a set of oscillations) is called an attractor, and its attracting set is called the basin of attraction.

• Thus, from a computational point of view in applied problems of nonlinear analysis of dynamical models, it is essential to regard attractors as self-excited attractors or hidden attractors depending on the simplicity of finding its basin of attraction in the phase space.

GA Leonov, NV Kuznetzov, Hidden attractors in dynamical systems. From hidden oscillations in Hilbert-Kolmogorov, Aizerman, and Kalman problems to hidden chaotic attractor in Chua circuits

navy.cs.vsb.cz96

Selected examplesHidden attractors

Remember, chaos has been observed inside EAs

Are hidden attractors in EAs dynamic too?

What happen if HA is met?

97 navy.cs.vsb.cz

Chaos – visualizations

navy.cs.vsb.cz98


• Emergent?

navy.cs.vsb.cz99

Selected examplesChaos - Links

• Lorenz attractor: http://www.youtube.com/watch?v=iu4RdmBVdps

• Chaos document:http://www.youtube.com/watch?v=EF5Wvi_Iiy4

• Double Pendulum Chaos:http://www.youtube.com/watch?v=QXf95_EKS6E

• Fractals – Hunting The Hidden Dimension –document about fractal geometry and chaos: https://www.youtube.com/watch?v=s65DSz78jW4

• The Strange New Science of Chaos: https://www.youtube.com/watch?v=fUsePzlOmxw

http://www.youtube.com/watch?v=iu4RdmBVdps

http://www.youtube.com/watch?v=EF5Wvi_Iiy4

http://www.youtube.com/watch?v=QXf95_EKS6E

https://www.youtube.com/watch?v=s65DSz78jW4

https://www.youtube.com/watch?v=fUsePzlOmxw

navy.cs.vsb.cz100

Presentation phase

navy.cs.vsb.cz101

Control EAs dynamics

Caenorhabditis elegans

• In its Neural Network:

– Neurons: ~ 300

– Synapses: ~ 3000

navy.cs.vsb.cz102


The worm Caenorhabditis elegans has 297 nerve cells. The neurons switch one another on or off, and, making 2345 connections among themselves. They form a network that stretches through the nematode’s millimeter-long body.How many neurons would you have to commandeer to control the network with complete precision?The answer is: 49

-- Adrian Cho, Science, 13 May 2011, vol. 332, p 777

navy.cs.vsb.cz103


“…very few individuals (approximately 5%) within honeybee swarms can guide the group to a new nest site.”

I.D. Couzin et al., Nature, 3 Feb 2005, vol. 433, p 513

These 5% of bees can be considered as “controlling” or “controlled” agents

navy.cs.vsb.cz104


• How many controllers to use?

• Where to put them (which nodes to “pin”)?

• Objective: To achieve cost-effective control (e.g. synchronization) with good performance

navy.cs.vsb.cz105

Complex system behavior Control

• Off line control

• Realtime control


navy.cs.vsb.cz106

Evolutionary algorithms as a feedback loop dynamical systems

navy.cs.vsb.cz107

Want to know more?

EA as CN

• Zelinka I, Davendra DD, Snasel V, Senkerik R, Jasek R (2010) Preliminary investigation on relations between complex networks and evolutionary algorithms dynamics. In: Computer Information Systems and Industrial Management Applications (CISIM) 31.

• Zelinka I, Davendra DD, Senkerik R, Jasek R (2011) Do evolutionary algorithm dynamics create complex network structures? In: Complex Systems, Vol. 20, Issue 2.

• Zelinka, Ivan, Snasel, Vaclav, Abraham, Ajith (Eds.) , Handbook of Optimization, Springer, 2013

• Zelinka I. et. al, Evolutionary algorithms dynamics and its hidden complex network structures. CEC 2014, p. 3246 – 3251, Beijing, China

• Davendra D, Zelinka I, Senkerik R, Pluhacek M (2014) Complex network analysis of discrete self-organising migrating algorithm. In: Nostradamus 2014: Prediction, Modeling and Analysis of Complex Systems, pp 161–174.

• Skanderova L., Fabian T., Differential Evolution Dynamic Analysis by Complex Network, submitted for publication.

• Davendra D., Metlicka M., Ensemble Centralities based Adaptive Artificial Bee Algorithm , CEC 2015, Sendai, Japan

navy.cs.vsb.cz108

Want to know more?

CN in Eas

• Soleimani-pouri M, Rezvanian A, Meybodi MR (2014) An Ant Based Particle Swarm Optimization Algorithm for Maximum Clique Problem in Social Networks, Springer International Publishing, pp 455–462.

• Wua Q, Hao JK (2014) A review on algorithms for maximum clique problems. European Journal of Operational Research.

• Gong M, Cai Q, Chen X, Ma L (2014) Complex network clustering by multiobjectivediscrete particle swarm optimization based on decomposition. IEEE Transactions on Evolutionary Computation 18:82–97 .

• Li Y, Liu J, Liu C (2014) A comparative analysis of evolutionary and memetic algorithms for community detection from signed social networks. Soft Computing 18:329–348

• Ashlock D, Smucker M, Walker J (1999) Graph based genetic algorithms. In: Evolutionary Computation, IEEE, pp 1362–1368.

• Mabu S, Hirasawa K, Hu J (2007) A graph-based evolutionary algorithm: Genetic network programming (gnp) and its extension using reinforcement learning. Evolutionary Computation 15:369–398 .

navy.cs.vsb.cz109

Conclusions

• EA behavior can be visualized as a complex network – CN

• EAs can exhibit different classes of behavior• Chaos can be also observed inside EA• CN can be evaluated by classical mathematical tools

developed for CN• CN can be converted to CML• CML can be controlled…• … then EAs dynamics can be controlled too• EA is feedback loop control system

navy.cs.vsb.cz110

Conclusions

• Graphical summarization

111 navy.cs.vsb.cz

THANK YOU FOR YOUR ATTENTION

[email protected]

www.ivanzelinka.eu

mailto:[email protected]

http://www.ivanzelinka.eu

navy.cs.vsb.cz112

Copyright

This didactic material is meant for the personal use of the student only,and is copyrighted. Its reproduction, even for a partial utilization, isstrictly forbidden in compliance with and in force of the law on Authorsrights.

Copyright©NAVY.CS.VSB.CZ

evolutionary algorithms as a complex dynamical systems

Documents