1

Evolvability Analysis for Evolutionary Robotics

Sung-Bae Cho

Yonsei University, Korea

2

Agenda

Motivation Analysis framework of evolution

– Adaptive evolution– Adaptive behaviors– Evolutionary pathways

Evolution of fuzzy logic controller Simulation results Summary

3

Motivation

Chances

Innovative

functional

structures

Increased

complexity

Desirable EvolutionEvolutionary Phenomena

Necessity

Random genetic drift

Adaptivity

4

Evolutionary Routes

Motivation

AdaptiveEvolution

HighEvolvability

LowEvolvability

Non-AdaptiveEvolution

GoodSolution

BadSolution

DesirableEvolutionary Causes

and Effects

High probability

Low probability

Emergence

AdaptiveBehavior

Can the same results be obtained? Adaptive evolution ( ) What properties are genetically preferred? Adaptive behaviors ( ) How the solutions are formed? Evolutionary pathways to the solutions ( ) Behavioral properties? Emergence ( )

1

2

1

2

4

3

3

4

5

Analysis Framework of Evolution

EvolutionAnalysis

EvolutionaryActivity

Statistics

EvolutionaryActivity

Statistics

SchemaAnalysis

SchemaAnalysis

ObservationalEmergence

ObservationalEmergence

Analysis ofEvolution

AdaptiveEvolution

AdaptiveBehavior

Emergence

EvolutionaryPathways

BehaviorAnalysis

6

Role of Analysis Components

Adaptive evolution– Does the evolving system maintains a good level of

evolvability, especially in a real-world problem?

Adaptive behavior– What properties make certain components more adaptive?

Evolutionary pathways– How the solutions have evolved, i.e., evolutionary pathways?

Application of the analysis framework to a real-world problem

Analysis Framework

7

Definitions of Evolvability

The capacity to produce good solutions via evolution

Genome’s ability to produce adaptive variants when acted on by the genetic system (Wagner and Altenberg, 1996)

Capacity to generate heritable phenotypic variation (Kirshner and Gerhart, 1998)

Capacity to create new adaptations, and especially new kinds of adaptations, through the evolutionary process (Bedau and Packard, 1992)

Analysis Framework

8

Evolvability Measures

Evolvability as the rate of complexity increase– By Chrystopher L. Nehaniv

– maxcpx gives the largest complexity of any entity at time t– The complexity of an entity is the least number of

hierarchically organized computing levels needed to construct an automata model of a target system

– Krohn-Rhodes algebraic automata theory and finite semigroup theory

Evolutionary activity statistics– By Mark A. Bedau

)()()( tmaxcpx1tmaxcpxtEv

Analysis Framework

9

Evolutionary Activity Statistics (1)

Evolutionary activity– A counter, , of the ith component at time t

– Updated as the component persistsInherited with reproductionInitialized when the component changes, e.g. mutationUpdate function should be chosen carefully

according to the problems at hand

tk

ii kta )()(

)(tai

)(ki

Analysis Framework

10

Evolutionary Activity Statistics (2)

Mean activity:

– D(t) is the number of component I at time t with ai(t)>0

– Represents continual adaptive success of components

New activity:

– is the number of components I with ai(t)>0

– Represents adaptive innovations flowing into the system

)(

)()(

tD

tatA i

i

cum

1

0

),()(

1)(

a

aanew atC

tDtA

),( atC

Analysis Framework

11

Evolutionary Activity Statistics (3)

Need to measure evolvability in two models– Target model– Shadow model

To screen off non adaptive evolutionary forces

Analysis Framework

12

Schema Analysis

Definition (Holland, 1968)– A similarity template that designates a set of chromosomes

having same alleles at certain loci Consists of a set of characters and don’t-cares Example

– Character set = {0,1}, don’t care=#– #0000 {10000, 00000}– #111# {01110, 01111, 11110, 11111}

Adaptive schema = the size of the set that this schema describes increases

Analysis Framework

13

Observational Emergence

Emergence– “creation of new properties” – Morgan, C.L., Emergent

Evolution, Williams and Norgate, 1923

Observational emergence– Proposed by N.A. Bass, 1992

S : structure (system, organization, organism, machine, …)

P : property observed by observational mechanism, Obs

Analysis Framework

14

Fuzzy Logic Controller for Mobile Robot

Evolution of Fuzzy Logic Controller

15

FLC Parameters for Khepera Robot

Input variables : 8 proximity sensors of Khepera mobile robot Output variables : 2 motors of Khepera mobile robot

Linguistic values of fuzzy sets

Membership function of fuzzy sets

Evolution of Fuzzy Logic Controller

16

Gene Encoding of FLC

• 8 proximity sensors

• 2 motors

8 INPUT 2 OUTPUT 20 RULES

Gene representation

for an individual

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

VF F M C VC

Encoding of a membership function

of a variable

00 40 31 21 0 41

d0 d1 d2 d3 d4 d5 d6 d7 v0 v1

21 01 30 10

1 2

variable toggle flag

rule toggle flag

1 conditional part

2 consequent part

Decoding of a rule

Evolution of Fuzzy Logic Controller

17

Experimental Setup

Population size : 50 Maximum generation : 1000 Overlapped population by 50% with elitism Crossover rate : 0.5 Mutation rate : 0.01

Evolutionary activity

Measuring evolvability in two models– Target model– Neutral shadow : no selective pressure

To screen off non adaptive evolutionary forces

otherwise0

tatexistsigenotypeif)()( 0

t

ii

dttnta

Simulation Results

18

Adaptive Evolution

0 200 400 600 800 10000

0.5

1

1.5

2

2.5

3

3.5x 10

4

Generation

To

tal A

ctiv

ity

Fuzzy Model Neutral Shadow

0 200 400 600 800 10000

0.02

0.04

0.06

0.08

0.1

0.12

Generation

Ne

w A

ctiv

ity

Fuzzy Model Neutral Shadow

Evolutionary activity

Mean activity

New activity

tk

ii kta )()(

)(

)()(

tD

tatA i

i

cum

1

0

),()(

1)(

a

anew atC

tDtA

Simulation Results

19

Adaptive Behavior

Salient Rules

0 100 200 300 400 500 600 700 800 900 10000

1000

2000

3000

4000

5000

6000

Generation

Act

ivat

ion

SR1 SR

2

SR3

SR4

SR7

SR6

SR5

SR8

SR9

SR10

SR12

SR

11

Simulation Results

With SR2 Without SR2 With SR8 Without SR8 With SR10 Without SR10

20

Schema Analysis

0 100 200 300 400 500 600 700 800 900 10000

1000

2000

3000

4000

5000

6000

Generation

Activa

tion

SR1 SR

2

SR3

SR4

SR7

SR6

SR5

SR8

SR9

SR10

SR12

SR

11

Best Individual

Salient Rules

Simulation Results: Evolutionary Pathways

21

Rule B2 and B7

S{1} S{4}B{2}

Activities of instances of

schemata S{1}, S{4}, and B{2}

Activities of instances of

schemata S{6} and B{7}

S{6} B{7}

Simulation Results: Evolutionary Pathways

22

Parameters of Emergence

Simulation Results: Observational Emergence

23

Turning Around

First-order structures Three Obs1s of first-order structures

Int

A Obs2 of a second-order structure S2

• The property observed by Obs2 of S2 constructed through the interactions of three first-order structures is quite different from the properties observed by Obs1( ),

By the definition of observational emergence

Turning around behavior (Obs2(S2)) is observationally emergent

17

15

12 111

,, SSS1

1iS }7,5,2{i

Simulation Results: Observational Emergence

24

Smooth Cornering

Int

First-order structures Two Obs1s of the first-order structures

A Obs2 of a second-order structure S2

• The property observed by Obs2 of S2 constructed through the interactions of the two first-order structures is quite different from the properties observed by Obs1( ),

By the definition of observational emergence

Smooth cornering behavior (Obs2(S2)) is observationally emergent

1

1iS }5,2{i

17

12 11,SS

Simulation Results: Observational Emergence

25

Summary

Application of evolvability measure to a real-world problem

Illustration of evolutionary pathways to the best individual

The evolvability measure shows that the performance of the best individual is the results of its rules’ adaptability

Schema analysis shows that most of the rules of the best individual are the combination of the rules of earlier stage of evolution