iwlcs'2006: a further look at ucs classifier system

A Further Look at UCS Cl ifi S tClassifier System

Albert Orriols-PuigEster Bernadó-Mansilla

Research Group in Intelligent SystemsEnginyeria i Arquitectura La Salle

Ramon Llull UniversityBarcelona, Spain, p

Aim

Provide a deep insight into UCSp g

Introduce a fitness sharing scheme in UCS

Highlight the differences between XCS and UCS

Slide 2GRSI Enginyeria i Arquitectura la Salle

Outline

1. Description of XCS

2. Description of UCS

3 Diff b t XCS d UCS3. Differences between XCS and UCS

4 Test-bed4. Test-bed

5. Experimentation

6. Conclusions


1. Description of XCS1. Description of XCS2. Description of UCS3. Differences b. XCS and UCS4. Test-bedp

In single-step tasks:

5. Experimentation6. Conclusions

Environment

g p

Problem instance

1 C A P ε F num as ts exp3 C A P ε F num as ts exp5 C A P ε F num as ts exp

Match Set [M]Selected

action


Population [P] Match set generation

5 C A P ε F num as ts exp6 C A P ε F num as ts exp

…Prediction Array

REWARD


…

A ti S t [A]

c1 c2 … cn

Random Action


C

Action Set [A]

Selection, Reproduction, mutation

Deletion ClassifierParameters

Update6 C A P ε F num as ts exp…Genetic Algorithm

Update


2. Description of UCS1. Description of XCS2. Description of UCS3. Differences b. XCS and UCS4. Test-bedp

Only for single-step tasks


y g p

Environment

M t h S t [M]P bl i t

Population [P]

1 C A acc F num cs ts exp3 C A acc F num cs ts exp5 C A acc F num cs ts exp

Match Set [M]Problem instance+

output class

1 C A acc F num cs ts exp2 C A acc F num cs ts exp3 C A acc F num cs ts exp4 C A acc F num cs ts exp

Population [P]

Classifier

6 C A acc F num cs ts exp…

correct set4 C A acc F num cs ts exp5 C A acc F num cs ts exp6 C A acc F num cs ts exp

…

ClassifierParameters

UpdateMatch set generation

C t S t [C]

correct setgeneration

ExperienceCorrectacc #

=Selection, Reproduction, mutation

Deletion 3 C A acc F num cs ts exp6 C A acc F num cs ts exp

…

Correct Set [C]

p

νaccFitness =Genetic Algorithm

…


3. Differences between XCS and UCS1. Description of XCS2. Description of UCS3. Differences b. XCS and UCS4. Test-bed5. Experimentation6. Conclusions

Three main differences:

– Explore regime

– Parameter updates

– Fitness computation



Explore Regime

XCS Populations evolved

c1 c2 … cnPrediction

Array

Random action

evolvedMaximal general classifiers predicting the correct classMaximal general classifiers predicting the incorrect classSo XCS also explores low rewarded niches

[A] 1. 000 0#######:0 1000 0 …2. 000 1#######:0 0 0 …

So, XCS also explores low rewarded niches

Complete action map

UCS

…action map

EnvironmentExample + class

Maximal general classifiers predicting the correct classAlways exploring the class of the input instance

[C]1. 000 0#######:0 1000 0 …2. 000 1#######:1 0 0 …

…Best

action map



Parameter Updates

XCS rdXCS

( )ttt pRpp −+=+ β1

e of

the

rew

ar

β=0.2

( )tttt pR εβεε −−+=+1

Influ

ence

UCS timet+8t+1 t+2 t+3 t+4 t+5 t+6 t+7

d

experiencecorrectnumber

=acc

of th

e re

war

dIn

fluen

ce

time


time


Fitness Sharing: XCS shares fitness but UCS does notThe advantages of fitness sharing are empiricallyThe advantages of fitness sharing are empirically demonstrated (Bull & Hurst, 2002)

Scheme of fitness sharing in UCS:

⎨⎧ >

=∈

accaccifk Ccl

1 0][

⎩⎨∈ otherwiseaccaccCcl να )/( 0

][We share the accuracywith all the classifiers

in [M]

k

∑∈

=

][·

·'

Mclclcl

clclcl

i

iinumk

numkk

∈ ][Mcli

)'·( FkFF −+= β


)(β

4. Test-bed1. Description of XCS2. Description of UCS3. Differences b. XCS and UCS4. Test-bed5. Experimentation6. Conclusions

Problems

– Parity: two-class problem

01001010

Condition length (l)

:1 Number of 1 mod 2

Complexity: It does not permit any generalization

– Decoder: multi-class problem

:5 Integer value of the input000110Condition length (l)

Complexity: the number of classes increases with the condition length



Problems– Imbalanced Multiplexer: Imbalanced two-class problem

000 10000100

Condition length (l)

:1Value of the position bit

indicated by the selection bits

The class labeled as 1 is under-sampled

Complexity: For high imbalances there is a poorir = proportion between majority and minority class examples

– Position: imbalanced multi-class problem

p y g psampling of minority class examples i = log2ir

:2Position of the left-most

one valued bit000110Condition length (l)

:2 one-valued bit

Complexity: the number of classes and the imbalance level increase with the condition length

000110


g


Problems– Multiplexer with Alternating noise

0000 1000010011100101 :1Value of the position bit

indicated by the selection bits

The output is flipped with probability Px

Complexity: The system receives noisy instancesComplexity: The system receives noisy instances


5. Experimentation1. Description of XCS2. Description of UCS3. Differences b. XCS and UCS4. Test-bedp5. Experimentation6. Conclusions

We used the five binary-input problems to test:– XCS– UCS without fitness sharing: UCSns– UCS with fitness sharing: UCSs

To permit comparison between XCS and UCS, we measured the percentage of the best action map achieved

We configured XCS with the following parameters:

N=25 |[O]|, α=0.1, ν=5, Rmax = 1000, ε0=1, θGA=25, β=0.2,χ=0.8, μ=0.4, θdel=20, δ=0.1, θsub=20, P#=0.6

selection=tournament mutation=niched

And for UCS, we added:

selection=tournament, mutation=niched,GAsub=true, [A]sub=false

acc0 = 0.999, ν=5


,

5. Experimentation5 2 The Parity Problem

1. Description of XCS2. Description of UCS3. Differences b. XCS and UCS4. Test-bed

5.2. The Parity Problem

Parity with l=3 to l=9


Parity with l 3 to l 9Complete Action Map Par3

000:0 100:1 000:1 100:0

001:1 101:0 001:0 101:1

010:1 110:0 010:0 110:1

011:0 111:1 011:1 111:0When an optimal classifier is - Correct optimal classifiers

- Incorrect optimal classifiers

pdiscovered, the fitness of the other classifiers in thepopulation is not affected

Difficulty: Lack of fitness guidance

XCS: 00#001#:0 P = 500 ε=500XCS: 00#001#:0 P = 500, ε=500UCS: 00#001#:0 acc = 0.5


5. Experimentation5 3 The Decoder Problem


5.3. The Decoder Problem

Decoder with l=3 to l=6


Decoder with l 3 to l 6Complete Action Map Dec3

000:0 1##:0 #1#:0 ##1:0XCS cannot solve Dec6 in 100 000001:1 1##:1 #1#:1 ##0:1

010:2 1##:2 #0#:2 ##1:2

011:3 1##:3 #0#:3 ##0:3

XCS cannot solve Dec6 in 100,000 learning iterations:

UCSs slightly improves UCSns

100:4 0##:4 #1#:4 ##1:4

101:5 0##:5 #1#:5 ##0:5

110:6 0##:6 #0#:6 ##1:6110:6 0##:6 #0#:6 ##1:6

111:7 0##:7 #0#:7 ##0:7- Correct optimal classifiers


Difficulty: Multiple classes


5. Experimentation5 3 The Decoder Problem


5.3. The Decoder Problem

Fit Dil i XCS (B t t l 2003)


Fitness Dilemma in XCS (Butz et al 2003)

Condition Class Correct R ti

P ErrorRatio

###1# 2 0.125 125 218.75

##01# 2 0.250 250 375

Error increases until P=500

#001# 2 0.500 500 500

0001# 2 1 1000 0


5. Experimentation5 4 The Imbalanced Multiplexer Problem


5.4. The Imbalanced Multiplexer Problem

Imbalanced 11 Mux for i=0 to i=9


Imbalanced 11-Mux for i=0 to i=9

Complete Action Map for the Multiplexer Problem000 0####### 0 000 1####### 1 000 0####### 1 000 1####### 0

Example: for i=6

000 0#######:0 000 1#######:1 000 0#######:1 000 1#######:0

001 #0######:0 001 #1######:1 001 #0######:1 001 #1######:0

010 ##0#####:0 010 ##1#####:1 010 ##0#####:1 010 ##1#####:0

011 ###0####:0 011 ###1####:1 011 ###0####:1 011 ###1####:0

Classifier acc F

### ########:0 0.9928 0.9302

000 0#######:0 1.00 1.00UCSs can solve the multiplexer t i 9 d XCS t i 8011 ###0####:0 011 ###1####:1 011 ###0####:1 011 ###1####:0

100 ####0###:0 100 ####1###:1 100 ####0###:1 100 ####1###:0

101 #####0##:0 101 #####1##:1 101 #####0##:1 101 #####1##:0

110 ######0#:0 110 ######1#:1 110 ######0#:1 110 ######1#:0

• Similar values of fitness• The overgeneral has more genetic opportunities

up to i=9 and XCS up to i=8

110 ######0#:0 110 ######1#:1 110 ######0#:1 110 ######1#:0

111 #######0:0 111 #######1:1 111 #######0:1 111 #######1:0

- Correct optimal classifiers- Incorrect optimal classifiers

The system were configured following the guidelines in (Orriols and Bernadó, 2006)

Difficulty: As the imbalance level increases, the sampling rate of minority class examples decreases.


That is, low search rate for promising rules predicting the minority class

5. Experimentation5 5 The Position Problem


5.5. The Position Problem

Position with l=3 to l=9


Position with l 3 to l 9

Complete Action Map for the Pos3

000:0 1##:0 #1#:0 ##1:0XCS h t l ll th t001:1 1##:1 #1#:1 ##0:0

01#:2 1##:2 #0#:2

1##:3 0##:3

XCS has to explore all the correct action map

UCS only explores the best action 1##:3 0##:3- Correct optimal classifiers


y pmap

Difficulty: Class imbalance and multiple classes.

Maximum imbalance ratio between classes:

irmax = 2l-1


5. Experimentation5 6 The Multiplexer with Alternating Noise


5.6. The Multiplexer with Alternating Noise

20-bit Multiplexer with alternating noise


g

In all cases, optimal classifiers are

Complete Action Map for the Multiplexer Problem

0000 0###############:0 0000 1###############:1 0000 0###############:1 0000 1###############:0

0001 #0##############:0 0001 #1##############:1 0001 #0##############:1 0001 #1##############:0pcontinuously created and removed

Windowed averages make oscillate the parameters of XCS’s classifiers

Optimal classifiers are considered as

0010 ##0#############:0 0010 ##1#############:1 0010 ##0#############:1 0010 ##1#############:0

0011 ###0############:0 0011 ###1############:1 0011 ###0############:1 0011 ###1############:0

0100 ####0###########:0 0100 ####1###########:1 0100 ####0###########:1 0100 ####1###########:0

0101 #####0########## 0 0101 #####1########## 1 0101 #####0########## 1 0101 #####1########## 0Optimal classifiers are considered as inaccurate

A non-fitness sharing scheme presents slightly better results

0101 #####0##########:0 0101 #####1##########:1 0101 #####0##########:1 0101 #####1##########:0

0110 ######0#########:0 0110 ######1#########:1 0110 ######0#########:1 0110 ######1#########:0

0111 #######0########:0 0111 #######1########:1 0111 #######0########:1 0111 #######1########:0

1000 ########0#######:0 0000 ########1#######:1 0000 ########0#######:1 0000 ########1#######:0

- Correct optimal classifiers- Incorrect optimal classifiers

1001 #########0######:0 0001 #########1######:1 0001 #########0######:1 0001 #########1######:0

1010 ##########0#####:0 0010 ##########1#####:1 0010 ##########0#####:1 0010 ##########1#####:0

1011 ###########0####:0 0011 ###########1####:1 0011 ###########0####:1 0011 ###########1####:0

1100 ############0###:0 0100 ############1###:1 0100 ############0###:1 0100 ############1###:01100 ############0###:0 0100 ############1###:1 0100 ############0###:1 0100 ############1###:0

1101 #############0##:0 0101 #############1##:1 0101 #############0##:1 0101 #############1##:0

1110 ##############0#:0 0110 ##############1#:1 0110 ##############0#:1 0110 ##############1#:0

1111 ###############0:0 0111 ###############1:1 0111 ###############0:1 0111 ###############1:0

Difficulty: The system receive examples labeled wrongly

XCS: Optimal incorrect classifiers will receive Px positive rewards


UCS: The system will need to create classifierscovering noisy examples. Lots of coverings.

6. Conclusions1. Description of XCS2. Description of UCS3. Differences b. XCS and UCS4. Test-bed5. Experimentation6. Conclusions

We introduced UCS, and specialization of XCS

We improved UCS by introducing fitness sharingWe improved UCS by introducing fitness sharing– Fitness sharing is necessary in imbalanced datasets, avoiding

overgeneral classifiers when the optimal classifiers are discoveredg p

UCS presents some advantages in the tested domains:It does not suffer from fitness dilemma– It does not suffer from fitness dilemma

– It only explores the correct class, decreasing the convergence time in problems with large complete action mapsp g p p

XCS is more general, and it can be applied to multi-step problems

As further work, we want to analyze the differences of UCSs and XCS with bilateral accuracy


A Further Look at UCSCl ifi S tClassifier System

Albert Orriols-PuigEster Bernadó-Mansilla

Research Group in Intelligent SystemsEnginyeria i Arquitectura La Salle

Ramon Llull UniversityBarcelona, Spain, p

iwlcs'2006: a further look at ucs classifier system

Education

ts exp3 ca p f num

ts exp5 ca p f num

ts exp6 ca p f num

ts exp2 ca p f num

ts expcaction set aselection

description of xcs2

description of ucs1

description of xcs1