effects of a deterministic hill climber on hboa

Effects of a Deterministic Hill Climber on hBOA

Elizabeth Radetic, Martin Pelikan, and David E. Goldberg

Missouri Estimation of Distribution Algorithms Laboratory (MEDAL)

University of Missouri, St. Louis, MOhttp://medal.cs.umsl.edu/

[email protected] [email protected]

Illinois Genetic Algorithms Laboratory

University of Illinois at Urbana-Champaign, Urbana, ILhttp://www-illigal.ge.uiuc.edu/ [email protected]

Elizabeth Radetic, Martin Pelikan, and David E. Goldberg Effects of a Deterministic Hill Climber on hBOA

http://medal.cs.umsl.edu/

[email protected]

[email protected]

http://www-illigal.ge.uiuc.edu/

[email protected]

Background & Motivation

Hybrid Estimation of Distribution Algorithms (EDAs)

I For very large or complex problems, scalability may not besufficient; additional efficiency enhancement may be required

I Hybridization is a popular method of efficiency enhancementfor EDAs and other GEAs

I A hybrid combines a local search algorithm with a globalsearcher such as an EDA

Hybrid hBOA

I The hierarchical Bayesian optimization algorithm (hBOA) isfrequently hybridized

I This study focuses on hBOA combined with a deterministichill climber (DHC)

I Examine the effects of varying the parameters of DHCI Study the effects of variance reduction on model building


Outline

1. Algorithms

2. Goals & Methodology

3. Test Problems

4. Results

5. Discussion of Results

6. Summary and conclusions.


Algorithms

Hierarchical Bayesian Optimization Algorithm (hBOA)

I Randomly initialize a population of binary strings

I Select promising solutions from the population

I Build a Bayesian network from the selected solutions

I Generate new solutions from the constructed model

I Incorporate new solutions into the population using restrictedtournament replacement


Algorithms

Deterministic Hill Climber (DHC)

I Flip one bit in the string that will produce the most fitnessimprovement

I Repeat until a specified quality has been reached, a maximumnumber of flips has been performed, or no more improvementis possible

I hBOA has frequently been combined with DHC to improveperformance (e.g., Pelikan & Goldberg, 2003; Pelikan &Hartmann, 2006; Pelikan, Katzgraber, & Kobe, 2008)


Goals & Methodology

Experiment Goals

I Examine the effects of DHC on performance of hBOA

I Determine optimal settings for DHC

Methodology

I Tune the frequency of local search, i.e. the proportion ofstrings in the population on which DHC operates

I Tune the duration of local search, i.e. the maximum numberof flips per string

I CompareI Population sizeI Number of generations until optimum


Test Problems

Test Problems

I Trap-5

I 2-D Ising Spin Glass

I 3-CNF MAXSAT


Test Problems

Trap-5

I Concatenation of non-overlapping subproblems of 5 bits

I The fitness of each subproblem is determined as

ftrap5(u) =

{

5 if u = 5,

4 − u otherwise,

where u is the number of ones in the subproblem

I All subproblems are added together to determine the string’sfitness


Test Problems

Trap-5

I PropertiesI Separable, non-overlapping subproblemsI Subproblems are fully deceptiveI Difficult for local search, simple GA

I ParametersI Problem sizes of 100-350 bitsI Population size determined empirically using bisection method

I 10 independent bisections of 10 runs each


Test Problems

2-D Ising Spin Glass

I Spins are arranged on a 2-D grid

I Each spin si gets a value from +1,−1

I Each edge between neighboring spins si and sj is assigned areal value Ji,j

I The task is to find a configuration of spins to minimize theenergy, specified as

H(s) = −

n∑

i,j=0

Jijsisj

I Spins are represented as bits in the binary string, 1 for +1, 0for −1


Test Problems

2-D Ising Spin Glass

I PropertiesI More complex fitness landscape than trapI Not fully decomposable into subproblems of bounded order,

substructures overlapI Solvable in polynomial time

I ParametersI Problem sizes 8x8, 10x10, 12x12, 14x14, 16x16 (64 - 256 bits)I 1000 instances

I For each instance, Jij randomly set to +1 or −1

I Population size determined using bisection method, 10successful runs per instance


Test Problems

MAXSAT

I Instances consist of 3-CNF formulas: conjunctions ofdisjunctions of 3 literals

I The task is to find an assignment of Boolean variables tosatisfy the maximum number of clauses

I Bits in the string represent the variables, with 1 representingtrue and 0 representing false


Test Problems

MAXSAT

I PropertiesI Complex fitness landscapeI Not fully decomposable into subproblems of bounded order,

substructures overlapI Not solvable in polynomial time

I ParametersI Problem sizes of 50 and 75 variablesI 100 randomly-generated instancesI Population size determined using bisection method, 10

successful runs per instance


Experimental Results

Trap-5 (350 bits)

I Reduction factor, speed-up: ratio of values without DHC tothose with DHC

0

10

20

30

40

50

60

0 0.2 0.4 0.6 0.8 1

Pop

ulat

ion

Siz

e R

educ

tion

Fac

tor

Max. Flips (% of Prob. Size)

pdhc0.10.20.30.4

0.50.60.70.8

0.91

0.8

0.9

1

1.1

1.2

1.3

1.4

1.5

1.6

1.7

1.8

0 0.2 0.4 0.6 0.8 1G

ener

atio

ns S

peed

-Up


pdhc0.10.20.30.4

0.50.60.70.8

0.91



Trap-5 (350 bits)

0

10

20

30

40

50

60

70

80

90

100

0 0.2 0.4 0.6 0.8 1

Eva

luat

ions

Spe

ed-U

p


pdhc0.10.20.30.4

0.50.60.70.8

0.91

0

1

2

3

4

5

6

7

0 0.2 0.4 0.6 0.8 1

Flip

s pe

r E

valu

atio

nMax. Flips (% of Prob. Size)

pdhc0.10.20.30.4

0.50.60.70.8

0.91



Trap-5

100

1000

10000

100000

350 300 250 200 150 100

Pop

ulat

ion

Siz

e

Problem size (# of bits)

No DHCWith DHC

1

10

100

350 300 250 200 150 100G

ener

atio

nsProblem size (# of bits)

No DHCWith DHC



Ising Spin Glass (16x16, 256 bits)

0

2

4

6

8

10

12

14

16

0 0.2 0.4 0.6 0.8 1

Pop

ulat

ion

Red

uctio

n F

acto

r


pdhc0.10.20.30.4

0.50.60.70.8

0.91.0

1

1.5

2

2.5

3

3.5

4

4.5

0 0.2 0.4 0.6 0.8 1

Gen

erat

ion

Spe

ed-u

pMax. Flips (% of Prob. Size)

pdhc0.10.20.30.4

0.50.60.70.8

0.91.0



Ising Spin Glass (16x16, 256 bits)

0

10

20

30

40

50

60

70

0 0.2 0.4 0.6 0.8 1

Eva

luat

ion

Spe

edup


pdhc0.10.20.30.4

0.50.60.70.8

0.91.0

0

1

2

3

4

5

6

7

8

9

0 0.2 0.4 0.6 0.8 1

Flip

s pe

r E

valu

atio


pdhc0.10.20.30.4

0.50.60.70.8

0.91.0



Ising Spin Glass

10

100

1000

10000

256 196 144 100 64

Pop

ulat

ion

Siz

e

Problem size (# of bits)

No DHCWith DHC

1

10

100

256 196 144 100 64G

ener

atio

nsProblem size (# of bits)

No DHCWith DHC



MAXSAT (75 bits)

0

20

40

60

80

100

120

0 0.2 0.4 0.6 0.8 1

Pop

ulat

ion

Red

uctio

n F

acto

r (o

rigin

al/r

educ

ed)


pdhc0.10.20.30.4

0.50.60.70.8

0.91.0

0.6 0.8

1 1.2 1.4 1.6 1.8

2 2.2 2.4 2.6 2.8

0 0.2 0.4 0.6 0.8 1

Gen

erat

ion

Spe

ed-u

pMax. Flips (% of Prob. Size)

pdhc0.10.20.30.4

0.50.60.70.8

0.91.0



MAXSAT (75 bits)

0

50

100

150

200

250

300

0 0.2 0.4 0.6 0.8 1

Eva

luat

ion

spee

dups


pdhc0.10.20.30.4

0.50.60.70.8

0.91.0

0

1

2

3

4

5

6

0 0.2 0.4 0.6 0.8 1

Flip

s pe

r E

valu

atio


pdhc0.10.20.30.4

0.50.60.70.8

0.91.0


Discussion of Results

Effects of DHC

I DHC improved performance on each test problem, for all sizes

I Substantial benefits were obtained despite the fact that

DHC by itself performs poorly on each of these problems

I Benefits of DHC are mainly due to variance reductionI Reduced variance makes model building easierI Also makes decision making easier



Effects of DHC on Model Building in hBOA: Trap-5

I Model qualityI Without DHC, the perfect model for trap-5 requires

connections between each variable within the partition, 10links in total

I With DHC, each partition contains either 11111 or 00000, sofewer dependencies (4) are required



Effects of DHC on Model Building in hBOA: Trap-5 (200 bits)

I Model quality: Proportion of correct dependenciesI Without DHC, much larger populations are required to obtain

the 10 necessary dependenciesI With DHC, smaller populations are required to find the 4

necessary dependencies

First generation Middle of the run

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

1

50 100

200 400

800 1600

3200 6400

12800

25600

51200

102400

204800

409600

prop

ortio

n of

cor

rect

dep

ende

ncie

s

Population Size

No DHCWith DHC

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

1

50 100

200 400

800 1600

3200 6400

12800

25600

51200

102400

204800

prop

ortio

n of

cor

rect

dep

ende

ncie

s

Population Size

No DHC, 10th generationWith DHC, 5th generation



Effects of DHC on Model Building in hBOA: Trap-5 (200 bits)

I Model quality: proportion of incorrect dependenciesI Without DHC, very many incorrect (spurious) dependencies

are discovered for all but very large populationsI With DHC, very few spurious dependencies are discovered,

except for very large populations

First generation Middle of the run

0

0.005

0.01

0.015

0.02

0.025

0.03

0.035

50 100

200 400

800 1600

3200 6400

12800

25600

51200

102400

204800

409600

prop

ortio

n of

inco

rrec

t dep

ende

ncie

s

Population Size

No DHCWith DHC

0

0.005

0.01

0.015

0.02

0.025

0.03

0.035

0.04

50 100

200 400

800 1600

3200 6400

12800

25600

51200

102400

204800

prop

ortio

n of

inco

rrec

t dep

ende

ncie

s

Population Size

No DHC, 10th generationWith DHC, 5th generation


Summary & Conclusions

DHC’s Effects on hBOA

I DHC significantly improves the performance of hBOA on eachof the test problems

I In these experiments the maximum improvement was attainedby allowing DHC to operate on the full population and rununtil it found the local optimum

I DHC helps by reducing the varianceI Model building is easierI Decision making is also easier


Acknowledgments

Acknowledgments

I NSF; NSF CAREER grant ECS-0547013.

I U.S. Air Force, AFOSR; FA9550-06-1-0096.

I University of Missouri; High Performance ComputingCollaboratory sponsored by Information Technology Services;Research Award; Research Board.


effects of a deterministic hill climber on hboa

Education

pelikan goldberg

goldberg eects

martin pelikan

instance elizabeth radetic

pelikan hartmann

optimum elizabeth radetic

false elizabeth radetic

stringstness elizabeth