a study on the convergence of genetic algorithms

Pergamon Computers ind. Engng Vol. 33, Nos 3--4, pp. 581-588, 1997

© 1997 Elsevier Science Ltd Printed in Great Britain. All rights reserved

0360-8352/97 $17.00 + 0.00 PII: S0360-8352(97)00198-8

A Study on the Convergence o f G e n e t i c A l g o r i t h m s

B.M. Kim Y.B. Kim C.H. Oh

Dept. o f Industr ia l The Research ins t i t u te o f Dept of Industr ia l

Engineering, Industr ial Technology Engineering,

Universi ty. o f UI-san Myon9 Ji Univ. The College of Su-won

Abstract

This paper extends genetic algorithms to achieve fast solut ions to d i f f i c u l t problem.

To accomplish this, we present empirical resul ts on the terminated condi t ion by bias

and the funct ionized model of mutation rate in genetic algerithms. The terminated

condi t ion by bias enable to reducing computation tima(CPU time) according to l imitted

and pre-astimated number o f generations. The funct ionized model o f mutation operator

reducing computation time and improving solut ion should be accomplished by applying

qui te low mutation rate on the continuing generation with remaining 95 percentage o f

b i a s . © 1997 E l s e v i e r S c i e n c e L td

Keywords : Bias, Generation, Mutation, Population

I. In t roduct ion

A successful e f f ec t o f system under the

dynamic environments usual ly demands the

adaptive solut ions. I t is so d i f f i c u l t to get

a optimal so lu t ion using noise, mul t i fu l

dimensions, a various react ion which are

uncertain in the various re lated appl icat ion

f ie lds . On these f ie lds , a successful resul t

can be obtained by using the appl icat ion o f

genetic algorithms for the global optimum. But

th is appl icat ion o f genetic algorithms to have

a 91obal optimum may be d i f f e ren t i a t ed by

f i tness function, i n i t i a l population, mutation

rate, crossover rate. The resul ts are also

changed depending on the number o f generation

o f the evolut ion or degree of convergence.

Such above parameters have been used as

various value under the given environments in

the genetic algorithms. I t would be not

guaranteed that these parameters are always

optimal. Many problems such as CPU time and

space o f storage would be influenced by the

parameters. Eventhough the adaptive parameters

were setted up to the given circumstances, a

proper combination on the parameters is

required to get more improved solut ion.

For instance Goldber9[5] researched the

technique o f computation for the re la t ionship

between populat ion size and computation time

used to convergence o f populat ion as applying

the expected number o f schemeta. I~ck[2]

su9gested mutation rate o f 1/l accelarated

search of so lu t ion in case o f pseudo-boolean

funct ion which the evaluat ion funct ion is

mul t imodal.

Meanwhile, genetic algorithma comes to an end

when i t a r r ives at the predetermined

generation upon convergence stage o f so lu t ion

or i f the increasing range o f f i tness funct ion

is less than s<:~te I imi ts(~:) [3] [7]. This

terminations techinque have the some problem

spending massive computation time and memory

according to the number of generation.

Moreover, the operat ion is continuous while

the value of f i tness funct ion increases fas t l y

on the i n i t i a l stage, slow or immaterial

increase on the some level. Such a rate o f

increase for f i tness funct ion becomes too

small on the f a l l i n g below the some level.

Hence, the total so lu t ion time may be reduced

and the qua l i t y of so lu t ion may be also

preserved well i f we terminate the algori thm

on the proper time which the increasing time

rate o f the f i tness funct ion value is too slow

or unnecessary. In th is aspect termination o f

condit ion for computation re f l ec t i ng the

character is t ics o f parameter is demanded. This

study suggests the terminated condi t ion o f

algori thm reducing the computation time which

appl ies terminated condi t ion by using bias in

the solving the problem as a genetic

581

582 Proceedings of 1996 ICC&IC

algorithms and pa r t i cu l a r l y in the case o f / <~

n, methods e levat ing the qua l i t y for so lu t ion

as improving the f i tness funct ion using the

character is t i cs o f murat ion operator.

2. The c h a r a c t e r i s t i c s o f parmmters us ing

by bias

This chapter 9ives p r i o r i t y to grasp a

charac te r is t i c point o f genetic system using

the l imit ted generation su i t ing the subject o f

problem. As grasping a character is t ics for the

various parameters o f such a system, the

analysis reducement to computation time and

pre-estimate the genetic algorithms for real

problem is enable to reduce total cost which

the number o f generation regards as a cost.

Bias d is t r i bu tes var ious ly between 50/, and

100/. as the s t r ing measures a rate of gene

wi th in population. Since such a bias is 100"/.

i f a l l s t r ings in population a r e al l the some

level on each loci. a so lut ion is not improved

any more as a l l s t r ings are coincident.

Let 's consider loci is b i t a on each

chromosome that the population size is 20, i f

15 uni ts have the value o f b i t i and the odd 5

uni ts have the value o f b i t 0 on chromosome

which loci is 9, th is bias becomes 75Y., and

th is bias appl ies the larger one out o f the

rate of 0 or 1. The s t r ing bias to use to

terminated condit ion of so lu t ion in th is study

is as fo l lowing ; i f each s t r ing bias is the

mean value o f I un i ts o f b i t biases. From such

a b i t bias, using s t r ing bias to grasp the

convergence state o f the whole population, and

we make i t the termination condit ion.

Applying the convergence rate o f genetic

a I 9or i thm, Operator i s cont i nuous and

converges to 100/. bias while get t ing connected

from the present generation to the next

generation, increasenent rate increases

gradual ly and on some level gets gradual ly

slow or immaterial, Hence, each chromosomes of

populat ion which has the bias above 95~ are

very s imi lar and converged to the speci f ied

str ing.

Af ter computing the measurement o f convergence

rate l i ke as bias, we should determine whether

i t is the termination o f algori thm or not. And

i t continuously comes into operat ion by stages

i f algori thm is not terminated.

The terminated condit ions applied to genetic

algorithms thus far is that algorithm is

continuously executed unt i l i t ' s ar r ived at

the determined maximum generation or the

increasement size o f f i tness funct ion is less

then some l imits(~). However, th is terminated

condithion have an important e f f ec t upon

population s ize and mutation operator.

To search the so lu t ion wi thin I imi ted

genration might be ef fected since the small

popu lat ion i s comparat ivel y converged upon

smaller generation then the large population.

Naximum generation arranged by population size

or the terminated condit ion by the increasing

size o f f i tness function gives r i se to trouble

to CPU time and storage space because o f the

res t r i c t i on by c ~ u t a t i o n time. Meanwhile, i f

the rate o f mutation operator is large, the

qua l i t y o f so lu t ion on i n i t i a l stage is

improved but the more number o f generation is

increased, the more i t impedes in convergence

state o f population because the mutation

operator at the convergence stage of so lu t ion

in ter feres with being a gene o f str ing. And

therefore, we are enable to ear ly confirm the

approached state to optimal so lu t ion according

to apply the terminated condit ion by bias.

Now we apply the method by bias as fol lows :

Aoplying Stage

Stage 1. Using the applying stage for ex is t ing

genetic algorithms,

Stage 2. In case that the bias is 95"/. and

over, terminating the algori thm i f

not increasing one percent or more

then the present bias, even a f te r

operat i ing from n time to 2n times.

At these above the method for termination,

operating times for algori thm could be

properly reduced according to the subject o f

problem.

3. Practical experiment

The purpose of th is experiment is to grasp the

problem point for the reduction o f computation

time, to make a pre-est imat ion and to improve

qua l i t y of so lu t ion using the terrniated

condit ion. The using computer is IB4 PC586

Pentium 100 N-Iz, the language is Nathematica

Vet 2.2.3. The subject o f experiment is the

Proceedings of 1996 ICC&IC 583

unconstraint condi t ion and the problem o f

quadrat ic fucnt ion, d iscont inu i ty ,

unimClel i ty, nonconvexity as fo l lowing :

f (x l , xz) = Integer(x, z + Xz z)

-5.12~x~, xz~5.12

This stage o f experiment suggests, select ing

character i st i cs o f parameter us i n8 B i as

applied from chapter 2, the technique that

keep the qua l i t y o f so lu t ion on time point

above rate for 95 percent a f te r reviewing and

comparing each individual genes for each

generation and solve a problera cause to

reduc i n9 c~pu ta t i on time.

3. I Influences by population size

One of the common problems in performing

genetic algorithms is how many influences

algori thm is given to populat ion size. In case

o f small population size, i t is very easy to

be prematurely converged cause occurrence o f

chromosome is too deminent or recessive in

search the so lu t ion space. And in case o f

large population size, some troubles are

ex is ted to perform the algori thm cause to a l l

search chromosome on every generation in

searching the so lu t ion space. I f the number o f

generation to be used l imit ted, some problems

are accurred to the convergent course o f

solut ion, consequently the population size to

be establ ished is adapt ively required to

perform genetic algorithms e f f ec t i ve l y to

improve the computation time for computer and

the qua l i t y o f solut ion. Fortunately, most o f

the experiment resu l ts are presented that 30

uni ts populat ion size su i t the subject of

problem, but th is case is influenced by

establ ished i n i t i a l population. Goldberg

recent ly proceeds an experiment using smaller

populat ion suggests an establishment through

the simulat ion about the size of adaptive

populat ion to e f f e c t i v e l y operate algorithm,

not to a f fec t the r e l i a b i l i t y o f solut ion.

On Figure 1, i f we consider the number of

generation of I00 percents convergence rate by

simulat ion in applying the population size

n=50 and mutation rate PaFO. OI, th is is about

14=35. Most of loss for gene in a whole

populat ion means convergence state o f

population. In such case, the i n c r e ~ t o f

populat ion size has an e f f ec t to considerably

loose the convergence state o f geno for

funct ion f, th is is understood to recluse the

poss i b i I i ty for premature convergence. Hence,

the increesement o f populat ion to avoid

premature convergence makes the convergence

rate o f so lut ion slow.

To confirm such an above assumption, Figure 2

appreared mutation rate P=--O, 01, population

size 30, 50, 100 appl ied algorithm performance

for funct ion f. And we can f ind out, in case

o f n=30 as c~mpared with population size n=50,

n=lO00 convergence rate o f 9ene wi th in

l imi t ted rect ion is remarkably reduced. That

is, the problem for premature convergene of

9erie could be e f f e c t i v e l y reduced as

increasing the population size.

In the s i tua t ion al lowed, the various

population size to maximum value f" and mean

calue j o f f i tness funct ion to measure the

f i tness o f funct ion f was applied. Figure 3

shows the maximum f i tness for funct ion f based

on population size. Convergence rate o f

algori thm is able to be quickly got from n=30

as compared with population size n=50, n=iO0

but the resu l t o f premature convergence which

is not optimal state is appeared. This

presents that a bet ter algori thm even though

convergene rate o f so I u t i on for Iarge

population is slower than small population.

Meenwhile, Figure 4 shows average f i tness for

funct ion f based upon population size.

3.2 The effect iveness for mutation rate

Mutation operator is genetic operator to be

decisive in applying genetic algorithms and

also one o f the genetic operator to apply

random walk method by the space o f s t r ing [ I ]

[11]. Mutation operator in previous research

about genetic algorithms appl ied mutation to

one b i t for every one thousand b i t s to get

bet ter e f fec t , however, in recent researches

and e f fec ts of experiment about genetic

algorithms, appl icat ion o f one mutation

operator for every one thousand b i t s has been

useless to improve the qua l i t y o f so lut ion in

case o f smaller Population size than length o f

s t r ing [5 ] [7 ] . Par t icu lar ly . in the e f fec t to

apply rate of mutation operator, the


discontinuous case for the subject o f problem

has been proved to be more r e l a t i ve l y

e f f i c i e n t than the continuous case in

improving qua l i t y o f so lu t ion [4].

Losed gone can be restored as increasing the

rate o f mutation operator since the degree of

convergence is reduced according as rate for

mutation operator is increased.

But, when the rate o f mutation operator is

increased, apparently suggested research for

being influence to the performance, degree o f

algori thm is acarcely existed, the reduced

gene by premature convergence can be only

improved as increasing rate o f mutation

operator. And an increasement o f the rate for

mutation operator increases the number o f gene

which samplin9 can be taken, th is exer ts

no-9ood influence upon algori thm performance

degree.

For instance, Figure 5 shows the e f fec t to

apply populat ion size n=50, mutation rate Pe

=0.001, 0.01, 0.03, 0.1. And Fi9ure 5 shows

average convergence rate o f gene based on rate

o f the various mutation operator. Since the

more rate o f mutation operator is increased.

the more convergence rate o f 9ene get low,

performance degree for algori thm is given some

troublers. In case that premature convergence

for 9ene is accurred, i t could be solved

problems as increasing the rate o f mutation

operator but convergence ef fect iveness based

on performance degree o f algorithm should be

surely considered.

Figure 6 is showing the ef fect iveness for

maximum performance degree of genetic

algorithms when the rate o f mutation operator

is increased.

In th is Figure 6, as B~ck presented, i t can' t

be v e r i f i e d maximum f i tness defree o f

algorithm is the best one when the rate o f

multat ion operat ion is 1/[. Increasement o f

rate for mutation operator have an

ef fect iveness to improve the f i tness o f

f i tness funct ion upon i n i t i a l stage when

algori thm is performed, but th is is considered

to be due to rather cause troubles for

improvement o f so lu t ion a f te r that the

generation is proceeded on a cer ta in degree.

In case that mutation operator o f low rate was

applied, f i tness o f f i tness funct ion cause by

premature convergence o f gene is reduced

according as generation is proceeded.

Figure 7 shows by a diagram for convergence

ef fect iveness of so lu t ion for average f i tness

degree based on increasement o f rate for

mutation operator when genetic algorithms is

appl led to funct ion f. And also the

increasement o f random rate for genetic

operator l i ke mutation reduce a performeance

degree o f genetic algorithms could be found

out.

Table 1 presented the resu l t to solve the

subject o f problem to f ind the computation

time and precision of so lut ion for terminated

condit ion by bias. By experiment data(30 uni ts

for population size 10, 50, 100), computation

time o f average 9W. was saved as compared with

1000 generations o f the typical genetic

algorithms which is the present, terminated

condit ion o f algorithm, and i t was maintained

so lu t ion qua l i t y of 92"/, from the problem

knowing optimal solut ion.

4. E f fec t i ve convergence of so lut ion

4. I Improvement of so lu t ion based on change

of b ias

One o f the problem points which could be

occurred in searchin9 so lu t ion o f the subject

o f problem based on genetic algorithms is that

the p o s s i b i l i t y o f premature convergence is

appeared on performin9 algorithm. This

premature convergence is var iously occurred

depending on value o f p robab i l i t y or number o f

generation. Especial ly, qua l i t y so lu t ion is

decreased is the case of />n, that is, most

solut ions would be prematurely converged in

the event that the size o f population is

smaller than s t r in9 length. However, there is

no technique to confirm in advance the

premature convergence state as the method o f

the present genetic algorithm. Improved

so lu t ion can be get, no enter ing a premature

convergence state as making populat ion size

large, but the expected ef fect iveness is not

much cause by r e s t r i c t i o n of computation time

and problem of memory. And the change o f

crossover rate can be considered, however, new

chromosomes consist ing o f next generation are

very herd to be expected due to the occurrance

of s imi lar chromosomes based on increasemant


fo r the number of generation in genetic

algorithms.

The change of mutation rate play a decisive

ro le for performance o f algori thm as the

generation is proceeded. Since the

predetermined mutation rate under the

performance o f algori thm is applied constant

value unt i l the termination for algorithm, i t

is o f no use in confirming premature

convergence. Hence a new technique to improve

the qua l i t y of solut ion, not entering

premature convergence state on performing

algorithm. This section presents the qua l i t y

o f so lu t ion based on increasing the mutation

rate on stage eighty percent [examPle;

populat ion size 10, ra ise mutation rate up to

0.3] that the p o s s i b i l i t y for new gene

occurrence is decreased.

To ident i fy above case, from Figure 8 to

Figure 10 showed the technique to improve

p o s s i b i l i t y for premature convergence as

ra is ing up the rate o f mutation operation on

stage o f bias 80"/. in the population size n=10

which has high p o s s i b i l i t y o f premature

convergence. From the experiment results, a

case of the small population size n=10,

pa r t i cu la r l y , on the discontinuous subject of

problem, the correctness degree for tota l

so lu t ion is appeared to about 95% of optimal

solut ion, and computation time is reduced to

91% as compared with the typical genetic

algor i thins techniques.

4.2 An appl ication of functionized mutation

operator for iBproving solution

Parameter exer t ing a d i rec t influence to

algori thm performance is a population size and

the rate of mutation operator. Especial ly the

rate o f operator act as a d i rec t factor to

improve so lu t ion in f ina l stage when we

perform the algorithm. Accordingly, th is

sect ion suggests that the large mutation rate

is appl ied to diminish the stage o f premature

convergence in the i n i t i a l stage o f algorithm

to improve the qua l i t y of solut ion, and that

the small mutation rate is applied to improve

so lu t ion in the convergent stage for optimal

so lu t ion on generation process.

Let us simply express the rate o f mutation

operator o f funct ionize form to improve the

quat i ty o f solut ion.

P= = a exp(-~tN), a < 0.1, /t ~ 0.1

From this, a and ~ are not exceeded 0.1,

which is mutation rate l i ke as tentave search

method in the i n i t i a l stage for algorithm, and

we apply the rate in so small that i t couldn' t

be interrupted to improve solut ion at a point

o f time maintained as bias above 95% on

9enerat ion process.

Since the crossover operation in the bias

s tate above 95% acts a rate of mutation

operator, and from a point of time maintained

as bias above 95%, i t may be e f f i c i e n t to

maintain the rate o f mutation operat ion

exert ing a influence to qua l i t y o f so lu t ion in

so small. An applying the case o f n=lO which

p o s s i b i l i t y of premature convergence is high,

n=50 which the improvement degree of algori thm

is also high and crossover rate 0.6, and the

experiment using terminated condit ion that is

previously suggested was presented by draw on

Figure ] l and Figure 12. The resu l t o f

experiment to apply a mutation rate for the

function form, the qua l i t y o f so lu t ion doesn't

have some change as compared with the present

techniques but computation time is come down

to 94%. This appears an ef fect iveness to save

the computation time.

5. Conclusion

In th is paper, a terminated condit ion o f

genetic a 19or i tl'lns reducing computaion

time(CPU t i m ) by using bias to solve the

given problem has been proposed. And also,

funct ionized mutation operator was u t i l i z e d to

improve f i tness function, as a resul t , the

method for ira Droving premature has been

studied.

The resul ts for various numerical

experimentations to apply terminated condit ion

by bias to the discontinuous funct ion have

been proved to 9et an optimal so lu t ion in

tota l experimentation even though there is a

some d i f ference according as populat ion rate,

mutation rate and crossover operator rate.

Accordingly, th is study set up the computation

time with in number o f generation enabling to

perd ic t ion and charac ter is t i c for parameter to

improve the qua l i t y of so lut ion as using the

s im i l a r i t y re la t ionship a~ong genes. I t was


confirmed through the experimentation that

th is technique, as compared with the typical

genetic algorithms, d idn ' t come down the

qua l i t y of so lu t ion and enabled to make

computation time reduced above 91~.

Addi t ional ly , we obtian that the funct ionized

mutation rate should be decreased during

convergence. The assumptions made for th is

explanation regarding the frequency o f

absorption end the losses due to mutation rate

can in p r inc ip le be v e r i f i e d experimentally.

This approach to determine the funct ionized

mutation rate seems fundamental and an

experimental ve r i f i ca t i on of the assumptions

used here is planned.

References

lO. Stephanie,

1. Yong-Beom Kim, "A Study on the

convergence o f Genetic Algorithms', PH.D

thesis, Univers i ty of Myeong-Ji,1996.

2. B&ck, T., "The Interact ion of

Mutation Rate, Selection, and

Self-Adaptat ion wi th in a Genetic

Algorithms', In Reinhard, M. and Bernard,

M., pp 85 - 94,1992.

3. De Jon9 K. A., "An Analysis o f the

Behaviour o f a Class o f Genetic Adaptive

Systems", P h . D . thesis, Univers i ty o f

Michgan, 1975.

4. Goldber9, D. E.,"Genetic Algorithms" in

Search, Optimization and Machine

Learning, Addison-Wesiey, Reading, MA, 1989.

5. Hesser, J., l~rmner, R., "Towards an

Optimal Mutation Probabi l i ty for Genetic

Algorithms', In Schwefel, H. and M~nner,

R., pp 23 - 32,1990.

6. Michalewicz, Z., "Genetic Algorithms

+Data Structures=EvolutionPrograms ", Spring

- Verlag, 1993.

7. Nicholas, J. R., Fe l i c i t y , A. W., "A

Study in Set Recombination', In Stephanie,

F., pp 23 -30,1993.

8. Reinhard, M., Bernard, M. , 'Para l le l

Problem Solving from Nature',Proceedings

o f the Second Conference on Paral le l

Problem Solving , North-Holland, 1992.

9. Scheefel, H., Minner, R. , "Para l le l

Problem Solving from Nature', 1st Workshop,

PPSN1, Springer-Verlag, 1990.

F. , 'Genet ic AI9orithms',

Proceedings of the f i f t h internat ional

conference on Genetic Algor i thm, Morgan

Kaufmann Publishers, Sen Mateo, CA, 1993.

i J= l

Figure 1. Bias o f f i tness funct ion f for

n=5O, P,=O, O. 001, O. 01, O. 05, O. 1

L ~

| =;

, o y Figure 2. Bias of f i tness funct ion ] for

n = 30,50,100

~o

t s

i ' °

20

Figure 3. Evaluate o f maximum f i tness

funct ion ] for n=30 ,50 ,100

o~ | i J

-LLo. ' ~ ' ° °

Figure 4. Evaluate of average f i tness

funct ion ] for n=30 ,50 ,100


Ii I I '~ l ,~/ t J

SO ~ ISO ~

Figure 5. Bias of fitness function J for

P.=O.O01,O.Ol,O.03,0.1

M

I"

¢ 1 1 u l

~s

=o

Figure 6. Evaluate of maximum f i tness

function ./ for P,=O.O01,O.Ol,O.03.0.1


function J for P.=0.001,0.01,0.03,0.1

if/ . [

Figure 8. Bias of J by mutation rate at the

state of bias 80"/. for n - - lO

s a m I m l

,! J o l

= = l

Figure 9.

j u

Evaluate of maxirnum fitness

function J by mutation rate at the state of

bias 80/, for n = ] O

* =

I" Jo

= l

= *


function j by mutation rate at the state of

bias 80"/. for n = l O

tD sB

l"

==

Figure 11. Evaluate of maximum fitness

function ] by functional mutation rate for

n=lO

,=

i"

Figure 12. Evaluate of maximum fitness

function ] by functional mutation rate for

n = 50

588 Proceedings of 19961CC&lC

Table 1. Comparative results between t radi t ional G.A, method of bias, method of Bias

80~ by mutation vari ty, and functionized mutation model

Terminate

method

Pop_size

Terminae

generation

Quality of

solution(%)

Computing

time(sec)

~ i t i o n ~ G.A

10 ~ 1~

1 ~ 1000 1 ~

~ .0 2 .8 97.4

5~.1 ~ . 8 9101

Method of Bias

10 50 100

21 60 108

82.0 96.8 97.7

15.0 234.6 932.6

Method of Bias 80%

by mutation varity

10 50 100

25 68 123

~ .7 97.0 97.5

18.9 267.7 1067.1

Functional

mutation Model

10 56 100

17 49 96

~ .0 ~ .3 ~.3

12.2 156.6 759.0

a study on the convergence of genetic algorithms

Documents