Dynamic Multi-Swarm Particle Swarm Optimizer with Local Search

J. J. Liang School of Electrical and Electronic Engineering,

Nanyang Technological University Singapore 639798

liangjing@pmail.ntu.edu.sg

P. N. Suganthan

School of Electrical and Electronic Engineering, Nanyang Technological University

Singapore 639798 epnsugan@ntu.edu.sg

Abstract-In this paper, the performance of a modified dynamic multi-swarm particle swarm optimizer (DMS-PSO) on the set of benchmark functions provided by CEC2005 is reported. Different from the existing multi-swarm PSOs and local versions of PSO, the swarms are dynamic and the swarms size is small. The whole population is divided into many small swarms, these swarms are regrouped frequently by using various regrouping schedules and information is exchanged among the swarms. The Quasi-Newton method is combined to improve its local search ability.

1 Introduction

Particle swarm optimizer (PSO), emulates flocking behavior of birds to solve the optimization problems, is introduced by Kennedy and Eberhart in 1995 [1][2]. Many optimization problems could be expressed as

Min ( )f x , 1 2[ , ,..., ]Dx x x=x where D is the number of the parameters to be optimized.

In PSO, each potential solution is regarded as a particle. All particles have fitness values and velocities. The particles fly through the D dimensional problem space by learning from the historical information of all the particles. Using the useful information collected in the search process, the particles have a tendency to fly towards better search area over the course of search process. The velocity Vid and position Xid updates of dth dimension of the ith particle are presented below:

1 1 ( )d d d d

i i i iV w V c rand pbest X= + di)di

d

(1) 2 2 (d dic rand gbest x+

d di i iX X V= + (2)

where c1 and c2 are the acceleration constants, rand1id and rand2id are two uniformly distributed random numbers in the range [0,1]. 1 2( , ,..., )Di i iX X X=iX is the position of the ith particle;

1 2( , ,..., )Di i ipbest pbest pbest=ipbest is the best previous position yielding the best fitness value pbesti for the ith particle; 1 2( , ,..., )Dgbest gbest gbest=gbest is the best position discovered by the whole population;

1 2( , ,..., )Di i iv v v=iV represents the rate of the position change (velocity) for particle i. w is the inertia weight used to balance between the global and local search abilities.

In the PSO domain, there are two main variants: global PSO and local PSO. In the local version of PSO, each particles velocity is adjusted according to its personal best and the best performance achieved so far within its neighborhood instead of learning from the personal best and the best position achieved so far by the whole population in the global version. The velocity updating equation becomes:

1 1 ( )d d d d

i i i iV w V c rand pbest X= + did

i

)

(3) 2 2 ( )d di ic rand lbest x+ where 1 2( , ,..., Di i ilbest lbest lbest=ilbest is the best position achieved within its neighborhood.

Focusing on improving the local version of PSO, different neighborhood structures are proposed and discussed [3][4][5][6][7]. Except these local PSO variants, some variants that use multi-swarm [8], subpopulation [9] can also be included in the local version PSOs if we treat the sub-groups as special neighborhood structures. In the existing local versions of PSO with different neighborhood structures and the

multi-swarm PSOs, the swarms are predefined or dynamically adjusted according to the distance. Hence, the freedom of the swarms is limited. In [10], we proposed a dynamic multi-swarm particle swarm optimizer (DMS-PSO) whose neighborhood topology is dynamic and randomly assigned. DMS-PSO gives a better performance on multimodal problems than some other PSO variants, but the local search performance is not so good. In this paper, we improved this DMS-PSO by combining a classic local search algorithm, Quasi-Newton method, and test this improved DMS-PSO on the 10-D test functions provided in CEC2005 real-parameter optimization special session.

2 DMS-PSO with Local Search

The dynamic multi-swarm particle swarm optimizer is constructed based on the local version of PSO with a new neighborhood topology. Many existing evolutionary algorithms prefer larger population, while PSO needs a comparatively smaller population size. A population with three to five particles can achieve satisfactory results for simple problems. And from many reported results on the local version of PSO [3][4], PSO with small neighborhoods performs better on complex problems. Hence, In order to slow down the populations convergence velocity and increase diversity and achieve better results on multimodal problems, in the DMS-PSO, small neighborhoods are used. The population is divided into small sized swarms. Each swarm uses its own members to search for better area in the search space.

Since the small sized swarms are searching using their own best historical information, they are easy to converge to a local optimum because of PSOs convergence property. In order to avoid it becomes a co-evolutionary PSO with these swarms searching in parallel, we must allow information exchange among the swarms. And in the information exchange schedule, we want to keep more information including the good ones and the not so good ones to add the varieties of the particles and achieve larger diversity. So a randomized regrouping schedule is introduced to make the particles have a dynamic changing neighborhood structures. Every R generations, the population is regrouped randomly and

starts searching using a new configuration of small swarms. Here R is called regrouping period. In this way, the information obtained by each swarm is exchanged among the swarms. Simultaneously the diversity of the population is increased. The new neighborhood structure has more freedom when compared with the classical neighborhood structure. It is not surprising that it performs better on complex multimodal problems.

For example, suppose we have three swarms with three particles in each swarm. First, the nine particles are divided into three swarms randomly. Then the three swarms use their own particles to search for better solutions. In this period, they may converge to near a local optimum. Then the whole population is regrouped into new swarms. The new swarms begin their search. This process is continued until a stop criterion is satisfied. With the randomly regrouping schedule, particles from different swarms are grouped in a new configuration so that each small swarms search space is enlarged and better solutions are possible to be found by the new small swarms. This procedure is shown in Figure 1. Regroup

Figure 1. DMS-PSOs Search

In this paper, we introduce two concepts to modify the DMS-PSO for seeking better performance. In steady just use the basic updating equation, when updating the positions of the particles, half of the dimensions are kept the same as its best historical position, pbest, to make better use of the particles historical information to improve its global search ability.

A larger diversity and a faster convergence velocity are always a trade-off problem. Since we achieve a larger diversity using DMS-PSO, at the same time, we lose the

fast convergence velocity. In order to alleviate this weakness and give a better search in the better local areas, a local search is added into DMS-PSO: 1) Every L generations, sort the groups according to

their fitness value and refine the lbest of the best 25% groups using the Quasi-Newton method.

2) In the end of the search, the best solution achieved so far is refined using Quasi-Newton method Combining with the local search schedule, the

DMS-PSO is modified to DMS-PSO with local search (DMS-L-PSO). In our algorithm, in order to constrain the particles within the range, we calculate the fitness value of a particle and update its pbest only if the particle is in the range. Since all exemplars are within the range, the particle will eventually return to the search range. The flowchart of DMS-L-PSO is given in Figure 2.

m: Each swarms population size n: Swarms number R: Regrouping period L: local refining period L_FEs: Max FEs using in the local search Max_FEs: Max fitness evaluations, stop criterion

Initialize m*n particles (position and velocity) Divide the population into n swarms randomly, with m particles in each swarm. FEs=0; gen=0 While FEs < 0.95*Max_FEs gen=gen+1; For i=1:m*n

Find ilbestFor d=1:D

If rand

error values achieved within the Max_FEs and Success Performance and Successful FEs for the 25 10-D test functions are presented in Table 1-5. Success here means achieving the desired error accuracy within the permitted number of FEs. The predefined tolerance values for the 25 test functions are 1e-6 for functions 1-5, 1e-2 for functions 6-14 and 1e-1 for functions 15-25.

Table 1. Best Functions Error Values Achieved When FES=1e+3,

FES=1e+4, FES=1e+5 for 10-D Functions 1-8 Func

FES 1 2 3 4 5 6 7 8

1st(Min) 5.5797e+002 2.8210e+003 4.8279e+006 5.1773e+003 6.2299e+003 3.5452e+006 9.1112e+002 2.0509e+001

7th 9.8257e+002 4.5581e+003 1.1366e+007 6.5605e+003 9.0299e+003 1.8916e+007 1.2399e+003 2.0604e+00113th(Median) 9.8257e+002 6.3502e+003 1.5739e+007 8.0928e+003 1.0351e+004 3.0379e+007 1.3891e+003 2.0711e+00119th 1.5922e+003 7.5542e+003 2.5972e+007 9.9281e+003 1.0777e+004 5.0488e+007 1.5263e+003 2.0797e+00125th (Max) 1.8506e+003 9.3023e+003 6.0234e+007 1.2189e+004 1.2652e+004 8.0001e+007 1.6892e+003 2.0929e+001Mean 1.2977e+003 6.0667e+003 2.0668e+007 8.3241e+003 9.7404e+003 3.3504e+007 1.3596e+003 2.0704e+001

1e+3

Std 3.7805e+002 1.8523e+003 1.3475e+007 2.0476e+003 1.6455e+003 2.0497e+007 2.1448e+002 1.2677e-001

1st(Min) 1.6247e-003 3.9238e+001 4.0429e+005 1.1703e+002 5.6631e+000 1.3379e+001 5.2479e-001 2.0147e+001

7th 2.9822e-003 6.2407e+001 5.2876e+005 2.0392e+002 3.5300e+001 3.7088e+001 7.1273e-001 2.0304e+00113th(Median) 4.3905e-003 9.6155e+001 9.0810e+005 2.6342e+002 5.1952e+001 5.9326e+001 8.2720e-001 2.0389e+00119th 6.4262e-003 1.2096e+002 1.0049e+006 3.0171e+002 6.2717e+001 1.1300e+002 9.3448e-001 2.0426e+00125th (Max) 1.1245e-002 2.1660e+002 1.4657e+006 5.5343e+002 1.1938e+002 2.4062e+002 1.4424e+000 2.0514e+001Mean 4.8732e-003 1.0070e+002 8.4376e+005 2.7052e+002 5.2905e+001 8.2235e+001 8.4928e-001 2.0365e+001

1e+4

Std 2.5251e-003 4.6623e+001 3.2567e+005 1.0251e+002 2.9579e+001 6.0921e+001 1.9711e-001 9.7501e-0021st(Min) 0 0 1.5364e-009 2.0784e-004 1.1267e-008 6.0652e-011 4.8879e-009 2.0000e+0017th 0 5.6843e-014 5.5371e-009 6.6984e-004 1.7025e-007 1.3616e-009 2.2151e-002 2.0000e+00113th(Median) 0 5.6843e-014 7.3142e-009 1.1399e-003 4.9662e-007 3.0839e-009 4.4283e-002 2.0000e+00119th 0 1.1369e-013 8.9301e-009 2.5429e-003 9.4387e-007 6.1451e-009 6.6493e-002 2.0000e+00125th (Max) 0 7.3896e-013 1.1275e-008 8.7836e-003 1.0268e-005 1.6000e-006 1.2799e-001 2.0000e+001Mean 0 1.2960e-013 7.0064e-009 1.8851e-003 1.1383e-006 6.8925e-008 4.5189e-002 2.0000e+001

1e+5

Std 0 1.5612e-013 2.6589e-009 1.8932e-003 2.1828e-006 3.1904e-007 3.2611e-002 5.5382e-009

Table 2. Best Functions Error Values Achieved When FES=1e+3,

FES=1e+4, FES=1e+5 for 10-D Functions 9-17 Func FES

9 10 11 12 13 14 15 16 17

1st(Min) 3.7276e+001 5.8644e+001 8.0748e+000 6.1444e+003 3.9536e+000 3.2187e+000 4.0762e+002 2.5848e+002 2.7300e+002

7th 4.8854e+001 7.0838e+001 9.4739e+000 1.5999e+004 4.9500e+000 4.0511e+000 4.9872e+002 2.7525e+002 3.1770e+00213th(Median) 5.5401e+001 8.0108e+001 9.9689e+000 2.3818e+004 5.6715e+000 4.1797e+000 5.5257e+002 2.9069e+002 3.3844e+00219th 6.2236e+001 8.5870e+001 1.0773e+001 2.9831e+004 6.3773e+000 4.2469e+000 6.0563e+002 3.1506e+002 3.6602e+00225th (Max) 6.8636e+001 9.1734e+001 1.1637e+001 5.3798e+004 6.7502e+000 4.4055e+000 6.9207e+002 3.6188e+002 4.3196e+002Mean 5.5690e+001 7.7811e+001 9.9952e+000 2.4357e+004 5.5898e+000 4.1159e+000 5.5430e+002 2.9740e+002 3.4092e+002

1e+3

Std 8.7204e+000 9.9802e+000 9.5799e-001 1.1226e+004 8.8343e-001 2.3195e-001 7.3464e+001 2.9359e+001 4.1921e+001

1st(Min) 4.2259e+000 1.2823e+001 6.1408e+000 1.5000e+002 6.2406e-001 2.6936e+000 8.2990e+001 1.1678e+002 1.3884e+002

7th 6.6033e+000 2.0938e+001 6.9765e+000 4.9357e+002 1.2522e+000 3.3075e+000 1.1028e+002 1.3187e+002 1.5133e+00213th(Median) 7.4948e+000 2.4708e+001 7.3711e+000 5.9180e+002 1.4223e+000 3.4396e+000 1.2302e+002 1.4128e+002 1.6128e+00219th 8.5720e+000 2.6821e+001 7.6442e+000 1.5895e+003 1.5919e+000 3.5710e+000 1.4025e+002 1.4990e+002 1.7109e+00225th (Max) 1.0540e+001 3.0281e+001 8.5022e+000 4.9612e+003 1.9244e+000 3.8485e+000 2.6380e+002 1.7003e+002 1.8608e+002Mean 7.4393e+000 2.3521e+001 7.3131e+000 1.1807e+003 1.3832e+000 3.3758e+000 1.3128e+002 1.4105e+002 1.6123e+002

1e+4

Std 1.6074e+000 4.8143e+000 5.8680e-001 1.1562e+003 3.0527e-001 2.8613e-001 3.9021e+001 1.3556e+001 1.3050e+0011st(Min) 0 1.9899e+000 2.7590e+000 3.3481e-011 2.5403e-001 1.4838e+000 0 5.1644e+001 9.9934e+0017th 0 2.9849e+000 4.4767e+000 3.0343e-010 3.2064e-001 2.1160e+000 0 9.2064e+001 1.0672e+00213th(Median) 0 3.9798e+000 4.6570e+000 7.2993e-010 3.6085e-001 2.3522e+000 0 9.5806e+001 1.1033e+00219th 0 3.9798e+000 5.0608e+000 1.4545e-005 4.0121e-001 2.6055e+000 0 9.9969e+001 1.1215e+00225th (Max) 0 4.9748e+000 5.2394e+000 1.0003e+001 4.7266e-001 2.8906e+000 4.0454e+001 1.0511e+002 1.1869e+002Mean 0 3.6217e+000 4.6229e+000 2.4007e+000 3.6865e-001 2.3601e+000 4.8539e+000 9.4756e+001 1.1009e+002

1e+5

Std 0 8.5590e-001 5.8400e-001 4.3602e+000 5.6411e-002 3.3750e-001 1.3415e+001 1.0086e+001 4.3453e+000

Table 3. Best Functions Error Values Achieved When FES=1e+3,

FES=1e+4, FES=1e+5 for 10-D Functions 18-25 Func

FES 18 19 20 21 22 23 24 25

1st(Min) 1.0593e+003 1.0611e+003 1.0882e+003 1.1039e+003 8.7826e+002 9.8774e+002 1.0828e+003 1.6620e+003

7th 1.0965e+003 1.1038e+003 1.1227e+003 1.2541e+003 9.5177e+002 1.2505e+003 1.2041e+003 1.7383e+00313th(Median) 1.1150e+003 1.1190e+003 1.1347e+003 1.2810e+003 9.8330e+002 1.2877e+003 1.2706e+003 1.7849e+00319th 1.1266e+003 1.1349e+003 1.1520e+003 1.2935e+003 1.0096e+003 1.3195e+003 1.2972e+003 1.8050e+003

25th (Max) 1.1712e+003 1.1732e+003 1.1912e+003 1.3144e+003 1.0942e+003 1.3487e+003 1.3509e+003 1.8620e+003Mean 1.1114e+003 1.1202e+003 1.1340e+003 1.2658e+003 9.8388e+002 1.2699e+003 1.2542e+003 1.7721e+003

1e+3

Std 2.8541e+001 2.7797e+001 2.4770e+001 4.4090e+001 5.0676e+001 7.6487e+001 6.4176e+001 4.9219e+001

1st(Min) 8.0039e+002 7.3891e+002 8.0067e+002 5.0713e+002 3.9761e+002 5.5947e+002 2.0002e+002 2.1249e+002

7th 8.0295e+002 8.0257e+002 8.0327e+002 8.0000e+002 7.6981e+002 7.2090e+002 2.0007e+002 1.0817e+00313th(Median) 9.4436e+002 8.3476e+002 9.4149e+002 8.9851e+002 7.7480e+002 9.6594e+002 2.0017e+002 1.1417e+00319th 9.7222e+002 9.7579e+002 9.7226e+002 9.5196e+002 7.7820e+002 9.7050e+002 2.0056e+002 1.2507e+00325th (Max) 1.0065e+003 1.0010e+003 1.0182e+003 1.0383e+003 8.0016e+002 1.1089e+003 5.0603e+002 1.3422e+003Mean 8.9728e+002 8.7985e+002 9.0132e+002 8.5141e+002 7.5664e+002 8.6076e+002 2.4875e+002 1.0057e+003

1e+4

Std 8.0994e+001 8.9503e+001 8.3251e+001 1.4205e+002 7.8894e+001 1.8700e+002 1.1324e+002 3.5474e+002

1st(Min) 3.0000e+002 3.0000e+002 8.0000e+002 3.0000e+002 3.0000e+002 5.5947e+002 200 2.0000e+002

7th 8.0000e+002 8.0000e+002 8.0000e+002 3.0000e+002 7.5070e+002 5.5947e+002 200 2.0000e+00213th(Median) 8.0000e+002 8.0000e+002 8.0000e+002 5.0000e+002 7.5484e+002 7.2122e+002 200 4.0656e+00219th 8.2639e+002 8.0000e+002 8.0211e+002 8.0000e+002 7.5915e+002 9.3796e+002 200 4.1533e+00225th (Max) 9.3500e+002 9.3323e+002 9.3937e+002 8.0000e+002 8.0000e+002 9.7050e+002 500 9.0000e+002Mean 7.6067e+002 7.1430e+002 8.2196e+002 5.3600e+002 6.9242e+002 7.3034e+002 2.2400e+002 3.6571e+002

1e+5

Std 1.8458e+002 2.0105e+002 4.5874e+001 2.1772e+002 1.5647e+002 1.6620e+002 8.3066e+001 1.5096e+002

Table 4. Best Error Functions Values Achieved in the MAX_FES &

Success Performance for 10-D

Func 1st(Min) 7th 13th (Median) 19th 25th (Max) Mean Std Success

rate Success

Performance1 0 0 0 0 0 0 0 100% 1.1912e+0042 0 5.6843e-014 5.6843e-014 1.1369e-013 7.3896e-013 1.2960e-013 1.5612e-013 100% 1.2052e+0043 1.5364e-009 5.5371e-009 7.3142e-009 8.9301e-009 1.1275e-008 7.0064e-009 2.6589e-009 100% 1.2480e+0044 2.0784e-004 6.6984e-004 1.1003e-003 2.5429e-003 8.7879e-003 1.8549e-003 1.9031e-003 0% - 5 9.2987e-009 2.0301e-007 3.9831e-007 9.4387e-007 1.0268e-005 1.1046e-006 2.1373e-006 80% 1.1336e+0056 6.0652e-011 1.3616e-009 3.0839e-009 6.1451e-009 1.6000e-006 6.8923e-008 3.1904e-007 100% 5.4677e+0047 4.8879e-009 2.2151e-002 4.4283e-002 6.6493e-002 1.2799e-001 4.5189e-002 3.2611e-002 16% 5.8672e+0058 2.0000e+001 2.0000e+001 2.0000e+001 2.0000e+001 2.0000e+001 2.0000e+001 5.5369e-009 0% - 9 0 0 0 0 0 0 0 100% 3.4612e+004

10 1.9899e+000 2.9849e+000 3.9798e+000 3.9798e+000 4.9748e+000 3.6217e+000 8.5590e-001 0% - 11 2.7590e+000 4.4767e+000 4.6580e+000 5.0764e+000 5.2394e+000 4.6289e+000 5.8798e-001 0% - 12 3.3481e-011 3.0343e-010 7.2993e-010 1.4545e-005 1.0003e+001 2.4007e+000 4.3602e+000 76% 5.4443e+00413 2.5403e-001 3.2064e-001 3.6085e-001 4.0121e-001 4.7266e-001 3.6865e-001 5.6411e-002 0% - 14 1.4838e+000 2.1160e+000 2.3522e+000 2.6055e+000 2.8906e+000 2.3601e+000 3.3750e-001 0% - 15 0 0 0 0 4.0454e+001 4.8539e+000 1.3415e+001 88% 5.6563e+00416 5.1247e+001 9.2064e+001 9.5806e+001 9.9969e+001 1.0511e+002 9.4741e+001 1.0157e+001 0% - 17 1.0034e+002 1.0909e+002 1.1081e+002 1.1448e+002 1.1964e+002 1.1132e+002 4.7171e+000 0% - 18 3.0000e+002 8.0000e+002 8.0000e+002 8.2639e+002 9.3500e+002 7.6067e+002 1.8457e+002 0% - 19 3.0000e+002 8.0000e+002 8.0000e+002 8.0000e+002 9.3323e+002 7.1430e+002 2.0105e+002 0% - 20 8.0000e+002 8.0000e+002 8.0000e+002 8.0163e+002 9.3937e+002 8.2194e+002 4.5885e+001 0% - 21 3.0000e+002 3.0000e+002 5.0000e+002 8.0000e+002 8.0000e+002 5.3600e+002 2.1772e+002 0% - 22 3.0000e+002 7.5082e+002 7.5484e+002 7.5915e+002 8.0000e+002 6.9224e+002 1.5672e+002 0% - 23 5.5947e+002 5.5947e+002 7.2122e+002 9.3796e+002 9.7050e+002 7.3034e+002 1.6620e+002 0% - 24 200 200 200 200 500 2.2400e+002 8.3066e+001 0% - 25 2.0000e+002 2.0000e+002 4.0656e+002 4.1533e+002 9.0000e+002 3.6597e+002 1.5111e+002 0% -

Table 5. Successful FES for 10-D

Func 1st(Min) 7th 13th (Median) 19th 25th (Max) Mean Std

1 1.1843e+004 1.1897e+004 1.1915e+004 1.1933e+004 1.1946e+004 1.1912e+004 2.5786e+0012 1.1843e+004 1.1981e+004 1.2046e+004 1.2123e+004 1.2340e+004 1.2052e+004 1.1488e+0023 1.2165e+004 1.2385e+004 1.2519e+004 1.2575e+004 1.2649e+004 1.2480e+004 1.2440e+0024 - - - - - - - 5 7.7125e+004 8.5671e+004 8.9185e+004 9.9592e+004 - - - 6 2.5912e+004 5.1969e+004 5.2022e+004 6.5011e+004 7.8003e+004 5.4677e+004 1.2942e+0047 4.9529e+004 - - - - - - 8 - - - - - - - 9 2.4651e+004 2.4984e+004 3.6919e+004 3.7828e+004 4.9663e+004 3.4612e+004 8.8073e+003

10 - - - - - - - 11 - - - - - - - 12 1.2482e+004 2.5000e+004 2.5687e+004 3.8515e+004 - - - 13 - - - - - - - 14 - - - - - - - 15 1.3568e+004 2.7567e+004 4.1189e+004 6.8074e+004 - - - 16 - - - - - - - 17 - - - - - - - 18 - - - - - - - 19 - - - - - - - 20 - - - - - - - 21 - - - - - - - 22 - - - - - - - 23 - - - - - - - 24 - - - - - - - 25 - - - - - - -

Best functions error values achieved when FEs=1e+3,

FEs=1e+4, FEs=1e+5, FEs=3e+5, best error values achieved within the Max_FEs and Success Performance and Successful FEs for the twenty 30-D test functions are listed in Table 6-10. And The convergence maps of DMS-PSO on 30-D functions 1-5, functions 6-10, functions 11-14 and functions 15-20 are plotted in Figure 3-6 respectively.

The first five functions are unimodal functions, function 1 is Shifted Sphere Function, function 2 is shifted Schwefels problem 1.2, and the function 3 is shifted rotated high conditioned elliptic function. These three functions have different condition numbers which make function 3 to be harder than function 2 and function 2 to be harder than function 1. From the results, we could observe that DMS-L-PSO achieves a better result for function 1 than function 2; a better result for function 2 than function 3. Function 4 is shifted Schwefels problem 1.2 with noise in fitness. With the

noise in the fitness, it disturbs the search process. DMS-L-PSO achieves 1e-4 error and 1e-2 error for 10-D and 30-D respectively, but does not meet the 1e-6 criterion. Function 5 is Schwefels problem 2.6 with global optimum on bounds. DMS-L-PSO achieves 80% success rate on this problem for 10-D but only get 1e-2 error for 30-D.

Table 6. Best Functions Error Values Achieved When FES=1e+3,

FES=1e+4, FES=1e+5, FES=3e+5 for 30-D Functions 1-8 Func

FES 1 2 3 4 5 6 7 8

1st(Min) 2.3093e+004 3.6767e+004 3.0516e+008 3.7431e+004 2.4056e+004 3.7339e+009 7.3637e+003 2.1030e+001

7th 2.8367e+004 4.8877e+004 4.9563e+008 5.9857e+004 2.6680e+004 7.1152e+009 8.1196e+003 2.1163e+00113th(Median) 3.0743e+004 5.2240e+004 5.3458e+008 6.8015e+004 2.8027e+004 8.8957e+009 8.5567e+003 2.1205e+00119th 3.3470e+004 6.2950e+004 6.4448e+008 7.3645e+004 2.8758e+004 1.1022e+010 8.7606e+003 2.1233e+00125th (Max) 3.8978e+004 7.5785e+004 7.9447e+008 9.2870e+004 3.1367e+004 1.3797e+010 9.4805e+003 2.1316e+001Mean 3.0920e+004 5.4941e+004 5.5174e+008 6.7769e+004 2.7842e+004 9.2351e+009 8.4766e+003 2.1198e+001

1e+3

Std 3.9401e+003 1.1008e+004 1.2252e+008 1.2654e+004 1.8639e+003 2.5567e+009 6.0802e+002 7.1891e-002

1st(Min) 1.6712e-011 1.4684e+003 9.2175e+006 1.7723e+004 5.1804e+003 3.7145e+005 6.0611e+000 2.0434e+001

7th 2.1657e-011 2.0758e+003 1.5697e+007 2.0021e+004 6.2969e+003 6.5127e+005 1.1538e+001 2.0555e+00113th(Median) 2.9161e-011 2.2573e+003 1.8598e+007 2.3720e+004 6.6363e+003 7.4886e+005 1.3364e+001 2.0602e+00119th 3.6380e-011 2.7972e+003 2.0161e+007 2.7668e+004 7.0852e+003 1.0973e+006 1.5782e+001 2.0673e+00125th (Max) 1.8105e-010 3.3752e+003 2.4520e+007 2.9494e+004 8.6114e+003 1.8697e+006 2.1629e+001 2.0733e+001Mean 4.4038e-011 2.3731e+003 1.7971e+007 2.3955e+004 6.6598e+003 8.8365e+005 1.3074e+001 2.0599e+001

1e+4

Std 4.5306e-011 5.1298e+002 3.8706e+006 3.8258e+003 8.0177e+002 4.1027e+005 3.7024e+000 8.1222e-0021st(Min) 0 9.2433e-007 3.3256e+005 4.3105e+003 2.9208e+003 8.1138e+000 4.5358e-007 2.0019e+0017th 0 1.5520e-006 7.1218e+005 5.9273e+003 3.0850e+003 2.1326e+001 2.5430e-006 2.0045e+00113th(Median) 0 2.5470e-006 8.2849e+005 6.4803e+003 3.2927e+003 8.6140e+001 9.8579e-003 2.0059e+00119th 0 3.5603e-006 1.0052e+006 7.3007e+003 3.4946e+003 1.2404e+002 9.8583e-003 2.0079e+00125th (Max) 5.6843e-014 6.2407e-006 1.2384e+006 8.8879e+003 3.7636e+003 1.7892e+002 1.2322e-002 2.0111e+001Mean 1.1369e-014 2.6544e-006 8.3716e+005 6.5738e+003 3.2984e+003 7.5322e+001 7.0000e-003 2.0062e+001

1e+5

Std 2.3206e-014 1.3747e-006 2.5239e+005 1.0922e+003 2.4016e+002 5.6248e+001 4.5372e-003 2.4750e-0021st(Min) 0 5.1705e-008 6.5673e-008 2.0255e+003 1.4786e+002 3.0861e-008 9.6634e-013 2.0000e+0017th 0 8.0356e-008 9.6756e-008 2.3335e+003 2.2760e+003 7.7005e-008 1.6518e-006 2.0000e+00113th(Median) 0 1.1080e-007 3.1589e-007 2.5813e+003 2.3972e+003 1.7511e-007 9.8573e-003 2.0000e+00119th 0 1.2708e-007 5.9519e-007 2.6583e+003 2.6268e+003 3.6054e-007 9.8573e-003 2.0000e+00125th (Max) 0 3.8895e-007 1.7141e-005 3.1666e+003 3.1264e+003 3.9866e+000 1.2321e-002 2.0001e+001Mean 0 1.1757e-007 1.6343e-006 2.5487e+003 2.1858e+003 4.7840e-001 6.9990e-003 2.0000e+001

3e+5

Std 0 6.5592e-008 3.9247e-006 3.0638e+002 8.2641e+002 1.3222e+000 4.5371e-003 2.3029e-004

Table 7. Best Functions Error Values Achieved When FES=1e+3,

FES=1e+4, FES=1e+5, FES=3e+5 for 30-D Functions 9-16 Func

FES 9 10 11 12 13 14 15 16

1st(Min) 2.9937e+002 4.1756e+002 3.7386e+001 5.7752e+005 3.3318e+001 1.3588e+001 7.8599e+002 5.9308e+002

7th 3.1527e+002 4.5312e+002 4.1645e+001 8.8508e+005 3.8618e+001 1.3931e+001 8.9137e+002 6.5911e+00213th(Median) 3.3700e+002 4.6589e+002 4.1986e+001 9.6172e+005 4.1767e+001 1.4036e+001 9.3656e+002 6.9970e+00219th 3.5061e+002 4.8940e+002 4.2952e+001 1.0265e+006 4.5558e+001 1.4125e+001 9.7630e+002 7.9826e+00225th (Max) 3.8094e+002 5.3387e+002 4.4306e+001 1.1876e+006 4.9738e+001 1.4216e+001 1.0234e+003 9.0607e+002Mean 3.3702e+002 4.6927e+002 4.1802e+001 9.5005e+005 4.1877e+001 1.4002e+001 9.2975e+002 7.2245e+002

1e+3

Std 2.3161e+001 3.2876e+001 1.5884e+000 1.4685e+005 4.5634e+000 1.6782e-001 6.1287e+001 8.9664e+001

1st(Min) 3.2952e+001 5.4340e+001 3.3480e+001 1.1101e+004 5.5823e+000 1.2467e+001 4.3342e+002 2.0804e+002

7th 4.4526e+001 7.1110e+001 3.5490e+001 2.3375e+004 7.1247e+000 1.2991e+001 4.7423e+002 2.2687e+00213th(Median) 5.3764e+001 8.3812e+001 3.6992e+001 2.9027e+004 7.4137e+000 1.3199e+001 4.8105e+002 2.5720e+00219th 5.9578e+001 9.3819e+001 3.7820e+001 3.6093e+004 8.0687e+000 1.3349e+001 4.9253e+002 2.9388e+00225th (Max) 7.1637e+001 1.0670e+002 3.8489e+001 6.4389e+004 9.0996e+000 1.3577e+001 5.8772e+002 5.4381e+002Mean 5.2623e+001 8.2489e+001 3.6703e+001 3.0558e+004 7.5396e+000 1.3124e+001 4.8671e+002 2.8613e+002

1e+4

Std 9.5962e+000 1.3585e+001 1.4517e+000 1.3497e+004 9.3062e-001 3.1427e-001 3.5724e+001 1.0075e+0021st(Min) 1.5919e+001 2.5869e+001 2.3810e+001 4.2560e-002 2.1011e+000 1.1655e+001 3.0001e+002 8.0650e+0017th 2.0940e+001 3.9798e+001 2.8889e+001 1.5350e+002 2.5321e+000 1.2113e+001 3.0002e+002 9.9425e+00113th(Median) 2.4874e+001 4.5768e+001 3.0035e+001 4.7821e+002 2.8599e+000 1.2321e+001 4.0000e+002 1.1246e+00219th 2.6864e+001 5.0743e+001 3.0690e+001 1.3612e+003 3.3337e+000 1.2446e+001 4.0000e+002 1.4007e+00225th (Max) 3.0844e+001 6.4672e+001 3.2049e+001 4.9761e+003 4.6229e+000 1.2850e+001 5.0000e+002 5.0000e+002Mean 2.3841e+001 4.4538e+001 2.9435e+001 9.6337e+002 2.9957e+000 1.2289e+001 3.5695e+002 1.5327e+002

1e+5

Std 4.0144e+000 9.0204e+000 1.9468e+000 1.3444e+003 6.6559e-001 3.2179e-001 5.7459e+001 1.1264e+0021st(Min) 1.2934e+001 2.5869e+001 2.2480e+001 1.5535e-007 1.3770e+000 1.0611e+001 3.0000e+002 5.0974e+0017th 1.4924e+001 3.4824e+001 2.6508e+001 3.2890e+000 1.9618e+000 1.1793e+001 3.0000e+002 6.4844e+00113th(Median) 1.7909e+001 3.8803e+001 2.7231e+001 9.6388e+001 2.3596e+000 1.2015e+001 3.0000e+002 7.6321e+00119th 1.8904e+001 4.0793e+001 2.8319e+001 4.7704e+002 2.7338e+000 1.2246e+001 4.0000e+002 9.7759e+00125th (Max) 2.4874e+001 4.5768e+001 2.9492e+001 9.9990e+002 3.3325e+000 1.2427e+001 4.0000e+002 5.0000e+002Mean 1.7591e+001 3.7410e+001 2.7278e+001 2.5359e+002 2.3595e+000 1.1961e+001 3.4400e+002 1.1950e+002

3e+5

Std 3.0222e+000 5.2883e+000 1.5739e+000 2.8883e+002 5.2823e-001 4.1146e-001 5.0662e+001 1.2068e+002

Table 8. Best Functions Error Values Achieved When FES=1e+3,

FES=1e+4, FES=1e+5, FES=3e+5 for 30-D Functions 17-20 Func

FES 17 18 19 20

1st(Min) 5.9513e+002 1.1602e+003 1.1632e+003 1.1813e+003

7th 8.9706e+002 1.2023e+003 1.1978e+003 1.2132e+00313th(Median) 9.5461e+002 1.2221e+003 1.2081e+003 1.2310e+00319th 1.0283e+003 1.2562e+003 1.2477e+003 1.2479e+00325th (Max) 1.2042e+003 1.3050e+003 1.2894e+003 1.2848e+003Mean 9.5948e+002 1.2272e+003 1.2188e+003 1.2326e+003

1e+3

Std 1.1676e+002 3.7515e+001 3.3518e+001 2.4708e+001

1st(Min) 2.4820e+002 9.3540e+002 9.3685e+002 9.3424e+002

7th 2.7989e+002 9.4119e+002 9.4278e+002 9.4309e+00213th(Median) 2.9648e+002 9.4410e+002 9.4431e+002 9.4613e+00219th 3.9439e+002 9.4633e+002 9.4533e+002 9.4852e+00225th (Max) 5.0963e+002 9.5580e+002 9.5321e+002 9.5260e+002Mean 3.3558e+002 9.4422e+002 9.4406e+002 9.4572e+002

1e+4

Std 7.5233e+001 4.3639e+000 3.3772e+000 4.3354e+000

1st(Min) 1.3775e+002 9.1051e+002 9.1150e+002 8.0000e+002

7th 1.6525e+002 9.1277e+002 9.1198e+002 9.1261e+00213th(Median) 1.7504e+002 9.1349e+002 9.1368e+002 9.1382e+00219th 1.9759e+002 9.1448e+002 9.1418e+002 9.1458e+00225th (Max) 4.4319e+002 9.1707e+002 9.1675e+002 9.1596e+002Mean 2.0394e+002 9.1354e+002 9.1346e+002 9.0478e+002

1e+5

Std 7.6767e+001 1.4685e+000 1.4175e+000 3.1565e+0011st(Min) 1.0100e+002 9.0706e+002 9.0729e+002 8.0000e+0027th 1.0761e+002 9.0968e+002 9.0977e+002 9.0943e+00213th(Median) 1.2125e+002 9.1082e+002 9.1092e+002 9.1078e+00219th 1.4671e+002 9.1144e+002 9.1147e+002 9.1227e+00225th (Max) 4.3285e+002 9.1377e+002 9.1256e+002 9.1369e+002Mean 1.4519e+002 9.1053e+002 9.1060e+002 9.0189e+002

3e+4

Std 7.3247e+001 1.5761e+000 1.3383e+000 3.0719e+001

Table 9. Best Error Functions Values Achieved in the MAX_FES &

Success Performance for 30-D

Func 1st(Min) 7th 13th (Median) 19th 25th (Max) Mean Std Success

rate Success

Performance1 0 0 0 0 0 0 0 100 5.0263e+0032 5.1705e-008 8.0356e-008 1.1080e-007 1.2708e-007 3.8895e-007 1.1757e-007 6.5592e-008 100 1.2552e+0053 6.5673e-008 9.6756e-008 3.1589e-007 5.9519e-007 1.7141e-005 1.6343e-006 3.9247e-006 84 3.4100e+0054 2.0255e+003 2.3335e+003 2.5813e+003 2.6583e+003 3.1666e+003 2.5487e+003 3.0638e+002 - - 5 1.4786e+002 2.2760e+003 2.3972e+003 2.6268e+003 3.1264e+003 2.1858e+003 8.2641e+002 - - 6 3.0861e-008 7.7005e-008 1.7511e-007 3.6054e-007 3.9866e+000 4.7840e-001 1.3222e+000 98 3.2781e+0057 9.6634e-013 1.6518e-006 9.8573e-003 9.8573e-003 1.2321e-002 6.9990e-003 4.5371e-003 96 5.9577e+0048 2.0000e+001 2.0000e+001 2.0000e+001 2.0000e+001 2.0001e+001 2.0000e+001 2.3029e-004 - - 9 1.2934e+001 1.4924e+001 1.7909e+001 1.8904e+001 2.4874e+001 1.7591e+001 3.0222e+000 - -

10 2.5869e+001 3.4824e+001 3.8803e+001 4.0793e+001 4.5768e+001 3.7410e+001 5.2883e+000 - - 11 2.2480e+001 2.6508e+001 2.7231e+001 2.8319e+001 2.9492e+001 2.7278e+001 1.5739e+000 - - 12 1.5535e-007 3.2890e+000 9.6388e+001 4.7704e+002 9.9990e+002 2.5359e+002 2.8883e+002 16 1.5108e+00613 1.3770e+000 1.9618e+000 2.3596e+000 2.7338e+000 3.3325e+000 2.3595e+000 5.2823e-001 - - 14 1.0611e+001 1.1793e+001 1.2015e+001 1.2246e+001 1.2427e+001 1.1961e+001 4.1146e-001 - - 15 3.0000e+002 3.0000e+002 3.0000e+002 4.0000e+002 4.0000e+002 3.4400e+002 5.0662e+001 - - 16 5.0974e+001 6.4844e+001 7.6321e+001 9.7759e+001 5.0000e+002 1.1950e+002 1.2068e+002 - - 17 1.0100e+002 1.0761e+002 1.2125e+002 1.4671e+002 4.3285e+002 1.4519e+002 7.3247e+001 - - 18 9.0706e+002 9.0968e+002 9.1082e+002 9.1144e+002 9.1377e+002 9.1053e+002 1.5761e+000 - - 19 9.0729e+002 9.0977e+002 9.1092e+002 9.1147e+002 9.1256e+002 9.1060e+002 1.3383e+000 - - 20 8.0000e+002 9.0943e+002 9.1078e+002 9.1227e+002 9.1369e+002 9.0189e+002 3.0719e+001 - -

Table 10. Successful FES for 30-D Func 1st(Min) 7th 13th (Median) 19th 25th (Max) Mean Std

1 4.9180e+003 4.9900e+003 5.0100e+003 5.0420e+003 5.2320e+003 5.0263e+003 7.2463e+001 2 9.6884e+004 1.1155e+005 1.2537e+005 1.3775e+005 1.5829e+005 1.2552e+005 1.7371e+004 3 2.8630e+005 2.8631e+005 2.8634e+005 2.8689e+005 - - - 4 - - - - - - - 5 - - - - - - - 6 2.8738e+005 2.8807e+005 2.8851e+005 2.8921e+005 - - - 7 4.1364e+004 4.8611e+004 6.2349e+004 6.2445e+004 - - - 8 - - - - - - - 9 - - - - - - -

10 - - - - - - - 11 - - - - - - - 12 1.1502e+005 - - - - - - 13 - - - - - - - 14 - - - - - - - 15 - - - - - - - 16 - - - - - - - 17 - - - - - - - 18 - - - - - - - 19 - - - - - - - 20 - - - - - - -

0 0.5 1 1.5 2 2.5 3

x 105

10-15

10-10

10-5

100

105

1010

1015

FEs

f(x)-f(x*)

function1function2function3function4function5

Figure 3. Convergence Graph for Function 1-5

0 0.5 1 1.5 2 2.5 3

x 105

10-4

10-2

100

102

104

106

108

1010

1012

FEs

f(x)-f(x*)

function6function7function8function9function10

Figure 4. Convergence Graph for Function 6-10

0 0.5 1 1.5 2 2.5 3

x 105

100

101

102

103

104

105

106

107

FEs

f(x)-f(x*)

function11function12function13function14

Figure 5. Convergence Graph for Function 11-14

0 0.5 1 1.5 2 2.5 3

x 105

102

103

104

FEs

f(x)-f(x*)

function15function16function17function18function19function20

Figure 6. Convergence Graph for Function 15-20

Functions 6-25 are multimodal problems. Function 6 is shifted Rosenbrocks Function, a problem between unimodal and multimodal and an algorithm with good local search ability can achieve good results on Rosenbrocks. DMS-L-PSO achieves 100% and 98% success rate on it for 10-D and 30-D owing to Quasi-Newtons good local search ability. Function 7 is shifted rotated Griewanks function without bounds, only the initialization range is given and the search range is[ , ]D + . Griewanks function is more difficult with low dimension and it is difficult to achieve the global optimum. DMS-L-PSO only achieves 16% success rate on this problem. Function 8 is shifted rotated Ackleys function with global optimum on bounds, which has a very narrow global basin and half of the dimensions of this basin are on the bounds. Hence, the search is almost like seeking a needle in a haystack. DMS-L-PSO fails on this problem in all 25 runs. Functions 9 and 10 are shifted Rastrigins function and shifted rotated Rastrigins function respectively, both of which have a huge number of local optima. For 10-D DMS-L-PSO hits the global optimum in all 25 runs for function 9, but the results are not that good for function 10 owing to the rotation. For 30-D, the results are not so good as 10-D owing to the small population size, a larger population is expected to given better results. Function 11 is shifted rotated Weierstrass function and the results are not very good. Function 12 is Schwefels problem and most time the global optimum is found, but when DMS-L-PSO fails, it falls into a local optimum with very bad fitness value.

The success rate for this problem is 76% and 16% for 10-D and 30-D respectively. Functions 13 and 14 are extended functions, and functions 15-25 are eleven novel composition functions. They all built up with basic functions. They give a big challenge to any search algorithm. DMS-L-PSO only achieves 88% success rate on function 15 for 10-D and fails on all others.

The algorithms complexity is calculated on 10, 30, 50 dimensions, to show the complexitys relationship with increasing dimensions.

Table 11. Algorithm Complexity

T0 T1 2T

( 2T

-T1)/T0

D=10 40.7510 30.6400 77.0380 1.1386

D=30 40.7510 38.6340 88.8812 1.2330

D=50 40.7510 46.4470 106.1240 1.4644

For the comparison of DMS-PSO with other

algorithms, please refer to other articles in this special session.

Bibliography

[1] R. C. Eberhart, and J. Kennedy, "A new optimizer using particle swarm theory", Proc. of the Sixth Int. Symposium on Micromachine and Human Science, Nagoya, Japan. pp. 39-43, 1995

[2] J. Kennedy, and R. C. Eberhart, "Particle swarm optimization ". Proceedings of IEEE International Conference on Neural Networks, Piscataway, NJ. pp. 1942-1948, 1995

[3] J. Kennedy, "Small worlds and mega-minds: effects of neighborhood topology on particle swarm performance". Proc. of IEEE Congress on Evolutionary Computation (CEC 1999), Piscataway, NJ. pp. 1931-1938, 1999

[4] J. Kennedy and R. Mendes, "Population structure and particle swarm performance ". Proceedings of the IEEE Congress on Evolutionary Computation (CEC 2002), Honolulu, Hawaii USA. 2002

[5] P. N. Suganthan, "Particle swarm optimizer with neighborhood operator," Proc. of the IEEE Congress on Evolutionary Computation (CEC 1999), Piscataway, NJ, pp. 1958-1962, 1999.

[6] X. Hu and R. C. Eberhart, "Multiobjective optimization using dynamic neighborhood particle swarm optimization," Proc. IEEE Congress on Evolutionary Computation, Hawaii, pp. 1677-1681, 2002.

[7] R. Mendes, J. Kennedy, and J. Neves, "The fully informed particle swarm: simpler, maybe better. IEEE Transactions on Evolutionary Computation, 8(3):204 - 210, June 2004

[8] T. M. Blackwell and J. Branke, "Multi-swarm optimization in dynamic environments". LNCS No. 3005: Proceedings of Applications of Evolutionary Computing: EvoWorkshops 2004: EvoBIO, EvoCOMNET, EvoHOT, EvoISAP, EvoMUSART, and EvoSTOC, Coimbra, Portugal. pp. 489-500, 2004

[9] M. Lvbjerg, T. K. Rasmussen and T. Krink, "Hybrid particle swarm optimiser with breeding and subpopulations". Proc. of the Genetic and Evolutionary Computation Conference, (GECCO 2001), 2001

[10] J. J. Liang and P. N. Suganthan, "Dynamic Multi-Swarm Particle Swarm Optimizer", IEEE International Swarm Intelligence Symposium, 2005

1 Introduction2 DMS-PSO with Local Search3 ExperimentsBibliography