newtonian law inspired optimization techniques based on gravitational search algorithm

A Thesis Seminar on

Newtonian Law Inspired Optimization

Techniques Based on Gravitational Search

Algorithm

Presented by-

Rajdeep Chatterjee

M.Tech, 2009-11

School of Computer Engineering

KIIT University

Under the guidance of-

Prof. (Dr.) Madhabananda Das

Dean, School of Computer Engineering

KIIT University

Overview Gravitational Search Algorithm (GSA)

PID Controller and GSA

Simulation Results

Mutation

Differential Evolution (DE)

Proposed Algorithm

◦ Gravitational Search Algorithm with mutation (GSA-m)

◦ Gravitational Differential Evolution (GDE)

Benchmark Functions & Experimental Results

Pareto optimality & Domination

Proposed Multi-objective Gravitational Optimization (MOGO)

Benchmark Functions & Simulations

Observations

Publications

References

Gravitational Search Algorithm

This algorithm is based on the Newtonian gravity: „„Every particle in

the universe attracts every other particle with a force that is directly

proportional to the product of their masses and inversely proportional to

the square of the distance between them”.

The position of the mass corresponds to a solution of the problem,

and its masses are determined using a fitness function.


Fig. 1


Major equations of GSA:

…(1)

…(2)

…(3)

…(4)

…(5)

…(6)


Major equations of GSA:

…(7)

…(8)

…(9)

…(10)

…(11)

…(12)

…(13)

GSA Flowchart

PID Controller

Fig. 5 PID Controller

Transfer Function

GSA and PID Controller

Simulation Result

GSA PSO BFO

KP 0.85 0.56 0.80

KI - - -

KD 0.57 0.62 0.53

Cost 15.3026 19.1416 15.7906

Table 3

GSA and PID Controller

Fig. 6 Closed Loop Response

Mutation

Optimization algorithm often trapped into local

optima.

We cannot obtain the global optima and rather

ended up with local optimal value.

Mutation operator is used to lift the population

from local optima to not yet explored search

space.

Mutation

Fig. 2

Differential Evolution

It is a heuristic approach for minimizing possibly

nonlinear and non differentiable continuous space.

It was introduced by Rainer Storn and Kenneth Price in

the year 1995.

Differential Evolution

Concept behind the DE:

…(15)

Where and NP is the population size.

…(16)

…(17)

…(18)

DE Crossover

Fig. 3

Proposed Algorithms

GSA–m :: Gravitational Search Algorithm with mutation

GDE :: Gravitational Differential Evolution

DE is used as mutation operator to improve the

convergence.

DE-1 :: DE/best/1/exp

DE-2 :: DE/best/2/exp

DE-3 :: DE/best/2/bin

Proposed GSA-m Algorithm 1. Generate initial population

2. For I: 0 to max-iteration or stop criteria is reached do

3. Evaluate the fitness for each agent

4. Update the G, best and worst of the population

5. Calculate M , F and a for each agent

6. Update velocity and position i.e updated-agent

7. If last r iterations give same result or (I mod k) == 0

8. Create Difference-Offspring from updated-agent

9. Evaluate fitness;

10. If an offspring is better than updated-agent

11. Then replace the updated-agent by offspring in the next generation;

12. End If;

13. End If;

14. End For

15. Return approximate global optima

Proposed GDE Algorithm

1. Generate initial population


3. Evaluate the fitness for each agent



6. Update velocity and position i.e updated-agent

7. Create Difference-Offspring from updated-agent

8. Evaluate fitness;

9. If an offspring is better than updated-agent

10. Then replace the updated-agent by offspring in the next generation;

11. End If;

12. End For

13. Return approximate global optima

Benchmark Functions

Unimodal & Multimodal Benchmark Functions

Table. 1

Experimental Results

Simulation Results on six Benchmark Functions

Table. 2

Test

Func

tions

GSA GSA-m GDE

DE-1 DE-2 DE-3 DE-1 DE-2 DE-3

F1(x) 7.3E-11 2.06

E-17

2.07

E-17

1.99

E-17

1.76

E-17

1.92

E-17

1.83

E-17

F2(x) 25.16 5.3E-2 6.5E-2 4.9E-2 2.53E-4 1.75E-4 2.35E-4

F3(x) -2.8E+3 -2.78

E+3

-2.82

E+3

-2.77

E+3

-1.045

E+4

-1.045

E+4

-9.5

E+3

F4(x) 6.9E-6 3.68E-9 3.65E-9 3.55E-9 3.2E-9 3.4E-9 3.5E-9

F5(x) 8.0E-3 5.4E-3 5.6E-3 5.8E-3 3.6E-3 3.2E-3 3.4E-3

F6(x) -9.33 -7.39 -7.39 -7.45 -10.04 -9.162 -9.162

Performance Graphs

Fig. 4 Fig. 5

Comparison Results

Test

Functions

GA PSO GSA GSA-m GDE

F1(x) 23.13 1.8E-3 7.3E-11 1.99E-17 1.76E-17

F2(x) 1.1E+3 3.6E+4 25.16 4.9E-2 1.75E-4

F3(x) -1.2E+4 -9.8E+3 -2.8E+3 -2.82E+3 -1.045E+4

F4(x) 2.14 9.0E-3 6.9E-6 3.55E-9 3.2E-9

F5(x) 4.0E-3 2.8E-3 8.0E-3 5.4E-3 3.2E-3

F6(x) -7.3421 -9.1120 -9.33 -7.45 -10.04

Table. 3

Pareto Optimality

A well formed Multi-objective problem, there should not be a single solution that simultaneously minimizes each objective to its fullest.

In each case we are looking for a solution for which each objective has been optimized to the extent that if we try to optimize it any further, then the other objective(s) will suffer as a result.

Finding such a solution, and quantifying how much better this solution is compared to other such solutions (there will generally be many) is the goal when setting up and solving a Multi-objective optimization problem.

Types of Domination

Given two decision or solution vectors x and y, we say that decision vector x weakly dominates (or simply dominates) the decision vector y (denoted by x

y) if and only if fi(x) fi(y)∀ i = 1, ...,M (i.e., the solution x is no worse than y in all objectives) and fi(x) ≺ fi(y) for at least one i ∈ 1, 2, ...,M (i.e., the solution x is strictly better than y in at least one objective).

A solution x strongly dominates a solution y (denoted by x ≺ y ), if solution x is strictly better than solution y in all M objectives.

Domination

Fig. 7 Strict & weakly domination

Pareto Front

Fig. 8 Non-dominated solutions

Multi-objective Gravitational Optimization

(MOGO)

Equation (8) is modified to (14)

…(8)

…(14)

Where m is the number of objectives; bestk and abestk are the

maximum and minimum fitness value among the solutions for kth

objective. Mass of an agent is the summation of the masses in all

dimensions of the objective space.

MOGO

1. Generate initial population and set the parameters

2. Evaluate fitness and add solutions to the archive




6. Update velocity and position

7. Evaluate fitness

8. Domination check for the new set of solutions with the

solutions in the archive

9. End For

10. Return set of non-dominated set of solutions

Benchmark Functions

Simulation Plots

Observations

GSA is implemented to optimize gain parameters in PID

Controller.

GSA provides better gain values than other popular

algorithms – PSO and BFO.

Proposed algorithm GSA-m produces better results

than Classical GSA except F6.

Again, proposed algorithm GDE generates better results

than Classical GSA as well as GSA-m for all the test

functions.

Observations

Results obtained from GSA, new algorithms GSA-m and

GDE has been compared with existing popular

optimization techniques GA and PSO.

In F3, GA and in F5 PSO outperform all the three physics

inspired algorithms.

But GDE outclasses GA and PSO in all other test

functions. Also, results of GDE not far from these

popular algorithms.

Hence, our new Hybrid Algorithm GDE is very much

competitive with the GA and PSO.

Observations

Distribution of non-dominated points is not so uniform

in nature in all the cases.

As far as the spreads of the Pareto fronts for the

benchmark test functions are concerned, our results are

well suited except for Deb benchmark function.

Unlike in MOPSO approaches, we have no leader

selection strategy in our proposed MOGO. This in turn

has reduced the computational complexity to a great

extent as compared to MOPSO approaches.

Hence, proposed MOGO is a novel algorithm and it

serves the purpose quite well. It could lead us to a

complete new arena with very high possibilities.

Publications

R. Chatterjee and M. N. DAS, “Physics Inspired Optimization

Algorithms: Introducing New Hybrid Gravitational

Evolution & Gravitational Search Algorithm with

mutation”, International Symposium on Devices MEMS

Intelligence System Communication 2011, SMU, Sikkim, India, APR

2011.

https://www.researchgate.net/publication/259193474_Physics_Inspi

red_Optimization_Algorithms_Introducing_New_Hybrid_Gravitati

onal_Differential_Evolution_and_Gravitational_Search_Algorithm_

with_mutation?ev=prf_pub

References

Darrell Whitley. A Genetic Algorithm Tutorial. Computer Science

Department, Colorado State University, page www.cs.colostate.edu/

genitor/MiscPubs/tutorial.pdf.

David E. Goldberg. Genetic Algorithms in Search, Optimization, and Machine

Learning. Addison-Wesley Longman Publishing Co. Inc, 1 edition, 1989.

E. Rashedi. Gravitational Search Algorithm. M.Sc Thesis, Shahid Bahonar

University of Kerman, 2007.

Esmat Rashedi, Hossein Nezamabadi-pour and Saeid Saryazdi. GSA: A

Gravitational Search Algorithm. Information Sciences, 179:2232–2248, 2009.

H. Nezamabadi-pour, S. Saryazdi and E. Rashedi. Edge detection using Ant

Algorithm, . Soft Computing, 10:623–628, 2006.

Hai Shen, Yunlong Zhu, Xiaoming Zhou, Haifenf Gho and Chuanguang

Chang. Bacterial Foraging Optimization Algorithm with Particle Swarm

Optimization Strategy for Global Numerical Optimization. In Proceeding

GEC '09 Proceedings of the rst ACM/SIGEVO Summit on Genetic and

Evolutionary Computation.

References Hui Liu, Zixing Cai and Yong Wang. Hybridizing particle swarm optimization

with differential evolution for constrained numerical and engineering

optimization. Applied Soft Computing, 10.

James Kennedy and Russell Eberhart . Particle Swarm Optimization. In

proceedings of the IEEE, International Conference on Neural Network,

pages 1942–1948,1995.

Wael Mansour Korani. Bacterial foraging oriented by particle swarm

optimization strategy for PID tuning. GECCO '08 Conference on Genetic

and evolutionary computation, ACM , 2008.

Xiaohui Hu, Russell C. Eberhert and Yuhui Shi. Particle Swarm with

Extended Memory for Multiobjective Optimization. In IEEE Swarm

Intelligence Symposium, pages 193–197, 2003.

Yuhui Shi and Russell C. Eberhert. Empirical study of Particle Swarm

Optimization. Evolutionary Computation, 1999. CEC 99. Proceedings of the

1999 Congress , 2009.

References

Liping Xie, Jianchao Zeng and Zhihua Cui. General framework of Artificial

Physics Optimization Algorithm. Nature & Biologically Inspired Computing

(NaBIC 2009), pages pp 1321–1326, 2009.

N. C. Jagan . Control System. B S Publication , 2 edition, 2008.

R. A. Formato. Central Force Optimization: A New Metaheuristic with

Applications in Applied Electromagnetic. Progress In Electromagnetic

Research, PIER,77:425–491, 2007.

Rainer Storn and Kenneth Price. Differential Evolution - A Simple and

Efficient Heuristic for Global Optimization over Continuous Spaces. Journal

of Global Optimization archive, 11 Issue 4, 1997.

THANK YOU