![Page 1: A Bi-population Particle Swarm Optimizer for Learning Automata … · 2016-12-09 · A Bi-population Particle Swarm Optimizer for Learning Automata based Slow Intelligent System Mohammad](https://reader030.vdocuments.net/reader030/viewer/2022041119/5f32115ffda2a37d633bea7c/html5/thumbnails/1.jpg)
A Bi-population Particle Swarm Optimizer for Learning Automata based Slow
Intelligent System
Mohammad H. Mofrad
University of Pittsburgh
Thursday, December 08, 2016
CS 2310 - Multimedia Software Engineering
![Page 2: A Bi-population Particle Swarm Optimizer for Learning Automata … · 2016-12-09 · A Bi-population Particle Swarm Optimizer for Learning Automata based Slow Intelligent System Mohammad](https://reader030.vdocuments.net/reader030/viewer/2022041119/5f32115ffda2a37d633bea7c/html5/thumbnails/2.jpg)
Preface
2
• Slow Intelligence System (SIS)
• Particle Swarm Optimization (PSO)
• Learning Automata (LA)
• Adaptive Intelligence Optimizer (AIO)
![Page 3: A Bi-population Particle Swarm Optimizer for Learning Automata … · 2016-12-09 · A Bi-population Particle Swarm Optimizer for Learning Automata based Slow Intelligent System Mohammad](https://reader030.vdocuments.net/reader030/viewer/2022041119/5f32115ffda2a37d633bea7c/html5/thumbnails/3.jpg)
Slow Intelligence System (SIS)
S.-K. Chang, “Slow intelligence systems,” in International Conference on Multimedia Modeling, 2010 3
• Enumeration (-enum<) of the different available solutions until finding the optimal solution
• Propagation (=prop+) of the achieved new information from the new solutions within a body of feasible solutions.
• Adaptation (+adap=) of the current solutions using the effective information gained from the elite solutions.
• Elimination (>elim-) of the worst solutions that exist in the problem space.
• Concentration (>conc=) on the elite solutions to produce new promising solutions.
![Page 4: A Bi-population Particle Swarm Optimizer for Learning Automata … · 2016-12-09 · A Bi-population Particle Swarm Optimizer for Learning Automata based Slow Intelligent System Mohammad](https://reader030.vdocuments.net/reader030/viewer/2022041119/5f32115ffda2a37d633bea7c/html5/thumbnails/4.jpg)
Particle Swarm Optimization (PSO)
J. Kennedy, “Swarm intelligence,” Handb. Nat.-Inspired Innov. Comput., pp. 187–219, 2006. 4
• Inspired from movement of animals
• Vi = w Vi + c1 r1 (pbesti - Xi) + c2 r2 (gbest - Xi)
• Xi = Xi + Vi
vi n
vi n, pbest pbesti
n
vi n, gbest
gbesti
xi n
Vi+1 n
xi +1n
![Page 5: A Bi-population Particle Swarm Optimizer for Learning Automata … · 2016-12-09 · A Bi-population Particle Swarm Optimizer for Learning Automata based Slow Intelligent System Mohammad](https://reader030.vdocuments.net/reader030/viewer/2022041119/5f32115ffda2a37d633bea7c/html5/thumbnails/5.jpg)
Learning Automata (LA)
M. Thathachar and P. S. Sastry, “Varieties of learning automata: an overview,” Syst. Man Cybern. Part B Cybern. IEEE Trans. On, vol. 32, no. 6, 2002. 5
• Belongs to the reinforcement learning family
Environment
Learning Automata
SignalAction
11
1 |
j j
j
j
p n a p n j ip n
p n a j j i
Reward signal
11
1 1 |
j
j
j
p n b j ip n
b r b p n j j i
Penalty signal
![Page 6: A Bi-population Particle Swarm Optimizer for Learning Automata … · 2016-12-09 · A Bi-population Particle Swarm Optimizer for Learning Automata based Slow Intelligent System Mohammad](https://reader030.vdocuments.net/reader030/viewer/2022041119/5f32115ffda2a37d633bea7c/html5/thumbnails/6.jpg)
Adaptive Intelligence Optimizer (AIO)
6
• 2 PSO populations
• SIS framework
• LA framework
![Page 7: A Bi-population Particle Swarm Optimizer for Learning Automata … · 2016-12-09 · A Bi-population Particle Swarm Optimizer for Learning Automata based Slow Intelligent System Mohammad](https://reader030.vdocuments.net/reader030/viewer/2022041119/5f32115ffda2a37d633bea7c/html5/thumbnails/7.jpg)
SIS + PSO
7
• Mapping SIS’s operators to PSO formula• cycle1: [guard1,2] P0 –enum< P1 =prop+ P2 >elim- P3 >conc= P4
• cycle2: [guard2,1] P0 –enum< P1 =prop+ P2 +adap= P3 >elim- P4 >conc= P5
• Simulating SIS’s slow & quick decision cycles• w = wmax – (0.75i (wmax - wmin)/imax)
• w = wmax – (i (wmax - wmin)/imax)
![Page 8: A Bi-population Particle Swarm Optimizer for Learning Automata … · 2016-12-09 · A Bi-population Particle Swarm Optimizer for Learning Automata based Slow Intelligent System Mohammad](https://reader030.vdocuments.net/reader030/viewer/2022041119/5f32115ffda2a37d633bea7c/html5/thumbnails/8.jpg)
Implementing SIS’s TDR system using AIO
8
LA1 LA2
d2d1
s1
sk
LAn
dn
{0,1}
{0,1} {0,1}
{0,1} {0,1}
{0,1}
T D R
![Page 9: A Bi-population Particle Swarm Optimizer for Learning Automata … · 2016-12-09 · A Bi-population Particle Swarm Optimizer for Learning Automata based Slow Intelligent System Mohammad](https://reader030.vdocuments.net/reader030/viewer/2022041119/5f32115ffda2a37d633bea7c/html5/thumbnails/9.jpg)
Implementing SIS Controller Component
9
SIS Controller
LA1 LA2 LAn
Swarm Membership
(LA Action selection)
D1 D2 Dn
s1 sk
PSO Population
CV (Si, gbesti)
Swarm Refinement
(LA Probability Update)
β1
βn
β2
αn
α2
α1
![Page 10: A Bi-population Particle Swarm Optimizer for Learning Automata … · 2016-12-09 · A Bi-population Particle Swarm Optimizer for Learning Automata based Slow Intelligent System Mohammad](https://reader030.vdocuments.net/reader030/viewer/2022041119/5f32115ffda2a37d633bea7c/html5/thumbnails/10.jpg)
The Isolation of PSO populations
10
{p0, p1}
LA1 LA2
d2d1
s1
sk
LAn
dn
{0,1}
{0,1} {0,1}
{0,1} {0,1}
{0,1}
LA1
LAk {p0, p1}
Bi-population PSO
CV (Si, gbesti)β1βn
α1
αn
s1 sk
![Page 11: A Bi-population Particle Swarm Optimizer for Learning Automata … · 2016-12-09 · A Bi-population Particle Swarm Optimizer for Learning Automata based Slow Intelligent System Mohammad](https://reader030.vdocuments.net/reader030/viewer/2022041119/5f32115ffda2a37d633bea7c/html5/thumbnails/11.jpg)
Implementation
11
• Python 3.4 (we extensively use NumPy package)
• Github repository @ https://goo.gl/V0vTRM
• Code• PSO ~200 lines• AIO ~400 lines
• XML interface for reading configurations
• 5 benchmark functions• Sphere, Rosenbrock, Ackley, Griewanks, Rastirign• 30 dimensions• 50 particles• …
![Page 12: A Bi-population Particle Swarm Optimizer for Learning Automata … · 2016-12-09 · A Bi-population Particle Swarm Optimizer for Learning Automata based Slow Intelligent System Mohammad](https://reader030.vdocuments.net/reader030/viewer/2022041119/5f32115ffda2a37d633bea7c/html5/thumbnails/12.jpg)
Results – Sphere Benchmark
12
1E-294
1E-282
1E-270
1E-258
1E-246
1E-234
1E-222
1E-210
1E-198
1E-186
1E-174
1E-162
1E-150
1E-138
1E-126
1E-114
1E-102
1E-90
1E-78
1E-66
1E-54
1E-42
1E-30
1E-18
1E-06
Fit
nes
s
Iteration
PSO AIO
![Page 13: A Bi-population Particle Swarm Optimizer for Learning Automata … · 2016-12-09 · A Bi-population Particle Swarm Optimizer for Learning Automata based Slow Intelligent System Mohammad](https://reader030.vdocuments.net/reader030/viewer/2022041119/5f32115ffda2a37d633bea7c/html5/thumbnails/13.jpg)
Results – Rosenbrock Benchmark
13
1E-08
0.0000001
0.000001
0.00001
0.0001
0.001
0.01
0.1
1
10
100
1000
10000
100000
1000000
10000000
100000000
1E+09
1E+10
1E+11
Fit
nes
s
Iteration
PSO AIO
![Page 14: A Bi-population Particle Swarm Optimizer for Learning Automata … · 2016-12-09 · A Bi-population Particle Swarm Optimizer for Learning Automata based Slow Intelligent System Mohammad](https://reader030.vdocuments.net/reader030/viewer/2022041119/5f32115ffda2a37d633bea7c/html5/thumbnails/14.jpg)
Results – Ackley Benchmark
14
0.0001
0.001
0.01
0.1
1
10
100
Fit
nes
s
Iteration
PSO AIO
![Page 15: A Bi-population Particle Swarm Optimizer for Learning Automata … · 2016-12-09 · A Bi-population Particle Swarm Optimizer for Learning Automata based Slow Intelligent System Mohammad](https://reader030.vdocuments.net/reader030/viewer/2022041119/5f32115ffda2a37d633bea7c/html5/thumbnails/15.jpg)
Results – Griewanks Benchmark
15
0.001
0.01
0.1
1
10
100
Fit
nes
s
Iteration
PSO AIO
![Page 16: A Bi-population Particle Swarm Optimizer for Learning Automata … · 2016-12-09 · A Bi-population Particle Swarm Optimizer for Learning Automata based Slow Intelligent System Mohammad](https://reader030.vdocuments.net/reader030/viewer/2022041119/5f32115ffda2a37d633bea7c/html5/thumbnails/16.jpg)
Results – Rastrigin Benchmark
16
1E-14
1E-13
1E-12
1E-11
1E-10
1E-09
1E-08
0.0000001
0.000001
0.00001
0.0001
0.001
0.01
0.1
1
10
100
1000
10000
100000
Fit
nes
s
Iteration
PSO AIO
![Page 17: A Bi-population Particle Swarm Optimizer for Learning Automata … · 2016-12-09 · A Bi-population Particle Swarm Optimizer for Learning Automata based Slow Intelligent System Mohammad](https://reader030.vdocuments.net/reader030/viewer/2022041119/5f32115ffda2a37d633bea7c/html5/thumbnails/17.jpg)
Future Work
17
• Clustering
• Dimensionality reduction
• Feature selection
• Parameter estimation
![Page 18: A Bi-population Particle Swarm Optimizer for Learning Automata … · 2016-12-09 · A Bi-population Particle Swarm Optimizer for Learning Automata based Slow Intelligent System Mohammad](https://reader030.vdocuments.net/reader030/viewer/2022041119/5f32115ffda2a37d633bea7c/html5/thumbnails/18.jpg)
Questions?
18