A Bi-population Particle Swarm Optimizer for Learning Automata based Slow
Intelligent System
Mohammad H. Mofrad
University of Pittsburgh
Thursday, December 08, 2016
CS 2310 - Multimedia Software Engineering
Preface
2
• Slow Intelligence System (SIS)
• Particle Swarm Optimization (PSO)
• Learning Automata (LA)
• Adaptive Intelligence Optimizer (AIO)
Slow Intelligence System (SIS)
S.-K. Chang, “Slow intelligence systems,” in International Conference on Multimedia Modeling, 2010 3
• Enumeration (-enum<) of the different available solutions until finding the optimal solution
• Propagation (=prop+) of the achieved new information from the new solutions within a body of feasible solutions.
• Adaptation (+adap=) of the current solutions using the effective information gained from the elite solutions.
• Elimination (>elim-) of the worst solutions that exist in the problem space.
• Concentration (>conc=) on the elite solutions to produce new promising solutions.
Particle Swarm Optimization (PSO)
J. Kennedy, “Swarm intelligence,” Handb. Nat.-Inspired Innov. Comput., pp. 187–219, 2006. 4
• Inspired from movement of animals
• Vi = w Vi + c1 r1 (pbesti - Xi) + c2 r2 (gbest - Xi)
• Xi = Xi + Vi
vi n
vi n, pbest pbesti
n
vi n, gbest
gbesti
xi n
Vi+1 n
xi +1n
Learning Automata (LA)
M. Thathachar and P. S. Sastry, “Varieties of learning automata: an overview,” Syst. Man Cybern. Part B Cybern. IEEE Trans. On, vol. 32, no. 6, 2002. 5
• Belongs to the reinforcement learning family
Environment
Learning Automata
SignalAction
11
1 |
j j
j
j
p n a p n j ip n
p n a j j i
Reward signal
11
1 1 |
j
j
j
p n b j ip n
b r b p n j j i
Penalty signal
Adaptive Intelligence Optimizer (AIO)
6
• 2 PSO populations
• SIS framework
• LA framework
SIS + PSO
7
• Mapping SIS’s operators to PSO formula• cycle1: [guard1,2] P0 –enum< P1 =prop+ P2 >elim- P3 >conc= P4
• cycle2: [guard2,1] P0 –enum< P1 =prop+ P2 +adap= P3 >elim- P4 >conc= P5
• Simulating SIS’s slow & quick decision cycles• w = wmax – (0.75i (wmax - wmin)/imax)
• w = wmax – (i (wmax - wmin)/imax)
Implementing SIS’s TDR system using AIO
8
LA1 LA2
d2d1
s1
sk
LAn
dn
{0,1}
{0,1} {0,1}
{0,1} {0,1}
{0,1}
T D R
Implementing SIS Controller Component
9
SIS Controller
LA1 LA2 LAn
Swarm Membership
(LA Action selection)
D1 D2 Dn
s1 sk
PSO Population
CV (Si, gbesti)
Swarm Refinement
(LA Probability Update)
β1
βn
β2
αn
α2
α1
The Isolation of PSO populations
10
{p0, p1}
LA1 LA2
d2d1
s1
sk
LAn
dn
{0,1}
{0,1} {0,1}
{0,1} {0,1}
{0,1}
LA1
LAk {p0, p1}
Bi-population PSO
CV (Si, gbesti)β1βn
α1
αn
s1 sk
Implementation
11
• Python 3.4 (we extensively use NumPy package)
• Github repository @ https://goo.gl/V0vTRM
• Code• PSO ~200 lines• AIO ~400 lines
• XML interface for reading configurations
• 5 benchmark functions• Sphere, Rosenbrock, Ackley, Griewanks, Rastirign• 30 dimensions• 50 particles• …
Results – Sphere Benchmark
12
1E-294
1E-282
1E-270
1E-258
1E-246
1E-234
1E-222
1E-210
1E-198
1E-186
1E-174
1E-162
1E-150
1E-138
1E-126
1E-114
1E-102
1E-90
1E-78
1E-66
1E-54
1E-42
1E-30
1E-18
1E-06
Fit
nes
s
Iteration
PSO AIO
Results – Rosenbrock Benchmark
13
1E-08
0.0000001
0.000001
0.00001
0.0001
0.001
0.01
0.1
1
10
100
1000
10000
100000
1000000
10000000
100000000
1E+09
1E+10
1E+11
Fit
nes
s
Iteration
PSO AIO
Results – Ackley Benchmark
14
0.0001
0.001
0.01
0.1
1
10
100
Fit
nes
s
Iteration
PSO AIO
Results – Griewanks Benchmark
15
0.001
0.01
0.1
1
10
100
Fit
nes
s
Iteration
PSO AIO
Results – Rastrigin Benchmark
16
1E-14
1E-13
1E-12
1E-11
1E-10
1E-09
1E-08
0.0000001
0.000001
0.00001
0.0001
0.001
0.01
0.1
1
10
100
1000
10000
100000
Fit
nes
s
Iteration
PSO AIO
Future Work
17
• Clustering
• Dimensionality reduction
• Feature selection
• Parameter estimation
Questions?
18