presentation ijcnn 2011: network-based learning through particle competition for data clustering
TRANSCRIPT
8/3/2019 Presentation IJCNN 2011: Network-Based Learning through Particle Competition for Data Clustering
http://slidepdf.com/reader/full/presentation-ijcnn-2011-network-based-learning-through-particle-competition 1/26
Thiago Christiano Silva
Liang Zhao
Institute of Mathematics and Computer Science
University of São Paulo, São Carlos, São Paulo, Brazil
8/3/2019 Presentation IJCNN 2011: Network-Based Learning through Particle Competition for Data Clustering
http://slidepdf.com/reader/full/presentation-ijcnn-2011-network-based-learning-through-particle-competition 2/26
Summary Introduction
Complex Networks
Communities
Competitive Learning
Proposed Technique
Description of the Technique
Mathematical Analysis of the Model
Time Complexity Analysis of the Model
Determining the optimal number of particles in the model
Computer Simulations
Artificial Data Sets
Real-world Data Sets
Conclusions
2
8/3/2019 Presentation IJCNN 2011: Network-Based Learning through Particle Competition for Data Clustering
http://slidepdf.com/reader/full/presentation-ijcnn-2011-network-based-learning-through-particle-competition 3/26
8/3/2019 Presentation IJCNN 2011: Network-Based Learning through Particle Competition for Data Clustering
http://slidepdf.com/reader/full/presentation-ijcnn-2011-network-based-learning-through-particle-competition 4/26
Communitiesy A sub-graph whose nodes are densely connected within
itself, but sparsely connected with the rest of the network
4M. Girvan and M. E. J. Newman. Community structure in social and biological networks. Proceedings of the National Academy of Sciences of the United States of America, 99(12):7821±7826.
8/3/2019 Presentation IJCNN 2011: Network-Based Learning through Particle Competition for Data Clustering
http://slidepdf.com/reader/full/presentation-ijcnn-2011-network-based-learning-through-particle-competition 5/26
Competitive Learningy Observed in nature and in many social systems sharing limited
resources
y Water, food, mates, territory, recognition, etc.
y Important field of Machine Learning
y Widely implemented in neural networks
y Several real-world applications
y Early works include:
y Self-organizing maps (SOM)
y Differential Competitive Learning
y Adaptive Resonance Theory (ART)
5
T. Kohonen, ³The self-organizing map,´ Proceedings of the IEEE, vol. 78, no. 9, pp. 1464 ±1480, 1990.
B. Kosko, ³Stochastic competitive learning,´ IEEE Trans. Neural Networks, vol. 2, no. 5, pp. 522±529, 1991.
S. Grossberg, ³Competitive learning: From interactive activation to adaptive resonance,´ Cognitive Science, vol.
11, pp. 23±63, 1987.
8/3/2019 Presentation IJCNN 2011: Network-Based Learning through Particle Competition for Data Clustering
http://slidepdf.com/reader/full/presentation-ijcnn-2011-network-based-learning-through-particle-competition 6/26
Prior Related Worky Originally proposed by Quiles et Al.
y Several particles walk in the network and compete with each
other to mark their own territory, while attempting to reject
intruder particles
y Each particle can perform: Random Walk or Deterministic
Walk
y Only a procedure of particle competition is introduced
without formal definition
y Only applied to community detection tasks
6
M. G. Quiles, L. Zhao, R. L. Alonso, and R. A. F. Romero, ³Particle competition for complex network
community detection,´ Chaos, vol. 18, no. 3, p. 033107, 2008.
8/3/2019 Presentation IJCNN 2011: Network-Based Learning through Particle Competition for Data Clustering
http://slidepdf.com/reader/full/presentation-ijcnn-2011-network-based-learning-through-particle-competition 7/26
Contributions of the Proposed Techniquey A new type of competitive learning mechanism inspired
by the work in Quiles et Al.
y Here, the particle competition is formally represented by
a stochastic dynamical system
y A mathematical analysis has been carried out to predict
the outcome of the technique
y
We have applied the model not only for communitydetection, but also for data clustering
y A procedure for estimating the number of clusters in a
data set is presented
7
8/3/2019 Presentation IJCNN 2011: Network-Based Learning through Particle Competition for Data Clustering
http://slidepdf.com/reader/full/presentation-ijcnn-2011-network-based-learning-through-particle-competition 8/26
Description of the Techniquey
8
8/3/2019 Presentation IJCNN 2011: Network-Based Learning through Particle Competition for Data Clustering
http://slidepdf.com/reader/full/presentation-ijcnn-2011-network-based-learning-through-particle-competition 9/26
Notationy
9
8/3/2019 Presentation IJCNN 2011: Network-Based Learning through Particle Competition for Data Clustering
http://slidepdf.com/reader/full/presentation-ijcnn-2011-network-based-learning-through-particle-competition 10/26
Particles Movement Policyy
10
RANDOM TERM
y Adventurous Behavior
y Does not take into account
the dominated vertices
DETERMINISTIC TERM
y Defensive Behavior
y Prefers visiting vertices with
high domination levels
8/3/2019 Presentation IJCNN 2011: Network-Based Learning through Particle Competition for Data Clustering
http://slidepdf.com/reader/full/presentation-ijcnn-2011-network-based-learning-through-particle-competition 11/26
Stochastic Dynamical Systemy Essentially, formed by two expressions:
1. Perform the transition from all particles: merely by
random number generation, whose probability transitiondistribution is equal to the transition matrix previously
given
2. Update of the number of visits received by all vertices by
the particles:
11
OBS.: One can see that the proposed dynamical system is Markovian, since it only depends
on the present state to completely define the immediate future state
8/3/2019 Presentation IJCNN 2011: Network-Based Learning through Particle Competition for Data Clustering
http://slidepdf.com/reader/full/presentation-ijcnn-2011-network-based-learning-through-particle-competition 12/26
12
8/3/2019 Presentation IJCNN 2011: Network-Based Learning through Particle Competition for Data Clustering
http://slidepdf.com/reader/full/presentation-ijcnn-2011-network-based-learning-through-particle-competition 13/26
Mathematical Analysis
y
13
8/3/2019 Presentation IJCNN 2011: Network-Based Learning through Particle Competition for Data Clustering
http://slidepdf.com/reader/full/presentation-ijcnn-2011-network-based-learning-through-particle-competition 14/26
14
2
4
3
113
14
176
5
7
8
9
15
16
18
19
20
10
11
12
8/3/2019 Presentation IJCNN 2011: Network-Based Learning through Particle Competition for Data Clustering
http://slidepdf.com/reader/full/presentation-ijcnn-2011-network-based-learning-through-particle-competition 15/26
Time Complexity Analysisy
15
8/3/2019 Presentation IJCNN 2011: Network-Based Learning through Particle Competition for Data Clustering
http://slidepdf.com/reader/full/presentation-ijcnn-2011-network-based-learning-through-particle-competition 16/26
16
Experiments conducted on random clustered networks
L. Danon, A. Díaz-Guilera, J. Duch, and A. Arenas, ³Comparing community structure identification,´ J. Stat. Mech., p.P09008, 2005.
8/3/2019 Presentation IJCNN 2011: Network-Based Learning through Particle Competition for Data Clustering
http://slidepdf.com/reader/full/presentation-ijcnn-2011-network-based-learning-through-particle-competition 17/26
Determination of the number of clustersy The number of clusters is not known a priori
y The proposed dynamical system carries a rich set of
information
y We are able to use this information to create a new
embedded measure that estimates the number of
clusters
y Since it is embedded, no extra processing is necessary
y We will verify that the optimal number of particles
happens exactly when it is equal to the number of
clusters
17
8/3/2019 Presentation IJCNN 2011: Network-Based Learning through Particle Competition for Data Clustering
http://slidepdf.com/reader/full/presentation-ijcnn-2011-network-based-learning-through-particle-competition 18/26
y
18
Determination of the number of clusters
8/3/2019 Presentation IJCNN 2011: Network-Based Learning through Particle Competition for Data Clustering
http://slidepdf.com/reader/full/presentation-ijcnn-2011-network-based-learning-through-particle-competition 19/26
8/3/2019 Presentation IJCNN 2011: Network-Based Learning through Particle Competition for Data Clustering
http://slidepdf.com/reader/full/presentation-ijcnn-2011-network-based-learning-through-particle-competition 20/26
20
Computer Simulations
Particles are randomly
inserted into vertices
8/3/2019 Presentation IJCNN 2011: Network-Based Learning through Particle Competition for Data Clustering
http://slidepdf.com/reader/full/presentation-ijcnn-2011-network-based-learning-through-particle-competition 21/26
2121
Computer Simulations
Particles are purposefully
inserted into the worst
case scenario at the
beginning (all in one
community)
8/3/2019 Presentation IJCNN 2011: Network-Based Learning through Particle Competition for Data Clustering
http://slidepdf.com/reader/full/presentation-ijcnn-2011-network-based-learning-through-particle-competition 22/26
22
8/3/2019 Presentation IJCNN 2011: Network-Based Learning through Particle Competition for Data Clustering
http://slidepdf.com/reader/full/presentation-ijcnn-2011-network-based-learning-through-particle-competition 23/26
2323
8/3/2019 Presentation IJCNN 2011: Network-Based Learning through Particle Competition for Data Clustering
http://slidepdf.com/reader/full/presentation-ijcnn-2011-network-based-learning-through-particle-competition 24/26
Real-world data sets
24
8/3/2019 Presentation IJCNN 2011: Network-Based Learning through Particle Competition for Data Clustering
http://slidepdf.com/reader/full/presentation-ijcnn-2011-network-based-learning-through-particle-competition 25/26
25
8/3/2019 Presentation IJCNN 2011: Network-Based Learning through Particle Competition for Data Clustering
http://slidepdf.com/reader/full/presentation-ijcnn-2011-network-based-learning-through-particle-competition 26/26
Conclusionsy We have proposed an unsupervised technique based on
competitive learning
y A rigorous definition has been provided using a nonlinear
stochastic dynamical system
y A mathematical analysis has been carried out
y The proposed method presents low time complexity
y A method for determining the optimal number of particles
and the number of clusters has been discussed
y Computer simulations have been performed and
satisfactory results have been obtained
y More importantly, this work is an attempt to provide an
alternative way to the study of competitive learning
26