radial basis function networks 20013627 표현아 computer science, kaist
Post on 20-Dec-2015
228 views
TRANSCRIPT
![Page 1: Radial Basis Function Networks 20013627 표현아 Computer Science, KAIST](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d4c5503460f94a29a51/html5/thumbnails/1.jpg)
Radial Basis Function Networks
20013627 표현아Computer Science,
KAIST
![Page 2: Radial Basis Function Networks 20013627 표현아 Computer Science, KAIST](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d4c5503460f94a29a51/html5/thumbnails/2.jpg)
contents
• Introduction
• Architecture
• Designing
• Learning strategies
• MLP vs RBFN
![Page 3: Radial Basis Function Networks 20013627 표현아 Computer Science, KAIST](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d4c5503460f94a29a51/html5/thumbnails/3.jpg)
introduction
• Completely different approach by viewing the design of a neural network as a curve-fitting (approximation) problem in high-dimensional space ( I.e MLP )
![Page 4: Radial Basis Function Networks 20013627 표현아 Computer Science, KAIST](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d4c5503460f94a29a51/html5/thumbnails/4.jpg)
In MLP
introduction
![Page 5: Radial Basis Function Networks 20013627 표현아 Computer Science, KAIST](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d4c5503460f94a29a51/html5/thumbnails/5.jpg)
In RBFN
introduction
![Page 6: Radial Basis Function Networks 20013627 표현아 Computer Science, KAIST](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d4c5503460f94a29a51/html5/thumbnails/6.jpg)
Radial Basis Function Network
• A kind of supervised neural networks
• Design of NN as curve-fitting problem
• Learning– find surface in multidimensional space best
fit to training data
• Generalization– Use of this multidimensional surface to
interpolate the test data
introduction
![Page 7: Radial Basis Function Networks 20013627 표현아 Computer Science, KAIST](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d4c5503460f94a29a51/html5/thumbnails/7.jpg)
Radial Basis Function Network
• Approximate function with linear combination of Radial basis functions
F(x) = wi h(x)
• h(x) is mostly Gaussian function
introduction
![Page 8: Radial Basis Function Networks 20013627 표현아 Computer Science, KAIST](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d4c5503460f94a29a51/html5/thumbnails/8.jpg)
architecture
Input layer
Hidden layer
Output layer
x1
x2
x3
xn
h1
h2
h3
hm
f(x)
W1
W2
W3
Wm
![Page 9: Radial Basis Function Networks 20013627 표현아 Computer Science, KAIST](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d4c5503460f94a29a51/html5/thumbnails/9.jpg)
Three layers
• Input layer– Source nodes that connect to the network
to its environment
• Hidden layer– Hidden units provide a set of basis function– High dimensionality
• Output layer– Linear combination of hidden functions
architecture
![Page 10: Radial Basis Function Networks 20013627 표현아 Computer Science, KAIST](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d4c5503460f94a29a51/html5/thumbnails/10.jpg)
Radial basis function
hj(x) = exp( -(x-cj)2 / rj2 )
f(x) = wjhj(x)j=1
m
Where cj is center of a region,
rj is width of the receptive field
architecture
![Page 11: Radial Basis Function Networks 20013627 표현아 Computer Science, KAIST](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d4c5503460f94a29a51/html5/thumbnails/11.jpg)
designing
• Require – Selection of the radial basis function width
parameter– Number of radial basis neurons
![Page 12: Radial Basis Function Networks 20013627 표현아 Computer Science, KAIST](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d4c5503460f94a29a51/html5/thumbnails/12.jpg)
Selection of the RBF width para.
• Not required for an MLP
• smaller width – alerting in untrained test data
• Larger width – network of smaller size & faster execution
designing
![Page 13: Radial Basis Function Networks 20013627 표현아 Computer Science, KAIST](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d4c5503460f94a29a51/html5/thumbnails/13.jpg)
Number of radial basis neurons
• By designer
• Max of neurons = number of input
• Min of neurons = ( experimentally determined)
• More neurons– More complex, but smaller tolerance
designing
![Page 14: Radial Basis Function Networks 20013627 표현아 Computer Science, KAIST](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d4c5503460f94a29a51/html5/thumbnails/14.jpg)
learning strategies
• Two levels of Learning– Center and spread learning (or
determination)– Output layer Weights Learning
• Make # ( parameters) small as possible– Principles of Dimensionality
![Page 15: Radial Basis Function Networks 20013627 표현아 Computer Science, KAIST](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d4c5503460f94a29a51/html5/thumbnails/15.jpg)
Various learning strategies
• how the centers of the radial-basis functions of the network are specified.
• Fixed centers selected at random
• Self-organized selection of centers
• Supervised selection of centers
learning strategies
![Page 16: Radial Basis Function Networks 20013627 표현아 Computer Science, KAIST](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d4c5503460f94a29a51/html5/thumbnails/16.jpg)
Fixed centers selected at random(1)
• Fixed RBFs of the hidden units
• The locations of the centers may be chosen randomly from the training data set.
• We can use different values of centers and widths for each radial basis function -> experimentation with training data is needed.
learning strategies
![Page 17: Radial Basis Function Networks 20013627 표현아 Computer Science, KAIST](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d4c5503460f94a29a51/html5/thumbnails/17.jpg)
Fixed centers selected at random(2)
• Only output layer weight is need to be learned.
• Obtain the value of the output layer weight by pseudo-inverse method
• Main problem– Require a large training set for a
satisfactory level of performance
learning strategies
![Page 18: Radial Basis Function Networks 20013627 표현아 Computer Science, KAIST](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d4c5503460f94a29a51/html5/thumbnails/18.jpg)
Self-organized selection of centers(1)
• Hybrid learning– self-organized learning to estimate the cent
ers of RBFs in hidden layer– supervised learning to estimate the linear
weights of the output layer
• Self-organized learning of centers by means of clustering.
• Supervised learning of output weights by LMS algorithm.
learning strategies
![Page 19: Radial Basis Function Networks 20013627 표현아 Computer Science, KAIST](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d4c5503460f94a29a51/html5/thumbnails/19.jpg)
Self-organized selection of centers(2)
• k-means clustering1. Initialization
2. Sampling
3. Similarity matching
4. Updating
5. Continuation
learning strategies
![Page 20: Radial Basis Function Networks 20013627 표현아 Computer Science, KAIST](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d4c5503460f94a29a51/html5/thumbnails/20.jpg)
Supervised selection of centers
• All free parameters of the network are changed by supervised learning process.
• Error-correction learning using LMS algorithm.
learning strategies
![Page 21: Radial Basis Function Networks 20013627 표현아 Computer Science, KAIST](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d4c5503460f94a29a51/html5/thumbnails/21.jpg)
Learning formula
learning strategies
• Linear weights (output layer)
• Positions of centers (hidden layer)
• Spreads of centers (hidden layer)
N
jCijj
ii
nGnenw
n
1
)||)((||)()(
)(tx Mi
nw
nEnwnw
iii ,...,2,1 ,
)(
)()()1( 1
N
jijiCijji
i
nnGnenwn
nEi
1
1' )]([)||)((||)()(2)(
)(txtx
tMi
n
nEnn
iii ,...,2,1 ,
)(
)()()1( 2
t
tt
N
jjiCijji
i
nnGnenwn
nEi
1
'1
)()||)((||)()()(
)(Qtx T
ijijji nnn )]()][([)( txtxQ
)(
)()()1(
1311
n
nEnn
iii
![Page 22: Radial Basis Function Networks 20013627 표현아 Computer Science, KAIST](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d4c5503460f94a29a51/html5/thumbnails/22.jpg)
MLP vs RBFN
Global hyperplane Local receptive field
EBP LMS
Local minima Serious local minima
Smaller number of hidden neurons
Larger number of hidden neurons
Shorter computation time Longer computation time
Longer learning time Shorter learning time
![Page 23: Radial Basis Function Networks 20013627 표현아 Computer Science, KAIST](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d4c5503460f94a29a51/html5/thumbnails/23.jpg)
Approximation
• MLP : Global network– All inputs cause an output
• RBF : Local network – Only inputs near a receptive field produce
an activation– Can give “don’t know” output
MLP vs RBFN
![Page 24: Radial Basis Function Networks 20013627 표현아 Computer Science, KAIST](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d4c5503460f94a29a51/html5/thumbnails/24.jpg)
in MLP
MLP vs RBFN
![Page 25: Radial Basis Function Networks 20013627 표현아 Computer Science, KAIST](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d4c5503460f94a29a51/html5/thumbnails/25.jpg)
in RBFN
MLP vs RBFN