functional link network
DESCRIPTION
Functional Link Network. Support Vector Machines. Support Vector Machines. support vectors. separator. margin. Support Vector Machines. Support Vector Machines. Support Vector Machines. Support Vector Machines. Support Vector Machines. Support Vector Machines. Support Vector Machines. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Functional Link Network](https://reader035.vdocuments.net/reader035/viewer/2022081514/56815884550346895dc5e73a/html5/thumbnails/1.jpg)
Functional Link Network
![Page 2: Functional Link Network](https://reader035.vdocuments.net/reader035/viewer/2022081514/56815884550346895dc5e73a/html5/thumbnails/2.jpg)
Support Vector Machines
![Page 3: Functional Link Network](https://reader035.vdocuments.net/reader035/viewer/2022081514/56815884550346895dc5e73a/html5/thumbnails/3.jpg)
![Page 4: Functional Link Network](https://reader035.vdocuments.net/reader035/viewer/2022081514/56815884550346895dc5e73a/html5/thumbnails/4.jpg)
)2,,( 2122
21 xxxx
![Page 5: Functional Link Network](https://reader035.vdocuments.net/reader035/viewer/2022081514/56815884550346895dc5e73a/html5/thumbnails/5.jpg)
Support Vector Machines
22
21
21
22
21
212
2
1
2
12 22)(
y
yy
y
x
xx
x
y
y
x
xyx
))()(( yx
),2,()( where 2221
21 xxxxx
![Page 6: Functional Link Network](https://reader035.vdocuments.net/reader035/viewer/2022081514/56815884550346895dc5e73a/html5/thumbnails/6.jpg)
margin
separator
support vectors
![Page 7: Functional Link Network](https://reader035.vdocuments.net/reader035/viewer/2022081514/56815884550346895dc5e73a/html5/thumbnails/7.jpg)
Support Vector Machines
}1{),(,),,( 11 Nll yxyx
bwbxw N ,,0)(
))((sign)( bxwxf
![Page 8: Functional Link Network](https://reader035.vdocuments.net/reader035/viewer/2022081514/56815884550346895dc5e73a/html5/thumbnails/8.jpg)
Support Vector Machines
0))(( bxwy ii
1))(( bxwy ii
|| minimize w
![Page 9: Functional Link Network](https://reader035.vdocuments.net/reader035/viewer/2022081514/56815884550346895dc5e73a/html5/thumbnails/9.jpg)
Support Vector Machines
![Page 10: Functional Link Network](https://reader035.vdocuments.net/reader035/viewer/2022081514/56815884550346895dc5e73a/html5/thumbnails/10.jpg)
Support Vector Machines
ii ii xxvw ctorssupport ve for the
i ii bxxvxf ))((sign)(
![Page 11: Functional Link Network](https://reader035.vdocuments.net/reader035/viewer/2022081514/56815884550346895dc5e73a/html5/thumbnails/11.jpg)
Support Vector Machines
FN :
))()((:),( yxyxk
![Page 12: Functional Link Network](https://reader035.vdocuments.net/reader035/viewer/2022081514/56815884550346895dc5e73a/html5/thumbnails/12.jpg)
Support Vector Machines
l
iii bxxkvxf
1
)),((sign)(
l
iii bxxkvxf
1
),()(
![Page 13: Functional Link Network](https://reader035.vdocuments.net/reader035/viewer/2022081514/56815884550346895dc5e73a/html5/thumbnails/13.jpg)
Support Vector Machines
![Page 14: Functional Link Network](https://reader035.vdocuments.net/reader035/viewer/2022081514/56815884550346895dc5e73a/html5/thumbnails/14.jpg)
Support Vector Machines
dyxyxk )(),(
))2/(exp(),( 22 yxyxk
))(tanh(),( yxyxk
![Page 15: Functional Link Network](https://reader035.vdocuments.net/reader035/viewer/2022081514/56815884550346895dc5e73a/html5/thumbnails/15.jpg)
Two Spiral Problem
![Page 16: Functional Link Network](https://reader035.vdocuments.net/reader035/viewer/2022081514/56815884550346895dc5e73a/html5/thumbnails/16.jpg)
SVM architecture
![Page 17: Functional Link Network](https://reader035.vdocuments.net/reader035/viewer/2022081514/56815884550346895dc5e73a/html5/thumbnails/17.jpg)
Application: text classification
• Reuters “newswire” messages
• Bag-of-words representation
• Dimension reduction
• Training SVM
![Page 18: Functional Link Network](https://reader035.vdocuments.net/reader035/viewer/2022081514/56815884550346895dc5e73a/html5/thumbnails/18.jpg)
Results
Break-even point = precision value at which precision and recall are nearly equal
![Page 19: Functional Link Network](https://reader035.vdocuments.net/reader035/viewer/2022081514/56815884550346895dc5e73a/html5/thumbnails/19.jpg)
Results
![Page 20: Functional Link Network](https://reader035.vdocuments.net/reader035/viewer/2022081514/56815884550346895dc5e73a/html5/thumbnails/20.jpg)
Application 2: face recognition
![Page 21: Functional Link Network](https://reader035.vdocuments.net/reader035/viewer/2022081514/56815884550346895dc5e73a/html5/thumbnails/21.jpg)
False detections
![Page 22: Functional Link Network](https://reader035.vdocuments.net/reader035/viewer/2022081514/56815884550346895dc5e73a/html5/thumbnails/22.jpg)
System architecture
![Page 23: Functional Link Network](https://reader035.vdocuments.net/reader035/viewer/2022081514/56815884550346895dc5e73a/html5/thumbnails/23.jpg)
Results
![Page 24: Functional Link Network](https://reader035.vdocuments.net/reader035/viewer/2022081514/56815884550346895dc5e73a/html5/thumbnails/24.jpg)
Results
![Page 25: Functional Link Network](https://reader035.vdocuments.net/reader035/viewer/2022081514/56815884550346895dc5e73a/html5/thumbnails/25.jpg)
Skin detection and real-time recognition
![Page 26: Functional Link Network](https://reader035.vdocuments.net/reader035/viewer/2022081514/56815884550346895dc5e73a/html5/thumbnails/26.jpg)
Neural Networks
![Page 27: Functional Link Network](https://reader035.vdocuments.net/reader035/viewer/2022081514/56815884550346895dc5e73a/html5/thumbnails/27.jpg)
Ccortex is a massive spiking neuron network emulation and will mimic the human cortex, the outer layer of gray matter at the cerebral hemispheres, largely responsible for higher brain functions. The emulation covers up to 20 billion layered neurons and 2 trillion 8-bit connections.
![Page 28: Functional Link Network](https://reader035.vdocuments.net/reader035/viewer/2022081514/56815884550346895dc5e73a/html5/thumbnails/28.jpg)
Spiking Neural Networks
• From neurones to neurons• Artificial Spiking Neural Networks
(ASNN)– Dynamic Feature Binding– Computing with spike-times
![Page 29: Functional Link Network](https://reader035.vdocuments.net/reader035/viewer/2022081514/56815884550346895dc5e73a/html5/thumbnails/29.jpg)
Neural Networks
• Artificial Neural Networks– (neuro)biology -> Artificial Intelligence (AI)
– Model of how we think the brain processes information
• New data on how the brain works!– Artificial Spiking Neural Networks
![Page 30: Functional Link Network](https://reader035.vdocuments.net/reader035/viewer/2022081514/56815884550346895dc5e73a/html5/thumbnails/30.jpg)
Real Neurons
• Real cortical neurons communicate with spikes or action potentials
![Page 31: Functional Link Network](https://reader035.vdocuments.net/reader035/viewer/2022081514/56815884550346895dc5e73a/html5/thumbnails/31.jpg)
Real Neurons
• The artificial sigmoidal neuron models the rate at which spikes are generated
• artificial neuron computes function of weighted input:
x = f( )w x ij ijjx
w x ij i
![Page 32: Functional Link Network](https://reader035.vdocuments.net/reader035/viewer/2022081514/56815884550346895dc5e73a/html5/thumbnails/32.jpg)
Artificial Neural Networks
• Artificial Neural Networks can:– approximate any function
• (Multi-Layer Perceptrons)
– act as associative memory• (Hopfield networks, Sparse Distributed Memory)
– learn temporal sequences• (Recurrent Neural Networks)
![Page 33: Functional Link Network](https://reader035.vdocuments.net/reader035/viewer/2022081514/56815884550346895dc5e73a/html5/thumbnails/33.jpg)
ANN’s
• BUT....
for understanding the brain the neuron model is wrong
• individual spikes are important, not just rate
![Page 34: Functional Link Network](https://reader035.vdocuments.net/reader035/viewer/2022081514/56815884550346895dc5e73a/html5/thumbnails/34.jpg)
Binding Problem
• When humans view a scene containing a red circle and a green square, some neurons – signal the presence of red,– signal the presence of green, – signal the circle shape,– Signal the square shape.
• The binding problem: – how does the brain represent the pairing of color and shape?
• Specifically, are the circles red or green?
![Page 35: Functional Link Network](https://reader035.vdocuments.net/reader035/viewer/2022081514/56815884550346895dc5e73a/html5/thumbnails/35.jpg)
Binding
• Synchronizing spikes?
![Page 36: Functional Link Network](https://reader035.vdocuments.net/reader035/viewer/2022081514/56815884550346895dc5e73a/html5/thumbnails/36.jpg)
New Data!
• neurons belonging to same percept tend to synchronize (Gray & Singer, Nature 1987)
• timing of (single) spikes can be remarkably reproducible
• Spikes are rare: average brain activity < 1Hz– “rates” are not energy efficient
![Page 37: Functional Link Network](https://reader035.vdocuments.net/reader035/viewer/2022081514/56815884550346895dc5e73a/html5/thumbnails/37.jpg)
Computing with Spikes
• Computing with precisely timed spikes is more powerful than with “rates”.(VC dimension of spiking neuron models)[W. Maass and M. Schmitt., 1999]
• Artificial Spiking Neural Networks??[W. Maass Neural Networks, 10, 1997]
![Page 38: Functional Link Network](https://reader035.vdocuments.net/reader035/viewer/2022081514/56815884550346895dc5e73a/html5/thumbnails/38.jpg)
Artificial Spiking Neuron
• The “state” (= membrane potential) is a weighted sum of impinging spikes– spike generated when potential crosses threshold, reset
potential
![Page 39: Functional Link Network](https://reader035.vdocuments.net/reader035/viewer/2022081514/56815884550346895dc5e73a/html5/thumbnails/39.jpg)
Artificial Spiking Neuron
• Spike-Response Model:
where ε(t) is the kernel describing how a single spike changes the potential:
t e (1-t/ )
PS P:
![Page 40: Functional Link Network](https://reader035.vdocuments.net/reader035/viewer/2022081514/56815884550346895dc5e73a/html5/thumbnails/40.jpg)
Artificial Spiking Neural Network
• Network of spiking neurons:
![Page 41: Functional Link Network](https://reader035.vdocuments.net/reader035/viewer/2022081514/56815884550346895dc5e73a/html5/thumbnails/41.jpg)
Error-backpropagation in ASNN
• Encode “X-OR” in (relative) spike-times
![Page 42: Functional Link Network](https://reader035.vdocuments.net/reader035/viewer/2022081514/56815884550346895dc5e73a/html5/thumbnails/42.jpg)
XOR in ASNN
• Change weights according to gradient descent using error-backpropagation (Bohte et al, Neurocomputing 2002)
• Also effective for unsupervised learning(Bohte etal, IEEE Trans Neural Net. 2002)
![Page 43: Functional Link Network](https://reader035.vdocuments.net/reader035/viewer/2022081514/56815884550346895dc5e73a/html5/thumbnails/43.jpg)
Oil Application