one of the learned stimuli
DESCRIPTION
Activity of a single neuron in the cortex. one of the learned stimuli. new stimulus. Hebbian plasticity. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/1.jpg)
one of the learned stimuli
newstimulus
Activity of a single neuron in the cortex
![Page 2: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/2.jpg)
Hebbian plasticity
“When an axon of cell A is near enough to excite cell B and repeatedly or persistently takes part in firing it, some growth process or metabolic change takes place in one or both cells such that A's efficiency, as one of the cells firing B, is increased”
Donald Hebb, 1949
“Neurons that fire together wire together”
![Page 3: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/3.jpg)
A cortical network
![Page 4: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/4.jpg)
A cortical network
![Page 5: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/5.jpg)
A cortical network
![Page 6: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/6.jpg)
A cortical network
![Page 7: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/7.jpg)
A cortical network
![Page 8: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/8.jpg)
A cortical network
![Page 9: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/9.jpg)
Hebbian plasticity
![Page 10: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/10.jpg)
A cortical network
Network can sustain activity even in the absence of input
![Page 11: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/11.jpg)
Specificity of sustained activity
![Page 12: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/12.jpg)
Specificity of sustained activity
![Page 13: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/13.jpg)
Specificity of sustained activity
![Page 14: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/14.jpg)
Specificity of sustained activity
![Page 15: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/15.jpg)
Specificity of sustained activity
![Page 16: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/16.jpg)
A model of associative memory
![Page 17: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/17.jpg)
A model of associative memory
![Page 18: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/18.jpg)
A model of associative memory
![Page 19: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/19.jpg)
A model of associative memory
![Page 20: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/20.jpg)
A model of associative memory
![Page 21: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/21.jpg)
A model of associative memory
![Page 22: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/22.jpg)
A model of associative memory
![Page 23: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/23.jpg)
A model of associative memory
![Page 24: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/24.jpg)
A model of associative memory
![Page 25: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/25.jpg)
A model of associative memory
![Page 26: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/26.jpg)
A model of associative memory
![Page 27: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/27.jpg)
A model of associative memory
![Page 28: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/28.jpg)
A model of associative memory
![Page 29: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/29.jpg)
A model of associative memory
![Page 30: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/30.jpg)
A model of associative memory
![Page 31: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/31.jpg)
A model of associative memory
![Page 32: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/32.jpg)
A model of associative memory
![Page 33: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/33.jpg)
‘Biological’ memories
• Associative: recall is based on content rather than on the address
• A transient cue induces a sustained recall
• Robust to minor failures of the hardware
• Distributed
![Page 34: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/34.jpg)
The mathematical model
I will use a slightly different model than the one presented in the last 10 minutes of Wednesday’s class
![Page 35: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/35.jpg)
The mathematical model
1 sgn
12
i
i
h tS t
Neurons are binary:
The activity of neuron i, Si = 0,1
at time t+1 input to neuron i at time t
![Page 36: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/36.jpg)
The mathematical model
i ij jj
h J S
1
2
5
3
4
J51
J21
J32
![Page 37: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/37.jpg)
A memory pattern is a vector of desired neural activities
For example:
1,0,0,1,1p
The mathematical model
1
2
5
3
4
![Page 38: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/38.jpg)
1 111 0.5 0.5 n n
ij ij i jJ n J n p pN
The Hopfield model
trial n +1
![Page 39: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/39.jpg)
1 111 0.5 0.5 n n
ij ij i jJ n J n p pN
The Hopfield model
![Page 40: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/40.jpg)
“Neurons that fire together wire together”
1 111 0.5 0.5 n n
ij ij i jJ n J n p pN
• local learning rule
• incremental, on-line
The Hopfield model
1
2
5
3
4
J51
J21
J32
0iiJ
![Page 41: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/41.jpg)
The Hopfield model
Questions:
1) What are the fixed points of the dynamics?
2) What is their relation with the memory pattern?
Network connections are symmetrical. It can be shown that with asynchronous updating, the dynamics necessarily converge to a fixed point.
![Page 42: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/42.jpg)
![Page 43: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/43.jpg)
![Page 44: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/44.jpg)
![Page 45: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/45.jpg)
![Page 46: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/46.jpg)
![Page 47: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/47.jpg)
![Page 48: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/48.jpg)
![Page 49: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/49.jpg)
![Page 50: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/50.jpg)
![Page 51: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/51.jpg)
![Page 52: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/52.jpg)
![Page 53: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/53.jpg)
![Page 54: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/54.jpg)
![Page 55: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/55.jpg)
![Page 56: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/56.jpg)
Hopfield.m
![Page 57: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/57.jpg)
The Hopfield model
Memory patterns:If Activities of neurons within and between patterns are independently chosen by tossing an unbiased coin then in the limit of large number o neurons, N the network can store ~N memory patterns
![Page 58: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/58.jpg)
The Perceptron
![Page 59: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/59.jpg)
![Page 60: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/60.jpg)
( )K tD
0 T
Affe
ren
ts
Vthr
V rest
tmax 0 tD
What does a neuron do?
spike
no spike
![Page 61: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/61.jpg)
Affe
ren
ts
0 Ttmax-
VthrNull
We consider a simplified case: input is synchronous
![Page 62: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/62.jpg)
Affe
ren
ts
Alternatively, input is constant
![Page 63: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/63.jpg)
The perceptron
j jj
h W X 11 sgn
2 Y h
1X 2X 3X 4X
Y1W 4W
11 sgn
2
Y W X
![Page 64: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/64.jpg)
Geometrical interpretation
1 1 2 2
W X W X W X1X 2X
Y
1W 2W
W
1W
2WX
1X
2X
![Page 65: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/65.jpg)
Geometrical interpretation
W
1W
2WX
1X
2X
1 1 2 2
cos cos sin sin
cos cos sin sin
cos
W X W X
W X W X
W X
W X W X W X
W X W X
W X
W X
![Page 66: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/66.jpg)
Geometrical interpretation
11 sgn
21
sgn cos2
Y W X
1X 2X
Y
1W 2W
W
1W
2W
X
1X
2X
![Page 67: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/67.jpg)
The perceptron
The Perceptron categorizes the space of inputs into inputs that should evoke a response and inputs that should not evoke a response
![Page 68: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/68.jpg)
Constraints on possible categorizations
1 11 sgn 1 sgn
2 2
i i
i
Y W X W X
1X
2X
![Page 69: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/69.jpg)
Constraints on possible categorizations
1X
2X
1 11 sgn 1 sgn
2 2
i i
i
Y W X W X
![Page 70: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/70.jpg)
Constraints on possible categorizations
1X
2X
1 11 sgn 1 sgn
2 2
i i
i
Y W X W X
![Page 71: one of the learned stimuli](https://reader035.vdocuments.net/reader035/viewer/2022062517/568131b2550346895d981f3f/html5/thumbnails/71.jpg)
Constraints on possible categorizations
1X
2X
22 1 0 X X