bez tytułu slajdu - warsaw university of technologymacukow/wspolne/nn/lecture3.pdf ·...
TRANSCRIPT
![Page 1: Bez tytułu slajdu - Warsaw University of Technologymacukow/wspolne/nn/Lecture3.pdf · 2015-12-30 · 6 Lower region of the brain and regions in the cerebellum are coordinating the](https://reader033.vdocuments.net/reader033/viewer/2022050119/5f4f493b2afa395c63033d85/html5/thumbnails/1.jpg)
1
Lecture 3
Neural Networks
![Page 2: Bez tytułu slajdu - Warsaw University of Technologymacukow/wspolne/nn/Lecture3.pdf · 2015-12-30 · 6 Lower region of the brain and regions in the cerebellum are coordinating the](https://reader033.vdocuments.net/reader033/viewer/2022050119/5f4f493b2afa395c63033d85/html5/thumbnails/2.jpg)
2
Principles to which
the nervous system works
![Page 3: Bez tytułu slajdu - Warsaw University of Technologymacukow/wspolne/nn/Lecture3.pdf · 2015-12-30 · 6 Lower region of the brain and regions in the cerebellum are coordinating the](https://reader033.vdocuments.net/reader033/viewer/2022050119/5f4f493b2afa395c63033d85/html5/thumbnails/3.jpg)
3
Some biology and neurophysiology
Nervous system • central nervous system • peripheral nervous system • autonomic nervous system
![Page 4: Bez tytułu slajdu - Warsaw University of Technologymacukow/wspolne/nn/Lecture3.pdf · 2015-12-30 · 6 Lower region of the brain and regions in the cerebellum are coordinating the](https://reader033.vdocuments.net/reader033/viewer/2022050119/5f4f493b2afa395c63033d85/html5/thumbnails/4.jpg)
4
Diagram of the nervous system
![Page 5: Bez tytułu slajdu - Warsaw University of Technologymacukow/wspolne/nn/Lecture3.pdf · 2015-12-30 · 6 Lower region of the brain and regions in the cerebellum are coordinating the](https://reader033.vdocuments.net/reader033/viewer/2022050119/5f4f493b2afa395c63033d85/html5/thumbnails/5.jpg)
5
Central nervous system has three hierarchical levels:
• the spinal cord level, • the lower brain level, • the cortical level.
The spinal cord acts as the organ controlling the simplest reaction of the organism (spinal reflexes)
Some biology and neurophysiology
![Page 6: Bez tytułu slajdu - Warsaw University of Technologymacukow/wspolne/nn/Lecture3.pdf · 2015-12-30 · 6 Lower region of the brain and regions in the cerebellum are coordinating the](https://reader033.vdocuments.net/reader033/viewer/2022050119/5f4f493b2afa395c63033d85/html5/thumbnails/6.jpg)
6
Lower region of the brain and regions in the cerebellum are coordinating the motor activities, orientation in space, general regulation of body (temperature, blood pressure etc.) Cerebral cortex establish interrelations between lower regions and coordinating their functions. Decision are taking, information is stored in cerebral cortex,
Some biology and neurophysiology
![Page 7: Bez tytułu slajdu - Warsaw University of Technologymacukow/wspolne/nn/Lecture3.pdf · 2015-12-30 · 6 Lower region of the brain and regions in the cerebellum are coordinating the](https://reader033.vdocuments.net/reader033/viewer/2022050119/5f4f493b2afa395c63033d85/html5/thumbnails/7.jpg)
7
Peripheral nervous system composed of the nerve processes running out from the brain and spinal cord. Nerves are the connections for communication between centers and organs.
Some biology and neurophysiology
![Page 8: Bez tytułu slajdu - Warsaw University of Technologymacukow/wspolne/nn/Lecture3.pdf · 2015-12-30 · 6 Lower region of the brain and regions in the cerebellum are coordinating the](https://reader033.vdocuments.net/reader033/viewer/2022050119/5f4f493b2afa395c63033d85/html5/thumbnails/8.jpg)
8
The task of the Autonomous nervous system is to control the most important vital processes such as breathing, blood circulation, concentration of chemicals in the blood etc.
Some biology and neurophysiology
![Page 9: Bez tytułu slajdu - Warsaw University of Technologymacukow/wspolne/nn/Lecture3.pdf · 2015-12-30 · 6 Lower region of the brain and regions in the cerebellum are coordinating the](https://reader033.vdocuments.net/reader033/viewer/2022050119/5f4f493b2afa395c63033d85/html5/thumbnails/9.jpg)
9
Functional scheme of connections of the nervous system:
1. an afferent system 2. a central association decision making
system 3. an efferent system
Some biology and neurophysiology
![Page 10: Bez tytułu slajdu - Warsaw University of Technologymacukow/wspolne/nn/Lecture3.pdf · 2015-12-30 · 6 Lower region of the brain and regions in the cerebellum are coordinating the](https://reader033.vdocuments.net/reader033/viewer/2022050119/5f4f493b2afa395c63033d85/html5/thumbnails/10.jpg)
10
Some biology and neurophysiology
Association – Decision
System
Efferent ways Afferent ways
Stimuli Activities
![Page 11: Bez tytułu slajdu - Warsaw University of Technologymacukow/wspolne/nn/Lecture3.pdf · 2015-12-30 · 6 Lower region of the brain and regions in the cerebellum are coordinating the](https://reader033.vdocuments.net/reader033/viewer/2022050119/5f4f493b2afa395c63033d85/html5/thumbnails/11.jpg)
11
Afferent ways an afferent system in which signals arriving from the environment are transmitted and analyzed, the degree and mode of analysis is controlled by superior coordinating and decision making system, multi level and hierarchical structures supplying the brain with information about external world (environment).
Some biology and neurophysiology
![Page 12: Bez tytułu slajdu - Warsaw University of Technologymacukow/wspolne/nn/Lecture3.pdf · 2015-12-30 · 6 Lower region of the brain and regions in the cerebellum are coordinating the](https://reader033.vdocuments.net/reader033/viewer/2022050119/5f4f493b2afa395c63033d85/html5/thumbnails/12.jpg)
12
The efferent system in which, on the basis of the decision taken a plan of reaction of the organism is worked out, on the base of static and dynamic situation, experience and optimization rules, output channels of a nervous system responsible for transmission and processing of signals
controlling the effectors
Some biology and neurophysiology
![Page 13: Bez tytułu slajdu - Warsaw University of Technologymacukow/wspolne/nn/Lecture3.pdf · 2015-12-30 · 6 Lower region of the brain and regions in the cerebellum are coordinating the](https://reader033.vdocuments.net/reader033/viewer/2022050119/5f4f493b2afa395c63033d85/html5/thumbnails/13.jpg)
13
The central association and decision making system where a decision about the reaction of the organism is worked out on the basis of the state of the environment, the state of the organism, previous experience, and a prediction of effect
Some biology and neurophysiology
![Page 14: Bez tytułu slajdu - Warsaw University of Technologymacukow/wspolne/nn/Lecture3.pdf · 2015-12-30 · 6 Lower region of the brain and regions in the cerebellum are coordinating the](https://reader033.vdocuments.net/reader033/viewer/2022050119/5f4f493b2afa395c63033d85/html5/thumbnails/14.jpg)
14
Some biology and neurophysiology
![Page 15: Bez tytułu slajdu - Warsaw University of Technologymacukow/wspolne/nn/Lecture3.pdf · 2015-12-30 · 6 Lower region of the brain and regions in the cerebellum are coordinating the](https://reader033.vdocuments.net/reader033/viewer/2022050119/5f4f493b2afa395c63033d85/html5/thumbnails/15.jpg)
15
Nerve cell models
The first model of neuron was proposed in 1943 by W.S. McCulloch and W.Pitts
The model came from the research into behavior of the neurons in the brain. It was very simple unit, thresholding the weighted sum of its inputs to get an output.
It was the result of the actual state of knowledge and used the methods of mathematical and formal logic.
The element was also called a formal neuron.
Θ
![Page 16: Bez tytułu slajdu - Warsaw University of Technologymacukow/wspolne/nn/Lecture3.pdf · 2015-12-30 · 6 Lower region of the brain and regions in the cerebellum are coordinating the](https://reader033.vdocuments.net/reader033/viewer/2022050119/5f4f493b2afa395c63033d85/html5/thumbnails/16.jpg)
16
Nerve cell models
Θ
![Page 17: Bez tytułu slajdu - Warsaw University of Technologymacukow/wspolne/nn/Lecture3.pdf · 2015-12-30 · 6 Lower region of the brain and regions in the cerebellum are coordinating the](https://reader033.vdocuments.net/reader033/viewer/2022050119/5f4f493b2afa395c63033d85/html5/thumbnails/17.jpg)
17
Nerve cell models
The formal neuron was characterized by describing its state (or output).
Changing of the state from inactive (0) to active (1) was when the weighted sum of input signals was greater than the threshold; and there was no inhibitory input.
![Page 18: Bez tytułu slajdu - Warsaw University of Technologymacukow/wspolne/nn/Lecture3.pdf · 2015-12-30 · 6 Lower region of the brain and regions in the cerebellum are coordinating the](https://reader033.vdocuments.net/reader033/viewer/2022050119/5f4f493b2afa395c63033d85/html5/thumbnails/18.jpg)
18
Model assumptions:
1. The element activity is based on the „all-or-none” principle.
2. The excitation (state 1) is preceded by a constant delay while accumulating the signals incoming to synapses (independent from the previous activity and localization of synapses).
3. The only neuronal delay between the input simulation and activity at the output, is the synaptic delay.
Nerve cell models
![Page 19: Bez tytułu slajdu - Warsaw University of Technologymacukow/wspolne/nn/Lecture3.pdf · 2015-12-30 · 6 Lower region of the brain and regions in the cerebellum are coordinating the](https://reader033.vdocuments.net/reader033/viewer/2022050119/5f4f493b2afa395c63033d85/html5/thumbnails/19.jpg)
19
Model assumptions:
3. Stimulation of any inhibitory input excludes a response at the output at the moment under consideration.
4. The net structure and neuron properties do not change with time.
Nerve cell models
![Page 20: Bez tytułu slajdu - Warsaw University of Technologymacukow/wspolne/nn/Lecture3.pdf · 2015-12-30 · 6 Lower region of the brain and regions in the cerebellum are coordinating the](https://reader033.vdocuments.net/reader033/viewer/2022050119/5f4f493b2afa395c63033d85/html5/thumbnails/20.jpg)
20
The discrete time is logical, because in the real neuron, after the action potential, the membrane is non-excitable, i.e. another impulse cannot be generated (appr. 1 ms). This interval is called the absolute refractory period.
It specifies the maximum impulse repetition rate to about 1000 impulses per second.
Nerve cell models
![Page 21: Bez tytułu slajdu - Warsaw University of Technologymacukow/wspolne/nn/Lecture3.pdf · 2015-12-30 · 6 Lower region of the brain and regions in the cerebellum are coordinating the](https://reader033.vdocuments.net/reader033/viewer/2022050119/5f4f493b2afa395c63033d85/html5/thumbnails/21.jpg)
21
Mathematical models of a nerve cell
The methods of selection of the properties of neural element depends not only on previous results, our level of knowledge – but mainly from the phenomena to be modeled. Another properties will be important while modeling the steady states, another for dynamic processes or for the learning processes.
But always, the model has to be as simple as possible.
![Page 22: Bez tytułu slajdu - Warsaw University of Technologymacukow/wspolne/nn/Lecture3.pdf · 2015-12-30 · 6 Lower region of the brain and regions in the cerebellum are coordinating the](https://reader033.vdocuments.net/reader033/viewer/2022050119/5f4f493b2afa395c63033d85/html5/thumbnails/22.jpg)
22
Functional model of a neuron: 1.input signals • adding signals (inhibitory and excitatory), • multiplying signals without memory , • multiplying signals with memory.
Physiologically –synapses (basically linear units).
Mathematical models of a nerve cell
![Page 23: Bez tytułu slajdu - Warsaw University of Technologymacukow/wspolne/nn/Lecture3.pdf · 2015-12-30 · 6 Lower region of the brain and regions in the cerebellum are coordinating the](https://reader033.vdocuments.net/reader033/viewer/2022050119/5f4f493b2afa395c63033d85/html5/thumbnails/23.jpg)
23
xi(t) – input signals, continuous time functions,
ui(t) – input to multiplying inputs with memory, weight control vi, - continuous time functions,
s(t) –facilitation hypothesis, (influence of long-lasting signals).
Mathematical models of a nerve cell
![Page 24: Bez tytułu slajdu - Warsaw University of Technologymacukow/wspolne/nn/Lecture3.pdf · 2015-12-30 · 6 Lower region of the brain and regions in the cerebellum are coordinating the](https://reader033.vdocuments.net/reader033/viewer/2022050119/5f4f493b2afa395c63033d85/html5/thumbnails/24.jpg)
24
weight control
tdttstut
vtsuv
itz
oiii
0
)()()exp()exp(
),,(
initial weight
storage coefficient forgetting
time-delay
weight increment
constrain
hence: ),,(*)()( tsuvtxtx iiivi
Mathematical models of a nerve cell
![Page 25: Bez tytułu slajdu - Warsaw University of Technologymacukow/wspolne/nn/Lecture3.pdf · 2015-12-30 · 6 Lower region of the brain and regions in the cerebellum are coordinating the](https://reader033.vdocuments.net/reader033/viewer/2022050119/5f4f493b2afa395c63033d85/html5/thumbnails/25.jpg)
25
2. analog adder 3. threshold impulse generator 4. delay element
Mathematical models of a nerve cell
![Page 26: Bez tytułu slajdu - Warsaw University of Technologymacukow/wspolne/nn/Lecture3.pdf · 2015-12-30 · 6 Lower region of the brain and regions in the cerebellum are coordinating the](https://reader033.vdocuments.net/reader033/viewer/2022050119/5f4f493b2afa395c63033d85/html5/thumbnails/26.jpg)
26
Functional model of a neuron
![Page 27: Bez tytułu slajdu - Warsaw University of Technologymacukow/wspolne/nn/Lecture3.pdf · 2015-12-30 · 6 Lower region of the brain and regions in the cerebellum are coordinating the](https://reader033.vdocuments.net/reader033/viewer/2022050119/5f4f493b2afa395c63033d85/html5/thumbnails/27.jpg)
27
Neural cell models
![Page 28: Bez tytułu slajdu - Warsaw University of Technologymacukow/wspolne/nn/Lecture3.pdf · 2015-12-30 · 6 Lower region of the brain and regions in the cerebellum are coordinating the](https://reader033.vdocuments.net/reader033/viewer/2022050119/5f4f493b2afa395c63033d85/html5/thumbnails/28.jpg)
28
Two kinds of neurons: static and dynamic.
Static model of neuron (used in continuous and discrete
static nets and dynamic discrete sets).
Input signals
x1
x2
xm
.
.
.
wk1
xj
.
.
.
wk2
wkj
wkm
m
.
.
.
.
.
.
g()
Weights
θk
Threshold
nk f()
Output
yk
Activation function
Transfer function
Activation signal
Output Signal
Bias
w
Inputs
Model of a static neuron k
Static model of neural cell
![Page 29: Bez tytułu slajdu - Warsaw University of Technologymacukow/wspolne/nn/Lecture3.pdf · 2015-12-30 · 6 Lower region of the brain and regions in the cerebellum are coordinating the](https://reader033.vdocuments.net/reader033/viewer/2022050119/5f4f493b2afa395c63033d85/html5/thumbnails/29.jpg)
29
Simplified models of static neuron
x1
x2
xm
.
.
.
xj
.
.
.
g
θk
yk
f
wk1
wk2
wkj
wkm
1
or
x1
x2
xm
.
.
.
xj
.
.
.
g yk
wk1
wk2
wkj
wkm
+1
wk0
![Page 30: Bez tytułu slajdu - Warsaw University of Technologymacukow/wspolne/nn/Lecture3.pdf · 2015-12-30 · 6 Lower region of the brain and regions in the cerebellum are coordinating the](https://reader033.vdocuments.net/reader033/viewer/2022050119/5f4f493b2afa395c63033d85/html5/thumbnails/30.jpg)
30
Dynamic model of neuron
External
recursion
Self
recursion
Internal self-feedback
![Page 31: Bez tytułu slajdu - Warsaw University of Technologymacukow/wspolne/nn/Lecture3.pdf · 2015-12-30 · 6 Lower region of the brain and regions in the cerebellum are coordinating the](https://reader033.vdocuments.net/reader033/viewer/2022050119/5f4f493b2afa395c63033d85/html5/thumbnails/31.jpg)
31
Activation functions
1. Adder
2. Product
3. Maximum
4. Minimum
5. Dominant
6. Cumulative sum
![Page 32: Bez tytułu slajdu - Warsaw University of Technologymacukow/wspolne/nn/Lecture3.pdf · 2015-12-30 · 6 Lower region of the brain and regions in the cerebellum are coordinating the](https://reader033.vdocuments.net/reader033/viewer/2022050119/5f4f493b2afa395c63033d85/html5/thumbnails/32.jpg)
32
Sum function
m m
jkjkkkk xwnorxn1j 0j
jj =w
Product function
m
1j
jjkk xwn
Maximum
jkk xwn j
jmax
Activation functions
(k – neuron’s number, j - number of neuron’s input)
![Page 33: Bez tytułu slajdu - Warsaw University of Technologymacukow/wspolne/nn/Lecture3.pdf · 2015-12-30 · 6 Lower region of the brain and regions in the cerebellum are coordinating the](https://reader033.vdocuments.net/reader033/viewer/2022050119/5f4f493b2afa395c63033d85/html5/thumbnails/33.jpg)
33
Minimum
Dominant function
Cumulative sum function
jkjk xwn min
j
m
1j
jkn where
otherwise
wif
0
01 ji,
j
m
1j
)i(
j
)i(
j,k
)i()1i( xwnn where i - iteration
(k – neuron’s number, j - number of neuron’s input)
Activation functions
![Page 34: Bez tytułu slajdu - Warsaw University of Technologymacukow/wspolne/nn/Lecture3.pdf · 2015-12-30 · 6 Lower region of the brain and regions in the cerebellum are coordinating the](https://reader033.vdocuments.net/reader033/viewer/2022050119/5f4f493b2afa395c63033d85/html5/thumbnails/34.jpg)
34
Unipolar function Bipolar function
otherwise
ify
1
0n0
otherwise
ify
1
0n1
Transfer functions
![Page 35: Bez tytułu slajdu - Warsaw University of Technologymacukow/wspolne/nn/Lecture3.pdf · 2015-12-30 · 6 Lower region of the brain and regions in the cerebellum are coordinating the](https://reader033.vdocuments.net/reader033/viewer/2022050119/5f4f493b2afa395c63033d85/html5/thumbnails/35.jpg)
35
Linear function Linear positive function
ny
otherwisen
ify
0n0
Transfer functions
![Page 36: Bez tytułu slajdu - Warsaw University of Technologymacukow/wspolne/nn/Lecture3.pdf · 2015-12-30 · 6 Lower region of the brain and regions in the cerebellum are coordinating the](https://reader033.vdocuments.net/reader033/viewer/2022050119/5f4f493b2afa395c63033d85/html5/thumbnails/36.jpg)
36
Linear function with
saturation (non symmetric)
Linear function with
saturation (symmetric)
nif
nifn
nif
y
1
0
00
nif
nifn
nif
y
1
1
Transfer functions
![Page 37: Bez tytułu slajdu - Warsaw University of Technologymacukow/wspolne/nn/Lecture3.pdf · 2015-12-30 · 6 Lower region of the brain and regions in the cerebellum are coordinating the](https://reader033.vdocuments.net/reader033/viewer/2022050119/5f4f493b2afa395c63033d85/html5/thumbnails/37.jpg)
37
Unipolar sigmoidal function
(log) (non symmetric)
Bipolar sigmoidal function
(log) (symmetric)
)exp(1
1
ny
1
)exp(1
2
ny
Transfer functions
}10 7; 5; 3; 2; 1.5; 1; 0.5;{
![Page 38: Bez tytułu slajdu - Warsaw University of Technologymacukow/wspolne/nn/Lecture3.pdf · 2015-12-30 · 6 Lower region of the brain and regions in the cerebellum are coordinating the](https://reader033.vdocuments.net/reader033/viewer/2022050119/5f4f493b2afa395c63033d85/html5/thumbnails/38.jpg)
38
Unipolar sigmoidal function
(th) (non symmetric)
Bipolar sigmoidal function
(th) (symmetric)
2
1
)exp()exp(
)exp()exp(
2
1
2
1)tanh(
2
1
nn
nnny
)exp()exp(
)exp()exp()tanh(
nn
nnny
Transfer functions
}10 7; 5; 3; 2; 1.5; 1; 0.5;{
![Page 39: Bez tytułu slajdu - Warsaw University of Technologymacukow/wspolne/nn/Lecture3.pdf · 2015-12-30 · 6 Lower region of the brain and regions in the cerebellum are coordinating the](https://reader033.vdocuments.net/reader033/viewer/2022050119/5f4f493b2afa395c63033d85/html5/thumbnails/39.jpg)
39
Gauss function
(Radial Basis)
2
2
nexpy
Transfer functions
}10 7; 5; 3; 2; 1.5; 1; 0.5;{
![Page 40: Bez tytułu slajdu - Warsaw University of Technologymacukow/wspolne/nn/Lecture3.pdf · 2015-12-30 · 6 Lower region of the brain and regions in the cerebellum are coordinating the](https://reader033.vdocuments.net/reader033/viewer/2022050119/5f4f493b2afa395c63033d85/html5/thumbnails/40.jpg)
40
Electronic models
Electronic neural cell model due to McGrogan
![Page 41: Bez tytułu slajdu - Warsaw University of Technologymacukow/wspolne/nn/Lecture3.pdf · 2015-12-30 · 6 Lower region of the brain and regions in the cerebellum are coordinating the](https://reader033.vdocuments.net/reader033/viewer/2022050119/5f4f493b2afa395c63033d85/html5/thumbnails/41.jpg)
41
Electronic neural cell model due to Harmon.
Electronic models
![Page 42: Bez tytułu slajdu - Warsaw University of Technologymacukow/wspolne/nn/Lecture3.pdf · 2015-12-30 · 6 Lower region of the brain and regions in the cerebellum are coordinating the](https://reader033.vdocuments.net/reader033/viewer/2022050119/5f4f493b2afa395c63033d85/html5/thumbnails/42.jpg)
42
Electronic neural cell model due to Harmon.
Electronic models
![Page 43: Bez tytułu slajdu - Warsaw University of Technologymacukow/wspolne/nn/Lecture3.pdf · 2015-12-30 · 6 Lower region of the brain and regions in the cerebellum are coordinating the](https://reader033.vdocuments.net/reader033/viewer/2022050119/5f4f493b2afa395c63033d85/html5/thumbnails/43.jpg)
43
Electronic models
Electronic neural cell model due to Taylor.
![Page 44: Bez tytułu slajdu - Warsaw University of Technologymacukow/wspolne/nn/Lecture3.pdf · 2015-12-30 · 6 Lower region of the brain and regions in the cerebellum are coordinating the](https://reader033.vdocuments.net/reader033/viewer/2022050119/5f4f493b2afa395c63033d85/html5/thumbnails/44.jpg)
44
Electronic neural cell model due to Küpfmüller and Janik
Electronic models
![Page 45: Bez tytułu slajdu - Warsaw University of Technologymacukow/wspolne/nn/Lecture3.pdf · 2015-12-30 · 6 Lower region of the brain and regions in the cerebellum are coordinating the](https://reader033.vdocuments.net/reader033/viewer/2022050119/5f4f493b2afa395c63033d85/html5/thumbnails/45.jpg)
45
Neuron model built in the Bionics Laboratory IA PAS, in 1969
Neural network model built in the Bionics Laboratory IA PAS, in 1969
Models built in the Bionics Laboratory, PAS
![Page 46: Bez tytułu slajdu - Warsaw University of Technologymacukow/wspolne/nn/Lecture3.pdf · 2015-12-30 · 6 Lower region of the brain and regions in the cerebellum are coordinating the](https://reader033.vdocuments.net/reader033/viewer/2022050119/5f4f493b2afa395c63033d85/html5/thumbnails/46.jpg)
46
Simplified model of a neural cell
![Page 47: Bez tytułu slajdu - Warsaw University of Technologymacukow/wspolne/nn/Lecture3.pdf · 2015-12-30 · 6 Lower region of the brain and regions in the cerebellum are coordinating the](https://reader033.vdocuments.net/reader033/viewer/2022050119/5f4f493b2afa395c63033d85/html5/thumbnails/47.jpg)
47
McCulloch Symbolism
![Page 48: Bez tytułu slajdu - Warsaw University of Technologymacukow/wspolne/nn/Lecture3.pdf · 2015-12-30 · 6 Lower region of the brain and regions in the cerebellum are coordinating the](https://reader033.vdocuments.net/reader033/viewer/2022050119/5f4f493b2afa395c63033d85/html5/thumbnails/48.jpg)
48
McCulloch-Pitts Model
In 1943 Warren McCulloch and Walter Pitts
proposed the first simple mathematics model of a
neuron as a two-values threshold element.
The McCulloch-Pitts neuron calculates the
weighted sum of input signals incoming from
other neurons and produce at the output value 1
(on) or 0 (off) depending the sum is greater or
smaller from the threshold value.
![Page 49: Bez tytułu slajdu - Warsaw University of Technologymacukow/wspolne/nn/Lecture3.pdf · 2015-12-30 · 6 Lower region of the brain and regions in the cerebellum are coordinating the](https://reader033.vdocuments.net/reader033/viewer/2022050119/5f4f493b2afa395c63033d85/html5/thumbnails/49.jpg)
49
McCulloch-Pitts Model
McCulloch and Pitts model of a single neuron
j
ijiji txwfty 1
Transfer function
otherwise
nifxf
0
0 1
yi n
x1
x2
.
.
.
.
. θi
![Page 50: Bez tytułu slajdu - Warsaw University of Technologymacukow/wspolne/nn/Lecture3.pdf · 2015-12-30 · 6 Lower region of the brain and regions in the cerebellum are coordinating the](https://reader033.vdocuments.net/reader033/viewer/2022050119/5f4f493b2afa395c63033d85/html5/thumbnails/50.jpg)
50
Neural cell models
inputs
inhibitory
excitatory
output
McCulloch and Pitts Model
n
j
jj
n
j
jj
n
j
jj
txv
txv
ty
or
txvty
1
1
1
)(0
)(1
)1(
2
1)(sgn
2
1)1(
when
when
x1
x2
xn
y
![Page 51: Bez tytułu slajdu - Warsaw University of Technologymacukow/wspolne/nn/Lecture3.pdf · 2015-12-30 · 6 Lower region of the brain and regions in the cerebellum are coordinating the](https://reader033.vdocuments.net/reader033/viewer/2022050119/5f4f493b2afa395c63033d85/html5/thumbnails/51.jpg)
51
McCulloch and Pitts models
Neural cell models
![Page 52: Bez tytułu slajdu - Warsaw University of Technologymacukow/wspolne/nn/Lecture3.pdf · 2015-12-30 · 6 Lower region of the brain and regions in the cerebellum are coordinating the](https://reader033.vdocuments.net/reader033/viewer/2022050119/5f4f493b2afa395c63033d85/html5/thumbnails/52.jpg)
52
From these simple elements - formal neurons - the nets simulating complex operations or some forms of the behavior of living organisms can be modeled
S1
S2
S3 2
S3 = S1 S2
Simple nets build from McCulloch &Pitts
elements
![Page 53: Bez tytułu slajdu - Warsaw University of Technologymacukow/wspolne/nn/Lecture3.pdf · 2015-12-30 · 6 Lower region of the brain and regions in the cerebellum are coordinating the](https://reader033.vdocuments.net/reader033/viewer/2022050119/5f4f493b2afa395c63033d85/html5/thumbnails/53.jpg)
53
S1
S2
S3 2
S3 = S1 S2
From these simple elements - formal neurons - the nets simulating complex operations or some forms of the behavior of living organisms can be modeled.
Simple nets build from McCulloch &Pitts
elements
![Page 54: Bez tytułu slajdu - Warsaw University of Technologymacukow/wspolne/nn/Lecture3.pdf · 2015-12-30 · 6 Lower region of the brain and regions in the cerebellum are coordinating the](https://reader033.vdocuments.net/reader033/viewer/2022050119/5f4f493b2afa395c63033d85/html5/thumbnails/54.jpg)
54
S1
S2
S3 2
S3 = S1 ~S2
Simple nets build from McCulloch &Pitts
elements
From these simple elements - formal neurons - the nets simulating complex operations or some forms of the behavior of living organisms can be modeled.
![Page 55: Bez tytułu slajdu - Warsaw University of Technologymacukow/wspolne/nn/Lecture3.pdf · 2015-12-30 · 6 Lower region of the brain and regions in the cerebellum are coordinating the](https://reader033.vdocuments.net/reader033/viewer/2022050119/5f4f493b2afa395c63033d85/html5/thumbnails/55.jpg)
55
4 5 3
2
1
6
2 2 2
2
input elements
Signal incoming to input: excites after 3 times repetition 2 6
Signals incoming to input: directly excite the element 1 6
Simple nets build from McCulloch &Pitts
elements, facilitation phenomenon
![Page 56: Bez tytułu slajdu - Warsaw University of Technologymacukow/wspolne/nn/Lecture3.pdf · 2015-12-30 · 6 Lower region of the brain and regions in the cerebellum are coordinating the](https://reader033.vdocuments.net/reader033/viewer/2022050119/5f4f493b2afa395c63033d85/html5/thumbnails/56.jpg)
56
First excitation of excite but is not enough to excite the others 2 3
4 3 3 2 4 Output from approaches (but below threshold). Totally and excite
3 This excitation yields to self excitation (positive feedback) of the
Simple nets build from McCulloch &Pitts
elements, facilitation phenomenon
4 5 3
2
1
6
2 2 2
2
input elements