neural network theory
Post on 30-Dec-2015
34 Views
Preview:
DESCRIPTION
TRANSCRIPT
NEURAL NETWORK THEORY
NEURAL DYNAMIC1:
ACTIVATIONHS AND SIGNALS
MAIN POINTS:
• NEURONS AS FUNCTIONS(神经元函数)
• SIGNAL MONOTONICITY(信号单调性)• BIOLOGICAL ACTIVATIONS AND SIGNALS
(生物激励与信号)• NEURON FIELDS(神经域)• NEURONAL DYNAMICAL SYSTEMS(神经诊断系统)• COMMON SIGNAL FUNCTION(一般信号方程)• PULSE-CODED SIGNAL FUNCTION
(脉冲编码信号方程)
NEURONS AS FUNCTION
Figure 1. Neuron Structure Model
ij
n
jjii xI
1
jIfy
,
Relationship of input-output:
•Common nonlinear transduction description: a sigmoidal or S-shaped curve
Fig.2 s(x) is a bounded monotone-nondecreasing function of x
Signal Function: cxe
xS
1
1)( )0(0)1(' cScS
dx
dSS
Neurons transduce an unbounded input activation x(t) at time t into a bounded output signal S(x(t)).
NEURONS AS FUNCTION
SIGNAL MONOTONICITY
0)(2
cexS cx
• In general, signal functions are monotone nondecreasing S’>=0. In practice this means signal functions have an upper bound or saturation value.
• An important exception: bell-shaped signal function or Gaussian signal functions
xScxeS cx ',2'2
The sign of the signal-activation derivation s’ is opposite the sign of the activation x. We shall assume signal functions are monotone nondecreasing unless stated otherwise.
SIGNAL MONOTONICITY
])(2
1exp[)( 2
2 n
j
ijj
ii xxS
2i
:
•Generalized Gaussian signal function define potential or radial basis function:
n
n Rxxx ),,( 1 input activation vector:
variance:
mean vector: ),,( 1
i
n
i
i
)( ii xS
Because the function depend on all neuronal activations not just the ith activation, we shall consider only scalar-input signal functions:
SIGNAL MONOTONICITY
• A property of signal monotonicity: semi-linearity
• Comparation:
a. Linear signal functions: computation and analysis is comparatively easy; do not suppress
noise.
b. Nonlinear signal functions: Increases a network’s computational richness and facilitates noise suppression; risks computational and analytical intractability;
SIGNAL MONOTONICITY
xSdt
dx
dx
dSS '
•Signal and activation velocities
the signal velocity:
=dS/dtS
Signal velocities depend explicitly on action velocities. This dependence will increase the number of unsupervised learning laws.
BIOLOGICAL ACTIVATIONS AND SIGNALS
Fig3. Key functional units of a biological neuron
•Introduction to units :
Dendrite: input
Axon: output
Synapse: transduce signal
Membrane: potential difference between inside and outside of neuron
BIOLOGICAL ACTIVATIONS AND SIGNALS
•Competitive Neuronal Signal
Signal values are usually binary and bipolar.
Bipolar signal functions :
Binary signal functions :
Tx
Txxs
0
1
Tx
Txxs
1
1
NEURON FIELDS In general, neural networks contain many fields of neurons. Neurons within a field are topological.
Denotation:ZYX FFF
XF : input field zF : output field
xs mm yxyxyx ,......,,, 2211
Neural system samples the function m times to generate the associated
•Classification: Zeroth-order topological (simplest)
Three-dimensional and volume topological (complex)
pairs
NEURONAL DYNAMICAL SYSTEMS •Description:
A system of first-order differential or difference equations that govern the time evolution of the neuronal activations or membrane potentials
Activation differential equations:
.....,,......,
.....,,......,
21
21
yyyFFhy
xxxFFgx
YX
YX
ii yx , denote the activation time functions of the ith neuron in XF
and jth neuron in YF
•Classification: Automomous systems: activations are independent of t
Nonautonomous systems: depend on t
NEURONAL DYNAMICAL SYSTEMS
pn
YX RRFF ,
ZYX FFF ]|[
p
n
n
x
RtytytY
RtxtxtX
,...
,...,
1
1
•Neuronal State spaces
So the state space of the entire neuronal dynamical system is:
Augmentation : pnz RF
Concatenate fields have different computational, metrical or other properties
NEURONAL DYNAMICAL SYSTEMS
Signal state spaces as hypercubes
)))((,)),((())(( 11 txStxStXS n
X
n
X
Fig.4 Neural and fuzzy computations conincide.
NEURONAL DYNAMICAL SYSTEMS
•Neuronal activations as short-term memory
Short-term memory(STM) : activation
Long-term memory(LTM) : synapse
COMMON SIGNAL FUNCTION
1 、 Liner Function
S(x) = cx + k , c>0
x
S
o
k
COMMON SIGNAL FUNCTION
2. Ramp Function
r if x≥θ
S(x)= cx if |x|<θ
-r if x≤-θ
r>0, r is a constant.
r
-r
θ -θ x
S
COMMON SIGNAL FUNCTION
else
cxif
cxif
cx
xS 0
1
0
1
)(
)),max(,min()( cxxS 01
3、 threshold linear signal function: a special Ramp Function
Another form:
0cS '
COMMON SIGNAL FUNCTION
xc
xc
xc
cx
ee
e
exS
22
2
1
1)(
4 、 logistic signal function:
Where c>0.
01 )(' ScSS
So the logistic signal function is monotone increasing.
COMMON SIGNAL FUNCTION
5 、 threshold signal function:
Txif
Txif
Txif
xSxSk
k
k
kk
1
1
1
1
0
)(
1
)(
Where T is an arbitrary real-valued threshold,and k indicates the discrete time step.
COMMON SIGNAL FUNCTION
)tanh()( cxxS
01 2 )(' ScS
6 、 hyperbolic-tangent signal function:
Another form:
cxcx
cxcx
ee
eecx
)tanh(
COMMON SIGNAL FUNCTION
),min()( cxexS 1
1cxe
0 cxceS '
7 、 threshold exponential signal function:
When
02 cxecS ''
0 cxnn ecS )(
COMMON SIGNAL FUNCTION
),max()( cxexS 10
0x
0 cxceS '
8 、 exponential-distribution signal function:
When
0'' 2 cxecS
COMMON SIGNAL FUNCTION
),max()(n
n
xc
xxS
0
1n
9 、 the family of ratio-polynomial signal function:
An example
For
02
1
)('
n
n
xc
cnxS
PULSE-CODED SIGNAL FUNCTION •Description: Pulse trains arriving in a sampling interval seems to be the bearer
of neuronal signal information.
t
ts
ii dsesxtS )()(
t
ts
jj dsesytS )()(
Pulse-coded formulation:
tatpulsenoif
tatoccurspulseaiftxi
0
1)(
where
ii yx , denote binary pulse functions that summarize the excitation of
membrane potential.
PULSE-CODED SIGNAL FUNCTION
)()()( tStxtS iii
)()()( tStytS jjj
tatpulsenoif
tatoccurspulseaiftxi
0
1)(
•Velocity-difference property of pulse-coded signals
Current pulse and current signal or expected pulse frequency are available quantities.
Another computational advantage:
If
arrivedpulsenots
arrivedpulsesustainedats
0
01
A simple form for the signal velocity:
欢迎大家提出意见建议!
常虹 2006.9.25
top related