stochastic resonance in continuous and spiking neuron...

Stochastic Resonance in Continuous and SpikingNeuron Models with Levy Noise

Ashok Patel and Bart Kosko

To appear in 2008 in the IEEE Transactions on Neural NetworksCurrent draft: 12 September 2008

Abstract—Levy noise can help neurons detect faint or sub-threshold signals. Levy noise extends standard Brownian noiseto many types of impulsive jump-noise processes found in realand model neurons as well as in models of finance and otherrandom phenomena. Two new theorems and the Ito calculus showthat white Levy noise will benefit subthreshold neuronal signaldetection if the noise process’s scaled drift velocity falls inside aninterval that depends on the threshold values. These results gener-alize earlier ‘forbidden interval’ theorems of neuronal ‘stochasticresonance’ or noise-injection benefits. Global and local Lipschitzconditions imply that additive white Levy noise can increase themutual information or bit count of several feedback neuronmodels that obey a general stochastic differential equation.Simulation results show that the same noise benefits still occur forsome infinite-variance stable Levy noise processes even thoughthe theorems themselves apply only to finite-variance Levy noise.The Appendix proves the two Ito-theoretic lemmas that underliethe new Levy noise-benefit theorems.

Index Terms—Levy noise, Signal detection, Stochastic reso-nance, Neuron models, Mutual information.

I. STOCHASTIC RESONANCE IN NEURAL SIGNALDETECTION

Stochastic resonance (SR) occurs when noise benefits asystem rather than harms it. Small amounts of noise can oftenenhance some forms of nonlinear signal processing whiletoo much noise degrades it [12], [13], [22], [27], [45], [49],[58], [60], [61], [69], [71], [72], [84]. SR has many usefulapplications in physics, biology, and medicine [5], [6], [7],[11], [14], [17], [18], [21], [23], [32], [40], [41], [43], [52],[53], [55], [56], [62], [70], [75], [83], [85], [89], [91]. SR inneural networks is itself part of the important and growingarea of stochastic neural networks [9], [10], [38], [86], [87],[88], [90]. We show that a wide range of general feedbackcontinuous neurons and spiking neurons benefit from a broadclass of additive white Levy noise. This appears to be the firstdemonstration of the SR effect for neuron models subject toLevy noise perturbations.

Figure 1 shows how impulsive Levy noise can enhancethe Kanisza-square visual illusion in which four dark-cornerfigures give rise to an illusory bright interior square. Eachpixel is the thresholded output of a noisy bistable neuronwhose input signals are subthreshold and quantized pixelvalues of the original noise-free Kanizsa image. The outputsof the bistable neurons do not depend on the input signalsif there is no additive noise because the input signals aresubthreshold. Figure 1(a) shows that adding infinite-variance

Ashok Patel and Bart Kosko are with Signal and Image Processing Institute,Department of Electrical Engineering, University of Southern California, LosAngeles, CA 90089-2564, USA. Email:[email protected]

Levy noise induces a slight correlation between the pixelinput and output signals. More intense Levy noise increasesthis correlation in Figures 1 (b) - (c). Still more intenseLevy noise degrades the image and undermines the visualillusion in Figures 1(d) -(e). Figure 2 shows typical samplepaths from different types of Levy noise. Figure 3 showsthe characteristic inverted-U or non-monotonic signature ofSR for white Levy noise that perturbs a continuous bistableneuron.

We generalize the recent ‘forbidden interval’ theorems[50], [51], [61], [65], [66] for continuous and spiking neuronmodels to a broad class of finite-second-moment Levy noisethat may depend on the neuron’s membrane potential. Theoriginal forbidden interval theorem [50], [51] states thatsimple threshold neurons will have an SR noise benefit in thesense that noise increases the neuron’s mutual information orbit count if and only if the noise mean or location parameterµ does not fall in a threshold-related interval: SR occurs ifand only if µ /∈ (T – A, T + A) for threshold T where− A < A < T for bipolar subthreshold signal ±A. Thetheorems below show that such an SR noise benefit will occurif the additive white Levy noise process has a bounded scaleddrift velocity that does not fall within a threshold-basedinterval. This holds for general feedback continuous neuronmodels that include common signal functions such as logisticsigmoids or Gaussians. It also holds for spiking neuronssuch as the FitzHugh-Nagumo, leaky integrate-and-fire, andreduced Type-I neuron models. We used the Ito stochasticcalculus to prove our results under the assumption that theLevy noise has a finite second moment. But Figure 1 andthe (c) sub-figures of Figures 3-8 all show that the SR noisebenefit still occurs in the more general infinite-variance caseof some types of α-stable Levy noise. So the SR effect isnot limited to finite-second-moment Levy noise. We were notable to prove that these stable infinite-variance SR effectsmust occur as we did prove with simpler neuron models [50],[51], [65].

Levy noise has advantages over standard Gaussian noise inneuron models despite its increased mathematical complexity.A Levy noise model more accurately describes how the neu-ron’s membrane potential evolves than does a simpler diffusionmodel because the more general Levy model includes not onlypure-diffusion and pure-jump models but jump-diffusion mod-els as well [35], [74]. Neuron models with additive Gaussiannoise are pure-diffusion models. These neuron models rely onthe classical central limit theorem for their Gaussian structureand thus they rely on special limiting-case assumptions of in-coming Poisson spikes from other neurons. These assumptions

2

(a) (b) (c) (d) (e)

Fig. 1. Stochastic resonance in the Kanisza square illusion with symmetric α-stable noise (α = 1.9) in a thick-tailed bell curve with infinite-variance but finite intensity or dispersion γ [30]. The Kanisza square illusion improves as the noise dispersion γ increases from 0.047 to0.3789 and then it degrades as the dispersion increases further. Each pixel represents the output of the noisy bistable potential neuron model(1)-(2) and (5) that uses the pixel values of the original Kanisza square image as subthreshold input signals. The additive α-stable noisedispersions are (a) γ = 0.047, (b) γ = 0.1015, (c) γ = 0.3789, (d) γ = 1, and (e) γ = 3.7321.

require at least that the number of impinging synapses is largeand that the synapses have small membrane effects due to thesmall coupling coefficient or the synaptic weights [28], [47].The Gaussian noise assumption may be more appropriatefor signal inputs from dendritic trees because of the sheer

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 t−0.2

−0.1

0

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 t

0

0.1

0.2

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 t

−0.2

0.1

−0.1

0

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 t

−0.2

−0.1

0

Lt

Lt

Lt

(a) Brownian diffusion

(b) jump diffusion

(c) normal inverse Gaussian process

Lt

(d) infinite−variance α−stable process

Fig. 2. Sample paths from one-dimensional Levy processes: (a)Brownian motion with drift µ = 0.1 and variance σ = 0.15, (b) jumpdiffusion with µ = 0.1, σ = 0.225, Poisson jump rate λ = 3, anduniformly distributed jump magnitudes in the interval [-0.2,0.2] (andso with Levy measure ν(dy) = (3/0.4)dy for y ∈ [−0.2, 0.2] andzero else), (c) normal inverse Gaussian (NIG) process with parametersα = 20, β = 0, δ = 0.1, and µ = 0, (d) infinite-variance α-stableprocess with α = 1.9 and dispersion κ = 0.0272 (µ = 0, σ = 0, andν(dy) is of the form k

|y|1+α dy).

number of dendrites. But often fewer inputs come fromsynapses near the post-synaptic neuron’s trigger zone andthese inputs produce impulses in noise amplitudes because ofthe higher concentration of voltage-sensitive sodium channelsin the trigger zone [29], [46], [64]. Engineering applicationsalso favor the more general Levy model because physicaldevices may be limited in their number of model-neuronconnections [59] and because real signals and noise can oftenbe impulsive [30], [63], [76].

II. NOISY FEEDBACK NEURON MODELS

We study Levy SR noise benefits in the noisy feedbackneuron models of the general form

x = −x(t) + f(x(t)) + s(t) + n(t) (1)y(t) = g(x(t)) (2)

with initial condition x(t0) = x0. Here s(t) is the additivenet excitatory or inhibitory input forcing signal—either s1 ors2. The additive noise term n(t) is Levy noise with mean orlocation µ and intensity scale κ (or dispersion γ for symmetricα-stable noise where γ = κα with characteristic function φ(u)= eiuµ−γ|u|

α

). The neuron feeds its activation or membranepotential signal x(t) back to itself through −x(t) + f(x(t))and emits the (observable) thresholded or spike signal y(t) asoutput. Here g is a static transformation function. We use thethreshold

g(x) =

1 if x > 00 else (3)

for continuous neuron models. We use a related thresholdg in spiking neuron models where g determines the spikeoccurrence. The neuronal signal function f(x) of (1) can beof quite general form for continuous neuron models [66]:

• Logistic. The logistic signal function [48] is sigmoidaland strictly increasing

f(x) =1

1 + e−cx(4)

for scaling constant c > 0. We use c = 8. This signal functiongives a bistable additive neuron model.

3

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 20

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

Scale κ of additive white Jump−Diffusion Levy noise

Mutu

al in

form

ation I(S

,R)

in b

its

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 20

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

Scale κ of additive white NIG Levy noise

Mutu

al in

form

ation I(S

,R)

in b

its

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 20

0.1

0.2

0.3

0.4

0.5

0.6

0.7

Scale κ of additive white α−stable Levy noise (α = 1.9)

Mutu

al in

form

ation I(S

,R)

in b

its

(a) (b) (c)

Fig. 3. Mutual information Levy noise benefits in the continuous bistable neuron (1)-(2) and (5). Additive white Levy noise dLt increasesthe mutual information of the bistable potential neuron for the subthreshold input signals s1 = −0.3 and s2 = 0.4. The types of Levy noisedLt are (a) Gaussian with uniformly distributed jumps, (b) pure-jump normal inverse Gaussian (NIG), and (c) symmetric α-stable noisewith α = 1.9 (thick-tailed bell curve with infinite variance [63]). The dashed vertical lines show the total min-max deviations of the mutualinformation in 100 simulation trials.

• Hyperbolic Tangent. This signal function is also sigmoidaland gives a bistable additive neuron model [2], [15], [37], [48]:

f(x) = 2 tanhx (5)

• Linear Threshold. This linear-threshold signal has theform [48]:

f(x) =

cx |cx| < 11 cx > 1−1 cx < −1

(6)

for constant c > 0. We use c = 2.

• Exponential. This signal function is asymmetric and hasthe form [48]

f(x) =

1− exp−cx if x > 00 else (7)


• Gaussian. The Gaussian or ‘radial basis’ signal function[48] differs in form from the signal functions above becauseit is nonmonotonic:

f(x) = exp−cx2 (8)


The above neuron models can have up to three fixed pointsdepending on the input signal and the model parameters.The input signal is subthreshold in the sense that switchingit from s1 to s2 or vice versa does not change the outputYt of (22). There exist θ1 and θ2 such that the input S issubthreshold when θ1 ≤ s1 < s2 ≤ θ2. The values of θ1and θ2 depend on the model parameters. Consider the linearthreshold neuron model (1)-(2) and (6) with c = 2. A simplecalculation shows that if the input signal St ∈ s1, s2satisfies −0.5 < s1 < s2 < 0.5 then the linear thresholdneuron has two stable fixed points (one positive and the othernegative) and has one unstable fixed point between them. TheGaussian neuron model (1)-(2) and (8) has only one fixed

point if 0 < s1 < s2. So the input is subthreshold becauseswitching it from s1 to s2 or vice versa does not changethe output Yt. Figure 3 shows the mutual information noisebenefits in the bistable neuron model (1)-(2) and (5) for threedifferent additive white Levy noise cases when the inputsignals are subthreshold. Note the signature nonmonotonicshape of all three SR noise-benefit curves in Figure 3.

The membrane potential dynamics (1) are one-dimensionalfor all our neuron models except for the two-dimensionalFHN spiking neuron model below. So next we briefly describemulti-dimensional Levy processes and set up a general multi-dimensional Levy stochastic differential equation frameworkfor our feedback continuous and spiking neuron models.

III. LEVY PROCESSES AND STOCHASTIC DIFFERENTIALEQUATIONS

Levy processes [68], [77] form a wide class of randomprocesses that include Brownian motion, α-stable processes,compound Poisson processes, generalized inverse Gaussianprocesses, and generalized hyperbolic processes. Figure 2shows some typical scalar Levy sample paths. Levy processescan account for the impulsiveness or discreteness of bothsignals and noise. Researchers have used Levy processes tomodel diverse phenomena in economics [4], [78], physics[81], electrical engineering [1], [4], [63], [67], biology [80],and seismology [82]. A Levy process Lt = (L1

t , ..., Lmt )′

for t ≥ 0 in a given probability space (Ω,F , (Ft)0≤t≤∞, P )is a stochastic process taking values in Rm with stationaryand independent increments (we assume that L0 = 0 withprobability 1). The Levy process Lt obeys three properties:

1) Lt−Ls is independent of sigma-algebra Fs for 0 ≤ s <t ≤ ∞

2) Lt − Ls has the same distribution as Lt−s3) Ls → Lt in probability if s → t.

4

The Levy-Khintchine formula gives the characteristic func-tion φ of Lt as [3]

φ(u) = E(ei〈u,Lt〉) = etη(u) for t ≥ 0 and u ∈ Rm (9)

where 〈·, ·〉 is the Euclidean inner product (so |u| = 〈u, u〉 12 ).The characteristic exponent or the so-called Levy exponent is

η(u) = i〈µ, u〉 − 12〈u,Ku〉

+∫Rm−0

[ei〈u,y〉- 1 -i〈u, y〉χ|y|<1(y)]ν(dy)(10)

for some µ ∈ Rm, a positive-definite symmetric m × mmatrix K, and measure ν on Borel subsets of Rm

0 = Rm\0(or ν(0) = 0). Then ν is a Levy measure such that∫

Rm0

min1, |y|2ν(dy) < ∞. (11)

A Levy process Lt combines a drift component, a Brownianmotion (Gaussian) component, and a jump component. TheLevy-Khintchine triplet (µ,K, ν) completely determines thesecomponents. The Levy measure ν determines both the averagenumber of jumps per unit time and the distribution of jumpmagnitudes in the jump component of Lt. Jumps of any sizein a Borel set B form a compound Poisson process with rate∫Bν(dy) and jump density ν(dy)/

∫Bν(dy) if the closure B

does not contain 0. µ gives the velocity of the drift component.K is the covariance matrix of the Gaussian component. IfK = 0 and ν = 0 then (9) becomes E(ei〈u,Lt〉) = eit〈µ,u〉.Then Lt = µt is a simple m-dimensional deterministic motion(drift) with velocity vector µ. If K 6= 0 and ν = 0 then Ltis a m-dimensional Brownian motion with drift because (9)takes the form E(ei〈u,Lt〉) = et[i〈µ,u〉−

12 〈u,Ku〉] and because

this exponential is the characteristic function of a Gaussianrandom vector with mean vector µt and covariance matrixtK. If K 6= 0 and ν(Rm) < ∞ then Lt is a jump-diffusionprocess while K = 0 and ν(Rm) < ∞ give a compoundPoisson process. If K = 0 and ν(Rm) = ∞ then Ltis a purely discontinuous jump process and has an infinitenumber of small jumps in any time interval of positive length.

We consider only the Levy processes whose components Lkthave finite second moments: E[(Lkt )]2 < ∞. This excludesthe important family of infinite-variance α-stable processes(including the α = 0.5 Levy stable case) where α ∈ (0,2]measures the tail thickness and where symmetric α-stabledistributions have characteristic functions φ(u) = eiuµ−γ|u|

α

[30], [50], [63], [76]. But a finite-moment assumption doesnot itself imply that the Levy measure is finite: (ν(R) <∞).Normal inverse Gaussian NIG(α, β, δ, µ) distributions areexamples of semi-thick-tailed pure-jump Levy processes thathave infinite Levy measure and yet have finite moments of allorder [33], [73]. They can model the risks of options hedgingand of credit default in portfolios of risky debt obligations[42], [79]. They have characteristic functions of the formφ(u) = e[iuµ+δ(

√α2−β2−

√α2−(β+iu)2)] where 0 ≤ |β| < α

and δ > 0.

Let Lt(µ,K, ν) = (L1t , ..., L

mt )′ be a Levy process that

takes values in Rm where Ljt (µj , σj , νj) are real-valued

independent Levy processes for j = 1, ...,m. We denote theLevy-Ito decomposition [3] of Ljt for each j = 1, ...,m andt ≥ 0 as

Ljt = µjt+ σjBjt +∫|yj |<1

yjN j(t, dyj)

+∫|yj |≥1

yjN j(t, dyj) (12)

= µjt+ σjBjt +∫ t

0

∫|yj |<1

yjN j(ds, dyj)

+∫ t

0

∫|yj |≥1

yjN j(ds, dyj). (13)

Here µ determines the velocity of the deterministic driftprocess µit while the Bjt are real-valued independentstandard Brownian motions. Then µ = (µ1, ..., µm)′ andK = diag[(σ1)2, ..., (σm)2]. The N j are independentPoisson random measures on R+ × R0 with compensated(mean-subtracted) Poisson processes N j and intensity/Levymeasures νj .

Define the Poisson random measure as

N j(t, B) = # ∆Ls ∈ B for 0 ≤ s ≤ t (14)

for each Borel set B in R0. The Poisson random measuregives the random number of jumps of Lt in the time interval[0, t] with jump size ∆Lt in the set B. N j(t, B) is a Poissonrandom variable with intensity νj(B) if νj(B) <∞ and if wefix t and B. But N j(t, ·)(ω) is a measure if we fix ω ∈ Ω andt ≥ 0. This measure is not a martingale. But the compensatedPoisson random measure

N j(t, B) = N j(t, B)− tνj(B) (15)

is a martingale and gives the compensated Poisson integral(12) (the second term on the right-hand side of (12)) as∫|yj |<1

yjN j(t, dyj) =∫|yj |<1

yjN j(t, dyj)

− t∫|yj |<1

yjνj(dyj) for j = 1, ...,m. (16)

We assume again that each Ljt has a finite second moment(E|Ljt |2 < ∞). But if Ljt is a Levy process with triplet(µj , σj , νj) then Ljt has a finite pth moment for p ∈ R+ ifand only if

∫|yj |>1

|yj |pνj(dyj) <∞ [77]. The drift velocityµj relates to the expected value of a Levy Process Ljt byE(Lj1) = µj +

∫|yj |>1

yjν(dyj) and E(Ljt ) = tE(Lj1). So ifLjt is a standard Brownian motion then νj = 0, E(Ljt ) = 0,and Var(Ljt ) = t(σj)2.

The variance of the Levy process in (12) is

Var(Ljt ) = Var(σjBjt ) + Var(∫|yj |<1

yjN j(t, dyj))

+ Var(∫|yj |≥1

yjN j(t, dyj)) (17)

because the underlying processes are independent. The vari-ance terms on the right-hand side of (17) have the following

5

form [3]:

Var(Ljt ) = t(σj)2 (18)

Var

(∫|yj |≥1

yjN j(t, dyj)

)= t

∫|yj |≥1

|yj |2νj(dyj) (19)

Var

(∫|yj |<1

yjN j(t, dyj)

)≤ E

(∫|yj |<1

yjN j(t, dyj)

)2

= t

∫|yj |<1

|yj |2νj(dyj). (20)

The last equality follows from the Ito isometry identity(Proposition 8.8 in [19]). Then (17) and (18)-(20) imply thatthe Var(Ljt ) → 0 if and only if σj → 0 and νj → 0.

We can rewrite (1)-(2) as a more general Ito stochasticdifferential equation (SDE) [3]

dXt = b(Xt−)dt+ c(Xt−)dLt (21)Yt = g(Xt) (22)

with initial condition X0. Here b(Xt−) = −Xt− + f(Xt−)+St is a Lipschitz continuous drift term, c(Xt−) is a boundedLevy diffusion term, and dLt is a white Levy noise withnoise scale κ.

Our continuous neuron models are again one-dimensionalbut the spiking FHN neuron model is two-dimensional. Soconsider the general d-dimensional SDE in the matrix formwith m-dimensional Levy noise Lt = (L1

t , ..., Lmt )′

dXt = b(Xt−)dt+ c(Xt−)dLt (23)

which is shorthand for the system of SDEs

dXit = bi(Xt−)dt+

m∑j=1

cij(Xt−)dLjt for i = 1, .., d (24)

with initial condition Xi0. Here Xt = (X1

t , ..., Xdt )′, b(Xt) =

(b1(Xt), ..., bd(Xt))′, and c is a d×m matrix with rows ci(Xt)= (ci1(Xt), ..., cim(Xt)). The functions bi: Rd → R are locallyor globally Lipschitz measurable functions. The functions cij :Rd → R are bounded globally Lipschitz measurable functionssuch that |cij |2 ≤ Hi

j ∈ R+. The Ljt terms are independentLevy processes as in (13) with µj = 0 for j = 1, ..., m. Then

dXit = bi(Xt−)dt+

m∑j=1

[cij(Xt−)µj ]dt+m∑j=1

[cij(Xt−)σj ]dBjt

+m∑j=1

∫|yj |<1

[cij(Xt−)yj ]N j(dt, dyj)

+m∑j=1

∫|yj |≥1

[cij(Xt−)yj ]N j(dt, dyj)

= bi(Xt−)dt+m∑j=1

[µij(Xt−)]dt+m∑j=1

σij(Xt−)dBjt

+m∑j=1

∫|yj |<1

F ij (Xt− , yj)N j(dt, dyj)

+m∑j=1

∫|yj |≥1

Gij(Xt− , yj)N j(dt, dyj). (25)

where µij(Xt−) = cij(Xt−)µj = 0, σij(Xt−) = cij(Xt−)σj ,F ij (Xt− , y

j) = cij(Xt−)yj , and Gij(Xt− , yj) = cij(Xt−)yj are

all globally Lipschitz functions. This equation has the integralform with initial condition Xi

0:

Xit = Xi

0 +∫ t

0

bi(Xs−)ds+m∑j=1

∫ t

0

σij(Xs−)dBjs

+m∑j=1

∫ t

0

∫|yj |<1

F ij (Xs− , yj)N j(ds, dyj)

+m∑j=1

∫ t

0

∫|yj |≥1

Gij(Xs− , yj)N j(ds, dyj). (26)

IV. LEVY NOISE BENEFITS IN CONTINUOUS NEURONMODELS

We now prove that Levy noise can benefit the noisy con-tinuous neurons (21)-(22) with signal functions (4)-(8) andsubthreshold input signals. We assume that the neuron receivesa constant subthreshold input signal St ∈ s1, s2 for time T .Let S denote the input signal and let Y denote the outputsignal Yt for a sufficiently large randomly chosen time t ≤ T .

Noise researchers have used various system performancemeasures to detect SR noise benefits [8], [17], [45], [52], [58],[60], [61], [65], [72]. These include the output signal-to-noiseratio, cross-correlation, error probability, and Shannon mutualinformation between input and output signals. We use Shannonmutual information to measure the Levy noise benefits. Mutualinformation measures the information that the neuron’s outputconveys about the input signal. It is a common detectionperformance measure when the input signal is random [8],[39], [61], [84].

Define the Shannon mutual information I(S, Y ) of thediscrete input random variable S and the output randomvariable Y as the difference between the output’s unconditionaland conditional entropy [20]:

I(S, Y ) = H(Y )−H(Y |S) (27)

= −∑y

PY (y) logPY (y)

+∑s

∑y

PSY (s, y) logPY |S(y|s) (28)

= −∑y

PY (y) logPY (y)

+∑s

PS(s)∑y

PY |S(y|s) logPY |S(y|s) (29)

=∑s,y

PSY (s, y) logPSY (s, y)PS(s)PY (y)

. (30)

So the mutual information is the expectation of the randomvariable log PSY (s,y)

PS(s)PY (y) :

I(S, Y ) = E[

logPSY (s, y)PS(s)PY (y)

]. (31)

Here PS(s) is the probability density of the input S, PY (y)is the probability density of the output Y , PY |S(y|s) is the

6

conditional density of the output Y given the input S, andPSY (s, y) is the joint density of the input S and the outputY . An SR noise benefit occurs in a system if and only if anincrease in the input noise variance or dispersion increasesthe system’s mutual information (31).

We need the following lemma to prove that noise improvesthe continuous neuron’s mutual information or bit count. TheAppendix gives the proof of Lemma 1.

Lemma 1: Let bi : Rd → R and cij : Rd → R in (23)-(24) be measurable functions that satisfy the global Lipschitzconditions

|bi(x1)− bi(x2)| ≤ K1‖x1 − x2‖ (32)|cij(x1)− cij(x2)| ≤ K2‖x1 − x2‖ (33)

and |cij(x1)|2 ≤ Hij (34)

for all x1, x2 ∈ Rd and for i = 1, ..., d and j = 1, ..., m.Suppose

dXt = b(Xt−)dt+ c(Xt−)dLt (35)dXt = b(Xt)dt (36)

where dLt is a Levy noise with µ = 0 and finite secondmoments. Then for every T ∈ R+ and for every ε > 0:

E[ sup0≤t≤T

‖Xt − Xt‖2]→ 0 as σj → 0 and νj → 0 (37)

for all j = 1, ..., m, and hence

P [ sup0≤t≤T

‖Xt − Xt‖2 > ε]→ 0 as σj → 0 and νj → 0 (38)

for all j = 1, ..., m since mean-square convergence impliesconvergence in probability.

We prove the Levy SR theorem with the stochastic calculusand a special limiting argument. This avoids trying to solvefor the process Xt in (21). The proof strategy follows thatof the ‘forbidden interval’ theorems [50], [51], [65]: whatgoes down must go up. Jensen’s inequality implies thatI(S, Y ) ≥ 0 [20]. Random variables S and Y are statisticallyindependent if and only if I(S, Y ) = 0. Hence I(S, Y ) > 0implies some degree of statistical dependence. So the systemmust exhibit the SR noise benefit if I(S, Y ) > 0 and ifI(S, Y ) → 0 when noise parameters σ → 0 and ν → 0.Theorem 1 uses Lemma 1 to show that I(S, Y ) → 0 whennoise parameters σ → 0 and ν → 0. So some increase in thenoise parameters must increase the mutual information.

Theorem 1: Suppose that the continuous neuron models(21)-(22) and (4)-(8) have a bounded globally Lipschitz Levydiffusion term c(Xt−) ≤ H and that the additive Levy noisehas drift velocity µ. Suppose also that the input signal S(t)∈ s1, s2 is subthreshold: θ1 ≤ s1 < s2 ≤ θ2 and that thereis some statistical dependence between the input randomvariable S and the output spike-rate random variable R sothat I(S,R) > 0. Then the neuron models (21)-(22) withsignal functions including (4)-(8) exhibit the nonmonotoneSR effect in the sense that I(S, Y ) → 0 as the Levy noiseparameters σ → 0 and ν → 0 if θ1 − s1 ≤ Hµ ≤ θ2 − s2.

Proof: Let σk, νk∞k=1 be any decreasing sequence ofLevy noise parameters such that σk → 0 and νk → 0 ask →∞. Define X(t)k and Y (t)k as solution processes of thecontinuous neuron models with Levy noise parameters σk andνk instead of σ and ν.

Suppose that µ 6= 0. We can absorb the drift c(Xt−)µ intothe input signal S because the Levy noise Lt is additive in theneuron models. Then the new input signal S′ = S+ c(Xt−)µand it does not affect the Lipschitz continuity of b(Xt−) inequation (21). Note that S′ is subthreshold (θ1 ≤ S′ ≤ θ2)if θ1 − s1 ≤ Hµ ≤ θ2 − s2. So we lose no generality ifwe consider the noise dLt with µ = 0 and let S ∈ s1, s2be subthreshold in the continuous neuron models (21). Thisallows us to use Lemma 1.

Let the symbol ‘0’ denote the input signal S = s1 and theoutput signal Y = 0. Let the symbol ‘1’ denote the inputsignal S = s2 and the output signal Y = 1. Assume that0 < PS(s) < 1 to avoid triviality when PS(s) = 0 or 1.We show that S and Y are asymptotically independent byusing the fact that I(S, Y ) = 0 if and only if S and Y arestatistically independent [20]. So we need to show only thatPSY (s, y) = PS(s)PY (y) or PY |S(y|s) = PY (y) as σk → 0and νk → 0 as k →∞ for signal symbols s ∈ S and y ∈ Y .The theorem of total probability and the two-symbol alphabetset S give

PY (y) =∑s

PY |S(y|s)PS(s)

= PY |S(y|0)PS(0) + PY |S(y|1)PS(1)= PY |S(y|0)PS(0) + PY |S(y|1)(1− PS(0))= (PY |S(y|0)− PY |S(y|1))PS(0) + PY |S(y|1)

So we need to show only that PYk|S(y|0) − PYk|S(y|1) = 0as σk → 0 and νk → 0 for y ∈ 0, 1 . We prove the casefor y = 0 only: limk→∞PYk|S(0|0)−PYk|S(0|1) = 0 sincethe proof for y = 1 is similar. Then the desired limit goes tozero because

limk→∞

PYk|S(0|0) − PYk|S(0|1)

= limk→∞

PYk|S(0|0)− limk→∞

PYk|S(0|1)

= limk→∞

P [Yk = 0|S = 0]− limk→∞

P [Yk = 0|S = 1]

= limk→∞

P [X(t)k < 0|S = 0]

− limk→∞

P [X(t)k < 0|S = 1] for large t

= limk→∞

P [X(t)k < 0, Xt < 0|S = 0]

+ P [X(t)k < 0, Xt > 0|S = 0]

− limk→∞

P [X(t)k > 0, Xt < 0|S = 1]

+ P [X(t)k > 0, Xt > 0|S = 1]

for large t

= limk→∞

P [X(t)k < 0|Xt < 0, S = 0]P [Xt < 0|S = 0]

+ P [X(t)k < 0|Xt > 0, S = 0]P [Xt > 0|S = 0]

7

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

Scale κ of additive white uniform jump−diffusion Levy noise

Mutu

al in

form

ation I(S

,R)

in b

its

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9


Mutu

al in

form

ation I(S

,R)

in b

its

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

Scale κ of additive white α−stable Levy noise

Mutu

al in

form

ation I(S

,R)

in b

its

(a) (b) (c)

Fig. 4. Mutual information SR Levy noise benefits in the logistic continuous neuron (1)-(2) and (4). Additive white Levy noise dLt increasesthe mutual information of the logistic neuron for the subthreshold input signal s1 = −0.6, s2 = −0.4, and c = 8. The types of Levy noisedLt are (a) Gaussian with uniformly distributed jumps, (b) pure-jump normal inverse Gaussian (NIG), and (c) symmetric α-stable noisewith α = 1.95 (thick-tailed bell curve with infinite variance [63]). The dashed vertical lines show the total min-max deviations of the mutualinformation in 100 simulation trials.

− limk→∞

P [X(t)k > 0|Xt < 0, S = 1]P [Xt < 0|S = 1]

+ P [X(t)k > 0|Xt > 0, S = 1]P [Xt > 0|S = 1]

for large t

= 1 · 12

+ 0 · 12 − 0 · 1

2+ 1 · 1

2

by Lemma 1 and the assumption thatP [Xt < 0|S = si] = P [Xt > 0|S = si] = 1/2

for i = 1, 2.= 0 Q.E.D.

Sub-figures (a) and (b) of Figures 4-6 show simulationinstances of Theorem 1 for finite-variance jump-diffusionand pure-jump additive white Levy noise in logistic, linear-threshold, and Gaussian neuron models. Small amounts ofadditive Levy noise in continuous neuron models producethe SR effect by increasing the Shannon mutual informationI(S, Y ) between realizations of a random (Bernoulli)subthreshold input signal S and the neuron’s thresholdedoutput random variable Y . The SR effect in Figures 2 (c)and 4-6 (c) lie outside the scope of the theorem because itoccurs for infinite-variance α-stable noise. Thus the SR effectin continuous neurons is not limited to finite-second-momentLevy noise.

V. LEVY NOISE BENEFITS IN SPIKING NEURON MODELS

We next demonstrate Levy SR noise benefits in threepopular spiking neuron models: the leaky integrate-and-firemodel [17], [28], the reduced Type I neuron model [54], andthe FitzHugh-Nagumo (FHN) model [26], [16]. This requiresthe use of Lemma 2 as we discuss below. These neuron modelshave a one- or two-dimensional form of (1). A spike occurs

when the membrane potential x(t) crosses a threshold valuefrom below. We measure the mutual information I(S,R) be-tween the input signal s(t) and the output spike-rate responseR of theses spiking neuron models. We define the averageoutput spike-rate response R in the time interval (t1, t2] as

R =N(t1, t2]t2 − t1

(39)

where N(t1, t2] is the number of spikes in the time interval(t1, t2].

The Leaky Integrate-and-Fire Neuron ModelThe leaky integrate-and-fire neuron model has the form [17]

v = −av + a− δ + S + n (40)

where v is the membrane voltage, a and δ are constants, δ/ais the barrier height of the potential, S is an input signal,and n is independent Gaussian white noise in the neuralliterature but here is Levy white noise. The input signal S issubthreshold when S < δ. The neuron emits a spike whenthe membrane voltage v crosses the threshold value of 1 frombelow to above. The membrane voltage v resets to 1 − δ/ajust after the neuron emits a spike.

The Reduced Type I Neuron ModelThe reduction procedure in [31], [36] gives a simple one-dimensional normal form [54] of the multi-dimensional dy-namics of Type I neuron models:

v = β + v2 + σn (41)

where v is the membrane potential, β is the value of inputsignal, and σ is the standard deviation of Gaussian white noisen in the neural literature but here is Levy white noise. Thisreduced model (41) operates in a subthreshold or excitableregime when the input β < 0.

8

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 20

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8


Mutu

al in

form

ation I(S

,R)

in b

its

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 20

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8


Mutu

al in

form

ation I(S

,R)

in b

its

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 20

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8


Mutu

al in

form

ation I(S

,R)

in b

its

(a) (b) (c)

Fig. 5. Mutual information Levy noise benefits in the linear-threshold continuous neuron (1)-(2) and (6). Additive white Levy noise dLt

increases the mutual information of the linear-threshold neuron for the subthreshold input signal s1 = −0.4, s2 = 0.4, and c = 2. The typesof Levy noise dLt are (a) Gaussian with uniformly distributed jumps, (b) pure-jump normal inverse Gaussian (NIG), and (c) symmetricα-stable noise with α = 1.95 (thick-tailed bell curve with infinite variance [63]). The dashed vertical lines show the total min-max deviationsof the mutual information in 100 simulation trials.

0 0.5 1 1.5 2 2.5 3 3.5 40

0.1

0.2

0.3

0.4

0.5

0.6


Mutu

al in

form

ation I(S

,R)

in b

its

0 0.5 1 1.5 2 2.5 3 3.5 40

0.1

0.2

0.3

0.4

0.5

0.6


Mutu

al in

form

ation I(S

,R)

in b

its

0 0.5 1 1.5 2 2.5 3 3.5 40

0.1

0.2

0.3

0.4

0.5

0.60.6


Mutu

al in

form

ation I(S

,R)

in b

its

(a) (b) (c)

Fig. 6. Mutual information Levy noise benefits in the Gaussian or ‘radial basis’ continuous neuron (1)-(2) and (8). Additive white Levynoise dLt increases the mutual information of the Gaussian neuron for the subthreshold input signal s1 = −0.4, s2 = 0.4, and c = 8. Thetypes of Levy noise dLt are (a) Gaussian with uniformly distributed jumps, (b) pure-jump normal inverse Gaussian (NIG), and (c) symmetricα-stable noise with α = 1.95 (thick-tailed bell curve with infinite variance [63]). The dashed vertical lines show the total min-max deviationsof the mutual information in 100 simulation trials.

The FitzHugh-Nagumo (FHN) Neuron ModelThe FHN neuron model [16], [26], [28] is a two-dimensionalsimplification of the Hodgkin and Huxley neuron model [34].It describes the response of a so-called Type II excitablesystem [28], [55] that undergoes a Hopf bifurcation. Thesystem first resides in the stable rest state for subthresholdinputs as do multistable systems. Then the system leaves thestable state in response to a strong input but returns to it afterpassing through firing and refractory states in a manner thatdiffers from the behavior of multistable systems. The FHNneuron model is a limit-cycle oscillator of the form

εv = −v(v2 − 14

)− w +A+ s(t) + n(t) (42)

w = v − w (43)

where v(t) is a fast (voltage) variable, w(t) is slow (re-covery) variable, A is a constant (tonic) activation signal,

and ε = 0.005. n(t) is a white Levy noise and s(t) is asubthreshold input signal—either s1 or s2. We measure theneuron’s response to the input signal s(t) in terms of thetransition (firing) rate r(t).

We can rewrite (42)-(43) as

εv = −v(v2 − 14

)− w +AT − (B − s(t)) + n(t) (44)

w = v − w (45)

where B is a positive constant parameter that corresponds tothe distance that the input signal s(t) must overcome to crossthe threshold. Then B−s(t) is the signal-to-threshold distanceand so s(t) is subthreshold when B−s(t) > 0. Our simulationsused B = 0.007 and hence A = −(5/(12

√3 + 0.007)).

The deterministic FHN model (n(t) ≡ 0 in (44)) performsrelaxation oscillations and has an action potential v(t) thatlies between -0.6 and 0.6. The system emits a spike when

9

v(t) crosses the threshold value θ = 0. We use a lowpass-filtered version of v(t) to avoid false spike detections due tothe additive noise. The lowpass filter is a 100-point moving-average smoother with a 0.001 second time-step.

We rewrite equations (42)-(43) as

x1 = −x1

ε((x1)2 − 1

4)− x2

ε+A

ε+s(t)ε

+n(t)ε

(46)

x2 = x1 − x2. (47)

Here x1 = v and x2 = w. The corresponding matrix Itostochastic differential equation is

dXt = b(Xt−)dt+ c(Xt−)dLt (48)

where Xt = (X1t , X

2t )T , Lt = (L1

t , L2t )T ,

b(Xt−) =[b1(X1

t− , X2t−)

b2(X1t− , X

2t−)

]=

[−X1

t−ε ((X1

t−)2 − 14 )− X2

t−ε + A

ε + stε

X1t− −X

2t−

],

and c(Xt−) =[

σε0

].

Thus all of the above spiking neuron models have the SDEform (23). Note that the drift term of the leaky integrate-and-fire neuron model is globally Lipschitz while the drift termof the reduced Type I neuron model is locally Lipschitz. TheLipschitz condition is not easy to verify in the FHN model.

We now show that the drift term b1(Xt) in the preced-ing equation does not satisfy the global Lipschitz condition.Note that b1(Xt) is differentiable on R2 because the partialderivatives of b1(X1

t− , X2t−) exist and are continuous on R2.

Suppose that b1(Xt) satisfies the following global Lipschitzcondition: There exists a constant K > 0 such that

|b1(Zt)− b1(Yt)| ≤ K||Zt − Yt||

for all Zt and Yt ∈ R2 and t ∈ [0, T ]. Then the mean-valuetheorem gives

b1(Zt)− b1(Yt) = [∂b1(ζ)∂X1

t

∂b1(ζ)∂X2

t

] · [Zt − Yt] (49)

=∂b1(ζ)∂X1

t

(Z1t − Y 1

t ) +∂b1(ζ)∂X2

t

(Z2t − Y 2

t )

for some ζ between Zt and Yt in R2. Then

|b1(Zt)− b1(Yt)| ≥ |∂b1(ζ)∂X1

t

||Z1t -Y 1

t | − |∂b1(ζ)∂X2

t

||Z2t -Y 2

t |

= |∂b1(ζ)∂X1

t

||Z1t − Y 1

t | −1ε|Z2t − Y 2

t |

because∂b1

∂X2t

= −1ε

= |∂b1(ζ)∂X1

t

||Z1t − Y 1

t | choosing

Zt and Yt such that Z2t = Y 2

t

= |∂b1(ζ)∂X1

t

|||Zt − Yt||

> K||Zt − Yt||

for some Zt ∈ R2 and Yt ∈ R2 because | ∂b1

∂X1t| =

|−3(X1t )

2

ε + 14ε | is unbounded and continuous on R2 and so

there is a domain D ⊂ R2 such that |∂b1(ζ)∂X1

t| > K for all ζ ∈

D. Thus b1(Xt) is not globally Lipschitz. So we cannot useLemma 1 to prove the sufficient condition for the SR effectin the FHN neuron model (44)-(45).

But b1(Xt) is locally Lipschitz. The partial derivatives ofb1(X1

t− , X2t−) exist and are continuous on R2. So ∂b1

∂X1t

and∂b1

∂X2t

achieve their respective maxima on the compact set ζ ∈R2 : ||ζ|| ≤ n. Then (49) gives the required local Lipschitzcondition:

|b1(Zt)− b1(Yt)| ≤ maxsupζ|∂b

1(ζ)∂X1

t

|,

supζ|∂b

1(ζ)∂X2

t

|||Zt − Yt||

= K ′n||Zt − Yt||

for all Zt and Yt ∈ R2 such that ||Zt|| ≤ n, ||Yt|| ≤ n, and||ζ|| ≤ n. Lemma 2 extends the conclusion of Lemma 1 tothe locally Lipschitz drift terms bi(Xt).

Theorem 2 below gives a ‘forbidden-interval’ sufficientcondition for a Levy SR noise benefits in spiking neuronmodels such as the leaky integrate-and-fire model [17], [28],the reduced Type I neuron model [54], and the FitzHugh-Nagumo (FHN) model [26], [16]. It shows that these neuronmodels enjoy SR noise benefits if the noise mean µ fallsto the left of a bound. Theorem 2 requires Lemma 2 toextend the conclusion of Lemma 1 to the locally Lipschitzdrift terms bi(Xt). The Appendix gives the proof of Lemma 2.

Lemma 2: Let bi : Rd → R and cij : Rd → R in (23)-(24)((40)-(43) for spiking neuron models) be measurable functionsthat satisfy the respective local and global Lipschitz conditions

|bi(z)− bi(y)| ≤ Cn‖z − y‖ (50)

when ‖z‖ ≤ n and ‖y‖ ≤ n,

|cij(z)− cij(y)| ≤ K1‖z − y‖ (51)

|cij(z)|2 ≤ Hij (52)

for all z and y ∈ Rd, and for i = 1, ..., d and j = 1, ..., m.Suppose



E[ sup0≤t≤T

‖Xt − Xt‖2]→ 0 as σj → 0 and νj → 0 (55)


P [ sup0≤t≤T

‖Xt − Xt‖2 > ε]→ 0 as σj → 0 and νj → 0 (56)


10

0 0.005 0.01 0.015 0.02 0.0250

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Scale κ of additive white Gaussian Levy noise

Mutu

al in

form

ation I(S

,R)

in b

its

0 0.005 0.01 0.015 0.02 0.0250

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1


Mutu

al in

form

ation I(S

,R)

in b

its

0 0.005 0.01 0.015 0.02 0.0250

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1


Mutu

al in

form

ation I(S

,R)

in b

its

(a) (b) (c)

Fig. 7. Mutual information Levy noise benefits in the leaky integrate-and-fire spiking neuron model (40). Additive white Levy noise dLt

increases the mutual information of the IF neuron with parameters a = 0.5 and δ = 0.02 for the subthreshold input signal s1 = 0.005 ands2 = 0.012. The types of Levy noise dLt are (a) Gaussian, (b) Gaussian with uniformly distributed jumps, and (c) symmetric α-stable noisewith α = 1.95 (thick-tailed bell curve with infinite variance [63]). The dashed vertical lines show the total min-max deviations of the mutualinformation in 100 simulation trials.

We can now state and prove Theorem 2.

Theorem 2: Suppose that the spiking neuron models(40)-(41) and (42)-(43) have the form of the Levy SDE(23) with a bounded globally Lipschitz Levy diffusion termc(Xt−) ≤ H and that the additive Levy noise has driftvelocity µ. Suppose that the input signal S(t) ∈ s1, s2 issubthreshold: S(t) < B. Suppose there is some statisticaldependence between the input random variable S and theoutput spike-rate random variable R so that I(S,R) > 0.Then the spiking neuron models (40)-(41) and (42)-(43)exhibit the SR effect in the sense that I(S,R) → 0 as theLevy noise parameters σ → 0 and ν → 0 if Hµ < B − s2.

Proof: Let σk, νk∞k=1 be any decreasing sequence of Levynoise parameters such that σk → 0 and νk → 0 as k → ∞.Define X(t)k and Rk as the respective solution process andspiking rate process of the FHN spiking neuron model (48)with Levy noise parameters σk and νk instead of σ and ν.

Suppose that µ 6= 0. We can absorb the drift c(Xt−)µ intothe input signal S because the Levy noise Lt is additive inall the neuron models. Then the new input signal S′ = S +c(Xt−)µ and this does not affect the Lipschitz continuity ofb(Xt−) in (21). S′ is subthreshold (S′ < B) because c(Xt−)µ< Hµ < B – s2 where s2 = maxs1, s2. So we lose nogenerality if we consider the noise dLt with µ = 0 and let S∈ s1, s2 be subthreshold in the continuous neuron models(21). This allows us to use Lemma 2.

Recall that I(S,R) = 0 if and only if S and R arestatistically independent [20]. So we need to show only thatfSR(s, r) = PS(s)fR(r) or fR|S(r|s) = fR(r) as σ → 0 andν → 0 for signal symbols s ∈ s1, s2 and for all r ≥ 0.Here fSR is the joint probability density function and fS|R isthe conditional density function. This is logically equivalent toFR|S = FR as σk → 0 and νk → 0 as k → 0 where FR|S isthe conditional distribution function [25]. Again the theoremof total probability and the two-symbol alphabet set s1, s2

give

FR(r) =∑s

FR|S(r|s)PS(s) (57)

= FR|S(r|s1)PS(s1) + FR|S(r|s2)PS(s2)= FR|S(r|s1)PS(s1) + FR|S(r|s2)(1− PS(s1))= (FR|S(r|s1)-FR|S(r|s2))PS(s1) + FR|S(r|s2).(58)

So we need to show that limk→∞ FRk|S(r|s1) − FRk|S(r|s2)= 0 for all r ≥ 0. This holds if and only if

limk→∞

P [Rk > r|S = s1]− P [Rk > r|S = s2] = 0 (59)

We prove that limk→∞ P [Rk > r|S = si] = 0 for i = 1 andi = 2. Note that if r > 0 for (48) then X1(t)k must cross thefiring or spike threshold θ. Then

P [Rk > r|S = si] ≤ P [ supt1≤t≤t2

X1(t)k > θ|S = si].

Then Lemma 2 shows that the required limit goes to zero:

limk→∞

P [Rk > r|S = si]

≤ limk→∞

P [ supt1≤t≤t2

X1(t)k > θ|S = si]

= limn→∞

P [ supt1≤t≤t2

X1(t)k > θ, X1(t) < θ|S = si]

because X1(t) converges to the FHN fixed-pointZFi < θ for large t

= 0 by Lemma 2. Q.E.D.

Sub-figures (a) and (b) of Figures 7-8 show simulationinstances of Theorem 2 for finite-variance diffusion and jump-diffusion white Levy noise in the leaky integrate-and-fire andthe FHN neuron models. Small amounts of additive Levynoise in these spiking neuron models produce the SR effectin terms of the noise-enhanced Shannon mutual informationI(S, Y ) between realizations of a random (Bernoulli)subthreshold input signal S and the neuron’s thresholded

11

0.5 1 1.5 2 2.5 3 3.5 4 4.5 5

x 10−3

0

0.1

0.2

0.3

0.4

0.5

0.6

Scale κ of additive white Gaussian Levy noise

Mutu

al in

form

ation I(S

,R)

in b

its

0.5 1 1.5 2 2.5 3 3.5 4 4.5 5

x 10−3

0

0.1

0.2

0.3

0.4

0.5

0.6


Mutu

al in

form

ation I(S

,R)

in b

its

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5

x 10−3

0

0.1

0.2

0.3

0.4

0.5

0.6


Mutu

al in

form

ation I(S

,R)

in b

its

(a) (b) (c)Fig. 8. Mutual information Levy noise benefits in the FHN spiking neuron (42)-(43). Additive white Levy noise dLt increases the mutualinformation of the FHN neuron for the subthreshold input signal s1 = −0.0045 and s2 = 0.0045. The types of Levy noise dLt are (a)Gaussian, (b) Gaussian with uniformly distributed jumps, and (c) symmetric α-stable noise with α = 1.9 (thick-tailed bell curve with infinitevariance [63]). The dashed vertical lines show the total min-max deviations of the mutual information in 100 simulation trials.

output random variable Y . The SR effects in Figures 7-8(c)again lie outside the scope of Theorem 2 because they occurfor infinite-variance α-stable noise and because Theorem 2requires noise with finite second moments. Thus the SR effectin spiking neurons is not limited to finite second momentLevy noise.

VI. CONCLUSION

Levy noise processes can benefit several continuous andspiking neuron models because general forms of the SR‘forbidden interval’ theorem hold for several types of Levynoise. The generality of Levy noise extends simple Brownianmodels of noise to more complex and realistic Poisson jumpmodels of noise that can affect biological and model neurons.But both Levy SR theorems require the finite second-momentrestrictions of the two lemmas. This rules out the importantclass of stable noise distributions in all but the Gaussian orpure-diffusion case.

Relaxing the second-moment assumption may producestochastic differential equations that are not mathematicallytractable. Yet the simulation evidence of Figure 1 andsub-figure (c) of Figures 3-8 shows that the SR noise benefitcontinues to hold for several stable models where the noisehas infinite variance and infinite higher-order moments. It isan open research question whether a more general Levy SRresult can include these and other observed noise benefits incontinuous and spiking neuron models.

APPENDIX

Proofs of Lemmas

The proof of Lemma 2 relies on the proof technique ofLemma 1 in which we bound a mean-squared term by fouradditive terms and then show that each of the four terms goesto zero in the limit.

Lemma 1: Let bi : Rd → R and cij : Rd → R in (23)-(24) be measurable functions that satisfy the global Lipschitzconditions

|bi(x1)− bi(x2)| ≤ K1‖x1 − x2‖ (60)|cij(x1)− cij(x2)| ≤ K2‖x1 − x2‖ (61)

and |cij(x1)|2 ≤ Hij (62)

for all x1, x2 ∈ Rd and for i = 1, ..., d and j = 1, ..., m.Suppose



E[ sup0≤t≤T

‖Xt − Xt‖2]→ 0 as σj → 0 and νj → 0 (65)


P [ sup0≤t≤T

‖Xt − Xt‖2 > ε]→ 0 as σj → 0 and νj → 0 (66)


Proof:The Lipschitz conditions (60) and (61) ensure that the processXt exists [3] for t ≥ 0 in (63). Then the proof commenceswith the inequality

sup0≤t≤T

‖Xt − Xt‖2 ≤d∑i=1

sup0≤t≤T

(Xit − Xi

t)2 (67)

which implies that

E[ sup0≤t≤T

‖Xt − Xt‖2] ≤d∑i=1

E[ sup0≤t≤T

(Xit − Xi

t)2]. (68)

12

Equations (26) and (63)-(64) imply

Xit − Xi

t =∫ t

0

[bi(Xs−)-bi(Xs−)]ds+m∑j=1

∫ t

0

σij(Xs−)dBjs

+m∑j=1

∫ t

0

∫|yj |<1


+m∑j=1

∫ t

0

∫|yj |≥1

Gij(Xs− , yj)N j(ds, dyj). (69)

This gives an upper bound on the squared difference as

(Xit − Xi

t)2 ≤ (3m+ 1)

(∫ t

0

[bi(Xs−)− bi(Xs−)]ds2

+m∑j=1

∫ t

0

σij(Xs−)dBjs

2

+m∑j=1

∫ t

0

∫|yj |<1


2

+m∑j=1

∫ t

0

∫|yj |≥1

Gij(Xs− , yj)N j(ds, dyj)

2 (70)

because (u1 + ... + un)2 ≤ n(u21 + ... + u2

n). The Cauchy-Schwartz inequality gives

∫ t

0

[bi(Xs−)− bi(Xs−)]ds2

≤(∫ t

0

ds

)(∫ t

0

[bi(Xs−)− bi(Xs−)]2ds). (71)

Now put (71) in the first term of (70) and then take expecta-tions of the supremum on both sides to get four additive termsas an upper bound:

E

[sup

0≤t≤T(Xi

t − Xit)

2

]≤ (3m+ 1)

(E

[sup

0≤t≤Tt

∫ t

0

[bi(Xs−)− bi(Xs−)]2ds]

+m∑j=1

E

[sup

0≤t≤T

∫ t

0

σij(Xs−)dBjs

2]

+m∑j=1

E

sup0≤t≤T

∫ t

0

∫|yj |<1


2

+m∑j=1

E

sup0≤t≤T

∫ t

0

∫|yj |≥1


2.(72)

We next show that each of the four terms goes to zero.

Consider the first term on the right-hand side of (72):

E

[sup

0≤t≤Tt

∫ t

0


≤ TE

[sup

0≤t≤T·∫ t

0


≤ TK21E

[sup

0≤t≤T

∫ t

0

‖Xs− − Xs−‖2ds]

by the Lipschitz condition (60)

≤ TK21

∫ T

0

E

[sup

0≤u≤s‖Xu− − Xu−‖2

]ds. (73)

The second term

E

[sup

0≤t≤T

∫ t

0

σij(Xs−)dBjs

2]

≤ 4E

∫ T

0

σij(Xs−)dBjs

2

because∫ t0σij(Xs−)dBjs is a martingale and so we can ap-

ply Doob’s Lp inequality [57]: E[supa≤t≤b |Ut|p

]≤(

pp−1

)pE|Ub|p if Utt≥0 is a real-valued martingale, [a, b]

is a bounded interval of R+, Ut ∈ Lp(Ω,R), and if p > 1(p = 2 in our case). But

4E

∫ T

0

σij(Xs−)dBjs

2 = 4

∫ T

0

E[σij(Xs−)2

]ds

by Ito isometry [3]: E

(∫ T0f(s, w)dBs

2)

=∫ T0E(|f(s, w)|2)ds if f ∈ H2([0, T ]) where H2([0, T ])

is the space of all real-valued measurable Ft-adaptedprocesses such that E

(∫ T0|f(s, w)|2ds

)<∞. Then

4∫ T

0

E[σij(Xs−)2

]ds ≤ 4(σj)2

∫ T

0

E[cij(Xs−)2

]ds

by definition of σij(Xs−)

≤ 4(σj)2THij because |cij |2 ≤ Hi

j . (74)

Note that

E

sup0≤t≤T

∫ t

0

∫|yj |<1


2

≤ 4E

∫ T

0

∫|yj |<1


2

by Doob’s Lp inequality

= 4E

∫ T

0

∫|yj |<1

cij(Xs−)yjN j(ds, dyj)

2

by definition of F ij (Xs− , yj)

≤ 4HijE

∫ T

0

∫|yj |<1

yjN j(ds, dyj)

2

because |cij |2 ≤ Hij

13

≤ 4HijE

∫|yj |<1

yjN j(T, dyj)

2

= 4HijT

∫|yj |<1

|yj |2νj(dyj) (75)

by Ito isometry and (20). Similar arguments and (19) give

E

sup0≤t≤T

∫ t

0

∫|yj |≥1


2

≤ 4HijT

∫|yj |≥1

|yj |2νj(dyj). (76)

Substituting the above estimates (73), (74), (75), and (76) ininequality (72) gives

E

[sup

0≤t≤T(Xi

t − Xit)

2

]≤ (3m+ 1)

(TK2

1 ·∫ T

0

E

[sup

0≤u≤s‖Xu− − Xu−‖2

]ds

+m∑j=1

4THij

[(σj)2 +

∫R

|yj |2νj(dyj)] . (77)

Inequalities (68) and (77) imply that we can write

z(T ) ≤ A+Q

∫ T

0

z(s)d(s) (78)

where z(T ) = E

[sup

0≤t≤T‖Xt − Xt‖2

],

A =d∑i=1

m∑j=1

(3m+1)4THij

[(σj)2 +

∫R

|yj |2νj(dyj)],

and Q = (3m + 1)dTK21 . Then we get z(T ) ≤ AeQT by

Gronwall’s inequality [24]: φ(t) ≤ αeβt for all t ∈ [0, T ]and for real continuous φ(t) in [0, T ] such that φ(t) ≤α + β

∫ t0φ(τ)dτ where t ∈ [0, T ] and β > 0. Note that

A→ 0 as σj → 0 and νj → 0. Hence

E[ sup0≤t≤T

‖Xt− Xt‖2] → 0 as σj → 0 and νj → 0 (79)

for each j = 1, ...,m. This implies the claim (66). Q.E.D.

Lemma 2: Let bi : Rd → R and cij : Rd → R in (23)-(24)((40)-(43) for spiking neuron models) be measurable functionsthat satisfy the respective local and global Lipschitz conditions

|bi(z)− bi(y)| ≤ Cn‖z − y‖ (80)

when ‖z‖ ≤ n and ‖y‖ ≤ n,

|cij(z)− cij(y)| ≤ K1‖z − y‖ (81)

|cij(z)|2 ≤ Hij (82)

for all z and y ∈ Rd, and for i = 1, ..., d and j = 1, ..., m.Suppose



E[ sup0≤t≤T

‖Xt − Xt‖2]→ 0 as σj → 0 and νj → 0 (85)


P [ sup0≤t≤T

‖Xt − Xt‖2 > ε]→ 0 as σj → 0 and νj → 0 (86)


Proof:First define the function bir such that

(i) bir(x) = bi(x) for ||x|| ≤ r(ii) bir(x) = 0 for ||x|| ≥ 2r

(iii) bir(x) = ((2r − ||x||)/r)bi(rx/||x||) for r ≤ ||x|| ≤ 2r.

We then show that the function bir is globally Lipschitz:

|bir(x)− bir(y)| ≤ C ′r||x− y|| for all x, y ∈ Rd.

Consider the function bir(x). Write

bir(x) =bi(x) if ||x|| ≤ rf(x)gi(x) if r ≤ ||x|| ≤ 2r (87)

where

f(x) = ((2r − ||x||)/r) and (88)gi(x) = bi(rx/‖x‖).

The definition of br implies that it is Lipschitz continuous onthe region D1 = ‖x‖ ≤ r:

‖bir(x)− bir(y)‖ ≤ Cr‖x− y‖ for all x, y ∈ D1. (89)

We first show that br(x) is Lipschitz continuous on the re-gion D2 = r ≤ ‖x‖ ≤ 2r. For x, y ∈ D2 = r ≤ ‖x‖ ≤ 2r:

|f(x)− f(y)| =|‖y‖ − ‖x‖|

rby definition of f (90)

≤ ‖x− y‖r

and

|gi(x)− gi(y)| = |bi(rx/‖x‖)− bi(ry/‖y‖)| (91)by definition of gi

≤ Cr‖rx

‖x‖− ry

‖y‖‖ (92)

because rs‖s‖ ∈ D1 for all s ∈ Rd

and bi is Lipschitz continuous on D1

≤ Cr2‖x− y‖ (93)

because r ≥ ‖x‖, ‖y‖ ≥ 2r.

Hence

|bir(x)− bir(y)| = |f(x)gi(x)− f(y)gi(y)| (94)≤ |f(x)gi(x)-f(z)gi(z)|+ |f(z)gi(z)-f(y)gi(y)| (95)= |f(x)gi(x)-f(x)gi(y)|+ |f(x)gi(y)-f(y)gi(y)| (96)

by choosing z on the line segment between 0 and ysuch that ‖z‖ = ‖x‖

14

= |f(x)||gi(x)− gi(y)|+ |gi(y)||f(x)− f(y)| (97)≤ ‖f‖∞,2|gi(x)− gi(y)|+ ‖g‖∞,2|f(x)− f(y)| (98)

where we define ‖v‖∞,i = sup‖v(s)‖ : s ∈ Di

≤ ‖f‖∞,2Cr2‖x− y‖+ ‖g‖∞,2

‖x− y‖r

(99)

≤ C ′r‖x− y‖ where C ′r = ‖f‖∞,2Cr2

+‖g‖∞,2r

. (100)

So br(x) is Lipschitz continuous on D2.

We next show that br(x) is Lipschitz continuous on D1 andD2. Choose x ∈ D1, y ∈ D2, and a point z of ∂D1 on theline segment between x and y. Then

|bir(x)− bir(y)| ≤ |bir(x)-bir(z)|+ |bir(z)-bir(y)| (101)≤ Cr‖x− z‖+ C ′r‖z − y‖ (102)≤ C ′r‖x− y‖ (103)

because C ′r ≥ Cr and ‖x− z‖ + ‖z− y‖ = ‖x− y‖. So br(x)is Lipschitz continuous with coefficient C ′r on ‖x‖ ≤ 2r. Wenow choose x ∈ (D1 ∪D2), y ∈ (D1 ∪D2)c, and a point zof ∂(D1 ∪D2)c on the line segment between x and y. Then

|bir(x)− bir(y)| ≤ |bir(x)-bir(z)|+ |bir(z)-bir(y)| (104)≤ Cr‖x− z‖+ 0 (105)≤ C ′r‖x− z‖+ C ′r‖z − y‖ (106)= C ′r‖x− y‖ . (107)

Then (89), (100), (103), and (107) show that bir(x) isLipschitz continuous with coefficient C ′r on Rd.

Consider next the SDE

dXt = br(Xt−)dt+ c(Xt−)dLt . (108)

Lemma 1 holds for (108) and so we can write

E[ sup0≤t≤Tr

‖Xt − Xt‖2]→ 0 as σj → 0 and νj → 0 (109)

for all j = 1, ..., m where Tr = inft ≥ 0 : ‖Xt‖ ≥ rand we choose r such that ‖Xt‖ < r for all t. Now defineTr = inf t ≥ 0 : ‖Xt‖ ≥ r and τr = inf t : ‖Xt‖ ≥r or ‖Xt‖ ≥ r = minTr, Tr. Then Xt and Xt satisfy (83)on [0, τr]. Note that Tr and Tr are stopping times and thus τris also a stopping time. So arguments similar to those of theproof of Lemma 1 ((68)-(76) with appropriate modifications)give

E

[sup

0≤u≤mint,τr‖Xu − Xu‖2

]

≤ Q′∫ t

0

E

[sup

0≤u≤mins,τr‖Xu − Xu‖2

]ds.(110)

Then

E

[sup

0≤u≤mint,τr‖Xu − Xu‖2

]≤ 0 (111)

by Gronwall’s inequality. Hence Xt = Xt holds almost surelyon [0, τr]. This result and (109) give

E[ sup0≤t≤τr

‖Xt − Xt‖2]→ 0 as σj → 0 and νj → 0 (112)

for all j = 1, ..., m.

We need now show only that τr → ∞ almost surely asr → ∞ to prove (86). Let XminT,τr be the value of theprocess Xt at time minT, τr. Note first that ‖XminT,τr‖2

=(Iτr>T‖XT ‖ + Iτr≤T‖Xτr‖

)2. Then

E[‖XminT,τr‖

2]

= E[(Iτr>T‖XT ‖

)2]+ E

[(Iτr≤T‖Xτr‖

)2]= E

[(Iτr>T‖XT ‖

)2]+ P (τr ≤ T )r2

because ‖Xτr‖ = r. Therefore

P (τr ≤ T ) ≤E[‖XminT,τr‖2

]r2

. (113)

Applying Ito’s lemma [3] to ‖XminT,τr‖2 gives

E[‖XminT,τr‖

2]≤ A′′ +Q′′

∫ T

0

E[‖Xmins,τr‖

2]ds.(114)

Thus

E[‖XminT,τr‖

2]≤ A′′eQ

′′T (115)

by Gronwall’s inequality where A′′ and Q′′ do not dependon r because we do not use the Lipschitz condition in thederivation of (114). Then (113) and (115) imply that

P (τr ≤ T ) ≤E[‖XminT,τr‖2

]r2

→ 0 as r →∞. (116)

Thus τr →∞ almost surely as r→∞. This implies the claim(85). Q.E.D.

REFERENCES

[1] H. Ahn and R.E. Feldman, “Optimal Filtering of GaussianSignals in the Presence of Levy Noise,” SIAM Journal of AppliedMathematics, vol. 60, no. 1, pp. 359-369, November 1999.

[2] S. Amari, “Neural Theory of Association and Concept Forma-tion,” Biological Cybernetics, vol. 26, pp. 175-185, May 1977.

[3] D. Applebaum, Levy Processes and Stochastic Calculus. Cam-bridge University Press, 2004.

[4] O. Barndoff-Nielson and N. Shephard, “Non-Gaussian Ornstein-Uhlenbeck-based Models and Some of Their Uses in FinancialEconomics,” Journal of the Royal Statistical Society, Series B,vol. 63, no. 2, pp. 167-241, 2001.

[5] S. M. Bezrukov and I. Vodyanov, “Noise-Induced Enhancementof Signal Transduction Across Voltage-Dependent Ion Chan-nels,” Nature, vol. 378, no. 6555, pp. 362-364, November 1995.

[6] A.R. Bulsara, E.W. Jacobs, T. Zhou, F. Moss, and L. Kiss,“Stochastic Resonance in a Single Neuron Model: Theory andAnalog Simulation,” Journal of Theoretical Biology, vol. 152,no. 4, pp. 531-555, October 1991.

[7] A.R. Bulsara, A.J. Maren, and G. Schmera, “Single EffectiveNeuron: Dendritic Coupling Effect and Stochastic Resonance,”Biological Cybernatics, vol. 70, no. 2, pp. 145-156, December1993.

[8] A.R. Bulsara and A. Zador, “Threshold Detection of WidebandSignals: A Noise Induced Maximum in the Mutual Information,”Physical Review E, vol. 54, no. 3, R2185-R2188, September1996.

[9] J. Cao and J. Lu, “Adaptive Synchronization of Neural net-works with and without Time-Varying Delays,” Chaos, vol. 16,013133-1 – 013133-6, March 2006.

15

[10] J. Cao, Z. Wang, and Y. Sun, “Synchronization in an Array ofLinearly Stochastically Coupled Networks with Time Delays,”Physica A, vol. 385, pp. 718-728, November 2007.

[11] F. Chapeau-Blondeau, X. Godivier, and N. Chambet, “Stochas-tic Resonance in a Neuron Model that Transmits Spike Trains,”Physical Review E, vol. 53, no. 1, pp. 1273-1275, January 1996.

[12] F. Chapeau-Blondeau and D. Rousseau, “Noise-Enhanced Per-formance for an Optimal Bayesian Estimator,” IEEE Transac-tions on Signal Processing, vol. 52, pp. 1327-1334, May 2004.

[13] F. Chapeau-Blondeau, S. Blanchard, and D. Rousseau, “Noise-Enhanced Fisher Information in Parallel Arrays of Sensors withSaturation,” Physical Review E, vol. 74, 0311021-03110210,September 2006.

[14] M. Chatterjee and M.E. Robert, “Noise Enhances ModulationSensitivity in Cochlear Implant Listeners: Stochastic Resonancein a Prosthetic Sensory System?” Journal of the Association forResearch in Otolaryngology, vol. 2, no. 2, pp. 159-171, August2001.

[15] M.A. Cohen and S. Grossberg, “Absolute Stability of GlobalPattern Formation and Parallel Memory Storage by CompetitiveNeural Networks,” IEEE Transactions on System, Man, and Cy-bernetics, vol. SMC-13, no. 5, pp. 815-826, September/October1983.

[16] J.J. Collins, C.C. Chow, A.C. Capela, and T.T. Imhoff, “Ape-riodic Stochastic Resonance in Excitable Systems,” PhysicalReview E, vol. 52, no. 4, pp. R3321-R3324, October 1995.

[17] J.J. Collins, C.C. Chow, A.C. Capela, and T.T. Imhoff, “Aperi-odic Stochastic Resonance,” Physical Review E, vol. 54, no. 5,pp. 5575-5584, November 1996.

[18] J.J. Collins, T.T. Imhoff, and P. Grigg, “Noise Enhanced Infor-mation Transmission in Rat SA1 Cutaneous Mechanoreceptorsvia Aperiodic Stochastic Resonance,” Journal of Neurophysiol-ogy, vol. 76, 642-645, July 1996.

[19] R. Cont and P. Tankov, Financial Modelling with Jump Pro-cesses. Chapman and Hall/CRC, 2003.

[20] T.M. Cover and J.A. Thomas, Elements of Information Theory.New York: Wiley, 1991.

[21] G. Deco and B. Schurmann, “Stochastic Resonance in theMutual Information Between Input and Output Spike Trains ofNoisy Central Neurons”, Physica D, vol. 117, pp. 276-282, June1998.

[22] T. Ditzinger, M. Stadler, D. Struber and J.A. Kelso, “NoiseImproves Three-Dimensional Perception: Stochastic Resonanceand other Impacts of Noise to the Perception of Autostere-ograms,” Physical Review E, vol. 62, no. 2, pp. 25662575,August 2000.

[23] J.K. Douglass, L. Wilkens, E. Pantazelou, and F. Moss, “NoiseEnhancement of Information Transfer in Crayfish Mechanore-ceptors by Stochastic Resonance,” Nature, vol. 365, pp. 337-340,September 1993.

[24] R. Durrett, Stochastic Calculus: A Practical Introduction. CRCPress, 1996.

[25] R. Durrett, Probability: Theory and Examples. Duxbury Press,2004.

[26] R.A. FitzHugh, “Impulses and Physiological States in Theoret-ical Models of Nerve Membrane,” Biophysical Journal, vol. 1,pp. 445-466, July 1961.

[27] L. Gammaitoni, “Stochastic Resonance and the Dithering Effectin Threshold Physical Systems,” Physical Review E, vol. 52, no.5, pp. 4691-4698, November 1995.

[28] W. Gerstner and W.M. Kistler, Spiking Neuron Models: SingleNeurons, Populations, Plasticity. Cambridge University Press,2002.

[29] M.T. Giraudo and L. Sacerdote, “Jump-Diffusion Processes asModels of Neuronal Activity,” BioSystems, vol. 40, no. 1-2, pp.75-82, 1997.

[30] M. Grigoriu, Applied Non-Gaussian Processes. EnglewoodCliffs, NJ: Prentice Hall, 1995.

[31] B.S. Gutkin and G.B. Ermentrout, “Dynamics of MembraneExcitability Determine Interspike Interval Variability: A LinkBetween Spike Generation Mechanism and Cortical Spike TrainStatistics,” Neural Computation, vol. 10, no. 5, pp. 1047-1065,July 1998.

[32] P. Hanggi, “Stochastic Resonance in Biology: How NoiseCan Enhance Detection of Weak Signals and Help ImproveBiological Information Processing,” Chemphyschem, vol. 3, pp.285-290, March 2002.

[33] A. Hanssen and T.A. Oigard, “The Normal Inverse GaussianDistribution: A Versatile Model For Heavy-tailed Stochastic Pro-cesses,” IEEE International Conference on Acoustics, Speech,and Signal Processing, ICASSP’01, pp. 3985-3988, May 2001.

[34] A.L. Hodgkin and A.F. Huxley, “A Quantitative Descriptionof Membrane Current and its Application to Conduction andExcitation in Nerve,” Journal of Physiology, vol. 117, pp. 500-540, August 1952.

[35] N. Hohn and A.N. Burkitt, “Shot Noise in Leaky Integrate-and-Fire Neuron,” Physical Review E, vol. 63, 031902-1 – 031902-11, March 2001.

[36] F.C. Hoppensteadt and E.M. Izhikevich, Weakly ConnectedNeural Networks. New York: Springer, pp. 111-142.

[37] J.J. Hopfield, “Neural Networks with Graded Response HaveCollective Computational Properties Like Those of Two-stateNeurons,” Proceeding of National Academy of Science, vol. 81,pp. 3088-3092, May 1984.

[38] H. Huang and J. Cao, “Exponential Stability Analysis ofUncertain Stochastic Neural Networks with Multiple Delays,”Nonlinear Analysis: Real World Applications, vol. 8, pp. 646-653, April 2007.

[39] M.E. Inchiosa, J.W.C. Robinson, and A.R. Bulsara,“Information-Theoretic Stochastic Resonance in Noise-FloorLimited Systems: The Case for Adding Noise,” Physical ReviewLetters, vol. 85, pp. 3369-3372, October 2000.

[40] F. Jaramillo and K. Wiesenfeld, “Mechanoelectrical Transduc-tion Assisted by Brownian Motion: A Role for Noise in theAuditory System,” Nature Neuroscience, vol. 1, pp. 384-388,1998.

[41] P. Jung, A. Cornell-Bell, F. Moss, S. Kadar, J. Wang, andK. Showalter, “Noise Sustained Waves in Subexcitable Media:From Chemical Waves to Brain Waves,” Chaos, vol. 8, no. 3,pp. 567-575, September 1998.

[42] A. Kalemanova, B. Schmid, and R. Werner, “The Normal In-verse Gaussian Distribution for Synthetic CDO Pricing,” Journalof Financial Derivatives, vol. 14, 2007.

[43] G.G. Karapetyan, “Application of Stochastic Resonance inGravitational-Wave Interferometer,” Physical Review D, vol. 73,122003, 1-9, June 2006.

[44] S.A. Kassam, Signal Detection in Non-Gaussian Noise.Springer-Verlag, 1988.

[45] S. Kay, “Can Detectability be Improved by Adding Noise?”IEEE Signal Processing Letters, vol. 7, pp. 8-10, January 2000.

[46] E.R. Kandel, J.H. Schwartz, and Thomas M. Jessell, Principlesof Neuroscience. McGraw-Hill, 4th Revised Edition, pp. 224,2000.

[47] A.F. Kohn, “Dendritic Transformations on Random SynapticInputs as Measured from Neuron’s Spike Train–Modeling andSimulation,” IEEE Transactions on Biomedical Engineering,vol. 36, no. 1, pp. 44-54, January 1989.

[48] B. Kosko, Neural Networks and Fuzzy Systems: A DynamicalSystems Approach to Machine Intelligence. Englewood Cliffs,NJ: Prentice-Hall, 1991.

[49] B. Kosko, Noise. Viking/Penguin, 2006.[50] B. Kosko and S. Mitaim, “Stochastic Resonance in Noisy

Threshold Neurons,” Neural Networks, vol. 14, no. 6-7, pp. 755-761, June 2003.

[51] B. Kosko and S. Mitaim, “Robust Stochastic Resonance for

16

Simple Threshold Neurons,” Physical Review E, vol. 70,031911-1 – 031911-10, 27 September 2004.

[52] I. Y. Lee, X. Liu, C. Zhou, and B. Kosko, “Noise-EnhancedDetection of Subthreshold Signals with Carbon Nanotubes,”IEEE Transactions on Nanotechnology, vol. 5, no. 6, pp. 613- 627, November 2006.

[53] J.E. Levin, and J.P. Miller, “Broadband Neural Encoding inthe Cricket Cercal Sensory System Enhanced by StochasticResonance,” Nature, vol. 380, pp. 165-168, March 1996.

[54] B. Lindner, A. Longtin, and A. R. Bulsara, “Analytic Expres-sions for Rate and CV of a Type I Neuron Driven by WhiteGaussian Noise,” Neural computation, vol. 15, pp. 1761-1788,August 2003.

[55] B. Lindner, J. Garcia-Ojalvo, A. Neiman, and L. Schimansky-Geier, “Effects of Noise in Excitable Systems,” Physics Reports,vol. 392, pp. 321-424, March 2004.

[56] A. Longtin, “Autonomous Stochastic Resonance in BurstingNeurons,” Physical Review E, vol. 55, no. 1, pp. 868-876,January 1997.

[57] X. Mao, Stochastic Differential Equations and their Applica-tions. Horwood Publishing, Chichester, 1997.

[58] M.D. McDonnell, N. Stocks, C.E.M. Pearce, and D. Abbott,“Quantization in the Presence of Large Amplitude ThresholdNoise,” Fluctuation Noise Letters, vol. 5, no. 3, L457-L468,September 2005.

[59] C. Mead, Analog VLSI and Neural Systems. Addison Wesley,1989.

[60] S. Mitaim and B. Kosko, “Adaptive Stochastic Resonance,”Proceedings of the IEEE: Special Issue on Intelligent SignalProcessing, vol. 86, no. 11, pp. 2152-2183, November 1998.

[61] S. Mitaim and B. Kosko, “Adaptive Stochastic Resonance inNoisy Neurons Based on Mutual Information,” IEEE Trans-actions on Neural Networks, vol. 15, no. 6, pp. 1526-1540,November 2004.

[62] F. Moss, L.M. Ward, and W.G. Sannita, “Stochastic Resonanceand Sensory Information Processing: A Tutorial and Review ofApplications,” Clinical Neurophysiology, vol. 115, no. 2, pp.267-281, February 2004.

[63] C.L. Nikias and M. Shao, Signal Processing with Alpha-StableDistributions and Applications. John Wiley and Sons, New York,1995.

[64] A. Pacut, “Separation of Deterministic and Stochastic Neuro-transmission,” International Joint Conference on Neural Net-works, IJCNN’01, vol. 1, pp. 55-60, July 2001.

[65] A. Patel and B. Kosko, “Stochastic Resonance in Noisy SpikingRetinal and Sensory Neuron Models,” Neural Networks, vol. 18,pp. 467-478, August 2005.

[66] A. Patel and B. Kosko, “Mutual-Information Noise Benefits inBrownian Models of Continuous and Spiking Neuron Models,”International Joint Conference on Neural Networks, IJCNN’06,pp. 2347-2354, July 2006.

[67] O. V. Poliannikov, Y. Bao and H. Krim, “Levy Processesfor Image Modelling,” Proceedings of IEEE Signal ProcessingWorkshop on Higher-Order Statistics, June 14-16, Caesarea,Israel, 1999.

[68] S.I. Resnick, Heavy-Tail Phenomena: Probabilistic and Statis-tical Modeling. Springer 2007.

[69] D. Rousseau, J. Rojas Varela, and F. Chapeau-Blondeau,“Stochastic Resonance for Nonlinear Sensors with Saturation,”Physical Review E, vol. 67, No.2, 0211021-02110216, February2003.

[70] D. Rousseau and F. Chapeau-Blondeau, “Neuronal SignalTransduction Aided by Noise at Threshold and at Saturation,”Neural Processing Letters, vol. 20, No.2, pp. 71-83, November2004.

[71] D. Rousseau and F. Chapeau-Blondeau, “Constructive Role ofNoise in Signal Detection From Parallel Array of Quantizers,”Signal Processing, vol. 85, no. 3, pp. 571-580, March 2005.

[72] D. Rousseau, G. V. Anand, and F. Chapeau-Blondeau, “Noise-enhanced Nonlinear Detector to Improve Signal Detection inNon-Gaussian noise,” Signal Processing, vol. 86, no. 11, pp.3456-3465, November 2006.

[73] T. Rydberg, “The Normal Inverse Gaussian Levy Processes:Simulation and Approximation,” Communications in Statistics:Stochastic Models, vol. 13, pp. 887-910, 1997.

[74] L. Sacerdote and R. Sirovich, “Multimodality of the InterspikeInterval Distribution in a Simple Jump-Diffusion Model,” Sci-entiae Mathematicae Japonicae, vol. 58, no. 2, pp. 307-322,September 2003.

[75] Y. Sakumara and K. Aihara, “Stochastic Resonance and Inci-dence Detection in Single Neuron,” Neural Processing Letters,vol. 16, pp. 235-242, December 2002.

[76] G. Samorodnitsky and M.s. Taqqu, Stable Non-Gaussian Ran-dom Processes: Stochastic Models with Infinite Variance. Chap-man & Hall/CRC, 1994.

[77] K-I. Sato, Levy Processes and Infinitely Divisible Distributions.Cambridge University Press, 1999.

[78] W. Schoutens, Levy Processes in Finance: Pricing FinancialDerivatives. John Wiley & Sons, 2003.

[79] W. Schoutens, “Exotic Options under Levy Models: AnOverview,” Journal of Computational and Applied Mathematics,vol. 189, no. 1, 526 - 538, May 2006.

[80] R. Segev, M. Benveniste, E. Hulata, N. Cohen, A. Palevski, E.Kapon, Y. Shapira, and E. Ben-Jacob, “Long Term Behaviorof Lithographically Prepared In Vitro Neuronal Networks,”Physical Review Letters, vol. 88, no. 11, 118102-1 – 118102-4,March 2002.

[81] M.F. Shlesinger, G. M. Zeslavsky, and U. Frisch, Eds., “LevyFlights and Related Topics in Physics: Proceedings of the Inter-national Workshop Held at Nice, France, June 27-30,” LectureNotes in Physics, vol. 450, Springer-Verlag, Berlin, 1995.

[82] O. Sotolongo-Costa, J.C. Antoranz, A. Posadas, F. Vidal and A.Vazquez, “Lvy Flights and Earthquakes,” Geophysical ResearchLetters, vol. 27, pp. 1965-1967, July 2000.

[83] W.C. Stacey and D.M. Durand, “Stochastic Resonance ImprovesSignal Detection in Hippocampal CA1 Neurons,” Journal ofNeurophysiology, vol. 83, pp. 1394-1402, September 2001.

[84] N.G. Stocks, “Information Transmission in Parallel ThresholdArrays: Suprathreshold Stochastic Resonance,” Physical ReviewE, vol. 63, no. 4, 041114-1 – 041114-9, March 2001.

[85] N.G. Stocks, D. Appligham, and R.P. Morse, “The Applicationof Suprathreshold Stochastic Resonance to Cochlear ImplantCoding,” Fluctuation and Noise Letters, vol. 2, no. 3, L169-L181, September 2002.

[86] Y. Sun and J. Cao, “Adaptive Lag Synchronization of UnknownChaotic Delayed Neural Networks with Noise Perturbations,”Physics Letters A, vol. 364, pp. 277-285, April 2007.

[87] Z. Wang, Y. Liu, M. Li, and X. Liu, “Stability Analysis forStochastic Cohen-Grossberg Neural Networks with Mixed TimeDelays,” IEEE Transactions on Neural Networks, vol. 17, no. 3,pp. 814-820, May 2006.

[88] Z. Wang, Y. Liu, K. Fraser, and X. Liu, “Stochastic Stabilityof Uncertain Hopfield Neural Networks with Discrete and Dis-tributed Delays,” Physics Letters A, vol. 354, no. 4, pp. 288-297,June 2006.

[89] K. Wiesenfeld and F. Moss, “Stochastic Resonance and theBenefits of Noise: From Ice Ages to Crayfish and SQUIDS,”Nature, vol. 373, pp. 33-36, January 1995.

[90] W. Yu and J. Cao, “Synchronization Control of StochasticDelayed neural Network,” Physica A, vol. 373, pp. 252-260,January 2007.

[91] F.G. Zeng, Q.J. Fu and R. Morse, “Human Hearing Enhancedby Noise,” Brain Research, vol. 869, no. 1-2, pp. 251–255, June2000.

17

Ashok Patel is a Ph.D. candidate in the Departmentof Electrical Engineering at the University of South-ern California (USC). He received his M.A. degreein applied mathematics and M.S. degree in electricalengineering from USC. He received his B.E. andM.E. degrees in electrical engineering from GujaratUniversity, Ahmedabad, India. His current researchinterests are in the areas of nonlinear and statisticalsignal processing.

Dr. Bart Kosko is a professor of electrical en-gineering at the University of Southern California(USC). He holds degrees in philosophy, economics,mathematics, electrical engineering, and law, and isa licensed attorney and former director of USC’sSignal and Image Processing Institute. Dr. Koskois an elected governor of the International NeuralNetwork Society. He has chaired or cochaired sev-eral international conferences on neural and fuzzysystems conferences and sits on the editorial boardof several scientific journals. Dr. Kosko is author of

the trade books Noise, Heaven in a Chip, and Fuzzy Thinking, as well asthe novel Nanotime. He has authored the textbooks Fuzzy Engineering andNeural Networks and Fuzzy Systems, edited the volume Neural Networks forSignal Processing, and co-edited the volume Intelligent Signal Processing. Heis author of the trade books Fuzzy Thinking, Heaven in a Chip, and the novelNanotime. His most recent book is Noise.

stochastic resonance in continuous and spiking neuron...

Documents