Deterministic Prediction andChaos in Squid Axon ResponseA. MeesK. AiharaM. AdachiK. JuddT. Ikeguchi

SFI WORKING PAPER: 1991-12-049

SFI Working Papers contain accounts of scientific work of the author(s) and do not necessarily represent theviews of the Santa Fe Institute. We accept papers intended for publication in peer-reviewed journals or proceedings volumes, but not papers that have already appeared in print. Except for papers by our externalfaculty, papers must be based on work done at SFI, inspired by an invited visit to or collaboration at SFI, orfunded by an SFI grant.©NOTICE: This working paper is included by permission of the contributing author(s) as a means to ensuretimely distribution of the scholarly and technical work on a non-commercial basis. Copyright and all rightstherein are maintained by the author(s). It is understood that all persons copying this information willadhere to the terms and constraints invoked by each author's copyright. These works may be reposted onlywith the explicit permission of the copyright holder.www.santafe.edu

SANTA FE INSTITUTE

Deterministic Prediction and Chaosin Squid Axon Response

A.Mees(1) , K.Aihara(2), M.Adachi(2), K.Judd(2) , T.Ikeguchi(3)and G.Matsumoto(4)

(l)Mathematics Department, The University of Western Australia, Nedlands,Perth 6009(2)Department of Electronic Engineering, Tokyo Denki University, 2-2 Kanda,Chiyoda, Tokyo 101(3)Department of Applied Electronics, Science University of Tokyo, Yamazaki,Noda 278(4)Electrotechnical Laboratory, 1-1-4 Umezono, Tsukuba 305, Japan

Abstract

We make deterministic predictive models of apparently complex squid axonresponse to periodic stimuli. The result provides evidence that the responseis chaotic (and therefore partially predictable) and implies the possibility ofidentifying deterministic chaos in other kinds of noisy data even when explicitmodels are not available.

1

Deterministic chaos is often characterised by its long-term unpredictabil­ity. Short term prediction may, however, be possible if a good enough modelis available, together with sufficiently accurate measurements. Recent workon building mathematical models directly from data[1-7] has made it possibleto detect deterministic structure in some time series data which appear atfirst sight to be relatively unpredictable.

Since biological systems are essentially nonlinear and nOlllequilibrium[8­10), it is natural to expect existence of deterministic chaos in such systems.Various studies have been directed toward discovering and understandingchaos in biological systems such as brains and hearts[1l-13]. For example, ithas been experimentally confirmed that when squid giant axons are forced pe­riodically, the nerve membranes fire irregulady under some cir.cumstances[14­17]. Fig.1 shows a time series in which the membrane potential is strobo­scopically sampled at the leading edge of each stimulating pulse. No clearregularity is apparent. The spectrum is shown in Fig.2: in spite of somepeaks, it is not indicative of a periodic signal, or of a simple superposition ofperiodic signals. It does not exclude the possibility that this axon responsecould be stochastic.

Because of non-stationarity due to axon fatigue, the data, which are ineffect from a Poincare section[18-21)' are not readily available in sufficientquantity to allow reliable calculation of such statistics as fractal dimension.Estimation of the latter is, in any case, a delicate procedure, and even areliable estimate with confidence intervals[22] will not necessarily distinguishdeterministic from stochastic response.

If an explanation for the system's behaviour is proposed-for example,in the form of a dynamical system-it can be tested by requiring it to makepredictions and then validating or falsifying them by experiment. The presentinvestigation adopted this approach. The Hodgkin-Huxley system[23] is theobvious candidate, and is an excellent mathematical model even for chaoticresponse[14,18-21], although it does not fit squid a..xon response well undercertain conditions[24]. However, we wanted to have a method that wouldalso work in other cases where no generally accepted model is available.

We chose instead to create an approximate model system directly fromthe data. The model is a function (usually defined via a computer algorithm)which estimates the value of the membrane potential at time t + k, given itsvalues at times 0, 1, ... , t. Several methods are available[1-7]; we selected thetesselation[5,7] and neural network[3,25] approximations.

2

Both methods assume that the data has been embedded[26]. Figure 3(a)shows an embedding in two dimensions; that is, a plot of Xl+! aLgainst Xl' Thisis strong evidence that the data is not stochastic; the points lie almost on aone-dimensional curve, and certainly do not densely fIll an area of the planeas a simple random process would. As the time series data seem to be wellembedded in the two-dimensional space, we used an embedding dimensionof 2 and a lag of 1 in the subsequent analysis.

The time series data of the squid axon response were di vided into firstand second halves for learning and testing, respectively. In deterministic pre­diction via tesselation[5,7], the first 250 points Xl, X2, .•. , X250 from our 500point data set were embedded in the two-dimensional space and tesselatedas the starting model for prediction. For each successive time t > 250, dataup to and including Xl were used to make predictions i l+k of the responseXl+k for k = 1,2,3,4 where k is prediction time.

In deterministic prediction via neural networks, the first half of the datawas used for learning with the back-propagation algorithm[25]. The structureof the neural network was feed forward with three layers, having respectively2 input, 9 hidden and 2 output neurons. The input signals to the first layerand the teaching signals to the last layer were (Xl, Xl+!) and (Xl+!' Xt+2),

respectively. For t > 250, the output of the second neuron in the last layergives one-step predictions. By feeding back the output signals to the inputlayer k times successively, we can also get k + I-step predictions. With eitherprediction method we can allow free-running to see whether an apparentstrange attractor is produced. A possible strange attractor produced by theneural network model is shown in Fig.3(b).

The result of the deterministic prediction is shown in Fig.4 where theordinate is the correlation coefficient between actual and predicted valuesand the abscissa is the number of steps ahead being predicted. The solidand dotted lines in Fig.4( a) correspond to the tesselation predictions andthe neural network predictions, respectively. The performance of the back­propagation network depends upon the initial choice of its connection weightsand thresholds as demonstrated in Fig.4(b); the dotted line in Fig.4(a) showsthe average performance over the five networks in Fig.4(b).

The one-step prediction it+! was excellent in both methods, with corre­lation better than 92% between actual and predicted values. Correlationsdecreased as prediction time k increased, as shown in Figure 4. As argued bySugihara and May[6], tltis is evidence for chaotic dynamics: chaos is charac-

3

terized by sensitivity to initial conditions, and the errors both from modeUingand from measurement will cause divergence between the predicted and thetrue val ues as prediction time increases.

The Lyapunov spectrum[27] is a useful index of sensitive dependence oninitial conditions. The largest exponent AI of the Lyapunov spectrum can beestimated from relationships between the prediction time T and statisticalquantities such as the root-mean-square error E [2,28] and the correlationcoefficient l' [29]. Using the relationship between T and E for iterated pre­diction given by Casdagli et al. [28], we obtain estimated values for AI of0.35 for the tesselation and 0.36 for the averaged neural nehvork model ofFig.4(a). Using Wales's result (equation (7) in Ref.(29]) relating r to AI andT, the corresponding values of AI are 0.35 and 0.37 for the tesselation andthe neural network, respectively. Moreover, the value of 0.39 is obtained bythe Sano-Sawada algorithm[30]. These similar values give credence to theexistence of a positive Lyapunov exponent in the dynamics of the squid axonresponse.

In all our tests there was good agreement between the two qui te differentmodelling methods, so it seems unlikely that the results are a lucky acci­dent. An alternative approach, which constructs a simple explicit modelfrom understancling of refractoriness and other neural characteristics, alsogives similar results[21].

Acknowledgements

AIM thanks Tokyo Denki University for hospit.ality and JSPS and the Aus­tralian Academy of Sciences for a travel grant. This research was partiallyfunded by ARC grant A69031387 and by a Grant-in-Aid(02255107) for Sci­entific Research on Priority Areas from the Ministry of Education, Scienceand Culture of Japan. A. I. Mees thanks the Santa Fe Insti-

tute and the Los Alamos National Laboratory.

References

[1] J.P. Crutchfield & B.S. McNamara, Comple:v Systems 1,417-452 (1987).

[2] J.D. Farmer & J.J. Sidorowich, Phys. Rev. Letters 59,845-848 (1987).

4

Nonequilibl'ium PhaseChemistr'y and Biology

[3] A. Lapedes & R. Farber, Nonlineal' signal pl'ocessing using neural net­wOl'ks: pl'ediction and system modelling (Los Alamos National Labora­tory, 1987).

[4] M. Casdagli, Physica D 35, 335-356 (1989).

[5] A.I. Mees, in Dynamics of Complex Intel'connected Biological Systems(eds. T. Vincent, A.I. Mees & L.S. Jennings) 104-124 (Birkhauser,Boston, 1990).

[6] G. Sugihara & R.M. May, Natul'e 344,734-741 (1990).

[7J A.I. Mees, Intemational Joul'nal of Biful'cation and C/wos 1, in press(1991).

[8] P. GlansdorIT & I. Progogine, Thel'modynamic theol'y of stl'uctUl'e, sta­bility and fluctuations (Wiley-Interscience, London, 1971).

[9J G. Nicolis & I. Prigogine, Self-ol'ganization in nonequilibl'ium sys­tems: f1'Om dissipative stl'uctUl'es to ol'del' th1'Ough fluctuations (Wiley­Interscience, London, 1977).

[10] H. I-Iaken, Synel'getics - an Introduction:Transitions and Self- 0l'ganisation in Physics,(Springer-Verlag, Berlin, 1977).

[I1J A.V. Holden, Chaos (Manchester University Press, Manchester, 1986).

[12] H. Degn, A.V. Holden & L.F. Olsen, Chaos in Biological Systems(Plenum Press, New York, 1987).

[13] B.J. West, Fractal Physiology and Chaos in Medicine (World Scientific,Singapore, 1990).

[14] K. Aihara & G. Matsumoto, in Chaos (eds. A.V. Holden) 257-269(Manchester University Press, Manchester, 1986).

[15] IC Aihara, T. Numajiri, G. Matsumoto & M. I<otani, Physics Letters A

116, 313-317 (1986).

[16] G. Matsumoto et aI., Physics Letters A 123, 162-166 (HI87).

5

[17] N. Takahashi et a!., Physica D 43, (1990).

[18] K. Aihara, G. Matsumoto & Y. Ikegaya, J. Theor. BioI. 109, 249-269(1984).

[19] K. Aihara & G. Matsumoto, in Chaos in Biological Systems (eds. H.Degn, A.V. Holden & L.F. Olsen) 121-131 (Plenum Press, New York,1987) .

[20] K. Aihara, in Bifurcation phenomena in nonlinear systems and theoryof dynamical systems (eds. H. Kawakami) 143-161 (World Scientific,Singapore, 1990).

[21] K. Judd, Y. Hanyu, N. Takahashi & G. Matsumoto, to be submitted toJ. Math. Bioi. (1992).

[22] IC Judd & A.I. Mees, Inte1'7lational Journal of Bifurcation and Chaos1, in press (1991).

[23] A.L. Hodgkin & A.F. Huxley, J. Physiol. (London) 117,500-544 (1952).

[24] Y. Hanyu & G. Matsumoto, Physica D , in press (1991).

[25] D.E. Rumelhart, G.E. Hinton & R.J. Williams, Nature 323, 533-536(1986).

[26] F. Takens, in Dynamical Systems and Turbulence (eds. D.A. Rand &L.S. Young) 365-381 (Springer, Berlin, 1981).

[27] I. Shimada & T. Nagashima, Progress in Theoretical Physics 61, 1605(1979).

[28] M. Casdagli, D. des Ja.rdins, S. Eubank, J.D. Farmer, J. Gibson, N.Hunter & H. Theiler, Los Alamos National Laboratory, Report LA-UR­91-1637 (1991).

[29] D.J. Wales, Nature 350,485-488 (1991).

[30] M. Sano & Y. Sawada, Phys. Rev. Lett. 55, 1082-1085 (1985).

6

Figure Captions

Fig.lSquid axon membrane potentials stroboscopically sampled at the leadingedge of each stimulating pulse. The squid axon in the resting state was stim­ulated by periodic pulses with amplitude 1.19 times the threshold current,pulse width 0.3 msec and pulse interval 3.8 msec. The temperature was14.0'0. The number of data points is 500. The data are normalized between-0.5 and 0.5.

Fig.2Power spectrum of the squid axon response time series in Fig.l, computedwith a Hanning window of width 64. The dashed lines are !~5% confidenceintervals.

Fig.3(a) An embedding in 2 dimensions of the first half of the squid data in Fig.1.(b) A possible strange attractor obtained by allowing a ba.ck-propagationnetwork modelling the squid data to free-run (that is, to simulate the systemfor many time steps).

Fig.4Correlation coefficient between actual and predicted values as a function ofincreasing prediction time. (a) The solid and dotted lines correspond topredictions using tesselation and using a back-propagation neural network,respectively. The dotted line is the average over five networks shown in (b).The initial values of connection weights and thresholds in each network ofFig.4(b) were determined randomly from a uniform distribution between -0.3and 0.3. The neural nets were trained for a fixed time; on termination, theroot mean square error between output and teaching signals in the last layerwas in all cases less than 0.039.

7

400200

0.5--r-........--------------------,

0.4

0.3

0.2

0.1

o-0.1

-0.2

-0.3

-0.4

-0.5-+-J----,-------r--!..---,.----r------/o

Time

Fi~. 1

CMt' es €t CLI.)

0.500000

0.000000

X( I +1) a• '",>"l

.._..~~,.~I. '

.." ."~~:.:~.

' ..,

.i·:···l~":i!-=....

..' "

f·

-0.500000 0.000000

-0.500000.j--~-~-~-~-~-~--~~X(I)

0.500000

X(I+I)

0.500000

0.000000

b .'--".'\);• ',-"' .... 1:\-'

"' ' "\ .

....... .,~ . .. ~ ' .

./

!r', .\

~

-0.500000 0.000000

-0.500000.j--~-~-~-~-~-~~-~--,x(t)

0.500000

F;~. ~ ( Mee s ·eA 1\/)

100-.---------------------.;..-----,

80

70 a....

.....' .............

'.....

60

50

40

'" ....

""".....".". ". ".". ".".".

30--L.....,...-----....,.------,..------,..--11 2 3 4

Prediction Timeo Tesselation + BP

4

.... , .. '.'0..

100""':""---------------....;.;...--------..,90­

80­

70­

60­

50­

40-

30-

20-

10-

0-t------------------------'-...,""'O-~

-10 .......--,,;---------r-,-------".---------"T"""--I1 2 3

Prediction Time