Predictive Coding as Stimulus Avoidancein Spiking Neural Networks

Atsushi MasumoriGraduate School of Arts and Sciences

The University of TokyoTokyo, Japan

masumori@sacral.c.u-tokyo.ac.jp

Takashi IkegamiGraduate School of Arts and Sciences

The University of TokyoTokyo, Japan

ikeg@sacral.c.u-tokyo.ac.jp

Lana SinapayenSony Computer Science Laboratories, Inc.

Tokyo, JapanEarth-Life Science Institute

Tokyo, Japanlana.sinapayen@gmail.com

Abstract—Predictive coding can be regarded as a functionwhich reduces the error between an input signal and a top-downprediction. If reducing the error is equivalent to reducing theinfluence of stimuli from the environment, predictive coding canbe regarded as stimulation avoidance by prediction. Our previousstudies showed that action and selection for stimulation avoid-ance emerge in spiking neural networks through spike-timingdependent plasticity (STDP). In this study, we demonstrate thatspiking neural networks with random structure spontaneouslylearn to predict temporal sequences of stimuli based solely onSTDP.

Keywords—Predictive coding; Spiking neural networks; Spike-timing dependent plasticity

I. INTRODUCTION

Prediction has recently been argued to be a importantfunction of the brain [1], [2] and predictive coding [3], [4]has attracted the attentions of researchers in many fields[5], [6]. Predictive coding can be regarded as a functionfor reducing errors between an input signal and a top-downprediction. If reducing errors is equivalent to reducing theinfluence of environmental stimuli, predictive coding can beregarded as stimulation avoidance by prediction. Our previousstudies showed that spiking neural networks with spike-timingdependent plasticity (STDP) [7] learn to avoid stimuli from theenvironment by their action [8], [9]. Cultured neural networkslearn actions to avoid stimulation in the same way [10]. Inaddition, we found that neuronal cultures avoid stimulationby selecting what external information is received or declined[11]. In our previous study, we demonstrated that spiking neu-ral networks with asymmetric STDP, in which the dynamics oflong-term potentiation and long-term depression are rotationalasymmetric, reproduced such selection. Therefore, based onSTDP, two types of stimulation avoidance emerge: stimulationavoidance by action and stimulation avoidance by selection.

In this study, we evaluate whether prediction also emerges inspiking neural networks based on STDP. Several studies haveexamined the prediction of temporal stimulation in spikingneural networks [12], [13], [14]. However, these predictivenetworks require specifically designed network topology orother synaptic functions than STDP, such as short-term plas-ticity. We hypothesise that there is no need to include suchstructures or functions other than STDP for learning to predicta simple sequence of stimuli because our previous studies

showed that neural networks with random initial weights learnto avoid stimuli by their action based on STDP, and for hiddenneurons, a prediction that inhibits the input neurons responseto stimulation is equivalent to an action that eliminates thestimulation.

We first demonstrate that minimal predictive networks con-sisting of 3 to 6 neurons with synaptic weight governed bySTDP spontaneously learn to predict sequences of stimuli. Wethen show that even larger random networks (of 100 neurons)without a specifically designed structure spontaneously learnto predict sequences of stimuli based solely on STDP.

II. METHODS

A. Izhikevich Neuron Model

The spiking neuron model proposed by Izhikevich [15] wasused to simulate excitatory and inhibitory neurons consistingsmall and large networks. This model is widely applied as indi-visual parameters can be adjusted to reproduce the dynamics ofmany types of neurons, and it is also computationally efficient.The basic equations of this neural model are as follows:

dv

dt= 0.04v2 + 5v + 140− u+ I,

du

dt= a(bv − u),

if v ≥ 30 mV, then

{v ← c

u← u+ d.

(1)

Here, v represents the membrane potential of the neuron, uis a variable related to membrane repolarization, I representsthe input current (with multiple components as explainedbelow), t is time, and a, b, c, and d are parameters controllingthe shape of the spike [15]. The neuron is regarded as firingwhen the membrane potential v ≥ 30 mV. The parametersfor excitatory neurons were set to a = 0.02, b = 0.2, c =−65 mV, and d = 8, and the parameters for inhibitoryneurons to a = 0.1, b = 0.2, c = −65 mV, and d = 2. Withthese parameters, excitatory neurons show regular spiking andinhibitory neurons show fast spiking (Fig. 1). The simulationtime step ∆t was set to 1 ms.

The variable I represents depolarization evoked by synapticcurrents, noise, and external stimuli, and was added to the

arX

iv:1

911.

0923

0v1

[cs

.NE

] 2

1 N

ov 2

019

membrane potential of each neuron ni at every time step asfollows:

Ii =

n∑j=0

fjwji + ei +mi

fj =

{1, if neuron j is firing0, otherwise.

(2)

Here, wji represents the weight of individual synapse be-tween presynaptic neuron j to postsynaptic neuron i, whereweights are positive for synapse from excitatory neurons andnegative for synapses from inhibitory neurons, m is zero-meanGaussian noise with standard deviation σ = 3 mV whichrepresents the internal noise, e represents the external stim-ulation (with conditions, frequency and strength of externalstimulation varying among on experiments).

Fig. 1. Firing patterns of regular spiking and fast spiking neurons simulatedusing the Izhikevich model. Regular-spiking neurons are used as excitatoryneurons and fast-spiking neurons are used as inhibitory neurons.

In most of the experiments, there was no synaptic delay;however, in the experiments with the longer temporal stimulussequence (described below), a synaptic delay were addedbetween an action potential of the presynaptic neuron andpostsynaptic potential. In these experiments, the function fj(Eq. 2) was modified by the synaptic delay according to

fj =

{1, t− tsj = tdij

0, otherwise.(3)

where t denotes the current simulation time, tsj representsspike timing of presynaptic neuron j, and tdij representsthe synaptic delay between neuron i and neuron j. In theexperiments with synaptic delay, each pair of excitatory andinhibitory neuron was connected by 15 synapses and the td ofeach synapse was varied from 1 to 15 ms.

B. Spike-Timing Dependent Plasticity

Spike-timing dependent plasticity was used as the mecha-nism for changing synaptic weights between spiking neurons.In this model, synaptic weight increased when the presynapticneuron fires before the postsynaptic neuron and decreasedwhen the presynaptic neuron fires after the postsynaptic neu-ron. The weight variation ∆w is defined by

∆w =

{A(1− 1

τ )∆t, if ∆t > 0

−A(1− 1τ )−∆t, if ∆t < 0.

(4)

Here, ∆t represents the relative spike timing between presy-naptic neuron a and postsynaptic neuron b: ∆t = tb − ta (tarepresents the spike timing of neuron a, and tb represents thespike timing of neuron b). For excitatory synapses, A = 0.1and τ = 20 ms. Figure 2 shows the variation of ∆w dependingon ∆t; ∆w is negative when the postsynaptic neuron firesfirst and positive when the presynaptic neuron fires first. Notethat STDP was applied not only to the connections betweenexcitatory neurons but also to connections from inhibitoryneurons to excitatory neurons. Although STDP at inhibitorysynapse is still controversial, here we applied the reverse shapeof the STDP function in Figure 2 for the inhibitory synapses(A = −0.1, τ = 20).

The weight value w varies as

wt = wt−1 + ∆w . (5)

The maximum possible weight was fixed at wmax = 80 forexcitatory synapses and wmax = 0 for inhibitory synapses,and if w > wmax, w was reset to wmax. Alternatively,the minimum possible weight was fixed at wmin = 0 forexcitatory synapses and wmin = −80 for inhibitory synapses,and if w < wmin, w was reset to wmin.

Fig. 2. The spike-timing depended plasticity (STDP) function used to changethe synaptic weights between neurons. The weight variation ∆w of a synapsefrom neuron a to neuron b depends on the relative spike timing (A = 0.1,τ = 20 ms).

C. Experimental Setup

We first performed experiments with minimal predictivenetworks consisting of 3 to 6 neurons that learn to predicttemporal sequences of stimuli. We then evaluated whether thislearning performance is scalable to larger random networks of100 neurons.

Figure 3 shows the basic network topology of the minimalpredictive networks used in the first series of experiments,where En represents excitatory neurons and I represents aninhibitory neuron. The network consisted of excitatory inputneurons and one inhibitory neuron. The input neurons were notconnected to each other, but all were connected to and fromthe inhibitory neuron. The number of input neurons was varied

from 3 to 5 across experiments. The initial weight values wbetween neurons were set to 15 (in arbitrary units).

E1E0 ・・・ En

random stimulationPart of stimulation sequence

I

Excitatory synapseInhibitory synapse

Fig. 3. Basic topology of a minimal predictive network. The network consistsof excitatory input neurons and one inhibitory neuron. E represents theexcitatory neurons and I represents the single inhibitory neuron. The numberof excitatory intput neurons was varied from 3 to 5 across the experiments.Neuron En receives random stimulus input as control, while others (Ei)receive part of the specific stimulation sequence. The excitatory neurons werenot connected to each other, but all projected an output to and received aninput from inhibitory neuron. The black arrows represent excitatory synapsesand the blue line with circles represent inhibitory synapses.

We also constructed larger networks to evaluate the scalabil-ity of the predictive networks. These larger networks consistedof 80 excitatory neurons and 20 inhibitory neurons, andthe network topology was random (i.e., neurons were fullyconnected with random weights). The weight values w wererandomly initialized between 0 and 5 (0 < w < 5) withuniform distributions for excitatory synapse and between -5and 0 (−5 < w < 0) with uniform distributions for inhibitorysynapse. Synaptic plasticity was applied to all connectionsexcept for connections between inhibitory neurons. There werethree input neuron groups (EG0–EG2), each consisting of 10excitatory neurons.

Three types of stimulus sequences were applied to thenetworks: a minimal pattern, a spatially extended pattern, anda temporally extended pattern (Fig. 4). The minimal patternconsisted of two stimuli, one signal stimulus followed byone target stimulus, where the signal stimulus was deliveredto a specific input neuron and after a fixed time delay thetarget stimulus was delivered to another specific input neuron(Fig. 4A). Therefore, the timing of the signal stimulus wasrandom (unpredictable) but the timing of the target stimuluswas predictable. For the large networks, the minimal patternconsisted of two groups of synchronous stimuli with eachstimulus group delivered to the corresponding input neurongroup.

In the spatially extended pattern, the sequence consistedof four stimuli, one signal stimulus and three subsequent

Fig. 4. Stimulation sequence patterns. The red squares represent the individualapplied stimuli within each stimulation sequence, while the green squaresrepresent random stimuli. A: Minimal pattern. The sequence consisted of twostimuli, an initial signal stimulus and subsequent target stimulus. B: Spatialextended pattern. The sequence consisted of four stimuli, one initial signalstimulus and three subsequent target stimuli. C: Temporal extended pattern.The sequence also consisted of four stimuli, one initial signal stimulus andthree successive target stimuli.

target stimuli. The signal stimulus was delivered to a specificinput neuron and after a fixed time delay the target stimuliwere delivered simultaneously to the three other specific inputneurons (Fig. 4B).

In the temporally extended pattern, the sequence also con-sisted of four stimuli, one signal stimulus and three subsequenttarget stimuli. The signal stimulus was delivered to a specificinput neuron and after a fixed time delay the first target stimu-lus was delivered to another specific input neuron followed bysuccessive target stimuli at fixed time interval to other neurons(Fig. 4C).

For every sequence pattern, each stimulus producess a 100mV depolarization in the minimal netowrk and a 10 mVdepolarization in the larger networks. The duration of eachstimulus was set to 1 ms, the interval between each stimulus inthe sequence to 10 ms (witch is sufficiently smaller than the 20ms working time window of STDP), and the interval betweeneach sequence to 300 ms (which is sufficiently larger thanthe working time window of STDP). In addition to sequentialstimulation, random stimulation was delivered into anotherspecific input neuron (or input neuron group in large networks)as the control.

Prediction is defined here as suppression of the inputneurons by the inhibitory neuron(s) at the time of stimulusinput. This decrease the influence of environmental stimulationon the network; thus, prediction can be regarded as stimulusavoidance.

III. RESULTS

A. Minimal Networks

We first examined predictive coding by the smallest minimalnetwork in response to the minimal stimulation pattern withoutsynaptic time delay. The network consisted of three excitatoryinput neurons and one inhibitory neuron. The input neuronswere not connected to each other, but all were connectedbidirectionally to the inhibitory neuron (Fig. 3). If the targetinputs are correctly predicted, spiking of input neurons shouldbe inhibited at the timing of stimulation. We examined thefiring rates of all input neurons and found that the firing ratesof neuron E1 receiving the target stimulus decreased, whereasthe firing rates of neuron E0 receiving the signal stimulus andE2 receiving the random stimulus did not change substantially(Fig. 5). Therefore, the network gradually learned to predictthe target stimuli (and exclude it) while the random stimuluswas not predicted.

Fig. 5. Time series of input neuron firing rates within small networks inresponse to the minimal pattern. The shaded regions represents the standarderror of the mean (n = 20 networks). The firing rate of neuron E1 receiving thetarget stimulus decreased substantially while the firing rates of E0 receivingthe signal stimulus and E2 receiving the random stimulus changed little.

Figures 6 and 7 show that the weight of the synapse fromthe neuron receiving signal stimulation (the signal neuronE0) to the inhibitory neuron increased with time, while theweight of the syanapse from the inhibitory neuron to the targetneuron (E1) decreased with time. The path from E0 to I toE1 is required to predict the timing of target stimuli at E1.In contrast, the weights to and from the random neuron E2changed little.

We then evaluated whether small networks could learn topredict more complex spatially extended stimulus patterns(Fig. 4B) and temporally extended stimulus patterns (Fig. 4C).We first applied the spatially extended pattern of stimulation tonetworks with two more additional input neurons compared tothe smallest minimal network shown in Fig. 3. Figure 8 showsthat the firing rates of neurons E1 to 3 receiving target stimuliin the spatial extended pattern decreased with time, whereasthe firing rates of neuron E0 receiving the signal stimulus and

Fig. 6. Time series of synaptic weight changes in small networks receivingthe minimal stimulation pattern. The shaded regions represent the standarderror of the mean (n = 20 networks). A: Weights from the inhibitory neuronto the excitatory neurons: E0, E1, and E2. B: Weight from the excitatoryneurons: E0, E1, and E2 to the inhibitory neuron.

E1E0 E2

I

-48.7

Excitatory synapseInhibitory synapse

Fig. 7. Final topology of the smallest network after stimulation with theminimal pattern. The pathway from E0 to the inhibitory neuron and from theinhibitory neuron to E1 was strengthened. This pathway is required to predicttarget stimuli at E1. The black arrows represent the excitatory synapsesand the blue connections represents inhibitory synapses. The weight valueof connections from the inhibitory neurons is negative.

E4 receiving the random stimulus were largely unchanged.Thus, these small networks gradually learned to predict thespatially extended target stimulus pattern, while the randomstimulus was not predicted. This implies that the inhibitoryneuron suppressed the firing of neurons E1–E3 at the time oftarget stimulation and that the network learned to predict thespatially extended pattern.

Fig. 8. Time series of input neuron (E0–E4) firing rates within smallnetworks in response to the spatially extended stimulation pattern. The shadedregions represent the standard error of the mean (n = 20 networks). Thefiring rates of E1–E3 receiving target stimuli in the spatially extendedpattern decreased remarkedly, while E0 receiving the signal stimulus andE4 receiving the random stimulus changed little.

On the other hand, these small network did not learn topredict the temporal extended pattern. Figure 9A shows thatthe only firing rate of neuron E1 decreased with time. Thisimplies that the network cannot learn to predict the longertemporal sequence than the minimal pattern consists of twostimuli, possibly, because there is only one inhibitory neuronand one connection to each excitatory neuron and longertemporal information cannot be encoded.

For neural networks to predict a longer temporal sequence,we hypothesized that they must have more inhibitory neuronsor more synapses between the input neurons and inhibitoryneurons with different synaptic delays to encode temporalinformation. Therefore, we constructed a small model with 15synapses between each input neuron and the inhibitory neuronand set different time delays (from 1 to 15 ms).

Figure 9B shows that the firing rates of neurons E1-E3receiving target stimuli in the temporal sequence decreasedwhereas the firing rates of E0 receiving the signal stimulus andE4 receiving the random stimulus changed little. Thus, thesenetworks gradually learned to predict the temporal sequence,while the random stimulus was not predicted. This implies thatthe inhibitory neuron suppressed spiking of neurons E1-3 atthe time of target input and that small networks with synapticdelay can learn to predict temporal sequences.

Fig. 9. Time series of the firing rate of the input neurons with the temporalextended pattern. The shaded regions represent standard errors of the mean (n= 20 networks). A: Without synaptic time delay. In this case, only the firingrate of E1 which got the first target stimuli decreased. This means that thenetwork without synaptic time delay cannot learn to predict longer temporalsequence. B: With synaptic time delay. In this case, the firing rate of E1-3which got the target stimuli of the temporal sequence decreased but E0 whichgot the signal stimulus and E4 which got the random stimulus did not changemuch. This means that the network with synaptic time delay learn to predictlonger temporal sequence.

B. Large Random Networks

We then examined whether the learning properties of theseminimal networks are scalable to large networks. We appliedthe minimal pattern of stimulation to random networks with100 neurons. Figure 10 shows that these larger networksgradually learned to predict the minimal pattern, while therandom stimulation was not predicted. The green line repre-sents the firing rate of hidden neurons, which can be regardedas the baseline firing rate of the network. The firing rate ofexcitatory neuron group 1 (EG1) receiving target stimuli grad-ually decreased to near baseline levels, indicating reasonableprediction accuracy.

Figure 11 shows a typical example of raster plot of spikesfor each neuron in the large network. In the first phase of theexperiment (first 3000 ms), almost all neurons of group EG1fired immediately in response to the target stimuli while in thelater phase (final 3000 ms), the firing rate was much lower andthe pattern was very similar to that of the hidden neurons. This

Fig. 10. Time series of the firing rates of the input neurons with the minimalpattern in the large networks (100 neurons). The shaded regions represent thestandard errors of the mean (20 networks). The firing rate of EG1 which gotthe target stimuli decreased but EG0 which got the signal stimuli and E2which got the random stimuli did not change much. The firing rate of hiddenneurons can be regarded as a baseline of the firing rate in the network.

implies that inhibitory neurons suppressed the firing of EG1neurons at the time of target stimulation, suggesting that largerandom networks can learn to predict the minimal pattern ofstimulation using only STDP.

Fig. 11. Raster plots of spikes in large networks. Each dot represents onespike: the red dots represent spikes of EG0 which got the signal stimuli, theblue dots represent spikes of EG1 which got the target stimuli, black dotsrepresent spikes of the other groups (EG2, Hidden and Inhibitory). A: Spikesin the first 3,000 ms. Almost all neurons in EG1 fired at the timing they gotthe target stimuli. B: Spikes in the last 3,000 ms. The neurons in EG1 didnot fire much at the timing of the stimulation and their firing patterns werealmost the same as the hidden neurons.

Moreover, the time series of synaptic weight changes be-tween neuron groups resembles those of the small networks(Fig. 12), suggesting that the mechanisms underlying predic-tion are similar.

Figure 13 shows the final topology of the network. Therewas a strong pathway from EG0 to inhibitory neurons andfrom inhibitory neurons to EG1; This pathway is requiredfor the prediction. In addition, there was a pathway fromEG0 to the hidden neurons, from the hidden neurons to theinhibitory neurons, and from the inhibitory neurons to EG1.

This pathway may be required to adjust the timing of EG1suppression.

Fig. 12. Time series of synaptic weight changes in large networks duringminimal pattern stimulation. A: Weights of connections from the inhibitoryneurons to the excitatory neuron groups EG0, EG1, EG2, and hiddenneurons. B: Weight from the excitatory neuron groups EG0, EG1, EG2,and hidden neurons to the inhibitory neurons.

Collectively, these findings demonstrate that the learningperformance of small minimal networks is scalable to largerrandom networks. Thus, spiking neural networks withoutspecific structure (random networks) can spontaneously learnto predict simple stimulus sequences based solely on STDP.

IV. DISCUSSION

We demonstrate that spiking neural networks can sponta-neously learn to predict input sequences from the environmentbased only on STDP. The firing of input neurons receivingtarget stimuli (those following signaling inputs) were predictedand suppressed by inhibitory neurons, thereby reducing theinfluence of environmental stimulation on network activity.In other words, neural networks can learn to avoid (ignore)specific input stimulation pattern through STDP.

We also demonstrate that this property is scalable fromsmall networks of a few neurons to larger random networksof 100 neurons without specific design structures or otherfunctions except STDP. These findings also suggest that likestimulation avoidance by action and selection in neuronalcultures [16], [11], prediction can emerge to avoid stimulation

Hidden

EG2EG0

Inhibitory

EG1

19.6

15.6

-15.

0

8.620.0

Excitatory synapseInhibitory synapse

Fig. 13. Final topology of the large network resulting from STDP. Aconnection between each group is depicted if the absolute mean weight valueis larger than 5.0. The thick lines represent weight values larger than 10.0and thin lines weight values lower than 10.0. The black arrows representexcitatory synapses and the blue lines inhibitory synapses. The weight valuesof connections from inhibitory neurons are negative.

in neuronal cultures via STDP, although this notion remainsto be confirmed.

Our previous studies showed that action and selection canemerge in spiking neural networks based on STDP and thatthese mechanisms can eliminate specific stimulus inputs [17].In this study, we found that stimulation avoidance by predic-tion also emerges in spiking neural networks under similarconditions. This findings implies that neural networks can es-tablish at least three mechanisms to eliminate (avoid) specificstimulation patterns based on STDP: action, prediction, andselection. We also speculate that these three mechanisms canemerge in the same neural network with STDP dependingon the quality of the stimuli. Specifically, controllable in-puts induce action, predictable inputs induce prediction (theremight be controllable and predictable input, and such inputinduce action or prediction) and uncontrollable inputs (noise)induce selection. In other words, distinct mechanisms canemerge for stimulus avoidance under specific environmentalconditions. We call this the principle of stimulus avoidance(PSA). In future work, we will test whether action, predictionand selection actually emerge in same network in silico andin vitro according to the PSA depending on differences instimulus input properties.

ACKNOWLEDGMENT

This work is partially supported by MEXT project “Study-ing a Brain Model based on Self-Simulation and Homeostasis”

in Grant-in-Aid for Scientific Research on Innovative Areas“Correspondence and Fusion of Artificial Intelligence andBrain Science” (19H04979)

REFERENCES

[1] A. M. Bastos, W. M. Usrey, R. A. Adams, G. R. Mangun, P. Fries,and K. J. Friston, “Canonical Microcircuits for Predictive Coding,” pp.695–711, 2012.

[2] A. Clark, “Whatever next? predictive brains, situated agents, and thefuture of cognitive science,” Behavioral and Brain Sciences, vol. 36,no. 3, pp. 181–204, 2013.

[3] R. P. N. Rao and D. H. Ballard, “Predictive coding in the visual cortex: Afunctional interpretation of some extra-classical receptive-field effects,”Nature Neuroscience, vol. 2, no. 1, pp. 79–87, 1999.

[4] Y. Huang and R. P. N. Rao, “Predictive coding,” Wiley InterdisciplinaryReviews: Cognitive Science, vol. 2, no. 5, pp. 580–593, 2011.

[5] K. Friston and S. Kiebel, “Predictive coding under the free-energyprinciple,” Philosophical Transactions of the Royal Society B: BiologicalSciences, vol. 364, no. 1521, pp. 1211–1221, 2009.

[6] W. Lotter, G. Kreiman, and D. Cox, “Deep predictive coding networksfor video prediction and unsupervised learning,” in 5th InternationalConference on Learning Representations, ICLR 2017, Toulon, France,April 24-26, 2017, Conference Track Proceedings, 2017.

[7] S. Song, K. D. Miller, and L. F. Abbott, “Competitive Hebbianlearning through spike-timing-dependent synaptic plasticity.” Natureneuroscience, vol. 3, no. 9, pp. 919–26, sep 2000.

[8] L. Sinapayen, A. Masumori, and T. Ikegami, “Learning by stimulationavoidance: A principle to control spiking neural networks dynamics,”PLOS ONE, vol. 12, no. 2, p. e0170388, feb 2017.

[9] A. Masumori, L. Sinapayen, and T. Ikegami, “Learning by stimulationavoidance scales to large neural networks,” in Proceedings of the 14thEuropean Conference on Artificial Life ECAL 2017. Cambridge, MA:MIT Press, sep 2017, pp. 275–282.

[10] A. Masumori, N. Maruyama, L. Sinapayen, T. Mita, U. Frey, D. Bakkum,H. Takahashi, and T. Ikegami, “Learning by Stimulation Avoidancein Cultured Neuronal Cells,” in The 2nd International Symposium onSwarm Behavior and Bio-Inspired Robotics (SWARM2017), 2017.

[11] A. Masumori, L. Sinapayen, N. Maruyama, T. Mita, D. Bakkum, U. Frey,H. Takahashi, and T. Ikegami, “Autonomous regulation of self and non-self by stimulation avoidance in embodied neural networks,” The 2018Conference on Artificial Life: A Hybrid of the European Conference onArtificial Life (ECAL) and the International Conference on the Synthesisand Simulation of Living Systems (ALIFE), pp. 163–170, 2018.

[12] D. V. Buonomano, “Decoding temporal information: A model based onshort-term synaptic plasticity,” Journal of Neuroscience, vol. 20, no. 3,pp. 1129–1141, 2000.

[13] R. P. N. Rao and T. J. Sejnowski, “Spike-timing-dependent hebbianplasticity as temporal difference learning,” Neural Comput., vol. 13,no. 10, pp. 2221–2237, Oct. 2001.

[14] C. Wacongne, J.-P. Changeux, and S. Dehaene, “A Neuronal Model ofPredictive Coding Accounting for the Mismatch Negativity,” Journal ofNeuroscience, vol. 32, no. 11, pp. 3665–3678, 2012.

[15] E. Izhikevich, “Simple model of spiking neurons,” IEEE Transactionson Neural Networks, vol. 14, no. 6, pp. 1569–1572, 2003.

[16] A. Masumori, N. Maruyama, L. Sinapayen, T. Mita, U. Frey, D. Bakkum,H. Takahashi, and T. Ikegami, “Emergence of Sense-Making Behaviorby the Stimulus Avoidance Principle: Experiments on a Robot BehaviorControlled by Cultured Neuronal Cells,” Proc. of the European Confer-ence on Artificial Life (ECAL) 2015, pp. 373–380, 2015.

[17] A. Masumori, L. Sinapayen, N. Maruyama, T. Mita, D. Bakkum, U. Frey,H. Takahashi, and T. Ikegami, “Neural Autopoiesis: Organizing Self-Boundary by Stimulus Avoidance in Biological and Artificial NeuralNetworks,” Artificial Life, vol. in press, 2019.