(dated: 23 november 2019)

9
Nonlinear resonances and multi-stability in simple neural circuits Leandro M. Alonso a) (Dated: 12 October 2021) We present an efficient numerical procedure to generate models of neural circuits which exhibit rich dynamical behavior. This is achieved by imposing that the circuits are close to many nonlinear resonances. The procedure is applied to generate circuits consisting of two interacting neural pop- ulations. When driven by external input, the resulting circuits present multiple stable patterns of periodic activity organized in complex tuning diagrams and signatures of low dimensional chaos. I. INTRODUCTION The brain is a multi-rhythmic system with spa- tiotemporal activity spanning several orders of mag- nitude which range from fast localized spiking at the single unit level to macroscopic oscillations that involve many units firing in synchronous patterns. Neural oscillations are believed to underlie a wide spectrum of brain functions (see Buszaki et al. for a review 1 ). The coordination of motor patterns uti- lized in animal locomotion and maintenance pro- cesses such as breathing are controlled in part by central pattern generators 2,3 . There is a growing view that rhythmic neural activity plays an ac- tive role in shaping neural processing and behav- ior by transiently binding cells into synchronized assemblies 4 . Different rhythms may shape the effec- tive connectivities of local circuits and alter the way information is processed and routed. Recently, ex- periments in rodents have shown that theta (8 Hz to 9 Hz) oscillations in the rats hippocampus encode the animals position in an arena 5 . The emerging view is that neuronal oscillators may provide a basis for a variety of neural functions including cognition 6 . Nonlinear resonances in periodically-driven neu- ral circuits have previously been studied both the- oretically and experimentally in several contexts. Thalamo-cortical interactions were studied using models of weakly connected oscillators in which com- munication between cortical columns is enabled by resonances 7 . Additional examples in the neuro- science literature include the amplification of gamma rhythms in inhibition-stabilized networks 8 . Re- cently, studies of periodic stimulation in models of spiking neurons have shown that intrinsic network oscillations can be shaped by changing the ampli- tude and frequency of the stimulation, suggesting possible implications for trans-cranial stimulation methods 9 . Periodic stimulation of nonlinear phys- ical and biological systems constitutes a traditional a) Electronic mail: [email protected] [email protected] approach to characterizing their intrinsic dynamical properties. This approach was used to investigate the intrinsic dynamics of a small network of elec- trically coupled neurons in the pyloric central pat- tern generator of the lobster 10 . More recently, it was shown that periodic stimulation of telencephalic nu- clei in songbirds creates subharmonic entrainments of the respiratory network 11 . The notion that chaotic dynamics might underlie neural dynamics has been explored by many authors. Two aspects of chaotic dynamics that are argued to be relevant for brain function are sparse exploration of the phase space and built-in multipurpose flexibil- ity. Even though the dimension of a chaotic attrac- tor is in general smaller than the allowed state space, chaotic motions explore a broad region of the be- haviors available to the system, providing a mean to seize opportunities upon environmental changes 12 . Neural systems are required for different purposes at different times and under different conditions. A chaotic system can accommodate this type of mul- tipurpose flexibility by switching the temporal pro- gramming of a small parametric perturbation in or- der to stabilize the different orbits embedded within the attractor 13 . More recently, Mindlin et al. ex- plored the possibility that the diversity of respira- tory motor gestures in birdsong can be partly ex- plained by nonlinear entrainments and bifurcations of a driven hierarchy of neural nuclei 14–19 . In this work we devise a procedure to tune mod- els of neural activity toward special regimes in which many dynamical behaviors are readily available. We consider simple circuits of interacting subpopula- tions of neurons which receive oscillatory input from elsewhere. The response of a circuit to a given stim- uli will vary depending on the parameters which define the architecture. Additionally, the response of a given circuit may be quite different as the fre- quency or the amplitude of the incoming stimuli are changed. We are interested in a hypothetical sce- nario in which the response of the circuit exhibits maximal diversity when receiving external stimuli within a given range. We show that this can be at- tained by asking that the circuits can be entrained to many different nonlinear resonances. The procedure arXiv:1609.02896v1 [nlin.CD] 9 Sep 2016

Upload: others

Post on 02-Feb-2022

1 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: (Dated: 23 November 2019)

Nonlinear resonances and multi-stability in simple neural circuitsLeandro M. Alonsoa)

(Dated: 12 October 2021)

We present an efficient numerical procedure to generate models of neural circuits which exhibit richdynamical behavior. This is achieved by imposing that the circuits are close to many nonlinearresonances. The procedure is applied to generate circuits consisting of two interacting neural pop-ulations. When driven by external input, the resulting circuits present multiple stable patterns ofperiodic activity organized in complex tuning diagrams and signatures of low dimensional chaos.

I. INTRODUCTION

The brain is a multi-rhythmic system with spa-tiotemporal activity spanning several orders of mag-nitude which range from fast localized spiking atthe single unit level to macroscopic oscillations thatinvolve many units firing in synchronous patterns.Neural oscillations are believed to underlie a widespectrum of brain functions (see Buszaki et al. for areview1). The coordination of motor patterns uti-lized in animal locomotion and maintenance pro-cesses such as breathing are controlled in part bycentral pattern generators2,3. There is a growingview that rhythmic neural activity plays an ac-tive role in shaping neural processing and behav-ior by transiently binding cells into synchronizedassemblies4. Different rhythms may shape the effec-tive connectivities of local circuits and alter the wayinformation is processed and routed. Recently, ex-periments in rodents have shown that theta (8 Hz to9 Hz) oscillations in the rats hippocampus encodethe animals position in an arena5. The emergingview is that neuronal oscillators may provide a basisfor a variety of neural functions including cognition6.

Nonlinear resonances in periodically-driven neu-ral circuits have previously been studied both the-oretically and experimentally in several contexts.Thalamo-cortical interactions were studied usingmodels of weakly connected oscillators in which com-munication between cortical columns is enabled byresonances7. Additional examples in the neuro-science literature include the amplification of gammarhythms in inhibition-stabilized networks8. Re-cently, studies of periodic stimulation in models ofspiking neurons have shown that intrinsic networkoscillations can be shaped by changing the ampli-tude and frequency of the stimulation, suggestingpossible implications for trans-cranial stimulationmethods9. Periodic stimulation of nonlinear phys-ical and biological systems constitutes a traditional

a)Electronic mail:[email protected]@gmail.com

approach to characterizing their intrinsic dynamicalproperties. This approach was used to investigatethe intrinsic dynamics of a small network of elec-trically coupled neurons in the pyloric central pat-tern generator of the lobster10. More recently, it wasshown that periodic stimulation of telencephalic nu-clei in songbirds creates subharmonic entrainmentsof the respiratory network11.

The notion that chaotic dynamics might underlieneural dynamics has been explored by many authors.Two aspects of chaotic dynamics that are argued tobe relevant for brain function are sparse explorationof the phase space and built-in multipurpose flexibil-ity. Even though the dimension of a chaotic attrac-tor is in general smaller than the allowed state space,chaotic motions explore a broad region of the be-haviors available to the system, providing a mean toseize opportunities upon environmental changes12.Neural systems are required for different purposesat different times and under different conditions. Achaotic system can accommodate this type of mul-tipurpose flexibility by switching the temporal pro-gramming of a small parametric perturbation in or-der to stabilize the different orbits embedded withinthe attractor13. More recently, Mindlin et al. ex-plored the possibility that the diversity of respira-tory motor gestures in birdsong can be partly ex-plained by nonlinear entrainments and bifurcationsof a driven hierarchy of neural nuclei14–19.

In this work we devise a procedure to tune mod-els of neural activity toward special regimes in whichmany dynamical behaviors are readily available. Weconsider simple circuits of interacting subpopula-tions of neurons which receive oscillatory input fromelsewhere. The response of a circuit to a given stim-uli will vary depending on the parameters whichdefine the architecture. Additionally, the responseof a given circuit may be quite different as the fre-quency or the amplitude of the incoming stimuli arechanged. We are interested in a hypothetical sce-nario in which the response of the circuit exhibitsmaximal diversity when receiving external stimuliwithin a given range. We show that this can be at-tained by asking that the circuits can be entrained tomany different nonlinear resonances. The procedure

arX

iv:1

609.

0289

6v1

[nl

in.C

D]

9 S

ep 2

016

Page 2: (Dated: 23 November 2019)

2

specifically consists of computing a very low resolu-tion approximation of the Arnold tongues diagramof the circuit over a fixed range of stimuli parame-ters. Each point in the range is assigned an integercorresponding to the period of the response (if it isperiodic) in units of the stimuli period. The result-ing diagrams are scored by counting how many sub-harmonic solutions occur in the inspected range andasking that there is an equal number of solutions foreach subharmonic type. We show that this approachyields simple circuits that accommodate this prop-erty. The resulting circuits exhibit patterns of pe-riodic activity with multiple timescales and diversewaveforms as well as complex non-periodic behavior.The procedure could be used to further explore linksbetween chaos theory and neural oscillations.

This work is organized as follows. In Section 2 wepresent a numerical procedure to generate circuitswhich respond sub-harmonically to a family of peri-odic stimuli. We also apply this procedure in a sim-ple setting of two interacting populations of neuronsdescribed in Section 2. Examples of such circuitsare presented in Section 3. Some final remarks andfuture directions are discussed in Section 4.

II. METHODS

A. Definition of locking period. Description of thealgorithm.

In this article we consider dynamical models ofneural activity which can be cast in the form of anordinary differential equation or vector field. Herewe introduce an objective function to drive the pa-rameters of these models towards a regime in whichthey can be nonlinearly entrained to an incoming pe-riodic signal. Let F (x, p) : Rm×q → Rm be a smoothvector field with state variables x ∈ Rm and param-eters p ∈ Rq. We further assume that this systemis driven by a family of periodic signals of period τ ,γα(t) parameterized by α. We consider the drivensystem,

x = F (x, p+ γα(t))

x(0) = x0. (1)

We are interested in finding parameters p suchthat the system will exhibit subharmonic entrain-ments with the driving signal γ, ie: that there is aninteger number of periods p ≥ 1 such that

x(t+ k(pτ)) = x(t) ∀k ∈ Z ∀t ∈ R. (2)

In order to test if a given solution is entrained tothe driving signal, we compute a Poincare section

of the flow by annotating the state of the systemx stroboscopically using the frequency of the driv-ing signal20,21. For this we take regularly spacedtimestamps every period of the forcing τ for a max-imum number of periods M . This yields a sequenceof states {x0, x1, ..., xM} which are used to computethe mismatch between the first state x0 and succes-sive states xn,

En =‖ xn − x0 ‖2 . (3)

Finally, we define the locking period P as the lowern such that En < ε,

P = minn{n : En < ε}, (4)

where ε is a parameter of the procedure whichdetermines the numerical accuracy utilized to deter-mine the mismatch between the state of the systemin the control sections. If there is no n that satisfiesEn < ε, we assign the value M + 1 indicating thatthe system does not entrain up to period M .

In order to compute the Poincare sections we in-tegrate the system using a Runge Kutta O4 methodusing dt = τ

10022. Before the Poincare section is com-

puted, the system is allowed to relax to the attractorfor a number of transient periods Mt. This providesa simple numerical way to check if the system is en-trained, but it may provide the wrong answer if thetransient decays to the attractors are too slow or ifthe numerical resolution parameter ε is set too high.An additional caveat is that a fixed point solutionwill be identified as a period 1 locking. This ambigu-ity was found to be unimportant for the purposes ofthis work. This definition provides a map betweenthe parameters of the driving signal α and an integernumber corresponding the period of the resulting so-lution. It is employed here to roughly quantify thediversity of dynamical responses available to a givenset of inputs. In the case of weakly interacting os-cillators these diagrams have specific shapes in thespace of frequencies and amplitudes which resemblea V. They are known as Arnold tongues after V.I.Arnold and they are the objects which inspired thisprocedure23.

B. Objective function

Next we introduce an objective function to drivethe parameters of the circuits towards nontrivial en-trainment regimes. In general, nonlinear systemswill exhibit complex resonances when driven withperiodic input. Small variations in the features ofthe input may result in different locking regimes

Page 3: (Dated: 23 November 2019)

3

which can be partially characterized by an integer.This maps the space of stimuli features into a set ofintegers which define the different entrainment re-gions. We specify the set of external stimuli by defin-ing a domain for the stimuli parameters α. Then wechoose a grid of N regularly spaced points in thisdomain and compute the locking period. We con-struct a histogram of locking periods by countinghow many solutions of each period were found andobtain a discrete distribution of locking periods {Lj}over the chosen domain. Since we are interested inobtaining circuits that are close to many resonances,we ask that the the distribution of locking periodsis flat. This is attained by minimizing an objectivefunction,

C(x0, p) =

M∑j=1

(LjN− 1

M)2. (5)

Evaluation of the objective function requires de-termining the locking period of a number of solu-tions. This is the computationally intensive part ofthe procedure. To summarize, the computationaleffort required to evaluate 5 scales linearly with thenumber of solutions to evaluate N and linearly withthe maximum number of periods M and with thetransient periods Mt. As we discuss in the next sec-tion, there are choices of these values for which eval-uation of this function is fast enough so that theproblem can be tackled by several heuristic opti-mization approaches.

C. The model

In this work we study the dynamics neural popu-lations using the celebrated Wilson-Cowan model24.This model and its extensions have been widely usedto model neural populations and it is a common ap-proach to address many problems in computationalneuroscience25. Here we consider the simplest caseof two populations of interconnected neurons of ex-citatory and inhibitory subtypes. The state vari-ables are a measure of the activity of each popula-tion. In the absence of stimulus, this system canpresent limit cycles and fixed-point behavior. How-ever, when driven by periodic input, the dynamicscan be extremely complex depending on the pre-cise weights of the connections and other parame-ters which define the architecture. Here we study afamily of circuits given by

1

τ1x1 = −x1 + S(C11x1 + C12x2 + ρ1 + γ(t)) (6)

1

τ2x2 = −x2 + S(C21x1 + C22x2 + ρ2),

where S is the sigmoid function,

S(x) =1

1 + e−x. (7)

.Here τi is the timescale of the population. Each

population receives input from the rest via the con-nectivities Cij and a constant input ρi. We assumethat an external input γ(t) is injected into popula-tion 1. We are interested in the possibility that thecircuits will respond in qualitatively different wayswhen driven by similar stimuli. Alternatively, theincoming signal can be thought of as produced byanother neural oscillator and therefore the full cir-cuit would present multiple stable patterns of peri-odic activity which can be switched by changing theweights of the connectivities or the offsets activities.We assume a particular family of periodic signals γwhich models an external neural oscillation definedby

γ(t) = ρ+AS(η(cos(ωt)− µ))). (8)

Here A is the amplitude, ω is the frequency andρ is a constant offset. The parameters η = 0.75 andµ = −1 are used to control the waveform of theoscillation and are kept fixed in this work.

III. RESULTS

In this section we discuss circuits of the form6 which were obtained by optimization of the ob-jective function 5. For visualization purposes, weconsider two scenarios independently. In the firstcase the driving signal γ can be modulated in am-plitude and frequency, α = (ω,A). In the secondcase the amplitude and constant offsets are allowedto vary, α = (ρ,A). We allow the connectivitiesto take a wide range of values Cij ∈ [−20, 20],ρj ∈ [−20, 20] and kept the timescales τi = 1 fixedfor simplicity. In order to evaluate the cost functionwe specified a domain for the stimulus parametersα = A ∈ [0, 10], ω ∈ [1− 0.2, 1 + 0.2], ρ ∈ [−5, 5].For each scenario, we took a regular grid of 10× 10values in the corresponding domains. We set themaximum number of locking periods M = 10 andthe numerical resolution parameter ε = 0.001. The

Page 4: (Dated: 23 November 2019)

4

solutions were allowed to decay to the attractors forMt = 10 periods. Evaluation of the cost functionthus entails determining the locking period for N =100 regularly spaced points in the corresponding αdomain up to period M = 10. For these parametersand in our current implementation we achieve about15 evaluations per second in a commercially avail-able linux server. This makes the problem tractableby many heuristic approaches which do not requireknowledge of the target function. The objectivefunction 5 is optimized by a genetic algorithm start-ing from 50 random seeds in the search domainwhich were evolved for 200 generations26. Each ofthe circuits presented were found in about 10 min-utes of computer work and the parameters are sum-marized in table I.

Figure 1 shows the locking diagrams correspond-ing to several evaluations of the objective function5. The locking period is computed in a grid and in-dicated in colors (color bar in Fig. 2). Values closeto 1 correspond to circuits which do not respondto the periodic stimuli or wherein the response istrivial, while lower values correspond to more color-ful diagrams for which several different entrainmentsoccur. The key realization is that the underlyingstructure of the locking regions is such that it canbe detected by sampling only a few points. The sys-tem can accommodate locking regions which consistof well-defined large blobs in the space of stimulusfeatures α. The problem of optimizing C is computa-tionally feasible because the structure of the lockingdiagrams can be partially characterized by samplingthe space of stimuli features with low resolutions.The main result of this article is that optimizationof this function is possible and yields a proceduralmechanism to generate neural circuits with rich dy-namical properties.

Figure 2 shows the dynamics of a circuit under pe-riodic forcing as the amplitude and the frequency ofthe stimuli are allowed to change. We are interestedin the case in which multiple timescales emerge outof the interaction of the populations. Figure (2A)shows the response of the circuit for different valuesof the amplitude. The system can be entrained to amultitude of oscillatory patterns with diverse tem-poral structure while the frequency of the stimuliremains fixed. Figure (2B) corresponds to the lock-ing period diagram over the inspected domain. Notethat while the structure of the diagram is complex,there are large connected regions for each locking pe-riod. These regions are in turn separated by smallernested structures of bands that exhibit beautiful pat-terns. In many cases, the waveform of the oscilla-tions is different for each region. The dashed line inFigure (2B) indicates the frequency of the stimulusused in (2A). This value was chosen so that several

different regions became available by changing onlythe amplitude.

The dynamics of the system in areas of the dia-grams in-between the larger regions is often associ-ated with strange attractors. Transitions from peri-odic to chaotic dynamics are known to exhibit uni-versal scaling rules and there are specific routes bywhich chaotic attractors can develop in low dimen-sional systems27,28. Figure (2C) shows the dynamicsof the circuit as the amplitude values are changed insmall decrements towards a transition between re-gions (A ∈ [2.705, 2.740]). The solutions are shownin full phase space (x1, x2, γ) in order to visualizechanges. As the amplitude is increased, the initialcurve seems to split in two copies of itself which failto intersect, in what appears to be a period dou-bling bifurcation. Further decrements of the ampli-tude result in successive folds of this mechanism un-til the leftmost attractor which is non-periodic andpossibly chaotic. Period doubling bifurcations andpossible chaotic behavior in neural models was pre-viously reported by Ermentrout29. The underlyingdynamical mechanisms by which these different os-cillations emerge is beyond the scope this work as itdepends on the specific details of each circuit. Whilein many cases the transitions from one region to an-other seem to correspond to local bifurcations, suchas period doubling, there are other mechanisms bywhich complicated dynamics may emerge. Despitethe complexity of the solutions there are a finitenumber of ways by which this can occur due to thelow dimensionality of the system. The mechanismsunderlying the generation of complex dynamics inlow dimensional systems can be partly characterizedand classified by topological analysis30,31.

This procedure can also be applied to the poten-tially more interesting scenario in which the stim-ulus frequency is fixed and only the connectivitiesare changed. This results in circuits which can beslowly modulated to produce patterns on multipletimescales while being driven at a unique fixed fre-quency. We obtained circuits with this property byallowing A and ρ to change and optimizing the ob-jective function 5. Figure 3 shows the locking dia-grams of one such circuit. The leftmost panel corre-sponds to the domain used in the optimization, whilethe successive panels zoom into the spiral. As be-fore, the diagrams exhibit beautiful nested patternswhich look self-similar. There are wide connectedregions which are associated with a particular lock-ing period, and typically, these regions are separatedby thin bands that correspond to different periods.We can think of an extended system by includingA = 0, ω = 0, ρ = 0 and then the diagrams can beinterpreted as different basins of attraction in thespace of initial conditions (A0, ω0, ρ0). From this

Page 5: (Dated: 23 November 2019)

5

FIG. 1. Evaluations of the objective function. The circuits are obtained by minimizing the objective function.A single evaluation of the objective function consists of calculating the locking period between the circuits responseand the stimulus in a 10×10 fixed grid of stimulus parameters (A,ω). This yields a map between stimulus parametersand integers and a distribution of locking periods over the computed domain. The objective function measures howfar this distribution is from being flat. (a) Distribution of locking periods over the computed domain (range shownin figure). (b) Map between stimulus parameters and locking period.Locking periods are color coded ranging fromperiod 1 (blue) to period higher than 10 or no locking (red). The color bar is shown in Fig. 2. (c) Increasing theresolution of (b) reveals the intricacies of the response diagrams.

point of view, the observed nested band structure isreminiscent of the Wada property for the basins ofattraction of multi-stable nonlinear systems32.

Finally, in order to allow for more biological real-ism in the resulting circuits, we performed the opti-mization in an extended domain including the pos-sibility that the time scales of the populations aredifferent. This additional degree of freedom resultsin lower values of the cost function and into the stun-ningly diverse locking diagrams shown in Figure 4.It is noteworthy that these type of models are ableto accommodate such remarkable properties despitebeing simple. This scenario might not be achiev-able in other systems for meaningful ranges of theirparameters. Neural models of the sort studied hereare remarkably flexible not only because they can beconnected in many different ways, but also becausethe intrinsic properties of the nodes can be different.

IV. CONCLUSIONS

We presented a procedure to tune the parametersof simple neural circuits towards regimes in which an

incoming periodic stimuli can result in many differ-ent dynamical behaviors. Simple architectures con-sisting of two populations of neurons can accommo-date such a scenario by nonlinearly entraining tothe incoming signals. These circuits can be foundby optimizing an objective function that measuresthe diversity of entrainment types by computing alow resolution approximation of the locking regionsof the circuits. While it is likely that a similar ap-proach would be applicable to other systems, it isunclear whether a given system may be able to ac-commodate many nonlinear resonances in ranges ofthe parameters that are physically and biologicallyplausible. Neural circuits of the sort discussed inthis article show a remarkable flexibility to accom-modate this type of dynamical behavior, even in thesimplest case of only two interacting populations.

Neural circuits that support an array of func-tions might be desirable in several contexts33. Theprocedure is useful to design multipurpose patterngenerators34. This in turn can be used as an en-coder. A long complex sequence of oscillatory com-mands can be encoded as different responses to sim-pler signals consisting of parametric modulations to

Page 6: (Dated: 23 November 2019)

6

FIG. 2. Emergence of multiple stable rhythms with complex organization. When neural circuits are closeto nonlinear resonances many dynamical behaviors become possible. The figure shows the response of a neural circuitconsisting of two interacting populations of neurons (x1, x2) receiving periodic input γ(t) as the amplitude of theinput is varied. (a) Responses of the circuit (red) as the amplitude of the stimulus (green) is increased. Both for lowand high values of A the system entrains 1 to 1 but the amplitude of the response is low. For intermediate values ofA, the system displays a multiplicity of stable periodic rhythms with complex waveforms. (b) Locking period diagramfor the considered domain of stimulus parameters A and ω. The locking period is color coded ranging from 1 (blue) to11 (red, no locking or higher than 10). The dashed line indicates the frequency of the stimulus for all shown solutions.The symbols indicate solutions shown in (a). There are large connected regions for any considered locking periodseparated by nested structures of bands. This structures are in turn associated with complex dynamics shown in (c).(c) Simulations for decreasing values of A near a transition between large regions plotted in phase space (A ≈ 2.705).As the amplitude is decreased the solutions undergo period doubling bifurcations and possibly low dimensional chaos.

the circuit35. Also, this procedure facilitates explor-ing the hypothesis that nonlinear resonances in neu-ral circuits may play a role in neural function. Itwould be interesting to investigate the dynamics oflarge scale models of weakly interacting neural cir-cuits when the parameters of each column are tunedclose to nonlinear resonances. From a dynamicalsystems perspective it would also be interesting tosee if circuits that exhibit these properties also shareother dynamical properties such as the bifurcations

diagrams of the non-driven system. Finally, the pro-cedure makes it easy to identify transitions betweenregimes in which universal scaling rules are expectedto arise. Therefore, the topological mechanisms bywhich complex dynamics emerge in simple neuralcircuits can be systematically investigated.

Page 7: (Dated: 23 November 2019)

7

FIG. 3. Locking period diagrams for modulations in amplitude and offset. Circuits can also be tuned closeto nonlinear resonance by considering modulations in other parameters (a). The diagrams show the locking period asthe amplitude A and the constant offset ρ of the stimuli are allowed to change. There are large regions correspondingto the same locking period which are separated by complex structures of nested bands. This is highlighted in thezoomed in diagrams (b,c,d) to the right. The rightmost panel is zooming to the point (A, ρ) = (8.010, 1.4965)±0.001.

FIG. 4. Stunning complexity of locking diagrams as more diversity is allowed in the circuits. Thediagrams correspond to circuits which were obtained by optimization of the objective function in a more generaldomain which includes the possibility of having different timescales for each population. The resulting circuits yieldbetter scores in the optimization function and they exhibit a remarkable diversity of behaviors which are availableby small modulations of the input.

Page 8: (Dated: 23 November 2019)

8

Location τ1 C11 C12 ρ1 τ2 C21 C22 ρ2

Fig 1.1 1.00 5.84 12.84 -16.00 1.00 -12.48 5.44 6.60

Fig 1.2 1.00 15.64 -14.32 -0.32 1.00 19.12 10.64 -8.36

Fig 1.3 1.00 1.08 -10.72 4.32 1.00 16.28 8.72 -18.96

Fig 1.4 1.00 15.56 -9.32 -4.80 1.00 13.08 14.00 -15.32

Fig 1.5 1.00 8.04 6.80 -10.00 1.00 -19.92 11.52 -3.28

Fig 2 1.00 4.92 -6.76 -3.00 1.00 14.96 18.76 -14.96

Fig 3 1.00 2.32 -17.32 8.52 1.00 15.16 16.44 -18.88

Fig 4.1.1 1.838 11.44 -8.76 -3.64 1.751 19.40 10.28 -7.12

Fig 4.1.2 1.8395 10.96 -12.00 -3.68 1.088 8.4 10.00 -6.84

Fig 4.1.3 1.7705 14.28 -9.72 -3.64 0.701 16.96 7.60 -4.92

Fig 4.2.1 1.9145 14.68 -9.72 -3.84 1.9085 11.32 10.56 -7.80

Fig 4.2.2 1.802 10.36 -9.44 0.56 0.545 16.60 7.24 5.00

Fig 4.2.3 1.9355 11.20 -8.24 -3.56 0.7745 12.88 19.28 -16.76

TABLE I. The parameters for all the circuits shown in this article are listed here. Circuits in Figure 1 are numbered1 through 5 from left to right. The circuits in Figure 4 are numbered according to their row and column numbers.

ACKNOWLEDGMENTS

Leandro M. Alonso’s research was supported byfunds from a Leon Levy Fellowship at The Rocke-feller University.

1Buzsaki, Gyorgy, and Andreas Draguhn. Neuronal oscil-lations in cortical networks. science 304, no. 5679 (2004):1926-1929.

2Golubitsky, Martin, Ian Stewart, Pietro-Luciano Buono,and J. J. Collins. Symmetry in locomotor central patterngenerators and animal gaits. Nature 401, no. 6754 (1999):693-695.

3Delcomyn, Fred. Neural basis of rhythmic behavior in ani-mals. Science 210, no. 4469 (1980): 492-498.

4Engel, Andreas K., Pascal Fries, and Wolf Singer. Dynamicpredictions: oscillations and synchrony in topdown process-ing. Nature Reviews Neuroscience 2, no. 10 (2001): 704-716.

5Agarwal, Gautam, Ian H. Stevenson, Antal Bernyi, KenjiMizuseki, Gyrgy Buzski, and Friedrich T. Sommer. ”Spa-tially distributed local fields in the hippocampus encode ratposition.” Science 344, no. 6184 (2014): 626-630.

6Buzsaki, Gyorgy. Rhythms of the Brain. Oxford UniversityPress, 2006.

7Hoppensteadt, Frank C., and Eugene M. Izhikevich.Thalamo-cortical interactions modeled by weakly connectedoscillators: could the brain use FM radio principles?.Biosystems 48, no. 1 (1998): 85-94.

8Veltz, Romain, and Terrence J. Sejnowski. Periodic forcingof inhibition-stabilized networks: Nonlinear resonances andphase-amplitude coupling. Neural computation (2015).

9Herrmann, Christoph S., Micah M. Murray, Silvio Ionta,Axel Hutt, and Jrmie Lefebvre. Shaping Intrinsic NeuralOscillations with Periodic Stimulation. The Journal of Neu-roscience 36, no. 19 (2016): 5328-5337. Harvard

10Szucs, Attila, Robert C. Elson, Michail I. Rabinovich,Henry DI Abarbanel, and Allen I. Selverston. Nonlinearbehavior of sinusoidally forced pyloric pacemaker neurons.Journal of Neurophysiology 85, no. 4 (2001): 1623-1638.

11Mendez, Jorge M., Gabriel B. Mindlin, and Franz Goller.Interaction between telencephalic signals and respiratorydynamics in songbirds. Journal of neurophysiology 107, no.11 (2012): 2971-2983.

12Rabinovich, M. I., and H. D. I. Abarbanel. The role of chaosin neural systems. Neuroscience 87, no. 1 (1998): 5-14.

13Ott, Edward, Celso Grebogi, and James A. Yorke. Control-ling chaos. Physical review letters 64.11 (1990): 1196.

14Trevisan, Marcos A., Gabriel B. Mindlin, and Franz Goller.Nonlinear model predicts diverse respiratory patterns ofbirdsong. Physical review letters 96, no. 5 (2006): 058103.

15Granada, A., M. Gabitto, G. Garcıa, J. Alliende, J. Mndez,M. A. Trevisan, and G. B. Mindlin. The generation of res-piratory rhythms in birds. Physica A: Statistical Mechanicsand its Applications 371, no. 1 (2006): 84-87.

16Arneodo, Ezequiel M., Leandro M. Alonso, Jorge A. Al-liende, and Gabriel B. Mindlin. The dynamical origin ofphysiological instructions used in birdsong production. Pra-mana 70, no. 6 (2008): 1077-1085.

17Alonso, Leandro M., Jorge A. Alliende, Franz Goller, andGabriel B. Mindlin. Low-dimensional dynamical model forthe diversity of pressure patterns used in canary song.Physical Review E 79, no. 4 (2009): 041929.

18Goldin, Matas A., and Gabriel B. Mindlin. Evidence andcontrol of bifurcations in a respiratory system. Chaos: AnInterdisciplinary Journal of Nonlinear Science 23, no. 4(2013): 043138.

19Alonso, Rodrigo G., Marcos A. Trevisan, Ana Amador,Franz Goller, and Gabriel B. Mindlin. A circular model forsong motor control in Serinus canaria. Frontiers in compu-tational neuroscience 9 (2015).

20Guckenheimer, John, and Philip Holmes. Nonlinear oscilla-tions, dynamical systems, and bifurcations of vector fields.Vol. 42. Springer Verlag: New York, 1983.

21Wiggins, Stephen. Introduction to applied nonlinear dy-namical systems and chaos. Vol. 2. Springer Science & Busi-ness Media, 2003.

22Press, William H. Numerical recipes 3rd edition: The artof scientific computing. Cambridge university press, 2007.

23Arnold, Vladimir. I. Geometrical Methods in the The-ory of Ordinary Differential Equations (Grundlehren dermathematischen Wissenschaften). Fundamental Principlesof Mathematical Science.Springer, Verlag, New York 250(1983).

24Wilson, Hugh R., and Jack D. Cowan. Excitatory and in-hibitory interactions in localized populations of model neu-rons. Biophysical journal 12, no. 1 (1972): 1-24.

Page 9: (Dated: 23 November 2019)

9

25Destexhe, Alain, and Terrence J. Sejnowski. The Wilson-Cowan model, 36 years later. Biological cybernetics 101,no. 1 (2009): 1-2.

26Holland, John H. Genetic algorithms. Scientific american267, no. 1 (1992): 66-72.

27Feigenbaum, Mitchell J. Quantitative universality for aclass of nonlinear transformations. Journal of statisticalphysics 19.1 (1978): 25-52.

28Libchaber, A., C. Laroche, and S. Fauve. Period doublingcascade in mercury, a quantitative measurement. Journalde Physique Lettres 43.7 (1982): 211-216.

29Ermentrout, G. Bard. Period doublings and possible chaosin neural models. SIAM Journal on Applied Mathematics44, no. 1 (1984): 80-95.

30Gilmore, Robert, and Marc Lefranc. The topology of chaos:Alice in stretch and squeezeland. John Wiley & Sons,(2008).

31Mindlin, Gabriel B., Xin-Jun Hou, Hernn G. Solari, R.Gilmore, and N. B. Tufillaro. Classification of strange at-tractors by integers. Physical review letters 64, no. 20(1990): 2350.

32Kennedy, Judy, and James A. Yorke. Basins of Wada.Physica D: Nonlinear Phenomena 51, no. 1-3 (1991): 213-225.

33Hoppenstead F. C. (Editor). Nonlinear oscillations in biol-ogy. Lectures in Applied Mathematics Volume 17. AmericanMathematical Society, (1979).

34Kopell, Nancy, and G. Bard Ermentrout. Coupled oscilla-tors and the design of central pattern generators. Mathe-matical biosciences 90, no. 1 (1988): 87-109.

35Alonso, Leandro M. Parameter estimation, nonlinearity,and Occam’s razor. Chaos: An Interdisciplinary Journal ofNonlinear Science 25, no. 3 (2015): 033104.