Superconducting Optoelectronic Neurons III: Synaptic Plasticity

Jeffrey M. Shainline, Adam N. McCaughan, Sonia M. Buckley, Christine A. Donnelly,

Manuel Castellanos-Beltran, Michael L. Schneider, Richard P. Mirin, and Sae Woo NamNational Institute of Standards and Technology, 325 Broadway, Boulder, CO, 80305

(Dated: July 5, 2018)

As a means of dynamically reconfiguring the synaptic weight of a superconducting optoelectronicloop neuron, a superconducting flux storage loop is inductively coupled to the synaptic current biasof the neuron. A standard flux memory cell is used to achieve a binary synapse, and loops capableof storing many flux quanta are used to enact multi-stable synapses. Circuits are designed to im-plement supervised learning wherein current pulses add or remove flux from the loop to strengthenor weaken the synaptic weight. Designs are presented for circuits with hundreds of intermediatesynaptic weights between minimum and maximum strengths. Circuits for implementing unsuper-vised learning are modeled using two photons to strengthen and two photons to weaken the synapticweight via Hebbian and anti-Hebbian learning rules, and techniques are proposed to control thelearning rate. Implementation of short-term plasticity, homeostatic plasticity, and metaplasticity inloop neurons is discussed.

I. INTRODUCTION

Information processing systems with differentiatedprocessing, information integration, and distributedmemory modeled after biological neural systems are ap-pealing as tools for understanding neural and nonlineardynamical systems as well as for computation in contextsrequiring complex contextualization and dynamic learn-ing. In such neural systems [1, 2], the synaptic weightsbetween nodes in the network are crucial memory ele-ments that affect dynamics and computation [3–6]. Forsome applications, it is important to have a means bywhich a user can interface with the system to externallycontrol the synaptic weights to implement learning algo-rithms [7]. In other applications, it is desirable for thesynaptic weights to dynamically update based on net-work activity in an unsupervised manner. It is beneficialfor a hardware platform to be capable of both.

It has been proposed [8, 9] that combining thestrengths of light for communication and superconduct-ing electronics for efficient computation offers a routeto large-scale neural systems. A circuit that transducessingle-photon communication signals to integrated super-current has been described in Ref. 10. In that reference,a means to modify the synaptic weight via a bias current(Isy) was identified. The present work explores circuitsthat dynamically control Isy.

The circuits implemented to control Isy should meetseveral criteria: 1) Transition between the minimum andmaximum values of Isy should be possible with a specifiednumber of increments to control the learning rate; 2) Thecircuit should not be able to set Isy outside of this rangeso that simple update rules or training algorithms do notresult in excessively large synaptic weights; 3) It shouldbe possible to cycle the value of Isy from minimum tomaximum and back repeatedly without degradation; 4)In addition to a means by which the synaptic weights canbe incremented by an external supervisor, there shouldbe a means by which correlated photon signals from thetwo neurons associated with a synapse can strengthen

or weaken the synaptic weight depending on the rela-tive arrival times of the signals from the two neurons;5) Within this unsupervised mode of operation, synapticupdate events should be induced by single-photon signalsto fully exploit the energy efficiency of the superconduct-ing optoelectronic hardware; 6) The transition probabil-ity between synaptic states should also be dynamicallyadjustable based on photonic signals to achieve meta-plastic behavior. This paper explores circuit designs sat-isfying all these criteria.

A schematic of the neuron under consideration isshown in Fig. 1(a). Operation is as follows. Photons fromafferent neurons are received by single-photon detectors(SPDs) [11–15] at a neuron’s synapses. Using Josephsonjunctions (JJs) [16–18], these detection events are con-verted into an integrated supercurrent that is stored in asuperconducting loop. The amount of current added tothe integration loop during a synaptic photon detectionevent is determined by the synaptic weight. The synapticweight is dynamically adjusted by another circuit com-bining SPDs and JJs. When the integrated current fromall the synapses of a given neuron reaches a threshold, anamplification cascade begins, resulting in the productionof light from a waveguide-integrated LED. The photonsthus produced fan out through a network of passive di-electric waveguides and arrive at the synaptic terminalsof other neurons where the process repeats.

The dashed box in Fig. 1(a) encloses the synapticweight control circuits that are the focus of this work.These signals add or remove flux from a storage loop,which is inductively coupled to the current bias line, Isy.This loop is referred to as the synaptic storage (SS) loop(Fig. 2), and the flux stored in this loop functions as thememory for the synapse.

The circuits described in this work modify Isy in ei-ther a supervised manner using JJs or unsupervised man-ner using SPDs in conjunction with JJs. Qualitativeexplanation of the memory update process in shown inFig. 1(b) for supervised learning and in Fig. 1(c) for un-supervised learning. For the simplest binary synapse, a

arX

iv:1

805.

0193

7v4

[cs

.NE

] 3

Jul

201

8

2

time

(ii)

synap

tic

bia

sdrive

(i)

DC-t

o-SFQ

flux

stor

age

time

(ii)

synap

tic

bia

s

(i)

drive

phot

onse

quen

cede

tect

or

flux

stor

age

+-

(b) (c)

(a) thresholdlearning rate

gain

Se

Si

W

W

T

W

S S

FIG. 1. (a) Schematic of the neuron showing excitatory (Se)and inhibitory synapses (Si) connected to an integration loopwith a variable threshold. The wavy, colored arrows are pho-tons, and the straight, black arrows are electrical signals. SeeRef. 10 for details. The dashed rectangle labels the synapticupdate box (W), which is the focus of this work. The numberof fluxons is determined by the synaptic bias current, whichis controlled by W. The photons from the left and right ac-complish spike-timing-dependent plasticity. The inset at thelower right depicts the electrical signals used for supervisedlearning. (b) Synaptic update in supervised mode. Squarepulses add or remove fluxons from a loop, which strengthensor weakens the synaptic weight. (c) Synaptic update in unsu-pervised mode. Photons from pre-synaptic and post-synapticneurons are used to change the flux in the loop.

flux-quantum memory cell can be used to switch betweenthe strong and weak synaptic states in 50 ps. This binarydesign can be extended to a multi-stable synapse thatcan modify the synaptic weight between the fully poten-tiated and fully depressed states with hundreds of stableintermediate levels, and implementations with more orless resolution are straightforward to achieve. For unsu-pervised learning, we consider a circuit that can imple-ment a Hebbian learning rule that potentiates a synap-tic connection using one photon from the pre-synapticneuron and one photon from the post-synaptic neuron.We generalize this circuit to implement full spike-timing-dependent plasticity (STDP) wherein a synaptic weightcan be either potentiated or depressed based on Hebbianand anti-Hebbian timing correlations. This STDP circuituses single-photon signals at four ports. Implementa-tions of short-term plasticity, homeostatic plasticity, andmetaplasticity are discussed in the Appendices. Combin-ing these synapse designs, it is possible to realize neuronswith a distribution of synapses that update at differentrates as well as ensembles of neurons wherein differentneurons store information about different stimuli learnedat different times, thus achieving a network with rapidadaptability and long memory retention times necessaryfor cognition, as discussed in Ref. 9.

II. CONCEPTUAL OVERVIEW

The field of neural computing [19, 20] is broad anddeep. A rich body of work exists wherein neural conceptsare implemented in software using conventional Booleanhardware [21–24]. Such technologies are usually referredto as neural networks. We make the distinction thatneural computing utilizes hardware with neural behav-ior present in the physics of the devices that implementthe required computations rather than implementing theneural computational functions with algorithms in soft-ware. Software neural nets and neural hardware are bothuseful and both have promise to affect the advanced com-puting landscape in coming years.

We further delineate two main modes of operation ofneural computers. In one mode, controlled inputs arepresented to the system, and the system provides anoutput. The output from the system is compared to adesired output, and an error is calculated based on acost function. This error is then used to update the con-figuration of the system, often through backpropagation[7]. We refer to this mode of operation as “supervisedlearning”. Most technologies commonly referred to asmachine learning or deep learning operate in this mode.The objective of supervised learning is often to train thehardware to perform a specific task [25].

For larger neural systems performing general cognitivefunctions, it is advantageous to operate in an unsuper-vised manner. In supervised learning, there is an externalmeans by which properties affecting network operationcan be adjusted (such as by explicitly changing synaptic

3

weights or neuron thresholds). In unsupervised learn-ing, no such external control is available. Unsupervisedlearning is scalable in that the user is not required to cal-culate or adjust the network parameters, so systems withmany more degrees of freedom can be realized. Yet un-supervised learning requires that internal activity of thenetwork be capable of adjusting the degrees of freedomto form a useful representation of the information it isexpected to process. Unsupervised learning usually oc-curs within spiking dynamical systems. In these dynami-cal systems, modification of the synaptic weights changesthe structure of the network and therefore adapts the dy-namical state space [26, 27] based on external stimulusand internal network activity.

In the present work, we are interested in both super-vised and unsupervised modes of operation, and we focuson the means by which synaptic weights can be modi-fied either externally or internally to enable training andlearning. In unsupervised learning, we are interested insystems that will interact continuously with their envi-ronment, be capable of immediately assimilating new in-formation, and also capable of remembering events aslong as possible. Such competing memory demands aresometimes referred to as the adaptability-precision trade-off [28], and the best-performing synapses in this regardare complex [29] and may have many stable levels [30].In human subjects, memories have been observed to fadewith a power law temporal dependence [31, 32]. It is dif-ficult to do better than power law forgetting with plasticsynapses that continually adapt [29], and simple synapseslose their memory trace most quickly [30]. In the presentwork, we show synapses with a number of stable statesranging from two to hundreds. These synapses havedynamically variable memory update rates, making thesynapses suitable for power law memory retention.

The neurons under consideration have been describedin Ref. 10, and they are referred to as loop neurons.They will be employed in the context of superconduct-ing optoelectronic networks described in Refs. 8 and 9.In such networks, light is used to communicate signalsbetween neurons, and when a single photon is receivedat a synapse, the signal is converted to a number of fluxquanta [16–18]. We refer to this as a synaptic firing event.In such a neuron, the role of the synaptic weight is tochange how many flux quanta are generated during asynaptic firing event. This determines how much currentis added to the neuron’s current integration loop [10], andtherefore how close the neuron is to reaching threshold.When one or more synaptic firing events add sufficientcurrent to the neuron’s integration loop to reach thresh-old, the neuron is induced to produce a pulse of light,which is distributed to that neuron’s downstream con-nections. Threshold detection and pulse production aretreated in Ref. 33.

The objective of this paper is to describe the means bywhich the synaptic weight can be modified to enable dy-namically reconfigurable synapses. The photon-to-fluxontransduction that occurs during a synaptic firing event is

implemented with an SPD in parallel with a JJ, as de-scribed in Ref. 10. To change the number of fluxons gen-erated during the synaptic firing event, one can simplychange the current bias across this JJ, referred to as thesynaptic firing junction. The circuits presented here aredesigned to dynamically modify the current bias to thesynaptic firing junction, Isy (see Fig. 2 of Ref. 10). Werefer to the circuits that modify Isy as the synaptic up-date circuits. In general, there will be a chosen weakestsynaptic strength and strongest synaptic strength at eachsynapse, and in general the weakest synaptic strengthwill be achieved with Imin

sy > 0. Thus, it is the goal ofthe synaptic update circuit to vary Isy over some rangeIminsy ≤ Isy ≤ Imax

sy . In certain contexts it is sufficient forIsy to only be able to take two values [20], while in otherlearning environments it may be advantageous to be ableto achieve many values of Isy between Imin

sy and Imaxsy .

Discussion of synapses for learning in various contexts ispresented in Sec. VI.

One means of modifying the current bias to the synap-tic firing junction is depicted in Fig. 2. For systems withmany neurons each with many synapses, we would liketo use a single current source to establish the baselinesynaptic bias to all synapses (I1 in Fig. 2), keeping inmind that we may need the baseline synaptic bias to bedifferent for different synapses. This can be achieved byusing a single current bias, I1, and using mutual induc-tors to couple this current to each synapse. The synapticfiring circuit is thus biased by a superconducting loop,referred to as the synaptic bias (SB) loop, and the ob-jective of the synaptic update circuit is to change thecurrent in the SB loop, also through mutual inductors.The circuits presented throughout this work achieve thevarious synaptic states by changing the amount of fluxtrapped in another superconducting loop, referred to asthe synaptic storage (SS) loop. This basic concept isshown in Fig. 2, where the SB loop is coupled to boththe main bias, I1, and the dynamic synaptic bias basedon the flux trapped in the SS loop. All circuits presentedin the remainder of this work provide a means to adjustthe flux stored in the SS loop.

To implement supervised learning, we would like tocontrol the flux stored in the SS loop using simple con-trol signals, which we take to be square current pulses.In Sec. III we show that such current pulses can be usedto modify the flux in the SS loop and therefore Isy to im-plement binary synapses as well as multistable synapseswith many hundreds of levels between Imin

sy and Imaxsy .

To implement unsupervised learning, it is necessary forneuron-generated signals to be capable of modifying theflux in the SS loop. In Secs. IV and V we show how pho-tonic signals can be used to change the state of flux in theSS loop and therefore implement learning rules based ontiming correlations between the two neurons associatedwith each synapse.

4

I+

SS SB

I1

Iss

Iss

b2

b1

Isy

I-

Jsu

Jsu

Lss

S

FIG. 2. Fluxon memory cell used to achieve binary synapse.The box labeled S is the synapse receiving the bias current(see Fig. 1). Circuit parameters are listed in Appendix A.

III. SUPERVISED LEARNING

Several quantities determine the behavior of a synapse.These include the minimum and maximum values of thesynaptic weight and the number of increments betweenthe two. For many applications in machine learning, neu-ral networks, and neuroscience, synapses are treated asbinary elements that can switch between strong (poten-tiated) and weak (depressed) states [20, 30, 34]. Memorystorage times can be improved if synapses have a largenumber of stable states between maximally potentiatedand depressed states [30]. In Ref. 10 it was shown that,in a superconducting optoelectronic neuron, changing Isyfrom 1 µA to 3 µA changes the contribution to the neu-ron’s integrated signal by a factor of 15. In this section,we demonstrate that the synaptic current can be changedover this range by adding anywhere from one to hundredsof fluxons to the SS loop, thereby achieving a range ofsynapses from a simple binary synapse to a multistablesynapse with a pseudocontinuum of stable values.

The circuit for enacting a binary synapse is shown inFig. 2. This circuit is a standard flux-quantum memorycell [17, 18] coupled to the SB loop via a mutual induc-tor [35]. The current delivered to the synapse, Isy, isshifted by a static value determined by the bias I1 andthe mutual inductors shown in Fig. 2. When there areno fluxons in the SS loop, Isy = 1 µA, the minimumvalue. In this state, the bias currents (Ib1ss and Ib2ss )are chosen such that a weakening synaptic update sig-nal (I−) cannot add a fluxon to the loop, so the synapticweight cannot be further depressed. A strengthening sig-nal can, however, switch Jsu and add one fluxon to theloop. This transitions the circuit to the potentiated state,wherein Isy = 3 µA. At this point, further potentiatingsignals cannot add additional flux to the loop. The loopcan store only a single fluxon, and it is characterizedby βL/2π = LIc/Φ0 = 1.8 [17, 18]. The junction andcircuit parameters are given in Appendix A. All param-eters are typical for superconducting electronic circuitsand straightforward to realize in hardware.

I+/- [µA

]

Time [ns]0 1

0

10

Isy [µA

]

1

3

I+ I-

∆t [ps]

Isy [µA

]

1

3

0

0.4

0 100

Vss [m

V]

0

10

I+ /

I- [µ

A]

Time [ns]

1

3

Isy [µA

]

0 100

(a)

(b)

(c)

3.33 µA

0.71 µA0.93 µA

1.03 µA

3.17 µA3.05 µA

FIG. 3. Operation of binary synapse. (a) Synaptic bias, Isy,as a function of time while potentiating and depressing drivesignals are applied. The red and green traces are the drivesignals across the two JJs, referenced to the left y axis. Theblue trace is Isy, referenced to the right y axis. (b) The op-eration of the storage loop driven with 50 ps switching time.(c) Temporal zoom of the data in (b).

Figure 3 shows WRSpice [36] simulations of the tempo-ral behavior of the circuit as it switches between states.In Fig. 3(a), the circuit is initially in the depressed state.A pulse of 10 µA drives the circuit to the potentiatedstate. Repeated current pulses do not switch the state,and after the input pulses cease, the cell holds the valueof Isy. Upon the application of a single 10 µA pulse intothe weakening port, the circuit switches back to the de-pressed state, and repeated applications of this signal donot further switch the circuit.

In Fig. 3(b) we show the synapse switching betweenthe depressed and potentiated states every 50 ps. Thetime scales of Fig. 3 are extremely fast compared to bio-logical neural circuits. The speed of these circuits offers

5

I+

SS

SB

I1

Isy

I-

S

FIG. 4. Diagram of the synaptic update circuit used forsupervised learning with multiple stable synaptic values. DC-to-SFQ converters (dashed box) add fluxons to the synapticstorage loop to increase or decrease the synaptic bias currentapplied to the synaptic firing circuit [10]. Values of the circuitparameters are listed in Appendix A.

intriguing possibilities, as discussed in Sec. VI. Figure3(c) shows a temporal zoom of a full cycle of the binarysynapse occurring within 50 ps.

For deep learning in neural networks, it is often nec-essary to increment the synaptic weights in small steps.To achieve fine weight update, a superconducting loopcapable of storing more than one flux quantum is uti-lized, as shown in Fig. 4. Flux quanta can be added oneby one using DC-to-SFQ converters [17, 18]. The binarysynapse of Figs. 2 and 3 has been modified to includetwo DC-to-SFQ converters: one for potentiating and onefor depressing. When a fluxon is produced by the po-tentiating DC-to-SFQ converter by the introduction ofa current pulse, I+, the fluxon is added to the SS loop.When a fluxon is produced by the depressing DC-to-SFQconverter by the introduction of a current pulse, I−, thefluxon counter propagates in the SS loop. The inductorsof the SS loop, Lss and Mss can be chosen over a broadrange of values to determine the learning rate and rangeof synaptic weights achieved.

Controlled increase of synaptic bias current is againdemonstrated using WRSpice [36]. The results are shownin Fig. 5. In this calculation, a periodic square wavedrives the DC-to-SFQ converter with 10 µA pulses of 1 nsduration and 2 ns period. Current is added to the SS loopin fluxon increments over many input cycles (Fig. 5(a)).In this case, the value of Isy before any flux has beenadded to the SS loop is 2 µA, chosen to be in the mid-dle of the operational range identified in Ref. 10. Forthis calculation, the inductance of the SS loop is 200 nH(βL/2π = LIc/Φ0 = 3.8 × 103), leading to the additionof 2.5 nA to Isy with the addition of each fluxon to the

0 1 2 3

1

3

Time [µs]

I sy [µA

]

3.2 µA 10 µA

0.8 µA

0.505 0.515

0

1

Time [µs]

∆I sy

[10

nA] 0 1

2

3

Time [µs]

I sy [µA

]

(a)

(b)

∆I = 2.5 nA∆t = 2 ns

FIG. 5. Operation of the synaptic update circuit for super-vised learning. (a) Synaptic current bias, Isy, is shown by theblue trace, referenced to the left y axis. The drive signal, I+

is shown without reference to an axis. The square wave hasa 10 µA amplitude and 2 ns period. The main panel showsdetail of the operation on a short time scale, and the insetshows a ramp from the middle synaptic weight until satura-tion at the maximum synaptic weight. In this calculation,Lss = 200 nH, ∆Iss = 10.3 nA per pulse, and ∆Isy = 2.5 nAper pulse. (b) Schematic of the circuit used to strengthen aswell as weaken synaptic weight. (c) Operation of the circuit asthe synaptic weight is repeatedly ramped between minimumand maximum values.

loop. This value of inductance (and therefore ∆Isy) canbe chosen over a broad range to set the synaptic updateincrement and number of synaptic levels. This value waschosen to create a SS loop that can store over 1000 flux-ons between the minimum and maximum values of Isy.The effects of the number of stable synaptic levels willbe discussed further in Sec. VI.

The inset of Fig. 5(a) shows the behavior of Isy as afunction of time as it is potentiated to saturation. Afluxon is added to the loop every two nanoseconds. Afterapproximately 500 fluxons have been added to the loop,the value of Isy saturates just above 3 µA. This saturationbehavior is advantageous so that a learning algorithmcannot cause a synaptic weight to grow without bound.

Figure 5(b) shows Isy as a function of time as the po-tentiating and depressing DC-to-SFQ converters are al-

6

ternately employed, analogous to the two drives of thebinary synapse in Fig. 2. For these calculations, an SSloop with 20 nH inductance was considered to reduce thetime required to achieve saturation. Initially, Isy = 2 µA.Fluxons are added to the SS loop for 200 ns, and Isyreaches its maximum value of 3.2 µA. Figure 5(b) showsthat while the synaptic strengthening drive (I+) is on,once the SS loop reaches saturation, the value of Isy can-not be increased. The figure further shows that after thesynaptic strengthening drive is turned off, Isy maintainsits value (i.e., during the time from 200 ns - 250 ns). After250 ns, fluxons of the opposite sign begin to be added tothe SS loop via the synaptic weakening drive (I−), andIsy can be driven down to the minimum value (800 nAin this case). Cycling these drives results in the periodicbehavior seen in Fig. 5(b). It can be seen that duringeach strengthening and weakening cycle Isy versus timehas two regions with different slopes. This is due to thefact that when the current in the SS loop is outside a cer-tain range, the DC-to-SFQ converter releases two fluxonsper drive cycle. This characteristic is likely of little con-sequence and may be eliminated with improved circuitdesign, possibly by separating the DC-to-SFQ converterfrom the SS loop with a JTL.

The circuits of Figs. 2 and 4 have several strengthswhen used to establish the synaptic weight of a supercon-ducting optoelectronic neuron. The nature of the flux-storage Josephson circuits enables cycling and modifyingthe synaptic weights as many times as necessary withoutmaterial degradation. The maximum and minimum val-ues of Isy can be designed to achieve a broad range ofoperating conditions. Upon reaching the maximum andminimum values, the device saturates, eliminating thepossibility of runaway values of synaptic weight. Synap-tic update can be carried out in a specified number ofincrements based on the choice of inductance of the SSloop. The size of these increments will determine thelearning/forgetting rate of the synapse.

While these characteristics of the circuits are conduciveto implementing a variety of training algorithms basedon back propagation [7] or in conjunction with designthrough genetic evolution [37, 38], we would also like toenable systems that learn using only activity within thenetwork. We next consider a Hebbian learning circuit,which strengthens the synaptic weight between two neu-rons that fire in succession. This will lead to the dis-cussion of a circuit achieving STDP based on the timingcorrelation between pre- and post-synaptic activity.

IV. HEBBIAN UPDATE

The Hebbian update circuit under consideration isshown in Fig. 6(a). The operation of this circuit is basedon a similar principle to the supervised learning circuitsdiscussed in Sec. III in that the synaptic bias current Isyis adjusted based on the amount of flux stored in theSS loop. In this section we will explore how the DC-

0 10 20 30 40 50 ∆t [ns]

∆I ss

/ I

sssa

t (%

)

Nfq (

x 10

0)

0

1

2

0

1

2

3

4

5

6

35 µA

36 µA

37 µA

38 µA

Isu

Ispd

Isu

r1

r2

L1

L2

L3

Jsu

Jss

SPD1

SPD2

I3

I2

I1

SSIsy

(a)

(b)

1 2

FIG. 6. Hebbian update. (a) Hebbian synaptic update cir-cuit diagram. Implements a rule based on temporally cor-related single photon detection events from the two neuronsassociated with the synapse. Circuit parameters are given inAppendix B. (b) Amount of current added to the synapticstorage loop, ∆Iss, as a percentage of the saturation currentof the loop, Isatss , versus time delay between upstream and lo-cal synaptic update photons, ∆t, for four values of Jsu biascurrent, Isu. In these calculations, Lss = 1 µH.

to-SFQ converter of Fig. 4 can be replaced by SPDs toenable flux to be added to the SS loop based on tem-porally correlated photonic activity within the network.In particular, we wish to implement a Hebbian updaterule that potentiates a synaptic connection between pre-and post-synaptic neurons when the pre-synaptic neuroncontributes to the firing of the post-synaptic neuron [1].

In Ref. 10, circuits transducing photonic signals tosupercurrent are discussed. The Hebbian rule re-quires a two-photon temporal-correlation circuit, like thetemporal-code receiver of Ref. 10, except the asymme-try of Hebbian update requires an asymmetrical initialbias to the two correlated SNSPDs. Operation of theHebbian update circuit discussed here can be describedqualitatively as follows. When no photons have been de-tected, the bias Ispd is directed through SPD1. The resis-tor r1 ensures that SPD2 is unbiased until SPD1 receivesa photon, and therefore photons incident on SPD2 haveno effect on the circuit unless they are incident during atime window following a detection event by SPD1. Oncea photon has been detected by SPD1, Ispd is redirected toI2 and I3. The current returns to I1 with a time constant

7

of τ1 = L1/r1. If a photon is detected by SPD2 duringτ1, Ispd is predominantly redirected to I3, which can besufficient to switch Jsu, the synaptic update JJ, perhapsmany times depending on the bias currents, Ispd and Isu,and the difference in arrival times between the two pho-tons, ∆t. More details of circuit design are included inAppendix B.

During circuit operation, we assume that when thepre-synaptic neuron fires a photonic pulse, one or morephotons will reach a synaptic firing circuit [10] of thepost-synaptic neuron and bring the neuron closer to itsthreshold [9]. We also assume additional photons havea probability of reaching SPD1 of the synaptic updatecircuit shown in Fig. 6(a) to perform the first step inimplementing the Hebbian rule. This photon is labeled“1” in Fig. 6(a). The probability of reaching SPD1 maybe controlled to modify the learning rate. Similarly, itis assumed that during a neuronal firing event, the lo-cal neuron will send photons to its downstream connec-tions, but also to its local synapse update circuits to ac-tivate learning by striking SPD2. This photon is labeled“2” in Fig. 6(a). This self-feedback is also illustrated inFig. 1(a).

The duration of the time window after the detection ofthe pre-synaptic update photon while the circuit is sensi-tive to the detection of the post-synaptic update photonis determined by the time constant L1/r1. In Fig. 6 weanalyze the current added to the SS loop as a functionof the delay, ∆t, for four values of Isu with Ispd fixedat 10 µA. We plot the change in current in the SS loop(∆Iss) as a percentage of the SS loop saturation cur-rent (Isatss ), during Hebbian update events characterizedby delay ∆t. This plot also shows the number of flux-ons created during each of the events. We see that theamount of synaptic weight modification depends stronglyon the temporal delay, dropping to zero after roughly τ1.We also see that the effect depends on Isu, providing ameans by which the memory update rate can be dynam-ically adjusted during operation via a DC bias current.This dependence on Isu provides a means to implementmetaplasticity, as will be discussed in Sec. VI. The quan-tity ∆Iss/I

satss represents the fraction of the synapse dy-

namic range that is acquired in a synaptic update event.Although the current in the SS loop (and therefore Isy)can only change by an integer number of flux quanta, theuse of high-kinetic-inductance flux storage loops whereinthousands of flux quanta can be stored makes this effec-tively an analog circuit. For the SS loop investigated inFig. 6(a), βL/2π = 1.9 × 104.

Hebbian learning rules may be based on average fir-ing rates of pre- and post-synaptic neurons or on timingbetween individual spikes from these neurons [2]. Herewe consider the latter. A timing-dependent learning ruleoften takes the form of exponential decay as a function ofthe difference in arrival times of pre- and post-synapticsignals. The form shown in Fig. 6(b) is slightly differentdue to Josephson nonlinearities. This modified tempo-ral dependence is likely of little consequence as it main-

tains the principal function of timing-dependent plastic-ity, which is to modify the synaptic weight based ontemporal correlation within a specified time window sur-rounding a neuronal firing event.

While the quantity ∆Iss represents the change insynaptic weight due to one Hebbian update event, thearea under the curves in Fig. 6(b) will be related tothe learning rate when averaged over many events, be-cause the delay between the two photons, ∆t, will varyacross events. In Fig. 6(b), the integral of the curve withIsu = 35 µA is 3.6% of the integral of the curve withIsu = 38 µA. For Isu = 36 µA, the value is 18%, and forIsu = 37 µA, the value is 48%. This indicates we can dy-namically change the learning rate across a broad rangeby adjusting Isu. A metaplasticity circuit accomplishingthis is discussed in Appendix F.

To further illustrate the performance of this deviceand provide intuition regarding operation, Fig. 7 showsdetails of the operation during Hebbian update eventsfor cases with ∆t = 0 ns (Fig. 7(a) and (b)) and with∆t = 25 ns (Fig. 7(c) and (d)). In these calculations, theSPDs were modeled in WRSpice as transient resistancesof 5 kΩ lasting for 200 ps introduced at a specified mo-ment of photon detection. Figures 7(a) and (c) show thecurrents I1, I2, and I3 during the time when the junctionis switching. The insets of Figs. 7(a) and (c) show a 100 nswindow, capturing the SPD recovery over a longer time.Figures 7(b) and (d) show the increase in the current cir-culating in the SS loop as well as the voltage pulses asfluxons enter the loop. We see that the Hebbian updateevents introduce 25 nA - 200 nA to the SS loop. Withthese values, 280 events with ∆t = 25 ns or 35 eventswith ∆t = 0 ns will saturate the SS loop. The number ofevents that saturate the loop can be adjusted with theSS loop inductance and with Isu.

While it is helpful to demonstrate a Hebbian updatemechanism using two photons coupled to a simple JJ cir-cuit, learning rules that can both strengthen and weakenthe synaptic connection are required for neural comput-ing.

V. SPIKE-TIMING-DEPENDENT PLASTICITY

The STDP we seek to implement performs the Heb-bian potentiating operation described in Sec. IV, but alsoenforces an anti-Hebbian depressing rule wherein a neu-ronal firing event at the post-synaptic neuron followedclosely by a neuronal firing event at a pre-synaptic neurondepresses the synaptic weight between the two neurons.A circuit capable of producing this STDP is depicted inFig. 8(a). Much as strengthening and weakening wereaccomplished in Sec. III by adding a mirror image of thestrengthening circuit to the SS loop, here we duplicatethe Hebbian circuit of Sec. IV to achieve STDP. The sim-ilarity of the SPD circuit of Fig. 8(a) and the JJ circuitof Fig. 5(a) is apparent.

The symmetry between the strengthening and weak-

8

0

5

10

15

20

25

Time [ns]

I ss [nA

]

0

0.4

Vss [m

V]

30 31

0

1

2

3

4

5

I [µ

A]

Time [ns]

0

2

4

6

0 100

7 µA, 25 ns

I [µ

A]

0

5

10

15

20

I ss [10

nA

]

6 10

0

0.4

Time [ns]

Vss [m

V]

0

2

4

6

I [µ

A]

7 µA, 0 ns

0 1000

2

4

6

Time [ns]

I [µ

A]

(a)

(b)

(c)

(d)

I3

I2

I1

FIG. 7. Circuit operation during synaptic update events. (a)Currents I1, I2, and I3 during a synaptic update event withIspd = 7 µA and ∆t = 0 ns. The inset shows the currentsover a longer time period after photon arrival. (b) The cur-rent stored in the synaptic storage loop and the voltage pulsescorresponding to 106 fluxons entering the loop. (c) CurrentsI1, I2, and I3 during synaptic update event with Ispd = 7 µAand ∆t = 25 ns. (d) The current stored in the synaptic stor-age loop and the voltage pulses corresponding to 13 fluxonsentering the loop.

ening receiver circuits in the STDP circuit of Fig. 8(a)is broken based on whether the SPD that is biased inthe steady state receives photons from the pre-synapticor post-synaptic neuron. In the synaptic-weakening re-ceiver circuit, a post-synaptic photon detected by SPD3

followed by a photon from a pre-synaptic neuron detectedby SPD4 introduces counter-circulating flux to the SSloop. The time constants and biases of the strengtheningand weakening receivers can be adjusted independently.

The WRSpice calculation shown in Fig. 8(b) and (c) il-lustrates the circuit in operation. Two synaptic strength-ening events and two synaptic weakening events occur.The currents associated with synaptic strengthening andweakening, I+ and I− are shown in Fig. 8(b). The synap-tic bias current delivered to the synaptic firing junction[10] is shown in Fig. 8(c). A synaptic strengthening eventoccurs with ∆t = 20 ns, followed by a weakening eventwith ∆t = 10 ns and another with ∆t = 25 ns. A finalstrengthening event occurs with ∆t = 5 ns. The synap-tic bias current, Isy, is observed to respond as expectedbased on the Hebbian analysis in Sec. IV. In this cal-culation, Lss = 20 nH, and we mention again that theamount of current added to Iss and therefore Isy duringa synaptic update event can be linearly scaled with Lss

in hardware and with Ispd dynamically. Below a certainvalue of Ispd photon detection will not occur, and mem-ory update will cease. The memory update rate of theSTDP synapse can be controlled by adjusting the fre-quency of photon absorption events. This considerationand others related to implementation of these circuits arediscussed in Sec. VI.

The circuit of Fig. 8(a) induces STDP based on photondetection events from the pre- and post-synaptic neurons.It may also be possible to achieve STDP entirely in theelectronic domain through the electrical signals producedduring synaptic firing events and neuronal firing events.One means to use fluxon signals while setting correlationtime constants with L/r (as shown in Fig. 8) is to usefluxons to switch the gate of an nTron [39]. In Ref. 33 weshow this operation in the context of a neuronal thresh-olding element. For STDP, the SPDs could be replacedwith nTrons. Fluxons generated by Jsf during synapticfiring events would represent pre-synaptic activity andwould switch the gates of nTrons replacing the left SPDsin Fig. 8(a). Fluxons generated by the thresholding junc-tion, Jth [33], would switch the gates of nTrons replacingthe right SPDs in Fig. 8(a). Hebbian and anti-Hebbianrules would be implemented based on temporal correla-tion between pre- and post-synaptic activity, and no pho-tons would need to be expended for the operation. Yetthe complexity of Josephson circuitry at each synapsewould increase.

While crucial to learning and the interplay betweenthe structure and function of neural systems, STDP isonly one of many synaptic plasticity mechanisms. De-spite their significance, discussion of short-term plastic-ity, homeostatic plasticity, and metaplasticity are rele-gated to Appendices D, E, and F. Discussion of how the

9

0

1

2

3

4

I [µ

A]

5 ns25 ns20 ns 10 ns

I

-

I

+

0 100 200 300 400

1.5

2.5

Time [ns]

I sy [µA

](a)

(b)

(c)

Ispd1

Isu

+

Isu

-Ispd2

SPD1

SPD2

SPD3

SPD4

I

+

I

-

SS

Isy

1 2

2 1

FIG. 8. Implementation of spike-timing-dependent plasticity.(a) Circuit under consideration. (b) The currents I+ and I−

during synaptic update events. (c) The synaptic bias current,Isy, delivered to the synaptic firing circuit [10] as a functionof time as the synaptic update events of (b) strengthen andweaken the synaptic connection. The full circuit with theSTDP module (a) delivering Isy to the synaptic firing junctionis shown in Fig. 9 of Appendix C.

parameters of the circuits presented here map to learn-ing rate and enable quick adaption alongside long-termmemory retention is presented in Appendix G.

VI. DISCUSSION

This work has explored synaptic update circuits capa-ble of delivering a variable synaptic bias current to thesynaptic firing circuits presented in Ref. 10. We have in-vestigated manipulation of the synaptic weight throughexternal input of square wave pulses, as would be desir-able for supervised learning, as well as manipulation ofsynaptic weight via photon detection events, as wouldbe desirable for unsupervised learning. As an extensionof supervised learning, it is interesting to consider usingJJ circuits for fast control of synaptic weights. UsingJosephson driver circuits [40–42], synaptic weights couldbe precisely dynamically controlled. In Sec. III we showedsimulations of cycling between weak and strong synapticweights with no perceptible hysteresis at 10 GHz. Fir-ing rates of superconducting optoelectronic neurons arelikely to be limited to below 1 GHz due to SPD recoverytime and emitter lifetime. Operation with network cy-cles having oscillation frequencies up to 20 MHz is likely.The potential to vary synaptic weight at much higher fre-quencies (10 GHz) introduces the possibility that synap-tic connections could be weighted in the frequency do-main. The same synaptic weight between two neuronscould be strong in some Fourier components and weak inothers.

For unsupervised learning, we considered circuits com-bining single–photon detectors and Josephson junctionsto implement unsupervised synaptic update rules basedon photons received from correlated neuronal firingevents. For full spike–timing–dependent plasticity, thesynaptic update circuits described here provide ports forfour photons: one strengthening photon from both thepre–synaptic and post–synaptic neuron, and one weaken-ing photon from both the pre–synaptic and post–synapticneuron. For a single synaptic strengthening or weaken-ing event, two of these photons must be present. Whenoptically implementing a synaptic update rule based ontiming correlation, it is difficult to achieve a circuit re-quiring fewer than two photons.

Other forms of photonic synapses have recently beendeveloped and offer utility in multiple neural contexts[43–47]. One can leverage phase shifts in microrings[44, 45] or Mach-Zehnder interferometers (MZIs) [46] toadjust synaptic weight. Thermal tuning is often em-ployed to implement the phase shifts. Thermal tuningrequires more power than is suitable for this hardwareplatform. Phase shifters may be also be large if MZIs areused, and phase shifters may require exotic materials,which limit scaling if electro-optic effects are leveraged.If different synaptic channels are addressed with differ-ent frequencies of light, the out-degree of a node in thenetwork is limited by the multiplexed channel spacing.Approaches using MZIs for weighting and routing havethe disadvantage that STDP cannot be implemented be-cause modifying a single phase shifter in the network af-fects many synaptic weights. One approach to synap-tic weighting in the photonic domain utilizes a variable

10

optical attenuator at each synaptic connection. Phase-change materials have been employed as such variableattenuators [47], and the absorption of phase change ma-terials can be affected with pulses of light, thus intro-ducing a Hebbian-type synaptic weight update process.While such an approach may be useful for certain typesof neural circuits, update of these synapses requires toomany photons to be useful for the energy-efficient neu-ral computing scheme developed here (billions of pho-tons per update operation for phase change versus sin-gle photons for superconducting optoelectronics). It isalso not clear how anti-Hebbian synaptic update can beintroduced to enable full spike-timing-dependent plastic-ity. It remains to be seen if other synaptic operationssuch as short-term plasticity, homeostatic plasticity, andmetaplasticity can be achieved with phase-change ma-terials. Synaptic weights that attenuate a signal in theoptical domain require more light from neuronal firingevents, and many photons are simply absorbed at weaksynapses. By contrast, using photons for communicationbut weighting in the superconducting domain, as pre-sented here, uses fluxons to change the synaptic weight,and they can be generated with orders of magnitude lessenergy than photons. While all of these approaches tosynaptic weighting may be useful in different contexts,we have developed the synapses presented in this workbased on simultaneous considerations of power, complex-ity, scalability, speed, and size in the context of the su-perconducting optoelectronic hardware platform [8, 9].

An important weakness of the synapses presented hereis they lose all memory when superconductivity is bro-ken. The neuromorphic system must remain below Tc topreserve what has been learned. This class of Frosty theSnowman memory may be augmented by devices that canbe heated, such as magnetic Josephson junctions [48–50].It would be appealing if the state of memory in the plasticsynapses described here could be transferred to long-termmagnetic memory, perhaps during a sleep phase.

Another potential challenge for this type of memory inloop neurons is flux trapping. The synaptic integrationloops discussed in Ref. 10 are likely to include resistorsto give a leak rate. Trapped flux in those loops will beless problematic. The synaptic storage loops that setthe synaptic weights are intended to store flux for a longtime to maintain memory, so they will not include resis-tors. In this case, trapped flux will produce variationsin the initial synaptic weights across an ensemble. Forbinary synapses, this will result in some synapses beinginitialized with strong synaptic weight, and some withweak. For SS loops with high inductance, stray flux willinduce a small current, so the perturbation may be smallrelative to the dynamic range of the synapse. For largeensembles of synapses, the statistical variation may betolerable or even advantageous. If flux proves problem-atic, techniques used to shield superconducting qubitscan be employed [51].

In Ref. 9 we argue that a dynamical system capableof differentiated processing and information integration

across spatial and temporal scales underlies cognition. InRef. 10 we introduced the relaxation oscillators and den-dritic processing loops capable of implementing the tem-poral synchronization operations necessary for integrat-ing information in time. Network synchronization andsynaptic plasticity are mutually constructive phenomenain that synaptic strengthening through spike timing ismore likely to occur when the firing of two neurons iscorrelated, and the strengthened synapses, in turn, makethe correlated neurons more likely to synchronize. Net-works with small-world structure [52, 53] and dynam-ics characterized by self-organized criticality are crucialto achieving information integration. Hebbian learningrules and STDP have also been shown to convert ran-dom networks into small-world networks and to give riseto self-organized criticality [5, 54]. Creation of hard-ware capable of supporting complex networks and synap-tic learning mechanisms will provide a powerful tool forthe investigation of the relation between critical networkdynamics and cognitive function. In the present workwe have shown the complex synaptic behavior necessaryfor rapid adaptation, long-term memory retention, andsynaptic update based on network activity. Networks ofneurons connected by these synapses will be capable ofintegrating information learned at many times in manycontexts in a single dynamical state.

This work has focused on changing synaptic weightsin superconducting optoelectronic neurons. A centralquestion of the hardware platform remains: how are thephotons created? This question is addressed in the nextpaper in this series, Ref. 33.

This is a contribution of NIST, an agency of the USgovernment, not subject to copyright.

11

Appendix A: Circuit parameters of supervisedmemory cells

The memory cell of Fig. 2 has been designed withthe following circuit parameters. Ib1ss = 38 µA, Ib2ss =20 µA, Lss = 90 pH. The four inductors comprising thetwo mutual inductors are labeled L1 − L4 from left toright. Their values are L1 = L2 = 45 pH, L3 = L4 =18 pH.

The memory cell of Fig. 4 has been designed with thefollowing circuit parameters. The inductors comprisingthe DC-to-SFQ converter are, from left to right, L1 =80 pH, L2 = 60 pH, L3 = 300 pH. The bias to the DC-to-SFQ converter is IDC = 73 µA. The drive current pulsesare I+ = 10 µA with 100 ps rise and fall time and 1 nsduration. The bias to the JJ in the SS loop is Ibss = 34 µA.The mutual inductor parameters between the SS loop andthe SB loop and from the SB loop to I1 are, from left toright L1 = 18 pH, L2 = 190 pH, L3 = 18 pH, L4 = 18 pH,and I1 = 27 µA. With Lss = 20 nH, ∆Iss = 103 nA perpulse, ∆Isy = 25 nA per pulse. The SS loop can store−4.94 µA < Iss < 4.96 µA.

In this work, all Josephson junctions have Ic = 40 µA.In contrast to the circuits of Ref. 10 where JJs withIc = 10 µA were used, these JJs do not switch with everysynaptic firing event, and consequently, using lower Ic forpower minimization is less important. Using Ic = 40 µAleads to circuits with wider operating margins and easeof fabrication. We argue in Refs. 33 and 55 that usingJJs with Ic = 40 µA for the circuits of Ref. 10 wouldalso be satisfactory. The JJs in this work [10] have beensimulated with βc = 0.95, corresponding to slightly over-damped junctions [17, 18].

Appendix B: Considerations for Hebbian circuitdesign

To achieve the desired Hebbian operation with thecircuit of Fig. 6(a), several considerations are pertinent.When SPD1 detects a photon, it needs to direct currentpredominantly to I2, and not to I3. When SPD2 de-tects a photon, it needs to direct current predominantlyto I3, and not to I1. These considerations inform us thatwe should choose L2 L3 and L3 L1. We chooseL2 = 12.5 nH for this study. Such a small SPD may havereduced detection efficiency, but the inefficiency is toler-able for this purpose, because synaptic update will occuronly rarely to optimize memory retention [20, 30]. Wethen choose L3 = 125 nH, and L1 = 1.25 µH. The choicesfor r1 and r2 are made to achieve the desired temporal be-havior. The L/r time constants must be long enough toensure the SPDs do not latch. Beyond this, they can bechosen to achieve the desired learning performance. Wechoose τ1 = 50 ns and τ2 = 5 ns to facilitate WRSpiceanalysis, but longer time constants may be necessary inpractice.

The circuit parameters relevant to Fig. 6(a) are as fol-

SS

DI/

NI/

NT

Jsf

Isu

Isu

SI

SB

Synapse

Weight

FIG. 9. Synaptic update circuit supplying Isy to synapticfiring circuit.

lows. Inductor values are L1 = 1.25 µH, L2 = 12.5 nH,L3 = 125 nH. Ispd = 7 µA - 10 µA. The bias to the synap-tic update junction is Ibsu = 38 µA, and the bias to thesynaptic storage junction is the same. The resistors r1and r2 can be chosen to achieve the desired correlationtime window.

Appendix C: Synaptic update circuit supplyingsynaptic firing circuit

The circuit configuration combining the synaptic up-date circuit of this work with the synaptic firing circuitof Ref. 10 is shown in Fig. 9. Bias current I1 can be usedto supply many synapses. A buffer stage (Jb1 and Jb2)isolates the SI loop from flux generated during synap-tic firing events. Initial simulations of this configurationshow that the buffer can employ junctions with Jb1 hav-ing 10 µA Ic (same as Jsf of Ref. 10), and Jb2 with 40 µAIc used throughout this work.

12

Appendix D: Short-term plasticity

Short term plasticity varies the post-synaptic responseto a pre-synaptic pulse train [3] on a time scale close tothe inter-spike interval [1]. Short term plasticity acts asa filter, and the response can be low-pass, high-pass, orband-pass depending on a number of factors. These var-ious filtering operations can be achieved in loop neuronswith the addition of typical SPD/JJ loop circuits thatchange their state in response to pre-synaptic activity,either from photons from the pre-synaptic neuron or flux-ons generated during the synaptic firing event. A circuitthat may be utilized to perform the filtering operationsof short-term plasticity using additional JJs and the flux-ons produced during a synaptic firing event is shown inFig. 10. This circuit expands upon the synaptic receivercircuit of Ref. 10. Two additional JJs have been added inseries to the junction in the Josephson transmission line,Jjtl. These JJs are coupled to independent loops thatare inductively coupled to the synaptic bias loop. In theabsence of synaptic activity, the bias is set by Isy, just asbefore. However, during a synaptic firing event, the twoadditional junctions also switch. Therefore, flux is cou-pled to the synaptic integrating loop, as before, but fluxis also added to two new loops, the short-term facilitatingloop (SF), and the short-term depressing loop (SD). TheSF loop will add current to Isy, effectively strengthen-ing the synaptic weight, and the SD loop will reduce thecurrent to Isy, effectively weakening the synaptic weight.Therefore, the sign of the mutual inductance of the twoloops is opposite, and their magnitude may differ. Thetime constants of the SF and SD loops can be set inde-pendently, and they will likely be slightly longer than theinter-spike interval of pulse trains in the system (≈ 1 µs).A given synapse may employ one or both loops as re-quired for information processing, and a given neuron islikely to benefit from an ensemble of synapses with di-verse short-term filtering responses.

Short-term facilitation may operate such that the firstpre-synaptic pulse evokes no post-synaptic response, andonly after several pulses has the synaptic weight beenfacilitated to the point of communicating subsequentpulses. The circuit of Fig. 10 would need to be modifiedto achieve this behavior, as facilitation and depressionboth depend on the switching of the junctions, which re-quires successful pre-synaptic transmission. Such short-term facilitating behavior can be accomplished by intro-ducing an additional SPD explicitly for short-term plas-ticity. This SPD would receive photons from the pre-synaptic neuron, just as the SPD in the receiver circuitshown in Fig. 10, but the additional SPD would add noflux to the SI loop upon firing, and would instead onlyadjust the flux in the SF and SD loops.

SI

SF

SDSB

Isy

Jsf

Jsi

FIG. 10. Circuit for implementing short-term plasticity.Synaptic firing events cause the junctions in the short-termfacilitating and short-term depressing loops to generate flux.The flux in the facilitating loop acts to temporarily achievesynaptic gain, and the flux in the depressing loop acts totemporarily suppress the synaptic efficacy. The inductancesof the loops determine the magnitude of their effects, and theL/r time constants determine the temporal envelope. Inde-pendent control of L and r in each of the loops shapes thefilter response.

Appendix E: Homeostatic plasticity

The function of homeostatic plasticity is to modulatesynaptic efficacy in response to a running average of post-synaptic neuronal activity to keep neuronal gain withina useful dynamic range [56]. A loop neuron circuit canimplement homeostatic plasticity with fluxons generatedby thresholding events of the post-synaptic neuron. Onemeans to achieve this operation is shown in Fig. 11.

Whereas the short-term plasticity circuit of AppendixD made use only of signals generated by a synaptic fir-ing event, the homeostatic plasticity circuit makes useonly of signals generated by post-synaptic neuronal fir-ing events. The homeostatic plasticity (HP) loop is nega-tively inductively coupled to the synaptic firing junction(Jsf), meaning flux added to HP reduces the bias to Jsf ,thereby depressing the synaptic efficacy. By choosing anL/r time constant for the HP loop that is longer thanthe neuron’s typical inter-spike interval, the negative HPfeedback depends on a sliding temporal average of post-synaptic neuronal firing activity [57].

Appendix F: Metaplasticity

Homeostatic plasticity (Appendix E) is one example ofa plasticity mechanism that compensates for neural activ-ity on longer time scales to adjust learning rate. Homeo-

13

SI

NT

HPSB

Isy

Jsf

Jsi

FIG. 11. Circuit for implementing homeostatic plasticity. Afluxon generated by the thresholding junction during a neu-ronal firing event changes the flux state of the homeostaticplasticity loop. This flux is inductively coupled to the bias ofthe synaptic firing junction.

static plasticity is a response to a sliding temporal aver-age of the post-synaptic neuron [57, 58]. Metaplasticityrefers more generally to mechanisms that adjust not thesynaptic efficacy, but the rate of change (or probabilityof change) of synaptic efficacy. Here we discuss a loopcircuit that achieves a metaplastic response [28, 29, 59]based on both pre-synaptic and post-synaptic activityusing similar SPD/JJ circuits to those developed for theSTDP circuit of Sec. V.

The circuit under consideration is shown in Fig. 12.The concept here is similar to many other operationsin loop neurons. The fractional change in synaptic effi-cacy (α, see Sec. VI) incurred during an STDP updateevent in the STDP circuit of Sec. V depends on the mag-nitude of the current through the junctions in parallelwith the SPDs. The function of the metaplastic circuitof Fig. 12 is to modify these bias currents based on corre-lated pre-synaptic and post-synaptic activity. To achievethis operation, the same circuit block that is employedto adjust Isy during a plasticity (efficacy update) eventis also employed to adjust the JJ bias during a metaplas-ticity (learning rate update) event. With this circuit, theamount the synaptic efficacy is adjusted during an STDPupdate event depends on the flux trapped in the meta-plasticity (MP) loops. The learning rate depends on boththe efficacy update frequency as well as the magnitude ofeach update (see Sec. VI). Thus, by changing the magni-tude of the updates, the metaplastic circuits modify thelearning rate.

Considering the metaplastic circuits in the context ofthe synapse as a whole [10], the state of the synapseis associated with the flux in the synaptic integratingloop. The rate of change of the state of the synapse isassociated with the flux in the synaptic storage loop. Therate of change of the rate of change of the state of thesynapse is associated with the flux in the metaplasticity

loops. By cascading additional loops, one can continuethe hierarchy of synaptic loops that record the state ofthe synapse and its derivatives. We suspect the threelevels of hierarchy presented here will suffice for manyapplications.

At this point, a basic algorithm for loop neuron designhas emerged. For each synaptic function, add an SPD,a JJ, and a loop. Inductively couple the loop bias cur-rents as functionally appropriate. Choose time constantscarefully. Repeat until there is no more space. As thenumber of plasticity operations, and therefore SPDs, JJs,and loops, grows large, it may be possible to reduce thecomponent count by using the same SPDs, JJs, and loopsfor multiple operations.

The circuits for various forms of synaptic plasticity pre-sented in these Appendices are motivated qualitatively,and superior designs are undoubtedly possible. The cir-cuit implementations for STDP, short-term plasticity,homeostatic plasticity, and metaplasticity are intendedto convey the potential for diverse synaptic functionalityachievable with superconducting optoelectronic circuitsin the context of loop neurons. Because synaptic op-erations use few photons and fluxons, they are energyefficient. Scaling in complexity will likely be limited byfabrication challenges and device real estate.

Appendix G: Learning rate and memory retention

To discuss synaptic update, it is helpful to define sev-eral parameters. We follow the conventions of Refs. [29]and [30]. We refer to the normalized synaptic weight asw, where w = 0 corresponds to the minimum synapticweight (in general not corresponding to a synaptic effi-cacy of zero), and w = 1 corresponds to the maximumsynaptic weight. The spacing between synaptic levels isdenoted by α. The total number of stable synaptic statesbetween w = 0 and w = 1 is 1/α. Reference 29 defines acandidate plasticity event as “the occurrence of a patternof activity that could potentially lead to synaptic modi-fication.” These event occur at a rate r. The probabilitythat one of these events is a candidate for strengthening(Hebbian) is f+, and the probability that it is a candi-date for weakening (anti-Hebbian) is f−. The symbol qdenotes the “size of the potentiation and depression mod-ifications” when synaptic update occurs [30]. Synapticstrengthening occurs at a rate qf+r, and weakening oc-curs at a rate qf−r. The synaptic efficacy update ratesas a fraction of the full synaptic efficacy range are givenby αqf+r and αqf−r. The initial signal–to–noise ratioof a memory upon storage is denoted by S0/N0, and isproportional to the number of synapses which have beenmodified by the memory.

In the context of the circuits described here, w is re-lated to the synaptic bias current, Isy, which determinesthe synaptic efficacy. The synaptic efficacy is manifestphysically as the current added to the NI loop duringa synaptic firing event [10]. In this work we have been

14

SS

Isy

Isy

MP

Isy

MP

STDP

Metaplasticity

FIG. 12. Circuit for achieving metaplasticity with single-photon update events. The metaplasticity loop stores a history ofpre-synaptic/post-synaptic correlation events, and this trapped flux is inductively coupled to the bias of the STDP updatecircuitry. Hebbian/anti-Hebbian correlation circuits are shown, but symmetrical correlation circuits [10] could also be used.Single-photon detectors could be replaced by nTrons to utilize only local electrical signals.

treating Isy = 1 µA as the w = 0 state of the synapse,and Isy = 3 µA as the w = 1 state of the synapse. Forthe binary synapse of Figs. 2 and 3, 1/α = 1. For themulti–stable synapse of Figs. 4 and 5, 1/α was shownto be nearly 1000. α is determined by the synaptic stor-age loop inductance, and can take a wide range of val-ues. The rate r at which candidate Hebbian and anti-Hebbian events occur depends on the firing rates of thenetwork, and this parameter is normalized out of analy-ses of memory retention times. The probabilities f+ andf− also depend on network activity and in general can-not be relied upon to be precisely balanced [30]. In the

circuits described here, f+ and f− can be engineered bychanging the number of photons that are directed to theSTDP receiver SPDs during each neuronal firing event.This number of photons can be much less than one sothat a single photon is rarely directed for plasticity andsynaptic update is infrequent. For example, we may op-erate in a mode wherein a pre–synaptic neuron sends onephoton to each downstream synaptic firing port, one pho-ton to each downstream synaptic update strengtheningport, and one photon to each downstream synaptic up-date weakening port during each neuronal firing event.The rate at which the post–synaptic neuron sends pho-

15

tons to its own synaptic update ports then controls f+and f−. This hardware-defined means of setting f+ andf− can vary across a synaptic population. This approachto slow stochastic learning has the benefit of requiringfew photons per neuronal firing event. A neuronal firingevent would need to produce 3kout photons, where kout isthe number of synaptic connections directed away fromthe firing neuron.

The size of synaptic modifications, q, is determined inthe STDP circuit by the values of Ispd1, Ispd2, I+su, andI−su. As shown in Fig. 6, changing Isu changes the amountof current added to the SS loop during a synaptic updateevent, and therefore changes the current bias, Isy, whichsets the synaptic weight during a synaptic firing event.

Investigation of the limits of memory retention in thepresence of ongoing plasticity [30] reveals that memorylifetimes can be improved linearly with the number ofstable synaptic states, 1/α. The expense is a decreasedsignal–to–noise ratio of stored memories, S0/N0. Refer-ence 30 further discovered that synapses in which q isa function of w (“soft bounds”) performed well for ex-tending memory storage times while maintaining highS0/N0. The circuits discussed in the present work canimplement such soft bounds by inductively coupling Isuto Iss so that Isu approaches some minimum value as thesynaptic storage loop approaches saturation.

In addition to plasticity and multi–stable synapses,power law memory retention is likely to make use of in-ternal synaptic states [29] that do not alter the efficacy ofthe synapse, but do affect the probability that a futureHebbian event will update the synaptic weight as wellas affect the magnitude of that update, should it occur.This corresponds to states of the synapse with differentvalues of q, but the same value of Iss. Circuit modifi-cations that adapt learning rate in response to internaland external activity are referred to as metaplastic [59].Using synapses with complex internal states allows forrapid incorporation of new information while maintain-ing stable, long–term memories [28–30, 34, 59]. A routeto achieve metaplasticity in the circuits presented hereis to vary Isu, the current that determines the amountof synaptic shift during an update event. Thus, in thecircuits presented here, we have q(Isu). By changing Isu

in time, plasticity can be present in an ensemble during acertain period of training, and then subsequently turnedoff, allowing those memories not to be corrupted by sub-sequent activity. Isu can by dynamically varied exter-nally to implement supervised metaplasticity, or Isu canbe modified by activity within the network using similarreceiver circuits to those presented for STDP (see Ap-pendix F). Activity dependent modification not only ofIsy, but also I+su1 and I−su (see Fig. 8) is likely to providemechanisms to adjust synaptic update rates to ensurethe dynamic range of the synapses is matched to corticalactivity [56, 57] (see Appendix E).

Reference 29 elucidates that a network of heteroge-neous synapses with a varying number of stable statesdoes not outperform a network of binary synapses withmultiple internal states, but a network of heterogeneoussynapses with different numbers of stable states as wellas multiple q states was not investigated. This combi-nation is likely to achieve the best of both worlds. Theoptoelectronic synapses of the present work have the op-portunity to achieve spike–timing–dependent plasticitywith a large number of stable levels as well as a largenumber of q states affecting adaptation rate. If the goalis to achieve a learning system that can rapidly incorpo-rate new information while retaining memories for a longtime, neural systems must incorporate synapses that varyby different amounts and over different time scales. Ona given neuron, or across an ensemble of neurons, a setof synapses may be heterogeneous in multiple capacities.The synapses may have a distribution in terms of numberof stable states (1/α), and they may have a distributionin learning rate, manifest in q(Isu). Synapses that are up-dated infrequently and in small increments will store oldwisdom. Binary synapses that switch readily bring fresheyes. Neurons comprising primarily fresh eyes synapsesbring fresh eyes to a network, and neurons comprisingprimarily synapses that were trained long ago and rarelychange bring old wisdom. The ability to integrate infor-mation from a network of synapses with different learningrates trained at different times is advantageous for net-works with optimal, power law forgetting rates. An en-semble of synapses is also likely to benefit from a diversityof short-term and homeostatic plasticity mechanisms, asdiscussed in Appendices D and E.

[1] P. Dayan and L.F. Abbott, Theoretical Neuroscience(The MIT Press, 2001).

[2] W. Gerstner and W. Kistler, Spiking neuron models, 1sted. (Cambridge University Press, Cambridge, 2002).

[3] L.F. Abbott and W.G. Regehr, “Synaptic computation,”Nature Reviews 431, 796 (2004).

[4] G. Buzsaki, Rhythms of the brain (Oxford UniversityPress, 2006).

[5] B. Siri, M. Quoy, B. Delord, B. Cessac, and H. Berry,“Effects of hebbian learning on the dynamics and struc-ture of random networks with inhibitory and excitatory

neurons,” Journal of Physiology Paris 101, 136 (2007).[6] J. Hawkins and S. Ahmad, “Why neurons have thousands

of synapses, a theory of sequence memory in neocortex,”Frontiers in Neural Circuits 10, 23 (2016).

[7] M.A. Nielsen, Neural networks and deep learning (Deter-mination Press, 2015).

[8] J.M. Shainline, S.M. Buckley, R.P. Mirin, and S.W.Nam, “Superconducting optoelectronic circuits for neuro-morphic computing,” Phys. Rev. App. 7, 034013 (2017).

[9] J.M. Shainline, S.M. Buckley, A.N. McCaughan,J. Chiles, R.P. Mirin, and S.W. Nam, “Superconduct-

16

ing optoelectronic neurons I: General principles,” arXiv, 1805.01929 (2018).

[10] J.M. Shainline, S.M. Buckley, A.N. McCaughan,M. Castellanos-Beltran, C.A. Donnelly, M.L. Schneider,R.P. Mirin, and S.W. Nam, “Superconducting optoelec-tronic neurons II: Receiver circuits,” arXiv , 1805.02599(2018).

[11] G.N. Gol’tsman, O. Okunev, G. Chulova, A. Lipatov,A. Semenov, K. Smirnov, B. Voronov, A. Dzardanov,C. Williams, and R. Sobolewski, “Picosecond super-conducting single-photon optical detector,” Appl. Phys.Lett. 79, 705 (2001).

[12] C.M. Natarajan, M.G. Tanner, and R.H. Hadfield, “Su-perconducting nanowire single-photon detectors: physicsand applications,” Supercond. Sci. Tech. 25, 063001(2012).

[13] D.K. Liu, L.X. You, S.J. Chen, X.Y. Yang, Z. Wang,Y.L. Wang, X.M. Xie, and M.H. Jiang, “Electri-cal characheristics of superconducting nanowire singlephoton detector,” IEEE Trans. Appl. Supercond. 23,2200804 (2013).

[14] F. Marsili, V.B. Verma, J.A. Stern, S. Harrington, A.E.Lita, T. Gerrits, I. Vayshnker, B. Baek, M.D. Shaw, R.P.Mirin, and S.W. Nam, “Detecting single infrared pho-tons with 93% system efficiency,” Nat. Photon. 7, 210(2013).

[15] V.B. Verma, B. Korzh, F. Bussieres, R.D. Horansky, S.D.Dyer, A.E. Lita, I. Vayshenker, F. Marsili, M.D. Shaw,H. Zbinden, R.P. Mirin, and S.W. Nam, “High-efficiencysuperconducting nanowire single-photon detectors fab-ricated from mosi thin-films,” Opt. Express 23, 33792(2015).

[16] M. Tinkham, Introduction to Superconductivity, 2nd ed.(Dover, 1996).

[17] T. Van Duzer and C.W. Turner, Principles of supercon-ductive devices and circuits, 2nd ed. (Prentice Hall, USA,1998).

[18] Alan M. Kadin, Introduction to superconducting circuits,1st ed. (John Wiley and Sons, USA, 1999).

[19] C.D. Schuman, T.E. Potok, R.M. Patton, J.D. Birdwell,M.E. Dean, G.S. Rose, and J.S. Plank, “A survey of neu-romorphic computing and neural networks in hardware,”arXiv , arXiv:1705.06963v1 (2017).

[20] S.-C. Liu, T. Delbruck, G. Indiveri, A. Whatley, andR. Douglas, eds., Event-based neuromorphic systems(John Wiley and Sons).

[21] J.J. Hopfield, “Neural networks and physical systemswith emergent computational abilities,” PNAS 79, 2554(1982).

[22] D.H. Ackley, G.E. Hinton, and T.J. Sejnowski, “A learn-ing algorithm for boltzmann machines,” Cognitive Sci-ence 9, 147 (1985).

[23] C.M. Bishop, “Neural networks and their applications,”Rev. Sci. Inst. 65, 1803 (1994).

[24] G.E. Hinton and R.R. Salakhutdinov, “Reducing the di-mensionality of data with neural networks,” Science 313,504 (2006).

[25] D. Silver, A. Huang, C.J. Maddison, A. Guez, L. Sifre,G. van den Driessche, J. Schrittwieser, I. Antonoglou,V. Panneershelvam, M. Lanctot, S. Dieleman, D. Grewe,J. Nham, N. Kalchbrenner, I. Sutskever, T. Lillicrap,M. Leach, K. Kavukcuoglu, T. Graepel, and D. Hass-abis, “Mastering the game of go with deep neural net-works and tree search,” Nature 529, 484 (2016).

[26] O. Sporns, G. Tononi, and G.M. Edelman, “Connectivityand complexity: the relationship between enuroanatomyand brain dynamics,” Neural networks 13, 909 (2000).

[27] G. Buzsaki and A. Draguhn, “Neuronal oscillations incortical networks,” Science 304, 1926 (2004).

[28] P. Khorsand and A. Soltani, “Optimal structure of meta-plasticity for adaptive learning,” PLOS ComputationalBiology (2017).

[29] S. Fusi, P.J. Drew, and L.F. Abbott, “Casdcade modelsof synaptically stored memories,” Neuron 45, 599 (2005).

[30] S. Fusi and L.F. Abbott, “Limits on the memory storagecapacity of bounded synapses,” Nature Neuroscience 10,485 (2007).

[31] J.T. Wixted and E. Ebbesen, “On the form of forget-ting,” Psychol. Sci. 2, 409 (1991).

[32] J.T. Wixted and E. Ebbesen, “Genuine power curves inforgetting,” Mem. Cognit. 25, 731 (19978).

[33] J.M. Shainline, A.N. McCaughan, A. Jafari-Salim, S.M.Buckley, R.P. Mirin, and S.W. Nam, “Superconductingoptoelectronic neurons IV: Transmitter circuits,” arXiv ,1805.01941 (2018).

[34] D.J. Amit and S. Fusi, “Learning in neural networks withmaterial synapses,” Neural Computation 6, 957 (1994).

[35] Y. Mizugaki, H. Hakii, M. Moriya, K. Usami, andT. Kobayashi, “Mutual inductance coupled through su-perconducting thin film in niobium josephson integratedcircuits,” Japanese Journal of Applied Physics 44, 1(2005).

[36] S.R. Whiteley, “Josephson junctions in spice3,” IEEETras. Mag. 27, 2902 (1991).

[37] D. Floreano, P. Durr, and C. Mattiussi, “Neuroevolu-tion: from architectures to learning,” Evol. Intel. 1, 47(2008).

[38] F.P. Such, V. Madhavan, E. Conti, J. Lehman, K.O.Stanley, and J. Clune, “Deep neuroevolution: geneticalgorithms are a competitive alternative for trainingdeep neural networks for reinforcement learning,” arXiv, 1712.06567 (2018).

[39] A.N. McCaughan and K.K. Berggren, “Asuperconducting-nanowire three-terminal electrothermaldevice,” Nano letters 14, 5748 (2014).

[40] Q.P. Herr, A.Y. Herr, O.T. Oberg, and A.G. Ioannidis,“Ultra-low-power superconductor logic,” J. Appl. Phys.109, 103903 (2011).

[41] K.K. Likharev, “Superconductor digital electronics,”Physica C 482, 6 (2012).

[42] S.P. Benz, S.B. Waltman, A.E. Fox, P.D. Dresselhaus,A. Rufenacht, J.M. Underwood, L.A. Howe, R.E Schwall,and C.J. Burroughs Jr., “One-volt josephson arbitrarywaveform synthesizer,” IEEE Trans. Appl. Supercond.25, 1300108 (2015).

[43] P.R. Prucnal and B.J. Shastri, Neuromorphic photonics,1st ed. (CRC Press, New York, 2017).

[44] A.N. Tait, M.A. Nahmias, B.J. Shastri, and P.R. Pruc-nal, “Broadcast and weight: an integrated network forscalable photonic spike processing,” J. Lightwave Tech-nol. 32, 3427 (2014).

[45] A.N. Tait, T. Ferreira de Lima, E. Zhou, A.X. Wu,M.A. Nahmias, B.J. Shastri, and P.R. Prucnal, “Neuro-morphic photonic networks using silicon photonic weightbanks,” Nature Sci. Rep. 7, 7430 (2017).

[46] Y. Shen, N.C. Harris, S. Skirlo, M. Prabhu, T. Baehr-Jones, M. Hochberg, X. Sun, S. Zhao, H. Larochelle,D. Englund, and M. Soljacic, “Deep learning with co-

17

herent nanophotonic circuits,” Nature Photonics 11, 441(2016).

[47] Z. Cheng, C. Rios, W.H.P. Pernice, C.D. Wright, andH. Bhaskaran, “On-chip photonic synapse,” Science Ad-vances 3, 1700160 (2017).

[48] S.E. Russek, C. Donnelly, M. Schneider, B. Baek, M. Pu-fall, W.H. Rippard, P.F. Hopkins, P.D. Dresselhaus, andS.P. Benz, “Stochastic Single Flux Quantum Neuromor-phic Computing using Magnetically Tunable JosephsonJunctions,” in IEEE International Conference on Reboot-ing Computing (IEEE, 2016).

[49] M.L. Schneider, C.A. Donnelly, S.E. Russek, B. Baek,M.R. Pufall, P.F. Hopkins, and W.H. Rippard, “Energy-efficient single-flux-quantum based neuromorphic com-puting,” in IEEE International Conference on RebootingComputing (IEEE, 2017).

[50] M.L. Schneider, C.A. Donnelly, S.E. Russek, B. Baek,M.R. Pufall, P.F. Hopkins, P. Dresselhaus, S.P. Benz,and W.H. Rippard, “Ultralow power artificial synapsesusing nanotextured magnetic josephson junctions,” Sci-ence Advances 4, 1701329 (2018).

[51] A.D. Corcoles, J.M. Chow, J.M. Gambetta, C. Rigetti,J.R. Rozen, G.A. Keefe, M.B. Rothwell, and M. Stef-fen, “Protecting superconducting qubits from radiation,”App. Phys. Lett. 99, 181906 (2011).

[52] D.J. Watts and S.H. Strogatz, “Collective dynamics ofsmall-world networks,” Nature 393, 440 (1998).

[53] O. Sporns, Networks of the Brain, 1st ed. (The MITPress, Cambridge, Massachusetts, 2010).

[54] M. Rubinov, O. Sporns, J.-P. Thivierge, and M. Break-spear, “Neurobiologically realistic determinants of self-organized criticality in networks of spiking neurons,”PLoS Computational Biology 7, 1 (2011).

[55] J.M. Shainline, J. Chiles, S.M. Buckley, A.N. Mc-Caughan, R.P. Mirin, and S.W. Nam, “Superconductingoptoelectronic neurons V: Networks and scaling,” arXiv, 1805.01942 (2018).

[56] L.N. Cooper and M.F. Bear, “The bcm theory of synapsemodification at 30: interaction of theory with experi-ment,” Nature Reviews Neuroscience 13, 798 (2012).

[57] E.L. Bienenstock, L.N. Cooper, and P.W. Munro, “The-ory for the development of neuron selectivity: orienta-tion specificity and binocular interaction in visual cor-tex,” The Journal of Neuroscience 2, 32 (1982).

[58] L.N. Cooper and M.F. Bear, “The bcm theory of synapsemodification at 30: interaction of theory with experi-ment,” Nature Reviews Neuroscience 13, 798 (2012).

[59] W.C. Abraham, “Metaplasticity: tuning synapses andnetworks for plasticity,” Nature Neuroscience 9, 387(2008).