Biologically Inspired Biologically Inspired ComputationComputation

Chris DiorioComputer Science & Engineering

University of Washingtondiorio@cs.washington.edu

C. Diorio, 10–8–00 2

Nature is telling us something...

Can add numbers together in nanoseconds Hopelessly beyond the

capabilities of brains

Can understand speech trivially Far ahead of digital computers …and Moore’s law will end

C. Diorio, 10–8–00 3

Problem: How do we build circuits that learn

One approach: Emulate neurobiology Dense arrays of synapses

output W Xjj

j2

X1 X2

learn signal

input vector X

error signal

output W Xjj

j1

synapseW11

synapseW12

learn signalerror signal

synapse synapseW21 W22

C. Diorio, 10–8–00 4

Silicon synapses

n–

electrontunneling

p

electroninjection

n+ n+ n+

Silicon Synapse Transistor Charge Q Sets the Weight

p– substrate 0 1 2 3 4 510

-11

10-10

10-9

10-8

10-7

10-6

10-5

control-gate–to–source voltage (V)

sour

ce c

urr e

nt (

A) Q1

Q2

Q4 Q5

Q3

floating gate(charge Q)

Use the silicon physics itself for learning Local, parallel adaptation Nonvolatile memory

5C. Diorio, 10–8–00

Silicon synapses can mimic biology

–10 0 10 20 30 40 500

1

2

3

4

5

time (min)

syna

pse

sour

ce c

urr e

nts

(nA

)

Biological Synapses Silicon Synapses

Mossy-fiber EPSC amplitudes plotted over time, before and after theinduction of LTP. Brief tetanic stimulation was applied at the time in- dicated. From Barrionuevo et al., J. Neurophysiol. 55:540-550, 1986.

Synapse transistor source currents plotted over time, before andafter we applied a tetanic stimulation of 2×105 coincident (row & column) pulses, each of 10µs duration, at the time indicated.

Local, autonomous learning

C. Diorio, 10–8–00 6

Synaptic circuits can learn complex functions

Synapse-based circuit operates on probability distributions Competitive learning Nonvolatile memory 11 transistors 0.35µm CMOS Silicon physics learns

“naturally”

Silicon learning circuit versus software neural network Both unmix a mixture of Gaussians Silicon circuit consumes nanowatts

Scaleable to many inputs and dimensions

tru e m ean sc ircu it o u tp u t

so ftw a re n eu ra ln e tw o rk

valu

e (

V)

num ber of tra in ing exam ples0 2000 4000

0.2

0.4

0.6

0.8

1

1000 3000

C. Diorio, 10–8–00 7

Technology spinoff: Adaptive filters

Synapse transistors for signal processing ~100× lower power and ~10× smaller size than digital

Mixed-signal FIR filter16-tap, 7-bits 225MHz, 2.5mW

Built and tested in 0.35µm CMOSAdjust synaptic tap weights off-line

FIR filter with on-chip learning64 taps, 10 bits, 200MHz, 25mWIn fabrication in 0.35µm CMOS

On-line synapse-based LMS

C. Diorio, 10–8–00 8

Problem: How to study neural basis of behavior

Measure neural signaling in intact animals Implant a microcontroller in Tritonia brain

Tritonia is a model organism Well studied neurophysiology 500µm neurons; tolerant immune response Work-in-progress

B. Brain with implanted chip: Dorsal view

A. Tritonia and seapen

Images courtesy James Beck & Russell Wyeth

MEMS probe tip,amplifier brain

battery

tether

memory microcontroller,A/D, cache

visceralcavity

Tritonia diomedea

C. Diorio, 10–8–00 9

An in-flight data recorder for insects

An autonomous microcontroller “in-the-loop” Study neural basis of flight control

Manduca Sexta or “hawk moth”