1

Win

ter

200

8Bi

oE33

2A

© Kwabena Boahen

LargeLarge--scale neural modelingscale neural modeling

KwabenaKwabena BoahenBoahenStanford BioengineeringStanford Bioengineeringboahen@stanford.eduboahen@stanford.edu

Goal: Link structure to function by Goal: Link structure to function by developing multideveloping multi--level level computational models of neural computational models of neural systems.systems.

Win

ter

2008

BioE

332A

© Kwabena Boahen

WeWe’’re acquiring brain data re acquiring brain data at an unprecedented rate at an unprecedented rate

Reid et al 2005

CaCa++++ imagingimaging

Computational Computational primitivesprimitives

Functional Functional behaviorbehavior

MicrocircuitryMicrocircuitry

Hausser et al 1997

Dendritic recordingDendritic recording Serial Scanning EMSerial Scanning EM

Denk et al 2005

Now all we have to is connect the dotsNow all we have to is connect the dots……

+ =

Win

ter

2008

BioE

332A

© Kwabena Boahen

MultiMulti--level simulations can link level simulations can link structure to functionstructure to functionThe problem is one of scaleThe problem is one of scale

7 levels of investigation7 levels of investigation

10 orders of magnitude10 orders of magnitude

Option 1: Option 1: ExperimentExperiment

Difficult to controlDifficult to control

Option 2: Option 2: TheoryTheory

Ignores detailsIgnores details

Option 3: Option 3: SimulationSimulation

Include all detailsInclude all detailsComplements theoryComplements theory

Control all parametersControl all parametersComplements experimentComplements experiment

Churchland & Sejnowski 1992

Levels of InvestigationLevels of Investigation

Win

ter

2008

BioE

332A

© Kwabena Boahen

2

Win

ter

2008

BioE

332A

© Kwabena Boahen Ray Kurzweil 2001

3GHz Dell Precision

Brunsviga Model 20

100Mz Compaq Presario

Win

ter

2008

BioE

332A

© Kwabena Boahen

The fastest supercomputers can simulate The fastest supercomputers can simulate only 10,000 neurons in realonly 10,000 neurons in real--timetime

CellCell

8M neurons connected by 4B synapses8M neurons connected by 4B synapses99°° visual field in V1visual field in V1

1sec of activity took 1hr 20mins to simulate1sec of activity took 1hr 20mins to simulate47504750×× slower then realslower then real--timetime

Had to perform 38 trillion evaluationsHad to perform 38 trillion evaluations8M neurons 8M neurons ×× 6 comp. 6 comp. ×× 8 8 eqeq. . ×× 101055 steps/sec steps/sec

CompartmentCompartment

( )

( )1

V

Vu u

α

β−

( )( )

u V ududt Vτ

∞ −=

( ) ( )( ) ( )V

u VV Vα

α β∞=

+

( ) ( ) ( )1V

V Vτ

α β=

+

IonIon--channelchannel Blue Gene supercomputerBlue Gene supercomputer

LansnerLansner et al. used one et al. used one 20482048--processor rack processor rack (3Tflops, $2M)(3Tflops, $2M)

Shenoy et al. 2006

Win

ter

2008

BioE

332A

© Kwabena Boahen

Physicists revolutionized astrophysics Physicists revolutionized astrophysics by building their own supercomputerby building their own supercomputer

Hubble Telescope 1999

Two spiral galaxiesTwo spiral galaxies

Hardwired to calculate gravitational forceHardwired to calculate gravitational force

A third as fast as Blue Gene rack (1Tflop)A third as fast as Blue Gene rack (1Tflop)

Sixteen times more costSixteen times more cost--effective ($42K)effective ($42K)First to show First to show gravothermalgravothermal oscillationsoscillations

Resulted in 40 papers in 2000 aloneResulted in 40 papers in 2000 alone

Univ. of Tokyo Univ. of Tokyo astrophysicist astrophysicist Jun MakinoJun Makino

2001

Point mass approx.Point mass approx. Law of gravityLaw of gravity

2i

j ji ij

mF Gmr

= ∑

GRAPE6 supercomputerGRAPE6 supercomputer

Win

ter

2008

BioE

332A

© Kwabena Boahen

NeurogridNeurogrid——an affordable supercomputer an affordable supercomputer for neuroscientistsfor neuroscientists

88××8844××44Neurogrid Neurogrid (chips)(chips)

18,20018,200280280Speed Speed (TF)(TF)64M64M1M1MTotal Total (neurons)(neurons)

1K1K××1K1K256256××256256Neurocore Neurocore (neurons)(neurons)

201120112008 (!)2008 (!)

NeurogridNeurogrid: Board with grid of chips: Board with grid of chipsProgrammable connectionsProgrammable connections

One chip per cortical cellOne chip per cortical cell--layer or typelayer or type

NeurocoreNeurocore:: Chip with array of neuronsChip with array of neuronsProgrammable ionProgrammable ion--channel propertieschannel properties

Multiple compartments per neuronMultiple compartments per neuron

Chip 1

Chip 2

0 1 2 3

0 1 2 3

0 1 2 31 3 2 0

prepostRAM

3

Win

ter

2008

BioE

332A

© Kwabena Boahen

DonDon’’t evaluate equationst evaluate equations——emulate physicsemulate physics

Emulate ionic currents with electronic currentsEmulate ionic currents with electronic currents

Exploit physical Exploit physical analogyanalogyIncluding stochastic behaviorIncluding stochastic behavior

Analog VLSIAnalog VLSIVery Large Scale Integration Very Large Scale Integration

Runs in realRuns in real--timetimeTakes 1sec instead of 1hr and 20minsTakes 1sec instead of 1hr and 20mins

( )

( )1

V

Vu u

α

β−

( )( )

u V ududt Vτ

∞ −=

( ) ( )( ) ( )V

u VV Vα

α β∞=

+

( ) ( ) ( )1V

V Vτ

α β=

+

DrainSource

Gate

Bulk

e-⇔ ⇔

Ion channelIon channelTransistorTransistor

Mead 1989

Win

ter

2008

BioE

332A

© Kwabena Boahen

MultiMulti--area cortical models area cortical models

Society for Neuroscience 2005Essen & Fellerman 1991

isual areasisual areas

Sillito ‘06

MTMT

V1V1

Bac

kwar

dB

ackw

ard

Forward

Forward

Feedforward view of motionFeedforward view of motion

V1: PartsV1: Parts

MT: ObjectMT: Object

Anatomy has feedbackAnatomy has feedback

MT projects to V1MT projects to V1

Hypotheses about feedback:Hypotheses about feedback:

Aggregates parts into Aggregates parts into coherent objectcoherent object

Composes cues into Composes cues into unambiguous perceptunambiguous percept

Win

ter

2008

BioE

332A

© Kwabena Boahen

BioE332BioE332’’s thousands thousand--neuron babyneuron baby

RAM

Computer

STDP Chip

USB

CPLD

Win

ter

2008

BioE

332A

© Kwabena Boahen

The chip: SpikeThe chip: Spike--timing dependent plasticitytiming dependent plasticity

INTER-NEURON

PRINCIPLE CELLS

PRINCIPLE CELLS

PLASTIC SYNAPSES

PLASTIC SYNAPSES

PLASTIC SYNAPSES

PLASTIC SYNAPSES

1024 excitatory principle cells1024 excitatory principle cells21 plastic synapses each21 plastic synapses each

256 inhibitory interneurons256 inhibitory interneurons

750,000 transistors750,000 transistors

10.2mm10.2mm22 in 0.25in 0.25μμm CMOSm CMOS

95µm

60µm

4

Win

ter

2008

BioE

332A

© Kwabena Boahen

The GUI: Memorizing patternsThe GUI: Memorizing patterns

Before learning After learning

Synaptic strengths

LTP

LTD

Synaptic strengths

Neuron array Spike trains Neuron array Spike trains

Win

ter

2008

BioE

332A

© Kwabena Boahen

Lab 1: Synapse ModelLab 1: Synapse Model

Sum

Win

ter

2008

BioE

332A

© Kwabena Boahen

Lab 2: Neuron ModelLab 2: Neuron Model

Win

ter

2008

BioE

332A

© Kwabena Boahen

Lab 3: Adaptation and BurstingLab 3: Adaptation and Bursting

5

Win

ter

2008

BioE

332A

© Kwabena Boahen

Lab 4: Phase ResponseLab 4: Phase Response

Win

ter

2008

BioE

332A

© Kwabena Boahen

Lab 5: SynchronyLab 5: SynchronyW

inte

r 20

08Bi

oE33

2A

© Kwabena Boahen

Lab 6: BindingLab 6: Binding

Win

ter

2008

BioE

332A

© Kwabena Boahen

Lab 7: Synaptic PlasticityLab 7: Synaptic Plasticity

6

Win

ter

2008

BioE

332A

© Kwabena Boahen

Lab 8: Plasticity and SynchronyLab 8: Plasticity and Synchrony

Win

ter

2008

BioE

332A

© Kwabena Boahen

Lab 9: Associative memoryLab 9: Associative memory

Before learning After learning

Win

ter

2008

BioE

332A

© Kwabena Boahen

Lab 10: AttentionLab 10: Attention