stockholm brain institute cortex modeling and cortex- inspired computation anders lansner dept of...
Post on 19-Dec-2015
217 views
TRANSCRIPT
STOCKHOLM BRAIN INSTITUTE
Cortex modeling and cortex-Cortex modeling and cortex-inspired computationinspired computation
Anders Lansner
Dept of Computational BiologyKTH and Stockholm University
November 15, 2007 Albanova Instrumentation Seminar 2
STOCKHOLM BRAIN INSTITUTE
SynopsisSynopsis
• Methods in neuronal network modeling
• Large-scale cortex model example
• Perspectives on modeling and brain-inspired computing
November 15, 2007 Albanova Instrumentation Seminar 3
STOCKHOLM BRAIN INSTITUTE
GoalsGoals
• Models of neurons and neuronal networks• 1985 - …• Today high demand from neuroscience labs• Enables understanding of the brain
• Brain-like/inspired algorithms and architectures• Beyond ”neural networks”, ”neurocomputing”• ”Artificial brains” … on silicon
November 15, 2007 Albanova Instrumentation Seminar 4
STOCKHOLM BRAIN INSTITUTE
Cortical areas and microcircuitsCortical areas and microcircuits
November 15, 2007 Albanova Instrumentation Seminar 5
STOCKHOLM BRAIN INSTITUTE
Advances in experimental Advances in experimental neuroscienceneuroscience
• Shortage of data, but rapid development…
• E.g. genetic fluorescent marking + confocal tracing of pathways
• Livet et al. Nature Nov 2007
November 15, 2007 Albanova Instrumentation Seminar 6
STOCKHOLM BRAIN INSTITUTE
Models at multiple levelsModels at multiple levels
• (Molecular dynamics)• Sub-cellular level models• Single neuron and synapse models• Microcircuits and networks• Full-scale global network models
November 15, 2007 Albanova Instrumentation Seminar 7
STOCKHOLM BRAIN INSTITUTE
Types of neuron modelsTypes of neuron models
• Summing threshold units• Connectionist model neural network• Integrate-and-fire• Hodgkin-Huxley formalism
November 15, 2007 Albanova Instrumentation Seminar 8
STOCKHOLM BRAIN INSTITUTE
Single cell models - signal Single cell models - signal processingprocessing
• An equivalent electrical circuit model
November 15, 2007 Albanova Instrumentation Seminar 9
STOCKHOLM BRAIN INSTITUTE
Equivalent electrical circuit of a Equivalent electrical circuit of a membrane membrane patchpatch
Ohm’s law:
Nernst eqn:
( )
ln
i i m i
outi
in
I g V E
CRTE
zF C
November 15, 2007 Albanova Instrumentation Seminar 10
STOCKHOLM BRAIN INSTITUTE
The gate modelThe gate model”Hodgkin-Huxley model””Hodgkin-Huxley model”
yydt
dy 1
First-order kinetics yields:
y
yydt
dy
1
y
p independent gating particles: py
1
1001.0)(
10
10Vn
e
VV 80125.0)(
V
n eV
K+ :
open closed
y y1
November 15, 2007 Albanova Instrumentation Seminar 11
STOCKHOLM BRAIN INSTITUTE
The Hodgkin-Huxley current The Hodgkin-Huxley current equationequation
1,
1
,1
1
jj
jj
jj
jj
k
jkkj
m R
VV
R
VVVEG
dt
dVC
November 15, 2007 Albanova Instrumentation Seminar 12
STOCKHOLM BRAIN INSTITUTE
An action potentialAn action potential
Nobel Prize 1963
November 15, 2007 Albanova Instrumentation Seminar 13
STOCKHOLM BRAIN INSTITUTE
Synaptic transmissionSynaptic transmission
• Simple conductance based model• Square pulse, Gamma function• Voltage dependence (NMDA)
• Detailed model of single spine• Postsynaptic receptor kinetics• Biochemical networks• Neuromodulation
• Electrical synapses• Graded transmitter release• Synaptic plasticity
• Short-term, ms - s• Long-term, s – yrs
• …
November 15, 2007 Albanova Instrumentation Seminar 14
STOCKHOLM BRAIN INSTITUTE
Real neuronal networksReal neuronal networks
• Several types of different neurons• Huge numbers• Modules and layers
• Quite similar over areas and species!• Computing power limitation …
November 15, 2007 Albanova Instrumentation Seminar 15
STOCKHOLM BRAIN INSTITUTE
• GENESIS• NEURON• SPLIT simulator
• Hammarlund & Ekeberg 1998
• SPLIT parallel setup, optimization• Djurfeldt et al. 2005
• PGENESIS, parallel NEURON• PDC/KTH
• Lenngren, KTH/PDC• Blue Gene/L
• 1024 dual core nodes (1/64 of full machine)
Simulators and simulation ofSimulators and simulation oflarge-scale models at KTHlarge-scale models at KTH
November 15, 2007 Albanova Instrumentation Seminar 16
STOCKHOLM BRAIN INSTITUTE
A large-scale cortex modelA large-scale cortex model
November 15, 2007 Albanova Instrumentation Seminar 17
STOCKHOLM BRAIN INSTITUTE
Hebbian synapses and cell assembliesHebbian synapses and cell assemblies
”LTP”Bliss and Lömo, 1973Levy and Steward, 1978
Hebb D O, 1949: The Organization of Behavior
• Cell assembly = mental object
• Gestalt perception• Perceptual completion• Figure-background separation• Perceptual rivalry
• Milner P: Lateral inhibition
• After activity 500 ms• Persistent, sustained• Fatigue = Adaptation, synaptic depression
• Association chains• Temporally asymmetric synaptic plasticity
November 15, 2007 Albanova Instrumentation Seminar 18
STOCKHOLM BRAIN INSTITUTE
The KTH layer 2/3 modelThe KTH layer 2/3 model
• Top-down driven model of associative memory• Generic “association cortex”, layers 2/3• Modular: Minicolumns, hypercolumns• 3 different cell types: Pyramidal cells, Basket cells,
Regular Spiking Non-Pyramidal• 2 000 – 20 000 000 model neurons
117% 2.5 mV 230% 0.30 mV
70% -1.5 mV mV
70% 1.2 mV
70% 2.5 mV
25% 2.4 mV
November 15, 2007 Albanova Instrumentation Seminar 19
STOCKHOLM BRAIN INSTITUTE
Neuron-synapse propertiesNeuron-synapse properties
• Realistic amplitude of PSP:s in largest network model
• Sparse connectivity (stochastic)• Synaptic depression• Asymmetric cell-cell connectivity• 3D geometry delays
• 0.1 - 1m/s conduction speed
Tsodyks, Uziel, Markram 2000
Local basket cell
Local pyramidal
Local RSNP
Distant pyramidal
November 15, 2007 Albanova Instrumentation Seminar 20
STOCKHOLM BRAIN INSTITUTE
Network layoutNetwork layout
• 1x1 mm patch• 9 hypercolumns• Each hypercolumn
• 100 minicolumns• 100 basket cells
• 100 patterns stored• 29700 neurons• 15 million synapses
Active minicolumn
(30 pyramidal cells)
Active basket cell
Active RSNP cells
One of the 9 hypercolumns
November 15, 2007 Albanova Instrumentation Seminar 21
STOCKHOLM BRAIN INSTITUTE
9 hypercolumns9 hypercolumns
• 1x1 mm patch• 9 hypercolumns• Each hypercolumn
• 100 minicolumns• 100 basket cells
• 100 patterns stored• 29700 neurons• 15 million synapses
November 15, 2007 Albanova Instrumentation Seminar 22
STOCKHOLM BRAIN INSTITUTE
100 hypercolumns100 hypercolumns
4x4 mm
• 330000 neurons
• 161 million synapses
November 15, 2007 Albanova Instrumentation Seminar 23
STOCKHOLM BRAIN INSTITUTE
8 rack BG/L simulation8 rack BG/L simulation
• 22x22 mm cortical patch• 22 million cells, 11 billion synapses
• 8K nodes, co-processor mode• used 360 MB memory/node
• Setup time = 6927 s• Simulation time = 1 s in 5942 s• >29000 cpu hours• Massive amounts of output data• 77 % of linear speedup
• Point-point communication slows (?)
• Currently (inofficial) world record!Djurfeldt M, Lundqvist M, Johansson C, Rehn M, Ekeberg Ö, and Lansner A (2007): Brain-scale simulation of the neocortex on the Blue Gene/L supercomputer. IBM J R&D (in press)
November 15, 2007 Albanova Instrumentation Seminar 24
STOCKHOLM BRAIN INSTITUTE
The three different cell typesThe three different cell types3 sec simulation3 sec simulation
Basket
RSNP
Pyramidal
November 15, 2007 Albanova Instrumentation Seminar 25
STOCKHOLM BRAIN INSTITUTE
• 2000+ neurons• 250000+ synapses• 5 s = 600 s on PC
Lundqvist M, Rehn M, Djurfeldt M and Lansner A (2006). Attractor dynamics in a modular network model of the neocortex. Network: Computation in Neural Systems: 17, 253-276
November 15, 2007 Albanova Instrumentation Seminar 26
STOCKHOLM BRAIN INSTITUTE
Perception and associative Perception and associative memory performancememory performance
• Pattern reconstruction• Figure-background• Pattern completion and rivalry• 50 – 100 ms
• Sustained after-activity• 150 ms – 2 sec• NMDACa, KCa modulation
• Robust to parameter changes and scaling• Cortical long-range recurrent excitation strong
enough to support attractor dynamics
November 15, 2007 Albanova Instrumentation Seminar 27
STOCKHOLM BRAIN INSTITUTE
Attractor dynamics:Attractor dynamics:Pattern rivalryPattern rivalry
Fast ”decision” <100 ms!
November 15, 2007 Albanova Instrumentation Seminar 28
STOCKHOLM BRAIN INSTITUTE
Bimodal membrane potentialBimodal membrane potential
Jeffrey Anderson, Ilan Lampl, Iva Reichova, Matteo Carandini, and David Ferster. Stimulus dependence of two-state fluctuations of membrane potential in cat visual cortex. Nat. Neurosci., 3(6):617–621, 2000.
Log(pISI)
Exponential fit
November 15, 2007 Albanova Instrumentation Seminar 29
STOCKHOLM BRAIN INSTITUTE
Bistable activity with irregular firing, Bistable activity with irregular firing, similar to similar to in vivoin vivo recordings recordings
• Ground state stable only in larger networks with many patterns stored
• Increase in irregularity in active cortical states is a challenges for persistent activity models
• This L2/3 network model• displays irregular fluctuation driven low-rate firing• operates in a high-conductance regime of balanced
excitatory and inhibitory currents• is stable to synchronization even with blocked NMDAR
• Details under investigation
November 15, 2007 Albanova Instrumentation Seminar 30
STOCKHOLM BRAIN INSTITUTE
Attentional blink – effect of GABAAttentional blink – effect of GABA↑↑
• Attractor activation correlates with percentage of correct probe detections
• Time scales different but qualitatively similar results
0 40 80 120 160 200 240 280 3200
20
40
60
80
100
milliseconds
% a
cti
va
ted
att
rac
tors
GABA baseline
GABA 150%
November 15, 2007 Albanova Instrumentation Seminar 31
STOCKHOLM BRAIN INSTITUTE
Ongoing workOngoing work+ Layer 4
• Selective feature detectors• V1 model with
• learned orientation map (LISSOM) patchy horizontal L2/3 connectivity
+ Layer 5• Martinotti cells, local (delayed) inhibition to superficial
layers• Pyramidals, cortico-cortical connections
• Analysing L2/3 dynamics, spiking statistics, conductances, intracellular potentials• Non-orthogonal stored memories
• Better synthetic VSD, BOLD signals• Modelling interacting areas … using parallel NEURON• Scalable abstract connectionist cortex model
• Cortical area module, on-line learning, network-of-networks,…
November 15, 2007 Albanova Instrumentation Seminar 32
STOCKHOLM BRAIN INSTITUTE
1980 1985 1990 1995 2000 2005 2010 2015 2020 2025
Computing PowerComputing PowerMoore’s law …Moore’s law …
year
GF
LO
P
IBM BlueGene/L 128K cores
Next generation supercomputers
>1M cores
100 ops/synapse/ms?
November 15, 2007 Albanova Instrumentation Seminar 33
STOCKHOLM BRAIN INSTITUTE
EU/FACETS – analog VLSIEU/FACETS – analog VLSI From cortex physiology to VLSIFrom cortex physiology to VLSI
EU/GOSPEL – NoE in Artificial olfactionSSF/Stockholm Brain Institute (SBI)OECD/INCF – International Neuroinformatics Coordinating Facility
November 15, 2007 Albanova Instrumentation Seminar 34
STOCKHOLM BRAIN INSTITUTE
ConclusionsConclusions
• Computational models are enabling tools in brain science
• Human brain level computing power in 10-15 yrs
• Brain mysteries likely to be largely uncovered at that time
• A principled understanding of brain function will emerge• Great benefits!
• Brain-like computing and AI• Consequences for society…?
November 15, 2007 Albanova Instrumentation Seminar 35
STOCKHOLM BRAIN INSTITUTE
CollaboratorsCollaborators
• Model development• Mikael Lundqvist, PhD student
• David Silverstein, Phd student
• Parallel simulation• Mikael Djurfeldt , PhD student
• Örjan Ekeberg, Assoc Prof
• Data analysis• Martin Rehn , postdoc