latency, duration and codes for objects in inferior …...latency, duration and codes for objects in...
TRANSCRIPT
Latency, duration and codes for objectsin inferior temporal cortex
Gabriel Kreiman, Chou Hung, Tomaso Poggio, James DiCarlo
Center for Biological and Computational LearningComputation and Systems Biology Initiative
McGovern Institute for Brain Research
Massachusetts Institute of Technology
Recognition can be very fast
Stimulus presentation
100 ms100 ms
time
Passive viewing(fixation task)
5 objects presentedper second
•Recordings of spiking activity from macaque monkeys
•Recordings in an area involved in object recognition (inferior temporal cortex)
•10-20 repetitions per stimulus
• Presentation order randomized
• 77 stimuli drawn from 8 pre-defined categories
Recordings made by Chou Hung and James DiCarlo
Reading the neuronal code
1 2 3 4 5
Neuron 1 Neuron 2 Neuron 3 Object
Yes No No 1
Neuron 1 Neuron 2 Neuron 3 Object
Yes No No 1Yes Yes No 2Yes Yes Yes 3
Neuron 1 Neuron 2 Neuron 3 Object
Yes No No 1
Yes Yes No 2
Mind reading Neuronal readingCan we read out what the monkey is seeing?
xLearning from (x,y) pairs y ∈ {1,…,8}
Input to the classifier
100 ms0 ms 200 ms 300 ms
w
MUA: spike counts in each bin
LFP: power in each bin
MUA+LFP: concatenation of MUA and LFP
Accurate read-out of object category and identity from a small population
Hung*, Kreiman*, Poggio, DiCarlo. Science 2005
Decoding requires very few spikes
w = 12.5 ms
Hung*, Kreiman*, Poggio, DiCarlo. Science 2005
Local field potentials (the “input”) also show selectivity
Kreiman*, Hung*, Kraskov, Quiroga, Poggio, DiCarlo. Neuron 2006
MUA: multi-unit spiking activity
SUA: single-unit spiking activity
LFP: local field potentials
MUA & LFP: MUA combined with LFP
The classifier extrapolates to new scales and positions
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7 n=31 n=24 n=9 n=9 n=17 n=10
chance
TRAIN
TEST
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7 n=31 n=24 n=9 n=9 n=17 n=10
chance
TRAIN
TEST
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7 n=31 n=24 n=9 n=9 n=17 n=10
chance
TRAIN
TEST
Other observations
• We can decode information from local field potentials. MUA+LFP > MUA > LFP
• Feature selection significantly improves performance. Choosing the “best” neurons >> randomly selecting neurons
• We can decode the time of stimulus onset
• We can also read out coarse “where” information
• Decoding is robust to internal and external perturbations
• The population can extrapolate to novel pictures within known categories
We can decode object information from the model units
Serre, Kouh, Cadieu, Knoblich, Kreiman, Poggio. MIT AI Memo 2005
The model shows scale and position invariance
Latency, duration and codes for objectsin inferior temporal cortex
Gabriel Kreiman, Chou Hung, Tomaso Poggio, James DiCarlo
Center for Biological and Computational LearningComputation and Systems Biology Initiative
McGovern Institute for Brain Research
Massachusetts Institute of Technology