Principles of neuronal coding

Grand Challenges WorkshopInstitute of Medicine Neuroscience Forum

25 June 2008

William BialekJoseph Henry Laboratories of Physics

Lewis-Sigler Institute for Integrative GenomicsPrinceton Center for Theoretical Science

Princeton University

http://www.princeton.edu/~wbialek/wbialek.html

The enormous variety of our sensory experiences,

abstract thoughts,

and actions

all are represented in our brain by sequences of identical electrical pulses,

called action potentials or “spikes.”

This is the problem of neural coding.

What are the elementary symbols in the code, and what do they represent?

Classical ideas about “rate,” evolution of ideas about “timing.”Classical ideas of feature selectivity, modern version as “dimensionality reduction.”

How are features extracted (invariantly, robustly)? Coding meets computation.*Are populations of cells more than the sum of their parts?

Why does the code have the structure that it does, as opposed to any other possible structure?

*Has the brain chosen codes that are, in some precise mathematical sense, optimal?Could this be a principle from which aspects of neural function can be predicted?

Are there principles that could unify our understanding of information flow in biological systems?

There is no way to give an exhaustive account of these efforts here, so I’ll choose some examples*(shamelessly drawn from work done by my colleagues and myself), and then point to lessons for the future.

Note: I am going to hide almost all of the math, but I give references.

Are populations of cells more than the sum of their parts?

cell #1

cell #2

cell #3

cell #4

cell #5

cell #6

cell #7

cell #8

cell #9

cell #10

states or patterns

a moment ago

no spike

spike no spike

no spike

no spike

spike no spike

spike no spike

spike 100010101

right now spikeno

spikeno

spike spikeno

spikeno

spikeno

spikeno

spike spikeno

spike 1001000010

the next moment

no spike

no spike spike

no spike spike

no spike spike

no spike

no spike

no spike 10101000

states of the network = binary patterns of spiking and silence

A common observation in many brain areas (including cortex) is that when we look at pairs of

cells, the 1s and 0s are only weakly correlated.

(e.g., pairs of ganglion cells in the vertebrate retina as it responds to natural movies)

Does this mean we can ignore correlations?Or could something dramatic be hiding in these

rather mundane observations?Analogous problems in statistical physics lead us

to be suspicious of widespread correlations ...

Although pairwise correlations are weak, there are collective effects in larger populations ...

What can pair correlations tell us about the distribution of states in the whole network?

In general, nothing. But there is a unique “simplest” or “least structured” model

consistent with the measured correlations(technically, the maximum entropy model).

Weak pairwise correlations imply strongly correlated network states in a neural population. E Schneidman, MJ Berry II, R Segev & W Bialek,

Nature 440, 1007-1012 (2006).

Ising models for networks of real neurons. G Tkacik, E Schneidman, MJ Berry II & W Bialek, arXiv.org/q-bio.NC/0611072 (2006).

(an example of N=40 cells)

independent neurons

simplestpairwisemodel

(an example of N=10 cells)

These minimal models are exactly the Ising model in statistical mechanics, and

also the Hopfield model of neural networks. Here these “physics-style” models emerge directly from the data!

Multiple attractors, phase transitions: more than the sum of its parts!

Optimal information transmission?In an efficient neural code, the input/output relation of neurons is matched to the distribution of inputs.Intuitively, we want the cell to be driven through its full dynamic range. This can be formalized by asking for a code that maximizes information transmission.

A simple coding procedure enhances a neuron's information capacity. SB Laughlin, Z Naturforsch 36c, 910-912 (1981).

10

1

0.1

0.01

0.001

1000-100stimulus

angular velocity (deg/s)

relativespike rate

distribution of input stimuli has large dynamic range

distribution of input stimuli has small dynamic range

Adaptive rescaling optimizes information transmission. N Brenner, W Bialek & R de Ruyter van Steveninck, Neuron 26, 695-702 (2000).

Efficiency and ambiguity in an adaptive neural code. AL Fairhall, GD Lewen, W Bialek & RR de Ruyter van Steveninck, Nature 412, 787-792 (2001).

Unifying principles?

Early development of the Drosophila embryomother puts mRNA

for Bicoid at (future)

head of the embryo

Bicoid activates expression

of Hunchback

intensity of bcd stain~ bcd concentration

inte

nsit

y of

hb

stai

n

Here we measure the input/output relations and noise ... Can we predict the distributions? Same principle as for neurons!

Information flow and optimization in transcriptional regulation.G Tkacik, CG Callan Jr & W Bialek, arXiv:0705.0313 [q-bio.MN] (2007).

Probing the limits to positional information. T Gregor, DW Tank, EF Wieschaus & W

Bialek, Cell 130, 153-164 (2007)

(no free parameters!)

Hunchback expression level

Prob

abili

ty d

ensi

ty

Theory (not just models) as a full partner with experiment

Maximum entropy for pairwise + recording many cells simultaneously leads to reconcile weak correlations with strong collective behavior;connect experiments directly to large body of neural network models;strength of correlations seems “just enough” for critical behavior.

Information maximization + measuring distributions (not means) leads to new forms of adaptation in the neural code;new questions about mechanisms;common principles across neural and genetic signaling.

Crucially, theory doesn’t just come after exp’t, it can and should come before: pointing to the next interesting experiment, and

motivating larger scale, more quantitative measurements.

A grand challenge: principled (not parameterized!) theories for real networks.