ONLY CONNECT

David WillshawInstitute for Adaptive & Neural Computation

School of InformaticsUniversity of Edinburgh

willshaw@inf.ed.ac.uk

ONLY CONNECT

Computational thought Hamming Seminars

Bell Labs Radar Family history

Bell Labs Information Theory My research

ONLY CONNECT

Only connect ! That was all her sermon. Only connect the pride and the passion and both will be exalted, and human love will be seen at its height. Live in fragments no longer. Only connect.....

From Howard’s End by E M Forster

This is my research area:

COMPUTATIONAL MODELLING OF THE DEVELOPMENT OF NEURAL

CONNECTIVITY

Why is this an important area?

Why is this an important area now?

Without the correct specific connectivity patterns between our neurons we cannot function correctly

We don’t yet know the mechanisms for how the brain is wired up

Computational modelling is used to explore particular hypotheses and suggest experiments to try to understand the underlying mechanisms

New technologies are giving us much better data about connectivity

Human Brain – MRI scan(Wellcome Images; Mark Lythgoe, Chloe Hutton)

7

Cerebellum

The cerebellar cortex contains nerve cells of several different types

8

Cajal(1905)

Purkinje cell (Wellcome Images; David Becker)

9

Shows the Purkinje cells lined up and the

parallel fibres (Cunningham, 1913)

10

Purkinje cells and parallel fibres (Wellcome Images; Spike Walker)

11

Vertebrate retina

13

Vertebrate retina

Visual pathways in mammals

15

Ocular dominance columns

The binocular projection from retina to cortex in mammals

Zebra stripes?

Reminiscent of Turing Patterns postulated to be formed in morphogenesis by mechanisms of reaction-diffusionAM Turing , Phil. Trans. Roy. Soc. B, 237, 37-52, 1952

Computational modelling in neuroscience

17

Molecules

Synapses

Neurons

Networks

Systems

CNS

1 cm

100 m

10 cm

m

m

A

Maps1 cm

19

Modelling at the nerve cell level (Wellcome Images; Benedict Campbell)

1952: The first computational neuroscience model

A quantitative description of membrane current and its application to conduction and excitation in nerve.

Hodgkin & Huxley, J Physiology (1952)

The Hodgkin-Huxley model

Impulse propagation caused by flow of K+ and Na+ currents through separate channels in the membrane

Permeability to ion flow in these channels is dependent on the potential difference across the membrane

Modelling a segment of the axon as an electrical circuit where the resistances are

voltage dependent

HH equations account for all the data

24

A model at the network level

25

Learning and Memory:Hebb’s rule

‘When an axon of cell A is near enough to excite cell B or repeatedly or consistently take part in firing it, some growth or metabolic change takes place

in one or both cells such that A’s efficiency, as one of the cells firing B, is increased.’

Hebb (1949)

A

B

C

26

Hebb's rule and associative memory

Distributed Memory:

The Associative Net (Willshaw, Buneman & Longuet-

Higgins, Nature, 1969)

Clipped Hebbian rule

Pre0 1

Post

0 0 01 0 +

Modelling of the development of nerve connections

27

J.F. Tello

Polyneuronal innervation in foetal human muscle (1917)

Connections between neonatal nerve and muscle(Wellcome Images, Ribchester & Gillingwater)

Visual pathways in mammals

30

FROG BRAIN

CAT BRAIN

Xenopus tadpoles

32

Frogs and toads

33

Xenopus

Frog visual system

34

What is the mechanism for the formation of ordered maps of nerve connections?

Both flexibility and rigidity in connection pattern are seen - probably more than one mechanism act together?

35From Jacobson (1967)

The main theories

36

1. Chemoaffinity – molecular cues guide each axon to its target cell or cells (usually associated with rigidity of connection)

2. Electrical signalling - e g, nearby cells that fire together may be more active than more distant cells and so can signal neighbour relations to the cells to which they are connected – usually associated with flexibility of connections.

So what is the link with Informatics?

37

“Informatics” means different things to different people?

38

“When I use a word” Humpty Dumpty said rather in a scornful tone “It means what I choose it to mean –neither more nor less”

Alice Through the Looking Glass, Lewis Carroll

39

Neuroinformatics INCFinformation-processing in the nervous system

Computational Models

inspire new hardware and software methodsNeural Engineering

collect, analyze, archive, share, simulate and visualize data and models

Software Systems

40

Neuroinformatics INCFINCF – International Neuroinformatics Coordinating Facility (www.incf.org)

An international organisation subscribed to by 15 governments

Dedicated to the coordination of neuroinformatics world wide.

Each country has its own local organisation; I am the UK Coordinator and scientific representative at INCF

41

Mike Fourman’s formulation:

“Informatics is the study of how natural and artificial systems store, process and communicate information”

42

The School of Informatics at Edinburgh is inclusive rather than exclusive.

++: Aren’t we lucky to be not constrained!

43

The School of Informatics at Edinburgh is inclusive rather than exclusive.

++: Cross-fertilisation

--: Because of the breadth there is a danger that individuals have a lack of understanding of other fields of research practised in Informatics

44

A snapshot, which I prepared for Mike Fourman, of the interactions between academic and research staff in the three departmental groupings in 1997, prior to the formation of the School of Informatics

AI/AIAI: Artificial Intelligence/AI Applications InstituteDCS: Department of Computer ScienceCCS/HCRC: Centre for Cognitive Science/Human Communications Research Centre

45

The importance of technology in the computational modelling of the

nervous system

Julius Bernstein (1839-1917), after whom the Bernstein Centres for Computational Neuroscience in Germany are named.

His membrane theory of the propagation of the nerve impulse (1902) was almost right.

But his equipment for measuring the properties of the nerve impulse was inadequate

Alan Hodgkin

Andrew Huxley

Once Hodgkin had been to Chicago (50 years later) to learn how to build an amplifier, he and Huxley could collect, analyse and model the required data, leading to a Nobel Prize for them

49

My current research problem

What is the mechanism for the formation of ordered maps of nerve connections?

Both flexibility and rigidity in connection pattern are seen - probably more than one mechanism act together?

50From Jacobson (1967)

The main theories

51

1. Chemoaffinity – molecular cues guide each axon to its target cell or cells (usually associated with rigidity of connection)

2. Electrical signalling - e g, nearby cells that fire together may be more active than more distant cells and so can signal neighbour relations to the cells to which they are connected – usually associated with flexibility of connections.

52

An example of mechanism 2: nearest neighbour interactions through correlated neural activity

According to the neural activity model, spontaneous electrical activity drives the process.

By a Hebbian-type mechanism, connections between neighbouring retinal cells and neighbouring tectal cells are strengthened; those between non-neighbours are weakened (Willshaw and von der Malsburg 1976).

53

New technology furnishes higher quality data:Mouse superior colliculus maps (Cang et al, J. Neurosci, 2008)

The colour-coded noisy X, Y coordinates of the receptive fields of each small part of a 2 mm square brain area including colliculus

X

Y

54

Distribute ‘recording’ positions

regularly over the colliculus

100m

55

Then join up nearest neighbours to form a lattice

56

Translating colours into field positions, plot out the receptive field position for each recording point, averaging over nearby collicular points

Then project the collicular lattice into the field

Colliculus Field

Wild type (normal)20 recording points, or nodes

57

Projection of collicular lattice to field 50 m separation between nodes

Projection of field lattice to colliculus

Wild type (#006, 170 nodes)

Theories for the formation of nerve connections can be tested in mice for which the genome is known

Genes that are thought to be determining developmental mechanisms can be manipulated

Their effects on connectivity can be observed and compared with the model predictions

58

Beta2 knockout

Knockout of the Beta2 component of the acetylcholine receptor is thought to diminish the strength of the correlated firing activity in the retina

And hence the precision of the map?

59

60

Remove correlated activity in a Beta2 knockout

Colliculus to field

Orientation: 19+/-17 degrees

The largest connected ordered submap – covers 138/145 nodes (95%)

Modern evidence for molecular guidance cues

from McLaughlin, Hindges and O’Leary (2003)

61

Are these molecules used in map-making?

If Ephs and ephrins are the labels of chemoaffinity, then changing them should result in abnormal patterns of connectivity

62

63

If all these ephrins are knocked out will the mapping along the rostrocaudal axis be destroyed??

EphrinA triple knock out

Three ephrinAs, ephrinA2, ephrinA3 and ephrinA5, are thought to label the rostrocaudal axis of the colliculus.

64

Remove molecular cues:Heterozygote triple EphA knockout[A2-/-A3+/-A5-

Largest ordered submap covering 90% of the nodes

Normal maps have quite high precision

Removing genes controlling activity cues and molecular cues still does not destroy the order

New models are needed!

65

In summary:

Collaborations are fun

66

67

At the 1st Connectionist

Summer School, Pittsburgh (1986)

68

69

Showsthe fledgling algorithm applied to a 30-city problem (Hopfield & Tank, 1985).

As presented to the 1986 Connectionists summer school

70

Application of the Elastic Net to a 100-City Problem

(Durbin & Willshaw, Nature, 1988)

Connections with the Bell Labs of Hamming’s era

Scientists at Bell Labs prided themselves in researching fundamental problems whereas in reality they were employed by a company to develop products

Is there a duality here? In this university or in Informatics many of us are employed to do fundamental research yet there is increasing pressure to go after the WOW Factor

71

But there has to be something behind the WOW

72

WOW!

But there has to be something behind the WOW

73

WOW!

Connections with Hamming’s aphorisms

from his Bell Communications Research Seminar

“You and your research”

74

1. “You have to learn to sell yourself, to write clearly”

75

1. “You have to learn to sell yourself, to write clearly”

2. “The closed door is symbolic of a closed mind”

76

1. “You have to learn to sell yourself, to write clearly”

2. “The closed door is symbolic of a closed mind”

3. Against working single-handedly with total control:

“If you learn to work with the system, you can go as far as the system will support you

77

1. “You have to learn to sell yourself, to write clearly”

2. “The closed door is symbolic of a closed mind”

3. Against working single-handedly with total control:

“If you learn to work with the system, you can go as far as the system will support you

4 “If you do some good work you will find yourself on all kinds of committee and unable to do any more work.”

78

1. “You have to learn to sell yourself, to write clearly”

2. “The closed door is symbolic of a closed mind”

3. Against working single-handedly with total control:

“If you learn to work with the system, you can go as far as the system will support you

4 “If you do some good work you will find yourself on all kinds of committee and unable to do any more work.”

5 “Plant the little acorns from which the mighty acorns grow”

79

If you want to find out more about computational modelling in neuroscience, you could consult our forthcoming Cambridge University Press book:

Principles of Computational

Modelling in Neuroscience

by

David Sterratt Bruce Graham

Andrew Gillies David Willshaw

PDF of almost-final version available on request!

80