how to design a cognitive system - elsevier.com

P A R T

1

How to Design a Cognitive System

The growing complexity of computer systems is a catalyst for their designers to lookto nature for ideas.

Part Contents

IntroductionSelf-organization in the Nervous SystemLarge-scale, Small-scale Systems

Morris-01.qxd 2/23/05 5:03 PM

Morris-01.qxd 2/23/05 5:03 PM

As computer systems become more com-plex, the likelihood of system failure increasesaccordingly. The designers of tomorrow’scomputer systems are starting to include theability of the system to self-repair at the topof their list of desirable characteristics.Scientists at IBM have recently come upwith a list of characteristics for the next gen-eration of computers which not onlyincludes the ability to self-repair but also theability to self-organize, the ability to adaptto changing environments or workload, theability to interact with other systems in adynamic way and the ability to anticipateusers’ actions.

The chapter ‘Large-scale, small-scale sys-tems’, written by Jim Austin, Dave Cliff,Robert Ghanea-Hercock and Andy Wright,sets out to present a biology-inspired view ofwhat these complex adaptive systems mightbe. As with neurobiology, they consider sys-tems made up of large numbers of relativelysimple components, for example ultra-massiveparallel processors. The components of thesesystems may interact in non-linear ways.These can then give rise to large-scale behav-iour which cannot necessarily be predictedfrom knowledge of the characteristics of theindividual components and their small-scalelocal interactions. This phenomenon hassometimes been described, perhaps unhelp-fully, as emergent behaviour or computation.

Cliff and Wright take the reader on awhistle-stop tour of artificial intelligence (AI),

at the beginning of which they elegantlydescribe the engineering approach as ‘seek-ing simply to create artificial systems thatreliably exhibit some desired level of cogni-tive performance or behaviour’. They pointout that, for much of its history, research inAI largely ignored biology. This is changing,prompted in part by the realization that theincreasingly large computing systems beingdesigned today are becoming more difficultto build and control. (It is no accident, how-ever, that the Foresight Project adopted theall-inclusive banner of cognitive systems,rather than that of artificial intelligence.)

Biology also strongly influences a recentdevelopment in AI, autonomous agents. Cliffand colleagues define autonomous agentsas ‘entities that are capable of coordinatingperception and action, for extended periodsof time, and without human interven-tion, in the pursuit of some set of goals’. Theyinclude in their review both physical auto-nomous agents, such as robots, and agentswith no physical embodiment, such as software agents that exist purely in virtualenvironments.

The interest in gathering insights frombiology has been fuelled by the increasingavailability of data concerning the propertiesand behaviour of the elements of complexbiological systems at the individual level, bethey genes, proteins or cells. David Willshaw,in Chapter 1, argues that one unifying prin-ciple of organization is self-organization,

Introduction: How to Design aCognitive System

Lionel Tarassenko and Richard Morris

3

Morris-01.qxd 2/23/05 5:03 PM

which is found throughout the biologicaland physical world.

Many of the intricate patterns seen innature, such as the patterns of zebra stripes,the paths formed by social insects, cloudconvection and snow-flake patterns, are exam-ples of self-organization. Willshaw definesself-organization as ‘those aspects of organ-ization that result from interactions betweenthe elements of the system as well as externalinfluences that do not themselves provideordering information’. He identifies threeforms of self-organization: self-organizationin development, self-organization as a com-plement to experiential changes and self-organization as a complement to damage.More than half of the chapter is devoted tothe first of these.

During development, self-organizationrelieves the genome of much of the burdenof specifying the exact numbers and pos-itioning of nerve cells and the connectionsthat they make. The internal, self-organizingdynamics combine with external influences,such as random activity in the participatingnerve cells.

There is much less that can be said abouthow self-organization operates during cog-nitive development, within the processes ofmemory storage and retrieval and as aresponse to insult, in all cases acting againsta background of continual neural change.Willshaw in Chapter 1 and Austin and col-leagues in Chapter 2 agree that knowledgeabout how the nervous system continuallyself-organizes in response to change will berelevant to the design of artificial cognitivesystems.

One issue to be faced in the design of thelarge distributed systems described byWright is how to organize large amounts ofdata for efficient storage and rapid retrieval.Willshaw suggests that these large-scalesystems may need to rely on softwareagents that independently harvest informa-tion for integration and self-organize tomaximize their utility to the overall system.

Cliff and colleagues in Chapter 2 paint apicture of a future in which federated net-works of computing facilities will housetens of thousands of servers, all connectedon an ultra-high bandwidth network andproviding computing on demand. Thesefacilities, which could come on-streamwithin the next five years, will use tech-niques inspired by biology to provide self-healing resilience to load fluctuations,component failures and attack by computerviruses and worms.

Willshaw speculates that the self-organizing capabilities of complex biologicalsystems could help to create a new genera-tion of hardware devices that dynamicallyand organically reconfigure themselves. Thisis echoed in the 20-year ‘vision’ sketchedout by Cliff and colleagues where silicon isno longer the dominant substrate for com-puting devices, being replaced instead bygenetically engineered organic substrates.However, Cliff and Ghanea-Hercock alsopoint out that this vision of the future isthreatened by the pace of developments inquantum computing. Does self-organizationplay a part at the quantum level?

4 INTRODUCTION: HOW TO DESIGN A COGNITIVE SYSTEM

Morris-01.qxd 2/23/05 5:03 PM

1 INTRODUCTION

The term self-organization is commonlyheld to describe the process by which indi-viduals organize their communal behaviourto create global order by interactionsamongst themselves rather than throughexternal intervention or instruction. Despitethis term receiving only scant mention indictionaries, it has been used to describemany different types of activities. Theclouds formed by birds in the sky, the coord-inated movement of schools of fish or thepaths formed by ants, as well as the intricatepatterns seen in snowflakes are all theresults of self-organization. Other complex

examples of spatial patterns are the manyman-made or natural crystal structures.

In physics, the simplest examples areclosed systems, where the system acts inde-pendently of external influences. The futurestate of the system is then controlled by itsconstitutive elements. Crucially, the emer-gence of a global pattern of order requiresinteractions between elements. Cooperativeinteractions will iron out local variationswhereas competitive interactions will exag-gerate them.

In the visually stunning Belouzov–Zhabotinsky reaction, two chemicals inhibiteach other’s autocatalysis, resulting in strik-ing periodic changes in colour, as indicated

C H A P T E R

1

Self-organization in the Nervous System

David Willshaw

1 Introduction1.1 Self-organization in the Nervous System1.2 Outline of the Chapter

2 Self-organization in Development2.1 Self-organization and Pattern Formation2.2 Making the Correct Numbers of Cells: Cell Death2.3 Development of Connections

3 The Role of Self-organization in Experiential Change3.1 Feature Maps3.2 Self-organization and the Acquisition of Cognitive Function

4 Self-organization as a Response to Damage4.1 Self-reorganization4.2 Can the Nervous System Regenerate After All?

5 Open Questions5.1 Questions for the Neurosciences5.2 Inspiration for Other Sciences – ‘Cognitive Systems’

5

Morris-01.qxd 2/23/05 5:03 PM

by an appropriate dye. In magnetic materials,it is energetically favourable for the dipoles ofneighbouring atoms to co-align, resulting in aglobal magnetic field. An example wherethere is a simple external influence is found ina laser. At low levels of excitation, individualatoms emit their light independently to pro-duce incoherent light; at higher levels, theemission of light from all the atoms becomeshighly coordinated through local interactions,producing coherent light.

Many systems exhibit both competitionand cooperation. A well-analysed exampleof a temporal pattern of self-organization inbiology is found in the statistics of the popu-lations of hares and their predators, lynxes,as recorded by the Hudson Bay TradingCompany in Canada between 1849 and 1930(Murray, 1993). Analysis of the number ofpelts collected suggests the following pat-tern of events: a large fluctuation in onepopulation can upset equilibrium states, inwhich the rates of reproduction and deathof both species balance out. For example, adecrease in the number of prey will cause acorresponding decrease in the number ofpredators, who will have less food. The pres-ence of fewer predators will then increasethe number of prey and consequently willincrease the number of predators, until finallythe preys will decrease in number again.This pattern of events will repeat over andover again, yielding the cyclical variation inboth prey and predator numbers over timethat is seen in the records. Clearly this behav-iour emerges from interactions betweenlynxes and hares and thus is an example ofself-organization.

1.1 Self-organization in the NervousSystem

As the words suggest, order in a self-organizing system emerges through localinteractions between individuals in theabsence of any external influence. As ahighly complex and dynamic system involv-ing many different elements interacting witheach other, the nervous system displaysmany features of self-organization. However,

there will be very few, if any, examples of trueself-organization within the nervous system.

It is very likely that the organization ofregions of the nervous system depends onexternal influences, either from other regionsof the nervous system of the body or underthe influence of external stimuli, such as sens-ory stimulation from the outside world. Theresulting organization will be the result ofinteractions between the elements of the sys-tem itself as constrained by the particularboundary conditions that are in force,together with ongoing external influences.

1.2 Outline of the Chapter

I take the term self-organization to referto those aspects of organization that resultfrom interactions between the elements ofthe system as well as with external influ-ences that do not themselves provide order-ing information. I identify three forms ofneural self-organization, which I shall dis-cuss in turn. These are:

● Self-organization in development Since akey challenge in our understanding of thenervous system is to comprehend howsuch a highly structured yet complex sys-tem can emerge from a single fertilizedegg, many phenomena displaying self-organization are concerned with how thenervous system develops. Many of thesedevelopmental processes are a result ofinteractions within the system itself.External influences exist but they can beregarded as initial constraints or bound-ary conditions acting on the system.

● Self-organization as a complement to experi-ential changes This refers to later stagesin development, when self-organizationplays a role along with other mechanismssuch as those involving external signalsarising from the sensory environment. Iexamine the effects of external influencesonly when these do not contain any pat-terning information. Therefore I do notdiscuss the neurobiology of learning andmemory, where specific patterns of activ-ity are required to be stored in or recalledfrom the system.

6 1 SELF-ORGANIZATION IN THE NERVOUS SYSTEM

Morris-01.qxd 2/23/05 5:03 PM

● Self-organization as a complement to damageThe adult nervous system can respond tosurgical or accidental damage. The facilityfor damaged brain to regenerate is eitherminimal or non-existent, which impliesthat the brain can self-organize, allowinghealthy regions to take over functions pre-viously carried out by other regions.

Section 2 considers development. I intro-duce some concepts of development at thegenetic and molecular level. I then describeself-organization in the formation of patternwithin collections of cells (section 2.1), inproducing the correct numbers of cells (sec-tion 2.2) and in the formation of orderednerve connections (section 2.3).

In section 3, I look at the role of self-organization in experiential changes. Sec-tion 3.1 describes the self-organization ofpatterns of feature selectivity in the cortexand section 3.2 provides a brief introductionto the self-organization of cognitive function.

Section 4 is concerned with self-organization as a response to injury, princi-pally in the adult.

Finally, in section 5 I discuss some openquestions that are relevant to the subject ofthis essay. Section 6 gives a short reading list.

2 SELF-ORGANIZATION INDEVELOPMENT

Generating nerve cells of the right type,in the right numbers, in the right places andwith the right connections is a formidabletask. It involves cell division, cell migration,cell death and the formation and withdrawalof synapses. The essential steps of embry-onic development are reviewed in manybooks. Wolpert (1991) provides a simple read-able introduction: Price and Willshaw (2000)discuss mammalian neural development.

Every organism is defined by the sets ofgenes in its genome. This contains the initialinstructions from which development pro-ceeds. The set of three-letter ‘words’obtained by reading the sequence of basesalong the DNA defines a sequence of amino

acids. Proteins are made out of amino acidsand cells are made out of proteins.

There has been considerable progress inour understanding of how genes controldevelopment. The fruit fly, Drosophilamelanogaster, has been used intensively ingenetic research for many decades. It is small,has a short life cycle of two weeks, and largenumbers of mutants have been identifiedand studied. The combination of the exten-sive knowledge of mutants and experimentalembryological and molecular biological tech-niques has provided a profound understand-ing of the genetic regulation (i.e. control) ofdevelopment in this species.

Remarkably, not only have many of thecontrol mechanisms that operate in Drosophilabeen conserved in mammals, but so havemany of the genes themselves. It is nowcommonplace to use information obtainedfrom studies of Drosophila to search for spe-cific regulatory genes in higher species and toformulate hypotheses regarding the generalprinciples that underlie development in allorganisms. In particular, work on Drosophilahas provided a comprehensive understand-ing of how different regions of a developingorganism can develop regional specificity.For example, certain morphogens – mol-ecules that control the development of form,or morphogenesis, a term coined by Turing(1952) – are distributed in gradients in theearly Drosophila embryo. They evoke differ-ent cellular responses at different concentra-tions, specifying the expression patterns ofother genes that themselves regulate later-expressed genes. In this way, complex pat-terns of later-expressed genes emerge toconfer positional identity on cells at eachposition in the embryo. The combined actionof the specific cocktail of regulatory genesthat each cell expresses is essential for con-ferring on each cell a particular phenotypeappropriate for its position.

Many groups have shown that verte-brates have genes that are similar to those ofDrosophila. Researchers have found verte-brate homologues for Drosophila genes thatact within cells to regulate the expression ofother genes (transcription factors) or that

2 SELF-ORGANIZATION IN DEVELOPMENT 7

Morris-01.qxd 2/23/05 5:03 PM

signal between cells to control processessuch as axonal guidance. A good example oftranscription factors is the large family of Hoxgenes (members of the homeotic clusters ofgenes) in mouse which have homology tothe genes of the Antennapedia complex ofDrosophila and which regulate the identityof segments of the Drosophila body. Anothergood example of conservation of develop-mental mechanisms is in the guidance ofaxons. Many of the receptor systems thathave been implicated in this process arehighly conserved between Drosophila, otherinvertebrate species and mammals.

It might be thought therefore that thegenome could hold a coordinate-by-coordinate blueprint of the nervous system,specifying where each nerve cell is to be sit-uated, what its functional properties are tobe, and which other cells are to be contacted.If homeotic genes control the production ofgross anatomical structure and cell differen-tiation, is it not possible that subordinate genefamilies subsequently control the remainingdevelopment processes? This is extremelyunlikely given the large numbers of nervecells and the many more connections thatthey make compared to the relatively smallsize of the genome.

If the 1014 connections between the 1010

neurons of the human neocortex were madeat random, this would require at least 1015

bits of information compared to the 109 bitsin the human genome. It is more likely thatthe genome contains the ‘rules of develop-ment’. For example, it is well known that theconnectivity of the brain is highly struc-tured, with topographic maps foundbetween many sensory structures and neo-cortex. The rules of development wouldspecify the general features of the mappingand the fine details could be arrangedthrough interactions between the constituentparts. Many of the features of the systemare, so to speak, arranged by the nervoussystem itself, or self-organized. The nextthree subsections describe different parts ofthe developmental process where mecha-nisms of self-organization make an importantcontribution.

2.1 Self-organization and PatternFormation

A central aim of developmental biologyis to understand how cells in different pos-itions develop differently; i.e. how regionalspecification comes about. This is as true forthe development of a structure as complexas the cerebral cortex – where each point inthe dorsal telencephalic wall (the precursorof the cortex) acquires a unique functionalproperty, with relative invariance in the lay-out of these properties between the individ-uals of the same species – as it is for thedevelopment of the five distinct digits of thehand or the patterns of markings on sea-shells, zebras or leopards. Many of the prin-ciples that govern the ways in whichregional specification arises during devel-opment have been identified in studies ofearly embryogenesis. We are relatively ignor-ant of the mechanisms that control brainregionalization, which makes it all the moreimportant to generate hypotheses withknowledge of principles deduced fromstudies of earlier developing systems.

The key questions are:

● How does a mass of developing cellsacquire differences one from another?

● How is this information used to deter-mine their different fates?

These two questions are interlinked. Forexample, does each cell acquire its ownidentity independently from the instruc-tions in the genome? Or do certain cells actas organizers and instruct their neighbours,which happens in limb morphogenesis?

Turing (1952) investigated one possibilitytheoretically. He analysed the emergence ofpattern in a collection of cells and showedthat, starting from a uniform concentrationof morphogens, which interact with eachother and diffuse between cells (giving riseto the term reaction-diffusion), the patternsof molecular concentration produced overthe cell population had peaks and troughsdefining characteristic periodicities. Turingargued that in a developing organism, a setof chemicals could be used in this way to set


Morris-01.qxd 2/23/05 5:03 PM

up a prepattern that will determine specificfeatures of the organism. The patterns arenot preprogrammed and emerge throughself-organization.

In the brain, generation of regional differ-ences has to occur at different levels. Howare particular areas or regions within a brainnucleus or system distinguished one fromanother? How is cellular identity within agiven region determined? I now discussregional specification within the forebrainand within the neocortex. I discuss specifi-cation within a brain region in section 2.3.

2.1.1 Pattern Formation in theSpecification of Forebrain

Much of our understanding derives fromwork on the amphibian Xenopus laevis. Twoimportant terms used to describe comple-mentary mechanisms that can generateregional specification are mosaicism andregulation, i.e. whether the development ofan individual cell is independent of ordependent on the development of other cells.

Regulative behaviour is commonly foundamong cells undergoing regional specifica-tion in the developing mammalian nervoussystem, which indicates that the mechanismof specification involves intercellular sig-nalling. In early embryogenesis, majorsources of such signals are well defined, andinclude the so-called Spemann organizer(Spemann, 1938). The signals that affect thedevelopmental pathway are termed induct-ive signals as the process involves transfer ofinformation from mesoderm to ectoderm,two of the three germ layers formed veryearly in development. Although inductivesignalling is almost certainly a widespreadmechanism in the later stages of corticalregionalization, its clearest roles are in theearly stages of forebrain development.

The very early regionalization of thedeveloping forebrain can be detected bymorphological criteria and by analysis ofthe discrete domains of expression of regu-latory genes. It is possible that the manygenes known to be involved give eachregion of the developing forebrain a unique

identity, probably through combinatorialactions. They may do this by controlling theexpression of numerous other genes requiredfor the characteristic morphological differen-tiation of that region. Amongst the moleculesknown to be involved are the diffusible pro-teins notch and delta, the wnt family of glyco-proteins and the hedgehog family of proteinsfirst identified in Drosophila.

Regional specificity of gene expression inthe telencephalon, from which the forebraindevelops, is likely to control regional dif-ferences in morphological characteristics,through actions on the cellular processes ofproliferation, migration and differentiation.How different regions come to express dif-ferent genes in the first place is a subject ofspeculation. One simple possibility is that a small number of genes distributed overthe neural plate and very early neural tube,the forerunners of the nervous system, gen-erate gradients of molecules.

Through transport and inter-cellularexchange, molecules at different levels ofconcentration would become localized indifferent cells. This can create domains ofgene expression with sharp boundaries.This type of process is known to generateregionalized domains of gene expression inthe early embryo of Drosophila. Althoughthere are homologues of these genes in themammalian forebrain, drawing close paral-lels between mammalian forebrain andDrosophila development may be dangerousgiven the differences between them at a cel-lular level. Nonetheless, the principle thatcontinuous molecular gradients may beread out to create domains of expression ofother genes distributed with discrete levelsis well established. There are various modelsfor how this can be done. These models areusually formulated according to the conceptof positional information and are constrainedby the regulatory phenomena often seen inembryogenesis.

Positional information Evidence from clas-sical embryological experiments on a massof cells after the removal of some cells or thetransposition of cells to a new positionresulted in the proposal that the fate of a cell


Morris-01.qxd 2/23/05 5:03 PM

is determined by its position within themorphogenetic field of cells. A particular setof cells that makes a single field can form itsown organ when transplanted to a foreignsite and cells within the field can regulate totake over the function of other cells that areremoved from it. How is each cell within thefield instructed or, as expressed by Wolpert(1969), how does the cell acquire its posi-tional information? As already discussed,one fundamental way in which informationis supplied in development is throughinducing signals supplied through extracel-lular means and various ways of assigningdifferences amongst cells by means of mor-phogens have been considered. In the sim-plest case of a one-dimensional field of cellsthat specifies the digits of the hand, forexample, a gradient of morphogen wouldenable different parts of the field to be dis-tinguished; specifying particular thresholdvalues of morphogen would determinewhich cells would develop into which digit.

Simple source/sink models Various dif-ferent ways of producing spatially varyingprofiles of a putative morphogen have beenconsidered. For a single dimension, mor-phogen flows from a single source to a singlesink to set up a graded variant of morphogendown the line of cells. Alternatively therecould be a single source and all cells acts assinks through leakage and other forms ofloss. These models have been found to beunsuitable. In particular, they do not adaptin the required fashion following perturb-ations such as the removal of a substantialnumber of cells.

Reaction–diffusion model Gierer andMeinhardt proposed a model of the reaction–diffusion type in which there are two mole-cules with different properties: an activator,which stimulates its own production, andan inhibitor, which diffuses at a faster rate(for a review, see Meinhardt, 1982). The acti-vator stimulates production of the inhibitorbut the inhibitor represses the production ofthe activator. A small local increase in theamount of activator will result in more acti-vator being produced, thus giving rise to alocal source of this molecule. The inhibitor

produced as a result will spread out morequickly than the activator and so a sink foractivator will be established nearby. In thisway, spatial patterns of activator andinhibitor become distributed across the arrayof cells. In these reaction–diffusion models,a crucial parameter is the size of the mor-phogenetic field over which the pattern isbeing formed compared with the diffusionlengths of the two molecules. If the field isvery much smaller than the diffusion lengths,periodically repeating patterns will be pro-duced; if the field is comparable in size tothese diffusion lengths, a single gradient ofmorphogen results. Imposing a weak gradi-ent of activator production to determinepolarity yields a single gradient. Diminutionof field size causes the full gradient to berestored, up to a limit. This is important asan explanation of the findings in develop-mental biology that in some animals struc-tures can regenerate from partial structures.

Reaction–diffusion mechanisms havebeen applied to the generation of many dif-ferent patterns, such as stripes, spots andother markings that appear on animal coats,and to other naturally occurring patterns,such as those on butterfly wings or those onsea-shells (Meinhardt, 1982; Murray, 1993).There is a close relationship between thesemechanisms, involving different types ofnon-linear interactions, and the self-organizing systems studied in physics.

Role of gradients The primary role to befulfilled by systems of gradients is to pro-vide a way for cells to be distinguished fromone another. The reaction–diffusion schemeat least provides a way of doing this whichis resistant (within limits) to changes inmorphogenetic field size. It is assumed thata separate mechanism translates an amountof morphogen into an instruction to build acellular structure. In some cases, patterns ofmorphogens are required to specify thecoordinate systems of developing organs. Itis natural, although not necessary, to assumethat the axes of the morphogens will matchthose of the required coordinate system. Forexample, a rectangular coordinate systemmight be provided by two morphogens,


Morris-01.qxd 2/23/05 5:03 PM

each identified with one axis. In cases wherethere is no such requirement, as long as cellscan be distinguished one from another, thepattern of morphogens can be arbitrary.

2.1.2 Pattern Formation in theSpecification of Neocortex

The neocortex, the uniquely mammalianstructure which has evolved rapidly andextensively in primates, is thought to be thesource of our highly developed cognitivefunctions. We can subdivide it into distinctareas, according to anatomical and functionalcriteria. There has been much discussion ofhow these distinct areas of neocortex develop

from the early cortical plate, with a rela-tively homogeneous appearance. It has beensuggested that neocortical organization isdetermined by:

● its afferents (inputs)● the significant amount of information

preprogrammed into the neocortex● interactions within the developing neo-

cortex, independently of its inputs.

In the adult mammal, the cerebral cortex hasbeen divided into areas according to theirhistological appearance. This is illustrated inFig. 1.1, which shows the cytoarchitectonicfields of the human brain as defined almost


11

1 256

78

9

10

17

17

18

18

19

19

20

20

36

35

28

2324

25

26

27

2930

31

33

34

4 3

37

181937

38

11

1 25

6

7

8

9

10

18

19

21

22

45 44

4241

43

40

39

43

38

47

FIGURE 1.1 Brodmann’s cytoarchitectonic fields of the human brain marked on the left cerebral hemisphere,seen from the lateral (top) and medial (bottom) aspects. (After Brodmann, 1909).

Morris-01.qxd 2/23/05 5:03 PM

a century ago by Brodmann (1909). Thesedistinct fields were defined according to therelative thickness of cell and fibre layers andBrodmann was able to delineate sharp bor-ders between neighbouring areas. This analy-sis of the human brain has been extended toother species, showing that their corticescan also be subdivided into cytoarchitec-tonic fields, with equivalent fields occupyingrelatively similar positions. Most impor-tantly, these anatomically defined regionshave different functional specializations.

The question of how much of the region-alization of the cortex is imposed on it by theorder of its afferents (i.e. inputs) and howmuch is specified before innervation hasbecome a major preoccupation. One extremeview is that the cortex is a naive sheet ofcells whose identities are determined by thenature, the order and the detailed connectiv-ity of the afferents that they receive. Anotherview is that the cells of the cortex do have atleast some regional identity before theybecome innervated.

There is now considerable electrophysio-logical evidence from the results of embry-onic and neonatal transplantations thatdifferences between distinct areas in theneocortex are induced by the afferent thala-mocortical axons that innervate the cortex.However, some area-specific differences aredetectable before any cortical innervationhas taken place. There is evidence for region-specific differences in the rates of prolifer-ation and expression of molecules in thecerebral cortex prior to innervation. In somecases these molecular differences are notaltered when expressing regions are trans-planted to non-expressing sites, suggestingthat region-specific differences may be deter-mined (i.e., irreversible) prior to innervation.The abolition of the activity in cortical affer-ents does not prevent the development in thecortex of characteristic region-specific distri-butions of molecules. In addition, there areseveral reports of cortical area-specific geneexpression that begins before or is independ-ent of afferent innervation.

In mouse, a specific gene is expressed inthe somatosensory cortex from before the

time of afferent innervation. The gene is stillexpressed if the developing somatosensorycortex is transplanted to an ectopic locationbut is not expressed by other regions ofdeveloping cortex even if they are trans-planted to the somatosensory cortex. Mostrecently, the arrangement of cortical areas intwo different mouse mutants that lack thal-amocortical connections have been found tobe abnormal. These results argue in favourof cortical cells having some positional iden-tity without innervation, although howmuch is far from clear.

Other experiments have addressed thisissue by studying the properties of differentcortical regions either in culture or aftertransplantation. Tissue culture experimentsto investigate the specificity of axons fromdifferent thalamic regions for different cor-tical areas showed that axons from thalamicexplants exhibit no preference for the area ofneocortex with which they were cultured.They grew equally well on their normal tar-get areas as on other non-target areas of theneocortex.

Other experiments have involved thetransplantation of regions of the developingcortex to abnormal sites. In transplantsbetween neocortical regions, the donor tissuewas found to develop attributes of the newhost region rather than retaining its normalattributes. Pieces of visual cortex graftedinto motor cortex developed persistent pro-jections to the spinal cord, as does normalmotor cortex but unlike normal visual cor-tex. When pieces of motor cortex weregrafted into the visual cortex, they devel-oped persistent projections to the superiorcolliculus, as does normal visual cortex, butunlike normal motor cortex.

Sur et al. (1988) carried out experimentson the regeneration of connections in ferretswhere target structures of sensory fibreswere removed, leading to the fibres beingdiverted to other cortical structures. Removalof lateral geniculate nucleus (a relay stationbetween retina and cortex) and visual cortexled to visual afferents innervating medialgeniculate nucleus (the destination of audi-tory fibres). The result was that cells of the


Morris-01.qxd 2/23/05 5:03 PM

auditory cortex became responsive to visualstimuli and acquired the functional charac-teristics of cells of the visual cortex.

All of these experiments indicate that dif-ferent regions of the embryonic and neonatalneocortex have a low level of commitmentto their specific regional fates. Although it iswidely accepted that the fates of embryoniccortical regions are not determined by birth(i.e. they are not irreversible), the degree towhich they are specified remains uncertain.The results of the experiments describedabove suggest that it is easy to deflect devel-oping neocortical regions from their normaldevelopmental pathways. However, recenttransplant experiments similar to those out-lined above have come up with oppositeresults with no clear explanation for the dif-ference. Furthermore, it may be harder toalter the fates of embryonic tissue trans-planted between the neocortex and othercortical areas, such as the limbic cortex.

2.1.3 Self-organization in Forebrain andNeocortical Development

To summarize section 2.1.1 and 2.1.2, thereis much evidence at the genetic, molecularand neural levels for the roles of self-organizing influences in the development ofregional specificity in these systems. Weknow most about the early stages of develop-ment of the forebrain, where inductive effectsare important. In the development of corticalregionalization, the influences of specificationprior to innervation and cortical afferentscontribute. Several mathematical and com-puter models have been developed but atpresent they lack specific application.

2.1.4 Pattern Formation in thePositioning of Cells

For nerve cells to function correctly, they have to be placed in the correct posi-tion. To investigate how this is done, weneed to look at systems where it is clearwhat it means to place cells correctly. This ismost easily accomplished in parts of thenervous system where there is a high degreeof order.

Nerve cells within invertebrates are wellordered, while the vertebrate nervous sys-tem is less highly ordered. The degree oforder varies greatly between different partsof the nervous system. In mammals, hippo-campus, olfactory cortex, cerebellar cortexand retina are examples of relatively orderedbrain structures.

In the cerebellar cortex, the Purkinje cells(the main output cells) form, with other celltypes, a regular three-dimensional lattice.These cells define regular, typically hex-agonal, neighbourhoods with intercellularspacings that grow steadily in size duringthe first few postnatal weeks.

In the retina, neurons form regulararrays, called mosaics. These exist in manyspecies, being more regular in invertebratesthan vertebrates. Several cell types are dis-tributed regularly in the two-dimensionalplane of the retina, giving a constant cell-spacing between adjacent cells. In insects,receptor cells are highly ordered, formingprecise hexagonal patterns.

It is important for cells to be arrangedregularly across the retina. The vertebrateretina is organized in such a way that manylocal circuits can analyse each part of theimage. This allows it to handle the vast infor-mation flux of a complex, ever-changingvisual scene using only slow, noisy neurons.The circuits work in parallel to assess differ-ent static or dynamic aspects of colour,brightness and contrast. As it is the regularrepetition of these circuits across the retinathat gives rise to mosaics, each mosaic canbe assumed to embody a unique function insensory processing.

There are different types of mosaics, suchas those involving cholinergic cells (amacrinecells with acetylcholine as transmitter), hori-zontal cells, different classes of ganglion cellsand the different types of photoreceptor cells.The mosaics appear early on in development.They form while the cell membership is stillforming, by the processes of neurogenesis,cell death and cell migration. For example, indeveloping cholinergic mosaics, as new cellsenter the array, neurons move sideways topreserve a constant inter-cell spacing.


Morris-01.qxd 2/23/05 5:03 PM

It is thought that the rudiments of thepattern are determined at a very coarse scaleby molecular markers, derived from genessuch as the pax family of homeotic genes, in the embryonic neuroepithelium, whichforms the eye. The ordered spacing can besimulated by a very simple local exclusionrule, applied to cells of the same type, whichspecifies the minimum distance betweencells.

Recent research suggests how this rulecan be implemented at the molecular level.A computer simulation study has shownthat regular, advancing arrays such as conemosaics can emerge from simple cellular-automaton rules that are applied to initiallyrandom arrangements, but also that manydifferent sets of these rules converge on thesame simple patterns.

Although information must be suppliedto position cells in the correct general region,this solution has the advantage that there isno need for a means of pre-specifying thepositioning of nerve cells precisely. There areother advantages. Since interactions are shortrange, neither local errors nor the introduc-tion of new cells at the periphery of thearray disturb the pattern. If the local interac-tions are restricted to cells of the same type,introduction of an array of cells of a differ-ent type will not perturb the arrangementsin the preexisting arrays. Finally, the abilityof cells to recognize others within a limitedrange could lay the basis for the formationof the topographically ordered maps of con-nections that exist in the retina and whichsubserve visual processing (see section 2.3).

2.2 Making the Correct Numbers ofCells: Cell Death

Of all the mechanisms involved in theformation and maintenance of the nervoussystem, cell death is the best understood,especially at the level of how genetic instruc-tions can bring about cellular self-destruction.It has long been realized that cell death canbe a physiological as well as a pathologicalprocess, i.e., that many cells die even duringnormal brain development.

Cell death during animal developmentwas first observed by Vogt in 1842, inamphibia. The basic findings are:

● there is substantial motor neuron deathin normal development

● removal of a developing limb bud fromchick embryos causes increased death ofthe motor neurons

● some of the motor neurons that wouldhave died during normal developmentcan be rescued by grafting in an extralimb bud.

Researchers have reported similar findingsin Xenopus. Subsequent studies have shownthat cell death is a normal occurrenceamongst many neuronal populations in thedeveloping vertebrate nervous system, tak-ing place when axons begin to reach andactivate their targets. The number of neuronsin a given population first rises then declinestowards the constant adult number. The pro-portion of cells that dies varies among differ-ent neuronal populations, ranging from theremoval of only a small number of the neu-rons in some regions to more than half thepopulation in others. In the retina, resultsfrom a range of mammalian species indicatethat 50–90% of the retinal ganglion cell pop-ulation will die during development.

That so much cell death occurs duringnormal development is counterintuitive.Thinking anthropomorphically, it appearswasteful. It is not clear why evolution shouldhave selected such a developmental process.

Blocking normal neuronal death – bymaking transgenic mice with mutations ofgenes that regulate cell death, therebyincreasing the numbers of neurons – doesnot affect lifespan. None the less, cell deathis a significant developmental process thatdemands an explanation both in terms of itsrole in development and the molecularmechanisms that control it.

Researchers have advanced variousexplanations for nerve cell death, amongstthem being:

● failure of neurons to find their target● failure to make the correct connections


Morris-01.qxd 2/23/05 5:03 PM

● the elimination of entire structures thatmay act as transient scaffolds; examplesof this are the subplate, a layer of cellsthat is critical for the development ofcerebral cortex, and the Rohan–Beardcells in amphibian embryos, which aretemporary sensory neurons

● removal of transient branches of the treeof lineage; this seems to be the case ininvertebrates – in the nematode C. elegans,around 20% of the 300 nerve cells gener-ated by cell division are preprogrammedto die

● lack of adequate innervation, as is thecase in insect optic lobe.

All these explanations seem to apply in spe-cial cases.

A commonly held view is that in manycases this substantial amount of nerve-celldeath results from the action of a mech-anism that matches the number of presynap-tic cells to the number of postsynaptic cells.One hypothesis for this is that neurons com-pete for a supply of one or more so-called‘neurotrophic’ factors that are known to beproduced by their target cells. According tothis neurotrophic hypothesis, insufficientneurotrophic factor is produced to supportthe excessive numbers of neurons generatedand those that are unsuccessful in the com-petition die.

Additional support for the idea of match-ing presynaptic and postsynaptic cell num-bers has come from experiments in whichchick lumbosacral cords were transplantedinto quails and vice versa before the limbsbecame innervated. Chicks are larger thanquails and have bigger muscles with moremuscle fibres. More quail motor neuronssurvived in the chick than in the nativequail, and fewer chick motor neurons sur-vived in the quail than in their own environ-ment. There was a correlation between thenumber of motor neurons surviving and the number of muscle fibres available forinnervation.

The nervous system contains controlmechanisms by which the numbers of presy-naptic and postsynaptic cells are matched

apparently automatically. This involves theapparently destructive side-effect of celldeath. This self-organizing mechanism hasgreat benefit, making it unnecessary to haveultra-precise controls over the generation ofneuron numbers. Whilst qualitatively theeffect is well established, the underlyingmechanisms and their quantitative implica-tions represent exciting challenges for thefuture.

2.3 Development of Connections

Once nerve cell axons have found theircorrect target within the nervous system,neuron numbers have been adjusted to theiradult levels and neurons have been correctlypositioned, the appropriate connections haveto be made. It is generally thought that thishappens in two stages. Initially, the axonalterminals are distributed across the targetrelatively diffusely. Subsequently, there is arearrangement or refinement of connections.This picture has emerged from many differ-ent animal preparations, principally fromexperiments on the innervation of skeletalmuscles and autonomic ganglia of neonatalrats, and on the visual pathways of Xenopus,kittens, ferrets and infant monkeys.

There are good reasons why some sort ofrefinement of connections is essential. First,far too many neurons exist for the position-ing of each to be controlled by a geneticallydetermined programme. Secondly, it is diffi-cult to see how such a programme candetermine the fine details of the connectionsbetween independently specified sets ofneurons. Finally, as a consequence of thecontinual growth of some animals there hasto be a continual remapping of connectionsduring development to accommodate thegeneration of new cells and consequentlynew connections.

The two stages are thought to involvemechanisms that guide axons to their initial,approximate destination to generate an ini-tial pattern of connections, followed by astage during which connections are remod-elled to form the adult configuration, whichinvolves the loss of existing connections and


Morris-01.qxd 2/23/05 5:03 PM

the generation of new ones. Many peoplethink of the first stage as being programmedgenetically and the second one driven byneural activity so that the refinement of con-nections is sculpted to fit the uses to whichthe neural system has to be put. The precisedivision between these mechanisms is notclear, particularly in that it is not clear howmuch specificity of connection is impartedduring each stage.

I will describe the role of self-organizationat two levels. I will first discuss the networklevel, as exemplified by the development ofordered maps of connections between thevertebrate retina and optic tectum in lowervertebrates (equivalent to the superior col-liculus in mammals). I will then consider thesingle-cell level, as exemplified by the elim-ination of connections from the developingneuromuscular junction.

2.3.1 Map FormationA striking feature of many of the connec-

tion patterns between collections of nervecells is that they are highly ordered. Evidencefor this comes mainly from two types ofexperiment. First, in electrophysiologicalexperiments, stimulation of a small region inone structure, such as a sensory surface or anucleus, leads to activation of cells in a smallregion of its target. As the stimulus is movedsystematically across the structure, the regionthat responds shifts in a corresponding fash-ion. The region in stimulus space that pro-duces a response at a particular targetposition is called the ‘receptive field’.

Secondly, the mapping between twopoints in different structures can often beestablished in anatomical experiments usingaxonal tracers (molecules that can be injectedat discrete points to label axons running toand from those points). Tracers placed atone point in one structure typically label asmall, circumscribed area in the target, thespatial layout of points of administration (indifferent animals) being reflected in the lay-out of points to which the tracers go in thetarget. Such ordered anatomical layouts ofconnections provide the substrates for the

ordering observed in electrophysiologicalexperiments.

Many neural maps are effectively projec-tions of one two-dimensional surface ontoanother. For example, axons from eachsmall cluster of ganglion cells in the mam-malian retina project, via the lateral genicu-late nucleus (LGN) of the thalamus, onto asmall area of visual cortex, with the resultthat a map of the retina is spread over thesurface of its target structure (Fig. 1.2a). Inamphibia and fish the retina projects directlyto the optic tectum where, once again, anorderly map of the retina is found. Auditorycortex contains an example of a one-dimensional map of frequency (Fig. 1.3a).

Another striking example is the existenceof precise maps of connections in somatosen-sory cortex. Rodents make much moreextensive use of tactile information than ofvisual information. Correspondingly, theirsomatosensory cortex is relatively largewhereas their visual cortex is relativelysmall and simple. A large area of primarysomatosensory cortex is occupied by anordered representation of the facial whiskerpad (Fig. 1.2b).

Anatomical investigations have revealeda set of barrel-like structures, with a one-to-one relationship between the arrangementof whiskers and the arrangement of barrels:where the muzzle contains an extra whisker,there is an extra barrel in the topograph-ically equivalent place and vice versa.Neurophysiological recordings have estab-lished that activation of an individualwhisker excites the cells in the correspond-ing barrel. There is also a topographical rep-resentation of the whiskers in each of thetwo nuclei which form the relay stationslinking the sensors with the somatosensorycortex. This ordered map is not preservedthroughout the length of the pathway fromsensorium to cortex but rather is recreatedat each individual relay station.

There is much evidence for plasticity inthe whisker-to-barrel pathway in rodents.An intact sensory periphery is required dur-ing a certain critical period of developmentfor the normal map to develop. When rows


Morris-01.qxd 2/23/05 5:03 PM

of follicles are injured at birth, before barrelsform, the corresponding row of barrels isabsent in the adult, and is replaced by asmall barrel-less territory. Barrels develop inthe first postnatal week and their morph-ology can be manipulated by the selectivelesioning of the whisker follicles. The earlierthe follicles are removed, the more extensivethe resulting morphological aberration.

Generally, where axonal projections are through intermediate structures, theintermediate maps are themselves well-ordered. Ordered projections are the rule,although there are a few apparently dis-ordered projections such as that of visual

space onto the pyramidal cells of mam-malian hippocampus, which in rats respondto specific locations in the animal’s environ-ment, and in the direct projection betweencortex and striatum.

The mapping of the elements in onestructure onto another is studied at thecoarse-grained level, as in the case of themapping of thalamic nuclei onto the cortex;or at a fine-grained level, as in the case of themapping of cells within a particular thala-mic nucleus onto a particular corticalregion. Such maps are readily understoodon the basis of zone-to-zone or point-to-point connections between the structures,


(a) (b)

1 cm

FIGURE 1.2 (a) Showing the ordered projection of the retina onto the visual cortex of monkey. The animal wasexposed to the image (upper) and activity in the visual cortex was revealed by processing for deoxyglucose uptake(lower). Note the ordered projection and non-linearities. (Reproduced from Tootell et al., 1982.{?3 permission?} (b) The pattern of whiskers on the adult rodent snout (lower) and the pattern of barrels in somatosensory cortex (upper,schematized in inset) to which the whiskers project in a one-to-one fashion. The pattern of whiskers is almost invariantfrom one animal to another. (Reproduced from Woolsey and van der Loos, 1970; Copyright acknowledged)

Morris-01.qxd 2/23/05 5:03 PM

mediated either by bundles of axons (in thecase of coarse-grained mapping) or by indi-vidual axons (in the case of fine-grainedmapping).

In a complex structure such as the cere-bral cortex, more complex properties of the sensory environment than position inthe field are detected by specific nerve cells.The properties of the stimulus that producesmaximal excitation varies over the cortex,defining ‘feature maps’ (see section 3.1).

2.3.2 Topographic Map Formation in theVisual System

Most work on topographic maps has beencarried out on the vertebrate visual system.Here I describe the direct projection of retinaonto optic tectum (the analogue of the super-ior colliculus in mammals) in amphibia andfish. The first crude maps were constructedfrom the results of axon degeneration stud-ies but the first maps with any precisionwere made by extracellular recording from


Cat Bat

2 mm

Dorsal

Anterior

(a)

10 mm

Right tectum

Dorsal

1

1 1

12

2 2

3

3 3

34

4

4

49

9

87

7

7

76

6

6

65

5

8

10

10

11

11

12

12

16

16

16

1617

17

17

1718

18

18

18

19

19

19

19

20

20

2122

23

20

20

15

15

15

14

14

14

13

13

15

13

13

21 22 23

VentralRight visual field

VentralLeft visual field

DorsalP

oste

rior

Pos

terio

r

Ant

erio

r

Ros

tral

Cau

dal

(b) Left tectum

32 168

42

61

100

949080

6262

6350

40 30 24

60

58

5450

10.5

0.250.13

A I

A IA I

TI

A IIEP

A I

12 1011 9 85

2

9

8

11

12

10

514

63

FIGURE 1.3 (a) The one-dimensional map of frequency in auditory cortex. (AI and All, primary andsecondary auditory cortex; EP, posterior ectosylvian gyrus; I, insula; T, temporal field; numbers 0.13–100 kHz.)There is a regular tonotopic representation in cat but a distortion of this regularity in bat by a large representationof 61–62 kHz, the frequency of its echolocating signal (based on Sugar, 1978 {?4} and Shepherd, 1994). (b) Theordered projection from retina to contralateral tectum in adult Xenopus loevis. The numbers indicate where in thevisual field a small point stimulus evoked maximal response at the correspondingly numbered tectal position.(Reproduced from Jacobson, 1967 {?5 permission})

Morris-01.qxd 2/23/05 5:03 PM

the optic tecta of goldfish, and of the frog,Rana, and Xenopus (Fig. 1.3b). The connec-tions are not precise at the cell-to-cell level(as in invertebrates) but in the retinotectalmap in Xenopus, for example, at least 50recording positions can be distinguished, allarranged in topographic order. The otherimportant attribute of such maps is that theyalways have a specific orientation. Allretinotectal maps are arranged so that tem-poral retina projects to rostral tectum anddorsal retina to medial tectum (Fig. 1.3b).

The problem of understanding howmaps are formed was originally formulatedin terms of the establishment of a functionalrelationship between the cells in the two sets of cells. With the advent of powerful methods for tracing patterns of connections,this problem has become that of specifyingthe connections themselves.

One powerful set of results, generatedfrom electrophysiological recordings car-ried out in the 1970s, involves experimentson the regeneration of connections in theretinotectal system of adult goldfish. Theparadigm experiment involves removingpart of the retina, allowing connections toregenerate and then making a map of theremaining part of the retina onto the tectum.After several months, retinal fibres hadexpanded to innervate the entire tectum.Complementary experiments revealed thatafter removal of half the tectum from gold-fish, the projection from the entire retinaeventually becomes compressed, in order,on to the surviving half-tectum. The basicresult in these so-called ‘mismatch’ experi-ments is that the retina and the tectummatch together as a system, independentlyof the actual sizes of both structures.

Although these results were found in theregeneration of connections, exactly thesame type of phenomenon occurs duringdevelopment. In Xenopus laevis, the retinaand tectum grow in different ways. Newretinal cells are added on in rings to the out-side of the retina whereas tectal growth pre-dominantly involves addition of cells to theback. None the less, there is an ordered pro-jection of retina onto tectum from a very

early stage. This implies a gradual shiftingof connections. For example, throughoutdevelopment, central retina always projectsto central tectum, the position of whichmoves progressively backwards as moretectal cells are added. This inference wasfirst made from extracellular recordings andthen confirmed by electron microscopystudies which demonstrated the degener-ation of synapses as axons shifted their pos-itions during development. Later work onfrog tadpoles that combined electrophysio-logical and electron microscopy confirmedthis by showing that retinal ganglion cellaxons move across the tectum during devel-opment, continually changing their tectalpartners as they do so.

These results demonstrate that connec-tions cannot be made by means of a simpleset of instructions specifying which cell con-nects to which other cell; more likely, thetwo populations of cells self-organize theirconnections so as to ensure the correct over-all pattern. Several different hypotheses forthe formation of nerve connections in thisparadigm system have been made. The twomain contenders are:

1. As first proposed by the Nobel laureateRoger Sperry (1963) in his doctrine ofchemospecificity, the two populations ofnerve cells, one in the retina and one in thetectum, are labelled separately by sets ofmolecular markers. By some means, thecorrespondence between the two sets ofmarkers is communicated to the partici-pating cells. Each retinal axon then usesthis information to find its correct tectalpartner. In addition, there is some meansof regulating the constitution of themolecular labels, when required, toaccount for the lability of connections dur-ing development and regeneration. Thisproposal has received renewed interestrecently due to the discovery of a type ofreceptor located in the retina, the Ephreceptor, and the associated neurotrophinslocated in the tectum, the ephrins, whichbind to these receptors. The Ephs and theephrins could be the markers that label the


Morris-01.qxd 2/23/05 5:03 PM

two sets of cells. The origin of the mark-ers themselves has not yet been linked tothe generation of regional specificity dis-cussed in section 2.1.

2. Initially, a roughly ordered map of connec-tions is made with subsequent refinementof the map through electrical activity.

At present, the status of both contendingproposals is unclear. They lack experimentalverification at the mechanistic level. For (1),it could be that the tectum acquires its mark-ers from the retina (Willshaw and von derMalsburg, 1979), thereby ensuring that thetwo sets of markers are properly coord-inated. For (2), a Hebbian type synapticmodification mechanism1 might operate. Byreinforcing the contacts made by neighbour-ing retinal cells to neighbouring tectal cellsat the expense of the connections made bynon-neighbouring cells (Willshaw and vonder Malsburg, 1976), each pair of neighbour-ing presynaptic cells comes to connect topostsynaptic cells that also are neighbours,resulting in a topographically ordered map.

2.3.3 The Elimination of Superinnervationfrom Developing Muscle

The second part of the two-stage processthought to underlie the development of nerveconnections is illustrated by cases where thedevelopment of the connections on individ-ual targets has been monitored, as in manyvertebrate skeletal muscles. In the adult eachmuscle fibre is innervated at its endplate bya single motor neuron. This pattern of inner-vation arises from an initial state in whichthere is innervation from a number of differ-ent axons at a single endplate.

During early development, contacts arewithdrawn until the adult configuration isreached (Fig. 1.4). The same pattern of eventstakes place in the adult after transection ofthe motor nerve. In the initial stages of rein-nervation, muscle fibres are superinnervatedand this pattern is transformed into one of

single innervation after a few weeks. Musclesvary in size. The soleus muscle is one of thelarger muscles in rat with around 3500 fibresinnervated by some 25 motor neurons, sothat in the adult each motor neuron innerv-ates on average 140 muscle fibres; the ratlumbrical muscle has about 100 musclefibres and 10 motor neurons.

This developmental loss of synaptic con-tacts has been observed at both central andperipheral sites, in systems as diverse as theneuromuscular junction of invertebratesand the cerebral cortex of primates. The pre-cise time course over which synapse elimin-ation occurs and the proportion of afferentslost varies greatly between areas of the nerv-ous system, even within a single species. Inneonatal rat skeletal muscles, there are onaverage four to five contacts per fibre ini-tially. This reduces to exactly one per musclefibre over the next two weeks.


Muscle

Muscle

Motor nerve axons

FIGURE 1.4 Schematic of the states of innervationof mammalian skeletal muscle: (top) the initial patternof superinnervation; (bottom) the adult state whereeach muscle fibre has contact from a single axon.(After Rasmussen and Willshaw, 1993)

1Referring to the hypothesis due to Hebb (1949) that synapses are strengthened by conjoint activity inthe presynaptic and the postsynaptic cells.

Morris-01.qxd 2/23/05 5:03 PM

In rat cerebellum, the elimination ofclimbing fibre synapses onto Purkinje cellsoccurs during the second postnatal week,about the same time course as at the neuro-muscular junction. In contrast, the elimin-ation of preganglionic synapses onto neuronsof the rat submandibular ganglion occursover at least five postnatal weeks, far longerthan is required for elimination at the neuro-muscular junction.

The number of cochlear nerve synapseson neurons of the chick cochlear nucleusdeclines rapidly, from about four to twoafferents and reaches a mature state evenbefore hatching. Therefore, synapse elimin-ation appears to be a widespread phenom-enon, although there are no general rulesabout the percentage of afferents that is lostor the duration of time required.

There is a long and established history ofthe role of neural activity in the developmentand regeneration of nerve connections in the neuromuscular system. Tenotomy (cut-ting the tendon) delays the withdrawal ofsuperinnervation by a moderate amount.Muscle paralysis results in increased, long-lasting levels of polyneuronal innervation, as does application of tetrodotoxin (TTX), aneural activity blocker, to the motor nerve.Chronic muscle stimulation accelerates theelimination of synapses during development.

Researchers can compare the effects ofactivity with inactivity by applying nerveblocking agents during the reinnervation ofneuromuscular connections to a rat musclewith two separate branches of the motornerve that can be manipulated independ-ently. After cutting or crushing both nervebranches, motor axons regenerate, theysuperinnervate the muscle fibres and thengradually the pattern of single innervationis reestablished. By blocking one of thenerves with TTX during reinnervation, acompetitive advantage can be given to theactive synapses as against the inactivesynapses (i.e., those made by the axons fromthe nerve blocked by TTX). It turns out thatactive synapses have an advantage overinactive synapses. For example, the abilityof a regenerating nerve to regain its territory

is enhanced if the other nerve is blocked andis diminished if its own nerve is blocked.

The final pattern of one contact per muscle fibre is both clear and unequivocaland several possible causes of the elimina-tion process have been considered. Here isan assessment of them.

● It is unlikely that withdrawal of connec-tions is random as this would leavemany fibres uninnervated, contrary toobservation.

● Nerve-cell death cannot provide theappropriate reduction in contacts asthere is no cell death during this stage ofdevelopment.

● Some terminals might withdraw if theywere misdirected to the wrong region orthe wrong fibre type. This is also unlikelyas the muscles are almost homogeneousin fibre type and the somatotopic order-ing of motor neurons across the muscle isvery low.

● Another possibility is that synapses arepreprogrammed to die. This is alsounlikely as when a proportion of the motorneurons are removed before connectionshave been established, the surviving motorneurons make more contacts than normal.

This last point provides strong evidence thatthe identity of the surviving contactsdepends on which other contacts are pre-sent; i.e., there is competition amongstsynaptic contacts for survival.

There are various formal models for thiscompetitive process. According to some mod-els, there is competition for a fixed amountof synaptic strength possessed by the motoraxons and shared amongst its terminals; inother models, the making of synapses isthought of in terms of the binding of neu-rotrophins onto the receptors on differentaxons, with competition amongst the recep-tors for the neurotrophins. How the variouseffects of activity can be interpreted is notyet clear. It is interesting that the mathemat-ics underlying these models bears a strongfamily resemblance to the mathematicsunderlying many self-organizing phenomenastudied in physics and the competitive


Morris-01.qxd 2/23/05 5:03 PM

interactions of the predator-prey type studiedin population biology and other disciplines.

In summary, the production of the preciseconnection pattern of one contact per musclefibre seems to be a task that is not fulfilledby the genome but instead is the responsi-bility of a competitive, ‘self-organizing’,mechanism involving interactions at thesynaptic level.

3 THE ROLE OF SELF-ORGANIZATION INEXPERIENTIAL CHANGE

I now examine situations where the puta-tive self-organizing mechanisms are morestrongly influenced by the signals impin-ging on the system from outside.

3.1 Feature Maps

In mammalian cerebral cortex, conver-gence of inputs onto neocortex causes cor-tical cells to respond to complex propertiesof the input. For example, in certain areas ofthe visual cortex, there are cells that are sen-sitive to the orientation of a stimulus or itsdirection of movement as well as its positionin the visual field. Such attributes of theexternal environment can be detected bymeans of the neural circuitry and the con-nectivity of the central nervous system. Theproperties of the stimulus that producesmaximal excitation in each small areachanges over the cortical surface, defining‘feature maps’.

3.1.1 Ocular Dominance, Orientation and Direction Selectivity

Visual cortical cells receive innervationfrom both eyes, but with differences in thestrength of the innervation from each eyethat vary systematically across the surface ofthe cortex (Fig. 1.5). The Nobel laureatesHubel and Wiesel (Hubel et al., 1977) dis-covered that cells in monkey binocular visualcortex vary in their responsiveness to the twoeyes. Similar ocularity preferences extend

down to layer IV of neocortex – one of the sixlayers in neocortex defined on anatomicalcriteria – where axons from the lateral genic-ulate nucleus terminate, and thus the conceptof ‘ocular dominance columns’ arose.

These systematic variations in oculardominance are superimposed on the basicretinotopic map (Hubel et al., 1977). Subse-quently, existence of such columns was con-firmed anatomically; the map of ocularityspecificity across the entire surface of binoc-ular cortex resembles a pattern of zebrastripes.

In cat and monkey, segregation begins ator around birth and is complete about sixweeks later. Ocular dominance columns seemto be the result of a competitive processbetween the axons from the two eyes. Theyresult from an initially overlapping distribu-tion of innervation originating from the left


2°

FIGURE 1.5 Computer reconstruction of thepattern of ocular dominance columns in layer IVc ofarea 17 of a macaque monkey, produced by reducedsilver staining, translated into the visual field.(Reproduced from Hubel and Wiesel, {?6} 1977)

Morris-01.qxd 2/23/05 5:03 PM

and right eyes. In another preparation, asimilar pattern of stripes can be producedthrough competition within the axons froma single eye. In Xenoptis, so-called ‘com-pound eyes’, constructed from two match-ing half eye rudiments, develop a singleoptic nerve which produces a striped projec-tion on the optic tectum.

Cells in certain visual areas of mam-malian neocortex are orientation selective;i.e., each cell is responsive to a small bar oflight when presented in a particular orienta-tion at a particular position in the visualfield (Fig. 1.6). The existence of orientationmaps was established by extracellularrecording and, more recently, the method ofoptical recording has been used to producedetailed orientation maps over the entiresurface of the visual cortex. The maps pro-duced are complex and have a number offeatures, such as periodically repeating pat-terns, and more complicated features, suchas saddle points and singularities (points onthe cortex around which orientation domainsare clustered in a pinwheel fashion). Thistype of data has provided an irresistiblechallenge to modellers.

The magnitude of the response fromsome cells elicited by a moving bar stimulusin a particular orientation, may depend onthe direction of movement (at right-angles

to the orientation of the bar). This forms adirectionally selective map.

3.1.2 Relations Between the DifferentFeature Maps

The different types of map are interrelated.The ocularity map effectively interrupts theretinotopic map; i.e., if all the pieces of cortexinnervated by one of the eyes were removedand the remaining pieces, innervated by theother eye, were pushed together, then a com-pletely ordered retinotopic map would result.In cat visual cortex, pinwheel centres in orien-tation maps are mainly located in the middleof ocular dominance columns. In addition,according to recent optical recording experi-ments in cat, the orientation domains tend tointersect at right angles the borders of oculardominance stripes. In the classic model ofHubel et al. (1977), developed for the cat, iso-orientation columns run in straight lines atright-angles to the inter digitating oculardominance columns.

3.1.3 The Effect of Neural ActivityIt is well established that lack of normal

visual experience early in postnatal life pre-vents the normal development of the visualcortex. This suggests that some informationrequired to generate the normal visual system derives from interactions of thedeveloping system with the external environ-ment. Clearly, many aspects of corticaldevelopment occur prenatally and areimmune to the effects of postnatal func-tional deprivation.

Studies in both cats and primates haveshown that before the neonate has receivedany visual experience, geniculocortical fibresfind their main target. In layer IV, they con-verge to generate immature orientationselective cells clustered into rudimentaryorientation columns, form a retinotopic mapand, at least in the primate, begin to segre-gate to form ocular dominance columns.

Visual deprivation early in life, duringthe so-called critical period, does not abolishthese features of early organization. The

3 THE ROLE OF SELF-ORGANIZATION IN EXPERIENTIAL CHANGE 23

FIGURE 1.6 Orientation preference map for area17 of the cat visual cortex, constructed by opticalrecording. The orientation at each point is indicatedon a grey-scale. The key below shows howorientations are assigned on the scale. The length ofthe bars corresponds to 0.5 mm. This type of picture isbest seen in colour (e.g., see Hubener et al., 1997{?7})

Morris-01.qxd 2/23/05 5:03 PM

continued refinement of cortical connec-tions is strongly influenced by patternedneural activity. Visual deprivation can havea devastating effect on the development ofthe detailed circuitry required for the nor-mal functional properties of visual corticalneurons. For example, in areas 17 and 18 ofthe cat – these are the primary (18) and sec-ondary (17) areas of cat visual cortex – thenormal appearance of a large proportion oforientation selective cells is prevented bydark-rearing or by binocular eyelid suture.

A variety of results shows how experi-mental interference can affect the develop-ment of ocular dominance columns. Most ofthese experiments involve the manipulationof activity levels by: monocular deprivation;rearing animals in the dark; distorting theretinal input by artificially inducing strabis-mus, and removal of spontaneous retinalactivity by administering the neural activityblocker TTX. The results of most of theseexperiments indicate the important role ofneural activity in the formation of oculardominance columns. Thus, deprivation pre-vents the emergence of the full richness offunctional architecture and receptive fieldproperties of the normal adult visual cortex.

The most striking effects of deprivationoccur when vision through only one eye isimpaired during the critical period. Thisresults in expansion of the cortical territoryof the projection serving the normal eyerelative to that of the projection serving thedeprived eye. Neuronal activity plays a crucial role in this organization, and itappears that monocular deprivation placesthe geniculocortical afferents from thedeprived eye at a competitive disadvantage.The effects of monocular deprivation can bereversed by opening the deprived eye andclosing the other before the end of the crit-ical period (reverse suture). These changesinvolve the sprouting and/or trimming ofgeniculocortical arbors. It appears that, asthe developing geniculocortical fibres elab-orate on their initial framework of immatureinputs, it is especially important that theprojections serving one eye should be asactive as those serving the other eye. A

balance of activity is required to ensure thatthe growth of terminals from each eye isrestricted within its own cortical territory.

3.1.4 Self-organization and the Formation of Feature Maps

Self-organization plays a role, in combin-ation with external signals, in determiningthe response properties of the individualcells in feature maps as well as the pattern ofresponse properties distributed over themap itself. In the development of ocularitymaps, determination of the ocular prefer-ence of an individual cell is influenced heav-ily by the nature of the afferent activity,whereas the pattern of ocular dominance isthe result of an interaction between activityand mechanisms of self-organization.

In the development of orientation maps,self-organization may be involved in speci-fying both single cell properties and theoverall pattern. Von der Malsburg (1973) pub-lished the first paper demonstrating that thepattern of orientation specificity in visual cor-tex could be developed in a self-organizingmodel under the instruction of simulatedorientated bar stimuli. Since then, researchhas established that patterned stimuli arenot needed to develop the individual cell’sresponse properties. Radially symmetricpatterns of activity drive the production oforientated receptive fields by a process ofsymmetry breaking. There has been muchrecent theoretical work developing modelsof all types of feature map and the relationbetween them. In these models, changes inthe external conditions (simulating, forexample visual deprivation) lead to themodel successfully self-organizing to adaptto the new conditions.

3.2 Self-organization and theAcquisition of Cognitive Function

All the examples of self-organization inthe nervous system we have discussed sofar concerned the development of the nerv-ous system in cases where we can relate theresults of the self-organization to structural


Morris-01.qxd 2/23/05 5:03 PM

and functional changes at the cellular and subcellular level. I now discuss self-organization and the acquisition of cogni-tive function.

Perhaps because of the lack of a strongstructural basis, less can be said about theprecise form of the self-organizing mech-anisms than in the examples of neural self-organization discussed already. It will beseen that many of the themes already dis-cussed are echoed, but with different vocabu-lary to place an emphasis on the cognitiveand psychological levels.

3.2.1 Cognitive Self-organizationThe term self-organization has been used

to understand how new cognitive behav-iours are acquired. As noted earlier, thesebehaviours could be derived from built-inknowledge (in cognitive science terms,nativism, attributed in this context to Fodor)or through learning (attributed in this con-text to Piaget). However, it has been pointedout that through self-organization, ordercan emerge from interactions within the sys-tem rather than by explicit instruction.

According to Karmiloff-Smith (1992),children’s brains are genetically prewired tocontain modules (or domains) of know-ledge which grow and interact during development. She proposes that in eachdomain, the child’s development is subjectto constraints that initially shape the waythat information in different domains isprocessed and represented. The child’s ini-tial knowledge in each domain is then pro-gressively redescribed according to existingknowledge and the child’s experiences. Thelevel of redescription in each domain thusdepends on a complex interaction betweenthe current state of knowledge in thedomain and the child’s experience. Alongwith mastering new behaviours, the childalso learns to introspect about what he/shehas done and ultimately constructs his/herown theories about how the world works.

While accounting for a large body of know-ledge, these theories of re-representation are difficult to interpret insofar that no

details are given about the mechanistic basisof re-representation. An attempt to make abridge between the cognitive and neurallevels of brain function has been suppliedunder the name of neural constructivism.This has been developed from Piaget’s con-structivism, a term that reflects his view thatthere is an active interaction between thedeveloping system and the environment inwhich it develops. Neural constructivismemphasizes the dynamic interaction betweenthe developing system and the develop-mental ‘task’ to be accomplished and in this respect it can be regarded as a form ofself-organization.

Neural constructivism lies between theextreme versions of the theories of chemo-specificity (the entire blueprint of the ner-vous system is specified in the genome) and atabula rasa theory (nothing is prespecified).It has been contrasted with the doctrine ofselectionism which, according to someauthors, is the idea that neural developmentproceeds by initial overproduction of neuralstructure followed by the selective pruningaway of the inappropriate parts. We canview selectionism as an extreme form of thetwo-stage process thought to underlie thedevelopment of connectivity, mentioned inSection 2.3. According to selectionism, thefirst stage is involved in the formation ofconnections and the second stage in thebreaking of connections.

Neural constructivism is concerned withhow neural activity guides the developmentof connections, the patterns of activity beinggenerated from the external environmentrather than through, for example, spontan-eous activity. It is suggested that, as neocor-tex evolved in mammals, there was aprogression toward more flexible represen-tational structures, rather than there beingan increase in the number and complexity ofinnate, specialized circuits. Different typesof evidence are cited in favour of neural con-structivism, some of which were reviewedearlier in this chapter:

● Changes in synaptic number During devel-opment in primates the number of

3 THE ROLE OF SELF-ORGANIZATION IN EXPERIENTIAL CHANGE 25

Morris-01.qxd 2/23/05 5:03 PM

synapses rises and eventually falls butthere is no decrease until post-puberty.

● Axonal growth In the visual system dur-ing the period of activity-dependentdevelopment there is evidence for axonalgrowth, contrary to the doctrine of selectionism.

● Dendritic growth Dendrites grow at amuch slower rate than axons, over alonger time scale which stretches overthe period of time prescribed by selec-tionism. Dendritic morphology can bemoulded by the patterned neural activityincident upon it.

● The extent of cortical development Humancortical postnatal development is alsomore extensive and protracted than gen-erally supposed, suggesting that the neo-cortex has evolved so as to maximize thecapacity of environmental structure toshape its structure and function throughconstructive learning.

● The power of constructive neural networksFinally, arguments based on the computa-tional power of artificial neural net-works, used to simulate the phenomenaobserved, are used to support neural con-structivism, particularly the view that net-works that develop their structure whilstthey are being trained are more powerfulthan fixed architecture neural networks.

3.2.2 Neural Constructivism and NeuralNetworks

Neural networks are collections of highlyinterconnected simple computing units mod-elled loosely on the architecture of the brain.In such a system, a computing unit is a styl-ized representation of a nerve cell. The networks are required to learn specific input/output relationships (Hertz et al., 1991; Bishop,1995) by selective adjustment of connectionstrengths between neurons. Most applicationsof neutral networks are to problems in pat-tern recognition, classification and predic-tion. Their behaviour is usually investigatedthrough computer simulation.

Neural networks can be trained by twomain methods. In supervised learning, many

examples of the input/output pairings to belearnt are presented to the network, in theforms of patterns of activity over the comput-ing units, which can be likened to patterns ofneural activity. As a result, the strengths ofthe connections (weights) between individ-ual computing units changes. Ultimately, theweights in the network become set so that thenetwork, when tested, gives the requiredoutput for any input presented during learn-ing. Hopefully, it generalizes its behaviour togive the appropriate response to an inputthat it has not seen before.

In unsupervised learning, there is noteaching signal, in the form of the requiredoutput being supplied for each input.Presentation of every input generates an out-put and the network modifies its weightstrengths ‘by itself’, with the result that in theinitial stages the output generated for eachinput may change. After many presentations,a stable output comes to be associated witheach input.

Neural networks have been said to be self-organizing in that, in both learning para-digms, learning depends critically on thestructure of the network and the interactionsbetween computing units. In supervisedlearning, both the pattern of weight strengthsthat emerge in learning a given mapping, andthe ability of the network to respond to novelinputs, is self-organized by the network itself.In unsupervised learning, the nature of theinput/output mapping produced dependson the interaction between network structureand the nature of the input patterns.

How a given task is learnt, or whether itwill be learnt at all, depends on the structureof the network. Using recently developedmethods, the structure of the neural networkcan be built up whilst the task is being learnt,rather than training a network with a fixedarchitecture. The network structure becomestailored to the specific problem and in manycases it is claimed to yield better perform-ance than networks with a fixed architecture.By analogy, neural constructivism describesthe idea that the development of structuressuch as the neocortex involves a dynamicinteraction between the mechanisms of


Morris-01.qxd 2/23/05 5:03 PM

growth and environmentally driven neuralactivity. How information is represented inthe cortex depends on the particular compu-tational task being learnt.

Neural network models have been usedto simulate human performance on specificcognitive tasks. This area of research isknown as connectionism. Typically, the mod-eller prescribes a specific network structure.The strengths (weights) of the connectionsbetween ‘neurons’ are chosen randomly, sothat the model does not contain any prepro-grammed knowledge. One early and influ-ential example of the use of neural networkmodels in this context was Rumelhart andMcLelland’s model of past tense learning(Rumelhart and McClelland, 1986a). Theyshowed that a neural network could learnthe mapping between the stem form of averb and the actual past tense without theneed for the rule that prescribes such trans-formations to be specified externally. Inaddition, the U-shaped pattern of learningcharacteristic of a child’s learning is pro-duced automatically by application of thismodel. Whilst this model has been subject tomuch criticism, its development and appli-cation did demonstrate that there are alter-native ways to viewing language acquisitionthan the application of preprogrammed rules.

4 SELF-ORGANIZATION AS ARESPONSE TO DAMAGE

It has long been known that the effects of brain injury early in life are much lesssevere than similar effects in the adult. Untilrelatively recently, the commonly held viewhas been that this is because the brain has acapacity for plasticity during developmentthat can be brought into play as a response toinjury. It is now clear that the adult brain hasa much higher capability for reorganizationthan previously thought. Recent experimen-tal studies illustrate that the adult mam-malian nervous system does have substantialcapacity to reorganize itself functionally.Plasticity should be thought as a property ofboth developing and adult systems.

4.1 Self-reorganization

A landmark study was made by Raismanin the early 1970s on experiments in rats.This involved partial denervation of theseptal nuclei, which are part of the limbicsystem that includes the hippocampal for-mation and the amygdala. The lateral sep-tum has two main inputs. When either oneof these was cut, there was a large tempor-ary reduction in the number of synapses onthe septal cells. Over the following twoweeks, the number of synapses returned tonormal levels but all the synapses were nowthe type characteristic of the intact input.The same result was obtained whicheverseptal input was cut. This study is import-ant as it provided the first solid evidence ofplasticity at the cellular level in the adultmammalian central nervous system.

Since that time, there have been severalstudies showing substantial plasticity inmammalian neocortex. Many sensory andmotor modalities have multiple representa-tions on the neocortex. Originally it wasthought that these maps would be relativelypermanent once they have formed. However,it is now established that the neocortexretains a large degree of plasticity through-out adult life.

In the 1980s, Merzenich and colleagues{?1} found in monkeys that, following periph-eral nerve injury, the ordered mapping ofthe body surface onto primary somatosen-sory cortex becomes reorganized substan-tially. Electrophysiological recordings weremade to examine the responsiveness of theareas that had responded to stimulation ofthe cut sensory nerve. When recordingswere made soon after surgery, it was foundthat a large part of each of these areas wasresponsive to cells from neighbouring areasof skin, which normally projected to neigh-bouring cortical areas. If the area was large,there was a region in the middle from whichno response to sensory stimulation could beobtained. Over the next month or two, thecortical representation of nearby somaticareas gradually expanded into the unre-sponsive area until the entire somatosensory

4 SELF-ORGANIZATION AS A RESPONSE TO DAMAGE 27

Morris-01.qxd 2/23/05 5:03 PM

region of the cortex responded to sensorystimulation, making a map that was a reor-ganized and distorted version of the origi-nal one.

Another manipulation that has been triedis removing part of the body instead of den-ervating it. For example, removal of wholedigits caused the affected area of cortex tocome under the control of the digits fromeither side of the ablated ones. Most likely,this reorganization involved two differentmechanisms. The gradual spreading of thesomatosensory projection seen in the longterm is likely to involve an anatomicalrearrangement involving the regrowth ofconnections. However, this type of change isrelatively slow and so cannot also accountfor the changes seen after surgery. The mostlikely explanation is that surgery triggers anunmasking of a population of silent synapses,agreeing with inferences drawn from earlierexperiments on the cat spinal cord.

Similar effects have been found in bothprimary auditory and visual cortex in mon-key. The auditory cortex contains a one-dimensional map of frequency, with highfrequency tones represented in caudalregions and low frequencies more rostrally.Destruction of nerve fibres in the cochleawhich are responsive to high frequenciescaused, a few months later, a reorganizationof this tonotopic map with low frequenciesbeing represented rostrally and mid-rangefrequencies more caudally. In primary visualcortex, small retinal lesions initially producean area of unresponsive visual cortex. Overthe next few months, this cortical regiongradually acquires new receptive fields fromplaces in the retina near the site of the lesion.

These effects are also seen in humans. Inthe 1990s, Ramachandran and colleagues{?1} carried out a series of studies on sensoryreorganization following limb amputation.On examination a few weeks after amputa-tion of an arm, patients reported sensationsfrom their phantom limb which were referredto regions of the face and the intact arm. Inseveral cases it was possible to define fieldsof sensation across regions of the face inwhich the normal somatotopic organization

of the digits of the hand was maintained. Inother cases these same types of map werereported in the region of the operated armabove the level of the amputation.

The most likely explanation of theseresults is again that the sensory fields in cortex that were adjacent to the part ofsomatosensory cortex that has suffereddeafferentation had invaded the deaffer-ented region. It might be noted that in thenormal somatosensory map, representationof the face is near to representations of thehand and the arm.

4.2 Can the Nervous SystemRegenerate After All?

The commonly held view is that dam-aged axons in the mammalian nervous sys-tem will regenerate in certain cases butnerve cells will not. Damage to major axontracts or large areas of nervous tissue leadsto permanent loss of function at the neur-onal level as neither damaged nor killedaxons will regenerate. In the mammalianperipheral nervous system, limited repair ispossible as axons can regenerate, leading tothe restoration of functional connectionswith other nerve cells and muscle fibres. Incontrast, invertebrates and non-mammalianvertebrates have the capacity to regenerateaxons and thereby nerve connectionsthroughout their nervous system.

The view that nerve cells cannot regener-ate is being challenged. Recent researchshows that an important class of cells, calledstem cells, exists in many parts of the adultmammalian nervous system. These cells candifferentiate into all types of cell, includingnerve cells.

Stem cells have been found in the dentategyrus of the hippocampus and in the olfac-tory lobe. It may be that nerve cells can begenerated from these cells, even duringadult life, possibly to replace damaged nervecells. One series of experiments observedcontinuous formation of cells (not identifiedas nerve cells) in adult mouse neocortex.Targeted destruction of nerve cells in a smallregion of neocortex then led to a population


Morris-01.qxd 2/23/05 5:03 PM

of new cells being generated, a small propor-tion being nerve cells. These new nerve cellsformed connections with other neural struc-tures, suggesting that indeed they were sub-stituting for the set of destroyed cells.

Research into the potentialities of stemcells is a current hot topic and new resultsare reported frequently. For example, a recentpaper (Mezey et al., 2003) reports postmortem analysis of humans who hadreceived bone marrow transplants as treat-ment for leukaemia. In the four patientsstudied, there was evidence for newly gen-erated nerve cells in the brain, which hadderived from the bone marrow transplants.These findings are still preliminary and con-troversial but may yet change the view thatindividual nerve cells cannot regenerate.

5 OPEN QUESTIONS

In attempting to understand a complexsystem such as the nervous system, it isimportant to identify important generalprinciples of operation. ‘Self-organization’is one such principle. It manifests itself dur-ing both development and the functioningof the nervous system. This section suggestskey questions for activities that could ariseout of the work reviewed here, both for thefurtherance of research into neuroscienceand for the construction of new types ofcomputational (‘cognitive’) systems. It isworth restating the obvious point that thesetwo activities have different goals. One is concerned with understanding a given sys-tem: the other is concerned with designing asystem to achieve a particular computa-tional task, where the nature of the internalworkings of the system are dictated by thetask to be accomplished rather than havingto reflect any biological plausibility.

5.1 Questions for the Neurosciences

The term self-organization refers to apostulated mechanism rather than a collec-tion of phenomena such as those embracedby, for example, perception or learning and

memory. The nature and scope of otherexamples of self-organization remain to bedetermined. Therefore, questions such as‘What is the future for research into neuralself-organization?’ are premature. Instead Ifocus on those aspects of the life scienceswhich are related to the areas of researchdescribed in this chapter.

5.1.1 Levels of AnalysisThis chapter describes how self-

organization operates at the synaptic, cellu-lar and network levels. At what other levelsmay self-organization apply?

● The cognitive level The role of self-organ-ization within developmental cognitionwas described briefly in section 3.2. Tomake sense of self-organization at thecognitive level, it is important to be ableto identify the nature of the elements thatdo self-organize. As this type of self-organization involves extensive regionsof the brain, this will require the use ofpowerful methods for assaying wholebrain activity to identify these elements.As mentioned already, modelling is a cru-cial experimental tool here. To apply com-puter modelling successfully, modelsmust contain the correct degree of neuro-biological realism.

● The subcellular level Although we nowhave at our disposal several completelysequenced genomes, we are still for themost part remarkably ignorant abouthow genes interact and regulate eachother to develop the nervous system. Aplausible picture of cellular regulationwould involve networks of multipleinteractions, with feedback between alllayers. There is great scope for exploitingthe parallels between maps of metabolicpathways and those of gene expression,both of which involve processes of globalcontrol by self-organization. More widely,the similarities between the patterns ofconnectivity over ecological, neural andbiochemical networks are being com-pared (Lee et al., 2002; Milo et al., 2002).

5 OPEN QUESTIONS 29

Morris-01.qxd 2/23/05 5:03 PM

Arguably, perhaps the only system ofmany interacting elements that has beencharacterized in detail is the artificialneural network. There must be a parallel,so far unexploited, between neural net-works and subcellular networks.

● Crossing the levels Quite often, testing apostulated mechanism requires observa-tion of phenomena at one level and theidentification of mechanism at lower levels. More links between levels arerequired to achieve this. For example,there is still no solid evidence linking thenature of the nerve connections makingup the topographically ordered maps andthe underlying molecular mechanisms.This will require a cross-disciplinaryapproach. In this case, what is needed is:experimental evidence of the nature ofthe connections made at both the electro-physiological and anatomical levels; evi-dence for the distribution of signallingmolecules or patterns of neural activityamongst nerve cells that carry the infor-mation enabling the correct connectionsto be made; and an explanatory frame-work to link the two levels. New imagingmethods that obtain information at manydifferent levels, particularly the protein,synaptic, cellular, network and wholebrain levels will be crucial.

5.1.2 The Use of Mathematical andComputer Models

Often the consequence of any givenhypothesis involving a large number ofinteracting elements can only be obtainedby constructing and analysing the proper-ties of computer and mathematical models.This approach is now recognized withinneuroscience as an important means of for-malizing ideas and concepts and testingthem out for their self-consistency andagainst the large amount of neuroscientificdata that is becoming available, and formspart of the field of neuroinformatics. Manyareas of neuroscience described in this chap-ter have profited from the application ofcomputer and mathematical models.

● Neural simulators There is an increasingtrend towards standard powerful neuralsimulators that will enable researchers toshare data, models and results.

● New types of model Completely new typesof model will be required in some cases,such as those required to model theregeneration of nerve cells (if indeed thisoccurs) as described in Section 4.2.

● Modelling of real nervous systems Thepower of the present generation of com-puters is now sufficient to enable simula-tion of the complex geometry of thenervous system, which is an importantconstraint on function. For example, uptill now, nerve cells have been often mod-elled as abstract entities rather than occu-pying particular positions in a complexthree-dimensional environment.

5.1.3 Evidence from InvertebratesThis chapter has been restricted almost

entirely to vertebrates, with very little dis-cussion about the organization of inverte-brate nervous systems. None the less, twogeneral comments can be made.

● Lessons to be learnt from the evidence:Observation of the same type of phenom-ena in invertebrates may suggest com-pletely different types of mechanism. Forexample, much of the evidence at thesynaptic and cellular level suggests a veryprecise and inflexible organization. Forexample, in the neuromuscular system ofthe fruit fly, Drosophila, there is a preciseand fixed relation between a motor neuronand the muscle fibre that it innervates.There are various possible reasons for this.In this case, for example, it may be that amore subtle form of self-organizationoperates; or it may be that for the smallernervous system the genome can afford tospecify precisely all the parameters valuesneeded which have a smaller number ofneurons. Knowledge of invertebrate ner-vous systems can provide another sourceof inspiration for the physical sciences.

● A complete explanation: To have a com-plete knowledge of how the underlying


Morris-01.qxd 2/23/05 5:03 PM

neural substrate generates and controlsthe animal’s behaviour it is necessary tounderstand it at many different levels,from genes through proteins, synapses,cells, networks to behaviour. In the chap-ter, emphasis has been on the propertiesof mammals and other vertebrates.However, it is more likely that the firstcomplete understanding will be obtainedfor an invertebrate system. One primeexample is Drosophila. It has a knowngenome, containing around 15 000 genes,with a variety of complex behavioursincluding those involving learning andmemory. In addition, newly developedimaging techniques can be applied thatenable visualization of nervous systemactivity at the subsynaptic, synaptic andcellular and network levels.

5.2 Inspiration for Other Sciences –‘Cognitive Systems’

Replication or inspiration? Whereas livingand artificial systems are made out of thesame basic set of elements, namely atoms,clearly their structures are different – siliconchips are different from pieces of brain. Thenature of the substrate constrains the prop-erties of the system. This means that whereasit may be possible to build a system thatmimics the input/output relations of a liv-ing system, at the mechanistic level the sys-tems will be different and will operate indifferent ways. As a consequence, resultsfrom neuroscience can at best only inspirethe construction of new types of computingdevice rather than lead to the constructionof exact replicas of living systems. To take awell-known example, neuroscience hasinspired the growth of the field of artificialneural networks. This can offer many inter-esting ideas for the construction of newcomputing devices themselves rather thangive a precise blueprint for such a system.

Three inspirations:

● As emphasized throughout this chapter,the mathematics developed to describeneural self-organization, itself influenced

from studies of physical systems, haswide application. Applications of self-organization to social systems and toeconomic systems are those that have notbeen mentioned so far.

● Using principles of self-organization tosolve problems of coordination amongautonomously interacting agents, such asthose that occur within e-communities, isan obvious specific application.

● Neural self-organization will serve as aninspiration to self-repair technology asan example of application to a hardwareproblem.

Acknowledgments

My very grateful thanks to FionaJamieson, Stephen Eglen, Kit Longden,David Price and Peter Dayan for readingand commenting on this chapter. Any errorsand omissions are mine.

Further Reading

This chapter draws on a large body of lit-erature. Instead of providing a long list ofpapers, I give below a number of texts thatcover most but probably not all of the workI have discussed, together with descriptionsof their various scopes. In addition, a fewimportant papers are included which arereferenced in the text.

Self-organization There is no book speci-fically about self-organization and the ner-vous system. Kauffman (1993) discussestheoretical aspects of self-organization inevolution and Camazine et al. (2003) is arecent book discussing self-organization inbiological communities, typical examplesbeing aggregates of bacteria, communitiesof fireflies, fish and ants.

Mathematical basis The book by Murray(1993) is a classic, concentrating on the math-ematical basis of theoretical developmentalbiology; Edelstein-Keshet (1987) is a verygood alternative.

Development Alberts et al. (1994) is a textat the molecular biology level and Wolpert

5 OPEN QUESTIONS 31

Morris-01.qxd 2/23/05 5:03 PM

(1991) is a readable introduction to embry-onic development for non-specialists.

Nervous system Nicholls et al. (2001) is aclassic treatise on the nervous system,which was first published in 1977 and hasundergone continual revision since then.Shepherd (1994) and Bear (2001) are recentresearch level text books.

Development of the nervous systemPurves and Lichtman (1985), Sanes et al.(2000) and Brown et al. (2001) are researchlevel texts on the development of the ner-vous system, the latter two being morerecent. The recent research monograph byPrice and Willshaw (2000) describes genetic,molecular, systems and modellingapproaches to understanding neocorticaldevelopment. Elman (1996) takes a connec-tionist approach to development.

Neural networks Excellent texts amongstthe plethora available are those by Hertz et al. (1991) and Bishop (1995). Rumelhartand McClelland (1986a,b) are two collec-tions of classic papers. Arbib (2003) is anencyclopaedic collection of short paperswritten by experts on many different aspectsof theories of brain function, connectionismand artificial neural networks. As the namesuggests, the treatment of the nervous sys-tem is generally at the cellular and networklevel.

Brain damage and repair Fawcett et al.(2001) is a collection of papers reviewingrecent research in this field.

ReferencesAlberts, B., Bray, D., Lewis, J., Raff, M., Roberts, K.

and Watson, J.D. (1994) Molecular Biology of theCell. New York: Garland Publishing.

Arbib, M.A. (ed.) (2003) The Handbook of BrainTheory and Neural Networks, 2nd edn.Cambridge, MA: MIT Press.

Bear, M.F. (2001) Neuroscience: Exploring the Brain,2nd edn. Baltimore, MD: Lippincott Williamsand Wilkins.

Bishop, C.M. (1995) Neural Networks for PatternRecognition. Oxford: The Clarendon Press.

Brodmann, K. (1909) Vergleichende Lokalisationslehreder Groshirnrinde in ihren Prinzipien dargestelltauf Grund des Zellenbaues. Leipzig: J.A. Barth.

Brown, M., Keynes, R. and Lumsden, A. (2001)The Developing Brain. Oxford: OxfordUniversity Press.

Camazine, S., Deneuborg, J.-L., Franks, N.R.,Sneyd, J., Theraulaz, G. and Bonebeau, E.(2003) Self-Organization in Biological Systems.Princeton, NJ: Princeton University Press.

Edelstein-Keshet, L. (1987) Mathematical Models inBiology. Birkhauser {?2}.

Elman, J.L. (1996) Rethinking Innateness: AConnectionist Perspective on Development.Cambridge, MA: MIT Press.

Fawcett, J.W., Rosser, A.E. and Dunnet, S.B. (2001)Brain Damage, Brain Repair. Oxford: OxfordUniversity Press.

Hebb, D. (1949) The Organization of Behavior. NewYork: Wiley.

Hertz, J., Krogh, A. and Palmer, R.G. (1991)Introduction to the Theory of Neural Computation.Reading, MA: Addison–Wesley.

Hubel, D.H., Wiesel, T.N. and LeVay, S. (1977)Plasticity of ocular dominance columns inmonkey striate cortex. Phil. Trans. R. Soc. Lond.Ser. B, 278: 377–409.

Karmiloff-Smith, A. (1992) Beyond Modularity: ADevelopmental Perspective on Cognitive Science.Cambridge, MA: MIT Press.

Kauffman, S.A. (1993) The Origins of Order.Oxford: Oxford University Press.

Lee, T.I., Rinaldi, N.J., Robert, F., Odom, D.T. et al.(2002) Transcriptional regulatory networks insaccharomyces cerevisiae. Science, 298:799–804.

Meinhardt, H. (1982) Models of Biological PatternFormation. New York: Academic Press.

Mezey, E., Key, S., Vogelsang, G., Szalayova, I.,Lange, G. and Crain, B. (2003) Transplantedbone marrow generates new neurons inhuman brains. Proc. Natl Acad. Sci. USA, 100:1364–1369.

Milo, R., Shen-Orr, S., Itzkovitz, S., Kashtan, N.,Chklovskii, D. and Alon, U. (2002) Networkmotifs: simple building blocks of complex net-works. Science, 298: 824–827.

Murray, J.D. (1993) Mathematical Biology, 2nd edn.Berlin: Springer-Verlag.

Nicholls, J.G., Fuchs, P.A., Martin, A.R. andWallace, B.G. (2001) From Neuron to Brain, 4thedn. Sunderland, MA: Sinauer Associates.

Price, D.J. and Willshaw, D.J. (2000) Mechanisms ofCortical Development. Oxford: Oxford Univer-sity Press.

Purves, D. and Lichtman, J.W. (1985) Principles ofNeural Development. Sunderland, MA: SinauerAssociates.


Morris-01.qxd 2/23/05 5:03 PM

Rasmussen, C.E. and Willshaw, D. (1993)Presynaptic and postsynaptic competition inmodels for the development of neuromuscu-lar connections. Biol. Cybern., 68: 409–419.

Rumelhart, D.E. and McClelland, J.L. (1986a)Parallel Distributed Processing: Explorations inthe Microstructure of Cognition. Vol. 1:Foundations. Cambridge, MA: MIT Press.

Rumelhart, D.E. and McClelland, J.L. (1986b)Parallel Distributed Processing: Explorations inthe Microstructure of Cognition. Vol. 2:Psychological and Biological Models. Cambridge,MA: MIT Press.

Sanes, D.H., Reh, T.A. and Harris, W.A. (2000)Development of the Nervous System. New York:Academic Press.

Shepherd, G.M. (1994) Neurobiology, 3rd edn.Oxford: Oxford University Press.

Spemann, H. (1938) Embryonic Development andInduction. New Haven, CT: Yale UniversityPress. (Reprinted, New York: Garland, 1988.)

Sperry, R.W. (1963) Chemoaffinity in the orderlygrowth of nerve fiber patterns and connec-tions. Proc. Natl Acad. Sci. USA, 50: 703–710.

Sur, M., Garraghty, P.E. and Roe, A.W. (1988)Experimentally induced visual projections inauditory thalamus and cortex. Science, 242:1434–1441.

Tootell, R.B.H., Silverman, M.S., Switkes, E. andDe Valois, R.L. (1982) Deoxyglucose analysisof retinotopic organization in primate striatecortex. Science, 218: 902–904.

Turing, A.M. (1952) The chemical basis of mor-phogenesis. Phil. Trans. R. Soc. Lond. Ser. B, 237:37–72.

Von der Malsburg, C. (1973) Self-organization oforientation sensitive cells in the striate cortex.Kybernetik, 14: 85–100.

Willshaw, D.J. and von der Malsburg, C. (1976)How patterned neural connexions can be setup by self-organization. Proc. R. Soc. Lond. B,194: 431–445.

Willshaw, D.J. and von der Malsburg, C. (1979) Amarker induction mechanism for the estab-lishment of ordered neural mappings: Itsapplication to the retinotectal problem. Proc.R. Soc. Lond. B, 287: 203–234.

Wolpert, L. (1969) Positional information and thespatial pattern of cellular differentiation. J. Theor. Biol., 25: 1–47.

Wolpert, L. (1991) The Triumph of the Embryo.Oxford: Oxford University Press.

Woolsey, T.A. and van der Loos, H. (1970) Thedescription of a cortical field composed of dis-crete cytoarchitectonic units. Brain Res., 17:205–242.

REFERENCES 33

Morris-01.qxd 2/23/05 5:03 PM

how to design a cognitive system - elsevier.com

Documents