NATIONAL PHYSICAL LABORATORY

SYMPOSIUM No. 10

Mechanisation of Thought Processes

Proceedings of a Symposium held at

the National Physical Laboratory

on 24th, 25th, 26th and 27th November 1958

VOLUME I

LONDON : HER MAJESTY'S S T A T I O N E R Y O F F I C E

1 9 5 9

(Reprinted 1962)

SESSION 3

PAPER 6

P A N D E M O N I U M : A P A R A D I G M F O R L E A R N I N G

by

D?. 0. G. S E L F R I D G E

Cllver G. Selt'rlci~e was born I n London 10 M?;; 15£'C He studied 2t the Massachusetts Institute of Tfcci-iriology frorr. 192-1945, returning post- ~-rar i .da tc j .y ??or I94?-1950. Alter 2 y e a r s a t S! gna l Corzs Laticrstorl e s at Fort Mor.mouth. Ncv Jersey, he 2 o l n e d Lincoln Labora- t o r i e s In GX'JD 34, Corr'iunicatlon Techniques, of W:-.lch he

Is now Group Leader.

PANDEMONIUM: A PARADIGM FOR L 7 A R M M

DR. 0. G. SELFRIDGE

INTRODUCTION

WE a r e proposing he re a model o f a process which we claim can a d a p t i v e l y improve i t s e l f t o handle c e r t a i n p a t t e r n r ecogn i t ion problems which cannot be adequate ly s p e c i f i e d In advance. Such p rob ie r s a r e usua l when t r y i n g t o b u i l d a macnine t o I m i t a t e any one of a very l a r g e c l a s s of human d a t a process ing techniques. A speech t y p e w r i t e r I s a good example of soneth ing chat very many people have been t r y i n g unsuccess fu l ly t o b u i l d f o r some time.

We do n o t sugges t t h a t we have proposed a model vhlcn can l e a r n t o typewri te from merely h e a r i n g speech. Pandemonium does no t , however, seem on paper t o have the same k inds of l nne ren t r e s t r i c t i o n s o r i n f l e x i b i l i t y t h a t many previous proposals have had. Trie b a s i c motif Dehind ou r model i s the n o t i o n of p a r a l l e l process ing . This i s suggested on two grounds: f i r s t , i t i s o f t e n e a s i e r t o handle d a t a In a p a r a l l e l nanner, and, Indeed, I t is u s u a l l y the more " n a t u r a l " manner t o handle i t in; and, secondly, i t I s e a s i e r t o modify an assembly of quasi-independent modules than a machine a l l of whose p a r t s i n t e r a c t immediately and in a complex way.

We a r e n o t going to apologize f o r a f requent use 01 antnropomorphic o r ~ i o m o r p h i c terminology., They seem t o be u s e f u l words t o desc r ibe our not ions .

What we a r e descr ib l r ig i s a process , or , r a t h e r , a model of a process. We s h a l l no? desc r ibe all t P n reasons thac l e d t o i t s particular formulation, out we s h a l l give some reasons f o r noping t h a t i t does lr f a c t possess the f l e x i b i l i t y and a d a p t a b i l i t y t n a t we a s c r i b e t o i t .

THE PROBLEM ENVIRONMENT FOR LEARNING

PandeTionium i s a -nodel hrhicn vie nope can l e a r n t o recognize p a t t e r n s which nave n o t been s p e c i f i e d . We nean t n a t i n cue f o l i o Jing sense: we p resen t t o the model examples of p a t t e r n s taken from some s e t of theri, each time I n f o m i n g tne nodel which p a t t e r n we n i d j u s t presented. Then, a f t e r some time tne model guesses Cor rec t ly which p a t t e r n has J u s t been presented beft -d

we inform i t . For a l s r g e c l a s s of p a t t e r n r e c o m i t l o n ensembles there has never e x i s t e d any adequate w r i t t e n o r s t a t a b l e d e s c r i p t i o n of t he d i s t l n c - t i o n s between the pa t t e rns . The on ly requirement '/le can p l a c e on our modal I s t h a t ;-;e want i t t o benave i n tC.5 same '::ay t h a t Xen observably behave In. In an abso lu te sense t h i s I s a v e r y u n s a t i s f a c t o r y de l ' i n l t i on of any task:, but i t may be appa ren t t h a t i t I s the ':;a; In which most t a s k s a r e def ined f o r most men. Lucky is he whose job can 'oe e x a c t l y s p e c i f i e d i n words withouc any ambiguity or necessa ry Inferences . The example we s h a l l I l l u s t r a t e i n some d e t a i l is t r a n s l a t i n g from manually keyed Morse code Into, say, t ypewr i t t en messages. Now i t I s t r u e t h a t when one l e a r n s Morse code one l e a r n s t h a t a dash should b s e x a c t l y th ree t imes the l e n g t h of a dot and so on, but i t t u r n s ou t t h a t cl-.ls I s r e a l l y mos t ly I r r e l e v a n t . What m a t t e r s is only what tne v a s t army of people who use Morse code and with whom one I s going to have t o communicate unders tand and p r a c t i s e when they u s e i t . I t t u r n s ou t t h a t t h i s Is n e a r l y alw?ys ve ry d i f f e r e n t from school book Morse.

In the same way the only adequate d e f i n i t i o n of the p a t t e r n of a spoken word, o r one hand-written, must be In terms of the consensus of the people who a r e u s i n g i t .

We use the term p a t t e r n r e c o g n i t i o n In a broad sense t o inc lude n o t only t h a t d a t a process ing by which images a r e ass igned t o one o r another p a t t e r n . in some s e t of p a t t e r n s , b u t a l s o the p rocesses by which the p a t t e r n s and d a t a proc,essing a re developed b y the organism .er machine; we g e n e r a l l y c a l l t h i s l a t t e r " l e a r n i n g f f e

PANDEMONIUM, IDEALIZED AND PRACTICAL

We first c o n s t r u c t an i d e a l i z e d pandemonium (figa 1 ) . Each poss ib l e P a t t e r n o f the s e t , r ep resen ted by a demon In a box, computes h i s s i m i l a r i t y w i t h tne image s imul taneously on view t o a l l o f them. and g ives an output depending monotonlca l ly on t h a t s i m i l a r i t y . The d e c i s i o n demon on t o p makes a choice of t h a t p a t t e r n belonging t o the demon whose output was the l a r g e s t . *

* This I s an e x a c t c o r r e l a t e of a c c m u n i c a t . l o n s sys tem ,;.herein given a r e c e i v e d message M W and a n- int tier of poss ib l e t r a n s m i t t e d messages !'.(!), t h a t . " 1s

chosen, t h a t I s , deemed t o have been t r a n s m i t t e d , ~ i h l c h minimizes ~ v c T ) - ~ ~ ~ ( I ) I2dI. (Such a procedure I s opt imum under c e r t a i n c o n d i t i o n s ) . T h i s I n t e g r a l Is, a s I t were, t he squa re of a d i s t a n c e in a s i g n a l phase space - f ie .2 - and t h u s t h a t trans-ri l t te ' i ne s sage I s s e l e c t e d t h a t Is most s i n l l a r t o t he r e c e i v e d one.

PICKS THE LARGEST OUTPUT FROM THE

COGNITIVE DEMONS, WHO INSPECT THE

CATA OR IMAGE.

Each demon may, f o r example, be ass igned one l e t t e r of the a lphabet , so t h a t the task: of the A-demon I s t o shout a s loud of the amount of 'A-nessl t h a t he s e e s In the image.* Now i t w i l l u s u a l l y happen t h a t w i th a reasonable c o l l e c t i o n ol' c a t e g o r i e s - l i k e the l e t t e r s of the a lphabe t - the conputa- l i o n s cerformed by each o f t h e s e idea l cogn i t ive demons w i l l be l a r g e l y the same. In many Ins tances a p a t t e r n I s n e a r l y equ iva len t t o some l o g i c a l funct ion of a s e t of f e a t u r e s , each of which is i n d i v i d u a l l y comon t o perhaps s e v e r a l p a t t e r n s and whose absence is a l s o comon t o s e v e r a l o the r pa t t e rns . ?

t he re f o r anend our i d e a l i z e d Pandemcnium. The amended ve r s ion - fie. 3 - has scme profound advantages, ch ie f among which is i t s suscept- i b i l i t y t o t h a t kind of adap t ive self-improvement t h a t I c a l l learning.

The d i f f e r e n c e between f i g . 1 and f i g .3 is t h a t the common p a r t s of the computations t h a t each c o g n i t i v e demon c a r r i e s o u t i n f i g . 1 have in f i g . 3 been a s s igned i n s t e a d t o a h o s t of subdemons. A t t h i s s t a g e the organiza- t i c n has four l e v e l s . A t the bottom the d a t a demons se rve merely t o s t o r e and pass on the da t a . A t the nex t l e v e l the computational demons o r sub- demons perform c e r t a i n more o r l e s s complicated computations on the d a t a and pass t h e r e s u l t s of t hese up t o the n e x t l e v e l , the cogn i t ive demons who weigh the evidence, a s i t were. Each cogn i t ive demon computes a s h r i e k , and from a l l the s h r i e k s the h i g h e s t l e v e l demon of a l l , the dec i s ion demon, mere ly s e l e c t s the loudes t .

THE CONCEPTION OF PANDEMONIUM

We cannot ab I n i t i o know the i d e a l cons t ruc t ion of our Pandemonium. We t r y t o a s s u r e t h a t I t con ta ins the seeds of self-improvement. Of the f o u r l e v e l s i n fip. 3. a l l bu t the t h i r d , t he subdemons, which compute, a r e s p e c i f i e d by the task. For the t h i r d l e v e l , t he re fo re , we c o l l e c t a l a r g e number of poss ib l e u s e f u l func t ions , e l lmlna t lng a p r i o r i Only those which could n o t conceivably be r e l e v a n t , and make a reasonable S e l e c t i o n of the o t h e r s , be ing bound by economy and space. We then guess reasonable weights f o r them. The behaviour a t t h i s poin t may even De accep tab ly good, bu t Usual ly i t must be improved by means of the adap t ive changes we are about t o d iscuss .

* I t I s s o s s l b l e a l s o t o ph rase I t so t h a t t h e A-denon I s concutlng the d i s t a n c e In sone phase of t h e l r a g e from scne Idea l A; I t seems t o m ' Jnnecessar l ly p l a t o n i c t o p o s t u l a t e the e x i s t e n c e o r ' I d e a l 1 r e p r e s e n t a t i v e s o f l a t t e r n s , ana , Indeed, t h e r e a r e o f t e n good r easons f o r n o t d o l n s so .

/Â See, fo r example, J e r o n e Bruner, % s t u d y of Thlnkl~Ig*,

COGNITIVE DEMONS

COMPUTATIONAL DEMONS

DATA OR IMAGE DEMONS

The Evo lu t i on of Pandemonium

There a r e seve ra l k inds of adapt ive changes which we w i l l d i scuss fo r our 7 Pandemonlim. They a r e a l l e s s e n t i a l l y very s i m i l a r , b d t tney nay be p r o g r m e a and discussed separa te ly . The first nay be c a l l e d "Feature Weighting".

Altnough we have n o t y e t s p e c i f i e d what tne cogn i t ive denons conpute, the s o l e t a s k a t present is t o add a weignted SLCT o trie ou tpu t s o f a l l t3e computational demons o r s-Jbdenons; the weightings w i l l of course d i f f e r for

Flg.4. F i r s t hill c l imb ing technique: p i c k v e c t o r s a t rand= ( p o i n t s In the s p a c e ) , s co re tre'n, ard s e l e c t the one t h a t s c a r e s h ighes t .

the d i f f e r e n t cogn i t ive demons, bu t the weight ings w i l l be the only d i f f e r e n c e between them. Feature weight ing Cons i s t s of a l t e r i n g the weigy.ts ass igned to t he subdemons by t h e c o g n i t i v e demons s o a s t o maximize the sco re of the e n t i r e 7 Pandemonium. How then can we do t h i s ?

The Score

What we mean b y the sco re nere I s how well the machine i s doing tne tasA we want i t t o do. This presumes t h a t we a r e moni tor ing the machine and t e l l i n g i t when I t makes an e r r o r and so on, and f o r the r e s t of the d i scuss ion we s n a i l be assuming t'qat we have a v a i l a b l e some such running sco re . Now a t sone po in t we s n a i l be very i n t e r e s t e d i n Having the machine r J n .-.'Ithout t z a t k ind of d i r e c t supe rv i s ion , aKd the ques t ion n a t u r a l l y a r i s e s w r e t r e r the rnacr-ine car. mean ingfu l ly monitor i t s own perforinance. Je answer t h a t ques t ion t e n t a t i v e l y yes, b ' ~ t de l ay d i scuss ing i t t i l l a l d c e r s ec t ion .

~ l g - 5 . secozd n l l l - c l lmblng techniiiue: pick vectors until o f the--, (lpuxltter 4) outscores the previous ones. Then take sho re i ardoj-l steps, retracing any that decrease the sco re .

Feature i feightang and Hi. Ll-Cl t'mbing

The output of any cogn i t ive demon i s

so t h a t the complete s e t o f weights f c r a l l t he cogn i t ive de-nons over a i l the subdemons I s a vec to r ;

Now f o r some (unknown) s e t of weights A, the behaviour of t h i s whole Pandenonimi I s optimum, and the problem of f e a t u r e weighting I s t o f ' ina

MAIN PEAK

KS

Flg.6. General space showing f a l s e peaks . One of t he f a l s e peai ts ~ 3 1 1 t e Isolated f r o " ^ t h e na in o? t rue peak-

t h a t s e t . This l a y be desc r ibed a s a h i l l - c l i m b i n g ~ ? r r t l e m . We have a space (of A) and a funct ion on the space ( t h e s c o r e ) , which we piay cons ider an a l t i t u d e , and whicn \:e ,vlsh t o maximize by a proper s ea rch through A. one poss ib l e technique I s t o s e l e c t weight ing v e c t o r s a t random, s c o r e tnem, and f i n a l l y t o s e l e c t the vec to r t n a t s co red h i g h e s t ( s ee f\g. 4). I t w i l l u sda l ly , however, t u rn o u t t o be p r o f i t a b l e to t ake ~ d v a n t a g e of the c o n t i n u i t y p r o p e r t i e s of tne space, which u s u a l l y e x i s t In some sense, i n t r e following way: s e l e c t v e c t o r s a t random u n t i l you f ind one t n a t s co res perceptiblymorfc tran the o t rers , ?-an t h i s point take s n a i l randor s t e p s In a l l d i r e c t i o n s ( t h a t I s , add smal l random v e c t o r s ) m t i l you f i n d a d i r @ c t ' o n t h a t improves your score. When you f i n e sucn a s t ep , take i t and r epea t the process. Tnis I s I l l u s t r a t e d In fq. 5 , and i s the case of a b l i n d man t r y i n g t o climb a h i l l . There may ce, of course, TanJ f a l s e peaks on wnicn one may f i n d onese l f t rapped In sdcn a procedure {fig. E).

' Ee prol-'le-"; of r a i s e p e x s in searc.'-.in% tec.'.nIc.ues I s an o ld an;; f a n l l i a r one. In gene ra l , one may hope ti-.a: in spaces of ve ry ::i3;h dlxen- s i o n a l i t y t ; ~ e Interdependence c : m e componer.ts and the score is so g r e a t a s t o inE.<e ve ry Qnl lKely L!".? ex i s t ence of f a l s e peaks c o n c l e t e l y I s o l a t e d from Cue m i n o r tree peak;. I t n u s t be r e a l i z e d , however, t h a t :rils is a pu re ly exps r l -~ . e r . t a l ~ ' - i e s t l o n t h a t has to be answered s e p a r a t e l y :or every hil l-cl1rn~~i1-g s i t u a t i o n . I t 50:s tl-irn out In l- . i l l -cl in3ing siti-i?tlor.s tl-.at the choice of s t a r t i n g point I s oc ten very important . The main peak may b e

very prominent, but u n l e s s I t n a s wide-spread foot-3111s it nay r ace a very long t i n e before we eve r 3ezin to ga in a l t i t u d e .

T'nis w.vf b e descr ibed a s or.? of the proolems of tr?.lnirig, n m e l y , t o encourage the nac'nine o r organism t o s e t enough on the f o o t - 5 i l l s so t h a t smal l changes in h i s Darar.etprs d i l l produce n o t i c e a b l e Improvenent i n h i s a l t i t u d e o r score , One can desc r ibe l e a r n i n g s i t u a t i o n s where n o s t of the d i f f i c , ~ l t y o: toe t a s k l i e s ir. f i n d i n g a s y ',-:ay o f irriproving onef s score , s x n a s l e a r n i n g t o r i d e a un lcyc le , where I t takes longer t o s t a y on f o r a second than I t does t o i m ~ r c v e t n a t one second to a minute; and o t h e r s where I t is easy t o do a l i t t l e well and v e r y &rd t o do very wel l , sucn a s l e a r n i n g to p l a y c5ess. I t i s a l s o t r u e t h a t o f t e n tne n a i n geai? i s I ~ l a t e a u r a t h e r than an i s o l a t e d s?i;<eà That i s t o say, o p t i n a l behaviour 01' the inecLanIsm, once re?-ched, pi5y be r a t h e r i n s e n s i t i v e t o the change of some of the parameters,

Subdemon S e l e c t i o n

The second kind of adap t ive change t h a t we wish t o inco rpora t e i n t o ou r Pandemcnluii I s subdemon s e l e c t i o n . A t the conception c f our demoniac assembly vre c o l l e c t e d sonewhat a r b i t r a r i l y a l a r g e nunber of subdemons :;'rile?. we guessed w w l d be u s e f u l and a s s igned thex weights a l s o a r b i t r a r i l y . The f i r s t adapt lve c?ange, f ea tu re weighting. optimlze-i t hese weights. but we have no assurance a t a l l t ha t the p a r t i c u l a r subdemons we s e l e c t e d a re good ones. Sutdemon s e l e c t i o n gene ra t e s new subdemor.~ f o r t r i a l and e l i m i n a t e s i n e f f i c i e n t ones, m a t , i s , ones t h a t ao n o t much h e l p improve the score.

We gropose t o do t h i s i n i t i a l l y by two di:f'.ferent techniques, which may be c a l l e d "mutated f i s s i o a t l and Nconjugat ion" . The first po in t t o note i s t h a t I t I s p o s s i b l e t o a s s ign a worth to each of the s u b d e m s . I t may t e done in seve ra l bays, and we nay, f o r example, wr i t e the worth I$ of t?.e i t h demon

so t h a t tine worthy demons are those hose o u t p u t s a r e l i k e l y t o a f f e c t most s t r o n g l y the dec i s ions ;nade.

We assume t h a t f e a t u r e weight ing h a s a l r e a d y run s o long t h a t the behaviour of the machine has been approximately optimized, and t h a t s c o r e s and worths of xiachine and i t s demons have been obtained. First we e l imina te those subdemons wi th low worths. Next we gene ra t e new subdemons by muta t ing the s u r v i v o r s and reweight ing the assembly. A t p r e sen t we p l an t o p i c k one subdemon and a l t e r some of h i s parameters more o r l e s s a t random. This will u s u a l l y r e q u i r e t h a t we reduce the subdemon himsel f t o some canon ica l form so t h a t the random changes we i n s e r t do n o t have the e f f e c t of r e w r i t i n g the program s o t h a t i t w i l l n o t run o r so t h a t i t w i l l e n t e r a c losed loop wi thout any hope o f g e t t i n g o u t of it.*

Besides muta Led f i s s i o n , we a r e proposing another method of subdeinon improvement c a l l e d l tconjugat ionl l . ou r purpose here is two-Sold: f i rs t t o provide a l o g i c a l v a r i e t y in the func t ions computed by the subdexions, and. secondly, t o provide l e n g t h and complexity In them.

What we do is t h i s : glven two ' u s e f u l ' subdemons. we genera te a new subdemon which is the continuous analogue of one of t he t e n n o n t r i v i a l b i n a r y two-variable func t ions on them. For example, the product of two subdemon output S, corresponding t o the l o g i c a l product , would Suggest trie simultaneous presence of two f ea tu res . The t en n o n - t r i v i a l such func t ions a r e l i s t e d In Table I.

A . BVwA .3 A.?13v-A B Table 1. Non-trivial b i n a r y functions on two var iab les .

CORRECT DECISION '\ (FROM ME)

COMPUTING SUBDEMONS

EVEN -NUMBERED DURATIONS ARE MARKS LETHE ODD- NUMERED DURATIONS ARE SPACES

A S we mentioned before , the e n t i r e n o t i o n of Pandemonium ,-iids conceived 2 S CL i r a c t i c a l way o f au tomat i ca l ly improving data-processing f o r p a t t e r n recogni t ion . our i n i t i a l model t a s k I s to d i s t i n g u i s h :dots and dashes In manually keyed Horse code, so t h a t ou r Pandemonium can be I l l u s t r a t e d In fig. 7. Vote that . the f l s . c t i o n s and behaviour of a l l de.nons have 'ceen s p e c i f i e d except f o r the computing subdemons. We s h a l l r e i t e r a t e tnose spec1 f i ca t ions .

(1) The dec i s ion demon1 s output I s ' d o t 1 o r ' da sh ' according a s tr.e d o t demon's output is g r e a t e r o r l e s s than ine dasfi demon's. (2 ) The cogni t ive demons, d o t and dash, each compute a ',deignted sum of the ou tpu t s of some 150 computing subdeinons. I n i t i a l weights we 'nave ass igned a r b i t r a r i l y , bu t , we hope, reasonably. (5) The data-handling demons r ece ive d a t a In me form of du ra t ions , a1 t e rna t ive l ; c ? marks and spaces, and they pass them ?own the l i n e .

OUT PUT

0 1 - 5 '0 / ̂

Tne computing sxbdernons a r e cons t ruc t ed Tram on ly a ve ry few o p e r a t i o n a l funct ions , which a r e c a r e f u l l y non-binary. For example the subdemons d = d p and do dr , hive t h e i r ou tpu t s shown i n fzg.3. The operational , functions

L

f o l l o w 1 ) l = ' . This funct ion computes the degree of e q u a l i t y of some s e t o l v a r i a b l e s ( see ftg, 5). (2) ' ' , l , compute the aegree to which some v a r i a b l e i s l e s s than o r 1s g r e a t e r than some o the r v a r i a b l e (see f izh 8). (3) 'm&, ' m a x ' , compute the degree t o which some v a r i a b l e I s the l a r g e s t of an a r b i t r a r y s e t of v a r i a b l e s o r an a r b i t r a r y s e t of consec- u t i v e v a r i a b l e s , (4) '(lL , t A t s t o x 1222 degree t o ~.&iIcF. :i;e izh d ~ i ~ a ~ l o r ~ h s been I d e n t i f i e 6 as a dot o r dash. (5) "W computes 2n average o? socie s e t 01 v a r i a b l e : , (S) ', '/I is a family of t r ack iEg means. For example, i t ~'iis::.t co;r@lite

CONCUJSI ON

',&at I s h a l l present a t the meet ing In November w i l l be the d e t a i l s of the p rogres s of Pandemonium on the Yorse code t r a n s l a t i o n problem. The i n i t i a l emblem we have given the machine I s t o dIst ln:?uIsh d o t s and dashes. When the behaviour of the machine has Improved I t s e l f to the p o i n t where l i t t l e f u r t h e r Improvement seems t o be occurr ing , we s n a i l add th ree more cogn i t ive demons, the symbol space, the l e t t e r space, and t h e w r d space. Presumably a f t e r sone f u r t h e r time t h i s new Pandemonium will s e t t l e down t o some unlriprovable s t a t e . Then we s h a l l r e p l a c e the s e n i o r o r decision-making demn '.-,'l th a row of some f o r t y o r so c h a r a c t e r demons wi th a new decls ion- inaKing demon above them, l e t t i n g the new cogn i t ive demons f o r the c h a r a c t e r demons use a l l the I n f e r i o r demons, c o g n i t i v e and otherwise, f o r t h e i r Inputs. I t I s probably a l s o d e s i r a b l e t h a t previous dec i s ions be a v a i l a b l e f o r present dec i s ions , so t h a t a couple of new func t iona l ope ra t ions might be added. There need be l i t t l e concern about 10giCal C i r c u l a r i t y , because we have no requirement fo r l o g i c a l cons i s t ency , bu t a r e merely seeking agreeable Morse t r a n s l a t i o n .

How much of m e whole Drogram w i l l have been run and t e s t e d by Nover/oer I cannot be s u r e of. A t the p re sen t ( J u l y ) we have had some f a i r t e s t i n g o f h i l l - c l imb ing procedures.

ACKNOWLEDGEMENT

I should l i k e t o acknowledge the va luab le con t r ibu t ions o f d i scuss ions wi th many of my f r i e n d s , Inc luding e s p e c i a l l y M. Mlnsky, U. Neisser , F. Fr ick and J. Let tv ln .

DISCUSSION ON THE P A P E R BY DR. 0. G. SELFRIDCrF +

DR. J. W. BRAY: May I a s k Dr . S e l f r i d g e what n e th inks of t h i s approach t o h i s problem? Let X = dura t ion o f t n e l a s t s i g n a l , which may oe a d o t o r a dash, a s i g n a l space, l e t t e r space o r ward Space. Let y = 2, i f i n f a c t i t was a dasn, 1 I f dot , 0 i f s i g n a l space, -1 I f l e t t e r space and -2 i f word space. . , 3 ~

To form the polynomial:

t ake a number of obse rva t ions , a s he sugges ts , and l e t tne machine l e a r n the code by determining t h e c o e f f i c i e n t s Aoetc. by simple c u r v i l i n e a r regress ion . The du ra t ion o f p rev ious and subsequent s i g a l s ana the i n t e r - p r e t a t i o n given t o p rev ious s i g n a l s could be added a s f u r t h e r v a r i a b l e s on the r i g n t hand s ide .

HR. C. STRACHEY: A t Lne end o f me Paper you promise u s some f u r t h e r information about t h e l a t e s t s t a t e of t he programme. Could you l e t u s know what t h i s is?

DP- J. McCARTHY; I would l i k e t o speaK b r i e f l y about some of tne advantages of tne pandemonium model a s an a c t u a l mooel o f conscious oehavlour. In observing a bra in , one should maKe a d l ~ t i n ~ t i ~ n between t h a t a spec t of the behaviour which I s a v a i l a b l e consciously, and tflose Denavlours, no doubt equa l ly important , but whicn proceed unconsciously. I f one conceives of the Drain a s a pandemonl'.~n - a r o l l e c t l o n o f demons - pernaps vhat I s going on wi th in the demons can be regarded a s t h e unconscious p a r t of thought, and what the demons a r e p u b l i c l y shou t ing f o r each o x e r to hear , a s the conscious p a r t o f thought.

DR. D. M. MACKAY: Dr. Sel f r i d g e ' S 'pandemonium' h a s a c e r t a i n fa~mlly resemblance to a c l a s s o f mechanism considered i n some e a r l i e r papers (ref. l ) (tnough I had never suspected i t s demonic iVn$ l l ca t ions . ' ) .

In one of these , ( re f . 2) a f t e r d i scuss ing the gene ra l p r i n c i p l e t n a t you p e n a l i s e t h e unsuccessful , I po in t ed c u t t n a t the amount of i r&forna t ion

1. MACK.iY, D. M. Menta l i t y In Machii-ies. f'rcc. A r i s t . Soc. Sup f t . , 1932, 61. 2. KACKAY, D. M. The Ep i s t emol?g lca l Proclem f o r ~ u t o m a t a , ea. J. McCarthy and

C . E. Shannan Automata S t d e s , ??-inceton (1955). S e e a l s o &it. J . ?syc^iol. , 1956, 47, 30 and Advancement of Science, 1956, 392.

p e r t ~ i c ~ l ( t o use FT. S e l f r i d = e i s neta^iicr) is , e r / m a l l u n l e s s the p r o s a s i l i Z Y of success arc f a i l u r e a r e w a l . I f you ^av - a sb s t e? ,mere t n e r e a r e a v a s t n - i i l e r c f ~ c s s i ~ i l l t i e s co be e l i n i n d t e a t-y r a t n e r fee-dess t r i a l and e r r o r , tnt-n of coarse f a i l u r e occurs "idcn nore o f t e n than success. Tne s o l u t i o n I s J L e s t e d was t o for-n a Kind of s y n d i c a t e d l e a r n i n g p rocess In wnlcn a t f i rs t l a r - t nufl'sers of elements, d e s t i n e d e v e n t u a l l y f o r independence, snouln n s c n n ~ l e d toge the r s o a s to reduce tne d i v e r s i t y of response. A hand, f o r example, n l ~ l t n o t a t f i r s t nave M C ^ f i n -- :er s e p a r a t e l y c o n t r o l l a 3 l e 1 bu t c & ~ l c worc clumsily a s a bdnole. I n t h a t d a ~ , you can g r e a t l y decrease t r e amount of groping &ich i s necessa ry be fo re a success fu l .e. act e

Gay De l e s s . On t h i s p r i n c i p l e , i f pursded t o trie l i m i t , even the e a r l i e s t T r i a l s c o ~ l d nave a non-ned.lgic% chance o f s u c c e s s , (tt-ough success i n a

e r 1 small way^). Given t h i s easy s t W t , s i n p l e se l f -o rgan i s ing subrou t ines can b u i l d up

f a i r l y quickly. A s they i n c r e a s e t h e i r number ana t h e i r success , however, t he Idea I s t h a t the coupl ings between elements should g radua l ly t e d i s s o l v e d t o inc rease toe complexity of t h e problem. I f ~ O J keep uhe complexity i n c r e a s i n g s t e p b,y s t e p wi th the degree o f development o f success fu l I n t e r n a l n a t c n i n g sub-routines, tnen f u l l y adap t ive behaviour can oe enornously 'nore quickly developed than i f m e sys t en s t a r t s vvith tne f u l l r e p e r t o i r e to oe explored. My ques t ion I s why Dr. S e l f r i d g e n a s n o t i nco rpora t ed t h i s p r i n c i p l e in n i s l~andemonium' , so t n a t eacn ' k i c ~ ' ~ o u l a nave sometnin," n e a r e r CO one oli, of i ~ i f o n n a ~ i o n i n s t e a a of an - i l ~ i i o s ~ n e ~ l l g i b l e f r ac t ion .

38, ? C ORICE \ w r i t t e n con t r lou t l on \ " Tnls i s a very I n t e r e s t i n g anc s t i n i ~ l a t i n g paper. I nave one coniflent t o m x e , and t h a t i s on t h e d iscus- s ion of "Feature Wei&tIne and dill-ClirnDingH.

I t h i n k t h a t in t h i s d i scuss ion the a u t h o r has not Drought cut, one impor tant d i s t i n c t i o n between types of f lhi l l-cl imbingfl problems - t h a t between determinate and s t o c h a s t i c problems. Whereas i n a de terminate problem Lne " h i l l u i s de f ined by a s i n g l e func t ion o f many v a r i a b l e s

i n a s t o c h a s t i c problem I t I s de f ined by the mean va lue of a number of funct ions :

and a is a random v a r i a b l e whose value depends on t h e p a r t i c u l a r t r i a l made.

Now I would have thought t h a t n o s t o f the more i n t e r e s t i n g p a t t e r n recogni t i on problems a r e e s s e n t i a l l y s t o c h a s t i c : t h e obj e c t i v e is t o maxe

a macnine t n a t w i l l o b t a i n a s n i g n a p r o p o r t i o n as p o s s l . - ' l e o r c o r r e c t answers t o a s e r i e s o f q u e s t i o n s Itwhat i s t h i s p a t t e r n ? " r e f e r r i n g t o a r3ndoia s e r i e s o f p a t t . e r n s .

Tile a u t n o r h a s felven an e x c e l l e n t account o f some of trie d i f f i c u l t i e s o f d e t e r m i n a t e h i l l - c l i n b i n s and o f t n e t e c h n i a u e s f o r overcornin& thel", b u t s t o c h a s t i c h i l l - c l i m b i n ^ : p r e s e n t s a d d i t i o n a l p rob lems , and I thinK t h i s :,';ay become v e r y a p p a r e n t when a pandemonium is ~ u i l t t o a e a l w i t n a p r a c t i c a l t ask . I n p a r t i c u l a r , two prob lems which w i l l n e e d Invest!.-ation a re :

(1) how l a r g e a s a n p l e o f t r i a l v a l u e s f(x) and f d y ) , b e l o n g i n g t o two p o i n t s x and y i n -.the v e c t o r s p a c e b e i n g e x p l o r e d , w i l l be needed t o o b t a i n a s a t i s f a c t o r y e s t i m a t e o f [ f ( x ) - fi)] f o r t h e p u r p o s e s o f h i l l - c l l r f A I n g ?

(11) hovi iruch l o n g e r w i l l a p a r t i c u l a r s r c h a s ti c h i l l - c l i r l ' s i n g p r o c e d u r e t a k e than t h e c o r r e s p o n d i n g d e t e r m i n a t e p r o c e d u r e , a n a w i l l t h i s r a t i o p rove t o be t o o l a r g e , i n some p r a c t i c a l i n s t a n c e s , t o a l l o w t h e evolu- t i o n a r y development o f t k e pandemonium t o taKe p l a c e ? I s h o u l d be v e r y I n t e r e s t e d t o l e a r n i f t h e a u t h o r h a s c o n s i d e r e d t h e s e

prob lems and deve loped any s o l u t i o n s . Tne problem o f s t o c h a s t i c h i l l - cllrnbine; Is o f P r a c t i c a l i n t e r e s t o u t s i d e t h e c o n t e x t o r QiIs p a p e r , f o r I n s t a n c e i n c o n n e c t i o n w i t h t n e e v o l u t i o n a r y o p e r a t i o n of c n e n i c a l p r o c e s s e s .

DR. 0. G. SELFRIDGE ( I n r e p i / ^ ) : S i n c e I have been * o r k i n on Morse code, rihich I an do ing i n a d d i t i o n t o wording on l e a r n i n g , I nave i-net p robaoly 50 p e o p l e and when I say t h a t I am working on a macnine t o do m a n u a l l y keyed Morse code, they s a y "1 w i l l t e l l yoa how t o do m a t ' ' ana t h e y then p r o c e e a t o cone up w i t n scme scheme. A c t u a l l y , t n i s is t h e f i r s t t i n e I have n e a r d t n i s p a r t i c u l a r one, whereas I nave CoTie a c r o s s m e O t h e r s many t i m e s , so D r . Bray should be c o n g r a t u l a t e d on a new scheme f o r s o l v i n g Morse code. L e t me a s s u r e him s a t i t w i l l n o t w o r ~ , we have ~ r i e d it. , i & r s e code m a n u a i i y

r e c e i v e d I s n o t t h a t s i n p l e . Tne c o n t e x t n e c e s s a r y a p p a r e n t l y e x t e n d s a t l e a s t 10 l e t t e r s on e i t h e r s i d e - n o t J d s t one s i d e , bo th s i d e s . I n f a c t , t h e - e a l cyesc ion a b o u t d o i n s th i s - N ~ J I chose t lo rse code w2s e x a c t l y L Y A C 1. had h a d t h i s k i n d o f i n t e r a c t i o n s ¥,l t h eacn o t n e r . YOL. cannot do i t by l o o k i n & a t b i n a r y f u n c t i o n s o f t h e d u r a t i o n s , t h a t i s , p x p r e s i n g durn . t ions a s b i n a r y d i g i t s and m e n l o o k i n g on b inary f u n c t i o r s of l o t s of them. I f I taKe 10 l e t t e r s on e i t n e r s i d e , each h a s t h r e e marks, vou s h o u l d c o n s i a e r t h e s p a c e s a s w e l l , b a t l e a v i n g t h o s e o u t , t n a t i s t h r e e t id ies 20, m i c h Is 60 d u r a t i o n s , and i f we s p e c i f y t h e n t o one p a r t 11- 32, t h a t I s 300 b i t s , and Dinar;, f u n c t i o n s o f 300 b i t s cannot DS p i c k e d a t rdridGm. 30 t n e q u e s t i o n is t o take some s t e p s i n t n e r i g h t d i r e c t i o n ana tnen n o r e t l e . s t e p s a r e r i g h t enoi-i&h so t h a t you w i l l g e t enough i m p r o v e ~ e n t s , s o t h a t t h e f o l l o w i n g s t e p s w i l l g e t you even r r r e so.

~ d i t e d ve rs ions by the autnor *"is not a v i l l a o l e .

I mainta in t R 2 t L:-?re I s some n e r i t In s tud>ing s e l f - L ~ r o v e n e n t systems and i f I an goin; t o do t h a t I an going to s tudy systems where I Know m a t I want, r a t h e r tnan .?ore 3 i f f i c ~ l t problems, nowever a t t r a c t i v e they may be t o natfie:naLiclans. . - la ihenat ic ians 11x2 to work on '-insolved problems for the g r e a t e r g lory , and pro 'slens a l r e a d y solved l l ~ e tne prl::!e nun'oer theorem a r e l e f t t o graduate s tuden t s . I have a susp ic ion m a t John McCarthy might l a t e r s r i n g up some Important p o i n t s about d e s c r i p t i o n s , and here I s ee my p o i n t about so lv ing u s e f u l pro'::lens, because Morse code I s a u s e f u l problen. Tne ,+nole ques t ion I s a t '.+hat l e v e l you a r e g i n : to d e a l x i th d e s c r i p t i o n s o f your data. I n most proLleli-is, e s p e c i a l l y I n d e a l i n g wi th tne amount o f tr,ln;-s which people do, b ina ry func t ions a re J u s t n o t adequate. For one tnlnr; t ,here is too xucn data , and one of the f i r s t ques t ions you t a l k about In the condl t l oned r e f l e x - remember t h a t cond i t iona l p r o b a b i l i t y assumes *- t h a t you nave a l r eady reco/ni .sed m e s t imulus ; i f J-ou know t h a t the s t imu lus celongs to one of a smal l c l a s s of funct ions , I t I s p r e t t y easy, i s a man, cut mostly i n human problems you do not know mls.

D r . Mackay nade scme very Kind remarm. I would go f u r t h e r baCK than he a i d , a s he well knows, i n c r e d i t i n g a e i n o n o l o ~ . A l l o f u s have s inned In Aaam, we have ea t en 01" m e t r e e 0 1 the knowledge of good and e v i l , and the demonological a l l e g o r i I s a very o ld one, indeed. As for n i s remarks about t he syndica te approach, tne evo lu t ion of d i v e r s a t l v e o r ~ a n l s a t i o n , I t h ink t n a t i s an extremely accu ra t e and zood po in t . One of the tn lngs a s p e c i e s l e a r n s I s n o t only how t o survive but t o have e x a c t l y the rljdit v a r i a n t s In U s memters. The reason r:orseshoe c rabs have n o t changed much is n o t Deca-ise they a o n ' t adapt; triey can adapt , but t h e r e is no v a r i a t i o n , so tney can only aaa? t ve ry l i t t l e , end tney a re a s o u t a s good a s they can be; whereas m e r e a r e a l l Xlnas of people.

I n p e h d ' sna i r - f c m a t l o n ~i-ieory, I rea l1 r - consider t h a t speed i n the c l a s s i c sense of computation I s so completely i r r e l e v a n t to t h i s problem - m e numser of b inary ope ra t ions one does a second - t h a t i t does n o t i n t e r e s t me vcirj' nuch though i t i n t e r e s t s conputer des igners . I woula r a t h e r H A G to s e e l e t s o f o p e r a t i o n s >v!hicii could conceivably be aone I n p a r a l l e l , 'oein;; done s e o u e n t i a l l y , because t h i s i s the only machine we nave, snd t 3 t 5 I nope people w i l l nave machines whlcn w i l l w0rK In CaI'2llel. so t2a.t I want a machine t o do w i c e a s d i f f i c u l t a problem I merely c u l l d m i c e as D I G a machine, i n s t e a d o f l c t t i n e , one macnine worA twice as long.

Dr. Strachey a s ~ e d a ' x - ~ t r e s u l t s , ana they a re roughly a s I have ind ica t ed . Irprovement dces take p l ace ; the n i l l - c l i m b i n g does N C ~ K . The s u t - d e m n s e l e c t i o n I n tr.e prozramnes t n a t we nave d i d not do what we hoped, s u t tne %c-clexcns \\'ere e f f e c t i v e l y thrown our, ana tile ones which .';ere kept ; e r e 1ar;"ely concent ra ted around those aerions which r e l a t e those