1 a new mathematical framework for language representation, association, processing, and...

1

A New Mathematical Framework for Language Representation, Association,

Processing, and Understanding

Robert Hecht-NielsenPrincipal Investigator

[email protected] Software Inc.

5935 Cornerstone CourtSan Diego, CA 92121

andUniversity of California, San Diego

AQUAINT Phase I Mid-Year Workshop Monterey, California10 – 13 June 2002

2 © 2002, Robert Hecht-Nielsen

THE BIG QUESTIONS

• How does thalamocortex represent objects?

• How does cerebral cortex learn and store information about objects?

• How does thalamocortex process information?

• In which brain nucleus does Homunculina reside?

And Now, The Answers!


How Does Cerebral Cortex Represent Objects? The cortical surface is tiled with thousands of regions (which will, for simplicity, be treated here as disjoint and non-interacting).

Each such region must (with momentary exceptions) be in one of a few thousand fixed-for-life states (the most common of which is the null state). Each state is a sparse random collection of active neurons: a standardized representational token.

Each object represented in cortex is expressed as a collection of tokens; one on each involved region. Each object has a tight pdf in cortical token space.

Cerebral cortex is thus fundamentally SYMBOLIC, just as some early AI researchers assumed.

Cerebral Cortex

Cortex is tiled with thousands of regions

Each region has a largely frozen-early-in-life lexicon of thousands of representational tokens


How Does Cerebral Cortex Represent Objects?

CorticalArea B

Cerebral Cortex

NRT

Cortical Area A

Thalamus

The lexicon of standardized tokens of region is used to describe one specific attribute of an

object(this lexicon is largely frozen early in life)

{Φ, T1, T2, . . . , TK}Neurons can randomly die and it still works!

THE BRAIN IS SYMBOLIC!

Cortical Area A

Cortical Region

Thalamic Region

Thalamic Area A

Cerebral cortex is tiled with thousands of

regions -- each forming the upper half of a

FEATURE ATTRACTORassociative memory

neural network


Mutual Significance EvaluatorAssociative Memory Neural

Network

p(j) p(i)

j)p(i, j)s(i,

s(i,j) is the mutual significance of tokens i and j on regions and . It is the ratio of the actual probability of co-occurrence of tokens i and j to the probability that these tokens would co-occur randomly.

MSE fascicles learn a monotonically increasing function of mutual significance, with low values set to zero. Thus, only a minute fraction of the token pairs of each MSE network need be connected.

How Does Cerebral Cortex Learn & Store Information?

CorticalArea A

CorticalArea B

MSE Fascicles

Region

Region



Significance is a new information theoretic quantity. Human cortex has roughly a million mutual significance evaluator networks; each linking a pair of regions. Significance values are guaranteed to converge because they are a ratio of probabilities that depend only on the information environment (internal and external). If the information environment changes (or if a particular token pair is exercised repeatedly – as in memory consolidation during sleep), the significances change. Significance learning remains active throughout life. Token pair mutual significance is the only information learned and stored in the cerebral cortex.

CorticalArea A

CorticalArea B

MSE fascicle linking Regions and

Region

Region

Connections are between neurons of one token and neurons of a paired token. However, almost all of these connections learn the same mutual significance value. Thus, as with the feature attractor network, the mutual significance evaluator network is inherently tolerant of extensive random neuron death.


Cortical Area A Cortical Area B

Cortical Area C

Hierarchical abstractor networks are built up from converging MSE networks. Random upward connections are used when new tokens need to be formed.


MSE fascicles(one for each region pair)

Higher-level region

Lower-level regions


Consensus Building: The Universal Thalamocortical Information Processing

Operation The essence of consensus building is to find the answer token(s) that have the highest consistent level of significance with all of the specified known facts in terms of all of the specified knowledge bases (mutual significance evaluator fascicles) being employed. Human cognition thus always proceeds on the basis of finding specified types of answers which, when considered using a specified body of knowledge, are uniformly highly consistent (in terms of high mutual significance) with all the assumed facts. Any inconsistency causes failure of the cognitive process. Unanswerable questions are usually resolved by further restricting the set of considered facts and/or the body of knowledge being used.

p(j) p(i)

j)p(i, j)s(i,

1

2

3

down the garden ???


CORTRONICS: A New Foundation for AIComputer Implementations of Thalamocortical Neural

Networks

Thalamus

Build Feature AttractorTrain Mutual

Significance Evaluator Insert Word & Apply MSE

Apply Feature Attractor

We have applied Cortronics to sound, vision, and English text. The examples presented here are for text. Experiments often use a diverse billion word training corpus equivalent to thousands of books (novels, encyclopedias, news, etc.). Many MSEs are trained in parallel during each training stage.

Cortex

Region Region

school bus

FA Networks

MSE Networks


driving at top speed traveling around the countrydriving: influence, expired, suspended traveling: mph, speed, directions,

unavailableat: university, time A.M., P.M. around: world, country, clock, globetop: priority, aides, aid, executives the: united, first, same,

company

man on the moon all of a suddenman: woman, murdering, moon, himself all: sudden, bear, are,

abroadon: #XXX (our universal numerical quantity) of: and, million, #XXX, world’sthe: united, first, same, company a: lot, few, share, new

all but a handful deep blue sky (three-word example)all: sudden, bear, are, abroad deep: cuts, in, the, divisionsbut: said, don’t, is, not blue: chips, jeans, chip,

shielda: lot, few, share, new

the future looks promising down the garden paththe: the, of, a, in down: unchanged, #XXX,

percent, fromfuture: Jerusalem, uncertain, the of the: of, states, is, andlooks: like, forward, pretty, at garden: grove, city, centers,

ceremony

Unites States of AmericaUnited: Canada, Britain, Japan, IndiaStates: Canada, Britain, Japan, Europeof: the, a, its, his

Consensus Building: Raw Knowledge Examples

12

3


driving at top speedtraveling around the countryman on the moonall of a suddenall but a handful1: deep blue sky2: deep blue eyes1: the future looks promising2: the future looks bright1: down the garden path2: down the garden lane

instructive mistake (a tie):1: United States of Canada1: United States of Japan1: United States of America

Consensus Building: Combining MSE Knowledge

12

3 MSE Fascicle 3: driving: influence, expired, suspended

MSE Fascicle 2: at: university, time, A.M., P.M.

MSE Fascicle 1: top: priority, aides, aid, executives


Consensus Building: Combining Disparate Knowledge

phrase-level

word-level

spoke on condition of anonymity##XXX degrees below zeroformer Prime Minister Margaret Thatcherformer President Ronald Reaganthe Dow Jones Industrial Averagethe Dow Jones Average rose, fell, climbedthe destruction of wildlife habitat##XXX on the Richter scalethe wishes of the school boardaccording to the school administrators, records, recordliving in New York, Jersey, Delhi, Orleanscomes in early afternoon, trading, September,

Springbased on new evidenceaccording to new laws, rulesearlier in the day, week, evening, sessionNew York Stock Exchange, market, index, . . .

Consensus building liberates us from Markov models, Bayesian networks, N-grams, etc. Now context of arbitrary type and quantity can be brought to bear on segmentations, classifications, and predictions.


… bus drove from Chicago to …

carauto

automobiletruck

Learning Word Level Synonymy


Word Synonymy ListsJohnJackIvan

JosephGeorge

PaulPeterDaniel

Edward

MaryBetty

ElisabethSueSallyKarenJackie

MargaretLisa

DetroitChicagoSydneySeattlePhoenixManila

CincinnatiMinneapolisVancouver

FrancePortugal

ChinaSingapore

HaitiCubaSpainChileJapan

ninetytwentythirtyeighty

seehearbeget

seengot

donebeen

minutesyearsdays

pounds

phonetelephone

telecommunicationscommunications

televisionTV

radionews

collegeschool

universitygraduate

presidentpresidents

leaderchairman


Phrase Level Synonymy Lists

French FrancsGerman Marks

Italian LiraSwiss Francs

Canadian Dollars

will beshould behave been

arecan be

New YorkLos Angeles

SeattleNew Orleans

Chicago

French Francs


Question Answering Approach Sketch

Jane works for ? (A question answering sentence) Motorola employs Jane. (A database sentence)

Consensus building operations, learned from examples, can be used with a sentence representation structure like this to logically compare the meaning of one sentence with another (even if one has missing elements).

phrase-level

sentence-level

word-level


The HNC AQUAINT Project Team

Dr. Robert Means -- Chief TechnologistKate Mark – Cortronics Project CoordinatorDavid Busby - Chief Brain Software ArchitectRion Snow - ResearcherDr. Syrus Nemat-Nasser, ConsultantChristopher Downing, Consultant

1 a new mathematical framework for language representation, association, processing, and...

Documents