A Cognitive Substrate for Natural Language

Understanding

Nick CassimatisArthi Murugesan

Magdalena Bugajska

Language & Cognition

N. L. Cassimatis, J. Trafton, M. Bugajska, A. Schultz (2004). Integrating Cognition, Perception and Action through Mental Simulation in Robots. Journal of Robotics and Autonomous Systems. Volume 49, Issues 1-2, 30 November 2004, Pages 13-23.

What is difficult?

Integration of various sources of information and constraints

• Language

• Social cues (pointing)

• Visual information

• Concepts like object

• Spatial, physics

Why integration is difficult?Fodor’s Modularity of Mind

Properties of Modular systems: • Domain

specificity : certain kinds of inputs

• Informational encapsulation

• Shallow outputs

Mind’s Central Processing

Vision Motor

ReasoningLanguage

Pictures

Sounds

Physicalobjects

?

???

Problems lead to a different goal and Tailored Evaluations

Current standards in AI have become: Not sentence understanding or question answering but

- Part of speech tagging (98%)- PCFG (Probabilistic Context Free

Grammar)- Evaluation Metrics

- Precision & Recall – 90%- Exact match – 20- 40%

HPSG semantics oriented – 70%

Our approach – Substrate

Non Modular

Focus of Attention(buffer)

Identity

Difference Temporal Constraint

IdentityHypothesis

World

EventSpace

Temporal Perception

Category

ConflictResolution

Substrate:•Representation•Procedural•Multiple processes

Language is a part of and interacts freely with the greater cognitive system

N.L. Cassimatis (2006).  A Cognitive Substrate for Human-Level Intelligence.  AI Magazine. Volume 27 Number 2.

Substrate Mappings:

The particular substrate : • Physical reasoning :

– N. L. Cassimatis (2002). Polyscheme: A Cognitive Architecture for Integrating Multiple Representation and Inference Schemes. Doctoral Dissertation, Media Laboratory, Massachusetts Institute of Technology, Cambridge , MA

• Word Learning : – M. Bugajska, N.L. Cassimatis (2006). Beyond Association: Social Cognition in

Word Learning.  In Proceedings of the International Conference on Development and Learning.

• Social Cognition: – P. Bello, N.L. Cassimatis (2006). Developmental Accounts of Theory-of-Mind

Acquisition:  Achieving Clarity via Computational Cognitive Modeling.  In Proceedings of 28th Annual Conference of the Cognitive Science Society.

– P. Bello & N.L. Cassimatis (2006).  Understanding other Minds: A Cognitive Modeling Approach.  In Proceedings of the International Conference on Cognitive Modeling.

Language to Substrate Mapping

HPSG Mapping:

A. Murugesan, N.L. Cassimatis (2006). A Model of Syntactic Parsing Based on Domain-General Cognitive Mechanisms. In Proceedings of 28th Annual Conference of the Cognitive Science Society.

Example of Syntax Semantics interaction

Semantics & Syntax Interaction

E. g. :

Given a sentence with an ambiguous word – choose the correct interpretation of the word;

“The bug needs a battery”

bug

animalinsect System

error

listeningdevice

annoy(verb)Eavesdrop

(verb)

Implementation(default)

Rules:1. By default the most probable [bug - animal insect] is

chosen- e.g. of such a sentence : “The bug crawled”

Phonology ?phrase ‘bug’ ~~> Lexicon ?phrase animalBug

Abnormality predicates are used to prioritize interpretations Phonology ?phrase ‘bug’ + Blocked ?phrase animalBug ~~> Lexicon ?

phrase systemBug Phonology ?phrase ‘bug’ + Blocked ?phrase systemBug ~~> Lexicon ?

phrase listeningDeviceBug Phonology ?phrase ‘bug’ + Blocked ?phrase listeningDeviceBug ~~>

Lexicon ?phrase annoyBug Phonology ?phrase ‘bug’ + Blocked ?phrase annoyBug ~~> Lexicon ?

phrase eavesdropBug

Blocked ?phrase ?prevLexicon = = > NOT Lexicon ?phrase ?prevLexicon

likely!

Implementation(Semantics)

Two implicit requirements here :

1. generate semantics of a sentence

2. availability of background information

Walk through the example :

“The bug needs a battery”

Background knowledge

Entity

Abstract Physical

Organic Inorganic

ISA(?obj, Inorganic) = = > ISA(?obj, Physical)ISA(?obj, Organic) = = > ISA(?obj, Physical)

ISA(?obj, Inorgainc ) = = > NOT ISA (?obj, Organic)

ISA(?obj, Orgainc ) = = > NOT ISA (?obj, Inorganic)

Category hierarchy

Need ( ?object, ?neededObj) + ISA(?neededObj,battery) = = >ISA(?object, Inorganic)

Conflict in Semantics

Lexicon(?phrase,animalBug) + Referent(?phrase, ?phraseRef) = = > ISA(?phraseRef,Organic)

According to default rule Lexicon(?phrase,animalBug) is likely true (l,?)

Therefore by the above rule ISA(?phraseRef,Organic) is also likely true

However once the sentence is formed and Needs(?phraseRef, ?batteryObj) is asserted ; according to background knowledge

?phraseRef must be Inorganic and NOT Organic!

i.e. ISA(?phraseRef,Organic) is Certainly false (?,C)

animalBug animalBug-1

(l,C) conflict

Contribution

A framework for integration

• Implausibility of non modular approach is reduced

• Learnability of language

• Seamless integration