cse@buffalo s.c. shapiro development of a cognitive agent stuart c. shapiro department of computer...
Post on 19-Dec-2015
217 views
TRANSCRIPT
S.C. Shapiro
cse@buff
alo
Development ofa Cognitive Agent
Stuart C. Shapiro
Department of Computer Science and Engineering
and Center for Cognitive Science
State University of New York at Buffalo
S.C. Shapiro
cse@buff
alo
Outline• Introduction• Intensional Representation• Logic for NLU and Commonsense Reasoning• Discussing Propositions and Sentences• Symbol-Grounding by Perception and Action • Representation and Use of Indexicals• A Personal Sense of Time• Summary
S.C. Shapiro
cse@buff
alo
Goal
• A computational cognitive agent that can:– Understand and communicate in English; – Discuss specific, generic, and “rule-like” information;– Reason;– Discuss acts and plans;– Sense;– Act;– Remember and report what it has sensed and done.
S.C. Shapiro
cse@buff
alo
Embodied Cassie
• A computational cognitive agent– Embodied in hardware– or Software-Simulated– Based on SNePS and GLAIR.
S.C. Shapiro
cse@buff
alo
SNePS• Knowledge Representation and Reasoning
– Propositions as Terms
• SNIP: SNePS Inference Package– Bi-Directional Inference
• SNeBR: SNePS Belief Revision
• SNeRE: SNePS Rational Engine
• Interface Languages– SNePSUL: Lisp-Like– SNePSLOG: Logic-Like– GATN for Fragments of English.
S.C. Shapiro
cse@buff
alo
GLAIR Architecture
Knowledge Level
Perceptuo-Motor Level
Sensory-Actuator Level
NL
Vision
Sonar
Motion
Proprioception
Grounded Layered Architecture with Integrated Reasoning
SNePS
S.C. Shapiro
cse@buff
alo
Interaction with CassieEnglish
(Statement, Question, Command)
(Current) Set of Beliefs[SNePS]
(Updated) Setof Beliefs[SNePS]
Actions[SNeRE]
(New Belief)[SNePS]
English sentence expressingnew belief answering question reporting actions
Answer[SNIP]
GATN Parser
GATN Generator
ReasoningClarification DialogueLooking in World
Reasoning
S.C. Shapiro
cse@buff
alo
Outline• Introduction• Intensional Representation• Logic for NLU and Commonsense Reasoning• Discussing Propositions and Sentences• Symbol-Grounding by Perception and Action • Representation and Use of Indexicals• A Personal Sense of Time• Summary
S.C. Shapiro
cse@buff
alo
Entities, Terms, Symbols, Objects
• Cassie’s mental entity: a person named Bill
• SNePS term: B5• Object in world:
S.C. Shapiro
cse@buff
alo
Intensional Representation
Intensional entities are distincteven if coreferential.
“The morning star is the evening star.”
“George IV wondered if Scott was the author of Waverly.”
S.C. Shapiro
cse@buff
alo
McCarthy’s Telephone Number Problem
Mary's telephone number is Mike's telephone number.
I understand that Mike's telephone number is Mary's telephone number.
Pat knew Mike's telephone number.
I understand that Pat knew Mike's telephone number.
Pat dialed Mike's telephone number.
I understand that Pat dialed Mike's telephone number.
S.C. Shapiro
cse@buff
alo
Answering the Telephone Number Problem
Did Pat dial Mary's telephone number?
Yes, Pat dialed Mary's telephone number.
Did Pat know Mary's telephone number?
I don't know.
S.C. Shapiro
cse@buff
alo
Outline• Introduction• Intensional Representation• Logic for NLU and Commonsense Reasoning• Discussing Propositions and Sentences• Symbol-Grounding by Perception and Action • Representation and Use of Indexicals• A Personal Sense of Time• Summary
S.C. Shapiro
cse@buff
alo
Logic for NLU &Commonsense Reasoning
Either Pat is a man or Pat is a woman or Pat is a robot.
I understand that Pat is a robot or Pat is a woman or Pat is a man.
Pat is a woman.
I understand that Pat is a woman.
What is Pat?
Pat is a woman and Pat is not a robotand Pat is not a man.
S.C. Shapiro
cse@buff
alo
Representation in FOPL?
Man(Pat) Woman(Pat) Robot(Pat)
but don’t want inclusive or
Man(Pat) Woman(Pat) Robot(Pat)+ +
T T T
F
TSo don’t want exclusive or either
S.C. Shapiro
cse@buff
alo
andor
andor(i, j){Pi, ..., Pn}
True iff at least i, and at most j of the Pi are True
S.C. Shapiro
cse@buff
alo
Outline• Introduction• Intensional Representation• Logic for NLU and Commonsense Reasoning• Discussing Propositions and Sentences• Symbol-Grounding by Perception and Action • Representation and Use of Indexicals• A Personal Sense of Time• Summary
S.C. Shapiro
cse@buff
alo
Discussing Propositions
That Bill is sweet is Mary's favorite proposition.
I understand that Mary's favorite proposition is that Bill is sweet.
Mike believes Mary's favorite proposition.
I understand that Mike believes that Bill is sweet.
S.C. Shapiro
cse@buff
alo
Discussing SentencesHamlet said `To be or not to be'.I understand that Hamlet said `To be or not To be'.
Polonius said `Neither a borrower nor a lender be'.I understand that Polonius said `Neither a
borrower nor a lender be'.
Who said `To be or not to be'?Hamlet said `To be or not To be'.
S.C. Shapiro
cse@buff
alo
Representing Propositionsand Sentences
Propositions and sentences must be
First-class entities of the domain
Represented by terms
S.C. Shapiro
cse@buff
alo
Outline• Introduction• Intensional Representation• Logic for NLU and Commonsense Reasoning• Discussing Propositions and Sentences• Symbol-Grounding by Perception and Action • Representation and Use of Indexicals• A Personal Sense of Time• Summary
S.C. Shapiro
cse@buff
alo
Symbol Grounding byPerception and Action
• Agent can demonstrate understanding
– Of properties and descriptions• By locating correct objects in the world
– Of verbs• By acting properly in the world
S.C. Shapiro
cse@buff
alo
Go to the green robot.
I went to the green robot.
I am near the green robot.
Symbol-Grounding 2
S.C. Shapiro
cse@buff
alo
Alignment
• Terms at GLAIR KR level
• Aligned with
• Descriptions and procedures
• At GLAIR PM level
S.C. Shapiro
cse@buff
alo
Outline• Introduction• Intensional Representation• Logic for NLU and Commonsense Reasoning• Discussing Propositions and Sentences• Symbol-Grounding by Perception and Action • Representation and Use of Indexicals• A Personal Sense of Time• Summary
S.C. Shapiro
cse@buff
alo
Representation and Use of Indexicals
• Words whose meanings are determined by occasion of use
• E.g. I, you, now, then, here, there
• Deictic Center <*I, *YOU, *NOW>
• *I: SNePS term representing Cassie
• *YOU: person Cassie is talking with
• *NOW: current time.
S.C. Shapiro
cse@buff
alo
Analysis of Indexicals(in input)
• First person pronouns: *YOU• Second person pronouns: *I• “here”: location of *YOU.
S.C. Shapiro
cse@buff
alo
Generation of Indexicals
• *I: First person pronouns
• *YOU: Second person pronouns
• *NOW: used to determine tense.
S.C. Shapiro
cse@buff
alo
Who am I?Your name is ‘Stu’and you are a person.
Who have you talked to?
I am talking to you.Talk to Bill.
I am talking to you, Bill.Come here.
Use of Indexicals 3
S.C. Shapiro
cse@buff
alo
Come here.
I came to you.I am near you.
I found you.I am looking at you.
Use of Indexicals 5
S.C. Shapiro
cse@buff
alo
Who am I?
I talked to Stuand I am talking to you.
Your name is ‘Bill’and you are a person.
Who are you?I am the FEVAHRand my name is ‘Cassie’.
Who have you talked to?
Use of Indexicals 6
S.C. Shapiro
cse@buff
alo
Outline• Introduction• Intensional Representation• Logic for NLU and Commonsense Reasoning• Discussing Propositions and Sentences• Symbol-Grounding by Perception and Action • Representation and Use of Indexicals• A Personal Sense of Time• Summary
S.C. Shapiro
cse@buff
alo
A Personal Sense of Time
• *NOW contains SNePS term representing current time.
• *NOW moves when Cassie acts.
S.C. Shapiro
cse@buff
alo
B6
Representation of Time
find
lex
action object
B1
!
agentact
eventtime
NOW
!!before after before after
?????????????
I
S.C. Shapiro
cse@buff
alo
Performing a Punctual Actt1 t3!before after
NOW NOW
t2!before after
!
time
event
S.C. Shapiro
cse@buff
alo
Performing a Durative Actt1
NOW
!before after t2
!
time
event
NOW
t3!
supintsubint
S.C. Shapiro
cse@buff
alo
Keeping Track of Time 1
Who are you looking at?I am looking at you, Stu.
Come here.
S.C. Shapiro
cse@buff
alo
Keeping Track of Time 2
Who are you looking at?I am looking at you.
Come here.
I came to you.I am near you.
Who are you looking at?I am looking at you.
S.C. Shapiro
cse@buff
alo
Keeping Track of Time 3
Find Bill.I found Bill.I am looking at Bill.
Who are you looking at?I looked at youand I am looking at Bill.
Who are you talking to?I am talking to you.
S.C. Shapiro
cse@buff
alo
Keeping Track of Time 4
Follow a red robot.I found a red robot.I am looking at a red robot.
S.C. Shapiro
cse@buff
alo
Keeping Track of Time 5
I went to a red robot.I am near a red robot.I am following a red robot.
Follow a red robot.I found a red robot.I am looking at a red robot.
S.C. Shapiro
cse@buff
alo
Keeping Track of Time 6
Who are you talking to?I am talking to you.
Who am I?Your name is ‘Stu’and you are a person.
Stop.I stopped.
S.C. Shapiro
cse@buff
alo
Keeping Track of Time 7
Who are you looking at?I looked at youand I looked at Billand I looked at a red robot.
Who are you following?I followed a red robot.
Who are you talking to?I am talking to you.
S.C. Shapiro
cse@buff
alo
Outline• Introduction• Intensional Representation• Logic for NLU and Commonsense Reasoning• Discussing Propositions and Sentences• Symbol-Grounding by Perception and Action • Representation and Use of Indexicals• A Personal Sense of Time• Summary
S.C. Shapiro
cse@buff
alo
Goal
• A computational cognitive agent/robot
• That can communicate in natural language.
S.C. Shapiro
cse@buff
alo
Intensional Representation
• SNePS terms represent mental entities.
• May assert that two entities are coreferential.
• Relations/acts may be declared transparent.
S.C. Shapiro
cse@buff
alo
Logic for NLU and Commonsense Reasoning
• Designed logical connectivesand rules of inference
More appropriate than in standard FOPC.
S.C. Shapiro
cse@buff
alo
Discussing Propositionsand Sentences
• Propositions and sentences are first-class entities.
S.C. Shapiro
cse@buff
alo
Symbol-Grounding by Perception and Action
• Use of GLAIR architecture to connect entities with descriptions/functions used by sensors and effectors.
S.C. Shapiro
cse@buff
alo
Representation and Useof Indexicals
• Use of Deictic Center for parser to interpret indexicals as current referents
• And for generator to generate indexicals from current referents.
S.C. Shapiro
cse@buff
alo
A Personal Sense of Time
• *NOW is current time.
• Updated when Cassie acts.