symbol-anchoring in cassie
DESCRIPTION
Symbol-Anchoring in Cassie. Stuart C. Shapiro and Haythem O. Ismail Department of Computer Science and Engineering and Center for Cognitive Science University at Buffalo { shapiro | hismail}@cse.buffalo.edu. Outline. Introduction Perceivable entities and properties Attentional Structures - PowerPoint PPT PresentationTRANSCRIPT
S.C. Shapiro
cse@buff
alo
Symbol-Anchoring in Cassie
Stuart C. Shapiro and Haythem O. Ismail
Department of Computer Science and Engineering
and Center for Cognitive Science
University at Buffalo
{shapiro | hismail}@cse.buffalo.edu
S.C. Shapiro
cse@buff
alo
Outline
• Introduction• Perceivable entities and properties• Attentional Structures• Actions• Time• Language• Examples• Summary
S.C. Shapiro
cse@buff
alo
Embodied Cassie
• A computational cognitive agent– Embodied in hardware– or Software-Simulated– Based on SNePS and GLAIR.
S.C. Shapiro
cse@buff
alo
Cassie, the FEVAHR
S.C. Shapiro
cse@buff
alo
FEVAHR/Cassie in the Lab
S.C. Shapiro
cse@buff
alo
SNePS• Knowledge Representation and Reasoning
– Intensional Representation
– Propositions as Terms
• SNIP: SNePS Inference Package– Specialized connectives and quantifiers
• SNeBR: SNePS Belief Revision• SNeRE: SNePS Rational Engine• Interface Languages
– SNePSUL: Lisp-Like
– SNePSLOG: Logic-Like
– GATN for Fragments of English.
S.C. Shapiro
cse@buff
alo
Interaction with CassieEnglish
(Statement, Question, Command)
(Current) Set of Beliefs[SNePS]
(Updated) Setof Beliefs[SNePS]
Actions[SNeRE]
(New Belief)[SNePS]
English sentence expressingnew belief answering question reporting actions
Answer[SNIP]
GATN Parser
GATN Generator
ReasoningClarification DialogueLooking in World
Reasoning
S.C. Shapiro
cse@buff
alo
Entities, Terms, Symbols, Objects
• Cassie’s mental entity: a person named Stu
• SNePS term: B4
• Object in world:
S.C. Shapiro
cse@buff
alo
GLAIR Architecture
Knowledge Level
Perceptuo-Motor Level
Sensory-Actuator Level NL
Vision
Sonar
MotionProprioception
Grounded Layered Architecture with Integrated Reasoning
SNePS
S.C. Shapiro
cse@buff
alo
Alignment
KL
Body (PML/SAL)
World
SNePS term
PML structure
Object/Phenomenon Action
S.C. Shapiro
cse@buff
alo
Outline
• Introduction
• Perceivable entities and properties
• Attentional Structures
• Actions
• Time
• Language
• Examples
• Summary
S.C. Shapiro
cse@buff
alo
World Objectsto Feature Vectors
<Height, Width, Texture, . . . >
WorldPML/SAL
S.C. Shapiro
cse@buff
alo
Feature Vectorsto KL Terms
<Height, Width, Texture, . . . >
PML/SALKL
ProperName(B4, Stu)
Alignment
S.C. Shapiro
cse@buff
alo
Incomplete Feature Vectors
<Height, nil, nil, . . . >
PML/SALKL
Height(B4, B12)
S.C. Shapiro
cse@buff
alo
Unifying PML-Descriptions
<x, nil, nil, ... >
PML/SALKL
B20
B30
B31
B6
<nil, y, nil, ... >
<nil, nil, z, ... >
<x, y, z, ... >
Isa
Prop
Prop
S.C. Shapiro
cse@buff
alo
Outline
• Introduction• Perceivable entities and properties• Attentional Structures• Actions• Time• Language• Examples• Summary
S.C. Shapiro
cse@buff
alo
Deictic & Modality Registers
IYouNow
Vision...
PMLKLTerms denoting
CassieAddressee
Current timeCurrent state(s) of looking at x
.
..
S.C. Shapiro
cse@buff
alo
Outline
• Introduction• Perceivable entities and properties• Attentional Structures• Actions• Time• Language• Examples• Summary
S.C. Shapiro
cse@buff
alo
Primitive Actions Alignedwith PML/SAL Functions
<... >
PMLKL
Find(B4)
Find(B4)
SAL
Modality Registers
World
S.C. Shapiro
cse@buff
alo
Outline
• Introduction• Perceivable entities and properties• Attentional Structures• Actions• Time• Language• Examples• Summary
S.C. Shapiro
cse@buff
alo
Aligning NOW using MTF
NOW
Semi-lattice of times, all of which contain *NOW,any of which could be meant by “now”Finite---only conceptualized times of conceptualized states
Maximal Temporal Frame based on *NOW
KL
PML
S.C. Shapiro
cse@buff
alo
Moving NOW with MTF
NOW
KL
PML
Moves when Cassie acts, newly observes a state,or is informed of a new state. Always includes times of states in modality registers.
S.C. Shapiro
cse@buff
alo
Providing a Feel for Time
NOW COUNT n
hom
0
KL
PML
t1 t2q
!before after
time duration!
S.C. Shapiro
cse@buff
alo
Outline
• Introduction• Perceivable entities and properties• Attentional Structures• Actions• Time• Language• Examples• Summary
S.C. Shapiro
cse@buff
alo
Aligning Lexemes/NL
“Stu”
PML/SALKL
ProperName(B4, Stu)
Isa(B4, person)
“person”
ctgynpr
ctgyn
numsing
Grammar NL
World
S.C. Shapiro
cse@buff
alo
Outline
• Introduction• Perceivable entities and properties• Attentional Structures• Actions• Time• Language• Examples• Summary
S.C. Shapiro
cse@buff
alo
Acting 1
S.C. Shapiro
cse@buff
alo
Acting 2
I found a red robot.I am looking at a red robot.
Follow a red robot.
S.C. Shapiro
cse@buff
alo
Acting 3
I went to a red robot.I am near a red robot.I am following a red robot.
I found a red robot.I am looking at a red robot.
Follow a red robot.
S.C. Shapiro
cse@buff
alo
Use of Indexicals 1
Come here.I found you.I am looking at you.
S.C. Shapiro
cse@buff
alo
Come here.
I came to you, Stu.I am near you.
Use of Indexicals 2
I found you.I am looking at you.
S.C. Shapiro
cse@buff
alo
Who am I?Your name is ‘Stu’and you are a person.
Who have you talked to?
I am talking to you.Talk to Bill.
I am talking to you, Bill.Come here.
Use of Indexicals 3
S.C. Shapiro
cse@buff
alo
Come here.
I found you.I am looking at you.
Use of Indexicals 4
S.C. Shapiro
cse@buff
alo
Come here.
I came to you.I am near you.
I found you.I am looking at you.
Use of Indexicals 5
S.C. Shapiro
cse@buff
alo
Who am I?
I talked to Stuand I am talking to you.
Your name is ‘Bill’and you are a person.
Who are you?I am the FEVAHRand my name is ‘Cassie’.
Who have you talked to?
Use of Indexicals 6
S.C. Shapiro
cse@buff
alo
Outline
• Introduction• Perceivable entities and properties• Attentional Structures• Actions• Time• Language• Examples• Summary
S.C. Shapiro
cse@buff
alo
Summary• KL terms denote mental entities.
• KL terms aligned to PML structures.
• PML variables/registers contain KL terms.
• PML variables/registers/structures grounded in world via sensors & effectors.
• PML attentional structures anchor symbols by changing only when motivated.
S.C. Shapiro
cse@buff
alo
For More Information
• Personnel
• Manual
• Tutorial
• Bibliography
• ftp’able SNePS source code
• etc.
• http://www.cse.buffalo.edu/sneps/