http:// 1 of 25 primatech: models and future directions gvu brownbag 9/30/1999 by jim davies, tim...
Post on 29-Dec-2015
218 Views
Preview:
TRANSCRIPT
http://www.cc.gatech.edu/gvu/perception/projects/primatech/ 1 of 25
Primatech: Models and Future Directions
GVU Brownbag 9/30/1999
by Jim Davies, Tim Keenan,
Wasinee Rungsarityotin &
Irfan Essa{jimmyd, tk, wasinee, irfan}
@cc.gatech.edu
http://www.cc.gatech.edu/gvu/perception/projects/primatech/ 2 of 25
What is Primatech?
• Chantek: An orangutan who can sign
• Will be on display at Zoo Atlanta; visitors can interact with him
• He will be overloaded. Primatech simulates having a conversation with him
http://www.cc.gatech.edu/gvu/perception/projects/primatech/ 4 of 25
Perception module
• 150 Signs to recognize (Starner et al. recognized 40)
• Controlled environment (any background, etc.)
• Novice users (few signs used)
• Supporting speech data
http://www.cc.gatech.edu/gvu/perception/projects/primatech/ 5 of 25
Graphics module
• Memory of sign animations
• Home positions?
• “Essence” with on the fly transformations
• Remember every transition? (22500)
http://www.cc.gatech.edu/gvu/perception/projects/primatech/ 8 of 25
An animated sign for “like”
QuickTime™ and aVideo decompressor
are needed to see this picture.
http://www.cc.gatech.edu/gvu/perception/projects/primatech/ 9 of 25
Ape language research
• Language trained apes know 140 signs, deaf children of 3 know 132
• Chantek– Dr. Lyn Miles– 150 signs (strict criterion)– Anthropological approach
http://www.cc.gatech.edu/gvu/perception/projects/primatech/ 10 of 25
Three cognitive models
1: Simple activation with Wordnet
2: Express words that correspond to desire satisfaction
3: Sentence verification
http://www.cc.gatech.edu/gvu/perception/projects/primatech/ 11 of 25
Model 1: Simple activation with Wordnet
• ACT-R uses spreading activation
• Relevance links only (from WordNet)
• distinguish words, signs, concepts
• Retrieve anything, let spreading activation from goal do the work
Model 1: Representation• Instead of:
(banana ISA fruit color yellow)
(yellow ISA color)
• I used:
(cup-concept ISA concept word cup-word sign cup-sign r1 drink-concept)
(cup-sign ISA sign meaning cup-concept)
(cup-word ISA sign meaning cup-concept)
What do people ask? (Collected by Tara Stoinski)
1. What do you like to eat? (5 people asked)
2. What do you like to do? (3)
3. Are you bored? (3)
4. What is it like to have fur? (1)
5. Do you like the animals that you live with? (1)
6. What is it like to live in a cage? (1)
7. Do you mind people staring at you all the time? (1)
8. Do you have a favorite keeper? (1)
9. Are you happy? (7)
10. What do you want that you don't have? (4)
11. Do you want to go back to the wild? (2)
12. Tell me about evolution (1)
http://www.cc.gatech.edu/gvu/perception/projects/primatech/ 14 of 25
Model 1: Results
1. What do you like to eat?
response: food-eat-sign
3. Are you bored?
response: you-sign
5. Do you like the animals that you live with?
response: you-sign
9. Are you happy?
response: you-sign
11. Do you want to go back to the wild?
response: go-sign
http://www.cc.gatech.edu/gvu/perception/projects/primatech/ 15 of 25
Why so bad?
• definitions too dictionary-like– banana n 1: any of several tropical and subtropical treelike herbs of the genus Musa
having a terminal crown of large entire leaves and usually bearing hanging clusters of elongated fruits
– What about being yellow? Eating it? Deliciousness?
• Only 20% of words linked• “You” “eat” and “go” were the only words in
questions that were in memory.
http://www.cc.gatech.edu/gvu/perception/projects/primatech/ 16 of 25
Model 2: Express words that correspond to desire satisfaction
• ACT-R has a goal structure but no way to express desires that are always around but no specifically acted upon all the time.
• Desires: food, drink, sex, freedom, and stimulation-- represented as memory chunks
• Words were linked to desires
http://www.cc.gatech.edu/gvu/perception/projects/primatech/ 17 of 25
Model 2: Relationships
(screwdriver-word-meaning ISA relation2 relation meaning-relation first screwdriver-word second screwdriver-concept)
(dog-isa-animal ISA relation2 relation subclass-relation first dog-concept second animal-concept)
Model 2: Input Representation (goal-1-1 ISA goal-talk sentence-id 1 word what-word next-word goal-1-2)
(goal-1-2 ISA goal-talk sentence-id 1 word do-word next-word goal-1-3)
(goal-1-3 ISA goal-talk sentence-id 1 word you-word next-word goal-1-4)
(goal-1-4 ISA goal-talk sentence-id 1 word like-word next-word unknown)
http://www.cc.gatech.edu/gvu/perception/projects/primatech/ 19 of 25
Model 2: Results
• Often responds appropriately when offered something desired
• Linguistically still impoverished
http://www.cc.gatech.edu/gvu/perception/projects/primatech/ 20 of 25
Model 3: Sentence verification
• Implemented a simple grammar• Part of speech relation added• Walked through input sentence, putting words
in sentence template• Checks the memory for a fact that has the
same subject, verb, and object• If found, express verb
http://www.cc.gatech.edu/gvu/perception/projects/primatech/ 21 of 25
Model 3: Results
• Could verify relations found in the memory (Apple is a fruit, Chantek likes meat, Banana is yellow, etc.)
• Combination of Models 2 and 3 might account for much of his conversation
http://www.cc.gatech.edu/gvu/perception/projects/primatech/ 22 of 25
Cognitive Model contributions
• Input sentence representation
• Desire representation in ACT-R
• New relation representation/ relations can become activated themselves
• First computer modeling of ape language
http://www.cc.gatech.edu/gvu/perception/projects/primatech/ 23 of 25
Future directions
• Perception: Apply machine learning to signs, installation design
• Graphics: Collision detection, closer modeling of the ape’s actual motion
• Cognitive Model: Modeling transcript data, integration of models 2 and 3
http://www.cc.gatech.edu/gvu/perception/projects/primatech/ 24 of 25
Thanks
• Graphics, Visualization & Usability seed grant 98-99
• Tara Stoinski for question collection
• Victor Zordan for graphics help
• Lyn Miles for Chantek
top related