the role of background knowledge in sentence processing raluca budiu july 9, 2001 thesis committee:...
TRANSCRIPT
The Role of Background Knowledge in Sentence Processing
Raluca Budiu
July 9, 2001
Thesis Committee:
John Anderson, Chair
Jaime Carbonell
David Plaut
Lynne Reder, Department of Psychology
Thesis Defense, July 9, 2001 2
Ambiguity of Language
My mouse behaves erratically lately.-- From an e-mail to CS facilities
Could you pass me the salt?
That's the sun of the egg. -- Child speaking about the yolk of a fried egg
Thesis Defense, July 9, 2001 3
Language and Noise
• Communication channels are noisy
• People make mistakesWe understand how unfair the death penalty is.
-- George W. Bush, speaking of death tax
• Listeners ignore semantic inconsistenciesWhen an aircraft crashes, where should the
survivors be buried?
Thesis Defense, July 9, 2001 4
Insight from this Research
Flexibility: stretching words’
meanings
Reliability: ignoring noise &
semantic inconsistencies
Thesis Defense, July 9, 2001 5
The Sentence-Processing Model
Model
Priorknowledge
Sentence
Noah took two animals of each kind on the ark
Napoleon was defeated at Waterloo in 1815
Plato was Socrate’s student
Sentenceinterpretation
Thesis Defense, July 9, 2001 6
Main Contribution
A model of language comprehension that:• Offers a unified explanation of several complex
linguistic phenomena
• Is incremental (on line)
• Is as fast as humans
• Uses prior knowledge and sentence context to understand vague words
• Is based on the ACT-R theory (Anderson & Lebiere, 1998)
Thesis Defense, July 9, 2001 7
ACT-R
• A cognitive architecture based on production systems
• A rigorous framework for building, running and testing computational models
• Based on verified assumptions about human cognition (e.g., memory properties, attention)
• Produces quantitative predictions about human behavior (e.g., accuracy and latency in a task)
Thesis Defense, July 9, 2001 8
Research Methodology
Human subjects
Experiment
Match?
Quantitative measures Quantitative measures
ComputationalACT-R model
predictions
no
Thesis Defense, July 9, 2001 9
Evaluation of the Model
The model• Can comprehend
– Literal or metaphoric, distorted or undistorted sentences
– Isolated or in-discourse sentences
• Can explain patterns of text recall• Compares well with people on psycholinguistic
experiments• Is fast, accurate, and scalable
Thesis Defense, July 9, 2001 10
Outline
• IntroductionThe sentence-processing model
– Evaluation
• Comprehension of sentences in discourse– Evaluation
• Scalability
• Future work and conclusions
Thesis Defense, July 9, 2001 11
The Sentence-Processing Model
Model
Background
knowledge
(words + thematic roles)
Noah took two animals of each kind on the arkNapoleon was defeated at Waterloo
in 1815
Plato was Socrate’s student
SentenceinterpretationInput sentence
Thesis Defense, July 9, 2001 12
Propositional Representation
take
arkanimals
Noah
Ark Prop
agent verb
place-obliquepatie
nt
Parent Ark PropChild animalsType patient
Noah took the animals on the ark
Thesis Defense, July 9, 2001 13
Associations
Noah took the animals on the ark
Napoleon was defeated at WaterlooNoah is Lamech’s son
PatriarchNoah
Moses
Napoleon
take
Noah
& Activation
Thesis Defense, July 9, 2001 14
Noah is Lamech’s son
Noah took the animals on the ark
Noah is Lamech’s son
Noah took the animals on the ark
Noah
Searchtook the animals on the arkNoahNoah
Noah
Napoleon was defeated at Waterloo
Patriarch
Moses
Napoleon
take
Thesis Defense, July 9, 2001 15
Noah
Matchtook the animals on the arkNoahNoah
NoahNoah took the animals on the ark
Napoleon was defeated at Waterloo
Noah is Lamech’s son
Patriarch
Moses
Napoleon
take
Noah is Lamech’s sonNoah
Thesis Defense, July 9, 2001 16
Noah
Matchtook the animals on the arkNoahNoah
NoahNoah took the animals on the ark
Napoleon was defeated at Waterloo
Noah is Lamech’s son
Patriarch
Moses
Napoleon
take
Noah is Lamech’s son is
took
take
Thesis Defense, July 9, 2001 17
Noah
Searchtook the animals on the arkNoahNoah
NoahNoah took the animals on the ark
Napoleon was defeated at Waterloo
Noah is Lamech’s son
Patriarch
Moses
Napoleon
take
took
take
Noah took the animals on the arkNoah took the animals on the ark
Thesis Defense, July 9, 2001 18
Final Interpretation
Noah took the animals on the ark
Noah took the animals on the ark
Napoleon was defeated at WaterlooNoah is Lamech’s son
PatriarchNoah
Moses
Napoleon
take
Thesis Defense, July 9, 2001 19
Failures of Comprehension
burnt offerings on the altar
Lamech Prop
Noah offered
Lamech Prop
No interpretation
word offeredrole verbinterpretation Lamech Prop
Bug
Thesis Defense, July 9, 2001 20
Summary of the Model
Read word
Bug
no
Integration
en
d o
f se
nte
nce
Interpretation?
yes
Search
no
Match?
yes
Thesis Defense, July 9, 2001 21
Answering True/False Queries
• False = a bug OR no final interpretation found
• True = no bug AND final interpretation found
Thesis Defense, July 9, 2001 22
Outline
• Introduction The sentence-processing model
– Empirical evaluation• Moses illusion
• Metaphor-position effects
• Comprehension of sentences in discourse– Evaluation
• Scalability• Future work and conclusions
Thesis Defense, July 9, 2001 23
Moses Illusion
• How many animals of each kind did Moses take on the ark?
• Good vs. bad distortions
How many animals of each kind did Adam take on the ark?
Thesis Defense, July 9, 2001 24
Moses-Illusion Data
Illusion rates for good and bad distortions (Ayers, Reder & Anderson, 1996)
• Percent correct distortions in the gist task (Ayers et al., 1996)
• Reading times in the literal and gist task (Reder & Kusbit, 1991)
Thesis Defense, July 9, 2001 25
Illusion rates (Ayers et al., 1996) and results of simulation
0
10
20
30
40
50
60
Undistorted Good distortions Bad distortions
Illu
sio
n R
ate
(%
)
Humans Model
Thesis Defense, July 9, 2001 28
Simulation of Moses Illusion
take
ark
animalsNoah
Ark Prop
agent
verb
place-oblique
patient
Moses
Adam
How many animals did Moses take on the ark
Zoo Prop Zoo Prop
Ark Prop
Adam
Zoo PropNo interpretation
Bug
Thesis Defense, July 9, 2001 29
Metaphor Comprehension
Effects of position on metaphor understanding (Gerrig & Healy, 1983)
• Metaphor-familiarity effects (Budiu & Anderson, 1999)
• Understanding metaphoric/literal sentences in context (Budiu & Anderson, 2000)
Thesis Defense, July 9, 2001 30
Metaphor Position
Stars Prop
Stars Prop
Container Prop Container Prop
Stars Prop
Drops of molten silver filled the night sky.
The night sky was filled with drops of molten silver.
4.30 s4.30 s
3.68 s3.68 s
ModelModel
3.53 s3.53 s
4.21 s4.21 s
HumansHumans
Thesis Defense, July 9, 2001 31
Outline
• IntroductionThe sentence-processing model
– Evaluation
Comprehension of sentences in discourse– Evaluation
• Scalability
• Future work and conclusions
Thesis Defense, July 9, 2001 32
Sentences in DiscourseCreate background knowledge from discourse propositions
King Lear had three daughtersGoneril and Regan declare their grand love
King Lear decided to divide his kingdom
Cordelia is disinherited
…Cordelia refuses to make an insincere speech
Cordelia marries the king of France
King Lear’sstory
Thesis Defense, July 9, 2001 33
Novel Sentences Use a partially matching interpretation to relate
to discourse
Cordelia marries the king of France
Prop 5Cordelia is disinherited
No interpretationProp 5No interpretation
word marriedInterpretation Prop 5 ……
Bug
Integration
<end>
Prop 5
Thesis Defense, July 9, 2001 34
Outline
• Introduction
• The sentence-processing model– Evaluation
• Comprehension of sentences in discourseEvaluation
• Scalability
• Future work and conclusions
Thesis Defense, July 9, 2001 35
Metaphor in Discourse
Experiments Metaphor vs. LiteralReading Time
Ortony et al., 1978Inhoff et al., 1984Shinjo & Myers, 1987Keysar, 1990
Gibbs, 1990Onishi & Murphy, 1993 slower
same
Comprehension
shallow
deepAnswer true/false
Comprehension(of novel sentences)
OurExperiments
Thesis Defense, July 9, 2001 36
Metaphoric Sentences in Context
During history seminars, a massive young man always yawned and never paid any attention to the discussions. He was a very good linebacker who had been all-state in football. The seminar always came after his training sessions, so he was very tired.
The bear slept quietlyThe bear yawned in class
Read new:
The athlete slept quietlyThe athlete yawned in class
True or false:
Thesis Defense, July 9, 2001 37
Metaphoric Sentences in Context
The bear yawned in class
True or false:
Find interpretationBug
Reevaluate bug
The athlete yawned in class
Find interpretation
The athlete slept quietly <end>
Interpretation No interpretation
Bug-basedintegration
The bear slept quietly
Read new:
No interpretation
bear
Bug
Thesis Defense, July 9, 2001 38
Outline
• Introduction
• The sentence-processing model• Evaluation
• Comprehension of sentences in discourse• Evaluation
Scalability
• Future work and conclusions
Thesis Defense, July 9, 2001 39
Computational Constraints
• Speed
• AccuracyScalability
- Word databaseSentence database
Thesis Defense, July 9, 2001 40
Scalability Test
• 436 noun-verb-noun sentences (Brown corpus via PennTreebank project)
• 999 distinct words
• One word repeated in at most 9 propositions• Associations based on LSA similarity measures
(Landauer & Dumais, 1997)
• Test for comprehension of a known sentence
Thesis Defense, July 9, 2001 41
Model PerformanceExperiment Accuracy Switches/word
Metaphor position
Moses illusion — literal
— gist > 94% < 0.72
Metaphor verification
Metaphor comprehension
Text Memory
Thesis Defense, July 9, 2001 42
Summary
• A model of sentence comprehension with a strong associative mechanism to speed up the search of an interpretation
• It offers a unified explanation for a variety of empirical psycholinguistic data
• It is scalable
• It is implemented in ACT-R
Thesis Defense, July 9, 2001 43
Future Work
• Extend the model to other empirical phenomena (e.g., priming, text inference, lexical ambiguity)
• Identify the ACT-R assumptions that are fundamental
• Eliminate some of the limitations
Thesis Defense, July 9, 2001 44
Conclusions
• Context can help the comprehension of metaphoric or semantically-flawed sentences
• Semantic associations between words are a powerful mechanism that allows fast and flexible comprehension
• “Peripheral” language phenomena can shedlight on deep cognitive processes
Thesis Defense, July 9, 2001 45
Limitations of the Model• No syntactic processing• Atomic word-phrases (e.g., drops of molten silver)
• Rudimentary discourse processing• Cannot account for sentences containing similar words
(e.g., George W.Bush is the son of George Bush)
• Relationship between discourse and background knowledge
• Similarities not from ratings• No thematic-role cues
Thesis Defense, July 9, 2001 47
Metaphor in Discourse
Experiments Metaphor vs. LiteralReading Time
Ortony et al., 1978Inhoff et al., 1984Shinjo & Myers, 1987Keysar, 1990
Gibbs, 1990Onishi & Murphy, 1993 slower
same
Comprehension
shallow
deep