anaphoric dependencies : a window into the architecture of the language system eye tracking...
TRANSCRIPT
Anaphoric dependencies: A window into the architecture of the language system Eye tracking experiments
Eric ReulandEric ReulandFrank WijnenFrank Wijnen
Arnout KoornneefArnout Koornneef
OVERVIEW OF LECTURE
• Part 1: discussion of method
- eye-tracking while reading
• Part 2: discussion of current research
- eye-tracking experiments
THE EYE-TRACKER
• monitors eye-movements from millisecond to millisecond
• provides information about where people look and for how long
TWO MAIN EYE-TRACKING PARADIGMS
• eye-tracking while reading
• eye-tracking while listening
(not discussed)
SOME FACTS ABOUT READING
• people do not read a text “smoothly”, but fixate a particular word (200 - 300 msec) and jump to the next
• a jump (or saccade) covers 7-9 letter spaces• during a saccade visual input is reduced• readers skip short words and words that are highly
predictable (these words are identified in the parafoveal region)
• readers regress (look back)• readers often undershoot on return sweeps (going from
the end of a line to the next line)• the perceptual span is asymmetrical to the right (to the
left for languages like Hebrew)
SOME FACTS ABOUT READING
• people do not read a text “smoothly”, but fixate a particular word (200 - 300 msec) and jump to the next
• a jump (or saccade) covers 7-9 letter spaces• during a saccade visual input is reduced• readers skip short words and words that are highly predictable
(these words are identified in the parafoveal region)• readers regress (look back)• readers often undershoot on return sweeps (going from the end of a
line to the next line)• the perceptual span is asymmetrical to the right (to the left for
languages like Hebrew)
SOME FACTS ABOUT READING
• people do not read a text “smoothly”, but fixate a particular word (200 - 300 msec) and jump to the next
• a jump (or saccade) covers 7-9 letter spaces• during a saccade visual input is reduced• readers skip short words and words that are highly predictable
(these words are identified in the parafoveal region)• readers regress (look back)• readers often undershoot on return sweeps (going from the end of a
line to the next line)• the perceptual span is asymmetrical to the right (to the left for
languages like Hebrew)
SOME FACTS ABOUT READING
• people do not read a text “smoothly”, but fixate a particular word (200 - 300 msec) and jump to the next
• a jump (or saccade) covers 7-9 letter spaces• during a saccade visual input is reduced• readers skip short words and words that are highly predictable
(these words are identified in the parafoveal region)• readers regress (look back)• readers often undershoot on return sweeps (going from the end of a
line to the next line)• the perceptual span is asymmetrical to the right (to the left for
languages like Hebrew)
SOME FACTS ABOUT READING
• people do not read a text “smoothly”, but fixate a particular word (200 - 300 msec) and jump to the next
• a jump (or saccade) covers 7-9 letter spaces• during a saccade visual input is reduced• readers skip short words and words that are highly predictable
(these words are identified in the parafoveal region)• readers regress (look back)• readers often undershoot on return sweeps (going from the end of a
line to the next line)• the perceptual span is asymmetrical to the right (to the left for
languages like Hebrew)
SOME FACTS ABOUT READING
• people do not read a text “smoothly”, but fixate a particular word (200 - 300 msec) and jump to the next
• a jump (or saccade) covers 7-9 letter spaces• during a saccade visual input is reduced• readers skip short words and words that are highly predictable
(these words are identified in the parafoveal region)• readers regress (look back)• readers often undershoot on return sweeps (going from the end of a
line to the next line)• the perceptual span is asymmetrical to the right (to the left for
languages like Hebrew)
SOME FACTS ABOUT READING
• people do not read a text “smoothly”, but fixate a particular word (200 - 300 msec) and jump to the next
• a jump (or saccade) covers 7-9 letter spaces• during a saccade visual input is reduced• readers skip short words and words that are highly predictable
(these words are identified in the parafoveal region)• readers regress (look back)• readers often undershoot on return sweeps (going from the end of a
line to the next line)• the perceptual span is asymmetrical to the right (to the left for
languages like Hebrew)
SOME FACTS ABOUT READING
• people do not read a text “smoothly”, but fixate a particular word (200 - 300 msec) and jump to the next
• a jump (or saccade) covers 7-9 letter spaces• during a saccade visual input is reduced• readers skip short words and words that are highly predictable
(these words are identified in the parafoveal region)• readers regress (look back)• readers often undershoot on return sweeps (going from the end of a
line to the next line)• the perceptual span is asymmetrical to the right (to the left for
languages like Hebrew)
A TYPICAL READING EXPERIMENT
Garden-path sentence
Since Jay always jogs a mile seems like a short distance to him.
Control sentence
Since Jay always jogs a mile this seems like a short distance to him.
READING PATTERN (Garden-path sentence)
Since Jay always jogs a mile seems
like a short distance to him.
READING PATTERN (Garden-path sentence)
Since Jay always jogs a mile
like a short distance to him.
= fixation after progressive saccade (first-pass)
= fixation after regressive saccade
= fixation after progressive saccade (second-pass)
seems
READING PATTERN
• If readers experience some sort of trouble they may fixate the difficult region longer and the may even regress to earlier parts of the sentence/text.
DIFFERENT MEASURES
• First fixation duration: duration of first fixation in a region
• First-pass duration: time spent in a region before moving on or looking back
• Regression path duration: time from first entering a region until moving the eyes beyond that region, includes regression time
• Second-pass duration: duration of re-fixations• Total duration: the sum of all fixations in a region• Probability of a regression: the percentage of
regressive eye-movements out of a region
DIFFERENT MEASURES
• First fixation duration: duration of first fixation in a region
• First-pass duration: time spent in a region before moving on or looking back
• Regression path duration: time from first entering a region until moving the eyes beyond that region, includes regression time
• Second-pass duration: duration of re-fixations• Total duration: the sum of all fixations in a region• Probability of a regression: the percentage of
regressive eye-movements out of a region
DIFFERENT MEASURES
• First fixation duration: duration of first fixation in a region
• First-pass duration: time spent in a region before moving on or looking back
• Regression path duration: time from first entering a region until moving the eyes beyond that region, includes regression time
• Second-pass duration: duration of re-fixations• Total duration: the sum of all fixations in a region• Probability of a regression: the percentage of
regressive eye-movements out of a region
DIFFERENT MEASURES
• First fixation duration: duration of first fixation in a region
• First-pass duration: time spent in a region before moving on or looking back
• Regression path duration: time from first entering a region until moving the eyes beyond that region, includes regression time
• Second-pass duration: duration of re-fixations• Total duration: the sum of all fixations in a region• Probability of a regression: the percentage of
regressive eye-movements out of a region
DIFFERENT MEASURES
• First fixation duration: duration of first fixation in a region
• First-pass duration: time spent in a region before moving on or looking back
• Regression path duration: time from first entering a region until moving the eyes beyond that region, includes regression time
• Second-pass duration: duration of re-fixations• Total duration: the sum of all fixations in a region• Probability of a regression: the percentage of
regressive eye-movements out of a region
DIFFERENT MEASURES
• First fixation duration: duration of first fixation in a region
• First-pass duration: time spent in a region before moving on or looking back
• Regression path duration: time from first entering a region until moving the eyes beyond that region, includes regression time
• Second-pass duration: duration of re-fixations• Total duration: the sum of all fixations in a region• Probability of a regression: the percentage of
regressive eye-movements out of a region
DIFFERENT MEASURES
• First fixation duration: duration of first fixation in a region
• First-pass duration: time spent in a region before moving on or looking back
• Regression path duration: time from first entering a region until moving the eyes beyond that region, includes regression time
• Second-pass duration: duration of re-fixations• Total duration: the sum of all fixations in a region• Probability of a regression: the percentage of
regressive eye-movements out of a region
EXPLANATION OF DIFFERENT MEASURES
Bart annoyed Homer because…1 2 3 45 7
Reading Times for word 3 (Homer)
First fixation duration = 3
First-pass duration = 3 + 4
Regression Path duration = 3 + 4 + 5
Second-pass duration = 6
Total duration = 3 + 4 + 6
6
DIFFERENT FIRST-PASS MEASURES
FIRST FIXATION DURATION
FIRST-PASS DURATION
REGRESSION PATH DURATION
TIME
EYE-MIND ASSUMPTION(JUST & CARPENTER, 1980)
• Readers retain fixation on a word until processing is completed
• This includes processes like word recognition, syntactic parsing, semantic integration, referential integration
AN IDEAL WORLD(VAN BERKUM, 2004)
Snowwhite kissed a dwarf
W P S R
W P S R
W P S R
W P S R
TIME
W = word recognition; P = parsing; S = semantic integration; R = referential integration
THE REAL WORLDIS A REAL MESS
(VAN BERKUM, 2004)
Snowwhite kissed a dwarf
W P S R
W P S R
W P S R
W P S R
TIME
W = word recognition; P = parsing; S = semantic integration; R = referential integration
THE REAL WORLDIS A REAL MESS
• In the real world the processing of word X continues while fixating word X + 1 (and possibly while fixating word X + 2 etc.)
• Thus, the effects of a manipulation are often visible more downstream (i.e. after the critical word or region). This is called spill-over.
THE REAL WORLDIS A REAL MESS
• In the real world the processing of word X continues while fixating word X + 1 (and possibly while fixating word X + 2 etc.)
• Thus, the effects of a manipulation are often visible more downstream (i.e. after the critical word or region). This is called spill-over.
LINKING ASSUMPTION(BOLAND, 2004)
• The eyes do not leave a word until it has been structurally integrated (tree building). Therefore, constraints that control structure-building influence first-pass reading time.
other measures (e.g., regression path duration) are sensitive for higher level processes
(semantic integration, discourse processes)
SOLUTION LINKING PROBLEM?
• Perhaps the different measures can provide information about what is happening? (this is an empirical question)
A DISADVANTAGE OF THE READING PARADIGM
• You can only use skilled readers (no children or language-disordered populations).
• This is possible in spoken language paradigms.
CAVEAT:
• the eyes tell us that something is happening at a specific point in time, but not what that something is!
TO CONCLUDE
• Eye-tracking (while reading and listening) excels in the “when” question.
• Not really suited for “what” question (use EEG/MEG instead).
BASIC ARCHITECTURE PROCESSING SYSTEM
• distinct modules syntax, semantics, discourse
• economy principle: cross-modular operations carry a cost (Reuland 2001)
• syntax first in time-course-model of processing phases (Friederici 1995-)
syntax is cheaper than semantics semantics is cheaper than discourse
CROSS-MODULAR OPERATIONS
Discourse storage (values) a a
C-I objects (variables) x1 x2
Syntactic objects (chains) C1 C2
Basic expressions α β
Discourse storage (values) a
C-I objects (variables) x1 x1
Syntactic objects (chains) C1 C2
Basic expressions α β
Discourse storage (values) a
C-I objects (variables) x1
Syntactic objects (chains) C1 C1
Basic expressions α β
SEMANTICS VS. DISCOURSE
• variable binding (semantic dependency)
Every clown thinks that he is not funny.
• co-reference (discourse dependency)
The clown has a big problem.
He is not funny.
PREDICTIONS OF THEORY
• variable binding is cheaper/faster than co-reference
• in an ambiguous situation variable binding has precedence over co-reference