prosody-driven sentence processing: an event-related brain potential study ann pannekamp, ulrike...
TRANSCRIPT
Prosody-driven Sentence Processing: An Event-related
Brain Potential Study
Ann Pannekamp, Ulrike Toepel, Kai Alter, Anja Hahne and Angela D. Friederici
Presented by Laura Matzen, 9/1/2005
Pannekamp et al. (2005)
• Goal of study:– Determine what causes the closure positive
shift (CPS)
– Is this effect driven by prosody alone or by other factors?
Pannekamp et al. (2005)
• Basic Design:– Systematically reduce linguistic content of
sentences
– Record ERPs in each condition to see if CPS is present in all cases or to see how it changes
Background onEvent-related Potentials (ERPs)
Background onEvent-related Potentials (ERPs)
Background
• Closure Positive Shift (CPS)– seen in response to normal spoken sentences– somehow related to intonation contour
• Intonation contour– “sentence melody”– provides information about syntactic structure,
sentence mode– fundamental frequency – F0
“Actually, we were the ones who said we didn’t want a regular cake, so you can blame it on us.”
Frequency
“Actually, we were the ones who said we didn’t want a regular cake, so you can blame it on us.”
Frequency
Intonational Phrases (IPh)
• One or more in each sentence
• Group and organize words into phrases
• Structure is usually determined by syntax
Intonational Phrases (IPh)
• “...defined as containing at least one nuclear accent and a boundary tone at their right edges”
• Pitch contour drops at the end, resets at start of the next IPh
• Last syllable usually lengthened
• Often followed by a pause
Examples
• She went to the store, then she picked up the dry cleaning on the way home.
• She went to the store, then she dropped the kids off at school, then she went home.
Intonational Phrases (IPh)
• Seem to be secondary to syntax in some ways– supporting role
• BUT, listeners can identify prosodic boundaries in the absence of semantic and syntactic information (de Rooij, 1975)
Intonational Phrases (IPh)
• Beckman (1996)“the prosodic structure of the utterance has to
be seen as a full grammatical property also requiring its own parsing”
• However, it might be harder to process prosody in absence of other linguistic information– Off-line behavioral studies can’t address this
issue
Steinhauer et al. (1999)
• Studied prosody with ERPs– Two sentence types: 1 or 2 IPh boundaries– Saw positive-going waveform in response to
all boundaries – CPS
– Possible Confound:• Close relationship between prosody and syntax• CPS could still be related to processing syntactic
boundaries, not just prosodic boundaries
Steinhauer & Friederici (2001)
• ERP study with delexicalized speech– Filtered to removed phonemic, semantic and
syntactic information– Only prosodic info left (pitch, amplitude,
rhythm)
CPS in this case could only be caused by prosody
Steinhauer & Friederici (2001)
• Results– Strange-looking CPS at first boundary– No CPS at second boundary– Contingent negative variation (CNV) across
whole sentence
Is this because the sentences are so unnatural?
Is this task completely different from language processing?
Meyer et al. (2002)
• fMRI study– Very different responses to natural speech
and delexicalized speech
– Response to prosody stronger in right hemisphere
• “strong evidence that pitch processing in the absence of additional linguistic information such as syntax and/or semantics takes place in right hemisphere regions”
Current Experiments
• Motivations:– Investigate partial replication of CPS findings
by Steinhauer & Friederici (2001)• Why didn’t they get the expected results?• Try using more natural stimuli
– See if CPS shifts to the right hemisphere as segmental information is removed
Experiments
• Experiment 1: Normal sentences– Normal semantic, syntactic and phonemic info
• Experiment 2: Jabberwocky sentences– Remove semantic information
• Experiment 3: Pseudo sentences– Remove semantic and syntactic information
• Experiment 4: Hummed sentences– Remove semantic, syntactic, and phonemic
info (only prosody left)
Expt 1: Normal Sentences
A1: [Kevin promises mom to sleep]IPh1
[and to be a good boy for a while.]
B1: [Kevin promises]IPh1 [mom to kiss]IPh2
[and to be a good boy for a while.]
Expt 1: Normal Sentences
A1: one IPh boundary at 1950 msec
B1: two IPh boundaries at 950 and 2700 msec• “First part” (subject + verb) of sentence longer than A1• High boundary tone at end of first part• Longer pause after first part
• Both conditions have IPh boundary after second verb (marked by high boundary tone)
Expt 2: Jabberwocky Sentences
All content words replaced with pseudo words
A2: [The bater rabels Onna to lubol]IPh1
[and the rado to nupe.]
B2: [The bater rabels ]IPh1 [Onna to lubol]IPh2
[and the rado to nupe.]
Expt 2: Jabberwocky Sentences
A2: one IPh boundary at 2100 msec
B2: two IPh boundaries at 1100 and 2600 msec
• “First part” (subject + verb) of sentence longer than A2• High boundary tone at end of first part• Longer pause after first part
• Both conditions have IPh boundary after second verb (marked by high boundary tone)
Expt 3: Pseudo Sentences
All function and content words replaced with pseudo words
A3: [Bater saklimm Onna ko labei keg ]IPh1
[nug som Rado lie nupes.]
B3: [Bater saklimm]IPh1 [Onna ko labei keg]IPh2 [nug som Rado lie nupes.]
Expt 3: Pseudo Sentences
A3: one IPh boundary at 2000 msec
B3: two IPh boundaries at 920 and 2400 msec• “First part” (subject + verb) of sentence longer than A3• High boundary tone at end of first part• Longer pause after first part
• Both conditions have IPh boundary after second verb (marked by high boundary tone)
Expt 4: Hummed Sentences
A4: [mm mmm mmmm mm mmmm]IPh1
[mmmm mmm mmm mmm mmmmm.]
B4: [mm mmm]IPh1 [mmmm mm mmm]IPh2 [mmmm mmm mmm mmm mmmmm.]
Expt 4: Hummed Sentences
A4: one IPh boundary at 1850 msec
B4: two IPh boundaries at 850 and 2150 msec• Longer pause after first part• High boundary tone at end of first part• (First part itself is NOT longer in this case)
• Both conditions have IPh boundary after second verb (marked by high boundary tone)
Procedure• Subjects only saw one type of sentence
• Auditory presentation for all sentences
• Trial followed by a probe word– Subjects had to say whether word was in
sentence or not• (Words were placed randomly into hummed filler
sentences)
Data Collection
• 23 Ag/AgCl electrodes
• 200 msec prestimulus baseline
• Averages computed over whole sentences – 4500 msec segments
Results
A1: IPh at 1950 ms
Response at 2000 ms
------------------------------
B1: IPh1 at 950 ms
Response at 1500 ms
IPh2 at 2700 ms
Response at 2700 ms
A2: IPh at 2100 ms
Response at 2200 ms
------------------------------
B2: IPh1 at 1100 ms
Response at 1500 ms
IPh2 at 2600 ms
Response at 2800 ms
A3: IPh at 2000 ms
Response at ms
------------------------------
B3: IPh1 at 920 ms
Response at 1500 ms
IPh2 at 2400 ms
Response at 2500 ms
A4: IPh at 1850 ms
Response at 2200 ms
------------------------------
B4: IPh1 at 850 ms
Response at 1000 ms
IPh2 at 2150 ms
Response at 2000-2500 ms
Negative peak from 500-1000 ms
Waveforms measured from offset of 1st sentence fragment
Their interpretation...
• Positivity resembles CPS
• Observed in all experimental conditions– Related to processing of prosodic boundaries
• Scalp distribution changes across conditions
Their interpretation...
• CPS associated with first IPh seems related to the amount of segmental content in sentences– Moved forward and rightward as info
decreased
• CPS associated with second IPh seems UNrelated to amount of segmental content
Scalp Distributions – IPh1
• Experiment 1: CPS over whole head
• Experiment 2: CPS at midline and lateral sites, also moves to anterior sites
• Experiment 3: Moves to right anterior sites
• Experiment 4: CPS broadly distributed
Scalp Distributions
• First IPh:
– CPS moved anterior sites as linguistic information decreased
– CPS for psuedo sentences moved rightward, but not for hummed sentences
(possible explanation – left hemisphere processing timing of hums?)
Scalp Distributions• Second IPh:
– CPS distributed broadly over midline sites• also over lateral sites for normal and pseudo
sentence (why not for other types of sentences?)
– CPS moved rightward for hummed sentences ONLY
(CPS not seen here at all in previous study- conclude that naturalness is important)
Negativity
• Early negativity in hummed sentences– Different processing mechanism for pure
prosody?– Search for early accent (that isn’t there)?
– Different task demands?
Their Conclusions
• CPS is independent of expt manipulations– Seems to be related to perception of prosodic
boundaries– Dependent exclusively on prosody
“The observed differences in the scalp distribution of the CPS as a function of the segmental content of the acoustic speech stream suggest that prosodic processing interacts with other information types involving different systems.”
Questions• Why did scalp distribution change?
(It doesn’t seem to change in a systematic way)
• Why are there different results for the 1st and 2nd IPh boundaries?
• Are the differences in timing meaningful?
• Why not compare sentences with no IPh boundaries? (one phrase)
• Is it reasonable to say that these are “natural” manipulations?