Mireille BessonEquipe Langage, Musique et Motricité

Institut de Neurosciences Cognitives de la Méditerranée

CNRS-Marseille

BrainTalk2 - 4 june, 2008 Lund, Sweden

Thanks to the Organizers!

Electrophysiological studies of prosody

Outline

o Modality and semantics (Astésano et al, 2004)

� On-line processing of modality

� Modality and Semantics : interactive processing

o Prosodic focus and pragmatics (Magne et et al, 2005)

� on-line processing of focal accents

� Processing of focal accents interacts with discourse context

o Meter and semantics (Magne et al, 2007)

� on-line processing of syllabic lengthening

� Meter and Semantics: interactive processing

Emotional function:Emotional function: express joy, sadness, anger express joy, sadness, anger ……

(Schirmer et al, 2002; Kotz et al, 2003, (Schirmer et al, 2002; Kotz et al, 2003, …… ))

Experiment 1

Men Women

F5 F6

P5 P6

F5 F6

P5 P6

-5 µV

800 ms

N400

Experiment 2

Men Women

F5 F6

P5 P6

F5 F6

P5 P6

-5 µV

800 ms

N400

Schirmer et al, Cognitive Brain Research, 2002

SOA 100 ms SOA 300 ms

Emotional function:Emotional function: express joy, sadness, anger express joy, sadness, anger ……

(Schirmer et al, 2001; Kotz et al, 2003, (Schirmer et al, 2001; Kotz et al, 2003, …… ))

Linguistic function:Linguistic function: modality , focus, segmentation, modality , focus, segmentation, …… through intonation, accents,through intonation, accents,meter, pauses meter, pauses (Alter at al, 2003; 2005; 2007; Astesano et al, 2003; B(Alter at al, 2003; 2005; 2007; Astesano et al, 2003; Bööcker et cker et al, 1999; Eckal, 1999; Eckstein & Friederici, 2005; Friedrich et al, 2004; Magne et al, 2stein & Friederici, 2005; Friedrich et al, 2004; Magne et al, 2005; 005; Meyer et al, Meyer et al, 2000; Steinhauer et al, 1999; 2000; Steinhauer et al, 1999; ……))

From Steinhauer, Alter & Friederici, 1999. Nature Neurosciences

Emotional function:Emotional function: express joy, sadness, anger express joy, sadness, anger ……

(Schirmer et al, 2001; Kotz et al, 2003, (Schirmer et al, 2001; Kotz et al, 2003, …… ))

Linguistic function:Linguistic function: modality , focus, segmentation, modality , focus, segmentation, …… through intonation, accents,through intonation, accents,meter, pauses meter, pauses (Alter at al, 2003; 2005; 2007; Astesano et al, 2003; B(Alter at al, 2003; 2005; 2007; Astesano et al, 2003; Bööcker et cker et al, 1999; Eckal, 1999; Eckstein & Friederici, 2005; Friedrich et al, 2004; Magne et al, 2stein & Friederici, 2005; Friedrich et al, 2004; Magne et al, 2005; 005; Meyer et al, Meyer et al, 2000; Steinhauer et al, 1999; 2000; Steinhauer et al, 1999; ……))

Acoustic parameters:Acoustic parameters: Pitch/FrequencyPitch/FrequencyLoudness/IntLoudness/IntensityensityDuration/TimDuration/TimeeTimbreTimbre

Modality and semantic processing

Astésano, Besson & Alter (Cognitive Brain Research, 2004)

ERP marker of an incongruous modality ?

Time course of modality and semantic processing

Independant vs interactive processes ?

Materials

2,5

3

3,5

4

4,5

5

5,5

6

6,5

7

S1 S2 S3 S4 S5 S6 Syllables

Yes/No

Questions

Statements

ERB Values

F0 contours of the Prosodically Congruous sentences

2,5

3

3,5

4

4,5

5

5,5

6

6,5

7

S1 S2 S3 S4 S5 S6 Syllables

*Questions

*Statements

ERB Values

F0 contours of the Prosodically Incongruous sentences

Le chauffeur conduisait?

The chauffeur was driving?

La lumière clignotait.

The light was flashing.

*Le piéton traversait.

*The pedestrian was crossing.

*Le cycliste pédalait?

*The cyclist was pedaling?

Procedure

16 French native speakers

2 judgment tasks

�Attention to Semantic: congruous/incongruous

�Attention Prosody: congruous/incongruous

Base Line Auditory Stimuli Yes/No Answer

‘ Le garcon coloriait’ (The boy was drawing) XXX

200 ms 2000 ms 2000 ms

� 28 scalp electrodes (International 10/20 system)

Modality and Semantics

P800 : ERP marker of modality processing

Semantic processing precedes modality processing(but differences in task difficulty)

Interaction between semantic and prosodic processing

Prosodic focus

Magne, Lacheret, Morel, Alter & Besson

(Jal of Cognitive Neuroscience, 2005)

Discourse : pragmatic structure organized around 2 main concepts:

topic : known information (le thème = what we are speaking about)

focus : new information /relevant (le rhème = what we are saying about

it)

Focus is marked :

prosodically : focal accent F0, duration and intensity

Objectives

Are focal accents processed in real time?

Time course of focal accent processing as afunction of pragmatic constraints

Stimuli

Questions :

Q1 Tu as eu une bonne ou une mauvaise note à ton contrôle d’histoire ?

Q2 Où as tu eu une mauvaise note, à ton contrôle d’histoire ou d’anglais ?

Answers :

Focus on sentence middle word (M)R1 En fait, j’ai eu une mauvaise note à mon contrôle d’histoire

Focus on sentence terminal word (T).R2 En fait, j’ai eu une mauvaise note à mon contrôle d’histoire

4 experimental conditions

Focus on sentence middle or terminal words (M vs T)

Focus at good or bad positions (+ vs -)

M+(T0) Focus good on middle word (none on terminal word)

M-(T0) Focus bad on middle word (none on terminal word)

T+(M0) Focus good on terminal word (none on middle word)

T-(M0) Focus bad on terminal word (none on middle word)

Task: is the intonation of the answer correct in the context?

EEG acquisition

| Question | ISI | Phrase Réponse | XXXX |1000 2000 1000 msec

|-------------------|-------|---------*--------*----------------------|----------------|--------| essai suivantM T

PROCEDURE

Participants: 16 native French speakers; right handed

Behavioural data

Conditions M+ M- T+ T-

% Erreurs (Sd) 5 (5) 9 (6) 5 (6) 6 (5)

Sorry for the delay.

With focal accent

Without focal accent

Congruous

Incongruous

Cz

Pz

Cz

Pz

Middle word

- 2 µV + 2 µV

300-1000 ms

ms

5 µV

3 0 0 3

300-450 ms 450-600 ms 600-750 ms 750-900 ms

3

900-1050 ms

a. Focal accent (M- - M+)

b. No focal accent (M0T- - M0T+)

30 3 3 3

300-450 ms 450-600 ms 600-750 ms 750-900 ms 900-1050 ms

0

150-300 ms

0

150-300 ms

Pragmatic congruence

(difference waves)

+ 3.5 µV- 3.5 µV 0

Congruent

Incongru

Cz

Pz

Cz

Pz

Terminal word

With focal accent

Without focal accent

- 2 µV + 2 µV

300-1000 ms

5 µV

ms

300-450 ms 450-600 ms 600-750 ms 750-900 ms 900-1050 ms

a. Focal accent (T- - T+)

b. No focal accent (M-T0 – M+T0)

300-450 ms 450-600 ms 600-750 ms 750-900 ms 900-1050 ms150-300 ms

150-300 ms

Pragmatic congruence

(difference waves)

+ 3.5 µV- 3.5 µV 0

40 0 0 0 3

00 0 3 3 3

ConclusionsConclusions

o Focal accents are processed in real time

� Psychobiological relevance of prosodic focus

o Interaction between prosodic focus and pragmatic context

� Pragmatic relevance of focal accent determines the observed effects (more than its presence - absence)

o Observed effects depend upon the position of focal accents in the sentence

� Middle word : positivity Surprise

� Terminal word : negativity Integration

Meter and semantic processing

Magne, Astésano, Ystad, Kronland-Martinet & Besson(Cerebral Cortex, 2007)

ERP marker of incongruous syllabic lengthening ?

Independent vs interactive processes ?

Influence of attention

Meter

Semantic

Congruous(M+)

Incongruous(M-)

Congruous (S+)M+S+

Le concours a regroupémille candidats

M-S+Le concours a regroupé mille

candidats

Incongruous (S-)M+S-

Le concours a regroupémille bigoudis

M-S-Le concours a regroupé mille

bigoudis

32 sentences / experimental condition (= 128 sentences)

Experimental conditions

Task Meter : Is the final word well-pronounced ?

Semantic : Is the final word meaningful in the context?

Participants: 14 native French speakers, right-handed

Protocole

Acquisition EEG + % errors + RTs Time course

| Phrase | Mot final | XXXX |200 2000 2000 ms

|------------------*------|-----------------|----------------|- … Essai suivantMarker

Mètre Sémantique

S+M+ S-M+ S+M- S-M- S+M+ S-M+ S+M- S-M-

%Err 13 (16) 19 (16) 15 (14) 17 (16) 11 (6) 15 (17) 24 (18) 15 (11)

TRs 932 (152) 1055 (179) 966 (102) 997 (135) 975 (139) 1127 (184) 1011 (164) 1138 (146)

Behavioural data

Cz

Pz

SemanticMeter

S+M+

S-M+

N400 N400

1000 ms

-5 µV

300-600 ms 600-1000 ms 600-1000 ms300-600 ms

Cz

Pz

S+M+

S+M-

P700

N400

1000 ms

-5 µV

300-600 ms 600-1000 ms300-600 ms 600-1000 ms

SemanticMeter

Cz

Pz

S+M+

S-M-

N400N400

P700 1000 ms

-5 µV

300-600 ms 600-1000 ms 300-600 ms 600-1000 ms

SemanticMeter

ConclusionsConclusions

o P700 to metric incongruity when attention is focused on meter

� participants are sensitive to the metric structure of words

o N400 to metric incongruity when attention is focused on semantics

� Interaction between metric and semantic processing

o N400 to semantic incongruity independently of attention.

� N400 also reflects automatic semantic processing

o Psychobiological validity of several aspects of prosodic

processing :

Modality

Focal accent

Meter

o Interactions between levels of processing :

Prosodic

Semantic Pragmatic

o Linguistic specificity ?

MusiqueLangage