Spontaneous speech,interaction& large databasesfor prosodic research

Roxane Bertrand & Cristel PortesUniversité de ProvenceLaboratoire Parole et Langage,UMR 6057 CNRSFrance

S2S04/20/2009

Spontaneous speech, interaction& large databases for prosodic research

Prosody & pragmatics/discourse analysis

Grammar approachGrammar approach

Constructed dataConstructed data

Information flow traditionInformation flow traditionMonologueMonologue

Naturally occuring dataNaturally occuring data

Contextualization traditionContextualization traditionDialogueDialogue

Spontaneous speech in large databasesSpontaneous speech in large databases

S2S04/20/2009

Spontaneous speech, interaction& large databases for prosodic research

Prosody & pragmatics/discourse analysis

Beyond the sentence, 4 dimensions 1. Using sentences:

speech acts, pragmatics

2. Combining sentences: coherence/cohesion, discourse analysis

3. Adapting to context (situation): discourse typology

4. Coordinating with the interlocutor(s): interaction, conversation analysis

S2S04/20/2009

Spontaneous speech, interaction& large databases for prosodic research

Contours

Non Final (without discourse value) Final L- H* / L- HL* (with discourse value)

Falling Non-falling ‘No revision anticipated’ ‘Revision anticipated’ H- L* (T%)

Rising Falling from penultimate peak

‘Speaker not ready ‘Speaker ready to revise’ to revise’ L- H+L* (T%)

Simple Rise Rising-falling

L- H* (T%) L- HL* (T%)

Marandin (2004, 2006) Contours as constructions

Prosody and illocution

S2S04/20/2009

Spontaneous speech, interaction& large databases for prosodic research

Prosody and illocution

Contour

Non stylized

Non-falling Stylized

Rising Fall from peak

Falling Simple rise Rising-Falling

A B D C

Marandin (2004, 2006) Contours as constructions

S2S04/20/2009

Spontaneous speech, interaction& large databases for prosodic research

Prosody & Illocution

Portes, Bertrand & Espesser 2007

S2S04/20/2009

Spontaneous speech, interaction& large databases for prosodic research

A discourse prosodic unit:the paratone

Discourse topic = what a piece of discourse is ‘about’ Paratone = a group of utterances corresponding to a

discourse topic and prosodically marked: Beginning phrases (vs intermediate vs final phrases)

higher and wider (resetting) Delayed first pitch peak downtrend Louder = between Slower intonational

phrases (IP) Final lowering Very long pause

Brown & Yule (1983), Grosz & Hirschberg (1992), Swerts (1994), Wichmann (2000)

S2S04/20/2009

Spontaneous speech, interaction& large databases for prosodic research

Phonological units are preserved in spontaneous data despite…

Disfluencies Orthogonal prosodic variation

Tempo Faster slower

Pitch range Register level Register span

Both have discursive and interactional specific functions …

S2S04/20/2009

Spontaneous speech, interaction& large databases for prosodic research

Disfluency insidethe Accentual phrase

S2S04/20/2009

Spontaneous speech, interaction& large databases for prosodic research

Tempo & register variation

S2S04/20/2009

Spontaneous speech, interaction& large databases for prosodic research

Slower tempo

S2S04/20/2009

Spontaneous speech, interaction& large databases for prosodic research

Register

S2S04/20/2009

Spontaneous speech, interaction& large databases for prosodic research

Interactional Linguistics Perspective (1)(Couper-Kuhlen 2001; Couper-Kuhlen & Selting, 1996)

Background

How linguistic structures are shaped by interaction? How, simultaneously, linguistic structures influence

interaction?

Attempt of elaboration of an Interaction Grammar

Interaction in langage? Linguistics in Interaction?

S2S04/20/2009

Spontaneous speech, interaction& large databases for prosodic research

Take into account verbal activities in which speakers are involved to reach some objectives or specific tasks in talk-in-interaction Locate interactional activities (such as narratives, requests, reported

speech, humor, etc.) Characterize these activities at various linguistic levels

What type of resources available? Role of prosodic cues?

Observables, tools and methods from the Conversation Analysis (Sacks et al. 1974)

The Interactional Phonetics (Local, Ogden, etc.)

Interactional Linguistics Perspective (2)

S2S04/20/2009

Spontaneous speech, interaction& large databases for prosodic research

Basic notions of an approach of talk-in-interaction (1)

Naturally-occurring data More often BUT : elicited conversations, task-oriented corpus, etc.

enable to analyze in a systematic way the whole resources availabe avoid to reject some phenomena (overlaps among others)(see Bertrand et al. 2008, The CID: Corpus of Interaction Data)

Activity collectively fulfilled Take into account all the partners Analysis unit based on their relevance in the interaction for the

participants

S2S04/20/2009

Spontaneous speech, interaction& large databases for prosodic research

Basic notions of an approach of talk-in-interaction (2)

Co-ordination, alignment and negociation according to the shared knowledge Specific units or phenomena to make this understandable:

adjacency pair (request~answer for example, extract from the CID)

Sp1 mais les euh les nanas du foyer elles étaient pas au courant

but girls of residence they did not knowSp2 non non

no noSp1 ah…

Localisation in specific points (TCU & TRP)

S2S04/20/2009

Spontaneous speech, interaction& large databases for prosodic research

Overview of the turn-taking system

Mechanism for the organization of turn-taking:

relies on 2 components related to the construction and the allocation of the turn

A turn constructional component = turn-constructional unit (TCU)

A turn-allocation component = transition relevance place (TRP)

S2S04/20/2009

Spontaneous speech, interaction& large databases for prosodic research

TCU and TRP

How do speakers build and recognize TCU and TRP?

What type of resources or cues are used?

Turn-Constructional Unit (TCU): the smallest interactionnally relevant complete linguistic unit (Selting 1998: 40) A TCU is a point of completeness (Ford &

Thompson, 1996)

Syntactic (clause) Prosodic (intonative unit achieved in

a terminal rising) Pragmatic (complete action of

request and answer)

Transition-Relevance Place (TRP) Completion points which make a transition

relevant but not necessarily accomplished (Schegloff 1996: 55)

S2S04/20/2009

Spontaneous speech, interaction& large databases for prosodic research

Crucial notion of Projection (1)

Large projects (Selting 2000) Consists of more than 1 TCU = multi-unit turns (explanation,

narratives, description, etc.)

Need to be projected by the SPEAKER

Typical prosodic features of turn-design are used locally to project more-to-come (Ogden 2005, Bertrand et al. 2007, Kern 2007) pitch contours (extracted from the CID): the rising contour (H*H%)

Turn-holding device

S2S04/20/2009

Spontaneous speech, interaction& large databases for prosodic research

Crucial Notion of Projection (2)

What about the RECIPIENT? Each TRP occurs in a point of potential

achievement built from the different linguistic criteria which are used by recipients as predictable cues.

S2S04/20/2009

Spontaneous speech, interaction& large databases for prosodic research

Backchannel signals (1)

BCs are short utterances produced by the recipient to co-construct discourse by orienting it in one or another way different functions

Continuer Acknowledgement Assesment Attitude statement, etc.

They provide information on interlocutor’s listening but also comprehension processes of discourse (Fox Tree 1999)

Prosodic role in the projection of this kind of response?

S2S04/20/2009

Spontaneous speech, interaction& large databases for prosodic research

Backchannel signals (2)

Context of production of BCs Higher occurrences of BC after terminal rising contours (continuation rise)

Different BC’s function according to a terminal rising vs a continuation rising (Bertrand et al. 2007) function as continuer for continuative rising function as assesment for terminal rising

Confirmation of the « more-to-come » intonation function Prosodic cues organize the floor by making specific recipient response

relevant.

Multimodal analysis (Bertrand et al. 2007): gestural resources also play a role in the context of production of BC More gestural BC after continuative rising than vocal BC less intrusive

Confirm that BC highlight some steps in the elaboration of discourse, and more precisely in the construction of different steps of the larger projects

S2S04/20/2009

Spontaneous speech, interaction& large databases for prosodic research

Direct Reported Speech in conversation

Who is speaking and for what? Reported speech is not only used to report words but also

to convey their assessment of the utterance while reproducing it (Holt 2000)

to increase one’s standing or saying something without really assuming it (Bertrand 2003)

to typify a character on which members of the same community shared knowledge and typical representations (Klewitz & Couper-Kuhlen 1999; Bertrand & Priego-Valverde, 2009)

Using of a specific prosodic delivery to make the another voice hearable and understable (Couper-Kuhlen 1996, Klewitz & Couper-Kuhlen 1999 Bertrand & Espesser 2002) Melodic Shift in the beginning of DRS But an absence of such a shift is yet a relevant cue Specific prosodic design of each voice in a reported dialog

S2S04/20/2009

Spontaneous speech, interaction& large databases for prosodic research

Reported dialog in conversation

ReLe ( N ) ( Rai ) ( N ) ReSp ( N ) ( Exp ) ( Com ) Pauses [722ms] [331ms]

oh puis tu sais le fait de suite de dire non mais je vais aller le voir moi le prof et tout je vais aller lui dire que c’est oh eh j’ai dit béa attend quand même eh

Re(gister)Span/Re(gister)Le(vel) N(ormal) = direct speech Rai(sed) and Exp(ansion) of span = reported speech (other figure) N(ormal)/Com(pressed) = reported speech (self-quotation)

S2S04/20/2009

Spontaneous speech, interaction& large databases for prosodic research

Prosodic orientation (Szczepek-Reed 2006)

Belongs to the general frame of the interactional orientation throughout the course of a conversation (…)

speakers display in their sequentially “next” turns an understanding of what the “prior” turn was about’ (Hutchby & Wooffitt 1998: 15).

Several cases of prosodic orientation complementation of a prior turn continuation of a previous unfinished prosodic pattern copy of a previous prosodic pattern

S2S04/20/2009

Spontaneous speech, interaction& large databases for prosodic research

Prosodic mapping in humor(Bertrand & Priego-Valverde 2009)

4 TCUs with the same syntactic and prosodic form:

little variation in f0 curve, slightly falling, final lengthening + filler (euh)

he works at I.R.A.

and he blows up uh

and he put the detonators uh

yeah he is watchmaker at I.R.A

S2S04/20/2009

Spontaneous speech, interaction& large databases for prosodic research

In Sum

Prosody is a crucial resource in the management of turn-taking system and in the structuring of various activities displayed in everyday conversations Construction of turn Projection of points of completion Projection of more-to-come Step by step constitution of the shared

knowledge

S2S04/20/2009

Spontaneous speech, interaction& large databases for prosodic research

Conclusion Of course, naturally occurring data present to the observation multiple

sources of information mixed together so that they may be hidden at first sight.

However, phonological (grammatical) units are recoverable in spontaneous speech. The confrontation of grammatical formalization with attested data and the reverse appear to be very fruitful.

In order to do so, we need a separate model of disfluencies (which are not grammatical but play important interactional roles).

We also need to treat orthogonal dimensions of prosody (tempo, pitch range, intensity) separately because of their specificity (gradience).

Finally, we need to dissociate the speaker from the addressee, and to study the way each of them takes its specific part in the co-construction of discourse.

S2S04/20/2009

Spontaneous speech, interaction& large databases for prosodic research

References

Bertrand R. & Espesser R. (2002) Voice diversity in conversation : a case study, in Bel B. & Marlien I (éds), Speech Prosody 2002, Aix-en-Provence, 171-174.

Bertrand R., Portes C., Sabio F. (2007) Distribution syntaxique, discursive et interactionnelle des contours intonatifs dufrançais dans un corpus de conversation Travaux neuchâtelois de linguistique, 47, 59-77

Bertrand R., Ferré G., Blache P., Espesser R. & Rauzy S. (2007) Backchannels revisited from a multimodal perspective' Proceedings of Auditory-visual Speech Processing. Hilvarenbeek, Cederom.

Bertrand, R., Blache, P., Espesser, R., & al., (2008) « Le CID - Corpus of Interactional Data -Annotation et Exploitation Multimodale de Parole Conversationnelle », in Traitement Automatique des Langues, 49 : 3 (in press)

Bertrand, R. & Priego-Valverde B. Does prosody play a specific role in conversational humor?, Pragmatics and Cognition (accepted)Couper-Kuhlen E. & Selting M (1996), Prosody in conversation. Interactional studies, Cambridge University Press,.Couper-Kuhlen E. (2001) The Handbook of Discourse analysis.Couper-Kuhlen E. & Ford C.E. (2004) Sound Patterns in Interaction, John Benjamins Publishing Company. Couper-Kuhlen E. (1999) Coherent voicing: On prosody in conversational reported speech, In Wolfram Bublitz & Uta Lenk (eds.) Coherence in

Spoken and Written Discourse: How to create it and how to describe it, Amsterdam: Benjamins, 11-32.Ford C. E. & Thompson S. A. (1996), Interactional Units in Conversation : syntactic, intonational and pragmatic resources for the management of

turns. In Interaction and Grammar, E. Ochs, E. A. Schegloff & S. A. Thompson (eds), 134-184, Cambridge UP.Fox Tree J.E. (1999) Listening in on Monologues and Dialogues", Discourse Processes 27(1): 35-53.Hirschberg J. (2002) The pragmatics of Intonational meaning, in Bel B. & Marlien I (éds), Speech Prosody 2002, Aix-en-Provence, 65-68.Hutchby I and Wooffitt R. (1998) Conversation Analysis. Principles, Practices and Applications (Cambridge: Polity Press)Jefferson G. (1978) Sequential aspects of storytelling in conversation, Studies in the organization of conversational interaction. New York (Academic

Press), 219-248. Kern F. (2007) Prosody as a resource in children’s game explanations: Some aspects of turn construction and recipiency,

Journal of Pragmatics 39, 111-133Klewitz G. & Couper-Kuhlen E. (1999) Quote-Unquote? The Role of Prosody in the Contextualization of Reported Speech Sequences, Pragmatics

9, 459-485.Labov W. & Waletzky J. (1966) Narrative analysis: oral versions of personal experience. In: J. Helm (ed.), Essays on the verbal and visual arts:

Proceedings of the 1966 Annual Spring Meeting of the American Ethnological Society. Seattle (University of Washington Press), 12-44. Local J. (2007) Phonetic detail in talk-in-interaction: on the deployment and interplay of sequential context and phonetic resources, Nouveaux

cahiers de linguistique française 28, 67-86, Université de Genève.Ogden R. & Routarine S. (2005) The communicative functions of Final Rises in Finnish Intonation, Phonetica, 62, 160-175.Sacks H.,Schegloff E. A., Jefferson G. (1974) A Simplest Systematics for the Organization of Turn-Taking for Conversation,

Language, 50, 4, 696-735.SchegloffF E. A. (1996), Turn organization: One intersection of grammar and interaction. In Interaction and Grammar, E. Ochs, E. A. Schegloff & S.

A. Thompson (Eds), Cambridge: Cambridge University Press, 52-133.Selting, M., (2000) The construction of 'units' in conversational talk", Language in Society 29: 477-517.Szczepek Reed B. (2006) Prosodic orientation in English Conversation, Palgrave Macmillan

S2S04/20/2009

Spontaneous speech, interaction& large databases for prosodic research

Prosodic cues in turn-taking