‘Speech-Smile’, ‘Speech-Laugh’, ‘Laugher’and their sequencing in dialogic interaction

Klaus J. Kohler

IPDS, Kiel, Germany

Satellite Workshop “The Phonetics of Laughter”, ICPhS Saarbrücken, 5 August, 2007

1 Types of laughing

• free laughter– complex breath control esssential element

° air stream direction° energy time course

– modified in fairly simple ways (supra)glottally– different from fairly simple subglottal and complex

(supra)glottal settings in speech

• speech-laugh: attributes of free laughter superimposed on speech – coordination of two types of breath control

• speech-smile– categorically different from laughter and speech

laugh– breath control of speech– super-imposed long-term attributes that are

related to smiling, physically and functionally° lip spreading° tongue fronting° pitch raising

– may be preceded by facial expression of smile without speech or paralinguistic vocalization

2 Functions of laughing

• detailed interpretation of laughing depends on contextual settings of speaker interactions

• general feature of – free laughter and speech-laugh seems to be the

expression of amusement, high-key hilarity– spech-smile a signal of happiness, low-key joy

3 Database and method

• two spontaneous data sources– stereo recording of dialogue session consisting of 6

sub-dialogues between two secretaries in the Appointment-making Scenario with overlap: f061

– stereo recording of two male speakers in the Video Task Scenario LINDENSTRASSE: l061 ° discussing differences in video clips from the

German TV series° presented to them separately beforehand

• speakers knew each other well, high degree of spontaneity and naturalness

• in f061, jm and mg arrange two 2-day business meetings in a 2-month period– cannot find mutually suitable dates– not enough successive daily slots in their calendars – mg suggests 2 meetings immediately following

each other, turning them into a 4-day meeting– jm considers it possible but not appropriate– finds clash between task and solution funny

° speech-laugh, followed by subdued laughter

– mg comments with speech-laugh ° points out that this concern is not important

in such an appoinment-making game– then turns into laughter – picked up by jm– followed by several exchanges of laughter

between the 2 speakers

Dialogue ex. 1

• in l061, speaker mpi sets theme of the “utter stupidity” of the German TV soap series LINDENSTRASSE– caps this with hilarious laughter– whole dialogue revolves round this theme between

speakers mpi and tra – leads to several exchanges of laughter

Dialogue ex. 2

Dialogue ex. 3

• phonetic and interactional data described in detail– through auditory assessment– visual inspection of acoustic records

° spectrograms° f0 and energy traces

– functional interpretation• for full analysis of speech-smiles

– video recordings necessary – not in the database– deduction from sound patterns

4 Phonetic exponents of speech-smile• The following sequence of speech-laugh and laughter

ends in a speech smile on the phrase ja, gut; speaker mg.

• lip spreading/fronting and rising pitch of gut, compared with na gut from same speaker in a non-smiling context

F1 468 Hz, F2 1071 Hz vs. F1 409 Hz, F2 936 Hz

Ex. 1.1 laugh + smile

Ex. 1.2 smile vs. non-smile

• The following is a sequence of speech-smile on the phrase und so (followed by hesitations) turning into laughter; speaker tra.

• lip spreading/fronting of und so, compared with Show from the same speaker in a non-smiling context

F1 590Hz, F2 1160 Hz vs. F1 450 Hz, F2 840 Hz

Dialogue Ex. 3.1 smile + laugh

Ex. 3.2 smile vs. non-smile

5 Phonetic exponents of loud laughter• laugh preceding smile in Ex. 1, female speaker mg

– egressive and ingressive air streams, oral – voice bursts, same duration, equi-distant, in

voiceless breathing, half-open to close vowel – energy control, equi-prominent, f0 grouping

Ex. 1.1 laugh (+ smile)

• laugh in Ex. 2, male speaker mpi

– egressive air stream, oral – voice bursts, in voiceless breathing, 2 short up-

beat, 4 longer bursts, open vowel resonance– energy control, 4 double-peaked energy bursts,

strong – weak: trochee patterns

Ex. 2. 1 laugh

• laugh following smile in Ex. 3, male speaker tra

– egressive air stream, oral – voice bursts, in voiceless breathing, less

regular, half-close vowel resonance– energy control, less rhythmic, f0 grouping

Ex. 3. 3 (smile) + laugh

6 Phonetic exponents of restrained laughter

• laugh in Ex. 4, same speaker as in Ex. 2, mpi

– egressive air stream, nasal, restrained laughter – 3 blocks of voice bursts, 4 short, 3 long, 3 very

short, each equi-distant, in voiceless breathing, – energy control, equi-prominent, f0 grouping

Ex. 4 laugh

• laugh in Ex. 1, after speech ne, female speaker jm

– egressive air stream, oral, subdued laughter – 3 blocks of weak - strong rhythmic patterns,

1 breath, 2 voice– former by duration, latter by energy control– sounds regular in spite of different distances

Ex. 1.3 speech + laugh

Ex. 1.4 laugh

• laugh in Ex. 1, after speech-laugh on glücklich female speaker jm

– long ingressive – short egressive air stream, oral, subdued laughter

– 2 strong – weak rhythmic patterns,– decreasing by energy control across the 2– sounds very regular

Ex. 1.5 speech + laugh

Ex. 1.6 laugh

7 Speech-laugh• laugh in Ex. 1.5, female speaker jm

– ingressive oral breath, continues into acc. glück– change to egressive + strong energy increase in glück,

phonation break– ingressive – egressive in lich, phonation break– double-peaked energy control in the fricative,

egressive, mouth opening

Ex. 1.5 laugh on “nicht so glücklich”

• laugh in Ex. 1.1, female speaker mg

– egressive oral air stream – double-peaked energy control in “ja”, 2nd peak

higher, vowel split in two– followed by strong egressive breath, gradually

trailing off over long period

Ex. 1.1 laugh on “ja”

8 Sequencing of laugh types

• any sequence is possible– speech-laugh – laughter – speech-smile, mg– speech-laugh – subdued laugh, jm– loud laughter – speech smile, mpi– speech-smile – laughter, tra

° laughter sets in after mip’s beginning of the speech-smile

– speech-smile – laughter, jm

9 Laughing interaction in dialogue

• laughing phenomena sequenced and timed for communicative functions – within one speaker– between speakers

• f061 coordinated action between the two speakers– part of their joint task-solving in the Appointment-

making game• in l061 the two speakers talk about differences they

have seen in video clips – mutually trigger laughter by facetious descriptions

of the soap opera excerpts

10 Conclusion

• kind of Conversation Analysis investigation of sampled instances of spontaneous occurrences of laughing phenomena

• small database suggests that fine phonetic detail is highly structured acoustically and functionally, including rhythmical factors

• extension of the auditory and acoustico-graphic investigation to broad databases from spontaneous speech of various scenarios, considering– speaker types – and different languages

• detailed feature description is prerequisite to large-scale acoustic analysis and introduction into speech technology