Jens Allwood

SCCIIL,Linguistics,

University of Gothenburg

Berlin, February, 2009

Feedback, coactivation and coconstruction

Feedback, coactivation and coconstruction

- an analysis of the relation of common ground to dialog (communication)

on the basis of

Activity based Communication

analysis

Why interesting?Human Communicative Feedback a

place where

Top-down and Bottom-up meet

• Top-down: anchorable and derivable from theory of Human Communication

• Bottom- up: anchorable and derivable from frequent type of data in human communication

• Many applications

Top - down• Humans are social, rational

animals

• The primary means for this is Communication (dialog)

Communication = def. • sharing of old or new factual, emotive and

conative aspects of cognition through • coactivation and coconstruction of content,

information or understanding• as a part of and means for joint social

activities involving coordination, attunement, collaboration and/or cooperation

• Often multimodal and interactive

Dialog = def. Interactive communication• Often multimodal and non-competitive and

non-conflictual

Thus communication involves

• Participating in a number of processes that need to be managed so that communicative behaviors can be adapted to reach the criteria of success of these processes

• We have therefore evolved several mechanisms for communication management, e.g. turn management, feedback, sequencing and own communication management

• The perhaps most important of these is the system for communicative feedback

Communicative feedback

• Provides mechanisms and criteria for management of the success of

Communication = i.e.

sharing through coactivation and coconstruction of factual, emotive and conative cognition

Through means for eliciting and giving

• positive and negative feedback

with varying degrees of awareness and intentionality

• These enable interactive management and success of communication

Positive and negative feedbackwith varying degrees of

awareness and intentionality• What is Positive feedback wrtsharing through coactivation (attunement) and

coconstruction of cognition (factual, emotive and conative) ?

Body movements and vocal

• What is Negative feedback wrtsharing through coactivation (attunement) and

coconstruction of cognition (factual, emotive and conative) ?

--> Body movements and vcocal

• How do we elicit reactions, responses - feedback?

--> Using Body movements and vcocal means

Interactive view on embodied communicationTwo communicators form one dynamic

system by establishing stabilizing communication links

Feedback

Feedback employs all of these levels, too.

What is embodied com. feedback?

Communication, in which two or more embodied communicators are taking turns in contributing information, requires a feedback system to make sure the contributed information is really shared.

• Use of vocal and non-vocal channels for feedback (speech, prosody, laughter, facial expression, gaze, head gesture, manual gesture, posture, body position and orientation).

• There is continuous and simultaneous sharing of information between speaker and hearer (e.g. Argyle & Cook, 1976).

• Feedback can range from conscious, intentional signals to nonconscious, unintentional indicators of how information processing is going.

• Underlying are automatic processes of mimicry, co-activation, synchrony of bodily or vocal action, and emotional contagion(e.g. Wallbott, 1995; Grammer et al., 1996).

Communicative Feedback(Allwood, 1976; Allwood, Nivre & Ahlsen, 1993)Every communicative expression can evoke

reactions and it can respond to evocative aspects of a previous contribution.

Responsive feedback includes short unobtrusive expressions whereby a recipient of information informs the contributor about– Ability & willingness to communicate (have

contact)– Ability & willingness to perceive the

information– Ability & willingness to understand the

information– Ability & willingness to accept the main

evocative intention (can statement be believed, question answered, request complied with)

– Emotions and attitudes triggered by the perceived information.

Dimensions of embodied feedback

Every feedback expression can be classified with respect to several dimensions, the most relevant being:

• Types of expression or modality used (e.g. Vocal, gestural)

• Types of function/content of the expressions

– ± Contact (C)– ± Perception (P) – ± Understanding (U)– ± Acceptance of main evocative intention (A)– Emotions/attitudes (E)

• Degrees of control and awareness– innately given causal influence, without control

or awareness– potentially aware and intentionally controllable– actually aware and intentionally controlled

Dimensions of embodied feedback (ctd.)

• Types of communicative intentionality (Allwood, 1976) – Indicated information that the sender is not

aware of ”showing”, or intending to convey– Displayed information is intended by the

sender to be shown– Signaled information is intended by the

sender to be recognized by the receiver as being displayed

• Types of reception its production is linked to– reactive and automatic vs. responsive and

aware

• Degrees of continuity – digital and discrete vs. continuous and

analogue

• Semiotic information carrying relations – indices vs. icons vs. symbols

Degrees of awareness, intentionality and

conventionalization

Index Icon Symbol

Indicate Moti-vated,

Natural

Display Moti-vated,

Manipulable

Signal Manipula-ble,

Arbitrary

Laughter• Modality of expression Vocal and gestural• Function/content of the expressions

– ± Contact (C)– ± Perception (P) – Emotions/attitudes (E)– --------------------------------------------– ± Understanding (U)– ± Acceptance of main evocative intention (A)– Both give feedback and elicit reaction

• Degrees of control and awarenessMostly innately given causal influence,

without control or awareness but is – potentially aware and intentionally

controllable or actually aware and intentionally controlled

• Mostly indicated, sometimes displayed, seldom signalled

• Mostly index. Seldom icon or symbol• Types of reception, mostly reactive,

spontaneous, contagious• Degrees of continuity, continuous and

analogue rather than digital and discrete

When does a communicative sound become a word (symbol)?

• Criteria• Similarity of expression and Similarity of

function/content

between communicators

• Conventionalization of link btween expression and content (arbitrary)

• Systematic difference between communication-communities

Most feedback expressions like mm, ng etc meet these criteria

• Inclusion in written language as a written word ( if this were necessary, no language without written languages would have words)

Feedback: Degrees of awareness, intentionality and conventionalization

Ontogenetic and phylogenetic development

Feedback behaviors

Indicate

index

coactivation, emotional contagion, orientation, synchrony, similar behavior (repetition), attunement, harmony,

Bodily movements, esp. Head

Laughter, smile, sounds

Conventionalization possible (bird dialects)

Display: Intentional, manipulable

orientation, synchrony, attunement, harmony

Bodily movements, esp. Head, laughter, smiles, sounds, esp. similar behavior, i.e. Icons - increased conventionalization

Signal Increased intentional manipulation -> less natural manipulation -> arbitrariness -> symbols, i.e. Conventional FB signals for orientation, attunement, coactivation, agreement

Dimensions of feedback in different types of expression

Aspects of bodily coordination

Facial expression, posture, prosody

Head gestures

Vocal verbal

Awareness andcontrol

Innate,automatic

Innate, potentially aware + controlled

Potentially/mostly aware + controlled

Potentially/mostly aware + controlled

Modality Visible Visible, audible

Visible Audible

Function/content

C, P, E C, P, E C, P, U, E, A C, P, U, E, A

Intentionality Indicate Indicate, display

Signal Signal

Type of reception

Reactive Reactive Responsive Responsive

Continuity Analogue Analogue, digital

Digital Digital

Semiotic sign type

Index Index, icon Symbol Symbol

Control architecture of a feedback-giving communicator

Must combine processes of high-level evaluation (e.g. Allwood et al., 1993)and low-level appraisal (e.g. Scherer, 1999)

Example

A: always raining in GothenburgB: (nodding) mhm yeah it is

(depressed)

The functions of A’s contribution

A: 1. ? Responsive: (? B present as pot. communicator)

2. Expressive: opinion (belief) - predication3. Referential: implicit (meteorological situation) made relevant by predication

4. Evocative: Continue Perceive Understand Share belief

Obligations: 1. A considers B cognitively and ethically

2. A desires continuationCommitments: A believes predication

B’s reaction/response and obligation

1. Nonconscious reactions (both internal and behavioral)

2. Evaluate own willingness/ability concerning evocative functions, i.e. Given my present emotional/volitional/cognitive state/social role

Can I/do I want to Continue

Perceive

Understand

Accept evocative function

Other reactions

3. React/respond on the basis of the evaluation,

i.e. feedback + possible continued coactivation of information

B: mhm yeah it is

Functions:1. Reactive/responsive: Contact, Perception,

Understanding

2. Expressive: CPU + Depression

reformulating opinion/ belief-

agreement

3. Referential: same as in A’s contrib.

4. Evocative: CPU (share belief)

Shared information/content understanding (common ground?)

Dialog creates/activates1. Shared contact (automatic reactions)2. Shared perception3. Shared understood content/information (including

emotions)4. Accepted evocative fuctions (agreements , commitments to

joint action)5. Continued relevant coconstruction

Drawing on a shared context/background of1. Shared perceptual environment (com & noncom)

2. Jointly engaged in activity (com & other instr action)

3. Other information activated by communication and other instrumental action

(iv) Other common traits (physical, biological, social, psychological, e.g. cultural

Representing a jointly activated and constructed content containing

• multimodal information - often using short 1-word utterances and gestures• other instrumental action + feedback concerning willingness/ability to perceive, understand, accept

How do we and/or the communicators know if information is shared (is

common ground)?

• Coherence of discourse i.e. - That evocative functions have a fit with

responsive functions (local relevance),- esp. relevant fedback (indicated,

displayed, signalled), - referential coherence,- continued relevant coconstruction

• Global relevance with joint purpose of social activity

• Relevance to shared interests• Lack of negative feedback• Requests for clarification handled

The build up of common ground is also influenced by the fact that

Communication involves Cooperation, Ethics & Trust

• Communication

Sharing through coactivation and coconstruction of information, content or understanding

mediated through

• mutual causal influence between organisms(mechanisms)engaging in

• (motivated, intentional, rational) action and interaction

The cooperative features of

communication mean that

• Common ground of coactivated and coconstructed old and new iformation can be more easily established

• Coordination, collaboration and cooperation facilitate reaching different levels of common ground

ButCommunication (dialog)

can be

• Unethical• Distrustful

And Unsuccessful communication (dialog)

Can be

• uncoordinated and

• lack collaboration on understanding and providing of information

Unsuccessful communication

• Even in these cases some types of common ground can be established

And is sometimes presupposed

- Subconcious CG- shared perception CG- -shared understanding CG

- Conflict and disagreement can sometimes involve lack of coordination, perception and understanding but does not always do so.

- Some types of conflict and disagreement presuppose understanding to be effective

Bottom -upEmpirical data on feedback

• Data on

• Indicative FB behaviorautomatic processes of mimicry, co-activation,

synchrony of bodily or vocal action, and emotional contagion

• Displayed FB behaviorDisplayed processes of mimicry, imitation,

coordination of bodily or vocal action, and emotional and conative attunement

• Signalled-symbolic FB behavior e.g gestural and verbal, see below

Collaborative and cooperative behavior

Indicative and/or Displayed FB30 interacting pairs of students, systematically varied with respect to sex and mutual acquaintance. Task to find out as much as possible about each other within 3 min. Self-reported rapport

L (00:00:13): Bist du im ersten Semester $R (00:00:15): Ich bin eigentlich im fünften Semester aber die ersten zwei hab ich nicht wirklich was gemacht und dann // $L (00:00:18): aha $R (00:00:20): jetzt bin ich im dritten $L (00:00:23): Zoologie oder Botaniker oder was $R (00:00:26): entweder Anthro oder Zoologie das weiss ich noch nicht so genau $ L (00:00:28): aha die Anthropologen sind viel besser $L (00:00:33): mhm /// hast du schon den Seidler gemacht $R (00:00:35): ja $….

Some empirical data on signalled, symbolic

Feedback

Rank German Swedish

1 Ja Ja

2 hm m

3 mhm nä

4 ja ja jaha

5 hm hm hm

6 nein ja ja

7 genau nej

8 ja ja ja ja just det

9 gut ha

10 aha m m

11 ne hej

12 und jaså

13 bitte va

14 eben okej

15 mhm mhm jo

16 nicht aha

17 aber tack

18 äh ja visst

19 ah ja a

20 hmmh ja precis

21 jaja nä nä

22 ja ja ja ja vad sade du

• Most common complete utterances in Swedish and German

German Swedish23 mhhm men24 nee nähä25 ach so precis26 nich så27 jawohl m ja28 nein nein öh29 na ja ja tack30 ja bitte javisst31 ich jaja32 ja eben just det33 okay ja okej34 mhm ja ja m35 wiedersehn eh36 also ja ja ja37 natürlich varsågod38 richtig tack så mycket39 danke ja det är det40 hm hm hm visst41 mh ja det42 ja hm ja men43 das det är det44 sicher vad45 ja sicher vadå46 ähä det47 guten tag m m m48 ja mhm tack ska du ha49 gell oj50 mm bra51 ja genau ja ha52 doch hej då53 ja aber nä just det54 wiederhören tjugo55 oh nej nej56 hm ja ah57 tja och58 so två59 eh äh60 ach tre61 ach ja ja absolut62 vier vier eins sieben sieben sieben japp

• German Swedish63 ja gut sju64 dankeschön tjugofem65 danke schön nä nä nä66 ja klar näeh67 wiedersehen ja ja visst68 auf wiederhören hm hm69 ah so fem70 genau ja fyra71 dann ja då72 oder åtta73 ja und yes74 m det tror du75 bitte schön tack hej76 was hej hej77 hier hallå78 herr ja det tror jag

79 ja das jaha ja80 ja natürlich jag vet inte81 gut ja det tycker du82 ja ich ett83 warum jag84 ähm gula sidornas informationsservice

god middag85 ah tolv86 bitte sehr det är87 hm hm hm hm ja det tycker jag88 aha ja nä men89 vielleicht eller90 das is sex91 schön m hm92 moment nio93 selbstverständlich femton94 vielen dank ja jo95 nee nee ja ja just det96 bitteschön tio97 jo tjugotre98 hallo du99 so ist es arton100mhmh sexton

100 most common utterances

Function German Swedish

FB-giving 70 62

FB-elicit 10 9

Gratitude 7 5

Greeting 6 5

Other 7 19

Complete utterances of length 1 (Danish and British English) Minimum frequency is 2 and the frequency measures are given as PPM. Danish (bysoc) British English (bnc) Collocation Frequency PPM Collocation Frequency PPM ja 22820 16592.9 yeah 32506 3116.2 m 11657 8476.0 m 24717 2369.5 nej 4104 2984.1 yes 10103 968.5 nå 4025 2926.7 no 9160 878.1 jo 708 514.8 oh 5395 517.2 uf 409 297.4 right 4066 389.8 aha 278 202.1 what 2823 270.6 så 269 195.6 erm 1577 151.2 det 134 97.4 okay 1471 141.0 men 132 96.0 ah 1470 140.9 altså 132 96.0 aha 1466 140.5 ik 115 83.6 well 1251 119.9 og 108 78.5 aye 1156 110.8 næ 99 72.0 why 921 88.3 åh 78 56.7 er 832 79.8 hva 54 39.3 pardon 830 79.6 ah 50 36.4 eh 801 76.8 cm 49 35.6 alright 650 62.3 tsk 48 34.9 sorry 643 61.6 ej 47 34.2 ooh 638 61.2 sådan 44 32.0 yep 628 60.2 ler 38 27.6 really 624 59.8 hvad 37 26.9 so 604 57.9 latter 30 21.8 hello 507 48.6 næh 29 21.1 who 460 44.1 m+ 28 20.4 and 415 39.8 gud 26 18.9 good 412 39.5 i 24 17.5 mum 407 39.0 hej 22 16.0 bye 345 33.1 hø 22 16.0 but 334 32.0 eller 21 15.3 mr 328 31.4 jamen 21 15.3 ha 327 31.3 på 20 14.5 where 278 26.7 okay 20 14.5 i 276 26.5 mor 19 13.8 mummy 271 26.0

Complete utterances of length 1- 10 (Danish (bysoc) & British English (bnc) Minimum frequency is 2 and the frequency measures are given as PPM. Danish (bysoc) British English (bnc) Collocation Frequency PPM Collocation Frequency PPM ja 22820 16592.9 yeah 32506 3116.2 m 11657 8476.0 m 24717 2369.5 nej 4104 2984.1 yes 10103 968.5 nå 4025 2926.7 no 9160 878.1 ja ja 1896 1378.6 oh 5395 517.2 m m 1582 1150.3 right 4066 389.8 jo 708 514.8 what 2823 270.6 uf 409 297.4 m m 1870 179.3 nej nej 325 236.3 oh yeah 1602 153.6 aha 278 202.1 erm 1577 151.2 så 269 195.6 that s right 1494 143.2 nå nå 221 160.7 okay 1471 141.0 nå ja 201 146.2 ah 1470 140.9 ja m 189 137.4 aha 1466 140.5 jo jo 141 102.5 well 1251 119.9 det 134 97.4 aye 1156 110.8 men 132 96.0 yeah yeah 1070 102.6 altså 132 96.0 i know 975 93.5 ik 115 83.6 why 921 88.3 og 108 78.5 oh yes 918 88.0 ja det 100 72.7 er 832 79.8 næ 99 72.0 pardon 830 79.6 nå m 93 67.6 eh 801 76.8 ja ja ja 90 65.4 thank you 739 70.8 hvad siger du

85 61.8 alright 650 62.3

m ja 84 61.1 sorry 643 61.6 åh 78 56.7 ooh 638 61.2 nej det 65 47.3 yep 628 60.2 ja ja ja ja 63 45.8 really 624 59.8 ja tak 59 42.9 oh right 622 59.6 og så 57 41.4 so 604 57.9 hva 54 39.3 no no 579 55.5

Parts of speech and one word utterances

tokens % of allutterances

% of all 1-word

utterancesAdj 798 0.80 3.27Adv 340 0.34 1.39conj 153 0.15 0.63Fb 20 048 20.01 82.21Int 714 0.71 2.93

Noun 829 0.83 3.40Num 646 0.64 2.65OCM 500 0.50 2.05Prep 42 0.04 0.18Pron 215 0.21 0.88Verb 100 0.10 0.41

Total 24385 24.33 100

Swedish Feedback reduplications

ja Init. Ensk. Med. Fin. n % n % n % n %

jaja… 2 129 35 27 66 51 12 9 16 12 3 8 4 50 2 25 1 13 1 13 4 1 0 0 1 100 0 0 0 0 ja_ 2 1341 611 46 379 28 279 21 72 5 3 109 50 46 38 35 17 16 4 4 4 26 12 46 8 31 4 15 2 8 5 7 5 71 1 14 1 14 0 0 6 1 0 0 0 0 0 0 1 100 8 1 1 100 0 0 0 0 0 0 12 1 0 0 1 100 0 0 0 0 jaha 2 9 2 22 4 44 2 22 1 11 aha 2 2 1 50 1 50 0 0 0 0 3 1 0 1 100 0 0 0 0 0 ha 2 35 4 11 5 14 17 49 9 26 3 8 1 13 2 25 2 25 3 38 4 4 0 0 0 0 0 0 4 100 jaså 2 2 1 50 0 1 50 0 0 0 javisst 2 2 1 50 1 50 0 0 0 0 ja_visst 2 2 1 50 0 0 1 50 0 0 jamen 2 1 1 100 0 0 0 0 0 0 3 1 1 100 0 0 0 0 0 0 ja_men 2 3 1 33 0 2 67 0 0 0 5 1 1 100 0 0 0 0 0 0 jadå 2 1 1 100 0 0 0 0 0 0

Some examples of reduplication

Some examples of reduplications Jaja (2) Medialt (12) A4627011 (Formal Meeting) 84. vad ni hade för olika definitioner och det var ju intressant / [70 för mej ]70 och att liksom att att rotera och [71 skriva ]71 rent som jag gjorde men sen börjar jag diskutera dom hela för att du ifrågasatte <17mattias >17 va >16 $J: [70 ja ]70 $J: [71 ja ]71 $J: ja [72 absolut ]72 $S: [72 jaja ]72 / jaja men då börjar vi ju en diskussion / $J: jo [73 men ]73 $S: [73 och det ]73 var inte tanken ja ja ja ja (4) Initialt (12) A3210011 (Inform al Conversation) 422. $A: (...) för mycket att göra så du inte komm er då $B: nä m en det ordnar sej nog $A: ja men det var ju slut [1 (...) som det ]1 $B: [1 ja ja ja ja m en ]1 m en det ordnar sej $A: ja men på vilket hur kan du veta $B: [1 ja ja ja ja m en ]1 m en det ordnar sej $A: ja m en på vilket hur kan du veta att det ordnar sej då

ja ja ja ja ja (5) Initialt (5) V0630022 (Discussion) 450. för att hon / m åste komm a lika långt som alla andra [51 utan ]51 [52 det ]52 bara för hon tar hon ärväldigt nogrann gör allting [53 så ]53 vill göra det så ordentligt och perfekt [54 som m öjligt ]54 va såhon behöver lite mer tid [55 helt enkelt ]55 och då får hon göra vissa saker [56 hemma ]56 $A: [49 ja ja ja ja ja <20 ja >20 m m ]49 $A: [50 m ]50 $A: [51 nä ]51 $A: [52 nä ]52 $A: [53 ja ]53 $A: ja ja ja ja ja ja ja ja (8) Initialt (1) A4627011 (Form al Meeting) 455. / [127 på ett ]127 abstra kt sätt är ju att / jag kan inte undvika att intresse ra mej för det va / e ftersom jag måste veta när konf likt och konfliktl ösning så att säg a NÄR [128 börjar dom bitarna och < när > börjar dom A NDRA bitarn a ]128 $O: [126 proje ktet gäller det nu va ]126 $O: [127 ja ]127 $O: [128 ja ja ja ja ja ja ja ja ]128 och sen så kommer det e n konflikt < okej > / det $J: och vad är det ja ja ja ja ja ja ja ja ja ja ja ja [= 12] Enskilt (1) A7720011 (Inte rview) 456. a har $J: med sin man $T: NEJ med varandra $J: dom gör det allt så [96 inom ]96 $T: [96 ja ja ]96 ja ja ja ja $J: dom har en grupp allts å [97 kan man säg a det / och så ]97 då som jag har förstått står det lite grann i motsät tning till dom män som då kan sitta kvar till sju åtta nio på kvällen där $T: [97 ja ja ja ja ja ja ja ja ja ja ja ja ]97 $T: ja j ag tror <262 janne >262 han går för sej det tror

Concluding remarksThis talk has agued that communicative

feedback both from a top-down and bottom-up perspective is an essential feature og human communication

• Feedback is a means for managing

the sharing of information through coactivation, and coconstruction in activity based interactions which is characteristic of all human communication

• Feedback occurs with several degrees of awareness, intentionality and conventionalization; from indicative behavior to signalled symbols