Exploiting Under-specification for Semantic Co-ordination

1. Dialogue as Co-ordination Problems

2. Two Dialogue Tasks:

The Maze Task

Verbal Dialogue: Spatial Reference Task

The Music Drawing Task Music Task

Graphical Dialogue: Musical ‘Pictionary’ Task

3. Group-specific sub-languages.

4. Repair-driven Co-ordination

Lewis’s (1969) Model of Convention

Many activities are co-ordinated by conventions• e.g., place and time to meet• specific place and time matter less than co-ordination of choice

Co-ordination problems have at least two alternative ‘co-ordination equilibria’

Two features:

1. Shared repertoire of possible co-ordination equilibria

2. Choice between them is arbitrary• salience• precedence

The Maze Task

• Recurrent problem of describing target locations• Target alternates and configuration changes on each trial

Player A Player B

Maze Task: Description Types

Figurative (Figural / Path): sensitive to particular configuration

A: right on the right hand side there are four boxes,B: mmhumA: then there are two shapes and then there's another four linked boxes, B: yes A: right it's the second from the bottom.

B: mmm, ummm, take the bottom left hand corner, A: yes B: up one box A: yesB: right one boxA: yes B: up one box

Maze Task Description Types

Abstract: (Line / Co-ordinate): abstracts underlying grid structure

A: ummm, fourth row down and the second from the right, [12]

B: okay it's the second row down and second in from the left, [13]

B: er: two two, [3]

A: six: six three, [4]

B: four three, [5]

(Kappa = 0.76, N =455, k= 2)

The Music Drawing Task:Exclusively graphical interaction via virtual whiteboard• Pairs seated in separate rooms

– 30 sec piano piece each - SAME or DIFFERENT?

• Draw picture of target: no letters or numbers

Room A Room B

Music Drawing Task: Drawing Types

Figurative: – Ad hoc associations: faces, figures, objects or situations

Abstract: – Graph-like representation of domain structure e.g., pitch,

intensity, rhythm

Composite:– Mixture of Abstract and Figurative

(Kappa = 0.9, N =287, k= 2)

Sequence of ‘Figurative’ Trials:

Sequence of ‘Abstract’ Trials:

Subject 2

Subject 1 Subject 6

Subject 5

Subject 3 Subject 4

Round 1 = Round 2 =Round 3 = Round 4 =

Phase1: Community Development

Between Group =

Within Group =

Phase 2: Experimental Manipulation

Both Tasks Phase 1:

1. Different partner on each round

2. Common ‘interaction history’ accumulates

3. Manipulations of group and dyad structure are hidden

• Music task: 10 ‘communities’ of 6 people– 4 rounds of 12 trials – reliable increase in speed and accuracy

• Maze Task: 4 ‘communities’ of 8 people– 5 rounds of 20 trials– Reliable increase in number of items completed

Chi2(2) =19.0, p=0.00

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

Within Community Between Community

Proportion

Abstract

Figurative

Composite

Music Drawing Results for Phase 2:

Music drawing Results for Phase 2:

Maze Task Results for Phase 2

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

Within-Group Cross-Group

AbstractFigurative

Maze Task Results for Phase 2

Crossing between sub-groups changes choice of description types

• Cross-group vs. Within-Group: Chi2(3) = 129.6, p=0.00

And provokes twice as many clarifications• Clarifications: Cross Group 37%, Within Group 16%

Cross-group pairs are not distinguishable from ‘naïve’ pairs on trial 1

• Cross-group vs. Naïve: Chi2(2) = 3.34, p=0.19.

Group-Specific Sub-languages

• In both tasks co-ordination is group specific

• Direct interaction plays an essential role in co-ordination– in addition to aggregate individual experience

• expert-ese not expertise

• In both tasks cross-group interaction is problematic– specifically de-stabilises ‘Abstract’ representations– Cross-group pairs are comparable to Naïve pairs

Why are the ‘abstract’ representations more unstable?

Co-ordination Equilibria?

Choice of ‘Abstract’ or ‘Figurative’ representations is not arbitrary

‘Abstract’ Semantic Models capture regularities across items • Musical structure - melody, tempo, intensity, • Grid structure - squares, rows, columns, diagonalsMore specifically:1. Systematicity: support direct comparison within and between

items2. Proto-compositionality: (relatively) consistently individuated

ontology

But therefore require closer co-ordination• several ontological schemes are possible• not consistently manifest in particular items

Selection of Co-ordination Equilibira?

Precedence?• Maze Task:

– converge on the least frequent initial description type– after problems people switch ‘down’ not ‘up’

Interactive Alignment?• Within and Cross-group pairs have same level of alignment

Explicit Negotiation?• Maze task: rare, agreement often violated, and most common

after co-ordination has developed. • Music Task: no meta-language• Bootstrapping problem

Emergent Semantic Co-ordination

Semantic convergence is a product of (not pre-condition for) interaction

• consistent migration to ‘Abstract’ representations – driven by direct interaction

• different varieties emerge in different communities• on inspection almost every pair’s solution differs• no convergence with passive overhearers

What are the mechanisms?

Repair Driven Co-ordination?

Miscommunication: breakdowns in understanding and their resolution are the key events

1. Try something.

2. If it works don’t worry (charitable interpretation)

3. If it fails

a) use under-specification (be ‘Figurative’ or ‘vague’)– semantic repair strategy

b) exploit potential for joint manipulation of representation – localise the representational problem

reprise, partial repeat, circle, underline

(‘meta-communicative’ interaction devices)?

Repair Driven Co-ordination?

Evidence?• Both tasks: initial choice appear to be random• Maze task: no convergence with passive overhearers• Music Task: preventing people from editing / annotating each

other’s drawings de-stabilises ‘Abstraction’.• Maze task: spoof clarification (“what?”, “row?” ) de-stabilises

‘Abstraction’.

• Communication is a special case of misunderstanding– persistent residual ambiguity– communicative success = mutual-indiscriminability

• Miscommunication is a pre-requisite for semantic convergence– U shaped curve?