Generation of Referring Expressions: Managing Structural Ambiguities

I.H. Khan G. Ritchie K. van Deemter

University of Aberdeen, UK

A natural language generator should avoid generating those phrases, which are too ambiguous to understand. But, how the generator can know whether a phrase is too ambiguous or not? We use corpus-based heuristics, backed by empirical evidence, that estimate the likelihood of different readings of a phrase, and guide the generator to choose an optimal phrase from the available alternatives.

Process of generating text in natural language (e.g., English) from some non-linguistic data (Reiter & Dale, 2000)

Example NLG system

– Pollen Forecast: generates reports from pollen forecast data

Natural Language Generation (NLG)

Grass pollen levels for Tuesday have decreased from the high levels of yesterday with values of around 4 to 5 across most parts of the country. However, in South Eastern areas, pollen levels will be high with values of 6. [courtesy E. Reiter]

Referring Expression = Noun Phrase – e.g., the black cat; the black cats and dogs (etc.)

A key component in most NLG systems Task of GRE:

– Given a set of intended referents, compute the properties of these referents that distinguish them from distractors in a KB

Generation of Referring Expressions (GRE)

GRE: An Example

Objects PropertiesObject1 (Type, Sheep), (Color, Black)

Object2 (Type, Sheep), (Color, Brown)

Object3 (Type, Sheep), (Color, Black)

Object4 (Type, Goat), (Color, Black)

Object5 (Type, Goat), (Color, Yellow)

Object6 (Type, Goat), (Color, Black)

Object7 (Type, Goat), (Color, Brown)

Object8 (Type, Cow), (Color, Black) Output: Distinguishing Description (DD)

– (Black Sheep) (Black Goat)

KB

Input: KB, Intended Referents R

Task: find properties that distinguish R from distractors

The Problem

NP1: The black sheep and the black goats

NP2: The black sheep and goats

(Black Sheep) (Black Goat) = {Object1,Object3,Object4,Object6}

(Black Sheep) Goat = {Object1,Object3,Object4,Object5,Object6,Object7}

NP1 unambiguous and long; NP2 ambiguous and brief

Question: How the generator might chose between NP1 and NP2?

Linguistic ambiguities can arise when DDs are realised

Our Approach Psycholinguistic evidence

– Avoidance of all ambiguity is not feasible (Abney, 1996)

Avoid only distractor interpretations– An interpretation is distractor if it is more likely or almost as

likely as the intended one.

Question– How to make distractor interpretation precise?

Our solution– Getting likelihood using word sketches (cf. Chantree et el., 2004)

– Word sketches provide detailed information about word relationships, based on corpus frequencies

– Relationships are grammatical

Hypothesis 1– If Adj modifies N1 more often than N2, then a narrow-scope

reading is likely (no matter how frequently N1 and N2 co-occur).

bearded men and women handsome men and women

Hypothesis 2– If Adj does not modify N1 more often than N2, then a wide-

scope reading is likely (no matter how frequently N1 and N2 co-occur)..

old men and women tall men and trees

Pattern: the Adj N1 and N2

Experiment 1

 

Please, remove the roaring lions and horses.

Experiment 1: Results

Hypothesis 2 (i.e., predictions for WS reading) is confirmed

Hypothesis 1 (i.e., predictions for NS reading) is not confirmed

– Tendency for WS (even though results are not stat. sig.)

Tentative conclusion– An intrinsic bias in favour of WS reading

BUT: The use of *unusual* features may have made people’s judgements unreliable

Experiment 2

 

Please, remove the figure containing the young lions and horses.

Experiment 2 (cont.)

 

Please, remove the figure containing the barking dogs and cats.

Results: Both hypotheses are confirmed

Word Sketches can make reasonable predictions about how an NP would be understood.

But we need more to know from generation point of view: which of the following two NPs is best?

The black sheep and the black goats

The black sheep and goats

(Black Sheep) (Black Goat)

(Black Sheep) Goat

We seek the answer in next experiment

Clarity-brevity trade-off

Recall the pattern: the Adj Noun1 and Noun2

Brief descriptions (+b) take the form– the Adj Noun1 and Noun2

Non-brief descriptions (-b) take the form– the Adj Noun1 and the Adj Noun2 (IR = WS)

– the Adj Noun1 and the Noun2 (IR = NS)

Clear descriptions (+c)– Which have no distractor interpretations

Non-clear descriptions (-c)– Which have some distractor interpretations

Hypothesis 1

– (+c, +b) descriptions are preferred over (+c, -b)

Hypothesis 2

– (+c, -b) descriptions are preferred over (-c, +b)

Each hypothesis is tested under two conditions– C1: intended reading is WS

– C2: intended reading is NS

The Hypotheses (Readers’ Preferences)

Experiment 3: NS Case

 

Which phrase works best to identify the filled area?

1. The barking dogs and cats

2. The barking dogs and the cats

Experiment 3: WS Case

 

Which phrase works best to identify the filled area?

1. The young lions and the young horses

2. The young lions and horses

Experiment 3: Results

Both hypotheses are confirmed:

– (+c, +b) descriptions are preferred over (+c, -b)

– (+c, -b) descriptions are preferred over (-c, +b)

Role of length:

– In WS cases preferences are very strong

– In NS cases preference is not as strong as in WS cases

Summary of Empirical Evidence

For the pattern the Adj Noun1 and Noun2

– Word Sketches can make reliable predictions

– Keeping clarity the same, a brief NP is better than a longer one

Algorithm Development

Main knowledge sources

– WordNet (for lexicalisation)

– SketchEngine (for predicting the most likely reading)

Main steps

1. Choose words

2. Use these to construct description in DNF

3. Use transformations to generate alternative structures from DNF

4. Select optimal phrase

Transformation Rules

Input

– Logical formula in DNF

Rule Base

1.(A B1) (A B2) A (B1 B2)

2.(X Y) (Y X)

[A = Adj, B1=B2=Noun, X=Y=(Adj and/or Noun)]

Output

– Set of logical formulae

Select optimal phrase

1. (black sheep) (black goats) DNF2. (black goats) (black sheep)3. black (goats sheep)4. black (sheep goats) Optimal

(4): Adj has high collocational frequency with N1 and N2, so the intended (wide-scope) reading is more likely.

Therefore, (4) is selected.

Conclusions

GRE should deal with surface ambiguities Word sketches can make distractor interpretation precise Keeping clarity the same, brief descriptions are preferred

over longer ones A GRE algorithm is sketched that balances clarity and

brevity

THANK YOU