semantics and pragmatics of nlp lexical semantics: machine ... › ... › spnlp › lectureslides...

university-logo

What is Logical Metonymy?Rule-Based Accounts

Some Shortcomings/GapsMachine Learning

Semantics and Pragmatics of NLPLexical Semantics: Machine Learning

Alex Lascarides

School of InformaticsUniversity of Edinburgh

Alex Lascarides SPNLP: Autmoated Lexical Acquisition

university-logo



Outline

1 What is Logical Metonymy?

2 Rule-Based Accounts

3 Some Shortcomings/Gaps

4 A probabilistic model for interpreting logical metonymies


university-logo



What is Logical Metonymy?

Semantic type of a syntactic complement to a word differs fromthe semantic type of the argument in logical form:

(1) a. Mary finished the cigarette.b. Mary finished smoking the cigarette.

(2) a. Mary finished her beer.b. Mary finished drinking her beer.

(3) a. easy problemb. difficult languagec. good cook


university-logo



Things in Common

1 Additional meaning is predictableThe event that’s finished/enjoyed/started is the purpose ofthe denotation of the noun

2 Interpretations can be rendered with a paraphrase


university-logo



Major Challenges

Semi-productivity??enjoy the tunnel, ??enjoy the door etc.Context-sensitivity

(4) My goat eats anything. He enjoyed your book

Ambiguityfast scientist: publishes quickly, does experiments quickly

researches quickly, persuades people quicklythinks quickly . . .

are all highly plausible interpretations

We will tackle ambiguity and discuss semi-productivity.


university-logo



Theoretical Accounts: Generative Lexicon

Against sense enumeration;meaning of adjective/verb depends on noun;nouns have qualia structures:

This represents very simple world knowledge:what object is made up of;its purpose;how it was created.

adjectives/verbs modify qualia for nouns.


university-logo



Example (Simplified): enjoy the book

book: inherited qualia enjoy: inherited info; begin, finish etc.2664bookSEM : book(y)

QUALIA :

"CONST : pagesTELIC : readAGENTIVE : write

#3775

266664coercing

CAT SUBCAT :

*24 npSEM : n [Q(y)]

QUALIA TELIC : P

35+

SEM : [e][enjoy(e, x, e′) ∧ act-on-pred/ P (e′, x, y) ∧ n ]

377775

enjoy the book:coercing

CAT SUBCAT :

⟨ npSEM : n book(y)

QUALIA TELIC : P read

⟩SEM : [e][enjoy(e, x , e′) ∧ / P read(e′, x , y) ∧ n book(y)]


university-logo



Gaps

They assume noun classes have one (perhaps default)telic role.So don’t investigate relative degree of ambiguity of variouscases of metonymy (e.g., fast scientist vs. fastprogrammer)Or degree of variation

for an N with different verbs:begin the house (agentive) vs. enjoy the house (telic)for verb with different Ns:begin the tunnel (agentive) vs. begin the book (telic)

Manually constructing a lexicon with very rich semanticinformation

so as to account for regular polysemy is impractical anyway.


university-logo



An Alternative: Machine Learning

Can the meanings of metonymies (and other forms ofregular polysemy) be acquired automatically from corpora?Can we constrain the number of interpretations byproviding a ranking on the set of meanings?

Finding Answers Empirically:

Provide a probabilistic modelModel parameters: exploit meaning paraphrases

co-occurrences of nouns, verbs and metonymicverbs/adjectives in the corpus

evaluate results against human judgements


university-logo



The Model: Metonymic Verbs enjoy book

Find e which maximises the probabilityP(e, book , enjoy) of seeing “enjoy e-ing book”.

The Equations: e=event; v=metonymic verb; n=noun

(5) P(e, n, v) = P(e) · P(v |e) · P(n|e, v)

Estimating the probabilities: P(e) = f (e)Pi

f (ei )

P(v |e) = f (v ,e)f (e)

P(n|e, v) = f (n,e,v)f (e,v)


university-logo



Sparse Data for f (n, e, v)! BNC

enjoy movie:

(6) I’ve always enjoyed watching spy movies.

begin speech:

(7) a. Churchill. . ., as he had begun to make public publicspeeches. . .

b. Liam sprang on to a table, raised a glass and beganto declaim a speech.

c. The Prince. . .he began to make increasinglyserious and significant speeches.

d. For the first time in ten years I’m gonna begindelivering a speech.

Grice predicts sparse data problem!Be brief:Use begin speech rather than begin V-ing speech if theinterpreter can infer the value of V anyway.


university-logo



Solving the Estimation Problem

Assume that the likelihood of seeing n as object of e isindependent of whether e is the complement of v

So:P(n|e, v) ≈ P(n|e)

P(n|e) = f (n,e)f (e)

P(e, n, v) ≈ f (v ,e)·f (n,e)Pi

f (ei )·f (e)


university-logo



Example: enjoy the film

f (enjoy , e) f (film, e)

play 44 make 176watch 42 be 154work with 35 see 89read 34 watch 65make 27 show 42see 24 produce 29meet 23 have 24go to 22 use 21use 17 do 20take 15 get 18

So events associated with enjoying films are:watching, making, seeing, using

Model is ignorant of context;determines most dominant meanings in the corpus.


university-logo



How We Estimated Parameters

Corpus: POS-tagged, lemmatised BNC (100 millionwords), parsed by Cass (Abney, 1996)

Verb-argument tuples: f (e, n)

Can extract verb-SUBJ and verb-OBJ (need just verb-OBJhere)Errors make filtering necessary:

e.g.: discard Vs that only occur once; particle Vs(come off heroin) retained only if particle isadjacent to N

Metonymic verb and its complement: f (v , e)

Metonymic verb v followed byVBG (progressive) or To0 (infinitival)


university-logo



Examples for f (e, v)

(8) a. I am going to start writing a bookstart write

b. I’ve really enjoyed working with youenjoy work

c. The phones began ringing off the hookbegin ring

(9) a. I had started to write a love-storystart write

b. She started to cook with simplicitystart cook

c. The suspect attempted to run offattempt run off


university-logo



Paraphrases from the Literature Verspoor 1997,Pustejovsky 1991, 1995

John began the book → reading/writingJohn began the sandwich → eating/makingJohn began the beer → drinkingJohn began the cigarette → smokingJohn began the coffee → drinkingJohn began the speech → writingJohn began the lesson → writing/takingJohn began the solo → playingJohn began the song → singingJohn began the story → tellingJohn enjoyed the symphony → listening toJohn enjoyed the film → watchingMary enjoyed the movie → watchingJohn quite enjoys his morning coffee → drinkingBill enjoyed Steven King’s last book → readingMary likes movies → to watchHarry wants another cigarette → to smokeJohn wants a beer → to drinkMary wants a job → to have


university-logo



Model-Derived Paraphrases (ranked by likelihood)

underline: value agrees with claims about meaning in theliterature

P(e, n, v) e1 e2 e3 e4 e5

P(e, book, begin) read write appear in publish leaf throughP(e, book, enjoy) read write browse through look through publishP(e, sandwich, begin) bite into eat munch unpack makeP(e, beer, begin) drink pour sip crack sellP(e, beer, want) drink buy sell weep into getP(e, cigarette, begin) smoke roll light take twitchP(e, cigarette, want) smoke take light put buyP(e, coffee, begin) pour drink sip make stirP(e, coffee, enjoy) browse through drink make take go forP(e, speech, begin) make read recite disclaim slurP(e, lesson, begin) learn teach take read reciteP(e, solo, begin) play sing tun hem work throughP(e, song, begin) sing rehearse write hum playP(e, story, begin) tell write read re-tell recountP(e, symphony, enjoy) play listen to write hear serveP(e, film, enjoy) watch make see go to work withP(e, movie, enjoy) watch go to make see eat inP(e, movie, like) see go to watch make filmP(e, job, want) get lose take make create


university-logo



Evaluation: Comparison Against Human Judgements

Randomly select 12 metonymic verbsattempt, begin, enjoy, expect, postpone, prefer, resist, start,survive, try, wantfrequency in BNC between 10.9 per million and 905.3 permillion

Randomly select 5 nouns which are attested as objects tothese verbs.Use model to derive meanings of the resulting 60combinations.Divide set of generated meanings into three probabilitybands

High, Medium, Low (equal size)


university-logo



Example Stimuli

60 V-N pairs × 3 bands = 180 stimuli

Michael attempted a smile Michael attempted to give a smileMichael attempted a smile Michael attempted to rehearse a smileMichael attempted a smile Michael attempted to look at a smileJean enjoyed the concert Jean enjoyed listening to the concertJean enjoyed the concert Jean enjoyed throwing the concertJean enjoyed the concert Jean enjoyed making the concert


university-logo



The Experimental Procedure

Magnitude estimation of linguistic judgements (ME):Subjects see a modulus item and assign it an arbitrarynumber; other stimuli are rated proportional to themodulus;ME yields highly robust and maximally delicate judgementdata;

Subjects

experiment administered over the Web using WebExp(Keller et al. 2001);60 subjects, each subject saw 90 stimuli;judge meaning paraphrases.


university-logo



Results

Performed analysis of variance (ANOVA) to test whetherparaphrases with high probs are perceived to be betterthan those with low probs.Probability bands yield desired differences!

Post-hoc Tukey tests show that the differences between allpairs of conditions were significant; α = 0.1Comparison between our model and human judgementsyields a Pearson correlation coeffecient of 0.64 (p < 0.01,N = 1742).

So model correlates reliably with human judgements.


university-logo



What’s the Upper Bound? Inter-Subject Agreement

Correlations computed via ‘leave-one-out cross-validation’:Start with m subjectsDivide subjects into 2 groups of size m − 1 and 1Correlate mean ratings of the responses of first subjectgroup with that of the latter subject.Repeat m times.

ResultGives average inter-subject agreement of .74So model doing OK (scoring .64), given this upper bound.


university-logo



The Lower Bound

Naive baseline model is to just take verb-nounco-occurrence data into account.

Don’t use f (e, v) to estimate P(e, v , n)

Like assuming that the metonymic verb is semanticallyempty.

Cf. begin the house vs. enjoy the house

Results:Naive model has Pearson correlation coefficient of 0.42Difference with our model is statistically significant(p < 0.05).And correlation between naive model and our model isrelatively low; 0.46


university-logo



Semi-ProductivityExamples from Verspoor 1997, Pustejovsky1995, Lascarides and Copestake 1998

John began a chair → sitting in/onJohn began the tunnel → driving throughJohn began a keyboard → typing onJohn began the trees → growing/planting/wateringJohn began the highway → driving onJohn began the film → watchingJohn began the nails → hammering inJohn began the door → opening/walking throughJohn began the dictionary → readingJohn enjoyed the path → hikingMary began the rock → ???


university-logo



What Grice (1975) Predicts

Maxim of MannerWhen metonymy is acceptable, you are relatively morelikely to use metonymic construction than its paraphrase.The opposite is true when metonymy is unacceptable.

So does our model comply with this prediction?


university-logo



Model-derived Paraphrases for ‘Bad’ Examples

P(e, n, v) e1 e2 e3 e4 e5

P(e, chair, begin) fidget on sink into rise from take moveP(e, tunnel, begin) waddle through dig walk towards emerge from buildP(e, keyboard, begin) use play operate assemble tuneP(e, tree, begin) climb climb towards sing in run towards growP(e, highway, begin) obstruct regain build use detachP(e, film, begin) make appear in show develop work onP(e, nail, begin) bite dig chew dig in driveP(e, door, begin) open walk towards knock on close move towardsP(e, dictionary, begin) compile flick through use publish advanceP(e, path, enjoy) walk follow ride take travel onP(e, rock, begin) crunch across climb run towards percolate through dissolve


university-logo



BNC Frequencies

Odd f (v, n) f (v, e, n)

begin chair 0 9begin tunnel 0 4begin keyboard 0 0begin tree 1 13begin highway 0 2begin film 0 7begin nail 0 4begin door 0 18begin dictionary 0 3begin rock 0 17enjoy path 1 2

Well-formed f (v, n) f (v, e, n)

begin book 35 17begin sandwich 4 0begin beer 2 1begin cigarette 0 0begin coffee 0 4begin speech 21 4begin solo 1 1begin song 19 8begin story 31 15enjoy symphony 34 30enjoy film 16 5enjoy movie 5 1enjoy coffee 8 1enjoy book 23 9like movie 18 3want cigarette 18 3want beer 15 8want job 116 60

Suggestive, but not conclusive: begin keyboard vs. begincigaretteThis is a model of interpretation, not of grammaticality


university-logo



Other Case Studies

Metonymic Adjectives fast scientist, good soup. . .

Similar model, with additional SUBJ vs. OBJ parameter:OBJ: good soup is soup that tastes goodSUBJ: good programmer is programmer who programs well

Performance of model very similar to metonymic verbsCompound Nouns hospital admission, patient admission. . .

Experiments performed on Medline. ExploitOccurrences of grammatical relations of modifier tocorresponding verb in the corpus;WordNet and UMLS (to smooth over sparse data)

Achieve 72.5% accuracy


university-logo



Conclusions

Regular polysemy is pervasive; it needs to be modelled.Manual modelling is undoableMachine learning can help, becauseyou can exploit meaning paraphrases and surface cues,and this makes unsupervised training feasible.Probabilistic models rank interpretationsand the rankings correlate reliably with human judgementsabout meaning.


semantics and pragmatics of nlp lexical semantics: machine ... › ... › spnlp › lectureslides...

Documents