anaphora resolution sobha lalitha devi au-kbc research centre mit campus of anna university...
Post on 22-Dec-2015
219 views
TRANSCRIPT
05/24/2011 Summer School, IIIT HYderabad 1
Anaphora Resolution
Sobha Lalitha DeviAU-KBC Research CentreMIT Campus of Anna [email protected]
05/24/2011 Summer School, IIIT HYderabad 2
Contents
Introduction to Anaphora and Anaphora Resolution
Types of Anaphora Process of Anaphora Resolution Tools Applications References
05/24/2011 Summer School, IIIT HYderabad 3
What is Cohesion
COHESION is the internal continuity or network of points of continuity within a text.
Text is not just a string of sentences. It is not simply a large grammatical unit
“something of the same kind as a sentence, but differing from it in size- a sort of super-sentence· A semantic unit”
Halliday & Hassan
05/24/2011 Summer School, IIIT HYderabad 4
Cohesive Relationships
Cohesive relationships between words and sentences have certain definable qualities that allow us to recognize the super sentence
Nature of cohesive relation Type of cohesion
Relatedness of form Substitution and ellipsis; Lexical collocation
Relatedness of reference Reference, Lexical reiteration
Semantic connection Conjunction
05/24/2011 Summer School, IIIT HYderabad 5
Relatedness of Form
Substitution: "Nice teapots! I'll take one.“
Ellipsis: "Turn on. Tune in. Drop out." ['you' is elided]
Collocation: "John went to the bank. He wanted to swim in the river." ['river' disambiguates 'bank']
05/24/2011 Summer School, IIIT HYderabad 6
Relatedness of Reference
Exophora: (extra linguistic feature: deitic markers , this that)
“what is this?”Anaphora:
"I used to have the key. But I lost it.”Cataphora:
"It is your turn, John”Reiteration: "He speaks only to the Huxleys; the Huxleys speak only
to the Darwins; and the Darwins speak only to God.“
05/24/2011 Summer School, IIIT HYderabad 7
Semantic Connection
Conjunction: "You tell me that you've got ev'rything you want, and your
bird can sing, but you don't get me, you don't get me!
You say you've seen seven wonders, and your bird is green, but you can't see me, you can't see me!
When your prized possessions, start to tear you down, then look in my direction, I'll be round, I'll be round."
[Beatles -- Lee Campbell]
05/24/2011 Summer School, IIIT HYderabad 8
Comparison
Halliday & Hassan also classify comparison as a form of cohesion
"She's more fun than a barrel of monkeys!"
"He's as tall as a six foot four inch tree."
05/24/2011 Summer School, IIIT HYderabad 9
CONTEXT DEPENDENCE
The interpretation of most expressions depends on the context in which they are used
Developing methods for interpreting context dependent expressions useful in many applications
We focus here on dependence of nominal expressions on context introduced LINGUISTICALLY,
for which we will use the term ANAPHORA
05/24/2011 Summer School, IIIT HYderabad 10
Introduction
What is Anaphora AntecedentAnaphora Resolution
1. Sabeer Bhatia arrived at Los Angeles International Airport at 6 p.m. on September 23, 1998. His flight from Bangalore had taken 22hrs and he was starving.
[RD, NOV 2000]
05/24/2011 Summer School, IIIT HYderabad 11
Etymology of Anaphora
ANA- Back, Upstream, Back upstream
Phora- Act of Carrying
Anaphora - Act of Carrying Back
05/24/2011 Summer School, IIIT HYderabad 12
What is Anaphora
Anaphora, in discourse, is a device for making an abbreviated reference (containing fewer bits of disambiguating information, rather than being lexically or phonetically shorter) to some entity (or entities) in the expectation that the receiver of the discourse will be able to disabbreviate the reference and, thereby, determine the identity of the entity.
(Hirst 1981)
05/24/2011 Summer School, IIIT HYderabad 13
Cataphora
When “anphora” precedes the antecedent
Because she was going to the departmental store, Mary was asked to pick up the vegetables.
05/24/2011 Summer School, IIIT HYderabad 14
Relevance from the Linguistics point of view
Binding Theory is one of the major results of the principles and parameters approach developed in Chomsky (1981) and is one of the mainstays of generative linguistics.
The Binding Theory deals with the relations between nominal expressions and possible antecedents.
It attempts to provide a structural account of the complementarity of distribution between pronouns, reflexives and R-expressions.
05/24/2011 Summer School, IIIT HYderabad 15
Dichotomy Between Linguistic and NLP
The Binding Theory (and its various formulations) deals only with intra-sentential anaphora,
A very small subset of the anaphoric phenomenon that practical NLP systems are interested in resolving.
A much larger set of anaphoric phenomenon is the resolution of pronouns inter-sententially.
This problem is dealt with by Discourse Representation Theory and more specifically by Centering Theory (Grosz et al., 1995)..
05/24/2011 Summer School, IIIT HYderabad 16
Types of Anaphors
The Prime Minister is yet to arrive and he is expected at the central hall at any time. [The Times of India, Feb 2001]
This book is about Anaphora Resolution. The book is designed to help beginners in the field and its author hopes that it will be useful.
VP Anaphor John screamed, as did Mary .
05/24/2011 Summer School, IIIT HYderabad 17
Pronominal anaphora Vajpayee hits back forcefully when he told the
opposition today “sometimes we fall prey to the media and sometimes you do. [Indian Express 2001]
Possessive Priyanka eats only chicken sandwiches
before going to take any exam; nothing else goes down her gullet that day.[Indian Express, 13 March 2001]
05/24/2011 Summer School, IIIT HYderabad 18
Reflexive Pronoun
Finally ,Danian heaved himself up and lay on a waiting stretcher.
Demonstrative PronounJohn had lots of packing to do before he shifted his
house. This was something he never liked….
Relative PronounStumper Sameer Dige, who made his test debut, failed
to show fast reflexives when it mattered.
05/24/2011 Summer School, IIIT HYderabad 19
Non Anaphoric Usage of Pronouns.
Pleonastic ItCognativea. It is believed that…..b. It appears that…..Modal Adjectivesc. It is dangerous……d. It is important…..Temporale. It is five o’clock f. It is winterWeather verbsg. It is rainingf. It is snowingDistanceh. How far it is to Chennai?
05/24/2011 Summer School, IIIT HYderabad 20
Non-anaphoric uses of pronounsHe that plants thorns must never expect to
gather roses.He who dares wins.
DeicticHe seems remarkably bright for a child of his
age.
05/24/2011 Summer School, IIIT HYderabad 21
Noun Phrase Anaphora
Definite descriptions and Proper names
Roy Kaene has warned Manchester United he may snub their pay deal. United’s skipper is even hinting that unless the future Old Trafford Package meets his demands, he could quit the club in June 2000. Irishman Keane, 27, still has 17 months to run on his current 23,000 pound a week contract and wants to commit himself to United for life. Alex Ferguson’s No 1 player confirmed: If it’s not the contract I want, I won’t sign”.
05/24/2011 Summer School, IIIT HYderabad 22
Coreference
Computational Linguists from many different countries attended the tutorial. The participants found it hard to cope with the speed of the presentation, nevertheless they manages to take extensive notes.
05/24/2011 Summer School, IIIT HYderabad 23
Coreference
Sophia Loren says she will always be grateful to Bono. The actress revealed that the U2 singer helped her calm down when she became scared by a thunderstorm while travelling by a plane.
She=> Sophia LorenThe actress=> Sophia LorenThe U2 Singer=> BonoHer=>Sophia LorenShe=>Sophia Loren
05/24/2011 Summer School, IIIT HYderabad 24
Coreference chain
Sophia Loren says she will always be grateful to Bono. The actress revealed that the U2 singer helped her calm down when she became scared by a thunderstorm while travelling by a plane.
Coreference chains {Sophia Loren, she, the actress, her, she} {Bono, the U2 singer}
05/24/2011 Summer School, IIIT HYderabad 25
Chains of object mentions in text
Toni Johnson pulls a tape measure across the front of what was once a stately Victorian home. A deep trench now runs along its north wall, exposed when the house lurched two feet off its foundation during last week's earthquake.
Once inside, she spends nearly four hours measuring and diagramming each room in the 80-year-old house, gathering enough information to estimate what it would cost to rebuild it.
While she works inside, a tenant returns with several friends to collect furniture and clothing.
One of the friends sweeps broken dishes and shattered glass from a countertop and starts to pack what can be salvaged from the kitchen.
(WSJ section of Penn Treebank corpus)
05/24/2011 Summer School, IIIT HYderabad 26
What is Anaphora Resolution
The Process of finding the antecedent for an Anaphor is Anaphora resolution
Anaphor-The reference that point to the previous item.
Antecedent-The entity to which the anaphor refers
05/24/2011 Summer School, IIIT HYderabad 27
RESEARCH ON ANAPHORA RESOLUTION: A QUICK SUMMARY
1970-1995Primarily theoreticalEmphasis: commonsense knowledge, salienceException: Hobbs 1977, Shalom Lappin
1995-2005First annotated corpora to be used to develop, evaluate and compare systems (MUC, Geand Charniak, ACE)First robust systems Heuristic-based: MitkovML: Vieira & Poesio 1998, 2000; Soon et al 2001, Ng and Cardie2002Emphasis: surface featuresExceptions: Poesio & Vieira, Harabagiu, Markert
2005-presentMore sophisticated ML techniques (global models, kernels)Richer features –especially semantic informationFirst tools
05/24/2011 Summer School, IIIT HYderabad 28
Application of Anaphora Resolution
Tasks that require determining the coherence of (segments of) textSegmentation
Post-hoc coherence check in summarization (Steinberger et al, 2007)
Tasks that require identifying the most important information ina text
Sentence selection in summarization (Steinberger et al 2005, 2007)Indexing
Information extraction: recognize which expressions refer to objects in the domain
Relation extraction from biomedical text (Sanchez-Grailletand Poesio, 2006, 2007)
Multimodal interfaces: recognize which objects in the visual scene are being referred to
05/24/2011 Summer School, IIIT HYderabad 29
Different Approaches In Anaphora Resolution
Rule Based Statistical Based Machine Learning Based
05/24/2011 Summer School, IIIT HYderabad 31
Hard Constraints on Coreference
Number agreement Person and case Gender Agreement Syntactic Agreement Selectional Restrictions
05/24/2011 Summer School, IIIT HYderabad 32
Number agreement
John and Mary loaned Sue a cup of coffee. Little did they know the magnitude of her addiction.
Singular Plural Unspecified
She,her, he, him, his, it
We, us, they, them
you
05/24/2011 Summer School, IIIT HYderabad 33
Person and Case Agreement
First Second
Third
Nominative I,we you he,she,they
Accusative me,us you Him,her,them
Genitive my,our
your His,her, their
05/24/2011 Summer School, IIIT HYderabad 34
Gender Agreement
*John has a coffee machine. She loves it.
05/24/2011 Summer School, IIIT HYderabad 35
Syntactic Agreement
Reflexives (himself, herself…) have strong constraints on what syntactic positions they can appear inJohn bought himself a cup of coffee.
*John bought him a cup of coffee.
05/24/2011 Summer School, IIIT HYderabad 36
Selectional Constraints
Jim bought a coffee from the store. He drank it quickly.
05/24/2011 Summer School, IIIT HYderabad 37
Also : Preferences
Recency Grammatical Role Repeated Mention Parallelism Verb Semantics
Based on Salience
05/24/2011 Summer School, IIIT HYderabad 38
Recency
John had a pop-tart. Bill had a jelly donut. Mary wanted it.
Recent Entities are more salient
05/24/2011 Summer School, IIIT HYderabad 39
Grammatical Role
“Sue bought a cup of coffee and a donut from Jane. She met John as she left.”
Entities in subject position are more salient
05/24/2011 Summer School, IIIT HYderabad 40
Repeated Mention
John went to the store to buy coffee. He loves coffee. He drinks 5 cups a day. At the store, Bill sold him a cup. He was delighed.
Entities mentioned more frequently are more salient
05/24/2011 Summer School, IIIT HYderabad 41
Parallelism
John bought coffee from Jim in the morning. Sue bought coffee from him in the evening.
Even with preferences to the contrary (grammatical role) the syntactic parallelism strongly prefers [him = Jim]
05/24/2011 Summer School, IIIT HYderabad 42
Verb Semantics
John telephoned Bill. He was jonesing for coffee.John criticized Bill. He was jonesing for coffee.
Perhaps salience of different elements in the sentence changes with respect to the verb used.
05/24/2011 Summer School, IIIT HYderabad 43
Algorithms --- How to integrate these preferences?
Constraints are easy to use : reject all hypothesis which violate the hard constraints
(if you can accurately detect the constraints!)
Preferences more difficult – how can one integrate these different preferences?
05/24/2011 Summer School, IIIT HYderabad 44
Hobbs Tree Search Algorithm
Given parse trees, search them in a specific order to find the most likely referent
05/24/2011 Summer School, IIIT HYderabad 45
Hobbs in Detail
1. Begin at NP
2. Go up tree to first NP or S. Call this X, and the path p.
3. Traverse all branches below X to the left of p. Propose as antecedent any NP that has a NP or S between it and X
4. If X is the highest S in the sentence, traverse the parse trees of the previous sentences in the order of recency. Traverse left-to-right, breadth first. When a NP is encountered, propose as antecedent. If not the highest node, go to step 5.
05/24/2011 Summer School, IIIT HYderabad 46
Hobbs cont.5. From node X, go up the tree to the first NP or S.
Call it X, and the path p.6. If X is an NP and the path to X did not pass through
the nominal that X dominates, propose X as antecedent
7. Traverse all branches below X to the right of the path, in a left-to-right, breadth first manner. Propose any NP encountered as the antecdent
8. If X is an S node, traverse all brnaches of X to the right of the path but do not go below any NP or S encountered. Propose any NP as the antecedent.
05/24/2011 Summer School, IIIT HYderabad 47
Lappin and Leass (1994) Anaphora Resolution Algorithm
The Lappin and Leass(1994) anaphora resolution algorithm uses
salience weight in determining the antecedent to the pronominals.
It requires as input a fully parsed sentence structure and
uses hierarchy in identifying the subject, object etc.
This algorithm uses syntactic criteria to rule out noun
phrases that cannot possibly corefer with it.
The antecedent is then chosen according to a ranking based
on salience weights.
05/24/2011 Summer School, IIIT HYderabad 48
The salience Factors and WeightsA pronoun P is non-coreferential with a (non-reflexive or non-
reciprocal) noun phrase N if any of the following conditions hold:
P and N have incompatible agreement features. P is in the argument domain of N. P is in the adjunct domain of N. P is an argument of a head
H, N is not a pronoun, and N is contained in H. P is in the NP domain of N. P is a determiner of a noun Q, and N is contained in Q.
05/24/2011 Summer School, IIIT HYderabad 49
Examples
Condition 1:The woman said that he is funny.
Condition 2:She likes her. John seems to want to see him.
Condition 3:She sat near her.
Condition 4:He believes that the man is amusing.This is the man he said John wrote about.
Condition 5:John’s portrait of him is interesting.
05/24/2011 Summer School, IIIT HYderabad 50
Salience Factors and Weights
Salience factor types with initial weightsFactor type Initial weightSentence recency 100Subject emphasis 80Existential emphasis 70Accusative emphasis 50Indirect object and oblique complement emphasis 40Head noun emphasis 80
Non-adverbial emphasis 50
05/24/2011 Summer School, IIIT HYderabad 51
Kennedy 1996The linguistic analysis for anaphora resolution includes
The output of a part of speech tagger,
Augmented with syntactic function annotations for each input token;
Using LINGSOFT
05/24/2011 Summer School, IIIT HYderabad 52
A set of patterns are used for identifying
The NP Chunking with position of the NP in the text: Nominal Sequencing in two subordinate syntactic
environments:a. in an adverbial adjunct b. in an NP (i.e. containment in a prepositional
or clausal complement of a noun, or containment in a relative clause)
Expletive “it”:
05/24/2011 Summer School, IIIT HYderabad 53
Anaphora Resolution
Uses Lappin and Lease algorithmSENT-S: 100 iff in the current sentenceCNTX-S: 50 iff in the current contextSUBJ-S: 80 iff GFUN = subjectEXST-S: 70 iff in an existential constructionPOSS-S: 65 iff GFUN = possessiveACC-S: 50 iff GFUN = direct objectDAT-S: 40 iff GFUN = indirect objectOBLQ-S: 30 iff the complement of a prepositionHEAD-S: 80 iff EMBED = NILARG-S: 50 iff ADJUNCT = NIL
05/24/2011 Summer School, IIIT HYderabad 54
Mitkov 1997
No Parsing of the Input Sentence
Boosting indicators
First Noun Phrases: A score of +1 is assigned to the first NP in a sentence.
Indicating Verbs: A score of +1 is assigned to those NPs immediately following a verb which is a member of a predefined set (including verbs such as discuss, present, illustrate, identify, summarise, examine, describe, define, show, check, develop, review,
05/24/2011 Summer School, IIIT HYderabad 55
MARS Cont….
Lexical Reiteration: A score of +2 is assigned to those NPs repeated twice or more in the paragraph in which the pronoun appears, a score of +1 is assigned to those NPs repeated once in that paragraph.
Section Heading Preference: A score of +1 is assigned to those NPs that also occur in the heading of the section in which the pronoun appears.
05/24/2011 Summer School, IIIT HYderabad 56
Boosting indicators contd..
Collocation Match: A score of +2 is assigned to those NPs that have an identical collocation pattern to the pronoun.
Immediate Reference: A score of +2 is assigned to those NPs appearing in constructions of the form
“… (You) V1 NP … con (you) V2 it (con (you) V3 it)”, where con Є {and/or/before/after…}.
Sequential Instructions: A score of +2 is applied to NPs in the NP1 position of constructions of the form: “To V1 NP1 V2 NP2. (Sentence). To V3 it, V4 NP4“ the noun phrase NP1 is the likely antecedent of the anaphor it (NP1 is assigned a score of 2).
Term Preference: A score of +1 is applied to those NPs identified as representing terms in the genre of the text.
05/24/2011 Summer School, IIIT HYderabad 57
Impeding indicators
Indefiniteness: Indefinite NPs are assigned a score of -1.
Prepositional Noun Phrases: NPs appearing in prepositional phrases are assigned a score of -1.
05/24/2011 Summer School, IIIT HYderabad 58
Indian Language
Types of anaphors Dravidian type with gender marked
pronouns avan, aval, atu (example) Telugu does not
have all other Dravidian languages have. Aryan Language Type without gender
marking in the pronouns “us” as in Hindi (example) FROM CHAPTER
05/24/2011 Summer School, IIIT HYderabad 59
<ANT 1,2,3> uutti </ANT> oru alakiya malai nakaram. Ooty(N) one(Q) beautiful(ADJ) hill(N) town(N). (Ooty is a beautiful hill station.)<ANA 1> ithu </ANA> malaikalin raani. This hill(N)+GEN queen(N). (This is Queen of hills.)<ANA 2> anku </ANA> puungkaa, pataku cavaari, malai rayil untu. There park(N), boating(VBN) hill train(N)
present(V+PRESENT+3S) (There are park, boating, hill train )<ANA 3> athu </ANA> oru cirantha currulaath thalam. it(PN) one(Q) best(ADJ) tourist(N) place(N). (It is a best tourist place.)
05/24/2011 Summer School, IIIT HYderabad 60
“Vasisth” a Rule Based Anaphora Resolution System
1. mo:han(i) avanRe(i) kuttiye kantu. mohan he-poss child-acc see-pst (Mohan saw his child.)2. mo:han(i) avanRe(i) kuttiye kantu ennu kRisnan paRannu. mohan he-poss child-acc see-pst compl krishnan say-pst (Krishnan said that Mohan saw his child.)3. *mo:han(i) avane(i) aticcu. mohan he-acc beat-pst (Mohan beat him.) 4. mo:han avane(i) aticcu ennu kRisnan(i) paRannu. mohan he-acc beat-pst compl krishnan say-pst (Krishnan said that Mohan beat him.)
05/24/2011 Summer School, IIIT HYderabad 61
The Algorithm for Intra-sentential Anaphora
A pronoun P is coreferential with an NP iff the following conditions hold:
a. P and NP have compatible P, N, G features. b. P does not precede NP. c. If P is possessive, then NP is the subject of the clause which contains P. d. If P is non-possessive, then NP is the subject
of the immediate clause which does not contain P.
05/24/2011 Summer School, IIIT HYderabad 62
Vasisth is a multilingual Anaphora Resolution system
Rule based With minimum Parsing Exploit the Morphology of Indian
Languages
05/24/2011 Summer School, IIIT HYderabad 63
“VASISTH” Using Salience Measure for Indian Languages
No In-depth Parsing
Exploit the Rich Morphology of the Language
The analysis depends on the salience weight of the candidate (NP) for the antecedent-hood of an anaphor from a list of probable candidates.
05/24/2011 Summer School, IIIT HYderabad 64
The salience weight assignment
a) The current sentence gets a score of 50 and it reduces by 10 for each preceding sentence till it reaches the fifth sentence. The system considers five sentences for identifying the antecedent.
b) The current clause gets a score of 75 if the pronoun present in the clause is a possessive pronoun and if it is a non-possessive pronoun it gets zero score.
c) The immediate clause gets the score 70 in the case of Possessive pronoun and gets a score of 75 for non-possessive pronouns.
d) For non-immediate clause, the possessive pronoun gets a score of 30 and non-possessive pronoun gets a score of 65.
05/24/2011 Summer School, IIIT HYderabad 65
e)The analysis showed that the subject could be the most probable antecedent for the pronoun. The case markings the subject of a sentence could take are nominative and dative.
A Nominative, a Dative and a Possessive NP with a nominative/Dative head could become a subject of a sentence.
05/24/2011 Summer School, IIIT HYderabad 66
f) The direct object of a sentence could be identified by the case markings and all the case markings other than the subject are considered for object. The next most probable NP for antecedent-hood is the direct object and hence it gets a score of 40.
g) The third NP in a clause, which is not identified as the subject or object, is considered as the indirect object and gets a low score of 30.
05/24/2011 Summer School, IIIT HYderabad 67
Salience factor weights for Indian Languages
Salience Factors Weights
Current sentence Possessive Current clauseImmediate clauseNon-immediate clauseNon-PossessiveCurrent clauseImmediate clauseNon-immediate clausePossessive and Non-PossessiveN.NomN.PossN.DatN.Acc, Loc, Instr…N.others(3rd NP)
50- Reduced by 10 for preceding sentences upto 5th sentence 75 7030 07565 8050504030
05/24/2011 Summer School, IIIT HYderabad 68
How it works
The salience weight to an NP is assigned in the following way
Identify the Pronoun
Consider Four sentences above the sentence containing the Pronoun
Consider all the NPs preceding the Pronoun ( This is the general rule)
05/24/2011 Summer School, IIIT HYderabad 69
Here we take some NPs which follow the the Pronoun since Tamil
All Indian languages are relatively free word Order
Assign Salience Weights.
The NP which gets the maximum salience weight and agrees in png with the anaphor is considered as the antecedent to the anaphor
05/24/2011 Summer School, IIIT HYderabad 71
Introduction
Goal of the task Automatically identify coreference chains in a document The coreference chains can include
Names Nominal mentions Pronouns verbs that are coreferenced with a noun phrases.
The data used is the Ontonotes English documents from Ontonotes This consists of five different types of genres
Newswire (NW), Broadcast News (BN), Telephonic Conversation (TC), Broadcast conversation (BC), Web blogs (WB)
05/24/2011 Summer School, IIIT HYderabad 72
Introduction
Coreferencing Two referential entities when they exist in the real
world Coreference analysis determines whether or not two
entities refer to the same entity Coreferents are classified into two types
Pronominal referents Non-pronominal referents
Pronominal referents are pronouns, which refer to other nouns in the text
Non-pronominal referents are names, nominal mentions and other noun phrases
05/24/2011 Summer School, IIIT HYderabad 73
A Sample Text
Eagle Clothes Inc. , which is operating under Chapter 11 of the Federal Bankruptcy Code , said it reached an agreement with its creditors .Under the accord , Albert Roth , chairman and chief executive officer , and Arthur Chase , Sam Beigel , and Louis Polsky will resign as officers and directors of the menswear retailer .Mr. Roth , who has been on leave from his posts , will be succeeded by Geoffrie D. Lurie of GDL Management Inc. , which is Eagle 's crisis manager .Mr. Lurie is currently co-chief executive.
05/24/2011 Summer School, IIIT HYderabad 74
Coreference Tagged Text
<Coref_chain Id="1">Eagle Clothes Inc.</Coref_chainId="1"> , which is operating under Chapter 11 of the federal Bankruptcy Code , said <Coref_chain Id="1">it</Coref_chain Id="1"> reached an agreement with <Coref_chain Id="1">its</Coref_chain Id="1"> creditors .Under the accord , <Coref_chain Id="2">Albert Roth , chairman and chief executive officer</Coref_chain Id=“2"> , and Arthur Chase , Sam Beigel , and Louis Polsky will resign as officers and directors of the <Coref_chain Id="1">menswear retailer</Coref_chain Id="1"> .<Coref_chain Id="2">Mr. Roth , who has been on leave from his posts ,</Coref_chain Id="2"> will be succeeded by <Coref_chain Id="3">Geoffrie D. Lurie of GDL Management Inc. , which is <Coref_chain Id="1">Eagle 's</Coref_chain Id="1"> crisis manager </Coref_chain Id="3">.<Coref_chain Id="3">Mr. Lurie </Coref_chain Id="3"> is currently co-chief executive .
05/24/2011 Summer School, IIIT HYderabad 75
Our Approach Two different approaches are used for the two
types of coreferents Mainly two modules
Pronominal resolution module Non-pronominal resolution module
Pronominal resolution refers to identification of a Noun phrase (NP) that is referred by a pronominal
Non-Pronominal resolution refers to identification of a NP referring to other NP.
05/24/2011 Summer School, IIIT HYderabad 76
Pronominal Resolution Module
Pronominal Resolution is done using refined salience measure.
All pronouns do not refer back to an entity For example in the sentence “It will rain today”
The pronoun “It” does not refer to any entity Such an instance of “It” is called as pleonastic “it”
Before we do pronominal resolution, we need to identify such non-anaphoric pronouns
05/24/2011 Summer School, IIIT HYderabad 77
Pronominal Resolution Module (Contd…)
The first step in the pronominal resolution module is identification of non-anaphoric pronouns and filter the non-anaphoric pronouns
from text The identification of non-anaphoric
pronouns is done using CRFs, a machine learning approach
05/24/2011 Summer School, IIIT HYderabad 78
Pronominal Resolution Module (Contd…)
Using CRFs we build a language model The features used for the training are
Word Part-of-speech (POS) tag
Window of five words Non-anaphoric pronoun filtering has
been observed to improve the results
05/24/2011 Summer School, IIIT HYderabad 79
Pronominal Resolution Module (Contd…)
The second step - core task of the pronoun resolution module the identification of antecedent for a pronoun
Use a refined salience measure based approach
WordNet and NE tag information are used to match the category of pronoun and the antecedent
05/24/2011 Summer School, IIIT HYderabad 80
Pronominal Resolution Module (Contd…) The table below shows the refined salience
factors and the weights assigned for themSalience Factors Weights
Current Sentence (sentence in which pronoun occurs) 100
For the preceding sentences up to four sentences from the current sentence Reduce sentence score by 10
Current Clause (clause in which pronoun occurs) 100 – for possessive pronoun50 – for non-possessive pronouns
Immediate Clause (clause preceding or following the current clause) 50 – for possessive pronoun100 – for non-possessive pronouns
Non-immediate Clause (neither the current or immediate clause) 50
Possessive NP 65
Existential NP 70
Subject 80
Direct Object 50
Indirect Object 40
Compliment of PP 30
05/24/2011 Summer School, IIIT HYderabad 81
Non-Pronominal Resolution Module
Use CRFs, a machine learning method
Preparing the Training data for CRFs learning
All positive pairs of NPs, and negative pairs of NPs are taken for training
Positive pairs are the NPs and the anaphor All NPs between an anaphor and antecedent are the
negative NPs Do not consider the NPs containing the pronouns, We consider the NP on the left side as antecedent NP
and NP on the right side as anaphor NP
05/24/2011 Summer School, IIIT HYderabad 82
Features – Non-pronominal Resolution Module
The features used for training are Distance feature
the possible values are 0,1,2,…. Calculated based on number of sentences between NPs
Definite NP If the antecedent NP is a definite NP, has value 1 else 0
Demonstrative NP If the antecedent NP is demonstrative, has value 1 else 0
String match the possible values are between 0 and 1 calculated as ratio of the number of words matched between
the NPs and the total number of words of the anaphor NP.
05/24/2011 Summer School, IIIT HYderabad 83
Features – Non-pronominal Resolution Module (Contd…)
Number Agreement use the gender data file (Bergsma and Lin, 2006) provided by CoNLL Also use POS information
Gender agreement use the gender data file (Bergsma and Lin, 2006) provided by CoNLL
Alias feature the possible values are 0 or 1. this is obtained using three methods
Comparing the head of the NPs, if both are same then scored as 1 If both the NPs start with NNP or NNPS POS tags, and if they are
same then scored as 1 Looks for Acronym match, if one is an acronym of other it is scored
as 1 Proper NP
If both NPs are proper NPs then value of 1 else 0 NE Tag Info
If NE tag present then value of 1 else 0
05/24/2011 Summer School, IIIT HYderabad 84
Non-Pronominal Resolution Module (Contd…)
The semantic class information (noun category) obtained from the WordNet is used for the filtering purpose. The pairs which do not have semantic
feature match are filtered out.
05/24/2011 Summer School, IIIT HYderabad 85
Complete Coreference Chain Building
The Coreferring pairs obtained from pronominal resolution system and Non-pronominal system are merged to generate the complete coreference chains.
The merging is done as follows: A member of a coreference pair is compared with all the
members of the coreference pairs identified if it occurs in anyone of the pair, then the two pairs are grouped this process is done for all the members of the identified pairs
and the members in each group are aligned based on their position in the document to form the chain
We show the sample outputs for building chain in the next few slides
05/24/2011 Summer School, IIIT HYderabad 86
Evaluation
The data used for training and testing is the English portion of the Ontonotes
The training and development data consisted of 1876 files from different genres viz., Newswire, Broadcast news, Broadcast conversation, Web blogs, magzine articles
The metrics used for evaluating the complete system are MUC B-Cubed CEAFE
05/24/2011 Summer School, IIIT HYderabad 87
Evaluation
We have performed evaluation of the Pronominal resolution module separately using the development data
We perform the evaluation of non-anaphor detection engine In the evaluation of pronominal resolution
module we studied how the non-anaphoric pronoun detection improves
05/24/2011 Summer School, IIIT HYderabad 88
Results – Pronominal Resolution Module
Type of pronoun
Actual (gold standard)
System identified Correctly
Accuracy (%)
Anaphoric Pronouns
939 908 96.6
Non-anaphoric pronouns
160 81 50.6
Total 1099 989 89.9
Evaluation of Non-anaphoric pronoun detection component
System type Total Anaphoric Pronouns
System identified pronouns
System correctly Resolved Pronouns
Prec (%)
Without non-anaphoric pronoun detection
939 1099 693 63.1
With non-anaphoric pronoun detection
939 987 693 70.2
Evaluation of Pronominal resolution module
05/24/2011 Summer School, IIIT HYderabad 89
Results– Complete System
Metric Mention Detection Coreference Resolution
Recall Precision F1 Recall Precision F1
MUC 68.1 61.5 64.6 52.1 49.9 50.9
B-CUBED 68.1 61.5 64.6 66.6 67.6 67.1
CEAFE 68.1 61.5 64.6 42.8 44.9 43.8
05/24/2011 Summer School, IIIT HYderabad 90
Results Discussion
On analysis of the output we found mainly three types of errors. They are Newly invented chains
The system identifies new chains that are not found in the gold standard annotation.
This reduces the precision of the system. This is because of the string match as one of
the features.
05/24/2011 Summer School, IIIT HYderabad 91
Results Discussion (Contd…)
Only head nouns in the chain system while selecting pair for identifying
coreference, the pair has only the head noun instead of the full phrase.
In the phrase “the letters sent in recent days”, the system identifies “the letters” instead of the whole phrase.
This affects both the precision and recall of the system.
05/24/2011 Summer School, IIIT HYderabad 92
Conclusion
Presented a coreference resolution system which combines the pronominal resolution using refined salience based approach with non-pronominal resolution using CRFs, machine learning approach.
Non-anaphoric pronouns identification improves the precision.
In non-pronominal resolution algorithm, the string match feature is an effective feature. But, this feature is found to introduce errors. We need to add additional contextual and semantic feature to
reduce above said errors. The results on the development set are encouraging.
05/24/2011 Summer School, IIIT HYderabad 93
Problems in Anaphora Resolution
Ambiguous Sentences We gave the bananas to the monkeys because
they were hungry. We gave the bananas to the monkeys because
they were ripe. We gave the bananas to the monkeys because
they were here. Complement anaphora
(1) Only a few of the children ate their ice-cream. They ate the strawberry flavour first.
(2) Only a few of the children ate their ice-cream. They threw it around the room instead
05/24/2011 Summer School, IIIT HYderabad 94
The Prime Minister of New Zealand visited us yesterday. The visit was the first time she had come to New York since 1998.
If the second sentence is quoted by itself, it is necessary to resolve the anaphor: The visit was the first time the Prime Minister of New
Zealand had come to New York since 1998. Although of course, as The Prime Minister of New Zealand
is an office of state and she would seem to refer to the person currently occupying that office, it could quite easily be that the Prime Minister of New Zealand had visited New York since 1998 and before the present day, whilst the present incumbent she had not.
05/24/2011 Summer School, IIIT HYderabad 95
Corpus Creation for ML Method
Annotation Guide lines PALINKA MUC GATE Give examples
05/24/2011 Summer School, IIIT HYderabad 96
Tools
GATE Java-RAP (pronouns) GUITAR (Poesio & Kabadjov, 2004;
Kabadjov, 2007) BART (Versleyet al, 2008)
05/24/2011 Summer School, IIIT HYderabad 97
TASKS and Conferences
CoNLL shared task Conference Discourse Anaphora and Anaphora
Resolution Colloquium: DAARC
05/24/2011 Summer School, IIIT HYderabad 98
Where it is required?
Machine Translation Information Extraction Summarization And in……….almost all NLU applications
05/24/2011 Summer School, IIIT HYderabad 99
References
Massimo Poesio Slides: “Anaphora resolution for Practical task”
Ruslan Mitkov: “MARS a Knowledge Poor anaphora resolution system”
MORE FROM OTHER SOURSES