advisor: hsin-his chen reporter: chi-hsin yu date: 2008.10.08 from acl 2008, regular paper...

FINDING CONTRADICTIONS IN TEXT

Advisor: Hsin-His ChenReporter: Chi-Hsin YuDate: 2008.10.08

From ACL 2008, regular paper

Marie-Catherine de Marneffe, Linguistics Department, Stanford University

Anna N. Rafferty and Christopher D. ManningComputer Science Department, Stanford University

OUTLINES

+ Introduction+ Related Work+ Typology ( 類型學 , the study of types) of

Contradictions+ System Overview+ Experiments+ Discussion and Conclusions

INTORDUCTION

+ Example

+ Applications of a contradiction detection system– Political candidate debates– Intelligence reports– Bioinformatics– Paper writing

Text Capital punishment is a catalyst ( 催化劑 ) for more crime.Hypothesis Capital punishment is a deterrent( 威嚇 ) to crime.

INTORDUCTION (CONT.)

+ Challenges – What is contradiction?

Typology of contradiction

– How do system detect the mismatch of information?

Coreference resolution Deeper understanding of text

Text Police specializing in explosives defused ( 去雷管 ) the rockets. Some 100 people were working inside the plant.

Hypothesis 100 people were injured.

Contradiction defused rockets cannot go off, and thus cannot injure anyone.

RELATED WORK

+ Text entailment– The PASCAL Recognizing Textual Entailment (RTE) Challenges

(Dagan et al., 2006; Bar-Haim et al., 2006; Giampiccolo et al., 2007) focused on textual inference in any domain.

– With probability and asymmetric inference direction– 在台灣很多人喜歡吃香蕉 香蕉很好吃 , 香蕉很便宜 – 在台灣很多人擁有自宅 X 房子很便宜 – 榴槤很好吃 X 在台灣很多人喜歡吃榴槤– 榴槤很好吃 在泰國很多人喜歡吃榴槤– 在台灣許多人喜歡吃榴槤 榴槤很好吃

RELATED WORK (CONT.)

+ Harabagiu et al. (2006) – providing the first empirical results for

contradiction detection, but they focus on specific kinds of contradiction

– accuracies of 75.63% on LCC negation and 62.55% on LCC paraphrase

WHAT IS CONTRADICTIONS?

+ A looser definition that more closely matches human intuitions is necessary.Text Sally sold a boat to John.

Hypothesis John sold a boat to Sally.

Contradiction tagged as contradictory even though it could be that each sold a boat to the other.

Text John thinks that he is incompetent.

Hypothesis His boss believes that John is not being given a chance.

Contradiction the targeted information in the two sentences is contradictory, even though the two sentences can be true simultaneously.

Text AIG计划出售亚洲子公司股权Hypothesis AIG 保留台灣壽險業務 南山人壽不賣了

WHAT IS CONTRADICTIONS? (CONT.)

+ For texts to be contradictory, they must involve the same event. Two phenomena must be considered in this determination: – implied coreference and embedded texts

–

– – –

Text Passions surrounding Germany’s final match turned violent when a woman stabbed her partner because she didn’t want to watch the game.

Hypothesis A woman passionately wanted to watch the game.

Remark the “woman” in two sentences is the same.

Text Eyewitnesses said de Menezes had jumped over the turnstile ( 十字轉門 ) at Stockwell subway station.

Hypothesis The documents leaked to ITV News suggest that Menezes walked casually into the subway station.

Remark Refer to the same event (de Menezes in a subway station),and display incompatible views of this event

TYPOLOGY OF CONTRADICTIONS

+ Two primary categories of contradiction: – Easy: antonym, negation, and date/number

mismatch– Hard: factive, modal words, structural and subtle

lexical contrasts, world knowledge (WK)

TYPOLOGY OF CONTRADICTIONS (CONT.)

CONTRADICTION CORPORA

+ Dataset– Annotating the RTE datasets for contradiction

RTE datasets are balanced between entailments and non-entailments.

– Containing pairs consisting of a short text and a one sentence hypothesis

CONTRADICTION CORPORA

131 contradiction pairs from news31% 55%

SYSTEM OVERVIEW

+ Our system– based on the stage architecture of the Stanford RTE system

(MacCartney et al., 2006), but adds a stage for event coreference decision

– apply logistic regression to classify the pair as contradictory or not.

The feature weights are hand-set, guided by linguistic intuition.

+ Added Steps– Linguistic analysis– Alignment between graphs– Filtering non-coreferent events– Extraction of contradiction features

FEATURES FOR CONTRADICTION DETECTION

+ Polarity features+ Number, date and time features+ Antonym features+ Structural features+ Factivity features+ Modality features+ Relational features

RESULTS

RESULTS (CONT.)

DISCUSSION AND CONCLUSIONS

+ Discussion and Conclusions– Contradiction detection is lack of feature

generalization.– Many contradictions in “hard” category require

multiple inferences.– Mismatching information is not sufficient to

indicate contradiction.Text Nike Inc. said that its profit grew 32 percent, as the company

posted broad gains in sales and orders.Hypothesis Nike said orders for footwear totaled $4.9 billion, including a

12 percent increase in U.S. orders.

THANKS!!