Sarcasm And ThwartingLekha

Deepali GuptaSagar Ahire

{lekha, gdeepali, sagarahire} @ cse.iitb.ac.in

Roadmap

Irony and Sarcasm

An Algorithm for Sarcasm Detection

Thwarting Detection

Irony and SarcasmLekha | 133050002

Verbal Irony

“An irony is a figure of speech which implicitly displays that the utterance situation was surrounded

by an ironic environment.”

There also exists Situational Irony

Ironic Environment

t0 t1

Expectation Expectation fails

Negative emotional attitude of the speaker

Reasons for Expectation to Fail

Expectation E is caused by an action A1. E failed because A failed or cannot be

performed because of another action B2. E failed because A was not performed

Expectation E is not caused by any action3. E failed by an action B4. E accidentally failed

Type 1 and 3 have victims

Sarcasm is irony with definite victims and counterfeited emotions

Properties of an Ironic Environment

An utterance implicitly displays all the three conditions for ironic environment when it: 1. Alludes to the speaker's expectation E2. Intentionally violates one of pragmatic principles3. Implies the speaker's emotional attitude toward the

failure of E

Irony is recognized if any 2 of these 3 are recognized.

Irony conveys the third unidentified property.

Allude to Speaker’s Expectation

Deepali baked a pizza to satisfy her hunger. She placed the pizza on the table and in the meantime Sagar came and gobbled up the whole pizza. Deepali said to Sagar:

a. I'm not hungry at all b. Have you seen my pizza on the table?c. I'll sleep well on a full stomach.d. I'm really satisfied to eat the pizza.e. Did you enjoy eating the pizza?

Violation of Pragmatic Principles

SincerityYou make a statement you believeYou ask a question whose answer you don’t knowYou offer advice which will benefit the receiverYou thank when you are really grateful

Propositional content You thank for something that has been done for you

Preparatory condition for Offer You offer something that you can really give

Maxim of relevance Politeness principle Maxim of quantity

Emotional Attitude

Tone and expressions

Interjections “Oh! The weather is so nice”

The context implies the emotional attitude of the speaker

Semi Supervised Recognition of Sarcastic

Sentences

Deepali Gupta | 13305R001

Sarcasm

The activity of saying or writing the opposite of what you mean, or of speaking in a way intended to make someone feel stupid or show that you are angry (Macmillan English Dictionary)

Sarcasm manifests in other ways...

● “Love the cover” (book)● “Be sure to save your purchase receipt”

(Smart Phone)● “Great idea, now try again with a real

product development team” (e-reader)● “Where am I?” (GPS device)

The Algorithm: Overview

1. Training Set: Sentences manually assigned scores 1 to 5 where five means clearly sarcastic and one absence of sarcasm

2. Create feature vectors from the labelled sentences

3. Use these feature vectors to build a model and assign scores to unlabelled examples

Step 1: Preprocessing of Data

1. Replace each appearance of a product/company/author by generalized [product], [company], [author], etc.

2. Remove all HTML tags and special symbols from review text.

Step 2: Creating Feature VectorsPattern Based Features:1. Classify words into High Frequency Words (HFWs) and

Content Words (CWs)All [product], [company] tags and punctuation marks are HFWs.

2. A pattern is a sequence of HFWs with slots for CWs.Example: “Garmin does not care about product quality or customer support” has patterns “[company] does not CW about CW CW” or “about CW CW or CW CW”, etc.

Pattern Matching

1: Exact Match

: Sparse Match - additional non-matching words can be inserted between pattern components

: Incomplete Match - only n of N pattern components appear in sentence, while some non-matching words can be inserted in between

Punctuation Based Features

● Sentence length in words● Number of “!” characters● Number of “?” characters● Number of quotes● Number of capitalized/all capital words

Features are normalized to be in [0-1] by dividing them by maximal observed value

Step 3: Data Enrichment

● For each sentence in the training set perform a search engine query containing this sentence

● Assign similar label to newly extracted sentence.

Step 4: Classification● Construct feature vectors for each sentence in the

training and test set● Compute Euclidean Distance to each of matching

vectors in training set

Let ti i=1..k be the k vectors with lowest Euclidean Distance to v.Then v is classified label l as follows:Count(l) = Count of vectors in the training set with label l

Label(v) =

Star Sentiment Baseline

● From a set of negative reviews (with 1-3 stars) classify those sentences as sarcastic with strong positive sentiment.

● Positive sentiment words can be eg. “great”, “best”, “top”, etc.

Results Precision Recall Accuracy F-Score

punctuation 0.256 0.312 0.821 0.281

patterns 0.743 0.788 0.943 0.765

pat+punct 0.868 0.763 0.945 0.812

enrich punct 0.4 0.390 0.832 0.395

enrich pat 0.762 0.777 0.937 0.769

all: SASI 0.912 0.756 0.947 0.827

Precision

Recall False Pos

False Neg

F-score

Baseline 0.5 0.16 0.05 0.44 0.242

SASI 0.766 0.813 0.11 0.12 0.788

Table 1:(Above): 5-fold cross validation results

Table 2 (Below): Comparison with baseline

Thwarting DetectionSagar Ahire | 133050073

“The actors were good, the story was great, the screenplay was a marvel of perfection and the music was good too, but the movie couldn’t hold my attention...”

Thwarting?

Detecting Thwarting: The Big Picture

● Ascertain attributes of entity using ontology● Find sentiment of each attribute in ontology

and the overall entity● If there is a contrast, conclude thwarting has

occured

Building the Domain Ontology

1. Identify key features of domain from a corpus

2. Arrange them in a hierarchy

Notes:● Very human-intensive● One-time requirement

An Example Ontology

Movie

Story Elements Acting Music

Main Story Dialogues Characters Songs Background Score

Approaches to Detect Thwarting

● Rule-based Approach● Machine Learning-based Approach

Rule-based Approach

1. Get dependency parse for adjective-noun dependencies

2. Identify polarities towards all nouns3. Tag corresponding ontology nodes with

found polarities4. If a contradiction across levels is found,

conclude that thwarting has taken place

Rule-based Approach: ExampleMovie

negative

Story Elementspositive

Actingpositive

Musicpositive

Main Storypositive

Dialoguespositive

Characterspositive

Songspositive

Background Score

negative

Machine Learning-based Approach

Proceeds in two phases:1. Learning weights2. Classifying documents

Learning Weights: Choices

1. Choices for loss function:a. Linear lossb. Hinge loss

2. Choices for percolation across ontology levels:a. Complete percolationb. Controlled percolation

Classification: Features

● Convert document into a feature vector.● Examples:

○ Document polarity○ No of flips of sign○ Longest contiguous subsequence of +ve values○ Longest contiguous subsequence of -ve values○ etc.

Results

● Random baseline: 50%● Rule-based approach: 56.3%● ML-based approach (Linear loss, controlled

percolation): 81%

What’s the catch?

Requires sentiment as input!

Current System

Document withSentiment Information Thwarted or Not Thwarted

Ideal System

Thwarted or Not Thwarted,Document SentimentDocument

Key Ideas● Irony indicates presence of an ironic environment,

with 3 properties ● 2 of those 3 are enough to recognize irony● Sarcasm is irony with victims and counterfeited

emotions● A semi supervised pattern based algorithm detects

sarcasm well● Thwarting is the phenomenon of polarity reversal at

a higher level of ontology compared to the polarity expressed at the lower level

● Rule based and machine learning based approaches have been attempted for thwarting

References● Akira Utsumi (1996) - A unified theory of irony and its

computational formalization. InCOLING, 962–967.

● Oren Tsur, Dmitry Davidov, Ari Rappoport (2010) - ICWSM – A Great Catchy Name: Semi-Supervised Recognition of Sarcastic Sentences in Online Product Reviews. In Association for the advancement of Artificial Intelligence

● Ankit Ramteke, Akshat Malu, Pushpak Bhattacharyya, J. Saketha Nath (2013) - Detecting Turnarounds in Sentiment Analysis: Thwarting. In ACL 2013.

Questions?