valitutti acii2015 talk
TRANSCRIPT
Inducing an Ironic Effect in Automated Tweets
Alessandro Valitu7 and Tony Veale University College Dublin
URL: www.whim-‐project.eu/whaDfmachine • The aim of WHIM project is the development of a "What-‐if machine", i.e. an AI program capable to produce new and unexpected fic'onal ideas.
• FicDonal ideaDon can be used to inspire people (writers, children, or copywriters) to write stories.
Examples of what-‐ifs: • What if there was a liPle dog who was afraid of bones? • What if beloved angels were to lose their purity, train in combat and become feared commandos?
ComputaDonal CreaDon
Genera'on of artefacts (e.g. poems, painDngs, songs, stories, newspaper headlines, etc.)
1. Idea'on 2. Rendering
IdeaDon = Idea GeneraDon
Objec've Idea Subjec've Point of View (opinions, emo'ons, personality traits, etc.)
Irony
Irony DefiniDon The term "irony" can refer to different concepts: • Situa'onal irony: situaDon characterized by contrast between
reality and human ideals or intenDons (also called "irony of the fate").
• Verbal irony: is a rhetorical device in which the intended meaning of statements is different from (and typically opposite of) the literal meaning. What is said is opposite of what is meant.
• If one looks out of his window at a rain storm and remarks to a friend, "Glorious day, isn't it?" the contradicDon between the facts and the implied descripDon is a form of verbal irony.
• SemanDc contrast can be employed in both types of irony. We focus on verbal irony.
• Polarity-‐based (verbal) irony: The polarity of what is said is opposite to the polarity of what is meant. ["glorious" -‐> "miserable”]
Irony
Conceptual CreaDvity LinguisDc CreaDvity
SituaDonal Irony Verbal Irony
Polarity opposiDon as a semanDc device for both conceptual and linguisDc creaDvity
Research QuesDons
• To what extent is it possible to generate verbal irony automaDcally?
• How can we evaluate verbal irony of a computer-‐generated text using a crowdsourcing system?
• To what extent the text meant to be ironic is really recognized as ironic?
@MetaphorMagnet (1) • It is a TwiPerbot @MetaphorMagnet, which uses a store of knowledge to create tweets that are meant to be meaningful (Veale 2014).
• The system can generate a rich range of creaDve statements, which are regularly posted on TwiPer.
Example: Remember when peace was encouraged by nonviolent peacemakers? Now, peace is a victory enjoyed only by conquering victors.
@MetaphorMagnet (2) A subsets of the tweet paPerns are designed to be intenDonally ironic. #Irony: When some sovereigns live in "magnificent" palaces the way rappers live in wretched ghePos. #Sovereign=#Rapper #Palace=#GhePo #Irony: Thinkers formulaDng promising ideas about fuDle fantasy. #Promising=#FuDle #IdeaAboutFantasy #Irony: The most dignified statesman is not more celebrated than the most unprofessional blogger. #Statesman=#Blogger
@MetaphorMagnet (3)
Linguis'c elements: • focus word • contrasDve comparison • scare quotes • #irony hashtag
Are ironic tweets really “ironic”? To what extent the tweets “meant to be ironic” are “recognized as ironic”?
Research QuesDon
Irony Factors, Irony Markers, Echo
There are three main building blocks: 1. SemanDc Content: Irony Factors 2. PragmaDc cues: Irony Markers 3. Background knowledge: Echoed Informa'on
#Irony: Thinkers formulaDng promising ideas about fuDle fantasy. #Promising=#FuDle #IdeaAboutFantasy
Irony Factors, Irony Markers, Echo
There are three main building blocks: 1. SemanDc Content: Irony Factors 2. PragmaDc cues: Irony Markers 3. Background knowledge: Echoed Informa'on
#Irony: Thinkers formulaDng promising ideas about fuDle fantasy.
Irony Factors, Irony Markers, Echo
There are three main building blocks: 1. SemanDc Content: Irony Factors 2. PragmaDc cues: Irony Markers 3. Background knowledge: Echoed Informa'on
#Irony: Thinkers formulaDng promising ideas about fuDle fantasy.
Irony Factors, Irony Markers, Echo
There are three main building blocks: 1. SemanDc Content: Irony Factors 2. PragmaDc cues: Irony Markers 3. Background knowledge: Echoed Informa'on
#Irony: Thinkers formulaDng promising ideas about fuDle fantasy.
Irony Factors, Irony Markers, Echo
There are three main building blocks: 1. SemanDc Content: Irony Factors 2. PragmaDc cues: Irony Markers 3. Background knowledge: Echoed Informa'on
#Irony: Thinkers formulaDng promising ideas about fuDle fantasy. (E.g., aker a poliDcal speech)
Ironic Similes Example 1: • X is as useful as a chocolate teapot. • Literal meaning: chocolat teapots are useful. • Non-‐literal (ironic) meaning: chocolat teapots are useless. Example 2: • The findings proposed in this paper are useful...as a
chocolate teapoint. Procedure: 1. Input property: "cold” 2. Antonym: "hot” 3. EnDty having the input property as typical: ice 4. Simile composiDon: "as hot as ice"
We are pleased to inform you that your paper (??) Has been “accepted” (...) for full publicaDon and ORAL presentaDon during the 6th InternaDonal Conference on AffecDve CompuDng and Intelligent InteracDon (ACII 2015). #irony #yeahright #Idonthinkso #youfellforit
Ironic Paper NoDficaDon
EvaluaDon of AutomaDc (Verbal) Irony • Use of @MetaphorMagnet as a test bed for the generaDon of randomized samples.
• Each experimental se7ng corresponds to different combinaDons of linguisDc features.
• Therefore, we can use the system to study the contribuDon of specific linguis'c features to the percepDon of irony.
• On the other hand, if the linguisDc features can be controlled by genera've parameters, the evaluaDon is not only about the irony of texts but also about the system capability to generate ironic texts.
• Example of linguisDc properDes that cannot be controlled by generaDve parameters: senDment of the whole sentence.
Issue about Men'oning Irony in the Evalua'on
Two-‐move Strategy: 1. Ask people to judge irony and valence and evaluate
their correlaDon 2. Use valence shiking as operaDonal measure of irony
• You want to know if a sentence is ironic. • You ask people if it is ironic. • You pay people to judge if it is ironic or not. • Now, you show a sentence starDng with #irony.
Would you trust the answer?
Experiment 1
Format 1 Kindergartens are educa?ng skinny kids about fat babies.
Format 2 The thief that fences vibrant jewels is disguised with lifeless masks.
Format 3 Some astrologers study “beloved” stars the way entomologists study ugly spiders.
Experiment 1 Condi'on Irony Surprise Humor Retweet
Format 1 2.43 2.69 2.36 1.56
Format 2 2.53 2.76 2.45 1.59
Format 3 2.69 2.80 2.62 1.66
NoAdj 2.51 2.71 2.46 1.59
Adj 2.59 2.78 2.50 1.62
Examples in Each Se7ng (Experiments 2 and 3)
BASE The vegetables are mixed in healthful salads.
QUOT The vegetables that are mixed in “healthful” salads.
HASH #Irony: The vegetables are mixed in healthful salads.
QUOT+COMP The vegetables that are mixed in “healthful” salads are treated with poisonous pes?cides.
QUOT+HASH #Irony: The vegetables that are mixed in “healthful” salads are treated with poisonous pes?cides.
Valence and Polarity Rate (Experiment 2)
SeWng Valence Polarity Rate
BASE 0.51 ± 0.38 0.91 ± 0.15
QUOT 0.41 ± 0.46 0.82 ± 0.13
COMP 0.29 ± 0.49 0.75 ± 0.15
QUOT+COMP 0.20 ± 0.54 0.64 ± 0.16
Valence Shiking -‐ Experiment 2 (P < 0.001)
SeWng QUOT COMP QUOT+COMP
BASE -‐0.10 -‐0.22 -‐0.31
QUOT — -‐0.12 -‐0.21
COMP — — -‐0.09
SeWng QUOT COMP QUOT+COMP
BASE -‐0.10 p < 0.001
-‐0.22 p < 0.001
-‐0.31 p < 0.001
QUOT — -‐0.12 p < 0.005
-‐0.21 p < 0.001
COMP — — -‐0.09 p < 0.001
Shallow Polarity Inversion -‐ Experiment 2
Valence and Polarity Rate (Experiment 3)
SeWng Valence Polarity Rate
COMP 0.11 ± 0.60 0.52 ± 0.29
QUOT -‐0.07 ± 0.60 0.35 ± 0.26
HASH 0.06 ± 0.59 0.50 ± 0.27
QUOT+HASH -‐0.05 ± 0.61 0.35 ± 0.25
Valence Shiking – Experiment 3
SeWng QUOT HASH QUOT+HASH
COMP -‐ 0.18 p < 0.001
-‐0.05 p > 0.5
-‐0.16 p < 0.001
SeWng QUOT HASH QUOT+HASH
COMP -‐0.17 p < 0.001
-‐0.02 p > 0.5
-‐0.17 p < 0.001
Shallow Polarity Inversion – Experiment 3
#Irony hashtag is not significantly effecDve!
Conclusive Remarks • Addressed Issue: difficulty of evaluaDng new tasks and subtle effects without a gold-‐standard textual corpus.
• Open issue: How can we employ this method to increase the size of the effect?
• DisDncDon between “meant effect” (e.g. polarity opposi?on) and “perceived effect” (e.g. valence shiIing)
• Valence shiking as an indicator of irony percepDon and its use in the indirect evaluaDon of irony
Possible ApplicaDons
• ComputaDonal Persuasion: context-‐based slanDng
• SenDment Analysis • InteracDve Systems (in parDcular, conversaDonal agents): interacDve irony + nonverbal irony markers + valence detecDon
Future Work
Extended experiments: – PosiDve valence shiking on posi've focus words – Role of features highligh'ng the focus word (scare quotes?)
– Study how to increase the size of the effect
References From the paper: • Y.Hao and T.Veale (2010). An ironic fist in a velvet glove:
CreaDve Mis-‐RepresentaDon in the ConstrucDon of Ironic Similes”. Minds and Machines 20(4).
• T.Veale (2014). A Service-‐Oriented Architecture for Metaphor Processing. Proceedings of the Second Workshop on Metaphor in NLP, at ACL 2014.
AddiDonal reference about the evaluaDon methodology (valence ⟺ humor appreciaDon): • A.Valitu7, A.Doucet, J.M.Toivanen, and H.Toivonen (2015).
ComputaDonal GeneraDon and DissecDon of Lexical Replacement Humor. Natural Language Engineering, April 2015.