Multilingual Connotation Frames: A Case Study on Social Media for Targeted Sentiment Analysis and ForecastHannah Rashkin, Eric Bell, Yejin Choi, and Svitlana Volkova

ABOUTPacific NorthwestNational LaboratoryThe Pacific Northwest National Laboratory,located in southeastern Washington State,is a U.S. Department of Energy Office ofScience laboratory that solves complexproblems in energy, national security, andthe environment, and advances scientificfrontiers in the chemical, biological, materials,environmental, and computational sciences.The Laboratory employs nearly 5,000 staffmembers, has an annual budget in excessof $1 billion, and has been managed byOhio-based Battelle since 1965.

For more information on the scienceyou see here, please contact:

Svitlana VolkovaPacific Northwest National LaboratoryP.O. Box 999Richland, WA 99352(509) 372-6585svitlana.volkova@pnnl.govhttps://www.cs.jhu.edu/~svitlana/

File

Nam

e//

File

Dat

e //

PNN

L-SA

-###

##

Brussels attacks+aftermath

News story about Clinton “killing” minesArrest of

terror suspect

Abdeslam

terror suspect accepts arrest and extradition

most negative about terror

suspect

Stories in English tweets from Russia about how Apple had “assaulted” FBI by refusing to

help in their investigations

Similar to the English tweets from Russia, the Russian

tweets about Apple are also less positive than towards most of the other entities.

+

-

-

-

-

-+

English Verb: surviveOther languages: survivre, sobrevivir, überleben…

“L'incroyable miraculé des explosions à Brussels: ce Mormon avait déjà survécu aux attentats de Boston et de Paris”

“Este joven ha sobrevivido a los atentados de Boston, de París y de Bruselas”

“US teenager … also survived Boston Marathon bombing”

“19-jähriger Missionar überlebt drei Terroranschläge”

Example Tweets

Writer

=Reader

Agent Theme

P(w!agent) P

(w!them

e)P(agent ! theme)

theme is some type

of hardship

agent is sympathetic

victim ———

Connotation Frame for surviving verbs:

+

+

—

—

—

1. Motivation 3. Multilingual Connotation Frames 5. Multidimensional Analysis

1. Extract multilingual tweets about news events

2. Extract agent-verb-theme from tweet

3. Aggregate connotation

What do people around the world say aboutcurrent events?

• Extracting via word-aligned translations for 10 languages (including low-resource)

US teenager … also survived Boston Marathon bombing

US teenager … also survived Boston Marathon bombing

US teenager … also survivedBoston Marathon bombing

Other tweets about bombing

Sentiment Distribution:

Expected sentiment:

- +/=/-

• Use frame F(v*) from most-likely English word alignment v*

4. Forecasting Opinion Dynamics

2. Aggregating Sentiment010

.3

.4

.3

.3

.6

.1

.3

.4

.3

LSTM

Fully connected

softmax layer

Input distribution

vectors

+=—

Mar

21

(UK →

Bru

ssel

s)

Mar

22

(UK →

Bru

ssel

s)

Mar

23

(UK →

Bru

ssel

s)

Mar

24

(UK →

Bru

ssel

s)

Predicted Distrib: Mar 25 (UK → Brussels)

}}

• LSTMs are good at predicting trend but not sudden spikes

• Multilingual extension to connotation frame lexicon from Rashkin et al. 2016 in 10 languages

• Tweets about news-worthy events for analysis of public sentiment across time, locations, languages

Conn. frame (survive)

• Predict tomorrow’s targeted sentiment distribution?

• Use aggregated sentiment per country towards entities

English Spanish French Dutch Russian German

Portuguese Italian Finnish Swedish Polish

0

2

4

6

NL PT IT FI ES EN DE FR PL RU SV

KL D

iver

genc

e

Forecast 1 day Forecast 4 days

Day

KL Divergences:• LSTM is best performing on average• Predicting 1 day ahead gives better performance

3.41

2.00

3.26

1.69

0 2 4

1 day

4 days

KL Divergence

LSTM SVM