exploring demographic language variations to …svitlana/posters/emnlp13-slides.pdfexploring...

Exploring Demographic Language Variations toImprove Multilingual Sentiment Analysis

in Social Media

Svitlana Volkova1, Theresa Wilson2 and David Yarowsky1,2,

1Center for Language and Speech Processing, Johns Hopkins University2Human-Language technology Center of Excellence

Motivation

Demographic language variations (DLV) have been studied bysocio-linguists for decades (Picard, 1997; Gefen & Ridings, 2005;Holmes & Meyerhoff, 2004; Macaulay, 2006; Tagliamonte, 2006).

DLV have been recently explored in personal email communication,blog posts, and public discussions (Boneva et al., 2001; Mohammad &Yang, 2011; Eisenstein et al., 2010; Bamman et al., 2012)

We propose to study differences in subjective language in socialmedia to support commercial applications:

personalized recommendation systems and targeted onlineadvertising (Fan & Chang, 2009),detecting helpful product reviews (Ott, Choi, Cardie, & Hancock,2011),tracking sentiment in real time (Resnik, 2013),large-scale, low-cost, passive polling (O’Connor, Eisenstein, Xing,& Smith, 2010).

S. Volkova, T. Wilson, D. Yarowsky (JHU) Demographic Language Variations in Twitter 2 / 35

Motivation

personalized recommendation systems and targeted onlineadvertising (Fan & Chang, 2009),detecting helpful product reviews (Ott et al., 2011),tracking sentiment in real time (Resnik, 2013),large-scale, low-cost, passive polling (O’Connor et al., 2010).

Motivation

personalized recommendation systems and targeted onlineadvertising (Fan & Chang, 2009),

detecting helpful product reviews (Ott et al., 2011),tracking sentiment in real time (Resnik, 2013),large-scale, low-cost, passive polling (O’Connor et al., 2010).

Motivation

personalized recommendation systems and targeted onlineadvertising (Fan & Chang, 2009),detecting helpful product reviews (Ott et al., 2011),

tracking sentiment in real time (Resnik, 2013),large-scale, low-cost, passive polling (O’Connor et al., 2010).

Motivation

personalized recommendation systems and targeted onlineadvertising (Fan & Chang, 2009),detecting helpful product reviews (Ott et al., 2011),tracking sentiment in real time (Resnik, 2013),

large-scale, low-cost, passive polling (O’Connor et al., 2010).

Motivation

Male ♂ and Female ♀ Twitter users use subjective terms differently:♀+ “Chocolate is my weakness”

♂− “Clearly they know our weakness. Argggg....”

Motivation

Male ♂ and Female ♀ Twitter users use subjective terms differently:♀+ “Chocolate is my weakness”♂− “Clearly they know our weakness. Argggg....”

Motivation

I. Explore gender bias in the use of subjective language in Twitter:

investigate multilingual subjective lexical variations;cross-cultural emoticon and hashtag usage.

II. Incorporate gender bias into models to improve sentimentanalysis for English, Spanish, and Russian:

demonstrate that simple, binary features representing authorgender are insufficient for gender-dependent sentiment analysis.

I. Explore gender bias in the use of subjective language in Twitter:investigate multilingual subjective lexical variations;

cross-cultural emoticon and hashtag usage.

I. Explore gender bias in the use of subjective language in Twitter:investigate multilingual subjective lexical variations;cross-cultural emoticon and hashtag usage.

Automatic gender label prediction using user first namemorphology (precision is above 0.98 across languages).

Sentiment labels from Mechanical Turk (5 annotations per tweet):

Positive: Как же приятно просто лечь в постель после тяжелогодня... (It is a great pleasure to go to bed after a long day at work...)Negative: Уважаемый господин Прохоров купите эти выборы!(Dear Mr. Prokhorov just buy the elections!)Both: Затолкали меня на местном рынке! но зато закупиласьподарками для всей семьи :) (It was crowded at the local market!But I got presents for my family:-))Neutral: Киев очень старый город (Kiev is a very old city).

Positive: Как же приятно просто лечь в постель после тяжелогодня... (It is a great pleasure to go to bed after a long day at work...)

Negative: Уважаемый господин Прохоров купите эти выборы!(Dear Mr. Prokhorov just buy the elections!)Both: Затолкали меня на местном рынке! но зато закупиласьподарками для всей семьи :) (It was crowded at the local market!But I got presents for my family:-))Neutral: Киев очень старый город (Kiev is a very old city).

Positive: Как же приятно просто лечь в постель после тяжелогодня... (It is a great pleasure to go to bed after a long day at work...)Negative: Уважаемый господин Прохоров купите эти выборы!(Dear Mr. Prokhorov just buy the elections!)

Both: Затолкали меня на местном рынке! но зато закупиласьподарками для всей семьи :) (It was crowded at the local market!But I got presents for my family:-))Neutral: Киев очень старый город (Kiev is a very old city).

Positive: Как же приятно просто лечь в постель после тяжелогодня... (It is a great pleasure to go to bed after a long day at work...)Negative: Уважаемый господин Прохоров купите эти выборы!(Dear Mr. Prokhorov just buy the elections!)Both: Затолкали меня на местном рынке! но зато закупиласьподарками для всей семьи :) (It was crowded at the local market!But I got presents for my family:-))

Neutral: Киев очень старый город (Kiev is a very old city).

Metrics Lexical Evaluation across Genders

Term ti subjectivity:

pti (subj |g) =c(ti ,P,g) + c(ti ,N,g)

c(ti ,g),

Term ti polarity:

pti (+|g) =c(ti ,P,g)

c(ti ,P,g) + c(ti ,N,g),

Polarity change across genders:

∆p+ti = |pti (+|F )− pti (+|M)|

s.t .∣∣∣∣∣1− tf subjti (F )

tf subjti (M)

∣∣∣∣∣ ≤ λ, tf subjti (M) 6= 0,

λ controls term frequency similarity.

c(ti ,g),

Term ti polarity:

∆p+ti = |pti (+|F )− pti (+|M)|

tf subjti (M)

∣∣∣∣∣ ≤ λ, tf subjti (M) 6= 0,

c(ti ,g),

Term ti polarity:

∆p+ti = |pti (+|F )− pti (+|M)|

tf subjti (M)

∣∣∣∣∣ ≤ λ, tf subjti (M) 6= 0,

Lexical Evaluation across Genders for English

Terms: 3 - from LI , 4 - bootstrapped lexicon LB, and � - hashtags

Lexical Evaluation for Spanish and Russian

Spanish:fiasco, triunfar (succeed) and #britneyspears used F+ but M−;horooriza (horrifies), #metallica and #latingrammy used F− but M+.

Russian:мечтайте (dream!), магический (magical) and совет (advice) usedF+ but M−;исскушение (temptation), сложны (complicated), #iphones and#spartak (soccer team) used F− but M+.

Lexical Evaluation for Spanish and Russian

Spanish:fiasco, triunfar (succeed) and #britneyspears used F+ but M−;horooriza (horrifies), #metallica and #latingrammy used F− but M+.

Russian:мечтайте (dream!), магический (magical) and совет (advice) usedF+ but M−;исскушение (temptation), сложны (complicated), #iphones and#spartak (soccer team) used F− but M+.

How gender differences in subjective languagecan help subjectivity and polarity classification in

social media?

Rule-based Subjectivity Classifiers

Gender-Independent (Riloff & Wiebe, 2003; Volkova et al., 2013):

GIndRBsubj =

{1 if ~w ·~f ≥ 0.5,0 otherwise.

Gender-Dependent:

GDepRBsubj =

{1 if ~wM · ~f M ≥ 0.5 ∧M,0 otherwise.

GIndRBsubj =

Gender-Dependent:

GDepRBsubj =

{1 if ~wM · ~f M ≥ 0.5 ∧M,0 otherwise.

GIndRBsubj =

Gender-Dependent:

GDepRBsubj =

{1 if ~wF · ~f F ≥ 0.5 ∧ F ,0 otherwise.

Rule-based Subjectivity Classification Results

Start with LI and incrementally add Emoticons, Adjectives, AdveRbs,Verbs, Nouns from LB.

Rule-based Subjectivity Classification Results

Rule-based Polarity Classifiers

GIndRBpol =

{1 if ~w+ · ~f+ ≥ ~w− · ~f−,0 otherwise

Gender-Dependent:

GDepRBpol =

{1 if ~wM+ · ~f M+ ≥ ~wM− · ~f M− ∧M,0 otherwise

GIndRBpol =

Gender-Dependent:

GDepRBpol =

{1 if ~wM+ · ~f M+ ≥ ~wM− · ~f M− ∧M,0 otherwise

GIndRBpol =

Gender-Dependent:

GDepRBpol =

{1 if ~wF+ · ~f F+ ≥ ~wF− · ~f F− ∧ F ,0 otherwise

Rule-based Polarity Classification Results

Experimental Setup

Gender-Independent features:V - unigram counts, LI ,LB - set-count features from the originaland bootstrapped lexicons, and E - emoticons

~f GIndsubj = [LI ,LB,E ,V ];

~f GIndpol = [L+

I ,L+B ,E

+,L−I ,L

−B ,E

−,V ].

Gender-Dependent joint features:

~f GDep−Jsubj = [LM

I ,LMB ,E

M ,LFI ,L

F ,V ];

~f Dep−Jpol = [LM+

I ,LM+B ,EM+,LF+

I ,LF+B ,EF+

LM−I ,LM−

B ,EM−,LF−I ,LF−

B ,EF−,V ].

Experimental Setup

Gender-Independent features:V - unigram counts, LI ,LB - set-count features from the originaland bootstrapped lexicons, and E - emoticons

~f GIndsubj = [LI ,LB,E ,V ];

~f GIndpol = [L+

I ,L+B ,E

+,L−I ,L

−B ,E

−,V ].

Gender-Dependent joint features:

~f GDep−Jsubj = [LM

I ,LMB ,E

M ,LFI ,L

F ,V ];

~f Dep−Jpol = [LM+

I ,LM+B ,EM+,LF+

I ,LF+B ,EF+

LM−I ,LM−

B ,EM−,LF−I ,LF−

B ,EF−,V ].

Subjectivity Classification Results using SL

Polarity Classification Results using SL

Summary

Empirical study of differences in subjective language betweenmale and female users in Twitter.

Analysis of hashtag and emoticon usage across cultures.Incorporating author gender as a model component cansignificantly improve subjectivity and polarity classification formultiple languages in social media.

Data: http://www.cs.jhu.edu/~svitlana/data/data_emnlp2013.zip

Summary

Empirical study of differences in subjective language betweenmale and female users in Twitter.Analysis of hashtag and emoticon usage across cultures.

Incorporating author gender as a model component cansignificantly improve subjectivity and polarity classification formultiple languages in social media.

Summary

Empirical study of differences in subjective language betweenmale and female users in Twitter.Analysis of hashtag and emoticon usage across cultures.Incorporating author gender as a model component cansignificantly improve subjectivity and polarity classification formultiple languages in social media.

Summary

Empirical study of differences in subjective language betweenmale and female users in Twitter.Analysis of hashtag and emoticon usage across cultures.Incorporating author gender as a model component cansignificantly improve subjectivity and polarity classification formultiple languages in social media.

References I

Bamman, D., Eisenstein, J., & Schnoebelen, T. (2012). Gender inTwitter: styles, stances, and social networks. ComputingResearch Repository.

Boneva, B., Kraut, R., & Frohlich, D. (2001). Using email for personalrelationships: The difference gender makes. AmericanBehavioral Scientist, 45(3), 530-549.

Eisenstein, J., O’Connor, B., Smith, N. A., & Xing, E. P. (2010). Alatent variable model for geographic lexical variation. InProceedings of the Conference on Empirical Methods in NaturalLanguage Processing (EMNLP’10) (p. 1277-1287).

Fan, T. K., & Chang, C. H. (2009). Sentiment-oriented contextualadvertising. Advances in Information Retrieval, 5478, 202-215.

References II

Gefen, D., & Ridings, C. M. (2005). If you spoke as she does, sir,instead of the way you do: a sociolinguistics perspective ofgender differences in virtual communities. SIGMIS Database,36(2), 78-92.

Holmes, J., & Meyerhoff, M. (2004). The handbook of language andgender. Blackwell Publishing.

Macaulay, R. (2006). Pure grammaticalization: The development of ateenage intensifier. Language Variation and Change, 18(03),267–283.

Mohammad, S., & Yang, T. (2011). Tracking sentiment in mail: Howgenders differ on emotional axes. In Proceedings of the 2ndWorkshop on Computational Approaches to Subjectivity andSentiment Analysis (WASSA’11) (p. 70-79).

References III

O’Connor, B., Eisenstein, J., Xing, E. P., & Smith, N. A. (2010). Amixture model of demographic lexical variation. In Proceedingsof NIPS Workshop on Machine Learning in Computational SocialScience (p. 1-7).

Ott, M., Choi, Y., Cardie, C., & Hancock, J. T. (2011). Finding deceptiveopinion spam by any stretch of the imagination. In Proceedingsof the 49th Annual Meeting of the Association for ComputationalLinguistics: Human Language Technologies (p. 309-319).

Picard, R. W. (1997). Affective computing. MIT Press.Resnik, P. (2013). Getting real(-time) with live polling.

(http://vimeo.com/68210812)Riloff, E., & Wiebe, J. (2003). Learning extraction patterns for

subjective expressions. In Proceedings of the Conference onEmpirical Methods in Natural Language Processing (EMNLP’03)(p. 105-112).

References IV

Tagliamonte, S. A. (2006). Analysing sociolinguistic variation.Cambridge University Press, 1st. Edition.

Volkova, S., Wilson, T., & Yarowsky, D. (2013). Exploring sentiment insocial media: Bootstrapping subjectivity clues from multilingualTwitter streams. In Proceedings of the 51st Annual Meeting ofthe Association for Computational Linguistics (ACL’13) (pp.505–510).

exploring demographic language variations to …svitlana/posters/emnlp13-slides.pdfexploring...

Documents

inflammation ph.d., md, assistant professor dzyha svitlana...

svitlana linda* historicism in the architecture of lviv

demographic dividend demographic dividend demographic...

svitlana boiko. portfolio

aetiomorphological and demographic variations of post

svitlana shytikova national tempus-office in ukraine

РОЗРОБКА КОМПЛЕКСУ ПРОГРАМ...

exploring demographic language variations to improve ... ·...

predicting user demographics, emotions and opinions in...

regional variations of demographic transitions: china’s...

exploring demographic language variations to improve...

variations in demographic characteristics of...

demographic transition, demographic dividend, and lewis

2.9. dr. svitlana moroz - rehabilitation centre

svitlana-tacienko.edukit.kr.uasvitlana-tacienko.edukit.kr.ua/files/downloads... ·...

svitlana mytryayeva 2011.11.17

“the mitten” /translated by edvard trela/ directed by...

on goldberg variations/variations

sponsoring teacher: svitlana dyka, school-gymnasium # 31...

by svitlana koshmai school # 7 of cherkasy cherkasy region