tecnolengua lingmotif at tass 2017: spanish twitter...

Tecnolengua Lingmotif at TASS 2017: SpanishTwitter Dataset Classification Combining

Wide-coverage Lexical Resources and Text Features

Tecnolengua Lingmotif en TASS 2017: Clasificacion depolaridad de tuits en espanol combinando recursos lexicos de

amplia cobertura con rasgos textuales.

Antonio Moreno-Ortiz & Chantal Perez HernandezUniversity of Malaga

Spain{amo, mph}@uma.es

Abstract: In this paper we describe our participation in TASS 2017 shared taskon polarity classification of Spanish tweets. For this task we built a classificationmodel based on the Lingmotif Spanish lexicon, and combined this with a number offormal text features, both general and CMC-specific, as well as single-word keywordsand n-gram keywords, achieving above-average results across all three datasets. Wereport the results of our experiments with different combinations of said feature setsand machine learning algorithms (logistic regression and SVM).Keywords: sentiment analysis, twitter, polarity classification

Resumen: En este artıculo describimos nuestra participacion en la tarea de clasi-ficacion de polaridad de tweets en espanol del TASS 2017. Para esta tarea hemosdesarrollado un modelo de clasificacion basado en el lexicon espanol de Lingmotif,combinado con una serie de rasgos formales de los textos, tanto generales como es-pecıficos de la comunicacion mediada por ordenador (CMC), junto con palabras yunidades fraseologicas clave, lo que nos ha permitido obtener unos resultados porencima de la media en los tres conjuntos de la prueba. Mostramos los resultadosde nuestros experimentos con diferentes combinaciones de conjuntos de funciones yalgoritmos de aprendizaje automatico (regresion logıstica y SVM).Palabras clave: analisis de sentimiento, twitter, clasificacion de polaridad

1 Introduction

The use of microblogging sites in general, andTwitter in particular, has become so well -established that it is now a common sourceto poll user opinion and even social happiness(Abdullah et al., 2015). Its relevance as a so-cial hub can hardly be overestimated, and itis now common for traditional media to refer-ence Twitter trending topics as an indicatorof social concerns and interests.

It is not surprising, then, that Twitterdatasets are increasingly being used for sen-timent analysis shared tasks. The SemEvalseries of shared tasks included SentimentAnalysis of English Twitter content in 2013(Nakov et al., 2013), and included other lan-guages in later editions. The TASS Work-shop on Sentiment Analysis at SEPLN seriesstarted in 2012, and continued on a yearly

basis, thus being a milestone not only forSpanish Twitter content, but for sentimentanalysis in general.

The General Corpus of TASS was pub-lished for TASS 2013 (Villena Roman et al.,2013), introducing aspect-based sentimentanalysis, consisting of over 68,000 polarity-annotated tweets. Its creation followed cer-tain design criteria in terms of topics (poli-tics, football, literature, and entertainment)and users.

TASS 2017 (Martınez-Camara et al.,2017) keeps the Spain-only General Corpusof TASS, and introduces a new internationalcorpus of Spanish tweets, named InterTASS.The InterTASS corpus adds considerable dif-ficulty to the tasks not only because of itsmulti-varietal nature, but also because, un-like the General Corpus of TASS, content has

TASS 2017: Workshop on Semantic Analysis at SEPLN, septiembre 2017, págs. 35-42

ISSN 1613-0073 Copyright © 2017 by the paper's authors. Copying permitted for private and academic purposes.

not been filtered or their users selected, whichintroduces many and varied decoding issues.

1.1 Classification tasks

TASS 2017 proposes two classification tasks.Task 1 focuses on sentiment analysis at thetweet level, while Task 2 deals with aspect-based sentiment classification. We took partin Task 1, since we have not yet tackledaspect-based sentiment analysis. The aimof this task is the automatic classification oftweets in one of 4 levels: positive, nega-tive, neutral, and none.

The neutral/none distinction intro-duces added difficulty to the classificationtask. Tweets annotated as none are sup-posed to express no sentiment whatsoever, asin informative or declarative texts, whereasthe neutral category of tweets is meant toqualify tweets where both positive and neg-ative opinion is expressed, but they canceleach other out, resulting in a neutral overallmessage.

We believe this distinction is too fuzzyto be annotated reliably. First, precise bal-ance of polarity is hardly ever found in anymessage where sentiment is expressed: themessage is usually ”negative/positive situa-tion x, somehow counterbalanced by posi-tive/negative situation y”, with an entail-ment that the result is tilted to either side.The following are examples of tweets taggedas neutral in the training set:

• 768547351443169284 Parece que las cosas no tevan muy bien, espero que todo mejore, que todoel mundo merece ser feliz.

• 770417499317895168 No hay nada mas bonito qsepararse d una persona y q al tiempo t diga qt echa de menos... pero a mi no m va a pasar

We also found a number of exampleswhere tweets that clearly fell into none cases,where wrongly annotated as neutral:

• 768588061496209408 Estas palabras, del Po-ema, INSTANTES, son de Nadine Stair. Es-critora norteamericana, a la q le gustan los hela-dos.

• 767846757996847104 pues imaginate en unacasa muy grande

• 769993102442524674 Ninguno de los clubes lohizo oficial pero se dice que sı

These annotation issues are to be ex-pected, due to the added cognitive load thatis placed on the annotators, as other re-searchers have pointed out (Mohammad andBravo-Marquez, 2017a). Also, its presence

makes it more difficult to compare resultswith those of other sentiment classificationshared tasks, where the none class is not con-sidered.

1.2 Lexicon-based SentimentAnalysis

Within Sentiment Analysis it is commonto distinguish corpus-based approaches fromlexicon-based approaches. Although a com-bination of both methods can be found in theliterature (Riloff, Patwardhan, and Wiebe,2006), Lexicon-based approaches are usu-ally preferred for sentence-level classification(Andreevskaia and Bergler, 2007), whereascorpus-based, statistical approaches are pre-ferred for document-level classification.

Using sentiment dictionaries has a longtradition in the field. WordNet (Fellbaum,1998) has been a recurrent source of lexicalinformation (Kim and Hovy, 2004; Hu andLiu, 2004; Adreevskaia and Bergler, 2006),either directly, as a source of lexical informa-tion, or for sentiment lexicon construction.Other common lexicons used in English sen-timent analysis research include The Gen-eral Inquirer (Stone and Hunt, 1963), MPQA(Wilson, Wiebe, and Hoffmann, 2005), andBing Liu’s Opinion Lexicon (Hu and Liu,2004). Yet other researchers have used acombination of existing lexicons or createdtheir own (Hatzivassiloglou and McKeown,1997; Turney, 2002). The use of lexiconshas sometimes been straightforward, wherethe mere presence of a sentiment word de-termines a given polarity. However, negationand intensification can alter the valence orpolarity of that word.1 Modification of sen-timent in context has also been widely rec-ognized and dealt with by some researchers(Kennedy and Inkpen, 2006; Polanyi and Za-enen, 2006; Choi and Cardie, 2008; Taboadaet al., 2011).

However, the valence of a given word mayvary greatly from one domain to another, afact well recognized in the literature (Aueand Gamon, 2005; Pang and Lee, 2008; Choi,Kim, and Myaeng, 2009), which causes prob-lems when a sentiment lexicon is the onlysource of knowledge. A number of solutionshave been proposed, mostly using ad hoc dic-

1The use of the terms valence and polarity is usedinconsistently in the literature. We use polarity torefer to the binary distinction positive/negative sen-timent, and valence to a value of intensity on a scale.

Antonio Moreno-Ortiz, Chantal Pérez Hernéndez

36

tionaries, sometimes created automaticallyfrom a domain-specific corpus (Tai and Kao,2013; Lu et al., 2011).

Our approach to using a lexicon takessome ideas from the aforementioned ap-proaches. We describe it in the next section.

2 System description

Our system for this polarity classificationtask relies on the availability of rich setsof lexical, sentiment, and (formal) text fea-tures, rather than on highly sophisticated al-gorithms. We basically used a logistic re-gression classifier trained on the optimal setof features after many feature combinationswere tried on the training set. We also trieda SVM classifier on the same feature sets, butwe consistently obtained poorer results com-pared to the logistic regression classifier. Pa-rameter finetuning on each classifier was verylimited; we simply performed a grid searchon the C parameter, which threw 100 as op-timal. For the SVM classifier we found theRBF kernel to perform better than the lin-ear kernel2. We mostly focused on featureselection and combination.

We obtained good results on the three testdatasets, with some important differences be-tween the InterTASS and General datasets.Results, however, were not as good as we hadanticipated based on our experiments on thetraining datasets. We discuss this in section3 below. Here we describe our general systemarchitecture and feature sets.

This TASS shared task is our first expe-rience with Twitter data sentiment classifi-cation proper, although we had the relatedexperience from our recent participation inWASSA-2017 Shared Task on Emotion In-tensity (Mohammad and Bravo-Marquez,2017b). From this shared task we learnt therelevance and impact that other, non-lexicaltext features can have in microblogging texts.

Since our focus was on identifying the pre-dictive power of classification features, andintended to perform many experiments withfeatures combinations, we designed a simpletool to facilitate this.

This tool, Lingmotif Learn, is a GUI-enabled convenience tool that managesdatasets and uses the Python-based scikit-learn (Pedregosa et al., 2011) machine learn-ing toolkit. It facilitates loading and prepro-

2For the RBF kernel we used gamma=0.001,C=100. For the linear kernel we used C=1000.

Figure 1: Lingmotif Learn

cessing datasets, getting the text run troughthe Lingmotif SA engine, and feeding theresulting data into one of several machinelearning algorithms. Lingmotif Learn is ableto extract both Sentiment features and non-sentiment features, such as raw text metricsand keywords, and it makes it easy to experi-ment with different feature set combinations.

2.1 The Lingmotif tool

Sentiment features are returned by the Ling-motif SA engine. Lingmotif (Moreno-Ortiz,2017a) is a user-friendly, multilingual, textanalysis application with a focus on senti-ment analysis that offers several modes oftext analysis. It is not specifically geared to-wards any particular type of text or domain.It can analyze long documents, such as nar-ratives, medium-sized ones, such as politicalspeeches and debates, and short to very shorttexts, such as user reviews and tweets. Foreach of these, the tool offers different outputsand metrics.

For large collections of short texts, suchas Twitter datasets, it provides a multi-document mode whose default output is clas-sification. In the current publicly availableversion this classification is entirely based onthe Text Sentiment Score (TSS), which at-tempts to summarize the text’s overall po-larity on a 0-100 scale. TSS is calculated asa function of the text’s positive and nega-tive scores and the sentiment intensity, whichreflects the proportion of sentiment to non-sentiment lexical items in the text. Spe-cific details on TSS calculation can be foundin Moreno-Ortiz (2017a). A description ofits applications is found in Moreno-Ortiz(2017b).

Lingmotif results are generated as aHTML/Javascript document, which is saved

Tecnolengua Lingmotif at TASS 2017: Spanish Twitter Dataset Classification Combining Wide-Coverage Lexical Resources and Text Features

37

Name Descriptiontss Text Sentiment Scoretsi Text Sentiment Intensitysent.it Number of lexical Itemspos.sc Positive scoreneg.sc Negative scorepos.it Number of positive itemsneg.it Number of negative itemsneu.it Number of neutral itemssplit1.tss TSS for split 1 of textsplit2.tss TSS for split 2 of textsentences Number of sentencesshifters Number of sentiment shifters

Table 1: Sentiment feature set

locally to a predefined location and auto-matically sent to the user’s default browserfor immediate display. Internally, the ap-plication generates results as an XML doc-ument containing all the relevant data; thisXML document is then parsed against one ofseveral available XSL templates, and trans-formed into the final HTML.

Lingmotif Learn simply plugs into the in-ternally generated XML document to retrievethe desired sentiment analysis data, and ap-pends the data to each tweet as features.

2.2 Sentiment features

Table 1 summarizes the sentiment-relatedfeature set generated by the Lingmotif en-gine.

Most of these features are included in theoriginal Lingmotif engine, but for this occa-sion we experimented with text splits to testthe relevance of the position of the sentimentwords in the tweet. The features split1.tssand split2.tss are the combined sentimentscore for each half of the tweet. The assump-tion was that sentiment words used towardsthe end of the tweet may have more weighton the overall tweet polarity. This might behelpful especially for the P/N/NEU distinc-tion. Neutral tweets are supposed to havesome balance between positivity and negativ-ity. In our tests with the training set, how-ever, adding these features did not improveresults. We also experimented with 3 splits,with the same results. These features werethus discarded for test set classification.

Some of these features are in fact re-dundant. Notably, tss already encapsulatespos.sc, neg.sc, and neu.it. In our tests,the classifier performed better using just thepos.sc and neg.sc values, than our calcu-lated tss, so we only used these two features.

Name Descriptionsentences Number of sentencestt.ratio Type/Token ratiolex.items Number of lexical itemsgram.items Number of grammatical itemsvb.items Number of verbsnn.items Number of nounsnnp.items Number of proper nounsjj.items Number of adjectivesrb.items Number of adverbschars Number of charactersintensifiers Number of intensifierscontrasters Number of contrast wordsemoticons Number of emoticons/emojisall.caps Number of upper case wordschar.ngrams Number of character ngramsx.marks Number of exclamation marksq.marks Number of question marksquote.marks Number of quotation markssusp.marks Number of suspension marksx.marks.seqs Number of x.marks sequencesq.marks.seqs Number of q.marks sequencesxq.marks.seqs Number of x/q marks sequenceshandles Number of Twitter handleshashtags Number of hashtagsurls Number of URL’s

Table 2: Text feature set

2.3 Text features

Raw text features are commonly used in sen-timent analysis shared tasks successfully (e.g.Mohammad, Kiritchenko, and Zhu (2013),Kiritchenko et al. (2014)), including previ-ous editions of TASS (Ceron-Guzman, 2016).The role of some of them is rather obvi-ous; the presence of emoticons or exclama-tion marks, for example, usually determines(strong) sentiment or opinion, thus being agood candidate predictor for the none vs restdistinction. The role of others, however, isnot as clear. For example, we consistently ob-tained better results using the gram.itemsfeature, whereas the number of lexical itemswas not a good predictor. The number ofverbs, adjectives and adverbs also proved tobe useful, whereas the number of nouns didnot.

Table 2 contains the full list of text fea-tures we experimented with.

2.4 Keyword features

In order to account for words and expressionsthat convey sentiment but may not be in-cluded in the sentiment lexicon, we experi-mented with automatic keyword extractionfor each of the classes in the training set.Automatic keyword and keyphrase extrac-tion is a well developed field and a numberof tools and methodologies have been pro-


38

Name Descriptionp.kw Positive keywordsp.ng.kw Positive ngram keywordsp.handles Positive handlesn.kw Negative keywordsn.ng.kw Negative ngram keywordsn.handles Negative handlesneu.kw Neutral keywordsneu.ng.kw Neutral ngram keywordsneu.handles Neutral handlesnone.kw None keywordsnone.ng.kw None ngram keywordsnone.handles None handles

Table 3: Keywords feature set

posed. Hasan and Ng (2014) provide a goodoverview of the state-of-the-art techniques forkeyphrase extraction.

We used a very simple approach thatconsisted in comparing frequencies of singlewords and ngrams (2 to 4 words) on a one-vs-rest basis for each of our four classes, forwords and ngrams with a minimum frequencyof 2. We calculated and ranked keyness basedon the chi-square statistic, and then manuallyremoved irrelevant results. We ended up witha list of 100 keywords and 100 keyphrases foreach class. We did the same for Twitter han-dles.

Using the keywords feature set improvedresults considerably in our tests with thetraining set. However, this improvement didnot transfer well to the test sets, especially inthe case of the InterTASS dataset. We fur-ther discuss this issue in section 3.

3 Experiments and Results

Tables 4, 5, and 6 show our results for each ofthe test sets. Although performance is strongacross all three, there clearly is a differencebetween the General TASS datasets, on theone hand, and the InterTASS dataset on theother.

Experiment Macro-F1 Accuracysent-only 0.456 0.582run3 0.441 0.576sent-only-fixed 0.441 0.595

Table 4: Official results for the InterTASStest set

We believe this is due to two main reasons.First, the General training set (7,218 tweets)is much larger than the InterTASS trainingset (1,514 tweets, using both the training anddevelopment datasets). This of course pro-vides a much more solid training base for

Experiment Macro-F1 Accuracyrun3 0.528 0.657final 0.517 0.632no ngrams 0.508 0.652

Table 5: Official results for the General TASStest set

Experiment Macro-F1 Accuracyrun3 0.521 0.638final 0.488 0.618run4 0.483 0.612

Table 6: Official results for the GeneralTASS-1k test set

the former than the latter. All our modelswere trained on one dataset where both train-ing datasets (General and InterTASS) wheremerged. Perhaps better results would havebeen obtained by training on each datasetseparately.

The other reason for poorer performanceon the InterTASS test set concerns the verydifferent nature of the datasets. The Gen-eral Corpus of TASS consists of tweets gen-erated by public figures (artists, politicians,journalists) with a large number of follow-ers. Such Twitter users are more predictableboth in terms of the content of their tweetsand the language they use. They are alsoCastilian Spanish speakers entirely. Most ofthese tweets contain very compact but care-fully chosen language, expressing users’ opin-ion or evaluation of polically or socially rel-evant events. On the other hand, the in-terTASS corpus shows much more variabil-ity; first, the tweets were collected not onlyfrom Spain, but from several Latin Americancountries, which introduces important lexi-cal variability. Second, no user selection isapparent. Tweets were randomly collectedfrom the whole Spanish speaking user base.This introduces spelling errors and a muchmore colloquial and chatty language. Non-lexical linguistic features, such as exclama-tion marks, emojis or emoticons, are recur-rent, as are, user-to user messages, whichare of course hard-to-decode, since they pre-suppose certain privately shared knowledge.These issues have obviously affected the per-formance of all TASS participants, as is clearfrom the final leader board.

We obtained the best results for the Gen-eral datasets with our run3 experiment,where we combined a selection of featuresfrom the three feature sets listed in tables


39

Featurespos.sc neu.kwneg.sc neu.ng.kwvb.items neu.handlesjj.items none.kwrb.items none.ng.kwgram.items none.handlesn.chars emoticonsintensifiers all.capscontrasters char.ngramsp.kw x.marksp.ng.kw q.marksp.handles susp.marksn.kw hashtagsn.ng.kw handlesn.handles urls

Table 7: run3 experiment feature set

Featurespos.sc handlesneg.sc emoticonsvb.items all.capsjj.items char.ngramsrb.items x.marksgram.items q.marksn.chars susp.marksintensifiers urlscontrasters hashtags

Table 8: sent-only experiment feature set

1, 2, and 3. This selection was in fact theoptimal we found during our cross-validationtests on the training dataset. Table 7 liststhe feature set used in this experiment.

Concerning the InterTass test set, the bestresults were obtained with the sent-only ex-periment, where a reduced set of features wasused. We list these features in table 8.

We obtained better results for the Inter-TASS test set using this reduced set of fea-tures because the keyword sets were caus-ing noise, since they were extracted using thewhole training set, which contained a muchlarger proportion of tweets from the GeneralTASS dataset.

Another important aspect is the large dif-ference that we encountered between our owntests on the training datasets and our final(official) results. For the General corpus ofTASS, we consistently obtained very high F1scores (upwards of 0.73) using the keywordset, but much closer to the official resultswithout them. This is a clear indication of

model overfitting, with an obvious negativeimpact on the classification of the test set.After this became apparent on our first re-sults upload, we corrected by reducing thesets of keywords, keyphrases and user han-dles, which resulted in better overall results.

4 Conclusions

This shared task has served us to assess theusefulness of many different features as pre-dictors of polarity classification in Spanishtweets. The differing sizes and characteristicsof the training and test datasets determinedto some extent our results, but we also felt weoverfitted our model with too large a selec-tion of keywords, which threw overoptimisticresults in our tests.

Our results on par with other participantswho used more sophisticated systems fromthe technical perspective, which is also anindication of the salient role that curated,high-quality lexical resources play in senti-ment analysis.

We also experienced the negative impactof model overfitting and learnt how to limitits effects. We plan to use this knowledge infuture versions of Lingmotif, which currentlyuses sentiment features exclusively. It is ob-vious that combining those with other formalfeatures can improve results considerably.

Acknowledgments

This research was supported by Spain’sMINECO through the funding of projectLingmotif2 (FFI2016-78141-P).

References

Abdullah, S., E. L. Murnane, J. M. Costa,and T. Choudhury. 2015. Collectivesmile: Measuring societal happiness fromgeolocated images. In Proceedings of the18th ACM Conference on Computer Sup-ported Cooperative Work & SocialComputing, CSCW ’15, pages 361–374,New York, NY, USA. ACM.

Adreevskaia, A. and S. Bergler. 2006. Min-ing wordnet for fuzzy sentiment: Senti-ment tag extraction from wordnet glosses.In 11th Conference of the European Chap-ter of the Association for ComputationalLinguistics, pages 209–216.

Andreevskaia, A. and S. Bergler. 2007.Clac and clac-nb: Knowledge-based and


40

corpus-based approaches to sentiment tag-ging. In Proceedings of the 4th In-ternational Workshop on Semantic Eval-uations, SemEval ’07, pages 117–120,Stroudsburg, PA, USA. Association forComputational Linguistics.

Aue, A. and M. Gamon. 2005. Customizingsentiment classifiers to new domains: Acase study. Borovets, Bulgaria.

Ceron-Guzman, J. A. 2016. Jacerong at tass2016: An ensemble classifier for sentimentanalysis of spanish tweets at global level.In Proceedings of TASS 2016: Work-shop on Sentiment Analysis at SEPLNco-located with 32nd SEPLN Conference(SEPLN 2016), pages 35–39, Salamanca,Spain. SEPLN.

Choi, Y. and C. Cardie. 2008. Learning withcompositional semantics as structural in-ference for subsentential sentiment analy-sis. In Proceedings of the Conference onEmpirical Methods in Natural LanguageProcessing, EMNLP ’08, pages 793–801,Stroudsburg, PA, USA.

Choi, Y., Y. Kim, and S.-H. Myaeng. 2009.Domain-specific sentiment analysis usingcontextual feature generation. In Proceed-ing of the 1st international CIKM work-shop on Topic-sentiment analysis for massopinion, pages 37–44, Hong Kong, China.ACM.

Fellbaum, C., editor. 1998. WordNet AnElectronic Lexical Database. The MITPress, Cambridge, MA; London, May.

Hasan, K. S. and V. Ng. 2014. Auto-matic keyphrase extraction: A survey ofthe state of the art. In Proceedings of the52nd Annual Meeting of the Associationfor Computational Linguistics (Volume 1:Long Papers), pages 1262–1273.

Hatzivassiloglou, V. and K. R. McKeown.1997. Predicting the semantic orientationof adjectives. In Proceedings of the eighthconference on European chapter of the As-sociation for Computational Linguistics,pages 174–181, Madrid, Spain. Associa-tion for Computational Linguistics.

Hu, M. and B. Liu. 2004. Mining and sum-marizing customer reviews. In Proceed-ings of the tenth ACM SIGKDD interna-tional conference on Knowledge discovery

and data mining, pages 168–177, Seattle,WA, USA. ACM.

Kennedy, A. and D. Inkpen. 2006. Sentimentclassification of movie reviews using con-textual valence shifters. ComputationalIntelligence, 22(2):110–125.

Kim, S.-M. and E. Hovy. 2004. Determin-ing the sentiment of opinions. In Pro-ceedings of the 20th international confer-ence on Computational Linguistics, page1367, Geneva, Switzerland. Associationfor Computational Linguistics.

Kiritchenko, S., X. Zhu, C. Cherry, andS. Mohammad. 2014. Nrc–canada-2014:Detecting aspects and sentiment in cus-tomer reviews. In Proceedings of the8th International Workshop on SemanticEvaluation (SemEval 2014), pages 437–442, Dublin, Ireland, August. Associationfor Computational Linguistics and DublinCity University.

Lu, Y., M. Castellanos, U. Dayal, andC. Zhai. 2011. Automatic construction ofa context-aware sentiment lexicon: An op-timization approach. In Proceedings of the20th International Conference on WorldWide Web, WWW ’11, pages 347–356,New York, NY, USA. ACM.

Martınez-Camara, E., M. C. Dıaz-Galiano,M. A. Garcıa-Cumbreras, M. Garcıa-Vega, and J. Villena-Roman. 2017.Overview of tass 2017. In J. Vil-lena Roman, M. A. Garcıa Cumbreras,E. Martınez-Camara, M. C. Dıaz Galiano,and M. Garcıa Vega, editors, Proceedingsof TASS 2017: Workshop on SemanticAnalysis at SEPLN (TASS 2017), vol-ume 1896 of CEUR Workshop Proceed-ings, Murcia, Spain, September. CEUR-WS.

Mohammad, S. and F. Bravo-Marquez.2017a. Emotion intensities in tweets. InProceedings of the sixth joint conferenceon lexical and computational semantics(*Sem), Vancouver, Canada.

Mohammad, S. and F. Bravo-Marquez.2017b. Wassa-2017 shared task on emo-tion intensity. In Proceedings of theEMNLP 2017 Workshop on Computa-tional Approaches to Subjectivity, Sen-timent, and Social Media, Copenhagen,Denmark, September.


41

Mohammad, S. M., S. Kiritchenko, andX. Zhu. 2013. Nrc-canada: Building thestate-of-the-art in sentiment analysis oftweets. In Proceedings of the seventh in-ternational workshop on Semantic Evalu-ation Exercises (SemEval-2013), Atlanta,Georgia, USA, June.

Moreno-Ortiz, A. 2017a. Lingmotif: A user-focused sentiment analysis tool. Proce-samiento del Lenguaje Natural, 58(0):133–140, March.

Moreno-Ortiz, A. 2017b. Lingmotif: Senti-ment analysis for the digital humanities.In Proceedings of the 15th Conference ofthe European Chapter of the Associationfor Computational Linguistics, pages 73–76, Valencia, Spain, April. Association forComputational Linguistics.

Nakov, P., Z. Kozareva, A. Ritter, S. Rosen-thal, V. Stoyanov, and T. Wilson. 2013.Semeval-2013 task 2: Sentiment analy-sis in twitter. In Proceedings of the Sev-enth International Workshop on Seman-tic Evaluation (SemEval 2013), Atlanta,Georgia, USA, June.

Pang, B. and L. Lee. 2008. Opinion miningand sentiment analysis. Foundations andTrends in Information Retrieval, 2(1-2):1–135.

Pedregosa, F., G. Varoquaux, A. Gram-fort, V. Michel, B. Thirion, O. Grisel,M. Blondel, P. Prettenhofer, R. Weiss,V. Dubourg, J. Vanderplas, A. Passos,D. Cournapeau, M. Brucher, M. Perrot,and E. Duchesnay. 2011. Scikit-learn:Machine learning in python. J. Mach.Learn. Res., 12:2825–2830, November.

Polanyi, L. and A. Zaenen. 2006. Contex-tual valence shifters. In Computing Atti-tude and Affect in Text: Theory and Ap-plications, volume 20 of The InformationRetrieval Series. Springer, Dordrecht, TheNetherlands, shanahan, james g., qu, yan,wiebe, janyce edition, pages 1–10.

Riloff, E., S. Patwardhan, and J. Wiebe.2006. Feature subsumption for opinionanalysis. In Proceedings of the 2006 Con-ference on Empirical Methods in NaturalLanguage Processing, EMNLP ’06, pages440–448, Stroudsburg, PA, USA. Associa-tion for Computational Linguistics.

Stone, P. J. and E. B. Hunt. 1963. Acomputer approach to content analysis:Studies using the general inquirer sys-tem. In Proceedings of the May 21-23,1963, Spring Joint Computer Conference,AFIPS ’63 (Spring), pages 241–256, NewYork, NY, USA. ACM.

Taboada, M., J. Brooks, M. Tofiloski,K. Voll, and M. Stede. 2011. Lexicon-based methods for sentiment analysis.Computational Linguistics, 37(2):267–307.

Tai, Y.-J. and H.-Y. Kao. 2013. Automaticdomain-specific sentiment lexicon genera-tion with label propagation. In Proceed-ings of International Conference on In-formation Integration and Web-based Ap-plications & Services, IIWAS ’13, pages53:53–53:62, New York, NY, USA. ACM.

Turney, P. D. 2002. Thumbs up or thumbsdown? semantic orientation applied tounsupervised classification of reviews. InProceedings of the 40th Annual Meetingof the Association for Computational Lin-guistics (ACL), pages 417–424, Philadel-phia, USA.

Villena Roman, J., S. Lana Serrano,E. Martınez Camara, and J. C.Gonzalez Cristobal. 2013. Tass -workshop on sentiment analysis at sepln.Procesamiento del Lenguaje Natural,50:37–44.

Wilson, T., J. Wiebe, and P. Hoffmann.2005. Recognizing contextual polarityin phrase-level sentiment analysis. InProceedings of the Conference on Hu-man Language Technology and EmpiricalMethods in Natural Language Processing,HLT ’05, pages 347–354, Stroudsburg, PA,USA. Association for Computational Lin-guistics.


42

tecnolengua lingmotif at tass 2017: spanish twitter...

Documents