redefining sentiment analysis

34
Redefining Sentiment Analysis Shlomo Argamon Subtext3, Inc. Illinois Institute of Technology

Upload: shlomo-argamon

Post on 26-Jan-2015

127 views

Category:

Technology


0 download

DESCRIPTION

Why current sentiment analysis theory and practice is fundamentally flawed and what to do about it. Talk at the research session of the Sentiment Symposium in San Francisco on Nov 8, 2011. Based on joint work with Dr. Ken Bloom.

TRANSCRIPT

Page 1: Redefining Sentiment Analysis

Redefining(Sentiment(Analysis1

Shlomo Argamon Subtext3, Inc.

Illinois Institute of Technology

Page 2: Redefining Sentiment Analysis

Strong(foundations…18 November, 2011 Sentiment Analysis Symposium, San Francisco

Page 3: Redefining Sentiment Analysis

…are(critical!18 November, 2011 Sentiment Analysis Symposium, San Francisco

Page 4: Redefining Sentiment Analysis

The(Archetype1Review classification – the earliest, and still

stereotypical sentiment analysis application

8 November, 2011 Sentiment Analysis Symposium, San Francisco

5.0$out$of$5$stars$Great!1This(Nikon(is(great.(It(takes(perfect(pictures(and(I(tried(the(shaking(effect;(it(was(the(bom.(I(shaked(the(camera(while(taking(a(picture(and(the(picture(came(out(picture(perfect.((Great(product!(1

1.0$out$of$5$stars$Buy*cheap,*buy*twice2Im(absolutely(livid(with(myself(for(buying(this(camera.(IFve(gone(through(a(divorce(and(been(left(with(nothing((including(the(loss(of(my(SLR(cameras)(and(this(was(the(one(I(could(afford(that(promised(to(be(the(most(similar(to(the(SLRs.(I(have(a(2(year(old(son,(and(armed(with(only(this(camera,(I(have(missed(documenting(the(last(6(months(of(his(time(with(me.(Every(picture(is(blurredNN(granted(heFs(2(and(moves(a(lot,(but(even(the(slightest(movement(by(the(photographer(blurs(the(image.(Blurred.(EVERYTHING.(I(wish(I(had(saved(up(and(bought(something(decent.(I(will(take(my(loss(on(this(one(for(an(Amazon(gift(card,(but(DONFT(WASTE(YOUR(MONEY.(*sigh*(1

5/5$stars$$.Opera*with*subtitles?**Who*knew?*I(had(the(great(fortune(to(aZend(a(dress(rehersal(for(The(Damnation(of(Faust.((This(updated(version(was(amazing.((1A(M(A(Z(I(N(G.(((The(theatre(is(gorgeous.((The(acoustics(are(incredible.(Go.((Go(now.1

Page 5: Redefining Sentiment Analysis

Assumptions1

8 November, 2011 Sentiment Analysis Symposium, San Francisco

Text(has(a(single(main(sentiment(about(a(single(main(topic1

Sentiment(is(graded(on(a(linear(scale1Usually$positive/negative.

Words(carry(sentiment(information1

Analysis(=(Classification1

Very(liZle(semantics((if(any)1

Page 6: Redefining Sentiment Analysis

8 November, 2011 Sentiment Analysis Symposium, San Francisco

Let’s*look*a*li=le*more*closely…2

Page 7: Redefining Sentiment Analysis

Do(they(fit(the(archetype?1

8 November, 2011 Sentiment Analysis Symposium, San Francisco

This(setup(using(the(CD(was(about(as(easy(as(learning(how(to(open(a(refrigerator(door(for(the(first(time.(1 Easy2

Page 8: Redefining Sentiment Analysis

Do(they(fit(the(archetype?1

8 November, 2011 Sentiment Analysis Symposium, San Francisco

This(setup(using(the(CD(was(about(as(easy(as(opening(a(wooden(crate(with(your(fingernails.(1 Hard2

Page 9: Redefining Sentiment Analysis

Do(they(fit(the(archetype?1

8 November, 2011 Sentiment Analysis Symposium, San Francisco

There(are(a(few(extremely(sexy(new(features(in(Final*Cut*Pro*7.(1

This(setup(using(the(CD(was(about(as(easy(as(opening(a(wooden(crate(with(your(fingernails.(1This(setup(using(the(CD(was(about(as(easy(as(learning(how(to(open(a(refrigerator(door(for(the(first(time.(1

Page 10: Redefining Sentiment Analysis

Do(they(fit(the(archetype?1

8 November, 2011 Sentiment Analysis Symposium, San Francisco

There(are(a(few(extremely(sexy(new(features(in(Final*Cut*Pro*7.(1

This(setup(using(the(CD(was(about(as(easy(as(opening(a(wooden(crate(with(your(fingernails.(1This(setup(using(the(CD(was(about(as(easy(as(learning(how(to(open(a(refrigerator(door(for(the(first(time.(1

…but(it’s(generally(lousy?1

Page 11: Redefining Sentiment Analysis

Do(they(fit(the(archetype?1

8 November, 2011 Sentiment Analysis Symposium, San Francisco

There(are(a(few(extremely(sexy(new(features(in(Final*Cut*Pro*7.(1

This(setup(using(the(CD(was(about(as(easy(as(opening(a(wooden(crate(with(your(fingernails.(1This(setup(using(the(CD(was(about(as(easy(as(learning(how(to(open(a(refrigerator(door(for(the(first(time.(1

The(rest(of(it,(however,(…1

Page 12: Redefining Sentiment Analysis

Do(they(fit(the(archetype?1

8 November, 2011 Sentiment Analysis Symposium, San Francisco

There(are(a(few(extremely(sexy(new(features(in(Final(Cut(Pro(7.(1

This(setup(using(the(CD(was(about(as(easy(as(opening(a(wooden(crate(with(your(fingernails.(1This(setup(using(the(CD(was(about(as(easy(as(learning(how(to(open(a(refrigerator(door(for(the(first(time.(1

When(the(tent(was(finally(completed,(the(result(was(much(nicer(than(that(Kiva(tent(…(1

1

Positive?2

Page 13: Redefining Sentiment Analysis

Do(they(fit(the(archetype?1

8 November, 2011 Sentiment Analysis Symposium, San Francisco

There(are(a(few(extremely(sexy(new(features(in(Final(Cut(Pro(7.(1

This(setup(using(the(CD(was(about(as(easy(as(opening(a(wooden(crate(with(your(fingernails.(1This(setup(using(the(CD(was(about(as(easy(as(learning(how(to(open(a(refrigerator(door(for(the(first(time.(1

When(the(tent(was(finally(completed,(the(result(was(much(nicer(than(that(Kiva(tent(

but(not(as(nice(as(the(Arctic(oven…1

Negative?2

Page 14: Redefining Sentiment Analysis

Do(they(fit(the(archetype?1

8 November, 2011 Sentiment Analysis Symposium, San Francisco

There(are(a(few(extremely(sexy(new(features(in(Final(Cut(Pro(7.(1

This(setup(using(the(CD(was(about(as(easy(as(opening(a(wooden(crate(with(your(fingernails.(1This(setup(using(the(CD(was(about(as(easy(as(learning(how(to(open(a(refrigerator(door(for(the(first(time.(1

Jurassic(Park(was(a(great(book(but(a(lousy(movie.(1

When(the(tent(was(finally(completed,(the(result(was(much(nicer(than(that(Kiva(tent(

but(not(as(nice(as(the(Arctic(oven…1

Positive?2Negative?2

Page 15: Redefining Sentiment Analysis

Do(they(fit(the(archetype?1

8 November, 2011 Sentiment Analysis Symposium, San Francisco

There(are(a(few(extremely(sexy(new(features(in(Final(Cut(Pro(7.(1

This(setup(using(the(CD(was(about(as(easy(as(opening(a(wooden(crate(with(your(fingernails.(1This(setup(using(the(CD(was(about(as(easy(as(learning(how(to(open(a(refrigerator(door(for(the(first(time.(1

Jurassic(Park(was(a(great(book(but(a(lousy(movie.(1

Loud(but(nice(room(at(a(great(price(with(bad(view.1

When(the(tent(was(finally(completed,(the(result(was(much(nicer(than(that(Kiva(tent(

but(not(as(nice(as(the(Arctic(oven…1

Generally*positive2

Page 16: Redefining Sentiment Analysis

Do(they(fit(the(archetype?1

8 November, 2011 Sentiment Analysis Symposium, San Francisco

There(are(a(few(extremely(sexy(new(features(in(Final(Cut(Pro(7.(1

This(setup(using(the(CD(was(about(as(easy(as(opening(a(wooden(crate(with(your(fingernails.(1This(setup(using(the(CD(was(about(as(easy(as(learning(how(to(open(a(refrigerator(door(for(the(first(time.(1

Loud(but(nice(room(at(a(great(price(with(bad(view.1

When(the(tent(was(finally(completed,(the(result(was(much(nicer(than(that(Kiva(tent(

but(not(as(nice(as(the(Arctic(oven…1

Negative*on*sound2

Jurassic(Park(was(a(great(book(but(a(lousy(movie.(1

Page 17: Redefining Sentiment Analysis

Do(they(fit(the(archetype?1

8 November, 2011 Sentiment Analysis Symposium, San Francisco

There(are(a(few(extremely(sexy(new(features(in(Final(Cut(Pro(7.(1

This(setup(using(the(CD(was(about(as(easy(as(opening(a(wooden(crate(with(your(fingernails.(1This(setup(using(the(CD(was(about(as(easy(as(learning(how(to(open(a(refrigerator(door(for(the(first(time.(1

Loud(but(nice(room(at(a(great(price(with(bad(view.1

When(the(tent(was(finally(completed,(the(result(was(much(nicer(than(that(Kiva(tent(

but(not(as(nice(as(the(Arctic(oven…1

Positive*on*price2

Jurassic(Park(was(a(great(book(but(a(lousy(movie.(1

Page 18: Redefining Sentiment Analysis

Do(they(fit(the(archetype?1

8 November, 2011 Sentiment Analysis Symposium, San Francisco

There(are(a(few(extremely(sexy(new(features(in(Final(Cut(Pro(7.(1

This(setup(using(the(CD(was(about(as(easy(as(opening(a(wooden(crate(with(your(fingernails.(1This(setup(using(the(CD(was(about(as(easy(as(learning(how(to(open(a(refrigerator(door(for(the(first(time.(1

Loud(but(nice(room(at(a(great(price(with(bad(view.1

When(the(tent(was(finally(completed,(the(result(was(much(nicer(than(that(Kiva(tent(

but(not(as(nice(as(the(Arctic(oven…1

Negative*on*view2

Jurassic(Park(was(a(great(book(but(a(lousy(movie.(1

Page 19: Redefining Sentiment Analysis

Does(it(maZer?1Pa=ern2 Results*(Google)2 Percentage2

X is good 4,820,000,000 X is as good as Y 2,530,000,000 52% X is better than Y 2,120,000,000 44% X is bad 1,730,000,000 X is worse than Y 437,000,000 25% X is a good Y 14,160,000,000 X is a better Y than a Z 781,000,000 6% X at a good price 888,000,000 X at a good price but ... bad Y 29,400,000 3%

8 November, 2011 Sentiment Analysis Symposium, San Francisco

Page 20: Redefining Sentiment Analysis

AZitude

1

Annotation(oddities1

8 November, 2011 Sentiment Analysis Symposium, San Francisco

We(believe(in(the(sincerity(of(the(United(States(in(promising(not(to(mix(up(its(counterNterrorism(drive(with(the(Taiwan(Strait(issue1

the(United(States(1 the(sincerity(of(the(United(States(in(promising(not(to(mix(up(its(counterNterrorism(drive(with(

the(Taiwan(Strait(issue1

Page 21: Redefining Sentiment Analysis

AZitude

1

Topic1

Annotation(oddities1

8 November, 2011 Sentiment Analysis Symposium, San Francisco

…what(is(important(is(that(your(Internet(connection(will(never(even(reach(the(speed(capabilities(of(this(router...1

important1 your(Internet(connection(will(never(even(reach(the(speed(capabilities(of(this(router1

Page 22: Redefining Sentiment Analysis

AZitude

1

Topic1

Annotation(oddities1

8 November, 2011 Sentiment Analysis Symposium, San Francisco

It’ll(make(life(a(lot(easier,(and(preclude(you(from(having(to(give(this(product(a(negative(review.((1

positive1 product1

Page 23: Redefining Sentiment Analysis

AZitude

1

Topic1

Annotation(oddities1

8 November, 2011 Sentiment Analysis Symposium, San Francisco

However,(this(product(performed(well(below(my(expectation.((1

negative1 router1

Page 24: Redefining Sentiment Analysis

So(what’s(really(going(on?1

8 November, 2011 Sentiment Analysis Symposium, San Francisco

Her(appearance(and(demeanor(are(excellently(suited(to(her(role.1

SheFs(the(most(heartless(coqueZe(in(France!1 Negative2

Positive2

Classification+

Page 25: Redefining Sentiment Analysis

The(Structure(of(Sentiment1

8 November, 2011 Sentiment Analysis Symposium, San Francisco

Her(appearance(and(demeanor(are(excellently(suited(to(her(role.1

SheFs(the(most(heartless(coqueZe(in(France!1 Negative2

Positive2

A-itude+

Page 26: Redefining Sentiment Analysis

The(Structure(of(Sentiment1

8 November, 2011 Sentiment Analysis Symposium, San Francisco

Her(appearance*and*demeanor*are(excellently(suited(to(her(role.1

SheFs(the(most(heartless(coqueZe(in(France!1

Target+

Page 27: Redefining Sentiment Analysis

The(Structure(of(Sentiment1

8 November, 2011 Sentiment Analysis Symposium, San Francisco

Her(appearance*and*demeanor*are(excellently(suited(to(her(role.1

SheFs(the(most(heartless(coqueZe(in(France!1

Superordinate+

Page 28: Redefining Sentiment Analysis

The(Structure(of(Sentiment1

8 November, 2011 Sentiment Analysis Symposium, San Francisco

Her*appearance*and*demeanor*are(excellently(suited(to(her(role.1

SheFs(the(most(heartless(coqueZe(in(France!1

Aspect+

Page 29: Redefining Sentiment Analysis

The(Sentiment(Complex1

8 November, 2011 Sentiment Analysis Symposium, San Francisco

…the(result(was(much(nicer(than(that(Kiva(tent…(1

Jurassic(Park(was(a(great(book(but(a(lousy(movie.(1

aZitude1target11 target21

target1 aZitude11 aZitude21

superordinate21superordinate11

Page 30: Redefining Sentiment Analysis

The(Semantics(of(Sentiment1

8 November, 2011 Sentiment Analysis Symposium, San Francisco

This unique dog or chicken house was made with green practices in mind.

Appreciation2Valuation1

Oh, and on principle I hate extremists / zealots of any ilk.

Affect2Affection1

Oh and Spooks, … with this really tough and sexy old lady in it, kicking ass.

Judgement2Tenacity1

Page 31: Redefining Sentiment Analysis

The(Semantics(of(Sentiment1

8 November, 2011 Sentiment Analysis Symposium, San Francisco

Page 32: Redefining Sentiment Analysis

Other(Key(Pieces1•  Modality

o  Maybe my throat will feel better o  The burgers are probably good but I went with the fish tacos.

•  Evidentiality o  Why are people always so angry? You're trying to prove that your mean

and it isn't cute nor do you need to be on a social website. o  -__- and she thinks that it's funny. BUT IT'S NOT!!! 

•  Sarcasm o  Nice to see Oakland Police attacking protesters again. Of course, it was

"agitators" at fault, not out of control cops.

•  Inferred sentiment o  College Without Weed Is Like A Notebook & No Pen..

8 November, 2011 Sentiment Analysis Symposium, San Francisco

Page 33: Redefining Sentiment Analysis

The(Sentiment(Analysis(Stack1Collections/Streams:(Analysis,(comparison,(trending,(…1

Texts:(Aggregating(and(aZributing(sentiments(from(expressions1

Expressions:(Contextualization(and(interpretation1

Expressions:(Extraction(and(linking1

8 November, 2011 Sentiment Analysis Symposium, San Francisco

Page 34: Redefining Sentiment Analysis

Shlomo Argamon [email protected] @ShlomoArgamon

Subtext3, Inc.

http://www.subtext3.com @Subtext3

Illinois Institute of Technology Linguistic Cognition Lab

http://lingcog.iit.edu

8 November, 2011 Sentiment Analysis Symposium, San Francisco