HCI Korea 2012 | Invited Talk!

Model-Driven Research in Social Computing!!

Ed H. Chi!!

Google Research!!Work done while at Palo Alto Research Center (PARC)!

!

!

2012-01-12 HCI Korea 2012 Invited Talk 1

It started with a search engine

This is also Google...

(and much, much more). 2012-01-12 HCI Korea 2012 Invited Talk 3

To organize the world’s information and make it

universally accessible and useful.

Our mission remains the same

1.8 billion internet users worldwide

4.6 billion mobile subscribers

800 exabytes of digital information in the world -- videos, photos, music, texts, etc.

2012-01-12 HCI Korea 2012 Invited Talk 5

!"#$%&""'($%!")*+(%!,+,-%!  !"#$###%&'()*%+(,%&(-..,/%,+01%2-/3.,%'/%45'66,(%7%

.1+.8*%9:#%2-55-'/%&'()*%+%)+;%

!  <=,(;%>:%*,0'/)*%?,'?5,%=-,&%,/'361%?1'.'*%@('2%

A-0+*+%B,C%D5C32*%.'%0'=,(%+/%,/.-(,%@''.C+55%E,5)%

!  <=,(;%F%2-/3.,*$%2'(,%?1'.'*%+(,%=-,&,)%'/%A-0+*+%

B,C%D5C32*%.1+/%,G-*.%-/%.1,%,/.-(,%H-2,7IJK<%?1'.'%

0'55,0.-'/%

2012-01-12 HCI Korea 2012 Invited Talk 7

."/0/1$%!,+,-%!  >"#%;,+(*%'@%L'3H3C,%=-),'%+(,%&+.01,)%,=,(;)+;%'/%

K+0,C''M%N3?%!O"G%;P;Q%

!  ,=,(;%2-/3.,%R##S%.&,,.*%0'/.+-/%L'3H3C,%5-/M*%%

T  N3?%9G%;P;Q%UV>%!#>>>W%

!  >##XS%?,'?5,%.+M,%+%*'0-+5%+0.-'/%&-.1%L'3H3C,%N5-M,*$%

*1+(,*$%0'22,/.*$%,.0Q%,=,(;%&,,M%N>#P>"P>#Q%

2012-01-12 HCI Korea 2012 Invited Talk 8

&""'($2%!,+,-%!  Y=,(%R#%2-55-'/%?,'?5,%Z'-/,)%[''65,%*-/0,%5+3/01O%

!  Y=,(%9OR%C-55-'/%?1'.'*%1+=,%+5(,+);%C,,/%*1+(,)%-/%

[''65,SO%

!  A,'?5,%+(,%!G79G%.-2,*%2'(,%5-M,5;%.'%*1+(,%0'/.,/.%&-.1%

'/,%'@%.1,-(%0-(05,*%.1+/%.'%2+M,%+%?3C5-0%?'*.O%

2012-01-12 HCI Korea 2012 Invited Talk 9

What can you do with all this data?

Google Trends Trendalyzer

Big Data Analytics!! Google Analytics Google Website Optimizer

2012-01-12 HCI Korea 2012 Invited Talk 10

Model-Driven and Living Laboratory Approach

Characterization

and Modeling

Unlock Understanding

of Collective Intelligence

Intelligent UI

and Data-mining

Applications / Products Living Laboratory

Productization

11

0+(3%*4%5%6),-%

!  D0.%J\%H1,%J/=-*-C5,%

T  ]'0-+5%],+(01%!  D0.%JJ\%H1,%^-*-C5,%

T  ]'0-+5%4('&*-/6%=-+%]1+(,)%D//'.+.-'/*%

!  D0.%JJJ\%H1,%DC*.(+0.,)%

T  ]1+(,)%_/'&5,)6,%]?+0,%

2012-01-12 HCI Korea 2012 Invited Talk

H1,%J2?'(.+/0,%'@%]'0-+5%]-6/+5*%-/%J/@'(2+.-'/%D00,**%

12

6),%78%%749*-*1($%!")*+(%!*'4+(-%:;"#%,<$%=;">?%

`'-/.%&'(M%&P%H'))%X;.M'&-0a$%b'&+/%c+-(/$%I+&(,/0,%I,,%

%

Ud1-%+/)%X;.M'&-0a$%e;?,(.,G.!##FW%

U_+22,(,(%,.%+5O$%deJ!##fW%

%

2012-01-12 HCI Korea 2012 Invited Talk

@;$(/?$8%6%A$;-"4+(%($+;4*4'%-,";B%H'\%01-g+02O'(6%

K('2\%4(+)%4+((-*1%hC(+)gi(,2'=,)O@'(O?(-=+0;iO0'2j%

]3CZ,0.\%A+/0(,+.-0%0+/0,(%

k+.,\%H13$%>%K,C%!##l%!>\9l\""%A]H%

%

e,;%<)O%Jm2%+%@,55'&%),5O-0-'O3*%3*,(%+/)%/'.-0,)%;'3%C''M2+(M%+%5'.%%%

'@%?+/0(,+.-0%0+/0,(%*.3nO%Jm2%+.%1'2,%&-.1%2;%)+)%&1'%&+*%)-+6/'*,)%%%

+%5-..5,%'=,(%+%;,+(%+6'%+/)%-*%/'&%+.%.1,%.+5,%,/)%'@%.1-/6*O%Jm=,%%%

5,+(/,)%+%5'.%.1('361%1-*%.(,+.2,/.*%+/)%+C'3.%&1+.m*%'3.%.1,(,O%J%%%

)3//'%-@%-.m*%*'2,.1-/6%;'3%'(%+%@+2-5;%2,2C,(%1+*$%C3.%Z3*.%&+/.,)%%%

.'%)('?%;'3%+/%,2+-5O%4,%&,55O%

%

4(+)%

14

C-*4'%74:";#+,*"4%0<$";B%,"%D"?$(%!")*+(%0+''*4'%EF?%GH%=<*I%0"??%DB,3">*)JI%6=D%GBA$;,$K,%LMMNO%

!"#$%&'(")%*#+&'

,&*-&' ."%/0*)+&'

!12&'!"#!$%

3)%"4$)2'.*%"4$)2'

&'()*%

2012-01-12 HCI Korea 2012 Invited Talk 15

0+''*4'%P$<+9*";%

GQ0+'R%-<">-%,+'%-+,/;+,*"4% GQS")%T%0+'RI%1;">-+1*(*,B%

2012-01-12 HCI Korea 2012 Invited Talk 16

7#A(*)+,*"4%

!Q"#$U%%&'R%%D/,/+(%74:";#+,*"4% V+*-$%*4%+9'H%,+'%W%1""3#+;3%

2012-01-12 HCI Korea 2012 Invited Talk 17

0+'!$+;)<8%D+AV$?/)$%7#A($#$4,+,*"4%

!  ]?(,+)-/6%D0.-=+.-'/%-/%+%C-76(+?1%

!  d'2?3.+.-'/%'=,(%+%=,(;%5+(6,%)+.+%*,.%

T  >"#%X-55-'/S%C''M2+(M*%

Tags URLs

P(URL|Tag)

P(Tag|URL)

2012-01-12 HCI Korea 2012 Invited Talk 18

2012-01-12

Guide

Web

Howto

Tips Help

Tools

Tip

Tricks

Tutorial

Tutorials

Reference

Semantic Similarity Graph

0+'!$+;)<8%C-$%!$#+4,*)%64+(B-*-%,"%V$?/)$%X"*-$%%%%%<,,A8WW#;,+''BH)"#%%

HCI Korea 2012 Invited Talk 19

2012-01-12 HCI Korea 2012 Invited Talk 20

FKA$;*#$4,%S$-*'4%%EY+##$;$;%$,%+(H%=G7LMMZO%

!  !%-/.,(@+0,%G%9%.+*M%)'2+-/%),*-6/%

T  !%J/.,(@+0,%NC,.&,,/7*3CZ,0.*Q%

!  <G?5'(+.'(;%=*O%4+*,5-/,%

T  9%.+*M%)'2+-/*%N&-.1-/7*3CZ,0.*Q%

!  K3.3(,%D(01-.,0.3(,$%[5'C+5%B+(2-/6$%B,C%X+*13?*%

!  9#%]3CZ,0.*%N!!%2+5,$%F%@,2+5,Q%

T  J/.,(2,)-+.,%'(%+)=+/0,)%0'2?3.,(%+/)%&,C%*,+(01%*M-55*%

T  e+5@%+**-6/,)%<G?5'(+.'(;$%1+5@%4+*,5-/,O%!  K'(%,+01%)'2+-/$%*-/65,%C5'0M%&-.1%9%.+*M%.;?,*\%

T  <+*;%+/)%k-o035.%A+6,%d'55,0.-'/%H+*M%U:2-/%,+01W%

T  ]322+(-a+.-'/%H+*M%U>!2-/W%

T  _,;&'()%[,/,(+.-'/%H+*M%U!2-/W%

2012-01-12 HCI Korea 2012 Invited Talk 21

F9+/+,*"4%V$-/(,-%EY+##$;$;%$,%+(HI%=G7LMMZO%!  <G?5'(+.'(;%-/.,(@+0,%3*,(*\%

T  ?,(@'(2,)%2'(,%p3,(-,*$%%

T  .''M%2'(,%.-2,$%%

T  &('.,%C,..,(%*322+(-,*%N-/%!P9%)'2+-/*Q$%%

T  6,/,(+.,)%2'(,%(,5,=+/.%M,;&'()*%N-/%!P9%)'2+-/*Q$%+/)%

T  1+)%+%1-61,(%0'6/-.-=,%5'+)O%!  ]366,*.-=,%'@%),,?,(%,/6+6,2,/.%+/)%C,..,(%5,+(/-/6O%

!  ]'2,%,=-),/0,%'@%*0+n'5)-/6%@'(%/'=-0,*%-/%.1,%M,;&'()%

6,/,(+.-'/%+/)%*322+(-a+.-'/%.+*M*O%

2012-01-12 HCI Korea 2012 Invited Talk 22

ZeroZero88.com: !Social Contextual Recommendation Engine"

Social Recommender

Engine

2010-10-22 23 IBM NPUC 2010

ZeroZero88 Recommender System"

My Friends’ URLs!

Popular URLs!

Recommendation Algorithm: Combining Sources and

Models!

Recommendations!

My Friends’ Network and Tweeting Pattern!

Social Ranking !Model!

My Tweets!

My Friends’ Tweets!

Topic Relevance Model!

2010-10-22 24 IBM NPUC 2010

6),%778%[*-*1($%!")*+(%!*'4+(-%:;"#%%!<+;$?%G*'<(*'<,*4'%

%

%

_3)'*%.'%I-01+/%e'/6$%I,*%c,5*'/%%

Ue'/6%,.%+5$%D^J!##FW%

Uc,5*'/%,.%+5O$%edJJ%!##fW%

\*4?*4'%+%V$-,+/;+4,%

!  D??('?(-+.,%@'(%

.1,%'00+*-'/%

2012-01-12 HCI Korea 2012 Invited Talk 26

Poor heuristic

Good heuristic

G$/;*-,*)-%

2012-01-12 HCI Korea 2012 Invited Talk 27

]G*4,-^%

Solo

Cooperative (“good hints”)

2012-01-12 HCI Korea 2012 Invited Talk 28

!A+;0+'H/-8%!")*+(%G*'<(*'<,*4'%%EG"4'%_%=<*I%=G7LMMZO%

!A+;0+'H/-8%!")*+(%G*'<(*'<,*4'%

!  !"#$%&'%.+66-/6%&1-5,%(,+)-/6%

!  e-615-61.-/6%

!  ]1+(,)%/'.,C''M-/6%%

!  ]1+(-/6q%

G*'<(*'<,*4'%+-%7#A";,+4)$%%74?*)+,";%

recall first-visit

!  ],/*,2+M-/6%.+*M%

T  K-/)%+/)%(,+)%2+.,(-+5%+C'3.%“</.,(?(-*,%!O#%2+*13?*”%-/%'(),(%.'%&(-.,%.&'%,**+;*%

!  ],,)*\%“,G?,(.”%0'/.,/.%@'(%*0+n'5)-/6%

T  H+6*%@('2%),5O-0-'O3*%

T  rbI*%@('2%[''65,PA+6,b+/M%

T  d'/*.(30.,)%+/)%.1,/%*1+(,)%.1('361%*'0-+5%2,01+/-*2*%N-O,O$%+%

]?+(H+6O3*%“@(-,/)”Q%

!  A,(@'(2+/0,%X,+*3(,*%

T  I,+(/-/6%6+-/\%A(,PA'*.%_/'&5,)6,%H,*.%

%

<=+53+.-'/%H+*M%s%X,.(-0%Uc,5*'/%,.%+5O$%edJJ!##fW%

scorePretest - scoreMax scorePretest -scorePosttest

=Gain

@;")$?/;$%

k,2'6(+?1-0*%

s%A(,.,*.%

SF ]?+(H+6O3*%

&-.1%‘K(-,/)’%

SO ]?+(H+6O3*%

Y/5;%

WS B-.1'3.%

]?+(H+6O3*%

A'*..,*.%

V$-/(,-8%`$+;4*4'%&+*4%

ct>F%%

!A+;0+'H/-%2%\;*$4?%-/A$;*";%,"%1",<%*4?*9*?/+(%)"4?*,*"4-%c'%)-n,(,/0,%C,.&,,/%.1,%.&'%0'/.('5%0'/)-.-'/*%

2012-01-12 HCI Korea 2012 Invited Talk 34

URL Kind Code Blog B Conference C Employment E MySpartagus M News N OpenSource O Search S Vendor V Wikipedia W Consultant X

a1-$;9+,*"4%rbI%_-/)% d'),%

45'6% 4%

d'/@,(,/0,% d%

<2?5';2,/.% <%

X;O]?+(.+6O3*% X%

c,&*% c%

Y?,/]'3(0,% Y%

],+(01% ]%

^,/)'(% ^%

B-M-?,)-+% B%

d'/*35.+/.% u%

2012-01-12 HCI Korea 2012 Invited Talk 35

["4%V$-,";b%7-"(+,*"4%Fb$),%EcZ55O%!  D*%+??5-,)%.'%1-615-61.*$%.1,%(#)*+,-.#/0*1-#2&.1#)*,0,$.*

*366,*.*%.1+.%(,+),(*\%

!  ()*#&+",#&-#.-/'$#-"#+/)%%!  (0*#1+)2"#34)&#%$#5)26+,$%%!  &1,.1,(%.1,%-/@'(2+.-'/%-*%-2?'(.+/.%'(%/'.O%

T  c-*.%+/)%e'6(,C,%Fl%

2012-01-12 HCI Korea 2012 Invited Talk 36

6),%7778%%61-,;+),$?%Y4">($?'$8%0<$%!)*$4)$%":%C4?$;-,+4?*4'%d*3*A$?*+%

_3)'*%.'%4'/6&'/%]31$%c-M-%_-..3(%

%

U_-..3(%,.%+5O$%deJ!##lW%

U]31%,.%+5O$%B-M-];2%!##fW%

%

FKA"4$4,*+(%&;">,<%":%d*3*A$?*+8%+4%+))$A,$?%e:+),f%

Number of Articles (Log Scale)

http://en.wikipedia.org/wiki/Wikipedia:Modelling_Wikipedia’s_growth

2012-01-12 HCI Korea 2012 Invited Talk 38

&;">,<%":%F?*,-%

2012-01-12 HCI Korea 2012 Invited Talk 39

!"#$,<*4'%<+AA$4$?%*4%$+;(B%LMMg%

2012-01-12 HCI Korea 2012 Invited Talk 40

&;">,<%":%6),*9$%F?*,";-%

*In thousands

2012-01-12 HCI Korea 2012 Invited Talk 41

!(">*4'%&;">,<%*4%&("1+(%6),*9*,B%

*In thousands

2012-01-12 HCI Korea 2012 Invited Talk 42

F+;(*$;%FKA"4$4,*+(%&;">,<%D"?$(%!  A(,@,(,/.-+5%D..+012,/.\%<)-.*%C,6,.%,)-.*%

T  2'(,%/32C,(%'@%?(,=-'3*%,)-.*$%2'(,%/32C,(%'@%/,&%,)-.*%

Growth rate of population

Current population

Growth rate depends on: N = current population r = growth rate of the population

2012-01-12 HCI Korea 2012 Invited Talk

!

dNdt

= r " N

!

N(t) = N0 " ert

43

`"'*-,*)%&;">,<%D"?$(%!  <0'5'6-0+5%?'?35+.-'/%6('&.1%2'),5%

T  D5*'%),?,/)%'/%,/=-('/2,/.+5%0'/)-.-'/*%

T  _$%0+((;-/6%0+?+0-.;%N)3,%.'%(,*'3(0,%5-2-.+.-'/Q%

!

dNdt

= rN(1" NK)

2012-01-12 HCI Korea 2012 Invited Talk 44

D+,)<%,"%S+,+8%h%":%X$>%6;,*)($-%!  K'55'&*%+%5'6-*.-0%6('&.1%03(=,%

New Article

2012-01-12 HCI Korea 2012 Invited Talk 45

!,;/''($%:";%FK*-,$4)$%i%S+;>*4%!  4-'5'6-0+5%*;*.,2%

T  d'2?,.-.-'/%-/0(,+*,*%+*%

?'?35+.-'/%1-.%.1,%5-2-.*%'@%.1,%

,0'5'6;%

T  D)=+/.+6,%6'%.'%2,2C,(*%'@%.1,%

?'?35+.-'/%.1+.%1+=,%0'2?,.-.-=,%

)'2-/+/0,%'=,(%'.1,(*%

!  D/+5'6;%

T  I-2-.,)%'??'(.3/-.-,*%.'%2+M,%

/'=,5%0'/.(-C3.-'/*%

T  J/0(,+*,)%?+..,(/*%'@%0'/v-0.%+/)%)'2-/+/0,%%

2012-01-12 HCI Korea 2012 Invited Talk 46

]!<">$;*4'^%GBA",<$-*-%B1+.%)(-=,*%0'/.(-C3.-'/*%.'%B-M-?,)-+w%

d''?,(+.-'/%-*%/'.%.1,%2+-/%)(-=,(w%

!  e;?'.1,*-*\%d'/v-0.*%)(-=,*%2'*.%'@%.1,%0'/.(-C3.-'/*O%

T  e'&%)'%&,%2,+*3(,%0'/v-0.*w%

!  d'/v-0.*%0+3*,%0''()-/+.-'/%0'*.*%.'%6'%3?O%

T  e'&%.'%2,+*3(,%0''()-/+.-'/%0'*.*w%

!  x/,6'.-+.-'/%-*%0(-.-0+5%.'%1,5?-/6%235.-?5,%?,(*?,0.-=,*%

.'%0'/=,(6,%'/%*1+(,)%M/'&5,)6,Oy%%

T  ].+15$%[('3?%d'6/-.-'/$%d1F$%!##R%

2012-01-12 HCI Korea 2012 Invited Talk 47

2012-01-12

="4j*),W="";?*4+,*"4%Fb$),-%*4%d*3*A$?*+%

60%

65%

70%

75%

80%

85%

90%

95%

100%

2001 2002 2003 2004 2005 2006

Perc

enta

ge o

f to

tal edits

Article

User

Article Talk

User Talk

Other

Maintenance

(Kittur, Suh, Pendleton, Chi, CHI2007)

HCI Korea 2012 Invited Talk 48

V+,*"%":%V$9$;,$?%="4,;*1/,*"4-%%

Monthly Ratio of Reverted Edits

2012-01-12 HCI Korea 2012 Invited Talk 49

[*-/+(%64+(B,*)-%"9$;%d*3*A$?*+%?+,+%D$?*+,";%@+,,$;4%i%0$;;*%!)<*+9" % %E!/<I%$,%+(HI%[6!0LMMgO%

Mediators

Sympathetic to parents

Sympathetic to husband

Anonymous (vandals/spammers)

2012-01-12 HCI Korea 2012 Invited Talk 50

d*3*S+-<1"+;?H)"#%

2012-01-12 HCI Korea 2012 Invited Talk 51

d<+,%S*?%d$%`$+;4k%!  H1,%d'22'/%H1(,+)\%

T  r.-5-a+.-'/%'@%]'0-+5%]-6/+5*%@'(%I,+(/-/6%+/)%J/@'(2+.-'/%D00,**%

T  B1,.1,(%-.%-*%-/=-*-C5,$%=-*-C5,$%+/)%+C*.(+0.,)O%

!  H1,%<*.+C5-*12,/.%'@%d'22'/%[('3/)%

T  J2?5-0-.%d''()-/+.-'/%

T  <G?5-0-.%d''()-/+.-'/%T  c,6'.-+.-'/%

!  xD55%0'55,0.-=,%+0.-'/*%+(,%C3-5.%'/%0'22'/%6('3/)%+/)%

-.*%+003235+.-'/Oy%

T  d5+(M%+/)%4(,//+/$%>ff>%

2012-01-12 HCI Korea 2012 Invited Talk 52

`">$;*4'%@+;,*)*A+,*"4%W%74,$;+),*"4%="-,-%

!  J/.,(+0.-'/%0'*.*%

),.,(2-/,%/32C,(%'@%

?,'?5,%&1'%?+(.-0-?+.,%

!  ]3(?53*%'@%+..,/.-'/%s%

2'.-=+.-'/%+.%*2+55%

.(+/*+0.-'/%0'*.*%

!  74+2+.-2+i%

!  J2?'(.+/.%.'%M,,?%

-/.,(+0.-'/%0'*.*%5'&%

Cost of participation #

Peop

le w

illin

g to

pro

duce

for “

free

”

2012-01-12 HCI Korea 2012 Invited Talk 53

0<+43%B"/l%!  01-g+02O'(6%

!  1..?\PP,)01-O/,.%

2012-01-12 HCI Korea 2012 Invited Talk 54

2012-01-12 HCI Korea 2012 Invited Talk

C-*4'%D+)<*4$%`$+;4*4'%,"%S$,$),%="4j*),-%!  d'3/.-/6%zd'/.('=,(*-+58%5+C,5*%

!  "G%0('**7=+5-)+.-'/$%b!%t%#OFfl%

0

1000

2000

3000

4000

5000

6000

7000

8000

9000

10000

0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000

Predicted controversial revisions

Actu

al c

ontr

over

sial r

evisi

ons

55

="?+8%6%=<+(($4'$8%6%#"?*m$?%("'*-,*)%#"?$(%!  d+((;-/6%d+?+0-.;%+*%+%@3/0.-'/%'@%.-2,O%

2012-01-12 HCI Korea 2012 Invited Talk 56