Introduction Recommender systems Related work A recommender system for Twitter Experiments Conclusions

Towards trust-aware recommender systems

Alberto LumbrerasRicard Gavaldà (Advisor)

July 2012

Introduction Recommender systems Related work A recommender system for Twitter Experiments Conclusions

1. Introduction

2. Recommender systems

3. Related work

4. A recommender system for Twitter

5. Experiments

6. Conclusions

Introduction Recommender systems Related work A recommender system for Twitter Experiments Conclusions

1. Introduction

2. Recommender systems

3. Related work

4. A recommender system for Twitter

5. Experiments

6. Conclusions

Introduction Recommender systems Related work A recommender system for Twitter Experiments Conclusions

Introduction

The paradox of choice: "more is less"

Recommender systems to reduceinformation overload

Social networks informationto enhance recommendations

Introduction Recommender systems Related work A recommender system for Twitter Experiments Conclusions

Aims of the Thesis

Recommend tweetsto users based on their

social network information

Studying the concept of trust in Twitter

Can trust improve tweet recommendations?

What other techniques are useful?

Introduction Recommender systems Related work A recommender system for Twitter Experiments Conclusions

1. Introduction

2. Recommender systems

3. Related work

4. A recommender system for Twitter

5. Experiments

6. Conclusions

Introduction Recommender systems Related work A recommender system for Twitter Experiments Conclusions

Recommender systems

Collaborative Filtering:• Look at "similar users" (similar ratings)• Average their ratings weighted by closeness.

• Sparsity• Cold-start problem

Content Based:• Based on items similarity / machine learning• Features extracted automatically or by experts

• Cold-start problem• What features are relevant?

Trust-aware:• Use trust to enhance recommendation methods

• What is trust?• How to compute and propagate trust?

Introduction Recommender systems Related work A recommender system for Twitter Experiments Conclusions

Computing Trust

Direct trust computation:• Explicit: Users annotations• Implicit: Inferred from users’ behavior and/or interactions

Trust propagation:• Algorithms fit network properties (decay, trust horizon,...)• Network as Markov Chain: PageRank, EigenTrust...• ...

Trust-aware recommendations:• Trust + Collaborative Filtering• Trust + Content Based Filtering• ...

Introduction Recommender systems Related work A recommender system for Twitter Experiments Conclusions

1. Introduction

2. Recommender systems

3. Related work

4. A recommender system for Twitter

5. Experiments

6. Conclusions

Introduction Recommender systems Related work A recommender system for Twitter Experiments Conclusions

Related work

TidalTrust:• Movie recommendation.• Customized algorithm for trust propagation• Direct trust explicitly annotated by users (0-10 range)

• Pro: Do not normalize trust• Con: Requires annotated direct trust

EigenTrust:• Reputation in P2P networks• Direct trust inferred from proportion of successful downloads• Trust propagation by Random Walk• Distributed computation

• Pro: Simple algorithm• Con: Do not explicitly consider network properties

Introduction Recommender systems Related work A recommender system for Twitter Experiments Conclusions

1. Introduction

2. Recommender systems

3. Related work

4. A recommender system for Twitter

5. Experiments

6. Conclusions

Introduction Recommender systems Related work A recommender system for Twitter Experiments Conclusions

Goals and challenges

GOAL:

Recommend tweets to users based on their social network information

CHALLENGES:

In GETTING the data:• APIs limits• Implement a crawler• Crawling criteria

In ANALYZING the data:• No explicit feedback mechanism• Textual items of 140 characters• Items volatility (hours or days)• Very high sparsity (items rated (retweets) by few users)

Introduction Recommender systems Related work A recommender system for Twitter Experiments Conclusions

Contributions

Trust metric (for social networks)

Trust-aware crawler (for social networks)

Recommender system prototype

Analysis of trust properties in Twitter

Introduction Recommender systems Related work A recommender system for Twitter Experiments Conclusions

Computing trust in TwitterInteractions in Twitter

tweet:• @xamat: Markov Random Fields for #recsys: http://t.co/zHggOl2r

mention:• @bob: Nice tutorial @xamat!• @bob: Not a very good tutorial @xamat...• trust or distrust (assumption: mostly trust)

retweet (forward) :• @charles: RT "@xamat: Markov Random Fields for #recsys: http://t.co/zHggOl2r"• @charles: So bad! RT "@xamat: Markov Random Fields for #recsys:http://t.co/zHggOl2r"

• trust or distrust (observation: mostly trust)

favorite:• store a friend’s tweet• observation: mostly trust

follow/unfollow :• users subscribe/unsubscribe to other users publications

Introduction Recommender systems Related work A recommender system for Twitter Experiments Conclusions

Computing trust in TwitterDirect trust

Mentions and retweets as a sign of trust

Trust tij as proportion of interactions of user i with user j .

tij =wN(m)

ij + (1− w)N(rt)ij

Ni(1)

Temporal decay of interactions:

~Ni (t) = λ ~Ni (t) + (1− λ) ~Ni (t − 1) (2)

Introduction Recommender systems Related work A recommender system for Twitter Experiments Conclusions

Computing trust in TwitterTrust propagation through random walk

Step 1: With direct trusts, build transition probabilities matrix P

P =

0 0.2 0.8 00 0 0.5 0.50 0 0 10.3 0.3 0.3 0

Introduction Recommender systems Related work A recommender system for Twitter Experiments Conclusions

Computing trust in TwitterTrust propagation through random walk

Step 2: Propagate trust

T 3 =13

(α1 P︸︷︷︸walk 1 step

+α2 P2︸︷︷︸walk 2 steps

+α3 P3︸︷︷︸walk 3 steps

) (3)

T s =1s

s∑n=1

αnPn (4)

P: direct trust matrix (transition matrix)s: trust horizonαn: path length penalization αn

Note: Assumes transitivity

Introduction Recommender systems Related work A recommender system for Twitter Experiments Conclusions

Architecture

Introduction Recommender systems Related work A recommender system for Twitter Experiments Conclusions

Crawler

Algorithm 1: Crawling cyclewhile True do

S ←− SeedUsers() ∪ TopTrustedUsers()foreach s ∈ S do

statuses ←− GetLastUpdates(s)UpdateInteracctionsMatrix(s, statuses)

endTruncateInteractionsMatrix() /*remove unsignificant users*/UpdateTrustMatrix()UpdateTopTrustedUsers()

end

Introduction Recommender systems Related work A recommender system for Twitter Experiments Conclusions

Recommender

Optional: Tweet expansion.

Optional: bag-of-words or tfPOS/tfNEG ratio.

Tweet candidates from top-trusted neighborhood

Learn to predict retweets.

Introduction Recommender systems Related work A recommender system for Twitter Experiments Conclusions

RecommenderQuery (tweet) expansion

Query expansion:• Bag of words probably not enough. Too few word coincidences between tweets• If expanding the query (i.e: synonyms) more chances to get coincidences• Query expansion proved useful in some scenarios (i.e: for QA systems with searchengines)

Tweet expansion:• Query Bing with tweets• Get first 200 results (summaries)• Add summaries words to tweet

Introduction Recommender systems Related work A recommender system for Twitter Experiments Conclusions

RecommenderScoring: Trust-aware + Content-based

Features:• has_url [True, False]• bag-of-words or tf ratio• trust [0,1]

Label:• retweet [True, False]

Introduction Recommender systems Related work A recommender system for Twitter Experiments Conclusions

1. Introduction

2. Recommender systems

3. Related work

4. A recommender system for Twitter

5. Experiments

6. Conclusions

Introduction Recommender systems Related work A recommender system for Twitter Experiments Conclusions

Dataset

20 target users

6 month crawling

Spanish, Catalan, English

314 instances/user (50% retweets, 50% non-retweets)

70% training, 30% test.

Offline testing (a posteriori prediction of retweets)

Introduction Recommender systems Related work A recommender system for Twitter Experiments Conclusions

Transitivity tests

Normalize trust rankings [0-10]

∆: Disagreement about ranking of common neighbors.

If transitivity: the higher trust on a user, the smaller ∆ between their ratings

Introduction Recommender systems Related work A recommender system for Twitter Experiments Conclusions

Transitivity testsInteractions

Question: are interactions of Twitter users transitive?

Table: Relation of interactions rank and delta

Ranking [0-10] ∆

[0− 1) 1.36[1− 2) 0.75[2− 3) 1.14[3− 4) 0.83[4− 5) 0.78[5− 6) 0.52[6− 7) 1.06[7− 8) 0.53[8− 9) 0.57

[9− 10) 0.17

Agreement about common neighbors (no matter the ∆)

... but we do not see transitivity.

Introduction Recommender systems Related work A recommender system for Twitter Experiments Conclusions

Transitivity testsTrust

different path decays (αn)

Table: Relation of trust rank and delta

Ranking(0-10)

∆No decay

∆Linear decay

∆Exp.decay

(0-1) 1.14 0.90 0.87(1-2) 1.10 1.22 1.09(2-3) 0.96 1.05 1.05(3-4) 1.07 1.18 1.20(4-5) 1.09 0.81 1.01(5-6) 1.13 1.11 0.92(6-7) 0.91 1.08 1.16(7-8) 0.90 1.08 0.99(8-9) 1.3 1.34 1.34(9-10) 1.26 1.11 1.16

we do not see transitivity

Introduction Recommender systems Related work A recommender system for Twitter Experiments Conclusions

Trust-aware recommendationsBenchmarking

trust slightly improves recommendations

Classif. Expand Encode Acc. Recall Precision F1 AUC

Trust

NByes

bow 50.76 59.43 52.07 55.51 50.93tf 52.38 58.99 52.52 55.57 52.58

nobow 48.79 53.01 47.76 50.25 49.88tf 51.97 50.49 50.60 50.44 51.51

SVMyes

bow 45.84 61.00 30.22 40.42 50.53tf 47.63 51.36 46.95 49.06 47.94

nobow 46.25 68.42 32.47 44.04 50.00tf 47.79 46.38 48.44 47.39 47.32

Averages 48.93 56.13 45.13 49.08 50.09

Notrust NB

yesbow 47.02 57.58 48.90 52.89 49.56tf 51.13 49.56 54.81 52.05 52.22

nobow 49.28 47.98 50.76 49.33 50.54tf 46.12 45.18 45.84 45.51 46.28

SVMyes

bow 45.22 55.83 27.05 36.44 50.06tf 49.23 49.36 51.47 59.39 49.80

nobow 43.40 34.87 17.59 23.38 50.22tf 47.11 41.55 48.87 44.91 47.32

Averages 47.31 47.73 43.16 45.49 49.50

Introduction Recommender systems Related work A recommender system for Twitter Experiments Conclusions

1. Introduction

2. Recommender systems

3. Related work

4. A recommender system for Twitter

5. Experiments

6. Conclusions

Introduction Recommender systems Related work A recommender system for Twitter Experiments Conclusions

Contributions and findings

Contributions:

Trust metric

Trust-aware crawler for social networks

Recommender system prototype

Analysis of trust properties in Twitter

Findings:

No transitivity of trust in Twitter or bad trust model...

...but trust model might be useful

Introduction Recommender systems Related work A recommender system for Twitter Experiments Conclusions

Publication

A. Lumbreras, R. Gavaldà, “Applying trust metrics based on user interactions torecommendation in social networks” in Social Knowledge Discovery and UtilizationWorkshop within IEEE/ACM ASONAM’2012, Istambul, August 2012.

Introduction Recommender systems Related work A recommender system for Twitter Experiments Conclusions

Future work

Other query expansion techniques

Further text analysis (e.g: LSA)

Apply temporal decay to tweets

Further study of network properties (trust, interactions, visualization...)

User tests

Study marginal contribution of retweets and mentions

Topic-aware trust (topic detection)

Open question: (how much) transitivity-based models can capture trust on anon-transitive trust network?

Introduction Recommender systems Related work A recommender system for Twitter Experiments Conclusions

Questions?