umap 2011: analyzing user modeling on twitter for personalized news recommendations

DelftUniversity ofTechnology

Analyzing User Modeling on Twitter for Personalized News RecommendationsUMAP, Girona, July 13, 2011

Fabian Abel, Qi Gao, Geert-Jan Houben, Ke TaoWeb Information Systems, TU Delft

2Analyzing User Modeling on Twitter for Personalized News Recommendations

The Social Web

Help me to tackle the

information overload!

Recommend me news articles

that now interest me!

Help me to find interesting (social) media!

Do not bother me with

advertisements that are not

interesting for me!

Give me personalized

support when I do my online training!

Who is this? What are his personal demands? How can we make him happy?

Personalize my Web

experience!


PersonalizedRecommendations

Personalized Search Adaptive Systems

What we do: Science and Engineering for the Personal Web

Social Web

Analysis and User Modeling

user/usage data

Semantic Enrichment, Linkage and Alignment

domains: news social media cultural heritage public data e-learning


User Modeling Challenge

I want my personalized

news recommendatio

ns!Analysis and User Modeling

Semantic Enrichment, Linkage and Alignment

Personalized News Recommender

Profile

?

(How) can we infer a Twitter-based user profile that

supports the news recommender?


User Modeling FrameworkBuilding Blocks for generating valuable user profiles

(a)hashtag-based(b)entity-based(c)topic-based

2. Profile Type

1. Temporal

Constraints

3. Semantic

Enrichment4.

Weighting Scheme

(a)time period(b)temporal patterns

(a)tweet-based(b)further enrichment

(a)concept frequency


User Modeling Building Blocks

Profile?concept weight

?

time

1. Which tweets of the user should be

analyzed?

Morning:Afternoon:Night:

1. Temporal

Constraints

June 27 July 4 July 11

(b) temporal patterns

weekendsstart

end

(a) time period




2. Profile Type

Francesca Schiavone won French Open #fo2010 ?

Francesca Schiavone

FrenchOpen

Francesca Schiavone French Open entity-

based

SportT

T topic-based

2. What type of concepts should represent

“interests”?

# fo2010

#fo2010# hashtag-

based

1. Temporal

Constraints

time





2. Profile Type

Francesca Schiavone won! http://bit.ly/2f4t7a

Francesca Schiavone

3. Further enrich the semantics of tweets?

1. Temporal

Constraints

3. Semantic

Enrichment

Francesca Schiavone

Francesca wins French Open

Thirty in women'stennis is primordially old, an age when agility and desire recedes as the …

French Open

Tennis

French OpenTennis

(b) further enrichment

(a) tweet-based



Profile? concept weight

2. Profile Type

4. How to weight the concepts?

1. Temporal

Constraints

3. Semantic

Enrichment

Francesca Schiavone

French OpenTennis

4. Weighting Scheme

time


?

weight(Francesca Schiavone)

Concept frequency

4

weight(French Open)

weight(Tennis)

36




2. Profile Type

1. Temporal

Constraints

3. Semantic

Enrichment4.

Weighting Scheme





AnalysisHow do the user modeling building blocks impact the (temporal) characteristics of Twitter-based user profiles?


2. Profile Type

1. Temporal

Constraints

3. Semantic

Enrichment4.

Weighting Scheme





Dataset

timeNov 15 Dec 15 Jan 15

20,000 Twitter users

10,000,000 tweets

2 months

more than:

75,000 news articles

WikiLeaks founder, Julian Assange, under arrest in

London


Size of user profiles

entity-based

topic-basedhashtag-based

~5% of the users do not make use of hashtags hashtag-based profiles are empty

Entity-based user modeling succeeds for 100% of the users

Profile Type


Tweet-based

further enrichment(e.g. exploiting links)

topic-based user profiles

More distinct entities per profile

further enrichment(e.g. exploiting links)

Tweet-based

entity-based user profiles

Impact of Semantic Enrichment

Exploiting external resources allows for significantly richer user profiles (quantitatively)

More distinct topics per profile

Semantic Enrichment


?

User Profiles change over time

d1-distance:

difference between current profile and

past profile

€

d1(r p x,

r p current ) = | px,i − pcurrent ,i |

i∑

Example:

€

d1(

0.5

0.5

0

⎛

⎝

⎜ ⎜ ⎜

⎞

⎠

⎟ ⎟ ⎟,

0.5

0

0.5

⎛

⎝

⎜ ⎜ ⎜

⎞

⎠

⎟ ⎟ ⎟) =1

music

football

tennis

old new

Hashtag-based profiles change stronger than entity-based and topic-based profiles

#

T

The older the profile the more it differs from the current profile

Temporal Constraint

s


Temporal patterns of user profiles

topic-based user profiles

weekday vs. weekend profilesd1(pweekday, pweekend)

day vs. night profilesd1(pday, pnight)

1. Weekend profiles differ significantly from weekday profiles

2. the difference is stronger than between day and night profiles

2

Temporal Constraint

s


Observations

• Semantic enrichment allows for richer user profiles

• Profiles change over time: fresh profiles seem to better reflect current user demands

• Temporal patterns: weekend profiles differ significantly form weekday profiles


EvaluationHow do the user modeling building blocks impact the quality of Twitter-based profiles for personalized news recommendations?


2. Profile Type

1. Temporal

Constraints

3. Semantic

Enrichment4.

Weighting Scheme




And can we benefit from the findings of the analysis to improve recommendations?


Twitter-based Profiles for Personalization

• Task: Recommending news articles (= tweets with URLs pointing to news articles)

• Recommender algorithm: cosine similarity between user profile and tweets

• Ground truth: re-tweets of users• Candidate items: news article tweets posted

during evaluation period

time

P(u)= ?

1 week

Recommendations = ?

5.5 relevant tweets per user

5529 candidate news articles


Overview: Performance of User Modeling strategies

Entity-based strategy improves the recommendation quality significantly (MRR & S@10)

Topic-based strategy improves S@10 significantly

T

#

Profile Type


Impact of Semantic Enrichment

Tweet-based

Further enrichment

Further semantic enrichment (exploiting links) improves the quality of the Twitter-based profiles!

T

Semantic Enrichment


Impact of temporal characteristics

Selection of temporal constraints depends on type of

user profile.

•Topic-based profiles: adapting to temporal context is beneficial• Entity-based profiles: long-term profiles perform better

Adapting to temporal context helps?

yes

no

yes

no

T

T

time

startcomplet

eend

complete: 2 months

Recommendations = ?

startfresh

fresh: 2 weeks

time

start end

Recommendations = ?

weekends

Temporal Constraint

s


Conclusions and Future Work

• What we did: Twitter-based User Modeling for Recommending News Articles

• Analysis: • Semantic enrichment results in richer user profiles (quantitative)• User interest profiles change over time (hashtag-based stronger

than others)• Weekend/weekday pattern more significant than day/night pattern

• Evaluation:• Best user modeling strategy: Entity-based > topic-based > hashtag-

based • Semantic enrichment improves recommendation quality• Adapting to temporal context helps for topic-based strategy

• Future work: for what type of personalization tasks can we exploit what type of Twitter profiles?


Thank you!

Fabian Abel, Qi Gao, Geert-Jan Houben, Ke Tao

Twitter: @perswebhttp://persweb.org/ http://u-sem.org/


Research Questions

1. What type of user interest profiles can we infer from Twitter activities?

2. Can we exploit Twitter-based profiles for personalizing users’ Social Web experience?

interest

?

Personalized news recommendationsin time:

time?

time

Good Morning! #tooearly

I like this http://bit.ly/5d4r2t

Why do people now blame Julian Assange?

Ajax deserves it! #sport

twitter


I like this http://bit.ly/4Gfd2

Analyzing Twitter-based Profiles for Personalized News Recommendations (in time)

Analysis and User Modeling

tweets

Semantic Enrichment, Linkage, Alignment

Francesca Schiavone is great!

Thirty in women'stennis is primordially old, an age when agility and desire recedes as the next wave of younger/faster/stronger players encroaches. It's uncommon for any athlete to have a breakthrough season at 30, but it's exceedingly…

topic:Tennis

oc:Sportsevent:FrenchOpen

dbpedia:Schiavone

Interests:TennisFootball

interest

time

Personalized news recommendations

News Recommendations in time:

interest

time

Ajax gives De Jong a breakAjax manager Frank deBoer announced that…

Nice, thank you!


User Modeling Challenge

Profile?

Personalized news recommender

I want my personalized

news recommendatio

ns!

Wednesday, July 13th 2011, 9:10am

(How) can we infer a Twitter-based user profile that

supports the news recommender?

?


time

Bob tweets…

Fr, 6am

Good Morning! #tooearly

Why do people now blame Julian Assange?

Fr, 3pm Fr, 8pm

I like this http://bit.ly/5d4r2t

Sa, 5pm

Ajax deserves it! #sport

People publish more than 60 million tweets per day!

umap 2011: analyzing user modeling on twitter for personalized news recommendations

Technology

based user modeling

twitterbased user profile

user modeling building

valuable user profiles6

fresh profiles

semantic enrichmentprofile

night profiles weekday

b temporal patternsprofile