lecture 5: personalization on the social web (2014)
DESCRIPTION
This is the fifth lecture in the Social Web course (2014) at the VU University Amsterdam. Visit the website for more information: http://thesocialweb2014.wordpress.com/TRANSCRIPT
Lecture V Personalization on the Social Web (some slides adopted from Fabian Abel)
Lora Aroyo The Network Institute13
VU University Amsterdam
Social Web 2014
theory amp techniques for 13how to design amp evaluate 13
recommenders amp user models 13to use in Social Web applications
Social Web 2014 Lora Aroyo
Fig 1 Functional model of tasks and sub-tasks specifically suited for SASs
Fig 1 Functional model of tasks and sub-tasks specifically suited for SASs (Ilaria Torre 2009)
Social Web 2014 Lora Aroyo
Kevin Kelly
How to infer amp represent 13user information that supports a given
application or context
User Modeling
Social Web 2014 Lora Aroyo
bull Application has to obtain understand amp exploit information about the user13
bull Information (need amp context) about user13
bull Inferring information about user amp representing it so that it can be consumed by the application13
bull Data relevant for inferring information about user
User Modeling Challenge
Social Web 2014 Lora Aroyo
bull People leave traces on the Web and on their computers
bull Usage data eg query logs click-through-data 13
bull Social data eg tags (micro-)blog posts comments bookmarks friend connections 13
bull Documents eg pictures videos13
bull Personal data eg affiliations locations 13
bull Products applications services - bought used installed13
bull Not only a userrsquos behavior but also interactions of other users
bull ldquopeople can make statements about merdquo13
bull ldquopeople who are similar to me can reveal information about merdquo13
bull ldquosocial learningrdquo collaborative recommender systems
Social Web 2014 Lora Aroyo
User amp Usage Data is Everywhere
bull User Profile = data structure = a characterization of a user at a particular moment represents what from a given (system) perspective there is to know about a user The data in the profile can be explicitly given by user or derived by system13
bull User Model = definitions amp rules for the interpretation of observations about the user amp about the translation of that interpretation into the characteristics in a user profile user model is the recipe for obtaining amp interpreting user profiles13
bull User Modeling = the process of representing the user
Social Web 2014 Lora Aroyo
UM Basic Concepts
bull Overlay User Modeling describe user characteristics eg ldquoknowledge of a userrdquo ldquointerests of a userrdquo with respect to ldquoidealrdquo characteristics13
bull Customizing user explicitly provides amp adjusts elements of the user profile13
bull User model elicitation ask amp observe the user learn amp improve user profile successively ldquointeractive user modelingrdquo13
bull Stereotyping stereotypical characteristics to describe a user13
bull User Relevance Modeling learninfer probabilities that a given item or concept is relevant for a user
Related scientific conference httpumap2011org Related journal httpumuaiorg
Social Web 2014 Lora Aroyo
User Modeling Approaches
httpfarm7staticflickrcom62406346803873_e756dd9bae_bjpgSocial Web 2014 Lora Aroyo
Which approach suits best the conditions of
applications
bull among the oldest user models13
bull used for modeling student knowledge13
bull the user is typically characterized in terms of domain concepts amp hypotheses of the userrsquos knowledge about these concepts in relation to an (ideal) expertrsquos knowledge13
bull concept-value pairs
Social Web 2014 Lora Aroyo
Overlay User Models
bull Ask the user explicitly learn13bull NLP intelligent dialogues13bull Bayesian networks Hidden Markov models13
bull Observe the user learn 13bull Logs machine learning13bull Clustering classification data mining
bull Interactive user modeling mixture of direct inputs of a user observations and inferences
Social Web 2014 Lora Aroyo
User Model Elicitation
httphunchcom
Social Web 2014 Lora Aroyo
httpfarm1staticflickrcom155413650229_31ef379b0b_bjpg
bull set of characteristics (eg attribute-value pairs) that describe a group of users13
bull user is not assigned to a single stereotype - user profile can feature characteristics of several different stereotypes
Social Web 2014 Lora Aroyo
User Stereotypes
based on slides from Fabien Abel
User Modeling (4 building blocks)
Semantic Enrichment Linkage and Alignment
Personalized News Recommender
Profile
I want my
personalized news recommendations
based on slides from Fabien Abel
Can we infer a Twitter-based User Profile
User Modeling Building Blocks
Profile concept weight
time
1 Which tweets of the user should be
analyzed
Morning Afternoon Night
1 Temporal Constraints
June 27 July 4 July 11
(b) temporal patterns
weekends start end
(a) time period
based on slides from Fabien Abel
User Modeling Building Blocks
Profile concept weight
2 Profile Type
Francesca Schiavone won French Open fo2010
Francesca Schiavone
French Open
Francesca Schiavone French Open entity-based
Sport T
T topic-based
2 What type of concepts should represent ldquointerestsrdquo
fo2010
fo2010 hashtag-based
1 Temporal Constraints
time
June 27 July 4 July 11
based on slides from Fabien Abel
User Modeling Building Blocks
Profile concept weight
2 Profile Type
Francesca Schiavone won httpbitly2f4t7a
Francesca Schiavone
3 Further enrich the semantics of tweets
1 Temporal Constraints
3 Semantic Enrichment
Francesca Schiavone
Francesca wins French Open Thirty in womens tennis is primordially old an age when agility and desire recedes as the hellip
French Open
Tennis
French Open
Tennis
(b) further enrichment
(a) tweet-based
based on slides from Fabien Abel
User Modeling Building Blocks
Profile concept weight
2 Profile Type
4 How to weight the concepts
1 Temporal Constraints
3 Semantic Enrichment
Francesca Schiavone
French Open
Tennis
4 Weighting Scheme
time
June 27 July 4 July 11
weight(Francesca Schiavone)
Concept frequency (TF)
4
3 6
TFxIDF Time-sensitive
weight(French Open)
weight(Tennis)
based on slides from Fabien Abel
Social Web 2014 Lora Aroyo
Observationsbull Profile characteristics
bull Semantic enrichment solves sparsity problems13bull Profiles change over time recent profiles reflect better
current user demands13bull Temporal patterns weekend profiles differ significantly
from weekday profiles13
bull Impact on recommendations bull The more fine-grained the concepts the better the
recommendation performance entity-based gt topic-based gt hashtag-based 13
bull Semantic enrichment improves recommendation quality 13bull Time-sensitivity (adapting to trends) improves
performance
Social Web 2014 Lora Aroyo
User Modelingit is not about putting everything in a user profile 13
it is about making the right choices
Social Web 2014 Lora Aroyo
User AdaptationKnowing the user to adapt a system or interface13
to improve the system functionality and user experience
A Jameson Adaptive interfaces and agents The HCI handbook fundamentals evolving technologies and emerging applications pp 305ndash330 2003
User-Adaptive Systems
user modeling
user profile
observations data and information about user
profile analysis
adaptation decisions
user modeling (infer current musical taste)
user profile interests in
genres artists tags
history of songs like ban pause skip
compare profile with possible next
songs to play
next song to be played
based on slides from Fabien Abel
Lastfm adapts to your music taste
bull Overfitting ldquobubble effectsrdquo loss of serendipity problem 13bull systems may adapt too strongly to the interestsbehavior13bull eg an adaptive radio station may always play the same or
very similar songs13bull We search for the right balance between novelty and
relevance for the user13
bull ldquoLost in Hyperspacerdquo problem 13bull when adapting the navigation ndash ie the links on which
users can click to findaccess information 13bull eg re-orderinghiding of menu items may lead to
confusion
Social Web 2014 Lora Aroyo
Issues in User-Adaptive Systems
Social Web 2014 Lora Aroyo
What is good user modelling amp personalisation
httpwwwflickrcomphotosbellarosebyliz4729613108
bull From the consumer perspective of an adaptive system 13
bull From the provider perspective of an adaptive system
Adaptive system maximizes satisfaction of the user
hard to measureobtain
Adaptive system maximizes the profit
influence of UM amp personalization may be hard to measureobtain
Social Web 2014 Lora Aroyo
Success Perspectives
bull User studies askobserve (selected) people whether you did a good job13
bull Log analysis Analyze (click) data and infer whether you did a good job13
bull Evaluation of user modeling13
bull measure quality of profiles directly eg measure overlap with existing (true) profiles or let people judge the quality of the generated user profiles 13
bull measure quality of application that exploits the user profile eg apply user modeling strategies in a recommender system
Social Web 2014 Lora Aroyo
Evaluation Strategies
time
item A
item B
item C
item D
item E
item G item H
item F
training data test data (ground truth)
Strategy X
User Modeling strategies to compare
Strategy Y
Strategy Z
Recommender
training data Recommendations
item H
X
item R
item M
Y
item H
item G
item N
item H
Z
item F
item M
measure quality
Social Web 2014 Lora Aroyo
Evaluating User Modeling in RecSys
Possible Metricsbull The usual IR metrics13
bull Precision fraction of retrieved items that are relevant13
bull Recall fraction of relevant items that have been retrieved13
bull F-Measure (harmonic) mean of precision and recall13
bull Metrics for evaluating recommendation (rankings)13
bull Mean Reciprocal Rank (MRR) of first relevant item13
bull Successk probability that relevant item occurs within the top k13
bull If a true ranking is given rank correlations 13
bull Precisionk Recallk amp F-Measurek13
bull Metrics for evaluating prediction of user preferences13
bull MAE = Mean Absolute Error13
bull TrueFalse PositivesNegatives runs
performance strategy X baseline
Is strategy X better than the baseline
Social Web 2014 Lora Aroyo
bull [Rae et al] a typical example of how to investigate and evaluate a proposal for improving (tag) recommendations (using social networks)13
bull Task test how well the different strategies (different tag contexts) can be used for tag predictionrecommendation13
bull Steps13
1 Gather a dataset of tag data part of which can be used as input and aim to test the recommendation on the remaining tag data13
2 Use the input data and calculate for the different strategies the predictions13
3 Measure the performance using standard (IR) metrics Precision of the top 5 recommended tags (P5) Mean Reciprocal Rank (MRR) Mean Average Precision (MAP)13
4 Test the results for statistical significance using T-test relative to the baseline (eg existing approach competitive approach)
[Rae et al Improving Tag Recommendations Using Social Networks RIAOrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
bull [Guy et al] another example of a similar evaluation approach13
bull The different strategies differ in the way people amp tags are used with tag-based systems there are complex relationships between users tags and items and strategies aim to find the relevant aspects of these relationships for modeling and recommendation13
bull The baseline is the lsquomost popularrsquo tags - often used to compare the most popular tags to the tags predicted by a particular personalization strategy - investigating whether the personalization is worth the effort and is able to outperform the easily available baseline
[Guy et al Social Media Recommendation based on People and Tags SIGIRrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
recommendation 13dimensions
Predict relevantusefulinteresting items13for a given user (in a given context)13
itrsquos often a ranking task
Social Web 2014 Lora Aroyo
Recommendation Systems
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
March 28 2013
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
commercial 13personalisation
httpwwwwiredcommagazine201111mf_artsyall1
Social Web 2014 Lora Aroyo
filter bubble
u1 likes u2
likes likes u1 likes Pulp Fiction
Social Web 2014 Lora Aroyo
Collaborative Filteringbull Memory-based User-Item matrix ratingspreferences of users =gt compute
similarity between users amp recommend items of similar users13
bull Model-based Item-Item matrix similarity (eg based on user ratings) between items =gt recommend items that are similar to the ones the user likes13
bull Model-based Clustering cluster users according to their preferences =gt recommend items of users that belong to the same cluster13
bull Model-based Bayesian networks P(u likes item B | u likes item A) = how likely is it that a user who likes item A will like item B learn probabilities from user ratingspreferences13
bull Others rule-based other data mining techniques13
bull complete input data is required13
bull pre-computation not possible13
bull does not scale well 13
bull high quality of recommendations13
bull abstraction (model) of input data13
bull pre-computation (partially) possible (model has to be re-built from time to time)13
bull scales better13
bull abstraction may reduce recommendation quality
Social Web 2014 Lora Aroyo
Memory vs Model-based
bull collaborative filtering lsquoneighborhoodsrsquo of people with similar interest amp recommending items based on likings in neighborhood13
bull limitations next to lsquocold startrsquo and lsquosparsityrsquo the lack of control (over onersquos neighborhood) is also a problem ie cannot add lsquotrustedrsquo people nor exclude lsquostrangersquo ones13
bull therefore interest in lsquosocial recommendersrsquo where presence of social connections defines the similarity in interests (eg social tagging CiteULike)13
bull does a social connection indicate user interest similarity13
bull how much users interest similarity depends on the strength of their connection13
bull is it feasible to use a social network as a personalized recommendation
[Lin amp Brusilovsky Social Networks and Interest Similarity The Case of CiteULike HTrsquo10]Social Web 2014 Lora Aroyo
Social Networks amp Interest Similarity
bull unilaterally connected pairs have more common itemsmetadatatags than non-connected pairs13
bull highest similarity for direct connections - decreasing with the increase of distance between users in SN 13
bull reciprocal relationship users - significantly larger similarity than users in a unidirectional relationship 13
bull traditional item-level similarity may be less reliable to find similar users in social bookmarking systems13
bull peers connected by self-defined social connections could be a useful source for cross-recommendation
Social Web 2014 Lora Aroyo
Conclusions
bull Input characteristics of items amp interests of a user into characteristics of items =gt Recommend items that feature characteristics which meet the userrsquos interests13
bull Techniques13
bull Data mining methods Cluster items based on their characteristics =gt Infer usersrsquo interests into clusters13
bull IR methods Represent items amp users as term vectors =gt Compute similarity between user profile vector and items13
bull Utility-based methods Utility function that gets an item as input the parameters of the utility function are customized via preferences of a user
Social Web 2014 Lora Aroyo
Content-based Recommendations
Government stops renovation of tower bridge Oct 13th 2011
Tower Bridge today Under construction
Tower Bridge is a combined bascule and suspension bridge in London England over the River Thames
Category politics england Related Twiper news bob Why do they stop tohellip [more] mary London stops renohellip [more]
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
02 0 0
02 04 01 01
= a
Weighting strategy - occurrence frequency - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
Content Features
RT Government stops renovation of tower bridge Oct 13th 2011
Userrsquos Twitter history
I am in London at the moment Oct 13th 2011
I am doing sports Oct 12th 2011
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
0 01 0
05 02 02 0
= u
Weighting strategy - occurrence frequency (eg smoothened by occurrence time recent concepts are more important - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
User Model
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
u 0
01 0
05 02 02 0
candidate items user a
02 0 0
02 04 01 01
b 0 0 0
08 02 0 0
c 0
05 02 0 0 0
03
cosine similarities
a b c
u 067 092 014
Ranking of recommended items 1 b 2 a 3 c
based on slides from Fabien Abel
Recommendations
Social Web 2014 Lora Aroyo
RecSys Issues
bull Cold-start problem (new user problem) nolittle data available to infer preferences of new users
bull Changing User Preferences user interests may change over time
bull Sparsity problem (new item problem) item descriptions are sparse eg not many user rated or tagged an item
bull Lack of Diversity (overfitting) when adapting too strongly to the preferences of users they might see samesimilar recommendations
bull Use the right context users do things which might not be relevant for their user model eg try out things do stuff for other people
bull Research challenge right balance between serendipity amp personalization
bull Research challenge right way to use the influence of recommendations on userrsquos behavior
one machine13vs 13
humans
image source httpwwwflickrcomphotosbionicteaching1375254387Social Web 2014 Lora Aroyo
Hands-on Teaser
bull Your Facebook Friendsrsquo popularity in a spread sheet13bull Locations of your Facebook Friends13bull Tag Cloud of your wall posts
theory amp techniques for 13how to design amp evaluate 13
recommenders amp user models 13to use in Social Web applications
Social Web 2014 Lora Aroyo
Fig 1 Functional model of tasks and sub-tasks specifically suited for SASs
Fig 1 Functional model of tasks and sub-tasks specifically suited for SASs (Ilaria Torre 2009)
Social Web 2014 Lora Aroyo
Kevin Kelly
How to infer amp represent 13user information that supports a given
application or context
User Modeling
Social Web 2014 Lora Aroyo
bull Application has to obtain understand amp exploit information about the user13
bull Information (need amp context) about user13
bull Inferring information about user amp representing it so that it can be consumed by the application13
bull Data relevant for inferring information about user
User Modeling Challenge
Social Web 2014 Lora Aroyo
bull People leave traces on the Web and on their computers
bull Usage data eg query logs click-through-data 13
bull Social data eg tags (micro-)blog posts comments bookmarks friend connections 13
bull Documents eg pictures videos13
bull Personal data eg affiliations locations 13
bull Products applications services - bought used installed13
bull Not only a userrsquos behavior but also interactions of other users
bull ldquopeople can make statements about merdquo13
bull ldquopeople who are similar to me can reveal information about merdquo13
bull ldquosocial learningrdquo collaborative recommender systems
Social Web 2014 Lora Aroyo
User amp Usage Data is Everywhere
bull User Profile = data structure = a characterization of a user at a particular moment represents what from a given (system) perspective there is to know about a user The data in the profile can be explicitly given by user or derived by system13
bull User Model = definitions amp rules for the interpretation of observations about the user amp about the translation of that interpretation into the characteristics in a user profile user model is the recipe for obtaining amp interpreting user profiles13
bull User Modeling = the process of representing the user
Social Web 2014 Lora Aroyo
UM Basic Concepts
bull Overlay User Modeling describe user characteristics eg ldquoknowledge of a userrdquo ldquointerests of a userrdquo with respect to ldquoidealrdquo characteristics13
bull Customizing user explicitly provides amp adjusts elements of the user profile13
bull User model elicitation ask amp observe the user learn amp improve user profile successively ldquointeractive user modelingrdquo13
bull Stereotyping stereotypical characteristics to describe a user13
bull User Relevance Modeling learninfer probabilities that a given item or concept is relevant for a user
Related scientific conference httpumap2011org Related journal httpumuaiorg
Social Web 2014 Lora Aroyo
User Modeling Approaches
httpfarm7staticflickrcom62406346803873_e756dd9bae_bjpgSocial Web 2014 Lora Aroyo
Which approach suits best the conditions of
applications
bull among the oldest user models13
bull used for modeling student knowledge13
bull the user is typically characterized in terms of domain concepts amp hypotheses of the userrsquos knowledge about these concepts in relation to an (ideal) expertrsquos knowledge13
bull concept-value pairs
Social Web 2014 Lora Aroyo
Overlay User Models
bull Ask the user explicitly learn13bull NLP intelligent dialogues13bull Bayesian networks Hidden Markov models13
bull Observe the user learn 13bull Logs machine learning13bull Clustering classification data mining
bull Interactive user modeling mixture of direct inputs of a user observations and inferences
Social Web 2014 Lora Aroyo
User Model Elicitation
httphunchcom
Social Web 2014 Lora Aroyo
httpfarm1staticflickrcom155413650229_31ef379b0b_bjpg
bull set of characteristics (eg attribute-value pairs) that describe a group of users13
bull user is not assigned to a single stereotype - user profile can feature characteristics of several different stereotypes
Social Web 2014 Lora Aroyo
User Stereotypes
based on slides from Fabien Abel
User Modeling (4 building blocks)
Semantic Enrichment Linkage and Alignment
Personalized News Recommender
Profile
I want my
personalized news recommendations
based on slides from Fabien Abel
Can we infer a Twitter-based User Profile
User Modeling Building Blocks
Profile concept weight
time
1 Which tweets of the user should be
analyzed
Morning Afternoon Night
1 Temporal Constraints
June 27 July 4 July 11
(b) temporal patterns
weekends start end
(a) time period
based on slides from Fabien Abel
User Modeling Building Blocks
Profile concept weight
2 Profile Type
Francesca Schiavone won French Open fo2010
Francesca Schiavone
French Open
Francesca Schiavone French Open entity-based
Sport T
T topic-based
2 What type of concepts should represent ldquointerestsrdquo
fo2010
fo2010 hashtag-based
1 Temporal Constraints
time
June 27 July 4 July 11
based on slides from Fabien Abel
User Modeling Building Blocks
Profile concept weight
2 Profile Type
Francesca Schiavone won httpbitly2f4t7a
Francesca Schiavone
3 Further enrich the semantics of tweets
1 Temporal Constraints
3 Semantic Enrichment
Francesca Schiavone
Francesca wins French Open Thirty in womens tennis is primordially old an age when agility and desire recedes as the hellip
French Open
Tennis
French Open
Tennis
(b) further enrichment
(a) tweet-based
based on slides from Fabien Abel
User Modeling Building Blocks
Profile concept weight
2 Profile Type
4 How to weight the concepts
1 Temporal Constraints
3 Semantic Enrichment
Francesca Schiavone
French Open
Tennis
4 Weighting Scheme
time
June 27 July 4 July 11
weight(Francesca Schiavone)
Concept frequency (TF)
4
3 6
TFxIDF Time-sensitive
weight(French Open)
weight(Tennis)
based on slides from Fabien Abel
Social Web 2014 Lora Aroyo
Observationsbull Profile characteristics
bull Semantic enrichment solves sparsity problems13bull Profiles change over time recent profiles reflect better
current user demands13bull Temporal patterns weekend profiles differ significantly
from weekday profiles13
bull Impact on recommendations bull The more fine-grained the concepts the better the
recommendation performance entity-based gt topic-based gt hashtag-based 13
bull Semantic enrichment improves recommendation quality 13bull Time-sensitivity (adapting to trends) improves
performance
Social Web 2014 Lora Aroyo
User Modelingit is not about putting everything in a user profile 13
it is about making the right choices
Social Web 2014 Lora Aroyo
User AdaptationKnowing the user to adapt a system or interface13
to improve the system functionality and user experience
A Jameson Adaptive interfaces and agents The HCI handbook fundamentals evolving technologies and emerging applications pp 305ndash330 2003
User-Adaptive Systems
user modeling
user profile
observations data and information about user
profile analysis
adaptation decisions
user modeling (infer current musical taste)
user profile interests in
genres artists tags
history of songs like ban pause skip
compare profile with possible next
songs to play
next song to be played
based on slides from Fabien Abel
Lastfm adapts to your music taste
bull Overfitting ldquobubble effectsrdquo loss of serendipity problem 13bull systems may adapt too strongly to the interestsbehavior13bull eg an adaptive radio station may always play the same or
very similar songs13bull We search for the right balance between novelty and
relevance for the user13
bull ldquoLost in Hyperspacerdquo problem 13bull when adapting the navigation ndash ie the links on which
users can click to findaccess information 13bull eg re-orderinghiding of menu items may lead to
confusion
Social Web 2014 Lora Aroyo
Issues in User-Adaptive Systems
Social Web 2014 Lora Aroyo
What is good user modelling amp personalisation
httpwwwflickrcomphotosbellarosebyliz4729613108
bull From the consumer perspective of an adaptive system 13
bull From the provider perspective of an adaptive system
Adaptive system maximizes satisfaction of the user
hard to measureobtain
Adaptive system maximizes the profit
influence of UM amp personalization may be hard to measureobtain
Social Web 2014 Lora Aroyo
Success Perspectives
bull User studies askobserve (selected) people whether you did a good job13
bull Log analysis Analyze (click) data and infer whether you did a good job13
bull Evaluation of user modeling13
bull measure quality of profiles directly eg measure overlap with existing (true) profiles or let people judge the quality of the generated user profiles 13
bull measure quality of application that exploits the user profile eg apply user modeling strategies in a recommender system
Social Web 2014 Lora Aroyo
Evaluation Strategies
time
item A
item B
item C
item D
item E
item G item H
item F
training data test data (ground truth)
Strategy X
User Modeling strategies to compare
Strategy Y
Strategy Z
Recommender
training data Recommendations
item H
X
item R
item M
Y
item H
item G
item N
item H
Z
item F
item M
measure quality
Social Web 2014 Lora Aroyo
Evaluating User Modeling in RecSys
Possible Metricsbull The usual IR metrics13
bull Precision fraction of retrieved items that are relevant13
bull Recall fraction of relevant items that have been retrieved13
bull F-Measure (harmonic) mean of precision and recall13
bull Metrics for evaluating recommendation (rankings)13
bull Mean Reciprocal Rank (MRR) of first relevant item13
bull Successk probability that relevant item occurs within the top k13
bull If a true ranking is given rank correlations 13
bull Precisionk Recallk amp F-Measurek13
bull Metrics for evaluating prediction of user preferences13
bull MAE = Mean Absolute Error13
bull TrueFalse PositivesNegatives runs
performance strategy X baseline
Is strategy X better than the baseline
Social Web 2014 Lora Aroyo
bull [Rae et al] a typical example of how to investigate and evaluate a proposal for improving (tag) recommendations (using social networks)13
bull Task test how well the different strategies (different tag contexts) can be used for tag predictionrecommendation13
bull Steps13
1 Gather a dataset of tag data part of which can be used as input and aim to test the recommendation on the remaining tag data13
2 Use the input data and calculate for the different strategies the predictions13
3 Measure the performance using standard (IR) metrics Precision of the top 5 recommended tags (P5) Mean Reciprocal Rank (MRR) Mean Average Precision (MAP)13
4 Test the results for statistical significance using T-test relative to the baseline (eg existing approach competitive approach)
[Rae et al Improving Tag Recommendations Using Social Networks RIAOrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
bull [Guy et al] another example of a similar evaluation approach13
bull The different strategies differ in the way people amp tags are used with tag-based systems there are complex relationships between users tags and items and strategies aim to find the relevant aspects of these relationships for modeling and recommendation13
bull The baseline is the lsquomost popularrsquo tags - often used to compare the most popular tags to the tags predicted by a particular personalization strategy - investigating whether the personalization is worth the effort and is able to outperform the easily available baseline
[Guy et al Social Media Recommendation based on People and Tags SIGIRrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
recommendation 13dimensions
Predict relevantusefulinteresting items13for a given user (in a given context)13
itrsquos often a ranking task
Social Web 2014 Lora Aroyo
Recommendation Systems
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
March 28 2013
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
commercial 13personalisation
httpwwwwiredcommagazine201111mf_artsyall1
Social Web 2014 Lora Aroyo
filter bubble
u1 likes u2
likes likes u1 likes Pulp Fiction
Social Web 2014 Lora Aroyo
Collaborative Filteringbull Memory-based User-Item matrix ratingspreferences of users =gt compute
similarity between users amp recommend items of similar users13
bull Model-based Item-Item matrix similarity (eg based on user ratings) between items =gt recommend items that are similar to the ones the user likes13
bull Model-based Clustering cluster users according to their preferences =gt recommend items of users that belong to the same cluster13
bull Model-based Bayesian networks P(u likes item B | u likes item A) = how likely is it that a user who likes item A will like item B learn probabilities from user ratingspreferences13
bull Others rule-based other data mining techniques13
bull complete input data is required13
bull pre-computation not possible13
bull does not scale well 13
bull high quality of recommendations13
bull abstraction (model) of input data13
bull pre-computation (partially) possible (model has to be re-built from time to time)13
bull scales better13
bull abstraction may reduce recommendation quality
Social Web 2014 Lora Aroyo
Memory vs Model-based
bull collaborative filtering lsquoneighborhoodsrsquo of people with similar interest amp recommending items based on likings in neighborhood13
bull limitations next to lsquocold startrsquo and lsquosparsityrsquo the lack of control (over onersquos neighborhood) is also a problem ie cannot add lsquotrustedrsquo people nor exclude lsquostrangersquo ones13
bull therefore interest in lsquosocial recommendersrsquo where presence of social connections defines the similarity in interests (eg social tagging CiteULike)13
bull does a social connection indicate user interest similarity13
bull how much users interest similarity depends on the strength of their connection13
bull is it feasible to use a social network as a personalized recommendation
[Lin amp Brusilovsky Social Networks and Interest Similarity The Case of CiteULike HTrsquo10]Social Web 2014 Lora Aroyo
Social Networks amp Interest Similarity
bull unilaterally connected pairs have more common itemsmetadatatags than non-connected pairs13
bull highest similarity for direct connections - decreasing with the increase of distance between users in SN 13
bull reciprocal relationship users - significantly larger similarity than users in a unidirectional relationship 13
bull traditional item-level similarity may be less reliable to find similar users in social bookmarking systems13
bull peers connected by self-defined social connections could be a useful source for cross-recommendation
Social Web 2014 Lora Aroyo
Conclusions
bull Input characteristics of items amp interests of a user into characteristics of items =gt Recommend items that feature characteristics which meet the userrsquos interests13
bull Techniques13
bull Data mining methods Cluster items based on their characteristics =gt Infer usersrsquo interests into clusters13
bull IR methods Represent items amp users as term vectors =gt Compute similarity between user profile vector and items13
bull Utility-based methods Utility function that gets an item as input the parameters of the utility function are customized via preferences of a user
Social Web 2014 Lora Aroyo
Content-based Recommendations
Government stops renovation of tower bridge Oct 13th 2011
Tower Bridge today Under construction
Tower Bridge is a combined bascule and suspension bridge in London England over the River Thames
Category politics england Related Twiper news bob Why do they stop tohellip [more] mary London stops renohellip [more]
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
02 0 0
02 04 01 01
= a
Weighting strategy - occurrence frequency - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
Content Features
RT Government stops renovation of tower bridge Oct 13th 2011
Userrsquos Twitter history
I am in London at the moment Oct 13th 2011
I am doing sports Oct 12th 2011
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
0 01 0
05 02 02 0
= u
Weighting strategy - occurrence frequency (eg smoothened by occurrence time recent concepts are more important - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
User Model
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
u 0
01 0
05 02 02 0
candidate items user a
02 0 0
02 04 01 01
b 0 0 0
08 02 0 0
c 0
05 02 0 0 0
03
cosine similarities
a b c
u 067 092 014
Ranking of recommended items 1 b 2 a 3 c
based on slides from Fabien Abel
Recommendations
Social Web 2014 Lora Aroyo
RecSys Issues
bull Cold-start problem (new user problem) nolittle data available to infer preferences of new users
bull Changing User Preferences user interests may change over time
bull Sparsity problem (new item problem) item descriptions are sparse eg not many user rated or tagged an item
bull Lack of Diversity (overfitting) when adapting too strongly to the preferences of users they might see samesimilar recommendations
bull Use the right context users do things which might not be relevant for their user model eg try out things do stuff for other people
bull Research challenge right balance between serendipity amp personalization
bull Research challenge right way to use the influence of recommendations on userrsquos behavior
one machine13vs 13
humans
image source httpwwwflickrcomphotosbionicteaching1375254387Social Web 2014 Lora Aroyo
Hands-on Teaser
bull Your Facebook Friendsrsquo popularity in a spread sheet13bull Locations of your Facebook Friends13bull Tag Cloud of your wall posts
Fig 1 Functional model of tasks and sub-tasks specifically suited for SASs
Fig 1 Functional model of tasks and sub-tasks specifically suited for SASs (Ilaria Torre 2009)
Social Web 2014 Lora Aroyo
Kevin Kelly
How to infer amp represent 13user information that supports a given
application or context
User Modeling
Social Web 2014 Lora Aroyo
bull Application has to obtain understand amp exploit information about the user13
bull Information (need amp context) about user13
bull Inferring information about user amp representing it so that it can be consumed by the application13
bull Data relevant for inferring information about user
User Modeling Challenge
Social Web 2014 Lora Aroyo
bull People leave traces on the Web and on their computers
bull Usage data eg query logs click-through-data 13
bull Social data eg tags (micro-)blog posts comments bookmarks friend connections 13
bull Documents eg pictures videos13
bull Personal data eg affiliations locations 13
bull Products applications services - bought used installed13
bull Not only a userrsquos behavior but also interactions of other users
bull ldquopeople can make statements about merdquo13
bull ldquopeople who are similar to me can reveal information about merdquo13
bull ldquosocial learningrdquo collaborative recommender systems
Social Web 2014 Lora Aroyo
User amp Usage Data is Everywhere
bull User Profile = data structure = a characterization of a user at a particular moment represents what from a given (system) perspective there is to know about a user The data in the profile can be explicitly given by user or derived by system13
bull User Model = definitions amp rules for the interpretation of observations about the user amp about the translation of that interpretation into the characteristics in a user profile user model is the recipe for obtaining amp interpreting user profiles13
bull User Modeling = the process of representing the user
Social Web 2014 Lora Aroyo
UM Basic Concepts
bull Overlay User Modeling describe user characteristics eg ldquoknowledge of a userrdquo ldquointerests of a userrdquo with respect to ldquoidealrdquo characteristics13
bull Customizing user explicitly provides amp adjusts elements of the user profile13
bull User model elicitation ask amp observe the user learn amp improve user profile successively ldquointeractive user modelingrdquo13
bull Stereotyping stereotypical characteristics to describe a user13
bull User Relevance Modeling learninfer probabilities that a given item or concept is relevant for a user
Related scientific conference httpumap2011org Related journal httpumuaiorg
Social Web 2014 Lora Aroyo
User Modeling Approaches
httpfarm7staticflickrcom62406346803873_e756dd9bae_bjpgSocial Web 2014 Lora Aroyo
Which approach suits best the conditions of
applications
bull among the oldest user models13
bull used for modeling student knowledge13
bull the user is typically characterized in terms of domain concepts amp hypotheses of the userrsquos knowledge about these concepts in relation to an (ideal) expertrsquos knowledge13
bull concept-value pairs
Social Web 2014 Lora Aroyo
Overlay User Models
bull Ask the user explicitly learn13bull NLP intelligent dialogues13bull Bayesian networks Hidden Markov models13
bull Observe the user learn 13bull Logs machine learning13bull Clustering classification data mining
bull Interactive user modeling mixture of direct inputs of a user observations and inferences
Social Web 2014 Lora Aroyo
User Model Elicitation
httphunchcom
Social Web 2014 Lora Aroyo
httpfarm1staticflickrcom155413650229_31ef379b0b_bjpg
bull set of characteristics (eg attribute-value pairs) that describe a group of users13
bull user is not assigned to a single stereotype - user profile can feature characteristics of several different stereotypes
Social Web 2014 Lora Aroyo
User Stereotypes
based on slides from Fabien Abel
User Modeling (4 building blocks)
Semantic Enrichment Linkage and Alignment
Personalized News Recommender
Profile
I want my
personalized news recommendations
based on slides from Fabien Abel
Can we infer a Twitter-based User Profile
User Modeling Building Blocks
Profile concept weight
time
1 Which tweets of the user should be
analyzed
Morning Afternoon Night
1 Temporal Constraints
June 27 July 4 July 11
(b) temporal patterns
weekends start end
(a) time period
based on slides from Fabien Abel
User Modeling Building Blocks
Profile concept weight
2 Profile Type
Francesca Schiavone won French Open fo2010
Francesca Schiavone
French Open
Francesca Schiavone French Open entity-based
Sport T
T topic-based
2 What type of concepts should represent ldquointerestsrdquo
fo2010
fo2010 hashtag-based
1 Temporal Constraints
time
June 27 July 4 July 11
based on slides from Fabien Abel
User Modeling Building Blocks
Profile concept weight
2 Profile Type
Francesca Schiavone won httpbitly2f4t7a
Francesca Schiavone
3 Further enrich the semantics of tweets
1 Temporal Constraints
3 Semantic Enrichment
Francesca Schiavone
Francesca wins French Open Thirty in womens tennis is primordially old an age when agility and desire recedes as the hellip
French Open
Tennis
French Open
Tennis
(b) further enrichment
(a) tweet-based
based on slides from Fabien Abel
User Modeling Building Blocks
Profile concept weight
2 Profile Type
4 How to weight the concepts
1 Temporal Constraints
3 Semantic Enrichment
Francesca Schiavone
French Open
Tennis
4 Weighting Scheme
time
June 27 July 4 July 11
weight(Francesca Schiavone)
Concept frequency (TF)
4
3 6
TFxIDF Time-sensitive
weight(French Open)
weight(Tennis)
based on slides from Fabien Abel
Social Web 2014 Lora Aroyo
Observationsbull Profile characteristics
bull Semantic enrichment solves sparsity problems13bull Profiles change over time recent profiles reflect better
current user demands13bull Temporal patterns weekend profiles differ significantly
from weekday profiles13
bull Impact on recommendations bull The more fine-grained the concepts the better the
recommendation performance entity-based gt topic-based gt hashtag-based 13
bull Semantic enrichment improves recommendation quality 13bull Time-sensitivity (adapting to trends) improves
performance
Social Web 2014 Lora Aroyo
User Modelingit is not about putting everything in a user profile 13
it is about making the right choices
Social Web 2014 Lora Aroyo
User AdaptationKnowing the user to adapt a system or interface13
to improve the system functionality and user experience
A Jameson Adaptive interfaces and agents The HCI handbook fundamentals evolving technologies and emerging applications pp 305ndash330 2003
User-Adaptive Systems
user modeling
user profile
observations data and information about user
profile analysis
adaptation decisions
user modeling (infer current musical taste)
user profile interests in
genres artists tags
history of songs like ban pause skip
compare profile with possible next
songs to play
next song to be played
based on slides from Fabien Abel
Lastfm adapts to your music taste
bull Overfitting ldquobubble effectsrdquo loss of serendipity problem 13bull systems may adapt too strongly to the interestsbehavior13bull eg an adaptive radio station may always play the same or
very similar songs13bull We search for the right balance between novelty and
relevance for the user13
bull ldquoLost in Hyperspacerdquo problem 13bull when adapting the navigation ndash ie the links on which
users can click to findaccess information 13bull eg re-orderinghiding of menu items may lead to
confusion
Social Web 2014 Lora Aroyo
Issues in User-Adaptive Systems
Social Web 2014 Lora Aroyo
What is good user modelling amp personalisation
httpwwwflickrcomphotosbellarosebyliz4729613108
bull From the consumer perspective of an adaptive system 13
bull From the provider perspective of an adaptive system
Adaptive system maximizes satisfaction of the user
hard to measureobtain
Adaptive system maximizes the profit
influence of UM amp personalization may be hard to measureobtain
Social Web 2014 Lora Aroyo
Success Perspectives
bull User studies askobserve (selected) people whether you did a good job13
bull Log analysis Analyze (click) data and infer whether you did a good job13
bull Evaluation of user modeling13
bull measure quality of profiles directly eg measure overlap with existing (true) profiles or let people judge the quality of the generated user profiles 13
bull measure quality of application that exploits the user profile eg apply user modeling strategies in a recommender system
Social Web 2014 Lora Aroyo
Evaluation Strategies
time
item A
item B
item C
item D
item E
item G item H
item F
training data test data (ground truth)
Strategy X
User Modeling strategies to compare
Strategy Y
Strategy Z
Recommender
training data Recommendations
item H
X
item R
item M
Y
item H
item G
item N
item H
Z
item F
item M
measure quality
Social Web 2014 Lora Aroyo
Evaluating User Modeling in RecSys
Possible Metricsbull The usual IR metrics13
bull Precision fraction of retrieved items that are relevant13
bull Recall fraction of relevant items that have been retrieved13
bull F-Measure (harmonic) mean of precision and recall13
bull Metrics for evaluating recommendation (rankings)13
bull Mean Reciprocal Rank (MRR) of first relevant item13
bull Successk probability that relevant item occurs within the top k13
bull If a true ranking is given rank correlations 13
bull Precisionk Recallk amp F-Measurek13
bull Metrics for evaluating prediction of user preferences13
bull MAE = Mean Absolute Error13
bull TrueFalse PositivesNegatives runs
performance strategy X baseline
Is strategy X better than the baseline
Social Web 2014 Lora Aroyo
bull [Rae et al] a typical example of how to investigate and evaluate a proposal for improving (tag) recommendations (using social networks)13
bull Task test how well the different strategies (different tag contexts) can be used for tag predictionrecommendation13
bull Steps13
1 Gather a dataset of tag data part of which can be used as input and aim to test the recommendation on the remaining tag data13
2 Use the input data and calculate for the different strategies the predictions13
3 Measure the performance using standard (IR) metrics Precision of the top 5 recommended tags (P5) Mean Reciprocal Rank (MRR) Mean Average Precision (MAP)13
4 Test the results for statistical significance using T-test relative to the baseline (eg existing approach competitive approach)
[Rae et al Improving Tag Recommendations Using Social Networks RIAOrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
bull [Guy et al] another example of a similar evaluation approach13
bull The different strategies differ in the way people amp tags are used with tag-based systems there are complex relationships between users tags and items and strategies aim to find the relevant aspects of these relationships for modeling and recommendation13
bull The baseline is the lsquomost popularrsquo tags - often used to compare the most popular tags to the tags predicted by a particular personalization strategy - investigating whether the personalization is worth the effort and is able to outperform the easily available baseline
[Guy et al Social Media Recommendation based on People and Tags SIGIRrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
recommendation 13dimensions
Predict relevantusefulinteresting items13for a given user (in a given context)13
itrsquos often a ranking task
Social Web 2014 Lora Aroyo
Recommendation Systems
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
March 28 2013
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
commercial 13personalisation
httpwwwwiredcommagazine201111mf_artsyall1
Social Web 2014 Lora Aroyo
filter bubble
u1 likes u2
likes likes u1 likes Pulp Fiction
Social Web 2014 Lora Aroyo
Collaborative Filteringbull Memory-based User-Item matrix ratingspreferences of users =gt compute
similarity between users amp recommend items of similar users13
bull Model-based Item-Item matrix similarity (eg based on user ratings) between items =gt recommend items that are similar to the ones the user likes13
bull Model-based Clustering cluster users according to their preferences =gt recommend items of users that belong to the same cluster13
bull Model-based Bayesian networks P(u likes item B | u likes item A) = how likely is it that a user who likes item A will like item B learn probabilities from user ratingspreferences13
bull Others rule-based other data mining techniques13
bull complete input data is required13
bull pre-computation not possible13
bull does not scale well 13
bull high quality of recommendations13
bull abstraction (model) of input data13
bull pre-computation (partially) possible (model has to be re-built from time to time)13
bull scales better13
bull abstraction may reduce recommendation quality
Social Web 2014 Lora Aroyo
Memory vs Model-based
bull collaborative filtering lsquoneighborhoodsrsquo of people with similar interest amp recommending items based on likings in neighborhood13
bull limitations next to lsquocold startrsquo and lsquosparsityrsquo the lack of control (over onersquos neighborhood) is also a problem ie cannot add lsquotrustedrsquo people nor exclude lsquostrangersquo ones13
bull therefore interest in lsquosocial recommendersrsquo where presence of social connections defines the similarity in interests (eg social tagging CiteULike)13
bull does a social connection indicate user interest similarity13
bull how much users interest similarity depends on the strength of their connection13
bull is it feasible to use a social network as a personalized recommendation
[Lin amp Brusilovsky Social Networks and Interest Similarity The Case of CiteULike HTrsquo10]Social Web 2014 Lora Aroyo
Social Networks amp Interest Similarity
bull unilaterally connected pairs have more common itemsmetadatatags than non-connected pairs13
bull highest similarity for direct connections - decreasing with the increase of distance between users in SN 13
bull reciprocal relationship users - significantly larger similarity than users in a unidirectional relationship 13
bull traditional item-level similarity may be less reliable to find similar users in social bookmarking systems13
bull peers connected by self-defined social connections could be a useful source for cross-recommendation
Social Web 2014 Lora Aroyo
Conclusions
bull Input characteristics of items amp interests of a user into characteristics of items =gt Recommend items that feature characteristics which meet the userrsquos interests13
bull Techniques13
bull Data mining methods Cluster items based on their characteristics =gt Infer usersrsquo interests into clusters13
bull IR methods Represent items amp users as term vectors =gt Compute similarity between user profile vector and items13
bull Utility-based methods Utility function that gets an item as input the parameters of the utility function are customized via preferences of a user
Social Web 2014 Lora Aroyo
Content-based Recommendations
Government stops renovation of tower bridge Oct 13th 2011
Tower Bridge today Under construction
Tower Bridge is a combined bascule and suspension bridge in London England over the River Thames
Category politics england Related Twiper news bob Why do they stop tohellip [more] mary London stops renohellip [more]
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
02 0 0
02 04 01 01
= a
Weighting strategy - occurrence frequency - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
Content Features
RT Government stops renovation of tower bridge Oct 13th 2011
Userrsquos Twitter history
I am in London at the moment Oct 13th 2011
I am doing sports Oct 12th 2011
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
0 01 0
05 02 02 0
= u
Weighting strategy - occurrence frequency (eg smoothened by occurrence time recent concepts are more important - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
User Model
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
u 0
01 0
05 02 02 0
candidate items user a
02 0 0
02 04 01 01
b 0 0 0
08 02 0 0
c 0
05 02 0 0 0
03
cosine similarities
a b c
u 067 092 014
Ranking of recommended items 1 b 2 a 3 c
based on slides from Fabien Abel
Recommendations
Social Web 2014 Lora Aroyo
RecSys Issues
bull Cold-start problem (new user problem) nolittle data available to infer preferences of new users
bull Changing User Preferences user interests may change over time
bull Sparsity problem (new item problem) item descriptions are sparse eg not many user rated or tagged an item
bull Lack of Diversity (overfitting) when adapting too strongly to the preferences of users they might see samesimilar recommendations
bull Use the right context users do things which might not be relevant for their user model eg try out things do stuff for other people
bull Research challenge right balance between serendipity amp personalization
bull Research challenge right way to use the influence of recommendations on userrsquos behavior
one machine13vs 13
humans
image source httpwwwflickrcomphotosbionicteaching1375254387Social Web 2014 Lora Aroyo
Hands-on Teaser
bull Your Facebook Friendsrsquo popularity in a spread sheet13bull Locations of your Facebook Friends13bull Tag Cloud of your wall posts
Kevin Kelly
How to infer amp represent 13user information that supports a given
application or context
User Modeling
Social Web 2014 Lora Aroyo
bull Application has to obtain understand amp exploit information about the user13
bull Information (need amp context) about user13
bull Inferring information about user amp representing it so that it can be consumed by the application13
bull Data relevant for inferring information about user
User Modeling Challenge
Social Web 2014 Lora Aroyo
bull People leave traces on the Web and on their computers
bull Usage data eg query logs click-through-data 13
bull Social data eg tags (micro-)blog posts comments bookmarks friend connections 13
bull Documents eg pictures videos13
bull Personal data eg affiliations locations 13
bull Products applications services - bought used installed13
bull Not only a userrsquos behavior but also interactions of other users
bull ldquopeople can make statements about merdquo13
bull ldquopeople who are similar to me can reveal information about merdquo13
bull ldquosocial learningrdquo collaborative recommender systems
Social Web 2014 Lora Aroyo
User amp Usage Data is Everywhere
bull User Profile = data structure = a characterization of a user at a particular moment represents what from a given (system) perspective there is to know about a user The data in the profile can be explicitly given by user or derived by system13
bull User Model = definitions amp rules for the interpretation of observations about the user amp about the translation of that interpretation into the characteristics in a user profile user model is the recipe for obtaining amp interpreting user profiles13
bull User Modeling = the process of representing the user
Social Web 2014 Lora Aroyo
UM Basic Concepts
bull Overlay User Modeling describe user characteristics eg ldquoknowledge of a userrdquo ldquointerests of a userrdquo with respect to ldquoidealrdquo characteristics13
bull Customizing user explicitly provides amp adjusts elements of the user profile13
bull User model elicitation ask amp observe the user learn amp improve user profile successively ldquointeractive user modelingrdquo13
bull Stereotyping stereotypical characteristics to describe a user13
bull User Relevance Modeling learninfer probabilities that a given item or concept is relevant for a user
Related scientific conference httpumap2011org Related journal httpumuaiorg
Social Web 2014 Lora Aroyo
User Modeling Approaches
httpfarm7staticflickrcom62406346803873_e756dd9bae_bjpgSocial Web 2014 Lora Aroyo
Which approach suits best the conditions of
applications
bull among the oldest user models13
bull used for modeling student knowledge13
bull the user is typically characterized in terms of domain concepts amp hypotheses of the userrsquos knowledge about these concepts in relation to an (ideal) expertrsquos knowledge13
bull concept-value pairs
Social Web 2014 Lora Aroyo
Overlay User Models
bull Ask the user explicitly learn13bull NLP intelligent dialogues13bull Bayesian networks Hidden Markov models13
bull Observe the user learn 13bull Logs machine learning13bull Clustering classification data mining
bull Interactive user modeling mixture of direct inputs of a user observations and inferences
Social Web 2014 Lora Aroyo
User Model Elicitation
httphunchcom
Social Web 2014 Lora Aroyo
httpfarm1staticflickrcom155413650229_31ef379b0b_bjpg
bull set of characteristics (eg attribute-value pairs) that describe a group of users13
bull user is not assigned to a single stereotype - user profile can feature characteristics of several different stereotypes
Social Web 2014 Lora Aroyo
User Stereotypes
based on slides from Fabien Abel
User Modeling (4 building blocks)
Semantic Enrichment Linkage and Alignment
Personalized News Recommender
Profile
I want my
personalized news recommendations
based on slides from Fabien Abel
Can we infer a Twitter-based User Profile
User Modeling Building Blocks
Profile concept weight
time
1 Which tweets of the user should be
analyzed
Morning Afternoon Night
1 Temporal Constraints
June 27 July 4 July 11
(b) temporal patterns
weekends start end
(a) time period
based on slides from Fabien Abel
User Modeling Building Blocks
Profile concept weight
2 Profile Type
Francesca Schiavone won French Open fo2010
Francesca Schiavone
French Open
Francesca Schiavone French Open entity-based
Sport T
T topic-based
2 What type of concepts should represent ldquointerestsrdquo
fo2010
fo2010 hashtag-based
1 Temporal Constraints
time
June 27 July 4 July 11
based on slides from Fabien Abel
User Modeling Building Blocks
Profile concept weight
2 Profile Type
Francesca Schiavone won httpbitly2f4t7a
Francesca Schiavone
3 Further enrich the semantics of tweets
1 Temporal Constraints
3 Semantic Enrichment
Francesca Schiavone
Francesca wins French Open Thirty in womens tennis is primordially old an age when agility and desire recedes as the hellip
French Open
Tennis
French Open
Tennis
(b) further enrichment
(a) tweet-based
based on slides from Fabien Abel
User Modeling Building Blocks
Profile concept weight
2 Profile Type
4 How to weight the concepts
1 Temporal Constraints
3 Semantic Enrichment
Francesca Schiavone
French Open
Tennis
4 Weighting Scheme
time
June 27 July 4 July 11
weight(Francesca Schiavone)
Concept frequency (TF)
4
3 6
TFxIDF Time-sensitive
weight(French Open)
weight(Tennis)
based on slides from Fabien Abel
Social Web 2014 Lora Aroyo
Observationsbull Profile characteristics
bull Semantic enrichment solves sparsity problems13bull Profiles change over time recent profiles reflect better
current user demands13bull Temporal patterns weekend profiles differ significantly
from weekday profiles13
bull Impact on recommendations bull The more fine-grained the concepts the better the
recommendation performance entity-based gt topic-based gt hashtag-based 13
bull Semantic enrichment improves recommendation quality 13bull Time-sensitivity (adapting to trends) improves
performance
Social Web 2014 Lora Aroyo
User Modelingit is not about putting everything in a user profile 13
it is about making the right choices
Social Web 2014 Lora Aroyo
User AdaptationKnowing the user to adapt a system or interface13
to improve the system functionality and user experience
A Jameson Adaptive interfaces and agents The HCI handbook fundamentals evolving technologies and emerging applications pp 305ndash330 2003
User-Adaptive Systems
user modeling
user profile
observations data and information about user
profile analysis
adaptation decisions
user modeling (infer current musical taste)
user profile interests in
genres artists tags
history of songs like ban pause skip
compare profile with possible next
songs to play
next song to be played
based on slides from Fabien Abel
Lastfm adapts to your music taste
bull Overfitting ldquobubble effectsrdquo loss of serendipity problem 13bull systems may adapt too strongly to the interestsbehavior13bull eg an adaptive radio station may always play the same or
very similar songs13bull We search for the right balance between novelty and
relevance for the user13
bull ldquoLost in Hyperspacerdquo problem 13bull when adapting the navigation ndash ie the links on which
users can click to findaccess information 13bull eg re-orderinghiding of menu items may lead to
confusion
Social Web 2014 Lora Aroyo
Issues in User-Adaptive Systems
Social Web 2014 Lora Aroyo
What is good user modelling amp personalisation
httpwwwflickrcomphotosbellarosebyliz4729613108
bull From the consumer perspective of an adaptive system 13
bull From the provider perspective of an adaptive system
Adaptive system maximizes satisfaction of the user
hard to measureobtain
Adaptive system maximizes the profit
influence of UM amp personalization may be hard to measureobtain
Social Web 2014 Lora Aroyo
Success Perspectives
bull User studies askobserve (selected) people whether you did a good job13
bull Log analysis Analyze (click) data and infer whether you did a good job13
bull Evaluation of user modeling13
bull measure quality of profiles directly eg measure overlap with existing (true) profiles or let people judge the quality of the generated user profiles 13
bull measure quality of application that exploits the user profile eg apply user modeling strategies in a recommender system
Social Web 2014 Lora Aroyo
Evaluation Strategies
time
item A
item B
item C
item D
item E
item G item H
item F
training data test data (ground truth)
Strategy X
User Modeling strategies to compare
Strategy Y
Strategy Z
Recommender
training data Recommendations
item H
X
item R
item M
Y
item H
item G
item N
item H
Z
item F
item M
measure quality
Social Web 2014 Lora Aroyo
Evaluating User Modeling in RecSys
Possible Metricsbull The usual IR metrics13
bull Precision fraction of retrieved items that are relevant13
bull Recall fraction of relevant items that have been retrieved13
bull F-Measure (harmonic) mean of precision and recall13
bull Metrics for evaluating recommendation (rankings)13
bull Mean Reciprocal Rank (MRR) of first relevant item13
bull Successk probability that relevant item occurs within the top k13
bull If a true ranking is given rank correlations 13
bull Precisionk Recallk amp F-Measurek13
bull Metrics for evaluating prediction of user preferences13
bull MAE = Mean Absolute Error13
bull TrueFalse PositivesNegatives runs
performance strategy X baseline
Is strategy X better than the baseline
Social Web 2014 Lora Aroyo
bull [Rae et al] a typical example of how to investigate and evaluate a proposal for improving (tag) recommendations (using social networks)13
bull Task test how well the different strategies (different tag contexts) can be used for tag predictionrecommendation13
bull Steps13
1 Gather a dataset of tag data part of which can be used as input and aim to test the recommendation on the remaining tag data13
2 Use the input data and calculate for the different strategies the predictions13
3 Measure the performance using standard (IR) metrics Precision of the top 5 recommended tags (P5) Mean Reciprocal Rank (MRR) Mean Average Precision (MAP)13
4 Test the results for statistical significance using T-test relative to the baseline (eg existing approach competitive approach)
[Rae et al Improving Tag Recommendations Using Social Networks RIAOrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
bull [Guy et al] another example of a similar evaluation approach13
bull The different strategies differ in the way people amp tags are used with tag-based systems there are complex relationships between users tags and items and strategies aim to find the relevant aspects of these relationships for modeling and recommendation13
bull The baseline is the lsquomost popularrsquo tags - often used to compare the most popular tags to the tags predicted by a particular personalization strategy - investigating whether the personalization is worth the effort and is able to outperform the easily available baseline
[Guy et al Social Media Recommendation based on People and Tags SIGIRrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
recommendation 13dimensions
Predict relevantusefulinteresting items13for a given user (in a given context)13
itrsquos often a ranking task
Social Web 2014 Lora Aroyo
Recommendation Systems
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
March 28 2013
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
commercial 13personalisation
httpwwwwiredcommagazine201111mf_artsyall1
Social Web 2014 Lora Aroyo
filter bubble
u1 likes u2
likes likes u1 likes Pulp Fiction
Social Web 2014 Lora Aroyo
Collaborative Filteringbull Memory-based User-Item matrix ratingspreferences of users =gt compute
similarity between users amp recommend items of similar users13
bull Model-based Item-Item matrix similarity (eg based on user ratings) between items =gt recommend items that are similar to the ones the user likes13
bull Model-based Clustering cluster users according to their preferences =gt recommend items of users that belong to the same cluster13
bull Model-based Bayesian networks P(u likes item B | u likes item A) = how likely is it that a user who likes item A will like item B learn probabilities from user ratingspreferences13
bull Others rule-based other data mining techniques13
bull complete input data is required13
bull pre-computation not possible13
bull does not scale well 13
bull high quality of recommendations13
bull abstraction (model) of input data13
bull pre-computation (partially) possible (model has to be re-built from time to time)13
bull scales better13
bull abstraction may reduce recommendation quality
Social Web 2014 Lora Aroyo
Memory vs Model-based
bull collaborative filtering lsquoneighborhoodsrsquo of people with similar interest amp recommending items based on likings in neighborhood13
bull limitations next to lsquocold startrsquo and lsquosparsityrsquo the lack of control (over onersquos neighborhood) is also a problem ie cannot add lsquotrustedrsquo people nor exclude lsquostrangersquo ones13
bull therefore interest in lsquosocial recommendersrsquo where presence of social connections defines the similarity in interests (eg social tagging CiteULike)13
bull does a social connection indicate user interest similarity13
bull how much users interest similarity depends on the strength of their connection13
bull is it feasible to use a social network as a personalized recommendation
[Lin amp Brusilovsky Social Networks and Interest Similarity The Case of CiteULike HTrsquo10]Social Web 2014 Lora Aroyo
Social Networks amp Interest Similarity
bull unilaterally connected pairs have more common itemsmetadatatags than non-connected pairs13
bull highest similarity for direct connections - decreasing with the increase of distance between users in SN 13
bull reciprocal relationship users - significantly larger similarity than users in a unidirectional relationship 13
bull traditional item-level similarity may be less reliable to find similar users in social bookmarking systems13
bull peers connected by self-defined social connections could be a useful source for cross-recommendation
Social Web 2014 Lora Aroyo
Conclusions
bull Input characteristics of items amp interests of a user into characteristics of items =gt Recommend items that feature characteristics which meet the userrsquos interests13
bull Techniques13
bull Data mining methods Cluster items based on their characteristics =gt Infer usersrsquo interests into clusters13
bull IR methods Represent items amp users as term vectors =gt Compute similarity between user profile vector and items13
bull Utility-based methods Utility function that gets an item as input the parameters of the utility function are customized via preferences of a user
Social Web 2014 Lora Aroyo
Content-based Recommendations
Government stops renovation of tower bridge Oct 13th 2011
Tower Bridge today Under construction
Tower Bridge is a combined bascule and suspension bridge in London England over the River Thames
Category politics england Related Twiper news bob Why do they stop tohellip [more] mary London stops renohellip [more]
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
02 0 0
02 04 01 01
= a
Weighting strategy - occurrence frequency - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
Content Features
RT Government stops renovation of tower bridge Oct 13th 2011
Userrsquos Twitter history
I am in London at the moment Oct 13th 2011
I am doing sports Oct 12th 2011
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
0 01 0
05 02 02 0
= u
Weighting strategy - occurrence frequency (eg smoothened by occurrence time recent concepts are more important - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
User Model
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
u 0
01 0
05 02 02 0
candidate items user a
02 0 0
02 04 01 01
b 0 0 0
08 02 0 0
c 0
05 02 0 0 0
03
cosine similarities
a b c
u 067 092 014
Ranking of recommended items 1 b 2 a 3 c
based on slides from Fabien Abel
Recommendations
Social Web 2014 Lora Aroyo
RecSys Issues
bull Cold-start problem (new user problem) nolittle data available to infer preferences of new users
bull Changing User Preferences user interests may change over time
bull Sparsity problem (new item problem) item descriptions are sparse eg not many user rated or tagged an item
bull Lack of Diversity (overfitting) when adapting too strongly to the preferences of users they might see samesimilar recommendations
bull Use the right context users do things which might not be relevant for their user model eg try out things do stuff for other people
bull Research challenge right balance between serendipity amp personalization
bull Research challenge right way to use the influence of recommendations on userrsquos behavior
one machine13vs 13
humans
image source httpwwwflickrcomphotosbionicteaching1375254387Social Web 2014 Lora Aroyo
Hands-on Teaser
bull Your Facebook Friendsrsquo popularity in a spread sheet13bull Locations of your Facebook Friends13bull Tag Cloud of your wall posts
bull Application has to obtain understand amp exploit information about the user13
bull Information (need amp context) about user13
bull Inferring information about user amp representing it so that it can be consumed by the application13
bull Data relevant for inferring information about user
User Modeling Challenge
Social Web 2014 Lora Aroyo
bull People leave traces on the Web and on their computers
bull Usage data eg query logs click-through-data 13
bull Social data eg tags (micro-)blog posts comments bookmarks friend connections 13
bull Documents eg pictures videos13
bull Personal data eg affiliations locations 13
bull Products applications services - bought used installed13
bull Not only a userrsquos behavior but also interactions of other users
bull ldquopeople can make statements about merdquo13
bull ldquopeople who are similar to me can reveal information about merdquo13
bull ldquosocial learningrdquo collaborative recommender systems
Social Web 2014 Lora Aroyo
User amp Usage Data is Everywhere
bull User Profile = data structure = a characterization of a user at a particular moment represents what from a given (system) perspective there is to know about a user The data in the profile can be explicitly given by user or derived by system13
bull User Model = definitions amp rules for the interpretation of observations about the user amp about the translation of that interpretation into the characteristics in a user profile user model is the recipe for obtaining amp interpreting user profiles13
bull User Modeling = the process of representing the user
Social Web 2014 Lora Aroyo
UM Basic Concepts
bull Overlay User Modeling describe user characteristics eg ldquoknowledge of a userrdquo ldquointerests of a userrdquo with respect to ldquoidealrdquo characteristics13
bull Customizing user explicitly provides amp adjusts elements of the user profile13
bull User model elicitation ask amp observe the user learn amp improve user profile successively ldquointeractive user modelingrdquo13
bull Stereotyping stereotypical characteristics to describe a user13
bull User Relevance Modeling learninfer probabilities that a given item or concept is relevant for a user
Related scientific conference httpumap2011org Related journal httpumuaiorg
Social Web 2014 Lora Aroyo
User Modeling Approaches
httpfarm7staticflickrcom62406346803873_e756dd9bae_bjpgSocial Web 2014 Lora Aroyo
Which approach suits best the conditions of
applications
bull among the oldest user models13
bull used for modeling student knowledge13
bull the user is typically characterized in terms of domain concepts amp hypotheses of the userrsquos knowledge about these concepts in relation to an (ideal) expertrsquos knowledge13
bull concept-value pairs
Social Web 2014 Lora Aroyo
Overlay User Models
bull Ask the user explicitly learn13bull NLP intelligent dialogues13bull Bayesian networks Hidden Markov models13
bull Observe the user learn 13bull Logs machine learning13bull Clustering classification data mining
bull Interactive user modeling mixture of direct inputs of a user observations and inferences
Social Web 2014 Lora Aroyo
User Model Elicitation
httphunchcom
Social Web 2014 Lora Aroyo
httpfarm1staticflickrcom155413650229_31ef379b0b_bjpg
bull set of characteristics (eg attribute-value pairs) that describe a group of users13
bull user is not assigned to a single stereotype - user profile can feature characteristics of several different stereotypes
Social Web 2014 Lora Aroyo
User Stereotypes
based on slides from Fabien Abel
User Modeling (4 building blocks)
Semantic Enrichment Linkage and Alignment
Personalized News Recommender
Profile
I want my
personalized news recommendations
based on slides from Fabien Abel
Can we infer a Twitter-based User Profile
User Modeling Building Blocks
Profile concept weight
time
1 Which tweets of the user should be
analyzed
Morning Afternoon Night
1 Temporal Constraints
June 27 July 4 July 11
(b) temporal patterns
weekends start end
(a) time period
based on slides from Fabien Abel
User Modeling Building Blocks
Profile concept weight
2 Profile Type
Francesca Schiavone won French Open fo2010
Francesca Schiavone
French Open
Francesca Schiavone French Open entity-based
Sport T
T topic-based
2 What type of concepts should represent ldquointerestsrdquo
fo2010
fo2010 hashtag-based
1 Temporal Constraints
time
June 27 July 4 July 11
based on slides from Fabien Abel
User Modeling Building Blocks
Profile concept weight
2 Profile Type
Francesca Schiavone won httpbitly2f4t7a
Francesca Schiavone
3 Further enrich the semantics of tweets
1 Temporal Constraints
3 Semantic Enrichment
Francesca Schiavone
Francesca wins French Open Thirty in womens tennis is primordially old an age when agility and desire recedes as the hellip
French Open
Tennis
French Open
Tennis
(b) further enrichment
(a) tweet-based
based on slides from Fabien Abel
User Modeling Building Blocks
Profile concept weight
2 Profile Type
4 How to weight the concepts
1 Temporal Constraints
3 Semantic Enrichment
Francesca Schiavone
French Open
Tennis
4 Weighting Scheme
time
June 27 July 4 July 11
weight(Francesca Schiavone)
Concept frequency (TF)
4
3 6
TFxIDF Time-sensitive
weight(French Open)
weight(Tennis)
based on slides from Fabien Abel
Social Web 2014 Lora Aroyo
Observationsbull Profile characteristics
bull Semantic enrichment solves sparsity problems13bull Profiles change over time recent profiles reflect better
current user demands13bull Temporal patterns weekend profiles differ significantly
from weekday profiles13
bull Impact on recommendations bull The more fine-grained the concepts the better the
recommendation performance entity-based gt topic-based gt hashtag-based 13
bull Semantic enrichment improves recommendation quality 13bull Time-sensitivity (adapting to trends) improves
performance
Social Web 2014 Lora Aroyo
User Modelingit is not about putting everything in a user profile 13
it is about making the right choices
Social Web 2014 Lora Aroyo
User AdaptationKnowing the user to adapt a system or interface13
to improve the system functionality and user experience
A Jameson Adaptive interfaces and agents The HCI handbook fundamentals evolving technologies and emerging applications pp 305ndash330 2003
User-Adaptive Systems
user modeling
user profile
observations data and information about user
profile analysis
adaptation decisions
user modeling (infer current musical taste)
user profile interests in
genres artists tags
history of songs like ban pause skip
compare profile with possible next
songs to play
next song to be played
based on slides from Fabien Abel
Lastfm adapts to your music taste
bull Overfitting ldquobubble effectsrdquo loss of serendipity problem 13bull systems may adapt too strongly to the interestsbehavior13bull eg an adaptive radio station may always play the same or
very similar songs13bull We search for the right balance between novelty and
relevance for the user13
bull ldquoLost in Hyperspacerdquo problem 13bull when adapting the navigation ndash ie the links on which
users can click to findaccess information 13bull eg re-orderinghiding of menu items may lead to
confusion
Social Web 2014 Lora Aroyo
Issues in User-Adaptive Systems
Social Web 2014 Lora Aroyo
What is good user modelling amp personalisation
httpwwwflickrcomphotosbellarosebyliz4729613108
bull From the consumer perspective of an adaptive system 13
bull From the provider perspective of an adaptive system
Adaptive system maximizes satisfaction of the user
hard to measureobtain
Adaptive system maximizes the profit
influence of UM amp personalization may be hard to measureobtain
Social Web 2014 Lora Aroyo
Success Perspectives
bull User studies askobserve (selected) people whether you did a good job13
bull Log analysis Analyze (click) data and infer whether you did a good job13
bull Evaluation of user modeling13
bull measure quality of profiles directly eg measure overlap with existing (true) profiles or let people judge the quality of the generated user profiles 13
bull measure quality of application that exploits the user profile eg apply user modeling strategies in a recommender system
Social Web 2014 Lora Aroyo
Evaluation Strategies
time
item A
item B
item C
item D
item E
item G item H
item F
training data test data (ground truth)
Strategy X
User Modeling strategies to compare
Strategy Y
Strategy Z
Recommender
training data Recommendations
item H
X
item R
item M
Y
item H
item G
item N
item H
Z
item F
item M
measure quality
Social Web 2014 Lora Aroyo
Evaluating User Modeling in RecSys
Possible Metricsbull The usual IR metrics13
bull Precision fraction of retrieved items that are relevant13
bull Recall fraction of relevant items that have been retrieved13
bull F-Measure (harmonic) mean of precision and recall13
bull Metrics for evaluating recommendation (rankings)13
bull Mean Reciprocal Rank (MRR) of first relevant item13
bull Successk probability that relevant item occurs within the top k13
bull If a true ranking is given rank correlations 13
bull Precisionk Recallk amp F-Measurek13
bull Metrics for evaluating prediction of user preferences13
bull MAE = Mean Absolute Error13
bull TrueFalse PositivesNegatives runs
performance strategy X baseline
Is strategy X better than the baseline
Social Web 2014 Lora Aroyo
bull [Rae et al] a typical example of how to investigate and evaluate a proposal for improving (tag) recommendations (using social networks)13
bull Task test how well the different strategies (different tag contexts) can be used for tag predictionrecommendation13
bull Steps13
1 Gather a dataset of tag data part of which can be used as input and aim to test the recommendation on the remaining tag data13
2 Use the input data and calculate for the different strategies the predictions13
3 Measure the performance using standard (IR) metrics Precision of the top 5 recommended tags (P5) Mean Reciprocal Rank (MRR) Mean Average Precision (MAP)13
4 Test the results for statistical significance using T-test relative to the baseline (eg existing approach competitive approach)
[Rae et al Improving Tag Recommendations Using Social Networks RIAOrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
bull [Guy et al] another example of a similar evaluation approach13
bull The different strategies differ in the way people amp tags are used with tag-based systems there are complex relationships between users tags and items and strategies aim to find the relevant aspects of these relationships for modeling and recommendation13
bull The baseline is the lsquomost popularrsquo tags - often used to compare the most popular tags to the tags predicted by a particular personalization strategy - investigating whether the personalization is worth the effort and is able to outperform the easily available baseline
[Guy et al Social Media Recommendation based on People and Tags SIGIRrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
recommendation 13dimensions
Predict relevantusefulinteresting items13for a given user (in a given context)13
itrsquos often a ranking task
Social Web 2014 Lora Aroyo
Recommendation Systems
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
March 28 2013
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
commercial 13personalisation
httpwwwwiredcommagazine201111mf_artsyall1
Social Web 2014 Lora Aroyo
filter bubble
u1 likes u2
likes likes u1 likes Pulp Fiction
Social Web 2014 Lora Aroyo
Collaborative Filteringbull Memory-based User-Item matrix ratingspreferences of users =gt compute
similarity between users amp recommend items of similar users13
bull Model-based Item-Item matrix similarity (eg based on user ratings) between items =gt recommend items that are similar to the ones the user likes13
bull Model-based Clustering cluster users according to their preferences =gt recommend items of users that belong to the same cluster13
bull Model-based Bayesian networks P(u likes item B | u likes item A) = how likely is it that a user who likes item A will like item B learn probabilities from user ratingspreferences13
bull Others rule-based other data mining techniques13
bull complete input data is required13
bull pre-computation not possible13
bull does not scale well 13
bull high quality of recommendations13
bull abstraction (model) of input data13
bull pre-computation (partially) possible (model has to be re-built from time to time)13
bull scales better13
bull abstraction may reduce recommendation quality
Social Web 2014 Lora Aroyo
Memory vs Model-based
bull collaborative filtering lsquoneighborhoodsrsquo of people with similar interest amp recommending items based on likings in neighborhood13
bull limitations next to lsquocold startrsquo and lsquosparsityrsquo the lack of control (over onersquos neighborhood) is also a problem ie cannot add lsquotrustedrsquo people nor exclude lsquostrangersquo ones13
bull therefore interest in lsquosocial recommendersrsquo where presence of social connections defines the similarity in interests (eg social tagging CiteULike)13
bull does a social connection indicate user interest similarity13
bull how much users interest similarity depends on the strength of their connection13
bull is it feasible to use a social network as a personalized recommendation
[Lin amp Brusilovsky Social Networks and Interest Similarity The Case of CiteULike HTrsquo10]Social Web 2014 Lora Aroyo
Social Networks amp Interest Similarity
bull unilaterally connected pairs have more common itemsmetadatatags than non-connected pairs13
bull highest similarity for direct connections - decreasing with the increase of distance between users in SN 13
bull reciprocal relationship users - significantly larger similarity than users in a unidirectional relationship 13
bull traditional item-level similarity may be less reliable to find similar users in social bookmarking systems13
bull peers connected by self-defined social connections could be a useful source for cross-recommendation
Social Web 2014 Lora Aroyo
Conclusions
bull Input characteristics of items amp interests of a user into characteristics of items =gt Recommend items that feature characteristics which meet the userrsquos interests13
bull Techniques13
bull Data mining methods Cluster items based on their characteristics =gt Infer usersrsquo interests into clusters13
bull IR methods Represent items amp users as term vectors =gt Compute similarity between user profile vector and items13
bull Utility-based methods Utility function that gets an item as input the parameters of the utility function are customized via preferences of a user
Social Web 2014 Lora Aroyo
Content-based Recommendations
Government stops renovation of tower bridge Oct 13th 2011
Tower Bridge today Under construction
Tower Bridge is a combined bascule and suspension bridge in London England over the River Thames
Category politics england Related Twiper news bob Why do they stop tohellip [more] mary London stops renohellip [more]
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
02 0 0
02 04 01 01
= a
Weighting strategy - occurrence frequency - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
Content Features
RT Government stops renovation of tower bridge Oct 13th 2011
Userrsquos Twitter history
I am in London at the moment Oct 13th 2011
I am doing sports Oct 12th 2011
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
0 01 0
05 02 02 0
= u
Weighting strategy - occurrence frequency (eg smoothened by occurrence time recent concepts are more important - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
User Model
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
u 0
01 0
05 02 02 0
candidate items user a
02 0 0
02 04 01 01
b 0 0 0
08 02 0 0
c 0
05 02 0 0 0
03
cosine similarities
a b c
u 067 092 014
Ranking of recommended items 1 b 2 a 3 c
based on slides from Fabien Abel
Recommendations
Social Web 2014 Lora Aroyo
RecSys Issues
bull Cold-start problem (new user problem) nolittle data available to infer preferences of new users
bull Changing User Preferences user interests may change over time
bull Sparsity problem (new item problem) item descriptions are sparse eg not many user rated or tagged an item
bull Lack of Diversity (overfitting) when adapting too strongly to the preferences of users they might see samesimilar recommendations
bull Use the right context users do things which might not be relevant for their user model eg try out things do stuff for other people
bull Research challenge right balance between serendipity amp personalization
bull Research challenge right way to use the influence of recommendations on userrsquos behavior
one machine13vs 13
humans
image source httpwwwflickrcomphotosbionicteaching1375254387Social Web 2014 Lora Aroyo
Hands-on Teaser
bull Your Facebook Friendsrsquo popularity in a spread sheet13bull Locations of your Facebook Friends13bull Tag Cloud of your wall posts
bull People leave traces on the Web and on their computers
bull Usage data eg query logs click-through-data 13
bull Social data eg tags (micro-)blog posts comments bookmarks friend connections 13
bull Documents eg pictures videos13
bull Personal data eg affiliations locations 13
bull Products applications services - bought used installed13
bull Not only a userrsquos behavior but also interactions of other users
bull ldquopeople can make statements about merdquo13
bull ldquopeople who are similar to me can reveal information about merdquo13
bull ldquosocial learningrdquo collaborative recommender systems
Social Web 2014 Lora Aroyo
User amp Usage Data is Everywhere
bull User Profile = data structure = a characterization of a user at a particular moment represents what from a given (system) perspective there is to know about a user The data in the profile can be explicitly given by user or derived by system13
bull User Model = definitions amp rules for the interpretation of observations about the user amp about the translation of that interpretation into the characteristics in a user profile user model is the recipe for obtaining amp interpreting user profiles13
bull User Modeling = the process of representing the user
Social Web 2014 Lora Aroyo
UM Basic Concepts
bull Overlay User Modeling describe user characteristics eg ldquoknowledge of a userrdquo ldquointerests of a userrdquo with respect to ldquoidealrdquo characteristics13
bull Customizing user explicitly provides amp adjusts elements of the user profile13
bull User model elicitation ask amp observe the user learn amp improve user profile successively ldquointeractive user modelingrdquo13
bull Stereotyping stereotypical characteristics to describe a user13
bull User Relevance Modeling learninfer probabilities that a given item or concept is relevant for a user
Related scientific conference httpumap2011org Related journal httpumuaiorg
Social Web 2014 Lora Aroyo
User Modeling Approaches
httpfarm7staticflickrcom62406346803873_e756dd9bae_bjpgSocial Web 2014 Lora Aroyo
Which approach suits best the conditions of
applications
bull among the oldest user models13
bull used for modeling student knowledge13
bull the user is typically characterized in terms of domain concepts amp hypotheses of the userrsquos knowledge about these concepts in relation to an (ideal) expertrsquos knowledge13
bull concept-value pairs
Social Web 2014 Lora Aroyo
Overlay User Models
bull Ask the user explicitly learn13bull NLP intelligent dialogues13bull Bayesian networks Hidden Markov models13
bull Observe the user learn 13bull Logs machine learning13bull Clustering classification data mining
bull Interactive user modeling mixture of direct inputs of a user observations and inferences
Social Web 2014 Lora Aroyo
User Model Elicitation
httphunchcom
Social Web 2014 Lora Aroyo
httpfarm1staticflickrcom155413650229_31ef379b0b_bjpg
bull set of characteristics (eg attribute-value pairs) that describe a group of users13
bull user is not assigned to a single stereotype - user profile can feature characteristics of several different stereotypes
Social Web 2014 Lora Aroyo
User Stereotypes
based on slides from Fabien Abel
User Modeling (4 building blocks)
Semantic Enrichment Linkage and Alignment
Personalized News Recommender
Profile
I want my
personalized news recommendations
based on slides from Fabien Abel
Can we infer a Twitter-based User Profile
User Modeling Building Blocks
Profile concept weight
time
1 Which tweets of the user should be
analyzed
Morning Afternoon Night
1 Temporal Constraints
June 27 July 4 July 11
(b) temporal patterns
weekends start end
(a) time period
based on slides from Fabien Abel
User Modeling Building Blocks
Profile concept weight
2 Profile Type
Francesca Schiavone won French Open fo2010
Francesca Schiavone
French Open
Francesca Schiavone French Open entity-based
Sport T
T topic-based
2 What type of concepts should represent ldquointerestsrdquo
fo2010
fo2010 hashtag-based
1 Temporal Constraints
time
June 27 July 4 July 11
based on slides from Fabien Abel
User Modeling Building Blocks
Profile concept weight
2 Profile Type
Francesca Schiavone won httpbitly2f4t7a
Francesca Schiavone
3 Further enrich the semantics of tweets
1 Temporal Constraints
3 Semantic Enrichment
Francesca Schiavone
Francesca wins French Open Thirty in womens tennis is primordially old an age when agility and desire recedes as the hellip
French Open
Tennis
French Open
Tennis
(b) further enrichment
(a) tweet-based
based on slides from Fabien Abel
User Modeling Building Blocks
Profile concept weight
2 Profile Type
4 How to weight the concepts
1 Temporal Constraints
3 Semantic Enrichment
Francesca Schiavone
French Open
Tennis
4 Weighting Scheme
time
June 27 July 4 July 11
weight(Francesca Schiavone)
Concept frequency (TF)
4
3 6
TFxIDF Time-sensitive
weight(French Open)
weight(Tennis)
based on slides from Fabien Abel
Social Web 2014 Lora Aroyo
Observationsbull Profile characteristics
bull Semantic enrichment solves sparsity problems13bull Profiles change over time recent profiles reflect better
current user demands13bull Temporal patterns weekend profiles differ significantly
from weekday profiles13
bull Impact on recommendations bull The more fine-grained the concepts the better the
recommendation performance entity-based gt topic-based gt hashtag-based 13
bull Semantic enrichment improves recommendation quality 13bull Time-sensitivity (adapting to trends) improves
performance
Social Web 2014 Lora Aroyo
User Modelingit is not about putting everything in a user profile 13
it is about making the right choices
Social Web 2014 Lora Aroyo
User AdaptationKnowing the user to adapt a system or interface13
to improve the system functionality and user experience
A Jameson Adaptive interfaces and agents The HCI handbook fundamentals evolving technologies and emerging applications pp 305ndash330 2003
User-Adaptive Systems
user modeling
user profile
observations data and information about user
profile analysis
adaptation decisions
user modeling (infer current musical taste)
user profile interests in
genres artists tags
history of songs like ban pause skip
compare profile with possible next
songs to play
next song to be played
based on slides from Fabien Abel
Lastfm adapts to your music taste
bull Overfitting ldquobubble effectsrdquo loss of serendipity problem 13bull systems may adapt too strongly to the interestsbehavior13bull eg an adaptive radio station may always play the same or
very similar songs13bull We search for the right balance between novelty and
relevance for the user13
bull ldquoLost in Hyperspacerdquo problem 13bull when adapting the navigation ndash ie the links on which
users can click to findaccess information 13bull eg re-orderinghiding of menu items may lead to
confusion
Social Web 2014 Lora Aroyo
Issues in User-Adaptive Systems
Social Web 2014 Lora Aroyo
What is good user modelling amp personalisation
httpwwwflickrcomphotosbellarosebyliz4729613108
bull From the consumer perspective of an adaptive system 13
bull From the provider perspective of an adaptive system
Adaptive system maximizes satisfaction of the user
hard to measureobtain
Adaptive system maximizes the profit
influence of UM amp personalization may be hard to measureobtain
Social Web 2014 Lora Aroyo
Success Perspectives
bull User studies askobserve (selected) people whether you did a good job13
bull Log analysis Analyze (click) data and infer whether you did a good job13
bull Evaluation of user modeling13
bull measure quality of profiles directly eg measure overlap with existing (true) profiles or let people judge the quality of the generated user profiles 13
bull measure quality of application that exploits the user profile eg apply user modeling strategies in a recommender system
Social Web 2014 Lora Aroyo
Evaluation Strategies
time
item A
item B
item C
item D
item E
item G item H
item F
training data test data (ground truth)
Strategy X
User Modeling strategies to compare
Strategy Y
Strategy Z
Recommender
training data Recommendations
item H
X
item R
item M
Y
item H
item G
item N
item H
Z
item F
item M
measure quality
Social Web 2014 Lora Aroyo
Evaluating User Modeling in RecSys
Possible Metricsbull The usual IR metrics13
bull Precision fraction of retrieved items that are relevant13
bull Recall fraction of relevant items that have been retrieved13
bull F-Measure (harmonic) mean of precision and recall13
bull Metrics for evaluating recommendation (rankings)13
bull Mean Reciprocal Rank (MRR) of first relevant item13
bull Successk probability that relevant item occurs within the top k13
bull If a true ranking is given rank correlations 13
bull Precisionk Recallk amp F-Measurek13
bull Metrics for evaluating prediction of user preferences13
bull MAE = Mean Absolute Error13
bull TrueFalse PositivesNegatives runs
performance strategy X baseline
Is strategy X better than the baseline
Social Web 2014 Lora Aroyo
bull [Rae et al] a typical example of how to investigate and evaluate a proposal for improving (tag) recommendations (using social networks)13
bull Task test how well the different strategies (different tag contexts) can be used for tag predictionrecommendation13
bull Steps13
1 Gather a dataset of tag data part of which can be used as input and aim to test the recommendation on the remaining tag data13
2 Use the input data and calculate for the different strategies the predictions13
3 Measure the performance using standard (IR) metrics Precision of the top 5 recommended tags (P5) Mean Reciprocal Rank (MRR) Mean Average Precision (MAP)13
4 Test the results for statistical significance using T-test relative to the baseline (eg existing approach competitive approach)
[Rae et al Improving Tag Recommendations Using Social Networks RIAOrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
bull [Guy et al] another example of a similar evaluation approach13
bull The different strategies differ in the way people amp tags are used with tag-based systems there are complex relationships between users tags and items and strategies aim to find the relevant aspects of these relationships for modeling and recommendation13
bull The baseline is the lsquomost popularrsquo tags - often used to compare the most popular tags to the tags predicted by a particular personalization strategy - investigating whether the personalization is worth the effort and is able to outperform the easily available baseline
[Guy et al Social Media Recommendation based on People and Tags SIGIRrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
recommendation 13dimensions
Predict relevantusefulinteresting items13for a given user (in a given context)13
itrsquos often a ranking task
Social Web 2014 Lora Aroyo
Recommendation Systems
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
March 28 2013
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
commercial 13personalisation
httpwwwwiredcommagazine201111mf_artsyall1
Social Web 2014 Lora Aroyo
filter bubble
u1 likes u2
likes likes u1 likes Pulp Fiction
Social Web 2014 Lora Aroyo
Collaborative Filteringbull Memory-based User-Item matrix ratingspreferences of users =gt compute
similarity between users amp recommend items of similar users13
bull Model-based Item-Item matrix similarity (eg based on user ratings) between items =gt recommend items that are similar to the ones the user likes13
bull Model-based Clustering cluster users according to their preferences =gt recommend items of users that belong to the same cluster13
bull Model-based Bayesian networks P(u likes item B | u likes item A) = how likely is it that a user who likes item A will like item B learn probabilities from user ratingspreferences13
bull Others rule-based other data mining techniques13
bull complete input data is required13
bull pre-computation not possible13
bull does not scale well 13
bull high quality of recommendations13
bull abstraction (model) of input data13
bull pre-computation (partially) possible (model has to be re-built from time to time)13
bull scales better13
bull abstraction may reduce recommendation quality
Social Web 2014 Lora Aroyo
Memory vs Model-based
bull collaborative filtering lsquoneighborhoodsrsquo of people with similar interest amp recommending items based on likings in neighborhood13
bull limitations next to lsquocold startrsquo and lsquosparsityrsquo the lack of control (over onersquos neighborhood) is also a problem ie cannot add lsquotrustedrsquo people nor exclude lsquostrangersquo ones13
bull therefore interest in lsquosocial recommendersrsquo where presence of social connections defines the similarity in interests (eg social tagging CiteULike)13
bull does a social connection indicate user interest similarity13
bull how much users interest similarity depends on the strength of their connection13
bull is it feasible to use a social network as a personalized recommendation
[Lin amp Brusilovsky Social Networks and Interest Similarity The Case of CiteULike HTrsquo10]Social Web 2014 Lora Aroyo
Social Networks amp Interest Similarity
bull unilaterally connected pairs have more common itemsmetadatatags than non-connected pairs13
bull highest similarity for direct connections - decreasing with the increase of distance between users in SN 13
bull reciprocal relationship users - significantly larger similarity than users in a unidirectional relationship 13
bull traditional item-level similarity may be less reliable to find similar users in social bookmarking systems13
bull peers connected by self-defined social connections could be a useful source for cross-recommendation
Social Web 2014 Lora Aroyo
Conclusions
bull Input characteristics of items amp interests of a user into characteristics of items =gt Recommend items that feature characteristics which meet the userrsquos interests13
bull Techniques13
bull Data mining methods Cluster items based on their characteristics =gt Infer usersrsquo interests into clusters13
bull IR methods Represent items amp users as term vectors =gt Compute similarity between user profile vector and items13
bull Utility-based methods Utility function that gets an item as input the parameters of the utility function are customized via preferences of a user
Social Web 2014 Lora Aroyo
Content-based Recommendations
Government stops renovation of tower bridge Oct 13th 2011
Tower Bridge today Under construction
Tower Bridge is a combined bascule and suspension bridge in London England over the River Thames
Category politics england Related Twiper news bob Why do they stop tohellip [more] mary London stops renohellip [more]
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
02 0 0
02 04 01 01
= a
Weighting strategy - occurrence frequency - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
Content Features
RT Government stops renovation of tower bridge Oct 13th 2011
Userrsquos Twitter history
I am in London at the moment Oct 13th 2011
I am doing sports Oct 12th 2011
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
0 01 0
05 02 02 0
= u
Weighting strategy - occurrence frequency (eg smoothened by occurrence time recent concepts are more important - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
User Model
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
u 0
01 0
05 02 02 0
candidate items user a
02 0 0
02 04 01 01
b 0 0 0
08 02 0 0
c 0
05 02 0 0 0
03
cosine similarities
a b c
u 067 092 014
Ranking of recommended items 1 b 2 a 3 c
based on slides from Fabien Abel
Recommendations
Social Web 2014 Lora Aroyo
RecSys Issues
bull Cold-start problem (new user problem) nolittle data available to infer preferences of new users
bull Changing User Preferences user interests may change over time
bull Sparsity problem (new item problem) item descriptions are sparse eg not many user rated or tagged an item
bull Lack of Diversity (overfitting) when adapting too strongly to the preferences of users they might see samesimilar recommendations
bull Use the right context users do things which might not be relevant for their user model eg try out things do stuff for other people
bull Research challenge right balance between serendipity amp personalization
bull Research challenge right way to use the influence of recommendations on userrsquos behavior
one machine13vs 13
humans
image source httpwwwflickrcomphotosbionicteaching1375254387Social Web 2014 Lora Aroyo
Hands-on Teaser
bull Your Facebook Friendsrsquo popularity in a spread sheet13bull Locations of your Facebook Friends13bull Tag Cloud of your wall posts
bull User Profile = data structure = a characterization of a user at a particular moment represents what from a given (system) perspective there is to know about a user The data in the profile can be explicitly given by user or derived by system13
bull User Model = definitions amp rules for the interpretation of observations about the user amp about the translation of that interpretation into the characteristics in a user profile user model is the recipe for obtaining amp interpreting user profiles13
bull User Modeling = the process of representing the user
Social Web 2014 Lora Aroyo
UM Basic Concepts
bull Overlay User Modeling describe user characteristics eg ldquoknowledge of a userrdquo ldquointerests of a userrdquo with respect to ldquoidealrdquo characteristics13
bull Customizing user explicitly provides amp adjusts elements of the user profile13
bull User model elicitation ask amp observe the user learn amp improve user profile successively ldquointeractive user modelingrdquo13
bull Stereotyping stereotypical characteristics to describe a user13
bull User Relevance Modeling learninfer probabilities that a given item or concept is relevant for a user
Related scientific conference httpumap2011org Related journal httpumuaiorg
Social Web 2014 Lora Aroyo
User Modeling Approaches
httpfarm7staticflickrcom62406346803873_e756dd9bae_bjpgSocial Web 2014 Lora Aroyo
Which approach suits best the conditions of
applications
bull among the oldest user models13
bull used for modeling student knowledge13
bull the user is typically characterized in terms of domain concepts amp hypotheses of the userrsquos knowledge about these concepts in relation to an (ideal) expertrsquos knowledge13
bull concept-value pairs
Social Web 2014 Lora Aroyo
Overlay User Models
bull Ask the user explicitly learn13bull NLP intelligent dialogues13bull Bayesian networks Hidden Markov models13
bull Observe the user learn 13bull Logs machine learning13bull Clustering classification data mining
bull Interactive user modeling mixture of direct inputs of a user observations and inferences
Social Web 2014 Lora Aroyo
User Model Elicitation
httphunchcom
Social Web 2014 Lora Aroyo
httpfarm1staticflickrcom155413650229_31ef379b0b_bjpg
bull set of characteristics (eg attribute-value pairs) that describe a group of users13
bull user is not assigned to a single stereotype - user profile can feature characteristics of several different stereotypes
Social Web 2014 Lora Aroyo
User Stereotypes
based on slides from Fabien Abel
User Modeling (4 building blocks)
Semantic Enrichment Linkage and Alignment
Personalized News Recommender
Profile
I want my
personalized news recommendations
based on slides from Fabien Abel
Can we infer a Twitter-based User Profile
User Modeling Building Blocks
Profile concept weight
time
1 Which tweets of the user should be
analyzed
Morning Afternoon Night
1 Temporal Constraints
June 27 July 4 July 11
(b) temporal patterns
weekends start end
(a) time period
based on slides from Fabien Abel
User Modeling Building Blocks
Profile concept weight
2 Profile Type
Francesca Schiavone won French Open fo2010
Francesca Schiavone
French Open
Francesca Schiavone French Open entity-based
Sport T
T topic-based
2 What type of concepts should represent ldquointerestsrdquo
fo2010
fo2010 hashtag-based
1 Temporal Constraints
time
June 27 July 4 July 11
based on slides from Fabien Abel
User Modeling Building Blocks
Profile concept weight
2 Profile Type
Francesca Schiavone won httpbitly2f4t7a
Francesca Schiavone
3 Further enrich the semantics of tweets
1 Temporal Constraints
3 Semantic Enrichment
Francesca Schiavone
Francesca wins French Open Thirty in womens tennis is primordially old an age when agility and desire recedes as the hellip
French Open
Tennis
French Open
Tennis
(b) further enrichment
(a) tweet-based
based on slides from Fabien Abel
User Modeling Building Blocks
Profile concept weight
2 Profile Type
4 How to weight the concepts
1 Temporal Constraints
3 Semantic Enrichment
Francesca Schiavone
French Open
Tennis
4 Weighting Scheme
time
June 27 July 4 July 11
weight(Francesca Schiavone)
Concept frequency (TF)
4
3 6
TFxIDF Time-sensitive
weight(French Open)
weight(Tennis)
based on slides from Fabien Abel
Social Web 2014 Lora Aroyo
Observationsbull Profile characteristics
bull Semantic enrichment solves sparsity problems13bull Profiles change over time recent profiles reflect better
current user demands13bull Temporal patterns weekend profiles differ significantly
from weekday profiles13
bull Impact on recommendations bull The more fine-grained the concepts the better the
recommendation performance entity-based gt topic-based gt hashtag-based 13
bull Semantic enrichment improves recommendation quality 13bull Time-sensitivity (adapting to trends) improves
performance
Social Web 2014 Lora Aroyo
User Modelingit is not about putting everything in a user profile 13
it is about making the right choices
Social Web 2014 Lora Aroyo
User AdaptationKnowing the user to adapt a system or interface13
to improve the system functionality and user experience
A Jameson Adaptive interfaces and agents The HCI handbook fundamentals evolving technologies and emerging applications pp 305ndash330 2003
User-Adaptive Systems
user modeling
user profile
observations data and information about user
profile analysis
adaptation decisions
user modeling (infer current musical taste)
user profile interests in
genres artists tags
history of songs like ban pause skip
compare profile with possible next
songs to play
next song to be played
based on slides from Fabien Abel
Lastfm adapts to your music taste
bull Overfitting ldquobubble effectsrdquo loss of serendipity problem 13bull systems may adapt too strongly to the interestsbehavior13bull eg an adaptive radio station may always play the same or
very similar songs13bull We search for the right balance between novelty and
relevance for the user13
bull ldquoLost in Hyperspacerdquo problem 13bull when adapting the navigation ndash ie the links on which
users can click to findaccess information 13bull eg re-orderinghiding of menu items may lead to
confusion
Social Web 2014 Lora Aroyo
Issues in User-Adaptive Systems
Social Web 2014 Lora Aroyo
What is good user modelling amp personalisation
httpwwwflickrcomphotosbellarosebyliz4729613108
bull From the consumer perspective of an adaptive system 13
bull From the provider perspective of an adaptive system
Adaptive system maximizes satisfaction of the user
hard to measureobtain
Adaptive system maximizes the profit
influence of UM amp personalization may be hard to measureobtain
Social Web 2014 Lora Aroyo
Success Perspectives
bull User studies askobserve (selected) people whether you did a good job13
bull Log analysis Analyze (click) data and infer whether you did a good job13
bull Evaluation of user modeling13
bull measure quality of profiles directly eg measure overlap with existing (true) profiles or let people judge the quality of the generated user profiles 13
bull measure quality of application that exploits the user profile eg apply user modeling strategies in a recommender system
Social Web 2014 Lora Aroyo
Evaluation Strategies
time
item A
item B
item C
item D
item E
item G item H
item F
training data test data (ground truth)
Strategy X
User Modeling strategies to compare
Strategy Y
Strategy Z
Recommender
training data Recommendations
item H
X
item R
item M
Y
item H
item G
item N
item H
Z
item F
item M
measure quality
Social Web 2014 Lora Aroyo
Evaluating User Modeling in RecSys
Possible Metricsbull The usual IR metrics13
bull Precision fraction of retrieved items that are relevant13
bull Recall fraction of relevant items that have been retrieved13
bull F-Measure (harmonic) mean of precision and recall13
bull Metrics for evaluating recommendation (rankings)13
bull Mean Reciprocal Rank (MRR) of first relevant item13
bull Successk probability that relevant item occurs within the top k13
bull If a true ranking is given rank correlations 13
bull Precisionk Recallk amp F-Measurek13
bull Metrics for evaluating prediction of user preferences13
bull MAE = Mean Absolute Error13
bull TrueFalse PositivesNegatives runs
performance strategy X baseline
Is strategy X better than the baseline
Social Web 2014 Lora Aroyo
bull [Rae et al] a typical example of how to investigate and evaluate a proposal for improving (tag) recommendations (using social networks)13
bull Task test how well the different strategies (different tag contexts) can be used for tag predictionrecommendation13
bull Steps13
1 Gather a dataset of tag data part of which can be used as input and aim to test the recommendation on the remaining tag data13
2 Use the input data and calculate for the different strategies the predictions13
3 Measure the performance using standard (IR) metrics Precision of the top 5 recommended tags (P5) Mean Reciprocal Rank (MRR) Mean Average Precision (MAP)13
4 Test the results for statistical significance using T-test relative to the baseline (eg existing approach competitive approach)
[Rae et al Improving Tag Recommendations Using Social Networks RIAOrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
bull [Guy et al] another example of a similar evaluation approach13
bull The different strategies differ in the way people amp tags are used with tag-based systems there are complex relationships between users tags and items and strategies aim to find the relevant aspects of these relationships for modeling and recommendation13
bull The baseline is the lsquomost popularrsquo tags - often used to compare the most popular tags to the tags predicted by a particular personalization strategy - investigating whether the personalization is worth the effort and is able to outperform the easily available baseline
[Guy et al Social Media Recommendation based on People and Tags SIGIRrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
recommendation 13dimensions
Predict relevantusefulinteresting items13for a given user (in a given context)13
itrsquos often a ranking task
Social Web 2014 Lora Aroyo
Recommendation Systems
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
March 28 2013
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
commercial 13personalisation
httpwwwwiredcommagazine201111mf_artsyall1
Social Web 2014 Lora Aroyo
filter bubble
u1 likes u2
likes likes u1 likes Pulp Fiction
Social Web 2014 Lora Aroyo
Collaborative Filteringbull Memory-based User-Item matrix ratingspreferences of users =gt compute
similarity between users amp recommend items of similar users13
bull Model-based Item-Item matrix similarity (eg based on user ratings) between items =gt recommend items that are similar to the ones the user likes13
bull Model-based Clustering cluster users according to their preferences =gt recommend items of users that belong to the same cluster13
bull Model-based Bayesian networks P(u likes item B | u likes item A) = how likely is it that a user who likes item A will like item B learn probabilities from user ratingspreferences13
bull Others rule-based other data mining techniques13
bull complete input data is required13
bull pre-computation not possible13
bull does not scale well 13
bull high quality of recommendations13
bull abstraction (model) of input data13
bull pre-computation (partially) possible (model has to be re-built from time to time)13
bull scales better13
bull abstraction may reduce recommendation quality
Social Web 2014 Lora Aroyo
Memory vs Model-based
bull collaborative filtering lsquoneighborhoodsrsquo of people with similar interest amp recommending items based on likings in neighborhood13
bull limitations next to lsquocold startrsquo and lsquosparsityrsquo the lack of control (over onersquos neighborhood) is also a problem ie cannot add lsquotrustedrsquo people nor exclude lsquostrangersquo ones13
bull therefore interest in lsquosocial recommendersrsquo where presence of social connections defines the similarity in interests (eg social tagging CiteULike)13
bull does a social connection indicate user interest similarity13
bull how much users interest similarity depends on the strength of their connection13
bull is it feasible to use a social network as a personalized recommendation
[Lin amp Brusilovsky Social Networks and Interest Similarity The Case of CiteULike HTrsquo10]Social Web 2014 Lora Aroyo
Social Networks amp Interest Similarity
bull unilaterally connected pairs have more common itemsmetadatatags than non-connected pairs13
bull highest similarity for direct connections - decreasing with the increase of distance between users in SN 13
bull reciprocal relationship users - significantly larger similarity than users in a unidirectional relationship 13
bull traditional item-level similarity may be less reliable to find similar users in social bookmarking systems13
bull peers connected by self-defined social connections could be a useful source for cross-recommendation
Social Web 2014 Lora Aroyo
Conclusions
bull Input characteristics of items amp interests of a user into characteristics of items =gt Recommend items that feature characteristics which meet the userrsquos interests13
bull Techniques13
bull Data mining methods Cluster items based on their characteristics =gt Infer usersrsquo interests into clusters13
bull IR methods Represent items amp users as term vectors =gt Compute similarity between user profile vector and items13
bull Utility-based methods Utility function that gets an item as input the parameters of the utility function are customized via preferences of a user
Social Web 2014 Lora Aroyo
Content-based Recommendations
Government stops renovation of tower bridge Oct 13th 2011
Tower Bridge today Under construction
Tower Bridge is a combined bascule and suspension bridge in London England over the River Thames
Category politics england Related Twiper news bob Why do they stop tohellip [more] mary London stops renohellip [more]
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
02 0 0
02 04 01 01
= a
Weighting strategy - occurrence frequency - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
Content Features
RT Government stops renovation of tower bridge Oct 13th 2011
Userrsquos Twitter history
I am in London at the moment Oct 13th 2011
I am doing sports Oct 12th 2011
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
0 01 0
05 02 02 0
= u
Weighting strategy - occurrence frequency (eg smoothened by occurrence time recent concepts are more important - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
User Model
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
u 0
01 0
05 02 02 0
candidate items user a
02 0 0
02 04 01 01
b 0 0 0
08 02 0 0
c 0
05 02 0 0 0
03
cosine similarities
a b c
u 067 092 014
Ranking of recommended items 1 b 2 a 3 c
based on slides from Fabien Abel
Recommendations
Social Web 2014 Lora Aroyo
RecSys Issues
bull Cold-start problem (new user problem) nolittle data available to infer preferences of new users
bull Changing User Preferences user interests may change over time
bull Sparsity problem (new item problem) item descriptions are sparse eg not many user rated or tagged an item
bull Lack of Diversity (overfitting) when adapting too strongly to the preferences of users they might see samesimilar recommendations
bull Use the right context users do things which might not be relevant for their user model eg try out things do stuff for other people
bull Research challenge right balance between serendipity amp personalization
bull Research challenge right way to use the influence of recommendations on userrsquos behavior
one machine13vs 13
humans
image source httpwwwflickrcomphotosbionicteaching1375254387Social Web 2014 Lora Aroyo
Hands-on Teaser
bull Your Facebook Friendsrsquo popularity in a spread sheet13bull Locations of your Facebook Friends13bull Tag Cloud of your wall posts
bull Overlay User Modeling describe user characteristics eg ldquoknowledge of a userrdquo ldquointerests of a userrdquo with respect to ldquoidealrdquo characteristics13
bull Customizing user explicitly provides amp adjusts elements of the user profile13
bull User model elicitation ask amp observe the user learn amp improve user profile successively ldquointeractive user modelingrdquo13
bull Stereotyping stereotypical characteristics to describe a user13
bull User Relevance Modeling learninfer probabilities that a given item or concept is relevant for a user
Related scientific conference httpumap2011org Related journal httpumuaiorg
Social Web 2014 Lora Aroyo
User Modeling Approaches
httpfarm7staticflickrcom62406346803873_e756dd9bae_bjpgSocial Web 2014 Lora Aroyo
Which approach suits best the conditions of
applications
bull among the oldest user models13
bull used for modeling student knowledge13
bull the user is typically characterized in terms of domain concepts amp hypotheses of the userrsquos knowledge about these concepts in relation to an (ideal) expertrsquos knowledge13
bull concept-value pairs
Social Web 2014 Lora Aroyo
Overlay User Models
bull Ask the user explicitly learn13bull NLP intelligent dialogues13bull Bayesian networks Hidden Markov models13
bull Observe the user learn 13bull Logs machine learning13bull Clustering classification data mining
bull Interactive user modeling mixture of direct inputs of a user observations and inferences
Social Web 2014 Lora Aroyo
User Model Elicitation
httphunchcom
Social Web 2014 Lora Aroyo
httpfarm1staticflickrcom155413650229_31ef379b0b_bjpg
bull set of characteristics (eg attribute-value pairs) that describe a group of users13
bull user is not assigned to a single stereotype - user profile can feature characteristics of several different stereotypes
Social Web 2014 Lora Aroyo
User Stereotypes
based on slides from Fabien Abel
User Modeling (4 building blocks)
Semantic Enrichment Linkage and Alignment
Personalized News Recommender
Profile
I want my
personalized news recommendations
based on slides from Fabien Abel
Can we infer a Twitter-based User Profile
User Modeling Building Blocks
Profile concept weight
time
1 Which tweets of the user should be
analyzed
Morning Afternoon Night
1 Temporal Constraints
June 27 July 4 July 11
(b) temporal patterns
weekends start end
(a) time period
based on slides from Fabien Abel
User Modeling Building Blocks
Profile concept weight
2 Profile Type
Francesca Schiavone won French Open fo2010
Francesca Schiavone
French Open
Francesca Schiavone French Open entity-based
Sport T
T topic-based
2 What type of concepts should represent ldquointerestsrdquo
fo2010
fo2010 hashtag-based
1 Temporal Constraints
time
June 27 July 4 July 11
based on slides from Fabien Abel
User Modeling Building Blocks
Profile concept weight
2 Profile Type
Francesca Schiavone won httpbitly2f4t7a
Francesca Schiavone
3 Further enrich the semantics of tweets
1 Temporal Constraints
3 Semantic Enrichment
Francesca Schiavone
Francesca wins French Open Thirty in womens tennis is primordially old an age when agility and desire recedes as the hellip
French Open
Tennis
French Open
Tennis
(b) further enrichment
(a) tweet-based
based on slides from Fabien Abel
User Modeling Building Blocks
Profile concept weight
2 Profile Type
4 How to weight the concepts
1 Temporal Constraints
3 Semantic Enrichment
Francesca Schiavone
French Open
Tennis
4 Weighting Scheme
time
June 27 July 4 July 11
weight(Francesca Schiavone)
Concept frequency (TF)
4
3 6
TFxIDF Time-sensitive
weight(French Open)
weight(Tennis)
based on slides from Fabien Abel
Social Web 2014 Lora Aroyo
Observationsbull Profile characteristics
bull Semantic enrichment solves sparsity problems13bull Profiles change over time recent profiles reflect better
current user demands13bull Temporal patterns weekend profiles differ significantly
from weekday profiles13
bull Impact on recommendations bull The more fine-grained the concepts the better the
recommendation performance entity-based gt topic-based gt hashtag-based 13
bull Semantic enrichment improves recommendation quality 13bull Time-sensitivity (adapting to trends) improves
performance
Social Web 2014 Lora Aroyo
User Modelingit is not about putting everything in a user profile 13
it is about making the right choices
Social Web 2014 Lora Aroyo
User AdaptationKnowing the user to adapt a system or interface13
to improve the system functionality and user experience
A Jameson Adaptive interfaces and agents The HCI handbook fundamentals evolving technologies and emerging applications pp 305ndash330 2003
User-Adaptive Systems
user modeling
user profile
observations data and information about user
profile analysis
adaptation decisions
user modeling (infer current musical taste)
user profile interests in
genres artists tags
history of songs like ban pause skip
compare profile with possible next
songs to play
next song to be played
based on slides from Fabien Abel
Lastfm adapts to your music taste
bull Overfitting ldquobubble effectsrdquo loss of serendipity problem 13bull systems may adapt too strongly to the interestsbehavior13bull eg an adaptive radio station may always play the same or
very similar songs13bull We search for the right balance between novelty and
relevance for the user13
bull ldquoLost in Hyperspacerdquo problem 13bull when adapting the navigation ndash ie the links on which
users can click to findaccess information 13bull eg re-orderinghiding of menu items may lead to
confusion
Social Web 2014 Lora Aroyo
Issues in User-Adaptive Systems
Social Web 2014 Lora Aroyo
What is good user modelling amp personalisation
httpwwwflickrcomphotosbellarosebyliz4729613108
bull From the consumer perspective of an adaptive system 13
bull From the provider perspective of an adaptive system
Adaptive system maximizes satisfaction of the user
hard to measureobtain
Adaptive system maximizes the profit
influence of UM amp personalization may be hard to measureobtain
Social Web 2014 Lora Aroyo
Success Perspectives
bull User studies askobserve (selected) people whether you did a good job13
bull Log analysis Analyze (click) data and infer whether you did a good job13
bull Evaluation of user modeling13
bull measure quality of profiles directly eg measure overlap with existing (true) profiles or let people judge the quality of the generated user profiles 13
bull measure quality of application that exploits the user profile eg apply user modeling strategies in a recommender system
Social Web 2014 Lora Aroyo
Evaluation Strategies
time
item A
item B
item C
item D
item E
item G item H
item F
training data test data (ground truth)
Strategy X
User Modeling strategies to compare
Strategy Y
Strategy Z
Recommender
training data Recommendations
item H
X
item R
item M
Y
item H
item G
item N
item H
Z
item F
item M
measure quality
Social Web 2014 Lora Aroyo
Evaluating User Modeling in RecSys
Possible Metricsbull The usual IR metrics13
bull Precision fraction of retrieved items that are relevant13
bull Recall fraction of relevant items that have been retrieved13
bull F-Measure (harmonic) mean of precision and recall13
bull Metrics for evaluating recommendation (rankings)13
bull Mean Reciprocal Rank (MRR) of first relevant item13
bull Successk probability that relevant item occurs within the top k13
bull If a true ranking is given rank correlations 13
bull Precisionk Recallk amp F-Measurek13
bull Metrics for evaluating prediction of user preferences13
bull MAE = Mean Absolute Error13
bull TrueFalse PositivesNegatives runs
performance strategy X baseline
Is strategy X better than the baseline
Social Web 2014 Lora Aroyo
bull [Rae et al] a typical example of how to investigate and evaluate a proposal for improving (tag) recommendations (using social networks)13
bull Task test how well the different strategies (different tag contexts) can be used for tag predictionrecommendation13
bull Steps13
1 Gather a dataset of tag data part of which can be used as input and aim to test the recommendation on the remaining tag data13
2 Use the input data and calculate for the different strategies the predictions13
3 Measure the performance using standard (IR) metrics Precision of the top 5 recommended tags (P5) Mean Reciprocal Rank (MRR) Mean Average Precision (MAP)13
4 Test the results for statistical significance using T-test relative to the baseline (eg existing approach competitive approach)
[Rae et al Improving Tag Recommendations Using Social Networks RIAOrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
bull [Guy et al] another example of a similar evaluation approach13
bull The different strategies differ in the way people amp tags are used with tag-based systems there are complex relationships between users tags and items and strategies aim to find the relevant aspects of these relationships for modeling and recommendation13
bull The baseline is the lsquomost popularrsquo tags - often used to compare the most popular tags to the tags predicted by a particular personalization strategy - investigating whether the personalization is worth the effort and is able to outperform the easily available baseline
[Guy et al Social Media Recommendation based on People and Tags SIGIRrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
recommendation 13dimensions
Predict relevantusefulinteresting items13for a given user (in a given context)13
itrsquos often a ranking task
Social Web 2014 Lora Aroyo
Recommendation Systems
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
March 28 2013
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
commercial 13personalisation
httpwwwwiredcommagazine201111mf_artsyall1
Social Web 2014 Lora Aroyo
filter bubble
u1 likes u2
likes likes u1 likes Pulp Fiction
Social Web 2014 Lora Aroyo
Collaborative Filteringbull Memory-based User-Item matrix ratingspreferences of users =gt compute
similarity between users amp recommend items of similar users13
bull Model-based Item-Item matrix similarity (eg based on user ratings) between items =gt recommend items that are similar to the ones the user likes13
bull Model-based Clustering cluster users according to their preferences =gt recommend items of users that belong to the same cluster13
bull Model-based Bayesian networks P(u likes item B | u likes item A) = how likely is it that a user who likes item A will like item B learn probabilities from user ratingspreferences13
bull Others rule-based other data mining techniques13
bull complete input data is required13
bull pre-computation not possible13
bull does not scale well 13
bull high quality of recommendations13
bull abstraction (model) of input data13
bull pre-computation (partially) possible (model has to be re-built from time to time)13
bull scales better13
bull abstraction may reduce recommendation quality
Social Web 2014 Lora Aroyo
Memory vs Model-based
bull collaborative filtering lsquoneighborhoodsrsquo of people with similar interest amp recommending items based on likings in neighborhood13
bull limitations next to lsquocold startrsquo and lsquosparsityrsquo the lack of control (over onersquos neighborhood) is also a problem ie cannot add lsquotrustedrsquo people nor exclude lsquostrangersquo ones13
bull therefore interest in lsquosocial recommendersrsquo where presence of social connections defines the similarity in interests (eg social tagging CiteULike)13
bull does a social connection indicate user interest similarity13
bull how much users interest similarity depends on the strength of their connection13
bull is it feasible to use a social network as a personalized recommendation
[Lin amp Brusilovsky Social Networks and Interest Similarity The Case of CiteULike HTrsquo10]Social Web 2014 Lora Aroyo
Social Networks amp Interest Similarity
bull unilaterally connected pairs have more common itemsmetadatatags than non-connected pairs13
bull highest similarity for direct connections - decreasing with the increase of distance between users in SN 13
bull reciprocal relationship users - significantly larger similarity than users in a unidirectional relationship 13
bull traditional item-level similarity may be less reliable to find similar users in social bookmarking systems13
bull peers connected by self-defined social connections could be a useful source for cross-recommendation
Social Web 2014 Lora Aroyo
Conclusions
bull Input characteristics of items amp interests of a user into characteristics of items =gt Recommend items that feature characteristics which meet the userrsquos interests13
bull Techniques13
bull Data mining methods Cluster items based on their characteristics =gt Infer usersrsquo interests into clusters13
bull IR methods Represent items amp users as term vectors =gt Compute similarity between user profile vector and items13
bull Utility-based methods Utility function that gets an item as input the parameters of the utility function are customized via preferences of a user
Social Web 2014 Lora Aroyo
Content-based Recommendations
Government stops renovation of tower bridge Oct 13th 2011
Tower Bridge today Under construction
Tower Bridge is a combined bascule and suspension bridge in London England over the River Thames
Category politics england Related Twiper news bob Why do they stop tohellip [more] mary London stops renohellip [more]
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
02 0 0
02 04 01 01
= a
Weighting strategy - occurrence frequency - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
Content Features
RT Government stops renovation of tower bridge Oct 13th 2011
Userrsquos Twitter history
I am in London at the moment Oct 13th 2011
I am doing sports Oct 12th 2011
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
0 01 0
05 02 02 0
= u
Weighting strategy - occurrence frequency (eg smoothened by occurrence time recent concepts are more important - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
User Model
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
u 0
01 0
05 02 02 0
candidate items user a
02 0 0
02 04 01 01
b 0 0 0
08 02 0 0
c 0
05 02 0 0 0
03
cosine similarities
a b c
u 067 092 014
Ranking of recommended items 1 b 2 a 3 c
based on slides from Fabien Abel
Recommendations
Social Web 2014 Lora Aroyo
RecSys Issues
bull Cold-start problem (new user problem) nolittle data available to infer preferences of new users
bull Changing User Preferences user interests may change over time
bull Sparsity problem (new item problem) item descriptions are sparse eg not many user rated or tagged an item
bull Lack of Diversity (overfitting) when adapting too strongly to the preferences of users they might see samesimilar recommendations
bull Use the right context users do things which might not be relevant for their user model eg try out things do stuff for other people
bull Research challenge right balance between serendipity amp personalization
bull Research challenge right way to use the influence of recommendations on userrsquos behavior
one machine13vs 13
humans
image source httpwwwflickrcomphotosbionicteaching1375254387Social Web 2014 Lora Aroyo
Hands-on Teaser
bull Your Facebook Friendsrsquo popularity in a spread sheet13bull Locations of your Facebook Friends13bull Tag Cloud of your wall posts
httpfarm7staticflickrcom62406346803873_e756dd9bae_bjpgSocial Web 2014 Lora Aroyo
Which approach suits best the conditions of
applications
bull among the oldest user models13
bull used for modeling student knowledge13
bull the user is typically characterized in terms of domain concepts amp hypotheses of the userrsquos knowledge about these concepts in relation to an (ideal) expertrsquos knowledge13
bull concept-value pairs
Social Web 2014 Lora Aroyo
Overlay User Models
bull Ask the user explicitly learn13bull NLP intelligent dialogues13bull Bayesian networks Hidden Markov models13
bull Observe the user learn 13bull Logs machine learning13bull Clustering classification data mining
bull Interactive user modeling mixture of direct inputs of a user observations and inferences
Social Web 2014 Lora Aroyo
User Model Elicitation
httphunchcom
Social Web 2014 Lora Aroyo
httpfarm1staticflickrcom155413650229_31ef379b0b_bjpg
bull set of characteristics (eg attribute-value pairs) that describe a group of users13
bull user is not assigned to a single stereotype - user profile can feature characteristics of several different stereotypes
Social Web 2014 Lora Aroyo
User Stereotypes
based on slides from Fabien Abel
User Modeling (4 building blocks)
Semantic Enrichment Linkage and Alignment
Personalized News Recommender
Profile
I want my
personalized news recommendations
based on slides from Fabien Abel
Can we infer a Twitter-based User Profile
User Modeling Building Blocks
Profile concept weight
time
1 Which tweets of the user should be
analyzed
Morning Afternoon Night
1 Temporal Constraints
June 27 July 4 July 11
(b) temporal patterns
weekends start end
(a) time period
based on slides from Fabien Abel
User Modeling Building Blocks
Profile concept weight
2 Profile Type
Francesca Schiavone won French Open fo2010
Francesca Schiavone
French Open
Francesca Schiavone French Open entity-based
Sport T
T topic-based
2 What type of concepts should represent ldquointerestsrdquo
fo2010
fo2010 hashtag-based
1 Temporal Constraints
time
June 27 July 4 July 11
based on slides from Fabien Abel
User Modeling Building Blocks
Profile concept weight
2 Profile Type
Francesca Schiavone won httpbitly2f4t7a
Francesca Schiavone
3 Further enrich the semantics of tweets
1 Temporal Constraints
3 Semantic Enrichment
Francesca Schiavone
Francesca wins French Open Thirty in womens tennis is primordially old an age when agility and desire recedes as the hellip
French Open
Tennis
French Open
Tennis
(b) further enrichment
(a) tweet-based
based on slides from Fabien Abel
User Modeling Building Blocks
Profile concept weight
2 Profile Type
4 How to weight the concepts
1 Temporal Constraints
3 Semantic Enrichment
Francesca Schiavone
French Open
Tennis
4 Weighting Scheme
time
June 27 July 4 July 11
weight(Francesca Schiavone)
Concept frequency (TF)
4
3 6
TFxIDF Time-sensitive
weight(French Open)
weight(Tennis)
based on slides from Fabien Abel
Social Web 2014 Lora Aroyo
Observationsbull Profile characteristics
bull Semantic enrichment solves sparsity problems13bull Profiles change over time recent profiles reflect better
current user demands13bull Temporal patterns weekend profiles differ significantly
from weekday profiles13
bull Impact on recommendations bull The more fine-grained the concepts the better the
recommendation performance entity-based gt topic-based gt hashtag-based 13
bull Semantic enrichment improves recommendation quality 13bull Time-sensitivity (adapting to trends) improves
performance
Social Web 2014 Lora Aroyo
User Modelingit is not about putting everything in a user profile 13
it is about making the right choices
Social Web 2014 Lora Aroyo
User AdaptationKnowing the user to adapt a system or interface13
to improve the system functionality and user experience
A Jameson Adaptive interfaces and agents The HCI handbook fundamentals evolving technologies and emerging applications pp 305ndash330 2003
User-Adaptive Systems
user modeling
user profile
observations data and information about user
profile analysis
adaptation decisions
user modeling (infer current musical taste)
user profile interests in
genres artists tags
history of songs like ban pause skip
compare profile with possible next
songs to play
next song to be played
based on slides from Fabien Abel
Lastfm adapts to your music taste
bull Overfitting ldquobubble effectsrdquo loss of serendipity problem 13bull systems may adapt too strongly to the interestsbehavior13bull eg an adaptive radio station may always play the same or
very similar songs13bull We search for the right balance between novelty and
relevance for the user13
bull ldquoLost in Hyperspacerdquo problem 13bull when adapting the navigation ndash ie the links on which
users can click to findaccess information 13bull eg re-orderinghiding of menu items may lead to
confusion
Social Web 2014 Lora Aroyo
Issues in User-Adaptive Systems
Social Web 2014 Lora Aroyo
What is good user modelling amp personalisation
httpwwwflickrcomphotosbellarosebyliz4729613108
bull From the consumer perspective of an adaptive system 13
bull From the provider perspective of an adaptive system
Adaptive system maximizes satisfaction of the user
hard to measureobtain
Adaptive system maximizes the profit
influence of UM amp personalization may be hard to measureobtain
Social Web 2014 Lora Aroyo
Success Perspectives
bull User studies askobserve (selected) people whether you did a good job13
bull Log analysis Analyze (click) data and infer whether you did a good job13
bull Evaluation of user modeling13
bull measure quality of profiles directly eg measure overlap with existing (true) profiles or let people judge the quality of the generated user profiles 13
bull measure quality of application that exploits the user profile eg apply user modeling strategies in a recommender system
Social Web 2014 Lora Aroyo
Evaluation Strategies
time
item A
item B
item C
item D
item E
item G item H
item F
training data test data (ground truth)
Strategy X
User Modeling strategies to compare
Strategy Y
Strategy Z
Recommender
training data Recommendations
item H
X
item R
item M
Y
item H
item G
item N
item H
Z
item F
item M
measure quality
Social Web 2014 Lora Aroyo
Evaluating User Modeling in RecSys
Possible Metricsbull The usual IR metrics13
bull Precision fraction of retrieved items that are relevant13
bull Recall fraction of relevant items that have been retrieved13
bull F-Measure (harmonic) mean of precision and recall13
bull Metrics for evaluating recommendation (rankings)13
bull Mean Reciprocal Rank (MRR) of first relevant item13
bull Successk probability that relevant item occurs within the top k13
bull If a true ranking is given rank correlations 13
bull Precisionk Recallk amp F-Measurek13
bull Metrics for evaluating prediction of user preferences13
bull MAE = Mean Absolute Error13
bull TrueFalse PositivesNegatives runs
performance strategy X baseline
Is strategy X better than the baseline
Social Web 2014 Lora Aroyo
bull [Rae et al] a typical example of how to investigate and evaluate a proposal for improving (tag) recommendations (using social networks)13
bull Task test how well the different strategies (different tag contexts) can be used for tag predictionrecommendation13
bull Steps13
1 Gather a dataset of tag data part of which can be used as input and aim to test the recommendation on the remaining tag data13
2 Use the input data and calculate for the different strategies the predictions13
3 Measure the performance using standard (IR) metrics Precision of the top 5 recommended tags (P5) Mean Reciprocal Rank (MRR) Mean Average Precision (MAP)13
4 Test the results for statistical significance using T-test relative to the baseline (eg existing approach competitive approach)
[Rae et al Improving Tag Recommendations Using Social Networks RIAOrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
bull [Guy et al] another example of a similar evaluation approach13
bull The different strategies differ in the way people amp tags are used with tag-based systems there are complex relationships between users tags and items and strategies aim to find the relevant aspects of these relationships for modeling and recommendation13
bull The baseline is the lsquomost popularrsquo tags - often used to compare the most popular tags to the tags predicted by a particular personalization strategy - investigating whether the personalization is worth the effort and is able to outperform the easily available baseline
[Guy et al Social Media Recommendation based on People and Tags SIGIRrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
recommendation 13dimensions
Predict relevantusefulinteresting items13for a given user (in a given context)13
itrsquos often a ranking task
Social Web 2014 Lora Aroyo
Recommendation Systems
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
March 28 2013
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
commercial 13personalisation
httpwwwwiredcommagazine201111mf_artsyall1
Social Web 2014 Lora Aroyo
filter bubble
u1 likes u2
likes likes u1 likes Pulp Fiction
Social Web 2014 Lora Aroyo
Collaborative Filteringbull Memory-based User-Item matrix ratingspreferences of users =gt compute
similarity between users amp recommend items of similar users13
bull Model-based Item-Item matrix similarity (eg based on user ratings) between items =gt recommend items that are similar to the ones the user likes13
bull Model-based Clustering cluster users according to their preferences =gt recommend items of users that belong to the same cluster13
bull Model-based Bayesian networks P(u likes item B | u likes item A) = how likely is it that a user who likes item A will like item B learn probabilities from user ratingspreferences13
bull Others rule-based other data mining techniques13
bull complete input data is required13
bull pre-computation not possible13
bull does not scale well 13
bull high quality of recommendations13
bull abstraction (model) of input data13
bull pre-computation (partially) possible (model has to be re-built from time to time)13
bull scales better13
bull abstraction may reduce recommendation quality
Social Web 2014 Lora Aroyo
Memory vs Model-based
bull collaborative filtering lsquoneighborhoodsrsquo of people with similar interest amp recommending items based on likings in neighborhood13
bull limitations next to lsquocold startrsquo and lsquosparsityrsquo the lack of control (over onersquos neighborhood) is also a problem ie cannot add lsquotrustedrsquo people nor exclude lsquostrangersquo ones13
bull therefore interest in lsquosocial recommendersrsquo where presence of social connections defines the similarity in interests (eg social tagging CiteULike)13
bull does a social connection indicate user interest similarity13
bull how much users interest similarity depends on the strength of their connection13
bull is it feasible to use a social network as a personalized recommendation
[Lin amp Brusilovsky Social Networks and Interest Similarity The Case of CiteULike HTrsquo10]Social Web 2014 Lora Aroyo
Social Networks amp Interest Similarity
bull unilaterally connected pairs have more common itemsmetadatatags than non-connected pairs13
bull highest similarity for direct connections - decreasing with the increase of distance between users in SN 13
bull reciprocal relationship users - significantly larger similarity than users in a unidirectional relationship 13
bull traditional item-level similarity may be less reliable to find similar users in social bookmarking systems13
bull peers connected by self-defined social connections could be a useful source for cross-recommendation
Social Web 2014 Lora Aroyo
Conclusions
bull Input characteristics of items amp interests of a user into characteristics of items =gt Recommend items that feature characteristics which meet the userrsquos interests13
bull Techniques13
bull Data mining methods Cluster items based on their characteristics =gt Infer usersrsquo interests into clusters13
bull IR methods Represent items amp users as term vectors =gt Compute similarity between user profile vector and items13
bull Utility-based methods Utility function that gets an item as input the parameters of the utility function are customized via preferences of a user
Social Web 2014 Lora Aroyo
Content-based Recommendations
Government stops renovation of tower bridge Oct 13th 2011
Tower Bridge today Under construction
Tower Bridge is a combined bascule and suspension bridge in London England over the River Thames
Category politics england Related Twiper news bob Why do they stop tohellip [more] mary London stops renohellip [more]
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
02 0 0
02 04 01 01
= a
Weighting strategy - occurrence frequency - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
Content Features
RT Government stops renovation of tower bridge Oct 13th 2011
Userrsquos Twitter history
I am in London at the moment Oct 13th 2011
I am doing sports Oct 12th 2011
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
0 01 0
05 02 02 0
= u
Weighting strategy - occurrence frequency (eg smoothened by occurrence time recent concepts are more important - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
User Model
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
u 0
01 0
05 02 02 0
candidate items user a
02 0 0
02 04 01 01
b 0 0 0
08 02 0 0
c 0
05 02 0 0 0
03
cosine similarities
a b c
u 067 092 014
Ranking of recommended items 1 b 2 a 3 c
based on slides from Fabien Abel
Recommendations
Social Web 2014 Lora Aroyo
RecSys Issues
bull Cold-start problem (new user problem) nolittle data available to infer preferences of new users
bull Changing User Preferences user interests may change over time
bull Sparsity problem (new item problem) item descriptions are sparse eg not many user rated or tagged an item
bull Lack of Diversity (overfitting) when adapting too strongly to the preferences of users they might see samesimilar recommendations
bull Use the right context users do things which might not be relevant for their user model eg try out things do stuff for other people
bull Research challenge right balance between serendipity amp personalization
bull Research challenge right way to use the influence of recommendations on userrsquos behavior
one machine13vs 13
humans
image source httpwwwflickrcomphotosbionicteaching1375254387Social Web 2014 Lora Aroyo
Hands-on Teaser
bull Your Facebook Friendsrsquo popularity in a spread sheet13bull Locations of your Facebook Friends13bull Tag Cloud of your wall posts
bull among the oldest user models13
bull used for modeling student knowledge13
bull the user is typically characterized in terms of domain concepts amp hypotheses of the userrsquos knowledge about these concepts in relation to an (ideal) expertrsquos knowledge13
bull concept-value pairs
Social Web 2014 Lora Aroyo
Overlay User Models
bull Ask the user explicitly learn13bull NLP intelligent dialogues13bull Bayesian networks Hidden Markov models13
bull Observe the user learn 13bull Logs machine learning13bull Clustering classification data mining
bull Interactive user modeling mixture of direct inputs of a user observations and inferences
Social Web 2014 Lora Aroyo
User Model Elicitation
httphunchcom
Social Web 2014 Lora Aroyo
httpfarm1staticflickrcom155413650229_31ef379b0b_bjpg
bull set of characteristics (eg attribute-value pairs) that describe a group of users13
bull user is not assigned to a single stereotype - user profile can feature characteristics of several different stereotypes
Social Web 2014 Lora Aroyo
User Stereotypes
based on slides from Fabien Abel
User Modeling (4 building blocks)
Semantic Enrichment Linkage and Alignment
Personalized News Recommender
Profile
I want my
personalized news recommendations
based on slides from Fabien Abel
Can we infer a Twitter-based User Profile
User Modeling Building Blocks
Profile concept weight
time
1 Which tweets of the user should be
analyzed
Morning Afternoon Night
1 Temporal Constraints
June 27 July 4 July 11
(b) temporal patterns
weekends start end
(a) time period
based on slides from Fabien Abel
User Modeling Building Blocks
Profile concept weight
2 Profile Type
Francesca Schiavone won French Open fo2010
Francesca Schiavone
French Open
Francesca Schiavone French Open entity-based
Sport T
T topic-based
2 What type of concepts should represent ldquointerestsrdquo
fo2010
fo2010 hashtag-based
1 Temporal Constraints
time
June 27 July 4 July 11
based on slides from Fabien Abel
User Modeling Building Blocks
Profile concept weight
2 Profile Type
Francesca Schiavone won httpbitly2f4t7a
Francesca Schiavone
3 Further enrich the semantics of tweets
1 Temporal Constraints
3 Semantic Enrichment
Francesca Schiavone
Francesca wins French Open Thirty in womens tennis is primordially old an age when agility and desire recedes as the hellip
French Open
Tennis
French Open
Tennis
(b) further enrichment
(a) tweet-based
based on slides from Fabien Abel
User Modeling Building Blocks
Profile concept weight
2 Profile Type
4 How to weight the concepts
1 Temporal Constraints
3 Semantic Enrichment
Francesca Schiavone
French Open
Tennis
4 Weighting Scheme
time
June 27 July 4 July 11
weight(Francesca Schiavone)
Concept frequency (TF)
4
3 6
TFxIDF Time-sensitive
weight(French Open)
weight(Tennis)
based on slides from Fabien Abel
Social Web 2014 Lora Aroyo
Observationsbull Profile characteristics
bull Semantic enrichment solves sparsity problems13bull Profiles change over time recent profiles reflect better
current user demands13bull Temporal patterns weekend profiles differ significantly
from weekday profiles13
bull Impact on recommendations bull The more fine-grained the concepts the better the
recommendation performance entity-based gt topic-based gt hashtag-based 13
bull Semantic enrichment improves recommendation quality 13bull Time-sensitivity (adapting to trends) improves
performance
Social Web 2014 Lora Aroyo
User Modelingit is not about putting everything in a user profile 13
it is about making the right choices
Social Web 2014 Lora Aroyo
User AdaptationKnowing the user to adapt a system or interface13
to improve the system functionality and user experience
A Jameson Adaptive interfaces and agents The HCI handbook fundamentals evolving technologies and emerging applications pp 305ndash330 2003
User-Adaptive Systems
user modeling
user profile
observations data and information about user
profile analysis
adaptation decisions
user modeling (infer current musical taste)
user profile interests in
genres artists tags
history of songs like ban pause skip
compare profile with possible next
songs to play
next song to be played
based on slides from Fabien Abel
Lastfm adapts to your music taste
bull Overfitting ldquobubble effectsrdquo loss of serendipity problem 13bull systems may adapt too strongly to the interestsbehavior13bull eg an adaptive radio station may always play the same or
very similar songs13bull We search for the right balance between novelty and
relevance for the user13
bull ldquoLost in Hyperspacerdquo problem 13bull when adapting the navigation ndash ie the links on which
users can click to findaccess information 13bull eg re-orderinghiding of menu items may lead to
confusion
Social Web 2014 Lora Aroyo
Issues in User-Adaptive Systems
Social Web 2014 Lora Aroyo
What is good user modelling amp personalisation
httpwwwflickrcomphotosbellarosebyliz4729613108
bull From the consumer perspective of an adaptive system 13
bull From the provider perspective of an adaptive system
Adaptive system maximizes satisfaction of the user
hard to measureobtain
Adaptive system maximizes the profit
influence of UM amp personalization may be hard to measureobtain
Social Web 2014 Lora Aroyo
Success Perspectives
bull User studies askobserve (selected) people whether you did a good job13
bull Log analysis Analyze (click) data and infer whether you did a good job13
bull Evaluation of user modeling13
bull measure quality of profiles directly eg measure overlap with existing (true) profiles or let people judge the quality of the generated user profiles 13
bull measure quality of application that exploits the user profile eg apply user modeling strategies in a recommender system
Social Web 2014 Lora Aroyo
Evaluation Strategies
time
item A
item B
item C
item D
item E
item G item H
item F
training data test data (ground truth)
Strategy X
User Modeling strategies to compare
Strategy Y
Strategy Z
Recommender
training data Recommendations
item H
X
item R
item M
Y
item H
item G
item N
item H
Z
item F
item M
measure quality
Social Web 2014 Lora Aroyo
Evaluating User Modeling in RecSys
Possible Metricsbull The usual IR metrics13
bull Precision fraction of retrieved items that are relevant13
bull Recall fraction of relevant items that have been retrieved13
bull F-Measure (harmonic) mean of precision and recall13
bull Metrics for evaluating recommendation (rankings)13
bull Mean Reciprocal Rank (MRR) of first relevant item13
bull Successk probability that relevant item occurs within the top k13
bull If a true ranking is given rank correlations 13
bull Precisionk Recallk amp F-Measurek13
bull Metrics for evaluating prediction of user preferences13
bull MAE = Mean Absolute Error13
bull TrueFalse PositivesNegatives runs
performance strategy X baseline
Is strategy X better than the baseline
Social Web 2014 Lora Aroyo
bull [Rae et al] a typical example of how to investigate and evaluate a proposal for improving (tag) recommendations (using social networks)13
bull Task test how well the different strategies (different tag contexts) can be used for tag predictionrecommendation13
bull Steps13
1 Gather a dataset of tag data part of which can be used as input and aim to test the recommendation on the remaining tag data13
2 Use the input data and calculate for the different strategies the predictions13
3 Measure the performance using standard (IR) metrics Precision of the top 5 recommended tags (P5) Mean Reciprocal Rank (MRR) Mean Average Precision (MAP)13
4 Test the results for statistical significance using T-test relative to the baseline (eg existing approach competitive approach)
[Rae et al Improving Tag Recommendations Using Social Networks RIAOrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
bull [Guy et al] another example of a similar evaluation approach13
bull The different strategies differ in the way people amp tags are used with tag-based systems there are complex relationships between users tags and items and strategies aim to find the relevant aspects of these relationships for modeling and recommendation13
bull The baseline is the lsquomost popularrsquo tags - often used to compare the most popular tags to the tags predicted by a particular personalization strategy - investigating whether the personalization is worth the effort and is able to outperform the easily available baseline
[Guy et al Social Media Recommendation based on People and Tags SIGIRrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
recommendation 13dimensions
Predict relevantusefulinteresting items13for a given user (in a given context)13
itrsquos often a ranking task
Social Web 2014 Lora Aroyo
Recommendation Systems
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
March 28 2013
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
commercial 13personalisation
httpwwwwiredcommagazine201111mf_artsyall1
Social Web 2014 Lora Aroyo
filter bubble
u1 likes u2
likes likes u1 likes Pulp Fiction
Social Web 2014 Lora Aroyo
Collaborative Filteringbull Memory-based User-Item matrix ratingspreferences of users =gt compute
similarity between users amp recommend items of similar users13
bull Model-based Item-Item matrix similarity (eg based on user ratings) between items =gt recommend items that are similar to the ones the user likes13
bull Model-based Clustering cluster users according to their preferences =gt recommend items of users that belong to the same cluster13
bull Model-based Bayesian networks P(u likes item B | u likes item A) = how likely is it that a user who likes item A will like item B learn probabilities from user ratingspreferences13
bull Others rule-based other data mining techniques13
bull complete input data is required13
bull pre-computation not possible13
bull does not scale well 13
bull high quality of recommendations13
bull abstraction (model) of input data13
bull pre-computation (partially) possible (model has to be re-built from time to time)13
bull scales better13
bull abstraction may reduce recommendation quality
Social Web 2014 Lora Aroyo
Memory vs Model-based
bull collaborative filtering lsquoneighborhoodsrsquo of people with similar interest amp recommending items based on likings in neighborhood13
bull limitations next to lsquocold startrsquo and lsquosparsityrsquo the lack of control (over onersquos neighborhood) is also a problem ie cannot add lsquotrustedrsquo people nor exclude lsquostrangersquo ones13
bull therefore interest in lsquosocial recommendersrsquo where presence of social connections defines the similarity in interests (eg social tagging CiteULike)13
bull does a social connection indicate user interest similarity13
bull how much users interest similarity depends on the strength of their connection13
bull is it feasible to use a social network as a personalized recommendation
[Lin amp Brusilovsky Social Networks and Interest Similarity The Case of CiteULike HTrsquo10]Social Web 2014 Lora Aroyo
Social Networks amp Interest Similarity
bull unilaterally connected pairs have more common itemsmetadatatags than non-connected pairs13
bull highest similarity for direct connections - decreasing with the increase of distance between users in SN 13
bull reciprocal relationship users - significantly larger similarity than users in a unidirectional relationship 13
bull traditional item-level similarity may be less reliable to find similar users in social bookmarking systems13
bull peers connected by self-defined social connections could be a useful source for cross-recommendation
Social Web 2014 Lora Aroyo
Conclusions
bull Input characteristics of items amp interests of a user into characteristics of items =gt Recommend items that feature characteristics which meet the userrsquos interests13
bull Techniques13
bull Data mining methods Cluster items based on their characteristics =gt Infer usersrsquo interests into clusters13
bull IR methods Represent items amp users as term vectors =gt Compute similarity between user profile vector and items13
bull Utility-based methods Utility function that gets an item as input the parameters of the utility function are customized via preferences of a user
Social Web 2014 Lora Aroyo
Content-based Recommendations
Government stops renovation of tower bridge Oct 13th 2011
Tower Bridge today Under construction
Tower Bridge is a combined bascule and suspension bridge in London England over the River Thames
Category politics england Related Twiper news bob Why do they stop tohellip [more] mary London stops renohellip [more]
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
02 0 0
02 04 01 01
= a
Weighting strategy - occurrence frequency - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
Content Features
RT Government stops renovation of tower bridge Oct 13th 2011
Userrsquos Twitter history
I am in London at the moment Oct 13th 2011
I am doing sports Oct 12th 2011
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
0 01 0
05 02 02 0
= u
Weighting strategy - occurrence frequency (eg smoothened by occurrence time recent concepts are more important - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
User Model
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
u 0
01 0
05 02 02 0
candidate items user a
02 0 0
02 04 01 01
b 0 0 0
08 02 0 0
c 0
05 02 0 0 0
03
cosine similarities
a b c
u 067 092 014
Ranking of recommended items 1 b 2 a 3 c
based on slides from Fabien Abel
Recommendations
Social Web 2014 Lora Aroyo
RecSys Issues
bull Cold-start problem (new user problem) nolittle data available to infer preferences of new users
bull Changing User Preferences user interests may change over time
bull Sparsity problem (new item problem) item descriptions are sparse eg not many user rated or tagged an item
bull Lack of Diversity (overfitting) when adapting too strongly to the preferences of users they might see samesimilar recommendations
bull Use the right context users do things which might not be relevant for their user model eg try out things do stuff for other people
bull Research challenge right balance between serendipity amp personalization
bull Research challenge right way to use the influence of recommendations on userrsquos behavior
one machine13vs 13
humans
image source httpwwwflickrcomphotosbionicteaching1375254387Social Web 2014 Lora Aroyo
Hands-on Teaser
bull Your Facebook Friendsrsquo popularity in a spread sheet13bull Locations of your Facebook Friends13bull Tag Cloud of your wall posts
bull Ask the user explicitly learn13bull NLP intelligent dialogues13bull Bayesian networks Hidden Markov models13
bull Observe the user learn 13bull Logs machine learning13bull Clustering classification data mining
bull Interactive user modeling mixture of direct inputs of a user observations and inferences
Social Web 2014 Lora Aroyo
User Model Elicitation
httphunchcom
Social Web 2014 Lora Aroyo
httpfarm1staticflickrcom155413650229_31ef379b0b_bjpg
bull set of characteristics (eg attribute-value pairs) that describe a group of users13
bull user is not assigned to a single stereotype - user profile can feature characteristics of several different stereotypes
Social Web 2014 Lora Aroyo
User Stereotypes
based on slides from Fabien Abel
User Modeling (4 building blocks)
Semantic Enrichment Linkage and Alignment
Personalized News Recommender
Profile
I want my
personalized news recommendations
based on slides from Fabien Abel
Can we infer a Twitter-based User Profile
User Modeling Building Blocks
Profile concept weight
time
1 Which tweets of the user should be
analyzed
Morning Afternoon Night
1 Temporal Constraints
June 27 July 4 July 11
(b) temporal patterns
weekends start end
(a) time period
based on slides from Fabien Abel
User Modeling Building Blocks
Profile concept weight
2 Profile Type
Francesca Schiavone won French Open fo2010
Francesca Schiavone
French Open
Francesca Schiavone French Open entity-based
Sport T
T topic-based
2 What type of concepts should represent ldquointerestsrdquo
fo2010
fo2010 hashtag-based
1 Temporal Constraints
time
June 27 July 4 July 11
based on slides from Fabien Abel
User Modeling Building Blocks
Profile concept weight
2 Profile Type
Francesca Schiavone won httpbitly2f4t7a
Francesca Schiavone
3 Further enrich the semantics of tweets
1 Temporal Constraints
3 Semantic Enrichment
Francesca Schiavone
Francesca wins French Open Thirty in womens tennis is primordially old an age when agility and desire recedes as the hellip
French Open
Tennis
French Open
Tennis
(b) further enrichment
(a) tweet-based
based on slides from Fabien Abel
User Modeling Building Blocks
Profile concept weight
2 Profile Type
4 How to weight the concepts
1 Temporal Constraints
3 Semantic Enrichment
Francesca Schiavone
French Open
Tennis
4 Weighting Scheme
time
June 27 July 4 July 11
weight(Francesca Schiavone)
Concept frequency (TF)
4
3 6
TFxIDF Time-sensitive
weight(French Open)
weight(Tennis)
based on slides from Fabien Abel
Social Web 2014 Lora Aroyo
Observationsbull Profile characteristics
bull Semantic enrichment solves sparsity problems13bull Profiles change over time recent profiles reflect better
current user demands13bull Temporal patterns weekend profiles differ significantly
from weekday profiles13
bull Impact on recommendations bull The more fine-grained the concepts the better the
recommendation performance entity-based gt topic-based gt hashtag-based 13
bull Semantic enrichment improves recommendation quality 13bull Time-sensitivity (adapting to trends) improves
performance
Social Web 2014 Lora Aroyo
User Modelingit is not about putting everything in a user profile 13
it is about making the right choices
Social Web 2014 Lora Aroyo
User AdaptationKnowing the user to adapt a system or interface13
to improve the system functionality and user experience
A Jameson Adaptive interfaces and agents The HCI handbook fundamentals evolving technologies and emerging applications pp 305ndash330 2003
User-Adaptive Systems
user modeling
user profile
observations data and information about user
profile analysis
adaptation decisions
user modeling (infer current musical taste)
user profile interests in
genres artists tags
history of songs like ban pause skip
compare profile with possible next
songs to play
next song to be played
based on slides from Fabien Abel
Lastfm adapts to your music taste
bull Overfitting ldquobubble effectsrdquo loss of serendipity problem 13bull systems may adapt too strongly to the interestsbehavior13bull eg an adaptive radio station may always play the same or
very similar songs13bull We search for the right balance between novelty and
relevance for the user13
bull ldquoLost in Hyperspacerdquo problem 13bull when adapting the navigation ndash ie the links on which
users can click to findaccess information 13bull eg re-orderinghiding of menu items may lead to
confusion
Social Web 2014 Lora Aroyo
Issues in User-Adaptive Systems
Social Web 2014 Lora Aroyo
What is good user modelling amp personalisation
httpwwwflickrcomphotosbellarosebyliz4729613108
bull From the consumer perspective of an adaptive system 13
bull From the provider perspective of an adaptive system
Adaptive system maximizes satisfaction of the user
hard to measureobtain
Adaptive system maximizes the profit
influence of UM amp personalization may be hard to measureobtain
Social Web 2014 Lora Aroyo
Success Perspectives
bull User studies askobserve (selected) people whether you did a good job13
bull Log analysis Analyze (click) data and infer whether you did a good job13
bull Evaluation of user modeling13
bull measure quality of profiles directly eg measure overlap with existing (true) profiles or let people judge the quality of the generated user profiles 13
bull measure quality of application that exploits the user profile eg apply user modeling strategies in a recommender system
Social Web 2014 Lora Aroyo
Evaluation Strategies
time
item A
item B
item C
item D
item E
item G item H
item F
training data test data (ground truth)
Strategy X
User Modeling strategies to compare
Strategy Y
Strategy Z
Recommender
training data Recommendations
item H
X
item R
item M
Y
item H
item G
item N
item H
Z
item F
item M
measure quality
Social Web 2014 Lora Aroyo
Evaluating User Modeling in RecSys
Possible Metricsbull The usual IR metrics13
bull Precision fraction of retrieved items that are relevant13
bull Recall fraction of relevant items that have been retrieved13
bull F-Measure (harmonic) mean of precision and recall13
bull Metrics for evaluating recommendation (rankings)13
bull Mean Reciprocal Rank (MRR) of first relevant item13
bull Successk probability that relevant item occurs within the top k13
bull If a true ranking is given rank correlations 13
bull Precisionk Recallk amp F-Measurek13
bull Metrics for evaluating prediction of user preferences13
bull MAE = Mean Absolute Error13
bull TrueFalse PositivesNegatives runs
performance strategy X baseline
Is strategy X better than the baseline
Social Web 2014 Lora Aroyo
bull [Rae et al] a typical example of how to investigate and evaluate a proposal for improving (tag) recommendations (using social networks)13
bull Task test how well the different strategies (different tag contexts) can be used for tag predictionrecommendation13
bull Steps13
1 Gather a dataset of tag data part of which can be used as input and aim to test the recommendation on the remaining tag data13
2 Use the input data and calculate for the different strategies the predictions13
3 Measure the performance using standard (IR) metrics Precision of the top 5 recommended tags (P5) Mean Reciprocal Rank (MRR) Mean Average Precision (MAP)13
4 Test the results for statistical significance using T-test relative to the baseline (eg existing approach competitive approach)
[Rae et al Improving Tag Recommendations Using Social Networks RIAOrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
bull [Guy et al] another example of a similar evaluation approach13
bull The different strategies differ in the way people amp tags are used with tag-based systems there are complex relationships between users tags and items and strategies aim to find the relevant aspects of these relationships for modeling and recommendation13
bull The baseline is the lsquomost popularrsquo tags - often used to compare the most popular tags to the tags predicted by a particular personalization strategy - investigating whether the personalization is worth the effort and is able to outperform the easily available baseline
[Guy et al Social Media Recommendation based on People and Tags SIGIRrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
recommendation 13dimensions
Predict relevantusefulinteresting items13for a given user (in a given context)13
itrsquos often a ranking task
Social Web 2014 Lora Aroyo
Recommendation Systems
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
March 28 2013
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
commercial 13personalisation
httpwwwwiredcommagazine201111mf_artsyall1
Social Web 2014 Lora Aroyo
filter bubble
u1 likes u2
likes likes u1 likes Pulp Fiction
Social Web 2014 Lora Aroyo
Collaborative Filteringbull Memory-based User-Item matrix ratingspreferences of users =gt compute
similarity between users amp recommend items of similar users13
bull Model-based Item-Item matrix similarity (eg based on user ratings) between items =gt recommend items that are similar to the ones the user likes13
bull Model-based Clustering cluster users according to their preferences =gt recommend items of users that belong to the same cluster13
bull Model-based Bayesian networks P(u likes item B | u likes item A) = how likely is it that a user who likes item A will like item B learn probabilities from user ratingspreferences13
bull Others rule-based other data mining techniques13
bull complete input data is required13
bull pre-computation not possible13
bull does not scale well 13
bull high quality of recommendations13
bull abstraction (model) of input data13
bull pre-computation (partially) possible (model has to be re-built from time to time)13
bull scales better13
bull abstraction may reduce recommendation quality
Social Web 2014 Lora Aroyo
Memory vs Model-based
bull collaborative filtering lsquoneighborhoodsrsquo of people with similar interest amp recommending items based on likings in neighborhood13
bull limitations next to lsquocold startrsquo and lsquosparsityrsquo the lack of control (over onersquos neighborhood) is also a problem ie cannot add lsquotrustedrsquo people nor exclude lsquostrangersquo ones13
bull therefore interest in lsquosocial recommendersrsquo where presence of social connections defines the similarity in interests (eg social tagging CiteULike)13
bull does a social connection indicate user interest similarity13
bull how much users interest similarity depends on the strength of their connection13
bull is it feasible to use a social network as a personalized recommendation
[Lin amp Brusilovsky Social Networks and Interest Similarity The Case of CiteULike HTrsquo10]Social Web 2014 Lora Aroyo
Social Networks amp Interest Similarity
bull unilaterally connected pairs have more common itemsmetadatatags than non-connected pairs13
bull highest similarity for direct connections - decreasing with the increase of distance between users in SN 13
bull reciprocal relationship users - significantly larger similarity than users in a unidirectional relationship 13
bull traditional item-level similarity may be less reliable to find similar users in social bookmarking systems13
bull peers connected by self-defined social connections could be a useful source for cross-recommendation
Social Web 2014 Lora Aroyo
Conclusions
bull Input characteristics of items amp interests of a user into characteristics of items =gt Recommend items that feature characteristics which meet the userrsquos interests13
bull Techniques13
bull Data mining methods Cluster items based on their characteristics =gt Infer usersrsquo interests into clusters13
bull IR methods Represent items amp users as term vectors =gt Compute similarity between user profile vector and items13
bull Utility-based methods Utility function that gets an item as input the parameters of the utility function are customized via preferences of a user
Social Web 2014 Lora Aroyo
Content-based Recommendations
Government stops renovation of tower bridge Oct 13th 2011
Tower Bridge today Under construction
Tower Bridge is a combined bascule and suspension bridge in London England over the River Thames
Category politics england Related Twiper news bob Why do they stop tohellip [more] mary London stops renohellip [more]
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
02 0 0
02 04 01 01
= a
Weighting strategy - occurrence frequency - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
Content Features
RT Government stops renovation of tower bridge Oct 13th 2011
Userrsquos Twitter history
I am in London at the moment Oct 13th 2011
I am doing sports Oct 12th 2011
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
0 01 0
05 02 02 0
= u
Weighting strategy - occurrence frequency (eg smoothened by occurrence time recent concepts are more important - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
User Model
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
u 0
01 0
05 02 02 0
candidate items user a
02 0 0
02 04 01 01
b 0 0 0
08 02 0 0
c 0
05 02 0 0 0
03
cosine similarities
a b c
u 067 092 014
Ranking of recommended items 1 b 2 a 3 c
based on slides from Fabien Abel
Recommendations
Social Web 2014 Lora Aroyo
RecSys Issues
bull Cold-start problem (new user problem) nolittle data available to infer preferences of new users
bull Changing User Preferences user interests may change over time
bull Sparsity problem (new item problem) item descriptions are sparse eg not many user rated or tagged an item
bull Lack of Diversity (overfitting) when adapting too strongly to the preferences of users they might see samesimilar recommendations
bull Use the right context users do things which might not be relevant for their user model eg try out things do stuff for other people
bull Research challenge right balance between serendipity amp personalization
bull Research challenge right way to use the influence of recommendations on userrsquos behavior
one machine13vs 13
humans
image source httpwwwflickrcomphotosbionicteaching1375254387Social Web 2014 Lora Aroyo
Hands-on Teaser
bull Your Facebook Friendsrsquo popularity in a spread sheet13bull Locations of your Facebook Friends13bull Tag Cloud of your wall posts
httphunchcom
Social Web 2014 Lora Aroyo
httpfarm1staticflickrcom155413650229_31ef379b0b_bjpg
bull set of characteristics (eg attribute-value pairs) that describe a group of users13
bull user is not assigned to a single stereotype - user profile can feature characteristics of several different stereotypes
Social Web 2014 Lora Aroyo
User Stereotypes
based on slides from Fabien Abel
User Modeling (4 building blocks)
Semantic Enrichment Linkage and Alignment
Personalized News Recommender
Profile
I want my
personalized news recommendations
based on slides from Fabien Abel
Can we infer a Twitter-based User Profile
User Modeling Building Blocks
Profile concept weight
time
1 Which tweets of the user should be
analyzed
Morning Afternoon Night
1 Temporal Constraints
June 27 July 4 July 11
(b) temporal patterns
weekends start end
(a) time period
based on slides from Fabien Abel
User Modeling Building Blocks
Profile concept weight
2 Profile Type
Francesca Schiavone won French Open fo2010
Francesca Schiavone
French Open
Francesca Schiavone French Open entity-based
Sport T
T topic-based
2 What type of concepts should represent ldquointerestsrdquo
fo2010
fo2010 hashtag-based
1 Temporal Constraints
time
June 27 July 4 July 11
based on slides from Fabien Abel
User Modeling Building Blocks
Profile concept weight
2 Profile Type
Francesca Schiavone won httpbitly2f4t7a
Francesca Schiavone
3 Further enrich the semantics of tweets
1 Temporal Constraints
3 Semantic Enrichment
Francesca Schiavone
Francesca wins French Open Thirty in womens tennis is primordially old an age when agility and desire recedes as the hellip
French Open
Tennis
French Open
Tennis
(b) further enrichment
(a) tweet-based
based on slides from Fabien Abel
User Modeling Building Blocks
Profile concept weight
2 Profile Type
4 How to weight the concepts
1 Temporal Constraints
3 Semantic Enrichment
Francesca Schiavone
French Open
Tennis
4 Weighting Scheme
time
June 27 July 4 July 11
weight(Francesca Schiavone)
Concept frequency (TF)
4
3 6
TFxIDF Time-sensitive
weight(French Open)
weight(Tennis)
based on slides from Fabien Abel
Social Web 2014 Lora Aroyo
Observationsbull Profile characteristics
bull Semantic enrichment solves sparsity problems13bull Profiles change over time recent profiles reflect better
current user demands13bull Temporal patterns weekend profiles differ significantly
from weekday profiles13
bull Impact on recommendations bull The more fine-grained the concepts the better the
recommendation performance entity-based gt topic-based gt hashtag-based 13
bull Semantic enrichment improves recommendation quality 13bull Time-sensitivity (adapting to trends) improves
performance
Social Web 2014 Lora Aroyo
User Modelingit is not about putting everything in a user profile 13
it is about making the right choices
Social Web 2014 Lora Aroyo
User AdaptationKnowing the user to adapt a system or interface13
to improve the system functionality and user experience
A Jameson Adaptive interfaces and agents The HCI handbook fundamentals evolving technologies and emerging applications pp 305ndash330 2003
User-Adaptive Systems
user modeling
user profile
observations data and information about user
profile analysis
adaptation decisions
user modeling (infer current musical taste)
user profile interests in
genres artists tags
history of songs like ban pause skip
compare profile with possible next
songs to play
next song to be played
based on slides from Fabien Abel
Lastfm adapts to your music taste
bull Overfitting ldquobubble effectsrdquo loss of serendipity problem 13bull systems may adapt too strongly to the interestsbehavior13bull eg an adaptive radio station may always play the same or
very similar songs13bull We search for the right balance between novelty and
relevance for the user13
bull ldquoLost in Hyperspacerdquo problem 13bull when adapting the navigation ndash ie the links on which
users can click to findaccess information 13bull eg re-orderinghiding of menu items may lead to
confusion
Social Web 2014 Lora Aroyo
Issues in User-Adaptive Systems
Social Web 2014 Lora Aroyo
What is good user modelling amp personalisation
httpwwwflickrcomphotosbellarosebyliz4729613108
bull From the consumer perspective of an adaptive system 13
bull From the provider perspective of an adaptive system
Adaptive system maximizes satisfaction of the user
hard to measureobtain
Adaptive system maximizes the profit
influence of UM amp personalization may be hard to measureobtain
Social Web 2014 Lora Aroyo
Success Perspectives
bull User studies askobserve (selected) people whether you did a good job13
bull Log analysis Analyze (click) data and infer whether you did a good job13
bull Evaluation of user modeling13
bull measure quality of profiles directly eg measure overlap with existing (true) profiles or let people judge the quality of the generated user profiles 13
bull measure quality of application that exploits the user profile eg apply user modeling strategies in a recommender system
Social Web 2014 Lora Aroyo
Evaluation Strategies
time
item A
item B
item C
item D
item E
item G item H
item F
training data test data (ground truth)
Strategy X
User Modeling strategies to compare
Strategy Y
Strategy Z
Recommender
training data Recommendations
item H
X
item R
item M
Y
item H
item G
item N
item H
Z
item F
item M
measure quality
Social Web 2014 Lora Aroyo
Evaluating User Modeling in RecSys
Possible Metricsbull The usual IR metrics13
bull Precision fraction of retrieved items that are relevant13
bull Recall fraction of relevant items that have been retrieved13
bull F-Measure (harmonic) mean of precision and recall13
bull Metrics for evaluating recommendation (rankings)13
bull Mean Reciprocal Rank (MRR) of first relevant item13
bull Successk probability that relevant item occurs within the top k13
bull If a true ranking is given rank correlations 13
bull Precisionk Recallk amp F-Measurek13
bull Metrics for evaluating prediction of user preferences13
bull MAE = Mean Absolute Error13
bull TrueFalse PositivesNegatives runs
performance strategy X baseline
Is strategy X better than the baseline
Social Web 2014 Lora Aroyo
bull [Rae et al] a typical example of how to investigate and evaluate a proposal for improving (tag) recommendations (using social networks)13
bull Task test how well the different strategies (different tag contexts) can be used for tag predictionrecommendation13
bull Steps13
1 Gather a dataset of tag data part of which can be used as input and aim to test the recommendation on the remaining tag data13
2 Use the input data and calculate for the different strategies the predictions13
3 Measure the performance using standard (IR) metrics Precision of the top 5 recommended tags (P5) Mean Reciprocal Rank (MRR) Mean Average Precision (MAP)13
4 Test the results for statistical significance using T-test relative to the baseline (eg existing approach competitive approach)
[Rae et al Improving Tag Recommendations Using Social Networks RIAOrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
bull [Guy et al] another example of a similar evaluation approach13
bull The different strategies differ in the way people amp tags are used with tag-based systems there are complex relationships between users tags and items and strategies aim to find the relevant aspects of these relationships for modeling and recommendation13
bull The baseline is the lsquomost popularrsquo tags - often used to compare the most popular tags to the tags predicted by a particular personalization strategy - investigating whether the personalization is worth the effort and is able to outperform the easily available baseline
[Guy et al Social Media Recommendation based on People and Tags SIGIRrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
recommendation 13dimensions
Predict relevantusefulinteresting items13for a given user (in a given context)13
itrsquos often a ranking task
Social Web 2014 Lora Aroyo
Recommendation Systems
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
March 28 2013
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
commercial 13personalisation
httpwwwwiredcommagazine201111mf_artsyall1
Social Web 2014 Lora Aroyo
filter bubble
u1 likes u2
likes likes u1 likes Pulp Fiction
Social Web 2014 Lora Aroyo
Collaborative Filteringbull Memory-based User-Item matrix ratingspreferences of users =gt compute
similarity between users amp recommend items of similar users13
bull Model-based Item-Item matrix similarity (eg based on user ratings) between items =gt recommend items that are similar to the ones the user likes13
bull Model-based Clustering cluster users according to their preferences =gt recommend items of users that belong to the same cluster13
bull Model-based Bayesian networks P(u likes item B | u likes item A) = how likely is it that a user who likes item A will like item B learn probabilities from user ratingspreferences13
bull Others rule-based other data mining techniques13
bull complete input data is required13
bull pre-computation not possible13
bull does not scale well 13
bull high quality of recommendations13
bull abstraction (model) of input data13
bull pre-computation (partially) possible (model has to be re-built from time to time)13
bull scales better13
bull abstraction may reduce recommendation quality
Social Web 2014 Lora Aroyo
Memory vs Model-based
bull collaborative filtering lsquoneighborhoodsrsquo of people with similar interest amp recommending items based on likings in neighborhood13
bull limitations next to lsquocold startrsquo and lsquosparsityrsquo the lack of control (over onersquos neighborhood) is also a problem ie cannot add lsquotrustedrsquo people nor exclude lsquostrangersquo ones13
bull therefore interest in lsquosocial recommendersrsquo where presence of social connections defines the similarity in interests (eg social tagging CiteULike)13
bull does a social connection indicate user interest similarity13
bull how much users interest similarity depends on the strength of their connection13
bull is it feasible to use a social network as a personalized recommendation
[Lin amp Brusilovsky Social Networks and Interest Similarity The Case of CiteULike HTrsquo10]Social Web 2014 Lora Aroyo
Social Networks amp Interest Similarity
bull unilaterally connected pairs have more common itemsmetadatatags than non-connected pairs13
bull highest similarity for direct connections - decreasing with the increase of distance between users in SN 13
bull reciprocal relationship users - significantly larger similarity than users in a unidirectional relationship 13
bull traditional item-level similarity may be less reliable to find similar users in social bookmarking systems13
bull peers connected by self-defined social connections could be a useful source for cross-recommendation
Social Web 2014 Lora Aroyo
Conclusions
bull Input characteristics of items amp interests of a user into characteristics of items =gt Recommend items that feature characteristics which meet the userrsquos interests13
bull Techniques13
bull Data mining methods Cluster items based on their characteristics =gt Infer usersrsquo interests into clusters13
bull IR methods Represent items amp users as term vectors =gt Compute similarity between user profile vector and items13
bull Utility-based methods Utility function that gets an item as input the parameters of the utility function are customized via preferences of a user
Social Web 2014 Lora Aroyo
Content-based Recommendations
Government stops renovation of tower bridge Oct 13th 2011
Tower Bridge today Under construction
Tower Bridge is a combined bascule and suspension bridge in London England over the River Thames
Category politics england Related Twiper news bob Why do they stop tohellip [more] mary London stops renohellip [more]
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
02 0 0
02 04 01 01
= a
Weighting strategy - occurrence frequency - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
Content Features
RT Government stops renovation of tower bridge Oct 13th 2011
Userrsquos Twitter history
I am in London at the moment Oct 13th 2011
I am doing sports Oct 12th 2011
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
0 01 0
05 02 02 0
= u
Weighting strategy - occurrence frequency (eg smoothened by occurrence time recent concepts are more important - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
User Model
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
u 0
01 0
05 02 02 0
candidate items user a
02 0 0
02 04 01 01
b 0 0 0
08 02 0 0
c 0
05 02 0 0 0
03
cosine similarities
a b c
u 067 092 014
Ranking of recommended items 1 b 2 a 3 c
based on slides from Fabien Abel
Recommendations
Social Web 2014 Lora Aroyo
RecSys Issues
bull Cold-start problem (new user problem) nolittle data available to infer preferences of new users
bull Changing User Preferences user interests may change over time
bull Sparsity problem (new item problem) item descriptions are sparse eg not many user rated or tagged an item
bull Lack of Diversity (overfitting) when adapting too strongly to the preferences of users they might see samesimilar recommendations
bull Use the right context users do things which might not be relevant for their user model eg try out things do stuff for other people
bull Research challenge right balance between serendipity amp personalization
bull Research challenge right way to use the influence of recommendations on userrsquos behavior
one machine13vs 13
humans
image source httpwwwflickrcomphotosbionicteaching1375254387Social Web 2014 Lora Aroyo
Hands-on Teaser
bull Your Facebook Friendsrsquo popularity in a spread sheet13bull Locations of your Facebook Friends13bull Tag Cloud of your wall posts
httpfarm1staticflickrcom155413650229_31ef379b0b_bjpg
bull set of characteristics (eg attribute-value pairs) that describe a group of users13
bull user is not assigned to a single stereotype - user profile can feature characteristics of several different stereotypes
Social Web 2014 Lora Aroyo
User Stereotypes
based on slides from Fabien Abel
User Modeling (4 building blocks)
Semantic Enrichment Linkage and Alignment
Personalized News Recommender
Profile
I want my
personalized news recommendations
based on slides from Fabien Abel
Can we infer a Twitter-based User Profile
User Modeling Building Blocks
Profile concept weight
time
1 Which tweets of the user should be
analyzed
Morning Afternoon Night
1 Temporal Constraints
June 27 July 4 July 11
(b) temporal patterns
weekends start end
(a) time period
based on slides from Fabien Abel
User Modeling Building Blocks
Profile concept weight
2 Profile Type
Francesca Schiavone won French Open fo2010
Francesca Schiavone
French Open
Francesca Schiavone French Open entity-based
Sport T
T topic-based
2 What type of concepts should represent ldquointerestsrdquo
fo2010
fo2010 hashtag-based
1 Temporal Constraints
time
June 27 July 4 July 11
based on slides from Fabien Abel
User Modeling Building Blocks
Profile concept weight
2 Profile Type
Francesca Schiavone won httpbitly2f4t7a
Francesca Schiavone
3 Further enrich the semantics of tweets
1 Temporal Constraints
3 Semantic Enrichment
Francesca Schiavone
Francesca wins French Open Thirty in womens tennis is primordially old an age when agility and desire recedes as the hellip
French Open
Tennis
French Open
Tennis
(b) further enrichment
(a) tweet-based
based on slides from Fabien Abel
User Modeling Building Blocks
Profile concept weight
2 Profile Type
4 How to weight the concepts
1 Temporal Constraints
3 Semantic Enrichment
Francesca Schiavone
French Open
Tennis
4 Weighting Scheme
time
June 27 July 4 July 11
weight(Francesca Schiavone)
Concept frequency (TF)
4
3 6
TFxIDF Time-sensitive
weight(French Open)
weight(Tennis)
based on slides from Fabien Abel
Social Web 2014 Lora Aroyo
Observationsbull Profile characteristics
bull Semantic enrichment solves sparsity problems13bull Profiles change over time recent profiles reflect better
current user demands13bull Temporal patterns weekend profiles differ significantly
from weekday profiles13
bull Impact on recommendations bull The more fine-grained the concepts the better the
recommendation performance entity-based gt topic-based gt hashtag-based 13
bull Semantic enrichment improves recommendation quality 13bull Time-sensitivity (adapting to trends) improves
performance
Social Web 2014 Lora Aroyo
User Modelingit is not about putting everything in a user profile 13
it is about making the right choices
Social Web 2014 Lora Aroyo
User AdaptationKnowing the user to adapt a system or interface13
to improve the system functionality and user experience
A Jameson Adaptive interfaces and agents The HCI handbook fundamentals evolving technologies and emerging applications pp 305ndash330 2003
User-Adaptive Systems
user modeling
user profile
observations data and information about user
profile analysis
adaptation decisions
user modeling (infer current musical taste)
user profile interests in
genres artists tags
history of songs like ban pause skip
compare profile with possible next
songs to play
next song to be played
based on slides from Fabien Abel
Lastfm adapts to your music taste
bull Overfitting ldquobubble effectsrdquo loss of serendipity problem 13bull systems may adapt too strongly to the interestsbehavior13bull eg an adaptive radio station may always play the same or
very similar songs13bull We search for the right balance between novelty and
relevance for the user13
bull ldquoLost in Hyperspacerdquo problem 13bull when adapting the navigation ndash ie the links on which
users can click to findaccess information 13bull eg re-orderinghiding of menu items may lead to
confusion
Social Web 2014 Lora Aroyo
Issues in User-Adaptive Systems
Social Web 2014 Lora Aroyo
What is good user modelling amp personalisation
httpwwwflickrcomphotosbellarosebyliz4729613108
bull From the consumer perspective of an adaptive system 13
bull From the provider perspective of an adaptive system
Adaptive system maximizes satisfaction of the user
hard to measureobtain
Adaptive system maximizes the profit
influence of UM amp personalization may be hard to measureobtain
Social Web 2014 Lora Aroyo
Success Perspectives
bull User studies askobserve (selected) people whether you did a good job13
bull Log analysis Analyze (click) data and infer whether you did a good job13
bull Evaluation of user modeling13
bull measure quality of profiles directly eg measure overlap with existing (true) profiles or let people judge the quality of the generated user profiles 13
bull measure quality of application that exploits the user profile eg apply user modeling strategies in a recommender system
Social Web 2014 Lora Aroyo
Evaluation Strategies
time
item A
item B
item C
item D
item E
item G item H
item F
training data test data (ground truth)
Strategy X
User Modeling strategies to compare
Strategy Y
Strategy Z
Recommender
training data Recommendations
item H
X
item R
item M
Y
item H
item G
item N
item H
Z
item F
item M
measure quality
Social Web 2014 Lora Aroyo
Evaluating User Modeling in RecSys
Possible Metricsbull The usual IR metrics13
bull Precision fraction of retrieved items that are relevant13
bull Recall fraction of relevant items that have been retrieved13
bull F-Measure (harmonic) mean of precision and recall13
bull Metrics for evaluating recommendation (rankings)13
bull Mean Reciprocal Rank (MRR) of first relevant item13
bull Successk probability that relevant item occurs within the top k13
bull If a true ranking is given rank correlations 13
bull Precisionk Recallk amp F-Measurek13
bull Metrics for evaluating prediction of user preferences13
bull MAE = Mean Absolute Error13
bull TrueFalse PositivesNegatives runs
performance strategy X baseline
Is strategy X better than the baseline
Social Web 2014 Lora Aroyo
bull [Rae et al] a typical example of how to investigate and evaluate a proposal for improving (tag) recommendations (using social networks)13
bull Task test how well the different strategies (different tag contexts) can be used for tag predictionrecommendation13
bull Steps13
1 Gather a dataset of tag data part of which can be used as input and aim to test the recommendation on the remaining tag data13
2 Use the input data and calculate for the different strategies the predictions13
3 Measure the performance using standard (IR) metrics Precision of the top 5 recommended tags (P5) Mean Reciprocal Rank (MRR) Mean Average Precision (MAP)13
4 Test the results for statistical significance using T-test relative to the baseline (eg existing approach competitive approach)
[Rae et al Improving Tag Recommendations Using Social Networks RIAOrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
bull [Guy et al] another example of a similar evaluation approach13
bull The different strategies differ in the way people amp tags are used with tag-based systems there are complex relationships between users tags and items and strategies aim to find the relevant aspects of these relationships for modeling and recommendation13
bull The baseline is the lsquomost popularrsquo tags - often used to compare the most popular tags to the tags predicted by a particular personalization strategy - investigating whether the personalization is worth the effort and is able to outperform the easily available baseline
[Guy et al Social Media Recommendation based on People and Tags SIGIRrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
recommendation 13dimensions
Predict relevantusefulinteresting items13for a given user (in a given context)13
itrsquos often a ranking task
Social Web 2014 Lora Aroyo
Recommendation Systems
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
March 28 2013
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
commercial 13personalisation
httpwwwwiredcommagazine201111mf_artsyall1
Social Web 2014 Lora Aroyo
filter bubble
u1 likes u2
likes likes u1 likes Pulp Fiction
Social Web 2014 Lora Aroyo
Collaborative Filteringbull Memory-based User-Item matrix ratingspreferences of users =gt compute
similarity between users amp recommend items of similar users13
bull Model-based Item-Item matrix similarity (eg based on user ratings) between items =gt recommend items that are similar to the ones the user likes13
bull Model-based Clustering cluster users according to their preferences =gt recommend items of users that belong to the same cluster13
bull Model-based Bayesian networks P(u likes item B | u likes item A) = how likely is it that a user who likes item A will like item B learn probabilities from user ratingspreferences13
bull Others rule-based other data mining techniques13
bull complete input data is required13
bull pre-computation not possible13
bull does not scale well 13
bull high quality of recommendations13
bull abstraction (model) of input data13
bull pre-computation (partially) possible (model has to be re-built from time to time)13
bull scales better13
bull abstraction may reduce recommendation quality
Social Web 2014 Lora Aroyo
Memory vs Model-based
bull collaborative filtering lsquoneighborhoodsrsquo of people with similar interest amp recommending items based on likings in neighborhood13
bull limitations next to lsquocold startrsquo and lsquosparsityrsquo the lack of control (over onersquos neighborhood) is also a problem ie cannot add lsquotrustedrsquo people nor exclude lsquostrangersquo ones13
bull therefore interest in lsquosocial recommendersrsquo where presence of social connections defines the similarity in interests (eg social tagging CiteULike)13
bull does a social connection indicate user interest similarity13
bull how much users interest similarity depends on the strength of their connection13
bull is it feasible to use a social network as a personalized recommendation
[Lin amp Brusilovsky Social Networks and Interest Similarity The Case of CiteULike HTrsquo10]Social Web 2014 Lora Aroyo
Social Networks amp Interest Similarity
bull unilaterally connected pairs have more common itemsmetadatatags than non-connected pairs13
bull highest similarity for direct connections - decreasing with the increase of distance between users in SN 13
bull reciprocal relationship users - significantly larger similarity than users in a unidirectional relationship 13
bull traditional item-level similarity may be less reliable to find similar users in social bookmarking systems13
bull peers connected by self-defined social connections could be a useful source for cross-recommendation
Social Web 2014 Lora Aroyo
Conclusions
bull Input characteristics of items amp interests of a user into characteristics of items =gt Recommend items that feature characteristics which meet the userrsquos interests13
bull Techniques13
bull Data mining methods Cluster items based on their characteristics =gt Infer usersrsquo interests into clusters13
bull IR methods Represent items amp users as term vectors =gt Compute similarity between user profile vector and items13
bull Utility-based methods Utility function that gets an item as input the parameters of the utility function are customized via preferences of a user
Social Web 2014 Lora Aroyo
Content-based Recommendations
Government stops renovation of tower bridge Oct 13th 2011
Tower Bridge today Under construction
Tower Bridge is a combined bascule and suspension bridge in London England over the River Thames
Category politics england Related Twiper news bob Why do they stop tohellip [more] mary London stops renohellip [more]
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
02 0 0
02 04 01 01
= a
Weighting strategy - occurrence frequency - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
Content Features
RT Government stops renovation of tower bridge Oct 13th 2011
Userrsquos Twitter history
I am in London at the moment Oct 13th 2011
I am doing sports Oct 12th 2011
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
0 01 0
05 02 02 0
= u
Weighting strategy - occurrence frequency (eg smoothened by occurrence time recent concepts are more important - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
User Model
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
u 0
01 0
05 02 02 0
candidate items user a
02 0 0
02 04 01 01
b 0 0 0
08 02 0 0
c 0
05 02 0 0 0
03
cosine similarities
a b c
u 067 092 014
Ranking of recommended items 1 b 2 a 3 c
based on slides from Fabien Abel
Recommendations
Social Web 2014 Lora Aroyo
RecSys Issues
bull Cold-start problem (new user problem) nolittle data available to infer preferences of new users
bull Changing User Preferences user interests may change over time
bull Sparsity problem (new item problem) item descriptions are sparse eg not many user rated or tagged an item
bull Lack of Diversity (overfitting) when adapting too strongly to the preferences of users they might see samesimilar recommendations
bull Use the right context users do things which might not be relevant for their user model eg try out things do stuff for other people
bull Research challenge right balance between serendipity amp personalization
bull Research challenge right way to use the influence of recommendations on userrsquos behavior
one machine13vs 13
humans
image source httpwwwflickrcomphotosbionicteaching1375254387Social Web 2014 Lora Aroyo
Hands-on Teaser
bull Your Facebook Friendsrsquo popularity in a spread sheet13bull Locations of your Facebook Friends13bull Tag Cloud of your wall posts
based on slides from Fabien Abel
User Modeling (4 building blocks)
Semantic Enrichment Linkage and Alignment
Personalized News Recommender
Profile
I want my
personalized news recommendations
based on slides from Fabien Abel
Can we infer a Twitter-based User Profile
User Modeling Building Blocks
Profile concept weight
time
1 Which tweets of the user should be
analyzed
Morning Afternoon Night
1 Temporal Constraints
June 27 July 4 July 11
(b) temporal patterns
weekends start end
(a) time period
based on slides from Fabien Abel
User Modeling Building Blocks
Profile concept weight
2 Profile Type
Francesca Schiavone won French Open fo2010
Francesca Schiavone
French Open
Francesca Schiavone French Open entity-based
Sport T
T topic-based
2 What type of concepts should represent ldquointerestsrdquo
fo2010
fo2010 hashtag-based
1 Temporal Constraints
time
June 27 July 4 July 11
based on slides from Fabien Abel
User Modeling Building Blocks
Profile concept weight
2 Profile Type
Francesca Schiavone won httpbitly2f4t7a
Francesca Schiavone
3 Further enrich the semantics of tweets
1 Temporal Constraints
3 Semantic Enrichment
Francesca Schiavone
Francesca wins French Open Thirty in womens tennis is primordially old an age when agility and desire recedes as the hellip
French Open
Tennis
French Open
Tennis
(b) further enrichment
(a) tweet-based
based on slides from Fabien Abel
User Modeling Building Blocks
Profile concept weight
2 Profile Type
4 How to weight the concepts
1 Temporal Constraints
3 Semantic Enrichment
Francesca Schiavone
French Open
Tennis
4 Weighting Scheme
time
June 27 July 4 July 11
weight(Francesca Schiavone)
Concept frequency (TF)
4
3 6
TFxIDF Time-sensitive
weight(French Open)
weight(Tennis)
based on slides from Fabien Abel
Social Web 2014 Lora Aroyo
Observationsbull Profile characteristics
bull Semantic enrichment solves sparsity problems13bull Profiles change over time recent profiles reflect better
current user demands13bull Temporal patterns weekend profiles differ significantly
from weekday profiles13
bull Impact on recommendations bull The more fine-grained the concepts the better the
recommendation performance entity-based gt topic-based gt hashtag-based 13
bull Semantic enrichment improves recommendation quality 13bull Time-sensitivity (adapting to trends) improves
performance
Social Web 2014 Lora Aroyo
User Modelingit is not about putting everything in a user profile 13
it is about making the right choices
Social Web 2014 Lora Aroyo
User AdaptationKnowing the user to adapt a system or interface13
to improve the system functionality and user experience
A Jameson Adaptive interfaces and agents The HCI handbook fundamentals evolving technologies and emerging applications pp 305ndash330 2003
User-Adaptive Systems
user modeling
user profile
observations data and information about user
profile analysis
adaptation decisions
user modeling (infer current musical taste)
user profile interests in
genres artists tags
history of songs like ban pause skip
compare profile with possible next
songs to play
next song to be played
based on slides from Fabien Abel
Lastfm adapts to your music taste
bull Overfitting ldquobubble effectsrdquo loss of serendipity problem 13bull systems may adapt too strongly to the interestsbehavior13bull eg an adaptive radio station may always play the same or
very similar songs13bull We search for the right balance between novelty and
relevance for the user13
bull ldquoLost in Hyperspacerdquo problem 13bull when adapting the navigation ndash ie the links on which
users can click to findaccess information 13bull eg re-orderinghiding of menu items may lead to
confusion
Social Web 2014 Lora Aroyo
Issues in User-Adaptive Systems
Social Web 2014 Lora Aroyo
What is good user modelling amp personalisation
httpwwwflickrcomphotosbellarosebyliz4729613108
bull From the consumer perspective of an adaptive system 13
bull From the provider perspective of an adaptive system
Adaptive system maximizes satisfaction of the user
hard to measureobtain
Adaptive system maximizes the profit
influence of UM amp personalization may be hard to measureobtain
Social Web 2014 Lora Aroyo
Success Perspectives
bull User studies askobserve (selected) people whether you did a good job13
bull Log analysis Analyze (click) data and infer whether you did a good job13
bull Evaluation of user modeling13
bull measure quality of profiles directly eg measure overlap with existing (true) profiles or let people judge the quality of the generated user profiles 13
bull measure quality of application that exploits the user profile eg apply user modeling strategies in a recommender system
Social Web 2014 Lora Aroyo
Evaluation Strategies
time
item A
item B
item C
item D
item E
item G item H
item F
training data test data (ground truth)
Strategy X
User Modeling strategies to compare
Strategy Y
Strategy Z
Recommender
training data Recommendations
item H
X
item R
item M
Y
item H
item G
item N
item H
Z
item F
item M
measure quality
Social Web 2014 Lora Aroyo
Evaluating User Modeling in RecSys
Possible Metricsbull The usual IR metrics13
bull Precision fraction of retrieved items that are relevant13
bull Recall fraction of relevant items that have been retrieved13
bull F-Measure (harmonic) mean of precision and recall13
bull Metrics for evaluating recommendation (rankings)13
bull Mean Reciprocal Rank (MRR) of first relevant item13
bull Successk probability that relevant item occurs within the top k13
bull If a true ranking is given rank correlations 13
bull Precisionk Recallk amp F-Measurek13
bull Metrics for evaluating prediction of user preferences13
bull MAE = Mean Absolute Error13
bull TrueFalse PositivesNegatives runs
performance strategy X baseline
Is strategy X better than the baseline
Social Web 2014 Lora Aroyo
bull [Rae et al] a typical example of how to investigate and evaluate a proposal for improving (tag) recommendations (using social networks)13
bull Task test how well the different strategies (different tag contexts) can be used for tag predictionrecommendation13
bull Steps13
1 Gather a dataset of tag data part of which can be used as input and aim to test the recommendation on the remaining tag data13
2 Use the input data and calculate for the different strategies the predictions13
3 Measure the performance using standard (IR) metrics Precision of the top 5 recommended tags (P5) Mean Reciprocal Rank (MRR) Mean Average Precision (MAP)13
4 Test the results for statistical significance using T-test relative to the baseline (eg existing approach competitive approach)
[Rae et al Improving Tag Recommendations Using Social Networks RIAOrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
bull [Guy et al] another example of a similar evaluation approach13
bull The different strategies differ in the way people amp tags are used with tag-based systems there are complex relationships between users tags and items and strategies aim to find the relevant aspects of these relationships for modeling and recommendation13
bull The baseline is the lsquomost popularrsquo tags - often used to compare the most popular tags to the tags predicted by a particular personalization strategy - investigating whether the personalization is worth the effort and is able to outperform the easily available baseline
[Guy et al Social Media Recommendation based on People and Tags SIGIRrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
recommendation 13dimensions
Predict relevantusefulinteresting items13for a given user (in a given context)13
itrsquos often a ranking task
Social Web 2014 Lora Aroyo
Recommendation Systems
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
March 28 2013
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
commercial 13personalisation
httpwwwwiredcommagazine201111mf_artsyall1
Social Web 2014 Lora Aroyo
filter bubble
u1 likes u2
likes likes u1 likes Pulp Fiction
Social Web 2014 Lora Aroyo
Collaborative Filteringbull Memory-based User-Item matrix ratingspreferences of users =gt compute
similarity between users amp recommend items of similar users13
bull Model-based Item-Item matrix similarity (eg based on user ratings) between items =gt recommend items that are similar to the ones the user likes13
bull Model-based Clustering cluster users according to their preferences =gt recommend items of users that belong to the same cluster13
bull Model-based Bayesian networks P(u likes item B | u likes item A) = how likely is it that a user who likes item A will like item B learn probabilities from user ratingspreferences13
bull Others rule-based other data mining techniques13
bull complete input data is required13
bull pre-computation not possible13
bull does not scale well 13
bull high quality of recommendations13
bull abstraction (model) of input data13
bull pre-computation (partially) possible (model has to be re-built from time to time)13
bull scales better13
bull abstraction may reduce recommendation quality
Social Web 2014 Lora Aroyo
Memory vs Model-based
bull collaborative filtering lsquoneighborhoodsrsquo of people with similar interest amp recommending items based on likings in neighborhood13
bull limitations next to lsquocold startrsquo and lsquosparsityrsquo the lack of control (over onersquos neighborhood) is also a problem ie cannot add lsquotrustedrsquo people nor exclude lsquostrangersquo ones13
bull therefore interest in lsquosocial recommendersrsquo where presence of social connections defines the similarity in interests (eg social tagging CiteULike)13
bull does a social connection indicate user interest similarity13
bull how much users interest similarity depends on the strength of their connection13
bull is it feasible to use a social network as a personalized recommendation
[Lin amp Brusilovsky Social Networks and Interest Similarity The Case of CiteULike HTrsquo10]Social Web 2014 Lora Aroyo
Social Networks amp Interest Similarity
bull unilaterally connected pairs have more common itemsmetadatatags than non-connected pairs13
bull highest similarity for direct connections - decreasing with the increase of distance between users in SN 13
bull reciprocal relationship users - significantly larger similarity than users in a unidirectional relationship 13
bull traditional item-level similarity may be less reliable to find similar users in social bookmarking systems13
bull peers connected by self-defined social connections could be a useful source for cross-recommendation
Social Web 2014 Lora Aroyo
Conclusions
bull Input characteristics of items amp interests of a user into characteristics of items =gt Recommend items that feature characteristics which meet the userrsquos interests13
bull Techniques13
bull Data mining methods Cluster items based on their characteristics =gt Infer usersrsquo interests into clusters13
bull IR methods Represent items amp users as term vectors =gt Compute similarity between user profile vector and items13
bull Utility-based methods Utility function that gets an item as input the parameters of the utility function are customized via preferences of a user
Social Web 2014 Lora Aroyo
Content-based Recommendations
Government stops renovation of tower bridge Oct 13th 2011
Tower Bridge today Under construction
Tower Bridge is a combined bascule and suspension bridge in London England over the River Thames
Category politics england Related Twiper news bob Why do they stop tohellip [more] mary London stops renohellip [more]
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
02 0 0
02 04 01 01
= a
Weighting strategy - occurrence frequency - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
Content Features
RT Government stops renovation of tower bridge Oct 13th 2011
Userrsquos Twitter history
I am in London at the moment Oct 13th 2011
I am doing sports Oct 12th 2011
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
0 01 0
05 02 02 0
= u
Weighting strategy - occurrence frequency (eg smoothened by occurrence time recent concepts are more important - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
User Model
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
u 0
01 0
05 02 02 0
candidate items user a
02 0 0
02 04 01 01
b 0 0 0
08 02 0 0
c 0
05 02 0 0 0
03
cosine similarities
a b c
u 067 092 014
Ranking of recommended items 1 b 2 a 3 c
based on slides from Fabien Abel
Recommendations
Social Web 2014 Lora Aroyo
RecSys Issues
bull Cold-start problem (new user problem) nolittle data available to infer preferences of new users
bull Changing User Preferences user interests may change over time
bull Sparsity problem (new item problem) item descriptions are sparse eg not many user rated or tagged an item
bull Lack of Diversity (overfitting) when adapting too strongly to the preferences of users they might see samesimilar recommendations
bull Use the right context users do things which might not be relevant for their user model eg try out things do stuff for other people
bull Research challenge right balance between serendipity amp personalization
bull Research challenge right way to use the influence of recommendations on userrsquos behavior
one machine13vs 13
humans
image source httpwwwflickrcomphotosbionicteaching1375254387Social Web 2014 Lora Aroyo
Hands-on Teaser
bull Your Facebook Friendsrsquo popularity in a spread sheet13bull Locations of your Facebook Friends13bull Tag Cloud of your wall posts
User Modeling (4 building blocks)
Semantic Enrichment Linkage and Alignment
Personalized News Recommender
Profile
I want my
personalized news recommendations
based on slides from Fabien Abel
Can we infer a Twitter-based User Profile
User Modeling Building Blocks
Profile concept weight
time
1 Which tweets of the user should be
analyzed
Morning Afternoon Night
1 Temporal Constraints
June 27 July 4 July 11
(b) temporal patterns
weekends start end
(a) time period
based on slides from Fabien Abel
User Modeling Building Blocks
Profile concept weight
2 Profile Type
Francesca Schiavone won French Open fo2010
Francesca Schiavone
French Open
Francesca Schiavone French Open entity-based
Sport T
T topic-based
2 What type of concepts should represent ldquointerestsrdquo
fo2010
fo2010 hashtag-based
1 Temporal Constraints
time
June 27 July 4 July 11
based on slides from Fabien Abel
User Modeling Building Blocks
Profile concept weight
2 Profile Type
Francesca Schiavone won httpbitly2f4t7a
Francesca Schiavone
3 Further enrich the semantics of tweets
1 Temporal Constraints
3 Semantic Enrichment
Francesca Schiavone
Francesca wins French Open Thirty in womens tennis is primordially old an age when agility and desire recedes as the hellip
French Open
Tennis
French Open
Tennis
(b) further enrichment
(a) tweet-based
based on slides from Fabien Abel
User Modeling Building Blocks
Profile concept weight
2 Profile Type
4 How to weight the concepts
1 Temporal Constraints
3 Semantic Enrichment
Francesca Schiavone
French Open
Tennis
4 Weighting Scheme
time
June 27 July 4 July 11
weight(Francesca Schiavone)
Concept frequency (TF)
4
3 6
TFxIDF Time-sensitive
weight(French Open)
weight(Tennis)
based on slides from Fabien Abel
Social Web 2014 Lora Aroyo
Observationsbull Profile characteristics
bull Semantic enrichment solves sparsity problems13bull Profiles change over time recent profiles reflect better
current user demands13bull Temporal patterns weekend profiles differ significantly
from weekday profiles13
bull Impact on recommendations bull The more fine-grained the concepts the better the
recommendation performance entity-based gt topic-based gt hashtag-based 13
bull Semantic enrichment improves recommendation quality 13bull Time-sensitivity (adapting to trends) improves
performance
Social Web 2014 Lora Aroyo
User Modelingit is not about putting everything in a user profile 13
it is about making the right choices
Social Web 2014 Lora Aroyo
User AdaptationKnowing the user to adapt a system or interface13
to improve the system functionality and user experience
A Jameson Adaptive interfaces and agents The HCI handbook fundamentals evolving technologies and emerging applications pp 305ndash330 2003
User-Adaptive Systems
user modeling
user profile
observations data and information about user
profile analysis
adaptation decisions
user modeling (infer current musical taste)
user profile interests in
genres artists tags
history of songs like ban pause skip
compare profile with possible next
songs to play
next song to be played
based on slides from Fabien Abel
Lastfm adapts to your music taste
bull Overfitting ldquobubble effectsrdquo loss of serendipity problem 13bull systems may adapt too strongly to the interestsbehavior13bull eg an adaptive radio station may always play the same or
very similar songs13bull We search for the right balance between novelty and
relevance for the user13
bull ldquoLost in Hyperspacerdquo problem 13bull when adapting the navigation ndash ie the links on which
users can click to findaccess information 13bull eg re-orderinghiding of menu items may lead to
confusion
Social Web 2014 Lora Aroyo
Issues in User-Adaptive Systems
Social Web 2014 Lora Aroyo
What is good user modelling amp personalisation
httpwwwflickrcomphotosbellarosebyliz4729613108
bull From the consumer perspective of an adaptive system 13
bull From the provider perspective of an adaptive system
Adaptive system maximizes satisfaction of the user
hard to measureobtain
Adaptive system maximizes the profit
influence of UM amp personalization may be hard to measureobtain
Social Web 2014 Lora Aroyo
Success Perspectives
bull User studies askobserve (selected) people whether you did a good job13
bull Log analysis Analyze (click) data and infer whether you did a good job13
bull Evaluation of user modeling13
bull measure quality of profiles directly eg measure overlap with existing (true) profiles or let people judge the quality of the generated user profiles 13
bull measure quality of application that exploits the user profile eg apply user modeling strategies in a recommender system
Social Web 2014 Lora Aroyo
Evaluation Strategies
time
item A
item B
item C
item D
item E
item G item H
item F
training data test data (ground truth)
Strategy X
User Modeling strategies to compare
Strategy Y
Strategy Z
Recommender
training data Recommendations
item H
X
item R
item M
Y
item H
item G
item N
item H
Z
item F
item M
measure quality
Social Web 2014 Lora Aroyo
Evaluating User Modeling in RecSys
Possible Metricsbull The usual IR metrics13
bull Precision fraction of retrieved items that are relevant13
bull Recall fraction of relevant items that have been retrieved13
bull F-Measure (harmonic) mean of precision and recall13
bull Metrics for evaluating recommendation (rankings)13
bull Mean Reciprocal Rank (MRR) of first relevant item13
bull Successk probability that relevant item occurs within the top k13
bull If a true ranking is given rank correlations 13
bull Precisionk Recallk amp F-Measurek13
bull Metrics for evaluating prediction of user preferences13
bull MAE = Mean Absolute Error13
bull TrueFalse PositivesNegatives runs
performance strategy X baseline
Is strategy X better than the baseline
Social Web 2014 Lora Aroyo
bull [Rae et al] a typical example of how to investigate and evaluate a proposal for improving (tag) recommendations (using social networks)13
bull Task test how well the different strategies (different tag contexts) can be used for tag predictionrecommendation13
bull Steps13
1 Gather a dataset of tag data part of which can be used as input and aim to test the recommendation on the remaining tag data13
2 Use the input data and calculate for the different strategies the predictions13
3 Measure the performance using standard (IR) metrics Precision of the top 5 recommended tags (P5) Mean Reciprocal Rank (MRR) Mean Average Precision (MAP)13
4 Test the results for statistical significance using T-test relative to the baseline (eg existing approach competitive approach)
[Rae et al Improving Tag Recommendations Using Social Networks RIAOrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
bull [Guy et al] another example of a similar evaluation approach13
bull The different strategies differ in the way people amp tags are used with tag-based systems there are complex relationships between users tags and items and strategies aim to find the relevant aspects of these relationships for modeling and recommendation13
bull The baseline is the lsquomost popularrsquo tags - often used to compare the most popular tags to the tags predicted by a particular personalization strategy - investigating whether the personalization is worth the effort and is able to outperform the easily available baseline
[Guy et al Social Media Recommendation based on People and Tags SIGIRrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
recommendation 13dimensions
Predict relevantusefulinteresting items13for a given user (in a given context)13
itrsquos often a ranking task
Social Web 2014 Lora Aroyo
Recommendation Systems
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
March 28 2013
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
commercial 13personalisation
httpwwwwiredcommagazine201111mf_artsyall1
Social Web 2014 Lora Aroyo
filter bubble
u1 likes u2
likes likes u1 likes Pulp Fiction
Social Web 2014 Lora Aroyo
Collaborative Filteringbull Memory-based User-Item matrix ratingspreferences of users =gt compute
similarity between users amp recommend items of similar users13
bull Model-based Item-Item matrix similarity (eg based on user ratings) between items =gt recommend items that are similar to the ones the user likes13
bull Model-based Clustering cluster users according to their preferences =gt recommend items of users that belong to the same cluster13
bull Model-based Bayesian networks P(u likes item B | u likes item A) = how likely is it that a user who likes item A will like item B learn probabilities from user ratingspreferences13
bull Others rule-based other data mining techniques13
bull complete input data is required13
bull pre-computation not possible13
bull does not scale well 13
bull high quality of recommendations13
bull abstraction (model) of input data13
bull pre-computation (partially) possible (model has to be re-built from time to time)13
bull scales better13
bull abstraction may reduce recommendation quality
Social Web 2014 Lora Aroyo
Memory vs Model-based
bull collaborative filtering lsquoneighborhoodsrsquo of people with similar interest amp recommending items based on likings in neighborhood13
bull limitations next to lsquocold startrsquo and lsquosparsityrsquo the lack of control (over onersquos neighborhood) is also a problem ie cannot add lsquotrustedrsquo people nor exclude lsquostrangersquo ones13
bull therefore interest in lsquosocial recommendersrsquo where presence of social connections defines the similarity in interests (eg social tagging CiteULike)13
bull does a social connection indicate user interest similarity13
bull how much users interest similarity depends on the strength of their connection13
bull is it feasible to use a social network as a personalized recommendation
[Lin amp Brusilovsky Social Networks and Interest Similarity The Case of CiteULike HTrsquo10]Social Web 2014 Lora Aroyo
Social Networks amp Interest Similarity
bull unilaterally connected pairs have more common itemsmetadatatags than non-connected pairs13
bull highest similarity for direct connections - decreasing with the increase of distance between users in SN 13
bull reciprocal relationship users - significantly larger similarity than users in a unidirectional relationship 13
bull traditional item-level similarity may be less reliable to find similar users in social bookmarking systems13
bull peers connected by self-defined social connections could be a useful source for cross-recommendation
Social Web 2014 Lora Aroyo
Conclusions
bull Input characteristics of items amp interests of a user into characteristics of items =gt Recommend items that feature characteristics which meet the userrsquos interests13
bull Techniques13
bull Data mining methods Cluster items based on their characteristics =gt Infer usersrsquo interests into clusters13
bull IR methods Represent items amp users as term vectors =gt Compute similarity between user profile vector and items13
bull Utility-based methods Utility function that gets an item as input the parameters of the utility function are customized via preferences of a user
Social Web 2014 Lora Aroyo
Content-based Recommendations
Government stops renovation of tower bridge Oct 13th 2011
Tower Bridge today Under construction
Tower Bridge is a combined bascule and suspension bridge in London England over the River Thames
Category politics england Related Twiper news bob Why do they stop tohellip [more] mary London stops renohellip [more]
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
02 0 0
02 04 01 01
= a
Weighting strategy - occurrence frequency - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
Content Features
RT Government stops renovation of tower bridge Oct 13th 2011
Userrsquos Twitter history
I am in London at the moment Oct 13th 2011
I am doing sports Oct 12th 2011
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
0 01 0
05 02 02 0
= u
Weighting strategy - occurrence frequency (eg smoothened by occurrence time recent concepts are more important - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
User Model
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
u 0
01 0
05 02 02 0
candidate items user a
02 0 0
02 04 01 01
b 0 0 0
08 02 0 0
c 0
05 02 0 0 0
03
cosine similarities
a b c
u 067 092 014
Ranking of recommended items 1 b 2 a 3 c
based on slides from Fabien Abel
Recommendations
Social Web 2014 Lora Aroyo
RecSys Issues
bull Cold-start problem (new user problem) nolittle data available to infer preferences of new users
bull Changing User Preferences user interests may change over time
bull Sparsity problem (new item problem) item descriptions are sparse eg not many user rated or tagged an item
bull Lack of Diversity (overfitting) when adapting too strongly to the preferences of users they might see samesimilar recommendations
bull Use the right context users do things which might not be relevant for their user model eg try out things do stuff for other people
bull Research challenge right balance between serendipity amp personalization
bull Research challenge right way to use the influence of recommendations on userrsquos behavior
one machine13vs 13
humans
image source httpwwwflickrcomphotosbionicteaching1375254387Social Web 2014 Lora Aroyo
Hands-on Teaser
bull Your Facebook Friendsrsquo popularity in a spread sheet13bull Locations of your Facebook Friends13bull Tag Cloud of your wall posts
User Modeling Building Blocks
Profile concept weight
time
1 Which tweets of the user should be
analyzed
Morning Afternoon Night
1 Temporal Constraints
June 27 July 4 July 11
(b) temporal patterns
weekends start end
(a) time period
based on slides from Fabien Abel
User Modeling Building Blocks
Profile concept weight
2 Profile Type
Francesca Schiavone won French Open fo2010
Francesca Schiavone
French Open
Francesca Schiavone French Open entity-based
Sport T
T topic-based
2 What type of concepts should represent ldquointerestsrdquo
fo2010
fo2010 hashtag-based
1 Temporal Constraints
time
June 27 July 4 July 11
based on slides from Fabien Abel
User Modeling Building Blocks
Profile concept weight
2 Profile Type
Francesca Schiavone won httpbitly2f4t7a
Francesca Schiavone
3 Further enrich the semantics of tweets
1 Temporal Constraints
3 Semantic Enrichment
Francesca Schiavone
Francesca wins French Open Thirty in womens tennis is primordially old an age when agility and desire recedes as the hellip
French Open
Tennis
French Open
Tennis
(b) further enrichment
(a) tweet-based
based on slides from Fabien Abel
User Modeling Building Blocks
Profile concept weight
2 Profile Type
4 How to weight the concepts
1 Temporal Constraints
3 Semantic Enrichment
Francesca Schiavone
French Open
Tennis
4 Weighting Scheme
time
June 27 July 4 July 11
weight(Francesca Schiavone)
Concept frequency (TF)
4
3 6
TFxIDF Time-sensitive
weight(French Open)
weight(Tennis)
based on slides from Fabien Abel
Social Web 2014 Lora Aroyo
Observationsbull Profile characteristics
bull Semantic enrichment solves sparsity problems13bull Profiles change over time recent profiles reflect better
current user demands13bull Temporal patterns weekend profiles differ significantly
from weekday profiles13
bull Impact on recommendations bull The more fine-grained the concepts the better the
recommendation performance entity-based gt topic-based gt hashtag-based 13
bull Semantic enrichment improves recommendation quality 13bull Time-sensitivity (adapting to trends) improves
performance
Social Web 2014 Lora Aroyo
User Modelingit is not about putting everything in a user profile 13
it is about making the right choices
Social Web 2014 Lora Aroyo
User AdaptationKnowing the user to adapt a system or interface13
to improve the system functionality and user experience
A Jameson Adaptive interfaces and agents The HCI handbook fundamentals evolving technologies and emerging applications pp 305ndash330 2003
User-Adaptive Systems
user modeling
user profile
observations data and information about user
profile analysis
adaptation decisions
user modeling (infer current musical taste)
user profile interests in
genres artists tags
history of songs like ban pause skip
compare profile with possible next
songs to play
next song to be played
based on slides from Fabien Abel
Lastfm adapts to your music taste
bull Overfitting ldquobubble effectsrdquo loss of serendipity problem 13bull systems may adapt too strongly to the interestsbehavior13bull eg an adaptive radio station may always play the same or
very similar songs13bull We search for the right balance between novelty and
relevance for the user13
bull ldquoLost in Hyperspacerdquo problem 13bull when adapting the navigation ndash ie the links on which
users can click to findaccess information 13bull eg re-orderinghiding of menu items may lead to
confusion
Social Web 2014 Lora Aroyo
Issues in User-Adaptive Systems
Social Web 2014 Lora Aroyo
What is good user modelling amp personalisation
httpwwwflickrcomphotosbellarosebyliz4729613108
bull From the consumer perspective of an adaptive system 13
bull From the provider perspective of an adaptive system
Adaptive system maximizes satisfaction of the user
hard to measureobtain
Adaptive system maximizes the profit
influence of UM amp personalization may be hard to measureobtain
Social Web 2014 Lora Aroyo
Success Perspectives
bull User studies askobserve (selected) people whether you did a good job13
bull Log analysis Analyze (click) data and infer whether you did a good job13
bull Evaluation of user modeling13
bull measure quality of profiles directly eg measure overlap with existing (true) profiles or let people judge the quality of the generated user profiles 13
bull measure quality of application that exploits the user profile eg apply user modeling strategies in a recommender system
Social Web 2014 Lora Aroyo
Evaluation Strategies
time
item A
item B
item C
item D
item E
item G item H
item F
training data test data (ground truth)
Strategy X
User Modeling strategies to compare
Strategy Y
Strategy Z
Recommender
training data Recommendations
item H
X
item R
item M
Y
item H
item G
item N
item H
Z
item F
item M
measure quality
Social Web 2014 Lora Aroyo
Evaluating User Modeling in RecSys
Possible Metricsbull The usual IR metrics13
bull Precision fraction of retrieved items that are relevant13
bull Recall fraction of relevant items that have been retrieved13
bull F-Measure (harmonic) mean of precision and recall13
bull Metrics for evaluating recommendation (rankings)13
bull Mean Reciprocal Rank (MRR) of first relevant item13
bull Successk probability that relevant item occurs within the top k13
bull If a true ranking is given rank correlations 13
bull Precisionk Recallk amp F-Measurek13
bull Metrics for evaluating prediction of user preferences13
bull MAE = Mean Absolute Error13
bull TrueFalse PositivesNegatives runs
performance strategy X baseline
Is strategy X better than the baseline
Social Web 2014 Lora Aroyo
bull [Rae et al] a typical example of how to investigate and evaluate a proposal for improving (tag) recommendations (using social networks)13
bull Task test how well the different strategies (different tag contexts) can be used for tag predictionrecommendation13
bull Steps13
1 Gather a dataset of tag data part of which can be used as input and aim to test the recommendation on the remaining tag data13
2 Use the input data and calculate for the different strategies the predictions13
3 Measure the performance using standard (IR) metrics Precision of the top 5 recommended tags (P5) Mean Reciprocal Rank (MRR) Mean Average Precision (MAP)13
4 Test the results for statistical significance using T-test relative to the baseline (eg existing approach competitive approach)
[Rae et al Improving Tag Recommendations Using Social Networks RIAOrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
bull [Guy et al] another example of a similar evaluation approach13
bull The different strategies differ in the way people amp tags are used with tag-based systems there are complex relationships between users tags and items and strategies aim to find the relevant aspects of these relationships for modeling and recommendation13
bull The baseline is the lsquomost popularrsquo tags - often used to compare the most popular tags to the tags predicted by a particular personalization strategy - investigating whether the personalization is worth the effort and is able to outperform the easily available baseline
[Guy et al Social Media Recommendation based on People and Tags SIGIRrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
recommendation 13dimensions
Predict relevantusefulinteresting items13for a given user (in a given context)13
itrsquos often a ranking task
Social Web 2014 Lora Aroyo
Recommendation Systems
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
March 28 2013
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
commercial 13personalisation
httpwwwwiredcommagazine201111mf_artsyall1
Social Web 2014 Lora Aroyo
filter bubble
u1 likes u2
likes likes u1 likes Pulp Fiction
Social Web 2014 Lora Aroyo
Collaborative Filteringbull Memory-based User-Item matrix ratingspreferences of users =gt compute
similarity between users amp recommend items of similar users13
bull Model-based Item-Item matrix similarity (eg based on user ratings) between items =gt recommend items that are similar to the ones the user likes13
bull Model-based Clustering cluster users according to their preferences =gt recommend items of users that belong to the same cluster13
bull Model-based Bayesian networks P(u likes item B | u likes item A) = how likely is it that a user who likes item A will like item B learn probabilities from user ratingspreferences13
bull Others rule-based other data mining techniques13
bull complete input data is required13
bull pre-computation not possible13
bull does not scale well 13
bull high quality of recommendations13
bull abstraction (model) of input data13
bull pre-computation (partially) possible (model has to be re-built from time to time)13
bull scales better13
bull abstraction may reduce recommendation quality
Social Web 2014 Lora Aroyo
Memory vs Model-based
bull collaborative filtering lsquoneighborhoodsrsquo of people with similar interest amp recommending items based on likings in neighborhood13
bull limitations next to lsquocold startrsquo and lsquosparsityrsquo the lack of control (over onersquos neighborhood) is also a problem ie cannot add lsquotrustedrsquo people nor exclude lsquostrangersquo ones13
bull therefore interest in lsquosocial recommendersrsquo where presence of social connections defines the similarity in interests (eg social tagging CiteULike)13
bull does a social connection indicate user interest similarity13
bull how much users interest similarity depends on the strength of their connection13
bull is it feasible to use a social network as a personalized recommendation
[Lin amp Brusilovsky Social Networks and Interest Similarity The Case of CiteULike HTrsquo10]Social Web 2014 Lora Aroyo
Social Networks amp Interest Similarity
bull unilaterally connected pairs have more common itemsmetadatatags than non-connected pairs13
bull highest similarity for direct connections - decreasing with the increase of distance between users in SN 13
bull reciprocal relationship users - significantly larger similarity than users in a unidirectional relationship 13
bull traditional item-level similarity may be less reliable to find similar users in social bookmarking systems13
bull peers connected by self-defined social connections could be a useful source for cross-recommendation
Social Web 2014 Lora Aroyo
Conclusions
bull Input characteristics of items amp interests of a user into characteristics of items =gt Recommend items that feature characteristics which meet the userrsquos interests13
bull Techniques13
bull Data mining methods Cluster items based on their characteristics =gt Infer usersrsquo interests into clusters13
bull IR methods Represent items amp users as term vectors =gt Compute similarity between user profile vector and items13
bull Utility-based methods Utility function that gets an item as input the parameters of the utility function are customized via preferences of a user
Social Web 2014 Lora Aroyo
Content-based Recommendations
Government stops renovation of tower bridge Oct 13th 2011
Tower Bridge today Under construction
Tower Bridge is a combined bascule and suspension bridge in London England over the River Thames
Category politics england Related Twiper news bob Why do they stop tohellip [more] mary London stops renohellip [more]
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
02 0 0
02 04 01 01
= a
Weighting strategy - occurrence frequency - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
Content Features
RT Government stops renovation of tower bridge Oct 13th 2011
Userrsquos Twitter history
I am in London at the moment Oct 13th 2011
I am doing sports Oct 12th 2011
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
0 01 0
05 02 02 0
= u
Weighting strategy - occurrence frequency (eg smoothened by occurrence time recent concepts are more important - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
User Model
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
u 0
01 0
05 02 02 0
candidate items user a
02 0 0
02 04 01 01
b 0 0 0
08 02 0 0
c 0
05 02 0 0 0
03
cosine similarities
a b c
u 067 092 014
Ranking of recommended items 1 b 2 a 3 c
based on slides from Fabien Abel
Recommendations
Social Web 2014 Lora Aroyo
RecSys Issues
bull Cold-start problem (new user problem) nolittle data available to infer preferences of new users
bull Changing User Preferences user interests may change over time
bull Sparsity problem (new item problem) item descriptions are sparse eg not many user rated or tagged an item
bull Lack of Diversity (overfitting) when adapting too strongly to the preferences of users they might see samesimilar recommendations
bull Use the right context users do things which might not be relevant for their user model eg try out things do stuff for other people
bull Research challenge right balance between serendipity amp personalization
bull Research challenge right way to use the influence of recommendations on userrsquos behavior
one machine13vs 13
humans
image source httpwwwflickrcomphotosbionicteaching1375254387Social Web 2014 Lora Aroyo
Hands-on Teaser
bull Your Facebook Friendsrsquo popularity in a spread sheet13bull Locations of your Facebook Friends13bull Tag Cloud of your wall posts
User Modeling Building Blocks
Profile concept weight
2 Profile Type
Francesca Schiavone won French Open fo2010
Francesca Schiavone
French Open
Francesca Schiavone French Open entity-based
Sport T
T topic-based
2 What type of concepts should represent ldquointerestsrdquo
fo2010
fo2010 hashtag-based
1 Temporal Constraints
time
June 27 July 4 July 11
based on slides from Fabien Abel
User Modeling Building Blocks
Profile concept weight
2 Profile Type
Francesca Schiavone won httpbitly2f4t7a
Francesca Schiavone
3 Further enrich the semantics of tweets
1 Temporal Constraints
3 Semantic Enrichment
Francesca Schiavone
Francesca wins French Open Thirty in womens tennis is primordially old an age when agility and desire recedes as the hellip
French Open
Tennis
French Open
Tennis
(b) further enrichment
(a) tweet-based
based on slides from Fabien Abel
User Modeling Building Blocks
Profile concept weight
2 Profile Type
4 How to weight the concepts
1 Temporal Constraints
3 Semantic Enrichment
Francesca Schiavone
French Open
Tennis
4 Weighting Scheme
time
June 27 July 4 July 11
weight(Francesca Schiavone)
Concept frequency (TF)
4
3 6
TFxIDF Time-sensitive
weight(French Open)
weight(Tennis)
based on slides from Fabien Abel
Social Web 2014 Lora Aroyo
Observationsbull Profile characteristics
bull Semantic enrichment solves sparsity problems13bull Profiles change over time recent profiles reflect better
current user demands13bull Temporal patterns weekend profiles differ significantly
from weekday profiles13
bull Impact on recommendations bull The more fine-grained the concepts the better the
recommendation performance entity-based gt topic-based gt hashtag-based 13
bull Semantic enrichment improves recommendation quality 13bull Time-sensitivity (adapting to trends) improves
performance
Social Web 2014 Lora Aroyo
User Modelingit is not about putting everything in a user profile 13
it is about making the right choices
Social Web 2014 Lora Aroyo
User AdaptationKnowing the user to adapt a system or interface13
to improve the system functionality and user experience
A Jameson Adaptive interfaces and agents The HCI handbook fundamentals evolving technologies and emerging applications pp 305ndash330 2003
User-Adaptive Systems
user modeling
user profile
observations data and information about user
profile analysis
adaptation decisions
user modeling (infer current musical taste)
user profile interests in
genres artists tags
history of songs like ban pause skip
compare profile with possible next
songs to play
next song to be played
based on slides from Fabien Abel
Lastfm adapts to your music taste
bull Overfitting ldquobubble effectsrdquo loss of serendipity problem 13bull systems may adapt too strongly to the interestsbehavior13bull eg an adaptive radio station may always play the same or
very similar songs13bull We search for the right balance between novelty and
relevance for the user13
bull ldquoLost in Hyperspacerdquo problem 13bull when adapting the navigation ndash ie the links on which
users can click to findaccess information 13bull eg re-orderinghiding of menu items may lead to
confusion
Social Web 2014 Lora Aroyo
Issues in User-Adaptive Systems
Social Web 2014 Lora Aroyo
What is good user modelling amp personalisation
httpwwwflickrcomphotosbellarosebyliz4729613108
bull From the consumer perspective of an adaptive system 13
bull From the provider perspective of an adaptive system
Adaptive system maximizes satisfaction of the user
hard to measureobtain
Adaptive system maximizes the profit
influence of UM amp personalization may be hard to measureobtain
Social Web 2014 Lora Aroyo
Success Perspectives
bull User studies askobserve (selected) people whether you did a good job13
bull Log analysis Analyze (click) data and infer whether you did a good job13
bull Evaluation of user modeling13
bull measure quality of profiles directly eg measure overlap with existing (true) profiles or let people judge the quality of the generated user profiles 13
bull measure quality of application that exploits the user profile eg apply user modeling strategies in a recommender system
Social Web 2014 Lora Aroyo
Evaluation Strategies
time
item A
item B
item C
item D
item E
item G item H
item F
training data test data (ground truth)
Strategy X
User Modeling strategies to compare
Strategy Y
Strategy Z
Recommender
training data Recommendations
item H
X
item R
item M
Y
item H
item G
item N
item H
Z
item F
item M
measure quality
Social Web 2014 Lora Aroyo
Evaluating User Modeling in RecSys
Possible Metricsbull The usual IR metrics13
bull Precision fraction of retrieved items that are relevant13
bull Recall fraction of relevant items that have been retrieved13
bull F-Measure (harmonic) mean of precision and recall13
bull Metrics for evaluating recommendation (rankings)13
bull Mean Reciprocal Rank (MRR) of first relevant item13
bull Successk probability that relevant item occurs within the top k13
bull If a true ranking is given rank correlations 13
bull Precisionk Recallk amp F-Measurek13
bull Metrics for evaluating prediction of user preferences13
bull MAE = Mean Absolute Error13
bull TrueFalse PositivesNegatives runs
performance strategy X baseline
Is strategy X better than the baseline
Social Web 2014 Lora Aroyo
bull [Rae et al] a typical example of how to investigate and evaluate a proposal for improving (tag) recommendations (using social networks)13
bull Task test how well the different strategies (different tag contexts) can be used for tag predictionrecommendation13
bull Steps13
1 Gather a dataset of tag data part of which can be used as input and aim to test the recommendation on the remaining tag data13
2 Use the input data and calculate for the different strategies the predictions13
3 Measure the performance using standard (IR) metrics Precision of the top 5 recommended tags (P5) Mean Reciprocal Rank (MRR) Mean Average Precision (MAP)13
4 Test the results for statistical significance using T-test relative to the baseline (eg existing approach competitive approach)
[Rae et al Improving Tag Recommendations Using Social Networks RIAOrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
bull [Guy et al] another example of a similar evaluation approach13
bull The different strategies differ in the way people amp tags are used with tag-based systems there are complex relationships between users tags and items and strategies aim to find the relevant aspects of these relationships for modeling and recommendation13
bull The baseline is the lsquomost popularrsquo tags - often used to compare the most popular tags to the tags predicted by a particular personalization strategy - investigating whether the personalization is worth the effort and is able to outperform the easily available baseline
[Guy et al Social Media Recommendation based on People and Tags SIGIRrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
recommendation 13dimensions
Predict relevantusefulinteresting items13for a given user (in a given context)13
itrsquos often a ranking task
Social Web 2014 Lora Aroyo
Recommendation Systems
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
March 28 2013
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
commercial 13personalisation
httpwwwwiredcommagazine201111mf_artsyall1
Social Web 2014 Lora Aroyo
filter bubble
u1 likes u2
likes likes u1 likes Pulp Fiction
Social Web 2014 Lora Aroyo
Collaborative Filteringbull Memory-based User-Item matrix ratingspreferences of users =gt compute
similarity between users amp recommend items of similar users13
bull Model-based Item-Item matrix similarity (eg based on user ratings) between items =gt recommend items that are similar to the ones the user likes13
bull Model-based Clustering cluster users according to their preferences =gt recommend items of users that belong to the same cluster13
bull Model-based Bayesian networks P(u likes item B | u likes item A) = how likely is it that a user who likes item A will like item B learn probabilities from user ratingspreferences13
bull Others rule-based other data mining techniques13
bull complete input data is required13
bull pre-computation not possible13
bull does not scale well 13
bull high quality of recommendations13
bull abstraction (model) of input data13
bull pre-computation (partially) possible (model has to be re-built from time to time)13
bull scales better13
bull abstraction may reduce recommendation quality
Social Web 2014 Lora Aroyo
Memory vs Model-based
bull collaborative filtering lsquoneighborhoodsrsquo of people with similar interest amp recommending items based on likings in neighborhood13
bull limitations next to lsquocold startrsquo and lsquosparsityrsquo the lack of control (over onersquos neighborhood) is also a problem ie cannot add lsquotrustedrsquo people nor exclude lsquostrangersquo ones13
bull therefore interest in lsquosocial recommendersrsquo where presence of social connections defines the similarity in interests (eg social tagging CiteULike)13
bull does a social connection indicate user interest similarity13
bull how much users interest similarity depends on the strength of their connection13
bull is it feasible to use a social network as a personalized recommendation
[Lin amp Brusilovsky Social Networks and Interest Similarity The Case of CiteULike HTrsquo10]Social Web 2014 Lora Aroyo
Social Networks amp Interest Similarity
bull unilaterally connected pairs have more common itemsmetadatatags than non-connected pairs13
bull highest similarity for direct connections - decreasing with the increase of distance between users in SN 13
bull reciprocal relationship users - significantly larger similarity than users in a unidirectional relationship 13
bull traditional item-level similarity may be less reliable to find similar users in social bookmarking systems13
bull peers connected by self-defined social connections could be a useful source for cross-recommendation
Social Web 2014 Lora Aroyo
Conclusions
bull Input characteristics of items amp interests of a user into characteristics of items =gt Recommend items that feature characteristics which meet the userrsquos interests13
bull Techniques13
bull Data mining methods Cluster items based on their characteristics =gt Infer usersrsquo interests into clusters13
bull IR methods Represent items amp users as term vectors =gt Compute similarity between user profile vector and items13
bull Utility-based methods Utility function that gets an item as input the parameters of the utility function are customized via preferences of a user
Social Web 2014 Lora Aroyo
Content-based Recommendations
Government stops renovation of tower bridge Oct 13th 2011
Tower Bridge today Under construction
Tower Bridge is a combined bascule and suspension bridge in London England over the River Thames
Category politics england Related Twiper news bob Why do they stop tohellip [more] mary London stops renohellip [more]
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
02 0 0
02 04 01 01
= a
Weighting strategy - occurrence frequency - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
Content Features
RT Government stops renovation of tower bridge Oct 13th 2011
Userrsquos Twitter history
I am in London at the moment Oct 13th 2011
I am doing sports Oct 12th 2011
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
0 01 0
05 02 02 0
= u
Weighting strategy - occurrence frequency (eg smoothened by occurrence time recent concepts are more important - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
User Model
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
u 0
01 0
05 02 02 0
candidate items user a
02 0 0
02 04 01 01
b 0 0 0
08 02 0 0
c 0
05 02 0 0 0
03
cosine similarities
a b c
u 067 092 014
Ranking of recommended items 1 b 2 a 3 c
based on slides from Fabien Abel
Recommendations
Social Web 2014 Lora Aroyo
RecSys Issues
bull Cold-start problem (new user problem) nolittle data available to infer preferences of new users
bull Changing User Preferences user interests may change over time
bull Sparsity problem (new item problem) item descriptions are sparse eg not many user rated or tagged an item
bull Lack of Diversity (overfitting) when adapting too strongly to the preferences of users they might see samesimilar recommendations
bull Use the right context users do things which might not be relevant for their user model eg try out things do stuff for other people
bull Research challenge right balance between serendipity amp personalization
bull Research challenge right way to use the influence of recommendations on userrsquos behavior
one machine13vs 13
humans
image source httpwwwflickrcomphotosbionicteaching1375254387Social Web 2014 Lora Aroyo
Hands-on Teaser
bull Your Facebook Friendsrsquo popularity in a spread sheet13bull Locations of your Facebook Friends13bull Tag Cloud of your wall posts
User Modeling Building Blocks
Profile concept weight
2 Profile Type
Francesca Schiavone won httpbitly2f4t7a
Francesca Schiavone
3 Further enrich the semantics of tweets
1 Temporal Constraints
3 Semantic Enrichment
Francesca Schiavone
Francesca wins French Open Thirty in womens tennis is primordially old an age when agility and desire recedes as the hellip
French Open
Tennis
French Open
Tennis
(b) further enrichment
(a) tweet-based
based on slides from Fabien Abel
User Modeling Building Blocks
Profile concept weight
2 Profile Type
4 How to weight the concepts
1 Temporal Constraints
3 Semantic Enrichment
Francesca Schiavone
French Open
Tennis
4 Weighting Scheme
time
June 27 July 4 July 11
weight(Francesca Schiavone)
Concept frequency (TF)
4
3 6
TFxIDF Time-sensitive
weight(French Open)
weight(Tennis)
based on slides from Fabien Abel
Social Web 2014 Lora Aroyo
Observationsbull Profile characteristics
bull Semantic enrichment solves sparsity problems13bull Profiles change over time recent profiles reflect better
current user demands13bull Temporal patterns weekend profiles differ significantly
from weekday profiles13
bull Impact on recommendations bull The more fine-grained the concepts the better the
recommendation performance entity-based gt topic-based gt hashtag-based 13
bull Semantic enrichment improves recommendation quality 13bull Time-sensitivity (adapting to trends) improves
performance
Social Web 2014 Lora Aroyo
User Modelingit is not about putting everything in a user profile 13
it is about making the right choices
Social Web 2014 Lora Aroyo
User AdaptationKnowing the user to adapt a system or interface13
to improve the system functionality and user experience
A Jameson Adaptive interfaces and agents The HCI handbook fundamentals evolving technologies and emerging applications pp 305ndash330 2003
User-Adaptive Systems
user modeling
user profile
observations data and information about user
profile analysis
adaptation decisions
user modeling (infer current musical taste)
user profile interests in
genres artists tags
history of songs like ban pause skip
compare profile with possible next
songs to play
next song to be played
based on slides from Fabien Abel
Lastfm adapts to your music taste
bull Overfitting ldquobubble effectsrdquo loss of serendipity problem 13bull systems may adapt too strongly to the interestsbehavior13bull eg an adaptive radio station may always play the same or
very similar songs13bull We search for the right balance between novelty and
relevance for the user13
bull ldquoLost in Hyperspacerdquo problem 13bull when adapting the navigation ndash ie the links on which
users can click to findaccess information 13bull eg re-orderinghiding of menu items may lead to
confusion
Social Web 2014 Lora Aroyo
Issues in User-Adaptive Systems
Social Web 2014 Lora Aroyo
What is good user modelling amp personalisation
httpwwwflickrcomphotosbellarosebyliz4729613108
bull From the consumer perspective of an adaptive system 13
bull From the provider perspective of an adaptive system
Adaptive system maximizes satisfaction of the user
hard to measureobtain
Adaptive system maximizes the profit
influence of UM amp personalization may be hard to measureobtain
Social Web 2014 Lora Aroyo
Success Perspectives
bull User studies askobserve (selected) people whether you did a good job13
bull Log analysis Analyze (click) data and infer whether you did a good job13
bull Evaluation of user modeling13
bull measure quality of profiles directly eg measure overlap with existing (true) profiles or let people judge the quality of the generated user profiles 13
bull measure quality of application that exploits the user profile eg apply user modeling strategies in a recommender system
Social Web 2014 Lora Aroyo
Evaluation Strategies
time
item A
item B
item C
item D
item E
item G item H
item F
training data test data (ground truth)
Strategy X
User Modeling strategies to compare
Strategy Y
Strategy Z
Recommender
training data Recommendations
item H
X
item R
item M
Y
item H
item G
item N
item H
Z
item F
item M
measure quality
Social Web 2014 Lora Aroyo
Evaluating User Modeling in RecSys
Possible Metricsbull The usual IR metrics13
bull Precision fraction of retrieved items that are relevant13
bull Recall fraction of relevant items that have been retrieved13
bull F-Measure (harmonic) mean of precision and recall13
bull Metrics for evaluating recommendation (rankings)13
bull Mean Reciprocal Rank (MRR) of first relevant item13
bull Successk probability that relevant item occurs within the top k13
bull If a true ranking is given rank correlations 13
bull Precisionk Recallk amp F-Measurek13
bull Metrics for evaluating prediction of user preferences13
bull MAE = Mean Absolute Error13
bull TrueFalse PositivesNegatives runs
performance strategy X baseline
Is strategy X better than the baseline
Social Web 2014 Lora Aroyo
bull [Rae et al] a typical example of how to investigate and evaluate a proposal for improving (tag) recommendations (using social networks)13
bull Task test how well the different strategies (different tag contexts) can be used for tag predictionrecommendation13
bull Steps13
1 Gather a dataset of tag data part of which can be used as input and aim to test the recommendation on the remaining tag data13
2 Use the input data and calculate for the different strategies the predictions13
3 Measure the performance using standard (IR) metrics Precision of the top 5 recommended tags (P5) Mean Reciprocal Rank (MRR) Mean Average Precision (MAP)13
4 Test the results for statistical significance using T-test relative to the baseline (eg existing approach competitive approach)
[Rae et al Improving Tag Recommendations Using Social Networks RIAOrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
bull [Guy et al] another example of a similar evaluation approach13
bull The different strategies differ in the way people amp tags are used with tag-based systems there are complex relationships between users tags and items and strategies aim to find the relevant aspects of these relationships for modeling and recommendation13
bull The baseline is the lsquomost popularrsquo tags - often used to compare the most popular tags to the tags predicted by a particular personalization strategy - investigating whether the personalization is worth the effort and is able to outperform the easily available baseline
[Guy et al Social Media Recommendation based on People and Tags SIGIRrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
recommendation 13dimensions
Predict relevantusefulinteresting items13for a given user (in a given context)13
itrsquos often a ranking task
Social Web 2014 Lora Aroyo
Recommendation Systems
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
March 28 2013
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
commercial 13personalisation
httpwwwwiredcommagazine201111mf_artsyall1
Social Web 2014 Lora Aroyo
filter bubble
u1 likes u2
likes likes u1 likes Pulp Fiction
Social Web 2014 Lora Aroyo
Collaborative Filteringbull Memory-based User-Item matrix ratingspreferences of users =gt compute
similarity between users amp recommend items of similar users13
bull Model-based Item-Item matrix similarity (eg based on user ratings) between items =gt recommend items that are similar to the ones the user likes13
bull Model-based Clustering cluster users according to their preferences =gt recommend items of users that belong to the same cluster13
bull Model-based Bayesian networks P(u likes item B | u likes item A) = how likely is it that a user who likes item A will like item B learn probabilities from user ratingspreferences13
bull Others rule-based other data mining techniques13
bull complete input data is required13
bull pre-computation not possible13
bull does not scale well 13
bull high quality of recommendations13
bull abstraction (model) of input data13
bull pre-computation (partially) possible (model has to be re-built from time to time)13
bull scales better13
bull abstraction may reduce recommendation quality
Social Web 2014 Lora Aroyo
Memory vs Model-based
bull collaborative filtering lsquoneighborhoodsrsquo of people with similar interest amp recommending items based on likings in neighborhood13
bull limitations next to lsquocold startrsquo and lsquosparsityrsquo the lack of control (over onersquos neighborhood) is also a problem ie cannot add lsquotrustedrsquo people nor exclude lsquostrangersquo ones13
bull therefore interest in lsquosocial recommendersrsquo where presence of social connections defines the similarity in interests (eg social tagging CiteULike)13
bull does a social connection indicate user interest similarity13
bull how much users interest similarity depends on the strength of their connection13
bull is it feasible to use a social network as a personalized recommendation
[Lin amp Brusilovsky Social Networks and Interest Similarity The Case of CiteULike HTrsquo10]Social Web 2014 Lora Aroyo
Social Networks amp Interest Similarity
bull unilaterally connected pairs have more common itemsmetadatatags than non-connected pairs13
bull highest similarity for direct connections - decreasing with the increase of distance between users in SN 13
bull reciprocal relationship users - significantly larger similarity than users in a unidirectional relationship 13
bull traditional item-level similarity may be less reliable to find similar users in social bookmarking systems13
bull peers connected by self-defined social connections could be a useful source for cross-recommendation
Social Web 2014 Lora Aroyo
Conclusions
bull Input characteristics of items amp interests of a user into characteristics of items =gt Recommend items that feature characteristics which meet the userrsquos interests13
bull Techniques13
bull Data mining methods Cluster items based on their characteristics =gt Infer usersrsquo interests into clusters13
bull IR methods Represent items amp users as term vectors =gt Compute similarity between user profile vector and items13
bull Utility-based methods Utility function that gets an item as input the parameters of the utility function are customized via preferences of a user
Social Web 2014 Lora Aroyo
Content-based Recommendations
Government stops renovation of tower bridge Oct 13th 2011
Tower Bridge today Under construction
Tower Bridge is a combined bascule and suspension bridge in London England over the River Thames
Category politics england Related Twiper news bob Why do they stop tohellip [more] mary London stops renohellip [more]
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
02 0 0
02 04 01 01
= a
Weighting strategy - occurrence frequency - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
Content Features
RT Government stops renovation of tower bridge Oct 13th 2011
Userrsquos Twitter history
I am in London at the moment Oct 13th 2011
I am doing sports Oct 12th 2011
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
0 01 0
05 02 02 0
= u
Weighting strategy - occurrence frequency (eg smoothened by occurrence time recent concepts are more important - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
User Model
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
u 0
01 0
05 02 02 0
candidate items user a
02 0 0
02 04 01 01
b 0 0 0
08 02 0 0
c 0
05 02 0 0 0
03
cosine similarities
a b c
u 067 092 014
Ranking of recommended items 1 b 2 a 3 c
based on slides from Fabien Abel
Recommendations
Social Web 2014 Lora Aroyo
RecSys Issues
bull Cold-start problem (new user problem) nolittle data available to infer preferences of new users
bull Changing User Preferences user interests may change over time
bull Sparsity problem (new item problem) item descriptions are sparse eg not many user rated or tagged an item
bull Lack of Diversity (overfitting) when adapting too strongly to the preferences of users they might see samesimilar recommendations
bull Use the right context users do things which might not be relevant for their user model eg try out things do stuff for other people
bull Research challenge right balance between serendipity amp personalization
bull Research challenge right way to use the influence of recommendations on userrsquos behavior
one machine13vs 13
humans
image source httpwwwflickrcomphotosbionicteaching1375254387Social Web 2014 Lora Aroyo
Hands-on Teaser
bull Your Facebook Friendsrsquo popularity in a spread sheet13bull Locations of your Facebook Friends13bull Tag Cloud of your wall posts
User Modeling Building Blocks
Profile concept weight
2 Profile Type
4 How to weight the concepts
1 Temporal Constraints
3 Semantic Enrichment
Francesca Schiavone
French Open
Tennis
4 Weighting Scheme
time
June 27 July 4 July 11
weight(Francesca Schiavone)
Concept frequency (TF)
4
3 6
TFxIDF Time-sensitive
weight(French Open)
weight(Tennis)
based on slides from Fabien Abel
Social Web 2014 Lora Aroyo
Observationsbull Profile characteristics
bull Semantic enrichment solves sparsity problems13bull Profiles change over time recent profiles reflect better
current user demands13bull Temporal patterns weekend profiles differ significantly
from weekday profiles13
bull Impact on recommendations bull The more fine-grained the concepts the better the
recommendation performance entity-based gt topic-based gt hashtag-based 13
bull Semantic enrichment improves recommendation quality 13bull Time-sensitivity (adapting to trends) improves
performance
Social Web 2014 Lora Aroyo
User Modelingit is not about putting everything in a user profile 13
it is about making the right choices
Social Web 2014 Lora Aroyo
User AdaptationKnowing the user to adapt a system or interface13
to improve the system functionality and user experience
A Jameson Adaptive interfaces and agents The HCI handbook fundamentals evolving technologies and emerging applications pp 305ndash330 2003
User-Adaptive Systems
user modeling
user profile
observations data and information about user
profile analysis
adaptation decisions
user modeling (infer current musical taste)
user profile interests in
genres artists tags
history of songs like ban pause skip
compare profile with possible next
songs to play
next song to be played
based on slides from Fabien Abel
Lastfm adapts to your music taste
bull Overfitting ldquobubble effectsrdquo loss of serendipity problem 13bull systems may adapt too strongly to the interestsbehavior13bull eg an adaptive radio station may always play the same or
very similar songs13bull We search for the right balance between novelty and
relevance for the user13
bull ldquoLost in Hyperspacerdquo problem 13bull when adapting the navigation ndash ie the links on which
users can click to findaccess information 13bull eg re-orderinghiding of menu items may lead to
confusion
Social Web 2014 Lora Aroyo
Issues in User-Adaptive Systems
Social Web 2014 Lora Aroyo
What is good user modelling amp personalisation
httpwwwflickrcomphotosbellarosebyliz4729613108
bull From the consumer perspective of an adaptive system 13
bull From the provider perspective of an adaptive system
Adaptive system maximizes satisfaction of the user
hard to measureobtain
Adaptive system maximizes the profit
influence of UM amp personalization may be hard to measureobtain
Social Web 2014 Lora Aroyo
Success Perspectives
bull User studies askobserve (selected) people whether you did a good job13
bull Log analysis Analyze (click) data and infer whether you did a good job13
bull Evaluation of user modeling13
bull measure quality of profiles directly eg measure overlap with existing (true) profiles or let people judge the quality of the generated user profiles 13
bull measure quality of application that exploits the user profile eg apply user modeling strategies in a recommender system
Social Web 2014 Lora Aroyo
Evaluation Strategies
time
item A
item B
item C
item D
item E
item G item H
item F
training data test data (ground truth)
Strategy X
User Modeling strategies to compare
Strategy Y
Strategy Z
Recommender
training data Recommendations
item H
X
item R
item M
Y
item H
item G
item N
item H
Z
item F
item M
measure quality
Social Web 2014 Lora Aroyo
Evaluating User Modeling in RecSys
Possible Metricsbull The usual IR metrics13
bull Precision fraction of retrieved items that are relevant13
bull Recall fraction of relevant items that have been retrieved13
bull F-Measure (harmonic) mean of precision and recall13
bull Metrics for evaluating recommendation (rankings)13
bull Mean Reciprocal Rank (MRR) of first relevant item13
bull Successk probability that relevant item occurs within the top k13
bull If a true ranking is given rank correlations 13
bull Precisionk Recallk amp F-Measurek13
bull Metrics for evaluating prediction of user preferences13
bull MAE = Mean Absolute Error13
bull TrueFalse PositivesNegatives runs
performance strategy X baseline
Is strategy X better than the baseline
Social Web 2014 Lora Aroyo
bull [Rae et al] a typical example of how to investigate and evaluate a proposal for improving (tag) recommendations (using social networks)13
bull Task test how well the different strategies (different tag contexts) can be used for tag predictionrecommendation13
bull Steps13
1 Gather a dataset of tag data part of which can be used as input and aim to test the recommendation on the remaining tag data13
2 Use the input data and calculate for the different strategies the predictions13
3 Measure the performance using standard (IR) metrics Precision of the top 5 recommended tags (P5) Mean Reciprocal Rank (MRR) Mean Average Precision (MAP)13
4 Test the results for statistical significance using T-test relative to the baseline (eg existing approach competitive approach)
[Rae et al Improving Tag Recommendations Using Social Networks RIAOrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
bull [Guy et al] another example of a similar evaluation approach13
bull The different strategies differ in the way people amp tags are used with tag-based systems there are complex relationships between users tags and items and strategies aim to find the relevant aspects of these relationships for modeling and recommendation13
bull The baseline is the lsquomost popularrsquo tags - often used to compare the most popular tags to the tags predicted by a particular personalization strategy - investigating whether the personalization is worth the effort and is able to outperform the easily available baseline
[Guy et al Social Media Recommendation based on People and Tags SIGIRrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
recommendation 13dimensions
Predict relevantusefulinteresting items13for a given user (in a given context)13
itrsquos often a ranking task
Social Web 2014 Lora Aroyo
Recommendation Systems
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
March 28 2013
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
commercial 13personalisation
httpwwwwiredcommagazine201111mf_artsyall1
Social Web 2014 Lora Aroyo
filter bubble
u1 likes u2
likes likes u1 likes Pulp Fiction
Social Web 2014 Lora Aroyo
Collaborative Filteringbull Memory-based User-Item matrix ratingspreferences of users =gt compute
similarity between users amp recommend items of similar users13
bull Model-based Item-Item matrix similarity (eg based on user ratings) between items =gt recommend items that are similar to the ones the user likes13
bull Model-based Clustering cluster users according to their preferences =gt recommend items of users that belong to the same cluster13
bull Model-based Bayesian networks P(u likes item B | u likes item A) = how likely is it that a user who likes item A will like item B learn probabilities from user ratingspreferences13
bull Others rule-based other data mining techniques13
bull complete input data is required13
bull pre-computation not possible13
bull does not scale well 13
bull high quality of recommendations13
bull abstraction (model) of input data13
bull pre-computation (partially) possible (model has to be re-built from time to time)13
bull scales better13
bull abstraction may reduce recommendation quality
Social Web 2014 Lora Aroyo
Memory vs Model-based
bull collaborative filtering lsquoneighborhoodsrsquo of people with similar interest amp recommending items based on likings in neighborhood13
bull limitations next to lsquocold startrsquo and lsquosparsityrsquo the lack of control (over onersquos neighborhood) is also a problem ie cannot add lsquotrustedrsquo people nor exclude lsquostrangersquo ones13
bull therefore interest in lsquosocial recommendersrsquo where presence of social connections defines the similarity in interests (eg social tagging CiteULike)13
bull does a social connection indicate user interest similarity13
bull how much users interest similarity depends on the strength of their connection13
bull is it feasible to use a social network as a personalized recommendation
[Lin amp Brusilovsky Social Networks and Interest Similarity The Case of CiteULike HTrsquo10]Social Web 2014 Lora Aroyo
Social Networks amp Interest Similarity
bull unilaterally connected pairs have more common itemsmetadatatags than non-connected pairs13
bull highest similarity for direct connections - decreasing with the increase of distance between users in SN 13
bull reciprocal relationship users - significantly larger similarity than users in a unidirectional relationship 13
bull traditional item-level similarity may be less reliable to find similar users in social bookmarking systems13
bull peers connected by self-defined social connections could be a useful source for cross-recommendation
Social Web 2014 Lora Aroyo
Conclusions
bull Input characteristics of items amp interests of a user into characteristics of items =gt Recommend items that feature characteristics which meet the userrsquos interests13
bull Techniques13
bull Data mining methods Cluster items based on their characteristics =gt Infer usersrsquo interests into clusters13
bull IR methods Represent items amp users as term vectors =gt Compute similarity between user profile vector and items13
bull Utility-based methods Utility function that gets an item as input the parameters of the utility function are customized via preferences of a user
Social Web 2014 Lora Aroyo
Content-based Recommendations
Government stops renovation of tower bridge Oct 13th 2011
Tower Bridge today Under construction
Tower Bridge is a combined bascule and suspension bridge in London England over the River Thames
Category politics england Related Twiper news bob Why do they stop tohellip [more] mary London stops renohellip [more]
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
02 0 0
02 04 01 01
= a
Weighting strategy - occurrence frequency - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
Content Features
RT Government stops renovation of tower bridge Oct 13th 2011
Userrsquos Twitter history
I am in London at the moment Oct 13th 2011
I am doing sports Oct 12th 2011
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
0 01 0
05 02 02 0
= u
Weighting strategy - occurrence frequency (eg smoothened by occurrence time recent concepts are more important - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
User Model
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
u 0
01 0
05 02 02 0
candidate items user a
02 0 0
02 04 01 01
b 0 0 0
08 02 0 0
c 0
05 02 0 0 0
03
cosine similarities
a b c
u 067 092 014
Ranking of recommended items 1 b 2 a 3 c
based on slides from Fabien Abel
Recommendations
Social Web 2014 Lora Aroyo
RecSys Issues
bull Cold-start problem (new user problem) nolittle data available to infer preferences of new users
bull Changing User Preferences user interests may change over time
bull Sparsity problem (new item problem) item descriptions are sparse eg not many user rated or tagged an item
bull Lack of Diversity (overfitting) when adapting too strongly to the preferences of users they might see samesimilar recommendations
bull Use the right context users do things which might not be relevant for their user model eg try out things do stuff for other people
bull Research challenge right balance between serendipity amp personalization
bull Research challenge right way to use the influence of recommendations on userrsquos behavior
one machine13vs 13
humans
image source httpwwwflickrcomphotosbionicteaching1375254387Social Web 2014 Lora Aroyo
Hands-on Teaser
bull Your Facebook Friendsrsquo popularity in a spread sheet13bull Locations of your Facebook Friends13bull Tag Cloud of your wall posts
Social Web 2014 Lora Aroyo
Observationsbull Profile characteristics
bull Semantic enrichment solves sparsity problems13bull Profiles change over time recent profiles reflect better
current user demands13bull Temporal patterns weekend profiles differ significantly
from weekday profiles13
bull Impact on recommendations bull The more fine-grained the concepts the better the
recommendation performance entity-based gt topic-based gt hashtag-based 13
bull Semantic enrichment improves recommendation quality 13bull Time-sensitivity (adapting to trends) improves
performance
Social Web 2014 Lora Aroyo
User Modelingit is not about putting everything in a user profile 13
it is about making the right choices
Social Web 2014 Lora Aroyo
User AdaptationKnowing the user to adapt a system or interface13
to improve the system functionality and user experience
A Jameson Adaptive interfaces and agents The HCI handbook fundamentals evolving technologies and emerging applications pp 305ndash330 2003
User-Adaptive Systems
user modeling
user profile
observations data and information about user
profile analysis
adaptation decisions
user modeling (infer current musical taste)
user profile interests in
genres artists tags
history of songs like ban pause skip
compare profile with possible next
songs to play
next song to be played
based on slides from Fabien Abel
Lastfm adapts to your music taste
bull Overfitting ldquobubble effectsrdquo loss of serendipity problem 13bull systems may adapt too strongly to the interestsbehavior13bull eg an adaptive radio station may always play the same or
very similar songs13bull We search for the right balance between novelty and
relevance for the user13
bull ldquoLost in Hyperspacerdquo problem 13bull when adapting the navigation ndash ie the links on which
users can click to findaccess information 13bull eg re-orderinghiding of menu items may lead to
confusion
Social Web 2014 Lora Aroyo
Issues in User-Adaptive Systems
Social Web 2014 Lora Aroyo
What is good user modelling amp personalisation
httpwwwflickrcomphotosbellarosebyliz4729613108
bull From the consumer perspective of an adaptive system 13
bull From the provider perspective of an adaptive system
Adaptive system maximizes satisfaction of the user
hard to measureobtain
Adaptive system maximizes the profit
influence of UM amp personalization may be hard to measureobtain
Social Web 2014 Lora Aroyo
Success Perspectives
bull User studies askobserve (selected) people whether you did a good job13
bull Log analysis Analyze (click) data and infer whether you did a good job13
bull Evaluation of user modeling13
bull measure quality of profiles directly eg measure overlap with existing (true) profiles or let people judge the quality of the generated user profiles 13
bull measure quality of application that exploits the user profile eg apply user modeling strategies in a recommender system
Social Web 2014 Lora Aroyo
Evaluation Strategies
time
item A
item B
item C
item D
item E
item G item H
item F
training data test data (ground truth)
Strategy X
User Modeling strategies to compare
Strategy Y
Strategy Z
Recommender
training data Recommendations
item H
X
item R
item M
Y
item H
item G
item N
item H
Z
item F
item M
measure quality
Social Web 2014 Lora Aroyo
Evaluating User Modeling in RecSys
Possible Metricsbull The usual IR metrics13
bull Precision fraction of retrieved items that are relevant13
bull Recall fraction of relevant items that have been retrieved13
bull F-Measure (harmonic) mean of precision and recall13
bull Metrics for evaluating recommendation (rankings)13
bull Mean Reciprocal Rank (MRR) of first relevant item13
bull Successk probability that relevant item occurs within the top k13
bull If a true ranking is given rank correlations 13
bull Precisionk Recallk amp F-Measurek13
bull Metrics for evaluating prediction of user preferences13
bull MAE = Mean Absolute Error13
bull TrueFalse PositivesNegatives runs
performance strategy X baseline
Is strategy X better than the baseline
Social Web 2014 Lora Aroyo
bull [Rae et al] a typical example of how to investigate and evaluate a proposal for improving (tag) recommendations (using social networks)13
bull Task test how well the different strategies (different tag contexts) can be used for tag predictionrecommendation13
bull Steps13
1 Gather a dataset of tag data part of which can be used as input and aim to test the recommendation on the remaining tag data13
2 Use the input data and calculate for the different strategies the predictions13
3 Measure the performance using standard (IR) metrics Precision of the top 5 recommended tags (P5) Mean Reciprocal Rank (MRR) Mean Average Precision (MAP)13
4 Test the results for statistical significance using T-test relative to the baseline (eg existing approach competitive approach)
[Rae et al Improving Tag Recommendations Using Social Networks RIAOrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
bull [Guy et al] another example of a similar evaluation approach13
bull The different strategies differ in the way people amp tags are used with tag-based systems there are complex relationships between users tags and items and strategies aim to find the relevant aspects of these relationships for modeling and recommendation13
bull The baseline is the lsquomost popularrsquo tags - often used to compare the most popular tags to the tags predicted by a particular personalization strategy - investigating whether the personalization is worth the effort and is able to outperform the easily available baseline
[Guy et al Social Media Recommendation based on People and Tags SIGIRrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
recommendation 13dimensions
Predict relevantusefulinteresting items13for a given user (in a given context)13
itrsquos often a ranking task
Social Web 2014 Lora Aroyo
Recommendation Systems
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
March 28 2013
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
commercial 13personalisation
httpwwwwiredcommagazine201111mf_artsyall1
Social Web 2014 Lora Aroyo
filter bubble
u1 likes u2
likes likes u1 likes Pulp Fiction
Social Web 2014 Lora Aroyo
Collaborative Filteringbull Memory-based User-Item matrix ratingspreferences of users =gt compute
similarity between users amp recommend items of similar users13
bull Model-based Item-Item matrix similarity (eg based on user ratings) between items =gt recommend items that are similar to the ones the user likes13
bull Model-based Clustering cluster users according to their preferences =gt recommend items of users that belong to the same cluster13
bull Model-based Bayesian networks P(u likes item B | u likes item A) = how likely is it that a user who likes item A will like item B learn probabilities from user ratingspreferences13
bull Others rule-based other data mining techniques13
bull complete input data is required13
bull pre-computation not possible13
bull does not scale well 13
bull high quality of recommendations13
bull abstraction (model) of input data13
bull pre-computation (partially) possible (model has to be re-built from time to time)13
bull scales better13
bull abstraction may reduce recommendation quality
Social Web 2014 Lora Aroyo
Memory vs Model-based
bull collaborative filtering lsquoneighborhoodsrsquo of people with similar interest amp recommending items based on likings in neighborhood13
bull limitations next to lsquocold startrsquo and lsquosparsityrsquo the lack of control (over onersquos neighborhood) is also a problem ie cannot add lsquotrustedrsquo people nor exclude lsquostrangersquo ones13
bull therefore interest in lsquosocial recommendersrsquo where presence of social connections defines the similarity in interests (eg social tagging CiteULike)13
bull does a social connection indicate user interest similarity13
bull how much users interest similarity depends on the strength of their connection13
bull is it feasible to use a social network as a personalized recommendation
[Lin amp Brusilovsky Social Networks and Interest Similarity The Case of CiteULike HTrsquo10]Social Web 2014 Lora Aroyo
Social Networks amp Interest Similarity
bull unilaterally connected pairs have more common itemsmetadatatags than non-connected pairs13
bull highest similarity for direct connections - decreasing with the increase of distance between users in SN 13
bull reciprocal relationship users - significantly larger similarity than users in a unidirectional relationship 13
bull traditional item-level similarity may be less reliable to find similar users in social bookmarking systems13
bull peers connected by self-defined social connections could be a useful source for cross-recommendation
Social Web 2014 Lora Aroyo
Conclusions
bull Input characteristics of items amp interests of a user into characteristics of items =gt Recommend items that feature characteristics which meet the userrsquos interests13
bull Techniques13
bull Data mining methods Cluster items based on their characteristics =gt Infer usersrsquo interests into clusters13
bull IR methods Represent items amp users as term vectors =gt Compute similarity between user profile vector and items13
bull Utility-based methods Utility function that gets an item as input the parameters of the utility function are customized via preferences of a user
Social Web 2014 Lora Aroyo
Content-based Recommendations
Government stops renovation of tower bridge Oct 13th 2011
Tower Bridge today Under construction
Tower Bridge is a combined bascule and suspension bridge in London England over the River Thames
Category politics england Related Twiper news bob Why do they stop tohellip [more] mary London stops renohellip [more]
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
02 0 0
02 04 01 01
= a
Weighting strategy - occurrence frequency - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
Content Features
RT Government stops renovation of tower bridge Oct 13th 2011
Userrsquos Twitter history
I am in London at the moment Oct 13th 2011
I am doing sports Oct 12th 2011
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
0 01 0
05 02 02 0
= u
Weighting strategy - occurrence frequency (eg smoothened by occurrence time recent concepts are more important - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
User Model
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
u 0
01 0
05 02 02 0
candidate items user a
02 0 0
02 04 01 01
b 0 0 0
08 02 0 0
c 0
05 02 0 0 0
03
cosine similarities
a b c
u 067 092 014
Ranking of recommended items 1 b 2 a 3 c
based on slides from Fabien Abel
Recommendations
Social Web 2014 Lora Aroyo
RecSys Issues
bull Cold-start problem (new user problem) nolittle data available to infer preferences of new users
bull Changing User Preferences user interests may change over time
bull Sparsity problem (new item problem) item descriptions are sparse eg not many user rated or tagged an item
bull Lack of Diversity (overfitting) when adapting too strongly to the preferences of users they might see samesimilar recommendations
bull Use the right context users do things which might not be relevant for their user model eg try out things do stuff for other people
bull Research challenge right balance between serendipity amp personalization
bull Research challenge right way to use the influence of recommendations on userrsquos behavior
one machine13vs 13
humans
image source httpwwwflickrcomphotosbionicteaching1375254387Social Web 2014 Lora Aroyo
Hands-on Teaser
bull Your Facebook Friendsrsquo popularity in a spread sheet13bull Locations of your Facebook Friends13bull Tag Cloud of your wall posts
Social Web 2014 Lora Aroyo
User Modelingit is not about putting everything in a user profile 13
it is about making the right choices
Social Web 2014 Lora Aroyo
User AdaptationKnowing the user to adapt a system or interface13
to improve the system functionality and user experience
A Jameson Adaptive interfaces and agents The HCI handbook fundamentals evolving technologies and emerging applications pp 305ndash330 2003
User-Adaptive Systems
user modeling
user profile
observations data and information about user
profile analysis
adaptation decisions
user modeling (infer current musical taste)
user profile interests in
genres artists tags
history of songs like ban pause skip
compare profile with possible next
songs to play
next song to be played
based on slides from Fabien Abel
Lastfm adapts to your music taste
bull Overfitting ldquobubble effectsrdquo loss of serendipity problem 13bull systems may adapt too strongly to the interestsbehavior13bull eg an adaptive radio station may always play the same or
very similar songs13bull We search for the right balance between novelty and
relevance for the user13
bull ldquoLost in Hyperspacerdquo problem 13bull when adapting the navigation ndash ie the links on which
users can click to findaccess information 13bull eg re-orderinghiding of menu items may lead to
confusion
Social Web 2014 Lora Aroyo
Issues in User-Adaptive Systems
Social Web 2014 Lora Aroyo
What is good user modelling amp personalisation
httpwwwflickrcomphotosbellarosebyliz4729613108
bull From the consumer perspective of an adaptive system 13
bull From the provider perspective of an adaptive system
Adaptive system maximizes satisfaction of the user
hard to measureobtain
Adaptive system maximizes the profit
influence of UM amp personalization may be hard to measureobtain
Social Web 2014 Lora Aroyo
Success Perspectives
bull User studies askobserve (selected) people whether you did a good job13
bull Log analysis Analyze (click) data and infer whether you did a good job13
bull Evaluation of user modeling13
bull measure quality of profiles directly eg measure overlap with existing (true) profiles or let people judge the quality of the generated user profiles 13
bull measure quality of application that exploits the user profile eg apply user modeling strategies in a recommender system
Social Web 2014 Lora Aroyo
Evaluation Strategies
time
item A
item B
item C
item D
item E
item G item H
item F
training data test data (ground truth)
Strategy X
User Modeling strategies to compare
Strategy Y
Strategy Z
Recommender
training data Recommendations
item H
X
item R
item M
Y
item H
item G
item N
item H
Z
item F
item M
measure quality
Social Web 2014 Lora Aroyo
Evaluating User Modeling in RecSys
Possible Metricsbull The usual IR metrics13
bull Precision fraction of retrieved items that are relevant13
bull Recall fraction of relevant items that have been retrieved13
bull F-Measure (harmonic) mean of precision and recall13
bull Metrics for evaluating recommendation (rankings)13
bull Mean Reciprocal Rank (MRR) of first relevant item13
bull Successk probability that relevant item occurs within the top k13
bull If a true ranking is given rank correlations 13
bull Precisionk Recallk amp F-Measurek13
bull Metrics for evaluating prediction of user preferences13
bull MAE = Mean Absolute Error13
bull TrueFalse PositivesNegatives runs
performance strategy X baseline
Is strategy X better than the baseline
Social Web 2014 Lora Aroyo
bull [Rae et al] a typical example of how to investigate and evaluate a proposal for improving (tag) recommendations (using social networks)13
bull Task test how well the different strategies (different tag contexts) can be used for tag predictionrecommendation13
bull Steps13
1 Gather a dataset of tag data part of which can be used as input and aim to test the recommendation on the remaining tag data13
2 Use the input data and calculate for the different strategies the predictions13
3 Measure the performance using standard (IR) metrics Precision of the top 5 recommended tags (P5) Mean Reciprocal Rank (MRR) Mean Average Precision (MAP)13
4 Test the results for statistical significance using T-test relative to the baseline (eg existing approach competitive approach)
[Rae et al Improving Tag Recommendations Using Social Networks RIAOrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
bull [Guy et al] another example of a similar evaluation approach13
bull The different strategies differ in the way people amp tags are used with tag-based systems there are complex relationships between users tags and items and strategies aim to find the relevant aspects of these relationships for modeling and recommendation13
bull The baseline is the lsquomost popularrsquo tags - often used to compare the most popular tags to the tags predicted by a particular personalization strategy - investigating whether the personalization is worth the effort and is able to outperform the easily available baseline
[Guy et al Social Media Recommendation based on People and Tags SIGIRrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
recommendation 13dimensions
Predict relevantusefulinteresting items13for a given user (in a given context)13
itrsquos often a ranking task
Social Web 2014 Lora Aroyo
Recommendation Systems
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
March 28 2013
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
commercial 13personalisation
httpwwwwiredcommagazine201111mf_artsyall1
Social Web 2014 Lora Aroyo
filter bubble
u1 likes u2
likes likes u1 likes Pulp Fiction
Social Web 2014 Lora Aroyo
Collaborative Filteringbull Memory-based User-Item matrix ratingspreferences of users =gt compute
similarity between users amp recommend items of similar users13
bull Model-based Item-Item matrix similarity (eg based on user ratings) between items =gt recommend items that are similar to the ones the user likes13
bull Model-based Clustering cluster users according to their preferences =gt recommend items of users that belong to the same cluster13
bull Model-based Bayesian networks P(u likes item B | u likes item A) = how likely is it that a user who likes item A will like item B learn probabilities from user ratingspreferences13
bull Others rule-based other data mining techniques13
bull complete input data is required13
bull pre-computation not possible13
bull does not scale well 13
bull high quality of recommendations13
bull abstraction (model) of input data13
bull pre-computation (partially) possible (model has to be re-built from time to time)13
bull scales better13
bull abstraction may reduce recommendation quality
Social Web 2014 Lora Aroyo
Memory vs Model-based
bull collaborative filtering lsquoneighborhoodsrsquo of people with similar interest amp recommending items based on likings in neighborhood13
bull limitations next to lsquocold startrsquo and lsquosparsityrsquo the lack of control (over onersquos neighborhood) is also a problem ie cannot add lsquotrustedrsquo people nor exclude lsquostrangersquo ones13
bull therefore interest in lsquosocial recommendersrsquo where presence of social connections defines the similarity in interests (eg social tagging CiteULike)13
bull does a social connection indicate user interest similarity13
bull how much users interest similarity depends on the strength of their connection13
bull is it feasible to use a social network as a personalized recommendation
[Lin amp Brusilovsky Social Networks and Interest Similarity The Case of CiteULike HTrsquo10]Social Web 2014 Lora Aroyo
Social Networks amp Interest Similarity
bull unilaterally connected pairs have more common itemsmetadatatags than non-connected pairs13
bull highest similarity for direct connections - decreasing with the increase of distance between users in SN 13
bull reciprocal relationship users - significantly larger similarity than users in a unidirectional relationship 13
bull traditional item-level similarity may be less reliable to find similar users in social bookmarking systems13
bull peers connected by self-defined social connections could be a useful source for cross-recommendation
Social Web 2014 Lora Aroyo
Conclusions
bull Input characteristics of items amp interests of a user into characteristics of items =gt Recommend items that feature characteristics which meet the userrsquos interests13
bull Techniques13
bull Data mining methods Cluster items based on their characteristics =gt Infer usersrsquo interests into clusters13
bull IR methods Represent items amp users as term vectors =gt Compute similarity between user profile vector and items13
bull Utility-based methods Utility function that gets an item as input the parameters of the utility function are customized via preferences of a user
Social Web 2014 Lora Aroyo
Content-based Recommendations
Government stops renovation of tower bridge Oct 13th 2011
Tower Bridge today Under construction
Tower Bridge is a combined bascule and suspension bridge in London England over the River Thames
Category politics england Related Twiper news bob Why do they stop tohellip [more] mary London stops renohellip [more]
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
02 0 0
02 04 01 01
= a
Weighting strategy - occurrence frequency - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
Content Features
RT Government stops renovation of tower bridge Oct 13th 2011
Userrsquos Twitter history
I am in London at the moment Oct 13th 2011
I am doing sports Oct 12th 2011
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
0 01 0
05 02 02 0
= u
Weighting strategy - occurrence frequency (eg smoothened by occurrence time recent concepts are more important - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
User Model
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
u 0
01 0
05 02 02 0
candidate items user a
02 0 0
02 04 01 01
b 0 0 0
08 02 0 0
c 0
05 02 0 0 0
03
cosine similarities
a b c
u 067 092 014
Ranking of recommended items 1 b 2 a 3 c
based on slides from Fabien Abel
Recommendations
Social Web 2014 Lora Aroyo
RecSys Issues
bull Cold-start problem (new user problem) nolittle data available to infer preferences of new users
bull Changing User Preferences user interests may change over time
bull Sparsity problem (new item problem) item descriptions are sparse eg not many user rated or tagged an item
bull Lack of Diversity (overfitting) when adapting too strongly to the preferences of users they might see samesimilar recommendations
bull Use the right context users do things which might not be relevant for their user model eg try out things do stuff for other people
bull Research challenge right balance between serendipity amp personalization
bull Research challenge right way to use the influence of recommendations on userrsquos behavior
one machine13vs 13
humans
image source httpwwwflickrcomphotosbionicteaching1375254387Social Web 2014 Lora Aroyo
Hands-on Teaser
bull Your Facebook Friendsrsquo popularity in a spread sheet13bull Locations of your Facebook Friends13bull Tag Cloud of your wall posts
Social Web 2014 Lora Aroyo
User AdaptationKnowing the user to adapt a system or interface13
to improve the system functionality and user experience
A Jameson Adaptive interfaces and agents The HCI handbook fundamentals evolving technologies and emerging applications pp 305ndash330 2003
User-Adaptive Systems
user modeling
user profile
observations data and information about user
profile analysis
adaptation decisions
user modeling (infer current musical taste)
user profile interests in
genres artists tags
history of songs like ban pause skip
compare profile with possible next
songs to play
next song to be played
based on slides from Fabien Abel
Lastfm adapts to your music taste
bull Overfitting ldquobubble effectsrdquo loss of serendipity problem 13bull systems may adapt too strongly to the interestsbehavior13bull eg an adaptive radio station may always play the same or
very similar songs13bull We search for the right balance between novelty and
relevance for the user13
bull ldquoLost in Hyperspacerdquo problem 13bull when adapting the navigation ndash ie the links on which
users can click to findaccess information 13bull eg re-orderinghiding of menu items may lead to
confusion
Social Web 2014 Lora Aroyo
Issues in User-Adaptive Systems
Social Web 2014 Lora Aroyo
What is good user modelling amp personalisation
httpwwwflickrcomphotosbellarosebyliz4729613108
bull From the consumer perspective of an adaptive system 13
bull From the provider perspective of an adaptive system
Adaptive system maximizes satisfaction of the user
hard to measureobtain
Adaptive system maximizes the profit
influence of UM amp personalization may be hard to measureobtain
Social Web 2014 Lora Aroyo
Success Perspectives
bull User studies askobserve (selected) people whether you did a good job13
bull Log analysis Analyze (click) data and infer whether you did a good job13
bull Evaluation of user modeling13
bull measure quality of profiles directly eg measure overlap with existing (true) profiles or let people judge the quality of the generated user profiles 13
bull measure quality of application that exploits the user profile eg apply user modeling strategies in a recommender system
Social Web 2014 Lora Aroyo
Evaluation Strategies
time
item A
item B
item C
item D
item E
item G item H
item F
training data test data (ground truth)
Strategy X
User Modeling strategies to compare
Strategy Y
Strategy Z
Recommender
training data Recommendations
item H
X
item R
item M
Y
item H
item G
item N
item H
Z
item F
item M
measure quality
Social Web 2014 Lora Aroyo
Evaluating User Modeling in RecSys
Possible Metricsbull The usual IR metrics13
bull Precision fraction of retrieved items that are relevant13
bull Recall fraction of relevant items that have been retrieved13
bull F-Measure (harmonic) mean of precision and recall13
bull Metrics for evaluating recommendation (rankings)13
bull Mean Reciprocal Rank (MRR) of first relevant item13
bull Successk probability that relevant item occurs within the top k13
bull If a true ranking is given rank correlations 13
bull Precisionk Recallk amp F-Measurek13
bull Metrics for evaluating prediction of user preferences13
bull MAE = Mean Absolute Error13
bull TrueFalse PositivesNegatives runs
performance strategy X baseline
Is strategy X better than the baseline
Social Web 2014 Lora Aroyo
bull [Rae et al] a typical example of how to investigate and evaluate a proposal for improving (tag) recommendations (using social networks)13
bull Task test how well the different strategies (different tag contexts) can be used for tag predictionrecommendation13
bull Steps13
1 Gather a dataset of tag data part of which can be used as input and aim to test the recommendation on the remaining tag data13
2 Use the input data and calculate for the different strategies the predictions13
3 Measure the performance using standard (IR) metrics Precision of the top 5 recommended tags (P5) Mean Reciprocal Rank (MRR) Mean Average Precision (MAP)13
4 Test the results for statistical significance using T-test relative to the baseline (eg existing approach competitive approach)
[Rae et al Improving Tag Recommendations Using Social Networks RIAOrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
bull [Guy et al] another example of a similar evaluation approach13
bull The different strategies differ in the way people amp tags are used with tag-based systems there are complex relationships between users tags and items and strategies aim to find the relevant aspects of these relationships for modeling and recommendation13
bull The baseline is the lsquomost popularrsquo tags - often used to compare the most popular tags to the tags predicted by a particular personalization strategy - investigating whether the personalization is worth the effort and is able to outperform the easily available baseline
[Guy et al Social Media Recommendation based on People and Tags SIGIRrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
recommendation 13dimensions
Predict relevantusefulinteresting items13for a given user (in a given context)13
itrsquos often a ranking task
Social Web 2014 Lora Aroyo
Recommendation Systems
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
March 28 2013
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
commercial 13personalisation
httpwwwwiredcommagazine201111mf_artsyall1
Social Web 2014 Lora Aroyo
filter bubble
u1 likes u2
likes likes u1 likes Pulp Fiction
Social Web 2014 Lora Aroyo
Collaborative Filteringbull Memory-based User-Item matrix ratingspreferences of users =gt compute
similarity between users amp recommend items of similar users13
bull Model-based Item-Item matrix similarity (eg based on user ratings) between items =gt recommend items that are similar to the ones the user likes13
bull Model-based Clustering cluster users according to their preferences =gt recommend items of users that belong to the same cluster13
bull Model-based Bayesian networks P(u likes item B | u likes item A) = how likely is it that a user who likes item A will like item B learn probabilities from user ratingspreferences13
bull Others rule-based other data mining techniques13
bull complete input data is required13
bull pre-computation not possible13
bull does not scale well 13
bull high quality of recommendations13
bull abstraction (model) of input data13
bull pre-computation (partially) possible (model has to be re-built from time to time)13
bull scales better13
bull abstraction may reduce recommendation quality
Social Web 2014 Lora Aroyo
Memory vs Model-based
bull collaborative filtering lsquoneighborhoodsrsquo of people with similar interest amp recommending items based on likings in neighborhood13
bull limitations next to lsquocold startrsquo and lsquosparsityrsquo the lack of control (over onersquos neighborhood) is also a problem ie cannot add lsquotrustedrsquo people nor exclude lsquostrangersquo ones13
bull therefore interest in lsquosocial recommendersrsquo where presence of social connections defines the similarity in interests (eg social tagging CiteULike)13
bull does a social connection indicate user interest similarity13
bull how much users interest similarity depends on the strength of their connection13
bull is it feasible to use a social network as a personalized recommendation
[Lin amp Brusilovsky Social Networks and Interest Similarity The Case of CiteULike HTrsquo10]Social Web 2014 Lora Aroyo
Social Networks amp Interest Similarity
bull unilaterally connected pairs have more common itemsmetadatatags than non-connected pairs13
bull highest similarity for direct connections - decreasing with the increase of distance between users in SN 13
bull reciprocal relationship users - significantly larger similarity than users in a unidirectional relationship 13
bull traditional item-level similarity may be less reliable to find similar users in social bookmarking systems13
bull peers connected by self-defined social connections could be a useful source for cross-recommendation
Social Web 2014 Lora Aroyo
Conclusions
bull Input characteristics of items amp interests of a user into characteristics of items =gt Recommend items that feature characteristics which meet the userrsquos interests13
bull Techniques13
bull Data mining methods Cluster items based on their characteristics =gt Infer usersrsquo interests into clusters13
bull IR methods Represent items amp users as term vectors =gt Compute similarity between user profile vector and items13
bull Utility-based methods Utility function that gets an item as input the parameters of the utility function are customized via preferences of a user
Social Web 2014 Lora Aroyo
Content-based Recommendations
Government stops renovation of tower bridge Oct 13th 2011
Tower Bridge today Under construction
Tower Bridge is a combined bascule and suspension bridge in London England over the River Thames
Category politics england Related Twiper news bob Why do they stop tohellip [more] mary London stops renohellip [more]
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
02 0 0
02 04 01 01
= a
Weighting strategy - occurrence frequency - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
Content Features
RT Government stops renovation of tower bridge Oct 13th 2011
Userrsquos Twitter history
I am in London at the moment Oct 13th 2011
I am doing sports Oct 12th 2011
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
0 01 0
05 02 02 0
= u
Weighting strategy - occurrence frequency (eg smoothened by occurrence time recent concepts are more important - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
User Model
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
u 0
01 0
05 02 02 0
candidate items user a
02 0 0
02 04 01 01
b 0 0 0
08 02 0 0
c 0
05 02 0 0 0
03
cosine similarities
a b c
u 067 092 014
Ranking of recommended items 1 b 2 a 3 c
based on slides from Fabien Abel
Recommendations
Social Web 2014 Lora Aroyo
RecSys Issues
bull Cold-start problem (new user problem) nolittle data available to infer preferences of new users
bull Changing User Preferences user interests may change over time
bull Sparsity problem (new item problem) item descriptions are sparse eg not many user rated or tagged an item
bull Lack of Diversity (overfitting) when adapting too strongly to the preferences of users they might see samesimilar recommendations
bull Use the right context users do things which might not be relevant for their user model eg try out things do stuff for other people
bull Research challenge right balance between serendipity amp personalization
bull Research challenge right way to use the influence of recommendations on userrsquos behavior
one machine13vs 13
humans
image source httpwwwflickrcomphotosbionicteaching1375254387Social Web 2014 Lora Aroyo
Hands-on Teaser
bull Your Facebook Friendsrsquo popularity in a spread sheet13bull Locations of your Facebook Friends13bull Tag Cloud of your wall posts
A Jameson Adaptive interfaces and agents The HCI handbook fundamentals evolving technologies and emerging applications pp 305ndash330 2003
User-Adaptive Systems
user modeling
user profile
observations data and information about user
profile analysis
adaptation decisions
user modeling (infer current musical taste)
user profile interests in
genres artists tags
history of songs like ban pause skip
compare profile with possible next
songs to play
next song to be played
based on slides from Fabien Abel
Lastfm adapts to your music taste
bull Overfitting ldquobubble effectsrdquo loss of serendipity problem 13bull systems may adapt too strongly to the interestsbehavior13bull eg an adaptive radio station may always play the same or
very similar songs13bull We search for the right balance between novelty and
relevance for the user13
bull ldquoLost in Hyperspacerdquo problem 13bull when adapting the navigation ndash ie the links on which
users can click to findaccess information 13bull eg re-orderinghiding of menu items may lead to
confusion
Social Web 2014 Lora Aroyo
Issues in User-Adaptive Systems
Social Web 2014 Lora Aroyo
What is good user modelling amp personalisation
httpwwwflickrcomphotosbellarosebyliz4729613108
bull From the consumer perspective of an adaptive system 13
bull From the provider perspective of an adaptive system
Adaptive system maximizes satisfaction of the user
hard to measureobtain
Adaptive system maximizes the profit
influence of UM amp personalization may be hard to measureobtain
Social Web 2014 Lora Aroyo
Success Perspectives
bull User studies askobserve (selected) people whether you did a good job13
bull Log analysis Analyze (click) data and infer whether you did a good job13
bull Evaluation of user modeling13
bull measure quality of profiles directly eg measure overlap with existing (true) profiles or let people judge the quality of the generated user profiles 13
bull measure quality of application that exploits the user profile eg apply user modeling strategies in a recommender system
Social Web 2014 Lora Aroyo
Evaluation Strategies
time
item A
item B
item C
item D
item E
item G item H
item F
training data test data (ground truth)
Strategy X
User Modeling strategies to compare
Strategy Y
Strategy Z
Recommender
training data Recommendations
item H
X
item R
item M
Y
item H
item G
item N
item H
Z
item F
item M
measure quality
Social Web 2014 Lora Aroyo
Evaluating User Modeling in RecSys
Possible Metricsbull The usual IR metrics13
bull Precision fraction of retrieved items that are relevant13
bull Recall fraction of relevant items that have been retrieved13
bull F-Measure (harmonic) mean of precision and recall13
bull Metrics for evaluating recommendation (rankings)13
bull Mean Reciprocal Rank (MRR) of first relevant item13
bull Successk probability that relevant item occurs within the top k13
bull If a true ranking is given rank correlations 13
bull Precisionk Recallk amp F-Measurek13
bull Metrics for evaluating prediction of user preferences13
bull MAE = Mean Absolute Error13
bull TrueFalse PositivesNegatives runs
performance strategy X baseline
Is strategy X better than the baseline
Social Web 2014 Lora Aroyo
bull [Rae et al] a typical example of how to investigate and evaluate a proposal for improving (tag) recommendations (using social networks)13
bull Task test how well the different strategies (different tag contexts) can be used for tag predictionrecommendation13
bull Steps13
1 Gather a dataset of tag data part of which can be used as input and aim to test the recommendation on the remaining tag data13
2 Use the input data and calculate for the different strategies the predictions13
3 Measure the performance using standard (IR) metrics Precision of the top 5 recommended tags (P5) Mean Reciprocal Rank (MRR) Mean Average Precision (MAP)13
4 Test the results for statistical significance using T-test relative to the baseline (eg existing approach competitive approach)
[Rae et al Improving Tag Recommendations Using Social Networks RIAOrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
bull [Guy et al] another example of a similar evaluation approach13
bull The different strategies differ in the way people amp tags are used with tag-based systems there are complex relationships between users tags and items and strategies aim to find the relevant aspects of these relationships for modeling and recommendation13
bull The baseline is the lsquomost popularrsquo tags - often used to compare the most popular tags to the tags predicted by a particular personalization strategy - investigating whether the personalization is worth the effort and is able to outperform the easily available baseline
[Guy et al Social Media Recommendation based on People and Tags SIGIRrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
recommendation 13dimensions
Predict relevantusefulinteresting items13for a given user (in a given context)13
itrsquos often a ranking task
Social Web 2014 Lora Aroyo
Recommendation Systems
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
March 28 2013
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
commercial 13personalisation
httpwwwwiredcommagazine201111mf_artsyall1
Social Web 2014 Lora Aroyo
filter bubble
u1 likes u2
likes likes u1 likes Pulp Fiction
Social Web 2014 Lora Aroyo
Collaborative Filteringbull Memory-based User-Item matrix ratingspreferences of users =gt compute
similarity between users amp recommend items of similar users13
bull Model-based Item-Item matrix similarity (eg based on user ratings) between items =gt recommend items that are similar to the ones the user likes13
bull Model-based Clustering cluster users according to their preferences =gt recommend items of users that belong to the same cluster13
bull Model-based Bayesian networks P(u likes item B | u likes item A) = how likely is it that a user who likes item A will like item B learn probabilities from user ratingspreferences13
bull Others rule-based other data mining techniques13
bull complete input data is required13
bull pre-computation not possible13
bull does not scale well 13
bull high quality of recommendations13
bull abstraction (model) of input data13
bull pre-computation (partially) possible (model has to be re-built from time to time)13
bull scales better13
bull abstraction may reduce recommendation quality
Social Web 2014 Lora Aroyo
Memory vs Model-based
bull collaborative filtering lsquoneighborhoodsrsquo of people with similar interest amp recommending items based on likings in neighborhood13
bull limitations next to lsquocold startrsquo and lsquosparsityrsquo the lack of control (over onersquos neighborhood) is also a problem ie cannot add lsquotrustedrsquo people nor exclude lsquostrangersquo ones13
bull therefore interest in lsquosocial recommendersrsquo where presence of social connections defines the similarity in interests (eg social tagging CiteULike)13
bull does a social connection indicate user interest similarity13
bull how much users interest similarity depends on the strength of their connection13
bull is it feasible to use a social network as a personalized recommendation
[Lin amp Brusilovsky Social Networks and Interest Similarity The Case of CiteULike HTrsquo10]Social Web 2014 Lora Aroyo
Social Networks amp Interest Similarity
bull unilaterally connected pairs have more common itemsmetadatatags than non-connected pairs13
bull highest similarity for direct connections - decreasing with the increase of distance between users in SN 13
bull reciprocal relationship users - significantly larger similarity than users in a unidirectional relationship 13
bull traditional item-level similarity may be less reliable to find similar users in social bookmarking systems13
bull peers connected by self-defined social connections could be a useful source for cross-recommendation
Social Web 2014 Lora Aroyo
Conclusions
bull Input characteristics of items amp interests of a user into characteristics of items =gt Recommend items that feature characteristics which meet the userrsquos interests13
bull Techniques13
bull Data mining methods Cluster items based on their characteristics =gt Infer usersrsquo interests into clusters13
bull IR methods Represent items amp users as term vectors =gt Compute similarity between user profile vector and items13
bull Utility-based methods Utility function that gets an item as input the parameters of the utility function are customized via preferences of a user
Social Web 2014 Lora Aroyo
Content-based Recommendations
Government stops renovation of tower bridge Oct 13th 2011
Tower Bridge today Under construction
Tower Bridge is a combined bascule and suspension bridge in London England over the River Thames
Category politics england Related Twiper news bob Why do they stop tohellip [more] mary London stops renohellip [more]
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
02 0 0
02 04 01 01
= a
Weighting strategy - occurrence frequency - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
Content Features
RT Government stops renovation of tower bridge Oct 13th 2011
Userrsquos Twitter history
I am in London at the moment Oct 13th 2011
I am doing sports Oct 12th 2011
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
0 01 0
05 02 02 0
= u
Weighting strategy - occurrence frequency (eg smoothened by occurrence time recent concepts are more important - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
User Model
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
u 0
01 0
05 02 02 0
candidate items user a
02 0 0
02 04 01 01
b 0 0 0
08 02 0 0
c 0
05 02 0 0 0
03
cosine similarities
a b c
u 067 092 014
Ranking of recommended items 1 b 2 a 3 c
based on slides from Fabien Abel
Recommendations
Social Web 2014 Lora Aroyo
RecSys Issues
bull Cold-start problem (new user problem) nolittle data available to infer preferences of new users
bull Changing User Preferences user interests may change over time
bull Sparsity problem (new item problem) item descriptions are sparse eg not many user rated or tagged an item
bull Lack of Diversity (overfitting) when adapting too strongly to the preferences of users they might see samesimilar recommendations
bull Use the right context users do things which might not be relevant for their user model eg try out things do stuff for other people
bull Research challenge right balance between serendipity amp personalization
bull Research challenge right way to use the influence of recommendations on userrsquos behavior
one machine13vs 13
humans
image source httpwwwflickrcomphotosbionicteaching1375254387Social Web 2014 Lora Aroyo
Hands-on Teaser
bull Your Facebook Friendsrsquo popularity in a spread sheet13bull Locations of your Facebook Friends13bull Tag Cloud of your wall posts
user modeling (infer current musical taste)
user profile interests in
genres artists tags
history of songs like ban pause skip
compare profile with possible next
songs to play
next song to be played
based on slides from Fabien Abel
Lastfm adapts to your music taste
bull Overfitting ldquobubble effectsrdquo loss of serendipity problem 13bull systems may adapt too strongly to the interestsbehavior13bull eg an adaptive radio station may always play the same or
very similar songs13bull We search for the right balance between novelty and
relevance for the user13
bull ldquoLost in Hyperspacerdquo problem 13bull when adapting the navigation ndash ie the links on which
users can click to findaccess information 13bull eg re-orderinghiding of menu items may lead to
confusion
Social Web 2014 Lora Aroyo
Issues in User-Adaptive Systems
Social Web 2014 Lora Aroyo
What is good user modelling amp personalisation
httpwwwflickrcomphotosbellarosebyliz4729613108
bull From the consumer perspective of an adaptive system 13
bull From the provider perspective of an adaptive system
Adaptive system maximizes satisfaction of the user
hard to measureobtain
Adaptive system maximizes the profit
influence of UM amp personalization may be hard to measureobtain
Social Web 2014 Lora Aroyo
Success Perspectives
bull User studies askobserve (selected) people whether you did a good job13
bull Log analysis Analyze (click) data and infer whether you did a good job13
bull Evaluation of user modeling13
bull measure quality of profiles directly eg measure overlap with existing (true) profiles or let people judge the quality of the generated user profiles 13
bull measure quality of application that exploits the user profile eg apply user modeling strategies in a recommender system
Social Web 2014 Lora Aroyo
Evaluation Strategies
time
item A
item B
item C
item D
item E
item G item H
item F
training data test data (ground truth)
Strategy X
User Modeling strategies to compare
Strategy Y
Strategy Z
Recommender
training data Recommendations
item H
X
item R
item M
Y
item H
item G
item N
item H
Z
item F
item M
measure quality
Social Web 2014 Lora Aroyo
Evaluating User Modeling in RecSys
Possible Metricsbull The usual IR metrics13
bull Precision fraction of retrieved items that are relevant13
bull Recall fraction of relevant items that have been retrieved13
bull F-Measure (harmonic) mean of precision and recall13
bull Metrics for evaluating recommendation (rankings)13
bull Mean Reciprocal Rank (MRR) of first relevant item13
bull Successk probability that relevant item occurs within the top k13
bull If a true ranking is given rank correlations 13
bull Precisionk Recallk amp F-Measurek13
bull Metrics for evaluating prediction of user preferences13
bull MAE = Mean Absolute Error13
bull TrueFalse PositivesNegatives runs
performance strategy X baseline
Is strategy X better than the baseline
Social Web 2014 Lora Aroyo
bull [Rae et al] a typical example of how to investigate and evaluate a proposal for improving (tag) recommendations (using social networks)13
bull Task test how well the different strategies (different tag contexts) can be used for tag predictionrecommendation13
bull Steps13
1 Gather a dataset of tag data part of which can be used as input and aim to test the recommendation on the remaining tag data13
2 Use the input data and calculate for the different strategies the predictions13
3 Measure the performance using standard (IR) metrics Precision of the top 5 recommended tags (P5) Mean Reciprocal Rank (MRR) Mean Average Precision (MAP)13
4 Test the results for statistical significance using T-test relative to the baseline (eg existing approach competitive approach)
[Rae et al Improving Tag Recommendations Using Social Networks RIAOrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
bull [Guy et al] another example of a similar evaluation approach13
bull The different strategies differ in the way people amp tags are used with tag-based systems there are complex relationships between users tags and items and strategies aim to find the relevant aspects of these relationships for modeling and recommendation13
bull The baseline is the lsquomost popularrsquo tags - often used to compare the most popular tags to the tags predicted by a particular personalization strategy - investigating whether the personalization is worth the effort and is able to outperform the easily available baseline
[Guy et al Social Media Recommendation based on People and Tags SIGIRrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
recommendation 13dimensions
Predict relevantusefulinteresting items13for a given user (in a given context)13
itrsquos often a ranking task
Social Web 2014 Lora Aroyo
Recommendation Systems
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
March 28 2013
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
commercial 13personalisation
httpwwwwiredcommagazine201111mf_artsyall1
Social Web 2014 Lora Aroyo
filter bubble
u1 likes u2
likes likes u1 likes Pulp Fiction
Social Web 2014 Lora Aroyo
Collaborative Filteringbull Memory-based User-Item matrix ratingspreferences of users =gt compute
similarity between users amp recommend items of similar users13
bull Model-based Item-Item matrix similarity (eg based on user ratings) between items =gt recommend items that are similar to the ones the user likes13
bull Model-based Clustering cluster users according to their preferences =gt recommend items of users that belong to the same cluster13
bull Model-based Bayesian networks P(u likes item B | u likes item A) = how likely is it that a user who likes item A will like item B learn probabilities from user ratingspreferences13
bull Others rule-based other data mining techniques13
bull complete input data is required13
bull pre-computation not possible13
bull does not scale well 13
bull high quality of recommendations13
bull abstraction (model) of input data13
bull pre-computation (partially) possible (model has to be re-built from time to time)13
bull scales better13
bull abstraction may reduce recommendation quality
Social Web 2014 Lora Aroyo
Memory vs Model-based
bull collaborative filtering lsquoneighborhoodsrsquo of people with similar interest amp recommending items based on likings in neighborhood13
bull limitations next to lsquocold startrsquo and lsquosparsityrsquo the lack of control (over onersquos neighborhood) is also a problem ie cannot add lsquotrustedrsquo people nor exclude lsquostrangersquo ones13
bull therefore interest in lsquosocial recommendersrsquo where presence of social connections defines the similarity in interests (eg social tagging CiteULike)13
bull does a social connection indicate user interest similarity13
bull how much users interest similarity depends on the strength of their connection13
bull is it feasible to use a social network as a personalized recommendation
[Lin amp Brusilovsky Social Networks and Interest Similarity The Case of CiteULike HTrsquo10]Social Web 2014 Lora Aroyo
Social Networks amp Interest Similarity
bull unilaterally connected pairs have more common itemsmetadatatags than non-connected pairs13
bull highest similarity for direct connections - decreasing with the increase of distance between users in SN 13
bull reciprocal relationship users - significantly larger similarity than users in a unidirectional relationship 13
bull traditional item-level similarity may be less reliable to find similar users in social bookmarking systems13
bull peers connected by self-defined social connections could be a useful source for cross-recommendation
Social Web 2014 Lora Aroyo
Conclusions
bull Input characteristics of items amp interests of a user into characteristics of items =gt Recommend items that feature characteristics which meet the userrsquos interests13
bull Techniques13
bull Data mining methods Cluster items based on their characteristics =gt Infer usersrsquo interests into clusters13
bull IR methods Represent items amp users as term vectors =gt Compute similarity between user profile vector and items13
bull Utility-based methods Utility function that gets an item as input the parameters of the utility function are customized via preferences of a user
Social Web 2014 Lora Aroyo
Content-based Recommendations
Government stops renovation of tower bridge Oct 13th 2011
Tower Bridge today Under construction
Tower Bridge is a combined bascule and suspension bridge in London England over the River Thames
Category politics england Related Twiper news bob Why do they stop tohellip [more] mary London stops renohellip [more]
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
02 0 0
02 04 01 01
= a
Weighting strategy - occurrence frequency - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
Content Features
RT Government stops renovation of tower bridge Oct 13th 2011
Userrsquos Twitter history
I am in London at the moment Oct 13th 2011
I am doing sports Oct 12th 2011
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
0 01 0
05 02 02 0
= u
Weighting strategy - occurrence frequency (eg smoothened by occurrence time recent concepts are more important - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
User Model
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
u 0
01 0
05 02 02 0
candidate items user a
02 0 0
02 04 01 01
b 0 0 0
08 02 0 0
c 0
05 02 0 0 0
03
cosine similarities
a b c
u 067 092 014
Ranking of recommended items 1 b 2 a 3 c
based on slides from Fabien Abel
Recommendations
Social Web 2014 Lora Aroyo
RecSys Issues
bull Cold-start problem (new user problem) nolittle data available to infer preferences of new users
bull Changing User Preferences user interests may change over time
bull Sparsity problem (new item problem) item descriptions are sparse eg not many user rated or tagged an item
bull Lack of Diversity (overfitting) when adapting too strongly to the preferences of users they might see samesimilar recommendations
bull Use the right context users do things which might not be relevant for their user model eg try out things do stuff for other people
bull Research challenge right balance between serendipity amp personalization
bull Research challenge right way to use the influence of recommendations on userrsquos behavior
one machine13vs 13
humans
image source httpwwwflickrcomphotosbionicteaching1375254387Social Web 2014 Lora Aroyo
Hands-on Teaser
bull Your Facebook Friendsrsquo popularity in a spread sheet13bull Locations of your Facebook Friends13bull Tag Cloud of your wall posts
bull Overfitting ldquobubble effectsrdquo loss of serendipity problem 13bull systems may adapt too strongly to the interestsbehavior13bull eg an adaptive radio station may always play the same or
very similar songs13bull We search for the right balance between novelty and
relevance for the user13
bull ldquoLost in Hyperspacerdquo problem 13bull when adapting the navigation ndash ie the links on which
users can click to findaccess information 13bull eg re-orderinghiding of menu items may lead to
confusion
Social Web 2014 Lora Aroyo
Issues in User-Adaptive Systems
Social Web 2014 Lora Aroyo
What is good user modelling amp personalisation
httpwwwflickrcomphotosbellarosebyliz4729613108
bull From the consumer perspective of an adaptive system 13
bull From the provider perspective of an adaptive system
Adaptive system maximizes satisfaction of the user
hard to measureobtain
Adaptive system maximizes the profit
influence of UM amp personalization may be hard to measureobtain
Social Web 2014 Lora Aroyo
Success Perspectives
bull User studies askobserve (selected) people whether you did a good job13
bull Log analysis Analyze (click) data and infer whether you did a good job13
bull Evaluation of user modeling13
bull measure quality of profiles directly eg measure overlap with existing (true) profiles or let people judge the quality of the generated user profiles 13
bull measure quality of application that exploits the user profile eg apply user modeling strategies in a recommender system
Social Web 2014 Lora Aroyo
Evaluation Strategies
time
item A
item B
item C
item D
item E
item G item H
item F
training data test data (ground truth)
Strategy X
User Modeling strategies to compare
Strategy Y
Strategy Z
Recommender
training data Recommendations
item H
X
item R
item M
Y
item H
item G
item N
item H
Z
item F
item M
measure quality
Social Web 2014 Lora Aroyo
Evaluating User Modeling in RecSys
Possible Metricsbull The usual IR metrics13
bull Precision fraction of retrieved items that are relevant13
bull Recall fraction of relevant items that have been retrieved13
bull F-Measure (harmonic) mean of precision and recall13
bull Metrics for evaluating recommendation (rankings)13
bull Mean Reciprocal Rank (MRR) of first relevant item13
bull Successk probability that relevant item occurs within the top k13
bull If a true ranking is given rank correlations 13
bull Precisionk Recallk amp F-Measurek13
bull Metrics for evaluating prediction of user preferences13
bull MAE = Mean Absolute Error13
bull TrueFalse PositivesNegatives runs
performance strategy X baseline
Is strategy X better than the baseline
Social Web 2014 Lora Aroyo
bull [Rae et al] a typical example of how to investigate and evaluate a proposal for improving (tag) recommendations (using social networks)13
bull Task test how well the different strategies (different tag contexts) can be used for tag predictionrecommendation13
bull Steps13
1 Gather a dataset of tag data part of which can be used as input and aim to test the recommendation on the remaining tag data13
2 Use the input data and calculate for the different strategies the predictions13
3 Measure the performance using standard (IR) metrics Precision of the top 5 recommended tags (P5) Mean Reciprocal Rank (MRR) Mean Average Precision (MAP)13
4 Test the results for statistical significance using T-test relative to the baseline (eg existing approach competitive approach)
[Rae et al Improving Tag Recommendations Using Social Networks RIAOrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
bull [Guy et al] another example of a similar evaluation approach13
bull The different strategies differ in the way people amp tags are used with tag-based systems there are complex relationships between users tags and items and strategies aim to find the relevant aspects of these relationships for modeling and recommendation13
bull The baseline is the lsquomost popularrsquo tags - often used to compare the most popular tags to the tags predicted by a particular personalization strategy - investigating whether the personalization is worth the effort and is able to outperform the easily available baseline
[Guy et al Social Media Recommendation based on People and Tags SIGIRrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
recommendation 13dimensions
Predict relevantusefulinteresting items13for a given user (in a given context)13
itrsquos often a ranking task
Social Web 2014 Lora Aroyo
Recommendation Systems
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
March 28 2013
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
commercial 13personalisation
httpwwwwiredcommagazine201111mf_artsyall1
Social Web 2014 Lora Aroyo
filter bubble
u1 likes u2
likes likes u1 likes Pulp Fiction
Social Web 2014 Lora Aroyo
Collaborative Filteringbull Memory-based User-Item matrix ratingspreferences of users =gt compute
similarity between users amp recommend items of similar users13
bull Model-based Item-Item matrix similarity (eg based on user ratings) between items =gt recommend items that are similar to the ones the user likes13
bull Model-based Clustering cluster users according to their preferences =gt recommend items of users that belong to the same cluster13
bull Model-based Bayesian networks P(u likes item B | u likes item A) = how likely is it that a user who likes item A will like item B learn probabilities from user ratingspreferences13
bull Others rule-based other data mining techniques13
bull complete input data is required13
bull pre-computation not possible13
bull does not scale well 13
bull high quality of recommendations13
bull abstraction (model) of input data13
bull pre-computation (partially) possible (model has to be re-built from time to time)13
bull scales better13
bull abstraction may reduce recommendation quality
Social Web 2014 Lora Aroyo
Memory vs Model-based
bull collaborative filtering lsquoneighborhoodsrsquo of people with similar interest amp recommending items based on likings in neighborhood13
bull limitations next to lsquocold startrsquo and lsquosparsityrsquo the lack of control (over onersquos neighborhood) is also a problem ie cannot add lsquotrustedrsquo people nor exclude lsquostrangersquo ones13
bull therefore interest in lsquosocial recommendersrsquo where presence of social connections defines the similarity in interests (eg social tagging CiteULike)13
bull does a social connection indicate user interest similarity13
bull how much users interest similarity depends on the strength of their connection13
bull is it feasible to use a social network as a personalized recommendation
[Lin amp Brusilovsky Social Networks and Interest Similarity The Case of CiteULike HTrsquo10]Social Web 2014 Lora Aroyo
Social Networks amp Interest Similarity
bull unilaterally connected pairs have more common itemsmetadatatags than non-connected pairs13
bull highest similarity for direct connections - decreasing with the increase of distance between users in SN 13
bull reciprocal relationship users - significantly larger similarity than users in a unidirectional relationship 13
bull traditional item-level similarity may be less reliable to find similar users in social bookmarking systems13
bull peers connected by self-defined social connections could be a useful source for cross-recommendation
Social Web 2014 Lora Aroyo
Conclusions
bull Input characteristics of items amp interests of a user into characteristics of items =gt Recommend items that feature characteristics which meet the userrsquos interests13
bull Techniques13
bull Data mining methods Cluster items based on their characteristics =gt Infer usersrsquo interests into clusters13
bull IR methods Represent items amp users as term vectors =gt Compute similarity between user profile vector and items13
bull Utility-based methods Utility function that gets an item as input the parameters of the utility function are customized via preferences of a user
Social Web 2014 Lora Aroyo
Content-based Recommendations
Government stops renovation of tower bridge Oct 13th 2011
Tower Bridge today Under construction
Tower Bridge is a combined bascule and suspension bridge in London England over the River Thames
Category politics england Related Twiper news bob Why do they stop tohellip [more] mary London stops renohellip [more]
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
02 0 0
02 04 01 01
= a
Weighting strategy - occurrence frequency - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
Content Features
RT Government stops renovation of tower bridge Oct 13th 2011
Userrsquos Twitter history
I am in London at the moment Oct 13th 2011
I am doing sports Oct 12th 2011
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
0 01 0
05 02 02 0
= u
Weighting strategy - occurrence frequency (eg smoothened by occurrence time recent concepts are more important - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
User Model
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
u 0
01 0
05 02 02 0
candidate items user a
02 0 0
02 04 01 01
b 0 0 0
08 02 0 0
c 0
05 02 0 0 0
03
cosine similarities
a b c
u 067 092 014
Ranking of recommended items 1 b 2 a 3 c
based on slides from Fabien Abel
Recommendations
Social Web 2014 Lora Aroyo
RecSys Issues
bull Cold-start problem (new user problem) nolittle data available to infer preferences of new users
bull Changing User Preferences user interests may change over time
bull Sparsity problem (new item problem) item descriptions are sparse eg not many user rated or tagged an item
bull Lack of Diversity (overfitting) when adapting too strongly to the preferences of users they might see samesimilar recommendations
bull Use the right context users do things which might not be relevant for their user model eg try out things do stuff for other people
bull Research challenge right balance between serendipity amp personalization
bull Research challenge right way to use the influence of recommendations on userrsquos behavior
one machine13vs 13
humans
image source httpwwwflickrcomphotosbionicteaching1375254387Social Web 2014 Lora Aroyo
Hands-on Teaser
bull Your Facebook Friendsrsquo popularity in a spread sheet13bull Locations of your Facebook Friends13bull Tag Cloud of your wall posts
Social Web 2014 Lora Aroyo
What is good user modelling amp personalisation
httpwwwflickrcomphotosbellarosebyliz4729613108
bull From the consumer perspective of an adaptive system 13
bull From the provider perspective of an adaptive system
Adaptive system maximizes satisfaction of the user
hard to measureobtain
Adaptive system maximizes the profit
influence of UM amp personalization may be hard to measureobtain
Social Web 2014 Lora Aroyo
Success Perspectives
bull User studies askobserve (selected) people whether you did a good job13
bull Log analysis Analyze (click) data and infer whether you did a good job13
bull Evaluation of user modeling13
bull measure quality of profiles directly eg measure overlap with existing (true) profiles or let people judge the quality of the generated user profiles 13
bull measure quality of application that exploits the user profile eg apply user modeling strategies in a recommender system
Social Web 2014 Lora Aroyo
Evaluation Strategies
time
item A
item B
item C
item D
item E
item G item H
item F
training data test data (ground truth)
Strategy X
User Modeling strategies to compare
Strategy Y
Strategy Z
Recommender
training data Recommendations
item H
X
item R
item M
Y
item H
item G
item N
item H
Z
item F
item M
measure quality
Social Web 2014 Lora Aroyo
Evaluating User Modeling in RecSys
Possible Metricsbull The usual IR metrics13
bull Precision fraction of retrieved items that are relevant13
bull Recall fraction of relevant items that have been retrieved13
bull F-Measure (harmonic) mean of precision and recall13
bull Metrics for evaluating recommendation (rankings)13
bull Mean Reciprocal Rank (MRR) of first relevant item13
bull Successk probability that relevant item occurs within the top k13
bull If a true ranking is given rank correlations 13
bull Precisionk Recallk amp F-Measurek13
bull Metrics for evaluating prediction of user preferences13
bull MAE = Mean Absolute Error13
bull TrueFalse PositivesNegatives runs
performance strategy X baseline
Is strategy X better than the baseline
Social Web 2014 Lora Aroyo
bull [Rae et al] a typical example of how to investigate and evaluate a proposal for improving (tag) recommendations (using social networks)13
bull Task test how well the different strategies (different tag contexts) can be used for tag predictionrecommendation13
bull Steps13
1 Gather a dataset of tag data part of which can be used as input and aim to test the recommendation on the remaining tag data13
2 Use the input data and calculate for the different strategies the predictions13
3 Measure the performance using standard (IR) metrics Precision of the top 5 recommended tags (P5) Mean Reciprocal Rank (MRR) Mean Average Precision (MAP)13
4 Test the results for statistical significance using T-test relative to the baseline (eg existing approach competitive approach)
[Rae et al Improving Tag Recommendations Using Social Networks RIAOrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
bull [Guy et al] another example of a similar evaluation approach13
bull The different strategies differ in the way people amp tags are used with tag-based systems there are complex relationships between users tags and items and strategies aim to find the relevant aspects of these relationships for modeling and recommendation13
bull The baseline is the lsquomost popularrsquo tags - often used to compare the most popular tags to the tags predicted by a particular personalization strategy - investigating whether the personalization is worth the effort and is able to outperform the easily available baseline
[Guy et al Social Media Recommendation based on People and Tags SIGIRrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
recommendation 13dimensions
Predict relevantusefulinteresting items13for a given user (in a given context)13
itrsquos often a ranking task
Social Web 2014 Lora Aroyo
Recommendation Systems
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
March 28 2013
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
commercial 13personalisation
httpwwwwiredcommagazine201111mf_artsyall1
Social Web 2014 Lora Aroyo
filter bubble
u1 likes u2
likes likes u1 likes Pulp Fiction
Social Web 2014 Lora Aroyo
Collaborative Filteringbull Memory-based User-Item matrix ratingspreferences of users =gt compute
similarity between users amp recommend items of similar users13
bull Model-based Item-Item matrix similarity (eg based on user ratings) between items =gt recommend items that are similar to the ones the user likes13
bull Model-based Clustering cluster users according to their preferences =gt recommend items of users that belong to the same cluster13
bull Model-based Bayesian networks P(u likes item B | u likes item A) = how likely is it that a user who likes item A will like item B learn probabilities from user ratingspreferences13
bull Others rule-based other data mining techniques13
bull complete input data is required13
bull pre-computation not possible13
bull does not scale well 13
bull high quality of recommendations13
bull abstraction (model) of input data13
bull pre-computation (partially) possible (model has to be re-built from time to time)13
bull scales better13
bull abstraction may reduce recommendation quality
Social Web 2014 Lora Aroyo
Memory vs Model-based
bull collaborative filtering lsquoneighborhoodsrsquo of people with similar interest amp recommending items based on likings in neighborhood13
bull limitations next to lsquocold startrsquo and lsquosparsityrsquo the lack of control (over onersquos neighborhood) is also a problem ie cannot add lsquotrustedrsquo people nor exclude lsquostrangersquo ones13
bull therefore interest in lsquosocial recommendersrsquo where presence of social connections defines the similarity in interests (eg social tagging CiteULike)13
bull does a social connection indicate user interest similarity13
bull how much users interest similarity depends on the strength of their connection13
bull is it feasible to use a social network as a personalized recommendation
[Lin amp Brusilovsky Social Networks and Interest Similarity The Case of CiteULike HTrsquo10]Social Web 2014 Lora Aroyo
Social Networks amp Interest Similarity
bull unilaterally connected pairs have more common itemsmetadatatags than non-connected pairs13
bull highest similarity for direct connections - decreasing with the increase of distance between users in SN 13
bull reciprocal relationship users - significantly larger similarity than users in a unidirectional relationship 13
bull traditional item-level similarity may be less reliable to find similar users in social bookmarking systems13
bull peers connected by self-defined social connections could be a useful source for cross-recommendation
Social Web 2014 Lora Aroyo
Conclusions
bull Input characteristics of items amp interests of a user into characteristics of items =gt Recommend items that feature characteristics which meet the userrsquos interests13
bull Techniques13
bull Data mining methods Cluster items based on their characteristics =gt Infer usersrsquo interests into clusters13
bull IR methods Represent items amp users as term vectors =gt Compute similarity between user profile vector and items13
bull Utility-based methods Utility function that gets an item as input the parameters of the utility function are customized via preferences of a user
Social Web 2014 Lora Aroyo
Content-based Recommendations
Government stops renovation of tower bridge Oct 13th 2011
Tower Bridge today Under construction
Tower Bridge is a combined bascule and suspension bridge in London England over the River Thames
Category politics england Related Twiper news bob Why do they stop tohellip [more] mary London stops renohellip [more]
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
02 0 0
02 04 01 01
= a
Weighting strategy - occurrence frequency - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
Content Features
RT Government stops renovation of tower bridge Oct 13th 2011
Userrsquos Twitter history
I am in London at the moment Oct 13th 2011
I am doing sports Oct 12th 2011
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
0 01 0
05 02 02 0
= u
Weighting strategy - occurrence frequency (eg smoothened by occurrence time recent concepts are more important - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
User Model
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
u 0
01 0
05 02 02 0
candidate items user a
02 0 0
02 04 01 01
b 0 0 0
08 02 0 0
c 0
05 02 0 0 0
03
cosine similarities
a b c
u 067 092 014
Ranking of recommended items 1 b 2 a 3 c
based on slides from Fabien Abel
Recommendations
Social Web 2014 Lora Aroyo
RecSys Issues
bull Cold-start problem (new user problem) nolittle data available to infer preferences of new users
bull Changing User Preferences user interests may change over time
bull Sparsity problem (new item problem) item descriptions are sparse eg not many user rated or tagged an item
bull Lack of Diversity (overfitting) when adapting too strongly to the preferences of users they might see samesimilar recommendations
bull Use the right context users do things which might not be relevant for their user model eg try out things do stuff for other people
bull Research challenge right balance between serendipity amp personalization
bull Research challenge right way to use the influence of recommendations on userrsquos behavior
one machine13vs 13
humans
image source httpwwwflickrcomphotosbionicteaching1375254387Social Web 2014 Lora Aroyo
Hands-on Teaser
bull Your Facebook Friendsrsquo popularity in a spread sheet13bull Locations of your Facebook Friends13bull Tag Cloud of your wall posts
bull From the consumer perspective of an adaptive system 13
bull From the provider perspective of an adaptive system
Adaptive system maximizes satisfaction of the user
hard to measureobtain
Adaptive system maximizes the profit
influence of UM amp personalization may be hard to measureobtain
Social Web 2014 Lora Aroyo
Success Perspectives
bull User studies askobserve (selected) people whether you did a good job13
bull Log analysis Analyze (click) data and infer whether you did a good job13
bull Evaluation of user modeling13
bull measure quality of profiles directly eg measure overlap with existing (true) profiles or let people judge the quality of the generated user profiles 13
bull measure quality of application that exploits the user profile eg apply user modeling strategies in a recommender system
Social Web 2014 Lora Aroyo
Evaluation Strategies
time
item A
item B
item C
item D
item E
item G item H
item F
training data test data (ground truth)
Strategy X
User Modeling strategies to compare
Strategy Y
Strategy Z
Recommender
training data Recommendations
item H
X
item R
item M
Y
item H
item G
item N
item H
Z
item F
item M
measure quality
Social Web 2014 Lora Aroyo
Evaluating User Modeling in RecSys
Possible Metricsbull The usual IR metrics13
bull Precision fraction of retrieved items that are relevant13
bull Recall fraction of relevant items that have been retrieved13
bull F-Measure (harmonic) mean of precision and recall13
bull Metrics for evaluating recommendation (rankings)13
bull Mean Reciprocal Rank (MRR) of first relevant item13
bull Successk probability that relevant item occurs within the top k13
bull If a true ranking is given rank correlations 13
bull Precisionk Recallk amp F-Measurek13
bull Metrics for evaluating prediction of user preferences13
bull MAE = Mean Absolute Error13
bull TrueFalse PositivesNegatives runs
performance strategy X baseline
Is strategy X better than the baseline
Social Web 2014 Lora Aroyo
bull [Rae et al] a typical example of how to investigate and evaluate a proposal for improving (tag) recommendations (using social networks)13
bull Task test how well the different strategies (different tag contexts) can be used for tag predictionrecommendation13
bull Steps13
1 Gather a dataset of tag data part of which can be used as input and aim to test the recommendation on the remaining tag data13
2 Use the input data and calculate for the different strategies the predictions13
3 Measure the performance using standard (IR) metrics Precision of the top 5 recommended tags (P5) Mean Reciprocal Rank (MRR) Mean Average Precision (MAP)13
4 Test the results for statistical significance using T-test relative to the baseline (eg existing approach competitive approach)
[Rae et al Improving Tag Recommendations Using Social Networks RIAOrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
bull [Guy et al] another example of a similar evaluation approach13
bull The different strategies differ in the way people amp tags are used with tag-based systems there are complex relationships between users tags and items and strategies aim to find the relevant aspects of these relationships for modeling and recommendation13
bull The baseline is the lsquomost popularrsquo tags - often used to compare the most popular tags to the tags predicted by a particular personalization strategy - investigating whether the personalization is worth the effort and is able to outperform the easily available baseline
[Guy et al Social Media Recommendation based on People and Tags SIGIRrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
recommendation 13dimensions
Predict relevantusefulinteresting items13for a given user (in a given context)13
itrsquos often a ranking task
Social Web 2014 Lora Aroyo
Recommendation Systems
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
March 28 2013
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
commercial 13personalisation
httpwwwwiredcommagazine201111mf_artsyall1
Social Web 2014 Lora Aroyo
filter bubble
u1 likes u2
likes likes u1 likes Pulp Fiction
Social Web 2014 Lora Aroyo
Collaborative Filteringbull Memory-based User-Item matrix ratingspreferences of users =gt compute
similarity between users amp recommend items of similar users13
bull Model-based Item-Item matrix similarity (eg based on user ratings) between items =gt recommend items that are similar to the ones the user likes13
bull Model-based Clustering cluster users according to their preferences =gt recommend items of users that belong to the same cluster13
bull Model-based Bayesian networks P(u likes item B | u likes item A) = how likely is it that a user who likes item A will like item B learn probabilities from user ratingspreferences13
bull Others rule-based other data mining techniques13
bull complete input data is required13
bull pre-computation not possible13
bull does not scale well 13
bull high quality of recommendations13
bull abstraction (model) of input data13
bull pre-computation (partially) possible (model has to be re-built from time to time)13
bull scales better13
bull abstraction may reduce recommendation quality
Social Web 2014 Lora Aroyo
Memory vs Model-based
bull collaborative filtering lsquoneighborhoodsrsquo of people with similar interest amp recommending items based on likings in neighborhood13
bull limitations next to lsquocold startrsquo and lsquosparsityrsquo the lack of control (over onersquos neighborhood) is also a problem ie cannot add lsquotrustedrsquo people nor exclude lsquostrangersquo ones13
bull therefore interest in lsquosocial recommendersrsquo where presence of social connections defines the similarity in interests (eg social tagging CiteULike)13
bull does a social connection indicate user interest similarity13
bull how much users interest similarity depends on the strength of their connection13
bull is it feasible to use a social network as a personalized recommendation
[Lin amp Brusilovsky Social Networks and Interest Similarity The Case of CiteULike HTrsquo10]Social Web 2014 Lora Aroyo
Social Networks amp Interest Similarity
bull unilaterally connected pairs have more common itemsmetadatatags than non-connected pairs13
bull highest similarity for direct connections - decreasing with the increase of distance between users in SN 13
bull reciprocal relationship users - significantly larger similarity than users in a unidirectional relationship 13
bull traditional item-level similarity may be less reliable to find similar users in social bookmarking systems13
bull peers connected by self-defined social connections could be a useful source for cross-recommendation
Social Web 2014 Lora Aroyo
Conclusions
bull Input characteristics of items amp interests of a user into characteristics of items =gt Recommend items that feature characteristics which meet the userrsquos interests13
bull Techniques13
bull Data mining methods Cluster items based on their characteristics =gt Infer usersrsquo interests into clusters13
bull IR methods Represent items amp users as term vectors =gt Compute similarity between user profile vector and items13
bull Utility-based methods Utility function that gets an item as input the parameters of the utility function are customized via preferences of a user
Social Web 2014 Lora Aroyo
Content-based Recommendations
Government stops renovation of tower bridge Oct 13th 2011
Tower Bridge today Under construction
Tower Bridge is a combined bascule and suspension bridge in London England over the River Thames
Category politics england Related Twiper news bob Why do they stop tohellip [more] mary London stops renohellip [more]
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
02 0 0
02 04 01 01
= a
Weighting strategy - occurrence frequency - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
Content Features
RT Government stops renovation of tower bridge Oct 13th 2011
Userrsquos Twitter history
I am in London at the moment Oct 13th 2011
I am doing sports Oct 12th 2011
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
0 01 0
05 02 02 0
= u
Weighting strategy - occurrence frequency (eg smoothened by occurrence time recent concepts are more important - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
User Model
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
u 0
01 0
05 02 02 0
candidate items user a
02 0 0
02 04 01 01
b 0 0 0
08 02 0 0
c 0
05 02 0 0 0
03
cosine similarities
a b c
u 067 092 014
Ranking of recommended items 1 b 2 a 3 c
based on slides from Fabien Abel
Recommendations
Social Web 2014 Lora Aroyo
RecSys Issues
bull Cold-start problem (new user problem) nolittle data available to infer preferences of new users
bull Changing User Preferences user interests may change over time
bull Sparsity problem (new item problem) item descriptions are sparse eg not many user rated or tagged an item
bull Lack of Diversity (overfitting) when adapting too strongly to the preferences of users they might see samesimilar recommendations
bull Use the right context users do things which might not be relevant for their user model eg try out things do stuff for other people
bull Research challenge right balance between serendipity amp personalization
bull Research challenge right way to use the influence of recommendations on userrsquos behavior
one machine13vs 13
humans
image source httpwwwflickrcomphotosbionicteaching1375254387Social Web 2014 Lora Aroyo
Hands-on Teaser
bull Your Facebook Friendsrsquo popularity in a spread sheet13bull Locations of your Facebook Friends13bull Tag Cloud of your wall posts
bull User studies askobserve (selected) people whether you did a good job13
bull Log analysis Analyze (click) data and infer whether you did a good job13
bull Evaluation of user modeling13
bull measure quality of profiles directly eg measure overlap with existing (true) profiles or let people judge the quality of the generated user profiles 13
bull measure quality of application that exploits the user profile eg apply user modeling strategies in a recommender system
Social Web 2014 Lora Aroyo
Evaluation Strategies
time
item A
item B
item C
item D
item E
item G item H
item F
training data test data (ground truth)
Strategy X
User Modeling strategies to compare
Strategy Y
Strategy Z
Recommender
training data Recommendations
item H
X
item R
item M
Y
item H
item G
item N
item H
Z
item F
item M
measure quality
Social Web 2014 Lora Aroyo
Evaluating User Modeling in RecSys
Possible Metricsbull The usual IR metrics13
bull Precision fraction of retrieved items that are relevant13
bull Recall fraction of relevant items that have been retrieved13
bull F-Measure (harmonic) mean of precision and recall13
bull Metrics for evaluating recommendation (rankings)13
bull Mean Reciprocal Rank (MRR) of first relevant item13
bull Successk probability that relevant item occurs within the top k13
bull If a true ranking is given rank correlations 13
bull Precisionk Recallk amp F-Measurek13
bull Metrics for evaluating prediction of user preferences13
bull MAE = Mean Absolute Error13
bull TrueFalse PositivesNegatives runs
performance strategy X baseline
Is strategy X better than the baseline
Social Web 2014 Lora Aroyo
bull [Rae et al] a typical example of how to investigate and evaluate a proposal for improving (tag) recommendations (using social networks)13
bull Task test how well the different strategies (different tag contexts) can be used for tag predictionrecommendation13
bull Steps13
1 Gather a dataset of tag data part of which can be used as input and aim to test the recommendation on the remaining tag data13
2 Use the input data and calculate for the different strategies the predictions13
3 Measure the performance using standard (IR) metrics Precision of the top 5 recommended tags (P5) Mean Reciprocal Rank (MRR) Mean Average Precision (MAP)13
4 Test the results for statistical significance using T-test relative to the baseline (eg existing approach competitive approach)
[Rae et al Improving Tag Recommendations Using Social Networks RIAOrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
bull [Guy et al] another example of a similar evaluation approach13
bull The different strategies differ in the way people amp tags are used with tag-based systems there are complex relationships between users tags and items and strategies aim to find the relevant aspects of these relationships for modeling and recommendation13
bull The baseline is the lsquomost popularrsquo tags - often used to compare the most popular tags to the tags predicted by a particular personalization strategy - investigating whether the personalization is worth the effort and is able to outperform the easily available baseline
[Guy et al Social Media Recommendation based on People and Tags SIGIRrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
recommendation 13dimensions
Predict relevantusefulinteresting items13for a given user (in a given context)13
itrsquos often a ranking task
Social Web 2014 Lora Aroyo
Recommendation Systems
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
March 28 2013
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
commercial 13personalisation
httpwwwwiredcommagazine201111mf_artsyall1
Social Web 2014 Lora Aroyo
filter bubble
u1 likes u2
likes likes u1 likes Pulp Fiction
Social Web 2014 Lora Aroyo
Collaborative Filteringbull Memory-based User-Item matrix ratingspreferences of users =gt compute
similarity between users amp recommend items of similar users13
bull Model-based Item-Item matrix similarity (eg based on user ratings) between items =gt recommend items that are similar to the ones the user likes13
bull Model-based Clustering cluster users according to their preferences =gt recommend items of users that belong to the same cluster13
bull Model-based Bayesian networks P(u likes item B | u likes item A) = how likely is it that a user who likes item A will like item B learn probabilities from user ratingspreferences13
bull Others rule-based other data mining techniques13
bull complete input data is required13
bull pre-computation not possible13
bull does not scale well 13
bull high quality of recommendations13
bull abstraction (model) of input data13
bull pre-computation (partially) possible (model has to be re-built from time to time)13
bull scales better13
bull abstraction may reduce recommendation quality
Social Web 2014 Lora Aroyo
Memory vs Model-based
bull collaborative filtering lsquoneighborhoodsrsquo of people with similar interest amp recommending items based on likings in neighborhood13
bull limitations next to lsquocold startrsquo and lsquosparsityrsquo the lack of control (over onersquos neighborhood) is also a problem ie cannot add lsquotrustedrsquo people nor exclude lsquostrangersquo ones13
bull therefore interest in lsquosocial recommendersrsquo where presence of social connections defines the similarity in interests (eg social tagging CiteULike)13
bull does a social connection indicate user interest similarity13
bull how much users interest similarity depends on the strength of their connection13
bull is it feasible to use a social network as a personalized recommendation
[Lin amp Brusilovsky Social Networks and Interest Similarity The Case of CiteULike HTrsquo10]Social Web 2014 Lora Aroyo
Social Networks amp Interest Similarity
bull unilaterally connected pairs have more common itemsmetadatatags than non-connected pairs13
bull highest similarity for direct connections - decreasing with the increase of distance between users in SN 13
bull reciprocal relationship users - significantly larger similarity than users in a unidirectional relationship 13
bull traditional item-level similarity may be less reliable to find similar users in social bookmarking systems13
bull peers connected by self-defined social connections could be a useful source for cross-recommendation
Social Web 2014 Lora Aroyo
Conclusions
bull Input characteristics of items amp interests of a user into characteristics of items =gt Recommend items that feature characteristics which meet the userrsquos interests13
bull Techniques13
bull Data mining methods Cluster items based on their characteristics =gt Infer usersrsquo interests into clusters13
bull IR methods Represent items amp users as term vectors =gt Compute similarity between user profile vector and items13
bull Utility-based methods Utility function that gets an item as input the parameters of the utility function are customized via preferences of a user
Social Web 2014 Lora Aroyo
Content-based Recommendations
Government stops renovation of tower bridge Oct 13th 2011
Tower Bridge today Under construction
Tower Bridge is a combined bascule and suspension bridge in London England over the River Thames
Category politics england Related Twiper news bob Why do they stop tohellip [more] mary London stops renohellip [more]
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
02 0 0
02 04 01 01
= a
Weighting strategy - occurrence frequency - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
Content Features
RT Government stops renovation of tower bridge Oct 13th 2011
Userrsquos Twitter history
I am in London at the moment Oct 13th 2011
I am doing sports Oct 12th 2011
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
0 01 0
05 02 02 0
= u
Weighting strategy - occurrence frequency (eg smoothened by occurrence time recent concepts are more important - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
User Model
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
u 0
01 0
05 02 02 0
candidate items user a
02 0 0
02 04 01 01
b 0 0 0
08 02 0 0
c 0
05 02 0 0 0
03
cosine similarities
a b c
u 067 092 014
Ranking of recommended items 1 b 2 a 3 c
based on slides from Fabien Abel
Recommendations
Social Web 2014 Lora Aroyo
RecSys Issues
bull Cold-start problem (new user problem) nolittle data available to infer preferences of new users
bull Changing User Preferences user interests may change over time
bull Sparsity problem (new item problem) item descriptions are sparse eg not many user rated or tagged an item
bull Lack of Diversity (overfitting) when adapting too strongly to the preferences of users they might see samesimilar recommendations
bull Use the right context users do things which might not be relevant for their user model eg try out things do stuff for other people
bull Research challenge right balance between serendipity amp personalization
bull Research challenge right way to use the influence of recommendations on userrsquos behavior
one machine13vs 13
humans
image source httpwwwflickrcomphotosbionicteaching1375254387Social Web 2014 Lora Aroyo
Hands-on Teaser
bull Your Facebook Friendsrsquo popularity in a spread sheet13bull Locations of your Facebook Friends13bull Tag Cloud of your wall posts
time
item A
item B
item C
item D
item E
item G item H
item F
training data test data (ground truth)
Strategy X
User Modeling strategies to compare
Strategy Y
Strategy Z
Recommender
training data Recommendations
item H
X
item R
item M
Y
item H
item G
item N
item H
Z
item F
item M
measure quality
Social Web 2014 Lora Aroyo
Evaluating User Modeling in RecSys
Possible Metricsbull The usual IR metrics13
bull Precision fraction of retrieved items that are relevant13
bull Recall fraction of relevant items that have been retrieved13
bull F-Measure (harmonic) mean of precision and recall13
bull Metrics for evaluating recommendation (rankings)13
bull Mean Reciprocal Rank (MRR) of first relevant item13
bull Successk probability that relevant item occurs within the top k13
bull If a true ranking is given rank correlations 13
bull Precisionk Recallk amp F-Measurek13
bull Metrics for evaluating prediction of user preferences13
bull MAE = Mean Absolute Error13
bull TrueFalse PositivesNegatives runs
performance strategy X baseline
Is strategy X better than the baseline
Social Web 2014 Lora Aroyo
bull [Rae et al] a typical example of how to investigate and evaluate a proposal for improving (tag) recommendations (using social networks)13
bull Task test how well the different strategies (different tag contexts) can be used for tag predictionrecommendation13
bull Steps13
1 Gather a dataset of tag data part of which can be used as input and aim to test the recommendation on the remaining tag data13
2 Use the input data and calculate for the different strategies the predictions13
3 Measure the performance using standard (IR) metrics Precision of the top 5 recommended tags (P5) Mean Reciprocal Rank (MRR) Mean Average Precision (MAP)13
4 Test the results for statistical significance using T-test relative to the baseline (eg existing approach competitive approach)
[Rae et al Improving Tag Recommendations Using Social Networks RIAOrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
bull [Guy et al] another example of a similar evaluation approach13
bull The different strategies differ in the way people amp tags are used with tag-based systems there are complex relationships between users tags and items and strategies aim to find the relevant aspects of these relationships for modeling and recommendation13
bull The baseline is the lsquomost popularrsquo tags - often used to compare the most popular tags to the tags predicted by a particular personalization strategy - investigating whether the personalization is worth the effort and is able to outperform the easily available baseline
[Guy et al Social Media Recommendation based on People and Tags SIGIRrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
recommendation 13dimensions
Predict relevantusefulinteresting items13for a given user (in a given context)13
itrsquos often a ranking task
Social Web 2014 Lora Aroyo
Recommendation Systems
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
March 28 2013
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
commercial 13personalisation
httpwwwwiredcommagazine201111mf_artsyall1
Social Web 2014 Lora Aroyo
filter bubble
u1 likes u2
likes likes u1 likes Pulp Fiction
Social Web 2014 Lora Aroyo
Collaborative Filteringbull Memory-based User-Item matrix ratingspreferences of users =gt compute
similarity between users amp recommend items of similar users13
bull Model-based Item-Item matrix similarity (eg based on user ratings) between items =gt recommend items that are similar to the ones the user likes13
bull Model-based Clustering cluster users according to their preferences =gt recommend items of users that belong to the same cluster13
bull Model-based Bayesian networks P(u likes item B | u likes item A) = how likely is it that a user who likes item A will like item B learn probabilities from user ratingspreferences13
bull Others rule-based other data mining techniques13
bull complete input data is required13
bull pre-computation not possible13
bull does not scale well 13
bull high quality of recommendations13
bull abstraction (model) of input data13
bull pre-computation (partially) possible (model has to be re-built from time to time)13
bull scales better13
bull abstraction may reduce recommendation quality
Social Web 2014 Lora Aroyo
Memory vs Model-based
bull collaborative filtering lsquoneighborhoodsrsquo of people with similar interest amp recommending items based on likings in neighborhood13
bull limitations next to lsquocold startrsquo and lsquosparsityrsquo the lack of control (over onersquos neighborhood) is also a problem ie cannot add lsquotrustedrsquo people nor exclude lsquostrangersquo ones13
bull therefore interest in lsquosocial recommendersrsquo where presence of social connections defines the similarity in interests (eg social tagging CiteULike)13
bull does a social connection indicate user interest similarity13
bull how much users interest similarity depends on the strength of their connection13
bull is it feasible to use a social network as a personalized recommendation
[Lin amp Brusilovsky Social Networks and Interest Similarity The Case of CiteULike HTrsquo10]Social Web 2014 Lora Aroyo
Social Networks amp Interest Similarity
bull unilaterally connected pairs have more common itemsmetadatatags than non-connected pairs13
bull highest similarity for direct connections - decreasing with the increase of distance between users in SN 13
bull reciprocal relationship users - significantly larger similarity than users in a unidirectional relationship 13
bull traditional item-level similarity may be less reliable to find similar users in social bookmarking systems13
bull peers connected by self-defined social connections could be a useful source for cross-recommendation
Social Web 2014 Lora Aroyo
Conclusions
bull Input characteristics of items amp interests of a user into characteristics of items =gt Recommend items that feature characteristics which meet the userrsquos interests13
bull Techniques13
bull Data mining methods Cluster items based on their characteristics =gt Infer usersrsquo interests into clusters13
bull IR methods Represent items amp users as term vectors =gt Compute similarity between user profile vector and items13
bull Utility-based methods Utility function that gets an item as input the parameters of the utility function are customized via preferences of a user
Social Web 2014 Lora Aroyo
Content-based Recommendations
Government stops renovation of tower bridge Oct 13th 2011
Tower Bridge today Under construction
Tower Bridge is a combined bascule and suspension bridge in London England over the River Thames
Category politics england Related Twiper news bob Why do they stop tohellip [more] mary London stops renohellip [more]
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
02 0 0
02 04 01 01
= a
Weighting strategy - occurrence frequency - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
Content Features
RT Government stops renovation of tower bridge Oct 13th 2011
Userrsquos Twitter history
I am in London at the moment Oct 13th 2011
I am doing sports Oct 12th 2011
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
0 01 0
05 02 02 0
= u
Weighting strategy - occurrence frequency (eg smoothened by occurrence time recent concepts are more important - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
User Model
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
u 0
01 0
05 02 02 0
candidate items user a
02 0 0
02 04 01 01
b 0 0 0
08 02 0 0
c 0
05 02 0 0 0
03
cosine similarities
a b c
u 067 092 014
Ranking of recommended items 1 b 2 a 3 c
based on slides from Fabien Abel
Recommendations
Social Web 2014 Lora Aroyo
RecSys Issues
bull Cold-start problem (new user problem) nolittle data available to infer preferences of new users
bull Changing User Preferences user interests may change over time
bull Sparsity problem (new item problem) item descriptions are sparse eg not many user rated or tagged an item
bull Lack of Diversity (overfitting) when adapting too strongly to the preferences of users they might see samesimilar recommendations
bull Use the right context users do things which might not be relevant for their user model eg try out things do stuff for other people
bull Research challenge right balance between serendipity amp personalization
bull Research challenge right way to use the influence of recommendations on userrsquos behavior
one machine13vs 13
humans
image source httpwwwflickrcomphotosbionicteaching1375254387Social Web 2014 Lora Aroyo
Hands-on Teaser
bull Your Facebook Friendsrsquo popularity in a spread sheet13bull Locations of your Facebook Friends13bull Tag Cloud of your wall posts
Possible Metricsbull The usual IR metrics13
bull Precision fraction of retrieved items that are relevant13
bull Recall fraction of relevant items that have been retrieved13
bull F-Measure (harmonic) mean of precision and recall13
bull Metrics for evaluating recommendation (rankings)13
bull Mean Reciprocal Rank (MRR) of first relevant item13
bull Successk probability that relevant item occurs within the top k13
bull If a true ranking is given rank correlations 13
bull Precisionk Recallk amp F-Measurek13
bull Metrics for evaluating prediction of user preferences13
bull MAE = Mean Absolute Error13
bull TrueFalse PositivesNegatives runs
performance strategy X baseline
Is strategy X better than the baseline
Social Web 2014 Lora Aroyo
bull [Rae et al] a typical example of how to investigate and evaluate a proposal for improving (tag) recommendations (using social networks)13
bull Task test how well the different strategies (different tag contexts) can be used for tag predictionrecommendation13
bull Steps13
1 Gather a dataset of tag data part of which can be used as input and aim to test the recommendation on the remaining tag data13
2 Use the input data and calculate for the different strategies the predictions13
3 Measure the performance using standard (IR) metrics Precision of the top 5 recommended tags (P5) Mean Reciprocal Rank (MRR) Mean Average Precision (MAP)13
4 Test the results for statistical significance using T-test relative to the baseline (eg existing approach competitive approach)
[Rae et al Improving Tag Recommendations Using Social Networks RIAOrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
bull [Guy et al] another example of a similar evaluation approach13
bull The different strategies differ in the way people amp tags are used with tag-based systems there are complex relationships between users tags and items and strategies aim to find the relevant aspects of these relationships for modeling and recommendation13
bull The baseline is the lsquomost popularrsquo tags - often used to compare the most popular tags to the tags predicted by a particular personalization strategy - investigating whether the personalization is worth the effort and is able to outperform the easily available baseline
[Guy et al Social Media Recommendation based on People and Tags SIGIRrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
recommendation 13dimensions
Predict relevantusefulinteresting items13for a given user (in a given context)13
itrsquos often a ranking task
Social Web 2014 Lora Aroyo
Recommendation Systems
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
March 28 2013
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
commercial 13personalisation
httpwwwwiredcommagazine201111mf_artsyall1
Social Web 2014 Lora Aroyo
filter bubble
u1 likes u2
likes likes u1 likes Pulp Fiction
Social Web 2014 Lora Aroyo
Collaborative Filteringbull Memory-based User-Item matrix ratingspreferences of users =gt compute
similarity between users amp recommend items of similar users13
bull Model-based Item-Item matrix similarity (eg based on user ratings) between items =gt recommend items that are similar to the ones the user likes13
bull Model-based Clustering cluster users according to their preferences =gt recommend items of users that belong to the same cluster13
bull Model-based Bayesian networks P(u likes item B | u likes item A) = how likely is it that a user who likes item A will like item B learn probabilities from user ratingspreferences13
bull Others rule-based other data mining techniques13
bull complete input data is required13
bull pre-computation not possible13
bull does not scale well 13
bull high quality of recommendations13
bull abstraction (model) of input data13
bull pre-computation (partially) possible (model has to be re-built from time to time)13
bull scales better13
bull abstraction may reduce recommendation quality
Social Web 2014 Lora Aroyo
Memory vs Model-based
bull collaborative filtering lsquoneighborhoodsrsquo of people with similar interest amp recommending items based on likings in neighborhood13
bull limitations next to lsquocold startrsquo and lsquosparsityrsquo the lack of control (over onersquos neighborhood) is also a problem ie cannot add lsquotrustedrsquo people nor exclude lsquostrangersquo ones13
bull therefore interest in lsquosocial recommendersrsquo where presence of social connections defines the similarity in interests (eg social tagging CiteULike)13
bull does a social connection indicate user interest similarity13
bull how much users interest similarity depends on the strength of their connection13
bull is it feasible to use a social network as a personalized recommendation
[Lin amp Brusilovsky Social Networks and Interest Similarity The Case of CiteULike HTrsquo10]Social Web 2014 Lora Aroyo
Social Networks amp Interest Similarity
bull unilaterally connected pairs have more common itemsmetadatatags than non-connected pairs13
bull highest similarity for direct connections - decreasing with the increase of distance between users in SN 13
bull reciprocal relationship users - significantly larger similarity than users in a unidirectional relationship 13
bull traditional item-level similarity may be less reliable to find similar users in social bookmarking systems13
bull peers connected by self-defined social connections could be a useful source for cross-recommendation
Social Web 2014 Lora Aroyo
Conclusions
bull Input characteristics of items amp interests of a user into characteristics of items =gt Recommend items that feature characteristics which meet the userrsquos interests13
bull Techniques13
bull Data mining methods Cluster items based on their characteristics =gt Infer usersrsquo interests into clusters13
bull IR methods Represent items amp users as term vectors =gt Compute similarity between user profile vector and items13
bull Utility-based methods Utility function that gets an item as input the parameters of the utility function are customized via preferences of a user
Social Web 2014 Lora Aroyo
Content-based Recommendations
Government stops renovation of tower bridge Oct 13th 2011
Tower Bridge today Under construction
Tower Bridge is a combined bascule and suspension bridge in London England over the River Thames
Category politics england Related Twiper news bob Why do they stop tohellip [more] mary London stops renohellip [more]
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
02 0 0
02 04 01 01
= a
Weighting strategy - occurrence frequency - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
Content Features
RT Government stops renovation of tower bridge Oct 13th 2011
Userrsquos Twitter history
I am in London at the moment Oct 13th 2011
I am doing sports Oct 12th 2011
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
0 01 0
05 02 02 0
= u
Weighting strategy - occurrence frequency (eg smoothened by occurrence time recent concepts are more important - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
User Model
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
u 0
01 0
05 02 02 0
candidate items user a
02 0 0
02 04 01 01
b 0 0 0
08 02 0 0
c 0
05 02 0 0 0
03
cosine similarities
a b c
u 067 092 014
Ranking of recommended items 1 b 2 a 3 c
based on slides from Fabien Abel
Recommendations
Social Web 2014 Lora Aroyo
RecSys Issues
bull Cold-start problem (new user problem) nolittle data available to infer preferences of new users
bull Changing User Preferences user interests may change over time
bull Sparsity problem (new item problem) item descriptions are sparse eg not many user rated or tagged an item
bull Lack of Diversity (overfitting) when adapting too strongly to the preferences of users they might see samesimilar recommendations
bull Use the right context users do things which might not be relevant for their user model eg try out things do stuff for other people
bull Research challenge right balance between serendipity amp personalization
bull Research challenge right way to use the influence of recommendations on userrsquos behavior
one machine13vs 13
humans
image source httpwwwflickrcomphotosbionicteaching1375254387Social Web 2014 Lora Aroyo
Hands-on Teaser
bull Your Facebook Friendsrsquo popularity in a spread sheet13bull Locations of your Facebook Friends13bull Tag Cloud of your wall posts
bull [Rae et al] a typical example of how to investigate and evaluate a proposal for improving (tag) recommendations (using social networks)13
bull Task test how well the different strategies (different tag contexts) can be used for tag predictionrecommendation13
bull Steps13
1 Gather a dataset of tag data part of which can be used as input and aim to test the recommendation on the remaining tag data13
2 Use the input data and calculate for the different strategies the predictions13
3 Measure the performance using standard (IR) metrics Precision of the top 5 recommended tags (P5) Mean Reciprocal Rank (MRR) Mean Average Precision (MAP)13
4 Test the results for statistical significance using T-test relative to the baseline (eg existing approach competitive approach)
[Rae et al Improving Tag Recommendations Using Social Networks RIAOrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
bull [Guy et al] another example of a similar evaluation approach13
bull The different strategies differ in the way people amp tags are used with tag-based systems there are complex relationships between users tags and items and strategies aim to find the relevant aspects of these relationships for modeling and recommendation13
bull The baseline is the lsquomost popularrsquo tags - often used to compare the most popular tags to the tags predicted by a particular personalization strategy - investigating whether the personalization is worth the effort and is able to outperform the easily available baseline
[Guy et al Social Media Recommendation based on People and Tags SIGIRrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
recommendation 13dimensions
Predict relevantusefulinteresting items13for a given user (in a given context)13
itrsquos often a ranking task
Social Web 2014 Lora Aroyo
Recommendation Systems
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
March 28 2013
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
commercial 13personalisation
httpwwwwiredcommagazine201111mf_artsyall1
Social Web 2014 Lora Aroyo
filter bubble
u1 likes u2
likes likes u1 likes Pulp Fiction
Social Web 2014 Lora Aroyo
Collaborative Filteringbull Memory-based User-Item matrix ratingspreferences of users =gt compute
similarity between users amp recommend items of similar users13
bull Model-based Item-Item matrix similarity (eg based on user ratings) between items =gt recommend items that are similar to the ones the user likes13
bull Model-based Clustering cluster users according to their preferences =gt recommend items of users that belong to the same cluster13
bull Model-based Bayesian networks P(u likes item B | u likes item A) = how likely is it that a user who likes item A will like item B learn probabilities from user ratingspreferences13
bull Others rule-based other data mining techniques13
bull complete input data is required13
bull pre-computation not possible13
bull does not scale well 13
bull high quality of recommendations13
bull abstraction (model) of input data13
bull pre-computation (partially) possible (model has to be re-built from time to time)13
bull scales better13
bull abstraction may reduce recommendation quality
Social Web 2014 Lora Aroyo
Memory vs Model-based
bull collaborative filtering lsquoneighborhoodsrsquo of people with similar interest amp recommending items based on likings in neighborhood13
bull limitations next to lsquocold startrsquo and lsquosparsityrsquo the lack of control (over onersquos neighborhood) is also a problem ie cannot add lsquotrustedrsquo people nor exclude lsquostrangersquo ones13
bull therefore interest in lsquosocial recommendersrsquo where presence of social connections defines the similarity in interests (eg social tagging CiteULike)13
bull does a social connection indicate user interest similarity13
bull how much users interest similarity depends on the strength of their connection13
bull is it feasible to use a social network as a personalized recommendation
[Lin amp Brusilovsky Social Networks and Interest Similarity The Case of CiteULike HTrsquo10]Social Web 2014 Lora Aroyo
Social Networks amp Interest Similarity
bull unilaterally connected pairs have more common itemsmetadatatags than non-connected pairs13
bull highest similarity for direct connections - decreasing with the increase of distance between users in SN 13
bull reciprocal relationship users - significantly larger similarity than users in a unidirectional relationship 13
bull traditional item-level similarity may be less reliable to find similar users in social bookmarking systems13
bull peers connected by self-defined social connections could be a useful source for cross-recommendation
Social Web 2014 Lora Aroyo
Conclusions
bull Input characteristics of items amp interests of a user into characteristics of items =gt Recommend items that feature characteristics which meet the userrsquos interests13
bull Techniques13
bull Data mining methods Cluster items based on their characteristics =gt Infer usersrsquo interests into clusters13
bull IR methods Represent items amp users as term vectors =gt Compute similarity between user profile vector and items13
bull Utility-based methods Utility function that gets an item as input the parameters of the utility function are customized via preferences of a user
Social Web 2014 Lora Aroyo
Content-based Recommendations
Government stops renovation of tower bridge Oct 13th 2011
Tower Bridge today Under construction
Tower Bridge is a combined bascule and suspension bridge in London England over the River Thames
Category politics england Related Twiper news bob Why do they stop tohellip [more] mary London stops renohellip [more]
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
02 0 0
02 04 01 01
= a
Weighting strategy - occurrence frequency - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
Content Features
RT Government stops renovation of tower bridge Oct 13th 2011
Userrsquos Twitter history
I am in London at the moment Oct 13th 2011
I am doing sports Oct 12th 2011
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
0 01 0
05 02 02 0
= u
Weighting strategy - occurrence frequency (eg smoothened by occurrence time recent concepts are more important - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
User Model
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
u 0
01 0
05 02 02 0
candidate items user a
02 0 0
02 04 01 01
b 0 0 0
08 02 0 0
c 0
05 02 0 0 0
03
cosine similarities
a b c
u 067 092 014
Ranking of recommended items 1 b 2 a 3 c
based on slides from Fabien Abel
Recommendations
Social Web 2014 Lora Aroyo
RecSys Issues
bull Cold-start problem (new user problem) nolittle data available to infer preferences of new users
bull Changing User Preferences user interests may change over time
bull Sparsity problem (new item problem) item descriptions are sparse eg not many user rated or tagged an item
bull Lack of Diversity (overfitting) when adapting too strongly to the preferences of users they might see samesimilar recommendations
bull Use the right context users do things which might not be relevant for their user model eg try out things do stuff for other people
bull Research challenge right balance between serendipity amp personalization
bull Research challenge right way to use the influence of recommendations on userrsquos behavior
one machine13vs 13
humans
image source httpwwwflickrcomphotosbionicteaching1375254387Social Web 2014 Lora Aroyo
Hands-on Teaser
bull Your Facebook Friendsrsquo popularity in a spread sheet13bull Locations of your Facebook Friends13bull Tag Cloud of your wall posts
bull [Guy et al] another example of a similar evaluation approach13
bull The different strategies differ in the way people amp tags are used with tag-based systems there are complex relationships between users tags and items and strategies aim to find the relevant aspects of these relationships for modeling and recommendation13
bull The baseline is the lsquomost popularrsquo tags - often used to compare the most popular tags to the tags predicted by a particular personalization strategy - investigating whether the personalization is worth the effort and is able to outperform the easily available baseline
[Guy et al Social Media Recommendation based on People and Tags SIGIRrsquo10]]
Social Web 2014 Lora Aroyo
Example Evaluation
recommendation 13dimensions
Predict relevantusefulinteresting items13for a given user (in a given context)13
itrsquos often a ranking task
Social Web 2014 Lora Aroyo
Recommendation Systems
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
March 28 2013
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
commercial 13personalisation
httpwwwwiredcommagazine201111mf_artsyall1
Social Web 2014 Lora Aroyo
filter bubble
u1 likes u2
likes likes u1 likes Pulp Fiction
Social Web 2014 Lora Aroyo
Collaborative Filteringbull Memory-based User-Item matrix ratingspreferences of users =gt compute
similarity between users amp recommend items of similar users13
bull Model-based Item-Item matrix similarity (eg based on user ratings) between items =gt recommend items that are similar to the ones the user likes13
bull Model-based Clustering cluster users according to their preferences =gt recommend items of users that belong to the same cluster13
bull Model-based Bayesian networks P(u likes item B | u likes item A) = how likely is it that a user who likes item A will like item B learn probabilities from user ratingspreferences13
bull Others rule-based other data mining techniques13
bull complete input data is required13
bull pre-computation not possible13
bull does not scale well 13
bull high quality of recommendations13
bull abstraction (model) of input data13
bull pre-computation (partially) possible (model has to be re-built from time to time)13
bull scales better13
bull abstraction may reduce recommendation quality
Social Web 2014 Lora Aroyo
Memory vs Model-based
bull collaborative filtering lsquoneighborhoodsrsquo of people with similar interest amp recommending items based on likings in neighborhood13
bull limitations next to lsquocold startrsquo and lsquosparsityrsquo the lack of control (over onersquos neighborhood) is also a problem ie cannot add lsquotrustedrsquo people nor exclude lsquostrangersquo ones13
bull therefore interest in lsquosocial recommendersrsquo where presence of social connections defines the similarity in interests (eg social tagging CiteULike)13
bull does a social connection indicate user interest similarity13
bull how much users interest similarity depends on the strength of their connection13
bull is it feasible to use a social network as a personalized recommendation
[Lin amp Brusilovsky Social Networks and Interest Similarity The Case of CiteULike HTrsquo10]Social Web 2014 Lora Aroyo
Social Networks amp Interest Similarity
bull unilaterally connected pairs have more common itemsmetadatatags than non-connected pairs13
bull highest similarity for direct connections - decreasing with the increase of distance between users in SN 13
bull reciprocal relationship users - significantly larger similarity than users in a unidirectional relationship 13
bull traditional item-level similarity may be less reliable to find similar users in social bookmarking systems13
bull peers connected by self-defined social connections could be a useful source for cross-recommendation
Social Web 2014 Lora Aroyo
Conclusions
bull Input characteristics of items amp interests of a user into characteristics of items =gt Recommend items that feature characteristics which meet the userrsquos interests13
bull Techniques13
bull Data mining methods Cluster items based on their characteristics =gt Infer usersrsquo interests into clusters13
bull IR methods Represent items amp users as term vectors =gt Compute similarity between user profile vector and items13
bull Utility-based methods Utility function that gets an item as input the parameters of the utility function are customized via preferences of a user
Social Web 2014 Lora Aroyo
Content-based Recommendations
Government stops renovation of tower bridge Oct 13th 2011
Tower Bridge today Under construction
Tower Bridge is a combined bascule and suspension bridge in London England over the River Thames
Category politics england Related Twiper news bob Why do they stop tohellip [more] mary London stops renohellip [more]
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
02 0 0
02 04 01 01
= a
Weighting strategy - occurrence frequency - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
Content Features
RT Government stops renovation of tower bridge Oct 13th 2011
Userrsquos Twitter history
I am in London at the moment Oct 13th 2011
I am doing sports Oct 12th 2011
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
0 01 0
05 02 02 0
= u
Weighting strategy - occurrence frequency (eg smoothened by occurrence time recent concepts are more important - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
User Model
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
u 0
01 0
05 02 02 0
candidate items user a
02 0 0
02 04 01 01
b 0 0 0
08 02 0 0
c 0
05 02 0 0 0
03
cosine similarities
a b c
u 067 092 014
Ranking of recommended items 1 b 2 a 3 c
based on slides from Fabien Abel
Recommendations
Social Web 2014 Lora Aroyo
RecSys Issues
bull Cold-start problem (new user problem) nolittle data available to infer preferences of new users
bull Changing User Preferences user interests may change over time
bull Sparsity problem (new item problem) item descriptions are sparse eg not many user rated or tagged an item
bull Lack of Diversity (overfitting) when adapting too strongly to the preferences of users they might see samesimilar recommendations
bull Use the right context users do things which might not be relevant for their user model eg try out things do stuff for other people
bull Research challenge right balance between serendipity amp personalization
bull Research challenge right way to use the influence of recommendations on userrsquos behavior
one machine13vs 13
humans
image source httpwwwflickrcomphotosbionicteaching1375254387Social Web 2014 Lora Aroyo
Hands-on Teaser
bull Your Facebook Friendsrsquo popularity in a spread sheet13bull Locations of your Facebook Friends13bull Tag Cloud of your wall posts
recommendation 13dimensions
Predict relevantusefulinteresting items13for a given user (in a given context)13
itrsquos often a ranking task
Social Web 2014 Lora Aroyo
Recommendation Systems
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
March 28 2013
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
commercial 13personalisation
httpwwwwiredcommagazine201111mf_artsyall1
Social Web 2014 Lora Aroyo
filter bubble
u1 likes u2
likes likes u1 likes Pulp Fiction
Social Web 2014 Lora Aroyo
Collaborative Filteringbull Memory-based User-Item matrix ratingspreferences of users =gt compute
similarity between users amp recommend items of similar users13
bull Model-based Item-Item matrix similarity (eg based on user ratings) between items =gt recommend items that are similar to the ones the user likes13
bull Model-based Clustering cluster users according to their preferences =gt recommend items of users that belong to the same cluster13
bull Model-based Bayesian networks P(u likes item B | u likes item A) = how likely is it that a user who likes item A will like item B learn probabilities from user ratingspreferences13
bull Others rule-based other data mining techniques13
bull complete input data is required13
bull pre-computation not possible13
bull does not scale well 13
bull high quality of recommendations13
bull abstraction (model) of input data13
bull pre-computation (partially) possible (model has to be re-built from time to time)13
bull scales better13
bull abstraction may reduce recommendation quality
Social Web 2014 Lora Aroyo
Memory vs Model-based
bull collaborative filtering lsquoneighborhoodsrsquo of people with similar interest amp recommending items based on likings in neighborhood13
bull limitations next to lsquocold startrsquo and lsquosparsityrsquo the lack of control (over onersquos neighborhood) is also a problem ie cannot add lsquotrustedrsquo people nor exclude lsquostrangersquo ones13
bull therefore interest in lsquosocial recommendersrsquo where presence of social connections defines the similarity in interests (eg social tagging CiteULike)13
bull does a social connection indicate user interest similarity13
bull how much users interest similarity depends on the strength of their connection13
bull is it feasible to use a social network as a personalized recommendation
[Lin amp Brusilovsky Social Networks and Interest Similarity The Case of CiteULike HTrsquo10]Social Web 2014 Lora Aroyo
Social Networks amp Interest Similarity
bull unilaterally connected pairs have more common itemsmetadatatags than non-connected pairs13
bull highest similarity for direct connections - decreasing with the increase of distance between users in SN 13
bull reciprocal relationship users - significantly larger similarity than users in a unidirectional relationship 13
bull traditional item-level similarity may be less reliable to find similar users in social bookmarking systems13
bull peers connected by self-defined social connections could be a useful source for cross-recommendation
Social Web 2014 Lora Aroyo
Conclusions
bull Input characteristics of items amp interests of a user into characteristics of items =gt Recommend items that feature characteristics which meet the userrsquos interests13
bull Techniques13
bull Data mining methods Cluster items based on their characteristics =gt Infer usersrsquo interests into clusters13
bull IR methods Represent items amp users as term vectors =gt Compute similarity between user profile vector and items13
bull Utility-based methods Utility function that gets an item as input the parameters of the utility function are customized via preferences of a user
Social Web 2014 Lora Aroyo
Content-based Recommendations
Government stops renovation of tower bridge Oct 13th 2011
Tower Bridge today Under construction
Tower Bridge is a combined bascule and suspension bridge in London England over the River Thames
Category politics england Related Twiper news bob Why do they stop tohellip [more] mary London stops renohellip [more]
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
02 0 0
02 04 01 01
= a
Weighting strategy - occurrence frequency - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
Content Features
RT Government stops renovation of tower bridge Oct 13th 2011
Userrsquos Twitter history
I am in London at the moment Oct 13th 2011
I am doing sports Oct 12th 2011
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
0 01 0
05 02 02 0
= u
Weighting strategy - occurrence frequency (eg smoothened by occurrence time recent concepts are more important - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
User Model
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
u 0
01 0
05 02 02 0
candidate items user a
02 0 0
02 04 01 01
b 0 0 0
08 02 0 0
c 0
05 02 0 0 0
03
cosine similarities
a b c
u 067 092 014
Ranking of recommended items 1 b 2 a 3 c
based on slides from Fabien Abel
Recommendations
Social Web 2014 Lora Aroyo
RecSys Issues
bull Cold-start problem (new user problem) nolittle data available to infer preferences of new users
bull Changing User Preferences user interests may change over time
bull Sparsity problem (new item problem) item descriptions are sparse eg not many user rated or tagged an item
bull Lack of Diversity (overfitting) when adapting too strongly to the preferences of users they might see samesimilar recommendations
bull Use the right context users do things which might not be relevant for their user model eg try out things do stuff for other people
bull Research challenge right balance between serendipity amp personalization
bull Research challenge right way to use the influence of recommendations on userrsquos behavior
one machine13vs 13
humans
image source httpwwwflickrcomphotosbionicteaching1375254387Social Web 2014 Lora Aroyo
Hands-on Teaser
bull Your Facebook Friendsrsquo popularity in a spread sheet13bull Locations of your Facebook Friends13bull Tag Cloud of your wall posts
Predict relevantusefulinteresting items13for a given user (in a given context)13
itrsquos often a ranking task
Social Web 2014 Lora Aroyo
Recommendation Systems
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
March 28 2013
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
commercial 13personalisation
httpwwwwiredcommagazine201111mf_artsyall1
Social Web 2014 Lora Aroyo
filter bubble
u1 likes u2
likes likes u1 likes Pulp Fiction
Social Web 2014 Lora Aroyo
Collaborative Filteringbull Memory-based User-Item matrix ratingspreferences of users =gt compute
similarity between users amp recommend items of similar users13
bull Model-based Item-Item matrix similarity (eg based on user ratings) between items =gt recommend items that are similar to the ones the user likes13
bull Model-based Clustering cluster users according to their preferences =gt recommend items of users that belong to the same cluster13
bull Model-based Bayesian networks P(u likes item B | u likes item A) = how likely is it that a user who likes item A will like item B learn probabilities from user ratingspreferences13
bull Others rule-based other data mining techniques13
bull complete input data is required13
bull pre-computation not possible13
bull does not scale well 13
bull high quality of recommendations13
bull abstraction (model) of input data13
bull pre-computation (partially) possible (model has to be re-built from time to time)13
bull scales better13
bull abstraction may reduce recommendation quality
Social Web 2014 Lora Aroyo
Memory vs Model-based
bull collaborative filtering lsquoneighborhoodsrsquo of people with similar interest amp recommending items based on likings in neighborhood13
bull limitations next to lsquocold startrsquo and lsquosparsityrsquo the lack of control (over onersquos neighborhood) is also a problem ie cannot add lsquotrustedrsquo people nor exclude lsquostrangersquo ones13
bull therefore interest in lsquosocial recommendersrsquo where presence of social connections defines the similarity in interests (eg social tagging CiteULike)13
bull does a social connection indicate user interest similarity13
bull how much users interest similarity depends on the strength of their connection13
bull is it feasible to use a social network as a personalized recommendation
[Lin amp Brusilovsky Social Networks and Interest Similarity The Case of CiteULike HTrsquo10]Social Web 2014 Lora Aroyo
Social Networks amp Interest Similarity
bull unilaterally connected pairs have more common itemsmetadatatags than non-connected pairs13
bull highest similarity for direct connections - decreasing with the increase of distance between users in SN 13
bull reciprocal relationship users - significantly larger similarity than users in a unidirectional relationship 13
bull traditional item-level similarity may be less reliable to find similar users in social bookmarking systems13
bull peers connected by self-defined social connections could be a useful source for cross-recommendation
Social Web 2014 Lora Aroyo
Conclusions
bull Input characteristics of items amp interests of a user into characteristics of items =gt Recommend items that feature characteristics which meet the userrsquos interests13
bull Techniques13
bull Data mining methods Cluster items based on their characteristics =gt Infer usersrsquo interests into clusters13
bull IR methods Represent items amp users as term vectors =gt Compute similarity between user profile vector and items13
bull Utility-based methods Utility function that gets an item as input the parameters of the utility function are customized via preferences of a user
Social Web 2014 Lora Aroyo
Content-based Recommendations
Government stops renovation of tower bridge Oct 13th 2011
Tower Bridge today Under construction
Tower Bridge is a combined bascule and suspension bridge in London England over the River Thames
Category politics england Related Twiper news bob Why do they stop tohellip [more] mary London stops renohellip [more]
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
02 0 0
02 04 01 01
= a
Weighting strategy - occurrence frequency - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
Content Features
RT Government stops renovation of tower bridge Oct 13th 2011
Userrsquos Twitter history
I am in London at the moment Oct 13th 2011
I am doing sports Oct 12th 2011
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
0 01 0
05 02 02 0
= u
Weighting strategy - occurrence frequency (eg smoothened by occurrence time recent concepts are more important - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
User Model
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
u 0
01 0
05 02 02 0
candidate items user a
02 0 0
02 04 01 01
b 0 0 0
08 02 0 0
c 0
05 02 0 0 0
03
cosine similarities
a b c
u 067 092 014
Ranking of recommended items 1 b 2 a 3 c
based on slides from Fabien Abel
Recommendations
Social Web 2014 Lora Aroyo
RecSys Issues
bull Cold-start problem (new user problem) nolittle data available to infer preferences of new users
bull Changing User Preferences user interests may change over time
bull Sparsity problem (new item problem) item descriptions are sparse eg not many user rated or tagged an item
bull Lack of Diversity (overfitting) when adapting too strongly to the preferences of users they might see samesimilar recommendations
bull Use the right context users do things which might not be relevant for their user model eg try out things do stuff for other people
bull Research challenge right balance between serendipity amp personalization
bull Research challenge right way to use the influence of recommendations on userrsquos behavior
one machine13vs 13
humans
image source httpwwwflickrcomphotosbionicteaching1375254387Social Web 2014 Lora Aroyo
Hands-on Teaser
bull Your Facebook Friendsrsquo popularity in a spread sheet13bull Locations of your Facebook Friends13bull Tag Cloud of your wall posts
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
March 28 2013
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
commercial 13personalisation
httpwwwwiredcommagazine201111mf_artsyall1
Social Web 2014 Lora Aroyo
filter bubble
u1 likes u2
likes likes u1 likes Pulp Fiction
Social Web 2014 Lora Aroyo
Collaborative Filteringbull Memory-based User-Item matrix ratingspreferences of users =gt compute
similarity between users amp recommend items of similar users13
bull Model-based Item-Item matrix similarity (eg based on user ratings) between items =gt recommend items that are similar to the ones the user likes13
bull Model-based Clustering cluster users according to their preferences =gt recommend items of users that belong to the same cluster13
bull Model-based Bayesian networks P(u likes item B | u likes item A) = how likely is it that a user who likes item A will like item B learn probabilities from user ratingspreferences13
bull Others rule-based other data mining techniques13
bull complete input data is required13
bull pre-computation not possible13
bull does not scale well 13
bull high quality of recommendations13
bull abstraction (model) of input data13
bull pre-computation (partially) possible (model has to be re-built from time to time)13
bull scales better13
bull abstraction may reduce recommendation quality
Social Web 2014 Lora Aroyo
Memory vs Model-based
bull collaborative filtering lsquoneighborhoodsrsquo of people with similar interest amp recommending items based on likings in neighborhood13
bull limitations next to lsquocold startrsquo and lsquosparsityrsquo the lack of control (over onersquos neighborhood) is also a problem ie cannot add lsquotrustedrsquo people nor exclude lsquostrangersquo ones13
bull therefore interest in lsquosocial recommendersrsquo where presence of social connections defines the similarity in interests (eg social tagging CiteULike)13
bull does a social connection indicate user interest similarity13
bull how much users interest similarity depends on the strength of their connection13
bull is it feasible to use a social network as a personalized recommendation
[Lin amp Brusilovsky Social Networks and Interest Similarity The Case of CiteULike HTrsquo10]Social Web 2014 Lora Aroyo
Social Networks amp Interest Similarity
bull unilaterally connected pairs have more common itemsmetadatatags than non-connected pairs13
bull highest similarity for direct connections - decreasing with the increase of distance between users in SN 13
bull reciprocal relationship users - significantly larger similarity than users in a unidirectional relationship 13
bull traditional item-level similarity may be less reliable to find similar users in social bookmarking systems13
bull peers connected by self-defined social connections could be a useful source for cross-recommendation
Social Web 2014 Lora Aroyo
Conclusions
bull Input characteristics of items amp interests of a user into characteristics of items =gt Recommend items that feature characteristics which meet the userrsquos interests13
bull Techniques13
bull Data mining methods Cluster items based on their characteristics =gt Infer usersrsquo interests into clusters13
bull IR methods Represent items amp users as term vectors =gt Compute similarity between user profile vector and items13
bull Utility-based methods Utility function that gets an item as input the parameters of the utility function are customized via preferences of a user
Social Web 2014 Lora Aroyo
Content-based Recommendations
Government stops renovation of tower bridge Oct 13th 2011
Tower Bridge today Under construction
Tower Bridge is a combined bascule and suspension bridge in London England over the River Thames
Category politics england Related Twiper news bob Why do they stop tohellip [more] mary London stops renohellip [more]
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
02 0 0
02 04 01 01
= a
Weighting strategy - occurrence frequency - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
Content Features
RT Government stops renovation of tower bridge Oct 13th 2011
Userrsquos Twitter history
I am in London at the moment Oct 13th 2011
I am doing sports Oct 12th 2011
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
0 01 0
05 02 02 0
= u
Weighting strategy - occurrence frequency (eg smoothened by occurrence time recent concepts are more important - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
User Model
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
u 0
01 0
05 02 02 0
candidate items user a
02 0 0
02 04 01 01
b 0 0 0
08 02 0 0
c 0
05 02 0 0 0
03
cosine similarities
a b c
u 067 092 014
Ranking of recommended items 1 b 2 a 3 c
based on slides from Fabien Abel
Recommendations
Social Web 2014 Lora Aroyo
RecSys Issues
bull Cold-start problem (new user problem) nolittle data available to infer preferences of new users
bull Changing User Preferences user interests may change over time
bull Sparsity problem (new item problem) item descriptions are sparse eg not many user rated or tagged an item
bull Lack of Diversity (overfitting) when adapting too strongly to the preferences of users they might see samesimilar recommendations
bull Use the right context users do things which might not be relevant for their user model eg try out things do stuff for other people
bull Research challenge right balance between serendipity amp personalization
bull Research challenge right way to use the influence of recommendations on userrsquos behavior
one machine13vs 13
humans
image source httpwwwflickrcomphotosbionicteaching1375254387Social Web 2014 Lora Aroyo
Hands-on Teaser
bull Your Facebook Friendsrsquo popularity in a spread sheet13bull Locations of your Facebook Friends13bull Tag Cloud of your wall posts
Social Web 2014 Lora Aroyo
March 28 2013
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
commercial 13personalisation
httpwwwwiredcommagazine201111mf_artsyall1
Social Web 2014 Lora Aroyo
filter bubble
u1 likes u2
likes likes u1 likes Pulp Fiction
Social Web 2014 Lora Aroyo
Collaborative Filteringbull Memory-based User-Item matrix ratingspreferences of users =gt compute
similarity between users amp recommend items of similar users13
bull Model-based Item-Item matrix similarity (eg based on user ratings) between items =gt recommend items that are similar to the ones the user likes13
bull Model-based Clustering cluster users according to their preferences =gt recommend items of users that belong to the same cluster13
bull Model-based Bayesian networks P(u likes item B | u likes item A) = how likely is it that a user who likes item A will like item B learn probabilities from user ratingspreferences13
bull Others rule-based other data mining techniques13
bull complete input data is required13
bull pre-computation not possible13
bull does not scale well 13
bull high quality of recommendations13
bull abstraction (model) of input data13
bull pre-computation (partially) possible (model has to be re-built from time to time)13
bull scales better13
bull abstraction may reduce recommendation quality
Social Web 2014 Lora Aroyo
Memory vs Model-based
bull collaborative filtering lsquoneighborhoodsrsquo of people with similar interest amp recommending items based on likings in neighborhood13
bull limitations next to lsquocold startrsquo and lsquosparsityrsquo the lack of control (over onersquos neighborhood) is also a problem ie cannot add lsquotrustedrsquo people nor exclude lsquostrangersquo ones13
bull therefore interest in lsquosocial recommendersrsquo where presence of social connections defines the similarity in interests (eg social tagging CiteULike)13
bull does a social connection indicate user interest similarity13
bull how much users interest similarity depends on the strength of their connection13
bull is it feasible to use a social network as a personalized recommendation
[Lin amp Brusilovsky Social Networks and Interest Similarity The Case of CiteULike HTrsquo10]Social Web 2014 Lora Aroyo
Social Networks amp Interest Similarity
bull unilaterally connected pairs have more common itemsmetadatatags than non-connected pairs13
bull highest similarity for direct connections - decreasing with the increase of distance between users in SN 13
bull reciprocal relationship users - significantly larger similarity than users in a unidirectional relationship 13
bull traditional item-level similarity may be less reliable to find similar users in social bookmarking systems13
bull peers connected by self-defined social connections could be a useful source for cross-recommendation
Social Web 2014 Lora Aroyo
Conclusions
bull Input characteristics of items amp interests of a user into characteristics of items =gt Recommend items that feature characteristics which meet the userrsquos interests13
bull Techniques13
bull Data mining methods Cluster items based on their characteristics =gt Infer usersrsquo interests into clusters13
bull IR methods Represent items amp users as term vectors =gt Compute similarity between user profile vector and items13
bull Utility-based methods Utility function that gets an item as input the parameters of the utility function are customized via preferences of a user
Social Web 2014 Lora Aroyo
Content-based Recommendations
Government stops renovation of tower bridge Oct 13th 2011
Tower Bridge today Under construction
Tower Bridge is a combined bascule and suspension bridge in London England over the River Thames
Category politics england Related Twiper news bob Why do they stop tohellip [more] mary London stops renohellip [more]
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
02 0 0
02 04 01 01
= a
Weighting strategy - occurrence frequency - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
Content Features
RT Government stops renovation of tower bridge Oct 13th 2011
Userrsquos Twitter history
I am in London at the moment Oct 13th 2011
I am doing sports Oct 12th 2011
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
0 01 0
05 02 02 0
= u
Weighting strategy - occurrence frequency (eg smoothened by occurrence time recent concepts are more important - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
User Model
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
u 0
01 0
05 02 02 0
candidate items user a
02 0 0
02 04 01 01
b 0 0 0
08 02 0 0
c 0
05 02 0 0 0
03
cosine similarities
a b c
u 067 092 014
Ranking of recommended items 1 b 2 a 3 c
based on slides from Fabien Abel
Recommendations
Social Web 2014 Lora Aroyo
RecSys Issues
bull Cold-start problem (new user problem) nolittle data available to infer preferences of new users
bull Changing User Preferences user interests may change over time
bull Sparsity problem (new item problem) item descriptions are sparse eg not many user rated or tagged an item
bull Lack of Diversity (overfitting) when adapting too strongly to the preferences of users they might see samesimilar recommendations
bull Use the right context users do things which might not be relevant for their user model eg try out things do stuff for other people
bull Research challenge right balance between serendipity amp personalization
bull Research challenge right way to use the influence of recommendations on userrsquos behavior
one machine13vs 13
humans
image source httpwwwflickrcomphotosbionicteaching1375254387Social Web 2014 Lora Aroyo
Hands-on Teaser
bull Your Facebook Friendsrsquo popularity in a spread sheet13bull Locations of your Facebook Friends13bull Tag Cloud of your wall posts
Social Web 2014 Lora Aroyo
Social Web 2014 Lora Aroyo
commercial 13personalisation
httpwwwwiredcommagazine201111mf_artsyall1
Social Web 2014 Lora Aroyo
filter bubble
u1 likes u2
likes likes u1 likes Pulp Fiction
Social Web 2014 Lora Aroyo
Collaborative Filteringbull Memory-based User-Item matrix ratingspreferences of users =gt compute
similarity between users amp recommend items of similar users13
bull Model-based Item-Item matrix similarity (eg based on user ratings) between items =gt recommend items that are similar to the ones the user likes13
bull Model-based Clustering cluster users according to their preferences =gt recommend items of users that belong to the same cluster13
bull Model-based Bayesian networks P(u likes item B | u likes item A) = how likely is it that a user who likes item A will like item B learn probabilities from user ratingspreferences13
bull Others rule-based other data mining techniques13
bull complete input data is required13
bull pre-computation not possible13
bull does not scale well 13
bull high quality of recommendations13
bull abstraction (model) of input data13
bull pre-computation (partially) possible (model has to be re-built from time to time)13
bull scales better13
bull abstraction may reduce recommendation quality
Social Web 2014 Lora Aroyo
Memory vs Model-based
bull collaborative filtering lsquoneighborhoodsrsquo of people with similar interest amp recommending items based on likings in neighborhood13
bull limitations next to lsquocold startrsquo and lsquosparsityrsquo the lack of control (over onersquos neighborhood) is also a problem ie cannot add lsquotrustedrsquo people nor exclude lsquostrangersquo ones13
bull therefore interest in lsquosocial recommendersrsquo where presence of social connections defines the similarity in interests (eg social tagging CiteULike)13
bull does a social connection indicate user interest similarity13
bull how much users interest similarity depends on the strength of their connection13
bull is it feasible to use a social network as a personalized recommendation
[Lin amp Brusilovsky Social Networks and Interest Similarity The Case of CiteULike HTrsquo10]Social Web 2014 Lora Aroyo
Social Networks amp Interest Similarity
bull unilaterally connected pairs have more common itemsmetadatatags than non-connected pairs13
bull highest similarity for direct connections - decreasing with the increase of distance between users in SN 13
bull reciprocal relationship users - significantly larger similarity than users in a unidirectional relationship 13
bull traditional item-level similarity may be less reliable to find similar users in social bookmarking systems13
bull peers connected by self-defined social connections could be a useful source for cross-recommendation
Social Web 2014 Lora Aroyo
Conclusions
bull Input characteristics of items amp interests of a user into characteristics of items =gt Recommend items that feature characteristics which meet the userrsquos interests13
bull Techniques13
bull Data mining methods Cluster items based on their characteristics =gt Infer usersrsquo interests into clusters13
bull IR methods Represent items amp users as term vectors =gt Compute similarity between user profile vector and items13
bull Utility-based methods Utility function that gets an item as input the parameters of the utility function are customized via preferences of a user
Social Web 2014 Lora Aroyo
Content-based Recommendations
Government stops renovation of tower bridge Oct 13th 2011
Tower Bridge today Under construction
Tower Bridge is a combined bascule and suspension bridge in London England over the River Thames
Category politics england Related Twiper news bob Why do they stop tohellip [more] mary London stops renohellip [more]
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
02 0 0
02 04 01 01
= a
Weighting strategy - occurrence frequency - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
Content Features
RT Government stops renovation of tower bridge Oct 13th 2011
Userrsquos Twitter history
I am in London at the moment Oct 13th 2011
I am doing sports Oct 12th 2011
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
0 01 0
05 02 02 0
= u
Weighting strategy - occurrence frequency (eg smoothened by occurrence time recent concepts are more important - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
User Model
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
u 0
01 0
05 02 02 0
candidate items user a
02 0 0
02 04 01 01
b 0 0 0
08 02 0 0
c 0
05 02 0 0 0
03
cosine similarities
a b c
u 067 092 014
Ranking of recommended items 1 b 2 a 3 c
based on slides from Fabien Abel
Recommendations
Social Web 2014 Lora Aroyo
RecSys Issues
bull Cold-start problem (new user problem) nolittle data available to infer preferences of new users
bull Changing User Preferences user interests may change over time
bull Sparsity problem (new item problem) item descriptions are sparse eg not many user rated or tagged an item
bull Lack of Diversity (overfitting) when adapting too strongly to the preferences of users they might see samesimilar recommendations
bull Use the right context users do things which might not be relevant for their user model eg try out things do stuff for other people
bull Research challenge right balance between serendipity amp personalization
bull Research challenge right way to use the influence of recommendations on userrsquos behavior
one machine13vs 13
humans
image source httpwwwflickrcomphotosbionicteaching1375254387Social Web 2014 Lora Aroyo
Hands-on Teaser
bull Your Facebook Friendsrsquo popularity in a spread sheet13bull Locations of your Facebook Friends13bull Tag Cloud of your wall posts
Social Web 2014 Lora Aroyo
commercial 13personalisation
httpwwwwiredcommagazine201111mf_artsyall1
Social Web 2014 Lora Aroyo
filter bubble
u1 likes u2
likes likes u1 likes Pulp Fiction
Social Web 2014 Lora Aroyo
Collaborative Filteringbull Memory-based User-Item matrix ratingspreferences of users =gt compute
similarity between users amp recommend items of similar users13
bull Model-based Item-Item matrix similarity (eg based on user ratings) between items =gt recommend items that are similar to the ones the user likes13
bull Model-based Clustering cluster users according to their preferences =gt recommend items of users that belong to the same cluster13
bull Model-based Bayesian networks P(u likes item B | u likes item A) = how likely is it that a user who likes item A will like item B learn probabilities from user ratingspreferences13
bull Others rule-based other data mining techniques13
bull complete input data is required13
bull pre-computation not possible13
bull does not scale well 13
bull high quality of recommendations13
bull abstraction (model) of input data13
bull pre-computation (partially) possible (model has to be re-built from time to time)13
bull scales better13
bull abstraction may reduce recommendation quality
Social Web 2014 Lora Aroyo
Memory vs Model-based
bull collaborative filtering lsquoneighborhoodsrsquo of people with similar interest amp recommending items based on likings in neighborhood13
bull limitations next to lsquocold startrsquo and lsquosparsityrsquo the lack of control (over onersquos neighborhood) is also a problem ie cannot add lsquotrustedrsquo people nor exclude lsquostrangersquo ones13
bull therefore interest in lsquosocial recommendersrsquo where presence of social connections defines the similarity in interests (eg social tagging CiteULike)13
bull does a social connection indicate user interest similarity13
bull how much users interest similarity depends on the strength of their connection13
bull is it feasible to use a social network as a personalized recommendation
[Lin amp Brusilovsky Social Networks and Interest Similarity The Case of CiteULike HTrsquo10]Social Web 2014 Lora Aroyo
Social Networks amp Interest Similarity
bull unilaterally connected pairs have more common itemsmetadatatags than non-connected pairs13
bull highest similarity for direct connections - decreasing with the increase of distance between users in SN 13
bull reciprocal relationship users - significantly larger similarity than users in a unidirectional relationship 13
bull traditional item-level similarity may be less reliable to find similar users in social bookmarking systems13
bull peers connected by self-defined social connections could be a useful source for cross-recommendation
Social Web 2014 Lora Aroyo
Conclusions
bull Input characteristics of items amp interests of a user into characteristics of items =gt Recommend items that feature characteristics which meet the userrsquos interests13
bull Techniques13
bull Data mining methods Cluster items based on their characteristics =gt Infer usersrsquo interests into clusters13
bull IR methods Represent items amp users as term vectors =gt Compute similarity between user profile vector and items13
bull Utility-based methods Utility function that gets an item as input the parameters of the utility function are customized via preferences of a user
Social Web 2014 Lora Aroyo
Content-based Recommendations
Government stops renovation of tower bridge Oct 13th 2011
Tower Bridge today Under construction
Tower Bridge is a combined bascule and suspension bridge in London England over the River Thames
Category politics england Related Twiper news bob Why do they stop tohellip [more] mary London stops renohellip [more]
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
02 0 0
02 04 01 01
= a
Weighting strategy - occurrence frequency - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
Content Features
RT Government stops renovation of tower bridge Oct 13th 2011
Userrsquos Twitter history
I am in London at the moment Oct 13th 2011
I am doing sports Oct 12th 2011
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
0 01 0
05 02 02 0
= u
Weighting strategy - occurrence frequency (eg smoothened by occurrence time recent concepts are more important - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
User Model
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
u 0
01 0
05 02 02 0
candidate items user a
02 0 0
02 04 01 01
b 0 0 0
08 02 0 0
c 0
05 02 0 0 0
03
cosine similarities
a b c
u 067 092 014
Ranking of recommended items 1 b 2 a 3 c
based on slides from Fabien Abel
Recommendations
Social Web 2014 Lora Aroyo
RecSys Issues
bull Cold-start problem (new user problem) nolittle data available to infer preferences of new users
bull Changing User Preferences user interests may change over time
bull Sparsity problem (new item problem) item descriptions are sparse eg not many user rated or tagged an item
bull Lack of Diversity (overfitting) when adapting too strongly to the preferences of users they might see samesimilar recommendations
bull Use the right context users do things which might not be relevant for their user model eg try out things do stuff for other people
bull Research challenge right balance between serendipity amp personalization
bull Research challenge right way to use the influence of recommendations on userrsquos behavior
one machine13vs 13
humans
image source httpwwwflickrcomphotosbionicteaching1375254387Social Web 2014 Lora Aroyo
Hands-on Teaser
bull Your Facebook Friendsrsquo popularity in a spread sheet13bull Locations of your Facebook Friends13bull Tag Cloud of your wall posts
commercial 13personalisation
httpwwwwiredcommagazine201111mf_artsyall1
Social Web 2014 Lora Aroyo
filter bubble
u1 likes u2
likes likes u1 likes Pulp Fiction
Social Web 2014 Lora Aroyo
Collaborative Filteringbull Memory-based User-Item matrix ratingspreferences of users =gt compute
similarity between users amp recommend items of similar users13
bull Model-based Item-Item matrix similarity (eg based on user ratings) between items =gt recommend items that are similar to the ones the user likes13
bull Model-based Clustering cluster users according to their preferences =gt recommend items of users that belong to the same cluster13
bull Model-based Bayesian networks P(u likes item B | u likes item A) = how likely is it that a user who likes item A will like item B learn probabilities from user ratingspreferences13
bull Others rule-based other data mining techniques13
bull complete input data is required13
bull pre-computation not possible13
bull does not scale well 13
bull high quality of recommendations13
bull abstraction (model) of input data13
bull pre-computation (partially) possible (model has to be re-built from time to time)13
bull scales better13
bull abstraction may reduce recommendation quality
Social Web 2014 Lora Aroyo
Memory vs Model-based
bull collaborative filtering lsquoneighborhoodsrsquo of people with similar interest amp recommending items based on likings in neighborhood13
bull limitations next to lsquocold startrsquo and lsquosparsityrsquo the lack of control (over onersquos neighborhood) is also a problem ie cannot add lsquotrustedrsquo people nor exclude lsquostrangersquo ones13
bull therefore interest in lsquosocial recommendersrsquo where presence of social connections defines the similarity in interests (eg social tagging CiteULike)13
bull does a social connection indicate user interest similarity13
bull how much users interest similarity depends on the strength of their connection13
bull is it feasible to use a social network as a personalized recommendation
[Lin amp Brusilovsky Social Networks and Interest Similarity The Case of CiteULike HTrsquo10]Social Web 2014 Lora Aroyo
Social Networks amp Interest Similarity
bull unilaterally connected pairs have more common itemsmetadatatags than non-connected pairs13
bull highest similarity for direct connections - decreasing with the increase of distance between users in SN 13
bull reciprocal relationship users - significantly larger similarity than users in a unidirectional relationship 13
bull traditional item-level similarity may be less reliable to find similar users in social bookmarking systems13
bull peers connected by self-defined social connections could be a useful source for cross-recommendation
Social Web 2014 Lora Aroyo
Conclusions
bull Input characteristics of items amp interests of a user into characteristics of items =gt Recommend items that feature characteristics which meet the userrsquos interests13
bull Techniques13
bull Data mining methods Cluster items based on their characteristics =gt Infer usersrsquo interests into clusters13
bull IR methods Represent items amp users as term vectors =gt Compute similarity between user profile vector and items13
bull Utility-based methods Utility function that gets an item as input the parameters of the utility function are customized via preferences of a user
Social Web 2014 Lora Aroyo
Content-based Recommendations
Government stops renovation of tower bridge Oct 13th 2011
Tower Bridge today Under construction
Tower Bridge is a combined bascule and suspension bridge in London England over the River Thames
Category politics england Related Twiper news bob Why do they stop tohellip [more] mary London stops renohellip [more]
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
02 0 0
02 04 01 01
= a
Weighting strategy - occurrence frequency - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
Content Features
RT Government stops renovation of tower bridge Oct 13th 2011
Userrsquos Twitter history
I am in London at the moment Oct 13th 2011
I am doing sports Oct 12th 2011
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
0 01 0
05 02 02 0
= u
Weighting strategy - occurrence frequency (eg smoothened by occurrence time recent concepts are more important - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
User Model
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
u 0
01 0
05 02 02 0
candidate items user a
02 0 0
02 04 01 01
b 0 0 0
08 02 0 0
c 0
05 02 0 0 0
03
cosine similarities
a b c
u 067 092 014
Ranking of recommended items 1 b 2 a 3 c
based on slides from Fabien Abel
Recommendations
Social Web 2014 Lora Aroyo
RecSys Issues
bull Cold-start problem (new user problem) nolittle data available to infer preferences of new users
bull Changing User Preferences user interests may change over time
bull Sparsity problem (new item problem) item descriptions are sparse eg not many user rated or tagged an item
bull Lack of Diversity (overfitting) when adapting too strongly to the preferences of users they might see samesimilar recommendations
bull Use the right context users do things which might not be relevant for their user model eg try out things do stuff for other people
bull Research challenge right balance between serendipity amp personalization
bull Research challenge right way to use the influence of recommendations on userrsquos behavior
one machine13vs 13
humans
image source httpwwwflickrcomphotosbionicteaching1375254387Social Web 2014 Lora Aroyo
Hands-on Teaser
bull Your Facebook Friendsrsquo popularity in a spread sheet13bull Locations of your Facebook Friends13bull Tag Cloud of your wall posts
httpwwwwiredcommagazine201111mf_artsyall1
Social Web 2014 Lora Aroyo
filter bubble
u1 likes u2
likes likes u1 likes Pulp Fiction
Social Web 2014 Lora Aroyo
Collaborative Filteringbull Memory-based User-Item matrix ratingspreferences of users =gt compute
similarity between users amp recommend items of similar users13
bull Model-based Item-Item matrix similarity (eg based on user ratings) between items =gt recommend items that are similar to the ones the user likes13
bull Model-based Clustering cluster users according to their preferences =gt recommend items of users that belong to the same cluster13
bull Model-based Bayesian networks P(u likes item B | u likes item A) = how likely is it that a user who likes item A will like item B learn probabilities from user ratingspreferences13
bull Others rule-based other data mining techniques13
bull complete input data is required13
bull pre-computation not possible13
bull does not scale well 13
bull high quality of recommendations13
bull abstraction (model) of input data13
bull pre-computation (partially) possible (model has to be re-built from time to time)13
bull scales better13
bull abstraction may reduce recommendation quality
Social Web 2014 Lora Aroyo
Memory vs Model-based
bull collaborative filtering lsquoneighborhoodsrsquo of people with similar interest amp recommending items based on likings in neighborhood13
bull limitations next to lsquocold startrsquo and lsquosparsityrsquo the lack of control (over onersquos neighborhood) is also a problem ie cannot add lsquotrustedrsquo people nor exclude lsquostrangersquo ones13
bull therefore interest in lsquosocial recommendersrsquo where presence of social connections defines the similarity in interests (eg social tagging CiteULike)13
bull does a social connection indicate user interest similarity13
bull how much users interest similarity depends on the strength of their connection13
bull is it feasible to use a social network as a personalized recommendation
[Lin amp Brusilovsky Social Networks and Interest Similarity The Case of CiteULike HTrsquo10]Social Web 2014 Lora Aroyo
Social Networks amp Interest Similarity
bull unilaterally connected pairs have more common itemsmetadatatags than non-connected pairs13
bull highest similarity for direct connections - decreasing with the increase of distance between users in SN 13
bull reciprocal relationship users - significantly larger similarity than users in a unidirectional relationship 13
bull traditional item-level similarity may be less reliable to find similar users in social bookmarking systems13
bull peers connected by self-defined social connections could be a useful source for cross-recommendation
Social Web 2014 Lora Aroyo
Conclusions
bull Input characteristics of items amp interests of a user into characteristics of items =gt Recommend items that feature characteristics which meet the userrsquos interests13
bull Techniques13
bull Data mining methods Cluster items based on their characteristics =gt Infer usersrsquo interests into clusters13
bull IR methods Represent items amp users as term vectors =gt Compute similarity between user profile vector and items13
bull Utility-based methods Utility function that gets an item as input the parameters of the utility function are customized via preferences of a user
Social Web 2014 Lora Aroyo
Content-based Recommendations
Government stops renovation of tower bridge Oct 13th 2011
Tower Bridge today Under construction
Tower Bridge is a combined bascule and suspension bridge in London England over the River Thames
Category politics england Related Twiper news bob Why do they stop tohellip [more] mary London stops renohellip [more]
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
02 0 0
02 04 01 01
= a
Weighting strategy - occurrence frequency - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
Content Features
RT Government stops renovation of tower bridge Oct 13th 2011
Userrsquos Twitter history
I am in London at the moment Oct 13th 2011
I am doing sports Oct 12th 2011
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
0 01 0
05 02 02 0
= u
Weighting strategy - occurrence frequency (eg smoothened by occurrence time recent concepts are more important - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
User Model
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
u 0
01 0
05 02 02 0
candidate items user a
02 0 0
02 04 01 01
b 0 0 0
08 02 0 0
c 0
05 02 0 0 0
03
cosine similarities
a b c
u 067 092 014
Ranking of recommended items 1 b 2 a 3 c
based on slides from Fabien Abel
Recommendations
Social Web 2014 Lora Aroyo
RecSys Issues
bull Cold-start problem (new user problem) nolittle data available to infer preferences of new users
bull Changing User Preferences user interests may change over time
bull Sparsity problem (new item problem) item descriptions are sparse eg not many user rated or tagged an item
bull Lack of Diversity (overfitting) when adapting too strongly to the preferences of users they might see samesimilar recommendations
bull Use the right context users do things which might not be relevant for their user model eg try out things do stuff for other people
bull Research challenge right balance between serendipity amp personalization
bull Research challenge right way to use the influence of recommendations on userrsquos behavior
one machine13vs 13
humans
image source httpwwwflickrcomphotosbionicteaching1375254387Social Web 2014 Lora Aroyo
Hands-on Teaser
bull Your Facebook Friendsrsquo popularity in a spread sheet13bull Locations of your Facebook Friends13bull Tag Cloud of your wall posts
filter bubble
u1 likes u2
likes likes u1 likes Pulp Fiction
Social Web 2014 Lora Aroyo
Collaborative Filteringbull Memory-based User-Item matrix ratingspreferences of users =gt compute
similarity between users amp recommend items of similar users13
bull Model-based Item-Item matrix similarity (eg based on user ratings) between items =gt recommend items that are similar to the ones the user likes13
bull Model-based Clustering cluster users according to their preferences =gt recommend items of users that belong to the same cluster13
bull Model-based Bayesian networks P(u likes item B | u likes item A) = how likely is it that a user who likes item A will like item B learn probabilities from user ratingspreferences13
bull Others rule-based other data mining techniques13
bull complete input data is required13
bull pre-computation not possible13
bull does not scale well 13
bull high quality of recommendations13
bull abstraction (model) of input data13
bull pre-computation (partially) possible (model has to be re-built from time to time)13
bull scales better13
bull abstraction may reduce recommendation quality
Social Web 2014 Lora Aroyo
Memory vs Model-based
bull collaborative filtering lsquoneighborhoodsrsquo of people with similar interest amp recommending items based on likings in neighborhood13
bull limitations next to lsquocold startrsquo and lsquosparsityrsquo the lack of control (over onersquos neighborhood) is also a problem ie cannot add lsquotrustedrsquo people nor exclude lsquostrangersquo ones13
bull therefore interest in lsquosocial recommendersrsquo where presence of social connections defines the similarity in interests (eg social tagging CiteULike)13
bull does a social connection indicate user interest similarity13
bull how much users interest similarity depends on the strength of their connection13
bull is it feasible to use a social network as a personalized recommendation
[Lin amp Brusilovsky Social Networks and Interest Similarity The Case of CiteULike HTrsquo10]Social Web 2014 Lora Aroyo
Social Networks amp Interest Similarity
bull unilaterally connected pairs have more common itemsmetadatatags than non-connected pairs13
bull highest similarity for direct connections - decreasing with the increase of distance between users in SN 13
bull reciprocal relationship users - significantly larger similarity than users in a unidirectional relationship 13
bull traditional item-level similarity may be less reliable to find similar users in social bookmarking systems13
bull peers connected by self-defined social connections could be a useful source for cross-recommendation
Social Web 2014 Lora Aroyo
Conclusions
bull Input characteristics of items amp interests of a user into characteristics of items =gt Recommend items that feature characteristics which meet the userrsquos interests13
bull Techniques13
bull Data mining methods Cluster items based on their characteristics =gt Infer usersrsquo interests into clusters13
bull IR methods Represent items amp users as term vectors =gt Compute similarity between user profile vector and items13
bull Utility-based methods Utility function that gets an item as input the parameters of the utility function are customized via preferences of a user
Social Web 2014 Lora Aroyo
Content-based Recommendations
Government stops renovation of tower bridge Oct 13th 2011
Tower Bridge today Under construction
Tower Bridge is a combined bascule and suspension bridge in London England over the River Thames
Category politics england Related Twiper news bob Why do they stop tohellip [more] mary London stops renohellip [more]
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
02 0 0
02 04 01 01
= a
Weighting strategy - occurrence frequency - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
Content Features
RT Government stops renovation of tower bridge Oct 13th 2011
Userrsquos Twitter history
I am in London at the moment Oct 13th 2011
I am doing sports Oct 12th 2011
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
0 01 0
05 02 02 0
= u
Weighting strategy - occurrence frequency (eg smoothened by occurrence time recent concepts are more important - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
User Model
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
u 0
01 0
05 02 02 0
candidate items user a
02 0 0
02 04 01 01
b 0 0 0
08 02 0 0
c 0
05 02 0 0 0
03
cosine similarities
a b c
u 067 092 014
Ranking of recommended items 1 b 2 a 3 c
based on slides from Fabien Abel
Recommendations
Social Web 2014 Lora Aroyo
RecSys Issues
bull Cold-start problem (new user problem) nolittle data available to infer preferences of new users
bull Changing User Preferences user interests may change over time
bull Sparsity problem (new item problem) item descriptions are sparse eg not many user rated or tagged an item
bull Lack of Diversity (overfitting) when adapting too strongly to the preferences of users they might see samesimilar recommendations
bull Use the right context users do things which might not be relevant for their user model eg try out things do stuff for other people
bull Research challenge right balance between serendipity amp personalization
bull Research challenge right way to use the influence of recommendations on userrsquos behavior
one machine13vs 13
humans
image source httpwwwflickrcomphotosbionicteaching1375254387Social Web 2014 Lora Aroyo
Hands-on Teaser
bull Your Facebook Friendsrsquo popularity in a spread sheet13bull Locations of your Facebook Friends13bull Tag Cloud of your wall posts
u1 likes u2
likes likes u1 likes Pulp Fiction
Social Web 2014 Lora Aroyo
Collaborative Filteringbull Memory-based User-Item matrix ratingspreferences of users =gt compute
similarity between users amp recommend items of similar users13
bull Model-based Item-Item matrix similarity (eg based on user ratings) between items =gt recommend items that are similar to the ones the user likes13
bull Model-based Clustering cluster users according to their preferences =gt recommend items of users that belong to the same cluster13
bull Model-based Bayesian networks P(u likes item B | u likes item A) = how likely is it that a user who likes item A will like item B learn probabilities from user ratingspreferences13
bull Others rule-based other data mining techniques13
bull complete input data is required13
bull pre-computation not possible13
bull does not scale well 13
bull high quality of recommendations13
bull abstraction (model) of input data13
bull pre-computation (partially) possible (model has to be re-built from time to time)13
bull scales better13
bull abstraction may reduce recommendation quality
Social Web 2014 Lora Aroyo
Memory vs Model-based
bull collaborative filtering lsquoneighborhoodsrsquo of people with similar interest amp recommending items based on likings in neighborhood13
bull limitations next to lsquocold startrsquo and lsquosparsityrsquo the lack of control (over onersquos neighborhood) is also a problem ie cannot add lsquotrustedrsquo people nor exclude lsquostrangersquo ones13
bull therefore interest in lsquosocial recommendersrsquo where presence of social connections defines the similarity in interests (eg social tagging CiteULike)13
bull does a social connection indicate user interest similarity13
bull how much users interest similarity depends on the strength of their connection13
bull is it feasible to use a social network as a personalized recommendation
[Lin amp Brusilovsky Social Networks and Interest Similarity The Case of CiteULike HTrsquo10]Social Web 2014 Lora Aroyo
Social Networks amp Interest Similarity
bull unilaterally connected pairs have more common itemsmetadatatags than non-connected pairs13
bull highest similarity for direct connections - decreasing with the increase of distance between users in SN 13
bull reciprocal relationship users - significantly larger similarity than users in a unidirectional relationship 13
bull traditional item-level similarity may be less reliable to find similar users in social bookmarking systems13
bull peers connected by self-defined social connections could be a useful source for cross-recommendation
Social Web 2014 Lora Aroyo
Conclusions
bull Input characteristics of items amp interests of a user into characteristics of items =gt Recommend items that feature characteristics which meet the userrsquos interests13
bull Techniques13
bull Data mining methods Cluster items based on their characteristics =gt Infer usersrsquo interests into clusters13
bull IR methods Represent items amp users as term vectors =gt Compute similarity between user profile vector and items13
bull Utility-based methods Utility function that gets an item as input the parameters of the utility function are customized via preferences of a user
Social Web 2014 Lora Aroyo
Content-based Recommendations
Government stops renovation of tower bridge Oct 13th 2011
Tower Bridge today Under construction
Tower Bridge is a combined bascule and suspension bridge in London England over the River Thames
Category politics england Related Twiper news bob Why do they stop tohellip [more] mary London stops renohellip [more]
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
02 0 0
02 04 01 01
= a
Weighting strategy - occurrence frequency - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
Content Features
RT Government stops renovation of tower bridge Oct 13th 2011
Userrsquos Twitter history
I am in London at the moment Oct 13th 2011
I am doing sports Oct 12th 2011
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
0 01 0
05 02 02 0
= u
Weighting strategy - occurrence frequency (eg smoothened by occurrence time recent concepts are more important - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
User Model
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
u 0
01 0
05 02 02 0
candidate items user a
02 0 0
02 04 01 01
b 0 0 0
08 02 0 0
c 0
05 02 0 0 0
03
cosine similarities
a b c
u 067 092 014
Ranking of recommended items 1 b 2 a 3 c
based on slides from Fabien Abel
Recommendations
Social Web 2014 Lora Aroyo
RecSys Issues
bull Cold-start problem (new user problem) nolittle data available to infer preferences of new users
bull Changing User Preferences user interests may change over time
bull Sparsity problem (new item problem) item descriptions are sparse eg not many user rated or tagged an item
bull Lack of Diversity (overfitting) when adapting too strongly to the preferences of users they might see samesimilar recommendations
bull Use the right context users do things which might not be relevant for their user model eg try out things do stuff for other people
bull Research challenge right balance between serendipity amp personalization
bull Research challenge right way to use the influence of recommendations on userrsquos behavior
one machine13vs 13
humans
image source httpwwwflickrcomphotosbionicteaching1375254387Social Web 2014 Lora Aroyo
Hands-on Teaser
bull Your Facebook Friendsrsquo popularity in a spread sheet13bull Locations of your Facebook Friends13bull Tag Cloud of your wall posts
bull complete input data is required13
bull pre-computation not possible13
bull does not scale well 13
bull high quality of recommendations13
bull abstraction (model) of input data13
bull pre-computation (partially) possible (model has to be re-built from time to time)13
bull scales better13
bull abstraction may reduce recommendation quality
Social Web 2014 Lora Aroyo
Memory vs Model-based
bull collaborative filtering lsquoneighborhoodsrsquo of people with similar interest amp recommending items based on likings in neighborhood13
bull limitations next to lsquocold startrsquo and lsquosparsityrsquo the lack of control (over onersquos neighborhood) is also a problem ie cannot add lsquotrustedrsquo people nor exclude lsquostrangersquo ones13
bull therefore interest in lsquosocial recommendersrsquo where presence of social connections defines the similarity in interests (eg social tagging CiteULike)13
bull does a social connection indicate user interest similarity13
bull how much users interest similarity depends on the strength of their connection13
bull is it feasible to use a social network as a personalized recommendation
[Lin amp Brusilovsky Social Networks and Interest Similarity The Case of CiteULike HTrsquo10]Social Web 2014 Lora Aroyo
Social Networks amp Interest Similarity
bull unilaterally connected pairs have more common itemsmetadatatags than non-connected pairs13
bull highest similarity for direct connections - decreasing with the increase of distance between users in SN 13
bull reciprocal relationship users - significantly larger similarity than users in a unidirectional relationship 13
bull traditional item-level similarity may be less reliable to find similar users in social bookmarking systems13
bull peers connected by self-defined social connections could be a useful source for cross-recommendation
Social Web 2014 Lora Aroyo
Conclusions
bull Input characteristics of items amp interests of a user into characteristics of items =gt Recommend items that feature characteristics which meet the userrsquos interests13
bull Techniques13
bull Data mining methods Cluster items based on their characteristics =gt Infer usersrsquo interests into clusters13
bull IR methods Represent items amp users as term vectors =gt Compute similarity between user profile vector and items13
bull Utility-based methods Utility function that gets an item as input the parameters of the utility function are customized via preferences of a user
Social Web 2014 Lora Aroyo
Content-based Recommendations
Government stops renovation of tower bridge Oct 13th 2011
Tower Bridge today Under construction
Tower Bridge is a combined bascule and suspension bridge in London England over the River Thames
Category politics england Related Twiper news bob Why do they stop tohellip [more] mary London stops renohellip [more]
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
02 0 0
02 04 01 01
= a
Weighting strategy - occurrence frequency - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
Content Features
RT Government stops renovation of tower bridge Oct 13th 2011
Userrsquos Twitter history
I am in London at the moment Oct 13th 2011
I am doing sports Oct 12th 2011
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
0 01 0
05 02 02 0
= u
Weighting strategy - occurrence frequency (eg smoothened by occurrence time recent concepts are more important - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
User Model
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
u 0
01 0
05 02 02 0
candidate items user a
02 0 0
02 04 01 01
b 0 0 0
08 02 0 0
c 0
05 02 0 0 0
03
cosine similarities
a b c
u 067 092 014
Ranking of recommended items 1 b 2 a 3 c
based on slides from Fabien Abel
Recommendations
Social Web 2014 Lora Aroyo
RecSys Issues
bull Cold-start problem (new user problem) nolittle data available to infer preferences of new users
bull Changing User Preferences user interests may change over time
bull Sparsity problem (new item problem) item descriptions are sparse eg not many user rated or tagged an item
bull Lack of Diversity (overfitting) when adapting too strongly to the preferences of users they might see samesimilar recommendations
bull Use the right context users do things which might not be relevant for their user model eg try out things do stuff for other people
bull Research challenge right balance between serendipity amp personalization
bull Research challenge right way to use the influence of recommendations on userrsquos behavior
one machine13vs 13
humans
image source httpwwwflickrcomphotosbionicteaching1375254387Social Web 2014 Lora Aroyo
Hands-on Teaser
bull Your Facebook Friendsrsquo popularity in a spread sheet13bull Locations of your Facebook Friends13bull Tag Cloud of your wall posts
bull collaborative filtering lsquoneighborhoodsrsquo of people with similar interest amp recommending items based on likings in neighborhood13
bull limitations next to lsquocold startrsquo and lsquosparsityrsquo the lack of control (over onersquos neighborhood) is also a problem ie cannot add lsquotrustedrsquo people nor exclude lsquostrangersquo ones13
bull therefore interest in lsquosocial recommendersrsquo where presence of social connections defines the similarity in interests (eg social tagging CiteULike)13
bull does a social connection indicate user interest similarity13
bull how much users interest similarity depends on the strength of their connection13
bull is it feasible to use a social network as a personalized recommendation
[Lin amp Brusilovsky Social Networks and Interest Similarity The Case of CiteULike HTrsquo10]Social Web 2014 Lora Aroyo
Social Networks amp Interest Similarity
bull unilaterally connected pairs have more common itemsmetadatatags than non-connected pairs13
bull highest similarity for direct connections - decreasing with the increase of distance between users in SN 13
bull reciprocal relationship users - significantly larger similarity than users in a unidirectional relationship 13
bull traditional item-level similarity may be less reliable to find similar users in social bookmarking systems13
bull peers connected by self-defined social connections could be a useful source for cross-recommendation
Social Web 2014 Lora Aroyo
Conclusions
bull Input characteristics of items amp interests of a user into characteristics of items =gt Recommend items that feature characteristics which meet the userrsquos interests13
bull Techniques13
bull Data mining methods Cluster items based on their characteristics =gt Infer usersrsquo interests into clusters13
bull IR methods Represent items amp users as term vectors =gt Compute similarity between user profile vector and items13
bull Utility-based methods Utility function that gets an item as input the parameters of the utility function are customized via preferences of a user
Social Web 2014 Lora Aroyo
Content-based Recommendations
Government stops renovation of tower bridge Oct 13th 2011
Tower Bridge today Under construction
Tower Bridge is a combined bascule and suspension bridge in London England over the River Thames
Category politics england Related Twiper news bob Why do they stop tohellip [more] mary London stops renohellip [more]
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
02 0 0
02 04 01 01
= a
Weighting strategy - occurrence frequency - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
Content Features
RT Government stops renovation of tower bridge Oct 13th 2011
Userrsquos Twitter history
I am in London at the moment Oct 13th 2011
I am doing sports Oct 12th 2011
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
0 01 0
05 02 02 0
= u
Weighting strategy - occurrence frequency (eg smoothened by occurrence time recent concepts are more important - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
User Model
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
u 0
01 0
05 02 02 0
candidate items user a
02 0 0
02 04 01 01
b 0 0 0
08 02 0 0
c 0
05 02 0 0 0
03
cosine similarities
a b c
u 067 092 014
Ranking of recommended items 1 b 2 a 3 c
based on slides from Fabien Abel
Recommendations
Social Web 2014 Lora Aroyo
RecSys Issues
bull Cold-start problem (new user problem) nolittle data available to infer preferences of new users
bull Changing User Preferences user interests may change over time
bull Sparsity problem (new item problem) item descriptions are sparse eg not many user rated or tagged an item
bull Lack of Diversity (overfitting) when adapting too strongly to the preferences of users they might see samesimilar recommendations
bull Use the right context users do things which might not be relevant for their user model eg try out things do stuff for other people
bull Research challenge right balance between serendipity amp personalization
bull Research challenge right way to use the influence of recommendations on userrsquos behavior
one machine13vs 13
humans
image source httpwwwflickrcomphotosbionicteaching1375254387Social Web 2014 Lora Aroyo
Hands-on Teaser
bull Your Facebook Friendsrsquo popularity in a spread sheet13bull Locations of your Facebook Friends13bull Tag Cloud of your wall posts
bull unilaterally connected pairs have more common itemsmetadatatags than non-connected pairs13
bull highest similarity for direct connections - decreasing with the increase of distance between users in SN 13
bull reciprocal relationship users - significantly larger similarity than users in a unidirectional relationship 13
bull traditional item-level similarity may be less reliable to find similar users in social bookmarking systems13
bull peers connected by self-defined social connections could be a useful source for cross-recommendation
Social Web 2014 Lora Aroyo
Conclusions
bull Input characteristics of items amp interests of a user into characteristics of items =gt Recommend items that feature characteristics which meet the userrsquos interests13
bull Techniques13
bull Data mining methods Cluster items based on their characteristics =gt Infer usersrsquo interests into clusters13
bull IR methods Represent items amp users as term vectors =gt Compute similarity between user profile vector and items13
bull Utility-based methods Utility function that gets an item as input the parameters of the utility function are customized via preferences of a user
Social Web 2014 Lora Aroyo
Content-based Recommendations
Government stops renovation of tower bridge Oct 13th 2011
Tower Bridge today Under construction
Tower Bridge is a combined bascule and suspension bridge in London England over the River Thames
Category politics england Related Twiper news bob Why do they stop tohellip [more] mary London stops renohellip [more]
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
02 0 0
02 04 01 01
= a
Weighting strategy - occurrence frequency - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
Content Features
RT Government stops renovation of tower bridge Oct 13th 2011
Userrsquos Twitter history
I am in London at the moment Oct 13th 2011
I am doing sports Oct 12th 2011
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
0 01 0
05 02 02 0
= u
Weighting strategy - occurrence frequency (eg smoothened by occurrence time recent concepts are more important - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
User Model
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
u 0
01 0
05 02 02 0
candidate items user a
02 0 0
02 04 01 01
b 0 0 0
08 02 0 0
c 0
05 02 0 0 0
03
cosine similarities
a b c
u 067 092 014
Ranking of recommended items 1 b 2 a 3 c
based on slides from Fabien Abel
Recommendations
Social Web 2014 Lora Aroyo
RecSys Issues
bull Cold-start problem (new user problem) nolittle data available to infer preferences of new users
bull Changing User Preferences user interests may change over time
bull Sparsity problem (new item problem) item descriptions are sparse eg not many user rated or tagged an item
bull Lack of Diversity (overfitting) when adapting too strongly to the preferences of users they might see samesimilar recommendations
bull Use the right context users do things which might not be relevant for their user model eg try out things do stuff for other people
bull Research challenge right balance between serendipity amp personalization
bull Research challenge right way to use the influence of recommendations on userrsquos behavior
one machine13vs 13
humans
image source httpwwwflickrcomphotosbionicteaching1375254387Social Web 2014 Lora Aroyo
Hands-on Teaser
bull Your Facebook Friendsrsquo popularity in a spread sheet13bull Locations of your Facebook Friends13bull Tag Cloud of your wall posts
bull Input characteristics of items amp interests of a user into characteristics of items =gt Recommend items that feature characteristics which meet the userrsquos interests13
bull Techniques13
bull Data mining methods Cluster items based on their characteristics =gt Infer usersrsquo interests into clusters13
bull IR methods Represent items amp users as term vectors =gt Compute similarity between user profile vector and items13
bull Utility-based methods Utility function that gets an item as input the parameters of the utility function are customized via preferences of a user
Social Web 2014 Lora Aroyo
Content-based Recommendations
Government stops renovation of tower bridge Oct 13th 2011
Tower Bridge today Under construction
Tower Bridge is a combined bascule and suspension bridge in London England over the River Thames
Category politics england Related Twiper news bob Why do they stop tohellip [more] mary London stops renohellip [more]
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
02 0 0
02 04 01 01
= a
Weighting strategy - occurrence frequency - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
Content Features
RT Government stops renovation of tower bridge Oct 13th 2011
Userrsquos Twitter history
I am in London at the moment Oct 13th 2011
I am doing sports Oct 12th 2011
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
0 01 0
05 02 02 0
= u
Weighting strategy - occurrence frequency (eg smoothened by occurrence time recent concepts are more important - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
User Model
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
u 0
01 0
05 02 02 0
candidate items user a
02 0 0
02 04 01 01
b 0 0 0
08 02 0 0
c 0
05 02 0 0 0
03
cosine similarities
a b c
u 067 092 014
Ranking of recommended items 1 b 2 a 3 c
based on slides from Fabien Abel
Recommendations
Social Web 2014 Lora Aroyo
RecSys Issues
bull Cold-start problem (new user problem) nolittle data available to infer preferences of new users
bull Changing User Preferences user interests may change over time
bull Sparsity problem (new item problem) item descriptions are sparse eg not many user rated or tagged an item
bull Lack of Diversity (overfitting) when adapting too strongly to the preferences of users they might see samesimilar recommendations
bull Use the right context users do things which might not be relevant for their user model eg try out things do stuff for other people
bull Research challenge right balance between serendipity amp personalization
bull Research challenge right way to use the influence of recommendations on userrsquos behavior
one machine13vs 13
humans
image source httpwwwflickrcomphotosbionicteaching1375254387Social Web 2014 Lora Aroyo
Hands-on Teaser
bull Your Facebook Friendsrsquo popularity in a spread sheet13bull Locations of your Facebook Friends13bull Tag Cloud of your wall posts
Government stops renovation of tower bridge Oct 13th 2011
Tower Bridge today Under construction
Tower Bridge is a combined bascule and suspension bridge in London England over the River Thames
Category politics england Related Twiper news bob Why do they stop tohellip [more] mary London stops renohellip [more]
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
02 0 0
02 04 01 01
= a
Weighting strategy - occurrence frequency - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
Content Features
RT Government stops renovation of tower bridge Oct 13th 2011
Userrsquos Twitter history
I am in London at the moment Oct 13th 2011
I am doing sports Oct 12th 2011
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
0 01 0
05 02 02 0
= u
Weighting strategy - occurrence frequency (eg smoothened by occurrence time recent concepts are more important - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
User Model
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
u 0
01 0
05 02 02 0
candidate items user a
02 0 0
02 04 01 01
b 0 0 0
08 02 0 0
c 0
05 02 0 0 0
03
cosine similarities
a b c
u 067 092 014
Ranking of recommended items 1 b 2 a 3 c
based on slides from Fabien Abel
Recommendations
Social Web 2014 Lora Aroyo
RecSys Issues
bull Cold-start problem (new user problem) nolittle data available to infer preferences of new users
bull Changing User Preferences user interests may change over time
bull Sparsity problem (new item problem) item descriptions are sparse eg not many user rated or tagged an item
bull Lack of Diversity (overfitting) when adapting too strongly to the preferences of users they might see samesimilar recommendations
bull Use the right context users do things which might not be relevant for their user model eg try out things do stuff for other people
bull Research challenge right balance between serendipity amp personalization
bull Research challenge right way to use the influence of recommendations on userrsquos behavior
one machine13vs 13
humans
image source httpwwwflickrcomphotosbionicteaching1375254387Social Web 2014 Lora Aroyo
Hands-on Teaser
bull Your Facebook Friendsrsquo popularity in a spread sheet13bull Locations of your Facebook Friends13bull Tag Cloud of your wall posts
RT Government stops renovation of tower bridge Oct 13th 2011
Userrsquos Twitter history
I am in London at the moment Oct 13th 2011
I am doing sports Oct 12th 2011
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
0 01 0
05 02 02 0
= u
Weighting strategy - occurrence frequency (eg smoothened by occurrence time recent concepts are more important - normalize vectors (1-norm sum of vector equals 1)
based on slides from Fabien Abel
User Model
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
u 0
01 0
05 02 02 0
candidate items user a
02 0 0
02 04 01 01
b 0 0 0
08 02 0 0
c 0
05 02 0 0 0
03
cosine similarities
a b c
u 067 092 014
Ranking of recommended items 1 b 2 a 3 c
based on slides from Fabien Abel
Recommendations
Social Web 2014 Lora Aroyo
RecSys Issues
bull Cold-start problem (new user problem) nolittle data available to infer preferences of new users
bull Changing User Preferences user interests may change over time
bull Sparsity problem (new item problem) item descriptions are sparse eg not many user rated or tagged an item
bull Lack of Diversity (overfitting) when adapting too strongly to the preferences of users they might see samesimilar recommendations
bull Use the right context users do things which might not be relevant for their user model eg try out things do stuff for other people
bull Research challenge right balance between serendipity amp personalization
bull Research challenge right way to use the influence of recommendations on userrsquos behavior
one machine13vs 13
humans
image source httpwwwflickrcomphotosbionicteaching1375254387Social Web 2014 Lora Aroyo
Hands-on Teaser
bull Your Facebook Friendsrsquo popularity in a spread sheet13bull Locations of your Facebook Friends13bull Tag Cloud of your wall posts
dbPolitics dbSports
dbEducation dbLondon
dbTower_Bridge dbGovernment
dbUK
u 0
01 0
05 02 02 0
candidate items user a
02 0 0
02 04 01 01
b 0 0 0
08 02 0 0
c 0
05 02 0 0 0
03
cosine similarities
a b c
u 067 092 014
Ranking of recommended items 1 b 2 a 3 c
based on slides from Fabien Abel
Recommendations
Social Web 2014 Lora Aroyo
RecSys Issues
bull Cold-start problem (new user problem) nolittle data available to infer preferences of new users
bull Changing User Preferences user interests may change over time
bull Sparsity problem (new item problem) item descriptions are sparse eg not many user rated or tagged an item
bull Lack of Diversity (overfitting) when adapting too strongly to the preferences of users they might see samesimilar recommendations
bull Use the right context users do things which might not be relevant for their user model eg try out things do stuff for other people
bull Research challenge right balance between serendipity amp personalization
bull Research challenge right way to use the influence of recommendations on userrsquos behavior
one machine13vs 13
humans
image source httpwwwflickrcomphotosbionicteaching1375254387Social Web 2014 Lora Aroyo
Hands-on Teaser
bull Your Facebook Friendsrsquo popularity in a spread sheet13bull Locations of your Facebook Friends13bull Tag Cloud of your wall posts
Social Web 2014 Lora Aroyo
RecSys Issues
bull Cold-start problem (new user problem) nolittle data available to infer preferences of new users
bull Changing User Preferences user interests may change over time
bull Sparsity problem (new item problem) item descriptions are sparse eg not many user rated or tagged an item
bull Lack of Diversity (overfitting) when adapting too strongly to the preferences of users they might see samesimilar recommendations
bull Use the right context users do things which might not be relevant for their user model eg try out things do stuff for other people
bull Research challenge right balance between serendipity amp personalization
bull Research challenge right way to use the influence of recommendations on userrsquos behavior
one machine13vs 13
humans
image source httpwwwflickrcomphotosbionicteaching1375254387Social Web 2014 Lora Aroyo
Hands-on Teaser
bull Your Facebook Friendsrsquo popularity in a spread sheet13bull Locations of your Facebook Friends13bull Tag Cloud of your wall posts
one machine13vs 13
humans
image source httpwwwflickrcomphotosbionicteaching1375254387Social Web 2014 Lora Aroyo
Hands-on Teaser
bull Your Facebook Friendsrsquo popularity in a spread sheet13bull Locations of your Facebook Friends13bull Tag Cloud of your wall posts
image source httpwwwflickrcomphotosbionicteaching1375254387Social Web 2014 Lora Aroyo
Hands-on Teaser
bull Your Facebook Friendsrsquo popularity in a spread sheet13bull Locations of your Facebook Friends13bull Tag Cloud of your wall posts