a user-centric evaluation of recommender algorithms for an event recommendation system
Post on 10-May-2015
452 Views
Preview:
DESCRIPTION
TRANSCRIPT
A User-centric Evaluation of Recommender Algorithms for an Event Recommendation SystemSimon Dooms, Toon De Pessemier, Luc Martens
@sidooms
Introduction
10/23/2011 Simon Dooms - Ghent University - UCERSTI 2 2
• Online context• Cultural events• 30,000 items
Recommender System?
Concl.ResultsQuestionsAlgorithmsFeedbackExperimentIntroIntro
The Experiment
Invitation mailDAY 1
Closed questionnaireDAY 56
DAY 41
End tracking
DAY 45
Send out recs
DAY 50Reminder mail
Reminder mailDAY 28
Concl.ResultsQuestionsAlgorithmsFeedbackIntroExperimentExperiment
10/23/2011 Simon Dooms - Ghent University - UCERSTI 2 3
FeedbackConcl.ResultsQuestionsAlgorithms
FeedbackFeedbackExperimentIntro
10/23/2011 Simon Dooms - Ghent University - UCERSTI 2 4
Algorithms
Content-Based CBUser-Based Collaborative Filtering UBCFHybrid recommender CB+UBCFSingular Value Decomposition SVDRandom recommender RAND
Concl.ResultsQuestionsAlgorithmsAlgorithmsFeedbackExperimentIntro
10/23/2011 Simon Dooms - Ghent University - UCERSTI 2 5
QuestionnaireConcl.Results
QuestionsQuestionsAlgorithmsFeedbackExperimentIntro
10/23/2011 Simon Dooms - Ghent University - UCERSTI 2 6
AccuracyFamiliarity
NoveltyDiversity
TransparencySatisfaction
TrustUsefulness
Results
612 users
Concl.ResultsResultsQuestionsAlgorithmsFeedbackExperimentIntro
10/23/2011 Simon Dooms - Ghent University - UCERSTI 2 7
232 users193 users
Concl.ResultsResultsQuestionsAlgorithmsFeedbackExperimentIntro
Average scores per question and algorithm
Accuracy Familiarity Novelty Diversity Trans. Satisfaction Trust Usefulness10/23/2011 Simon Dooms - Ghent University - UCERSTI 2 8
Concl.ResultsResultsQuestionsAlgorithmsFeedbackExperimentIntro
Average scores per question and algorithm
Accuracy Satisfaction910/23/2011 Simon Dooms - Ghent University - UCERSTI 2
CorrelationsConcl.
ResultsResultsQuestionsAlgorithmsFeedbackExperimentIntro
10/23/2011 Simon Dooms - Ghent University - UCERSTI 2 10
AccuracyFamiliarity
NoveltyDiversity
TransparencySatisfaction
TrustUsefulness
RegressionsConcl.
ResultsResultsQuestionsAlgorithmsFeedbackExperimentIntro
10/23/2011 Simon Dooms - Ghent University - UCERSTI 2 11
Conclusions
• Online user-centric evaluation experiment• 5 popular algorithms: CB, UBCF, CB+UBCF, SVD, RAND
• CB+UBCF winner algorithm• SVD ‘loser’ algorithm• Accuracy and Transparency Satisfaction• Diversity NOT correlated with Satisfaction
Concl.Concl.ResultsQuestionsAlgorithmsFeedbackExperimentIntro
10/23/2011 Simon Dooms - Ghent University - UCERSTI 2 12
Simon Dooms, Toon De Pessemier, Luc Martens
A User-centric Evaluationof Recommender Algorithms for an Event Recommendation System
With the support of IWT Vlaanderen and FWO Vlaanderen
top related