hybrid web recommender systems robin burke presentation by jae-wook ahn 10/04/05

44
Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05

Upload: margery-thomas

Post on 28-Dec-2015

225 views

Category:

Documents


1 download

TRANSCRIPT

Page 1: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05

Hybrid Web Recommender Systems

Robin Burke

Presentation by Jae-wook Ahn10/04/05

Page 2: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05

10/5/05 Hybrid Web Recommender Systems 2

References

• Entrée system & dataset• Burke, R. (2002). Semantic ratings and heuristic similarity for colla

borative filtering. AAAI Workshop on Knowledge-based Electronic Markets 2000.

• Feature augmentation, mixed hybrid example• Torres, R., McNee, S., Abel, M., Konstan J., & Riedl J. (2004). En

hancing Digital Libraries with TechLens+. Proceedings of the 2004 Joint ACM/IEEE Conference on Digital Libraries.

• Hybrid recommender system UI issue• Schafer, J. (2005). DynamicLens: A Dynamic User-Interface for a

Meta-Recommendation System. Workshop: Beyond Personalization 2005, IUI’05.

• Collaborative filtering algorithm• Sarwar, B., Karypis, G., Konstan, J., & Riedl, J. (2001). Item-based

collaborative filtering recommendation algorithms. In Proceedings of the 10th international conference on World Wide Web.

Page 3: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05

Concepts and Techniques

Page 4: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05

10/5/05 Hybrid Web Recommender Systems 4

Hybrid Recommender Systems

• Mix of recommender systems • Recommender system classification – knowledge source

• Collaborative (CF)• User’s ratings “only”

• Content-based (CN)• Product features, user’s ratings• Classifications of user’s likes/dislikes

• Demographic• User’s ratings, user’s demographics

• Knowledge-based (KB)• Domain knowledge, product features, user’s need/query• Inferences about a use’s needs and preferences

Page 5: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05

10/5/05 Hybrid Web Recommender Systems 5

CF vs. CN

• User-based CF• Searches for similar users

in user-item “rating” matrix

• Item-based CF• Searches for similar items

in user-item “rating” matrix

• CN• Searches for similar items

in item-feature matrix• Example – TF*IDF term

weight vector for news recommendation

Items

Users Ratings

Page 6: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05

10/5/05 Hybrid Web Recommender Systems 6

Recommender System Problems

• Cold-start problem • Learning based techniques• Collaborative, content-based, demographic Hybrid techniques

• Stability vs. plasticity problem• Difficulty to change established user’s profile Temporal discount – older rating with less influence

KB – fewer cold start problem (no need of historical data)

CF/Demographic – cross-genre niches, jump outside of the familiar (novelty, serendipity)

Page 7: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05

10/5/05 Hybrid Web Recommender Systems 7

Strategies for Hybrid Recommendation

• Combination of multiple recommendation techniques together for producing output

• Different techniques of different types• Most common implementations

• Most promise to resolve cold-start problem

• Different techniques of the same type• Ex) NewsDude – naïve Bayes + kNN

Page 8: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05

10/5/05 Hybrid Web Recommender Systems 8

Seven Types of Recommender Systems

• Taxonomy by Burke (2002)

1. Weighted 2. Switching3. Mixed4. Feature combination5. Feature augmentation6. Cascade7. Meta-level

Page 9: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05

10/5/05 Hybrid Web Recommender Systems 9

Weighted Hybrid

• Concept

• Each component of the hybrid scores a given item and the scores are combined using a linear formula

• When recommenders have consistent relative accuracy across the product space

• Uniform performance among recommenders (otherwise other hybrids)

Page 10: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05

10/5/05 Hybrid Web Recommender Systems 10

Weighted Hybrid Procedure

1. Training

2. Joint rating• Intersection –

candidates shared between the candidates

• Union – case with no possible rating neutral score (neither liked nor disliked)

3. Linear combination

Page 11: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05

10/5/05 Hybrid Web Recommender Systems 11

Mixed Hybrid

• Concepts• Presentation of different components side-by-side in a

combined list

• If lists are to be combined, how are rankings to be integrated?• Merging based on predicted rating or on recommender

confidence

• Not fit with retrospective data• Cannot use actual ratings to test if right items ranked highly

• Example• CF_rank(3) + CN_rank(2) Mixed_rank(5)

Page 12: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05

10/5/05 Hybrid Web Recommender Systems 12

Mixed Hybrid Procedure

1. Candidate generation

2. Multiple ranked lists

3. Combined display

Page 13: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05

10/5/05 Hybrid Web Recommender Systems 13

Switching Hybrid

• Concepts

• Selects a single recommender among components based on recommendation situation

• Different profile different recommendation

• Components with different performance for some types of users

• Existence of criterion for switching decision• Ex) confidence value, external criteria

Page 14: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05

10/5/05 Hybrid Web Recommender Systems 14

Switching Hybrid Procedure

1. Switching decision

2. Candidate generation

3. Scoring

• No role for unchosen recommender

Page 15: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05

10/5/05 Hybrid Web Recommender Systems 15

Feature Combination Hybrid

• Concepts

• Inject features of one source into a different source for processing different data

• Features of “contributing recommender” are used as a part of the “actual recommender”

• Adding new features into the mix

• Not combining components, just combining knowledge source

Page 16: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05

10/5/05 Hybrid Web Recommender Systems 16

Feature Combination Hybrid Procedure

1. Feature combination In training stage

2. Candidate generation

3. Scoring

Page 17: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05

10/5/05 Hybrid Web Recommender Systems 17

Feature Augmentation Hybrid

• Concepts• Similar to Feature Combination

• Generates new features for each item by contributing domain

• Augmentation/combination – done offline

• Comparison with Feature Combination• Not raw features (FC), but the result of computation

from contribution (FA)

• More flexible to apply

• Adds smaller dimension

Page 18: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05

10/5/05 Hybrid Web Recommender Systems 18

Feature Augmentation Hybrid Procedure

Page 19: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05

10/5/05 Hybrid Web Recommender Systems 19

Cascade Hybrid

• Concepts• Tie breaker

• Secondary recommender• Just tie breaker• Do refinements

• Primary recommender• Integer-valued scores – higher probability for ties• Real-valued scores – low probability for ties• Precision reduction

• Score: 0.8348694 0.83

Page 20: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05

10/5/05 Hybrid Web Recommender Systems 20

Cascade Hybrid Procedure

• Procedure1. Primary recommender

2. Ranks

3. Break ties by secondary recommender

Page 21: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05

10/5/05 Hybrid Web Recommender Systems 21

Meta-level Hybrid

• Concepts

• A model learned by contributing recommender

input for actual recommender

• Contributing recommender completely replaces the original knowledge source with a learned model

• Not all recommenders can produce the intermediary model

Page 22: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05

10/5/05 Hybrid Web Recommender Systems 22

Meta-level Hybrid Procedure

• Procedure

1. Contributing recommender

Learned model

2. Knowledge Source Replacement

3. Actual Recommender

Page 23: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05

Experiments

Page 24: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05

10/5/05 Hybrid Web Recommender Systems 24

Testbed – Entrée Restaurant Recommender

• Entrée System• Case-based reasoning• Interactive critiquing dialog

• Ex) Entry Candidates “Cheaper” Candidates “Nicer” Candidates Exit

• Not “narrowing” the search by adding constrains, but changing the focus in the feature space

Page 25: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05

10/5/05 Hybrid Web Recommender Systems 25

Testbed – Entrée Restaurant Recommender (cont’d)

• Entrée Dataset• Rating

• Entry, ending point – “positive” rating

• Critiques – “negative” rating

• Mostly negative ratings

• Validity test for positive ending point assumption – strong correlation between original vs. modified (entry points with positive ratings)

• Small in size

Page 26: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05

10/5/05 Hybrid Web Recommender Systems 26

Evaluation Methodology

• Measures • ARC (Average Rank of the Correct recommendations)

• Accuracy of retrieval • At different size retrieval set• Fraction of the candidate set (0 ~ 1.0)

• Training & Test set• 5 fold cross validation – random partition of training/test set

• “Leave one out” methodology – randomly remove one item and check whether the system can recommend it

• Sessions Sizes• Single visit profiles – 5S, 10S, 15S

• Multiple visit profiles – 10M, 20M, 30M

Page 27: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05

10/5/05 Hybrid Web Recommender Systems 27

Baseline Algorithms• Collaborative Pearson (CFP)

• Pearson’s correlation coefficient for similarity• Collaborative Heuristic (CFH)

• Heuristics for calculating distances between critiques• “nicer” and “cheaper” dissimilar• “nicer” & “quieter” similar

• Content-based (CN)• Naïve Bayes algorithm – compute probability that a item is “liked” / “d

isliked”• Too few “liked” items modified candidate generation

• Retrieve items with common features with the “liked” vector of the naïve Bayes profile

• Knowledge-based (KB)• Knowledge-based comparison metrics of Entrée• Nationality, price, atmosphere, etc.

Page 28: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05

10/5/05 Hybrid Web Recommender Systems 28

Baseline Evaluations

• Techniques vary in performance on the Entrée data• Content-based (CN) –

weak• Knowledge-based (KB) –

better on single-session than multi-session

• Heuristic collaborative (CFH) – better than correlation-based (CFP) for short profiles

• Room for improvement• Multi-session profiles

Page 29: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05

10/5/05 Hybrid Web Recommender Systems 29

Baseline Evaluations

Page 30: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05

10/5/05 Hybrid Web Recommender Systems 30

Hybrid Comparative Study

• Missing components

• Mixed hybrid• Not possible with retrospective data

• Demographic recommender• No demographic data

Page 31: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05

10/5/05 Hybrid Web Recommender Systems 31

Results – Weighted

• Hybrid performance better in only 10 of 30

• CN/CFP – consistent synergy (5 of 6)

• Lacks uniform performance • KB, CFH

• Linear weighting scheme assumption – fault

Page 32: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05

10/5/05 Hybrid Web Recommender Systems 32

Results – Switching

• KB hybrids – best switching hybrids

Page 33: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05

10/5/05 Hybrid Web Recommender Systems 33

Results – Feature Combination

• CN/CFH, CN/CFP• Contributing CN

• Identical to CFH, CFP

• CFH maintains accuracy with reduced dataset

• CF/CN Winnow – modest improvement

Page 34: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05

10/5/05 Hybrid Web Recommender Systems 34

Results – Feature Augmentation

• Best performance so far• Particularly CN*/CF*

• Good for multi-session profiles

Page 35: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05

10/5/05 Hybrid Web Recommender Systems 35

Results – Cascade

• CFP/KB, CFP/CN• Great improvement

• Also good for multi-profile sessions

Page 36: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05

10/5/05 Hybrid Web Recommender Systems 36

Results – Meta-level Hybrids

• CN/CF, CN/KB, CF/KB, CF/CN

• Not effective• No synergy

• Weakness of KB/CN in Entrée dataset

• Both components should be strong

Page 37: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05

10/5/05 Hybrid Web Recommender Systems 37

Discussion

• Dominance of the hybrids over basic recommenders

• Synergy was found under• Smaller profile size

• Sparse recommendation density

hybridization conquers cold start problem

Page 38: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05

10/5/05 Hybrid Web Recommender Systems 38

Discussion (cont’d)

• Best hybrids• Feature augmentation, cascade

• FA allows a contributing recommender to make a positive impact • without interfering with the performance of the better algorithm

Page 39: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05

10/5/05 Hybrid Web Recommender Systems 39

Conclusions

• Knowledge-based recommendation is not limited• Numerously combined to build hybrids

• Good for secondary or contributing components

• Cascade hybrids are effective• Though rare in literatures

• Effective for combining recommender with different strengths

• Different performance characteristics• Six hybridization techniques

• Relative accuracy & consistency of hybrid components

Page 40: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05

System Example & Related Issues

Page 41: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05

10/5/05 Hybrid Web Recommender Systems 41

System Example – TechLens+• Hybrid recommender system

• Recommenders – CF, CN• Hybrid algorithms – CF/CN FA, CN/CF FA, Fusion (Mixed)

• Corpus• CiteSeer• Title, abstract (CN), citations (CF)

• Methodology• Offline experiment, Online user study with questionnaire (by asking satisfa

ction on the recommendation)• Results

• Fusion was the best• Some FA were not good due the their sequential natures• Different algorithms should be used for recommending different papers• Users with different levels of experiences perceive recommendations differ

ently

Page 42: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05

10/5/05 Hybrid Web Recommender Systems 42

Meta-recommender – DynamicLens

• Can user provided information improve hybrid recommender system output?

• Meta-recommender• Provide users with personal

ized control over the generation of a recommendation list from hybrid recommender system

• MetaLens• IF (Information Filtering),

CF

Page 43: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05

10/5/05 Hybrid Web Recommender Systems 43

Meta-recommender – DynamicLens (cont’d)

• Dynamic query• Merges preference & recommendation interfaces

• Immediate feedback

• Discover why a given set of ranking recommendations were made

Page 44: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05

Questions & Comments