[sac2014]splitting approaches for context-aware recommendation: an empirical study

Splitting Approaches for Context-Aware Recommendation:

An Empirical Study

Yong Zheng, Robin Burke, Bamshad MobasherCenter for Web Intelligence

DePaul University, Chicago, IL USA

ACM SIGAPP the 29th Symposium On Applied Computing

Gyeongju, South Korea, March 26, 2014

Center for Web Intelligence DePaul University, Chicago, IL USA

Contents

Context-aware Recommender Systems

Context-aware Splitting Approaches

Empirical Study & Evaluation Results

Discussions, Conclusions & Future work



2


Contents







3

Recommender Systems

Recommender Systems (RS)

Two-dimension rating space: Users × Items Ratings


M1 M2 M3

U1

U2

U3

U4

4

?


Context-aware Recommender Systems (CARS)

Multi-dimensional space: Users × Items × Contexts Ratings

Center for Web Intelligence DePaul University, Chicago, IL USA5



Assumptions and Viewpoints in CARS:

Users’ preferences or decisions usually differ from contexts to contexts,

even towards the same item. E.g. buy a gift for someone.

It’s better to infer user’s preferences by rating profiles within the same or

similar contexts. E.g. look at music others choose within same contexts

Context is defined as “any information that can be used to characterize

the situation of an entity” by Dey, Anind K. (2001).However, the actual

contexts in CARS and the contextual effects are domain specific.

Movie domain: time, location, companion, mood, etc

Music domain: time, activity, mood, etc

Travel domain: season, weather, companion or trip type, etc




Example of CARS applications: Tour Plan Recommender

Yu, Chien-Chih and Chang, Hsiao-ping, "Towards Context-Aware Recommendation for Personalized

Mobile Travel Planning". International Conference on Context-Aware Systems and Applications, 2012



How to incorporate contexts into RS?

There are two methods to categorize those incorporations.

1).In terms of how contexts interacted with the RS algorithms



How to incorporate contexts into RS?

There are two methods to categorize those incorporations.

2).In terms of whether new CARS algorithms required to be developed

It can be simply categorized into:

a).Transformation Algorithms

A transformation is required, then all traditional RS algorithms can

be applied to. Do NOT need to develop new CARS algorithms.

such as Dimensions as Virtual Items (DaVI) and context-aware

splitting approaches (CASA).

b).Adaptation Algorithms

CARS algorithms are required, traditional algs can be modified.

Such as context-aware matrix factorization (CAMF).



Contents







10

Context-aware Splitting Approaches (CASA)


In terms of the two categorizations (i.e. how to incorporate contexts into

recommender systems) above, CASA belongs to pre-filtering and

transformation algorithms.

There are three context-aware splitting approaches:

1). Item splitting by L. Baltrunas, F. Ricci, ACM RecSys, 2009

2). User splitting by A. Said et al, CARS@ACM RecSys, 2011

3). UI splitting by Y. Zheng et al, Decisions@ACM RecSys, 2013

User splitting and UI splitting are two approaches derived from

Item splitting, examined and evaluated by different authors.

According to feedbacks from researchers, CASA is one of most efficient

CARS algorithms, but there are no empirical study over them.



Item SplittingThe underlying idea is that the nature of an item, from the user's point of

view, may change in different contextual conditions, hence it may be

useful to consider it as two different items. (L. Baltrunas, F. Ricci,

RecSys'09) – In short, contexts are dependent with items.

Any dependent patterns involved in those ratings?


At Cinema At Home At Swimming Pool

12


Item Splitting -- Example


User Item Location Rating

U1 M1 Pool 5

U2 M1 Pool 5

U3 M1 Pool 5

U1 M1 Home 2

U4 M1 Home 3

U2 M1 Home 2

High Rating

Low Rating

Significant difference?

Let’s split it !!!

M11: being seen at Pool

M12: being seen at Home

M1

Same movie,

different IDs.

13


Item Splitting

Step 1. Choose a contextual condition to split each item; The selection

process is done by measuring significance of rating differences (such

as the two-sample t test);

Step 2. Contexts are fused to items and removed from original

multidimensional matrix. We get a 2D rating matrix, then traditional

algorithms like CF, MF can be applied to;

How to select an appropriate contextual conditions for splitting?

a). Binary contextual condition

b). Impurity criteria and significance test

See example in the next.



Item Splitting – Binary Contextual Condition


User Item Loc Rating

U1 M1 Pool 5

U2 M1 Pool 5

U3 M1 Pool 5

U1 M1 Home 2

U4 M1 Home 3

U2 M1 Cinema 2

User Item Rating

U1 M11 5

U2 M11 5

U3 M11 5

U1 M12 2

U4 M12 3

U2 M12 2

Transformation

If there is qualified split,

one item will be split to

two new ones.

A binary contextual condition

for splitting:

“Pool” vs. “Non-Pool”

Why use a binary condition? To alleviate or avoid cold-start problems!

15


Item Splitting – Impurity CriteriaThere could be several binary context conditions, for example, “Pool” vs

“Non-Pool”, “Home” vs “Non-Home”, “Weekend” vs “Non-Weekend”.

Impurity criteria and significance test are used to make the selection.

There are 4 impurity criteria for splitting by L. Baltrunas, et al, RecSys'09;

tmean (t-test), tprop (z-test), tchi (chi-square test), tIG (Information gain)

Take tmean for example, tmean, is defined using the two-sample t test and

computes how significantly different are the means of the rating in the

two rating subsets, when the split c (c is a context condition, e.g.

location = Pool) is used. The bigger the t value of the test is, the

more likely the difference of the means in the two partitions is

significant (at 95% confidence value). Choose the largest one!



User Splitting and UI SplittingSimilarly, the splitting approach can be applied to user too!

• User Splitting: is a similar one. Instead of splitting items, it may be

useful to consider one user as two different users, if user demonstrates

significantly different preferences across contexts. (A. Said et al.,

CARS@RecSys 2011) In short, contexts are dependent with users.

• UI Splitting: simply a combination of item splitting and user splitting –

both approaches are applied to create a new rating matrix – new users

and new items are created in the rating matrix. (Y. Zheng, et al,

Decisions@ACM RecSys 2013). In short, it fuses dependent contexts

to users and items simultaneously at the same time.



An Example of Three CASA


After transformation:

Item Splitting: User + NewItem;

User Splitting: NewUser + Item;

UI Splitting: NewUser + NewItem;

UI Splitting fuses contexts to both

users and items, where it may

enlarge the contextual effects,

but it also increases sparsity. It is

hard to say whether UI splitting

will outperform the other two algs

or not. It varies from data to data.

18


Contents







19

Empirical Study and Evaluations

Experimental Goals

a> Comparison Among Three Splitting Approaches

Which one performs the best?

Which splitting criteria is the best appropriate one?

Any underlying patterns to indicate which one should be used?

b> Comparison Between CASA and other Contextual Algorithms

Which one performs the best?

How about CASA competing with other CARS algorithms?



Data Sets

Contextual variables in the three survey data sets:

Food data: degree of hungriness in real and supposed situations;

Movie data: Location (home/cinema), Time (weekend/weekday), Companion (family, etc);

LDOS-CoMoDa: Location, Time, Companion, Weather, Emotions, Seasons, etc

We use a 5-fold cross validation for all data sets and examined algorithms.



Baseline AlgorithmsWe choose two other context-aware algorithms as the baselines:

1). Differential Context Modeling (DCM) by Y. Zheng, et al, 2012

There are two approaches falling into this category:Differential Context Relaxation (DCR)Differential Context Weighting (DCW)

Basic idea: Using rating profiles with same or similar contexts for rating predictions;Take user-based collaborative filtering for example:

Segment alg to various components;

Apply context filter to each component;

Filters could be different, and not necessary

to be the same.

Filter could be realized by context relaxation

to find same contexts, or context weighting

to find similar contexts.

Generally, DCW works better than DCR.



Baseline AlgorithmsWe choose two other context-aware algorithms as the baselines:

2). Context-aware Matrix Factorization (CAMF) by L. Baltrunas, et al, 2011

There are three approaches falling into this category: CAMF_C, CAMF_CI, CAMF_CU

CAMF_C: Assume contextual effect is associated with each contextual condition only.CAMF_CI: Assume contextual effect is associated with item-context interactions.CAMF_CU: Assume contextual effect is associated with user-context interactions.

CAMF is a kind of contextual modeling approach, where context-aware splitting approaches are contextual pre-filtering approaches. Both of them take advantage of the dependency between contexts and users or items.



Context-aware Splitting Approaches (CASA)They are pre-filtering approaches. Any traditional recommendation algorithms can be applied to, after the original multi-dimensional rating matrix was transformed to a 2D rating matrix. We evaluate three CASA based on the configuration as follows:

1). Evaluated by different traditional RS algorithms

User-based Collaborative Filtering (UBCF), Item-based Collaborative Filtering (IBCF)Traditional Matrix Factorization techniques (MF) without taking contexts into consideration

Implemented and evaluated by open-source Toolkit MyMediaLite v3.07

CF algorithms were tuned up by varying # of neighbors;MF algorithms were examined by varying # of factors and training iterations;

2). Evaluated by different impurity criteria in splitting processestmean (t-test), tprop (z-test), tchi (chi-square test), tIG (Information gain)



Evaluation MetricsWe choose three metrics: RMSE, Precision, ROC.

RMSE is used to evaluate the accuracy of predicted ratings. Prediction error, is the most popular and common used metric in CARS area, since context-aware data are usually sparse and few users rated a same item for several times.

ROC = a visualization between recall and FPR by varying # of N in Top-N recommendations.= x axis is FPT, y axis is Recall

In measuring Precision and ROC, we use a rating threshold to judge “relevance”.For Movie data, the threshold is set as 7, and it is set as 3 for the other two data sets.



Evaluation MetricsTraditional way to measure Precision and ROC:

1).We have training and testing set;2).Train a model based on the training set and evaluate it on the testing set;3).In the testing set, aggregate rating profiles by <user, a list of items he/she rated>;4).For evaluation purpose, provide a list of ranked Top-N items to each user;5).And examine the hit ratio between the Top-N list and the list of items rated in the testing;

However, in CARS, contexts should be taken into account; CPrecision and CROC curve

1).We have training and testing set;2).Train a model based on the training set and evaluate it on the testing set;3).In the testing set, aggregate rating profiles by <user, a list of items he/she rated, contexts>;4).For evaluation purpose, provide a list of ranked Top-N items to each <user, contexts>;5).And examine the hit ratio between the Top-N list and the list of items rated in the testing;

NOTICE: <user, a list of items he/she rated, contexts>; the list is pretty short and even just one item, because users seldom rated items for several times within different contexts.

Thus the value of CPrecision and CROC will be much smaller than traditional ones.



Evaluation Challenge in CASA (Optional Part)

RMSE can be directly evaluated based on the transformed rating matrix in CASAIt is because the number of rating profiles in data is NOT changed.

CPrecision and CROC cannot be directly evaluated on the transformed rating matrix1). # of users and # of items could be DIFFERENT2). It is not comparable to other CARS algorithms

Solution: We only use transformed matrix to predict ratings, but evaluate IR metrics on the original multi-dimensional rating matrix.

User Item Loc Rating

U1 M1 Pool 5

U2 M1 Pool 5

U3 M1 Pool 5

U1 M1 Home 2

U4 M1 Home 3

U2 M1 Cinema 2

User Item Rating

U1 M11 5

U2 M11 5

U3 M11 5

U1 M12 2

U4 M12 3

U2 M12 2

Transformation


Experimental Results

Experimental Results (in RMSE)Goal-1: Comparisons among the three context-aware splitting approaches (in RMSE)

Q: Which one performs the best? The best impurity criteria?A: UI Splitting using MF as the recommendation algorithm. MF works better than CFs.The best choice varies from data to data. No consistent patterns.

Q: Any other patterns?A: For Movie data, item splitting is better than user splitting; But user splitting is better than item splitting for the other two ones, where they have emotional or feeling contextual variables, we assume those contexts are more dependent with users.



Experimental Results (in RMSE)Goal-2: Comparisons with other CARS algorithms (in terms of RMSE)

Q: Which one performs the best? The best impurity criteria?A: UI Splitting using MF as the recommendation algorithm in terms of RMSE.

Q: Any other patterns?A: If item splitting is better than user splitting, CAMF_CI is better than CAMF_CU;If user splitting is better than item splitting, then CAMF_CU is better than CAMF_CI;It is because both of them take advantage of context-dependency patterns!!



Experimental Results (CPrecision & CROC)

Goal-1: Comparisons among the three context-aware splitting approachesIn CPrecision, UI splitting > Item splitting > User Splitting;In ROC Curve, UI splitting > User splitting > Item Splitting;

Goal-2: Comparisons with other CARS algorithmsIn CPrecision, UI splitting > CAMF_CI > CAMF_CU > DCW > DCR;In ROC Curve, UI splitting > CAMF_CU > CAMF_CI > DCW > DCR;

Patterns:UI Splitting is the best in RMSE and IR metrics for LDOS-CoMoDa;Consistent findings in context-dependency pattern in EACH METRIC;In RMSE, context is more dependent with user;




Goal-1: Comparisons among the three context-aware splitting approachesIn CPrecision, Item splitting > UI splitting > User splitting;In ROC Curve, same patter as above;

Goal-2: Comparisons with other CARS algorithmsIn CPrecision, Item splitting > UI splitting > CAMF_CI > CAMF_CU > DCW;In ROC Curve, Item splitting > UI splitting > CAMF_CI > DCW > CAMF_CU;

Patterns:Item Splitting is the best in RMSE and IR metrics for Movie data;Consistent findings in context-dependency pattern in EACH METRIC;




Goal-1: Comparisons among the three context-aware splitting approachesIn CPrecision, UI splitting > User splitting > Item Splitting;In ROC Curve, same pattern as above;

Goal-2: Comparisons with other CARS algorithmsIn CPrecision, UI splitting > CAMF_CU > CAMF_CI > DCR > DCW;In ROC Curve, DCR > UI splitting > DCW > CAMF_CU > CAMF_CI

Patterns:Overall, UI Splitting is the best in RMSE and IR metrics for Food Data;Consistent findings in context-dependency pattern;



Contents







33

Conclusions

Conclusions & Future WorkWhich one performs the best?Generally speaking, UI splitting is the best;In Movie data, UI splitting is the best on RMSE, but item splitting is the best on IR metrics;If context is not that dependent with users, merging effects by UI splitting may decrease the joint effect on recommendations.

Any patterns or guidelines to choose which context-aware algorithms?In terms of choices between item splitting & user splitting, and CAMF_CI & CAMF_CU, it totally depends on which one contexts are more dependent to, user or item?

Whether UI splitting performs the best depends on three factors:1). The dependency between contexts and users and items;2). The sparsity after rating matrix transformation – cold-start problems in CASA;3). The performance difference between user splitting and item splitting. If one of them performs bad, it is not guaranteed that the joint effect UI splitting will perform better;

Future work: 1).how to judge contexts are more dependent with users or items? Any numeric metrics to validate it? PS: Impurity values? no consistent patterns.2). How to alleviate the cold-start problems in UI splitting.


Conclusions

ReferencesItem Splitting

1) L. Baltrunas, and F. Ricci. "Context-based splitting of item ratings in collaborative filtering." ACM RecSys, 2009.

2) L. Baltrunas, and F. Ricci. "Experimental evaluation of context-dependent collaborative filtering using item splitting." User Modeling and

User-Adapted Interaction (2013): 1-28.

User Splitting

1) L. Batrunas and X. Amatriain."Towards Time-Dependent Recommendation Based on Implicit Feedback." CARS@RecSys, 2009

2) A. Said, E. Luca, S. Albayrak. "Inferring contextual user profiles—improving recommender performance.“ CARS@RecSys, 2011

UI Splitting

1) Y. Zheng, R. Burke, B. Mobasher. "The Role of Emotions in Context-aware Recommendation". Decisons@RecSys, 2013

2) Y. Zheng, R. Burke, B. Mobasher, “Splitting Approaches for Context-Aware Recommendation: An Empirical Study”, ACM SAC, 2014

Context-aware Matrix Factorization

1) L. Baltrunas, B. Ludwig, F. Ricci. "Matrix factorization techniques for context aware recommendation." ACM RecSys 2011.

Differential Context Modeling

1) Y. Zheng, R. Burke, B. Mobasher. "Differential Context Relaxation for Context-aware Travel Recommendation". EC-WEB, 2012

2) Y. Zheng, R. Burke, B. Mobasher. "Optimal Feature Selection for Context-Aware Recommendation using Differential Relaxation".

CARS@RecSys, 2012

3) Y. Zheng, R. Burke, B. Mobasher. "Recommendation with Differential Context Weighting". In UMAP, 2013


[sac2014]splitting approaches for context-aware recommendation: an empirical study

Technology

contextaware systems

contexts context

contextaware recommendation

user splitting

ui splitting

rs algorithms center

il usa acm sigapp

example center