1. INCREMENTALCOLLABORATIVE FILTERINGVIA EVOLUTIONARY CO-CLUSTERINGAUTHORS / MOHAMMAD KHOSHNESHIN AND W. NICK STREETSOURCE / RECSYS10AFFILIATION / UNIVERSITY OF IOWAPRESENTER / ALLEN WU1

2. OUTLINE Introduction Incremental CF Incremental evolutionary co-clustering Experimental Results Conclusion 2 3. INTRODUCTION (1/3) Recommender system suggest items of interest to users. Collaborative filtering (CF) users rating information to recommenditems based on similarity. The drawback: more appropriate for static settings. In real world data, the new users and items should be incorporatedinto model recommendations in an online manner. Theincremental CF can handle the need. 3 4. INTRODUCTION (2/3) A few published approaches of the incremental CF: Sarwar et al. proposed an online CF strategy using singular valuedecomposition, SVD. Das et al. proposed a scalable online CF using MinHashclustering, PLSI and co-visitation counts. In K-NN, similarity parameters such as correlation can beupdated incrementally during online phase. George and Merugu used Bregman co-clustering as a scalableincremental CF approach for dynamic settings. (ICDM05)4 5. INTRODUCTION (3/3) This paper propose an incremental CF method that is bothscalable and accurate. The main contribution of this paper: An evolutionary Bregman co-clustering algorithm An ensemble strategy to give better predictions. 5 6. INCREMENTAL CF(1/3) In a CF problem, there are U users and V items. Users have provided a number of explicit ratings for items. rui is the rating of user u for item i. There are two phases in a CF algorithm: Offline phase: training based on known ratings Online phase: unknown ratings are estimated using the output of offlinephase. In incremental CF, the data available during online phase isincorporated into future predictions.6 7. BASELINEALGORITHM The simplest way to predict a rating is the global average r- of allratings. However, some users tend to rate higher and some items are morepopular. Including user bias and item bias in rating, the prediction isgiven by: r-u: the average ratings by user u. r-i: the average of ratings for item i. nu: the number of ratings for user u. ni: the number of ratings for item i. Snu,w and Sni, w are the support function for user u and item i.7 8. INCREMENTAL CF VIA CO-CLUSTERING (ICDM05) (1/2) Clustering refers to partitioning similar objects into groups, while co-clustering partitions two different kinds of objects simultaneously. As suggested in Georges paper, the prediction is as follows: where k=(u) is the user cluster assigned to user u. l=(i) is the item cluster assigned to item i. r-kl is the average of ratings belonging to users in user cluster kand items in item cluster l. (r-u-r-k) is the bias of user u. (r-i-r-l) is the bias of item i. 8 9. INCREMENTAL CF VIA CO-CLUSTERING (ICDM05) (2/2) George used the Bregman co-clustering algorithm, which has twophases, updating user clusters and updating item clusters, to producethe co-cluster results. In the online phase, the prediction is as follows: Incremental training is achieved by using new ratings to update theaverage parameters (r-kl, r-u, r-k, r-i, r-l). However, new users or items are not assigned to clusters during theonline phase.9 10. INCREMENTAL EVOLUTIONARYCO-CLUSTERING (1/4) If the support Sv,w (number of available ratings) for a user or items islow, the co-clustering approach will not provide good predictions forthem. As a strategy, users and items with low support are removed from thetraining phase so that training is both more effective and efficient. The drawback of Eq. (3), It incorporates (r-kl, r-k, r-l) from co-clustering solution that is notnecessarily reliable. (r-k and r-l is close to r-) Using only the block average r-kl for prediction ignores user anditem bias which results in poor accuracy as well. 10 11. INCREMENTAL EVOLUTIONARYCO-CLUSTERING (2/4) The revised rating prediction with co-clustering residuals is model as Eq.(5) is come from the support function of Eq. (1) set to 1. The ui is the correction parameter for (1). For known rating, Eq. (5) can be rewritten as ui can be interpreted as the residual of the prediction via (1). For implementing co-clustering, it is enough to work with the following objectivefunction. Where wui is 1 if rating rui exists in training data and otherwise is 0. (u)(i): the block average of residuals for user cluster (u) and item cluster (i).11 12. INCREMENTAL EVOLUTIONARYCO-CLUSTERING (3/4) The prediction strategy of old user - old item, and otherwise The ensembles are used to improve the accuracy of a method using a groupof predictors, while increasing the running time linearly with the number ofensemble elements. Let p denote a co-clustering solution and P be the number of co-clusteringsolutions we use in the model. We can predict with zulp is the average error of prediction for user u and item cluster l in p. zikp is the average error of prediction for item i and user cluster k in p. 12 13. INCREMENTAL EVOLUTIONARYCO-CLUSTERING (4/4) In this paper, it is trivial to find an appropriate cluster for a new user or newitem. Let u be a new user who has provided some ratings. If a sufficient number of rated items exits in the current co-clusteringsolution (sub-matrix), then the new users cluster can be found using nuh is the number of times user u has rated the items belonging to item cluster h during the online phase. -uh is the average of residuals for those ratings. g: user cluster A similar procedure finds the cluster of a new item.13 14. INCREMENTAL TRAININGALGORITHM numberIn() is the number ofratings a user u (item i) has inthe co-clustering solutionwhich is defined by hnuh(gnig). Trust the information for incorporating new user or new item. The new users and items willnot receive any prediction,those are predicted by Eq. (1). 14 15. EVOLUTIONARY ALGORITHM A group of co-clustering solutions is randomly generated and locally A population-based searchoptimized via Bregman co-clustering.approach Goal: find better solutions bycombining the currentsolutions. Every evolutionary algorithmhas three main step Selection Crossover Replacement Worst solution 15 16. CROSSOVERALGORITHM Let X be a NK assignmentmatrix An element x=(u, k) is 1, if objectu is assigned to cluster k and 0otherwise. qr is the intersection betweencluster q and cluster r. (k) is the largest intersection.16 17. ILLUSTRATION EXAMPLE p=1p=2 (1, 1) l1l2 (2, 2) l1 l1 54 0 0 k2 54 k1 0 0 12 3 4 k1 1 2k2 34 31 0 1 Bregman k2 3 1k2 01 00 1 1 co-clustering(3, 3) l1l1 (3, 3) l1 l2 54 3 2 k1 00 k1 1 1 21 2 1 k2 5 4k2 32 p=3p=4 k2 2 1k2 211X1 kk22X2 k1k2k1 k2 k31u1 01 u1 10 crossover u1 0 0 1u2 1 0u2 01 u2 0 0 1u3 0 1u3 01 u3 0 1 0(k) k1 k2 17u31 0 18. EXPERIMENTALRESULTS (1/3) The experiment dataset: Movielens dataset consisting of 100,000 ratings(1-5) by 943 users on 1682 movies. Evaluation metrics: Mean Absolute Error (MAE) Comparison methods: Baseline COCL: George, ICDM05 ECOCL: Evolutionary co-clustering without ensembles ECOCLE: Evolutionary co-clustering with ensembles. IKNN: Incremental KNN method. SVD18 19. EXPERIMENTALRESULTS (2/3) The experiment use the 5-foldcv. to get average MAE. The incremental training basedon three different strategies. 20%-80%: 20% of data wasused for offline training, 80%for incremental training. 19 20. EXPERIMENTALRESULTS (3/3) The offline phase of ECOCLEneeds more time due to theevolutionary algorithm. Online time is the sum of bothincremental training andprediction. ECOCL and IKNN havesimilar online speeds, whilethe accuracy for ECOCLE ismuch higher. 20 21. CONCLUSION Online CF methods that can incorporate new data in real time areadvantageous in many practical situations. However, this problem has not been adequately addressed. This paper extended the idea of CF via co-clustering to satisfy this need. The empirical results showed the proposed ECOCLE avchived very goodaccuracy compared to other incremental methods. Training time was comparatively slow, but still manageable. 21 22. THANK YOUFOR LISTENING!Q & A 22