CoFiRank : Maximum Margin Matrix Factorization for Collabora

tive Ranking

Markus Weimer, Alexandros Karatzoglou, Quoc Viet Le and Alex Smola

NIPS’07

Idea

• Maximum Margin Matrix Factorization

• Structured Estimation for Ranking

• Bundle Method Solver

Collaborative Filtering

• Based on partial observed matrix to predict unobserved entries

Matrix Factorization

• Low Rank Approximation

• SVD for fully observed Y

• Non-convex

Maximum Margin Matrix Factorization

• Trace norm+Hinge loss: Convex

• Semi-Definite Programming

Regularized Matrix Factorization

• Formulation

• Probabilistic Matrix Factorization (PMF)

• CoFiRank

• Linear Convex Upper Bound

TTtrain

U,VVVUUYUVL trtr

2),( minimize

2

2),( traintrain YUVYUVL

),NDCG(1),( traintrain YY Non-Convex

ffcYYfL traintrain ,),(max:),(

Solved by linear programming

Alternating optimizing

How to Compute Loss?

• Linear Convex Upper Bound

• Solved by Linear Programming

ffcYYfL traintrain ,),(max:),(

i j jijiX

XC ,, min

Can this explain in simple way?

Useful Links

• CoFiRank

http://www.cofirank.org

• MMMF http://ttic.uchicago.edu/~nati/mmmf/

• MFhttp://helikoid.si/mf/index.html

Famous Researchers in Optimization

• Yurii Nesterov – “Introductory Lectures on Convex Optimization: A Basic Course”http://www.core.ucl.ac.be/~nesterov/

• Arkadi Nemirovski – “Efficient methods in convex programming” http://www2.isye.gatech.edu/~nemirovs/

• Stephen P. Boyd – “Convex Optimization”http://www.stanford.edu/~boyd/

• Stephen J. Wright – “Numerical Optimization” http://pages.cs.wisc.edu/~swright/

• Dimitri Bertsekas – “Nonlinear Programming” http://web.mit.edu/dimitrib/www/home.html

Questions?

Normalized Discounted Cumulative Gain (NDCG)

How to set c?

• ci is set decreasing, is maximized with respect toπ for argsort(f)

• ci =(i+1)-0.25

fcf ,:),(

Convex Upper Bound

• Linear Convex Upper Bound

ffcYYfL traintrain ,),(max:),(

),(,),(),( **traintraintrain YffcYYfL

baba ,)(sort),(sort

Bundle Method

• General convex optimization solver with tight convergence bound O(1/)

Bundle Method for CoFiRank