Deep Recommender System:Fundamentals and Advances

Xiangyu Zhao1, Wenqi Fan2, Dawei Yin3, Jiliang Tang1

1Michigan State University, 2The Hong Kong Polytechnic University, 3Baidu Inc. Tutorial website: https://deeprs-tutorial.github.io

1Data Science and Engineering Lab

2

Recommender Systems

Recommender Systems

Informationoverload

Age of Information Explosion

Recommend item X to user

Items can be: Products, News, Movies, Videos, Friends, etc.

3

Recommender Systems

A B C

Recommendation has been widely applied in online services:- E-commerce, Content Sharing, Social Networking ...

Product Recommendation

4

Recommender SystemsRecommendation has been widely applied in online services:- E-commerce, Content Sharing, Social Networking ...

News/Video/Image Recommendation

5

Recommender SystemsRecommendation has been widely applied in online services:- E-commerce, Content Sharing, Social Networking ...

Friend Recommendation

Problem Formulation

Historical user-item interactions or additional side information (e.g., social relations, item’s knowledge, etc.)

INPUTPredict how likely a user would interact with a target Item (e.g., click, view, or purchase)

OUTPUT

6

Item set

User set social relations, age, gender, occupation, etc.

year, genre, actor,reviews, etc.

Side information

Side information

! users

! items (movies) …

…

User-item Interaction History

55

5 5

3

5

14

Spider Man

Iron ManToy Story

MinionsCaptainAmerica

Lily LalaPeter David

7

Recommender Systems

• Collaborative Filtering (CF) is the most well-known technique for recommendation.- Similar users (with respect to their historical interactions) have similar preferences.- Modelling users’ preference on items based on their past interactions (e.g., ratings and clicks).

• Learning representations of users and items is the key of CF.

items …

users …! users

! items (movies) …

…

User-item Interaction History

55

5 5

3

5

14

Spider Man

Iron ManToy Story

MinionsCaptainAmerica

Lily LalaPeter David

Task: predicting missing movie ratings in Netflix.

5 4…

5…

5…

5 12 5

! users

" items (movies)

Lily

…

?…

? ?? ? ? ?

…? ?

…? ? ?

Spider Man

Iron ManToy Story

MinionsCaptainAmerica

…

User-item Rating Matrix𝐑

8

Matrix Factorization

≈𝒒!𝒑"#

𝑑

𝑑 Predicted rating of item 𝒋 for user 𝒊 :

�̂�"! ≈ 𝒑"#𝒒! = *$%&

'

𝑝"$ 𝑞!$

User representations 𝐏 ∈ ℝ!×#

Items representationsQT ∈ ℝ#×$

𝐑 ≈ 𝐏 × 𝐐T

× ≪ 𝒎𝒊𝒏(𝒏,𝒎)

User-item Rating Matrix𝐑

5 4…

5

…5

…5 1

2 5

𝑛us

ers

𝑚 items (movies)

Lily

Lala

Peter

David

…

Spider Man Iron Man

Toy Story Minions

CaptainAmerica

…

Lily

Lala

Peter

David

…

…

Spider Man Iron Man

Toy Story Minions

CaptainAmerica

?…

? ?

? ? ? ?

…? ?

…? ? ?

… …

4𝒓𝒊𝒋

Ø Learn representations to describe users and items based on user-item rating matrix 𝐑.

…

…

…

Objective with rating reconstruction error:

𝑚𝑖𝑛 𝐏, 𝐐 *",!∈.

(𝑟"! − �̂�"!)/ = *",!∈.

(𝑟"! − 𝒑"#𝒒!)/

Given 𝑛×𝑚 matrix 𝐑, the goal is to learn: Users/Items representations: 𝐏 ∈ ℝ0×', 𝐐 ∈ ℝ2×'

Observed user-item interactions (known): 𝑺

observed rating scorepredicted rating score

≈𝒒!𝒑"#

𝒅

𝒅

User representations 𝐏 ∈ ℝ!×#

Item representationsQT ∈ ℝ#×$

×

≪ 𝒎𝒊𝒏(𝒏,𝒎)

User-item Rating Matrix𝐑

5 4…

5

…5

…5 1

2 5

𝑛us

ers

𝒎 items (movies)

…

…

Task: rating prediction in Netflix

9

Matrix Factorization

10

Deep Learning is Changing Our Lives

11

Reinforcement Learning (RL)

Graph Neural Networks (GNNs)

Automated Machine Learning(AutoML)

Deep Recommender Systems

Fundamentals of Deep Recommender Systems

00 1

Field1 Fieldm FieldM

01 0 10 0

Manually Deisgned Architectures§ Expert knowledge§ Time and engineering efforts

Graph-structured Data§ Information Isolated Island

Issue: ignore implicit/explicit relationships among instances

ItemsUsers!! "!!! ""!! "#!" "!

… …

Recommendation Policies§ Offline optimization§ Short-term reward

12

Agenda

• 9:00 – 9:10 Introduction to Recommender Systems (Jiliang Tang)• 9:10 – 9: 35 Fundamentals of Deep Recommender Systems (Wenqi Fan)• 9: 35 – 10:15 Reinforcement Learning for Recommendations (Xiangyu Zhao)• 10:15 – 10:25 Coffee Break (10 mins)• 10:25 – 11:00 Graph Neural Network for Recommendations (Wenqi Fan)• 11:05 – 11:35 AutoML for recommendations (Xiangyu Zhao)• 11:35 – 11:45 Conclusion and QA session