how to build a recommender system

Recommender System

How to build a

Võ Duy TuấnTechnical Director @ dienmay.com

PHP 5 Zend Certified Engineer

Mobile App Developer

Web Developer & Designer

Interest: o PHP

o Large System & Data Mining

o Web Performance Optimization

o Mobile Development

Introduction

Collaborative Filtering

Question & Answer

AGENDA

1. Introduction

APPLICATIONS

• Personalized recommendation• Social recommendation• Item recommendation• Combination of 3 approaches above

AMAZON.COM | BOOKS

PLAY.GOOGLE.COM | APPS

SKILLSHARE.COM | CLASSES

PROCESS DIAGRAM

Preprocessing Data Analysis Adjustment

INPUT OUTPUT

TYPE OF RECOMMENDER SYSTEM

• Collaborative filtering• Content-based filtering• Hybrid

2. Collaborative Filtering

USER & ITEM

ORDER DATA

ORDER DATA (cont.)

ORDER DATA (cont.)

VECTOR & DIMENSION

VECTOR & DIMENSION

VECTORS

VECTORS

SIMILARITY CALCULATION

USER SIMILARITY MATRIX

SIMILARITY CALCULATION

SIMILARITY CALCULATION

SIMILARITY CALCULATION EXAMPLE

K-NEAREST-NEIGHBOR

K-NEAREST-NEIGHBOR

NEIGHBORS’ ORDER

REMOVE BOUGHT ITEMS

CALCULATING FINAL SCORE

OTHER SIMILARITY MEASURES

More at: http://favi.com.vn/wp-content/uploads/2012/05/pg049_Similarity_Measures_for_Text_Document_Clustering.pdf

Problem ?!

COLLABORATIVE FILTERING PROBLEM

• Fail with cold start problemo New Usero New Item

• Performanceo Large Data seto Pre-calculate

PERFORMANCE EXAMPLE

• We have 1,000,000 users (customers)• We sell 10,000 items

- Total of similarity calculating = 1,000,000 x 1,000,000 = 1,000,000,000,000- Each similarity calculate need 0.006s (on my MacBook Pro 2.2GHz Core i7, 8G Ram)

=> We need 1,000,000,000,000 x 0.006 = 6,000,000,000(s)≈ 70,000 days ≈ 191 years

- If store each similarity in 8 bytes, we need = 8,000,000,000,000 bytes≈ 8,000 GB (on Memory or File)

ITEM-TO-ITEM COLLABORATIVE FILTERING (AMAZON.COM )

Download Paper: http://www.cs.umd.edu/~samir/498/Amazon-Recommendations.pdf

ADJUSTMENTS

• Hybrid Recommender System• Sale forecast system• Context of User• Type of Item, Action• External (3rd-party) information.

BOOKS

Programming Collective IntelligenceToby Segaran

Recommender Systems HandbookMany Authors

Big Data For DummiesMarcia Kaufman, Fern Halper

OPEN SOURCES

Thank you!

CONTACT ME:

tuanmaster2002@yahoo.com

0938 916 902

http://bloghoctap.com/