copyright © 2015 criteo large-scale real-time product recommendation at criteo simon dollé recsys...
TRANSCRIPT
![Page 1: Copyright © 2015 Criteo Large-Scale Real-Time Product Recommendation at Criteo Simon Dollé RecSys FR, December 1 st, 2015](https://reader035.vdocuments.net/reader035/viewer/2022062519/5697bfa11a28abf838c95e62/html5/thumbnails/1.jpg)
Copyright © 2015 Criteo
Large-Scale Real-Time Product Recommendation at Criteo
Simon Dollé
RecSys FR, December 1st, 2015
![Page 2: Copyright © 2015 Criteo Large-Scale Real-Time Product Recommendation at Criteo Simon Dollé RecSys FR, December 1 st, 2015](https://reader035.vdocuments.net/reader035/viewer/2022062519/5697bfa11a28abf838c95e62/html5/thumbnails/2.jpg)
Copyright © 2015 Criteo
![Page 3: Copyright © 2015 Criteo Large-Scale Real-Time Product Recommendation at Criteo Simon Dollé RecSys FR, December 1 st, 2015](https://reader035.vdocuments.net/reader035/viewer/2022062519/5697bfa11a28abf838c95e62/html5/thumbnails/3.jpg)
Copyright © 2015 Criteo
We buy
Ad spaces
![Page 4: Copyright © 2015 Criteo Large-Scale Real-Time Product Recommendation at Criteo Simon Dollé RecSys FR, December 1 st, 2015](https://reader035.vdocuments.net/reader035/viewer/2022062519/5697bfa11a28abf838c95e62/html5/thumbnails/4.jpg)
Copyright © 2015 Criteo
We buy
Ad spaces
We sell
Clicks
![Page 5: Copyright © 2015 Criteo Large-Scale Real-Time Product Recommendation at Criteo Simon Dollé RecSys FR, December 1 st, 2015](https://reader035.vdocuments.net/reader035/viewer/2022062519/5697bfa11a28abf838c95e62/html5/thumbnails/5.jpg)
Copyright © 2015 Criteo
We buy
Ad spaces
We sell
Clicksthat convert
![Page 6: Copyright © 2015 Criteo Large-Scale Real-Time Product Recommendation at Criteo Simon Dollé RecSys FR, December 1 st, 2015](https://reader035.vdocuments.net/reader035/viewer/2022062519/5697bfa11a28abf838c95e62/html5/thumbnails/6.jpg)
Copyright © 2015 Criteo
We buy
Ad spaces
We sell
Clicksthat converta lot
![Page 7: Copyright © 2015 Criteo Large-Scale Real-Time Product Recommendation at Criteo Simon Dollé RecSys FR, December 1 st, 2015](https://reader035.vdocuments.net/reader035/viewer/2022062519/5697bfa11a28abf838c95e62/html5/thumbnails/7.jpg)
Copyright © 2015 Criteo
We buy
Ad spaces
We sell
Clicksthat converta lot
We take the risk
![Page 8: Copyright © 2015 Criteo Large-Scale Real-Time Product Recommendation at Criteo Simon Dollé RecSys FR, December 1 st, 2015](https://reader035.vdocuments.net/reader035/viewer/2022062519/5697bfa11a28abf838c95e62/html5/thumbnails/8.jpg)
10 000 displays
![Page 9: Copyright © 2015 Criteo Large-Scale Real-Time Product Recommendation at Criteo Simon Dollé RecSys FR, December 1 st, 2015](https://reader035.vdocuments.net/reader035/viewer/2022062519/5697bfa11a28abf838c95e62/html5/thumbnails/9.jpg)
10 000 displays
leads to
50 clicks
![Page 10: Copyright © 2015 Criteo Large-Scale Real-Time Product Recommendation at Criteo Simon Dollé RecSys FR, December 1 st, 2015](https://reader035.vdocuments.net/reader035/viewer/2022062519/5697bfa11a28abf838c95e62/html5/thumbnails/10.jpg)
10 000 displays
leads to
50 clicks
leads to
1 sale
![Page 11: Copyright © 2015 Criteo Large-Scale Real-Time Product Recommendation at Criteo Simon Dollé RecSys FR, December 1 st, 2015](https://reader035.vdocuments.net/reader035/viewer/2022062519/5697bfa11a28abf838c95e62/html5/thumbnails/11.jpg)
3 billion ads/day3 billion products
![Page 12: Copyright © 2015 Criteo Large-Scale Real-Time Product Recommendation at Criteo Simon Dollé RecSys FR, December 1 st, 2015](https://reader035.vdocuments.net/reader035/viewer/2022062519/5697bfa11a28abf838c95e62/html5/thumbnails/12.jpg)
10ms to pick relevant products
![Page 13: Copyright © 2015 Criteo Large-Scale Real-Time Product Recommendation at Criteo Simon Dollé RecSys FR, December 1 st, 2015](https://reader035.vdocuments.net/reader035/viewer/2022062519/5697bfa11a28abf838c95e62/html5/thumbnails/13.jpg)
7 data centers15 000 servers
1200-node hadoop cluster
![Page 14: Copyright © 2015 Criteo Large-Scale Real-Time Product Recommendation at Criteo Simon Dollé RecSys FR, December 1 st, 2015](https://reader035.vdocuments.net/reader035/viewer/2022062519/5697bfa11a28abf838c95e62/html5/thumbnails/14.jpg)
Catalog data3B+ products
![Page 15: Copyright © 2015 Criteo Large-Scale Real-Time Product Recommendation at Criteo Simon Dollé RecSys FR, December 1 st, 2015](https://reader035.vdocuments.net/reader035/viewer/2022062519/5697bfa11a28abf838c95e62/html5/thumbnails/15.jpg)
Browsing history2B events / day
Catalog data3B+ products
![Page 16: Copyright © 2015 Criteo Large-Scale Real-Time Product Recommendation at Criteo Simon Dollé RecSys FR, December 1 st, 2015](https://reader035.vdocuments.net/reader035/viewer/2022062519/5697bfa11a28abf838c95e62/html5/thumbnails/16.jpg)
Ad display data20B events / day
Browsing history2B events / day
Catalog data3B+ products
![Page 17: Copyright © 2015 Criteo Large-Scale Real-Time Product Recommendation at Criteo Simon Dollé RecSys FR, December 1 st, 2015](https://reader035.vdocuments.net/reader035/viewer/2022062519/5697bfa11a28abf838c95e62/html5/thumbnails/17.jpg)
Copyright © 2015 Criteo
How do we do it ?
![Page 18: Copyright © 2015 Criteo Large-Scale Real-Time Product Recommendation at Criteo Simon Dollé RecSys FR, December 1 st, 2015](https://reader035.vdocuments.net/reader035/viewer/2022062519/5697bfa11a28abf838c95e62/html5/thumbnails/18.jpg)
Copyright © 2015 Criteo
Recommend products for a user
• What we want: reco(user) = products
• 1B users x 3B products !• But we need to scale and keep it fresh
• What we can do :
Pre-select products offline
Refine scoring online to get final candidates
![Page 19: Copyright © 2015 Criteo Large-Scale Real-Time Product Recommendation at Criteo Simon Dollé RecSys FR, December 1 st, 2015](https://reader035.vdocuments.net/reader035/viewer/2022062519/5697bfa11a28abf838c95e62/html5/thumbnails/19.jpg)
Bob saw orange shoes
![Page 20: Copyright © 2015 Criteo Large-Scale Real-Time Product Recommendation at Criteo Simon Dollé RecSys FR, December 1 st, 2015](https://reader035.vdocuments.net/reader035/viewer/2022062519/5697bfa11a28abf838c95e62/html5/thumbnails/20.jpg)
Bob saw orange shoes
Some candidate products
Historical
![Page 21: Copyright © 2015 Criteo Large-Scale Real-Time Product Recommendation at Criteo Simon Dollé RecSys FR, December 1 st, 2015](https://reader035.vdocuments.net/reader035/viewer/2022062519/5697bfa11a28abf838c95e62/html5/thumbnails/21.jpg)
Bob saw orange shoes
Some candidate products
Historical
Most viewed
![Page 22: Copyright © 2015 Criteo Large-Scale Real-Time Product Recommendation at Criteo Simon Dollé RecSys FR, December 1 st, 2015](https://reader035.vdocuments.net/reader035/viewer/2022062519/5697bfa11a28abf838c95e62/html5/thumbnails/22.jpg)
Bob saw orange shoes
Some candidate products
Historical
Most viewed
![Page 23: Copyright © 2015 Criteo Large-Scale Real-Time Product Recommendation at Criteo Simon Dollé RecSys FR, December 1 st, 2015](https://reader035.vdocuments.net/reader035/viewer/2022062519/5697bfa11a28abf838c95e62/html5/thumbnails/23.jpg)
Bob saw orange shoes
Some candidate products
Historical
Similar
Most viewed
![Page 24: Copyright © 2015 Criteo Large-Scale Real-Time Product Recommendation at Criteo Simon Dollé RecSys FR, December 1 st, 2015](https://reader035.vdocuments.net/reader035/viewer/2022062519/5697bfa11a28abf838c95e62/html5/thumbnails/24.jpg)
Bob saw orange shoes
Some candidate products
Historical
Similar
Most viewed
![Page 25: Copyright © 2015 Criteo Large-Scale Real-Time Product Recommendation at Criteo Simon Dollé RecSys FR, December 1 st, 2015](https://reader035.vdocuments.net/reader035/viewer/2022062519/5697bfa11a28abf838c95e62/html5/thumbnails/25.jpg)
Bob saw orange shoes
Some candidate products
Historical
Similar
Complementary
Most viewed
![Page 26: Copyright © 2015 Criteo Large-Scale Real-Time Product Recommendation at Criteo Simon Dollé RecSys FR, December 1 st, 2015](https://reader035.vdocuments.net/reader035/viewer/2022062519/5697bfa11a28abf838c95e62/html5/thumbnails/26.jpg)
Recommendation Service20K qps
![Page 27: Copyright © 2015 Criteo Large-Scale Real-Time Product Recommendation at Criteo Simon Dollé RecSys FR, December 1 st, 2015](https://reader035.vdocuments.net/reader035/viewer/2022062519/5697bfa11a28abf838c95e62/html5/thumbnails/27.jpg)
HADOOPBrowsing
history
Recommendation Service
50B
20K qps
Preselection computation Map-Reduce jobs
![Page 28: Copyright © 2015 Criteo Large-Scale Real-Time Product Recommendation at Criteo Simon Dollé RecSys FR, December 1 st, 2015](https://reader035.vdocuments.net/reader035/viewer/2022062519/5697bfa11a28abf838c95e62/html5/thumbnails/28.jpg)
HADOOPBrowsing
history
Preselections
Recommendation Service
50B
12h
20K qps
Preselection computation Map-Reduce jobs
500M
![Page 29: Copyright © 2015 Criteo Large-Scale Real-Time Product Recommendation at Criteo Simon Dollé RecSys FR, December 1 st, 2015](https://reader035.vdocuments.net/reader035/viewer/2022062519/5697bfa11a28abf838c95e62/html5/thumbnails/29.jpg)
Copyright © 2015 Criteo
Online: sources
Similarities Most viewed Most bought
![Page 30: Copyright © 2015 Criteo Large-Scale Real-Time Product Recommendation at Criteo Simon Dollé RecSys FR, December 1 st, 2015](https://reader035.vdocuments.net/reader035/viewer/2022062519/5697bfa11a28abf838c95e62/html5/thumbnails/30.jpg)
Copyright © 2015 Criteo
Online: merge of products
Similarities Most viewed Most bought
![Page 31: Copyright © 2015 Criteo Large-Scale Real-Time Product Recommendation at Criteo Simon Dollé RecSys FR, December 1 st, 2015](https://reader035.vdocuments.net/reader035/viewer/2022062519/5697bfa11a28abf838c95e62/html5/thumbnails/31.jpg)
Copyright © 2015 Criteo
ML model
• Logistic regression models because : • They scale• They are fast• They can handle lots of features
Product-specific User-specific User-product interactions Display-specific
![Page 32: Copyright © 2015 Criteo Large-Scale Real-Time Product Recommendation at Criteo Simon Dollé RecSys FR, December 1 st, 2015](https://reader035.vdocuments.net/reader035/viewer/2022062519/5697bfa11a28abf838c95e62/html5/thumbnails/32.jpg)
HADOOPBrowsing
history
Recommendation Service
50B
12h
20K qps
Preselection computation Map-Reduce jobs
500M
Preselections
![Page 33: Copyright © 2015 Criteo Large-Scale Real-Time Product Recommendation at Criteo Simon Dollé RecSys FR, December 1 st, 2015](https://reader035.vdocuments.net/reader035/viewer/2022062519/5697bfa11a28abf838c95e62/html5/thumbnails/33.jpg)
HADOOPBrowsing
history
Prediction models
Recommendation Service
50B
12h
6h
20K qps
Preselection computation Map-Reduce jobs
500M
Preselections
![Page 34: Copyright © 2015 Criteo Large-Scale Real-Time Product Recommendation at Criteo Simon Dollé RecSys FR, December 1 st, 2015](https://reader035.vdocuments.net/reader035/viewer/2022062519/5697bfa11a28abf838c95e62/html5/thumbnails/34.jpg)
HADOOPBrowsing
history
Prediction models
Recommendation Service
50B
12h
6h
20K qps
Display, Click, Sale logs
Preselection computation Map-Reduce jobs
500M
Preselections
![Page 35: Copyright © 2015 Criteo Large-Scale Real-Time Product Recommendation at Criteo Simon Dollé RecSys FR, December 1 st, 2015](https://reader035.vdocuments.net/reader035/viewer/2022062519/5697bfa11a28abf838c95e62/html5/thumbnails/35.jpg)
HADOOPBrowsing
history
Prediction models
Recommendation Service
50B
12h
6h
20K qps
Display, Click, Sale logs
Preselection computation Map-Reduce jobs
500M
Preselections
![Page 36: Copyright © 2015 Criteo Large-Scale Real-Time Product Recommendation at Criteo Simon Dollé RecSys FR, December 1 st, 2015](https://reader035.vdocuments.net/reader035/viewer/2022062519/5697bfa11a28abf838c95e62/html5/thumbnails/36.jpg)
Copyright © 2015 Criteo
Online: scoring
Similarities Most viewed Most bought
0,02 0,12 0,06 0,18 0,03 0,05 0,01 0,005 0,011 0,013 0,004 0,007
![Page 37: Copyright © 2015 Criteo Large-Scale Real-Time Product Recommendation at Criteo Simon Dollé RecSys FR, December 1 st, 2015](https://reader035.vdocuments.net/reader035/viewer/2022062519/5697bfa11a28abf838c95e62/html5/thumbnails/37.jpg)
Copyright © 2015 Criteo
Online: scoring
Similarities Most viewed Most bought
0,18 0,12 0,06 0,05 0,03 0,02 0,013 0,011 0,01 0,007 0,005 0,004
![Page 38: Copyright © 2015 Criteo Large-Scale Real-Time Product Recommendation at Criteo Simon Dollé RecSys FR, December 1 st, 2015](https://reader035.vdocuments.net/reader035/viewer/2022062519/5697bfa11a28abf838c95e62/html5/thumbnails/38.jpg)
Copyright © 2015 Criteo
Online: candidates
0,18 0,12 0,06 0,05 0,03 0,02 0,013 0,011 0,01 0,007 0,005 0,004
SHOP SHOP SHOP SHOP
-50%
![Page 39: Copyright © 2015 Criteo Large-Scale Real-Time Product Recommendation at Criteo Simon Dollé RecSys FR, December 1 st, 2015](https://reader035.vdocuments.net/reader035/viewer/2022062519/5697bfa11a28abf838c95e62/html5/thumbnails/39.jpg)
Copyright © 2015 Criteo
What’s next ?
![Page 40: Copyright © 2015 Criteo Large-Scale Real-Time Product Recommendation at Criteo Simon Dollé RecSys FR, December 1 st, 2015](https://reader035.vdocuments.net/reader035/viewer/2022062519/5697bfa11a28abf838c95e62/html5/thumbnails/40.jpg)
Copyright © 2015 Criteo
What’s next for us: Upcoming challenges
• Long(er)-term user profiles
![Page 41: Copyright © 2015 Criteo Large-Scale Real-Time Product Recommendation at Criteo Simon Dollé RecSys FR, December 1 st, 2015](https://reader035.vdocuments.net/reader035/viewer/2022062519/5697bfa11a28abf838c95e62/html5/thumbnails/41.jpg)
Copyright © 2015 Criteo
What’s next for us: Upcoming challenges
• Long(er)-term user profiles
• More and better product information (images, semantic, NLP)
![Page 42: Copyright © 2015 Criteo Large-Scale Real-Time Product Recommendation at Criteo Simon Dollé RecSys FR, December 1 st, 2015](https://reader035.vdocuments.net/reader035/viewer/2022062519/5697bfa11a28abf838c95e62/html5/thumbnails/42.jpg)
Copyright © 2015 Criteo
What’s next for us: Upcoming challenges
• Long(er)-term user profiles
• More and better product information (images, semantic, NLP)
• Instant-update of similarities
![Page 43: Copyright © 2015 Criteo Large-Scale Real-Time Product Recommendation at Criteo Simon Dollé RecSys FR, December 1 st, 2015](https://reader035.vdocuments.net/reader035/viewer/2022062519/5697bfa11a28abf838c95e62/html5/thumbnails/43.jpg)
Copyright © 2015 Criteo
What’s next for us: Upcoming challenges
• Long(er)-term user profiles
• More and better product information (images, semantic, NLP)
• Instant-update of similarities
• Joint product scoring • (score full banner and not products independently)
![Page 44: Copyright © 2015 Criteo Large-Scale Real-Time Product Recommendation at Criteo Simon Dollé RecSys FR, December 1 st, 2015](https://reader035.vdocuments.net/reader035/viewer/2022062519/5697bfa11a28abf838c95e62/html5/thumbnails/44.jpg)
Copyright © 2015 Criteo
What’s next for you: Fancy a try?
On your own:
With us !
http://labs.criteo.com/jobs/
• We published datasets for click prediction• 4GB display-click data: Kaggle challenge in 2014 http://bit.ly/1vgw2XC• 1TB Display-Click data (industry’s largest dataset): http://bit.ly/1PyH4Vq
• 4 billion of observations• 156 billion feature-value• available on Microsoft Azure• used by edX (UC Berkeley)
![Page 45: Copyright © 2015 Criteo Large-Scale Real-Time Product Recommendation at Criteo Simon Dollé RecSys FR, December 1 st, 2015](https://reader035.vdocuments.net/reader035/viewer/2022062519/5697bfa11a28abf838c95e62/html5/thumbnails/45.jpg)
Copyright © 2015 Criteo
![Page 46: Copyright © 2015 Criteo Large-Scale Real-Time Product Recommendation at Criteo Simon Dollé RecSys FR, December 1 st, 2015](https://reader035.vdocuments.net/reader035/viewer/2022062519/5697bfa11a28abf838c95e62/html5/thumbnails/46.jpg)
Copyright © 2015 Criteo
Questions?
![Page 47: Copyright © 2015 Criteo Large-Scale Real-Time Product Recommendation at Criteo Simon Dollé RecSys FR, December 1 st, 2015](https://reader035.vdocuments.net/reader035/viewer/2022062519/5697bfa11a28abf838c95e62/html5/thumbnails/47.jpg)
Copyright © 2015 Criteo
Thank you [email protected]
@simondolle@recsysfr
Credits: Creative Stall, Gilbert Bages