Upload File

applied data science: building a beer recommender | data science md - oct 2014

  • Home
  • Data & Analytics
  • Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014
96
Applied Data Science with Yhat Greg Lamp Data Science MD Meetup October 2014

Upload: yhat

Post on 14-Jun-2015

727 views

Category:

Data & Analytics


0 download

Report

Tags:

DESCRIPTION

Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014 ----------- Slides from a talk by Greg Lamp, CTO of Yhat, about building recommendation systems using Python and deploying them to production.

TRANSCRIPT

Page 1: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

Applied Data Science with Yhat

Greg Lamp

Data Science MD MeetupOctober 2014

Page 2: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

1) Intro2) The Problem3) Solutions4) Case Study: Beer Recommender5) Demo6) Q/A

Page 3: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

Here I am on the Internet.

Founder/CTO @ Yhat

Hi, I’m Greg!

Page 4: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014
Page 5: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

Founders

Company

Investors

Greg Lamp, CTO

Austin Ogilvie, CEO

● Launched in 2013

● HQ in Brooklyn

Page 6: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014
Page 7: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014
Page 8: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

Data sciencein the real world.

regression

Page 9: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

Get Raw Data

Strategic Insights

Real World Scoring

Data Driven Products

Business Impact

Clean Data

Stages of the Analytics Project Life Cycle

Expert data teams

Management

Customers & Front Line Employees

Page 10: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014
Page 11: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

What makes building analytical apps hard?

Page 12: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

Hi, I’m Trey.

Meet Trey, the Data Scientist

Page 13: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

We need to reduce churn.

Okay. I'll look into it.

Page 14: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

I figured out that....some complex stuff about vector space that'll improve...

....and that's how we'll reduce churn.

Sounds good. Let's do that...

Page 15: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

Any of you know what Gradient Boosting is?

So when can we go live with the new model?

Page 16: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

Now what?

Page 17: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

1)Translate Code

Page 18: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014
Page 19: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

2 Rebel Policeme

n 2

Page 20: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

2)PMML

Page 21: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014
Page 22: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

?

Page 23: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

3)Batch Jobs

Page 24: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014
Page 25: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014
Page 26: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

use your tools

Page 27: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

use your tools move quickly

Page 28: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

use your tools move quickly

any workflow

Page 29: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

use your tools move quickly

any workflow no translating

Page 30: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

Case Study

Page 31: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

+ =?

Page 32: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

A Beer Recommender in Python

Page 33: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

https://beers.yhathq.com

https://beers.yhathq.com/
https://beers.yhathq.com/
Page 34: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

The Data

Page 35: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

http://snap.stanford.edu/data/web-BeerAdvocate.html

Page 36: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014
Page 37: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014
Page 38: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014
Page 39: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014
Page 40: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014
Page 41: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014
Page 42: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014
Page 43: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014
Page 44: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014
Page 45: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014
Page 46: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

Beers

Page 47: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

Users

Page 48: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

Ratings

Page 49: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

Distance

Page 50: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

vs

Page 51: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014
Page 52: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014
Page 53: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014
Page 54: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014
Page 55: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014
Page 56: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

vs

Page 57: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014
Page 58: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

calculating distance

Page 59: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

eeny

? ?

Page 60: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

eeny meeny

?

Page 61: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

?Cosine

eeny meeny miny

Page 62: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

?Cosine

moe

Page 63: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

pick one.you can always

change

Page 64: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014
Page 65: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

Thank you,

Page 66: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014
Page 67: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014
Page 68: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014
Page 69: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014
Page 70: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014
Page 71: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014
Page 72: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014
Page 73: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

Scoring

Page 74: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014
Page 75: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

Aggregate

Page 76: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

Sort

Page 77: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

Filter

Page 78: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

Return

Page 79: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014
Page 80: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

Deployment

Page 81: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

What does this mean?

Page 82: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014
Page 83: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

Import Yhat

Page 84: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

Create a YhatModel

Page 85: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

Define execute

Page 86: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

Grab incoming data

Page 87: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

Call your function

Page 88: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

Format and return results

Page 89: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014
Page 90: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014
Page 91: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014
Page 92: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

Demohttp://cloud.yhathq.com/http://beers.yhathq.com/

http://cloud.yhathq.com/
http://beers.yhathq.com/
Page 93: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014
Page 94: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

deploy your [email protected]:yhat/beer-rec.git

Python/R Source on githubIPython Notebook

https://github.com/yhat/beer-rec
https://github.com/yhat/yhat-examples/tree/master/beer-recommender
http://nbviewer.ipython.org/gist/glamp/9260985
Page 95: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

Thanks!@yhathq

[email protected]

Page 96: Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014

Questions?


Data Modeling in IPTV and OTT Recommender Systems

Recommender Systemsdia.apice.unibo.it/download/slides/recommendation...Recommender Systems (1) Recommender Systems Prima parte Programmazione di Applicazioni Data Intensive Laurea

Streaming Recommender Systemspapers.…Streaming, recommender system, online learning, continu-ous time, data stream. 1. INTRODUCTION Recommender systems help to overcome information

Health Recommender System using Big data analyticsibii-us.org/Journals/JMSBI/V2N2/Publish/V2N2_3.pdf · 4. Healthcare Recommender System Framework Healthcare recommender system is

Network Science @ Recommender Systems ... Science @ Recommender Systems László Grad-Gyenge Electronic Commerce Group Vienna University of Technology, Vienna, Austria [email protected]

Recommender Systems and Linked Open Data

Data Recommender System: Improving the Discovery of ...€¦ · Search functions in PANGAEA Dataset and its related research objects Devaraju, A. et al., Data Recommender System:

Building Recommender Systems - Mendeley and Science Direct

Data Science: What It Is and How It Can Help Your …...Data Science approach: Build a recommender system using historical data, recommend future purchases. Identify which products

Kontextsensitive Recommender Systeme für … Science Research Meets Business: App and Mobile Development - October 2014, Bolzano Kontextsensitive Recommender Systeme für Mobile Endgeräte

GeoSRS: A Hybrid Social Recommender System for Geolocated Data

Item2Vec-based Approach to a Recommender System...UNIVERSITY OF TARTU Institute of Computer Science Computer Science Curriculum Vitali Kuzmin Item2Vec-based Approach to a Recommender

Recommender Systems in the Linked Data era

Recommender Systems Recommender Systems

Interactive Visualization of Recommender Systems Data (2017... · 2017-09-24 · Interactive Visualization of Recommender Systems Data SHCIS’17, June 21-22, 2017, Neuchatel, Switzerlandˆ

Recommender Systems: The Art and Science of Matching Items to Users - A LinkedIn open data talk by Deepak Agarwal from Yahoo Research!

SSSW 2013 - Feeding Recommender Systems with Linked Open Data

SURVEY ON RECOMMENDER SYSTEMS FOR SOLVING COLD … · Keywords Cold Start, Recommender Systems, User Behavior, Big Data, Informat ion Filtering. I. INTRODUCTION Recommender systems

Semantics-aware Graph-based Recommender Systems exploiting Linked Open Data

Taxonomy of Recommender Systems for Educational Data Mining … · 2019-10-31 · Taxonomy of Recommender Systems for Educational Data Mining (EDM) Techniques: A Systematic Review

DR-Advisor: A Data-Driven Demand Response Recommender System

CS 277: Data Mining Recommender Systems Padhraic Smyth Department of Computer Science University of California, Irvine

M.TECH. DATA SCIENCE DEPARTMENT OF COMPUTER SCIENCE … · 19DS611 Big Data Mining 3 0 1 4 ... Data Compression 3 0 0 3 19DS714 Modeling and Simulation 3 0 0 3 19DS715 Recommender

Shaping Feedback Data in Recommender Systems …...Shaping Feedback Data in Recommender Systems with Interventions Based on Information Foraging Theory Tobias Schnabel Cornell University

Recommender Systemsaris.me/contents/teaching/data-mining-2016/slides/recommender-sy… · Ricci, Rokach and Shapira: Introduction to Recommender Systems Handbook [link] Bobadilla

The Barclays Data Science Hackathon: Building Retail Recommender Systems based on Customer Shopping Behaviour - Gianmario Spacagna, Pirelli

Data-Driven Destination Recommender Systems · 2020-02-17 · Data-Driven Destination Recommender Systems Linus W. Dietz Technical University of Munich Department of Informatics Chair

Master of Science (MS) in Data Science€¦ · recommender systems, and social networks. He has published over 35 articles in referred journals and conference proceedings, and participated

Recommender Systems - Aris Anagnostopoulos - Homearis.me/.../slides/recommender-systems.pdf · Recommender Systems Class Algorithmic Methods of Data Mining Program M. Sc. Data Science

COMP 465: Data Mining Recommender Systems

dayta.me - A Personal News + Data Recommender for Your Daypeople.csail.mit.edu/emax/papers/dayta.pdf · 2010-10-01 · Dayta.me - A Personal News + Data Recommender for Your Day 3

15-388/688 -Practical Data Science: Recommender systems · 2019-12-06 · Collaborative filtering Collaborative filtering refers to recommender systems that make recommendations based

Recommender Systems...Recommender Systems Jee-Hyong Lee Department of Computer Science & Engineering Sungkyunkwan University. Outline 1. Introduction 2. Collaborative Filtering Overview

Machine Learning and Data Mining Methods for Recommender

The Science and the Magic of User Feedback for Recommender Systems