a few challenges to make machine learning easy

June 3rd, 2013BigML Inc, 2013

Challenges to Make Machine Learning Easy

ACM San Francisco Bay Area Professional Chapter

Francisco J Martin, Ph.D.BigML Co-founder & CEO

eBay Whitman Campus

http://www.sfbayacm.org/

http://www.sfbayacm.org/

http://maps.google.com/maps?q=2065+Hamilton+Ave%2C+San+Jose%2C+CA

http://maps.google.com/maps?q=2065+Hamilton+Ave%2C+San+Jose%2C+CA

June 3rd, 2013BigML Inc, 2013 2

Expert: Published papers at KDD, ICML, NIPS, etc or developed own ML algorithms used at large scale.

Sampling the Audience

Aficionado: Understands pros/cons of different techniques and/or can tweak algorithms as needed.

Newbie: Just taking Coursera ML class or reading an introductory book to ML.

Absolute beginner: ML sounds like science fiction

Practitioner: Very familiar with ML packages (Weka, Scikit, R, etc).


Data, data everywhereA special report on managing information

Why make ML easy?

In the age of data, Machine Learning is the key component to:

‣ make data-driven decisions‣ develop smart applications‣ build predictive analytics

June 3rd, 2013BigML Inc, 2013

However, Machine Learning is COMPLEX:

‣tools are complicated and do not scale well‣solutions are costly‣e x p e r t s w i t h i n d u s t r y experience are scarce

4

Why make ML easy?

http://ttic.uchicago.edu/~samory/

http://people.tuebingen.mpg.de/samory/

http://people.tuebingen.mpg.de/samory/


Why make ML easy?

April, 2013BigML Inc, 2013 7

BigML A cloud-based service that makes

Machine Learning SIMPLE

$ bigmler --train customer2012.csv \ --test new_customers.csv \ --objective churn

>>> from bigml.api import BigML>>> api = BigML()>>> source = api.create_source("s3://bigml-public/csv/sales.csv")>>> dataset = api.create_dataset(source)>>> model = api.create_model(dataset)

$ curl https://bigml.io/model?$BIGML_AUTH \ -X POST \ -H "content-type: application/json" \ -d '{"dataset": "dataset/50ca447b3b56356ae0000029"}'

https://bigml.io/dataset?$BIGML_AUTH



AgendaBigML web-based interface (10-15 min)

Questions (10-15 min)

$ bigmler --train customer2012.csv \ --test new_customers.csv \ --objective churn

>>> from bigml.api import BigML>>> api = BigML()>>> source = api.create_source("s3://bigml-public/csv/iris.csv")>>> dataset = api.create_dataset(source)>>> model = api.create_model(dataset)

$ curl https://bigml.io/dataset?$BIGML_AUTH \ -X POST \ -H "content-type: application/json" \ -d '{"source": "source/50ca447b3b56356ae0000029"}'

BigML API, API Bindings, BigMLer (5 min)

Challenges (10-15 min)

#1 Machine Learning Breadth and Depth#2 User Diversity #3 Simplicity#4 Scalability #5 Measuring Impact#6 Pricing




How it works


BigML Resources

csv, arff, xlshttps, s3, azure, odata

Sources local and remote

Datasets

Stream histogramsStatistics

ModelsInteractive Compoundable Random Decision Forests

Actionable: exportable to rules, code, pmml

PredictionsForm-based PredictionsQuestion by QuestionLocal predictions

Evaluations

ClassificationRegressionComparison


BigML API


3,500+ users

35,000+ models

BigML


FREE subscription?

mail your username to: [email protected]

mailto:[email protected]

mailto:[email protected]


Challenges

#1 Machine Learning breadth and depth#2 User Diversity #3 Simplicity#4 Scalability #5 Measuring Machine Learning Impact#6 Pricing


...or you can deal with that!

#1 Supervised learning #2 Unsupervised learning#3 Semi-supervised learning #4 Reinforcement learning#5 Learning to Learn

#1 machine learning breadth and depth


...or you can deal with that!#1 machine learning breadth and depth


Phrase a problem as an ML task

The stages of an ML application

Data Wrangling

Feature Engineering

Learn from Data

Pre-evaluate

Measure Impact


Problems

Techniques

Applications

ClassificationRegressionClusteringDensity EstimationManifold learningActive learningetc. Just solving a couple of

problems and using a few techniques thousands of

applications can be developed

churn prevention, date matching, decision making, diagnostics, fraud detection, detecting tumors, detecting investment opportunities, human body pose estimation, pedestrian tracking, predictive analytics, recommendation systems, risk analysis, spam detection, etc



Understanding the past

Predicting the future

Why Trees first?


Why Trees?


A Machine Learning application requires more tasks (that are even more important) than just learning from data.

Just solving one problem more will enable a huge number of applications more.

What problem(s) to tackle next and which techniques to use?



Experts

Aficionados

Practitioner

Newbies

Absolute beginners

#2 user diversity

How to prioritize what to build next? More features for the

expert or simplifying more for the newbies?


Tim

e-to

-pro

duct

ivit

y

+

+

Expertise

#2 user diversity


#2 user diversity

MBs PBs

MBs PBs

Actual size

Size

Most users believe their data is much bigger than it really is


Num

ber

of J

obs

+

+

Size of Job

#2 user diversity


#3 simplicity


“Any fool can make something complicated. It takes a genius to

make it simple.”

― Woody Guthrie

#3 simplicity

http://www.goodreads.com/author/show/57836.Woody_Guthrie

http://www.goodreads.com/author/show/57836.Woody_Guthrie


‣install ‣configure‣use‣train ‣understand‣test‣pre-evaluate‣measure impact‣deploy‣scale ‣access programmatically (API)

#3 simplicity Simple means much more than a easy-to-use interface


#4 scalability

N CONCURRENT

JOBSfrom

1 CUSTOMER

1 JOBfrom

1 USER

N JOBSfrom

M CUSTOMERS


Infrastructure


#5 measuring machine learning impact


Measuring “actual” impact is complex and goes beyond traditional performance evaluation.

Imagine that an algorithm predicts that user Alice is going to buy a Magic Potion.

‣ But Magic Potions are out of stock.

‣ Should we blame ‣ the algorithm for the “false positive” prediction?‣ the data scientist for not including that feature?‣ operations for running out of stock on things that customers want to buy?

#5 measuring machine learning impact


Kiri Wagstaff, Machine Learning that Matters, ICML, 2012

The stages of an ML research program

Very inspirational!!!


Phrase a problem as an ML task

Data Wrangling

Learn from Data

The stages of an ML application

Feature Engineering

Pre-evaluate

Measure Impact !!!!!


#6 pricing


Pre-pay-as-you-go


Subscriptions



BigML 1-click model

You can deal with this...

Machine Learning made easy?


BigML 1-click model

You can deal with this...




Ease

-of-

use

+

+

2013



Ease

-of-

use

+

+

2013 2014 2015 2016 2017 2018

Machine Learning made Easy!!!


Questions


Unknown Modelf : X -> Y

Example: ideal credit approval formula

ModelsM

Example: set of candidate credit approval formulas

Learning from Data

LearningAlgorithm

Based on Learning from Data by Y. Abu-Mostafa, M. Magdon-Ismail and H. Lin

Final Modelg ~ f

Example: learned credit approval formula

Training Examples(x1, l1), (x2, l2), ..., (xN, lN)

Example: historical records of credit customers

x1

xN

labelf1 f2 fn

a few challenges to make machine learning easy

Technology

bigml source

bigml resourcescsv

import bigml api

machine learning breadth

machine learning iscomplex

machine learning application

unsupervised learning

reinforcement learning