using predictive apis to create smarter apps
TRANSCRIPT
@louisdorard
#apidays
PredictiveAPIs
–Mike Gualtieri, Principal Analyst at Forrester
“Predictive apps are the next big thing
in app development.”
–Waqar Hasan, VISA
“Predictive is the ‘killer app’ for big data.”
1. Machine Learning 2. Data
BUT
–McKinsey & Co. (2011)
“A significant constraint on realizing value from big data will be a shortage of talent, particularly of people with
deep expertise in statistics and machine learning.”
What the @#?~% is ML?
“Which type of email is this? — Spam/Ham”-> Classification
“Which type of email is this? — Spam/Ham”
I
O
• Need examples of inputs AND outputs • Need enough examples
??
Predictive APIs
The two phases of machine learning: • TRAIN a model • PREDICT with a model
The two methods of predictive APIs: • TRAIN a model • PREDICT with a model
The two methods of predictive APIs: • model = create_model(dataset)
• predicted_output = create_prediction(model, new_input)
from bigml.api import BigML
# create a modelapi = BigML()source = api.create_source('training_data.csv')dataset = api.create_dataset(source)model = api.create_model(dataset)
# make a predictionprediction = api.create_prediction(model, new_input)print "Predicted output value: ",prediction['object']['output']
http://bit.ly/bigml_wakari
from bigml.api import BigML
# create a modelapi = BigML()source = api.create_source('training_data.csv')dataset = api.create_dataset(source)model = api.create_model(dataset)
# make a predictionprediction = api.create_prediction(model, new_input)print "Predicted output value: ",prediction['object']['output']
http://bit.ly/bigml_wakari
Different needs, different Predictive APIs
AB
STRA
CTIO
N
Algorithmic Generic
Generic for text Problem-specific
Fixed-model
AB
STRA
CTIO
N
Algorithmic Generic
Generic for text Problem-specific
Fixed-model
“Is this email important? — Yes/No”
Generic Predictive APIs:• Google Prediction API • BigML • PredicSis • WolframCloud • Amazon
AMAZON GOOGLE PREDICSIS BIGML
ACCURACY 0.862 0.743 0.858 0.790
TRAINING TIME 135s 76s 17s 5s
TEST TIME 188s 369s 5s 1s
louisdorard.com/blog/machine-learning-apis-comparison
AB
STRA
CTIO
N
Algorithmic Generic
Generic for textProblem-specific
Fixed-model
“Which project does this document relate to? — A/B/C”
Text classification APIs:• uClassify.com • MonkeyLearn.com • Cortical.io
AB
STRA
CTIO
N
Algorithmic Generic
Generic for text Problem-specific
Fixed-model
“Is this customer going to leave next month? — Yes/No”
“Is this transaction fraudulent? — Yes/No”
Problem-specific Predictive APIs:• Churn: ChurnSpotter.io, Framed.io • Fraud detection: SiftScience.com • Lead scoring: Infer.com • Personal assistant (Siri): Wit.ai
AB
STRA
CTIO
N
Algorithmic Generic
Generic for text Problem-specific
Fixed-model
“What is the sentiment of this tweet? — Positive/Neutral/Negative”
“Is this email spam? — Yes/No”
Fixed-model Predictive APIs:• Text: Datumbox.com, Semantria.com • Vision: Indico, AlchemyAPI.com • Siri-like: maluuba.com
AB
STRA
CTIO
N
Algorithmic Generic
Generic for text Problem-specific
Fixed-model
CU
STOM
IZATION
Algorithmic Generic
Generic for text Problem-specific
Fixed-model
Own MLalgorithm?
Own data processingalgorithm?
Data processing APIfication
MLlib
• “Open source prediction server” • Based on Spark • Expose predictive models as (scalable
& robust) APIs
= PAPI+
• Send (new) data • Send prediction queries
cli = predictionio.EventClient("<my_app_id>") cli.record_user_action_on_item("view","John","Item") eng = predictionio.EngineClient("<my_engine_url>") rec = eng.send_query({"uid":"John","n":5})
Use cases
• Increase e-commerce revenue with recos • Reduce attrition (“churn”) • Automate decision making • Predict real-estate prices w. open data • Predict crowds (SNCF & La Poste) • …
