overview on azure machine learning
TRANSCRIPT
Overview on Azure Machine LearningJames SerraBig Data [email protected]
About Me Microsoft, Big Data Evangelist In IT for 30 years, worked on many BI and DW projects Worked as desktop/web/database developer, DBA, BI and DW architect and
developer, MDM architect, PDW/APS developer Been perm employee, contractor, consultant, business owner Presenter at PASS Business Analytics Conference, PASS Summit, Enterprise Data
World conference Certifications: MCSE: Data Platform, Business Intelligence; MS: Architecting
Microsoft Azure Solutions, Design and Implement Big Data Analytics Solutions, Design and Implement Cloud Data Platform Solutions
Blog at JamesSerra.com Former SQL Server MVP Author of book “Reporting with Microsoft SQL Server 2012”
Advanced Analytics Defined
Microsoft Azure Machine Learning Built for a cloud-first, mobile-first world Fully managed
Integrated Best in Class Algorithms
Deploy in minutes
No software to install, no hardware to manage, and one portal to view and update.
Simple drag, drop and connect interface for Data Science. No need for programming for common tasks.
Built-in collection of best of breed algorithms. Support for R and Python for extensibility.
Operationalize models with a single click. Monetize in Machine Learning Marketplace.
Advanced analytics architectureData to model to web services in minutes
Data preparation Business valueModeling Deployment
• HDFS• RDBMS• NoSQL stores• Blobs and tables
Data
• Desktop files• Spreadsheets• Server stores• Sensors
Cloud
Local
Apps, dashboardsand processes
Storage space
Integrated development environment for machine
learning
MLStudio
http://studio.azureml.net
API
Model is now a web svc
Monetize this API
MMarketplace
Web
• Data factory• Stream analytics
• Machine learning• HDInsight
• Marketplace• Azure portal
• Power BI• Apps
Establish mechanisms to conduct data science activities end-to-end in the cloud or on premises, friction free.
Cortana Analytics Process (CAP)
http://aka.ms/adapt
• Set up a Data Science Environment in the cloud• Move data from on premise to cloud• Explore and understand your data• Build a model with Azure Machine Learning• Deploy model as web-service and consume it• End-to-End walkthroughs with real datasets
Process
Setup Cloud Environment
Load DataExplore DataEngineer FeaturesSample Data
Build Model Deploy Model Consume Model
CAPSet up a Data Science Environment in the cloudVirtual Machines for Data ScienceHadoop clusters for Data ScienceAzureML Workspace, Storage accounts etc.
Move data from on premise to cloudLarge, Medium, Small data scenariosDifferent sources and destinations
Explore and understand your data Visualize data – IPython Notebook, PowerBIPrepare data for modeling - Pre-process data, clean data, engineer featuresSample data for Azure Machine Learning
CAP (contd.)Build a model with Azure Machine LearningFurther manipulate dataTrain, Evaluate and Tune Model(s)Deploy and Consume Model(s)
End-to-End walkthroughsWith real datasets. Code in GitHubHands on workshops, tutorials, training
The FutureNew technology: Revolution R, SQL Server Enterprise Edition, SQL-IPNew Data Types: Periodic refresh data, Streaming data, Structured and Unstructured data, Text data, Social data etc.New Data Sources: On-Premise, Cloud based
Scenario
11
This is Karl.Karl owns a company that
operates vending machines in Washington state.
His job is to make sure that his 100 vending machines are selling drinks & obtaining
revenue.
Karl wants revenue to always be high & his
business to be profitable
Scenario
12
Sadly, vending machine will occasionally break & may take up to 7 days to fix, thus hurting
sales.
To eliminate this occurrence, Karl must maintain operations & figure out the best way to utilize
resources in order to optimize revenue.
Questions & Solutions
3. How Do We Plan Maintenance?
13
Azure Cloud Services + Machine Learning to the Rescue!
1. Which Machines Have Failed?
2. Which Machines Will Soon Fail?
Application Phases
14
• Damage is reported by customer or during weekly restocking routes
• Technician must be scheduled to investigate
• Process take up to 8 days to fixa broken machine
• Sensor data is used to monitor cooler condition in real-time
• Broken coolers are identifiedat time of failure
• Lost sales remain due to maintenance lead teams(parts & repair technicians)
• Azure ML predicts where, when,& what failures will occur based on sensor data
• Spare parts & repairs can be scheduled before machines shut down leading to no lost sales
CURRENT SCENARIO REAL-TIME SENSORS SENSORS & MACHINE LEARNINGDays: Days:Days:
Cloud
Microsoft Azure PortalPublish API
Publish API in Minutes
Web
Workspace & Data Science
Easily Make Changes
ResultsRun & RefineTest ModelsSocial DataLearning Feedback Loop
Stream Analytics
Event Hubs
Azure architecture
>PowerBI Dashboard>Excel API
ML Studio ML API Service
MicrosoftAzure Portal
Blob Storage
ML Apps Marketplace
ML Operationalization
ML Studio
ML Algorithms
Azure ML Marketplace
ProducerConsumer
What can Azure ML do for you…?
Social network analysis
Weather forecasting
Healthcare outcomes
Predictive maintenance
Targeted advertising
Natural resource exploration
Fraud detection
Telemetry data analysis
Buyer propensity models
Churn analysis
Life sciences research
Web app optimization
Network intrusion detection
Smart meter monitoring
MICROSOFT CONF IDENTIAL – INTERNAL ONLY
Health and home
How can I update the layout of my store based on where customers actually go?
Retail
How can I predict which time to perform tasks with autonomous devices in the home?
How can I automate checkout and identify potential fraudulent transactions?
How can I proactively react to abnormal patterns of data?
How can I optimize the movement of my fleet and assets?
How can I heat and cool my buildings based on usage and weather?
How can I fix equipment proactively before it fails?
How can I stock my trucks appropriately for the day and week?
How can I track customers within my store and create personalized offers for them?
How can I predict consumer’s health progress?
How can I identify if there is an intruder trying to break in?
ML Questions within IoT ScenariosOperations and workforce
Model Your Way: Open source/our sourceScript with R, SQLite or Python CPython 2.7 support from inside AML Studionumpy/scipy/panda/scikit-learn/etc. Anaconda distro pre-installed
Python client library Analyze data using Python and its librariesUse IPython, PTVS, Eclipse to edit/debug
Big learning with countsTB scale datasets Modular: tune/monitor/replace in isolationMonitorable and debuggable
Deploy in MinutesOne click to production Publish as a Web Service or to Gallery Continuous updates to streamline process Stay tuned to our blog for more
New in-product GalleryDiscover what others have built Learn by dropping these into your workspaceShare your work with others
Expand your Reach
• Accessible through a web browser, no software to install;
• Collaborative work with anyone, anywhere via Azure workspace;
• Visual composition with end2end support for data science workflow;
• Best in class ML algorithms; Immutable library of models, search discover and reuse;
• Extensible, support for R & Python;• Rapidly try a range of features, ML
algorithms and modeling strategies
ML Studio
Cortana Intelligence SuiteIntegrated as part of an end-to-end suite
Action
People
Automated Systems
Apps
Web
Mobile
Bots
Intelligence
Dashboards & Visualizations
Cortana
Bot Framework
Cognitive Services
Power BI
Information Management
Event Hubs
Data Catalog
Data Factory
Machine Learning and Analytics
HDInsight (Hadoop and Spark)
Stream Analytics
Intelligence
Data Lake Analytics
Machine Learning
Big Data Stores
SQL Data Warehouse
Data Lake Store
Data Sources
Apps
Sensors and devices
Data
Event Hub Stores
Streaming Data
Stream Analytics processes events as
they arrive in the EventHub
AML Model Web Service
BES endpoint
Power BI / D3
Dashboard
Data for Real-time Processing
Aggregations
External Data
Azure Services
Azure SQLContains Historical
Energy Consumption Data
Real time data stats
Azure Data Factory
Pipeline invokes AML Web Service
Real
Tim
eBa
tch
Example Architecture
Real Time Telemetry
Data
Azure Data Factory
Pipeline Moves Data
Batch updates of predictions
AML Model Web Service
RRS endpoint
Architecture
Real Time Energy
Consumption Data (Public
Source)
Event Hub Stores
Streaming Data
Stream Analytics processes events as
they arrive in the EventHub
AML Model Web Service
BES endpoint
Power BI / D3
Dashboard
Data for Real-time Processing
Data Stream
Job
Hourly Prediction Updates
External Data
Azure Services
Copy to Azure SQL for batch
predictions
Scrape Data
5 mins
Azure WebJob Runs jobs to scrape
data from public source
Azure SQLContains Historical
Energy Consumption Data
Real time data stats
Azure Data Factory
Pipeline invokes AML Web Service
Real
Tim
eBa
tch
Q & A ?James Serra, Big Data EvangelistEmail me at: [email protected] me at: @JamesSerra Link to me at: www.linkedin.com/in/JamesSerra Visit my blog at: JamesSerra.com (where this slide deck is posted via the “Presentations” link on the top menu)