1© 2015 The MathWorks, Inc.

Machine Learning for Predictive Modelling

Rory Adams

2

Machine Learning

– What is Machine Learning and why do we need it?

– Common challenges in Machine Learning

Example: Human activity learning using mobile phone data

Example: Real-time object identification using images

Example: Load forecasting using weather data

Summary & Key Takeaways

Agenda

3

Machine Learning is Everywhere

Image Recognition

Speech Recognition

Stock Prediction

Medical Diagnosis

Data Analytics

Robotics

and more…

[TBD]

4

Machine Learning

Machine learning uses data and produces a model to perform a task

Standard Approach Machine Learning Approach

𝑚𝑜𝑑𝑒𝑙 = <𝑴𝒂𝒄𝒉𝒊𝒏𝒆𝑳𝒆𝒂𝒓𝒏𝒊𝒏𝒈𝑨𝒍𝒈𝒐𝒓𝒊𝒕𝒉𝒎

>(𝑠𝑒𝑛𝑠𝑜𝑟_𝑑𝑎𝑡𝑎, 𝑎𝑐𝑡𝑖𝑣𝑖𝑡𝑦)

Computer

Program

Machine

Learning

𝑚𝑜𝑑𝑒𝑙: Inputs → OutputsHand Written Program Formula or Equation

If X_acc > 0.5

then “SITTING”

If Y_acc < 4 and Z_acc > 5

then “STANDING”

…

𝑌𝑎𝑐𝑡𝑖𝑣𝑖𝑡𝑦= 𝛽1𝑋𝑎𝑐𝑐 + 𝛽2𝑌𝑎𝑐𝑐+ 𝛽3𝑍𝑎𝑐𝑐 +

…

Task: Human Activity Detection

𝑚𝑜𝑑𝑒𝑙: Predictors → Response

5

Different Types of Learning

Machine Learning

Supervised Learning

Classification

Regression

Unsupervised Learning

• Discover a good internal representation

• Learn a low dimensional representation

• Response is a continuous number

(temperature, stock prices).

• Response is a choice between classes

• (True, False) (Red, Blue, Green)

6

Example: Human Activity Learning Using Mobile Phone Data

Machine

Learning

Data:

3-axial Accelerometer data

3-axial Gyroscope data

7

“essentially, all models are wrong,

but some are useful”

– George Box

8

Steps Challenge

Access, explore and analyze

dataData diversity

Numeric, Images, Signals, Text – not always tabular

Preprocess dataLack of domain tools

Filtering and feature extraction

Feature selection and transformation

Train modelsTime consuming

Train many models to find the “best”

Assess model performanceAvoid pitfalls

Over Fitting

Speed-Accuracy-Complexity tradeoffs

Iterate

Challenges in Machine LearningHard to get started

Steps Challenge

Access, explore and analyze

dataData diversity

Numeric, Images, Signals, Text – not always tabular

Preprocess dataLack of domain tools

Filtering and feature extraction

Feature selection and transformation

Train modelsTime consuming

Train many models to find the “best”

Assess model performanceAvoid pitfalls

Over Fitting

Speed-Accuracy-Complexity tradeoffs

Iterate

9

PREDICTION

Machine Learning Workflow

Train: Iterate till you find the best model

Predict: Integrate trained models into applications

MODELSUPERVISED

LEARNING

CLASSIFICATION

REGRESSION

PREPROCESS

DATA

SUMMARY

STATISTICS

PCAFILTERS

CLUSTER

ANALYSIS

LOAD

DATAPREPROCESS

DATA

SUMMARY

STATISTICS

PCAFILTERS

CLUSTER

ANALYSIS

NEW

DATA

PREPROCESS

DATA

SUMMARY

STATISTICS

PCAFILTERS

CLUSTER

ANALYSIS

MODEL

10

Machine Learning

– What is Machine Learning and why do we need it?

– Common challenges in Machine Learning

Example: Human activity learning using mobile phone data

Example: Real-time object identification using images

Example: Load forecasting using weather data

Summary & Key Takeaways

Agenda

11

Example 1: Human Activity Learning Using Mobile Phone Data

Objective: Train a classifier to classify

human activity from sensor data

Data:

Approach:

– Extract features from raw sensor signals

– Train and compare classifiers

– Test results on new sensor data

Predictors 3-axial Accelerometer and

Gyroscope data

Response Activity:

(Classification)

12

PREDICTIONMODEL

Machine Learning Workflow for Example 1

Train: Iterate till you find the best model

Predict: Integrate trained models into applications

MODELSUPERVISED

LEARNING

CLASSIFICATION

REGRESSION

PREPROCESS

DATA

SUMMARY

STATISTICS

PCAFILTERS

CLUSTER

ANALYSIS

LOAD

DATA

PREPROCESS

DATA

SUMMARY

STATISTICS

PCAFILTERS

CLUSTER

ANALYSIS

TEST

DATA

1. Mean

2. Standard

deviation

3. PCA

Classification

Learner

1. Mean

2. Standard

deviation

3. PCA

13

Machine Learning

– What is Machine Learning and why do we need it?

– Common challenges in Machine Learning

Example: Human activity learning using mobile phone data

Example: Real-time object identification using images

Example: Load forecasting using weather data

Summary & Key Takeaways

Agenda

14

Example 2: Real-time Toy Identification Using Images

Objective: Train a classifier to identify toy

type from a webcam video

Data:

Approach:

– Extract features using Bag-of-words

– Train and compare classifiers

– Classify streaming video from a webcam

Predictors Several images of cars:

Response CAR, HELICOPTER, PLANE, BIKE

(Classification)

15

PREDICTIONMODEL

Machine Learning Workflow for Example

Train: Iterate till you find the best model

Predict: Integrate trained models into applications

MODELSUPERVISED

LEARNING

CLASSIFICATION

REGRESSION

PREPROCESS

DATA

SUMMARY

STATISTICS

PCAFILTERS

CLUSTER

ANALYSIS

LOAD

DATA

PREPROCESS

DATA

SUMMARY

STATISTICS

PCAFILTERS

CLUSTER

ANALYSIS

WEBCAM

1. Build Bag-of-

features

2. Encode images

as new features

Classification

Learner

Encode images as

new features

16

Machine Learning

– What is Machine Learning and why do we need it?

– Common challenges in Machine Learning

Example: Human activity learning using mobile phone data

Example: Real-time object identification using images

Example: Load forecasting using weather data

Summary & Key Takeaways

Agenda

17

Example 3: Day-Ahead System Load Forecasting

Objective: Train a neural network to predict

the required system load for a zone

Data:

Approach:

– Extract additional features

– Train neural network

– Predict load

Predictors Temperature, Dew point, Month,

Day of week, Prior day load, Prior

week load

Response LOAD

(Regression)

18

PREDICTIONMODEL

Machine Learning Workflow for Example 1

Train: Iterate till you find the best model

Predict: Integrate trained models into applications

MODELSUPERVISED

LEARNING

CLASSIFICATION

REGRESSION

PREPROCESS

DATA

SUMMARY

STATISTICS

PCAFILTERS

CLUSTER

ANALYSIS

LOAD

DATA

PREPROCESS

DATA

SUMMARY

STATISTICS

PCAFILTERS

CLUSTER

ANALYSIS

TEST

DATA

Temp, Dew point

Day of week

Prior day load

Prior week load

Neural Network

Temp, Dew point

Day of week

Prior day load

Prior week load

19

Machine Learning

– What is Machine Learning and why do we need it?

– Common challenges in Machine Learning

Example: Human activity learning using mobile phone data

Example: Real-time object identification using images

Example: Load forecasting using weather data

Summary & Key Takeaways

Agenda

20

Steps Challenge

Accessing, exploring and

analyzing dataData diversity

Preprocess data Lack of domain tools

Train models Time consuming

Assess model

performance

Avoid pitfallsOver Fitting,

Speed-Accuracy-Complexity

Iterate

Challenges in Machine Learning

21

MATLAB Strengths for Machine Learning

Challenge Solution

Data diversityExtensive data support

Import and work with signal, images, financial,

Textual, geospatial, and several others formats

Lack of domain toolsHigh-quality libraries

Industry-standard algorithms for Finance, Statistics, Signal,

Image processing & more

Time consuming Interactive, app-driven workflows

Focus on machine learning, not programing

Avoid pitfallsOver Fitting,

Speed-Accuracy-Complexity

Integrated best practicesModel validation tools built into app

Rich documentation with step by step guidance

Flexible architecture for customized workflowsComplete machine learning platform

22

Consider Machine Learning when:

– Hand written rules and equations are too complex

Face recognition, speech recognition, recognizing patterns

– Rules of a task are constantly changing

Fraud detection from transactions, anomaly in sensor data

– Nature of the data changes and the program needs to adapt

Automated trading, energy demand forecasting, predicting shopping trends

MATLAB for Machine Learning

Key Takeaways

23

Additional Resources

Documentation:Machine Learning with MATLAB:

24

Q & A

Topic of interest Session / Demo Station

Working with IoT data Session: MATLAB and the Internet of Things (IoT): Collecting and

Analysing IoT Data

Accessing, analysing and

visualising data

Session: Analysis of Experimental and Test Data

Working with big data sets Session: Tackling Big Data with MATLAB

Deploying machine learning

algorithms

Demo: Building MATLAB Apps to Visualise Complex Data

Machine learning with computer

vision

Demo: Identification of Objects in Real-Time Video