“let the computers do the hard work”...how did caterpillar do with our tools? semi-automated...
TRANSCRIPT
![Page 1: “Let the computers do the hard work”...How Did Caterpillar Do with Our Tools? Semi-automated labeling process –“We go from having to label 100 percent of our data to only having](https://reader033.vdocuments.net/reader033/viewer/2022050101/5f408fb057e3291b76296ef6/html5/thumbnails/1.jpg)
1© 2015 The MathWorks, Inc.
Demystifying Deep Learning“Let the computers do the hard work”
Pitambar Dayal
Product MarketingDeep Learning, Image Processing
![Page 2: “Let the computers do the hard work”...How Did Caterpillar Do with Our Tools? Semi-automated labeling process –“We go from having to label 100 percent of our data to only having](https://reader033.vdocuments.net/reader033/viewer/2022050101/5f408fb057e3291b76296ef6/html5/thumbnails/2.jpg)
2
![Page 3: “Let the computers do the hard work”...How Did Caterpillar Do with Our Tools? Semi-automated labeling process –“We go from having to label 100 percent of our data to only having](https://reader033.vdocuments.net/reader033/viewer/2022050101/5f408fb057e3291b76296ef6/html5/thumbnails/3.jpg)
3
Why MATLAB for Deep Learning?
▪ MATLAB is Productive
▪ MATLAB Supports the Entire Deep Learning
Workflow
▪ MATLAB Integrates with Open Source
![Page 4: “Let the computers do the hard work”...How Did Caterpillar Do with Our Tools? Semi-automated labeling process –“We go from having to label 100 percent of our data to only having](https://reader033.vdocuments.net/reader033/viewer/2022050101/5f408fb057e3291b76296ef6/html5/thumbnails/4.jpg)
4
What is Deep Learning?
A. Deep learning is learning done really far underground.
B. I don’t know, I’m just here for the free snacks.
C. Deep learning is machine learning with automatic
feature extraction. It uses a neural network architecture
to perform tasks like classification, detection, and
regression.
![Page 5: “Let the computers do the hard work”...How Did Caterpillar Do with Our Tools? Semi-automated labeling process –“We go from having to label 100 percent of our data to only having](https://reader033.vdocuments.net/reader033/viewer/2022050101/5f408fb057e3291b76296ef6/html5/thumbnails/5.jpg)
5
Deep Learning
What is Deep Learning?
Machine
Learning
Deep
Learning
▪ Subset of machine learning with automatic feature extraction
– Learns features and tasks directly from data
– More Data = better model
6
Deep Learning
What is Deep Learning?
Machine
Learning
Deep
Learning
§ Subset of machine learning with automatic feature extraction
– Learns features and tasks directly from data
– More Data = better model
![Page 6: “Let the computers do the hard work”...How Did Caterpillar Do with Our Tools? Semi-automated labeling process –“We go from having to label 100 percent of our data to only having](https://reader033.vdocuments.net/reader033/viewer/2022050101/5f408fb057e3291b76296ef6/html5/thumbnails/6.jpg)
6
Deep Learning Uses a Neural Network Architecture
Input
Layer Hidden Layers (n)
Output
Layer
![Page 7: “Let the computers do the hard work”...How Did Caterpillar Do with Our Tools? Semi-automated labeling process –“We go from having to label 100 percent of our data to only having](https://reader033.vdocuments.net/reader033/viewer/2022050101/5f408fb057e3291b76296ef6/html5/thumbnails/7.jpg)
7
Deep Learning Datatypes
SignalImage
TextNumeric
![Page 8: “Let the computers do the hard work”...How Did Caterpillar Do with Our Tools? Semi-automated labeling process –“We go from having to label 100 percent of our data to only having](https://reader033.vdocuments.net/reader033/viewer/2022050101/5f408fb057e3291b76296ef6/html5/thumbnails/8.jpg)
8
Deep Learning is Versatile
Iris Recognition – 99.4% accuracy2
Rain Detection and Removal1Detection of cars and road in autonomous driving systems
1. Deep Joint Rain Detection and Removal from a Single Image" Wenhan Yang,
Robby T. Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan
2. Source: An experimental study of deep convolutional features for iris recognition
Signal Processing in Medicine and Biology Symposium (SPMB), 2016 IEEE
Shervin Minaee ; Amirali Abdolrashidiy ; Yao Wang; An experimental study of
deep convolutional features for iris recognition
![Page 9: “Let the computers do the hard work”...How Did Caterpillar Do with Our Tools? Semi-automated labeling process –“We go from having to label 100 percent of our data to only having](https://reader033.vdocuments.net/reader033/viewer/2022050101/5f408fb057e3291b76296ef6/html5/thumbnails/9.jpg)
9
Why MATLAB for Deep Learning?
▪ MATLAB is Productive
▪ MATLAB Supports the Entire Deep Learning
Workflow
▪ MATLAB integrates with Open Source
![Page 10: “Let the computers do the hard work”...How Did Caterpillar Do with Our Tools? Semi-automated labeling process –“We go from having to label 100 percent of our data to only having](https://reader033.vdocuments.net/reader033/viewer/2022050101/5f408fb057e3291b76296ef6/html5/thumbnails/10.jpg)
10
Deep Learning in 6 Lines of MATLAB Code
![Page 11: “Let the computers do the hard work”...How Did Caterpillar Do with Our Tools? Semi-automated labeling process –“We go from having to label 100 percent of our data to only having](https://reader033.vdocuments.net/reader033/viewer/2022050101/5f408fb057e3291b76296ef6/html5/thumbnails/11.jpg)
11
“I love to label and
preprocess my data”
~ Said no engineer, ever.
![Page 12: “Let the computers do the hard work”...How Did Caterpillar Do with Our Tools? Semi-automated labeling process –“We go from having to label 100 percent of our data to only having](https://reader033.vdocuments.net/reader033/viewer/2022050101/5f408fb057e3291b76296ef6/html5/thumbnails/12.jpg)
12
Caterpillar Case Study
▪ World’s leading manufacturer of
construction and mining
equipment.
▪ Similarity between these
projects?
– Autonomous haul trucks
– Pedestrian detection
– Equipment classification
– Terrain mapping
![Page 13: “Let the computers do the hard work”...How Did Caterpillar Do with Our Tools? Semi-automated labeling process –“We go from having to label 100 percent of our data to only having](https://reader033.vdocuments.net/reader033/viewer/2022050101/5f408fb057e3291b76296ef6/html5/thumbnails/13.jpg)
13
Computer Must Learn from Lots of Data
▪ ALL data must first be labeled to create these autonomous systems.
“We were spending way too much time ground-truthing [the data]”
--Larry Mianzo, Caterpillar
![Page 14: “Let the computers do the hard work”...How Did Caterpillar Do with Our Tools? Semi-automated labeling process –“We go from having to label 100 percent of our data to only having](https://reader033.vdocuments.net/reader033/viewer/2022050101/5f408fb057e3291b76296ef6/html5/thumbnails/14.jpg)
14
How Did Caterpillar Do with Our Tools?
▪ Semi-automated labeling process– “We go from having to label 100 percent of our data to only having to
label about 80 to 90 percent”
▪ Used MATLAB for entire development workflow.
– “Because everything is in MATLAB, development time is short”
![Page 15: “Let the computers do the hard work”...How Did Caterpillar Do with Our Tools? Semi-automated labeling process –“We go from having to label 100 percent of our data to only having](https://reader033.vdocuments.net/reader033/viewer/2022050101/5f408fb057e3291b76296ef6/html5/thumbnails/15.jpg)
15
How Does MATLAB Come into Play?
![Page 16: “Let the computers do the hard work”...How Did Caterpillar Do with Our Tools? Semi-automated labeling process –“We go from having to label 100 percent of our data to only having](https://reader033.vdocuments.net/reader033/viewer/2022050101/5f408fb057e3291b76296ef6/html5/thumbnails/16.jpg)
16
![Page 17: “Let the computers do the hard work”...How Did Caterpillar Do with Our Tools? Semi-automated labeling process –“We go from having to label 100 percent of our data to only having](https://reader033.vdocuments.net/reader033/viewer/2022050101/5f408fb057e3291b76296ef6/html5/thumbnails/17.jpg)
17
![Page 18: “Let the computers do the hard work”...How Did Caterpillar Do with Our Tools? Semi-automated labeling process –“We go from having to label 100 percent of our data to only having](https://reader033.vdocuments.net/reader033/viewer/2022050101/5f408fb057e3291b76296ef6/html5/thumbnails/18.jpg)
18
![Page 19: “Let the computers do the hard work”...How Did Caterpillar Do with Our Tools? Semi-automated labeling process –“We go from having to label 100 percent of our data to only having](https://reader033.vdocuments.net/reader033/viewer/2022050101/5f408fb057e3291b76296ef6/html5/thumbnails/19.jpg)
19
MATLAB is Productive
▪ Image Labeler App semi-automates labeling workflow
▪ Bootstrapping
– Improve automatic labeling by updating algorithm as you label
more images correctly.
▪ Easy to load metadata even when labeling manually
![Page 20: “Let the computers do the hard work”...How Did Caterpillar Do with Our Tools? Semi-automated labeling process –“We go from having to label 100 percent of our data to only having](https://reader033.vdocuments.net/reader033/viewer/2022050101/5f408fb057e3291b76296ef6/html5/thumbnails/20.jpg)
20
Why MATLAB for Deep Learning?
▪ MATLAB is Productive
▪ MATLAB Supports the Entire Deep Learning
Workflow
▪ MATLAB integrates with Open Source
![Page 21: “Let the computers do the hard work”...How Did Caterpillar Do with Our Tools? Semi-automated labeling process –“We go from having to label 100 percent of our data to only having](https://reader033.vdocuments.net/reader033/viewer/2022050101/5f408fb057e3291b76296ef6/html5/thumbnails/21.jpg)
21
Deep Learning Workflow
Files
Databases
Sensors
ACCESS AND EXPLORE
DATA
DEVELOP PREDICTIVE
MODELS
Hardware-Accelerated
Training
Hyperparameter Tuning
Network Visualization
LABEL AND PREPROCESS
DATA
Data Augmentation/
Transformation
Labeling Automation
Import Reference
Models
INTEGRATE MODELS WITH
SYSTEMS
Desktop Apps
Enterprise Scale Systems
Embedded Devices and
Hardware
![Page 22: “Let the computers do the hard work”...How Did Caterpillar Do with Our Tools? Semi-automated labeling process –“We go from having to label 100 percent of our data to only having](https://reader033.vdocuments.net/reader033/viewer/2022050101/5f408fb057e3291b76296ef6/html5/thumbnails/22.jpg)
22
Files
Databases
Sensors
ACCESS AND EXPLORE
DATA
▪ Datastore
useful for large
datasets
▪ Built in read
support for
many file types
![Page 23: “Let the computers do the hard work”...How Did Caterpillar Do with Our Tools? Semi-automated labeling process –“We go from having to label 100 percent of our data to only having](https://reader033.vdocuments.net/reader033/viewer/2022050101/5f408fb057e3291b76296ef6/html5/thumbnails/23.jpg)
23
LABEL AND PREPROCESS
DATA
Data Augmentation/
Transformation
Labeling Automation
Import Reference
Models
![Page 24: “Let the computers do the hard work”...How Did Caterpillar Do with Our Tools? Semi-automated labeling process –“We go from having to label 100 percent of our data to only having](https://reader033.vdocuments.net/reader033/viewer/2022050101/5f408fb057e3291b76296ef6/html5/thumbnails/24.jpg)
24
DEVELOP PREDICTIVE
MODELS
Hardware-Accelerated
Training
Hyperparameter Tuning
Network Visualization
![Page 25: “Let the computers do the hard work”...How Did Caterpillar Do with Our Tools? Semi-automated labeling process –“We go from having to label 100 percent of our data to only having](https://reader033.vdocuments.net/reader033/viewer/2022050101/5f408fb057e3291b76296ef6/html5/thumbnails/25.jpg)
25
DEVELOP PREDICTIVE
MODELS
Hardware-Accelerated
Training
Hyperparameter Tuning
Network Visualization
AlexNetPRETRAINED MODEL
VGG-16PRETRAINED MODEL
VGG-19PRETRAINED MODEL
ResNet-50PRETRAINED MODEL
GoogLeNetPRETRAINED MODEL
ResNet-101PRETRAINED MODEL
Inception-v3PRETRAINED MODEL
DenseNet-201PRETRAINED MODEL
SqueezeNetPRETRAINED MODEL
ResNet-18PRETRAINED MODEL
ResNet-50PRETRAINED MODEL
Inception-
ResNet-v2PRETRAINED MODEL
Easy access to many
pre-trained models
![Page 26: “Let the computers do the hard work”...How Did Caterpillar Do with Our Tools? Semi-automated labeling process –“We go from having to label 100 percent of our data to only having](https://reader033.vdocuments.net/reader033/viewer/2022050101/5f408fb057e3291b76296ef6/html5/thumbnails/26.jpg)
26
DEVELOP PREDICTIVE
MODELS
Hardware-Accelerated
Training
Hyperparameter Tuning
Network Visualization
Single CPU
Single CPUSingle GPU
Single CPU, Multiple GPUs
On-prem server with GPUs
Cloud GPUs(AWS)
![Page 27: “Let the computers do the hard work”...How Did Caterpillar Do with Our Tools? Semi-automated labeling process –“We go from having to label 100 percent of our data to only having](https://reader033.vdocuments.net/reader033/viewer/2022050101/5f408fb057e3291b76296ef6/html5/thumbnails/27.jpg)
27
INTEGRATE MODELS WITH
SYSTEMS
Desktop Apps
Enterprise Scale Systems
Embedded Devices and
HardwareMATLAB Code
Coder
Products
Deployment
Target
C/C++/HDL/etc.
![Page 28: “Let the computers do the hard work”...How Did Caterpillar Do with Our Tools? Semi-automated labeling process –“We go from having to label 100 percent of our data to only having](https://reader033.vdocuments.net/reader033/viewer/2022050101/5f408fb057e3291b76296ef6/html5/thumbnails/28.jpg)
28
Deploying Deep Learning Models for Inference
Coder
Products
Deep Learning
Networks
NVIDIA
TensorRT &
cuDNN
Libraries
ARM
Compute
Library
Intel
MKL-DNN
Library
![Page 29: “Let the computers do the hard work”...How Did Caterpillar Do with Our Tools? Semi-automated labeling process –“We go from having to label 100 percent of our data to only having](https://reader033.vdocuments.net/reader033/viewer/2022050101/5f408fb057e3291b76296ef6/html5/thumbnails/29.jpg)
29
Desktop CPU
Raspberry Pi board
Deploying to Various Targets
Coder
Products
Deep Learning
Networks
NVIDIA
TensorRT &
cuDNN
Libraries
![Page 30: “Let the computers do the hard work”...How Did Caterpillar Do with Our Tools? Semi-automated labeling process –“We go from having to label 100 percent of our data to only having](https://reader033.vdocuments.net/reader033/viewer/2022050101/5f408fb057e3291b76296ef6/html5/thumbnails/30.jpg)
30
With GPU Coder, MATLAB is fast
Intel® Xeon® CPU 3.6 GHz - NVIDIA libraries: CUDA9 - cuDNN 7 - Frameworks: TensorFlow 1.8.0, MXNet 1.2.1, PyTorch 0.3.1
GPU Coder is faster
than TensorFlow,
MXNet and Pytorch
TensorFlow
MXNet
GPU Coder
PyTorch
![Page 31: “Let the computers do the hard work”...How Did Caterpillar Do with Our Tools? Semi-automated labeling process –“We go from having to label 100 percent of our data to only having](https://reader033.vdocuments.net/reader033/viewer/2022050101/5f408fb057e3291b76296ef6/html5/thumbnails/31.jpg)
31
Deep Learning Workflow
Files
Databases
Sensors
ACCESS AND EXPLORE
DATA
DEVELOP PREDICTIVE
MODELS
Hardware-Accelerated
Training
Hyperparameter Tuning
Network Visualization
LABEL AND PREPROCESS
DATA
Data Augmentation/
Transformation
Labeling Automation
Import Reference
Models
INTEGRATE MODELS WITH
SYSTEMS
Desktop Apps
Enterprise Scale Systems
Embedded Devices and
Hardware
![Page 32: “Let the computers do the hard work”...How Did Caterpillar Do with Our Tools? Semi-automated labeling process –“We go from having to label 100 percent of our data to only having](https://reader033.vdocuments.net/reader033/viewer/2022050101/5f408fb057e3291b76296ef6/html5/thumbnails/32.jpg)
32
Why MATLAB?
▪ MATLAB is Productive
▪ MATLAB Supports the Entire Deep Learning
Workflow
▪ MATLAB Integrates with Open Source
![Page 33: “Let the computers do the hard work”...How Did Caterpillar Do with Our Tools? Semi-automated labeling process –“We go from having to label 100 percent of our data to only having](https://reader033.vdocuments.net/reader033/viewer/2022050101/5f408fb057e3291b76296ef6/html5/thumbnails/33.jpg)
33
Used MATLAB and Open Source Together
1. Deep Joint Rain Detection and Removal from a Single
Image" Wenhan Yang, Robby T. Tan, Jiashi Feng,
Jiaying Liu, Zongming Guo, and Shuicheng Yan
▪ Used Caffe and MATLAB
together
▪ Achieved significantly
better results than an
engineered rain model.
▪ Use our tools where it
makes your workflow
easier!
![Page 34: “Let the computers do the hard work”...How Did Caterpillar Do with Our Tools? Semi-automated labeling process –“We go from having to label 100 percent of our data to only having](https://reader033.vdocuments.net/reader033/viewer/2022050101/5f408fb057e3291b76296ef6/html5/thumbnails/34.jpg)
34
MATLAB Integrates with Open Source Frameworks
Pre-trained models
Import models from other
frameworks via ONNX or direct from
Keras-TensorFlow and Caffe
![Page 35: “Let the computers do the hard work”...How Did Caterpillar Do with Our Tools? Semi-automated labeling process –“We go from having to label 100 percent of our data to only having](https://reader033.vdocuments.net/reader033/viewer/2022050101/5f408fb057e3291b76296ef6/html5/thumbnails/35.jpg)
35
ONNX is an open format to represent deep learning models
ONNX = Open Neural Network Exchange Format
![Page 36: “Let the computers do the hard work”...How Did Caterpillar Do with Our Tools? Semi-automated labeling process –“We go from having to label 100 percent of our data to only having](https://reader033.vdocuments.net/reader033/viewer/2022050101/5f408fb057e3291b76296ef6/html5/thumbnails/36.jpg)
36
ONNX Converter
Import from ONNX format
Export to ONNX format
![Page 37: “Let the computers do the hard work”...How Did Caterpillar Do with Our Tools? Semi-automated labeling process –“We go from having to label 100 percent of our data to only having](https://reader033.vdocuments.net/reader033/viewer/2022050101/5f408fb057e3291b76296ef6/html5/thumbnails/37.jpg)
37
MATLAB Integrates with Open Source
Frameworks
▪ Use MATLAB for parts of workflow
▪ Access to latest models
▪ Improved collaboration with other users
![Page 38: “Let the computers do the hard work”...How Did Caterpillar Do with Our Tools? Semi-automated labeling process –“We go from having to label 100 percent of our data to only having](https://reader033.vdocuments.net/reader033/viewer/2022050101/5f408fb057e3291b76296ef6/html5/thumbnails/38.jpg)
38
Why MATLAB for Deep Learning?
▪ MATLAB is Productive
▪ MATLAB Supports Entire Deep Learning
Workflow
▪ MATLAB Integrates with Open Source
![Page 39: “Let the computers do the hard work”...How Did Caterpillar Do with Our Tools? Semi-automated labeling process –“We go from having to label 100 percent of our data to only having](https://reader033.vdocuments.net/reader033/viewer/2022050101/5f408fb057e3291b76296ef6/html5/thumbnails/39.jpg)
39
Other Deep Learning Expo Events
▪ Deep Learning Workshops – 1pm, 2pm, 3:30pm
▪ Deep Learning Booth
▪ Deep Learning for Signals – 3:30pm