cs 410: intro to software engineering deepcuremarc/cs410/swe02-21.pdf · 29 deep learning software...
TRANSCRIPT
Overview
Introduction ( ~5 mins )
Deep Learning 101 ( ~25 mins )
Notable Applications ( ~ 15 mins )
Hands-on coding ( ~ 30 mins )
Discussion ( @Piazza, after class )
Historical Overview
Slide adapted from: https://www.slideshare.net/LuMa921/deep-learning-the-past-present-and-future-of-artificial-intelligence
14 mil. images
25k movie reviews
40 mil. tweets> 100 datasets
$700 for 1 TeraFlop
6
1000s of startups on top and around DL Tech
Slide adapted from: https://www.slideshare.net/LuMa921/deep-learning-the-past-present-and-future-of-artificial-intelligence
7
Supervised Learning
Machine Learning Tasks - Classification
© DeepCure, Inc. All Rights Reserved.
8
Unsupervised Learning
Machine Learning Tasks - Classification
© DeepCure, Inc. All Rights Reserved.
© DeepCure, Inc. All Rights Reserved. 9
Unsupervised Learning
Machine Learning Tasks - Classification
10
Machine Learning Tasks - Regression
165 CM??? CM
177 CM182 CM
© DeepCure, Inc. All Rights Reserved.
© DeepCure, Inc. All Rights Reserved. 11
Machine Learning Tasks - Generative Modeling
© DeepCure, Inc. All Rights Reserved. 12
Machine Learning Tasks - Reinforcement Learning
© DeepCure, Inc. All Rights Reserved. 13
Deep Learning V. Machine Learning
© DeepCure, Inc. All Rights Reserved. 14
Deep Learning V. Machine Learning
© DeepCure, Inc. All Rights Reserved. 15
Deep Learning V. Machine Learning
Multi-Layer Perceptrons (MLPs)
Slide adapted from: http://cs231n.stanford.edu/slides/2017/cs231n_2017_lecture4.pdf
Multi-Layer Perceptrons (MLPs)
Slide adapted from: http://cs231n.stanford.edu/slides/2017/cs231n_2017_lecture4.pdf
0.8
The “3” detector
18
https://cs.stanford.edu/people/karpathy/convnetjs/demo/classify2d.html
Demo: 2D Classification with MLPs
19
Convolutional Neural Networks (CNNs)
Demo: Large scale scene classification with CNNs
http://people.csail.mit.edu/torralba/research/drawCNN/drawNet.html
21
Recurrent Neural Networks (RNNs)… Lorem Ipsum is simply dummy text of the printing and typesetting industry ...
… Lorem Ipsum es simplemente un texto de relleno de la industria de impresión y tipografía ...
Recurrent Neural Networks
Long Short Term Memory (LSTM)
Vanishing Gradients!
23
Demo: Speech & Text translation with LSTM RNNs
https://distill.pub/2016/augmented-rnns/
© DeepCure, Inc. All Rights Reserved. 24
Evaluation Mechanisms - Data Splits
25
Evaluation Mechanisms - Metrics
Classification Regression
● Mean Squared Error
● Explained Variance
● R2 Score
● Classification accuracy
● Cross entropy
● ROC AUC
● PRC AUC
CE
© DeepCure, Inc. All Rights Reserved.
Train a DNN to learn the error distribution of the experiment
https://academic.oup.com/bioinformatics/article/33/14/i225/3953958
28
Deep Learning Software - Tools
https://en.wikipedia.org/wiki/Comparison_of_deep_learning_software
Frameworks
Web extensions
Mobile
© DeepCure, Inc. All Rights Reserved.
29
Deep Learning Software - Common Features
● Automatic Differentiation
● Parallelism (support for OpenMP, OpenCL, CUDA)
● Distributed Execution (CPU/GPU, cloud)
● Agile Development
● Fast Execution (compilation, optimizations)
● Portability, Export/Import models and parameters
© DeepCure, Inc. All Rights Reserved.
30Image Credits: Tensorflow.org, PyTorch.org
Deep Learning Software - DataFlow Programming
Rapid hands-on tutorial
CS 410: Intro to Software Engineering
DEEPCURE
https://github.com/thrakar9/dl_bio_workshop