october 18, 2017 and machine learning - internet2...2017/10/18 · if you’re new to tensorflow,...
TRANSCRIPT
Intro to TensorFlow and Machine LearningOctober 18, 2017
Edward DoanGoogle Cloud Customer Engineering@edwardd
@edwarddgoo.gl/stbR1K
Confidential & Proprietary
Deep Learning
What is TensorFlow
Under the hood
Resources
Agenda
@edwarddgoo.gl/stbR1K
@edwarddgoo.gl/stbR1K
Deep Learning
@edwarddgoo.gl/stbR1K
Input
Out
Deep Learning
@edwarddgoo.gl/stbR1K
Input
Out
Results with Deep Learning
@edwarddgoo.gl/stbR1K
Growing Use of Deep Learning at Google# of directories containing model description files
Time
Uniq
ue P
roje
ct D
irect
orie
s
@edwarddgoo.gl/stbR1K
Important Property of Neural Networks
Results get better with
More data +
Bigger models +
More computation
(Better algorithms, new insights, and improved techniques always help, too!)
@edwarddgoo.gl/stbR1K
What is TensorFlow?
@edwarddgoo.gl/stbR1K
Like so many good things, it started as a research paper...
@edwarddgoo.gl/stbR1K
repositoryfor “machine learning”
category on GitHub
#1
@edwarddgoo.gl/stbR1K
● TensorFlow is a machine learning library enabling researchers and developers to build the next generation of intelligent applications.
● Provides distributed, parallel machine learning based on general-purpose dataflow graphs
● Targets heterogeneous devices:○ single PC with CPU or GPU(s)○ mobile device○ clusters of 100s or 1000s of CPUs and GPUs
What is TensorFlow?
@edwarddgoo.gl/stbR1K
Flexible● General computational infrastructure
○ Works well for Deep Learning
○ Deep Learning is a set of libraries on top of the core
○ Also useful for other machine learning algorithms, maybe even more
traditional high performance computing (HPC) work
○ Abstracts away the underlying devices
A “TPU pod” built with 64 second-generation TPUs delivers up to 11.5 petaflops of machine
learning acceleration.
@edwarddgoo.gl/stbR1K
@edwarddgoo.gl/stbR1K
TensorFlow powered Cucumber Sorter
From: http://workpiles.com/2016/02/tensorflow-cnn-cucumber/
@edwarddgoo.gl/stbR1K
Under the hood
@edwarddgoo.gl/stbR1K
TensorFlow
CPU GPU Android iOS ...
C++ front end Python front end ...
Kernels
Bindings + Compound Ops
add mul print reshape ...
Core TensorFlow Execution System
Ops
@edwarddgoo.gl/stbR1K
Computation is a dataflow graph
MatMul
Add Relu
biases
weights
examples
labels
Xent
Graph of Nodes, also called Operations or ops.
@edwarddgoo.gl/stbR1K
Dataflow graphwith tensors
MatMul
Add Relu
biases
weights
examples
labels
Xent
Edges are N-dimensional arrays: Tensors
@edwarddgoo.gl/stbR1K
Dataflow graphwith state
Add Mul
biases
...
learning rate
−=...
'Biases' is a variable −= updates biasesSome ops compute gradients
@edwarddgoo.gl/stbR1K
Parallelism● Parallel op implementations
MatMul
...
...
@edwarddgoo.gl/stbR1K
Task Parallelism in the Dataflow Graph
MatMul
...
...
MatMul
...
...
...
@edwarddgoo.gl/stbR1K
Data parallelism by replicating the graph
MatMul
Input
MatMulParam
...
...MatMul
MatMul
...
...
@edwarddgoo.gl/stbR1K
Model parallelism by splitting one op into many
MatMul
Matrix
MatrixMatMul
Split Concat
@edwarddgoo.gl/stbR1K
MatMul
Add Relu
biases
weights
input
input = ...biases = tf.get_variable(’biases’, ...)weights = tf.get_variable(’weights’, ...)out = tf.matmul(input, weights)out = tf.add(out, biases)out = tf.nn.relu(out)
@edwarddgoo.gl/stbR1K
MatMul
Add Relu
biases
weights
input
input = ...output = tf.layers.fully_connected(input, ...)
@edwarddgoo.gl/stbR1K
TensorFlow contains complete algorithmsLinear{Classifier,Regressor}
DNN{Classifier,Regressor}
DNNLinearCombined{Classifier,Regressor}
SVM
KMeansClustering
GMM
...
@edwarddgoo.gl/stbR1K
Simple machine learningclassifier = learn.LinearClassifier(feature_columns=feature_columns,
n_classes=10)
classifier.fit(data, labels, batch_size=100, steps=1000)
classifier.evaluate(eval_data, eval_labels)
Tooling provided for distributed training and evaluation, graphical debugging, and export to production server (tensorflow/serving).
@edwarddgoo.gl/stbR1K
Visualizing learning
@edwarddgoo.gl/stbR1K
Let’s play!
Visit https://jhub.edwarddoan.com (sorry for the SSL cert error )
Click this button ->
Log in as [email protected] / gogoogle (where x is unique to you)
Start “My Server”
If you’re new to Tensorflow, play with 2_getting_started.ipynb
If you’re kinda familiar with Tensorflow, play with 3_mnist_from_scratch.ipynb
Click the Play button to go through the markdown and code cell-by-cell
@edwarddgoo.gl/stbR1K
Resources
Website: https://www.tensorflow.org
Code: https://github.com/tensorflow/tensorflow
WhitePaper: https://www.tensorflow.org/whitepaper2015.pdf
@edwarddgoo.gl/stbR1K
? : !@edwardd cloud.google.com