introduction to tensorflow, by machine learning at berkeley
TRANSCRIPT
![Page 1: Introduction to TensorFlow, by Machine Learning at Berkeley](https://reader031.vdocuments.net/reader031/viewer/2022020119/58e502d61a28abdf698b6785/html5/thumbnails/1.jpg)
Introduction to TensorFlow
![Page 2: Introduction to TensorFlow, by Machine Learning at Berkeley](https://reader031.vdocuments.net/reader031/viewer/2022020119/58e502d61a28abdf698b6785/html5/thumbnails/2.jpg)
AgendaOur goals for tonight
Neural Network Review
What is TensorFlow?
Building Neural Nets
TensorBoard Visualization
1
2
3
4
![Page 3: Introduction to TensorFlow, by Machine Learning at Berkeley](https://reader031.vdocuments.net/reader031/viewer/2022020119/58e502d61a28abdf698b6785/html5/thumbnails/3.jpg)
Neural Networks Review
● Layers that combine previous features to form new features● Each layer applies linear transformation, “weighing” previous features● Nonlinear activation function● Predicts through a feedforward propagation● Learns through gradient descent and backpropagation
FEEDFORWARD
BACKPROPAGATE
![Page 4: Introduction to TensorFlow, by Machine Learning at Berkeley](https://reader031.vdocuments.net/reader031/viewer/2022020119/58e502d61a28abdf698b6785/html5/thumbnails/4.jpg)
Neural Network Libraries
Pros: - Strength in CNNs - Image processing - Python interface
Cons: - Inflexible - C++
Pros: - Widely used - High performance - Python
Cons: - Somewhat bulky - Can get low-level
Pros: - Slimmer - High performance - Modular
Cons: - Academic use - Lua
Pros: - Gaining support - TensorBoard - Python
Cons: - Improving performance - Low content
![Page 5: Introduction to TensorFlow, by Machine Learning at Berkeley](https://reader031.vdocuments.net/reader031/viewer/2022020119/58e502d61a28abdf698b6785/html5/thumbnails/5.jpg)
Why Learn TensorFlow?
➢ Backed by Google○ Constant development and frequent updates○ DeepMind moving from Torch to TensorFlow
➢ Growing Community○ Amount of example code and tutorials growing○ Most commonly mentioned ML library on Stack Overflow
➢ Long term support○ Recent TensorFlow 1.0 update, all code will be compatible
with 1.x updates➢ Performance is not very good, but getting better.
○ About an order of magnitude slower than Theano
![Page 6: Introduction to TensorFlow, by Machine Learning at Berkeley](https://reader031.vdocuments.net/reader031/viewer/2022020119/58e502d61a28abdf698b6785/html5/thumbnails/6.jpg)
Tensors (side note)
➢ For Programmers: Tensors generalize multidimensional arrays.➢ For Mathematicians: Tensors generalize scalars, vectors,
matrices and linear operators!➢ TensorFlow describe data as tensors, and pass them through its
computation graph.➢ Tensors flow through the network
![Page 7: Introduction to TensorFlow, by Machine Learning at Berkeley](https://reader031.vdocuments.net/reader031/viewer/2022020119/58e502d61a28abdf698b6785/html5/thumbnails/7.jpg)
TensorFlow Basics
Computation Graph
Construction Execution
![Page 8: Introduction to TensorFlow, by Machine Learning at Berkeley](https://reader031.vdocuments.net/reader031/viewer/2022020119/58e502d61a28abdf698b6785/html5/thumbnails/8.jpg)
TensorFlow Basics *
Variables➢ Stores parameters in graph➢ Can be trainable (optimized
during backprop) or untrainable➢ Variety of initializers (e.g.
constant, normal, etc)
Operations➢ Takes in variable and/or
outputs from other operations➢ Can be fed into other ops and
linked in the graph
tf.constant(5.0)
tf.constant(3.0)
tf.random_normal(mean=1, stddev=2)
tf.add()
tf.mul()
![Page 9: Introduction to TensorFlow, by Machine Learning at Berkeley](https://reader031.vdocuments.net/reader031/viewer/2022020119/58e502d61a28abdf698b6785/html5/thumbnails/9.jpg)
TensorFlow Basics *
Sessions➢ Handles post-construction interactions with the graph➢ Call the run method to evaluate tensors
3.0
5.0
1.68
8.0
13.44
tf.constant(5.0)
tf.constant(3.0)
tf.random_normal(mean=1, stddev=2)
tf.add()
tf.mul()
sess = tf.Session()sess.run(tf.global_variables_initializer())sess.run(mult_op)
mult_op
![Page 10: Introduction to TensorFlow, by Machine Learning at Berkeley](https://reader031.vdocuments.net/reader031/viewer/2022020119/58e502d61a28abdf698b6785/html5/thumbnails/10.jpg)
TensorFlow Basics *
Optimizers➢ Subclasses of tf.train.Optimizer➢ Main functions: compute_gradients, apply_gradients, and
minimize
def minimize(self, loss_fn):self.compute_gradients(loss_fn
)self.apply_gradients(loss_fn)
Backpropagation on ops
Update trainable variables
Some loss functions are built into TensorFlow➢ For example, tf.losses.mean_squared_error➢ You can also define your own loss functions by combining ops
![Page 11: Introduction to TensorFlow, by Machine Learning at Berkeley](https://reader031.vdocuments.net/reader031/viewer/2022020119/58e502d61a28abdf698b6785/html5/thumbnails/11.jpg)
TensorFlow Basics
Placeholders➢ A placeholder variable that can be filled in during execution➢ On evaluation, specify a dictionary with placeholder key-value pairs
P2
P1
tf.random_normal(mean=1, stddev=2)
tf.add()
tf.mul()
1.0
2.0
0.94
3.0
2.82
sess.run(mul, feed_dict={P1: 1.0, P2:2.0})
![Page 12: Introduction to TensorFlow, by Machine Learning at Berkeley](https://reader031.vdocuments.net/reader031/viewer/2022020119/58e502d61a28abdf698b6785/html5/thumbnails/12.jpg)
TensorBoard
Graph Visualization➢ See visual representation of the graph➢ Check and debug construction
Scoping➢ Used to create abstractions➢ Without scoping, graph can become a
convoluted mess➢ Created by using a with statement➢ Scope gets name prepended to variable
and operation names
![Page 13: Introduction to TensorFlow, by Machine Learning at Berkeley](https://reader031.vdocuments.net/reader031/viewer/2022020119/58e502d61a28abdf698b6785/html5/thumbnails/13.jpg)
TensorBoard
Visualizing Learning➢ See tensor values during training➢ Check and debug execution
![Page 14: Introduction to TensorFlow, by Machine Learning at Berkeley](https://reader031.vdocuments.net/reader031/viewer/2022020119/58e502d61a28abdf698b6785/html5/thumbnails/14.jpg)
Review
➢ TensorFlow is one of several ML libraries, each with pros and cons➢ Expected long term support for TensorFlow➢ Two stages: construction and execution➢ Tensors are passed through chained operations➢ Operations are evaluated at the execution stage with a session object➢ Use optimizers to find and apply gradients for the training step➢ TensorBoard used for graph visualization and visualizing learning
![Page 15: Introduction to TensorFlow, by Machine Learning at Berkeley](https://reader031.vdocuments.net/reader031/viewer/2022020119/58e502d61a28abdf698b6785/html5/thumbnails/15.jpg)
Thank You for Coming!Please fill out this feedback form: https://mlab.typeform.com/to/t51Y09
Like our page on Facebook: www.facebook.com/berkeleymlEmail us: [email protected] our website: ml.berkeley.eduCheck out our blog: ml.berkeley.edu/blog