università degli studi di pavia deep learning and tensorflow · deep learning and tensorflow...
TRANSCRIPT
[1]Deep Learning and TensorFlow – Episode 4
Deep Learningand TensorFlowEpisode 4TensorFlow Basics
Part 1
Università degli Studi di Pavia
[2]Deep Learning and TensorFlow – Episode 4
Summary
[3]Deep Learning and TensorFlow – Episode 4
Main References
[4]Deep Learning and TensorFlow – Episode 4
What is TensorFlow?
[5]Deep Learning and TensorFlow – Episode 4
What is TensorFlow?
[6]Deep Learning and TensorFlow – Episode 4
Why TensorFlow? (alternatives?)
[7]Deep Learning and TensorFlow – Episode 4
Why TensorFlow?
[8]Deep Learning and TensorFlow – Episode 4
Example application: classify skin cancer
[9]Deep Learning and TensorFlow – Episode 4
Example application: Neural Style Transfer
[10]Deep Learning and TensorFlow – Episode 4
The structure of a TensorFlow program
𝑦 = (𝑎 ∗ 𝑏)𝑚𝑢𝑙
+ (𝑎 + 𝑏)𝑎𝑑𝑑
𝑎𝑑𝑑
[11]Deep Learning and TensorFlow – Episode 4
Graphs: an intuitive view
[12]Deep Learning and TensorFlow – Episode 4
Graphs: an intuitive view
[13]Deep Learning and TensorFlow – Episode 4
Graphs: an intuitive view
[14]Deep Learning and TensorFlow – Episode 4
Graphs: an intuitive view
[15]Deep Learning and TensorFlow – Episode 4
Graphs: an intuitive view
[16]Deep Learning and TensorFlow – Episode 4
Graphs: an intuitive view
[17]Deep Learning and TensorFlow – Episode 4
Graphs: an intuitive way (with matrices)
[18]Deep Learning and TensorFlow – Episode 4
Graphs: an intuitive way (with matrices)
[19]Deep Learning and TensorFlow – Episode 4
Graphs: an intuitive way (with matrices)
1 23 4
5 67 8
[20]Deep Learning and TensorFlow – Episode 4
Graphs: an intuitive way (with matrices)
1 23 4
5 67 8
5 67 8
1 23 4
[21]Deep Learning and TensorFlow – Episode 4
Graphs: an intuitive way (with matrices)
5 1221 32
6 810 12
[22]Deep Learning and TensorFlow – Episode 4
Graphs: an intuitive way (with matrices)
11 2031 44
[23]Deep Learning and TensorFlow – Episode 4
What is Tensorflow?
[24]Deep Learning and TensorFlow – Episode 4
Sessions: computing flow graphs
[25]Deep Learning and TensorFlow – Episode 4
The first TensorFlow program
import tensorflow as tf
a = 3
b = 7
y = tf.multiply(a, b) + tf.add(a, b)
[26]Deep Learning and TensorFlow – Episode 4
The first TensorFlow program
import tensorflow as tf
a = 3
b = 7
y = tf.multiply(a, b) + tf.add(a, b)
[27]Deep Learning and TensorFlow – Episode 4
The first TensorFlow program
import tensorflow as tf
a = 3
b = 7
y = tf.multiply(a, b) + tf.add(a, b)
print(y)
[28]Deep Learning and TensorFlow – Episode 4
The first TensorFlow program
import tensorflow as tf
a = 3
b = 7
y = tf.multiply(a, b) + tf.add(a, b)
print(y)
>> Tensor("Add:0", shape=(), dtype=int32)
[29]Deep Learning and TensorFlow – Episode 4
The first TensorFlow program
import tensorflow as tf
a = 3
b = 7
y = tf.multiply(a, b) + tf.add(a, b)
print(y)
>> Tensor("Add:0", shape=(), dtype=int32)
[30]Deep Learning and TensorFlow – Episode 4
Sessions
[31]Deep Learning and TensorFlow – Episode 4
Sessions
sess
y
[32]Deep Learning and TensorFlow – Episode 4
Sessions
sess
y
import tensorflow as tfy = tf.add(3, 7)sess = tf.Session()print(sess.run(y))sess.close()
[33]Deep Learning and TensorFlow – Episode 4
Sessions
sess
y
import tensorflow as tfy = tf.add(3, 7)sess = tf.Session()print(sess.run(y))sess.close()
with
[34]Deep Learning and TensorFlow – Episode 4
The first TensorFlow program
import tensorflow as tf
a = 3
b = 7
y = tf.multiply(a, b) + tf.add(a, b)
[35]Deep Learning and TensorFlow – Episode 4
The first TensorFlow program
import tensorflow as tf
a = 3
b = 7
y = tf.multiply(a, b) + tf.add(a, b)
with tf.Session() as sess:
print(sess.run(y))
[36]Deep Learning and TensorFlow – Episode 4
The first TensorFlow program
import tensorflow as tf
a = 3
b = 7
y = tf.multiply(a, b) + tf.add(a, b)
with tf.Session() as sess:
print(sess.run(y))
>> 31
[37]Deep Learning and TensorFlow – Episode 4
A graph in TensorFlow
import tensorflow as tf
a = 3
b = 7
y = tf.multiply(a, b) + tf.add(a, b)
[38]Deep Learning and TensorFlow – Episode 4
A graph in TensorFlow
import tensorflow as tf
a = 3
b = 7
y = tf.multiply(a, b) + tf.add(a, b)
with tf.Session() as sess:
print(sess.run(y))
[39]Deep Learning and TensorFlow – Episode 4
A graph in TensorFlow
import tensorflow as tf
a = 3
b = 7
y = tf.multiply(a, b) + tf.add(a, b)
with tf.Session() as sess:
print(sess.run(y))
>> 31
[40]Deep Learning and TensorFlow – Episode 4
A graph in TensorFlow (with matrices)
import tensorflow as tf
a = tf.constant([1,2,3,4], shape=(2,2))
b = tf.constant([5,6,7,8], shape=(2,2))
y = tf.multiply(a, b) + tf.add(a, b)
[41]Deep Learning and TensorFlow – Episode 4
A graph in TensorFlow (with matrices)
import tensorflow as tf
a = tf.constant([1,2,3,4], shape=(2,2))
b = tf.constant([5,6,7,8], shape=(2,2))
y = tf.multiply(a, b) + tf.add(a, b)
with tf.Session() as sess:
print(sess.run(y))
[42]Deep Learning and TensorFlow – Episode 4
A graph in TensorFlow (with matrices)
import tensorflow as tf
a = tf.constant([1,2,3,4], shape=(2,2))
b = tf.constant([5,6,7,8], shape=(2,2))
y = tf.multiply(a, b) + tf.add(a, b)
with tf.Session() as sess:
print(sess.run(y))
>> [[11 20]
[31 44]]
[43]Deep Learning and TensorFlow – Episode 4
A graph in TensorFlow (with matrices)
import tensorflow as tf
a = tf.constant([1,2,3,4], shape=(2,2))
b = tf.constant([5,6,7,8], shape=(2,2))
y = tf.multiply(a, b) + tf.add(a, b)
with tf.Session() as sess:
print(sess.run(y))
>> [[11 20]
[31 44]]
tf.constant
shape
[44]Deep Learning and TensorFlow – Episode 4
Why graphs?
[45]Deep Learning and TensorFlow – Episode 4
Why graphs?
[46]Deep Learning and TensorFlow – Episode 4
Why graphs?
[47]Deep Learning and TensorFlow – Episode 4
Why graphs?
[48]Deep Learning and TensorFlow – Episode 4
Why graphs?
[49]Deep Learning and TensorFlow – Episode 4
Why graphs?
[50]Deep Learning and TensorFlow – Episode 4
Visualization with TensorBoard
import tensorflow as tf
a = tf.constant(3, name='a') # equivalent to a = 3
b = tf.constant(7, name='b')
y = tf.multiply(a, b, name='mul_op') + tf.add(a, b, name='add_op')
with tf.Session() as sess:
print(sess.run(y))
[51]Deep Learning and TensorFlow – Episode 4
Visualization with TensorBoard
import tensorflow as tf
a = tf.constant(3, name='a') # equivalent to a = 3
b = tf.constant(7, name='b')
y = tf.multiply(a, b, name='mul_op') + tf.add(a, b, name='add_op')
with tf.Session() as sess:
print(sess.run(y))
[52]Deep Learning and TensorFlow – Episode 4
Visualization with TensorBoard
import tensorflow as tf
a = tf.constant(3, name='a') # equivalent to a = 3
b = tf.constant(7, name='b')
y = tf.multiply(a, b, name='mul_op') + tf.add(a, b, name='add_op')
with tf.Session() as sess:
print(sess.run(y))
tf.add
[53]Deep Learning and TensorFlow – Episode 4
Visualization with TensorBoard
import tensorflow as tf
a = tf.constant(3, name='a') # equivalent to a = 3
b = tf.constant(7, name='b')
with tf.Session() as sess:
print(sess.run(y))
tf.add
y = tf.add(tf.multiply(a, b, name='mul_op'), tf.add(a, b, name='add_op'))
[54]Deep Learning and TensorFlow – Episode 4
Visualization with TensorBoard
import tensorflow as tf
a = tf.constant(3, name='a') # equivalent to a = 3
b = tf.constant(7, name='b')
writer = tf.summary.FileWriter('./graphs', tf.get_default_graph())
with tf.Session() as sess:
print(sess.run(y))
writer.close() # close the writer when you’re done using it
y = tf.add(tf.multiply(a, b, name='mul_op'), tf.add(a, b, name='add_op'))
[55]Deep Learning and TensorFlow – Episode 4
Visualization with TensorBoard
import tensorflow as tf
a = tf.constant(3, name='a') # equivalent to a = 3
b = tf.constant(7, name='b')
writer = tf.summary.FileWriter('./graphs', tf.get_default_graph())
with tf.Session() as sess:
print(sess.run(y))
writer.close() # close the writer when you’re done using it
y = tf.add(tf.multiply(a, b, name='mul_op'), tf.add(a, b, name='add_op'))
[56]Deep Learning and TensorFlow – Episode 4
Visualization with TensorBoard
import tensorflow as tf
a = tf.constant(3, name='a') # equivalent to a = 3
b = tf.constant(7, name='b')
with tf.Session() as sess:writer = tf.summary.FileWriter('./graphs', sess.graph)print(sess.run(y))
y = tf.add(tf.multiply(a, b, name='mul_op'), tf.add(a, b, name='add_op'))
[57]Deep Learning and TensorFlow – Episode 4
Run it!
$ python3 [yourprogram].py$ tensorboard --logdir="./graphs" --port 6006
[58]Deep Learning and TensorFlow – Episode 4
Run it!
$ python3 [yourprogram].py$ tensorboard --logdir="./graphs" --port 6006
http://localhost:6006/
[59]Deep Learning and TensorFlow – Episode 4
The same graph in Tensorboard
a = tf.constant(3, name='a')b = tf.constant(7, name='b')y = tf.multiply(a, b, name='mul_op') +
tf.add(a, b, name='add_op')
[60]Deep Learning and TensorFlow – Episode 4
The same graph in Tensorboard
name
a = tf.constant(3, name='a')b = tf.constant(7, name='b')y = tf.multiply(a, b, name='mul_op') +
tf.add(a, b, name='add_op')
[61]Deep Learning and TensorFlow – Episode 4
x = 2y = 3
add_op = tf.add(x, y)mul_op = tf.multiply(x, y)
useless = tf.multiply(x, add_op,name='useless')
pow_op = tf.pow(add_op, mul_op)
with tf.Session() as sess:z = sess.run(pow_op)
Subgraphs
[62]Deep Learning and TensorFlow – Episode 4
x = 2y = 3
add_op = tf.add(x, y)mul_op = tf.multiply(x, y)
useless = tf.multiply(x, add_op,name='useless')
pow_op = tf.pow(add_op, mul_op)
with tf.Session() as sess:z = sess.run(pow_op)
Subgraphs
[63]Deep Learning and TensorFlow – Episode 4
x = 2y = 3
add_op = tf.add(x, y)mul_op = tf.multiply(x, y)
useless = tf.multiply(x, add_op,name='useless')
pow_op = tf.pow(add_op, mul_op)
with tf.Session() as sess:z = sess.run(pow_op)
Subgraphs
pow_oppow_op uselessuseless
[64]Deep Learning and TensorFlow – Episode 4
Subgraphs
x = 2y = 3
add_op = tf.add(x, y)mul_op = tf.multiply(x, y)
useless = tf.multiply(x, add_op,name='useless')
pow_op = tf.pow(add_op, mul_op)
with tf.Session() as sess:z, not_useless =
sess.run([useless, pow_op])
[65]Deep Learning and TensorFlow – Episode 4
Subgraphs
x = 2y = 3
add_op = tf.add(x, y)mul_op = tf.multiply(x, y)
useless = tf.multiply(x, add_op,name='useless')
pow_op = tf.pow(add_op, mul_op)
with tf.Session() as sess:z, not_useless =
sess.run([useless, pow_op])
sess.run
[66]Deep Learning and TensorFlow – Episode 4
TensorFlow and GPUs
[67]Deep Learning and TensorFlow – Episode 4
Distributed computation
[68]Deep Learning and TensorFlow – Episode 4
Distributed computation
# Creates a graph.
with tf.device('/gpu:2'):
a = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], name='a')
b = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], name='b')
y = tf.multiply(a, b)
# Creates a session with log_device_placement set to True.
sess = tf.Session(config=tf.ConfigProto(log_device_placement=True))
# Runs the op.
print(sess.run(y))
[69]Deep Learning and TensorFlow – Episode 4
Protocol buffers
print(tf.get_default_graph().as_graph_def())
[70]Deep Learning and TensorFlow – Episode 4
Protocol buffers
print(tf.get_default_graph().as_graph_def())
node {name: "mul_op/a"op: "Const"attr {key: "dtype"value { type: DT_INT32 }
}attr {key: "value"value { tensor
{ dtype: DT_INT32tensor_shape {}int_val: 7 }
}}
}
[71]Deep Learning and TensorFlow – Episode 4
Protocol buffers
print(tf.get_default_graph().as_graph_def())
node {name: "mul_op"op: "Mul"input: "mul_op/a"input: "mull_op/b"attr {key: "T"value {type: DT_INT32
}}
}
[72]Deep Learning and TensorFlow – Episode 4
Protocol buffers
print(tf.get_default_graph().as_graph_def())
node {name: "add"op: "Add"input: "mul_op"input: "add_op"attr {key: "T"value {type: DT_INT32
}}
}
[73]Deep Learning and TensorFlow – Episode 4
Tensors, constants and ops
[74]Deep Learning and TensorFlow – Episode 4
What is a tensor?
[75]Deep Learning and TensorFlow – Episode 4
Tensors as objects tf.Tensor
float32 int32 string
[76]Deep Learning and TensorFlow – Episode 4
Tensors as objects tf.Tensor
float32 int32 string
[77]Deep Learning and TensorFlow – Episode 4
Tensors as objects tf.Tensor
float32 int32 string
r = tf.rank(cat_image) # r is a tensors = tf.shape(cat_image) # s is a tensor# after evaluation r will be 3# s will be (28, 28, 3)
[78]Deep Learning and TensorFlow – Episode 4
Tensors as objects tf.Tensor
float32 int32 string
r = tf.rank(cat_image) # r is a tensors = tf.shape(cat_image) # s is a tensor# after evaluation r will be 3# s will be (28, 28, 3)
r s
[79]Deep Learning and TensorFlow – Episode 4
Tensors Containers
[80]Deep Learning and TensorFlow – Episode 4
Tensors Containers
[81]Deep Learning and TensorFlow – Episode 4
Tensors Containers
[82]Deep Learning and TensorFlow – Episode 4
Tensorflow Operations
[83]Deep Learning and TensorFlow – Episode 4
Graph representations of tensors and containers
a = tf.constant(3, name='a')b = tf.constant(7, name='b')y = tf.multiply(a, b, name='mul_op') +
tf.add(a, b, name='add_op')
[84]Deep Learning and TensorFlow – Episode 4
Graph representations of tensors and containers
a = tf.constant(3, name='a')b = tf.constant(7, name='b')y = tf.multiply(a, b, name='mul_op') +
tf.add(a, b, name='add_op')
[85]Deep Learning and TensorFlow – Episode 4
Constants tf.constant(value,
dtype=None,shape=None,name='Const',verify_shape=False)
[86]Deep Learning and TensorFlow – Episode 4
Constants tf.constant(value,
dtype=None,shape=None,name='Const',verify_shape=False)
# constant of 1d tensor (vector)a = tf.constant([2, 2], name="vector")
# constant of 2x2 tensor (matrix)b = tf.constant([[0, 1], [2, 3]], name="matrix")
x = tf.multiply(a, b, name='mul_op')
>> x = [[0 2][4 6]]
[87]Deep Learning and TensorFlow – Episode 4
Constants tf.constant(value,
dtype=None,shape=None,name='Const',verify_shape=False)
# constant of 1d tensor (vector)a = tf.constant([2, 2], name="vector")
# constant of 2x2 tensor (matrix)b = tf.constant([[0, 1], [2, 3]], name="matrix")
x = tf.multiply(a, b, name='mul_op')
>> x = [[0 2][4 6]]
[88]Deep Learning and TensorFlow – Episode 4
Constants filled with specific valuestf.zeros(shape, dtype=tf.float32, name=None)
tf.zeros_like(input_tensor, dtype=None, name=None, optimize=True)
tf.ones(shape, dtype=tf.float32, name=None)
tf.ones_like (input_tensor, dtype=None, name=None, optimize=True)
tf.fill(shape, value, name=None)
[89]Deep Learning and TensorFlow – Episode 4
Constants filled with specific valuestf.zeros(shape, dtype=tf.float32, name=None)
tf.zeros_like(input_tensor, dtype=None, name=None, optimize=True)
tf.ones(shape, dtype=tf.float32, name=None)
tf.ones_like (input_tensor, dtype=None, name=None, optimize=True)
tf.fill(shape, value, name=None)
tf.zeros([2, 3], tf.int32)
tf.ones([2, 3], tf.int32)
0 0 00 0 0
1 1 11 1 1
[90]Deep Learning and TensorFlow – Episode 4
Constants filled with specific valuestf.zeros(shape, dtype=tf.float32, name=None)
tf.zeros_like(input_tensor, dtype=None, name=None, optimize=True)
tf.ones(shape, dtype=tf.float32, name=None)
tf.ones_like (input_tensor, dtype=None, name=None, optimize=True)
tf.fill(shape, value, name=None)
tf.zeros([2, 3], tf.int32)
tf.ones([2, 3], tf.int32)
# input_tensor is [[0, 1], [2, 3], [4, 5]]
tf.zeros_like(input_tensor)
tf.ones_like(input_tensor)
0 0 00 0 0
1 11 11 1
1 1 11 1 1
0 00 00 0
[91]Deep Learning and TensorFlow – Episode 4
Constants filled with specific valuestf.zeros(shape, dtype=tf.float32, name=None)
tf.zeros_like(input_tensor, dtype=None, name=None, optimize=True)
tf.ones(shape, dtype=tf.float32, name=None)
tf.ones_like (input_tensor, dtype=None, name=None, optimize=True)
tf.fill(shape, value, name=None)
tf.zeros([2, 3], tf.int32)
tf.ones([2, 3], tf.int32)
# input_tensor is [[0, 1], [2, 3], [4, 5]]
tf.zeros_like(input_tensor)
tf.ones_like(input_tensor)
tf.fill([2, 3], 8)
0 0 00 0 0
1 11 11 1
8 88 88 8
1 1 11 1 1
0 00 00 0
[92]Deep Learning and TensorFlow – Episode 4
Constants as sequences
tf.lin_space(start, stop, num, name=None)
tf.lin_space(10.0, 13.0, 4) ==> [10. 11. 12. 13.]
tf.range(start, limit=None, delta=1, dtype=None, name='range')
tf.range(3, 18, 3) ==> [3 6 9 12 15]
tf.range(5) ==> [0 1 2 3 4]
[93]Deep Learning and TensorFlow – Episode 4
Constants as sequences
tf.lin_space(start, stop, num, name=None)
tf.lin_space(10.0, 13.0, 4) ==> [10. 11. 12. 13.]
tf.range(start, limit=None, delta=1, dtype=None, name='range')
tf.range(3, 18, 3) ==> [3 6 9 12 15]
tf.range(5) ==> [0 1 2 3 4]
tf.range
for _ in tf.range(4):# TypeError
[94]Deep Learning and TensorFlow – Episode 4
Randomly generated Constants
tf.random_normal
tf.truncated_normal
tf.random_uniform
tf.random_shuffle
tf.random_crop
tf.multinomial
tf.random_gamma
[95]Deep Learning and TensorFlow – Episode 4
Randomly generated Constants
tf.random_normal
tf.truncated_normal
tf.random_uniform
tf.random_shuffle
tf.random_crop
tf.multinomial
tf.random_gamma
tf.set_random_seed(seed)
[96]Deep Learning and TensorFlow – Episode 4
Data types
[97]Deep Learning and TensorFlow – Episode 4
Data types
t_0 = 19 # scalars are treated like 0-d tensorstf.zeros_like(t_0) # ==> constant 0-d tensor with value 0tf.ones_like(t_0) # ==> constant 0-d tensor with value 1
t_1 = ["apple", "peach", "grape"] # 1-d arrays are treated like 1-d tensorstf.zeros_like(t_1) # ==> ['' '' '']tf.ones_like(t_1) # ==> TypeError
t_2 = [[True, False, False],[False, False, True],[False, True, False]] # 2-d arrays are treated like 2-d tensors
tf.zeros_like(t_2) # ==> 3x3 tensor, all elements are Falsetf.ones_like(t_2) # ==> 3x3 tensor, all elements are True
[98]Deep Learning and TensorFlow – Episode 4
Data types
[99]Deep Learning and TensorFlow – Episode 4
Variables
[100]Deep Learning and TensorFlow – Episode 4
Shortcomings of constants
[101]Deep Learning and TensorFlow – Episode 4
Shortcomings of constants
attr {key: "value"value {
tensor {dtype: DT_FLOAT
tensor_shape {dim { size: 2 }
}tensor_content:
"\000\000\200?\000\000\000@"}
}}
my_const = tf.constant([1.0, 2.0], name="my_const")with tf.Session() as sess:
print(sess.graph.as_graph_def())
[102]Deep Learning and TensorFlow – Episode 4
Shortcomings of constants
attr {key: "value"value {
tensor {dtype: DT_FLOAT
tensor_shape {dim { size: 2 }
}tensor_content:
"\000\000\200?\000\000\000@"}
}}
my_const = tf.constant([1.0, 2.0], name="my_const")with tf.Session() as sess:
print(sess.graph.as_graph_def())
[103]Deep Learning and TensorFlow – Episode 4
Variables
tf.Variable
[104]Deep Learning and TensorFlow – Episode 4
Variables
tf.Variable
# create variables with tf.Variable
s = tf.Variable(2, name="scalar") m = tf.Variable([[0, 1], [2, 3]], name="matrix") W = tf.Variable(tf.zeros([784,10]))
[105]Deep Learning and TensorFlow – Episode 4
Variables
tf.Variable
# create variables with tf.Variable
s = tf.Variable(2, name="scalar") m = tf.Variable([[0, 1], [2, 3]], name="matrix") W = tf.Variable(tf.zeros([784,10]))
# create variables with tf.get_variable
s = tf.get_variable("scalar", initializer=tf.constant(2)) m = tf.get_variable("matrix", initializer=tf.constant([[0, 1], [2, 3]]))W = tf.get_variable("big_matrix", shape=(784, 10),
initializer=tf.zeros_initializer())
[106]Deep Learning and TensorFlow – Episode 4
Variables
tf.Variable
# create variables with tf.Variable
s = tf.Variable(2, name="scalar") m = tf.Variable([[0, 1], [2, 3]], name="matrix") W = tf.Variable(tf.zeros([784,10]))
# create variables with tf.get_variable
s = tf.get_variable("scalar", initializer=tf.constant(2)) m = tf.get_variable("matrix", initializer=tf.constant([[0, 1], [2, 3]]))W = tf.get_variable("big_matrix", shape=(784, 10),
initializer=tf.zeros_initializer())
get_variable
[107]Deep Learning and TensorFlow – Episode 4
Variables operator tf.Variable
x = tf.Variable(…)
x.initializer # init opx.value # read op
# and more..
[108]Deep Learning and TensorFlow – Episode 4
Variables operator tf.Variable
x = tf.Variable(…)
x.initializer # init opx.value # read op
# and more..
W = tf.get_variable("big_matrix", shape=(784, 10),initializer=tf.zeros_initializer())
with tf.Session() as sess:
print(sess.run(W))
[109]Deep Learning and TensorFlow – Episode 4
Variables operator tf.Variable
x = tf.Variable(…)
x.initializer # init opx.value # read op
# and more..
W = tf.get_variable("big_matrix", shape=(784, 10),initializer=tf.zeros_initializer())
with tf.Session() as sess:
print(sess.run(W))
>> FailedPreconditionError: Attempting to use uninitialized value Variable
[110]Deep Learning and TensorFlow – Episode 4
Variables initialization
with tf.Session() as sess:sess.run(tf.global_variables_initializer())
[111]Deep Learning and TensorFlow – Episode 4
Variables initialization
with tf.Session() as sess:sess.run(tf.global_variables_initializer())
with tf.Session() as sess:sess.run(tf.variables_initializer([a, b]))
[112]Deep Learning and TensorFlow – Episode 4
Variables initialization
with tf.Session() as sess:sess.run(tf.global_variables_initializer())
with tf.Session() as sess:sess.run(tf.variables_initializer([a, b]))
W = tf.Variable(tf.zeros([784,10]))
with tf.Session() as sess:sess.run(W.initializer)
[113]Deep Learning and TensorFlow – Episode 4
# W is a random 700 x 100 variable object
W = tf.Variable(tf.truncated_normal([700, 10]))
with tf.Session() as sess:
sess.run(W.initializer)
Variables evaluation
print(W)
[114]Deep Learning and TensorFlow – Episode 4
# W is a random 700 x 100 variable object
W = tf.Variable(tf.truncated_normal([700, 10]))
with tf.Session() as sess:
sess.run(W.initializer)
>> Tensor("Variable/read:0", shape=(700, 10), dtype=float32)
Variables evaluation
print(W)
[115]Deep Learning and TensorFlow – Episode 4
# W is a random 700 x 100 variable object
W = tf.Variable(tf.truncated_normal([700, 10]))
with tf.Session() as sess:
sess.run(W.initializer)
Variables evaluation
print(W.eval())
[116]Deep Learning and TensorFlow – Episode 4
# W is a random 700 x 100 variable object
W = tf.Variable(tf.truncated_normal([700, 10]))
with tf.Session() as sess:
sess.run(W.initializer)
Variables evaluation
print(sess.run(W))print(W.eval())
[117]Deep Learning and TensorFlow – Episode 4
# W is a random 700 x 100 variable object
W = tf.Variable(tf.truncated_normal([700, 10]))
with tf.Session() as sess:
sess.run(W.initializer)
>> [[-0.76781619 -0.67020458 1.15333688 ..., -0.98434633 -1.25692499-0.90904623]
..., [ 0.19076447 -0.62968266 -1.97970271 ..., -1.48389161 0.681706431.46369624]]
Variables evaluation
print(sess.run(W))print(W.eval())
[118]Deep Learning and TensorFlow – Episode 4
Variable Assignments
W = tf.Variable(10)
W.assign(100)
with tf.Session() as sess:
sess.run(W.initializer)
print(W.eval())
[119]Deep Learning and TensorFlow – Episode 4
Variable Assignments
W = tf.Variable(10)
W.assign(100)
with tf.Session() as sess:
sess.run(W.initializer)
print(W.eval())
# >> 10
[120]Deep Learning and TensorFlow – Episode 4
Variable Assignments
W = tf.Variable(10)
W.assign(100)
with tf.Session() as sess:
sess.run(W.initializer)
print(W.eval())
# >> 10
W.assign(100)
[121]Deep Learning and TensorFlow – Episode 4
Variable Assignments
W = tf.Variable(10)
W.assign(100)
with tf.Session() as sess:
sess.run(W.initializer)
print(W.eval())
# >> 10
W = tf.Variable(10)
assign_op = W.assign(100)
with tf.Session() as sess:
sess.run(W.initializer)
sess.run(assign_op)
print(W.eval())
W.assign(100)
[122]Deep Learning and TensorFlow – Episode 4
Variable Assignments
W = tf.Variable(10)
W.assign(100)
with tf.Session() as sess:
sess.run(W.initializer)
print(W.eval())
# >> 10
W = tf.Variable(10)
assign_op = W.assign(100)
with tf.Session() as sess:
sess.run(W.initializer)
sess.run(assign_op)
print(W.eval())
# >> 100
W.assign(100)
[123]Deep Learning and TensorFlow – Episode 4
Variable Assignments
# create a variable whose original value is 2
my_var = tf.Variable(2, name="my_var")
# assign a*2 to a and call that op a_times_two
my_var_times_two = my_var.assign(2*my_var)
[124]Deep Learning and TensorFlow – Episode 4
Variable Assignments
# create a variable whose original value is 2
my_var = tf.Variable(2, name="my_var")
# assign a*2 to a and call that op a_times_two
my_var_times_two = my_var.assign(2*my_var)
2*my_var my_varmy_var_times_two
[125]Deep Learning and TensorFlow – Episode 4
Variable Assignments
# create a variable whose original value is 2
my_var = tf.Variable(2, name="my_var")
# assign a*2 to a and call that op a_times_two
my_var_times_two = my_var.assign(2*my_var)
with tf.Session() as sess:
sess.run(my_var.initializer)
sess.run(my_var_times_two) # >> the value of my_var now is 4
2*my_var my_varmy_var_times_two
[126]Deep Learning and TensorFlow – Episode 4
Variable Assignments
# create a variable whose original value is 2
my_var = tf.Variable(2, name="my_var")
# assign a*2 to a and call that op a_times_two
my_var_times_two = my_var.assign(2*my_var)
with tf.Session() as sess:
sess.run(my_var.initializer)
sess.run(my_var_times_two) # >> the value of my_var now is 4
sess.run(my_var_times_two) # >> the value of my_var now is 8
2*my_var my_varmy_var_times_two
[127]Deep Learning and TensorFlow – Episode 4
Variable Assignments
# create a variable whose original value is 2
my_var = tf.Variable(2, name="my_var")
# assign a*2 to a and call that op a_times_two
my_var_times_two = my_var.assign(2*my_var)
with tf.Session() as sess:
sess.run(my_var.initializer)
sess.run(my_var_times_two) # >> the value of my_var now is 4
sess.run(my_var_times_two) # >> the value of my_var now is 8
sess.run(my_var_times_two) # >> the value of my_var now is 16
2*my_var my_varmy_var_times_two
[128]Deep Learning and TensorFlow – Episode 4
Variables in multiple sessions
W = tf.Variable(10)
sess1 = tf.Session()
sess2 = tf.Session()
sess1.run(W.initializer)
sess2.run(W.initializer)
sess1.close()
sess2.close()
[129]Deep Learning and TensorFlow – Episode 4
Variables in multiple sessions
W = tf.Variable(10)
sess1 = tf.Session()
sess2 = tf.Session()
sess1.run(W.initializer)
sess2.run(W.initializer)
print(sess1.run(W.assign_add(10)))
print(sess2.run(W.assign_sub(2)))
sess1.close()
sess2.close()
[130]Deep Learning and TensorFlow – Episode 4
Variables in multiple sessions
W = tf.Variable(10)
sess1 = tf.Session()
sess2 = tf.Session()
sess1.run(W.initializer)
sess2.run(W.initializer)
print(sess1.run(W.assign_add(10)))
print(sess2.run(W.assign_sub(2)))
sess1.close()
sess2.close()
# >> 20# >> 8
[131]Deep Learning and TensorFlow – Episode 4
Variables in multiple sessions
W = tf.Variable(10)
sess1 = tf.Session()
sess2 = tf.Session()
sess1.run(W.initializer)
sess2.run(W.initializer)
print(sess1.run(W.assign_add(10)))
print(sess2.run(W.assign_sub(2)))
print(sess1.run(W.assign_add(100)))
print(sess2.run(W.assign_add(100)))
sess1.close()
sess2.close()
# >> 20# >> 8
[132]Deep Learning and TensorFlow – Episode 4
Variables in multiple sessions
W = tf.Variable(10)
sess1 = tf.Session()
sess2 = tf.Session()
sess1.run(W.initializer)
sess2.run(W.initializer)
print(sess1.run(W.assign_add(10)))
print(sess2.run(W.assign_sub(2)))
print(sess1.run(W.assign_add(100)))
print(sess2.run(W.assign_add(100)))
sess1.close()
sess2.close()
# >> 20# >> 8
# >> 120# >> 108
[134]Deep Learning and TensorFlow – Episode 4
Placeholders
[135]Deep Learning and TensorFlow – Episode 4
Placeholders
[136]Deep Learning and TensorFlow – Episode 4
Placeholders
[137]Deep Learning and TensorFlow – Episode 4
Placeholders
[138]Deep Learning and TensorFlow – Episode 4
Placeholders
[139]Deep Learning and TensorFlow – Episode 4
Placeholders
[140]Deep Learning and TensorFlow – Episode 4
Placeholder and feed dicts tf.placeholder(dtype, shape=None, name=None)
[141]Deep Learning and TensorFlow – Episode 4
Placeholder and feed dicts tf.placeholder(dtype, shape=None, name=None)
# create a placeholder for a vector of 3 elements, type tf.float32
a = tf.placeholder(tf.float32, shape=[3])
b = tf.constant([5, 5, 5], tf.float32)
# use the placeholder as you would a constant or a variable
c = a + b # short for tf.add(a, b)
[142]Deep Learning and TensorFlow – Episode 4
Placeholder and feed dicts tf.placeholder(dtype, shape=None, name=None)
# create a placeholder for a vector of 3 elements, type tf.float32
a = tf.placeholder(tf.float32, shape=[3])
b = tf.constant([5, 5, 5], tf.float32)
# use the placeholder as you would a constant or a variable
c = a + b # short for tf.add(a, b)
[143]Deep Learning and TensorFlow – Episode 4
Placeholder and feed dicts tf.placeholder(dtype, shape=None, name=None)
# create a placeholder for a vector of 3 elements, type tf.float32
a = tf.placeholder(tf.float32, shape=[3])
b = tf.constant([5, 5, 5], tf.float32)
# use the placeholder as you would a constant or a variable
c = a + b # short for tf.add(a, b)
with tf.Session() as sess:print(sess.run(c))
[144]Deep Learning and TensorFlow – Episode 4
Placeholder and feed dicts tf.placeholder(dtype, shape=None, name=None)
# create a placeholder for a vector of 3 elements, type tf.float32
a = tf.placeholder(tf.float32, shape=[3])
b = tf.constant([5, 5, 5], tf.float32)
# use the placeholder as you would a constant or a variable
c = a + b # short for tf.add(a, b)
>> InvalidArgumentError: a doesn’t have an>> actual value
with tf.Session() as sess:print(sess.run(c))
[145]Deep Learning and TensorFlow – Episode 4
Placeholder and feed dicts tf.placeholder(dtype, shape=None, name=None)
# create a placeholder for a vector of 3 elements, type tf.float32
a = tf.placeholder(tf.float32, shape=[3])
b = tf.constant([5, 5, 5], tf.float32)
# use the placeholder as you would a constant or a variable
c = a + b # short for tf.add(a, b)
with tf.Session() as sess:print(sess.run(c, feed_dict={a: [1, 2, 3]}))
[146]Deep Learning and TensorFlow – Episode 4
Placeholder and feed dicts tf.placeholder(dtype, shape=None, name=None)
# create a placeholder for a vector of 3 elements, type tf.float32
a = tf.placeholder(tf.float32, shape=[3])
b = tf.constant([5, 5, 5], tf.float32)
# use the placeholder as you would a constant or a variable
c = a + b # short for tf.add(a, b)
with tf.Session() as sess:print(sess.run(c, feed_dict={a: [1, 2, 3]}))
>> [6, 7, 8]
[147]Deep Learning and TensorFlow – Episode 4
Feed dicts
[148]Deep Learning and TensorFlow – Episode 4
Feed dicts
print(sess.run(some_op, feed_dict={a: [1, 2, 3]}))
[149]Deep Learning and TensorFlow – Episode 4
Feed dicts
print(sess.run(some_op, feed_dict={a: [1, 2, 3]}))
[150]Deep Learning and TensorFlow – Episode 4
Feed dicts
print(sess.run(some_op, feed_dict={a: [1, 2, 3]}))
None
[151]Deep Learning and TensorFlow – Episode 4
Feed dicts
print(sess.run(some_op, feed_dict={a: [1, 2, 3]}))
None
a = tf.placeholder(tf.float32, shape=None)
[152]Deep Learning and TensorFlow – Episode 4
Feed dicts
print(sess.run(some_op, feed_dict={a: [1, 2, 3]}))
None
a = tf.placeholder(tf.float32, shape=None)
[153]Deep Learning and TensorFlow – Episode 4
Feed dicts
print(sess.run(some_op, feed_dict={a: [1, 2, 3]}))
None
a = tf.placeholder(tf.float32, shape=None)
[154]Deep Learning and TensorFlow – Episode 4
Feed dicts
print(sess.run(some_op, feed_dict={a: [1, 2, 3]}))
None
a = tf.placeholder(tf.float32, shape=None)
with tf.Session() as sess:
for a_value in list_of_values_for_a:
print(sess.run(some_op, {a: a_value}))
[155]Deep Learning and TensorFlow – Episode 4
Feed dicts
[156]Deep Learning and TensorFlow – Episode 4
Feed dicts
[157]Deep Learning and TensorFlow – Episode 4
Feed dicts
a = tf.add(2, 5)
b = tf.multiply(a, 3)
with tf.Session() as sess:
# compute the value of b given a is 15
sess.run(b, feed_dict={a: 15})
[158]Deep Learning and TensorFlow – Episode 4
Feed dicts
a = tf.add(2, 5)
b = tf.multiply(a, 3)
with tf.Session() as sess:
# compute the value of b given a is 15
sess.run(b, feed_dict={a: 15})
[159]Deep Learning and TensorFlow – Episode 4
Feed dicts
a = tf.add(2, 5)
b = tf.multiply(a, 3)
with tf.Session() as sess:
# compute the value of b given a is 15
sess.run(b, feed_dict={a: 15})
tf.Graph.is_feedable(tensor
[160]Deep Learning and TensorFlow – Episode 4
A common mistake
[161]Deep Learning and TensorFlow – Episode 4
A common mistake
[162]Deep Learning and TensorFlow – Episode 4
A common mistake
[163]Deep Learning and TensorFlow – Episode 4
Lazy loading
x = tf.Variable(10, name='x')
y = tf.Variable(20, name='y')
z = tf.add(x, y)
writer = tf.summary.FileWriter('./graphs/normal_loading', tf.get_default_graph())
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
for _ in range(10):
sess.run(z)
writer.close()
[164]Deep Learning and TensorFlow – Episode 4
Lazy loading
x = tf.Variable(10, name='x')
y = tf.Variable(20, name='y')
z = tf.add(x, y)
writer = tf.summary.FileWriter('./graphs/normal_loading', tf.get_default_graph())
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
for _ in range(10):
sess.run(z)
writer.close()
[165]Deep Learning and TensorFlow – Episode 4
Lazy loading
x = tf.Variable(10, name='x')
y = tf.Variable(20, name='y')
writer = tf.summary.FileWriter('./graphs/normal_loading', tf.get_default_graph())
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
for _ in range(10):
sess.run(tf.add(x, y))
writer.close()
[166]Deep Learning and TensorFlow – Episode 4
Lazy loading
x = tf.Variable(10, name='x')
y = tf.Variable(20, name='y')
writer = tf.summary.FileWriter('./graphs/normal_loading', tf.get_default_graph())
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
for _ in range(10):
sess.run(tf.add(x, y))
writer.close()
[167]Deep Learning and TensorFlow – Episode 4
Lazy loading
[168]Deep Learning and TensorFlow – Episode 4
Lazy loading
node {
name: "Add"
op: "Add"
input: "x/read"
input: "y/read"
attr {
key: "T"
value {
type: DT_INT32
}
}
}
[169]Deep Learning and TensorFlow – Episode 4
Lazy loading
node {
name: "Add"
op: "Add"
input: "x/read"
input: "y/read"
attr {
key: "T"
value {
type: DT_INT32
}
}
}
node {
name: "Add_1"
op: "Add"
...
}
...
node {
name: "Add_10"
op: "Add"
...
}
[170]Deep Learning and TensorFlow – Episode 4
Lazy loading
node {
name: "Add"
op: "Add"
input: "x/read"
input: "y/read"
attr {
key: "T"
value {
type: DT_INT32
}
}
}
node {
name: "Add_1"
op: "Add"
...
}
...
node {
name: "Add_10"
op: "Add"
...
}
[171]Deep Learning and TensorFlow – Episode 4
Lazy loading
[172]Deep Learning and TensorFlow – Episode 4
Lazy loading
@property
[173]Deep Learning and TensorFlow – Episode 4
Next step…build the first machine learning model!
[174]Deep Learning and TensorFlow – Episode 4
The first machine learning model in TF
[175]Deep Learning and TensorFlow – Episode 4
Now's the time for part 2!