teaching recurrent neural networks using tensorflow (may 2016)

Teaching Recurrent Neural Networks

using Tensorflow Rajiv Shah

twitter: rajcs4 rshah@pobox.com www.rajivshah.com

Teaching Recurrent Neural Networks using Tensorflow

• How they are used

• Theory - http://playground.tensorflow.org/

• Learn a sine wave - http://tinyurl.com/raj-sine

• Learn to add - http://tinyurl.com/raj-add

• Learn handwriting - https://github.com/hardmaru/write-rnn-tensorflow

Theory

Recurrent Neural Networks: Theory

Unrolling an RNN

RNN

LSTM

Examples of RNNs

Little more theory . . . but fun

Tensorflow Playground• http://playground.tensorflow.org/

Sine Wave

High Level Architecture

• Function to get batches

• Determined type of RNN cells and initialize them

• Decide on cost function and hyper parameters (dropout, learning rate, optimizer, batch size, sequence length)

• Run session

Go to the python notebook

Friendly Tensorflow - Using R / Shiny

• Wrapped calls to tensorflow using the rPython language

• Shiny provided a nice visual interface and the ability to quickly change different sorts of inputs / models

• https://github.com/rajshah4/tensorflow_shiny

Adding

Teaching Kids to Add

Teaching to Add(array([[7], [4], [1], [8], [0], [3], [1], [5], [6], [9], [2], [0], [0], [0], [0]]), 46.0)

Go to the python notebook

Learning Handwriting

Links

• http://www.cs.toronto.edu/~graves/handwriting.html

• https://github.com/hardmaru/write-rnn-tensorflow

Take Aways

• RNNs are sequential learners and state of the art

• Understand why deep learning is powerful

• See how a RNN learns to model a sine wave, add, learn handwriting . . .

Teaching Recurrent Neural Networks

using Tensorflow Rajiv Shah

twitter: rajcs4 rshah@pobox.com www.rajivshah.com