"practical machine learning with ruby" by iqbal farabi (id ruby community)

Practical Machine Learning with Ruby

Tech in Asia ConferenceJakarta, 2016/11/16

About Me• Iqbal Farabi• Cofounder of: • Virkea Empresa Sistema• Starqle Indonesia

• Member of ID Ruby community• Contact: farabi.Iqbal@gmail.com

What to Expect

Basic concepts of Machine Learning

Introduction to Classification with Neural Network

Neural Network Implementation with Ruby

Let’s learn together!

Overview

Machine Learning

Mathematical Models• Simple Regression• Handwritten Recognition

Ruby Implementation• Simple Regression• Handwritten Recognition

Machine Learning

Machine LearningMotivations

Data Science Full Stack

Data Science Full Stack

Machine Learning

SciRuby ManifestoRuby has for some time lacked libraries implementing the basic tools needed for scientific computing and data visualization.

We believe that the time for a Ruby science and visualization package has come. Sometimes when a solution of sugar and water becomes super-saturated, from it precipitates a pure, delicious, and diabetes-causing crystal of sweetness, induced by no more than the tap of a finger. So it is, we believe, with the need for numeric and visualization libraries in Ruby.

Machine LearningBasic Concepts

DefinitionMachine learning is a type of artificial intelligence (AI) that provides computers with the ability to learn without being explicitly programmed.

Machine learning focuses on the development of computer programs that can teach themselves to grow and change when exposed to new data.

Types of Learning• Supervised Learning• Unsupervised Learning• Reinforcement Learning

Supervised LearningInput data is called training data and has a known label or result such as spam/not-spam or a stock price at a time.

A model is prepared through a training process where it is required to make predictions and is corrected when those predictions are wrong. The training process continues until the model achieves a desired level of accuracy on the training data.

Supervised Learningx y1 22 43 64 ?

Supervised LearningSize (ft) # of Bedrooms # of Floors Age of home Price ($1000)

1 2104 5 1 45 460

1 1416 3 2 40 232

1 1534 3 2 30 315

1 852 2 1 36 178

Mathematical ModelsSimple Regression

Regression (1)x y1 22 43 64 ?

Regression (2)

Regression (3)

Regression (4)

Regression (5)

Regression (6)

Regression (7)

Regression (8)

Regression (10)

Ruby ImplementationSimple Regression

Some Gems You’ll Need• iRuby - https://github.com/SciRuby/iruby• NMatrix - https://github.com/SciRuby/nmatrix• Nyaplot - https://github.com/domitry/nyaplot• Ruby GSL - https://github.com/SciRuby/rb-gsl

How do We Translate this to Ruby?

Matrix Representation

Math Matrix

Math Ruby

Gradient Descent

Demo!source: my github repo

Mathematical ModelsNeural Network

Perceptron (1)

x1

x2

x3

output

Perceptron (2)• x1, x2, x3 single binary inputs• add “weight” for each input: w1, w2, w3

ouput0 if ≤ threshold

1 if > threshold

Perceptron Example (1)Should I go to TIA Conference?• x1 = can I take a leave in November 16th and 17th?• x2 = are my friends going to the conference?• x3 = is there any talk that interests me?

Add “weight” for each input, for instance: • w1 = 6 (means this is the most important factor in my decision) • w2 = 1 (means this is the least important factor in my decision)• w3 = 3 (means this is less important factor in my decision)

Perceptron Example (2)Decide threshold• Lower threshold means that I am more likely to go to TIA Conference• Higher threshold means that I am less likely to go to TIA Conference• Example: threshold = 3

Calculate perceptron, example:• output = (0 * 6) + (1 * 1) + (1 * 3) = 4• output > threshold I go to TIA Conference!

Perceptron Modified (1)

(vectorized)

Perceptron Modified (2)

ouput0 if ≤ threshold

1 if > thresholdouput

0 if w . x + b ≤ 0

1 if w . x + b > 0

b = bias = -threshold

Perceptron as Logic Function (1)Example: NAND Perceptron

x1 x2 Calculation Result

0 0 0*(-2) + 0*(-2) + 3 = 3 True

1 0 1*(-2) + 0*(-2) + 3 = 1 True

0 1 0*(-2) + 1*(-2) + 3 = 1 Tue

1 1 1*(-2) + 1*(-2) + 3 = -1 False

3

x1

x2

-2

-2

Perceptron as Logic Function (2)Example: Bitwise Copy

-2

-2-2

-2

-2

-2

-2-2

-4

3

3

3

3

3

source: Neural Networks and Deep Learning

Neural Network’s Goal• To devise learning algorithms which can automatically tune the

weights and biases of a network of artificial neurons. • This tuning happens in response to external stimuli, without direct

intervention by a programmer. • Enable the use of artificial neurons in a way which is radically different

to conventional logic gates• Learn to solve problems, sometimes where it would be extremely

difficult to directly design a conventional circuit

Ruby ImplementationHandwritten Recognition

Handwritten Recognition

Convert Each Image to 28x28 Pixels

Neural Network

Ruby ImplementantionInput Layer• x1 = greyscale value of pixel at (0, 0)• …• x784 = greyscale value of pixel at (27, 27)

Output Layer: • y1 = 0, y2 = 1, y3 = 2, y4 = 3, y5 = 4, y6 = 5, y7 = 6, y8 = 7, y9 = 8, y10 = 9

Neural Network’s goal: based on training data,• Find weight• Find bias

Cost Function• w = weight• b = bias• x = network input• y(x) = label for each input• a = network output

Gradient Descent• Given the cost function, try to minimize cost function

Gems You’ll Need• Chunky PNG (https://github.com/wvanbergen/chunky_png)• Nmatrix (https://github.com/tangledpath/ruby-fann)• Ruby FANN (https://github.com/tangledpath/ruby-fann)• Sinatra (https://github.com/sinatra/sinatra)

Training Neural Network (1)

Training Neural Network (3)

Training Neural Network (2)

Predict

Demo!source: Geoff Buesing

What’s Next?

What’s Next?• Explore more SciRuby Libraries• Implement more learning algorithms in Ruby• Learn more Math XD• Join ID Ruby community! (http://tinyurl.com/id-ruby-slack)

Thank You!