"practical machine learning with ruby" by iqbal farabi (id ruby community)
TRANSCRIPT
Practical Machine Learning with Ruby
Tech in Asia ConferenceJakarta, 2016/11/16
About Me• Iqbal Farabi• Cofounder of: • Virkea Empresa Sistema• Starqle Indonesia
• Member of ID Ruby community• Contact: [email protected]
What to Expect
Basic concepts of Machine Learning
Introduction to Classification with Neural Network
Neural Network Implementation with Ruby
Let’s learn together!
Overview
Machine Learning
Mathematical Models• Simple Regression• Handwritten Recognition
Ruby Implementation• Simple Regression• Handwritten Recognition
Machine Learning
Machine LearningMotivations
Data Science Full Stack
Data Science Full Stack
Machine Learning
SciRuby ManifestoRuby has for some time lacked libraries implementing the basic tools needed for scientific computing and data visualization.
We believe that the time for a Ruby science and visualization package has come. Sometimes when a solution of sugar and water becomes super-saturated, from it precipitates a pure, delicious, and diabetes-causing crystal of sweetness, induced by no more than the tap of a finger. So it is, we believe, with the need for numeric and visualization libraries in Ruby.
Machine LearningBasic Concepts
DefinitionMachine learning is a type of artificial intelligence (AI) that provides computers with the ability to learn without being explicitly programmed.
Machine learning focuses on the development of computer programs that can teach themselves to grow and change when exposed to new data.
Types of Learning• Supervised Learning• Unsupervised Learning• Reinforcement Learning
Supervised LearningInput data is called training data and has a known label or result such as spam/not-spam or a stock price at a time.
A model is prepared through a training process where it is required to make predictions and is corrected when those predictions are wrong. The training process continues until the model achieves a desired level of accuracy on the training data.
Supervised Learningx y1 22 43 64 ?
Supervised LearningSize (ft) # of Bedrooms # of Floors Age of home Price ($1000)
1 2104 5 1 45 460
1 1416 3 2 40 232
1 1534 3 2 30 315
1 852 2 1 36 178
Mathematical ModelsSimple Regression
Regression (1)x y1 22 43 64 ?
Regression (2)
Regression (3)
Regression (4)
Regression (5)
Regression (6)
Regression (7)
Regression (8)
Regression (10)
Ruby ImplementationSimple Regression
Some Gems You’ll Need• iRuby - https://github.com/SciRuby/iruby• NMatrix - https://github.com/SciRuby/nmatrix• Nyaplot - https://github.com/domitry/nyaplot• Ruby GSL - https://github.com/SciRuby/rb-gsl
How do We Translate this to Ruby?
Matrix Representation
Math Matrix
Math Ruby
Gradient Descent
Demo!source: my github repo
Mathematical ModelsNeural Network
Perceptron (1)
x1
x2
x3
output
Perceptron (2)• x1, x2, x3 single binary inputs• add “weight” for each input: w1, w2, w3
ouput0 if ≤ threshold
1 if > threshold
Perceptron Example (1)Should I go to TIA Conference?• x1 = can I take a leave in November 16th and 17th?• x2 = are my friends going to the conference?• x3 = is there any talk that interests me?
Add “weight” for each input, for instance: • w1 = 6 (means this is the most important factor in my decision) • w2 = 1 (means this is the least important factor in my decision)• w3 = 3 (means this is less important factor in my decision)
Perceptron Example (2)Decide threshold• Lower threshold means that I am more likely to go to TIA Conference• Higher threshold means that I am less likely to go to TIA Conference• Example: threshold = 3
Calculate perceptron, example:• output = (0 * 6) + (1 * 1) + (1 * 3) = 4• output > threshold I go to TIA Conference!
Perceptron Modified (1)
(vectorized)
Perceptron Modified (2)
ouput0 if ≤ threshold
1 if > thresholdouput
0 if w . x + b ≤ 0
1 if w . x + b > 0
b = bias = -threshold
Perceptron as Logic Function (1)Example: NAND Perceptron
x1 x2 Calculation Result
0 0 0*(-2) + 0*(-2) + 3 = 3 True
1 0 1*(-2) + 0*(-2) + 3 = 1 True
0 1 0*(-2) + 1*(-2) + 3 = 1 Tue
1 1 1*(-2) + 1*(-2) + 3 = -1 False
3
x1
x2
-2
-2
Perceptron as Logic Function (2)Example: Bitwise Copy
-2
-2-2
-2
-2
-2
-2-2
-4
3
3
3
3
3
source: Neural Networks and Deep Learning
Neural Network’s Goal• To devise learning algorithms which can automatically tune the
weights and biases of a network of artificial neurons. • This tuning happens in response to external stimuli, without direct
intervention by a programmer. • Enable the use of artificial neurons in a way which is radically different
to conventional logic gates• Learn to solve problems, sometimes where it would be extremely
difficult to directly design a conventional circuit
Ruby ImplementationHandwritten Recognition
Handwritten Recognition
Convert Each Image to 28x28 Pixels
Neural Network
Ruby ImplementantionInput Layer• x1 = greyscale value of pixel at (0, 0)• …• x784 = greyscale value of pixel at (27, 27)
Output Layer: • y1 = 0, y2 = 1, y3 = 2, y4 = 3, y5 = 4, y6 = 5, y7 = 6, y8 = 7, y9 = 8, y10 = 9
Neural Network’s goal: based on training data,• Find weight• Find bias
Cost Function• w = weight• b = bias• x = network input• y(x) = label for each input• a = network output
Gradient Descent• Given the cost function, try to minimize cost function
Gems You’ll Need• Chunky PNG (https://github.com/wvanbergen/chunky_png)• Nmatrix (https://github.com/tangledpath/ruby-fann)• Ruby FANN (https://github.com/tangledpath/ruby-fann)• Sinatra (https://github.com/sinatra/sinatra)
Training Neural Network (1)
Training Neural Network (3)
Training Neural Network (2)
Predict
Demo!source: Geoff Buesing
What’s Next?
What’s Next?• Explore more SciRuby Libraries• Implement more learning algorithms in Ruby• Learn more Math XD• Join ID Ruby community! (http://tinyurl.com/id-ruby-slack)
Thank You!