ml 3 - goto amsterdam - willem meints › uploads › slides › conference...• building neural...
TRANSCRIPT
![Page 1: ML 3 - GOTO Amsterdam - Willem Meints › uploads › slides › conference...• Building neural networks with DeepLearning4J • How to get started yourself. Setting the stage. How](https://reader034.vdocuments.net/reader034/viewer/2022042406/5f20a19e594bd166d8440219/html5/thumbnails/1.jpg)
![Page 2: ML 3 - GOTO Amsterdam - Willem Meints › uploads › slides › conference...• Building neural networks with DeepLearning4J • How to get started yourself. Setting the stage. How](https://reader034.vdocuments.net/reader034/viewer/2022042406/5f20a19e594bd166d8440219/html5/thumbnails/2.jpg)
![Page 3: ML 3 - GOTO Amsterdam - Willem Meints › uploads › slides › conference...• Building neural networks with DeepLearning4J • How to get started yourself. Setting the stage. How](https://reader034.vdocuments.net/reader034/viewer/2022042406/5f20a19e594bd166d8440219/html5/thumbnails/3.jpg)
Agenda• Setting the stage • Building neural networks with DeepLearning4J • How to get started yourself
![Page 4: ML 3 - GOTO Amsterdam - Willem Meints › uploads › slides › conference...• Building neural networks with DeepLearning4J • How to get started yourself. Setting the stage. How](https://reader034.vdocuments.net/reader034/viewer/2022042406/5f20a19e594bd166d8440219/html5/thumbnails/4.jpg)
Setting the
stage
![Page 5: ML 3 - GOTO Amsterdam - Willem Meints › uploads › slides › conference...• Building neural networks with DeepLearning4J • How to get started yourself. Setting the stage. How](https://reader034.vdocuments.net/reader034/viewer/2022042406/5f20a19e594bd166d8440219/html5/thumbnails/5.jpg)
![Page 6: ML 3 - GOTO Amsterdam - Willem Meints › uploads › slides › conference...• Building neural networks with DeepLearning4J • How to get started yourself. Setting the stage. How](https://reader034.vdocuments.net/reader034/viewer/2022042406/5f20a19e594bd166d8440219/html5/thumbnails/6.jpg)
![Page 7: ML 3 - GOTO Amsterdam - Willem Meints › uploads › slides › conference...• Building neural networks with DeepLearning4J • How to get started yourself. Setting the stage. How](https://reader034.vdocuments.net/reader034/viewer/2022042406/5f20a19e594bd166d8440219/html5/thumbnails/7.jpg)
How does the bot find an answer to your question?
![Page 8: ML 3 - GOTO Amsterdam - Willem Meints › uploads › slides › conference...• Building neural networks with DeepLearning4J • How to get started yourself. Setting the stage. How](https://reader034.vdocuments.net/reader034/viewer/2022042406/5f20a19e594bd166d8440219/html5/thumbnails/8.jpg)
A search problem• The question somehow relates to a specific answer.
• Two parts to the answer: • We need to finger print the questions • That fingerprint then needs to be related to a specific answer
![Page 9: ML 3 - GOTO Amsterdam - Willem Meints › uploads › slides › conference...• Building neural networks with DeepLearning4J • How to get started yourself. Setting the stage. How](https://reader034.vdocuments.net/reader034/viewer/2022042406/5f20a19e594bd166d8440219/html5/thumbnails/9.jpg)
Let’s teach the computer howBy building a neural network
![Page 10: ML 3 - GOTO Amsterdam - Willem Meints › uploads › slides › conference...• Building neural networks with DeepLearning4J • How to get started yourself. Setting the stage. How](https://reader034.vdocuments.net/reader034/viewer/2022042406/5f20a19e594bd166d8440219/html5/thumbnails/10.jpg)
Input layer Hidden layer Output layer
Quick introduction to neural networks
![Page 11: ML 3 - GOTO Amsterdam - Willem Meints › uploads › slides › conference...• Building neural networks with DeepLearning4J • How to get started yourself. Setting the stage. How](https://reader034.vdocuments.net/reader034/viewer/2022042406/5f20a19e594bd166d8440219/html5/thumbnails/11.jpg)
Input Prediction
[0 . 0…1.0 ] [
0 . 73…1.0 ]
Input layer Hidden layer Output layer
Quick introduction to neural networks
![Page 12: ML 3 - GOTO Amsterdam - Willem Meints › uploads › slides › conference...• Building neural networks with DeepLearning4J • How to get started yourself. Setting the stage. How](https://reader034.vdocuments.net/reader034/viewer/2022042406/5f20a19e594bd166d8440219/html5/thumbnails/12.jpg)
𝑦 = 𝑓(𝑛
∑𝑖=1
(𝑤𝑖 ∗ 𝑥𝑖))
x1
x2
x3
y
w1
w2
w3
∑ 𝑓
![Page 13: ML 3 - GOTO Amsterdam - Willem Meints › uploads › slides › conference...• Building neural networks with DeepLearning4J • How to get started yourself. Setting the stage. How](https://reader034.vdocuments.net/reader034/viewer/2022042406/5f20a19e594bd166d8440219/html5/thumbnails/13.jpg)
Build a neural network
with DeepLearning4J
![Page 14: ML 3 - GOTO Amsterdam - Willem Meints › uploads › slides › conference...• Building neural networks with DeepLearning4J • How to get started yourself. Setting the stage. How](https://reader034.vdocuments.net/reader034/viewer/2022042406/5f20a19e594bd166d8440219/html5/thumbnails/14.jpg)
![Page 15: ML 3 - GOTO Amsterdam - Willem Meints › uploads › slides › conference...• Building neural networks with DeepLearning4J • How to get started yourself. Setting the stage. How](https://reader034.vdocuments.net/reader034/viewer/2022042406/5f20a19e594bd166d8440219/html5/thumbnails/15.jpg)
ND4J – Scientific computation for the JVM
DeepLearning4J – Deep learning framework
NLP ETLNeural
networksSpark
integration
GPU support with CUDA CPU with/without Intel MKL
![Page 16: ML 3 - GOTO Amsterdam - Willem Meints › uploads › slides › conference...• Building neural networks with DeepLearning4J • How to get started yourself. Setting the stage. How](https://reader034.vdocuments.net/reader034/viewer/2022042406/5f20a19e594bd166d8440219/html5/thumbnails/16.jpg)
Shared library with model definition
Training application
Microsoft bot connector service
Chatbot
Application architecture
![Page 17: ML 3 - GOTO Amsterdam - Willem Meints › uploads › slides › conference...• Building neural networks with DeepLearning4J • How to get started yourself. Setting the stage. How](https://reader034.vdocuments.net/reader034/viewer/2022042406/5f20a19e594bd166d8440219/html5/thumbnails/17.jpg)
We’re going to follow a 4 step recipe• Step 1: Encode the input • Step 2: Map the answers • Step 3: Build the neural network • Step 4: Train the neural network
![Page 18: ML 3 - GOTO Amsterdam - Willem Meints › uploads › slides › conference...• Building neural networks with DeepLearning4J • How to get started yourself. Setting the stage. How](https://reader034.vdocuments.net/reader034/viewer/2022042406/5f20a19e594bd166d8440219/html5/thumbnails/18.jpg)
Step 1: Encode the input
![Page 19: ML 3 - GOTO Amsterdam - Willem Meints › uploads › slides › conference...• Building neural networks with DeepLearning4J • How to get started yourself. Setting the stage. How](https://reader034.vdocuments.net/reader034/viewer/2022042406/5f20a19e594bd166d8440219/html5/thumbnails/19.jpg)
Encoding text as a bag of wordsThree steps:
1. Create a vector equal to the size of your vocabulary 2. Count word ocurrences 3. Assign the count each word a unique index in the vector
![Page 20: ML 3 - GOTO Amsterdam - Willem Meints › uploads › slides › conference...• Building neural networks with DeepLearning4J • How to get started yourself. Setting the stage. How](https://reader034.vdocuments.net/reader034/viewer/2022042406/5f20a19e594bd166d8440219/html5/thumbnails/20.jpg)
𝑋𝑡𝑟𝑎𝑖𝑛 = [011]
Hello
World
![Page 21: ML 3 - GOTO Amsterdam - Willem Meints › uploads › slides › conference...• Building neural networks with DeepLearning4J • How to get started yourself. Setting the stage. How](https://reader034.vdocuments.net/reader034/viewer/2022042406/5f20a19e594bd166d8440219/html5/thumbnails/21.jpg)
TokenizerFactory tokenizerFactory = new DefaultTokenizerFactory();tokenizerFactory.setTokenPreProcessor(new CommonPreprocessor());// This vectorizer uses the TF-IDF algorithm to produce// a unique fingerprint for every question we feed it.BagOfWordsVectorizer vectorizer = new BagOfWordsVectorizer.Builder() .setTokenizerFactory(tokenizerFactory) .setIterator(new CSVSentenceIterator(inputFile)) .build();
Create a bag of words in DL4J
![Page 22: ML 3 - GOTO Amsterdam - Willem Meints › uploads › slides › conference...• Building neural networks with DeepLearning4J • How to get started yourself. Setting the stage. How](https://reader034.vdocuments.net/reader034/viewer/2022042406/5f20a19e594bd166d8440219/html5/thumbnails/22.jpg)
Step 2: Encode answers
![Page 23: ML 3 - GOTO Amsterdam - Willem Meints › uploads › slides › conference...• Building neural networks with DeepLearning4J • How to get started yourself. Setting the stage. How](https://reader034.vdocuments.net/reader034/viewer/2022042406/5f20a19e594bd166d8440219/html5/thumbnails/23.jpg)
Encode answers
Answer 1
Answer 2
Answer 3
Answer 4
![Page 24: ML 3 - GOTO Amsterdam - Willem Meints › uploads › slides › conference...• Building neural networks with DeepLearning4J • How to get started yourself. Setting the stage. How](https://reader034.vdocuments.net/reader034/viewer/2022042406/5f20a19e594bd166d8440219/html5/thumbnails/24.jpg)
Map neurons to answerstry (CSVRecordReader reader = new CSVRecordReader(1, ',')) { reader.initialize(new FileSplit(inputFile)); Map<Integer, String> answers = new HashMap<>(); while(reader.hasNext()) { List<Writable> record = reader.next(); // Note: The answer index needs a -1, so that we get an offset mapping. answers.put(record.get(0).toInt() - 1, record.get(1).toString()); } return answers;}
![Page 25: ML 3 - GOTO Amsterdam - Willem Meints › uploads › slides › conference...• Building neural networks with DeepLearning4J • How to get started yourself. Setting the stage. How](https://reader034.vdocuments.net/reader034/viewer/2022042406/5f20a19e594bd166d8440219/html5/thumbnails/25.jpg)
Step 3: Build the neural network
![Page 26: ML 3 - GOTO Amsterdam - Willem Meints › uploads › slides › conference...• Building neural networks with DeepLearning4J • How to get started yourself. Setting the stage. How](https://reader034.vdocuments.net/reader034/viewer/2022042406/5f20a19e594bd166d8440219/html5/thumbnails/26.jpg)
Fingerprint the data with an auto-encoder
![Page 27: ML 3 - GOTO Amsterdam - Willem Meints › uploads › slides › conference...• Building neural networks with DeepLearning4J • How to get started yourself. Setting the stage. How](https://reader034.vdocuments.net/reader034/viewer/2022042406/5f20a19e594bd166d8440219/html5/thumbnails/27.jpg)
Relate the fingerprint to an answer
Auto-encoder
Feed forward network
![Page 28: ML 3 - GOTO Amsterdam - Willem Meints › uploads › slides › conference...• Building neural networks with DeepLearning4J • How to get started yourself. Setting the stage. How](https://reader034.vdocuments.net/reader034/viewer/2022042406/5f20a19e594bd166d8440219/html5/thumbnails/28.jpg)
MultiLayerConfiguration networkConfiguration = new NeuralNetConfiguration.Builder() .seed(1337) .list() .layer(0, new VariationalAutoencoder.Builder() .nIn(inputLayerSize).nOut(1024) .encoderLayerSizes(1024, 512, 256, 128) .decoderLayerSizes(128, 256, 512, 1024) .lossFunction(Activation.RELU, LossFunctions.LossFunction.MSE) .gradientNormalization(GradientNormalization.ClipElementWiseAbsoluteValue) .dropOut(0.8) .build()) .layer(1, new OutputLayer.Builder() .nIn(1024).nOut(outputLayerSize) .activation(Activation.SOFTMAX) .lossFunction(LossFunctions.LossFunction.NEGATIVELOGLIKELIHOOD) .build()) .updater(new RmsProp(0.01)) .pretrain(true) .backprop(true) .build();
![Page 29: ML 3 - GOTO Amsterdam - Willem Meints › uploads › slides › conference...• Building neural networks with DeepLearning4J • How to get started yourself. Setting the stage. How](https://reader034.vdocuments.net/reader034/viewer/2022042406/5f20a19e594bd166d8440219/html5/thumbnails/29.jpg)
MultiLayerNetwork network = new MultiLayerNetwork(networkConfiguration);network.setListeners(new ScoreIterationListener(1));network.init();
![Page 30: ML 3 - GOTO Amsterdam - Willem Meints › uploads › slides › conference...• Building neural networks with DeepLearning4J • How to get started yourself. Setting the stage. How](https://reader034.vdocuments.net/reader034/viewer/2022042406/5f20a19e594bd166d8440219/html5/thumbnails/30.jpg)
Step 4: Train the neural network
![Page 31: ML 3 - GOTO Amsterdam - Willem Meints › uploads › slides › conference...• Building neural networks with DeepLearning4J • How to get started yourself. Setting the stage. How](https://reader034.vdocuments.net/reader034/viewer/2022042406/5f20a19e594bd166d8440219/html5/thumbnails/31.jpg)
Map<Integer, String> answers = AnswersMappingFactory.create( new File("data/answers.csv"));TextVectorizer vectorizer = QuestionVectorizerFactory.create( new File("data/"));
vectorizer.fit();vectorizer.save(new File("model/vectorizer.bin"));QuestionClassifier classifier = QuestionClassifierFactory.create( vectorizer, answers);
classifier.fit(new File("data/questions_train.csv"));classifier.save(new File("model/classifier.bin"));
The final training application
![Page 32: ML 3 - GOTO Amsterdam - Willem Meints › uploads › slides › conference...• Building neural networks with DeepLearning4J • How to get started yourself. Setting the stage. How](https://reader034.vdocuments.net/reader034/viewer/2022042406/5f20a19e594bd166d8440219/html5/thumbnails/32.jpg)
Making a prediction
INDArray prediction = network.output(vectorizer.transform(text));int answerIndex = prediction.argMax(1).getInt(0,0);return answers.get(answerIndex);
String replyText = classifier.predict(context.activity().text());
Inside the bot framework adapter
At neural network level
![Page 33: ML 3 - GOTO Amsterdam - Willem Meints › uploads › slides › conference...• Building neural networks with DeepLearning4J • How to get started yourself. Setting the stage. How](https://reader034.vdocuments.net/reader034/viewer/2022042406/5f20a19e594bd166d8440219/html5/thumbnails/33.jpg)
How to get
started yourself
![Page 34: ML 3 - GOTO Amsterdam - Willem Meints › uploads › slides › conference...• Building neural networks with DeepLearning4J • How to get started yourself. Setting the stage. How](https://reader034.vdocuments.net/reader034/viewer/2022042406/5f20a19e594bd166d8440219/html5/thumbnails/34.jpg)
You too can use deep learning• Three tips
• Explore the model zoo • Starts with small experiments and expand if it has potential • Choose a framework that allows you to experiment quickly
![Page 35: ML 3 - GOTO Amsterdam - Willem Meints › uploads › slides › conference...• Building neural networks with DeepLearning4J • How to get started yourself. Setting the stage. How](https://reader034.vdocuments.net/reader034/viewer/2022042406/5f20a19e594bd166d8440219/html5/thumbnails/35.jpg)
Useful resources• The code:
https://github.com/wmeints/qna-bot
• The model zoo:http://www.asimovinstitute.org/neural-network-zoo/
• DeepLearning4J website: http://deeplearning4j.org
• Machine learning simplified: https://www.youtube.com/watch?v=b99UVkWzYTQ&t=5s
![Page 37: ML 3 - GOTO Amsterdam - Willem Meints › uploads › slides › conference...• Building neural networks with DeepLearning4J • How to get started yourself. Setting the stage. How](https://reader034.vdocuments.net/reader034/viewer/2022042406/5f20a19e594bd166d8440219/html5/thumbnails/37.jpg)