pycon2016- applying deep learning in information retrieval system
Post on 22-Jan-2017
1.672 Views
Preview:
TRANSCRIPT
Applying Deep Learning in Information Retrieval System
http://twchrislin.com/
My projecthttp://twchrislin.com/project/
Tools: python numpy pandas scikit-learn xgboost theano lasagne
id LifeTime dira FlightDistance FlightDistanceError … signal
18453471 0.00157832377590239050.999999344348907614.0333347320556640.6814009547233582 … 0
5364094 0.00098804314620792870.99970477819442765.536156654357910.3023407459259033 … 0
11130990 0.00087667378829792140.99998414516448966.1173019409179690.27646270394325256… 0
15173787 0.0008541746647097170.99990290403366085.2280673980712890.22073912620544436… 0
1102544 0.00112933525815606140.999995291233062939.069534301757811.8981972932815552 … 0
https://youtu.be/_SjgCoR7e1s
http://www.emotibot.com/
:
python numpy pandas
scikit-learn xgboost theano lasagne
Machine Learning Deep Learning
Outline
1. Information Retrieval 2. Neural Networks and Convolutional Neural Networks 3. Feature Vector Representation 4. Sentence Retrieval 5. Image Retrieval 6. Conclusion
Information Retrieval
Input Query
Encoding Database Matching
Ranking
Neural Networks
http://speech.ee.ntu.edu.tw/~tlkagk/courses/MLDS_2015_2/Lecture/DNN%20(v4).pdf
Single Neuron, Logistic Regression
Sigmoid funciton
28 pixels
28 pixels
784 pixels
Neural Networks
http://speech.ee.ntu.edu.tw/~tlkagk/courses/MLDS_2015_2/Lecture/DNN%20(v4).pdf
Multiclass 10
Neural Networks
http://speech.ee.ntu.edu.tw/~tlkagk/courses/MLDS_2015_2/Lecture/DNN%20(v4).pdf
MLP Multiclass Z1xL1=X1x785 W785xL1
+f(Z1xL1)
Sigmoid funciton
Z1xL2=X1xL1 WL1xL2
+f(Z1xL2)
Convolutional Neural Networks
http://www.slideshare.net/tw_dsconf/ss-62245351
translation invariance
subsampling
Conv + Pooling Conv + Pooling MLP
Feature Vector Representation
Bengio, 2014, Representation Learning: A Review and New Perspectives
Feature Vector
Representation Learning
http://deeplearning4j.org/convolutionalnets
NN feature
extractor
Feature Vector Representation
InputNeural Networks
including MLP, CNN, RNN
Feature Vector
[ 0.25, 0.33…, 0.78 ]
Encoding
Matching
Feature Vector 1
Feature Vector 2
Similarity Function Score
• cosine similarity • v1TM v2
Outline
1. Information Retrieval 2. Neural Networks and Convolutional Neural Networks 3. Feature Vector Representation 4. Sentence Retrieval 5. Image Retrieval 6. Conclusion
<Pad> <Pad> <Pad> <Pad>
Sentence Retrieval
http://emnlp2014.org/papers/pdf/EMNLP2014181.pdf
Yoon Kim, 2014, Convolutional Neural Networks for Sentence Classification
Neural Net Architecture
NN feature
extractor
1 of K (one-hot) +
word embedding
Conv + MaxPooling
Sentence Retrieval
Results: • Training acc: 99% up • Validation acc: 99% up
Experiment: • 5 classes sentences • Training Set 80% • Validation Set 20%
Feature vector PCA tSNE
Sentence Retrieval
Recurrent Neural Network
Kyunghyun Cho et. al., 2014, Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
https://devblogs.nvidia.com/parallelforall/introduction-neural-machine-translation-gpus-part-2/
Skip-Thought Vectors
Skip-Thought Vectors http://arxiv.org/pdf/1506.06726v1.pdf
Encode a sentence into a thought vector hi by predicting its neighbor sentence.
Ryan Kiros et. al., 2015, Skip-Thought Vectors
… I just got back home. I could see the cat on the steps. This was strange. …
Channel A
http://www.cherry123.com/data/book/14299/book14299_2.html
Skip-Thought Vectors
? ?
?
?
?
?
?
! ? !
?
, ,
Semantic Similarity
Skip-Thought Vectors
query sen skt result skt score
… … 0.934
0.918
0.901
0.898
0.889
0.861
0.836
0.831
0.812
0.777
0.765
Reference Relationship
Skip-Thought Vectors
query sen skt result skt score
0.898
0.916
0.836
… … 0.755
0.757
0.763
LimitationsSkip-Thought Vectors
• Requiring huge size of corpus • … I just got back home. I could see the cat on the steps. This was strange. … • … I got back to office. I could see the cat on the steps. This was cool. …
11,051 novels with 17,515,150 sentences
• Scenario dependency
query sen skt result skt score
0.697
… … 0.697
0.697
Something InterestingSkip-Thought Vectors
query sen skt result skt score
0.889
0.697
skip-thought knows: = I’m ?
skip-thought knows: = ?
Image Retrieval
“A cat” Image Input
Expected Outcome:
Feature Vector Calculation
Matching Database
Image Retrieval
Input Image
pre-processing
(2)
(1)
Hashing Vector Layer
FC + Softmax Layer
.
.
.
.
.
.
CNN network
1st: various targets
Step1 Training
copy
CNN network
Step2Training
Kevin Lin et. al, 2015, Deep Learning of Binary Hash Codes for Fast Image Retrieval
.
.
.
.
.
.
2nd: task specific targets
encoding application Hashing
Vector Layer
copy
CNN network
copy
Feature Vector
AlexNet VGG
Image Retrieval Kevin Lin et. al, 2015, Deep Learning of Binary Hash Codes for Fast Image Retrieval
Demohttp://tweddielin.pythonanywhere.com/
https://github.com/tweddielin/flask-imsearchGithub
Conclusion
1. Deep Learning => Trainable Feature Extractor 2. Type of Neural Nets: MLP, CNN, RNN 3.
• http://www.slideshare.net/tw_dsconf/ss-62245351
• http://speech.ee.ntu.edu.tw/~tlkagk/courses_MLSD15_2.html
Backup
Convolutional Neural Networks
http://cs231n.github.io/convolutional-networks/More Information
1 1 2 4
5 6 7 8
3 2 1 -2
1 2 3 4
ReLU
max(0,x)
Convolved Feature
MaxPooling
top related