1. 1. Deep LearningDeep Learning2014/08/26 Preferred Networks
2. 2. l (Seiya Tokui)l Preferred Networks l Jubatus Jubatus: NTTPFI http://jubat.us/ l 2
3. 3. l SGDl Pylearn2, Torch7, Caffe, Cuda-convnet2l l 3
4. 4.
5. 5. Deep LearningNeural Network: Convolutional NN: NNDeep Belief Network(),Deep Boltzmann Machine():Deep 35
6. 6. Feed-Forward Neural Networkx 1wj1x2wj2x3wj3x4wj4hj = f(wj1x1 + + wj4x4 + bj) = f(wj x + bj)
7. 7. Feed-Forward Neural Networkh = f(Wx+ b)3y = f3(W3f2(W2f1(W1x + b1) + b2) + b3)W1 W2 W3x h1 h2 y
8. 8. l lossl L(W) =1NXNi=1`(y(xi;W), yi)l `y(x;W)(xi, yi)i=1,...,NWl 8minimize WL(W)`(y, yi) = ky y1k2
9. 9. gradient W W rL(W)9L
10. 10. Stochastic Gradient Descent (SGD)l l 10B|B|W W |B|Xi2Br`(y(xi;W), yi)
11. 11. l SGD l 11h h + 1|B|Xni2Br`(y(xi;W), yi),W W h.W(t)r`W(t