everyone can be vango
TRANSCRIPT
Everyone can be van GoghIntroduction to Neural Art
物理三 王昱翔
2016. 11. 08
Reference
• Leon A. Gatys, Alexander S. Ecker, Matthias Bethge, A Neural Algorithm of Artistic Style
• Mark Chang’s slide, http://www.slideshare.net/ckmarkohchang/a-neural-algorithm-of-artistic-style?next_slideshow=1
Outline
• Few examples of Neural Art
• Introduction to CNN
• Implement Neural Art
• Application
• Conclusion
Paintings of van Gogh
Can we paint like van Gogh? (I’m not 9.2%)
Yes! (Use Convolutional Neural Network)
Ref: Mark Chang’s slide
Neural Network
Partial CNN Process
Image(Input)
Convolution Max Pooling
CNN– Convolution
0 0 1 1 0 0
0 1 1 1 1 0
1 1 1 1 1 1
1 1 1 1 1 1
0 0 0.5 0.5 0 0
0 0 0.5 0.5 0 0
0 0 1
0 0 1
0 0 1
Filter_2
Number of filters depends on Users !
0 0 1
0 0.5 0
1 0 0
Filter_1
Find pattern
Input image
Filter -- Element wise matrix multiplication
0 0 1 1 0 0
0 1 1 1 1 0
1 1 1 1 1 1
1 1 1 1 1 1
0 0 0.5 0.5 0 0
0 0 0.5 0.5 0 0
0 0 1
0 0.5 0
1 0 0
0 0 1
0 0 1
0 0 1
2.5
CNN -- convolution 0
1
2
3
4
5
6
w0 = 𝐹11
0 1 2 3 4 56 7 8 9 10 11
0 0 1 1 0 0
0 1 1 1 1 0
Corresponding Index
2.5
w1 = 𝐹12
w2 = 𝐹13
w6 = 𝐹21
0 0 1 1 0 0
0 1 1 1 1 0
1 1 1 1 1 1
1 1 1 1 1 1
0 0 0.5 0.5 0 0
0 0 0.5 0.5 0 0
0 0 1
0 0.5 0
1 0 0
0 0 1
0 0 1
0 0 1
2.5
CNN – Max Pooling
2.5 2.5 1.5 1.5
2.5 2.5 2.5 1.5
1.5 1.5 2 2
1 1.5 1.75 1.5
2.5 2.5
1.5 2
0 0 1 1 0 0
0 1 1 1 1 0
1 1 1 1 1 1
1 1 1 1 1 1
0 0 0.5 0.5 0 0
0 0 0.5 0.5 0 0
0 0 1
0 0.5 0
1 0 0
Max pooling
2.5
Repeat Convolution + Max pooling
Input Image
ConvolutionMax pooling
ConvolutionMax pooling
Whole CNN Process
Image ConvolutionMax
Pooling
N times
FlattenFully
ConnectedNeural
Network
Neural Art
Content
Style
Neural Art- Create Content
0.3 0.7 1 0 0.6
0.2 0.8 0.7 0.4 0.4
0.7 0.1 0.2 0.3 0.5
0.8 0 0.3 0.9 0.1
0.5 0.9 0.6 1 0.8
CNN
CNN
Canvas x
Output X
Output P
𝐿𝑐𝑜𝑛𝑡𝑒𝑛𝑡 =1
2 𝑖,𝑗(𝑋𝑖,𝑗 − 𝑃𝑖,𝑗)
2
Gradient Descent (Back Propagation)
Photo p
Canvas x: Our paintings
CNN: Already trained byother people
CNN
Neural Art – Create Style
0.3 0.7 1 0 0.6
0.2 0.8 0.7 0.4 0.4
0.7 0.1 0.2 0.3 0.5
0.8 0 0.3 0.9 0.1
0.5 0.9 0.6 1 0.8
CNN
Canvas x
Output
Output 𝐿𝑠𝑡𝑦𝑙𝑒 =1
2 𝑖,𝑗(𝑋𝑖,𝑗 − 𝑆𝑖,𝑗)
2
Gradient Descent (Back Propagation)
Style sGram S
Gram X
G
G
∗ 𝐺𝑟𝑎𝑚𝑖,𝑗 = <𝑣𝑖 , 𝑣𝑗>
Gram S_00= <O_s row[0], O_s col[0]>
Update Canvas
• 𝐿𝑡𝑜𝑡𝑎𝑙 = 𝛼𝐿𝑐𝑜𝑛𝑡𝑒𝑛𝑡 + 𝛽𝐿𝑠𝑡𝑦𝑙𝑒 x = x - 𝜂𝜕𝐿𝑡𝑜𝑡𝑎𝑙
𝜕𝑥
Application -- DeepDream
Ref: Deep Dream
Application -- Prisma
Ref: Prisma
Real time Neural Art
Ref: Fast Style transfer in Tensorflow
Conclusion
• Yeah! Computer can paint by themselves!
• Really? They still need a photo for painting.
Thank You For Your Listening !