generative - idc.ac.il · metz, luke, et al. "unrolled generative adversarial networks."...
Embed Size (px)
TRANSCRIPT
GenerativeAdversarialNetworks
Yoav Orlev - Deep Image Processing Seminar
Generative ModelsAnd why we want them
Discriminative vs Generative
Discriminative vs Generative
What are generative models?
Given a training set drawn from some distribution,
Tries to fit a model to best represent the data probability.
What are generative models?
Given a training set drawn from some distribution,
Tries to fit a model to best represent the data probability.
Why do we want generative models?
Why do we want generative models?
“What I cannot create
I can not understand”*Richard Feynman
Why do we want generative models?
● Understanding and Compressing Knowledge
Why do we need generative models?
Radford et al (2015).
100 Million Parameters
● Understanding and Compressing Knowledge
DCGAN Radford et al.
200GB -> 10MB
Generated Results From The ImageNet Dataset
Why do we want generative models?
● Understanding and Compressing Knowledge
● Semi supervised Learning
Why do we want generative models?
● Semi supervised Learning
Assumes some
Structure to the underlying
distribution of the data.
Why do we want generative models?
● Semi supervised Learning
99.14% Accuracy on MNIST
Only 10 labels per class.
OpenAI Research
Why do we need generative models?
● Understanding and Compressing Knowledge
● Semi supervised Learning
● Multi modal outputs
Why do we need generative models?
● Multi modal outputs
Lotter et al. (2015)
Why do we need generative models?
● Understanding and Compressing Knowledge
● Semi supervised Learning
● Multi modal outputs
● Generating Data!
Why do we need generative models?
● Understanding and Compressing Knowledge
● Semi supervised Learning
● Multi modal outputs
● Generating Data!
● And more...
Generative Model Types
● Naive Bayes
● Variational Autoencoders
● Hidden Markov Models
● Generative Adversarial Network
● And more...
Generative Model Types
● Naive Bayes
● Variational Autoencoders
● Hidden Markov Models
● Generative Adversarial Network
● And more...
Generative adversarial networksQuick Introduction
End Goal
Training set
End Goal
Input Output
1
2
7
5
2
1
code
End Goal
Input Output
8
2
7
9
3
4
code
How do we learn to generate?
Every Month Different Show.
Every Month Different Show.
Concert name 1
You can’t come in!The ticket color should be red!
Concert name 2
You can’t come in!The ticket date should be in Arial font!
Concert name 3
You can’t come in!The ticket … should be ...!
Concert name x
Concert name 1011
Come In!
Credits
DiscriminatorGenerator Training Data
Notice!
Generator Training Data
X
Generative adversarial networksIan J. Goodfellow et el. [2014]
GAN has 2 players
D Discriminator
GeneratorG
GAN has 2 players
D Discriminator
GeneratorG
Given a sample x, Outputs the probability of x coming from the training set
GAN has 2 players
D Discriminator
GeneratorG
Given a sample x, Outputs the probability of x coming from the training set
Given a random z, Output an imageWhich look as if it came from the training
set
GAN Architecture
Gz
G(z)
x
D
D(x)
D(G(z)
Slide taken from Kevin Mcguinness -
https://www.slideshare.net/xavigiro/deep-learning-for-computer-vision-generative-models-and-adversarial-training-upc-2016
GAN Architecture
Gz
G(z)
x
D
D(x)
D(G(z)
Slide taken from Kevin Mcguinness -
https://www.slideshare.net/xavigiro/deep-learning-for-computer-vision-generative-models-and-adversarial-training-upc-2016
1
0
GAN Training
Gz
G(z)
x
D
D(x)
D(G(z)
Slide taken from Kevin Mcguinness -
https://www.slideshare.net/xavigiro/deep-learning-for-computer-vision-generative-models-and-adversarial-training-upc-2016
GAN Training
Gz
G(z)
x
D
D(x)
D(G(z)
Slide taken from Kevin Mcguinness -
https://www.slideshare.net/xavigiro/deep-learning-for-computer-vision-generative-models-and-adversarial-training-upc-2016
GAN Process
GAN Process
GAN Process
GAN Process
GAN Process
GAN Architecture
Slide taken from :
https://www.slideshare.net/xavigiro/deep-learning-for-computer-vision-generative-models-and-adversarial-training-upc-2016
Loss Function - MiniMax Game
Loss Function - MiniMax Game
Mini-Max Game
What is the max value V(D, G) can take?
Mini-Max Game
What is the max value V(D, G) can take? 0
Mini-Max Game
What is the max value V(D, G) can take? 0
What is the min value V(D, G) can take?
Mini-Max Game
What is the max value V(D, G) can take? 0
What is the min value V(D, G) can take? -
8
Why D wants to maximize
Why D wants to maximize
training set
generated sample
Why D wants to maximize
training set
generated sample
Why G wants to Minimize
generated sample
Why G wants to Minimize
generated sample
Loss - In practice
Discriminator Generator
Given (x, 1) the loss is :
Given (x=G(z), 0) the loss is :
Given (G(z), 1) the loss is :
So basically we have cross entropy loss :
-
-
In practice we would maximize :
Game Equilibrium
log(x)
0.5
log(1-x)
GAN Recap
Gz
G(z)
x
D
D(x)
D(G(z)
Slide taken from Kevin Mcguinness -
https://www.slideshare.net/xavigiro/deep-learning-for-computer-vision-generative-models-and-adversarial-training-upc-2016
GAN Algorithm
Process
Generative Adversarial NetworksResults
Original GAN (2014)
Goodfellow et el. (2014)
DCGAN (2015)
Radford et el. (2014)
PPGN (2016)
Nguyen et el. (2016)
Progressive Growing of GANs (2018)
Karras et el. (2018)
Generative adversarial networksProblems
Non Convergence
Unlike traditional Deep Learning approaches GAN involve two players
● D is trying to maximize its reward● G is trying to minimize D’s reward
● SGD was not designed to find the NE of a game● Might not converge!
Non Convergence example
Non Convergence example
● State 1: [x > 0 | y > 0 | V > 0]
Increase y | Decrease x
Non Convergence example
● State 1: [x > 0 | y > 0 | V > 0]
Increase y | Decrease x
● State 2: [x < 0 | y > 0 | V < 0]
Decrease y | Decrease x
Non Convergence example
● State 1: [x > 0 | y > 0 | V > 0]
Increase y | Decrease x
● State 2: [x < 0 | y > 0 | V < 0]
Decrease y | Decrease x
● State 3: [x < 0 | y < 0 | V > 0]
Non Convergence example
● State 1: [x > 0 | y > 0 | V > 0]
Increase y | Decrease x
● State 2: [x < 0 | y > 0 | V < 0]
Decrease y | Decrease x
● State 3: [x < 0 | y < 0 | V > 0]
Non Convergence example
● State 1: [x > 0 | y > 0 | V > 0]
Increase y | Decrease x
● State 2: [x < 0 | y > 0 | V < 0]
Decrease y | Decrease x
● State 3: [x < 0 | y < 0 | V > 0]
Mode collapse
Mode collapse
Mode collapse
Progressive growing of GANs for improved quality, Stability and variation.Karras, Tero, et al. (2018)
Progressive Growing
Smooth Transition
Network architecture and learning
Network architecture and learning
Network architecture and learning
Network architecture and learning
Network architecture and more
Minibatch Standard Deviation
Results
Results
Results
Thank You!
References
● Metz, Luke, et al. "Unrolled Generative Adversarial Networks." arXiv preprint arXiv:1611.02163 (2016).
● Zhu, Jun-Yan, et al. "Unpaired image-to-image translation using cycle-consistent adversarial networks." arXiv preprint
arXiv:1703.10593 (2017).
● Karras, Tero, et al. "Progressive growing of gans for improved quality, stability, and variation." arXiv preprint arXiv:1710.10196
(2017).
● Goodfellow, Ian, et al. "Generative adversarial nets." Advances in neural information processing systems. 2014.