generative adversarial networks and their applicationsshapiro/ee596/notes/bindita19.pdfgenerative...
TRANSCRIPT
![Page 1: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:](https://reader034.vdocuments.net/reader034/viewer/2022042220/5ec5f5be90ca1d693c7061c5/html5/thumbnails/1.jpg)
Generative Adversarial Networks
And Their Applications
Bindita Chaudhuri
![Page 2: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:](https://reader034.vdocuments.net/reader034/viewer/2022042220/5ec5f5be90ca1d693c7061c5/html5/thumbnails/2.jpg)
Unsupervised Learning: Autoencoders
![Page 3: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:](https://reader034.vdocuments.net/reader034/viewer/2022042220/5ec5f5be90ca1d693c7061c5/html5/thumbnails/3.jpg)
Unsupervised Learning: Autoencoders
![Page 4: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:](https://reader034.vdocuments.net/reader034/viewer/2022042220/5ec5f5be90ca1d693c7061c5/html5/thumbnails/4.jpg)
Unsupervised Learning: Variational Autoencoders
![Page 5: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:](https://reader034.vdocuments.net/reader034/viewer/2022042220/5ec5f5be90ca1d693c7061c5/html5/thumbnails/5.jpg)
Generative Adversarial Networks: Idea
![Page 6: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:](https://reader034.vdocuments.net/reader034/viewer/2022042220/5ec5f5be90ca1d693c7061c5/html5/thumbnails/6.jpg)
Generative Adversarial Networks: Idea
![Page 7: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:](https://reader034.vdocuments.net/reader034/viewer/2022042220/5ec5f5be90ca1d693c7061c5/html5/thumbnails/7.jpg)
Generative Adversarial Networks
![Page 8: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:](https://reader034.vdocuments.net/reader034/viewer/2022042220/5ec5f5be90ca1d693c7061c5/html5/thumbnails/8.jpg)
Generative Adversarial Networks
![Page 9: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:](https://reader034.vdocuments.net/reader034/viewer/2022042220/5ec5f5be90ca1d693c7061c5/html5/thumbnails/9.jpg)
Distributions during training
![Page 10: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:](https://reader034.vdocuments.net/reader034/viewer/2022042220/5ec5f5be90ca1d693c7061c5/html5/thumbnails/10.jpg)
GAN: Sample Architecture (DC-GAN)
![Page 11: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:](https://reader034.vdocuments.net/reader034/viewer/2022042220/5ec5f5be90ca1d693c7061c5/html5/thumbnails/11.jpg)
Generated Samples
MNIST Faces CIFAR 10
![Page 12: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:](https://reader034.vdocuments.net/reader034/viewer/2022042220/5ec5f5be90ca1d693c7061c5/html5/thumbnails/12.jpg)
Bidirectional GAN (BiGAN)
![Page 13: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:](https://reader034.vdocuments.net/reader034/viewer/2022042220/5ec5f5be90ca1d693c7061c5/html5/thumbnails/13.jpg)
Conditional GAN (cGAN)
Generator Discriminator
![Page 14: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:](https://reader034.vdocuments.net/reader034/viewer/2022042220/5ec5f5be90ca1d693c7061c5/html5/thumbnails/14.jpg)
Conditional GAN (cGAN)
Generator Discriminator
![Page 15: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:](https://reader034.vdocuments.net/reader034/viewer/2022042220/5ec5f5be90ca1d693c7061c5/html5/thumbnails/15.jpg)
Pix2Pix: Type of cGAN
L1 loss
![Page 16: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:](https://reader034.vdocuments.net/reader034/viewer/2022042220/5ec5f5be90ca1d693c7061c5/html5/thumbnails/16.jpg)
CycleGAN: Unsupervised Pix2Pix
![Page 17: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:](https://reader034.vdocuments.net/reader034/viewer/2022042220/5ec5f5be90ca1d693c7061c5/html5/thumbnails/17.jpg)
CycleGAN: Unsupervised Pix2Pix
![Page 18: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:](https://reader034.vdocuments.net/reader034/viewer/2022042220/5ec5f5be90ca1d693c7061c5/html5/thumbnails/18.jpg)
CycleGAN Results
![Page 19: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:](https://reader034.vdocuments.net/reader034/viewer/2022042220/5ec5f5be90ca1d693c7061c5/html5/thumbnails/19.jpg)
Progressive Growing of GANs
![Page 20: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:](https://reader034.vdocuments.net/reader034/viewer/2022042220/5ec5f5be90ca1d693c7061c5/html5/thumbnails/20.jpg)
Progressive GAN Results
Celebrities Bedrooms Objects
![Page 21: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:](https://reader034.vdocuments.net/reader034/viewer/2022042220/5ec5f5be90ca1d693c7061c5/html5/thumbnails/21.jpg)
Application: Neural Style transfer
![Page 22: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:](https://reader034.vdocuments.net/reader034/viewer/2022042220/5ec5f5be90ca1d693c7061c5/html5/thumbnails/22.jpg)
Application: 3D GAN
![Page 23: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:](https://reader034.vdocuments.net/reader034/viewer/2022042220/5ec5f5be90ca1d693c7061c5/html5/thumbnails/23.jpg)
My Project: Facial Motion Retargeting
![Page 24: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:](https://reader034.vdocuments.net/reader034/viewer/2022042220/5ec5f5be90ca1d693c7061c5/html5/thumbnails/24.jpg)
My Project: Facial Motion Retargeting
Network
![Page 25: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:](https://reader034.vdocuments.net/reader034/viewer/2022042220/5ec5f5be90ca1d693c7061c5/html5/thumbnails/25.jpg)
• Compute facial landmarks:
2D-to-3D CycleGAN
![Page 26: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:](https://reader034.vdocuments.net/reader034/viewer/2022042220/5ec5f5be90ca1d693c7061c5/html5/thumbnails/26.jpg)
• Compute facial landmarks:
• Convert 3D model to 2D position map:
2D-to-3D CycleGAN
![Page 27: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:](https://reader034.vdocuments.net/reader034/viewer/2022042220/5ec5f5be90ca1d693c7061c5/html5/thumbnails/27.jpg)
3D discriminator
landmark loss
• Compute facial landmarks:
• Convert 3D model to 2D position map:
• Train CycleGAN:
generator A→B
generator B→A
2D discriminator
2D-to-3D CycleGAN
![Page 28: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:](https://reader034.vdocuments.net/reader034/viewer/2022042220/5ec5f5be90ca1d693c7061c5/html5/thumbnails/28.jpg)
Input Landmark only CycleGAN
Results
![Page 29: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:](https://reader034.vdocuments.net/reader034/viewer/2022042220/5ec5f5be90ca1d693c7061c5/html5/thumbnails/29.jpg)
Input Landmark only CycleGAN
Results
![Page 30: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:](https://reader034.vdocuments.net/reader034/viewer/2022042220/5ec5f5be90ca1d693c7061c5/html5/thumbnails/30.jpg)
Useful links
• GAN Zoo: https://github.com/hindupuravinash/the-gan-zoo
• GAN hacks: https://github.com/soumith/ganhacks
• Code Bases:• Tensorflow: https://www.tensorflow.org/tutorials/generative/dcgan• Keras: https://github.com/eriklindernoren/Keras-GAN• Pytorch: https://github.com/pytorch/examples/tree/master/dcgan
• References:• http://cs231n.stanford.edu/slides/2017/cs231n_2017_lecture13.pdf• https://www.cs.toronto.edu/~rgrosse/courses/csc321_2018/slides/lec19.pdf• https://www.cs.cmu.edu/~bhiksha/courses/deeplearning/Fall.2015/slides/lec13.GAN.pdf
![Page 31: Generative Adversarial Networks And Their Applicationsshapiro/EE596/notes/Bindita19.pdfGenerative Adversarial Networks And Their Applications Bindita Chaudhuri. ... Unsupervised Learning:](https://reader034.vdocuments.net/reader034/viewer/2022042220/5ec5f5be90ca1d693c7061c5/html5/thumbnails/31.jpg)