Upload File

generative adversarial nets - semantic scholargenerative adversarial text to image synthesis. scott...

  • Home
  • Documents
  • Generative Adversarial Nets - Semantic Scholargenerative adversarial text to image synthesis. Scott Reed, ZeynepAkata. ... Conditional Sequence Generative Adversarial Nets[J]. arXiv
29
Generative Adversarial Nets 王文富 20170329

Upload: others

Post on 19-Jun-2020

11 views

Category:

Documents


0 download

Report

TRANSCRIPT

Page 1: Generative Adversarial Nets - Semantic Scholargenerative adversarial text to image synthesis. Scott Reed, ZeynepAkata. ... Conditional Sequence Generative Adversarial Nets[J]. arXiv

GenerativeAdversarialNets

王文富

20170329

Page 2: Generative Adversarial Nets - Semantic Scholargenerative adversarial text to image synthesis. Scott Reed, ZeynepAkata. ... Conditional Sequence Generative Adversarial Nets[J]. arXiv

“There are many interesting recent development in deep learning…The mostimportant one, in my opinion, is adversarial training (also called GAN forGenerative Adversarial Networks). This, and the variations that are nowbeing proposed is the most interesting idea in the last 10 years in ML, in myopinion.”

– Facebook, Yann LeCun

GoogleTrendsforGANs（2017/02/01）

研究&发展

Page 3: Generative Adversarial Nets - Semantic Scholargenerative adversarial text to image synthesis. Scott Reed, ZeynepAkata. ... Conditional Sequence Generative Adversarial Nets[J]. arXiv

生成式模型 通过观测数据，学习样本与标签的联合概率分布P(X, Y)，训练好的模型能够生成符合样本分布的新数据。可用于有监督学习和无监督学习。

有监督生成式模型-朴素贝叶斯（Naïve Bayes, NB）-高斯混合模型（Gaussian Mixture Model, GMM）-隐马尔科夫模型（Hidden Markov Model, HMM）。

无监督生成式模型深度生成式模型在无监督深度学习方面占据主要位置，它通过深层网络捕捉数据的内在特征，生成与训练样本相似的新数据。典型无监督生成式模型：-受限玻尔兹曼机（Restricted Boltzmann Machine, RBM）-深度信念网络（Deep Belief Network, DBN）-变分自编码器（Variational Autoencoders, VAE）-自回归模型（Autoregressive models）-生成对抗网络（Generative Adversarial Networks, GANs）

生成式模型

Page 4: Generative Adversarial Nets - Semantic Scholargenerative adversarial text to image synthesis. Scott Reed, ZeynepAkata. ... Conditional Sequence Generative Adversarial Nets[J]. arXiv

对抗网络/对抗训练

Page 5: Generative Adversarial Nets - Semantic Scholargenerative adversarial text to image synthesis. Scott Reed, ZeynepAkata. ... Conditional Sequence Generative Adversarial Nets[J]. arXiv

极小极大博弈

判别模型的训练目的是 大化自己的判别准确率:

生成模型的训练目的是 小化判别模型的判别准确率：

http://papers.nips.cc/paper/5423-generative-adversarial-nets IanJ.Goodfellow

判别器(D)、生成器(G)交替迭代学习，使得对方的错误 大化， 终G能估测出样本数据的分布。[Goodfellow2014]论文证明了这个极小化极大博弈当且仅当𝑃" =𝑃#$%$时存在 优解，即达到纳什均衡，此时生成模型G恢复了训练数据的分布，判别模型D的准确率等于50%。

Page 6: Generative Adversarial Nets - Semantic Scholargenerative adversarial text to image synthesis. Scott Reed, ZeynepAkata. ... Conditional Sequence Generative Adversarial Nets[J]. arXiv

极小极大博弈

http://papers.nips.cc/paper/5423-generative-adversarial-nets IanJ.Goodfellow

Page 7: Generative Adversarial Nets - Semantic Scholargenerative adversarial text to image synthesis. Scott Reed, ZeynepAkata. ... Conditional Sequence Generative Adversarial Nets[J]. arXiv

生成真实的、想要的数据

https://arxiv.org/pdf/1411.1784.pdfConditionalGenerativeAdversarialNets.MehdiMirza,SimonOsindero.

ConditionalGAN

Page 8: Generative Adversarial Nets - Semantic Scholargenerative adversarial text to image synthesis. Scott Reed, ZeynepAkata. ... Conditional Sequence Generative Adversarial Nets[J]. arXiv

生成真实的、想要的数据

https://arxiv.org/pdf/1411.1784.pdfConditionalGenerativeAdversarialNets.MehdiMirza,SimonOsindero.

ConditionalGAN

Page 9: Generative Adversarial Nets - Semantic Scholargenerative adversarial text to image synthesis. Scott Reed, ZeynepAkata. ... Conditional Sequence Generative Adversarial Nets[J]. arXiv

http://www.jmlr.org/proceedings/papers/v48/reed16.pdfgenerativeadversarialtexttoimagesynthesis.ScottReed,Zeynep Akata.

基于文本描述的图像生成

ConditionalGAN

Page 10: Generative Adversarial Nets - Semantic Scholargenerative adversarial text to image synthesis. Scott Reed, ZeynepAkata. ... Conditional Sequence Generative Adversarial Nets[J]. arXiv

三个优势 [OpenAI IanGoodfellow的Quora问答]：

-能更好建模数据分布（图像更锐利、清晰）。

-理论上，生成对抗式网络框架能训练任何一种生成器网络。其他的框架需要生成器网络有一些特定的函数形式，比如输出层是高斯的。

-无需利用马尔科夫链反复采样，无需在学习过程中进行推断（Inference），没有复杂的变分下界，避开近似计算棘手的概率的难题。

优势与缺陷

两个缺陷：

-难训练，不稳定。D与G之间需要很好的同步，但是在实际训练中很容易D收敛，G发散。D/G的训练需精心设计。

-模式缺失（ModeCollapse）问题。GAN的学习过程可能出现模式缺失，生成器开始退化，总是生成同样的样本点，无法继续学习。

Page 11: Generative Adversarial Nets - Semantic Scholargenerative adversarial text to image synthesis. Scott Reed, ZeynepAkata. ... Conditional Sequence Generative Adversarial Nets[J]. arXiv

模型结构：- AvoidSparseGradients:ReLU,MaxPool。使用LeakyReLU、AveragePooling、

Conv2d+stride- BatchNorm- DCGAN,（DeepConvolutionalGAN,应用 广）

训练策略：- UsetheADAMOptimizer- UseSoftandNoisyLabels：(0.8-1.2)forrealsample,(0-0.3)forfakesample- Addnoisetoinputs,decayovertime

……

稳定策略

https://github.com/soumith/ganhackshttp://www.inference.vc/instance-noise-a-trick-for-stabilising-gan-training/ImprovedTechniquesforTrainingGANs

没有从原理上分析为什么GAN难训练，也没有给出根本的解决方案

Page 12: Generative Adversarial Nets - Semantic Scholargenerative adversarial text to image synthesis. Scott Reed, ZeynepAkata. ... Conditional Sequence Generative Adversarial Nets[J]. arXiv

WhyhardtotrainGAN

• 原始形式GAN(公式2)的问题：判别器训练的太好，生成器梯度消失。判别器训练的不好，生成器梯度不准，四处乱跑。只有判别器训练得不好不坏才行。但是这个火候很难把握。

• -logDtrickGAN(公式3)的问题：小化第二种生成器loss函数，会等价于 小化一个不合理的距离度量，导致

两个问题，一是梯度不稳定，二是collapsemode即多样性不足。

https://arxiv.org/pdf/1701.07875v1.pdf Wasserterin Gan.https://openreview.net/pdf?id=Hk4_qw5xe TowardsPrincipledmethodsfortraininggenerativeadversarialnetworks

WGAN前作:TowardsPrincipledmethodsfortraininggenerativeadversarialnetworks. MartinArjovsky

Page 13: Generative Adversarial Nets - Semantic Scholargenerative adversarial text to image synthesis. Scott Reed, ZeynepAkata. ... Conditional Sequence Generative Adversarial Nets[J]. arXiv

WhyhardtotrainGAN

原始形式GAN的问题（从生成器的等价loss入手）

https://arxiv.org/pdf/1701.07875v1.pdf Wasserterin Gan.https://openreview.net/pdf?id=Hk4_qw5xe TowardsPrincipledmethodsfortraininggenerativeadversarialnetworks

WGAN前作:TowardsPrincipledmethodsfortraininggenerativeadversarialnetworks. MartinArjovsky

给定任意一个样本x，它可能来自真实分布也可能来自生成分布，它对公示1损失函数的贡献是：

对D(x)求导，得到 优判别器：

Page 14: Generative Adversarial Nets - Semantic Scholargenerative adversarial text to image synthesis. Scott Reed, ZeynepAkata. ... Conditional Sequence Generative Adversarial Nets[J]. arXiv

WhyhardtotrainGAN

原始形式GAN的问题（从生成器的等价loss入手）

https://arxiv.org/pdf/1701.07875v1.pdf Wasserterin Gan.https://openreview.net/pdf?id=Hk4_qw5xe TowardsPrincipledmethodsfortraininggenerativeadversarialnetworks

WGAN前作:TowardsPrincipledmethodsfortraininggenerativeadversarialnetworks. MartinArjovsky

小化公式2等价于 小化：

代入公式4，变换得到：

Page 15: Generative Adversarial Nets - Semantic Scholargenerative adversarial text to image synthesis. Scott Reed, ZeynepAkata. ... Conditional Sequence Generative Adversarial Nets[J]. arXiv

WhyhardtotrainGAN

原始形式GAN的问题（从生成器的等价loss入手）

https://arxiv.org/pdf/1701.07875v1.pdf Wasserterin Gan.https://openreview.net/pdf?id=Hk4_qw5xe TowardsPrincipledmethodsfortraininggenerativeadversarialnetworks

WGAN前作:TowardsPrincipledmethodsfortraininggenerativeadversarialnetworks. MartinArjovsky

后，公式5变换为：

KL,JS散度定义：

Page 16: Generative Adversarial Nets - Semantic Scholargenerative adversarial text to image synthesis. Scott Reed, ZeynepAkata. ... Conditional Sequence Generative Adversarial Nets[J]. arXiv

WhyhardtotrainGAN

原始形式GAN的问题（从生成器的等价loss入手）

https://arxiv.org/pdf/1701.07875v1.pdf Wasserterin Gan.https://openreview.net/pdf?id=Hk4_qw5xe TowardsPrincipledmethodsfortraininggenerativeadversarialnetworks

WGAN前作:TowardsPrincipledmethodsfortraininggenerativeadversarialnetworks. MartinArjovsky

作者从测度论、拓扑学证明：Pr与Pg几乎不可能有不可忽略的重叠，所以无论它们相距多远，JS散度是常数log2。因此公式8（生成器loss）为常数，梯度为0。

一个训练trick：就是别把判别器训练得太好，否则在实验中生成器会完全学不动（loss降不下去）

Page 17: Generative Adversarial Nets - Semantic Scholargenerative adversarial text to image synthesis. Scott Reed, ZeynepAkata. ... Conditional Sequence Generative Adversarial Nets[J]. arXiv

WhyhardtotrainGAN

-logDtrickGAN的问题（从生成器的等价loss入手）

https://arxiv.org/pdf/1701.07875v1.pdf Wasserterin Gan.https://openreview.net/pdf?id=Hk4_qw5xe TowardsPrincipledmethodsfortraininggenerativeadversarialnetworks

WGAN前作:TowardsPrincipledmethodsfortraininggenerativeadversarialnetworks. MartinArjovsky

小化公式3等价于 小化

这个等价 小化目标存在两个严重的问题。第一是它同时要 小化生成分布与真实分布的KL散度，却又要 大化两者的JS散度，一个要拉近，一个却要推远！这在直观上非常荒谬，在数值上则会导致梯度不稳定太好，容易出现mode collapse。

Page 18: Generative Adversarial Nets - Semantic Scholargenerative adversarial text to image synthesis. Scott Reed, ZeynepAkata. ... Conditional Sequence Generative Adversarial Nets[J]. arXiv

WGAN

https://arxiv.org/pdf/1701.07875v1.pdf Wasserterin Gan.https://openreview.net/pdf?id=Hk4_qw5xe TowardsPrincipledmethodsfortraininggenerativeadversarialnetworks

WGAN本作:WassersteinGAN. MartinArjovsky

• 解决GAN训练不稳定的问题，不再需要小心平衡生成器和判别器的训练程度

• 基本解决了collapse mode的问题，确保了生成样本的多样性

• 训练过程中终于有一个像交叉熵、准确率这样的数值来指示训练的进程，这个数值越小代表GAN训练得越好，代表生成器产生的图像质量越高

• 不需要精心设计的网络架构， 简单的多层全连接网络就可以做到

优势

Page 19: Generative Adversarial Nets - Semantic Scholargenerative adversarial text to image synthesis. Scott Reed, ZeynepAkata. ... Conditional Sequence Generative Adversarial Nets[J]. arXiv

WGAN

https://arxiv.org/pdf/1701.07875v1.pdf Wasserterin Gan.https://openreview.net/pdf?id=Hk4_qw5xe TowardsPrincipledmethodsfortraininggenerativeadversarialnetworks

WGAN本作:WassersteinGAN. MartinArjovsky

• 判别器 后一层去掉sigmoid

• 生成器和判别器的loss不取log

• 每次更新判别器的参数之后把它们的绝对值截断到不超过一个固定常数c

• 不要用基于动量的优化算法（包括momentum和Adam），推荐RMSProp，SGD也行

具体实现

Page 20: Generative Adversarial Nets - Semantic Scholargenerative adversarial text to image synthesis. Scott Reed, ZeynepAkata. ... Conditional Sequence Generative Adversarial Nets[J]. arXiv

l图像领域

1图像超分辨率（Twitter,Ledigetal.）

GAN的应用

LedigC,TheisL,HuszarF,CaballeroJ,CunninghamA,AcostaA,AitkenA,TejaniA,TotzJ,WangZH,ShiWZ.Photo-realisticsingleimagesuper-resolutionusingagenerativeadversarialnetwork.arXivpreprintarXiv:1609.04802,2016

Page 21: Generative Adversarial Nets - Semantic Scholargenerative adversarial text to image synthesis. Scott Reed, ZeynepAkata. ... Conditional Sequence Generative Adversarial Nets[J]. arXiv

GAN的应用

LedigC,TheisL,HuszarF,CaballeroJ,CunninghamA,AcostaA,AitkenA,TejaniA,TotzJ,WangZH,ShiWZ.Photo-realisticsingleimagesuper-resolutionusingagenerativeadversarialnetwork.arXivpreprintarXiv:1609.04802,2016

l图像领域

1图像超分辨率（Twitter,Ledigetal.）

Page 22: Generative Adversarial Nets - Semantic Scholargenerative adversarial text to image synthesis. Scott Reed, ZeynepAkata. ... Conditional Sequence Generative Adversarial Nets[J]. arXiv

GAN的应用

2 图像仿真，人眼检测（Apple,Gouetal.）

GouC,WuY,WangK,WangKF,WangFY,JiQ.Ajointcascadedframeworkforsimultaneouseyedetectionandeyestateestimation.PatternRecognition,2017,67:23-31

l图像领域

Page 23: Generative Adversarial Nets - Semantic Scholargenerative adversarial text to image synthesis. Scott Reed, ZeynepAkata. ... Conditional Sequence Generative Adversarial Nets[J]. arXiv

GAN的应用

IsolaP,ZhuJY,ZhouT,etal.Image-to-imagetranslationwithconditionaladversarialnetworks[J].arXivpreprintarXiv:1611.07004,2016.

l图像领域

3图像到图像翻译

Page 24: Generative Adversarial Nets - Semantic Scholargenerative adversarial text to image synthesis. Scott Reed, ZeynepAkata. ... Conditional Sequence Generative Adversarial Nets[J]. arXiv

1诗歌序列生成

GAN的应用

https://arxiv.org/abs/1609.05473 Sequencegenerativeadversarialnetswithpolicygradient.AAAI2017.https://arxiv.org/pdf/1701.06547.pdf LiJ,MonroeW,ShiT,etal.AdversarialLearningforNeuralDialogueGeneration[J].arXivpreprintarXiv:1701.06547,2017.https://arxiv.org/abs/1703.04887 YangZ,ChenW,WangF,XuB.ImprovingNeuralMachineTranslationwithConditionalSequenceGenerativeAdversarialNets[J].arXivpreprintarXiv:1703.04887,2017.

lNLP领域

2对话生成

3机器翻译（+2BLEU）

Page 25: Generative Adversarial Nets - Semantic Scholargenerative adversarial text to image synthesis. Scott Reed, ZeynepAkata. ... Conditional Sequence Generative Adversarial Nets[J]. arXiv

GAN的应用

GENERATIVEADVERSARIALNETWORK-BASEDPOSTFILTERFORSTATISTICALPARAMETRICSPEECHSYNTHESIS.ICASSP2017

l语音合成

TTS后处理

Page 26: Generative Adversarial Nets - Semantic Scholargenerative adversarial text to image synthesis. Scott Reed, ZeynepAkata. ... Conditional Sequence Generative Adversarial Nets[J]. arXiv

GAN的应用

GENERATIVEADVERSARIALNETWORK-BASEDPOSTFILTERFORSTATISTICALPARAMETRICSPEECHSYNTHESIS.ICASSP2017

l语音合成

TTS后处理

Page 27: Generative Adversarial Nets - Semantic Scholargenerative adversarial text to image synthesis. Scott Reed, ZeynepAkata. ... Conditional Sequence Generative Adversarial Nets[J]. arXiv

GAN的应用

GENERATIVEADVERSARIALNETWORK-BASEDPOSTFILTERFORSTATISTICALPARAMETRICSPEECHSYNTHESIS.ICASSP2017

l语音合成

TTS后处理

Page 28: Generative Adversarial Nets - Semantic Scholargenerative adversarial text to image synthesis. Scott Reed, ZeynepAkata. ... Conditional Sequence Generative Adversarial Nets[J]. arXiv

GAN的应用

l其他领域

增强学习等

Page 29: Generative Adversarial Nets - Semantic Scholargenerative adversarial text to image synthesis. Scott Reed, ZeynepAkata. ... Conditional Sequence Generative Adversarial Nets[J]. arXiv

Generative adversarial networks...Generative adversarial nets. NeurIPS. Longer version Goodfellow, (2016). NIPS 2016 Tutorial: Generative Adversarial Networks. Extra Radford et al,

Synthesizing Designs with Inter-Part Dependencies using ...ideal.umd.edu/assets/pdfs/chen_hgan_jmd_2019.pdf · Generative adversarial nets [7] model a game between a generative model

Generative Adversarial Nets - Ehsan Hoque … · InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets Slightly modified original GAN--instead

Extra Domain Data Generation with Generative Adversarial Netsmwiering/GROUP/ARTICLES/... · Extra Domain Data Generation with Generative Adversarial Nets Luuk Boulogne Bernoulli Institute

GAN-OPC: Mask Optimization with Lithography-guided Generative … · 2018. 6. 29. · GAN-OPC: Mask Optimization with Lithography-guided Generative Adversarial Nets Haoyu Yang, Shuhe

Face Aging with Contextual Generative Adversarial Netsliusi-group.com/pdf/faceaging-acmmm2017.pdf · Face Aging with Contextual Generative Adversarial Nets MM ’17, October 23–27,

Generative Adversarial Networksbhiksha/courses/deeplearning/Spring.2018/... · 2018. 3. 29. · Generative Adversarial Networks Generative adversarial networks (GANs) are relatively

Semi-supervised Learning on Graphs with Generative ...Graphs with Generative Adversarial Nets . In The 27th ACM International Conference on Information and Knowledge Management (CIKM

Autoencoders and Generative Adversarial Nets€¦ · Autoencoders and Generative Adversarial Nets Chapter 1 [ 5 ] Fixing corrupted data with denoising autoencoders The autoencoders

Generative Adversarial Nets: Applications and Extensionsice.dlut.edu.cn/valse2018/ppt/Generative... · Generative Adversarial Nets: Applications and Extensions Wangmeng Zuo Vision

Lipschitz Generative Adversarial Nets11-14-00)-11-15-10-4628-lipschitz_gener.pdf · Zhiming Zhou Lipschitz Generative Adversarial Nets ICML, 2019 2 / 11. The Gradient Uninformativeness

Generative Adversarial Perturbations - arXiv

Generative Adversarial Nets - courses.cs.washington.edu

Dual Discriminator Generative Adversarial Nets · Dual Discriminator Generative Adversarial Nets Tu Dinh Nguyen, Trung Le, Hung Vu, Dinh Phung Deakin University, Geelong, Australia

Generative Adversarial Networks (GANs)

InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets

Brief overview of neural networks Generative …V11: Generative Modeling with Neural Nets Brief overview of neural networks Generative Adversarial Nets Use case: image inpainting With

Lecture 9: Unsupervised, Generative & Adversarial Networks · Lecture 9: Unsupervised, Generative & Adversarial Networks ... GENERATIVE & ADVERSARIAL NETWORKS - 20 o Autoencoders

Generative Adversarial Networks (GANs) - Sciencesconf.org · Generative Adversarial Networks (GANs) Anirban Mukhopadhyay TU Darmstadt, Germany. Introduction Generative vs. Discriminative

Adversarial Generative Nets: Neural Network Attacks on ... · PDF fileAdversarial Generative Nets: Neural Network Attacks on State-of-the-Art Face Recognition Mahmood Sharif, Sruti

Generative Adversarial Imitation Learning

Conditional generative adversarial nets for convolutional ... · Conditional generative adversarial nets for convolutional face generation Jon Gauthier Symbolic Systems Program, Natural

Generative Adversarial Networkstat.snu.ac.kr/mcp/Lecture10_GAN.pdf · 2019-11-11 · Generative Adversarial Network Generative adversarial network: Setup Let y = 1 if the data is

Generative Adversarial Networks - HAW Hamburgubicomp/projekte/master... · Generative Adversarial Networks ... Outline 1. Basic idea 2. Generative Adversarial ... com/2017/01/generative-adversarial-networks-hot-topic-machine-learning

EmotiGAN: Emoji Art using Generative Adversarial Networkscs229.stanford.edu/proj2017/final-reports/5244346.pdfA. Generative Adversarial Networks A Generative Adversarial Network[4]

1 An Introduction to Image Synthesis with Generative Adversarial … · 2018. 11. 20. · 1 An Introduction to Image Synthesis with Generative Adversarial Nets He Huang, Philip S

Generative Adversarial Transformers

Generative adversarial nets

Generative Adversarial Networks, and Applications Adversarial Networks, and Applications Outline: •Generative Models vs Discriminative Models (Background) •Generative Adversarial

Generative Adversarial Nets - ice.dlut.edu.cnice.dlut.edu.cn/valse2018/ppt/Generative_Adversarial_Nets_JSFeng.pdf · Generative Adversarial Nets 生成对抗网络 冯佳时. 机器学习与视觉实验室

Generative Adversarial Networks (part 2)deeplearning.cs.cmu.edu/document/slides/bstriner_gans_part_2.pdf · Unrolled Generative Adversarial Networks Unrolled Generative Adversarial

Generative Adversarial Nets - cosy.sbg.ac.atheld/teaching/wiss_arbeiten/slides_19-2… · paper "Generative Adversarial Networks" from Ian Goodfellow et al. (2014) [2]. Schuiki Johannes,

Face Aging with Contextual Generative Adversarial Nets · Face Aging with Contextual Generative Adversarial Nets MM ’17, October 23–27, 2017, Mountain View, CA, USA lïvï í

Adversarial Distillation of Bayesian Neural Network Posteriorsproceedings.mlr.press/v80/wang18i/wang18i.pdf · 2.3. Generative Adversarial Networks Generative Adversarial Networks

Triple Generative Adversarial Netspapers.nips.cc/paper/6997-triple-generative-adversarial-nets.pdf · the labels. To address the problems, we present triple generative adversarial