infogan : interpretable representation learning by information ... · infogan : interpretable...

InfoGAN : Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets

ISL Lab Seminar Hansol Kang

: Mutual Information

Contents

Review

InfoGAN

Experiment

Summary

2019-04-09

2

I. Review

ReviewVanilla GAN, DCGAN

Review• Concept of GAN

2019-04-09

4

VsD GF1F1

F1

F1

FakeR1

Review• Concept of GAN

2019-04-09

5

VsD GFake?R1

@

F1

Review• Adversarial nets

2019-04-09

6

)))]((1[log()]([log),(maxmin )(~)(~ zGDExDEGDV zpzxpxDG zdata−+=

Smart D

)))]((1[log()]([log )(~)(~ zGDExDE zpzxpx zdata−+Real case

Fake case

1

)))]((1[log()]([log )(~)(~ zGDExDE zpzxpx zdata−+

0

should be 0

should be 0

1

Log(x)cf.

Stupid D

)))]((1[log()]([log )(~)(~ zGDExDE zpzxpx zdata−+Real case

Fake case

0


1

should be negative infinity

should be negative infinityD perspective, it should be maximum.

Review• Adversarial nets

2019-04-09

7

)))]((1[log()]([log),(maxmin )(~)(~ zGDExDEGDV zpzxpxDG zdata−+=

Generator


1

should be negative infinity1

Log(x)cf.

G perspective, it should be minimum.

Smart G

Stupid G )))]((1[log()]([log )(~)(~ zGDExDE zpzxpx zdata−+

0

should be 0

Review2019-04-09

8

• GAN

1) Global Optimality of datag pp =

2) Convergence of Algorithm

D GVs

x

)(xpdata

“Generative Adversarial Networks”

Goal Method

Review2019-04-09

9

• DCGAN : network

DG

“쟤들뭐하냐?”

“CNN이 MLP보다훨씬낫지롱”D“우리가짱이야.” G

Vanilla GAN DCGAN

Review2019-04-09

10

• DCGAN : latent space

0 1

0.1

0.15

0.18

0.143

0.5

0.45

0.47

0.473

0.9

0.95

0.96

0.9370.607±

II. InfoGAN

InfoGANConcept, Mutual Information, Variational method, Results

InfoGAN2019-04-09

12

• Concept

D

GNoise

Real(Kurt Cobain)

Fake(Not Kurt Cobain)

…

Latent code

Dataset

InfoGAN2019-04-09

13

• Concept

D

GNoise

Real(Kurt Cobain)


…

Latent code

Dataset

0

Simplify

InfoGAN2019-04-09

14

• Concept

D

GNoise

Real(Kurt Cobain)


…

Latent code

Dataset

0.5

Simplify

InfoGAN2019-04-09

15

• Concept

D

GNoise

Real(Kurt Cobain)


…

Latent code

Dataset

1

Simplify

InfoGAN2019-04-09

16

• Concept

D

GNoise

Real(Kurt Cobain)


…

Latent code

Dataset

0.0010.0081.0000.007

…0.005

But…

? : 실제 latent code의 구조는 복잡하여해석이어려움(entangled).

InfoGAN2019-04-09

17

• Concept

GNoise

Latent code

0.0010.0081.0000.007

…0.005

? : 실제 latent code의 구조는 복잡하여해석이어려움(entangled).

Let's make the latent code simple.

The proper generation is difficult.

[0.001, 0.008, …, 005] [005]

Latent code

0.0010.0081.0000.007

…0.005

0000…1

Z C

Z C : Condition

How about adding latent code?Idea

InfoGAN2019-04-09

18

• Concept

GLatent code

Z C

“뭐야? 그러면 C를 Z 옆에바로붙이면되는거야?”

[0.001, 0.008, …, 005 | 0, 0, … 1]

z c[0.001, 0.008, …, 005 | 1, 0, … 0]

z c

[0.001, 0.008, …, 005 ]

z[0.001, 0.008, …, 005 ]

z

Ignore the additional latent code c

Cost function을수정하여 c의영향을만듦.),(maxmin GDVDG

(Mutual Information)

InfoGAN2019-04-09

19

• Mutual Information

)|()();( YXHXHYXI −=)()()();(

YPXPYXPYXI

=

ISL Browserhttps://isl.cnu.ac.kr

Supervised Learning

검색결과약 107,000,000개

Unsupervised Learning

검색결과약 13,400,000개

Clustering

검색결과약 40,900,000개

Supervised Learning Clustering

검색결과약 25,300,000개

Unsupervised Learning Clustering

검색결과약 7,770,000개

Deep Learning

검색결과약 1,380,000,000개

07754.0)( =SLP

00971.0)( =ULP

02964.0)( =CP

0.01833)( =CSLP

0.00563)( =CULP

97551.702964.007754.0

01833.0)()()(

=×

=CPSLPCSLP

56190.1902964.000971.0

00563.0)()()(

=×

=CPULPCULP

“두사건사이의연관성파악”

InfoGAN2019-04-09

20

• Mutual Information

: Generator와 c 사이의연관성을 cost로정의( )),(;),(),(maxmin czGcIGDVGDVIDGλ−=

Maximize

Hard to maximize directly as it requires access to the posterior )|( xcP

( )[ ] ( ))(||)|()(|log),,( )|( zpxzqKLzgxExL xzq φθφθφ +−=

),,(min xL θφReconstruction Error Regularization

VAE Seminar (18.07.23)

InfoGAN2019-04-09

21

• Variational method

)|( xcP )|( xcQIntractable(Very complicated) Tractable(e.g Gaussian)

( )),(;),(),(maxmin czGcIGDVGDVIDGλ−=

( ) ( )),(|)(),(; czGcHcHczGcI −=

( ) ( ) ( ) ),(),(|ln),(,),(| czdcdGczGcPczGcPczGcH ∫∫−=

( ) dydxxyPxyPxyH )|(ln),(| ∫∫−=

dydxxyPxPxyP )|(ln)()|(∫∫−=

c.f Conditional Entropy

Product rule

( ) ( ) ( ) ),(),(|ln),(),(| czdcdGczGcPczGPczGcP∫∫−=

( ) ( ) ( ) ),(),(|ln),(|),( czdcdGczGcPczGcPczGP∫∫−=

( ) [ ] ),()|'(ln),( )|(~' czdGxcPEczGP xcPc∫−=[ ][ ])|'(ln)|(~'),(~ xcPEE xcPcczGx−= ( ) [ ][ ])|'(ln)(),(; )|(~'),(~ xcPEEcHczGcI xcPcczGx+=

(1)

(2)

InfoGAN2019-04-09

22


( ) [ ][ ])|'(ln)(),(; )|(~'),(~ xcPEEcHczGcI xcPcczGx+= (2)

( )

=

)'()|'(ln)|'(||)|'( )|(~' xcQ

xcPExcQxcPD xcPcKL

)|'(ln)'(ln )|(~')|(~' xcQExcPE xcPcxcPc −=

( ) )|'(ln)|'(||)|'()'(ln )|(~')|(~' xcQExcQxcPDxcPE xcPcKLxcPc +=

( ) ( )[ ])|'(ln)|'(||)|'()(),(; )|(~'),(~ xcQExcQxcPDEcHczGcI xcPcKLczGx ++= (3)0≥

( )

=

)()(ln|| ~ xQ

xPEQPD PxKL

c.f KL Divergence

( ) 0|| =QPDKL : 동일분포

[ ])|'(ln)( )|(~'),(~ xcQEEcH xcPcczGx+≥ (4)

)|( xcQ Tractable distribution

InfoGAN2019-04-09

24

• Results

InfoGAN2019-04-09

25

• Results

InfoGAN2019-04-09

26

• Results

InfoGAN2019-04-09

27

• Results

III. Experiment

ExperimentMNIST, FashionMNIST, LSUN

Experiment• Results#1 MNIST (continuous)

2019-04-09

29

Epoch 1 Epoch 5 Epoch 10

Epoch 30 Epoch 50 GIF

Experiment• Results#1 MNIST (categorical)

2019-04-09

30



Experiment• Results#2 Fashion MNIST (continuous)

2019-04-09

31



Experiment• Results#2 Fashion MNIST (categorical)

2019-04-09

32



Experiment• Results#3 LSUN (continuous)

2019-04-09

33


Epoch 4 Epoch 5

Experiment• Results#3 LSUN (categorical)

2019-04-09

34


Epoch 4 Epoch 5

Experiment• Results#3 LSUN (categorical, ep 5)

2019-04-09

35

IV. Summary

SummarySummary, Future Work

Summary2019-04-09

37

• Latent code에추가적인 code를할당하여학습함.

• 기존의 GAN 학습법으로는추가된 code를무시하기에새로운학습방법이필요함.

• Mutual information을통해추가된 code와네트워크간의상호연관성을부여함.

• 주어진 code는그형태에따라 categorical(discrete)or continuous로구분되며, 실제실험을통해적절히학습되는것을확인함.

(cGAN과비슷한접근법)

Future work2019-04-09

38

GAN Research

Vanilla GAN

DCGAN

InfoGAN

LS GAN

BEGAN

Pix2Pix

Cycle GAN

Novel GAN(about depth)

Tools

Document

Programming

PyTorch

Python executable & UI

I Know What You Did Last Faculty

C++ Coding Standard

Mathematical theory

LSM applications

Other Research

Level Processor

Ice Propagation

infogan : interpretable representation learning by information ... · infogan : interpretable...

Documents