t-cvae: transformer-based conditioned variational
TRANSCRIPT
T-CVAE: Transformer-Based Conditioned Variational Autoencoder for Story Completion
Source : IJCAI 2019
Speaker : Ya-Fang, Hsiao
Advisor : Jia-Ling, Koh
Date : 2020/04/24
for Story Completion
My Dad loves chocolate chip cookies. _______________. I decided I would learn how to make them. I made my first batch the other day. My Dad was very surprised and quite happy!
My Mom doesn't like to make cookies because they take too long.
He has been making them all week.
He always ate them.
T-CVAE:
โ ๐ฅ = โKL ๐ ๐ง ๐ฅ ||๐(๐ง) + ๐ผ๐ ๐ง ๐ฅ [๐๐๐๐(๐ฅโฒ|๐ง)]
โ ๐ฅ, ๐ฅโฒ = โKL ๐ ๐ง ๐ฅ, ๐ ||๐(๐ง|๐ฅ) + ๐ผ๐ ๐ง ๐ฅ, ๐ [๐๐๐๐(๐ฅโฒ|๐, ๐ง)]
VAE:
CVAE:
T-CVAE:
โ ๐ฅ, ๐ฅโฒ = โKL ๐ ๐ง ๐ฅ, ๐ ||๐(๐ง|๐ฅ) + ๐ผ๐ ๐ง ๐ฅ, ๐ [๐๐๐๐(๐ฅโฒ|๐, ๐ง)]
๐ถ๐ก = tanh( ๐ง, ๐ท๐๐ข๐ก,๐ก๐ฟ ๐๐)
๐๐ก = ๐ถ๐ก๐๐ + ๐๐
Combination layer:
๐๐ก = ๐ ๐๐๐ก๐๐๐ฅ(๐๐ก)