variational autoencoders with jointly optimized …mori/research/papers/he-iclr19.pdf ·...

Published as a conference paper at ICLR 2019

VARIATIONAL AUTOENCODERS WITH JOINTLYOPTIMIZED LATENT DEPENDENCY STRUCTURE

Jiawei He1∗ & Yu Gong1∗

{jha203, gongyug}@sfu.caJoseph Marino2

[email protected]

Greg [email protected]

Andreas M. [email protected]

1School of Computing Science, Simon Fraser UniversityBurnaby, BC, V5B1Z1, Canada

2California Institute of TechnologyPasadena, CA, 91125, USA

ABSTRACT

We propose a method for learning the dependency structure between latent vari-ables in deep latent variable models. Our general modeling and inference frame-work combines the complementary strengths of deep generative models and prob-abilistic graphical models. In particular, we express the latent variable space ofa variational autoencoder (VAE) in terms of a Bayesian network with a learned,flexible dependency structure. The network parameters, variational parametersas well as the latent topology are optimized simultaneously with a single objec-tive. Inference is formulated via a sampling procedure that produces expectationsover latent variable structures and incorporates top-down and bottom-up reasoningover latent variable values. We validate our framework in extensive experimentson MNIST, Omniglot, and CIFAR-10. Comparisons to state-of-the-art structuredvariational autoencoder baselines show improvements in terms of the expressive-ness of the learned model.

1 INTRODUCTION

Deep latent variable models offer an effective method for automatically learning structure fromdata. By explicitly modeling the data distribution using latent variables, these models are capableof learning compressed representations that are then relevant for downstream tasks. Such modelshave been applied across a wide array of domains, such as images (Gregor et al., 2014; Kingma &Welling, 2014; Rezende et al., 2014; Gregor et al., 2015), audio (Chung et al., 2015; Fraccaro et al.,2016), video (He et al., 2018; Yingzhen & Mandt, 2018), and text (Bowman et al., 2016; Krishnanet al., 2017). However, despite their success, latent variable models are often formulated with simple(e.g. Gaussian) distributions, making independence assumptions among the latent variables. That is,each latent variable is sampled independently. Ignoring the dependencies between latent variableslimits the flexibility of these models, negatively impacting the model’s ability to fit the data.

In general, structural dependencies can be incorporated into all phases of a forward process, includ-ing inference, latent model, and output space: Normalizing flows (Rezende & Mohamed, 2015),for instance, accounts for dependencies during inference by learning a mapping from a simple dis-tribution to a more complex distribution that contains these dependencies. Structured output net-works (Lehrmann & Sigal, 2017), on the other hand, directly predict an expressive non-parametricoutput distribution. On the modeling side, one can add dependencies by constructing a hierarchi-cal latent representation (Dayan et al., 1995). These structures consist of conditional (empirical)priors, in which one latent variable forms a prior on another latent variable. While this conditional

∗Equal Contribution.

1


distribution may take a simple form, marginalizing over the parent variable can result in an arbitrar-ily complex distribution. Models with these more flexible latent dependency structures have beenshown to result in improved performance (Sønderby et al., 2016; Burda et al., 2016; Kingma et al.,2016). However, despite the benefits of including additional structure in these models, their depen-dency structures have so far been predefined, potentially limiting the performance of this approach.

In this work, we propose a method for learning dependency structures in latent variable models.Structure learning is a difficult task with a long history in the graphical models community (Koller& Friedman, 2009). Over the years, it has been tackled from several perspectives, includingconstraint-based approaches (Cheng et al., 2002; Lehmann & Romano, 2008), optimization of struc-ture scores (Kass & Raftery, 1995; Heckerman et al., 1995; Barron et al., 1998), Bayesian modelaveraging (Heckerman et al., 1999; Koivisto & Sood, 2004), and many more. Unfortunately, theunderlying objectives are often limited to graphs of a particular form (e.g., limited tree width),prohibitively expensive, or difficult to integrate with the gradient-based optimization techniques ofmodern neural networks. Here, we discuss an end-to-end approach for general graph structures in-troducing minimal complexity overhead. In particular, we introduce a set of binary global variablesto gate the latent dependencies. The whole model (including its structure) is jointly optimized witha single stochastic variational inference objective. In our experimental validation, we show that thelearned dependency structures result in models that more accurately model the data distribution,outperforming several common predefined latent dependency structures.

2 BACKGROUND

2.1 VARIATIONAL INFERENCE & VARIATIONAL AUTOENCODERS

A latent variable model, defined by the joint distribution, pθ(x,z) = pθ(x∣z)pθ(z), models eachdata example, x, using a local latent variable, z, and global parameters, θ. pθ(x∣z) denotes the con-ditional likelihood, and pθ(z) denotes the prior. Latent variable models are capable of capturing thestructure present in data, with z forming a compressed representation of each data example. Unfortu-nately, inferring the posterior, pθ(z∣x), is typically computationally intractable, prompting the use ofapproximate inference techniques. Variational inference (Jordan et al., 1999) introduces an approxi-mate posterior, qφ(z∣x), and optimizes variational parameters, φ, to minimize the KL-divergence tothe true posterior, KL(qφ(z∣x)∣∣pθ(z∣x)). As this quantity cannot be evaluated directly, the follow-ing relation is used:

log pθ(x) = KL(qφ(z∣x)∥pθ(z∣x)) +L(x; θ, φ), (1)

where L(x; θ, φ) is the evidence lower bound (ELBO), defined as

L(x; θ, φ) = Eqφ(z∣x) [log pθ(x∣z)] −KL(qφ(z∣x)∣∣pθ(z)). (2)

In Eq. (1), log pθ(x) is independent of φ, so we can minimize the KL divergence term, i.e. performapproximate inference, by maximizing L(x; θ, φ) w.r.t. qφ(z∣x). Further, because KL divergenceis non-negative, L(x; θ, φ) is a lower bound on log pθ(x), meaning we can then learn the modelparameters by maximizing L(x; θ, φ) w.r.t. θ.

Variational autoencoders (VAEs) (Kingma & Welling, 2014; Rezende et al., 2014) amortize infer-ence optimization across data examples by parameterizing qφ(z∣x) as a separate inference model,then jointly optimizing the model parameters θ and φ. VAEs instantiate both the inference modeland latent variable model with deep networks, allowing them to scale to high-dimensional data.However, VAEs are typically implemented with basic graphical structures and simple, unimodaldistributions (e.g. Gaussians). For instance, the dimensions of the prior are often assumed tobe independent, pθ(z) = ∏m pθ(zm), with a common assumption being a fixed standard Gaus-sian: pθ(z) = N (z;0, I). Similarly, approximate posteriors often make the mean field assump-tion, qφ(z∣x) = ∏m qφ(zm∣x). Independence assumptions such as these may be overly restrictive,thereby limiting modeling capabilities.

2.2 EMPIRICAL PRIORS THROUGH LATENT DEPENDENCY STRUCTURE

One technique for improving the expressive capacity of latent variable models is through incorpo-rating dependency structure among the latent variables, forming a hierarchy (Dayan et al., 1995;

2


fixed | chain-structured<latexit sha1_base64="hPRkNjEm+gQwq88DXy1G2kWr7vA=">AAACP3icbZDPSgMxEMaz9V+tVlu96SVYBS+W3V70KHjxWMG2Qi0lm51tg9nsksxKS13wabzqK/gYPoE38erNtN2DVT8IfPxmhpl8fiKFQdd9cwpLyyura8X10sZmeWu7Ut1pmzjVHFo8lrG+8ZkBKRS0UKCEm0QDi3wJHf/uYlrv3IM2IlbXOE6gF7GBEqHgDC3qV/ZCMYKAHj4cUj5kQp0Y1CnHVEPQr9TcujsT/Wu83NRIrma/6pRvg5inESjkkhnT9dwEexOmUXAJWek2NZAwfscG0LVWsQhMbzL7REaPLAloGGv7FNIZ/TkxYZEx48i3nRHDofldm8L/at0Uw7PeRKgkRVB8vihMJcWYThOhgdDAUY6tYVwLe+s0Cs042twWtgw0S4aCj7JFCmp21wL0o6xkA/R+x/XXtBt1z617V43a+VkeZZHskwNyTDxySs7JJWmSFuHkkTyRZ/LivDrvzofzOW8tOPnMLlmQ8/UNhH+vJA==</latexit><latexit sha1_base64="hPRkNjEm+gQwq88DXy1G2kWr7vA=">AAACP3icbZDPSgMxEMaz9V+tVlu96SVYBS+W3V70KHjxWMG2Qi0lm51tg9nsksxKS13wabzqK/gYPoE38erNtN2DVT8IfPxmhpl8fiKFQdd9cwpLyyura8X10sZmeWu7Ut1pmzjVHFo8lrG+8ZkBKRS0UKCEm0QDi3wJHf/uYlrv3IM2IlbXOE6gF7GBEqHgDC3qV/ZCMYKAHj4cUj5kQp0Y1CnHVEPQr9TcujsT/Wu83NRIrma/6pRvg5inESjkkhnT9dwEexOmUXAJWek2NZAwfscG0LVWsQhMbzL7REaPLAloGGv7FNIZ/TkxYZEx48i3nRHDofldm8L/at0Uw7PeRKgkRVB8vihMJcWYThOhgdDAUY6tYVwLe+s0Cs042twWtgw0S4aCj7JFCmp21wL0o6xkA/R+x/XXtBt1z617V43a+VkeZZHskwNyTDxySs7JJWmSFuHkkTyRZ/LivDrvzofzOW8tOPnMLlmQ8/UNhH+vJA==</latexit><latexit sha1_base64="hPRkNjEm+gQwq88DXy1G2kWr7vA=">AAACP3icbZDPSgMxEMaz9V+tVlu96SVYBS+W3V70KHjxWMG2Qi0lm51tg9nsksxKS13wabzqK/gYPoE38erNtN2DVT8IfPxmhpl8fiKFQdd9cwpLyyura8X10sZmeWu7Ut1pmzjVHFo8lrG+8ZkBKRS0UKCEm0QDi3wJHf/uYlrv3IM2IlbXOE6gF7GBEqHgDC3qV/ZCMYKAHj4cUj5kQp0Y1CnHVEPQr9TcujsT/Wu83NRIrma/6pRvg5inESjkkhnT9dwEexOmUXAJWek2NZAwfscG0LVWsQhMbzL7REaPLAloGGv7FNIZ/TkxYZEx48i3nRHDofldm8L/at0Uw7PeRKgkRVB8vihMJcWYThOhgdDAUY6tYVwLe+s0Cs042twWtgw0S4aCj7JFCmp21wL0o6xkA/R+x/XXtBt1z617V43a+VkeZZHskwNyTDxySs7JJWmSFuHkkTyRZ/LivDrvzofzOW8tOPnMLlmQ8/UNhH+vJA==</latexit><latexit sha1_base64="hPRkNjEm+gQwq88DXy1G2kWr7vA=">AAACP3icbZDPSgMxEMaz9V+tVlu96SVYBS+W3V70KHjxWMG2Qi0lm51tg9nsksxKS13wabzqK/gYPoE38erNtN2DVT8IfPxmhpl8fiKFQdd9cwpLyyura8X10sZmeWu7Ut1pmzjVHFo8lrG+8ZkBKRS0UKCEm0QDi3wJHf/uYlrv3IM2IlbXOE6gF7GBEqHgDC3qV/ZCMYKAHj4cUj5kQp0Y1CnHVEPQr9TcujsT/Wu83NRIrma/6pRvg5inESjkkhnT9dwEexOmUXAJWek2NZAwfscG0LVWsQhMbzL7REaPLAloGGv7FNIZ/TkxYZEx48i3nRHDofldm8L/at0Uw7PeRKgkRVB8vihMJcWYThOhgdDAUY6tYVwLe+s0Cs042twWtgw0S4aCj7JFCmp21wL0o6xkA/R+x/XXtBt1z617V43a+VkeZZHskwNyTDxySs7JJWmSFuHkkTyRZ/LivDrvzofzOW8tOPnMLlmQ8/UNhH+vJA==</latexit>

p✓(z) =Q

m p✓(zm)<latexit sha1_base64="1dUS6S6ffy0cF47jR+3gVKUA8lE=">AAACV3icbVDBThsxFPQuLaQptAk99mI1QUou0W4ucEFC4tIjSA0gZaOV13mbWNhey35bEVb7H3wN1/ILfA04IVIb0pEsjWbe0xtPZqRwGEXPQbjz4ePuXuNT8/P+wZevrfbhlStKy2HEC1nYm4w5kELDCAVKuDEWmMokXGe350v/+jdYJwr9CxcGJorNtMgFZ+iltDXsmjTBOSCjvUQxnGd5dV/36SlNjC2mqaJ//fu0UnW/m7Y60SBagW6TeE06ZI2LtB0cJNOClwo0csmcG8eRwUnFLAouoW4mpQPD+C2bwdhTzRS4SbX6XE2PvDKleWH900hX6r8bFVPOLVTmJ5f53XtvKf7PG5eYn0wqoU2JoPnbobyUFAu6bIpOhQWOcuEJ41b4rJTPmWUcfZ8bV2aWmbngd/WmCnqVa0PMVN30Bcbv69omV8NBHA3iy2Hn7GRdZYN8Jz9Ij8TkmJyRn+SCjAgnD+SR/CFPwXPwEu6GjbfRMFjvfCMbCNuvGtC1Ig==</latexit><latexit sha1_base64="1dUS6S6ffy0cF47jR+3gVKUA8lE=">AAACV3icbVDBThsxFPQuLaQptAk99mI1QUou0W4ucEFC4tIjSA0gZaOV13mbWNhey35bEVb7H3wN1/ILfA04IVIb0pEsjWbe0xtPZqRwGEXPQbjz4ePuXuNT8/P+wZevrfbhlStKy2HEC1nYm4w5kELDCAVKuDEWmMokXGe350v/+jdYJwr9CxcGJorNtMgFZ+iltDXsmjTBOSCjvUQxnGd5dV/36SlNjC2mqaJ//fu0UnW/m7Y60SBagW6TeE06ZI2LtB0cJNOClwo0csmcG8eRwUnFLAouoW4mpQPD+C2bwdhTzRS4SbX6XE2PvDKleWH900hX6r8bFVPOLVTmJ5f53XtvKf7PG5eYn0wqoU2JoPnbobyUFAu6bIpOhQWOcuEJ41b4rJTPmWUcfZ8bV2aWmbngd/WmCnqVa0PMVN30Bcbv69omV8NBHA3iy2Hn7GRdZYN8Jz9Ij8TkmJyRn+SCjAgnD+SR/CFPwXPwEu6GjbfRMFjvfCMbCNuvGtC1Ig==</latexit><latexit sha1_base64="1dUS6S6ffy0cF47jR+3gVKUA8lE=">AAACV3icbVDBThsxFPQuLaQptAk99mI1QUou0W4ucEFC4tIjSA0gZaOV13mbWNhey35bEVb7H3wN1/ILfA04IVIb0pEsjWbe0xtPZqRwGEXPQbjz4ePuXuNT8/P+wZevrfbhlStKy2HEC1nYm4w5kELDCAVKuDEWmMokXGe350v/+jdYJwr9CxcGJorNtMgFZ+iltDXsmjTBOSCjvUQxnGd5dV/36SlNjC2mqaJ//fu0UnW/m7Y60SBagW6TeE06ZI2LtB0cJNOClwo0csmcG8eRwUnFLAouoW4mpQPD+C2bwdhTzRS4SbX6XE2PvDKleWH900hX6r8bFVPOLVTmJ5f53XtvKf7PG5eYn0wqoU2JoPnbobyUFAu6bIpOhQWOcuEJ41b4rJTPmWUcfZ8bV2aWmbngd/WmCnqVa0PMVN30Bcbv69omV8NBHA3iy2Hn7GRdZYN8Jz9Ij8TkmJyRn+SCjAgnD+SR/CFPwXPwEu6GjbfRMFjvfCMbCNuvGtC1Ig==</latexit><latexit sha1_base64="1dUS6S6ffy0cF47jR+3gVKUA8lE=">AAACV3icbVDBThsxFPQuLaQptAk99mI1QUou0W4ucEFC4tIjSA0gZaOV13mbWNhey35bEVb7H3wN1/ILfA04IVIb0pEsjWbe0xtPZqRwGEXPQbjz4ePuXuNT8/P+wZevrfbhlStKy2HEC1nYm4w5kELDCAVKuDEWmMokXGe350v/+jdYJwr9CxcGJorNtMgFZ+iltDXsmjTBOSCjvUQxnGd5dV/36SlNjC2mqaJ//fu0UnW/m7Y60SBagW6TeE06ZI2LtB0cJNOClwo0csmcG8eRwUnFLAouoW4mpQPD+C2bwdhTzRS4SbX6XE2PvDKleWH900hX6r8bFVPOLVTmJ5f53XtvKf7PG5eYn0wqoU2JoPnbobyUFAu6bIpOhQWOcuEJ41b4rJTPmWUcfZ8bV2aWmbngd/WmCnqVa0PMVN30Bcbv69omV8NBHA3iy2Hn7GRdZYN8Jz9Ij8TkmJyRn+SCjAgnD+SR/CFPwXPwEu6GjbfRMFjvfCMbCNuvGtC1Ig==</latexit>

x<latexit sha1_base64="yMKvcsbQNkvFrBnVkm2K+W26W+g=">AAACGXicbVC7TgJBFL3rE/GFWtpMBBMrskujJdHGEhN5JLAhs8NdmDA7u5mZNRDCR9jK19gZWys/xsRhoRDwJJOcnHNv7pkTJIJr47rfztb2zu7efu4gf3h0fHJaODtv6DhVDOssFrFqBVSj4BLrhhuBrUQhjQKBzWD4MPebL6g0j+WzGSfoR7QvecgZNVZqljpBSEalbqHolt0MZJN4S1KEJWrdwk+nF7M0QmmYoFq3PTcx/oQqw5nAab6TakwoG9I+ti2VNELtT7K4U3JtlR4JY2WfNCRT/25MaKT1OArsZETNQK97c/E/r52a8M6fcJmkBiVbHApTQUxM5n8nPa6QGTG2hDLFbVbCBlRRZmxDK1f6iiYDzkbTVRVllitv+/LW29kkjUrZc8veU6VYvV82l4NLuIIb8OAWqvAINagDgyG8whvMnJnz7nw4n4vRLWe5cwErcL5+AY50oP0=</latexit><latexit sha1_base64="yMKvcsbQNkvFrBnVkm2K+W26W+g=">AAACGXicbVC7TgJBFL3rE/GFWtpMBBMrskujJdHGEhN5JLAhs8NdmDA7u5mZNRDCR9jK19gZWys/xsRhoRDwJJOcnHNv7pkTJIJr47rfztb2zu7efu4gf3h0fHJaODtv6DhVDOssFrFqBVSj4BLrhhuBrUQhjQKBzWD4MPebL6g0j+WzGSfoR7QvecgZNVZqljpBSEalbqHolt0MZJN4S1KEJWrdwk+nF7M0QmmYoFq3PTcx/oQqw5nAab6TakwoG9I+ti2VNELtT7K4U3JtlR4JY2WfNCRT/25MaKT1OArsZETNQK97c/E/r52a8M6fcJmkBiVbHApTQUxM5n8nPa6QGTG2hDLFbVbCBlRRZmxDK1f6iiYDzkbTVRVllitv+/LW29kkjUrZc8veU6VYvV82l4NLuIIb8OAWqvAINagDgyG8whvMnJnz7nw4n4vRLWe5cwErcL5+AY50oP0=</latexit><latexit sha1_base64="yMKvcsbQNkvFrBnVkm2K+W26W+g=">AAACGXicbVC7TgJBFL3rE/GFWtpMBBMrskujJdHGEhN5JLAhs8NdmDA7u5mZNRDCR9jK19gZWys/xsRhoRDwJJOcnHNv7pkTJIJr47rfztb2zu7efu4gf3h0fHJaODtv6DhVDOssFrFqBVSj4BLrhhuBrUQhjQKBzWD4MPebL6g0j+WzGSfoR7QvecgZNVZqljpBSEalbqHolt0MZJN4S1KEJWrdwk+nF7M0QmmYoFq3PTcx/oQqw5nAab6TakwoG9I+ti2VNELtT7K4U3JtlR4JY2WfNCRT/25MaKT1OArsZETNQK97c/E/r52a8M6fcJmkBiVbHApTQUxM5n8nPa6QGTG2hDLFbVbCBlRRZmxDK1f6iiYDzkbTVRVllitv+/LW29kkjUrZc8veU6VYvV82l4NLuIIb8OAWqvAINagDgyG8whvMnJnz7nw4n4vRLWe5cwErcL5+AY50oP0=</latexit><latexit sha1_base64="yMKvcsbQNkvFrBnVkm2K+W26W+g=">AAACGXicbVC7TgJBFL3rE/GFWtpMBBMrskujJdHGEhN5JLAhs8NdmDA7u5mZNRDCR9jK19gZWys/xsRhoRDwJJOcnHNv7pkTJIJr47rfztb2zu7efu4gf3h0fHJaODtv6DhVDOssFrFqBVSj4BLrhhuBrUQhjQKBzWD4MPebL6g0j+WzGSfoR7QvecgZNVZqljpBSEalbqHolt0MZJN4S1KEJWrdwk+nF7M0QmmYoFq3PTcx/oQqw5nAab6TakwoG9I+ti2VNELtT7K4U3JtlR4JY2WfNCRT/25MaKT1OArsZETNQK97c/E/r52a8M6fcJmkBiVbHApTQUxM5n8nPa6QGTG2hDLFbVbCBlRRZmxDK1f6iiYDzkbTVRVllitv+/LW29kkjUrZc8veU6VYvV82l4NLuIIb8OAWqvAINagDgyG8whvMnJnz7nw4n4vRLWe5cwErcL5+AY50oP0=</latexit>

✓<latexit sha1_base64="xWmXra5P90YYt2hGo0QW8XWb5is=">AAACGnicbVDLTsJAFJ3iC/GFunTTCCauSMtGl0Q3LjGRRwINmQ63dMJ02szcGgnhJ9zK17gzbt34MSYOpQsBTzLJyTn35p45fiK4Rsf5tgpb2zu7e8X90sHh0fFJ+fSsreNUMWixWMSq61MNgktoIUcB3UQBjXwBHX98v/A7z6A0j+UTThLwIjqSPOCMopG61T6GgLQ6KFecmpPB3iRuTiokR3NQ/ukPY5ZGIJEJqnXPdRL0plQhZwJmpX6qIaFsTEfQM1TSCLQ3zfLO7CujDO0gVuZJtDP178aURlpPIt9MRhRDve4txP+8XorBrTflMkkRJFseClJhY2wvPm8PuQKGYmIIZYqbrDYLqaIMTUUrV0aKJiFnL7NVFWSWq2T6ctfb2STtes11au5jvdK4y5srkgtySa6JS25IgzyQJmkRRgR5JW9kbs2td+vD+lyOFqx855yswPr6Belyob0=</latexit><latexit sha1_base64="xWmXra5P90YYt2hGo0QW8XWb5is=">AAACGnicbVDLTsJAFJ3iC/GFunTTCCauSMtGl0Q3LjGRRwINmQ63dMJ02szcGgnhJ9zK17gzbt34MSYOpQsBTzLJyTn35p45fiK4Rsf5tgpb2zu7e8X90sHh0fFJ+fSsreNUMWixWMSq61MNgktoIUcB3UQBjXwBHX98v/A7z6A0j+UTThLwIjqSPOCMopG61T6GgLQ6KFecmpPB3iRuTiokR3NQ/ukPY5ZGIJEJqnXPdRL0plQhZwJmpX6qIaFsTEfQM1TSCLQ3zfLO7CujDO0gVuZJtDP178aURlpPIt9MRhRDve4txP+8XorBrTflMkkRJFseClJhY2wvPm8PuQKGYmIIZYqbrDYLqaIMTUUrV0aKJiFnL7NVFWSWq2T6ctfb2STtes11au5jvdK4y5srkgtySa6JS25IgzyQJmkRRgR5JW9kbs2td+vD+lyOFqx855yswPr6Belyob0=</latexit><latexit sha1_base64="xWmXra5P90YYt2hGo0QW8XWb5is=">AAACGnicbVDLTsJAFJ3iC/GFunTTCCauSMtGl0Q3LjGRRwINmQ63dMJ02szcGgnhJ9zK17gzbt34MSYOpQsBTzLJyTn35p45fiK4Rsf5tgpb2zu7e8X90sHh0fFJ+fSsreNUMWixWMSq61MNgktoIUcB3UQBjXwBHX98v/A7z6A0j+UTThLwIjqSPOCMopG61T6GgLQ6KFecmpPB3iRuTiokR3NQ/ukPY5ZGIJEJqnXPdRL0plQhZwJmpX6qIaFsTEfQM1TSCLQ3zfLO7CujDO0gVuZJtDP178aURlpPIt9MRhRDve4txP+8XorBrTflMkkRJFseClJhY2wvPm8PuQKGYmIIZYqbrDYLqaIMTUUrV0aKJiFnL7NVFWSWq2T6ctfb2STtes11au5jvdK4y5srkgtySa6JS25IgzyQJmkRRgR5JW9kbs2td+vD+lyOFqx855yswPr6Belyob0=</latexit><latexit sha1_base64="xWmXra5P90YYt2hGo0QW8XWb5is=">AAACGnicbVDLTsJAFJ3iC/GFunTTCCauSMtGl0Q3LjGRRwINmQ63dMJ02szcGgnhJ9zK17gzbt34MSYOpQsBTzLJyTn35p45fiK4Rsf5tgpb2zu7e8X90sHh0fFJ+fSsreNUMWixWMSq61MNgktoIUcB3UQBjXwBHX98v/A7z6A0j+UTThLwIjqSPOCMopG61T6GgLQ6KFecmpPB3iRuTiokR3NQ/ukPY5ZGIJEJqnXPdRL0plQhZwJmpX6qIaFsTEfQM1TSCLQ3zfLO7CujDO0gVuZJtDP178aURlpPIt9MRhRDve4txP+8XorBrTflMkkRJFseClJhY2wvPm8PuQKGYmIIZYqbrDYLqaIMTUUrV0aKJiFnL7NVFWSWq2T6ctfb2STtes11au5jvdK4y5srkgtySa6JS25IgzyQJmkRRgR5JW9kbs2td+vD+lyOFqx855yswPr6Belyob0=</latexit>

�<latexit sha1_base64="H9c4lv8scahYYuN6NzGl+NKdbDc=">AAACGHicbVDLTsJAFL31ifhCXbppBBNXpGWjS6Ibl5jII4GGTIdbGJlOm5mpkTT8g1v5GnfGrTs/xsShdCHgSSY5Oefe3DPHjzlT2nG+rY3Nre2d3cJecf/g8Oi4dHLaUlEiKTZpxCPZ8YlCzgQ2NdMcO7FEEvoc2/74bu63n1EqFolHPYnRC8lQsIBRoo3UqvTiEav0S2Wn6mSw14mbkzLkaPRLP71BRJMQhaacKNV1nVh7KZGaUY7TYi9RGBM6JkPsGipIiMpLs7RT+9IoAzuIpHlC25n6dyMloVKT0DeTIdEjterNxf+8bqKDGy9lIk40Cro4FCTc1pE9/7o9YBKp5hNDCJXMZLXpiEhCtSlo6cpQEtMLfZkuqyiyXEXTl7vazjpp1aquU3UfauX6bd5cAc7hAq7AhWuowz00oAkUnuAV3mBmzax368P6XIxuWPnOGSzB+voFM8Gg1A==</latexit><latexit sha1_base64="H9c4lv8scahYYuN6NzGl+NKdbDc=">AAACGHicbVDLTsJAFL31ifhCXbppBBNXpGWjS6Ibl5jII4GGTIdbGJlOm5mpkTT8g1v5GnfGrTs/xsShdCHgSSY5Oefe3DPHjzlT2nG+rY3Nre2d3cJecf/g8Oi4dHLaUlEiKTZpxCPZ8YlCzgQ2NdMcO7FEEvoc2/74bu63n1EqFolHPYnRC8lQsIBRoo3UqvTiEav0S2Wn6mSw14mbkzLkaPRLP71BRJMQhaacKNV1nVh7KZGaUY7TYi9RGBM6JkPsGipIiMpLs7RT+9IoAzuIpHlC25n6dyMloVKT0DeTIdEjterNxf+8bqKDGy9lIk40Cro4FCTc1pE9/7o9YBKp5hNDCJXMZLXpiEhCtSlo6cpQEtMLfZkuqyiyXEXTl7vazjpp1aquU3UfauX6bd5cAc7hAq7AhWuowz00oAkUnuAV3mBmzax368P6XIxuWPnOGSzB+voFM8Gg1A==</latexit><latexit sha1_base64="H9c4lv8scahYYuN6NzGl+NKdbDc=">AAACGHicbVDLTsJAFL31ifhCXbppBBNXpGWjS6Ibl5jII4GGTIdbGJlOm5mpkTT8g1v5GnfGrTs/xsShdCHgSSY5Oefe3DPHjzlT2nG+rY3Nre2d3cJecf/g8Oi4dHLaUlEiKTZpxCPZ8YlCzgQ2NdMcO7FEEvoc2/74bu63n1EqFolHPYnRC8lQsIBRoo3UqvTiEav0S2Wn6mSw14mbkzLkaPRLP71BRJMQhaacKNV1nVh7KZGaUY7TYi9RGBM6JkPsGipIiMpLs7RT+9IoAzuIpHlC25n6dyMloVKT0DeTIdEjterNxf+8bqKDGy9lIk40Cro4FCTc1pE9/7o9YBKp5hNDCJXMZLXpiEhCtSlo6cpQEtMLfZkuqyiyXEXTl7vazjpp1aquU3UfauX6bd5cAc7hAq7AhWuowz00oAkUnuAV3mBmzax368P6XIxuWPnOGSzB+voFM8Gg1A==</latexit><latexit sha1_base64="H9c4lv8scahYYuN6NzGl+NKdbDc=">AAACGHicbVDLTsJAFL31ifhCXbppBBNXpGWjS6Ibl5jII4GGTIdbGJlOm5mpkTT8g1v5GnfGrTs/xsShdCHgSSY5Oefe3DPHjzlT2nG+rY3Nre2d3cJecf/g8Oi4dHLaUlEiKTZpxCPZ8YlCzgQ2NdMcO7FEEvoc2/74bu63n1EqFolHPYnRC8lQsIBRoo3UqvTiEav0S2Wn6mSw14mbkzLkaPRLP71BRJMQhaacKNV1nVh7KZGaUY7TYi9RGBM6JkPsGipIiMpLs7RT+9IoAzuIpHlC25n6dyMloVKT0DeTIdEjterNxf+8bqKDGy9lIk40Cro4FCTc1pE9/7o9YBKp5hNDCJXMZLXpiEhCtSlo6cpQEtMLfZkuqyiyXEXTl7vazjpp1aquU3UfauX6bd5cAc7hAq7AhWuowz00oAkUnuAV3mBmzax368P6XIxuWPnOGSzB+voFM8Gg1A==</latexit>

N<latexit sha1_base64="P8oqS/zsceICK4985TkO8rf6tpY=">AAACFXicbVC7TsMwFL0pr1JeBUYWixaJqUq6wFjBwoSKoA+pjSrHdVKrjhPZDqKK+gms9GvYECszH4OEm2agLUeydHTOvbrHx4s5U9q2v63CxubW9k5xt7S3f3B4VD4+aasokYS2SMQj2fWwopwJ2tJMc9qNJcWhx2nHG9/O/c4zlYpF4klPYuqGOBDMZwRrIz1W76uDcsWu2RnQOnFyUoEczUH5pz+MSBJSoQnHSvUcO9ZuiqVmhNNpqZ8oGmMyxgHtGSpwSJWbZlGn6MIoQ+RH0jyhUab+3UhxqNQk9MxkiPVIrXpz8T+vl2j/2k2ZiBNNBVkc8hOOdITm/0ZDJinRfGIIJpKZrIiMsMREm3aWrgQSxyNGXqbLKhVZrpLpy1ltZ5206zXHrjkP9UrjJm+uCGdwDpfgwBU04A6a0AICAbzCG8ysmfVufVifi9GCle+cwhKsr1+MPZ9n</latexit><latexit sha1_base64="P8oqS/zsceICK4985TkO8rf6tpY=">AAACFXicbVC7TsMwFL0pr1JeBUYWixaJqUq6wFjBwoSKoA+pjSrHdVKrjhPZDqKK+gms9GvYECszH4OEm2agLUeydHTOvbrHx4s5U9q2v63CxubW9k5xt7S3f3B4VD4+aasokYS2SMQj2fWwopwJ2tJMc9qNJcWhx2nHG9/O/c4zlYpF4klPYuqGOBDMZwRrIz1W76uDcsWu2RnQOnFyUoEczUH5pz+MSBJSoQnHSvUcO9ZuiqVmhNNpqZ8oGmMyxgHtGSpwSJWbZlGn6MIoQ+RH0jyhUab+3UhxqNQk9MxkiPVIrXpz8T+vl2j/2k2ZiBNNBVkc8hOOdITm/0ZDJinRfGIIJpKZrIiMsMREm3aWrgQSxyNGXqbLKhVZrpLpy1ltZ5206zXHrjkP9UrjJm+uCGdwDpfgwBU04A6a0AICAbzCG8ysmfVufVifi9GCle+cwhKsr1+MPZ9n</latexit><latexit sha1_base64="P8oqS/zsceICK4985TkO8rf6tpY=">AAACFXicbVC7TsMwFL0pr1JeBUYWixaJqUq6wFjBwoSKoA+pjSrHdVKrjhPZDqKK+gms9GvYECszH4OEm2agLUeydHTOvbrHx4s5U9q2v63CxubW9k5xt7S3f3B4VD4+aasokYS2SMQj2fWwopwJ2tJMc9qNJcWhx2nHG9/O/c4zlYpF4klPYuqGOBDMZwRrIz1W76uDcsWu2RnQOnFyUoEczUH5pz+MSBJSoQnHSvUcO9ZuiqVmhNNpqZ8oGmMyxgHtGSpwSJWbZlGn6MIoQ+RH0jyhUab+3UhxqNQk9MxkiPVIrXpz8T+vl2j/2k2ZiBNNBVkc8hOOdITm/0ZDJinRfGIIJpKZrIiMsMREm3aWrgQSxyNGXqbLKhVZrpLpy1ltZ5206zXHrjkP9UrjJm+uCGdwDpfgwBU04A6a0AICAbzCG8ysmfVufVifi9GCle+cwhKsr1+MPZ9n</latexit><latexit sha1_base64="P8oqS/zsceICK4985TkO8rf6tpY=">AAACFXicbVC7TsMwFL0pr1JeBUYWixaJqUq6wFjBwoSKoA+pjSrHdVKrjhPZDqKK+gms9GvYECszH4OEm2agLUeydHTOvbrHx4s5U9q2v63CxubW9k5xt7S3f3B4VD4+aasokYS2SMQj2fWwopwJ2tJMc9qNJcWhx2nHG9/O/c4zlYpF4klPYuqGOBDMZwRrIz1W76uDcsWu2RnQOnFyUoEczUH5pz+MSBJSoQnHSvUcO9ZuiqVmhNNpqZ8oGmMyxgHtGSpwSJWbZlGn6MIoQ+RH0jyhUab+3UhxqNQk9MxkiPVIrXpz8T+vl2j/2k2ZiBNNBVkc8hOOdITm/0ZDJinRfGIIJpKZrIiMsMREm3aWrgQSxyNGXqbLKhVZrpLpy1ltZ5206zXHrjkP9UrjJm+uCGdwDpfgwBU04A6a0AICAbzCG8ysmfVufVifi9GCle+cwhKsr1+MPZ9n</latexit>

z<latexit sha1_base64="qv/I6fmAzetHscBr4SeDOeeZ+LU=">AAACF3icbVA9TwJBFHznJ+IXammzkZhYkTsarQyJjSUmHpAAIXvLO9iwt3fZ3TPihd9gK7/Gztha+mNMXA4KASfZZDLzXt7sBIng2rjut7OxubW9s1vYK+4fHB4dl05OGzpOFUOfxSJWrYBqFFyib7gR2EoU0igQ2AxGdzO/+YRK81g+mnGC3YgOJA85o8ZKficIyUuvVHYrbg6yTrwFKcMC9V7pp9OPWRqhNExQrduem5huRpXhTOCk2Ek1JpSN6ADblkoaoe5medgJubRKn4Sxsk8akqt/NzIaaT2OAjsZUTPUq95M/M9rpya86WZcJqlByeaHwlQQE5PZz0mfK2RGjC2hTHGblbAhVZQZ28/SlYGiyZCz58myijLPVbR9eavtrJNGteK5Fe+hWq7dLporwDlcwBV4cA01uIc6+MCAwyu8wdSZOu/Oh/M5H91wFjtnsATn6xfKk6Cf</latexit><latexit sha1_base64="qv/I6fmAzetHscBr4SeDOeeZ+LU=">AAACF3icbVA9TwJBFHznJ+IXammzkZhYkTsarQyJjSUmHpAAIXvLO9iwt3fZ3TPihd9gK7/Gztha+mNMXA4KASfZZDLzXt7sBIng2rjut7OxubW9s1vYK+4fHB4dl05OGzpOFUOfxSJWrYBqFFyib7gR2EoU0igQ2AxGdzO/+YRK81g+mnGC3YgOJA85o8ZKficIyUuvVHYrbg6yTrwFKcMC9V7pp9OPWRqhNExQrduem5huRpXhTOCk2Ek1JpSN6ADblkoaoe5medgJubRKn4Sxsk8akqt/NzIaaT2OAjsZUTPUq95M/M9rpya86WZcJqlByeaHwlQQE5PZz0mfK2RGjC2hTHGblbAhVZQZ28/SlYGiyZCz58myijLPVbR9eavtrJNGteK5Fe+hWq7dLporwDlcwBV4cA01uIc6+MCAwyu8wdSZOu/Oh/M5H91wFjtnsATn6xfKk6Cf</latexit><latexit sha1_base64="qv/I6fmAzetHscBr4SeDOeeZ+LU=">AAACF3icbVA9TwJBFHznJ+IXammzkZhYkTsarQyJjSUmHpAAIXvLO9iwt3fZ3TPihd9gK7/Gztha+mNMXA4KASfZZDLzXt7sBIng2rjut7OxubW9s1vYK+4fHB4dl05OGzpOFUOfxSJWrYBqFFyib7gR2EoU0igQ2AxGdzO/+YRK81g+mnGC3YgOJA85o8ZKficIyUuvVHYrbg6yTrwFKcMC9V7pp9OPWRqhNExQrduem5huRpXhTOCk2Ek1JpSN6ADblkoaoe5medgJubRKn4Sxsk8akqt/NzIaaT2OAjsZUTPUq95M/M9rpya86WZcJqlByeaHwlQQE5PZz0mfK2RGjC2hTHGblbAhVZQZ28/SlYGiyZCz58myijLPVbR9eavtrJNGteK5Fe+hWq7dLporwDlcwBV4cA01uIc6+MCAwyu8wdSZOu/Oh/M5H91wFjtnsATn6xfKk6Cf</latexit><latexit sha1_base64="qv/I6fmAzetHscBr4SeDOeeZ+LU=">AAACF3icbVA9TwJBFHznJ+IXammzkZhYkTsarQyJjSUmHpAAIXvLO9iwt3fZ3TPihd9gK7/Gztha+mNMXA4KASfZZDLzXt7sBIng2rjut7OxubW9s1vYK+4fHB4dl05OGzpOFUOfxSJWrYBqFFyib7gR2EoU0igQ2AxGdzO/+YRK81g+mnGC3YgOJA85o8ZKficIyUuvVHYrbg6yTrwFKcMC9V7pp9OPWRqhNExQrduem5huRpXhTOCk2Ek1JpSN6ADblkoaoe5medgJubRKn4Sxsk8akqt/NzIaaT2OAjsZUTPUq95M/M9rpya86WZcJqlByeaHwlQQE5PZz0mfK2RGjC2hTHGblbAhVZQZ28/SlYGiyZCz58myijLPVbR9eavtrJNGteK5Fe+hWq7dLporwDlcwBV4cA01uIc6+MCAwyu8wdSZOu/Oh/M5H91wFjtnsATn6xfKk6Cf</latexit>

...<latexit sha1_base64="GeFPqah0GabDOiVnSUd8mdAsBt0=">AAACGnicbVDLTgIxFO34RHyhLt00gokrMsNGl0Q3LjGRRwIT0ul0oKHTTto7RDLhJ9zK17gzbt34MSYWmIWAJ2lycs69uacnSAQ34Lrfztb2zu7efuGgeHh0fHJaOjtvGZVqyppUCaU7ATFMcMmawEGwTqIZiQPB2sHoYe63x0wbruQzTBLmx2QgecQpASt1Kr1xqMBU+qWyW3UXwJvEy0kZ5Wj0Sz+9UNE0ZhKoIMZ0PTcBPyMaOBVsWuylhiWEjsiAdS2VJGbGzxZ5p/jaKiGOlLZPAl6ofzcyEhsziQM7GRMYmnVvLv7ndVOI7vyMyyQFJunyUJQKDArPP49DrhkFMbGEUM1tVkyHRBMKtqKVKwNNkiGnL9NVlclFrqLty1tvZ5O0alXPrXpPtXL9Pm+ugC7RFbpBHrpFdfSIGqiJKBLoFb2hmTNz3p0P53M5uuXkOxdoBc7XLxVxodc=</latexit><latexit sha1_base64="GeFPqah0GabDOiVnSUd8mdAsBt0=">AAACGnicbVDLTgIxFO34RHyhLt00gokrMsNGl0Q3LjGRRwIT0ul0oKHTTto7RDLhJ9zK17gzbt34MSYWmIWAJ2lycs69uacnSAQ34Lrfztb2zu7efuGgeHh0fHJaOjtvGZVqyppUCaU7ATFMcMmawEGwTqIZiQPB2sHoYe63x0wbruQzTBLmx2QgecQpASt1Kr1xqMBU+qWyW3UXwJvEy0kZ5Wj0Sz+9UNE0ZhKoIMZ0PTcBPyMaOBVsWuylhiWEjsiAdS2VJGbGzxZ5p/jaKiGOlLZPAl6ofzcyEhsziQM7GRMYmnVvLv7ndVOI7vyMyyQFJunyUJQKDArPP49DrhkFMbGEUM1tVkyHRBMKtqKVKwNNkiGnL9NVlclFrqLty1tvZ5O0alXPrXpPtXL9Pm+ugC7RFbpBHrpFdfSIGqiJKBLoFb2hmTNz3p0P53M5uuXkOxdoBc7XLxVxodc=</latexit><latexit sha1_base64="GeFPqah0GabDOiVnSUd8mdAsBt0=">AAACGnicbVDLTgIxFO34RHyhLt00gokrMsNGl0Q3LjGRRwIT0ul0oKHTTto7RDLhJ9zK17gzbt34MSYWmIWAJ2lycs69uacnSAQ34Lrfztb2zu7efuGgeHh0fHJaOjtvGZVqyppUCaU7ATFMcMmawEGwTqIZiQPB2sHoYe63x0wbruQzTBLmx2QgecQpASt1Kr1xqMBU+qWyW3UXwJvEy0kZ5Wj0Sz+9UNE0ZhKoIMZ0PTcBPyMaOBVsWuylhiWEjsiAdS2VJGbGzxZ5p/jaKiGOlLZPAl6ofzcyEhsziQM7GRMYmnVvLv7ndVOI7vyMyyQFJunyUJQKDArPP49DrhkFMbGEUM1tVkyHRBMKtqKVKwNNkiGnL9NVlclFrqLty1tvZ5O0alXPrXpPtXL9Pm+ugC7RFbpBHrpFdfSIGqiJKBLoFb2hmTNz3p0P53M5uuXkOxdoBc7XLxVxodc=</latexit><latexit sha1_base64="GeFPqah0GabDOiVnSUd8mdAsBt0=">AAACGnicbVDLTgIxFO34RHyhLt00gokrMsNGl0Q3LjGRRwIT0ul0oKHTTto7RDLhJ9zK17gzbt34MSYWmIWAJ2lycs69uacnSAQ34Lrfztb2zu7efuGgeHh0fHJaOjtvGZVqyppUCaU7ATFMcMmawEGwTqIZiQPB2sHoYe63x0wbruQzTBLmx2QgecQpASt1Kr1xqMBU+qWyW3UXwJvEy0kZ5Wj0Sz+9UNE0ZhKoIMZ0PTcBPyMaOBVsWuylhiWEjsiAdS2VJGbGzxZ5p/jaKiGOlLZPAl6ofzcyEhsziQM7GRMYmnVvLv7ndVOI7vyMyyQFJunyUJQKDArPP49DrhkFMbGEUM1tVkyHRBMKtqKVKwNNkiGnL9NVlclFrqLty1tvZ5O0alXPrXpPtXL9Pm+ugC7RFbpBHrpFdfSIGqiJKBLoFb2hmTNz3p0P53M5uuXkOxdoBc7XLxVxodc=</latexit>




...<latexit sha1_base64="GeFPqah0GabDOiVnSUd8mdAsBt0=">AAACGnicbVDLTgIxFO34RHyhLt00gokrMsNGl0Q3LjGRRwIT0ul0oKHTTto7RDLhJ9zK17gzbt34MSYWmIWAJ2lycs69uacnSAQ34Lrfztb2zu7efuGgeHh0fHJaOjtvGZVqyppUCaU7ATFMcMmawEGwTqIZiQPB2sHoYe63x0wbruQzTBLmx2QgecQpASt1Kr1xqMBU+qWyW3UXwJvEy0kZ5Wj0Sz+9UNE0ZhKoIMZ0PTcBPyMaOBVsWuylhiWEjsiAdS2VJGbGzxZ5p/jaKiGOlLZPAl6ofzcyEhsziQM7GRMYmnVvLv7ndVOI7vyMyyQFJunyUJQKDArPP49DrhkFMbGEUM1tVkyHRBMKtqKVKwNNkiGnL9NVlclFrqLty1tvZ5O0alXPrXpPtXL9Pm+ugC7RFbpBHrpFdfSIGqiJKBLoFb2hmTNz3p0P53M5uuXkOxdoBc7XLxVxodc=</latexit><latexit sha1_base64="GeFPqah0GabDOiVnSUd8mdAsBt0=">AAACGnicbVDLTgIxFO34RHyhLt00gokrMsNGl0Q3LjGRRwIT0ul0oKHTTto7RDLhJ9zK17gzbt34MSYWmIWAJ2lycs69uacnSAQ34Lrfztb2zu7efuGgeHh0fHJaOjtvGZVqyppUCaU7ATFMcMmawEGwTqIZiQPB2sHoYe63x0wbruQzTBLmx2QgecQpASt1Kr1xqMBU+qWyW3UXwJvEy0kZ5Wj0Sz+9UNE0ZhKoIMZ0PTcBPyMaOBVsWuylhiWEjsiAdS2VJGbGzxZ5p/jaKiGOlLZPAl6ofzcyEhsziQM7GRMYmnVvLv7ndVOI7vyMyyQFJunyUJQKDArPP49DrhkFMbGEUM1tVkyHRBMKtqKVKwNNkiGnL9NVlclFrqLty1tvZ5O0alXPrXpPtXL9Pm+ugC7RFbpBHrpFdfSIGqiJKBLoFb2hmTNz3p0P53M5uuXkOxdoBc7XLxVxodc=</latexit><latexit sha1_base64="GeFPqah0GabDOiVnSUd8mdAsBt0=">AAACGnicbVDLTgIxFO34RHyhLt00gokrMsNGl0Q3LjGRRwIT0ul0oKHTTto7RDLhJ9zK17gzbt34MSYWmIWAJ2lycs69uacnSAQ34Lrfztb2zu7efuGgeHh0fHJaOjtvGZVqyppUCaU7ATFMcMmawEGwTqIZiQPB2sHoYe63x0wbruQzTBLmx2QgecQpASt1Kr1xqMBU+qWyW3UXwJvEy0kZ5Wj0Sz+9UNE0ZhKoIMZ0PTcBPyMaOBVsWuylhiWEjsiAdS2VJGbGzxZ5p/jaKiGOlLZPAl6ofzcyEhsziQM7GRMYmnVvLv7ndVOI7vyMyyQFJunyUJQKDArPP49DrhkFMbGEUM1tVkyHRBMKtqKVKwNNkiGnL9NVlclFrqLty1tvZ5O0alXPrXpPtXL9Pm+ugC7RFbpBHrpFdfSIGqiJKBLoFb2hmTNz3p0P53M5uuXkOxdoBc7XLxVxodc=</latexit><latexit sha1_base64="GeFPqah0GabDOiVnSUd8mdAsBt0=">AAACGnicbVDLTgIxFO34RHyhLt00gokrMsNGl0Q3LjGRRwIT0ul0oKHTTto7RDLhJ9zK17gzbt34MSYWmIWAJ2lycs69uacnSAQ34Lrfztb2zu7efuGgeHh0fHJaOjtvGZVqyppUCaU7ATFMcMmawEGwTqIZiQPB2sHoYe63x0wbruQzTBLmx2QgecQpASt1Kr1xqMBU+qWyW3UXwJvEy0kZ5Wj0Sz+9UNE0ZhKoIMZ0PTcBPyMaOBVsWuylhiWEjsiAdS2VJGbGzxZ5p/jaKiGOlLZPAl6ofzcyEhsziQM7GRMYmnVvLv7ndVOI7vyMyyQFJunyUJQKDArPP49DrhkFMbGEUM1tVkyHRBMKtqKVKwNNkiGnL9NVlclFrqLty1tvZ5O0alXPrXpPtXL9Pm+ugC7RFbpBHrpFdfSIGqiJKBLoFb2hmTNz3p0P53M5uuXkOxdoBc7XLxVxodc=</latexit>

Ladder VAE<latexit sha1_base64="rak78yFFnkMmPssnckSD+ST52Qo=">AAACHHicbVDLSsNAFJ3UV62vqks3g0VwVZJudCUVEVy4qGAf2IYymdy0QyeTMDMRS+hfuLVf407cCn6M4DTNwrYeGDiccy/3zPFizpS27W+rsLa+sblV3C7t7O7tH5QPj1oqSiSFJo14JDseUcCZgKZmmkMnlkBCj0PbG93M/PYzSMUi8ajHMbghGQgWMEq0kZ7uie+DxK3r2365YlftDHiVODmpoByNfvmn50c0CUFoyolSXceOtZsSqRnlMCn1EgUxoSMygK6hgoSg3DRLPMFnRvFxEEnzhMaZ+ncjJaFS49AzkyHRQ7XszcT/vG6ig0s3ZSJONAg6PxQkHOsIz76PfSaBaj42hFDJTFZMh0QSqk1JC1cGksRDRl8miyqILFfJ9OUst7NKWrWqY1edh1qlfpU3V0Qn6BSdIwddoDq6Qw3URBQJ9Ire0NSaWu/Wh/U5Hy1Y+c4xWoD19QsoAKJb</latexit><latexit sha1_base64="rak78yFFnkMmPssnckSD+ST52Qo=">AAACHHicbVDLSsNAFJ3UV62vqks3g0VwVZJudCUVEVy4qGAf2IYymdy0QyeTMDMRS+hfuLVf407cCn6M4DTNwrYeGDiccy/3zPFizpS27W+rsLa+sblV3C7t7O7tH5QPj1oqSiSFJo14JDseUcCZgKZmmkMnlkBCj0PbG93M/PYzSMUi8ajHMbghGQgWMEq0kZ7uie+DxK3r2365YlftDHiVODmpoByNfvmn50c0CUFoyolSXceOtZsSqRnlMCn1EgUxoSMygK6hgoSg3DRLPMFnRvFxEEnzhMaZ+ncjJaFS49AzkyHRQ7XszcT/vG6ig0s3ZSJONAg6PxQkHOsIz76PfSaBaj42hFDJTFZMh0QSqk1JC1cGksRDRl8miyqILFfJ9OUst7NKWrWqY1edh1qlfpU3V0Qn6BSdIwddoDq6Qw3URBQJ9Ire0NSaWu/Wh/U5Hy1Y+c4xWoD19QsoAKJb</latexit><latexit sha1_base64="rak78yFFnkMmPssnckSD+ST52Qo=">AAACHHicbVDLSsNAFJ3UV62vqks3g0VwVZJudCUVEVy4qGAf2IYymdy0QyeTMDMRS+hfuLVf407cCn6M4DTNwrYeGDiccy/3zPFizpS27W+rsLa+sblV3C7t7O7tH5QPj1oqSiSFJo14JDseUcCZgKZmmkMnlkBCj0PbG93M/PYzSMUi8ajHMbghGQgWMEq0kZ7uie+DxK3r2365YlftDHiVODmpoByNfvmn50c0CUFoyolSXceOtZsSqRnlMCn1EgUxoSMygK6hgoSg3DRLPMFnRvFxEEnzhMaZ+ncjJaFS49AzkyHRQ7XszcT/vG6ig0s3ZSJONAg6PxQkHOsIz76PfSaBaj42hFDJTFZMh0QSqk1JC1cGksRDRl8miyqILFfJ9OUst7NKWrWqY1edh1qlfpU3V0Qn6BSdIwddoDq6Qw3URBQJ9Ire0NSaWu/Wh/U5Hy1Y+c4xWoD19QsoAKJb</latexit><latexit sha1_base64="rak78yFFnkMmPssnckSD+ST52Qo=">AAACHHicbVDLSsNAFJ3UV62vqks3g0VwVZJudCUVEVy4qGAf2IYymdy0QyeTMDMRS+hfuLVf407cCn6M4DTNwrYeGDiccy/3zPFizpS27W+rsLa+sblV3C7t7O7tH5QPj1oqSiSFJo14JDseUcCZgKZmmkMnlkBCj0PbG93M/PYzSMUi8ajHMbghGQgWMEq0kZ7uie+DxK3r2365YlftDHiVODmpoByNfvmn50c0CUFoyolSXceOtZsSqRnlMCn1EgUxoSMygK6hgoSg3DRLPMFnRvFxEEnzhMaZ+ncjJaFS49AzkyHRQ7XszcT/vG6ig0s3ZSJONAg6PxQkHOsIz76PfSaBaj42hFDJTFZMh0QSqk1JC1cGksRDRl8miyqILFfJ9OUst7NKWrWqY1edh1qlfpU3V0Qn6BSdIwddoDq6Qw3URBQJ9Ire0NSaWu/Wh/U5Hy1Y+c4xWoD19QsoAKJb</latexit>

VAE<latexit sha1_base64="n8AseNkGoH4W/vGtOIIc+PD6iF8=">AAACFXicbVDLSsNAFL3xWeur6tJNsAiuStKNrqQigsuK9gFtKJPpTTt0MgkzE7GEfoJb+zXuxK1rP0ZwmmZhWw8MHM65l3vm+DFnSjvOt7W2vrG5tV3YKe7u7R8clo6OmypKJMUGjXgk2z5RyJnAhmaaYzuWSEKfY8sf3c781jNKxSLxpMcxeiEZCBYwSrSRHps3d71S2ak4GexV4uakDDnqvdJPtx/RJEShKSdKdVwn1l5KpGaU46TYTRTGhI7IADuGChKi8tIs6sQ+N0rfDiJpntB2pv7dSEmo1Dj0zWRI9FAtezPxP6+T6ODKS5mIE42Czg8FCbd1ZM/+bfeZRKr52BBCJTNZbTokklBt2lm4MpAkHjL6MllUUWS5iqYvd7mdVdKsVlyn4j5Uy7XrvLkCnMIZXIALl1CDe6hDAygM4BXeYGpNrXfrw/qcj65Z+c4JLMD6+gUBSZ+p</latexit><latexit sha1_base64="n8AseNkGoH4W/vGtOIIc+PD6iF8=">AAACFXicbVDLSsNAFL3xWeur6tJNsAiuStKNrqQigsuK9gFtKJPpTTt0MgkzE7GEfoJb+zXuxK1rP0ZwmmZhWw8MHM65l3vm+DFnSjvOt7W2vrG5tV3YKe7u7R8clo6OmypKJMUGjXgk2z5RyJnAhmaaYzuWSEKfY8sf3c781jNKxSLxpMcxeiEZCBYwSrSRHps3d71S2ak4GexV4uakDDnqvdJPtx/RJEShKSdKdVwn1l5KpGaU46TYTRTGhI7IADuGChKi8tIs6sQ+N0rfDiJpntB2pv7dSEmo1Dj0zWRI9FAtezPxP6+T6ODKS5mIE42Czg8FCbd1ZM/+bfeZRKr52BBCJTNZbTokklBt2lm4MpAkHjL6MllUUWS5iqYvd7mdVdKsVlyn4j5Uy7XrvLkCnMIZXIALl1CDe6hDAygM4BXeYGpNrXfrw/qcj65Z+c4JLMD6+gUBSZ+p</latexit><latexit sha1_base64="n8AseNkGoH4W/vGtOIIc+PD6iF8=">AAACFXicbVDLSsNAFL3xWeur6tJNsAiuStKNrqQigsuK9gFtKJPpTTt0MgkzE7GEfoJb+zXuxK1rP0ZwmmZhWw8MHM65l3vm+DFnSjvOt7W2vrG5tV3YKe7u7R8clo6OmypKJMUGjXgk2z5RyJnAhmaaYzuWSEKfY8sf3c781jNKxSLxpMcxeiEZCBYwSrSRHps3d71S2ak4GexV4uakDDnqvdJPtx/RJEShKSdKdVwn1l5KpGaU46TYTRTGhI7IADuGChKi8tIs6sQ+N0rfDiJpntB2pv7dSEmo1Dj0zWRI9FAtezPxP6+T6ODKS5mIE42Czg8FCbd1ZM/+bfeZRKr52BBCJTNZbTokklBt2lm4MpAkHjL6MllUUWS5iqYvd7mdVdKsVlyn4j5Uy7XrvLkCnMIZXIALl1CDe6hDAygM4BXeYGpNrXfrw/qcj65Z+c4JLMD6+gUBSZ+p</latexit><latexit sha1_base64="n8AseNkGoH4W/vGtOIIc+PD6iF8=">AAACFXicbVDLSsNAFL3xWeur6tJNsAiuStKNrqQigsuK9gFtKJPpTTt0MgkzE7GEfoJb+zXuxK1rP0ZwmmZhWw8MHM65l3vm+DFnSjvOt7W2vrG5tV3YKe7u7R8clo6OmypKJMUGjXgk2z5RyJnAhmaaYzuWSEKfY8sf3c781jNKxSLxpMcxeiEZCBYwSrSRHps3d71S2ak4GexV4uakDDnqvdJPtx/RJEShKSdKdVwn1l5KpGaU46TYTRTGhI7IADuGChKi8tIs6sQ+N0rfDiJpntB2pv7dSEmo1Dj0zWRI9FAtezPxP6+T6ODKS5mIE42Czg8FCbd1ZM/+bfeZRKr52BBCJTNZbTokklBt2lm4MpAkHjL6MllUUWS5iqYvd7mdVdKsVlyn4j5Uy7XrvLkCnMIZXIALl1CDe6hDAygM4BXeYGpNrXfrw/qcj65Z+c4JLMD6+gUBSZ+p</latexit>

✓<latexit sha1_base64="xWmXra5P90YYt2hGo0QW8XWb5is=">AAACGnicbVDLTsJAFJ3iC/GFunTTCCauSMtGl0Q3LjGRRwINmQ63dMJ02szcGgnhJ9zK17gzbt34MSYOpQsBTzLJyTn35p45fiK4Rsf5tgpb2zu7e8X90sHh0fFJ+fSsreNUMWixWMSq61MNgktoIUcB3UQBjXwBHX98v/A7z6A0j+UTThLwIjqSPOCMopG61T6GgLQ6KFecmpPB3iRuTiokR3NQ/ukPY5ZGIJEJqnXPdRL0plQhZwJmpX6qIaFsTEfQM1TSCLQ3zfLO7CujDO0gVuZJtDP178aURlpPIt9MRhRDve4txP+8XorBrTflMkkRJFseClJhY2wvPm8PuQKGYmIIZYqbrDYLqaIMTUUrV0aKJiFnL7NVFWSWq2T6ctfb2STtes11au5jvdK4y5srkgtySa6JS25IgzyQJmkRRgR5JW9kbs2td+vD+lyOFqx855yswPr6Belyob0=</latexit><latexit sha1_base64="xWmXra5P90YYt2hGo0QW8XWb5is=">AAACGnicbVDLTsJAFJ3iC/GFunTTCCauSMtGl0Q3LjGRRwINmQ63dMJ02szcGgnhJ9zK17gzbt34MSYOpQsBTzLJyTn35p45fiK4Rsf5tgpb2zu7e8X90sHh0fFJ+fSsreNUMWixWMSq61MNgktoIUcB3UQBjXwBHX98v/A7z6A0j+UTThLwIjqSPOCMopG61T6GgLQ6KFecmpPB3iRuTiokR3NQ/ukPY5ZGIJEJqnXPdRL0plQhZwJmpX6qIaFsTEfQM1TSCLQ3zfLO7CujDO0gVuZJtDP178aURlpPIt9MRhRDve4txP+8XorBrTflMkkRJFseClJhY2wvPm8PuQKGYmIIZYqbrDYLqaIMTUUrV0aKJiFnL7NVFWSWq2T6ctfb2STtes11au5jvdK4y5srkgtySa6JS25IgzyQJmkRRgR5JW9kbs2td+vD+lyOFqx855yswPr6Belyob0=</latexit><latexit sha1_base64="xWmXra5P90YYt2hGo0QW8XWb5is=">AAACGnicbVDLTsJAFJ3iC/GFunTTCCauSMtGl0Q3LjGRRwINmQ63dMJ02szcGgnhJ9zK17gzbt34MSYOpQsBTzLJyTn35p45fiK4Rsf5tgpb2zu7e8X90sHh0fFJ+fSsreNUMWixWMSq61MNgktoIUcB3UQBjXwBHX98v/A7z6A0j+UTThLwIjqSPOCMopG61T6GgLQ6KFecmpPB3iRuTiokR3NQ/ukPY5ZGIJEJqnXPdRL0plQhZwJmpX6qIaFsTEfQM1TSCLQ3zfLO7CujDO0gVuZJtDP178aURlpPIt9MRhRDve4txP+8XorBrTflMkkRJFseClJhY2wvPm8PuQKGYmIIZYqbrDYLqaIMTUUrV0aKJiFnL7NVFWSWq2T6ctfb2STtes11au5jvdK4y5srkgtySa6JS25IgzyQJmkRRgR5JW9kbs2td+vD+lyOFqx855yswPr6Belyob0=</latexit><latexit sha1_base64="xWmXra5P90YYt2hGo0QW8XWb5is=">AAACGnicbVDLTsJAFJ3iC/GFunTTCCauSMtGl0Q3LjGRRwINmQ63dMJ02szcGgnhJ9zK17gzbt34MSYOpQsBTzLJyTn35p45fiK4Rsf5tgpb2zu7e8X90sHh0fFJ+fSsreNUMWixWMSq61MNgktoIUcB3UQBjXwBHX98v/A7z6A0j+UTThLwIjqSPOCMopG61T6GgLQ6KFecmpPB3iRuTiokR3NQ/ukPY5ZGIJEJqnXPdRL0plQhZwJmpX6qIaFsTEfQM1TSCLQ3zfLO7CujDO0gVuZJtDP178aURlpPIt9MRhRDve4txP+8XorBrTflMkkRJFseClJhY2wvPm8PuQKGYmIIZYqbrDYLqaIMTUUrV0aKJiFnL7NVFWSWq2T6ctfb2STtes11au5jvdK4y5srkgtySa6JS25IgzyQJmkRRgR5JW9kbs2td+vD+lyOFqx855yswPr6Belyob0=</latexit>


fixed | independent<latexit sha1_base64="gYR545rgH3Kn7mTe5KJTJ+wWapM=">AAACKXicbVDLTsJAFJ36RHxVjSs3E8HEFWnZyJLEjUtM5JEAIdPpLUyYTpuZqYFUPsatfI07det3mDiULgS8yUxOzrkn9+R4MWdKO86ntbW9s7u3XzgoHh4dn5zaZ+ctFSWSQpNGPJIdjyjgTEBTM82hE0sgoceh7Y3vF3r7GaRikXjS0xj6IRkKFjBKtKEG9mXAJuDj8ksZM+FDDOYTemCXnIqTDd4Ebg5KKJ/GwP7p+RFNQuOlnCjVdZ1Y91MiNaMcZsVeoiAmdEyG0DVQkBBUP83iz/CNYXwcRNI8oXHG/nWkJFRqGnpmMyR6pNa1Bfmf1k10UOunTMSJBkGXh4KEYx3hRRfYZxKo5lMDCJXMZMV0RCSh2jS2cmUoSTxidDJbZUFkuYqmL3e9nU3QqlZcp+I+Vkv1Wt5cAV2ha3SLXHSH6ugBNVATUZSiV/SG5tbcerc+rK/l6paVey7Qyljfv8nSp04=</latexit><latexit sha1_base64="gYR545rgH3Kn7mTe5KJTJ+wWapM=">AAACKXicbVDLTsJAFJ36RHxVjSs3E8HEFWnZyJLEjUtM5JEAIdPpLUyYTpuZqYFUPsatfI07det3mDiULgS8yUxOzrkn9+R4MWdKO86ntbW9s7u3XzgoHh4dn5zaZ+ctFSWSQpNGPJIdjyjgTEBTM82hE0sgoceh7Y3vF3r7GaRikXjS0xj6IRkKFjBKtKEG9mXAJuDj8ksZM+FDDOYTemCXnIqTDd4Ebg5KKJ/GwP7p+RFNQuOlnCjVdZ1Y91MiNaMcZsVeoiAmdEyG0DVQkBBUP83iz/CNYXwcRNI8oXHG/nWkJFRqGnpmMyR6pNa1Bfmf1k10UOunTMSJBkGXh4KEYx3hRRfYZxKo5lMDCJXMZMV0RCSh2jS2cmUoSTxidDJbZUFkuYqmL3e9nU3QqlZcp+I+Vkv1Wt5cAV2ha3SLXHSH6ugBNVATUZSiV/SG5tbcerc+rK/l6paVey7Qyljfv8nSp04=</latexit><latexit sha1_base64="gYR545rgH3Kn7mTe5KJTJ+wWapM=">AAACKXicbVDLTsJAFJ36RHxVjSs3E8HEFWnZyJLEjUtM5JEAIdPpLUyYTpuZqYFUPsatfI07det3mDiULgS8yUxOzrkn9+R4MWdKO86ntbW9s7u3XzgoHh4dn5zaZ+ctFSWSQpNGPJIdjyjgTEBTM82hE0sgoceh7Y3vF3r7GaRikXjS0xj6IRkKFjBKtKEG9mXAJuDj8ksZM+FDDOYTemCXnIqTDd4Ebg5KKJ/GwP7p+RFNQuOlnCjVdZ1Y91MiNaMcZsVeoiAmdEyG0DVQkBBUP83iz/CNYXwcRNI8oXHG/nWkJFRqGnpmMyR6pNa1Bfmf1k10UOunTMSJBkGXh4KEYx3hRRfYZxKo5lMDCJXMZMV0RCSh2jS2cmUoSTxidDJbZUFkuYqmL3e9nU3QqlZcp+I+Vkv1Wt5cAV2ha3SLXHSH6ugBNVATUZSiV/SG5tbcerc+rK/l6paVey7Qyljfv8nSp04=</latexit><latexit sha1_base64="gYR545rgH3Kn7mTe5KJTJ+wWapM=">AAACKXicbVDLTsJAFJ36RHxVjSs3E8HEFWnZyJLEjUtM5JEAIdPpLUyYTpuZqYFUPsatfI07det3mDiULgS8yUxOzrkn9+R4MWdKO86ntbW9s7u3XzgoHh4dn5zaZ+ctFSWSQpNGPJIdjyjgTEBTM82hE0sgoceh7Y3vF3r7GaRikXjS0xj6IRkKFjBKtKEG9mXAJuDj8ksZM+FDDOYTemCXnIqTDd4Ebg5KKJ/GwP7p+RFNQuOlnCjVdZ1Y91MiNaMcZsVeoiAmdEyG0DVQkBBUP83iz/CNYXwcRNI8oXHG/nWkJFRqGnpmMyR6pNa1Bfmf1k10UOunTMSJBkGXh4KEYx3hRRfYZxKo5lMDCJXMZMV0RCSh2jS2cmUoSTxidDJbZUFkuYqmL3e9nU3QqlZcp+I+Vkv1Wt5cAV2ha3SLXHSH6ugBNVATUZSiV/SG5tbcerc+rK/l6paVey7Qyljfv8nSp04=</latexit>

zm<latexit sha1_base64="FQUFB8LNxtSLjqOl+XLS2P4F8MM=">AAACKHicbVDNTgIxGGz9RRQFPXppBBNPZJeLHkm8eMTEBRLYkG7pQkPb3bRdI272GbzqK/g03gxXn8SycHDBSZpMZr4v33SCmDNtHGcBd3b39g8OS0fl45PK6Vm1dt7VUaII9UjEI9UPsKacSeoZZjjtx4piEXDaC2b3S7/3TJVmkXwy85j6Ak8kCxnBxkpe43UkGqNq3Wk6OdA2cdekDtbojGqwMhxHJBFUGsKx1gPXiY2fYmUY4TQrDxNNY0xmeEIHlkosqPbTPG2Grq0yRmGk7JMG5erfjRQLrecisJMCm6ne9Jbif94gMeGdnzIZJ4ZKsjoUJhyZCC2/jsZMUWL43BJMFLNZEZlihYmxBRWuTBSOp4y8ZEWVyjxXQQxEVrYFupt1bZNuq+k6TfexVW+31lWWwCW4AjfABbegDR5AB3iAAAbewDv4gJ/wC37DxWp0B653LkAB8OcXcdKmEA==</latexit><latexit sha1_base64="FQUFB8LNxtSLjqOl+XLS2P4F8MM=">AAACKHicbVDNTgIxGGz9RRQFPXppBBNPZJeLHkm8eMTEBRLYkG7pQkPb3bRdI272GbzqK/g03gxXn8SycHDBSZpMZr4v33SCmDNtHGcBd3b39g8OS0fl45PK6Vm1dt7VUaII9UjEI9UPsKacSeoZZjjtx4piEXDaC2b3S7/3TJVmkXwy85j6Ak8kCxnBxkpe43UkGqNq3Wk6OdA2cdekDtbojGqwMhxHJBFUGsKx1gPXiY2fYmUY4TQrDxNNY0xmeEIHlkosqPbTPG2Grq0yRmGk7JMG5erfjRQLrecisJMCm6ne9Jbif94gMeGdnzIZJ4ZKsjoUJhyZCC2/jsZMUWL43BJMFLNZEZlihYmxBRWuTBSOp4y8ZEWVyjxXQQxEVrYFupt1bZNuq+k6TfexVW+31lWWwCW4AjfABbegDR5AB3iAAAbewDv4gJ/wC37DxWp0B653LkAB8OcXcdKmEA==</latexit><latexit sha1_base64="FQUFB8LNxtSLjqOl+XLS2P4F8MM=">AAACKHicbVDNTgIxGGz9RRQFPXppBBNPZJeLHkm8eMTEBRLYkG7pQkPb3bRdI272GbzqK/g03gxXn8SycHDBSZpMZr4v33SCmDNtHGcBd3b39g8OS0fl45PK6Vm1dt7VUaII9UjEI9UPsKacSeoZZjjtx4piEXDaC2b3S7/3TJVmkXwy85j6Ak8kCxnBxkpe43UkGqNq3Wk6OdA2cdekDtbojGqwMhxHJBFUGsKx1gPXiY2fYmUY4TQrDxNNY0xmeEIHlkosqPbTPG2Grq0yRmGk7JMG5erfjRQLrecisJMCm6ne9Jbif94gMeGdnzIZJ4ZKsjoUJhyZCC2/jsZMUWL43BJMFLNZEZlihYmxBRWuTBSOp4y8ZEWVyjxXQQxEVrYFupt1bZNuq+k6TfexVW+31lWWwCW4AjfABbegDR5AB3iAAAbewDv4gJ/wC37DxWp0B653LkAB8OcXcdKmEA==</latexit><latexit sha1_base64="FQUFB8LNxtSLjqOl+XLS2P4F8MM=">AAACKHicbVDNTgIxGGz9RRQFPXppBBNPZJeLHkm8eMTEBRLYkG7pQkPb3bRdI272GbzqK/g03gxXn8SycHDBSZpMZr4v33SCmDNtHGcBd3b39g8OS0fl45PK6Vm1dt7VUaII9UjEI9UPsKacSeoZZjjtx4piEXDaC2b3S7/3TJVmkXwy85j6Ak8kCxnBxkpe43UkGqNq3Wk6OdA2cdekDtbojGqwMhxHJBFUGsKx1gPXiY2fYmUY4TQrDxNNY0xmeEIHlkosqPbTPG2Grq0yRmGk7JMG5erfjRQLrecisJMCm6ne9Jbif94gMeGdnzIZJ4ZKsjoUJhyZCC2/jsZMUWL43BJMFLNZEZlihYmxBRWuTBSOp4y8ZEWVyjxXQQxEVrYFupt1bZNuq+k6TfexVW+31lWWwCW4AjfABbegDR5AB3iAAAbewDv4gJ/wC37DxWp0B653LkAB8OcXcdKmEA==</latexit>

p✓(z) =Q

n p✓(zn|zn+1)<latexit sha1_base64="UhYXzIE19f4UKdU35cK5F+u1svA=">AAACbXicbVDRShtBFJ2srU1jtdHSJ0sZjKIihN28mBch4EseFRoVsmGZndxNhszODjN3xWTdX/Jr+tKH+gv+gpMYsNEeGDhzzr3ce0+spbDo+38q3tqHj+ufqp9rG182t77Wt3eubJYbDj2eyczcxMyCFAp6KFDCjTbA0ljCdTw5n/vXt2CsyNQvnGoYpGykRCI4QydF9e6+jkIcAzJ6FKYMx3FSzMpjekZDbbJhpOirX4RxQmdlVKiS3tPX30lQHu9H9Ybf9Beg70mwJA2yxEW0XdkMhxnPU1DIJbO2H/gaBwUzKLiEshbmFjTjEzaCvqOKpWAHxeLkkh44ZUiTzLinkC7UfzsKllo7TWNXOb/KvvXm4v+8fo5Je1AIpXMExV8GJbmkmNF5fnQoDHCUU0cYN8LtSvmYGcbRpbwyZWSYHgt+V66qoBZ7rYhxWtZcgMHbuN6Tq1Yz8JvBZavRaS+jrJJdskeOSEBOSYd0yQXpEU4eyG/ylzxWnrzv3g/v50upV1n2fCMr8A6fAb9fvNA=</latexit><latexit sha1_base64="UhYXzIE19f4UKdU35cK5F+u1svA=">AAACbXicbVDRShtBFJ2srU1jtdHSJ0sZjKIihN28mBch4EseFRoVsmGZndxNhszODjN3xWTdX/Jr+tKH+gv+gpMYsNEeGDhzzr3ce0+spbDo+38q3tqHj+ufqp9rG182t77Wt3eubJYbDj2eyczcxMyCFAp6KFDCjTbA0ljCdTw5n/vXt2CsyNQvnGoYpGykRCI4QydF9e6+jkIcAzJ6FKYMx3FSzMpjekZDbbJhpOirX4RxQmdlVKiS3tPX30lQHu9H9Ybf9Beg70mwJA2yxEW0XdkMhxnPU1DIJbO2H/gaBwUzKLiEshbmFjTjEzaCvqOKpWAHxeLkkh44ZUiTzLinkC7UfzsKllo7TWNXOb/KvvXm4v+8fo5Je1AIpXMExV8GJbmkmNF5fnQoDHCUU0cYN8LtSvmYGcbRpbwyZWSYHgt+V66qoBZ7rYhxWtZcgMHbuN6Tq1Yz8JvBZavRaS+jrJJdskeOSEBOSYd0yQXpEU4eyG/ylzxWnrzv3g/v50upV1n2fCMr8A6fAb9fvNA=</latexit><latexit sha1_base64="UhYXzIE19f4UKdU35cK5F+u1svA=">AAACbXicbVDRShtBFJ2srU1jtdHSJ0sZjKIihN28mBch4EseFRoVsmGZndxNhszODjN3xWTdX/Jr+tKH+gv+gpMYsNEeGDhzzr3ce0+spbDo+38q3tqHj+ufqp9rG182t77Wt3eubJYbDj2eyczcxMyCFAp6KFDCjTbA0ljCdTw5n/vXt2CsyNQvnGoYpGykRCI4QydF9e6+jkIcAzJ6FKYMx3FSzMpjekZDbbJhpOirX4RxQmdlVKiS3tPX30lQHu9H9Ybf9Beg70mwJA2yxEW0XdkMhxnPU1DIJbO2H/gaBwUzKLiEshbmFjTjEzaCvqOKpWAHxeLkkh44ZUiTzLinkC7UfzsKllo7TWNXOb/KvvXm4v+8fo5Je1AIpXMExV8GJbmkmNF5fnQoDHCUU0cYN8LtSvmYGcbRpbwyZWSYHgt+V66qoBZ7rYhxWtZcgMHbuN6Tq1Yz8JvBZavRaS+jrJJdskeOSEBOSYd0yQXpEU4eyG/ylzxWnrzv3g/v50upV1n2fCMr8A6fAb9fvNA=</latexit><latexit sha1_base64="UhYXzIE19f4UKdU35cK5F+u1svA=">AAACbXicbVDRShtBFJ2srU1jtdHSJ0sZjKIihN28mBch4EseFRoVsmGZndxNhszODjN3xWTdX/Jr+tKH+gv+gpMYsNEeGDhzzr3ce0+spbDo+38q3tqHj+ufqp9rG182t77Wt3eubJYbDj2eyczcxMyCFAp6KFDCjTbA0ljCdTw5n/vXt2CsyNQvnGoYpGykRCI4QydF9e6+jkIcAzJ6FKYMx3FSzMpjekZDbbJhpOirX4RxQmdlVKiS3tPX30lQHu9H9Ybf9Beg70mwJA2yxEW0XdkMhxnPU1DIJbO2H/gaBwUzKLiEshbmFjTjEzaCvqOKpWAHxeLkkh44ZUiTzLinkC7UfzsKllo7TWNXOb/KvvXm4v+8fo5Je1AIpXMExV8GJbmkmNF5fnQoDHCUU0cYN8LtSvmYGcbRpbwyZWSYHgt+V66qoBZ7rYhxWtZcgMHbuN6Tq1Yz8JvBZavRaS+jrJJdskeOSEBOSYd0yQXpEU4eyG/ylzxWnrzv3g/v50upV1n2fCMr8A6fAb9fvNA=</latexit>

zn<latexit sha1_base64="C9SlwxFRYjXvyZMtIqwdEOkDqc4=">AAACLnicbVDLSsNAFJ34rNVqq0s3g63gqiTd6LLgxmUF+4A0lMl00g6dR5iZiDXkM9zqL/g1ggtx62c4TbswrQcuHM65l3s4YcyoNq776Wxt7+zu7ZcOyodHleOTau20p2WiMOliyaQahEgTRgXpGmoYGcSKIB4y0g9ntwu//0iUplI8mHlMAo4mgkYUI2Mlv5EOwwg+ZyPRGFXrbtPNATeJtyJ1sEJnVHMqw7HECSfCYIa09j03NkGKlKGYkaw8TDSJEZ6hCfEtFYgTHaR55gxeWmUMI6nsCANz9e9FirjWcx7aTY7MVK97C/E/z09MdBOkVMSJIQIvH0UJg0bCRQFwTBXBhs0tQVhRmxXiKVIIG1tT4ctEoXhK8VNWVInIcxXEkGdlW6C3Xtcm6bWantv07lv1dmtVZQmcgwtwBTxwDdrgDnRAF2AgwQt4BW/Ou/PhfDnfy9UtZ3VzBgpwfn4BS5qoiQ==</latexit><latexit sha1_base64="C9SlwxFRYjXvyZMtIqwdEOkDqc4=">AAACLnicbVDLSsNAFJ34rNVqq0s3g63gqiTd6LLgxmUF+4A0lMl00g6dR5iZiDXkM9zqL/g1ggtx62c4TbswrQcuHM65l3s4YcyoNq776Wxt7+zu7ZcOyodHleOTau20p2WiMOliyaQahEgTRgXpGmoYGcSKIB4y0g9ntwu//0iUplI8mHlMAo4mgkYUI2Mlv5EOwwg+ZyPRGFXrbtPNATeJtyJ1sEJnVHMqw7HECSfCYIa09j03NkGKlKGYkaw8TDSJEZ6hCfEtFYgTHaR55gxeWmUMI6nsCANz9e9FirjWcx7aTY7MVK97C/E/z09MdBOkVMSJIQIvH0UJg0bCRQFwTBXBhs0tQVhRmxXiKVIIG1tT4ctEoXhK8VNWVInIcxXEkGdlW6C3Xtcm6bWantv07lv1dmtVZQmcgwtwBTxwDdrgDnRAF2AgwQt4BW/Ou/PhfDnfy9UtZ3VzBgpwfn4BS5qoiQ==</latexit><latexit sha1_base64="C9SlwxFRYjXvyZMtIqwdEOkDqc4=">AAACLnicbVDLSsNAFJ34rNVqq0s3g63gqiTd6LLgxmUF+4A0lMl00g6dR5iZiDXkM9zqL/g1ggtx62c4TbswrQcuHM65l3s4YcyoNq776Wxt7+zu7ZcOyodHleOTau20p2WiMOliyaQahEgTRgXpGmoYGcSKIB4y0g9ntwu//0iUplI8mHlMAo4mgkYUI2Mlv5EOwwg+ZyPRGFXrbtPNATeJtyJ1sEJnVHMqw7HECSfCYIa09j03NkGKlKGYkaw8TDSJEZ6hCfEtFYgTHaR55gxeWmUMI6nsCANz9e9FirjWcx7aTY7MVK97C/E/z09MdBOkVMSJIQIvH0UJg0bCRQFwTBXBhs0tQVhRmxXiKVIIG1tT4ctEoXhK8VNWVInIcxXEkGdlW6C3Xtcm6bWantv07lv1dmtVZQmcgwtwBTxwDdrgDnRAF2AgwQt4BW/Ou/PhfDnfy9UtZ3VzBgpwfn4BS5qoiQ==</latexit><latexit sha1_base64="C9SlwxFRYjXvyZMtIqwdEOkDqc4=">AAACLnicbVDLSsNAFJ34rNVqq0s3g63gqiTd6LLgxmUF+4A0lMl00g6dR5iZiDXkM9zqL/g1ggtx62c4TbswrQcuHM65l3s4YcyoNq776Wxt7+zu7ZcOyodHleOTau20p2WiMOliyaQahEgTRgXpGmoYGcSKIB4y0g9ntwu//0iUplI8mHlMAo4mgkYUI2Mlv5EOwwg+ZyPRGFXrbtPNATeJtyJ1sEJnVHMqw7HECSfCYIa09j03NkGKlKGYkaw8TDSJEZ6hCfEtFYgTHaR55gxeWmUMI6nsCANz9e9FirjWcx7aTY7MVK97C/E/z09MdBOkVMSJIQIvH0UJg0bCRQFwTBXBhs0tQVhRmxXiKVIIG1tT4ctEoXhK8VNWVInIcxXEkGdlW6C3Xtcm6bWantv07lv1dmtVZQmcgwtwBTxwDdrgDnRAF2AgwQt4BW/Ou/PhfDnfy9UtZ3VzBgpwfn4BS5qoiQ==</latexit>

zn+1<latexit sha1_base64="rHCOl6RFYoxdcBQQIKMJMWSVD5s=">AAACNHicbVDLSsNAFJ34rNXaVpduBltBEErSjS4LblxWsA9oQ5hMJ+3QmUmYmYg15Evc6i/4L4I7ces3OE2zMK0HLhzOuZd7OH7EqNK2/WFtbe/s7u2XDsqHR5Xjaq1+0ldhLDHp4ZCFcugjRRgVpKepZmQYSYK4z8jAn98u/cEjkYqG4kEvIuJyNBU0oBhpI3m1ajMZ+wF8Tr1EXDlp06s17JadAW4SJycNkKPr1a3KeBLimBOhMUNKjRw70m6CpKaYkbQ8jhWJEJ6jKRkZKhAnyk2y5Cm8MMoEBqE0IzTM1L8XCeJKLbhvNjnSM7XuLcX/vFGsgxs3oSKKNRF49SiIGdQhXNYAJ1QSrNnCEIQlNVkhniGJsDZlFb5MJYpmFD+lRZWILFdB9HlaNgU663Vtkn675dgt577d6LTzKkvgDJyDS+CAa9ABd6ALegCDGLyAV/BmvVuf1pf1vVrdsvKbU1CA9fMLxJGqNg==</latexit><latexit sha1_base64="rHCOl6RFYoxdcBQQIKMJMWSVD5s=">AAACNHicbVDLSsNAFJ34rNXaVpduBltBEErSjS4LblxWsA9oQ5hMJ+3QmUmYmYg15Evc6i/4L4I7ces3OE2zMK0HLhzOuZd7OH7EqNK2/WFtbe/s7u2XDsqHR5Xjaq1+0ldhLDHp4ZCFcugjRRgVpKepZmQYSYK4z8jAn98u/cEjkYqG4kEvIuJyNBU0oBhpI3m1ajMZ+wF8Tr1EXDlp06s17JadAW4SJycNkKPr1a3KeBLimBOhMUNKjRw70m6CpKaYkbQ8jhWJEJ6jKRkZKhAnyk2y5Cm8MMoEBqE0IzTM1L8XCeJKLbhvNjnSM7XuLcX/vFGsgxs3oSKKNRF49SiIGdQhXNYAJ1QSrNnCEIQlNVkhniGJsDZlFb5MJYpmFD+lRZWILFdB9HlaNgU663Vtkn675dgt577d6LTzKkvgDJyDS+CAa9ABd6ALegCDGLyAV/BmvVuf1pf1vVrdsvKbU1CA9fMLxJGqNg==</latexit><latexit sha1_base64="rHCOl6RFYoxdcBQQIKMJMWSVD5s=">AAACNHicbVDLSsNAFJ34rNXaVpduBltBEErSjS4LblxWsA9oQ5hMJ+3QmUmYmYg15Evc6i/4L4I7ces3OE2zMK0HLhzOuZd7OH7EqNK2/WFtbe/s7u2XDsqHR5Xjaq1+0ldhLDHp4ZCFcugjRRgVpKepZmQYSYK4z8jAn98u/cEjkYqG4kEvIuJyNBU0oBhpI3m1ajMZ+wF8Tr1EXDlp06s17JadAW4SJycNkKPr1a3KeBLimBOhMUNKjRw70m6CpKaYkbQ8jhWJEJ6jKRkZKhAnyk2y5Cm8MMoEBqE0IzTM1L8XCeJKLbhvNjnSM7XuLcX/vFGsgxs3oSKKNRF49SiIGdQhXNYAJ1QSrNnCEIQlNVkhniGJsDZlFb5MJYpmFD+lRZWILFdB9HlaNgU663Vtkn675dgt577d6LTzKkvgDJyDS+CAa9ABd6ALegCDGLyAV/BmvVuf1pf1vVrdsvKbU1CA9fMLxJGqNg==</latexit><latexit sha1_base64="rHCOl6RFYoxdcBQQIKMJMWSVD5s=">AAACNHicbVDLSsNAFJ34rNXaVpduBltBEErSjS4LblxWsA9oQ5hMJ+3QmUmYmYg15Evc6i/4L4I7ces3OE2zMK0HLhzOuZd7OH7EqNK2/WFtbe/s7u2XDsqHR5Xjaq1+0ldhLDHp4ZCFcugjRRgVpKepZmQYSYK4z8jAn98u/cEjkYqG4kEvIuJyNBU0oBhpI3m1ajMZ+wF8Tr1EXDlp06s17JadAW4SJycNkKPr1a3KeBLimBOhMUNKjRw70m6CpKaYkbQ8jhWJEJ6jKRkZKhAnyk2y5Cm8MMoEBqE0IzTM1L8XCeJKLbhvNjnSM7XuLcX/vFGsgxs3oSKKNRF49SiIGdQhXNYAJ1QSrNnCEIQlNVkhniGJsDZlFb5MJYpmFD+lRZWILFdB9HlaNgU663Vtkn675dgt577d6LTzKkvgDJyDS+CAa9ABd6ALegCDGLyAV/BmvVuf1pf1vVrdsvKbU1CA9fMLxJGqNg==</latexit>

(a) Traditional Latent Variable Models


c<latexit sha1_base64="L5vsFdfC3adaiAy6SN/mhTZlff4=">AAACGXicbVC7TgJBFL2LL8QXamkzEUysyC6NlkQbS0zkkcCGzA53YcLs7GZm1kgIH2ErX2NnbK38GBOHhULAk0xycs69uWdOkAiujet+O7mt7Z3dvfx+4eDw6PikeHrW1HGqGDZYLGLVDqhGwSU2DDcC24lCGgUCW8Hofu63nlFpHssnM07Qj+hA8pAzaqzUKneDkLByr1hyK24Gskm8JSnBEvVe8afbj1kaoTRMUK07npsYf0KV4UzgtNBNNSaUjegAO5ZKGqH2J1ncKbmySp+EsbJPGpKpfzcmNNJ6HAV2MqJmqNe9ufif10lNeOtPuExSg5ItDoWpICYm87+TPlfIjBhbQpniNithQ6ooM7ahlSsDRZMhZy/TVRVllqtg+/LW29kkzWrFcyveY7VUu1s2l4cLuIRr8OAGavAAdWgAgxG8whvMnJnz7nw4n4vRnLPcOYcVOF+/awSg6A==</latexit><latexit sha1_base64="L5vsFdfC3adaiAy6SN/mhTZlff4=">AAACGXicbVC7TgJBFL2LL8QXamkzEUysyC6NlkQbS0zkkcCGzA53YcLs7GZm1kgIH2ErX2NnbK38GBOHhULAk0xycs69uWdOkAiujet+O7mt7Z3dvfx+4eDw6PikeHrW1HGqGDZYLGLVDqhGwSU2DDcC24lCGgUCW8Hofu63nlFpHssnM07Qj+hA8pAzaqzUKneDkLByr1hyK24Gskm8JSnBEvVe8afbj1kaoTRMUK07npsYf0KV4UzgtNBNNSaUjegAO5ZKGqH2J1ncKbmySp+EsbJPGpKpfzcmNNJ6HAV2MqJmqNe9ufif10lNeOtPuExSg5ItDoWpICYm87+TPlfIjBhbQpniNithQ6ooM7ahlSsDRZMhZy/TVRVllqtg+/LW29kkzWrFcyveY7VUu1s2l4cLuIRr8OAGavAAdWgAgxG8whvMnJnz7nw4n4vRnLPcOYcVOF+/awSg6A==</latexit><latexit sha1_base64="L5vsFdfC3adaiAy6SN/mhTZlff4=">AAACGXicbVC7TgJBFL2LL8QXamkzEUysyC6NlkQbS0zkkcCGzA53YcLs7GZm1kgIH2ErX2NnbK38GBOHhULAk0xycs69uWdOkAiujet+O7mt7Z3dvfx+4eDw6PikeHrW1HGqGDZYLGLVDqhGwSU2DDcC24lCGgUCW8Hofu63nlFpHssnM07Qj+hA8pAzaqzUKneDkLByr1hyK24Gskm8JSnBEvVe8afbj1kaoTRMUK07npsYf0KV4UzgtNBNNSaUjegAO5ZKGqH2J1ncKbmySp+EsbJPGpKpfzcmNNJ6HAV2MqJmqNe9ufif10lNeOtPuExSg5ItDoWpICYm87+TPlfIjBhbQpniNithQ6ooM7ahlSsDRZMhZy/TVRVllqtg+/LW29kkzWrFcyveY7VUu1s2l4cLuIRr8OAGavAAdWgAgxG8whvMnJnz7nw4n4vRnLPcOYcVOF+/awSg6A==</latexit><latexit sha1_base64="L5vsFdfC3adaiAy6SN/mhTZlff4=">AAACGXicbVC7TgJBFL2LL8QXamkzEUysyC6NlkQbS0zkkcCGzA53YcLs7GZm1kgIH2ErX2NnbK38GBOHhULAk0xycs69uWdOkAiujet+O7mt7Z3dvfx+4eDw6PikeHrW1HGqGDZYLGLVDqhGwSU2DDcC24lCGgUCW8Hofu63nlFpHssnM07Qj+hA8pAzaqzUKneDkLByr1hyK24Gskm8JSnBEvVe8afbj1kaoTRMUK07npsYf0KV4UzgtNBNNSaUjegAO5ZKGqH2J1ncKbmySp+EsbJPGpKpfzcmNNJ6HAV2MqJmqNe9ufif10lNeOtPuExSg5ItDoWpICYm87+TPlfIjBhbQpniNithQ6ooM7ahlSsDRZMhZy/TVRVllqtg+/LW29kkzWrFcyveY7VUu1s2l4cLuIRr8OAGavAAdWgAgxG8whvMnJnz7nw4n4vRnLPcOYcVOF+/awSg6A==</latexit> ✓<latexit sha1_base64="xWmXra5P90YYt2hGo0QW8XWb5is=">AAACGnicbVDLTsJAFJ3iC/GFunTTCCauSMtGl0Q3LjGRRwINmQ63dMJ02szcGgnhJ9zK17gzbt34MSYOpQsBTzLJyTn35p45fiK4Rsf5tgpb2zu7e8X90sHh0fFJ+fSsreNUMWixWMSq61MNgktoIUcB3UQBjXwBHX98v/A7z6A0j+UTThLwIjqSPOCMopG61T6GgLQ6KFecmpPB3iRuTiokR3NQ/ukPY5ZGIJEJqnXPdRL0plQhZwJmpX6qIaFsTEfQM1TSCLQ3zfLO7CujDO0gVuZJtDP178aURlpPIt9MRhRDve4txP+8XorBrTflMkkRJFseClJhY2wvPm8PuQKGYmIIZYqbrDYLqaIMTUUrV0aKJiFnL7NVFWSWq2T6ctfb2STtes11au5jvdK4y5srkgtySa6JS25IgzyQJmkRRgR5JW9kbs2td+vD+lyOFqx855yswPr6Belyob0=</latexit><latexit sha1_base64="xWmXra5P90YYt2hGo0QW8XWb5is=">AAACGnicbVDLTsJAFJ3iC/GFunTTCCauSMtGl0Q3LjGRRwINmQ63dMJ02szcGgnhJ9zK17gzbt34MSYOpQsBTzLJyTn35p45fiK4Rsf5tgpb2zu7e8X90sHh0fFJ+fSsreNUMWixWMSq61MNgktoIUcB3UQBjXwBHX98v/A7z6A0j+UTThLwIjqSPOCMopG61T6GgLQ6KFecmpPB3iRuTiokR3NQ/ukPY5ZGIJEJqnXPdRL0plQhZwJmpX6qIaFsTEfQM1TSCLQ3zfLO7CujDO0gVuZJtDP178aURlpPIt9MRhRDve4txP+8XorBrTflMkkRJFseClJhY2wvPm8PuQKGYmIIZYqbrDYLqaIMTUUrV0aKJiFnL7NVFWSWq2T6ctfb2STtes11au5jvdK4y5srkgtySa6JS25IgzyQJmkRRgR5JW9kbs2td+vD+lyOFqx855yswPr6Belyob0=</latexit><latexit sha1_base64="xWmXra5P90YYt2hGo0QW8XWb5is=">AAACGnicbVDLTsJAFJ3iC/GFunTTCCauSMtGl0Q3LjGRRwINmQ63dMJ02szcGgnhJ9zK17gzbt34MSYOpQsBTzLJyTn35p45fiK4Rsf5tgpb2zu7e8X90sHh0fFJ+fSsreNUMWixWMSq61MNgktoIUcB3UQBjXwBHX98v/A7z6A0j+UTThLwIjqSPOCMopG61T6GgLQ6KFecmpPB3iRuTiokR3NQ/ukPY5ZGIJEJqnXPdRL0plQhZwJmpX6qIaFsTEfQM1TSCLQ3zfLO7CujDO0gVuZJtDP178aURlpPIt9MRhRDve4txP+8XorBrTflMkkRJFseClJhY2wvPm8PuQKGYmIIZYqbrDYLqaIMTUUrV0aKJiFnL7NVFWSWq2T6ctfb2STtes11au5jvdK4y5srkgtySa6JS25IgzyQJmkRRgR5JW9kbs2td+vD+lyOFqx855yswPr6Belyob0=</latexit><latexit sha1_base64="xWmXra5P90YYt2hGo0QW8XWb5is=">AAACGnicbVDLTsJAFJ3iC/GFunTTCCauSMtGl0Q3LjGRRwINmQ63dMJ02szcGgnhJ9zK17gzbt34MSYOpQsBTzLJyTn35p45fiK4Rsf5tgpb2zu7e8X90sHh0fFJ+fSsreNUMWixWMSq61MNgktoIUcB3UQBjXwBHX98v/A7z6A0j+UTThLwIjqSPOCMopG61T6GgLQ6KFecmpPB3iRuTiokR3NQ/ukPY5ZGIJEJqnXPdRL0plQhZwJmpX6qIaFsTEfQM1TSCLQ3zfLO7CujDO0gVuZJtDP178aURlpPIt9MRhRDve4txP+8XorBrTflMkkRJFseClJhY2wvPm8PuQKGYmIIZYqbrDYLqaIMTUUrV0aKJiFnL7NVFWSWq2T6ctfb2STtes11au5jvdK4y5srkgtySa6JS25IgzyQJmkRRgR5JW9kbs2td+vD+lyOFqx855yswPr6Belyob0=</latexit>




Graph VAE<latexit sha1_base64="Rj/FAiJJYYuonId+cd7GHSaDnUc=">AAACG3icbVDLSsNAFL2pr1pfVZduBovgqiTd6EoqIrqsYB/ShjKZTtqhk0mYmYgl9Cvc2q9xJ25d+DGCkzQL23pg4HDOvdwzx4s4U9q2v63C2vrG5lZxu7Szu7d/UD48aqkwloQ2SchD2fGwopwJ2tRMc9qJJMWBx2nbG9+kfvuZSsVC8agnEXUDPBTMZwRrIz3dSRyNUOv6tl+u2FU7A1olTk4qkKPRL//0BiGJAyo04ViprmNH2k2w1IxwOi31YkUjTMZ4SLuGChxQ5SZZ4Ck6M8oA+aE0T2iUqX83EhwoNQk8MxlgPVLLXir+53Vj7V+6CRNRrKkg80N+zJEOUfp7NGCSEs0nhmAimcmKyAhLTLTpaOHKMG2GkZfpokpFlqtk+nKW21klrVrVsavOQ61Sv8qbK8IJnMI5OHABdbiHBjSBQACv8AYza2a9Wx/W53y0YOU7x7AA6+sXaC+h9w==</latexit><latexit sha1_base64="Rj/FAiJJYYuonId+cd7GHSaDnUc=">AAACG3icbVDLSsNAFL2pr1pfVZduBovgqiTd6EoqIrqsYB/ShjKZTtqhk0mYmYgl9Cvc2q9xJ25d+DGCkzQL23pg4HDOvdwzx4s4U9q2v63C2vrG5lZxu7Szu7d/UD48aqkwloQ2SchD2fGwopwJ2tRMc9qJJMWBx2nbG9+kfvuZSsVC8agnEXUDPBTMZwRrIz3dSRyNUOv6tl+u2FU7A1olTk4qkKPRL//0BiGJAyo04ViprmNH2k2w1IxwOi31YkUjTMZ4SLuGChxQ5SZZ4Ck6M8oA+aE0T2iUqX83EhwoNQk8MxlgPVLLXir+53Vj7V+6CRNRrKkg80N+zJEOUfp7NGCSEs0nhmAimcmKyAhLTLTpaOHKMG2GkZfpokpFlqtk+nKW21klrVrVsavOQ61Sv8qbK8IJnMI5OHABdbiHBjSBQACv8AYza2a9Wx/W53y0YOU7x7AA6+sXaC+h9w==</latexit><latexit sha1_base64="Rj/FAiJJYYuonId+cd7GHSaDnUc=">AAACG3icbVDLSsNAFL2pr1pfVZduBovgqiTd6EoqIrqsYB/ShjKZTtqhk0mYmYgl9Cvc2q9xJ25d+DGCkzQL23pg4HDOvdwzx4s4U9q2v63C2vrG5lZxu7Szu7d/UD48aqkwloQ2SchD2fGwopwJ2tRMc9qJJMWBx2nbG9+kfvuZSsVC8agnEXUDPBTMZwRrIz3dSRyNUOv6tl+u2FU7A1olTk4qkKPRL//0BiGJAyo04ViprmNH2k2w1IxwOi31YkUjTMZ4SLuGChxQ5SZZ4Ck6M8oA+aE0T2iUqX83EhwoNQk8MxlgPVLLXir+53Vj7V+6CRNRrKkg80N+zJEOUfp7NGCSEs0nhmAimcmKyAhLTLTpaOHKMG2GkZfpokpFlqtk+nKW21klrVrVsavOQ61Sv8qbK8IJnMI5OHABdbiHBjSBQACv8AYza2a9Wx/W53y0YOU7x7AA6+sXaC+h9w==</latexit><latexit sha1_base64="Rj/FAiJJYYuonId+cd7GHSaDnUc=">AAACG3icbVDLSsNAFL2pr1pfVZduBovgqiTd6EoqIrqsYB/ShjKZTtqhk0mYmYgl9Cvc2q9xJ25d+DGCkzQL23pg4HDOvdwzx4s4U9q2v63C2vrG5lZxu7Szu7d/UD48aqkwloQ2SchD2fGwopwJ2tRMc9qJJMWBx2nbG9+kfvuZSsVC8agnEXUDPBTMZwRrIz3dSRyNUOv6tl+u2FU7A1olTk4qkKPRL//0BiGJAyo04ViprmNH2k2w1IxwOi31YkUjTMZ4SLuGChxQ5SZZ4Ck6M8oA+aE0T2iUqX83EhwoNQk8MxlgPVLLXir+53Vj7V+6CRNRrKkg80N+zJEOUfp7NGCSEs0nhmAimcmKyAhLTLTpaOHKMG2GkZfpokpFlqtk+nKW21klrVrVsavOQ61Sv8qbK8IJnMI5OHABdbiHBjSBQACv8AYza2a9Wx/W53y0YOU7x7AA6+sXaC+h9w==</latexit>

zn<latexit sha1_base64="C9SlwxFRYjXvyZMtIqwdEOkDqc4=">AAACLnicbVDLSsNAFJ34rNVqq0s3g63gqiTd6LLgxmUF+4A0lMl00g6dR5iZiDXkM9zqL/g1ggtx62c4TbswrQcuHM65l3s4YcyoNq776Wxt7+zu7ZcOyodHleOTau20p2WiMOliyaQahEgTRgXpGmoYGcSKIB4y0g9ntwu//0iUplI8mHlMAo4mgkYUI2Mlv5EOwwg+ZyPRGFXrbtPNATeJtyJ1sEJnVHMqw7HECSfCYIa09j03NkGKlKGYkaw8TDSJEZ6hCfEtFYgTHaR55gxeWmUMI6nsCANz9e9FirjWcx7aTY7MVK97C/E/z09MdBOkVMSJIQIvH0UJg0bCRQFwTBXBhs0tQVhRmxXiKVIIG1tT4ctEoXhK8VNWVInIcxXEkGdlW6C3Xtcm6bWantv07lv1dmtVZQmcgwtwBTxwDdrgDnRAF2AgwQt4BW/Ou/PhfDnfy9UtZ3VzBgpwfn4BS5qoiQ==</latexit><latexit sha1_base64="C9SlwxFRYjXvyZMtIqwdEOkDqc4=">AAACLnicbVDLSsNAFJ34rNVqq0s3g63gqiTd6LLgxmUF+4A0lMl00g6dR5iZiDXkM9zqL/g1ggtx62c4TbswrQcuHM65l3s4YcyoNq776Wxt7+zu7ZcOyodHleOTau20p2WiMOliyaQahEgTRgXpGmoYGcSKIB4y0g9ntwu//0iUplI8mHlMAo4mgkYUI2Mlv5EOwwg+ZyPRGFXrbtPNATeJtyJ1sEJnVHMqw7HECSfCYIa09j03NkGKlKGYkaw8TDSJEZ6hCfEtFYgTHaR55gxeWmUMI6nsCANz9e9FirjWcx7aTY7MVK97C/E/z09MdBOkVMSJIQIvH0UJg0bCRQFwTBXBhs0tQVhRmxXiKVIIG1tT4ctEoXhK8VNWVInIcxXEkGdlW6C3Xtcm6bWantv07lv1dmtVZQmcgwtwBTxwDdrgDnRAF2AgwQt4BW/Ou/PhfDnfy9UtZ3VzBgpwfn4BS5qoiQ==</latexit><latexit sha1_base64="C9SlwxFRYjXvyZMtIqwdEOkDqc4=">AAACLnicbVDLSsNAFJ34rNVqq0s3g63gqiTd6LLgxmUF+4A0lMl00g6dR5iZiDXkM9zqL/g1ggtx62c4TbswrQcuHM65l3s4YcyoNq776Wxt7+zu7ZcOyodHleOTau20p2WiMOliyaQahEgTRgXpGmoYGcSKIB4y0g9ntwu//0iUplI8mHlMAo4mgkYUI2Mlv5EOwwg+ZyPRGFXrbtPNATeJtyJ1sEJnVHMqw7HECSfCYIa09j03NkGKlKGYkaw8TDSJEZ6hCfEtFYgTHaR55gxeWmUMI6nsCANz9e9FirjWcx7aTY7MVK97C/E/z09MdBOkVMSJIQIvH0UJg0bCRQFwTBXBhs0tQVhRmxXiKVIIG1tT4ctEoXhK8VNWVInIcxXEkGdlW6C3Xtcm6bWantv07lv1dmtVZQmcgwtwBTxwDdrgDnRAF2AgwQt4BW/Ou/PhfDnfy9UtZ3VzBgpwfn4BS5qoiQ==</latexit><latexit sha1_base64="C9SlwxFRYjXvyZMtIqwdEOkDqc4=">AAACLnicbVDLSsNAFJ34rNVqq0s3g63gqiTd6LLgxmUF+4A0lMl00g6dR5iZiDXkM9zqL/g1ggtx62c4TbswrQcuHM65l3s4YcyoNq776Wxt7+zu7ZcOyodHleOTau20p2WiMOliyaQahEgTRgXpGmoYGcSKIB4y0g9ntwu//0iUplI8mHlMAo4mgkYUI2Mlv5EOwwg+ZyPRGFXrbtPNATeJtyJ1sEJnVHMqw7HECSfCYIa09j03NkGKlKGYkaw8TDSJEZ6hCfEtFYgTHaR55gxeWmUMI6nsCANz9e9FirjWcx7aTY7MVK97C/E/z09MdBOkVMSJIQIvH0UJg0bCRQFwTBXBhs0tQVhRmxXiKVIIG1tT4ctEoXhK8VNWVInIcxXEkGdlW6C3Xtcm6bWantv07lv1dmtVZQmcgwtwBTxwDdrgDnRAF2AgwQt4BW/Ou/PhfDnfy9UtZ3VzBgpwfn4BS5qoiQ==</latexit>

learned | flexible graph<latexit sha1_base64="OeurA8LYF6xtHZMoethndPNQXWs=">AAACP3icbVDLTgIxFO34RBQF3emmEUxckRk2siRx4xITeSQwIZ1yBxo6nUnbMZBxEr/Grf6Cn+EXuDNu3VkGFgKepMnJOffk3h4v4kxp2/6wtrZ3dvf2cwf5w6PC8UmxdNpWYSwptGjIQ9n1iALOBLQ00xy6kQQSeBw63uR27nceQSoWigc9i8ANyEgwn1GijTQonnMgUsAQV54q2OcwZSaJR5JE40GxbFftDHiTOEtSRks0ByWr0B+GNA5AaMqJUj3HjrSbEKkZ5ZDm+7GCiNAJGUHPUEECUG6SfSLFV0YZYj+U5gmNM/VvIiGBUrPAM5MB0WO17s3F/7xerP26mzARxRoEXSzyY451iOeN4CGTQDWfGUKoZOZWTMdEEqpNbytbsloYnaarKojsrhXRC9K8KdBZr2uTtGtVx64697Vyo76sMocu0CW6Rg66QQ10h5qohSh6Ri/oFb1Z79an9WV9L0a3rGXmDK3A+vkFG3iu5w==</latexit><latexit sha1_base64="OeurA8LYF6xtHZMoethndPNQXWs=">AAACP3icbVDLTgIxFO34RBQF3emmEUxckRk2siRx4xITeSQwIZ1yBxo6nUnbMZBxEr/Grf6Cn+EXuDNu3VkGFgKepMnJOffk3h4v4kxp2/6wtrZ3dvf2cwf5w6PC8UmxdNpWYSwptGjIQ9n1iALOBLQ00xy6kQQSeBw63uR27nceQSoWigc9i8ANyEgwn1GijTQonnMgUsAQV54q2OcwZSaJR5JE40GxbFftDHiTOEtSRks0ByWr0B+GNA5AaMqJUj3HjrSbEKkZ5ZDm+7GCiNAJGUHPUEECUG6SfSLFV0YZYj+U5gmNM/VvIiGBUrPAM5MB0WO17s3F/7xerP26mzARxRoEXSzyY451iOeN4CGTQDWfGUKoZOZWTMdEEqpNbytbsloYnaarKojsrhXRC9K8KdBZr2uTtGtVx64697Vyo76sMocu0CW6Rg66QQ10h5qohSh6Ri/oFb1Z79an9WV9L0a3rGXmDK3A+vkFG3iu5w==</latexit><latexit sha1_base64="OeurA8LYF6xtHZMoethndPNQXWs=">AAACP3icbVDLTgIxFO34RBQF3emmEUxckRk2siRx4xITeSQwIZ1yBxo6nUnbMZBxEr/Grf6Cn+EXuDNu3VkGFgKepMnJOffk3h4v4kxp2/6wtrZ3dvf2cwf5w6PC8UmxdNpWYSwptGjIQ9n1iALOBLQ00xy6kQQSeBw63uR27nceQSoWigc9i8ANyEgwn1GijTQonnMgUsAQV54q2OcwZSaJR5JE40GxbFftDHiTOEtSRks0ByWr0B+GNA5AaMqJUj3HjrSbEKkZ5ZDm+7GCiNAJGUHPUEECUG6SfSLFV0YZYj+U5gmNM/VvIiGBUrPAM5MB0WO17s3F/7xerP26mzARxRoEXSzyY451iOeN4CGTQDWfGUKoZOZWTMdEEqpNbytbsloYnaarKojsrhXRC9K8KdBZr2uTtGtVx64697Vyo76sMocu0CW6Rg66QQ10h5qohSh6Ri/oFb1Z79an9WV9L0a3rGXmDK3A+vkFG3iu5w==</latexit><latexit sha1_base64="OeurA8LYF6xtHZMoethndPNQXWs=">AAACP3icbVDLTgIxFO34RBQF3emmEUxckRk2siRx4xITeSQwIZ1yBxo6nUnbMZBxEr/Grf6Cn+EXuDNu3VkGFgKepMnJOffk3h4v4kxp2/6wtrZ3dvf2cwf5w6PC8UmxdNpWYSwptGjIQ9n1iALOBLQ00xy6kQQSeBw63uR27nceQSoWigc9i8ANyEgwn1GijTQonnMgUsAQV54q2OcwZSaJR5JE40GxbFftDHiTOEtSRks0ByWr0B+GNA5AaMqJUj3HjrSbEKkZ5ZDm+7GCiNAJGUHPUEECUG6SfSLFV0YZYj+U5gmNM/VvIiGBUrPAM5MB0WO17s3F/7xerP26mzARxRoEXSzyY451iOeN4CGTQDWfGUKoZOZWTMdEEqpNbytbsloYnaarKojsrhXRC9K8KdBZr2uTtGtVx64697Vyo76sMocu0CW6Rg66QQ10h5qohSh6Ri/oFb1Z79an9WV9L0a3rGXmDK3A+vkFG3iu5w==</latexit>

p✓(z|c) =QN

n=1 p✓(zn|zpa(n), cpa(n),n)<latexit sha1_base64="L1Gq4rFGVX/tI76oD2wHMomV1h8=">AAACsXicbVHNbhMxEPYufyVQSOGCxMUkQkqkKtqFQ3uJFIkLJ1Qk0hZlw+J1ZhMrXq9lz1YNi9+Ah+C1eA2eAG9SpCTtSJY+f998M6OZTEthMYr+BOG9+w8ePjp43Hry9PDZ8/bRi3NbVobDmJeyNJcZsyCFgjEKlHCpDbAik3CRLT80+sUVGCtK9QVXGqYFmyuRC87QU2n7l04TXAAy2ksKhossr384+pP+/3DXp0OaaFPO0loNY/et/uToXSYvbxsbIkG4RlPUmrme6rvjrar74jFVrp+2u9EgWge9DeIb0B11hvo3IeQsPQoOk1nJqwIUcsmsncSRxmnNDAouwbWSyoJmfMnmMPFQsQLstF6vzdG3npnRvDT+KaRrdttRs8LaVZH5zGZuu6815F3apML8dFoLpSsExTeN8kpSLGlzAzoTBjjKlQeMG+FnpXzBDOPoL7XTZW6YXgh+7XZZUOu5dsiscC2/wHh/XbfB+btBHA3iz3F3dEo2cUBekw7pkZickBH5SM7ImHDyN3gVvAk64fvwa/g9zDapYXDjeUl2Ilz+A8zU2LE=</latexit><latexit sha1_base64="fQpv7JZtpfP0XLbEFHyrbCrR/b0=">AAACsXicbVHNbhMxEPYufyVQmsIFiYtJhJRIVbQLB3qJFIkLJ1Qk0hZlw+J1ZhMrXq9lz6KGxW/AW/BQiNfgCfAmRUrSjmTp8/fNNzOaybQUFqPoTxDeuXvv/oODh61Hjw+fHLWPn57bsjIcxryUpbnMmAUpFIxRoIRLbYAVmYSLbPmu0S++gbGiVJ9wpWFasLkSueAMPZW2f+o0wQUgo72kYLjI8vq7oz/o/w93fTqkiTblLK3VMHZf6g+O3mby8raxIRKEKzRFrZnrqb472aq6L55Q5fppuxsNonXQmyC+Bt1RZ6h//W4Nz9Lj4DCZlbwqQCGXzNpJHGmc1syg4BJcK6ksaMaXbA4TDxUrwE7r9docfeWZGc1L459Cuma3HTUrrF0Vmc9s5rb7WkPepk0qzE+ntVC6QlB80yivJMWSNjegM2GAo1x5wLgRflbKF8wwjv5SO13mhumF4FdulwW1nmuHzArX8guM99d1E5y/HsTRIP4Yd0enZBMH5AXpkB6JyVsyIu/JGRkTTv4Gz4OXQSd8E34Ov4bZJjUMrj3PyE6Ey3+TZNnS</latexit><latexit sha1_base64="fQpv7JZtpfP0XLbEFHyrbCrR/b0=">AAACsXicbVHNbhMxEPYufyVQmsIFiYtJhJRIVbQLB3qJFIkLJ1Qk0hZlw+J1ZhMrXq9lz6KGxW/AW/BQiNfgCfAmRUrSjmTp8/fNNzOaybQUFqPoTxDeuXvv/oODh61Hjw+fHLWPn57bsjIcxryUpbnMmAUpFIxRoIRLbYAVmYSLbPmu0S++gbGiVJ9wpWFasLkSueAMPZW2f+o0wQUgo72kYLjI8vq7oz/o/w93fTqkiTblLK3VMHZf6g+O3mby8raxIRKEKzRFrZnrqb472aq6L55Q5fppuxsNonXQmyC+Bt1RZ6h//W4Nz9Lj4DCZlbwqQCGXzNpJHGmc1syg4BJcK6ksaMaXbA4TDxUrwE7r9docfeWZGc1L459Cuma3HTUrrF0Vmc9s5rb7WkPepk0qzE+ntVC6QlB80yivJMWSNjegM2GAo1x5wLgRflbKF8wwjv5SO13mhumF4FdulwW1nmuHzArX8guM99d1E5y/HsTRIP4Yd0enZBMH5AXpkB6JyVsyIu/JGRkTTv4Gz4OXQSd8E34Ov4bZJjUMrj3PyE6Ey3+TZNnS</latexit><latexit sha1_base64="7bdoCrnTb7yQXt+8f8414jfh570=">AAACsXicbVFNbxMxEPUuXyVQmsIFiYtphJRIVbQLB3qpVIkLJ1Qk0hZlw+J1ZhMrtteyZ1HD4n/AH+Rv8AvwpkFK0o5k6fk9P8/oTWGkcJgkf6L43v0HDx/tPe48ebr/7KB7+PzCVbXlMOKVrOxVwRxIoWGEAiVcGQtMFRIui8WHVr/8AdaJSn/BpYGJYjMtSsEZBirv/jZ5hnNARvuZYjgvyuanp7/o/wv3A3pKM2Orad7o09R/az55epcpyJvGlsgQrtGqxjDf1wN/vPHrrnhMtR/k3V4yTFZFb4N0DXpkXef5YbSfTSteK9DIJXNunCYGJw2zKLgE38lqB4bxBZvBOEDNFLhJs4rN0zeBmdKysuFopCt209Ew5dxSFeFlO7fb1VryLm1cY3kyaYQ2NYLmN43KWlKsaLsDOhUWOMplAIxbEWalfM4s4xg2tdVlZpmZC37tt1nQq7m2yEL5Tggw3Y3rNrh4O0yTYfo57Z2drKPcI6/IEemTlLwnZ+QjOScjwsnf6GX0OjqK38Vf4+9xcfM0jtaeF2Sr4sU/IjrXAg==</latexit>

zpa(n)<latexit sha1_base64="BxEDPI6F5q5NyMlo13JxEGjaF0s=">AAACP3icbZDLSgMxFIYzXmu9tbrTTbAKuikz3ehScOOygm2FtpRMeqYNJpkhOSPWYcCncauv4GP4BO7ErTvTy8KqPwR+vnMO5+QPEyks+v6bt7C4tLyyWlgrrm9sbm2XyjtNG6eGQ4PHMjY3IbMghYYGCpRwkxhgKpTQCm8vxvXWHRgrYn2NowS6ig20iARn6FCvtHeYdcKIPuS9rINwj0ZlCcuP9Ul+2CtV/Ko/Ef1rgpmpkJnqvbK32enHPFWgkUtmbTvwE+xmzKDgEvJiJ7WQMH7LBtB2VjMFtptNPpHTI0f6NIqNexrphP6cyJiydqRC16kYDu3v2hj+V2unGJ11M6GTFEHz6aIolRRjOk6E9oUBjnLkDONGuFspHzLDOLrc5rYMDEuGgt/n8xT05K45GKq86AIMfsf11zRr1cCvBle1ynltFmWB7JMDckwCckrOySWpkwbh5JE8kWfy4r16796H9zltXfBmM7tkTt7XNwTLr2I=</latexit><latexit sha1_base64="BxEDPI6F5q5NyMlo13JxEGjaF0s=">AAACP3icbZDLSgMxFIYzXmu9tbrTTbAKuikz3ehScOOygm2FtpRMeqYNJpkhOSPWYcCncauv4GP4BO7ErTvTy8KqPwR+vnMO5+QPEyks+v6bt7C4tLyyWlgrrm9sbm2XyjtNG6eGQ4PHMjY3IbMghYYGCpRwkxhgKpTQCm8vxvXWHRgrYn2NowS6ig20iARn6FCvtHeYdcKIPuS9rINwj0ZlCcuP9Ul+2CtV/Ko/Ef1rgpmpkJnqvbK32enHPFWgkUtmbTvwE+xmzKDgEvJiJ7WQMH7LBtB2VjMFtptNPpHTI0f6NIqNexrphP6cyJiydqRC16kYDu3v2hj+V2unGJ11M6GTFEHz6aIolRRjOk6E9oUBjnLkDONGuFspHzLDOLrc5rYMDEuGgt/n8xT05K45GKq86AIMfsf11zRr1cCvBle1ynltFmWB7JMDckwCckrOySWpkwbh5JE8kWfy4r16796H9zltXfBmM7tkTt7XNwTLr2I=</latexit><latexit sha1_base64="BxEDPI6F5q5NyMlo13JxEGjaF0s=">AAACP3icbZDLSgMxFIYzXmu9tbrTTbAKuikz3ehScOOygm2FtpRMeqYNJpkhOSPWYcCncauv4GP4BO7ErTvTy8KqPwR+vnMO5+QPEyks+v6bt7C4tLyyWlgrrm9sbm2XyjtNG6eGQ4PHMjY3IbMghYYGCpRwkxhgKpTQCm8vxvXWHRgrYn2NowS6ig20iARn6FCvtHeYdcKIPuS9rINwj0ZlCcuP9Ul+2CtV/Ko/Ef1rgpmpkJnqvbK32enHPFWgkUtmbTvwE+xmzKDgEvJiJ7WQMH7LBtB2VjMFtptNPpHTI0f6NIqNexrphP6cyJiydqRC16kYDu3v2hj+V2unGJ11M6GTFEHz6aIolRRjOk6E9oUBjnLkDONGuFspHzLDOLrc5rYMDEuGgt/n8xT05K45GKq86AIMfsf11zRr1cCvBle1ynltFmWB7JMDckwCckrOySWpkwbh5JE8kWfy4r16796H9zltXfBmM7tkTt7XNwTLr2I=</latexit><latexit sha1_base64="BxEDPI6F5q5NyMlo13JxEGjaF0s=">AAACP3icbZDLSgMxFIYzXmu9tbrTTbAKuikz3ehScOOygm2FtpRMeqYNJpkhOSPWYcCncauv4GP4BO7ErTvTy8KqPwR+vnMO5+QPEyks+v6bt7C4tLyyWlgrrm9sbm2XyjtNG6eGQ4PHMjY3IbMghYYGCpRwkxhgKpTQCm8vxvXWHRgrYn2NowS6ig20iARn6FCvtHeYdcKIPuS9rINwj0ZlCcuP9Ul+2CtV/Ko/Ef1rgpmpkJnqvbK32enHPFWgkUtmbTvwE+xmzKDgEvJiJ7WQMH7LBtB2VjMFtptNPpHTI0f6NIqNexrphP6cyJiydqRC16kYDu3v2hj+V2unGJ11M6GTFEHz6aIolRRjOk6E9oUBjnLkDONGuFspHzLDOLrc5rYMDEuGgt/n8xT05K45GKq86AIMfsf11zRr1cCvBle1ynltFmWB7JMDckwCckrOySWpkwbh5JE8kWfy4r16796H9zltXfBmM7tkTt7XNwTLr2I=</latexit>

(b) Proposed Model

Figure 1: Overview: Model Comparison. We show the graphical representations of (a) traditionallatent variable models (VAE, ladder VAE) and (b) the proposed graph VAE. Solid lines denotegeneration, dashed lines denote inference, and the dotted area indicates the latent space governedby variational parameters φ and generative parameters θ. Both VAE and ladder VAE use a fixedgraph structure with limited expressiveness (VAE: independent; ladder VAE: chain-structured). Incontrast, graph VAE jointly optimizes a distribution over latent structures c and model parameters(φ,θ), allowing test-time sampling of a flexible, data-driven latent structure.

Rezende et al., 2014; Goyal et al., 2017; Vikram et al., 2018; Webb et al., 2018). These dependen-cies provide empirical priors, learned priors that are conditioned on other latent variables. With Mlatent dimensions, the full prior takes the following auto-regressive form:

pθ(z) =M

∏m=1

pθ(zm∣zpa(m)), (3)

where zpa(m) denotes the vector of latent variables constituting the parents of zm. Each conditionaldistribution can be parameterized by deep networks that output the parameters of distributions, e.g.mean and variance of a Gaussian. While these conditional distributions may be relatively simple, themarginal empirical prior, pθ(zm) = ∫ pθ(zm∣zpa(m))pθ(zpa(m))dzpa(m), can be arbitrarily complex.By using more flexible priors, models with latent dependency structure are less restrictive in theirlatent representations, hopefully capturing more information and enabling a better fit to the data.

With the added latent dependencies in the model, the independence assumption in the approximateposterior is even less valid in this setting. While normalizing flows (Rezende & Mohamed, 2015)offers one technique for overcoming the mean field assumption, a separate line of work has in-vestigated the use of structured approximate posteriors, particularly in the context of models withempirical priors (Johnson et al., 2016). This technique introduces dependencies between the dimen-sions of the approximate posterior, often mirroring the dependency structure of the latent variablemodel. An explanation for this was provided by Marino et al. (2018): optimizing the approximateposterior requires knowledge of the prior, which is especially relevant in models with empirical pri-ors where the prior can vary with the data. Ladder VAE (Sønderby et al., 2016) incorporates theseprior dependencies by using a structured approximate posterior of the form

qφ(z∣x) =M

∏m=1

qφ(zm∣x,zpa(m)). (4)

Unlike the mean field approximate posterior, which conditions each dimension only on the dataexample, x, the distributions in Eq. (4) account for latent dependencies by conditioning on sam-ples from the parent variables. Ladder VAE performs this conditioning by reusing the empiricalprior during inference, forming the approximate posterior by combining a “bottom-up” recognitiondistribution and the “top-down” prior.

While Eqs. (3) and (4) permit separate latent dependencies for each individual latent dimension, thedimensions are typically partitioned into a set of nodes, with dimensions within each node sharing

3


the same parents. This improves computational efficiency by allowing priors and approximate pos-teriors within each node to be calculated in parallel. Using zn to denote latent node n of N andzpa(n) to denote the concatenation of its parent nodes, we can write the ELBO (Eq. (2)) as

L(x; θ, φ) = Eqφ(z∣x) [log pθ(x∣z)] −N

∑n=1

Eqφ(z∣x) [logqφ(zn∣x,zpa(n))

pθ(zn∣zpa(n))] . (5)

Note that the KL divergence term in the ELBO can no longer be evaluated analytically, now requiringa sampling-based estimate of the expectation (Kingma & Welling, 2014). While this can lead tohigher variance in ELBO estimates and the resulting gradients, models with latent dependenciesstill tend to empirically outperform models with independence assumptions (Burda et al., 2016;Sønderby et al., 2016). However, by increasing the number of nodes, the burden of devising asuitable dependency structure falls upon the experimental practitioner. This is non-trivial, as thestructure may depend on the data and other model hyperparameters, such as the number of layersin the deep networks, non-linearities, latent distributions, etc. Rather than relying on pre-definedfully-connected structures (Kingma et al., 2016) or chain structures (Sønderby et al., 2016), we seekto automatically learn the latent dependency structure as part of the variational optimization process.A comparison of these approaches is visualized in Fig. 1.

3 VARIATIONAL OPTIMIZATION OF LATENT STRUCTURES

Unlike the model parameters (φ, θ), which are optimized over a continuous domain, the latent de-pendency structure is discrete, without a clear ordering. The discrete nature of the latent space’stopological structure introduces discontinuities in the optimization landscape, complicating thelearning process. Fortunately, unlike the related setting of neural architecture search (Zoph & Le,2016), there is only a finite number of possible dependency structures over a fixed number of latentdimensions: In a directed graphical model, a fully-connected directed acyclic graph (DAG) modelsall possible dependencies. In this model, an ordering is induced over the latent nodes, and the par-ents of node n (of N ) are given as zpa(n) = {zn+1, . . . ,zN}.1 Thus, to learn an appropriate latentdependency structure, we can maintain all dependencies in a fully-connected DAG, modifying theirpresence or absence during training. This is accomplished by introducing a set of binary dependencygates (Section 3.1). We convert discrete optimization over dependency structures into a continuousoptimization problem by parameterizing these gates as samples from Bernoulli distributions, thenlearning the distribution parameters (Section 3.3). These gating distributions induce an additionallower bound on L, which becomes tight when the distribution converges to a delta function, yieldinga single, optimized dependency structure (Section 3.2). Indeed, we observe this process empirically,with the learned dependency structures outperforming their predefined counterparts (Section 4).

3.1 GATED DEPENDENCIES

To control the dependency structure of the model, we introduce a set of binary global variables,c = {ci,j}i,j , which gate the latent dependencies. The element ci,j denotes the gate variable fromzi to zj (i > j), specifying the presence or absence of this latent dependency. Because each elementof c takes values in {0,1}, dependencies can be preserved or removed simply through (broadcasted)element-wise multiplication with the corresponding parent nodes. Removing a dependency entailsmultiplying the corresponding input parent node by 0. Each possible latent dependency structurecan now be expressed through its corresponding value of c.

Each fixed latent dependency structure, c′, induces a separate latent variable model pθ(x,z,c) =

pθ(x∣z,c)pθ(z∣c)δc,c′ , where δ⋅,⋅ is the Kronecker delta, which effectively selects a single structure.Similar to Eq. (3), the prior on the latent variables can now be expressed as

pθ(z∣c) =N

∏n=1

pθ(zn∣zpa(n),cpa(n),n), (6)

where cpa(n),n denotes the gate variables associated with the dependencies between node zn andits parents, zpa(n). Note that zpa(n) denotes the set of all possible parents of node zn in the fully-connected DAG, i.e. zpa(n) = {zn+1, . . . ,zN}. To give a concrete example of the gating procedure,

1We follow convention (Dayan et al., 1995; Rezende et al., 2014; Sønderby et al., 2016), with parent nodeshaving a larger index than their children.

4


e n = (eµn, e⌃n)<latexit sha1_base64="ptRQcu8EBy9hpix9hKjb7KBpEqs=">AAACa3icbZDLSsNAFIan8VbrrbU7dRGsgoKUpBvdCAVBXFa0F2hKmUxO26EzkzAzUUvIG/k07kRfwXdwmnZhW38Y+PnOOZwzvx8xqrTjfOastfWNza38dmFnd2//oFg6bKkwlgSaJGSh7PhYAaMCmppqBp1IAuY+g7Y/vpvW2y8gFQ3Fs55E0ON4KOiAEqwN6hfvz7xXGoCmLIDE87kXKZr2xe3FEuaxoVdL8IkOOTb88qxfrDhVJ5O9aty5qaC5Gv1Sbs8LQhJzEJowrFTXdSLdS7DUlDBIC16sIMJkjIfQNVZgDqqXZB9O7XNDAnsQSvOEtjP6dyLBXKkJ900nx3qklmtT+F+tG+vBTS+hIoo1CDJbNIiZrUN7mp4dUAlEs4kxmEhqbrXJCEtMtMl4YctQ4mhEyVu6SEFkdy1An6cFE6C7HNeqadWqrlN1H2uVem0eZR4do1N0gVx0jeroATVQExH0jj7QF/rO/Vhl68g6mbVauflMGS3IOv8FzH++iw==</latexit><latexit sha1_base64="ptRQcu8EBy9hpix9hKjb7KBpEqs=">AAACa3icbZDLSsNAFIan8VbrrbU7dRGsgoKUpBvdCAVBXFa0F2hKmUxO26EzkzAzUUvIG/k07kRfwXdwmnZhW38Y+PnOOZwzvx8xqrTjfOastfWNza38dmFnd2//oFg6bKkwlgSaJGSh7PhYAaMCmppqBp1IAuY+g7Y/vpvW2y8gFQ3Fs55E0ON4KOiAEqwN6hfvz7xXGoCmLIDE87kXKZr2xe3FEuaxoVdL8IkOOTb88qxfrDhVJ5O9aty5qaC5Gv1Sbs8LQhJzEJowrFTXdSLdS7DUlDBIC16sIMJkjIfQNVZgDqqXZB9O7XNDAnsQSvOEtjP6dyLBXKkJ900nx3qklmtT+F+tG+vBTS+hIoo1CDJbNIiZrUN7mp4dUAlEs4kxmEhqbrXJCEtMtMl4YctQ4mhEyVu6SEFkdy1An6cFE6C7HNeqadWqrlN1H2uVem0eZR4do1N0gVx0jeroATVQExH0jj7QF/rO/Vhl68g6mbVauflMGS3IOv8FzH++iw==</latexit><latexit sha1_base64="ptRQcu8EBy9hpix9hKjb7KBpEqs=">AAACa3icbZDLSsNAFIan8VbrrbU7dRGsgoKUpBvdCAVBXFa0F2hKmUxO26EzkzAzUUvIG/k07kRfwXdwmnZhW38Y+PnOOZwzvx8xqrTjfOastfWNza38dmFnd2//oFg6bKkwlgSaJGSh7PhYAaMCmppqBp1IAuY+g7Y/vpvW2y8gFQ3Fs55E0ON4KOiAEqwN6hfvz7xXGoCmLIDE87kXKZr2xe3FEuaxoVdL8IkOOTb88qxfrDhVJ5O9aty5qaC5Gv1Sbs8LQhJzEJowrFTXdSLdS7DUlDBIC16sIMJkjIfQNVZgDqqXZB9O7XNDAnsQSvOEtjP6dyLBXKkJ900nx3qklmtT+F+tG+vBTS+hIoo1CDJbNIiZrUN7mp4dUAlEs4kxmEhqbrXJCEtMtMl4YctQ4mhEyVu6SEFkdy1An6cFE6C7HNeqadWqrlN1H2uVem0eZR4do1N0gVx0jeroATVQExH0jj7QF/rO/Vhl68g6mbVauflMGS3IOv8FzH++iw==</latexit><latexit sha1_base64="DPsD6WpYCZ2+0ZNmOdaBbg+2QWQ=">AAACFXicbZC9TsMwFIVv+C2hhXZmiaiQmKqEBUYkFsYi0R+prSrHuWmt2k5kO4gqyguw8go8DRti4m1w0w6k5UqWjr5j6x6fMOVMG9//cfb2Dw6Pjmsn7mndbZydN+t9nWSKYo8mPFHDkGjkTGLPMMNxmCokIuQ4CBcPK3/wgkqzRD6bZYoTQWaSxYwSY1F32mz7Hb8cb1cEG9GGzUxbTmMcJTQTKA3lROtR4KdmkhNlGOVYuONMY0rogsxwZKUkAvUkL3MW3pUlkRcnyh5pvJL+fZETofVShPamIGaut70V/M8bZSa+m+RMpplBSdeL4ox7JvFWn/YippAavrSCUMVsVo/OiSLU2GoqW2aKpHNGX4sqRVnmqsBQFK7tL9hua1f0bzqB3wmefKjBBVzCNQRwC/fwCF3oAYUI3uDd+XA+na91z3vOpvAWVMb5/gXodaHk</latexit><latexit sha1_base64="A22+Gk6pR+Pc61LNtg13Ik/2agk=">AAACYHicbZDLSgMxFIZPx1utt1Z36mLwAgpSZtzoRhAEcVnRVqFTSiZz2oYmmSHJqGWYN/Jp3Im+gu9gelnY1h8CP9+fcE7+MOFMG8/7LDgLi0vLK8XV0tr6xuZWubLe0HGqKNZpzGP1HBKNnEmsG2Y4PicKiQg5PoX9m2H+9IJKs1g+mkGCLUG6knUYJcaidvn2KHhlERrGI8yCUASJZnlbXp3MYJFaejYDH1hXEMtPj9rlQ6/qjeTOG39iDmGiWrtS2AiimKYCpaGcaN30vcS0MqIMoxzzUpBqTAjtky42rZVEoG5low/n7rElkduJlT3SuCP690VGhNYDEdqbgpiens2G8L+smZrOZStjMkkNSjoe1Em5a2J32J4bMYXU8IE1hCpmd3VpjyhCje14akpXkaTH6Fs+TVGO9pqCochLtkB/tq550ziv+l7Vv/egCHtwACfgwwVcwx3UoA4U3uEDvuC78OPsOLvjqp3CpPNtmJKz/wujYL3G</latexit><latexit sha1_base64="A22+Gk6pR+Pc61LNtg13Ik/2agk=">AAACYHicbZDLSgMxFIZPx1utt1Z36mLwAgpSZtzoRhAEcVnRVqFTSiZz2oYmmSHJqGWYN/Jp3Im+gu9gelnY1h8CP9+fcE7+MOFMG8/7LDgLi0vLK8XV0tr6xuZWubLe0HGqKNZpzGP1HBKNnEmsG2Y4PicKiQg5PoX9m2H+9IJKs1g+mkGCLUG6knUYJcaidvn2KHhlERrGI8yCUASJZnlbXp3MYJFaejYDH1hXEMtPj9rlQ6/qjeTOG39iDmGiWrtS2AiimKYCpaGcaN30vcS0MqIMoxzzUpBqTAjtky42rZVEoG5low/n7rElkduJlT3SuCP690VGhNYDEdqbgpiens2G8L+smZrOZStjMkkNSjoe1Em5a2J32J4bMYXU8IE1hCpmd3VpjyhCje14akpXkaTH6Fs+TVGO9pqCochLtkB/tq550ziv+l7Vv/egCHtwACfgwwVcwx3UoA4U3uEDvuC78OPsOLvjqp3CpPNtmJKz/wujYL3G</latexit><latexit sha1_base64="O73u71OdA6cn87SJQLXXxV/TJn8=">AAACa3icbZDNSsNAFIWn8b9arXanLoJtoYKUxI1uhIIgLhXtDzQlTCa37eDMJMxM1BLyRj6NO9FX8B2c1i5M64GBw3fv5d45Qcyo0o7zUbBWVtfWNza3its7pd298v5BR0WJJNAmEYtkL8AKGBXQ1lQz6MUSMA8YdIOn62m9+wxS0Ug86kkMA45Hgg4pwdogv3xT815oCJqyEFIv4F6saOaLq8YC5omhZwvwgY44Nvy05perTtOZyV427txU0Vx3/n6h5IURSTgITRhWqu86sR6kWGpKGGRFL1EQY/KER9A3VmAOapDOPpzZdUNCexhJ84S2Z/TvRIq5UhMemE6O9Vgt1qbwv1o/0cPLQUpFnGgQ5HfRMGG2juxpenZIJRDNJsZgIqm51SZjLDHRJuPclpHE8ZiS1yxPQczuysGAZ0UToLsY17LpnDddp+neO9XW+TzKTXSETlADuegCtdAtukNtRNAbekef6KvwbVWsQ+v4t9UqzGcqKCer/gPL376J</latexit><latexit sha1_base64="ptRQcu8EBy9hpix9hKjb7KBpEqs=">AAACa3icbZDLSsNAFIan8VbrrbU7dRGsgoKUpBvdCAVBXFa0F2hKmUxO26EzkzAzUUvIG/k07kRfwXdwmnZhW38Y+PnOOZwzvx8xqrTjfOastfWNza38dmFnd2//oFg6bKkwlgSaJGSh7PhYAaMCmppqBp1IAuY+g7Y/vpvW2y8gFQ3Fs55E0ON4KOiAEqwN6hfvz7xXGoCmLIDE87kXKZr2xe3FEuaxoVdL8IkOOTb88qxfrDhVJ5O9aty5qaC5Gv1Sbs8LQhJzEJowrFTXdSLdS7DUlDBIC16sIMJkjIfQNVZgDqqXZB9O7XNDAnsQSvOEtjP6dyLBXKkJ900nx3qklmtT+F+tG+vBTS+hIoo1CDJbNIiZrUN7mp4dUAlEs4kxmEhqbrXJCEtMtMl4YctQ4mhEyVu6SEFkdy1An6cFE6C7HNeqadWqrlN1H2uVem0eZR4do1N0gVx0jeroATVQExH0jj7QF/rO/Vhl68g6mbVauflMGS3IOv8FzH++iw==</latexit><latexit sha1_base64="ptRQcu8EBy9hpix9hKjb7KBpEqs=">AAACa3icbZDLSsNAFIan8VbrrbU7dRGsgoKUpBvdCAVBXFa0F2hKmUxO26EzkzAzUUvIG/k07kRfwXdwmnZhW38Y+PnOOZwzvx8xqrTjfOastfWNza38dmFnd2//oFg6bKkwlgSaJGSh7PhYAaMCmppqBp1IAuY+g7Y/vpvW2y8gFQ3Fs55E0ON4KOiAEqwN6hfvz7xXGoCmLIDE87kXKZr2xe3FEuaxoVdL8IkOOTb88qxfrDhVJ5O9aty5qaC5Gv1Sbs8LQhJzEJowrFTXdSLdS7DUlDBIC16sIMJkjIfQNVZgDqqXZB9O7XNDAnsQSvOEtjP6dyLBXKkJ900nx3qklmtT+F+tG+vBTS+hIoo1CDJbNIiZrUN7mp4dUAlEs4kxmEhqbrXJCEtMtMl4YctQ4mhEyVu6SEFkdy1An6cFE6C7HNeqadWqrlN1H2uVem0eZR4do1N0gVx0jeroATVQExH0jj7QF/rO/Vhl68g6mbVauflMGS3IOv8FzH++iw==</latexit><latexit sha1_base64="ptRQcu8EBy9hpix9hKjb7KBpEqs=">AAACa3icbZDLSsNAFIan8VbrrbU7dRGsgoKUpBvdCAVBXFa0F2hKmUxO26EzkzAzUUvIG/k07kRfwXdwmnZhW38Y+PnOOZwzvx8xqrTjfOastfWNza38dmFnd2//oFg6bKkwlgSaJGSh7PhYAaMCmppqBp1IAuY+g7Y/vpvW2y8gFQ3Fs55E0ON4KOiAEqwN6hfvz7xXGoCmLIDE87kXKZr2xe3FEuaxoVdL8IkOOTb88qxfrDhVJ5O9aty5qaC5Gv1Sbs8LQhJzEJowrFTXdSLdS7DUlDBIC16sIMJkjIfQNVZgDqqXZB9O7XNDAnsQSvOEtjP6dyLBXKkJ900nx3qklmtT+F+tG+vBTS+hIoo1CDJbNIiZrUN7mp4dUAlEs4kxmEhqbrXJCEtMtMl4YctQ4mhEyVu6SEFkdy1An6cFE6C7HNeqadWqrlN1H2uVem0eZR4do1N0gVx0jeroATVQExH0jj7QF/rO/Vhl68g6mbVauflMGS3IOv8FzH++iw==</latexit><latexit sha1_base64="ptRQcu8EBy9hpix9hKjb7KBpEqs=">AAACa3icbZDLSsNAFIan8VbrrbU7dRGsgoKUpBvdCAVBXFa0F2hKmUxO26EzkzAzUUvIG/k07kRfwXdwmnZhW38Y+PnOOZwzvx8xqrTjfOastfWNza38dmFnd2//oFg6bKkwlgSaJGSh7PhYAaMCmppqBp1IAuY+g7Y/vpvW2y8gFQ3Fs55E0ON4KOiAEqwN6hfvz7xXGoCmLIDE87kXKZr2xe3FEuaxoVdL8IkOOTb88qxfrDhVJ5O9aty5qaC5Gv1Sbs8LQhJzEJowrFTXdSLdS7DUlDBIC16sIMJkjIfQNVZgDqqXZB9O7XNDAnsQSvOEtjP6dyLBXKkJ900nx3qklmtT+F+tG+vBTS+hIoo1CDJbNIiZrUN7mp4dUAlEs4kxmEhqbrXJCEtMtMl4YctQ4mhEyVu6SEFkdy1An6cFE6C7HNeqadWqrlN1H2uVem0eZR4do1N0gVx0jeroATVQExH0jj7QF/rO/Vhl68g6mbVauflMGS3IOv8FzH++iw==</latexit><latexit sha1_base64="ptRQcu8EBy9hpix9hKjb7KBpEqs=">AAACa3icbZDLSsNAFIan8VbrrbU7dRGsgoKUpBvdCAVBXFa0F2hKmUxO26EzkzAzUUvIG/k07kRfwXdwmnZhW38Y+PnOOZwzvx8xqrTjfOastfWNza38dmFnd2//oFg6bKkwlgSaJGSh7PhYAaMCmppqBp1IAuY+g7Y/vpvW2y8gFQ3Fs55E0ON4KOiAEqwN6hfvz7xXGoCmLIDE87kXKZr2xe3FEuaxoVdL8IkOOTb88qxfrDhVJ5O9aty5qaC5Gv1Sbs8LQhJzEJowrFTXdSLdS7DUlDBIC16sIMJkjIfQNVZgDqqXZB9O7XNDAnsQSvOEtjP6dyLBXKkJ900nx3qklmtT+F+tG+vBTS+hIoo1CDJbNIiZrUN7mp4dUAlEs4kxmEhqbrXJCEtMtMl4YctQ4mhEyVu6SEFkdy1An6cFE6C7HNeqadWqrlN1H2uVem0eZR4do1N0gVx0jeroATVQExH0jj7QF/rO/Vhl68g6mbVauflMGS3IOv8FzH++iw==</latexit><latexit sha1_base64="ptRQcu8EBy9hpix9hKjb7KBpEqs=">AAACa3icbZDLSsNAFIan8VbrrbU7dRGsgoKUpBvdCAVBXFa0F2hKmUxO26EzkzAzUUvIG/k07kRfwXdwmnZhW38Y+PnOOZwzvx8xqrTjfOastfWNza38dmFnd2//oFg6bKkwlgSaJGSh7PhYAaMCmppqBp1IAuY+g7Y/vpvW2y8gFQ3Fs55E0ON4KOiAEqwN6hfvz7xXGoCmLIDE87kXKZr2xe3FEuaxoVdL8IkOOTb88qxfrDhVJ5O9aty5qaC5Gv1Sbs8LQhJzEJowrFTXdSLdS7DUlDBIC16sIMJkjIfQNVZgDqqXZB9O7XNDAnsQSvOEtjP6dyLBXKkJ900nx3qklmtT+F+tG+vBTS+hIoo1CDJbNIiZrUN7mp4dUAlEs4kxmEhqbrXJCEtMtMl4YctQ4mhEyVu6SEFkdy1An6cFE6C7HNeqadWqrlN1H2uVem0eZR4do1N0gVx0jeroATVQExH0jj7QF/rO/Vhl68g6mbVauflMGS3IOv8FzH++iw==</latexit>

. . .<latexit sha1_base64="4ARCFSnpLWM82MNV3ecWYZMIQ8Q=">AAACK3icbVDLSgMxFE181mq11aWbwVZwVWa60ZUU3LisYB/QDiWTybSheQxJRixlfsKt/oJf40px63+YTmfhtB4IHM65l3tygphRbVz3E25t7+zu7ZcOyodHleOTau20p2WiMOliyaQaBEgTRgXpGmoYGcSKIB4w0g9md0u//0SUplI8mnlMfI4mgkYUI2OlQWPEQml0Y1ytu003g7NJvJzUQY7OuAYro1DihBNhMENaDz03Nv4CKUMxI2l5lGgSIzxDEzK0VCBOtL/IAqfOpVVCJ5LKPmGcTP27sUBc6zkP7CRHZqrXvaX4nzdMTHTjL6iIE0MEXh2KEuYY6Sx/74RUEWzY3BKEFbVZHTxFCmFjOypcmSgUTyl+TosqEVmughjwtGwL9Nbr2iS9VtNzm95Dq96+zassgXNwAa6AB65BG9yDDugCDBh4Aa/gDb7DD/gFv1ejWzDfOQMFwJ9fFGindg==</latexit><latexit sha1_base64="4ARCFSnpLWM82MNV3ecWYZMIQ8Q=">AAACK3icbVDLSgMxFE181mq11aWbwVZwVWa60ZUU3LisYB/QDiWTybSheQxJRixlfsKt/oJf40px63+YTmfhtB4IHM65l3tygphRbVz3E25t7+zu7ZcOyodHleOTau20p2WiMOliyaQaBEgTRgXpGmoYGcSKIB4w0g9md0u//0SUplI8mnlMfI4mgkYUI2OlQWPEQml0Y1ytu003g7NJvJzUQY7OuAYro1DihBNhMENaDz03Nv4CKUMxI2l5lGgSIzxDEzK0VCBOtL/IAqfOpVVCJ5LKPmGcTP27sUBc6zkP7CRHZqrXvaX4nzdMTHTjL6iIE0MEXh2KEuYY6Sx/74RUEWzY3BKEFbVZHTxFCmFjOypcmSgUTyl+TosqEVmughjwtGwL9Nbr2iS9VtNzm95Dq96+zassgXNwAa6AB65BG9yDDugCDBh4Aa/gDb7DD/gFv1ejWzDfOQMFwJ9fFGindg==</latexit><latexit sha1_base64="4ARCFSnpLWM82MNV3ecWYZMIQ8Q=">AAACK3icbVDLSgMxFE181mq11aWbwVZwVWa60ZUU3LisYB/QDiWTybSheQxJRixlfsKt/oJf40px63+YTmfhtB4IHM65l3tygphRbVz3E25t7+zu7ZcOyodHleOTau20p2WiMOliyaQaBEgTRgXpGmoYGcSKIB4w0g9md0u//0SUplI8mnlMfI4mgkYUI2OlQWPEQml0Y1ytu003g7NJvJzUQY7OuAYro1DihBNhMENaDz03Nv4CKUMxI2l5lGgSIzxDEzK0VCBOtL/IAqfOpVVCJ5LKPmGcTP27sUBc6zkP7CRHZqrXvaX4nzdMTHTjL6iIE0MEXh2KEuYY6Sx/74RUEWzY3BKEFbVZHTxFCmFjOypcmSgUTyl+TosqEVmughjwtGwL9Nbr2iS9VtNzm95Dq96+zassgXNwAa6AB65BG9yDDugCDBh4Aa/gDb7DD/gFv1ejWzDfOQMFwJ9fFGindg==</latexit><latexit sha1_base64="4ARCFSnpLWM82MNV3ecWYZMIQ8Q=">AAACK3icbVDLSgMxFE181mq11aWbwVZwVWa60ZUU3LisYB/QDiWTybSheQxJRixlfsKt/oJf40px63+YTmfhtB4IHM65l3tygphRbVz3E25t7+zu7ZcOyodHleOTau20p2WiMOliyaQaBEgTRgXpGmoYGcSKIB4w0g9md0u//0SUplI8mnlMfI4mgkYUI2OlQWPEQml0Y1ytu003g7NJvJzUQY7OuAYro1DihBNhMENaDz03Nv4CKUMxI2l5lGgSIzxDEzK0VCBOtL/IAqfOpVVCJ5LKPmGcTP27sUBc6zkP7CRHZqrXvaX4nzdMTHTjL6iIE0MEXh2KEuYY6Sx/74RUEWzY3BKEFbVZHTxFCmFjOypcmSgUTyl+TosqEVmughjwtGwL9Nbr2iS9VtNzm95Dq96+zassgXNwAa6AB65BG9yDDugCDBh4Aa/gDb7DD/gFv1ejWzDfOQMFwJ9fFGindg==</latexit>

zpa(n)<latexit sha1_base64="sJClZMD4xijwVmMOFhy/Y1fbQQI=">AAACP3icbZDPSgMxEMaz/q31X9WbXoKtUC9ltxc9Cl48VrAqtKVk09k2mGSXZFasy4JP41VfwcfwCbyJV2+mtQfb+kHg4zczzOQLEyks+v67t7C4tLyyWlgrrm9sbm2XdnavbZwaDk0ey9jchsyCFBqaKFDCbWKAqVDCTXh3Pqrf3IOxItZXOEygo1hfi0hwhg51S/uVrB1G9DHvZm2EBzQqS1he1cd5pVsq+zV/LDpvgokpk4ka3R1vs92LeapAI5fM2lbgJ9jJmEHBJeTFdmohYfyO9aHlrGYKbCcbfyKnR470aBQb9zTSMf07kTFl7VCFrlMxHNjZ2gj+V2ulGJ12MqGTFEHz30VRKinGdJQI7QkDHOXQGcaNcLdSPmCGcXS5TW3pG5YMBH/Ipyno8V1TMFR50QUYzMY1b67rtcCvBZf18tnpJMoCOSCHpEoCckLOyAVpkCbh5Ik8kxfy6r15H96n9/XbuuBNZvbIlLzvHwaZr2g=</latexit><latexit sha1_base64="sJClZMD4xijwVmMOFhy/Y1fbQQI=">AAACP3icbZDPSgMxEMaz/q31X9WbXoKtUC9ltxc9Cl48VrAqtKVk09k2mGSXZFasy4JP41VfwcfwCbyJV2+mtQfb+kHg4zczzOQLEyks+v67t7C4tLyyWlgrrm9sbm2XdnavbZwaDk0ey9jchsyCFBqaKFDCbWKAqVDCTXh3Pqrf3IOxItZXOEygo1hfi0hwhg51S/uVrB1G9DHvZm2EBzQqS1he1cd5pVsq+zV/LDpvgokpk4ka3R1vs92LeapAI5fM2lbgJ9jJmEHBJeTFdmohYfyO9aHlrGYKbCcbfyKnR470aBQb9zTSMf07kTFl7VCFrlMxHNjZ2gj+V2ulGJ12MqGTFEHz30VRKinGdJQI7QkDHOXQGcaNcLdSPmCGcXS5TW3pG5YMBH/Ipyno8V1TMFR50QUYzMY1b67rtcCvBZf18tnpJMoCOSCHpEoCckLOyAVpkCbh5Ik8kxfy6r15H96n9/XbuuBNZvbIlLzvHwaZr2g=</latexit><latexit sha1_base64="sJClZMD4xijwVmMOFhy/Y1fbQQI=">AAACP3icbZDPSgMxEMaz/q31X9WbXoKtUC9ltxc9Cl48VrAqtKVk09k2mGSXZFasy4JP41VfwcfwCbyJV2+mtQfb+kHg4zczzOQLEyks+v67t7C4tLyyWlgrrm9sbm2XdnavbZwaDk0ey9jchsyCFBqaKFDCbWKAqVDCTXh3Pqrf3IOxItZXOEygo1hfi0hwhg51S/uVrB1G9DHvZm2EBzQqS1he1cd5pVsq+zV/LDpvgokpk4ka3R1vs92LeapAI5fM2lbgJ9jJmEHBJeTFdmohYfyO9aHlrGYKbCcbfyKnR470aBQb9zTSMf07kTFl7VCFrlMxHNjZ2gj+V2ulGJ12MqGTFEHz30VRKinGdJQI7QkDHOXQGcaNcLdSPmCGcXS5TW3pG5YMBH/Ipyno8V1TMFR50QUYzMY1b67rtcCvBZf18tnpJMoCOSCHpEoCckLOyAVpkCbh5Ik8kxfy6r15H96n9/XbuuBNZvbIlLzvHwaZr2g=</latexit><latexit sha1_base64="sJClZMD4xijwVmMOFhy/Y1fbQQI=">AAACP3icbZDPSgMxEMaz/q31X9WbXoKtUC9ltxc9Cl48VrAqtKVk09k2mGSXZFasy4JP41VfwcfwCbyJV2+mtQfb+kHg4zczzOQLEyks+v67t7C4tLyyWlgrrm9sbm2XdnavbZwaDk0ey9jchsyCFBqaKFDCbWKAqVDCTXh3Pqrf3IOxItZXOEygo1hfi0hwhg51S/uVrB1G9DHvZm2EBzQqS1he1cd5pVsq+zV/LDpvgokpk4ka3R1vs92LeapAI5fM2lbgJ9jJmEHBJeTFdmohYfyO9aHlrGYKbCcbfyKnR470aBQb9zTSMf07kTFl7VCFrlMxHNjZ2gj+V2ulGJ12MqGTFEHz30VRKinGdJQI7QkDHOXQGcaNcLdSPmCGcXS5TW3pG5YMBH/Ipyno8V1TMFR50QUYzMY1b67rtcCvBZf18tnpJMoCOSCHpEoCckLOyAVpkCbh5Ik8kxfy6r15H96n9/XbuuBNZvbIlLzvHwaZr2g=</latexit>

zn+1<latexit sha1_base64="rHCOl6RFYoxdcBQQIKMJMWSVD5s=">AAACNHicbVDLSsNAFJ34rNXaVpduBltBEErSjS4LblxWsA9oQ5hMJ+3QmUmYmYg15Evc6i/4L4I7ces3OE2zMK0HLhzOuZd7OH7EqNK2/WFtbe/s7u2XDsqHR5Xjaq1+0ldhLDHp4ZCFcugjRRgVpKepZmQYSYK4z8jAn98u/cEjkYqG4kEvIuJyNBU0oBhpI3m1ajMZ+wF8Tr1EXDlp06s17JadAW4SJycNkKPr1a3KeBLimBOhMUNKjRw70m6CpKaYkbQ8jhWJEJ6jKRkZKhAnyk2y5Cm8MMoEBqE0IzTM1L8XCeJKLbhvNjnSM7XuLcX/vFGsgxs3oSKKNRF49SiIGdQhXNYAJ1QSrNnCEIQlNVkhniGJsDZlFb5MJYpmFD+lRZWILFdB9HlaNgU663Vtkn675dgt577d6LTzKkvgDJyDS+CAa9ABd6ALegCDGLyAV/BmvVuf1pf1vVrdsvKbU1CA9fMLxJGqNg==</latexit><latexit sha1_base64="rHCOl6RFYoxdcBQQIKMJMWSVD5s=">AAACNHicbVDLSsNAFJ34rNXaVpduBltBEErSjS4LblxWsA9oQ5hMJ+3QmUmYmYg15Evc6i/4L4I7ces3OE2zMK0HLhzOuZd7OH7EqNK2/WFtbe/s7u2XDsqHR5Xjaq1+0ldhLDHp4ZCFcugjRRgVpKepZmQYSYK4z8jAn98u/cEjkYqG4kEvIuJyNBU0oBhpI3m1ajMZ+wF8Tr1EXDlp06s17JadAW4SJycNkKPr1a3KeBLimBOhMUNKjRw70m6CpKaYkbQ8jhWJEJ6jKRkZKhAnyk2y5Cm8MMoEBqE0IzTM1L8XCeJKLbhvNjnSM7XuLcX/vFGsgxs3oSKKNRF49SiIGdQhXNYAJ1QSrNnCEIQlNVkhniGJsDZlFb5MJYpmFD+lRZWILFdB9HlaNgU663Vtkn675dgt577d6LTzKkvgDJyDS+CAa9ABd6ALegCDGLyAV/BmvVuf1pf1vVrdsvKbU1CA9fMLxJGqNg==</latexit><latexit sha1_base64="rHCOl6RFYoxdcBQQIKMJMWSVD5s=">AAACNHicbVDLSsNAFJ34rNXaVpduBltBEErSjS4LblxWsA9oQ5hMJ+3QmUmYmYg15Evc6i/4L4I7ces3OE2zMK0HLhzOuZd7OH7EqNK2/WFtbe/s7u2XDsqHR5Xjaq1+0ldhLDHp4ZCFcugjRRgVpKepZmQYSYK4z8jAn98u/cEjkYqG4kEvIuJyNBU0oBhpI3m1ajMZ+wF8Tr1EXDlp06s17JadAW4SJycNkKPr1a3KeBLimBOhMUNKjRw70m6CpKaYkbQ8jhWJEJ6jKRkZKhAnyk2y5Cm8MMoEBqE0IzTM1L8XCeJKLbhvNjnSM7XuLcX/vFGsgxs3oSKKNRF49SiIGdQhXNYAJ1QSrNnCEIQlNVkhniGJsDZlFb5MJYpmFD+lRZWILFdB9HlaNgU663Vtkn675dgt577d6LTzKkvgDJyDS+CAa9ABd6ALegCDGLyAV/BmvVuf1pf1vVrdsvKbU1CA9fMLxJGqNg==</latexit><latexit sha1_base64="rHCOl6RFYoxdcBQQIKMJMWSVD5s=">AAACNHicbVDLSsNAFJ34rNXaVpduBltBEErSjS4LblxWsA9oQ5hMJ+3QmUmYmYg15Evc6i/4L4I7ces3OE2zMK0HLhzOuZd7OH7EqNK2/WFtbe/s7u2XDsqHR5Xjaq1+0ldhLDHp4ZCFcugjRRgVpKepZmQYSYK4z8jAn98u/cEjkYqG4kEvIuJyNBU0oBhpI3m1ajMZ+wF8Tr1EXDlp06s17JadAW4SJycNkKPr1a3KeBLimBOhMUNKjRw70m6CpKaYkbQ8jhWJEJ6jKRkZKhAnyk2y5Cm8MMoEBqE0IzTM1L8XCeJKLbhvNjnSM7XuLcX/vFGsgxs3oSKKNRF49SiIGdQhXNYAJ1QSrNnCEIQlNVkhniGJsDZlFb5MJYpmFD+lRZWILFdB9HlaNgU663Vtkn675dgt577d6LTzKkvgDJyDS+CAa9ABd6ALegCDGLyAV/BmvVuf1pf1vVrdsvKbU1CA9fMLxJGqNg==</latexit>

zN<latexit sha1_base64="zgYPV75jS6Lo2oIeht0WD944b4c=">AAACMHicbVDLTgIxFG19IoqCLt00gokrMsNGlyRuXBlM5JHAhHRKBxrazth2jDiZ73Crv+DX6Mq49SsswywEPMlNTs65N/fk+BFn2jjOJ9zY3Nre2S3sFfcPSodH5cpxR4exIrRNQh6qno815UzStmGG016kKBY+p11/ej33u49UaRbKezOLqCfwWLKAEWys5NWSgR+g53SY3Ka1Ybnq1J0MaJ24OamCHK1hBZYGo5DEgkpDONa67zqR8RKsDCOcpsVBrGmEyRSPad9SiQXVXpKlTtG5VUYoCJUdaVCm/r1IsNB6Jny7KbCZ6FVvLv7n9WMTXHkJk1FsqCSLR0HMkQnRvAI0YooSw2eWYKKYzYrIBCtMjC1q6ctY4WjCyFO6rFKZ5VoSfZEWbYHual3rpNOou07dvWtUm428ygI4BWfgArjgEjTBDWiBNiDgAbyAV/AG3+EH/ILfi9UNmN+cgCXAn18YsKl1</latexit><latexit sha1_base64="zgYPV75jS6Lo2oIeht0WD944b4c=">AAACMHicbVDLTgIxFG19IoqCLt00gokrMsNGlyRuXBlM5JHAhHRKBxrazth2jDiZ73Crv+DX6Mq49SsswywEPMlNTs65N/fk+BFn2jjOJ9zY3Nre2S3sFfcPSodH5cpxR4exIrRNQh6qno815UzStmGG016kKBY+p11/ej33u49UaRbKezOLqCfwWLKAEWys5NWSgR+g53SY3Ka1Ybnq1J0MaJ24OamCHK1hBZYGo5DEgkpDONa67zqR8RKsDCOcpsVBrGmEyRSPad9SiQXVXpKlTtG5VUYoCJUdaVCm/r1IsNB6Jny7KbCZ6FVvLv7n9WMTXHkJk1FsqCSLR0HMkQnRvAI0YooSw2eWYKKYzYrIBCtMjC1q6ctY4WjCyFO6rFKZ5VoSfZEWbYHual3rpNOou07dvWtUm428ygI4BWfgArjgEjTBDWiBNiDgAbyAV/AG3+EH/ILfi9UNmN+cgCXAn18YsKl1</latexit><latexit sha1_base64="zgYPV75jS6Lo2oIeht0WD944b4c=">AAACMHicbVDLTgIxFG19IoqCLt00gokrMsNGlyRuXBlM5JHAhHRKBxrazth2jDiZ73Crv+DX6Mq49SsswywEPMlNTs65N/fk+BFn2jjOJ9zY3Nre2S3sFfcPSodH5cpxR4exIrRNQh6qno815UzStmGG016kKBY+p11/ej33u49UaRbKezOLqCfwWLKAEWys5NWSgR+g53SY3Ka1Ybnq1J0MaJ24OamCHK1hBZYGo5DEgkpDONa67zqR8RKsDCOcpsVBrGmEyRSPad9SiQXVXpKlTtG5VUYoCJUdaVCm/r1IsNB6Jny7KbCZ6FVvLv7n9WMTXHkJk1FsqCSLR0HMkQnRvAI0YooSw2eWYKKYzYrIBCtMjC1q6ctY4WjCyFO6rFKZ5VoSfZEWbYHual3rpNOou07dvWtUm428ygI4BWfgArjgEjTBDWiBNiDgAbyAV/AG3+EH/ILfi9UNmN+cgCXAn18YsKl1</latexit><latexit sha1_base64="zgYPV75jS6Lo2oIeht0WD944b4c=">AAACMHicbVDLTgIxFG19IoqCLt00gokrMsNGlyRuXBlM5JHAhHRKBxrazth2jDiZ73Crv+DX6Mq49SsswywEPMlNTs65N/fk+BFn2jjOJ9zY3Nre2S3sFfcPSodH5cpxR4exIrRNQh6qno815UzStmGG016kKBY+p11/ej33u49UaRbKezOLqCfwWLKAEWys5NWSgR+g53SY3Ka1Ybnq1J0MaJ24OamCHK1hBZYGo5DEgkpDONa67zqR8RKsDCOcpsVBrGmEyRSPad9SiQXVXpKlTtG5VUYoCJUdaVCm/r1IsNB6Jny7KbCZ6FVvLv7n9WMTXHkJk1FsqCSLR0HMkQnRvAI0YooSw2eWYKKYzYrIBCtMjC1q6ctY4WjCyFO6rFKZ5VoSfZEWbYHual3rpNOou07dvWtUm428ygI4BWfgArjgEjTBDWiBNiDgAbyAV/AG3+EH/ILfi9UNmN+cgCXAn18YsKl1</latexit>

zN�1<latexit sha1_base64="xvdISp2SGWA7IK+MXYZNMO/JTwY=">AAACNHicbVDLSsNAFJ3UV63Wtrp0M9gKbixJN7osuHElFewD2hAm00k7dGYSZiZiDfkSt/oL/ovgTtz6DU7TLkzrgQuHc+7lHo4fMaq0bX9Yha3tnd294n7p4LB8VKnWjnsqjCUmXRyyUA58pAijgnQ11YwMIkkQ9xnp+7Obhd9/JFLRUDzoeURcjiaCBhQjbSSvWmkkIz+Az6mX3F06acOr1u2mnQFuEmdF6mCFjlezyqNxiGNOhMYMKTV07Ei7CZKaYkbS0ihWJEJ4hiZkaKhAnCg3yZKn8NwoYxiE0ozQMFP/XiSIKzXnvtnkSE/VurcQ//OGsQ6u3YSKKNZE4OWjIGZQh3BRAxxTSbBmc0MQltRkhXiKJMLalJX7MpEomlL8lOZVIrJcOdHnackU6KzXtUl6raZjN537Vr3dWlVZBKfgDFwAB1yBNrgFHdAFGMTgBbyCN+vd+rS+rO/lasFa3ZyAHKyfX4+Xqhg=</latexit><latexit sha1_base64="xvdISp2SGWA7IK+MXYZNMO/JTwY=">AAACNHicbVDLSsNAFJ3UV63Wtrp0M9gKbixJN7osuHElFewD2hAm00k7dGYSZiZiDfkSt/oL/ovgTtz6DU7TLkzrgQuHc+7lHo4fMaq0bX9Yha3tnd294n7p4LB8VKnWjnsqjCUmXRyyUA58pAijgnQ11YwMIkkQ9xnp+7Obhd9/JFLRUDzoeURcjiaCBhQjbSSvWmkkIz+Az6mX3F06acOr1u2mnQFuEmdF6mCFjlezyqNxiGNOhMYMKTV07Ei7CZKaYkbS0ihWJEJ4hiZkaKhAnCg3yZKn8NwoYxiE0ozQMFP/XiSIKzXnvtnkSE/VurcQ//OGsQ6u3YSKKNZE4OWjIGZQh3BRAxxTSbBmc0MQltRkhXiKJMLalJX7MpEomlL8lOZVIrJcOdHnackU6KzXtUl6raZjN537Vr3dWlVZBKfgDFwAB1yBNrgFHdAFGMTgBbyCN+vd+rS+rO/lasFa3ZyAHKyfX4+Xqhg=</latexit><latexit sha1_base64="xvdISp2SGWA7IK+MXYZNMO/JTwY=">AAACNHicbVDLSsNAFJ3UV63Wtrp0M9gKbixJN7osuHElFewD2hAm00k7dGYSZiZiDfkSt/oL/ovgTtz6DU7TLkzrgQuHc+7lHo4fMaq0bX9Yha3tnd294n7p4LB8VKnWjnsqjCUmXRyyUA58pAijgnQ11YwMIkkQ9xnp+7Obhd9/JFLRUDzoeURcjiaCBhQjbSSvWmkkIz+Az6mX3F06acOr1u2mnQFuEmdF6mCFjlezyqNxiGNOhMYMKTV07Ei7CZKaYkbS0ihWJEJ4hiZkaKhAnCg3yZKn8NwoYxiE0ozQMFP/XiSIKzXnvtnkSE/VurcQ//OGsQ6u3YSKKNZE4OWjIGZQh3BRAxxTSbBmc0MQltRkhXiKJMLalJX7MpEomlL8lOZVIrJcOdHnackU6KzXtUl6raZjN537Vr3dWlVZBKfgDFwAB1yBNrgFHdAFGMTgBbyCN+vd+rS+rO/lasFa3ZyAHKyfX4+Xqhg=</latexit><latexit sha1_base64="xvdISp2SGWA7IK+MXYZNMO/JTwY=">AAACNHicbVDLSsNAFJ3UV63Wtrp0M9gKbixJN7osuHElFewD2hAm00k7dGYSZiZiDfkSt/oL/ovgTtz6DU7TLkzrgQuHc+7lHo4fMaq0bX9Yha3tnd294n7p4LB8VKnWjnsqjCUmXRyyUA58pAijgnQ11YwMIkkQ9xnp+7Obhd9/JFLRUDzoeURcjiaCBhQjbSSvWmkkIz+Az6mX3F06acOr1u2mnQFuEmdF6mCFjlezyqNxiGNOhMYMKTV07Ei7CZKaYkbS0ihWJEJ4hiZkaKhAnCg3yZKn8NwoYxiE0ozQMFP/XiSIKzXnvtnkSE/VurcQ//OGsQ6u3YSKKNZE4OWjIGZQh3BRAxxTSbBmc0MQltRkhXiKJMLalJX7MpEomlL8lOZVIrJcOdHnackU6KzXtUl6raZjN537Vr3dWlVZBKfgDFwAB1yBNrgFHdAFGMTgBbyCN+vd+rS+rO/lasFa3ZyAHKyfX4+Xqhg=</latexit>

zn<latexit sha1_base64="kKihJzslZNQcGREdxEzSoq/rx3Q=">AAACMHicbVDLTgIxFG19IoqCLt00gokrMsNGV4bEjUtM5JHAhHRKBxrazth2jDiZ73Crv+DX6Mq49SsswywEPMlNTs65N/fk+BFn2jjOJ9zY3Nre2S3sFfcPSodH5cpxR4exIrRNQh6qno815UzStmGG016kKBY+p11/ejP3u49UaRbKezOLqCfwWLKAEWys5NWSgR+g53SYyLQ2LFedupMBrRM3J1WQozWswNJgFJJYUGkIx1r3XScyXoKVYYTTtDiINY0wmeIx7VsqsaDaS7LUKTq3yggFobIjDcrUvxcJFlrPhG83BTYTverNxf+8fmyCKy9hMooNlWTxKIg5MiGaV4BGTFFi+MwSTBSzWRGZYIWJsUUtfRkrHE0YeUqXVSqzXEuiL9KiLdBdrWuddBp116m7d41q8zqvsgBOwRm4AC64BE1wC1qgDQh4AC/gFbzBd/gBv+D3YnUD5jcnYAnw5xdUjKmh</latexit><latexit sha1_base64="kKihJzslZNQcGREdxEzSoq/rx3Q=">AAACMHicbVDLTgIxFG19IoqCLt00gokrMsNGV4bEjUtM5JHAhHRKBxrazth2jDiZ73Crv+DX6Mq49SsswywEPMlNTs65N/fk+BFn2jjOJ9zY3Nre2S3sFfcPSodH5cpxR4exIrRNQh6qno815UzStmGG016kKBY+p11/ejP3u49UaRbKezOLqCfwWLKAEWys5NWSgR+g53SYyLQ2LFedupMBrRM3J1WQozWswNJgFJJYUGkIx1r3XScyXoKVYYTTtDiINY0wmeIx7VsqsaDaS7LUKTq3yggFobIjDcrUvxcJFlrPhG83BTYTverNxf+8fmyCKy9hMooNlWTxKIg5MiGaV4BGTFFi+MwSTBSzWRGZYIWJsUUtfRkrHE0YeUqXVSqzXEuiL9KiLdBdrWuddBp116m7d41q8zqvsgBOwRm4AC64BE1wC1qgDQh4AC/gFbzBd/gBv+D3YnUD5jcnYAnw5xdUjKmh</latexit><latexit sha1_base64="kKihJzslZNQcGREdxEzSoq/rx3Q=">AAACMHicbVDLTgIxFG19IoqCLt00gokrMsNGV4bEjUtM5JHAhHRKBxrazth2jDiZ73Crv+DX6Mq49SsswywEPMlNTs65N/fk+BFn2jjOJ9zY3Nre2S3sFfcPSodH5cpxR4exIrRNQh6qno815UzStmGG016kKBY+p11/ejP3u49UaRbKezOLqCfwWLKAEWys5NWSgR+g53SYyLQ2LFedupMBrRM3J1WQozWswNJgFJJYUGkIx1r3XScyXoKVYYTTtDiINY0wmeIx7VsqsaDaS7LUKTq3yggFobIjDcrUvxcJFlrPhG83BTYTverNxf+8fmyCKy9hMooNlWTxKIg5MiGaV4BGTFFi+MwSTBSzWRGZYIWJsUUtfRkrHE0YeUqXVSqzXEuiL9KiLdBdrWuddBp116m7d41q8zqvsgBOwRm4AC64BE1wC1qgDQh4AC/gFbzBd/gBv+D3YnUD5jcnYAnw5xdUjKmh</latexit><latexit sha1_base64="kKihJzslZNQcGREdxEzSoq/rx3Q=">AAACMHicbVDLTgIxFG19IoqCLt00gokrMsNGV4bEjUtM5JHAhHRKBxrazth2jDiZ73Crv+DX6Mq49SsswywEPMlNTs65N/fk+BFn2jjOJ9zY3Nre2S3sFfcPSodH5cpxR4exIrRNQh6qno815UzStmGG016kKBY+p11/ejP3u49UaRbKezOLqCfwWLKAEWys5NWSgR+g53SYyLQ2LFedupMBrRM3J1WQozWswNJgFJJYUGkIx1r3XScyXoKVYYTTtDiINY0wmeIx7VsqsaDaS7LUKTq3yggFobIjDcrUvxcJFlrPhG83BTYTverNxf+8fmyCKy9hMooNlWTxKIg5MiGaV4BGTFFi+MwSTBSzWRGZYIWJsUUtfRkrHE0YeUqXVSqzXEuiL9KiLdBdrWuddBp116m7d41q8zqvsgBOwRm4AC64BE1wC1qgDQh4AC/gFbzBd/gBv+D3YnUD5jcnYAnw5xdUjKmh</latexit>

cN,n<latexit sha1_base64="HAXZPqdENJeu1znHRsy3NStWHE0=">AAACNHicbVDLSsNAFJ34rNXaVpduBlvBhZSkG10W3LiSCvYBbQiT6aQdOjMJMxOxhHyJW/0F/0VwJ279BqdpFqb1wIXDOfdyD8ePGFXatj+sre2d3b390kH58KhyXK3VT/oqjCUmPRyyUA59pAijgvQ01YwMI0kQ9xkZ+PPbpT94IlLRUDzqRURcjqaCBhQjbSSvVm0mYz+AOPWS+yuRNr1aw27ZGeAmcXLSADm6Xt2qjCchjjkRGjOk1MixI+0mSGqKGUnL41iRCOE5mpKRoQJxotwkS57CC6NMYBBKM0LDTP17kSCu1IL7ZpMjPVPr3lL8zxvFOrhxEyqiWBOBV4+CmEEdwmUNcEIlwZotDEFYUpMV4hmSCGtTVuHLVKJoRvFzWlSJyHIVRJ+nZVOgs17XJum3W47dch7ajU47r7IEzsA5uAQOuAYdcAe6oAcwiMELeAVv1rv1aX1Z36vVLSu/OQUFWD+/0BaqPQ==</latexit><latexit sha1_base64="HAXZPqdENJeu1znHRsy3NStWHE0=">AAACNHicbVDLSsNAFJ34rNXaVpduBlvBhZSkG10W3LiSCvYBbQiT6aQdOjMJMxOxhHyJW/0F/0VwJ279BqdpFqb1wIXDOfdyD8ePGFXatj+sre2d3b390kH58KhyXK3VT/oqjCUmPRyyUA59pAijgvQ01YwMI0kQ9xkZ+PPbpT94IlLRUDzqRURcjqaCBhQjbSSvVm0mYz+AOPWS+yuRNr1aw27ZGeAmcXLSADm6Xt2qjCchjjkRGjOk1MixI+0mSGqKGUnL41iRCOE5mpKRoQJxotwkS57CC6NMYBBKM0LDTP17kSCu1IL7ZpMjPVPr3lL8zxvFOrhxEyqiWBOBV4+CmEEdwmUNcEIlwZotDEFYUpMV4hmSCGtTVuHLVKJoRvFzWlSJyHIVRJ+nZVOgs17XJum3W47dch7ajU47r7IEzsA5uAQOuAYdcAe6oAcwiMELeAVv1rv1aX1Z36vVLSu/OQUFWD+/0BaqPQ==</latexit><latexit sha1_base64="HAXZPqdENJeu1znHRsy3NStWHE0=">AAACNHicbVDLSsNAFJ34rNXaVpduBlvBhZSkG10W3LiSCvYBbQiT6aQdOjMJMxOxhHyJW/0F/0VwJ279BqdpFqb1wIXDOfdyD8ePGFXatj+sre2d3b390kH58KhyXK3VT/oqjCUmPRyyUA59pAijgvQ01YwMI0kQ9xkZ+PPbpT94IlLRUDzqRURcjqaCBhQjbSSvVm0mYz+AOPWS+yuRNr1aw27ZGeAmcXLSADm6Xt2qjCchjjkRGjOk1MixI+0mSGqKGUnL41iRCOE5mpKRoQJxotwkS57CC6NMYBBKM0LDTP17kSCu1IL7ZpMjPVPr3lL8zxvFOrhxEyqiWBOBV4+CmEEdwmUNcEIlwZotDEFYUpMV4hmSCGtTVuHLVKJoRvFzWlSJyHIVRJ+nZVOgs17XJum3W47dch7ajU47r7IEzsA5uAQOuAYdcAe6oAcwiMELeAVv1rv1aX1Z36vVLSu/OQUFWD+/0BaqPQ==</latexit><latexit sha1_base64="HAXZPqdENJeu1znHRsy3NStWHE0=">AAACNHicbVDLSsNAFJ34rNXaVpduBlvBhZSkG10W3LiSCvYBbQiT6aQdOjMJMxOxhHyJW/0F/0VwJ279BqdpFqb1wIXDOfdyD8ePGFXatj+sre2d3b390kH58KhyXK3VT/oqjCUmPRyyUA59pAijgvQ01YwMI0kQ9xkZ+PPbpT94IlLRUDzqRURcjqaCBhQjbSSvVm0mYz+AOPWS+yuRNr1aw27ZGeAmcXLSADm6Xt2qjCchjjkRGjOk1MixI+0mSGqKGUnL41iRCOE5mpKRoQJxotwkS57CC6NMYBBKM0LDTP17kSCu1IL7ZpMjPVPr3lL8zxvFOrhxEyqiWBOBV4+CmEEdwmUNcEIlwZotDEFYUpMV4hmSCGtTVuHLVKJoRvFzWlSJyHIVRJ+nZVOgs17XJum3W47dch7ajU47r7IEzsA5uAQOuAYdcAe6oAcwiMELeAVv1rv1aX1Z36vVLSu/OQUFWD+/0BaqPQ==</latexit>

cn+1,n<latexit sha1_base64="qheHfKTctYBtCM1JxodXMpawjCU=">AAACNnicbVDLSsNAFJ34rNVqq0s3g60gKCXpRpcFNy4r2Ae0IUymk3bozCTMTNQS8ilu9Rf8FTfuxK2f4DTNwrQeuHA4517u4fgRo0rb9oe1sbm1vbNb2ivvH1QOj6q1454KY4lJF4cslAMfKcKoIF1NNSODSBLEfUb6/ux24fcfiVQ0FA96HhGXo4mgAcVIG8mr1hrJyA8gTr1EXDpXIm141brdtDPAdeLkpA5ydLyaVRmNQxxzIjRmSKmhY0faTZDUFDOSlkexIhHCMzQhQ0MF4kS5SZY9hedGGcMglGaEhpn69yJBXKk5980mR3qqVr2F+J83jHVw4yZURLEmAi8fBTGDOoSLIuCYSoI1mxuCsKQmK8RTJBHWpq7Cl4lE0ZTi57SoEpHlKog+T8umQGe1rnXSazUdu+nct+rtVl5lCZyCM3ABHHAN2uAOdEAXYPAEXsAreLPerU/ry/perm5Y+c0JKMD6+QX+DqrN</latexit><latexit sha1_base64="qheHfKTctYBtCM1JxodXMpawjCU=">AAACNnicbVDLSsNAFJ34rNVqq0s3g60gKCXpRpcFNy4r2Ae0IUymk3bozCTMTNQS8ilu9Rf8FTfuxK2f4DTNwrQeuHA4517u4fgRo0rb9oe1sbm1vbNb2ivvH1QOj6q1454KY4lJF4cslAMfKcKoIF1NNSODSBLEfUb6/ux24fcfiVQ0FA96HhGXo4mgAcVIG8mr1hrJyA8gTr1EXDpXIm141brdtDPAdeLkpA5ydLyaVRmNQxxzIjRmSKmhY0faTZDUFDOSlkexIhHCMzQhQ0MF4kS5SZY9hedGGcMglGaEhpn69yJBXKk5980mR3qqVr2F+J83jHVw4yZURLEmAi8fBTGDOoSLIuCYSoI1mxuCsKQmK8RTJBHWpq7Cl4lE0ZTi57SoEpHlKog+T8umQGe1rnXSazUdu+nct+rtVl5lCZyCM3ABHHAN2uAOdEAXYPAEXsAreLPerU/ry/perm5Y+c0JKMD6+QX+DqrN</latexit><latexit sha1_base64="qheHfKTctYBtCM1JxodXMpawjCU=">AAACNnicbVDLSsNAFJ34rNVqq0s3g60gKCXpRpcFNy4r2Ae0IUymk3bozCTMTNQS8ilu9Rf8FTfuxK2f4DTNwrQeuHA4517u4fgRo0rb9oe1sbm1vbNb2ivvH1QOj6q1454KY4lJF4cslAMfKcKoIF1NNSODSBLEfUb6/ux24fcfiVQ0FA96HhGXo4mgAcVIG8mr1hrJyA8gTr1EXDpXIm141brdtDPAdeLkpA5ydLyaVRmNQxxzIjRmSKmhY0faTZDUFDOSlkexIhHCMzQhQ0MF4kS5SZY9hedGGcMglGaEhpn69yJBXKk5980mR3qqVr2F+J83jHVw4yZURLEmAi8fBTGDOoSLIuCYSoI1mxuCsKQmK8RTJBHWpq7Cl4lE0ZTi57SoEpHlKog+T8umQGe1rnXSazUdu+nct+rtVl5lCZyCM3ABHHAN2uAOdEAXYPAEXsAreLPerU/ry/perm5Y+c0JKMD6+QX+DqrN</latexit><latexit sha1_base64="qheHfKTctYBtCM1JxodXMpawjCU=">AAACNnicbVDLSsNAFJ34rNVqq0s3g60gKCXpRpcFNy4r2Ae0IUymk3bozCTMTNQS8ilu9Rf8FTfuxK2f4DTNwrQeuHA4517u4fgRo0rb9oe1sbm1vbNb2ivvH1QOj6q1454KY4lJF4cslAMfKcKoIF1NNSODSBLEfUb6/ux24fcfiVQ0FA96HhGXo4mgAcVIG8mr1hrJyA8gTr1EXDpXIm141brdtDPAdeLkpA5ydLyaVRmNQxxzIjRmSKmhY0faTZDUFDOSlkexIhHCMzQhQ0MF4kS5SZY9hedGGcMglGaEhpn69yJBXKk5980mR3qqVr2F+J83jHVw4yZURLEmAi8fBTGDOoSLIuCYSoI1mxuCsKQmK8RTJBHWpq7Cl4lE0ZTi57SoEpHlKog+T8umQGe1rnXSazUdu+nct+rtVl5lCZyCM3ABHHAN2uAOdEAXYPAEXsAreLPerU/ry/perm5Y+c0JKMD6+QX+DqrN</latexit>

cN�1,n<latexit sha1_base64="bDZlai9xeNyQNdV70P7y2VFq1FM=">AAACNnicbVC7TsMwFHXKqxQKLYwsFi0SA1RJFxgrsTChItGH1EaR4zqtVduJbAeoonwKK/wCv8LChlj5BNw0A2050pWOzrlX9+j4EaNK2/aHVdjY3NreKe6W9vbLB4eV6lFXhbHEpINDFsq+jxRhVJCOppqRfiQJ4j4jPX96M/d7j0QqGooHPYuIy9FY0IBipI3kVar1ZOgHEKdecnfpXIi07lVqdsPOANeJk5MayNH2qlZ5OApxzInQmCGlBo4daTdBUlPMSFoaxopECE/RmAwMFYgT5SZZ9hSeGWUEg1CaERpm6t+LBHGlZtw3mxzpiVr15uJ/3iDWwbWbUBHFmgi8eBTEDOoQzouAIyoJ1mxmCMKSmqwQT5BEWJu6lr6MJYomFD+nyyoRWa4l0edpyRTorNa1TrrNhmM3nPtmrdXMqyyCE3AKzoEDrkAL3II26AAMnsALeAVv1rv1aX1Z34vVgpXfHIMlWD+/yNiqrw==</latexit><latexit sha1_base64="bDZlai9xeNyQNdV70P7y2VFq1FM=">AAACNnicbVC7TsMwFHXKqxQKLYwsFi0SA1RJFxgrsTChItGH1EaR4zqtVduJbAeoonwKK/wCv8LChlj5BNw0A2050pWOzrlX9+j4EaNK2/aHVdjY3NreKe6W9vbLB4eV6lFXhbHEpINDFsq+jxRhVJCOppqRfiQJ4j4jPX96M/d7j0QqGooHPYuIy9FY0IBipI3kVar1ZOgHEKdecnfpXIi07lVqdsPOANeJk5MayNH2qlZ5OApxzInQmCGlBo4daTdBUlPMSFoaxopECE/RmAwMFYgT5SZZ9hSeGWUEg1CaERpm6t+LBHGlZtw3mxzpiVr15uJ/3iDWwbWbUBHFmgi8eBTEDOoQzouAIyoJ1mxmCMKSmqwQT5BEWJu6lr6MJYomFD+nyyoRWa4l0edpyRTorNa1TrrNhmM3nPtmrdXMqyyCE3AKzoEDrkAL3II26AAMnsALeAVv1rv1aX1Z34vVgpXfHIMlWD+/yNiqrw==</latexit><latexit sha1_base64="bDZlai9xeNyQNdV70P7y2VFq1FM=">AAACNnicbVC7TsMwFHXKqxQKLYwsFi0SA1RJFxgrsTChItGH1EaR4zqtVduJbAeoonwKK/wCv8LChlj5BNw0A2050pWOzrlX9+j4EaNK2/aHVdjY3NreKe6W9vbLB4eV6lFXhbHEpINDFsq+jxRhVJCOppqRfiQJ4j4jPX96M/d7j0QqGooHPYuIy9FY0IBipI3kVar1ZOgHEKdecnfpXIi07lVqdsPOANeJk5MayNH2qlZ5OApxzInQmCGlBo4daTdBUlPMSFoaxopECE/RmAwMFYgT5SZZ9hSeGWUEg1CaERpm6t+LBHGlZtw3mxzpiVr15uJ/3iDWwbWbUBHFmgi8eBTEDOoQzouAIyoJ1mxmCMKSmqwQT5BEWJu6lr6MJYomFD+nyyoRWa4l0edpyRTorNa1TrrNhmM3nPtmrdXMqyyCE3AKzoEDrkAL3II26AAMnsALeAVv1rv1aX1Z34vVgpXfHIMlWD+/yNiqrw==</latexit><latexit sha1_base64="bDZlai9xeNyQNdV70P7y2VFq1FM=">AAACNnicbVC7TsMwFHXKqxQKLYwsFi0SA1RJFxgrsTChItGH1EaR4zqtVduJbAeoonwKK/wCv8LChlj5BNw0A2050pWOzrlX9+j4EaNK2/aHVdjY3NreKe6W9vbLB4eV6lFXhbHEpINDFsq+jxRhVJCOppqRfiQJ4j4jPX96M/d7j0QqGooHPYuIy9FY0IBipI3kVar1ZOgHEKdecnfpXIi07lVqdsPOANeJk5MayNH2qlZ5OApxzInQmCGlBo4daTdBUlPMSFoaxopECE/RmAwMFYgT5SZZ9hSeGWUEg1CaERpm6t+LBHGlZtw3mxzpiVr15uJ/3iDWwbWbUBHFmgi8eBTEDOoQzouAIyoJ1mxmCMKSmqwQT5BEWJu6lr6MJYomFD+nyyoRWa4l0edpyRTorNa1TrrNhmM3nPtmrdXMqyyCE3AKzoEDrkAL3II26AAMnsALeAVv1rv1aX1Z34vVgpXfHIMlWD+/yNiqrw==</latexit>

⇠<latexit sha1_base64="ZMThSV6AJuqyqDZRp9cO5+n+/oc=">AAACKXicbVC9TsMwGLT5LYVCCyOLRYvEVCVdYKzEwlgk+iO1UeW4TmtqO5HtIKoo78AKr8DTsAErL4KbZiAtJ1k63X2fvvP5EWfaOM4X3Nre2d3bLx2UD48qxyfV2mlPh7EitEtCHqqBjzXlTNKuYYbTQaQoFj6nfX9+u/T7T1RpFsoHs4ioJ/BUsoARbKzUa4w0E41xte40nQxok7g5qYMcnXENVkaTkMSCSkM41nroOpHxEqwMI5ym5VGsaYTJHE/p0FKJBdVeksVN0aVVJigIlX3SoEz9u5FgofVC+HZSYDPT695S/M8bxia48RImo9hQSVaHgpgjE6Ll39GEKUoMX1iCiWI2KyIzrDAxtqHClanC0YyR57SoUpnlKoi+SMu2QHe9rk3SazVdp+net+rtVl5lCZyDC3AFXHAN2uAOdEAXEPAIXsAreIPv8AN+wu/V6BbMd85AAfDnFz7cpnk=</latexit><latexit sha1_base64="ZMThSV6AJuqyqDZRp9cO5+n+/oc=">AAACKXicbVC9TsMwGLT5LYVCCyOLRYvEVCVdYKzEwlgk+iO1UeW4TmtqO5HtIKoo78AKr8DTsAErL4KbZiAtJ1k63X2fvvP5EWfaOM4X3Nre2d3bLx2UD48qxyfV2mlPh7EitEtCHqqBjzXlTNKuYYbTQaQoFj6nfX9+u/T7T1RpFsoHs4ioJ/BUsoARbKzUa4w0E41xte40nQxok7g5qYMcnXENVkaTkMSCSkM41nroOpHxEqwMI5ym5VGsaYTJHE/p0FKJBdVeksVN0aVVJigIlX3SoEz9u5FgofVC+HZSYDPT695S/M8bxia48RImo9hQSVaHgpgjE6Ll39GEKUoMX1iCiWI2KyIzrDAxtqHClanC0YyR57SoUpnlKoi+SMu2QHe9rk3SazVdp+net+rtVl5lCZyDC3AFXHAN2uAOdEAXEPAIXsAreIPv8AN+wu/V6BbMd85AAfDnFz7cpnk=</latexit><latexit sha1_base64="ZMThSV6AJuqyqDZRp9cO5+n+/oc=">AAACKXicbVC9TsMwGLT5LYVCCyOLRYvEVCVdYKzEwlgk+iO1UeW4TmtqO5HtIKoo78AKr8DTsAErL4KbZiAtJ1k63X2fvvP5EWfaOM4X3Nre2d3bLx2UD48qxyfV2mlPh7EitEtCHqqBjzXlTNKuYYbTQaQoFj6nfX9+u/T7T1RpFsoHs4ioJ/BUsoARbKzUa4w0E41xte40nQxok7g5qYMcnXENVkaTkMSCSkM41nroOpHxEqwMI5ym5VGsaYTJHE/p0FKJBdVeksVN0aVVJigIlX3SoEz9u5FgofVC+HZSYDPT695S/M8bxia48RImo9hQSVaHgpgjE6Ll39GEKUoMX1iCiWI2KyIzrDAxtqHClanC0YyR57SoUpnlKoi+SMu2QHe9rk3SazVdp+net+rtVl5lCZyDC3AFXHAN2uAOdEAXEPAIXsAreIPv8AN+wu/V6BbMd85AAfDnFz7cpnk=</latexit><latexit sha1_base64="ZMThSV6AJuqyqDZRp9cO5+n+/oc=">AAACKXicbVC9TsMwGLT5LYVCCyOLRYvEVCVdYKzEwlgk+iO1UeW4TmtqO5HtIKoo78AKr8DTsAErL4KbZiAtJ1k63X2fvvP5EWfaOM4X3Nre2d3bLx2UD48qxyfV2mlPh7EitEtCHqqBjzXlTNKuYYbTQaQoFj6nfX9+u/T7T1RpFsoHs4ioJ/BUsoARbKzUa4w0E41xte40nQxok7g5qYMcnXENVkaTkMSCSkM41nroOpHxEqwMI5ym5VGsaYTJHE/p0FKJBdVeksVN0aVVJigIlX3SoEz9u5FgofVC+HZSYDPT695S/M8bxia48RImo9hQSVaHgpgjE6Ll39GEKUoMX1iCiWI2KyIzrDAxtqHClanC0YyR57SoUpnlKoi+SMu2QHe9rk3SazVdp+net+rtVl5lCZyDC3AFXHAN2uAOdEAXEPAIXsAreIPv8AN+wu/V6BbMd85AAfDnFz7cpnk=</latexit> ⇠<latexit sha1_base64="ZMThSV6AJuqyqDZRp9cO5+n+/oc=">AAACKXicbVC9TsMwGLT5LYVCCyOLRYvEVCVdYKzEwlgk+iO1UeW4TmtqO5HtIKoo78AKr8DTsAErL4KbZiAtJ1k63X2fvvP5EWfaOM4X3Nre2d3bLx2UD48qxyfV2mlPh7EitEtCHqqBjzXlTNKuYYbTQaQoFj6nfX9+u/T7T1RpFsoHs4ioJ/BUsoARbKzUa4w0E41xte40nQxok7g5qYMcnXENVkaTkMSCSkM41nroOpHxEqwMI5ym5VGsaYTJHE/p0FKJBdVeksVN0aVVJigIlX3SoEz9u5FgofVC+HZSYDPT695S/M8bxia48RImo9hQSVaHgpgjE6Ll39GEKUoMX1iCiWI2KyIzrDAxtqHClanC0YyR57SoUpnlKoi+SMu2QHe9rk3SazVdp+net+rtVl5lCZyDC3AFXHAN2uAOdEAXEPAIXsAreIPv8AN+wu/V6BbMd85AAfDnFz7cpnk=</latexit><latexit sha1_base64="ZMThSV6AJuqyqDZRp9cO5+n+/oc=">AAACKXicbVC9TsMwGLT5LYVCCyOLRYvEVCVdYKzEwlgk+iO1UeW4TmtqO5HtIKoo78AKr8DTsAErL4KbZiAtJ1k63X2fvvP5EWfaOM4X3Nre2d3bLx2UD48qxyfV2mlPh7EitEtCHqqBjzXlTNKuYYbTQaQoFj6nfX9+u/T7T1RpFsoHs4ioJ/BUsoARbKzUa4w0E41xte40nQxok7g5qYMcnXENVkaTkMSCSkM41nroOpHxEqwMI5ym5VGsaYTJHE/p0FKJBdVeksVN0aVVJigIlX3SoEz9u5FgofVC+HZSYDPT695S/M8bxia48RImo9hQSVaHgpgjE6Ll39GEKUoMX1iCiWI2KyIzrDAxtqHClanC0YyR57SoUpnlKoi+SMu2QHe9rk3SazVdp+net+rtVl5lCZyDC3AFXHAN2uAOdEAXEPAIXsAreIPv8AN+wu/V6BbMd85AAfDnFz7cpnk=</latexit><latexit sha1_base64="ZMThSV6AJuqyqDZRp9cO5+n+/oc=">AAACKXicbVC9TsMwGLT5LYVCCyOLRYvEVCVdYKzEwlgk+iO1UeW4TmtqO5HtIKoo78AKr8DTsAErL4KbZiAtJ1k63X2fvvP5EWfaOM4X3Nre2d3bLx2UD48qxyfV2mlPh7EitEtCHqqBjzXlTNKuYYbTQaQoFj6nfX9+u/T7T1RpFsoHs4ioJ/BUsoARbKzUa4w0E41xte40nQxok7g5qYMcnXENVkaTkMSCSkM41nroOpHxEqwMI5ym5VGsaYTJHE/p0FKJBdVeksVN0aVVJigIlX3SoEz9u5FgofVC+HZSYDPT695S/M8bxia48RImo9hQSVaHgpgjE6Ll39GEKUoMX1iCiWI2KyIzrDAxtqHClanC0YyR57SoUpnlKoi+SMu2QHe9rk3SazVdp+net+rtVl5lCZyDC3AFXHAN2uAOdEAXEPAIXsAreIPv8AN+wu/V6BbMd85AAfDnFz7cpnk=</latexit><latexit sha1_base64="ZMThSV6AJuqyqDZRp9cO5+n+/oc=">AAACKXicbVC9TsMwGLT5LYVCCyOLRYvEVCVdYKzEwlgk+iO1UeW4TmtqO5HtIKoo78AKr8DTsAErL4KbZiAtJ1k63X2fvvP5EWfaOM4X3Nre2d3bLx2UD48qxyfV2mlPh7EitEtCHqqBjzXlTNKuYYbTQaQoFj6nfX9+u/T7T1RpFsoHs4ioJ/BUsoARbKzUa4w0E41xte40nQxok7g5qYMcnXENVkaTkMSCSkM41nroOpHxEqwMI5ym5VGsaYTJHE/p0FKJBdVeksVN0aVVJigIlX3SoEz9u5FgofVC+HZSYDPT695S/M8bxia48RImo9hQSVaHgpgjE6Ll39GEKUoMX1iCiWI2KyIzrDAxtqHClanC0YyR57SoUpnlKoi+SMu2QHe9rk3SazVdp+net+rtVl5lCZyDC3AFXHAN2uAOdEAXEPAIXsAreIPv8AN+wu/V6BbMd85AAfDnFz7cpnk=</latexit> ⇠<latexit sha1_base64="ZMThSV6AJuqyqDZRp9cO5+n+/oc=">AAACKXicbVC9TsMwGLT5LYVCCyOLRYvEVCVdYKzEwlgk+iO1UeW4TmtqO5HtIKoo78AKr8DTsAErL4KbZiAtJ1k63X2fvvP5EWfaOM4X3Nre2d3bLx2UD48qxyfV2mlPh7EitEtCHqqBjzXlTNKuYYbTQaQoFj6nfX9+u/T7T1RpFsoHs4ioJ/BUsoARbKzUa4w0E41xte40nQxok7g5qYMcnXENVkaTkMSCSkM41nroOpHxEqwMI5ym5VGsaYTJHE/p0FKJBdVeksVN0aVVJigIlX3SoEz9u5FgofVC+HZSYDPT695S/M8bxia48RImo9hQSVaHgpgjE6Ll39GEKUoMX1iCiWI2KyIzrDAxtqHClanC0YyR57SoUpnlKoi+SMu2QHe9rk3SazVdp+net+rtVl5lCZyDC3AFXHAN2uAOdEAXEPAIXsAreIPv8AN+wu/V6BbMd85AAfDnFz7cpnk=</latexit><latexit sha1_base64="ZMThSV6AJuqyqDZRp9cO5+n+/oc=">AAACKXicbVC9TsMwGLT5LYVCCyOLRYvEVCVdYKzEwlgk+iO1UeW4TmtqO5HtIKoo78AKr8DTsAErL4KbZiAtJ1k63X2fvvP5EWfaOM4X3Nre2d3bLx2UD48qxyfV2mlPh7EitEtCHqqBjzXlTNKuYYbTQaQoFj6nfX9+u/T7T1RpFsoHs4ioJ/BUsoARbKzUa4w0E41xte40nQxok7g5qYMcnXENVkaTkMSCSkM41nroOpHxEqwMI5ym5VGsaYTJHE/p0FKJBdVeksVN0aVVJigIlX3SoEz9u5FgofVC+HZSYDPT695S/M8bxia48RImo9hQSVaHgpgjE6Ll39GEKUoMX1iCiWI2KyIzrDAxtqHClanC0YyR57SoUpnlKoi+SMu2QHe9rk3SazVdp+net+rtVl5lCZyDC3AFXHAN2uAOdEAXEPAIXsAreIPv8AN+wu/V6BbMd85AAfDnFz7cpnk=</latexit><latexit sha1_base64="ZMThSV6AJuqyqDZRp9cO5+n+/oc=">AAACKXicbVC9TsMwGLT5LYVCCyOLRYvEVCVdYKzEwlgk+iO1UeW4TmtqO5HtIKoo78AKr8DTsAErL4KbZiAtJ1k63X2fvvP5EWfaOM4X3Nre2d3bLx2UD48qxyfV2mlPh7EitEtCHqqBjzXlTNKuYYbTQaQoFj6nfX9+u/T7T1RpFsoHs4ioJ/BUsoARbKzUa4w0E41xte40nQxok7g5qYMcnXENVkaTkMSCSkM41nroOpHxEqwMI5ym5VGsaYTJHE/p0FKJBdVeksVN0aVVJigIlX3SoEz9u5FgofVC+HZSYDPT695S/M8bxia48RImo9hQSVaHgpgjE6Ll39GEKUoMX1iCiWI2KyIzrDAxtqHClanC0YyR57SoUpnlKoi+SMu2QHe9rk3SazVdp+net+rtVl5lCZyDC3AFXHAN2uAOdEAXEPAIXsAreIPv8AN+wu/V6BbMd85AAfDnFz7cpnk=</latexit><latexit sha1_base64="ZMThSV6AJuqyqDZRp9cO5+n+/oc=">AAACKXicbVC9TsMwGLT5LYVCCyOLRYvEVCVdYKzEwlgk+iO1UeW4TmtqO5HtIKoo78AKr8DTsAErL4KbZiAtJ1k63X2fvvP5EWfaOM4X3Nre2d3bLx2UD48qxyfV2mlPh7EitEtCHqqBjzXlTNKuYYbTQaQoFj6nfX9+u/T7T1RpFsoHs4ioJ/BUsoARbKzUa4w0E41xte40nQxok7g5qYMcnXENVkaTkMSCSkM41nroOpHxEqwMI5ym5VGsaYTJHE/p0FKJBdVeksVN0aVVJigIlX3SoEz9u5FgofVC+HZSYDPT695S/M8bxia48RImo9hQSVaHgpgjE6Ll39GEKUoMX1iCiWI2KyIzrDAxtqHClanC0YyR57SoUpnlKoi+SMu2QHe9rk3SazVdp+net+rtVl5lCZyDC3AFXHAN2uAOdEAXEPAIXsAreIPv8AN+wu/V6BbMd85AAfDnFz7cpnk=</latexit>

x<latexit sha1_base64="/TgEZ8DA1yQEHqOaCU6+0xYhQdo=">AAACKHicbVDLTgIxFL3jE1EUdOmmkZi4IjNsdGMkceMSEwdIgJBO6UBD25m0HQOZTOIfuNVf8Af8DXeGrV9ieSwEPEmTk3PuzT09QcyZNq47dba2d3b39nMH+cOjwvFJsXTa0FGiCPVJxCPVCrCmnEnqG2Y4bcWKYhFw2gxG9zO/+UyVZpF8MpOYdgUeSBYygo2V/E4QonGvWHYr7hxok3hLUr77zN++AEC9V3IKnX5EEkGlIRxr3fbc2HRTrAwjnGb5TqJpjMkID2jbUokF1d10njZDl1bpozBS9kmD5urfjRQLrScisJMCm6Fe92bif147MeFNN2UyTgyVZHEoTDgyEZp9HfWZosTwiSWYKGazIjLEChNjC1q5MlA4HjIyzlZVKue5VsRAZHlboLde1yZpVCueW/EevXKtCgvk4Bwu4Ao8uIYaPEAdfCDA4BXe4N35cL6cb2e6GN1yljtnsALn5xeEPqf6</latexit><latexit sha1_base64="gK8QTxAlLx2DlXggTG1H43s+Lts=">AAACKHicbVDNTgIxGOziH66ioEcvjcTEE9nlohcjiRePmLhAAhvSLV1oaLubtmsgm30Gr3r25gv4Gt4IV5/EsnAQcJImk5nvyzedIGZUaceZW4Wd3b39g+KhfXRcOjktV85aKkokJh6OWCQ7AVKEUUE8TTUjnVgSxANG2sH4YeG3X4hUNBLPehoTn6OhoCHFSBvJ6wUhnPTLVafm5IDbxF2R6v2XfRd/zOxmv2KVeoMIJ5wIjRlSqus6sfZTJDXFjGR2L1EkRniMhqRrqECcKD/N02bwyigDGEbSPKFhrv7dSBFXasoDM8mRHqlNbyH+53UTHd76KRVxoonAy0NhwqCO4OLrcEAlwZpNDUFYUpMV4hGSCGtT0NqVoUTxiOJJtq4SkedaEwOe2aZAd7OubdKq11yn5j651UYdLFEEF+ASXAMX3IAGeARN4AEMKHgFb+Dd+rS+rZk1X44WrNXOOViD9fMLzP2pbg==</latexit><latexit sha1_base64="gK8QTxAlLx2DlXggTG1H43s+Lts=">AAACKHicbVDNTgIxGOziH66ioEcvjcTEE9nlohcjiRePmLhAAhvSLV1oaLubtmsgm30Gr3r25gv4Gt4IV5/EsnAQcJImk5nvyzedIGZUaceZW4Wd3b39g+KhfXRcOjktV85aKkokJh6OWCQ7AVKEUUE8TTUjnVgSxANG2sH4YeG3X4hUNBLPehoTn6OhoCHFSBvJ6wUhnPTLVafm5IDbxF2R6v2XfRd/zOxmv2KVeoMIJ5wIjRlSqus6sfZTJDXFjGR2L1EkRniMhqRrqECcKD/N02bwyigDGEbSPKFhrv7dSBFXasoDM8mRHqlNbyH+53UTHd76KRVxoonAy0NhwqCO4OLrcEAlwZpNDUFYUpMV4hGSCGtT0NqVoUTxiOJJtq4SkedaEwOe2aZAd7OubdKq11yn5j651UYdLFEEF+ASXAMX3IAGeARN4AEMKHgFb+Dd+rS+rZk1X44WrNXOOViD9fMLzP2pbg==</latexit><latexit sha1_base64="3bLQSHvauaU7/tGQr69K5GT0Cc4=">AAACKHicbVDNTgIxGGz9RRQFPXppJCaeyC4XPZJ48YiJCySwId3ShYa2u2m7BrLZZ/Cqr+DTeDNcfRLLsgcXnKTJZOb78k0niDnTxnFWcG//4PDouHJSPT2rnV/UG5c9HSWKUI9EPFKDAGvKmaSeYYbTQawoFgGn/WD+uPb7r1RpFskXs4ypL/BUspARbKzkjYIQLcb1ptNycqBd4hakCQp0xw1YG00ikggqDeFY66HrxMZPsTKMcJpVR4mmMSZzPKVDSyUWVPtpnjZDt1aZoDBS9kmDcvXvRoqF1ksR2EmBzUxve2vxP2+YmPDBT5mME0Ml2RwKE45MhNZfRxOmKDF8aQkmitmsiMywwsTYgkpXpgrHM0YWWVmlMs9VEgORVW2B7nZdu6TXbrlOy312m512UWUFXIMbcAdccA864Al0gQcIYOANvIMP+Am/4DdcbUb3YLFzBUqAP7/CW6Y9</latexit>

Encoder<latexit sha1_base64="FabAt1BdB4lmBXTXHYJUGkGfcMM=">AAACKnicbVDLSsNAFJ34rNVqq0s3wSK4Kkk3uiyI4LKCfUAbymRy0w6dmYSZiVhCPsKt/oJf46649UOcplmY1gMDh3Pu5Z45fsyo0o6ztHZ29/YPDitH1eOT2ulZvXHeV1EiCfRIxCI59LECRgX0NNUMhrEEzH0GA39+v/IHLyAVjcSzXsTgcTwVNKQEayMNHgSJApCTetNpOTnsbeIWpIkKdCcNqzYOIpJwEJowrNTIdWLtpVhqShhk1XGiIMZkjqcwMlRgDspL87yZfW2UwA4jaZ7Qdq7+3UgxV2rBfTPJsZ6pTW8l/ueNEh3eeSkVcaJBkPWhMGG2juzV5+2ASiCaLQzBRFKT1SYzLDHRpqLSlanE8YyS16ysgshzlUSfZ1VToLtZ1zbpt1uu03Kf2s1Ou6iygi7RFbpBLrpFHfSIuqiHCJqjN/SOPqxP68taWt/r0R2r2LlAJVg/v9hjp1Y=</latexit><latexit sha1_base64="FabAt1BdB4lmBXTXHYJUGkGfcMM=">AAACKnicbVDLSsNAFJ34rNVqq0s3wSK4Kkk3uiyI4LKCfUAbymRy0w6dmYSZiVhCPsKt/oJf46649UOcplmY1gMDh3Pu5Z45fsyo0o6ztHZ29/YPDitH1eOT2ulZvXHeV1EiCfRIxCI59LECRgX0NNUMhrEEzH0GA39+v/IHLyAVjcSzXsTgcTwVNKQEayMNHgSJApCTetNpOTnsbeIWpIkKdCcNqzYOIpJwEJowrNTIdWLtpVhqShhk1XGiIMZkjqcwMlRgDspL87yZfW2UwA4jaZ7Qdq7+3UgxV2rBfTPJsZ6pTW8l/ueNEh3eeSkVcaJBkPWhMGG2juzV5+2ASiCaLQzBRFKT1SYzLDHRpqLSlanE8YyS16ysgshzlUSfZ1VToLtZ1zbpt1uu03Kf2s1Ou6iygi7RFbpBLrpFHfSIuqiHCJqjN/SOPqxP68taWt/r0R2r2LlAJVg/v9hjp1Y=</latexit><latexit sha1_base64="FabAt1BdB4lmBXTXHYJUGkGfcMM=">AAACKnicbVDLSsNAFJ34rNVqq0s3wSK4Kkk3uiyI4LKCfUAbymRy0w6dmYSZiVhCPsKt/oJf46649UOcplmY1gMDh3Pu5Z45fsyo0o6ztHZ29/YPDitH1eOT2ulZvXHeV1EiCfRIxCI59LECRgX0NNUMhrEEzH0GA39+v/IHLyAVjcSzXsTgcTwVNKQEayMNHgSJApCTetNpOTnsbeIWpIkKdCcNqzYOIpJwEJowrNTIdWLtpVhqShhk1XGiIMZkjqcwMlRgDspL87yZfW2UwA4jaZ7Qdq7+3UgxV2rBfTPJsZ6pTW8l/ueNEh3eeSkVcaJBkPWhMGG2juzV5+2ASiCaLQzBRFKT1SYzLDHRpqLSlanE8YyS16ysgshzlUSfZ1VToLtZ1zbpt1uu03Kf2s1Ou6iygi7RFbpBLrpFHfSIuqiHCJqjN/SOPqxP68taWt/r0R2r2LlAJVg/v9hjp1Y=</latexit><latexit sha1_base64="FabAt1BdB4lmBXTXHYJUGkGfcMM=">AAACKnicbVDLSsNAFJ34rNVqq0s3wSK4Kkk3uiyI4LKCfUAbymRy0w6dmYSZiVhCPsKt/oJf46649UOcplmY1gMDh3Pu5Z45fsyo0o6ztHZ29/YPDitH1eOT2ulZvXHeV1EiCfRIxCI59LECRgX0NNUMhrEEzH0GA39+v/IHLyAVjcSzXsTgcTwVNKQEayMNHgSJApCTetNpOTnsbeIWpIkKdCcNqzYOIpJwEJowrNTIdWLtpVhqShhk1XGiIMZkjqcwMlRgDspL87yZfW2UwA4jaZ7Qdq7+3UgxV2rBfTPJsZ6pTW8l/ueNEh3eeSkVcaJBkPWhMGG2juzV5+2ASiCaLQzBRFKT1SYzLDHRpqLSlanE8YyS16ysgshzlUSfZ1VToLtZ1zbpt1uu03Kf2s1Ou6iygi7RFbpBLrpFHfSIuqiHCJqjN/SOPqxP68taWt/r0R2r2LlAJVg/v9hjp1Y=</latexit>

MLP(TD)n

<latexit sha1_base64="6gZYS0e+iWJJa4JZnk7QuzMEPZQ=">AAACPnicbZDPTgIxEMa7+B9FUU/Gy0YwwQvZ5aJHEj140AQTURNA0i0DNLTdTTtrIJuNT+NVX8HX8AW8Ga8eLchBxC9p8uU3M5npF0SCG/S8NyezsLi0vLK6ll3fyG1u5bd3bkwYawZ1FopQ3wXUgOAK6shRwF2kgcpAwG0wOB3Xbx9AGx6qaxxF0JK0p3iXM4oWtfN7xSbCELVMLi9q6X1Suj47StuJSovtfMErexO588afmgKZqtbednLNTshiCQqZoMY0fC/CVkI1ciYgzTZjAxFlA9qDhrWKSjCtZPKH1D20pON2Q22fQndCf08kVBozkoHtlBT75m9tDP+rNWLsnrQSrqIYQbGfRd1YuBi640DcDtfAUIysoUxze6vL+lRThja2mS09TaM+Z8N0loKa3DUDA5lmbYD+37jmzU2l7Htl/6pSqFamUa6SfXJASsQnx6RKzkmN1Akjj+SJPJMX59V5dz6cz5/WjDOd2SUzcr6+AZ6Frqg=</latexit><latexit sha1_base64="6gZYS0e+iWJJa4JZnk7QuzMEPZQ=">AAACPnicbZDPTgIxEMa7+B9FUU/Gy0YwwQvZ5aJHEj140AQTURNA0i0DNLTdTTtrIJuNT+NVX8HX8AW8Ga8eLchBxC9p8uU3M5npF0SCG/S8NyezsLi0vLK6ll3fyG1u5bd3bkwYawZ1FopQ3wXUgOAK6shRwF2kgcpAwG0wOB3Xbx9AGx6qaxxF0JK0p3iXM4oWtfN7xSbCELVMLi9q6X1Suj47StuJSovtfMErexO588afmgKZqtbednLNTshiCQqZoMY0fC/CVkI1ciYgzTZjAxFlA9qDhrWKSjCtZPKH1D20pON2Q22fQndCf08kVBozkoHtlBT75m9tDP+rNWLsnrQSrqIYQbGfRd1YuBi640DcDtfAUIysoUxze6vL+lRThja2mS09TaM+Z8N0loKa3DUDA5lmbYD+37jmzU2l7Htl/6pSqFamUa6SfXJASsQnx6RKzkmN1Akjj+SJPJMX59V5dz6cz5/WjDOd2SUzcr6+AZ6Frqg=</latexit><latexit sha1_base64="6gZYS0e+iWJJa4JZnk7QuzMEPZQ=">AAACPnicbZDPTgIxEMa7+B9FUU/Gy0YwwQvZ5aJHEj140AQTURNA0i0DNLTdTTtrIJuNT+NVX8HX8AW8Ga8eLchBxC9p8uU3M5npF0SCG/S8NyezsLi0vLK6ll3fyG1u5bd3bkwYawZ1FopQ3wXUgOAK6shRwF2kgcpAwG0wOB3Xbx9AGx6qaxxF0JK0p3iXM4oWtfN7xSbCELVMLi9q6X1Suj47StuJSovtfMErexO588afmgKZqtbednLNTshiCQqZoMY0fC/CVkI1ciYgzTZjAxFlA9qDhrWKSjCtZPKH1D20pON2Q22fQndCf08kVBozkoHtlBT75m9tDP+rNWLsnrQSrqIYQbGfRd1YuBi640DcDtfAUIysoUxze6vL+lRThja2mS09TaM+Z8N0loKa3DUDA5lmbYD+37jmzU2l7Htl/6pSqFamUa6SfXJASsQnx6RKzkmN1Akjj+SJPJMX59V5dz6cz5/WjDOd2SUzcr6+AZ6Frqg=</latexit><latexit sha1_base64="6gZYS0e+iWJJa4JZnk7QuzMEPZQ=">AAACPnicbZDPTgIxEMa7+B9FUU/Gy0YwwQvZ5aJHEj140AQTURNA0i0DNLTdTTtrIJuNT+NVX8HX8AW8Ga8eLchBxC9p8uU3M5npF0SCG/S8NyezsLi0vLK6ll3fyG1u5bd3bkwYawZ1FopQ3wXUgOAK6shRwF2kgcpAwG0wOB3Xbx9AGx6qaxxF0JK0p3iXM4oWtfN7xSbCELVMLi9q6X1Suj47StuJSovtfMErexO588afmgKZqtbednLNTshiCQqZoMY0fC/CVkI1ciYgzTZjAxFlA9qDhrWKSjCtZPKH1D20pON2Q22fQndCf08kVBozkoHtlBT75m9tDP+rNWLsnrQSrqIYQbGfRd1YuBi640DcDtfAUIysoUxze6vL+lRThja2mS09TaM+Z8N0loKa3DUDA5lmbYD+37jmzU2l7Htl/6pSqFamUa6SfXJASsQnx6RKzkmN1Akjj+SJPJMX59V5dz6cz5/WjDOd2SUzcr6+AZ6Frqg=</latexit>

MLP(BU)n

<latexit sha1_base64="U/UizcLQvVoX2qSMQh+/wo9LbpQ=">AAACPnicbZDPSgMxEMaz/q3Vaqsn8bLYCnopu73oUfTiQaGCrUJbSzadtqFJdklmxbIsPo1XfQVfwxfwJl49mv452OoHgY/fzDCTL4gEN+h5787C4tLyympmLbu+kdvcyhe26yaMNYMaC0Wo7wJqQHAFNeQo4C7SQGUg4DYYnI/qtw+gDQ/VDQ4jaEnaU7zLGUWL2vndUhPhEbVMri6r6X1yeFY7StuJSkvtfNEre2O5f40/NUUyVbVdcHLNTshiCQqZoMY0fC/CVkI1ciYgzTZjAxFlA9qDhrWKSjCtZPyH1D2wpON2Q22fQndMf08kVBozlIHtlBT7Zr42gv/VGjF2T1oJV1GMoNhkUTcWLobuKBC3wzUwFENrKNPc3uqyPtWUoY1tZktP06jP2WM6S0GN75qBgUyzNkB/Pq6/pl4p+17Zv64UTyvTKDNkj+yTQ+KTY3JKLkiV1AgjT+SZvJBX5835cD6dr0nrgjOd2SEzcr5/AJysrqc=</latexit><latexit sha1_base64="U/UizcLQvVoX2qSMQh+/wo9LbpQ=">AAACPnicbZDPSgMxEMaz/q3Vaqsn8bLYCnopu73oUfTiQaGCrUJbSzadtqFJdklmxbIsPo1XfQVfwxfwJl49mv452OoHgY/fzDCTL4gEN+h5787C4tLyympmLbu+kdvcyhe26yaMNYMaC0Wo7wJqQHAFNeQo4C7SQGUg4DYYnI/qtw+gDQ/VDQ4jaEnaU7zLGUWL2vndUhPhEbVMri6r6X1yeFY7StuJSkvtfNEre2O5f40/NUUyVbVdcHLNTshiCQqZoMY0fC/CVkI1ciYgzTZjAxFlA9qDhrWKSjCtZPyH1D2wpON2Q22fQndMf08kVBozlIHtlBT7Zr42gv/VGjF2T1oJV1GMoNhkUTcWLobuKBC3wzUwFENrKNPc3uqyPtWUoY1tZktP06jP2WM6S0GN75qBgUyzNkB/Pq6/pl4p+17Zv64UTyvTKDNkj+yTQ+KTY3JKLkiV1AgjT+SZvJBX5835cD6dr0nrgjOd2SEzcr5/AJysrqc=</latexit><latexit sha1_base64="U/UizcLQvVoX2qSMQh+/wo9LbpQ=">AAACPnicbZDPSgMxEMaz/q3Vaqsn8bLYCnopu73oUfTiQaGCrUJbSzadtqFJdklmxbIsPo1XfQVfwxfwJl49mv452OoHgY/fzDCTL4gEN+h5787C4tLyympmLbu+kdvcyhe26yaMNYMaC0Wo7wJqQHAFNeQo4C7SQGUg4DYYnI/qtw+gDQ/VDQ4jaEnaU7zLGUWL2vndUhPhEbVMri6r6X1yeFY7StuJSkvtfNEre2O5f40/NUUyVbVdcHLNTshiCQqZoMY0fC/CVkI1ciYgzTZjAxFlA9qDhrWKSjCtZPyH1D2wpON2Q22fQndMf08kVBozlIHtlBT7Zr42gv/VGjF2T1oJV1GMoNhkUTcWLobuKBC3wzUwFENrKNPc3uqyPtWUoY1tZktP06jP2WM6S0GN75qBgUyzNkB/Pq6/pl4p+17Zv64UTyvTKDNkj+yTQ+KTY3JKLkiV1AgjT+SZvJBX5835cD6dr0nrgjOd2SEzcr5/AJysrqc=</latexit><latexit sha1_base64="U/UizcLQvVoX2qSMQh+/wo9LbpQ=">AAACPnicbZDPSgMxEMaz/q3Vaqsn8bLYCnopu73oUfTiQaGCrUJbSzadtqFJdklmxbIsPo1XfQVfwxfwJl49mv452OoHgY/fzDCTL4gEN+h5787C4tLyympmLbu+kdvcyhe26yaMNYMaC0Wo7wJqQHAFNeQo4C7SQGUg4DYYnI/qtw+gDQ/VDQ4jaEnaU7zLGUWL2vndUhPhEbVMri6r6X1yeFY7StuJSkvtfNEre2O5f40/NUUyVbVdcHLNTshiCQqZoMY0fC/CVkI1ciYgzTZjAxFlA9qDhrWKSjCtZPyH1D2wpON2Q22fQndMf08kVBozlIHtlBT7Zr42gv/VGjF2T1oJV1GMoNhkUTcWLobuKBC3wzUwFENrKNPc3uqyPtWUoY1tZktP06jP2WM6S0GN75qBgUyzNkB/Pq6/pl4p+17Zv64UTyvTKDNkj+yTQ+KTY3JKLkiV1AgjT+SZvJBX5835cD6dr0nrgjOd2SEzcr5/AJysrqc=</latexit>

b n = (bµn, b⌃n)<latexit sha1_base64="rmMY0VPQvCYxTerVVNICu3+w3B0=">AAACa3icbZDdSsNAEIW38b/+Vb1TL4JVUJCS9EZvBEEQLxWtCk0pk+20Xbq7CbsbtYS8kU/jnegr+A5uYhGjHlg4fDPDzJ4w5kwbz3utOFPTM7Nz8wvVxaXlldXa2vqtjhJFsUUjHqn7EDRyJrFlmOF4HysEEXK8C0dnef3uAZVmkbwx4xg7AgaS9RkFY1G3dr6bBo+sh0MwaRCKINYsy7ryZP+bFlgkOT0sw2s2EJDzg91ure41vELuX+NPTJ1MdNldqywHvYgmAqWhHLRu+15sOikowyjHrBokGmOgIxhg21oJAnUnLT6cuXuW9Nx+pOyTxi3oz4kUhNZjEdpOAWaof9dy+F+tnZj+cSdlMk4MSvq1qJ9w10Runp7bYwqp4WNrgCpmb3XpEBRQYzMubRkoiIeMPmVlirK4qwRDkVVtgP7vuP6a22bD9xr+VbN+2pxEOU+2yA7ZJz45IqfkglySFqHkmbyQN/Je+XA2nE1n+6vVqUxmNkhJzt4nkXu+9A==</latexit><latexit sha1_base64="rmMY0VPQvCYxTerVVNICu3+w3B0=">AAACa3icbZDdSsNAEIW38b/+Vb1TL4JVUJCS9EZvBEEQLxWtCk0pk+20Xbq7CbsbtYS8kU/jnegr+A5uYhGjHlg4fDPDzJ4w5kwbz3utOFPTM7Nz8wvVxaXlldXa2vqtjhJFsUUjHqn7EDRyJrFlmOF4HysEEXK8C0dnef3uAZVmkbwx4xg7AgaS9RkFY1G3dr6bBo+sh0MwaRCKINYsy7ryZP+bFlgkOT0sw2s2EJDzg91ure41vELuX+NPTJ1MdNldqywHvYgmAqWhHLRu+15sOikowyjHrBokGmOgIxhg21oJAnUnLT6cuXuW9Nx+pOyTxi3oz4kUhNZjEdpOAWaof9dy+F+tnZj+cSdlMk4MSvq1qJ9w10Runp7bYwqp4WNrgCpmb3XpEBRQYzMubRkoiIeMPmVlirK4qwRDkVVtgP7vuP6a22bD9xr+VbN+2pxEOU+2yA7ZJz45IqfkglySFqHkmbyQN/Je+XA2nE1n+6vVqUxmNkhJzt4nkXu+9A==</latexit><latexit sha1_base64="rmMY0VPQvCYxTerVVNICu3+w3B0=">AAACa3icbZDdSsNAEIW38b/+Vb1TL4JVUJCS9EZvBEEQLxWtCk0pk+20Xbq7CbsbtYS8kU/jnegr+A5uYhGjHlg4fDPDzJ4w5kwbz3utOFPTM7Nz8wvVxaXlldXa2vqtjhJFsUUjHqn7EDRyJrFlmOF4HysEEXK8C0dnef3uAZVmkbwx4xg7AgaS9RkFY1G3dr6bBo+sh0MwaRCKINYsy7ryZP+bFlgkOT0sw2s2EJDzg91ure41vELuX+NPTJ1MdNldqywHvYgmAqWhHLRu+15sOikowyjHrBokGmOgIxhg21oJAnUnLT6cuXuW9Nx+pOyTxi3oz4kUhNZjEdpOAWaof9dy+F+tnZj+cSdlMk4MSvq1qJ9w10Runp7bYwqp4WNrgCpmb3XpEBRQYzMubRkoiIeMPmVlirK4qwRDkVVtgP7vuP6a22bD9xr+VbN+2pxEOU+2yA7ZJz45IqfkglySFqHkmbyQN/Je+XA2nE1n+6vVqUxmNkhJzt4nkXu+9A==</latexit><latexit sha1_base64="rmMY0VPQvCYxTerVVNICu3+w3B0=">AAACa3icbZDdSsNAEIW38b/+Vb1TL4JVUJCS9EZvBEEQLxWtCk0pk+20Xbq7CbsbtYS8kU/jnegr+A5uYhGjHlg4fDPDzJ4w5kwbz3utOFPTM7Nz8wvVxaXlldXa2vqtjhJFsUUjHqn7EDRyJrFlmOF4HysEEXK8C0dnef3uAZVmkbwx4xg7AgaS9RkFY1G3dr6bBo+sh0MwaRCKINYsy7ryZP+bFlgkOT0sw2s2EJDzg91ure41vELuX+NPTJ1MdNldqywHvYgmAqWhHLRu+15sOikowyjHrBokGmOgIxhg21oJAnUnLT6cuXuW9Nx+pOyTxi3oz4kUhNZjEdpOAWaof9dy+F+tnZj+cSdlMk4MSvq1qJ9w10Runp7bYwqp4WNrgCpmb3XpEBRQYzMubRkoiIeMPmVlirK4qwRDkVVtgP7vuP6a22bD9xr+VbN+2pxEOU+2yA7ZJz45IqfkglySFqHkmbyQN/Je+XA2nE1n+6vVqUxmNkhJzt4nkXu+9A==</latexit>

F<latexit sha1_base64="c3quR4YZHm3h9y6tXEoXw0DuW1g=">AAACJHicbVDLSsNAFJ3UV61WW126CRbBVUm60WVBEJct2Ae0oUymN+3QmUmYmYgl5Avc6i/4Ne7EhRu/xWmahWk9MHA4517umeNHjCrtON9WaWd3b/+gfFg5Oq6enNbqZ30VxpJAj4QslEMfK2BUQE9TzWAYScDcZzDwF3crf/AEUtFQPOplBB7HM0EDSrA2Uvd+Ums4TSeDvU3cnDRQjs6kblXH05DEHIQmDCs1cp1IewmWmhIGaWUcK4gwWeAZjAwVmIPykixpal8ZZWoHoTRPaDtT/24kmCu15L6Z5FjP1aa3Ev/zRrEObr2EiijWIMj6UBAzW4f26tv2lEogmi0NwURSk9Umcywx0aacwpWZxNGckue0qILIchVEn6cVU6C7Wdc26beartN0u61Gu5VXWUYX6BJdIxfdoDZ6QB3UQwQBekGv6M16tz6sT+trPVqy8p1zVID18wub8KSg</latexit><latexit sha1_base64="c3quR4YZHm3h9y6tXEoXw0DuW1g=">AAACJHicbVDLSsNAFJ3UV61WW126CRbBVUm60WVBEJct2Ae0oUymN+3QmUmYmYgl5Avc6i/4Ne7EhRu/xWmahWk9MHA4517umeNHjCrtON9WaWd3b/+gfFg5Oq6enNbqZ30VxpJAj4QslEMfK2BUQE9TzWAYScDcZzDwF3crf/AEUtFQPOplBB7HM0EDSrA2Uvd+Ums4TSeDvU3cnDRQjs6kblXH05DEHIQmDCs1cp1IewmWmhIGaWUcK4gwWeAZjAwVmIPykixpal8ZZWoHoTRPaDtT/24kmCu15L6Z5FjP1aa3Ev/zRrEObr2EiijWIMj6UBAzW4f26tv2lEogmi0NwURSk9Umcywx0aacwpWZxNGckue0qILIchVEn6cVU6C7Wdc26beartN0u61Gu5VXWUYX6BJdIxfdoDZ6QB3UQwQBekGv6M16tz6sT+trPVqy8p1zVID18wub8KSg</latexit><latexit sha1_base64="c3quR4YZHm3h9y6tXEoXw0DuW1g=">AAACJHicbVDLSsNAFJ3UV61WW126CRbBVUm60WVBEJct2Ae0oUymN+3QmUmYmYgl5Avc6i/4Ne7EhRu/xWmahWk9MHA4517umeNHjCrtON9WaWd3b/+gfFg5Oq6enNbqZ30VxpJAj4QslEMfK2BUQE9TzWAYScDcZzDwF3crf/AEUtFQPOplBB7HM0EDSrA2Uvd+Ums4TSeDvU3cnDRQjs6kblXH05DEHIQmDCs1cp1IewmWmhIGaWUcK4gwWeAZjAwVmIPykixpal8ZZWoHoTRPaDtT/24kmCu15L6Z5FjP1aa3Ev/zRrEObr2EiijWIMj6UBAzW4f26tv2lEogmi0NwURSk9Umcywx0aacwpWZxNGckue0qILIchVEn6cVU6C7Wdc26beartN0u61Gu5VXWUYX6BJdIxfdoDZ6QB3UQwQBekGv6M16tz6sT+trPVqy8p1zVID18wub8KSg</latexit><latexit sha1_base64="c3quR4YZHm3h9y6tXEoXw0DuW1g=">AAACJHicbVDLSsNAFJ3UV61WW126CRbBVUm60WVBEJct2Ae0oUymN+3QmUmYmYgl5Avc6i/4Ne7EhRu/xWmahWk9MHA4517umeNHjCrtON9WaWd3b/+gfFg5Oq6enNbqZ30VxpJAj4QslEMfK2BUQE9TzWAYScDcZzDwF3crf/AEUtFQPOplBB7HM0EDSrA2Uvd+Ums4TSeDvU3cnDRQjs6kblXH05DEHIQmDCs1cp1IewmWmhIGaWUcK4gwWeAZjAwVmIPykixpal8ZZWoHoTRPaDtT/24kmCu15L6Z5FjP1aa3Ev/zRrEObr2EiijWIMj6UBAzW4f26tv2lEogmi0NwURSk9Umcywx0aacwpWZxNGckue0qILIchVEn6cVU6C7Wdc26beartN0u61Gu5VXWUYX6BJdIxfdoDZ6QB3UQwQBekGv6M16tz6sT+trPVqy8p1zVID18wub8KSg</latexit>

F<latexit sha1_base64="c3quR4YZHm3h9y6tXEoXw0DuW1g=">AAACJHicbVDLSsNAFJ3UV61WW126CRbBVUm60WVBEJct2Ae0oUymN+3QmUmYmYgl5Avc6i/4Ne7EhRu/xWmahWk9MHA4517umeNHjCrtON9WaWd3b/+gfFg5Oq6enNbqZ30VxpJAj4QslEMfK2BUQE9TzWAYScDcZzDwF3crf/AEUtFQPOplBB7HM0EDSrA2Uvd+Ums4TSeDvU3cnDRQjs6kblXH05DEHIQmDCs1cp1IewmWmhIGaWUcK4gwWeAZjAwVmIPykixpal8ZZWoHoTRPaDtT/24kmCu15L6Z5FjP1aa3Ev/zRrEObr2EiijWIMj6UBAzW4f26tv2lEogmi0NwURSk9Umcywx0aacwpWZxNGckue0qILIchVEn6cVU6C7Wdc26beartN0u61Gu5VXWUYX6BJdIxfdoDZ6QB3UQwQBekGv6M16tz6sT+trPVqy8p1zVID18wub8KSg</latexit><latexit sha1_base64="c3quR4YZHm3h9y6tXEoXw0DuW1g=">AAACJHicbVDLSsNAFJ3UV61WW126CRbBVUm60WVBEJct2Ae0oUymN+3QmUmYmYgl5Avc6i/4Ne7EhRu/xWmahWk9MHA4517umeNHjCrtON9WaWd3b/+gfFg5Oq6enNbqZ30VxpJAj4QslEMfK2BUQE9TzWAYScDcZzDwF3crf/AEUtFQPOplBB7HM0EDSrA2Uvd+Ums4TSeDvU3cnDRQjs6kblXH05DEHIQmDCs1cp1IewmWmhIGaWUcK4gwWeAZjAwVmIPykixpal8ZZWoHoTRPaDtT/24kmCu15L6Z5FjP1aa3Ev/zRrEObr2EiijWIMj6UBAzW4f26tv2lEogmi0NwURSk9Umcywx0aacwpWZxNGckue0qILIchVEn6cVU6C7Wdc26beartN0u61Gu5VXWUYX6BJdIxfdoDZ6QB3UQwQBekGv6M16tz6sT+trPVqy8p1zVID18wub8KSg</latexit><latexit sha1_base64="c3quR4YZHm3h9y6tXEoXw0DuW1g=">AAACJHicbVDLSsNAFJ3UV61WW126CRbBVUm60WVBEJct2Ae0oUymN+3QmUmYmYgl5Avc6i/4Ne7EhRu/xWmahWk9MHA4517umeNHjCrtON9WaWd3b/+gfFg5Oq6enNbqZ30VxpJAj4QslEMfK2BUQE9TzWAYScDcZzDwF3crf/AEUtFQPOplBB7HM0EDSrA2Uvd+Ums4TSeDvU3cnDRQjs6kblXH05DEHIQmDCs1cp1IewmWmhIGaWUcK4gwWeAZjAwVmIPykixpal8ZZWoHoTRPaDtT/24kmCu15L6Z5FjP1aa3Ev/zRrEObr2EiijWIMj6UBAzW4f26tv2lEogmi0NwURSk9Umcywx0aacwpWZxNGckue0qILIchVEn6cVU6C7Wdc26beartN0u61Gu5VXWUYX6BJdIxfdoDZ6QB3UQwQBekGv6M16tz6sT+trPVqy8p1zVID18wub8KSg</latexit><latexit sha1_base64="c3quR4YZHm3h9y6tXEoXw0DuW1g=">AAACJHicbVDLSsNAFJ3UV61WW126CRbBVUm60WVBEJct2Ae0oUymN+3QmUmYmYgl5Avc6i/4Ne7EhRu/xWmahWk9MHA4517umeNHjCrtON9WaWd3b/+gfFg5Oq6enNbqZ30VxpJAj4QslEMfK2BUQE9TzWAYScDcZzDwF3crf/AEUtFQPOplBB7HM0EDSrA2Uvd+Ums4TSeDvU3cnDRQjs6kblXH05DEHIQmDCs1cp1IewmWmhIGaWUcK4gwWeAZjAwVmIPykixpal8ZZWoHoTRPaDtT/24kmCu15L6Z5FjP1aa3Ev/zRrEObr2EiijWIMj6UBAzW4f26tv2lEogmi0NwURSk9Umcywx0aacwpWZxNGckue0qILIchVEn6cVU6C7Wdc26beartN0u61Gu5VXWUYX6BJdIxfdoDZ6QB3UQwQBekGv6M16tz6sT+trPVqy8p1zVID18wub8KSg</latexit>

precision-weighted fusion<latexit sha1_base64="0R8zlBHJLV+S9zOdpYKeq8mzIa0=">AAACPnicbVDLSsNAFJ3UV61WW12Jm2AR3FiSbnRZcOOygn1AG8pketMMnUzCzEQtofg1bvUX/A1/wJ24dekkzcK0Hhg4nHMv585xI0alsqwPo7SxubW9U96t7O1XDw5r9aOeDGNBoEtCFoqBiyUwyqGrqGIwiATgwGXQd2c3qd9/ACFpyO/VPAInwFNOPUqw0tK4dqKnCU3ty0egU1/BxPRimXkNq2llMNeJnZMGytEZ143qaBKSOACuCMNSDm0rUk6ChaKEwaIyiiVEmMzwFIaachyAdJLsDwvzXCs6OhT6cWVm6t+NBAdSzgNXTwZY+XLVS8X/vGGsvGsnoTyKFXCyDPJiZqrQTAsxJ1Q3oNhcE0wE1beaxMcCE6VrK6RMBY58Sp4WRRV4dldBdINFRRdor9a1Tnqtpm017btWo93KqyyjU3SGLpCNrlAb3aIO6iKCntELekVvxrvxaXwZ38vRkpHvHKMCjJ9fFU2veQ==</latexit><latexit sha1_base64="0R8zlBHJLV+S9zOdpYKeq8mzIa0=">AAACPnicbVDLSsNAFJ3UV61WW12Jm2AR3FiSbnRZcOOygn1AG8pketMMnUzCzEQtofg1bvUX/A1/wJ24dekkzcK0Hhg4nHMv585xI0alsqwPo7SxubW9U96t7O1XDw5r9aOeDGNBoEtCFoqBiyUwyqGrqGIwiATgwGXQd2c3qd9/ACFpyO/VPAInwFNOPUqw0tK4dqKnCU3ty0egU1/BxPRimXkNq2llMNeJnZMGytEZ143qaBKSOACuCMNSDm0rUk6ChaKEwaIyiiVEmMzwFIaachyAdJLsDwvzXCs6OhT6cWVm6t+NBAdSzgNXTwZY+XLVS8X/vGGsvGsnoTyKFXCyDPJiZqrQTAsxJ1Q3oNhcE0wE1beaxMcCE6VrK6RMBY58Sp4WRRV4dldBdINFRRdor9a1Tnqtpm017btWo93KqyyjU3SGLpCNrlAb3aIO6iKCntELekVvxrvxaXwZ38vRkpHvHKMCjJ9fFU2veQ==</latexit><latexit sha1_base64="0R8zlBHJLV+S9zOdpYKeq8mzIa0=">AAACPnicbVDLSsNAFJ3UV61WW12Jm2AR3FiSbnRZcOOygn1AG8pketMMnUzCzEQtofg1bvUX/A1/wJ24dekkzcK0Hhg4nHMv585xI0alsqwPo7SxubW9U96t7O1XDw5r9aOeDGNBoEtCFoqBiyUwyqGrqGIwiATgwGXQd2c3qd9/ACFpyO/VPAInwFNOPUqw0tK4dqKnCU3ty0egU1/BxPRimXkNq2llMNeJnZMGytEZ143qaBKSOACuCMNSDm0rUk6ChaKEwaIyiiVEmMzwFIaachyAdJLsDwvzXCs6OhT6cWVm6t+NBAdSzgNXTwZY+XLVS8X/vGGsvGsnoTyKFXCyDPJiZqrQTAsxJ1Q3oNhcE0wE1beaxMcCE6VrK6RMBY58Sp4WRRV4dldBdINFRRdor9a1Tnqtpm017btWo93KqyyjU3SGLpCNrlAb3aIO6iKCntELekVvxrvxaXwZ38vRkpHvHKMCjJ9fFU2veQ==</latexit><latexit sha1_base64="0R8zlBHJLV+S9zOdpYKeq8mzIa0=">AAACPnicbVDLSsNAFJ3UV61WW12Jm2AR3FiSbnRZcOOygn1AG8pketMMnUzCzEQtofg1bvUX/A1/wJ24dekkzcK0Hhg4nHMv585xI0alsqwPo7SxubW9U96t7O1XDw5r9aOeDGNBoEtCFoqBiyUwyqGrqGIwiATgwGXQd2c3qd9/ACFpyO/VPAInwFNOPUqw0tK4dqKnCU3ty0egU1/BxPRimXkNq2llMNeJnZMGytEZ143qaBKSOACuCMNSDm0rUk6ChaKEwaIyiiVEmMzwFIaachyAdJLsDwvzXCs6OhT6cWVm6t+NBAdSzgNXTwZY+XLVS8X/vGGsvGsnoTyKFXCyDPJiZqrQTAsxJ1Q3oNhcE0wE1beaxMcCE6VrK6RMBY58Sp4WRRV4dldBdINFRRdor9a1Tnqtpm017btWo93KqyyjU3SGLpCNrlAb3aIO6iKCntELekVvxrvxaXwZ38vRkpHvHKMCjJ9fFU2veQ==</latexit>

⇠<latexit sha1_base64="ZMThSV6AJuqyqDZRp9cO5+n+/oc=">AAACKXicbVC9TsMwGLT5LYVCCyOLRYvEVCVdYKzEwlgk+iO1UeW4TmtqO5HtIKoo78AKr8DTsAErL4KbZiAtJ1k63X2fvvP5EWfaOM4X3Nre2d3bLx2UD48qxyfV2mlPh7EitEtCHqqBjzXlTNKuYYbTQaQoFj6nfX9+u/T7T1RpFsoHs4ioJ/BUsoARbKzUa4w0E41xte40nQxok7g5qYMcnXENVkaTkMSCSkM41nroOpHxEqwMI5ym5VGsaYTJHE/p0FKJBdVeksVN0aVVJigIlX3SoEz9u5FgofVC+HZSYDPT695S/M8bxia48RImo9hQSVaHgpgjE6Ll39GEKUoMX1iCiWI2KyIzrDAxtqHClanC0YyR57SoUpnlKoi+SMu2QHe9rk3SazVdp+net+rtVl5lCZyDC3AFXHAN2uAOdEAXEPAIXsAreIPv8AN+wu/V6BbMd85AAfDnFz7cpnk=</latexit><latexit sha1_base64="ZMThSV6AJuqyqDZRp9cO5+n+/oc=">AAACKXicbVC9TsMwGLT5LYVCCyOLRYvEVCVdYKzEwlgk+iO1UeW4TmtqO5HtIKoo78AKr8DTsAErL4KbZiAtJ1k63X2fvvP5EWfaOM4X3Nre2d3bLx2UD48qxyfV2mlPh7EitEtCHqqBjzXlTNKuYYbTQaQoFj6nfX9+u/T7T1RpFsoHs4ioJ/BUsoARbKzUa4w0E41xte40nQxok7g5qYMcnXENVkaTkMSCSkM41nroOpHxEqwMI5ym5VGsaYTJHE/p0FKJBdVeksVN0aVVJigIlX3SoEz9u5FgofVC+HZSYDPT695S/M8bxia48RImo9hQSVaHgpgjE6Ll39GEKUoMX1iCiWI2KyIzrDAxtqHClanC0YyR57SoUpnlKoi+SMu2QHe9rk3SazVdp+net+rtVl5lCZyDC3AFXHAN2uAOdEAXEPAIXsAreIPv8AN+wu/V6BbMd85AAfDnFz7cpnk=</latexit><latexit sha1_base64="ZMThSV6AJuqyqDZRp9cO5+n+/oc=">AAACKXicbVC9TsMwGLT5LYVCCyOLRYvEVCVdYKzEwlgk+iO1UeW4TmtqO5HtIKoo78AKr8DTsAErL4KbZiAtJ1k63X2fvvP5EWfaOM4X3Nre2d3bLx2UD48qxyfV2mlPh7EitEtCHqqBjzXlTNKuYYbTQaQoFj6nfX9+u/T7T1RpFsoHs4ioJ/BUsoARbKzUa4w0E41xte40nQxok7g5qYMcnXENVkaTkMSCSkM41nroOpHxEqwMI5ym5VGsaYTJHE/p0FKJBdVeksVN0aVVJigIlX3SoEz9u5FgofVC+HZSYDPT695S/M8bxia48RImo9hQSVaHgpgjE6Ll39GEKUoMX1iCiWI2KyIzrDAxtqHClanC0YyR57SoUpnlKoi+SMu2QHe9rk3SazVdp+net+rtVl5lCZyDC3AFXHAN2uAOdEAXEPAIXsAreIPv8AN+wu/V6BbMd85AAfDnFz7cpnk=</latexit><latexit sha1_base64="ZMThSV6AJuqyqDZRp9cO5+n+/oc=">AAACKXicbVC9TsMwGLT5LYVCCyOLRYvEVCVdYKzEwlgk+iO1UeW4TmtqO5HtIKoo78AKr8DTsAErL4KbZiAtJ1k63X2fvvP5EWfaOM4X3Nre2d3bLx2UD48qxyfV2mlPh7EitEtCHqqBjzXlTNKuYYbTQaQoFj6nfX9+u/T7T1RpFsoHs4ioJ/BUsoARbKzUa4w0E41xte40nQxok7g5qYMcnXENVkaTkMSCSkM41nroOpHxEqwMI5ym5VGsaYTJHE/p0FKJBdVeksVN0aVVJigIlX3SoEz9u5FgofVC+HZSYDPT695S/M8bxia48RImo9hQSVaHgpgjE6Ll39GEKUoMX1iCiWI2KyIzrDAxtqHClanC0YyR57SoUpnlKoi+SMu2QHe9rk3SazVdp+net+rtVl5lCZyDC3AFXHAN2uAOdEAXEPAIXsAreIPv8AN+wu/V6BbMd85AAfDnFz7cpnk=</latexit> sampling<latexit sha1_base64="DrhDvjcQOJv2Gu3evUdNaaiXyLA=">AAACK3icbVC9TsMwGLTLXykUWhhZLCokpirpAmMlFsYi0R+pjSrHdVKrthPZDqKK8hKs8Ao8DROIlffATTOQlk+ydLr7Pt35/JgzbRznE1Z2dvf2D6qHtaPj+slpo3k20FGiCO2TiEdq5GNNOZO0b5jhdBQrioXP6dBf3K304RNVmkXy0Sxj6gkcShYwgo2lRhoL6yLDaaPltJ180DZwC9ACxfSmTVifzCKSCCoN4VjrsevExkuxMoxwmtUmiaYxJgsc0rGFEguqvTQPnKEry8xQECn7pEE5+/cixULrpfDtpsBmrje1FfmfNk5McOulTMaJoZKsjYKEIxOh1e/RjClKDF9agIliNisic6wwMbajkkuocDxn5Dkrs1TmuUqkL7KaLdDdrGsbDDpt12m7D51Wt1NUWQUX4BJcAxfcgC64Bz3QBwRw8AJewRt8hx/wC36vVyuwuDkHpYE/vxDzp/s=</latexit><latexit sha1_base64="DrhDvjcQOJv2Gu3evUdNaaiXyLA=">AAACK3icbVC9TsMwGLTLXykUWhhZLCokpirpAmMlFsYi0R+pjSrHdVKrthPZDqKK8hKs8Ao8DROIlffATTOQlk+ydLr7Pt35/JgzbRznE1Z2dvf2D6qHtaPj+slpo3k20FGiCO2TiEdq5GNNOZO0b5jhdBQrioXP6dBf3K304RNVmkXy0Sxj6gkcShYwgo2lRhoL6yLDaaPltJ180DZwC9ACxfSmTVifzCKSCCoN4VjrsevExkuxMoxwmtUmiaYxJgsc0rGFEguqvTQPnKEry8xQECn7pEE5+/cixULrpfDtpsBmrje1FfmfNk5McOulTMaJoZKsjYKEIxOh1e/RjClKDF9agIliNisic6wwMbajkkuocDxn5Dkrs1TmuUqkL7KaLdDdrGsbDDpt12m7D51Wt1NUWQUX4BJcAxfcgC64Bz3QBwRw8AJewRt8hx/wC36vVyuwuDkHpYE/vxDzp/s=</latexit><latexit sha1_base64="DrhDvjcQOJv2Gu3evUdNaaiXyLA=">AAACK3icbVC9TsMwGLTLXykUWhhZLCokpirpAmMlFsYi0R+pjSrHdVKrthPZDqKK8hKs8Ao8DROIlffATTOQlk+ydLr7Pt35/JgzbRznE1Z2dvf2D6qHtaPj+slpo3k20FGiCO2TiEdq5GNNOZO0b5jhdBQrioXP6dBf3K304RNVmkXy0Sxj6gkcShYwgo2lRhoL6yLDaaPltJ180DZwC9ACxfSmTVifzCKSCCoN4VjrsevExkuxMoxwmtUmiaYxJgsc0rGFEguqvTQPnKEry8xQECn7pEE5+/cixULrpfDtpsBmrje1FfmfNk5McOulTMaJoZKsjYKEIxOh1e/RjClKDF9agIliNisic6wwMbajkkuocDxn5Dkrs1TmuUqkL7KaLdDdrGsbDDpt12m7D51Wt1NUWQUX4BJcAxfcgC64Bz3QBwRw8AJewRt8hx/wC36vVyuwuDkHpYE/vxDzp/s=</latexit><latexit sha1_base64="DrhDvjcQOJv2Gu3evUdNaaiXyLA=">AAACK3icbVC9TsMwGLTLXykUWhhZLCokpirpAmMlFsYi0R+pjSrHdVKrthPZDqKK8hKs8Ao8DROIlffATTOQlk+ydLr7Pt35/JgzbRznE1Z2dvf2D6qHtaPj+slpo3k20FGiCO2TiEdq5GNNOZO0b5jhdBQrioXP6dBf3K304RNVmkXy0Sxj6gkcShYwgo2lRhoL6yLDaaPltJ180DZwC9ACxfSmTVifzCKSCCoN4VjrsevExkuxMoxwmtUmiaYxJgsc0rGFEguqvTQPnKEry8xQECn7pEE5+/cixULrpfDtpsBmrje1FfmfNk5McOulTMaJoZKsjYKEIxOh1e/RjClKDF9agIliNisic6wwMbajkkuocDxn5Dkrs1TmuUqkL7KaLdDdrGsbDDpt12m7D51Wt1NUWQUX4BJcAxfcgC64Bz3QBwRw8AJewRt8hx/wC36vVyuwuDkHpYE/vxDzp/s=</latexit>

multiplication<latexit sha1_base64="LJDwo0WavaX6dTU+7247sb/m3GY=">AAACMXicbVDLTgIxFL3jE1EUdOmmkZi4IjNsdEnixiUm8khgJJ3SgYa2M2k7RjKZ/3Crv+DXsDNu/QnLMAsBT9Lk9Nx7c05OEHOmjesunJ3dvf2Dw9JR+fikcnpWrZ13dZQoQjsk4pHqB1hTziTtGGY47ceKYhFw2gtm98t574UqzSL5ZOYx9QWeSBYygo2VnkXCDbM+q++oWncbbg60TbyC1KFAe1RzKsNxRBJBpSEcaz3w3Nj4KVaGEU6z8jDRNMZkhid0YKnEgmo/zWNn6NoqYxRGyj5pUK7+vUix0HouArspsJnqzdlS/G82SEx456dMxomhkqyMwoQjE6FlB2jMFCWGzy3BRDGbFZEpVpgY29Say0TheMrIa7auUpnnWhMDkZVtgd5mXduk22x4bsN7bNZbzaLKElzCFdyAB7fQggdoQwcIKHiDd/hwPp2F8+V8r1Z3nOLmAtbg/PwCjLyqyg==</latexit><latexit sha1_base64="LJDwo0WavaX6dTU+7247sb/m3GY=">AAACMXicbVDLTgIxFL3jE1EUdOmmkZi4IjNsdEnixiUm8khgJJ3SgYa2M2k7RjKZ/3Crv+DXsDNu/QnLMAsBT9Lk9Nx7c05OEHOmjesunJ3dvf2Dw9JR+fikcnpWrZ13dZQoQjsk4pHqB1hTziTtGGY47ceKYhFw2gtm98t574UqzSL5ZOYx9QWeSBYygo2VnkXCDbM+q++oWncbbg60TbyC1KFAe1RzKsNxRBJBpSEcaz3w3Nj4KVaGEU6z8jDRNMZkhid0YKnEgmo/zWNn6NoqYxRGyj5pUK7+vUix0HouArspsJnqzdlS/G82SEx456dMxomhkqyMwoQjE6FlB2jMFCWGzy3BRDGbFZEpVpgY29Say0TheMrIa7auUpnnWhMDkZVtgd5mXduk22x4bsN7bNZbzaLKElzCFdyAB7fQggdoQwcIKHiDd/hwPp2F8+V8r1Z3nOLmAtbg/PwCjLyqyg==</latexit><latexit sha1_base64="LJDwo0WavaX6dTU+7247sb/m3GY=">AAACMXicbVDLTgIxFL3jE1EUdOmmkZi4IjNsdEnixiUm8khgJJ3SgYa2M2k7RjKZ/3Crv+DXsDNu/QnLMAsBT9Lk9Nx7c05OEHOmjesunJ3dvf2Dw9JR+fikcnpWrZ13dZQoQjsk4pHqB1hTziTtGGY47ceKYhFw2gtm98t574UqzSL5ZOYx9QWeSBYygo2VnkXCDbM+q++oWncbbg60TbyC1KFAe1RzKsNxRBJBpSEcaz3w3Nj4KVaGEU6z8jDRNMZkhid0YKnEgmo/zWNn6NoqYxRGyj5pUK7+vUix0HouArspsJnqzdlS/G82SEx456dMxomhkqyMwoQjE6FlB2jMFCWGzy3BRDGbFZEpVpgY29Say0TheMrIa7auUpnnWhMDkZVtgd5mXduk22x4bsN7bNZbzaLKElzCFdyAB7fQggdoQwcIKHiDd/hwPp2F8+V8r1Z3nOLmAtbg/PwCjLyqyg==</latexit><latexit sha1_base64="LJDwo0WavaX6dTU+7247sb/m3GY=">AAACMXicbVDLTgIxFL3jE1EUdOmmkZi4IjNsdEnixiUm8khgJJ3SgYa2M2k7RjKZ/3Crv+DXsDNu/QnLMAsBT9Lk9Nx7c05OEHOmjesunJ3dvf2Dw9JR+fikcnpWrZ13dZQoQjsk4pHqB1hTziTtGGY47ceKYhFw2gtm98t574UqzSL5ZOYx9QWeSBYygo2VnkXCDbM+q++oWncbbg60TbyC1KFAe1RzKsNxRBJBpSEcaz3w3Nj4KVaGEU6z8jDRNMZkhid0YKnEgmo/zWNn6NoqYxRGyj5pUK7+vUix0HouArspsJnqzdlS/G82SEx456dMxomhkqyMwoQjE6FlB2jMFCWGzy3BRDGbFZEpVpgY29Say0TheMrIa7auUpnnWhMDkZVtgd5mXduk22x4bsN7bNZbzaLKElzCFdyAB7fQggdoQwcIKHiDd/hwPp2F8+V8r1Z3nOLmAtbg/PwCjLyqyg==</latexit>N<latexit sha1_base64="tOLXbGCbkZmkCpa4aX/cGWrFPpw=">AAACJnicbVC9TsMwGLTLXykUWhhZLFokpirpAmMlFiZUBP2R2qhyXCe16jiR7SCqKI/ACq/A07AhxMaj4KYZSMtJlk5336fvfG7EmdKW9Q1LW9s7u3vl/crBYfXouFY/6aswloT2SMhDOXSxopwJ2tNMczqMJMWBy+nAnd8s/cETlYqF4lEvIuoE2BfMYwRrIz0075qTWsNqWRnQJrFz0gA5upM6rI6nIYkDKjThWKmRbUXaSbDUjHCaVsaxohEmc+zTkaECB1Q5SZY1RRdGmSIvlOYJjTL170aCA6UWgWsmA6xnat1biv95o1h7107CRBRrKsjqkBdzpEO0/DiaMkmJ5gtDMJHMZEVkhiUm2tRTuOJLHM0YeU6LKhVZroLoBmnFFGiv17VJ+u2WbbXs+3aj086rLIMzcA4ugQ2uQAfcgi7oAQJ88AJewRt8hx/wE36tRksw3zkFBcCfX3LXpQQ=</latexit><latexit sha1_base64="tOLXbGCbkZmkCpa4aX/cGWrFPpw=">AAACJnicbVC9TsMwGLTLXykUWhhZLFokpirpAmMlFiZUBP2R2qhyXCe16jiR7SCqKI/ACq/A07AhxMaj4KYZSMtJlk5336fvfG7EmdKW9Q1LW9s7u3vl/crBYfXouFY/6aswloT2SMhDOXSxopwJ2tNMczqMJMWBy+nAnd8s/cETlYqF4lEvIuoE2BfMYwRrIz0075qTWsNqWRnQJrFz0gA5upM6rI6nIYkDKjThWKmRbUXaSbDUjHCaVsaxohEmc+zTkaECB1Q5SZY1RRdGmSIvlOYJjTL170aCA6UWgWsmA6xnat1biv95o1h7107CRBRrKsjqkBdzpEO0/DiaMkmJ5gtDMJHMZEVkhiUm2tRTuOJLHM0YeU6LKhVZroLoBmnFFGiv17VJ+u2WbbXs+3aj086rLIMzcA4ugQ2uQAfcgi7oAQJ88AJewRt8hx/wE36tRksw3zkFBcCfX3LXpQQ=</latexit><latexit sha1_base64="tOLXbGCbkZmkCpa4aX/cGWrFPpw=">AAACJnicbVC9TsMwGLTLXykUWhhZLFokpirpAmMlFiZUBP2R2qhyXCe16jiR7SCqKI/ACq/A07AhxMaj4KYZSMtJlk5336fvfG7EmdKW9Q1LW9s7u3vl/crBYfXouFY/6aswloT2SMhDOXSxopwJ2tNMczqMJMWBy+nAnd8s/cETlYqF4lEvIuoE2BfMYwRrIz0075qTWsNqWRnQJrFz0gA5upM6rI6nIYkDKjThWKmRbUXaSbDUjHCaVsaxohEmc+zTkaECB1Q5SZY1RRdGmSIvlOYJjTL170aCA6UWgWsmA6xnat1biv95o1h7107CRBRrKsjqkBdzpEO0/DiaMkmJ5gtDMJHMZEVkhiUm2tRTuOJLHM0YeU6LKhVZroLoBmnFFGiv17VJ+u2WbbXs+3aj086rLIMzcA4ugQ2uQAfcgi7oAQJ88AJewRt8hx/wE36tRksw3zkFBcCfX3LXpQQ=</latexit><latexit sha1_base64="tOLXbGCbkZmkCpa4aX/cGWrFPpw=">AAACJnicbVC9TsMwGLTLXykUWhhZLFokpirpAmMlFiZUBP2R2qhyXCe16jiR7SCqKI/ACq/A07AhxMaj4KYZSMtJlk5336fvfG7EmdKW9Q1LW9s7u3vl/crBYfXouFY/6aswloT2SMhDOXSxopwJ2tNMczqMJMWBy+nAnd8s/cETlYqF4lEvIuoE2BfMYwRrIz0075qTWsNqWRnQJrFz0gA5upM6rI6nIYkDKjThWKmRbUXaSbDUjHCaVsaxohEmc+zTkaECB1Q5SZY1RRdGmSIvlOYJjTL170aCA6UWgWsmA6xnat1biv95o1h7107CRBRrKsjqkBdzpEO0/DiaMkmJ5gtDMJHMZEVkhiUm2tRTuOJLHM0YeU6LKhVZroLoBmnFFGiv17VJ+u2WbbXs+3aj086rLIMzcA4ugQ2uQAfcgi7oAQJ88AJewRt8hx/wE36tRksw3zkFBcCfX3LXpQQ=</latexit>

n<latexit sha1_base64="XaolyvL8VAtgkC0BkS4EduTFz6s=">AAACJnicbVC9TsMwGLT5LYVCCyOLRYvEVCVdYKzEwlgE/ZHaqHJcJ7VqO5HtIKooj8AKr8DTsCHExqPgph1Iy0mWTnffp+98fsyZNo7zDbe2d3b39ksH5cOjyvFJtXba01GiCO2SiEdq4GNNOZO0a5jhdBArioXPad+f3S78/hNVmkXy0cxj6gkcShYwgo2VHhqyMa7WnaaTA20Sd0XqYIXOuAYro0lEEkGlIRxrPXSd2HgpVoYRTrPyKNE0xmSGQzq0VGJBtZfmWTN0aZUJCiJlnzQoV/9upFhoPRe+nRTYTPW6txD/84aJCW68lMk4MVSS5aEg4chEaPFxNGGKEsPnlmCimM2KyBQrTIytp3AlVDieMvKcFVUq81wF0RdZ2Rborte1SXqtpus03ftWvd1aVVkC5+ACXAEXXIM2uAMd0AUEhOAFvII3+A4/4Cf8Wo5uwdXOGSgA/vwCqvelJA==</latexit><latexit sha1_base64="XaolyvL8VAtgkC0BkS4EduTFz6s=">AAACJnicbVC9TsMwGLT5LYVCCyOLRYvEVCVdYKzEwlgE/ZHaqHJcJ7VqO5HtIKooj8AKr8DTsCHExqPgph1Iy0mWTnffp+98fsyZNo7zDbe2d3b39ksH5cOjyvFJtXba01GiCO2SiEdq4GNNOZO0a5jhdBArioXPad+f3S78/hNVmkXy0cxj6gkcShYwgo2VHhqyMa7WnaaTA20Sd0XqYIXOuAYro0lEEkGlIRxrPXSd2HgpVoYRTrPyKNE0xmSGQzq0VGJBtZfmWTN0aZUJCiJlnzQoV/9upFhoPRe+nRTYTPW6txD/84aJCW68lMk4MVSS5aEg4chEaPFxNGGKEsPnlmCimM2KyBQrTIytp3AlVDieMvKcFVUq81wF0RdZ2Rborte1SXqtpus03ftWvd1aVVkC5+ACXAEXXIM2uAMd0AUEhOAFvII3+A4/4Cf8Wo5uwdXOGSgA/vwCqvelJA==</latexit><latexit sha1_base64="XaolyvL8VAtgkC0BkS4EduTFz6s=">AAACJnicbVC9TsMwGLT5LYVCCyOLRYvEVCVdYKzEwlgE/ZHaqHJcJ7VqO5HtIKooj8AKr8DTsCHExqPgph1Iy0mWTnffp+98fsyZNo7zDbe2d3b39ksH5cOjyvFJtXba01GiCO2SiEdq4GNNOZO0a5jhdBArioXPad+f3S78/hNVmkXy0cxj6gkcShYwgo2VHhqyMa7WnaaTA20Sd0XqYIXOuAYro0lEEkGlIRxrPXSd2HgpVoYRTrPyKNE0xmSGQzq0VGJBtZfmWTN0aZUJCiJlnzQoV/9upFhoPRe+nRTYTPW6txD/84aJCW68lMk4MVSS5aEg4chEaPFxNGGKEsPnlmCimM2KyBQrTIytp3AlVDieMvKcFVUq81wF0RdZ2Rborte1SXqtpus03ftWvd1aVVkC5+ACXAEXXIM2uAMd0AUEhOAFvII3+A4/4Cf8Wo5uwdXOGSgA/vwCqvelJA==</latexit><latexit sha1_base64="XaolyvL8VAtgkC0BkS4EduTFz6s=">AAACJnicbVC9TsMwGLT5LYVCCyOLRYvEVCVdYKzEwlgE/ZHaqHJcJ7VqO5HtIKooj8AKr8DTsCHExqPgph1Iy0mWTnffp+98fsyZNo7zDbe2d3b39ksH5cOjyvFJtXba01GiCO2SiEdq4GNNOZO0a5jhdBArioXPad+f3S78/hNVmkXy0cxj6gkcShYwgo2VHhqyMa7WnaaTA20Sd0XqYIXOuAYro0lEEkGlIRxrPXSd2HgpVoYRTrPyKNE0xmSGQzq0VGJBtZfmWTN0aZUJCiJlnzQoV/9upFhoPRe+nRTYTPW6txD/84aJCW68lMk4MVSS5aEg4chEaPFxNGGKEsPnlmCimM2KyBQrTIytp3AlVDieMvKcFVUq81wF0RdZ2Rborte1SXqtpus03ftWvd1aVVkC5+ACXAEXXIM2uAMd0AUEhOAFvII3+A4/4Cf8Wo5uwdXOGSgA/vwCqvelJA==</latexit>

1<latexit sha1_base64="sOKZWwYLP0D8eYA/vKWe/syd2xo=">AAACJnicbVC9TsMwGLT5LYVCCyOLRYvEVCVdYKzEwlgE/ZHaqHJcJ7VqO5HtIKooj8AKr8DTsCHExqPgph1Iy0mWTnffp+98fsyZNo7zDbe2d3b39ksH5cOjyvFJtXba01GiCO2SiEdq4GNNOZO0a5jhdBArioXPad+f3S78/hNVmkXy0cxj6gkcShYwgo2VHhpuY1ytO00nB9ok7orUwQqdcQ1WRpOIJIJKQzjWeug6sfFSrAwjnGblUaJpjMkMh3RoqcSCai/Ns2bo0ioTFETKPmlQrv7dSLHQei58Oymwmep1byH+5w0TE9x4KZNxYqgky0NBwpGJ0OLjaMIUJYbPLcFEMZsVkSlWmBhbT+FKqHA8ZeQ5K6pU5rkKoi+ysi3QXa9rk/RaTddpuveteru1qrIEzsEFuAIuuAZtcAc6oAsICMELeAVv8B1+wE/4tRzdgqudM1AA/PkFP/qk5w==</latexit><latexit sha1_base64="sOKZWwYLP0D8eYA/vKWe/syd2xo=">AAACJnicbVC9TsMwGLT5LYVCCyOLRYvEVCVdYKzEwlgE/ZHaqHJcJ7VqO5HtIKooj8AKr8DTsCHExqPgph1Iy0mWTnffp+98fsyZNo7zDbe2d3b39ksH5cOjyvFJtXba01GiCO2SiEdq4GNNOZO0a5jhdBArioXPad+f3S78/hNVmkXy0cxj6gkcShYwgo2VHhpuY1ytO00nB9ok7orUwQqdcQ1WRpOIJIJKQzjWeug6sfFSrAwjnGblUaJpjMkMh3RoqcSCai/Ns2bo0ioTFETKPmlQrv7dSLHQei58Oymwmep1byH+5w0TE9x4KZNxYqgky0NBwpGJ0OLjaMIUJYbPLcFEMZsVkSlWmBhbT+FKqHA8ZeQ5K6pU5rkKoi+ysi3QXa9rk/RaTddpuveteru1qrIEzsEFuAIuuAZtcAc6oAsICMELeAVv8B1+wE/4tRzdgqudM1AA/PkFP/qk5w==</latexit><latexit sha1_base64="sOKZWwYLP0D8eYA/vKWe/syd2xo=">AAACJnicbVC9TsMwGLT5LYVCCyOLRYvEVCVdYKzEwlgE/ZHaqHJcJ7VqO5HtIKooj8AKr8DTsCHExqPgph1Iy0mWTnffp+98fsyZNo7zDbe2d3b39ksH5cOjyvFJtXba01GiCO2SiEdq4GNNOZO0a5jhdBArioXPad+f3S78/hNVmkXy0cxj6gkcShYwgo2VHhpuY1ytO00nB9ok7orUwQqdcQ1WRpOIJIJKQzjWeug6sfFSrAwjnGblUaJpjMkMh3RoqcSCai/Ns2bo0ioTFETKPmlQrv7dSLHQei58Oymwmep1byH+5w0TE9x4KZNxYqgky0NBwpGJ0OLjaMIUJYbPLcFEMZsVkSlWmBhbT+FKqHA8ZeQ5K6pU5rkKoi+ysi3QXa9rk/RaTddpuveteru1qrIEzsEFuAIuuAZtcAc6oAsICMELeAVv8B1+wE/4tRzdgqudM1AA/PkFP/qk5w==</latexit><latexit sha1_base64="sOKZWwYLP0D8eYA/vKWe/syd2xo=">AAACJnicbVC9TsMwGLT5LYVCCyOLRYvEVCVdYKzEwlgE/ZHaqHJcJ7VqO5HtIKooj8AKr8DTsCHExqPgph1Iy0mWTnffp+98fsyZNo7zDbe2d3b39ksH5cOjyvFJtXba01GiCO2SiEdq4GNNOZO0a5jhdBArioXPad+f3S78/hNVmkXy0cxj6gkcShYwgo2VHhpuY1ytO00nB9ok7orUwQqdcQ1WRpOIJIJKQzjWeug6sfFSrAwjnGblUaJpjMkMh3RoqcSCai/Ns2bo0ioTFETKPmlQrv7dSLHQei58Oymwmep1byH+5w0TE9x4KZNxYqgky0NBwpGJ0OLjaMIUJYbPLcFEMZsVkSlWmBhbT+FKqHA8ZeQ5K6pU5rkKoi+ysi3QXa9rk/RaTddpuveteru1qrIEzsEFuAIuuAZtcAc6oAsICMELeAVv8B1+wE/4tRzdgqudM1AA/PkFP/qk5w==</latexit>

bottom-up inference<latexit sha1_base64="VqVw2dSnengxfF4U+onKgCOAyOQ=">AAACOHicbVC7SgNBFJ31GeMrmtJmMAg2ht00WgZsLBVMIsQQZid3k8F5LDN3xSXst9jqL/gndnZi6xc4iSlM4oGBwzn3cs+cOJXCYRi+Byura+sbm6Wt8vbO7t5+5eCw7UxmObS4kcbexcyBFBpaKFDCXWqBqVhCJ364nPidR7BOGH2LeQo9xYZaJIIz9FK/Uo0NolFnWUqFTsCC5tCv1MJ6OAVdJtGM1MgM1/2DYPd+YHimQCOXzLluFKbYGzOLgksoyveZg5TxBzaErqeaKXC98TR9QU+8MqCJsf5ppFP178aYKedyFftJxXDkFr2J+J/XzTC56I2FTjP03/o9lGSSoqGTKuhAWOAoc08Yt8JnpXzELOPoC5u7MrQsHQn+VMyroKe55sRYFWVfYLRY1zJpN+pRWI9uGrVmY1ZliRyRY3JKInJOmuSKXJMW4SQnz+SFvAZvwUfwGXz9jq4Es50qmUPw/QORMay1</latexit><latexit sha1_base64="VqVw2dSnengxfF4U+onKgCOAyOQ=">AAACOHicbVC7SgNBFJ31GeMrmtJmMAg2ht00WgZsLBVMIsQQZid3k8F5LDN3xSXst9jqL/gndnZi6xc4iSlM4oGBwzn3cs+cOJXCYRi+Byura+sbm6Wt8vbO7t5+5eCw7UxmObS4kcbexcyBFBpaKFDCXWqBqVhCJ364nPidR7BOGH2LeQo9xYZaJIIz9FK/Uo0NolFnWUqFTsCC5tCv1MJ6OAVdJtGM1MgM1/2DYPd+YHimQCOXzLluFKbYGzOLgksoyveZg5TxBzaErqeaKXC98TR9QU+8MqCJsf5ppFP178aYKedyFftJxXDkFr2J+J/XzTC56I2FTjP03/o9lGSSoqGTKuhAWOAoc08Yt8JnpXzELOPoC5u7MrQsHQn+VMyroKe55sRYFWVfYLRY1zJpN+pRWI9uGrVmY1ZliRyRY3JKInJOmuSKXJMW4SQnz+SFvAZvwUfwGXz9jq4Es50qmUPw/QORMay1</latexit><latexit sha1_base64="VqVw2dSnengxfF4U+onKgCOAyOQ=">AAACOHicbVC7SgNBFJ31GeMrmtJmMAg2ht00WgZsLBVMIsQQZid3k8F5LDN3xSXst9jqL/gndnZi6xc4iSlM4oGBwzn3cs+cOJXCYRi+Byura+sbm6Wt8vbO7t5+5eCw7UxmObS4kcbexcyBFBpaKFDCXWqBqVhCJ364nPidR7BOGH2LeQo9xYZaJIIz9FK/Uo0NolFnWUqFTsCC5tCv1MJ6OAVdJtGM1MgM1/2DYPd+YHimQCOXzLluFKbYGzOLgksoyveZg5TxBzaErqeaKXC98TR9QU+8MqCJsf5ppFP178aYKedyFftJxXDkFr2J+J/XzTC56I2FTjP03/o9lGSSoqGTKuhAWOAoc08Yt8JnpXzELOPoC5u7MrQsHQn+VMyroKe55sRYFWVfYLRY1zJpN+pRWI9uGrVmY1ZliRyRY3JKInJOmuSKXJMW4SQnz+SFvAZvwUfwGXz9jq4Es50qmUPw/QORMay1</latexit><latexit sha1_base64="VqVw2dSnengxfF4U+onKgCOAyOQ=">AAACOHicbVC7SgNBFJ31GeMrmtJmMAg2ht00WgZsLBVMIsQQZid3k8F5LDN3xSXst9jqL/gndnZi6xc4iSlM4oGBwzn3cs+cOJXCYRi+Byura+sbm6Wt8vbO7t5+5eCw7UxmObS4kcbexcyBFBpaKFDCXWqBqVhCJ364nPidR7BOGH2LeQo9xYZaJIIz9FK/Uo0NolFnWUqFTsCC5tCv1MJ6OAVdJtGM1MgM1/2DYPd+YHimQCOXzLluFKbYGzOLgksoyveZg5TxBzaErqeaKXC98TR9QU+8MqCJsf5ppFP178aYKedyFftJxXDkFr2J+J/XzTC56I2FTjP03/o9lGSSoqGTKuhAWOAoc08Yt8JnpXzELOPoC5u7MrQsHQn+VMyroKe55sRYFWVfYLRY1zJpN+pRWI9uGrVmY1ZliRyRY3JKInJOmuSKXJMW4SQnz+SFvAZvwUfwGXz9jq4Es50qmUPw/QORMay1</latexit>

top-down inference<latexit sha1_base64="lbZ0CP+3ZgKNjsmr8vh4ery5VQ4=">AAACN3icbVDLTgIxFO3gC1EUcOmmkZi4kcyw0SWJG5eYyCMBQjqdO9DQaSdtRyFkfsWt/oKf4sqdcesfWGAWAp6kyck59+aeHj/mTBvX/XByO7t7+wf5w8LRcfHktFSutLVMFIUWlVyqrk80cCagZZjh0I0VkMjn0PEndwu/8wRKMykezSyGQURGgoWMEmOlYaliZHwdyGeBmQhBgaAwLFXdmrsE3iZeRqooQ3NYdor9QNIkAmEoJ1r3PDc2gzlRhlEOaaGfaIgJnZAR9CwVJAI9mC/Dp/jSKgEOpbJPGLxU/27MSaT1LPLtZETMWG96C/E/r5eY8HYwZyJOjP3W6lCYcGwkXjSBA6aAGj6zhFDFbFZMx0QRamxfa1dGisRjRqfpugpimWtN9KO0YAv0NuvaJu16zXNr3kO92qhnVebRObpAV8hDN6iB7lETtRBFU/SCXtGb8+58Ol/O92o052Q7Z2gNzs8voW+sPA==</latexit><latexit sha1_base64="lbZ0CP+3ZgKNjsmr8vh4ery5VQ4=">AAACN3icbVDLTgIxFO3gC1EUcOmmkZi4kcyw0SWJG5eYyCMBQjqdO9DQaSdtRyFkfsWt/oKf4sqdcesfWGAWAp6kyck59+aeHj/mTBvX/XByO7t7+wf5w8LRcfHktFSutLVMFIUWlVyqrk80cCagZZjh0I0VkMjn0PEndwu/8wRKMykezSyGQURGgoWMEmOlYaliZHwdyGeBmQhBgaAwLFXdmrsE3iZeRqooQ3NYdor9QNIkAmEoJ1r3PDc2gzlRhlEOaaGfaIgJnZAR9CwVJAI9mC/Dp/jSKgEOpbJPGLxU/27MSaT1LPLtZETMWG96C/E/r5eY8HYwZyJOjP3W6lCYcGwkXjSBA6aAGj6zhFDFbFZMx0QRamxfa1dGisRjRqfpugpimWtN9KO0YAv0NuvaJu16zXNr3kO92qhnVebRObpAV8hDN6iB7lETtRBFU/SCXtGb8+58Ol/O92o052Q7Z2gNzs8voW+sPA==</latexit><latexit sha1_base64="lbZ0CP+3ZgKNjsmr8vh4ery5VQ4=">AAACN3icbVDLTgIxFO3gC1EUcOmmkZi4kcyw0SWJG5eYyCMBQjqdO9DQaSdtRyFkfsWt/oKf4sqdcesfWGAWAp6kyck59+aeHj/mTBvX/XByO7t7+wf5w8LRcfHktFSutLVMFIUWlVyqrk80cCagZZjh0I0VkMjn0PEndwu/8wRKMykezSyGQURGgoWMEmOlYaliZHwdyGeBmQhBgaAwLFXdmrsE3iZeRqooQ3NYdor9QNIkAmEoJ1r3PDc2gzlRhlEOaaGfaIgJnZAR9CwVJAI9mC/Dp/jSKgEOpbJPGLxU/27MSaT1LPLtZETMWG96C/E/r5eY8HYwZyJOjP3W6lCYcGwkXjSBA6aAGj6zhFDFbFZMx0QRamxfa1dGisRjRqfpugpimWtN9KO0YAv0NuvaJu16zXNr3kO92qhnVebRObpAV8hDN6iB7lETtRBFU/SCXtGb8+58Ol/O92o052Q7Z2gNzs8voW+sPA==</latexit><latexit sha1_base64="lbZ0CP+3ZgKNjsmr8vh4ery5VQ4=">AAACN3icbVDLTgIxFO3gC1EUcOmmkZi4kcyw0SWJG5eYyCMBQjqdO9DQaSdtRyFkfsWt/oKf4sqdcesfWGAWAp6kyck59+aeHj/mTBvX/XByO7t7+wf5w8LRcfHktFSutLVMFIUWlVyqrk80cCagZZjh0I0VkMjn0PEndwu/8wRKMykezSyGQURGgoWMEmOlYaliZHwdyGeBmQhBgaAwLFXdmrsE3iZeRqooQ3NYdor9QNIkAmEoJ1r3PDc2gzlRhlEOaaGfaIgJnZAR9CwVJAI9mC/Dp/jSKgEOpbJPGLxU/27MSaT1LPLtZETMWG96C/E/r5eY8HYwZyJOjP3W6lCYcGwkXjSBA6aAGj6zhFDFbFZMx0QRamxfa1dGisRjRqfpugpimWtN9KO0YAv0NuvaJu16zXNr3kO92qhnVebRObpAV8hDN6iB7lETtRBFU/SCXtGb8+58Ol/O92o052Q7Z2gNzs8voW+sPA==</latexit>

Figure 2: Local Distributions. We illustrate the parametrization of a local variable zn in our struc-tured representation. The local prior (Eq. (6)) is defined in terms of a top-down process (in black)predicting the node’s parameters ψ̂n from a gate-modulated sample of zn’s parents zpa(n). The localapproximate posterior (Eq. (7)) additionally performs a precision-weighted fusion of these parame-ters with the result of a bottom-up process using a node-specific MLP to predict input-conditionedparameters ψ̃n from a generic encoding of x.

consider the case in which pθ(zn∣zpa(n),cpa(n),n) is given by a Gaussian density with parametersψ̂n = (µ̂n, Σ̂n). We obtain these parameters recursively by multiplying samples of node zn’sparent variables zpa(n) with their corresponding gating variables cpa(n),n and input a concatenation ofthe results of this operation into a multi-layer perceptron MLP(TD)n predicting ψ̂n (see AppendixB for additional details on the MLP architecture). The top-down recursion starts at the root nodezN ∼ p(zN) = N (0, I). An illustration of this process is shown in black in Fig. 2.

The approximate posterior, qφ(z∣x,c), must approximate pθ(z∣x,c). We express the approximateposterior as

qφ(z∣x,c) =N

∏n=1

qφ(zn∣x,zpa(n),cpa(n)). (7)

We note that the dependency structures of the generative model and its corresponding posteriorare, in general, not independent: choosing a particular structure in the generative model induces aparticular structure in the posterior (Webb et al., 2018). A simple way to guarantee enough capacityin the encoder to account for the dependencies implied by the decoder is thus to keep the encodergraph fully-connected and learn a decoder graph only. Instead, we share the gating variables cbetween approximate posterior and generative model (see Section 3.3), i.e., we assume that theencoder dependencies mirror those of the decoder. As a consequence, the posterior implied by thegenerative model could lie outside of the model class representable by the encoder. In practice, thisis not an issue and we observe significant performance improvements over both traditional VAEs(where prior and posterior match but are limited in their expressiveness) and Graph VAEs withfully-connected encoder graph. See Section 4.3 for quantitative experiments and Section 5 for adiscussion on the relationship between learned structures and fully-connected structures.

Parameter prediction for the local factors qφ(zn∣x,zpa(n),cpa(n)) consists of a precision-weightedfusion of the top-down prediction ψ̂n described above and a bottom-up prediction ψ̃n. The latter isobtained by encoding x into a generic feature that is used as an input to a node-specific multi-layerperceptron MLP(BU)n predicting ψ̃n. This is shown in blue in Fig. 2. Additional details on the fusionprocess can be found in Appendix B.3.

5


Algorithm 1 Optimizing VAEs with Latent Dependency StructureRequire: Data x, number of latent nodes N , number of dimensions per node N ′.

1: Initialize θ, φ, µ.2: repeat3: Sample c using Eq. (9) and determine zpa(n) for each zn based on the sampled structure.4: For each node, compute qφ(zn∣x,zpa(n)) using Eq. (7).5: Sample z from qφ(z∣x) using Eq. (6) and compute pθ(x∣z).6: Update θ, φ,µ based on the gradients derived from Eq. (8).7: until Convergence.

3.2 LEARNING STRUCTURE BY INDUCING AN ADDITIONAL LOWER BOUND

The formulation in Section 3.1 provides the form of the model for a particular configuration of thelatent dependency structure. Finding the optimal structure corresponds to a discrete optimizationover all values of c, potentially optimizing the model parameters of each possible configuration. Toavoid this intractable procedure, we place a distribution over c, then directly optimize the parametersof this distribution to arrive at a single, learned latent dependency structure. Specifically, we treateach ci,j as an independent random variable, sampled from a Bernoulli distribution with mean µi,j ,i.e. ci,j ∼ p(ci,j) = B(µi,j). We denote the set of these Bernoulli means as µ. Introducing thisdistribution allows us to express the following additional lower bound on L, derived in Appendix A:

L ≥ L̃ = Ep(c) [Eqφ(z∣x) [log pθ(x∣z,c)] −KL(qφ(z∣x)∣∣pθ(z∣c))] = Ep(c) [Lc] , (8)

where Lc is the ELBO for a particular value of dependency gating variables. Thus, L̃ can be in-terpreted as the expected ELBO under the distribution of dependency structures induced by p(c),which we estimate by sampling c ∼ p(c) and evaluating Lc. We note that L̃ is not a proper varia-tional bound, as it is not guaranteed to recover the marginal likelihood if the approximate posteriormatches the true posterior. Rather, optimizing L̃ provides a method for learning the latent structure.For fixed parameters (φ, θ), the optimal L̃ w.r.t. µ is a δ-distribution at the MAP configuration, c∗,yielding L = L̃ = Lc∗ . This is because Lc∗ is always greater than or equal to the expected ELBOover all dependency gates, L̃. In practice, we jointly optimize φ, θ, and µ. While this non-convexoptimization procedure may result in local optima, we find this empirically works well, with p(c)converging to a fixed distribution (Fig. 3). Thus, by the end of training, we are effectively optimizingthe ELBO for a single dependency structure. The training procedure is outlined in Algorithm 1.

3.3 LEARNING THE DISCRETE GATING VARIABLE DISTRIBUTIONS

For a given latent dependency structure, gradients for the parameters θ and φ can be estimated usingMonte Carlo samples and the reparameterization trick (Kingma & Welling, 2014; Rezende et al.,2014). To obtain gradients for the gate means, µ, we make use of recent advances in differentiatingthrough discrete operations (Maddison et al., 2017; Jang et al., 2017), allowing us to differenti-ate through the sampling of the dependency gating variables, c. Specifically, we recast the gatingvariables using the Gumbel-Softmax estimator from Jang et al. (2017), re-expressing ci,j as:

ci,j =exp((log(µi,j) + ε1)/τ)

exp((log(µi,j) + ε1)/τ) + exp((log(1 − µi,j) + ε2)/τ), (9)

where ε1 and ε2 are i.i.d samples drawn from a Gumbel(0, 1) distribution and τ is a temperatureparameter. The Gumbel-Softmax distribution is differentiable for τ > 0, allowing us to estimate thederivative ∂ci,j

∂µi,j. For large values of τ , we obtain smoothed versions of c, essentially interpolating

between different dependency structures. As τ → 0, we recover binary values for c, yielding thedesired discrete sampling of dependency structures at the cost of high-variance gradient estimates.Thus, we anneal τ during training to learn the dependency gate means, µ, eventually arriving at thediscrete setting.

6


(a) (b)

Figure 3: Structure Learning. In (a), we show trainingof the Bernoulli parameters µi,j governing the distributionover graph structures in architecture space. All edges arecolor-coded and can be located in (b), where we show arandom sample from the resulting steady-state distributionwith the same color scheme.

Figure 4: Ablation Study on MNIST.For a fixed latent dimension M (color-coded), we report the log-likelihood ofall possible factorizations of node di-mension N ′ (x-axis) and number ofnodes N =M/N ′.

4 EVALUATION

We evaluate the proposed latent dependency learning approach on three benchmark datasets:MNIST (Lecun et al., 1998; Larochelle & Murray, 2011), Omniglot (Lake et al., 2013), and CIFAR-10 (Krizhevsky, 2009). After discussing the experimental setup in Section 4.1, we provide a setof qualitative experiments in Section 4.2 to gain insight into the learning process, hyper-parameterselection, and the nature of the inferred structures. In Section 4.3, we provide quantitative com-parisons with common predefined latent dependency structures on benchmark datasets. Additionalresults on training robustness and latent space embeddings can be found in Appendix C.

4.1 EXPERIMENTAL SETUP

To provide a fair comparison, the encoders of all structured methods use the same MLP architecturewith batch normalization (Ioffe & Szegedy, 2015) and ReLU non-linearities (Nair & Hinton, 2010)in all experiments.2 Decoder structures are the reverse of the encoders. Likewise, the number oflatent dimensions is the same in all models and experiments (M = 80). As discussed in Section 3.1,all latent dependencies are modeled by non-linear MLPs as well.

For MNIST and Omniglot, we binarize the data and model pθ(x∣z) as a Bernoulli distribution, usinga sigmoid non-linearity on the output layer to produce the mean of this distribution. For CIFAR-10,we model pθ(x∣z) with a Gaussian density, with mean and log-variance predicted by sigmoid andlinear functions, respectively. Further implementation details, including the model architectures andtraining criteria, can be found in Appendix B.

4.2 QUALITATIVE ANALYSIS

We first explore the structure learning process. As described in Section 3.2 and Appendix A, op-timizing L̃ w.r.t. the dependency gating means, µ, should push this lower bound toward L. Thus,µ should converge to either 0 or 1, yielding a fixed, final latent dependency structure. In Fig. 3,we visualize this process during training on MNIST. The model has N = 5 nodes and a total latentdimension of M = 80 (i.e., N ′ =M/N = 16 dimensions per node). As shown in Fig. 3a, the gatingmeans converge in practice, with 3 out of 10 edges removed and the rest retained. The resultingstatic dependency structure is visualized in Fig. 3b. We observed that the learned structure is stableacross training runs with different seeds for parameter initialization and mini-batch sampling, sup-porting the hypothesis that the inferred structure indeed depends on the model parameterization andthe dataset.

2We implement classic VAEs using a more complex encoder to match the number of parameters of thestructured methods. All baselines use the same or more parameters than Graph VAE.

7


Dataset Method LL KL ELBO

MN

IST VAE −87.30 ± 0.04 26.28 ± 0.05 −95.61 ± 0.07

Ladder VAE −84.98 ± 0.10 24.25 ± 0.12 −92.80 ± 0.14FC-VAE −82.80 ± 0.15 22.47 ± 0.09 −88.49 ± 0.12Graph VAE −82.58 ± 0.08 22.49 ± 0.10 −88.35 ± 0.14

Om

nigl

ot VAE −109.62 ± 0.09 32.11 ± 0.07 −115.43 ± 0.12Ladder VAE −105.92 ± 0.16 29.46 ± 0.10 −110.62 ± 0.17FC-VAE −105.01 ± 0.26 30.74 ± 0.12 −108.79 ± 0.21Graph VAE −104.57 ± 0.19 29.93 ± 0.10 −108.17 ± 0.24

CIF

AR

-10 VAE −6.37 ±△ 0.10 ±△ −6.42 ±△

Ladder VAE −6.10 ± 0.02 0.08 ±△ −6.13 ± 0.02FC-VAE −5.89 ± 0.04 0.07 ±△ −5.90 ± 0.04Graph VAE −5.81 ± 0.04 0.07 ±△ −5.82 ± 0.04

Table 1: Quantitative Analysis. Test log-likelihood (LL), DKL(qφ(z∣x)∣∣pθ(z)) (KL), and ELBOfor the proposed Graph VAE model and the baseline models with predefined dependency structures.We show mean and standard deviation over 5 independent runs, where △ indicates a value < 0.01.

We next investigate the influence of the total latent dimension, M , and the trade-off between thenumber of nodes, N , and the node dimension, N ′ = M/N . Our results for various models trainedon MNIST are shown in Fig. 4. Models with the same total latent dimension are shown in thesame color. We observe that the performance improves with increasing total latent dimension, likelyresulting from the additional flexibility of the higher-dimensional latent space. We also observethat, for a fixed number of latent dimensions, models with fewer node dimensions (and thereforemore nodes with a more complex dependency structure) typically perform better. This highlightsthe importance of using an expressive dependency structure for obtaining a flexible model.

4.3 QUANTITATIVE COMPARISON

To quantitatively evaluate the improvements due to learning the latent dependency structure, wecompare with a range of common, predefined baseline structures. These baselines include classicVAEs (Kingma & Welling, 2014; Rezende et al., 2014), which contain no dependencies in the prior,Ladder VAEs (Sønderby et al., 2016), which contain chain-like dependencies in the prior, and fully-connected VAEs (FC-VAEs) (cf. Kingma et al. (2016)), which contain all possible dependencies inthe prior (corresponding to all gating variable parameters µi,j set to a fixed value of 1). We note thatour approach is orthogonal and could be complemented by a number of other approaches attemptingto overcome the limitations of classic VAEs. Similar to ladder VAEs, Zhao et al. (2017) use chain-structured latent dependencies to learn disentangled representations. Normalizing flows (Rezende& Mohamed, 2015), on the other hand, adds dependencies to the approximate posterior through aseries of invertible transformations.

We evaluate the performance of all models using their test log-likelihood, log pθ(x), in 5 indepen-dent runs (Table 1). All values were estimated using 5,000 importance-weighted samples. Follow-ing standard practice, we report log pθ(x) in nats on MNIST/Omniglot and in bits/input dimensionon CIFAR-10. The learned dependency structure in our proposed Graph VAE consistently outper-forms models with both fewer (VAE, ladder VAE) and more (FC-VAE) latent dependencies. Wediscuss potential reasons in Section 5. To provide further insight into the training objective, Table 1also reports DKL(qφ(z∣x)∣∣pθ(z)) and the ELBO for each model on the test set.

Encoder-Decoder Relationship. From a purely theoretical point of view, learning the structureof the generative model implies the need for a fully-connected graph in the approximate posterior(see Section 3.1). In practice, we share the gating variables c between encoder and decoder (seeEq. (8)), because we observed improved empirical performance when doing so: training a GraphVAE decoder with a predefined, fully-connected encoder graph results in a mean test log-likelihoodof −83.40 nats on MNIST, which is worse than the performance of FC-VAE (−82.80 nats) andGraph VAE (−82.58 nats). We believe these are noteworthy empirical results, but further research is

8


required to understand this behavior at a theoretical level. A full visualization of the training processis provided in Appendix C.1.

5 DISCUSSION

Performance vs. Speed/Memory. As shown in Fig. 4, the number of latent nodes can significantlyimpact the performance of our model. While allowing more complex dependency structures througha lowM/N -ratio is typically beneficial, it also has an adverse effect on the training time and memoryconsumption. Fortunately, the ability to freely select this ratio allows a simple adaption to theavailable processing power, hardware constraints, and application scenarios.

Optimization Order. It is worth noting that the learning process optimizes the model parameters(c, φ, θ) in a clear temporal order. While the latent structure, governed by c, converges during thefirst ≈ 200 epochs (Fig. 3a), it takes over 10× as long until the variational and generative parameters(φ, θ) converge. There is no external force enforcing this behaviour, indicating that the loss caninitially most easily be decreased by limiting the latent structure to the complexity prescribed by theobserved training data.

Performance Improvement over Fully-Connected VAEs. There is an intricate relationship be-tween fully-connected graphs vs. learned graphs along one axis and between prior structures vs.posterior structures along a separate axis: FC-VAEs model all conditional latent dependencies andare thus potentially more expressive and flexible than other latent dependency structures. It is there-fore somewhat surprising that the learned latent structures in Graph VAE consistently outperformthe FC-VAE baseline. We speculate that this may be due to difficulties in optimization, which isa known problem in hierarchical latent variable models (Bowman et al., 2016; Burda et al., 2016).Graph VAE and FC-VAE both contain the same hypothesis space of possible models that can belearned. If all dependencies are needed, Graph VAE could set all dependency gate parameters to 1.Likewise, if a latent dependency was unnecessary, FC-VAE could set all of the model parameters inthat dependency to 0. However, this would require many coordinated steps along multiple parameterdimensions. It is plausible that the benefit of learning the dependency structure may stem from theability to alter the optimization landscape, using the dependency gates to move through the modelparameter space more coarsely and rapidly. The resulting latent dependency structures may thus beless expressive, but easier to optimize. While only an intuition, this hypothesis is also in line withthe observations and results in our experiments with fully-connected encoder graphs and learneddecoder graphs, where the theoretically more flexible FC-encoder is outperformed by our parametersharing approach. Follow-up work will be required to test this intuition.

6 CONCLUSION

We presented a novel method for structure learning in latent variable models, which uses depen-dency gating variables, together with a modified objective and discrete differentiation techniques,to effectively transform discrete structure learning into a continuous optimization problem. In ourexperiments, the learned latent dependency structures improve the performance of latent variablemodels over comparable baselines with predefined dependency structures. The approach presentedhere provides directions for further research in structure learning for other tasks, including undi-rected graphical models, time-series models, and discriminative models.

9


REFERENCES

A. Barron, J. Rissanen, and B. Yu. The Minimum Description Length Principle in Coding andModeling. In IEEE Transactions on Information Theory, 1998.

Samuel R Bowman, Luke Vilnis, Oriol Vinyals, Andrew Dai, Rafal Jozefowicz, and Samy Bengio.Generating sentences from a continuous space. In Proceedings of The 20th SIGNLL Conferenceon Computational Natural Language Learning, pp. 10–21, 2016.

Yuri Burda, Roger Grosse, and Ruslan Salakhutdinov. Importance weighted autoencoders. In Inter-national Conference on Learning Representations, 2016.

Jie Cheng, Russell Greiner, Jonathan Kelly, David Bell, and Weiru Liu. Learning Bayesian Networksfrom Data: An Information-theoretic Approach. In Artificial Intelligence, 2002.

Junyoung Chung, Kyle Kastner, Laurent Dinh, Kratarth Goel, Aaron C Courville, and Yoshua Ben-gio. A recurrent latent variable model for sequential data. In Advances in neural informationprocessing systems, pp. 2980–2988, 2015.

Peter Dayan, Geoffrey E Hinton, Radford M Neal, and Richard S Zemel. The helmholtz machine.Neural computation, 7(5):889–904, 1995.

Marco Fraccaro, Søren Kaae Sønderby, Ulrich Paquet, and Ole Winther. Sequential neural modelswith stochastic layers. In Advances in neural information processing systems, pp. 2199–2207,2016.

Prasoon Goyal, Zhiting Hu, Xiaodan Liang, Chenyu Wang, Eric P Xing, and Carnegie Mellon.Nonparametric variational auto-encoders for hierarchical representation learning. In ICCV, pp.5104–5112, 2017.

Karol Gregor, Ivo Danihelka, Andriy Mnih, Charles Blundell, and Daan Wierstra. Deep autoregres-sive networks. In International Conference on Machine Learning, pp. 1242–1250, 2014.

Karol Gregor, Ivo Danihelka, Alex Graves, Danilo Rezende, and Daan Wierstra. Draw: A recurrentneural network for image generation. In International Conference on Machine Learning, pp.1462–1471, 2015.

Jiawei He, Andreas Lehrmann, Joseph Marino, Greg Mori, and Leonid Sigal. Probabilistic videogeneration using holistic attribute control. In European Conference on Computer Vision, 2018.

D. Heckerman, C. Meek, and G. Cooper. A Bayesian Approach to Causal Discovery. In Computa-tion, Causation, Discovery, 1999.

David Heckerman, Dan Geiger, and David M. Chickering. Learning Bayesian Networks: The Com-bination of Knowledge and Statistical Data. In Machine Learning, 1995.

Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training byreducing internal covariate shift. In International Conference on Machine Learning, pp. 448–456,2015.

Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparametrization with gumble-softmax. InInternational Conference on Learning Representations, 2017.

Matthew Johnson, David K Duvenaud, Alex Wiltschko, Ryan P Adams, and Sandeep R Datta. Com-posing graphical models with neural networks for structured representations and fast inference.In D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, and R. Garnett (eds.), Advances in NeuralInformation Processing Systems 29, pp. 2946–2954. Curran Associates, Inc., 2016.

Michael I Jordan, Zoubin Ghahramani, Tommi S Jaakkola, and Lawrence K Saul. An introductionto variational methods for graphical models. Machine learning, 37(2):183–233, 1999.

R. Kass and A. Raftery. Bayes Factors. In Journal of the American Statistical Association, 1995.

Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. InternationalConference on Learning Representations, 2015.

10


Diederik P Kingma and Max Welling. Auto-encoding variational bayes. In International Conferenceon Learning Representations, 2014.

Diederik P Kingma, Tim Salimans, Rafal Jozefowicz, Xi Chen, Ilya Sutskever, and Max Welling.Improved variational inference with inverse autoregressive flow. In Advances in Neural Informa-tion Processing Systems, pp. 4743–4751, 2016.

M. Koivisto and K. Sood. Exact Bayesian Structure Discovery in Bayesian Networks. In Journal ofMachine Learning Research, 2004.

Daphne Koller and Nir Friedman. Probabilistic Graphical Models. MIT Press, 2009.

Rahul G Krishnan, Dawen Liang, and Matthew Hoffman. On the challenges of learning with infer-ence networks on sparse, high-dimensional data. arXiv preprint arXiv:1710.06085, 2017.

Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, Citeseer,2009.

Brenden M Lake, Ruslan R Salakhutdinov, and Josh Tenenbaum. One-shot learning by invertinga compositional causal process. In C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, andK. Q. Weinberger (eds.), Advances in Neural Information Processing Systems 26, pp. 2526–2534.Curran Associates, Inc., 2013.

Hugo Larochelle and Iain Murray. The neural autoregressive distribution estimator. In Proceedingsof the Fourteenth International Conference on Artificial Intelligence and Statistics, pp. 29–37,2011.

Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to documentrecognition. Proceedings of the IEEE, 86(11):2278–2324, Nov 1998. ISSN 0018-9219. doi:10.1109/5.726791.

Erich L. Lehmann and Joseph P. Romano. Testing Statistical Hypotheses. Springer, 2008.

Andreas M. Lehrmann and Leonid Sigal. Non-parametric structured output networks. In Advancesin neural information processing systems, 2017.

Chris J Maddison, Andriy Mnih, and Yee Whye Teh. The concrete distribution: A continuousrelaxation of discrete random variables. International Conference on Learning Representations,2017.

Joseph Marino, Yisong Yue, and Stephan Mandt. Iterative amortized inference. In InternationalConference on Machine Learning, pp. 3400–3409, 2018.

Vinod Nair and Geoffrey E Hinton. Rectified linear units improve restricted boltzmann machines. InProceedings of the 27th international conference on machine learning (ICML-10), pp. 807–814,2010.

Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito,Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation inpytorch. In NIPS-W, 2017.

Danilo Rezende and Shakir Mohamed. Variational inference with normalizing flows. In Interna-tional Conference on Machine Learning, pp. 1530–1538, 2015.

Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. Stochastic backpropagation and ap-proximate inference in deep generative models. In International Conference on Machine Learn-ing, pp. 1278–1286, 2014.

Casper Kaae Sønderby, Tapani Raiko, Lars Maaløe, Søren Kaae Sønderby, and Ole Winther. Laddervariational autoencoders. In Advances in neural information processing systems, pp. 3738–3746,2016.

Sharad Vikram, Matthew D Hoffman, and Matthew J Johnson. The loracs prior for vaes: Letting thetrees speak for the data. arXiv preprint arXiv:1810.06891, 2018.

11


Stefan Webb, Adam Golinski, Robert Zinkov, N Siddharth, Tom Rainforth, Yee Whye Teh, andFrank Wood. Faithful inversion of generative models for effective amortized inference. In Ad-vances in Neural Information Processing Systems, 2018.

Li Yingzhen and Stephan Mandt. Disentangled sequential autoencoder. In International Conferenceon Machine Learning, pp. 5656–5665, 2018.

Shengjia Zhao, Jiaming Song, and Stefano Ermon. Learning hierarchical features from deep gener-ative models. In International Conference on Machine Learning, pp. 4091–4099, 2017.

Barret Zoph and Quoc V Le. Neural architecture search with reinforcement learning. arXiv preprintarXiv:1611.01578, 2016.

12


A LOWER BOUND DERIVATION

Introducing the distribution over dependency gating variables, p(c) = B(µ), modifies the evidencelower bound (ELBO) from Eq. (2), as we now have an additional set of random variables over whichto marginalize. To see this, we can start by re-expressing Eq. (2) as

L = Eqφ(z∣x) [log pθ(x,z)] −Eqφ(z∣x) [log qφ(z∣x)] . (10)

pθ(x,z) can be expressed as a marginalization over the gating variables, effectively averaging overan ensemble of models with a distribution of dependency structures:

pθ(x,z) = ∫ pθ(x,z,c)dc = ∫ pθ(x,z∣c)p(c)dc = Ep(c) [pθ(x,z∣c)] . (11)

Plugging this into Eq. (10):

L = Eqφ(z∣x) [logEp(c) [pθ(x,z∣c)]] −Eqφ(z∣x) [log qφ(z∣x)] . (12)

Using Jensen’s inequality, we bring the log inside of the expectation, Ep(c) [⋅], yielding

L ≥ L̃ = Eqφ(z∣x) [Ep(c) [log pθ(x,z∣c)]] −Eqφ(z∣x) [log qφ(z∣x)] , (13)

where L̃ is a lower bound on L. Swapping the order of expectation, we rewrite L̃ as

L̃ = Ep(c) [Eqφ(z∣x) [log pθ(x,z∣c)]] −Eqφ(z∣x) [log qφ(z∣x)] , (14)

and because the second term is independent of p(c), we can include both terms inside of a singleouter expectation:

L̃ = Ep(c) [Eqφ(z∣x) [log pθ(x,z∣c) − log qφ(z∣x)]]

= Ep(c) [Eqφ(z∣x) [log pθ(x∣z,c)] −KL(qφ(z∣x)∣∣pθ(z∣c))]

= Ep(c) [Lc] ,

(15)

where we have defined Lc as the ELBO for a given dependency structure. Note that L̃ is not aproper variational bound, as it cannot recover the marginal log likelihood. Rather, L̃ allows us tooptimize the distribution over gating variables and, thus, the model structure. When we arrive at afixed structure, we will be optimizing the variational bound for that particular dependency structure.

To see this, note that the bound in Eq. 13 becomes tight when p(c) is any fixed distribution, in whichthe dependency gating means, µ, are all either 1 or 0. Plugging in a delta distribution for p(c) at aparticular configuration, c′, i.e. p(c) = δc,c′ , we have

L = Eqφ(z∣x) [logEδc,c′ [pθ(x,z∣c)]] −Eqφ(z∣x) [log qφ(z∣x)]

= Eqφ(z∣x) [Eδc,c′ [log pθ(x,z∣c)]] −Eqφ(z∣x) [log qφ(z∣x)]

= L̃.

(16)

Intuitively, this is simply the case of a latent variable model with predefined, fixed dependencies.By optimizing L̃ w.r.t. µ, we hope to collapse p(c) to a delta distribution at the optimal (MAP)configuration, c∗, because

Lc∗ = Eδc,c∗ [Lc] ≥ Ep(c) [Lc] = L̃. (17)

Effectively, we can search for the dependency structure with the highest ELBO by optimizing thedistribution parameters of p(c). Although this optimization procedure may arrive at locally optimaldependency structures, our hope is that these learned structures will still perform better than anarbitrary, predefined dependency structure.

13


B IMPLEMENTATION DETAILS

B.1 NETWORK ARCHITECTURE

We document the network architectures used in our experiments. We use the same network archi-tectures for all datasets. Input size (pixels per image) is the only difference across datasets. Theinput size is 28 × 28 for MNIST and Omniglot, and 3 × 32 × 32 for CIFAR-10.

Encoders:

fc(input size,512) → batch norm → ELU → fc(512,512) → batch norm → ELU →fc(512,256) → batch norm → ELU → fc(256,128)

Latent Dependencies: Latent dependencies are modelled by non-linear MLPs. Note that the top-down architecture is shared between inference model and generative model, but the MLPs are opti-mized independently.

bottom-up: For each node with N ′ dimensions, the local potential is predicted by a mapping fromthe encoded feature:

fc(128,128) → batch norm → ELU → fc(128,N ′). The output feature is then mapped toµ and log var with two independent fc(N ′,N ′) layers, respectively.

top-down: For each node (N ′ dimensions) with a set of parent nodes, the top-down inference/gen-eration is implemented as:

fc(sum of parent nodes’ dimension, 128) → batch norm → ELU →

fc(128,N ′). The output feature is then mapped to µ and log var with two independentfc(N ′, N ′) layers, respectively.

Decoders:

fc(N ′,256) → batch norm → ELU → fc(256,512) → batch norm → ELU →fc(512,512)→ batch norm→ ELU→ fc(512,input size)→ output function()

output function for MNIST and Omniglot is sigmoid(), which predicts the mean µ ofBernoulli observations; and sigmoid() predicting µ, fc(input size,input size) pre-dicting log var of Gaussion observations for CIFAR-10.

B.2 TRAINING

All models were implemented with PyTorch (Paszke et al., 2017) and trained using the Adam(Kingma & Ba, 2015) optimizer with a mini-batch size of 64 and learning rate of 1e−3. Learn-ing rate is decresed by 0.25 every 200 epochs. The Gumbel-softmax temperature was initialized at1 and decreased to 0.99epoch at each epoch. MNIST and Omniglot took 2000 epochs to converge,and CIFAR took 3000 epochs to converge.

B.3 INFERENCE MODULE DETAILS

Parameter prediction for the local factors qφ(zn∣x,zpa(n)) consists of a precision-weighted fusionof the top-down prediction ψ̂n and a bottom-up prediction ψ̃n. Specifically, for a latent variable zn,ψ̂n ∶= {µ̂n, σ̂n}, and ψ̃n ∶= {µ̃n, σ̃n}.

Bottom-up Inference. A high-dimensional input is first mapped to a feature vector hx by an encoderMLP. hx is then used to predict µ̃n and σ̃n with non-linear MLPBUn .

Top-down Inference. µ̂n and σ̂n are predicted by µ̂n = MLPTDn ([zpa(n) ⊙ cpa(n),n]) and σ̂n =

MLPTDn ([zpa(n) ⊙ cpa(n),n]), respectively. [,] denotes concatenation operation, and ⊙ denoteselement-wise multiplication.

Precision-weighted fusion. Having ψ̂n and ψ̃n, the parameters of the local conditional distributionis given by qφ(zn∣x,zpa(n),cpa(n),n) ∼ N (zn∣µn, σ

2n), with

14


σn =1

σ̂−2n + σ̃−2n,

µn =µ̂nσ̂

−2n + µ̃nσ̃

−2n

σ̂−2n + σ̃−2n.

(18)

15


C ADDITIONAL RESULTS

C.1 ROBUSTNESS OF LOG-LIKELIHOODS

In Fig. 5, we report the averaged test log-likelihoods and associated standard deviations of GraphVAE and our baselines at different epochs. All calculations are based on 5 independent runs.

Figure 5: Comparison of Test-time Log-Likelihoods on MNIST. Models are trained 5 times indi-vidually, and test-time performances are evaluated every 300 epochs, with mean log-likelihood andstandard deviation indicated by the colored bars.

C.2 LATENT EMBEDDINGS

Our training objective optimizes intrinsic structure (which does not necessarily correlate with se-mantic meaning) and does not incentivize a disentanglement of latent factors. Interestingly, a TSNE-visualization of the data as well as latent embeddings of Graph VAE and VAE on MNIST (Fig. 6)shows that the latent embedding of Graph VAE exhibits a large (semantic) gap between differentclasses, even though the model is trained in an unsupervised fashion. We will further investigate thisbehavior in future work.

(a) Graph VAE (b) VAE (c) Data

Figure 6: TSNE-Visualization of Latent Embeddings on MNIST. We visualize embeddings of (a)Graph VAE, (b) VAE, and (c) the data itself.

16

variational autoencoders with jointly optimized …mori/research/papers/he-iclr19.pdf ·...

Documents