relativistic monte carlo

Relativistic Monte Carlo

Xiaoyu Lu ∗ Valerio Perrone ∗University of Oxford University of Warwick

Leonard Hasenclever Yee Whye Teh Sebastian J. VollmerUniversity of Oxford University of Oxford University of Warwick

Abstract

Hamiltonian Monte Carlo (HMC) is a pop-ular Markov chain Monte Carlo (MCMC)algorithm that generates proposals for aMetropolis-Hastings algorithm by simulatingthe dynamics of a Hamiltonian system. How-ever, HMC is sensitive to large time discretiza-tions and performs poorly if there is a mis-match between the spatial geometry of thetarget distribution and the scales of the mo-mentum distribution. In particular the massmatrix of HMC is hard to tune well.In order to alleviate these problems we pro-pose relativistic Hamiltonian Monte Carlo, aversion of HMC based on relativistic dynam-ics that introduces a maximum velocity onparticles. We also derive stochastic gradientversions of the algorithm and show that theresulting algorithms bear interesting relation-ships to gradient clipping, RMSprop, Adagradand Adam, popular optimisation methods indeep learning. Based on this, we developrelativistic stochastic gradient descent by tak-ing the zero-temperature limit of relativisticstochastic gradient Hamiltonian Monte Carlo.In experiments we show that the relativis-tic algorithms perform better than classicalNewtonian variants and Adam.

1 Introduction

Markov chain Monte Carlo (MCMC) techniques basedon continuous-time physical systems allow the efficient

* These authors contributed equally to this work.

Proceedings of the 20th International Conference on Artifi-cial Intelligence and Statistics (AISTATS) 2017, Fort Laud-erdale, Florida, USA. JMLR: W&CP volume 54. Copyright2017 by the author(s).

simulation of posterior distributions, and are an impor-tant mainstay of Bayesian machine learning and statis-tics. Hamiltonian Monte Carlo (HMC) [1, 2, 3, 4, 5]is based on Newtonian dynamics on a frictionless sur-face, and has been argued to be more efficient thantechniques based on diffusions [6]. On the other hand,stochastic gradient MCMC techniques based on dif-fusive dynamics [7, 8, 9, 10] have allowed scalableBayesian learning using mini-batches.

An important consideration when designing suchMCMC algorithms is adaptation or tuning to the ge-ometry of the space under consideration [11, 12, 13].To give a concrete example, consider HMC. Let f(θ) bea target density that can be written as f(θ) ∝ e−U(θ)

where U(θ) is interpreted as the potential energy ofa particle in location θ. HMC introduces an auxil-iary momentum variable p so that the joint distribu-tion is f(θ, p) ∝ e−H(θ,p) where the Hamiltonian isH(θ, p) = U(θ) + 1

2mp>p. The quantity 1

2mp>p, where

m is the mass of the particle, represents the kineticenergy. Denoting by θ and p the time derivative ofθ and p, the leapfrog discretisation [2] of Hamilton’sequations θ = ∂H

∂p and p = −∂H∂θ gives

pt+1/2 ← pt − 12ε∇U(θt),

θt+1 ← θt + εpt+1/2m ,

pt+1 ← pt+1/2 − 12ε∇U(θt+1)

where ε is the time discretisation and the velocity ispt+1/2m . If m is too small, the particle travels too fast

leading to an accumulation of discretisation error. Tocompensate, ε needs to be set small and the compu-tational cost required increases. On the other hand,if m is too large, the particle travels slowly resultingin slow mixing for the resulting Markov chain. Whilethe mass parameter can be tuned, e.g., to optimiseacceptance rate according to theory [12], it only inci-dentally controls the velocity which ultimately affectsthe discretisation error and algorithm stability.

In this paper, we are interested in making MCMC al-


gorithms based on physical simulations more robust bydirectly controlling the velocity of the particle. Thisis achieved by replacing the Newtonian dynamics inHMC with the relativistic dynamics [14], whereby par-ticles cannot travel faster than the “speed of light”. Wealso develop relativistic variants of stochastic gradientMCMC algorithms and show that they work better andare more robust than the classical Newtonian variants.

The relativistic MCMC algorithms we develop haveinteresting relationships with a number of optimisa-tion algorithms popular in deep learning. Firstly, themaximum allowable velocity (speed of light) is remi-niscent of gradient clipping [15]. Our framework givesBayesian alternatives to gradient clipping, in the sensethat our algorithms demonstrably sample from insteadof optimising the target distribution (exactly or ap-proximately). Secondly, the resulting formulas (see(3)), which include normalisations by L2 norms, bearstrong resemblances to (but are distinct from) RM-Sprop, Adagrad and Adam [16, 17, 18]. Motivated bythese connections, we develop a relativistic stochasticgradient descent (SGD) algorithm by taking the zero-temperature limit of relativistic SGHMC, and showin experiments on feedforward networks trained onMNIST that it achieves better performance than Adam.

2 Relativistic Hamiltonian Dynamics

Our starting point is the Hamiltonian that governs thedynamics in special relativity [14],

H(θ, p) = U(θ) +K(p) (1)

K(p) = mc2(p>p

m2c2 + 1) 1

2

(2)

where the target density is f(θ) ∝ e−U(θ), for θ ∈ Rdinterpreted as the position of the particle, p ∈ Rd is amomentum variable, and K(p) is the relativistic kineticenergy. The two tunable hyperparameters are a scalar“rest mass” m and the “speed of light” c which boundsthe particle’s speed. The joint distribution f(θ, p) ∝e−H(θ,p) is separable, with the momentum variablehaving marginal distribution ∝ e−K(p), a multivariategeneralisation of the symmetric hyperbolic distribution.

The resulting dynamics is given by Hamilton’s equa-tions, which read

θ = ∂H

∂p= M−1(p)p, M(p) = m

(p>p

m2c2 + 1) 1

2

p = −∂H∂θ

= −∇U(θ), (3)

where M(p) can be interpreted as the relativistic massand M−1(p)p as the velocity of the particle (c.f. the ve-locity under Newtonian dynamics is m−1p). Note that

the relativistic mass is lower bounded by and increasesasymptotically to ‖p‖/c as the momentum increases,so that the speed M−1(p)‖p‖ is upper bounded by andasymptototes to c. On the other hand, the larger therest mass m, the smaller the typical “cruising” speedof the particle. Conversely, as m→ 0 the particle willtravel at the speed of light at all times, i.e. it behaveslike a photon. This gives an intuition for tuning bothhyperparameters c and m based on knowledge aboutthe length scale of the target density: we choose c asan upper bound on the speed at which the parameterof interest θ changes at each iteration, while we choosem to control the typical sensible speed at which theparameter changes. We will demonstrate this intuitionin the experimental Section 5.

A crucial property of the relativistic dynamics is thatthe mass is a function of the momentum. We will showin the next sections that this represents a simple way toadapt to the geometry of the target distribution that,as opposed to differential geometric approaches [11],does not require high order geometric information andmore complex integrators.

In very high dimensional problems (e.g. those in deeplearning, collaborative filtering or probabilistic mod-elling), the maximum overall speed imposed on thesystem might need to be very large so that reasonablylarge changes in each coordinate are possible at eachstep of the algorithm. This means that each coordinatecould in principle achieve a much higher speed thandesirable. An alternative approach is to upper boundthe speed at which each coordinate changes by choosingthe following relativistic kinetic energy

K(p) =d∑j=1

mjc2j

(p2j

m2jc

2j

+ 1) 1

2

, (4)

where j indexes the coordinates of the d-dimensionalsystem, and each coordinate can have its own mass mj

and speed of light cj . This leads to the same Hamilto-nian dynamics (3), but with all variables interpretedas vectors, and all arithmetic operations interpretedas element-wise operations. Experimental results willbe based on this independent-momenta variant, whichshowed consistently better performance. For simplicity,in the theoretical sections we will describe only theversion (2).

2.1 Relativistic Hamiltonian Monte Carlo

As a demonstration of the relativistic Monte Carloframework, we derive a relativistic variant of the Hamil-tonian Monte Carlo (HMC) algorithm [2, 1]. In thefollowing, we will refer to all classical variants as Newto-nian, since they follow Newtonian dynamics (e.g. New-tonian HMC (NHMC) vs relativistic HMC (RHMC)).

Lu, Perrone, Hasenclever, Teh and Vollmer

Each iteration of HMC involves first sampling the mo-mentum variable, followed by a series of L leapfrogsteps, followed by a Metropolis-Hastings accept/rejectstep. The momentum can be simulated by first simulat-ing the speed ‖p‖ followed by simulating p uniformly onthe sphere with radius ‖p‖. The speed ‖p‖ has marginaldistribution given by a symmetric hyperbolic distribu-tion, for which specialised random variate generatorsexist. Alternatively, the density is log-concave, and weused adaptive rejection sampling to simulate it. Theleapfrog steps [19] with stepsize ε follows (3) directly:set θ0, p0 to the current location and momentum andfor t = 1, . . . , L,

pt+1/2 ← pt − 12ε∇U(θt)

θt+1 ← θt + εM−1(pt+1/2)pt+1/2

pt+1 ← pt+1/2 − 12ε∇U(θt+1)

The leapfrog steps leave the Hamiltonian H ap-proximately invariant and are volume-preserving [20],so that the MH acceptance probability is simplymin(1, exp(−H(θL, pL) +H(θ0, p0))).

Observe that the momentum p is unbounded and maybecome very large in the presence of large gradientsin the potential energy. However, the size of the θupdate is bounded by εc and therefore the stabilityof the proposed sampler can be controlled. This be-haviour is essential for good algorithmic performanceon complex models such as neural networks, where thescales of gradients can vary significantly across differentparameters and may not be indicative of the optimalscales of parameter changes. This is consistent withpast experiences optimising neural networks, where itis important to adapt the learning rates individuallyfor each parameter so that typical parameter changesstay in a sensible range [15, 16, 17, 18, 21]. Such adap-tation techniques have also been explored for stochasticgradient MCMC techniques [22, 23], but we will ar-gue in Sections 3.2 and 5 that they introduce anotherform of instability that is not present in the relativisticapproach.

3 Relativistic Stochastic GradientMarkov Chain Monte Carlo

In recent years stochastic gradient MCMC (SGMCMC)algorithms have been very well explored as methodsto scale up Bayesian learning by using mini-batchesof data [7, 10, 9, 8, 24]. In this section we developrelativistic variants of SGHMC [10] and SGNHT [9, 24].These algorithms include momenta, which serve asreservoirs of previous gradient computations, thus canintegrate and smooth out gradient signals from previousmini-batches of data. As noted earlier, because themomentum can be large, particularly as the stochastic

gradients can have large variance, the resulting updatesto θ can be overly large, and small values of the stepsize are required for stability, leading potentially toslower convergence. This motivates our developmentof relativistic variants.

We make use of the framework of [8] for deriving SGM-CMC algorithms. Let z be a collection of variableswith target distribution f(z) ∝ e−H(z). Let D(z)be a symmetric positive-definite diffusion matrix andQ(z) be a skew-symmetric matrix describing an energy-conserving dynamics. [8] showed that a diffusion pro-cess has f(z) as its equilibrium distribution if and onlyif it has the following form:

dz = −[D(z) +Q(z)]∇H(z)dt+ Γ(z)dt+√

2D(z)dW

Γi(z) =d∑j=1

∂[Dij(z)+Qij(z)]∂zj

, (5)

where W is the d-dimensional Wiener process (Brow-nian motion) and Γ(z) is a correction factor whichcan be computed from D and Q. To derive a correctSGMCMC algorithm, one just has to choose D andQ, compute Γ, discretise the SDE, and substitute astochastic estimate ∇U(z) for ∇U(z). The stochasticgradient has asymptotically negligible variance com-pared to the noise injected by W for small step sizes.In the following, we will derive a range of relativisticalgorithms using different choices of the state space, Dand Q.

As a historical aside, the work of [8] brings togetherbeautifully a number of disparate partial results inthe applied mathematics and physics literature whichwe would like to highlight. In the case of constantdiffusion matrix D(z) = D0, the result goes back to[25]. Moreover, the characterization result of [8] followsfrom two observations. The first is to characterise SDEswith a prescribed invariant distribution using implicitdivergence free vector fields, which appeared in, e.g.,[26]. The second consists of an explicit expression ofthese vector fields in terms of skew-symmetric “steam”matrices Q. This is well known in the stochastic fluiddynamic literature, see e.g. [27].

3.1 Relativistic Stochastic GradientHamiltonian Monte Carlo

Suppose our noisy gradient estimate ∇U(θ) of ∇U(θ)is based on a minibatch of data. Then, appealing to thecentral limit theorem, we can assume that ∇U(θ) ≈∇U(θ) +N (0, B(θ)). Let z = (θ, p) and H(z) be therelativistic Hamiltonian in (2). Choosing

D(z) =(

0 00 D

), Q(z) =

(0 −II 0

), (6)


and thus Γ(z) = 0, where D is a fixed symmetricdiffusion matrix results in the following relativisticSGHMC dynamics:(dθdp

)=(

M−1(p)p−∇U(θ)−DM−1(p)p

)dt+

(0 00√

2D

)dWt

Using a simple Euler-Maruyama discretisation, therelativistic SGHMC algorithm is,

pt+1 ← pt − εt∇U(θt)− εtDM−1(pt)pt+N (0, εt(2D − εtBt))

θt+1 ← θt + εtM−1(pt+1)pt+1 (7)

where B is an estimate of the noise coming from thestochastic gradient B(θ). The term DM−1(p)p canbe interpreted as friction, which prevents the kineticenergy to build up and corrects for the noise comingfrom the stochastic gradient.

It is useful to compare RSGHMC with preconditionedSGLD [22, 23], which attempts to adapt the SGLDalgorithm to the geometry of the space, using adap-tations similar to RMSProp, Adagrad or Adam. Therelevant term above is the update M−1(pt+1)pt+1 toθt+1:

M−1(pt+1)pt+1 = pt+1√p>

t+1pt+1

c2 +m2(8)

Note the surprising similarity to RMSProp, Adagradand Adam, with the main difference being that therelativistic mass adaptation uses the current momen-tum instead of being separately estimated using thesquare of the gradient. This has the advantage thatthe relativistic SGHMC enforces a maximum speed ofchange. In contrast, as we also observe in Section 5,preconditioned SGLD has the following failure mode:when the gradient is small, the adaptation scales up thegradient so that the gradient update has a reasonablesize. However it also scales up the injected noise, whichcan end up being significantly larger than the gradientupdate, and making the algorithm unstable.

3.2 Relativistic Stochastic Gradient Descent(with Momentum)

Motivated by the relationship to RMSprop, Adagradand Adam, we develop a relativistic stochastic gradientdescent (RSGD) algorithm with momentum by takingthe zero-temperature limit of the RSGHMC dynamics.This idea connects to Santa [28], a recently developedalgorithm where an annealing scheme on the systemtemperature makes it possible to obtain a stochasticoptimization algorithm starting from a Bayesian one.

From thermodynamics [29], the canonical (Gibbs Boltz-mann) density is proportional to e−βU(z), where β is

the inverse temperature. Previously we have been usingβ = 1, which corresponds to the posterior distribution.For general β,(

dθdp

)=(

βM−1(p)pβ(−∇U(θ)−DM−1(p)p

)) dt+(

0 00√

2D

)dW

By taking β → ∞ the target distribution becomesmore peaked around the MAP estimator. Simulatedannealing [30, 31, 28], which increases β → ∞ overtime, forces the sampler to converge to a MAP estima-tor. We can derive RSGD by rescaling time as well,guaranteeing a non-degenerate limit process. Lettingθ(t) = θ(βt), p(t) = p(βt), so that(

dθdp

)=(


)dt

+(

0 00√

2Dβ

)dW

and letting β →∞, we obtain the following ODE:(dθdp

)=(


)dt

Discretising the above then gives RSGD. Notice that ifthe above converges, i.e. θ = p = 0, it does so at a crit-ical point of U . Similar to other adaptation schemes,RSGD adaptively rescales the learning rates for dif-ferent parameters, which enables effective learning es-pecially in high dimensional settings. Moreover, theupdate in each iteration is upper bounded by the speedof light. Our algorithm differs from others throughthe use of a momentum, and adapting based on themomentum instead of the average of squared gradients.

4 A Stochastic Gradient Nosé-HooverThermostat for RelativisticHamiltonian Monte Carlo

Borrowing a second concept from physics, SGHMC canbe improved by introducing a dynamic variable ξ thatadaptively increases or decreases the momenta. Thenew variable ξ can be thought of as a thermostat in astatistical physics setting and its dynamics expressedas

dξ = 1d

(pT p− d

)dt. (9)

The idea is that the system adaptively changes thefriction for the momentum, ‘heating’ or ‘cooling down’the system. The dynamics of this new variable, knownas Nosé-Hoover [32] thermostat due to its links to sta-tistical physics, has been shown to be able to remove


the additional bias due to the stochastic gradient pro-vided that the noise is isotropic Gaussian and spatiallyconstant ([9],[20]). In general, the noise is neither Gaus-sian, spatially constant or isotropic. Nevertheless, thereis numerical evidence that the thermostat increases sta-bility and mixing. Heuristically, the dynamics for ξ canbe motivated by the fact that in equilibrium we have

E[∂2K

∂p2i

]=∫∂2K

∂p2i

e−K(p)dp

= −∫∂K

∂pi

(−∂K∂pi

e−K(p))dp

= E

[(∂K

∂pi

)2].

Since ∂K∂pi

= pi and ∂2K∂p2

i= 1 this implies

E[dξ

dt

]= E

[1d

∑i

((∂K

∂pi

)2− ∂2K

∂p2i

)]= 0

The additional dynamics pushes the system towardsdξdt = 0, suggesting that the distribution will be movedcloser to the equilibrium. This gives a recipe for astochastic gradient Nosé-Hoover thermostat with ageneral kinetic energy K(p).

We first augment the Hamiltonian with ξ:

H(q, p, ξ) = U(q) +K(p) + d

2(ξ −D)2.

We are now in the position to derive the SDE preservingthe probability density ∝ exp(−H) by adopting theframework of [8] and defining:

H(θ, p, ξ) = U(θ) +K(p) + d

2(ξ −D)2

D(θ, p, ξ) =

0 0 00 D · I 00 0 0

Q(θ, p, ξ) =

0 −I 0I 0 ∇K(p)/d0 −∇K(p)T /d 0

.

From (5) it follows that Γ = (0 0 −∆K(p)/d)T andthe dynamics becomesdθdp

dξ

=

∇K(p)−∇Udt− ξ∇K(p)

1d

(‖∇K(p)‖2 −∆K(p)

) dt

+

0 0 00√

2D 00 0 0

dWt

where ∆ is the Laplace operator defined as ∆K(p) =∑i∂2K(p)∂p2

i. For the relativistic kinetic energy

K(p), we have that ∇pK(p) = M−1(p)p with

M(p) := m(pT pm2c2 + 1

) 12 a scalar and that ∆K(p) =

tr(ddp

( 1dM

−1(p)p))

. The Stochastic Gradient Nosé-Hoover Thermostat for relativistic HMC follows:

dθdpdξ

=

M−1(p)p−∇U − ξM−1(p)p

pT pd

(M−2(p) + c−2M−3(p)

)−M−1(p)

dt

+

0 0 00√

2D 00 0 0

dWt.

5 Experiments

5.1 Synthetic data

All the experimental results in this section are basedon the independent-momenta versions (4) as they givesuperior results. We first explore the performancesof the algorithms on a set of small examples includ-ing a two dimensional banana function (Banana) [33]with density p(x) ∝ exp{−0.5(0.01x2

1 + (x2 + 0.1x21 −

10)2)}, and Gaussian mixture models (GMM1, GMM2,GMM3) obtained by combining the three followingGaussian random variables with equal mixing pro-portions: N (−5, 1/σ2), N (0, σ2), N (5, 1/σ2), whereσ2 = 1, 0.5, 0.3. When σ2 = 1 the three Gaussianshave the same variance, while a lower σ2 means largerdiscrepancies between their variances and thus a widerrange of length scales and log density gradients. Thedensity plots of the examples can be found in the toprow of Figure 3.

We start with an exploration of the behaviour of RHMCas the tuning parameters m, c and ε are varied. Firstwe considered the effective sample sizes (ESS) of the al-gorithm on the Banana and GMM1 datasets. We variedboth ε and ε× c over a grid, and computed the averageESS, over 20 chains, each of length 104 for Banana,and over 100 chains of length 105 for GMM1. The ESScontour plots can be found in Figure 1, which suggeststhat εc and ε can be independently tuned. While εcontrols the time discretisation of the continuous-timedynamics, εc controls the maximum change in the pa-rameters at each leapfrog step. Next we varied the massparameter m for GMM1, showing plots in Figure 2. Asexpected the ESS is optimised at an intermediate valueof m, and the average “cruising speed” v decreaseswith m. In order to understand how to tune m, on thethird panel we overlaid two contour plots: one for ESSand the other for v. We see that the cruising speedv correlates much better with the ESS than m does,which suggests that m should be tuned via v, e.g. by


stepsize

0 1 2 3 4

1500

1000

500

1

2000

Color

0

2

4

6

8

stepsi

ze*c

ESS (GMM 1)

Figure 1: ESS contour plots of ε× c versus ε for Banana and GMM1 models.

mass

0 5 10

0

500

1000

1500

2000

2500

ESS

mass

0 5 10

0.2

0.3

0.4

0.5

0.6

0.7

Cruising speed

Figure 2: Varying m for GMM1. From left to right: ESS, cruising speed (the red horizontal line is c), and ESSand relative cruising speed v/c contour plots versus m and ε.

the user specifying a desired value for v and m beingadapted to achieve the speed (noting that m and vhave a monotonic relationship, which makes for easyadaptation).

We next compare the performances of NHMC andRHMC for a wide range of stepsizes, via the ESS (higherbetter), the mean absolute error (MAE) between thetrue probabilities and the histograms of the sample fre-quencies (lower better), and the log Stein discrepancy[34] (lower better). The Stein discrepancy is a more ac-curate measure of sample quality, the reason being thatit can be used to bound the Wasserstein distance, thusaccounting for bias and insufficient exploration of thetarget. The results can be found in rows 2-4 of Figure 3.It can be seen that RHMC achieves better performanceand is strikingly more robust to the step size ε thanNHMC. As expected, this behaviour is particularly pro-nounced when the step size is large. Moreover, whenthe gradients of the target model span a large range ofvalues (GMM2, GMM3), the improvements yielded bythe relativistic variants are more pronounced. Theseresults confirm that, since the speed of particles isbounded by c, RHMC is less sensitive to the presenceof large gradients in the target density and more stablewith respect to the choice of ε, allowing for a moreefficient exploration of the target density.

Next we compare both the Newtonian and relativisticvariants of HMC and SGMCMC algorithms on a sim-ulated 3-dimensional logistic regression example with500 observations. For the stochastic versions of thealgorithms, we use mini-batches of size 100. After aburn-in period of 1000 iterations, we calculated theStein discrepancy for different ε while keeping the prod-uct ε× c fixed. To make a fair comparison in terms ofcomputational time, we used 200 samples for NHMCand RHMC and 1000 samples for the SGMCMC al-gorithms. From Figure 4, we see that the relativisticvariants are significantly more robust than the Newto-nian variants. The NHT algorithms were able to correctfor stochastic gradient noise and performed better thanSGHMC algorithms. Particularly, RSGNHT had lowerStein discrepancies than the other algorithms for mostvalues of ε.

5.2 Neural Networks

Turning to more complex models, we first considered aneural network with 50 hidden units and initialised itsweights by the widely used Xavier initialisation. Weused the Pima Indians dataset for binary classification(552 observations and 8 covariates) to compare therelativistic and the preconditioning approach. Indeed,


-10 -5 0 5 10

0.00

0.05

0.10

0.15

0.20

0.25

GMM 1

-10 -5 0 5 10

0.00

0.05

0.10

0.15

0.20

0.25

GMM 2

-10 -5 0 5 10

0.00

0.05

0.10

0.15

0.20

0.25

GMM 3

stepsize

0 2 4 6 8

0

50

100

150

200

RHMCNHMC

stepsize

0 2 4 6 8

0

50

100

150

200

RHMCNHMC

stepsize

0 2 4 6 8

0

50

100

150

200

RHMCNHMC

stepsize

0 2 4 6 8

0

50

100

150

200

RHMCNHMC

stepsize

0 2 4 6 8

0

50

100

150

200

RHMCNHMC

stepsize

0 2 4 6 8

0

50

100

150

200

RHMCNHMC

stepsize

0 2 4 6 8

-4

-3

-2

-1

0

1

2

RHMCNHMC

stepsize

0 2 4 6 8

-4

-3

-2

-1

0

1

2

RHMCNHMC

stepsize

0 2 4 6 8

-4

-3

-2

-1

0

1

2

RHMCNHMC

Figure 3: Left to right: Banana, GMM1, GMM2, GMM3 datasets. Top to bottom: density plot, ESS versus stepsize ε, MAE versus ε, log stein discrepancy versus ε.these methods represent two different ways to normalisegradients so that the update sizes are reasonable for thelocal lengthscale of the target distribution. In particu-lar we consider SGLD Adam, namely a preconditionedSGLD algorithm with an additional Adam-style debias-ing of the preconditioner. Figure 5 compares the test-set accuracy of SGLD Adam with RSGD and RHMC,showing that the first is significantly outperformed bythe relativistic algorithms. Due to Xavier initializationall of the weights are small, which causes small gradi-ents; therefore, the injected noise becomes very largedue to the rescaling by the inverse of squared root of theaverage gradients, which makes SGLD Adam unstable.The histograms reveal that at the first iteration the pre-conditioning causes the weights to become extremelylarge and this strongly compromises the performanceof SGLD Adam, which takes a long time to recover.The relativistic framework represents therefore a muchbetter approach to perform adaptation of the learningrates specific to each parameter.

We then apply our algorithms to the standard MNISTdataset, which consists of 28 × 28 handwritten digi-tal images from 10 classes with a training set of size60, 000 and a test set of size 10, 000. We tested ouroptimization algorithm on a single layer with 100 hid-

Figure 4: Stein discrepancy versus stepsize ε for logisticregression. NSGHMC and NSGNHT were unstable forε > 6 × 10−3 and thus their Stein discrepancies werenot plotted.

den units and two multi-layer neural networks with500∗300 and 400∗400 hidden units. In Figure 6 a com-parison with Adam and Santa [28] is displayed (theirrelation is discussed in more detail in Section 3.2). As


weights

-1.0 -0.5 0.0 0.5 1.0

0.0

0.5

1.0

1.5

2.0

RSGD

weights

-1.0 -0.5 0.0 0.5 1.0

0.0

0.5

1.0

1.5

RHMC

weights

-4.0×10⁴ -2.0×10⁴ 0 2.0×10⁴ 4.0×10⁴

0.00000

0.00005

0.00010

0.00015

0.00020

SGLD Adam

Figure 5: Comparison between RSGD, RHMC and SGLD Adam on the Pima Indians dataset using 50 hiddenunits. The histograms show the neural network weights at the first iteration.

Figure 6: Comparison of the test-set error rate on the MNIST dataset. From left to right: 100 hidden units;500 ∗ 300 hidden units; 400 ∗ 400 hidden units.

the goal of these experiments is to compare stochasticoptimizers, we consider Santa SGD, namely Santa inits zero-temperature limit. In addition, we adopt anEuler integration scheme for all algorithms, since theimprovements yielded by higher-order integrators areorthogonal to the specific algorithm. It can be observedthat our algorithm is competitive with Adam and isable to achieve a lower error rate, particularly withthe 100 hidden units architecture. Moreover, RSGDperforms significantly better than Santa SGD on allthe considered architectures.

6 Conclusion

Our numerical experiments demonstrate that the rel-ativistic algorithms developed in this paper are muchmore stable and robust to the choice of parameters andnoise in stochastic gradients compared to the Newto-nian counterparts. Moreover, while the standard New-tonian algorithms also require parameter tuning, inthe relativistic setting we have a clearer understandingof how each parameter affects performance: εc and mrespectively control the maximal parameter change andthe average speed. This direct interpretation simplifiestuning and is used consistently across experiments toselect parameters, achieving improved performance fora wider range of parameters.

The connection of our algorithms with popular stochas-tic optimizers such as Adam and RMSProp is novel andgives an interesting perspective to understand them.Moreover, the relativistic stochastic gradient descentshowed to be very competitive with state of the artstochastic gradient methods for fitting neural nets. Weanticipate a variety of algorithms with different kineticenergies to be developed following our work.

Each of the proposed methodologies has scope for fur-ther research. The HMC version of the algorithm couldbe improved by some more advanced HMC methodol-ogy such as the NUTS version [4] and partial momentrefreshment instead of Adaptive Rejection Sampling[2]. Additionally, better numerical integration schemescould be employed. Finally, we leave an in-depth er-godic theory as an interesting avenue for future work.

Acknowledgements

XL thanks the PAG scholarship and New College forsupport. VP and LH are funded by the EPSRC CDTOxWaSP through EP/L016710/1. SJV thanks EPSRCfor funding through EPSRC grants EP/N000188/1 andEP/K009850/1. YWT acknowledges the EuropeanResearch Council under the European Union’s SeventhFramework Programme (FP7/2007-2013) ERC grantagreement no. 617071 and EPSRC for funding throughEP/K009362/1.


References

[1] S. Duane, A.D. Kennedy, B.J. Pendleton, andD. Roweth. Hybrid Monte Carlo. Physics letters B,195(2):216–222, 1987.

[2] R. M. Neal. MCMC using Hamiltonian dynamics.Handbook of Markov Chain Monte Carlo, 54:113–162,2010.

[3] B. Carpenter, D. Lee, A. M. Brubaker, A. Riddell,A. Gelman, B. Goodrich, J. Guo, M. Hoffman, M. Be-tancourt, and P. Li. Stan: A probabilistic programminglanguage. Journal of Statistical Software, 2015.

[4] Matthew D. Hoffman and Andrew Gelman. The No-U-Turn Sampler: Adaptively Setting Path Lengthsin Hamiltonian Monte Carlo. Journal of MachineLearning Research, 15:1593–1623, 2014.

[5] S. Livingstone, M. Betancourt, S. Byrne, and M. Giro-lami. On the Geometric Ergodicity of HamiltonianMonte Carlo. ArXiv e-print 1601.08057, 2016.

[6] T. Xifara, C. Sherlock, S. Livingstone, S. Byrne, andM. Girolami. Langevin diffusions and the Metropolis-adjusted Langevin algorithm. Statistics & ProbabilityLetters, 91(C):14–19, 2014.

[7] M. Welling and Y.W. Teh. Bayesian Learning viaStochastic Gradient Langevin Dynamics. In ICML,pages 681–688. Omnipress, 2011.

[8] Y. Ma, T. Chen, and E. B. Fox. A Complete Recipefor Stochastic Gradient MCMC. In NIPS, 2015.

[9] N. Ding, Y. Fang, R. Babbush, C. Chen, R. D. Skeel,and H. Neven. Bayesian Sampling Using StochasticGradient Thermostats. In NIPS, pages 3203–3211,2014.

[10] T. Chen, E. Fox, and C. Guestrin. Stochastic GradientHamiltonian Monte Carlo. In ICML, pages 1683–1691,2014.

[11] M. Girolami and B. Calderhead. Riemann mani-fold Langevin and Hamiltonian Monte Carlo meth-ods. Journal of the Royal Statistical Society: Series B(Statistical Methodology), 73(2):123–214, March 2011.

[12] A. Beskos, N. Pillai, G. O. Roberts, J. M. Sanz-Serna,and A. M. Stuart. Optimal tuning of hybrid MonteCarlo algorithm. Bernoulli, 19:1501–1534, 2013.

[13] S. Patterson and Y. W. Teh. Stochastic gradient Rie-mannian Langevin dynamics on the probability sim-plex. In Advances in Neural Information ProcessingSystems, 2013.

[14] A. Einstein. On the Electrodynamics of Moving Bodies.Annalen der Physik, 17, 1905.

[15] R. Pascanu, T. Mikolov, and Y. Bengio. On the diffi-culty of training recurrent neural networks. In ICML,2013.

[16] T. Tieleman and G. Hinton. Lecture 6.5-RMSProp:Divide the gradient by a running average of its recentmagnitude, 2012. COURSERA: Neural Networks forMachine Learning.

[17] J. Duchi, E. Hazan, and Y. Singer. Adaptive Sub-gradient Methods for Online Learning and StochasticOptimization. JMLR, 12:2121–2159, July 2011.

[18] D. P. Kingma and J. Ba. Adam: A method for stochas-tic optimization. In ICLR, 2015.

[19] J. M. Sanz-Serna and M.P. Calvo. Numerical Hamilto-nian problems. Applied mathematics and mathematicalcomputation. Chapman & Hall, 1994.

[20] B. Leimkuhler and X. Shang. Adaptive Thermostats forNoisy Gradient Systems. SIAM Journal on ScientificComputing, 38(2):A712–A736, 2016.

[21] Umut Şimşekli, Roland Badeau, A Taylan Cemgil,and Gaël Richard. Stochastic Quasi-Newton LangevinMonte Carlo. ICML, 2016.

[22] Chunyuan Li, Changyou Chen, David Carlson, andLawrence Carin. Preconditioned Stochastic GradientLangevin Dynamics for Deep Neural Networks. InAAAI Conference on Artificial Intelligence, 2016.

[23] L. Hasenclever, S. Webb, T. Lienart, Y. Whye Teh,S. Vollmer, B. Lakshminarayanan, and C. Blundell.Distributed Bayesian Learning with Stochastic Natural-gradient Expectation Propagation and the PosteriorServer. ArXiv e-print 1512.09327, December 2015.

[24] X. Shang, Z. Zhu, B. Leimkuhler, and A. J. Storkey.Covariance-Controlled Adaptive Langevin Thermostatfor Large-Scale Bayesian Sampling. In NIPS, pages37–45, 2015.

[25] Chii-Ruey Hwang, Shu-Yin Hwang-Ma, and Shuenn-Jyi Sheu. Accelerating gaussian diffusions. The Annalsof Applied Probability, pages 897–913, 1993.

[26] G. A. Pavliotis. Stochastic processes and applications,volume 60 of Texts in Applied Mathematics. Springer,New York, 2014. Diffusion processes, the Fokker-Planckand Langevin equations.

[27] Tomasz Komorowski, Claudio Landim, and StefanoOlla. Fluctuations in Markov processes: time symmetryand martingale approximation, volume 345. SpringerScience & Business Media, 2012.

[28] Changyou Chen, David Carlson, Zhe Gan, ChunyuanLi, and Lawrence Carin. Bridging the gap betweenstochastic gradient MCMC and stochastic optimization.AISTATS, 2016.

[29] Normand M Laurendeau. Statistical Thermodynamics:Fundamentals and Applications. Cambridge UniversityPress, 2005.

[30] S. Geman and C. Hwang. Diffusions for global opti-mization. SIAM Journal on Control and Optimization,24(5):1031–1043, 1986.

[31] C. Andrieu and A. Doucet. Simulated annealing formaximum a posteriori parameter estimation of hiddenMarkov models. IEEE Transactions on InformationTheory, 46(3):994–1004, 2000.

[32] B. Leimkuhler and S. Reich. A Metropolis adjustedNosé-Hoover thermostat. ESAIM: Mathematical Mod-elling and Numerical Analysis, 43:743–755, 7 2009.


[33] H. Haario, E. Saksman, and J. Tamminen. AdaptiveProposal Distribution for Random Walk MetropolisAlgorithm. Computational Statistics, 14(3):375–396,1999.

[34] J. Gorham and L. W. Mackey. Measuring SampleQuality with Stein’s Method. In NIPS, pages 226–234,2015.

relativistic monte carlo

Documents