Weighted Ensemble Milestoning (WEM): A Combined Method For RareEvent Simulations

Dhiman Ray1 and Ioan Andricioaei1, a)

Department of Chemistry, University of California Irvine, California 92697, USA

Direct simulation of rare events using atomistic molecular dynamics is a significant challenge in computational bio-physics. Well established enhanced sampling techniques like umbrella sampling, metadynamics and adaptive-biasingforces are able to calculate the free energy landscapes for rare events involving barrier crossing. But developing meth-ods for obtaining kinetics of long timescale processes from molecular dynamics simulation is still an emerging area ofscience. Milestoning and weighted ensemble (WE) are two different path sampling based strategies which have shownpromises for computing timescales of complex biomolecular processes. Nevertheless, both require a significant invest-ment of computational resources. We have combined WE and milestoning, in a novel way, to calculate experimentalobservables in orders of magnitude less CPU and wall-clock time. Our method uses WE simulation to converge thetransition probability and first passage times between milestones, followed by the utilization of the theoretical frame-work of milestoning to extract thermodynamic and kinetic properties of the entire process. We tested our method for asimple one dimensional double well potential, an eleven dimensional potential energy surface with energy barrier, andon a molecular system of alanine dipeptide. We were able to recover the free energy profiles, time correlation func-tions, and mean first passage times for barrier crossing events at a significantly small computational cost. Our methodpromises to extend the applicability of molecular dynamics simulation to the slow dynamics of large systems which arebeyond the scope of present day computer power.

Keywords: Milestoning, Weighted Ensemble, enhancedsampling, rare event, kinetics

I. INTRODUCTION

Computing thermodynamics and kinetics of complex bio-physical processes from molecular dynamics (MD) simulationis challenging nevertheless of significant interest1,2. Biologi-cally relevant molecular processes take place at a wide rangeof timescales spanning from picoseconds to seconds. Exam-ples of such processes include protein side chain motion (ps-ns), relative motion of different protein domains (ns-µs), pro-tein folding, ligand binding and allosteric transitions (µs-s)3,4.But due to the fast bond vibrational motions at fs timescale,the integration time-step of MD trajectories can not exceed be-yond 2-4 fs to avoid instability in numerical integration5. Be-sides, aqueous environment is necessary for most biomolecu-lar processes in cell. To include the effect of water and ionsexplicitly, the size of the simulation box becomes of the or-der of 104-106 atoms. Despite the enormous development ofcomputing hardware in past few decades, a µs to ms timescalesimulation for such system is difficult to perform with cur-rently available computational resources. Specialized com-puting hardware, like Anton supercomputer, has been pre-pared by Shaw et al. which can routinely perform µs to mssimulation of a hundred thousand atom system6. Distributedcomputing over Graphical Processing Unit (GPU) and GPU-grid based hardware has also revolutionized the field of com-putational biophysics by enabling µs timescale simulation forbiologically relevant systems7,8.

Nevertheless, developing methodologies for calculatingfree energy profile and kinetics of long time processes from

a)Electronic mail: andricio@uci.edu.

short time molecular dynamics simulation remains of sig-nificant interest9. One single long MD trajectory is proneto remain confined in a low energy minima and avoid sam-pling high energy regions10. Although conventional enhancedsampling methods like umbrella sampling (US)9,11, metady-namics (MtD)12, adaptive biasing force (ABF)13, and steeredmolecular dynamics (sMD)14 can provide the free energylandscape along pre-defined reaction coordinate, accurate es-timation of kinetic properties from biomolecular simulationremains elusive. Recently, Tiwary and Parinello have de-vised a scheme for calculating rate constants from infrequentmetadynamics simulations, using an Arrhenius rate equationbased approach15. This method has been proved to be suc-cessful for µs scale ligand unbinding and folding simulationsof small proteins16. Alternatively, Doshi and Hamelberg17,and Frank and Andricioaei have developed a method of es-timating rate constants of slow biomolecular processes usingKramers rate theory from potential scaled simulations2. Thismethod has been used by Deb and Frank for determining firstpassage times of drug unbinding from protein from µs scalesimulation18.

The biggest challenge for obtaining converged kinetics for aslow process is sampling many successful transitions from theinitial state to final state19 which requires significantly longerMD simulations than the average first passage time of the par-ticular event. As most processes of physiological interest hap-pen at beyond µs timescale, simulating many such events goeswell beyond the scope of currently available computational re-sources. Besides, one long MD trajectory often ends up over-sampling a local minimum close to the starting configurationbut fails to cross the free energy barrier to reach the targetstate for purely statistical reason12. This necessitates runningmultiple trajectories with different initial conditions leadingto increase in computational cost by a significant degree.

Path sampling based strategies has been proved to be usefulfor estimating kinetics from MD simulations20.

arX

iv:1

912.

1065

0v2

[co

nd-m

at.s

tat-

mec

h] 1

2 Ja

n 20

20

2

A variety of methods like transition path sampling(TPS)21,22, transition interface sampling (TIS)23, forwardflux sampling (FFS)24,25, weighted ensemble (WE)26–28 andmilestoning29–31 have appeared in the molecular dynamics lit-erature in the past two decades. All these methods discretizethe reaction coordinate into many bins or intermediate states(except for steered transition path sampling which does notrequire definition of interfaces32). In WE and FFS methodMD trajectories are stopped and spawned if they cross the binboundaries, increasing the number of trajectories in relativelyunder-sampled regions of the configurational space. In TPS,TIS and milestoning, short trajectories are initiated from in-termediate values of reaction coordinate and the probabilitiesof them reaching the initial, final or other intermediate inter-faces are monitored. These short trajectories can also be ac-celerated by applying biasing force followed by re-weightingbased on stochastic path integral to get the appropriate ki-netic information19,33,34. Earlier studies confirmed that theprocess of decomposing a longer biased trajectory into manyshorter ones accelerate the convergence of free energies35,36.Trajectory stratification technique using non-equilibrium um-brella sampling (NEUS)37–39 and its modified versions40–43

could also calculate averages of stationary and dynamic prop-erties with reasonable efficiency and accuracy44. But unlikeother path sampling methods the re-weighting of the biasedtrajectories in NEUS is not performed at the individual tra-jectory level but at a specific region of the configurationalspace like US44. A relatively older method, dynamic impor-tance sampling (DIMS),45 utilizes similar stratification tech-nique to sample biomolecular conformation46 and obtain ki-netics of barrier crossing processes. But unless the biasingforce is chosen carefully47, small biases tend to accumulateand consequently, quantitative observables (e.g. rates) do notconverge32,46. Still, DIMS can efficiently generate initial un-biased paths for TPS through an annealing scheme48 and thesepaths often provide reasonable mechanistic insight32,46.

Markov state modeling (MSM) has been developed andused since the last decade for estimating kinetics of longtimescale processes from MD simulation. It decomposes largeamount of trajectory data into meta-stable states or clustersdepending on structural or kinetic criteria, and subsequentlybuilds a Markov chain between the states49–52. The input datacan either be a very long single MD trajectory or multipleshort trajectories with significant amount of inter conversionsin-between the meta-stable states. The primary advantage ofMSM is that it does not require a predefined reaction coor-dinate. Techniques like time-lagged independent componentanalysis (TICA) can be used to identify the slowly varyingdegrees of freedom53,54. The thermodynamics and kineticsalong them can later be captured from the eigenvalues of theMarkovian transition probability matrix.

Recently, milestoning and weighted ensemble based pathsampling methods have gained popularity in computationalbiophysics community because of their open source imple-mentation in commonly used molecular dynamics packages.The Weighted Ensemble Simulation Toolkit with Paralleliza-tion and Analysis (WESTPA) is built by Chong and cowork-ers for performing WE simulation in conjunction with molec-

ular dynamics and Brownian dynamics55. It has found itsapplications in studying the free energy and kinetics of aplethora of interesting biophysical processes including proteinfolding56,57, formation of host guest complex58, protein ligandbinding59,60, ion permeation through protein channel61, viralcapsid assembly62 etc. Lately Lotz and Dickson have calcu-lated the kinetics of an 11 minutes timescale drug unbindingprocess with qualitative agreement from WExplore63 simula-tion based on WE scheme64,65.

Likewise Elber and coworkers have recently shown the rig-orous derivation of expressions of free energy landscape andmean first passage time from milestoning, a method they pro-posed in the last decade. They also proved that milestoningcan be considered statistically exact at the infinite samplinglimit31. Different variants of milestoning have incorporatedinnovative strategies for constructing milestoning boundaries(e.g. Voronoi tessellations) to effectively sample the reactioncoordinate on the way to the product state. Examples of theminclude the directional milestoning by Majek et al66 and theMarkovian milestoning by Vanden-Eijnden et al.67. Takencollectively, the milestoning methods have been successfullyapplied to problems like allosteric transitions68, membranepermeation by small molecules69,70 and other biological mem-brane systems67,71. Votapka et al. have simulated the bindingof a ligand to a protein using multi-scale simulations involvingBrownian dynamics and molecular dynamics in conjunctionwith milestoning72. They have implemented their techniquein a open source package SEEKR which can run alongsidepopular MD simulation package NAMD73 and BD packageBrownDye4,74. Moreover, Grazioli and Andricioaei have de-veloped an enhanced milestoning protocol by applying bias-ing force to obtain quick convergence for milestone to mile-stone trajectories19. They also have come up with a strategyto calculate time correlation functions from milestoning sim-ulation using stochastic path integral75.

In its current form, WE method requires the trajectoriesreaching final state to be regenerated from the starting pointto maintain a steady state of probability flow27. This requiressignificantly more computing time unlike methods like mile-stoning where the trajectories are killed upon reaching the sinkstate. Another major limitation of WE simulation is that itgenerates correlated trajectories as large part of their historyremains exactly identical to each other. Trajectories, whichare not independent, are prone to give wrong results for freeenergy and time correlation function76.

Milestoning, on the other hand, has precise ways of cal-culating free energy, kinetics and time correlation functionfrom the milestone to milestone transition probabilities andlifetimes. However, milestones are to be placed sufficientlyfar from each other to ensure that trajectories starting froma given milestone are independent from their history of pre-viously visited milestones66. It requires longer simulationtimescales unlike closely spaced interfaces, and effectivelyincurs large computational cost when summed over all mile-stones. The wind-assisted reweighted milestoning (WARM)attempts to reduce the cost by accelerating the trajectories to-wards the adjacent milestone by applying a biasing force19.

In the current work, we propose a novel path sampling

3

algorithm combining weighted ensemble and milestoning toproduce a computationally more robust technique than ei-ther of the individual techniques. In this method we performweighted ensemble simulation in-between milestones for fastconvergence of the transition probability within each segmentbetween milestones. The converged transition probability ma-trix is then utilized to calculate the free energy, kinetics andtime correlation function from the framework of milestoningtheory described in Ref. 31.

This paper is organized in the following manner: In sec-tion II we review the theoretical framework of milestoning andweighted ensemble in brief, and proceed to describe our com-bined Weighted ensemble milestoning (WEM) procedure. InSection III we show the results of our method on a 1D doublewell potential, a coupled (10+1)D potential with enthalpic bar-rier and conformational transition in Alanine dipeptide. Wecompared the results of WEM with WE, milestoning and reg-ular MD simulation wherever applicable.

II. THEORY

A. Milestoning

The milestoning method was first proposed by Ron Elberand coworkers in the last decade29,30. Recently they have re-visited their earlier work and proposed the ‘exact milestoning’formalism as well as provided rigorous derivations of free en-ergy and kinetics from milestoning calculations31. Here wehave included a brief description on some topics from mile-stoning which are directly relevant to the our work. The readeris referred to Ref. 31 and Ref. 4 for the details of the deriva-tion.

Milestones are non-interacting hypersurfaces, preferablyorthogonal to the reaction coordinate in a given phase space30.M milestones (including the initial and final state) dividesthe configurational space into M− 1 domains. The primarygoal of milestoning is to estimate the flux of probability qi(t)through the milestone i for i ∈ [1,M]. To accomplish this,multiple trajectories are initiated from each milestone. Thetrajectories are stopped when they reached either of the adja-cent milestones. A transition kernel K is constructed from theprobabilities of transition between adjacent milestones in thefollowing way

Ki j =ni→ j

N; j = i±1

= 0; otherwise(1)

where ni→ j is the number of trajectories initiating at mile-stones i and ending at milestone j while N is the total numberof trajectories started from milestone i. Also a lifetime vec-tor T is obtained which contains the average lifetime of eachmilestone. Average lifetime of a milestone i is defined as theaverage time spent by the trajectories initiated from i beforethey reach either of milestone i− 1 or i+ 1. So the elementsof T are given by the following formula

T i =∑

Nl=1 tlN

(2)

where tl is the time spent by the l’th trajectory before hittingany of the adjacent milestones. To compute the free energyprofile along the reaction coordinate the stationary flux vectorqstat has to be computed. The stationary flux vector is theeigenvector of the transition kernel K with eigenvalue 1.

K ·qstat = qstat (3)

The equilibrium probability distribution Peq at the milestonesis obtained from stationary flux vector by element wise multi-plication of qstat with the lifetime vector T.

Peq,i = qstat,iT i (4)

From the equilibrium probability distribution the Free energy∆Gi at milestone i can be calculated as

∆Gi =−kBT ln(

Peq,i

Peq,0

)(5)

where Peq,0 is the probability corresponding to a reference freeenergy.

To calculate the mean first passage time (MFPT) τ , an ab-sorbing boundary condition has to be set at the last milestone,the target state31. The milestone at the endpoint works as asink and transition probability from that milestone to any othermilestone is set to zero. So we define a new square matrix Kwith the following property

Ki j = 0; if i≥ m, j = i±1= Ki j; otherwise

(6)

where milestone m is referring to the final state of the problemin hand. It is possible that there are milestones after m and Ki jwill be zero for them as well. Now the MFPT is calculated bythe following expression

τ = p0(I− K)−1T (7)

The derivation of the Equation 7 is presented in Ref. 31. Thep0 is the probability distribution at each milestone at the be-ginning. For example, if we want to study the transition frommilestone 2 to milestone 5 for a 5 milestone system we wantthe initial probability to be at milestone 2 is 1 and zero else-where. So p0 = (0,1,0,0,0).

Recently Grazioli and Andricioaei have shown that timecorrelation functions can also be calculated by using the tran-sition kernel and the first passage time distribution betweenmilestones75. For that, a move from milestone i to j is pro-posed with probability pi→ j where

pi→ j =Ki j

∑ j∈i−1,i+1Ki j(8)

The time taken for the move is a random number sampledfrom the first passage time distribution between those givenmilestones. This process is repeated many times to generate atrajectory in the milestone space

X = (tn,xi) | n ∈ [1,N]; i ∈ [1,M] (9)

4

where M is the number of milestones, N is the total number ofsteps and xi is the value of the coordinate corresponding to themilestone i, where the system is in at step n. This method isinspired by the kinetic Monte Carlo (KMC) scheme suggestedby Voter77. The time tn is defined as

tn =N

∑k=1

tsk (10)

where tsk is the sampled first passage time for kth move. This

long trajectory X can be extended much beyond the timescalesachievable from conventional MD simulation. There are twoways to calculate the time correlation function X. One is toconstruct a time dependent conditional probability distribu-tion function Pi(t | x0) by interpolating the sparse time trajec-tory X for all milestones. Here i ∈ [1,M] and x0 is the initialposition. From this distribution time correlation function canbe estimated by evaluating the following expression75

C(t) =M

∑i=1

(A(xi)Pi(∞)

M

∑s=1

A(xs)Ps(t | x0)

)(11)

where A is the observable, xi is the position at milestone i andP(∞) is the stationary probability distribution obtained fromEquation 4.

The other option is to interpolate the long trajectory X to getpositions and times in between milestones to produce anothertrajectory of the same length but with much higher resolutionin time and space. From this trajectory the time correlationcan be calculated by conventional time averaging

C(t) =1T

∫ T

0A(x(t ′))A(x(t ′+ t))dt ′ (12)

Grazioli and Andricioaei75 have shown that the latter methodperforms better in simple 1D model systems. They appliedit to Alanine dipeptide to yield bond vector time correlationfunction using Lipari-Szabo formalism78 in very good agree-ment with the exact results. But all their results involved onelong brute-force Langevin trajectory to begin with, which theydiscretized into the milestone space. They did not implementtheir technique for short milestone to milestone trajectories.

B. Weighted ensemble (WE)

Weighted ensemble simulation is a statistically exact pathsampling strategy introduced by Huber and Kim26 and laterstudied in detail by Zuckerman and coworkers27,28,76. Thedetails of the weighted ensemble simulation are presented inRef. 27. A brief discussion regarding the portion relevant toour work is given below.

In WE simulation the configurational space is divided intomultiple (say M) bins. A certain number of trajectories (sayN) are started from the initial state. Each trajectory has weight1/N in the first iteration. The position is monitored at a giventime interval δ t. After each time interval any trajectory reach-ing a new bin is stopped and some new trajectories are gener-ated from its endpoint. The weight of the old (parent) trajec-

tory is equally distributed amount the newly created (daugh-ter) trajectories. This process is continued so that every occu-pied bin will contain exact N trajectories. If the number in-crease beyond N, the excess ones are killed and their weightsare added to the surviving trajectories. Thus the total proba-bility remains conserved. The philosophy of the WE simula-tion is that the number of trajectories gets increased and theweights get reduced as we go further from the starting state.Some of this trajectories with very low weight might eventu-ally reach the target state. From these weights it is possibleto estimate the rate and mean first passage time. For exam-ple if a trajectory with weight 10−6 reaches the target state atcertain time t then it is possible to suggest that in time du-ration t, approximately 1 out of 106 trajectories reaches thattarget state without propagating all of them during the entirecourse of time. To maintain a steady state of probability flowthe trajectories reaching the target state are re-initiated fromthe starting state. Rate constant k and mean first passage time(MFPT) can be estimated by the Hill relation:76

k =1

MFPT(A→ B)= flux(SS,A→ B) =

∑i wi

t(13)

where SS refers to steady state, A and B are the initial andtarget states respectively, wi is the weight of the ith reactivetrajectory and t is the simulation time.

C. Combined weighted ensemble milestoning (WEM)

In WEM method we blend milestoning and WE methodsinto one so that convergence of milestone to milestone transi-tion probabilities and timescales can be achieved with reducedcomputational effort. We divide the space between milestonesinto bins and apply the WE technique of producing and killingnew trajectories. The milestone in question (say i) becomesour starting state of the WE simulation and milestone i−1 andi+ 1 becomes two target states. A schematic of our methodis shown in Figure 1. In spite of the apparent similarity be-tween WEM method and the adaptive weighted ensemble pro-cedure (aWEP) by Bhatt and Bahar79, they are fundamentallyvery different. Unlike aWEP method, WEM utilizes the back-ground of milestoning to calculate free energy profile and timecorrelation function. Also the primary aim of WEM is not toobtain equilibrium rate constant in between milestones but tocalculate the transition kernel K, the lifetime vector T, andthe first passage time distribution between each pair of mile-stones. So, we made some modifications in the weighted en-semble scheme.

• The trajectories reaching either of the adjacent mile-stones are killed.

• No new trajectories are generated when a trajectoryreaches a target milestone, because we do not enforcea steady state in our simulation. This is typical for con-ventional milestoning.

• The average lifetime of milestone i is estimated by

T i = ∑k

tkwk (14)

5

ii-1 i+1

FIG. 1. A schematic diagram of combined weighted ensemble mile-stoning (WEM) method. The thick vertical lines are milestones andthe dotted lines are WE bin boundaries. The trajectory in red is thestarting trajectory which branches out into daughter trajectories uponcrossing WE bin boundaries. The blue and green trajectories haveweights 1/2 and 1/4 of the red one respectively. According to thisfigure, Ki i+1 =

38 and Ki i−1 =

12 .

where wk and tk are, respectively, the weight and timeat which the kth trajectory that reaches either of mile-stones i−1 or i+1.

• The elements of K are given by

Ki j = ∑k∈Γ(i→ j)

wk; j = i±1

= 0; otherwise(15)

where Γ(i→ j) is the set of trajectories starting at mile-stone i and ending at j. If all the trajectories reachedeither of the milestones Ki i+1 +Ki i−1 = 1. This canbe used as a check for convergence for each milestone.Yet, if the milestone i is placed in a deep free energyminima, this condition might not be satisfied. So, wealternatively define

flux(i→ j) = ∑k∈Γ(i→ j)

(wk

tk

)(16)

If this flux gets converged to a specific value as timeprogresses, we say that the simulation for a particularpair of milestones is converged. We have used this cri-teria throughout our work to monitor convergence. Thiscriteria is very general and applicable to any milestoneirrespective of its position.

• The first passage time distribution (FPT Di→ j(t)) is cal-culated as a function of time by summing over theweights of the trajectories arrived at j from i within thetime interval of t and t +δ t.

The transition kernel, lifetime vector and first passage timedistribution calculated in this manner are used in the mileston-ing theory as described in Section II A to elucidate the MFPT

τ , stationary probability distribution aka free energy profile,and time correlation function for the overall process.

III. RESULTS

A. One Dimensional Double Well potential

We tested our WEM method on a 1D double well potential.The chosen potential was of the following form:

V (x) = c(1− x2)2 (17)

where c is a parameter which can be varied to control the bar-rier height (Figure 2). The functional form of the potential isinspired by earlier studies in milestoning19,75 and weightedensemble simulations28. We have used kBT as the unit ofenergy, so the barrier height is ckBT in Equation 17. Wehave performed WEM simulations for three different barrierheights: 0.5 kBT , 1.0 kBT and 2.0 kBT . Two different mile-stoning schemes with 5 and 9 milestones has been tested (Fig-ure 2). The milestones were placed between x = −2.0 andx = 2.0 at ∆x = 1.0 interval for the former and ∆x = 0.5 in-terval for the latter. We studied the transitions between thetwo minima situated at x = ±1. Classical trajectories werepropagated using over-damped Langevin dynamics27:

x(t +∆t) = x(t)− kBTmγ

(dVdx

)∆t +∆xR (18)

Time step ∆t is chosen to be unity. The frictional coef-ficient γ determines the variance of random Gaussian noise∆xR through fluctuation dissipation theorem by the followingexpression27

σ2 = 2

kBTmγ

∆t (19)

The value of γ was chosen to be 2000 which resulted inσ2 = 0.001. Typical values and propagation algorithm wastaken from Ref. 27. Multiple Langevin trajectories startingfrom x = −1.0 and ending at x = 1.0 were used to calcu-late mean first passage time which is compared with that fromother milestoning and weighted ensemble simulations. Addi-tionally, a long trajectory of 106 steps has been propagated tocalculate the time correlation function for each of the differentbarrier heights.

Weighted ensemble (WE) simulations have been performedusing the WESTPA package55. The x space has been dis-cretized into bins of width δx = 0.1. The position of the binsare depicted in Figure 2 with dotted lines. WE trajectorieswere started from x =−1.0 and ended at x = 1.0. The trajec-tories were stopped when they reach the target state i.e x = 1.N = 10 trajectories were run in each bin for barrier height0.5kBT and 20 for the other two systems. New trajectorieswere initiated or killed at a time interval of δ t = 20 time-steps.Total 2000 iterations of time interval δ t has been performed.First passage times were calculated by recording the time stepat which the trajectories reach the end-point.

6

2 1 0 1 2x

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0V(

x)/k

BT

c = 0.5c = 1.0c = 2.0

2.0 1.5 1.0 0.5 0.0 0.5 1.0 1.5 2.0x

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

V(x)

/kBT

c = 0.5c = 1.0c = 2.0

FIG. 2. The 1D double well systems with different barrier heightsand milestone configurations. The positions of the milestones aredepicted by solid black lines and the edges of the WE bins are de-picted by black dotted lines. (top) 5 milestone system; (bottom) 9milestone system

In combined weighted-ensemble milestoning (WEM)method, the same δx and δ t values were used as of normalWE simulations. A maximum of 20 trajectories were run foreach bin which was sufficient to give converged results for allthe double well systems. The results were analyzed using thew_ipa module of WESTPA package to elucidate the lifetimeof each milestone and the transition probabilities to the nearestmilestones.

Conventional milestoning simulations were also performedto make a comparison with WEM results. The positioning ofmilestones, number of trajectories, and simulation timescaleswere kept the same as that of the WEM scheme.

Three different observable quantities were calculated fromthese simulations: the mean first passage time (MFPT), thestationary distribution along the milestone space and positionauto-correlation function C(t) = 〈x(0)x(t)〉. MFPT’s are di-rectly related to the rate constant of the transition betweenthe two minima. The stationary distribution can be used tocalculate the free energy profile along the chosen coordinate.On the other hand, the time correlation function is the central

quantity in the non-equilibrium thermodynamics that denotesthe decay of memory in a system. The mean first passagetimes for different barrier heights have been computed frommilestoning and WEM simulations using Equation 7. The re-sults have been depicted in Table I. All error bars are com-puted from three sets of independent WEM, WE or mileston-ing simulation unless mentioned otherwise.

The results of the combined weighted ensemble mileston-ing (WEM) are in reasonable agreement with the results fromconventional Langevin dynamics simulation, WE simulationand the conventional milestoning simulation for different bar-rier heights. But the WEM simulations require significantlyless simulation steps i.e. total number of force evaluationsto get converged result (Table II). Although the accelerationcompared to traditional WE simulation is not pronounced sig-nificantly for this simple 1D system, converged results couldbe obtained with 2-10 times less simulation steps. It shouldalso be noted that the simulations starting from different mile-stones are independent and can be run in parallel resultingin significant reduction in wall clock time. One should notcompare the number of total force evaluations performed forconventional milestoning and WEM simulations because con-ventional milestoning simulation failed to sample some of theenergetically unfavorable transitions, particularly those at theboundaries. We tried increasing the number of walkers andtotal simulation time by 1-2 orders of magnitude but we couldnot see any transition to very high energy milestones. In or-der to converge statistics for such transitions we may needmany orders of magnitude more simulation time or numberof walkers which have been avoided because it is not veryrelevant for our study. Because of this under-sampling issuewe did not perform kinetic Monte-Carlo simulation from con-ventional milestoning results for calculating time correlationfunction.

The exact Boltzmann stationary distribution of probabilityalong x was compared with that computed from WEM simu-lation in Figure 3. The Boltzmann probability has been calcu-lated only at the 9 milestone points (x=−2.0+0.5i, i∈ [0,8]).The results of WEM simulation agrees very well with the ex-act stationary distribution. The apparent discrepancy for theWEM 5 milestone case resulted due to the normalization oftotal probability over the milestones:

M

∑i=1

Peq(xi)∆x = 1 (20)

If the total number of milestones changes, the normalizedprobability at a given milestone also changes.

Although better results can be expected with more num-ber of milestones, very closely spaced interfaces will imposeerror in the calculation as the memory of previously visitedmilestones will not be completely lost. This trade off shouldbe taken into consideration for biological application of thismethod.

We implemented the interpolation based method for calcu-lating time correlation function using Equation 12 in SectionII A for the short milestone to milestone trajectories propa-gated using weighted ensemble scheme. We compared our

7

TABLE I. First passage times in (×103 steps) for different simulation scheme.

Barrier height (kBT ) WEM 5 milestone WEM 9 milestone Regular 5 milestone Regular 9 milestone WE Regular Langevina

0.5 7.5±0.7 7.2±0.8 6.8±0.1 7.3±0.2 7.0±0.1 6.7±5.81.0 7.0±0.4 7.3±0.4 7.3±0.3 7.8±0.4 7.5±0.1 7.2±6.22.0 9.6±0.3 13.7±1.7 9.4±0.1 10.3±1.5 10.0±0.1 10.4±9.7

a Error bars for regular Langevin dynamics results are standard deviation of many transition events observed in long Langevin dynamics simulation.

TABLE II. Total number of simulation steps (Number of trajectories × time steps × number of milestones) required to get converged result(×106 time steps)

Barrier height (kBT ) WEM 5 milestone WEM 9 milestone Regular 5 milestone Regular 9 milestone WE Regular Langevin0.5 15.30 4.86 16.72 23.77 30.66 103

1.0 18.20 4.82 16.72 23.77 56.28 103

2.0 16.13 5.10 16.72 23.77 49.56 103

results with a long brute-force over-damped Langevin dynam-ics simulation of 106 steps for all different barrier heights. Alltrajectories were propagated long enough to obtain convergedresult for the time correlation function. Position-positiontime correlations functions C(t) = 〈x(0)x(t)〉 calculated fromWEM simulations with two different milestone configurationhas been shown in Figure 4. For lower barrier heights e.g.0.5kBT and 1.0kBT , the 〈x(0)x(t)〉 obtained from both mile-stone configurations closely resembles that from the long con-ventional Langevin dynamics simulation. For barrier height2.0kBT the C(t) obtained from 5 milestones did not showquantitative agreement with Langevin dynamics result unlikethe 9 milestone case. This is not surprising since Grazioli andAndricioaei have shown that the accuracy of C(t) increaseswith increasing number of milestones75.

The results obtained for the 1D double well potential servesas a proof of concept for WEM method. We could show thatit can reproduce the MFPT, stationary distribution and timecorrelation function with reasonable accuracy. The compu-tational gain is not evident from our results for 1D potentialbecause there is no additional degree of freedom. In atomisticmolecular dynamics simulation there are many coupled de-grees of freedom which the system will sample before transi-tioning to a new state along the reaction coordinate. In SectionIII B and III C we show that our method can be successfullyapplied in such scenarios as well.

B. (10+1) Dimensional Coupled Potential

In order to study the effect of additional coupled degreesof freedom on the reaction coordinate, we have tested WEMmethod on a (10+1) dimensional potential19 where the onedimensional reaction coordinate (x) is coupled with 10 lowbarrier double well potentials (y1,y2, ...y10). The form of thepotential function is given by:

V (x,y1,y2, ...y10) = (1− x2)2− 12

10

∑n=1

y2nx2 +

10

∑n=1

y4n (21)

The projection of V in a 2D plane is depicted in the Fig-

ure 5. Standard weighted ensemble (WE) simulations wereperformed without milestones with δ t = 500 time steps for1000 iterations. The WE bins were chosen to be of width δx= 0.1 and trajectories were regenerated or stopped keeping 5trajectories per occupied bin. Trajectories were initiated atx = −1.0 and stopped when they reached x = 1.0. For WEMsimulations, milestones have been placed at x = −2.0, −1.0,−0.5, 0.0, 0.5, 1.0, 2.0 for 7 milestone case and two additionalmilestones at x = ±1.5 for the 9 milestone case. WE trajec-tories were propagated in between the milestones accordingto Equation 18. Each WEM iteration involved δ t = 20 timesteps. All other parameters were same as what was used for1D double well potential.

The transition matrix K and lifetime vector T was com-puted from the WEM simulation to obtain the equilibriumprobability distribution in the milestone space. The free en-ergy profile was computed from the probability distributionusing equation 5. The results of both seven and nine milestonescheme are well within 1 kcal/mol agreement with that ob-tained from long regular Langevin dynamics simulation (Fig-ure 6). The results tend to improve with increasing number ofmilestones.

Position-position time correlation function of reaction co-ordinate 〈x(0)x(t)〉 was computed from both WEM simula-tion and regular LD simulation and the results agree with eachother (Figure 6). The MFPT of transition from x=−1 to x= 1obtained from WEM simulation is also in agreement with thatobtained from WE and regular Langevin dynamics (Table III).These results indicate that our method can be extended to mul-tidimensional systems with many degrees of freedom coupledto the reaction coordinate and experimental observables canbe calculated in significantly less computational effort.

C. Alanine Dipeptide

To test the applicability of our method on biologicallyrelevant systems we studied the transition between αRand C7eq conformation in an artificially stiffened Alaninedipeptide2,33. The 22 atom small molecule has been mod-

8

2.0 1.5 1.0 0.5 0.0 0.5 1.0 1.5 2.0x

0.0

0.2

0.4

0.6

0.8

1.0

P eq(

x)

Barrier = 0.5 kBT

ExactWEM 5 milestonesWEM 9 milestones

2.0 1.5 1.0 0.5 0.0 0.5 1.0 1.5 2.0x

0.0

0.2

0.4

0.6

0.8

1.0

P eq(

x)

Barrier = 1.0 kBT

2.0 1.5 1.0 0.5 0.0 0.5 1.0 1.5 2.0x

0.0

0.2

0.4

0.6

0.8

1.0

P eq(

x)

Barrier = 2.0 kBT

FIG. 3. Comparison of the stationary probability distribution of 1Ddouble well systems, calculated from WEM simulations and exactBoltzmann probability. Error bars are obtained from 3 sets of inde-pendent WEM simulation.

eled by CHARMM36 force field80 in Generalized Born Im-plicit solvent (GBIS) environment81,82. Standard transitiontimescales and free energy surface as a function of the back-bone dihedral angles φ and ψ has been computed from a long1 µs simulation. All MD simulations were performed us-ing NAMD 2.1273 package. Newtonian equations of motionwere integrated with a 2 fs time-step with SHAKE algorithmto constrain the bond lengths. We have artificially stiffenedthe backbone dihedral angle ψ by applying harmonic wallsto avoid the periodicity of the collective variable. Regular

0 2000 4000 6000 8000 10000timesteps (t)

0.0

0.2

0.4

0.6

0.8

1.0

<x(0

)x(t)

>

Barrier = 0.5 kBT

WEM 9 MilestonesWEM 5 MilestonesRegular LD

0 2000 4000 6000 8000 10000timesteps (t)

0.0

0.2

0.4

0.6

0.8

1.0

<x(0

)x(t)

>

Barrier = 1.0 kBT

0 2000 4000 6000 8000 10000timesteps (t)

0.0

0.2

0.4

0.6

0.8

1.0

<x(0

)x(t)

>

Barrier = 2.0 kBT

FIG. 4. Comparison of the position-position time correlation func-tions of 1D double well systems calculated from WEM simulationsand regular Langevin dynamics (LD) simulation

WE and WEM simulations were performed using WESTPA55

package. The ψ space has been divided into bins of width of10 degrees for WE simulation. The temperature has been keptconstant at 300 K using Langevin thermostat with a damp-ing constant γ = 80 ps−1 which corresponds to water-likeviscosity57. The use of a stochastic thermostat is necessaryto make sure that the dynamics is not fully deterministic. TheWE simulation was performed for 500 iterations of δ t = 20ps. The starting state was defined by ψ = −60 (αR) and thefinal state is defined at ψ = 150 (C7eq). The φ dihedral anglewas constrained between −180 and 0 by a harmonic wall

9

2.5 2.0 1.5 1.0 0.5 0.0 0.5 1.0 1.5 2.0 2.5x

2.0

1.5

1.0

0.5

0.0

0.5

1.0

1.5

2.0y

0

1

2

3

4

5

V(x,

y)/k

BT

FIG. 5. A 2D projection of the (10+1)D coupled potential used in thisstudy. In two dimension the potential function is given by V (x,y) =(1− x2)2− 0.5x2y2 + y4. The black solid lines indicate milestonesand the dotted lines indicate WE bin boundaries. The two additionalmilestones for 9 milestone case (at x = ±1.5) are not shown in thisfigure.

2.0 1.5 1.0 0.5 0.0 0.5 1.0 1.5 2.0x

0

1

2

3

4

5

Free

Ene

rgy

(kca

l/mol

)

Regular LDWEM 7 MilestonesWEM 9 Milestones

0 20000 40000 60000 80000 100000timesteps (t)

0.0

0.2

0.4

0.6

0.8

1.0

<x(0

)x(t)

>

Regular LDWEM 7 MilestonesWEM 9 Milestones

FIG. 6. Free energy profile and auto-correlation function along xfrom the WEM scheme and from long regular Langevin dynamics(LD) simulation for (10+1) dimensional coupled potential. The errorbars are computed from 3 independent WEM simulations.

TABLE III. Mean first passage times (MFPT) for different simu-lation scheme for (10+1) dimensional coupled potential. Error barsare obtained from 3 independent sets of simulation unless mentionedotherwise. Total simulation times required (including all walkers) toobtain converged MFPT results have also been indicated.

Simulation scheme MFPT Total simulation time(×103 time steps) (×106 time steps)

Regular LD 105.2±104.8a 10.0WE 93.5±6.7 16.1

WEM 7 milestone 87.4±0.4 1.9WEM 9 milestone 113.3±56.9 1.1

a Error bar is standard deviation of multiple transition events observed inone long LD trajectory

180 120 60 0 60( )

180

100

60

20

20

60

100

150180

()

R

C7eq

0

1

2

3

4

5

Free

Ene

rgy

(kca

l/mol

)

FIG. 7. Free energy profile of the artificially stiffened Alanine dipep-tide obtained from 1 µs long conventional MD simulation. The po-sition of the milestones are depicted by solid horizontal lines and theedges of the WE bins are shown by dotted lines.

constraint of 0.04 kcal/mol deg−2.For WEM simulations 8 milestones were placed at the fol-

lowing ψ angles: -100, -60, -20, 20, 60, 100, 150 and180. For each of the milestone the starting point of the simu-lation is generated by performing a 100 ps equilibration sim-ulation starting from the energy minimized structure. Duringequilibration the ψ angle was constrained at the specific valueof the milestone by a force constant of 0.12 kcal/mol deg −2.The same bins and γ as WE simulation were used. WE trajec-tories were initiated from the final structure of the 100 ps sim-ulation and propagated until it’s daughter trajectories reachedeither of the nearby milestone. An iteration time δ t of 0.2 pswas used.

The mean first passage times for transition between αR andC7eq states have been summarized in Table IV for all threemethods. The total simulation times required to obtain con-verged results have also been depicted (Table IV). The MFPTand its standard deviation from conventional MD simulationshows very similar values indicating a Poisson distribution oftimescales of barrier crossing event. This Poisson type be-havior agrees with previous work by Salvalaglio et al.83 As

10

TABLE IV. Mean first passage times (MFPT) and computationalcosts for obtaining converged results from different simulationschemes for Alanine dipeptide. Error bars are computed from 3 in-dependent sets of simulation unless specified otherwise.

Simulation MFPT (ns) MFPT(ns) Total simulationscheme (αR→C7eq) (C7eq→ αR) time (ns)

Brute Force MD 2.4±2.7a 1.2±1.2 1000WE 2.1±0.2 - 545

WEM 3.6±1.1 0.7±0.2 11.6

a Error bar is standard deviation over many transition events observed froma long trajectory

mentioned in Section III A, the computational gain is clearlypronounced for a molecular system with many degrees of free-dom. The total simulation time for all walkers for WEM sim-ulation is ∼30 times less than a conventional WE calculation.Moreover, the WEM method can be parallelised over mile-stones. The longest time required to get converged transi-tion probability and lifetime on a single milestone is about3-4 ns. So, provided the availability of the parallel computingresources, one effectively spends two orders of magnitude lesswall clock wall clock time for WEM simulation than a tradi-tional WE or MD simulation. The mean first passage timeof αR to C7eq transition is in reasonable agreement with theresults from long MD and WE simulations (Table IV).

In traditional WE method we need to specify a startingand a target state. It prevents us from obtaining the ratesof the backward process without performing a second set ofcalculation. As an additional advantage over traditional WEmethod, the mean first passage time of the backward transition(C7eq → αR) can also be estimated by using the transpose oftransition kernel KT and T′ in place of K and T, respectively,in Equation 7, where

T ′i = T M−i; i ∈ [1,M] (22)

where M is the number of milestones. The predictedtimescales of the C7eq→ αR are in order of magnitude agree-ment with the result obtained from 1 µs long, regular classicalMD simulation.

Moreover, about 4.5 ns simulation time was spent in the lasttwo milestones (ψ = 150.0 and 180.0). If calculating therate constant is the only objective, these calculations are notnecessary, because the transition probabilities from the targetmilestone and beyond does not appear the modified transitionkernel K in Equation 6. But for the calculation of the free en-ergy profile and time correlation function, transition probabil-ities and lifetimes at these milestones are required. Also, 100ps of additional equilibration was performed to sample initialequilibrium configurations at each milestone. These addedupto 800 ps more simulation time to that reported for WEMsimulations in Table IV.

The stationary probability distribution in milestone space iscalculated and the values have been used in Equation 5 to cal-culate the free energy profile. The reference (P0

eq) was chosento be the probability of the most populated milestone. Thefree energy profile from WEM is in quantitative agreement

100 60 20 20 60 100 150 180( )

0

1

2

3

4

5

Free

Ene

rgy

(kca

l/mol

) WEM 8 Milestones1 s Molecular Dynamics

0.00 0.25 0.50 0.75 1.00 1.25 1.50 1.75 2.00time (ns)

0.0

0.2

0.4

0.6

0.8

1.0

<(0

)(t)

>

WEM 8 Milestones1 s Molecular Dynamics

FIG. 8. Comparison of the Free energy profile along ψ (top) andthe time correlation function 〈ψ(0)ψ(t)〉 (bottom) calculated fromWEM simulation with 8 milestones and long conventional MD sim-ulation. The error bars are computed from 3 independent WEM sim-ulations.

with the one computed from long unbiased MD simulation(Figure 8). It could also predict the barrier height to be 2.9kcal/mol which is very close to the value obtained from theconventional MD simulation (3.3 kcal/mol). Also, the auto-correlation function of the ψ dihedral angle (〈ψ(0)ψ(t)〉) wascomputed as function of time and and compared with the re-sults of the 1 µs MD simulation. The results agree very wellwith each other (Figure 8). The convergence of the First Pas-sage Time Distribution (FPT D(t)) for one of the milestonesis depicted in Figure 9. The milestone space trajectory gener-ated from the transition statistics, and the interpolated trajec-tory used for the calculation on the time correlation functionis shown in Figure 9.

IV. DISCUSSIONS AND CONCLUSIONS

In this paper we have developed and tested combinedweighted ensemble milestoning (WEM) method for calculat-ing kinetic and thermodynamic properties of barrier crossingevents. We have tested our model for a 1D double well model

11

0.0 2.5 5.0 7.5 10.0 12.5 15.0 17.5 20.0time (ps)

0.00

0.02

0.04

0.06

0.08

0.10

0.12FP

TD(t)

milestone 4 to 5milestone 4 to 3

0 2000 4000 6000 8000 10000time (ps)

10060202060

100

150180

()

Interpolated trajectoryTrajectory in milestone space

FIG. 9. (Top) Convergence of the First Passage Time Distributionfunction from milestone 4 (ψ = 20) to milestone 3 (ψ = −20)and milestone 5 (ψ = 60). (Bottom) The milestone to milestonetrajectory and the interpolated continuous trajectory generated fromthe first passage time distribution between milestones (as describedin Section II A)

.

with different barrier heights, a 11D potential with 10 degreesof freedom coupled to reaction coordinate (RC), and an atom-istic model of Alanine dipetide. We have computed stationaryprobability distribution, free energy profile along RC, meanfirst passage time and time correlation function for these sys-tems. The WEM simulations were able to reproduce the re-sults of regular MD or Langevin dynamics, conventional mile-stoning, and weighted ensemble simulations within a smallfraction of the computing time. Also the possibility of paral-lelizing the calculation over different milestones reduces thewall clock time by two orders of magnitude compared to regu-lar MD or WE simulation. Besides, WEM method allows thecalculation of the mean first passage time and rate for the re-verse process which is not directly accessible in conventionalWE calculation.

Unlike the WARM method proposed by Grazioli andAndricioaei19, WEM does not require application of biasingforces. The unidirectional wind forces, used in WARM, can

increase sampling in the forward direction, but, on the otherhand, decrease sampling on the backward direction. For ex-ample, applying a biasing force along +x direction for our 1Dmodel will cause more trajectories to reach milestone i + 1from milestone i with very less number of trajectories reach-ing milestone i− 1, resulting in significant reduction of thestatistics for i→ i− 1 transition. For calculating observableproperties like free energy and kinetics from milestoning, itis necessary to properly sample both back and forth transi-tions. Our WEM method does not create any directional biasand increases sampling in all directions. This is a significantimprovement over the WARM technique.

Grazioli and Andricioaei have also developed the techniquefor calculating time correlation function by decomposing asingle long trajectory into milestones75. We have extendedtheir technique for short milestone to milestone trajectoriesand showed the applicability of their method for enhancedmilestoning simulations. Pure WE simulation can not repro-duce time correlation functions very well because of the cor-related nature of the trajectories as the daughter trajectoriesfrom the same parent have the exact same evolution history76.We showed that it is possible to recover the correct time cor-relation functions from WE simulation using milestoning.

A typical problem for milestoning simulation is the opti-mal placement of the milestones. If they are placed too closeto each other, the transition times might be shorter than therelaxation timescales. This causes preservation of the mem-ory of previously visited milestones and the master equationbased formalism becomes no longer applicable. To avoid this,spacing between milestones need to be increased so that theirtransition statistics are independent. But it leads to signifi-cant increase in the length of the trajectories and computa-tional cost. WEM method decreases the computational costof simulating transitions between such distant milestones byperforming weighted ensemble simulations in stead of tradi-tional molecular dynamics.

In conventional milestoning scheme, one needs to performan equilibrium simulation on each milestone constraining thereaction coordinate and allowing sampling of the other or-thogonal degrees of freedom. Then, a large number of trajec-tories (usually of the order of 102− 103) are to be generatedfrom different starting points sampled from that equilibriumtrajectory. But WEM method does not require one to startmany trajectories from each milestone. As one WE trajec-tory splits into many trajectories with smaller weights, one ora few starting trajectories are sufficient to reasonably samplethe space in between milestones and also the transition prob-abilities. So starting points can be obtained from a very shortconstrained equilibration at each milestone. All our calcula-tions were performed with just one starting point from eachmilestone, but a few more might be required for problemswith more complex energy landscape like the ones involved inprotein folding. However, the number of starting trajectoriesrequired is significantly less compared to regular milestoning,bringing enormous simplicity in the simulation workflow.

Our method does not facilitate the choice of optimal RCwhich itself is an extremely involved exercise in biomolecularsystems. But once a good order parameter space is known,

12

it can help in obtaining reach physics by spending very smallamount of computational effort.

In summary, we proposed a combination of weighted en-semble and milestoning method which is superior to ei-ther of the individual techniques both in computational effi-ciency and in versatility of applications, by sharing the ad-vantages and mitigating the deficiencies of each other. Wehave implemented our method in WESTPA toolkit84 throughNAMD 2.12 molecular simulation package73. Our imple-mentation can be extended to higher dimensional milestonesthrough Voronoi bins as suggested by Vanden-Eijnden andcoworkers67. We hope WEM method will find applicationin computational biophysics in deciphering thermodynamicsand kinetics of complex and challenging processes, and it willhelp discovering new physics about biomolecular systems infuture.

V. ACKNOWLEDGEMENTS

The authors thank Trevor Gokey for his help in generatingthe scripts for analyzing WEM results. The authors are grate-ful to Moises Romero, Shane Flynn, Bridgett Kohno, BrianNgyuen, and Saswata Roy for their valuable comments andsuggestions. The authors thank UC Irvine High PerformanceComputing (HPC) cluster for computational resources.

REFERENCES

1S. J. Klippenstein, V. S. Pande, and D. G. Truhlar, “Chemical Kineticsand Mechanisms of Complex Systems: A Perspective on Recent Theoret-ical Advances,” Journal of the American Chemical Society 136, 528–546(2014).

2A. T. Frank and I. Andricioaei, “Reaction Coordinate-Free Approach toRecovering Kinetics from Potential-Scaled Simulations: Application ofKramers’ Rate Theory,” The Journal of Physical Chemistry B 120, 8600–8605 (2016).

3M. C. Zwier and L. T. Chong, “Reaching biological timescales with all-atom molecular dynamics simulations,” Current Opinion in Pharmacology10, 745–752 (2010).

4L. W. Votapka, B. R. Jagger, A. L. Heyneman, and R. E. Amaro, “SEEKR:Simulation Enabled Estimation of Kinetic Rates, A Computational Tool toEstimate Molecular Kinetics and Its Application to Trypsin–BenzamidineBinding,” The Journal of Physical Chemistry B 121, 3597–3606 (2017).

5C. W. Hopkins, S. Le Grand, R. C. Walker, and A. E. Roitberg, “Long-Time-Step Molecular Dynamics through Hydrogen Mass Repartitioning,”Journal of Chemical Theory and Computation 11, 1864–1874 (2015).

6D. E. Shaw, J. C. Chao, M. P. Eastwood, J. Gagliardo, J. P. Grossman, C. R.Ho, D. J. Lerardi, I. Kolossváry, J. L. Klepeis, T. Layman, C. McLeavey,M. M. Deneroff, M. A. Moraes, R. Mueller, E. C. Priest, Y. Shan, J. Spen-gler, M. Theobald, B. Towles, S. C. Wang, R. O. Dror, J. S. Kuskin, R. H.Larson, J. K. Salmon, C. Young, B. Batson, and K. J. Bowers, “Anton, aspecial-purpose machine for molecular dynamics simulation,” Communi-cations of the ACM 51, 91 (2008).

7I. Buch, M. J. Harvey, T. Giorgino, D. P. Anderson, and G. De Fabritiis,“High-Throughput All-Atom Molecular Dynamics Simulations Using Dis-tributed Computing,” Journal of Chemical Information and Modeling 50,397–403 (2010).

8I. Buch, T. Giorgino, and G. De Fabritiis, “Complete reconstruction ofan enzyme-inhibitor binding process by molecular dynamics simulations.”Proceedings of the National Academy of Sciences of the United States ofAmerica 108, 10184–9 (2011).

9R. C. Bernardi, M. C. Melo, and K. Schulten, “Enhanced sampling tech-niques in molecular dynamics simulations of biological systems,” Biochim-ica et Biophysica Acta (BBA) - General Subjects 1850, 872–877 (2015).

10B. Bagchi, Statistical Mechanics for Chemistry and Materials Science(CRC Press, 2018).

11G. Torrie and J. Valleau, “Nonphysical sampling distributions in MonteCarlo free-energy estimation: Umbrella sampling,” Journal of Computa-tional Physics 23, 187–199 (1977).

12A. Laio and M. Parrinello, “Escaping free-energy minima.” Proceedingsof the National Academy of Sciences of the United States of America 99,12562–6 (2002).

13E. Darve, D. Rodríguez-Gómez, and A. Pohorille, “Adaptive biasing forcemethod for scalar and vector free energy calculations,” The Journal ofChemical Physics 128, 144120 (2008).

14S. Park, F. Khalili-Araghi, E. Tajkhorshid, and K. Schulten, “Free energycalculation from steered molecular dynamics simulations using Jarzynski’sequality,” The Journal of Chemical Physics 119, 3559–3566 (2003).

15P. Tiwary and M. Parrinello, “From Metadynamics to Dynamics,” PhysicalReview Letters 111, 230602 (2013).

16Y. Wang, O. Valsson, P. Tiwary, M. Parrinello, and K. Lindorff-Larsen,“Frequency adaptive metadynamics for the calculation of rare-event kinet-ics,” The Journal of Chemical Physics 149, 072309 (2018).

17U. Doshi and D. Hamelberg, “Extracting Realistic Kinetics of Rare Ac-tivated Processes from Accelerated Molecular Dynamics Using Kramers’Theory,” Journal of Chemical Theory and Computation 7, 575–581 (2011).

18I. Deb and A. T. Frank, “Accelerating Rare Dissociative Processes inBiomolecules Using Selectively Scaled MD Simulations,” Journal ofChemical Theory and Computation 15, 5817–5828 (2019).

19G. Grazioli and I. Andricioaei, “Advances in milestoning. I. Enhanced sam-pling via wind-assisted reweighted milestoning (WARM),” The Journal ofChemical Physics 149, 084103 (2018).

20L. T. Chong, A. S. Saglam, and D. M. Zuckerman, “Path-sampling strate-gies for simulating rare events in biomolecular systems,” Current Opinionin Structural Biology 43, 88–94 (2017).

21C. Dellago, P. G. Bolhuis, F. S. Csajka, and D. Chandler, “Transition pathsampling and the calculation of rate constants,” The Journal of ChemicalPhysics 108, 1964–1977 (1998).

22P. G. Bolhuis, D. Chandler, C. Dellago, and P. L. Geissler, “Transition PathSampling : Throwing Ropes Over Rough Mountain Passes, in the Dark,”Annual Review of Physical Chemistry 53, 291–318 (2002).

23T. S. van Erp, D. Moroni, and P. G. Bolhuis, “A novel path samplingmethod for the calculation of rate constants,” The Journal of ChemicalPhysics 118, 7762–7774 (2003).

24R. J. Allen, D. Frenkel, and P. R. ten Wolde, “Forward flux sampling-typeschemes for simulating rare events: Efficiency analysis,” The Journal ofChemical Physics 124, 194111 (2006).

25R. J. Allen, C. Valeriani, and P. Rein ten Wolde, “Forward flux sampling forrare event simulations,” Journal of Physics: Condensed Matter 21, 463102(2009).

26G. Huber and S. Kim, “Weighted-ensemble Brownian dynamics simula-tions for protein association reactions,” Biophysical Journal 70, 97–110(1996).

27D. Bhatt, B. W. Zhang, and D. M. Zuckerman, “Steady-state simulationsusing weighted ensemble path sampling,” The Journal of Chemical Physics133, 014110 (2010).

28B. W. Zhang, D. Jasnow, and D. M. Zuckerman, “The “weighted ensemble”path sampling method is statistically exact for a broad class of stochasticprocesses and binning procedures,” The Journal of Chemical Physics 132,054107 (2010).

29A. K. Faradjian and R. Elber, “Computing time scales from reaction coordi-nates by milestoning,” The Journal of Chemical Physics 120, 10880–10889(2004).

30A. M. A. West, R. Elber, and D. Shalloway, “Extending molecular dynam-ics time scales with milestoning: Example of complex kinetics in a solvatedpeptide,” The Journal of Chemical Physics 126, 145104 (2007).

31J. M. Bello-Rivas and R. Elber, “Exact milestoning,” The Journal of Chem-ical Physics 142, 094102 (2015).

32N. Guttenberg, A. R. Dinner, and J. Weare, “Steered transition path sam-pling,” The Journal of Chemical Physics 136, 234103 (2012).

13

33J. Nummela and I. Andricioaei, “Exact Low-Force Kinetics from High-Force Single-Molecule Unfolding Events,” Biophysical Journal 93, 3373–3381 (2007).

34C. Xing and I. Andricioaei, “On the calculation of time correlation func-tions by potential scaling,” The Journal of Chemical Physics 124, 034110(2006).

35S. X. Sun, “Equilibrium free energies from path sampling of nonequi-librium trajectories,” The Journal of Chemical Physics 118, 5769–5775(2003).

36G. Ozer, E. F. Valeev, S. Quirk, and R. Hernandez, “Adaptive SteeredMolecular Dynamics of the Long-Distance Unfolding of Neuropeptide Y,”Journal of Chemical Theory and Computation 6, 3026–3038 (2010).

37A. Dickson, A. Warmflash, and A. R. Dinner, “Separating forward andbackward pathways in nonequilibrium umbrella sampling,” The Journal ofChemical Physics 131, 154104 (2009).

38A. Warmflash, P. Bhimalapuram, and A. R. Dinner, “Umbrella sampling fornonequilibrium processes,” The Journal of Chemical Physics 127, 154112(2007).

39E. Vanden-Eijnden and M. Venturoli, “Exact rate calculations by trajectoryparallelization and tilting,” The Journal of Chemical Physics 131, 044120(2009).

40X. Xu, S. A. Rice, and A. R. Dinner, “Relation between ordering and shearthinning in colloidal suspensions.” Proceedings of the National Academyof Sciences of the United States of America 110, 3771–6 (2013).

41A. Dickson, A. Warmflash, and A. R. Dinner, “Nonequilibrium umbrellasampling in spaces of many order parameters,” The Journal of ChemicalPhysics 130, 074104 (2009).

42A. Dickson and A. R. Dinner, “Enhanced Sampling of NonequilibriumSteady States,” Annual Review of Physical Chemistry 61, 441–459 (2010).

43A. Dickson, M. Maienschein-Cline, A. Tovo-Dwyer, J. R. Hammond, andA. R. Dinner, “Flow-Dependent Unfolding and Refolding of an RNA byNonequilibrium Umbrella Sampling,” Journal of Chemical Theory andComputation 7, 2710–2720 (2011).

44A. R. Dinner, J. C. Mattingly, J. O. B. Tempkin, B. V. Koten, and J. Weare,“Trajectory Stratification of Stochastic Dynamics,” SIAM Review 60, 909–938 (2018).

45D. M. Zuckerman and T. B. Woolf, “Efficient dynamic importance samplingof rare events in one dimension,” Physical Review E 63, 016702 (2000).

46J. R. Perilla, O. Beckstein, E. J. Denning, and T. B. Woolf, “Computing en-sembles of transitions from stable states: Dynamic importance sampling,”Journal of Computational Chemistry 32, 196–209 (2011).

47E. Vanden-Eijnden and J. Weare, “Rare Event Simulation of Small NoiseDiffusions,” Communications on Pure and Applied Mathematics 65, 1770–1803 (2012).

48J. Hu, A. Ma, and A. R. Dinner, “Bias annealing: A method for obtainingtransition paths <i>de novo</i>,” The Journal of Chemical Physics 125,114101 (2006).

49V. S. Pande, K. Beauchamp, and G. R. Bowman, “Everything you wantedto know about Markov State Models but were afraid to ask,” Methods 52,99–105 (2010).

50G. R. Bowman, K. A. Beauchamp, G. Boxer, and V. S. Pande, “Progressand challenges in the automated construction of Markov state models forfull protein systems,” The Journal of Chemical Physics 131, 124101 (2009).

51F. Noé and S. Fischer, “Transition networks for modeling the kinetics ofconformational change in macromolecules,” Current Opinion in StructuralBiology 18, 154–162 (2008).

52G. R. Bowman, X. Huang, and V. S. Pande, “Using generalized ensemblesimulations and Markov state models to identify conformational states,”Methods 49, 197–201 (2009).

53G. Pérez-Hernández, F. Paul, T. Giorgino, G. De Fabritiis, and F. Noé,“Identification of slow molecular order parameters for Markov model con-struction,” The Journal of Chemical Physics 139, 015102 (2013).

54C. R. Schwantes and V. S. Pande, “Improvements in Markov State ModelConstruction Reveal Many Non-Native Interactions in the Folding ofNTL9,” Journal of Chemical Theory and Computation 9, 2000–2009(2013).

55M. C. Zwier, J. L. Adelman, J. W. Kaus, A. J. Pratt, K. F. Wong, N. B.Rego, E. Suárez, S. Lettieri, D. W. Wang, M. Grabe, D. M. Zuckerman, andL. T. Chong, “WESTPA: An Interoperable, Highly Scalable Software Pack-age for Weighted Ensemble Simulation and Analysis,” Journal of Chemical

Theory and Computation 11, 800–809 (2015).56B. Abdul-Wahid, H. Feng, D. Rajan, R. Costaouec, E. Darve, D. Thain, and

J. A. Izaguirre, “AWE-WQ: Fast-Forwarding Molecular Dynamics Usingthe Accelerated Weighted Ensemble,” Journal of Chemical Information andModeling 54, 3033–3043 (2014).

57U. Adhikari, B. Mostofian, J. Copperman, S. R. Subramanian, A. A. Pe-tersen, and D. M. Zuckerman, “Computational Estimation of Microsecondto Second Atomistic Folding Times,” Journal of the American ChemicalSociety 141, 6519–6526 (2019).

58M. C. Zwier, J. W. Kaus, and L. T. Chong, “Efficient Explicit-Solvent Molecular Dynamics Simulations of Molecular Association Kinet-ics: Methane/Methane, Na+/Cl-, Methane/Benzene, and K+/18-Crown-6Ether,” Journal of Chemical Theory and Computation 7, 1189–1197 (2011).

59M. C. Zwier, A. J. Pratt, J. L. Adelman, J. W. Kaus, D. M. Zuckerman, andL. T. Chong, “Efficient Atomistic Simulation of Pathways and Calculationof Rate Constants for a Protein–Peptide Binding Process: Application to theMDM2 Protein and an Intrinsically Disordered p53 Peptide,” The Journalof Physical Chemistry Letters 7, 3440–3445 (2016).

60A. Dickson and S. D. Lotz, “Ligand Release Pathways Obtained with WEx-plore: Residence Times and Mechanisms,” The Journal of Physical Chem-istry B 120, 5377–5385 (2016).

61J. L. Adelman and M. Grabe, “Simulating Current–Voltage Relationshipsfor a Narrow Ion Channel Using the Weighted Ensemble Method,” Journalof Chemical Theory and Computation 11, 1907–1918 (2015).

62J. Spiriti and D. M. Zuckerman, “Tabulation as a high-resolution alternativeto coarse-graining protein interactions: Initial application to virus capsidsubunits,” The Journal of Chemical Physics 143, 243159 (2015).

63A. Dickson and C. L. Brooks, “WExplore: Hierarchical Exploration ofHigh-Dimensional Spaces Using the Weighted Ensemble Algorithm,” TheJournal of Physical Chemistry B 118, 3532–3542 (2014).

64S. D. Lotz and A. Dickson, “Unbiased Molecular Dynamics of 11 minTimescale Drug Unbinding Reveals Transition State Stabilizing Interac-tions,” Journal of the American Chemical Society 140, 618–628 (2018).

65A. Dickson and S. D. Lotz, “Multiple Ligand Unbinding Pathways andLigand-Induced Destabilization Revealed by WExplore,” Biophysical Jour-nal 112, 620–629 (2017).

66P. Májek and R. Elber, “Milestoning without a Reaction Coordinate,” Jour-nal of Chemical Theory and Computation 6, 1805–1817 (2010).

67E. Vanden-Eijnden and M. Venturoli, “Markovian milestoning with Voronoitessellations,” The Journal of Chemical Physics 130, 194101 (2009).

68R. Elber, “A Milestoning Study of the Kinetics of an Allosteric Transition:Atomically Detailed Simulations of Deoxy Scapharca Hemoglobin,” Bio-physical Journal 92, L85–L87 (2007).

69L. W. Votapka, C. T. Lee, and R. E. Amaro, “Two Relations to Esti-mate Membrane Permeability Using Milestoning,” The Journal of PhysicalChemistry B 120, 8606–8616 (2016).

70A. Fathizadeh and R. Elber, “Ion Permeation through a Phospholipid Mem-brane: Transition State, Path Splitting, and Calculation of Permeability,”Journal of Chemical Theory and Computation 15, 720–730 (2019).

71A. E. Cardenas and R. Elber, “Markovian and Non-Markovian Modelingof Membrane Dynamics with Milestoning,” The Journal of Physical Chem-istry B 120, 8208–8216 (2016).

72L. W. Votapka and R. E. Amaro, “Multiscale Estimation of Binding Ki-netics Using Brownian Dynamics, Molecular Dynamics and Milestoning,”PLOS Computational Biology 11, e1004381 (2015).

73J. C. Phillips, R. Braun, W. Wang, J. Gumbart, E. Tajkhorshid, E. Villa,C. Chipot, R. D. Skeel, L. Kalé, and K. Schulten, “Scalable moleculardynamics with NAMD,” Journal of Computational Chemistry 26, 1781–1802 (2005).

74G. A. Huber and J. A. McCammon, “Browndye: A software package forBrownian dynamics,” Computer Physics Communications 181, 1896–1905(2010).

75G. Grazioli and I. Andricioaei, “Advances in milestoning. II. Calculat-ing time-correlation functions from milestoning using stochastic path in-tegrals,” The Journal of Chemical Physics 149, 084104 (2018).

76D. M. Zuckerman and L. T. Chong, “Weighted Ensemble Simulation: Re-view of Methodology, Applications, and Software,” Annual Review of Bio-physics 46, 43–57 (2017).

77A. F. Voter, “Introduction to the Kinetic Monte Carlo method,” in RadiationEffects in Solids (Springer Netherlands, Dordrecht, 2007) pp. 1–23.

14

78G. Lipari and A. Szabo, “Model-free approach to the interpretation ofnuclear magnetic resonance relaxation in macromolecules. 1. Theory andrange of validity,” Journal of the American Chemical Society 104, 4546–4559 (1982).

79D. Bhatt and I. Bahar, “An adaptive weighted ensemble procedure for effi-cient computation of free energies and first passage rates,” The Journal ofChemical Physics 137, 104101 (2012).

80J. Huang, S. Rauscher, G. Nawrocki, T. Ran, M. Feig, B. L. de Groot,H. Grubmüller, and A. D. MacKerell, “CHARMM36m: an improved forcefield for folded and intrinsically disordered proteins,” Nature Methods 14,71–73 (2017).

81Alexey Onufriev, D. Bashford, , and D. A. Case*, “Modification ofthe Generalized Born Model Suitable for Macromolecules,” (2000),

10.1021/JP994072S.82A. Onufriev, D. Bashford, and D. A. Case, “Exploring protein native states

and large-scale conformational changes with a modified generalized bornmodel,” Proteins: Structure, Function, and Bioinformatics 55, 383–394(2004).

83M. Salvalaglio, P. Tiwary, and M. Parrinello, “Assessing the Reliabilityof the Dynamics Reconstructed from Metadynamics,” Journal of ChemicalTheory and Computation 10, 1420–1425 (2014).

84M. C. Zwier, J. L. Adelman, J. W. Kaus, A. J. Pratt, K. F. Wong, N. B.Rego, E. Suárez, S. Lettieri, D. W. Wang, M. Grabe, D. M. Zuckerman, andL. T. Chong, “WESTPA: An Interoperable, Highly Scalable Software Pack-age for Weighted Ensemble Simulation and Analysis,” Journal of ChemicalTheory and Computation 11, 800–809 (2015).