1

STOCHASTIC CHEMICAL KINETICS

Dan Gillespie

GillespieDT@mailaps.org

Current Support: Caltech (DARPA / AFOSR) The Molecular Sciences Institute (Sandia / DOE)

Past Support: ONR

Collaborators: Linda Petzold, Yang Cao (UCSB) John Doyle (Caltech) Muruhan Rathinam (UMBC) Larry Lok (MSI)

A Chemically Reacting System

• Molecules of N chemical species 1, , NS S .

- In a volume Ω , at temperature T .

- Different conformations or excitation levels are considered different species if they behave differently.

• M “elemental” reaction channels 1, , MR R .

- each jR describes a single instantaneous physical event, which

changes the population of at least one species. Thus, jR is either

iS∅ → ,

or something elseiS → ,

or something elsei iS S′+ → .

2

Question: How does this system evolve in time?

The traditional answer, for spatially homogeneous systems:

“According to the reaction rate equation (RRE).”

• A set of coupled, first-order ODEs.

• Derived using ad hoc, phenomenological reasoning.

• Implies the system evolves continuously and deterministically.

• Empirically accurate for large systems.

• Often not adequate for small systems.

* * *

The question deserves a more carefully considered answer.

Molecular Dynamics (MD)

• The most exact way of describing the system’s evolution.

• Tracks the position and velocity of every molecule in the system.

• Simulates every collision, non-reactive as well as reactive.

• Shows changes in species populations and their spatial distributions.

• But . . . it’s unfeasibly slow for nearly all realistic systems.

3

A great simplification occurs if successive reactive collisions tend to be separated in time by very many non-reactive collisions.

• The overall effect of the non-reactive collisions is to randomize the positions of the molecules (and also maintain thermal equilibrium).

• The non-reactive collisions merely serve to keep the system well-stir red or spatially homogeneous for the reactive collisions.

• Can describe the state of the system by ( )1( ) ( ), , ( )Nt X t X tX ,

( )iX t the number of iS molecules at time t .

But this well-stirred simplification, which . . .

• ignores the non-reactive collisions, • truncates the definition of the system’s state,

. . . comes at a price:

( )tX must be viewed as a stochastic process.

In fact, the system was never deterministic to begin with! Even if molecules moved according to classical mechanics . . . - monomolecular reactions always involve QM. - bimolecular reactions require collisions, whose extreme sensitivity to initial conditions renders them essentially random. - bimolecular reactions usually involve QM too.

But stochastic processes can be handled.

4

For well-stir red systems, each jR is completely characterized by:

• a propensity function ( )ja x : Given the system in state x ,

( )ja dtx the probability that one jR event will occur in the next dt .

- The existence and form of ( )ja x follow from kinetic theory.

- ( )ja x is roughly equal to, but is not derived from, the RRE “rate”.

• a state change vector ( )1 , ,j j N jvν≡ : i jν the change in the

number of iS molecules caused by one jR event.

- jR induces j→ +x x . i jν ≡ the “stoichiometric matrix.”

E.g.,

1

21 2 12

c

cS S S+

1 1 1 2 1

1 12 2 2

( ) , ( 1, 1,0, ,0)

( 1)( ) , ( 1, 1,0, ,0)

2

a c x x

x xa c

= = + − −= = − +

x

x

5

Two exact, r igorously der ivable consequences . . .

1. The chemical master equation (CME):

0 00 0 0 0

1

( , | , )( ) ( , | , ) ( ) ( , | , )

M

j j j jj

P t ta P t t a P t t

t =

∂ = − − − ∂

x xx x x x x x .

• Gives 0 0 0 0( , | , ) Prob ( ) , given ( )P t t t t= =x x X x X x for 0t t≥ .

• The CME follows from the probability statement

( )

( )

0 0 0 01

0 01

( , | , ) ( , | , ) 1 ( )

( , | , ) ( ) .

M

jj

M

j j jj

P t dt t P t t a dt

P t t a dt

=

=

+ = × −

+ − × −

x x x x x

x x x

• But it’s practically always intractable (analytically and numerically).

• With ( ) 0 0( ) ( ) ( , | , )f t f P t tx

X x x x , can show from CME that

( )1

( )( )

M

j jj

d ta t

dt ==

XX .

• I f there were no fluctuations, we would have

( ) ( ) ( )( ) ( ) ( )f t f t f t= =X X X ,

and the above would reduce to the reaction-rate equation (RRE):

( )1

( )( )

M

j jj

d ta t

dt ==

XX .

(Usually written in terms of the concentration ( ) ( )t t ΩZ X .)

But as yet, we have no justification for ignoring fluctuations.

6

2. The stochastic simulation algorithm (SSA):

• A procedure for constructing sample paths or realizations of ( )tX .

• Approach: Generate the time to the next reaction and the index of that reaction.

• Theoretical justification: With ( , | , )p j tτ x defined by

( , | , )p j t dτ τx prob, given ( )t =X x , that the next reaction will

occur in [ , )t t dτ τ τ+ + + , and will be an jR ,

can prove that

( )0 01

( , | , ) ( ) exp ( ) , where ( ) ( )M

j jj

p j t a a a aτ τ ′′=

= − x x x x x .

This implies that the time τ to the next reaction event is an exponential random variable with mean 01 ( )a x , and the index j of

that reaction is an integer random variable with prob 0( ) ( )ja ax x .

The “ Direct” Version of the SSA

1. With the system in state x at time t, evaluate 01

( ) ( )M

jj

a a ′′=x x .

2. Draw two unit-interval uniform random numbers 1r and 2r , and

compute τ and j according to

• 0 1

1 1ln

( )a rτ

=

x,

• j = the smallest integer satisfying 2 01

( ) ( )j

jj

a r a′′=

> x x .

3. Replace t t τ← + and j← +x x .

4. Record ( , )tx . Return to Step 1, or else end the simulation.

7

A Simple Example: 11 0cS → .

1 1 1 1 1( ) , 1a x c x ν= = − . Take 01 1(0)X x= .

RRE: 11 1

( )( )

dX tc X t

dt= − . Solution is 10

1 1( ) e c tX t x −= .

CME: 0

0 01 11 1 1 1 1 1 1

( , | ,0)( 1) ( 1, | ,0) ( , | ,0)

P x t xc x P x t x x P x t x

t

∂ = + + − ∂

.

Solution: ( )01 1

1 1 1

00 01

1 1 1 101 1 1

!( , | ,0) e 1 e ( 0,1, , )

!( )!

x xc x t c txP x t x x x

x x x

−− −= − =−

which implies 101 1( ) e c tX t x −= , ( )1 10

1 1sdev ( ) e 1 ec t c tX t x − −= − .

SSA: Given 1 1( )X t x= , generate 1 1

1 1ln

c x rτ =

, then update:

1 1, 1t t x xτ← + ← − .

0 1 2 3 4 5 60

20

40

60

80

100

S1 → 0

c1 = 1, X1(0) = 100

8

0 1 2 3 4 5 60

20

40

60

80

100

S1 → 0

c1 = 1, X1(0) = 100

0 1 2 3 4 5 60

20

40

60

80

100

S1 → 0

c1 = 1, X1(0) = 100

9

0 1 2 3 4 5 60

20

40

60

80

100

S1 → 0

c1 = 1, X1(0) = 100

The SSA . . .

• Is exact.

• Is equivalent to (but is not derived from) the CME.

• Does not entail approximating “dt” by “ t∆ ” .

• Is procedurally simple, even when the CME is intractable. • Has been redesigned to be faster and more efficient for very large

systems (though more complicated to code) by Gibson and Bruck. • Remains too slow for most practical problems: Simulating every

reaction event, one at a time, is just too much work if any reactant is present in very large numbers.

10

We would be willing to sacr ifice a little exactness . . . . . . if that would buy us a faster simulation.

One way of doing this: Tau-Leaping

• Approximately advances the process by a pre-selected time τ , which may encompass more than one reaction event.

• The size of τ is limited by the Leap Condition: The changes induced in the propensity functions during τ must be “ small” .

• If τ can also be taken large enough to encompass many reaction events, tau-leaping will be faster than the SSA.

• Real speed-ups of 100× obtained for some simple model systems.

• But must use with care. Not as automatic and foolproof as the SSA.

Basics of Tau-Leaping

• Some math: The Poisson random variable ( , )a τ the number of events that will occur in time τ , given that the probability of an event occurring in any dt is adt .

• So, with ( )t =X x , if τ is such that ( )ja x ≈ constant in [ , ]t t τ+ ,

then the number of jR reactions that will occur in [ , ]t t τ+ is

approximately ( )( ),ja τx ; hence,

( )1

( ) ( ),M

j j jj

t aτ τ=

+ +X x x .

• Often feasible because . . . - Reliable procedures exist for generating samples of ( , )a τ .

- A way has been found to estimate in advance the largest τ for a specified degree of adherence to the Leap Condition.

11

The Tau-Leap Simulation Algorithm

• 1. In state x at time t , choose τ so that the expected change in every propensity function in [ , ]t t τ+ is 0( )aε≤ x . (This can be done.)

• 2. Generate the number of firings jk of channel jR in [ , ]t t τ+ as

( )( ), ( 1, , )j jk a j Mτ= =x .

• 3. Leap: Replace t t τ← + and 1

M

j jj

k=

← +x x .

• 4. Record ( , )tx . Then return to Step 1, or else end the simulation.

0

5000

10000

15000

20000

25000

30000

35000

40000

45000

50000

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30

Time

mo

no

mer

X1,

un

stab

le d

imer

X2,

sta

ble

dim

er X

3

X2

X3

X1

Decaying-Dimerizing Reaction Set S1--> 0 c1 = 1 S1 + S1 --> S2 c2 = 0.002 S2 --> S1 + S1 c3 = 0.5 S2 --> S3 c4 = 0.04 - Exact SSA Run - 500 steps (reactions) per plotted dot. - Intially: X1=100,000; X2=X3=0. - Last reaction at T=48.07. - Final X(3)=17,106. - 524,112 steps (reactions) total.

12

0

5000

10000

15000

20000

25000

30000

35000

40000

45000

50000

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30

Time

mo

no

mer

X1,

un

stab

le d

imer

X2,

sta

ble

dim

er X

3

X2

X3

X1

Decaying-Dimerizing Reaction Set S1--> 0 c1 = 1 S1 + S1 --> S2 c2 = 0.002 S2 --> S1 + S1 c3 = 0.5 S2 --> S3 c4 = 0.04 - Explicit Tau Leaping Run (Epsilon=0.03) - 1 leap per plotted dot. - Intially: X1=100,000; X2=X3=0. - Last reaction at T=44.52. - Final X(3)=17,033. - 592 leaps total.

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

0 5 10 15 20 25 30

Time

AJ

/ A0

A1/A0

A2/A0

A3/A0

A4/A0

Decaying-Dimerizing Reaction Set

R1: S1 ---> 0 A1 = C1*X1 R2: S1 + S1 ---> S2 A2 = C2*X1*(X1-1)/2 R3: S2 ---> S1 + S1 A3 = C3*X2 R4: S2 ---> S3 A4 = C4*X2

Plots of AJ/A0, where A0=A1+A2+A3+A4. Exact SSA run. 500 reactions per plotted point.

A2/A0

A3/A0

13

1800 2100 2400 2700 3000

X1(t=10)

0.000

0.002

0.004

0.006

0.008

rela

tive

freq

uenc

y

3 x 10,000 Simulation Runs

t=0: X1=4150, X2=39565, X3=3445

Gaussian-windowed histograms ofX1 at t=10

Exact SSA (32 hours).

Tau-leaping, =0.02, (11 minutes).

Tau-leaping, =0.03, (6.5 minutes).

13200 13550 13900 14250 14600

X2(t=10)

0.000

0.001

0.002

0.003

0.004

0.005

rela

tive

freq

uenc

y

3 x 10,000 Simulation Runs

t=0: X1=4150, X2=39565, X3=3445

Gaussian-windowed histograms ofX2 at t=10

Exact SSA

Tau-leaping ( =0.02)

Tau-leaping ( =0.03)

14

13100 13375 13650 13925 14200

X3(t=10)

0.000

0.002

0.004

0.006

rela

tive

freq

uenc

y

3 x 10,000 Simulation Runs

t=0: X1=4150, X2=39565, X3=3445

Gaussian-windowed histograms ofX3 at t=10

Exact SSA

Tau-leaping ( =0.02)

Tau-leaping ( =0.03)

We can push Tau-Leaping fur ther . . .

• Some more math: ( , )a τ has mean and variance aτ . And when

1aτ , can approximate ( , ) ( , ) (0,1)a a a a aτ τ τ τ τ≡ + .

• So, with ( )t =X x , suppose we can choose τ to satisfy the Leap

Condition, and also the conditions ( ) 1,ja jτ ∀x . Then

( )1

( ) ( ),M

j j jj

t aτ τ=

+ +X x x

can be further approximated to

( ) ( )1 1

( ) (0,1)M M

j j j j jj j

t a aτ τ τ= =

+ + + X x x x .

• Valid iff τ is chosen small enough to satisfy the Leap Condition, yet large enough that every jR fires many more times than once in τ .

• It’s not always possible to find such a τ ! But it usually is if all the reactant populations are “sufficiently large”.

15

( ) ( )1 1

( ) (0,1)M M

j j j j jj j

t a aτ τ τ= =

+ + + X x x x

• Is called the Langevin leaping formula. • It’s faster than ordinary tau-leaping. • It directly implies (and is entirely equivalent too) a SDE called the chemical Langevin equation (CLE):

( ) ( )1 1

( )( ) ( ) ( )

M M

j j j j jj j

d ta t a t t

dtΓ

= =+

XX X .

jΓ ’ s are “Gaussian white noises” , ( ) ( ) ( )j j j jt t t tΓ Γ δ δ′ ′′ ′= − .

• Our discrete stochastic process ( )tX has now been approximated as a continuous stochastic process.

• Again, the CLE can’ t always be invoked. But it usually can if all the reactant molecular populations are “ sufficiently large” .

The Thermodynamic Limit

Def: All iX → ∞ , and Ω → ∞ , with iX Ω constant.

• Can prove that, in this limit, all propensity functions grow linearly with the system size.

• So as we approach this limit, in the CLE

( ) ( )1 1

( )( ) ( ) ( )

M M

j j j j jj j

d ta t a t t

dtΓ

= =+

XX X ,

the deterministic term grows like (system size), while the

stochastic term grows like system size .

• This establishes the well know rule-of-thumb: Relative fluctuations scale as the inverse square root of the system size.

• Very near the thermodynamic limit, the CLE approximates to

( )1

( )( )

M

j jj

d ta t

dt =

XX ,

the reaction rate equation (RRE)! ( )tX has now become a continuous deterministic process. And we’ve derived the RRE!

16

Summarizing . . .

Star t with Molecular Dynamics • The Gold Standard. • “State” = positions and velocities of all the molecules. • Simulates all collisions, non-reactive as well as reactive. • But is way too slow.

The “ well-stir red” simplification gives the CME/SSA • Assumes the non-reactive collisions occur frequently enough to keep

the system well-stirred (spatially homogeneous) for the reactive collisions.

• Simulates only the reactive collisions. • “State” = the molecular populations of all the species, ( )tX .

• ( )tX is a discrete stochastic process.

The Spectrum of Analytical Approaches for Well-Stirred Systems

,( ) const ( ) 1

constin [ , ],

ij j

i i

Xa a

Xt t j j

ΩτΩτ

→∞ →∞≈→+ ∀ ∀

x x

- → → →CME/SSA Tau Leaping CLE RRE

approximate approximate approximate

continuou

exact

discrete discrete

stochastic stochastic stochastic

s continuous

determinis

tic

Is it simply a matter of picking the “ right tool” from this spectrum?

Not quite.

17

Complications from “ Multiscale”

• Some jR occur very much more frequently than others.

• Some iX are very much larger than others.

• Fast & slow & small & large usually all occur coupled – not easy to separate.

• Leads to dynamical stiffness, which usually restricts τ to the smallest time scale in the system.

• One recently proposed fix: Implicit tau-leaping – a stochastic adaptation of the implicit numerical solution method for stiff ODEs.

• Other proposed fixes: stochastic versions of the quasi-steady state and partial equilibrium approximation methods of deterministic chemical kinetics.

Spatially Inhomogeneous Systems

• Spatial homogeneity does not require that all equal-size subvolumes of Ω contain the same number of molecules!

• The CME and SSA require only that the center of a randomly chosen

iS molecule be found with equal probability at any point inside Ω .

• A system consisting of only one molecule can be “well-stirred”.

But the well-stir red assumption can’ t always be made.

In that case, we must do something different; however, the traditional reaction-diffusion equation (RDE) is not always the answer:

• The RDE (like the RRE) is continuous and deterministic. • It assumes that each dΩ contains a spatially homogeneous mixture of

infinitely many molecules. • Not the case in most cellular systems, where spatial inhomogeneity

arises not from slow mixing but rather from compartmentalization caused by highly heterogeneous structures within the cell.

18

We can formulate a spatially inhomogeneous CME/SSA: Subdivide Ω into K spatially homogeneous subvolumes kΩ . Mathematically is

the same as in the homogeneous case, except now we have

• KN species i kS , and KM chemical reactions j kR ,

• plus a whole bunch of diffusive transfer reactions diffi kkR ′ .

Challenges:

• Enormous increases in the numbers of species and reactions, so simulations will run orders of magnitude slower.

• No unambiguous definition of spatial homogeneity, hence no clear criterion for making the partitioning kΩ Ω→ .

• Dynamically altering the kΩ will probably be necessary, and will

require bookkeeping that is complicated and time-consuming. • Conflicting ways to compute the diffusive transfer rate constants.