an introduction to moment closure techniques

An introduction to moment closure techniques

Colin Gillespie

School of Mathematics & StatisticsNewcastle University

July 30, 2008

Colin Gillespie An introduction to moment closure techniques

Modelling

Let’s start with a simple birth-death model.

Birth-death model

X −→ X − 1 and X −→ X + 1

which has the propensity functions µX and λX .

The deterministic model is

dX (t)dt

= (λ− µ)X (t) ,

which can be solved to give X (t) = X (0) exp[(λ− µ)t ].


Deterministic Solution: λ < µ

Time

Popu

latio

n

0

10

20

30

40

50

0 1 2 3 4


Stochastic Simulation

It’s very easy to simulate the birth-death process usingGillespie’s method:

1 Update reaction clock;2 Choose a reaction to occur;3 Repeat.


Four Stochastic Simulations

4 Simulations of a birth-death process

Time

Popu

latio

n

0

10

20

30

40

50

0 1 2 3 4

Simulation 1 Simulation 2

Simulation 3

0 1 2 3 4

0

10

20

30

40

50

Simulation 4


Stochastic Mean and Variance

If we simulated the process a large number of times (say109), then we could calculate the population mean andvariance.We could construct an approximate 95% tolerance interval

Mean ± 2√

Variance


Four Stochastic SimulationsMean (Green), tolerance interval (red), simulation(blue)

Time

Popu

latio

n

0

10

20

30

40

50

0 1 2 3 4

Simulation 1 Simulation 2

Simulation 3

0 1 2 3 4

0

10

20

30

40

50

Simulation 4


Mean and Variance

In this talk we will look at a quick method for estimating themean and variance, without using stochastic simulation


Moment generating function

Let pn(t) be the probability that the population is of size nat time t .The moment generating function is defined as

M(θ; t) ≡∞∑

n=0

pn(t)enθ .

If we differentiate M(θ; t) w.r.t θ and set θ = 0, we getE[N(t)], i.e. the mean.If we differentiate M(θ; t) w.r.t θ twice, and set θ = 0, weget E[N(t)2] and hence

Var[N(t)] = E[N(t)2]− E[N(t)]2 .


General idea

The birth-death process has the following CME:

dpn

dt= λ(n − 1)pn−1 + µ(n + 1)pn+1 − (λ + µ)npn

After multiplying the CME by enθ and summing over n, weobtain

∂M∂t

= [λ(eθ − 1) + µ(e−θ − 1)]∂M∂θ


Moment Equations

If we differentiate this p.d.e. w.r.t θ and set θ = 0, we get:

dE[N(t)]dt

= (λ− µ)E[N(t)]

where E[N(t)] is the mean. This is a single ODE that wecan solve to obtain a value for the mean.If we differentiate the p.d.e. w.r.t θ twice and set θ = 0, weget:

dE[N(t)2]

dt= (λ− µ)E[N(t)] + 2(λ− µ)E[N(t)2]

and hence the variance Var[N(t)] = E[N(t)2]− E[N(t)]2

So instead of simulating the process 109 to estimate themean and variance, we can simply solve two ODEs.


Part I

Examples


Simple Dimerisation model

The dimerisation model has the following biochemicalreactions:

Dimerisation

2X1 −→ X2 and X2 −→ 2X1

We can formulate the dimer model in terms of momentequations, namely,

dE[X1]

dt= 0.5k1(E[X 2

1 ]− E[X1])− k2E[X1]

dE[X 21 ]

dt= k1(E[X 2

1 X2]− E[X1X2]) + 0.5k1(E[X 21 ]− E[X1])

+ k2(E[X1]− 2E[X 21 ])

where E[X1] is the mean of X1 and E[X 21 ]− E[X1]

2 is thevariance of X1.




Dimerisation

2X1 −→ X2 and X2 −→ 2X1


dE[X1]

dt= 0.5k1(E[X 2

1 ]− E[X1])− k2E[X1]

dE[X 21 ]

dt= k1(E[X 2

1 X2]− E[X1X2]) + 0.5k1(E[X 21 ]− E[X1])

+ k2(E[X1]− 2E[X 21 ])


2 is thevariance of X1.The i th moment equation depends on the (i + 1)th

equation.Colin Gillespie An introduction to moment closure techniques



Dimerisation

2X1 −→ X2 and X2 −→ 2X1


dE[X1]

dt= 0.5k1E [X1](E[X1]− 1) + 0.5k1Var[X1]− k2E[X1]


2 is thevariance of X1.The deterministic equation is an approximation to thestochastic mean.



To close the equations, we usually assume that theunderlying distribution is Normal or Lognormal.The easiest option is to assume an underlying Normaldistribution, i.e.

E[X 31 ] = 3E[X 2

1 ]E[X1]− 2E[X1]3

But we could also use, the Poisson

E[X 31 ] = E[X1] + 3E[X1]

2 + E[X1]3

or the Lognormal

E[X 31 ] =

(E[X 2

1 ]

E[X1]

)3



Time

Pro

tein

Pop

ulat

ion

150

200

250

300

0 5 10 15 20 25


Heat Shock Model

Proctor et al, 2005 - 23 reactions, 17 chemical speciesA single stochastic simulation up to t = 2000 takes about35 minutes.If we convert the model to moment equations, we get 139equations.

A python script automatically generates the ODEs from anSBML file

These can be solved in less than 5 minutes using MapleHopefully I’ll start outputting in sundials, so this should beeven quicker


Heat Shock Model

Time

AD

P

200

400

600

800

1000

1200

0 500 1000 1500 2000

Time

Nat

ive

Pro

tein

(10,

000’

s)

560

570

580

590

600

0 500 1000 1500 2000


Univariate Distributions

ADP

Den

sity

0.000

0.002

0.004

0.006

600 800 1000 1200 1400

Time t=200

600 800 1000 1200 1400

Time t=2000


Bivariate Distributions at time t = 2000

ADP

Nat

P

800 900 1000 1100 1200

4e+0

65e

+06

6e+0

67e

+06


P53-Mdm2 Oscillations model

Proctor and Grey, 2008 - 16 chemical species and about adozen reactions.The model contains two events.If we convert the model to moment equations, we get 139equations.However, in this case the moment closure approximationdoesn’t do to well!


P53 MeanMC(black), True (red)

Time

P53

Pop

ulat

ion

0

50

100

150

200

250

300

0 5 10 15 20 25 30


P53 MeanMC(black), True (red), Deterministic(green)

Time

P53

Pop

ulat

ion

0

50

100

150

200

250

300

0 5 10 15 20 25 30


What went wrong?

The Moment closure (tends) to fail when there is a largedifference between the deterministic and stochasticformulations.I believe it failed because of strongly correlated speciesTypically when the MC approximation fails, it gives anegative varianceThe MC approximation does work well for other parametervalues for the p53 model.


Software

Python script that takes in a SBML file and outputs themoment equations.Currently outputs as a Maple file (University has a sitelicence)Hopefully it will soon output as a sundials/GSL C file(Sort of) supports events.Currently only handles polynomial rate laws, but could beupgrade to handle more complicated rate laws.


References

For an introduction to Moment closure see papers by Matiset al over the last 20 years.Gillespie, C.S. Moment closure approximations formass-action models. IET Systems Biology, in press


an introduction to moment closure techniques

Technology