Numerical Methods for SPDEs driven by Levy JumpProcesses: Probabilistic and Deterministic Approaches

Mengdi Zheng, George EmKarniadakis (Brown University)

2015 SIAM Conference onComputational Science and Engineering

March 17, 2015

Contents

� Motivation� Introduction

� Levy process� Dependence structure of multi-dim pure jump process� Generalized Fokker-Planck (FP) equation

� Overdamped Langevin equation driven by 1D TαS process� by MC and PCM (probabilistic methods)� by FP equation (deterministic method, tempered fractional PDE)

� Diffusion equation driven by multi-dimensional jump processes� SPDE w/ 2D jump process in LePage’s rep� SPDE w/ 2D jump process by Levy copula� SPDE w/ 10D jump process in LePage’s rep (ANOVA decomposition)

� Future work

2 of 25

Section 1: motivation

D. Xiu, J.S. Hesthaven, High order collocation methods for differential

equations with random inputs, SIAM J. Sci. Comput., 27(3) (2005),

pp. 1118–1139.

R. Cont, P. Tankov, Financial Modelling with Jump Processes,Chapman & Hall/CRC Press, 2004.

3 of 25

Section 2.1: introduction of Levy processes� Definition of a Levy process Xt (a continuous random walk):

� Independent increments: for t0 < t1 < ... < tn, random variables(RVs) Xt0 , Xt1 − Xt0 ,..., Xtn−1 − Xtn−1 are independent;

� Stationary increments: the distribution of Xt+h − Xt does not dependon t;

� RCLL: right continuous with left limits;� Stochastic continuity: ∀ε > 0, limh→0 P(|Xt+h − Xt | ≥ ε) = 0;� X0 = 0 P-a.s..

� Decomposition of a Levy process Xt = Gt + Jt + vt: a Gaussianprocess (Gt), a pure jump process (Jt), and a drift (vt).

� Definition of the jump: 4Jt = Jt − Jt− .

� Definition of the Poisson random measure (an RV):N(t,U) =

∑0≤s≤t I4Js∈U , U ∈ B(Rd

0 ), U ⊂ Rd0 .

1

1S. Ken-iti, Levy Processes and Infinitely Divisible Distributions, CambridgeUniversity Press, Cambridge, 1999.

4 of 25

Section 2.2: Pure jump process Jt

� Levy measure ν: ν(U) = E[N(1,U)], U ∈ B(Rd0 ), U ⊂ Rd

0 .

� 3 ways to describe dependence structure between components ofa multi-dimensional Levy process:

5 of 25

Section 2.2: LePage’s multi-d jump processes (1)

� Example 1: d-dim tempered α-stable processes (TαS) inspherical coordinates (”size” and ”direction” of jumps):

� Levy measure (dependence structure):

νrθ(dr , d~θ) = σ(dr , ~θ)p(d~θ) = ce−λrdrr1+α p(d~θ) = ce−λrdr

r1+α2πd/2d~θΓ(d/2) ,

r ∈ [0,+∞], ~θ ∈ Sd .� Series representation by Rosinksi (simulation)2:

~L(t) =∑+∞

j=1

(εj [(

αΓj

2cT )−1/α ∧ ηjξ1/αj ]

)(θj1, θj2, ..., θjd)I{Uj≤t},

for t ∈ [0,T ].P(εj = 0, 1) = 1/2, ηj ∼ Exp(λ), Uj ∼ U(0,T ), ξj ∼U(0, 1).{Γj} are the arrival times in a Poisson process with unit rate.(θj1, θj2, ..., θjd) is uniformly distributed on the sphereSd−1.

2J. Rosınski, Series representations of infinitely divisible random vectors and ageneralized shot noise in Banach spaces, Technical Report No. 195, (1987).J. Rosınski, On series representations of infinitely divisible random vectors, Ann.Probab., 18 (1990), pp. 405–430.

6 of 25

Section 2.2: dependence structure by Levy copula (2)� Example 2: 2-dim jump process (L1, L2) w/ TαS components3

� (L++1 , L++

2 ), (L+−1 , L+−

2 ), (L−+1 , L−+

2 ), and (L−−1 , L−−2 )

Figure : Construction of Levy measure for (L++1 , L++

2 ) as an example

3J. Kallsen, P. Tankov, Characterization of dependence ofmultidimensional Levy processes using Levy copulas, Journal of MultivariateAnalysis, 97 (2006), pp. 1551–1572.

7 of 25

Section 2.2: dependence structure by (Levy copula)� Example 2 (continued):

� Simulation of (L1, L2) ((L++1 , L++

2 ) as an example) by seriesrepresentation 4

L++1 (t) =

∑+∞j=1 ε1j

((

αΓj

2(c/2)T )−1/α ∧ ηjξ1/αj

)I[0,t](Vj),

L++2 (t) =∑+∞j=1 ε2jU

++(−1)2

(F−1(Wi

∣∣∣∣U++1 (

αΓj

2(c/2)T )−1/α ∧ ηjξ1/αj )

)I[0,t](Vj)

� F−1(v2|v1) = v1

(v− τ

1+τ

2 − 1

)−1/τ

.

� {Vi} ∼Uniform(0, 1) and {Wi} ∼Uniform(0, 1). {Γi} is the i-tharrival time for a Poisson process with unit rate. {Vi}, {Wi} and {Γi}are independent.

� Concept: we ’can’ represent Levy processes correlated by Levycopula by RVs4R. Cont, P. Tankov, Financial Modelling with Jump Processes, Chapman

& Hall/CRC Press, 2004.8 of 25

Section 2.3: generalized Fokker-Planck (FP) equations

� For an SODE system d~u = ~C (~u, t) + d~L(t), where ~C (~u, t) is alinear operator on ~u.

� Let us assume that the Levy measure of the pure jump process~L(t) ∈ Rd has the symmetry ν(~x) = ν(−~x).

� The generalized FP equation for the joint PDF satisfies5:

∂P(~u, t)

∂t= −∇·(~C (~u, t)P(~u, t))+

∫Rd−{0}

ν(d~z)

[P(~u+~z , t)−P(~u, t)

].

(1)

� Available in literature: if ~C (~u, t) is non-linear, if the noise ismultiplicative in Ito’s or Marcus’s integral form, the FP eqn isderived.

5X. Sun, J. Duan, Fokker-Planck equations for nonlinear dynamical systemsdriven by non-Gaussian Levy processes. J. Math. Phys., 53 (2012), 072701.

9 of 25

Section 3: 1D SODE driven by 1D TαS process

� We solve6:dx(t;ω) = −σx(t;ω)dt + dLt(ω), x(0) = x0.

� Levy measure of Lt is: ν(x) = ce−λ|x|

|x |α+1 , for x ∈ R, 0 < α < 2

� FP equation as a tempered fractional PDE (TFPDE)� When 0 < α < 1, D(α) = c

αΓ(1− α)

∂∂t P(x , t) = ∂

∂x

(σxP(x , t)

)−D(α)

(−∞Dα,λ

x P(x , t)+xDα,λ+∞P(x , t)

)� −∞Dα,λ

x and xDα,λ+∞ are left and right Riemann-Liouville tempered

fractional derivatives7.� We solve this by 3 methods: MC, PCM, TFPDE.

6M. Zheng, G.E. Karniadakis, Numerical methods for SPDEs withtempered stable processes, SIAM J. Sci. Comput., accepted in 2015.

7M.M. Meerschaert, A. Sikorskii, Stochastic Models for FractionalCalculus, De Gruyter Studies in Mathematics Vol. 43, 2012.

10 of 25

Section 3: PCM vs. TFPDE in E[x2(t;ω)]

0 0.2 0.4 0.6 0.8 110 4

10 3

10 2

10 1

100

t

err 2n

d

fractional density equation

PCM/CP

0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.4510 3

10 2

10 1

100

t

err 2n

d

fractional density equation

PCM/CP

Figure : err2nd versus time by: 1) TFPDEs; 2) PCM. α = 0.5, c = 2,λ = 10, σ = 0.1, x0 = 1 (left); α = 1.5, c = 0.01, λ = 0.01, σ = 0.1, x0 = 1(right). However, TFPDE costs less CPU time than PCM.

11 of 25

Section 3: MC vs. TFPDE in density

4 2 0 2 4 60

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

x(T = 0.5)

dens

ity P

(x,t)

histogram by MC/CPdensity by fractional PDEs

4 2 0 2 4 60

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

x(T=1)de

nsity

P(x

,t)

histogram by MC/CPdensity by fractional PDEs

Figure : Zoomed in plots of P(x ,T ) by TFPDEs and MC at T = 0.5 (left)and T = 1 (right): α = 0.5, c = 1, λ = 1, x0 = 1 and σ = 0.01 (left andright). The agreement can be quantified by the Kolmogorov–Smirnov test.

12 of 25

Section 4: SPDE w/ multi-dim jump process

� We solve :du(t, x ;ω) = µ∂

2u∂x2 dt +

∑di=1 fi (x)dLi (t;ω), x ∈ [0, 1]

u(t, 0) = u(t, 1) = 0

u(0, x) = u0(x) =∑d

i=1 fi (x)

� ~L(t;ω): {Li (t;ω), i = 1, ..., d} are mutually dependent.

� fk(x) =√

2sin(πkx), k = 1, 2, 3, ... are orthonormal on [0, 1].

� By u(x , t;ω) =∑+∞

i=1 ui (t;ω)fi (x) and Galerkin projection onto{fi (x)}, we obtain an SODE system, where Dmm = −(πm)2:

du1(t) = µD11u1(t)dt + dL1, u1(0) = 1du2(t) = µD22u2(t)dt + dL2, u2(0) = 1...dud(t) = µDddud(t)dt + dLd , ud(0) = 1

13 of 25

Section 4.1: SPDEs driven by multi-d jump processes

Figure : probabilistic and deterministic methods: M. Zheng, G.E.Karniadakis, Numerical methods for SPDEs with additivemulti-dimensional Levy jump processes, in preparation.

14 of 25

Section 4.2: FP eqn when ~Lt (2D) is in LePage’s rep

� When the Levy measure of ~Lt is given by (d = 2)

νrθ(dr , d~θ) = ce−λrdrr1+α

2πd/2d~θΓ(d/2) , for r ∈ [0,+∞], ~θ ∈ Sd

� The generalized FP equation for the joint PDF P(~u, t) of solutionsin the SODE system is:∂P(~u,t)∂t = −

∑di=1

[µDii (P + ui

∂P∂ui

)

]− cαΓ(1− α)

∫Sd−1

Γ(d/2)dσ(~θ)

2πd/2

[rDα,λ

+∞P(~u + r~θ, t)

], where ~θ is a

unit vector on the unit sphere Sd−1.

� xDα,λ+∞ is the right Riemann-Liouville TF derivative.

� Solved by a multi-grid solver; I.C. introduces error

� Later, for d = 10, we will use ANOVA decomposition to obtainequations for marginal distributions from this FP equation.

15 of 25

Section 4.2: simulation if ~Lt (2D) is in LePage’s rep

Figure : TFPDE (3D contour) vs. MC (2D contour): P(u1, u2, t) of SODEsystem, slices at the peak. t = 1 , c = 1, α = 0.5, λ = 5, µ = 0.01,NSR = 16.0% at t = 1. NSR = ‖

√Var [u]‖L∞([0,1])/‖E[u]‖L∞([0,1]).16 of 25

Section 4.2: simulation if ~Lt (2D) is in LePage’s rep

0.2 0.4 0.6 0.8 110−10

10−8

10−6

10−4

10−2

l2u2

(t)

t

PCM/S Q=5, q=2PCM/S Q=10, q=2TFPDE

NSR 5 4.8%

0.2 0.4 0.6 0.8 110−7

10−6

10−5

10−4

10−3

10−2

l2u2

(t)t

PCM/S Q=10, q=2PCM/S Q=20, q=2TFPDE

NSR 5 6.4%

Figure : TFPDE vs. PCM: L2 error norm E[u] by PCM and TFPDE.α = 0.5, λ = 5, µ = 0.001 (left and right). c = 0.1 (left); c = 1 (right).(Talk about the restriction of 2 methods here.)

17 of 25

Section 4.3: FP eqn if ~Lt (2D) is from Levy copula

� The dependence structure btw components of ~Lt is given by Levycopula on each corners (++,+−,−+,−−)

� dependence structure is described by the Clayton family of copulaswith correlation length τ on each corner

� The generalized FP eqn is :∂P(~u,t)∂t = −∇ · (~C (~u, t)P(~u, t))

+∫ +∞

0 dz1

∫ +∞0 dz2ν

++(z1, z2)[P(~u + ~z , t)− P(~u, t)]

+∫ +∞

0 dz1

∫ 0−∞ dz2ν

+−(z1, z2)[P(~u + ~z , t)− P(~u, t)]

+∫ 0−∞ dz1

∫ +∞0 dz2ν

−+(z1, z2)[P(~u + ~z , t)− P(~u, t)]

+∫ 0−∞ dz1

∫ 0−∞ dz2ν

−−(z1, z2)[P(~u + ~z , t)− P(~u, t)]� We solve this by a multi-grid solver.� M. Zheng, G.E. Karniadakis, Numerical methods for SPDEs

with additive multi-dimensional Levy jump processes, inpreparation.

18 of 25

Section 4.3: FP eqn if ~Lt (2D) is from Levy copula

Figure : FP (3D contour) vs. MC (2D contour): P(u1, u2, t) of SODEsystem. t = 1 , c = 1, α = 0.5, λ = 5, µ = 0.005, τ = 1, NSR = 30.1%.

19 of 25

Section 4.3: if ~Lt (2D) is from Levy copula

0.2 0.4 0.6 0.8 110−5

10−4

10−3

10−2

t

l2u2

(t)

TFPDEPCM/S Q=1, q=2PCM/2 Q=2, q=2

NSR 5 6.4%

0.2 0.4 0.6 0.8 110−3

10−2

10−1

100

tl2

u2(t)

TFPDEPCM/S Q=2, q=2PCM/S Q=1, q=2

NSR 5 30.1%

Figure : FP vs. PCM: L2 error of E[u] in heat equation α = 0.5, λ = 5,τ = 1 (left and right). c = 0.05, µ = 0.001 (left). c = 1, µ = 0.005 (right).

20 of 25

Section 4.3: FP eqn if ~Lt is in LePage’s rep by ANOVA

� The unanchored analysis of variance (ANOVA) decomposition is 8:P(~u, t) ≈ P0(t) +

∑1≤j1≤d Pj1(uj1 , t) +

∑1≤j1<j2≤d Pj1,j2(uj1 , uj2 , t)

+...+∑

1≤j1<j2...<jκ≤d Pj1,j2,...,jκ(uj1 , uj2 , ..., uκ, t)

� κ is the effective dimension� ANOVA modes of P(~u, t) are related to marginal distributions

� P0(t) =∫

Rd P(~u, t)d~u� Pi (ui , t) =

∫Rd−1 du1...dui−1dui+1...dudP(~u, t)− P0(t) =

pi (ui , t)− P0(t)� Pij(xi , xj , t) =

∫Rd−1 du1...dui−1dui+1...duj−1duj+1...dudP(~u, t)

−Pi (ui , t)− Pj(uj , t)− P0(t) =pij(x1, x2, t)− pi (x1, t)− pj(x2, t) + P0(t)

8M. Bieri, C. Schwab, Sparse high order FEM for elliptic sPDEs, Tech.Report 22, ETH, Switzerland, (2008).X. Yang, M. Choi, G. Lin, G.E. Karniadakis,Adaptive ANOVAdecomposition of stochastic incompressible and compressible flows, Journal ofComputational Physics, 231 (2012), pp. 1587–1614.

21 of 25

Section 4.3: FP eqn if ~Lt is in LePage’s rep by ANOVA

� When the Levy measure of ~Lt is given by

νrθ(dr , d~θ) = ce−λrdrr1+α

2πd/2d~θΓ(d/2) , for r ∈ [0,+∞], ~θ ∈ Sd (for

0 < α < 1)

�∂pi (ui ,t)

∂t = −(∑d

k=1 µDkk

)pi (xi , t)− µDiixi

∂pi (xi ,t)∂xi

− cΓ(1−α)α

(Γ( d

2)

2πd2

2πd−1

2

Γ( d−12

)

)∫ π0 dφsin(d−2)(φ)

[rDα,λ

+∞pi (ui +rcos(φ), t)

]�

∂pij (ui ,uj ,t)∂t =

−(∑d

k=1 µDkk

)pij−µDiiui

∂pij∂ui−µDjjuj

∂pij∂uj− cΓ(1−α)

α

(Γ( d

2)

2πd2

2πd−2

2

Γ( d−22

)

)∫ π

0 dφ1

∫ π0 dφ2sin8(φ1)sin7(φ2)

[rDα,λ

+∞pij(ui + rcosφ1, uj +

rsinφ1cosφ2, t)

]22 of 25

Section 4.3: 1D-ANOVA-FP is enough for E[u] in 10D

0 0.2 0.4 0.6 0.8 1−2

0

2

4

6

8

10

12

x

E[u(

x,T=

1)]

E[uPCM]E[u1D−ANOVA−FP]E[u2D−ANOVA−FP]

0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 13.4

3.6

3.8

4

4.2

4.4

4.6

4.8

5

5.2x 10−4

T

L 2 nor

m o

f diff

eren

ce in

E[u

]

||E[u1D−ANOVA−FP−E[uPCM]||L2([0,1])/||E[uPCM]||L2([0,1])||E[u2D−ANOVA−FP−E[uPCM]||L2([0,1])/||E[uPCM]||L2([0,1])

Figure : 1D-ANOVA-FP vs. 2D-ANOVA-FP vs. PCM in 10D: the mean(left) for heat eqn at T = 1. The L2 norms of difference in E[u] (right).c = 1, α = 0.5, λ = 10, µ = 10−4.NSR ≈ 18.24% at T = 1.

23 of 25

Section 4.3: 2D-ANOVA-FP is enough for E[u] in 10D

0 0.2 0.4 0.6 0.8 10

20

40

60

80

100

120

x

E[u2 (x

,T=1

)]

E[u2PCM]

E[u21D−ANOVA−FP]

E[u22D−ANOVA−FP]

0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 10

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

T

L 2 nor

m o

f diff

eren

ce in

E[u

2 ]

||E[u21D−ANOVA−FP−E[u2

PCM]||L2([0,1])/||E[u2PCM]||L2([0,1])

||E[u22D−ANOVA−FP−E[u2

PCM]||L2([0,1])/||E[u2PCM]||L2([0,1])

Figure : 1D-ANOVA-FP vs. 2D-ANOVA-FP vs. PCM in 10D: E[u2] (left)for heat eqn. The L2 norms of difference in E[u2] (right). c = 1, α = 0.5,λ = 10, µ = 10−4. NSR ≈ 18.24% at T = 1.

24 of 25

Future work

� multiplicative noise (now we have additive noise)

� nonlinear SPDE (now we have linear SPDE)

� higher dimensions (we computed up to < 20 dimensions)

This work is partially supported by OSD-MURI (grantFA9550-09-1-0613), NSF/DMS (grant DMS-1216437) and the newDOE Center on Mathematics for Mesoscopic Modeling of Materials(CM4).Thanks!

25 of 25