approximation of stochastic partial differential equations ... · approximation of stochastic...

Approximation of Stochastic Partial DifferentialEquations by a Kernel-based Collocation Method

Department of Applied MathematicsIllinois Institute of Technology

Joint work with Prof. I. Cialenco and Prof. G. E. Fasshauer

February 2012

qye3@iit.edu MCQMC 2012 February 2012

Introduction

Outline

1 Introduction

2 Background

3 Kernel-based Collocation Methods

4 Numerical Examples

5 Acknowledgments

Introduction

Meshfree Methods

Stochastic AnalysisStatistical Learning

Introduction Books

Monographs

2010-12-11http://images.barnesandnoble.com/images/48560000/48568864.JPG

2010-12-11http://images.barnesandnoble.com/images/34380000/34380775.jpg

2010-12-11http://i.ebayimg.com/22/!B+SFnlw!Wk~$(KGrHqV,!jUEzKMBd-(3BM+JDHMq3w~~0...

Background

Outline

1 Introduction

2 Background

5 Acknowledgments

Background The method in a nutshell

Parabolic Stochastic Equations =⇒ Elliptic Stochastic Equations

Here, we only consider the simple high-dimensional elliptic SPDE∆u = f + ξ, in D ⊂ Rd ,

u = 0, on ∂D,

where∆ =

∑dj=1

∂x2j

is the Laplacian operator,

suppose that u ∈ Sobolev space Hm(D) with m > 2 + d/2 a.s.,f : D → R is a deterministic function,ξ : D × Ωξ → R is a Gaussian field with mean zero and covariancekernel W : D ×D → R defined on a probability space (Ωξ,Fξ,Pξ),i.e.,

E(ξx ) = 0, Cov(ξx , ξy ) = W (x ,y).

The proposed numerical method for solving a parabolic SPDE can bedescribed as follows:

1 We choose a reproducing kernel

K : D ×D → R

whose reproducing kernel Hilbert space HK (D) is embedded intoHm(D).

Noise Covariance Kernel W → Smoothness of Exact Solution u↓ ↓

Convergent Rates ← Reproducing Kernel K

2 We simulate the Gaussian field with covariance structure W at afinite collection of predetermined collocation points

XD := x1, · · · ,xN ⊂ D, X∂D := xN+1, · · · ,xN+M ⊂ ∂D,

yj := f (x j) + ξx j , j = 1, · · · ,N, yN+j := 0, j = 1, · · · ,M,

ξ := (ξx1 , · · · , ξxN ) ∼ N (0,W) , W :=(W (x j ,xk )

)N,Nj,k=1 .

We also let the random vector

yξ := (y1, · · · , yN+M)T .

3 We also define its integral-type kernel

∗K (x ,y) :=

K (x , z)K (y , z)dz ,∗K ∈ Hm,m(D ×D).

4 The kernel-based collocation solution is written as

u(x) ≈ u(x) :=N∑

ck ∆2∗K (x ,xk ) +

M∑k=1

cN+k∗K (x ,xN+k ),

where the unknown random coefficients

c := (c1, · · · , cN+M)T

are obtained by solving a random system of linear equations, i.e.,

∗Kc = yξ.

Background Advantages

Advantages

The kernel-based collocation method is a meshfree approximationmethod. It does not require an underlying triangular mesh as theGalerkin finite element method does.

The kernel-based collocation method can be applied to ahigh-dimensional domain D with complex boundary ∂D.

To obtain the truncated Gaussian noise ξn for the finite elementmethod, it is difficult for us to compute the eigenvalues andeigenfunctions of the noise covariance kernel W . For thekernel-based collocation method we need not worry about thisissue.

Once the reproducing kernel is fixed, the error of the collocationsolution only depends on the collocation points.

Background Difference for Finite Element Methods

Given a finite element basis φ, we shall compute the right-hand side forthe Galerkin finite element methods.

Popular Methods:∫Dξxφ(x)dx ≈

∫Dξn

xφ(x)dx =n∑

√λkek (x)φ(x)dx ,

where the truncated Gaussian noise

ξx ≈ ξnx =

n∑k=1

ζk√λkek (x), ζ1, . . . , ζn ∼ i.i.d.N (0,1),

W (x ,y) ≈W n(x ,y) =n∑

λkek (x)ek (y).

Background Difference for Finite Element Methods

Monte Carlo Methods:For each fixed sample path ω ∈ Ωξ, ξx (ω) is a function defined onD. However, we do not know its exact form. We can only useMonte Carlo methods to approximate the right-hand side, i.e.,∫

Dξxφ(x)dx ≈

N∑j=1

ξx jφ(x j).

Kernel-based Methods:

ξx ≈ ξx := w(x)T W−1ξ,

w(x) := (W (x ,x1), · · · ,W (x ,xN))T , W :=(W (x j ,xk )

)N,Nj,k=1 .

Kernel-based Collocation Methods

Outline

1 Introduction

2 Background

5 Acknowledgments

Kernel-based Collocation Methods Gaussian Fields

According to [Cialenco, Fasshauer and Ye 2011 SPDE, Theorem 3.1],for a given µ ∈ HK (D), there exists a probability measure Pµ defined on

(ΩK ,FK ) = (HK (D),B(HK (D)))

such that the stochastic fields ∆S, S given by

∆Sx (ω) = ∆S(x , ω) := (∆ω)(x), x ∈ D, ω ∈ ΩK = HK (D),

Sx (ω) = S(x , ω) := ω(x), x ∈ D ∪ ∂D, ω ∈ ΩK = HK (D),

are Gaussian with means ∆µ, µ and covariance kernels ∆1∆2∗K ,

defined on (ΩK ,FK ,Pµ), respectively.

For any fixed z ∈ R, we let

Ex (z) := ω ∈ ΩK : ω(x) = z = ω ∈ ΩK : Sx (ω) = z .

Kernel-based Collocation Methods Gaussian Fields

[Cialenco, Fasshauer and Ye 2011 SPDE, Corollary 3.2], shows thatthe random vector

S := (∆Sx1 , · · · ,∆SxN ,SxN+1 , · · · ,SxN+M ) ∼ N (mµ,∗K),

wheremµ := (∆µ(x1), · · · ,∆µ(xN), µ(xN+1), · · · , µ(xN+M))T

∗K :=

(∆1∆2∗K (x j ,xk ))N,N

j,k=1, (∆1∗K (x j ,xN+k ))N,M

(∆2∗K (xN+j ,xk ))M,N

j,k=1, (∗K (xN+j ,xN+k ))M,M

For any given y = (y1, · · · , yN+M)T ∈ RN+M , we let

EX (y) := ω ∈ ΩK : ∆ω(x1) = y1, . . . , ω(xN+M) = yN+M= ω ∈ ΩK : S(ω) = y .

Kernel-based Collocation Methods Approximation and Convergence

For each fixed x ∈ D and ω2 ∈ Ωξ, we obtain the "optimal" estimator

u(x , ω2) ≈ u(x , ω2) = argmaxz∈R

supµ∈HK (D)

Pµξ(Ex (z)× Ωξ

∣∣EX(yξ(ω2)

= argmaxz∈R

supµ∈HK (D)

Pµξ(Sx = z

∣∣S = yξ(ω2)),

= argmaxz∈R

supµ∈HK (D)

pµx (z|yξ(ω2)),

= k(x)T∗K−1yξ(ω2)

where k(x) := (∆2∗K (x ,x1), · · · ,

∗K (x ,xN+M))T and

ΩK ξ := ΩK × Ωξ, FK ξ := FK ⊗Fξ, Pµξ := Pµ ⊗ Pξ,

so that ∆S, S and ξ can be extended to the product space whilepreserving the original probability distributional properties.

Error Bound Analysis

For any ε > 0, we define

Eεx :=ω1 × ω2 ∈ ΩK × Ωξ : |ω1(x)− u(x , ω2)| ≥ ε,

s.t. ∆ω1(x1) = y1(ω2), . . . , ω1(xN+M) = yN+M(ω2).

Let the fill distance

hX := supx∈D

min1≤j≤N+M

‖x − x j‖2.

We can deduce that

supµ∈HK (D)

Pµξ (Eεx ) = O

(hm−2−d/2

where m is the order of the Sobolev space corresponded to the exactsolution of the SPDE.

Since |u(x , ω2)− u(x , ω2)| ≥ ε if and only if u ∈ Eεx , we have

supµ∈HK (D)

Pµξ(‖u − u‖L∞(D) ≥ ε

)≤ sup

µ∈HK (D),x∈DPµξ (Eεx )→ 0,

when hX → 0.

Numerical Examples

Outline

1 Introduction

2 Background

5 Acknowledgments

Numerical Examples Stochastic Laplace’s Equations

Let the domainD := (0,1)2 ⊂ R2.

We choose the deterministic function

f (x) := −2π2 sin(πx1) sin(πx2)− 8π2 sin(2πx1) sin(2πx2),

and the covariance kernel of the Gaussian noise ξ to be

W (x ,y) :=4π4 sin(πx1) sin(πx2) sin(πy1) sin(πy2)

+ 16π4 sin(2πx1) sin(2πx2) sin(2πy1) sin(2πy2).

Then the exact solution of the above elliptic SPDE has the form

u(x) := sin(πx1) sin(πx2) + sin(2πx1) sin(2πx2)

+ ζ1 sin(πx1) sin(πx2) +ζ2

2sin(2πx1) sin(2πx2),

where ζ1, ζ2 ∼ i.i.d. N (0,1).

For the collocation methods, we use the C4-Matérn function with shapeparameter θ > 0

gθ(r) := (3 + 3θr + θ2r2)e−θr , r > 0,

to construct the reproducing kernel (Sobolev-spline kernel)

Kθ(x ,y) := gθ(‖x − y‖2).

According to [Fasshauer and Ye 2011 Distributional Operators,Fasshauer and Ye 2011 Differential and Boundary Operators], we candeduce that

HKθ(D) ∼= H3+1/2(D) ⊂ C2(D).

Approximate Mean

Relative Absolute Error0.02 0.04 0.06 0.08

Approximate Variance

Relative Absolute Error0.01 0.02 0.03 0.04 0.05 0.06

−4 −2 0 2 40

PDF, x1 = 0.52632, x2 = 0.52632

Empirical Theoretical

0 0.2 0.4 0.6 0.8 10

1Collocation Points

Figure: N = 65, M = 28 and θ = 0.9qye3@iit.edu MCQMC 2012 February 2012

0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.2 0.220

fill distance hX

lative

Mean, θ = 0.9

Variance, θ = 0.9Mean, θ = 1.9

Variance, θ = 1.9Mean, θ = 2.9

Variance, θ = 2.9

Figure: Convergence of Mean and Varianceqye3@iit.edu MCQMC 2012 February 2012

Acknowledgments

Outline

1 Introduction

2 Background

5 Acknowledgments

Acknowledgments

THANK YOU for the invitation and the NSF support from Prof. Owen.

Appendix References

References I

R. A. Adams and J. J. F. Fournier,Sobolev Spaces (2nd Ed.),Pure and Applied Mathematics, Vol. 140, Academic Press, 2003.

A. Berlinet and C. Thomas-Agnan,Reproducing Kernel Hilbert Spaces in Probability and Statistics,Kluwer Academic Publishers, 2004.

M. D. Buhmann,Radial Basis Functions: Theory and Implementations,Cambridge University Press (Cambridge), 2003.

G. E. Fasshauer,Meshfree Approximation Methods with MATLAB,Interdisciplinary Mathematical Sciences, Vol. 6, World ScientificPublishers (Singapore), 2007.

Appendix References

References II

L. Hörmander,The analysis of linear partial differential operators I,Classics in Mathematics, Springer, 2004.

P. E. Kloeden and E. PlatenNumerical Solution of Stochastic Differential Equations, Vol. 23,Springer, 2011.

B. ØksendalStochastic Differential Equations: An Introduction withApplications, 6th edition,Springer, 2010.

I. Steinwart and A. Christmann,Support Vector Machines,Springer Science Press, 2008.

Appendix References

References III

G. Wahba,Spline Models for Observational Data,CBMS-NSF Regional Conference Series in Applied Mathematics59, SIAM (Philadelphia), 1990.

H. Wendland,Scattered Data Approximation,Cambridge University Press, 2005.

G. E. Fasshauer and Q. Ye,Reproducing Kernels of Generalized Sobolev Spaces via a GreenFunction Approach with Distributional Operator,Numerische Mathematik, Volume 119, Number 3, Pages 585-611,2011.

Appendix References

References IV

G. E. Fasshauer and Q. Ye,Reproducing Kernels of Sobolev Spaces via a Green FunctionApproach with Differential Operators and Boundary Operators,Advances in Computational Mathematics, 2011, to appear, DOI:10.1007/s10444-011-9264-6.

G. E. Fasshauer and Q. Ye,Kernel-based Collocation Methods versus Galerkin Finite ElementMethods for Approximating Elliptic Stochastic Partial DifferentialEquations,in preparation.

I. Cialenco, G. E. Fasshauer and Q. Ye,Approximation of Stochastic Partial Differential Equations by aKernel-based Collocation Method,submitted.

Appendix References

References V

S. Koutsourelakis and J. WarnerLearning Solutions to Multiscale Elliptic Problems with GaussianProcess Models,Research report at Cornell University, 2009.

Q. Ye,Reproducing Kernels of Generalized Sobolev Spaces via a GreenFunction Approach with Differential Operator,IIT technical report, 2010.

approximation of stochastic partial differential equations ... · approximation of stochastic...

Documents

stochastic differential equations - kampen

1 multidimensional stochastic approximation: adaptive...

approximation algorithms for stochastic combinatorial...

solving stochastic partial differential equations as …....

stochastic calculus and stochastic differential...

on weak approximation of stochastic differential equations...

approximation in stochastic integer programming

sample approximation methods for stochastic program

simulation & stochastic differential equations ·...

smooth approximation of stochastic differential...

stochastic approximation algorithms for partition function...

optimal execution and stochastic approximation: learning...

approximation of stochastic partial differential equations...

stochastic approximation and simulated annealing

how does a stochastic optimization/approximation algorithm...

stochastic approximation applications

stochastic differential equations

wick−malliavin approximation to nonlinear stochastic...

solving linear stochastic differential equations · solving...

stochastic differential dynamic logic for …3 stochastic...