variance reduction and brownian simulation methods

Variance reduction and Brownian Simulation Methods

Yossi Shamai

Raz Kupferman

The Hebrew University

All (incompressible) fluids are governed by mass-momentum conservation equations

u(x,t) = velocity

(x,t) = polymeric stress

QuickTime™ and a decompressor

are needed to see this picture.





Dumbbell models

Dumbbell models

(q,x,t) = pdf.

The polymers are modeled by two beads connected by a spring (dumbbell) . The conformation is modeled by an end-to-end vector q.

less affect

more affect

q

The (random) conformations are distributed according to a density function (q,x,t), which satisfies an evolution equation

advection deformation diffusion

The stress is an ensemble average of polymeric conformations,



q

g(q) = qF(q)

The stress

Conservation laws (macroscopic dynamics)

Polymeric density distribution (microscopic dynamics)

• Problem: high dimensionality





• Assumption: 1-D

Closable systemsIn certain cases, a PDE for (x,t) can be derived, yielding a closed-form system for u(x,t), (x,t).



• Closable systems can be solved by standard methods.• Brownian simulations can be used for non-closable systems.

Example (Semi-linear systems): if g(q) = q2, and b(q,u) = b(u) q then (x,t) satisfies the PDE



Outline

1. Brownian simulation methods

2. Some mathematical preliminaries on spatial correlations

3. A variance reduction mechanism in Brownian simulations

4. Examples



Brownian simulationsThe average stress (x,t) is an expectation with respect to a stochastic process q(x,t) with PDF (q,x,t)

q(x,t) is simulated by a collection of realizations qi(x,t).

The stress is approximated by an empirical mean QuickTime™ and a

decompressorare needed to see this picture.

PDE SPDE

A reminder: real-valued Brownian motion

1. B(t) is a random function of time.

2. Almost surely continues.

3. Independent increments.

4. B(t)-B(s) ~ N(0,t-s).L2-valued Brownian motion

1. B(x,t) is a random function of time and space.

2. For fixed x, B(x,t) is a real-valued Brownian motion.

3. Finite normQuickTime™ and a


Spatial correlationsB(x,t) is characterized by the spatial correlation function



1. Symmetry: c(x,y) = c(y,x).

2. c(x,x) = 1.

3. L2 - function:



Spatial correlations (cont.)An L2 - function is a correlation function iff

a. c(x,x) = 1.

b. It has a “square root” in L2



Oscillatory

Discretization QuickTime™ and a decompressor


UniformQuickTime™ and a


Piecewise constant uncorrelated





• No spatially uncorrelated L2-valued Brownian motion.

• Spatially uncorrelated noise has meaning only in a discrete setting. It is a sequence of piecewise constant standard Brownian motions, uncorrelated at any two distinct steps, that converges to 0.



Spatial correlations (cont.)

Spatial correlations (cont.)Spatial correlations can be alternatively

described by Correlation operators



QuickTime™ and a decompressorare needed to see this picture. QuickTime™ and a decompressorare needed to see this picture.

• C is nonnegative, symmetric and trace class.



• For any f,g in L2

• No Id-correlated Brownian motion (trace Id = ∞ ).



SDEs versus SPDEs



SDEs (Stochastic Differential Equations)





• F,G are operators

SPDEs (Stochastic Partial Differential Equations)

Ito’s integral

Ito’s integral




are needed to see this picture.QuickTime™ and a decompressorare needed to see this picture.

• q(x,t) has spatial correlation.

PDE (Fokker-Plank)

SDE

SDEs





PDE (Fokker-Plank)

SPDE

SPDEsQuickTime™ and a


SDEs versus SPDEs

Brownian simulationsunifying approach

• Equivalence class insensitive to spatial correlations.

€



• Consistency: for every x, q(x,0) ~ (q,x,0).



Lemma: Let (u,,q) be a solution for the stochastic system on some time interval [0,T]. Let (q,x,t) be the PDF corresponding to q(x,t). Then (u,,) is a solution for the deterministic system on [0,T].



Brownian simulation methodsThe stochastic process q is simulated by n “realizations” driven by i.i.d Brownian motions. Expectation is approximated by an empirical mean with respect to the realizations: QuickTime™ and a




Advantages: 1. No Fokker-plank equation.

2. Easy to simulate.

Disadvantages:1. No error

analysis.2. Variance is

O(n-1).



Brownian simulation methodsThe approximation The system

CONNFFESSIT (Calculations of Non Newtonian Fluids Finite Elements and Stochastic Simulation Techniques) - Piecewise constant uncorrelated noise (Ottinger et al. 1993)

BCF - Spatially uniform noise (Hulsen et al. 1997)



Correlation affects approximation but not the exact solution

Error reduction ?

1. Prove that e(n,t)0.

2. Reduce the error by choosing the spatial correlation of the Brownian noise:

Step 1. Express e(n,t) as a function F(c).

Step 2. Minimize F(c).

The error of the Brownian simulations is

Goals



The idea of adapting correlation to minimize variance first proposed by Jourdain et al. (2004) in the context of shear flow with a specific FEM scheme.

An “integral-type” system



The Brownian simulation is



Example

Results:

Brownian simulation

The stress

n = 2000 with spatially uniform noise ( c(x,y) = 1 ).

The (normalized) error as a function of time

Large error (1.47)





“smooth” simulation

The Brownian simulation at t=20

(dotted curve)

Brownian simulation

Stress

Results:


(dotted curve)

“noisy” simulations

n = 2000 with piecewise constant uncorrelated noise.


reduced error (1.06)





Error analysisWe want to analyze the error of the Brownian simulations



Lets demonstrate the analysis for semi-linear system…

Closable systemsIn certain cases, a PDE for (x,t) can be derived, yielding a closed-form system for u(x,t), (x,t).



• Closable systems can be solved by standard methods.• Brownian simulations can be used for non-closable systems.

Example (Semi-linear systems): if g(q) = q2, and b(q,u) = b(u) q then (x,t) satisfies the PDE



Error analysis for Semi-linear systems

• Linearize (properly)

In semi-linear systems, the stress field (x,t) satisfies a PDE

We want to estimate the error of the Brownian simulations QuickTime™ and a




An analogous evolution equation for T(x,t) is derived







Linearized system



andQuickTime™ and a


Theorem 1. To leading order:



and k is a kernel function determined by the parameters.



where



The function F can be also expressed in terms of the correlation operator C,

• F is convex

• In principle, the analysis is the same

• Proof is restricted to closable systems

Theorem 1. To leading order in n,



Error analysis for Closable systems

The optimization problem

Minimize F(c) over the domain:

S = {c(x,y) : c has a root in L2, c(x,x) = 1}

Difficulties:

A. Infinite dimensional optimization problem.

B. S is not compact.

• In general, there is no minimizer

Find a sequence of correlations cn S such that F(cn)

converge toQuickTime™ and a


Finite dimensional approximations

1. Set a natural k.

2. Discretize the problem to a k-point mesh





Theorem 2. The sequence of errors

converges (as k∞ ) to the optimal error





The F-D optimization problem

The F-D optimization problem is:

Minimize F(A), A is k-by-k symmetric PSD

Subject to Aii = 1, i=1,…,k

We want to minimize F(ck) over Sk.

• ck(x,y) is indexed by k2 mesh points (xi ,xj) (matrix).

• Symmetric Positive-Semi-Definite.

• ck (xi ,xi) = 1.

F is convex SDP algorithms (Semi-Definite Programming)

So what did we do?

Developed a unifying approach for a variance reduction mechanism in Brownian simulations.

Formulated an optimization problem (in infinite dimensions).

Showed that it is amenable to a standard algorithm (SDP).

Example 1

A linear advection-dissipation equation in [0,1].







In stochastic formulation,



The Brownian simulation is



The error is

Variance independent of correlations (no reduction)

Insights: the dynamics (advection and dissipation) do not mix different points in space. Thus, the error only ‘sees’ diagonal elements of the correlations, which are fixed by the constraints.



An “integral-type” system (x[0,1]):



Closable:

Example 2



Results:

Brownian simulation

The stress

n = 2000 with spatially uniform noise ( c(x,y) = 1 ).(BCF)


Large error (1.47)





“smooth” simulation


(dotted curve)

Brownian simulation

Stress

Results:


(dotted curve)

“noisy” simulations

n = 2000 with piecewise constant uncorrelated noise (CONNFFESSIT).


optimal error (1.06)





Why?…

the optimal error is obtained by taking c 0 (CONNFESSIT).



• g(x,y,t) is singular on the diagonal (x=y), and a smooth positive function off the diagonal.

• c(x,x) = 1

The error is

Example 3: 1-D planar Shear flow model. (Jourdain et al. 2004)

The system:







Closable: set QuickTime™ and a




To leading order, the error of the Brownian simulations is

• C - the spatial correlation operator.



• K(t) - a nonnegative bounded operator.



• Any sequence ck 0 yields the optimal error (e.g, spatially constant uncorrelated)



So is CONFFESSIT always optimal?

• No!

We can construct a problem for which e(n,t) = n-1(const + Tr[K(t)C]) for K(t) bounded and not PSD.

Theorem. If the semi-groups are Hilbert-Schmidt (they have L2-kernels) then CONNFFESSIT is optimal.

Some further thoughts…

• The spatial correlation of the initial data q(x,0) may also be considered.

• Non-closable systems?

• Gain insights about the optimal correlation by understand relations between type of equation and optimal correlation.

variance reduction and brownian simulation methods

Documents

brownian noise

brownian simulations4

brownian simulation

brownian motion bt

spatial correlationsbx

spatial correlations3

brownian motion trace

velocity x