a narrow-stencil finite difference method for hamilton-jacobi-bellman equations · 2016-11-25 · a...

A Narrow-Stencil Finite Difference Method forHamilton-Jacobi-Bellman Equations

Xiaobing Feng

Department of MathematicsThe University of Tennessee, Knoxville, U.S.A.

Linz, November 23, 2016

Collaborators

Tom Lewis, North Carolina Stefan Schnake, Tennessee

The work to be presented here has been partially supported by NSF

Outline

• Motivation and Background

• A Narrow-Stencil Finite Difference Method

• High Order Extensions

• Numerical Experiments

• Conclusion

We consider second order fully nonlinear PDEs

F (D2u,Du,u, x) = 0

Two best known classes of equations:

• Monge-Ampére equation: det(D2u) = f• HJB equations: infν∈V (Lνu − fν) = 0, where

Lνu := Aν(x) : D2u + bν(x) · ∇u + cν(x)u, ν ∈ V.

Both equations arise from many applications such asdifferential geometry, optimal mass transfer, stochastic optimalcontrol, mathematical finance etc.

Remark: cν ≡ 0 in several applications (e.g., stochastic optimalcontrol, Bellman reformulation of Monge-Ampère equation).

Example: (Stochastic Optimal Control) Suppose a stochasticprocess x(τ) is governed by the stochastic differential equation

dx(τ) = f (τ,x(τ),u(τ)) dt + σ (τ,x(τ),u(τ)) dW (τ), τ ∈ (t ,T ]

x(t) = x ∈ Ω ⊂ Rn,

W : Wiener process u : control vector

and let

J (t , x ,u) = E t x

[∫ T

tL (τ,x(τ),u(τ)) dτ + g (x(T ))

].

Stochastic optimal control problem involves minimizingJ (t , x ,u) over all u ∈ U for each (t , x) ∈ (0,T ]× Ω.

Bellman Principle

Suppose u∗ ∈ U such that

u∗ ∈ argminu∈U

J (t , x ,u) ,

and define the value function

v (t , x) = J (t , x ,u∗) .

Then, v is the minimal cost achieved starting from the initialvalue x(t) = x , and u∗ is the optimal control that attains theminimum.

Bellman Principle (Continued)

Let Ω ⊂ Rn, T > 0, and U ⊂ Rm. The Bellman Principle says vis the solution of

vt = F (D2v ,∇v , v , x , t) in (0,T ]× Ω, (1)

for

F (D2v ,∇v , v , x , t) = infu∈U

(Luv − hu) ,

Luv =n∑

i=1

n∑j=1

aui,j(t , x)vxi xj +

n∑i=1

bui (t , x)vxi + cu (t , x) v

with

Au :=12σσT bu := f (t , x ,u)

cu := 0 hu := L (t , x ,u)

Ellipticity

Definition: Let F [u] := F (D2u,∇u,u, x) and A,B ∈ SL(n).

(a) F is said to be uniformly elliptic if ∃Λ > λ > 0 such that

λtr(A− B) ≤ F (A,p, r , x)− F (B,p, r , x) ≤ Λtr(A− B) ∀A ≥ B.

(b) F is said to be proper elliptic if

F (A,p, v , x) ≤ F (B,p,w , x) ∀A ≥ B; v ,w ∈ Rd , v ≤ w .

(c) F is said to be degenerate elliptic if

F (A,p, r , x) ≤ F (B,p, r , x) ∀A ≥ B.

Viscosity solutions

Definitions: Assume F is elliptic in a function class A ⊂ B(Ω)(set of bounded functions),

(i) u ∈ A is called a viscosity subsolution of F [u] = 0 if∀ϕ ∈ C2, when u∗ − ϕ has a local maximum at x0 then

F∗(D2ϕ(x0),Dϕ(x0),u∗(x0), x0) ≤ 0

(ii) u ∈ A is called a viscosity supersolution of F [u] = 0 if∀ϕ ∈ C2, when u∗ − ϕ has a local minimum at x0 then

F ∗(D2ϕ(x0),Dϕ(x0),u∗(x0), x0) ≥ 0

(iii) u ∈ A is called a viscosity solution of F [u] = 0 if u is both asub- and supersolution of F [u] = 0

where u∗(x) := lim supx ′→x

u(x ′) and u∗(x) := lim infx ′→x

u(x ′) are the

upper and lower semi-continuous envelops of u

Barles-Souganidis Framework I

For approximating viscosity solutions, we first recall theBarles-Souganidis framework.

Theorem (Barles-Souganidis (’91))Suppose that the elliptic problem F [u](x) = 0 in Ω satisfies thecomparison principle. Assume that the (approximation)operator S : R+ × Ω× R× B(Ω)→ R is consistent, monotoneand stable (as well as admissible), then the solution uρ ofproblem:

S(ρ, x ,uρ(x),uρ) = 0 in Ω,

converges locally uniformly to the unique viscosity of u.

Barles-Souganidis Framework II

(i) Admissibility and Stability. For all ρ > 0, there exists a solutionuρ ∈ B(Ω) to the following problem:

S(ρ, x ,uρ(x),uρ) = 0 in Ω.

Moreover, there exists a ρ-independent constant C > 0 such that‖uρ‖L∞(Ω) ≤ C.

(ii) Monotonicity. For all x ∈ Ω, t ∈ R and ρ > 0

S(ρ, x , t ,u) ≤ S(ρ, x , t , v) ∀u, v ∈ B(Ω), u ≥ v .

(iii) Consistency. For all x ∈ Ω and φ ∈ C∞(Ω) there hold

lim supρ→0y→xξ→0

S(ρ, y , φ(y) + ξ, φ+ ξ)

ρ≤ F (D2φ(x),∇φ(x), φ(x), x),

lim infρ→0y→xξ→0

S(ρ, y , φ(y) + ξ, φ+ ξ)

ρ≥ F (D2φ(x),∇φ(x), φ(x), x).

Wasow-Motzkin TheoremTheorem (Wasow-Motzkin (’53))Any monotone and consistent method has to be a wide stencilscheme.

Theorem (Bonnans and Zidani (’03))If Aν in the HJB equation is not diagonally dominant, then widestencils are required to preserve monotonicity.

Remark: The difficulty is the directional resolution. Widerstencils are used to increase the resolution.

Remarks on Monotone Schemes and Wide-stencils

I Why monotonicity? For the numerical scheme to identifythe correct viscosity solution, it needs to respect orderingin some sense. The monotonicity provides such anordering, which is the best known one so far.

I A “drawback" of monotonicity is that one must use widestencils according to Wasow-Motzkin (’53), which could bevery problematic for anisotropic problems because veryfine directional resolution is required, besides the difficultyfrom handling boundary conditions.

I Consequently, in order to avoid using wide stencils, onemust relax (or abandon) the concept of monotonicity (in thesense of Barles and Souganidis).

Outline





• Conclusion

Goals:

I To construct finite difference methods (FDMs) whosesolutions converge to viscosity solutions of the underlyingfully nonlinear 2nd order PDE problems, especially, to gobeyond the domain of Barles-Souganidis’ framework andto be more suitable for FDMs and DG methods.

Remark: A few existing “narrow-stencil" methods areI Glowinski et al. (’04-’12): Mixed FE for MA eqns (H2 solns).I Brenner et al. (’09-’13): DG for MA eqns (classical solns).I Jensen-Smears (’12): Linear FE for isotropic HJB eqns.I Smears-Süli (’14): DG-FE for (Cordes-) HJB (H2 solns).I F.-Neilan (’07-’11): FE and DG based on the vanishing

moment approach.I · · ·

Finite Difference Operators

Let ejdj=1 denote the canonical basis of Rd . Define

δ+xk ,hk

v(x) ≡ v(x + hk ek )− v(x)

hk, δ−xk ,hk

v(x) ≡ v(x)− v(x− hk ek )

hk

δµνxk ,hkv(x) ≡ δνxk ,hk

δµxk ,hkv(x), δµνxk ,hk ;x`,h`v(x) ≡ δνx`,h`δ

µxk ,hk

v(x)

Discrete Gradients: Two natural "sided" choices[∇±h]

k ≡ δ±xk ,hk

Discrete Hessians: Four natural "sided" choices[Dµν

h]

k ,` ≡ δµνxk ,hk ;x`,h`

, µ, ν ∈ −,+

Remark: Low-regularity can be resolved by using “sided"gradient and Hessian approximations.

Ideas Used for 1st Order Hamilton-Jacobi Equations

(Crandall and Lions, ’84) FD schemes with the form

H(∇−h Uα,∇+h Uα,Uα, xα) = 0

converge to the viscosity solution of a Hamilton-Jacobi equationassuming

I Consistency: H(q,q,u, x) = H(q,u, x),

I Monotonicity: H(↑, ↓,u, x).

H is called a numerical Hamiltonian.

I H is a function of both ∇−h Uα and ∇+h Uα.

I The monotonicity requirement is compatible with a discrete firstderivative test.

Vanishing Viscosity and Numerical Viscosity

H(∇u,u, x) = 0 7−→ −ε∆uε + H(∇uε,uε, x) = 0

(E. Tadmor, ’97) Every convergent monotone finite differencescheme for HJ equations implicitly approximates the differentialequation

−βh“∆u” + H(∇u, x) = 0

for sufficiently large and possibly nonlinear β > 0, where −βh“∆u” iscalled a numerical viscosity.

Note: If hk ≡ h, then 1 ·(∇+

h Uα −∇−h Uα)

= h∆hUα ≡ hd∑

k=1

δ2xk ,hUα.

Lax-Friedrichs numerical Hamiltonian

H(q−,q+,u, x) ≡ H(q− + q+

2,u, x

)− b · (q+ − q−)︸︷︷︸

Numerical Viscosity

FD Approximations of F (D2u,∇u,u, x) = 0

I Since uxi xj may be discontinuous at xα, it needs beapproximated from multiple directions.

I There are 4 possible FD approximations of uxi ,xj (xα):

uxi ,xj (xα) ≈ δµxiδνxj

u(xα) µ, ν ∈ +,−.

andD2u(xα) ≈ Dµν

h u(xα) µ, ν ∈ +,−.

FD Approximations of F (D2u,∇u,u, x) = 0

Inspired by the above observations, we propose the followingform of FD methods for F (D2u,∇u,u, x) = 0:

F (D++h Uα,D+−

h Uα,D−+h Uα,D−−h Uα,∇−h Uα,∇+

h Uα,Uα, xα) = 0

F is called a numerical operator, which needs to satisfy

I Consistency: F (P,P,P,P,q,q,u, x) = F (P,q,u, x),I Generalized Monotonicity: F (↑, ↓, ↓, ↑, ↑, ↓, ↑, x), uses the

natural (partial) orderings for symmetric matrices, vectors,and scalars

I Solvability/Admissibility and Stability: ∃h0 > 0 andC0 > 0, which is independent h, such that F [Uα, xα] = 0has a (unique) solution U and ‖U‖`∞(Th) < C0 for h < h0.

Remarks on Numerical Operator F

I F depends on all four "sided" Hessians and both "sided"gradients.

I The generalized monotonicity requirement is an extensionof the Crandall and Lions framework that enforces theelliptic structure of the PDE with respect to the mixeddiscrete Hessians D±∓h .

I If F is not continuous, F may not be continuous either. Inthis case the consistency definition should be replaced by

lim infPk →P,r→qs→t,y→x

F (P1,P2,P3,P4, r, s, y) ≥ F∗(P,q, t , x),

lim supPk →P,r→qn

s→t,y→x

F (P1,P2,P3,P4, r, s, y) ≤ F∗(P,q, t , x).

I Above consistency and monotonicity are different fromBarles-Souganidis’ definitions.

Examples of Numerical Operators F I

Lax-Friedrichs-like schemes: 1-D (F-Lewis-Kao, ’14)

F1(p1,p2,p3, x) := F(p1 + p2 + p3

3, x)

+ α(p1 − 2p2 + p3)

F2(p1,p2,p3, x) := F (p2, x) + α(p1 − 2p2 + p3)

F3(p1,p2,p3, x) := F(p1 + p3

2, x)

+ α(p1 − 2p2 + p3)

Examples of Numerical Operators F II

Godunov-like schemes: 1-D (F-Lewis-Kao, ’14):

F4(p1,p2,p3, x) := extp∈I(p1,p2,p3)

F (p, x)

where I(p1,p2,p3) := [p1 ∧ p2 ∧ p3, p1 ∨ p2 ∨ p3] and

extp∈I(p1,p2,p3)

:=

minp∈I(p1,p2,p3)

if p2 ≥ maxp1,p3,

maxp∈I(p1,p2,p3)

if p2 ≤ minp1,p3,

minp1≤p≤p2

if p1 < p2 < p3,

minp3≤p≤p2

if p3 < p2 < p1.

Examples of Numerical Operators F III

Godunov-like schemes: 1-D (F-Lewis-Kao, ’14) (continued):

F5(p1,p2,p3, x) := extrp∈I(p1,p2,p3)

F (p, x)

where I(p1,p2,p3) := [p1 ∧ p2 ∧ p3, p1 ∨ p2 ∨ p3] and

extrp∈I(p1,p2,p3)

:=

minp∈I(p1,p2,p3)

if p2 ≥ maxp1,p3,

maxp∈I(p1,p2,p3)

if p2 ≤ minp1,p3,

maxp2≤p≤p3

if p1 < p2 < p3,

maxp2≤p≤p1

if p3 < p2 < p1.

Higher Dimensional Lax-Friedrichs-like SchemesCentral Discrete Hessian:

D2hUα ≡

14(D++

h + D+−h + D−+

h + D−−h)

Uα

Central Discrete Gradient:

∇hUα ≡12(∇+

h +∇−h)

Uα

Lax-Friedrichs-like Numerical Operator:

F [Uα, xα] ≡ F(

D2hUα,∇2

hUα,Uα, xα)

+ A(Uα,xα) :(D++

h Uα − D+−h Uα − D−+

h Uα + D−−h Uα

)− b(Uα,xα) ·

(∇+

h Uα −∇−h Uα

)

Numerical Moment I

The discrete operator

A(Uα,xα) :(D++



)is called a numerical moment.

Observation:(δ+

xk ,hkδ+

x`,h`− δ+

xk ,hkδ−x`,h` − δ

−xk ,hk

δ+x`,h`

+ δ−xk ,hkδ−x`,h`

)Uα

= hkh`δ2xk ,hk

δ2x`,h`Uα

= hkh`δ2x`,h`δ

2xk ,hk

Uα,

an O(h2

k + h2`

)approximation of uxk xk x`x`(xα) scaled by hkh`.

1d×d :(D++



)≈ h2∆2u(xα),

Numerical Moment II

Remark:I Here the numerical moment plays a similar role for the 2rd

order fully nonlinear PDEs to what the numerical viscositydoes for the 1st order fully nonlinear PDEs.

I The introduction of numerical moments is very muchconsistent with the vanishing moment method (at the PDElevel) proposed and analyzed by F. and Neilan (’07-’12):

ε∆2uε + F (D2uε,∇uε,uε, x) = 0,

with the special choice of the parameter ε = αh2.I Is there an analogue of E. Tadmor’s result for 2rd order

PDEs?

Convergence I

Assumption: F is uniformly elliptic and Lipschitz continuous inthe Hessian argument, and A and ~b are uniformly bounded forthe family of linear operators defining the HJB problem withDirichlet boundary data. Assume c ≡ 0.

Theorem (F-Lewis, ’16)The Lax-Friedrichs-like scheme is admissible, consistent,stable, and generalized-monotone.

Main ingredients of proof:

I Consistency is trivial.I Generalized monotonicity is also “easy" by choosing the

numerical moment large enough (related to the Lipschitzconstant of F ).

Convergence II

I Admissibility and stability are the most difficult to verify.The idea is to show that the mappingMρ : Uα → Uα

defined by∆hUα = ∆hUα − ρF [Uα]

is monotone and a contraction for small enough ρ > 0. ∆his an enhanced discrete Laplacian.

I Stability follows from applying Crandall-Tartar lemma toMρ which commutes with additive constants (this is thereason to assume c ≡ 0).

Convergence IIITheorem (F-Lewis, ’16)In addition, suppose F [u] = 0 satisfies the comparisonprinciple. Let U be the solution to the Lax-Friedrichs-likescheme and uh be its piecewise constant extension. Then uhconverges to u locally uniformly as h→ 0+.

Main ingredients of proofI Follow and modify Barles-Souganidis’ proof to show that

u(x) := lim inf uh(x) and u(x) := lim sup uh(x) arerespectively viscosity super- and sub-solution.

I First work with quadratic test functions, then with generaltest functions.

I Use the numerical moment to control the off-diagonalentries in the discrete Hessians (this is the most difficultand technical step).

I Use the consistency and comparison principle to get u = u.

The Local Stencil (2D)

I The extra nodes in the Cartesian directions smooth theapproximation through the diagonal components of thevanishing moment.

I Directional resolution is no longer necessary avoiding theneed for a wide-stencil.

Overcoming a Lack of Monotonicity

xk

v

x0

δ2xk ,hk

v(x0 ± hk ek ) < δ2xk ,hk

v(x0) < 0

δ2xk ,hk

v(x0)− δ2xk ,hk

v(x0) < 0

xk

v

x0

δ2xk ,hk

v(x0 ± hk ek ) > 0 and δ2xk ,hk

v(x0) < 0

δ2xk ,hk

v(x0)− δ2xk ,hk

v(x0) > 0

Outline





• Conclusion

ExtensionsGoals: To develop high order methods and to use unstructuredmeshes

I “Bad" news: It is not possible to construct higher than 2ndorder monotone FD schemes (we have yet given up sinceg-monotonicity is a weaker requirement)

I “Good" news: Inspired by a work of Yan-Osher (JCP, ’11)for fully nonlinear 1st order Hamilton-Jacobi equations, weare able to construct high order MDG (mixed DG) and LDG(local DG) methods.

I Ideas of MDG: write F (uxx , x) = 0 as

F (↑

pR ,↓

pM ,↑

pR , x) = 0

pL = u−xx

pM = uaxx

pR = u+xx

and use different numerical fluxes on the linear equations.

Extensions (continued)

I Ideas of LDG: write F (uxx ,ux ,u, x) = 0 as

F (↑

p1,↓

p2,↓

p3,↑

p4,↑

q1,↓

q2,u, x) = 0,

q1 = ux (x−),

q2 = ux (x+),

p1 = q1x (x−),

p2 = q1x (x+),

p3 = q2x (x−),

p4 = q2x (x+),

where ux (x−) (resp. qjx (x−)) and ux (x+) (resp. qjx (x+))denote the left and right limits of ux (resp. qjx ) at x . Theydictate how numerical fluxes should be chosen in thediscretization. ↑ and ↓ stands for monotone increasingand decreasing, respectively. r = 0 is allowed!

Outline





• Conclusion

1-D simulationsTest 1: Consider the problem

−u2xx + 1 = 0, 0 < x < 1

u(0) = 0, u(1) = 0.5

This problem has the pointwise solutions

u+(x) = 0.5x2 (convex), u−(x) = −0.5x2 + x (concave)

Computed using Lax-Friedrichs-like scheme F1 with α = 1 (left)and α = −1 (right)

Test 2: Consider the problem

minθ∈1,2

−Aθuxx − S(x) = 0, −1 < x < 1

u(−1) = −1, u(1) = 1

A1 = 1, A2 = 2, S(x) =

12x2, if x < 0−24x2, if x ≥ 0

This problem has the exact solution u(x) = x |x |3

Test 3: Consider the problem

−u3xx + 8 sign(x) = 0, −1 < x < 1

u(−1) = −1, u(1) = 1.

This problem has the exact solution u(x) = x |x | ∈ C1([−1,1])which is not classical

2-D simulationsTest 4: Let Ω = (0,1)2. Consider Monge-Ampére equation

det(D2u) = 1 in Ω

u = 0 on ∂Ω

Remark: No explicit solution formula and solution is notclassical.

( α < 0: concave solution) ( α > 0: convex solution)

Test 5: Let Ω = (−1,1)2. Consider Monge-Ampére equation

det(D2u(x , y)) = 0 in Ω

u = g on ∂Ω

Choose g such that the exact solution is u(x , y) = |x |.

hx L∞ norm order L2 norm order2.50E-01 3.86E-02 3.42E-021.25E-01 2.08E-02 0.89 1.85E-02 0.888.33E-02 1.38E-02 1.02 1.24E-02 0.99

Top: α = I, r = 1, and hy = 1/3 fixed. Bottom: α = I, r = 1, andodd number of intervals in the x-direction.

3-D Simulations

Test 6: Let Ω = (0,1)3. Consider Monge-Ampére equation

det(D2u) = f in Ω

u = g on ∂Ω

f (x , y , z) = (1 + x2 + y2 + z2)e3(x2+y2+z2)

2

g(x , y , z) =

ey2+z2

2 if x = 0

ex2+z2

2 if y = 0

ex2+y2

2 if z = 0

ey2+z2+1

2 if x = 1

ex2+z2+1

2 if y = 1

ex2+y2+1

2 if z = 1

The exact solution is u0(x , y , z) = ex2+y2+z2

2

(computed solution) (error function)

Test 7: Let Ω = (0,1)3. Consider Monge-Ampére equation

det(D2u) = 1 in Ω

u = 0 on ∂Ω

Remark: No explicit solution formula and solution is notclassical.

computed solution (α > 0)

Outline





• Conclusion

Concluding RemarksI The generalized monotone structure allows us to deal with low

regularity functions by considering multiple gradient and Hessianapproximations.

I The key new concept is that of a numerical moment which canbe used to design generalized monotone methods such as theLax-Friedrichs-like scheme.

I By using narrow stencil schemes that can be expressed usingonly forward and backward difference quotients, the methodscan be easily extended to higher-order and non-Cartesiandomains/grids using the DG techniques (or DG Finite ElementCalculus).

I Formulation easily extends to Monge-Ampère type equationseither directly or based on its HJB reformulation, although theanalysis is not yet.

I Many open problems/issues: such as extension to degeneratePDES, rate of convergence in C0-norm, boundary layers,parabolic PDEs, nonlinear solvers, etc.

References

I XF, R. Glowinski and M. Neilan, “Recent Developments inNumerical Methods for Fully Nonlinear 2nd Order PDEs", SIAMReview, 55:1-64, 2013

I M. Neilan, A. Salgado and W. Zhang, “Numerical Analysis ofStrongly Nonlinear PDEs", arxiv.org/abs/1610.07992, toappear in Acta Numerica.

I Check arxiv.org for more (and new) references.

Thanks for Your Attention!

a narrow-stencil finite difference method for hamilton-jacobi-bellman equations · 2016-11-25 · a...

Documents