# hierarchical bayesian level set inversion

Post on 05-Dec-2021

0 views

Embed Size (px)

TRANSCRIPT

Hierarchical Bayesian Level Set Inversion Matt Dunlop

SIAM UQ 2016, Lausanne, Switzerland,

April 7th.

Matt Dunlop (University of Warwick) Hierarchical Bayesian Level Set Inversion April 7th 2016 1 / 21

Outline

4 Hierarchical Bayesian Level Set Inversion

5 Conclusions

Matt Dunlop (University of Warwick) Hierarchical Bayesian Level Set Inversion April 7th 2016 2 / 21

Outline

4 Hierarchical Bayesian Level Set Inversion

5 Conclusions

Matt Dunlop (University of Warwick) Hierarchical Bayesian Level Set Inversion April 7th 2016 2 / 21

Level Set Representation

S.Osher and J.Sethian Fronts propagating with curvature-dependent speed . . . . J. Comp. Phys. 79(1988), 12–49.

Piecewise constant function v defined through thresholding a continuous level set function u. Let

1 = c0 < c1 < · · · < cK1 < cK = 1.

v(x) = KX

k=1

vk I{ck1<uck}(x); v = F (u).

F : X ! Z is the level-set map, X cts fns. Z piecewise cts fns. Matt Dunlop (University of Warwick) Hierarchical Bayesian Level Set Inversion April 7th 2016 3 / 21

Example 1: Image Reconstruction

H.W. Engl, M. Hanke and A. Neubauer Regularization of Inverse Problems Kluwer(1994)

Forward Problem Define K : Z ! RJ by (Kv)j = v(xj), xj 2 D Rd . Given v 2 Z

y = Kv .

Let 2 RJ be a realization of an observational noise.

Inverse Problem Given prior information v = F (u), u 2 X and y 2 RJ , find v :

y = Kv + .

Matt Dunlop (University of Warwick) Hierarchical Bayesian Level Set Inversion April 7th 2016 4 / 21

Example 2: Groundwater Flow M. Dashti and A.M. Stuart The Bayesian approach to inverse problems. Handbook of Uncertainty Quantification Editors: R. Ghanem, D.Higdon and H. Owhadi, Springer, 2017. arXiv:1302.6989

Forward Problem: Darcy Flow

Let X+ := {v 2 Z : essinfx2Dv > 0}. Given 2 X+, find y := G() 2 RJ

where yj = `j(p),V := H1 0 (D), `j 2 V , j = 1, . . . , J and

( r · (rp) = f in D p = 0 on @D.

Let 2 RJ be a realization of an observational noise.

Inverse Problem Given prior information = F (u), u 2 X and y 2 RJ , find :

y = G() + .

Matt Dunlop (University of Warwick) Hierarchical Bayesian Level Set Inversion April 7th 2016 5 / 21

D

el

Example 3: Electrical Impedance Tomography Apply currents I` on e`, ` = 1, . . . , L. Induces voltages ` on e`, ` = 1, . . . , L. Input is (, I), output is (,). We have an Ohm’s law = R()I.

8 >>>>>>>>>><

e`

SL `=1 e`

(x) + z`(x) @

(PDE)

Matt Dunlop (University of Warwick) Hierarchical Bayesian Level Set Inversion April 7th 2016 6 / 21

Example 3: Electrical Impedance Tomography

M. M. Dunlop and A. M. Stuart The Bayesian formulation of EIT. arXiv:1509.03136 Inverse Problems and Imaging, Submitted, 2015.

Forward Problem Let X L1(D), and denote X+ := {u 2 X : essinfx2D u > 0}. Given u 2 X ,F : X ! X+ and = F (u), find (,) 2 H1(D) RL

solving (PDE). This gives = R(F (u))I.

Let 2 RL be a realisation of an observational noise.

Inverse Problem Given prior information = F (u), u 2 X , and given currents I and y 2 RL, find :

y = R()I + .

Matt Dunlop (University of Warwick) Hierarchical Bayesian Level Set Inversion April 7th 2016 7 / 21

Outline

4 Hierarchical Bayesian Level Set Inversion

5 Conclusions

Matt Dunlop (University of Warwick) Hierarchical Bayesian Level Set Inversion April 7th 2016 7 / 21

Tikhonov Regularization

G : Z ! RJ and 2 RJ a realization of an observational noise.

Inverse Problem Find u 2 X , given y 2 RJ satisfying y = G F (u) + .

Model-Data Misfit

1 2 y G F (u)

2 .

Tikhonov Regularization Minimize, for some E compactly embedded into X ,

I(u; y) := (u; y) + 1 2 kuk2

E .

Matt Dunlop (University of Warwick) Hierarchical Bayesian Level Set Inversion April 7th 2016 8 / 21

Issue 1: Discontinuity of Level Set Map

F (·) is continuous at u F (·) is discontinuous at u

v = F (u) := v+ I{u0}(x) + v I{u<0}(x).

Causes problems in classical level set inversion.

Matt Dunlop (University of Warwick) Hierarchical Bayesian Level Set Inversion April 7th 2016 9 / 21

Issue 2: Length-Scale Matters

Figure demonstrates role of length-scale in level-set function. Classical Tikhonov-Phillips regularization does not allow for direct control of the length-scale.

New ideas needed.

Matt Dunlop (University of Warwick) Hierarchical Bayesian Level Set Inversion April 7th 2016 10 / 21

Issue 3: Amplitude Matters

Y. Lu Probabilistic Analysis of Interface Problems PhD Thesis (2016), Warwick University

Consider case of K = 2, c1 = 0. For contradiction assume u? is a minimizer of I(u; y). Now define the sequence u = u?. Then if 0 < < 1,

(u; y) = (u?; y), kukE = ku?kE ) I(u; y) < I(u?; y).

Hence I(u; y) is an infimizing sequence. But u ! 0 and I(0; y) > I(u; y) for 1. Thus infimum cannot be attained. This issue caused by thresholding at 0. But idea generalizes.

Choice of threshold matters.

Matt Dunlop (University of Warwick) Hierarchical Bayesian Level Set Inversion April 7th 2016 11 / 21

Outline

4 Hierarchical Bayesian Level Set Inversion

5 Conclusions

Matt Dunlop (University of Warwick) Hierarchical Bayesian Level Set Inversion April 7th 2016 11 / 21

Bayes’ Formula

µ0(du)

Likelihood Since y = G(u) + , if N(0, ), then y |u N(G(u), ). The model-data misfit is the negative log-likelihood:

P(y |u) / exp (u; y)

, (u; y) =

1 2

µy (du) / exp (u; y)

µ0(du).

Matt Dunlop (University of Warwick) Hierarchical Bayesian Level Set Inversion April 7th 2016 12 / 21

Rigorous Statement M. Iglesias, Y. Lu and A. M . Stuart A level-set approach to Bayesian geometric inverse problems. arXiv:1504.00313

M. Iglesias A regularizing iterative ensemble Kalman method for PDE constrained inverse problems. arXiv:1505.03876

L2 (X ;S) = {f : X ! S : Ekf (u)k2

S < 1}.

Theorem (Iglesias, Lu and S) Let µ0(du) = P(du) and µy (du) = P(du|y). Assume that u 2 X µ0a.s. Then for Examples 1–3 (and more) µy µ0 and y 7! µy (du) is Lipschitz in the Hellinger metric. Furthermore, if S is a separable Banach space and f 2 L2

µ0 (X ;S), then

S C|y1 y2|.

Key idea in proof is that F : X ! Z is continuous µ0 a.s.

Matt Dunlop (University of Warwick) Hierarchical Bayesian Level Set Inversion April 7th 2016 13 / 21

Whittle-Matern Priors

( |x y |)K( |x y |).

controls smoothness – draws from Gaussian fields with this covariance have fractional Sobolev and Hölder derivatives. is an inverse length-scale. is an amplitude scale.

Ignoring boundary conditions, corresponding covariance operator given by

CWM(,) / 22(2I 4) d 2 .

Matt Dunlop (University of Warwick) Hierarchical Bayesian Level Set Inversion April 7th 2016 14 / 21

Example 2: Groundwater Flow

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

1.5

2

2.5

3

3.5

4

4.5

5

5.5

6

6.5

Figure: (Row 1, = 15) Logarithm of the true hydraulic conductivity. (Row 2 = 10, 30, 50, 70, 90) samples of F (u). (Row 3) F (E(u)). (Row 4) E(F (u)).

Matt Dunlop (University of Warwick) Hierarchical Bayesian Level Set Inversion April 7th 2016 15 / 21

Outline

4 Hierarchical Bayesian Level Set Inversion

5 Conclusions

Matt Dunlop (University of Warwick) Hierarchical Bayesian Level Set Inversion April 7th 2016 15 / 21

Hierarchical Whittle-Matern Priors (The Problem)

Prior: P(u, ) = P(u|)P().

Recall CWM(,) / 22(2I 4) d 2 .

Theorem For fixed the family of measures N(0,CWM(,·)) are mutually singular.

Algorithms which attempt to move between singular measures perform badly under mesh refinement (since they fail completely in the fully resolved limit).

Matt Dunlop (University of Warwick) Hierarchical Bayesian Level Set Inversion April 7th 2016 16 / 21

Hierarchical Whittle-Matern Priors (The Solution) M. Dunlop, M. Iglesias and A. M . Stuart Hierarchical Bayesian Level Set Inversion arXiv:1601.03605

Hence choose = and define: C / (2I 4) d 2 . Prior:

P(u, ) = P(u|)P().

1 = c0 < c1 < · · · < cK1 < cK = 1.

v(x) = KX

k=1

vk I{ck1<uck}(x); v = F (u, ).

Matt Dunlop (University of Warwick) Hierarchical Bayesian Level Set Inversion April 7th 2016 17 / 21

Hierarchical Posterior

Likelihood

. where the

1 2 y G F (u, )

2 .

µy (du, d) / exp (u, ; y)

µ0(du, d).

Matt Dunlop (University of Warwick) Hierarchical Bayesian Level Set Inversion April 7th 2016 18 / 21

MCMC

We implement a Metropolis-within-Gibbs algorithm to generate a posterior-invariant Markov chain (uk , k ):

Propose vk+1 from a P(u|k )reversible kernel. uk+1 = vk+1 w.p. 1 ^ exp((uk , k ) (vk+1, k )), uk+1 = uk otherwise. Propose tk+1 from a P()reversible kernel. k+1 = tk+1 w.p. 1 ^ exp((uk+1, k ) (uk+1, tk+1))w(k , tk+1), k+1 = k otherwise.

Here w(, t) is density of N(0,Ct) with respect to N(0,C ).

Matt Dunlop (University of Warwick) Hierarchical Bayesian Level Set Inversion April 7th 2016 19 / 21

Example 2: Groundwater Flow

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

1.5

2

2.5

3

3.5

4

4.5

5

5.5

6

6.5

Figure: (Row 1, = 15) Logarithm of the true hydraulic conductivity (middle, top). (Row 2 = 10, 30, 50, 70, 90) samples of F (u, ). (Row 3) F (E(u),E()). (Row 4) E(F (u, )).

Matt Dunlop (University of Warwick) Hierarchical Bayesian Level Set Inversion April 7th 2016 20 / 21

Outline

4 Hierarchical Bayesian Level Set Inversion

5 Conclusions

Matt Dunlop (University of Warwick) Hierarchical Bayesian Level Set Inversion April 7th 2016 20 / 21

Summary Last 5 years development of a theoretical and computational framework for infinite dimensional Bayesian inversion with wide applicability:

1 Framework allows for noisy data and uncertain prior information. 2 Probabilistic well-posedness. 3 Theory clearly delineates (and links) analysis and probability. 4 Theory leads to new algorithms (defined on Banach space). 5 Grid-independent convergence rates for MCMC.

The methodology has been extended to solve interface problems. This is achieved via the level set representation.

1 Level set method becomes well-posed in this setting. 2 Discontinuity in level set map is a probability zero event. 3 Hierarchical choice of length-scale improves performance. 4 Amplitude scale is linked to length-scale via measure equivalence. 5 Algorithms which reflect this link are mesh-invariant.

Potential for many future developments for both applications and theory: 1 Applications: subsurface imaging, medical imaging. 2 Theory: inhomogenous length-scales, new hierarchies.

Matt Dunlop (University of Warwick) Hierarchical Bayesian Level Set Inversion April 7th 2016 21 / 21

Introduction

Conclusions

SIAM UQ 2016, Lausanne, Switzerland,

April 7th.

Matt Dunlop (University of Warwick) Hierarchical Bayesian Level Set Inversion April 7th 2016 1 / 21

Outline

4 Hierarchical Bayesian Level Set Inversion

5 Conclusions

Matt Dunlop (University of Warwick) Hierarchical Bayesian Level Set Inversion April 7th 2016 2 / 21

Outline

4 Hierarchical Bayesian Level Set Inversion

5 Conclusions

Matt Dunlop (University of Warwick) Hierarchical Bayesian Level Set Inversion April 7th 2016 2 / 21

Level Set Representation

S.Osher and J.Sethian Fronts propagating with curvature-dependent speed . . . . J. Comp. Phys. 79(1988), 12–49.

Piecewise constant function v defined through thresholding a continuous level set function u. Let

1 = c0 < c1 < · · · < cK1 < cK = 1.

v(x) = KX

k=1

vk I{ck1<uck}(x); v = F (u).

F : X ! Z is the level-set map, X cts fns. Z piecewise cts fns. Matt Dunlop (University of Warwick) Hierarchical Bayesian Level Set Inversion April 7th 2016 3 / 21

Example 1: Image Reconstruction

H.W. Engl, M. Hanke and A. Neubauer Regularization of Inverse Problems Kluwer(1994)

Forward Problem Define K : Z ! RJ by (Kv)j = v(xj), xj 2 D Rd . Given v 2 Z

y = Kv .

Let 2 RJ be a realization of an observational noise.

Inverse Problem Given prior information v = F (u), u 2 X and y 2 RJ , find v :

y = Kv + .

Matt Dunlop (University of Warwick) Hierarchical Bayesian Level Set Inversion April 7th 2016 4 / 21

Example 2: Groundwater Flow M. Dashti and A.M. Stuart The Bayesian approach to inverse problems. Handbook of Uncertainty Quantification Editors: R. Ghanem, D.Higdon and H. Owhadi, Springer, 2017. arXiv:1302.6989

Forward Problem: Darcy Flow

Let X+ := {v 2 Z : essinfx2Dv > 0}. Given 2 X+, find y := G() 2 RJ

where yj = `j(p),V := H1 0 (D), `j 2 V , j = 1, . . . , J and

( r · (rp) = f in D p = 0 on @D.

Let 2 RJ be a realization of an observational noise.

Inverse Problem Given prior information = F (u), u 2 X and y 2 RJ , find :

y = G() + .

Matt Dunlop (University of Warwick) Hierarchical Bayesian Level Set Inversion April 7th 2016 5 / 21

D

el

Example 3: Electrical Impedance Tomography Apply currents I` on e`, ` = 1, . . . , L. Induces voltages ` on e`, ` = 1, . . . , L. Input is (, I), output is (,). We have an Ohm’s law = R()I.

8 >>>>>>>>>><

e`

SL `=1 e`

(x) + z`(x) @

(PDE)

Matt Dunlop (University of Warwick) Hierarchical Bayesian Level Set Inversion April 7th 2016 6 / 21

Example 3: Electrical Impedance Tomography

M. M. Dunlop and A. M. Stuart The Bayesian formulation of EIT. arXiv:1509.03136 Inverse Problems and Imaging, Submitted, 2015.

Forward Problem Let X L1(D), and denote X+ := {u 2 X : essinfx2D u > 0}. Given u 2 X ,F : X ! X+ and = F (u), find (,) 2 H1(D) RL

solving (PDE). This gives = R(F (u))I.

Let 2 RL be a realisation of an observational noise.

Inverse Problem Given prior information = F (u), u 2 X , and given currents I and y 2 RL, find :

y = R()I + .

Matt Dunlop (University of Warwick) Hierarchical Bayesian Level Set Inversion April 7th 2016 7 / 21

Outline

4 Hierarchical Bayesian Level Set Inversion

5 Conclusions

Matt Dunlop (University of Warwick) Hierarchical Bayesian Level Set Inversion April 7th 2016 7 / 21

Tikhonov Regularization

G : Z ! RJ and 2 RJ a realization of an observational noise.

Inverse Problem Find u 2 X , given y 2 RJ satisfying y = G F (u) + .

Model-Data Misfit

1 2 y G F (u)

2 .

Tikhonov Regularization Minimize, for some E compactly embedded into X ,

I(u; y) := (u; y) + 1 2 kuk2

E .

Matt Dunlop (University of Warwick) Hierarchical Bayesian Level Set Inversion April 7th 2016 8 / 21

Issue 1: Discontinuity of Level Set Map

F (·) is continuous at u F (·) is discontinuous at u

v = F (u) := v+ I{u0}(x) + v I{u<0}(x).

Causes problems in classical level set inversion.

Matt Dunlop (University of Warwick) Hierarchical Bayesian Level Set Inversion April 7th 2016 9 / 21

Issue 2: Length-Scale Matters

Figure demonstrates role of length-scale in level-set function. Classical Tikhonov-Phillips regularization does not allow for direct control of the length-scale.

New ideas needed.

Matt Dunlop (University of Warwick) Hierarchical Bayesian Level Set Inversion April 7th 2016 10 / 21

Issue 3: Amplitude Matters

Y. Lu Probabilistic Analysis of Interface Problems PhD Thesis (2016), Warwick University

Consider case of K = 2, c1 = 0. For contradiction assume u? is a minimizer of I(u; y). Now define the sequence u = u?. Then if 0 < < 1,

(u; y) = (u?; y), kukE = ku?kE ) I(u; y) < I(u?; y).

Hence I(u; y) is an infimizing sequence. But u ! 0 and I(0; y) > I(u; y) for 1. Thus infimum cannot be attained. This issue caused by thresholding at 0. But idea generalizes.

Choice of threshold matters.

Matt Dunlop (University of Warwick) Hierarchical Bayesian Level Set Inversion April 7th 2016 11 / 21

Outline

4 Hierarchical Bayesian Level Set Inversion

5 Conclusions

Matt Dunlop (University of Warwick) Hierarchical Bayesian Level Set Inversion April 7th 2016 11 / 21

Bayes’ Formula

µ0(du)

Likelihood Since y = G(u) + , if N(0, ), then y |u N(G(u), ). The model-data misfit is the negative log-likelihood:

P(y |u) / exp (u; y)

, (u; y) =

1 2

µy (du) / exp (u; y)

µ0(du).

Matt Dunlop (University of Warwick) Hierarchical Bayesian Level Set Inversion April 7th 2016 12 / 21

Rigorous Statement M. Iglesias, Y. Lu and A. M . Stuart A level-set approach to Bayesian geometric inverse problems. arXiv:1504.00313

M. Iglesias A regularizing iterative ensemble Kalman method for PDE constrained inverse problems. arXiv:1505.03876

L2 (X ;S) = {f : X ! S : Ekf (u)k2

S < 1}.

Theorem (Iglesias, Lu and S) Let µ0(du) = P(du) and µy (du) = P(du|y). Assume that u 2 X µ0a.s. Then for Examples 1–3 (and more) µy µ0 and y 7! µy (du) is Lipschitz in the Hellinger metric. Furthermore, if S is a separable Banach space and f 2 L2

µ0 (X ;S), then

S C|y1 y2|.

Key idea in proof is that F : X ! Z is continuous µ0 a.s.

Matt Dunlop (University of Warwick) Hierarchical Bayesian Level Set Inversion April 7th 2016 13 / 21

Whittle-Matern Priors

( |x y |)K( |x y |).

controls smoothness – draws from Gaussian fields with this covariance have fractional Sobolev and Hölder derivatives. is an inverse length-scale. is an amplitude scale.

Ignoring boundary conditions, corresponding covariance operator given by

CWM(,) / 22(2I 4) d 2 .

Matt Dunlop (University of Warwick) Hierarchical Bayesian Level Set Inversion April 7th 2016 14 / 21

Example 2: Groundwater Flow

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

1.5

2

2.5

3

3.5

4

4.5

5

5.5

6

6.5

Figure: (Row 1, = 15) Logarithm of the true hydraulic conductivity. (Row 2 = 10, 30, 50, 70, 90) samples of F (u). (Row 3) F (E(u)). (Row 4) E(F (u)).

Matt Dunlop (University of Warwick) Hierarchical Bayesian Level Set Inversion April 7th 2016 15 / 21

Outline

4 Hierarchical Bayesian Level Set Inversion

5 Conclusions

Matt Dunlop (University of Warwick) Hierarchical Bayesian Level Set Inversion April 7th 2016 15 / 21

Hierarchical Whittle-Matern Priors (The Problem)

Prior: P(u, ) = P(u|)P().

Recall CWM(,) / 22(2I 4) d 2 .

Theorem For fixed the family of measures N(0,CWM(,·)) are mutually singular.

Algorithms which attempt to move between singular measures perform badly under mesh refinement (since they fail completely in the fully resolved limit).

Matt Dunlop (University of Warwick) Hierarchical Bayesian Level Set Inversion April 7th 2016 16 / 21

Hierarchical Whittle-Matern Priors (The Solution) M. Dunlop, M. Iglesias and A. M . Stuart Hierarchical Bayesian Level Set Inversion arXiv:1601.03605

Hence choose = and define: C / (2I 4) d 2 . Prior:

P(u, ) = P(u|)P().

1 = c0 < c1 < · · · < cK1 < cK = 1.

v(x) = KX

k=1

vk I{ck1<uck}(x); v = F (u, ).

Matt Dunlop (University of Warwick) Hierarchical Bayesian Level Set Inversion April 7th 2016 17 / 21

Hierarchical Posterior

Likelihood

. where the

1 2 y G F (u, )

2 .

µy (du, d) / exp (u, ; y)

µ0(du, d).

Matt Dunlop (University of Warwick) Hierarchical Bayesian Level Set Inversion April 7th 2016 18 / 21

MCMC

We implement a Metropolis-within-Gibbs algorithm to generate a posterior-invariant Markov chain (uk , k ):

Propose vk+1 from a P(u|k )reversible kernel. uk+1 = vk+1 w.p. 1 ^ exp((uk , k ) (vk+1, k )), uk+1 = uk otherwise. Propose tk+1 from a P()reversible kernel. k+1 = tk+1 w.p. 1 ^ exp((uk+1, k ) (uk+1, tk+1))w(k , tk+1), k+1 = k otherwise.

Here w(, t) is density of N(0,Ct) with respect to N(0,C ).

Matt Dunlop (University of Warwick) Hierarchical Bayesian Level Set Inversion April 7th 2016 19 / 21

Example 2: Groundwater Flow

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

1.5

2

2.5

3

3.5

4

4.5

5

5.5

6

6.5

Figure: (Row 1, = 15) Logarithm of the true hydraulic conductivity (middle, top). (Row 2 = 10, 30, 50, 70, 90) samples of F (u, ). (Row 3) F (E(u),E()). (Row 4) E(F (u, )).

Matt Dunlop (University of Warwick) Hierarchical Bayesian Level Set Inversion April 7th 2016 20 / 21

Outline

4 Hierarchical Bayesian Level Set Inversion

5 Conclusions

Matt Dunlop (University of Warwick) Hierarchical Bayesian Level Set Inversion April 7th 2016 20 / 21

Summary Last 5 years development of a theoretical and computational framework for infinite dimensional Bayesian inversion with wide applicability:

1 Framework allows for noisy data and uncertain prior information. 2 Probabilistic well-posedness. 3 Theory clearly delineates (and links) analysis and probability. 4 Theory leads to new algorithms (defined on Banach space). 5 Grid-independent convergence rates for MCMC.

The methodology has been extended to solve interface problems. This is achieved via the level set representation.

1 Level set method becomes well-posed in this setting. 2 Discontinuity in level set map is a probability zero event. 3 Hierarchical choice of length-scale improves performance. 4 Amplitude scale is linked to length-scale via measure equivalence. 5 Algorithms which reflect this link are mesh-invariant.

Potential for many future developments for both applications and theory: 1 Applications: subsurface imaging, medical imaging. 2 Theory: inhomogenous length-scales, new hierarchies.

Matt Dunlop (University of Warwick) Hierarchical Bayesian Level Set Inversion April 7th 2016 21 / 21

Introduction

Conclusions