with contributions from: michael jacobsen, toke koldborg jensen - phd students

Large-Scale Methods in Inverse Problems 1

With contributions from:

• Michael Jacobsen, Toke Koldborg Jensen - PhD students

• Line H. Clemmensen, Iben Kraglund, Kristine Horn,Jesper Pedersen, Marie-Louise H. Rasmussen - Master students

Large-Scale Methods in Inverse Problems

Per Christian Hansen

Informatics and Mathematical Modelling

Technical University of Denmark

Large-Scale Methods in Inverse Problems 2

Overview of Talk

A survey of numerical methods for large-scale inverse problems

1. Some examples.

2. The need for regularization algorithms.

3. Krylov subspace methods for large-scale problems.

4. Preconditioning for regularization problems.

5. Signal subspaces and (semi)norms.

6. GMRES as a regularization method.

7. Alternatives to spectral filtering.

Many details are skipped, to get the big picture!!!

Large-Scale Methods in Inverse Problems 3

Related WorkMany people work on similar problems and algorithms:• Åke Björck, Lars Eldén, Tommy Elfving• Martin Hanke, James G. Nagy, Robert Plemmons• Misha E. Kilmer, Dianne P. Oleary• Daniela Calvetti, Lothar Reichel, Brian Lewis• Gene H. Golub, Urs von Matt• Uri Asher, Eldad Haber, Douglas Oldenburg• Jerry Eriksson, Mårten Gullikson, Per-Åke Wedin• Marielba Rojas, Trond Steihaug• Tony Chan, Stanley Osher, Curtis R. Vogel• Jesse Barlow, Raymond Chan, Michael Ng

Recent Matlab software packages:• Restore Tools (Nagy, Palmer, Perrone, 2004)• MOORe Tools (Jacobsen, 2004)• GeoTools (Pedersen, 2005)

Large-Scale Methods in Inverse Problems 4

Inverse Geomagnetic Problems

Large-Scale Methods in Inverse Problems 5

Inverse Acoustic Problems

Oticon/Rhinometrics

Large-Scale Methods in Inverse Problems 6

Image Restoration Problems

blurring

deblurring

Io (moon of Saturn)

You cannot depend on your eyes whenyour imagination is out of focus

– Mark Twain

Large-Scale Methods in Inverse Problems 7

Model Problem and Discretization

Vertical component ofmagnetic field from a dipole

Large-Scale Methods in Inverse Problems 8

The Need for Regularization

Regularization:

keep the “good” SVD components

and discard the noisy ones!

Large-Scale Methods in Inverse Problems 9

Regularization – TSVD & Tikhonov

Large-Scale Methods in Inverse Problems 10

Singular Vectors (Always) Oscillate

Large-Scale Methods in Inverse Problems 11

Large-Scale Aspects (the easy case)

Large-Scale Methods in Inverse Problems 12

Large-Scale Aspects (the real problems)

Toeplitz matrix-vectormultiplication flop count.

Large-Scale Methods in Inverse Problems 13

Large-Scale Tikhonov Regularization

Large-Scale Methods in Inverse Problems 14

Difficulties and Remedies I

Large-Scale Methods in Inverse Problems 15

Difficulties and Remedies II

Large-Scale Methods in Inverse Problems 16

The Art of Preconditioning

Large-Scale Methods in Inverse Problems 17

Explicit Subspace Preconditiong

Large-Scale Methods in Inverse Problems 18

Krylov Signal Subspaces

Smiley Crater, Mars

Large-Scale Methods in Inverse Problems 19

Pros and Cons of Regularizing Iterations

Large-Scale Methods in Inverse Problems 20

Projection, then Regularization

Large-Scale Methods in Inverse Problems 21

Bounds on “Everything”

Large-Scale Methods in Inverse Problems 22

A Dilemma With Projection + Regular.

Large-Scale Methods in Inverse Problems 23

Better Basis Vectors!

Large-Scale Methods in Inverse Problems 24

Considerations in 2D

…

…

Large-Scale Methods in Inverse Problems 25

Good Seminorms for 2D Problems

Large-Scale Methods in Inverse Problems 26

Seminorms and Regularizing Iterations

Large-Scale Methods in Inverse Problems 27

Krylov Implementation

Large-Scale Methods in Inverse Problems 28

Avoiding the Transpose: GMRES

Large-Scale Methods in Inverse Problems 29

GMRES and CGLS Basis Vectors

Large-Scale Methods in Inverse Problems 30

CGLS and GMRES Solutions

Large-Scale Methods in Inverse Problems 31

The “Freckles’’

DCT spectrum spatial domain

Large-Scale Methods in Inverse Problems 32

Preconditioning for GMRES

Large-Scale Methods in Inverse Problems 33

A New and Better Approach

Large-Scale Methods in Inverse Problems 34

(P)CGLS and (P)GMRES

Large-Scale Methods in Inverse Problems 35

Away From 2-Norms

Io (

moo

n of

Sat

urn)

q = 2 q = 1.1

Large-Scale Methods in Inverse Problems 36

Functionals Defined on Sols. to DIP

Large-Scale Methods in Inverse Problems 37

Large-Scale Algorithm MLFIP

Large-Scale Methods in Inverse Problems 38

Confidence Invervals with MLFIP

Large-Scale Methods in Inverse Problems 39

• Algorithms for other norms (p and q ≠ 2).• In particular, total variation (TV).• Nonnegativity constraints.• General linear inequality constraints.• Compression of dense coefficient matrix A.• Color images (and color TV).• Implementation aspects and software.• The choice the of regularization parameter.

Many Topics Not Covered …

Large-Scale Methods in Inverse Problems 40

“Conclusions and Further Work” I hesitate to give any conclusion –

• the work is ongoing;• there are many open problems,• lots of challenges (mathematical and numerical),• and a multitude of practical problems waiting to be solved.