presentation - signal processing and speech communication

TU Graz - Advanced Signal Processing 1, SeminarInstitute of Signal Processing and Speech Communication

Graz University of Technology

”Fighting the Curse of Dimensionality:Compressive Sensing in Exploration Seismology”

Herrmann, F.J.; Friedlander, M.P.; Yilmat, O.Signal Processing Magazine, IEEE, vol.29, no.3, pp.88-100

Andreas Gaich, Andrea Zabaznoska

Advanced Signal Processing 1, SeminarInstitute of Signal Processing and Speech Communication


Dec 10, 2012

Andreas Gaich, Andrea Zabaznoska Dec 10, 2012 page 1/68



Outline

Introduction

Mathematical Background of CS

The Wavelet-Domain

Reflection Seismology

Acquisition Schemes

Seismic Wavefield Representation

Compressive Seismic Computation and Imaging

SPGL1 Solver

Full Waveform Inversion and PGS

Pareto Curves




Introduction – On Compressive Sensing I

Shortcomings of the typical workflows

I Current seismic techniques rely on massive datavolumes as all conducted experiments produceenormous amounts of data.

I Most of the data consists of reflected energywithin a frequency content of roughly [5 - 100]Hz.

I Moving into geologically more complex areas ofthe Earth which makes the correction forwavepaths along which the reflected datatraveled impossible (geometry opticsapproximation of wave propagation).




Introduction – On Compressive Sensing II

Current trends

I Alternative sampling strategy that leveragesrecent insights from compressive sensing (CS)towards seismic data acquisition and processing.

I Introducing CS as a novel nonlinear samplingparadigm, a randomized dimensionalityreduction approach effective for acquiring signalsthat have a sparse representation in sometransform domain.




Introduction – On Compressive Sensing III

Current trends (cont’d)

I We need a workflow, a framework.

I We need to come up with sub-Nyquist samplingschemes whose sampling is proportional to thesparsity of the problem and not to thedimensionality.

I Luckily audio, image and seismic signals admitsparse approximations, i.e. they can be wellapproximated by linear superposition andnonlinear recovery algorithms, such as thel1 − norm.

I l0−, l1−, l2 − norm




Introduction – On Compressive Sensing VI

Compressive sensing (CS)

l1 − normSparsity promotion

Wavelets, curvelets, noiselets

Exploration seismology

Seismic data acquisition

Seismic imaging

Convex optimization

Pareto curves

Seismic inversion




The Math Behind CS – I

x ∈ RN where ‖x‖0 ≤ ky = Ψ · x . . .Ψ is an nxN matrix

n << N (#msrs.<<ambient dims.)

Init. assumptions (1)

x . . . is a high-dimensional signal

Ψ . . . is an full-rank nxN matrix

y . . . non-adaptive linear measurements

Goal: obtain x from non-adaptive linear measurements y




The Math Behind CS – II

b = Ψ · z where z is sparse

minimize‖z‖1 s.t. Ψ · z = b

Ψ . . . measurement matrix

Sparse recovery problem (2)

Goal: find x∗ as the solution of b = Ψ · z1st Problem: b = Ψ · z has infinitely many solutions.

2nd Problem: Sensitivity to the sparsity assumption.

3rd Problem: Sensitivity to additive noise, thus not useful inpractice.




The Math Behind CS – III

Theorem 1Suppose Ψ is an nxN Gaussian random matrix. If n & k · log(Nn )then with overwhelming probability we can recover all k-sparse xfrom y = Ψ · x.

ProblemIt is naive to expect signals in practice to be sparse. Morerealistically, the magnitude of the coefficients decays rapidly andthe coefficient vector contains only few significant entries.




The Math Behind CS – IV

Theorem 2

b = Ψ · z + e, ‖e‖2 ≤ εminimize‖z‖1 subject to ‖Ψ · z − b‖2 ≤ εx∗ . . . the solution for z

(3)

Corollary

‖x− x∗‖2 ≤ C1ε+ C2k− 1

2σk(x)

n = O(k · logNn

)

σk(x) . . . the best k-term approx. error

(4)




The Math Behind CS – V

I It has been empirically shown that seismic signals and CSyield small recovery errors, even in scenarios with muchdegraded sampling ratios.

I The recovered result is within the noise level and nearly asaccurate as the approximation we would obtain by measuringdirectly the largest k entries of x.




Solving the Sparse-Optimization Problem

I The sparsest solution of a severely underdetermined linearsystem can be recovered exactly by seeking the minimumone-norm (l1 ) solution.

I The l1 − norm finds sparse solutions (whiteboard)




Meanwhile in the Wavelet-Domain... I

In general seismic signals admit sparse approximations interms of curvelets:

f ∈ RN

f = SH · x and SPXN , P ≥ Nb = Ψ · f = Ψ · SH · x

Init. assumptions

(5)

f . . . a compressed data vector

SH . . . the superscript denotes the adjoint

Goal: Choose Ψ for a given S s.t. Ψ · SH is a goodmeasurement matrix.

(whiteboard)




Meanwhile in the Wavelet-Domain... II

ProblemWhat happens if the sparsifying dictionary is over-complete:

the columns of the SH matrix are correlated?

there are infinitely many x that explain the same signal f?

SPxN . . . P > N?

Solving the sparse optimization problem

A universal strategy for choosing Ψ that doesn’t require priorknowledge of the sparsity basis S: if we choose Ψ to be appropriaterandom measurement matrix, then Ψ · SH is guaranteed to also bea good measurement matrix independent of S.




Meanwhile in the Wavelet-Domain... III

Bottomline:

Many random sensing matrices are universally incoherent with anyfixed basis with very high probability.

The smaller the coherence between the randomly chosen matrixand the fixed basis, the fewer the samples required.

This matches numerical and practical experience.

Promote sparsity as a prior via one-norm regularization toovercome the singular nature of S.




Reflection Seismology - Introduction

Proceeding

I Generatation of Seismic source signal

I Seismic waves are reflected at the layers of the subsurface

I Reflected waves are measured by receivers at the ground




Reflection Seismology - Introduction - cont’d

Characteristic Acoustic Impedance

I Seismic waves travel in Earth at a speed governed by theAcoustic Impedance

Z0 = ρ · c

I Material Property

I Reflection occurs at the boundary between two materials withdifferent Characteristic Acoustic Impedances

R =Z1 − Z0

Z1 + Z0




Reflection Seismology - Introduction - cont’d

Reflection at normal and non-normal incidence




Reflection Seismology - Introduction cont’d

Sources

I Should ideally only emit P-WavesI Impulse

I f.e. Explotions, Airgun, EarthquakesI Source hardly predictible

I SweepI Vibrator truckI Source exactly knownI Correlation at the receivers necessary




Reflection Seismology - Introduction cont’d




Acquisition Schemes

Receiver

I Land receiver are so called ”Geophones”

I consider only vertical movement of the earth

Common Gather Types




Traditional Acquisition

I Encountered to the traditional ”Nyquist Theorem” to avoidSpatial Aliasing

I Needs for higher resolution images leads to exponentiallyincreasing costs

I Acquisition of spatio-temporal wavefield in up to fivedimensions

f(t, x) ∈ L2((0, T ]× [−L,L])

I T in the order of seconds; L in the order of kilometer

I Sampling intervalls in the order of milliseconds and meters

⇒ Dimension reduction needed




Compressive Sensing Approach

I Combines sampling and coding in one single step by arandomized subsampling technique

I Encoding is linear and does not require access tohigh-resolution data during encoding

Based on:

I Randomized Sampling

I Sparsifying Transforms

I Sparsity-promotion recovery by convex optimization




Examples

Periodic versus uniformly random subsampling

I If signals permits a sparse transform-domain representation itsuffices to sample at a rate that is lower than Nyquist.

I Recover signals from far fewer randomly placed samples

I In Seismology: Use seismic arrays with fewer geophonesselected uniformly random from regular sampling grids withspacings defined by Nyquist

This turns coherent aliases into Gaussian white noise




I In Geophysical: subsampling-related artifacts are commonlyknown as spectral leakage

I Depend on the degree of subsampling

I Characteristics depend on the irregularity of the sampling

I Remove noisy artifacts by sparse-recovery procedures




Seismic Wavefield Representation

Sparse Signal representation concerning CS

I Leverage structure within signals to reduce sampling

I Look for transform-domains that concentrate the signalsenergy in a few number of coefficients

I Consider transforms that are fast, multiscale andmultidirectional

Appropriate Transforms

I Curvelet Transform

I Wave Atom Transform




Wavelet Transformation

I Generalizes the Fourier transform by using a basis thatrepresents both location and spatial frequency

I Define orthogonal basis functions as dilations and translationsof ”Mother functions” also called ”analyzing wavelets”

Φ(sl)(x) = 2−s2 Φ(2−sx− l)




Wavelet Transformation - cont’d




Wavelet Transformation - cont’d

I Wavelets do not utilize geometric properties of wavefields

I Curved structures as superposition of ”Multiscale Fat Dots”




Curvelet Transformation

I Extension of wavelets by additional ”Orientation inLocalization”

ϕj,l,k(x) = ϕj

(RΘl

(x− x(j,l)k )

)

RΘl=

[cos Θl sin Θl

− sin Θl cos Θl

], x

(j,l)k = R−1

Θl(k1·2−j , k2·2−j/2)

Θl = 2π · 2−bj/2c · l, l = 0, 1, ..., 0 ≤ Θl < 2π




Curvelet Transformation - cont’d




Curvelets ⇐⇒ Waveatoms

Both obey a so called parabolic scaling

I Curvelets: ”needle-like” shapes

I Wave Atoms: ”Oscillatory patterns




Performance measure for transforms

Approximation Error

I real signals are not strictly sparse

I but transfer-domain coefficients often decay rapidly

I Orthonormal basis: Decay rate directly linked to the decay ofthe nonlinear approximation error

σ(k) = ‖f − fk‖2

I This does not hold for redundant transforms




Alternative Approximation Error

I Based on solution of a sparsity promoting program

min‖x‖1 subject to SHx = f

I To account for different redundancies plot SNR as a functionof the sparsity ratio ρ = k/P

SNR(ρ) = −20 log‖f − f

ρ‖

‖f‖




Subsampling of shots

Aim

I Breaking the periodicity of coherent sampling

Oppurtunities

I Selections of subsets of sources

I Design of incoherent simultaneous-source experiments




Subsampling of shots - cont’d

Measurement basises

I Sequential-source acquisition

I = INs ⊗ INt

I Simultaneous-source acquisition

M = GNs ⊗ INt

I Incomplete data

R = Rns ⊗ INt ns Ns




Subsampling of shots cont’d

Coherence depends on the interplay between restriction,

measurement and synthesis matrices

Another performance measure

δ = n/N, SNR(δ) = −20 log‖f − f

δ‖

‖f‖

where

fδ

= SHxδ, xδ = argmin‖x‖1 subject to Aδx = b

Aδ = RδMδSH




Compressive Seismic Computation

Overview

I Compressive Simulation

I Compressive Imaging

I Compressive Inversion

I SPGL1 - Spectral Projected-Gradient for L1 norm




Compressive Simulation

I Aim: Simulation of P-WavesI Traditionally solved with the time-harmonic Helmholtz

equationI large linear system of PDEs that discretizes the underlying

wave equation

I Use linearity in the sources to reduce the number of sequentialshots into a small number of ”supershots”

I complexity reduction from O(n4) to O(n3 log n)




Compressive Imaging

Motivation

I Locate mineral and oil sources

I Building Industry




The Seismic Image Problem

I Requires inversion of the linearized time-harmonicBorn-scattering matrix

I Seismic data is decomposed through Fourier transform intomonochromatic wavefields

minimize‖b− Ax‖22 =

K∑i=1

‖bi − Aix‖22

b ∈ CNfNrNs , x ∈ RM , K = NfNs

I Each iteration needs solutions of a big set of PDEs




Solution by batching

I Take ”Mini-Batches” with K ′ K monochromaticsupershots

I Solve reduced system

minimize‖b− Ax‖22 =

K′∑

i=1

‖bj − Ajx‖22

bj =

K∑i=1

wijbi, Aj =

K∑i=1

wijAi




Solution by batching - cont’d

Sub-selection and mixing in the source-frequency space:

‖RM(b− Ax)‖22

R = RΣ ⊗ I⊗ RΩ, M = MΣ ⊗ I⊗ I

RΣ ∈ Rn′s×ns , RΩ ∈ Rn

′f×nf , M ∈ R(KNr(×(KNr)

I ⇒ Solution reduced by a factor of K′/K




Solution by sparsity promotion

I Averaging is not able to remove the source crosstalk efficiently

I use transform-domain sparsity promotion

I Additionaly: Redraw supershots at each iteration




SPGL1 Solver

I large-scale sparse reconstruction solverI Basis pursuit

minimize ‖x‖1 subject to Ax = b

I Basis pursuit denoise

minimize ‖x‖1 subject to ‖Ax− b‖2 ≤ σ

I Lasso

minimize ‖Ax− b‖2 subject to ‖x‖1 ≤ τ




SPGL Solver - cont’d

Basis pursuit denoise

I Solve BPσ by a sequence of subproblems LSτI At each iteration k refine τk such that τk → τσ

I Derive sequence of estimates τk by applying Newton’s methodto find the root of value function φ(τ) = σ





LSτ solved by ”Spectral projected gradient”

Pτ [b] := argmin‖b− x‖2 subject to ‖x‖1 ≤ τ




Full Waveform Inversion – FWI I

I Most industry practice is based on a geometric opticsapproximation of the wave propagation.

I A ”smooth velocity model” that describes the propagationspeed of the waves in the subsurface (lack of knowledge).

I FWI relies on modeling the data by solving the wave equation(PDE), and adapting the model parameters in order tominimize the data misfit based on the LS-optimization.

I Minimization of the data misfit requires low wavenumbers.And yes, this is now becoming possible!




Full Waveform Inversion – FWI II

Ghost-free seismic data




Full Waveform Inversion – FWI III

More Wavenumbers




Full Waveform Inversion – FWI IV

More Wavenumbers (cont’d)

I Broadband frequency signals propagatingthrough the earth and the ocean.

I The wavenumber of any seismic signal is relatedin a rather complex manner to its seismicfrequency, the seismic velocity and the offsetbetween source and receiver locations, recordedalong a towed streamer.

I Removing the receiver ghost and deep streamertowing allows recording data in a low noiseenvironment.




Full Waveform Inversion – FWI V

More Wavenumbers (cont’d)

I ”Wavefield separation” during processing allowsbuilding high resolution velocity models and highresolution imaging.

I LOW WAVENUMBERS: data rich in lowfrequencies.




Full Waveform Inversion – FWI VI

Wavefield Separation

I Two interfering wavefields: up-going wavefieldscattered upwards from the earth and thetime-delayed version that is reflected downwardsfrom the sea surface.

I Key differentiation technique of GeoStreamer.




Full Waveform Inversion – FWI VII

I Underdetermined system of linear equations:

f = SHx

A = RMSH ⇒ Ax = b

minimize‖x‖0 s.t. Ax = b

Basic problem (6)

l0 . . .#of nonzero elements

I This cannot be approximated by a convex approximationproblem.

I One of the major findings of CS is that under some conditionson A and x, the solution can be recovered by solving theconvex optimization problem.




Full Waveform Inversion – FWI VIII

I Basis Pursuit (BP):

minimize‖x‖1 s.t. Ax = b (7)

I Recovers the correct sparse signal depending on the sparsitylevel of x, the #of measurements and the restricted isometryproperty (RIP) of A.

I The RIP constant measures how far the matrix A is from aunitary matrix when acting on sparse vectors (A∗A = I).

I The columns of an unitary matrix form an orthonormal basis,as well as the rows.




Full Waveform Inversion – FWI IX

I Quadratic Programming (QP):

minimize1

2‖Ax− b‖22 + λ‖x‖1 (8)

I Basis Pursuit Denoise Problem (BPDP):

minimize‖x‖1 s.t. ‖b−Ax‖2 ≤ σ (9)

I LASSO (LS):

min1

2‖b−Ax‖2 s.t. ‖x‖1 ≤ τ (10)

I For each σ, there are unique values of λ and τ so that thesolutions for QP and LASSO coincide.




Full Waveform Inversion – FWI X

LASSO (cont’d)

I The estimate of the one norm of the solution(τ) is seldom available for geophysical problems,but is a key internal problem used by SpectralProjected Gradient Method (SPGL).

I The SPGL can be used to quickly solve the LSequation for very large linear systems and it hasalready been proven to be very successful insolving large-scale CS problems in seismicexploration.




Pareto Curves – I

Pareto Curves:

Trace the optimal trade-off between the datamisfit and some prior model.

Are commonly used in problems with two-normpriors and recently explored in terms ofone-norm regularization.

Are convex and decreasing, i.e. regular. Thisregularity means it is possible to obtain a goodapproximation to the curve with very fewinterpolation points.




Pareto Curves – II

Pareto Curves (cont’d)




References I

F.J. Herrmann, M.P. Friedlander, O. Yilmaz,

“Fighting the curse of dimensionality: compressive sensing in exploration seismology,”IEEE Signal Processing Magazine, vol. 29, no. 3, pp. 88–100, 2012.

E.J. Candes, M.B. Wakin,

“An Introduction to Compressive Sampling,”IEEE Signal Processing Magazine, vol. 25, no. 2, pp. 21–30, 2008.

R. Baraniuk,

“Compressive Sensing,”Lecture Notes in IEEE Signal Processing Magazine, vol. 24, pp. 118–121.

A. Aravkin, X. Li, F.J. Hermann,

“Fast Seismic Imaging For Marine Data,”The University of British Columbia Technical Report., October 2011.

J. Ma, G. Plonka,

“The Curvelet Transform,”IEEE Signal Processing Magazine, vol. 27, no. 2, pp. 118–133, March 2010.

J. Ma,

“Characterization of textural surfaces using wave atoms,”Applied Physics Letters, vol. 90, no. 26, June 2007.




References II

E.v.d. Berg, M.P. Friedlander,

“Probing the Pareto Frontier for Basis Pursuit Solutions,”The University of British Columbia Technical Report., May 2008.

E. Hager,

“Full Azimuth Seismic Acquisition with Coil Shooting,”8. Biennial International Conference and Exposition on Petroleum Geophysics Hyderabad, 2010.

F.J. Herrmann, X. Li, A.Y. Aravkin, T. Leeuwen,

“A modified, sparsity promoting, Gauss-Newton algorithm for seismic waveform inversion,”Dept. of Earth and Ocean Sciences, University of British Columbia.

F.J. Herrmann,

“Randomized sampling and sparsity: getting more information from fewer samples,”The University of British Columbia Technical Report, TR-2010-01, 2010-05-19.

G. Hennenfent, E.v.d. Berg, M.P. Friedlander, F.J. Herrmann,

“New Insights into one-norm solvers from the Pareto curve,”Geophysics, vol. 73, no. 4, pp. A23–A26.


presentation - signal processing and speech communication

Documents