sparse sampling: theory and applicationspld/talks/icms09.pdf · such as ltered streams of diracs,...

46
Sparse Sampling: Theory and Applications Pier Luigi Dragotti November 24, 2009 Pier Luigi Dragotti Sparse Sampling: Theory and Applications

Upload: others

Post on 15-May-2020

5 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Sparse Sampling: Theory and Applicationspld/talks/ICMS09.pdf · such as ltered streams of Diracs, multi-dimensional signals and piecewise sinusoidal signals. 50 100 150 200 250 300

Sparse Sampling: Theory and Applications

Pier Luigi Dragotti

November 24, 2009

Pier Luigi DragottiSparse Sampling: Theory and Applications

Page 2: Sparse Sampling: Theory and Applicationspld/talks/ICMS09.pdf · such as ltered streams of Diracs, multi-dimensional signals and piecewise sinusoidal signals. 50 100 150 200 250 300

OutlineI Problem Statement

I Signals with Finite Rate of Innovation

I Sampling Kernels: E-splines and B-splines

I Sparse Sampling: the Basic Set-up and Extensions

I The Noisy Scenario

I Applications

I CompressionI Image Super-resolution

I Conclusions and Outlook

Pier Luigi DragottiSparse Sampling: Theory and Applications

Page 3: Sparse Sampling: Theory and Applicationspld/talks/ICMS09.pdf · such as ltered streams of Diracs, multi-dimensional signals and piecewise sinusoidal signals. 50 100 150 200 250 300

Problem Statement

You are given a class of functions. You have a sampling device, typically, alow-pass filter. Given the measurements yn = 〈x(t), ϕ(t/T − n)〉, you want toreconstruct x(t).

Tx(t)

!

Acquisition Device

h(t)= (!t/T) y =<x(t), (t/T!n)>n !y(t)

Natural questions:

I When is there a one-to-one mapping between x(t) and yn?

I What signals can be sampled and what kernels ϕ(t) can be used?

I What reconstruction algorithm?

Pier Luigi DragottiSparse Sampling: Theory and Applications

Page 4: Sparse Sampling: Theory and Applicationspld/talks/ICMS09.pdf · such as ltered streams of Diracs, multi-dimensional signals and piecewise sinusoidal signals. 50 100 150 200 250 300

Problem Statement

Observedscene

Samples

AcquisitionSystem

Lens CCDArray

I The low-quality lens blurs the images.

I The images are under-sampled by the low resolution CCD array.

I You need a good post-processing algorithm to undo the blurring andupsample the images.

Pier Luigi DragottiSparse Sampling: Theory and Applications

Page 5: Sparse Sampling: Theory and Applicationspld/talks/ICMS09.pdf · such as ltered streams of Diracs, multi-dimensional signals and piecewise sinusoidal signals. 50 100 150 200 250 300

Signals with Finite Rate of Innovation

I Sampling is an ill-posed inverse problem. To make it well-posed we needto impose constraints on the signals. Typically signals are assumed to bebandlimited: x(t) =

Pn ynsinc(t/T − n).

I What is so special about those signals?

I The signal x(t) =P

n ynsinc(t/T − n) is exactly specified by oneparameter yn every T seconds, it has a finite number ρ = 1/T of degreesof freedom per unit of time.

In the classical formulation, innovation is uniform. How about signals wherethe rate of innovation is finite but non-uniform? E.g.

I Piecewise sinusoidal signals (Frequency Hopping modulation).

I Pulse position modulation (UWB).

I Edges in images.

Pier Luigi DragottiSparse Sampling: Theory and Applications

Page 6: Sparse Sampling: Theory and Applicationspld/talks/ICMS09.pdf · such as ltered streams of Diracs, multi-dimensional signals and piecewise sinusoidal signals. 50 100 150 200 250 300

Signals with Finite Rate of Innovation

I We assume that the signals considered are completely determined by afinite number of parameters per unit of time.

I The number ρ of degrees of freedom per unit of time is called rate ofinnovation. A signal with a finite ρ is called signal with finite rate ofinnovation (FRI).

I Notice: this is a parametric sparsity.

FRI signals include:

I Bandlimited signals and signals belonging to shift-invariant subspaces.

I K-sparse discrete signals (like in Compressed Sensing).

I Signals with point-like innovation, (point source phenomena), piecewisesinusoidal signals (OFDM, FH), filtered Diracs (UWB, Neuronal signals).

Pier Luigi DragottiSparse Sampling: Theory and Applications

Page 7: Sparse Sampling: Theory and Applicationspld/talks/ICMS09.pdf · such as ltered streams of Diracs, multi-dimensional signals and piecewise sinusoidal signals. 50 100 150 200 250 300

Examples of Signals with Finite Rate of Innovation

0 100 200 300 400 500 600−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1

20 40 60 80 100 120−0.5

0

0.5

1

1.5

2

Filtered Streams of Diracs Piecewise Polynomial Signals

20 40 60 80 100 120

−1

−0.5

0

0.5

1

1.5

2

Piecewise Sinusoidal Signals Mondrian paintings ;-)

Pier Luigi DragottiSparse Sampling: Theory and Applications

Page 8: Sparse Sampling: Theory and Applicationspld/talks/ICMS09.pdf · such as ltered streams of Diracs, multi-dimensional signals and piecewise sinusoidal signals. 50 100 150 200 250 300

Sampling Kernels

Any kernel ϕ(t) that can reproduce exponentials:Xn

cm,nϕ(t − n) = eαmt , αm = α0 + mλ and m = 0, 1, ..., L.

This includes any composite kernel of the form φ(t) ∗ β~α(t) whereβ~α(t) = βα0 (t) ∗ βα1 (t) ∗ ... ∗ βαL (t) and βαi (t) is an Exponential Spline of first order[UnserB:05].

eα t

E−Spline βα(t)

βα(t)⇔ β(ω) =1− eα−jω

jω − αNotice:

I α can be complex.

I E-Spline is of compact support.

I E-Spline reduces to the classical polynomialspline when α = 0.

Pier Luigi DragottiSparse Sampling: Theory and Applications

Page 9: Sparse Sampling: Theory and Applicationspld/talks/ICMS09.pdf · such as ltered streams of Diracs, multi-dimensional signals and piecewise sinusoidal signals. 50 100 150 200 250 300

Kernels Reproducing Exponentials

0 1 t

eα0 t

0 1 2 3 4 5 6 7 80

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

t

The E-spline of first order βα0 (t) reproduces the exponential eα0t :Xn

c0,nβα0 (t − n) = eα0t .

In this case c0,n = eα0n. In general, cm,n = cm,0eαmn.

Pier Luigi DragottiSparse Sampling: Theory and Applications

Page 10: Sparse Sampling: Theory and Applicationspld/talks/ICMS09.pdf · such as ltered streams of Diracs, multi-dimensional signals and piecewise sinusoidal signals. 50 100 150 200 250 300

Trigonometric E-Splines

I The exponent α of the E-splines can be complex. This means βα(t) canbe a complex function.

I However if pairs of exponents are chosen to be complex conjugate thenthe spline stays real.

I Example:

βα0+jω0 (t) ∗ βα0−jω0 (t) =

8>>>><>>>>:

sinω0tω0

eα0t 0 ≤ t < 1

− sinω0(t−2)ω0

eα0t 1 ≤ t < 2

0 Otherwise

When α0 = 0 (i.e., purely imaginary exponents), the spline is called

trigonometric spline.

Pier Luigi DragottiSparse Sampling: Theory and Applications

Page 11: Sparse Sampling: Theory and Applicationspld/talks/ICMS09.pdf · such as ltered streams of Diracs, multi-dimensional signals and piecewise sinusoidal signals. 50 100 150 200 250 300

Trigonometric E-Splines

−10 −8 −6 −4 −2 0 2 4 6 8 10 12−1.5

−1

−0.5

0

0.5

1

1.5

t

ExponentialReproductionScaled and shifted E−splines

−10 −8 −6 −4 −2 0 2 4 6 8 10 12−1.5

−1

−0.5

0

0.5

1

1.5

t

ExponentialReproductionScaled and shifted E−splines

−10 −8 −6 −4 −2 0 2 4 6 8 10 12−1.5

−1

−0.5

0

0.5

1

1.5

t

ExponentialReproductionScaled and shifted E−splines

−10 −8 −6 −4 −2 0 2 4 6 8 10 12−1.5

−1

−0.5

0

0.5

1

1.5

t

ExponentialReproductionScaled and shifted E−splines

Here ~α = (−jω0, jω0) and ω0 = 0.2.P

n cn,mβ~α(t − n) = e jmω0 m = −1, 1.

Notice: β~α(t) is a real function, but the coefficients cm,n are complex.

Pier Luigi DragottiSparse Sampling: Theory and Applications

Page 12: Sparse Sampling: Theory and Applicationspld/talks/ICMS09.pdf · such as ltered streams of Diracs, multi-dimensional signals and piecewise sinusoidal signals. 50 100 150 200 250 300

E-Splines and B-splines

When αm = 0, m = 0, 1, ..., L. The E-spline reduce to the classical B-spline and isthen able to reproduce polynomials up to degree L.

−2 −1.5 −1 −0.5 0 0.5 1 1.5 2

0

0.2

0.4

0.6

0.8

1

!0(t)−2 −1.5 −1 −0.5 0 0.5 1 1.5 2

0

0.2

0.4

0.6

0.8

1

!1(t)

β0(t) β1(t)

−2 −1.5 −1 −0.5 0 0.5 1 1.5 2−0.1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

!2(t)−2 −1.5 −1 −0.5 0 0.5 1 1.5 2

−0.1

0

0.1

0.2

0.3

0.4

0.5

0.6

!3(t)

β2(t) β3(t)

Pier Luigi DragottiSparse Sampling: Theory and Applications

Page 13: Sparse Sampling: Theory and Applicationspld/talks/ICMS09.pdf · such as ltered streams of Diracs, multi-dimensional signals and piecewise sinusoidal signals. 50 100 150 200 250 300

E-Splines and B-splines

The E-spline reduces to the classical cubic B-spline when αm = 0, m = 0, 1, ..., L andL = 3. In this case it can reproduce polynomials up to degree L = 3.

−6 −4 −2 0 2 4 6−3

−2

−1

0

1

2

3

−6 −4 −2 0 2 4 6−3

−2

−1

0

1

2

3

c0,n = (1, 1, 1, 1, 1, 1, 1) c1,n = (−3,−2,−1, 0, 1, 2, 3)

−6 −4 −2 0 2 4 6

−6

−4

−2

0

2

4

6

−6 −4 −2 0 2 4 6−20

−15

−10

−5

0

5

10

15

20

c2,n ∼ (8.7, 3.7, 0.7,−0.333, 0.7, 3.7, 8.7) c3,n ∼ (−24,−6,−0.001, 0, 0.001, 6, 24)

Pier Luigi DragottiSparse Sampling: Theory and Applications

Page 14: Sparse Sampling: Theory and Applicationspld/talks/ICMS09.pdf · such as ltered streams of Diracs, multi-dimensional signals and piecewise sinusoidal signals. 50 100 150 200 250 300

Kernel Reproducing Exponential

Any functions with rational Fourier transform:

ϕ(ω) =

Qi (jω − bi )Q

m(jω − am)m = 0, 1, ..., L.

is a generalized E-splines. This includes practical devices as common as an RC circuit:

x(t) y(t)

R

C

+

-

+

-

Pier Luigi DragottiSparse Sampling: Theory and Applications

Page 15: Sparse Sampling: Theory and Applicationspld/talks/ICMS09.pdf · such as ltered streams of Diracs, multi-dimensional signals and piecewise sinusoidal signals. 50 100 150 200 250 300

Sparse Sampling: Basic Set-up

Assume that x(t) is the signal we have sampled and want to reconstructand assume the sampling kernel ϕ(t) is any function that can reproduceexponentials of the form∑

n

cm,nϕ(t − n) = eαmt m = 0, 1, ..., L,

We want to retrieve x(t), from the samples yn = 〈x(t), ϕ(t/T − n)〉.

Pier Luigi DragottiSparse Sampling: Theory and Applications

Page 16: Sparse Sampling: Theory and Applicationspld/talks/ICMS09.pdf · such as ltered streams of Diracs, multi-dimensional signals and piecewise sinusoidal signals. 50 100 150 200 250 300

Sparse Sampling: Basic Set-up

We have that (we assume T = 1 for simplicity)

sm =∑

n cm,nyn

= 〈x(t),∑

n cm,nϕ(t − n)〉

=∫∞−∞ x(t)eαmtdt, m = 0, 1, ..., L.

I sm is the bilateral Laplace transform of x(t) evaluated at αm.

I When αm = jωm then sm = x(ωm).

I When αm = 0, the sm’s are the polynomial moments of x(t).

Pier Luigi DragottiSparse Sampling: Theory and Applications

Page 17: Sparse Sampling: Theory and Applicationspld/talks/ICMS09.pdf · such as ltered streams of Diracs, multi-dimensional signals and piecewise sinusoidal signals. 50 100 150 200 250 300

Sampling Streams of Diracs

I Assume x(t) is a stream of K Diracs: x(t) =∑K−1

k=0 xkδ(t − tk).

I We restrict αm = α0 + mλ m = 0, 1, ..., L and L ≥ 2K − 1.

I We obtain

sm =∫∞−∞ x(t)eαmtdt,

=∑K−1

k=0 xkeαmtk

=∑K−1

k=0 xkeλmtk

=∑K−1

k=0 xkumk , m = 0, 1, ..., L.

Pier Luigi DragottiSparse Sampling: Theory and Applications

Page 18: Sparse Sampling: Theory and Applicationspld/talks/ICMS09.pdf · such as ltered streams of Diracs, multi-dimensional signals and piecewise sinusoidal signals. 50 100 150 200 250 300

The Annihilating Filter Method

I The quantity

sm =K−1∑k=0

xkumk , m = 0, 1, ..., L

is a power-sum series.

I We can retrieve the locations uk and the amplitudes xk with theannihilating filter method (also known as Prony’s method since itwas discovered by Gaspard de Prony in 1795).

I Given the pairs {uk , xk}, then tk = (ln uk)/λ and xk = xk/eα0tk .

Pier Luigi DragottiSparse Sampling: Theory and Applications

Page 19: Sparse Sampling: Theory and Applicationspld/talks/ICMS09.pdf · such as ltered streams of Diracs, multi-dimensional signals and piecewise sinusoidal signals. 50 100 150 200 250 300

The Annihilating Filter Method

1. Call hm the filter with z-transform H(z) =PK

i=0 hiz−i =

QK−1k=0 (1− ukz

−1).We have that

hm ∗ sm =KX

i=0

hi sm−i =KX

i=0

K−1Xk=0

xkhium−ik =

K−1Xk=0

xkumk

KXi=0

hiu−ik| {z }

0

= 0.

This filter is thus called the annihilating filter. In matrix/vector form, using thefact that h0 = 1, we obtain26666666664

sK−1 sK−2 · · · s0

sK sK−1 · · · s1

......

. . ....

sL−1 sL−2 · · · sL−K

37777777775

0BBBBBBBBB@

h1

h2

...

hK

1CCCCCCCCCA= −

0BBBBBBBBB@

sK

sK+1

...

sL

1CCCCCCCCCA.

Solve the above system to find the coefficients of the annihilating filter.

Pier Luigi DragottiSparse Sampling: Theory and Applications

Page 20: Sparse Sampling: Theory and Applicationspld/talks/ICMS09.pdf · such as ltered streams of Diracs, multi-dimensional signals and piecewise sinusoidal signals. 50 100 150 200 250 300

The Annihilating Filter Method

2. Given the coefficients {1, h1, h2, ..., hk}, we get the locations uk by findingthe roots of H(z).3. Solve the first K equations in sm =

PK−1k=0 xku

mk to find the amplitudes xk .

In matrix/vector form26666666664

1 1 · · · 1

u0 u1 · · · uK−1

......

. . ....

uK−10 uK−1

1 · · · uK−1K−1

37777777775

0BBBBBBBBB@

x0

x1

...

xK−1

1CCCCCCCCCA=

0BBBBBBBBB@

s0

s1

...

sK−1

1CCCCCCCCCA. (1)

Classic Vandermonde system. Unique solution for distinct uks.

Pier Luigi DragottiSparse Sampling: Theory and Applications

Page 21: Sparse Sampling: Theory and Applicationspld/talks/ICMS09.pdf · such as ltered streams of Diracs, multi-dimensional signals and piecewise sinusoidal signals. 50 100 150 200 250 300

Sampling Streams of Diracs: Numerical Example

0 0.2 0.4 0.6 0.8 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

t−4 −3 −2 −1 0 1 2 3 40

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

t

(a) Original Signal (b) Sampling Kernel (β7(t))

0 0.2 0.4 0.6 0.8 10

0.5

1

1.5

2

2.5

3

3.5

4

4.5

t0 0.2 0.4 0.6 0.8 1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

t

(c) Samples (d) Reconstructed Signal

Pier Luigi DragottiSparse Sampling: Theory and Applications

Page 22: Sparse Sampling: Theory and Applicationspld/talks/ICMS09.pdf · such as ltered streams of Diracs, multi-dimensional signals and piecewise sinusoidal signals. 50 100 150 200 250 300

Sparse Sampling: Extensions

Using variations of the annihilating filter methods other signals can be sampledsuch as filtered streams of Diracs, multi-dimensional signals and piecewisesinusoidal signals.

50 100 150 200 250 300 350 400 450 500

−1.5

−1

−0.5

0

0.5

1

1.5

Pier Luigi DragottiSparse Sampling: Theory and Applications

Page 23: Sparse Sampling: Theory and Applicationspld/talks/ICMS09.pdf · such as ltered streams of Diracs, multi-dimensional signals and piecewise sinusoidal signals. 50 100 150 200 250 300

Sampling Piecewise Sinusoidal Signals

We consider signals of the type:

x(t) =DX

d=1

NXn=1

Ad,n cos(ωd,nt + ϕd,n)ξd(t),

whereξd(t) = u(t − td)− u(t − td+1) and −∞ < t1 < ... < td < ... < tD+1 <∞.Why is it difficult to sample them?

I Piecewise sinusoidal signals contain innovation in both spectral andtemporal domains.

I They are not bandlimited.

I They are not sparse in time nor in a basis or a frame.

Pier Luigi DragottiSparse Sampling: Theory and Applications

Page 24: Sparse Sampling: Theory and Applicationspld/talks/ICMS09.pdf · such as ltered streams of Diracs, multi-dimensional signals and piecewise sinusoidal signals. 50 100 150 200 250 300

Sampling Piecewise Sinusoidal Signals [BerentD:09]

From the samples we can obtain the Laplace transform of x(t) atαm = α0 + mλ, m = 0, 1, ..., L:

sm =DX

d=1

2NXn=1

Ad,n[etd+1(jωd,n+αm) − etd (jωd,n+αm)]

(jωd,n + αm),

where Ad,n = Ad,nejϕd,n . We define the polynomial

Q(αm) =DY

d=1

2NYn=1

(jωd,n + αm) =JX

j=0

rjαjm.

Pier Luigi DragottiSparse Sampling: Theory and Applications

Page 25: Sparse Sampling: Theory and Applicationspld/talks/ICMS09.pdf · such as ltered streams of Diracs, multi-dimensional signals and piecewise sinusoidal signals. 50 100 150 200 250 300

Sampling Piecewise Sinusoidal Signals [BerentD:09]

Multiplying both side of the equation by Q(αm) we obtain:

Q(αm)sm =DX

d=1

2NXn=1

Ad,nP(αm)[etd+1(jωd,n+αm) − etd (jωd,n+αm)], (2)

where P(αm) is a polynomial. Since αm = α0 + λm the right-hand side of (2)can be annihilated:

Q(αm)sm ∗ hm = 0.

Pier Luigi DragottiSparse Sampling: Theory and Applications

Page 26: Sparse Sampling: Theory and Applicationspld/talks/ICMS09.pdf · such as ltered streams of Diracs, multi-dimensional signals and piecewise sinusoidal signals. 50 100 150 200 250 300

Sampling Piecewise Sinusoidal Signals [BerentD:09]

In matrix/vector form (assuming h0 = 1), we have:

266666666664

sK · · · αJK sK · · · s0 · · · αJ

0s0

sK+1 · · · αJK+1sK+1 · · · s0 · · · αJ

1s1

......

. . ....

sL · · · αJLsL · · · s0 · · · αJ

(L−K)s(L−K)

377777777775

0BBBBBBBBBBBBBBBBBBBBBBBBBBBB@

r0

...

rJ

h1r0

h1r1

...

hK rJ

...hK rK

1CCCCCCCCCCCCCCCCCCCCCCCCCCCCA

= 0.

Pier Luigi DragottiSparse Sampling: Theory and Applications

Page 27: Sparse Sampling: Theory and Applicationspld/talks/ICMS09.pdf · such as ltered streams of Diracs, multi-dimensional signals and piecewise sinusoidal signals. 50 100 150 200 250 300

Sampling Piecewise Sinusoidal Signals

I From the coefficients rj , j = 0, 1, ...J, we obtain Q(αm).

I The roots of the filter H(z) and of the polynomial Q(αm) give thelocations of the switching points and the frequencies of the sinewaves respectively.

I To solve the system we need L ≥ 4D3N2 + 4D2N2 + 4D2N + 6DN.

Pier Luigi DragottiSparse Sampling: Theory and Applications

Page 28: Sparse Sampling: Theory and Applicationspld/talks/ICMS09.pdf · such as ltered streams of Diracs, multi-dimensional signals and piecewise sinusoidal signals. 50 100 150 200 250 300

Numerical Example

0 0.25 0.5 0.75 1 [sec]−5

0

5

(a)

0 0.25 0.5 0.75 1 [sec]−2

0

2

(b)

0 0.25 0.5 0.75 1 [sec]−5

0

5

(c)

Pier Luigi DragottiSparse Sampling: Theory and Applications

Page 29: Sparse Sampling: Theory and Applicationspld/talks/ICMS09.pdf · such as ltered streams of Diracs, multi-dimensional signals and piecewise sinusoidal signals. 50 100 150 200 250 300

Robust Sparse Sampling

In the presence of noise, the annihilation equation

SH = 0

is only approximately satisfied.Minimize: ‖SH‖2 under the constraint ‖H‖2 = 1.This is achieved by performing an SVD of S:

S = UλVT.

Then H is the last column of V.Notice: this is similar to Pisarenko’s method in spectral estimation.

Pier Luigi DragottiSparse Sampling: Theory and Applications

Page 30: Sparse Sampling: Theory and Applicationspld/talks/ICMS09.pdf · such as ltered streams of Diracs, multi-dimensional signals and piecewise sinusoidal signals. 50 100 150 200 250 300

Robust Sparse Sampling: Cadzow’s algorithm

For small SNR use Cadzow’s method to denoise S before applying TLS.The basic intuition behind this method is that, in the noiseless case, S isrank deficient (rank K ) and Toeplitz, while in the noisy case S is full rank.Algorithm:

I SVD of S = UλVT.

I Keep the K largest diagonal coefficients of λ and set the others tozero.

I Reconstruct S′ = Uλ′VT.

I This matrix is not Toeplitz, make it so by averaging along thediagonals.

I Iterate.

Pier Luigi DragottiSparse Sampling: Theory and Applications

Page 31: Sparse Sampling: Theory and Applicationspld/talks/ICMS09.pdf · such as ltered streams of Diracs, multi-dimensional signals and piecewise sinusoidal signals. 50 100 150 200 250 300

Robust Sparse Sampling

I Samples are corrupted by additive noise.

I This is a parametric estimation problem.

I Unbiased algorithms have a covariance matrix lower bounded by CRB.

I The proposed algorithm reaches CRB down to SNR of 5dB.

Pier Luigi DragottiSparse Sampling: Theory and Applications

Page 32: Sparse Sampling: Theory and Applicationspld/talks/ICMS09.pdf · such as ltered streams of Diracs, multi-dimensional signals and piecewise sinusoidal signals. 50 100 150 200 250 300

Robust Sparse Sampling

0 0.2 0.4 0.6 0.8 1−5

0

5

10

Noisy Samples (N=128)

0 0.2 0.4 0.6 0.8 10

2

4

6

8

10

Noiseless Samples (N=128)

0 0.2 0.4 0.6 0.8 1

0

0.2

0.4

0.6

0.8

1

1.2

Original and estimated Diracs : SNR = 5 dB;

t

original Diracsestimated Diracs

Pier Luigi DragottiSparse Sampling: Theory and Applications

Page 33: Sparse Sampling: Theory and Applicationspld/talks/ICMS09.pdf · such as ltered streams of Diracs, multi-dimensional signals and piecewise sinusoidal signals. 50 100 150 200 250 300

Robust Sparse Sampling

Piecewise sinusoidal signal

For R

eview O

nly

IEEE TRANSACTIONS ON SIGNAL PROCESSING 23

0 5 10 15 20 25 300

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Input SNR [dB]

loca

tio

n [

se

co

nd

s]

(a)

0 5 10 15 20 25 30

10!4

10!3

10!2

10!1

100

101

102

103

Input SNR [dB]

Sta

nd

ard

de

via

tio

n f

or

loca

tio

n t

1

Observed Standard Deviation

Cramer!Rao Bound

(b)

Fig. 8. Retrieval of the switching point of a step sine (#1 = 12.23) [1/sec] and t1 = 0.4907 [sec]) in 128 noisy samples. (a)

Scatter plot of the estimated location. (b) Standard deviation (averages over 1000 iterations) of the location retrieval compared

to the Cramer-Rao bound.

C. Performance evaluation

It is of interest here to evaluate the performance of the reconstruction algorithm in the presence of

different noise levels. Therefore, we consider the Cramer-Rao bound that provides an answer to the

best possible performance of an unbiased estimator. The derivation of the Cramer-Rao bound in the

case of additive white Gaussian noise is presented in Appendix A. In this experiment, the acquisition

device observes 128 noisy samples with T = 1/128 [sec] of a truncated sine wave with frequency

#1 = 12.23" [rad/sec] and switching points t1 = 0.4907 [sec] and t2 = 1 [sec]. Since we assume

that the switching points are sufficiently far apart, their locations can be estimated independently. We

therefore show the results only for the first discontinuity. The frequency of the sine wave is estimated using

Matlab’s rootmusic function and the location of the switching point is estimated using a four phase

approach and additional hard thresholding. Note that we have also experimented with other frequency

estimation methods as well as using the ground truth frequency. Similar results are obtained in all cases.

Figure 8(a) shows the scatter plot for the reconstruction of the switching point t1 for different SNR

levels. The standard deviation of the error (averages over 1000 iterations) of the location retrieval is

shown in Figure 8(b). These simulations show that the proposed reconstruction algorithm behaves well

down to noise levels of about 7 [dB]. Figure 9 illustrates an example of the recovery of a continuous-

February 20, 2009 DRAFT

Page 23 of 108

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

Pier Luigi DragottiSparse Sampling: Theory and Applications

Page 34: Sparse Sampling: Theory and Applicationspld/talks/ICMS09.pdf · such as ltered streams of Diracs, multi-dimensional signals and piecewise sinusoidal signals. 50 100 150 200 250 300

Robust Sparse Sampling

For R

eview O

nly

IEEE TRANSACTIONS ON SIGNAL PROCESSING 24

0 0.5 1 [sec]!1

!0.5

0

0.5

1

(b)

Ground Truth

Reconstructed Signal

0 0.5 1 [sec]!2

!1

0

1

2

(a)

Fig. 9. Recovery of a truncated sine wave at SNR = 8 [dB]. (a) The observed noisy samples. (b) The reconstructed signal

along with the ground truth signal (dashed).

time piecewise sinusoidal signal (with t1 = 0.2441 [sec], t2 = 0.7324 [sec] and #1 = 12.23" [rad/sec])

given 128 noisy samples at an SNR of 8 [dB]. Note that despite the small error in the estimation of the

frequency of the sine wave, the estimation of the switching points are accurate.

VIII. CONCLUSION

We have set out to show that piecewise sinusoids belong to the family of signals with finite rate

of innovation and can be sampled and perfectly reconstructed using sampling kernels that reproduce

exponentials or polynomials. These classes of kernels are physically realizable and are of compact support.

Moreover, combinations of piecewise sinusoids and polynomials also have a finite rate of innovation and

can be dealt with using similar sampling schemes. This combination gives rise to a very general type of

signal.

Since the sampling scheme is limited by the rate of innovation rather than the actual frequency

of the continuous-time signal, we are, in theory, capable of retrieving piecewise sine waves with an

arbitrarily high frequency along with the exact location of the switching points. We believe therefore that

the sampling scheme presented may find applications, for example, in spread spectrum and wide band

communications.

Finally, we studied the effect of noise on the performance of the estimation of the switching points. In

doing so, we derived a polyphase reconstruction algorithm that together with hard thresholding behaves

well with respect to the Cramer-Rao bounds down to SNRs of 7 [dB].

February 20, 2009 DRAFT

Page 24 of 108

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

SNR= 8dB, N=128.

Pier Luigi DragottiSparse Sampling: Theory and Applications

Page 35: Sparse Sampling: Theory and Applicationspld/talks/ICMS09.pdf · such as ltered streams of Diracs, multi-dimensional signals and piecewise sinusoidal signals. 50 100 150 200 250 300

Compression

I FRI Signals can be sparsely sampled. Can they also be compressed? Whathappens when the samples are quantized?

I Traditional Compression is based on complex encoders and simpledecoders.

I New sampling theories are characterized by a linear acquisition butnon-linear reconstruction.

Pier Luigi DragottiSparse Sampling: Theory and Applications

Page 36: Sparse Sampling: Theory and Applicationspld/talks/ICMS09.pdf · such as ltered streams of Diracs, multi-dimensional signals and piecewise sinusoidal signals. 50 100 150 200 250 300

Compression

Signals are piecewise smooth, with α-Lipschitz regular pieces.

Traditional compression algorithms use the wavelet transform and compressonly the large wavelet coefficients. They achieve the optimal D(R)performance:

D(R) ∼ R−2α

The proposed algorithm compresses and transmits only the low-pass

coefficients of the wavelet transform (linear encoding), but uses FRI techniques

to estimate the discontinuities in the signal from the low-pass coefficients

(non-linear decoding).

Pier Luigi DragottiSparse Sampling: Theory and Applications

Page 37: Sparse Sampling: Theory and Applicationspld/talks/ICMS09.pdf · such as ltered streams of Diracs, multi-dimensional signals and piecewise sinusoidal signals. 50 100 150 200 250 300

Performance Analysis

I Any piecewise smooth signals can be decomposed into a piecewisepolynomial and a globally smooth signal [DragottiV:03].

I The low-pass coefficients are a sufficient representation of the piecewisepolynomial signal, but quantization and the smooth signal act as noiseand this reduces the reconstruction fidelity.

I We treat both contributions as additive noise and evaluate the CR-boundsfor this estimation problem. The quantization noise depends on thebit-rate R. This leads to a connection between CR-Bounds andrate-distortion analysis and leads to this performance bound:

DFRI (R) ∼ R−2α.

Pier Luigi DragottiSparse Sampling: Theory and Applications

Page 38: Sparse Sampling: Theory and Applicationspld/talks/ICMS09.pdf · such as ltered streams of Diracs, multi-dimensional signals and piecewise sinusoidal signals. 50 100 150 200 250 300

Simulation Results

102 103 104 10510−10

10−8

10−6

10−4

10−2

100

R (bits)

D

Semi−Parametricc1R−2!

Linear Approx.c2R−1

0.3 0.4 0.5 0.6−1

258

original signal

0.3 0.4 0.5 0.6−1

258

linear approx. compression, R = 736 bits

0.3 0.4 0.5 0.6−1

258

semi−parametric compression, R = 544 bits

Pier Luigi DragottiSparse Sampling: Theory and Applications

Page 39: Sparse Sampling: Theory and Applicationspld/talks/ICMS09.pdf · such as ltered streams of Diracs, multi-dimensional signals and piecewise sinusoidal signals. 50 100 150 200 250 300

Application: Image Super-Resolution

Super-Resolution is a multichannel sampling problem with unknown shifts. Usemoments to retrieve the shifts or the geometric transformation between images.

(a)Original (512× 512) (b) Low-res. (64× 64) (c) Super-res ( PSNR=24.2dB)

I Forty low-resolution and shifted versions of the original.

I The disparity between images has a finite rate of innovation and can be retrieved.

I Accurate registration is achieved by retrieving the continuous moments of the‘Tiger’ from the samples.

I The registered images are interpolated to achieve super-resolution.Pier Luigi DragottiSparse Sampling: Theory and Applications

Page 40: Sparse Sampling: Theory and Applicationspld/talks/ICMS09.pdf · such as ltered streams of Diracs, multi-dimensional signals and piecewise sinusoidal signals. 50 100 150 200 250 300

Application: Image Super-Resolution

Image super-resolution basic building blocks

RestorationSuper-resolved

image

Set of low-resolution images Image Registration HR grid estimation

LR image 0...

LR image k

Pier Luigi DragottiSparse Sampling: Theory and Applications

Page 41: Sparse Sampling: Theory and Applicationspld/talks/ICMS09.pdf · such as ltered streams of Diracs, multi-dimensional signals and piecewise sinusoidal signals. 50 100 150 200 250 300

Application: Image Super-Resolution

I For each blurred image I (x , y):

I A pixel Pm,n in the blurred image is given by

Pm,n = 〈I (x , y), ϕ(x/T − n, y/T −m)〉,

where ϕ(t) represents the point spread function of the lens.I We assume ϕ(t) is a spline that can reproduce polynomials:X

n

Xm

c (l,j)m,n ϕ(x − n, y −m) = x ly j l = 0, 1, ...,N; j = 0, 1, ...,N.

I We retrieve the exact moments of I (x , y) from Pm,n:

τl,j =X

n

Xm

c (l,j)m,n Pm,n =

Z ZI (x , y)x ly jdxdy .

I Given the moments from two or more images, we estimate the geometricaltransformation and register them. Notice that moments of up to order threealong the x and y coordinates allows the estimation of an affine transformation.

Pier Luigi DragottiSparse Sampling: Theory and Applications

Page 42: Sparse Sampling: Theory and Applicationspld/talks/ICMS09.pdf · such as ltered streams of Diracs, multi-dimensional signals and piecewise sinusoidal signals. 50 100 150 200 250 300

Application: Image Super-Resolution

−6 −4 −2 0 2 4 6

Pixels (perpendicular)

Line

spr

ead

func

tion

1

0

(a)Original (2014× 3039) (b) Point Spread function

Pier Luigi DragottiSparse Sampling: Theory and Applications

Page 43: Sparse Sampling: Theory and Applicationspld/talks/ICMS09.pdf · such as ltered streams of Diracs, multi-dimensional signals and piecewise sinusoidal signals. 50 100 150 200 250 300

Application: Image Super-Resolution

(a)Original (128× 128) (b) Super-res (1024× 1024)

Pier Luigi DragottiSparse Sampling: Theory and Applications

Page 44: Sparse Sampling: Theory and Applicationspld/talks/ICMS09.pdf · such as ltered streams of Diracs, multi-dimensional signals and piecewise sinusoidal signals. 50 100 150 200 250 300

Application: Image Super-Resolution

(a)Original (48× 48) (b) Super-res (480× 480)

Pier Luigi DragottiSparse Sampling: Theory and Applications

Page 45: Sparse Sampling: Theory and Applicationspld/talks/ICMS09.pdf · such as ltered streams of Diracs, multi-dimensional signals and piecewise sinusoidal signals. 50 100 150 200 250 300

Conclusions

Sampling signals at their rate of innovation:

I New framework that allows the sampling and reconstruction of signals ata rate smaller than Nyquist rate.

I Robust and fast algorithms with complexity proportional to the number ofdegrees of freedom.

I Provable optimality (i.e. CRB achieved over wide SNR ranges).

I Intriguing connections with multi-resolution analysis, Fourier theory andanalogue circuit theory.

But also

I There is no such thing as a free lunch. Core application is difficult.

I Still many open questions from theory to practice.

Pier Luigi DragottiSparse Sampling: Theory and Applications

Page 46: Sparse Sampling: Theory and Applicationspld/talks/ICMS09.pdf · such as ltered streams of Diracs, multi-dimensional signals and piecewise sinusoidal signals. 50 100 150 200 250 300

ReferencesOn sampling

I T. Blu, P.L. Dragotti, M. Vetterli, P. Marziliano and L. Coulot ‘Sparse Samplingof Signal Innovations: Theory, Algorithms and Performance Bounds,’ IEEESignal Processing Magazine, vol. 25(2), pp. 31-40, March 2008

I P.L. Dragotti, M. Vetterli and T. Blu, ‘Sampling Moments and ReconstructingSignals of Finite Rate of Innovation: Shannon meets Strang-Fix’, IEEE Trans.on Signal Processing, vol.55 (5), pp.1741-1757, May 2007.

I J.Berent and P.L. Dragotti, and T. Blu,‘Sampling Piecewise Sinusoidal Signalswith Finite Rate of Innovation Methods,’ accepted for publication in IEEETransactions on Signal Processing, July 2009

On Image Super-Resolution

I L. Baboulaz and P.L. Dragotti, ‘Exact Feature Extraction using Finite Rate ofInnovation Principles with an Application to Image Super-Resolution’, IEEETrans. on Image Processing, vol.18(2), pp. 281-298, February 2009.

On compression

I V. Chaisinthop and P.L. Dragotti, ‘ Semi-Parametric Compression ofPiecewise-Smooth Functions’, in Proc. of European Conference on SignalProcessing (EUSIPCO), Glasgow, UK, August 2009.

Pier Luigi DragottiSparse Sampling: Theory and Applications