fisher metric, support stability and optimal number of ... · joint work with clarice poon...

Fisher metric, support stability and optimal number of measurements in

compressive off-the-grid recovery

Nicolas KerivenEcole Normale Supérieure (Paris)

CFM-ENS chair in Data Science

Joint work with Clarice Poon (Cambridge Uni.), Gabriel Peyré (ENS)

Discrete compressive sensing

1/14

• Signal: vector


1/14

• Signal: vector• Sparsity: few non-zeros coefficients


1/14

• Signal: vector• Sparsity: few non-zeros coefficients• Dimensionality reduction (often random matrix)


1/14

• Signal: vector• Sparsity: few non-zeros coefficients• Dimensionality reduction (often random matrix)• Recovery: convex relaxation LASSO


1/14

Continuous sparsity ?

2/14

- Fourier coeff- Convolution


2/14



• Signal: Radon measure

2/14



• Signal: Radon measure• Sparsity:

2/14



• Signal: Radon measure• Sparsity:• Dimensionality reduction (e.g. first Fourier coefficients)

2/14



• Signal: Radon measure• Sparsity:• Dimensionality reduction (e.g. first Fourier coefficients)• Recovery: convex relaxation?

See Keriven 2017, Gribonval 2017

2/14



• Signal: Radon measure• Sparsity:• Dimensionality reduction (e.g. first Fourier coefficients)• Recovery: convex relaxation? BLASSO [De Castro, Gamboa 2012]


2/14



• Signal: Radon measure• Sparsity:• Dimensionality reduction (e.g. first Fourier coefficients)• Recovery: convex relaxation? BLASSO [De Castro, Gamboa 2012]

Other approaches: « Prony-like » ESPRIT, MUSIC… (but only 1d noiseless Fourier)


2/14

Example of applications

Fluorescence microscopy (3D) PALM, STORM… [Betzig 2006]

3/14



Astronomy (2D)[Puschmann 2017]

3/14



Compressive mixture model learning(many D)[Keriven 2017]


3/14



Compressive mixture model learning(many D)[Keriven 2017]


• Neuro-imaging with EEG (3D) [Gramfort 2013]

• 1-layer neural network (many D) [Bach 2017]

• Radar• Geophysics• …

3/14

A seminal result [Candès, Fernandez-Granda 2012]

4/14


First Fourier coefficients

4/14



Inverse Fourier

4/14



Inverse Fourier

BLASSO

4/14



• Regular Fourier on the Torus

• Minimal separation

Inverse Fourier

BLASSO

4/14





• Reconstruction: formulated as SDP (other: Frank-Wolfe,

greedy, Prony-like…)

Inverse Fourier

BLASSO

4/14







• Noise: weak convergence (mass of concentrated

around true Diracs)

Inverse Fourier

BLASSO

4/14







• Noise: weak convergence (mass of concentrated

around true Diracs)

Inverse Fourier

BLASSO

4/14

Many extensions since…

Previous work, contribution

5/14

Relevant previous works:

• [Tang, Recht 2013]: random Fourier coefficients are sufficient


5/14


• [Tang, Recht 2013]: random Fourier coefficients are sufficient• Random signs assumption• 1D discrete Fourier


5/14



• [Bendory et al. 2016]: extension to other measurement operators


5/14



• [Bendory et al. 2016]: extension to other measurement operators• Minimal separation does not take into account geometry of the meas. operator


5/14




• [Duval, Peyré 2015]: In some cases, support stability in the small noise regime


5/14




• [Duval, Peyré 2015]: In some cases, support stability in the small noise regime• Noise level under which support stability is achieved?


5/14





Contributions:

• Generalize to many multi-d measurement operators, express the minimal separation as a geometry-aware Fisher metric


5/14





Contributions:


• 1: Remove the random sign assumption (weak convergence)


5/14





Contributions:


• 1: Remove the random sign assumption (weak convergence)

• 2: Prove support stability when (with random signs)

Outline

Background on dual certificates

Minimal separation and Fisher metric

Conclusion, outlooks

Main results, applications

Dual certificates

6/14

Random linear operator:

Dual certificates

6/14

Noisy measurement:Random linear operator:

Dual certificates

6/14

The BLASSO problem:


Dual certificates

First-order conditions

solution ofBLASSO

6/14

The BLASSO problem:


solution of

Dual certificates

First-order conditions

Dual certificate (noiseless case)

solution ofBLASSO

6/14

The BLASSO problem:


What is a dual certificate ?

What does it look like ?

7/14

Case :



7/14

Case :



7/14

Ensures uniqueness and robustness…

Non-degenerate dual certif.

Case :



7/14

Ensures uniqueness and robustness…

Non-degenerate dual certif.

Intuitively:

Larger -> easierinterpolation

Proof strategy

8/14

Step 1: Study the limit case to derive an appropriate notion of minimal separation

Proof strategy

8/14


Step 2: bound the deviation for finite number of measurements

Proof strategy

m=10

8/14



Proof strategy

m=10 m=50

8/14



Proof strategy

m=500m=10 m=50

8/14



Proof strategy

m=500m=10 m=50

8/14



Step 3: recovery• Adaptation of [Azaïs 2015] for weak convergence• Quantitative Implicit Function Theorem [Denoyelle 2015] for support stability

Outline





Limit covariance kernels

9/14

How to construct a certificate ?Study limit covariance kernel when :


9/14


Sub-sampled version:


9/14


Strategy under minimal separation



Min. Separation

9/14





Min. Separation

1: kernel at each saturation point

9/14





Min. Separation

2: Small adjustements(minimal separation)

1: kernel at each saturation point

9/14





10/14

Which metric for separation ?


10/14


Classical case: translation-invariant kernel


10/14


Classical case: translation-invariant kernel

natural


10/14


Classical case: translation-invariant kernel Non translation-inv. ?

Kernel for microscopynatural


10/14



Kernel for microscopy

Riemannian metric associated to a kernel [Amari 99]:

: metric tensor : geodesic distance

natural


10/14



Kernel for microscopy

Riemannian metric associated to a kernel [Amari 99]:

: metric tensor : geodesic distance

Thm: under some hypothesis, for , there exists non-degenerate

natural

Examples

16/04/2018

Features

Fisher metric and minimal separation

Kernel

11/14

Examples

16/04/2018

Features


Discrete Fourier on Torus:Féjer kernel

Kernel

11/14

Examples

16/04/2018

Features



Continuous Gaussian Fourier:Gaussian kernel

Kernel

11/14

Examples

16/04/2018

Features



Continuous Gaussian Fourier:Gaussian kernel

Microscopy (Laplace transform):

Kernel

11/14

Outline





Main results

Thm: Eliminating the random signs

12/14

Main results


Depend on kernel

12/14

Main results


• The recovered measure concentrate around true Diracs

Depend on kernel

12/14

Main results



• Proof: golfing scheme [Gross 2009, Candès Plan 2011]

Depend on kernel

12/14

Main results




Thm: Support stability

Depend on kernel

12/14

Main results





Depend on kernel

With random signs Without random signs

12/14

Main results





Depend on kernel


12/14

• Quantified small noise : if , then:

Main results





Depend on kernel


12/14

• Quantified small noise : if , then:

• The recovered measure is formed of exactly Diracs

Applications

16/04/2018

• Féjer kernel (discrete Fourier):

13/14

Applications

16/04/2018


• Microscopy with Laplace transform:

13/14

Applications

16/04/2018



• Application: Gaussian Mixture Model learning

13/14

Applications

16/04/2018




• Given

13/14

Applications

16/04/2018




• Given

• Compute with Gaussian

13/14

(streaming, distributed)

Applications

16/04/2018




• Given


• Solve the BLASSO with as characteristic function of a Gaussian

13/14


Applications

16/04/2018




• Given


• Solve the BLASSO with as characteristic function of a Gaussian

Then: if

The BLASSO yields exactly s Diracs: non-asymptotic model selection !

13/14


Outline





Summary, outlooks

• Summary: generalization of existing results on super-resolutionwith random measurements (and minimal separation)

• Introduction of the kernel Fisher metric to measure minimal separation• Application in particular to a non-translation-invariant example for microscopy

• No need of random signs for weak convergence (golfing scheme)• Quantitative support stability

14/14

Summary, outlooks

• Summary: generalization of existing results on super-resolutionwith random measurements (and minimal separation)

• Introduction of the kernel Fisher metric to measure minimal separation• Application in particular to a non-translation-invariant example for microscopy

• No need of random signs for weak convergence (golfing scheme)• Quantitative support stability

14/14

• Outlooks• Implication of support stability for algorithms ? (active field)• Better characterization of the « universality » of the geodesic distance• More quantified treatment of dimension• Other practical applications (eg 1-layer neural networks with continuum of

neurons [Bach 2017])

Poon, Keriven, Peyré. A Dual Certificates Analysis of Compressive Off-the-Grid Recovery. Preprint arxiv:1802.08464

Poon, Keriven, Peyré. Support Localization and the Fisher Metric for off-the-grid Sparse Regularization. Preprint arxiv:1810.03340

Thank you !

14/14

data-ens.github.ioEnter the data challenges!Come to the colloquium!Come to the Laplace seminars!

fisher metric, support stability and optimal number of ... · joint work with clarice poon...

Documents