inverse problems and sparse representations€¦ · r. gribonval - sparse methods - sisea 2011...
Post on 10-Jun-2020
4 Views
Preview:
TRANSCRIPT
Inverse problems and sparse representations
Rémi Gribonval, DR INRIAEPI METISS (Speech and Audio Processing)INRIA Rennes - Bretagne Atlantique
remi.gribonval@inria.frhttp://www.irisa.fr/metiss/members/remi/talks
mercredi 30 novembre 2011
NOVEMBER 2011R. GRIBONVAL - SPARSE METHODS - SISEA 2011
Further material on sparsity
2
• Books with a Signal Processing perspective✦ S. Mallat, «Wavelet Tour of Signal Processing», 3rd edition, 2008✦ M. Elad, «Sparse and Redundant Representations: From Theory to
Applications in Signal and Image Processing», 2009.
• Review paper: ✦ Bruckstein, Donoho, Elad, SIAM Reviews, 2009
• Video lectures✦ E. Candès, MLSS’09✦ F. Bach, NIPS 2009✦ Sparsity in Machine Learning and Statistics SMLS’09
• Slides of this course:✦ http://www.irisa.fr/metiss/gribonval/Teaching/SISEA-2011/
mercredi 30 novembre 2011
NOVEMBER 2011R. GRIBONVAL - SPARSE METHODS - SISEA 2011
Reminder of last session
• Session 1: Introduction✓ sparsity & data compression✓ inverse problems in signal and image processing
✦ image deblurring, image inpainting, ✦ channel equalization, signal separation, ✦ tomography, MRI
✓ sparsity & under-determined inverse problems✦ well-posedness
3
mercredi 30 novembre 2011
NOVEMBER 2011R. GRIBONVAL - SPARSE METHODS - SISEA 2011
Overview of the course
• Session 2: Complexity & Feasibility✓ NP-completeness of ideal sparse approximation✓ Relaxations✓ L1 is sparsity-inducing and convex
• Session 3: Pursuit Algorithms
• Session 4: Recovery Guarantees
• Session 5: How to choose dictionaries
4
mercredi 30 novembre 2011
NOVEMBER 2011 - R. GRIBONVAL - SPARSE METHODS - SISEA 2011
Session 2: Complexity & Feasibility
NP-completeness of ideal sparse approximationRelaxed optimization principlesL1 induces sparsity
mercredi 30 novembre 2011
NOVEMBER 2011R. GRIBONVAL - SPARSE METHODS - SISEA 2011
Ideal sparse approximation
• Input:m x N matrix A, with m < N, m-dimensional vector b
• Possible objectives:find the sparsest approximation within tolerance
find best approximation with given sparsity
find a solution x to
6
arg minx�x�0, s.t.�b−Ax� ≤ �
arg minx�b−Ax�, s.t.�x�0 ≤ k
�b−Ax� ≤ �, and �x�0 ≤ k
mercredi 30 novembre 2011
NOVEMBER 2011R. GRIBONVAL - SPARSE METHODS - SISEA 2011
Ideal Sparse Approximation
• Brute force search
• Theorem (Davies et al, Natarajan)
It is NP-hard!• Are there other more efficient alternatives ?
7
mercredi 30 novembre 2011
NOVEMBER 2011R. GRIBONVAL - SPARSE METHODS - SISEA 2011
Ideal Sparse Approximation
• Brute force search
• Theorem (Davies et al, Natarajan)
It is NP-hard!• Are there other more efficient alternatives ?
7
mercredi 30 novembre 2011
NOVEMBER 2011R. GRIBONVAL - SPARSE METHODS - SISEA 2011
Ideal Sparse Approximation
• Brute force search
• Theorem (Davies et al, Natarajan)
It is NP-hard!• Are there other more efficient alternatives ?
7
mercredi 30 novembre 2011
NOVEMBER 2011R. GRIBONVAL - SPARSE METHODS - SISEA 2011
Ideal Sparse Approximation
• Brute force search
• Theorem (Davies et al, Natarajan)
It is NP-hard!• Are there other more efficient alternatives ?
7
mercredi 30 novembre 2011
NOVEMBER 2011R. GRIBONVAL - SPARSE METHODS - SISEA 2011
Ideal Sparse Approximation
• Brute force search
• Theorem (Davies et al, Natarajan)
It is NP-hard!• Are there other more efficient alternatives ?
7
mercredi 30 novembre 2011
NOVEMBER 2011R. GRIBONVAL - SPARSE METHODS - SISEA 2011
Ideal Sparse Approximation
• Brute force search
• Theorem (Davies et al, Natarajan)
It is NP-hard!• Are there other more efficient alternatives ?
7
mercredi 30 novembre 2011
NOVEMBER 2011R. GRIBONVAL - SPARSE METHODS - SISEA 2011
Ideal Sparse Approximation
• Brute force search
• Theorem (Davies et al, Natarajan)
It is NP-hard!• Are there other more efficient alternatives ?
7
mercredi 30 novembre 2011
NOVEMBER 2011R. GRIBONVAL - SPARSE METHODS - SISEA 2011
Ideal Sparse Approximation
• Brute force search
• Theorem (Davies et al, Natarajan)
It is NP-hard!• Are there other more efficient alternatives ?
7
mercredi 30 novembre 2011
NOVEMBER 2011R. GRIBONVAL - SPARSE METHODS - SISEA 2011
Ideal Sparse Approximation
• Brute force search
• Theorem (Davies et al, Natarajan)
It is NP-hard!• Are there other more efficient alternatives ?
7
mercredi 30 novembre 2011
NOVEMBER 2011R. GRIBONVAL - SPARSE METHODS - SISEA 2011
Ideal Sparse Approximation
• Brute force search
• Theorem (Davies et al, Natarajan)
It is NP-hard!• Are there other more efficient alternatives ?
7
mercredi 30 novembre 2011
NOVEMBER 2011R. GRIBONVAL - SPARSE METHODS - SISEA 2011
Ideal Sparse Approximation
• Brute force search
• Theorem (Davies et al, Natarajan)
It is NP-hard!• Are there other more efficient alternatives ?
7
mercredi 30 novembre 2011
NOVEMBER 2011R. GRIBONVAL - SPARSE METHODS - SISEA 2011
Ideal Sparse Approximation
• Brute force search
• Theorem (Davies et al, Natarajan)
It is NP-hard!• Are there other more efficient alternatives ?
Many k-tuples to try!
7
mercredi 30 novembre 2011
NOVEMBER 2011R. GRIBONVAL - SPARSE METHODS - SISEA 2011
Elements of Complexity Theory
• Polynomial algorithm: given input of size N, compute output in cost poly(N)
• A problem is polynomial (is in P) if there exists a polynomial algorithm computing the solution to each instance of the problem
•8
mercredi 30 novembre 2011
NOVEMBER 2011R. GRIBONVAL - SPARSE METHODS - SISEA 2011
Example
• Problem: find the nearest neighbor to an m-dimensional vector from a collection of N such vectors
✓ input size: m x (N+1)
✓ complexity: O(Nm) [N distances in ]
9
Rm
mercredi 30 novembre 2011
NOVEMBER 2011R. GRIBONVAL - SPARSE METHODS - SISEA 2011
Complexity: the class NP
• Decision problem: of the type “does there exist x satisfying a given set of constraints”
• A decision problem is «non-deterministic polynomial» (in the class NP) if there exists a polynomial algorithm that checks (for any instance of the problem) if a given candidate solution x satisfies the constraint.✦ warning: the algorithm is not required to find a solution. It merely has
to check if a solution x (given by an “oracle”) is acceptable.
10
mercredi 30 novembre 2011
NOVEMBER 2011R. GRIBONVAL - SPARSE METHODS - SISEA 2011
Complexity comparisons
• Polynomial reducibility : ✓ Problem A can be reduced to Problem B if every
instance of Problem A can be transformed into an instance of Problem B in polynomial time
11
A “at most as complex” as B
reduction
Problem BProblem A
polynomial
find or checksolution
polynomial
mercredi 30 novembre 2011
NOVEMBER 2011R. GRIBONVAL - SPARSE METHODS - SISEA 2011
Complexity: NP-complete problems
• Problem B is NP-hard if every Problem A in NP can be polynomially reduced to B.
• NP-complete problems: NP-hard + in NP• Fact: there exists at least one NP-complete problem
(satisfiability problem = SAT)
12
Class NPClass P
mercredi 30 novembre 2011
NOVEMBER 2011R. GRIBONVAL - SPARSE METHODS - SISEA 2011
Complexity: NP-complete problems
• Problem B is NP-hard if every Problem A in NP can be polynomially reduced to B.
• NP-complete problems: NP-hard + in NP• Fact: there exists at least one NP-complete problem
(satisfiability problem = SAT)
12
Class NPClass P
SAT
mercredi 30 novembre 2011
NOVEMBER 2011R. GRIBONVAL - SPARSE METHODS - SISEA 2011
NP complete problems
13
• Fact: there exists at least one NP-complete problem (satisfiability problem = SAT)
• A problem in NP is NP-complete if and only if some known NP-complete problem reducible to it
• All NP-complete problems are «as hard»
Class NPClass P
SAT
mercredi 30 novembre 2011
NOVEMBER 2011R. GRIBONVAL - SPARSE METHODS - SISEA 2011
Complexity of sparse approximation
• Step 1: express it as a decision problem:✓ description of an instance
m x N matrix A, m-dimensional vector b, parameters
✓ size of an instance = approximately mN✓ decision problem: does there exists x such that
• Step 2: prove it is in NP. Indeed, one can check in polynomial time O(mN) whether a given x satisfies the constraints
• Step 3: reduce an existing NP-complete problem to it to show it is NP-complete
14
(�, k)
�b−Ax� ≤ �, and �x�0 ≤ k
mercredi 30 novembre 2011
NOVEMBER 2011R. GRIBONVAL - SPARSE METHODS - SISEA 2011
NP-completeness of sparse approximation
• Which known NP-complete problem? ✓ 3SET : Exact-cover by 3-sets
✦ Description of an instance: • The integer interval
• A collection of N subsets of size 3
✦ Decision problem:• does there exist an exact cover (=disjoint partition) of E from elements of C ?
15
[Davis & al 1997](other problem chosen in [Natarajan 1995])
E = �1, 3k�C = {Fn, 1 ≤ n ≤ N}, Fn ⊂ E, �Fn = 3
∃?Λ,∪n∈ΛFn = E n �= n� ∈ Λ ⇒ Fn ∩ Fn� = ∅
mercredi 30 novembre 2011
NOVEMBER 2011R. GRIBONVAL - SPARSE METHODS - SISEA 2011
NP-completeness of Sparse Approximation
• Reduction of 3-SETS to sparse approximation✓ m=3k✓ vector✓ matrix✓ tolerance
• Exact cover implies existence of x such that
• Non-exact cover implies the opposite
A = (ain)1≤i≤m,1≤n≤N ain =�
1, i ∈ Fn
0, otherwise
b = (bi)mi=1 bi = 1,∀i
� < 1
�b−Ax� ≤ �, and �x�0 ≤ k
9gap=1
1 3k
1 3k
mercredi 30 novembre 2011
NOVEMBER 2011R. GRIBONVAL - SPARSE METHODS - SISEA 2011
Ideal sparse approximation: a summary
• Well-posed if:✓ Kruskal rank large enough
• But: ✓ ... L0 minimization NP-complete to solve✓ ... Kruskal rank NP-complete to compute
• So do we give up ?
17
arg minx�b−Ax�, s.t.�x�0 ≤ k
mercredi 30 novembre 2011
NOVEMBER 2011 - R. GRIBONVAL - SPARSE METHODS - SISEA 2011
The End
mercredi 30 novembre 2011
NOVEMBER 2011R. GRIBONVAL - SPARSE METHODS - SISEA 2011
• Audio = superimposition of structures
• Example : glockenspiel
✓ transients = short, small scale✓ harmonic part = long, large scale
• Gabor atoms
Sparse ApproximationWith Time-Frequency Atoms
�gs,τ,f (t) =
1√sw
�t− τ
s
�e2iπft
�
s,τ,f
19
mercredi 30 novembre 2011
NOVEMBER 2011R. GRIBONVAL - SPARSE METHODS - SISEA 2011
• Audio = superimposition of structures
• Example : glockenspiel
✓ transients = short, small scale✓ harmonic part = long, large scale
• Gabor atoms
Sparse ApproximationWith Time-Frequency Atoms
�gs,τ,f (t) =
1√sw
�t− τ
s
�e2iπft
�
s,τ,f
19
mercredi 30 novembre 2011
NOVEMBER 2011R. GRIBONVAL - SPARSE METHODS - SISEA 2011
• Audio = superimposition of structures
• Example : glockenspiel
✓ transients = short, small scale✓ harmonic part = long, large scale
• Gabor atoms
Sparse ApproximationWith Time-Frequency Atoms
�gs,τ,f (t) =
1√sw
�t− τ
s
�e2iπft
�
s,τ,f
19
mercredi 30 novembre 2011
NOVEMBER 2011R. GRIBONVAL - SPARSE METHODS - SISEA 2011
• Audio = superimposition of structures
• Example : glockenspiel
✓ transients = short, small scale✓ harmonic part = long, large scale
• Gabor atoms
Sparse ApproximationWith Time-Frequency Atoms
�gs,τ,f (t) =
1√sw
�t− τ
s
�e2iπft
�
s,τ,f
19
mercredi 30 novembre 2011
NOVEMBER 2011R. GRIBONVAL - SPARSE METHODS - SISEA 2011
Ideal Sparse Approximation: overcoming NP-completeness
• Seek approximate solution in polynomial time ?
• Find sub-problems with polynomial time solution ?
• Use randomized algorithm with polynomial time and high probability of correctness of the solution ?
• Use heuristic algorithm (greedy, etc.) ?
20
mercredi 30 novembre 2011
NOVEMBER 2011 - R. GRIBONVAL - SPARSE METHODS - SISEA 2011
Session 2: Complexity & Feasibility
NP-completeness of ideal sparse approximationRelaxed optimization principlesL1 induces sparsity
mercredi 30 novembre 2011
NOVEMBER 2011R. GRIBONVAL - SPARSE METHODS - SISEA 2011
Overall compromise
• Approximation quality
• Ideal sparsity measure : “norm”
• “Relaxed” sparsity measures
�Ax− b�2
�0
22
0 < p < ∞, �x�p :=� �
n
|xn|p�1/p
�x�0 := �{n, xn �= 0} =�
n
|xn|0
mercredi 30 novembre 2011
NOVEMBER 2011R. GRIBONVAL - SPARSE METHODS - SISEA 2011
Norms ?
23
mercredi 30 novembre 2011
NOVEMBER 2011R. GRIBONVAL - SPARSE METHODS - SISEA 2011
Lp norms / quasi-norms
• Norms when
• Quasi-norms when
• “Pseudo”-norm for p=0
24
1 ≤ p <∞
0 < p < 1
Triangle inequality
�x�p = 0 ⇔ x = 0�λx�p = |λ|�x�p,∀λ, x
�x + y�p ≤ �x�p + �y�p,∀x, y
Quasi-triangle inequality�x + y�p ≤ 21/p
��x�p + �y�p
�,∀x, y
�x + y�pp ≤ �x�p
p + �y�pp,∀x, y
�x + y�0 ≤ �x�0 + �y�0,∀x, y
= convex
= nonconvex
mercredi 30 novembre 2011
NOVEMBER 2011R. GRIBONVAL - SPARSE METHODS - SISEA 2011
Optimization problems
• Approximation
• Sparsification
• Regularization
25
minx�b−Ax�2 s.t. �x�p ≤ τ
minx�x�p s.t. �b−Ax�2 ≤ �
minx
12�b−Ax�2 + λ�x�p
mercredi 30 novembre 2011
NOVEMBER 2011R. GRIBONVAL - SPARSE METHODS - SISEA 2011
Lp “norms” level sets
• Strictly convex when p>1
• Convex p=1 • Nonconvex p<1
26
{x s.t.b = Ax}Observation: the minimizer is sparse
mercredi 30 novembre 2011
NOVEMBER 2011 - R. GRIBONVAL - SPARSE METHODS - SISEA 2011
Session 2: Complexity & Feasibility
NP-completeness of ideal sparse approximationRelaxed optimization principlesL1 induces sparsity
mercredi 30 novembre 2011
NOVEMBER 2011R. GRIBONVAL - SPARSE METHODS - SISEA 2011
L1 induces sparsity (1)
• Real-valued case✓ A = an m x N real-valued matrix✓ b = an m-dimensional real-valued vector✓ X = set of all minimum L1 norm solutions to
• Fact 1: X is convex and contains a “sparse” solution
28
Ax = b
∃x0 ∈ X, �x0�0 ≤ m
x̃ ∈ X ⇔ �x̃�1 = min �x�1 s.t. Ax = b
mercredi 30 novembre 2011
NOVEMBER 2011R. GRIBONVAL - SPARSE METHODS - SISEA 2011
Proof ? Exercice!
29
mercredi 30 novembre 2011
NOVEMBER 2011R. GRIBONVAL - SPARSE METHODS - SISEA 2011
Convex sets
• Definition:
30
θu + (1− θ)v ∈ X, ∀u,v ∈ X, 0 ≤ θ ≤ 1
Convex Not convex
mercredi 30 novembre 2011
NOVEMBER 2011R. GRIBONVAL - SPARSE METHODS - SISEA 2011
Convex sets
• Definition:
30
θu + (1− θ)v ∈ X, ∀u,v ∈ X, 0 ≤ θ ≤ 1
Convex Not convex
mercredi 30 novembre 2011
NOVEMBER 2011R. GRIBONVAL - SPARSE METHODS - SISEA 2011
Convex sets
• Definition:
30
θu + (1− θ)v ∈ X, ∀u,v ∈ X, 0 ≤ θ ≤ 1
Convex Not convex
mercredi 30 novembre 2011
NOVEMBER 2011R. GRIBONVAL - SPARSE METHODS - SISEA 2011
Convex functions
• Definition: f is convex if
31
f(θu + (1− θ)v) ≤ θf(u) + (1− θ)f(v)
∀u,v, 0 ≤ θ ≤ 1
ConvexNot
convex
mercredi 30 novembre 2011
NOVEMBER 2011R. GRIBONVAL - SPARSE METHODS - SISEA 2011
Convex functions
• Definition: f is convex if
31
f(θu + (1− θ)v) ≤ θf(u) + (1− θ)f(v)
∀u,v, 0 ≤ θ ≤ 1
ConvexNot
convex
mercredi 30 novembre 2011
NOVEMBER 2011R. GRIBONVAL - SPARSE METHODS - SISEA 2011
Convex functions
• Definition: f is convex if
31
f(θu + (1− θ)v) ≤ θf(u) + (1− θ)f(v)
∀u,v, 0 ≤ θ ≤ 1
ConvexNot
convex
mercredi 30 novembre 2011
NOVEMBER 2011R. GRIBONVAL - SPARSE METHODS - SISEA 2011
Proof ? Exercice!
• Convexity of the set of solutions X: ✓ let✓ convexity of constraint
✓ by definition
✓ convexity of objective function
✓ hence
32
Ax = Ax� = A(θx + (1− θ)x�) = b
x, x� ∈ X, 0 ≤ θ ≤ 1
�x�1 = �x��1 = min �x̃�1 s.t. Ax̃ = b
�(θx + (1− θ)x�)�1 ≤ θ�x�1 + (1− θ)�x��1 = �x�1
θx + (1− θ)x� ∈ X
mercredi 30 novembre 2011
NOVEMBER 2011R. GRIBONVAL - SPARSE METHODS - SISEA 2011
• Existence of a sparse solution✓ let x satisfy with
✦ support
✦ sub-matrix
✓ existence of nontrivial null space vector✓ other solution ✓ for small
Proof? Exercice!
33
I := supp(x) := {i, xi �= 0}
A AI
� ≥ m + 1
→m
N
AIz = 0
��x + �z�1 =
�
i∈I
|xi + �zi| +�
i/∈I
|xi + �zi|
Ax = b �x�0 ≥ m + 1
m
x� = x + �z
mercredi 30 novembre 2011
NOVEMBER 2011R. GRIBONVAL - SPARSE METHODS - SISEA 2011
• Existence of a sparse solution✓ let x satisfy with
✦ support
✦ sub-matrix
✓ existence of nontrivial null space vector✓ other solution ✓ for small
Proof? Exercice!
33
I := supp(x) := {i, xi �= 0}
A AI
� ≥ m + 1
→m
N
AIz = 0
��x + �z�1 =
�
i∈I
|xi + �zi| +�
i/∈I
|xi + �zi|
Ax = b �x�0 ≥ m + 1
m
x� = x + �z
mercredi 30 novembre 2011
NOVEMBER 2011R. GRIBONVAL - SPARSE METHODS - SISEA 2011
• Existence of a sparse solution✓ let x satisfy with
✦ support
✦ sub-matrix
✓ existence of nontrivial null space vector✓ other solution ✓ for small
Proof? Exercice!
33
I := supp(x) := {i, xi �= 0}
A AI
� ≥ m + 1
→m
N
AIz = 0
��x + �z�1 =
�
i∈I
|xi + �zi| +�
i/∈I
|xi + �zi|
=�
i∈I
sign(xi)(xi + �zi)
Ax = b �x�0 ≥ m + 1
m
x� = x + �z
mercredi 30 novembre 2011
NOVEMBER 2011R. GRIBONVAL - SPARSE METHODS - SISEA 2011
• Existence of a sparse solution✓ let x satisfy with
✦ support
✦ sub-matrix
✓ existence of nontrivial null space vector✓ other solution ✓ for small
Proof? Exercice!
33
I := supp(x) := {i, xi �= 0}
A AI
� ≥ m + 1
→m
N
AIz = 0
��x + �z�1 =
�
i∈I
|xi + �zi| +�
i/∈I
|xi + �zi|
=�
i∈I
sign(xi)(xi + �zi) = �x�1 + ��
i∈I
sign(xi)zi
Ax = b �x�0 ≥ m + 1
m
x� = x + �z
mercredi 30 novembre 2011
NOVEMBER 2011R. GRIBONVAL - SPARSE METHODS - SISEA 2011
• Existence of a sparse solution✓ let x satisfy with
✦ support
✦ sub-matrix
✓ existence of nontrivial null space vector✓ other solution ✓ for small
Proof? Exercice!
33
I := supp(x) := {i, xi �= 0}
A AI
� ≥ m + 1
→m
N
AIz = 0
��x + �z�1 =
�
i∈I
|xi + �zi| +�
i/∈I
|xi + �zi|
=�
i∈I
sign(xi)(xi + �zi) = �x�1 + ��
i∈I
sign(xi)zi
Ax = b �x�0 ≥ m + 1
m
x� = x + �zcost function
is not minimum
mercredi 30 novembre 2011
NOVEMBER 2011R. GRIBONVAL - SPARSE METHODS - SISEA 2011
Convexity of the set of minimizers
• Unique solution • Non unique solution
34
mercredi 30 novembre 2011
NOVEMBER 2011R. GRIBONVAL - SPARSE METHODS - SISEA 2011
L1 induces sparsity (2)
• Real-valued case✓ A = an m x N real-valued matrix✓ b = an m-dimensional real-valued vector✓ X = set of al solutions to regularization problem
• Fact 2: X is a convex set and contains a “sparse” solution
35
∃x0 ∈ X, �x0�0 ≤ m
L(x) :=12�Ax− b�2
2 + λ�x�1
x̃ ∈ X ⇔ L(x̃) = minxL(x)
mercredi 30 novembre 2011
NOVEMBER 2011R. GRIBONVAL - SPARSE METHODS - SISEA 2011
Proof ? Exercice!
36
mercredi 30 novembre 2011
NOVEMBER 2011R. GRIBONVAL - SPARSE METHODS - SISEA 2011
L1 induces sparsity
• A word of caution: this does not hold true in the complex-valued case
• Counter example: there is a construction where✓ A = a 2 x 3 complex-valued matrix✓ b = a 2-dimensional complex-valued vector✓ the minimum L1 norm solution is unique and has 3
nonzero components
37
[E. Vincent, Complex Nonconvex Optimization l_p norm minimization for underdetermined source separation, Proc. ICA 2007.]
mercredi 30 novembre 2011
NOVEMBER 2011R. GRIBONVAL - SPARSE METHODS - SISEA 2011
Global Optimization : from Principles to Algorithms
• Optimization principle
✓ Sparse representation✓ Sparse approximation
local minima convex : global minimum
NP-hard combinatorial Iterative thresholding / proximal algo.FOCUSS / IRLS Linear
Lasso [Tibshirani 1996], Basis Pursuit (Denoising) [Chen, Donoho & Saunders, 1999]
Linear/Quadratic programming (interior point, etc.) Homotopy method [Osborne 2000] / Least Angle Regression [Efron &al 2002]
Iterative / proximal algorithms [Daubechies, de Frise, de Mol 2004, Combettes & Pesquet 2008, ...]
38
λ→ 0λ > 0
minx
12�Ax− b�2
2 + λ�x�pp
Ax = bAx ≈ b
mercredi 30 novembre 2011
top related