sparse approximation by wavelet frames and applications
DESCRIPTION
Sparse Approximation by Wavelet Frames and Applications. Bin Dong Department of Mathematics The University of Arizona 2012 International Workshop on Signal Processing , Optimization, and Control June 30- July 3, 2012 USTC, Hefei, Anhui, China. Outlines. - PowerPoint PPT PresentationTRANSCRIPT
Sparse Approximation by Wavelet Frames and Applications
Bin Dong
Department of Mathematics
The University of Arizona
2012 International Workshop on
Signal Processing , Optimization, and ControlJune 30- July 3, 2012
USTC, Hefei, Anhui, China
OutlinesI. Wavelet Frame Based Models for Linear
Inverse Problems (Image Restoration)
II. Applications in CT Reconstruction
1-norm based models
Connections to variational model
0-norm based model
Comparisons: 1-norm v.s. 0-norm
Quick Intro of Conventional CT Reconstruction
CT Reconstruction with Radon Domain Inpainting
Tight Frames in Orthonormal basis
Riesz basis
Tight frame: Mercedes-Benz frame
Expansions:Unique
Not unique
Tight Frames General tight frame systems
Tight wavelet frames
Construction of tight frame: unitary extension principles [Ron and Shen, 1997]
• They are redundant systems satisfying Parseval’s identity
• Or equivalently
where and
Tight Frames Example:
Fast transforms
Lecture notes: [Dong and Shen, MRA-Based Wavelet Frames and Applications, IAS Lecture Notes Series,2011]
Decomposition
Reconstruction
Perfect Reconstruction
Redundancy
Image Restoration Model Image Restoration Problems
Challenges: large-scale & ill-posed
• Denoising, when is identity operator
• Deblurring, when is some blurring operator
• Inpainting, when is some restriction operator
• CT/MR Imaging, when is partial Radon/Fourier
transform
Frame Based Models Image restoration model:
Balanced model for image restoration [Chan, Chan, Shen and Shen, 2003], [Cai, Chan and Shen, 2008]
When , we have synthesis based model [Daubechies, Defrise and De Mol, 2004; Daubechies, Teschke and Vese, 2007]
When , we have analysis based model [Stark, Elad and Donoho, 2005; Cai, Osher and Shen, 2009]
Resembles Variational Models
Connections: Wavelet Transform and Differential Operators Nonlinear diffusion and iterative wavelet and wavelet
frame shrinkageo 2nd-order diffusion and Haar wavelet: [Mrazek,
Weickert and Steidl, 2003&2005]o High-order diffusion and tight wavelet frames in 1D:
[Jiang, 2011] Difference operators in wavelet frame transform:
True for general wavelet frames with various vanishing moments [Weickert et al., 2006; Shen and Xu, 2011]
Filters
Transform
Approximation
Connections: Analysis Based Model and Variational Model [Cai, Dong, Osher and Shen, Journal of the AMS,
2012]:
The connections give us
Leads to new applications of wavelet frames:
Converges
• Geometric interpretations of the wavelet frame transform (WFT)
• WFT provides flexible and good discretization for differential operators
• Different discretizations affect reconstruction results
• Good regularization should contain differential operators with varied
orders (e.g., total generalized variation [Bredies, Kunisch, and Pock,
2010])
Image segmentation: [Dong, Chien and Shen, 2010] Surface reconstruction from point clouds: [Dong and Shen, 2011]
For any differential operator when proper parameter is chosen.
Standard Discretization Piecewise Linear WFT
Frame Based Models: 0-Norm Nonconvex analysis based model [Zhang, Dong and
Lu, 2011]
Motivations:
Related work:
• Restricted isometry property (RIP) is not
satisfied for many applications
• Penalizing “norm” of frame coefficients
better balances sparsity and smoothnes
• “norm” with : [Blumensath and Davies,
2008&2009]
• quasi-norm with : [Chartrand,
2007&2008]
Fast Algorithm: 0-Norm Penalty decomposition (PD) method [Lu and Zhang, 2010]
Algorithm:
Change of variables
Quadratic penalty
Fast Algorithm: 0-Norm Step 1:
Subproblem 1a): quadratic
Subproblem 1b): hard-thresholding
Convergence Analysis [Zhang, Dong and Lu, 2011] :
Numerical Results Comparisons (Deblurring)
Balanced
Analysis
0-Norm
PFBS/FPC: [Combettes and Wajs, 2006] /[Hale, Yin and Zhang, 2010]
Split Bregman: [Goldstein and Osher, 2008] & [Cai, Osher and Shen, 2009]PD Method: [Zhang, Dong and Lu, 2011]
Numerical Results Comparisons
Portrait
Couple
Balanced Analysis
Faster Algorithm: 0-Norm Start with some fast optimization method for nonsmooth
and convex optimizations: doubly augmented Lagrangian (DAL) method [Rockafellar, 1976].
Given the problem:
The DAL method:
where
We solve the joint optimization problem of the DAL method using an
inexact alternative optimization scheme
Faster Algorithm: 0-Norm Start with some fast optimization method for nonsmooth
and convex optimizations: doubly augmented Lagrangian (DAL) method [Rockafellar, 1976].
The inexact DAL method:
Given the problem:
The DAL method:
where
Hard thresholding
Faster Algorithm: 0-Norm However, the inexact DAL method does not seem to
converge!! Nonetheless, the sequence oscillates and is bounded.
The mean doubly augmented Lagrangian method (MDAL) [Dong and Zhang, 2011] solve the convergence issue by using arithmetic means of the solution sequence as outputs instead:MDAL:
Comparisons: Deblurring Comparisons of best PSNR values v.s. various noise level
Comparisons: Deblurring Comparisons of computation time v.s. various noise level
Comparisons: Deblurring What makes “lena” so special?
Decay of the magnitudes of the wavelet frame coefficients is very fast, which is what 0-norm prefers.
Similar observation was made earlier by [Wang and Yin, 2010].
1-norm 0-norm: PD 0-norm: MDAL
APPLICATIONS IN CT RECONSTRUCTION
With the Center for Advanced Radiotherapy and Technology (CART), UCSD
Cone Beam CT
3D Cone Beam CT
xy
z
u
v
0
g(u)
f(x)
n0
xS
x
u*
Discrete
3D Cone Beam CT
=
Animation created by Dr. Xun Jia
Goal: solve
Difficulties:
Related work:
Cone Beam CT Image Reconstruction
Unknown Image
Projected Image
• In order to reduce dose, the system is highly underdetermined. Hence the solution is not unique.•Projected image is noisy.
Total Variation (TV): [Sidkey, Kao and Pan 2006], [Sidkey
and Pan, 2008], [Cho et al. 2009], [Jia et al. 2010];
EM-TV: [Yan et al. 2011]; [Chen et al. 2011];
Wavelet Frames: [Jia, Dong, Lou and Jiang, 2011];
Dynamical CT/4D CT: [Chen, Tang and Leng, 2008],
[Jia et al. 2010], [Tian et al., 2011]; [Gao et al. 2011];
CT Image Reconstruction with Radon Domain Inpainting Idea: start with
Benefits:
Instead of solving
We find both and such that:
• is close to but with better quality
•
• Prior knowledge of them should be used
• Safely increase imaging dose• Utilizing prior knowledge we have for both CT images and the projected images
CT Image Reconstruction with Radon Domain Inpainting Model [Dong, Li and Shen, 2011]
Algorithm: alternative optimization & split Bregman.
where
• p=1, anisotropic• p=2, isotropic
CT Image Reconstruction with Radon Domain Inpainting Algorithm [Dong, Li and Shen, 2011]: block
coordinate descend method [Tseng, 2001]
Convergence Analysis
Problem:
Algorithm:
Note: If each subproblem is solved exactly, then the convergence analysis was given by [Tseng, 2001], even for nonconvex problems.
CT Image Reconstruction with Radon Domain Inpainting Results: N denoting number of projections
N=15 N=20
CT Image Reconstruction with Radon Domain Inpainting Results: N denoting number of projections
N=15
N=20
W/O Inpainting With Inpainting
Thank You
Collaborators:
Mathematics Stanley Osher, UCLA Zuowei Shen, NUS Jia Li, NUS Jianfeng Cai, University of Iowa Yifei Lou, UCLA/UCSD Yong Zhang, Simon Fraser University, Canada Zhaosong Lu, Simon Fraser University, Canada
Medical School Steve B. Jiang, Radiation Oncology, UCSD Xun Jia, Radiation Oncology, UCSD Aichi Chien, Radiology, UCLA