robust orthonormal subspace learning: efficient recovery of corrupted low-rank matrices
DESCRIPTION
Low-rank matrix recovery from a corrupted observation has many applications in computer vision. Conventional methods address this problem by iterating between nuclear norm minimization and sparsity minimization. However, iterative nuclear norm minimization is computationally prohibitive for large-scale data (e.g., video) analysis. In this paper, we propose a Robust Orthogonal Subspace Learning (ROSL) method to achieve efficient low-rank recovery. Our intuition is a novel rank measure on the low-rank matrix that imposes the group sparsity of its coefficients under orthonormal subspace. We present an efficient sparse coding algorithm to minimize this rank measure and recover the low-rank matrix at quadratic complexity of the matrix size. We give theoretical proof to validate that this rank measure is lower bounded by nuclear norm and it has the same global minimum as the latter. To further accelerate ROSL to linear complexity, we also describe a faster version (ROSL+) empowered by random sampling. Our extensive experiments demonstrate that both ROSL and ROSL+ provide superior efficiency against the state-of-the-art methods at the same level of recovery accuracy.TRANSCRIPT
Xianbiao Shu1, Fatih Porikli2, Narendra Ahuja1
1xshu2,[email protected] [email protected]
1 2
Robust Orthonormal Subspace Learning: Efficient Recovery of Corrupted Low-rank Matrices
Introduction
• Low-rank recovery on Large-Scale Data and its Vision Applications
• Problem: to recover low-rank matrix from its corrupted observation
CVPR2014 2
Image UnderstandingClustering
classificationRecognition
Video Surveillancedenoising, compression background subtraction tracking, saliency alarm
Imagingcompressive
sensing[Shu11] dynamical MRI
Camera Registrationcamera calibration video stabilization
3D reconstruction[Mobahi11]
20 Miss Korean Contestants only 6 principal components [Huang14]
Robust Orthonormal Subspace Learning: Efficient Recovery of Corrupted Low-rank Matrices
Overview of Existing Methods
• Robust Principal Component Analysis (RPCA) [Candes11]
– State-of-the-art: convex, global minima guaranteed.
– Cubic complexity: , due to multiple rounds of SVD
– Running time: >300s on a small video clip (size:160x128, 1060 frames)
• Its accelerated methods
– Partial RPCA [Lin09]: only computes major singular values determine ?
– RP_RPCA [Mu11]: random projection , and minimize unstable
– GoDec [Zhou11]: uses bilateral random projection slow convergence
• Matrix factorization methods
– RMF[Ke05], LMaFit [Shen11] require exact rank estimate
• Efficient, stable, automatic(without requiring rank estimate) method?
CVPR2014 3
n
Robust Orthonormal Subspace Learning: Efficient Recovery of Corrupted Low-rank Matrices
Robust Orthonormal Subspace Learning(ROSL)
• Orthonormal Subspace Decomposition ,
– Rank initial subspace dimension
• New rank measures: given ,
number of nonzero rows
sum of magnitude of rows
• Problem formulation
Fast sparse coding at quadratic complexity .
Subspace dimension shrinks from to no requiring rank estimate
Non-convex optimization,
CVPR2014 4
rn k
m
Robust Orthonormal Subspace Learning: Efficient Recovery of Corrupted Low-rank Matrices
• ROSL replaces nuclear norm in RPCA by a new rank measure:
row-1 group sparsity under orthonormal subspace.
• Thus, ROSL shares the same global minima as the performance-
guaranteed RPCA.
Given a matrix A , define conventional and new rank measures
respectively as nuclear norm and row-1 group sparsity under
orthonormal subspace:
Then
holds.
Performance of ROSL
CVPR2014 5
Proposition
Robust Orthonormal Subspace Learning: Efficient Recovery of Corrupted Low-rank Matrices
Efficient Algorithm
• Lagrange Function
• Alternating Direction Method of Multipliers (ADMM)
– Subspace learning:
Group sparsity shrinkage sequentially updates
automatically shrinks subspace dim from to rank
– Solve error component:
– Update Lagrangian multiplier
• Overall complexity is quadratic
CVPR2014 6Robust Orthonormal Subspace Learning: Efficient Recovery of Corrupted Low-rank Matrices
ROSL+
• ROSL+: linear-complexity algorithm by random sampling (RS)
• Nystrom method[Talwalkar10]:
• Three major steps:
– Obtain XTL by random sampling h rows and l columns
– Solve and by applying ROSL on
+
– Solve coefficients by minimizing
||
• Final low-rank recovery
CVPR2014 7
AT =[ATL , ATR]
AL =[ATL ; ABL]
Robust Orthonormal Subspace Learning: Efficient Recovery of Corrupted Low-rank Matrices
Experimental Results
• Synthetic data (m=n, r=10, k=30, h=l=100), generated by
– Multiplying a matrix and a matrix, which obeys N(0,1).
– then adding a sparse error (sparsity:10%), drawn from Unif[-50, 50].
CVPR2014 9
rm k
m
MAE: Mean of Absolute Error
Robust Orthonormal Subspace Learning: Efficient Recovery of Corrupted Low-rank Matrices
Experimental Results
• ROSL at varying initial subspace dimension and weight
• ROSL+ at varying random sampling density (h = l)
CVPR2014 10
r=10, k=30
Robust Orthonormal Subspace Learning: Efficient Recovery of Corrupted Low-rank Matrices
Experimental Results
• Background subtraction in surveillance video (160x128,1060frames)
CVPR2014 11
Original RPCA(time:334s) ROSL(time:34.6s) ROSL+(time:3.61s)
Robust Orthonormal Subspace Learning: Efficient Recovery of Corrupted Low-rank Matrices
Experimental Results
• Illumination removal in Yale-B face dataset (168x192, 55 frames)
CVPR2014 12
Original RPCA (time:12.16s) ROSL(time: 5.85s)
Robust Orthonormal Subspace Learning: Efficient Recovery of Corrupted Low-rank Matrices
Reference• [Huang14]: J. Huang, http://jbhuang0604.blogspot.com/2013/04/miss-korea-2013-contestants-face.html
http://misskorea.mpluskorea.com/missdaegu2013_poll
• [Shu11]: X. Shu and N. Ahuja. Imaging via three-dimensional compressive sampling (3DCS). In ICCV, 2011.
• [Mobahi11]: H. Mobahi, Z. Zhou, A. Yang, and Y. Ma. Holistic Reconstruction of Urban Structures from Low-rank Textures In ICCV_3dRR,
2011.
• [Candes11]: E. Candes, X. Li, Y. Ma, and J. Wright. Robust principal component analysis? Journal of the ACM, 58(3):article 11, 2011.
• [Lin10]: Z. Lin, M. Chen, and Y. Ma. The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices.
Technical Report UILU-ENG-09-2214, UIUC, 2010
• [Mu11]: Y. Mu, J. Dong, X. Yuan, and S. Yan. Accelerated low-rank visual recovery by random projection. In CVPR, 2011.
• [Zhou11]: T. Zhou and D. Tao. GoDec: randomized low-rank andsparse matrix decomposition in noisy case. In ICML, 2011.
• [Ke05]: Q. Ke and T. Kanade. Robust l1 norm factorization in the presence of outliers and missing data by alternative convex programming.
In CVPR, volume 1, pages 739–746, 2005.
• [Shen11]: Y. Shen, Z.Wen, and Y. Zhang. Augmented lagrangian alternatingdirection method for matrix separation based on low-rank
factorization. Technical Report TR11-02, Rice University, 2011.
• [Talwalkar10]: A. Talwalkar and A. Rostamizadeh. Matrix coherence and the nystrom method. In Proceedings of the Twenty-Sixth
Conference Annual Conference on Uncertainty in Artificial Intelligence (UAI-10), 2010.
CVPR2014 13Robust Orthonormal Subspace Learning: Efficient Recovery of Corrupted Low-rank Matrices
Summary
• Robust Orthonormal Subspace Learning(ROSL)
– A new rank measure using row-1 group sparsity under orthogonal subspace,
which is lower bounded by nuclear norm.
– A fast low-rank recovery method (ROSL) with the same global minima as RPCA.
– An efficient algorithm is given to solve ROSL with stable convergence behavior
– An accelerated version (ROSL+) with linear complexity by random sampling
– Experimental results show that our ROSL/ROSL+ are much faster than the
performance-guaranteed method (RPCA) at the same level of accuracy.
CVPR2014 14
rn k
m
Robust Orthonormal Subspace Learning: Efficient Recovery of Corrupted Low-rank Matrices
Summary
Code is available
https://sites.google.com/site/xianbiaoshu/
Xianbiao Shu Fatih Porikli Narendra Ahuja
CVPR2014 15Robust Orthonormal Subspace Learning: Efficient Recovery of Corrupted Low-rank Matrices
Thanks
And
Questions?
CVPR2014 16Robust Orthonormal Subspace Learning: Efficient Recovery of Corrupted Low-rank Matrices