recent advances on low-rank and sparse decomposition for moving object detection
TRANSCRIPT
![Page 1: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/1.jpg)
Recent advances on low-rank and sparse decomposition for moving object detection
Matrix and Tensor-based approaches
Andrews Cordolino SobralPh.D. Student, Computer Vision
L3i / MIA, Université de La Rochellehttp://andrewssobral.wix.com/home
Atelier : Enjeux dans la détection d’objets mobiles par soustraction de fond
![Page 2: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/2.jpg)
Summary
● Context– Understanding an Intelligent Video Surveillance Framework
– Introduction to Background Subtraction
● Decomposition into Additive Matrices
– Case 1: Low-rank Approximation and Matrix Completion
– Case 2: Robust Principal Component Analysis (RPCA)
– Case 3: Stable decomposition● Constrained RPCA
● Introduction to Tensors
– Tensor Decomposition● Tucker/HoSVD● CANDECOMP-PARAFAC (CP)● Applications to Background Subtraction
![Page 3: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/3.jpg)
Behind the Scenes of an Intelligent Video Surveillance Framework
!
Video Content Analysis(VCA)
Behavior Analysis
Image acquisition and preprocessing
ObjectDetection
ObjectTracking
event location
Intrusion detection Collision
prevention
Target detection and tracking
Anomaly detection
Target behavior analysis
Traffic data collection and
analysis
activity report
Understanding an Intelligent Video Surveillance Framework
supervisor
!
Example of automatic incident detection
our focus
![Page 4: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/4.jpg)
Introduction to Background Subtraction
Initialize Background Model
frame modelForegroundDetection
Background Model Maintenance
![Page 5: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/5.jpg)
Background Subtraction Methods
Traditional methods:• Basic methods, mean and variance over time• Fuzzy based methods• Statistical methods • Non-parametric methods• Neural and neuro-fuzzy methods
Matrix and Tensor Factorization methods:• Eigenspace-based methods (PCA / SVD)• RPCA, LRR, NMF, MC, ST, etc.• Tensor Decomposition, NTF, etc.
BGSLibrary (C++)https://github.com/andrewssobral/bgslibrary
A large number of algorithms have been proposed for background subtraction over the last few years:
LRSLibrary (MATLAB)https://github.com/andrewssobral/lrslibrary
our focus
Andrews Sobral and Antoine Vacavant. A comprehensive review of background subtraction algorithms evaluated with synthetic and real videos. Computer Vision and Image Understanding (CVIU), 2014.
Bouwmans, Thierry; Sobral, Andrews; Javed, Sajid; Ki Jung, Soon; Zahzah, El-Hadi. "Decomposition into Low-rank plus Additive Matrices for Background/Foreground Separation: A Review for a Comparative Evaluation with a Large-Scale Dataset". Submitted to Computer Science Review, 2015.
![Page 6: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/6.jpg)
Background Subtraction Methods
Traditional methods:• Basic methods, mean and variance over time• Fuzzy based methods• Statistical methods • Non-parametric methods• Neural and neuro-fuzzy methods
Matrix and Tensor Factorization methods:• Eigenspace-based methods (PCA / SVD)• RPCA, LRR, NMF, MC, ST, etc.• Tensor Decomposition, NTF, etc.
BGSLibrary (C++)https://github.com/andrewssobral/bgslibrary
A large number of algorithms have been proposed for background subtraction over the last few years:
LRSLibrary (MATLAB)https://github.com/andrewssobral/lrslibrary
our focus
Andrews Sobral and Antoine Vacavant. A comprehensive review of background subtraction algorithms evaluated with synthetic and real videos. Computer Vision and Image Understanding (CVIU), 2014.
Bouwmans, Thierry; Sobral, Andrews; Javed, Sajid; Ki Jung, Soon; Zahzah, El-Hadi. "Decomposition into Low-rank plus Additive Matrices for Background/Foreground Separation: A Review for a Comparative Evaluation with a Large-Scale Dataset". Submitted to Computer Science Review, 2015.
Glossary of terms:
PCA Principal Component AnalysisSVD Singular Value Decomposition
LRA Low-rank ApproximationMC Matrix CompletionNMF Non-negative Matrix Factorization
RPCA Robust Principal Component AnalysisLRR Low-rank RecoveryRNMF Robust NMFST Subspace Tracking
Stable RPCA Stable version of RPCATTD Three-Term Decomposition
TD Tensor DecompositionNTF Non-negative Tensor Factorization
![Page 7: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/7.jpg)
Decomposition into Additive Matrices
● The decomposition is represented in a general formulation:
● where K usually is equal to 1, 2, or 3. For K = 3, M1 … M3 are commonly defined by:
● The characteristics of the matrices MK are as follows:
– The first matrix M1 = L is the low-rank component.
– The second matrix M2 = S is the sparse component.
– The third matrix M3 = E is generally the noise component.
● When K = 1, the matrix A ≈ L and S (implicit) can be given by S = A – L. e.g.: LRA, MC, NMF, ...
● When K = 2, A = L + S. This decomposition is called explicit. e.g.: RPCA, LRR, RNMF, …
● When K = 3, A = L + S + E. This decomposition is called stable. e.g.: Stable RPCA / Stable PCP.
Bouwmans, Thierry; Sobral, Andrews; Javed, Sajid; Ki Jung, Soon; Zahzah, El-Hadi. "Decomposition into Low-rank plus Additive Matrices for Background/Foreground Separation: A Review for a Comparative Evaluation with a Large-Scale Dataset". Submitted to Computer Science Review, 2015.
![Page 8: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/8.jpg)
Decomposition into Additive Matrices
● The decomposition is represented in a general formulation:
● where K usually is equal to 1, 2, or 3. For K = 3, M1 … M3 are commonly defined by:
● The characteristics of the matrices MK are as follows:
– The first matrix M1 = L is the low-rank component.
– The second matrix M2 = S is the sparse component.
– The third matrix M3 = E is generally the noise component.
● When K = 1, the matrix A ≈ L and S (implicit) can be given by S = A – L. e.g.: LRA, MC, NMF, ...
● When K = 2, A = L + S. This decomposition is called explicit. e.g.: RPCA, LRR, RNMF, …
● When K = 3, A = L + S + E. This decomposition is called stable. e.g.: Stable RPCA / Stable PCP.
Bouwmans, Thierry; Sobral, Andrews; Javed, Sajid; Ki Jung, Soon; Zahzah, El-Hadi. "Decomposition into Low-rank plus Additive Matrices for Background/Foreground Separation: A Review for a Comparative Evaluation with a Large-Scale Dataset". Submitted to Computer Science Review, 2015.
![Page 9: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/9.jpg)
Low Rank Approximation
▪ Low-rank approximation (LRA) is a minimization problem, in which the cost function measures the fit between a given matrix (the data) and an approximating matrix (the optimization variable), subject to a constraint that the approximating matrix has reduced rank.
![Page 10: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/10.jpg)
Low Rank Approximation
▪ Low-rank approximation (LRA) is a minimization problem, in which the cost function measures the fit between a given matrix (the data) and an approximating matrix (the optimization variable), subject to a constraint that the approximating matrix has reduced rank.
!!! Singular Value Decomposition !!!
![Page 11: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/11.jpg)
Singular Value Decomposition
▪ Formally, the singular value decomposition of an m×n real or complex matrix A is a factorization of the form:
▪ where U is a m×m real or complex unitary matrix, D is an m×n rectangular diagonal matrix with non-negative real numbers on the diagonal, and VT (the transpose of V if V is real) is an n×n real or complex unitary matrix. The diagonal entries D are known as the singular values of A.
▪ The m columns of U and the n columns of V are called the left-singular vectors and right-singular vectors of A, respectively.
generalization of eigenvalue decomposition
![Page 12: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/12.jpg)
Best rank r Approximation
![Page 13: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/13.jpg)
Background Model Estimation
![Page 14: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/14.jpg)
What about LRA for corrupted entries?
![Page 15: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/15.jpg)
Introduction to Matrix Completion (MC)
▪ Matrix Completion (MC) can be formulated as the problem of o recover a low rank matrix L from the partial observations of its entries (represented by A):
L
Underlying low-rank matrix
A
Matrix of partial observations
http://perception.csl.illinois.edu/matrix-rank/home.html
![Page 16: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/16.jpg)
Introduction to Matrix Completion (MC)
▪ Matrix Completion (MC) can be formulated as the problem of o recover a low rank matrix L from the partial observations of its entries (represented by A):
L
Underlying low-rank matrix
A
Matrix of partial observations
http://perception.csl.illinois.edu/matrix-rank/home.html
![Page 17: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/17.jpg)
Demo: Matrix Completion
http://cvxr.com/tfocs/demos/matrixcompletion/
Matrix completion via TFOCS
![Page 18: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/18.jpg)
Demo: Matrix Completion
http://cvxr.com/tfocs/demos/matrixcompletion/
![Page 19: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/19.jpg)
MC Algorithms
● LRSLibrary:
– MC: Matrix Completion (14)● FPC: Fixed point and Bregman iterative methods for matrix rank minimization (Ma et al. 2008)
● GROUSE: Grassmannian Rank-One Update Subspace Estimation (Balzano et al. 2010)
● IALM-MC: Inexact ALM for Matrix Completion (Lin et al. 2009)
● LMaFit: Low-Rank Matrix Fitting (Wen et al. 2012)
● LRGeomCG: Low-rank matrix completion by Riemannian optimization (Bart Vandereycken, 2013)
● MC_logdet: Top-N Recommender System via Matrix Completion (Kang et al. 2016)
● MC-NMF: Nonnegative Matrix Completion (Xu et al. 2011)
● OP-RPCA: Robust PCA via Outlier Pursuit (Xu et al. 2012)
● OptSpace: Matrix Completion from Noisy Entries (Keshavan et al. 2009)
● OR1MP: Orthogonal rank-one matrix pursuit for low rank matrix completion (Wang et al. 2015)
● RPCA-GD: Robust PCA via Gradient Descent (Yi et al. 2016)
● ScGrassMC: Scaled Gradients on Grassmann Manifolds for Matrix Completion (Ngo and Saad, 2012)
● SVP: Guaranteed Rank Minimization via Singular Value Projection (Meka et al. 2009)
● SVT: A singular value thresholding algorithm for matrix completion (Cai et al. 2008)
https://github.com/andrewssobral/lrslibrary
![Page 20: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/20.jpg)
MC Algorithms
● LRSLibrary:
– MC: Matrix Completion (14)● FPC: Fixed point and Bregman iterative methods for matrix rank minimization (Ma et al. 2008)
● GROUSE: Grassmannian Rank-One Update Subspace Estimation (Balzano et al. 2010)
● IALM-MC: Inexact ALM for Matrix Completion (Lin et al. 2009)
● LMaFit: Low-Rank Matrix Fitting (Wen et al. 2012)
● LRGeomCG: Low-rank matrix completion by Riemannian optimization (Bart Vandereycken, 2013)
● MC_logdet: Top-N Recommender System via Matrix Completion (Kang et al. 2016)
● MC-NMF: Nonnegative Matrix Completion (Xu et al. 2011)
● OP-RPCA: Robust PCA via Outlier Pursuit (Xu et al. 2012)
● OptSpace: Matrix Completion from Noisy Entries (Keshavan et al. 2009)
● OR1MP: Orthogonal rank-one matrix pursuit for low rank matrix completion (Wang et al. 2015)
● RPCA-GD: Robust PCA via Gradient Descent (Yi et al. 2016)
● ScGrassMC: Scaled Gradients on Grassmann Manifolds for Matrix Completion (Ngo and Saad, 2012)
● SVP: Guaranteed Rank Minimization via Singular Value Projection (Meka et al. 2009)
● SVT: A singular value thresholding algorithm for matrix completion (Cai et al. 2008)
https://github.com/andrewssobral/lrslibrary
![Page 21: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/21.jpg)
Demo: LRSLibrary for MC
https://github.com/andrewssobral/lrslibrary
https://github.com/andrewssobral/lrslibrary/blob/master/algorithms/mc/GROUSE/run_alg.m
![Page 22: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/22.jpg)
![Page 23: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/23.jpg)
Decomposition into Additive Matrices
● The decomposition is represented in a general formulation:
● where K usually is equal to 1, 2, or 3. For K = 3, M1 … M3 are commonly defined by:
● The characteristics of the matrices MK are as follows:
– The first matrix M1 = L is the low-rank component.
– The second matrix M2 = S is the sparse component.
– The third matrix M3 = E is generally the noise component.
● When K = 1, the matrix A ≈ L and S (implicit) can be given by S = A – L. e.g.: LRA, MC, NMF, ...
● When K = 2, A = L + S. This decomposition is called explicit. e.g.: RPCA, LRR, RNMF, …
● When K = 3, A = L + S + E. This decomposition is called stable. e.g.: Stable RPCA / Stable PCP.
Bouwmans, Thierry; Sobral, Andrews; Javed, Sajid; Ki Jung, Soon; Zahzah, El-Hadi. "Decomposition into Low-rank plus Additive Matrices for Background/Foreground Separation: A Review for a Comparative Evaluation with a Large-Scale Dataset". Submitted to Computer Science Review, 2015.
![Page 24: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/24.jpg)
Robust Principal Component Analysis (RPCA)
▪ RPCA can be formulated as the problem of decomposing a data matrix A into two components L and S, where A is the sum of a low-rank matrix L and a sparse matrix S:
Sparse error matrix
SL
Underlying low-rank matrix
A
Matrix of corrupted observations
![Page 25: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/25.jpg)
Robust Principal Component Analysis (RPCA)
Video Low-rank Sparse Foreground
Background model Moving objects Classification
▪ Candès et al. (2009) show that L and S can be recovered by solving a convex optimization problem, named as Principal Component Pursuit (PCP):
![Page 26: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/26.jpg)
Solving PCP
One effective way to solve PCP for the case of large matrices is to use a standard augmented Lagrangian multiplier method (ALM) (Bertsekas, 1982).
and then minimizing it iteratively by setting
where:
More information: (Qiu and Vaswani, 2011), (Pope et al. 2011), (Rodríguez and Wohlberg, 2013)
![Page 27: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/27.jpg)
RPCA solvers
For more information see: (Lin et al., 2010) http://perception.csl.illinois.edu/matrix-rank/sample_code.html
![Page 28: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/28.jpg)
What about RPCA for very dynamic background?
http://changedetection.net/http://www.svcl.ucsd.edu/projects/background_subtraction/demo.htm
![Page 29: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/29.jpg)
Decomposition into Additive Matrices
● The decomposition is represented in a general formulation:
● where K usually is equal to 1, 2, or 3. For K = 3, M1 … M3 are commonly defined by:
● The characteristics of the matrices MK are as follows:
– The first matrix M1 = L is the low-rank component.
– The second matrix M2 = S is the sparse component.
– The third matrix M3 = E is generally the noise component.
● When K = 1, the matrix A ≈ L and S (implicit) can be given by S = A – L. e.g.: LRA, MC, NMF, ...
● When K = 2, A = L + S. This decomposition is called explicit. e.g.: RPCA, LRR, RNMF, …
● When K = 3, A = L + S + E. This decomposition is called stable. e.g.: Stable RPCA / Stable PCP.
Bouwmans, Thierry; Sobral, Andrews; Javed, Sajid; Ki Jung, Soon; Zahzah, El-Hadi. "Decomposition into Low-rank plus Additive Matrices for Background/Foreground Separation: A Review for a Comparative Evaluation with a Large-Scale Dataset". Submitted to Computer Science Review, 2015.
![Page 30: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/30.jpg)
Stable PCP
▪ The PCP is limited, the low-rank component needs to be exactly low-rank and the sparse component needs to be exactly sparse, but in real applications the observations are often corrupted by noise.
▪ Zhou et al. (2010) proposed a stable version of PCP, named Stable PCP (SPCP), adding a third component that guarantee stable and accurate recovery in the presence of entry-wise noise. The observation matrix A is represented as A = L + S + E, where E is a noise term.
![Page 31: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/31.jpg)
Constrained RPCA (example 1)
▪ Some authors added an additional constraint to improve the background/foreground separation:
– Oreifej et al. (2013) use a turbulance model that quantify the scene’s motion in terms of the motion of the particles which are driven by dense optical flow.
http://www.cs.ucf.edu/~oreifej/papers/3-Way-Decomposition.pdf
![Page 32: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/32.jpg)
Constrained RPCA (example 2)
▪ Yang et al. (2015) propose a robust motion-assisted matrix restoration (RMAMR) where a dense motion field is estimated for each frame by dense optical flow, and mapped into a weighting matrix which indicates the likelihood that each pixel belongs to the background.
http://projects.medialab-tju.org/bf_separation/
![Page 33: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/33.jpg)
Double-constrained RPCA?
▪ Sobral et al. (2015) propose a double-constrained Robust Principal Component Analysis (RPCA), named SCM-RPCA (Shape and Confidence Map-based RPCA), is proposed to improve the object foreground detection in maritime scenes. It combine some ideas of Oreifej et al. (2013) and Yang et al. (2015).
– The weighting matrix proposed by Yang et al. (2015) can be used as a shape constraint (or region constraint), while the confidence map proposed by Oreifej et al. (2013) reinforces the pixels belonging from the moving objects.
▪ The original 3WD was modified adding the shape constraint as has been done in the RMAMR. We chose to modify the 3WD instead of RMAMR due its capacity to deal more robustly with the multimodality of the background.
https://sites.google.com/site/scmrpca/
![Page 34: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/34.jpg)
Solving the SCM-RPCA
Is important to note that the double constraints (confidence map and shape) can be built from two different types of source (i.e. from spatial, temporal, or spatio-temporal information), but in this work we focus only on spatial saliency maps.
![Page 35: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/35.jpg)
SCM-RPCA - Visual results on UCSD data set
From left to right: (a) input frame, (b) saliency map generated by BMS, (c) ground truth, (d) proposed approach, (e) 3WD, and (f) RMAMR.
Dataset:http://www.svcl.ucsd.edu/projects/background\_subtraction/ucsdbgsub\_dataset.htm
![Page 36: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/36.jpg)
SCM-RPCA - Visual results on MarDT data set
Is important to note that in the UCSD scenes we have used the original spatial saliency map provided by BMS, while for the MarDT scenes we have subtracted its temporal median due to the high saliency from the buildings around the river.
Dataset:http://www.dis.uniroma1.it/~labrococo/MAR/index.htm
![Page 37: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/37.jpg)
Infinity and beyond
![Page 38: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/38.jpg)
What about multidimensional data?
![Page 39: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/39.jpg)
Introduction to tensors
![Page 40: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/40.jpg)
Introduction to tensors
● Tensors are simply mathematical objects that can be used to describe physical properties. In fact tensors are merely a generalization of scalars, vectors and matrices; a scalar is a zero rank tensor, a vector is a first rank tensor and a matrix is the second rank tensor.
![Page 41: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/41.jpg)
Introduction to tensors
● Subarrays, tubes and slices of a 3rd order tensor.
![Page 42: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/42.jpg)
Introduction to tensors
● Matricization and unfolding a 3rd order tensor.
![Page 43: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/43.jpg)
Introduction to tensors
● Horizontal, vertical and frontal slices from a 3rd order tensor.
110
1928
3746 51
19
1725
3341
48
1
9
17
25
33
41
48
kj
i
110
1928
3746 51
19
1725
3341
48
1
9
17
25
33
41
48
kj
i
110
1928
3746 51
19
1725
3341
48
1
9
17
25
33
41
48
kj
i
110
1928
3746 51
19
1725
3341
48
1
9
17
25
33
41
48
kj
i
110
1928
3746 51
19
1725
3341
48
1
9
17
25
33
41
48
kj
i
Frontal Vertical Horizontal
![Page 44: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/44.jpg)
Tensor decomposition methods
● Approaches:
– Tucker / HOSVD
– CANDECOMP-PARAFAC (CP)
– Hierarchical Tucker (HT)
– Tensor-Train decomposition (TT)
– NTF (Non-negative Tensor Factorization)
– NTD (Non-negative Tucker Decomposition)
– NCP (Non-negative CP Decomposition)
![Page 45: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/45.jpg)
![Page 46: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/46.jpg)
Tucker / HoSVD
![Page 47: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/47.jpg)
CP
● The CP model is a special case of the Tucker model, where the core tensor is superdiagonal and the number of components in the factor matrices is the same.
Solving by ALS (alternating least squares) framework
![Page 48: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/48.jpg)
Background Model Estimation via Tensor Factorization
https://github.com/andrewssobral/mtt/blob/master/tensor_demo_subtensors_ntf_hals.m
![Page 49: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/49.jpg)
Background Subtraction via Tensor Decomposition
https://github.com/andrewssobral/lrslibrary
![Page 50: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/50.jpg)
Incremental Tensor LearningInterested in stream processing?
![Page 51: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/51.jpg)
Incremental Tensor Subspace Learing
![Page 52: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/52.jpg)
Incremental and Multifeature
https://github.com/andrewssobral/imtsl
![Page 53: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/53.jpg)
Incremental and Multifeature
A total of 8 features are extracted:
1) red channel,
2) green channel,
3) blue channel,
4) gray-scale,
5) local binary patterns (LBP),
6) spatial gradients in horizontal direction,
7) spatial gradients in vertical direction, and
8) spatial gradients magnitude.
values
pixels
features
tensor model
…
![Page 54: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/54.jpg)
Incremental and Multifeature
https://github.com/andrewssobral/imtsl
Feature Extraction+
iHoSVD
Initialize Background Model
frame modelForegroundDetection
Background Model Maintenance
Low Rank Tensor Model
Weighted combination of
similarity measures
x1
x3
x3
w1
w2
w3
yΣ ⁄
![Page 55: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/55.jpg)
Online Stochastic
https://github.com/andrewssobral/ostd
![Page 56: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/56.jpg)
![Page 57: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/57.jpg)
![Page 58: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/58.jpg)
LRSLibrary
The LRSLibrary provides a collection of low-rank and sparse decomposition algorithms in MATLAB. The library was designed for motion segmentation in videos, but it can be also used or adapted for other computer vision problems. Currently the LRSLibrary contains a total of 103 matrix-based and tensor-based algorithms.
https://github.com/andrewssobral/lrslibrary
![Page 59: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/59.jpg)
BGSLibrary
The BGSLibrary provides an easy-to-use C++ framework based on OpenCV to perform background subtraction (BGS) in videos. The BGSLibrary compiles under Linux, Mac OS X and Windows. Currently the library offers 37 BGS algorithms.
![Page 60: Recent advances on low-rank and sparse decomposition for moving object detection](https://reader030.vdocuments.net/reader030/viewer/2022020314/58713b421a28abf0568b6d83/html5/thumbnails/60.jpg)