[6pt] cvpr 2017 - a new tensor algebra -...
TRANSCRIPT
![Page 1: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/1.jpg)
CVPR 2017A New Tensor Algebra - Tutorial
Lior Horesh Misha [email protected] [email protected]
July 26, 2017
![Page 2: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/2.jpg)
Outline
MotivationBackground and notationNew t-product and associated algebraic frameworkImplementation considerationsThe t-SVD and optimality
Application in Facial RecognitionProper Orthogonal Decomposition
- Dynamic Model RedcutionA tensor Nuclear Norm from the t-SVD
Applications in video completion
CVPR 2017 New Tensor Algebra Lior Horesh & Misha Kilmer 2
![Page 3: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/3.jpg)
Tensor Applications:
Machine vision: understanding theworld in 3D, enable understandingphenomena such as perspective,occlusions, illumination
Latent semantic tensor indexing:common terms vs. entries vs. parts,co-occurrence of terms
Tensor subspace Analysis for Viewpoint Recognition, T. Ivanov, L. Mathies, M.A.O. Vasilescu, ICCV, 2nd IEEEInternational Workshop on Subspace Methods, September, 2009
CVPR 2017 New Tensor Algebra Lior Horesh & Misha Kilmer 3
![Page 4: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/4.jpg)
Tensor Applications:
Medical imaging: naturally involves 3D(spatio) and 4D (spatio-temporal)correlations
Video surveillance and Motionsignature: 2D images + 3rd dimensionof time, 3D/4D motion trajectory
Multi-target Tracking with Motion Context in Tenor Power Iteration X. Shi, H. Ling, W. Hu, C. Yuan, and J.Xing IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), Columbus OH, 2014
CVPR 2017 New Tensor Algebra Lior Horesh & Misha Kilmer 4
![Page 5: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/5.jpg)
Tensors: Historical Review
1927 F.L. Hitchcock: “The expression of a tensor or a polyadic as asum of products” (Journal of Mathematics and Physics)
1944 R.B. Cattell introduced a multiway model: “Parallelproportional profiles and other principles for determining the choiceof factors by rotation” (Psychometrika)
1960 L.R. Tucker: “Some mathematical notes on three-mode factoranalysis” (Psychometrika)
1981 tensor decomposition was first used in chemometrics
Past decade, computer vision, image processing, data mining, graphanalysis, etc.
F.L. Hitchcock
R.B. Cattell
L.R. TuckerCVPR 2017 New Tensor Algebra Lior Horesh & Misha Kilmer 5
![Page 6: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/6.jpg)
The Power of Proper Representation
What is that ?
Let’s observe the same data but in a different (matrix rather than vector) representation
Representation matters! some correlations can only be realized in appropriaterepresentation
CVPR 2017 New Tensor Algebra Lior Horesh & Misha Kilmer 6
![Page 7: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/7.jpg)
The Power of Proper Representation
What is that ?
Let’s observe the same data but in a different (matrix rather than vector) representation
Representation matters! some correlations can only be realized in appropriaterepresentation
CVPR 2017 New Tensor Algebra Lior Horesh & Misha Kilmer 7
![Page 8: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/8.jpg)
The Power of Proper Representation
What is that ?
Let’s observe the same data but in a different (matrix rather than vector) representation
Representation matters! some correlations can only be realized in appropriaterepresentationCVPR 2017 New Tensor Algebra Lior Horesh & Misha Kilmer 8
![Page 9: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/9.jpg)
Motivation
Much real-world data is inherently multidimensionalcolor video data – 4 way3D medical image, evolving in time (4 way); multiple patients (5 way)
Many operators and models are also multi-wayTraditional matrix-based methods based on data vectorization (e.g. matrix PCA) generallyagnostic to possible high dimensional correlations
Can we uncover hidden patterns in tensor data by computing an appropriate tensordecomposition/approximation?
Need to decide on the tensor decomposition – application dependent!
What do we mean by ‘decompose’?
CVPR 2017 New Tensor Algebra Lior Horesh & Misha Kilmer 9
![Page 10: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/10.jpg)
Tensors: Background and Notation
Notation : An1×n2...,×nj - jth order tensorExamples
0th order tensor - scalar
1st order tensor - vector
2nd order tensor - matrix
3rd order tensor ...
CVPR 2017 New Tensor Algebra Lior Horesh & Misha Kilmer 10
![Page 11: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/11.jpg)
Tensors: Background and Notation
Notation : An1×n2...,×nj - jth order tensorExamples
0th order tensor
- scalar
1st order tensor - vector
2nd order tensor - matrix
3rd order tensor ...
CVPR 2017 New Tensor Algebra Lior Horesh & Misha Kilmer 11
![Page 12: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/12.jpg)
Tensors: Background and Notation
Notation : An1×n2...,×nj - jth order tensorExamples
0th order tensor - scalar
1st order tensor - vector
2nd order tensor - matrix
3rd order tensor ...
CVPR 2017 New Tensor Algebra Lior Horesh & Misha Kilmer 12
![Page 13: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/13.jpg)
Tensors: Background and Notation
Notation : An1×n2...,×nj - jth order tensorExamples
0th order tensor - scalar
1st order tensor
- vector
2nd order tensor - matrix
3rd order tensor ...
CVPR 2017 New Tensor Algebra Lior Horesh & Misha Kilmer 13
![Page 14: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/14.jpg)
Tensors: Background and Notation
Notation : An1×n2...,×nj - jth order tensorExamples
0th order tensor - scalar
1st order tensor - vector
2nd order tensor - matrix
3rd order tensor ...
CVPR 2017 New Tensor Algebra Lior Horesh & Misha Kilmer 14
![Page 15: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/15.jpg)
Tensors: Background and Notation
Notation : An1×n2...,×nj - jth order tensorExamples
0th order tensor - scalar
1st order tensor - vector
2nd order tensor
- matrix
3rd order tensor ...
CVPR 2017 New Tensor Algebra Lior Horesh & Misha Kilmer 15
![Page 16: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/16.jpg)
Tensors: Background and Notation
Notation : An1×n2...,×nj - jth order tensorExamples
0th order tensor - scalar
1st order tensor - vector
2nd order tensor - matrix
3rd order tensor ...
CVPR 2017 New Tensor Algebra Lior Horesh & Misha Kilmer 16
![Page 17: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/17.jpg)
Tensors: Background and Notation
Notation : An1×n2...,×nj - jth order tensorExamples
0th order tensor - scalar
1st order tensor - vector
2nd order tensor - matrix
3rd order tensor ...
CVPR 2017 New Tensor Algebra Lior Horesh & Misha Kilmer 17
![Page 18: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/18.jpg)
Tensors: Background and Notation
Notation : An1×n2...,×nj - jth order tensorExamples
0th order tensor - scalar
1st order tensor - vector
2nd order tensor - matrix
3rd order tensor ...
CVPR 2017 New Tensor Algebra Lior Horesh & Misha Kilmer 18
![Page 19: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/19.jpg)
Notation
Ai,j,k = element of A in row i, column j, tube k
← A4,7,1
← A:,3,1
← A:,:,3
CVPR 2017 New Tensor Algebra Lior Horesh & Misha Kilmer 19
![Page 20: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/20.jpg)
Notation
Ai,j,k = element of A in row i, column j, tube k
← A4,7,1
← A:,3,1
← A:,:,3
CVPR 2017 New Tensor Algebra Lior Horesh & Misha Kilmer 20
![Page 21: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/21.jpg)
Notation
Ai,j,k = element of A in row i, column j, tube k
← A4,7,1
← A:,3,1
← A:,:,3
CVPR 2017 New Tensor Algebra Lior Horesh & Misha Kilmer 21
![Page 22: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/22.jpg)
Notation
Ai,j,k = element of A in row i, column j, tube k
← A4,7,1
← A:,3,1
← A:,:,3
CVPR 2017 New Tensor Algebra Lior Horesh & Misha Kilmer 22
![Page 23: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/23.jpg)
Tensors: Background and Notation
Fiber - a vector defined by fixing all but one index while varying the rest
Slice - a matrix defined by fixing all but two indices while varying the rest
CVPR 2017 New Tensor Algebra Lior Horesh & Misha Kilmer 23
![Page 24: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/24.jpg)
Tensor Multiplication
Definition : The k - mode multiplication of a tensor X ∈ Rn1×n2×...,×nd with a matrixU ∈ RJ×nk is denoted by X×kU and is of size n1 × · · · × nk−1 × J × nk+1 × · · · × nd
Element-wise
(X×kU)i1···ik−1jik+1···id=
nd∑ik=1
xi1i2···idujik
1-mode multiplication
CVPR 2017 New Tensor Algebra Lior Horesh & Misha Kilmer 24
![Page 25: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/25.jpg)
Tensor Holy Grail and the Matrix Analogy
Find a way to express a tensor that leads to the possibility for compressed representation(near redundancy removed) that maintains important features of the original tensor
min ‖A−B‖F s.t. B has rank p ≤ r
B =∑p
i=1 σi(V u(i) V v
(i)) where A =∑r
i=1 σi(u(i) v(i))
CVPR 2017 New Tensor Algebra Lior Horesh & Misha Kilmer 25
![Page 26: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/26.jpg)
Tensor Decompositions - CPCP (CANDECOMP-PARAFAC) Decomposition 1 :
X ≈r∑
i=1ai bi ci
Outer product T = u v w ⇒ Tijk = uivjwk
Columns of A = [a1, . . . , ar], B = [b1, . . . , br], C = [c1, . . . , cr] are not orthogonalIf r is minimal, then r is called the rank of the tensorNo perfect procedure for fitting CP for a given number of components 2
1R. Harshman, 1970; J. Carroll and J. Chang, 19702V. de Silva, L. Lim, Tensor Rank and the Ill-Posedness of the Best Low-Rank Approximation Problem, 2008
CVPR 2017 New Tensor Algebra Lior Horesh & Misha Kilmer 26
![Page 27: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/27.jpg)
Tensor Decompositions - Tucker
Tucker Decomposition :
X ≈ C ×1 G×2 T ×3 S =r1∑
i=1
r2∑j=1
r3∑k=1
cijkgi tj sk
C is the core tensorG, T , S are the components of factorsCan either have diagonal core or orthogonal columns in components [DeLathauwer et al.]Truncated Tucker decomposition is not optimal in approximating the norm of the difference
‖X − C ×1 G×2 T ×3 S‖CVPR 2017 New Tensor Algebra Lior Horesh & Misha Kilmer 27
![Page 28: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/28.jpg)
Tensor Decompositions - t-product
t-product : Let A be n1 × n2 × n3 and B be n2 × `× n3. Then the t-product A ∗ B is then1 × `× n3 tensor
A ∗ B = fold(circ(A) · vec(B))
circ (A) · vec (B) =
A1 An3 An3−1 · · · A2A2 A1 An3 · · · A3
.... . . . . . . . .
...An3−1 An3−2 An3−3 · · · An3
An3 An3−1 An3−2 · · · A1
B1B2B3...Bn3
fold(vec(B)) = B
Ai, Bi, i = 1, . . . , n3 are frontal slices of A and B
M.E. Kilmer and C.D. Martin. Factorization strategies for third-order tensors, Linear Algebra and itsApplications, Special Issue in Honor of G. W. Stewart’s 70th birthday, vol. 435(3):641–658, 2011
CVPR 2017 New Tensor Algebra Lior Horesh & Misha Kilmer 28
![Page 29: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/29.jpg)
Block Circulants
A block circulant can be block-diagonalized by a (normalized) DFT in the 2nd dimension:
(F⊗ I)circ (A) (F∗ ⊗ I) =
A1 0 · · · 00 A2 0 · · ·
0 · · ·. . . 0
0 · · · 0 An
Here ⊗ is a Kronecker product of matricesIf F is n× n, and I is m×m, (F⊗ I) is the mn×mn block matrix, of n block rows andcolumns, each block is m×m, where the ijth block is fi,jIBut we never implement it this way because an FFT along tube fibers of A yields a tensor,A whose frontal slices are the Ai
CVPR 2017 New Tensor Algebra Lior Horesh & Misha Kilmer 29
![Page 30: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/30.jpg)
t-product Identity
Definition: The n× n× ` identity tensor Inn` is the tensor whose frontal face is the n× nidentity matrix, and whose other faces are all zeros
Class Exercise: Let A be n1 × n× n3, show that
A ∗ I = A and I ∗ A = A
A ∗ I = fold (circ (A) · vec (I)) =
A1 An3 An3−1 · · · A2A2 A1 An3 · · · A3
.... . . . . . . . .
...An3−1 An3−2 An3−3 · · · An3
An3 An3−1 An3−2 · · · A1
I00...0
= A
CVPR 2017 New Tensor Algebra Lior Horesh & Misha Kilmer 30
![Page 31: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/31.jpg)
t-product Identity
Definition: The n× n× ` identity tensor Inn` is the tensor whose frontal face is the n× nidentity matrix, and whose other faces are all zeros
Class Exercise: Let A be n1 × n× n3, show that
A ∗ I = A and I ∗ A = A
A ∗ I = fold (circ (A) · vec (I)) =
A1 An3 An3−1 · · · A2A2 A1 An3 · · · A3
.... . . . . . . . .
...An3−1 An3−2 An3−3 · · · An3
An3 An3−1 An3−2 · · · A1
I00...0
= A
CVPR 2017 New Tensor Algebra Lior Horesh & Misha Kilmer 31
![Page 32: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/32.jpg)
t-product Identity
Definition: The n× n× ` identity tensor Inn` is the tensor whose frontal face is the n× nidentity matrix, and whose other faces are all zeros
Class Exercise: Let A be n1 × n× n3, show that
A ∗ I = A and I ∗ A = A
A ∗ I
= fold (circ (A) · vec (I)) =
A1 An3 An3−1 · · · A2A2 A1 An3 · · · A3
.... . . . . . . . .
...An3−1 An3−2 An3−3 · · · An3
An3 An3−1 An3−2 · · · A1
I00...0
= A
CVPR 2017 New Tensor Algebra Lior Horesh & Misha Kilmer 32
![Page 33: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/33.jpg)
t-product Identity
Definition: The n× n× ` identity tensor Inn` is the tensor whose frontal face is the n× nidentity matrix, and whose other faces are all zeros
Class Exercise: Let A be n1 × n× n3, show that
A ∗ I = A and I ∗ A = A
A ∗ I = fold (circ (A) · vec (I))
=
A1 An3 An3−1 · · · A2A2 A1 An3 · · · A3
.... . . . . . . . .
...An3−1 An3−2 An3−3 · · · An3
An3 An3−1 An3−2 · · · A1
I00...0
= A
CVPR 2017 New Tensor Algebra Lior Horesh & Misha Kilmer 33
![Page 34: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/34.jpg)
t-product Identity
Definition: The n× n× ` identity tensor Inn` is the tensor whose frontal face is the n× nidentity matrix, and whose other faces are all zeros
Class Exercise: Let A be n1 × n× n3, show that
A ∗ I = A and I ∗ A = A
A ∗ I = fold (circ (A) · vec (I)) =
A1 An3 An3−1 · · · A2A2 A1 An3 · · · A3
.... . . . . . . . .
...An3−1 An3−2 An3−3 · · · An3
An3 An3−1 An3−2 · · · A1
I00...0
= A
CVPR 2017 New Tensor Algebra Lior Horesh & Misha Kilmer 34
![Page 35: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/35.jpg)
t-product Identity
Definition: The n× n× ` identity tensor Inn` is the tensor whose frontal face is the n× nidentity matrix, and whose other faces are all zeros
Class Exercise: Let A be n1 × n× n3, show that
A ∗ I = A and I ∗ A = A
A ∗ I = fold (circ (A) · vec (I)) =
A1 An3 An3−1 · · · A2A2 A1 An3 · · · A3
.... . . . . . . . .
...An3−1 An3−2 An3−3 · · · An3
An3 An3−1 An3−2 · · · A1
I00...0
= A
CVPR 2017 New Tensor Algebra Lior Horesh & Misha Kilmer 35
![Page 36: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/36.jpg)
t-product Transpose
Definition: If A is n1×n2×n3, then A> is the n2×n1×n3 tensor obtained by transposingeach of the frontal faces and then reversing the order of transposed faces 2 through n3
Example: If A ∈ Rn1×n2×4 and its frontal faces are given by the n1 × n2 matricesA1,A2,A3,A4, then
A> = fold
A>1A>4A>3A>2
Mimetic property: when n = 1, the ∗ operator collapses to traditional matrix multiplicationbetween two matrices and tranpose becomes matrix transposition
CVPR 2017 New Tensor Algebra Lior Horesh & Misha Kilmer 36
![Page 37: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/37.jpg)
t-product Orthogonality
Definition: An n× n× l real-valued tensor Q is orthogonal if
Q> ∗Q = Q ∗Q> = I
Note that this means that
Q(:, i, :)> ∗ Q(:, j, :) =e1 i = j0 i 6= j
CVPR 2017 New Tensor Algebra Lior Horesh & Misha Kilmer 37
![Page 38: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/38.jpg)
t-SVD and Trunction Optimality
Theorem: Let the T -SVD of A ∈ R`×m×n be given by A = U ∗ S ∗ V>, with `× `× northogonal tensor U , m×m× n orthogonal tensor V, and `×m× n f-diagonal tensor S
For k < min(l, m), define
Ak = U(:, 1 : k, :) ∗ S(1 : k, 1 : k, :) ∗ V>(:, 1 : k, :) =k∑
i=1
U(:, i, :) ∗ S(i, i, :) ∗ V(:, i, :)>
ThenAk = arg min
A∈M
‖A − A‖
where M = C = X ∗ Y | X ∈ R`×k×n,Y ∈ Rk×m×n
CVPR 2017 New Tensor Algebra Lior Horesh & Misha Kilmer 38
![Page 39: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/39.jpg)
t-SVD and Optimality in Truncation
Let A ∈ Rm×p×n, for k < min(m, p), define
Ak =k∑
i=1U(:, i, :) ∗ S(i, i, :) ∗V(:, i, :)>
ThenAk = arg min
A∈M
‖A− A‖
where M = C = X ∗ Y |X ∈ Rm×k×n,Y ∈ Rk×p×n
CVPR 2017 New Tensor Algebra Lior Horesh & Misha Kilmer 39
![Page 40: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/40.jpg)
t-SVD example
Let A be 2× 2× 2
(F⊗ I)circ (A) (F∗ ⊗ I) =[A1 00 A2
]∈ C4×4
[A1 00 A2
]=[U1 00 U2
][σ
(1)1 00 σ
(1)2
][σ
(2)1 00 σ
(2)2
][V∗1 0
0 V∗2
]
The U,S,VT are formed by putting the hat matrices as frontal slices, then ifft along tubes
e.g. S(1,1,:) obtained from ifft of vector[σ
(1)1σ
(2)1
]oriented into screen
CVPR 2017 New Tensor Algebra Lior Horesh & Misha Kilmer 40
![Page 41: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/41.jpg)
T -SVD and Multiway PCA
Xj , j = 1, 2, . . . ,m are the training imagesM is the mean imageA(:, j, :) = Xj −M stores the mean-subtracted imagesK = A ∗ A> = U ∗ S ∗ S> ∗ U> is the covariance tensorLeft orthogonal U contains the principal components with respect to K
A(:, j, :) ≈ U(:, 1 : k, :) ∗ U(:, 1 : k, :)> ∗ A(:, j, :) =k∑
t=1U(:, t, :) ∗ C(t, j, :)
CVPR 2017 New Tensor Algebra Lior Horesh & Misha Kilmer 41
![Page 42: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/42.jpg)
T -QR Decomposition
Theorem: Let A be an `×m× n real-valued tensor, then A can be factored as
A ∗ P = Q ∗R
where Q is orthogonal `× `× n, R is `×m× n f-upper triangular, and P is a permutationtensor
Cheaper for updating and downdating
CVPR 2017 New Tensor Algebra Lior Horesh & Misha Kilmer 42
![Page 43: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/43.jpg)
Face Recognition Task
Multilinear (Tensor) ICA and Dimensionality Reduction”, M.A.O. Vasilescu, D. Terzopoulos, Proc. 7thInternational Conference on Independent Component Analysis and Signal Separation (ICA07), London, UK,September, 2007. In Lecture Notes in Computer Science, 4666, Springer-Verlag, New York, 2007, 818-826
CVPR 2017 New Tensor Algebra Lior Horesh & Misha Kilmer 43
![Page 44: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/44.jpg)
Face Recognition Task
Experiment 1: randomly selected 15 images of each person as training set and test allremaining imagesExperiment 2: randomly selected 5 images of each person as the training set and test allremaining imagesPreprocessing: decimated the images by a factor of 3 to 64× 56 pixels20 trials for each experiment
The Extended Yale Face Database B, http://vision.ucsd.edu/˜leekc/ExtYaleDatabase/ExtYaleB.html
CVPR 2017 New Tensor Algebra Lior Horesh & Misha Kilmer 44
![Page 45: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/45.jpg)
T -SVD vs. PCA
N. Hao, M.E. Kilmer, K. Braman, R.C. Hoover, Facial Recognition Using Tensor-Tensor Decompositions,SIAM J. Imaging Sci., 6(1), 437-463
CVPR 2017 New Tensor Algebra Lior Horesh & Misha Kilmer 45
![Page 46: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/46.jpg)
T -SVD vs. PCA
N. Hao, M.E. Kilmer, K. Braman, R.C. Hoover, Facial Recognition Using Tensor-Tensor Decompositions,SIAM J. Imaging Sci., 6(1), 437-463
CVPR 2017 New Tensor Algebra Lior Horesh & Misha Kilmer 46
![Page 47: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/47.jpg)
T -QR vs. PCA
N. Hao, M.E. Kilmer, K. Braman, R.C. Hoover, Facial Recognition Using Tensor-Tensor Decompositions,SIAM J. Imaging Sci., 6(1), 437-463
CVPR 2017 New Tensor Algebra Lior Horesh & Misha Kilmer 47
![Page 48: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/48.jpg)
T -QR vs. PCA
N. Hao, M.E. Kilmer, K. Braman, R.C. Hoover, Facial Recognition Using Tensor-Tensor Decompositions,SIAM J. Imaging Sci., 6(1), 437-463
CVPR 2017 New Tensor Algebra Lior Horesh & Misha Kilmer 48
![Page 49: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/49.jpg)
Non-Negative Tensor Decompositions - t-product
Given a nonnegative third-order tensor T ∈ R`×m×n and a positive integer k < min(l,m, n)Find nonnegative G ∈ R`×k×n, H ∈ Rk×m×n such that
minG,H‖T − G ∗ H‖2
F
Facial Recognition Example:Dataset: The Center for Biological and Computational Learning (CBCL) DatabaseTraining images: 200k = 10
CVPR 2017 New Tensor Algebra Lior Horesh & Misha Kilmer 49
![Page 50: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/50.jpg)
Non-Negative Tensor Decompositions - t-product
Given a nonnegative third-order tensor T ∈ R`×m×n and a positive integer k < min(l,m, n)Find nonnegative G ∈ R`×k×n, H ∈ Rk×m×n such that
minG,H‖T − G ∗ H‖2
F
Facial Recognition Example:Dataset: The Center for Biological and Computational Learning (CBCL) DatabaseTraining images: 200k = 10
CVPR 2017 New Tensor Algebra Lior Horesh & Misha Kilmer 50
![Page 51: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/51.jpg)
Reconstructed Images Based on NMF, NTF-CP and NTF-GH
N. Hao, L. Horesh, M. Kilmer, Non-negative Tensor Decomposition, Compressed Sensing & Sparse Filtering,Springer, 123–148, 2014
CVPR 2017 New Tensor Algebra Lior Horesh & Misha Kilmer 51
![Page 52: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/52.jpg)
Tensor Nuclear Norm
If A is an `×m, ` ≥ m matrix with singular values σi, the nuclear norm ‖A‖~ =∑m
i=1 σi.
However, in the t-SVD, we have singular tubes (the entries of which need not be positive),which sum up to a singular tube!
The entries in the jth singular tube are the inverse Fourier coefficients of the length-n vectorof the jth singular values of A:,:,i, i = 1..n.
DefinitionFor A ∈ R`×m×n, our tensor nuclear norm is‖A‖~ =
∑min(`,m)i=1 ‖
√nFV si‖1 =
∑min(`,m)i=1
∑nj=1 Si,i,j . (Same as the matrix nuclear norm
of circ (A)).
CVPR 2017 New Tensor Algebra Lior Horesh & Misha Kilmer 52
![Page 53: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/53.jpg)
Tensor Nuclear Norm
Theorem (Semerci,Hao,Kilmer,Miller)The tensor nuclear norm is a valid norm.
Since the t-SVD extends to higher-order tensors [Martin et al, 2012], the norm does, as well.
CVPR 2017 New Tensor Algebra Lior Horesh & Misha Kilmer 53
![Page 54: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/54.jpg)
Tensor Completion
Given unknown tensor TM of size n1 × n2 × n3, given a subset of entriesTMijk : (i, j, k) ∈ Ω where Ω is an indicator tensor of size n1 × n2 × n3. Recover theentire TM :
min ‖TX‖~subject to PΩ(TX) = PΩ(TM)
The (i, j, k)th component of PΩ(TX) is equal to TMijk if (i, j, k) ∈ Ω and zero otherwise.
Similar to the previous problem, this can be solved by ADMM, with 3 update steps, onewhich decouples, one that is a shrinkage / thresholding step.
CVPR 2017 New Tensor Algebra Lior Horesh & Misha Kilmer 54
![Page 55: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/55.jpg)
Numerical Results
TNN minimization, Low Rank Tensor Completion (LRTC) [Liu, et al, 2013] based ontensor-n-rank [Gandy, et al, 2011], and the nuclear norm minimization on the vectorizedvideo data [Cai, et al, 2010].MERL3 video, Basketball video
3with thanks to A. AgrawalCVPR 2017 New Tensor Algebra Lior Horesh & Misha Kilmer 55
![Page 56: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/56.jpg)
Numerical Results
CVPR 2017 New Tensor Algebra Lior Horesh & Misha Kilmer 56
![Page 57: [6pt] CVPR 2017 - A New Tensor Algebra - Tutorialusers.cecs.anu.edu.au/.../present/horesh_lior_tmcv2017.pdfCVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh Misha Kilmer lhoresh@us.ibm.com](https://reader035.vdocuments.net/reader035/viewer/2022062605/5fcdb33ba21a1474b32c18ae/html5/thumbnails/57.jpg)
Toolbox and References
Brett W. Bader, Tamara G. Kolda and others. MATLAB Tensor Toolbox Version 2.5,Available online, January 2012, http://www.sandia.gov/˜tgkolda/TensorToolbox/
T. G. Kolda and B. W. Bader, Tensor Decompositions and Applications, SIAM Review51(3):455-500, 2009
A. Cichocki, R. Zdunek, A.H. Phan, S.i. Amari, Nonnegative Matrix and TensorFactorizations: Applications to Exploratory Multi-way Data Analysis and Blind SourceSeparation, 2009
M.E. Kilmer and C.D. Martin. Factorization strategies for third-order tensors, Linear Algebraand its Applications, Special Issue in Honor of G. W. Stewart’s 70th birthday, vol.435(3):641–658, 2011
N. Hao, L. Horesh, M. Kilmer, Non-negative Tensor Decomposition, Compressed Sensing &Sparse Filtering, Springer, 123–148, 2014
CVPR 2017 New Tensor Algebra Lior Horesh & Misha Kilmer 57