maa704: matrix factorization and canonical forms
TRANSCRIPT
![Page 1: MAA704: Matrix factorization and canonical forms](https://reader030.vdocuments.net/reader030/viewer/2022020703/61fb3c8e2e268c58cd5bc7e7/html5/thumbnails/1.jpg)
MAA704:Matrix
factorizationand canonical
forms
Matrixfactorization
Matrixproperties
Triangularmatrix
Hessenbergmatrix
Hermitian matrix
Unitary matrices
Positive definitematrix
Matrixfactorization
Spectralfactorization
Rankfactorization
LU factorization
Choleskyfactorization
QR factorization
Canonicalforms
Reduced rowechelon form
Jordan normalform
Singular valuefactorization
Similar matrices
Summary
MAA704: Matrix factorization and canonicalforms
Karl Lundengard
November 23, 2012
![Page 2: MAA704: Matrix factorization and canonical forms](https://reader030.vdocuments.net/reader030/viewer/2022020703/61fb3c8e2e268c58cd5bc7e7/html5/thumbnails/2.jpg)
MAA704:Matrix
factorizationand canonical
forms
Matrixfactorization
Matrixproperties
Triangularmatrix
Hessenbergmatrix
Hermitian matrix
Unitary matrices
Positive definitematrix
Matrixfactorization
Spectralfactorization
Rankfactorization
LU factorization
Choleskyfactorization
QR factorization
Canonicalforms
Reduced rowechelon form
Jordan normalform
Singular valuefactorization
Similar matrices
Summary
Contents of todays lecture
I Some interesting / useful / important properties ofmatrices
I Matrix factorization
I Canonical forms
![Page 3: MAA704: Matrix factorization and canonical forms](https://reader030.vdocuments.net/reader030/viewer/2022020703/61fb3c8e2e268c58cd5bc7e7/html5/thumbnails/3.jpg)
MAA704:Matrix
factorizationand canonical
forms
Matrixfactorization
Matrixproperties
Triangularmatrix
Hessenbergmatrix
Hermitian matrix
Unitary matrices
Positive definitematrix
Matrixfactorization
Spectralfactorization
Rankfactorization
LU factorization
Choleskyfactorization
QR factorization
Canonicalforms
Reduced rowechelon form
Jordan normalform
Singular valuefactorization
Similar matrices
Summary
Matrix factorization
I Rewriting a matrix as a product of several matrices.
I Choosing these factor matrices wisely can make problemseasier to solve.
I Also known as matrix decomposition
![Page 4: MAA704: Matrix factorization and canonical forms](https://reader030.vdocuments.net/reader030/viewer/2022020703/61fb3c8e2e268c58cd5bc7e7/html5/thumbnails/4.jpg)
MAA704:Matrix
factorizationand canonical
forms
Matrixfactorization
Matrixproperties
Triangularmatrix
Hessenbergmatrix
Hermitian matrix
Unitary matrices
Positive definitematrix
Matrixfactorization
Spectralfactorization
Rankfactorization
LU factorization
Choleskyfactorization
QR factorization
Canonicalforms
Reduced rowechelon form
Jordan normalform
Singular valuefactorization
Similar matrices
Summary
Diagonalizable matrix
DefinitionIf B = S−1DS where D is a diagonal matrix then B is diagonal.
Motivation.Using elementary row operations we want to turn Bx = y intoDx = y . This can be written as SBx = Sy . Since elementaryrow operations are invertible SBS−1Sx = Sy . Let x = Sx andy = Sy , then
D = SBS−1 ⇔ B = S−1DS
![Page 5: MAA704: Matrix factorization and canonical forms](https://reader030.vdocuments.net/reader030/viewer/2022020703/61fb3c8e2e268c58cd5bc7e7/html5/thumbnails/5.jpg)
MAA704:Matrix
factorizationand canonical
forms
Matrixfactorization
Matrixproperties
Triangularmatrix
Hessenbergmatrix
Hermitian matrix
Unitary matrices
Positive definitematrix
Matrixfactorization
Spectralfactorization
Rankfactorization
LU factorization
Choleskyfactorization
QR factorization
Canonicalforms
Reduced rowechelon form
Jordan normalform
Singular valuefactorization
Similar matrices
Summary
Triangular matrix
A =
F F . . . F0 F . . . F...
.... . .
...0 0 . . . F
![Page 6: MAA704: Matrix factorization and canonical forms](https://reader030.vdocuments.net/reader030/viewer/2022020703/61fb3c8e2e268c58cd5bc7e7/html5/thumbnails/6.jpg)
MAA704:Matrix
factorizationand canonical
forms
Matrixfactorization
Matrixproperties
Triangularmatrix
Hessenbergmatrix
Hermitian matrix
Unitary matrices
Positive definitematrix
Matrixfactorization
Spectralfactorization
Rankfactorization
LU factorization
Choleskyfactorization
QR factorization
Canonicalforms
Reduced rowechelon form
Jordan normalform
Singular valuefactorization
Similar matrices
Summary
Triangular matrix
I Can be lower (left) or upper (right) triangular
I Easy to solve equation systems involving triangularmatrices
I Diagonal values are also eigenvalues
![Page 7: MAA704: Matrix factorization and canonical forms](https://reader030.vdocuments.net/reader030/viewer/2022020703/61fb3c8e2e268c58cd5bc7e7/html5/thumbnails/7.jpg)
MAA704:Matrix
factorizationand canonical
forms
Matrixfactorization
Matrixproperties
Triangularmatrix
Hessenbergmatrix
Hermitian matrix
Unitary matrices
Positive definitematrix
Matrixfactorization
Spectralfactorization
Rankfactorization
LU factorization
Choleskyfactorization
QR factorization
Canonicalforms
Reduced rowechelon form
Jordan normalform
Singular valuefactorization
Similar matrices
Summary
Hessenberg matrix
A =
F F F · · · F F FF F F · · · F F F0 F F · · · F F F0 0 F · · · F F F...
......
. . ....
......
0 0 0 · · · F F F0 0 0 · · · 0 F F
![Page 8: MAA704: Matrix factorization and canonical forms](https://reader030.vdocuments.net/reader030/viewer/2022020703/61fb3c8e2e268c58cd5bc7e7/html5/thumbnails/8.jpg)
MAA704:Matrix
factorizationand canonical
forms
Matrixfactorization
Matrixproperties
Triangularmatrix
Hessenbergmatrix
Hermitian matrix
Unitary matrices
Positive definitematrix
Matrixfactorization
Spectralfactorization
Rankfactorization
LU factorization
Choleskyfactorization
QR factorization
Canonicalforms
Reduced rowechelon form
Jordan normalform
Singular valuefactorization
Similar matrices
Summary
Hessenberg matrix
I ’Almost’ triangular
I Multiplication of two Hessenberg matrices gives a newHessenberg matrix
I Useful in the QR-method (Lecture 9)
![Page 9: MAA704: Matrix factorization and canonical forms](https://reader030.vdocuments.net/reader030/viewer/2022020703/61fb3c8e2e268c58cd5bc7e7/html5/thumbnails/9.jpg)
MAA704:Matrix
factorizationand canonical
forms
Matrixfactorization
Matrixproperties
Triangularmatrix
Hessenbergmatrix
Hermitian matrix
Unitary matrices
Positive definitematrix
Matrixfactorization
Spectralfactorization
Rankfactorization
LU factorization
Choleskyfactorization
QR factorization
Canonicalforms
Reduced rowechelon form
Jordan normalform
Singular valuefactorization
Similar matrices
Summary
Hermitian matrix
DefinitionThe Hermitian conjugate of a matrix A is denoted AH and isdefined by (AH)ij = (A)ji .
DefinitionA matrix is said to be Hermitian (or self-adjoint) if AH = A
![Page 10: MAA704: Matrix factorization and canonical forms](https://reader030.vdocuments.net/reader030/viewer/2022020703/61fb3c8e2e268c58cd5bc7e7/html5/thumbnails/10.jpg)
MAA704:Matrix
factorizationand canonical
forms
Matrixfactorization
Matrixproperties
Triangularmatrix
Hessenbergmatrix
Hermitian matrix
Unitary matrices
Positive definitematrix
Matrixfactorization
Spectralfactorization
Rankfactorization
LU factorization
Choleskyfactorization
QR factorization
Canonicalforms
Reduced rowechelon form
Jordan normalform
Singular valuefactorization
Similar matrices
Summary
Hermitian matrix
I Notice the similarities with a symmetric matrix A> = A.
I All eigenvalues real.
I Always diagonalizable.
I Important in theoretical physics, quantum physics,electroengineering and in certain problems in statistics.
![Page 11: MAA704: Matrix factorization and canonical forms](https://reader030.vdocuments.net/reader030/viewer/2022020703/61fb3c8e2e268c58cd5bc7e7/html5/thumbnails/11.jpg)
MAA704:Matrix
factorizationand canonical
forms
Matrixfactorization
Matrixproperties
Triangularmatrix
Hessenbergmatrix
Hermitian matrix
Unitary matrices
Positive definitematrix
Matrixfactorization
Spectralfactorization
Rankfactorization
LU factorization
Choleskyfactorization
QR factorization
Canonicalforms
Reduced rowechelon form
Jordan normalform
Singular valuefactorization
Similar matrices
Summary
Unitary matrices
DefinitionA matrix, A, is said to be unitary if AH = A−1.
![Page 12: MAA704: Matrix factorization and canonical forms](https://reader030.vdocuments.net/reader030/viewer/2022020703/61fb3c8e2e268c58cd5bc7e7/html5/thumbnails/12.jpg)
MAA704:Matrix
factorizationand canonical
forms
Matrixfactorization
Matrixproperties
Triangularmatrix
Hessenbergmatrix
Hermitian matrix
Unitary matrices
Positive definitematrix
Matrixfactorization
Spectralfactorization
Rankfactorization
LU factorization
Choleskyfactorization
QR factorization
Canonicalforms
Reduced rowechelon form
Jordan normalform
Singular valuefactorization
Similar matrices
Summary
Properties of unitary matrices
TheoremLet U be a unitary matrix, then
a) U is always invertible.
b) U−1 is also unitary.
c) | det(U)| = 1
d) (UV )H = (UV )−1 if V is also unitary.
e) For any λ that is an eigenvalue of U, λ = e iω, 0 ≤ ω ≤ 2π.
f) Let v be a vector, then |Uv | = |v | (for any vector norm1).
g) The rows/columns of U are orthonormal, that is Ui .UHj . = 0,
i 6= j , Uk.UHk. = 1.
h) U preserves eigenvalues.
1see lecture 7
![Page 13: MAA704: Matrix factorization and canonical forms](https://reader030.vdocuments.net/reader030/viewer/2022020703/61fb3c8e2e268c58cd5bc7e7/html5/thumbnails/13.jpg)
MAA704:Matrix
factorizationand canonical
forms
Matrixfactorization
Matrixproperties
Triangularmatrix
Hessenbergmatrix
Hermitian matrix
Unitary matrices
Positive definitematrix
Matrixfactorization
Spectralfactorization
Rankfactorization
LU factorization
Choleskyfactorization
QR factorization
Canonicalforms
Reduced rowechelon form
Jordan normalform
Singular valuefactorization
Similar matrices
Summary
Example of a unitary matrix
I The C matrix below rotates a vector by the angle θaround the x-axis
C =
1 0 00 cos(θ) − sin(θ)0 sin(θ) cos(θ)
and is a unitary matrix.
![Page 14: MAA704: Matrix factorization and canonical forms](https://reader030.vdocuments.net/reader030/viewer/2022020703/61fb3c8e2e268c58cd5bc7e7/html5/thumbnails/14.jpg)
MAA704:Matrix
factorizationand canonical
forms
Matrixfactorization
Matrixproperties
Triangularmatrix
Hessenbergmatrix
Hermitian matrix
Unitary matrices
Positive definitematrix
Matrixfactorization
Spectralfactorization
Rankfactorization
LU factorization
Choleskyfactorization
QR factorization
Canonicalforms
Reduced rowechelon form
Jordan normalform
Singular valuefactorization
Similar matrices
Summary
Positive definite matrix
DefinitionWe consider a square symmetric real valued n × n matrix A,then:
I A is positive definite if xTAx is positive for all non-zerovectors x .
I A is positive semidefinite if xTAx is non-negative for allnon-zero vectors x .
I A is positive definite ⇔ λ > 0 for all λ eigenvalue of A.
I Can also define negative definite and semi-definitematrices, more about this in lecture 10.
![Page 15: MAA704: Matrix factorization and canonical forms](https://reader030.vdocuments.net/reader030/viewer/2022020703/61fb3c8e2e268c58cd5bc7e7/html5/thumbnails/15.jpg)
MAA704:Matrix
factorizationand canonical
forms
Matrixfactorization
Matrixproperties
Triangularmatrix
Hessenbergmatrix
Hermitian matrix
Unitary matrices
Positive definitematrix
Matrixfactorization
Spectralfactorization
Rankfactorization
LU factorization
Choleskyfactorization
QR factorization
Canonicalforms
Reduced rowechelon form
Jordan normalform
Singular valuefactorization
Similar matrices
Summary
Matrix factorization
I Diagonalizable A = S−1DS with D diagonalI Other important factorizations:
I Spectral factorization QΛQ−1
I LU-factorizationI Cholesky factorization GGH
I QR-factorizationI Rank factorization CFI Jordan canonical form S−1JSI Singular value factorization UΣV H
![Page 16: MAA704: Matrix factorization and canonical forms](https://reader030.vdocuments.net/reader030/viewer/2022020703/61fb3c8e2e268c58cd5bc7e7/html5/thumbnails/16.jpg)
MAA704:Matrix
factorizationand canonical
forms
Matrixfactorization
Matrixproperties
Triangularmatrix
Hessenbergmatrix
Hermitian matrix
Unitary matrices
Positive definitematrix
Matrixfactorization
Spectralfactorization
Rankfactorization
LU factorization
Choleskyfactorization
QR factorization
Canonicalforms
Reduced rowechelon form
Jordan normalform
Singular valuefactorization
Similar matrices
Summary
Spectral factorization
I Spectral factorization is a special version of diagonalfactorization.
I It is sometimes referred to as eigendecomposition.
I Let A be an square (n × n) matrix with linearlyindependent rows. Then
A = QΛQ−1
where AQ.i = ΛiiQ.i for all 1 ≤ i ≤ n.
![Page 17: MAA704: Matrix factorization and canonical forms](https://reader030.vdocuments.net/reader030/viewer/2022020703/61fb3c8e2e268c58cd5bc7e7/html5/thumbnails/17.jpg)
MAA704:Matrix
factorizationand canonical
forms
Matrixfactorization
Matrixproperties
Triangularmatrix
Hessenbergmatrix
Hermitian matrix
Unitary matrices
Positive definitematrix
Matrixfactorization
Spectralfactorization
Rankfactorization
LU factorization
Choleskyfactorization
QR factorization
Canonicalforms
Reduced rowechelon form
Jordan normalform
Singular valuefactorization
Similar matrices
Summary
Rank factorization
I Let A be an m × n matrix with rank(A) = r (A has rindependent rows/columns). Then
A = CF
where C ∈Mm×r and F ∈Mr×n
![Page 18: MAA704: Matrix factorization and canonical forms](https://reader030.vdocuments.net/reader030/viewer/2022020703/61fb3c8e2e268c58cd5bc7e7/html5/thumbnails/18.jpg)
MAA704:Matrix
factorizationand canonical
forms
Matrixfactorization
Matrixproperties
Triangularmatrix
Hessenbergmatrix
Hermitian matrix
Unitary matrices
Positive definitematrix
Matrixfactorization
Spectralfactorization
Rankfactorization
LU factorization
Choleskyfactorization
QR factorization
Canonicalforms
Reduced rowechelon form
Jordan normalform
Singular valuefactorization
Similar matrices
Summary
Rank factorization
I How can we find this factorization?
I Rewrite matrix on an echelon form
B =
0 ~ ∗ ∗ ∗ ∗ ∗ ∗0 0 0 ~ ∗ ∗ ∗ ∗0 0 0 0 ~ ∗ ∗ ∗0 0 0 0 0 ~ ∗ ∗0 0 0 0 0 0 0 ~0 0 0 0 0 0 0 00 0 0 0 0 0 0 00 0 0 0 0 0 0 0
![Page 19: MAA704: Matrix factorization and canonical forms](https://reader030.vdocuments.net/reader030/viewer/2022020703/61fb3c8e2e268c58cd5bc7e7/html5/thumbnails/19.jpg)
MAA704:Matrix
factorizationand canonical
forms
Matrixfactorization
Matrixproperties
Triangularmatrix
Hessenbergmatrix
Hermitian matrix
Unitary matrices
Positive definitematrix
Matrixfactorization
Spectralfactorization
Rankfactorization
LU factorization
Choleskyfactorization
QR factorization
Canonicalforms
Reduced rowechelon form
Jordan normalform
Singular valuefactorization
Similar matrices
Summary
Rank factorization
I Create C by removing all columns in A that correspondsto a non-pivot column in B.
I In this example
C =[A.2 A.4 A.5 A.6 A.8
]I Create F by removing all zero rows in B.
I In this example
F =[B1. B2. B3. B4. B5.
]
![Page 20: MAA704: Matrix factorization and canonical forms](https://reader030.vdocuments.net/reader030/viewer/2022020703/61fb3c8e2e268c58cd5bc7e7/html5/thumbnails/20.jpg)
MAA704:Matrix
factorizationand canonical
forms
Matrixfactorization
Matrixproperties
Triangularmatrix
Hessenbergmatrix
Hermitian matrix
Unitary matrices
Positive definitematrix
Matrixfactorization
Spectralfactorization
Rankfactorization
LU factorization
Choleskyfactorization
QR factorization
Canonicalforms
Reduced rowechelon form
Jordan normalform
Singular valuefactorization
Similar matrices
Summary
LU-factorization
I A = LR = LU, L lower (left) triangular
I L is a n × n lower triangular matrix.
I U is a n ×m upper triangular matrix.
I Solve Ax = L(Ux) = b by first solving Ly = b and thensolve Ux = y . Both these systems are easy to solve sinceL and U are both triangular.
I Not every matrix A have a LU factorization, not evenevery square invertible matrix.
![Page 21: MAA704: Matrix factorization and canonical forms](https://reader030.vdocuments.net/reader030/viewer/2022020703/61fb3c8e2e268c58cd5bc7e7/html5/thumbnails/21.jpg)
MAA704:Matrix
factorizationand canonical
forms
Matrixfactorization
Matrixproperties
Triangularmatrix
Hessenbergmatrix
Hermitian matrix
Unitary matrices
Positive definitematrix
Matrixfactorization
Spectralfactorization
Rankfactorization
LU factorization
Choleskyfactorization
QR factorization
Canonicalforms
Reduced rowechelon form
Jordan normalform
Singular valuefactorization
Similar matrices
Summary
LUP-factorization
TheoremEvery n ×m matrix A have a matrix factorization
PA = LU
. where
I P is a n × n permutation matrix.
I L is a n × n lower triangular matrix.
I U is a n ×m upper triangular matrix.
![Page 22: MAA704: Matrix factorization and canonical forms](https://reader030.vdocuments.net/reader030/viewer/2022020703/61fb3c8e2e268c58cd5bc7e7/html5/thumbnails/22.jpg)
MAA704:Matrix
factorizationand canonical
forms
Matrixfactorization
Matrixproperties
Triangularmatrix
Hessenbergmatrix
Hermitian matrix
Unitary matrices
Positive definitematrix
Matrixfactorization
Spectralfactorization
Rankfactorization
LU factorization
Choleskyfactorization
QR factorization
Canonicalforms
Reduced rowechelon form
Jordan normalform
Singular valuefactorization
Similar matrices
Summary
Cholesky factorization
I Systems involving triangular matrices are often easy tosolve.
I Try to rewrite a matrix as a product that contains atriangular matrix seems like a good idea.
I One way is using LU-factorization where PA = LU whereP is a permutation matrix, L is a lower- and U is an uppertriangular matrix.
I More on the LU-factorization in lecture 10.
I There is also the Cholesky factorization, A = GGH , whereA is Hermitian and positive-definite and G is lowertriangular.
![Page 23: MAA704: Matrix factorization and canonical forms](https://reader030.vdocuments.net/reader030/viewer/2022020703/61fb3c8e2e268c58cd5bc7e7/html5/thumbnails/23.jpg)
MAA704:Matrix
factorizationand canonical
forms
Matrixfactorization
Matrixproperties
Triangularmatrix
Hessenbergmatrix
Hermitian matrix
Unitary matrices
Positive definitematrix
Matrixfactorization
Spectralfactorization
Rankfactorization
LU factorization
Choleskyfactorization
QR factorization
Canonicalforms
Reduced rowechelon form
Jordan normalform
Singular valuefactorization
Similar matrices
Summary
Cholesky factorization
I Consider the equation Ax = y . If a can be Choleskyfactorized, A = GGH , this equation can be turned into twonew equations: {
Gz = y
GHx = z
both of these equations are easy to solve.
![Page 24: MAA704: Matrix factorization and canonical forms](https://reader030.vdocuments.net/reader030/viewer/2022020703/61fb3c8e2e268c58cd5bc7e7/html5/thumbnails/24.jpg)
MAA704:Matrix
factorizationand canonical
forms
Matrixfactorization
Matrixproperties
Triangularmatrix
Hessenbergmatrix
Hermitian matrix
Unitary matrices
Positive definitematrix
Matrixfactorization
Spectralfactorization
Rankfactorization
LU factorization
Choleskyfactorization
QR factorization
Canonicalforms
Reduced rowechelon form
Jordan normalform
Singular valuefactorization
Similar matrices
Summary
Applications of Cholesky factorization
I Are there any interesting matrices that can be easyCholesky factorized?
I Any covariance matrix is positive-definite and anycovariance matrix based on measured data is going to besymmetric and real-valued. From the last two properties itfollows that this matrix is Hermitian.
I Example application: generating variates according to amultivariate distribution with covariance matrix Σ andexpected value µUsing the Cholesky factorization you get the simpleformula X = µ+ G>Z where X is the variate, Σ = GGH
and Z is a vector of standard normal variates.
![Page 25: MAA704: Matrix factorization and canonical forms](https://reader030.vdocuments.net/reader030/viewer/2022020703/61fb3c8e2e268c58cd5bc7e7/html5/thumbnails/25.jpg)
MAA704:Matrix
factorizationand canonical
forms
Matrixfactorization
Matrixproperties
Triangularmatrix
Hessenbergmatrix
Hermitian matrix
Unitary matrices
Positive definitematrix
Matrixfactorization
Spectralfactorization
Rankfactorization
LU factorization
Choleskyfactorization
QR factorization
Canonicalforms
Reduced rowechelon form
Jordan normalform
Singular valuefactorization
Similar matrices
Summary
QR factorization
TheoremEvery n ×m matrix A have a matrix decomposition
A = QR
. where
I R is a n ×m upper triangular matrix..
I Q is a n × n unitary matrix.
![Page 26: MAA704: Matrix factorization and canonical forms](https://reader030.vdocuments.net/reader030/viewer/2022020703/61fb3c8e2e268c58cd5bc7e7/html5/thumbnails/26.jpg)
MAA704:Matrix
factorizationand canonical
forms
Matrixfactorization
Matrixproperties
Triangularmatrix
Hessenbergmatrix
Hermitian matrix
Unitary matrices
Positive definitematrix
Matrixfactorization
Spectralfactorization
Rankfactorization
LU factorization
Choleskyfactorization
QR factorization
Canonicalforms
Reduced rowechelon form
Jordan normalform
Singular valuefactorization
Similar matrices
Summary
QR factorization
I Given a QR-factorization we can solve a linear systemAx = b by solving Rx = Q−1b = QHb. Which is can bedone fast since R is a triangular matrix.
I QR-factorization can also used in solving the linear leastsquare problem.
I It plays an important role in the QR-method used tocalculate eigenvalues of a matrix numerically.
![Page 27: MAA704: Matrix factorization and canonical forms](https://reader030.vdocuments.net/reader030/viewer/2022020703/61fb3c8e2e268c58cd5bc7e7/html5/thumbnails/27.jpg)
MAA704:Matrix
factorizationand canonical
forms
Matrixfactorization
Matrixproperties
Triangularmatrix
Hessenbergmatrix
Hermitian matrix
Unitary matrices
Positive definitematrix
Matrixfactorization
Spectralfactorization
Rankfactorization
LU factorization
Choleskyfactorization
QR factorization
Canonicalforms
Reduced rowechelon form
Jordan normalform
Singular valuefactorization
Similar matrices
Summary
Canonical form
I A canonical form is a standard way of describing an object.
I There can be several different kinds of canonical forms foran object.
I Some examples for matrices:I Diagonal form (for diagonalizable matrices)I Reduced row echelon form (for all matrices)I Jordan canonical form (for square matrices)I Singular value factorization form (for all matrices)
![Page 28: MAA704: Matrix factorization and canonical forms](https://reader030.vdocuments.net/reader030/viewer/2022020703/61fb3c8e2e268c58cd5bc7e7/html5/thumbnails/28.jpg)
MAA704:Matrix
factorizationand canonical
forms
Matrixfactorization
Matrixproperties
Triangularmatrix
Hessenbergmatrix
Hermitian matrix
Unitary matrices
Positive definitematrix
Matrixfactorization
Spectralfactorization
Rankfactorization
LU factorization
Choleskyfactorization
QR factorization
Canonicalforms
Reduced rowechelon form
Jordan normalform
Singular valuefactorization
Similar matrices
Summary
Reduced row echelon form
DefinitionA matrix is written on reduced row echelon form when they arewritten on echelon form and their pivot elements are all equalto one and all other elements in a pivot column are zero.
B =
0 1 ∗ 0 0 0 ∗ 00 0 0 1 0 0 ∗ 00 0 0 0 1 0 ∗ 00 0 0 0 0 1 ∗ 00 0 0 0 0 0 0 10 0 0 0 0 0 0 00 0 0 0 0 0 0 00 0 0 0 0 0 0 0
TheoremAll matrices are similar to some reduced row echelon matrix.
![Page 29: MAA704: Matrix factorization and canonical forms](https://reader030.vdocuments.net/reader030/viewer/2022020703/61fb3c8e2e268c58cd5bc7e7/html5/thumbnails/29.jpg)
MAA704:Matrix
factorizationand canonical
forms
Matrixfactorization
Matrixproperties
Triangularmatrix
Hessenbergmatrix
Hermitian matrix
Unitary matrices
Positive definitematrix
Matrixfactorization
Spectralfactorization
Rankfactorization
LU factorization
Choleskyfactorization
QR factorization
Canonicalforms
Reduced rowechelon form
Jordan normalform
Singular valuefactorization
Similar matrices
Summary
Jordan normal form
Definition (Jordan block)
A Jordan block is a square matrix of the form
Jm(λ) =
λ 1 0 . . . 00 λ 1 . . . 0...
.... . .
. . ....
0 0 . . . λ 10 0 . . . 0 λ
![Page 30: MAA704: Matrix factorization and canonical forms](https://reader030.vdocuments.net/reader030/viewer/2022020703/61fb3c8e2e268c58cd5bc7e7/html5/thumbnails/30.jpg)
MAA704:Matrix
factorizationand canonical
forms
Matrixfactorization
Matrixproperties
Triangularmatrix
Hessenbergmatrix
Hermitian matrix
Unitary matrices
Positive definitematrix
Matrixfactorization
Spectralfactorization
Rankfactorization
LU factorization
Choleskyfactorization
QR factorization
Canonicalforms
Reduced rowechelon form
Jordan normalform
Singular valuefactorization
Similar matrices
Summary
Jordan normal form
Definition (Jordan matrix)
A Jordan matrix is a square matrix of the form
J =
Jm1(λ1) 0 . . . 0
0 Jm2(λ2) . . . 0...
.... . .
...0 0 . . . Jm1(λk)
![Page 31: MAA704: Matrix factorization and canonical forms](https://reader030.vdocuments.net/reader030/viewer/2022020703/61fb3c8e2e268c58cd5bc7e7/html5/thumbnails/31.jpg)
MAA704:Matrix
factorizationand canonical
forms
Matrixfactorization
Matrixproperties
Triangularmatrix
Hessenbergmatrix
Hermitian matrix
Unitary matrices
Positive definitematrix
Matrixfactorization
Spectralfactorization
Rankfactorization
LU factorization
Choleskyfactorization
QR factorization
Canonicalforms
Reduced rowechelon form
Jordan normalform
Singular valuefactorization
Similar matrices
Summary
Jordan normal form
TheoremAll square matrices are similar to a Jordan matrix. The Jordanmatrix is unique except for the order of the Jordan blocks. ThisJordan matrix is called the Jordan normal form of the matrix.
Theorem (Some other interesting properties of the Jordannormal form)
Let A = S−1JS
a) The eigenvalues of J is the same as the diagonal elementsof J.
b) J has one eigenvector per Jordan block.
c) The rank of J is equal to the number of Jordan blocks.
d) The normal form is sensitive to perturbations. This meansthat a small change in the normal form can mean a largechange in the A matrix and vice versa.
![Page 32: MAA704: Matrix factorization and canonical forms](https://reader030.vdocuments.net/reader030/viewer/2022020703/61fb3c8e2e268c58cd5bc7e7/html5/thumbnails/32.jpg)
MAA704:Matrix
factorizationand canonical
forms
Matrixfactorization
Matrixproperties
Triangularmatrix
Hessenbergmatrix
Hermitian matrix
Unitary matrices
Positive definitematrix
Matrixfactorization
Spectralfactorization
Rankfactorization
LU factorization
Choleskyfactorization
QR factorization
Canonicalforms
Reduced rowechelon form
Jordan normalform
Singular valuefactorization
Similar matrices
Summary
Singular value factorization
TheoremAll A ∈Mm×n can be factorized as
A = UΣV H
where U and V are unitary matrices and
Σ =
[Sr 00 0
]where Sr is a diagonal matrix with r = rank(A). The diagonalelements of Sr are called the singular values. The singularvalues are uniquely determined by the matrix A (but notnecessarily their order).
![Page 33: MAA704: Matrix factorization and canonical forms](https://reader030.vdocuments.net/reader030/viewer/2022020703/61fb3c8e2e268c58cd5bc7e7/html5/thumbnails/33.jpg)
MAA704:Matrix
factorizationand canonical
forms
Matrixfactorization
Matrixproperties
Triangularmatrix
Hessenbergmatrix
Hermitian matrix
Unitary matrices
Positive definitematrix
Matrixfactorization
Spectralfactorization
Rankfactorization
LU factorization
Choleskyfactorization
QR factorization
Canonicalforms
Reduced rowechelon form
Jordan normalform
Singular valuefactorization
Similar matrices
Summary
Singular value factorization
I Very often referred to as the SVD (singular valuedecomposition).
I Used a lot in statistics and information processing.
I Can be used to quantify many different qualities ofmatrices, more on this in later lectures.
![Page 34: MAA704: Matrix factorization and canonical forms](https://reader030.vdocuments.net/reader030/viewer/2022020703/61fb3c8e2e268c58cd5bc7e7/html5/thumbnails/34.jpg)
MAA704:Matrix
factorizationand canonical
forms
Matrixfactorization
Matrixproperties
Triangularmatrix
Hessenbergmatrix
Hermitian matrix
Unitary matrices
Positive definitematrix
Matrixfactorization
Spectralfactorization
Rankfactorization
LU factorization
Choleskyfactorization
QR factorization
Canonicalforms
Reduced rowechelon form
Jordan normalform
Singular valuefactorization
Similar matrices
Summary
Similar matrices
I In everyday language two matrices are ’similar’ if theyhave almost the same elements or structure. But there isalso a precise mathematical relation between two matricesthat is called similar.
DefinitionTwo matrices, A and B, are similar if A = S−1BS .
![Page 35: MAA704: Matrix factorization and canonical forms](https://reader030.vdocuments.net/reader030/viewer/2022020703/61fb3c8e2e268c58cd5bc7e7/html5/thumbnails/35.jpg)
MAA704:Matrix
factorizationand canonical
forms
Matrixfactorization
Matrixproperties
Triangularmatrix
Hessenbergmatrix
Hermitian matrix
Unitary matrices
Positive definitematrix
Matrixfactorization
Spectralfactorization
Rankfactorization
LU factorization
Choleskyfactorization
QR factorization
Canonicalforms
Reduced rowechelon form
Jordan normalform
Singular valuefactorization
Similar matrices
Summary
Interesting properties of similar matrices
I Similar matrices share several properties:I Eigenvalues (but generally not eigenvectors)I DeterminantI TraceI Rank
I We have already seen some examples of why similarmatrices are interesting:
I Diagonalizable matrices A = S−1BSI Permutation matrices A = PBP>
I Jordan normal form A = S−1JS
I Similarity between matrices mean they represent the samelinear mapping described in different basis. More on this inlecture 8.
![Page 36: MAA704: Matrix factorization and canonical forms](https://reader030.vdocuments.net/reader030/viewer/2022020703/61fb3c8e2e268c58cd5bc7e7/html5/thumbnails/36.jpg)
MAA704:Matrix
factorizationand canonical
forms
Matrixfactorization
Matrixproperties
Triangularmatrix
Hessenbergmatrix
Hermitian matrix
Unitary matrices
Positive definitematrix
Matrixfactorization
Spectralfactorization
Rankfactorization
LU factorization
Choleskyfactorization
QR factorization
Canonicalforms
Reduced rowechelon form
Jordan normalform
Singular valuefactorization
Similar matrices
Summary
Summary
I Triangular and Hessenberg matrices
I Hermitian matrices
I Unitary matrices
I Positive definite matrices
![Page 37: MAA704: Matrix factorization and canonical forms](https://reader030.vdocuments.net/reader030/viewer/2022020703/61fb3c8e2e268c58cd5bc7e7/html5/thumbnails/37.jpg)
MAA704:Matrix
factorizationand canonical
forms
Matrixfactorization
Matrixproperties
Triangularmatrix
Hessenbergmatrix
Hermitian matrix
Unitary matrices
Positive definitematrix
Matrixfactorization
Spectralfactorization
Rankfactorization
LU factorization
Choleskyfactorization
QR factorization
Canonicalforms
Reduced rowechelon form
Jordan normalform
Singular valuefactorization
Similar matrices
Summary
Summary
I Matrix factorizationI Spectral factorization QΛQ−1
I LU-factorizationI Cholesky factorization GGH
I QR-factorizationI Rank factorization CFI Jordan canonical form S−1JSI Singular value factorization UΣV H
![Page 38: MAA704: Matrix factorization and canonical forms](https://reader030.vdocuments.net/reader030/viewer/2022020703/61fb3c8e2e268c58cd5bc7e7/html5/thumbnails/38.jpg)
MAA704:Matrix
factorizationand canonical
forms
Matrixfactorization
Matrixproperties
Triangularmatrix
Hessenbergmatrix
Hermitian matrix
Unitary matrices
Positive definitematrix
Matrixfactorization
Spectralfactorization
Rankfactorization
LU factorization
Choleskyfactorization
QR factorization
Canonicalforms
Reduced rowechelon form
Jordan normalform
Singular valuefactorization
Similar matrices
Summary
Next lecture
I Matrix functions and matrix equations with SergeiSilvestrov
![Page 39: MAA704: Matrix factorization and canonical forms](https://reader030.vdocuments.net/reader030/viewer/2022020703/61fb3c8e2e268c58cd5bc7e7/html5/thumbnails/39.jpg)
MAA704:Matrix
factorizationand canonical
forms
Matrixfactorization
Matrixproperties
Triangularmatrix
Hessenbergmatrix
Hermitian matrix
Unitary matrices
Positive definitematrix
Matrixfactorization
Spectralfactorization
Rankfactorization
LU factorization
Choleskyfactorization
QR factorization
Canonicalforms
Reduced rowechelon form
Jordan normalform
Singular valuefactorization
Similar matrices
Summary
Remember the seminar assignment!
Remember the projects!
Have a nice weekend!