second year mathematics revision...second year mathematics revision linear algebra - part 2 rui tong...
TRANSCRIPT
Second Year Mathematics RevisionLinear Algebra - Part 2
Rui Tong
UNSW Mathematics Society
Rui Tong Second Year Mathematics Revision
Eigenvalues and Eigenvectors
Today’s plan
1 Eigenvalues and EigenvectorsSingular Value Decomposition (MATH2601)
2 The Jordan Canonical FormFinding Jordan FormsThe Cayley-Hamilton Theorem
3 Matrix ExponentialsComputing Matrix ExponentialsApplication to Systems of Differential Equations
Rui Tong Second Year Mathematics Revision
Eigenvalues and Eigenvectors Singular Value Decomposition (MATH2601)
Singular Values (MATH2601 only section)
Definition 1: Singular Values
A singular value of a m × n matrix A is the square root of aneigenvalue of A∗A.
Recall: A∗A denotes the adjoint of A.
Definition 2: Singular Value Decomposition
A SVD for an m × n matrix A is of the form A = UΣV ∗ where
U is an m ×m unitary matrix.
V is an n × n unitary matrix.
Σ has entries
σii > 0. (These are determined by the singular values.)σij = 0 for all i 6= j .
Rui Tong Second Year Mathematics Revision
Eigenvalues and Eigenvectors Singular Value Decomposition (MATH2601)
Nice properties of A∗A
Lemma 1: Properties of A∗A
1 All eigenvalues of A∗A are real and non-negative (even if A hascomplex entries!)
2 ker(A∗A) = ker(A)
3 rank(A∗A) = rank(A)
The first one is pretty much why everything works.
Rui Tong Second Year Mathematics Revision
Eigenvalues and Eigenvectors Singular Value Decomposition (MATH2601)
SVD Algorithm
Algorithm 1: Finding a SVD
1 Find all eigenvalues λi of A∗A and write in descending order.Also find their associated eigenvectors of unit length vi .
2 Find an orthonormal set of eigenvectors for A∗A.
Automatically occurs when all eigenvalues are distinct, whichwill usually be the case. Otherwise require Gram-Schmidt forany eigenspace with dimension strictly greater than 1..
3 Compute ui = 1√λiAvi for each non-zero eigenvalue.
4 State U and V from the vectors you found, and Σ from thesingular values.
Lemma 2: Used to speed up step 1
A∗A and AA∗ share the same non-zero eigenvalues.
If rank(A) = r , then A∗A has r non-zero eigenvalues. All othereigenvalues are 0.
Rui Tong Second Year Mathematics Revision
Eigenvalues and Eigenvectors Singular Value Decomposition (MATH2601)
SVD Example
Example 1: MATH2601 2017 Q2 c)
For the matrix A =
(3 1 1−1 3 1
)1 Find the eigenvalues of AA∗.
2 Explain why the eigenvalues in part 1 are also eigenvalues ofA∗A, and state any other eigenvalues of A∗A.
3 Find all eigenvectors of A∗A.
4 Find a singular value decomposition for A.
Rui Tong Second Year Mathematics Revision
Eigenvalues and Eigenvectors Singular Value Decomposition (MATH2601)
SVD Example
Part 1: We compute
AA∗ =
(3 1 1−1 3 1
)3 −11 31 1
=
(11 11 11
).
This matrix has two eigenvalues, and they sum to tr(AA∗) = 22 andmultiply to det(AA∗) = 120. By inspection, λ1 = 12 and λ2 = 10.
Rui Tong Second Year Mathematics Revision
Eigenvalues and Eigenvectors Singular Value Decomposition (MATH2601)
SVD Example
Part 2: Quoted word for word from the answers...
”We know that A∗A and AA∗ have the same nonzero eigenvalues,so 12 and 10 are eigenvalues of A∗A.
Also, all eigenvalues of A∗A are real and nonnegative, so its thirdeigenvalue is 0.”
Rui Tong Second Year Mathematics Revision
Eigenvalues and Eigenvectors Singular Value Decomposition (MATH2601)
SVD Example
Part 3: We compute
A∗A =
3 −11 31 1
( 3 1 1−1 3 1
)=
10 0 20 10 42 4 2
For λ = 12:
A∗A− 12I =
−2 0 20 −2 42 4 −10
.
Looking at row 1, arbitrarily set first component to 1, and then thethird component is 1. Equating row 2, the second component is 2.
∴ v1 = t
121
.
Rui Tong Second Year Mathematics Revision
Eigenvalues and Eigenvectors Singular Value Decomposition (MATH2601)
SVD Example
Part 3: We compute
A∗A =
3 −11 31 1
( 3 1 1−1 3 1
)=
10 0 20 10 42 4 2
For λ = 10:
A∗A− 10I =
0 0 20 0 42 4 −8
.
Rows 1 and 2 force the third component to be 0. Looking at row 3,it is easier to set the second component to 1, and then the firstcomponent will be −2.
∴ v2 = t
−210
.
Rui Tong Second Year Mathematics Revision
Eigenvalues and Eigenvectors Singular Value Decomposition (MATH2601)
SVD Example
Part 3: We compute
A∗A =
3 −11 31 1
( 3 1 1−1 3 1
)=
10 0 20 10 42 4 2
For λ = 0, looking at A∗A itself, there really are many possibilities
we can go about it. But I follow the answers, which arbitrarily setthe first component to −1. See if you can then show that
v3 = t
−1−25
.
Note: In each case, t ∈ R.
Rui Tong Second Year Mathematics Revision
Eigenvalues and Eigenvectors Singular Value Decomposition (MATH2601)
SVD Example
Part 4: In each case, choose the value of t that normalises theeigenvectors:
v1 =1√6
121
(λ1 = 12)
v2 =1√5
−210
(λ2 = 10)
v3 =1√30
−1−25
(λ3 = 0)
Rui Tong Second Year Mathematics Revision
Eigenvalues and Eigenvectors Singular Value Decomposition (MATH2601)
SVD Example
Part 4: Compute ui = 1√λiAvi for each non-zero eigenvector:
u1 =1√12
(3 1 1−1 3 1
)1√6
121
=1√2
(11
)
u2 =1√10
(3 1 1−1 3 1
)1√5
−210
=1√2
(−11
)
Rui Tong Second Year Mathematics Revision
Eigenvalues and Eigenvectors Singular Value Decomposition (MATH2601)
SVD Example
Part 4: We conclude that a SVD for A is A = UΣV ∗, where
U =1√2
(1 −11 1
)Σ =
(√12 0 0
0√
10 0
)
V =
1√6− 2√
5− 1√
302√6
1√5− 2√
301√6
0 5√30
Rui Tong Second Year Mathematics Revision
Eigenvalues and Eigenvectors Singular Value Decomposition (MATH2601)
Remark
For A = UΣV ∗, where A ∈ Mn×n(C):
The columns of U =(u1 . . . um
)are called the left singular
vectors.
The columns of V =(v1 . . . vn
)are called the right
singular vectors.
Word of advice: Write some of these numbers very quickly! SVDsare instructive when you know the method, but it always takesforever to do.
Rui Tong Second Year Mathematics Revision
Eigenvalues and Eigenvectors Singular Value Decomposition (MATH2601)
Reduced SVD
I don’t see these examined, but I should still mention them.
1 Obtain Σ̂ by removing any zero columns in Σ
2 Obtain V̂ by removing the corresponding columns in V .
3 Then, A = UΣ̂V̂ ∗.
For the earlier example:
Σ̂ =
(√12 0
0√
10
)
V̂ =
1√6− 2√
52√6
1√5
1√6
0
Rui Tong Second Year Mathematics Revision
The Jordan Canonical Form
Today’s plan
1 Eigenvalues and EigenvectorsSingular Value Decomposition (MATH2601)
2 The Jordan Canonical FormFinding Jordan FormsThe Cayley-Hamilton Theorem
3 Matrix ExponentialsComputing Matrix ExponentialsApplication to Systems of Differential Equations
Rui Tong Second Year Mathematics Revision
The Jordan Canonical Form Finding Jordan Forms
Jordan Blocks
Definition 3: Jordan blocks
The k × k Jordan block for λ is the matrix
Jk(λ) =
λ 1 0 . . . 00 λ 1 . . . 00 0 λ . . . 0...
......
. . ....
0 0 0 . . . 10 0 0 . . . λ
∈ Mk×k(C).
That is, put λ on every entry along the main diagonal, and a 1immediately above each λ wherever possible.
Rui Tong Second Year Mathematics Revision
The Jordan Canonical Form Finding Jordan Forms
Jordan Blocks
Quick examples:
J3(−4) =
−4 1 00 −4 10 0 −4
J4(0) =
0 1 0 00 0 1 00 0 0 10 0 0 0
Rui Tong Second Year Mathematics Revision
The Jordan Canonical Form Finding Jordan Forms
Powers of Jordan Forms
Find the pattern.
J1(λ)n =(λn)
J2(λ)n =
(λn
(n1
)λn−1
0 λn
)
J3(λ)n =
λn (n1
)λn−1
(n2
)λn−2
0 λn(n
1
)λn−1
0 0 λn
J4(λ)n =
λn
(n1
)λn−1
(n2
)λn−2
(n3
)λn−3
0 λn(n
1
)λn−1
(n2
)λn−2
0 0 λn(n
1
)λn−1
0 0 0 λn
Rui Tong Second Year Mathematics Revision
The Jordan Canonical Form Finding Jordan Forms
Powers of Jordan Forms
Lemma 3: Computing powers of Jordan forms
1 Start with λn on every diagonal entry.
2 Put(n
1
)λn−1 wherever you can immediately above λn
3 Put(n
2
)λn−2 wherever you can immediately above
(n1
)λn−1
4 Keep doing this, increasing the binomial coefficient anddecreasing the power on λ.
Note: Not quite the above. If you ever bump into(nn
), that’s the
last diagonal you fill. Just put 0’s everywhere else above.
Rui Tong Second Year Mathematics Revision
The Jordan Canonical Form Finding Jordan Forms
Matrix Direct Sums
Definition 4: Direct sums of matrices
The direct sum of matrices A1,A2, . . . ,An is the matrix formed byputting these matrices on the diagonals and zeroes everywhere else.
A1 ⊕ A2 ⊕ · · · ⊕ An =
A1 0 . . . 00 A2 . . . 0...
.... . .
...0 0 . . . An
In MATH2501 and MATH2601, we only worry about this withJordan blocks.
Rui Tong Second Year Mathematics Revision
The Jordan Canonical Form Finding Jordan Forms
Matrix Direct Sums
Quick example:
J2(3)⊕ J4(5) =
3 1 0 0 0 00 3 0 0 0 00 0 5 1 0 00 0 0 5 1 00 0 0 0 5 10 0 0 0 0 5
Rui Tong Second Year Mathematics Revision
The Jordan Canonical Form Finding Jordan Forms
Matrix Direct Sums
Lemma 4: Powers on direct sums of Jordan blocks
The power of a Jordan form is the direct sum of powers on eachindividual block. I.e.,
(J1 ⊕ · · · ⊕ Jm)n = Jn1 ⊕ · · · ⊕ Jnm.
Example:
[J2(3)⊕ J1(2)]n =
3n(n
1
)3n−1 0
0 3n 00 0 2n
Rui Tong Second Year Mathematics Revision
The Jordan Canonical Form Finding Jordan Forms
The Generalised Eigenvector
Definition 5: Generalised Eigenvector
A generalised eigenvector corresponding to eigenvalue λ is a
non-zero vector v satisfying the property (A− λI )kv = 0 , for
some k ≥ 1.
This differs from the (usual) eigenvector in the sense that thosemust satisfy (A− λI )v = 0, i.e. we must take k = 1.
Rui Tong Second Year Mathematics Revision
The Jordan Canonical Form Finding Jordan Forms
The Generalised Eigenvector
Example 2: MATH2601 2016 Q4 c)
Let C =
9 7 −3−2 2 12 5 2
and v =
1−2−3
.
Show that for the matrix C , v is a generalised eigenvectorcorresponding to λ = 5.
Rui Tong Second Year Mathematics Revision
The Jordan Canonical Form Finding Jordan Forms
The Generalised Eigenvector
We compute that
C − 5I =
4 7 −3−2 −3 12 5 −3
and continuously left-multiplying to v,
(C − 5I )v =
−111
(C − 5I )2v =
000
so v ∈ GE5.
Rui Tong Second Year Mathematics Revision
The Jordan Canonical Form Finding Jordan Forms
Generalised Eigenspaces
Definition 6: Generalised Eigenspace
The generalised eigenspace of λ, denoted GEλ, is the set of allgeneralised eigenvectors corresponding to λ.
GEλ = {v ∈ Cn | (A− λI )kv = 0 for some k ≥ 1}
Lemma 4: Alternate representation of GEλ
GEλ = ker(A− λI ) ∪ ker(A− λI )2 ∪ ker(A− λI )3 ∪ . . .
Rui Tong Second Year Mathematics Revision
The Jordan Canonical Form Finding Jordan Forms
Computing Jordan Forms
Definition 7: Jordan matrix
A Jordan matrix J is a direct sum of Jordan blocks.
Lemma 5: Uniqueness
Every matrix A has one unique Jordan matrix, up to somepermutation (arrangement) of the Jordan blocks.
Rui Tong Second Year Mathematics Revision
The Jordan Canonical Form Finding Jordan Forms
Computing Jordan Forms
Theorem 1: Useful properties in computing Jordan forms
Let dim ker(A− λI )k , i.e. nullity(A− λI )k = dk . Set d0 = 0. Then
1 ker(A− λI ) ⊆ ker(A− λI )2 ⊆ . . .2 d0 ≤ d1 ≤ d2 ≤ d3 ≤ . . .3 d1 − d0 ≥ d2 − d1 ≥ d3 − d2 ≥ . . .
That is to say, the nullities must not decrease, but the difference innullities must not INcrease.
Remark: Multiplicity
As a corollary, the algebraic multiplicity of an eigenvalue λ equals todimGEλ. This allows us to not compute (A− λI )k forever - westop when nullity(A− λI )k = AM.
Rui Tong Second Year Mathematics Revision
The Jordan Canonical Form Finding Jordan Forms
Computing Jordan Forms
We use Jordan chains to find the matrices P and J, such thatA = PJP−1. For an eigenvalue λ with algebraic multiplicity k , weneed to start with some vector v1 such that on multiplication, wehave
v1A−λI−−−→ v2
A−λI−−−→ . . .A−λI−−−→ vk
A−λI−−−→ 0.
We then include (note the reverse order!)(vk . . . v2 v1
)to P . This corresponds to one Jordan block Jk(λ) in the direct sumfor the Jordan matrix J of A.
Rui Tong Second Year Mathematics Revision
The Jordan Canonical Form Finding Jordan Forms
Computing Jordan Forms
(Or maybe your lecturer taught things the other way around.) Weconsider this chain
vkA−λI−−−→ vk−1
A−λI−−−→ . . .A−λI−−−→ v1
A−λI−−−→ 0.
and we include this to P instead.(v1 . . . vk−1 vk
)We still use the Jordan block Jk(λ).
Rui Tong Second Year Mathematics Revision
The Jordan Canonical Form Finding Jordan Forms
Computing Jordan Forms: Example 1
Example 3: MATH2601 2016 Q4 c)
Let C =
9 7 −3−2 2 12 5 2
and v =
1−2−3
.
1 Calculate (C − 5I )v and (C − 5I )2v. (Done earlier)
2 Without using any matrix calculations, write down all theeigenvalues of C and their algebraic and geometricmultiplicities. Give reasons for your answers.
3 (Not originally in the exam:) Find an invertible matrix P and aJordan matrix J such that C = PJP−1.
Rui Tong Second Year Mathematics Revision
The Jordan Canonical Form Finding Jordan Forms
Computing Jordan Forms: Example 1
Part 2: The trace is usually helpful, because it is the sum of theeigenvalues.
tr(C ) = 9 + 2 + 2 = 13
From part 1, 5 is an eigenvalue of C with algebraic multiplicity atleast 2. The third eigenvalue λ3 satisfies
5 + 5 + λ3 = 13 =⇒ λ3 = 3.
Which is, of course, the only remaining eigenvalue and hence musthave AM = 1. So we have:
Eigenvalue 5: AM = 2, GM = 1
Eigenvalue 3: AM = 1, GM = 1
Note: I haven’t justified the GM’s! Try doing that yourself!
Rui Tong Second Year Mathematics Revision
The Jordan Canonical Form Finding Jordan Forms
Computing Jordan Forms: Example 1
Part 3: Row reducing C − 3I ,
C − 3I =
6 7 −3−2 −1 12 5 −1
∼
0 −12 0−2 −1 10 4 0
∼
−2 −1 10 1 00 0 0
so we can take a corresponding eigenvector v3 =
102
.
Rui Tong Second Year Mathematics Revision
The Jordan Canonical Form Finding Jordan Forms
Computing Jordan Forms: Example 1
So our chains are: 1−2−3
C−5I−−−→
−111
C−5I−−−→ 0
102
C−3I−−−→ 0
and hence A = PJP−1 where
J =
5 1 00 5 00 0 3
and P =
−1 1 11 −2 01 −3 2
Rui Tong Second Year Mathematics Revision
The Jordan Canonical Form Finding Jordan Forms
Computing Jordan Forms: Example 2 (time permitting...)
Example 4: MATH2601 2017 Q3 a)
Let A =
3 1 −22 6 −72 2 −2
. We are given that
GE2 = span
1
32
,
011
and GE3 = span
1
42
.
1 Find the Jordan chain for λ = 2 starting with
011
.
2 Without any calculation, write down the geometric multiplicityof λ = 2, giving reasons for your answer.
3 Find a Jordan form J and invertible matrix P for A, such thatA = PJP−1.
Rui Tong Second Year Mathematics Revision
The Jordan Canonical Form Finding Jordan Forms
Computing Jordan Forms: Example 2 (time permitting...)
Part 1: 011
A−2I−−−→
−1−3−2
A−2I−−−→
000
.
Part 2: The algebraic multiplicity is 2, since we have anotherdistinct eigenvalue, so GM ≤ 2. But GM 6= 2 since we have a chainof length 2, so GM = 1.
Rui Tong Second Year Mathematics Revision
The Jordan Canonical Form Finding Jordan Forms
Computing Jordan Forms: Example 2 (time permitting...)
Part 3: A = PJP−1 where
P =
−1 0 1−3 1 4−2 1 2
J =
2 1 00 2 00 0 3
Rui Tong Second Year Mathematics Revision
The Jordan Canonical Form Finding Jordan Forms
Computing Jordan Forms: Example 2 (time permitting...)
Example 4: MATH2601 2017 Q3 a)
4 Find v ∈ GE2 and w ∈ GE3 such that v + w =
210
Theorem 2: Cn and the generalised eigenspaces
The direct sum of generalised eigenspaces of any A ∈ Mn×n spanCn.
Hence we just need to express
210
as a linear combination of132
,
011
and
142
.
Rui Tong Second Year Mathematics Revision
The Jordan Canonical Form Finding Jordan Forms
Computing Jordan Forms: Example 2 (time permitting...)
You can have fun with the row reduction... I’ll just state the finalanswer: 2
10
= 3
132
− 4
011
−1
42
=
352
︸ ︷︷ ︸
v
+
−1−4−2
︸ ︷︷ ︸
w
Rui Tong Second Year Mathematics Revision
The Jordan Canonical Form Finding Jordan Forms
Remark: Similarity Invariants
Theorem 3: Jordan forms are the complete similarity invariant
Two matrices A and B are similar, i.e. A = PBP−1 for someinvertible matrix P, if and only if they have the same Jordan forms.(At least, to within a different arrangement of direct sums.)
Rui Tong Second Year Mathematics Revision
The Jordan Canonical Form Finding Jordan Forms
Jordan forms given nullities
The Jordan matrix J can sometimes be found with less informationif we don’t need to find P.
Example 5: MATH2601 2016 Q4 b)
Let B be a 10× 10 matrix and let λ be a scalar. Suppose it isknown that
nullity(B − λI ) = 5,
nullity(B − λI )2 = 8,
nullity(B − λI )3 = 9.
Find all possible Jordan forms of B.
Idea: Our Jordan chains can be drawn on an onion diagram.
Rui Tong Second Year Mathematics Revision
The Jordan Canonical Form Finding Jordan Forms
Jordan forms given nullities
There are 5 eigenvectors in ker(B − λI ). The idea is that there are8− 5 = 3 more generalised eigenvectors in ker(B − λI )2. This isbecause we know that ker(B − λI ) ⊆ ker(B − λI )2.
Similarly, there is another 9− 8 = 1 in ker(B − λI )3.
Rui Tong Second Year Mathematics Revision
The Jordan Canonical Form Finding Jordan Forms
Jordan forms given nullities
We’ve addressed 9 of the 10 eigenvalues for B. There is only onemore left to go.
Case 1: The tenth eigenvalue is NOT λ.Then it must be some other value µ 6= λ. It can only correspond toone eigenvector, so we include J1(µ) to the direct sum.
The Jordan chains for λ have lengths 3, 2, 2, 1 and 1, so therefore(up to some permutation),
J = J3(λ)⊕ J2(λ)⊕ J2(λ)⊕ J1(λ)⊕ J1(λ)⊕ J1(µ).
(again, up to some permutation of the Jordan blocks).
Rui Tong Second Year Mathematics Revision
The Jordan Canonical Form Finding Jordan Forms
Jordan forms given nullities
We’ve addressed 9 of the 10 eigenvalues for B. There is only onemore left to go.
Case 2: The tenth eigenvalue IS also λ.Problem: We cannot add it in ker(B − λI ), ker(B − λI )2 orker(B − λI )3 without screwing up the nullities!
Recall that the difference is nullities is non-increasing. This meansthat the last generalised eigenvector must be in ker(B − λI )4. Ouroriginal chain of length 3 also becomes chain of length 4. So we get
J = J4(λ)⊕ J2(λ)⊕ J2(λ)⊕ J1(λ)⊕ J1(λ)
(again, up to some permutation of the Jordan blocks).
Rui Tong Second Year Mathematics Revision
The Jordan Canonical Form Finding Jordan Forms
Jordan forms given nullities
Remark: Why ker(B − λI )4? For completeness sake, here’s a quickcontradiction.
Suppose, say, the remaining generalised eigenvector was inker(B − λI )5 but not in ker(B − λI )4. Then ker(B − λI )4 must infact be equal to ker(B − λI )3, so d4 = d3, i.e. d4 − d3 = 0. Yetd5 − d4 = 1. Therefore d5 − d4 > d4 − d3, which cannot happen.
Rui Tong Second Year Mathematics Revision
The Jordan Canonical Form Finding Jordan Forms
Invalid nullities
The property d1 − d0 ≥ d2 − d1 ≥ d3 − d2 ≥ . . . helps determinethings that are impossible.
Example 6: David Angell’s MATH2601 notes
Let A be a matrix with eigenvalue λ. Explain why this is notpossible:
nullity(A− λI ) = 5,
nullity(A− λI )2 = 8,
nullity(A− λI )3 = 9,
nullity(A− λI )4 = 12,
nullity(A− λI )k = 12 for all k > 4.
Answer: d4 − d3 = 3 > 1 = d3 − d2, which can’t happen.
Rui Tong Second Year Mathematics Revision
The Jordan Canonical Form Finding Jordan Forms
From Jordan forms back to nullities
Example 7: Peter Brown’s MATH2501 notes
If A is similar to J = J2(−4)⊕ J2(−4)⊕ J2(−4)⊕ J3(5)⊕ J1(5).find
nullity(A + 4I )k and nullity(A− 5I )k
for all k ≥ 1.
Solution: Go backwards!
Rui Tong Second Year Mathematics Revision
The Jordan Canonical Form Finding Jordan Forms
From Jordan forms back to nullities
We know the lengths of the chains...
Rui Tong Second Year Mathematics Revision
The Jordan Canonical Form Finding Jordan Forms
From Jordan forms back to nullities
So we see that:
nullity(A + 4I ) = 3
nullity(A + 4I )k = 6 for all k ≥ 2
nullity(A− 5I ) = 2
nullity(A− 5I )2 = 3
nullity(A− 5I )k = 4 for all k ≥ 3
Rui Tong Second Year Mathematics Revision
The Jordan Canonical Form Finding Jordan Forms
Remark: T -invariance (MATH2601)
I’ve seen questions on this pop up in tutorials and exams, so I’ll givean example involving proving this result. I won’t have time to goover it in class though.
Definition 8: Invariance under T
A subspace U of V is said to be invariant under a transformation Tif T (U) ⊆ U.
Example 8: MATH2601 2018 Q3 b)
Let V be a vector space, let S and T be linear transformations fromV to V , and write W = ker(S − T ). Show that if ST = TS thenW is invariant under T .
Rui Tong Second Year Mathematics Revision
The Jordan Canonical Form Finding Jordan Forms
Remark: T -invariance (MATH2601)
Let V be a vector space and let S and T be linear transformationsfrom V to V . Let W = ker(S − T ) and suppose that ST = TS .
Let v ∈ T (W ). Then v = T (w) for some w ∈W .
Goal: Show that v ∈W = ker(S − T ), i.e. (S − T )(v) = 0 .
Then,
(S − T )(v) = S(v)− T (v)
= S(T (w))− T (T (w))
= T (S(w))− T (T (w))
since ST = TS .
Rui Tong Second Year Mathematics Revision
The Jordan Canonical Form Finding Jordan Forms
Remark: T -invariance (MATH2601)
Further, since T is linear,
(S − T )(v) = T (S(w)− T (w))
= T ((S − T )(w)) .
But since w ∈W = ker(S − T ), we know that (S − T )(w) = 0.Hence
(S − T )(v) = T (0)
= 0.
Therefore v ∈W , so T (W ) ⊆W and henceW is invariant under T .
Rui Tong Second Year Mathematics Revision
The Jordan Canonical Form The Cayley-Hamilton Theorem
The Companion Matrix (MATH2501)
The companion matrix allows us to go backwards from acharacteristic polynomial to a matrix. (Or at least, one such matrix.)
Definition 9: Companion matrix
Consider the polynomial
f (λ) = λn + an−1λn−1 + an−2λ
n−2 + · · ·+ a1λ+ a0.
A matrix C whose characteristic polynomial is f (λ) is
C =
0 0 . . . 0 0 −a0
1 0 . . . 0 0 −a1
0 1 . . . 0 0 −a2...
. . ....
0 0 . . . 1 0 −an−2
0 0 . . . 0 1 −an−1
Rui Tong Second Year Mathematics Revision
The Jordan Canonical Form The Cayley-Hamilton Theorem
The Companion Matrix (MATH2501)
Example: The companion matrix corresponding top(λ) = λ3 − 3λ2 + 2λ− 5 is
C =
0 0 51 0 −20 1 3
.
Here, we set a0 = −5, a1 = −2 and a2 = 3.
Rui Tong Second Year Mathematics Revision
The Jordan Canonical Form The Cayley-Hamilton Theorem
Recursively finding matrix powers
Theorem 4: The Cayley-Hamilton Theorem
Let A be an n × n matrix and f (z) be its characteristic polynomial.
Then f (A) = 0 , the zero matrix.
Example 9: Peter Brown’s MATH2501 notes
1 Verify the Cayley-Hamilton Theorem for A =
(1 34 −2
)2 Use the Cayley-Hamilton Theorem to express A4 and A−1 in
terms of A and I , where I is the 2× 2 identity matrix.
Rui Tong Second Year Mathematics Revision
The Jordan Canonical Form The Cayley-Hamilton Theorem
Recursively finding matrix powers
Part 1: Begin by computing
cpA(z) =
∣∣∣∣1− z 34 −2− z
∣∣∣∣= (z − 1)(z + 2)− 12
= z2 + z − 14
Then observe that
cpA(A) =
(13 −3−4 16
)+
(1 34 −2
)− 14
(1 00 1
)=
(0 00 0
)where we note that the constant term gets multiplied to theidentity matrix.
Rui Tong Second Year Mathematics Revision
The Jordan Canonical Form The Cayley-Hamilton Theorem
Recursively finding matrix powers
Part 2: Using the Cayley-Hamilton Theorem, we know thatA2 = −A + 14I . Hence
A3 = −A2 + 14A
= −(−A + 14I ) + 14A
= 15A− 14I
A4 = 15A2 − 14A
= 15(−A + 14I )− 14A
= −29A + 210I
Also A = −I + 14A−1, so A−1 = 114A + 1
14 I .
Rui Tong Second Year Mathematics Revision
The Jordan Canonical Form The Cayley-Hamilton Theorem
Minimal polynomials (MATH2501)
Note: This has been taken out of the higher syllabus.
Definition 10: Minimal polynomial of a matrix
Let A be an n × n matrix. The minimal polynomial m of A is thepolynomial:
of smallest degree possible
and monic (i.e. the leading coefficient is 1)
such that m(A) = 0.
Lemma 6: Minimal polynomials and characteristic polynomials
The minimal polynomial is a factor of the characteristic polynomial.(Not really useful for computations, but it can be a nice sanitycheck.)
We won’t delve much into the theory, we just illustratehow to find it.
Rui Tong Second Year Mathematics Revision
The Jordan Canonical Form The Cayley-Hamilton Theorem
Minimal polynomials (MATH2501)
Theorem 5: Explicit form for the minimal polynomial
Let A be an n × n matrix and denote the distinct eigenvalues of Aas λ1, λ2, . . . , λr .
For the i-th eigenvalue λi , let bi be the size of the largest Jordanblock corresponding to λi .
Then the minimal polynomial of A is
m(z) = (z − λ1)b1(z − λ2)b2 . . . (z − λr )br .
Rui Tong Second Year Mathematics Revision
The Jordan Canonical Form The Cayley-Hamilton Theorem
Minimal polynomials (MATH2501)
Example 10: Peter Brown’s MATH2501 notes
The Jordan form of A ∈ M15×15 is
J5(2)⊕ J2(2)⊕ J3(−2)⊕ J3(−2)⊕ J2(−2).
What is its minimal polynomial?
The largest block for λ = 2 has size 5, and the largest block forλ = −2 has size 3. Therefore
m(z) = (z − 2)5(z + 2)3.
Rui Tong Second Year Mathematics Revision
The Jordan Canonical Form The Cayley-Hamilton Theorem
Minimal polynomials (MATH2501)
Example 11: Peter Brown’s MATH2501 notes
The matrix A =
3 5 −4−2 −4 4−1 −3 4
has characteristic polynomial
cpA(z) = z(z − 1)(z − 2).
What is its minimal polynomial?
The characteristic polynomial shows that the 3× 3 matrix A hasthree distinct eigenvalues, so it must be diagonalisable. HenceJ = J1(0)⊕ J1(1)⊕ J1(2), so for this matrix,
m(z) = cpA(z) = z(z − 1)(z − 2).
Rui Tong Second Year Mathematics Revision
Matrix Exponentials
Today’s plan
1 Eigenvalues and EigenvectorsSingular Value Decomposition (MATH2601)
2 The Jordan Canonical FormFinding Jordan FormsThe Cayley-Hamilton Theorem
3 Matrix ExponentialsComputing Matrix ExponentialsApplication to Systems of Differential Equations
Rui Tong Second Year Mathematics Revision
Matrix Exponentials Computing Matrix Exponentials
Matrix Exponential
Definition 11: Exponential of a matrix
The matrix exponential exp(tA) is defined as
exp(tA) = etA =∞∑k=0
tk
k!Ak .
Rui Tong Second Year Mathematics Revision
Matrix Exponentials Computing Matrix Exponentials
Computing matrix exponentials
We will illustrate the ideas...
Lemma 7: Properties of matrix exponentials
1 If A = PBP−1, then exp(A) = P exp(B)P−1.
2 If A = A1 ⊕ · · · ⊕ An, then exp(A) = exp(A1)⊕ · · · ⊕ exp(An)
3 exp (tJk(λ)) = eλt
1 t t2
2! . . . tk−1
(k−1)!
0 1 t
0 0 1. . .
.... . .
0 0 0 . . . t0 0 0 . . . 1
Also nice to note is that if AB = BA, thenexp(A) exp(B) = exp(A + B).
Rui Tong Second Year Mathematics Revision
Matrix Exponentials Computing Matrix Exponentials
Computing matrix exponentials
Example for a Jordan block:
exp (tJ5(2)) = e2t
1 t t2
2!t3
3!t4
4!
0 1 t t2
2!t3
3!
0 0 1 t t2
2!0 0 0 1 t0 0 0 0 1
We’re really filling the matrix in with terms from the power series ofet , but then leaving a usual exponential in front.
Rui Tong Second Year Mathematics Revision
Matrix Exponentials Computing Matrix Exponentials
Computing matrix exponentials
Example 12: Not really an example...
Consider C =
9 7 −3−2 2 12 5 2
from earlier. We want exp(tC ).
We have
P =
−1 1 11 −2 01 −3 2
and J =
5 1 00 5 00 0 3
.
Rui Tong Second Year Mathematics Revision
Matrix Exponentials Computing Matrix Exponentials
Computing matrix exponentials
The earlier results show that we can do powers of Jordan blocksone at a time. So we obtain
exp(J) =
e5t te5t 00 e5t 00 0 e3t
and hence
exp(C ) = P
e5t te5t 00 e5t 00 0 e3t
P−1
Rui Tong Second Year Mathematics Revision
Matrix Exponentials Computing Matrix Exponentials
A huge pain, as you can see.
So you probably won’t be asked to do that in an exam. But youmay be asked something else.
Rui Tong Second Year Mathematics Revision
Matrix Exponentials Application to Systems of Differential Equations
The ‘Columns’ technique
Theorem 6: Matrix Exponential times Generalised Eigenvector
If we have the Jordan chain
v1A−λI−−−→ v2
A−λI−−−→ v3A−λI−−−→ . . .
A−λI−−−→ vk
then
exp(tA)v1 = eλt(v1 + tv2 +
t2
2v3 + · · ·+ tk−1
(k − 1)!vk
)
Rui Tong Second Year Mathematics Revision
Matrix Exponentials Application to Systems of Differential Equations
The ‘Columns’ technique
This does come with a caveat in that v1 must be a generalisedeigenvector corresponding to λ.
(Otherwise, we have to decompose it into a sum of generalisedeigenvectors first.)
Rui Tong Second Year Mathematics Revision
Matrix Exponentials Application to Systems of Differential Equations
Solving Homogeneous systems of DEs
More often than not, we just need to compute exp(tA)v for somevector v, instead of the actual matrix exponential itself.
Theorem 7: Solution to a homogeneous system
The solution to dydt = Ay with initial condition y(0) = c is
y = exp(tA)c.
Rui Tong Second Year Mathematics Revision
Matrix Exponentials Application to Systems of Differential Equations
Solving Homogeneous systems of DEs
Example 13: MATH2601 2016 Q4 c)
Recall for C =
9 7 −3−2 2 12 5 2
and v =
1−2−3
we have the chain
1−2−3
C−5I−−−→
−111
C−5I−−−→ 0.
Use this to solve the initial value problem y′ = Cy, y(0) = v.
Rui Tong Second Year Mathematics Revision
Matrix Exponentials Application to Systems of Differential Equations
Solving Homogeneous systems of DEs
The solution is
y = exp(tA)
1−2−3
.
So using the columns method,
y = e5t
1−2−3
+ t
−111
= e5t
1− t−2 + t−3 + t
Rui Tong Second Year Mathematics Revision
Matrix Exponentials Application to Systems of Differential Equations
The more general case (if time permits)
In general, if we can decompose c into a sum of generalisedeigenvectors, we work our way around this issue.
Example 14: MATH2601 2017 Q3 a)
For A =
3 1 −22 6 −72 2 −2
, solve y′ = Ay with initial condition
y(0) =
210
, given that
210
=
352
︸ ︷︷ ︸∈GE2
+
−1−4−2
︸ ︷︷ ︸∈GE3
.
Rui Tong Second Year Mathematics Revision
Matrix Exponentials Application to Systems of Differential Equations
The more general case (if time permits)
Construct the chains:352
A−2I−−−→
4128
A−2I−−−→ 0
−1−4−2
A−3I−−−→ 0
Our solution will thus be
y = etA
352
+ etA
−1−4−2
= e2t
3 + 4t5 + 12t2 + 8t
+ e3t
−1−4−2
.
Rui Tong Second Year Mathematics Revision
Matrix Exponentials Application to Systems of Differential Equations
Solving Non-homogeneous systems of DEs
Lemma 8: Solution to non-homogeneous systems
The general solution to y′ = Ay can be expressed as y = yH + yPwhere
yH is the general solution to y′ = Ay
yP is any particular solution to y′ = Ay + b
Lemma 9: Variation of parameters
We can approach the particular solution by subbing y = etAz.
Lemma 10: Derivative of matrix exponential
d
dtetA = AetA
Rui Tong Second Year Mathematics Revision
Matrix Exponentials Application to Systems of Differential Equations
Variation of parameters
In general, upon substituting in y = etAz intody
dt= Ay + b, we
have
d(etAz)
dt= AetAz + b
etAz′ + AetAz = AetAz + b
etAz′ = b
z′ = e−tAb
So what we can do is:
1 Find z′, probably using the columns technique again.
2 Integrate out to find z.
3 Recompute y = etAz for our particular solution.
Rui Tong Second Year Mathematics Revision
Matrix Exponentials Application to Systems of Differential Equations
Solving Non-homogeneous systems of DEs
Example 15: MATH2601 2016 Q4 c)
Find a particular solution of y′ = Cy + te5tw, where
C =
9 7 −3−2 2 12 5 2
and w =
−82−4
, given that w is a
generalised eigenvector of C .
Subbing y = etCz gives
CetCz + etCz′ = CetCz + te5tw
z′ = te5te−tCw
Rui Tong Second Year Mathematics Revision
Matrix Exponentials Application to Systems of Differential Equations
Solving Non-homogeneous systems of DEs
We need to construct a Jordan chain starting at w first:−82−4
C−5I−−−→
−666
C−5I−−−→
000
so therefore
etCw = e5t
−82−4
+ t
−666
.But observe how we want the negative exponent e−tC ! This meanswhat we’re really interested in is
e−tCw = e−5t
−82−4
− t
−666
Rui Tong Second Year Mathematics Revision
Matrix Exponentials Application to Systems of Differential Equations
Solving Non-homogeneous systems of DEs
Therefore
z′ = te5te−tCw = t
−82−4
− t2
−666
so upon integrating,
z =t2
2
−82−4
− t3
3
−666
+ c
Question: How to deal with that constant of integration?
Rui Tong Second Year Mathematics Revision
Matrix Exponentials Application to Systems of Differential Equations
Solving Non-homogeneous systems of DEs
In general, you can only deal with it when you know what y(0) is,i.e. you have an initial value for the original systems of DEs. Whenthat’s the case, you let z(0) = y(0) to find it.
Here we don’t, so we just proceed as usual.
yP = etCz =t2
2etC
−82−4
− t3
3etC
−666
+ etCc.
Rui Tong Second Year Mathematics Revision
Matrix Exponentials Application to Systems of Differential Equations
Solving Non-homogeneous systems of DEs
To finish this off, we can recycle our Jordan chain from earlier:−82−4
C−5I−−−→
−666
C−5I−−−→
000
.
This gives us
yP =t2
2e5t
−82−4
− t
−666
− t3
3e5t
−666
+ etCc
The remainder is trivial and is left as an exercise to the audience.
Rui Tong Second Year Mathematics Revision
Matrix Exponentials Application to Systems of Differential Equations
Solving Non-homogeneous systems of DEs
To finish this off, we can recycle our Jordan chain from earlier:−82−4
C−5I−−−→
−666
C−5I−−−→
000
.
This gives us
yP =t2
2e5t
−82−4
− t
−666
− t3
3e5t
−666
+ etCc
The remainder is trivial and is left as an exercise to the audience.
Rui Tong Second Year Mathematics Revision
Matrix Exponentials Application to Systems of Differential Equations
Solving Non-homogeneous systems of DEs
Note: The harsh reality is that if we knew what c was, that wouldpotentially be another Jordan chain we need to deal with. Fingerscrossed you don’t have to do that in your exam.
Rui Tong Second Year Mathematics Revision
Final remark: One single eigenvalue
When you’re told that the matrix A only has one eigenvalue, youcan take care of things more easily. The following comments assumen = 3, but the analogy can be adapted for all n × n matrices.
Use the trace to find that eigenvalue λ.
You automatically know that GEλ = C3, so it’s less difficult toconstruct a Jordan chain. Find ker(A− λI ), and onlyker(A− λI )2 if you don’t already have two eigenvectors.
Then, just pick a third vector out of thin air, not linearlyindepedent to the other two. Construct a chain using thatvector.
Done!
You’ll see this in all the examples in your tutorials...
Rui Tong Second Year Mathematics Revision