Gaussian elimination revisited
βΆ Example. Keeping track of the elementary matrices duringGaussian elimination on π΄:
π΄ = β‘β’β£2 14 β6
β€β₯β¦
πΈπ΄ = β‘β’β£
1 0β2 1
β€β₯β¦
β‘β’β£2 14 β6
β€β₯β¦
= β‘β’β£2 10 β8
β€β₯β¦
.
Note thatπ΄ = πΈβ1 β‘β’
β£2 10 β8
β€β₯β¦
= β‘β’β£1 02 1
β€β₯β¦
β‘β’β£2 10 β8
β€β₯β¦
We factored π΄ as the product of a lower and upper triangularmatrix! We say that π΄ has triangular factorization.
1
Gaussian elimination revisited
βΆ Example. Keeping track of the elementary matrices duringGaussian elimination on π΄:
π΄ = β‘β’β£2 14 β6
β€β₯β¦
πΈπ΄ = β‘β’β£
1 0β2 1
β€β₯β¦
β‘β’β£2 14 β6
β€β₯β¦
= β‘β’β£2 10 β8
β€β₯β¦
.
Note thatπ΄ = πΈβ1 β‘β’
β£2 10 β8
β€β₯β¦
= β‘β’β£1 02 1
β€β₯β¦
β‘β’β£2 10 β8
β€β₯β¦
We factored π΄ as the product of a lower and upper triangularmatrix! We say that π΄ has triangular factorization.
1
Gaussian elimination revisited
βΆ Example. Keeping track of the elementary matrices duringGaussian elimination on π΄:
π΄ = β‘β’β£2 14 β6
β€β₯β¦
πΈπ΄ = β‘β’β£
1 0β2 1
β€β₯β¦
β‘β’β£2 14 β6
β€β₯β¦
= β‘β’β£2 10 β8
β€β₯β¦
.
Note thatπ΄ = πΈβ1 β‘β’
β£2 10 β8
β€β₯β¦
= β‘β’β£1 02 1
β€β₯β¦
β‘β’β£2 10 β8
β€β₯β¦
We factored π΄ as the product of a lower and upper triangularmatrix! We say that π΄ has triangular factorization.
1
Gaussian elimination revisited
βΆ Example. Keeping track of the elementary matrices duringGaussian elimination on π΄:
π΄ = β‘β’β£2 14 β6
β€β₯β¦
πΈπ΄ = β‘β’β£
1 0β2 1
β€β₯β¦
β‘β’β£2 14 β6
β€β₯β¦
= β‘β’β£2 10 β8
β€β₯β¦
.
Note thatπ΄ = πΈβ1 β‘β’
β£2 10 β8
β€β₯β¦
= β‘β’β£1 02 1
β€β₯β¦
β‘β’β£2 10 β8
β€β₯β¦
We factored π΄ as the product of a lower and upper triangularmatrix! We say that π΄ has triangular factorization.
1
Gaussian elimination revisited
π΄ = πΏπ is known as the LU decomposition of π΄.
βΆ Definition.lower triangular
β‘β’β’β’β’β’β’β£
β 0 0 0 0β β 0 0 0β β β 0 0β β β β 0β β β β β
β€β₯β₯β₯β₯β₯β₯β¦
upper triangular
β‘β’β’β’β’β’β’β£
β β β β β0 β β β β0 0 β β β0 0 0 β β0 0 0 0 β
β€β₯β₯β₯β₯β₯β₯β¦
2
Gaussian elimination revisited
π΄ = πΏπ is known as the LU decomposition of π΄.
βΆ Definition.lower triangular
β‘β’β’β’β’β’β’β£
β 0 0 0 0β β 0 0 0β β β 0 0β β β β 0β β β β β
β€β₯β₯β₯β₯β₯β₯β¦
upper triangular
β‘β’β’β’β’β’β’β£
β β β β β0 β β β β0 0 β β β0 0 0 β β0 0 0 0 β
β€β₯β₯β₯β₯β₯β₯β¦
2
πΏπ decompostion
βΆ Example. Factor π΄ =β‘β’β’β£
2 1 14 β6 0
β2 7 2
β€β₯β₯β¦
as π΄ = πΏπ.
βΆ Solution.
πΈ1π΄ =β‘β’β’β£
1 0 0β2 1 00 0 1
β€β₯β₯β¦
β‘β’β’β£
2 1 14 β6 0
β2 7 2
β€β₯β₯β¦
=β‘β’β’β£
2 1 10 β8 β2
β2 7 2
β€β₯β₯β¦
πΈ2(πΈ1π΄) =β‘β’β’β£
1 0 00 1 01 0 1
β€β₯β₯β¦
β‘β’β’β£
2 1 10 β8 β2
β2 7 2
β€β₯β₯β¦
=β‘β’β’β£
2 1 10 β8 β20 8 3
β€β₯β₯β¦
πΈ3πΈ2πΈ1π΄ =β‘β’β’β£
1 0 00 1 00 1 1
β€β₯β₯β¦
β‘β’β’β£
2 1 10 β8 β20 8 3
β€β₯β₯β¦
=β‘β’β’β£
2 1 10 β8 β20 0 1
β€β₯β₯β¦
= π
3
πΏπ decompostion
βΆ Example. Factor π΄ =β‘β’β’β£
2 1 14 β6 0
β2 7 2
β€β₯β₯β¦
as π΄ = πΏπ.
βΆ Solution.
πΈ1π΄ =β‘β’β’β£
1 0 0β2 1 00 0 1
β€β₯β₯β¦
β‘β’β’β£
2 1 14 β6 0
β2 7 2
β€β₯β₯β¦
=β‘β’β’β£
2 1 10 β8 β2
β2 7 2
β€β₯β₯β¦
πΈ2(πΈ1π΄) =β‘β’β’β£
1 0 00 1 01 0 1
β€β₯β₯β¦
β‘β’β’β£
2 1 10 β8 β2
β2 7 2
β€β₯β₯β¦
=β‘β’β’β£
2 1 10 β8 β20 8 3
β€β₯β₯β¦
πΈ3πΈ2πΈ1π΄ =β‘β’β’β£
1 0 00 1 00 1 1
β€β₯β₯β¦
β‘β’β’β£
2 1 10 β8 β20 8 3
β€β₯β₯β¦
=β‘β’β’β£
2 1 10 β8 β20 0 1
β€β₯β₯β¦
= π
3
πΏπ decompostion
βΆ Example. Factor π΄ =β‘β’β’β£
2 1 14 β6 0
β2 7 2
β€β₯β₯β¦
as π΄ = πΏπ.
βΆ Solution.
πΈ1π΄ =β‘β’β’β£
1 0 0β2 1 00 0 1
β€β₯β₯β¦
β‘β’β’β£
2 1 14 β6 0
β2 7 2
β€β₯β₯β¦
=β‘β’β’β£
2 1 10 β8 β2
β2 7 2
β€β₯β₯β¦
πΈ2(πΈ1π΄) =β‘β’β’β£
1 0 00 1 01 0 1
β€β₯β₯β¦
β‘β’β’β£
2 1 10 β8 β2
β2 7 2
β€β₯β₯β¦
=β‘β’β’β£
2 1 10 β8 β20 8 3
β€β₯β₯β¦
πΈ3πΈ2πΈ1π΄ =β‘β’β’β£
1 0 00 1 00 1 1
β€β₯β₯β¦
β‘β’β’β£
2 1 10 β8 β20 8 3
β€β₯β₯β¦
=β‘β’β’β£
2 1 10 β8 β20 0 1
β€β₯β₯β¦
= π
3
πΏπ decompostion
βΆ Example. Factor π΄ =β‘β’β’β£
2 1 14 β6 0
β2 7 2
β€β₯β₯β¦
as π΄ = πΏπ.
βΆ Solution.
πΈ1π΄ =β‘β’β’β£
1 0 0β2 1 00 0 1
β€β₯β₯β¦
β‘β’β’β£
2 1 14 β6 0
β2 7 2
β€β₯β₯β¦
=β‘β’β’β£
2 1 10 β8 β2
β2 7 2
β€β₯β₯β¦
πΈ2(πΈ1π΄) =β‘β’β’β£
1 0 00 1 01 0 1
β€β₯β₯β¦
β‘β’β’β£
2 1 10 β8 β2
β2 7 2
β€β₯β₯β¦
=β‘β’β’β£
2 1 10 β8 β20 8 3
β€β₯β₯β¦
πΈ3πΈ2πΈ1π΄ =β‘β’β’β£
1 0 00 1 00 1 1
β€β₯β₯β¦
β‘β’β’β£
2 1 10 β8 β20 8 3
β€β₯β₯β¦
=β‘β’β’β£
2 1 10 β8 β20 0 1
β€β₯β₯β¦
= π
3
πΏπ decompostion
πΈ3πΈ2πΈ1π΄ = π
βΉ π΄ = πΈβ11 πΈβ1
2 πΈβ13 π
The factor πΏ is given by
πΏ = πΈβ11 πΈβ1
2 πΈβ13
=β‘β’β’β£
1 0 02 1 00 0 1
β€β₯β₯β¦
β‘β’β’β£
1 0 00 1 0
β1 0 1
β€β₯β₯β¦
β‘β’β’β£
1 0 00 1 00 β1 1
β€β₯β₯β¦
=β‘β’β’β£
1 0 02 1 0
β1 β1 1
β€β₯β₯β¦
4
πΏπ decompostion
πΈ3πΈ2πΈ1π΄ = π βΉ π΄ = πΈβ11 πΈβ1
2 πΈβ13 π
The factor πΏ is given by
πΏ = πΈβ11 πΈβ1
2 πΈβ13
=β‘β’β’β£
1 0 02 1 00 0 1
β€β₯β₯β¦
β‘β’β’β£
1 0 00 1 0
β1 0 1
β€β₯β₯β¦
β‘β’β’β£
1 0 00 1 00 β1 1
β€β₯β₯β¦
=β‘β’β’β£
1 0 02 1 0
β1 β1 1
β€β₯β₯β¦
4
πΏπ decompostion
πΈ3πΈ2πΈ1π΄ = π βΉ π΄ = πΈβ11 πΈβ1
2 πΈβ13 π
The factor πΏ is given by
πΏ = πΈβ11 πΈβ1
2 πΈβ13
=β‘β’β’β£
1 0 02 1 00 0 1
β€β₯β₯β¦
β‘β’β’β£
1 0 00 1 0
β1 0 1
β€β₯β₯β¦
β‘β’β’β£
1 0 00 1 00 β1 1
β€β₯β₯β¦
=β‘β’β’β£
1 0 02 1 0
β1 β1 1
β€β₯β₯β¦
4
πΏπ decompostion
We found the following πΏπ decomposition of π΄:
π΄ =β‘β’β’β£
2 1 14 β6 0
β2 7 2
β€β₯β₯β¦
= πΏπ =β‘β’β’β£
1 0 02 1 0
β1 β1 1
β€β₯β₯β¦
β‘β’β’β£
2 1 10 β8 β20 0 1
β€β₯β₯β¦
.
5
Why πΏπ decomposition?Once we have π΄ = πΏπ, it is simple to solve π΄π₯π₯π₯ = πππ.
π΄π₯π₯π₯ = ππππΏ(ππ₯π₯π₯) = πππ
πΏπππ = πππ and ππ₯π₯π₯ = πππ.Both of the final systems are triangular and hence easily solved:
β’ πΏπππ = πππ by forward substitution to find πππ, and thenβ’ ππ₯π₯π₯ = πππ by backward substitution to find π₯π₯π₯.
βΆ Example. Solve
β‘β’β’β£
2 1 14 β6 0
β2 7 2
β€β₯β₯β¦
π₯π₯π₯ =β‘β’β’β£
410β3
β€β₯β₯β¦
.
6
Why πΏπ decomposition?Once we have π΄ = πΏπ, it is simple to solve π΄π₯π₯π₯ = πππ.
π΄π₯π₯π₯ = ππππΏ(ππ₯π₯π₯) = πππ
πΏπππ = πππ and ππ₯π₯π₯ = πππ.Both of the final systems are triangular and hence easily solved:
β’ πΏπππ = πππ by forward substitution to find πππ, and thenβ’ ππ₯π₯π₯ = πππ by backward substitution to find π₯π₯π₯.
βΆ Example. Solve
β‘β’β’β£
2 1 14 β6 0
β2 7 2
β€β₯β₯β¦
π₯π₯π₯ =β‘β’β’β£
410β3
β€β₯β₯β¦
.
6
Why πΏπ decomposition?Once we have π΄ = πΏπ, it is simple to solve π΄π₯π₯π₯ = πππ.
π΄π₯π₯π₯ = ππππΏ(ππ₯π₯π₯) = πππ
πΏπππ = πππ and ππ₯π₯π₯ = πππ.Both of the final systems are triangular and hence easily solved:
β’ πΏπππ = πππ by forward substitution to find πππ, and thenβ’ ππ₯π₯π₯ = πππ by backward substitution to find π₯π₯π₯.
βΆ Example. Solve
β‘β’β’β£
2 1 14 β6 0
β2 7 2
β€β₯β₯β¦
π₯π₯π₯ =β‘β’β’β£
410β3
β€β₯β₯β¦
.
6
The inverse of a matrix
βΆ Definition. An π Γ π matrix π΄ is invertible if there is a matrix π΅such that
π΄π΅ = π΅π΄ = πΌπΓπ.
In that case, π΅ is the inverse of π΄ and is denoted by π΄β1.
βΆ Remark.
β’ The inverse of a matrix is unique. (Why?)β’ Do not write π΄
π΅ .
7
The inverse of a matrix
βΆ Definition. An π Γ π matrix π΄ is invertible if there is a matrix π΅such that
π΄π΅ = π΅π΄ = πΌπΓπ.
In that case, π΅ is the inverse of π΄ and is denoted by π΄β1.βΆ Remark.
β’ The inverse of a matrix is unique. (Why?)
β’ Do not write π΄π΅ .
7
The inverse of a matrix
βΆ Definition. An π Γ π matrix π΄ is invertible if there is a matrix π΅such that
π΄π΅ = π΅π΄ = πΌπΓπ.
In that case, π΅ is the inverse of π΄ and is denoted by π΄β1.βΆ Remark.
β’ The inverse of a matrix is unique. (Why?)β’ Do not write π΄
π΅ .
7
The inverse of a matrix
βΆ Example. Let π΄ = β‘β’β£π ππ π
β€β₯β¦
. If ππ β ππ β 0, then
π΄β1 = 1ππ β ππ
β‘β’β£
π βπβπ π
β€β₯β¦
.
βΆ Example. The matrix π΄ = β‘β’β£0 10 0
β€β₯β¦
is not invertible.
βΆ Example. A 2 Γ 2 matrix β‘β’β£π ππ π
β€β₯β¦
is invertible if and only if
ππ β ππ β 0.
8
The inverse of a matrix
βΆ Example. Let π΄ = β‘β’β£π ππ π
β€β₯β¦
. If ππ β ππ β 0, then
π΄β1 = 1ππ β ππ
β‘β’β£
π βπβπ π
β€β₯β¦
.
βΆ Example. The matrix π΄ = β‘β’β£0 10 0
β€β₯β¦
is not invertible.
βΆ Example. A 2 Γ 2 matrix β‘β’β£π ππ π
β€β₯β¦
is invertible if and only if
ππ β ππ β 0.
8
The inverse of a matrix
βΆ Example. Let π΄ = β‘β’β£π ππ π
β€β₯β¦
. If ππ β ππ β 0, then
π΄β1 = 1ππ β ππ
β‘β’β£
π βπβπ π
β€β₯β¦
.
βΆ Example. The matrix π΄ = β‘β’β£0 10 0
β€β₯β¦
is not invertible.
βΆ Example. A 2 Γ 2 matrix β‘β’β£π ππ π
β€β₯β¦
is invertible if and only if
ππ β ππ β 0.
8
The inverse of a matrix
Suppose π΄ and π΅ are invertible. Then
β’ π΄β1 is invertible and (π΄β1)β1 = π΄.
β’ π΄π is invertible and (π΄π )β1 = (π΄β1)π .β’ π΄π΅ is invertible and (π΄π΅)β1 = π΅β1π΄β1. (Why?)
9
The inverse of a matrix
Suppose π΄ and π΅ are invertible. Then
β’ π΄β1 is invertible and (π΄β1)β1 = π΄.β’ π΄π is invertible and (π΄π )β1 = (π΄β1)π .
β’ π΄π΅ is invertible and (π΄π΅)β1 = π΅β1π΄β1. (Why?)
9
The inverse of a matrix
Suppose π΄ and π΅ are invertible. Then
β’ π΄β1 is invertible and (π΄β1)β1 = π΄.β’ π΄π is invertible and (π΄π )β1 = (π΄β1)π .β’ π΄π΅ is invertible and (π΄π΅)β1 = π΅β1π΄β1. (Why?)
9
Solving systems using matrix inverse
Theorem. Let π΄ be invertible. Then the system π΄π₯π₯π₯ = πππ has theunique solution π₯π₯π₯ = π΄β1πππ.
10
Computing the inverse
βΆ To solve π΄π₯π₯π₯ = πππ, we do row reduction on [ π΄ β£ π ].
βΆ To solve π΄π = πΌ, we do row reduction on [ π΄ β£ πΌ ].βΆ To compute π΄β1 (The Gauss-Jordan Method):
β’ Form the augmented matrix [ π΄ β£ πΌ ].β’ Compute the reduced echelon form.β’ If π΄ is invertible, the result is of the form [ πΌ β£ π΄β1 ].
11
Computing the inverse
βΆ To solve π΄π₯π₯π₯ = πππ, we do row reduction on [ π΄ β£ π ].βΆ To solve π΄π = πΌ, we do row reduction on [ π΄ β£ πΌ ].
βΆ To compute π΄β1 (The Gauss-Jordan Method):
β’ Form the augmented matrix [ π΄ β£ πΌ ].β’ Compute the reduced echelon form.β’ If π΄ is invertible, the result is of the form [ πΌ β£ π΄β1 ].
11
Computing the inverse
βΆ To solve π΄π₯π₯π₯ = πππ, we do row reduction on [ π΄ β£ π ].βΆ To solve π΄π = πΌ, we do row reduction on [ π΄ β£ πΌ ].βΆ To compute π΄β1 (The Gauss-Jordan Method):
β’ Form the augmented matrix [ π΄ β£ πΌ ].β’ Compute the reduced echelon form.β’ If π΄ is invertible, the result is of the form [ πΌ β£ π΄β1 ].
11
Computing the inverse
βΆ Example. Find the inverse of π΄ =β‘β’β’β£
2 1 14 β6 0
β2 7 2
β€β₯β₯β¦, if it exists.
βΆ Solution.
β‘β’β’β£
2 1 1 1 0 04 β6 0 0 1 0
β2 7 2 0 0 1
β€β₯β₯β¦
π 2βΆπ 2β2π 1βββββββββββπ 3βΆπ 3+π 1
β‘β’β’β£
2 1 1 1 0 00 β8 β2 β2 1 00 8 3 1 0 1
β€β₯β₯β¦
π 3βΆπ 3+π 2βββββββββββ‘β’β’β£
2 1 1 1 0 00 β8 β2 β2 1 00 0 1 β1 1 1
β€β₯β₯β¦
βββββββ β¦
12
Computing the inverse
βΆ Example. Find the inverse of π΄ =β‘β’β’β£
2 1 14 β6 0
β2 7 2
β€β₯β₯β¦, if it exists.
βΆ Solution.
β‘β’β’β£
2 1 1 1 0 04 β6 0 0 1 0
β2 7 2 0 0 1
β€β₯β₯β¦
π 2βΆπ 2β2π 1βββββββββββπ 3βΆπ 3+π 1
β‘β’β’β£
2 1 1 1 0 00 β8 β2 β2 1 00 8 3 1 0 1
β€β₯β₯β¦
π 3βΆπ 3+π 2βββββββββββ‘β’β’β£
2 1 1 1 0 00 β8 β2 β2 1 00 0 1 β1 1 1
β€β₯β₯β¦
βββββββ β¦
12
Computing the inverse
βΆ Example. Find the inverse of π΄ =β‘β’β’β£
2 1 14 β6 0
β2 7 2
β€β₯β₯β¦, if it exists.
βΆ Solution.
β‘β’β’β£
2 1 1 1 0 04 β6 0 0 1 0
β2 7 2 0 0 1
β€β₯β₯β¦
π 2βΆπ 2β2π 1βββββββββββπ 3βΆπ 3+π 1
β‘β’β’β£
2 1 1 1 0 00 β8 β2 β2 1 00 8 3 1 0 1
β€β₯β₯β¦
π 3βΆπ 3+π 2βββββββββββ‘β’β’β£
2 1 1 1 0 00 β8 β2 β2 1 00 0 1 β1 1 1
β€β₯β₯β¦
βββββββ β¦
12
Computing the inverse
βΆ Example. Find the inverse of π΄ =β‘β’β’β£
2 1 14 β6 0
β2 7 2
β€β₯β₯β¦, if it exists.
βΆ Solution.
β‘β’β’β£
2 1 1 1 0 04 β6 0 0 1 0
β2 7 2 0 0 1
β€β₯β₯β¦
π 2βΆπ 2β2π 1βββββββββββπ 3βΆπ 3+π 1
β‘β’β’β£
2 1 1 1 0 00 β8 β2 β2 1 00 8 3 1 0 1
β€β₯β₯β¦
π 3βΆπ 3+π 2βββββββββββ‘β’β’β£
2 1 1 1 0 00 β8 β2 β2 1 00 0 1 β1 1 1
β€β₯β₯β¦
βββββββ β¦
12
Computing the inverse
βΆ Example. Find the inverse of π΄ =β‘β’β’β£
2 1 14 β6 0
β2 7 2
β€β₯β₯β¦, if it exists.
βΆ Solution.
β‘β’β’β£
2 1 1 1 0 04 β6 0 0 1 0
β2 7 2 0 0 1
β€β₯β₯β¦
π 2βΆπ 2β2π 1βββββββββββπ 3βΆπ 3+π 1
β‘β’β’β£
2 1 1 1 0 00 β8 β2 β2 1 00 8 3 1 0 1
β€β₯β₯β¦
π 3βΆπ 3+π 2βββββββββββ‘β’β’β£
2 1 1 1 0 00 β8 β2 β2 1 00 0 1 β1 1 1
β€β₯β₯β¦
βββββββ β¦
12
Computing the inverse
ββββββββ‘β’β’β£
1 0 0 1216
β516
β616
0 1 0 48
β38
β28
0 0 1 β1 1 1
β€β₯β₯β¦
π΄β1 =β‘β’β’β£
1216
β516
β616
48
β38
β28
β1 1 1
β€β₯β₯β¦
13
Why does it work?
β’ Each row reduction corresponds to multiplying with anelementary matrix πΈ:
[ π΄β£ πΌ ] β [ πΈ1π΄β£ πΈ1πΌ ] β [ πΈ2πΈ1π΄β£ πΈ2πΈ1 ] β β¦
β¦ β [ πΉπ΄ β£ πΉ] where πΉ = πΈπ β¦ πΈ2πΈ1.
β’ If we manage to reduce [ π΄β£ πΌ ] to [ πΌβ£ πΉ ], this means
πΉπ΄ = πΌ βΉ π΄β1 = πΉ.
14
Why does it work?
β’ Each row reduction corresponds to multiplying with anelementary matrix πΈ:
[ π΄β£ πΌ ] β [ πΈ1π΄β£ πΈ1πΌ ] β [ πΈ2πΈ1π΄β£ πΈ2πΈ1 ] β β¦
β¦ β [ πΉπ΄ β£ πΉ] where πΉ = πΈπ β¦ πΈ2πΈ1.
β’ If we manage to reduce [ π΄β£ πΌ ] to [ πΌβ£ πΉ ], this means
πΉπ΄ = πΌ βΉ π΄β1 = πΉ.
14
Why does it work?
β’ Each row reduction corresponds to multiplying with anelementary matrix πΈ:
[ π΄β£ πΌ ] β [ πΈ1π΄β£ πΈ1πΌ ] β [ πΈ2πΈ1π΄β£ πΈ2πΈ1 ] β β¦
β¦ β [ πΉπ΄ β£ πΉ] where πΉ = πΈπ β¦ πΈ2πΈ1.
β’ If we manage to reduce [ π΄β£ πΌ ] to [ πΌβ£ πΉ ], this means
πΉπ΄ = πΌ βΉ π΄β1 = πΉ.
14