eigenvalue algorithms: the qr algorithm
TRANSCRIPT
![Page 1: Eigenvalue algorithms: The QR algorithm](https://reader035.vdocuments.net/reader035/viewer/2022062607/62b5cf18a5c5247a4a18a05f/html5/thumbnails/1.jpg)
L20-S00
Eigenvalue algorithms: The QR algorithm
MATH 6610 Lecture 20
October 23, 2020
Trefethen & Bau: Lecture 28
MATH 6610-001 – U. Utah The QR algorithm
![Page 2: Eigenvalue algorithms: The QR algorithm](https://reader035.vdocuments.net/reader035/viewer/2022062607/62b5cf18a5c5247a4a18a05f/html5/thumbnails/2.jpg)
L20-S01Simultaneous power iteration, I
Let p�j , vjqnj“1 be the ordered eigenpairs of A, with |�j | ° |�j`1|.As relatively ineffective as power iteration is, consider applying it to 2 vectorsv, w, which have expansions
v “nÿ
j“1
cjvj , w “Nÿ
j“1
djvj .
For large k, then:
Ak rv ws «ˆ
�k1c1v1 �k
1
„d1v1 `
´�2�1
¯kv2
⇢ ˙
“ pv1 v2q˜
c1�k1 d1�k
1
0 d2´
�2�1
¯k
¸“: QR
MATH 6610-001 – U. Utah The QR algorithm
![Page 3: Eigenvalue algorithms: The QR algorithm](https://reader035.vdocuments.net/reader035/viewer/2022062607/62b5cf18a5c5247a4a18a05f/html5/thumbnails/3.jpg)
![Page 4: Eigenvalue algorithms: The QR algorithm](https://reader035.vdocuments.net/reader035/viewer/2022062607/62b5cf18a5c5247a4a18a05f/html5/thumbnails/4.jpg)
L20-S01Simultaneous power iteration, I
Let p�j , vjqnj“1 be the ordered eigenpairs of A, with |�j | ° |�j`1|.As relatively ineffective as power iteration is, consider applying it to 2 vectorsv, w, which have expansions
v “nÿ
j“1
cjvj , w “Nÿ
j“1
djvj .
For large k, then:
Ak rv ws «ˆ
�k1c1v1 �k
1
„d1v1 `
´�2�1
¯kv2
⇢ ˙
“ pv1 v2q˜
c1�k1 d1�k
1
0 d2´
�2�1
¯k
¸“: QR
MATH 6610-001 – U. Utah The QR algorithm
![Page 5: Eigenvalue algorithms: The QR algorithm](https://reader035.vdocuments.net/reader035/viewer/2022062607/62b5cf18a5c5247a4a18a05f/html5/thumbnails/5.jpg)
L20-S02Simultaneous power iteration, II
Extending this argument, if W is some square, full-rank matrix, then
AkW “ QR « V R,
where V is the eigenvector matrix for A.
We’ll slightly modify simultaneous power iteration: perform orthogonalizationat every step:
Initialize QPI0 “ I. For k “ 0, 1, . . .,
1. APIk`1 :“ AQPI
k
2. QPIk`1R
PIk`1 :“ API
k`1
For large k, we expect QPIk Ñ V . In fact, we can show that if we have the
QR decomposition Ak “ QkRk, then
QPIk RPI
k RPIk´1 ¨ ¨ ¨RPI
1 “ QkRk
So simultaneous power iteration compute Qk implicitly.
MATH 6610-001 – U. Utah The QR algorithm
![Page 6: Eigenvalue algorithms: The QR algorithm](https://reader035.vdocuments.net/reader035/viewer/2022062607/62b5cf18a5c5247a4a18a05f/html5/thumbnails/6.jpg)
L20-S02Simultaneous power iteration, II
Extending this argument, if W is some square, full-rank matrix, then
AkW “ QR « V R,
where V is the eigenvector matrix for A.
We’ll slightly modify simultaneous power iteration: perform orthogonalizationat every step:
Initialize QPI0 “ I. For k “ 0, 1, . . .,
1. APIk`1 :“ AQPI
k
2. QPIk`1R
PIk`1 :“ API
k`1
For large k, we expect QPIk Ñ V .
In fact, we can show that if we have theQR decomposition Ak “ QkRk, then
QPIk RPI
k RPIk´1 ¨ ¨ ¨RPI
1 “ QkRk
So simultaneous power iteration compute Qk implicitly.
MATH 6610-001 – U. Utah The QR algorithm
![Page 7: Eigenvalue algorithms: The QR algorithm](https://reader035.vdocuments.net/reader035/viewer/2022062607/62b5cf18a5c5247a4a18a05f/html5/thumbnails/7.jpg)
![Page 8: Eigenvalue algorithms: The QR algorithm](https://reader035.vdocuments.net/reader035/viewer/2022062607/62b5cf18a5c5247a4a18a05f/html5/thumbnails/8.jpg)
L20-S02Simultaneous power iteration, II
Extending this argument, if W is some square, full-rank matrix, then
AkW “ QR « V R,
where V is the eigenvector matrix for A.
We’ll slightly modify simultaneous power iteration: perform orthogonalizationat every step:
Initialize QPI0 “ I. For k “ 0, 1, . . .,
1. APIk`1 :“ AQPI
k
2. QPIk`1R
PIk`1 :“ API
k`1
For large k, we expect QPIk Ñ V . In fact, we can show that if we have the
QR decomposition Ak “ QkRk, then
QPIk RPI
k RPIk´1 ¨ ¨ ¨RPI
1 “ QkRk
So simultaneous power iteration compute Qk implicitly.MATH 6610-001 – U. Utah The QR algorithm
![Page 9: Eigenvalue algorithms: The QR algorithm](https://reader035.vdocuments.net/reader035/viewer/2022062607/62b5cf18a5c5247a4a18a05f/html5/thumbnails/9.jpg)
L20-S03The QR algorithm
The QR algorithm is a procedure for computing eigenvalues.
(It is distinct from the QR decomposition, but does use QR decompositions.)
The algorithm is so striking that we’ll introduce it first without explanation.
As usual we assume A is Hermitian, so that it has a unitary diagonalization:A “ V ⇤V ˚.
1. Compute A “ QR, the QR decomposition of A2. Replace A by the procedure A – RQ
3. Return to step 1
In the limit of an infinite number of iterations, A converges to ⇤.
At face value, it’s remarkable that this algorithm does anything useful.
In fact, this actually is performing simultaneous power iteration in disguise.
MATH 6610-001 – U. Utah The QR algorithm
![Page 10: Eigenvalue algorithms: The QR algorithm](https://reader035.vdocuments.net/reader035/viewer/2022062607/62b5cf18a5c5247a4a18a05f/html5/thumbnails/10.jpg)
L20-S03The QR algorithm
The QR algorithm is a procedure for computing eigenvalues.
(It is distinct from the QR decomposition, but does use QR decompositions.)
The algorithm is so striking that we’ll introduce it first without explanation.
As usual we assume A is Hermitian, so that it has a unitary diagonalization:A “ V ⇤V ˚.
1. Compute A “ QR, the QR decomposition of A2. Replace A by the procedure A – RQ
3. Return to step 1
In the limit of an infinite number of iterations, A converges to ⇤.
At face value, it’s remarkable that this algorithm does anything useful.
In fact, this actually is performing simultaneous power iteration in disguise.
MATH 6610-001 – U. Utah The QR algorithm
![Page 11: Eigenvalue algorithms: The QR algorithm](https://reader035.vdocuments.net/reader035/viewer/2022062607/62b5cf18a5c5247a4a18a05f/html5/thumbnails/11.jpg)
L20-S03The QR algorithm
The QR algorithm is a procedure for computing eigenvalues.
(It is distinct from the QR decomposition, but does use QR decompositions.)
The algorithm is so striking that we’ll introduce it first without explanation.
As usual we assume A is Hermitian, so that it has a unitary diagonalization:A “ V ⇤V ˚.
1. Compute A “ QR, the QR decomposition of A2. Replace A by the procedure A – RQ
3. Return to step 1
In the limit of an infinite number of iterations, A converges to ⇤.
At face value, it’s remarkable that this algorithm does anything useful.
In fact, this actually is performing simultaneous power iteration in disguise.MATH 6610-001 – U. Utah The QR algorithm
![Page 12: Eigenvalue algorithms: The QR algorithm](https://reader035.vdocuments.net/reader035/viewer/2022062607/62b5cf18a5c5247a4a18a05f/html5/thumbnails/12.jpg)
![Page 13: Eigenvalue algorithms: The QR algorithm](https://reader035.vdocuments.net/reader035/viewer/2022062607/62b5cf18a5c5247a4a18a05f/html5/thumbnails/13.jpg)
L20-S04The QR algorithm and simultaneous power iteration
We can now understand why the QR algorithm and simultaneous poweriteration are performing similar operations.
A useful fact from the QR algorithm is the following relationship:
RQRj QQR
j “ QQj`1RRQ
j`1R, j • 1.
This last relation yields the following result via induction:
Ak “´QQR
1 QQR2 ¨ ¨ ¨QQR
k
¯ ´RQR
k RQRk´1 ¨ ¨ ¨RQR
1
¯
MATH 6610-001 – U. Utah The QR algorithm
![Page 14: Eigenvalue algorithms: The QR algorithm](https://reader035.vdocuments.net/reader035/viewer/2022062607/62b5cf18a5c5247a4a18a05f/html5/thumbnails/14.jpg)
L20-S04The QR algorithm and simultaneous power iteration
We can now understand why the QR algorithm and simultaneous poweriteration are performing similar operations.
A useful fact from the QR algorithm is the following relationship:
RQRj QQR
j “ QQj`1RRQ
j`1R, j • 1.
This last relation yields the following result via induction:
Ak “´QQR
1 QQR2 ¨ ¨ ¨QQR
k
¯ ´RQR
k RQRk´1 ¨ ¨ ¨RQR
1
¯
MATH 6610-001 – U. Utah The QR algorithm
![Page 15: Eigenvalue algorithms: The QR algorithm](https://reader035.vdocuments.net/reader035/viewer/2022062607/62b5cf18a5c5247a4a18a05f/html5/thumbnails/15.jpg)
L20-S04The QR algorithm and simultaneous power iteration
We can now understand why the QR algorithm and simultaneous poweriteration are performing similar operations.
A useful fact from the QR algorithm is the following relationship:
RQRj QQR
j “ QQj`1RRQ
j`1R, j • 1.
This last relation yields the following result via induction:
Ak “´QQR
1 QQR2 ¨ ¨ ¨QQR
k
¯ ´RQR
k RQRk´1 ¨ ¨ ¨RQR
1
¯
MATH 6610-001 – U. Utah The QR algorithm
![Page 16: Eigenvalue algorithms: The QR algorithm](https://reader035.vdocuments.net/reader035/viewer/2022062607/62b5cf18a5c5247a4a18a05f/html5/thumbnails/16.jpg)
![Page 17: Eigenvalue algorithms: The QR algorithm](https://reader035.vdocuments.net/reader035/viewer/2022062607/62b5cf18a5c5247a4a18a05f/html5/thumbnails/17.jpg)
L20-S05The QR algorithm’s QR decomposition
Finally, we can uncover what the QR algorithm is doing since we haveuncovered two QR decompositions of A:
Ak “´QQR
1 QQR2 ¨ ¨ ¨QQR
k
¯ ´RQR
k RQRk´1 ¨ ¨ ¨RQR
1
¯
Ak “ QPIk
`RPI
k RPIk´1 ¨ ¨ ¨RPI
1
˘
Therefore: with A “ V ⇤V ˚ the eigenvalue decomposition of A, then:´QQR
1 QQR2 ¨ ¨ ¨QQR
k
¯“ QPI
kkÒ8›Ñ V.
Great, but the QR algorithm doesn’t compute the entire matrix,´QQR
1 QQR2 ¨ ¨ ¨QQR
k
¯,
it just computes QQRk .
MATH 6610-001 – U. Utah The QR algorithm
![Page 18: Eigenvalue algorithms: The QR algorithm](https://reader035.vdocuments.net/reader035/viewer/2022062607/62b5cf18a5c5247a4a18a05f/html5/thumbnails/18.jpg)
L20-S05The QR algorithm’s QR decomposition
Finally, we can uncover what the QR algorithm is doing since we haveuncovered two QR decompositions of A:
Ak “´QQR
1 QQR2 ¨ ¨ ¨QQR
k
¯ ´RQR
k RQRk´1 ¨ ¨ ¨RQR
1
¯
Ak “ QPIk
`RPI
k RPIk´1 ¨ ¨ ¨RPI
1
˘
Therefore: with A “ V ⇤V ˚ the eigenvalue decomposition of A, then:´QQR
1 QQR2 ¨ ¨ ¨QQR
k
¯“ QPI
kkÒ8›Ñ V.
Great, but the QR algorithm doesn’t compute the entire matrix,´QQR
1 QQR2 ¨ ¨ ¨QQR
k
¯,
it just computes QQRk .
MATH 6610-001 – U. Utah The QR algorithm
![Page 19: Eigenvalue algorithms: The QR algorithm](https://reader035.vdocuments.net/reader035/viewer/2022062607/62b5cf18a5c5247a4a18a05f/html5/thumbnails/19.jpg)
L20-S05The QR algorithm’s QR decomposition
Finally, we can uncover what the QR algorithm is doing since we haveuncovered two QR decompositions of A:
Ak “´QQR
1 QQR2 ¨ ¨ ¨QQR
k
¯ ´RQR
k RQRk´1 ¨ ¨ ¨RQR
1
¯
Ak “ QPIk
`RPI
k RPIk´1 ¨ ¨ ¨RPI
1
˘
Therefore: with A “ V ⇤V ˚ the eigenvalue decomposition of A, then:´QQR
1 QQR2 ¨ ¨ ¨QQR
k
¯“ QPI
kkÒ8›Ñ V.
Great, but the QR algorithm doesn’t compute the entire matrix,´QQR
1 QQR2 ¨ ¨ ¨QQR
k
¯,
it just computes QQRk .
MATH 6610-001 – U. Utah The QR algorithm
![Page 20: Eigenvalue algorithms: The QR algorithm](https://reader035.vdocuments.net/reader035/viewer/2022062607/62b5cf18a5c5247a4a18a05f/html5/thumbnails/20.jpg)
L20-S06The QR algorithm computes eigenvalues
What does the QR algorithm compute? Basically just AQRk .
The final property to note is that AQRk is unitarily equivalent to A:
AQRk “
´QQR
1 QQR2 ¨ ¨ ¨QQR
k
¯˚A
´QQR
1 QQR2 ¨ ¨ ¨QQR
k
¯.
But we know that´QQR
1 QQR2 ¨ ¨ ¨QQR
k
¯Ñ V . Therefore:
AQRk “
´QQR
1 QQR2 ¨ ¨ ¨QQR
k
¯˚A
´QQR
1 QQR2 ¨ ¨ ¨QQR
k
¯
kÒ8Ñ V ˚AV “ ⇤.
MATH 6610-001 – U. Utah The QR algorithm
![Page 21: Eigenvalue algorithms: The QR algorithm](https://reader035.vdocuments.net/reader035/viewer/2022062607/62b5cf18a5c5247a4a18a05f/html5/thumbnails/21.jpg)
L20-S06The QR algorithm computes eigenvalues
What does the QR algorithm compute? Basically just AQRk .
The final property to note is that AQRk is unitarily equivalent to A:
AQRk “
´QQR
1 QQR2 ¨ ¨ ¨QQR
k
¯˚A
´QQR
1 QQR2 ¨ ¨ ¨QQR
k
¯.
But we know that´QQR
1 QQR2 ¨ ¨ ¨QQR
k
¯Ñ V . Therefore:
AQRk “
´QQR
1 QQR2 ¨ ¨ ¨QQR
k
¯˚A
´QQR
1 QQR2 ¨ ¨ ¨QQR
k
¯
kÒ8Ñ V ˚AV “ ⇤.
MATH 6610-001 – U. Utah The QR algorithm
![Page 22: Eigenvalue algorithms: The QR algorithm](https://reader035.vdocuments.net/reader035/viewer/2022062607/62b5cf18a5c5247a4a18a05f/html5/thumbnails/22.jpg)
L20-S06The QR algorithm computes eigenvalues
What does the QR algorithm compute? Basically just AQRk .
The final property to note is that AQRk is unitarily equivalent to A:
AQRk “
´QQR
1 QQR2 ¨ ¨ ¨QQR
k
¯˚A
´QQR
1 QQR2 ¨ ¨ ¨QQR
k
¯.
But we know that´QQR
1 QQR2 ¨ ¨ ¨QQR
k
¯Ñ V . Therefore:
AQRk “
´QQR
1 QQR2 ¨ ¨ ¨QQR
k
¯˚A
´QQR
1 QQR2 ¨ ¨ ¨QQR
k
¯
kÒ8Ñ V ˚AV “ ⇤.
MATH 6610-001 – U. Utah The QR algorithm
![Page 23: Eigenvalue algorithms: The QR algorithm](https://reader035.vdocuments.net/reader035/viewer/2022062607/62b5cf18a5c5247a4a18a05f/html5/thumbnails/23.jpg)
L20-S07The QR algorithm and convergence
The QR algorithm therefore performs an eigenvalue decomposition. For real,symmetric matrices, we have convergence
AQRk Ñ ⇤,
with error ck, where c depends on the ratio of magnitude of consecutive(ordered) eigenvalues.
In fact: the real symmetric assumption is not necessary. For (fairly) generalmatrices A, the QR algorithm computes AQR
k that converges to the Schurfactor T in the Schur decomposition
A “ QTQ˚.
MATH 6610-001 – U. Utah The QR algorithm
![Page 24: Eigenvalue algorithms: The QR algorithm](https://reader035.vdocuments.net/reader035/viewer/2022062607/62b5cf18a5c5247a4a18a05f/html5/thumbnails/24.jpg)
L20-S07The QR algorithm and convergence
The QR algorithm therefore performs an eigenvalue decomposition. For real,symmetric matrices, we have convergence
AQRk Ñ ⇤,
with error ck, where c depends on the ratio of magnitude of consecutive(ordered) eigenvalues.
In fact: the real symmetric assumption is not necessary. For (fairly) generalmatrices A, the QR algorithm computes AQR
k that converges to the Schurfactor T in the Schur decomposition
A “ QTQ˚.
MATH 6610-001 – U. Utah The QR algorithm