uppsala, april 2014 subspace methodssleij101/uppsala/lecture4.pdf · shift & invert uk+1 = (a...
TRANSCRIPT
![Page 1: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/1.jpg)
Uppsala, April 2014
Subspace methods
Gerard SleijpenDepartment of Mathematics
http://www.staff.science.uu.nl/∼sleij101/
![Page 2: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/2.jpg)
Ax = b Ax = λx
Subspace methods
Iterate until sufficiently accurate:
• Expansion. Expand the search subspace Vk.Restart if dim(Vk) is too large.
• Extraction. Extract an appropriate approximatesolution (ϑ,u) from the search subspace.
Practical aspects.
Construct basis v1, . . . ,vk of the search subspace Vk.
Notation. Vk = [v1, . . . , vk]; Vk = span(Vk).
• Compute expansion vector t, “orthogonalise” to vk+1.orthogonalize, A-orthogonalize, bi-orthogonalize, . . .
• Approximate solution uk ≡ Vk yk; compute yk.
![Page 3: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/3.jpg)
Program Lecture 4
• Expansion
◦ Krylov subspace approach
Lanczos, Arnoldi, Shift and Invert Arnoldi
◦ Convergence
◦ Accelerated Rayleigh Quotient Iteration
Rational Krylov Sequence method
◦ Optimal expansion
Jacobi-Davidson
• Restart
LOCG, Implicitly Restarted Arnoldi Method
• Deflation
![Page 4: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/4.jpg)
Program Lecture 4
• Expansion
◦ Krylov subspace approach
Lanczos, Arnoldi, Shift and Invert Arnoldi
◦ Convergence
◦ Accelerated Rayleigh Quotient Iteration
Rational Krylov Sequence method
◦ Optimal expansion
Jacobi-Davidson
• Restart
LOCG, Implicitly Restarted Arnoldi Method
• Deflation
![Page 5: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/5.jpg)
Krylov subspace expansion
The columns of Vk = [v1, . . . ,vk] form a (orthonormal)
Krylov basis: then t = Avk is an expansion vector.
Examples. For Ax = λx.
• Arnoldi ((harmonic) Ritz extraction).
Lanczos in case A is Hermitian.
![Page 6: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/6.jpg)
Krylov subspace expansion
The columns of Vk = [v1, . . . ,vk] form a (orthonormal)
Krylov basis: then t = Avk is an expansion vector.
Examples. For Ax = λx.
• Arnoldi ((harmonic) Ritz extraction).
Lanczos in case A is Hermitian.
• Shift and Invert Arnoldi:
t = (A− τ I)−1vk to generate a basis
for the search subspace Kk((A− τ I)−1,v1).
Shift and Invert Lanczos for the Hermitian case
![Page 7: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/7.jpg)
Krylov subspace expansion
The columns of Vk = [v1, . . . ,vk] form a (orthonormal)
Krylov basis: then t = Avk is an expansion vector.
Examples. For Ax = λx.
• Arnoldi ((harmonic) Ritz extraction).
Lanczos in case A is Hermitian.
• Shift and Invert Arnoldi:
t = (A− τ I)−1vk to generate a basis
for the search subspace Kk((A− τ I)−1,v1).
SI expansion amplifies components of eigenvectors with
eigenvalue close to the target τ .
![Page 8: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/8.jpg)
Program Lecture 4
• Expansion
◦ Krylov subspace approach
Lanczos, Arnoldi, Shift and Invert Arnoldi
◦ Convergence
◦ Accelerated Rayleigh Quotient Iteration
Rational Krylov Sequence method
◦ Optimal expansion
Jacobi-Davidson
• Restart
LOCG, Implicitly Restarted Arnoldi Method
• Deflation
![Page 9: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/9.jpg)
Convergence without subspace acceleration
Shift & Invert uk+1 = (A− τ I)−1uk. (s&i)
Linear convergence towards eigenvector with eigenvalue
closest to τ .
Rayleigh Quotient Iteration
uk+1 = (A− ρk I)−1uk, where ρk ≡ ρ(uk).
Asymptotic quadratic convergence towards an eigenvector.
Asymptotic cubic convergence with two-sided Rayleigh quo-
tient iteration, in particular, if A Hermitian.
![Page 10: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/10.jpg)
Program Lecture 4
• Expansion
◦ Krylov subspace approach
Lanczos, Arnoldi, Shift and Invert Arnoldi
◦ Convergence
◦ Accelerated Rayleigh Quotient Iteration
Rational Krylov Sequence method
◦ Optimal expansion
Jacobi-Davidson
• Restart
LOCG, Implicitly Restarted Arnoldi Method
• Deflation
![Page 11: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/11.jpg)
Accelerated RQI. Expand with
t = (A− ρ(u)I)−1u, where ρ(u) = u∗Au (rqi)
and u approximate eigenvector in span(V).
![Page 12: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/12.jpg)
Accelerated RQI. Expand with
t = (A− ρ(u)I)−1u, where ρ(u) = u∗Au (rqi)
and u approximate eigenvector in span(V).
+ Fast convergence (RQI)
+ Subspace allows better control of the process.
Search subspace is not a Krylov subspace.
![Page 13: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/13.jpg)
Accelerated RQI. Expand with
t = (A− ρ(u)I)−1u, where ρ(u) = u∗Au (rqi)
and u approximate eigenvector in span(V).
+ Fast convergence (RQI)
+ Subspace allows better control of the process.
Search subspace is not a Krylov subspace.
[Ruhe 82]Rational Krylov Sequence method allows
different shifts in Shift and Invert (generating a subspace of
rational functions in A times u0, rather than polynomials),
yet it works with Hessenberg matrices (making computa-
tions a little more efficient).
![Page 14: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/14.jpg)
Accelerated RQI. Expand with
t = (A− ρ(u)I)−1u, where ρ(u) = u∗Au (rqi)
and u approximate eigenvector in span(V).
+ Fast convergence (RQI)
+ Subspace allows better control of the process.
Search subspace is not a Krylov subspace.
[Ruhe 82]Rational Krylov Sequence method allows
efficient computations.
Computational costs
S& I: Solving (A− τ I)uk = uk+1 requires one LU-dec. in total
RQI: Solving (A− ρkI)uk = uk+1 requires one LU-dec. each step
May not be feasible if n large.
![Page 15: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/15.jpg)
Program Lecture 4
• Expansion
◦ Krylov subspace approach
Lanczos, Arnoldi, Shift and Invert Arnoldi
◦ Convergence
◦ Accelerated Rayleigh Quotient Iteration
Rational Krylov Sequence method
◦ Optimal expansion
Jacobi-Davidson
• Restart
LOCG, Implicitly Restarted Arnoldi Method
• Deflation
![Page 16: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/16.jpg)
Ax = λx.
If (ϑ,u) is an approximate eigenpair, r ≡ Au− ϑu
(e.g., ϑ = ρ(u)), then the solution t ⊥ u of
(I− uu∗)(A− ϑI)(I− uu∗)t = −r
is the (first order wrt ‖r‖2) correction of u.
[Sleijpen vd Vorst 95] Jacobi-Davidson
![Page 17: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/17.jpg)
(Approximately) solving for t ⊥ u
(I− uu∗)(A− ϑI)(I− uu∗)t = −r (jd)
If solved exactly ⇒ asymptotic quadratic convergence
![Page 18: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/18.jpg)
(Approximately) solving for t ⊥ u
(I− uu∗)(A− ϑI)(I− uu∗)t = −r (jd)
If solved exactly ⇒ asymptotic quadratic convergence
The Jacobi-Davidson correction equation (jd) can be
viewed as a Newton correction equation for the non-linear
problem
F(λ, x) ≡ (1− ‖x‖22,Ax− λx) = (0,0) :
(jd) can be related to the Jacobian of F : Cn+1 → Cn+1.
![Page 19: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/19.jpg)
(Approximately) solving for t ⊥ u
(I− uu∗)(A− ϑI)(I− uu∗)t = −r (jd)
If solved exactly ⇒ asymptotic quadratic convergence
Note. If ϑ ≈ λ,
then the system (A− ϑI)t = u is ill conditioned.
Whereas, if λ is simple and (ϑ,u) ≈ (λ, x), then the system
(jd) as a system in the space u⊥ is well-conditioned.
Krylov subspace solvers with initial guess x0 = 0 are suit-
able for solving in u⊥.
Better conditioning
• faster convergence of the linear solver(deflation of small eigenvalue),
• more stability.
![Page 20: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/20.jpg)
(Approximately) solving for t ⊥ u
(I− uu∗)(A− ϑI)(I− uu∗)t = −r (jd)
If solved exactly ⇒ asymptotic quadratic convergence
Note. If ϑ ≈ λ,
then the system (A− ϑI)t = u is ill conditioned.
Whereas, if λ is simple and (ϑ,u) ≈ (λ, x), then the system
(jd) as a system in the space u⊥ is well-conditioned.
There are block-versions to deal with the non-simple eigen-
value case (but the simple eigenvalue version often works
well in the non-simple case).
Block versions require to know the multiplicity in advance.
Block versions are attractive for computational reasons
(exploit parallelism).
![Page 21: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/21.jpg)
(Approximately) solving for t ⊥ u
(I− uu∗)(A− ϑI)(I− uu∗)t = −r (jd)
If solved exactly ⇒ asymptotic quadratic convergence
Solve approximately with preconditioned iterative linear sol-
ver as GMRES (or MINRES of A∗ = A).
Often a fixed modest number of steps already lead to fast
convergence.
Issue. How many steps are optimal?
Many steps in the inner loop (to solve (jd))
high quality search subspace of low dimension.
Optimal strategy is available in case A = A∗.
![Page 22: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/22.jpg)
(Approximately) solving for t ⊥ u
(I− uu∗)(A− ϑI)(I− uu∗)t = −r (jd)
If solved exactly ⇒ asymptotic quadratic convergence
Approximate solves. M ≈ A− ϑI
t ⊥ u such that (I− uu∗)M(I− uu∗)t = −r
⇔ Mt = −r+ βu with β st t ⊥ u
⇔ t = −M−1r+ βM−1u ⊥ u ⇒ β = u∗M−1ru∗M−1u
⇔ t = −(I− wu∗u∗w)M−1r, where w ≡M−1u
![Page 23: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/23.jpg)
(Approximately) solving for t ⊥ u
(I− uu∗)(A− ϑI)(I− uu∗)t = −r (jd)
If solved exactly ⇒ asymptotic quadratic convergence
Approximate solves. M ≈ A− ϑI
t ⊥ u such that (I− uu∗)M(I− uu∗)t = −r
⇔ t = −(I− wu∗u∗w)M−1r, where w ≡M−1u
![Page 24: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/24.jpg)
(Approximately) solving for t ⊥ u
(I− uu∗)(A− ϑI)(I− uu∗)t = −r (jd)
If solved exactly ⇒ asymptotic quadratic convergence
Approximate solves. M ≈ A− ϑI
t ⊥ u such that (I− uu∗)M(I− uu∗)t = −r
⇔ t = −(I− wu∗u∗w)M−1r, where w ≡M−1u
Expansion by t = −M−1r (d) Davidson ’75
Expansion by t = −(I− wu∗u∗w)M−1r (o) Olsen ’93
![Page 25: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/25.jpg)
(Approximately) solving for t ⊥ u
(I− uu∗)(A− ϑI)(I− uu∗)t = −r (jd)
If solved exactly ⇒ asymptotic quadratic convergence
Approximate solves. M ≈ A− ϑI
t ⊥ u such that (I− uu∗)M(I− uu∗)t = −r
⇔ t = −(I− wu∗u∗w)M−1r, where w ≡M−1u
Interpretation. If M = A− ϑI, then u = M−1r. Hence,
effective expansion with w = M−1u: RQI expansion.
![Page 26: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/26.jpg)
(Approximately) solving for t ⊥ u
(I− uu∗)(A− ϑI)(I− uu∗)t = −r (jd)
If solved exactly ⇒ asymptotic quadratic convergence
Approximate solves. M ≈ A− ϑI
t ⊥ u such that (I− uu∗)M(I− uu∗)t = −r
⇔ t = −(I− wu∗u∗w)M−1r, where w ≡M−1u
Interpretation. If M = A− ϑI, then u = M−1r. Hence,
effective expansion with w = M−1u: RQI expansion.
+ expansion equation (jd) better conditioned then (rqi).
![Page 27: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/27.jpg)
0 5 10 15 20 25 30 35 4010
−10
10−8
10−6
10−4
10−2
100
102
norm
of JD
resid
ual
number of outer iterations
5 steps GMRES
− Jacobi−Davidson
−. Gen. Davidson
... Inexact Shift & Invert
A is SHERMAN4 finding the eigenvalue closest to τ = 0.5
(which is the fifth smallest eigenvalue),M is ILU(2) of A− τ I
![Page 28: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/28.jpg)
0 5 10 15 20 25 30 35 4010
−12
10−10
10−8
10−6
10−4
10−2
100
102
norm
of JD
resid
ual
number of outer iterations
10 steps GMRES
− Jacobi−Davidson
−. Gen. Davidson
... Inexact Shift & Invert
A is SHERMAN4 finding the eigenvalue closest to τ = 0.5
(which is the fifth smallest eigenvalue),M is ILU(2) of A− τ I
![Page 29: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/29.jpg)
0 5 10 15 20 25 30 35 4010
−14
10−12
10−10
10−8
10−6
10−4
10−2
100
102
norm
of JD
resid
ual
number of outer iterations
25 steps GMRES
− Jacobi−Davidson
−. Gen. Davidson
... Inexact Shift & Invert
A is SHERMAN4 finding the eigenvalue closest to τ = 0.5
(which is the fifth smallest eigenvalue),M is ILU(2) of A− τ I
![Page 30: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/30.jpg)
(Approximately) solving for t ⊥ u
(I− uu∗)(A− ϑI)(I− uu∗)t = −r (jd)
If solved exactly ⇒ asymptotic quadratic convergence
Approximate solves. M ≈ A− ϑI
t ⊥ u such that (I− uu∗)M(I− uu∗)t = −r
⇔ t = −(I− wu∗u∗w)M−1r, where w ≡M−1u
Precondition (jd) by (I− wu∗u∗w)M−1, where w ≡M−1u
With x0 = 0, preconditioned Krylov requires mult. by
(I− wu∗u∗w)M−1(A− ϑI)
with right-hand side vector (I− wu∗u∗w)M−1r.
![Page 31: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/31.jpg)
(Approximately) solving for t ⊥ u
(I− uu∗)(A− ϑI)(I− uu∗)t = −r (jd)
If solved exactly ⇒ asymptotic quadratic convergence
Approximate solves. M ≈ A− ϑI
t ⊥ u such that (I− uu∗)M(I− uu∗)t = −r
⇔ t = −(I− wu∗u∗w)M−1r, where w ≡M−1u
Precondition (jd) by (I− wu∗u∗w)M−1, where w ≡M−1u
With x0 = 0, preconditioned Krylov requires mult. by
(I− wu∗u∗w)M−1(A− ϑI)
with right-hand side vector (I− wu∗u∗w)M−1r.
Additional costs. (additional to RQI)
per step mult. by I− wu∗u∗w : 1 AXPY, 1 DOT per step
per Krylov run: 1 solve of Mw = u.
![Page 32: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/32.jpg)
Jacobi–Davidson
• Subspace method
+ Accelerated convergence
+ Steering possibilities
+ variety of selection methods
More costly steps
• Expansion vectors from JD equation
+ Locally optimal expansion (with exact solves)
+ Asymptotic quadratic convergence possible(with exact solves)
+ Well-conditioned (when λ is simple)
+ Fast convergence with moderate accurate solves
+ Preconditioners can be exploited
Additional costs per step
![Page 33: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/33.jpg)
Program Lecture 4
• Expansion
◦ Krylov subspace approach
Lanczos, Arnoldi, Shift and Invert Arnoldi
◦ Convergence
◦ Accelerated Rayleigh Quotient Iteration
Rational Krylov Sequence method
◦ Optimal expansion
Jacobi-Davidson
• Restart
LOCG, Implicitly Restarted Arnoldi Method
• Deflation
![Page 34: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/34.jpg)
For ease of discussion . . .
Recall that harmonic Ritz vectors are better suited for se-
lecting approximate eigenpairs than Ritz vectors. In our
discussion below, we refer to Ritz pairs, but the discussion
can be extended to harmonic Ritz pairs.
For stability we need a well-conditioned bases of the search
subspace. We assume the basis v1, . . . ,vk to be orthonor-
mal and the expansion vector t is to be orthonormalised
against v1, . . . ,vk to obtain the next basis vector vk+1.
However, it may be more efficient to have orthonormality
with respect to other non-standard inner product (as the
A-, or M-inner product, . . . ) or to have bi-orthogonality.
![Page 35: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/35.jpg)
The most promising Ritz vectors
If Vk = [v1, . . . , vk] spans the search subspace Vk,
then we can compute k Ritz pairs (ϑ1,u1), . . . , (ϑk,uk),
i.e., uj ∈ Vk and ϑj ∈ C such that
Auj − ϑj uj ⊥ Vk (j = 1, . . . , k)
or, equivalently, with Hk ≡ V∗kAVk,
uj = Vk yj such that Hk yj = ϑj yj (j = 1, . . . , k).
Note the change in notation:we now suppress the iteration index k, while before we suppressed theordering index j in the set of all Ritz pairs at iteration step k:
including the iteration index k as well in the present notation, the Ritz
pairs (ϑk,uk) that we considered before would correspond to (ϑ(k)j ,u
(k)j )
for some j.
![Page 36: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/36.jpg)
The most promising Ritz vectors
If Vk = [v1, . . . , vk] spans the search subspace Vk,
then we can compute k Ritz pairs (ϑ1,u1), . . . , (ϑk,uk),
i.e., uj ∈ Vk and ϑj ∈ C such that
Auj − ϑj uj ⊥ Vk (j = 1, . . . , k)
or, equivalently, with Hk ≡ V∗kAVk,
uj = Vk yj such that Hk yj = ϑj yj (j = 1, . . . , k).
The ℓ most promising Ritz vectors u1, . . . ,uℓ are the ones
with Ritz value that ‘best’ have the property that we want
our wanted eigenvalue of A to have.
Examples. • Re(ϑj) ≥ Re(ϑj+1) (j = 1, . . . , k − 1)
if the eigenvalue of A with largest real part is wanted.
• |ϑj − τ | ≤ |ϑj+1 − τ | (j = 1, . . . , k − 1)
if the eigenvalue of A closest to some target τ ∈ C is wanted.
![Page 37: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/37.jpg)
The most promising Ritz vectors
If Vk = [v1, . . . , vk] spans the search subspace Vk,
then we can compute k Ritz pairs (ϑ1,u1), . . . , (ϑk,uk).
For stability, we rather compute the Schur decomposition
Hk = USU∗
of Hk rather than the eigenvector decomposition:
here U is k × k unitary and S is k × k upper triangular.
The Ritz values are the diagonal entries of S: ϑj = Sjj.
If Szj = ϑj zj then yj = U zj.
Theorem. The first ℓ columns of U form an orthonormal
basis of the space spanned by the eigenvectors y1, . . . , yℓ of
Hk with eigenvalues the top ℓ diagonal entries S11, . . . , Sℓℓ
of S.
![Page 38: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/38.jpg)
The most promising Ritz vectors
If Vk = [v1, . . . , vk] spans the search subspace Vk,
then we can compute k Ritz pairs (ϑ1,u1), . . . , (ϑk,uk).
For stability, we rather compute the Schur decomposition
Hk = USU∗
of Hk rather than the eigenvector decomposition:
here U is k × k unitary and S is k × k upper triangular.
A Schur decomposition can be re-ordered (using unitary
transforms, Givens rotations) such that the most promising
Ritz values are the top diagonal entries of ‘new’ S:
Theorem. There is a Schur decomposition Hk = USU∗ of
Hk such that the diagonal elements of S appear in prescri-
bed order.
Assumption. u1, . . . ,uℓ are the most promising Ritz vec-
tors and span(u1, . . . ,uℓ) = span(VkUe1, . . . ,VkUeℓ).
![Page 39: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/39.jpg)
Restart
If dim(V) = k is too high, k = kmax, then we have to
restart to limit
• high memory demands (to store Vk+1 = [v1, . . . ,vk,vk+1]),
• high computational costs (to orthonormalise t against Vk).
Simple restart. Take V = span(u1), i.e., V1 = [u1].
Thick restart. Take V = span(u1, . . . ,uℓ) with ℓ = kmin
(i.e., Vℓ = Vk U(:,1 : ℓ) ).
Why thick restart? i.e., why ℓ = kmin > 1?
• to (partially) maintain super linear convergence
• to maintain a space that provides a good initial guess whenthe search to a next eigenpair is started.
Explanation. The second Ritz pair is likely to converge to the secondeigenpair (recall the convergence proof of the power method). Themain component of the error in u1 is probably in the direction of u2,etc.
![Page 40: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/40.jpg)
Restart
If dim(V) = k is too high, k = kmax, then we have to
restart to limit
• high memory demands (to store Vk+1 = [v1, . . . ,vk,vk+1]),
• high computational costs (to orthonormalise t against Vk).
Simple restart. Take V = span(u1), i.e., V1 = [u1].
Thick restart. Take V = span(u1, . . . ,uℓ) with ℓ = kmin
(i.e., Vℓ = Vk U(:,1 : ℓ) ).
Example. For the case where A∗ = A.
Locally Optimal CG takes kmax = 3 and kmin = 2 and
restarts with the two dimensional space spanned by the
best Ritz vector and the best Ritz vector from the prece-
ding step.LOCG expands with a Davidson step, solve t from Mt = r.
![Page 41: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/41.jpg)
Restarting Arnoldi
Arnoldi’s method requires a start with an Arnoldi relation.
If, with η ≡ hk+1,k,
AVk = Vk+1Hk = VkHk + ηvk+1e∗k,
then, with q∗ ≡ e∗kU(:,1 : ℓ) using Matlab’s notation,
AVkU(:,1 : ℓ) = VkU(:,1 : ℓ)S + ηvk+1q∗. (∗)
The ℓ-vector q will not be a multiple of eℓ and (∗) is not
an Arnoldi relation.
![Page 42: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/42.jpg)
Restarting Arnoldi
Arnoldi’s method requires a start with an Arnoldi relation.
If, with η ≡ hk+1,k,
AVk = Vk+1Hk = VkHk + ηvk+1e∗k,
then, with q∗ ≡ e∗kU(:,1 : ℓ) using Matlab’s notation,
AVkU(:,1 : ℓ) = VkU(:,1 : ℓ)S + ηvk+1q∗. (∗)
The ℓ-vector q will not be a multiple of eℓ and (∗) is not
an Arnoldi relation.
Theorem. There is a unitary matrix Q (product of Hou-
seholder reflections) such that[Q 00∗ 1
]∗ [S
ηq∗
]Q is (ℓ+1)× ℓ upper Hessenberg.
Note. Except for vector updates Vk(U(:,1 : ℓ)Q) this allowsa restart at the cost of only low dimensional operations.
![Page 43: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/43.jpg)
Implicitly Restarted Arnoldi Method
More elegant and more efficient and stable implementation
to compute U(:,1 : ℓ)Q by Sorenson is based on idea QR-
algorithm.
![Page 44: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/44.jpg)
Implicitly Restarted Arnoldi Method
Select shifts µk, µk−1 . . . , µℓ+1 in C. Let p be the polynomial
p(ζ) ≡ (ζ − µk) · . . . · (ζ − µℓ+1) (ζ ∈ C).
Let v1 ≡ p(A)v1. Consider the Arnoldi relation
AVℓ = Vℓ+1 Hℓ with Vℓe1 = v1.
Theorem. This Arnoldi relation can be obtained by ap-
plying k − ℓ steps of the shifted QR-algorithm to Hk with
shifts µj finding HkU = UHk and then use the first ℓ co-
lumns of U to form Vℓ = VkU(:,1 : ℓ).
IRAM selects µj = ϑj (the less best Ritz values). This
gives the desired Arnoldi: the columns of Vℓ form an or-
thonormal basis of the space spanned by u1, . . . ,uℓ. The
eigenvalues of Hℓ are precisely ϑ1, . . . , ϑℓ.
![Page 45: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/45.jpg)
Implicitly Restarted Arnoldi Method
eigs.m in Matlab is IRAM (with deflation)
eigs.m is based on ARPACK, a collection of FORTRAN
routines that implements IRAM
Lehoucq, Sorensen and Yang 1998
![Page 46: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/46.jpg)
Program Lecture 4
• Expansion
◦ Krylov subspace approach
Lanczos, Arnoldi, Shift and Invert Arnoldi
◦ Convergence
◦ Accelerated Rayleigh Quotient Iteration
Rational Krylov Sequence method
◦ Optimal expansion
Jacobi-Davidson
• Restart
LOCG, Implicitly Restarted Arnoldi Method
• Deflation
![Page 47: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/47.jpg)
Towards the next eigenpair
If an eigenvector x with eigenvalue λ has been detected
(computed), how to continue the computational process
for the next eigenpair?
![Page 48: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/48.jpg)
Towards the next eigenpair
If an eigenvector x with eigenvalue λ has been detected
(computed), how to continue the computational process
for the next eigenpair?
1) Continue that process, but, at restart, do not select λ.
![Page 49: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/49.jpg)
Towards the next eigenpair
If an eigenvector x with eigenvalue λ has been detected
(computed), how to continue the computational process
for the next eigenpair?
1) Continue that process, but, at restart, do not select λ.
Very unstable: the computed eigenpair will be computed
again, which is a waste of computational effort.
![Page 50: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/50.jpg)
Towards the next eigenpair
If an eigenvector x with eigenvalue λ has been detected
(computed), how to continue the computational process
for the next eigenpair?
1) Continue that process, but, at restart, do not select λ.
2) Deflation: remove the detected eigenvector x from the
eigenvalue problem (and, in each step, from the computa-
tional process).
![Page 51: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/51.jpg)
Deflation
For stability reason, aim for Schur decomposition
AQ = QS
with Q unitary and S upper triangular.
Suppose partial Schur decomposition is available, i.e.,
AQ1 = Q1S11 with Q1 the n× ℓ left block Q andS11 the ℓ× ℓ left upper block of S.
Note. S11y = λy, then Ax = λx if x ≡Q1y.
![Page 52: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/52.jpg)
Deflation
For stability reason, aim for Schur decomposition
AQ = QS
with Q unitary and S upper triangular.
Suppose partial Schur decomposition is available, i.e.,
AQ1 = Q1S11 with Q1 the n× ℓ left block Q andS11 the ℓ× ℓ left upper block of S.
Note. S11y = λy, then Ax = λx if x ≡Q1y.
Theorem. q = qℓ+1 with λ = sℓ+1,ℓ+1 satisfies
(I−Q1Q∗1)A(I−Q1Q
∗1)q = λq
![Page 53: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/53.jpg)
Deflation
Deflation can be incorporated in two ways.
• Either by keeping the detected Schur vectors in the search
subspace and working with A. This approach fits Arnoldi’s
method well; see next transparency.
• Or by removing these vectors from the search subspace
and working with the deflated matrix. This approach fits
Jacobi–Davidson well.
In JD, it leads to the following JD correction equation:
(I− [Q1,u][Q∗1,u])(A− ϑI)(I− [Q1,u][Q1,u]
∗)t = −r.
Recall that enhancing the conditioning is one of the be-
nefits of including a projection in the ‘Shift and Invert’
equation. Including Q1 in the projections amplifies this ef-
fect. Note that the solution t is orthogonal to Q1 and the
search subspace gets automatically expanded with vectors
⊥Q1.
![Page 54: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/54.jpg)
Deflating Arnoldi’s relation
We want to retain an Arnoldi relation at deflation.
Theorem. If AVk = Vk+1Hk, then the following properties
are equivalent
• One of the Ritz vectors uj is an eigenvector.
• A sub-diagonal element, say hℓ+1,ℓ, of Hk is 0.
• span(Vℓ) is A-invariant (whence, spanned by eigenvectors).
Idea 1. Replace small sub-diagonal elements of Hk, say
|hℓ+1,ℓ| ≤ ǫ, by 0.
Idea 2. Keep Vℓ in the Arnoldi basis for orthogonalisation,
use V(:, [ℓ+1 : k]) for selection (restart).
Note. span(Vℓ) = span(Q1). If Vℓ is in the Arnoldi
basis then orthonormalising Avk against Vk is equivalent
to orthonormalising (I − Q1Q∗1)A(I − Q1Q
∗1)vk against
V(:, [ℓ+1 : k]).
![Page 55: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/55.jpg)
Deflating Arnoldi’s relation
We want to retain an Arnoldi relation at deflation.
Theorem. If AVk = Vk+1Hk, then the following properties
are equivalent
• One of the Ritz vectors uj is an eigenvector.
• A sub-diagonal element, say hℓ+1,ℓ, of Hk is 0.
• span(Vℓ) is A-invariant (whence, spanned by eigenvectors).
Idea 1. Replace small sub-diagonal elements of Hk, say
|hℓ+1,ℓ| ≤ ǫ, by 0.
Idea 2. Keep Vℓ in the Arnoldi basis for orthogonalisation,
use V(:, [ℓ+1 : k]) for selection (restart).
Note. Idea 1 is unstable and needs stabilisation.
![Page 56: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/56.jpg)
Example: JD + options
The next transparencies illustrate Jacobi-Davidson with
thick restart and deflation
![Page 57: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/57.jpg)
1. Start: Choose an initial z.
• Set Λ = [ ], U = [ ], V = [ ].
2. Repeat:
a. V ← ModGS([V |z ]), W ← AV
b. Compute H ← V∗W.
c. Compute H = QRQ∗
with Q∗Q = I, R upper ∆,
|R11 − τ | ≤ |R22 − τ | ≤ . . ..
d. ϑ ← R11, y ← Qe1, u ← Vy ,
r ← ModGSU(Au− ϑu).
e. If ‖ r ‖ ≤ tol
• Λ ← [ Λ |ϑ ], U ← [U |u ].
• J=[ e2 | . . . | edim(V) ],
V← V(QJ), Q ← I, R← J∗RJ.
• Goto 2.d.
f . Solve (app.) z ⊥ U ≡ [U |u ]
(I− U U∗) (A− ϑ I) (I− U U
∗) z = −r.
![Page 58: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/58.jpg)
-10
-5
0 10 20 30 40 50 60
log
10
of
resid
ua
l n
orm
number of flops x 1e6
GMRES Ritz1
-10
-5
log
10
of
resid
ua
l n
orm
GMRES Harmonic1
max(dim(span(V))) = 15, after restart dim(V) = 10
![Page 59: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/59.jpg)
Spectrum MHD test problem
−600
−450
−300
−150
0
150
300
450
600
−50 −40 −30 −20 −10 0 10
C
Entire spectrum of the MHD test problem (n = 416)
showing the correct location ( ) of the eigenvalues.
![Page 60: Uppsala, April 2014 Subspace methodssleij101/Uppsala/Lecture4.pdf · Shift & Invert uk+1 = (A −τI)−1uk. (s&i) Linear convergence towards eigenvector with eigenvalue closest to](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed6bcb6973aa21e5d3a0995/html5/thumbnails/60.jpg)
Alfven spectrum
0.0
0.2
0.4
0.6
0.8
1.0
−1.0 −0.8 −0.6 −0.4 −0.2 0.0
C
∗�
The MHD test problem (n = 416).
Correct eigenvalues ( ◦ ), target ( ∗� ).