krylov 2 filimon
Post on 03-Jun-2018
238 Views
Preview:
TRANSCRIPT
-
8/12/2019 Krylov 2 Filimon
1/24
Krylov Subspace Iteration Methods
Anastasia Filimon
ETH Zurich
29 May 2008
Anastasia Filimon (ETH Zurich) Krylov Subspace Iteration Methods 29/05/08 1 / 24
http://find/ -
8/12/2019 Krylov 2 Filimon
2/24
Set up
The methods represent iterative techniques for solving large linearsystems
Ax=b,
where Ais non-singular nxn-matrix, bis n-vector, n is large.
They are based on projection processes onto Krylov subspaces.
Anastasia Filimon (ETH Zurich) Krylov Subspace Iteration Methods 29/05/08 2 / 24
http://find/ -
8/12/2019 Krylov 2 Filimon
3/24
Krylov Subspaces
Krylov subspace generated by an nxn-matrix A, and an n-vector b is
the subspace spanned by the vectors of the Krylov sequence:
Km =spanfb, Ab, A2b, ..., Am1bg.The projection method seeks an approximate solution xm from anane subspace x0+ Km by imposing condition b
Axm
?Lm , Lm is
another subspace of dimension m, x0 is an initial guess to thesolution. In the case of Krylov subspace methods Km = Km(A, r0),r0 =b Ax0 is an n-vector
Km =spanfr0 , Ar0 , A
2r0 , ..., Am1r0
g.
Property 1. Km is the subspace of all vectors in Rn which can be
written asx=p(A)v,
where pis a polynomial of degree not exceeding m 1.Anastasia Filimon (ETH Zurich) Krylov Subspace Iteration Methods 29/05/08 3 / 24
http://find/ -
8/12/2019 Krylov 2 Filimon
4/24
Krylov Subspaces
The dierent versions of Krylov subspace methods arise from dierentchoices of the subspace Lm and from the ways in which the system ispreconditioned. Two broad choices for Lm give rise to the best-knowntechniques:
Lm =Km FOMLm =AKm GMRES, MINRES.
Anastasia Filimon (ETH Zurich) Krylov Subspace Iteration Methods 29/05/08 4 / 24
http://find/ -
8/12/2019 Krylov 2 Filimon
5/24
Arnoldi Modied Gram-Schmidt Method
This is an algorithm for building an orthogonal basis of the Krylovsubspace Km .
At each step, the algorithm multiplies Arnoldi vector vj byAand thenorthonormalizes the resulting vector wjagainst all previous vjs by a
standard Gram-Schmidt procedure.Anastasia Filimon (ETH Zurich) Krylov Subspace Iteration Methods 29/05/08 5 / 24
http://find/http://goback/ -
8/12/2019 Krylov 2 Filimon
6/24
Arnoldis Method
Proposition 1. Assume the Arnoldi Algorithm does not stop beforethe m-th step. Then the vectors v1 , v2 , ..., vm form an orthonormalbasis of the Krylov subspace
Km =spanfv1 , Av1 , ..., Am1v1g.Proposition 2. Projection method onto the subspace Kjwill beexact when a breakdown occurs at step j.Proposition 3. Denote by Vm the nxmmatrix with column vectors
v1 , ..., vm , byH m , the (m+1)xm Hessenberg matrix whose nonzeroentries hijare dened by Arnoldi Modied Gram-Schmidt Algorithmand by Hm the matrix obtained from H m by deleting its last row.
Then the following relations hold:
AVm =VmHm+ wmeTm
=Vm+1Hm
VTmAVm =Hm
.
Anastasia Filimon (ETH Zurich) Krylov Subspace Iteration Methods 29/05/08 6 / 24
http://find/http://goback/ -
8/12/2019 Krylov 2 Filimon
7/24
Full Orthogonalization Method (FOM)
Given the initial guess x0 to the original linear system Ax=b,consider the projection method described before, which takesLm =Km(A, r0), where r0 =b Ax0 . Ifv1 = r0kr0k2 in Arnoldismethod and set =kr0k2, then
VTmAVm = Hm from Proposition 3, and
VTmr0 = V
Tm
(v1) = e1 .
As a result, the approximate solution using the above mdimensionalsubspaces is given by
xm =x0+ Vmym , ym =H1m
(e1)
Anastasia Filimon (ETH Zurich) Krylov Subspace Iteration Methods 29/05/08 7 / 24
http://find/ -
8/12/2019 Krylov 2 Filimon
8/24
Full Orthogonalization Method (FOM)
The presented algorithm depends on a parameter m which is thedimension of the Krylov subspace. In practice it is desirable to selectm in a dynamic fashion. This would be possible if the residual norm
of xm is available without compution xm itself.Proposition 4. The residual vector of the approximate solution xmcomputed by the FOM Algorithm is such that
kb
Axm
k2 =hm+1,m eTm ym .
Anastasia Filimon (ETH Zurich) Krylov Subspace Iteration Methods 29/05/08 8 / 24
http://find/ -
8/12/2019 Krylov 2 Filimon
9/24
Full Orthogonalization Method (FOM)
Anastasia Filimon (ETH Zurich) Krylov Subspace Iteration Methods 29/05/08 9 / 24
http://find/ -
8/12/2019 Krylov 2 Filimon
10/24
Generalized Minimum Residual Method (GMRES)
The method is a projection method based on taking Lm =AKm ,inwhich Km is the m-th Krylov subspace with v1 =r0/
kr0k2.
Such atechnique minimizes the residual norm over all vectors in x0+ Km .The implementation of an algorithm based on this approach is similarto that of the FOM algorithm.
Any vector x in x0+ Km can be written as x=x0+ Vmy, where y is
an m-vector.Dene
J(y) =kb Axk2 =kb A(x0+ Vmy)k2 ,Using relation from Proposition 3
b Ax = b A(x0+ Vmy) = roAVmy= v1 Vm+1Hmy= Vm+1(e1Hmy).
Since the column-vectors ofVm+1 are orthonormal, then
J(y) =
kb
A(x0+ Vmy)
k2 = e1Hmy2 .
Anastasia Filimon (ETH Zurich) Krylov Subspace Iteration Methods 29/05/08 10 / 24
http://find/http://goback/ -
8/12/2019 Krylov 2 Filimon
11/24
Generalized Minimum Residual Method (GMRES)
The GMRES approximation is the unique vector ofx0+ Km whichminimizes J(y), i.e.
xm = x0+ Vmym,
whereym = arg min
y
e1Hmy2The minimizer is inexpensive to compute since it requires the solutionof an (m+1)xm least-squares problem where m is typically small.
Anastasia Filimon (ETH Zurich) Krylov Subspace Iteration Methods 29/05/08 11 / 24
http://find/ -
8/12/2019 Krylov 2 Filimon
12/24
Generalized Minimum Residual Method (GMRES)
Anastasia Filimon (ETH Zurich) Krylov Subspace Iteration Methods 29/05/08 12 / 24
http://find/ -
8/12/2019 Krylov 2 Filimon
13/24
Generalized Minimum Residual Method (GMRES).
Practical Implementation issues
A clear diculty with GMRES algorithm is that it does not providethe approximate solution xm explicitly at each step. As a result, it is
not easy to determine when to stop. However, there is a solutionrelated to the way in which the least-squares problem is solved.
In order to solve the least-squares problem mine1Hmy
, it isnatural to transform the Hessenberg matrix into upper triangular form
by using plane rotations.
Anastasia Filimon (ETH Zurich) Krylov Subspace Iteration Methods 29/05/08 13 / 24
http://find/ -
8/12/2019 Krylov 2 Filimon
14/24
Generalized Minimum Residual Method (GMRES).
Practical Implementation issues
Proposition 5. Dene the rotation matrices to transform Hm intoupper triangular form
i =
0BBBBBBBBBB@
1...
1ci sisi ci
1...
1
1CCCCCCCCCCA
c2i + s2i =1, si=
hi+1,iq(h
(i1)ii )
2 +h2i+1,i
, ci= h
(i1)iiq
(h(i1)ii )
2 +h2i+1,i
.
Anastasia Filimon (ETH Zurich) Krylov Subspace Iteration Methods 29/05/08 14 / 24
http://find/ -
8/12/2019 Krylov 2 Filimon
15/24
Generalized Minimum Residual Method (GMRES).
Practical Implementation issues
Dene the product of matrices i
Qm = mm1 ...1 ,
Rm,
gm = (1, ...,
m+1)T
the resulting matrix and right-hand side
Rm = H(m)m =QmHm ,
gm = Qm(e1) = (1 , ..., m+1)T
.
Denote by Rm the mxm upper triangular matrix obtained from Rm bydeleting its last row and by gm the mdimensional vector obtainedfrom gm by deleting its last component.
Anastasia Filimon (ETH Zurich) Krylov Subspace Iteration Methods 29/05/08 15 / 24
( )
http://find/ -
8/12/2019 Krylov 2 Filimon
16/24
Generalized Minimum Residual Method (GMRES).
Practical Implementation issues
Then,1. Vector ym which minimizes
e1Hmy2 is given byym =R
1m gm .
2. The residual vector at step m satises
bAxm =Vm+1(e1Hmy) =Vm+1QTm(m+1em+1) and, as a result,kb Axmk2 =jm+1j .
This was the process for computing the least-squares solution ym .Theprocess must be stopped if the residual normjm+1j is small enough.The last rows ofRm and gm are deleted and the resulting uppertriangular system is solved to obtain ym . Then the approximatesolution xm =x0+ Vmym is computed.
Anastasia Filimon (ETH Zurich) Krylov Subspace Iteration Methods 29/05/08 16 / 24
Th S i L Al i h
http://find/ -
8/12/2019 Krylov 2 Filimon
17/24
The Symmetric Lanczos Algorithm
This algorithm can be viewed as a simplication of Arnoldis methodfor the particular case when the matrix is symmetric. When Ais
symmetric, then the Hessenberg matrix Hm becomes symmetrictridiagonal. 0
BBBB@1 22 2 3
...
m1 m1 mm m
1CCCCA
Anastasia Filimon (ETH Zurich) Krylov Subspace Iteration Methods 29/05/08 17 / 24
Th C j G di Al i h
http://find/ -
8/12/2019 Krylov 2 Filimon
18/24
The Conjugate Gradient Algorithm
This algorithm is one of the best known iterative techniques forsolving sparse Symmetric Positive Denite linear systems.
Assume we need to minimize the following function
f(x) = 12xTAx xTb
where Ais nxn-matrix positive denite and symmetric, b is n-vector.
The minimum value off(x) isbTA1b/2, achieved by settingx= A
1
b. Therefore, minimizingf(x) and solving Ax=bareequivalent problems ifA is symmetric positive denite.
Anastasia Filimon (ETH Zurich) Krylov Subspace Iteration Methods 29/05/08 18 / 24
Th C j G di Al i h
http://find/ -
8/12/2019 Krylov 2 Filimon
19/24
The Conjugate Gradient Algorithm
The vector xj+1 can be expressed as .
xj+1 =xj+jpj.
Therefore, the residual vectors must satisfy the recurrence
rj+1 =rjjApj.
To have rjs orthogonal it is necessary that
(rj
jApj, rj) =0
and, as a result,
j = (rj,rj)
(Apj, rj).
Anastasia Filimon (ETH Zurich) Krylov Subspace Iteration Methods 29/05/08 19 / 24
Th C j t G di t Al ith
http://find/ -
8/12/2019 Krylov 2 Filimon
20/24
The Conjugate Gradient Algorithm
The rst basis vector p1 is the gradient off at x0 , which equals to
Ax0 b . The other vectors in the basis will be conjugate to thegradient . Each nextpk+1 is dened to be in the direction closest tothe gradient rkunder the conjugacy constraint. This direction is givenby the projection ofrkonto the space orthogonal to pk with respectto the inner product induced by A.
pj+1 = rj+1+jpj,
(Apj, rj) = (Apj, pjj1pj1) = (Apj, pj)
j =
(rj,rj)
(Apj, pj),
j=
(rj+1,Apj)
(pj, Apj)
Apj = 1j
(rj+1 rj),j= 1
j
(rj+1,(rj+1 rj))(Apj, pj)
=(rj+1 , rj+1)
(rj, rj)
Anastasia Filimon (ETH Zurich) Krylov Subspace Iteration Methods 29/05/08 20 / 24
Th C j t G di t Al ith
http://find/ -
8/12/2019 Krylov 2 Filimon
21/24
The Conjugate Gradient Algorithm
The process stops if rj+1 is "suciently small".
Anastasia Filimon (ETH Zurich) Krylov Subspace Iteration Methods 29/05/08 21 / 24
Convergence Analysis
http://find/ -
8/12/2019 Krylov 2 Filimon
22/24
Convergence Analysis
One of the main tool used in the analysis of convergence behavior isChebyshev polynomials.
Lemma 1. Let xm be the approximate solution obtained from themth step of the CG algorithm, and let dm =x xm , where x isthe exact solution. Then xm is of the form
xm =x0+ qm(A)r0 ,
where qm is a polynomial of degree m 1 such thatk(IAqm(A))d0kA = min
q2Pm1k(IAq(A))d0kA .
Theorem 2. Let xm be the approximate solution obtained from themth step of the CG algorithm, and x is the exact solution. Then
kx xmkA2"p
k 1pk+1
#mkx x0kA .
Anastasia Filimon (ETH Zurich) Krylov Subspace Iteration Methods 29/05/08 22 / 24
Convergence Analysis
http://find/ -
8/12/2019 Krylov 2 Filimon
23/24
Convergence Analysis
Anastasia Filimon (ETH Zurich) Krylov Subspace Iteration Methods 29/05/08 23 / 24
Convergence Analysis
http://find/ -
8/12/2019 Krylov 2 Filimon
24/24
Convergence Analysis
Anastasia Filimon (ETH Zurich) Krylov Subspace Iteration Methods 29/05/08 24 / 24
http://find/
top related