![Page 1: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/1.jpg)
CSE 245: Computer Aided Circuit Simulation and Verification
Matrix Computations: Iterative Methods (II)
Chung-Kuan Cheng
![Page 2: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/2.jpg)
2
Outline
Introduction Direct Methods Iterative Methods
Formulations Projection Methods Krylov Space Methods Preconditioned Iterations Multigrid Methods Domain Decomposition Methods
![Page 3: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/3.jpg)
3
Introduction
Direct MethodLU Decomposition
Iterative Methods
Jacobi
Gauss-Seidel
Conjugate GradientGMRES
Multigrid
Domain Decomposition
PreconditioningGeneral and Robust but can be complicated ifN>= 1M
Excellent choice for SPD matricesRemain an art for arbitrary matrices
![Page 4: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/4.jpg)
4
Formulation
Error in A norm Matrix A is SPD (Symmetric and Positive
Definite Minimal Residue
Matrix A can be arbitrary
![Page 5: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/5.jpg)
5
Formulation: Error in A norm
Min E(x)=1/2 xTAx – bTx, A is SPDSuppose that we knew Ax* =b
E(x)=1/2 (x-x*)TA(x-x*).
For Krylov space approach,search space: x=x0+Vy,
where x0 is an initial solution,
matrix Vnxm is given and full ranked
vector ym contains the m variables.
![Page 6: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/6.jpg)
6
Solution and ErrorMin E(x)=1/2xTAx - bTx,Search space: x=x0+Vy, r=b-Ax
Derivation1. VTAV is nonsingular (A is SPD & V is full
ranked2. The variable: y=(VTAV)-1VTr0 (derivation)
3. Thus, solution: x=x0+V(VTAV)-1VTr0
Property of the solution4. Residue: VTr=0 (proof)5. Error: E(x)=E(x0)-1/2 r0
TV(VTAV)-1VTr0
![Page 7: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/7.jpg)
7
Solution and ErrorMin E(x)=1/2xTAx - bTx,Search space: x=x0+Vy, r=b-Ax
The variable: y=(VTAV)-1VTr0
Derivation of y: Use the condition that dE/dy=0We have VTAVy+VTAx0-VTb=0
Thus y=(VTAV)-1VT(b-Ax0)
![Page 8: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/8.jpg)
8
Solution and ErrorMin E(x)=1/2xTAx - bTx,Search space: x=x0+Vy, r=b-Ax
Property of Solution1. Residue: VTr=0 (proof)Proof: VTr=VT(b-Ax)=VT(b-Ax0-AV(VTAV)-1VTr0)
=VT(r0-AV(VTAV)-1VTr0)=0
The residue of the solution is orthogonal to the previous bases.
![Page 9: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/9.jpg)
9
Transformation of the basesFor any V’=VW, where Vnxm is given and Wmxm is nonsigular, solution x remains the same for search space: x=x0+V’y
Proof:1. V’TAV’ is nonsingular (A is SPD & V’ is full
ranked)2. x=x0+V’(V’TAV’)-1V’Tr0=x0+V(VTAV)-1VTr0
3. V’Tr=WTVTr=04. E(x)=E(x0)-1/2 r0
TV(VTAV)-1VTr0
![Page 10: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/10.jpg)
10
Steepest Descent: Error in A norm
Min E(x)=1/2 xTAx - bTx,Gradient: -r = Ax-bSet x=x0+yr0
1. r0TAr0 is nonsingular (A is SPD)
2. y=(r0TAr0)-1r0
Tr0
3. x=x0+r0(r0TAr0)-1r0
Tr0
4. r0Tr=0
5. E(x)=E(x0)-1/2 (r0Tr0)2/(r0
TAr0)
![Page 11: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/11.jpg)
11
Lanczos: Error in A norm
Min E(x)=1/2 xTAx-bTx,Set x=x0+Vy
1. v1=r0
2. vi is in K{r0, A, i}
3. V=[v1,v2, …,vm] is orthogonal
4. AV=VHm+vm+1 emT
5. VTAV=Hm
Note since A is SPD, Hm is Tm Tridiagonal
![Page 12: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/12.jpg)
12
Lanczos: Derive from Arnoldi ProcessArnoldi Process: AV=VH+hm+1vm+1em
T
Input A, v1=r0/|r0|
Output V =[v1,v2, …, vm], H and hm+1, vm+1
k=0, h10=1
While hk+1,k!=0, k<=m
vk+1=rk/hk+1,k
k=k+1, rk=Avk
For i=1, khik=vi
Trk
rk=rk-hikvi
Endhk+1,k=|rk|2
End
![Page 13: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/13.jpg)
13
Lanczos:Create V=[v1,v2, …,vm] which is orthogonal
Input A, r0, Output V and T=VTAV
Initial: k=0, β0=|r0|2, v0=0, v1=r0/β0
while k≤ K or βk!= 0
vk+1=rk/βk
k=k+1ak=vk
TAvk
rk=Avk-akvk- βk-1vk-1
βk=|rk|2
End
![Page 14: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/14.jpg)
14
Lanczos:Create V=[v1,v2, …,vm] which is orthogonal
Input A, r0, Output V and T=VTAV
Tii=ai, Tij=bi (j=i+1), =bj (j=i-1), =0 (else)
Proof: By induction that vj
TAvj= 0, if i<j-1
bj, if i=j-1
aj, if i=j
Since vivj=0 if i!=j
vjTAvi=vj
T(bi-1vi-1+aivi+bivi+1)
=bi-1vjTvi-1+aivj
Tvi+bivjTvi+1
![Page 15: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/15.jpg)
15
Conjugate Gradient:
Min E(x)=1/2 xTAx-bTx,Set x=x0+Vy
1. v1=r0
2. vi is in K{r0, A, i}
3. Set V=[v1,v2, …,vm] orthogonal in A norm, i.e. VTAV= [diag(vi
TAvi)]=D
4. x=x0+VD-1VTr0,
5. x= x0+∑i=1,mdiviviTr0, where di=(vi
TAvi)-1
![Page 16: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/16.jpg)
16
Conjugate Gradient Method Steepest Descent
Repeat search direction Why take exact one step
for each direction?
Search direction of Steepestdescent method
![Page 17: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/17.jpg)
17
Orthogonal A-orthogonal Instead of orthogonal search direction, we make search
direction A –orthogonal (conjugate)
0)()( iTj Add
![Page 18: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/18.jpg)
Conjugate Search Direction
How to construct A-orthogonal search directions, given a set of n linear independent vectors.
Since the residue vector in steepest descent method is orthogonal, a good candidate to start with
110 ,,, nrrr
18
![Page 19: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/19.jpg)
19
Conjugate Gradient:
Min E(x)=1/2 xTAx-bTx,Set x=x0+Vy
Lanczos Method to derive V and T x=x0+VT-1VTr0
Decompose T to LDU=T (U=LT)Thus, we have x=x0+V(LDLT)-1VTr0=x0+VL-TD-1L-1VTr0
![Page 20: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/20.jpg)
Conjugate Gradient Algorithm
)()1()1()1(
)1(
)()()()1(
)()()()1(
)(
)0()0()0(
)()(
)1()1(
)()(
)()(
iiii
rr
rr
i
iiii
iiii
Add
rr
i
drd
Adarr
daxx
a
Axbrd
iT
i
iT
i
iT
i
iT
i
Given x0, iterate until residue is smaller than error tolerance
20
![Page 21: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/21.jpg)
21
Conjugate gradient: Convergence
In exact arithmetic, CG converges in n steps (completely unrealistic!!)
Accuracy after k steps of CG is related to: consider polynomials of degree k that is
equal to 1 at step 0. how small can such a polynomial be at
all the eigenvalues of A? Eigenvalues close together are good. Condition number: κ(A) = ||A||2 ||A-1||2 =
λmax(A) / λmin(A) Residual is reduced by a constant factor by
O(κ1/2(A)) iterations of CG.
![Page 22: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/22.jpg)
22
Preconditioners Suppose you had a matrix B such that:
1. condition number κ(B-1A) is small2. By = z is easy to solve
Then you could solve (B-1A)x = B-1b instead of Ax = b
B = A is great for (1), not for (2) B = I is great for (2), not for (1) Domain-specific approximations sometimes work B = diagonal of A sometimes works
Better: blend in some direct-methods ideas. . .
![Page 23: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/23.jpg)
23
Preconditioned conjugate gradient iteration
x0 = 0, r0 = b, d0 = B-1 r0, y0 = B-1 r0
for k = 1, 2, 3, . . .
αk = (yTk-1rk-1) / (dT
k-1Adk-1) step length
xk = xk-1 + αk dk-1 approx solution
rk = rk-1 – αk Adk-1 residual
yk = B-1 rk preconditioning solve
βk = (yTk rk) / (yT
k-1rk-1) improvement
dk = yk + βk dk-1 search direction One matrix-vector multiplication per iteration One solve with preconditioner per iteration
![Page 24: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/24.jpg)
24
Other Krylov subspace methods
Nonsymmetric linear systems: GMRES:
for i = 1, 2, 3, . . . find xi Ki (A, b) such that ri = (Axi – b) Ki (A, b)
But, no short recurrence => save old vectors => lots more space (Usually “restarted” every k iterations to use less space.)
BiCGStab, QMR, etc.:Two spaces Ki (A, b) and Ki (AT, b) w/ mutually orthogonal
bases Short recurrences => O(n) space, but less robust Convergence and preconditioning more delicate than CG Active area of current research
Eigenvalues: Lanczos (symmetric), Arnoldi (nonsymmetric)
![Page 25: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/25.jpg)
25
Formulation: Residual
Min |r|2=|b-Ax|2, for an arbitrary square matrix A
Min R(x)=(b-Ax)T(b-Ax)Search space: x=x0+Vy
where x0 is an initial solution,
matrix Vnxm has m bases of subspace K
vector ym contains the m variables.
![Page 26: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/26.jpg)
26
Solution: Residual
Min R(x)=(b-Ax)T(b-Ax)Search space: x=x0+Vy
1. VTATAV is nonsingular if A is nonsingular and V is full ranked.
2. y=(VTATAV)-1VTATr0
3. x=x0+V(VTATAV)-1VTATr0
4. VTATr= 05. R(x)=R(x0)-r0
TAV(VTATAV)-1VTATr0
![Page 27: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/27.jpg)
27
Steepest Descent: Residual
Min R(x)=(b-Ax)T(b-Ax)Gradient: -2AT(b-Ax)=-2ATrLet x=x0+yATr0
1. VTATAV is nonsingular if A is nonsingular where V=ATr0.
2. y=(VTATAV)-1VTATr0
3. x=x0+V(VTATAV)-1VTATr0
4. VTATr= 05. R(x)=R(x0)-r0
TAV(VTATAV)-1VTATr0
![Page 28: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/28.jpg)
28
GMRES: Residual
Min R(x)=(b-Ax)T(b-Ax)Gradient: -2AT(b-Ax)=-2ATrLet x=x0+yATr0
1. v1=r0
2. vi is in K{r0, A, i}
3. V=[v1,v2, …,vm] is orthogonal
4. AV=VHm+vm+1 emT=Vm+1Hm
5. x=x0+V(VTATAV)-1VTATr0
6. =x0+V(HmTHm)-1 Hm
Te1|r0|2
![Page 29: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/29.jpg)
29
Conjugate Residual: Residual
Min R(x)=(b-Ax)T(b-Ax)Gradient: -2AT(b-Ax)=-2ATrLet x=x0+yATr0
1. v1=r0
2. vi is in K{r0, A, i}
3. (AV)TAV= D Diagonal Matrix4. x=x0+V(VTATAV)-1VTATr0
5. =x0+VD-1VTATr0
![Page 30: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/30.jpg)
Outline Iterative Method
Stationary Iterative Method (SOR, GS,Jacob)
Krylov Method (CG, GMRES) Multigrid Method
30
![Page 31: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/31.jpg)
What is the multigrid A multilevel iterative method to solve
Ax=b Originated in PDEs on geometric grids Expend the multigrid idea to
unstructured problem – Algebraic MG Geometric multigrid for presenting
the basic ideas of the multigrid method.
31
![Page 32: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/32.jpg)
The model problem
+ v1 v2 v3 v4 v5 v6 v7 v8vs
0
0
0
0
0
0
0
21
121
121
121
121
121
121
12
8
7
6
5
4
3
2
1 sv
v
v
v
v
v
v
v
v Ax = b
32
![Page 33: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/33.jpg)
Simple iterative method
x(0) -> x(1) -> … -> x(k) Jacobi iteration
Matrix form : x(k) = Rjx(k-1) + Cj
General form: x(k) = Rx(k-1) + C (1) Stationary: x* = Rx* + C (2)
))(/1(1
)(1
1
)()1(
n
ij
kjik
i
j
kjikiii
ki xaxabax
33
![Page 34: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/34.jpg)
Error and Convergence
Definition: error e = x* - x (3) residual r = b – Ax (4)
e, r relation: Ae = r (5) ((3)+(4))
e(1) = x*-x(1) = Rx* + C – Rx(0) – C =Re(0) Error equation e(k) = Rke(0) (6) ((1)+(2)+(3))
Convergence:0lim )(
k
ke
34
![Page 35: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/35.jpg)
Error of diffenent frequency Wavenumber k and frequency
w = k/n
• High frequency error is more oscillatory between points
k= 1
k= 4 k= 2
35
![Page 36: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/36.jpg)
Iteration reduce low frequency error efficiently
Smoothing iteration reduce high frequency error efficiently, but not low frequency error
Error
Iterations
k = 1
k = 2
k = 4
36
![Page 37: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/37.jpg)
Multigrid – a first glance Two levels : coarse and fine grid
1 2 3 4 5 6 7 8
1 2 3 4 A2hx2h=b2h
Ahxh=bhh
Ax=b
2h
37
![Page 38: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/38.jpg)
Idea 1: the V-cycle iteration
Also called the nested iteration
A2hx2h = b2h A2hx2h = b2h
h
2h
Ahxh = bh
Iterate =>
)0(2holdx
)0(2hnewx
Start with
Prolongation: )0(2hnewx
)0(holdx Restriction: )0(h
newx)1(2h
oldx
Iterate to get )0(hnewx
Question 1: Why we need the coarse grid ?
38
![Page 39: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/39.jpg)
Prolongation Prolongation (interpolation) operator
xh = x2h
1 2 3 4 5 6 7 8
1 2 3 4
hhI2
hhI2
21
21
21
21
21
21
21
2
1
1
1
1
hhI
39
![Page 40: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/40.jpg)
Restriction Restriction operator
xh = x2hhhI2
hhI2
1 2 3 4 5 6 7 8
1 2 3 4
010
010
010
01
2hhI
40
![Page 41: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/41.jpg)
Smoothing The basic iterations in each level
In ph: xphold xph
new
Iteration reduces the error, makes the error smooth geometrically.So the iteration is called smoothing.
41
![Page 42: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/42.jpg)
Why multilevel ? Coarse lever iteration is cheap. More than this…
Coarse level smoothing reduces the error more efficiently than fine level in some way .
Why ? ( Question 2 )
42
![Page 43: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/43.jpg)
Error restriction Map error to coarse grid will make the
error more oscillatory
K = 4, = /2
K = 4, =
43
![Page 44: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/44.jpg)
Idea 2: Residual correction
Known current solution x
Solve Ax=b eq. to
MG do NOT map x directly between levels
Map residual equation to coarse level1. Calculate rh
2. b2h= Ih2h rh ( Restriction )
3. eh = Ih2h x2h ( Prolongation )
4. xh = xh + eh
**
*
exx
rAeSolve
44
![Page 45: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/45.jpg)
Why residual correction ? Error is smooth at fine level, but the
actual solution may not be. Prolongation results in a smooth error
in fine level, which is suppose to be a good evaluation of the fine level error.
If the solution is not smooth in fine level, prolongation will introduce more high frequency error.
45
![Page 46: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/46.jpg)
`
Revised V-cycle with idea 2
Smoothing on xh
Calculate rh
b2h= Ih2h rh
Smoothing on x2h
eh = Ih2h x2h
Correct: xh = xh + eh
Restriction
Prolongation
2h h
46
![Page 47: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/47.jpg)
What is A2h
Galerkin condition
hh
hhh
h IAIA 222
hhhhhh
hhh
hhh
hhhh
hhhhhh
h
bxArIxIAI
rxIAeAxIe22222
22
22
22
47
![Page 48: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/48.jpg)
Going to multilevels
V-cycle and W-cycle
Full Multigrid V-cycle
h
2h
4h
h
2h
4h
8h
48
![Page 49: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/49.jpg)
Performance of Multigrid Complexity comparison
Gaussian elimination O(N2)
Jacobi iteration O(N2log)Gauss-Seidel O(N2log)SOR O(N3/2log)Conjugate gradient O(N3/2log)Multigrid ( iterative ) O(Nlog)Multigrid ( FMG ) O(N)
49
![Page 50: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/50.jpg)
Summary of MG ideasImportant ideas of MG1. Hierarchical iteration2. Residual correction3. Galerkin condition4. Smoothing the error:
high frequency : fine grid low frequency : coarse grid
50
![Page 51: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/51.jpg)
AMG :for unstructured grids Ax=b, no regular grid structure Fine grid defined from A
1
2
3 4
5 6
311000
130010
102001
000411
010131
001113
A
51
![Page 52: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/52.jpg)
Three questions for AMG How to choose coarse grid
How to define the smoothness of errors
How are interpolation and prolongation done
52
![Page 53: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/53.jpg)
How to choose coarse grid Idea:
C/F splitting As few coarse grid
point as possible For each F-node, at
least one of its neighbor is a C-node
Choose node with strong coupling to other nodes as C-node
1
2
3 4
5 6
53
![Page 54: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/54.jpg)
How to define the smoothness of error
AMG fundamental concept: Smooth error = small residuals ||r|| << ||e||
54
![Page 55: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/55.jpg)
How are Prolongation and Restriction done
Prolongation is based on smooth error and strong connections
Common practice: I
55
![Page 56: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/56.jpg)
AMG Prolongation (2)
56
![Page 57: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/57.jpg)
AMG Prolongation (3)
Thh
hh ICI )( 22
Restriction :
57
![Page 58: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/58.jpg)
Summary Multigrid is a multilevel iterative
method. Advantage: scalable If no geometrical grid is available, try
Algebraic multigrid method
58
![Page 59: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/59.jpg)
59
Pivoting
LU
GMRES,
BiCGSTAB, …
Cholesky
Conjugate gradient
MultiGrid
DirectA = LU
Iterativey’ = Ay
Non-symmetric
Symmetricpositivedefinite
More Robust Less Storage (if sparse)
More Robust
More General
The landscape of Solvers
![Page 60: CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods (II) Chung-Kuan Cheng](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c745503460f94926592/html5/thumbnails/60.jpg)
60
References
G.H. Golub and C.F. Van Loan, Matrix Computataions, 4th Edition, Johns Hopkins, 2013
Y. Saad, Iterative Methods for Sparse Linear Systems, Second Edition, SIAM, 2003.