multigrid tutorial

APPM 7400

Topics in Applied Math

Section 003

Multigrid Methods

APPM 7400

Topics in Applied Math

Section 003

Multigrid Methods

U-Boulder 1 of 312

Steve McCormickECOT 233 492-0662/442-0724

[email protected]://amath.colorado.edu/faculty/stevemOffice hours: M@11, W@9, others TBA

APPM 7400-003Syllabus

• Coursework– No exams– Homework exercises & computing assignments– Project & presentation--joint is OK– Can tailor some or all work & later lectures to your interests

• Philosophy– Big on fundamentals, principles, & simple ideas– Think about the scientific method in general

U-Boulder 2 of 312

– Think about the scientific method in general– Understand by concrete examples & experience– Know the whole picture– Ask questions & interject comments– Make sure I explain what you need

• Text– A Multigrid Tutorial, 2nd edition, 2nd printing– Supplemental material as needed

Sources

MGNet http://www.mgnet.org/

NewsletterSoftware repository

Copper Mountain ConferenceMarch 30--April 4, 2003We hope to support your attendance.

U-Boulder 3 of 312

We hope to support your attendance.

Course web site:

http://amath.colorado.edu/courses/7400/2003Spr/003/

Tech support: John <[email protected]>

Homework: Ollie <[email protected]>

Homework exercises

1: 3,4

2: 1-3, 8, 10, 13, 16

6: 1, 2, 6

7: 1, 2, 10, 12, 15, 21

Due 1 week after the relevant chapter is covered.

U-Boulder 4 of 312

2: 1-3, 8, 10, 13, 16

3: 2-6

4: 6, 7, 11

5: 1-3, 12, 13

7: 1, 2, 10, 12, 15, 21

8: 2, 6, 8

9: 1, 3

10: 1, 5, 8, 9

Computing assignments

2: 20 4: 13, 15 6: 6 by computer!!!

Due 2 weeks after the relevant chapter is covered.Start assignments very early: computers are merciless!!!Use whatever computing language/platform you like.

U-Boulder 5 of 312

7: Convert your 2-D code developed for Chapter 4 (standard coarsening & point relaxation) to solve (7.14). Verify the code using εεεε = 1. Test its performance using εεεε = 1/2,1/5, 1/10, 2, 5, 10. Interpret your results.

Project & presentation

• I am VERY flexible as to what constitutes a project.

• Think about what has captured your attention.

• See me, but only after you’ve thought a lot about it.

• Computing &/or theory &/or application &/or exploration.

Due at the end of the semester.

U-Boulder 6 of 312

• Computing &/or theory &/or application &/or exploration.

• Presentation = in-class short talk &/or hand-in.

• In-class talk: Prepare! Speak to your peers, not me.

• This is meant to be instructive & FUN!

• Check course web site for a bit more detail.

My thoughts on what’s good sciencein order of importance to me

• Attitude– You can do it! Be positive.– But is it really right? Be critical.– Think, think, think! Don’t just hope or guess.– Control your emotions! Expect ups & downs in order to control them.

• Method– Start simply. Reduce issue to the simplest possible case.– Think with tiny steps, but keep the big picture in mind.– Study concrete examples.

U-Boulder 7 of 312

– Study concrete examples.– Look for analogies. Can A be done in any way like how B was done?

• Creativity– What do you really want? What end are you really aiming for?– What do you really need? What you’re trying may be sufficient to do what you want, but would an easier-to-prove weaker result do instead?

• Intelligence– OK, so it doesn’t hurt to try to be “smart” too.

A Multigrid Tutorial2nd Edition, 2nd Printing

By

William L. BriggsCU-Denver

U-Boulder 8 of 312

CU-Denver

Van Emden HensonLLNL

Steve McCormickCU-Boulder

← THANKS!

Outlineby chapter

1. Model Problems

2. Basic Iterative Methods

Convergence tests

Analysis

3. Elements of Multigrid

Relaxation

6. Nonlinear ProblemsFull approximation scheme

7. Selected ApplicationsNeumann boundariesAnisotropic problemsVariable meshesVariable coefficients

U-Boulder 9 of 312

Relaxation

Coarsening

4. Implementation

Complexity

Diagnostics

5. Some Theory

Spectral vs. algebraic

Variable coefficients 8. Algebraic Multigrid (AMG)

Matrix coarsening9. Multilevel Adaptive Methods

FAC10. Finite Elements

Variational methodology

Suggested readingCHECK MY MG LIBRARY & MGNET REPOSITORY

• A. Brandt, “Multi-level Adaptive Solutions to Boundary Value Problems,”Math Comp., 31, 1977, pp 333-390.

• A. Brandt, “Multigrid techniques: 1984 guide with applications to computational fluid dynamics,” GMD, 1984.

• W. Hackbusch, “Multi-Grid Methods & Applications,” Springer, 1985.

• W. Hackbusch & U. Trottenberg, “Multigrid Methods”, Springer-Verlag,

U-Boulder 10 of 312

• W. Hackbusch & U. Trottenberg, “Multigrid Methods”, Springer-Verlag, 1982.

• S. McCormick, ed., “Multigrid Methods,” SIAM Frontiers in Applied Math. III, 1987.

• U. Trottenberg, C. Oosterlee, & A. Schüller, “Multigrid,” Academic Press, 2000.

• P. Wesseling, “An Introduction to Multigrid Methods,” Wylie, 1992.

Multilevel methods have been developed for...

• PDEs, CFD, porous media, elasticity, electromagnetics.• Purely algebraic problems, with no physical grid; for example, network & geodetic survey problems.

• Image reconstruction & tomography.• Optimization (e.g., the traveling salesman & long transportation problems).

U-Boulder 11 of 312

• Optimization (e.g., the traveling salesman & long transportation problems).

• Statistical mechanics, Ising spin models.• Quantum chromo dynamics.• Quadrature & generalized FFTs.• Integral equations.

Everyone uses multilevel methods

• Multigrid, multilevel, multiscale, multiphysics, …

Use local “governing rules” on the finest level to resolve the state of the system at these detailed scales, but--recognizing that these “rules” have broader implications that are hard to determine there--use coarser levels to resolve larger scales.

U-Boulder 12 of 312

there--use coarser levels to resolve larger scales. Continual feedback is essential because improving one scale impacts other scales.

• Common uses

Sight, art, politics, thinking (scientific research), cooking, team sports, …

1. Model problems• 1-D boundary value problem:

• Grid:

≥σ,<<)(=)(σ+)(″− xxfxuxu 010

uu =)(=)( 010

h =1 ...,,=,=, Nihixi 10

U-Boulder 13 of 312

• Let & for .

Nh =

xuv ii )(≈ xff ii )(≈

...,,=,=, Nihixi 10

Ni ...,,= 10

x0 x1 x2 xi xN

x = 0 x = 1

This discretizes the variables, but what about the equations?

Approximate u’’(x) via Taylor series

• Approximate 2nd derivative using Taylor series:

hOxuh

xuh

xuhxuxu )(+)(′′′+)(″+)(′+)(=)( iiiii +4

32

1 !3!2

hh32

0 0

U-Boulder 14 of 312

hOxuh

xuh

xuhxuxu )(+)(′′′−)(″+)(′−)(=)( iiiii −4

32

1 !3!2

hOh

xuxuxuxu )(+

)(+)(−)(=)(″

iii

i

−+ 2

2

11 2

• Summing & solving:

Approximate equation via finite differences

• Approximate the BVP

by a finite difference scheme:

≥σ,<<)(=)(σ+)(″− xxfxuxu 010

uu =)(=)( 010

U-Boulder 15 of 312

by a finite difference scheme:

−...,,==σ+−+−

Nifvh

vvv

ii

iii +−

2

11121

2

vv0 == N 0

Discrete model problem

Letting &

we obtain the matrix equation Av = f, where Ais (N-1) x (N-1), symmetric, positive definite, &

v ),...,,(= vvv 121T

N −

f ),...,,(= fff 121T

N −

U-Boulder 16 of 312

is (N-1) x (N-1), symmetric, positive definite, &

f

f

f

f

f

f

v

v

v

v

v

h

h

h

h

h

hA

=,

=,

σ+−

−σ+−

−σ+−

−σ+−

−σ+

=

21

121

121

121

12

1

1

2

3

2

1

1

2

3

2

1

2

2

2

2

2

2v

N

N

N

N

−

−−

−

Stencil notation

A = [-1 2 -1]dropping h -2 & σ for convenience

U-Boulder 17 of 312

2

-1-1

Basic solution methods• Direct

– Gaussian elimination– Factorization– Fast Poisson solvers (FFT-based, reduction-based, …)

• Iterative– Richardson, Jacobi, Gauss-Seidel, … – Steepest Descent, Conjugate Gradients, …– Incomplete Factorization, ...

U-Boulder 18 of 312

– Incomplete Factorization, ...

• Notes: – This simple 1-D problem can be solved efficiently in many ways. Pretend it can’t & that it’s very hard, because it shares many characteristics with some very hard problems. If we keep things as simple as possible by studying this model, we’ve got a chance to really understand what’s going on.

– But, to keep our feet on the ground, let’s go to 2-D anyway…

2-D model problem

• Consider the problem

• Consider the grid

yyxxu ≥σ;=,=,=,=,= 010100

<<,<<,),(=σ+−− yxyxfuuu yyxx 1010

U-Boulder 19 of 312

• Consider the grid

Nh

Mh yx ,=,=

11

),(=),( hjhiyx yxji

0 ≤≤ Mi

0 ≤≤ Nj

x

y

z

Discretizing the 2-D problem

• Let & . Again, using 2nd-order finite differences to approximate & we arrive at the approximate equation for the unknown , for i =1,2, … M-1 & j =1,2, …, N-1:

yxuv jiji ),(≈ yxff jiji ),(≈u xx u yy

yxu ),( ji

−+−−+− vvvvvv jijijijijiji +,−,,+,− 1111 22

U-Boulder 20 of 312

• Ordering the unknowns (& also the vector f )lexicographically by y-lines:

=σ+−+−

+−+−

fvh

vvv

h

vvv

jijiy

jijiji

x

jijiji +,−,,+,−

2

11

2

11 22

MjjMiiv ji =,=,=,=,= 000

v = (v1,1 , v1,2 ,..., v1,N −1 , v2,1 , v2 ,2 ,..., v2 ,N−1 ,..., vM−1,1 ,vM−1,2 ,..., vM−1,N−1 )T

Resulting linear system• We obtain a block-tridiagonal system Av = f :

where Iy is a diagonal matrix with on the diagonal &

=

−

−−

−−

−−

−

f

f

f

f

v

v

v

v

AI

IAI

IAI

IAI

IA

1

3

2

1

1

3

2

1

1

2

3

2

1

NN

Ny

yNy

yy

yy

y

−−−

−

1

U-Boulder 21 of 312

diagonal & 1

hy2

Ai =

2

hx

2 +2

hy

2 +σ −1

hx

2

−1

hx

2

2

hx

2+

2

hy

2+σ −

1

hx

2

−1

hx2

2

hx2

+2

hy2

+σ −1

hx2

−1

hx

2

2

hx

2+

2

hy

2+ σ

Stencilspreferred for grid issues

Stencils are much better for showing the grid picture:

• • •

• • •

0 −1

hy2

0

again dropping h -2 & σ

-1

-1-1

U-Boulder 22 of 312

• • •

• • •

hy

−1

hx2

2

hx2 +

2

hy2 +σ

−1

hx2

0 −1

hy2

0

Stencils show local relationships--grid point interactions.

-1

-1-1 4

Outline

√√√√• Model Problems

• Basic Iterative Methods

– Convergence tests

– Analysis

• Elements of Multigrid

– Relaxation

• Nonlinear Problems

– Full approximation scheme

• Selected Applications

– Neumann boundaries

– Anisotropic problems

– Variable meshes

Chapters 1-5: Chapters 6-10:






– Variable meshes

U-Boulder 23 of 312

– Relaxation

– Coarsening

• Implementation

– Complexity

– Diagnostics

• Some Theory

– Spectral vs. algebraic

– Variable meshes

– Variable coefficients

• Algebraic Multigrid (AMG)

– Matrix coarsening

• Multilevel Adaptive Methods

– FAC

• Finite Elements

– Variational methodology

– Variable meshes





– FAC

• Finite Elements


2. Basic iterative methods

• Consider Au = f where A is NxN & let v be an approximation to u. (Generic A uses N not N-1.)

• Two important measures:

– The Error: with norms

U-Boulder 24 of 312

– The Error: with norms

– The Residual: with

vue −=

||=|||| ee ∞ xam i =|||| ee 212 ∑ i

N

i =

vAfr −=

|||| r ∞ |||| r 2

What does||r|| measure???

&

& Why have bothr & e ???

Residual correction

r = f - Av= Au - Av= A(u - v)= Ae

• Since e = u - v, we can write Au = f as A(v + e) = f

which means that Ae = f - Au ≡ r.

• Residual Equation:

U-Boulder 25 of 312

reA =

evu +=

What does thisdo for us???

= Ae

• Residual Correction:

Relaxation

• Consider the 1-D model problem

• Jacobi (simultaneous displacement): Solve the i th

==−≤≤=−+− uuNifhuuu Niiii +− 02

11 0112

U-Boulder 26 of 312

• Jacobi (simultaneous displacement): Solve the i thequation for holding all other variables fixed:

vi

Nifhvvv idlo

idlo

iwen

i)(

+)(

−)( −≤≤)++(= 11

2

1 211

Jacobi in matrix form

• Let A = D - L - U, where D is diagonal & L & U are the strictly lower & upper triangular parts of A.

• Then becomesfuA =

fuULuD +)+(==)−−( fuULD

U-Boulder 27 of 312

• Let .– RJ is called the error propagation or iteration matrix.

• Then the iteration is

fuULuD +)+(=

fDuULDu +)+(= −− 11

ULDRJ )+(= − 1

fDvRv−)()( dlo

Jwen += 1

J = D-1(D-A) = I-D-1A

Error propagation matrix & the error• From the derivation,

• the iteration is

• subtracting,

fDuULDu +)+(= −− 11

fDuRu += J− 1

fDvRv−)()( dlo

Jwen += 1

U-Boulder 28 of 312

• subtracting,

• or

• hence

fDvRv )()( dloJ

wen +=

e )()( dloJ

wen = eR

vRuRvu −=− )()( dloJJ

wen

fDvRfDuRvu )+(−+=− −)(−)( dloJJ

wen 11

Error propagation! J = I-D-1A

A pictureRJ = D-1 (L + U) = [ 0 ]

so Jacobi is an error averaging process:

ei(new) ← (ei-1(old) + ei+1(old))/2

12

12

U-Boulder 29 of 312

But…

U-Boulder 30 of 312

Another matrix look at Jacobiv (new) ← D-1 (L + U) v (old) + D-1 f (L + U = D-A)

= (I - D-1 A) v (old) + D-1 f

v (new) = v (old) - D-1 (Av (old) - f) = v (old) - D-1 r

• Exact: u = u - D-1 (Au - f)

• Subtracting: e (new) = e (old)- D-1 Ae (old)

U-Boulder 31 of 312

• Subtracting: e (new) = e (old)- D-1 Ae (old)

• General form: u = u - B (Au - f) with B ~ A-1

• Damped Jacobi: u = u - ωD-1 (Au - f) with 0<ω<2

• Gauss-Seidel: u = u - (D - L)-1 (Au - f)

• Exact: u = u - A-1 (Au - f) = A-1f

Weighted Jacobisafer (0<ω<1) changes; politics in the ‘hood

• Consider the iteration

• Letting A = D-L-U, the matrix form is

fhvvvv idlo

idlo

idlo

iwen

i)(

+)(

−)()( )++(

ω+)ω−(←

21

211

fDhvULDIv−)(−)( dlowen ω+)+(ω+)ω−(= 1

121

• • •

• o •

• • •

U-Boulder 32 of 312

• Note that

• It is easy to see that if , then

fDhvULDIv−)(−)( dlowen ω+)+(ω+)ω−(= 1

121

ω+= fDhvR−)(ω dlo 12

RIRω ]ω+)ω−([= 1 J

uue −≡ )()( xorppatcaxe

eRe )(ω)( dlowen =

Gauss-Seidel (1-D)

• Solve equation i for ui & update immediately.• Equivalently: set each component of r to zero in turn.• Component form: for set

• Matrix form:

Ni ,−...,,= 121

fhvvv iiii )++(←2

1+−

211

U-Boulder 33 of 312

• Let

• Then iterate:

• Error propagation:

ULDA )−−(=

+=)−( fuUuLD

fLDuULDu )−(+)−(= −− 11

ULDRG )−(= − 1

fLDvRv−)()( dlo

Gwen )−(+← 1

eRe )()( dloG

wen ←

Red-black Gauss-Seidel• Update the EVEN points:

• Update the ODD points:

fhvvv 22

12122 iiii )++(←2

1+−

fhvvv 122

22212 iiii +++ )++(←2

1

U-Boulder 34 of 312

• 2-D:

1222212 iiii +++ 2

x0 xN

Numerical experiments• Solve ,

• Use Fourier modes as initial iterates, with N = 64:uA = 0 =−+− uuu iii +− 11 02

vk

π

=)(=N

kiv ki nis 1111 −≤≤,−≤≤ NkNi

component mode

sin(kπx), x =i/N

U-Boulder 35 of 312

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

k = 3

k = 1

k = 6

sin(kπx), x =i/N

Convergence factors differ for different error components

Error, ||e||∞∞∞∞ , in weighted (ω=2/3) Jacobi on Au = 0for 100 iterations using initial guesses v1, v3, & v6

0.9

1k = 1

U-Boulder 36 of 312

0 20 40 60 80 100 1200

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

k = 3

k = 6

Stalling convergencerelaxation shoots itself in the foot

• Weighted (ω=2/3) Jacobi on 1-D problem.

• Initial guess:

• Error plotted against iteration number:N

j

N

j

N

jv0

π

+

π

+

π

=23

nis6

nisnis3

1

|||| e ∞

U-Boulder 37 of 312

0 10 20 30 40 50 60 70 80 90 1000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Analysis of stationary linear iteration

• Let v(new) = Rv(old) + g . The exact solution is unchanged by the iteration: u = Ru + g .

• Subtracting: e(new) = Re(old) .

• Let e(0) be the initial error & e(i) be the error after the i th iteration. After n iterations, we have

e(n) = Rne(0) .

U-Boulder 38 of 312

e(n) = Rne(0) .

I can deal with 24, but ???

a b c

d e f

g h i

4

a b c

d e f

g h i

4

What if e = 24 e ???

Quick review of eigenvectors & eigenvalues

• The number λ is an eigenvalue of a matrix B & w ≠ 0its associated eigenvector if Bw = λw.

• The eigenvalues & eigenvectors are characteristicsof a given matrix.

• Eigenvectors are linearly independent, & if there is a complete set of N distinct eigenvectors for an NxN

U-Boulder 39 of 312

• Eigenvectors are linearly independent, & if there is a complete set of N distinct eigenvectors for an NxNmatrix, then they form a basis: for any v, there exist unique scalars vk such that

• Propagation: v = vk

k=1

N

Σ wk

Bnv = λn vk

k=1

N

Σ wk

Why is aneigenvectoruseful???

“Fundamental Theorem of Iteration”

• R is convergent (Rn → 0 as n → ∞) iff.

Thus, e(n) = Rne(0) → 0 for any initial vector v (0)

iff ρ(R)<1.

ρ(R) = max1≤k≤N λk <1

U-Boulder 40 of 312

• ρ(R)<1 assures convergence of the iteration for R.

• ρ(R) is the spectral convergence factor.– But ρ is doesn’t tell you much by itself--it’s generally valid only asymptotically. It’s useful for the symmetric case in particular, so we’ll accept it for now, but first a little background & then a warning…

Rayleigh quotient vs. spectral radius assume A is symmetric (wk orthogonal) & nonnegative definite (λ ≥ 0)

• RQ(v) ≤ ρ(A): v = Σvkwk

RQ(v)=

< Av,v >< v,v >

=< A Σvk wk ,Σvk wk >< Σ vk wk , Σ vk wk >

< Σλ v w , Σv w > Σλ v2

≤ λλλλ = ρρρρ(A)≥ λλλλ

U-Boulder 41 of 312

• supv ≠ 0 RQ(v) = ρ(A):

RQ(wN ) =< AwN ,wN >< wN ,wN >

=< λN wN ,wN >

< wN , wN >= λN = ρ(A)

=< Σλk vk wk , Σvk wk >< Σvk wk , Σvk wk >

= Σλk vk2

Σvk2

≤ λλλλN = ρρρρ(A)≥ λλλλ1

Euclidean norm vs. spectral radius use RQ

• ||R||2 = ρ1/2(RTR):

||R||22 = supe≠0 ||Re||22 /||e||22

= supe≠0 <Re,Re >/<e,e>= sup <RTRe,e >/<e,e> = ρ(RTR)

U-Boulder 42 of 312

= supe≠0 <RTRe,e >/<e,e> = ρ(RTR)

note: ||Re||2 ≤ ||R||2·||e||2

• ||A||2 = ρ1/2(A2) = ρ(A) for symmetric A!!!

ρ(R) vs. ||R||2

ρ(R) = sup |λ(R)|.

ρ(R) is norm independent.

ρ(R) is asymptotic: could get large ||R||2 but ρ(R)<1⇒ “convergence”. Next slide.

||R||2=supe≠0 ||Re||2 /||e||2= ρ1/2(RTR).

||R||2 depends on ||· ||2.

||R||2 bounds ||e(n + 1)||2 /||e(n)||2: worst case! Probably pessimistic initially, but sharp sooner or later.

U-Boulder 43 of 312

2

⇒ “convergence”. Next slide. initially, but sharp sooner or later.

ρ(R) = inf ||R||taken over all matrix norms.

ρ(R) = ||R||2for symmetric R.

Example: R = ( ), K largeρ(R) = 0 but ||R||2 = K !

e(0) =

0

1

⇒

e(1)

2

e(0)

2

=Re(0 )

2

e(0 )

2

= K

Thus, 1 iteration with e(0) shows dramatic L2 divergence!

U-Boulder 44 of 312

Thus, 1 iteration with e(0) shows dramatic L2 divergence!But R2 = 0, so e(2) = Re(1) = R2e(0) = 0!

Thus, 2 iterations with e(0) show complete convergence!

On one hand, this is special (λ=0, large K, 2x2), so thisbehavior would be more subtle & persistent in general.

On the other, this behavior would vanish for symmetric R.

Convergence factor & rate• How many iterations are needed to reduce the initial error by 10-d ?

• So, we have

e(n )

e(0 )≤ R

n ≤ Rn

~ 10−d

d

U-Boulder 45 of 312

• Convergence factor = ||R|| or ρ(R).

• Convergence rate = or -log10(ρ(R)).

n ~d

− log10

ρ(R)

− log10 R digits

iteration

Convergence analysis: Weighted Jacobi

ω−= ADI− 1

IRω

−−

−−−

ω−= 121

121

12

21-D

U-Boulder 46 of 312

IRω −

⋅⋅⋅

⋅⋅⋅

⋅⋅⋅

−−

−=

21

1212

For our 1-D model, the eigenvectors of weighted Jacobi Rω & the eigenvectors of A are the same!

Why???

)(λω

−=)(λ ARω2

1

The eigenvalues are related as well.

1-D

Eigenpairs of A

The eigenvectors of A are Fourier modes!

π

=,

π

=)(λ jkk N

kjw

N

kA nis

2nis4 2

,[-1 2 -1]

λλλλN-1 ≅≅≅≅ 4λλλλ1 ≅ π≅ π≅ π≅ πh2

0.4

0.6

0.8

1

0.4

0.6

0.8

1

N = 64

U-Boulder 47 of 312

0 10 20 30 40 50 60

-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0 10 20 30 40 50 60

-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0 10 20 30 40 50 60

-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

0 10 20 30 40 50 60

-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

0 10 20 30 40 50 60

-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

N = 64

k = 1 k = 2

k = 4 k = 8 k = 16

Eigenvectors of Rω = eigenvectors of A

• Expand the initial error in terms of the eigenvectors:

π

ω−=)(λk N

kRω

2nis21 2

e(0) = ck wk

N−1

∑

U-Boulder 48 of 312

• After n iterations:

• The kth error mode is reduced by λk (Rω) each iteration.

Rne

(0 ) = ckλk

nwk

k=1

N −1

∑

e = ck wk

k=1

∑

Relaxation suppresses eigenmodes unevenly

• Look carefully at .

π

ω−=)(λk N

kRω

2nis21 2

Note that if 0 < ω ≤ 1, then for

.<|)(λ| k Rω 10.6

0.8

1

/=ω 31

U-Boulder 49 of 312

.

For 0 < ω ≤ 1,Nk −,...,,= 121

π

ω−=λ 21 2

nis21N

π

ω−=2

nis21 2 h

≈)(−= 11 hO2-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

/=ω 21

/=ω 32=ω 1

N

2

0 Nk

λλλλ

Low frequencies are “undamped”

Notice that no value of will efficiently damp out long waves or low frequencies.

ω

What value of gives the best damping of short waves or high

ω

0.6

0.8

1

/=ω 31

U-Boulder 50 of 312

short waves or highfrequencies N/2≤k≤N-1?

Choose such that

)(λ−=)(λ NN

2

RR ωω

ω

=ω⇒3

2-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

/=ω 21

/=ω 32=ω 1

N

2

0 Nk axis

λλλλaxis

Smoothing factor

• The smoothing factor is the largest magnitude of the iteration matrix eigenvalues corresponding to the oscillatory Fourier modes:smoothing factor = max |λk(R)| for N/2≤k≤N-1.

• Why only the upper spectrum? “MG” spectral radius?

U-Boulder 51 of 312

• Why only the upper spectrum?• For Rω with ω=2/3, the smoothing factor is 1/3:

|λN/2|=|λN|=1/3 & |λk|<1/3 for N/2<k<N.

• But |λk| ≈ 1 - ωk2π2h2 for long waves (k << N/2).

“MG” spectral radius?

Convergence of Jacobi on Au = 0

20

30

40

50

60

70

80

90

100

20

30

40

50

60

70

80

90

100Unweighted Jacobi Weighted Jacobi

U-Boulder 52 of 312

• Jacobi on Au = 0 with N = 64. Number of iterations needed to reduce initial error ||e||∞ by 0.01.

• Initial guess :

=N

jkvkj

πsin

0 10 20 30 40 50 600

10

20

0 10 20 30 40 50 600

10

Wavenumber, k Wavenumber, k

Weighted Jacobi = smoother (error)

• Initial error:

- 2

- 1

0

1

2

Many relaxation schemes are smoothers:

N

j

N

j

N

jv jk

π

+

π

+

π

=23

nis2

161nis

2

12nis

U-Boulder 53 of 312

• Error after 35 iteration sweeps:

0 0 . 5 1

- 2

0 0 . 5 1

- 2

- 1

0

1

2

are smoothers: oscillatory error modes are quickly eliminated,

butsmooth modes are slowly

damped.

Similar analysis for other smoothers

• Gauss-Seidel relaxation applied to the 3-point difference matrix A (1-D model problem):

• A little algebra & trigonometry shows thatULDRG )−(=

− 1

w = cosj kπ sin

jkπ λ (R ) = cos2 kπ

U-Boulder 54 of 312

wk , j = cosj kπ

N

sin

jkπN

0 10 20 30 40 50 600

0.2

0.4

0.6

0.8

1

λλλλk

sin(3ππππ xj )

What’s wk look like for large k ?

λλλλ3 sin(3ππππxj )j

0 10 20 30 40 50 60

-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

jk

λk (RG ) = cos2 kπ

N

Gauss-Seidel eigenvectors

0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

k=1,n=64

0 0.2 0.4 0.6 0.8 1−0.02

0

0.02

0.04

0.06

0.08

0.1

k=30,n=64

U-Boulder 55 of 312

0 0.2 0.4 0.6 0.8 10

0 0.2 0.4 0.6 0.8 1−0.02

0 0.2 0.4 0.6 0.8 1−0.5

−0.4

−0.3

−0.2

−0.1

0

0.1

0.2

k=48,n=64

0 0.2 0.4 0.6 0.8 1−1

−0.5

0

0.5

1

k=60,n=64

Gauss-Seidel convergenceAu = 0

Eigenvectors of RG are not the same as those of A.Gauss-Seidel mixes the modes of A.

90

100 Gauss-Seidel on Au = 0, withN = 64. Number of iterations

U-Boulder 56 of 312

0 10 20 30 40 50 600

10

20

30

40

50

60

70

80N = 64. Number of iterations needed to reduce initial error ||e||∞ by 0.01.

Initial guess (modes of A):

=N

jkvkj

πsin

Outline




– Analysis


– Relaxation






– Variable meshes


√√√√

U-Boulder 57 of 312

– Relaxation

– Coarsening

• Implementation

– Complexity

– Diagnostics

• Some Theory


– Variable meshes





– FAC

• Finite Elements


• Many relaxation schemes have the smoothing property: oscillatory error modes are quickly eliminated, while smooth modes are often very slow to disappear.

• We’ll turn this adversity around: the idea is to

3. Elements of multigrid

1st observation toward multigrid

U-Boulder 58 of 312

• We’ll turn this adversity around: the idea is to use coarse grids to take advantage of smoothing.

x0 xN

x0 xN

2

Ωh

Ω2h

How?

Reason #1 for coarse grids:Nested iteration

• Coarse grids can be used to compute an improved initial guess for the fine-grid relaxation. This is advantageous because:

– Relaxation on the coarse-grid is much cheaper: half as many points in 1-D, one-fourth in 2-D, one-eighth in 3-D,…

U-Boulder 59 of 312

many points in 1-D, one-fourth in 2-D, one-eighth in 3-D,…

– Relaxation on the coarse grid has a marginally faster(|λ1(R)| ≈ 1 - ωπ2h2 ):

instead of

1 )(− hO241 )(− hO

2

Idea! Nested iteration

• Relax on Au = f on Ω4h to obtain initial guess v2h.

• Relax on Au = f on Ω2h to obtain initial guess vh.

• Relax on Au = f on Ωh to obtain … final solution???

•••

U-Boulder 60 of 312

• What is A2hu2h = f2h? Analogous to Ahuh = fh for now.• How do we migrate between grids? Hang on…

• What if the error still has large smooth components when we get to the fine grid Ωh ? Stay tuned for 2nd

observation toward multigrid…

Reason #2 for coarse grids:Smooth error becomes more oscillatory

• A smooth function:

0 0 . 5 1- 1

- 0 . 5

0

0 . 5

1On the coarse grid, the smooth error appears tobe relatively higher in

frequency: in this exampleit is the 4-mode out ofa possible 15 on the finegrid, ~1/4 the way up thespectrum. On the coarse

U-Boulder 61 of 312

can be represented by linear interpolation from a coarser grid:

0 0 . 5 1- 1 spectrum. On the coarse

grid, it is the 4-mode outof a possible 7, ~1/2 theway up the spectrum.

Relaxation on 2h is(cheaper &) faster

on this mode!!!

For k=1,2,…N/2, the kth mode is preserved on the coarse grid.

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

wN

kj

N

kjw h

jkh

jk ,,2

2 =/

π

=

π

=2

nis2

nis

Also, note that

U-Boulder 62 of 312

0 2 4 6 8 10 12

-1

-0.8

0 1 2 3 4 5 6

-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

Also, note that

on the coarse grid.wh

N/2 → 0k=4 mode, N=12 grid

k=4 mode, N=6 grid

What happens to the modes

between N/2 & N ?

For k > N/2, wkh is disguised on the coarse grid: aliasing!!!

• For k > N/2, the kth mode on the fine grid is aliased & appears as the (N - k)th

mode on the coarse grid:

-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

π)( kj2

U-Boulder 63 of 312

0 2 4 6 8 10 12

-1

0 1 2 3 4 5 6

-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

)−(π

−=2

nisN

kNj

π)(

=)(N

kjw

j

hk

2

2nis

/

)−(π

−=2

nisN

jkN

)(−= w hkN −

2

k=9 mode, N=12 grid

k=3 mode, N=12 grid

j

1-D interpolation (prolongation)to migrate from coarse to fine grids

• Mapping from the coarse grid to the fine grid:

• Let , be defined on , . Thenvh v2h Ωh Ω2h

I2

2hhh

h Ω→Ω:

U-Boulder 64 of 312

where

vh v2h Ωh Ω2h

vv 22

hi

hi =

vvv 21

212

hi

hi

hi ++ )+(=

2

1 for .12

0 −≤≤N

i

vvI 22

hhhh =

1-D interpolation (prolongation)

• Values at points on the coarse grid map unchanged to the fine grid.

• Values at fine-grid points NOT on the coarse grid

are the averages of their coarse-grid neighbors.

U-Boulder 65 of 312

1-D prolongation operator P

• P = is a linear operator: ℜN/2-1 ℜN-1 .

• N = 8:

//

/ 2121

1

21

3

2

1

21

h

h

h

h v

v

v

v

I2hh

U-Boulder 66 of 312

• has full rank, so η(P ) = 0 .

=

/

//

// 21

1

2121

1

2121

177

6

5

4

3

13

23

22

21

37

x

h

h

h

h

h

x

h

h

h

xv

v

v

v

v

v

v

v

I2hh

When is v2h = 0? I2hh

“Give To” stencil for P

] 1/2 1 1/2 [

o x o x o x o1/2 1 1/2

U-Boulder 67 of 312

o x o x o x o1/2 1 1/2

How well could v 2h approximate u?

• Imagine that a coarse-grid approximation v2h has been found. How well could it approximate the exact solution u ?

U-Boulder 68 of 312

• If u is smooth, a coarse-grid interpolant v2h mightdo very well.

How well could v 2h approximate u?

• Imagine that a coarse-grid approximation v2h has been found. How well could it approximate the exact solution u ?

U-Boulder 69 of 312

• If u is oscillatory, a coarse-grid interpolant v2hcannot work well.

Moral• If what we want to compute is smooth, a coarse-grid interpolant could do very well.

• If what we want to compute is oscillatory, a coarse-grid interpolant cannot do very well.

• What if u is not smooth? Can we make it so?• Can we make something smooth?

U-Boulder 70 of 312

• How about e ? Can we smooth it? How would we get e & use it to get u ? Ae = r & u ← v + e !

• Just because the coarse grid can approximate e well doesn’t mean we know how to do it! But we will soon!

• Thus, use nested iteration on residual equationto approximate the error after smoothing!!!

2nd observation toward multigrid

• The residual equation: Let v be an approximation to the solution of Au = f, where the residual r = f -Av. Then the error e = u - v satisfies Ae = r.

• After relaxing on Au = f on the fine grid, e will be smooth, so the coarse grid can approximate e well. This will be cheaper & e should be more oscillatory there, so relaxation will be more effective.

U-Boulder 71 of 312

there, so relaxation will be more effective.

• Therefore, we go to a coarse grid & relax on the residual equation Ae = r.

e = 0 !What’s a good initial guess on grid 2h?

How do we get back to grid 2h? Stay tuned…

• Assume we’ve relaxed so much that e is smooth.• Ansatz: e = Pv2h for some coarse-grid v2h.• How do we characterize e so we can hope to compute it?

Ae = r ⇒ A P v2h = r7x7 7x3 3x1 = 7x1

• Too many equations now & too few unknowns!

A way to coarsen Ae = r

U-Boulder 72 of 312

• Too many equations now & too few unknowns!

• Multiply both sides by some 3x7 matrix?

P T

P T A P v2h = P Tr3x7 7x7 7x3 3x1 = 3x1)) A2h

3x3

Idea! Coarse-grid correction2-grid

• Relax on Au = f on to get an approximation .

• Compute .

• Relax on Ae = r on to obtain an approximation to

the error, .Ω2h

Ωh

vAfr −= h

U-Boulder 73 of 312

the error, .

• Correct the approximation v h ← v h + e2h. e2h

I2hh

This is the essence of multigrid.

• Clearly, we need a mapping for Ωh → Ω2h.

• Mapping from the fine grid to the coarse grid:

• Let , be defined on , . Then

where .

vh v2h Ωh Ω2h

Ihhh

h22 Ω→Ω:

vvI hhhh

22 =

1-D restriction by injection

U-Boulder 74 of 312

where .vv h

ih

i 22 =

vvIh =

Ωh

Ω2h

1-D restriction by full weighting

• Let , be defined on , . Then

where

.

vh v2h Ωh Ω2h

vvvv hi

hi

hi

hi 122122 )++(= 2

4

1+−

vvI hhhh

22 =

U-Boulder 75 of 312

4

Ωh

Ω2h

1-D restriction (full-weighting)

• R = is a linear operator: ℜN-1 ℜN/2-1 .

• N = 8:

/// 412141 vv

v

v

213

2

1

hh

h

h

Ih

h2

U-Boulder 76 of 312

• has rank , so dim(η(R)) .

=

///

//////

412141

412141

412141

v

v

v

v

v

v

v

v

23

22

1

7

6

5

4

3

h

h

h

h

h

h

∼N

2∼

N

2I

hh2

Look at the columns of R associated with grid 2h.

Prolongation & restriction are often nicely related

• For the 1-D examples, linear interpolation & full weighting are

11

2

1

1 1211

U-Boulder 77 of 312

• So they’re related by the variational condition

for c in ℜ.

IcI2

2

Thh

hh )(=

I2hh

=

1

2

11

2

11

2

1I

hh2

=

121

121

121

4

1

2-D prolongation

vvv 12

212h

jihji

hji ,+,+ )+(=

2

1

vvv 12

122h

jihji

hji +,+, )+(=

2

1

vvvvv 11112

1212h

jih

jih

jihji

hji +,++,,++,+ )+++(=

4

14

1

2

1

4

1

2

11

2

1

4

1

2

1

4

1

U-Boulder 78 of 312

11111212 jijijijiji +,++,,++,+ 4

We denote the operator by using a “give to” stencil ] [. Centered over a C-point , it shows what fraction of

the C-point’s value contributes to a neighboring

F-point .4

1

2

1

4

1

2

11

2

1

4

1

2

1

4

1

“Get From” interpolation

[1/2 1/2]

1/4 1/4

1/4 1/4

Centered over a coarse grid point .

U-Boulder 79 of 312

1/2 1/2

1/4 1/4

1/4 1/4

2-D restriction (full weighting)

61

1

8

1

61

1

8

1

4

1

8

1

61

1

8

1

61

1

U-Boulder 80 of 312

We denote the operator by using a “get from” stencil [ ]. Centered over a C-point , it shows what fraction of the value of the neighboring

F-point contributes to the value at the C-point.

61861

61

1

8

1

61

1

8

1

4

1

8

1

61

1

8

1

61

1

Now we put all these ideas together

• Nested Iteration– effective on smooth solution (components).

• Relaxation – effective on oscillatory error (components).

U-Boulder 81 of 312

• Residual Equation– characterizes the error.– enables nested iteration for smooth error (components)!!!

• Prolongation (variables) & Restriction (equations)– provides pathways between coarse & fine grids.

2-grid coarse-grid correction

1) Relax times on on with arbitrary initial guess .

2) Compute .

3) Compute .

fuAhhh = Ωh

vh

fvGCvhhh ) α,α,,(← 21

α1

vAfr hhhh −=

U-Boulder 82 of 312

3) Compute .

4) “Solve” on .

5) Correct fine-grid solution .

6) Relax times on on with initial guess .

fuAhhh = Ωh

vh

α2

reA 222 hhh = Ω2h

rIr22 hhh

h =

eIvv hhh

hh +← 22

2-grid coarse-grid correction

Relax on fvAhhh

=vAfr hhhh −=Compute

Correct

evv hhh +←

U-Boulder 83 of 312

rIr22 hhh

h =Restrict

Solve reA 222 hhh =rAe 2122 hhh )(= −

eIe hhh

h = 22

Interpolate

What is A2h?

• For this scheme to work, we must have , a coarse-grid operator. For the moment, we will simply assume that is “the coarse-grid version ” of the fine-grid operator .

A2h

A2h

Ah

U-Boulder 84 of 312

• Later we’ll return to the question of constructing .

A2h

Ah

How do we “solve” the coarse-grid residual equation? Recursion!

vIe hh

hh ← 2

2

vIe 422 hhh←

fvGvhhh ),(←

α1

f0Gv222 hhh ),(←

α1

vAfIf22 hhhh

h

h)−(←

vAfIf 22242

4 hhhh

h

h)−(←

evv 222 hhh +←

evv hhh +←α2 = 0

U-Boulder 85 of 312

vIe 424

2 hhh

h←

vIe 84

84 hh

hh ←

f0Gv444 hhh ),(← α1

f0Gv888 hhh

),(←α1

vAfIf 22

hh )−(←

vAfIf 44484

8 hhhh

h

h)−(←

evv 888 hhh +←

evv 444 hhh +←

fAeHHH

)(=− 1

vH=

V-cycle (recursive form)

1) Relax times on with initial given.

2) If is the coarsest grid, go to 4;else:

α1 fuAhhh = vh

Ωh

vAfIf 2 hhh2hh

h)−(←

fvVMvhhhh ),(← , α1, α 2

U-Boulder 86 of 312

3) Correct: .

4) Relax times on with initial guess .

h )−(←

v2h ← 0

fvVMv2222 hhhh ),(←

vIvv hhh

hh +← 22

α2 fuAhhh = vh

Outline




– Analysis


– Relaxation






– Variable meshes







– Variable meshes

√√√√

√√√√

U-Boulder 87 of 312

– Relaxation

– Coarsening

• Implementation

– Complexity

– Diagnostics

• Some Theory


– Variable meshes





– FAC

• Finite Elements


– Variable meshes





– FAC

• Finite Elements


√√√√

Storage cost: vh & fh on each level.

In 1-D, a coarse grid has about half as many points as the fine grid.

In 2-D, a coarse grid has about one-fourth

.

4. Implementation

U-Boulder 88 of 312

In 2-D, a coarse grid has about one-fourth

as many points as the fine grid.

In d-dimensions, a coarse grid has about 2-d

as many points as the fine grid.

21

2222212

NN

d

ddMdddd

−<)+…++++(

−−−−− 32

Total storage cost:

less than 2, 4/3, & 8/7 the cost of storage on the fine

grid for 1-D, 2-D, & 3-D problems, respectively.

Computational cost

• Let one Work Unit (WU) be the cost of one relaxation sweep on the fine grid.

• Ignore the cost of restriction & interpolation (typically about 20% of the total cost).

• Consider a V-cycle with 1 pre-Coarse-Grid (α1 = 1)correction sweep & 1 post-Coarse-Grid (α = 1)

U-Boulder 89 of 312

correction sweep & 1 post-Coarse-Grid (α2 = 1)correction sweep.

• Cost of a V-cycle (in WUs):

• Cost is about 4, 8/3, & 16/7 WUs per

V-cycle in 1, 2, & 3 dimensions.

21

2222212

−<)+...++++(

−−−−−

d

dMddd 32

Convergence analysis• First, a heuristic argument:

– The convergence factor for the oscillatory error modes (smoothing factor) is small & bounded uniformly in h.

smoothing factor = max |λk(R)| for N/2≤k≤N-1.

– Multigrid focuses the relaxation process on attenuating the oscillatory components on each level.

U-Boulder 90 of 312

⇒ The overall multigrid convergence factor is small & bounded uniformly in h !

smoothk=1

oscillatoryk=Nk=N/2

Relax on fine gridRelax 1st coarse grid

Bounded uniformly in h ≠ independent of h.

Convergence analysisa little more precisely...

• Continuous problem: Au = f, u i = u(x i)• Discrete problem: Ah uh = fh, vh ≈ uh

• Global/discretization error: Ei = u(x i) - uih

||E || ≤ Kh p(p = 2 for model problem & proper norm)

U-Boulder 91 of 312

• Algebraic error: eih = uih - vih

• For tolerance ε, assume the aim is to find vh so that the total error ||e || = ||u(h) - vh || ≤ ε , where u(h) = (u (x i)).

• Then this objective is assured as follows:||e || ≤ ||u (h) - u h || + ||u h - v h || = ||E|| + ||e h|| ≤ ε .

We can satisfy the convergence objective by imposing two conditions

1) ||E|| ≤ ε/2. Achieve this condition by choosing an appropriately small grid spacing h*:

K(h*)p = ε/2

2) ||e h|| ≤ ε/2. Achieve this condition by iterating until

||E|| ≤ = K(h*) on grid h; then we’ve

U-Boulder 92 of 312

||E|| ≤ ε/2 = K(h*)p on grid h; then we’ve

converged to the level of discretization error.

Once discretization error & algebraic errorare in balance, then it would be better to

go to grid h/2 than to iterate more!

→→→→

Convergence to the level of discretization error

• Use an MV scheme with convergence factor γ < 1bounded uniformly in h (fixed α1 & α2).

• Assume a d-dimensional problem on an NxNx…xNgrid with h = N -1.

U-Boulder 93 of 312

• Initial error is ||e h || = ||u h - 0 || = ||u h || = O(1) .

• Must reduce this to ||e h || = O(h p) = O(N -p) .

• We can determine θ, the number of V-cycles needed to do this, if we can bound its convergence factor γ.

Work to converge to the level of discretization error

• Using θ V-cycles with convergence factor γ gives an overall convergence factor of γ θ.

• We must therefore have , or .

• Since 1 V-cycle costs O(1) WUs & 1 WUs is O(N d ),

)(∼γ−θ NO

p

)(∼θ NO gol

U-Boulder 94 of 312

• Since 1 V-cycle costs O(1) WUs & 1 WUs is O(N d ), then converging to the level of discretization error using the MV method costs

• This compares to fast direct methods (fast Poisson solvers). But we’ll see that MV can do even better.

)(∼θ NO gol

NNO )( dgol

Numerical example

• Consider the 2-D model problem (with σ = 0):

in the unit square, with u = 0 Dirichlet boundary.)−()−(+)−()−(=−− xxyyyxuu yyxx 1611612 222222

U-Boulder 95 of 312

• The solution to this problem is

• We examine the effectiveness of MV cycling to solve this problem on NxN grids [(N-1) x (N-1)interior points] for N = 16, 32, 64, 128.

yyxxyxu )−()−(=),( 2442

Numerical resultsMV cycling

Shown are the results of 16 V(2,1)-cycles. We display, after each cycle, residual norms, total error norms, & ratios of these norms to their values after the previous

U-Boulder 96 of 312

these norms to their values after the previous cycle.N = 16, 32, 64, 128.

||r h||h = h ||r h||2scaled residual error

||e ||h = h ||u (h) - v h||2scaled discrete total error

A warning about bounds

• Bounds like ||en + 1|| ≤ γ ||en || & ||u (h) - u h|| = O(h)are only just that--bounds!

• If you see behavior that suggests that these bounds are sharp (e.g., halving h halves the discretization error), then great. If you don’t see this behavior, don’t assume things are wrong.

U-Boulder 97 of 312

this behavior, don’t assume things are wrong.

• Think about this:

O(h2 ) = O(h) but generally O(h) ≠ O(h2 ) !!!

(Any process that is O(h2) is also O(h), but the converse isn’t necessarily true.)

Reconsideration

You want to approximate uh.

A good iteration is the V-cycle.

What’s a good way to start it?

Can you do better than vh 0?

U-Boulder 98 of 312

Can you do better than vh ← 0?

Start on the coarse grid!

Use nested iteration for the V-cycle!

Look again at nested iteration

• Idea: It’s cheaper to solve a problem (fewer iterations) if the initial guess is good.

• How to get a good initial guess:

U-Boulder 99 of 312

– “Solve” the problem on the coarse grid first.

– Interpolate the coarse solution to the fine grid.

• Now, let’s use the V-cycle as the solver on each grid level! This defines the Full Multigrid (FMG) cycle.

Full multigrid (FMG)

• Initialize

• Solve on coarsest grid

ffffHhhh

,...,,,42

fAvHHH )(=

− 1

fGMFvhh )(←

U-Boulder 100 of 312

• Interpolate initial guess

• Perform V-cycle

• Interpolate initial guess

• Perform V-cycle

vIv 424

2 hhh

h ←

vIv hhh

h ← 22

fvVMv2222 hhhh ),(←

fvVMvhhhh ),(←

FMG-cycle

• Restriction• Interpolation• High-Order Interpolation?


FMG-cycle (recursive form)

1) Initialize

2) If is the coarsest grid, then solve exactly;

else:Ωh

ffffHhhh

,...,,,42

fGMFvhh )(←

fGMFv22 hh )(←

, η


else:

3) Set initial guess: .

4) Perform η times.vIv hh

hh ← 2

2

fGMFv22 hh )(←

fvVMvhhh ),(←

FMG cycle cost

One V-cycle is performed per level, at a cost ofWUs per grid (where the WU is for the

size of the finest grid involved).The size of the WU for coarse-grid j is times the size of the WU for the fine grid (grid 0).

2− dj

21

2

− − d


the size of the WU for the fine grid (grid 0).Hence, the cost of the FMG(1,1) cycle in WUs is less than

d = 1: 8 WUs; d = 2: 32/9 WUs; d = 3: 128/49 WUs.)−(

=)...+++(−

21

2221

21

2

−

−−− d

dd

d 2

2

Has discretization error been reached by FMG?

• If discretization error is achieved, then ||eh|| = O(h2)& the error norms at the “solution” points in the cycle should form a Cauchy sequence:

||eh|| 0.25 ||e2h||


||eh|| ≈ 0.25 ||e2h||

Let’s see this algebraically…

Interpolation stabilityhow interpolation affects error

• Property: || P e 2h || ≤ β|| e 2h ||• Reasoning:

|| P e 2h || ≤ ||P|| || e 2h ||= ||PT P||1/2 || e 2h ||


⇒ ||PT P||1/2 ≤ √2

Approximation propertyhow the discrete solutions approximate each other

Property: ||u h - P u 2h || ≤ αKh2

Reasoning:|(u (h) - P u (2h))i| = |u(xi) - (u(xi - 1 ) + u(xi + 1))/2| ≤ |u”(ξ)|h2/2

⇒ ||u (h) - P u (2h) || ≤ cKh2


⇒ ||u (h) - P u (2h) || ≤ cKh2

⇒ ||u h - P u 2h || ≤ ||u h - u (h) || + ||u (h) - P u (2h) || + || P u (2h) - P u 2h ||≤ Kh2 + cKh2 + || P ||·||u (2h) - u 2h ||≤ Kh2 + cKh2 + βK(2h)2

≤ (1 + c + 4 β)Kh2

FMG accuracy||e 2h || ≤ K(2h)2

Assume:

||e 2h || ≤ K(2h)2 induction hypothesis

||u h - P u 2h || ≤ αKh2 approximation property (α≈5)

|| P w 2h || ≤ β||w 2h || interpolation stability (β≈1)

Triangle inequality:

||e h || = ||u h - P v 2h ||


||e h || = ||u h - P v 2h || ≤ ||u h - P u 2h || + || P (u 2h - v 2h)||≤ αKh2 + β K(2h)2

= (α + 4β)Kh2

⇒ ||e h || ≤ “9”Kh2

So we need only reduce ||eh|| by “0.1”!!!

Numerical example

• Consider again the 2-D model problem (with σ = 0):

inside the unit square, with u = 0 on the boundary.

)−()−(+)−()−(=−− xxyyyxuu yyxx 1611612 222222


• We examine the effectiveness of FMG cycling to solve this problem on NxN grids [(N-1) x (N-1)interior] for N = 16, 32, 64, 128.

FMG results3 FMG cycles & comparison with MV cycle results

1.03e-04


||e ||h = h ||u (h) - v h||2scaled discrete total error

1.03e-042.58e-056.44e-061.61e-06

Diagnostic toolsfor debugging the code, the method, the problem

• Finding mistakes in codes, algorithms, concepts, & the problem itself challenges our scientific abilities.

• This challenge is especially tough for multigrid:– Interactions between multilevel processes can be very subtle.– It’s often not easy to know how well multigrid should perform.

• Achi Brandt:


• Achi Brandt:– “The amount of computational work should be proportional to the amount of real physical changes in the computed solution.”

– “Stalling numerical processes must be wrong.”

• The “computational culture” is best learned by lots of experience & interaction, but some discussion helps.

Tool # 1: Be methodical

• Modularize your code.

• Test the algebraic solver first.

• Test the discretization next.


• Test the FMG solver last.

• Beware of boundaries, scales, & concepts.

• Ask whether the problem itself is well posed.

Tool # 2: Start simply

• Start from something that already works if you can.

• Introduce complexities slowly & methodically, testing

thoroughly along the way.


• Start with a very coarse fine grid (no oxymoron intended).

• Start with two levels if you can, using a direct solver or lots

of cycles on coarse grids if nothing else.

Tool # 3: Expose trouble

Start simply, but don’t let niceties mask trouble:

• Set reaction/Helmholtz terms to zero.


• Take infinite or very big time steps.

• Don’t take 1-D too seriously, not even 2-D.

Tool # 4: Test fixed point property

Relaxation shouldn’t alter the exact solution of

the linear system (up to machine precision).

• Create a right side: f = Au* with u* given.

• Make sure u* satisfies the right boundary conditions.


• Make sure u* satisfies the right boundary conditions.

• Test relaxation starting with u* : Is r = 0, is it zero

after relaxation, does u* change?

• Test coarse-grid correction starting with u* : Is the

correction zero?

Tool # 5: Test on Au =0

• The exact solution u* = 0 is known!

• Residual norm ||Au|| & error norm ||u|| are computable.

• Errors ||Au|| & ||u|| should eventually decrease steadily with a rate that might be predicted by mode analysis.


with a rate that might be predicted by mode analysis.

• Multigrid can converge so fast that early stalling suggests trouble when it’s just that all machine-representable numbers in a nonzero u* have already been computed! Computing r = f - Au & updating u shouldn’t have trouble with machine precision if you have u* = 0 & thus f = 0.

Tool # 6: Zero out residual

• Using a normal test, try multiplying the residual by 0

before you go to the coarse grid.

• Check to see that the coarse-grid corrections are 0.


• Check to see that the coarse-grid corrections are 0.

• Compare this test with a relaxation-only test--the

results should be identical.

Tool # 7: Print out residual norms

• Use the discrete L 2 norm:

||r||h = (hdΣ ri2)1/2 = hd/2 ||r ||2

• Output ||r||h after each pre- & post-relaxation sweep.

• These norms should decline to zero steadily for each h.


• These norms should decline to zero steadily for each h.

• The norm after post-relaxation should be consistently

smaller than after pre-relaxation--by the predictable

convergence factor at least.

A warning about residuals• Residuals could mask large smooth errors:

– If ||e||=O(h2), then ||r||=O(h2) for smooth e, but ||r||=O(1) for oscillatory e. (λN /λ1 = O(h-2)!!!)

– This means that if we are controlling the error by monitoring ||r||, if we don’t know the nature of e, & if we don’t want to risk large e by requiring only


& if we don’t want to risk large e by requiring only ||r||=O(1), then we may have to work very hard to make ||r||=O(h2). (We could in effect be trying to make ||e||=O(h4) !!!)

• Multigrid tends to balance the errors, so ||r||=O(h2)tends to mean ||e||=O(h2).

Tool # 8: Graph the error

• Run a test on a problem with known solution (Au = 0).

• Plot algebraic error before & after fine-grid relaxation.

• Is the error oscillatory after coarse-grid correction?


• Is the error much smoother after fine-grid relaxation?

• Are there any strange characteristics near boundaries,

interfaces, or other special phenomena?

Tool # 9: Test two-level cycling

• Replace the coarse-grid V-cycle recursive call with a direct

solver if possible, or iterate many times with some method

known to “work” (test ||r|| to be sure it’s very small), or use many recursive V-cycle calls.


• This can be used to test performance between two coarser

levels, especially if residual norm behavior identifies trouble

on a particular level.

Tool # 10: Beware of boundaries

• Boundaries usually require special treatment of the

stencils, intergrid transfers, & sometimes relaxation.

• Special treatment often means special trouble, typically

exposed in later cycles as it begins to infect the interior.

• Replace the boundary by periodic or Dirichlet conditions.


• Replace the boundary by periodic or Dirichlet conditions.

• Relax more at the boundary, perhaps using direct solvers.

• Make sure your coarse-grid approximation at the boundary

is guided by good discretization at the fine-grid boundary.

Tool # 11: Test for symmetry

• If your problem is symmetric or includes a symmetric

case, test for it.

• Check symmetry of the fine-grid & coarse-grid matrices:


• Check symmetry of the fine-grid & coarse-grid matrices:

are aij & aji relatively equal (to machine precision).

• Be especially watchful for asymmetries near boundaries.

Tool # 12: Check for compatibility

• This is a bit ahead of schedule, but consider the problem

-u” = f with u ’(0) = u ’(1) = 0.

• It’s singular: If u = 1, then -u ” = 0 & u ’(0) = u ’(1) = 0.

• It’s is solvable iff f ∈ Range(∂xx) = η⊥(∂xx ) = 1⊥ or f ⊥ 1.


• It’s is solvable iff f ∈ Range(∂xx) = η (∂xx ) = 1 or f ⊥ 1.

• First fix the grid h right side: fh ← f h - (<f h, 1>/<1,1>)1.

• Do this on coarse grids too: f2h ← f2h - (<f2h, 1>/<1,1>)1.

• Uniqueness is also a worry: uh ← uh - (<uh, 1>/<1,1>)1.

Tool # 13: Test for linearity

• Again, this is a bit ahead of schedule, but if you’re writing

a nonlinear FAS code, it should agree with the linear code

when you test it on a linear problem. Try it.


• Move gradually to the target nonlinear test problem by

putting a parameter in front of the nonlinear term, then

running tests as the parameter changes slowly from 0 to 1.

Tool # 14: Use a known PDE solution• Set up the source term (f = Au* in Ω) & data (g = u* on Γ).

• Do multigrid results compare qualitatively with sampled u*?

• Monitor ||u* - u h||h .

• Test a case with no discretization error (let u* = 2nd degree polynomial). The algebraic error should tend steadily to 0.

• Test a case with discretization error (let u* have a nonzero


• Test a case with discretization error (let u* have a nonzero 3rd derivative). The algebraic error should decrease rapidly at first, then stall at discretization error level. Check error behavior as you decrease h. Does it behave like O(h2) (hhalved ⇒ error halved) or whatever it should behave like?

Tool # 15: Test FMG accuracy

• Make sure first that the algebraic solver converges as

predicted, with uniformly bounded convergence factors.

• Test the discretization using Tool # 14.


• Test the discretization using Tool # 14.

• Compare FMG total error to discretization error for

various h. You might need to tune the FMG process here

(play with the number of cycles & relaxation sweeps).

Computing assignments• Document:

norms/weights, V(ν1, ν2), errors, labels (table, graph)• Use various scenarios:

Ax = 0, Ax = f, varying N & νi & ω, Jacobi/Gauss-Seidel• Thoroughly test:

don’t stop until you get what you expect.compare with known solution, text, others. study discretization & algebraic errors. report on “asymptotic” factors.


study discretization & algebraic errors. report on “asymptotic” factors.

• Be kind to the reader:code = zzz… tables = + tables & graphs = ++tables & graphs & discussion (clear, concise) = +++

• Discuss, discuss, discuss:what do you see & think? what did you learn?

by hand ok↓↓↓↓ ↓↓↓↓

Outline




– Analysis


– Relaxation






– Variable meshes







– Variable meshes

√√√√

√√√√


– Relaxation

– Coarsening

• Implementation

– Complexity

– Diagnostics

• Some Theory


– Variable meshes





– FAC

• Finite Elements


– Variable meshes





– FAC

• Finite Elements


√√√√

√√√√

What is A2h?• Recall the 2-grid coarse-grid correction scheme:

– 1) Relax on Ah u h = f h on Ωh to get v h. – 2) Compute f2h = Ih2h (f h - Ah v h) . – 4) Solve A2h u 2h = f 2h on Ω2h .

– 5) Correct fine-grid solution vh ← vh + I2h u2h.h

5. Some theory


– 5) Correct fine-grid solution v ← v + I2h u .

• Assume that eh ∈ Range(I2h ). Then the residual equation can be written

rh = Aheh = Ah I2h u2h for some u2h ∈ Ω2h.

This characterizes u 2h, but with too many equations.

• How does Ah act on Range(I2h )?

h

h

h

h

How does act on ?Ah

IegnaR )( 2hh

u2h

uI 22

hhh


• Therefore, the odd rows of Ah I2h are zero (for 1-D only) & r2i + 1 = 0. We therefore keep the even rows of Ah I2h for the residual equations on Ω2h. These rows can be picked out by applying restriction!

uIA hhh

h 22

h

h

rIuIAI hhh

hhh

hhh

222

2 =

Building A2h: The Galerkin condition

• The residual equation on the coarse grid is

rIuIAI hhh

hhh

hhh

222

2 =

• We thus identify the coarse-grid operator as


IAIA 222 h

hhh

hh = A2h = RAP

• How do we know RAP is symmetric?

• We thus identify the coarse-grid operator as

If PT = R & RT = P, then(RAP)T = PTATRT = RAP.

↑↑↑↑ Why is this the i th row of A2h ?

Computing the i th row of . • Compute where

A2h

eA22 hi

he

Thi2

),...,,,,...,,(= 001000

ii − 1 i + 1


0 01 eh

i2

0 01 2

1

2

1eI

22

hi

hh

0 0 −2

1

h2

−2

1

h2

1

h2

eIAh

ihh

h 22

2

1

h2

−4

1

h2

−4

1

h2

eIAIh

ihh

hhh

22

2

• The i th row of is ,

which is the version of .

• Note that IF relaxation on leaves only error in the range of interpolation, then solving

The i th row of looks a lot like the i th row of !

A2h

Ah

A2h

1212

1−−

)( h2

AhΩ2h

Ωh


in the range of interpolation, then solvingA 2h u 2h = f 2h

determines the error exactly!

• This is generally not feasible, but this logic leads to a very plausible representation for .

A2h

Ωh

Variational properties of coarsening

• The definition for that resulted from the foregoing line of reasoning is useful for both theoretical & practical reasons. Together with the commonly used relationship between restriction & prolongation, we have the variational properties:

A2h


variational properties:

IAIA 222 h

hhh

hh =

IcI2

2

Thh

hh )(=

Galerkin Condition

for c in ℜ.

Properties of restrictionin a little more detail…

• Full Weighting: or

ℜN-1 ℜN/2-1

• N = 8:

Ihhh

h22 Ω→Ω:

Ih

h2 :


• N = 8:

• has rank & null space η( ) with dim .

Ih

h

73

2 =

121

121

121

4

1

×

N1

2−I

hh2 N

2Ih

2h

Spectral properties of restriction

• How does act on the eigenvectors of ?

• Consider , 1 ≤ k ≤ N - 1, 0 ≤ j ≤ N.

• A little algebra & trigonometry shows that

Ih

h2 A

h

N

kjwh

jk,

π

= nis


for 1 ≤ k ≤ N/2.

π

=)( wN

kwI h

jkj

hk

hh

222

2soc ,

≡ wc hjkk ,

2

Spectral properties (cont’d)

• i.e., [kth mode on ] = ck [ kth mode on ]

: N = 8, k = 2

Ih

h2 Ωh Ω2h

Ωh•• • •

•• • •

•


: N = 4, k = 2

Ω2h

Ih

h2

• • •

•

•

•

•

•


• Let for , so that .

• A little algebra & trigonometry shows that

kNk −=′ NkN

2<′<

21 <≤

Nk

π k


π

−=)( wN

kwI h

jkj

hk

hh

222,′ 2

nis

≡ ws hjkk ,

2


i.e., [(N-k)th mode on ] = -sk [ kth mode on ]

: N = 8, k = 6

Ih

h2 Ωh Ω2h

Ωh••

•

•

••

•

•


: N = 4, k = 2

Ω2h

Ih

h2

•• •

•

•

•

•

•


• Summarizing:

wcwI hkk

hk

hh

22 =

wswI hkk

hk

hh

22′ −=

wI hh2 = 0

21 <<

Nk

kNk −=′


• Complementary modes: wI h

Nh

h 22

/ = 0

wwW hk

hkk ,= naps ′

wWI hkk

hh

22 → ∞

Null space of restriction

• Observe that η(Ih2h) = span(Ahêih), where i is odd & êih is the i th unit vector.

• Let ηi = Ahêih.

• While the ηi looks oscillatory, it generally contains 0 000 − 1 − 12 0 0


• While the ηi looks oscillatory, it generally contains all Fourier modes of :

• All the Fourier modes of are needed to represent the null space of restriction!

Ah

=η kk

N

ki ∑

−

=

1

1

wa ak ≠ 0

Ah

0 000 − 1 − 12 0 0

Properties of interpolation

• Interpolation: or

: ℜN/2-1 ℜN-1

• N = 8:

I2hh

I2

2hhh

h Ω→Ω:

2

1


• has full rank & null space 0.

I2hh =

1

2

11

2

11

2

2

1

I2hh

Spectral properties of interpolation

• How does act on the eigenvectors of ?

• Consider , 1 ≤ k ≤ N/2-1,0 ≤ j ≤ N/2.

I2hh A

2h

/

π

=)(N

kjw

j

hk2

2nis


• A bit of work shows that the modes of are NOT “preserved” by , but that the space is “preserved”:

A2h

I2hh W k

wN

kw

N

kwI 222

2hk

hk

hk

hh

π

−

π

=2

nis2

soc ′

−= wswc hkk

hkk ′

Spectral properties of interpolation

• Interpolation of smooth modes excites oscillatory modes on .

• Note that if , then

Ω2h

Ωh

N

I2hh wswc h

kkhkk −= ′wk

2h


• Note that if , then

• is 2nd-order interpolation.

Nk

2«

wN

kOw

N

kOwI

2

2

2

22

2hk

hk

hk

hh

+

−

= 1 ′

≈ whk

I2hh

Range of interpolation

• The range of is the span of the columns of .

• Let be the i th column of .

I2hh I2

hh

ξ i I2hh

:ξi


• All the Fourier modes of are needed to represent Range( ).

:ξi

I2hh

≠,=ξ khkk

N

k

hi ∑

−

=

1

1

bwb 0

Ah

Use all the facts to analyze the coarse-grid correction scheme

relax only before correction

1) Relax once on :

2) Compute & restrict residual

3) Solve residual equation

vAfIf22 hhhhh

h )−(←

fAv2122 hhh )(=

−

Ωh vRv hh ← + B f h


4) Correct fine-grid solution .

• The entire process appears as

• The exact solution satisfies

fAv222 hhh )(=

vIvv hhh

hh +← 22

AfIAIvRvhhh

hhh

hhh )−()(+ −α 212

2 )vR h + B f h(←

AfIAIuRvhhh

hhh

hhh )−()(+ − 212

2 )uR h + B f h(=

CG error propagation

• Subtracting the previous two expressions, we get

• How does CG act on the modes of ? Assume eh

eRAIAIIe hhhh

hhh

h )(−←212

2

−

eGCe hh ←


• How does CG act on the modes of ? Assume ehconsists of the modes & for & .

• We know how act on & .

eGCe hh ←

Ah

whk′wh

k 21 /≤≤ Nk

kNk −=′

IAIAR−α ,)(,,, h

hhh

hh

2

122

whk wh

k′

CG error propagation

• For now, assume no relaxation. Then

is invariant under CG:wwW h

khkk ,= naps ′

wswswGC hkk

hkk

hk += ′


where

wswswGC kkkkk += ′

wcwcwGC hkk

hkk

hk ′′ +=

N

kck

π

=2

soc 2

N

ksk

π

=2

nis 2

CG error propagation for k<<N

• Consider the case k << N (extremely smooth & oscillatory modes):

wN

kOw

N

kOw kkk

+

→2

2

2

2

′


• Hence, CG eliminates the smooth modes but does not damp the oscillatory modes of the error!

NN

wN

kOw

N

kOw kkk

−

+

−

→ 112

2

2

2

′′

CG with relaxation

• Next, include one relaxation sweep. Assume that the relaxation preserves the modes of (although this is often unnecessary). Let denote the eigenvalue of associated with . For k < < N/2:

R Ah

λkR wk


For k < < N/2:

Small!

Small!

wswsw kkkkkkk λ+λ→ ′

wcwcw kkkkkkk ′′′′ λ+λ→

Crucial observation

• Between relaxation & coarse-grid correction, both smooth & oscillatory components of the error are effectively damped.


• This is the “spectral” picture of how multigrid works. We examine now another viewpoint, the “algebraic” picture of multigrid.

Recall the variational properties

• All the analysis that follows assumes that the variational properties hold:


IAIA 222 h

hhh

hh =

IcI2

2

Thh

hh )(=

Galerkin Condition

for c in ℜ.

Algebraic interpretation of CG• Consider the subspaces that make up & .Ωh Ω2h

IR )( 2hh IN )( h

h2Ωh

From now on, ‘R( )’ refers to the Range of a linear operator & ‘N( )’ to its Null Space.


Ω2hIR )( hh2

I2hh I

hh2

From the Fundamental Theorem of Linear Algebra:

orIRIN ))((⊥)( Thh

hh

22

IRIN )(⊥)( hh

hh 22

But we really care about N (Ih2hAh ) !?!?

Subspace decomposition of Ω h

• If uh ∈ N (Ih2hAh ), then for any u 2h we have0 = ⟨ Ih2hA hu h, u 2h ⟩ = ⟨ Ahu h, I2hu 2h ⟩ ,

so

,

where means .

• Moreover, any can be written as ,

AINIR )(⊥)( 22

hhhA

hh h

yx ⊥ h yxAh =, 0

h

“energy”inner product


• Moreover, any can be written as ,

where & .

• Hence,

.

yx ⊥A

h yxAh =, 0

eh tsehhh +=

IRshh

h )(∈ 2 AINthh

hh )(∈ 2

)(⊕)(=Ω hhh

hh

hAINIR

22

inner product

Characteristics of the subspaces

• Since for some , we associate with the smooth components of . But, generally has all Fourier modes in it. Recall the basis vectors for :

• Similarly, we associate with oscillatory

qIs hhh

h = 22 q

22 hh Ω∈sh eh

sh

I2hh


• Similarly, we associate with oscillatory components of , although generally has all Fourier modes in it as well. Recall that is spanned by , so is spanned by the unit vectors for odd i, which “look” oscillatory.

th

eh th

IN )( hh2

=η ih

i eA AIN )( hhh2

eTh

i ),...,,,,...,,(= 001000

Algebraic analysis CG

• Recall that (without relaxation)

• First note that if , then . This follows since for some & therefore

AIAIIGC )(−= 2122

hhh

hhh

−

IRshh

h )(∈ 2 sGC h = 0qIs hh

hh = 2

2 q22 hh Ω∈


qIAIAIIsGC hhh

hhh

hhh

h )(−= 22

2122

−

& therefore

• It follows that N(CG)=R( ), that is, the null space of CG is the range of interpolation.

qIs hhh

h = 22 q

22 hh Ω∈

= 0

A2h by Galerkin property

I2h

h

More algebraic analysis of CG

• Next, note that if , thenAINthh

hh )(∈ 2

tAIAIItGChhh

hhh

hh )(−= 212

2

−


• Thus, CG is the identity on .

ttGChh =

0

AIN )( hhh2

⇒

How does the algebraic picture fit with the spectral view?

• We may view in two ways:

=

that is,

Ωh

Low frequency modes

1 ≤ k ≤ N/2

High frequency modes

N/2 < k < NΩh


that is,

or⊕=Ωh

HL

)(⊕)(=Ω hhh

hh

hAINIR

22

Actually, each view is just part of the picture

• The operations we have examined work on different spaces!

• While is mostly oscillatory, it isn’t , & while is mostly smooth, it isn’t . H

IR )( h LAIN )( hh

h2


& while is mostly smooth, it isn’t .

• Relaxation eliminates error from .

• Coarse-grid correction eliminates error from . H

IR )( 2hh L

IR )( 2hh

How it actually works (cartoon)AIN )( hh

h2

eh

th

H


IR )( 2hh

sh

L

Relaxation eliminates H, but increases the error in .

AIN )( hhh2H

IR )( 2hh


IR )( 2hh

eh

sh

th

L

CG eliminates error in ,but increases error in H

AIN )( hhh2H

IR )( 2hh


IR )( 2hh

eh

sh

th

L

Outline




– Analysis


– Relaxation






– Variable meshes







– Variable meshes

√√√√

√√√√


– Relaxation

– Coarsening

• Implementation

– Complexity

– Diagnostics

• Some Theory


– Variable meshes





– FAC

• Finite Elements


– Variable meshes





– FAC

• Finite Elements


√√√√

√√√√

√√√√

Some reflection

• What do you want out of this course?

• Do you seem to be getting there?

• Can we get there more directly?


• Can we get there more directly?

• Start thinking about your project!

• Think about doing it in two parts.

6. Nonlinear problems

• How should we approach the nonlinear system

& can we use MG to solve it?

• A fundamental relation we’ve relied on is the linear

fuA =)(


• A fundamental relation we’ve relied on is the linearresidual equation:

We can’t rely on this now since a nonlinear A(u)generally means

reAvAfvAuA =⇒−=−

eAvAuA )(≠)(−)(

The nonlinear residual equation

• We still base our development around the residual equation, now the nonlinear residual equation:

vAfvAuA )(−=)(−)(

fuA =)(


• How can we use this equation as the basis for a solution method?

vAfvAuA )(−=)(−)(

rvAuA =)(−)(

Let’s consider Newton’s method

• Best known & most important nonlinear problem solver!• We wish to solve F(x) = 0 . • Suppose for the moment that F is scalar F : ℜ→ ℜ.• Expand F in a Taylor series about x :


• Dropping higher order terms (hot), if x + s is a solution,

• We thus arrive at Newton’s method:0 )(′/)(−=∴)(′+)(= xFxFsxFsxF

xF

xFxx

)(′)(

−←

F( x + s) = F( x) + sF ′( x) + s2 F ″ ( ξ)

Newton’s method for systems

• The system A(u) = f in vector form is

a1(u1,u2 ,...,uN )

a2 (u1,u2 ,...,uN )

M

=

f1

f2

M


• Expanding A(v + e) in a Taylor series about v :

evJvAevA +)(+)(=)+( hot

J(u) =∂ai

∂u j

M

aN (u1,u2 ,...,uN )

M

fN

Newton for systems (cont’d)

• J(v) is the Jacobian

J(v) =

∂a1

∂u1

∂a1

∂u2

L∂a1

∂uN

∂a2

∂u1

∂a2

∂u2

L∂a2

∂uN

M M O M


• If u = v + e is a solution, f = A(v) + J(v)e + hot, soe ≈ [J(v)]-1(f - A(v))

• This leads to the iteration

M M O M∂aN

∂u1

∂aN

∂u2

L∂aN

∂uN

u = v

v ← v + [J(v)]-1(f - A(v))

Newton’s via the residual equation

• The nonlinear residual equation is

• Expanding A(v + e) in a two-term Taylor series about v& ignoring hot:

rvAevA =)(−)+(


• Newton’s method is thus:

v ← v + J( v)− 1

r

vAfr )(−=

rvAevJvA =)(−)(+)(

revJ =)(

How does multigrid fit in?

• One obvious method is to use multigrid to solve J(v)e = r at each iteration step. This method is called Newton-MG & can be very effective.

• However, we would like to use multigrid ideas to


• However, we would like to use multigrid ideas to treat the nonlinearity directly.

• Hence, we need to specialize the multigrid components (relaxation, grid transfers, coarsening) for the nonlinear case.

What is nonlinear relaxation?

• Several of the common relaxation schemes have nonlinear counterparts. For A(u) = f, we describe nonlinear Gauss-Seidel:

For each j = 1, 2, …, N :Set the jth component of the residual to 0


Set the j component of the residual to 0

& solve for vj . That is, solve (A(v))j = fj .

• Equivalently,

For each j = 1, 2, …, N :Find s ∈ ℜ such that ,

where εj is the canonical jth unit basis vector. =))ε+(( fsvA jjj

How is nonlinear Gauss-Seidel done?• Each is a nonlinear scalar equation for vj . We can use the scalar Newton’s method to solve!

• Example: may be discretized so that is given by

=))(( fvA jj

,=)(+)(″− fexuxu xu )(

=))(( fvA jj

−+− vvv 2


• Newton iteration for vj is given by=+

−+−fev

h

vvv

jv

j

jjj +−

2

11 2j 11 −≤≤ Nj

ve

fev

vv

j

v

h

jv

jh

vvv

jj

)+(+

−+

−←

−+−

j

jjjj +−

2

2

11

2

2

1

How do we do coarsening for nonlinear multigrid?

• Recall the nonlinear residual equation

• In multigrid, we obtain an approximate solution vhon the fine grid, then solve the residual equation on the coarse grid.

rvAevA =)(−)+(


on the coarse grid.

• The residual equation on appears as

rvAevA 222222 hhhhhh =)(−)+(

Ω2h

Look at the coarse residual equation

• We must evaluate the quantities on in

• Given vh, a fine-grid approximation, we restrict the residual to the coarse grid:

Ω2h

rvAevA 222222 hhhhhh =)(−)+(


• For v2h we restrict vh by vAfIr

22 hhhhh

h ))(−(=

vIv22 hhh

h =

vAfIvIAevIA222222 hhhhh

hhh

hhhhh

h ))(−(+)(=)+(

Thus,

We’ve obtained a coarse-grid equation of the form A2h(u2h) = f2h

• Consider the coarse-grid residual equation:


hhh

hhhhh

h ))(−(+)(=)+(

f2hu2h


• We solve for & obtain

• We then apply the correction:

f2h

all quantities are known

u2h

coarse-grid unknown

fuA222 hhh =)( evIu 222 hhh

hh +=

vIue222 hhh

hh −=

eIvv hhh

hh += 22

Full approximation scheme (FAS)2-grid form

• Perform nonlinear relaxation on to obtain an approximation .

• Restrict the approximation & its residual

• Solve the coarse-grid equation

fuAhhh =)(

vh

vIv22 hhh

h = vAfIr22 hhhh

h ))(−(=


• Solve the coarse-grid equation

• Extract 2h approximation to h error

• Interpolate & correct

eIvv hhh

hh += 22

vIv2 hh

h = vAfIr2 hh

h ))(−(=

rvAuA 22222 hhhhh +)(=)(

vue 222 hhh −=

A few observations about FAS

• If A is a linear operator, then FAS reduces directly to

the linear two-grid correction scheme.


hhh

hhhhh

h ))(−(+)(=)+(


• An exact solution to the fine-grid problem is a fixed

point of the FAS iteration.

A few more observations about FAS

• The FAS coarse-grid equation can be written as

where is the so-called tau correction term & is the original 2h source term, provided you choose it that way: .

fuA 2222 hh

hhh τ+=)(

τh2h =A(Ih

2hvh)−Ih2hA(vh)

f=I2 hhhf

2h

f2h


• In general, since , the solution to the FAS coarse-grid equation is not the same as the solution to the original coarse-grid problem

• The tau correction is as a way to alter the coarse-grid equation to enhance its approximation properties.

≠τ hh2 0 u2h

fuA222 hhh =)(

f=Ihf

Still more observations about FAS

• FAS may be viewed as an inner & outer iteration: the outer iteration is the coarse-grid correction, the inner iteration the relaxation method.


• A true multilevel FAS process is recursive, using FAS to solve the nonlinear Ω2h problem using Ω4h. Hence, FAS is generally employed in a V- or W-cycling scheme.

Even more observations about FAS

• For linear problems, we use FMG to obtain a good initial guess on the fine grid. Convergence of nonlinear iterations depends critically on having a good initial guess.

• When FMG is used for nonlinear problems, the interpolant is generally accurate enough to be in the basin of attraction of the fine-grid uI 2hh


be in the basin of attraction of the fine-grid solver.

• Thus, whether FAS, Newton, or Newton-multigrid is used on each level, one FMG cycle should provide a solution accurate to the level of discretization, unless the nonlinearity is extremely strong.

uI 22

hhh

Intergrid transfers for FAS

• Generally speaking, the standard operators (linear interpolation, full weighting) work effectively in FAS schemes.

• For strongly nonlinear problems, higher-order interpolation (e.g., cubic interpolation) may be beneficial.


interpolation (e.g., cubic interpolation) may be beneficial.

• For an FMG scheme, where is the interpolation of a coarse-grid solution to become a fine-grid initial guess, higher-order interpolation is often advised.

uI 22

hhh

What is A2h(u2h) in FAS?As in the linear case, there are two basic possibilities:

1. A2h(u2h) is determined by discretizing the nonlinear operator A(u) in the same fashion as was employed to obtain Ah(uh) , except that the coarser mesh spacing is used.

2. A2h(u2h) is determined from the Galerkin condition


2. A2h(u2h) is determined from the Galerkin condition

where the action of the Galerkin product can be captured in an implementable formula.

The first method is usually easier & more common.

A2h

( u2h) = Ih2h

Ah( u2h)I2h

h

Nonlinear problems: An example

• Consider

on the unit square [0,1] x [0,1] with homogeneous Dirichlet boundary conditions & a regular h = 1/128 Cartesian grid.

− ∆u( x, y) + γ u( x, y) eu( x, y) = f ( x, y)


Dirichlet boundary conditions & a regular h = 1/128 Cartesian grid.

• Suppose the exact solution is

u( x, y) = ( x 2 − x 3 ) sin ( 3 π y)

Discretization of nonlinear example

• The operator can be written (sloppily) as

1

141

11

feuuh

2=γ+

−

−−−

jiuh

jih

ji ,,,

hji,

uA hh )(


• Relaxation is given byuA hh )(

eu

fuA

uuuh

jih

jiji

hh

hji

hji

,

,,

,,)+(γ+

−))((

−←4

21 ji,

h

FAS & Newton’s method on

• FAS V(2,1)-cycles until ||r|| < 10-10

− ∆u( x, y) + γ u( x, y) eu( x, y) = f ( x, y)

1 10 100 1000

convergence factor 0.135 0.124 0.098 0.072

γ

N = 128


• Newton’s Method with exact inner solves until ||r|| < 10-10

convergence factor 0.135 0.124 0.098 0.072

number of FAS cycles 12 11 11 10

1 10 100 1000

convergence factor 4.00E-05 7.00E-05 3.00E-04 2.00E-04

number of Newton iterations 3 3 3 4

γ

Newton, Newton-MG, & FAS on

• Newton uses exact solves, Newton-MG is inexact Newton with a fixed number of inner V(2,1)-cycles for the Jacobian problem, overall stopping criterion ||r|| < 10-10.

− ∆u( x, y) + γ u( x, y) eu( x, y) = f ( x, y)

Outer Inner

Method iterations iterations Megaflops

N = 128


Method iterations iterations Megaflops

Newton 3 1660.6

Newton-MG 3 20 56.4

Newton-MG 4 10 38.5

Newton-MG 5 5 25.1

Newton-MG 10 2 22.3

Newton-MG 19 1 24.6

FAS 11 27.1

Comparing FMG-FAS & FMG-Newton− ∆u( x, y) + γ u( x, y) e

u( x, y) = f ( x, y)

• We will do one FMG cycle using a single FAS V(2,1) -cycle as the “solver” at each new level. We then follow that with sufficiently many FAS V(2,1)-cycles as is necessary to obtain ||r|| < 10-10.


as is necessary to obtain ||r|| < 10 .

• Next, we will do one FMG cycle using a Newton-multigrid step at each new level (with a single linear V(2,1)-cycle as the Jacobian “solver.”) We then follow that with sufficiently many Newton-multigrid steps as is necessary to obtain ||r|| < 10-10.

Comparing FMG-FAS & FMG-Newton− ∆u( x, y) + γ u( x, y) e

u( x, y) = f ( x, y)

Cycle Mflops Mflops Cycle

FMG-FAS 1.10E-02 2.00E-05 3.1 1.06E-02 2.50E-05 2.4 FMG-Newton

FAS V 6.80E-04 2.40E-05 5.4 6.70E-04 2.49E-05 4.1 Newton-MG



|||| rh |||| rh|||| eh |||| eh

N = 128








5.50E-09 2.49E-05 17.7 Newton-MG

1.80E-09 2.49E-05 19.4 Newton-MG

5.60E-10 2.49E-05 21.1 Newton-MG

1.80E-10 2.49E-05 22.8 Newton-MG

5.70E-11 2.49E-05 24.5 Newton-MG

Remembering coarse-grid correctionAu = f

• Relax (Jacobi) to smooth error e = u - v :v ← v - D-1(Av - f)

• Form the residual equation Ae = r :Ae = A(u - v) = f - Av = r

Use premise that smooth error ⇒ e = I e2h :

hI2h ↑↑↑↑


• Use premise that smooth error ⇒ e = I2he2h :AI2he2h = r

• Use transpose I2h = (I2h)T to reduce equations:

I2hAI2h e2h = I2hr

h

h

hh

h h

fewer unknowns

fewer equationsh

A2h

Motivating FAS for nonlinear AA(v + e) = f → A2h(v2h + e2h) = f2h

• Analogous to asking how to discretize a PDE of this form: A(v + e) = f

with v known.


hhh

hhhhh

h ))(−(+)(=)+(


with v known.• Example:

γuu’ - u’’ = f →→→→ γ (v + e) (v + e)’ - (v + e)’’ = f• Expand to get an equation in e of form g + ae + be’ + …:

γvv’ - v’’ + γ (v’e + ve’ + ee’ ) - e’’ = f

A(v)

Differential residual equationγ (v’e + ve’ + ee’ ) - e’’ = f - A(v) ≡ r(v)

• Right side (analogous to injection):

r(v) →→→→ rjh = r(xj)

• Coefficients:

v →→→→ vjh = v(xj), v’ →→→→ (v’)jh = (v(xj+1) - v(xj-1))/(2h)


v →→→→ vjh = v(xj), v’ →→→→ (v’)jh = (v(xj+1) - v(xj-1))/(2h)

• Unknowns:

e →→→→ ejh, e’ →→→→ (e’)jh = (ej+1 - ej-1)/(2h),

e’’ →→→→ (e’’)jh = (ej+1 - 2ej + ej-1)/h2h h

h h h

Leading to FAS…γuu’-u’’ = f → A(v+e) = f → γ (v’e+ve’+ee’ )-e’’ = f-A(v)

• Fine-grid residual equation (at h point 2j):

• Coarse-grid residual equation (at 2h point j):

+ γv2 j

h e2 j +1

h−e2 j−1

h

2h

−e2 j +1

h+ 2e2 j

h−e2 j−1

h

h2+γ e2 j

hv2 j +1

h−v2 j−1

h

2h+ γe2 j

h e2 j +1

h−e2 j−1

h

2h= r2 j

h


• Coarse-grid residual equation (at 2h point j):

• FAS

−ej +1

2h+ 2ej

2h−ej−1

2h

(2h)2+ γv2j

h ej +1

2h−ej−1

2h

4h+γ ej

2hv2j+2

h−v2j−2

h

4h+γej

2h ej +1

2h−ej−1

2h

4h= Ih

2hr

h( )j


hhh

hhhhh

h))(−(=)(−)+(

Example: Newton-MG vs. FAS

• PDE (er, ODE):

• Discretization:

,=)(+)(″− fexuxu xu )(


• Discretization:

=+−+−

fevh

vvv

jv

j

jjj +−

2

11 2j

One Newton-MG step• Step 0. Given v, form the grid h linear correction equation:

Initialize the Newton correction approximation: e = 0. • Step 1: Relax on the grid h linear equation. • Step 2: Solve the grid 2h error correction equation:

−ej−1 + 2ej − ej +1

h2 + (1+ v j )e

v jej = rj ≡ f j −

−v j−1 + 2v j − v j +1

h2 + v j e

v j( )

−e2h

+ 2e2h

− e2h

( −e + 2e − e)


• Step 3: Correct the grid h Newton correction:

• Step 4: Stop if you’ve “solved” the linear equation well enough for Newton correction e & set v ← v + e. Else, leave v alone & return to Step 1.

−ej−1

2h+ 2ej

2h− ej +1

2h

(2h)2 + (1+ v2j)e

v 2jej

2h= r2 j −( −e2 j−1 + 2e2 j − e2 j +1

h2 + (1+ v2 j )e

v2je2 j)

e ← e + I2 h

he

2 h

One FAS step

• Step 0. Given v, form the grid h nonlinear equation:

• Step 1: Relax on the grid h nonlinear equation to improve v. • Step 2: Solve the grid 2h FAS correction equation:

=+−+−

fevh

vvv

jv

j

jjj +−

2

11 2j

−e2h

+ 2e2h

−e2h


• Step 3: Correct the grid h approximation v :

• Step 4: Stop if you’ve “solved” the nonlinear problem well enough for the solution u. Else, return to Step 1.

v ← v + I2 h

he

2 h

−ej−1

2h+ 2ej

2h−ej +1

2h

(2h)2 + v2 j

h+ ej

2h( )e

v2 jh +e j

2h

−v2 j

he

v2 jh

= Ih

2hr

h( )

j

Outline




– Analysis


– Relaxation






– Variable meshes







– Variable meshes

√√√√

√√√√

√√√√


– Relaxation

– Coarsening

• Implementation

– Complexity

– Diagnostics

• Some Theory


– Variable meshes





– FAC

• Finite Elements


– Variable meshes





– FAC

• Finite Elements


√√√√

√√√√

√√√√

7a. Neumann boundary conditions

• Consider the 1-D problem

• We discretize this on the interval [0,1] with

)(=)(″− xfxu 10 << x

uu =)(′=)(′ 010 1

7. Selected applications


-1 N+2

• We discretize this on the interval [0,1] with grid spacing & nodes , j=1,2, …, N+1 .

• We extend the interval with two ghost points

uu =)(′=)(′ 010

hjx j =N

h+

=1

1

0 1 2 3 … j ... N-1 N N+1

0 x 1

Central differences at boundary• We use differences as before, but now also for the derivative in the Neumann condition:

• This yields the system

-1 N+20 1 j-1 j j+1 N N+1

h

uuu

−≈)(′

20

11 −

h

uuu

−≈)(′

21

NN + 2uuu

xu

−+−≈)(″

jjj

j

+−

2

11 2


• This yields the systemhu ≈)(′

20

hu ≈)(′

21

hxu ≈)(″ j 2

=−+−

fh

uuu

j

jjj +−

2

11 2

h

uu 11=

− −0

2 h

uu NN + 2=

−0

2

10 +≤≤ Nj

Eliminating the ghost points

• Use the boundary conditions to eliminate ,

• Eliminating the ghost points in the j = 0 & j = N + 1equations gives the (N+2)x(N+2) system of equations:

u − 1 uN + 2

h

uu 11=

− −0

2

uu NN + 2 =h

uu NN + 2=

−0

2uu − 11 =


• Eliminating the ghost points in the j = 0 & j = N + 1equations gives the (N+2)x(N+2) system of equations:

=−+−

fh

uuu

j

jjj +−

2

11 210 +≤≤ Nj

22f

h

uu

02

10=

−=

+− 22f

h

uu

N

NN

++

12

1

Write the system in matrix form

• We can write , wherefuAhhh =

Ah =

1

h2 2 − 2

− 1 2 − 1

− 1 2 − 1⋅ ⋅⋅

⋅ ⋅ ⋅


• Note that is (N+2)x(N+2) & nonsymmetric, & the system involves unknowns & at the boundaries.

h2

⋅ ⋅⋅

⋅ ⋅ ⋅− 1 2 − 1

− 1 2

Ah

u0h uN + 1

h

2

We must consider compatibility

• The problem , for & with is not well-posed!

• If u(x) is a solution, so is u(x) + constant.• We cannot be certain a solution exists. If one does, it must satisfy

)(=)(″− xfxu 10 << x

uu =)(′=)(′ 010


• This integral compatibility condition is necessary! If f(x) doesn’t satisfy it, there is no solution!

− ⌠⌡0

1

u″( x) dx = ⌠⌡0

1

f ( x) dx − [ u′( 1) − u′( 0) ] = ⌠⌡0

1

f ( x) dx

0 = ⌠⌡0

1

f ( x) dx

The well-posed system• The compatibility condition is necessary for a solution to exist. In general, it is also sufficient:

is a well-behaved operator on the space of functions u(x) that have zero mean.

• Thus we may conclude that if f(x) satisfies the compatibility condition, this problem is well-posed:

∂

∂−

2

2

x


compatibility condition, this problem is well-posed:

• The last says: of all possible solutions u(x) + constant, we choose the one with zero mean.

)(=)(″− xfxu 10 << x

uu =)(′=)(′ 010

=)(⌡⌠1

0

xdxu 0

The discrete problem is not well posed• Since all row sums of are zero, then

• Putting into row-echelon form shows that

hence

• By the Fundamental Theorem of Linear Algebra, has a solution if & only if

Ah

1h ∈ NS ( A

h )

Ah

1SNmid =))(( Ah

1napsSN )(=)( Ahh

Ah

ASNfThh ))((⊥


• It is easy to show that

• Thus, has a solution if & only if

• That is,

ASNfThh ))((⊥

cASN )/,,...,,,/(=))(( TTh2111121

fuAhhh =cf

Th )/,,...,,,/(⊥ 2111121

02

1

2

1fff 1

10

hN

hj

N

j

h =++∑ +=

We have two issues to consider

• Solvability. A solution exists iff

• Uniqueness. If solves , then so does

• Note that if , then & solvability & uniqueness can be handled together

ASNfThh ))((⊥

uh Ahuh = f

h uh + c1h

Ah = ( A

h) TNS ( A

h) = NS ( ( Ah) T)


• Note that if , then & solvability & uniqueness can be handled together

• This is easily done. Multiply 1st & last equations by 1/2, giving

Ah = ( A

h) TNS ( A

h) = NS ( ( Ah) T)

Ah

=1

h2

1 − 1

− 1 2 − 1

− 1 2 − 1⋅ ⋅ ⋅

⋅⋅ ⋅− 1 2 − 1

− 1 1

The new system is symmetric• We have the symmetric system :

1

h2

1 − 1

− 1 2 − 1

− 1 2 − 1⋅⋅ ⋅

⋅⋅ ⋅− 1 2 − 1

u0

h

u1h

u2h

M

uhN

=

f

h0 /2

fh1

fh2

M

fhN

Ah

uh = fh


• Solvability is guaranteed by ensuring that is orthogonal to the constant vector :

− 1 2 − 1

− 1 1

uN

uhN + 1

f

hN

fhN + 1/2

fh

1h

ff j

hN

j

hh==, 01 ∑

+

=

1

0

The well-posed discrete system

• The (N+3)x(N+2) system is:

=−+−

fh

uuu

j

jjj +−

2

11 210 +≤≤ Nj

u0 − u1

h2

=f 0

2


or, more simply

− uN + uN + 1

h2

=f N + 1

2

Ah

uh = fh

∑ hi

N

i

+

=

1

0

u = 0

uhh =, 01

(choose the zero mean solution)

Multigrid for the Neumann problem• We must have the interval endpoints on all grids

• Relaxation is performed at all points, including endpoints:

xNh

/2xh0 xh

1 xNh

x 22N

h/x 4

2N

h/

x02h


• We add a global Gram-Schmidt step after relaxation on each level to enforce the zero-mean condition

fhvvh

hh0

210 +← fhvv

h

NhNN

h++ 1

21 +←

fhvv

v

h

jhj

hj

jh

+++←

+−2

11

2

v

vvh

hh

hh

hh

,

,−← 1

11

1

Interpolation must include the endpoints

• We use linear interpolation:


I2hh

//

//

//

//

=

1

2121

1

2121

1

2121

1

2121

1

Restriction also treats the endpoints

• For restriction, we use , yielding the values

fff 10

2

0

hhh+=

4

1

2

1

IITh

hh

h 22 )(=

2

1


fff N

hh

N

h

N ++ 1

2

1 +=2

1

4

1

ffffh

jj

hh

j

h

j 12212

2++=

4

1

2

1

4

1+−

The coarse-grid operator

• We compute the coarse-grid operator using the Galerkin condition

IAIA 222 h

h

hhh

h=

j2h = 0 1


A2h

=1

4h2

1 − 1

− 1 2 − 1

− 1 2 − 1⋅ ⋅ ⋅

⋅⋅ ⋅− 1 2 − 1

− 1 1

−ε

−ε

ε

ε

22202

2

22202

2

202

20

hhh

hhh

hhh

h

h

hhh

h

4

1

4

1

2

10

2

1

02

11

01

hhIAI

hhIA

I

Coarse-grid solvability

• Assuming satisfies , the solvability condition, we can show that theoretically the coarse-grid problem is also solvable.

• To be certain numerical round-off does not perturb

fh

fhh

=, 01

vAfIuA22

2h

hhhh

hh

)−(=


• To be certain numerical round-off does not perturb solvability, we incorporate a Gram-Schmidt-like step each time a new right-hand side is generated for the coarse grid: f

2h

f

ff2

22

22

22 h

hh

hh

hh

,

,

−← 111

1

Neumann problem: An example• Consider the problem ,

which has as a solution for any c(c = -1/12 gives the zero mean solution).

−=)(″− xxu 12

10 << x uu =)(′=)(′ 010

cxx

xu +−=)(32

32


grid size average number

N conv. factor of cycles31 6.30E-11 0.079 9.70E-05 9

63 1.90E-11 0.089 2.40E-05 10

127 2.60E-11 0.093 5.90E-06 10

255 3.70E-11 0.096 1.50E-06 10

511 5.70E-11 0.100 3.70E-07 10

1027 8.60E-11 0.104 9.20E-08 10

2047 2.10E-11 0.112 2.30E-08 10

4095 5.20E-11 0.122 5.70E-09 10

|||| rh |||| eh

V(2,1)cycles

• All problems considered thus far have had as the off-diagonal entries.

• We consider two situations when the matrix has two different constants on the off-diagonals.

−1

h2

7b. Anisotropic problems



two different constants on the off-diagonals. These situations arise when– the (2-d) differential equation has constant but different coefficients for the derivatives in the coordinate directions

– the discretization has constant but different mesh spacing in the different coordinate directions

We consider two types of anisotropy• Different coefficients on the derivatives

discretized on a uniform grid with spacing h .

• Different mesh spacings:

=α−− fuu yyxx


Nhhx ==

1

hh

xy

α=

hy

hx

Both problems lead to the same stencil

−+−α+

−+−

h

uuu

h

uuu kjkjkjkjkjkj +,,−,,+,,−

2

11

2

11 22

h −α+−

α−=

1


−+−+

−+− uuu

h

uuu

h

kjkjkjkjkjkj

α

+,,−,,+,,−

2

11

2

11 22

hA

h α−

−α+−

= 12211

2

Why standard multigrid fails

• Note that Ah has weak connections in the y-direction. MG convergence factors degrade as α gets small, with poor performance already at α = 0.1 .

• Consider the limiting case α ⇒ 0:

• Collection of disconnected 1-D problems!hA

h

−α+−

⇒

0

1221

01

22

hA

h α−

−α+−α−

= 12211

2


• Collection of disconnected 1-D problems!

• Point relaxation will smooth oscillatory errors in the x-direction (strong connections), but with no connections in the y-direction, the errors in that direction will generally be random; point relaxation provides no smoothing in the y-direction.

h 02

We analyze weighted Jacobi

• The eigenvalues of the weighted Jacobi iteration matrix for this problem are

λi, l = 1 −2ω

1 + α

sin2

iπ

2N

+ α sin2

lπ

2N


0

5

10

15 0

5

10

150

0.2

0.4

0.6

0.8

1

l

k

0 2 4 6 8 10 12 14 160.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 2 4 6 8 10 12 14 160.3

0.4

0.5

0.6

0.7

0.8

0.9

1

ii ll

Two strategies for anisotropy

• Semicoarsening: Because we expect MG-like convergence for the 1-D problems along lines of constant y, we could coarsen the grid in the x-direction, but not in the y-direction.


• Line relaxation: Because the the equations are strongly coupled in the x-direction, we could solve simultaneously for entire lines of unknowns in the x-direction (along lines of constant y).

Semicoarsening with point relaxation

• Point relaxation on smooths in the

x-direction. Coarsen by removing every other y-line. hA

h α−

−α+−α−

= 12211

2

Ω2hΩh


• We do not coarsen along the remaining y-lines.

• Semicoarsening is not as “fast” as full coarsening. The number of points on is about half the number of points on , instead of the usual one-fourth.

Ω

Ω2h

Ω

Ωh

Interpolation with semicoarsening

• We interpolate in the 1-D way along each line of constant y.

• The coarse-grid correction equations are


vvv 222

hkj

hkj

hkj ,,, +=

vv

vv

21

2

1212

hkj

hkj

hkj

hkj

,+,,+,+

++=

2

Line relaxation with full coarsening

• The other approach to this problem is to do full coarsening, but to relax entire lines (constant y) of variables simultaneously.

• Write Ah in block form as

D − cI


where &

Ah =

D − cI

− cI D − cI

− cI D − cI⋅ ⋅⋅

⋅⋅ ⋅ − cI

− cI D

hc

α=

2 D =1

h2

2 + 2α − 1

− 1 2 + 2α − 1⋅ ⋅⋅− 1 2 + 2α

Line relaxation

• One sweep of line relaxation consists of solving a tridiagonal system for each line of constant y.

• The kth such system has the form , where is the kth subvector of with entries & the kth right-hand side subvector is

gvD hk

hk = vh

k

vh =)( vv kjh

jhk ,


the kth right-hand side subvector is

• Because D is tridiagonal, the kth system can be solved very efficiently.

vh =)( vv kjh

jhk ,

)+(α

+=)( vvh

fg hkj

hkj

hkjj

hk +,−,, 112

Why line relaxation works• The eigenvalues of the weighted block Jacobi iteration matrix are

π

α+

π

α+

ω−=λ

N

ili

π, 2

nis2

nis

nis2

21 22

22

N

l

N

i


0

5

10

15

0

5

10

15

-0.2

0

0.2

0.4

0.6

0.8

ili l

0 2 4 6 8 10 12 14 16

-0.2

0

0.2

0.4

0.6

0.8

0 2 4 6 8 10 12 14 16

-0.2

0

0.2

0.4

0.6

0.8

Semicoarsening & line relaxation

• We might not know the direction of weak coupling or it might vary.

• Suppose we want a method that can handle either

or − α − 1


or

• We could use semicoarsening in the x-direction to handle & line relaxation in the y-direction to take care of .

A1h =

1

h2 − α

− 1 2 + 2α − 1

− α

A2h =

1

h2 − 1

− α 2 + 2α − α− 1

A1h

A2h

Semicoarsening & line relaxation


• The original grid. • Original grid viewed as a stack of “pencils.” Line relaxation is used to solve problem along each “pencil”.

• Coarsening is done by deleting every other pencil.

An anisotropic example

• Consider with u = 0 on the boundaries of the unit square, & stencil given by

• Suppose so the exact

=α−− fuu yyxx

hA

h α−

−α+−α−

= 12211

2


• Suppose so the exact solution is given by

• Observe that if α is small, the x-direction dominates, while if α is large, the y-direction dominates

f ( x, y) = 2 ( y − y2) + 2α ( x − x2)u( x, y) = ( y − y2) ( x − x2)

What is smooth error?

• Consider α = 0.001 & suppose point Gauss-Seidel is applied to a random initial guess. The error after 50 sweeps appears as:

0.08

0.1

0.12


00.2

0.40.6

0.81

0

0.5

10

0.02

0.04

0.06

0.08

0.1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.02

0.04

0.06

0.08

0.1

0.12

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.02

0.04

0.06

Error along line of constant x

Error along line of constant yxy

We experiment with 3 methods

• Standard V(2,1)-cycling, with point Gauss-Seidel relaxation, full coarsening, & linear interpolation

• Semicoarsening in the x-direction. Coarse & fine grids have the same number of points in the y-direction. 1-D full weighting & linear interpolation


direction. 1-D full weighting & linear interpolation are used in the x-direction, with no y-coupling in the intergrid transfers

• Semicoarsening in the x-direction combined with line relaxation in the y-direction. 1-D full weighting & interpolation.

With semicoarsening, the operator must change

• To account for unequal mesh spacing, the residual & relaxation operators must use a modified stencil

−

α

hy2


• Note that as grids become coarser, grows while remains constant.

A =

−1

hx2 2

hx2

+2α

hy2

−1

hx2

−1

hy2

hxhy

How do the 3 methods work for various values of α ?

• Asymptotic convergence factors:

scheme 1000 100 10 1 0.1 0.01 0.001 1E-04

V(2,1)-cycles 0.95 0.94 0.58 0.13 0.58 0.90 0.95 0.95

α

N = 16


• Note: semicoarsening in x works well for α < .001but degrades noticeably even at α = .1.

V(2,1)-cycles 0.95 0.94 0.58 0.13 0.58 0.90 0.95 0.95

semiC in x 0.94 0.99 0.98 0.93 0.71 0.28 0.07 0.07

semiC / line relax 0.04 0.08 0.08 0.08 0.07 0.07 0.08 0.08

y-direction strong x-direction strong

A semicoarsening subtlety• Suppose α is small, so that semicoarsening in x is used. As we progress to coarser grids, gets small but remains constant.

• If, on some coarse grid, becomes comparable to , then the problem effectively becomes recoupled in the y-direction. Continued

hx− 2

hy− 2

hx− 2

α h− 2


recoupled in the y-direction. Continued semicoarsening can produce artificial anisotropy, strong in the y-direction.

• When this occurs, it is best to stop semicoarsening & use full coarsening on any further coarse grids.

α hy− 2

• Non-uniform grids are commonly used for domain or data irregularities or emerging solution features

• Consider how we might approach the 1-D problem

7c. Variable meshes



posed on the following grid:

− u″( x) = f ( x) 0 < x < 1

u( 0) = u( 1 ) = 0

x0x j − 1 x j

x j + 1 xN

x = 0

We need some notation for the mesh spacing

• Let N be a positive integer. We define the spacing interval between & : x j x j + 1

hj + 1/2 ≡ x j + 1 − x j j = 0, 1, ... , N − 1


x0x j − 1 x j

x j + 1 xN

hj + 1/2

Second divided differences

u u

uj-1/2 = (uj+1/2 - uj-1/2)/hj’’ ’ ’

uj-1/2 = (uj - uj-1)/hj-1/2' Uj+1/2 = (uj+1 - uj)/hj+1/2'

↓ ↓


uj-1 ujuj+1

hj = (hj-1/2 + hj+1/2)/2

hj-1/2 hj+1/2

| |

↓ ↓

We define the discrete differential operator

• Using second-order finite differences (& wading through a mess of algebra!), we obtain the discrete representation

− αjh uj − 1

h + ( αjh + βj

h ) ujh − βj

huj + 1

h = f jh

1 ≤ j ≤ N − 1


where

• Multiply by (hj-1/2 + hj+1/2)/2 yields an SPD matrix.

u0h = uN

h = 0

αjh =

2

hj − 1/2 ( hj − 1/2 + hj + 1/2 )βj

h=

2

hj + 1/2 (hj − 1/2 + hj + 1/2 )

We modify standard multigrid to accommodate variable spacing

• We choose every second fine-grid point as a coarse-grid point

xh0

xNhx2 j

h


• In [x0 , x2 ], linear means

v (x) = v0 + (v1 - v0 ) (x - x0 )/(x2 - x0 )

• Now just plug in x = x1 to get…

x02h x 2

2N

h/x j

2h

h hh2h 2h2h

hh

h

Interpolation formula

• If , then for we have vh = I2hh

v2h 1 ≤ j ≤ N/2 − 1


v2 jh = vj

2h v2 j + 1h =

h2 j + 3/2 v j2h + h2 j + 1/2 v j + 1

2h

h2 j + 1/2 + h2 j + 3/2

We use the variational properties to derive restriction & A2h

• This produces a stencil on that is similar, but not identical, to the fine-grid stencil. If the resulting system is scaled by , then

A2h = Ih

2hA

hI2h

h Ih2h =

1

2( I2h

h )T

Ω2h


resulting system is scaled by , then the Galerkin product is the same as the fine-grid stencil.

• For 2-d problems this approach can be generalized readily to logically rectangular grids. However, for irregular grids that are not logically rectangular, AMG is a better choice.

( hj − 1/2 + hj + 1/2 )

• A common difficulty is variable coefficients, exemplified in 1-D by

where a(x) is a positive function on [0,1] • We seek to develop a conservative, or self-adjoint,

− ( a( x) u′( x) ) ′ = f ( x) 0 < x < 1

u( 0 ) = u( 1 ) = 0

7d. Variable coefficients



• We seek to develop a conservative, or self-adjoint, method for discretizing this problem.

• Assume we have available to us the values of a(x)at the midpoints of the grid

a j + 1/2 ≡ a( x j + 1/2 )

xhj xh

j + 1xh

j − 1xh0 xh

N

We discretize using central differences

• We can use second-order differences to approximate the derivatives. To use a grid spacing of h, we evaluate a(x)u’(x) at points midwaybetween the gridpoints:

( au′) |x − ( a u′) |x


x j

xhj xh

j + 1xh

j − 1xh0 xh

N

Points used to evaluate (au’ )’ at x j

( a( x) u′( x) ) ′ ≈( au′) |xj + 1/2 − ( a u′) |xj − 1/2

h+ O( h

2 )=

We discretize using central differences(cont’d)

• To evaluate , we must sample a(x) at the point & use second order differences:

where

( au′) |x j + 1/2

x j + 1/2

( a u′ ) |x j + 1/2≈ aj + 1/2

uj + 1 − uj

h( a u′ ) |x j − 1/2

≈ aj − 1/2

uj − uj − 1

h


where

a j + 1/2 ≡ a( x j + 1/2 )

xhj xh

j + 1xh

j − 1xh0 xh

N

Points used to evaluate (au’ )’ at x j

Points used to evaluate u’ at

x j + 1/2

The basic stencil

• We combine the differences for u’ & for (au’ )’ toobtain the operator

−aj + 1/2

uj + 1 − uj

h− aj − 1/2

uj − uj − 1

h

h− ( a( x j) u′ ( x j) ) ′ ( x j) ≈


& the problem becomes, for

hj j j

1

h2 ( − aj − 1/2 uj − 1 + (aj − 1/2 + aj + 1/2 ) uj − aj + 1/2 uj + 1 ) = f j

1 ≤ j ≤ N − 1

u0 = uN = 0

Coarsening the variable coefficient problem

• A reasonable approach is to use a standard multigrid algorithm with linear interpolation, full weighting, & the stencil

where

A2h =

1

( 2h) 2( − a j − 1/2

2h a j − 1/22h + a j + 1/2

2h − a j + 1/22h )


where

• The same stencil is obtained by the Galerkin relation

( 2h)

a j + 1/22h =

a2 j + 1/2h + a2 j + 3/2

h

2

Variable mesh vs. variable coefficientsafter scaling by ηj = (hj-1/2 + hj+1/2)/2 & h

• Variable mesh

• Variable coefficients

−1

hj−1/ 2

uj−1

h+ (

1

hj−1/ 2

+1

hj +1/ 2

)uj

h+

1

hj +1/ 2

uj +1

h= ηj f j

h


• Correspondenceh

1( − aj − 1/2 uj − 1 + (aj − 1/2 + aj + 1/2 ) uj − aj + 1/2 uj + 1 ) =

⇔aj−1/ 2

h⇔

aj +1/ 2

h

1

hj−1/ 2

1

hj +1/ 2

h f j

h

Standard multigrid degrades if a(x) is highly variable

• So MG for variable coefficients is equivalent to MG (with simple averaging) for Poisson’s equation on a variable mesh.

-(au’ )’ = f


• But simple averaging won’t accurately represent smooth components if is close to but far from .

-u’’ = f

x2j + 1h x2 j

h x2 j + 2h

x2j + 1hx2 j

h x2 j + 2h

x j2h x j + 1

2h

One remedy is to apply operator interpolation

• Assume that relaxation does not change smooth error, so the residual is approximately zero. Applying at yields

− a2 j + 1/2h e2 j

h + ( a2 j + 1/2h + a2 j + 3/2

h ) e2 j + 1h − a2 j + 3/2

h e2 j + 2h

= 0

x2j + 1h


• Solving for : h2

= 0

e2j + 1h

e2j + 1h =

a2 j + 1/2h ej

2h + a2 j + 3/2h ej + 1

2h

a2j + 1/2h + a2j + 3/2

h

Thus, operator induced interpolation is

v2j + 1h =

a2 j + 1/2h vj

2h + a2 j + 3/2h vj + 1

2h

ah + ah

v2 jh = vj

2h


• As usual, the restriction & coarse-grid operators are defined by the Galerkin relations

v2j + 1 =a2j + 1/2

h + a2j + 3/2h

A2h = Ih

2hA

hI2h

hIh

2h = c( I2hh ) T

A variable coefficient example

• We use V(2,1) cycle, full weighting, linear interpolation.

• We use a(x) = 1 + ρ sin(kπx) & a(x) = 1 + ρ rand(kπx)

a(x) = 1 + ρ sin(kπx)1 + ρ rand(kπx)

N = 128


k=3 k=25 k=50 k=100 k=200 k=400

0 0.085 0.085 0.085 0.085 0.085 0.085 0.085

0.25 0.084 0.098 0.098 0.094 0.093 0.083 0.083

0.5 0.093 0.185 0.194 0.196 0.195 0.187 0.173

0.75 0.119 0.374 0.387 0.391 0.39 0.388 0.394

0.85 0.142 0.497 0.511 0.514 0.514 0.526 0.472

0.95 0.191 0.681 0.69 0.694 0.699 0.745 0.672

ρ

Outline




– Analysis


– Relaxation






– Variable meshes


√√√√

√√√√

√√√√

√√√√


– Relaxation

– Coarsening

• Implementation

– Complexity

– Diagnostics

• Some Theory


– Variable meshes





– FAC

• Finite Elements


√√√√

√√√√

√√√√

8. Algebraic multigrid (AMG)

Automatically defines coarse “grid”

AMG has two distinct phases:— setup phase: define MG components— solution phase: perform MG cycles

unstructured grids, variable coefficients,…


AMG approach is opposite of geometric MG— fix relaxation (point Gauss-Seidel)— choose coarse “grids” & prolongation, P, so that error not reduced by relaxation is in range(P)

(in contrast, geometric MG fixes coarse grids,then defines suitable operators & smoothers)

— use Galerkin principle so that coarse-grid correction eliminates error in range(P)

AMG has two phases

• Setup Phase– Select coarse “grids,”

– Define interpolation,

– Define restriction & coarse-grid operators

...,,=,Ωm + 1m 21

mImm + 1 ...,,=, 21


Solve PhaseStandard MG processes: V-cycle, W-cycle, FMG, FAS, …

– Define restriction & coarse-grid operators

IITm

mmm +

+1

1 )(= IAIAmm

mmm

m+

++1

11 =

Only the selection of coarse grids does not parallelize well using existing techniques!

AMG fundamental concept:Smooth error = “small” residuals

• Consider the iterative method error recurrence

• Error that is slow to converge satisfies

eAQIe kk −+ 11 )−(=

≈)−( eeAQI−1 ≈⇒ eAQ

−10


• More precisely, it can be shown that smooth error satisfies

≈)−( eeAQI ≈⇒ eAQ 0

≈⇒ r 0

er AD− 1 « )(1

AMG uses strong connection to determine MG components

• It is easy to show from (1) that smooth error

satisfies

• Define i is strongly connected to j by

eeDeeA ,, « )(2


• Define i is strongly connected to j by

• For M-matrices, we have from (2)

implying that smooth error varies slowly in the

direction of strong connections

≤θ<,−θ≥− aa kiik

ji 10xam≠

1«22

1∑

i

ji

ii

ji

ji ≠

−

−

e

ee

a

a 2

Some useful set definitions

• The set of strong connections of a variable , that is, the variables upon whose values the value of depends, is defined as

S i

−θ>−:

= aaj jiij

ji xam≠

ui

ui


• The set of points strongly connected to a variable is denoted .

• The set of coarse-grid variables is denoted C.• The set of fine-grid variables is denoted F.• The set of interpolatory coarse-grid variables used to interpolate the value of the fine-grid variable is denoted .

ij ≠

STi ∈:= Sjj i

C iui

ui

Choosing the coarse grid

• Two Criteria

– (C1) For each , each point should either be in or should be strongly connected to at least one point in .

i ∈ F CC i

Sj ∈ i


– (C2) should be a maximal subset with the property that no two -points are strongly connected to each other.

• Satisfying (C1) & (C2) is sometimes impossible. We use (C2) as a guide while enforcing (C1).

CC

Selecting the coarse-grid points

C-point selected (point with largest “value”)

Neighbors of C-point


Neighbors of C-point become F-points

Next C-point selected (after updating “values”)

F-points selected, etc.

Examples: Laplacian operator5-pt FD, 9-pt FE (quads), & 9-pt FE (stretched quads)

9-pt FE (stretched quads)


−−−

−−−−−

111

181

111−

−−−

1

141

1

−−−

−−− 141

282

1415-pt FD 9-pt FE (quads)

Prolongation is based on smooth error, strong connections (from

M-matrices)

Smooth error is given by:C

FF

eaear jjiNj

iiii ≈+= ∑∈ i

0


Prolongation :i

CC

F ∈,ω

∈,

=)( Fie

Cie

ePkki

Ck

i

i ∑∈ i

Prolongation is based on smooth error, strong connections (from

M-matrices) Sets:

Strongly connected -pts.

Strongly connected -pts.

Weakly connected points.

CF

F

Ci C

Dsi

Dwi

F


The definition of smooth error,i

CC

F

eaea jjiij

iii −≈ ∑≠

gives

ea iii

Strong C Strong F Weak pts.

−−−≈ ∑∑∑ jji

Dj

jji

Dj

jjiCj ∈∈∈ w

isii

eaeaea

Finally, the prolongation weights are defined

• In the smooth-error relation, use for weak connections. For the strong F-points use :

ee ij =

aeae kjCk

kkjCk

j

⁄

= ∑∑∈∈ ii


yielding the prolongation weights:

aa

a

w

ni

Dn

ii

a

aa

Dj

ji

ji+

+

−=∑

∑

∈

∈

wi

mk

iCm

jkki

si∑∈

AMG setup costs:A bad rap

• Many geometric MG methods need to compute prolongation & coarse-grid operators

• The only additional expense in the AMG setup phase is


• The only additional expense in the AMG setup phase is the coarse-grid selection algorithm

• AMG setup phase is only 10-25% more expensive than in geometric MG & may be considerably less than that!

AMG performance:Sometimes a success story

• AMG performs extremely well on the model problem (Poisson’s equation, regular grid)- optimal convergence factor (e.g., 0.14) & scalability w.r.t. problem size.

• AMG appears to be both scalable & efficient on


• AMG appears to be both scalable & efficient on diffusion problems on unstructured grids (e.g., 0.1-0.3).

• AMG handles anisotropic diffusion coefficients on irregular grids reasonably well.

• AMG handles anisotropic operators on structured & unstructured grids relatively well (e.g., 0.35).

So, what could go wrong?Strong F-F connections: weights depend on each other

• For point the value is interpolated from , , & is needed to make the interpolation weights for approximating .

• For point the value is interpolated from , , & is needed to make the interpolation weights for approximating .

ei

ej k1i

k2

k2

j k1ei


approximating .

• It’s an implicit system!

Ck

Ck

1

1

∈

∈

j

i

Ck

Ck

2

2

∈

∈

j

i

Ck 3 ∈ iCk 4 ∈ ji j

ej

Is there a fix?

A Gauss-Seidel-like iterative approach to weight

definition is implemented. Usually, two passes

suffice. But does it work? Yes, frequently:

θ standard iterative2D Laplacian, N = 10,000,stretched quadrilaterals

convergence factors


θ standard iterative

0.25 0.47 0.14

0.5 0.24 0.14

0.25 0.93 0.93

0.5 0.55 0.28

∆=∆ yx 01

∆=∆ yx 001

stretched quadrilaterals

V(1,1) cycles/Gauss-Seidel (used throughout chapter 8).

How’s it perform (vol I)?regular grids, plain, old, vanilla problems,

unit square, N = 64, Dirichlet boundaries

• The Laplace operator: Convergence Time Setup

Stencil per cycle Complexity per Cycle Times

5-pt 0.054 2.21 0.29 1.63


• Anisotropic 5-Point Laplacian:

5-pt 0.054 2.21 0.29 1.63

5-pt skew 0.067 2.12 0.27 1.52

9-pt (-1,8) 0.078 1.30 0.26 1.83

9-pt (-1,-4,20) 0.109 1.30 0.26 1.83

−ε− UU yyxx

Epsilon 0.001 0.01 0.1 0.5 1 2 10 100 1000

Convergence/cycle 0.084 0.093 0.058 0.069 0.054 0.079 0.087 0.093 0.083

How’s it perform (vol II)?structured meshes, rectangular domains

• 5-point Laplacian on regular rectangular gridsConvergence factor (y-axis) plotted against number of nodes (x-axis)

0 .1 4 60 .1 4 2

0 .1 3 5 0 .1 3 3

0 .1 2

0 .1 4

0 .1 6


0 .0 5 5

0 .0 8 2

0

0 .0 2

0 .0 4

0 .0 6

0 .0 8

0 .1

0 .1 2

0 5 0 0 0 0 0 1 0 0 0 0 0 0

How’s it perform (vol III)?unstructured meshes, rectangular domains

• Laplacian on random unstructured grids (regular triangulations, 15-20% nodes randomly collapsed into neighboring nodes)

Convergence factor (y-axis) plotted against number of nodes (x-axis)

0 .2 5 30 . 2 5

0 . 3


0 .0 9 70 .1 1 1

0 .2 5 3

0 .1 7 5

0 .2 2 3

0

0 . 0 5

0 . 1

0 . 1 5

0 . 2

0 . 2 5

0 2 0 0 0 0 4 0 0 0 0 6 0 0 0 0

How’s it perform (vol IV)?isotropic diffusion, structured/unstructured grids

on structured, unstructured grids

0

0 . 0 5

0 . 1

0 . 1 5

0 . 2

0 . 2 5

0 . 3

0 . 3 5

0 . 4

6 a

6 b

7 a

7 b

8 a

8 b

9 a

9 b

)∇),((•∇ uyxd


Problems used: “a” means parameter c = 10, “b” means c = 1000

0

016 −+.=),(: yxcyxd

50

50017

.>

.≤.

=),(:xc

xyxd

52050505210018

.≤.−,.−≤..

=),(:c

yxyxd

esiwrehto

xam

52050505210019 .≤).−(+).−(≤..

=),(:c

yxyxd22

esiwrehto

N=16642 N=66049 N=13755 N=54518Structured Structured Unstruct. Unstruct.

How’s it perform (vol V)?Laplacian operator, unstructured grids

Convergence factor

0.2198 0.2237

0.2754

0.2

0.25

0.3


0.1002

0.1715

0.1888

0

0.05

0.1

0.15

0.2

0 50000 100000 150000 200000 250000 300000 350000

Gridpoints

AMG for systems

• How can we do AMG on systems?

• Naïve approach: “Block” AMG (block Gauss-Seidel, using scalar AMG to “solve” at each cycle)

=

g

f

v

u

AA

AA

2212

2111


using scalar AMG to “solve” at each cycle)

uAgAv

vAfAu

)−()(←

)−()(←

121

22

211

11

−

−

Great Idea! Except that it often doesn’t work!

Block AMG doesn’t account for strong inter-variable coupling.

AMG for systems: A solution

• To solve the system problem, allow interaction between the unknowns at all levels:

& AA

AAA

kk

kkk

=2212

2111

I

I

I

v

kk

u

kk

kk

+

++

1

1

1

)(

)(

=0

0


• This is called the “unknown” approach.

• 2-D biharmonic, Dirichlet boundaries, unit square, uniform quadrilateral mesh:

AA 2212I

vk +1 )( 0

Mesh spacing 0.1250.0625 0.03135 0.015625

Convergence factor 0.22 0.35 0.42 0.44

Adaptive AMG (ααααAMG)to broaden applicability

adaptive interpolation

to determine sense of smoothness

+


+

adaptive C-point choice

to determine strength of connection

Adaptive interpolation

Standard AMG collapses stencils by assuming smooth error is locally constant (Poisson “sense of smoothness”):

aeae kjkkjj

⁄

= ∑∑*** smooth ≈ 1 ***


ea iii

Strong C Strong F Weak pts.

−−−≈ ∑∑∑ jji

Dj

jji

Dj

jjiCj ∈∈∈ w

isii

eaeaea

kjCk

kkjCk

j

∑∑∈∈ ii

= eije

*** smooth ≈ 1 ***

Pre-relaxation

• What if we found a smooth error x that’s far from 1 ?

• If we found that xj = 1.6xi for j∈Diw, say, then we could

just make the replacement ej → 1.6ei .

• If, say, xj = 0.2xk for j∈Dis & all k∈Ci, then we could set


ej → ( Σ ajk 0.2ek )/( Σ ajk ).

j∈Ci j∈Ci

• How do we obtain a representative smooth error x ?

Relax on Ax = 0 !!!

Adaptive C-point choice

• AMG uses the ansatz that residuals are 0 at F-points.

• What if we had rf = 0 exactly?

A ff A fc

Acf Acc

e f

ec

=

0

rc

⇒ A ff e f + A fcec = 0 ⇒

e f

ec

=

−A ff

−1A fc

I

ec


• Choosing (ideal interpolation) makes MG exact:P =

−A ff

−1A fc

I

MGe = [CG]Pec = I − P[PTAP]−1

PTA[ ]Pec

= Pec − P[PTAP]−1[PT

AP]ec = 0.

Compatible relaxation

• How do we make the residuals zero at F-points?

Relax on Affxf = -Afcxc !!!

• This is the same as relaxing on the F-points while maintaining 0 at the C-points.


maintaining 0 at the C-points.

• If Aff is well-conditioned, then this converges quickly.• Quick “CR” convergence means that the F-point residuals will be small after relaxation & that there is thus a P(the ideal one) that gives good MG convergence.

• Is there a good local P ? Maybe, since CR is local & fast.

Outline




– Analysis


– Relaxation






– Variable meshes







– Variable meshes

√√√√

√√√√

√√√√

√√√√


– Relaxation

– Coarsening

• Implementation

– Complexity

– Diagnostics

• Some Theory


– Variable meshes





– FAC

• Finite Elements


– Variable meshes





– FAC

• Finite Elements


√√√√

√√√√

√√√√

√√√√

9. Multilevel adaptive methodsfast adaptive composite grid method (FAC)

two-spike problem


Model 1-D problem

- u”(x) = f(x), 0 < x < 1u(0) = u(1) = 0


h-2 (- uhi-1 + 2uhi - uhi-1 ) = fhi ,1 ≤ i ≤ 7,

uh0 = uh8 = 0, h = 1/8

0 1

Local refinement

suppose f(x) has a spike at x = 3/4but is smooth elsewhere

y= f(x)


Ofine grid ( = interface)

coarse grid

x

Strategy

• Recognize that there’s little value of having the fine grid in the smooth region, [0, 1/2].

• Start with standard MG– eliminate relaxation– eliminate intergrid transfers & residual calculations


– eliminate intergrid transfers & residual calculations

• Interpret this process via the composite grid (= 2h-points in [0, 1/2] + h-points in [1/2, 1]).

• We’ll try absurdly hard to eliminate work @ x = 1/2, but we have in mind multi-dimensions & smaller patches.

Eliminate relaxation

• Initialize vh = 0 & f12h (f1h + 2f2h + f3h )/4 . • Relax on vh on the local fine grid.• Compute rh = fh - Ah vh & transfer to 2h:

notationvh = (vih )


• Compute r = f - A v & transfer to 2h:f22h (r3h + 2r4h + r5h )/4 & f32h (r5h + 2r6h + r7h )/4 .

• Compute an approximation, v2h, to the solution of the 2hresidual equation, A2h u2h = f2h .

• Update the residual at x12h for later cycles:f12h f12h - (- v02h + 2 v12h - v22h )/(2h)2.

• Correct: vh vh + Ih2hv2h.

Eliminate more

• Initialize vh = 0, w12h = 0, & f12h (f1h + 2f2h + f3h )/4 . • Relax on vh on the local fine grid.• Compute rh = fh - Ah vh (not at x1h or x2h) & transfer to 2h:f 2h (r h + 2r h + r h )/4 & f 2h (r h + 2r h + r h )/4 .

f12h involves rh at x3h & x4h!

Store vh only @ x3h - x7h & save v2h w12h !


f22h (r3h + 2r4h + r5h )/4 & f32h (r5h + 2r6h + r7h )/4 .• Compute an approximation, v2h, to the solution of the 2hresidual equation, A2h u2h = f2h .


• Accumulate the 2h approximation: w12h w12h + v12h.• Correct: vh vh + Ih2hv2h (not @ x1h or x2h).

Eliminate the restr3h doesn’t change on h.Compute change in r3h.

• Initialize vh = 0, w12h = 0, & f12h (f1h + 2f2h + f3h )/4 . • Relax on vh on the local fine grid.• Compute the right sides for 2h: f22h g12h - (- w12h + 2 w22h - w32h )/(2h)2 &

2h residual @ x1h = g12h - (- w12h + 2 w22h - w32h )/(2h)2 !


f2 g1 - (- w1 + 2 w2 - w3 )/(2h) & f32h (r5h + 2r6h + r7h )/4 .

• Compute an approximation, v2h, to the solution of the 2hresidual equation, A2h u2h = f2h .


• Accumulate the 2h approximation: w12h w12h + v12h.• Correct: vh vh + Ih2hv2h (not @ x1h or x2h or x3h).

Interpretationcomposite grid


(- v0c + 2 v1c - v2c )/(2h)2 = f1c ≡ (f1h + 2 f2h + f3h )/4

(- v1c + 2 v2c - v3c )/(2h)2 = f2c ≡ (f3h + 2 f4h)/4

(- vci-1 + 2 vic - vci+1 )/h = fic ≡ fhi+2

Issues

• Adaptivity

• Error estimates

• Norms (proper scaling)

• Multiple dimensions– Slave points


– Slave points

– More complicated stencils

• Data structures

• Parallel algorithms (AFAC)

• Time-space

Two-spike example (Laplacian)

• Global grid h & one two-patch refined level h/2.• V(1,0)-cycles, Gauss-Seidel

• Asymptotic convergence of the solver

• Scaled L2 discretization error estimate


Outline




– Analysis


– Relaxation






– Variable meshes







– Variable meshes

√√√√

√√√√

√√√√

√√√√


– Relaxation

– Coarsening

• Implementation

– Complexity

– Diagnostics

• Some Theory


– Variable meshes





– FAC

• Finite Elements


– Variable meshes





– FAC

• Finite Elements


√√√√

√√√√

√√√√

√√√√

√√√√

10. Finite elementsFE, a variational methodology

FE methodologyFD methodology


method methodmethod

Discretization

• •

What do you see?


•

• • ••

• •••

Localize

••

What choice do you have?


•

Continuous piecewise linear functions

• • •• •••


•

• • ••

• ••

Representationa basis of “hat” functions

• • • • •••••

ε(i) is piecewise linear &0 at all nodes exceptnode i where it’s 1

h

hε(i)


• • • • ••••a general piecewise linear

function can be represented by

uh = Σi ui ε(i)h h

How do we use piecewise linears?

• • ••

How do you take 2nd derivatives of this ???


•

• • ••

• •••

Weak form!

−u"= f

⇒ − u"v = fv ∀v

⇒ (−u")vdx =∫ fvdx ∀v∫


∫ ∫⇒ u'v'dx − u'v |0

1 =∫ fvdx ∀v∫⇒ u'v'dx =∫ fvdx ∀v∫ ∋ v(0) = v(1) = 0

⇒ u',v '( )= f ,v( ) ∀v ∋ v(0) = v(1) = 0

h hΣjuj ε(j)↓

ε(i)h↓

FE basics

• Model problem Ω ⊂ ℜ2

Lu ≡ - uxx - uyy = f in Ω

u = 0 on ∂Ω

• Sobolev spaces

L2(Ω) = u : ∫Ωu2dΩ < ∞

H 1( ) = u : u, u , u L2( ), u| = 0

• Duality: Solving Lu = f is equivalent to minimizing

the functional

F(u) = “(Lu, u)”/2 - (f, u)

• Short story:


H01(Ω) = u : u, ux , uy ε L2(Ω), u|∂Ω = 0

(u, v) = ∫Ωu v dΩ

• L is self-adjoint positive definite:

(Lu, v) = (∇u, ∇v) = ∫Ω(uxvx+uyvy)dΩ = (u, Lv)(Lu, u) > 0 if u ≠ 0

• Short story:

1st derivative test

∇F(u) = Lu - f = 02nd derivative test

F”(u) = L > 0

∇u =ux

uy

Long story

Using symmetry & linearity of L & bilinearity of the inner product:

F(u + v) = (L(u + v), u + v)/2 - (f, u + v)

= [(Lu, u) + 2(Lu, v) + (Lv, v)]/2

- (f, u) - (f, v)

= F(u) + (Lu, v) - (f, v) + (Lv, v)/2


= F(u) + (Lu, v) - (f, v) + (Lv, v)/2

= F(u) + (Lu - f, v) + (Lv, v)/2

The last term is positive for v ≠ 0. Thus,

F(u + v) ≥ F(u) ⇔ (Lu, v) = (f, v) ∀ v ε H01(Ω)

⇔ Lu = f

FE constructs

• Assume that Ω is the unit square.

• Consider an nxn grid of square “cells”.

• Continuous piecewise bilinear elements:Hh ⊂ H01(Ω).

• Each uh in Hh is determined by its nodevalues. This is how we’ll represent them!

• Within each square:


• Within each square:uh = axy + bx + cy + duh is linear in a coordinate direction

• If uh on one side of an element matches uh

on the other at the nodes, then it matches on the common edge. So uh is continuous.

Continuity

••


•

FE discretization

• Minimize F(u) = (Luh, uh)/2 - (f, uh)

over uh ε Hh.

• Equilavent to solving

(Luh, vh) = (f, vh) ∀ vh ε Hh.


• Basis: ε(ij) is the element of Hh that equals 1 @ node ij & 0 elsewhere.

• Expansion: uh(x,y) = Σij uij ε(ij) (x,y).

• Problem: What is Luh ≡ - uxx - uyy ???

h h

h

hh

Weak form

• The Gauss Divergence Theorem & homogeneous

boundary conditions yield

(Lu, v) = (- uxx - uyy, v) = (-∇•∇u, v) = (∇u, ∇v)

• Note:

(∇u, ∇v) = ∫ (uxvx + uyvy)dΩ.


(∇u, ∇v) = ∫Ω(uxvx + uyvy)dΩ.

• So the problem becomes

(∇uh, ∇vh) = (f, vh) ∀ vh ε Hhor

∫Ω(uxvx + uyvy)dΩ = ∫ΩfvhdΩ ∀ vh ε Hh.

h h h h

Towards the matrix equationfor nodal values

(∇uh, ∇vh) = (f, vh) ∀ vh ε H h

• Using uh(x,y) = Σij uij ε(ij) (x,y) & choosing vh = ε(kl) :Σij uij (∇ε(ij), ∇ε(kl)) = (f, ε(kl)) ∀ k, l

• Matrix terms (∇ε(ij), ∇ε(kl)) are 0 when |i - k| or |j - l| > 1.

h h h

h h h h

h h


|i - k| or |j - l| > 1.• We compute

(∇ε(ij), ∇ε(ij)) = 8/3 & (∇ε(ij), ∇ε(i±1j ±1)) = -1/3.• Assume f is bilinear:

(f, ε(kl)) = ∫Ωf ε(kl)dΩ ≈ h2f(xkl).hh h h

h h

actually,Bh fh

(mass matrix)huh = (uij) & fh = (h2f(xkl))

The matrix equationAhuh = fh

where

uh = (uij) & fh = (h2f(xkl))

& the matrix is given by the stencil

h


Aij

h =1

3

−1 −1 −1

−1 8 −1

−1 −1 −1

. stiffness matrix

Some matrix properties

• Symmetric! ij “reaches” to i±1j±1 as i±1j±1 to ij. • Singular? Ah 1 = 0 ?! Not next to boundaries!

Aij

h =1

3

−1 −1 −1

−1 8 −1

−1 −1 −1


• Singular? Ah 1 = 0 ?! Not next to boundaries!

@ west boundary:

• Positive definite!

Diagonally dominant (strictly so @ boundaries).

Aij

h =1

3

−1 −1

8 −1

−1 −1

Abstract FE relaxationfocus on functions, not nodal vectors

• Relaxation involves “local” changes: uh ← uh - s εh for some scalar s & εh = ε(ij).

But how do we pick s ???

• Use FE principle of minimizing F(uh - sεh) over s :

F(uh - sεh) = (L(uh - sεh), uh - sεh)/2 - (f, uh - sεh)

h


F(uh - sεh) = (L(uh - sεh), uh - sεh)/2 - (f, uh - sεh) = F(uh) - s (Luh - f, εh) + (s2/2) (Lεh, εh).

• So just pick s by taking the derivative of this quadratic polynomial & setting it to 0: s = (Luh - f, εh)/(Lεh, εh).

This is just Gauss-Seidel!

Abstract FE coarseningagain with focus on functions

• Coarsening involves a “global” change: uh ← uh + w2h for some coarse-grid function w2h.

But how do we pick w2h ???

• Use FE principle of minimizing F(uh + w2h) over w2h:

F(u + w ) = (L(u + w ), u + w )/2 - (f, u + w )


F(uh + w2h) = (L(uh + w2h), uh + w2h)/2 - (f, uh + w2h) = F(uh) + (Luh - f, w2h) + (Lw2h, w2h)/2.

• So just pick w2h by taking the gradient of this quadratic functional & setting it to 0. This is tricky because you need to write the gradient as a function in the subspace H2h. We go instead from abstract functions to nodal vectors…

Concrete FE coarsening: I2hnow with focus on nodal vectors

• Adding nodal representations of vh & v2h (ε = ε(ij)) :v2h(x,y) = Σij v2hij ε2h (x,y)

2h nodal values

= Σij vhij εh (x,y) ???h nodal values

(sum over 2h indices↔ even h indices)

h


• We should be able to do this because v2h εH2h ⊂ Hh.

• Row 2j:vh2i 2j = v2hijvh2i+1 2j = (v2hij + v2hi+1 j)/2vh2i 2j+1 = (v2hij + v2hij+1)/2vh2i+1 2j+1 = (v2hij + v2hi+1 j +

v2hij+1 + v2hi+1 j+1)/4

Concrete FE coarsening (cont’d)Ahuh = fh

• Solving this matrix equation is equivalent to minimizing

Fh(vh) ≡ (Ah vh, vh)/2 - (fh, vh) (parens here mean Euclidean norm)over vh ε Hh. So how do we now correct vh ???

• We minimize Fh(vh + I2h v2h) over v2h ε H2h :

Fh(vh + I2h v2h) h


Fh(vh + I2h v2h)

= (Ah (vh + I2h v2h), vh + I2h v2h)/2 - (fh, vh + I2h v2h)

= Fh(vh) + (A2h v2h, v2h)/2 - (f2h, v2h)

h

h h h

F2h (v2h)≡ Fh(vh) +

variational conditions

f2h = I2h(fh- Ah vh)A2h = I2hAhI2h

hT

hhT

Outline




– Analysis


– Relaxation






– Variable meshes







– Variable meshes

√√√√

√√√√

√√√√

√√√√


– Relaxation

– Coarsening

• Implementation

– Complexity

– Diagnostics

• Some Theory


– Variable meshes





– FAC

• Finite Elements


– Variable meshes





– FAC

• Finite Elements


√√√√

√√√√

√√√√

√√√√

√√√√

√√√√

Multigrid rules!

We conclude with a few observations:

o We have barely scratched the surface of the myriad ways that multigrid has been, & can be, employed.


o With diligence & care, multigrid can be made to handle many types of complications in a robust, efficient manner.

o Further extensions to multigrid methodology are being sought by many people working on many different problems.

Multigrid/multilevel/multiscalean important methodology

• Multigrid has proved successful on a wide variety of problems, especially elliptic PDEs, but has also found application in parabolic & hyperbolic PDEs, integral equations, evolution problems, geodesic problems, …


• Multigrid can be optimal, often O(N).

• Multigrid can be robust in a practical sense.

• Multigrid is of great interest because it is one of the very few scalable algorithms, & it can be parallelized readily & efficiently!

multigrid tutorial

Documents