![Page 1: 05/11/2005 Carnegie Mellon School of Computer Science Aladdin Lamps 05 Combinatorial and algebraic tools for multigrid Yiannis Koutis Computer Science](https://reader035.vdocuments.net/reader035/viewer/2022062620/5519e3b15503467a178b487b/html5/thumbnails/1.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
Combinatorial and algebraic tools for
multigridYiannis Koutis
Computer Science DepartmentCarnegie Mellon University
![Page 2: 05/11/2005 Carnegie Mellon School of Computer Science Aladdin Lamps 05 Combinatorial and algebraic tools for multigrid Yiannis Koutis Computer Science](https://reader035.vdocuments.net/reader035/viewer/2022062620/5519e3b15503467a178b487b/html5/thumbnails/2.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
multilevel methods
www.mgnet.org• 3500 citations• 25 free software packages• 10 special conferences since 1983
Algorithms not always workingLimited theoretical understanding
![Page 3: 05/11/2005 Carnegie Mellon School of Computer Science Aladdin Lamps 05 Combinatorial and algebraic tools for multigrid Yiannis Koutis Computer Science](https://reader035.vdocuments.net/reader035/viewer/2022062620/5519e3b15503467a178b487b/html5/thumbnails/3.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
multilevel methods: our goals
• provide theoretical understanding• solve multilevel design problems• small changes in current software
• study structure of eigenspaces of Laplacians
• extensions for multilevel eigensolvers
![Page 4: 05/11/2005 Carnegie Mellon School of Computer Science Aladdin Lamps 05 Combinatorial and algebraic tools for multigrid Yiannis Koutis Computer Science](https://reader035.vdocuments.net/reader035/viewer/2022062620/5519e3b15503467a178b487b/html5/thumbnails/4.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
Overview
• Quick definitions• Subgraph preconditioners• Support graph preconditioners• Algebraic expressions• Low frequency eigenvectors and good
partitionings• Multigrid introduction and current state • Multigrid – Our contributions
![Page 5: 05/11/2005 Carnegie Mellon School of Computer Science Aladdin Lamps 05 Combinatorial and algebraic tools for multigrid Yiannis Koutis Computer Science](https://reader035.vdocuments.net/reader035/viewer/2022062620/5519e3b15503467a178b487b/html5/thumbnails/5.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
quick definitions
• Given a graph G, with weights wij
• Laplacian: A(i,j) = -wij, row sums =0
• Normalized Laplacian:
• (A,B) is a measure of how well B approximates A (and vice-versa)
![Page 6: 05/11/2005 Carnegie Mellon School of Computer Science Aladdin Lamps 05 Combinatorial and algebraic tools for multigrid Yiannis Koutis Computer Science](https://reader035.vdocuments.net/reader035/viewer/2022062620/5519e3b15503467a178b487b/html5/thumbnails/6.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
linear systems : preconditioning
• Goal: Solve Ax = b via an iterative method
• A is a Laplacian of size n with m edges. Complexity depends on (A,I) and m
• Solution: Solve B-1Ax = B-1b• Bz=y must be easily solvable• (A,B) is small• B is the preconditioner
![Page 7: 05/11/2005 Carnegie Mellon School of Computer Science Aladdin Lamps 05 Combinatorial and algebraic tools for multigrid Yiannis Koutis Computer Science](https://reader035.vdocuments.net/reader035/viewer/2022062620/5519e3b15503467a178b487b/html5/thumbnails/7.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
Overview
• Quick definitions• Subgraph preconditioners• Support graph preconditioners• Algebraic expressions• Low frequency eigenvectors and good
partitionings• Multigrid introduction and current state • Multigrid – Our contributions
![Page 8: 05/11/2005 Carnegie Mellon School of Computer Science Aladdin Lamps 05 Combinatorial and algebraic tools for multigrid Yiannis Koutis Computer Science](https://reader035.vdocuments.net/reader035/viewer/2022062620/5519e3b15503467a178b487b/html5/thumbnails/8.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
combinatorial preconditionersthe Vaidya thread
• B is a sparse subgraph of A, possibly with additional edges
Solving Bz=y is performed as follows:1. Gaussian elimination on degree ·2 nodes
of B2. A new system must be solved 3. Recursively call the same algorithm on
to get an approximate solution.
![Page 9: 05/11/2005 Carnegie Mellon School of Computer Science Aladdin Lamps 05 Combinatorial and algebraic tools for multigrid Yiannis Koutis Computer Science](https://reader035.vdocuments.net/reader035/viewer/2022062620/5519e3b15503467a178b487b/html5/thumbnails/9.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
combinatorial preconditionersthe Vaidya thread
• Graph Sparsification [Spielman, Teng]• Low stretch trees [Elkin, Emek,
Spielman, Teng]• Near optimal O(m poly( log n))
complexity
![Page 10: 05/11/2005 Carnegie Mellon School of Computer Science Aladdin Lamps 05 Combinatorial and algebraic tools for multigrid Yiannis Koutis Computer Science](https://reader035.vdocuments.net/reader035/viewer/2022062620/5519e3b15503467a178b487b/html5/thumbnails/10.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
combinatorial preconditionersthe Vaidya thread
• Graph Sparsification [Spielman, Teng]• Low stretch trees [Elkin, Emek, Spielman,
Teng]• Near optimal O(m poly( log n)) complexity
• Focus on constructing a good B• (A,B) is well understood – B is sparser than
A• B can look complicated even for simple
graphs A
![Page 11: 05/11/2005 Carnegie Mellon School of Computer Science Aladdin Lamps 05 Combinatorial and algebraic tools for multigrid Yiannis Koutis Computer Science](https://reader035.vdocuments.net/reader035/viewer/2022062620/5519e3b15503467a178b487b/html5/thumbnails/11.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
Overview
• Quick definitions• Subgraph preconditioners• Support graph preconditioners• Algebraic expressions• Low frequency eigenvectors and good
partitionings• Multigrid introduction and current state • Multigrid – Our contributions
![Page 12: 05/11/2005 Carnegie Mellon School of Computer Science Aladdin Lamps 05 Combinatorial and algebraic tools for multigrid Yiannis Koutis Computer Science](https://reader035.vdocuments.net/reader035/viewer/2022062620/5519e3b15503467a178b487b/html5/thumbnails/12.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
combinatorial preconditioners
the Gremban - Miller thread• the support graph S is bigger than A
1 12 3 1 2 2 1 1
![Page 13: 05/11/2005 Carnegie Mellon School of Computer Science Aladdin Lamps 05 Combinatorial and algebraic tools for multigrid Yiannis Koutis Computer Science](https://reader035.vdocuments.net/reader035/viewer/2022062620/5519e3b15503467a178b487b/html5/thumbnails/13.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
combinatorial preconditioners
the Gremban - Miller thread• the support graph S is bigger than A
2
12 3 1 2 2 1 1
1 12 3 1 2 2 1 1
3 2 1
1 3 3 4 4 3 4 3 2 1
Quotient
1
![Page 14: 05/11/2005 Carnegie Mellon School of Computer Science Aladdin Lamps 05 Combinatorial and algebraic tools for multigrid Yiannis Koutis Computer Science](https://reader035.vdocuments.net/reader035/viewer/2022062620/5519e3b15503467a178b487b/html5/thumbnails/14.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
combinatorial preconditioners
the Gremban - Miller thread• The preconditioner S is often a
natural graph • S inherits the sparsity properties of A• S is equivalent to a dense graph B of
size equal to that of A : (A,S) = (A,B)• Analysis of (A,S) made easy by work of
[Maggs, Miller, Ravi, Woo, Parekh]
• Existence of good S by work of [Racke]
![Page 15: 05/11/2005 Carnegie Mellon School of Computer Science Aladdin Lamps 05 Combinatorial and algebraic tools for multigrid Yiannis Koutis Computer Science](https://reader035.vdocuments.net/reader035/viewer/2022062620/5519e3b15503467a178b487b/html5/thumbnails/15.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
Overview
• Quick definitions• Subgraph preconditioners• Support graph preconditioners• Algebraic expressions• Low frequency eigenvectors and good
partitionings• Multigrid introduction and current state • Multigrid – Our contributions• Other results
![Page 16: 05/11/2005 Carnegie Mellon School of Computer Science Aladdin Lamps 05 Combinatorial and algebraic tools for multigrid Yiannis Koutis Computer Science](https://reader035.vdocuments.net/reader035/viewer/2022062620/5519e3b15503467a178b487b/html5/thumbnails/16.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
• Suppose we are given m clusters in A
• R(i,j) = 1 if the jth cluster contains node i
• R is n x m • Quotient
• R is the clustering matrix
algebraic expressions
![Page 17: 05/11/2005 Carnegie Mellon School of Computer Science Aladdin Lamps 05 Combinatorial and algebraic tools for multigrid Yiannis Koutis Computer Science](https://reader035.vdocuments.net/reader035/viewer/2022062620/5519e3b15503467a178b487b/html5/thumbnails/17.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
• The inverse preconditioner
• The normalized version
• RT D1/2 is the weighted clustering matrix
algebraic expressions
![Page 18: 05/11/2005 Carnegie Mellon School of Computer Science Aladdin Lamps 05 Combinatorial and algebraic tools for multigrid Yiannis Koutis Computer Science](https://reader035.vdocuments.net/reader035/viewer/2022062620/5519e3b15503467a178b487b/html5/thumbnails/18.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
Overview
• Quick definitions• Subgraph preconditioners• Support graph preconditioners• Algebraic expressions• Low frequency eigenvectors and good
partitionings• Multigrid introduction and current state • Multigrid – Our contributions• Other results
![Page 19: 05/11/2005 Carnegie Mellon School of Computer Science Aladdin Lamps 05 Combinatorial and algebraic tools for multigrid Yiannis Koutis Computer Science](https://reader035.vdocuments.net/reader035/viewer/2022062620/5519e3b15503467a178b487b/html5/thumbnails/19.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
good partitions and low frequency invariant
subspaces• Suppose the graph A has a good
clustering defined by the clustering matrix R
• Let• Let y be any vector such that
![Page 20: 05/11/2005 Carnegie Mellon School of Computer Science Aladdin Lamps 05 Combinatorial and algebraic tools for multigrid Yiannis Koutis Computer Science](https://reader035.vdocuments.net/reader035/viewer/2022062620/5519e3b15503467a178b487b/html5/thumbnails/20.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
• Suppose the graph A has a good clustering defined by the clustering matrix R
• Let• Let y be any vector such that
Theorem: The inequality is tight up to a constant for
certain graphs
good partitions and low frequency invariant
subspaces
quality
test?
![Page 21: 05/11/2005 Carnegie Mellon School of Computer Science Aladdin Lamps 05 Combinatorial and algebraic tools for multigrid Yiannis Koutis Computer Science](https://reader035.vdocuments.net/reader035/viewer/2022062620/5519e3b15503467a178b487b/html5/thumbnails/21.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
good partitions and low frequency invariant
subspaces• Let y be any vector such that • Let x be mostly a linear combination of
eigenvectors corresponding to eigenvalues close to
Theorem: • Prove ?• We can find random vector x and check
the distance to the closest y
![Page 22: 05/11/2005 Carnegie Mellon School of Computer Science Aladdin Lamps 05 Combinatorial and algebraic tools for multigrid Yiannis Koutis Computer Science](https://reader035.vdocuments.net/reader035/viewer/2022062620/5519e3b15503467a178b487b/html5/thumbnails/22.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
Overview
• Quick definitions• Subgraph preconditioners• Support graph preconditioners• Algebraic expressions• Low frequency eigenvectors and good
partitionings• Multigrid introduction and current state • Multigrid – Our contributions
![Page 23: 05/11/2005 Carnegie Mellon School of Computer Science Aladdin Lamps 05 Combinatorial and algebraic tools for multigrid Yiannis Koutis Computer Science](https://reader035.vdocuments.net/reader035/viewer/2022062620/5519e3b15503467a178b487b/html5/thumbnails/23.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
multigrid – short introduction
• General class of algorithms
• Richardson iteration:
• High frequency components are reduced:
![Page 24: 05/11/2005 Carnegie Mellon School of Computer Science Aladdin Lamps 05 Combinatorial and algebraic tools for multigrid Yiannis Koutis Computer Science](https://reader035.vdocuments.net/reader035/viewer/2022062620/5519e3b15503467a178b487b/html5/thumbnails/24.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
initial and smoothed error
initial error smoothed error
![Page 25: 05/11/2005 Carnegie Mellon School of Computer Science Aladdin Lamps 05 Combinatorial and algebraic tools for multigrid Yiannis Koutis Computer Science](https://reader035.vdocuments.net/reader035/viewer/2022062620/5519e3b15503467a178b487b/html5/thumbnails/25.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
• Define a smaller graph Q• Define a projection operator
Rproject• Define a lift operator Rlift
the basic multigrid algorithm
1. Apply t rounds of smoothing 2. Take the residual r = b-Axold
3. Solve Qz = Rprojectr4. Form new iterate xnew = xold + Rlift z
5. Apply t rounds of smoothing
how many? which
iteration ?
recursion
is this needed ?
![Page 26: 05/11/2005 Carnegie Mellon School of Computer Science Aladdin Lamps 05 Combinatorial and algebraic tools for multigrid Yiannis Koutis Computer Science](https://reader035.vdocuments.net/reader035/viewer/2022062620/5519e3b15503467a178b487b/html5/thumbnails/26.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
algebraic multigrid (AMG)
Goals: The range of Rproject must approximate the unreduced error very well. The error not reduced by smoothing must be reduced by the smaller grid.
![Page 27: 05/11/2005 Carnegie Mellon School of Computer Science Aladdin Lamps 05 Combinatorial and algebraic tools for multigrid Yiannis Koutis Computer Science](https://reader035.vdocuments.net/reader035/viewer/2022062620/5519e3b15503467a178b487b/html5/thumbnails/27.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
algebraic multigrid (AMG)Goals: The range of Rproject must approximate
the unreduced error very well. The error not reduced by smoothing must be reduced in the smaller grid.
• Jacobi iteration: • or ‘scaled’ Richardson:• Find a clustering • Rproject = (Rlift)T
• Q = RprojectT A Rproject
![Page 28: 05/11/2005 Carnegie Mellon School of Computer Science Aladdin Lamps 05 Combinatorial and algebraic tools for multigrid Yiannis Koutis Computer Science](https://reader035.vdocuments.net/reader035/viewer/2022062620/5519e3b15503467a178b487b/html5/thumbnails/28.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
algebraic multigrid (AMG)Goals: The range of Rproject must approximate
the unreduced error very well. The error not reduced by smoothing must be reduced in the smaller grid.
• Jacobi iteration: • or ‘scaled’ Richardson• Find a clustering [heuristic]• Rproject = (Rlift)T [heuristic]
• Q = RprojectT A Rproject
![Page 29: 05/11/2005 Carnegie Mellon School of Computer Science Aladdin Lamps 05 Combinatorial and algebraic tools for multigrid Yiannis Koutis Computer Science](https://reader035.vdocuments.net/reader035/viewer/2022062620/5519e3b15503467a178b487b/html5/thumbnails/29.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
two level analysis
• Analyze the maximum eigenvalue of
• where
• The matrix T1 eliminates the error in
• A low frequency eigenvector has a significant component in
![Page 30: 05/11/2005 Carnegie Mellon School of Computer Science Aladdin Lamps 05 Combinatorial and algebraic tools for multigrid Yiannis Koutis Computer Science](https://reader035.vdocuments.net/reader035/viewer/2022062620/5519e3b15503467a178b487b/html5/thumbnails/30.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
two level analysis
• Starting hypothesis: Let X be the subspace corresponding to eigenvalues smaller than . Let Y be the null space of Rproject. Assume, <X,Y>2 · /
• Two level convergence : error reduced by
• Proving the hypothesis ? Limited cases
![Page 31: 05/11/2005 Carnegie Mellon School of Computer Science Aladdin Lamps 05 Combinatorial and algebraic tools for multigrid Yiannis Koutis Computer Science](https://reader035.vdocuments.net/reader035/viewer/2022062620/5519e3b15503467a178b487b/html5/thumbnails/31.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
current state
‘there is no systematic AMG approach that has proven effective in any kind of general context’
[BCFHJMMR, SIAM Journal on Scientific Computing, 2003]
![Page 32: 05/11/2005 Carnegie Mellon School of Computer Science Aladdin Lamps 05 Combinatorial and algebraic tools for multigrid Yiannis Koutis Computer Science](https://reader035.vdocuments.net/reader035/viewer/2022062620/5519e3b15503467a178b487b/html5/thumbnails/32.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
Overview
• Quick definitions• Subgraph preconditioners• Support graph preconditioners• Algebraic expressions• Low frequency eigenvectors and good
partitionings• Multigrid introduction and current state • Multigrid – Our contributions
![Page 33: 05/11/2005 Carnegie Mellon School of Computer Science Aladdin Lamps 05 Combinatorial and algebraic tools for multigrid Yiannis Koutis Computer Science](https://reader035.vdocuments.net/reader035/viewer/2022062620/5519e3b15503467a178b487b/html5/thumbnails/33.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
our contributions – two level
• There exists a good clustering given by R. The quality is measured by the condition number (A,S)
• Q = RT A R• Richardson’s with
• Projection matrix
![Page 34: 05/11/2005 Carnegie Mellon School of Computer Science Aladdin Lamps 05 Combinatorial and algebraic tools for multigrid Yiannis Koutis Computer Science](https://reader035.vdocuments.net/reader035/viewer/2022062620/5519e3b15503467a178b487b/html5/thumbnails/34.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
our contributions - two level analysis
• Starting hypothesis: Let X be the subspace corresponding to eigenvalues smaller than . Let Y be the null space of Rproject = RTD1/2 Assume, <X,Y>2 · /
• Two level convergence : error reduced by • Proving the hypothesis ? Yes! Using (A,S)• Result holds for t=1 smoothing• Additional smoothings do not help
![Page 35: 05/11/2005 Carnegie Mellon School of Computer Science Aladdin Lamps 05 Combinatorial and algebraic tools for multigrid Yiannis Koutis Computer Science](https://reader035.vdocuments.net/reader035/viewer/2022062620/5519e3b15503467a178b487b/html5/thumbnails/35.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
our contributions - recursion
• There is a matrix M which characterizes the error reduction after one full multigrid cycle
• We need to upper bound its maximum eigenvalue as a function of the two-level eigenvalues
the maximum eigenvalue of M is upper bounded by the sum of the maximum
eigenvalues over all two-levels
![Page 36: 05/11/2005 Carnegie Mellon School of Computer Science Aladdin Lamps 05 Combinatorial and algebraic tools for multigrid Yiannis Koutis Computer Science](https://reader035.vdocuments.net/reader035/viewer/2022062620/5519e3b15503467a178b487b/html5/thumbnails/36.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
towards full convergence
• Goal: The error not reduced by smoothing must be reduced by the smaller grid
A different point of viewThe small grid does not reduce part
of the error. It rather changes its spectral profile.
![Page 37: 05/11/2005 Carnegie Mellon School of Computer Science Aladdin Lamps 05 Combinatorial and algebraic tools for multigrid Yiannis Koutis Computer Science](https://reader035.vdocuments.net/reader035/viewer/2022062620/5519e3b15503467a178b487b/html5/thumbnails/37.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
full convergence for regular d-dimensional toroidal
meshes• A simple change in the
implementation of the algorithm:
• where
• T2 has eigenvalues 1 and -1
• T2 xlow = xhigh
![Page 38: 05/11/2005 Carnegie Mellon School of Computer Science Aladdin Lamps 05 Combinatorial and algebraic tools for multigrid Yiannis Koutis Computer Science](https://reader035.vdocuments.net/reader035/viewer/2022062620/5519e3b15503467a178b487b/html5/thumbnails/38.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
full convergence for regular d-dimensional toroidal
meshes• With t=O(log log n) smoothings
• Using recursive analysis: max(M) · 1/2
• Both pre-smoothings and post-smoothings are needed
• Holds for perturbations of toroidal meshes
![Page 39: 05/11/2005 Carnegie Mellon School of Computer Science Aladdin Lamps 05 Combinatorial and algebraic tools for multigrid Yiannis Koutis Computer Science](https://reader035.vdocuments.net/reader035/viewer/2022062620/5519e3b15503467a178b487b/html5/thumbnails/39.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
Overview
• Quick definitions• Subgraph preconditioners• Support graph preconditioners• Algebraic expressions• Low frequency eigenvectors and good
partitionings• Multigrid introduction and current state • Multigrid – Our contributions
![Page 40: 05/11/2005 Carnegie Mellon School of Computer Science Aladdin Lamps 05 Combinatorial and algebraic tools for multigrid Yiannis Koutis Computer Science](https://reader035.vdocuments.net/reader035/viewer/2022062620/5519e3b15503467a178b487b/html5/thumbnails/40.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
Thanks!