![Page 1: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/1.jpg)
Approxima)ng Graphs and
Solving Systems of Linear Equa)ons
Daniel A. Spielman Yale University
BMS, May 13, 2011
![Page 2: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/2.jpg)
Outline
Complexity of solving linear equa)ons Matrix Inversion, Sparse LU, CG
Nearly-‐linear )me for Laplacian Linear Systems What Sparsifica)on Low-‐stretch spanning trees Ultra-‐sparsifica)on
The future
Ax = b
![Page 3: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/3.jpg)
Matrix Inversion:
Can invert an n-‐by-‐n matrix in )me O(n3)
x = A−1b
![Page 4: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/4.jpg)
Matrix Inversion:
Can invert an n-‐by-‐n matrix in )me
Strassen ‘69: Can actually do it in )me
Coppersmith-‐Winograd ‘90: in )me
O(n3)
O(n2.81)
O(n2.38)
x = A−1b
![Page 5: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/5.jpg)
Matrix Inversion ≈ Matrix Mul)plica)on
= n1 n2
n3 n2
n1
n3
*
Easy to do in )me O(n1n2n3)
If can do it faster, can invert matrices faster.
![Page 6: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/6.jpg)
Group Theore)c Approach [Cohn-‐Umans ’03]
Can do matrix mul)plica)on in algebra of a group G if G has three sets of elements
|S1| = n1 |S2| = n2 |S3| = n3
such that for ai, bi ∈ Si
a1b−11 a2b
−12 a3b
−13 = 1 aib
−1i = 1
Fast by discrete Fourier Transform if
i d3i < n1n2n3
(dimensions of irreducible representa)ons)
![Page 7: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/7.jpg)
Group Theore)c Approach [Cohn-‐Umans ’03]
If small enough, could invert in )me
i d
3i
n2+o(1)
Cohn-‐Kleinberg-‐Szegedy-‐Umans ’05 find groups that allow matrix inversion in )me O(n2.41)
![Page 8: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/8.jpg)
Group Theore)c Approach [Cohn-‐Umans ’03]
If small enough, could invert in )me
i d
3i
n2+o(1)
Can we do it in )me ? n2+o(1)
Cohn-‐Kleinberg-‐Szegedy-‐Umans ’05 find groups that allow matrix inversion in )me O(n2.41)
![Page 9: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/9.jpg)
LU-‐Factoriza)on (Gaussian Elimina)on)
Write A = L U, lower-‐ and upper-‐triangular
1 -‐1 0 0 0 -‐1 2 -‐1 0 0 0 -‐1 2 -‐1 0 0 0 -‐1 2 -‐1 0 0 0 -‐1 2
1 0 0 0 0 -‐1 1 0 0 0 0 -‐1 1 0 0 0 0 -‐1 1 0 0 0 0 -‐1 1
1 -‐1 0 0 0 0 1 -‐1 0 0 0 0 1 -‐1 0 0 0 0 1 -‐1 0 0 0 0 1
=
Can be very fast if L and U have few non-‐zeros.
The inverse usually has non-‐zeros, L and U can have non-‐zero entries.
Ω(n2)O(n)
![Page 10: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/10.jpg)
Conjugate Gradient for Sparse Systems
Let A have m non-‐zero entries.
Conjugate Gradient (as a direct method) solves
in )me
O(mn)
If m is close to n, is much becer than inversion!
[Hestenes ‘51, S)efel ’52]
Ax = b
![Page 11: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/11.jpg)
Laplacian Linear Systems
Solve in )me where = number of non-‐zeros entries of A
)mes for -‐approximate solu)on.
Enables solu)on of all symmetric, diagonally-‐dominant systems.
O(m logc m)m
log(1/) x−A−1b
A≤
A−1bA
![Page 12: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/12.jpg)
Laplacian Quadra)c Form of
For x : V → IR
xTLGx =
(u,v)∈E
(x (u)− x (v))2
G = (V,E)
![Page 13: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/13.jpg)
Laplacian Quadra)c Form of
For x : V → IR
−1−3 01
3
x :
xTLGx =
(u,v)∈E
(x (u)− x (v))2
G = (V,E)
![Page 14: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/14.jpg)
Laplacian Quadra)c Form of
For x : V → IR
−1−3 0x :
xTLGx = 15
22 1212
32
xTLGx =
(u,v)∈E
(x (u)− x (v))2
1
3
G = (V,E)
![Page 15: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/15.jpg)
Laplacian Quadra)c Form for Weighted Graphs
xTLGx =
(u,v)∈E
w(u,v) (x (u)− x (v))2
G = (V,E,w)
w : E → IR+ assigns a posi)ve weight to every edge
Matrix LG is posi)ve semi-‐definite nullspace spanned by const vector, if connected
![Page 16: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/16.jpg)
Laplacian Matrix of a Weighted Graph
LG(u, v) =
−w(u, v) if (u, v) ∈ E
d(u) if u = v
0 otherwise
4 -1 0 -1 -2! -1 4 -3 0 0! 0 -3 4 -1 0! -1 0 -1 2 0! -2 0 0 0 2!
1 2
3 4
5 1!
1!
2!
1!
3!
d(u) =
(v,u)∈E w(u, v)
the weighted degree of u
is a diagonally dominant matrix
![Page 17: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/17.jpg)
Networks of Resistors [Kirchhoff]
Ohm’s laws gives
In general, with w(u,v) = 1/r(u,v)
Minimize dissipated energy
i = v/r
i = LGv
vTLGv
1V
0V
![Page 18: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/18.jpg)
1V
0V
0.5V
0.5V
0.625V 0.375V
By solving Laplacian
1V
0V
Networks of Resistors
Ohm’s laws gives
In general, with w(u,v) = 1/r(u,v)
Minimize dissipated energy
i = v/r
i = LGv
vTLGv
![Page 19: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/19.jpg)
Learning on Graphs [Zhu-‐Ghahramani-‐Lafferty ’03]
Infer values of a func)on at all ver)ces from known values at a few ver)ces.
Minimize xTLGx =
(u,v)∈E
w(u,v) (x (u)− x (v))2
Subject to known values
0
1
![Page 20: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/20.jpg)
0
1 0.5
0.5
0.625 0.375
By solving Laplacian
Infer values of a func)on at all ver)ces from known values at a few ver)ces.
Minimize xTLGx =
(u,v)∈E
w(u,v) (x (u)− x (v))2
Subject to known values
Learning on Graphs [Zhu-‐Ghahramani-‐Lafferty ’03]
![Page 21: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/21.jpg)
Other Applica)ons
Solving Ellip)c PDEs.
Solving Maximum Flow Problems.
Compu)ng Eigenvectors and Eigenvalues of Laplacians of graphs.
![Page 22: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/22.jpg)
Solving Laplacian Linear Equa)ons Quickly
Fast when graph is simple, by elimina)on.
Fast approxima)on when graph is complicated*, by Conjugate Gradient
* = random graph or expander
![Page 23: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/23.jpg)
Cholesky Factoriza)on of Laplacians
Also known as Y-‐Δ
When eliminate a vertex, connect its neighbors.
3 -1 0 -1 -1! -1 2 -1 0 0! 0 -1 2 -1 0! -1 0 -1 2 0! -1 0 0 0 1!
1 2
3 4
5 1!
1!
1!
1!
1!
![Page 24: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/24.jpg)
Cholesky Factoriza)on of Laplacians
Also known as Y-‐Δ
When eliminate a vertex, connect its neighbors.
3 -1 0 -1 -1! -1 2 -1 0 0! 0 -1 2 -1 0! -1 0 -1 2 0! -1 0 0 0 1!
1 2
3 4
5 1!
1!
1!
1!
1!.33!
.33!
.33!
3 0 0 0 0! 0 1.67 -1.00 -0.33 -0.33! 0 -1.00 2.00 -1.00 0! 0 -0.33 -1.00 1.67 -0.33! 0 -0.33 0 -0.33 0.67!
![Page 25: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/25.jpg)
Cholesky Factoriza)on of Laplacians
Also known as Y-‐Δ
When eliminate a vertex, connect its neighbors.
3 -1 0 -1 -1! -1 2 -1 0 0! 0 -1 2 -1 0! -1 0 -1 2 0! -1 0 0 0 1!
1 2
3 4
5
1!
1!.33!
.33!
.33!
3 0 0 0 0! 0 1.67 -1.00 -0.33 -0.33! 0 -1.00 2.00 -1.00 0! 0 -0.33 -1.00 1.67 -0.33! 0 -0.33 0 -0.33 0.67!
![Page 26: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/26.jpg)
Complexity of Cholesky Factoriza)on
#ops ~ Σv (degree of v when eliminate)2
Tree #ops ~ O(|V|)
![Page 27: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/27.jpg)
Complexity of Cholesky Factoriza)on
#ops ~ Σv (degree of v when eliminate)2
#ops ~ O(|V|) Tree
![Page 28: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/28.jpg)
Complexity of Cholesky Factoriza)on
#ops ~ Σv (degree of v when eliminate)2
Tree #ops ~ O(|V|)
![Page 29: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/29.jpg)
Complexity of Cholesky Factoriza)on
#ops ~ Σv (degree of v when eliminate)2
Tree #ops ~ O(|V|)
![Page 30: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/30.jpg)
Complexity of Cholesky Factoriza)on
#ops ~ Σv (degree of v when eliminate)2
Tree #ops ~ O(|V|)
![Page 31: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/31.jpg)
Complexity of Cholesky Factoriza)on
#ops ~ Σv (degree of v when eliminate)2
Tree #ops ~ O(|V|)
![Page 32: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/32.jpg)
Complexity of Cholesky Factoriza)on
#ops ~ Σv (degree of v when eliminate)2
Tree #ops ~ O(|V|)
Planar #ops ~ O(|V|3/2) Lipton-‐Rose-‐Tarjan ‘79
![Page 33: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/33.jpg)
Complexity of Cholesky Factoriza)on
#ops ~ Σv (degree of v when eliminate)2
Tree #ops ~ O(|V|)
Planar #ops ~ O(|V|3/2) Lipton-‐Rose-‐Tarjan ‘79
Expander like random, but O(|V|) edges
#ops ≳ Ω(|V|3) Lipton-‐Rose-‐Tarjan ‘79
![Page 34: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/34.jpg)
For S ⊂ V
ΦG = minS⊂V Φ(S)
Φ(S) = # edges leaving Ssum degrees on smaller side, S or V − S
S
Conductance and Cholesky Factoriza)on
![Page 35: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/35.jpg)
Conductance and Cholesky Factoriza)on
Cholesky slow when conductance high Cholesky fast when low for G and all subgraphs
For S ⊂ V
ΦG = minS⊂V Φ(S)
Φ(S) = # edges leaving Ssum degrees on smaller side, S or V − S
S
![Page 36: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/36.jpg)
Conductance
S
![Page 37: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/37.jpg)
Conductance
Φ(S) = 3/5
S
![Page 38: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/38.jpg)
Conductance
S
![Page 39: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/39.jpg)
Cheeger’s Inequality and the Conjugate Gradient
Cheeger’s inequality (degree-‐d unweighted case)
12λ2d ≤ ΦG ≤
2λ2
d
= second-‐smallest eigenvalue of LG ~ d/mixing )me of random walk
λ2
near d for expanders and random graphs
![Page 40: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/40.jpg)
Cheeger’s Inequality and the Conjugate Gradient
Cheeger’s inequality (degree-‐d unweighted case)
12λ2d ≤ ΦG ≤
2λ2
d
= second-‐smallest eigenvalue of LG ~ d/mixing )me of random walk
λ2
Conjugate Gradient finds ∊ -‐approx solu)on to LG x = b
in mults by LG O(d/λ2 log −1)
is ops O(mΦ−1G log −1)
![Page 41: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/41.jpg)
Fast solu)on of linear equa)ons
Conjugate Gradient fast when conductance high.
Elimina)on fast when low for G and all subgraphs.
![Page 42: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/42.jpg)
Fast solu)on of linear equa)ons
Elimina)on fast when low for G and all subgraphs.
Planar graphs
Want speed of extremes in the middle
Conjugate Gradient fast when conductance high.
![Page 43: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/43.jpg)
Fast solu)on of linear equa)ons
Elimina)on fast when low for G and all subgraphs.
Planar graphs
Want speed of extremes in the middle
Not all graphs fit into these categories!
Conjugate Gradient fast when conductance high.
![Page 44: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/44.jpg)
Precondi)oned Conjugate Gradient
Solve LG x = b by
Approxima)ng LG by LH (the precondi)oner)
In each itera)on solve a system in LH mul)ply a vector by LG
∊ -‐approximate solu)on aver
O(κ(LG, LH) log −1) itera)ons
condi6on number/approx quality
![Page 45: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/45.jpg)
Inequali)es and Approxima)on
if is positive semi-definite, !i.e. for all x,
xTLHx xTLGx
LH LG LG − LH
Example: if H is a subgraph of G
xTLGx =
(u,v)∈E
w(u,v) (x (u)− x (v))2
![Page 46: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/46.jpg)
Inequali)es and Approxima)on
if is positive semi-definite, !i.e. for all x,
κ(LG, LH) ≤ t
LH LG tLH
LH LG LG − LH
if
iff for some c cLH LG ctLH
xTLHx xTLGx
![Page 47: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/47.jpg)
Inequali)es and Approxima)on
if is positive semi-definite, !i.e. for all x,
κ(LG, LH) ≤ t
LH LG tLH
LH LG LG − LH
if
iff for some c cLH LG ctLH
Call H a t-‐approx of G if κ(LG, LH) ≤ t
xTLHx xTLGx
![Page 48: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/48.jpg)
Other defini)ons of rela)ve condi)on number
pseudo-inverse
min non-zero eigenvalue
κ(LG, LH) =λmax(LGL
+H)
λmin(LGL+H)
![Page 49: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/49.jpg)
Spectral Sparsifica)on of Graphs [S-‐Teng]
For every graph G with ver)ces there is a sparse graph H such that
κ(LG, LH) ≤ 1 +
n
![Page 50: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/50.jpg)
Spectral Sparsifica)on of Graphs [S-‐Teng]
κ(LG, LH) ≤ 1 +
Sparse: exists H with edges 4n/2
Can find H with edges in nearly-‐linear )me.
O(n log n/2)
For every graph G with ver)ces there is a sparse graph H such that
n
[Batson-‐S-‐Srivastava]
[S-‐Srivastava]
![Page 51: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/51.jpg)
Sparsifica)on by Random Sampling [S-‐Srivastava]
Include edge with probability (u, v)
Reff(u, v) = effec)ve resistance between u and v = 1/(current flow at one volt)
0V
0.53V
0.27V
0.33V 0.2V
1V
0V
u
v
pu,v ∼ wu,vReff(u, v)
![Page 52: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/52.jpg)
Sparsifica)on by Random Sampling [S-‐Srivastava]
Include edge with probability (u, v)
pu,v ∼ wu,vReff(u, v)
If include edge, give weight wu,v/pu,v
Can do all this in )me O(n log3 n)
![Page 53: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/53.jpg)
Spectral Sparsifica)on of Graphs [S-‐Teng]
For every graph G with n ver)ces there is a sparse graph H such that
κ(LG, LH) ≤ 1 +
Can solve in )me LGx = b
O(n2 log n log −1)
Using CG as direct solver for LH
![Page 54: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/54.jpg)
Vaidya’s Subgraph Precondi)oners
Precondi)on G by a subgraph H
LH LG so just need t for which LG tLH
Easy to bound t if H is a spanning tree
And, easy to solve equa)ons in LH by elimina)on
H
![Page 55: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/55.jpg)
The Stretch of Spanning Trees
Where
Boman-‐Hendrickson ‘01:
stT (G) =
(u,v)∈E
path-lengthT (u, v)
LG stG(T )LT
![Page 56: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/56.jpg)
The Stretch of Spanning Trees
Where
Boman-‐Hendrickson ‘01:
stT (G) =
(u,v)∈E
path-lengthT (u, v)
LG stG(T )LT
![Page 57: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/57.jpg)
The Stretch of Spanning Trees
path-‐len 3
Where
Boman-‐Hendrickson ‘01:
stT (G) =
(u,v)∈E
path-lengthT (u, v)
LG stG(T )LT
![Page 58: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/58.jpg)
The Stretch of Spanning Trees
path-‐len 5
Where
Boman-‐Hendrickson ‘01:
stT (G) =
(u,v)∈E
path-lengthT (u, v)
LG stG(T )LT
![Page 59: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/59.jpg)
The Stretch of Spanning Trees
path-‐len 1
Where
Boman-‐Hendrickson ‘01:
stT (G) =
(u,v)∈E
path-lengthT (u, v)
LG stG(T )LT
![Page 60: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/60.jpg)
The Stretch of Spanning Trees
In weighted case, measure resistances of paths
Where
Boman-‐Hendrickson ‘01:
stT (G) =
(u,v)∈E
path-lengthT (u, v)
LG stG(T )LT
![Page 61: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/61.jpg)
61
Fundamental Graphic Inequality
1!8!
1!2! 3!
8! 7!
4!5!
6!
edge k times path of length k
With weights, corresponds to resistors in serial!(Poincaré inequality)!
1!2! 3!
8! 7!
4!5!
6!
2! 3!
7!
4!5!
6!
![Page 62: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/62.jpg)
62
When T is a Spanning Tree
G T
Every edge of G not in T has unique path in T
![Page 63: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/63.jpg)
63
When T is a Spanning Tree
![Page 64: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/64.jpg)
Low-‐Stretch Spanning Trees
(Alon-‐Karp-‐Peleg-‐West ’91)
(Elkin-‐Emek-‐S-‐Teng ’04, Abraham-‐Bartal-‐Neiman ’08)
For every G there is a T with
where m = |E|
Solve linear systems in )me O(m3/2 logm)
stT (G) ≤ m1+o(1)
stT (G) ≤ O(m logm log2 logm)
![Page 65: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/65.jpg)
Low-‐Stretch Spanning Trees
(Alon-‐Karp-‐Peleg-‐West ’91)
(Elkin-‐Emek-‐S-‐Teng ’04, Abraham-‐Bartal-‐Neiman ’08)
For every G there is a T with
where m = |E|
Solve linear systems in )me O(m3/2 logm)
stT (G) ≤ m1+o(1)
stT (G) ≤ O(m logm log2 logm)
With sparsifica)on, O(m+ n3/2 log n)
![Page 66: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/66.jpg)
Ultra-‐Sparsifiers [S-‐Teng]
Approximate G by a tree plus edges
Sparsifiers Low-‐Stretch Trees
n/ log2 n
LH LG c log2 n LH
![Page 67: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/67.jpg)
Ultra-‐Sparsifiers
Solve systems in H by: 1. Cholesky elimina)ng degree 1 and 2 nodes
2. recursively solving reduced system
Time
O(m logc m)
![Page 68: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/68.jpg)
Kou)s-‐Miller-‐Peng ‘11
Solve in )me O(m log n log2 log n log(1/))
Build Ultra-‐Sparsifier by: 1. Construc)ng low-‐stretch spanning tree 2. Adding other edges with probability
pu,v ∼ path-lengthT (u, v)
![Page 69: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/69.jpg)
Conclusions
Laplacian Solvers are a powerful primi)ve! Faster Maxflow: Chris)ano-‐Kelner-‐Madry-‐S-‐Teng
Faster Random Spanning Trees: Kelner-‐Madry-‐Propp
All Effec)ve Resistances: S-‐Srivastava
Can we solve all well-‐condi)oned graph problems in nearly-‐linear )me?
Don’t fear large constants
![Page 70: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/70.jpg)
Open Problems
Faster and becer Low-‐Stretch Spanning Trees.
Faster high-‐quality sparsifica)on.
Other families of linear systems. From op)miza)on, machine learning, etc.
![Page 71: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/71.jpg)
Given vertex of interest find nearby cluster S with small conductance in )me O(|S|)
Local Graph Clustering [S-‐Teng ‘04]
![Page 72: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/72.jpg)
Prove: if S has small conductance u is a random node in S probably find a set of small conductance, in )me
Local Graph Clustering [S-‐Teng ‘04]
u S
![Page 73: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/73.jpg)
Using Approximate Personal PageRank Vectors
Jeh-‐Widom ‘03, Berkhin ‘06, Andersen-‐Chung-‐Lang ’06
Spilling paint in a graph: start at one node at each step, α frac)on dries of wet paint, half stays put, half to neighbors
![Page 74: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/74.jpg)
Using Approximate Personal PageRank Vectors
Jeh-‐Widom ‘03, Berkhin ‘06, Andersen-‐Chung-‐Lang ’06
Spilling paint in a graph: start at one node at each step, α frac)on dries of wet paint, half stays put, half to neighbors
1 0 0
![Page 75: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/75.jpg)
Using Approximate Personal PageRank Vectors
Jeh-‐Widom ‘03, Berkhin ‘06, Andersen-‐Chung-‐Lang ’06
Spilling paint in a graph: start at one node at each step, α frac)on dries of wet paint, half stays put, half to neighbors
1/2 0 0
1/2
with α = 1/2
![Page 76: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/76.jpg)
Using Approximate Personal PageRank Vectors
Jeh-‐Widom ‘03, Berkhin ‘06, Andersen-‐Chung-‐Lang ’06
Spilling paint in a graph: start at one node at each step, α frac)on dries of wet paint, half stays put, half to neighbors
1/4 1/4 0
1/2
with α = 1/2
![Page 77: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/77.jpg)
Using Approximate Personal PageRank Vectors
Jeh-‐Widom ‘03, Berkhin ‘06, Andersen-‐Chung-‐Lang ’06
Spilling paint in a graph: start at one node at each step, α frac)on dries of wet paint, half stays put, half to neighbors
1/8 1/8 0
5/8 1/8
with α = 1/2
![Page 78: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/78.jpg)
Using Approximate Personal PageRank Vectors
Jeh-‐Widom ‘03, Berkhin ‘06, Andersen-‐Chung-‐Lang ’06
Spilling paint in a graph: start at one node at each step, α frac)on dries of wet paint, half stays put, half to neighbors
3/32 1/8 1/32
5/8 1/8
with α = 1/2
![Page 79: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/79.jpg)
Using Approximate Personal PageRank Vectors
Jeh-‐Widom ‘03, Berkhin ‘06, Andersen-‐Chung-‐Lang ’06
Spilling paint in a graph: start at one node at each step, α frac)on dries of wet paint, half stays put, half to neighbors
Time doesn’t macer, can push paint whenever
Approximate: only push when a lot of paint
![Page 80: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/80.jpg)
Using Approximate Personal PageRank Vectors
Jeh-‐Widom ‘03, Berkhin ‘06, Andersen-‐Chung-‐Lang ’06
Spilling paint in a graph: start at one node at each step, α frac)on dries of wet paint, half stays put, half to neighbors
Time doesn’t macer, can push paint whenever
Approximate: only push when a lot of paint
1 0 0
![Page 81: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/81.jpg)
Spilling paint in a graph: start at one node at each step, α frac)on dries of wet paint, half stays put, half to neighbors
1/2
Using Approximate Personal PageRank Vectors
Jeh-‐Widom ‘03, Berkhin ‘06, Andersen-‐Chung-‐Lang ’06
Time doesn’t macer, can push paint whenever
Approximate: only push when a lot of paint
1/4 1/4 0
![Page 82: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/82.jpg)
Spilling paint in a graph: start at one node at each step, α frac)on dries of wet paint, half stays put, half to neighbors
Using Approximate Personal PageRank Vectors
Jeh-‐Widom ‘03, Berkhin ‘06, Andersen-‐Chung-‐Lang ’06
Time doesn’t macer, can push paint whenever
Approximate: only push when a lot of paint
1/4 1/4 0
1/2
![Page 83: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/83.jpg)
Spilling paint in a graph: start at one node at each step, α frac)on dries of wet paint, half stays put, half to neighbors
1/8
Using Approximate Personal PageRank Vectors
Jeh-‐Widom ‘03, Berkhin ‘06, Andersen-‐Chung-‐Lang ’06
Time doesn’t macer, can push paint whenever
Approximate: only push when a lot of paint
5/16 1/16 1/16
1/2
![Page 84: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/84.jpg)
Volume-‐Biased Evolving Set Markov Chain
[Andersen-‐Peres ‘09]
Walk on sets of ver)ces starts at one vertex, ends at V
Dual to random walk on graph
When start inside set of conductance find set of conductance φ1/2 log1/2 n
with work |S| logc n/φ1/2
![Page 85: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/85.jpg)
Volume-‐Biased Evolving Set Markov Chain
[Andersen-‐Peres ‘09]
Walk on sets of ver)ces starts at one vertex, ends at V
Dual to random walk on graph
When start inside set of conductance find set of conductance φ1/2 log1/2 n
with work |S| logc n/φ1/2
can we eliminate this?
![Page 86: ApproximangGraphs and SolvingSystemsofLinearEquaons · 2013-09-02 · LUFactorizaon(GaussianEliminaon) WriteA=LU,loweranduppertriangular 11000 12100 01210 00121 00012 10000 11000](https://reader033.vdocuments.net/reader033/viewer/2022042117/5e9523039d3cce37710ac2ba/html5/thumbnails/86.jpg)
Open Problems
Faster and becer Low-‐Stretch Spanning Trees.
Faster high-‐quality sparsifica)on.
Other families of linear systems.
Faster and becer local clustering.