sparsification,of,graphs,and,matrices, · sparsification,of,graphs,and,matrices,, daniela....
TRANSCRIPT
![Page 1: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/1.jpg)
Sparsification of Graphs and Matrices
Daniel A. Spielman
Yale University
joint work with
Joshua Batson (MIT) Nikhil Srivastava (MSR) Shang-‐Hua Teng (USC)
HUJI, May 21, 2014
![Page 2: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/2.jpg)
Objective of Sparsification:
Approximate any (weighted) graph by a
sparse weighted graph.
![Page 3: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/3.jpg)
Objective of Sparsification:
Approximate any (weighted) graph by a
sparse weighted graph. Spanners -‐ Preserve Distances [Chew ’89] Cut-‐Sparsifiers – preserve wt of edges leaving every set S ⊆ V
[Benczur-‐Karger ’96]
S S
![Page 4: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/4.jpg)
Spectral Sparsification [S-‐Teng]
Approximate any (weighted) graph by a
sparse weighted graph.
Graph Laplacian X
(u,v)2E
wu,v(x(u)� x(v))2G = (V,E,w)
![Page 5: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/5.jpg)
Laplacian Quadratic Form, examples
All edge-‐weights are 1 x : 0
1
0
1
1
![Page 6: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/6.jpg)
Laplacian Quadratic Form, examples
All edge-‐weights are 1 x : 0
1
0
1
1
Sum of squares of differences across edges
x
TLGx =
X
(u,v)2E
(x(u)� x(v))2
=
![Page 7: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/7.jpg)
Laplacian Quadratic Form, examples
All edge-‐weights are 1 x : 0
1
0
1
1
Sum of squares of differences across edges
00
01
= 1
x
TLGx =
X
(u,v)2E
(x(u)� x(v))2
=
![Page 8: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/8.jpg)
Laplacian Quadratic Form, examples
When x is the characteristic vector of a set S, sum the weights of edges on the boundary of S
00
0
1
1
1
S
x
TLGx =
X
(u,v)2Eu2Sv 62S
wu,v
0
![Page 9: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/9.jpg)
Learning on Graphs (Zhu-‐Ghahramani-‐Lafferty ’03)
Infer values of a function at all vertices from known values at a few vertices.
Minimize
Subject to known values
X
(a,b)2E
(x(a)� x(b))2
0
1
![Page 10: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/10.jpg)
X
(a,b)2E
(x(a)� x(b))2
0
1 0.5
0.5
0.625 0.375
Infer values of a function at all vertices from known values at a few vertices.
Minimize
Subject to known values
Learning on Graphs (Zhu-‐Ghahramani-‐Lafferty ’03)
![Page 11: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/11.jpg)
View edges as resistors connecting vertices Apply voltages at some vertices. Measure induced voltages and current flow.
1V
0V
Graphs as Resistor Networks
![Page 12: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/12.jpg)
View edges as resistors connecting vertices Apply voltages at some vertices. Measure induced voltages and current flow.
1V
0V
0.5V
0.5V
0.625V 0.375V
Graphs as Resistor Networks
![Page 13: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/13.jpg)
View edges as resistors connecting vertices Apply voltages at some vertices. Measure induced voltages and current flow. Induced voltages minimize
X
(a,b)2E
(v(a)� v(b))2
Subject to fixed voltages (by battery)
Graphs as Resistor Networks
![Page 14: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/14.jpg)
Graphs as Resistor Networks
Effective Resistance between s and t = potential difference of unit flow
s t
s t
2 1 0
0
0.5
0.5
1
1
1.5 Re↵(s, t) = 1.5
Re↵(s, t) = 2
![Page 15: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/15.jpg)
Laplacian Matrices
x
TLGx =
X
(u,v)2E
wu,v(x(u)� x(v))2
L1,2 =
✓1 �1�1 1
◆
=
✓1�1
◆�1 �1
�
E.g.
LG =X
(u,v)2E
wu,vLu,v
![Page 16: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/16.jpg)
Laplacian Matrices
x
TLGx =
X
(u,v)2E
wu,v(x(u)� x(v))2
LG =X
(u,v)2E
wu,vLu,v
=X
(u,v)2E
wu,v(bu,vbTu,v)
where bu,v = �u � �vSum of outer products
![Page 17: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/17.jpg)
Laplacian Matrices
x
TLGx =
X
(u,v)2E
wu,v(x(u)� x(v))2
LG =X
(u,v)2E
wu,vLu,v
=X
(u,v)2E
wu,v(bu,vbTu,v)
Positive semidefinite
If connected, nullspace = Span(1)
![Page 18: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/18.jpg)
Inequalities on Graphs and Matrices
For graphs and G = (V,E,w) H = (V, F, z)
For matrices and fMM
M 4 fMx
TMx x
T fMx
if for all x
G 4 H if LG 4 LH
![Page 19: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/19.jpg)
Inequalities on Graphs and Matrices
For graphs and G = (V,E,w) H = (V, F, z)
For matrices and fMM
M 4 fMx
TMx x
T fMx
if for all x
G 4 H if LG 4 LH
if LG 4 kLHG 4 kH
![Page 20: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/20.jpg)
Approximations of Graphs and Matrices
For graphs and G = (V,E,w) H = (V, F, z)
M ⇡✏fM
1
1 + ✏M 4 fM 4 (1 + ✏)Mif
LG ⇡✏ LHG ⇡✏ H if
![Page 21: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/21.jpg)
Approximations of Graphs and Matrices
For graphs and G = (V,E,w) H = (V, F, z)
M ⇡✏fM
1
1 + ✏M 4 fM 4 (1 + ✏)Mif
LG ⇡✏ LHG ⇡✏ H if
That is, for all
1
1 + �
P(u,v)2F zu,v(x(u)� x(v))2
P(u,v)2E wu,v(x(u)� x(v))2
1 + �
![Page 22: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/22.jpg)
Implications of Approximation
LH and LG have similar eigenvalues
Solutions to systems of linear equations are similar.
G ⇡✏ H
L+G ⇡✏ L
+H
Boundaries of sets are similar.
Effective resistances are similar.
![Page 23: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/23.jpg)
Spectral Sparsification [S-‐Teng]
so that
For an input graph G with n vertices, find a sparse graph H having edges O(n)
G ⇡✏ H
![Page 24: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/24.jpg)
Why? Solving linear equations in Laplacian Matrices
key part of nearly-‐linear time algorithm use for learning on graphs, maxflow, PDEs, …
Preserve Eigenvectors, Eigenvalues and electrical properties Generalize Expanders Certifiable cut-‐sparsifiers
![Page 25: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/25.jpg)
Approximations of Complete Graphs are Expanders
Expanders: d-‐regular graphs on n vertices (n grows, d fixed) every set of vertices has large boundary random walks mix quickly incredibly useful
![Page 26: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/26.jpg)
Approximations of Complete Graphs are Expanders
Expanders: d-‐regular graphs on n vertices (n grows, d fixed) weak expanders: eigenvalues bounded from 0 strong expanders: all eigenvalues near d
![Page 27: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/27.jpg)
Example: Approximating a Complete Graph
For G the complete graph on n verts. all non-‐zero eigenvalues of LG are n.
For , x ? 1 x
TLGx = n
kxk = 1
![Page 28: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/28.jpg)
Example: Approximating a Complete Graph
For G the complete graph on n verts. all non-‐zero eigenvalues of LG are n.
For , x ? 1 x
TLGx = n
For H a d-‐regular strong expander, all non-‐zero eigenvalues of LH are close to d.
For , x ? 1 x
TLHx ⇠ d
kxk = 1
kxk = 1
![Page 29: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/29.jpg)
Example: Approximating a Complete Graph
For G the complete graph on n verts. all non-‐zero eigenvalues of LG are n.
For , x ? 1 x
TLGx = n
For H a d-‐regular strong expander, all non-‐zero eigenvalues of LH are close to d.
For , x ? 1 x
TLHx ⇠ d
kxk = 1
kxk = 1
n
dH is a good approximation of G
![Page 30: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/30.jpg)
Best Approximations of Complete Graphs
Ramanujan Expanders [Margulis, Lubotzky-‐Phillips-‐Sarnak]
d� 2pd� 1 �(LH) d+ 2
pd� 1
![Page 31: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/31.jpg)
Best Approximations of Complete Graphs
Ramanujan Expanders [Margulis, Lubotzky-‐Phillips-‐Sarnak]
d� 2pd� 1 �(LH) d+ 2
pd� 1
Cannot do better if n grows while d is fixed [Alon-‐Boppana]
![Page 32: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/32.jpg)
Best Approximations of Complete Graphs
Ramanujan Expanders [Margulis, Lubotzky-‐Phillips-‐Sarnak]
Can we approximate every graph this well?
d� 2pd� 1 �(LH) d+ 2
pd� 1
![Page 33: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/33.jpg)
33
Example: Dumbbell
Complete graph
on n vertices Complete graph
on n vertices
1
d-‐regular Ramanujan, times n/d
d-‐regular Ramanujan, times n/d
1
G
H
![Page 34: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/34.jpg)
34
Example: Dumbbell
Kn Kn
d-‐regular Ramanujan, times n/d
d-‐regular Ramanujan, times n/d
1
1
![Page 35: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/35.jpg)
35
Example: Dumbbell
Kn Kn
d-‐regular Ramanujan, times n/d
d-‐regular Ramanujan, times n/d
1
1
G1
G2 G3
H1
H2
H3
G = G1 +G2 +G3
H = H1 +H2 +H3
G1 4 (1 + �)H1
G2 = H2
G3 4 (1 + �)H3
G 4 (1 + �)H
![Page 36: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/36.jpg)
Cut-‐Sparsifiers [Benczur-‐Karger ‘96]
S S
For every G, is an H with edges
for all x � {0, 1}nO(n log n/�2)
1
1 + ✏
x
TLHx
x
TLGx
1 + ✏
![Page 37: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/37.jpg)
Cut-‐approximation is different
k
m� 1
![Page 38: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/38.jpg)
Cut-‐approximation is different
k
m� 1
0 1 2 m … 3
![Page 39: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/39.jpg)
Cut-‐approximation is different
k
m� 1
0 1 2 m … 3
x
TLGx = km+m
2
Need long edge if k < m
![Page 40: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/40.jpg)
Main Theorems for Graphs
For every , there is a s.t.
G = (V,E,w) H = (V, F, z)
|F | � |V | (2 + �)2/�2and G ⇡✏ H
![Page 41: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/41.jpg)
Main Theorems for Graphs
For every , there is a s.t.
G = (V,E,w) H = (V, F, z)
|F | � |V | (2 + �)2/�2and G ⇡✏ H
Within a factor of 2 of the Ramanujan bound
![Page 42: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/42.jpg)
Main Theorems for Graphs
For every , there is a s.t.
G = (V,E,w) H = (V, F, z)
|F | � |V | (2 + �)2/�2and G ⇡✏ H
By careful random sampling, get
(S-‐Srivastava 08)
O(|E| log2 |V | log(1/�))
(Koutis-‐Levin-‐Peng ‘12)
In time
![Page 43: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/43.jpg)
Sparsification by Random Sampling
Assign a probability pu,v to each edge (u,v) Include edge (u,v) in H with probability pu,v. If include edge (u,v), give it weight wu,v /pu,v
E [ LH ] =X
(u,v)2E
pu,v(wu,v/pu,v)Lu,v = LG
![Page 44: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/44.jpg)
Sparsification by Random Sampling
Choose pu,v to be wu,v times the effective resistance between u and v. Low resistance between u and v means there are many alternate routes for current to flow and that the edge is not critical. Proof by random matrix concentration bounds (Rudelson, Ahlswede-‐Winter, Tropp, etc.)
![Page 45: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/45.jpg)
Matrix Sparsification
� �=
� �✓ ◆M BTB
� �=
� �✓ ◆fM
1
(1 + ✏)M 4 fM 4 (1 + ✏)M
![Page 46: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/46.jpg)
Matrix Sparsification
� �=
� �✓ ◆M BTB
� �=
� �✓ ◆fM
1
(1 + ✏)M 4 fM 4 (1 + ✏)M
most se = 0
=P
e sebebTe
=P
e bebTe
![Page 47: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/47.jpg)
Main Theorem (Batson-‐S-‐Srivastava)
For , there exist so that for
M =P
e bebTe se
fM =P
e sebebTe
n(2 + ✏)2/✏2at most are non-‐zero se
M ⇡✏fM
and
![Page 48: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/48.jpg)
Simplification of Matrix Sparsification
1
(1 + ✏)M 4 fM 4 (1 + ✏)M
1
(1 + ✏)I 4 M�1/2fMM�1/2 4 (1 + ✏)I
is equivalent to
![Page 49: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/49.jpg)
Simplification of Matrix Sparsification
1
(1 + ✏)I 4 M�1/2fMM�1/2 4 (1 + ✏)I
ve = M�1/2beSet X
e
vevTe = I
“Decomposition of the identity”
X
e
hu, vei2 = kuk2
![Page 50: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/50.jpg)
Simplification of Matrix Sparsification
1
(1 + ✏)I 4 M�1/2fMM�1/2 4 (1 + ✏)I
ve = M�1/2beSet X
e
vevTe = I
We need X
e
sevevTe ⇡✏ I
![Page 51: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/51.jpg)
Simplification of Matrix Sparsification
1
(1 + ✏)I 4 M�1/2fMM�1/2 4 (1 + ✏)I
ve = M�1/2beSet X
e
vevTe = I
X
e
sevevTe ⇡✏ I
Random sampling sets
We need
pe = kvek2
![Page 52: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/52.jpg)
What happens when we add a vector?
![Page 53: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/53.jpg)
Interlacing
![Page 54: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/54.jpg)
More precisely Characteristic Polynomial:
![Page 55: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/55.jpg)
More precisely Characteristic Polynomial: Rank-‐one update:
pA+vvT =⇣1 +
Pi<v,ui>
2
�i�x
⌘pA
Where Aui = �iui
![Page 56: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/56.jpg)
More precisely Characteristic Polynomial: Rank-‐one update:
pA+vvT =⇣1 +
Pi<v,ui>
2
�i�x
⌘pA
are zeros of
![Page 57: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/57.jpg)
Physical model of interlacing λi = positive unit charges resting at barriers on a slope
![Page 58: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/58.jpg)
Physical model of interlacing
ui is eigenvector is added vector charge on barrier
v
hv, uii2
![Page 59: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/59.jpg)
Physical model of interlacing
Barriers repel eigs.
ui is eigenvector is added vector charge on barrier
v
hv, uii2
![Page 60: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/60.jpg)
Physical model of interlacing
Barriers repel eigs.
gravity
Inverse law repulsion
![Page 61: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/61.jpg)
Examples
![Page 62: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/62.jpg)
Ex1: All weight on u1
v = u1
![Page 63: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/63.jpg)
Ex1: All weight on u1
v = u1
![Page 64: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/64.jpg)
Ex1: All weight on u1
v = u1
![Page 65: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/65.jpg)
Ex1: All weight on u1
Pushed up against next barrier
Gravity keeps resting on barrier
v = u1
![Page 66: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/66.jpg)
Ex2: Equal weight on u1, u2
![Page 67: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/67.jpg)
Ex2: Equal weight on u1, u2
![Page 68: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/68.jpg)
Ex2: Equal weight on u1, u2
![Page 69: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/69.jpg)
Ex3: Equal weight on all u1, u2 , …, un
+1/n
+1/n
+1/n
![Page 70: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/70.jpg)
Ex3: Equal weight on all u1, u2 , …, un
+1/n
+1/n
+1/n
![Page 71: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/71.jpg)
Adding a random ve
Because are decomposition of identity, ve
Ee
⇥PA+vevT
e
⇤=
1 +
1
m
X
i
1
�i � x
!PA
= PA � 1
m
d
dxPA
Ee
hhve, uii2
i= 1/m
![Page 72: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/72.jpg)
Many random ve
Ee
⇥PA+vevT
e(x)
⇤= 1� 1
m
d
dx
PA(x)
Ee1,...,ek
hPve1v
Te1
+···+vekvTek(x)
i=
✓1� 1
m
d
dx
◆k
x
n
![Page 73: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/73.jpg)
Many random ve
Ee
⇥PA+vevT
e(x)
⇤= 1� 1
m
d
dx
PA(x)
Ee1,...,ek
hPve1v
Te1
+···+vekvTek(x)
i=
✓1� 1
m
d
dx
◆k
x
n
Is an associated Laguerre polynomial!
For , roots lie between and
k = n/✏2
(1� ✏)2n
m✏2(1 + ✏)2
n
m✏2
![Page 74: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/74.jpg)
Matrix Sparsification Proof Sketch
Have X
e
vevTe = I Want
X
e
sevevTe ⇡✏ I
✏ ⇡ 2.6
Will do with |{e : se 6= 0}| 6n
All eigenvalues between 1 and 13,
![Page 75: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/75.jpg)
Broad outline: moving barriers
0 n -‐n
![Page 76: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/76.jpg)
Step 1
0 n -‐n
![Page 77: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/77.jpg)
Step 1
0 n -‐n
0 n -‐n
![Page 78: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/78.jpg)
Step 1
0 n -‐n
0
+1/3 +2
-‐n+1/3 n+2
![Page 79: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/79.jpg)
Step 1
0 n -‐n
0
+1/3 +2
-‐n+1/3 n+2
tighter constraint
looser constraint
![Page 80: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/80.jpg)
Step i+1
0
![Page 81: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/81.jpg)
Step i+1
0
+1/3 +2
![Page 82: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/82.jpg)
Step i+1
0
![Page 83: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/83.jpg)
Step i+1
0
+1/3 +2
![Page 84: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/84.jpg)
Step i+1
0
![Page 85: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/85.jpg)
Step i+1
0
+1/3 +2
![Page 86: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/86.jpg)
Step i+1
0
![Page 87: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/87.jpg)
Step i+1
0
![Page 88: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/88.jpg)
Step i+1
0
![Page 89: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/89.jpg)
Step i+1
0
![Page 90: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/90.jpg)
Step 6n
0 … 13n n
![Page 91: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/91.jpg)
Step 6n
0 … n
2.6-approximation with 6n vectors.
13n
![Page 92: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/92.jpg)
Problem
need to show that an appropriate
always exists.
![Page 93: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/93.jpg)
Problem
need to show that an appropriate
always exists.
Is not strong enough for induction
![Page 94: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/94.jpg)
Problems
If many small eigenvalues, can only move one
Bunched large eigenvalues repel the highest one
![Page 95: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/95.jpg)
The Lower Barrier Potential Function
�⇥(A) =P
i1
�i�⇥ = Tr�(A� �I)�1
�
![Page 96: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/96.jpg)
The Lower Barrier Potential Function
�⇥(A) =P
i1
�i�⇥ = Tr�(A� �I)�1
�
��(A) 1 =) �min(A) � ⇥+ 1
![Page 97: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/97.jpg)
No λi within dist. 1 No 2 λi within dist. 2 No 3 λi within dist. 3
.
. No k λi within dist. k
The Lower Barrier Potential Function
�⇥(A) =P
i1
�i�⇥ = Tr�(A� �I)�1
�
��(A) 1 =) �min(A) � ⇥+ 1
![Page 98: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/98.jpg)
The Upper Barrier Potential Function
�u(A) =P
i1
u��i= Tr
�(uI �A)�1
�
![Page 99: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/99.jpg)
The Beginning
0 n-n
![Page 100: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/100.jpg)
The Beginning
0 n-n
![Page 101: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/101.jpg)
Step i+1
0
![Page 102: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/102.jpg)
Step i+1
0
+1/3 +2
Lemma. can always choose so that potentials do not increase
+sevevTe
![Page 103: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/103.jpg)
Step i+1
0
![Page 104: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/104.jpg)
Step i+1
0
![Page 105: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/105.jpg)
Step i+1
0
![Page 106: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/106.jpg)
Step i+1
0
![Page 107: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/107.jpg)
Step 6n
0 … 13n n
![Page 108: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/108.jpg)
Step 6n
0 … n
2.6-approximation with 6n vectors.
13n
![Page 109: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/109.jpg)
Goal
+1/3 +2
Lemma. can always choose so that potentials do not increase
+sevevTe
+sevevTe
![Page 110: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/110.jpg)
Upper Barrier Update
Add and set
+2
svvT
![Page 111: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/111.jpg)
Upper Barrier Update
Add and set
+2
�u0(A+ svvT )
= Tr�(u0I �A� svvT )�1
�
svvT
![Page 112: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/112.jpg)
Upper Barrier Update
Add and set
+2
svvT
�u0(A+ svvT )
= Tr�(u0I �A� svvT )�1
�
= �u0(A) +
svT (u0I �A)�2v
1� svT (u0I �A)�1v
By Sherman-‐Morrison Formula
![Page 113: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/113.jpg)
Upper Barrier Update
Add and set
+2
svvT
�u0(A+ svvT )
= Tr�(u0I �A� svvT )�1
�
= �u0(A) +
svT (u0I �A)�2v
1� svT (u0I �A)�1v
Need �u(A)
![Page 114: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/114.jpg)
How much of can we add?
Rearranging:
�u0(A+ svvT ) �u(A)
1 � svT
✓(u0I �A)�2
�u(A)� �u0(A)+ (u0I �A)�1
◆v
iff
vvT
![Page 115: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/115.jpg)
How much of can we add?
Rearranging:
�u0(A+ svvT ) �u(A)
1 � svT
✓(u0I �A)�2
�u(A)� �u0(A)+ (u0I �A)�1
◆v
iff
Write as 1 � svTUAv
vvT
![Page 116: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/116.jpg)
Lower Barrier
Similarly:
iff
Write as
�l0(A+ svvT ) �l(A)
1 svTLAv
1 svT
✓(A� l0I)�2
�l0(A)� �l(A)� (A� l0I)�1
◆v
![Page 117: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/117.jpg)
Goal
Show that we can always add some vector while respecting both barriers.
+1/3 +2
Need: svTUAv 1 svTLAv
+svvT
![Page 118: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/118.jpg)
Two expectations
Need:
Can show:
svTUAv 1 svTLAv
Ee
⇥vTe UAve
⇤ 3/2m
Ee
⇥vTe LAve
⇤� 2/m
![Page 119: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/119.jpg)
Two expectations
Need:
Can show:
svTUAv 1 svTLAv
So:
And, exists
Ee
⇥vTe UAve
⇤ 3/2m
Ee
⇥vTe LAve
⇤� 2/m
Ee
⇥vTe UAve
⇤ E
e
⇥vTe LAve
⇤
e : vTe UAve vTe LAve
![Page 120: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/120.jpg)
Two expectations
Need:
Can show:
svTUAv 1 svTLAv
So:
And, exists
And that puts between them s 1
Ee
⇥vTe UAve
⇤ 3/2m
Ee
⇥vTe LAve
⇤� 2/m
Ee
⇥vTe UAve
⇤ E
e
⇥vTe LAve
⇤
e : vTe UAve vTe LAve
![Page 121: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/121.jpg)
Two expectations
Need:
Can show:
svTUAv 1 svTLAv
So:
And, exists
And that puts between them s 1
Ee
⇥vTe UAve
⇤ 3/2m
Ee
⇥vTe LAve
⇤� 2/m
Ee
⇥vTe UAve
⇤ E
e
⇥vTe LAve
⇤
e : vTe UAve vTe LAve
![Page 122: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/122.jpg)
Bounding expectations
vTUAv = Tr�UAvv
T�
Ee
⇥Tr
�UAvev
Te
� ⇤= Tr
⇣UA E
e
⇥vev
Te
⇤⌘
= Tr (UAI)
= Tr (UA)
/m
/m
![Page 123: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/123.jpg)
Bounding expectations
Tr (UA) =Tr
�(u0I �A)�2
�
�u(A)� �u0(A)+ Tr
�(u0I �A)�1
�
= �u0(A)
�u(A)
1
As barrier function is monotone decreasing
![Page 124: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/124.jpg)
Bounding expectations
Tr (UA) =Tr
�(u0I �A)�2
�
�u(A)� �u0(A)+ Tr
�(u0I �A)�1
�
= �u0(A)
�u(A)
1
Numerator is derivative of barrier function. As barrier function is convex,
1
u0 � u
Tr(UA) 1/2 + 1 = 3/2
![Page 125: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/125.jpg)
Bounding expectations
Similarly, Tr (LA) �1
l0 � l� 1
=1
1/3� 1
= 2
![Page 126: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/126.jpg)
Step i+1
0
+1/3
Lemma. can always choose so that potentials do not increase
+2
![Page 127: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/127.jpg)
Twice-‐Ramanujan Sparsifiers
Fixing steps and tightening parameters gives ratio
Less than twice as many edges as used by Ramanujan Expander of same quality
�max
(A)
�min
(A) d+ 1 + 2
pd
d+ 1� 2pd
![Page 128: Sparsification,of,Graphs,and,Matrices, · Sparsification,of,Graphs,and,Matrices,, DanielA. Spielman, Yale,University,,,,, joint,work,with, JoshuaBatson,(MIT) NikhilSrivastava(MSR)](https://reader035.vdocuments.net/reader035/viewer/2022070821/5f1f56abe9df115aa0181f70/html5/thumbnails/128.jpg)
Open Questions
The Ramanujan bound Properties of vectors from graphs?
Faster algorithm union of random Hamiltonian cycles?