graph partitioning
DESCRIPTION
Graph Partitioning. Donald Nguyen October 24 , 2011. Overview. Reminder: 1D and 2D partitioning for dense MVM Parallel sparse MVM as a graph algorithm Partitioning sparse MVM as a graph problem Metis approach to graph partitioning. Dense MVM. Matrix-Vector Multiply. x. y. A. =. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Graph Partitioning](https://reader035.vdocuments.net/reader035/viewer/2022062310/5681663d550346895dd9a8af/html5/thumbnails/1.jpg)
Graph Partitioning
Donald NguyenOctober 24, 2011
![Page 2: Graph Partitioning](https://reader035.vdocuments.net/reader035/viewer/2022062310/5681663d550346895dd9a8af/html5/thumbnails/2.jpg)
2
Overview
• Reminder: 1D and 2D partitioning for dense MVM
• Parallel sparse MVM as a graph algorithm• Partitioning sparse MVM as a graph problem• Metis approach to graph partitioning
![Page 3: Graph Partitioning](https://reader035.vdocuments.net/reader035/viewer/2022062310/5681663d550346895dd9a8af/html5/thumbnails/3.jpg)
3
Dense MVM
• Matrix-Vector Multiply
y xA=
![Page 4: Graph Partitioning](https://reader035.vdocuments.net/reader035/viewer/2022062310/5681663d550346895dd9a8af/html5/thumbnails/4.jpg)
4
1D Partitioning
=
![Page 5: Graph Partitioning](https://reader035.vdocuments.net/reader035/viewer/2022062310/5681663d550346895dd9a8af/html5/thumbnails/5.jpg)
5
2D Partitioning
=
![Page 6: Graph Partitioning](https://reader035.vdocuments.net/reader035/viewer/2022062310/5681663d550346895dd9a8af/html5/thumbnails/6.jpg)
6
Summary
• 1D and 2D dense partitioning– 2D more scalable
• Reuse partitioning over iterative MVMs– y becomes x in next iteration– use AllReduce to distribute results
![Page 7: Graph Partitioning](https://reader035.vdocuments.net/reader035/viewer/2022062310/5681663d550346895dd9a8af/html5/thumbnails/7.jpg)
7
Sparse MVM
y xA=
00 00
0 0
i
j
i
jAij
yi
xj
xi
yj
• A is incidence matrix of graph• y and x are labels on nodes
![Page 8: Graph Partitioning](https://reader035.vdocuments.net/reader035/viewer/2022062310/5681663d550346895dd9a8af/html5/thumbnails/8.jpg)
8
Graph Partitioning for Sparse MVM
a b
c
e
d
f
• Assign nodes to partitions of equal size minimizing edges cut– AKA find graph edge separator
• Analogous to 1D partitioning– assign nodes to processors
![Page 9: Graph Partitioning](https://reader035.vdocuments.net/reader035/viewer/2022062310/5681663d550346895dd9a8af/html5/thumbnails/9.jpg)
9
Partitioning Strategies
• Spectral partitioning– compute eigenvector of Laplacian– random walk approximation
• LP relaxation• Multilevel (Metis, …)– By far, most common and fastest
![Page 10: Graph Partitioning](https://reader035.vdocuments.net/reader035/viewer/2022062310/5681663d550346895dd9a8af/html5/thumbnails/10.jpg)
10
Metis
• Multilevel– Use short range and
long range structure
• 3 major phases– coarsening– initial partitioning– refinement
G1
Gn
… …
… …
coarsening
refin
emen
t
initial partitioning
![Page 11: Graph Partitioning](https://reader035.vdocuments.net/reader035/viewer/2022062310/5681663d550346895dd9a8af/html5/thumbnails/11.jpg)
11
Coarsening• Find matching– related problems:
• maximum (weighted) matching (O(V1/2E))• minimum maximal matching (NP-hard), i.e., matching with
smallest #edges– polynomial 2-approximations
![Page 12: Graph Partitioning](https://reader035.vdocuments.net/reader035/viewer/2022062310/5681663d550346895dd9a8af/html5/thumbnails/12.jpg)
12
Coarsening
• Edge contract a b
c
*
c
![Page 13: Graph Partitioning](https://reader035.vdocuments.net/reader035/viewer/2022062310/5681663d550346895dd9a8af/html5/thumbnails/13.jpg)
13
Initial Partitioning
• Breadth-first traversal– select k random nodes
a
b
![Page 14: Graph Partitioning](https://reader035.vdocuments.net/reader035/viewer/2022062310/5681663d550346895dd9a8af/html5/thumbnails/14.jpg)
14
Initial Partitioning
• Kernighan-Lin– improve partitioning by greedy swaps
cd
Dc = Ec – Ic = 3 – 0 = 3
Dd = Ed – Id = 3 – 0 = 3
Benefit(swap(c, d)) = Dc + Dd – 2Acd = 3 + 3 – 2 = 4c
d
![Page 15: Graph Partitioning](https://reader035.vdocuments.net/reader035/viewer/2022062310/5681663d550346895dd9a8af/html5/thumbnails/15.jpg)
15
Refinement
• Random K-way refinement– Randomly pick boundary
node– Find new partition which
reduces graph cut and maintains balance
– Repeat until all boundary nodes have been visited
a
a
![Page 16: Graph Partitioning](https://reader035.vdocuments.net/reader035/viewer/2022062310/5681663d550346895dd9a8af/html5/thumbnails/16.jpg)
16
Parallelizing Multilevel Partitioning
• For iterative methods, partitioning can be reused and relative cost of partitioning is small
• In other cases, partitioning itself can be a scalability bottleneck– hand-parallelization: ParMetis– Metis is also an example of amorphous data-
parallelism
![Page 17: Graph Partitioning](https://reader035.vdocuments.net/reader035/viewer/2022062310/5681663d550346895dd9a8af/html5/thumbnails/17.jpg)
17
Operator Formulation• Algorithm
– repeated application of operator to graph• Active node
–node where computation is started• Activity
–application of operator to active node– can add/remove nodes from graph
• Neighborhood– set of nodes/edges read/written by activity– can be distinct from neighbors in graph
• Ordering on active nodes–Unordered, ordered
i1
i2
i3
i4
i5
: active node: neighborhood
Amorphous data-parallelism: parallel execution of activities, subject to neighborhood and ordering constraints
![Page 18: Graph Partitioning](https://reader035.vdocuments.net/reader035/viewer/2022062310/5681663d550346895dd9a8af/html5/thumbnails/18.jpg)
18
ADP in Metis
• Coarsening– matching– edge contraction
• Initial partitioning• Refinement
![Page 19: Graph Partitioning](https://reader035.vdocuments.net/reader035/viewer/2022062310/5681663d550346895dd9a8af/html5/thumbnails/19.jpg)
19
ADP in Metis
• Coarsening– matching– edge contraction
• Initial partitioning• Refinement
![Page 20: Graph Partitioning](https://reader035.vdocuments.net/reader035/viewer/2022062310/5681663d550346895dd9a8af/html5/thumbnails/20.jpg)
20
ADP in Metis
• Coarsening• Initial partitioning• Refinement
![Page 21: Graph Partitioning](https://reader035.vdocuments.net/reader035/viewer/2022062310/5681663d550346895dd9a8af/html5/thumbnails/21.jpg)
21
Parallelism Profile
t60k benchmark graph
![Page 22: Graph Partitioning](https://reader035.vdocuments.net/reader035/viewer/2022062310/5681663d550346895dd9a8af/html5/thumbnails/22.jpg)
22
Dataset
• Public available large sparse graphs from University of Florida Sparse Matrix Collection and DIMACS shortest path competition
![Page 23: Graph Partitioning](https://reader035.vdocuments.net/reader035/viewer/2022062310/5681663d550346895dd9a8af/html5/thumbnails/23.jpg)
23
Scalability
memchip(2.3s)
Freesca
le1(2.9s)
circu
it5M dc(3
.2s)
rajat31(4.0s)
kkt p
ower(10.4s)
cage1
5(23.7s)
patents(
51.7s)
cit-Patents(
58.7s)0
1
2
3
4
5
6
Best ParMetisBest GHMetis
Dataset (Metis time in seconds)
memchip(2.3s)
Freesca
le1(2.9s)
circu
it5M dc(3
.2s)
rajat31(4.0s)
kkt p
ower(10.4s)
cage1
5(23.7s)
patents(
51.7s)
cit-Patents(
58.7s)0
1
2
3
4
5
6
Best ParMetisBest GHMetis
![Page 24: Graph Partitioning](https://reader035.vdocuments.net/reader035/viewer/2022062310/5681663d550346895dd9a8af/html5/thumbnails/24.jpg)
24
Summary
• Graph partitioning arises in many applications– sparse MVM, …
• Multilevel partitioning is most common graph partitioning algorithm– 3 phases: coarsening, initial partitioning,
refinement
![Page 25: Graph Partitioning](https://reader035.vdocuments.net/reader035/viewer/2022062310/5681663d550346895dd9a8af/html5/thumbnails/25.jpg)
25