4.1.2 compressed sparse row format: csr
TRANSCRIPT
![Page 1: 4.1.2 Compressed Sparse Row Format: CSR](https://reader031.vdocuments.net/reader031/viewer/2022020701/61f6317bce54f11b8c51003a/html5/thumbnails/1.jpg)
5
4.1.2 Compressed Sparse Row
Format: CSR
AA 2 1 5 3 4 6 7 8 9 10 11 12
JA 4 1 4 1 2 1 3 4 5 3 4 5
Values
Column indices
row 1 row 2 row 3 row 4 row 5
IA: 1 3 6 10 12 13 pointer to row i
Storage:
n and nnz,
n+nnz+1 integer,
nnz float.
![Page 2: 4.1.2 Compressed Sparse Row Format: CSR](https://reader031.vdocuments.net/reader031/viewer/2022020701/61f6317bce54f11b8c51003a/html5/thumbnails/2.jpg)
6
AA 2 1 5 3 4 6 7 8 9 10 11 12
JA 4 1 4 1 2 1 3 4 5 3 4 5
Values
Column indices
row 1 row 2 row 3 row 4 row 5
IA: 1 3 6 10 12 13 pointer to row i
Code for computing c = A*b:
c = 0;for i = 1 : n
for j = IA(i) : IA(i+1)-1ci = ci + AA(j)*bJA(j) ;
endend
Indirect addressing only in b.
Columnwise compressed sparse column format
![Page 3: 4.1.2 Compressed Sparse Row Format: CSR](https://reader031.vdocuments.net/reader031/viewer/2022020701/61f6317bce54f11b8c51003a/html5/thumbnails/3.jpg)
7
CSR with extracted main diagonal
AA 1 4 7 11 12 * 2 3 5 6 8 9 10
JA 7 8 10 13 14 14 4 2 4 1 4 5 3
main diagonal entries | nondiagonal entries in CSR
Pointer to begin of i-th row:
Storage: n and nnz, nnz + 1 integer, nnz + 1 float
![Page 4: 4.1.2 Compressed Sparse Row Format: CSR](https://reader031.vdocuments.net/reader031/viewer/2022020701/61f6317bce54f11b8c51003a/html5/thumbnails/4.jpg)
8
AA 1 4 7 11 12 * 2 3 5 6 8 9 10
JA 7 8 10 13 14 14 4 2 4 1 4 5 3
main diagonal entries | nondiagonal entries in CSR
for i = 1 : n ci = AAi * bi ; for j = JA(i) : JA(i+1)-1 ci = ci + AAj*bJA(j) ; endend
Code for c=A*b
![Page 5: 4.1.2 Compressed Sparse Row Format: CSR](https://reader031.vdocuments.net/reader031/viewer/2022020701/61f6317bce54f11b8c51003a/html5/thumbnails/5.jpg)
9
4.1.4 Diagonalwise storage
=
121100000900807600504300201
A
Diagonal number -1 0 2
Values in: ( )201,
*1211*10987654321*
−=
= IOFFDIAG
Storage: n, nd = number of diagonals, nd integers for IOFF and n*nd float
![Page 6: 4.1.2 Compressed Sparse Row Format: CSR](https://reader031.vdocuments.net/reader031/viewer/2022020701/61f6317bce54f11b8c51003a/html5/thumbnails/6.jpg)
10
4.1.5 Rectangular Storage Scheme by Pressing from the Right
=
012110109876543021
1211000010900807600504300201
COEFgives
=
*54*43532421*31
JCOEF
Storage: n, n*nl integer and float
nl := nnz of longest row.
![Page 7: 4.1.2 Compressed Sparse Row Format: CSR](https://reader031.vdocuments.net/reader031/viewer/2022020701/61f6317bce54f11b8c51003a/html5/thumbnails/7.jpg)
11
Code for c = A b:
c = 0;for i = 1 : n for j = 1 : nl ci = ci + COEFF(i,j) * b(JCOEFF(i,j)); endend
This format was used in ELLPACK (package of subroutines for elliptic PDE).
Coordinate form is used by MATLAB.
![Page 8: 4.1.2 Compressed Sparse Row Format: CSR](https://reader031.vdocuments.net/reader031/viewer/2022020701/61f6317bce54f11b8c51003a/html5/thumbnails/8.jpg)
12
4.1.6 Jagged Diagonal FormPrestep: Sort rows after their length. Long rows first.
=⇒
=
1211000010900002018076005043
1211000010900807600504300201
PAA
Length 3
Length 2
Pointer to beginning of j-th diagonal: ( )131161=IDIAG
( )545433243121=JDIAGColumn indices:
( )851210274119163=DJValues of PA:
First jagged diagonal second jagged diag.
![Page 9: 4.1.2 Compressed Sparse Row Format: CSR](https://reader031.vdocuments.net/reader031/viewer/2022020701/61f6317bce54f11b8c51003a/html5/thumbnails/9.jpg)
13
NDIAG = number of jagged diagonals
Storage: n, NDIAG, nnz float, nnz + NDIAG integer
Code for c = A b:
c = 0;for j = 1 : NDIAG for i = 1 : IDIAG(j+1) - IDIAG(j) k = IDIAG(j) + i - 1; ci = ci + DJ(k) * b(JDIAG(k)); endend
Advantages: - Always start with row 1.- More operations on neighboring data.- Less indirect addressing.- Pre-permutation changes only rows. Can be done implicitly.
Length of j-th jagged diag.
![Page 10: 4.1.2 Compressed Sparse Row Format: CSR](https://reader031.vdocuments.net/reader031/viewer/2022020701/61f6317bce54f11b8c51003a/html5/thumbnails/10.jpg)
14
4.2 Sparse Matrices and Graphs4.2.1 Graph G(A) for symmetric positive definite spd A=AT >0
n x n –matrix: vertices e1, … , en with edges (ei,ek) for aik ≠ 0 , undirected Graph
=
**0****00****0**
A G(A): e1 e2 e3 e4
G(A) as directed graph: e1 e2 e3 e4
![Page 11: 4.1.2 Compressed Sparse Row Format: CSR](https://reader031.vdocuments.net/reader031/viewer/2022020701/61f6317bce54f11b8c51003a/html5/thumbnails/11.jpg)
15
Adjacency Matrix for G(A) or A:
=
1101111001111011
))(( AGA can be obtained directly by replacing in A each nonzero entry by 1.
Symmetric permutations of A in the form P A PT change the ordering of the rows and columns of A simultaneously.
Therefore, the graph of P A PT can be obtained by the graph of A by renumbering the vertices:
![Page 12: 4.1.2 Compressed Sparse Row Format: CSR](https://reader031.vdocuments.net/reader031/viewer/2022020701/61f6317bce54f11b8c51003a/html5/thumbnails/12.jpg)
16
Matrix A with graph G(A)
Symmetric permutation P A PT with graph G(P A PT)
G(PAPT): e1 e2 e4 e3
G(A): e1 e2 e3 e4
Example: P permutation that changes 3 4:
e1 e2 e3 e4
![Page 13: 4.1.2 Compressed Sparse Row Format: CSR](https://reader031.vdocuments.net/reader031/viewer/2022020701/61f6317bce54f11b8c51003a/html5/thumbnails/13.jpg)
17
4.2.2 Matrix A nonsymmetric, G(A) directed
=⇒
=
1001111001100011
))((
*00****00**000**
AGAA
G(A): e1 e2 e3 e4
How can we characterize „good“ sparsity patterns?
„good“: Gaussian Elimination can be reduced to smaller subproblems or produces no (or small) fill-in.
![Page 14: 4.1.2 Compressed Sparse Row Format: CSR](https://reader031.vdocuments.net/reader031/viewer/2022020701/61f6317bce54f11b8c51003a/html5/thumbnails/14.jpg)
18
Block Diagonal Pattern
=⇒
=
1010010110100101
))((
*0*00*0**0*00*0*
AGAA
G(A): e1 e2 e3 e4
=
=2
1
00
**00**0000**00**
AA
PAPTe1 e3 e2 e42 3:
By this permutation, A can be transformed into block diagonalform easy to solve!
=
−
−−
12
11
1
2
1
00
00
AA
AA
![Page 15: 4.1.2 Compressed Sparse Row Format: CSR](https://reader031.vdocuments.net/reader031/viewer/2022020701/61f6317bce54f11b8c51003a/html5/thumbnails/15.jpg)
19
Banded Pattern
=
+−
+−
nnqnn
npn
q
p
aa
a
a
aa
A
1,
,1
1
111
=
+− nnqnn
q
ll
l
l
L
1,
1
11
=+−
nn
npn
p
u
u
uu
U
,1
111
Gauss Elimination without pivoting preserves the sparsity pattern
With pivoting the bandwidth in U grows, but remains <= p+q.
![Page 16: 4.1.2 Compressed Sparse Row Format: CSR](https://reader031.vdocuments.net/reader031/viewer/2022020701/61f6317bce54f11b8c51003a/html5/thumbnails/16.jpg)
20
Overlapping Block Diagonal
=A
Pattern is preserved by Gaussian Elimination (in case of restricted pivoting).
![Page 17: 4.1.2 Compressed Sparse Row Format: CSR](https://reader031.vdocuments.net/reader031/viewer/2022020701/61f6317bce54f11b8c51003a/html5/thumbnails/17.jpg)
21
Dissection Form
0
0
0
0
0
0
0
0
0
0 0
00
0
0 0
Nested (recursive) dissection:
Pattern are preservedduring GE withoutpivoting. No fill in
![Page 18: 4.1.2 Compressed Sparse Row Format: CSR](https://reader031.vdocuments.net/reader031/viewer/2022020701/61f6317bce54f11b8c51003a/html5/thumbnails/18.jpg)
22
Schur Complement Reduction
=
++
=
⋅
−−
−
−
−
II
SBDBBBSBDBI
SDB
BBBB
*0
0
!
143
113
121
1
11
43
21
Write matrix B in terms of smaller submatrices:
To satisfy this equation we have to set:
12
112
1134
14
12
113
143
12
11
121 0
−−−
−−−−
−−−
−=−=⇒
+−=⇒=+
−=⇒=+
SBBDandBBBBS
SBSBBBIISBDB
SBBDSBDB
S Schur Complement
![Page 19: 4.1.2 Compressed Sparse Row Format: CSR](https://reader031.vdocuments.net/reader031/viewer/2022020701/61f6317bce54f11b8c51003a/html5/thumbnails/19.jpg)
23
⋅
=
= − S
BBIBB
IBBBB
B0
0 2111343
21
Therefore, solving linear system in B is reduced to solving two smaller linear systems, one in B1 and the other in the Schur complement S.
B sparse B1 also sparse, but S usually dense!
Example: Schur complement and dissection form:
=
321
22
11
00
AGGFAFA
ASchur complement:
( )
21221
1113
2
112
11
213 00
FAGFAGA
FF
AA
GGAS
−−
−
−
−−=
=
⋅
⋅−=
![Page 20: 4.1.2 Compressed Sparse Row Format: CSR](https://reader031.vdocuments.net/reader031/viewer/2022020701/61f6317bce54f11b8c51003a/html5/thumbnails/20.jpg)
24
Direct derivation of Schur complement:
3332211
23222
13111
3
2
1
3
2
1
321
22
11
00
bxAxGxGbxFxAbxFxA
bbb
xxx
AGGFAFA
=++=+=+
⇒
=
⋅
32122
122
311
111
11
xFAbAx
xFAbAx
−−
−−
−=
−=⇒
33
21221
111332
1221
1113
333321222
12231
1111
111
~
)(
)()(
bSx
bAGbAGbxFAGFAGA
bxAxFAGbAGxFAGbAG
=⇒
−−=−−⇒
=+−+−⇒
−−−−
−−−−
![Page 21: 4.1.2 Compressed Sparse Row Format: CSR](https://reader031.vdocuments.net/reader031/viewer/2022020701/61f6317bce54f11b8c51003a/html5/thumbnails/21.jpg)
25
Algorithm for solving Ax=b based on Schur complement:
1. Compute S by using inv(A1) and inv(A2)
2. Solve Sx3 =b~3
3. Compute x1 and x2 by using inv(A1) and inv(A2)
The explicit computation of S can be avoided by solving the linearSystem in S iteratively, e.g. Jacobi, pcg, ….Then we need only part of S and in every iteration step we have to compute S * intermediate vector.
To achieve fast convergence, a preconditioner (approximation) for S has to be used!
Iterative methods and preconditioning will be subject of later chapters.