lu factorization - unc charlotte faq - unc charlottelu decomposition a matrix a can be decomposed...

LU Factorization

� To further improve the efficiency of solving linear systems

� Factorizations of matrix A : LU and QR

� LU Factorization Methods:

◦ Using basic Gaussian Elimination (GE)◦ Factorization of Tridiagonal Matrix◦ Using Gaussian Elimination with pivoting◦ Direct LU Factorization◦ Factorizing Symmetrix Matrices (Cholesky Decomposition)

� Applications

� Analysis

ITCS 4133/5133: Intro. to Numerical Methods 1 LU/QR Factorization

LU Decomposition

� A matrix A can be decomposed into a lower triangular matrix L andupper triangular matrix U so that

A = LU

� LU decomposition is performed once; can be used to solve multipleright hand sides.

� Similar to Gaussian elimination, care must be taken to avoid roundofferrors (partial or full pivoting)

� Special Cases: Banded matrices, Symmetric matrices.


LU Decomposition: Motivation


LU DecompositionForward pass of Gaussian elimination results in the upper triangular ma-trix

U =

1 u12 u13 · · · u1n

0 1 u23 · · · u2n

0 0 1 · · · u3n... ... ... ... ...0 0 0 · · · 1

We can determine a matrix L such that

LU = A , and

L =

l11 0 0 · · · 0l21 l22 0 · · · 0l31 l32 l33 · · · 0... ... ... ... ...

ln1 ln2 ln3 · · · 1


LU Decomposition via Basic GaussianElimination


LU Decomposition via Basic GaussianElimination: Algorithm


Banded Matrices

� Matrices that have non-zero elements close to the main diagonal

� Example: matrix with a bandwidth of 3 (or half band of 1)a11 a12 0 0 0a21 a22 a23 0 00 a32 a33 a34 00 0 a43 a44 a450 0 0 a54 a55

� Efficiency: Reduced pivoting needed, as elements below bands are

zero.

� Iterative techniques are generally more efficient for sparse matrices,such as banded systems.


LU Decomposition of Tridiagonal Matrix


LU Decomposition of Tridiagonal Matrix:Algorithm


LU Decomposition via GE with Pivoting


LU Decomposition via GE with Pivoting:Algorithm


Direct LU Decomposition

l11 0 0 · · · 0l21 l22 0 · · · 0l31 l32 l33 · · · 0... ... ... ... ...

ln1 ln2 ln3 · · · 1

1 u12 u13 · · · u1n

0 1 u23 · · · u2n

0 0 1 · · · u3n... ... ... ... ...0 0 0 · · · 1

=

a11 a12 a13 · · · a1n

a21 a22 a23 · · · a2n

a31 a32 a33 · · · a2n... ... ... ... ...

an1 an2 an3 · · · ann

The above equation shows the coefficient matrix A can be decomposedinto L and U ; L and U can be determined by the computing the productand equating like terms.


Determining L and U

li1 = ai1, i = 1, 2, . . . , n

u1j =a1j

l11, j = 2, 3, . . . , n

lij = aij −j−1∑k=1

likukj,

for j = 2, 3, . . . , n− 1 and i = j, j + 1, . . . , n

uji =aji −

∑j−1k=1 ljkuki

ljj,

for j = 2, 3, . . . , n− 1 and i = j + 1, j + 2, . . . , n

lnn = ann −n−1∑k=1

lnkukn


Direct LU Decomposition: Example


Direct LU Decomposition: Algorithm


Symmetric Matrices

� Symmetric square matrices - common in engineering, for examplestiffness matrix (stiffness properties of structures).

� For a symmetric matrix A

A = AT

where AT is the transpose of A. Hence

� For a symmetric matrix A

A = LLT


Symmetric Matrices: LU Decomposition

� We can use Cholesky Decomposition to compute the LU decompo-sition of A

l11 =√

a11

lii =

√√√√aii −i−1∑k=1

l2ik, for i = 2, 3, . . . , n

lij =aij −

∑j−1k=1 likljk

ljj, for j = 2, 3, . . . , i− 1, and j < 1


LU (Cholesky) Decomposition of SymmetricMatrices


LU (Cholesky) Decomposition: Algorithm


Applications of LU Decomposition:SolvingLinear systems

[A] ~X = [L][U ][ ~X ] = C

= [L]~e = C

where ~e = [U ] ~X

� Solve for ~e using L (forward substitution)

� Solve for ~X using U (back substitution)


Solving Linear Systems

Forward Substitution

Solve for ~e,

e1 =C1

l11

ei =Ci −

∑nj=1 lijej

lii, for i = 2, 3, . . . n

Back SubstitutionSolve for ~X

Xn = en

Xi = ei −n∑

j=i+1

uijXj, for i = n− 1, n− 2, . . . 1


Solving Linear Systems: Example


Solving Linear Systems: Algorithm


Application 2: Tridiagonal Systems


Application 3: Matrix Inverse


Analysis: LU Decomposition� Why does LU decomposition work?

� Consider the first elimination step: 1 0 0m21 1 0m31 0 1

a11 a12 a13a21 a22 a23a31 a32 a33

=

a11 a12 a13

0 a′

22 a′

230 a

′

32 a′

33

� Define

M1 =

1 0 0m21 1 0m31 0 1

, M2 =

1 0 00 1 00 m32 1

� Hence

M1−1 =

1 0 0−m21 1 0−m31 0 1

, M2−1 =

1 0 00 1 00 −m32 1

,


Analysis: LU Decomposition

� Gradual tranformation of I towards LU:

I.A = A

I.M1−1.M1.A = A

I.M1−1.M2

−1.M2.M1.A = A

� M2.M1.A is U and I.M1−1.M2

−1 is L, given by 1 0 0−m21 1 0−m31 −m32 1


Analysis:LU Decomposition with Pivoting� Decomposition results in the factorization of PA, where P is the

permutation matrix.[1 0 00 1 00 0 1

] [a11 a12 a13

a21 a22 a23

a31 a32 a33

]=

[a11 a12 a13

a21 a22 a23

a31 a32 a33

]� Assume no pivoting in column 1, pivoting in column 2:

I.A = A

I.M1−1.M1.A = A

I.M1−1.P.P.M1.A = A

I.M1−1.P.M2

−1.M2.P.M1.A = A

� M2.P.M1.A is upper triangular, but I.M1−1.P.M2

−1 is not lower tri-angular.

� Premultiply both sides by P :

PM1−1.P.M2

−1.M2.P.M1.A = P.AITCS 4133/5133: Intro. to Numerical Methods 28 LU/QR Factorization

lu factorization - unc charlotte faq - unc charlottelu decomposition a matrix a can be decomposed...

Documents