ch10 lu decomposition 21

8/18/2019 Ch10 LU Decomposition 21

1/34

The Islamic University of Gaza

Faculty of Engineering

Civil Engineering Department

Numerical Analysis

ECIV 3306

Chapter 10

LU Decomposition and Matrix Inversion


2/34

Introduction

• Gauss elimination solves [A] {x} ={B}

• It becomes insufficient when solving theseequations for different values of {B}

• LU decomposition works on the matrix [A] and thevector {B} separately.

• LU decomposition is very useful when the vector ofvariables {x} is estimated for different parametervectors {B} since the forward elimination process isnot performed on {B}.


3/34

LU Decomposition

If:L: lower triangular matrix

U: upper triangular matrix

Then,

[A]{X}={B} can be decomposed into two

matrices [L] and [U] such that:

1. [L][U] = [A] ([L][U]){X} = {B}


4/34

LU Decomposition

Consider:

[U]{X} = {D}

So, [L]{D} = {B}

2. [L]{D} = {B} is used to generate an

intermediate vector {D} by forward substitution 3. Then, [U]{X}={D} is used to get {X} by back

substitution

([L][U]){X} = {B}


5/34

Summary of LU Decomposition


6/34

LU Decomposition

As in Gauss elimination, LU decomposition must

employ pivoting to avoid division by zero and to

minimize round off errors. The pivoting is done

immediately after computing each column.


7/34

LU Decomposition

3

2

1

3

2

1

333231

232221

131211

b

b

b

x

x

x

aaa

aaa

aaa

11 12 13

/ /

22 23

//

33

[ ] 0

0 0

a a a

U a a

a

21

31 32

1 0 0

[ ] 1 0

1

L l

l l

Step 1: Forward elimination

System of linear equations [A]{x}={B}

][]][[ AU L

11

2121

a

al

11

3131

a

al

'

'

32

22

32

a

al


8/34

LU Decomposition

Step 2: Generate an intermediate vector {D} by forward

substitution

Step 3: Get {X} by back substitution.

3

2

1

3

2

1

3231

21

101

001

bb

b

d d

d

l l l

3

2

1

3

2

1

33

2322

131211

''00

''0

d

d

d

x

x

x

a

aa

aaa

[L]{D} = {B}

[U]{X}={D}


9/34

LU Decomposition-Example

102030

307 10

20103

A

..

..

.. 3 0.1 0.2

0 7.003 0.293

0 0.19 10.02

21 31

\

3232 \

22

0.1 0.30.03333; 0.10003 3

0.190.02713

7.003

l l

al

a

0121000

29300037 0

20103

U

.

..

.. 1 0 0

[ ] 0.03333 1 0

0.1000 .02713 1

L


10/34

LU Decomposition-Example (cont’d)

4.71

3.19

85.7

1

102.03.0

3.071.0

2.01.03

3

2

1

x

x

x

0843.70

5617.1985.7

4.71

3.1985.7

102713.01000.0

010333.0001

3

2

1

3

2

1

d

d d

d

d d

Step 2: Find the intermediate vector {D} by forward substitution

Use previous [L] and {D} matrices to solve the system:


11/34

LU Decomposition-Example (cont’d)

Step 3: Get {X} by back substitution.

00003.7

5.2

3

0843.70

5617.19

85.7

012.1000

2933.00033.70

2.01.03

3

2

1

3

2

1

x

x

x

x

x

x


12/34

Decomposition Step

• % Decomposition Step

for k=1:n-1

[a,o]= pivot(a,o,k,n);

for i = k+1:n

a(i,k) = a(i,k)/a(k,k);a(i,k+1:n)= a(i,k+1:n)-a(i,k)*a(k,k+1:n);

end

end


13/34

Partial Pivoting

• %Partial Pivoting

function [a,o] = pivot(a,o,k,n)

[big piv]=max(abs(a(k:n,k)));

piv=piv+(k-1);

if piv ~= ktemp = a(piv,:);

a(piv,:)= a(k,:);

a(k,:)=temp;

temp = o(piv);

o(piv)=o(k);

o(k)=temp;

end


14/34

Substitution Steps

%Forward Substitution

d(1)=bn(1);

for i=2:n

d(i)=bn(i)-a(i,1:i-1)*(d(1:i-1))';

end

• % Back Substitution

x(n)=d(n)/a(n,n);

for i=n-1:-1:1

x(i)=(d(i)-a(i,i+1:n)*x(i+1:n)')/a(i,i);

end


15/34

Matrix Inverse Using the LU

Decomposition

• LU decomposition can be used to obtain the

inverse of the original coefficient matrix [A].

• Each column j of the inverse is determined by

using a unit vector (with 1 in the jth raw).


16/34

Matrix Inverse: LU Decomposition

0

0

1

}{

1

11

x A

b x A

0

1

0

}{

2

22

x A

b x A

1

0

0

}{

3

33

x A

b x A

1st column

of [A]-12nd column

of [A]-13rd column

of [A]-1

3211

x x x A }{}{}{

[A] [A]-1 = [A]-1[A] = I


17/34

Matrix inverse using LU decomposition Example

102030

307 1020103

A

..

..

..

0121000

29300037 020103

U

.

....1 0 0

[ ] 0.03333 1 0

0.1000 .02713 1

L

1009.0

03333.0

1

0

0

1

102713.01000.0

010333.0

001

3

2

1

3

2

1

d

d

d

d

d

d

1A. [L]{d}1 = {b}1

1B. Then, [U]{X}1={d}1

01008.0

00518.0

33249.0

1009.0

03333.0

1

012.1000

2933.00033.70

2.01.03

3

2

1

3

2

1

x

x

x

x

x

x

1st column

of [A]-1


18/34

2A. [L]{d}2 = {b}2

Matrix inverse using LU decomposition

Example (cont’d)

02713.0

1

0

0

1

0

102713.01000.0

010333.0

001

3

2

1

3

2

1

d

d

d

d

d

d

00271.0

142903.0

004944.0

02713.0

1

0

012.1000

2933.00033.70

2.01.03

3

2

1

3

2

1

x

x

x

x

x

x

2B. Then, [U]{X}2={d}22nd column

of [A]-1


19/34

3A. [L]{d}3 = {b}3


Example (cont’d)

1

0

0

1

0

0

102713.01000.0

010333.0

001

3

2

1

3

2

1

d

d

d

d

d

d

09988.0

004183.0006798.0

1

00

012.1000

2933.00033.702.01.03

3

2

1

3

2

1

x

x x

x

x x

3B. Then, [U]{X}3={d}33rd column

of [A]-1


20/34


Example (cont’d)

09988.000271.001008.0004183.0142903.000518.0

006798.0004944.033249.0

][

1

A


21/34

ERROR ANALYSIS AND SYSTEM CONDITION

The inverse provides a means to discern whethersystems are ill-conditioned.

1. Scale the matrix of coefficients [A] so that the largest

element in each row is 1. Invert the scaled matrix and if

there are elements of [A]−1

that are several orders ofmagnitude greater than one, it is likely that the system is ill-

conditioned.

2. Multiply the inverse by the original coefficient matrix and

assess whether the result is close to the identity matrix. If

not, it indicates ill-conditioning.

3. Invert the inverted matrix and assess whether the result is

sufficiently close to the original coefficient matrix. If not, it

again indicates that the system is ill-conditioned.


22/34

Vector and Matrix Norms

Norm is a real-valued function that provides a measure ofsize or “length” of vectors and matrices.

Norms are useful in studying the error behavior of

algorithms.

y.repectivelaxes,zandy,x,alongdistancesthearecand b,a,where

asdrepresente becanthat

spaceEuclideanldimensiona-in threevectoraisexamplesimpleA

cba F


23/34

Vector and Matrix Norms (cont’d)


24/34


• The length of this vector can be simply computed as

222 cba F e

n

i

ji

n

j

e

n

i

ie

n

a

x

x x x X

1

2

,

1

1

2

21

A

[A]matrixaFor

X

ascomputedisnormEuclideana

Length or Euclidean norm of [F]

• For an n dimensional vector

Frobenius norm

(10.24)


25/34


For vectors, there are alternatives called p norms

that can be represented generally by

1/

1

X

P-Norm p

p

n

i pi

x

We can also see that Euclidean norm and the 2

norm, ||X||2, are identical for vectors.

In contrast to vectors, the 2 norm and Euclidean

norm for a matrix are not the same.


26/34


• Uniform vector norm

• Uniform matrix norm (row sum Norm)

ini

xmax1X

n

j ji

ni a1 ,1maxA

11

,11 1

1-Norm

For Vector

CFor olumn Sum Norm

X

( )Matrix

A max

n

ii

n

i j j n i

x

a


27/34


• Matrix Condition Number Defined as:

• For a matrix [A], this number will be greater than or

equal to 1.

• Relative error of the norm

• For example, if the coefficients of [A] are known to

t -digit precision (rounding errors~10-t) and

Cond [A]=10c, the solution [X] may be valid to only

t-c digits (rounding errors~10c-t).

1 A A ACond

A

A ACond

X

X


28/34

The condition number is considerably greater than

unity suggests that the

system is ill-conditioned.


29/34

Iterative Refinement

• Round-off errors can be reduced by the following procedure:

• Suppose an approximate solution vectors given by

• Substitute the result in the original system

....(Eq.1)

]~~~[~

321 x x x X T

3333232131

2323222121

1313212111

~~~~

~~~~

~~~~

b xa xa xa

b xa xa xab xa xa xa

3333232131

2323222121

1313212111

b xa xa xa

b xa xa xa

b xa xa xa


30/34

Iterative Refinement (cont’d)

• Now, assume the exact solution is:

• Then:

… …….(Eq.2)

333

222

111

~

~

~

x x x

x x x

x x x

3333322321131

2332322221121

1331322121111

)~()~()~(

)~()~()~(

)~()~()~(

b x xa x xa x xa

b x xa x xa x xa

b x xa x xa x xa


31/34

Iterative Refinement (cont’d)

Subtract Eq.2 from Eq.1, the result is a system of linear

equations that can be solved to obtain the correction factors

The factors then can be applied to improve the solution as

specified by the equation:

333333232131

222323222121

111313212111

~

~

~

E bb xa xa xa

E bb xa xa xa

E bb xa xa xa

x

333

222

111

~

~

~

x x x

x x x

x x x


32/34

Iterative Refinement- Example (cont’d)

Solve:

The exact solution is ………

1- Solve the equations using [A]-1, such as {x}=[A]-1{c}

]00.100.100.1[T X

]00.1997.0991.0[~ T X

58.232.211.285.1

22.598.077.653.2

28.511.206.123.4

321

321

321

x x x

x x x

x x x


33/34

Iterative Refinement - Example

(cont’d)

2- Substitute the result in the original system [A]{x}={c}

}

~

{}

~

]{[ C X A

59032.222246.5

24511.5

}~

{C

0103.0

00246.0

0348.0

59032.258.2

22246.522.5

24511.528.5

}~

{}{ C C E


34/34

Iterative Refinement - Example

(cont’d)

3- Solve the system of linear equations

using [A]-1 to find the correction factors to yield

}{}~

]{[ E X A

x

]00000757.000300.000822.0[ T X

00.1~00.1~

999.0~

333

222

111

x x x x x x

x x x

ch10 lu decomposition 21

Documents