1

Chapter: 3aSystem of Linear Equations

Dr. Asaf Varol

asvarol@mail.wvu.edu

2

Introduction (I)

For example5x + 2y = -13x - 2y = 1

is a system of two linear equations with two unknowns x and y. There may be casesthat the equation contains exponents of the unknown other than one, but by algebraicmanipulation it can be reduced to a linear form. Consider the following system ofequations

5x2 + 2xy2 -13x = 0x - y2 = 1

Dividing the first equation by x (x can not be zero in this case), and substituting z = y2

the above system reduces to

5x + 2z = 13 x - z = 1

• Linear Equation: an equations where only the first exponent of the unknown is present

• System of Linear Equations: the set of linear equations where there are more than one unknown

3

Introduction (II)

4

Introduction Example - Vibration of Three Railroad Cars

m1

k1 k2 k3

x1 x3 x2

m2 m3

Vibration of three railroad cars connected to each other by a spring on a barge.

Newton’s second law to each car yields

-k1 x1 + k2 (x2 – x1) = m1 a1

-k2 (x2 – x1) + k3 (x3 – x2) = m2 a2

-k3 (x3 – x2) = m3 a3

5

Introduction Example - Vibration of Three Railroad Cars

L e t k 1 = k 2 = k 3 = k = 1 0 0 0 0 k g / s 2 , a n d m 1 = 2 0 0 0 k g , m 2 = 3 0 0 0 k g , a n d m 3 = 1 0 0 0 k g .D e t e r m i n e t h e r e l a t i v e l o c a t i o n o f e a c h c a r w h e n t h e i r a c c e l e r a t i o n a r e a l l t h e s a m e a n de q u a l t o 1 m / s 2 . S u b s t i t u t i n g t h e s e v a l u e s i n t o t h e a b o v e e q u a t i o n s a n d r e a r r a n g i n g y i e l d

- 2 x 1 + x 2 = 0 . 2 x 1 - 2 x 2 + x 3 = 0 . 3

x 2 - x 3 = 0 . 1

T h i s p a r t i c u l a r f o r m i s v e r y s u i t a b l e f o r m a t r i x r e p r e s e n t a t i o n o f t h e s y s t e m o f e q u a t i o n ss u c h t h a t

[ A ] { X } = { C }

w h e r e [ A ] d e n o t e s t h e m a t r i x o f c o e f f i c i e n t s , { X } r e p r e s e n t s t h e v e c t o r o f t h e u n k n o w n s ,x 1 , x 2 , x 3 , a n d { C } r e p r e s e n t s t h e r i g h t h a n d s i d e c o e f f i c i e n t s ( o r c o n s t a n t s ) a s a v e c t o r :

2 1 0

1 2 1

0 1 1

0 2

0 3

0 1

1

2

3

x

x

x

.

.

.

6

Introduction Example - Forces Acting on a Simple Truss

Drawing a free body diagram at every joint and setting the summation of forces in x- andy-directions separately to zero give the following set of equations.from joint a:

Fax + Fab cos45 + Fad cos30 = 0Fay + Fab sin45 + Fad sin30 = 0

from joint b:

-Fab cos45 + Fbc cos45 + 3 = 0-Fab sin45 – Fbc sin45 –Fbd = 0

from joint c:

-Fcd cos30 – Fbc cos45 = 0Fcy + Fcd sin30 + Fbc sin45 = 0

from joint d:

-Fad cos30 + Fcd cos30 = 0Fbd – Fad sin30 – Fcd sin30

7

Introduction Example - Forces Acting on a Simple Truss

Rearranging and organizing the equations:

Fax + cos(45) Fab + cos(30) Fad = 0Fay + sin(45) Fab + sin(30) Fad = 0

- cos(45) Fab + cos(45) Fbc = -3- sin(45) Fab - sin(45) Fbc - Fbd = 0

- cos(45) Fbc - cos(30) Fcd = 0sin(45) Fbc + sin(30) Fcd + Fcy = 0

- cos(30) Fad + cos(30) Fcd = 0- sin(30) Fad + Fbd - sin(30) Fcd = 0

Again we have arranged the equations in such an organized fashion that it can be easilyput into the form of a matrix equation. Let

F1 = Fax, F2 = Fay, F3 = Fab, F4 = Fad, F5 = Fbc, F6 = Fbd, F7 = Fcd, and F8 = Fcy

The matrix equation then can be written as

[G]{F} = {L}

Where the [G] is the geometry matrix, {F} is the force vector, and {L} is the load vector.

8

Introduction Example - Forces Acting on a Simple Truss

Colm.

Row i

Gi1

1

Gi2

2

Gi3

3

Gi4

4

Gi5

5

Gi6

6

Gi7

7

Gi8

8

Fi Li

1 1 0 .707 .866 0 0 0 0 F1 02 0 1 .707 .5 0 0 0 0 F2 03 0 0 -.707 0 .707 0 0 0 F3 -34 0 0 -.707 0 -.707 -1 0 0 F4 05 0 0 0 0 -.707 0 -.866 0 F5 06 0 0 0 0 .707 0 .5 1 F6 07 0 0 0 -.866 0 0 .866 0 F7 08 0 0 0 -.5 0 1 -.5 0 F8 0

The system of equations Equation 3.1.11 should be solved for the 8 unknowns. Thesolutions are Fax = -3 kN, Fay = -1.5 kN, Fab = -0.776 kN, Fad = 4.10 kN, Fbc = -5.02kN, Fbd = 4.10 kN, Fcd = 4.10 kN, Fcy = 1.5 kN. Solving 8 equations for 8 unknowns byhand is, of course, not a desirable assignment. Thanks to the computers and the solutionalgorithms we shall present in this chapter this task can be performed with relative ease.

9

Review of Matrix Algebra

Let [A] be a general 3x3 matrix denoted as

a11 a12 a13

[A] = aij = a21 a22 a23

a31 a32 a33

where the first index i=1,2,3 denotes the row number, and the second index j=1,2,3denotes the column number. We call the elements aii, i.e. a11, a22, a33 the dioganalelements, those elements above the diagonal the upper diagonal elements, and thosebelow the diagonal the lower diagonal elements.When the lower diagonal elements are zero the matrix is called an upper triangular matrix,and when the upper diagonal elements are zero it is called lower triangular matrix.

In general a matrix with m rows and n columns is donated by

[A] = aij ; i=1,2,3,...,m; j=1,2,3,...,n

When m=n the matrix is called a square matrix. The size of the matrix is denoted by nxm.

10

Review of Matrix Algebra - Addition

In order that two matrices can be added to each other they must be of the same size. Theelements of the sum of two matrices is the sum of the corresponding elements of the twomatrices, hence

[A] + [B] = [C]

means

aij + bij = cij

i.e. c11 = a11 + b11; c12 = a12 + b12; c21 = a21 + b21 etc.

11

Review of Matrix Algebra - Multiplication

In order that two matrices can be multiplied the latter matrix must have the samenumber of rows as the number of columns of the first. That is

[A]mxn [B]nxk = [C]mxk

The resulting product matrix has the same number of rows as the first and the samenumber of columns as the second matrix. In index notation the multiplication of twomatrices is written as

aij bjk = cik

where the summation is over the index “j”. It is important to note that the order ofmultiplication can not be changed, that is

[A][B] [B][A]

12

Example E3.2.1

Problem: Find the product of the matrices given below:

1 -1 0[A] = aij = 2 -2 1

3 0 -1

Solution:

i=1, k=1 c11 = a11b11 + a12b21 + a13b31 = 1 * 2 + -1 * 0 + 0 * 3 = 2

i=2,k=1 c21 = a21b11 + a22b21 + a23b31 = 2 * 2 + -2 * 0 + 1 * 3 = 7

i=3,k=1 c31 = a31b11 + a32b21 + a33b31 = 3 * 2 + 0 * 0 + -1 * 3 = 3

i=1,k=2 c12 = a11b12 + a12b22 + a13b32 = 1 * -1 + -1 * 1 + 0 * 0 = -2

i=2,k=2 c12 = a21b12 + a22b22 + a23b32 = 2 * -1 + -2 * 1 + 1 * 0 = -4

i=3,k=2 c12 = a31b12 + a32b22 + a33b32 = 3 * -1 + 0 * 1 + -1 * 0 = -3

2 -2[C] = cik = 7 -4

3 -3

2 -1[B] = bjk = 0 1

3 0

13

Transpose and Determinant of a Matrix

The transpose of a matrix, [A]T is obtained by interchanging the rows of the originalmatrix by the columns.

[B] = [A]T ; bij = aji

For a symmetric matrix [A] it follows then, [A]T = [A].

The Determinant of a matrix is defined by

det[A] = (-1)i+j aij (determinants of the minor matrices)

The minor matrix is a submatrix of the original matrix obtained by closing the ith row andthe jth column. If two rows are interchanged the determinant changes sign. The smallestsubmatrix is a 2x2 matrix and its determinant is given by

deta a

a aaa aa11 12

21 22

1122 2112

If det[A] = 0, then the matrix [A] is said to be singular and the system has no uniquesolution.

14

Example E3.2.2 - Find the Determinant of a Matrix

15

Identity Matrix and the Inverse of a Matrix

The inverse, [A]-1, of a matrix, [A], is defined such that [A][A]-1 = [I] = [A]-1[A] where the matrix [I] is called the identity matrix with diagonal elements equal to one, and all other elements are zero. For example a 4x4 identity matrix is given by

I

1 0 0 0

0 1 0 0

0 0 1 0

0 0 0 1

16

Inverse of a matrix

17

Inverse of a matrix (Cont’d)

18

Rules Governing Matrix Algebra

Associative laws ([A]+[B])+[C] = [A]+([B]+[C])([A][B])[C] = [A]([B][C])

Commutative law [A] + [B] = [B] + [A]

Distributive law [A]([B]+[C]) = [A][B] + [A][C]

([A][B])T = [B]T [A]T

([A] + [B])T =[A]T + [B]T

Inverse ([A][B])-1 = [B]-1 [A]-1

(c=constant) (c[A])-1 = [A]-1/c

Determinant det([A][B]) = det[A]det[B]det([A]T) = det[A]

(c=constant) det(c[A]) = cn det[A] (n=order)

19

Matrix Norms

T h e n o r m o f a v e c t o r o r a m a t r i x i s a n o n - n e g a t i v e n u m b e r t h a t i s a m e a s u r e o f t h em a g n i t u d e o f t h a t v e c t o r o r m a t r i x . T h e m a g n i t u d e ( o r t h e n o r m ) o f a s c a l a r n u m b e r i s i t sa b s o l u t e v a l u e . S i n c e t h e r e a r e m o r e t h a n o n e s c a l a r i n v o l v e d i n v e c t o r s a n d m a t r i c e s t h en o r m s f o r v e c t o r s a n d m a t r i c e s c a n b e d e f i n e d a n d c o m p u t e d i n m a n y w a y s . T h em a g n i t u d e o f v e c t o r i s u s u a l l y d e f i n e d a s t h e s q u a r e r o o t o f t h e s u m o f t h e s q u a r e s o f i t se l e m e n t s ( o r c o m p o n e n t s ) , i . e . f o r { v } = - 1 i + 2 j - 3 k ; v 1 = - 1 , v 2 = 2 , v 3 = - 3 t h e m a g n i t u d ei s g i v e n b y

{ V } = v v v1

2

2

2

3

2 121 4 9 3 7 4 ( ) .

T h i s n o r m i s k n o w n a s t h e E u c l i d i a n n o r m , a n d i s g i v e b y

{ }V v ii

n

2

1

12

I n g e n e r a l a “ p ” n o r m c a n b e d e f i n e d b y

{ }V v i

p

i

n p

1

1

20

Matrix Norms (II)

A n o t h e r c o m m o n l y u s e d n o r m i s t h e s o - c a l l e d u n i f o r m v e c t o r n o r m g i v e n b y

{ } m a xV v i f o r 1 i n

S i m i l a r n o r m s a r e d e f i n e d f o r a m a t r i x , [ A ] o f s i z e n x n a s f o l l o w s :

F r o b e n i u s n o r m :

[ ]A ae i j

j

n

i

n

2

11

12

U n i f o r m m a t r i x n o r m ( o r r o w - s u m n o r m ) :

[ ] m a xA a i jj

n

1 f o r 1 i n

C o l u m n n o r m ( o r c o l u m n - s u m n o r m ) :

[ ] m a xA ac i j

i

n

1 f o r 1 j n

21

Example E3.2.3

P r o b l e m : C o m p u t e t h e t h r e e c o m m o n n o r m s f o r t h e f o l l o w i n g m a t r i x

A

1 2 1

2 3 2

3 4 5

S o l u t i o n : U s i n g t h e d e f i n i t i o n s , w e o b t a i n

F r o b e n i u s n o r m :

[ ]Ae

{ ( 1 ) 2 + ( 2 ) 2 + ( 3 ) 2 + ( - 2 ) 2 + ( 3 ) 2 + ( 4 ) 2 + ( - 1 ) 2 + ( - 2 ) 2 + ( 5 ) 2 } 1 / 2 = 8 . 5 4

U n i f o r m m a t r i x n o r m :

[ ]A

m a x { ( 1 + 2 + 1 ) , ( 2 + 3 + 2 ) , ( 3 + 4 + 5 ) }

= m a x ( 4 , 7 , 1 2 ) = 1 2

C o l u m n n o r m :

[ ]Ac

m a x { ( 1 + 2 + 3 ) , ( 2 + 3 + 4 ) , ( 1 + 2 + 5 ) }

= m a x ( 6 , 9 , 8 ) = 9

22

Condition Number and Ill-Conditioned Systems

The condition number of a matrix, A. is defined by 

Cond([A]) =  where the symbol indicates any norm of the matrix. It can be shown by matrix algebra, that the condition number of a matrix is always greater than or equal to one., i.e. 

Cond ([A]) =  If the condition number of a matrix is “large” it is said to be ill-conditioned. Systems of equations that involve ill-conditioned matrices are difficult to solve. These systems must be first preconditioned before attempting to find a numerical solution. How large should the condition number be before one can call a matrix truly ill-conditioned is a matter of order of magnitudes. For ill-conditioned systems, a small change in coefficients leads to large changes in the solution.

[ ] [ ]A A 1

[ ] [ ] [ ][ ]A A A A I = = 1 1 1

23

Example E3.3.1

Problem:Find the condition number of the following matrix: 

[A] = [A]-1 = (1/56)  

Solution: Using the uniform matrix norm:  = Max (4, 7, 12) = 12; = Max[(1/56) (36, 24, 18)] = 36/56 = 9/14

Cond ([A]) = (12)(9/14) = 54/7  Using the Column norm  = Max (6, 9, 8) = 9; = Max[(1/56) (40, 24, 14)] = 40/56 = 5/7 Cond ([A]) = (9)(5/7) = 45/7

543

232

121 23 6 7

16 8 0

1 10 7

[ ]A [ ]A

1

[ ]Ac

[ ]Ac

1

24

Rules of Thumb for Checking the Condition of a Matrix

Apply the following tests after scaling (or normalizing) the matrix, [A], such that the largest element in each row is one.

(i) If the sum of the absolute value of off-diagonal elements is less than the absolute value of the diagonal element for each row separately, this matrix is most likely to be well-conditioned. Note that interchanging the rows, i.e. partial pivoting, does not improve the condition number of a matrix. 

(ii) If det[A] 0. This matrix is ill-conditioned. 

(iii) If there are elements of [A]-1 which are several order of magnitude larger than one, it is most likely that this matrix is ill-conditioned. 

(iv) Let [I]* = [A] [A]-1; if [I]* is significantly different (i.e. the diagonal elements are not close to one) than the identity matrix, [I], then the matrix [A] is likely to be ill-conditioned. 

(v) Let [A]* = {[A]-1}-1 ; if [A]* is not close to the original matrix [A] the matrix is most likely to be ill-conditioned.

25

Example E3.3.2

Problem: Given the following  Show that the condition number of the matrix does not change when the rows are

interchanged. Solution:  Now interchange the rows to obtain 

 

One can show that the inverse of the matrices are right by checking if the product [B][B]-1 = I is satisfied.Using the Froberius norm we find 

cond([A]) = ||[A]||e||[A]-1||e = (1.4177)(1.4177) = 2.001

 cond([B]) = ||[B]||e||[B]-1||e = (1.4177)(1.4177) = 2.001

 

1000.00000.1

0000.10001.0A

0001.00000.1

0000.11000.0A 1

0000.10001.0

1000.00000.1B

0000.10001.0

1000.00000.1B 1

26

Direct Methods for Solving Linear Systems

Survey of several commonly used direct methods meaning that when the outlined procedure or (algorithm) is completed we get the solution, hence there is no need for iteration to improve an approximate solution. Cramer’s Method: This method is the oldest but the most cumbersome and expensive method to solve a given system of linear equations. It requires computation of (n+1) determinants of nxn matrices, where n is the number of unknowns. The rule is given by 

xi = det{[A]*}/det[A] for i=1,2,3,...,n

(3.4.1) where [A]* is the modified matrix where the “i”th column is replaced by the right hand side of the equations, i.e. by {c}T = (c1,c2,c3, ...

27

Example E3.4.1

Problem: Find the solution of the system of equations given by Equation (3.1.4). Solution: First we write the equations in matrix form [A]{X}={C}and observe that

 

   x1 = a0 = det[A*]1/det[A]; x2 = a1 = det[A*]2/det[A]; x3 = a2 = det[A*]3/det[A]

 where 

 It follows then det[A] = 6 a0 = 8/6 a1 = 3/6 a2 = -5/6

 

A

1 1 1

1 1 1

1 2 4

X

a

a

a

0

1

2

C

0

1

1

A*

1

0 1 1

1 1 1

1 2 4

A*

2

1 0 1

1 1 1

1 1 4

A*

3

1 1 0

1 1 1

1 2 1

28

References

• Celik, Ismail, B., “Introductory Numerical Methods for Engineering Applications”, Ararat Books & Publishing, LCC., Morgantown, 2001

• Fausett, Laurene, V. “Numerical Methods, Algorithms and Applications”, Prentice Hall, 2003 by Pearson Education, Inc., Upper Saddle River, NJ 07458

• Rao, Singiresu, S., “Applied Numerical Methods for Engineers and Scientists, 2002 Prentice Hall, Upper Saddle River, NJ 07458

• Mathews, John, H.; Fink, Kurtis, D., “Numerical Methods Using MATLAB” Fourth Edition, 2004 Prentice Hall, Upper Saddle River, NJ 07458

• Varol, A., “Sayisal Analiz (Numerical Analysis), in Turkish, Course notes, Firat University, 2001

• http://math.uww.edu/faculty/mcfarlat/inverse.htm