6 system of linear equations

TMS3133 Numerical Methods

Learning Unit 6 System of Linear Equa>ons

REVIEW

Linear Algebraic Equa>ons and Matrices

Overview A matrix consists of a rectangular array of elements represented by a single symbol (example: [A]).

An individual entry of a matrix is an element (example: a23)

Overview (cont)

A horizontal set of elements is called a row and a ver>cal set of elements is called a column.

The rst subscript of an element indicates the row while the second indicates the column.

The size of a matrix is given as m rows by n columns, or simply m by n (or m x n).

1 x n matrices are row vectors. m x 1 matrices are column vectors.

Special Matrices

Matrices where m=n are called square matrices. There are a number of special forms of square matrices:

Symmetric

A[ ]=5 1 21 3 72 7 8

Diagonal

A[ ]=a11

a22a33

Identity

A[ ]=1

11

Upper Triangular

A[ ]=a11 a12 a13

a22 a23a33

Lower Triangular

A[ ]=a11a21 a22a31 a32 a33

Banded

A[ ]=

a11 a12a21 a22 a23

a32 a33 a34a43 a44

Matrix Opera>ons

Two matrices are considered equal if and only if every element in the rst matrix is equal to every corresponding element in the second. This means the two matrices must be the same size.

Matrix addi>on and subtrac>on are performed by adding or subtrac>ng the corresponding elements. This requires that the two matrices be the same size.

Scalar matrix mul>plica>on is performed by mul>plying each element by the same scalar.

Matrix Mul>plica>on

The elements in the matrix [C] that results from mul>plying matrices [A] and [B] are calculated using:

cij = aikbkjk=1

n

Matrix Inverse and Transpose

The inverse of a square, nonsingular matrix [A] is that matrix which, when mul>plied by [A], yields the iden>ty matrix. [A][A]-1=[A]-1[A]=[I]

The transpose of a matrix involves transforming its rows into columns and its columns into rows. (aij)T=aji

Represen>ng Linear Algebra

Matrices provide a concise nota>on for represen>ng and solving simultaneous linear equa>ons:

a11x1 + a12x2 + a13x3 = b1a21x1 + a22x2 + a23x3 = b2a31x1 + a32x2 + a33x3 = b3

a11 a12 a13a21 a22 a23a31 a32 a33

x1x2x3

=

b1b2b3

[A]{x} = {b}

Solving With MATLAB

MATLAB provides two direct ways to solve systems of linear algebraic equa>ons [A]{x}={b}: LeZ-division x = A\b

Matrix inversion x = inv(A)*b

The matrix inverse is less ecient than leZ-division and also only works for square, non-singular systems.

What have you learned thus far?

Understanding matrix nota>on. Being able to iden>fy the following types of matrices: iden>fy, diagonal, symmetric, triangular, and tridiagonal.

Knowing how to perform matrix mul>plica>on and being able to assess when it is feasible.

Knowing how to represent a system of linear equa>ons in matrix form.

Knowing how to solve linear algebraic equa>ons with leZ division and matrix inversion in MATLAB.

Contents

Elimina>on methods Gauss elimina>on method GE with pivo>ng

What is a system of linear equa>ons?

General form of the system

Matrix Form

Augmented Matrix Form

Solu>ons

In this course, only the case of unique solution where matrix A must be square matrix will be discussed

Graphical Method

For small sets of simultaneous equa>ons, graphing them and determining the loca>on of the intercept provides a solu>on.

Graphical Method (cont)

Graphing the equa>ons can also show systems where: a) No solu>on exists b) Innite solu>ons exist c) System is ill-condi>oned

Concept of Elimina>on method

Deni>on

Gauss Elimina>on Method (1/6)

Discuss the algorithm by example Consider 3 linear equa>ons with 3 unknowns below:

This concept can be extended to n linear equa>ons with n unknowns

Gauss Elimina>on Algorithm (2/6)

The system of linear equa>ons can be wricen as

In augmented matrix form

Gauss Elimina>on Summary

1. Transform augmented matrix to upper triangular form

Do elimina>on process 2. Back subs>tu>on

Nave Gauss Elimina>on (summary) Forward elimina>on

Star>ng with the rst row, add or subtract mul>ples of that row to eliminate the rst coecient from the second row and beyond.

Con>nue this process with the second row to remove the second coecient from the third row and beyond.

Stop when an upper triangular matrix remains.

Back subs>tu>on Star>ng with the last row, solve for the

unknown, then subs>tute that value into the next highest row.

Because of the upper-triangular nature of the matrix, each row will contain only one more unknown.

GE Hand Calcula>on

Solve the system Ax = b where

GE Hand Calcula>on (another example)

Solve the system

143213432123431234

4321

4321

4321

4321

=+++

=+++

=+++

=+++

xxxxxxxxxxxxxxxx

General nonsingular system of n linear equa>ons

Consider

For k = 1, 2, , n-1, carry out the following elimina>on step Step k : Eliminate xk from Eq k+1 through Eq n.

nbxaa

bxaxa

nnnnn

nn

Eq

1Eq

1

11111

=++

=++

!

Elimina>on Step (1/3)

The result of the preceding step will yield a new system

Assume and dene nfxexe

kfxexe

dxcxcbxaxaxa

nnnnknk

knknkkk

nn

nn

Eq

Eq

2Eq1Eq

22222

11212111

=++

=++

=++

=+++

!"

!"#

!!

0kke

nkieemkk

ikik ,,1, +==


For equa>ons i = k+1,,n, subtract mik >mes Eqk from Eqi, elimina>ng xk from Eqi.

The new coecients and the RHS numbers in Eqk+1 through Eqn are dened by

nkifmffnkjiemee

kikii

kjikijij

,,1,

,,1,,

+==

+==


When step n-1 completed, the linear system will be in upper triangular form denoted by

nnnn

nn

gxu

gxuxu

=

=++

!" 11111

Back Subs>tu>on

Solve successively for using back subs>tu>on

11 ,,, xxx nn

1,,1,

,

1 =

=

=

+= niu

xugx

ugx

ii

n

ijjiji

i

nn

nn

Gaussian Elimina>on Pseudocode func>on X = mygauss(A,b) E = [A b]; [r,c] = size(E); %forward elimina>on for i = 1:r-1 for k = i+1:r m(k,i) = E(k,i)/E(i,i);

for j = i+1:c E(k,j) = E(k,j) - m(k,i)* E(i,j); end

end end

%backward subs>tu>on X(r) = E(r,c)/E(r,r); for i = r-1:-1:1

sum = 0; for j = i+1:n sum = sum + E(i,j)*X(j); end X(i) = (E(i,c)-sum)/E(i,i);

end

GE Cartoon

for i = 1

Gaussian Elimina>on func>on X = mygauss(A,b) E = [A b]; [r,c] = size(E); %forward elimina>on

for i = 1:r-1 for k = i+1:r m(k,i) = E(k,i)/E(i,i);


end

end



end

GE Cartoon

k = 2

k = 3

k = r

i = 1

Gaussian Elimina>on func>on X = mygauss(A,b) E = [A b]; [r,c] = size(E); %forward elimina>on for i = 1:r-1

for k = i+1:r m(k,i) = E(k,i)/E(i,i);


end end



end

GE Cartoon

i = 1 k = 2

j = 2 : c

k = 3

j = 2 : c

k = r

j = 2 : c

Gaussian Elimina>on func>on X = mygauss(A,b) E = [A b]; [r,c] = size(E); %forward elimina>on for i = 1:r-1 for k = i+1:r m(k,i) = E(k,i)/E(i,i);

for j = i+1:c E(k,j) = E(k,j) - m(k,i)* E(i,j);

end end end



end

GE Cartoon

i = 2

k = 3

j = 3 : c

k = r

j = 3 : c

GE Cartoon

i = r-1

k = r

j = r : c

GE Summary

GE is an orderly process of transforming an augmented matrix into an equivalent upper triangular form.

The elimina>on opera>on is with nkjiemee kjikijij ,,1,, +== nkie

emkk

ikik ,,1, +==

Nave Gauss Elimina>on Program

Gauss Program Eciency The execu>on of Gauss elimina>on depends on the amount of

oa:ng-point opera:ons (or ops). The op count for an n x n system is:

Conclusions: As the system gets larger, the computa>on >me increases greatly. Most of the eort is incurred in the elimina>on step.

ForwardElimination

2n33 +O n

2( )Back

Substitution n2 +O n( )

Total 2n3

3 +O n2( )

Pivo>ng

Problems arise with nave Gauss elimina>on if a coecient along the diagonal is 0 (problem: division by 0) or close to 0 (problem: round-o error)

One way to combat these issues is to determine the coecient with the largest absolute value in the column below the pivot element. The rows can then be switched so that the largest element is the pivot element. This is called par:al pivo:ng.

If the rows to the right of the pivot element are also checked and columns switched, this is called complete pivo:ng.

Ques>on

How does your GE func>on change to include par>al pivo>ng?

Par>al Pivo>ng Program

Tridiagonal Systems A tridiagonal system is a banded system with a bandwidth of 3:

Tridiagonal systems can be solved using the same method as Gauss elimina>on, but with much less eort because most of the matrix elements are already 0.

f1 g1e2 f2 g2

e3 f3 g3

en1 fn1 gn1en fn

x1x2x3xn1xn

=

r1r2r3rn1rn

Tridiagonal System Solver

What have you learned today? Knowing how to solve small sets of linear equa>ons with the

graphical method. Understanding how to implement forward elimina>on and

back subs>tu>on as in Gauss elimina>on. Understanding how to count ops to evaluate the eciency

of an algorithm. Understanding the concepts of singularity and ill-condi>on. Understanding how par>al pivo>ng is implemented and how

it diers from complete pivo>ng. Recognizing how the banded structure of a tridiagonal system

can be exploited to obtain extremely ecient solu>ons.

6 system of linear equations

Documents