linear systems of equations - sutherland, utah...gaussian elimination! • algorithm to solve...

Linear Systems of Equations

ChEn 2450

1 LinearSystems-direct.key - August 25, 2014

v_

+

I6

I1I2

R2

R4

R3I3

I4

R5

R1

I5

Kirchoff ’s Laws give the following equations for the currents (I) in terms of the resistances (R)

Assuming the voltage (v) and resistances (Ri) are known, solve for the currents, Ij.

Circuit analysis (also used in heat transfer)

Example

1. Define the ordering for the unknowns (x)

2. Define the “A” matrix

3. Define the “b” (rhs) vector.

4. Solve for “x”

v �R2I2 �R4I4 = 0�R2I2 + R1I1 + R3I3 = 0�R4I4 �R3I3 + R5I5 = 0

I6 = I1 + I2

I2 + I3 = I4

I1 = I3 + I5


Systems of Linear Equations⇤

⌥⌥⌥⇧

a11 a12 · · · a1n

a21 a22 · · · a2n...

.... . .

...am1 am2 · · · amn

⌅

��⌃

�

↵↵↵

x1

x2...

xn

⇥

��⌦=

�

↵↵↵

b1

b2...

bn

⇥

��⌦

Any system of linear equations

may be written as:

Alternatively, Ax=b, or n�

j=1

ai,jxj = bi

Note: in MATLAB, you may easily solve a system of linear equations by defining a square matrix, A, a column vector, b, and setting x=A\b;

Unknowns

A = [ a11 a12 a13 a14 a15; ...! a21 a22 a23 a24 a25; ...! a31 a32 a33 a34 a35; ...! a41 a42 a43 a44 a45; ...! a51 a52 a53 a54 a55; ];!!b = [ b1 b2 b3 b4 b5 ]';! !x = A\b;

Example: 5 equations1. Define the “A” matrix (here we assume

that aij have been defined previously, or substitute appropriate numbers).

2. Define the “b” (rhs) vector.

3. Solve for “x”

to be well-posed, what is the relationship between m and n?

a11x1 + a12x2 + · · ·+ a1nxn = b1

a21x1 + a22x2 + · · ·+ a2nxn = b2

...

am1x1 + am2x2 + · · ·+ amnxn = bm


Gaussian Elimination • Algorithm to solve systems of linear equations

• Basically a methodic approach to solving equations ‘by hand.’

• Cost scales as n3 where n is the number of equations.

LU Factorization • Useful when you need to solve Ax=b for different b but same A.

• L - Lower diagonal matrix, U - Upper diagonal matrix.

• Determining L, U is “expensive”, but solving the resulting system is simple.

• See “help lu” in MATLAB

Direct Solution Methods

�

⇧⇧⇧⇤

a11 a12 · · · a1m

a21 a22 · · · a2m...

.... . .

...an1 an2 · · · anm

⇥

⌃⌃⌃⌅=

�

⇧⇧⇧⇧⇤

1 0 · · · 0

�21 1. . .

......

. . . 1 0�n1 · · · �n,m�1 1

⇥

⌃⌃⌃⌃⌅

�

⇧⇧⇧⇤

u11 u12 · · · u1m

0 u22 · · · u2m... 0

. . ....

0 · · · 0 unm

⇥

⌃⌃⌃⌅

A = LU � LUx = b

Lb̃ = b Ux = b̃

Hoffman §1.3-1.4

NOTE: if you are rusty on solution of linear systems, read through Hoffman §1.1-1.3 carefully & do the examples!


Sparse Linear SystemsSparse systems of equations have mostly zeros in the matrix.

Often arise from solving ODEs or PDEs • Also in “networked” systems

‣ Networked flow systems - separation units (you will see this in later ChEn courses)

‣ Networked “spring” systems

‣ Electrical circuits

‣ many others...

Idea: instead of storing all of the zeros, only store the nonzero entries, along with the row & column that they belong in. • For k non-zeros per equation, this reduces storage from n2 to kn.

• For “small” n, storing all of the zeros isn’t a problem. For “large” n, it is.

Typically solved by iterative methods...

Hoffman §1.7


Sparse MatricesT (0) = To

dT

dx

��x=L

= 0

x=0 x=L

Example: Steady-state, 1-dimensional heat transfer

solve at a “discrete” set of points:

more on how to do this later in the semester...

The

se a

re “

trid

iago

nal”

mat

rice

s (s

peci

al k

inds

of s

pars

e m

atri

ces)

.

Discrete solution for n=7 points.

1�x2

⇤

⌥⌥⌥⌥⌥⌥⌥⌥⇧

�3 1 0 0 0 0 01 �2 1 0 0 0 00 1 �2 1 0 0 00 0 1 �2 1 0 00 0 0 1 �2 1 00 0 0 0 1 �2 10 0 0 0 0 1 �1

⌅

��⌃

�

↵↵↵↵↵↵↵↵

T1

T2

T3

T4

T5

T6

T7

⇥

��⌦

=

�

↵↵↵↵↵↵↵↵

�S1� �

2�x2 To

�S2�

�S3�

�S4�

�S5�

�S6�

�S7�

⇥

��⌦

d2T

dx2= �S

�

Discrete solution for n=5 points.

1�x2

⇤

⌥⌥⌥⌥⇧

�3 1 0 0 01 �2 1 0 00 1 �2 1 00 0 1 �2 10 0 0 1 �1

⌅

��⌃

�

↵↵↵↵

T1

T2

T3

T4

T5

⇥

��⌦=

�

↵↵↵↵

�S1� �

2�x2 To

�S2�

�S3�

�S4�

�S5�

⇥

��⌦

How would you write a MATLAB code to generate the matrix & rhs vector (given n)?


MATLAB Tip: Creating Banded Matrices

d = [-3 -2 -2 -2 -1];!ud = [1 1 1 1];!ld = [1 1 1 1];! !A = diag(ld,-1) + diag(d,0) + diag(ud,1);

A =

�

⇧⇧⇧⇧⇤

�3 1 0 0 01 �2 1 0 00 1 �2 1 00 0 1 �2 10 0 0 1 �1

⇥

⌃⌃⌃⌃⌅

n = 5;!d = -2*ones(n,1);!ud = ones(n-1,1);!ld = ones(n-1,1);! !A = diag(ld,-1) + diag(d,0) + diag(ud,1);

A =

�

⇧⇧⇧⇧⇤

�2 1 0 0 01 �2 1 0 00 1 �2 1 00 0 1 �2 10 0 0 1 �2

⇥

⌃⌃⌃⌃⌅

diag(v,k) - create a matrix with the vector v on its kth diagonal. • k = 0 ⇒ main diagonal

• k = 1 ⇒ upper diagonal

• k = -1 ⇒ lower diagonal

• k = 2 ⇒ 2nd upper diagonal.

This creates a dense matrix. If you get into large systems of equations, you will need to use sparse matrices. See the sparse function in MATLAB.


Tridiagonal Matrices (“Special” Kinds of Sparse Matrices)

No need to store n2 entries of T - there are only 3n-2 non-zero entries.

ℓ⇒ lower diagonal (n-1 entries)

d ⇒ main diagonal (n entries)

u ⇒ upper diagonal (n-1 entries)

Tx = b

Hoffman §1.5

NOTE: there are many different varieties of the Thomas algorithm.

They are all equivalent.

The Thomas Algorithm: (Given ℓ, u, d, b)

1. for i=2,3,...,n

2.

di = di �ui�1�i�1

di�1

bi = bi �bi�1�i�1

di�1

xn = bndn

,

xi = bi�uixi+1di

i = n� 1, . . . , 1

T =

�

⇧⇧⇧⇧⇧⇤

a11 a12

a21 a22 a23

. . . . . . . . .an�1,n�2 an�1,n�1 an�1,n

an,n�1 an,n

⇥

⌃⌃⌃⌃⌃⌅

=

�

⇧⇧⇧⇧⇧⇤

d1 u1

�1 d2 u2

. . . . . . . . .�n�2 dn�1 un�1

�n�1 dn

⇥

⌃⌃⌃⌃⌃⌅


Thomas Algorithm - Example2x1 � 3x2 = �4

x1 + 2x2 � x3 = 24x2 � x3 + x4 = 9

2x3 � x4 = 2

� =

�

⇤142

⇥

⌅ d =

�

⇧⇧⇤

22�1�1

⇥

⌃⌃⌅ u =

�

⇤�3�1

1

⇥

⌅

⇤

⌥⌥⇧

2 �31 2 �1

4 �1 12 �1

⌅

��⌃

�

↵↵

x1

x2

x3

x4

⇥

��⌦ =

�

↵↵

�4292

⇥

��⌦

d2 = d2 �u1⇥1d1

= 2� (�3)(1)2

=72

d3 = d3 �u2⇥2d2

= �1� (�1)(4)7/2

=17

d4 = d4 �u3⇥3d3

= �1� (1)(2)1/7

= �15

b2 = b2 �b1⇥1d1

= 2� (�4)(1)2

= 4

b3 = b3 �b2⇥2d2

= 9� (4)(4)7/2

=317

b4 = b4 �b3⇥3d3

= 2� (31/7)(2)1/7

= �60

2. Setxn = bn

dn, i = n,

xi = bi�uixi+1di

i = n� 1, . . . , 1

x4 =b4

d4=�60�15

= 4

x3 =b3 � u3x4

d3=

31/7� (1)(4)1/7

= 3

x2 =b2 � u2x3

d2=

4� (�1)(3)7/2

= 2

x1 =b1 � u1x2

d1=�4� (�3)(2)

2= 1

What kinds of error are present?

• Iteration error? • Roundoff error? • Approximation error?

1. Recall: d1 and b1 remain unchanged.di = di �

ui�1⇤i�1

di�1, bi = bi �

bi�1⇤i�1

di�1i = 2, 3, . . . , n


linear systems of equations - sutherland, utah...gaussian elimination! • algorithm to solve...

Documents