1 gauss elimination small matrices for small numbers of equations, solvable by hand graphical ...

Gauss EliminationGauss Elimination

Small Matrices

For small numbers of equations, solvable by hand Graphical Cramer's rule Elimination

nnnn22n11n

2nn2222121

1nn1212111

bxaxaxa

b xaxaxa

x3x3x2x

rearrange 3xx

2x1 – x2 = 3x1 + x2 = 3

One solution

Graphical Method

2x1 – x2 = 3

2x1 – x2 = – 1

No solution

Graphical Method

6x1 – 3x2 = 92x1 – x2 = 3

Infinite many solutions

Graphical Method

2x1 – x2 = 3

2.1x1 – x2 = 3

ill conditioned

Graphical Method 方程式斜率非常接近，視覺上無法判定交點

Cramer’s Rule: Determinant

Compute the determinant D ( 行列式 ) 2 x 2 matrix

3 x 3 matrix

211222112221

1211 aaaaaa

222113

232112

232211

333231

232221

131211

Cramer’s Rule

To find xk for the following system

Replace kth column of as with bs (i.e., aik bi )

nnnn22n11n

2nn2222121

1nn1212111

bxa...xaxa

bxa...xaxabxa...xaxa

ijk D(a

matrix newDx

Cramer’s Rule ：例

3x3 matrix

333231

232221

131211

ILL-Conditioned System

What happen if the determinant D is very small or zero?

Divided by zero (linearly dependent system) Divided by a small number: Round-off error Loss of significant digits

0AdetD

Eliminate x2

Subtract to get

2222121

1212111

2122221212112

1222122211122

baxaaxaa

babaxaaxaa

21122211

1212112

21121122

2121221

2121221211211122

Not practical for large number (> 4) of equations

Elimination Method

MATLAB’s Methods

Forward slash ( / ) Back-slash ( \ ) Multiplication by the inverse of the quantity under

the slash

b*Ainvx

bAxbAx

Gauss Elimination

Manipulate equations to remove one of the unknowns Develop algorithm to do this repeatedly The goal is to set up upper triangular matrix

Back substitution to find solution (root)13

n22322

n1131211

Basic Gauss Elimination

Direct method (no iteration required) Forward elimination Column-by-column elimination of the below-

diagonal elements Reduce to upper triangular matrix Back-substitution

Naive Gauss Elimination ( 單純高斯消去法 )

Begin with

Multiply the first equation by a21 / a11 and subtract from second equation

nnnn22n11n

2nn2222121

1nn1212111

bxa...xaxa

nnnn22n11n

212nn1

21n2212

2122111

1nn212111

bxa...xaxa

aa...xa

bxa...xaxa

Forward Elimination

Reduce to

Repeat the forward elimination to get

nnnn22n11n

2nn2222

1nn1212111

bxa...xaxa

bxa...xa

bxa...xaxa

nnnn22n

2nn2222

1nn1212111

bxa...xa

bxa...xaxa

Forward Elimination

First equation is pivot equation ( 軸元方程式 ) a11 is pivot element ( 軸元元素 ) Now multiply second equation by a'32 /a'22 and

subtract from third equation

323nn2

32n3323

2nn2323222

1nn1313212111

bxaxaxa

bxaxaxaxa

Forward Elimination

Repeat the elimination of ai2 and get

Continue and get

nnnn33n

3nn3333

2nn2323222

1nn1313212111

bxaxaxa

bxaxaxaxa

)()( 1nnn

3nn3333

2nn2323222

1nn1313212111

bxaxaxa

bxaxaxaxa

Back Substitution ( 向後代換 )

Now we can perform back substitution to get {x} By simple division

Substitute this into (n-1)th equation

Solve for xn-1

Repeat the process to solve for xn-2 , xn-3 , …. x2, x1

2nn1n1n

2n1n1n bxaxa

Back Substitution

Back substitution: starting with xn

Solve for xn1 , xn2 , … , x3, x2, x1

for i = n1, n2, …, 1

0a 1iii )(

Naive Gauss EliminationNaive Gauss Elimination

4444342

3343332

2242322

114131211

114141

113131

112121

444434241

334333231

224232221

114131211

Elimination of First Column

2242322

114131211

224242

223232

4444342

3343332

2242322

114131211

Elimination of Second Column

Upper triangular matrix

2242322

114131211

33434344443

2242322

114131211

Elimination of Third Column

1141431321211

2242432322

3343433

a/)xaxaxab(x

a/)xaxab(x

a/)xab(x

Upper triangular matrix

0a,a,a,a 44332211

2242322

114131211

Back Substitution

5141020

Example

5141400

5141020

Forward Elimination

3370000

5141420

Upper Triangular Matrix

13/70x3x21x

8/35x2x

4/352x4x

33/707033/x

3370000

Back Substitution

function x = GaussNaive(A,b)% x = GaussNaive(A,b):% Gauss elimination without pivoting.% input:% A = coefficient matrix% b = right hand side vector% output:% x = solution vector[m,n] = size(A);if m~=n, error('Matrix A must be square'); endnb = n+1;Aug = [A b];% forward eliminationfor k = 1:n-1 for i = k+1:n factor = Aug(i,k)/Aug(k,k); Aug(i,k:nb) = Aug(i,k:nb)-factor*Aug(k,k:nb); end disp(Aug);end% back substitutionx = zeros(n,1);x(n) = Aug(n,nb)/Aug(n,n);for i = n-1:-1:1 x(i) = (Aug(i,nb)-Aug(i,i+1:n)*x(i+1:n))/Aug(i,i);end

Print all factor and Aug

(do not suppress output)Eliminate first column

Eliminate second column

Eliminate third column Back-substitution

Aug = [A, b]

>> format short>> x = GaussNaive(A,b)m = 4n = 4Aug = 1 0 2 3 1 -1 2 2 -3 -1 0 1 1 4 2 6 2 2 4 1factor = -1Aug = 1 0 2 3 1 0 2 4 0 0 0 1 1 4 2 6 2 2 4 1factor = 0Aug = 1 0 2 3 1 0 2 4 0 0 0 1 1 4 2 6 2 2 4 1factor = 6Aug = 1 0 2 3 1 0 2 4 0 0 0 1 1 4 2 0 2 -10 -14 -5

factor = 0.5000Aug = 1 0 2 3 1 0 2 4 0 0 0 0 -1 4 2 0 2 -10 -14 -5factor = 1Aug = 1 0 2 3 1 0 2 4 0 0 0 0 -1 4 2 0 0 -14 -14 -5

factor = 14Aug = 1 0 2 3 1 0 2 4 0 0 0 0 -1 4 2 0 0 0 -70 -33

x = 0 0 0 0.4714x = 0 0 -0.1143 0.4714x = 0 0.2286 -0.1143 0.4714x = -0.1857 0.2286 -0.1143 0.4714

Gauss Elimination: Algorithm

Forward elimination for each equation j, j = 1 to n-1

for all equations k greater than j

(a) multiply equation j by akj /ajj

(b) subtract the result from equation k This leads to an upper triangular matrix

Back substitution (a) determine xn from

(b) put xn into (n-1)th equation, solve for xn-1

(c) repeat from (b), moving back to n-2, n-3, etc. until all equations are solved

)1n(nn

)1n(nn a/bx

Operation Count For Gauss Elimination

Elimination routine on the order of O(n3/3) operations

Total operation counts for elimination stage = 2n3/3 + O(n2)

Back-substitution uses O(n2/2)

Total operation counts for back substitution stage = n2 + O(n)

))(())((,

3121nn1n

2knkn1knknn1kk

n2n1n2nn32

1n1nn1nn21

iontion/DivisMultiplica

ubtractionAddition/S

Loop Inner

Loop Outer

Operation Count For Gauss Elimination

#flops (floating-point operations) for Naive Gauss elimination

Computation time increases rapidly with n Most effort incurs in the elimination step Improve efficiency by reducing the elimination effort

859910676106861000000106761000

539866666768155010000671550100

588766780510070510

nEliminatio to

Due Percentage

2nFlops

onSbustituti

nElimination

Partial Pivoting ( 部分軸元法 )

Problems with Gauss elimination division by zero round off errors ill conditioned systems

Use “pivoting” to avoid this Find the row with largest absolute coefficient below the

pivot element Switch rows (“partial pivoting”) Complete pivoting switch columns (rarely used)

Round-off Errors ( 捨入誤差 )

A lot of chopping with more than n3/3 operations More important - error is propagated For large systems (more than 100 equations),

round-off error is important (machine dependent) Ill conditioned systems - small changes in

coefficients lead to large changes in solution Round-off errors are especially important for ill-

conditioned systems

2x1 – x2 = 3

2.1x1 – x2 = 3

Ill-conditioned System

Consider

Since slopes are almost equal

2222121

1212111

1211 Divided by small number

Determinant

Calculate determinant using Gauss elimination

)1n(nn

n22322

n1131211

)(detdet 1nnn332211 aaaaUA

Gauss Elimination with Partial Pivoting

Forward elimination for each equation j, j = 1 to n-1

• first scale each equation k greater than j• then pivot (switch rows)• Now perform the elimination

(a) multiply equation j by akj /ajj

(b) subtract the result from equation

1x4x2x2x6

1x3x2x2x

1 x3x2x

Partial (Row) Pivoting

Interchange rows 1 & 4

5/67/35/31/30

5/67/37/37/30

Forward Elimination

No interchange required

5/72200

33/145000

5/67/37/37/30

5/67/35/31/30

5/67/37/37/30

Forward Elimination

Interchange rows 3 & 4

13/70/6x 2x 2x 41x

8/357/3/x 7/3x 7/35/6x

4/35/2x 25/7x

33/70/533/14x

33/145000

5/72200

5/67/37/37/30

Back-Substitution

function x = GaussPivot(A,b)% x = GaussPivot(A,b):% Gauss elimination without pivoting.

[m,n]=size(A);if m~=n, error('Matrix A must be square'); endnb=n+1;Aug=[A b];% forward eliminationfor k = 1:n-1 % partial pivoting [big,i]=max(abs(Aug(k:n,k))); ipr=i+k-1; if ipr~=k Aug([k,ipr],:)=Aug([ipr,k],:); end for i = k+1:n factor=Aug(i,k)/Aug(k,k); Aug(i,k:nb)=Aug(i,k:nb)-factor*Aug(k,k:nb); end disp(Aug);end% back substitutionx=zeros(n,1);x(n)=Aug(n,nb)/Aug(n,n);for i = n-1:-1:1 x(i)=(Aug(i,nb)-Aug(i,i+1:n)*x(i+1:n))/Aug(i,i);end

Partial pivoting (switch rows)

[big,i] = max(x)

largest element in {x}

index of the largest element

>> format short>> x=GaussPivot0(A,b)Aug = 1 0 2 3 1 -1 2 2 -3 -1 0 1 1 4 2 6 2 2 4 1big = 6i = 4ipr = 4Aug = 6 2 2 4 1 -1 2 2 -3 -1 0 1 1 4 2 1 0 2 3 1factor = -0.1667Aug = 6.0000 2.0000 2.0000 4.0000 1.0000 0 2.3333 2.3333 -2.3333 -0.8333 0 1.0000 1.0000 4.0000 2.0000 1.0000 0 2.0000 3.0000 1.0000factor = 0Aug = 6.0000 2.0000 2.0000 4.0000 1.0000 0 2.3333 2.3333 -2.3333 -0.8333 0 1.0000 1.0000 4.0000 2.0000 1.0000 0 2.0000 3.0000 1.0000factor = 0.1667Aug = 6.0000 2.0000 2.0000 4.0000 1.0000 0 2.3333 2.3333 -2.3333 -0.8333 0 1.0000 1.0000 4.0000 2.0000 0 -0.3333 1.6667 2.3333 0.8333

Aug = [A b]

Interchange rows 1 and 4

Find the first pivot element and its index

Eliminate first columnNo need to interchange

big = 2.3333i = 1ipr = 2factor = 0.4286Aug = 6.0000 2.0000 2.0000 4.0000 1.0000 0 2.3333 2.3333 -2.3333 -0.8333 0 0 0 5.0000 2.3571 0 -0.3333 1.6667 2.3333 0.8333factor = -0.1429Aug = 6.0000 2.0000 2.0000 4.0000 1.0000 0 2.3333 2.3333 -2.3333 -0.8333 0 0 0 5.0000 2.3571 0 0 2.0000 2.0000 0.7143big = 2i = 2ipr = 4Aug = 6.0000 2.0000 2.0000 4.0000 1.0000 0 2.3333 2.3333 -2.3333 -0.8333 0 0 2.0000 2.0000 0.7143 0 0 0 5.0000 2.3571factor = 0Aug = 6.0000 2.0000 2.0000 4.0000 1.0000 0 2.3333 2.3333 -2.3333 -0.8333 0 0 2.0000 2.0000 0.7143 0 0 0 5.0000 2.3571

x = 0 0 0 0.4714x = 0 0 -0.1143 0.4714x = 0 0.2286 -0.1143 0.4714x =

-0.1857 0.2286 -0.1143 0.4714

Back substitutionSecond pivot element and index

Third pivot element and index

Eliminate second column

Eliminate third column

Interchange rows 3 and 4

No need to interchange

Save factors fij for

LU decomposition

Banded Matrix

HBW: Half band width

Banded Matrix

0000000

000000

0000000ai,j= 0if j > i + HBor j < i - HB

HB: Half bandwidthB: Bandwidth

B = 2*HB + 1

In this exampleHB = 1 & B = 3

Tridiagonal Matrix ( 三對角線系統 )

Only three nonzero elements in each equation (3n instead of n2 elements) Subdiagonal, diagonal, superdiagonal

Solve by Gauss elimination

Tridiagonal Matrix ：例

Special case of banded matrix with bandwidth = 3 Save storage, 3 n instead of n n

1n1n1n

Tridiagonal Matrix

Forward elimination

Back substitution

1232n1nk f

Use factor = ek / fk1

to eliminate subdiagonal element

Apply the same matrix operations to right hand side

Tridiagonal Matrix: Hand Calculations

2515000

50251g

(a) Forward elimination (b) Back substitution

function x = Tridiag(e,f,g,r)% x = Tridiag(e,f,g,r):% Tridiagonal system solver.% input:% e = subdiagonal vector% f = diagonal vector% g = superdiagonal vector% r = right hand side vector% output:% x = solution vectorn=length(f);% forward eliminationfor k = 2:n factor = e(k)/f(k-1); f(k) = f(k) - factor*g(k-1); r(k) = r(k) - factor*r(k-1);end% back substitutionx(n) = r(n)/f(n);for k =n-1:-1:1 x(k) = (r(k)-g(k)*x(k+1))/f(k);end

» [e,f,g,r] = example

e = 0 -2.0000 4.0000 -0.5000 1.5000 -3.0000f = 1.0000 6.0000 9.0000 3.2500 1.7500 13.0000g = -2.0000 4.0000 -0.5000 1.5000 -3.0000 0r = -3.0000 22.0000 35.5000 -7.7500 4.0000 -33.0000

» x = Tridiag (e, f, g, r)x = 1 2 3 -1 -2 -3

function [e,f,g,r] = example

e=[ 0 -2 4 -0.5 1.5 -3];

f=[ 1 6 9 3.25 1.75 13];

g=[-2 4 -0.5 1.5 -3 0];

r=[-3 22 35.5 -7.75 4 -33];

375.15.1

5.125.35.0

Note: e(1) = 0 and g(n) = 0

Tridiagonal Matrix: 運算例

補充： Big-O

Concept of Order of Growth

We say fA(n)=30n+8 is (at most) order n, or O(n)

It is, at most, roughly proportional to n fB(n)=n2+1 is order n2, or O(n2)

It is (at most) roughly proportional to n2

Any function whose exact (tightest) order is O(n2) is faster-growing than any O(n) function Later we will introduce Θ for expressing exact order

Definition: O(g) (Asymptotic Upper Bound)

“f is at most order g”, or “f is O(g)”, or “f = O(g)” all just mean that fO(g)

Let g be any function RR. Define “at most order g”, written O(g), to be:

{f:RR | c,k: x>k: f(x) cg(x)} “Beyond some point k, function f is at most a constant c

times g (i.e., proportional to g)”

About the Definition O(g)

Big-O 範例 (1/3)

Big-O 範例 (2/3)

Big-O 範例 (3/3)

Show that 30n+8 is O(n) To show c,k: n > k: 30n+8 cn

• Let c = 31, k = 8. Assume n > k = 8. Thencn = 31n = 30n + n > 30n+8, so 30n+8 < cn

Show that n2+1 is O(n2) To show c,k: n > k: n2+1 cn2

• Let c = 2, k = 1. Assume n > 1. Then cn2 = 2n2 = n2+n2 > n2+1, or n2+1< cn2

Note 30n+8 isn’t less than n anywhere (n>0) It isn’t even less than 31n everywhere But it is less than 31n everywhere to the right of n=8

Big-O Example

Increasing n

30n+8cn =31n

30n+8 O(n)

應知的定理

定理應用範例

Summary

For any g:RR, “at most order g”,O(g) {f:RR | c,k x > k |f(x)| |cg(x)| } Often, we deal only with positive functions and can

ignore absolute value symbols

“fO(g)” often written “f is O(g)” or “f = O(g)” The latter form is an instance of a more general

convention...

(g) {f:RR | c,k x > k, |f(x)| > |cg(x)|}

Remarks O(.) gives worst-case guarantees (good news), while

gives a lower bound (bad news)

的定義 (Asymptotic Lower Bound)

Definition: (g), Exactly Order g (Asymptotic Tight Bound)

If fO(g) and gO(f ), then we say “g and f are of the same order” or “f is (exactly) order g” and write f(g)

Another equivalent definition:(g) {f:RR | c1c2k > 0 x > k: |c1g(x)| |f(x)| |c2g(x)| } “Everywhere beyond some point k, f(x) lies in between

two multiples of g(x)”

1 gauss elimination small matrices for small numbers of equations, solvable by hand graphical ...

n factor

1x1 x2

92x1 x2

3x1 x2

32x1 x2

solutiongraphical method2x1

function x

equation a11

Documents

sitting in judgment, myron c. cramer's experiences in … in...

waste elimination orientation video. waste elimination

hilltop plantfest! 2019 · hilltop plantfest! 2019 plant...

cramer's rule - math with mrs. brown€¦ · in linear...

jim cramer's 'mad money' charitable trust performance and...

mthsc 3110 section 3.3 cramer's rule

cramer's united states, spelling book : with appropriate

cramer's rule and loop ensembles 1. introduction the

secured options - cramer's 25 rules for investing

clayton cramer's home page

karydis, dimitris__beethoven's annotations to cramer's...

bowel elimination si.ppt [read-only] -...

cramer's rule

study of an rna helicase implicates small rna–noncoding...

cramer's...

national pollutant discharge elimination · 2011-01-21 ·...

chapter 9 gauss elimination z9.1 solving small numbers of...

gaussian elimination

cramer's rule gabriel cramer was a swiss mathematician...

rpah elimination diet handbook - emerge...