g aussian e limination presented by: max le max le justin lavorgna data structures ii: algorithm...
TRANSCRIPT
GGAUSSIANAUSSIAN EELIMINATIONLIMINATION
Presented by:Presented by:
Max LeMax Le
Justin LavorgnaJustin Lavorgna
Data Structures II: Data Structures II:
Algorithm Design & Analysis Algorithm Design & Analysis
Gaussian Gaussian Elimination ItineraryElimination Itinerary
Introduction:Introduction:
Who was Carl Friedrich Gauss ?Who was Carl Friedrich Gauss ?What is Gaussian Elimination ?What is Gaussian Elimination ?Some Basic Terminology ?Some Basic Terminology ?
Main Presentation:Main Presentation:
Backward SubstitutionBackward Substitution method methodLULU method methodCompute Compute DeterminantDeterminant of a matrix of a matrix
Conclusion:Conclusion:
Which Algorithm is more efficient ?Which Algorithm is more efficient ?Which Algorithm is more practical ?Which Algorithm is more practical ?
CARL FRIEDRICH CARL FRIEDRICH GAUSSGAUSS
Who Was He?Who Was He?
Born: April 30Born: April 30thth, 1777 Brunswick, 1777 Brunswick
Died: February 23Died: February 23rdrd,1822 Göttingen,1822 Göttingen
One of the all time great German One of the all time great German Mathematicians. His field of study Mathematicians. His field of study consisted of most every aspect in consisted of most every aspect in mathematics today. mathematics today.
Gauss’s work contributed to a variety of Gauss’s work contributed to a variety of different aspects such as:different aspects such as:
Gaussian Elimination (Linear Algebra) Gaussian Elimination (Linear Algebra) Gaussian Primes & Gauss Sums (Number Gaussian Primes & Gauss Sums (Number
Theory)Theory) Gaussian Distribution (Statistics)Gaussian Distribution (Statistics) Gauss (Electromagnetism)Gauss (Electromagnetism) Gaussian Curvature & Gauss-Bonnet Formula Gaussian Curvature & Gauss-Bonnet Formula
(Differential Geometry)(Differential Geometry) Gaussian Quadrature (Numerical Analysis)Gaussian Quadrature (Numerical Analysis) Gauss’s Identity (Hypergeometric Functions)Gauss’s Identity (Hypergeometric Functions)
GAUSSIAN GAUSSIAN ELIMINATIONELIMINATION
What is it?What is it?
For complex systems of equations where:For complex systems of equations where: The number of equations are equal to The number of equations are equal to nn The number of unknowns are equal to The number of unknowns are equal to nn
We must solve by a process of elimination. We must solve by a process of elimination.
EliminationElimination is implied by reducing is implied by reducing the amount of unknowns and the amount of unknowns and
equations in the system.equations in the system.
1. Subtract multiples of the first 1. Subtract multiples of the first equation from all other equations equation from all other equations
The elimination The elimination process:process:
2.2. The goal is to eliminate the first The goal is to eliminate the first variables in each equation.variables in each equation.
a11 a12 a13a11 a12 a13
a21 a22 a23a21 a22 a23
a31 a32 a33a31 a32 a33
u11 u12 u13u11 u12 u13
0 u22 u230 u22 u23
0 0 u330 0 u33
A=A=
U=U=
TERMINOLOGYTERMINOLOGY
The above conditions are what we call a The above conditions are what we call a linear linear equationequation. If . If h = 0h = 0 then the linear equation is then the linear equation is said to be said to be homogeneoushomogeneous. All linear equations . All linear equations make up a make up a linear systemlinear system while all homogenous while all homogenous linear equations make up a linear equations make up a homogenous homogenous linear systemlinear system..
Equation:Equation: ax + by + cz + dw = hax + by + cz + dw = h where:where: aa,,bb,,cc,,dd,, and and hh are known are known
numbersnumbers
while:while: xx,,yy,,zz,, and and ww are unknown are unknown numbersnumbers
HOW IT WORKSHOW IT WORKS
Given System:Given System:
x + y + z = 0x + y + z = 0
x - 2y + 2z = 4x - 2y + 2z = 4
Equation 1Equation 1
x + 2y - z = 2x + 2y - z = 2 Equation 3Equation 3
Equation 2Equation 2
Make system into a matrix:Make system into a matrix:
x + y + z | 0x + y + z | 0 x - 2y + 2z | 4x - 2y + 2z | 4
Equation 1Equation 1
x + 2y - z | 2x + 2y - z | 2 Equation 3Equation 3
Equation 2Equation 2
Objective:Objective:
Kill ‘ Kill ‘ XX ’ Variable in Equation 2 ’ Variable in Equation 2
x + y + z | 0x + y + z | 0
xx - 2y + 2z | 4 - 2y + 2z | 4
Equation 1Equation 1
x + 2y - z | 2x + 2y - z | 2 Equation 3Equation 3
Equation 2Equation 2
1x +1y + 1z | 01x +1y + 1z | 0
1x - 2y + 2z | 41x - 2y + 2z | 4Multiple: 1Multiple: 11x + 1y - 1z | 01x + 1y - 1z | 0
-- ____________________ - 3y + z | 4- 3y + z | 4 New Equation 2New Equation 2
Equation 1Equation 1
Equation 2Equation 2
MatrixMatrix
1x + 2y - 1z | 2 1x + 2y - 1z | 2 Equation 3Equation 3
Step 1. Observe first column from left of the Step 1. Observe first column from left of the matrix. matrix.
Step 2. If leading term is any number other than 1, Step 2. If leading term is any number other than 1, multiply the row by its reciprocal to obtain a 1. multiply the row by its reciprocal to obtain a 1.
1x1x
Step 3. Subtract first row from all other rows until a Step 3. Subtract first row from all other rows until a zero appears below the leading column.zero appears below the leading column.
‘‘X’ unknown is DEAD…X’ unknown is DEAD…
Equation 1Equation 1
x + y + z | 0x + y + z | 0
xx - 2y + 2z | 4 - 2y + 2z | 4
x + 2y - z | 2x + 2y - z | 2
Objective:Objective:
Kill ‘ Kill ‘ XX ’ Variable in Equation 3 ’ Variable in Equation 3
x + y + z | 0x + y + z | 0
xx - 2y + 2z | 4 - 2y + 2z | 4
Equation 1Equation 1
xx + 2y - z | 2 + 2y - z | 2 Equation 3Equation 3
Equation 2Equation 2
1x +1y + 1z | 01x +1y + 1z | 0
1x + 2y - 1z | 21x + 2y - 1z | 2
Multiple: 1Multiple: 11x + 1y - 1z | 01x + 1y - 1z | 0
-- ____________________ y - 2z | 2y - 2z | 2 New Equation 3New Equation 3
Equation 1Equation 1
Equation 3Equation 3
MatrixMatrix
- 3y + 1z | 4- 3y + 1z | 4 Equation 2Equation 2
Step 4. Observe first column from left of the Step 4. Observe first column from left of the matrix. matrix.
Step 5. If leading term is any number other than 1, Step 5. If leading term is any number other than 1, multiply the row by its reciprocal to obtain a 1. multiply the row by its reciprocal to obtain a 1.
1x1x
Step 6. Subtract first row from all other rows until a Step 6. Subtract first row from all other rows until a zero appears below the leading column.zero appears below the leading column.
‘‘X’ unknown is DEAD…X’ unknown is DEAD…
Equation 1Equation 1 x + y + z | 0x + y + z | 0
xx - 2y + 2z | 4 - 2y + 2z | 4
xx + 2y - z | 2 + 2y - z | 2
Objective:Objective:
Kill ‘ Kill ‘ YY ’ Variable in ’ Variable in EquationEquation 3 3
x + y + z | 0x + y + z | 0
xx - 2y + 2z | 4 - 2y + 2z | 4
Equation 1Equation 1
x + x + 22yy - z | 2 - z | 2 Equation 3Equation 3
Equation 2Equation 2
1y - 2z | 21y - 2z | 2
- 3y + 1z | 4- 3y + 1z | 4Multiple: 3Multiple: 33y - 6z | 63y - 6z | 6
--____________________ - 5z | 10- 5z | 10 New Equation 3New Equation 3
Equation 3Equation 3
Equation 2Equation 2
MatrixMatrix
1x +1y + 1z | 01x +1y + 1z | 0 Equation 1Equation 1
Step 7. Observe first column from left of the Step 7. Observe first column from left of the matrix. matrix.
Step 8. If leading term is any number other than 1, Step 8. If leading term is any number other than 1, multiply the row by its reciprocal to obtain a 1. multiply the row by its reciprocal to obtain a 1.
1y1y
Step 9. Add second row to the third row until a Step 9. Add second row to the third row until a zero appears below the leading column.zero appears below the leading column.
‘‘Y’ unknown is DEAD…Y’ unknown is DEAD…
x + y + z | 0x + y + z | 0
xx - 2y + 2z | 4 - 2y + 2z | 4
x + x + 2y2y - z | 2 - z | 2
Objective:Objective:
Solve for Unknowns using Solve for Unknowns using Backward SubstitutionBackward Substitution
BACKWARDBACKWARD
SUBSTITUTIONSUBSTITUTION
Linear System Solution:Linear System Solution:
Final matrix:Final matrix:
x + y + z | 0x + y + z | 0 - 3y + z | 4- 3y + z | 4
Equation 1Equation 1
- 5z | 10- 5z | 10 Equation 3Equation 3
Equation 2Equation 2 U =U =
Solving for z in equation 3:Solving for z in equation 3:
- 5 10- 5 10 - 5 - 5- 5 - 5___ ______ ___-2-2z =z =
Solving for y in equation 2:Solving for y in equation 2:
- 3 4- 3 4
- 3 - 3- 3 - 3___ _________ ______= -2= -2
y y ( + 2)( + 2)==+ (-2)+ (-2)
Solving for x in equation 1:Solving for x in equation 1:
00
1 11 1___ ____________ _________ = 4= 4 ( + 2 + 2)( + 2 + 2)==+ (-2) + (-2)+ (-2) + (-2)
x x
Objective:Objective:
Solve using LU DecompositionSolve using LU Decomposition
LU DECOMPOSITIONLU DECOMPOSITION
LU DecompositionLU Decomposition
Information on LU MethodInformation on LU MethodExampleExample
LU DecompositionLU Decomposition
Given a system A*x = bGiven a system A*x = b
Solving the system Lp*y=b to solve for y. Solving the system Lp*y=b to solve for y.
We want to find Lp and U matrices such We want to find Lp and U matrices such that Lp*U=Athat Lp*U=A
Then solving the system U*x=y to get xThen solving the system U*x=y to get x
OBJECTIVE:OBJECTIVE:
Given a system A*x = bGiven a system A*x = b
LU DecompositionLU DecompositionConsider the system:Consider the system:
2x + 2y + 1z2x + 2y + 1z 2x + 3y - 2z2x + 3y - 2z
4x + 1y - 2z4x + 1y - 2z
Equation in matrix form for A:Equation in matrix form for A:
2 2 12 2 1
2 3 -22 3 -2
4 1 -24 1 -2
A A
11
11 11
xx
zz yy
XX
==
bb
11
11 11
==
==
==
==
Identity MatrixIdentity Matrix
Identity Matrix is a matrix with all 1 in its Identity Matrix is a matrix with all 1 in its diagonal and zeros elsewherediagonal and zeros elsewhere
11 0 0 0 0
0 0 11 0 0 I =I =
0 0 0 0 11
Find the U matrix and keep track of Elementary Find the U matrix and keep track of Elementary matricesmatrices
First Operation:First Operation:
Elementary operation equivalent:Elementary operation equivalent:
2 2 12 2 1 2 3 -22 3 -2 4 1 -24 1 -2
2 2 12 2 1 0 1 -30 1 -3 4 1 -24 1 -2
Multiply row 1 by -1Multiply row 1 by -1
Add row 1 to row 2Add row 1 to row 2
1 0 01 0 0 0 1 00 1 0 0 0 10 0 1
Multiply row 1 by -1Multiply row 1 by -1
Add row 1 to row 2Add row 1 to row 2
1 0 01 0 0 -1 1 0-1 1 0 0 0 10 0 1
= E1= E1
Find the U matrix and keep track of Elementary Find the U matrix and keep track of Elementary matricesmatrices
Second Operation:Second Operation:
Elementary operation equivalent:Elementary operation equivalent:
2 2 12 2 1 0 1 -30 1 -3 4 1 -24 1 -2
2 2 12 2 1 0 1 -30 1 -3 0 -3 -40 -3 -4
Multiply row 1 by -2Multiply row 1 by -2
Add row 1 to row 3Add row 1 to row 3
1 0 01 0 0 0 1 00 1 0 0 0 10 0 1
Multiply row 1 by -2Multiply row 1 by -2
Add row 1 to row 3Add row 1 to row 3
1 0 01 0 0 0 1 00 1 0 -2 0 1-2 0 1
= E2= E2
Find the U matrix and keep track of Elementary Find the U matrix and keep track of Elementary matricesmatrices
Third Operation:Third Operation:
Elementary operation equivalent:Elementary operation equivalent:
2 2 12 2 1 0 1 -30 1 -3 0 -3 -40 -3 -4
2 2 12 2 1 0 1 -30 1 -3 0 0 -130 0 -13
Multiply row 2 by 3Multiply row 2 by 3
Add row 2 to row 3Add row 2 to row 3
1 0 01 0 0 0 1 00 1 0 0 0 10 0 1
Multiply row 2 by 3Multiply row 2 by 3
Add row 2 to row 3Add row 2 to row 3
1 0 01 0 0 0 1 00 1 0 0 3 10 3 1
= E3= E3
= U= U
1 0 01 0 0 -1 1 0-1 1 0 0 0 10 0 1
E1 =E1 = E2 =E2 = 1 0 01 0 0 0 1 00 1 0 -2 0 1-2 0 1
E2 =E2 = 1 0 01 0 0 0 1 00 1 0 0 3 10 3 1
1 0 01 0 0 -1 1 0-1 1 0 0 0 10 0 1
1 0 01 0 0 0 1 00 1 0 -2 0 1-2 0 1
1 0 01 0 0 0 1 00 1 0 0 3 10 3 1
S =S =
S = E1*E2*E3S = E1*E2*E3
Forming the Lp MatrixForming the Lp Matrix
Forming the Lp MatrixForming the Lp MatrixProduct of the elementary matricesProduct of the elementary matrices
1 0 01 0 0 -1 1 0-1 1 0 -5 3 1-5 3 1
S =S =
Inverse of a S will give LpInverse of a S will give Lp
1 0 01 0 0 1 1 01 1 0 2 -3 12 -3 1
1/S =1/S = = Lp= Lp
OBJECTIVE:OBJECTIVE:We want to find Lp and U We want to find Lp and U
matrices such that Lp*U=Amatrices such that Lp*U=A
LU DecompositionLU Decomposition
We can verify that Lp*U = AWe can verify that Lp*U = A
= A= A 1 0 01 0 0 -1 1 0-1 1 0
2 -3 12 -3 1
2 2 12 2 1 0 1 -30 1 -3 0 0 -130 0 -13
LpLp UU
== 2 2 12 2 1 2 3 -22 3 -2 4 1 -24 1 -2
LU DecompositionLU Decomposition
Recall that Upper Triangular Matrix:Recall that Upper Triangular Matrix:
2 2 12 2 1 0 1 -30 1 -3 0 0 -130 0 -13
U =U =
Lp =Lp =
Recall that Lower Triangular Matrix:Recall that Lower Triangular Matrix:
1 0 01 0 0 1 1 01 1 0 2 -3 12 -3 1
OBJECTIVE:OBJECTIVE:
Solving the system Lp*y=b to Solving the system Lp*y=b to solve for y. solve for y.
Solve Lp*y = bSolve Lp*y = bThe system:The system:
Solve for y1, y2 and y3Solve for y1, y2 and y3
y1 = 1y1 = 1
y1+y2 = 1y1+y2 = 1
2*y1-3*y2 + y3 = 12*y1-3*y2 + y3 = 1
bb
11
11 11
y1y1
y3y3 y2y2
YY ==
==
LpLp
=> => y2 = 0y2 = 0 => => y3 = -1y3 = -1
1 0 01 0 0 1 1 01 1 0 2 -3 12 -3 1
OBJECTIVE:OBJECTIVE:
Then solving the system U*x=y Then solving the system U*x=y to get xto get x
Solve U*x = ySolve U*x = yThe system :The system :
Solve for x1, x2 and x3Solve for x1, x2 and x3
-13*X3 = -1 -13*X3 = -1
X2 – 3*X3 = 1X2 – 3*X3 = 1
2*X1+2*X2 + X3 = 12*X1+2*X2 + X3 = 1
YY
11
-1-1 00
X1X1
X3X3 X2X2
XX ==
== 2 2 12 2 1 0 1 -30 1 -3 0 0 -130 0 -13
UU
=> => X3 = 1/13X3 = 1/13
=> => X2 = 3/13X2 = 3/13
=> => X1 = 3/13X1 = 3/13
Objective:Objective:
Solve for linear system Solve for linear system using Determinant of Matrixusing Determinant of Matrix
DETERMINANT of DETERMINANT of MATRIXMATRIX
Make equations into a matrix:Make equations into a matrix:
1 + 1 + 11 + 1 + 11 - 2 + 21 - 2 + 2 1 + 2 - 11 + 2 - 1
Given Equations:Given Equations:
x + y + z = 0x + y + z = 0x - 2y + 2z = 4x - 2y + 2z = 4
Equation 1Equation 1
x + 2y - z = 2x + 2y - z = 2 Equation 3Equation 3
Equation 2Equation 2
How Determinant works:How Determinant works:
aa1111 a a1212
a a2121 a a2222 ==detdet
aa1111-a-a2222 – a – a1212aa2121
det A =det A = ΣΣ ssjjaa11jj det A det Ajj
aa11 11 det [ adet [ a2222 ] – a ] – a1212 det [ a det [ a2121 ] = a ] = a1111-a-a2222 – a – a1212aa2121
aa11 11 det [ adet [ a2222 ] – a ] – a1212 det [ a det [ a2121 ] = ] =
nn
jj = 1 = 1Formula:Formula:
Matrix:Matrix:
-2-2 22 -1-122
1111 111111 -2 2 -2 2 2 -1 2 -1
== detdet detdet-- detdet++
2 2
-1 -1
1 1
1 1
1 -2 1 -2 1 2 1 2
11 11 11
-2 2 -2 2 2 -1 2 -1detdet detdet-- detdet++
2 2
-1 -1
1 1 1 1
1 -2 1 -2 1 21 2
== 1 ( 2 – 4 ) – 1 ( -1 – 2 ) + 1 (2 – (-2) )1 ( 2 – 4 ) – 1 ( -1 – 2 ) + 1 (2 – (-2) )
== -2 + 3 + 4-2 + 3 + 4
== 55detdet
detdet
detdet
CONCLUSIONCONCLUSION
Time ComplexityTime Complexity Backward substitution O(n^3)Backward substitution O(n^3) LU method O(n^3)LU method O(n^3)
Space ComplexitySpace Complexity Backward substitution O(n^2)Backward substitution O(n^2) LU method O(n^2)LU method O(n^2)
ApplicationApplication
Gaussian Elimination plays a major role Gaussian Elimination plays a major role in solving Linear systems, including:in solving Linear systems, including:- Digital Signal processingDigital Signal processing- Linear system analysisLinear system analysis- Image processing.Image processing.- Etc…Etc…