chapter 7 numerical methods for the solution of systems of equations
DESCRIPTION
Chapter 7 Numerical Methods for the Solution of Systems of Equations. Introduction. This chapter is about the techniques for solving linear and nonlinear systems of equations. Two important problems from linear algebra: The linear systems problem: The nonlinear systems problem:. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/1.jpg)
1
Chapter 7Numerical Methods for the Solution of
Systems of Equations
![Page 2: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/2.jpg)
2
Introduction
This chapter is about the techniques for solving linear and nonlinear systems of equations.
Two important problems from linear algebra:– The linear systems problem:
– The nonlinear systems problem:
![Page 3: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/3.jpg)
3
7.1 Linear Algebra Review
![Page 4: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/4.jpg)
4
![Page 5: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/5.jpg)
5
Theorem 7.1 and Corollary 7.1
Singular v.s. nonsingular
![Page 6: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/6.jpg)
6
Tridiagonal Matrices
Upper triangular:
Lower triangular:
Symmetric matrices, positive definite matrices…… The concepts of independence/dependence, spanning, b
asis, vector space/subspace, dimension, and orthogonal/orthonormal should review……
![Page 7: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/7.jpg)
7
7.2 Linear Systems and Gaussian Elimination
In Section 2.6, the linear system can be written as a single augmented matrix:
Elementary row operations to solve the linear system problems:
Row equivalent: if we can manipulate from one matrix to another using only elementary row operations, then the two matrices are said to be row equivalent.
![Page 8: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/8.jpg)
8
Theorem 7.2
![Page 9: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/9.jpg)
9
Example 7.1
![Page 10: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/10.jpg)
10
Example 7.1 (con.)
![Page 11: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/11.jpg)
11
Partial Pivoting
![Page 12: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/12.jpg)
12
The Problem of Naive Gaussian Elimination
The problem of naive Gaussian elimination is the potential division by a zero pivot.
For example: consider the following system
The exact solution:
What happens when we solve this system using the naive algorithm and the pivoting algorithm?
![Page 13: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/13.jpg)
13
Discussion
Using the naive algorithm:
Using the pivoting algorithm:
correct
incorrect
![Page 14: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/14.jpg)
14
7.3 Operation Counts
You can trace Algorithms 7.1 and 7.2 to evaluate the computational time.
![Page 15: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/15.jpg)
15
7.4 The LU Factorization
Our goal in this section is to develop a matrix factorization that allows us save the work from the elimination step.
Why don’t we just compute A-1 (to check if A is nonsingular)?– The answer is that it is not cost-effective to do so.– The total cost is (Exercise 7)
What we will do is show that we can factor the matrix A into the product of a lower triangular and an upper triangular matrix:
![Page 16: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/16.jpg)
16
The LU Factorization
![Page 17: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/17.jpg)
17
Example 7.2
![Page 18: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/18.jpg)
18
Example 7.2 (con.)
![Page 19: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/19.jpg)
19
The Computational Cost The total cost of the above process:
If we already have done the factorization, then the cost of the two solution steps:
Constructing the LU factorization is surprisingly easy.– The LU factorization is nothing more than a very slight
reorganization of the same Gaussian elimination algorithm we studied earlier in this chapter.
![Page 20: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/20.jpg)
20
The LU Factorization : Algorithms 7.5 and 7.6
![Page 21: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/21.jpg)
21
![Page 22: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/22.jpg)
22
![Page 23: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/23.jpg)
23
Example 7.3
![Page 24: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/24.jpg)
24
Example 7.3 (con.)
L
U
![Page 25: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/25.jpg)
25
Pivoting and the LU Decomposition
Can we pivoting in the LU decomposition without destroying the algorithm?– Because of the triangular structure of the LU factors,
we can implement pivoting almost exactly as we did before.
– The difference is that we must keep track of how the rows are interchanged in order to properly apply the forward and backward solution steps.
![Page 26: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/26.jpg)
26
Example 7.4
Next page
![Page 27: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/27.jpg)
27
Example 7.4 (con.)
We need to keeptrack of the row interchanges.
![Page 28: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/28.jpg)
28
Discussion
How to deep track of the row interchanges?– Using an index array
– For example: In Example 7.4, the final version of J is
you can check that this is correct.
![Page 29: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/29.jpg)
29
7.5 Perturbation, Conditioning, and Stability
Example 7.5
![Page 30: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/30.jpg)
30
7.5.1 Vector and Matrix Norms
For example:– Infinity norm:
– Euclidean 2-norm:
![Page 31: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/31.jpg)
31
Matrix Norm
The properties of matrix norm: (1) (2) For example:
– The matrix infinity norm:
– The matrix 2-norm:
![Page 32: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/32.jpg)
32
Example 7.6
0
14211
25622
112217
![Page 33: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/33.jpg)
33
7.5.2 The Condition Number and Perturbations
Note thatCondition number
![Page 34: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/34.jpg)
34
Definition 7.3 and Theorem 7.3
![Page 35: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/35.jpg)
35
AA-1= I
![Page 36: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/36.jpg)
36
Theorem 7.4
![Page 37: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/37.jpg)
37
Theorems 7.5 and 7.6
![Page 38: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/38.jpg)
38
Theorem 7.7
![Page 39: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/39.jpg)
39
Definition 7.4
An example: Example 7.7
![Page 40: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/40.jpg)
40
![Page 41: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/41.jpg)
41
Theorem 7.9
![Page 42: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/42.jpg)
42
Discussion Is Gaussian elimination with partial pivoting a stable process?
– For a sufficiently accurate computer (u small enough) and a sufficiently small problem (n small enough), then Gaussian elimination with partial pivoting will produce solutions that are stable and accurate.
![Page 43: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/43.jpg)
43
7.5.3 Estimating the Condition Number
Singular matrices are perhaps something of a rarity, and all singular matrices are arbitrarily close to a nonsingular matrix.
If the solution to a linear system changes a great deal when the problem changes only very slightly, then we suspect that the matrix is ill conditioned (nearly singular).
The condition number is an important indicator to find the ill conditioned matrix.
![Page 44: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/44.jpg)
44
Estimating the Condition Number
Estimate the condition number
![Page 45: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/45.jpg)
45
Example 7.8
![Page 46: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/46.jpg)
46
7.5.4 Iterative Refinement Since Gaussian elimination can be adversely affected by rounding
error, especially if the matrix is ill condition. Iterative refinement (iterative improvement) algorithm can use to
improve the accuracy of a computed solution.
![Page 47: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/47.jpg)
47
Example 7.9
![Page 48: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/48.jpg)
48
Example 7.9 (con.)
compare
![Page 49: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/49.jpg)
49
7.6 SPD Matrices and The Cholesky Decomposition
SPD matrices: symmetric, positive definite matrices
You can prove this theorem using induction method.
![Page 50: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/50.jpg)
50
The Cholesky Decomposition
There are a number of different ways of actually constructing the Cholesky decomposition.
All of these constructions are equivalent, because the Cholesky factorization is unique.
One common scheme uses the following formulas:
This is a very efficient algorithm. You can read Section 9.22 to learn more about Cholesky
method.
n
![Page 51: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/51.jpg)
51
7.7 Iterative Method for Linear Systems: a Brief Survey
If the coefficient matrix is a very large and sparse, then Gaussian elimination may not be the best way to solve the linear system problem.
Why? – Even though A=LU is sparse, the individual factors L and U may not be
as sparse as A.
![Page 52: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/52.jpg)
52
Example 7.10
![Page 53: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/53.jpg)
53
Example 7.10 (con.)
![Page 54: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/54.jpg)
54
Splitting Methods (details see Chapter 9)
![Page 55: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/55.jpg)
55
Theorem 7.13
![Page 56: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/56.jpg)
56
Definition 7.6
![Page 57: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/57.jpg)
57
Theorem 7.14
Conclusion:
![Page 58: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/58.jpg)
58
Example of Splitting Methods--Jacobi Iteration
Jacobi iteration:
In this method, matrix M = D.
![Page 59: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/59.jpg)
59
Example 7.12
![Page 60: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/60.jpg)
60
Example 7.12 (con.)
![Page 61: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/61.jpg)
61
Example of Splitting Methods--Gauss-Seidel Iteration
Gauss-Seidel Iteration :
In this method, matrix M = L.
![Page 62: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/62.jpg)
62
Example 7.13
![Page 63: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/63.jpg)
63
Theorem 7.15
![Page 64: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/64.jpg)
64
Example of Splitting Methods--SOR Iteration
SOR: successive over-relaxation iteration
![Page 65: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/65.jpg)
65
Example 7.14
![Page 66: Chapter 7 Numerical Methods for the Solution of Systems of Equations](https://reader035.vdocuments.net/reader035/viewer/2022062314/56812bd1550346895d9032a7/html5/thumbnails/66.jpg)
66
Theorem 7.16