numerical analysis
DESCRIPTION
Numerical Analysis. EE, NCKU Tien-Hao Chang (Darby Chang). In the previous slide. Rootfinding multiplicity Bisection method Intermediate Value Theorem convergence measures False position yet another simple enclosure method advantage and disadvantage in comparison with bisection method. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/1.jpg)
Numerical Analysis
1
EE, NCKUTien-Hao Chang (Darby Chang)
![Page 2: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/2.jpg)
In the previous slide Rootfinding
– multiplicity
Bisection method– Intermediate Value Theorem
– convergence measures
False position– yet another simple enclosure method
– advantage and disadvantage in comparison with bisection method
2
![Page 3: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/3.jpg)
In this slide Fixed point iteration scheme
– what is a fixed point?
– iteration function
– convergence
Newton’s method– tangent line approximation
– convergence
Secant method3
![Page 4: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/4.jpg)
Rootfinding Simple enclosure
– Intermediate Value Theorem
– guarantee to converge• convergence rate is slow
– bisection and false position
Fixed point iteration– Mean Value Theorem
– rapid convergence• loss of guaranteed convergence
4
![Page 5: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/5.jpg)
2.3
5
Fixed Point Iteration Schemes
![Page 6: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/6.jpg)
6
![Page 7: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/7.jpg)
7
There is at least one point on the graph at which the tangent lines is parallel to the secant line
![Page 8: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/8.jpg)
Mean Value Theorem
We use a slightly different formulation
An example of using this theorem– proof the inequality
8
![Page 9: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/9.jpg)
9
![Page 10: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/10.jpg)
Fixed points Consider the function
– thought of as moving the input value of to the output value
– the sine function maps to • the sine function fixes the location of
– is said to be a fixed point of the function
10
![Page 11: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/11.jpg)
11
![Page 12: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/12.jpg)
Number of fixed points According to the previous figure, a
trivial question is– how many fixed points of a given
function?
12
![Page 13: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/13.jpg)
13
![Page 14: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/14.jpg)
14
𝑔 ′ (𝑥 )≤𝑘<1
![Page 15: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/15.jpg)
Only sufficient conditions
Namely, not necessary conditions– it is possible for a function to violate one or more of the
hypotheses, yet still have a (possibly unique) fixed point
15
![Page 16: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/16.jpg)
Fixed point iteration
16
![Page 17: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/17.jpg)
Fixed point iteration If it is known that a function has a
fixed point, one way to approximate the value of that fixed point is fixed point iteration scheme
These can be defined as follows:
17
![Page 18: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/18.jpg)
18
In action
http://thomashawk.com/hello/209/1017/1024/Jackson%20Running.jpg
![Page 19: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/19.jpg)
19
![Page 20: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/20.jpg)
20
![Page 21: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/21.jpg)
Any Questions?
21
About fixed point iteration
![Page 22: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/22.jpg)
Relation to rootfinding Now we know what fixed point
iteration is, but how to apply it on rootfinding?
More precisely, given a rootfinding equation, f(x)=x3+x2-3x-3=0, what is its iteration function g(x)?
22
hint
![Page 23: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/23.jpg)
Iteration function Algebraically transform to the form
– …
Every rootfinding problem can be transformed into any number of fixed point problems– (fortunately or unfortunately?)
23
![Page 24: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/24.jpg)
24
![Page 25: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/25.jpg)
25
In action
http://thomashawk.com/hello/209/1017/1024/Jackson%20Running.jpg
![Page 26: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/26.jpg)
26
![Page 27: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/27.jpg)
Analysis #1 iteration function converges
– but to a fixed point outside the interval
#2 fails to converge– despite attaining values quite close to #1
#3 and #5 converge rapidly– #3 add one correct decimal every iteration
– #5 doubles correct decimals every iteration
#4 converges, but very slow
27
![Page 28: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/28.jpg)
Convergence This analysis suggests a trivial question
– the fixed point of is justified in our previous theorem
28
![Page 29: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/29.jpg)
29
![Page 30: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/30.jpg)
30
![Page 31: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/31.jpg)
31
![Page 32: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/32.jpg)
𝑘 demonstrates the importance of the
parameter – when , rapid
– when , dramatically slow
– when , roughly the same as the bisection method
32
![Page 33: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/33.jpg)
Order of convergence of fixed point iteration schemes
33
All about the derivatives,
![Page 34: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/34.jpg)
34
![Page 35: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/35.jpg)
35
![Page 36: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/36.jpg)
36
![Page 37: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/37.jpg)
37
![Page 38: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/38.jpg)
38
![Page 39: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/39.jpg)
Stopping condition
39
![Page 40: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/40.jpg)
40
![Page 41: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/41.jpg)
Two steps
41
![Page 42: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/42.jpg)
The first step
42
![Page 43: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/43.jpg)
The second step
43
![Page 44: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/44.jpg)
Any Questions?
44
2.3 Fixed Point Iteration Schemes
![Page 45: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/45.jpg)
2.4
45
Newton’s Method
![Page 46: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/46.jpg)
46
![Page 47: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/47.jpg)
47
![Page 48: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/48.jpg)
Newton’s Method
Definition
48
![Page 49: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/49.jpg)
49
In action
http://thomashawk.com/hello/209/1017/1024/Jackson%20Running.jpg
![Page 50: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/50.jpg)
50
![Page 51: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/51.jpg)
In the previous example Newton’s method used 8 function
evaluations Bisection method requires 36
evaluations starting from False position requires 31 evaluations
starting from
51
![Page 52: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/52.jpg)
52
![Page 53: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/53.jpg)
Any Questions?
53
![Page 54: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/54.jpg)
Initial guess Are these comparisons fair?
– , converges to after 5 iterations
– , fails to converges after 5000 iterations
– , converges to after 42 iterations
54
example
answer
![Page 55: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/55.jpg)
Initial guess Are these comparisons fair?
– , converges to after 5 iterations
– , fails to converges after 5000 iterations
– , converges to after 42 iterations
55
answer
![Page 56: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/56.jpg)
Initial guess Are these comparisons fair?
– , converges to after 5 iterations
– , fails to converges after 5000 iterations
– , converges to after 42 iterations
56
![Page 57: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/57.jpg)
in Newton’s method Not guaranteed to converge
– , fails to converge
May converge to a value very far from – , converges to
Heavily dependent on the choice of
57
![Page 58: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/58.jpg)
Convergence analysis for Newton’s method
58
![Page 59: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/59.jpg)
59
The simplest plan is to apply the general fixed point iteration convergence theorem
![Page 60: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/60.jpg)
Analysis strategy To do this, it is must be shown that
there exists such an interval, , which contains the root , for which
60
![Page 61: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/61.jpg)
61
![Page 62: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/62.jpg)
62
![Page 63: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/63.jpg)
63
![Page 64: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/64.jpg)
64
![Page 65: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/65.jpg)
Any Questions?
65
![Page 66: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/66.jpg)
Newton’s Method
Guaranteed to Converge?
Why sometimes Newton’s method does not converge?
This theorem guarantees that exists But it may be very small
66
hint
answer
![Page 67: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/67.jpg)
Newton’s Method
Guaranteed to Converge?
Why sometimes Newton’s method does not converge?
This theorem guarantees that exists But it may be very small
67
answer
![Page 68: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/68.jpg)
Newton’s Method
Guaranteed to Converge?
Why sometimes Newton’s method does not converge?
This theorem guarantees that exists But it may be very small
68
![Page 69: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/69.jpg)
69
Oh no! After these annoying analyses, the Newton’s method is still not guaranteed to converge!?
http://img2.timeinc.net/people/i/2007/startracks/071008/brad_pitt300.jpg
![Page 70: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/70.jpg)
Don’t worry Actually, there is an intuitive method Combine Newton’s method and bisection
method– Newton’s method first
– if an approximation falls outside current interval, then apply bisection method to obtain a better guess
(Can you write an algorithm for this method?)
70
![Page 71: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/71.jpg)
Newton’s Method
Convergence analysis At least quadratic
–
– , since
Stopping condition–
71
![Page 72: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/72.jpg)
72http://www.dianadepasquale.com/ThinkingMonkey.jpg
Recall that
![Page 73: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/73.jpg)
73
Is Newton’s method always faster?
![Page 74: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/74.jpg)
74
![Page 75: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/75.jpg)
75
In action
http://thomashawk.com/hello/209/1017/1024/Jackson%20Running.jpg
![Page 76: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/76.jpg)
76
![Page 77: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/77.jpg)
Any Questions?
77
2.4 Newton’s Method
![Page 78: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/78.jpg)
2.5
78
Secant Method
![Page 79: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/79.jpg)
Secant method Because that Newton’s method
– 2 function evaluations per iteration
– requires the derivative
Secant method is a variation on either false position or Newton’s method– 1 additional function evaluation per iteration
– does not require the derivative
Let’s see the figure first
79
answer
![Page 80: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/80.jpg)
80
![Page 81: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/81.jpg)
Secant method Secant method is a variation on
either false position or Newton’s method– 1 additional function evaluation per
iteration
– does not require the derivative
– does not maintain an interval
– is calculated with and
81
![Page 82: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/82.jpg)
82
![Page 83: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/83.jpg)
83
![Page 84: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/84.jpg)
84
![Page 85: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/85.jpg)
85
![Page 86: Numerical Analysis](https://reader038.vdocuments.net/reader038/viewer/2022103101/56814106550346895daccc22/html5/thumbnails/86.jpg)
Any Questions?
86
2.5 Secant Method