kfupm (term 092) section 08
DESCRIPTION
CISE-301: Numerical Methods Topic 1: Introduction to Numerical Methods and Taylor Series Lectures 1-4:. KFUPM (Term 092) Section 08. Lecture 1 Introduction to Numerical Methods. What are NUMERICAL METHODS ? Why do we need them? Topics covered in CISE301 . - PowerPoint PPT PresentationTRANSCRIPT
CISE301_Topic1 KFUPM - T092 - Section 08 1
CISE-301: Numerical Methods
Topic 1: Introduction to Numerical Methods and
Taylor Series
Lectures 1-4:
KFUPM(Term 092)Section 08
CISE301_Topic1 KFUPM - T092 - Section 08 2
Lecture 1Introduction to Numerical
Methods
What are NUMERICAL METHODS? Why do we need them? Topics covered in CISE301.
Reading Assignment: Pages 3-10 of textbook
CISE301_Topic1 KFUPM - T092 - Section 08 3
Numerical Methods
Numerical Methods: Algorithms that are used to obtain numerical
solutions of a mathematical problem.
Why do we need them? 1. No analytical solution exists,
2. An analytical solution is difficult to obtain or not practical.
CISE301_Topic1 KFUPM - T092 - Section 08 4
What do we need?
Basic Needs in the Numerical Methods: Practical: Can be computed in a reasonable amount of time. Accurate:
Good approximate to the true value, Information about the approximation error
(Bounds, error order,… ).
CISE301_Topic1 KFUPM - T092 - Section 08 5
Outlines of the Course Taylor Theorem Number
Representation Solution of nonlinear
Equations Interpolation Numerical
Differentiation Numerical Integration
Solution of linear Equations
Least Squares curve fitting
Solution of ordinary differential equations
Solution of Partial differential equations
CISE301_Topic1 KFUPM - T092 - Section 08 6
Solution of Nonlinear Equations Some simple equations can be solved analytically:
Many other equations have no analytical solution:
31
)1(2
)3)(1(444solution Analytic
034
2
2
xandx
roots
xx
solution analytic No052 29
xex
xx
CISE301_Topic1 KFUPM - T092 - Section 08 7
Methods for Solving Nonlinear Equations
o Bisection Method
o Newton-Raphson Method
o Secant Method
CISE301_Topic1 KFUPM - T092 - Section 08 8
Solution of Systems of Linear Equations
unknowns. 1000in equations 1000
have weif do What to
123,2
523,3
:asit solvecan We
52
3
12
2221
21
21
xx
xxxx
xx
xx
CISE301_Topic1 KFUPM - T092 - Section 08 9
Cramer’s Rule is Not Practical
this.compute toyears 10 than more needscomputer super A
needed. are tionsmultiplica102.3 system, 30by 30 a solve To
tions.multiplica
1)N!1)(N(N need weunknowns, N with equations N solve To
problems. largefor practicalnot is Rule sCramer'But
2
21
11
51
31
,1
21
11
25
13
:system thesolve toused becan Rule sCramer'
20
35
21
xx
CISE301_Topic1 KFUPM - T092 - Section 08 10
Methods for Solving Systems of Linear Equations
o Naive Gaussian Elimination
o Gaussian Elimination with Scaled Partial Pivoting
o Algorithm for Tri-diagonal Equations
CISE301_Topic1 KFUPM - T092 - Section 08 11
Curve Fitting Given a set of data:
Select a curve that best fits the data. One choice is to find the curve so that the sum of the square of the error is minimized.
x 0 1 2
y 0.5 10.3 21.3
CISE301_Topic1 KFUPM - T092 - Section 08 12
Interpolation Given a set of data:
Find a polynomial P(x) whose graph passes through all tabulated points.
xi 0 1 2
yi 0.5 10.3 15.3
tablein the is)( iii xifxPy
CISE301_Topic1 KFUPM - T092 - Section 08 13
Methods for Curve Fitting o Least Squares
o Linear Regressiono Nonlinear Least Squares Problems
o Interpolationo Newton Polynomial Interpolationo Lagrange Interpolation
CISE301_Topic1 KFUPM - T092 - Section 08 14
Integration Some functions can be integrated
analytically:
?
:solutions analytical no have functionsmany But
42
1
2
9
2
1
0
3
1
23
1
2
dxe
xxdx
ax
CISE301_Topic1 KFUPM - T092 - Section 08 15
Methods for Numerical Integration
o Upper and Lower Sums
o Trapezoid Method
o Romberg Method
o Gauss Quadrature
CISE301_Topic1 KFUPM - T092 - Section 08 16
Solution of Ordinary Differential Equations
only. cases special
for available are solutions Analytical *
equations. thesatisfies that function a is
0)0(;1)0(
0)(3)(3)(
:equation aldifferenti theosolution tA
x(t)
xx
txtxtx
CISE301_Topic1 KFUPM - T092 - Section 08 17
Solution of Partial Differential Equations
Partial Differential Equations are more difficult to solve than ordinary differential equations:
)sin()0,(,0),1(),0(
022
2
2
2
xxututut
u
x
u
CISE301_Topic1 KFUPM - T092 - Section 08 18
Summary Numerical Methods: Algorithms that are
used to obtain numerical solution of a mathematical problem.
We need them when No analytical solution
exists or it is difficult to obtain it.
Solution of Nonlinear Equations Solution of Linear Equations Curve Fitting
Least Squares Interpolation
Numerical Integration Numerical Differentiation Solution of Ordinary Differential
Equations Solution of Partial Differential
Equations
Topics Covered in the Course
CISE301_Topic1 KFUPM - T092 - Section 08 19
Number Representation Normalized Floating Point Representation Significant Digits Accuracy and Precision Rounding and Chopping
Reading Assignment: Chapter 3
Lecture 2 Number Representation and
Accuracy
CISE301_Topic1 KFUPM - T092 - Section 08 20
Representing Real Numbers You are familiar with the decimal system:
Decimal System: Base = 10 , Digits (0,1,…,9)
Standard Representations:
21012 10510410210110345.312
part part
fraction integralsign
54.213
CISE301_Topic1 KFUPM - T092 - Section 08 21
Normalized Floating Point Representation Normalized Floating Point Representation:
No integral part, Advantage: Efficient in representing very small or very
large numbers.
integer:,0
exponent mantissasign
10.0
1
4321
nd
dddd n
CISE301_Topic1 KFUPM - T092 - Section 08 22
Calculator Example Suppose you want to compute: 3.578 * 2.139 using a calculator with two-digit fractions
3.57 * 2.13 7.60=
7.653342True answer:
CISE301_Topic1 KFUPM - T092 - Section 08 23
Binary System
Binary System: Base = 2, Digits {0,1}
exponent mantissasign
21.0 432nbbb
10 11 bb
1010321
2 )625.0()212021()101.0(
CISE301_Topic1 KFUPM - T092 - Section 08 24
7-Bit Representation(sign: 1 bit, mantissa: 3bits, exponent: 3 bits)
CISE301_Topic1 KFUPM - T092 - Section 08 25
Fact Numbers that have a finite expansion in one numbering
system may have an infinite expansion in another numbering system:
You can never represent 0.1 exactly in any computer.
210 ...)011000001100110.0()1.0(
CISE301_Topic1 KFUPM - T092 - Section 08 26
Representation
Hypothetical Machine (real computers use ≥ 23 bit mantissa)
Mantissa: 3 bits Exponent: 2 bits Sign: 1 bit
Possible positive machine numbers:
.25 .3125 .375 .4375 .5 .625 .75 .875
1 1.25 1.5 1.75
CISE301_Topic1 KFUPM - T091 - Section 07 27
Representation
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
Gap near zero
CISE301_Topic1 KFUPM - T092 - Section 08 28
Remarks
Numbers that can be exactly represented are called machine numbers.
Difference between machine numbers is not uniform
Sum of machine numbers is not necessarily a machine number:
0.25 + .3125 = 0.5625 (not a machine number)
CISE301_Topic1 KFUPM - T092 - Section 08 29
Significant Digits
Significant digits are those digits that can be used with confidence.
CISE301_Topic1 KFUPM - T092 - Section 08 30
48.9
Significant Digits - Example
CISE301_Topic1 KFUPM - T092 - Section 08 31
Accuracy and Precision
Accuracy is related to the closeness to the true value.
Precision is related to the closeness to other
estimated values.
CISE301_Topic1 KFUPM - T092 - Section 08 32
CISE301_Topic1 KFUPM - T092 - Section 08 33
Rounding and Chopping
Rounding: Replace the number by the nearest machine number.
Chopping: Throw all extra digits.
CISE301_Topic1 KFUPM - T092 - Section 08 34
Rounding and Chopping - Example
CISE301_Topic1 KFUPM - T092 - Section 08 35
Can be computed if the true value is known:
100* valuetrue
ionapproximat valuetrue
Error RelativePercent Absolute
ionapproximat valuetrue
Error True Absolute
t
tE
Error Definitions – True Error
CISE301_Topic1 KFUPM - T092 - Section 08 36
When the true value is not known:
100*estimatecurrent
estimate previous estimatecurrent
Error RelativePercent Absolute Estimated
estimate previous estimatecurrent
Error Absolute Estimated
a
aE
Error Definitions – Estimated
Error
CISE301_Topic1 KFUPM - T092 - Section 08 37
We say that the estimate is correct to n decimal digits if:
We say that the estimate is correct to n decimal digits rounded if:
n10Error
n 102
1Error
Notation
CISE301_Topic1 KFUPM - T092 - Section 08 38
Summary Number Representation
Numbers that have a finite expansion in one numbering system may have an infinite expansion in another numbering system.
Normalized Floating Point Representation Efficient in representing very small or very large numbers, Difference between machine numbers is not uniform, Representation error depends on the number of bits used in
the mantissa.
CISE301_Topic1 KFUPM - T092 - Section 08 39
Lectures 3-4Taylor Theorem
Motivation Taylor Theorem Examples
Reading assignment: Chapter 4
CISE301_Topic1 KFUPM - T092 - Section 08 40
Motivation
We can easily compute expressions like:
?)6.0sin(,4.1 computeyou do HowBut,
)4(2
103 2
x
way?practical a thisis
?)6.0sin(
compute todefinition theusecan We
0.6
ab
CISE301_Topic1 KFUPM - T092 - Section 08 41
Taylor Series
∑∞
000
)(
000
)(
30
0)3(
20
0)2(
00'
0
0
)()(!
1)(
:can write weconverge, series theIf
)()(!
1
...)(!3
)()(
!2
)()()()(
:about )( ofexpansion seriesTaylor The
k
kk
k
kk
xxxfk
xf
xxxfk
SeriesTaylor
or
xxxf
xxxf
xxxfxf
xxf
CISE301_Topic1 KFUPM - T092 - Section 08 42
Taylor Series – Example 1
∞.xfor converges series The
!)()(
!
1
11)0()(
1)0()(
1)0(')('
1)0()(
∑∑∞
0
∞
000
)(
)()(
)2()2(
k
k
k
kkx
kxk
x
x
x
k
xxxxf
ke
kforfexf
fexf
fexf
fexf
:0about )( of expansion seriesTaylor Obtain xexf x
CISE301_Topic1 KFUPM - T092 - Section 08 43
Taylor SeriesExample 1
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 10
0.5
1
1.5
2
2.5
3
1
1+x
1+x+0.5x2
exp(x)
CISE301_Topic1 KFUPM - T092 - Section 08 44
Taylor Series – Example 2
∞.xfor converges series The
....!7!5!3
)(!
)()sin(
1)0()cos()(
0)0()sin()(
1)0(')cos()('
0)0()sin()(
753∞
00
0)(
)3()3(
)2()2(
∑
xxxxxx
k
xfx
fxxf
fxxf
fxxf
fxxf
k
kk
:0about )sin()( of expansion seriesTaylor Obtain xxxf
CISE301_Topic1 KFUPM - T092 - Section 08 45
-4 -3 -2 -1 0 1 2 3 4-4
-3
-2
-1
0
1
2
3
4
x
x-x3/3!
x-x3/3!+x5/5!
sin(x)
CISE301_Topic1 KFUPM - T092 - Section 08 46
Convergence of Taylor Series(Observations, Example 1)
The Taylor series converges fast (few terms are needed) when x is near the point of expansion. If |x-c| is large then more terms are needed to get a good approximation.
CISE301_Topic1 KFUPM - T092 - Section 08 47
Taylor Series – Example 3
....xxx1x1
1 :ofExpansion SeriesTaylor
6)0(1
6)(
2)0(1
2)(
1)0('1
1)('
1)0(1
1)(
:0about of expansion seriesTaylor Obtain
32
)3(4
)3(
)2(3
)2(
2
fx
xf
fx
xf
fx
xf
fx
xf
xx1
1f(x)
CISE301_Topic1 KFUPM - T092 - Section 08 48
Example 3 – Remarks
Can we apply Taylor series for x>1??
How many terms are needed to get a good approximation???
These questions will be answered using Taylor’s Theorem.
CISE301_Topic1 KFUPM - T092 - Section 08 49
Taylor’s Theorem
. and between is)()!1(
)(
:where
)(!
)()(
:b][a,∈cany for then
b],[a, interval closed ain 1)(n ..., 2, 1, orders of
sderivative continuous possesses f(x)function a If
1)1(
1
1
n
0
)(
∑
xcandcxn
fE
Ecxk
cfxf
nn
n
nk
kk
(n+1) terms Truncated Taylor Series
Remainder
CISE301_Topic1 KFUPM - T092 - Section 08 50
Taylor’s Theorem
.applicablenot is TheoremTaylor
defined.not are sderivative
its andfunction then the,1includes],[If
.1||if0expansion ofpoint with the1
1
:for theoremsTaylor'apply can We
xba
xcx
f(x)
CISE301_Topic1 KFUPM - T092 - Section 08 51
Error Term
. and between allfor
)()!1(
)(
:on boundupper an derivecan we
error,ion approximat about the ideaan get To
1)1(
1
xcofvalues
cxn
fE n
n
n
CISE301_Topic1 KFUPM - T092 - Section 08 52
Error Term – Example 4
0514268.82.0)!1(
)()!1(
)(
1≥≤)()(
41
2.0
1
1)1(
1
2.0)()(
EEn
eE
cxn
fE
kforefexf
nn
nn
n
kxk
?2.0=0=about
expansion seriesTaylor its of3)=(n terms4first the
by =)( replaced weiferror theis large How
xwhenx
exf x
CISE301_Topic1 KFUPM - T092 - Section 08 53
Alternative form of Taylor’s Theorem
hxxwherehn
fE
Ehk
xfhxf
[a,b]hx[a,b]x
[a,b]
xfLet
nn
n
n
n
k
kk
and between is)!1(
)(
!
)()(
:then, and and
, intervalan on 1)(n ..., 2, 1, orders
of sderivative continuous have)(
1)1(
1
10
)(
CISE301_Topic1 KFUPM - T092 - Section 08 54
Taylor’s Theorem – Alternative forms
. and between is
)!1(
)(
!
)()(
,
. and between is
)()!1(
)()(
!
)()(
1)1(
0
)(
1)1(
0
)(
hxxwhere
hn
fh
k
xfhxf
xchxx
xcwhere
cxn
fcx
k
cfxf
nnn
k
kk
nnn
k
kk
CISE301_Topic1 KFUPM - T092 - Section 08 55
Mean Value Theorem
(b-a)dx
df(ξf(a)f(b)
bhxaxn
(b-a)
f(a)f(b)
dx
df(ξ
baξ
)
, ,0for Theorem sTaylor' Use:Proof
)
],[ exists then there
b)(a, intervalopen on the defined is derivative its and
b][a, interval closed aon function continuous a isf(x) If
CISE301_Topic1 KFUPM - T092 - Section 08 56
Alternating Series Theorem
termomittedFirst :
n terms)first theof (sum sum Partial:
converges series The
then
0lim
If
S
:series galternatin heConsider t
1
1
4321
4321
n
n
nnnn
a
S
aSS
and
a
and
aaaa
aaaa
CISE301_Topic1 KFUPM - T092 - Section 08 57
Alternating Series – Example 5
!7
1
!5
1
!3
11)1(s
!5
1
!3
11)1(s
:Then
0lim
:since series galternatin convergent a is This!7
1
!5
1
!3
11)1(s:usingcomputed becansin(1)
4321
in
in
aandaaaa
in
nn
CISE301_Topic1 KFUPM - T092 - Section 08 58
Example 6
? 1with xe eapproximat to
used are terms1)(n when beerror can the largeHow
expansion) ofcenter (the5.0aboutef(x) of
expansion seriesTaylor theObtain
12x
12x
c
CISE301_Topic1 KFUPM - T092 - Section 08 59
Example 6 – Taylor Series
...!
)5.0(2...
!2
)5.0(4)5.0(2
)5.0(!
)5.0(
2)5.0(2)(
4)5.0(4)(
2)5.0('2)('
)5.0()(
22
222
∞
0
)(12
2)(12)(
2)2(12)2(
212
212
∑
k
xe
xexee
xk
fe
efexf
efexf
efexf
efexf
kk
k
kk
x
kkxkk
x
x
x
5.0,)( of expansion seriesTaylor Obtain 12 cexf x
CISE301_Topic1 KFUPM - T092 - Section 08 60
Example 6 – Error Term
)!1(
max)!1(
)5.0(2
)!1(
)5.01(2
)5.0()!1(
)(
2)(
3
12
]1,5.0[
11
1121
1)1(
12)(
n
eError
en
Error
neError
xn
fError
exf
nn
nn
nn
xkk
CISE301_Topic1 KFUPM - T092 - Section 08 61
Remark
In this course, all angles are assumed to be in radian unless you are told otherwise.
CISE301_Topic1 KFUPM - T092 - Section 08 62
Maclaurin Series Find Maclaurin series expansion of cos (x).
Maclaurin series is a special case of Taylor series with the center of expansion c = 0.
CISE301_Topic1 KFUPM - T092 - Section 08 63
Maclaurin Series – Example 7
∞.xfor converges series The
....!6!4!2
1)(!
)0()cos(
0)0()sin()(
1)0()cos()(
0)0(')sin()('
1)0()cos()(
642∞
0
)(
)3()3(
)2()2(
∑
xxxx
k
fx
fxxf
fxxf
fxxf
fxxf
k
kk
)cos()( :of expansion series Maclaurin Obtain xxf