numerical methods for ordinary differential …bnikolic/teaching/phys660/lectures...numerical...
TRANSCRIPT
Numerical Methods for Ordinary Differential Equations
Branislav K. NikolićDepartment of Physics and Astronomy, University of Delaware, U.S.A.
PHYS 460/660: Computational Methods of Physics http://www.physics.udel.edu/~bnikolic/teaching/phys660/phys660.html
PHYS 460/660: Numerical Methods for ODE
Ordinary Differential Equations
2
2, ( ), ( ), ..., ( ) 0
n
n
d d dF y y t y t y t
dt dt dt
=
Ordinary: only one independent variable
Differential: unknown functions enter into the equation through its derivatives
Order: highest derivative in F
Degree: exponent of the highest derivative3
2
2Example: ( ) ( ) 0
dy t y t
dt
− =
PHYS 460/660: Numerical Methods for ODE
What is Solution of ODE?
A problem involving ODE is not completely specified by its equation
ODE has to be supplemented with boundary conditions:
( )y y t=
•Initial value problem: is given at some starting value , and it is desired to find at some final points or at some discrete list of points (for example, at tabulated intervals).
•Two point bondary value problem: Boundary conditions are specified at more than one ; typically some of the conditions will be specified at and some at .
yit
y ft
tit
ft
PHYS 460/660: Numerical Methods for ODE
What is Numerical Solution to the Initial Value Problem?
( ) 0 0
( ), ( ) ; ( )
dy tf t y t y t y
dt= =
A numerical solution to this problem generates sequence of values for the independent variable
and a corresponding sequence of values of the dependent variable
so that each approximates solution at :
1 2, , , nt t t…
1 2, , , ny y y…
ny nt
( ) , 0,1,n ny t y n≈ = …
PHYS 460/660: Numerical Methods for ODE
Euler Metod
tru ey
t∆
y
t
E u lery
All finite difference methods start from the same conceptual idea: Add small increments to your function corresponding to derivatives (right-hand side of the equations) multiplied by the stepsize.
Euler method is an implementation of this idea in the simplest and most direct form.
Single-Step Forward
Propagation
PHYS 460/660: Numerical Methods for ODE
Euler Algorithm for First-Order ODE
( , )dy
f t ydt
= ( , )y f t y t∆ = ∆
1 1 1
1
1
initialize , ( )
do while
( , )
end do
i i i
i i
t y y t
i n
y y f t y t
t t t
+
+
≡
≤
= + ∆
= + ∆
PHYS 460/660: Numerical Methods for ODE
Step Size Effects in Radioactive Decay
Analytics: ( 0)t
U UU U
dN NN N t e
dtτ
τ
−
= − ⇒ = =
( )2
1
Numerics (Euler):
( ) (0) ( )UU U
ii i
dNN t N t O t
dt
NN N t
τ+
∆ = + ∆ + ∆
≈ − ∆
Numerical solution will depend Numerical solution will depend on the step sizeon the step size
t∆
PHYS 460/660: Numerical Methods for ODE
Stability of Euler Algorithm
t∆
1
(0) 1,
After n Euler steps of size :
(1 )
at
n
n n n n
dyay y y e
dt
t
y y ay t y a t
−
+
= − ⇒ = =
∆
= − ∆ ⇒ = − ∆
Approximate solution will decay monotonically only if is small enough:
For a single decaying exponential-like solution (i.e. if there is only one first order equation) the existence of a stability criterion is not a problem because has to be small for the reasons of accuracy.t∆
Step size if often limited by the stability criterion:
max
1t t
a∆ ≤ ∆ ≡
PHYS 460/660: Numerical Methods for ODE
Accuracy:Discretization and Roundoff Errors
Local:
Global:
1 1 1
( , )( )
( )
n n
n n n n
n n n
duf u t
LE y u tdt
u t y+ + +
=
⇒ = −=
( )n n nGE y y t= −
0
1
0
1
0
1
0
1
0
( ) ( ) ( ) ( )
( ) ( )
( ) ( )
N
n
n
N
Nt
n nt
n
t
n n nt
Nt
n n nt
n
N
n n
n
f f t y t f d h f t
LE h f t f d
GE h f t f d
GE LE
τ τ
τ τ
τ τ
+
−
=
−
=
−
=
= ⇒ = ≈
= −
= −
=
∑∫
∫
∑ ∫
∑
Method is of order n iff:1 1( )n n
n nLE O h LE Ch+ += ⇔ ≤
1n nh t t t+= − ≡ ∆
1
1
Number of steps for roundoff error to be comparable with the discretization error:
pCN L
Lε
+ ≈
Integrate over interval: Full Error: 0 p
f
LL t t Ch
h
ε= − ⇒ +
PHYS 460/660: Numerical Methods for ODE
Global Discretization Error Example
2 3
2 3
2 3
2 2
2
( ) , 1
( ) ( )( ) 1
2! 3!
( 1) ( ) ( 1)( 2) ( )1
2! 3!
1 ( ) 3 ( ) 1( )
2! 3! 2
n
aT
n
n
aT
n
Ty T e y a
n
aT aTy T aT
n n aT n n n aTy aT
n n
aT aT a ty T y O aTe
n n n
−
−
= = −
= − + − +
− − −= − + − +
∆ − = − + +
…
…
… ∼
Suppose we want to find the solution over the interval Suppose we want to find the solution over the interval →→Divide Divide the interval into the interval into nn equal steps so that equal steps so that
[ ]0,Tt T n∆ =
This is a measure of theglobal truncation errorglobal truncation error,i.e., the error over a fixed range in t. It is proportional to the first power of the step size and hence the Euler the Euler method is a first order method is a first order methodmethod (do not confuse this with the fact that we are applying it in this case to a first order equation).
PHYS 460/660: Numerical Methods for ODE
Reducing Higher Order ODE to System of First Order ODE
Solve higher order ODEs by splitting them into sets of first order equations:
2
2( ) ( ) ( )
( ) ( ) ( )
There is no unique way to do this:
( )( ) ( )
( )
( )
d y dyp t q t y g t
dt dt
dzg t p t z q t y
dy dtz
dydtz
dt
dz dp tg t q t y
dy dt dtz p t y
dt dyz p t y
dt
+ + =
= − −
= ⇒ =
= + − = + ⇒
= −
PHYS 460/660: Numerical Methods for ODE
Example: Realistic Motion of Baseball
1 1
1
1
initialize , ( )
do while
( , )
end do
i i i i
i i
t y t
i n
y y f t y t
t t t
+
+
≤
= + ∆
= + ∆
v rω−2
2
2 02
d r vm mg B v S v
dt vω= − + ×
v rω+
d r a gF
v
ω
1
21
1
1
1
01
x
i i i
x x x
i i i
y
i i i
y y
i i
z
i i i
z z xi i
x x v t
Bv v vv t
m
y y v t
v v g t
z z v t
S vv v t
m
ω
+
+
+
+
+
+
= + ∆
= − ∆
= + ∆
= − ∆
= + ∆
= − ∆
⇔
PHYS 460/660: Numerical Methods for ODE
More Realistic Modeling of Air Flow
PHYS 460/660: Numerical Methods for ODE
ODE of Linear Harmonic Oscillator
2
2
2
2
sin 0
for small θ sin
0,
d g
dt l
d g g
dt l l
θθ
θ θ
θθ
+ =
⇒ ≈
+ = Ω =
2
2 21 1must be conserved!
2 2total
dE ml mgl
dt
θθ
= +
PHYS 460/660: Numerical Methods for ODE
Euler Method for Linear Harmonic Oscillator
Switch to dimensionless quantities: 2
02
2
2
0 sin( )
1 1
2 2total
dt
dt
dE
dt
θθ θ θ φ
θθ
+ = ⇒ = Ω +
= +
( )
( )
2 211 1
12
1
1
2
1
n n ntotal n n
n n n
total nn n
tE
t
E E tt t t
ω ω θω θ
θ θ ω+
+ +
+
+
= − ∆ = +
= + ∆ ⇒ = + ∆= + ∆
Euler discretization algorithm:
PHYS 460/660: Numerical Methods for ODE
Euler Fails on ( )tθ
PHYS 460/660: Numerical Methods for ODE
Euler Fails on ( )tω
PHYS 460/660: Numerical Methods for ODE
Euler Fails on Phase Space Trajectory
PHYS 460/660: Numerical Methods for ODE
Can We Resurrect Euler by Using Smaller Step Size?
PHYS 460/660: Numerical Methods for ODE
Cromer Fixed Euler Method for LHO
1
1 1 1
1
n n n
n n n n n
n n
t
t
t t t
ω ω θ
ω ω θ θ ω
+
+ + +
+
= − ∆
→ ⇒ = + ∆ = + ∆
Apparently trivial trick, but:
( ) ( )
2 2 20 0
2 2 2 3
1
0 0 0 0
cos2( ) 0
1
2
sin( ), cos( )
over a period
n n n n
t t
E E t O t
t t t t
ω θ θ
ω θ
θ θ ω θ
+
− = − =
= + − ∆ + ∆
= − = −
PHYS 460/660: Numerical Methods for ODE
From Euler to Higher Order Algorithms
1
1
( , )n n n n
n n
y y f t y
t t h
+
+
= +
= + ( ) ( )m
exact
ty t t y t dy dt t+ ∆ = + ∆
vs.
Mean value theorem
PHYS 460/660: Numerical Methods for ODE
Midpoint Method: Second Order Runge-Kutta
1
2 1
( , )
( , )2 2
n n
n n
s f t y
h hs f t y s
=
= + +
3
1 2
1
( )n n
n n
y y hs O h
t t h
+
+
= + +
= +
PHYS 460/660: Numerical Methods for ODE
Classical Runge-Kutta
5
1 1 2 3 4
1
( 2 2 ) ( )6
n n
n n
hy y s s s s O h
t t h
+
+
= + + + + +
= +
Fourth-order method
1
2 1
3 2
4 3
( , )
( , )2 2
( , )2 2
( , )
n n
n n
n n
n n
s f t y
h hs f t y s
h hs f t y s
s f t h y hs
=
= + +
= + +
= + +
PHYS 460/660: Numerical Methods for ODE
Classical Runge-Kutta F90 Subroutine
PHYS 460/660: Numerical Methods for ODE
General Single-Step Methods
1
,
1
( , ), 1, ,i
i n i n i j j
j
s f t h y h s i kα β−
=
= + + =∑ …
Each of the k stages of the algorithm computes slope by evaluating for a particular value of and a value of obtained by taking linear combinations of the previous slopes:
is( , )f t y t y
The proposed step is also a linear combination of the slopes:
1
1
k
n n i i
i
y y h sγ+=
= + ∑Error is estimated from yet another linear combination of the slopes:
1
1
k
n i i
i
e h sδ+=
= ∑
The parameters are determined by matching terms in the Taylor series expansion of the slopes → the order of the method is the exponent of the smallest power of h that cannot be matched.
In MATLAB ODE numerical routines are named as odennxx, where nn indicates the order and xx is some special feature of the method.
PHYS 460/660: Numerical Methods for ODE
Example: MATLAB ode23 Function(Bogacki and Shampine BS23 Algorithm)
1
2 1
3 2
( , )
( , )2 2
3 3( , )
4 4
n n
n n
n n
s f t y
h hs f t y s
s f t h y hs
=
= + +
= + +
1 1 2 3
1 4 1 1
(2 3 4 )9
; ( , )
n n
n n n n
hy y s s s
t t h s f t y
+
+ + +
= + + +
= + =
1 1 2 3 4( 5 6 8 9 )72
n
he s s s s+ = − + + −
PHYS 460/660: Numerical Methods for ODE
Beyond Runge-Kutta Methods
Runge-Kutta methods propagates a solution over an interval by combining the information from several Euler-style steps (each involving one evaluation of the right-hand side f’s), and then using the information obtained to match Taylor series expansion up to some higher order.
Richardson extrapolation method used the powerful idea of extrapolating computed result to the value that would have been obtained if the stepsize had been very much smaller than it actually was. In particular, extrapolation to zero stepsize is the desired goal – implemented by Burlich-Stoer algorithm.
Predictor-corrector methods store the solution along the way, and use those results to extrapolate the solution one step advanced; they correct the extrapolation using derivative information at the new point.
PHYS 460/660: Numerical Methods for ODE
Stiff Systems of Differential Equations
Stiffness arises in systems of ODE where there are two or more very different scales of the independent variable:
2998 1998 , 999 1999
(0) 1, (0) 0
du dvu y zu v u v
dt dtv y z
u v
= −= + = − −
⇒ = − += =
1000
1000
2 t t
t t
u e e
v e e
− −
− −
= −⇒
= − +
Follow the variation in the solution on the shortest length scale to maintain stability of the integration even though accuracy requirements allow for a much larger step size → use implicit methods:
explicit
1
implicit
1 1 1
, 0 (1 )
2 as
1
n n n n
n
nn n n n
y cy c y y ty c t y
t c y n
yy cy y y ty y
c t
+
+ + +
′ ′= − > ⇒ = +∆ = − ∆
∆ > ⇔ →∞ →∞
′ ′= − ⇒ = +∆ ⇒ =+ ∆
PHYS 460/660: Numerical Methods for ODE
Solutions to Stiffness Beyond Implicit Euler
Generalizations of Runge-Kutta methods →Rosenbrock methods and Kaps-Rentrop methods.
Burlich-Stoer algorithm generalized to Bader-Deuflhard semi-implicit extrapolation method.
Predictor-corrector methods generalized toGear backward differentiation method.
To improve higher-order (than Euler, which is first-order) methods use: