Introduction to Numerical Solutions Introduction to Numerical Solutions of Ordinary Differential Equationsof Ordinary Differential Equations

Larry Caretto

Mechanical Engineering 501AB

Seminar in Engineering Analysis

October 29, 2003

2

Outline• Review last class and homework• Midterm Exam November 4 covers

material up to and including last week’s lecture

• Overivew of numerical solutions– Initial value problems in first-order

equations– Systems of first order equations and initial

value problems in higher order equations– Boundary value problems– Stiff systems and eigenvalues

3

Review Last Class• Systems of equations

– Combine to higher order equation– Matrix approaches– Reduction of order

• Laplace transforms for ODEs– Transform ODE from y(t) to Y(s)– Solve for Y(s)– Rearrange solution– Transform back to y(t)

4

Simultaneous Solution• Differentiate first (y1) equation to obtain

derivative(s) of y2 that are present in second equation

• Solve first equation for y2

• Substitute equations for y2 and its deriv-atives into the result of the first step

• Result is a higher order equation that can be solved (if possible) by usual methods

5

Matrix Differential Equations II

• Matrix components in rAyy

dt

d

ny

y

y

y

3

2

1

y

dtdy

dtdydt

dydt

dy

y

y

y

y

dt

d

dt

d

nn

3

2

1

3

2

1

y

nnnnn

n

n

n

aaaa

aaaa

aaaa

aaaa

321

3333231

2232221

1131211

A

Tnrrrr 321r

6

Solving

• Assume that A(n x n) has n linearly independent eigenvectors

• Eigenvectors are columns of matrix, X

• Define a new vector s = X-1y (y = Xs)

• Substitute y = Xs into the matrix differential equation

• Obtain equation for s using = X-1AX that gives independent equations

rAyy

dt

d

7

Terms in Solution of pss

dt

d

• With these definitions, si = Cieit + pi/i

becomes s = EC + -1p (E(0) = I)

t

t

t

t

ne

e

e

e

t

000

000

000

000

)(3

2

1

E

n

np

p

p

p

3

3

2

2

1

1

1p

nC

C

C

C

3

2

1

C

8

Apply Initial Conditions on y(0)• Get y=Xs = XEC + X-1p• At t = 0, E = I, and y = y0 = initial y

components, giving y0 = XIC + X-1p• Premultiply by X-1 to obtain X-1y0 = X-

1XC + X-1X-1p = C + -1p• Constant vector, C = X-1y0 - -1p• Result: y=XE [X-1y0 - -1p] + X-1p• Homogenous(p = 0): y=XEX-1y0

9

Reduction of Order

• An nth order equation can be written as a system of n first order equations

• We can write a general nonlinear nth order equation as shown below

1

1

,,,,n

n

n

n

dx

yd

dx

dyyxf

dx

yd

• Redefine y as z0 and define the following derivatives

21

20

2

2

2

10 z

dx

dz

dx

zd

dx

ydz

dx

dz

dx

dy

10

Reduction of Order II

• z0 to zn-1 are n variables that satisfy simultaneous, first-order ODEs

1101

1

1

,,,,,,,,

nn

nn

n

n

n

zzzxfdx

dz

dx

yd

dx

dyyxf

dx

dy

• Continue this definition up to zn-1

dx

dz

dx

dz

dx

zd

dx

zd

dx

zd

dx

yd nnn

n

n

n

n

n

n

n1

22

22

2

11

10

2,,1,01 nkzdx

dzk

k

11

Reduction of Order III

• The main application of reduction of order is to numerical methods

• Numerical solutions of ODEs develop methods to solve a system of first order equations

• Higher order equations are solved by converting them to a system of first order equations

12

Simple Laplace Transformsf(t) F(s) f(t) F(s)

ctn cn!/sn+1 ceatsin t

ctx c(x+1)/sx+1

ceat c/(s – a) ceatcos t

csin t c/(s2 + 2)

ccos t cs/(s2 + 2) Additional transforms in Table 5.9, pp 297-299 of Kreyszig

csinh t c/(s2 - 2)

ccosh t cs/(s2 - 2)

22)( as

c

22)(

)(

as

asc

13

Transforms of Derivatives• The main formulas for differential

equations are the Laplace transforms of derivatives

• These have transform of desired function, L [f(t)] = F(s) and the initial conditions, f(0), f’(0), etc.

)0()()]('[ fssFtf

)0(')0()()](''[ 2 fsfsFstf

)0()0()0(')0()()]([ )1()2(21)( nnnnnn fsffsfssFstf

14

Solving Differential Equations

• Transform all terms in the differential equation to get an algebraic equation– For a differential equation in y(t) we get the

transforms Y(s) = L [y(t)]– Similar notation for other transformed

functions in the equation R(s) = L [r(t)]

• Solve the algebraic equation for Y(s)• Obtain the inverse transform for Y(s)

from tables to get y(t)

15

Partial Fractions

• Method to convert fraction with several factors in denominator into sum of individual factors (in denominator)

• Necessary to modify complicated solution for Y(s) into simpler functions whose transforms are known

• Special rules for repeated factors and for complex conjugates

16

Shifting Theorems• Sometimes required for transform table

– First theorem is defined for function f(t) with transform F(s)

– F(s - a) has inverse of e-atf(t)– Example Y(s) = [2(s+1)+4]/[(s + 1)2 + 1]– For F(s) = s/(s2 + 2); f(t) = cos t and f(t) =

cos t for F(s) = /(s2 + 2)– Here Y(s+1) = 2(s+1)+4]/[(s + 1)2 + 1] +

4/[(s + 1)2 + 1] so y(t) = e-t times Y(s) inverse

– y(t) = e-t[2 cos t + 4 sin t]

17

Shifting Theorems II

• Second shifting theorem– Applies to e-asF(s), where F(s) is known

transform of a function f(t)– Inverse transform is f(t – a) u(t – a) where

u(t – a) is the unit step function– If we have e-ass/(s2 + 2), we recognize

s/(s2 + 2) as F(s) for f(t) = cos t– Thus e-ass/(s2 + 2) is the Laplace transform

for cos[t(t – a)] u(t – a)

18

Numerical Analysis Problems• Numerical solution of algebraic

equations and eigenvalue problems• Solution of one or more nonlinear

algebraic equations f(x) = 0• Linear and nonlinear optimization• Constructing interpolating polynomials• Numerical quadrature• Numerical differentiation• Numerical differential equations

19

Interpolation

• Start with N data pairs xi, yi

• Find a function (polynomial) that can be used for interpolation

• Basic rule: the interpolation polynomial must fit all points exactly

• Denote the polynomial as p(x)

• The basic rule is that p(xi) = yi

• Many different forms

20

Newton Polynomials• p(x) = a0 + a1(x – x0) + a2(x – x0)(x – x1) +

a3(x – x0)(x – x1)(x – x2) + … + an-1(x – x0)(x – x1)(x – x2) … (x – xn-2)

• Terms with factors of x – xi are zero when x = xi

– Use this and rule that p(xi) = yi to find ai

• a0 = y0, a1 = (y1 – y0) / (x1 – x0) • y2 = a0 + a1(x2 – x0) + a2(x2 – x0)(x2 – x1)

– Solve for a2 using results for a0 and a1

21

Newton Polynomials II

• y2 = a0 + a1(x2 – x0) + a2(x2 – x0)(x2 – x1)

))((

)(

))((

)(

1202

0201

0102

1202

021022 xxxx

xxxx

yyyy

xxxx

xxaaya

• y2 = a0 + a1(x2 – x0) + a2(x2 – x0)(x2 – x1)

• Data determine coefficients

• Develop scheme known as divided difference table to compute ak

22

Divided Difference Tablex0 y0 a0

a1

x1 y1 a2

x2 y2 a3

x3 y3

01

010 xx

yyF

12

121 xx

yyF

23

232 xx

yyF

02

010 xx

FFS

13

121 xx

FFS

03

010 xx

SST

23

Divided Difference Example0 0 a0

a1

10 10 a2

20 40 a3

30 100

1010

0100

F

31020

10401

F

62030

401002

F

1.020

130

S

15.1030

361

S600

1

030

15.2.0

T

24

Divided Difference Example II

• Divided difference table gives a0 = 0, a1 = 1, a2 = .1, and a3 = 1/600

• Polynomial p(x) = a0 + a1(x – x0) + a2(x – x0)(x – x1) + a3(x – x0)(x – x1)(x – x2) = 0 + 1(x – 0) + 0.1(x – 0)(x – 10) + (1/600)(x – 0)(x – 10)(x – 20) = x + 0.1x(x – 10) + (1/600)x(x – 10)(x – 20)

• Check p(30) = 30 + .1(30)(20) + (1/600) (30)(20)(10) = 30 + 60 + 10 = 100 (correct)

25

Constant Step Size• Divided differences work for equal or

unequal step size in x• If x = h is a constant we have simpler

results– Fk = yk/h = (yk+1 – yk)/h– Sk = 2yk/h2 = (yk+2 – 2yk-1 + yk)/h2

– Tk = 3yk/h3 = (yk+3 – 3yk+2 + 3yk+1 – yk)/h3

– nyk is called the nth forward difference– Can also define backwards and central

differences

26

Interpolation Approaches• When we have N data points how do we

interpolate among them?– Order N-1 polynomial not good choice– Use piecewise polynomials of lower order

(linear or quadratic)– Can match first and or higher derivatives

where piecewise polynomials join– Cubic splines are piecewise cubic

polynomials that match first and second derivatives (as well as values)

27

Cubic Spline Interpolation

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0 1 2 3 4 5 6

x values

y va

lues Known f'

Natural

No Knot

Data

28

Newton Interpolating Polynomial

-1

0

1

2

3

4

5

0 1 2 3 4 5 6

X Values

Y V

alues

Polynomial

Data

29

Polynomial Applications• Data interpolation• Approximation functions in numerical

quadrature and solution of ODEs• Basis functions for finite element

methods• Can obtain equations for numerical

differentiation• Statistical curve fitting (not discussed

here) usually used in practice

30

Derivative Expressions

• Obtain from differentiating interpolation polynomials or from Taylor series

• Series expansion for f(x) about x = a

....)-(!3

1)-(

!2

1)()()( 3

3

32

2

2

axdx

fdax

dx

fdax

dx

dfafxf

axaxax

0

)-(!

1)(

n

n

ax

n

n

axdx

fd

nxf

• Note: d0f/dx0 = f and 0! = 1

• What is error from truncating series?

31

Truncation Error

• If we truncate series after m terms

10

)-(!

1)-(

!

1)(

mn

n

ax

n

nm

n

n

ax

n

n

axdx

fd

nax

dx

fd

nxf

Terms used Truncation error, m

• Can write truncation error as single term at unknown location (derivation based on the theorem of the mean)

11

1

1

)-(!)1(

1)-(

!

1

m

x

m

m

mn

n

ax

n

n

m axdx

fd

max

dx

fd

n

32

Derivative Expressions

• Look at finite-difference grid with equal spacing: h = x so xi = x0 + ih

• Taylor series about x = xi gives f(xi + kh) = f[x0 + (i+k)h] = fi+k in terms of f(xi) = fi

.....)(!3

1)(

!2

1)()( 3

3

32

2

2

khdx

fdkh

dx

fdkh

dx

dfxfkhxf

iii xxxxxxii

iii xx

n

nni

xx

ixx

i dx

fdf

dx

fdf

dx

dff

... 2

2'''

• Compact derivative notation

33

Derivative Expressions II

• Combine all definitions for compact series notation

.....)(!3

1)(

!2

1)()( 3

3

32

2

2

khdx

fdkh

dx

fdkh

dx

dfxfkhxf

iii xxxxxxii

• Use this formula to get expansions for various grid locations about x = xi and use results to get derivative expressions

.....!3

)(

!2

)( 3'''2'''

khfkhfkhfff iiiiki

34

Derivative Expressions III

• Apply general equation for k = 1 and k = –1

.....!3!2

3'''2'''

1 hfhf

hfff iiiii

Ahh

ffhfhf

h

fff iiiiiii

1

'''''1' .....

!3!2

.....!3

)(

!2

)( 3'''2'''

khfkhfkhfff iiiiki

.....!3!2

3'''2'''

1 hfhf

hfff iiiii

Ahh

ffhfhf

h

fff iiiiiii

1

'''''1' .....

!3!2

Forward

Backward

35

Derivative Expressions IV

• Subtract fi+1 and fi-1 expressions

.....!3!2

3'''2'''

1 hfhf

hfff iiiii

.....!3!2

3'''2'''

1 hfhf

hfff iiiii

2114'''''2'''

11'

2.....

!5!3Ah

h

ffhfhf

h

fff iiiiiii

• Result called central difference expression

.....!5

2

!3

22

5'''3''''

11 hfhf

hfff iiiii

36

Order of the Error

• Forward and backward derivative have error term that is proportional to h

• Central difference error is proportional to h2

• Error proportional to hn called nth order

• Reducing step size by a factor of a reduces nth order error by an

n

h

h

1

212

37

Order of the Error Notation• Write the error term for nth error term as

O(hn) – Big oh notation, O, denotes order– Recognizes that factor multiplying hn may

change slightly with h

)(2

211' hOh

fff iii

)(1' hOh

fff iii

)(1' hO

h

fff iii

First order forward First order backward

Second order central

38

Higher Order Derivatives

• Add fi+1 and fi-1 expressions

.....!3!2

3'''2'''

1 hfhf

hfff iiiii

.....!3!2

3'''2'''

1 hfhf

hfff iiiii

2211

4'''''2'''

211'' 2

.....!5!3

2hO

h

fffhfhf

h

ffff iiiiiiiii

• f’’ is second-order, central difference

.....!6

2

!4

2

!222

6''''''4''''2''

11 hfhfhf

fff iiiiii

39

Higher Order Directional

• We can get higher order derivative expressions at the expense of more computations

• Get second order forward and backward derivative expressions from fi+2 and fi-2, respectively

• Combine with previous expressions for fi+1 and fi-1 to eliminate first order error term

40

Specific Taylor Series

• General equation

.....!3

)(

!2

)( 3'''2'''

khfkhfkhfff iiiiki

.....!3

8!2

423'''2''

'2

hfhfhfff iiiii

• k = -2

• k = 2

.....!3

8!2

423'''2''

'2

hfhfhfff iiiii

• k = -3

• k = 3 .....!3

27!2

933'''2''

'3

hfhfhfff iiiii

.....!3

27!2

933'''2''

'3

hfhfhfff iiiii

41

Second Order Forward

• Subtract 4fi+1 from fi+2 to eliminate h2 term

..62

4

...6

82

424

3'''

2'''

3'''

2'''

12

hf

hfhff

hf

hfhffff

iiii

iiiiii

...6

42343

''''12

hfhffff iiiii

...32

34 2'''12'

h

fh

ffff i

iiii

Second order error

42

Second Order Backwards

• Add 4fi-1 to –fi-2 to eliminate h2 term

..62

4

...6

82

424

3'''

2'''

3'''

2'''

12

hf

hfhff

hf

hfhffff

iiii

iiiiii

...6

42343

''''12

hfhffff iiiii

...32

34 2'''12'

h

fh

ffff i

iiii

Second order error

43

Other Derivative Expressions• Can continue in this fashion

– Write Taylor series for fi+1, fi-1, fi+2, fi-2, fi+3, fi-

3, etc.– Create linear combinations with factors that

eliminate desired terms– Eliminate fi term to obtain central difference– Keep only terms in fk with k i for forward

difference expressions– Keep only terms in fk with k i for forward

difference expressions– See results on page 271 of Hoffman

44

Order of Error Examples• Table 2-1 in notes shows error in first

derivative for ex around x = 1– Using first- and second-order forward and

second-order central differences– Step h = 0.4, 0.2, and 0.1– Error ratio for doubling step size

• 4.01 to 4.02 for central differences• 2.07 to 2.15 for first-order forward differences• 4.32 to 4.69 for second-order forward

)log()log(

)log()log(

log

log

12

12

1

2

1

2

hhh

hn

45

Roundoff Error• Possible in derivative expressions from

subtracting close differences• Example f(x) = ex: f’(x) (ex+h – ex-h)/(2h)

and error at x = 1 is (e1+h – e1-h)/(2h) - e

3105.4718282.2)1.0(2

722815.2004166.3

xE

9105.4597182818284.2)0001.0(2

7180100139.27185536702.2

xE

9109.5718281828.2)0000001.0(2

03887182815566.287247182821002.2

xE

Second order error

46

Figure 2-1. Effect of Step Size on Error

1.E-11

1.E-10

1.E-09

1.E-08

1.E-07

1.E-06

1.E-05

1.E-04

1.E-03

1.E-02

1.E-01

1.E+00

1.E+01

1.E-17 1.E-15 1.E-13 1.E-11 1.E-09 1.E-07 1.E-05 1.E-03 1.E-01

Step Size

Err

or

47

Numerical ODE Solutions

• The initial value problem

• Euler method as a prototype for the general algorithm

• Local and global errors

• More accurate methods

• Step-size control for error control

• Applications to systems of equations

• Higher-order equations

48

The Initial Value Problem• dy/dx = f(x,y) (known) with y(x0) = y0

• Basic numerical approach– Use a finite difference grid: xi+1 – xi = h– Replace derivative by finite-difference

approximation: dy/dx (yi+1 – yi) / (xi+1 – xi) = (yi+1 – yi) / h

– Derive a formula to compute favg the average value of f(x,y) between xi and xi+1

– Replace dy/dx = f(x,y) by (yi+1 – yi) / h = favg

– Repeatedly compute yi+1 = yi + h favg

49

Notation• xi is the value of the independent variable

at point i on the grid– Determined from the user-selected value of

step size (or the series of hi values)– Can always specify exactly the independent

variable’s value, xi

• yi is the value of the numerical solution at the point where x = xi

• fi is derivative value found from xi and the numerical value, yi. I.e., fi = f(xi, yi).

50

More Notation

• y(xi) is the exact value of y at x = xi

– Usually not known but notation is used in error analysis of algorithms

• f(xi,y(xi)) is the exact value of the

derivative at x = xi

• e1 = y(x1) – y1 is the local truncation error

– This is error for one step of algorithm starting from known initial condition

51

Local versus Global Error• At the initial condition we know the

solution, y, exactly• First step introduces some error• Remaining steps have single step error

plus previous accumulated error• Ej = y(xj) – yj is global truncation error

– Difference between numerical and exact solution after several steps

– This is the error we want to control

52

Euler’s Method• Simplest algorithm, example used for

error analysis, not for practical use• Define favg = f(xi, yi)• Euler’s method algorithm is yi+1 = yi + hifi

= yi + hi f(xi, yi)• Example dy/dx = x + y, y = 0 at x = 0 • Choose h = 0.1• We have x0 = 0, y0 = 0, f0 = x0 + y0 = 0 x1

= x0 + h = 0.1, y1 = y0 + hf0 = 0

53

Euler Example Continued

• Next step is from x1 = 0.1 to x2 = 0.2

• f1 = x1 + y1 = 0.1 + 0 = 0.1

• y2 = y1 + hf1 = 0 + (.1)(.1) = .01

• For dy/dx = x + y, we know the exact solution is (x0 + y0 + 1)ex-x

0 – x – 1

• For x0 = y0 = 0, y = ex – x – 1

• Look at application of Euler algorithm for a few steps and compute the error

54

Euler Example

xi yi fi y(xi) E(xi)

0 0 0 0 0

.1 0 .1 .0052 .0052

.2 .01 .21 .0214 .0114

.3 .031 .331 .04986 .01886

.4 .0641 .4641 .091825 .027725

55

Euler Example Plot

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

x

y Exact

Numerical

56

Error Propagation

• Behavior of Euler algorithm is typical of all algorithms for numerical solutions

• Error grows at each step

• We usually do not know this global error, but we would like to control it

• Look at local error for Euler algorithm

• Then discuss general relationship between local and global error

57

Taylor Series to Get Error

• Expand y(x) in Taylor series about x = a

....)-(!3

1)-(

!2

1)()()( 3

3

32

2

2

axdx

ydax

dx

ydax

dx

dyayxy

axaxax

• Look at one step from known initial condition, a = x0, to x0 + h so x – a = h

....!3

1

!2

1)()( 3

03

32

02

2

000 h

dx

ydh

dx

ydh

dx

dyxyhxy

• In ODE notation, dy/dx|0 = f(x0, y(x0))

58

Local Euler Error

• Result of Taylor series on last chart

....!3

1

!2

1))(,()()( 3

0

3

32

0

2

2

0000 hdx

ydh

dx

ydxyxhfxyhxy

• This is only the Euler algorithm for the first step when we know f(x0,y(x0))

• This gives the local truncation error• Local truncation error for Euler algorithm

is second order

Euler Algorithm Truncation Error

59

Global Error

• We will show that a local error of order n, has a global error of order n-1

• To show this consider the global error at x = x0 + kh after k algorithm steps

– Is approximately k times the local error– If local error is O(hn), approximate global

error after k steps is k O(hn) kAhn

– A new step size, h/r, takes kr steps to get to the same x value

60

Global Error Concluded

• Compare error for same x = kh with step sizes h and h/r

• Ex=kh(h) kAhn

• Ex=kh(h/r) = krA(h/r)n

1

1

)(

nn

n

khx

khx

rhkrhkr

hErhE

• When we reduce the step size by a factor of 1/r we reduce the error by a factor of 1/rn-1; this is the behavior of an algorithm whose error is order n-1

61

Euler Local and Global Error

• Previously showed Euler algorithm to have second order local error

• Should have first order global error

• Results for previous Euler example at x = 1 with different step sizes

Step size First step Final errorh = 0.1 5.17x10-3 1.25 x10-1

h = 0.01 5.02 x10-5 1.35 x10-2

h = 0.001 5.00 x10-7 1.36 x10-3

62

Better Algorithms• Seek high accuracy with low

computational work• Could improve Euler accuracy by

cutting step size, but this is not efficient• Use other algorithms that have higher

order errors• Runge-Kutta methods typically used

– This is a class of methods that use several function evaluation methods per step

63

Second-order Runge Kutta

• Huen’s method

• Modified Euler method

2

),(),(),(

2

),(0

1110

1011

11

1110

1

iiiiiiiii

iii

iiiiiiii

yxfhyyyxfyxf

hyy

hxxyxfhyy

),(

2),(

2

21

2111

1

21

1

21

iiiii

iiiii

iii

yxfhyy

hxxyxf

hyy

64

Fourth-order Runge Kutta

• Uses four derivative evaluations per step

),(

2,2

2,2

),(6

22

3114

2113

1112

11

114321

1

kyhxfhk

kyhxfhk

kyhxfhk

yxfhk

hxxkkkk

yy

iiii

ii

ii

ii

ii

iii

iiiii

65

Comparison of Methods• Look at Euler, Heun, Modified Euler and

fourth-order Runge-Kutta • Solve dy/dx = e-y-x with y(0) = 1• Compare numerical values to exact

solution y = ln( ey0 + e-x

0 – e-x)• Look at errors in the methods at x = 1

as a function of step size• Compare error propagation (increase in

error as x increases)

66

Error versus Step Size for Simple ODE Solvers

1.E-16

1.E-15

1.E-14

1.E-13

1.E-12

1.E-11

1.E-10

1.E-09

1.E-08

1.E-07

1.E-06

1.E-05

1.E-04

1.E-03

1.E-02

1.E-01

1.E+00

1.E-07 1.E-06 1.E-05 1.E-04 1.E-03 1.E-02 1.E-01

Step size, h

Err

or Euler

Huen

Modified Euler

Runge-Kutta

01 00 xxatyywithedxdy yx

67

Error Propagation in Solutions of ODEs

1.E-14

1.E-13

1.E-12

1.E-11

1.E-10

1.E-09

1.E-08

1.E-07

1.E-06

1.E-05

1.E-04

1.E-03

1.E-02

1.E-01

1.E+00

0 0.2 0.4 0.6 0.8 1

x

Err

or

Euler (N = 10)

Euler (N = 100)

Euler N = 1,000)

Huen (N =10)

Huen (N = 100)

Huen (N = 1,000)

RK4 (H = 10)

RK4 (N = 100)

68

Some Basic Concepts• A numerical method is convergent with

the solution of the ODE if the numerical solution approaches the actual solution as h 0 (with increase in numerical precision at smaller h)– Mainly a theoretical concept; algorithms in

use are convergent

• Stability refers to the ability of a numerical algorithm to damp any errors introduced during the solution

69

More on Stability• Finite difference equations in numerical

algorithms, when iterated, may be numerically increase without bound

• Stability usually is obtained by keeping step size h small, sometimes smaller than the h required for accuracy

• For most ODEs stability is not a problem, but it is for stiff systems of ODEs and for partial differential equations

70

Euler Solution of Decaying Exponential

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

0 0.5 1 1.5 2 2.5 3x

y

y(0) = 1

y(0) = 2

y(0) = 3

y(0) = 4

y(0) = 5

Euler

Euler algorithm errors move numerical solution of dy/dx = -y

to solution for anohter initial condition. Error bounded by

decreasing exponential

71

Euler Solution for Increasing Exponential

0

10

20

30

40

50

60

70

80

90

100

0 0.5 1 1.5 2 2.5 3

x

y

y(0) = 1

y(0) = 2

y(0) = 3

y(0) = 4

y(0) = 5

Euler

Euler algorithm errors move numerical solution of dy/dx = -y

to solution for anohter initial condition. Error becomes large

for increasing exponential

72

Error Control

• How do we choose h?

• Want to obtain a result with some desired small global error

• Can just repeat calculations with smaller h until two results are sufficiently close

• Can use algorithms that estimate error and adjust step size during the calculation based on the error

73

Runge-Kutta-Fehlberg• Uses two equations to compute yn+1,

one has O(h5), the other O(h6) error• Requires six derivative evaluations per

step (same evaluations used for both equations)

• The error estimate can be used for step size control based on an overall 5th order error

• Cask-Karp version different coefficients

74

Runge-Kutta-Fehlberg II• See page 377 in Hoffman for algorithm• Typical formula components below

• yn+1= yn + (16k1/135 + 6656k2/12825 …

• k3 = hf(xn + 3h/8, yn + 3k1/32 + 9k2/32)

• Error = k1/360 - 128k3/4275 …

• hnew = hold|EDesired/Error|1/5

• EDesired is set by user

• RKF45 code by Watts and Shampine

75

Other Error Controls

• Do fourth-order Runge-Kutta two times, with step sizes different by a factor of two and compare results

• Runge-Kutta-Verner is similar to Runge-Kutta-Fehlberg, but uses higher order expressions

76

Systems of Equations• Any problem with one or more ODEs of

any order can be reduced to a system for first order ODEs

• Last week we reduced a system of two second order equations to a system of four first order equations

• In this process the sum of the orders is constant

• Example from last week

77

Reduction of Order Example

• Define variables y3 = dy1/dt and y4 = dy2/dt • Then dy3/dt = d2y1/dt2 and dy4/dt = d2y2/dt2 • Have four simultaneous first-order ODEs

00 22

231

2

22

22

21

21

1

212

12

ym

kky

m

k

dt

ydy

m

ky

m

kk

dt

yd

),(

),(

),(),(

422

231

2

24

321

21

1

213

242

131

y

y

yy

xfym

kky

m

k

dt

dy

xfym

ky

m

kk

dt

dy

xfydt

dyxfy

dt

dy

78

Solving Simultaneous ODEs

• Apply same algorithms used for single ODEs– Must apply each step and substep to all

equations in system

– Key is consistent determination of fi(x,y)

– All yi values in y must be available at the same x point when computing the fi

– E.g., in Runge-Kutta we must evaluate k1 for all equations before finding k2

79

Runge-Kutta for ODE System – y(i) is vector of dependent variables at x = xi

– k(1), k(2), k(3), and k(4), are vectors containing intermediate Runge-Kutta results

– f is a vector containing the derivatives

– k(1) = hf(xi, y(i))

– k(2) = hf(xi + h/2, y(i) + k(1)/2)

– k(3) = hf(xi + h/2, y(i) + k(2)/2)

– k(4) = hf(xi + h, y(i) + k(3))

– y(i+1) = (k(1) + 2k(2) + 2k(3) + k(4))/6

80

ODE System by RK4

• dy/dx = -y + z and dz/dx = y – z with y(0) = 1 and z(0) = -1 with h = .1

• k(1)y = h[-y + z] = 0.1[-1 + (-1)] = -.2

• k(1)z = h[y - z] = 0.1[1 - (-1)] = .2

• k(2)y = h[-(y+ k(1)y/2) + z + k(1)z/2] = 0.1[ -(1 + -0.2/2) + (-1 + .2/2)] = -.18

• k(2)z = h[(y+ k(1)y/2) –(z + k(1)z/2)] = 0.1[(1 + -0.2)/2 - (-1 + .2/2)] = .18

81

ODE System by RK4 II

• k(3)y = h[-(y+ k(2)y/2) + z + k(2)z/2] = 0.1[ -(1 + -0.18/2) + (-1 + .18/2)] = -.182

• k(3)z = h[(y+ k(2)y/2) –(z + k(2)z/2)] = 0.1[(1 + -0.18)/2 - (-1 + .18/2)] = .182

• k(4)y = h[-(y+ k(3)y) + z + k(3)z] = 0.1[ -(1 + -0.182) + (-1 + .182)] = -.1636

• k(4)z = h[(y+ k(3)y) –(z + k(3)z)] = 0.1[ (1 + -0.182) - (-1 + .182)] = .1636

82

ODE System by RK4 III

• yi+1 = yi + (k(1)y + 2k(2)y + 2k(3)y + k(4)y )/6 = 1 + [ (–.2) + 2(–.18) + 2(–.182) + (–.1636)]/6 = .8187

• zi+1 = zi + (k(1)z + 2k(2)z + 2k(3)z + k(4)z )/6 = –1 + [(.2) + 2(.18) + 2(.182) + (.1636)]/6 = –.8187

• Continue in this fashion until desired final x value is reached

• No x dependence for f in this example

83

Numerical Software for ODEs

• Usually written to solve a system of N equations, but will work for N = 1

• User has to code a subroutine or function to compute the f array– Input variables are x and y; f is output – Some codes have one dimensional

parameter array to pass additional information from main program into the function that computes derivatives

84

Derivative Subroutine Example

• Visual Basic code for system of ODEs at right is shown below

Sub fsub( x as Double, y() as Double, _ f() as Double )f(1) = -y(1) + Sqr(y(2)) + y(3)*Exp(2*x)

f(2) = -2 * y(1)^2f(3) = -3 * y(1) * y(2)

End Sub

2132

12

2321

1

32 yydx

dyy

dx

dy

eyyydx

dy x

85

Derivative Subroutine Example

• Fortran code for system of ODEs at right is shown below

subroutine fsub( x, y, f ) real(KIND=8) x, y(*), f(*)f(1) = -y(1) + sqrt(y(2)) +y(3)*exp(2*x)

f(2) = -2 * y(1)**2f(3) = -3 * y(1) * y(2)

end subroutine fsub

2132

12

2321

1

32 yydx

dyy

dx

dy

eyyydx

dy x

86

Derivative Subroutine Example

• C++ code for system of ODEs at right is shown below

void fsub(double x, double y[], double f[])

{ f[1] = -y[1] + sqrt(y[2]) + y[3]*exp(2*x);

f[2] = -2 * y[1] * y[1];f[3] = -3 * y[1] * y[2];

}

2132

12

2321

1

32 yydx

dyy

dx

dy

eyyydx

dy x