numerical differentiation and integration - unc charlotte · numerical differentiation and...

Numerical Differentiation and Integration

� Numerical Differentiation

◦ Finite Differences◦ Interpolating Polynomials◦ Taylor Series Expansion◦ Richardson Extrapolation

� Numerical Integration

◦ Basic Numerical Integration◦ Improved Numerical Integration⇒ Trapezoidal, Simpson’s Rules

◦ Rhomberg Integration

ITCS 4133/5133: Numerical Comp. Methods 1 Numerical Differentiation and Integration

Numerical Differentiation and Integration

� Many engineering applications require numerical estimates ofderivatives of functions

� Especially true, when analytical solutions are not possible

� Differentiation: Use finite differences

� Integration (definite integrals): Weighted sum of function valuesat specified points (area under the curve).


Application:Integral of a Normal Distribution

◦ Represented as a Gaussian, a scaled form of f (x) = e−x2

, veryimportant function in statistics

◦ Not easy to determine indefinite integral - use numerical techniques

A =

∫ b

a

e−x2


Application:Integral of a Sinc

f (x) =sin(x)

x


Numerical Differentiation:Approach

⇒ Evaluate the function at two consecutive points, separated by ∆x, ofthe independent variable, and taking their difference:

df (x)

dx≈ f (x + ∆x)− f (x)

∆x

⇒ Fit a function to a set of points that define the relationship betweenthe independent and dependent variables, such as an nth orderpolynomial; then differentiate the polynomial


Finite Differences

Forward Difference

df (x)

dx=f (x + ∆x)− f (x)

∆x

Backward Difference

df (x)

dx=f (x)− f (x−∆x)

∆x

Two Step Method, Central Difference

df (x)

dx=f (x + ∆x)− f (x−∆x)

2∆x


Finite Differences: Example


Forward Differences: Derivation

� Finite difference formulas can be derived from the Taylor series.

f (x + h) = f (x) + hf ′(x) +h2

2f ′′(η),

For h = xi+1 − xi,

f ′(xi) =f (xi+1)− f (xi)

h− h

2f ′′(η),

where xi < η < xi+1.


Backward Differences: Derivation

f (x + h) = f (x) + hf ′(x) +h2

2f ′′f (η),

For h = xi−1 − xi,

f (xi + h) = f (xi + xi−1 − xi) = f (xi−1) = f (xi) + hf ′(xi) +h2

2f ′′f (η)

f ′(xi) =f (xi−1)− f (xi)

h− h

2f ′′(η)

where xi−1 < η < xi.


Central Differences: Derivation

Use the next higher order Taylor polynomial,

f (xi+1) = f (xi) + hf ′(xi) +h2

2f ′′(xi) +

h3

6f ′′′(η1),

f (xi−1) = f (xi)− hf ′(xi) +h2

2f ′′(xi)−

h3

6f ′′′(η2)

with x < η1 < x + h, x− h < η2 < xThus,

f (xi+1)− f (xi−1) = 2hf ′(xi) +h3

6[f ′′′(η1) + f ′′′(η2)]

or,

f ′(xi) =f (xi+1)− f (xi−1)

2h− h2

6f ′′′(η), xi−1 < η < xi + 1


Second Derivatives

f (x + h) = f (x) + hf ′(x) +h2

2f ′′(x) +

h3

3!f ′′′(x) +

h4

4!f (4)(x)(η1)

f (x− h) = f (x)− hf ′(x) +h2

2f ′′(x)− h3

3!f ′′′(x) +

h4

4!f (4)(x)(η2)

Thus

f (x + h) + f (x− h) = 2f (x) + h2f ′′(x) +h4

4![f (4)(x)(η1) + f (4)(x)(η2)]

f ′′(x) ≈ 1

h2[f (x + h)− 2f (x) + f (x− h)]

with truncation error of the O(h4)


Partial Derivatives

� Generally, interested in partial derivatives of functions of 2 variables,(xi, yj), based on a mesh of points.

� Subscripts denote partial derivatives.


Partial Derivatives (contd)

� Laplacian operator: ∆2u = uxx + uyy

� Biharmonic Operator: ∆4u = uxxxx + uxxyy + uyyyy


Using Interpolating Polynomials

� Given a function in discrete form, the data can be interpolated to fitan nth order polynomial

� For those functions that are in analytical form, but difficult to differ-entiate, the function can be discretized and fitted by a polynomial.

f (x) = bnxn + bn−1x

n−1 + · · · + b1x + b0

whose derivative is

df (x)

dx= nbnx

n−1 + (n− 1)bn−2xn−2 + · · · + b1


Power Series Type Interpolating Polynomial

Consider the nth order polynomial passing through (n+1) points; the (n+1) coefficients can be uniquely determined.

Example

f (x) = a0 + a1x + a2x2

Consider f (xi), f (xi + h), f (xi + 2h)

f (xi) = a0 + a1xi + a2x2i

f (xi + h) = a0 + a1(xi + h) + a2(xi + h)2

f (xi + 2h) = a0 + a1(xi + 2h) + a2(xi + 2h)2


Example (contd)

For the 3 points, xi = 0, h, 2h,

fi = a0

fi+1 = a0 + a1h + a2h2

fi+2 = a0 + 2a1h + 4a2h2

which has the solution,

a0 = fi

a1 =−fi+2 + 4fi+1 − 3fi

2h

a2 =fi+2 − 2fi+1 + fi

2h2


Example (contd)

f (x) = a0 + a1x + a2x2

f ′(xi) = f′

i = f ′(xi = 0)

= a1 + 2a2xi

= a1

=−fi+2 + 4fi+1 − 3fi

2h

Similarly, second derivatives can also be obtained.


Numerical Integration

� Integration can be thought of as considering some continuous func-tion f (x) and the area A subtended by it; for instance, within a par-ticular interval

A =

∫ b

a

f (x)dx

� Numerical Integration is needed when f (x) does not have a knownanalytical solution, or, if f(x) is only defined at discrete points.

� There are two approaches to a numerical solution:

⇒ Fitting polynomials to f (x) and integrating using analytical cal-culus,

⇒ Area under f (x) can be approximated using geometric shapesdefined by adjacent points.


Interpolation Approach

We can derive an nth order polynomial (for instance, Gregory-Newtonmethod), which has the general form as

f (x) = b1xn + b2x

n−1 + · · · + bnx + bn+1

After the bis are determined, this can be integrated as∫f (x)dx =

b1xn+1

n + 1+b2x

n

n+ · · · + bnx

2

2+ bn+1x

which can be solved for, within the limits of the integration


Basic Numerical Integration (Newton-CotesClosed Formulas)

� Trapezoid Rule: Approximates function by a straightline to computearea under curve.∫ b

a

f (x)dx ≈ h

2[f (x0) + f (x1)]

� Solution is exact, for polynomials of degree≤ 1, i.e. linear functions.


Basic Numerical Integration (Newton-CotesClosed Formulas)

� Simpson’s Rule: Instead of a linear approximation, use a quadraticpolynomial:

h =b− a

2, x0 = a, x1 = x0 + h =

b + a

2, x2 = b

Approximate integral is given by∫ b

a

f (x)dx ≈ h

3[f (x0) + 4f (x1) + f (x2)


Basic Simpson’s Rule: Examp1e 1


Basic Simpson’s Rule: Examp1e 2


Improved Numerical Integration

� Improve the accuracy by applying lower order methods repeatedlyon several subintervals.

� Known as composite integration

� Simpson, Trapezoid rules use equal subintervals; using unequal(adaptive) intervals leads to Gaussian Quadrature methods.


Composite Trapezoid Rule� Approximates the area under the function f (x) by a set of discrete

trapezoids fitted between each pair of points of the dependent vari-able (and the X axis)

∫ xn

x1

f (x)dx ≈n−1∑i=1

(xi+1 − xi)f (xi+1 + f (xi)

2

f(x)

f(x)

xx1 x2 x3 x4

If the intervals are the same, h = (b− a)/n, we get∫ xn

x1

f (x)dx ≈ b− a2n

[f (a) + 2f (x1) + . . . + f (b)]


Composite Trapezoid Rule:Example


Composite Trapezoid Rule:Algorithm


Composite Trapezoid Rule: Notes

⇒ f(xi+1+f(xi)2 represents the average height of the trapezoid.

⇒ Linear approximation between pairs of successive points used; erroris proportional to the distance between sample points.

Error

⇒ Error in the trapezoidal can be approximated by Taylor’s series

xi+1 − xi

2f ′′(x)

where f ′′(x) is evaluated at the value of x that maximizes the secondderivative between the two sample points.


Composite Simpson’s Rule� Uses the same idea as the composite Trapezoid rule, by using mul-

tiple sub-intervals to improve the Simpson rule approximation.

� For 2 subintervals, Simpson’s rule has the following form:∫ b

a

f (x)dx =

∫ x2

a

f (x)dx +

∫ b

x2

f (x)dx

≈ h

3[f (a) + 4f (x1) + f (x2)] +

h

3[f (x2) + 4f (x3) + f (b)]

≈ h

3[f (a) + 4f (x1) + 2f (x2) + 4f (x3) + f (b)])

Trapezoidal Rule

Simpson’s Rule

f(x)

a b(a+b)/2

f(x)

xITCS 4133/5133: Numerical Comp. Methods 29 Numerical Differentiation and Integration

Composite Simpson’s Rule (contd)

� For n (n must be even) subintervals, h = (b− a)/n we get∫ b

a

f (x)dx =h

3[f (a) + 4f (x1) + 2f (x2) + 4f (x3) +

2f (x4) + . . . + 2f (xn−2 + 4f (xn−1) + f (b)

� n must be even number of subintervals.

� Error Bound: Based on the 4th derivative:

Error =(xi+1 − xi)

5

90f (4)(xi)

where f (4)(xi) is evaluated at the value of x that maximizes the 4thderivative between the two sample points.


Composite Simpson’s Rule:Example


Composite Simpson’s Rule:Algorithm


Romberg Integration

� Simpson’s rule improves on Trapezoid rule with additional evalua-tions of f(x), required to fit a more accurate polynomial

� More evaluations will further improve the integral, leading to theRomberg Integration

I01 =b− a

2(f (a) + f (b))

A second estimate can be obtained with subdividing the interval ab into2 equal intervals is given by

I11 =b− a

2

(1

2f (a) + f (m) +

1

2f (b)

)=

1

2

[I01 + (b− a)f

(a +

b− a2

)]ITCS 4133/5133: Numerical Comp. Methods 33 Numerical Differentiation and Integration

Romberg Integration(contd.)

A third estimate, I21, using 3 intermediate points m1,m2,m3 is given by

I21 =b− a

2

[1

4f (a) +

1

2f (m1) +

1

2f (m2) +

1

2f (m3) +

1

4f (b)

]=

1

2

[I11 +

b− a2

3∑k=1,k 6=2

f

(a +

b− a4

k

)]This leads to the recursive relationship

Ii1 =1

2

[Ii−1,1 +

b− a2i−1

2i−1∑k=1,3,5

f

(a +

b− a2i

k

)], for i = 1, 2, . . .



Ii1 =1

2

[Ii−1,1 +

b− a2i−1

2i−1∑k=1,3,5

f

(a +

b− a2i

k

)], for i = 1, 2, . . .

We can combine these estimates using Richardson’s formula,

Iij =4j−1Ii+1,j−1 − Ii,j−1

4j−1 − 1

for i = 0, 1, . . . , N − j + 1, j = 1, 2, . . . N . The estimates form an uppertriangular matrix:

I01 I02 I03 I04 · · · I0,N−1 I0,N I0,N+1

I11 I12 I13 . · · · I1,N−1 I1,N

I21 I22 . . · · · I2,N−1

I31 . . . · · ·. . . . · · ·. . IN−2,3

. IN−1,2

IN1



Algorithm (To solve for I0N� Determine I01 (Trapezoid formula)

� Determine Ii1, for i = 1, 2, . . .

� Determine Ii2, Ii3, etc., using Richardson’s formula.


numerical differentiation and integration - unc charlotte · numerical differentiation and...

Documents