Approximations and Numerical Methods

Kenny TM∼

August 20, 2006

1 Why do we need approximation?

Approximation means an inexact representation of something that is still closeenough to be useful. In physics and all other kinds of sciences use approxima-tion extensively.

Why do we need approximation? All natural scientific theories, unlikemathematical one, derived from observations of experiments. As apparatuscannot made exact, we can only obtain an estimation of what is going on.Ensuring exact in calculation would be nitpicking.

Approximation usually leads to much simpler expression, without losing alot of information or making excessive assumption. Therefore it is used a lot oftime. Sometimes an exact solution is even impossible, and approximation is amust.

2 Some common approximations

The following shows some common approximations, when x is much less than1, usually written as x� 1. I will state why these approximations are possiblein the next section.

1

(1 + x)n≈ 1 + nx (1)

sin x ≈ x (2)

cos x ≈ 1 −12

x2≈ 1 (3)

tan x ≈ x (4)ex≈ 1 + x (5)

ln(1 + x) ≈ x (6)

For example, when a� b, i.e., a is much greater than b, then

(a + b)n = an(1 +

ba

)n

(7)

≈ an(1 +

nba

). (8)

Approximates can be applied recursively. For example, the

1ex + x

≈1

1 + x + x

=1

1 + 2x≈ 1 − 2x.

A practical example is the simple pendulum. The equation of motion of asimple pendulum is

ml2θ̈ = −mgl sinθ.

However, this ODE cannot be solved with elementary solution. Nevertheless,if we restrict the range of θ to be θ� 1, then the equation can be simplified to

ml2θ̈ = −mglθ

as sinθ ≈ θ. We can easily identify it as an SHM, and the angular frequency is√gl .

3 Taylor series

Taylor series is a direction of how approximation can be done.

2

Suppose we want to approximate a given function f (x) near x = x0. Theeasiest approximation is f (x) ≈ f (x0). This is known as the zeroth order approx-imation. The result of this approximation is often unsatisfactory. A better oneis to use a line to approximate it. The line should, of course, pass through(x0, f (x0)) and has a slope f ′(x0) so that the line is tangent to the curve. Usingthe usual point-slope form we have

f (x) ≈ f (x0) + f ′(x0)(x − x0). (9)

This is known as the first order approximation. This approximation is the mostwidely used one, since the form is linear and thus easy enough to interpret, yetdoes not deviate much from the real function.

y

x0th

1st

Figure 1: Zeroth and first order approximation.

It is possible, though, that first order approximation is not enough. We canmake it more precise by approximating with a parabola, thus yielding a secondorder approximation

f (x) ≈ f (x0) + f ′(x0)(x − x0) +12

f ′′(x0)(x − x0)2, (10)

and, in general, the n-th order approximation,

f (x) ≈n∑

k=0

1k!∂k f∂xk

∣∣∣∣∣∣x=x0

(x − x0)k (11)

3

If n = ∞, the right hand side is an exact representation of f . This is called theTaylor series, or Taylor expansion of f . In particular, if x0 = 0, then the expressionreduced to

f (x) ≈n∑

k=0

1k!∂k f∂xk

∣∣∣∣∣∣x=0

xk

which is known as the Maclaurin series.

All the approximates in the last section are in fact first order approximations(except the cosine one, which is second order). If you substitute everything into(9), you will find how they are derived.

Example: Find the first order approximation of cos−1 x near x = 0. Using(9),

cos−1 x ≈ cos−1 0 +d

dxcos−1 x

∣∣∣∣∣x=0

x

=π2− x.

4 Non-numerical approximation

Approximation does not appear just in formulae, but overwhelmingly a lotmore on assumptions of the system. Here gives some common assumptions:

Symmetry — Ignoring defect in a system, and assumes a certain kind of sym-metrym, since symmetric objects often eliminated the need of specialadvanced functions. Examples are:

Spherical symmetry — the Earth is a sphere.

Circular symmetry — the trajectory of Moon around the Earth is a circle.

Infinity — When one observes an object close enough to it, it seems to ex-tend to infinity. Assuming it as infinite or semi-infinite usually providesimplification and new symmetry, if the calculation really converges atall.

Infinite rod — a straight rod is infinite in length. It then exhibits transla-tional symmetry along the direction of the rod, and rotational sym-metry about the rod1.

Infinite plane — a flat sheet is infinite in area. It then exhibits transla-tional symmetry, and rotational symmetry about the normal direc-tion of the plane.

1The rotational symmetry is always there even if the rod is not infinite, though

4

Lattice — a lattice fills the whole space. It again exhibits translationaland rotational symmetry.

Infinitesimal — the collision time between two objects are infinitesimal.

Averaging — a fast-varying quantity can be replaced by its average value,which is constant, since we cannot catch up with its pace. For instance,in AC circuit the power consumed by a load calculated using P = IV is infact an averaged value.

Perfect — a fluid with zero viscosity, a gas totally satisfy the ideal gas law, amirror with infinite optical density to reflect all incident light.

Uniform — the density and temperature of a fluid is uniformly distributed.

Constant — the acceleration of a car, and due to Earth’s gravity also, is constant,a heater providing constant power, etc.

Point mass/charge — an object can be assumed as a point mass or point chargefor easy calculation. For the former, the rotating motion of a finite-sizedmass can be ignored. For the latter, the irregularity of electric field canbe neglected. Also, when one is far enough from a finite object, it can beconsidered as a point also.

Vacuum — the atmosphere is often taken the same as vacuum, ignoring theair friction and refractive index, etc.

Isolated system — no energy lost or gained from surroundings, no externalforce or any other influence from the outer space, etc.

Perpetual — a system that has been running infinitely long, and will runinfinitely long more, such as SHM and waves.

Classical physics — applying classical physics to quantum-scaled object, e.g.explaining the diamagnetism of material.

5 Numerical root-finding

When the system is so complicated that approximation cannot simplify every-thing down-to-earth, and no solution found, we will have to apply numericalmethods. Numerical methods are methods to find a numerical, not analyticalanswer, which is very often the case encountered by engineers. The usual nu-merical methods needed in this level is to solve an algebraic equation, evaluatean integral, or solve an ODE. In this notes we will only deal with root-finding,which is the most frequently needed one.

5

5.1 Iterative method

Suppose we have an equation, x = f (x), how do we solve it?

The simple analysis is, if x = f (x), then x = f ( f (x)) = f ( f ( f (x))) = . . . adinfitinum. The iterative method is built upon this. The general procedure is

1. Write the equation to solve in the form x = f (x)

2. Make a reasonable guess of x, say, x = x0.

3. Calculate x1 = f (x0).

4. Continue with xn+1 = f (xn)

5. Stop when xn reaches the precision needed.

Example: find the solution to f (x) = x7− x13 = 1. This equation cannot

be solved algebraically, and hence numerical method is needed. Firstly, werewrite the equation into

x = x7− x13 + x − 1

so that iterative method can be done. Now, when x = −2, f (x) = 8064 > 1; whenx = 2, f (x) = −8064 < 1, therefore the solution should lie somewhere between−2 and 2. Let’s suppose x0 = 0. Then applying iterative method we have

n xn

0 0

1 −1

2 −2

3 2.04516

4 8061

5 −6.06815 × 1050

It seems not working! It is one of the big problem of iterative methods and anyother numerical root-finding methods — if you chose the wrong starting point,the method fails. Another reason of failing maybe that our x = f (x) is not quitegood. Observe that when |x| > 1, the expression blows up extremely fast, andthat’s why the method fails also. Instead, if we write

x =13√

x7 − 1,

6

then the following table is obtained:

n xn

0 0

1 −1

2 −1.05477

3 −1.07144

4 −1.07694

5 −1.07878

6 −1.07941

7 −1.07962

8 −1.07969

Much nicer! We see that x stablizes around−1.080, and we may say the solutionis x ≈ −1.080.

y

xx0x1 x2x3 x4x5 x6x7 x8x9

Figure 2: Iterative method, where it fails.

5.2 Bisection method

Owing to the unstability of iterative method, another method, the bisectionmethod is used when it fails.

7

y

xx0 x1x2 x3x4x5

Figure 3: Iterative method, where it succeeds.

The basic idea of bisection method is that, if we found that f (a) < 0 andf (b) > 0 for a continuous function f , then then must be a point x such thata < x < b and f (x) = 0.

As a numerical method, we often pick the x that lies exactly in the middleof a and b, i.e., x = a+b

2 . If f (x) = 0, we are done. More often it is not, and weneed to find another x to test. However, we can actually half the interval lengthby knowing the sign of f (x) — if f (x) and f (a) are of the same sign, then theroot must appear in the interval (x, b). Otherwise, it is in (a, x). By keeping thehalving, we could eventually reach the exact root.

The general procedure of using bisection method to solve an equation is

1. Rewrite the equation into the form f (x) = 0.

2. Make a reasonable guess of x−2 and x−1 such that x−2 < x−1 and f (x−2) andf (x−1) are of different sign.

3. Take x0 as the average value of x−2 and x−1, i.e., x0 =x−2+x−1

2 .

4. Evaluate f (x0).

• If f (x0) f (x−2) < 0, then set x1 =x−2+x0

2 .

• If f (x0) f (x−1) < 0, then set x1 =x−1+x0

2 .

5. Continue with evaluating f (xn) and comparing the signs with f (xn−1) andf (xn−2).

8

6. Stop when xn reaches the precision needed.

Example: find the solution to f (x) = x7−x13 = 1. Firstly the equation should

be rewritten asg(x) = x7

− x13− 1 = 0.

Again we knew f (−2) > 0 and f (2) < 0, hence we set x−2 = −2 and x−1 = 2. Andthus x0 = 0. Continue gives the following table:

n xn f (xn)

−2 −2 +

−1 2 −

0 0 −

1 −1 −

2 −1.5 +

3 −1.25 +

4 −1.125 +

5 −1.0625 −

6 −1.09375 +

7 −1.078125 −

8 −1.0859375 +

9 −1.08203125 +

10 −1.080078125 +

we can see that x settles down around−1.08, and therefore x ≈ −1.08. Comparedwith iterative method, bisection method can always find the root, because itjust bracket the interval where the root exists, but using iterative method thesuccessive values may diverge away. On the other hand, bisection method isslow, as we can see, we repeated 10 times in bisection method to arrive at −1.08,but only 4 in (the successful) iterative method.

5.3 Newton’s method

Newton’s method is a numerical method that is more stable than the iterativemethod, and much faster than bisection method.

Newton’s method can be considered as the application of first order ap-proximation. Given a guess x0 to the equation f (x) = 0, we first the first order

9

y

x

Figure 4: Bisection method.

approximation of f near x = x0. Then, we find the x-intercept of this line, whichis supposed to be the zero of f . Of course it is not, but this new x can be usedto approximate another line, and eventually the exact solution can be found.

The general procedure is:

1. Rewrite the equation into the form f (x) = 0.

2. Make a reasonable guess of x0.

3. Define xn+1 = xn −f (xn)f ′(xn) .

4. Repeat until xn is precise enough.

Example: find the solution to f (x) = x7−x13 = 1. Firstly the equation should

be rewritten as g(x) = x7− x13

− 1 = 0. Therefore g′(x) = 7x6− 13x12. Using

10

Newton’s method with an initial guess x0 = −1, then

n xn

0 −1

1 −1.166666667

2 −1.113191097

3 −1.08615279

4 −1.080008019

5 −1.079731886

6 −1.079731352

7 −1.079731352

8 −1.079731352

We can say that x ≈ −1.079731352 then.

Notice how Newton’s method converges to the solution. Once it foundwhere the solution is, the error diminishes quadratically, i.e., the precision isdoubled each iteration. Compared with bisection method, whose speed is onlylinear, i.e., the precision is increasing in a steady pace.

Despite Newton’s method’s great speed, it still has limitations where us-ing such method is not prefered. Newton’s method involves derivative of afunction. If the derivative is difficult to calculate, we cannot use Newton’smethod efficiently. Also, if f ′(xn) is very small, then the next term may be sentto somewhere totally irrelevant, and the method will fail.

5.4 Using your calculator wisely

Modern scientific calculators often include numerical integration as a built-infunction, and newer models include also numerical differentiation and evenNewton’s method. Using these calculators we can eliminate the need of ana-lyzing the behavior of the function, and pay more attention to the real physicalproblems. Some calculators even provide the programming (macro) functionso that repetative tasks can be done automatically. Table 1 lists calculatorswhere numerical methods are built-in.

11

y

xx0

x1

x2

Figure 5: Newton’s method.

Model∫

f (x)dx d fdx

∣∣∣∣x=x0

f (x) = 0 Prog

Casio fx-50F

Casio fx-3900Pv X X

Casio fx-3650P X X X

Casio fx-991MS* X X X

Casio fx-991ES* X X X

Sharp EL-506V X X •

Citizen SRP-285II •

Hewlett Packard HP-30S •

Table 1: Comparison of calculators. Note: “*” means the calculator is notHKEAA approved. “•” means programming function is limited,and no complex programs can be written.

12