11

Chapter 3 Chapter 3 Root FindingRoot Finding

2

3.1 The Bisection Method3.1 The Bisection Method

► Let Let ff be a continues function. Suppose we know that be a continues function. Suppose we know that ff((aa) ) ff((bb) < 0) < 0, th, then there is a root between en there is a root between aa and and bb..

3

Example 3.1Example 3.1

► A formal statement is given in Algorithm 3.1.A formal statement is given in Algorithm 3.1.

4

5

Theorem 3.1 Bisection Theorem 3.1 Bisection Convergence and ErrorConvergence and Error

6

7

Bisection MethodBisection Method

► Advantage:Advantage: A global methodA global method: it always converge no matter : it always converge no matter

how far you start from the actual root.how far you start from the actual root.

►Disadvantage:Disadvantage: It cannot be used to find roots when the It cannot be used to find roots when the

function is tangent is the axis and does not function is tangent is the axis and does not pass through the axis.pass through the axis.►For example: For example:

It It converges slowlyconverges slowly compared with other compared with other methods.methods.

8

3.2 Newton’s Method: 3.2 Newton’s Method: Derivation and ExamplesDerivation and Examples

►Newton’s method is the classic Newton’s method is the classic algorithm for finding roots of functions.algorithm for finding roots of functions.

►Two good derivations of Newton’s Two good derivations of Newton’s method:method: Geometric derivationGeometric derivation Analytic derivationAnalytic derivation

9

Newton’s Method :Newton’s Method :Geometric DerivationGeometric Derivation

10

Newton’s Method :Newton’s Method :Geometric DerivationGeometric Derivation

► The fundamental idea in Newton’s method is to use The fundamental idea in Newton’s method is to use the tangent line approximation to the functionthe tangent line approximation to the function ff at at point .point .

► The point-slope formula for the equation of the The point-slope formula for the equation of the straight line gives us:straight line gives us:

► Continue the process with another straight line to getContinue the process with another straight line to get

11

Newton’s Method :Newton’s Method :Analytic DerivationAnalytic Derivation

12

Example 3.2Example 3.2

13

14

Newton’s MethodNewton’s Method► Advantage:Advantage:

Very fastVery fast

►Disadvantage:Disadvantage: Not a global methodNot a global method

►For example: Figure 3.3 (root For example: Figure 3.3 (root xx = 0.5)= 0.5)

►Another example: Figure 3.4 (root Another example: Figure 3.4 (root xx = 0.05) = 0.05)► In these example, the initial point should be carefully In these example, the initial point should be carefully

chosen.chosen.

Newton’s method will cycle indefinitely.Newton’s method will cycle indefinitely.►Newton’s method will just hop back and forth between Newton’s method will just hop back and forth between

two values.two values.►For example: Consider (root For example: Consider (root xx = 0)= 0)

15

16

Very close to the actual root

Initial value

Wrong predictions,because the root

is positive

17

3.3 How to Stop Newton’s 3.3 How to Stop Newton’s MethodMethod

► Ideally, we would want to stop when the error Ideally, we would want to stop when the error is sufficiently small. is sufficiently small.

(p. 12)

0)( f

)()('

)(1

1

1nn

n

n xxxf

xf

18To make sure f(xn) is also small enough

19

3.4 Application:3.4 Application:Division using Newton’s MethodDivision using Newton’s Method► The purpose is to illustrate the use of The purpose is to illustrate the use of

Newtown’s method and the analysis of the Newtown’s method and the analysis of the resulting iteration.resulting iteration.

f ’(x)f (x)

20

►Questions:Questions: When does this iteration converge and how When does this iteration converge and how

fast?fast? What initial guesses What initial guesses xx00 will work for us? will work for us?

► The way that computer stores numbers:The way that computer stores numbers:

21

xaxf

1)( a

rx

ae n

nn 1

|||)1

(|

|)21

(||21

|

|))2((1

||1

|||

22

22

2

11

nn

nnnn

nnnn

aexa

a

xxaa

aaxxa

axxa

xa

e

n

rr

a

ar

en

nn 2

0|||1

|||

axax

xax

axr

211 if

011 if

1|1|1||

00

00

00

22

From (2.11) p.53

p.56

Initial x0

1)1( and 2)2

1(

1)( ff

xxf

23

Example 3.3Example 3.3

24

3.5 The Newton Error 3.5 The Newton Error FormulaFormula

25

Definition 3.1Definition 3.1

► The requirement that The requirement that CC be nonzero and finite actually fo be nonzero and finite actually forces rces pp to be a single unique value. to be a single unique value.

► Linear convergence: Linear convergence: pp = 1 = 1► Quadratic convergence: Quadratic convergence: pp = 2 = 2► Superlinearly convergence: butSuperlinearly convergence: but

26

Example 3.6Example 3.6

1|| 0 x

27

3.6 Newton’s Method:3.6 Newton’s Method:Theory and ConvergenceTheory and Convergence

► Its proof is shown at pp. 106-108.Its proof is shown at pp. 106-108.

28

3.7 Application: 3.7 Application: Computation of the Square RootComputation of the Square Root

29

►Questions:Questions: Can we find an initial guess such that Newton’s Can we find an initial guess such that Newton’s

method will always converge for method will always converge for bb on this on this interval?interval?

How rapidly will it converge?How rapidly will it converge?

► The Newton error formula (3.12) applied to The Newton error formula (3.12) applied to : :

(3.25) (3.25)

► The relative error satisfiesThe relative error satisfies (3.26)(3.26)

30

relative relative errorerror

31

How to find the initial value?How to find the initial value?► Choose the midpoint of the intervalChoose the midpoint of the interval

For example: For example:

If ,If ,

► Using linear interpolationUsing linear interpolation For example:For example:

b is known

32

3.8 The Secant Method:3.8 The Secant Method:Derivation and ExamplesDerivation and Examples

► An obvious drawback of Newton’s method is that it An obvious drawback of Newton’s method is that it requires a formula for the derivative of requires a formula for the derivative of ff..

► One obvious way to deal with this problem is to use an One obvious way to deal with this problem is to use an approximation to the derivative in the Newton formula.approximation to the derivative in the Newton formula. For example: For example:

► Another method: the secant methodAnother method: the secant method Used a secant lineUsed a secant line

33

The Secant MethodThe Secant Method

34

The Secant MethodThe Secant Method

35

The Secant MethodThe Secant Method

► Its advantages over Newton’s method:Its advantages over Newton’s method: It not require the derivative.It not require the derivative. It can be coded in a way requiringIt can be coded in a way requiring only only a single function a single function

evaluation per iteration. evaluation per iteration. ►Newton’s requires two, one for the function and one for Newton’s requires two, one for the function and one for

the derivative.the derivative.

36

Example 3.7Example 3.7

37

38

Error EstimationError Estimation

► The error formula for the secant method:The error formula for the secant method:

39

The ConvergenceThe Convergence

► This is almost the same as Newton’s This is almost the same as Newton’s method.method.

40

3.9 Fixed-point Iteration3.9 Fixed-point Iteration

► The goal of this section is to use the added understanding of The goal of this section is to use the added understanding of simple iteration to simple iteration to enhanceenhance our understanding of and ability our understanding of and ability to solve root-finding problems.to solve root-finding problems.

► The root of The root of f f is equal to the fixed-point of is equal to the fixed-point of gg..

root

41

Fixed-point IterationFixed-point Iteration► Because show that this kind of point Because show that this kind of point

is called a is called a fixed pointfixed point of the function of the function gg, and an iteration of , and an iteration of

the form (3.33) is called the form (3.33) is called a fixed-point iterationa fixed-point iteration for for gg. .

42

Fixed point

Root

43

44

45

46

Example 3.8Example 3.8

47

48

g (x)

49

Theorem 3.5Theorem 3.5

50

Theorem 3.5 (con.)Theorem 3.5 (con.)

51

52

3.10 Special Topics in Root-3.10 Special Topics in Root-finding Methodfinding Method

► 3.10.1 Extrapolation and Acceleration3.10.1 Extrapolation and Acceleration The examples have some mistakes, so we jump The examples have some mistakes, so we jump

this subsection. this subsection.

53

3.10.2 Variants of Newton’s 3.10.2 Variants of Newton’s MethodMethod

► Newton’s method v.s. the chord methodNewton’s method v.s. the chord method

v.s.v.s.

► How much do we lose?How much do we lose? The chord method is only linear, and only locally convergent.The chord method is only linear, and only locally convergent.

► The chord method is useful in solving nonlinear systems of equatioThe chord method is useful in solving nonlinear systems of equations.ns.

► One interesting variant of the chord method updates the point at One interesting variant of the chord method updates the point at which the derivative is evaluated, but not every iteration.which the derivative is evaluated, but not every iteration.

54

Example 3.12Example 3.12

55

56

Other Approximations to the Other Approximations to the DerivativeDerivative

► In Section 3.8, a method using a finite difference In Section 3.8, a method using a finite difference approximation to the derivative in Newton’s method.approximation to the derivative in Newton’s method.

Only linear convergence (shown on pages 133 to 134)Only linear convergence (shown on pages 133 to 134)

57

3.10.3 The Secant Method:3.10.3 The Secant Method:Theory and ConvergenceTheory and Convergence

The proof is shown on pages 136 to 139. You can study it by yourselves.

58

3.10.4 Multiple Roots3.10.4 Multiple Roots

► So far our study of root-finding methods has assumed that So far our study of root-finding methods has assumed that the derivative of the function does not vanish at the root:the derivative of the function does not vanish at the root:

► What happens if the derivative does vanish at the root?What happens if the derivative does vanish at the root?

59

Example 3.13Example 3.13

-1

60

L’Hopital’s Rule for forms of typL’Hopital’s Rule for forms of type 0/0e 0/0

.or ,,, symbols theofany for standmay Here

)('

)('lim

)(

)(lim

thensense, infiniteor finite either the in exists )]()([lim If

.0)(lim)(lim that Suppose

aaau

xg

xf

xg

xf

x/g'xf'

xgxf

uxux

ux

uxux

61

Another example (Another example (ff((xx)=1-)=1-xexe1-1-xx))

62

Another example (Another example (ff((xx)=1-)=1-xexe1-1-xx))

► The data (Table 3.10a) suggests that both iterations are The data (Table 3.10a) suggests that both iterations are converging, but neither one is converging as rapidly as we converging, but neither one is converging as rapidly as we might have expected.might have expected.

► Can we explain this?Can we explain this? The fact that will have an effect on both Newton The fact that will have an effect on both Newton

and secant methods.and secant methods. The error formulas (3.12) and (3.50) and limits (3.24) The error formulas (3.12) and (3.50) and limits (3.24)

and (3.47) all require that .and (3.47) all require that . Can we find anything more in the way of an explanation?Can we find anything more in the way of an explanation?

63

Discussion—Newton’s Discussion—Newton’s MethodMethod

► Assume Assume f f has a double root and has a double root and

► Note that we no longer have , therefore (according to Note that we no longer have , therefore (according to Theorem 3.7 page 124) we no longer have quadratic Theorem 3.7 page 124) we no longer have quadratic convergence. convergence.

64

Discussion—Newton’s Discussion—Newton’s MethodMethod

► If we change the Newton iteration to beIf we change the Newton iteration to be

now we have . (quadratic convergence)now we have . (quadratic convergence)

► More generally,More generally,

► The problem with this technique is that it requires that we The problem with this technique is that it requires that we knowknow the degree of multiplicity of the root ahead of time. the degree of multiplicity of the root ahead of time.

65

Discussion—Newton’s Discussion—Newton’s MethodMethod

► So an alternative is needed.So an alternative is needed.

► The draw back of this method is that applying Newton’s The draw back of this method is that applying Newton’s method to method to uu will require that we have a formula for the will require that we have a formula for the second second derivativederivative of of ff..

66

Discussion—Newton’s Discussion—Newton’s MethodMethod

► (3.60)(3.60)

► (3.61)(3.61)

67

Table 3.10Table 3.10

68

Discussion Discussion

► From Table 3.10, we can see the accuracy is not as good as From Table 3.10, we can see the accuracy is not as good as

past. What is going on?past. What is going on?

► Let’s look at a graph of the polynomialLet’s look at a graph of the polynomial

Fig. 3.11 shows a plot of 8000 points from this curve on the Fig. 3.11 shows a plot of 8000 points from this curve on the

interval [0.45, 0.55] (root = 0.5)interval [0.45, 0.55] (root = 0.5)

Premature convergencePremature convergence

► This is not caused by the root-finding method. It is because This is not caused by the root-finding method. It is because

using finite precision arithmetic. using finite precision arithmetic.

69

70

3.10.5 In Search of Fast Global 3.10.5 In Search of Fast Global Convergence: Hybrid AlgorithmConvergence: Hybrid Algorithm

► Bisection method: Bisection method: slow but steady and reliableslow but steady and reliable

► Newton’s method and the secant method: Newton’s method and the secant method: fast but potentially unreliablefast but potentially unreliable

► Brent’s algorithm: Brent’s algorithm: incorporate these basic ideas into an algorithmincorporate these basic ideas into an algorithm Algorithm 3.6Algorithm 3.6

71

72

Example 3.14Example 3.14

Step 3. (b)

Step 3. (c)

Step 1.

Step 1.

Step 2. (a)

Step 2. (b)

Step 2. (b)

73

Another ExampleAnother Example