web view1 3 -2.842e-014. ans = 3.0000000. surprisingly, only one iteration ... at an initial guess x...

[100] 005 Chapter 5 Root and Minimum Search, Tutorial by www.msharpmath.com

[100] 005 revised on 2012.10.30 cemmath

The Simple is the Best

Chapter 5 Root and Minimum Search

5-1 Roots of a Single Nonlinear Equation5-2 Collection of Roots and Complex Root5-3 Coupled Nonlinear Equations5-4 Pseudo-Linear System5-5 Minimum Search5-6 Summary

The term ‘roots’ are used as a terminology for a set of solution to a given system of equations, either single or coupled. Unfortunately, there exist no cure-all methods to solve all kinds of nonlinear equations. In this regard, experience and gifted talent of intuition could be the most powerful approach in solving nonlinear equations. Although no guarantee of success is provided, root-finding methods available in the literature are discussed in this chapter. Also, several methods to find minimum of a function (without constraints though) are addressed.

Section 5-1 Roots of a Single Nonlinear Equation

In this section, we introduce the Umbrella ‘solve’ to find roots of a nonlinear equation of a single-variable. The default method of finding roots is based on the secant method which is a modified version of the Newton method. To aid users’ choice, several well-known methods are also provided with appropriate Spokes.

■ Umbrella ‘solve’. Our mission is to find a root x which satisfies the following equation

1


f ( x )=g (x)

where f ( x ) and g(x ) are functions of a single-variable x When g(x )≡ 0, the root is frequently called the zero of the equation f ( x )=0.

Various methods available in the literature are implemented through the Umbrella ‘solve’ and Spokes listed below

solve .x ( <<opt>>, f(x) = g(x) ) .bisect( a,b ) // bisection solve .x ( <<opt>>, f(x) = g(x) ) .secant( a,b ) // secant methodsolve .x ( <<opt>>, f(x) = g(x) ) .muller( a,b,c ) // Muller methodsolve .x ( <<opt>>, f(x) = g(x) ) .march ( a,b,dx ) // marching methodsolve .x ( <<opt>>, f(x) = g(x) ) .span[n=100,g=1](a,b) // bisections

solve .x {a} ( <<opt>>, f(x) = g(x) ) .newton ( f'(x)-g'(x) ) // Newton solve .x {a} ( <<opt>>, f(x) = g(x) ) .multiple( f'(x)-g'(x) ) // multiple

These methods will be discussed briefly together with the performance test. To avoid any confusion in using syntax, both the commands in the following

solve .x { c } ( f(x) = g(x) ) .bisect( a,b ) // bisection methodsolve .x ( f(x) = g(x) ) .bisect( a,b ) // bisection method

are exactly the same, since the Hub field { c } is eliminated by Spoke ‘bisect’.

Other Spokes which control the solution procedures are

.peep show all the iteration procedures

.peeptail show the final stage of the iteration

.return( … ) return manipulated data

.maxiter(200) maximum number of iteration

.tol/abstol(1.e-7) absolute tolerance. tol is equivalent to abstol

.reltol(1.e-7) relative tolerance

where constants inside the parenthesis are the default values, and slash means that either Spoke can be alternatively used.

■ Default Method. If not specified explicitly, the default method to find a root is the secant method near x=a. The second point x=b is selected

2


b=a+h ,h=|a|ϵ (if h<ϵ ,h=ϵ )

where ϵ=10−3. Therefore, the syntax for the default method reads

solve .x { a } ( <<opt>>, f(x) = g(x) ) // a=0.41 by default

which is simple enough for repeated use.

(Example 5-1-1) Let us find a root of the following equation

ex−3=cos x

The motive of expression-look-alike approach leads us to write

#> solve.x { 0 } ( exp(x)-3 = cos(x) );

which results in

ans = 1.2098915 ◀

(Example 5-1-2) However, for an equation ex−3=sin x, we find that

#> solve.x { 0 } ( exp(x)-3 = sin(x) );

yields the following warning

The secant method fails due to near-zero slopeans = 1.#INF000

This is because the derivative of ex−3−sin x at x=0 vanishes to zero. Therefore, we have to select another starting point, e.g.

#> solve.x { 0.5 } ( exp(x)-3 = sin(x) );ans = 1.3818344

We find successfully the root x=1.3818344. As in this example, it is crucial to select a starting point for the secant method. ◀

3


■ Spoke ‘peep’, ‘return’ and ‘tol’. Not only the root but also the procedure of root-finding may be of interest. In this case, Spoke ‘peep’ enables to watch the iteration procedure. Also, Spoke ‘tol’ controls the termination of iteration. For example

#> solve.x { 0.5 } ( exp(x)-3 = sin(x) ) .tol(0.01).peep;

shows the iteration procedure and the final root as follows.

--------- secant method -----------iter x f(x)

0 0.5 -1.831 0 0.501 -1.83 1 2.8707546 14.38 2 0.76768751 -1.54 3 0.97105291 -1.185 4 1.6498706 1.209 5 1.3069639 -0.2705 6 1.3696323 -0.04593 7 1.3824521 0.002345ans = 1.3824521

It can be seen that the total number of iterations is 7 until the absolute tolerance of 0.01 is met. Also, an error 1.3824521−1.3818344=0.0006177 has occurred.

An interesting feature of Spoke ‘return’ is that it can manipulate the final return data, even changing from scalar to matrix. If one argument is returned, the results is a scalar, otherwise a matrix. This can be confirmed by the following commands

#> solve.x { 0.5 } ( exp(x)-3 = sin(x) ) .return( x/100 ); #> solve.x { 0.5 } ( exp(x)-3 = sin(x) ) .return( x+1, x, sin(x) ); ans = 0.0138183ans = [ 2.3818 ]

[ 1.3818 ][ 0.9822 ]

Such an interesting performance of Spoke ‘return’ will be judiciously used to solve coupled quations, as can be seen later.

4


■ Bisection Method. For an interval of a ≤ x≤ b it is evident that the equation f ( x )=0 has at least one root inside the interval, if f ( a ) f (b )<0 This is the premise of the bisection method. For the same problem discussed above, let us try to apply the bisection method over an interval 1 ≤ x ≤2 by

#> solve.x ( exp(x)-3 = sin(x) ) .bisect( 1,2 ).peeptail;

which results in

----------------------- bisection method --------------------------- iter x f(x) xa xb 23 1.3818344 7.724e-008 ( 1.3818343 1.3818345 )ans = 1.3818344

A total of 23 iterations are necessary to find the root x=1.3818344.

■ Newton Method. The Newton method utilizes the derivative of function to find a root of equation ex−3=sin x

xn+1=xn−f n

f n'

where f n= f (xn) and f n' = f ' (xn). The performance of the Newton method is

examined by

#> solve.x { 2 } ( exp(x)-3-sin(x) = 0 ) .newton( exp(x)-cos(x) ).peep;

which yields

--------- Newton method ----------- iter x f(x) 0 2 3.48 1 1.5541745 0.7313 2 1.3990555 0.06608 3 1.3820259 0.0007269 4 1.3818344 9.105e-008ans = 1.3818344

5


Since the convergence rate of the Newton method is of second-order, only 4 iterations are required as above.

■ Secant Method. The secant method selects two points, x1 and x2. Then,

the slope ( f 2−f 1 ) /( x2−x1) plays a similar role of the tangent f ' (x) in the Newton method, i.e.

xn+1=xn−f n

xn−xn−1

f n−f n−1

is repeated until convergence. When |f 1|<¿ f 2∨¿, the initial direction is

reversed, i.e. x1 and x2 are interchanged. The secant method with two points x1=1 , x2=2 is tested by

#> solve.x ( exp(x)-3-sin(x) = 0 ) .secant( 1,2 ).peep;--------- secant method -----------

iter x f(x) 0 1 -1.123 0 2 3.48 1 1.2440152 -0.4776 2 1.4245115 0.1665 3 1.377849 -0.01508 4 1.3817247 -0.0004161 5 1.3818347 1.087e-006 6 1.3818344 -7.796e-011ans = 1.3818344

■ Muller Method. The Muller method employs three different points x=x1 , x2 , x3, and determines a parabola passing through three points

( x1 , f 1 ) , ( x2 , f 2 ) ,(x3 , f 3)

Then, two roots of the parabola are examined to discard one of them, and this procedure is repeated. Similarly, the performance test is done by

#> solve.x ( exp(x)-3 = sin(x) ) .muller( 0,1,2 ).peep;--------- Muller method ----------- iter x f(x) 0 0 -2

6


0 1 -1.123 0 2 3.48 1 1.3340585 -0.1757 2 1.3797657 -0.007839 3 1.3818408 2.445e-005 4 1.3818344 -4.388e-010ans = 1.3818344

Note that the convergence rate is similar to that of the Newton method.

■ Marching Method. The marching method starts from x=a and marches by an increment dx until x=b. During this process, the bisection method is applied whenever sign change is noticed. Otherwise, not-a-number (NaN) is returned. The performance test is as follows

#> solve.x ( exp(x)-3 = sin(x) ) .march( 1,2, 0.1 ).peep;--------- marching method -----------

step x f(x) 0 1 -1.123 1 1.1 -0.887 2 1.2 -0.6119 3 1.3 -0.2943 4 1.4 0.06975ans = 1.3818343

Note that sign change has occurred over an interval 1.3 ≤ x ≤1.4, and thus the bisection method is applied to find a root.

■ Multiple-Roots Method. The multiple-roots method is applied to the equation of type

F ( x )=( x−c )n g ( x )=0

Since F ( x ) / F ' ( x )=( x−c )h ( x ) has a sinlge root, the derivative is delivered through the Spoke ‘multiple’. Subsequently, the secant method is applied to find a single root of the modified equation. Consider an equation which has multiple roots

F ( x )=( x−3 )4=x 4−12 x3+54 x2−108 x+81=0

7


It should be noted that the bisection method fails to find a root since F ( x ) ≥0 always. Using Spoke ‘multiple’ enables to solve this equation near x=2

#> solve.x { 2 } ( 81 -108*x +54*x*x -12*x^3 +x^4 = 0 ) .multiple( -108 +108*x -36*x*x +4*x^3 ).peep;

--------- Multiple method ----------- iter x f(x) 0 2 -0.25 1 3 -2.842e-014ans = 3.0000000

Surprisingly, only one iteration is sufficient to find a root of even multiplicity.

Section 5-2 Collection of Roots and Complex Root

■ Collection of Roots. For an equation f ( x )=0, there can exist more than one root inside an interval a ≤ x≤ b (e.g. sin x=0.5, 0≤ x≤ 5π). Under this circumstance, it is desired to find them all, if possible. Though not deliberate, the interval can be divided into many subintervals and the bisection method can be applied to each subinterval. This approach may require huge amount of computational cost, but serves as a convenient tool.

Collection of roots over an interval of a ≤ x≤ b is possible by Spoke ‘span’ which implies spanning subintervals over an interval of a ≤ x≤ b and applying

the bisection method. The syntax is

solve .x ( <<opt>>, f(x) = g(x) ) .span [n=100,g=1] (a,b)

It should be remembered that the return value is a matrix in this case, while all other methods return by default a scalar. Under favorable conditions, it is straightforward to locate collection of roots. For example, the following equation

sin x=0.5 , 0≤ x ≤5 π

can be solved and plotted by

8


#> solve.x ( sin(x) = 0.5 ) .span(0,5*pi) /pi*6; #> plot.x(0,5*pi) ( sin(x), 0.5 );

The plot is shown in Figure 1, and the returned matrix is

ans = [ 1 5.0055e-008 ] [ 5 -1.0602e-016 ] [ 13 -6.0066e-007 ] [ 17 -6.0066e-007 ] [ 25 1.0602e-015 ] [ 29 -1.2013e-006 ]

Since the roots were divided by π /6, the real roots can be listed as

x=π6[1,5,13,17,25,29 ]

and agree with the exact solutions.

Figure 1 Plot of more than one root

■ Discontinuities. When a given function includes discontinuities, Spoke ‘span’ returns n× 4 matrix the last two columns of which designate the points of discontinuities. This can be confirmed by

#> solve.x ( tan(x) = x ) . span (0,5*pi) /pi*2;ans =

[ 0 0 1 -7.012e+006 ] [ 2.8606 -3.731e-006 3 3.506e+006 ] [ 4.918 1.7415e-005 5 -1.0518e+006 ]

9


[ 6.9418 -3.3665e-005 7 5.259e+006 ] [ 8.9548 -5.7542e-005 9 2.6295e+006 ]

Note that the first column is indeed representing the roots (but divided by π / 2). The second column is the corresponding residues of function. Large values in the fourth column designate that the third column are discontinuities. In the above, a total of 5 points

xdiscontinuties=π2

[1,3,5,7,9]

indeed turn out to be discontinuities, as expected. The actual roots are

xroot=π2

[0 ,2.8606 , 4.918 ,6.9418 , 8.9548]

■ Difficulty in Finding Roots. Let us consider a curve including the Bessel function

f ( x )=J 0 (3 x )−2 [sin (4 x ) ]2

over an interval of 0≤ x≤ 5. The plot is shown in Figure 2 by the following command

#> plot.x[1001](0,5) ( .J_0(3*x)-2*sin(4*x)^2, 0 );

Figure 2 Function with ill-behaved roots

10


Although a total of 9 roots exist inside the interval 0≤ x≤ 5 it would be readily understandable how difficult it is to find roots of f ( x )=0 from Figure 2. Two consecutive roots are too close to be identified near x=4. The best way of finding such ill-behaved roots is of course plotting the curve and examines the possible range of roots.

In the below, the difference between two grid resolutions can be obviously perceived. The former locates 7 roots, while the latter locates 9 roots including ill-behaved roots near x=4.

#> solve.x ( .J_0(3*x)-2*sin(4*x)^2 ) . span[101](0,5); ans = [ 0.18672 7.0457e-008 ] [ 0.72046 -4.8847e-008 ] [ 0.79823 -8.1143e-008 ] [ 2.2583 -8.5721e-007 ] [ 2.4526 -9.7138e-007 ] [ 4.6356 -1.2173e-006 ] [ 4.7744 1.2927e-006 ]

#> solve.x ( .J_0(3*x)-2*sin(4*x)^2 ) . span[1001](0,5);ans = [ 0.18672 -5.5199e-008 ] [ 0.72046 3.8437e-008 ] [ 0.79823 9.7982e-009 ] [ 2.2583 2.6107e-007 ] [ 2.4526 -2.9943e-007 ] [ 3.9314 -1.2532e-008 ] [ 3.9444 3.0044e-008 ] [ 4.6356 -8.9246e-007 ] [ 4.7744 -3.621e-007 ]

■ Complex Root. It is also possible to find a complex root by using Umbrella ‘solve’. Since no inequality is defined for complex numbers, only the secant method will be adopted in finding complex roots. A proper syntax is

solve .z { complex number } ( <<opt>>, f(z) = g(z) ) // default only

11


This means that, once a complex number is specified inside the Hub field, Spokes describing root-finding methods cannot be used.

Consider an equation in terms of a complex variable

f ( z )=ez+1=0 , z=x+iy

It is evident that f ( z )=0 has no real root since |e z|>0 for all z. Therefore, let us find a complex root starting from z=1+10 i by the following commands

#> solve.z { 1+10! } ( exp(z)+1 = 0 ).peep / pi;

The results include the iteration procedure due to the use of Spoke ‘peep’ as follows.

------------ secant method for complex root ------------ iter real imag |f(z)| 0 ( 1 10 ) 1.956 0 ( 1.011 10.03 ) 2.008 1 ( 0.65476666 9.9056557 ) 1.136 2 ( 0.42414361 9.8163938 ) 0.7145 …

25 ( 3.5603844e-008 9.4247781 ) 1.403e-007 26 ( 1.7801913e-008 9.424778 ) 7.013e-008ans = 5.66653e-009 + 3!

In the above, note that the returned complex number has been divided by π. Thus the final root is found to be

z=3 πi

Since the complex function ez is multi-valued, starting from a different point would yield another root. For example,

#> solve.z { 1+20! } ( exp(z)+1 = 0 ) / pi ;ans = -1.73861e-008 + 7!

i.e., a new root z=7 πi is obtained.

12


Section 5-3 Coupled Nonlinear Equations

For n equations in terms of n variables, no success-proof method to solve coupled nonlinear equations is available unfortunately. However, several methods such as the Newton method and the Broyden method are available in the literature. Due to stable convergence with proper under-relaxation, the default method of solving coupled nonlinear equations is selected to be the Newton method.

■ Syntax for Coupled Nonlinear Equations. The syntax to solve coupled nonlinear equations is

solve .x.y.z … { xo,yo,zo, … } ( <<opt>>, f = 0, g = 0, h = 0, … )

where the number of variables, the number of fields in the Hub field, and the number of equations in the Stem must be all the same. Without the Hub field, all-zeros are used to initialize the dummy Hub variables.

The solution can be stored at each local variable by an alternative syntax

solve [ x=xo, y=yo, z=zo ] ( <<opt>>, f = 0, g = 0, h = 0, … )

To control the iteration procedure, several Spokes are employed

.relax(0.7) relaxation factor to improve convergence

.broyden use Broyden method instead of the default method

along with Spokes ‘return’, ‘maxiter’, ‘tol/abstol’ stated earlier.

The iteration history may be heavily affected by under-relaxation factor, especially when the nonlinearity of the problem is extremely severe. Under these circumstances, use of much smaller relaxation is desirable to get a converged solution.

(Example 5-3-1) To illustrate solving coupled nonlinear equations, let us consider

13


x2 y+ y3=39 , x2+3 xy− y2=13

with initial guess ( x , y )=(1,1). This is solved twice by

#> solve.x.y { 1,1 } ( x*x*y+y^3 = 39, x*x+3*x*y-y*y = 13 ) .peep;#> solve [ x=1, y=1 ] ( x*x*y+y^3 = 39, x*x+3*x*y-y*y = 13 ); #> x ;#> y ;--- Newton method with relaxation (default 0.7) --- iter residue |dx| 0 38.33 1 380.5 9.168 2 169 2.416 3 72.73 1.706 …

18 2.206e-006 2.714e-007 19 6.618e-007 8.142e-008ans = [ 2 ] [ 3 ]ans = [ 2 ] [ 3 ]x = 2.0000000y = 3.0000000

The solution is found to be ( x , y )=(2,3). ◀

■ Effect of Relaxation and Initial Guess. The performance of root-finding methods for coupled nonlinear equation is heavily dependent on the initial guess. With a poor guess, the method frequently fails to get solution. To circumvent this unfavorable situation, one may need to reduce the relaxation factor α relax.

Let us investigate the effect of the relaxation and the initial guess by the following example.

u2 (1+750 f )=300 , 1√ f

=2 log10 ( 4 × 104 u√ f )−0.8

Before stating the Cemmath commands, it is wise to remove singularity related

14


to √ f

u2 (1+750 F2 )=300 ,1=F ¿

Since the derivative of √ f blows up near f =0, new variable F=√ f is

introduced. Also, 1 / F is also eliminated to clear a singularity around F=0. Whenever an excessive update f new=f old+df is acted, either f or u may take

negative values. Therefore, the argument of log10 is made non-negative by taking absolute value. This kind of technique is helpful to solve ill-behaved equations.

With an initial guess f =0.01 , u=1 (i.e., F=0.1 , u=1) and a default relaxation factor α relax=0.7, one can write and obtain the results as follows.

#> solve.F.u { 0.1,1 } ( u*u*(1+750*F^2) = 300, 1 = F*( 2*log10(4.e4*fabs(u*F))-0.8 )

) .peep.return( F*F, u, F, u*u*(1+750*F^2)-300, -1+F*(2*log10(4.e4*u*F)-0.8) );

--- Newton method with relaxation (default 0.7) --- iter residue |dx| 0 291.5 1 32.9 18.68 2 5531 25.29 3 1401 25.55 …

19 8.679e-006 2.78e-007 20 2.604e-006 8.34e-008

ans = [ 0.015739 ]

[ 4.8404 ][ 0.12546 ][ 2.6037e-006 ][ -4.3467e-010 ]

The final returned matrix is 1×5 matrix as was specified with Spoke ‘return’. The first element of matrix is indeed the wanted value of f =F2=0.0157402. A use of Spoke ‘return’ proves to be useful in manipulating the solution data.

15


A naïve approach using the equations as they appear

#> solve.f.u { 0.01,1 } ( u*u*(1+750*f) = 300, 1 = sqrt(f)*( 2*log10(4.e4*u*sqrt(f))-0.8 )

).peep;

unfortunately fails to get a solution, and is left to readers. However, using a smaller relaxation factor α relax=0.2, the above-mentioned naïve approach

#> solve.f.u { 0.01,1 } ( u*u*(1+750*f) = 300,

1 = sqrt(f)*( 2*log10(4.e4*u*sqrt(f))-0.8 ) ).relax(0.2).peeptail;

--- Newton method with relaxation (default 0.7) --- iter residue |dx| 74 1.934e-005 9.135e-008ans = [ 0.015739 ]

[ 4.8404 ]

is also successful to get a converged solution. The present example emphasizes the influence of under-relaxation as well as the importance of transforming given equations. When the solution is diverged, users should remind this point.

■ Linear Systems. In closing this section, a remark is made on the solution of linear equations. It is also possible to solve linear equations by the Newton method. For efficiency, the relaxation factor of α relax=1 must be used. For example, we solve the following linear equations

3 x+6 y=25+5 y2 x−4 y=2−3 x

by

#> solve.x.y ( 3*x+ 6*y = 25+5*y, 2*x-4*y = 2-3*x ) .relax(1).peep;

--- Newton method with relaxation (default 0.7) --- iter residue |dx| 0 25.08

16


1 9.482e-010 9.22 2 0 3.624e-010ans = [ 6 ]

[ 7 ]

In this sense, it can be said that the Umbrella ‘solve’ would be the most versatile tool to solve equations, either linear or nonlinear (even if success-proof is not always guaranteed).

Section 5-4 Pseudo-Linear System

■ Umbrella ‘linsys’ for Pseudo-Linear System. As a matter of fact, the Umbrella ‘linsys’ here is designed to solve pseudo-linear systems. To illustrate this point in detail, let us consider two coupled equations in terms of two independent variables written as

f ( x , y )=0g ( x , y )=0

where x , y are independent variables, and f , g are functions of x , y . Rather than solving directly the above-prescribed coupled equations, the Umbrella ‘linsys’ solves the following approximately linearized systems

f ( x0 , y0 )+ ∂ f∂ x ( x0 , y0 ) ( x−x0 )+ ∂ f

∂ y ( x0 , y0 ) ( y− y0 )=0

g ( x0 , y0 )+ ∂ g∂ x ( x0 , y0 ) ( x−x0 )+ ∂ g

∂ y ( x0 , y0 ) ( y− y0 )=0

instead of the original coupled equations. Of course, when the given system of coupled equations is linear, Umbrella ‘linsys’ yields a direct solution of the coupled equations.

■ Syntax of Umbrella ‘linsys’. The syntax of Umbrella ‘linsys’ reads

linsys .x.y. … ( <<opt>>, f(x,y,…),g(x,y,…), … )

17


when there is no need of initializing the dummy Hub variables (variables are initialized by default to be zero). Obviously, the number of variables must match the number of equations listed in the Stem.

It is worthy of note that, in solving linear system of coupled equations, there is essentially no need of initial guess. But for pseudo-linear system, the corresponding syntax becomes

linsys .x.y. … { a,b, … } ( <<opt>>, f(x,y,…),g(x,y,…), … )

where the Hub field { a,b, … } initializes the dummy Hub variables in order of appearance.

When one wants to treat the Hub variables as local variables useable outside of the Umbrella ‘linsys’, the following syntax

linsys [ x=a, y=b, … ] ( <<opt>>, f(x,y,…),g(x,y,…), … )

or equivalently

x=a; y=b; … linsys [ x, y, … ] ( <<opt>>, f(x,y,…),g(x,y,…), … )

should be used.

■ Examples. Let us consider the following coupled linear equations

3 x+6 y=25+5 y2 x−4 y=2−2 x

and solve them by

#> linsys.x.y ( 3*x + 6*y = 25 + 5*y, 2*x - 4*y = 2 - 3*x );ans = [ 6 7 ]

Note that there is no need of initializing the Hub variables via the Hub field, since the given system is straightforward linear.

When it is desired to extract the solution field in terms of a priori designated

18


variables, it is helpful to utilize local Hub variables

#> linsys [ x=0,y=0 ] ( 3*x + 6*y = 25 + 5*y, 2*x - 4*y = 2 - 3*x );#> x; y;

ans = [ 6 7 ]

x = 6.0000000y = 7.0000000

In the above, initialization of local Hub variables is compulsory, since those variables have never been defined a priori. Otherwise, one should declare these variables outside of the Umbrella ‘linsys’

#> (x,y) = (0,0);; #> Sol = linsys [ x,y ] ( 3*x+ 6*y = 25+5*y, 2*x-4*y = 2-3*x );#> x; y;

Sol = [ 6 7 ]

x = 6.0000000y = 7.0000000

Note that assignment has been peformed twice. The first is the return matrix Sol from the Umbrella ‘linsys’. The other is the local Hub variables x and y.

■ Application to Nonlinear Equations. Although the solution methods for coupled nonlinear equations are very complicated, it is sometimes possible to obtain the desired solution through iteration of solving linear equations. However, it should be remembered that the success cannot be guaranteed and depends on the case-by-case situation.

As an example, we consider the following coupled nonlinear equations

x2+ y2=42 x− y=1

At an initial guess x0= y0=1, the Jacobian matrix (discussed in Chap. 3) and the initial residue are

19


J0=[ ∂ f∂ x

∂ f∂ y

∂ g∂ x

∂ g∂ y ]=[2 x0 2 y0

2 −1 ]=[2 22 −1]

r0=[ −( x02+ y0

2 )+4−(2 x0− y0 )+1]=[ −(12+1 )+4

−(2 ⋅1−1 )+1]=[20 ]At this stage, the first step is solving the following pseudo-linear system

J0 ( x1−x0 )=r 0

⇒[ x1

y1]=[ x0

y0]+ 13 [12]=1

3 [45]=[1.33331.6667 ]

This is the reason why we get the first solution from iteration, as listed below.

#> x = y = 1;; #> for.i(1,4) linsys [x,y] ( x*x+y*y = 4, 2*x-y = 1 );

The results are

ans = [ 1.3333 1.6667 ] ans = [ 1.2738 1.5476 ] ans = [ 1.2718 1.5436 ] ans = [ 1.2718 1.5436 ]

Note that the solution at the first line agrees exactly with the discussion above. The solution method through this kind of iteration is known as the Newton method. This approach was adopted in solving coupled nonlinear equations, whereas the secant method was adopted in solving single nonlinear equations.

Section 5-5 Minimum Search

■ Local Minimum of Function. For a scalar function in terms of more than one variable

f (x1 , x2 , …, xn)

20


we will attempt to find only a local minimum, but not the global mininum. The minimum search methods handled here are the golden-section method, the quadratic method, and the Fletcher-Powell method among others. The default method is the golden-section method.

The syntax for Umbrella ‘minfind’ is

minfind .x1.x2 … .xn { c1,c2, … , cn } ( <<opt>>, F(x1,x2, … ,xn) )

and returns a solution as a matrix. Several Spokes relevant to Umbrella ‘minfind’ are

.tol/abstol/reltol(1.e-5) tolerance to terminate iteration

.maxiter(200) maximum iteration permitted

.maxstep(1) maximum step size toward negative gradient direction

.peep show the iteration procedure

.peeptail show the final stage of iteration

.golden use the golden-section method (this is default)

.quadratic use the quadratic method

.fletcher use the Fletcher-Powell method

The maximum step size may be required to be controlled, if excessive or too small movement makes the solution procedure unstable or time-consuming. The accuracy of the solution can be controlled by tolerance.

When the converged solution needs to be stored individually at a priori designated variables, an alternative syntax

minfind [ x1=c1, x2=c2, … , xn=cn ] ( <<opt>>, F(x1,x2, … ,xn) )

can be used.

(Example 5-5-1) Let us find a local minimum of the function

f ( x , y )=5 x2−6 xy+2 y2−2 y+4

starting from ( x , y )=(1,1). This can be done twice by

21


// x,y not defined#> minfind .x.y {1,1} ( 5*x*x -6*x*y +2*y*y -2*y +4 ).peep; // x,y defined#> minfind [ x=1,y=1 ] ( 5*x*x -6*x*y +2*y*y -2*y +4 );#> x; y;

which result in

------------ golden section method --------------------------- iter f <<--- f_old with |grad| by |dx| 0 3 1 1.769236 3 5.657 0.4351 2 1.011811 1.769236 1.305 0.8703 3 0.7125144 1.011811 2.647 0.2262 …

63 -0.9999007 -0.9998572 0.03366 0.002583 64 -0.9999326 -0.9999007 0.007637 0.007637 65 -0.9999515 -0.9999326 0.02216 0.001709ans =

[ 2.99034 ] [ 4.98455 ] [ -0.999933 ]

ans = [ 2.99034 ]

[ 4.98455 ] [ -0.999933 ]

x = 2.9903393y = 4.9845496

The first line for iter = 0 represents the function value f (1,1). The second line states that the magnitude of the gradient at ( x , y )=(1,1) is 5.657, and the position is moved by amount of 0.4351 where the new value of function becomes 1.769236. In this way, the starting point is repeatedly moved toward the negative direction of the gradient. The final converged solution is f min=−0.9999953 at ( x , y )=(2.9969,49952), which agrees well with the exact solution f min=−1 at ( x , y )=(3,5) ◀

■ Application to Solving Coupled Nonlinear Equations. Solving coupled nonlinear equations

22


f 1 ( x1 , x2 , …, xn )=0f 2 ( x1 , x2 , …, xn )=0

…f n ( x1 , x2 , …, xn )=0

is equivalent to finding a minimum of the following scalar equation

F ( x1 , x2 , …, xn )=f 12+ f 2

2+…+ f n2

If there exists a solution vector to the coupled equations, the global minimum of the function F ( x1 , x2 ,…,xn ) must be 0. However, the numerical method adopted here finds only the local minimum starting from a guessed solution vector. This means that an alternative approach via the minimum search may not be successful.

In order to implement this approach, the syntax is just listing functions inside the Stem

minfind .x1.x2 … .xn { c1,c2, … , cn } ( <<opt>>, f1,f2, … , fn )

If the Stem includes more than one equation, ‘minfind’ tries to find the local minimum of the square sums

f1^2 + f2^2 + … + fn^2

(Example 5-5-2) Through the minimum search, try to solve the following coupled equations

x2+ y2=4 ,2 x− y=1

starting from ( x , y )=(1,1). The syntax is very simple, and just lists the coupled equations inside the Stem

#> minfind.x.y {1,1} ( x*x+y*y = 4, 2*x-y = 1 ).peep .return( x,y, x*x+y*y-4, 2*x-y-1 );

23


------------ golden section method --------------------------- iter f <<--- f_old with |grad| by |dx| 0 4 1 2.846914 4 11.31 0.9417 2 0.070731 2.846914 15.81 0.3811 3 0.002392806 0.070731 1.125 0.1187 4 0.0001263653 0.002392806 0.3829 0.01188 5 6.380063e-006 0.0001263653 0.04893 0.004893 6 3.37887e-007 6.380063e-006 0.01987 0.0006147

ans = [ 1.27196 ]

[ 1.54336 ] [ 6.38006e-006 ] [ 0.000560276 ]

The final solution ( x , y )=(1.272,1 .5434) yields

√ ( x2+ y2−4 )2+(2x− y−1 )2=√3.37887 ×10−7=0.0005813

In other words, the accuracy of the solution may not be satisfactory. This is already reflected in the 3rd- and 4th columns of the returned matrix, listing each residue. Since the information on the n-equations has been degenerated to that of a single equation, such a loss in accuracy is inevitable.

The above-obtained solution depends on the initial guess, since there exists two roots for the coupled equations. This can be confirmed by

#> plot.x(-4,4).y(-4,4) ( .norm2(x*x+y*y-4, 2*x-y-1) );

where the result is shown in Figure 3. ◀

24


Figure 3 Minimum finding

Section 5-6 Summary

■ Umbrella ‘solve’.

solve .x { a } ( <<opt>>, f(x) = g(x) ) // default method

solve .x ( <<opt>>, f(x) = g(x) ) .bisect( a,b ) // bisection solve .x ( <<opt>>, f(x) = g(x) ) .secant( a,b ) // secant methodsolve .x ( <<opt>>, f(x) = g(x) ) .muller( a,b,c ) // Muller methodsolve .x ( <<opt>>, f(x) = g(x) ) .march ( a,b,dx ) // marching methodsolve .x ( <<opt>>, f(x) = g(x) ) .span[n=100,g=1](a,b) // bisections

solve .x {a} ( <<opt>>, f(x) = g(x) ) .newton ( f'(x)-g'(x) ) // Newton solve .x {a} ( <<opt>>, f(x) = g(x) ) .multiple( f'(x)-g'(x) )

relevant Spokes are

.peep show all the iteration procedures

.peeptail show the final stage of the iteration

.return( … ) return manipulated data

.maxiter(500) maximum number of iteration

.tol/abstol(1.e-7) absolute tolerance. tol is equivalent to abstol

.reltol(1.e-7) relative tolerance

.relax(0.7) relaxation factor to improve convergence

.broyden use Broyden method instead of the default method

25


For complex roots,

solve .z { complex number } ( <<opt>>, f(z) = g(z) ) // default method only

To solve coupled nonlinear equations

solve .x.y.z … { xo,yo,zo, … } ( <<opt>>, f = 0, g = 0, h = 0, … )solve [ x=xo, y=yo, z=zo ] ( <<opt>>, f = 0, g = 0, h = 0, … )

■ Umbrella ‘linsys’.

linsys .x.y. … ( <<opt>>, f(x,y,…),g(x,y,…), … ) linsys .x.y. … { a,b, … } ( <<opt>>, f(x,y,…),g(x,y,…), … ) linsys [ x=a, y=b, … ] ( <<opt>>, f(x,y,…),g(x,y,…), … )x=a; y=b; … linsys [ x, y, … ] ( <<opt>>, f(x,y,…),g(x,y,…), … )

■ Umbrella ‘minfind’.

To search local minimum of a scalar function

minfind .x1.x2 … .xn { c1,c2, … , cn } ( <<opt>>, F(x1,x2, … ,xn) )minfind [ x1=c1, x2=c2, … , xn=cn ] ( <<opt>>, F(x1,x2, … ,xn) )

and relevant Spokes

.tol/abstol/reltol(1.e-5) tolerance to terminate iteration

.maxiter(200) maximum iteration permitted

.maxstep(1) maximum step size toward negative gradient direction

.peep show the iteration procedure

.peeptail show the final stage of iteration

.quadratic use the quadratic method

.fletcher use the Fletcher-Powell method

.golden use the golden-section method (this is default)

Application to solve coupled nonlinear equations via minimum search

minfind .x1.x2 … .xn { c1,c2, … , cn } ( <<opt>>, f1,f2, … , fn )

where vector function is listed in the Stem.

26


27

web view1 3 -2.842e-014. ans = 3.0000000. surprisingly, only one iteration ... at an initial guess x...

Documents