P e r g a m o n Appl. Math. Lett. Vol. 11, No. 5, pp. 35-37, 1998

© 1998 Elsevier Science Ltd. All rights reserved Printed in Great Britain

0893-9659/98 $19.00 + 0.00 PII: S0893-9659(98)00076-7

C o n v e r g e n c e o f B r o y d e n - L i k e M a t r i x

DONGHUI LI , J INPING ZENG AND SHUZI ZHOU Department of Applied Mathematics

Hunan University, Changsha, P.R. China

(Received March 1996; accepted November 1997)

A b s t r a c t - - I n this paper, the convergence of Broyden-like matrices generated by Broyden-like method for solving nonlinear equations F(x) = 0 is discussed. Under suitable conditions, it is proved that the matrix sequence converges to the gradiauent of F(x). © 1998 Elsevier Science Ltd. All rights reserved.

Keywords--C~nvergence, Broyden-like matrix, Nonlinear equations, Nonlinear programming.

1. I N T R O D U C T I O N

Broyden-like methods are important iterative methods for solving nonlinear equations, nonlinear programming, and nonlinear complementarity problems. They possess local superlinear conver- gence properties. If a suitable line search technique is used, the methods converge globally and locally superlinearly (see [1-4], etc.).

Let F : R n --* R n. Then the iterative process of Broyden-like methods for solving nonlinear equations

F ( x ) = 0 (1)

is as follows.

Bese + F ( z e ) = 0, k = 0, 1 , . . . , (2)

Xk+l ---- Xk ~- 8k, k = 0, 1, . . . . (3)

The iterative matr ix Be is updated by the following formula:

Be+l -~ Be + Ok (Ye - B e s e ) s T ii ell + , (4)

where sk = Xk+l - xk , Yk = F ( x k + l ) - F ( x k ) , and Ilskll = sTsk, ee is chosen so that for some constant 8 e (0,1), 10e - 11 < 8, and when Be is nonsingular, so is Be+l. We refer to [5] for details about the choice of 0e.

Denote x* as a solution of (1). Let F E C 1. F I stands for the Jacobi matrix of F. It is well known that a necessary and sufficient condition of superlinear convergence for the iterative method (2) and (3) is I](Be - F'(x*))pell/l[pe[[ --* O. This means that in some sense the super- linear convergence depends on the approximation of Bk to F ' along direction Pc. Therefore, it is interesting to s tudy the convergence of matrix sequence {Be}. For unconstrained optimization

Typeset by A.,v~TmX

35

36 D. LI et al.

problems, the authors of [6-8] studied the convergence of the matrix generated by the symmetric rank one method, DFP, BFGS methods, and restricted Broyden's class methods. The conver- gence of Broyden's rank one method has not been solved till now. There are examples which show that Broyden's matrix may not converge to F'(x*). However, under certain conditions, this convergence holds. The purpose of this paper is to give a condition under which the matrix generated by Broyden-like methods converges to F'(x*).

2. C O N V E R G E N C E O F B R O Y D E N - L I K E M A T R I X

In this section, we study the convergence of the Broyden-like matrix. First, we introduce a lemma (see [2,3]).

LEMMA 1. Let F E C 1. ALso Jet F ' be Lipschitz continuous with constant L, i.e., there exists a constant L > 0 such that

IlF'(x) -F'(Y)IIF ~ LIIx-yll, Vx, y e R", (s)

where IIAII~ = (E,~,~=I a~j) I/u denote the b-~obenius norm of matrix A and Ilxll denote the Euclidian norm of vector x. Choose Bo nonsingular, f f {xk } is well defined and

then we have

o o

I l Z k ÷ Z - xkll < ~ , (6) k=O

o o Ily~ - n ~ II 2 ~=0 Wsj-qV < oo. (7)

I t is shown by [2,3] that for nonlinear equations and an unconstrained optimization problem,

(7) holds.

To analyze the convergence of {Bk}, we make the following assumption (similar to [6] for symmetric rank one method).

ASSUMPTION (A).

(i) {xk} ~ x*, (k ~ co), and (6) holds. (ii) F E C 1.

(iii) {sk} is uniformly lineaxly independent, i.e., there is a constant p > O, and integers ko > O, ra > n such that when k > ko, we can find indices k < kz < k2 < . . . < kn < k + m such that O'min(Qk) >_ p, where ~rmin(A) denotes the smadlest singular value of matrix A and

Qk = [ ~ sk2 s ~ \ IlSkl I1' Ilsk~ I1"" "' Ilsl,,, I I )"

THEOREM 2. Let the conditions of Assumption (A) hold. {Bk} is generated by Broyden-like methods. Then

lira Bk = F' (x*). (S) k---*oc

PROOF. Denote aa = HY~- BaskH/HskH • By Assumption A(i) and Lemma 1, we see that {ak} --* 0, as k -* oo. We prove the conclusion by the following two steps.

First, we show that for all k , j , if k > j + 1, then

IlYj - S~s~ll _< (x + O)2k-~(~k,~ + ak,~)llsjll, (9)

where r/k,j = max{llxp - zqll I J -< P -< q -< k} and ak,# = max{ap [ j < p < k}. We verify (9) by induction on k.

Broyden-Like Matrix 37

For k = j % 1, we have from (4) tha t (B~+I - B j ) s j = Oj(yj - B j s j ) and 0 < 1 - 0 < Oj < 1 + O.

Therefore,

HY~ - B~+Isj][ ~_ II~ - Bjs~[I + ]I(BJ+I - B~)sj[[

This means that (9) holds for k = j + 1. Now, assume that (9) holds for some k >_ j + 1. Then,

llyj - Bk÷:jll ~ IIY~ - Bksjll + ll(Bk+1 - Bk)s~ll

___ (1 + 0)2k-J(r/k,j + ak,j)lls~ll + 0:kllsjll

< (1 + 0)2k-J~Tk,j[[S#[[ + (i + 0) (2 k-j + 1) ,Tk,jlls~ll

<_ (i + 0)2k+1-J(,Tk,j + "k+l,#)lls~ll,

Tha t is, (9) holds for k + 1. Now we prove (8).

Set Aj = f~ F ( z j + t s j ) dr. Then yj = A#sj and {Aj} ~ F' (x*) as j ~ oo. Substituting m + 1 + k for k in (9), we get that for all j = kl , k 2 , . . . , kn,

[[(Bk+~+l - F' (x*))sill < H(Bk+~+I - A.#),#[[ + [[A.~ - F' (x*)[[F

Ils~ll - Ilsjll _< (1 + o)2k+rn+l-J(Y~k+rn+l, j + 0"k+m+l,j) "-~ IIA.~ - F' (x*)llF

___~ (1 "~ O)2Yn+l(~k+m+l,j "Jr- O'k+m+l,j) "Jr" IIAj - F' (x*)[[F.

Since for j = k l , k 2 , . . . , kn, ~}k+m+l,j --~ O, a k + m + l j --* 0 as k --* c¢, we c la im t h a t

llBk+m+l - F' (x')ll < p-1 ll(Bk+~+1 - F' (x*)) Okll ~ o.

In other words, (8) holds. The proof is now completed.

R E F E R E N C E S 1. C.G. Broyden, J.E. Dennis, Jr. and J.J. Mord, On the local and superlinear convergence of quasi-Newton

methods, J. Inst. Math. Appl. 12, 223-236, (1973). 2. A. Griewank, The 'global' convergence of Broyden-like methods with a suitable line search, J. Austrialian

Math. Soc. Ser. B 28, 75-92, (1986). 3. D. Li, Global convergence of nonsingular Broyden's methods for solving unconstrained optimizations, Chinese

J. Numer.. Math. Appl. 17, 85-94, (1995). 4. S. Zhou, D. Li and J. Zeng, A successive approximation quasi-Newton process for nonlinear complementarity

problem, In Recent Advances in Nonsmooth Opgmization, (Edited by D.-Z. Du, L. Qi and R. Womersley), pp. 459-472, World Scientific Publishers, (1995).

5. J.J. Mord and A. Trangenstein, On the global convergence of Broyden's method, Math. Prog. 30, 523-540, (1976).

6. A.R. Conu, N.I.M. Could and Ph.L. Toint, Convergence of quasi-Newton matrices generated by the symmetric rank one update, Math. Prog. 50, 177-195, (1991).

7. M.J.D. Powell and R.-P. Ge, The convergence of variable metric matrix in unconstrained optimization, Math. Proff. 27, 123-143, (1983).

8. J. Stoer, The convergence of matrix generated by rank-2 methods from the restricted /9-class of Broyden, Numer. Math. 4,1, 37-52, (1984).

9. J.E. Dennis, Jr. and J.J. Mord, A characterization of superlinear convergence to quasi-Newton methods, Math. Comp. 126, 549--560, (1974).