1998 Fifth /E€€ International Workshop on Cellular Neural Networks and their Applications, London, England, 14-17 April 1998

Estimation of the Basin of Attractions of Stable Equilibrium Points in CNNs

Valeri M. Mladenov ' and Domine M. W. Leenaerts "

' Author is at the Department of Theor. Electrotechnics, Technical University Sofia, 1756 Sofia, Bulgaria, e-mail:valerim@vmei.acad.bg

" Author is at the Department of Electrical Engineering, Technical University Eindhoven, 5600MB Eindhoven, The Netherlands, e-mai1:dominel @eeb.ele.tue.nl

ABSTRACT: We present an approach to estimate the basin of attraction of stable equilibrium points in CNNs. The approach is based on the determination of the so called tree of regions connected with each stable equilibrium points described in a previous authors work. The new contribution is connected with additional separation of the regions where the boundaries between different basins are located. The suggested separation with internal hyperplanes will help to estimate more precisely the boundaries between different basins, because the previous algorithm to obtain the trees could not give the exact description of the basin of attractions.

1. Introduction

In 1988, Chua and Yang introduced the Cellular Neural Network (CNN) as a local interconnected network of non-linear dynamic cells [ 1-21. The behaviour of the CNN is defined by the template matrices A and B and the template vector I. CNNs are non-linear dynamic systems which give rise to complex dynamics. Since the invention of the CNN system, many properties of the CNN are derived. Due to the complexity of the problems, they are mainly valid for special cases. Generally, the problems related to the dynamics and the basin of attraction of CNNs are not solved thoroughly. The dynamic properties of dynamic systems with saturated non linearity's (e.g. neural networks) are considered in the book written by Liu and Michel [3] and in the references given there. This work is primarily concerned with the existence and the location of equilibrium points, with the qualitative properties of these points and with the extension of the basin of attraction of asymptotically stable equilibrium. An approach for estimation the basin of attractions of Hopfield Neural Networks, based on Lyapunov functions is given in [4]. The approach hold especially when the output functions are sigmoidal and hence there is no extension for the piecewise linear output functions.

The subject of the presented paper is to consider the case when initial patterns are given at instance zero to the state nodes of the CNN and to find an estimation of the basin of attractions of the stable equilibrium points. This problem is of great importance for CNN implementation because it is connected with the robustness of the CNN. The problem is related with stability and convergence analysis of CNNs.

A normalised definition of CNNs and convergence properties are considered in [5]. A complete description on the basin of attractions for a simple 2 cell CNN is given in [6]. In [7] an approach for the estimation of the basin of attractions for an n-cell CNN is suggested and here we will try to extend the approach in order to obtain a precise description.

The outline of this paper is as follows. In section 2 we make a brief presentation of the known results which we will use later. Then in section 3 we give a new idea for internal separation of the regions belonging to several basin of attractions. In section 4 we give an example to illustrate the outlined approach and we end up with some conclusions in section 5.

.

2. Preliminaries

In this paper we will consider the continuous-time CNNs described by the non-linear ordinary differential equation in vector-matrix form:

d~(t)/dt = - X ( t ) + A y ( t ) + b =-X(t)+Af(X(t))+b, (1)

0-78034867-21981$10.0001998 IEEE

62

Here the vector and matrices will be denoted by bold fashion, x(t)=(xl(t), x2(t), ..., xn(t)jT is the state vector at time t , A is the feedback matrix, & represent the time-invariant inputs and the threshold [l], [5 ] and y=f(x)=lf(xr), f(x2), ..., f ( ~ ~ ) ) ~ , where f(.) is the piecewise linear function

Owing the piecewise linear function (2), the state space of (1) can be partitioned into 3" disjoint regions which are

a) linear region DO wheref(x,)=x, for every i; b) 2" saturation regions DS, whereflx,) is + l or -1 for every i; c) 3"-2"-1 partial saturation regions DP, where for some i, f(xJ is +1 or -1 and f(x,,)=x, for the others.

If the condition a,,>O holds (where afi are the diagonal elements of the matrix A in (l)), then the stable equilibrium points are only in the saturation regions. The technique described in [7] allows to construct a tree of regions connected to each saturation region with a stable equilibrium point. These trees give a first order estimation of the basin of attractions of the equilibrium points in the corresponding saturation regions (roots of the trees). The technique is based on the determination of the direction of the state trajectories for the boundaries of all regions. Here we will discuss the problem with those regions in which boundaries are located between different basin of attractions. This problem is stated in [7] as open and we make an attempt to give more insight for the internal separation of the above mentioned regions.

3. Description of the approach

classified as follows [7], [XI:

We shall explain our new approach for the case of symmetric matrix A (CNN with a symmetric template). Without loss of generality we consider the linear region DO (f(q)=xi). If there are two or more stable equilibrium points, the boundaries of the basin of attractions are always located in DO (DO belongs to all trees). In this case the state equation (1) becomes a linear differential equation

Our idea is based on the transformation of (3) into

via non-singular matrix P , where AL = P- 'ALP , b̂ = P-'b and X = P -lx.

Equation (4) is valid in a region DO,, which can be obtained using a transformation of the boundary hyperplanes of the region DO through X = P - ' x . Furthermore if the matrix P is appropriately chosen (for instance the matrix of eigen vectors in order to receive a eigen-factorization), A, is a diagonal matrix of eigen values of AL and as AL is symmetric its eigenvalues are real. Therefore it is easy to see that the equations of (4) are independent.

It should be stressed that the above presentation is possible also for each of the partial saturation regions DP. If we assume, without loss of generality, that only xi is saturated, then the corresponding matrix A L can still be expressed by A-I except that the elements of i-th column inA are set to zero. In this case the corresponding i-th eigen value is equal to -1 and the other eigen values corresponding to the symmetric (n-l)x(n-l) part are real.

-

Let us consider the i-th equation of (4) expressed by

&; /d t=ai j2+b^=Ai(x^+ci ) (5)

where Ai is the i-th eigenvalue of AL and ci = bi / Ai . The solution of (5) for each initial condition Zi (0) from DO, can be expressed as

63

Using (6) one can conclude that the state variable x,(t) can not cross the hyperplane given by xti=-ci. Furthermore is positive (in the frame of the region DOJ. Therefore the it will tend to -ci if & is negative and will leave -ci if

space of DO, is separated via hyperplanes Xi = -ci , i = l , ..., n , or in vector form:

Now we can return back to DO and using the inverse transform x = P i we can conclude that the region DO can be separated internally via hyperplanes which can not be crossed by the state trajectories. The equations of these hyperplanes can be found from (8) and x̂ = P - ' x as

P - I X = - C (9)

where the i-th row of (9) represent the equation of the i-th separation hyperplane. The state trajectories tend to it if Ai is negative and tend to leave it if Ai is positive. The intersection of these hyperplanes with each of the boundaries of region DO can be found easy considering the corresponding equations simultaneously. For example, the intersection of all internal hyperplanes with the boundary hyperplane xj=l is given by (9) except that the j-th column of P' becomes zero column and the right side becomes the old one (-c) minus the j-th column of PI.

Now we are in a position to extend the results in [7]. First of all we should generate the trees of regions connected with each saturation region containing stable equilibrium point (following the technique described in [7]). Then for the regions in which boundaries of different basin of attractions are located we should find the description (9) of the internal hyperplanes. These boundaries can not be crossed by state trajectories and then a new subset of regions can be obtained using the separation technique.

Now after separation we have a new set of regions and we simply should apply the approach in [7] again in order to get a new estimation of the basin of attractions. The advantage of the outlined separation is that the new regions fulfil more precisely the stopping criteria [7] when the trees are constructed. This is due the fact that some of their boundaries (internal) can not be crossed by state trajectories.

4. Example

Let us consider a two cell CNN described by the system of equations

The above CNN has two stable equilibrium points, namely El=(x1,xz)=(3.5, 3.5) and E2=(xl,x2)=(-1.S, -1.5). Due to the piecewise linear functions yi=fi(xJ the CNN can operate in nine linear regions, numerated from left top (1) to right bottom (9). The regions are separated by the lines x ~ = l , X I = -1, xz=l and x2= -1. The linear region is denoted by DO={5}, the set of saturation regions DS=( 1 , 3, 7, 91, and the set of partial saturation regions DP=( 2, 4, 6, 8) (exact as in [7]).

Let us consider the linear region 5 (fig. 1). The description of our two cell CNN here is dx, / d t =05x1 + x 2 + 1 ~ 2 / d t = x 1 + 0 5 x 2 + 1 (1 1)

or AL =[1 0.5 05]. 1 b = [ : ] .

We apply the transformation of the region 5 into 5t via non-singular matrix P, x = P2 and finally obtain equation (4),where iL = P - ' A L P , b ^ = P - ' b .

If the matrix A is symmetric (symmetric CNN), then all its eigenvalues hl and & are real and if we choose P to be - -0.5 0

a matrix of eigenvectors A, = In this case, P =

64

Using the linear transformation X = P - ’ r region 5 (fig 1) is transformed into region 5t (point (1,l) into point (O,l), point (1,-I) into point (l,O), point (-1,-1) into point (0,-1), point (-1,l) into point (-l,O).where the description of our CNN in the new basis is

(12) a5, / d t = -052, &,/dt=15X2+1

Therefore the transient can be written in a form of two independent equalities

or vector -c in (8) is -c=[O -2/3IT. Now we should return back in region 5 via transformation x = Px̂ . The line between points (-1, -1) and (1, 1)

corresponds to the line xI=O in region 5t and the line between points (-1, -U3) and (-1/3, -1) corresponds to the line x2=-2/3 in 5t.

The equations of the above separation lines in a region 5 follow also from (9). Here P 1 is

P-’ = [:: -E] and the two equations become: 0.5x1-0.5x2=0 ; 0.5x1+0.5~2=-2/3 which give the lines stated

above and separate region 5 in four sub regions 5a, 5b, 5c and 5d as shown in fig. 2. The above procedure can be done also for the regions 8 and 4. In this case one of the separation lines falls

completely in one of the sub regions determined by the other line. So for the regions 4 and 8 only one internal separation line is necessary. This is the line xz=O for region 4 and the line x1=0 for region 8. Using these lines regions 4 and 8 are separated respectively in 4a, 4b and Sa, 8b as it is shown in fig. 2.

Now using the new set of regions and the rules described in [7] we can construct the trees for the stable equilibrium points El and E2 (in a regions 3 and 7). It is easy to check that the tree for El include regions { 3, 2, 6, 1, 9, 4a, Sa, 5a and 5b) and the tree for E2 include regions(7, 4b, Sb, 5c and 5dJ. If the initial conditions are chosen from region 5, i.e. lu,l<I then this solution represents the basin of attractions of E l and E2. Hence we can state that the trajectories started from regions 5a and 5b will finish in El and the trajectories started from 5c and 5d will finish in E2 after the transient settles down. If however we may choose any initial condition, then we must conclude that the obtained solution is an estimation. This because a small region in 8b still belongs to the basin of attraction of El.

5. Conclusions

In this paper we suggest a method for determining an estimation of the basin of attractions for the stable equilibrium points in Cellular Neural Networks. It is valid also for continuous time Hopfield networks if the output functions are piecewise instead of sigmoid functions. The method is based on determining the so called tree of regions connected with each stable equilibrium point described in the previous authors work. The suggested separation of some regions (where the boundaries between different basin are located) allows the construction of the trees and hence the basins of attraction with a refinement.

6. References

[ 11 L.O. Chua and L. Yang, ’Cellular Neural Networks: Theory,‘ IEEE Trans. Circuits and Systems, CAS-35, 1257- 1272 (1988) [2] L.O. Chua and L. Yang, ‘Cellular Neural Networks: Applications,’ IEEE Trans. Circuits and Systems, CAS-35,

[3] D. Liu, A.N. Michel, Dynamical Systems with Saturation Nonlinearities, Lecture Notes in Control and Information Sciences 195, Springer-Verlag, London 1994 [4] A. Michel, J. Farrell, W. Porod, ‘Qualitative Analysis of Neural Networks,’ IEEE Trans. on Circuits and Systems., CAS-36, No. 2,229-243 (1989) [5] J.A. Nossek, G. Seiler, T. Roska, L.O. Chua, “Cellular Neural Networks: Theory and Circuit Design”, Int. J. Cir. Theor Appl., Vol.-20,533-553 (1992). [6] G. Seiler, A.J. Schuler, J.A. Nossek, ‘Design of Robust Cellular Neural Networks,’ IEEE Trans. Circuits and Systems- I., Vol-40,358-364 (1993)

1273-1290 (1988)

65

[7] V.M.Mladenov, D.M.W. Leenaerts, F.H. Uhlmann, ’First Order Estimation of the Basin of Attraction of Stable Equilibrium Points in CNNs”, ECCTD’97, Budapest, Hungary, pp. 684-689, 1997. [8] F. Zou, J.A. Nossek, ’Bifurcation and chaos in cellular neural networks’, ZEEE Trans. on Circuits and Systems - I, CAS-40, NO 3, 166-173 (1993)

rx2

! i

I i xl=l

Figure 1: Regions 5 and 5t. The arrows indicate the directions of the trajectories at the boundaries.

66

x2

(-1, -1/3)

xl=-1

(-1/3, -1)

x2= 1

x2=- 1

x l = l

Figure 2: New division of the regions based on the computed additional separation lines.

67