Pergamon 0305-0548(95)00017-8

Computers Ops Res. Vol. 23, No. 2, pp. 121-129, 1996 Copyright © 1995 Elsevier Science Ltd

Printed in Great Britain. All rights reserved 0305-0548/96 $9.50 + 0.00

"CONSCIENTIOUS" N E U R A L NETS F O R T O U R C O N S T R U C T I O N IN THE T R A V E L I N G S A L E S M A N

PROBLEM: THE V I G I L A N T NET

L a u r a B u r k e t Department of Industrial and Manufacturing Systems Engineering, Lehigh University,

Bethlehem, PA 18015, U.S.A.

(Received August 1993; in revised form March 1995)

Scope and Purpose--This paper describes an adaptive neural network technique for constructing short tours for the well-known traveling salesman problem (TSP). The TSP asks for a minimum length closed path through a given set of points or cities. Its technical difficulty has attracted a variety of solution approaches, including various neural network methods. Adaptive neural network methods offer a computationally feasible approach, even without hardware simulation. The vigilant net, described herein, addresses some of the shortcomings of adaptive neural approaches. Moreover, we investigate strategies for fixing or adapting various critical parameter values so that good tours can be reliably generated. The results of the algorithm are compared to a benchmark heuristic for speed, and a more computationally intensive heuristic for performance. The vigilant net proves to be very comparable to the more computationally demanding heuristic and far outperforms the faster method, to which it is comparable in speed if implemented in hardware.

Abstract--Previous research has established the feasibility of applying adaptive neural network approaches to the traveling salesman problem. Such methods (unlike the Hopfield network) gradually adjust a ring of nodes until the nodes correspond to actual data points, making them comparable to tour construction heuristics. However, they typically require a few thousand iterations and may not yield "node separation"--a one-to-one correspondence between nodes and cities--in a reasonable processing time. The vigilant net addresses these shortcomings by utilizing a hardware implementable mechanism--that of vigilance, from adaptive resonance theory networks--to help separate nodes without excessive processing. Further, for these solution methods to be truly viable, they must not demand fine-tuning of various parameters, as presently they do. Finally, an advantage to using these methods must be established. Results from previous research and new insights help determine appropriate parameter settings and strategies for the algorithm. Two strategies for selecting the important vigilance parameter are investigated here. In one, feedback from the network helps to adjust vigilance. Results indicate that the vigilant network can yield acceptable results in very short processing times, and the two strategies perform virtually identically. Comparisons with conventional heuristics yield further insights.

1. I N T R O D U C T I O N

T h e p e r c e p t i o n o f n e u r a l n e t w o r k a p p r o a c h e s to o p e r a t i o n s r e sea rch p r o b l e m s has suffered in t w o

ways : first, the h y p e s u r r o u n d i n g the n e w m e t h o d o l o g y has u n d e r s t a n d a b l y c a u s e d m a n y o p e r a t i o n s

r e sea rche r s to l o o k skep t i ca l ly o n the c l a ims p r o p o u n d e d ; a n d second , the resul ts o f ea r ly

e x p e r i m e n t a t i o n - - t r u m p e t e d to the n e u r a l n e t w o r k c o m m u n i t y w i t h p r i d e - - r e v e a l e d n e u r a l net-

w o r k s to be s h o c k i n g l y i n a d e q u a t e dev ices fo r m o s t O R re la t ed p r o b l e m s . In pa r t i cu l a r , the first

a p p l i c a t i o n to the t r a v e l i n g s a l e s m a n p r o b l e m in 1985 r e v e a l e d t h a t feas ib le t ou r s c o u l d s o m e t i m e s

be g u a r a n t e e d a n d t h a t g o o d t ou r s c o u l d be f o u n d fo r 10 a n d p e r h a p s th i r ty c i ty p r o b l e m s [1]. E v e n

these u n i m p r e s s i v e resul t s c o u l d n o t be re l i ab ly d u p l i c a t e d [2]. T h e s e u n f o r t u n a t e causes h a v e led

m a n y to o v e r l o o k the swif t p r o g r e s s t h a t has ensued s ince the first T S P app l i ca t i ons . E v e n

e x t e n s i o n s to a n d m o d i f i c a t i o n s o f the H o p f i e l d - T a n k n e t w o r k h a v e y ie lded a m o r e v iab le

a p p r o a c h , a l t h o u g h in s o f t w a r e s i m u l a t i o n it still r equ i r e s excess ive c o m p u t a t i o n t ime. M o r e o v e r ,

tLaura Burke is Associate Professor of Industrial and Manufacturing Systems Engineering at Lehigh University. Her research interests include artificial neural networks in prediction and optimization problems, and combinatorial optimization and logistics.

121

122 Laura Burke

adaptive neural networks (Hopfield-Tank networks are non-adaptive in their weights) have emerged as an alternative with a very real potential in the application to the traveling salesman problem.

Adaptive neural network methods appear to be promising constructive approaches to solving the traveling salesman problem [3-7]. They are properly compared with conventional constructive methods such as the nearest neighbor insertion method, cheapest insertion method, and the newer space filling curve approach--al l of which build tours which may then be improved using an improvement algorithm such as the one due to Lin and Kernighan [8]. To establish the applicability of new neural network methods it is necessary to discern what, if any, practical or potential advantages exist with respect to other constructive heuristics, and this paper aims to do that.

Adaptive neural networks construct tours by preserving an (arbitrary) sequence of nodes, and gradually changing node positions until they correspond with city positions. A key issue with these approaches has been the separation of nodes (ensuring that each corresponds to a unique city), which has been addressed by the authors cited above in a variety of ways. In this paper we give new results for an adaptive network utilizing the vigilance concept of adaptive resonance theory [9], the vigilant network. The concept of the vigilant network, first introduced in [7], is fully expanded here. We draw on well-known concepts from conventional approaches to the TSP and suitably transfer them to adaptive neural implementations. We restrict modifications to those realizable with hardware implementable mechanisms, which thus renders such a method a potentially highly efficient mode of tour construction. Even in software simulation, however, the vigilant network approach yields a tour very quickly.

The problem with previous adaptive neural network approaches is that fine-tuning various parameters impacts performance considerably. Also, an investigation into what kinds and sizes of problems suit the approach is lacking. Our goal is to find rules for initialization and automated parameter adaptation strategies that produce good results and to determine the sensitivity, if any, of these choices to problem size and type. We investigate adaptation of the vigilance parameter based on locally available feedback. Finally, we show that a key advantage to the method is that an accelerated processing time may be selected, with virtually no negative impact on performance.

It is important to note that the techniques described here are tour construction (or more appropriately, tour evolution) methods; thus, significantly shorter tours can be obtained using familiar tour improvement procedures [8, 10, 11]. Since we focus only on construction, our goal is to determine if the method offers a fast, consistent way to generate acceptable tours. Since the space filling curve heuristic [12] is perhaps the fastest available constructive approach, we use its performance as one benchmark. In hardware implementation, the vigilant network would be quite competitive with this method in terms of speed. The multiple nearest neighbor heuristic typically gives much shorter tours but at a much higher computational expense, so it gives another appropriate comparison. Moreover, we test the algorithm on both uniformly distributed planar data and "clumped" data with varying degrees of density (number of cities per unit).

2. BACKGROUND

In this section we briefly review adaptive neural network approaches to the traveling salesman problem. These approaches have significant advantages over non-adaptive neural network methods-- the Hopfield and Tank approach, specifically--in terms of their ability to quickly yield feasible tours.

2.1. Adaptive neural network TSP methods

Adaptive neural network approaches differ drastically from non-adaptive, Hopfield systems [1, 13]. While many instances of application of Hopfield-Tank networks to the TSP and other combinatorial optimization problems exist, these applications are not relevant to the adaptive methods we pursue. Non-adaptive methods will prove efficient only with hardware implementation; adaptive approaches can be considerably speeded by exploitation of parallelism but also operate efficiently in software simulation.

Adaptive approaches to the TSP vary widely. The elastic net [14] set the stage for the basic approach, which can be described as follows. A "ring" of connected nodes is adjusted until node

"Conscientious" neural nets for TSP 123

positions ultimately coincide with actual city positions. The order of visitation is set by the relationship on the ring between the nodes. The elastic net required many thousands of iterations to yield good solutions for reasonably sized problems (on the order of 100 cities).

Self-organizing feature map (SOFM) based networks [15] offer a much faster approach similar in spirit to the elastic net [3-5]. The network consists of an input node layer and an output node layer which represents the "ring". The input nodes receive city positions and transmit distance informa- tion with respect to the current position of each output node. A competition identifies the output node closest to the present city, and that node--a long with its neighbors on the ring, to varying degrees--moves closer to the claimed city. Ultimately, each output node should represent a single city. However, without additional (somewhat artificial) modifications, the approach generates "duplicates": nodes which claim more than one city, while leaving other nodes to claim no cities. Fort [3] and Angeniol et al. [4] addressed the problem by using 2N or 3N output nodes, where N is the number of cities. Angeniol et al. also used heuristic node duplication and deletion steps. With these methods, it is possible to achieve node separation, i.e. a single node for each city.

Favata and Walker [5] used a SOFM based approach without node separation and found that, for uniformly distributed problems, relatively good results could be achieved by arbitrarily ordering multiple cities claimed by one node. Burke and Damany [6] showed that their modified approach, the guilty net, could use a conscience mechanism [16], to penalize "greedy" nodes and subsidize "dead" nodes so that a more even distribution of winners evolves. Even if conscience is used, some duplicates may be allowed and arbitrary insertion can be used to finalize the tour. Finally, in recent work, Burke has shown that the node separation mechanism of the guilty net can improve tours at a statistically significant level [7], and that additional features of the algorithm can affect performance.

While the guilty net's "conscience mechanism" can aid in producing good tours quickly, it requires the selection of a weighting factor for accomplishing node separation, and the schedule for increasing or lessening its effect. If the weight is too large, the quality of the tour is compromised at the expense of eliminating duplicates; if too small, conscience exerts no effect. In this case, the large number of duplicates must be inserted into the tour, and this can increase computational effort (if done intelligently) and/or lead to inferior tours with higher variance. The vigilant net offers an alternative approach to node separation for SOFM based TSP methods.

All of the adaptive methods may be viewed as requiring O(N 2) computations per iteration (a full pass through the list of cities) when simulated in software. However, they may be speeded up by an order of magnitude if they are implemented in hardware. The number of iterations, however, may vary with the actual implementation. In most of the adaptive systems described above, on the order of N iterations are required. In the vigilant net, we will investigate much faster processing times by restricting the number of iterations.

2.2. The Space Filling Curve TSP approach

As mentioned earlier, conventional constructive heuristic approaches to the TSP can perform well for uniformly generated problems, on average. In addition to simple heuristics which obtain good solutions, the Space Filling Curve procedure (SFC) [12] has been shown to obtain acceptable results very quickly for the TSP. In fact, the tour length yielded by the SFC on uniformly generated problems can be predicted with high confidence, without execution of the heuristic, from the formula

TLsfc = 1.25TLopt

(almost surely, as N grows large, for points that are statistically independent and identically distributed) where TLsfc = tour length generated by the SFC procedure, and TLopt = optimal tour length [12]. Thus, average and worst case behavior coincide. The algorithm constructs tours extremely quickly [O(Nlog N), where N is the number of cities]. Bartholdi and Platzman also show that the heuristic, unlike many other TSP heuristics, obtains smaller tour lengths for non- uniformly generated problems (truncated normal). Non-uniformly generated problems typically should possess smaller tour lengths, and many heuristics are unable to capitalize on this expectation.

2.3. Multiple nearest neighbor heuristic

The multiple nearest neighbor heuristic (MNN) is a representative constructive heuristic requiring

124 Laura Burke

O(N 3) computat ions for an N-city problem. The nearest neighbor simply constructs a path through the N cities by selecting, out of cities not yet on the tour, the city nearest to the end of the path. A tour is then made by connecting the endpoints of the path. The M N N gives the best results for starting the nearest neighbor heuristic at each of the N points. Its average case performance has been found to be 15% above optimal [11]. Since the best of N tours is selected, the heuristic is also relatively insensitive to clumping and density of the city distribution.

The M N N heuristic used here was coded in Fortran and implemented on a PC-486 25 MHz computer, as was the vigilant net.

3. V I G I L A N T N E T

The vigilant net, an approach to automating the separation of nodes in SOFM based TSP approaches, was introduced in [7] and is further expanded and investigated herein. It uses notions from adaptive resonance theory [9], in that i fa node which wins a competition (is closest to the input city) does not satisfy a particular criterion, it is reset--i .e , turned off, and a previously uncommitted node allowed to win. The parameter which facilitates this mechanism in adaptive resonance neural networks is known as the vigilance parameter. Vigilance is used in tour evolution as follows. On a particular pass through the list of cities, a node which wins a competit ion has its "win factor" increased by 1 (this update may be accomplished by adaptive, local methods similar to those used for the conscience mechanism; see [17]. I f a city is presented to the network, and the winning node has a win factor exceeding some pa ramete r - -wh ich we will call the vigilance pa rame te r - - t hen that node is reset. The competit ion is restarted for uncommitted nodes within the neighborhood of the reset node. At the start of the next pass through the cities, all win factors are zeroed out. This vigilance mechanism has some appealing properties:

(1) It treats node separation as a "hard constraint". The conscience mechanism of the guilty net treats it as a soft constraint, with an accordingly subjective penalty parameter.

(2) It will not allow the number of cities claimed by a node to exceed a bound set by the user.

Since the vigilance parameter is the bound described above, in te rpre ta t ion- -and thus se lec t ion--of the parameter is much more straightforward than for the conscience parameter of the guilty net. Less refinement and tuning appears necessary, as we report in Section 4. The size of the problem should affect the vigilance level in a straightforward manner. A decrease in computat ional effort results when fewer duplicates emerge and reduction in tour length variance can occur since the tour will depend much less on insertion techniques. The algorithm follows.

Algorithm

0. Initialization. Let N = number of cities; d = number of city coordinates (Euclidean dimension); c~ = 1;/3 = 10 [c~ is the learning rate of the network and/3 is a distance factor which appears in (3)]. Randomize city order, and label cities 1 , . . . , N. Let m be the index of the city in presentation. Set m = 1 and win(j) = 0 f o r j = 1 , . . . , N. Set vigilance level p and a strategy for its adaptat ion to be described in Section 5.

Output node positions are represented by the weight vectors, wj for node j , associated with the network. The initialization of these vectors is described in Section 4.

1. Select a city, m, to be presented to network (represented by the vector Xm)- 2. Allow a bot tom-up competit ion to find the node i closest to city m (Euclidean distance). The

winner i is found from a competit ion as follows: i is the node with the minimum

net(j) = [Ix,, - wj',] (1)

where [',a - bl] is Euclidean distance. We say the winning node "claimed" city m. However, a vigilance parameter p is used to disqualify node i if win( i )>p, where win(j) = number of wins for node j .

3. Adapt the weights of node i and its neighbors (defined below) using Kohonen 's formula [15]

w~TW w~}d + aF(aO.)(Xk m old, = --Wkj) for node j (2)

k = 1 , . . . , d

j = I , . . . , N

"Conscientious" neural nets for TSP 125

where c~ is a fractional learning rate, i indexes the winning node, w# represents the weight vector for node j , Xm gives coordinates for city m, and F(di~) is given as

F(dik ) = (1 -d ik / t ) #, dik <L (3)

= 0 otherwise

L = parameter defining boundaries of neighborhoods.

dik = min[li - kl, N - li - k[] where lxl represents absolute value.

The "win factor" for vigilance is updated for the winning node i as

win(i) = win(i) + 1.

4. Set m = m + 1. I f m < N + 1, go to step 1. I f m = N + 1, set m = 1, and decrease learning rate a. Set all win factors to zero. Increase parameter L in this step (see Section 4.2.1). Go to step 5.

5. Convergence tests: (a) I f node positions are within an acceptable distance of cities they claim (e.g. 0.009) A N D nodes have settled on cities (competition winners are not changing), then stop. Otherwise, go to step 1. (b) I f a user specified number of iterations have occurred, stop. Otherwise, go to step 1.

We use convergence 5b in this work. Using criterion 5a requires many more interations of the algorithm and produces only marginally improved tours [7]. The algorithm was implemented in For t ran and run on a PC-486 computer (25 MHz). A single pass of the algorithm (one presenta- tion of the list of N cities) requires O(N 2) computat ion in software simulation. In hardware implementations, the complexity could be reduced to O(N).

The next sections describe in detail how the parameters of the algorithm above are set, and why.

4. I N I T I A L I Z A T I O N C H O I C E S A N D P A R A M E T E R S E L E C T I O N

To aid in achieving the goals stated in the Introduction, the following subsections describe the central issues on which we focus. These issues relate to how we can realistically implement the algorithm for reliable results. Then, Section 5 gives results for the strategies used on a variety of problem types.

The central question addressed is, what choices must be made so that a vigilant net approach can effectively solve a TSP? The critical "open" choices and parameters include weight vector initialization, learning rate initialization and adaptation, neighborhood size initialization and adaptation, and vigilance parameter initialization and adaptation. The number of iterations must also be selected. In Section 4.1, we discuss how we addressed weight vector and learning rate choices on the basis of previous research. In Section 4.2, we describe newly chosen strategies for neighborhood size decisions and number of iterations. Section 5 then addresses vigilance parameter selection and adaptation.

In the following sections we review the impact of previous findings for the guilty net. The investigation used such findings to decide on intelligent choices for weight vector initialization, learning rate, and neighborhood adaptat ion for the vigilant net. Previous research on adaptive neural networks is brought together and applied here.

4.1. Weight initialization Experimentation has revealed the validity of the following conjecture: since a convex hullt can

provide the foundation for a good starting subtour for conventional TSP algorithms [11], a similar approximat ion should be effective for a neural approach [6]. It is well-known that ordering the points on the convex hull gives a good starting subtour for a number of conventional TSP heuristics. Previous research [6] demonstrated for the guilty net that initializing weights into a ring or square derived f rom the minimum and maximum coordinates of all points to be traversed proves effective for both uniformly and non-uniformly generated data. The intuitive reasoning behind its use is supported by experimental results showing that it can lead to improved (shorter) tours. The variations yielded by random starting weights are avoided by the hull initialization. Moreover, it can

tThe convex hull of a set of N points is the smallest convex set which contains all N points.

126 Laura Burke

be easily found from the data itself since, for an adaptive net, only a rectangle containing the data points is needed (and not the actual convex hull itself).

4.2. Learning rate adaptation

With the goal of automating all parameter choices and ensuring fast termination of the algorithm, we used a simple rule for setting and altering learning rate. Analogies to stochastic approximation theory and empirical results both support the basic strategy of gradually decreasing the learning rate [18, 19]. Initially, we set the learning rate a(0) to one, and linearly decrease it as follows

a(t + 1) = a(t) - a(O)/K (4)

where K is the number of iterations (complete passes through the data set). Clearly, then, after K iterations the learning rate o(K) will equal zero. Such a strategy forces fast termination of the algorithm. Favata and Walker [5] also support a linear decrease in the learning rate in their experimentation. The decision then becomes, what should be the value of K? We address this issue next.

4.3. Number of iterations, K

The selection of number of iterations, K, becomes a critical selection given that learning rate decay is linear and dependent on K. The assumption is that larger values of K will yield somewhat improved tours, but the improvement is weighed against the greater need for speed in a constructive heuristic such as the vigilant net. We have previously used a multiplicatively decreasing learning rate and found that the processing of the guilty net could be terminated before full convergence without significant effect. Nodes which corresponded to more than one data point might result, but arbitrarily ordering those clustered cities did not appear to significantly affect tour length [5].

Favata and Walker [5] used a linearly decreasing learning rate and found, for their SOFM-based network (without separation mechanisms) that the necessary number of iterations actually decreased with problem size. Further, experimentation has suggested that processing the net for N iterations, where N is the number of cities, did not yield significantly better results than for K = fN , where f <0.5 [7]. When K increases with N, an O(N 3) algorithmic complexity results (in software simulation). Such complexity is not especially attractive for a tour construction heuristic. To compare, the multiple nearest neighbor algorithm has complexity O(N 3) with an average case performance of 15% above optimal. The space filling curve heuristic is much faster, with complexity O(N log N) and average (guaranteed) performance of 25% above optimal. We use these comparisons to guide experimentation with the guilty net.

The adaptive nature of the approach suggests that very short processing times may be selected a priori to generate less refined but still acceptable tours. In the event that a tour improvement heuristic would be applied to the constructed tour, this option appears even more attractive. Thus we will investigate the network's performance under the fast processing assumption. Since the learning rate decreases from one to zero independent of the size of the problem, 10 iterations, for example, would effect a fast decrease and serve as a minimally feasible number. However, the network relies on somewhat gradual linear decrease in learning rate and neighborhood size (Section 4.2), so 20 iterations (double the absolute minimum) seems reasonable to use as a fast (but sufficient) number. To assess the advantage of further processing, we also used 40 iterations (double the "fast" number).

4.4. Neighborhood adaptation

Although the guilty net used a heuristically chosen, fixed neighborhood size (parameter L in the algorithm) and an increasing /3 (distance effect) parameter, further experimentation here revealed a superior approach. The effect of changing/3 appears to be minimal, and once set at 10 we leave it fixed. Instead, following the ideas first used by Durbin and Willshaw [14], we gradually decreased the neighborhood size, L. Beginning with a "large" neighborhood (consisting of about 10°/O or so of the nodes in the network) and decreasing the size to two nodes (a minimal neighborhood) yielded superior results overall, and was the strategy chosen.

5. VIGILANCE STRATEGIES--RESULTS

Section 4 reviewed the bases for a priori choices regarding parameter strategies. We have exploited

"Conscientious" neural nets for TSP 127

previous results detailed above, including those which showed that a fixed, "convex hull" starting point yields good tours. Thus, we eliminate randomness of initialization. We linearly decrease learning rate from one, and linearly decrease the size of the neighborhood (decrease the L parameter).

The goal of the current investigation is to determine if automated parameter selection/tuning can yield good tours. Thus the selection of a strategy for implementing vigilance remains. We investigated two strategies, selected because they require no problem dependent "tuning" and satisfy our need for automation. They are adaptive vigilance and strict vigilance, and the discussion below describes them in greater detail.

5.1. Vigilance strategies

A fixed "strict" vigilance strategy was investigated. While the strictest vigilance level would correspond to a parameter p = 1, we chose a parameter p = 2 to suitably implement strict vigilance. This fixed vigilance level ensures, at all times, that no node claims more than two cities. Using a parameter value of two allows more freedom in node competition (than usingp = 1), which is useful for tour refinement, and also bounds the number of duplicates. When a node claims two cities, insertion into the tour of the extra city is unlikely to significantly affect tour length and is a trivial operation.

Adaptive vigilance refers to a strategy which initializes vigilance as a "strict level" and then increases or decreases the level on the basis of the ring length. The ring length is the total distance on the ring containing all nodes, which corresponds to tour length in an approximate, but gradually more precise, manner. If the change in ring length from one iteration to the next is positive (indicating a worsening of the tour) then vigilance level is decreased (leading to a higher vigilance parameter, p). Typically, the vigilance parameter p never increases to more than 20 (for a 100 city problem), though it may change nonmonotonically throughout. Thus, feedback from the ring of nodes itself is used to adjust vigilance.

5.2. Problem characteristics and comparisons

The data types consist of eight separate categories, which can be further described as either uniform or clumped, and average, high or low density. We shall refer to high density problems as dense, and low density problems as sparse. In each of the eight categories, ten problems were randomly generated. The categories are given in Table 1. The variety of data types aided in identifying strengths and weaknesses of the strategies.

For "truncated normal" categories, problems were generated from the family of truncated normal distributions on the unit square

f~ = cexp{ - [ (x - 0.5) 2 + (y - 0.5)2]/2tr 2}

where c is such that

J f ~ = 1.

As the value of c~ increases above 1, the distribution generates points resembling uniformly generated points. As ~r decreases, points tend to cluster about the point (x, y). Bartholdi and Platzman showed that the SFC heuristic has the appealing property of yielding shorter tour lengths

Table 1

Uniform/ Density Description clump? (cities/unit)

1. Uniformly distributed unit square 100 city problems Uniform 2. Uniformly distributed unit square 200 city problems Uniform 3. Uniformly distributed over (0, 2) 100-city problems Uniform 4. Uniformly distributed over (0, 1) 50-city problems Uniform 5. One truncated normal over (0, 1) 100-city ("single clump") Clump 6. One truncated normal over (0, 1) 200-city Clump 7. One truncated normal over (0, 2) 100-city Clump 8. Two truncated normals over (0, 1) 100-city ("two clumps") Clump

Average (100) Dense (200) Sparse (25) Sparse (50) Average (100) Dense (200) Sparse (25) Average (100)

128 Laura Burke

for problems with ~r< 1; such a result cannot be claimed for many heuristics. In fact, the popular strip heuristic will not yield shorter tours for such problems, even though the optimal tour length is expected to decrease as points become concentrated. In categories 6-8, cr values of 0.1 and 0.2 were used. In the double clump data, clumps were centered at roughly opposite corners of the unit square.

We compared all vigilant net results to the solution obtained using the multiple nearest neighbor heuristic, and the predicted performance using the SFC heuristic. The multiple nearest neighbor (MNN) heuristic yields an O(N 3) constructive heuristic with an empirically determined average case performance of 15% above optimal [11]. Recall that the very fast SFC heuristic finds tours in O(NlogN) time which are almost surely 25% longer than optimal. Thus, the MNN heuristic typically performs better than the SFC but is much slower, giving a benchmark on performance, while the SFC gives a benchmark for speed.

Primarily the investigation focused on how the two "automated" vigilance strategies would perform empirically, how data characteristics would affect results, and how the number of iterations influenced performance. Comparison with the asymptotic optimal and SFC predicted performance (where possible) and with the nearest neighbor benchmarks guided assessment.

5.3. Results--full processing time

As discussed earlier, we do not allow the procedure to reach convergence in the strictest sense (criterion 5a in the algorithm). Previous research [6, 7] indicates that the improvement in tour length may not be justified by the excessive computation time.

Using 40 iterations, the fixed strict level of p = 2 performed slightly better than the adaptive vigilance strategy--an improvement of only 0.6%. However, for twenty iterations, the two strategies performed nearly identically, with an overall difference of only 0.007%. The slight relative differences between the two approaches suggest that they may both represent reasonable, easy to implement versions of the vigilant network. Neither approach requires parameter tuning.

Even more interesting is the comparison of results between forty and twenty iterations. This finding suggests the attractive possibility that the neural network method can yield good results for short processing times. For the adaptive strategy, the difference between 40 and 20 iterations is only 0.7%.

Comparing minimal processing neural network results (adaptive vigilance strategy) to multiple nearest neighbor and predicted SFC performance yields further insights (Table 2). While the MNN heuristic always performs better, the vigilant net yields an average of only 5.9% longer tour lengths (at a software simulated speedup of an order of magnitude). More interesting is that the disparity between the two approaches diminishes with problem size. For 200 city problems, the average difference is only about 3.7%. However, when compared to the fast SFC heuristic, the vigilant net

Table 2. Comparison of SFC, VGN, and MNN tour lengths shown are averaged over ten problems per type

Problem type SFC (Pred.) VGN MNN

1 9.56 9.50 8.86 2 13.52 12.72 1229 3 19.12 17.53 17.05 4 6.76 6.81 6.26 5 9.14 7.29 6.83 6 12.92 10.25 9.86 7 12.92 7.57 7.16 8 9.14 (lower bound) 8.72 8.00

Table 3. Computation time comparisons: vigilant net and MNN heuristic times shown are in seconds (PC-486 25 MHz)

Problem size: 200 Problem size: 100 Problem size: 50

Vigilant net 60 11 6 (40 iter.) Vigilant net 30 5 4 (20 iter.) M N N 70 13 2

"Consc ien t ious" neural nets for TSP 129

generally always performs considerably better than the predicted SFC performance (17.7% longer tours on average yielded by SFC). It is difficult to tell if any trend toward a real difference in performance for uniform or clumped, sparse or dense data is emerging, More telling perhaps is that no significant differences in performance emerge except for problem size.

Typical computation times are shown in Table 3. Clearly the 20 iteration approach is much faster than the 40 iteration approach, as expected. Also, the fast processing approach improves considerably on the MNN method, especially as the number of cities increases. The vigilant network computer code also generates much more output than did the MNN computer code, accounting for a large part of the computation time (and explaining the results for small problem sizes).

6. C O N C L U S I O N

While the vigilant net results compare modestly to those obtained with the multiple nearest neighbor heuristic--a representative constructive heuristic having O(N 3) computation time--some promising features emerge. The neural network-based vigilant net, even in software simulation, processes an order of magnitude more quickly than the MNN heuristic. Its tour lengths exceed those yielded by the MNN by only 5.3% on average. However, the difference is only 3.7% for the largest (200 city) problems. The SFC heuristic, an order of magnitude faster than the software simulated vigilant network, provides tours on average 15.8% longer (as a conservative estimate). In hardware implementations, the vigilant net could be speeded up by an order of magnitude, yielding a comparable heuristic to the SFC which may yield tours of quality comparable to the MNN.

Acknowledgements--This research was supported in part by NSF Grant ECS-9110846, and by a NSF National Young Investigator Award.

R E F E R E N C E S

1. J. J. Hopfield and D. W. Tank, Neural computation of decisions in optimization problems. Biol. Cybernet. 52, 141-152 (1985).

2. G.V. Wilson and G. S. Pawley, On the stability of the travelling salesman problem of Hopfield and Tank. Biol. Cybernet. 58, 63-70 (1988).

3. J. C. Fort, Solving a combinatorial problem via self-organizing process: an application of the Kohonen algorithm to the traveling salesman problem. Biol. Cybernet. 59, 33-40 (1988).

4. B. Angeniol et al. Self-Organizing feature maps and the traveling salesman problem. Neural Networks 1,289-293 (1988). 5. F. Favata and R. Walker, A study of the application of Kohonen-tye neural networks to the traveling salesman problem.

Biol. Cybernet. 64, 463-468 (1991). 6. L.I. Burke and P. Damany, The guilty net for the traveling salesman problem. Computers Ops Res. 19, 255-265 (1992). 7. L. I. Burke, Neural methods for the traveling salesman problem: insights from operations research. Neural Networks 7,

681-690 (1994). 8. S. Lin and B. W. Kernighan, An effective heuristic for the traveling salesman problem. Opns Res. 21,498-516 (1973). 9. G. Carpenter and S. Grossberg, ART2: self-organization of stable category recognition codes for analog input patterns.

Appl. Opt. 26, 4919-4946 (1987). 10. D. Rosenkrantz, R. Stearns and P. Lewis, Approximate algorithms for the TSP. Proc. 15th IEEE Symp. on Switching and

Automata Theory, pp. 33-42 (1974). 11. E. L. Lawler, J. K. Lenstra, A. H. G. Rinnooy Kan and D. B. Shmoys, The Traveling Salesman Problem. Wiley, New

York (1985). 12. L. K. Platzman and J. J. Bartholdi, Spacefilling Curves and the Planar Travelling Salesman Problem. J. A.C.M. 36,

719-737 (1989). 13. X. Xu and W. T. Tsai, Effective neural algorithms for the traveling salesman problem. Neural Networks 4, 193-205

(1991). 14. R. Durbin and D. Willshaw, An analogue approach to the traveling salesman problem using an elastic net method.

Nature 326, 689-691 (1987). 15. T. Kohonen, Self-organization and Associative Memory. Springer, Berlin (1984). 16. D. Desieno, Adding a conscience to competitive learning. Proc. Int. Conf. on Neural Networks I, 117-124. IEEE Press,

New York (1988). 17. Hecht-Nielsen R, Neurocomputing. Addison-Wesley, Reading, Mass. (1990). 18. C. Darken and J. Moody, Fast adaptive K-means clustering: some empirical results. Proc. Int. Conf. on NeuralNetworks.

IEEE Press, New York (1990). 19. J. Hertz, A. Krogh and R. G. Palmer, Introduction to the Theory of Neural Computation. Addison-Wesley, Reading,

Mass. (1991).