a fuzzy valuation-based local search framework for combinatorial problems

A Fuzzy Valuation-Based Local Search Frameworkfor Combinatorial Problems

1

ARMANDO BLANCO [email protected]

Depto. de Ciencias de la Computacion e I.A, E.T.S. Ing. Informatica, 18071 - Granada, Spain

DAVID A. PELTA2 [email protected]


JOSE-L. VERDEGAY3 [email protected]


Abstract. A novel local search method is presented. One of the new elements of this Fuzzy Adaptive

Neighborhood Search (FANS ) algorithm is a fuzzy valuation, which is used to measure the degree to which the

solutions that are considered at the decision stages accomplish a certain qualitative property.

FANS is analyzed from two perspectives: first, it is shown how FANS may be adapted to behave like other

traditional local search techniques by means of suitable definitions for the fuzzy valuation component. Second,

comparisons are made to show the potential of the method as a general purpose optimization tool, when none or

minimal knowledge of the problem being solved is available.

Both aspects make FANS a valuable tool regarding further developments within the context of decision support

systems involving heuristic algorithms.

1. Introduction

The design, construction and search for exact algorithms solving real life problems are, as

a whole, key objectives of Computer Science. Although this kind of problems generally

has a high level of difficulty, they need to be solved because of their importance. Both

facts, difficulty and importance, together with increasing computer power, encourage the

development of heuristics which, even though it may lead to non-optimal solutions, can

solve the problem at hand based on the decision makers’ satisfaction. In this way the

decision maker may prefer to obtain satisfying solutions according to his wishes than

optimal ones.

Consequently, in order to face a problem in terms of satisfaction, and not only

optimization, heuristic methods must search for solutions not only providing good values

for the objective function, but also having additional characteristics predefined by the

decisor. In general, those characteristics will be of a subjective nature and therefore they

could be well modelized by fuzzy sets.

From our point of view, the last decades have witnessed a flow of information from

classical fields, such as Operational Research or Control Theory, to the area of Fuzzy Sets

and Systems, which have provided very fruitful results. Now, terms like ‘‘fuzzy

mathematical programming’’, ‘‘fuzzy control’’, ‘‘fuzzy rule based systems’’, etc. sound

familiar to a wider audience (IFSA99 (1999), Delgado et al (1994)). However, it has not

Fuzzy Optimization and Decision Making, 1, 177–193, 2002# 2002 Kluwer Academic Publishers. Printed in The Netherlands.

been so usual to see this continuous interaction in the other direction. Classical areas do

not take into account the possible benefits of ‘‘merging’’ with fuzzy logic ideas.

In order to bridge this gap, the approach here proposed will consider a classical

method from the optimization problems area, and by making use of very basic elements

of the Fuzzy Sets and Systems field, it will provide a robust and adaptive heuristic

optimization tool.

It is well known that for a variety of problems and situations several heuristic methods,

sharing a common core and differing in minor elements, may be available. A set of

heuristics implies a set of possibly different solutions for a problem at hand; so it would

therefore be desirable for decision makers to have some kind of shell or framework of

heuristics that can be tailored for specific purposes, and it can be used as a kind of

heuristic guide.

In this work we will show how Fuzzy Sets and Systems may help to design and obtain

a novel (fuzzy set-based) heuristic method oriented to deal with difficult and well

defined problems. Besides, and because of the presentation of just another optimization

heuristic might be not relevant, we will also show how the proposed method may be

interpreted as a local search framework, which is able to capture or reflect the qualitative

behaviour of other heuristics.

The method we introduce here is called FANS (Fuzzy Adaptive Neighborhood Search)

and its very basic ideas were previously presented in Pelta et al (2000a, 2000b). FANS is

termed Fuzzy, because solutions are also evaluated by means of fuzzy valuations and

Adaptive because its behavior is adapted as a function of the search state. Fuzzy

valuations are represented in FANS by fuzzy sets, and are used as a mechanism to

represent abstract concepts like ‘‘Acceptability’’ or ‘‘Similarity’’. Within FANS, a fuzzy

valuation is used to define a semantic neighborhood and to obtain, for example, the

degree of ‘‘Acceptability’’ of a neighborhood solution. Such degrees are used to guide

the search.

Ideally, this key component of FANS may be used to capture vague or imprecise

knowledge, but here we propose to analyze the fuzzy valuation as a useful mechanism to

induce particular behaviors of the algorithm. We will show how it can be defined to obtain

qualitative behaviors like those induced by Random Walks, Hill Climbing and Simulated

Annealing procedures. In addition, the simulation of Tabu Search-like behavior will be

briefly considered.

Summarizing, the aim of this article is twofold: first, to show how FANS is able to

capture or reflect the behavior of some other traditional local search methods, and second

to compare its optimization performance against other optimization tools when minimal

knowledge of the problem being solved is available.

In order to achieve both objectives, the article is organized as follows: in Section 2 the

basic elements of neighborhood search methods are presented. Then in Section 3 the main

components of FANS are reviewed and the use of fuzzy valuations is described. In Section

4 it is shown how fuzzy valuations could be defined for FANS to simulate the behaviour of

traditional local search techniques. Experiments and comparisons against Simulated

Annealing and Genetic Algorithms are presented in Section 5 to evaluate the potential

of FANS as a general purpose optimization tool.

BLANCO, PELTA AND VERDEGAY178

2. Neighborhood Search Algorithms

Awide range of optimization methods rest on the idea of generate and test. In some way, a

solution to the problem of interest is generated and then tested for quality. If it is not good

enough, the process is repeated.

This kind of methods, known as stochastic iterative methods, could be roughly classified

in two classes (Corne et al (1999)): local search and population-based search. Here we will

only focus on local search methods, and therefore population-based search methods will

not be described. The interested readers in these last methods are referred to Michalewicz

(1998).

Local search is also known as neighborhood search and these kinds of methods are

simple and powerful tools to deal with combinatorial optimization problems, which is the

discipline of decision making in the case of discrete alternatives. Following Aarts and

Lenstra (1997), and assuming a minimization problem, we define an instance of a

combinatorial optimization problem as a pair (L, f ), where the solution set L is the set

of feasible solutions and the cost function f is a mapping f : L ! R. The problem is to

find a globally optimal solution; i.e., an i*2 L such that f (i*) � f (i) for all i 2 L.Now, let (L, f ) be an instance of a combinatorial optimization problem. Then a

neighborhood function is a mapping N : L ! 2L, which defines for each solution i 2 La set N ðiÞ � L of solutions that are in some sense close to i. N ðiÞ is called the

neighborhood of solution i.

Given an initial solution s0 and an adequate definition of the neighborhood N ðs0Þ, localsearch methods look for a solution s1 2 N ðs0Þ verifying a certain condition; for example,

improving the cost. If such a solution exists, s1 is taken as the current solution and the

process is repeated looking for some s2 2 N ðs1Þ.The algorithm stops when a certain external stopping condition holds or a non suitable

solution skþ1 2 N ðskÞ was found. The solution sk is considered as a local optimum.

Figure 1 shows the basic steps of a local search method. The routine improve (x) returns,

if possible, a new solution y from the neighborhood such that f ( y) > f (x) for some cost

function f and a maximization problem. In the other case, ‘‘NO’’ is returned and the local

optimum found is the output of the algorithm.

Figure 1. Local Search Scheme.

A FUZZY VALUATION-BASED LOCAL SEARCH 179

This scheme, where the accepted move is always done to a better solution, is called Hill

Climbing or iterative improvement. If the solution taken is any one from the neighbor-

hood, the obtained scheme is called Random Walks.

It is well known that these kinds of methods get stuck in local optimal solutions. There-

fore, several extensions have been proposed in order to overcome this limitation. Exam-

ples of those extended methods are Simulated Annealing (SA ) and Tabu Search (TS ).

SA differs from local search in three main aspects. The first one is related with the

stopping criterion: SA uses an external condition (for example number of cost function

evaluations) while local search stops when a local optimum is reached. Another difference

lies in the fact that not only better solutions are taken into account. The acceptance is

governed by a parameter T enabling the probabilistic acceptance of worse solutions.

Because T is varied as the run progress, the acceptance criterion changes, while in local

search the acceptance criterion is fixed.

TS is quite similar to SA, but now the generation of solutions is controlled in terms of the

‘‘history’’ H of the search. The history is a list of non-valid or tabu moves and its main

intention is to avoid the risk of cycling. TS always moves to the best available

neighborhood solution, although its cost could be worse. From a practical point of view,

it is usual to perform several runs of the chosen method, each one starting from a different

initial solution, finally reporting the best solution ever found.

In terms of exploitation capacities, the lower level is achieved by the Random Walks

procedure while the highest level of exploitation corresponds to Hill Climbing. SA and TS

lie somewhere in the middle, not in a fixed position but moving as long as the execution

progresses.

3. Description of FANS

Having reviewed the basic components and some examples of local search methods, we will

now present the basic elements of our Fuzzy Adaptive Neighborhood Search algorithm.

FANS presents two novel elements. First, and in addition to the usual cost function

evaluation, solutions are also evaluated in terms of a fuzzy valuation. Fuzzy valuations are

represented in FANS by means of fuzzy sets and sometimes they are called fuzzy

properties. In this way, we can talk about solutions verifying the property to some degree.

Second, a new mechanism is used to escape from local optimum. This mechanism,

which is described below, is based on the fact that the neighborhood of a solution is

induced by the modification or move operator. A change in the operator implies a change

of neighborhood.

If we think of the search space as a graph, with nodes representing the solutions, each

operator represents a particular set of edges. An edge between solutions s1 and s2 will

appear if s2 could be obtained through one application of the operator over s1. When the

operator is changed, the set of edges is modified. In this way, a set of different transitions is

obtained. For example, if we consider a binary solution s and two operators modifying one

and three variables respectively, it is clear that one application of each operator over s will

produce different solutions.


FANS is based on four main components, which interact in a suggested way. Each

component has a clear responsibility and must comply with certain characteristics which

are described below.

3.1. Modification Operator OOO

The operator is used to construct new solutions si from a current one s. Each application of

the operator over the same s returns a different solution, meaning that a randomization

element must be present in its definition.

This operator has to be adaptable in a certain sense, for example providing tunable

parameters to control its operation. As an example, we can consider the class of k –

exchange operators for permutation based solutions. These operators obtain new solutions

making k, possibly random, exchanges from a reference solution.

3.2. Fuzzy Valuation ��( )

Within FANS, a first evaluation of the solutions is performed with the objective function.

After that, a fuzzy valuation is applied in order to obtain a fuzzy measure for the solution:

i.e. we evaluate the membership degree of the solution to a fuzzy set. Having the fuzzy set

of ‘‘good’’ solutions, we can measure the goodness of the solution of interest.

In this way, given a current solution a and a generated solution b we could think about

how similar a and b are, or how close they are, or even how different b is from a. Similar,

Close, Different will be fuzzy sets represented by appropriate membership functions � and

FANS will deal with ‘‘degrees of membership’’ at the decision stages to be meant as

accomplishment degrees of the qualitative property under consideration.

3.3. Operator Scheduler OS

The modification operator O is adapted as a way to search for neighborhood solutions in

different modes.

OS encapsulates the adaptation strategy defined and it will be executed when certain

conditions are met. For example, when the search is trapped in a local optimum, or no

progress has been made in a certain number of iterations.

As response, the tunable parameters of the operator will be adapted and, therefore, a

modified version of it will be returned.

If a k – exchange operator is used, OS will vary the value of k in some way. For

example, OS may increase/decrease k until some limit is reached, and then the value may

be decreased/increased.

3.4. Neighborhood Scheduler NS

This component is used to generate and select a new solution from the neighborhood.

Given the current operator O, the current solution s, the operational neighborhood of s is


defined as:

N ðsÞ ¼ fsi j si ¼ OiðsÞg

where OiðsÞ represents the ith call to O over s.

Now taking the fuzzy valuation �, the semantic neighborhood is defined as:

N^

ðsÞ ¼ fs 2 N ðsÞ j �ðsÞ � �g

In other words, ^N ðsÞ represents the �-cut of �; the solutions of interest are those satisfyingthe fuzzy valuation to some specified degree.

Two procedures are applied within the scheduler. First, a generator is executed to obtain

solutions from the semantic neighborhood sampling the search space with O. After that, a

selector procedure has to decide which one is returned, taking as a basis the degrees of

membership of the obtained solutions. For example, we may return ‘‘the most different

solution’’ or ‘‘the less similar solution’’, where different and similar are fuzzy valuations.

Fuzzy valuations act as the acceptance criterion, so different definitions lead to different

criteria, which in turn induce different behaviors for FANS.

Several situations could arise in the scheduler, for example when the generator could

not obtain any solution s 2 ^N ðsÞ. In such a case, the interaction between components

must ensure that any of the elements used by the generator change. In this way, the

generator would change its behavior leading to a possibly different result the next time it

is executed.

The elements described above suffice to construct a local search algorithm, but the

problem of local optimum is not solved. It is impossible for any of these algorithms to

avoid being trapped in a local optimum, so escape mechanisms are needed.

In order to escape from local optimal solutions, FANS provides two mechanisms: one is a

‘‘classical’’ Restart operator which resumes the search from a completely new solution or

from a special modification of the current one.

The other mechanism is related with the fact that the modification operator determines

the landscape or search space (Jones (1995)).

When the search is trapped, and because a small fraction of the neighborhood is

explored, we can only say that ‘‘it seems’’ that we are trapped in a local optimum. FANS

tries several versions of the operator (produced by the operator scheduler) in order to

search for solutions in a different way (in other words, on different landscapes). If the

search also fails with those different operators, we have more evidence about the local

optimality of the current solution. At this point, the ‘‘classical’’ mechanism could be

applied.

The pseudo-code of FANS is shown in Figure 2. The algorithm will finish when a certain

external condition holds.

Each iteration starts with a call to the neighborhood scheduler NS with the parameters:

current solution scur, fuzzy valuation �( ), and modification operator O. Two results are

possible: either an ‘‘acceptable’’ neighborhood solution snew was found or none was.


In the first case snew is taken as the current solution and �( ) parameters are adapted. In

this way, we are varying our fuzzy valuation as a function of the context.

If NS could not return any acceptable solution, an exception condition (OK ¼ false) is

raised. No solutions were good enough in the neighborhood induced by the operator. In

this case, the new escaping mechanism is applied. The operator scheduler OS is executed,

returning a modified version of O. The next time it is executed, NS will have a modified

operator to search for solutions. In some way, this behaviour resembles that of variable

neighborhood search (Hansen and Mladenovic (1999)).

Finally, another exception condition, trappedCondition(), is raised when Top iterations

were done without improvements in the best solution found; this situation may indicate

cycling. In this case, the doEscape() procedure is executed, the cost of the new solution is

evaluated and �( ) is adapted (Pelta et al (2000a, 2000b)).

4. Fuzzy Valuations as a Tool to Induce FANS Behaviour

The global behavior of FANS is a function of its components and of their interactions.

Bearing in mind Figure 2, we will discuss how FANS may be adapted to behave or reflect

the behavior of other local search techniques. This behavior has to be understand in

qualitative terms. We will show how this adaptation may be mainly obtained through the

appropriate definition of the fuzzy valuation.

First, we need a more precise definition for the neighborhood scheduler. For simpli-

city, we will use a FirstFound scheme with this simple definition: given a solution s,

Figure 2. FANS Pseudo Code.


the fuzzy valuation �, and the minimum level � of required ‘‘quality’’, the scheduler

will return the first solution r 2 ^N ðsÞ found (within a maximum number of trials).

Suppose the fuzzy valuation � represents some notion of ‘‘Acceptability’’. We will also

assume a minimization problem, with a current solution s and N ðsÞ the neighborhood

of s.

With the above elements, we are in position to indicate how FANS is able to capture the

behavior of traditional local search algorithms.

4.1. FANS with Random Search-like Behavior

A trivial way to achieve this behavior is setting TrappedSituation ¼ True and canceling the

call to NS. In this way, at each iteration the procedure doEscape( ) is executed generating a

completely new solution. The best solution found is then saved.

To obtain this behavior making use of the fuzzy valuation, we have to consider any of

the solutions from the neighborhood of the current solution as not good enough. Using

�( f (s)) ¼ 0, 8 s 2 N ðsÞ, and setting � ¼ 1, each call to the neighborhood scheduler

will fail. The operator scheduler will be called and this situation will be repeated for a

number of times. After that, the procedure doEscape( ) will be executed generating a

new solution.

This second approach is quite inefficient from the point of view of how the available cost

function evaluations are used. Obviously, anyone interested in performing a random search

will apply the first approach.

4.2. FANS with Random Walks-like Behavior

In order to obtain this behavior, all we need is to consider any solution from the

neighborhood as equally acceptable. Using a fuzzy valuation with �( f (s)) ¼ 1 8 s 2N ðsÞ, any solution from the operational neighborhood will have the chance of being

selected. In this way, the desired behavior is obtained.

4.3. FANS with Hill Climbing-like Behavior

Again, we could also obtain the required behavior manipulating the fuzzy valuation. Using

any valuation where �( f (s)) ¼ 1 if f (s) < f (s) and setting � ¼ 1, we will only consider

those solutions improving the current cost as acceptable. The classical Hill Climbing

method will stop when a local optimum is reached. In the context of FANS, the

implementation of a multi-start version is straightforward.

4.4. FANS with Simulated Annealing-like Behavior

SA uses an external parameter, called ‘‘Temperature’’ to manage the acceptance of new

solutions. Better solutions are always accepted, but initially, SAwith high temperature may

accept bad solutions with high probability. As the execution goes on, the temperature is


lowered, decreasing the probability of acceptance of worse solutions. Towards the end of

the run, only better solutions will be considered.

In order to achieve this behavior with FANS, it is enough to use the following definition

for the fuzzy valuation ‘‘Acceptability’’:

�aðq; s; �Þ ¼

0:0 if f ðqÞ > �

��f ðqÞ��f ðsÞ if f ðsÞ � f ðqÞ � �

1:0 if f ðqÞ < f ðsÞ

8>>>><>>>>:

with f the objective function, s the current solution, q ¼ O(s) a generated solution, and �the limit for what is considered as acceptable.

It is easy to see that the key element is the definition of the � value which determines

what solutions belongs to the neighborhood or not.

Defining � as some function h(s, q, t) where t is an external parameter representing, for

example, the current number of iterations or the number of cost function evaluations

performed, the deterioration limit may be reduced as the simulation progresses. Towards

the end of the run, only those solutions better than the current one will be taken into

account. For example, h may be defined as h ¼ f ðsÞ* ð1þ 11þeð f ðsÞ�f ðqÞÞ=TÞ where the second

term of the sum is the logistic probability used within Boltzmann trials.

4.5. FANS with Tabu Search-like Behavior

The basic scheme of TS, makes use of a history or memory H in order to constrain and

guide the search. Typically, the history H is used to constrain or discard the generation of

particular neighborhood solutions, which are included in a tabu list. In this way, cycling

situations are reduced, and the search can be guided to promising or unexplored regions

(Glover and Laguna (1997)).

The use of such memory can not be captured by the fuzzy valuation, or at least not

easily. However, this is not a problem because the use of a memory structure, if

desired, could be added within the generator procedure of the neighborhood scheduler,

which is the component responsible for the generation of solutions. Then, the selector

procedure must also be changed in order to return the best available solution (in the

sense of the fuzzy valuation) leading to a new neighborhood scheduler, which may be

called BestFit.

4.6. FANS as General Purpose Heuristic

In the previous paragraphs we showed how FANS may be tailored to reflect the behavior of

other techniques, mainly through the definition of adequate fuzzy valuations. Up until this

point, we have emphasized FANS as a local search framework, but we now wish to point

out the potential of FANS as a general purpose optimization tool.


In order to apply FANS to a particular problem, a particular instance of FANS has to be

implemented: particular definitions for the components must be provided, and some

parameters have to be set.

General purpose definitions are easily obtained in some cases. For example, we may use

a modification operator based on k-exchange and an operator scheduler modifying k in

some way. As a fuzzy valuation, the ‘‘Acceptability’’ property could be used within a

FirstFound scheme for the neighborhood scheduler.

With such definitions, a canonical and very simple version of FANS is obtained. Of

course, the more problem-dependent the definitions are, the better will be the results.

Each instantiation of FANS leads to an algorithm with its particular, possibly novel,

behavior which is mainly related with the definition of the fuzzy valuation and the value of

the parameter �.We are now in a position to analyze the second aspect of FANS: its performance as a

general purpose optimization tool when compared against other heuristic optimization

algorithms. As test bed we perform experiments over a set of instances of the Knapsack

Problem.

We choose to make comparisons against particular implementations of SA and GA. In

this way, one member of each class of stochastic iterative methods is considered.

5. Algorithms, Experiments and Results

In this section, we present the formulation of the knapsack problem and the test instances

used. Then, particular definitions for FANS components will be presented, and details

about the implemented versions of GA and SA will be shown. Finally, comparison results

will be analyzed.

5.1. Knapsack Problem Instances

The knapsack problem is one of the most studied in both the Operational Research and

Computer Science areas. The mathematical formulation is as follows:

MaxXn

j¼1

pj * xj

s:t:Xn

j¼1

wj * xj � C; xj 2 f0; 1g; j ¼ 1; . . . ; n

where n is the number of items, xj indicates if the item j is included or not in the knapsack,

pj is the profit associated with the item j, wj 2 [0, . . . , r] is the weight of item j, and C is the

capacity of the knapsack. It is also assumed wj < C, 8j (every item fits in the knapsack);

andPn

j=1 wj > C (the whole set of items does not fit).

Although it seems very simple, knapsack problems are a real challenge for a variety of

search algorithms (see Pissinger (1995) for recent developments). To the best of our


knowledge, no standard set of classical knapsack problems exists. That’s why we used

random instances to create our own test set.

Given wi ¼ U(1, max), where U is a uniform random number generator, Martello and

Toth (1990) propose the following instances obtained by means of the correlation between

weights and profits:

� Uncorrelated UN: pi ¼ U(1, max)� Weakly Correlated CW: pi ¼ U(wi � t, wi þ t)� Strongly Correlated CS: pi ¼ wi þ k.

where t and k are arbitrary constants. Experimental evidence shows that higher correlation

implies increasing difficulty and it seems that weakly correlated instances are closer to real

problems (Martello and Toth (1990)). Knapsack capacity is calculated as:

C ¼ *

Xni¼1

wi; with 2 ½0 . . . 1�

To construct our test set, 45 problems were generated: 5 for each type of instance and

value of ¼ [0.25, 0.5, 0.75]. The reader must note that higher values of potentially implies

more items in the knapsack, enlarging the number of feasible solutions. For every problem,

we used n ¼ 100, max ¼ 1000, t ¼ max/10 and k ¼ 10. Because the optimum is unknown,

we used the solution of the continuous version (Dantzig bound) for comparison purposes.

Knapsack solutions are represented by binary vectors X, where position i represents the

variable xi. This representation is used in FANS, SA and GA. With no loss of generality, we

assume that the items are sorted verifying the following condition: p1 � p2 � . . . � pn.

In our experiments, infeasible solutions will not be taken into account and they will be

discarded. No repair procedure will be used.

The initial solutions for each method will only contain a unique position set at one. This

reflects the fact that all we know about the instance is that solutions with just one element

are always feasible.

5.2. FANS Specifications for the Knapsack Problem

The proposed definitions here for FANS components are shown below.

5.2.1. Modification Operator: k-BitFlip

Given a solution s, this operator randomly selects k positions and flips the associated bit

values. ‘‘Back mutation’’ is not allowed.

5.2.2. Fuzzy Valuation: Acceptability

The generated solutions will be qualified in terms of ‘‘Acceptability’’; in other words, we

will measure how well the solutions reflects the property or concept ‘‘Acceptable’’.


This property captures the following idea: solutions improving the current cost will have

a higher degree of acceptability than those with lower cost. Solutions with cost below a

certain threshold will not be considered as acceptable. Thus, given the objective function f,

the current solution s, a new solution q ¼ OðsÞ, and the limit � for what is considered as

acceptable, the following membership function comprises those ideas:

�aðq; s; �Þ ¼

0:0 if f ðqÞ < �

ð f ðqÞ��Þð f ðsÞ��Þ if � � f ðqÞ � f ðsÞ

1:0 if f ðqÞ > f ðsÞ

8>>>><>>>>:

As a first approximation, we use � ¼ f (s) * (1 þ scaleFactor), with scaleFactor

2 [0. . .1]. In this work we use scaleFactor ¼ 0.05. Every time the current solution s

changes, the value � is recalculated. In this way, the notion of ‘‘Acceptability’’ is modified

as the current situation changes.

5.2.3. Operator Scheduler

The k-BitFlip operator will be adapted through changes on the parameter k.

Each time the scheduler is called, the current value of k will be replaced by a new

value k obtained as a random integer value from [1, 2 * k]. Also, if k > top ¼ n10,

where n is the number of items of the instance, then k ¼ top.

5.2.4. Neighborhood Scheduler

We will use a Quality Based Grouping Scheme (Pelta et al (2000b)) which makes use of

the membership values provided by the fuzzy valuation in order to select the solution to

return.

The Quality Based Grouping Scheme or RjSjT scheme tries to generate R ‘‘Acceptable’’

solutions with O in at most maxTrials trials, then those solutions are grouped into S fuzzy

sets based on their acceptability degree, and finally T solutions are returned. The second

step may be viewed as a primitive clustering process.

We use here a R ¼ 5jS ¼ 3jT ¼ 1 scheme with maxTrials ¼ 25. The S ¼ 3 fuzzy sets or

clusters are represented by overlapped triangular membership functions. Their boundaries

are adjusted to fit the range [�, 1.0] being � ¼ 0.98 the minimum level of acceptability

required. The sets represent the terms Low, Medium, High for the linguistic variable

‘‘Quality’’.

At the end of the process, T ¼ 1 solution must be returned. The selection rule returns any

solution of the highest quality available. If High quality solutions exist, any one of them is

returned. If the set is empty, the same procedure is tried with the Medium quality set and if

it is also empty, the Low level quality is used.


If no acceptable solution was found, an exception condition is raised. This condition is

then managed in the main loop of FANS.

5.2.5. Additional Elements

At least two options are available to implement for the doEscape( ) procedure. The first

one may be regarded as a classical restart operator, where a new random solution is used to

resume the search from.

The other option, and the one implemented here, applies a special perturbation over the

current solution in a similar way as the used in iterated local search methods. The

perturbation will set to zero 13of randomly selected variables in one. In knapsack terms,

this is equivalent to ‘‘throwing out some elements’’.

The definition of FANS components leads to a behavior ‘‘different’’ from that show by

the local search techniques already described.

In order to diversify/intensify the search, three options are available: to adapt � and �using a fixed modification operator; to modify the operator keeping both parameters fixed;

or to adapt both the operator and the parameters.

Within FANS context, the second option is chosen, but extensions to the other options

are easy to implement.

5.3. GA and SA Details

Although several versions of GA and SA are available from a variety of sources, in order to

produce here fair comparisons with FANS, we decided to code and use our own

implementations. The GA implemented may be regarded as a ‘‘traditional’’ one. The

mutation operator is the same as the modification operator of FANS, but given that no

operator scheduler exists in the GA, the BitFlip operator uses k ¼ 2.

Because no repair procedure is used, the following consideration is taken into account: if

the solution obtained after mutation is infeasible, it is discarded and the operator is applied

again over the original solution. This process is repeated at most four times. If no feasible

solution could be obtained by mutation, the original individual is kept. Mutation is applied

to all individuals with a certain probability.

As the crossover operator, two classical ones are implemented: 1 point and uniform

crossover. In this way we obtain two algorithms: GAop and GAux respectively. Elitism4 is

also used in both versions.

The selection process uses Tournament Selection with tournament size q ¼ 2 within a

(� ¼ 50 þ � ¼ 75) scheme. The population size was PopSize ¼ 100 and crossover and

mutation probabilities were fixed at Pxover ¼ 0.8 and Pmut ¼ 0.2.

In order to implement SA, we may use FANS with SA like behaviour, but we decided to

code it apart following the guidelines presented in Diaz et al (1996). As in the GA case, the

k-BitFlip operator is used with k ¼ 2.

The initial temperature was set to T0 ¼ 10 and proportional cooling is used with Tk+1 ¼Tk * with ¼ 0.93. The temperature is adapted when 15 neighbour solutions were


accepted, or when 2 * n neighbour solutions were generated, being n the dimension of the

problem.

5.4. Results

The experiments compare the behavior of the three algorithms under none or minimal

knowledge of the problem being solved (a fact reflected by the use of very simple

operators’ definitions) and when they are given a fixed amount of resources.

We consider this as a reasonable scenario for a decision maker doing the first tests trying

to gain knowledge on the problem.

Of course, knapsack oriented versions of the algorithms obtain better results than those

shown here. See for example Chu and Beasley (1998) for a successful application of a GA

for knapsack problems.

To compare the algorithms, 30 runs were made for each test instance and algorithm.

Each one ends when maxEvals ¼ 15000 cost function evaluations were done, or when

maxEvals * 1.5 solutions were generated. This limit is defined because only feasible

solutions are evaluated.

Table 1 shows the results obtained. Each value represents the mean of the errors taken

over the 15 problems of each type of instance and its corresponding 30 runs’ results.

Variances are also shown. The error is calculated as:

error ¼ 100 *Dantzig Bound � Obtained Value

Dantzig Bound

Table 1 shows clear differences for UN instances among GAop and the other algorithms.

FANS achieves the better value, also in terms of variance. For CW instances, FANS and SA

have similar values, with important differences against both GA versions. FANS is also

better on CS instances while the other algorithms achieve very similar values.

It must be noted that for FANS, SA and GAop, lower mean error values are reached on

instances CS, contradicting the idea which relates higher correlation with higher difficulty.

This is a point that deserves further research.

To detect significant differences in the mean of errors, t-tests were done for each type of

instance. In addition, t-tests were done for the whole set of instances. The results appear in

Table 2 where a plus sign at position (i, j) stands for algorithm on row i better than that of

Table 1. Mean and Variance of Errors by type of instance.

Uncorr Weakly Corr Strong

Method Mean Var. Mean Var. Mean Var.

FANS 1.0775 0.547 1.7477 1.078 0.8392 1.015

SA 1.6100 3.168 1.8035 1.366 1.3150 1.252

GAop 2.0642 0.802 2.7390 1.176 1.3239 1.257

GAux 1.3174 0.770 2.6290 1.247 1.3656 1.252


column j with confidence level of 95%. A minus sign stands for i worse than j and¼ stands

for no significant differences.

When the type of instance is taken into account, the results show that FANS

consistently outperformed both GA in all three types. FANS also presents better results

in UN and CS than SA, achieving the same performance on CW instances. Both GA

achieve the same performance over CW and CS instances, but GAux is better over UN.

SA performance is better than both GA over UN and CW instances; the three algorithms

are equally good on CS.

If we consider the whole set of results over the 45 test instances without taking into

account its type, we found that FANS achieves significantly lower mean error values than

SA, GAop and GAux. It is followed by SA, which outperforms both GA, and GAux which

results better than GAop, mainly due to the results over UN instances.

To conclude, the relation between the value of used for the calculation of the capacity

and the performance of the algorithms is analyzed. Results are shown in Table 3.

Table 2. T-tests Results. A þ sign at (i, j) stands for algorithm i better than j with

confidence level of 95%. A � sign stands for i worse than j and ¼ stands for no

significant differences.

Uncorrelated Weakly Corr.

FANS SA GAop GAux FANS SA GAop GAux

FANS + + + FANS = + +

SA � + + SA = + +

GAop � � � GAop � � =

GAux � � + GAux � � =

Strongly Corr. All Instances

FANS SA GAop GAux FANS SA GAop GAux

FANS + + + FANS + + +

SA � = = SA � + +

GAop � = = GAop � � �GAux � = = GAux � � +

Table 3. Mean of Errors by Instance Type and Capacity.

Uncorr Weakly Corr Strong

0.25 0.50 0.75 0.25 0.50 0.75 0.25 0.50 0.75

FANS 1.86 0.87 0.50 2.85 1.38 1.01 0.92 1.31 0.29

SA 3.55 0.99 0.30 3.12 1.53 0.77 1.81 1.68 0.45

GAop 2.37 2.44 1.39 3.81 2.64 1.77 1.81 1.71 0.45

GAux 2.07 1.24 0.63 3.78 2.45 1.66 1.81 1.77 0.52


Apart from a few exceptions, within each type of instance it is verified that as increased, the mean error decreased.

We think that the reason is related with the fact of as increases, potentially more items

will fit in the knapsack; consequently the chance to obtain infeasible solutions decreases.

In this way, the search space is more ‘‘continuous’’, with less ‘‘holes’’ or ruggedness,

enabling the algorithms to perform the search without ‘‘jumps’’.

About the values per instance type and value, the mean errors were lower in FANS in

almost all cases. SA achieved lower values when ¼ 0.75 for instances UN and CW. In

general, both GA’s were quite far from the minimum value obtained for each and

instance type.

6. Conclusions

In this work, we have presented a novel Fuzzy Adaptive Neighborhood Search algorithm

(FANS ), and we have analyzed it from two points of view. First, as a framework of local

search techniques, we showed how FANS might be tuned to behave like other local search

methods. Second, about FANS as a general purpose optimization tool, taking into account

the experimental conditions, considering the results obtained here and those presented

elsewhere, we may conclude that FANS is a valid option to cope with combinatorial

optimization problems.

A great part of the success, mainly to achieve the first point, may be attributed to the use

of the fuzzy valuation. Being conceptually very simple, this key component enabled us to

obtain a wide range of behaviors. If we consider that each situation has to be tackled with a

particular strategy, a tunable tool like FANS has to be viewed as a valuable option with

regard to possible developments within decision support systems.

In order to make FANS available, a distribution version is being prepared. Meanwhile, a

preliminary version may be obtained contacting [email protected].

Acknowledgments

The authors want to explicitly thank the reviewers of this paper for their very useful

comments, which truly helped to improve the quality and clarity of the article. David Pelta

wishes to thank Natalio Krasnogor, for his very useful suggestions in the first stages of

FANS development.

Notes

1. Research supported in part by Projects PB98-1305 and TIC 99-0563.

2. Grant holder from Consejo Nac. de Invest. Cientıficas y Tecnicas (Argentina).

3. Corresponding author: Tel: þ34 958 243195 Fax: þ34 958 243317.

4. The best individual of each generation is passed to the next one without changes.


References

Aarts, E. and J.K. Lenstra (eds.). (1997). Local Search in Combinatorial Optimization. John Wiley & Sons.

Chu, P. and J. Beasley. (1998). ‘‘A Genetic Algorithm for the Multidimensional Knapsack Problem,’’ Journal of

Heuristics 4, 63–86.

Corne, D., M. Dorigo, and F. Glover (eds.). (1999). New Ideas in Optimization. McGraw-Hill.

Delgado, M., J. Kacprzyk, J. Verdegay, and M. Vila (eds.). (1994). Fuzzy Optimization. Recent Advances. Physica

Verlag.

Diaz, A., F. Glover, H. Ghaziri, J. Gonzalez, M. Laguna, P. Moscato, and F. Tseng. (1996). Heuristic Optimization

and Neural Nets. Ed. Paraninfo. In Spanish.

Glover, F. and M. Laguna. (1997). Tabu Search. London: Kluwer Academic Publishers.

Hansen, P. and N. Mladenovic. (1999). ‘‘An Introduction to Variable Neighborhood Search.’’ In S. Voss,

S. Martello, I. Osman, and C. Roucairol (eds.), Metaheuristics: Advances and Trends in Local Search Proce-

dures for Optimization, pp. 433–458. Kluwer.

IFSA99. (1999). ‘‘8th International Fuzzy Systems Associations World Conference.’’ Taipei, Taiwan.

Jones, T. (1995). ‘‘Evolutionary Algorithms, Fitness Landscapes and Search.’’ Ph.D. thesis, University of New

Mexico, Albuquerque.

Martello, S. and P. Toth. (1990). Knapsack Problems: Algorithms and Computer Implementations. John Wiley &

Sons.

Michalewicz, Z. (1998). Genetic Algorithms þ Data Structures ¼ Evolution Programs. Springer Verlag.

Pelta, D., A. Blanco, and J.L. Verdegay. (2000a). ‘‘A Fuzzy Adaptive Neighborhood Search for Function

Optimization.’’ In Fourth International Conference on Knowledge-Based Intelligent Engineering Systems &

Allied Technologies, KES 2000, 2, 594–597.

Pelta, D., A. Blanco, and J.L. Verdegay. (2000b). ‘‘Introducing FANS: a Fuzzy Adaptive Neighborhood Search.’’

In Eigth International Conference on Information Processing and Management of Uncertainty in Knowledge-

based Systems, IPMU 2000, 3, 1349–1355.

Pissinger, D. (1995). ‘‘Algorithms for Knapsack Problems.’’ Ph.D. thesis, Dept. of Computer Science, University

of Copenhagen, Denmark.


a fuzzy valuation-based local search framework for combinatorial problems

Documents