visory systems [46], organization and strategy planning [7,8], analysis of business performance indicators [9],

* Corresponding author.E-mail addresses: mehdi@iust.ac.ir (M. Ghazanfari), s_alizadeh@ind.iust.ac.ir (S. Alizadeh), Fathian@iust.ac.ir (M. Fathian),

jimk@pme.duth.gr (D.E. Koulouriotis).

Applied Mathematics and Computation 192 (2007) 5668

www.elsevier.com/locate/amc0096-3003/$ - see front matter 2007 Elsevier Inc. All rights reserved.1. Introduction

Cognitive Maps contain components and their corresponding relations, which may be positive, negative, orneutral. A Cognitive Map is a directed graph that its nodes correspond to relevant concepts and the edges statethe relation between these two nodes by a sign. A positive sign implies a positive relation; moreover, anyincrease in its source value leads to increase in its target value. A negative sign presents negative relationand any increase or decrease in its source value leads to reverse eect to its target value. In a CognitiveMap if there is no edge between two nodes it means that, there is no relation between them.

Cognitive Maps were initially introduced by Robert Axelrod in 1976 and applied in political science [1].Also it was used in numerous areas of application such as analysis of electrical circuits [2], medicine [3], super-Abstract

Fuzzy Cognitive Map (FCM) is a directed graph, which shows the relations between essential components in complexsystems. It is a very convenient, simple, and powerful tool, which is used in numerous areas of application. Experts who arefamiliar with the system components and their relations can generate a related FCM. There is a big gap when humanexperts cannot produce FCM or even there is no expert to produce the related FCM. Therefore, a new mechanism mustbe used to bridge this gap. In this paper, a novel learning method is proposed to construct FCM by using some metaheu-ristic methods such as genetic algorithm (GA) and simulated annealing (SA). The proposed method not only is able toconstruct FCM graph topology but also is able to extract the weight of the edges from input historical data. The eciencyof the proposed method is shown via comparison of its results of some numerical examples with those of some othermethods. 2007 Elsevier Inc. All rights reserved.

Keywords: Fuzzy Cognitive Map (FCM); Learning; Genetic algorithm; Simulated annealingComparing simulated annealing and genetic algorithmin learning FCM

M. Ghazanfari a, S. Alizadeh a,*, M. Fathian a, D.E. Koulouriotis b

a Industrial Engineering Department, Iran University of Science and Technology (IUST) Narmak, Tehran, Iranb Department of Production Engineering and Management, Democritos University of Thrace, Xanthi, Greecedoi:10.1016/j.amc.2007.02.144

software project management [10,11], Information retrievals [12], modeling of plant control [13], systemdynamics and complex systems [1421] and modeling virtual world [22], etc.

In 1988, Kosko introduced a new extension concept for Cognitive Map and named it Fuzzy CognitiveMaps (FCM) [2326]. In a FCM, the relation between two nodes is determined by taking a value in interval[1,1]. While 1 corresponds to the strongest negative, +1 corresponds to strongest positive one. The othervalues express dierent levels of inuence. This model can be presented by a square matrix called connectionmatrix. The value of relation between two nodes is set in their corresponding cell. In the connection matrix,row and column are associated with a source node and a target node, respectively. An FCM consists of nodes,Ci; i 1; . . . ;N , where N is the total number of concepts Each arc between two nodes Ci and Cj, has a weightFij, which is the strength of the causal link between Ci and Cj. The sign of Fij indicates whether the relationbetween two concepts is direct or inverse. The direction of causality indicates whether the concept Ci causes

M. Ghazanfari et al. / Applied Mathematics and Computation 192 (2007) 5668 57the concept Cj or vice versa. Thus, there are three types of weights:

Fij > 0 express positive causalityFij < 0 express negative causalityFij = 0 express no relation

A simple FCM with ve nodes and 10 weighted arcs is depicted in Fig. 1.Experts develop a FCM or a mental model manually based on their knowledge in the area under study. At

rst, they identify key domain issues or concepts. Secondly, they identify the causal relationships among theseconcepts and thirdly, they estimate causal relationships strengths. The achieved graph (FCM) shows not onlythe components and their relations but also the strengths.

A group of experts can be utilized to improve the results. All experts are asked to determine the relevantfactors in a brain storm meeting. They discuss about main characteristics of the system, for example, numberand kinds of concepts and relation between nodes, which are in the FCM. Then, they determine the structureand the interconnections of the network using fuzzy conditional statements or fuzzy rules. Each expert maydraw his own individual FCM, which can be dierent from the others. In order to deal with these diagrams,the assigned weights by each expert can be considered and a new FCM will be constructed by all experts. Thus,this constructed FCM will represent the knowledge and experience of all related experts. [27,28].

FCMs can be produced by expert manually or generated by other source of information computationally.They named manual FCMs and automated FCMs.

In Fuzzy Cognitive Maps like Cognitive Map, the inuence of a concept on the others is considered asnegative, positive or neutral, but all relations are expressed in fuzzy terms, e.g. very weak, weak, med-ium, strong and very strong. The following set of linguistic variables is also considered:

{negatively very strong, negatively strong, negatively medium, negatively weak, zero, positively weak, medium,positively strong and positively very strong}.

The corresponding membership functions for these terms are shown in Fig. 2.In a FCM, all fuzzy variables are mapped into interval [1,1]. A simple way is to map fuzzy expression to

numerical value in a range of [1,1]. For example, positively weak is mapped to 0.25, negatively medium to 0.5, positively strong to 0.75 [29].

C1

C4

C5

C3

C2+F21

+F12

-F23

-F43

-F24

+F54

-F53

+F35-F15

+F41Fig. 1. A simple Fuzzy Cognitive Map (FCM).

58 M. Ghazanfari et al. / Applied Mathematics and Computation 192 (2007) 5668Then, all the suggested linguistic variables, are considered and an overall linguistic weight is obtained, withthe defuzzication method of Centre of Gravity (COG) [30], is transformed to a numerical weight belonging tothe interval [1,1].

In general, the manual procedures for developing FCM have occurred, when at least there is one expert whohas expertise in the area under study. In some situations, a FCM could not be constructed manually such as:

There is no expert to dene a FCM. The experts knowledge is dierent with each other and they draw dierent FCM. There are large amount of concepts and connections between them, which could not be drawn withoutmistakes.

The above situation shows that in many cases, to develop a FCM manually becomes very dicult andexperts intervention could not resolve the problem. Therefore, a systematic way should be found in orderto bridge this gap. For example designing a new method could eliminate the existing weakness. The relatedknowledge can be extracted by analyzing past information about the given systems. This paper is organizedas follows: Section 2 states some basic concepts and denitions of Fuzzy Cognitive Map (FCM) and history ofFCM Automatic construction. The proposed learning model is presented in Section 3. While, Section 4 pre-sents the experimental evaluation and discussion of the achieved results and model eectiveness. Finally, Sec-tion 5 covers conclusions and future research directions.

2. Automated FCM construction (related works)

When the experts are not able to express their expertise or even there is no expert in the area under studiedto add some expression based on her/his expertise, therefore a new way should be dened. For these reasons,the development of computational methods for learning FCM is necessary [31]. In this method, not onlycasual relations between nodes, but also the strength on each edge must be achieved based on historical data.The required knowledge is extracted from historical data by means and new computational procedures. Manymethods for learning FCM model structure have been recently proposed. In general, these methods are cat-egorized in two min groups:

Hebbian algorithm

nvs ns nm nw z pw pm ps pvs

-1 -0.75 -0.5 -0.25 0 0.25 0.5 0.75 1

Fig. 2. Membership functions. Genetic algorithm

Soft computing approach such as neural networks and genetic algorithm can be used to discover appropri-ate knowledge from historical data in the form of a graph or a FCM. Many researches worked on these areasby investigating FCM learning methods using historical data.

Kosko proposed a new model by use of simple Dierential Hebbian Learning law (DHL) in 1994, but heused this model to learning FCMs without any applications [32]. This learning process modied weights ofedges existing in a FCM in order to nd the desired connection matrix. In general, when the correspondingconcept changes, the value of the related edges for that nodes will be modied too.

In 2002, Vazquez introduced a new extension to DHL algorithm presented by Kosko. He used a new ideato update edge values in a new formula [33]. Another method of learning FCMs based on the rst approach

M. Ghazanfari et al. / Applied Mathematics and Computation 192 (2007) 5668 59(Hebbian algorithm), was introduced in 2003 by Papageorgiou et al. He developed another extension to Heb-bian algorithm, called Nonlinear Hebbian Learning (NHL) [34]. Active Hebbian Algorithm (AHL) intro-duced by Papageorgiu et al. in 2004. In the recent method, experts not only determined the desired set ofconcepts, initial structure and the interconnections of the FCM structure, but also identied the sequenceof activation concepts [35].

Another category in learning connection matrix of FCM is application of genetic algorithms or evolution-ary algorithms. Koulouriotis et al. applied the Genetic Strategy (GS) to learn FCMs structure in 2001 [36]. Inmentioned model, they focused on the development of an ES-based procedure that determines the values ofthe cause3eect relationships (causality). Parsopoulos et al. also published other related papers in 2003. They

Table 1Overview of some learning approaches applied to FCMs

Algorithm Learning goal Humanintervention

Type of dataused

Transformationfunction

No. of node Learningalgorithm

DHL Connectionmatrix

No Single N/A N/A Hebbian

BDA Connectionmatrix

No Single Binary 579 ModiedHebbian

NHL Connectionmatrix

Yes and No Single Continuous 5 ModiedHebbian

AHL Connectionmatrix

Yes and No Single Continuous 8 ModiedHebbian

GS Connectionmatrix

No Multiple Continuous 7 Genetic

PSO Connectionmatrix

No Multiple Continuous 5 Swarm

GA Initial vector N/A N/A Continuous 11 GeneticRCGA Connection

matrixNo Single Continuous 4; 6; 8; 10 Genetic

SA (Thispaper)

Connectionmatrix

No Single Continuous AnyNumber

SA

GA/SA(This paper)

Connectionmatrix

No Single Continuous AnyNumber

GA/SAtried to apply Particle Swarm Optimization (PSO) method, which belongs to the class of Swarm Intelligencealgorithms, to learn FCM structure [37,38]. Khan and Chong worked on learning initial state vector of FCMin 2003. They performed a goal-oriented analysis of FCM and their learning method did not aim to computethe connection matrix, and their model focused on nding initial state vector for FCM [39]. In 2005, Stachet al. applied real-coded genetic algorithm (RCGA) to develop FCM model from a set of historical data in2005 [28].

Other work to train a FCM was done by Konar in 2005. He worked on reasoning and unsupervised learn-ing in a FCM. In this paper, a new model was introduced for unsupervised learning and reasoning on a specialtype of Cognitive Maps realized with Petri nets [40]. In 2006, Parsopoulos et al. combined these two categoriesand published a paper about using evolutionary algorithms to train Fuzzy Cognitive Maps. In their model,they investigated a coupling of dierential evolution algorithm and unsupervised Hebbian learning algorithm[29]. Our model based on Simulated Annealing and genetic algorithm is a new method to learn connectionmatrix rapidly. Table 1 shows a comparison between some existing methods:

The table compares the methods based on several factors, such as learning goal, kind of input historicaldata, type of transformation function, size of FCM model, type of learning strategy and that whether expertsare involved in model or not.

3. The proposed learning method by GA and SA

This new method proposed a solution for automatic construction of Fuzzy Cognitive Map by using Sim-ulated Annealing and Genetic Algorithm. The focus of this model is to determine causeeect relationships(causality) and their strength.

Eqs.

nectio

i

estimated real

wher

i i

AsStachpaperon simulated annealing as a heuristic model in learning FCM. Secondly, these two algorithms are hybrid to

60 M. Ghazanfari et al. / Applied Mathematics and Computation 192 (2007) 5668improve the result and a new method is proposed. Fig. 3 shows the outline of the proposed method:The proposed learning model uses Genetic algorithm to nd the near-optimum solution and simulated

annealing is used to escape the local minimum solution and to improve the optimum solution found byGA. The following sections provide details about simulated annealing and assume that readers are familiarwith GA. A useful summary about relevant GA and SA can be found in [4143]. Also, it is tried to demon-

stratementioned in Section 3, there are many methods for automatic constructing FCM matrix, for example,et al. constructed this matrix by a Real Code Genetic Algorithm (RCGA) with simple operators. In this, at rst, we deal with genetic algorithm to adjust it by dening proper operators and then we concentrateIf Inputitn dened as an initial vector, and Outputitn1 as system response, K 1 pairs in the form of {initialvector, system response} can be generated from the input data.e N is the number of nodes and K is the iteration.

Input tn ! Output tn1 8t 0; . . . ;K 1: 7Total Error n1 i1

Outputi tn1 Outputi tn1; 6

XK XNi

According to Eq. (4), Outputestimatedi tn1 is the estimated output and Inputitn is its corresponding input forthe ith node. Eproposed is the new proposed matrix. The real output is Outputreali tn1 and the dierence be-tween real and estimated outputs is calculated by:

Error Outputestimatedi tn1 Outputreali tn1: 5By using the later two equations, the objective is dened as minimizing the dierence between real and esti-mated outputs. This objective is dened for all nodes as:Outputestimatedtn1 Eproposed Input tn: 4proposed FCM learning methods forms structure of a FCM and is able to generate state vector sequences thattransform the input vectors into the output vectors. When all real input and output values of a FCM are inhand, the most important step is to nd a new solution for the FCM and calculate the estimated output relatedto this new FCMn Matrix) is a vital factor in Eq. (3) which should be determined in the FCM learning process. TheInputitn is input data for node i, Outputitn1 is its corresponding output data and E is the Connection ma-trix of FCM. Eq. (3) implies that for each node i its corresponding output can be calculated. E (Related con-(1) and (2) can be expressed by Eq. (3):

Outputitn1 E Inputitn: 3f x 1 ecx : 213.1. Problem denition

As mentioned before, a causeeect relation is specied by a related Connection matrix. The elements ofthis matrix are the values of edges in the FCM. The aim of the proposed method is to nd these elements.The relations between nodes and edges are calculated as:

Cit 1 fXni1

ejiCjt !

; 1

where eijs are the elements of the matrix and f is a transform function which includes recurring relation ont > 0 between Ct 1 and Ct that can be presented by a logistic function like:all essential elements of propose method, including structure of solution coding (chromosomes), gener-

00.20.40.60.8

11.2

1 5 7 9 10 11 12 13 14 15 16 17 18 19 20

node1 node2 node3 node4 node5 node6 node7

Simulated Annealing

Genetic Algorithm

C1

C4

C5

C3

C2+e21

+e12

-e23

-e43

-e24

+e54

-e53

+e35-e15

+e41

2 3 4 6 8

M. Ghazanfari et al. / Applied Mathematics and Computation 192 (2007) 5668 61ation of initial solution, initial temperature, tness function, stopping condition, genetic operators in GA,Neighboring solutions in SA, and selection strategy.

3.2. A proposed SA_GA method for learning FCM

In this section, SA and GA algorithm for learning FCM is introduced. At rst, we focused on GA param-eters and new proper operators.

3.2.1. Proposed genetic algorithms with new operators for learning FCM

As mentioned in Chapter 2, the others worked on learning FCMs with GA, but in this paper, new operatorsand new chromosome structure are proposed which dened here. At rst, six principle factors in GA are con-sidered as follows:

Solution coding (chromosome structure). The chromosome structure or a solution for FCM is formed as amatrix showed in Fig. 4 [44]. In GA algorithms the initial solution generates random.

Fitness value. The tness value is a criterion for the quality measurement of a chromosome or feasible solution.An o spring or new solution is accepted when its objective function value is minimum as compared with itsparents.

Mating pool selection strategy. For creating the new generation, it is necessary to select some chromosomes(mating pool) with the latest tness in the current generation for recombining or creating chromosomes relatedto the new generation. In this paper, at rst sort the population from the best to the worst, assign the selectionprobability of each chromosome, and select them according to their ranking numbers.

Improved GA operators. In this paper, the chromosome structure is formed as a matrix. Thus, the GA linear

Fig. 3. The diagram of new model.operators cannot be used to a matrix type as the traditional forms. These operators should be improved pro-portional to the matrix type. Therefore, considering the nature of the matrix, each of three operators calledcrossover, mutation, and inversion are considered as follows:

e11 e12 e13 e14 e15

e21 e22 e23 e24 e25

e31 e32 e33 e34 e35

e41 e42 e43 e44 e45

e51 e52 e53 e54 e55

Fig. 4. The chromosome structure.

Horizontal operator, vertical operator and diametric operator. In horizontal operator, at rst two positions inrow selected randomly and operation is exercised over this selected area. Fig. 5 shows this operator. Verticaloperator is similar to the horizontal operator but two positions selected in column randomly. Fig. 6 shows thisoperator. In diametric operator, two numbers in the relevant matrix column or row limits and one of the direc-tions of primary or secondary are selected randomly. Then, the operation is exercised over obtained diameters.For example in Fig. 7, a random diameter is selected and then the inversion operator is used.

e11 e12 e13 e14 e15 e11 e12 e13 e14 e15

e21 e22 e23 e24 e25 e25 e24 e23 e22 e21

e31 e32 e33 e34 e35 e35 e34 e33 e32 e31

e41 e42 e43 e44 e45 e41 e42 e43 e44 e45

e51 e52 e53 e54 e55 e51 e52 e53 e54 e55

Fig. 5. Horizontal operator.

e11 e12 e13 e14 e15 e11 e12 e13 e54 e55

e21 e22 e23 e24 e25 e21 e22 e23 e44 e45

e31 e32 e33 e34 e35 e31 e32 e33 e34 e35

e41 e42 e43 e44 e45 e41 e42 e43 e24 e25

e51 e52 e53 e54 e55 e51 e52 e53 e14 e15

Fig. 6. Vertical operator.

62 M. Ghazanfari et al. / Applied Mathematics and Computation 192 (2007) 5668e11 e12 e13 e14 E15 e11 e45 e13 e14 e15

e21 e22 e23 e24 E25 e21 e22 e34 e24 e25

e31 e32 e33 e34 e35 e31 e32 e33 e23 e35 e41 e42 e43 e44 E45 e41 e42 e43 e44 e12

e51 e52 e53 e54 E55 e51 e52 e53 e54 e55

Fig. 7. Diametric operator.

e11 e12 e13 e14 e15 e11 e12 e13 e14 e15

e21 e22 e23 e24 e25 e21 E22 E23 e24 e25

e31 e32 e33 e34 e35 e31 E32 E33 e34 e35

e41 e42 e43 e44 e45 e41 e42 e43 e44 e45

e51 e52 e53 e54 e55 e51 e52 e53 e54 e55

E11 E12 E13 E14 E15 E11 E12 E13 E14 E15

E21 E22 E23 E24 E25 E21 e22 e23 E24 E25

E31 E32 E33 E34 E35 E31 e32 e33 E34 E35

E41 E42 E43 E44 E45 E41 E42 E43 E44 E45

E51 E52 E53 E54 E55 E51 E52 E53 E54 E55

Fig. 8. Operation on part of matrix.

an

Algor Numbcic n

3.2

creanat

M. Ghazanfari et al. / Applied Mathematics and Computation 192 (2007) 5668 63n n 1Consider xneighbour at random in the neighbourhoodIf F xneighbour F x then counter = counter+1 and change = falseIf F xneighbour < F x then x xneighbour

If F xneighbour < F best then F best F xneighbour and xbest xneighbourIf F xneighbour > F xbest

Consider R a random number in [0, 1],D F xneighbour F besta 1

1eDTIf R < a Then x xneighbourEnd loopT n1 1

1eDT T n, N = 0, Change = false

End loopdecrease as the algorithm progresses. Important parts of the simulated annealing algorithm for learning FCMare explained as follows:

SA algorithm.

*Initialization: Select an initial solution x1 in XInitialize the best value Fbest of F and the corresponding solution xbest:F best F x1xbest x1Initialize N ; T , a

Do while not stopping condition is fullled T n1 > aDo (n < N and (Not Change)to a user-dene annealing schedule. It terminates, when either the optimal solution is attained or the problembecomes frozen at a local optimum that cannot be improved. To avoid freezing at a local optimum, the algo-rithm moves slowly (with respect to the objective value) through the solution space. This controlled improve-ment of the objective value is accomplished by accepting non-improving moves with a certain probability thatBest solution not improved. If the best solution F x was not improved after a period, the algorithm willstop. (Users determine this period.)Best Solutions Found. In this case, the algorithm terminates, if the best solution nd and the error functiondoes not change after a period.

.2. Proposed simulated annealing for Learning FCM

In this section, SA algorithm for learning FCM is introduced. Simulated annealing is an algorithm for dis-te optimization backs to the early 1980s. It was originally developed as a simulation model for a physicalnealing process and hence it is referred to as simulated annealing. In simulated annealing, a problem startsan initial solution, and a series of moves (i.e., changing the values of decision variables) are made accordingFor dcriteria. The stopping criteria condition can be dened by two dierent ways:

ithm time. In this case, the algorithm terminates, if time exceeds the specic time.er of generations. In this case, the algorithm terminates, if the number of generations exceeds the spe-umber.

bestStoppingrelevant matrix row limits are selected randomly. Then, the operation is exercised over obtained district.r example in Fig. 8, the block obtained from cross related to two elected chromosomes is selected randomlyd then the crossover operator is used.Operation on part of matrix. In this case, two numbers in the relevant matrix column limits and two numbers intheFoesigning the SA algorithm, many principle factors considered and introduced here:

1Neigh

curreopera

64 M. Ghazanfari et al. / Applied Mathematics and Computation 192 (2007) 5668Stopping criteria. Here are two stopping rules:

The number of temperature transitions is used as a stopping criterion. Furthermore, the SA algorithm canbe terminated, when the term T n1 > e or stopping condition is satised. e can be a constant or calculatedby other parameters.

If the neighbor solution F xneighbour was not improved after a period, it will stop. (Users determine thisperiod.)

4. Computational resultsThgoalinputsimilastateconceT n1 a T n ) T n1 1 eDT T n; 10

boring solutions. Neighboring solutions are the set of feasible solutions that can be generated from thent solution. Each feasible solution can be directly reached from current solution by a move (as genetictions mutation or inversion) and resulted neighboring solution.Solution coding. The solution coding for SA is equal to GA in Section 3.2.1.

Initial solution. An initial solution is a starting solution (point) that will be used in the search process and con-sidered as a random solution. In this research, When SA algorithm is used to optimize FCM structure alone,its initial solution generates random and in combined GA and SA model, the best solution dened by GAalgorithm is used as initial solution in SA model.

Initial temperature and cooling schedule. An initial temperature T0 and a cooling schedule a are used to controlthe series of moves in the search process. In general, the initial temperature should be high enough to allow allcandidate solutions to be accepted. Cooling schedule a is the rate at which the temperature is reduced. In thispaper, a classical schedule is represented in Fig. 9. Starting from T0, the temperature is decreased throughmultiplication by a xed factor a0 < a < 1.

a 11 eDT ; 8

D F xneighbour F best 9

1T

2T

3T

0 1 2 3 ..

Fig. 9. Temperature schedule.0Te aim of the experiments is to assess quality of the proposed method for learning FCMs. In general, theof learning FCM is to nd FCM connection matrix that generates the same state vector sequence as thedata for a given initial state vector. Now, an important parameter considered. This variable measuresrity between the input data, and data generated by simulating the candidate FCM with the same initialvector as the input data. The criterion is dened as a normalized average error between correspondingpt values at each iteration between the two state vector sequences:

error 1K 1 N XK

n1

XNi1Outputestimatedi tn1 Outputreali tn12: 11

As mentioned before, in above formula Outputreali tn1 is the value of a node at iteration t in the input,Outputestimatedi tn1 is the estimated value of that node at iteration t from simulation of the candidateFCM, K is the input data length and N is the number of nodes. All experiments were performed with logistictransformation function. This function is parameterized as:

F x 11 e0:2x : 12

In our experiment, for comparing the metaheuristic results, some test problems were solved by using GA, SAon a PC Pentium IV, 1.6 GHz. The metaheuristic algorithms were developed using Visual Basic 6. Two algo-rithms, the genetic algorithm, Simulated Annealing ran with dierent Node numbers: 2, 3, 4, 5, 6, 7, 8, 9, 10,11, 12, 13, 14, 15 and for every run the Error and time consuming saved. Each considered FCM, in terms ofthe number of nodes, was simulated 5 times with the two algorithms, which totaled in 140 experiments. In GA,one of the stopping criteria is Algorithm time, in this program, the algorithm terminates, if the time exceeds 40(s), while, SA nd the best solution in less than 3 (s). The obtained results are shown in Table 2.

The results of these experiments show that these algorithms gradually converge into a high-quality candi-date FCM. Two examples of FCM learning experiments based on GA and SA are plotted in Fig. 10.1 and10.2.

Fig. 11 shows the error of GA and SA algorithms. This gure shows that SA algorithm produce better solu-tion with less error for FCM with node number more than 10. That means SA is appropriate for learningFCM with more nodes.

Table 2Experiment results with dierent tness functions and times

NO node Genetic algorithm Simulated annealing

Fitness function L2 (error) Time (s) Fitness function L2 (error) Time (s)

2 1.7998E06 40 3.5000E05 2.963 1.8308E06 40 3.8308E05 2.684 3.0140E06 40 4.8140E05 2.395 5.6714E06 40 5.2714E05 2.186 1.5381E05 40 5.4381E05 1.857 2.4262E05 40 6.4262E05 1.64

0.000.0

0.00

M. Ghazanfari et al. / Applied Mathematics and Computation 192 (2007) 5668 6500.00020.00040.0006

00.000020.000040.00006080112

0.000080.0001

0.000128 3.1149E05 40 7.1432E05 1.519 6.4874E05 40 7.4320E05 1.2510 1.0917E04 40 7.6345E05 1.0911 1.5211E04 40 8.2345E05 0.9612 1.9053E04 40 8.7432E05 0.7513 2.6694E04 40 1.0060E04 0.614 3.2980E04 40 1.0604E04 0.4815 4.0198E04 40 1.2029E04 0.43

0.00140.00160.00180.002

0.000140.000160.00018

0.0002Fig. 10. An example of (10.1) SA and (10.2) GA tness function which show that error converges to near zero.

66 M. Ghazanfari et al. / Applied Mathematics and Computation 192 (2007) 566851015202530354045

0.0000E+005.0000E-05

2 4 6 7 10 11 12 13 14 15GA SA

3 5 8 9

Fig. 11. GA and SA errors for dierent nodes.1.0000E-041.5000E-042.0000E-042.5000E-043.0000E-043.5000E-044.0000E-044.5000E-04Fig. 12 compares the time consuming with GA and SA algorithms. This gure shows that simulated anneal-ing algorithm, as a learning algorithm for FCM is faster than genetic algorithms in the same problem. There-fore, The SA algorithm found related solutions in less computational times than GA.

In these two algorithms, the error of GA in FCM with little Node is less that Simulated annealing. How-ever, in FCM with more nodes the error of SA as a new learning algorithm is less that GA. Considering theresults of Table 1 shows that the presented metaheuristic algorithms are able to nd and report the near-opti-mal and promising solutions in a reasonable computational time. This indicates the success of the proposedmethod in learning FCM. In general, we can conclude the following results:

1. The SA algorithm meanly found better solutions than GA in less computational times.2. To combine GA and SA algorithm produce the best method not only in computational time but also in

FCM with large nodes. In general, In FCM with little node number it is better to use GA and in FCM withmore nodes, it is better to use SA. Fig. 13 shows this issue.

5. Conclusion

In this paper, we have developed a new method for learning FCMs by SA algorithm. It has been shown thatSA not only can improve the speed of learning process, but also can improve the quality of learning FCMs

02 9 10 11 12 13 14 15

GA SA

3 4 5 6 7 8

Fig. 12. Time consuming with GA and SA algorithms.

0.0000E+005.0000E-051.0000E-041.5000E-042.0000E-042.5000E-043.0000E-043.5000E-044.0000E-044.5000E-04

1 5 6 9 10 11 12 13 14

GA SA GA (in more nodes) , SA (in less nodes)

2 3 4 7 8

Fig. 13. The new method.

M. Ghazanfari et al. / Applied Mathematics and Computation 192 (2007) 5668 67with more nodes. The quality of learning method based on GA deteriorates with the increasing size of themaps but SA over comes this diculty and when the size of maps increase the GA algorithms replaced withSA algorithms. According to these properties, a new method proposed. In this new method, GA algorithm isused in learning small size FCMs and SA in learning large size FCMs. The results show this new method isvery eective, and generates FCM models that can almost perfectly represent the input data. In general, theproposed method achieves excellent quality for maps in every size of FCM. The produced results could pro-vide some guidelines for other learning methods. The future work will concern on the improvement of the pro-posed learning method. One of interesting and open issues is using the other heuristic methods for learningFCMs and comparing them with the others.

References

[1] R. Axelrod, Structure of Decision: The Cognitive Maps of Political Elites, Princeton University Press, Princeton, NJ, 1976.[2] M.A. Styblinski, B.D. Meyer, Signal ow graphs versus fuzzy cognitive maps in application to qualitative circuit analysis, Int. J. Man

Mach. Studies 35 (1991) 175186.[3] V.C. Georgopoulos, G.A. Malandraki, C.D. Stylios, A fuzzy cognitive map approach to dierential diagnosis of specic language

impairment, J. Artif. Intell. Med. 29 (3) (2003) 261278.[4] C.D. Stylios, P.P. Groumpos, The challenge of modeling supervisory systems using fuzzy cognitive maps, J. Intel. Manuf. 9 (4) (1998)

339345.[5] C.D. Stylios, P.P. Groumpos, Fuzzy cognitive maps: a model for intelligent supervisory control systems, Comput. Ind. 39 (3) (1999)

229238.[6] C.D. Stylios, P.P. Groumpos, Fuzzy cognitive map in modeling supervisory control systems, J. Intell. Fuzzy Syst. 8 (2) (2000) 8398.[7] M.G. Bougon, Congregate cognitive maps: a unied dynamic theory of organization and strategy, J. Manage. Stud. 29 (1992) 369

389.[8] K.C. Lee, W.J. Lee, O.B. Kwon, J.H. Han, P.I. Yu, Strategic planning simulation based on fuzzy cognitive map knowledge and

dierential game, Simulation 71 (5) (1998) 316327.[9] D. Kardaras, G. Mentzas, Using fuzzy cognitive maps to model and analyze business performance assessment, in: J. Chen, A. Mital

(Eds.), Advances in Industrial Engineering Applications and Practice II, 1997, pp. 6368.[10] W. Stach, L. Kurgan, Modeling software development project using fuzzy cognitive maps, in: Proc. 4th ASERC Workshop on

Quantitative and Soft Software Engineering (QSSE04), 2004, pp. 5560.[11] W. Stach, L. Kurgan,W. Pedrycz, M. Reformat, Parallel fuzzy cognitive maps as a tool for modeling software development project,

in: Proc. 2004 North American Fuzzy Information Processing Society Conf. (NAFIPS04), Ban, AB, 2004, pp. 2833.[12] A.R.Montazemi, D.W. Conrath, The use of cognitive mapping for information requirements analysis, MIS Quarterly 10 (1986) 4455.[13] K. Gotoh, J. Murakami, T. Yamaguchi, Y. Yamanaka, Application of fuzzy cognitive maps to supporting for plant control, in: Proc.

SICE Joint Symp. 15th Systems Symp. and Tenth Knowledge Engineering Symp., 1989, pp. 99104.[14] J.P. Carvalho, J.A. Tome, Rule based fuzzy cognitive maps and fuzzy cognitive maps a comparative study, in: Proceedings of the

18th International Conference of the North American Fuzzy Information Processing Society, NAFIPS99, New York.[15] J.P. Carvalho, J.A. Tome, Rule based fuzzy cognitive maps fuzzy causal relations, in: M. Mohammadian (Ed.), Computational

Intelligence for Modelling, Control and Automation, 1999.[16] J.P. Carvalho, J.A. Tome, Fuzzy mechanisms for causal relations, in: Proceedings of the Eighth International Fuzzy Systems

Association World Congress, IFSA99, Taiwan.[17] J.P. Carvalho, J.A. Tome, Rule based fuzzy cognitive maps qualitative systems dynamics, in: Proceedings of the 19th International

Conference of the North American Fuzzy Information Processing Society, NAFIPS2000, Atlanta.[18] S. Alizadeh, M. Ghazanfari, M. Jafari, An approach for solving fuzzy system dynamics problems, in: 21st International System

Dynamics Conference, USA, July 2003.[19] D.E. Koulouriotis, I.E. Diakoulakis, D.M. Emiris, E.N. Antonidakis, I.A. Kaliakatsos, Eciently modeling and controlling complex

dynamic systems using evolutionary fuzzy cognitive maps (Invited Paper), Int. J. Comput. Cognition 1 (2) (2003) 4165.[20] D.E. Koulouriotis, I.E. Diakoulakis, D.M. Emiris, C.D. Zopounidis, Development of dynamic cognitive networks as complex

systems approximators: validation in nancial time series, Appl. Soft Comput. 5 (2005) 157179.[21] C.D. Stylios, P.P. Groumpos, Modeling complex systems using fuzzy cognitive maps, IEEE Trans. Syst. Man, Cybernet Part A: Syst.

Humans 34 (1) (2004).[22] D.E. Koulouriotis, I.E. Diakoulakis, D.M. Emiris, Anamorphosis of fuzzy cognitive maps for operation in ambiguous and multi-

stimulus real world environments, in: 10th IEEE Int. Conf. on Fuzzy Systems, 2001, pp. 11561159.[23] B. Kosko, Hidden patterns in combined and adaptive knowledge networks, Int. J.Approx. Reason. 2 (1988) 377393.[24] B. Kosko, Fuzzy cognitive maps, Int. J. ManMach. Studies 24 (1986) 6575.[25] B. Kosko, Neural Networks and Fuzzy Systems, Prentice-Hall, Englewood Clis, NJ, 1992.[26] B. Kosko, Fuzzy Engineering, Prentice-Hall, Englewood Clis, NJ, 1997.[27] M. Khan, M. Quaddus, Group decision support using fuzzy cognitive maps for causal reasoning, Group Decision Negotiation J. 13

(5) (2004) 463480.

[28] Wojciech Stach, Lukasz Kurgan, Witold Pedrycz, Marek reformat genetic learning of fuzzy cognitive maps, Fuzzy Sets Syst. 153(2005) 371401.

[29] E. Papageorgiou, C. Stylios, P. Groumpos, Unsupervised learning techniques for ne-tuning fuzzy cognitive map causal links, Int. J.HumanComputer Studies 64 (2006) 727743.

[30] C. Lee, Fuzzy logic in control systems: fuzzy logic controller, Part 1, 2, IEEE Trans. Syst. Man Cybernet 20 (2) (1990) 404435.[31] M. Schneider, E. Shnaider, A. Kandel, G. Chew, Automatic construction of FCMs, Fuzzy Sets Syst. 93 (2) (1998) 161172.[32] J.A. Dickerson, B. Kosko, Fuzzy virtual worlds, Artif. Intell. Expert 7 (1994) 2531.[33] A. Vazquez, A balanced dierential learning algorithm in fuzzy cognitive maps, Technical Report, Departament de Llenguatges I

Sistemes Informatics, Universitat Politecnica de Catalunya (UPC), 2002.[34] E. Papageorgiou, C.D. Stylios, P.P. Groumpos, Fuzzy cognitive map learning based on nonlinear Hebbian rule, in: Australian Conf.

on Articial Intelligence, 2003, pp. 256268.[35] E. Papageorgiou, C.D. Stylios, P.P. Groumpos, Active Hebbian learning algorithm to train fuzzy cognitive maps, Int. J. Approx.

Reason. 37 (3) (2004) 219249.[36] D.E. Koulouriotis, I.E. Diakoulakis, D.M. Emiris, Learning fuzzy cognitive maps using evolution strategies: a novel schema for

modeling and simulating high-level behavior, in: IEEE Congr. On Evolutionary Computation (CEC2001), 2001, pp. 364371.[37] E. Papageorgiou, K.E. Parsopoulos, C.D. Stylios, P.P. Groumpos, M.N. Vrahatis, Fuzzy cognitive maps learning using particle

swarm optimization, J. Intell. Inform. Syst. (2005).[38] K.E. Parsopoulos, E.I. Papageorgiou, P.P. Groumpos, M.N. Vrahatis, A rst study of fuzzy cognitive maps learning using particle

swarm optimization, in: Proc. IEEE 2003 Congr. on Evolutionary Computation, 2003, pp. 14401447.[39] M. Khan, A. Chong, Fuzzy cognitive map analysis with genetic algorithm, in: Proc. 1st Indian Internat. Conf. on Articial

Intelligence (IICAI-03), 2003.[40] Amit Konar, Uday K. Chakraborty, Reasoning and unsupervised learning in a fuzzy cognitive map, Inform. Sci. 170 (2005) 419441.[41] Duc Truong Pham, Intelligent Optimisation Techniques: Genetic Algorithms, Tabu Search, Simulated Annealing and Neural

Networks.[42] Marc Pirlot, General local search methods, Eur. J. Operat. Res. 92 (1996) 493511.[43] van Laarhoven, E. Aarts, Simulated Annealing: Theory and Applications, Reidel, Dordrect, 1987.[44] R. Tavakkoli-Moghaddam, M.B. Aryanezhad, N. Safaei, A. Azaron, Solving a dynamic cell formation problem using metaheuristics,

Appl. Math. Comput. 170 (2005) 761780.

68 M. Ghazanfari et al. / Applied Mathematics and Computation 192 (2007) 5668

Comparing simulated annealing and genetic algorithm in learning FCMIntroductionAutomated FCM construction (related works)The proposed learning method by GA and SAProblem definitionA proposed SA_GA method for learning FCMProposed genetic algorithms with new operators for learning FCMSolution coding (chromosome structure)Fitness valueMating pool selection strategyImproved GA operatorsHorizontal operator, vertical operator and diametric operatorOperation on part of matrixStopping criteria

Proposed simulated annealing for Learning FCMSA algorithmSolution codingInitial solutionInitial temperature and cooling scheduleNeighboring solutionsStopping criteria

Computational resultsConclusionReferences