metaheuristic applications in structures and infrastructures || metaheuristic algorithms in modeling...

1 Metaheuristic Algorithms inModeling and Optimization

Amir Hossein Gandomi1, Xin-She Yang 2,

Siamak Talatahari 3 and Amir Hossein Alavi 4

1Department of Civil Engineering, The University of Akron, Akron, OH, USA,2School of Science and Technology, Middlesex University, London, UK,3Marand Faculty of Engineering, University of Tabriz, Tabriz, Iran,4School of Civil Engineering, Iran University of Science and Technology,Tehran, Iran

1.1 Introduction

In metaheuristic algorithms, meta- means “beyond” or “higher level.” They gener-

ally perform better than simple heuristics. All metaheuristic algorithms use some

trade-off of local search and global exploration. The variety of solutions is often

realized via randomization. Despite the popularity of metaheuristics, there is no

agreed upon definition of heuristics and metaheuristics in the literature. Some

researchers use “heuristics” and “metaheuristics” interchangeably. However, the

recent trend tends to name all stochastic algorithms with randomization and global

exploration as metaheuristic. Randomization provides a good way to move away

from local search to the search on the global scale. Therefore, almost all metaheuris-

tic algorithms are usually suitable for nonlinear modeling and global optimization.

Metaheuristics can be an efficient way to use trial and error to produce

acceptable solutions to a complex problem in a reasonably practical time. The com-

plexity of the problem of interest makes it impossible to search every possible solu-

tion or combination; the aim is to find good, feasible solutions in an

acceptable timescale. There is no guarantee that the best solutions can be found;

we do not even know whether an algorithm will work, and if it does work, why

(Yang, 2008, 2010a). The idea is to have an efficient and practical algorithm that

will work most of the time and is able to produce good-quality solutions. Among

the quality solutions found, it can be expected that some of them are nearly opti-

mal, though there is no guarantee for such optimality.

The main components of any metaheuristic algorithm are: intensification and

diversification, or exploitation and exploration (Blum and Roli, 2003).

Diversification means generating diverse solutions so as to explore the search space

on the global scale, while intensification means focusing on the search in a local

Metaheuristic Applications in Structures and Infrastructures. DOI: http://dx.doi.org/10.1016/B978-0-12-398364-0.00001-2

© 2013 Elsevier Inc. All rights reserved.

http://dx.doi.org/10.1016/B978-0-12-398364-0.00001-2

region by exploiting the information that a current good solution is found in this

region. This is in combination with the selection of the best solutions (Yang, 2011a).

The selection of the best ensures that the solutions will converge to the optimality.

On the other hand, the diversification via randomization increases the diversity of

the solutions while keeping the solutions from being trapped at local optima. The

good combination of these two major components will usually ensure that the global

solution is achievable.

Metaheuristic algorithms can be classified in many ways. One way is to classify

them as either population-based or trajectory-based (Yang, 2010a). For example,

genetic algorithms (GAs) and genetic programming (GP) are population-based as

they use a set of strings; the particle swarm optimization (PSO) algorithm, which

uses multiple agents or particles, is also population-based (Kennedy and Eberhart,

1995). On the other hand, simulated annealing (SA) (Kirkpatrick et al., 1983) uses

a single solution that moves through the design space or search space, while artifi-

cial neural networks (ANNs) use a different approach.

Modeling and optimization may have a different emphasis, but for solving real-

world problems, we often have to use both modeling and optimization because

modeling makes sure that the objective functions are evaluated using the correct

mathematical/numerical model for the problem of interest, while optimization can

achieve the optimal settings of the design parameters. For optimization, the essen-

tial parts are the optimization algorithms. For this reason, we will focus on the

algorithms, especially metaheuristic algorithms.

1.2 Metaheuristic Algorithms

1.2.1 Characteristics of Metaheuristics

Throughout history, especially during the early periods of human history, the main

approach to problem-solving has always been heuristic or metaheuristic—by trial

and error. Many important discoveries were done by “thinking outside the box,”

and often by accident; that is heuristics. Archimedes’ Eureka moment was a

heuristic triumph. In fact, our daily learning experiences (at least as children) are

dominantly heuristic (Yang, 2010a). There are many reasons for the popularity and

success of metaheuristics, and one of the main reasons is that these algorithms

have been developed by mimicking the most successful processes in nature, includ-

ing biological systems, and physical and chemical processes. For most algorithms,

we know their fundamental components, but how exactly these components interact

to achieve efficiency still remains largely a mystery, which inspires more active

studies. Convergence analysis of a few algorithms shows some insight, but in gen-

eral the mathematical analysis of metaheuristic algorithms still has many open

questions and is still an ongoing active research topic (Yang, 2011a,c).

The notable performance of metaheuristic algorithms is due to how they imitate

the best features in nature. Intensification and diversification are two main features

of metaheuristic algorithms (Blum and Roli, 2003; Gandomi and Alavi, 2012a;

Yang, 2010a, 2011c). The intensification phase, also called the exploitation phase,

2 Metaheuristic Applications in Structures and Infrastructures

searches the current best solutions and selects the best candidates or solutions. The

diversification phase, also called the exploration phase, ensures that the algorithm

explores the search space more efficiently. The overall efficiency of an algorithm

is mainly influenced by a fine balance between these two components. The system

may be trapped in local optima if there is too little exploration or too much exploi-

tation. In this case, it would be very difficult or even impossible to find the global

optimum. On the other hand, if there is too much exploration but too little exploita-

tion, it may be difficult for the system to converge. In this case, the overall search

performance decelerates. Balancing these two components is itself a major optimi-

zation problem (Yang, 2011c). Evidently, simple exploitation and exploration are a

part of the search. During the search, a proper mechanism or criterion should be

considered to select the best solutions. “Survival of the fittest” is a common crite-

rion. It is based on repeatedly updating the current best solution found so far.

Moreover, certain elitism should be used. This is to verify that the best or fittest

solutions are not lost and are passed onto the next generations.

Each algorithm and its variants use different ways to obtain a balance between

exploration and exploitation. Certain randomization in combination with a deter-

ministic procedure can be considered an efficient way to achieve exploration or

diversification. This makes sure that the newly generated solutions distribute as

diversely as possible in the feasible search space. From the implementation view-

point, the actual way of implementing the algorithm does affect the performance to

some degree. Hence, validation and testing of implementation of any algorithm are

important (Talbi, 2009).

1.2.2 No Free Lunch Theorems

There are the so-called “no free lunch theorems,” which can have significant impli-

cations in the field of optimization (Wolpert and Macready, 1997). One of the theo-

rems states that if algorithm A outperforms algorithm B for some optimization

functions, then B will be superior to A for other functions. In other words, if aver-

aged over all possible function space, both algorithms A and B will perform, on

average, equally well. That is to say, there are no universally better algorithms. An

alternative viewpoint is that there is no need to find the average over all the possi-

ble functions for a given optimization problem. In this case, the major task is to

find the best solutions, which has nothing to do with the average over all possible

function space. Other researchers believe that there is no universal tool and, based

on experiences, some algorithms outperform others for given types of optimization

problems. Thus, the main objective would be either to choose the most

suitable algorithm for a given problem or to design better algorithms for most types

of problems, not necessarily for all the problems.

1.3 Metaheuristic Algorithms in Modeling

Various methodologies can be employed for nonlinear system modeling. Each

method has its own advantages or drawbacks. The need to determine both the

3Metaheuristic Algorithms in Modeling and Optimization

structure and the parameters of the engineering systems makes the modeling of

these systems a difficult task. In general, models are classified into two main

groups: (i) phenomenological and (ii) behavioral (Metenidis et al., 2004). The first

class is established by taking into account the physical relationships governing a

system. The structure of a phenomenological model is chosen on the basis of a

priori knowledge about the system. To cope with the design complexity of

phenomenological models, behavioral models are usually used. The behavioral

models capture the relationships between the inputs and the outputs on the basis of

a measured set of data. Thus, there is no need for a priori knowledge about the

mechanisms that produced the experimental data. Such models are beneficial

because they can provide very good results with minimal effort (Gandomi and

Alavi, 2011, 2012a,b; Metenidis et al., 2004). Statistical regression techniques are

widely used in behavioral modeling approaches.

Several alternative metaheuristic approaches have been developed for behavioral

modeling. Developments in computer hardware during the last two decades have

made it much easier for these techniques to grow into more efficient frameworks.

In addition, various metaheuristics may be used as efficient tools in problems

where conventional approaches fail or perform poorly. Two well-known classes of

metaheuristic algorithms used in nonlinear modeling are ANNs (Haykin, 1999) and

GP (Koza, 1992). ANNs have been used for a wide range of structural engineering

problems (Alavi and Gandomi, 2011a; Sakla and Ashour, 2005). In spite of the suc-

cessful performance of ANNs, they usually do not give a deep insight into the pro-

cess for which they obtain a solution. GP, as an extension of GAs, possess

completely new characteristics. GP is essentially a supervised machine-learning

approach that searches a program space instead of a data space and automatically

generates computer programs that are represented as tree structures and expressed

using a functional programming language (Gandomi and Alavi, 2011; Koza, 1992).

The ability to generate prediction models without assuming the form of the existing

relationships is surely a main advantage of GP over regression and ANN techni-

ques. GP and its variants are widely used for solving real-world problems (Alavi

and Gandomi, 2011b; Gandomi et al., 2011a,b). There are some other metaheuristic

algorithms have been described in the literature for modeling; these include, fuzzy

logic (FL) and support vector machine (SVM). These algorithms (ANNs, GP, FL,

and SVM) are explained in the following sections.

1.3.1 Artificial Neural Networks

ANNs emerged as a result of simulating a biological nervous system. The ANN

method was developed in the early 1940s by McCulloch and coworkers (Perlovsky,

2001). The first studies were focused on building simple neural networks to model

simple logic functions. At present, ANNs have been applied to problems that do

not have algorithmic solutions or problems with complex solutions. In this study,

the approximation ability of two of the most well-known ANN architectures, multi-

layer perceptron (MLP) and radial basis function (RBF), are investigated.


1.3.1.1 Multilayer Perceptron Network

MLP networks are a class of ANN structures using feed-forward architecture. They

are among the most widely used metaheuristics for modeling complex systems in

real-world applications (Alavi et al., 2010a). The MLP networks are usually applied

to perform supervised learning tasks, which involve iterative training methods to

adjust the connection weights within the neural network. MLPs are universal

approximators; that is, they are capable of approximating essentially any continuous

function to an arbitrary degree of accuracy. They are often trained using back propa-

gation (BP) (Rumelhart et al., 1986) algorithms. MLPs consist of an input layer, at

least one hidden layer of neurons, and an output layer. Each of these layers has sev-

eral processing units, and each unit is fully interconnected with weighted connec-

tions to units in the subsequent layer. Each layer contains a number of nodes. Every

input is multiplied by the interconnection weights of the nodes. The output (hj) is

obtained by passing the sum of the product through an activation function. Further

details of MLPs can be found in Cybenko (1989) and Haykin (1999).

1.3.1.2 Radial Basis Function

RBFs have feed-forward architectures. Compared with other ANN structures such

as MLPs, the RBF procedure for finding complex relationships is generally faster,

and their training is much less computationally intensive. The structure of the RBF

network consists of an input layer, a hidden layer with a nonlinear RBF activation

function, and a linear output layer. Input vectors are transformed into RBFs by

means of the hidden layer (Alavi et al., 2009).

The transformation functions used are based on a Gaussian distribution as an

activation function. The center and width are two important parameters that are

related to the Gaussian basis function. As the distance, usually Euclidean distance,

between the input vector and its center increases, as the output given by the activa-

tion function decays to zero. The rate of decrease in the output is controlled by the

width of RBF. The RBF networks with Gaussian basis functions have been shown

to be universal function approximators with high point-wise convergence (Girosi

and Poggio, 1990).

1.3.2 Genetic Programming

GP is a symbolic optimization technique that creates computer programs to solve a

problem using the principle of Darwinian natural selection (Koza, 1992). Friedberg

(1958) left the first footprints in the area of GP by using a learning algorithm to

stepwise improve a program in a stepwise manner. Much later, Cramer (1985)

applied GAs and tree-like structures to evolve programs. The breakthrough in GP

then came in the late 1980s with the experiments of Koza (1992) on symbolic

regression. GP was introduced by Koza (1992) as an extension of GA. The main

difference between GP and GA is the representation of the solution. The GP solu-

tions are computer programs that are represented as tree structures and expressed in


a functional programming language (like LISP) (Koza, 1992). GA first creates a

string of numbers that represent the solutions. In GP, the evolving programs (indi-

viduals) are parse trees that, unlike fixed-length binary strings, can vary in length

throughout the run. Essentially, this was the beginning of computer programs that

could program themselves (Koza, 1992). Since GP often evolves computer pro-

grams, the solutions can be executed without post-processing, while coded binary

strings typically evolved by GA require post-processing. The optimization techni-

ques, like GA, are generally used in parameter optimization to evolve so as to find

the best values for a given set of model parameters. GP, on the other hand, provides

the basic structure of the approximation model, together with the values of its para-

meters (Javadi and Rezania, 2009). GP optimizes a population of computer pro-

grams according to a fitness landscape determined by a program’s ability to

perform a given computational task. The fitness of each program in the population

is evaluated using a predefined fitness function. Thus, the fitness function is the

objective function GP aims to optimize (Torres et al., 2009).

This classical GP approach is referred to as tree-based GP. In addition to the tra-

ditional tree-based GP, there are other types of GP where programs are represented

in different ways (Figure 1.1). These are linear GP and graph-based GP (Alavi

et al., 2012; Banzhaf et al., 1998). The emphasis of the present study is on the

linear-based GP techniques.

1.3.2.1 Linear-Based GP

There are a number of reasons for using linear GP. Basic computer architectures

are fundamentally the same now as they were 20 years ago when GP began.

Almost all the architectures represent computer programs in a linear fashion. Also,

computers do not naturally run tree-shaped programs. Hence, slow interpreters

have to be used as part of tree-based GP. Conversely, by evolving the binary bit

patterns, in fact, used by computers, the use of an expensive interpreter (or com-

piler) is avoided and GP can run several orders of magnitude faster (Poli et al.,

2007). Several linear variants of GP have been recently proposed. Some of them

are (Oltean and Grossan, 2003a): linear genetic programming (LGP) (Brameier and

Banzhaf, 2007), gene expression programming (GEP) (Ferreira, 2001), multiexpres-

sion programming (MEP) (Oltean and Dumitrescu, 2002), Cartesian genetic pro-

gramming (CGP) (Miller and Thomson, 2002), genetic algorithm for deriving

software (GADS) (Patterson, 2002), and infix form genetic programming (IFGP)

(Oltean and Grossan, 2003b). LGP, GEP, and MEP are the most common linear-

based GP methods. These variants make a clear distinction between the genotype

Genetic programming

Tree-based GP Linear-based GP Graph-based GP

Figure 1.1 Different types of GP.


and the phenotype of an individual. The individuals in these variants are repre-

sented as linear strings (Oltean and Grossan, 2003a).

1.3.2.1.1 Linear Genetic ProgrammingLGP is a subset of GP with a linear representation of individuals. There are several

main differences between LGP and the traditional tree-based GP. Figure 1.2 pre-

sents a comparison of program structures in LGP and tree-based GP. Linear genetic

programs can be seen as a data flow graph generated by multiple usage of register

content. LGP operates on genetic programs that are represented as linear sequences

of instructions of an imperative programming language (like C/C11) (see

Figure 1.2A). As shown in Figure 1.2B, the data flow in tree-based GP is more rig-

idly determined by the tree structure of the program (Brameier and Banzhaf, 2001;

Gandomi et al., 2010).

In the LGP system described here, an individual program is interpreted as a

variable-length sequence of simple C instructions. The instruction set or function

set of LGP consists of arithmetic operations, conditional branches, and function

calls. The terminal set of the system is composed of variables and constants. The

instructions are restricted to operations that accept a minimum number of constants

or memory variables, called registers (r), and assign the result to a destination reg-

ister, e.g., r05 r11 1. LGPs can be converted into a functional representation by

successive replacements of variables, starting with the last effective instruction

(Oltean and Grossan, 2003a). Automatic induction of machine code by genetic pro-

gramming (AIMGP) is a particular form of LGP. In AIMGP, evolved programs are

stored as linear strings of native binary machine code and are directly executed by

the processor during fitness calculation. The absence of an interpreter and complex

memory handling results in a significant speedup in the AIMGP execution com-

pared to tree-based GP. This machine-code-based LGP approach searches for the

computer program and the constants at the same time. Comprehensive descriptions

of the basic parameters used to direct a search for a linear genetic program can be

found in Brameier and Banzhaf (2007).

1.3.2.1.2 Gene Expression ProgrammingGEP is a natural development of GP. It was first presented by Ferreira (2001). GEP

consists of five main components: function set, terminal set, fitness function, control

parameters, and termination condition. Unlike the parse-tree representation in the

conventional GP, GEP uses a fixed length of character strings to represent solutions

y = f [0] = (v[1] / 3) + v[4]

f [0] = 0;

L0: f [0] + = v[1];

L1: f [0] / = 3;

+

v[4]/ L2: f [0] + = v[4];

return f [0]; v[1] 3

(B)(A) Figure 1.2 Comparison of the GP

structures. (A) LGP and (B) Tree-

based GP.

Source: After Alavi et al. (2010).


to the problems, which are afterward expressed as parse trees of different sizes and

shapes. These trees are called GEP expression trees (ETs). One advantage of the

GEP technique is that the creation of genetic diversity is extremely simplified as

genetic operators work at the chromosome level. Another strength of GEP is that it

refers to its unique; the multigenic nature allows the evolution of more complex pro-

grams composed of several subprograms (Gandomi and Alavi, 2011b, 2012c).

GEP genes have a fixed length, which is predetermined for a given problem.

Thus, what varies in GEP is not the length of genes but the size of the correspond-

ing ETs.

1.3.2.1.3 Multiexpression ProgrammingMEP is a subarea of GP developed by Oltean and Dumitrescu (2002). MEP uses

linear chromosomes for solution encoding. It has a special ability to encode multi-

ple solutions (computer programs) of a problem in a single chromosome. Based on

the fitness values of the individuals, the best encoded solution is chosen to repre-

sent the chromosome. There is no increase in the complexity of the MEP decoding

process, compared with the other GP variants that store a single solution in a chro-

mosome. The exception is for the situations where the set of training data is not

known (Oltean and Grossan, 2003a,c). The evolutionary steady-state MEP algo-

rithm typically starts by the creation of a random population of individuals.

MEP is represented in a similar way to that of C and Pascal compilers translat-

ing mathematical expressions into machine code. The number of MEP genes per

chromosome is constant, which specifies the length of the chromosome. A terminal

(an element in the terminal set T) or a function symbol (an element in the function

set F) is encoded by each gene. A gene that encodes a function includes pointers

toward the function arguments. Function parameters always have indices of lower

values than the position of that function itself in the chromosome. The first symbol

in a chromosome must be a terminal symbol as stated by the proposed representa-

tion scheme.

The fitness of each expression in an MEP chromosome is calculated to designate

the best encoded expression in that chromosome (Alavi et al., 2010b).

1.3.3 Fuzzy Logic

FL is a process of mapping an input space onto an output space using membership

functions and linguistically specified rules (Ceven and Ozdemir, 2007). The con-

cept of “fuzzy set” was preliminarily introduced by Zadeh (1965). The fuzzy

approach is more in line with human thought as it provides possible rules relating

input variables to the output variable. FL is well suited to implementing control

rules that can only be expressed verbally. It can also be used for the modeling of

systems that cannot be modeled with linear differential equations (Afandizadeh-

Zargari et al., 2012).

The essential idea in FL is the concept of partial belongings of any object to dif-

ferent subsets of the universal set instead of full belonging to a single set. Partial

belonging to a set can be described numerically by a membership function


(Topcu and Sarıdemir, 2008). A membership function is a curve, mapping an input

element to a value between 0 and 1, showing the degree to which it belongs to a

fuzzy set. Membership degree is the value of every element, varying between 0 and 1.

A membership function can have different shapes for different kinds of fuzzy sets,

such as bell, sigmoid, triangle, and trapezoid (Ceven and Ozdemir, 2007). In FL, rules

and membership sets are used to make a decision. The idea of a fuzzy set is basic and

simple: an object is allowed to have a gradual membership of a set. It means the

degree of truth of a statement can range between 0 and 1, which is not limited to just

two logic values {true, false}.

When linguistic variables are used, these degrees may be managed by specific

functions. A fuzzy system consists of output and input variables. For each variable,

fuzzy sets that characterize those variables are formulated, and for each fuzzy set a

membership function can be defined. After that, the rules that relate the output and

input variables to their fuzzy sets are defined. Figure 1.3 depicts a typical FL sys-

tem where a general fuzzy inference system has basically four components: fuzzifi-

cation, fuzzy rule base, fuzzy inference engine, and defuzzification (Topcu and

Sarıdemir, 2008).

1.3.4 Support Vector Machines

SVM is a well-known machine-learning method, based on statistical learning theory

(Boser et al., 1992; Vapnik, 1995, 1998). Similar to ANNs, the SVM procedure

involves a training phase in which a series of input and target output values are fed

into the model. A trained algorithm is then employed to evaluate a separate set of test-

ing data. Two fundamental concepts underlying the SVM are (Goh and Goh, 2007):

1. An optimum margin classifier. This is a linear classifier that constructs a separating

hyperplane (decision surface) such that the distance between the positive and the negative

examples is maximized.

2. Use of kernel functions. A kernel function is a function that calculates the dot product of two

vectors. A suitable nonlinear kernel can map the original example data onto a new data set

that become linearly separable in a high-dimensional feature space, even though they are

nonseparable in the original input space (Goh and Goh, 2007; Vapnik, 1995, 1998).

Outputdata

Fuzzification Defuzzification

Rules

InferenceFuzzy input sets Fuzzy output sets

Input data

Figure 1.3 FL system.


The SVM procedure can be outlined as follows (Goh and Goh, 2007):

a. Choosing a kernel function with related kernel parameters.

b. Solving a Lagrange cost function and obtaining the Lagrange multipliers.

c. Carrying out the binary classification task, with training input data points.

Comprehensive descriptions of SVM can be found in more advanced literature

(Goh and Goh, 2007; Vapnik, 1995).

1.4 Metaheuristic Algorithms in Optimization

To find an optimal solution to an optimization problem is often a very challenging

task, depending on the choice and the correct use of the right algorithm. The choice

of an algorithm may depend on the type of problem, the available of algorithms,

computational resources, and time constraints. For large-scale, nonlinear, global

optimization problems, there is often no agreed guideline for algorithm choice, and

in many cases, there is no efficient algorithm. For hard optimization problems,

especially for nondeterministic polynomial-time hard, or NP-hard, optimization

problems, there is no efficient algorithm at all. In most applications, an optimiza-

tion problem can be commonly expressed in the following generic form (Yang,

2010a, 2011e):

minimize xARnfiðxÞ; ði5 1; 2; . . .;MÞ; ð1:1Þ

subject to

hjðxÞ5 0; ðj5 1; 2; . . .; JÞ; ð1:2Þ

gkðxÞ# 0; ðk5 1; 2; . . .;KÞ; ð1:3Þ

where fi(x), hj(x), and gk(x) are functions of the design vector x5 (x1, x2, . . ., xn)T.

Here, the components xi of x are called design or decision variables, and they can

be real continuous, discrete, or a mix of these two. The functions fi(x), where i5 1,

2, . . ., M are called the objective functions, or simply cost functions, and in the

case of M5 1, there is only a single objective. The space spanned by the decision

variables is called the design space or search space. The equalities for hj and

inequalities for gk are called constraints. It is worth pointing out that we can also

write the inequalities in the other way $ 0, and we can also formulate the objec-

tives as a maximization problem.

Various algorithms may be used for solving optimization problems. The conven-

tional or classic algorithms are mostly deterministic. As an instance, the simplex

method in linear programming is deterministic. Some other deterministic optimization

algorithms, such as Newton�Raphson algorithm, use the gradient information and are

called gradient-based algorithms. Nongradient-based, or gradient-free/derivative-free,

algorithms only use the function values, not any derivative (Yang, 2011b).


Heuristic and metaheuristic are the main types of the stochastic algorithms. The

difference between heuristic and metaheuristic algorithms is negligible. Heuristic

means “to find” or “to discover by trial and error.” Quality solutions to a tough

optimization problem can be found in a reasonable amount of time, but there is no

guarantee that optimal solutions are reached. This is useful when good solutions,

but not necessarily the best solutions, are needed within a reasonable amount of

time (Koziel and Yang, 2011; Yang, 2010a).

As discussed earlier in this chapter, metaheuristic optimization algorithms are

often inspired by nature. These metaheuristic algorithms can be classified into dif-

ferent categories based on the source of inspiration as shown in Figure 1.4. The

main category is the biology-inspired algorithms, which generally use biological

evolution and/or collective behavior of animals as their models. Science is another

source of inspiration for metaheuristic algorithms. These algorithms are usually

inspired by physics and chemistry. Moreover, art-inspired algorithms have been

successful for global optimization. These are generally inspired by the creative

behavior of artists such as musicians and architects. Social behavior is another

source of inspiration and the socially inspired algorithms simulate social behavior

to solve optimization.

Although there are different sources of inspiration for the metaheuristic optimi-

zation algorithms, they have similarities in their structures. Therefore, they can also

be classified into two main categories: evolutionary algorithms and swarm

algorithms.

1.4.1 Evolutionary Algorithms

The evolutionary algorithms generally use an iterative procedure based on a biolog-

ical evolution progress to solve optimization problems. Some of the evolutionary

algorithms are described below.

1.4.1.1 Genetic Algorithm

GAs are a powerful optimization method based on the principles of genetics and

natural selection (Holland, 1975). Holland (1975) was the first to use the crossover

and recombination, mutation, and selection in the study of adaptive and artificial

systems. These genetic operators form the essential part of GA for problem-solving.

Up to now, many variants of GA have been developed and applied to a wide

range of optimization problems (Nikjoofar and Zarghami, 2013; Rani et al., 2012).

Nature-inspired

Bio-inspired Science-inspired Art-inspired Social-inspired

Figure 1.4 Source of inspiration in metaheuristic optimization algorithms.


One of the main advantages of GA is that it is a gradient-free method with the flexi-

bility to deal with various types of optimization, whether the objective function is

stationary or nonstationary, linear or nonlinear, continuous or discontinuous, or with

random noise. In GA, a population can simultaneously find the search space in many

directions because multiple offsprings in the population act like independent agents.

This feature idealizes the parallelization of the algorithms for implementation.

Moreover, different parameters and groups of encoded strings can be manipulated at

the same time. Despite several advantages, Gas have some disadvantages pertaining

to the formulation of fitness function, the usage of population size, the choice of the

important parameters, and the selection criteria of a new population. The conver-

gence of GA can be seriously dependent on the appropriate choice of these

parameters.

1.4.1.2 Differential Evolution

Differential evolution (DE) was developed by Storn and Price (1997). It is a

vector-based evolutionary algorithm and can be considered as a further develop-

ment to GAs. It is a stochastic search algorithm with a self-organizing tendency

and does not use the information of derivatives. DE carries out operations over

each component (or each dimension of the solution). Solutions are represented in

terms of vectors, and then mutation and crossover are carried out using these

vectors (Gandomi et al., 2012a). For example, in GAs, mutation is carried out at

one site or multiple sites of a chromosome; but in DE, a difference vector of two

randomly chosen vectors is used to perturb an existing vector. Such vectorized

mutation can be viewed as a self-organizing search, directed toward optimality

(Yang, 2008, 2010a). This kind of perturbation is carried out over each population

vector, and thus can be expected to be more efficient. Similarly, crossover is also a

vector-based component-wise exchange of chromosomes or vector segments.

1.4.1.3 Harmony Search

Harmony search (HS) algorithm is a music-inspired algorithm, based on the impro-

visation process of a musician (Geem et al. 2001). Previous reviews of the HS liter-

ature have focused on applications in civil engineering such as engineering

optimization (Lee and Geem, 2005), design of structures (Lee et al., 2005), design

of water-distribution networks (Geem, 2006), geometry design of geodesic domes

(Saka, 2007), design of steel frames (Degertekin, 2008a,b), groundwater manage-

ment problems (Ayvaz and Elci, 2012), and geotechnical engineering problems

(Cheng and Geem, 2012).

HS algorithms include a number of optimization operators, such as the harmony

memory (HM), the harmony memory size (HMS), the harmony memory consider-

ing rate (HMCR), and the pitch adjusting rate (PAR). In the HS algorithm, the HM

stores the feasible vectors, which are all in the feasible space. The HMR determines

the number of vectors to be stored.


During the optimization process, a new harmony vector is generated from the

HM, based on memory considerations, pitch adjustments, and randomization. After

generating a new harmony vector, if it is better than the worst harmony in the HM,

judged in terms of the objective function value, the new harmony is included in the

HM and the existing worst harmony is excluded from the HM. Pitch adjustment is

similar to the mutation operator in GAs. Although adjusting pitch has a similar

role, it is limited to a certain local pitch adjustment and thus corresponds to a local

search. The use of randomization can drive the system further to explore various

regions with high solution diversity so as to find the global optimality.

1.4.2 Swarm-Intelligence-Based Algorithms

Swarm-intelligence-based algorithms use the collective behavior of animals such as

birds, insects, or fishes. Here, we introduce briefly some of the most widely used

swarm algorithms.

1.4.2.1 Particle Swarm Optimization

The PSO algorithm, inspired by social behavior simulation, was initially proposed

by Kennedy and Eberhart (1995). PSO used the idea that social sharing of informa-

tion among members may have some evolutionary advantage (Kennedy et al.,

2001). PSO has been applied to many real-world problems (Talatahari et al.,

2012a; Yang, 2008, 2010a). A standard PSO algorithm is initialized with a popula-

tion (swarm) of random potential solutions (particles). Each particle iteratively

moves across the search space and is attracted to the position of the best fitness his-

torically achieved by the particle itself (local best) and by the best among the

neighbors of the particle (global best) (Kaveh and Talatahari, 2009a). In fact, in the

PSO, instead of using more traditional genetic operators, each particle adjusts its

flying according to its own flying experience and its companions’ flying experience

(Hadidi et al., 2011; Kaveh and Talatahari, 2008, 2009b). Chaos theory can also

improve the performance of the PSO by tuning its main constants or random vari-

ables (Gandomi et al., 2013a).

The original PSO, developed by Kennedy and Eberhart (1995), used an equation

to calculate the velocity of each particle according to previous velocity, direction

of the best position of each particle itself and direction of the best swarm, and then

update the particle position.

After many numerical simulations, Shi (1998) added a weighting/inertia factor

to the velocity equation to control the trade-off between the global exploration and

the local exploitation abilities of the flying particles.

A well-chosen weight can stabilize the swarm as well as speed up the conver-

gence. By using the linearly decreasing inertia weight, the PSO lacks global search

ability at the end of run even when the global search ability is required to jump out

of the local minimum in some cases. Nevertheless, the results shown in literature

illustrate that by using a linearly decreasing inertia weight the performance of the

PSO can be improved greatly and have better results than that of both a simple


PSO and an evolutionary programming as reported in Angeline (1998) and Shi and

Eberhart (1999). Eusuff and Lansey (2003) combined the benefits of the local

search tool of the PSO and the idea of mixing information from parallel local

searches (Duan et al., 1993) to solve global optimization problems. They called

this algorithm as shuffled frog-leaping (SFL) algorithm.

1.4.2.2 Ant Colony Optimization

In 1992, Dorigo developed a paradigm known as the ant colony optimization

(ACO), a cooperative search technique that mimics the foraging behavior of real-

life ant colonies (Dorigo, 1992; Dorigo et al., 1996). The ant algorithms mimic the

characteristics of real ants that can rapidly establish the shortest route from food

source to their nest and vice versa. Ants start searching the area surrounding their

nest in a random manner. Ethologists observed that ants can construct the shortest

path from their colony to the feed source and back using pheromone trails

(Deneubourg and Goss, 1989; Goss et al., 1990). When ants encounter an obstacle,

at first, there is an equal probability for all ants to move right or left, but after a

while, the number of ants choosing the shorter path increases because of the

increase in the amount of the pheromone on that path. With the increase in the

number of ants and pheromone on the shorter path, all of the ants will choose and

move along the shorter one (Kaveh and Talatahari, 2010a, Talatahari et al., 2012b).

In fact, real ants use their pheromone trails as a medium for communication of

information among them. When an isolated ant comes across some food source in

its random sojourn, it deposits a quantity of pheromone on that location. Other ran-

domly moving ants in the neighborhood can detect this marked pheromone trail.

Furthermore, these ants can follow this trail with a very high degree of probability

and simultaneously enhance the trail by depositing their own pheromone. More and

more ants follow the pheromone-rich trail and the probability of the trail being fol-

lowed by other ants is further enhanced by the increased trail deposition. This is an

autocatalytic (positive feedback) process that favors the path along which more

ants previously traversed. The ant algorithms are based on the indirect communica-

tion capabilities of the ants. In ACO algorithms, virtual ants are deputed to generate

rules by using heuristic information or visibility and the principle of indirect phero-

mone communication capabilities for iterative improvement of rules.

The general procedure of the ACO algorithm manages the scheduling of three

steps: initialization, solution construction, and pheromone updating. The initializa-

tion of the ACO includes two parts: the first part is initialization of the pheromone

trail. In the second part, a number of ants are arbitrarily placed on the nodes chosen

randomly. Then each of the distributed ants will perform a tour on the graph by

constructing a path according to the node transition rule described next.

For generation of a solution, each ant constructs a complete solution to the prob-

lem according to a probabilistic state transition rule. The state transition rule

depends mainly on the state of the pheromone and visibility of ants. Visibility is an

additional ability used to make this method more efficient. When every ant has con-

structed a solution, the intensity of pheromone trails on each edge is updated by the


pheromone updating rule, which is applied in two phases: first, an evaporation phase

where a fraction of the pheromone evaporates, and then a reinforcement phase,

where the elitist ant, which has the best solution among the others, deposits an

amount of pheromone. At the end of each movement, local pheromone update

reduces the level of pheromone trail on paths selected by the ant colony during the

preceding iteration.

1.4.2.3 Bee Algorithms

Bee algorithms are another class of metaheuristic algorithms that mimic the behav-

ior of bees (Karaboga, 2005; Yang, 2005, 2008). Different variants of bee algo-

rithms use slightly different characteristics of the behavior of bees. For example, in

the honeybee-based algorithms, forager bees are allocated to different food sources

(or flower patches) so as to maximize the total nectar intake (Karaboga, 2005;

Nakrani and Tovey, 2004; Pham et al., 2006; Yang, 2005). In the virtual bee algo-

rithm (VBA), developed by Yang (2005), pheromone concentrations can be linked

with the objective functions more directly. On the other hand, the artificial bee col-

ony (ABC) optimization algorithm was first developed by Karaboga (2005). In the

ABC algorithm, the bees in a colony are divided into three groups. Unlike the

honey bee algorithm, which has two groups of the bees (forager bees and observer

bees), bees in ABC are more specialized (Afshar et al., 2007; Karaboga, 2005).

In the ABC algorithm, the colony of the artificial honey bees contains three

groups of bees including employed bees (forager bees), onlooker bees (observer

bees), and scouts. The first half of the colony consists of the employed artificial

bees and the second half includes the onlookers. The position of a food source

represents a possible solution to the considered optimization problem and the

amount of nectar at the food source corresponds to the quality or fitness of the

associated solution. At first, the ABC algorithm generates a randomly distributed,

predefined number of initial population. After initialization, the population of the

positions (solutions) is subjected to repeated cycles of the search process of the

employed bees, onlooker bees, and scout bees. An employed bee produces a modi-

fication on the position (solution) in its memory depending on the local information

(visual information) and tests the nectar amount (fitness value) of the new food

source (new solution). Provided that the nectar amount of the new source is higher

than that of the previous one, the bee memorizes the new position and forgets the

old one. Otherwise, it keeps the position of the previous source in its memory.

When all the employed bees complete the search process, they share the nectar

information of the food sources and their position information with the onlooker

bees in the dance area.

1.4.2.4 Firefly Algorithm

The firefly algorithm (FA) was first developed by Yang (2008, 2009), and was

based on the flashing patterns and behavior of fireflies. In essence, FA uses the fol-

lowing three idealized rules: (i) fireflies are unisexual so that one firefly will be

attracted to other fireflies regardless of their sex; (ii) The attractiveness is


proportional to the brightness and they both decrease as their distance increases.

Thus, for any two flashing fireflies, the less bright one will move toward the bright-

er one. If neither is brighter, they will each move randomly. (iii) The brightness of

a firefly is determined by the landscape of the objective function.

A demo version of FA implementation, without Levy flights, can be found at

Mathworks file exchange website.1 FA has attracted much attention (Apostolopoulos

and Vlachos, 2011; Gandomi et al., 2011c; Sayadi et al., 2010; Talatahari et al.,

2012c; Yang et al., 2012). A discrete version of FA can efficiently solve NP-hard

scheduling problems (Sayadi et al., 2010), while a detailed analysis has demonstrated

the efficiency of FA over a wide range of test problems, including multiobjective load

dispatch problems (Apostolopoulos and Vlachos, 2011). A chaos-enhanced FA with a

basic method for automatic parameter tuning is also developed (Yang, 2011b), and the

use of various chaotic maps can significantly improve the performance of FA, though

different chaotic maps may have different effects (Gandomi et al., 2013b).

1.4.2.5 Cuckoo Search

Cuckoo search (CS) is one of the latest nature-inspired metaheuristic algorithms,

developed by Yang and Deb (2009). CS is based on the brood parasitism of some

cuckoo species. In addition, this algorithm is enhanced by the so-called Levy

flights (Pavlyukevich, 2007), rather than by simple isotropic random walks. Recent

studies show that CS is potentially far more efficient than PSO and GAs (Yang and

Deb, 2010). For simplicity in describing the standard CS, we now use the following

three idealized rules: (i) each cuckoo lays one egg at a time, and dumps it in a ran-

domly chosen nest; (ii) the best nests with high-quality eggs will be carried over to

the next generations; and (iii) the number of available host nests is fixed, and the

egg laid by a cuckoo is discovered by the host bird with a probability between

0 and 1.

A Matlab implementation is given by the author, and can be downloaded.2 CS is

very efficient in solving engineering optimization problems (Gandomi et al.,

2011d, 2012c).

1.4.2.6 Bat Algorithm

Bat algorithm (BA) is a relatively new metaheuristic, developed by Yang (2010b).

It was inspired by the echolocation behavior of microbats. Microbats use a type of

sonar, called echolocation, to detect prey, avoid obstacles, and locate their roosting

crevices in the dark. These bats emit a very loud sound pulse and listen for the

echo that bounces back from the surrounding objects. Their pulses vary in proper-

ties and can be correlated with their hunting strategies, depending on the species.

Most bats use short, frequency-modulated signals to sweep through about an

octave, while others more often use constant-frequency signals for echolocation.

1 http://www.mathworks.com/matlabcentral/fileexchange/29693-firefly-algorithm2 www.mathworks.com/matlabcentral/fileexchange/29809-cuckoo-search-cs-algorithm


http://www.mathworks.com/matlabcentral/fileexchange/29693-firefly-algorithm

http://www.mathworks.com/matlabcentral/fileexchange/29809-cuckoo-search-cs-algorithm

Their signal bandwidth varies depends on the species and is often increased by

using more harmonics.

BA has been extended to multiobjective bat algorithm (MOBA) by Yang

(2011d), and preliminary results suggest that it is very efficient (Yang and

Gandomi, 2012; Gandomi et al., 2012d).

1.4.2.7 Charged System Search

The charged system search (CSS) is another of the more recently introduced meta-

heuristic algorithms (Kaveh and Talatahari, 2010b). This algorithm has been used

to solve different types of optimization problems such as the design of skeletal

structures (Kaveh and Talatahari, 2010c), the design of grillage systems (Kaveh

and Talatahari, 2010d), parameter identification of MR dampers (Talatahari et al.,

2012d), and the design of composite open channels (Kaveh et al., 2012).This

algorithm was inspired by the governing laws of charged systems. Like the swarm

algorithms, CSS uses multiple agents/charged particles, and each agent can be con-

sidered as a charged sphere. However, these agents are treated as charged particles

(CP) they can affect each other according to the Coulomb and Gauss laws of

electrostatics.

The governing laws of motion is from the Newtonian mechanics. CPs can

impose electrical forces on the others, and the forces vary with the separation dis-

tance between the CPs, and for a CP located outside the sphere is inversely propor-

tional to the square of the separation distance between the particles. At each

iteration, each CP moves toward its new position considering the resultant forces

and its previous velocity. If each CP exits from the allowable search space, its posi-

tion is corrected using the HS-based handling approach as described by Kaveh and

Talatahari (2009c). In addition, to store the best design, a charged memory (CM) is

considered containing the CM number of positions for the so far best agents.

1.4.2.8 Krill Herd

Krill herd (KH) is another novel biologically inspired algorithm, proposed by

Gandomi and Alavi (2012a). The KH algorithm is based on simulating the herding

behavior of krill individuals. The minimum distances of each individual krill from

food and from highest density of the herd are considered as the objectives for the

Lagrangian movement. The time-dependent positions of the krill individuals are

updated by three main components/factors:

1. movement induced by the presence of other individuals,

2. foraging activity, and

3. random diffusion.

This algorithm is also a gradient-free method because derivatives are not needed

in the KH algorithm, and KH is also a metaheuristic algorithm because it uses a

stochastic/random search in addition to some deterministic components.

For any metaheuristic algorithm, it is important to tune its related parameters.

One of the interesting parts of the KH algorithm is that it can carefully simulate the


krill behavior, and the values of these coefficients are based on empirical studies of

the real-world krill systems. For this reason, only time interval needs fine-tuning in

the KH algorithm. This can be considered a first attempt to use a real-world system

to derive algorithm-dependent parameters, which can be advantageous. The prelim-

inary results indicate that the KH method is very encouraging for its further appli-

cation to optimization tasks.

1.5 Challenges in Metaheuristics

As we have seen from this review, metaheuristic algorithms have been used suc-

cessfully for solving a variety of real-world problems. However, there remain some

challenging issues concerning metaheuristics. First, theoretical analysis of these

algorithms still lacks a unified framework, and there are many open problems as

outlined by Yang in a recent review (Yang, 2011c). For example, in what ways do

algorithm-dependent parameters affect the efficiency of an algorithm? What is the

optimal balance between exploration and exploitation for metaheuristic algorithms

so that they can perform most efficiently? How can memory in algorithm help to

improve the performance of an algorithm?

Another important issue is the gap between theory and practice because meta-

heuristic applications are expanding rapidly, far more rapidly than mathematical

analysis. At the same time, most applications are concerned with small-scale pro-

blems. Future applications and studies should focus on larger-scale applications.

On the other hand, there are many new algorithms, but more algorithms make it

even harder to understand the working mechanisms of metaheuristics in general. We

may need a unified approach to analyze algorithms, and ideally to classify these

algorithms, so that we can understand all metaheuristics in a more insightful way

(Yang, 2011c). These challenges also provide some timely and hot research opportu-

nities for researchers so that important progress can be made in the near future.

References

Afandizadeh-Zargari, S., Zabihi, S., Alavi, A.H., Gandomi, A.H., 2012. A computational

intelligence based approach for short-term traffic flow prediction. Expert Syst. 29 (2),

124�142.

Afshar, A., Haddad, O.B., Marino, M.A., Adams, B.J., 2007. Honey-bee mating optimization

(HBMO) algorithm for optimal reservoir operation. J. Franklin Inst. 344, 452�462.

Alavi, A.H., Gandomi, A.H., Heshmat, A.A.R., 2010. Discussion on “Soft computing approach

for real-time estimation of missing wave heights” by S.N. Londhe [Ocean Engineering 35

(2008) 1080–1089]. Ocean Eng. 37 (13), 1239�1240.

Alavi, A.H., Gandomi, A.H., 2011a. Prediction of principal ground-motion parameters using

a hybrid method coupling artificial neural networks and simulated annealing. Comput.

Struct. 89 (23�24), 2176�2194.


Alavi, A.H., Gandomi, A.H., 2011b. A robust data mining approach for formulation of geo-

technical engineering systems. Int. J. Comput. Aided Meth. Eng. Eng. Comput. 28 (3),

242�274.

Alavi, A.H., Gandomi, A.H., Gandomi, M., Sadat Hosseini, S.S., 2009. Prediction of

maximum dry density and optimum moisture content of stabilized soil using RBF neural

networks. IES J. A Civ. Struct. Eng. 2 (2), 98�106.

Alavi, A.H., Gandomi, A.H., Mollahasani, A., Heshmati, A.A.R., Rashed, A., 2010a.

Modeling of maximum dry density and optimum moisture content of stabilized soil

using artificial neural networks. J. Plant Nutr. Soil Sci. 173 (3), 368�379.

Alavi, A.H., Gandomi, A.H., Sahab, M.G., Gandomi, M., 2010b. Multi expression program-

ming: a new approach to formulation of soil classification. Eng. Comput. 26 (2),

111�118.

Alavi, A.H., Gandomi, A.H., Bolury, J., Mollahasani, A., 2012. Linear and tree-based genetic

programming for solving geotechnical engineering problems. In: Yang, X.S., et al.,

(Eds.), Metaheuristics in Water Resources, Geotechnical and Transportation

Engineering. Elsevier, Waltham, MA, pp. 289�310. (Chapter 12).

Angeline P.J., 1998. Evolutionary optimization versus particle swarm optimization: philoso-

phy and performance difference. Proceedings of Annual Conference on Evolutionary

Programming, San Diego, 1998, pp. 6012 610.

Apostolopoulos, T., Vlachos, A., 2011. Application of the firefly algorithm for solving the

economic emissions load dispatch problem. Int. J. Combin. (Volume 2011, Article ID

523806. ,http://www.hindawi.com/journals/ijct/2011/523806.html. (accessed

14.01.2012.).

Ayvaz, M.T., Elci, A., 2012. Application of the hybrid HS solver algorithm to the solution

of groundwater management problems. In: Yang, X.S., et al., (Eds.), Metaheuristics in

Water Resources, Geotechnical and Transportation Engineering. Elsevier, pp. 79�97.

(Chapter 4).

Banzhaf, W., Nordin, P., Keller, R., Francone, F., 1998. Genetic Programming—An

Introduction on the Automatic Evolution of Computer Programs and its Application.

dpunkt/Morgan Kaufmann, Heidelberg/San Francisco, CA.

Blum, C., Roli, A., 2003. Metaheuristics in combinatorial optimization: overview and con-

ceptual comparison. ACM Comput. Surv. 35, 268�308.

Boser B.E., Guyon I.M., Vapnik V.N., 1992. A training algorithm for optimal margin classi-

fiers. In: Proceedings of the Fifth Annual ACM Workshop on Computational Learning

Theory, vol. 5. Pittsburgh, pp. 144�152.

Brameier, M., Banzhaf, W., 2001. A comparison of linear genetic programming and neural

networks in medical data mining. IEEE Trans. Evol. Comput. 5 (1), 17�26.

Brameier, M., Banzhaf, W., 2007. Linear Genetic Programming. Springer Science 1

Business Media LLC, New York, NY.

Ceven, E.K., Ozdemir, O., 2007. Using fuzzy logic to evaluate and predict Chenille Yarn’s

shrinkage behaviour. Fibres Textiles East. Europe. 15 (3), 55�59.

Cheng, Y.M., Geem, Z.W., 2012. Hybrid heuristic optimization methods in geotechnical

engineering. In: Yang, X.S., et al., (Eds.), Metaheuristics in Water Resources,

Geotechnical and Transportation Engineering. Elsevier, Waltham, MA, pp. 205�229.

(Chapter 9).

Cortes, C., Vapnik, V., 1995. Support vector networks. Mach. Learn. 20, 273�297.

Cramer, N.L., 1985. A representation for the adaptive generation of simple sequential pro-

grams. In: Genetic Algorithms and Their Applications, Davis, L., Ed. Pittsburgh, PA.

pp. 1832 187.


http://www.hindawi.com/journals/ijct/2011/523806.html

Cybenko, J., 1989. Approximations by superpositions of a sigmoidal function. Math. Cont.

Sign. Syst. 2, 303�314.

Degertekin, S.O., 2008a. Optimum design of steel frames using harmony search algorithm.

Struct. Multidiscip. Optim. 36, 393�401.

Degertekin, S.O., 2008b. Harmony search algorithm for optimum design of steel frame struc-

tures: a comparative study with other optimization methods. Struct. Eng. Mech. 29,

391�410.

Deneubourg, J.L., Goss, S., 1989. Collective patterns and decision-making. Ethnol. Ecol.

Evol. 1, 295�311.

Dorigo, M., 1992. Optimization, Learning and Natural Algorithms. Dip. Elettronica

e Informazione, Politecnico di Milano, Milano, PhD Thesis.

Dorigo, M., Maniezzo, V., Colorni, A., 1996. The ant system: optimization by a colony of

cooperating agents. IEEE Trans. Syst. Man, Cybern. B Cybern. 26 (1), 29�41.

Duan, Q.Y., Gupta, V.K., Sorooshian, S., 1993. Shuffled complex evolution approach for

effective and efficient global minimization. J. Optim. Theor. Appl. 76, 502�521.

Eberhart R.C., Kennedy J., 1995. A new optimizer using particle swarm theory. In:

Proceedings of the Sixth International Symposium on Micro Machine and Human

Science, Nagoya, Japan, 1995.

Eusuff, M.M., Lansey, K.E., 2003. Optimization of water distribution network design using

the shuffled frog leaping algorithm. J. Water. Res. Plan. Manage. 29 (3), 10�25.

Ferreira, C., 2001. Gene expression programming: a new adaptive algorithm for solving pro-

blems. Complex Syst. 13 (2), 87�129.

Ferreira, C., 2006. Gene Expression Programming: Mathematical Modeling by an Artificial

Intelligence. second ed. Springer, Germany.

Francone, F.D., Deschaine, L.M., 2004. Extending the boundaries of design optimization by

integrating fast optimization techniques with machine-code-based, linear genetic pro-

gramming. Inf. Sci. 161, 99�120.

Friedberg, R.M., 1958. A learning machine: Part I. IBM J. Res. Dev. 2, 2�13.

Gandomi, A.H., Alavi, A.H., 2011. Multi-stage genetic programming: a new strategy to non-

linear system modeling. Inf. Sci. 23, 5227�5239.

Gandomi, A.H., Alavi, A.H., 2012a. Krill herd: a new bio-inspired optimization algorithm.

Commun. Nonlinear Sci. Numer. Simul. 17 (12), 4831�4845.

Gandomi, A.H., Alavi, A.H., 2012b. A new multi-gene genetic programming approach to

nonlinear system modeling. Part I: materials and structural engineering problems.

Neural Comput. Appl. 21 (1), 171�187.

Gandomi, A.H., Alavi, A.H., 2012c. A new multi-gene genetic programming approach to

nonlinear system modeling. Part II: geotechnical and earthquake engineering problems.

Neural Comput. Appl. 21 (1), 189�201.

Gandomi, A.H., Alavi, A.H., Sahab, M.G., 2010. New formulation for compressive strength

of CFRP confined concrete cylinders using linear genetic programming. Mater. Struct.

43 (7), 963�983.

Gandomi, A.H., Alavi, A.H., Mirzahosseini, M.R., Moqaddas Nejad, F., 2011a. Nonlinear

genetic-based models for prediction of flow number of Asphalt mixtures. J. Mater. Civil

Eng. ASCE. 23 (3), 248�263.

Gandomi, A.H., Alavi, A.H., Yun, G.J., 2011b. Nonlinear modeling of shear strength of

SFRC beams using linear genetic programming. Struct. Eng. Mech. 38 (1), 1�25.

Gandomi, A.H., Yang, X.S., Alavi, A.H., 2011c. Mixed variable structural optimization

using firefly algorithm. Comput. Struct. 89 (23�24), 2325�2336.


Gandomi, A.H., Yang, X.S., Alavi, A.H., 2011d. Cuckoo search algorithm: a metaheuristic

approach to solve structural optimization problems. Eng. Comput. doi: 10.1007/

s00366-011-0241-y (in press).

Gandomi, A.H., Yang, X.S., Talatahari, S., Deb, S., 2012a. Coupled eagle strategy and dif-

ferential evolution for unconstrained and constrained global optimization. Comput.

Math. Appl. 63 (1), 191�200.

Gandomi, A.H., Babanajad, S.K., Alavi, A.H., Farnam, Y., 2012b. A novel approach to

strength modeling of concrete under triaxial compression. J. Mater. Civ. Eng. 24 (9),

1132�1143.

Gandomi, A.H., Talatahari, S., Yang, X.S., Deb, S., 2012c. Design optimization of truss

structures using cuckoo search algorithm. Struct. Des. Tall Spec. Buildings. 10.1002/

tal.1033.

Gandomi, A.H., Yang, X.S., Talatahari, S., Alavi, A.H., 2012d. Bat algorithm for constrained

optimization tasks. Neural Comput. Appl. doi: 10.1007/s00521-012-1028-9.

Gandomi, A.H., Yun, G.J., Yang, X.S., Talatahari, S., 2013a. Chaos-enhanced accelerated

particle swarm algorithm. Commun. Nonlinear Sci. Numer. Simul. 18 (2), 327�340.

Gandomi, A.H., Yang, X.S., Talatahari, S., Alavi, A.H., 2013b. Firefly algorithm with chaos.

Commun. Nonlinear Sci. Numer. Simul. 18 (1), 89�98.

Geem, Z.W., 2006. Optimal cost design of water distribution networks using harmony

search. Eng. Optim. 38, 259�277.

Geem, Z.W., Kim, J.H., Loganathan, G.V., 2001. A new heuristic optimization algorithm;

harmony search. Simulation. 76, 60�68.

Girosi, F., Poggio, T., 1990. Networks and the best approximation property. Biol. Cybern.

63 (3), 169�176.

Goh, A.T.C., Goh, S.H., 2007. Support vector machines: their use in geotechnical engineer-

ing as illustrated using seismic liquefaction data. Comput. Geotech. 34, 410�421.

Goss, S., Beckers, R., Deneubourg, J.L., Aron, S., Pasteels, J.M., 1990. How trail laying and

trail following can solve foraging problems for ant colonies. In: Hughes, R.N. (Ed.),

Behavioural Mechanisms in Food Selection, NATO-ASI Series, vol. G 20, Berlin.

Hadidi, A., Kaveh, A., Farahmand Azar, B., Talatahari, S., Farahmandpour, C., 2011. An

efficient optimization algorithm based on particle swarm and simulated annealing for

space trusses. Int. J. Optim. Civ. Eng. 1 (3), 375�395.

Haykin, S., 1999. Neural Networks—A Comprehensive Foundation. second ed. Prentice

Hall, Englewood Cliffs, NJ.

Holland, J., 1975. Adaptation in Natural and Artificial Systems. University of Michigan

Press, Ann Anbor, MI.

Javadi, A.A., Rezania, M., 2009. Applications of artificial intelligence and data mining tech-

niques in soil modeling. Geomech. Eng. 1 (1), 53�74.

Karaboga, D., 2005. An Idea Based on Honey Bee Swarm for Numerical Optimization.

Erciyes University, Computer Engineering Department (Technical Report-TR06).

Kaveh, A., Talatahari, S., 2008. A discrete particle swarm ant colony optimization for design

of steel frames. Asian J. Civ. Eng. 9 (6), 563�575.

Kaveh, A., Talatahari, S., 2009a. Hybrid algorithm of harmony search, particle swarm and

ant colony for structural design optimization, studies in computational intelligence,

Harmony Search Algorithms for Structural Design Optimization, vol. 239. Springer,

Berlin, Heidelberg (pp. 159�198).

Kaveh, A., Talatahari, S., 2009b. A particle swarm ant colony optimization algorithm for

truss structures with discrete variables. J. Construct. Steel Res. 65 (8�9), 1558�1568.


Kaveh, A., Talatahari, S., 2009c. Particle swarm optimizer, ant colony strategy and harmony

search scheme hybridized for optimization of truss structures. Comput. Struct. 87

(5�6), 267�283.

Kaveh, A., Talatahari, S., 2010a. An improved ant colony optimization for constrained engi-

neering design problems, engineering computations. Int. J. Comput. Aided Eng. Softw.

27 (1), 155�182.

Kaveh, A., Talatahari, S., 2010b. A novel heuristic optimization method: charged system

search. Acta Mech. 213 (3�4), 267�289.

Kaveh, A., Talatahari, S., 2010c. Optimal design of skeletal structures via the charged sys-

tem search algorithm. Struct. Multidiscip. Optim. 41 (6), 893�911.

Kaveh, A., Talatahari, S., 2010d. Charged system search for optimum grillage systems

design using the LRFD-AISC code. J. Construct. Steel Res. 66 (6), 767�771.

Kaveh, A., Talatahari, S., Farahmand Azar, B., 2012. Optimum Design of Composite Open

Channels Using Charged System Search Algorithm. Iranian Journal of Science &

Technology, Transaction B: Engineering. 36 (C1), 67�77.

Kennedy, J., Eberhart, R., 1995. Particle swarm optimization. In: Proceedings of the IEEE

International Conference on Neural Networks, Piscataway, NJ, pp. 19422 1948.

Kennedy, J., Eberhart, R.C., Shi, Y., 2001. Swarm Intelligence. Morgan Kaufman

Publishers, San Francisco, CA.

Kirkpatrick, S., Gellat, C.D., Vecchi, M.P., 1983. Optimization by simulated annealing.

Science. 220, 671�680.

Koza, J.R., 1992. Genetic Programming: on the Programming of Computers by means of

Natural Selection. MIT Press, Cambridge, MA.

Koziel, S., Yang, X.S., 2011. Computational Optimization, Methods and Algorithms, Studies

in Computational Intelligence, 356. Springer, Berlin, Germany.

Lee, K.S., Geem, Z.W., 2005. A new meta-heuristic algorithm for continuous engineering

optimization: harmony search theory and practice. Comput. Method Appl. Mech. Eng.

194, 3902�3933.

Lee, K.S., Geem, Z.W., Lee, S.-H., et al., 2005. The harmony search heuristic algorithm for

discrete structural optimization. Eng. Optim. 37, 663�684.

Metenidis, M.F., Witczak, M., Korbicz, J., 2004. A novel genetic programming approach to

nonlinear system modelling: application to the DAMADICS benchmark problem. Eng.

Appl. Artif. Intell. 17, 363�370.

Miller, J., Thomson, P., 2002. Cartesian genetic programming. In: Poli, R., Banzhaf, W.,

Langdon, B., Miller, J., Nordin, P., Fogarty, T.C. (Eds.), Genetic Programming.

Springer, Berlin.

Nakrani, S., Tovey, C., 2004. On Honey Bees and Dynamic Server Allocation in Internet

Hosting Centers. Adaptive Behaviour. 12 (3�4), 223�240.

Nikjoofar, A., Zarghami, M., 2013. Water distribution networks designing by the multiobjec-

tive genetic algorithm and game theory. In: Yang, X.S., et al., (Eds.), Metaheuristics in

Water Resources, Geotechnical and Transportation Engineering. Elsevier, Waltham,

MA, pp. 43�77. (Chapter 5).

Oltean, M., Dumitrescu, D., 2002. Multi Expression Programming. Babes-Bolyai University,

Cluj-Napoca, Romania (Technical Report, UBB-01-2002).

Oltean, M., Grossan, C., 2003a. A comparison of several linear genetic programming techni-

ques. Adv. Complex Syst. 14 (4), 1�29.

Oltean, M., Grossan, C., 2003b. Solving classification problems using infix form genetic pro-

gramming. In: Berthold, M. (Ed.), Intelligent Data Analysis. Springer, Berlin,

pp. 242�252. (LNCS 2810).


Oltean, M., Grossan, C., 2003c. Evolving evolutionary algorithms using multi expression

programming. In Artificial life, LNAI, vol. 2801. Springer, pp. 651�658.

Patterson, N., 2002. Genetic Programming with Context-Sensitive Grammars. School of

Computer Science, University of Scotland, Scotland, UK (Ph.D. Thesis).

Pavlyukevich, I., (2007). Levy flights, non-local search and simulated annealing,

J. Computational Physics, vol. 226 , 1830�1844.

Perlovsky, L.I., 2001. Neural Networks and Intellect. Oxford University Press, Oxford, UK.

Pham, D.T., Ghanbarzadeh, A., Koc, E., Otri, S., Rahim, S., Zaidi, M., 2006. The bees algo-

rithm: a novel tool for complex optimisation problems. Proceedings of IPROMS 2006

Conference, pp.4542 461.

Poli, R., Langdon, W.B., McPhee, N.F., Koza, J.R., 2007. Genetic programming: an intro-

ductory tutorial and a survey of techniques and applications. University of Essex, UK

(Technical report [CES-475], 2007).

Rani, D., Jain, S.K., Srivastava, D.K., Perumal, M., 2012. Genetic algorithms and their appli-

cations to water resources systems. In: Yang, X.S., et al., (Eds.), Metaheuristics in

Water Resources, Geotechnical and Transportation Engineering. Elsevier, Waltham,

MA, pp. 43�77. (Chapter 3).

Rumelhart, D.E., Hinton, G.E., Williams, R.J., 1986. Learning internal representations by

error propagation. Proceedings Parallel Distributed Processing. MIT Press, Cambridge,

MA.

Saka, M.P., 2007. Optimum geometry design of geodesic domes using harmony search algo-

rithm. Adv. Struct. Eng. 10, 595�606.

Sakla, S.S., Ashour, A.F., 2005. Prediction of tensile capacity of single adhesive anchors

using neural networks. Comput. Struct. 83 (21�22), 1792�1803.

Sayadi, M.K., Ramezanian, R., Ghaffari-Nasab, N., 2010. A discrete firefly meta-heuristic

with local search for make span minimization in permutation flow shop scheduling pro-

blems. Int. J. Ind. Eng. Comput. 1, 1�10.

Shi Y., Eberhart R.C., 1998. A modified particle swarm optimizer. Proceedings of IEEE

International Conference on Evolutionary Computation, Alaska, 1998, pp. 69�73.

Shi Y., Eberhart R.C., 1999. Empirical study of particle swarm optimization. Proceedings of

the 1999 IEEE Congress on Evolutionary Computation 1999. vol. 3, pp. 19452 1950.

Storn, R., Price, K.V., 1997. Differential evolution—a simple and efficient heuristic for

global optimization over continuous spaces. J. Global Opt. 11 (4), 341�359.

Talatahari, S., Kheirollahi, M., Farahmandpour, C., Gandomi, A.H., 2012a. A multi-stage

particle swarm for optimum design of truss structures. Neural Comput. Appl. doi:

10.1007/s00521-012-1072-5.

Talatahari, S., Singh, V.P., Hassanzadeh, Y., 2012b. Ant colony optimization for estimating

parameters of flood frequency distributions. In: Yang, X.S., et al., (Eds.),

Metaheuristics in Water Resources, Geotechnical and Transportation Engineering.

Elsevier, Waltham, MA, pp. 121�146. (Chapter 6).

Talatahari, S., Gandomi, A.H., Yun, G.Y., 2012c. Optimum design of tower structures by

firefly algorithms. Struct. Des. Tall Spec. Build. doi: 10.1002/tal.1043.

Talatahari, S., Kaveh, A., Mohajer Rahbari, N., 2012d. Parameter identification of Bouc-

Wen model for MR fluid dampers using adaptive charged system search optimization.

J. Mech. Sci. Technol. 26 (8), 2523�2534.

Talbi, E.G., 2009. Metahueristics: From Design to Implementation. Wiley, Hoboken, NJ.

Topcu, I.B., Sarı, M., 2008. Prediction of compressive strength of concrete containing

fly ash using artificial neural networks and fuzzy logic. Comput. Mater. Sci. 41,

305�311.


Torres, R.S., Falcao, A.X., Goncalves, M.A., Papa, J.P., Zhang, B., Fan, W., et al., 2009. A

genetic programming framework for content-based image retrieval. Pattern Recognit.

42 (2), 283�292.

Vapnik, V., 1995. The Nature of Statistical Learning. Springer, New York, NY.

Vapnik, V., 1998. Statistical Learning Theory. Wiley, New York, NY.

Wolpert, D.H., Macready, W.G., 1997. No free lunch theorems for optimization. IEEE

Trans. Evol. Comput. 1 (1), 67�82.

Yang, X.S., 2005. Engineering optimization via nature-inspired virtual bee algorithms.

Artificial Intelligence and Knowledge Engineering Applications: A Bioinspired

Approach. Springer, Berlin, Germany (Lecture Notes in Computer Science, 3562,

pp. 3172 323).

Yang, X.S., 2008. Nature-Inspired Metaheuristic Algorithms. first ed. Luniver Press, Frome.

Yang, X.S., 2009. Firefly algorithms for multimodal optimization. In: Watanabe, O.,

Zeugmann, T. (Eds.), Fifth Symposium on Stochastic Algorithms, Foundation and

Applications (SAGA 2009), LNCS, 5792, pp. 169�178.

Yang, X.S., 2010a. Engineering Optimization: An Introduction with Metaheuristic

Applications. John Wiley and Sons, Hoboken, NJ.

Yang, X.S., 2010b. A new metaheuristic bat-inspired algorithm. In: Gonzalez, J.R., et al.,

(Eds.), Nature-Inspired Cooperative Strategies for Optimization (NICSO 2010).

Springer, Berlin, pp. 65�74. (SCI 284).

Yang, X.S., 2011a. Review of metaheuristics and generalized evolutionary walk algorithm.

Int. J. Bio-Inspired Comput. 3 (2), 77�84.

Yang, X.S., 2011b. Chaos-enhanced firefly algorithm with automatic parameter tuning. Int.

J. Swarm Intell. Res. 2 (4), 1�11.

Yang, X.S., 2011c. Metaheuristic optimization: algorithm analysis and open problems.

In: Pardalos, P.M., Rebennack, S. (Eds.), Proceedings of the Tenth Symposium of

Experimental Algorithms (SEA2011), vol. 6630. Springer (Lecture Notes in Computer

Science, pp. 212 32).

Yang, X.S., 2011d. Bat algorithm for multi-objective optimisation. Int. J. Bio-Inspired

Comput. 3 (5), 267�274.

Yang, X.S., 2011e. Metaheuristic optimization. Scholarpedia. 6 (8), 11472, http://www

.scholarpedia.org/article/Metaheuristic_Optimization. doi:10.4249/scholarpedia.11472.

Yang, X.S., Deb, S., 2009. Cuckoo search via Levy flights. In: Proceedings of World

Congress on Nature and Biologically Inspired Computing (NaBic 2009), IEEE

Publications, USA, pp. 2102 214. doi:10.1109/NABIC.2009.5393690.

Yang, X.S., Deb, S., 2010. Engineering optimization by cuckoo search. Int. J. Math.

Modelling Num. Optimisation. 1 (4), 330�343.

Yang, X.S., Gandomi, A.H., 2012. Bat algorithm: a novel approach for global engineering

optimization. Eng. Comput. 29 (5), 464�483.

Yang, X.S., Sadat, H.S.S., Gandomi, A.H., 2012. Firefly algorithm for solving non-convex

economic dispatch problems with valve loading effect. Appl. Soft Comput. 12 (3),

1180�1186.

Zadeh, L.A., 1965. Fuzzy sets. Inf. Control. 8, 338�353.


metaheuristic applications in structures and infrastructures || metaheuristic algorithms in modeling...

Documents