Modern Method of OptimizationIng. Jesus Ixta Morales

September 23, 2015

1 IntroductionOptimization is the act of obtaining the best result under given circumstances. In design,construction, and maintenance of any engineering system, engineers have to take many tech-nological and managerial decisions at several stages. The ultimate goal of all such decisionsis either to minimize the effort required or to maximize the desired benefit. Since the effortrequired or the benefit desired in any practical situation can be expressed as a function ofcertain decision variables, optimization can be defined as the process of finding the conditionsthat give the maximum or minimum value of a function. It can be seen from Figure(1) thatif a point x corresponds to the minimum value of function f(x), the same point also corre-sponds to the maximum value of the negative of the function, f(x). Thus without loss ofgenerality, optimization can be taken to mean minimization since the maximum of a functioncan be found by seeking the minimum of the negative of the same function. In addition, thefollowing operations on the objective function will not change the optimum solution x (seeFigure(2)).

1. Multiplication (or division) of f(x) by positive constant c.

2. Addition (or subtraction) of a positive constant c to (or from) f(x).

There is no single method available for solving all optimization problems efficiently. Hencea number of optimization methods have been developed for solving different types of optimiza-tion problems. The optimum seeking methods are also known as mathematical programmingtechniques and are generally studied as a part of operations research. Operations research isa branch of mathematics concerned with the application of scientific methods and techniquesto decision making problems and with establishing the best or optimal solutions. The begin-nings of the subject of operations research can be traced to the early period of World War II.During the war, the British military faced the problem of allocating very scarce and limitedresources (such as fighter airplanes, radars, and submarines) to several activities (deploymentto numerous targets and destinations). Because there were no systematic methods availableto solve resource allocation problems, the military called upon a team of mathematicians todevelop methods for solving the problem in a scientific manner. The methods developed bythe team were instrumental in the winning of the Air Battle by Britain. These methods, suchas linear programming, which were developed as a result of research on (military) operations,subsequently became known as the methods of operations research.

1

Figure 1. Minimum of f(x) is same as maximum of f(x)

Figure 2. Optimum solution of cf(x) or c+ f(x) same as that of f(x)

Table1 lists various mathematical programming techniques together with other well-defined areas of operations research. The classification given in Table 1.1 is not unique;it is given mainly for convenience.

2

Mathematical programming techniques are useful in finding the minimum of a functionof several variables under a prescribed set of constraints. Stochastic process techniques canbe used to analyze problems described by a set of random variables having known proba-bility distributions. Statistical methods enable one to analyze the experimental data andbuild empirical models to obtain the most accurate representation of the physical situation.This book deals with the theory and application of mathematical programming techniquessuitable for the solution of engineering design problems.

Tabla 1. Methods of Operations Research

Mathematical programming oroptimization techniques

Stochastic process tech-niques

Statistical method

Calculus methods Statistical decision theory Regression analysisCalculus of variations Markov processes Cluster analysis, patternNonlinear programming Queueing theory recognitionGeometric programming Renewal theory Design of experimentsQuadratic programming Simulation methods Discriminate analysisLinear programming Reliability theory (factor analysis)Dynamic programmingInteger programmingStochastic programmingSeparable programmingMultiobjective programmingNetwork methods: CPM andPERTGame theory

Modern or nontraditional opti-mization techniques

Genetic algorithmsSimulated annealingAnt colony optimizationParticle swarm optimizationNeural networksFuzzy optimization

2 Modern Method of Optimization.Modern Method of Optimization. The modern optimization methods, also sometimescalled nontraditional methods, have emerged as powerful and popular methods for solving

3

complex engineering optimization problems in recent years. These methods include genetic al-gorithms, simulated annealing, particle swarm optimization, ant colony optimization, neuralnetwork-based optimization, and fuzzy optimization. The genetic algorithms are comput-erized search and optimization algorithms based on the mechanics of natural genetics andnatural selection. The genetic algorithms were originally proposed by John Holland in 1975.The simulated annealing method is based on the mechanics of the cooling process of moltenmetals through annealing. The method was originally developed by Kirkpatrick, Gelatt, andVecchi.

The particle swarm optimization algorithm mimics the behavior of social organisms suchas a colony or swarm of insects (for example, ants, termites, bees, and wasps), a flock ofbirds, and a school of fish. The algorithm was originally proposed by Kennedy and Eberhartin 1995. The ant colony optimization is based on the cooperative behavior of ant colonies,which are able to find the shortest path from their nest to a food source. The method wasfirst developed by Marco Dorigo in 1992. The neural network methods are based on theimmense computational power of the nervous system to solve perceptional problems in thepresence of massive amount of sensory data through its parallel processing capability. Themethod was originally used for optimization by Hopfield and Tank in 1985. The fuzzy op-timization methods were developed to solve optimization problems involving design data,objective function, and constraints stated in imprecise form involving vague and linguisticdescriptions. The fuzzy approaches for single and multiobjective optimization in engineeringdesign were first presented by Rao in 1986.

3 Engineering applications of optimizationOptimization, in its broadest sense, can be applied to solve any engineering problem. Sometypical applications from different engineering disciplines indicate the wide scope of the sub-ject:

1. Design of aircraft and aerospace structures for minimum weight

2. Finding the optimal trajectories of space vehicles

3. Design of civil engineering structures such as frames, foundations, bridges, towers,chimneys, and dams for minimum cost

4. Minimum-weight design of structures for earthquake, wind, and other types of randomloading

5. Design of water resources systems for maximum benefit

6. Optimal plastic design of structures

7. Optimum design of linkages, cams, gears, machine tools, and other mechanical compo-nents

4

8. Selection of machining conditions in metal-cutting processes for minimum productioncost

9. Design of material handling equipment, such as conveyors, trucks, and cranes, forminimum cost

10. Design of pumps, turbines, and heat transfer equipment for maximum efficiency

11. Optimum design of electrical machinery such as motors, generators, and transformers

12. Optimum design of electrical networks

13. Shortest route taken by a salesperson visiting various cities during one tour

14. Optimal production planning, controlling, and scheduling

15. Analysis of statistical data and building empirical models from experimental results toobtain the most accurate representation of the physical phenomenon

16. Optimum design of chemical processing equipment and plants

17. Design of optimum pipeline networks for process industries

18. Selection of a site for an industry

19. Planning of maintenance and replacement of equipment to reduce operating costs

20. Inventory control

21. Allocation of resources or services among several activities to maximize the benefit

22. Controlling the waiting and idle times and queueing in production lines to reduce thecosts

23. Planning the best strategy to obtain maximum profit in the presence of a competitor

24. Optimum design of control systems

4 Statement of an optimization problemAn optimization or a mathematical programming problem can be stated as follows.

find X =

x1x2...xn

which minimizes f(X)

5

gj(x) 0, j = 1, 2, ...,mlj(x) = 0, j = 1, 2, ..., p

subject to the constraints

where X is an n-dimensional vector called the design vector, f(X) is termed the objectivefunction, and gj(X) and lj(X) are known as inequality and equality constraints, respectively.The number of variables n and the number of constraints m and/or p need not be relatedin any way. The problem stated in Eq. (1.1) is called a constrained optimization problem.Some optimization problems do not involve any constraints and can be stated as

find X =

x1x2...xn

which minimizes f(X)

Such problems are called unconstrained optimization problems.

5 Classification of optimization problemsOptimization problems can be classified in several ways, as described below. Classification Based on the Existence of Constraints: As indicated earlier, any

optimization problem can be classified as constrained or unconstrained, depending onwhether constraints exist in the problem.

Classification Based on the Nature of the Design Variables: Based on thenature of design variables encountered, optimization problems can be classified intotwo broad categories. In the first category, the problem is to find values to a set ofdesign parameters that make some prescribed function of these parameters minimumsubject to certain constraints.

Classification Based on the Physical Structure of the Problem: Dependingon the physical structure of the problem, optimization problems can be classified asoptimal control and nonoptimal control problems.

Classification Based on the Nature of the Equations Involved: Another impor-tant classification of optimization problems is based on the nature of expressions for theobjective function and the constraints. According to this classification, optimizationproblems can be classified as linear, nonlinear, geometric, and quadratic programmingproblems. This classification is extremely useful from the computational point of viewsince there are many special methods available for the efficient solution of a particularclass of problems. Thus the first task of a designer would be to investigate the class ofproblem encountered. This will, in many cases, dictate the types of solution proceduresto be adopted in solving the problem.

6

Classification Based on the Permissible Values of the Design Variables: De-pending on the values permitted for the design variables, optimization problems canbe classified as integer and real-valued programming problems.

Classification Based on the Deterministic Nature of the Variables: Basedon the deterministic nature of the variables involved, optimization problems can beclassified as deterministic and stochastic programming problems.

Classification Based on the Separability of the Functions: Optimization prob-lems can be classified as separable and nonseparable programming problems based onthe separability of the objective and constraint functions.

Classification Based on the Number of Objective Functions: Depending on thenumber of objective functions to be minimized, optimization problems can be classifiedas single- and multiobjective programming problems.

5.1 Examples:5.1.1 Classification Based on the Existence of Constraints

min x2 + y2x+ y = 3

L(x, y, ) = x2 + y2 + (3 x y)

Terms:

5~xL(x, y, ) = (2x , 2y ) = (0, 0)L

(x, y, ) = 3 x y = 0

Initial condition L

= 0

H~xL(x, y, ) =[2 00 2

]positive definite

Result[32

32

]5.1.2 Classification Based on the Nature of the Design Variables

Classification based on the nature of the design variables (first category: the objective is tofind a set of design parameters that make a prescribed function of these parameters minimumor maximum subject to certain constraints.) for example to find the minimum weight design

7

Figure 3. Estructure

of a strip footing with two loads shown in the figure, subject to a limitation on the maximumsettlement of the structure.

the problem can be defined as follows

Find X =[bd

]which minimizes

f(X) = h(b, d)

subject to the constraints

s(X) 6 max

b 1 0

d 1 0

The length of the footing(L) the loads P1 and P2, the distance between the loads are assumedto be constant and the required optimization is achieved by varying b and d. Such problemsare called parameter or static optimization problems.

5.1.3 Classification Based on the Nature of the Equations Involved

Linear Programming.

Problem the objective function and constraints are linear.

For a couple of sessions with a carpenter (our client), this tells us that only manufac-tures furniture and sells all the tables and chairs that makes a market. However, it hasa steady income and want to optimize this situation.

The goal is to determine how many tables and should make to maximize net incomechairs. We began focusing on a time horizon, that is, within planning to revise our

8

solution weekly if necessary. To learn more about this issue , we must go to businesscarpenter and observe what happens and measure what we need to develop (to model)of your problem. We confirm that your goal is to maximize net income. We mustcommunicate with the client.

Carpenter The problem is to determine how many tables and chairs must be manufac-tured per week; but first must establish an objective function.

The objective function is 5x1+3x2, where x1 and x2 they represent the number of tablesand chairs; 5 and 3 represent net income (for example, in dollars or tens of dollars)from the sale of a table and chair, respectively. The limiting factors that typically comefrom outside, are the limitations of labor (this limitation comes from the family of thecarpenter) and raw material resources (this limitation comes from scheduled delivery).

Production time required for a table and a chair at different times of day are measuredand calculated in 2 hours and 1 hour, respectively. The total working hours per weekare only 40. The raw material required for a table and a chair is 1 and 2 units, re-spectively. The total supply of raw material is 50 units per week . Accordingly, theformulation of PL is:

Maximize 5x1 + 3x2

Subject to:2x1 + x2 6 40 restriction of laborx1 + 2x2 6 50 restriction of materialsboth x1 and x2 are non negative.

This is a mathematical model for the problem Carpenter. The decision variables, thatis to say, the controllable inputs are x1 and x2. The output or result of this model aretotal net revenues 5x1 + 3x2. All functions used in this model are linear (the decisionvariables are raised to the first power). The ratio of these restrictions is called techno-logical factors (matrix). The review period is one week, a review period is one week, an appropriate period within which it is less likely to change (fluctuate) controllableinputs (all parameters such as 5,50,2 , ...). Even within such a short planning, weperform scenario analysis to respond to any changes in these inputs in order to controlthe problem, that is to say, update the prescribed solution.

Note that since the carpenter is not going to go bankrupt at the end of the planningperiod , we added the conditions that both x1 and x2 must be non-negative ratherthan the requirements x1 and x2 must be positive integers. remember that no nega-tivity conditions are also called implicit restrictions. Again, a linear program officerwell for this problem if the carpenter continued manufacturing these products. Partialitems simply be counted as work in progress and eventually be transformed into finished

9

products, in the following week.

We can try to solve x1 and x2 by listing possible solutions for each and selecting thepair (x1, x2) that maximizes 5x1 + 3x2 (net income). However, it takes a long time tolist all the possible alternatives and if all the options are not listed, we can not be surethat the selected pair (as a solution) is the best of all alternatives. Other (more efficientand effective), preferred methodologies known as the techniques of linear programmingsolutions.

The optimal solution, that is to say, the optimal strategy is to set x1 = x2 = 10 tablesand 20 chairs. We schedule activities weekly carpenter to manufacture 10 tables and20 chairs. With this strategy (optimal), net income is $ 110. This prescribed solutioncarpenter surprised because due to higher net income from the sale of a table ($ 5), heused to make more tables than chairs.

hire or not hire an assistant? Suppose the carpenter could hire a helper at a cost of US$ 2 per hour (additional $ 2) suits the carpenter hire an assistant? if so, for how manyhours?

x3 is the amount of overtime, then the modified problem is:

maximice 5x1 + 3x2 2x3

subject to: 2x1 + x2 6 40 + x3 restriction of labor with overtime unknown.

x1 + 2X2 6 50 restriction of materials

in this new condition, we see that the optimal solution is x1 = 50, x2 = 0, x3 = 60,with optimal net income of US $ 130. Therefore, the carpenter should hire an assistantfor 60 hours.

Geometric problem

Find the economic order quantity of a product, that is to say, it decides which itemquantity should store periodically; the total costs associated with the product and itsstorage can be expressed CT = CCI + CHP + V C where:

CT = Qh2 +adQ

+ dQ

(P + kQ)

Where:

10

CT Total costCCI Loaded costCHP Total cost of orderVC Purchase priceQ Economic order quantityh Annual storage cost per unita I cost of orderingd Average consumption per yeark,P Constant

The objective function has the following general formula:

Min CT = C1Q+C2Q

later

1 + 2 = 01 2 = 0

So that to solve the above system of simultaneous equations we reach that 1 = 2 andthe variable Q must be such that the two terms of the objective function are the same:

C1Q =C2Q

Q = 2C2C1

C1 =h

2 , C2 = d(a+ p)

Q = 2

2d(a+ p)h

CT

Q= C1 C2

Q2;

2CT

Q2= 2C

2

Q3> (Minimo).

Contribution of the solution methods for linear programming problems not alreadymentioned some of the known are:

Dimensional search techniques: Minimax, simultaneous search: two experiments,simultaneous search: n experiments resolution, distinctness, scaling, sequentialsearch method Bolzano, block search, search couple blocks dichotomous search,Fibonacci search, search unknown resolution, golden section search, Fibonaccisearch and reverse search using odd blocks, among others.

11

Search multidimensional techniques: some models are multivariable Elimination,geometric methods, logical methods, random search, methods of stochastic ap-proximation search form mesh pattern search method: Hooke-Jeeves, quadraticinterpolation method of Powell method accelerated promotion, method of Newton-Raphson method Davidon-Fletcher-Powell method Broyden-Fletcher, method ofFletcher-Reeves method of Smith.

Other methods: method of Levenberg-Marquardt, Quasi-Newton, conjugate gra-dient, subgradient, Zoutendijk, successive linear programming (PSL), successivequadratic programming (PSC), Rosen, Zangwill and unrestricted technique se-quential minimization (SUMT), among others.

Quadratic problem

Solve the following quadratic programming problem by the method of Wolfe:Max Z = 10X1 + 25X2 10X21 X22 4X1X2with restriction:

X1 + 2X2 6 10X1 +X2 6 9X1X2 > 10

Applying the Lagrange multipliers have:

F ( ~X,~, ~) = 10X1 + 25X2 10X21 X22 4X1X21(X1 + 2X2 10)2(X1 +X29) 1(X1) 2(X2)

The first partial derivatives are:

F

X= 10 20X 4X2 1 2 + 1 = 0

F

X= 25 2X2 4X121 2 + 2 = 0

The problem equivalent to the original linear programming method according to Wolfeis:

Min W = V1 + V2

subjet to:

12

20X1 + 4X2 + 1 + 2 1 + V1 = 104X1 + 2X2 + 21 + 2 2 + V2 = 25X1 + 2X2 + Y1 = 10X1 +X2 + Y2 = 9

With the following restrictions complementary slackness:

X11 = 0X22 = 01Y1 = 02Y2 = 0

Using the Simplex method is that the initial basic solution is:

W = 35; V1 = 10; V2 = 25; Y1 = 10; Y1 = 9

In the first iteration enters X2(2 = 0) and leaves X1 (It is clear that although theSimplex choose 1 y 2 to enter the base before it occurs X2, 1, 2 they are not ac-ceptable because Y1 y Y2 It is positive). The end point is recalculated after:

W = 20; X2 = 52; V2 = 20; Y1 = 5; Y2 =132

In the third iteration can not enter the base 1 y 2 y Y 1 y Y 2 they are positive;Simplex taken as the next candidate 1 and output Y1; the endpoint after iterate is:

W = 15; X2 = 5; V2 = 15; 1 = 10; Y2 = 4

In the last iteration (V1 = 0 y V2 = 0) you must go X1 but can not because 1 It ispositive ; the next item to enter the base is 1 which replaces V2 after recalculating(pivoting) end point is:

W = 0; X2 = 5; 1 =152 ; 1 =

352 ; Y1 = 4

The above solution corresponds to the optimum:

X1 = 0; X2 = 5; Z = 100

13

5.1.4 Classification Based on the Permissible Values of the Design Variables

Some company produces article formed with four component A and three pieces of B.The parts can be manufactured in any of three different machines that owns the companywhich transforms the two raw materials in the parts that go to final product assembly .The following table shows the number of grams of each raw material to be used in each ma-chine to cycle production of components. The same table shows the number of componentsof each type obtained in each production cycle of each of the machines as well as the numberof grams of raw materials available .

Machine M.P. 1(g/cicle) M.P. 2(g/cicle) Component A(u/cicle) Component A(u/cicle)1 8 6 7 52 5 9 6 93 3 8 8 4

Avalible 100 200

it should be programmed for maximum production quantity of the items ? Model buildingfor a better understanding we draw a diagram of the situation.

Definition of variablesxi = number of production batches making the machine i.Each batch of production machines uses certain amount of raw materials and produces acertain amount of components A and B , with which the assembly of the final product isobtained.

As for each unit of the assembly four units of component A and three component B areused, it is concluded that the total number of assemblies obtained will be the result of di-viding by four the number of components type A, but also must equal the number of type Bcomponents, divided by three.

Then you also need to define:

XA = number of components of type A obtained.XB = number of type B components obtained.

14

Maximize: number of assemblies = XA4The restrictions are of two kinds:

Related resources or commodities.

8X1 + 5X2 + 3X3 6 100 grams of mp16X1 + 9X2 + 8X3 6 200 grams of mp2

Related to the total amount of each component.

XA = 7X1 + 6X2 + 8X3 units of component A.XB = 5X1 + 9X2 + 4X3 units of component B.

5.1.5 Classification Based on the Deterministic Nature of the Variables

The stochastic program is considered feasible set with g(x) > ~,where:x Rn, g(x) = (g1(x)), (g2(x)), ..., (gm(x))T

It contains no random element and = (1, 2, , m) It is a random vector of dimen-sion m. In this case for p [0, 1] It must be the feasible set of deterministic equivalent torestrictions joint chance is

C(p) = {x Rn | P (g(x) > ) > p} ={x Rn | F~(g(x)) > p

},

where F~ it is the distribution function of the random vector .

Ci(pi) ={x Rn | F~(gi(x)) > pi

}= {x Rn | gi(x) > i},

where i =1F~

(pi).

5.1.6 Classification Based on the Separability of the Functions

Below we solve a problem of separable programming using the method of the restricted base.

Max Z = X1 +X2

with your restriction

15

3X1 + 2XX22 6 9X1, X2 > 0

approximation method suggests that the separable variables are:

K ak2 f2(ak2) g21(ak2)1 0 0 02 1 1 23 2 16 84 3 81 18

later:f2(X2) T2f2(a2) + T 22 f2(a22) + T 32 f2(a32) + T 42 f2(a42)

f2(X2) 0(T2) + 1(T 22 ) + 16(T 32 ) + 81(T 42 )f2(X2) (T 22 ) + 16(T 23 ) + 81(T 42 )g21(X2) 2(T 22 ) + 8(T 32 ) + 18(T 42 )

then the original problem by approximation becomes:

Max Z = X1 + T 22 + 16T 32 + 81T 42

subject to:

Z = X1 + 2T 22 + 8T 32 + 18T 42 6 9T 12 + T 22 + T 32 + T 42 = 1T k2 > 0, K = 1, 2, 3, 4X1 > 0

the initial board simplex corresponds to:

where s1 is a slack variable (padding).The optimal solution by the simplex equivalent to this problem is:

16

T 32 =910; T

42 =

110; y Z

= 452 then the optimum in terms of X1 and X2x is:

X1 = 0; X2 2T 32 + 3T 42 ; X2 2{ 9

10

}+ 3

{ 110

}; X2 2, 1; Z =

452

5.1.7 Classification Based on the Number of Objective Functions

17

References

18

Contents1 Introduction 1

2 Modern Method of Optimization. 3

3 Engineering applications of optimization 4

4 Statement of an optimization problem 5

5 Classification of optimization problems 65.1 Examples: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

5.1.1 Classification Based on the Existence of Constraints . . . . . . . . . . 75.1.2 Classification Based on the Nature of the Design Variables . . . . . . 75.1.3 Classification Based on the Nature of the Equations Involved . . . . . 85.1.4 Classification Based on the Permissible Values of the Design Variables 145.1.5 Classification Based on the Deterministic Nature of the Variables . . 155.1.6 Classification Based on the Separability of the Functions . . . . . . . 155.1.7 Classification Based on the Number of Objective Functions . . . . . . 17

19