mdp 631 industrial operations research lecture 9...mdp 631 industrial operations research lecture 9...

MDP 631Industrial Operations Research

Lecture 9

MIP and NLP Models

Today’s lecture

• Sample MIP models

– The multi-period product mix problem

– The single machine scheduling problem

• Standard IP models

• Computational complexity of IP problems

• Non-linear programming

• Separable programming

• Using LINGO for solving non-linear programming problems

Example 1: multi-period product mix problem

It is required to determine the production plan for the next

l weeks for m products. Each product i requires its own

setup operation with weekly setup cost of si, and each unit

costs ci to be produced. There are n different resources

that are utilized in the production of all products. Each

product i consumes qij units from resource j. The

available quantity of resource j in week t is Qjt. The total

demand requirement for each product, Di , must be

satisfied by the end of the l weeks.

The MIP model

Elements of the MIP model:

1) sets:

P : Set of products = {1,2,3…m }T : Set of weeks = {1,2,3…l }R : Set of resources = {1,2,3…n }

2) Parameters:

si : setup cost for product ici : production cost of product iqij : quantity required by product i from resource jQjt : quantity of resource j available in week tDi : total demand requirement of product i

The MIP model

Elements of the MIP model (cont.):

3) Decision variables:xit : quantity to be produced of product i in week t

yit : a binary variable that equals 1 if product i is produced in week t and zero otherwise

4) Objective function:

( )i it i it

t T i P

c x s y

+Minimize total cost Z =

5) Constraints:

Resources limits: andij it jt

i P

q x Q j R t T

Demand satisfaction: it i

t T

x D i P

=

andit i itx D y i P t T x and y relationship:

The MIP model

Domain constraints: 0 anditx i P t T

{0,1} andity i P t T

x-y relationship constraints

andit i itx D y i P t T

x and y relationship constraints:

We can see that if yit = 0, xit will be forced to be zero due to the inequality defined in these constraints.

At the same time if xit needs to have a value greater than zero, the corresponding yit will take the value of 1, and hence the setup cost will be incurred.

The last case that may occur is that yit take the value of 1, while xit is zero. We can see that as the objective function contains the minimization of a positive coefficient multiplied by yit, then yit will be forced to be zero in this case.

x-y relationship constraints

andit i itx D y i P t T

multi-period product mix problem: a case

Number of products = m = 6Number of weeks = l = 5Number of resources = n = 3

Product

1 2 3 4 5 6

si 10 15 12 10 8 5

ci 0.1 0.15 0.05 0.08 0.18 0.2

Di 1000 550 620 430 500 1200

multi-period product mix problem: a case

qij

Product

1 2 3 4 5 6

Reso

urce

1 0.1 0.2 0.2 0.15 0.25 0.05

2 1.2 1.5 0.9 0.7 1.8 2.5

3 2.1 3.2 4.5 2 1.5 0.5

Qjt

Period

1 2 3 4 5

Reso

urce

1 100 240 130 80 120

2 1500 1200 1800 2000 1200

3 2000 1800 1500 2200 2000

LINGO modelMODEL:

SETS:

P / 1..6 /: s, c, D;

T / 1..5 /;

R / 1..3 /;

PXR( P, R): q;

PXT( P, T): x, y;

RXT( R, T): _Q;

ENDSETS

DATA:

s c D =

10 0.1 1000

15 0.15 550

12 0.05 620

10 0.08 430

8 0.18 500

5 0.2 1200;

q = 0.1 1.2 2.1

0.2 1.5 3.2

0.2 0.9 4.5

0.15 0.7 2.0

0.25 1.8 1.5

0.05 2.5 0.5;

_Q = 100 240 130 80 120

1500 1200 1800 2000 1200

2000 1800 1500 2200 2000;

ENDDATA

LINGO model (cont.)! Objective function, minimize total production and setup costs;

MIN = @SUM( T(_t): @SUM( P(_i): c(_i) * x(_i, _t))) +

@SUM( T(_t): @SUM( P(_i): s(_i) * y(_i, _t)));

! Resources limits;

@FOR( R( _j):

@FOR( T( _t):

@SUM( P( _i): q(_i, _j) * x(_i, _t)) <= _Q(_j, _t)

)

);

! Demand satisfaction;

@FOR( P( _i):

@SUM( T( _t): x(_i, _t)) = D(_i)

);

! x-y relationship;

@FOR( T( _t):

@FOR( P( _i): x(_i, _t) <= D(_i) * y(_i, _t)

)

);

@FOR( PXT(_i, _t): @BIN( y(_i, _t)));

END

Extension to example 1

• As an extension to example 1, consider a situation in which product 1 should be produced whenever product 2 is produced due to technological considerations.

• We know that the binary variable y2t equals 1 if x2t > 0 and zero otherwise. Then by adding the following constraint, we satisfy the above condition:

1 20.0001t tx y t T

Example 2: Single machine scheduling

• In the single machine scheduling problem, there is a set of jobs waiting to be processed on a single machine, and the processing time of job i is denoted pi.

• The machine can process only one job at a time.

• It is required to determine the processing sequence of jobs such that the average completion time is minimized.

• Whenever a job starts its processing, it cannot be stopped and the machine is assumed to be continuously available without breakdown.

The MIP model

Elements of the MIP model:

1) sets:J : Set of jobs = {1,2,3…n }

2) Parameters:pi : processing time of job i

3) Decision variables:xij : a binary variable that equals 1 if job i is processed

before job j, and zero otherwisesi : start time of job i

4) Objective function:

Minimize mean completion time Z =

( )i i

i J

s p

n

+

The MIP model

5) constraints:

Processing sequence:

(1 ) , wherej i i ijs s p M x i j J j i− − −

, wherei j j ijs s p Mx i j J j i− −

Domain constraints:

{0,1} , whereijx i j J j i *

is i J

These two constraints form whatis known asdisjunctive constraints

Combinatorial explosion: In this example, we will have a total number of n(n-1) disjunctive constraints, and n(n-1)/2 binary variables. For each binary variable, we have two possible values, leading to a total of 2n(n-1)/2

possible combinations.

Single machine scheduling: a case

• Consider a case in which n = 10 and the processing times are given as:

Job (i)

1 2 3 4 5 6 7 8 9 10

pi 12 4 8 15 7 3 6 4 9 7

LINGO modelMODEL:

SETS:

J / 1..10 /: s, p;

JXJ( J, J) | &2 #GT# &1: x;

ENDSETS

DATA:

p = 12 4 8 15 7 3 6 4 9 7;

M = 1000;

ENDDATA

! Objective function, minimize the average completion time;

MIN = @SUM( J(_i): s(_i) + p(_i)) / 10;

! Disjunctive constraints;

@FOR( JXJ( _i, _j):

s(_j) - s(_i) >= p(_i) - M * (1 - x(_i, _j));

s(_i) - s(_j) >= p(_j) - M * x(_i, _j);

);

! Domain constraints;

@FOR( JXJ(_i, _j): @BIN( x(_i, _j)));

END

A note on the solution method

• For the single machine scheduling problem, using LINGO to solve its MIP model is not the best way to solve this problem.

• The problem of minimizing the average completion time can be solved easily by sorting the operations in a non-decreasing order of their processing times.

• As an exercise, solve the given model using LINGO and compare the final solution with the solution method just mentioned.

Standard IP models

• There is a set of well-known and frequently encountered IP models that can appear as sub-problems in many applications. This set includes:

– Set covering problem

– Set packing problem

– Set partitioning problem

– Knapsack problem

– Assignment problem

– Travelling salesman problem

– Quadratic assignment problem

Set covering problem

Given a set of objects S = {O1, O2, …, Om},

set P of items where each item is a subset of S,

i.e. P = { Ij: Ij S, j = 1,…, n }, and a cost Cj is

associated with each item Ij P. The objective is to

select a group of items from set P, such that all the

items in set S are covered with the minimum

associated cost.

Binary variables are used to represent the decision of

whether the jth member of set P is included or not.


=

n

j

jj xC1

11

=

n

j

j

i

j xU

i

jU

Min.

s.t.

xj = 0 or 1 j = 1, …, n

is a constant that equals 1 if object Oi Ij and

zero otherwise.

i = 1, …, m


Let S = {1, 2, 3, 4, 5} and P = {{1, 3}, {2, 3, 4}, {4, 5}, {3, 5}, {1, 2}},

and the costs associated with each item in set P are 2, 5, 3, 2, 3

respectively. The corresponding IP model is formulated as follows:

Decision variables:

x1 = 1 if subset {1, 3} is included; 0 otherwise.

x2 = 1 if subset {2, 3, 4} is included; 0 otherwise.




Min. 2 x1 + 5 x2 + 3 x3 + 2 x4 + 3 x5

s.t. x1 + x5 1

x2 + + x5 1

x1 + x2 + x4 1

x2 + x3 + 1

x3 + x4 1

SOLUTION: x3 = x4 = x5 = 1, x1 = x2 = 0.

Set packing problem

Given a set of objects S = {O1, O2, …, Om},


i.e. P = { Ij: Ij S, j = 1,…, n }, and a gain value Vj is

associated with each item Ij P.

The objective is to select a group of items from set P,

such that the total gain value of the items included

(packed) is maximized and objects from set S are

considered at most once.

Binary variables are used to represent the decision of

whether the jth member of set P is included or not.

Set packing problem

1

n

j j

j

V x=

1

1n

i

j j

j

U x=

i

jU

Max.

s.t.

xj = 0 or 1 j = 1, …, n


zero otherwise.

i = 1, …, m

Set packing problem

Let S = {1, 2, 3, 4, 5} and P = {{1, 3}, {2, 3, 4}, {4, 5}, {3, 5}, {1, 2}},

and the costs associated with each item in set P are 2, 5, 3, 2, 3

respectively. The corresponding IP model is formulated as follows:

Decision variables:


x2 = 1 if subset {2, 3, 4} is included; 0 otherwise.




Max. 2 x1 + 5 x2 + 3 x3 + 2 x4 + 3 x5

s.t. x1 + x5 ≤ 1

x2 + + x5 ≤ 1

x1 + x2 + x4 ≤ 1

x2 + x3 + ≤ 1

x3 + x4 ≤ 1

SOLUTION: x3 = x5 = 1, x1 = x2 = x4 = 0.

Set partitioning problem

This problem can be considered as a combination of previous

set covering and set packing problems.

Here we again have a set of objects S = {O1, O2, …, Om} and


i.e. P = { Ij: Ij S, j = 1,…, n }.

We want to decide which items of set P will be included such

that no overlap of objects occurs. In order to represent the no-

overlap condition, we use equality constraints.

Set partitioning problem

1

n

j j

j

V x=

1

1n

i

j j

j

U x=

=

i

jU

Max. Or Min.

s.t.

xj = 0 or 1 j = 1, …, n


zero otherwise.

i = 1, …, m

=

n

j

jj xC1

Note: Set packing and set partitioning problems can be transformed from one to another by introducing binary slack variables.

Knapsack problem

In the knapsack problem, we have n items with specific value (pi) and

weight (wi) for each. We want to decide how many or how much to

include from each item such that the total value is maximized and the

total weight limit (W) is not exceeded. This problem is represented as

follows:

=

n

i

ii xp1

Wxwn

i

ji =1

Max.

s.t.

where xi can be continuous or integer values

The 0-1 knapsack problem has the same definition of the knapsack problem,

but with the constraint that all xi values are binary.

Assignment problem

The assignment problem is concerned with assigning n people to n tasks.

There is exactly one person for each task and if person i is assigned to task

j, a cost cij will be incurred. The objective is to minimize the cost of

assignment.

In this problem a binary decision variable xij = 1 if person i does task j and

zero otherwise.

= =

n

i

n

j

ijij xc1 1

11

==

n

j

ijx

1

1n

ij

i

x=

=

Min.

s.t. for i = 1,…, n

for j = 1,…, n

Travelling salesman problem (TSP)

A salesman must visit each of n cities exactly once and then

return to his starting point. The cost of traveling from city i to

city j is cij. It is required to find the order in which he should

make his tour in order to minimize the total travel cost.

To model this problem, a binary decision variable xij is used

to represent the decision that the salesman will travel from

city i to city j or not. The model is very similar to the

assignment problem, except with the addition of what is

called subtour elimination constraints.

Travelling salesman problem (TSP)

Quadratic assignment problem

This problem arises in the application of facility layout problem.

We have a set of n machines and a set of n locations. It is

required to assign machines to locations in a similar manner as

we did in the assignment problem, but we have a different cost

structure.

A cost cijkl will be incurred if machine i is assigned to location j

and machine k is assigned to location l. Accordingly the IP

formulation is done by using a binary variable xij the same way

we did in the assignment problem to represent the decision of

assigning machine i to location j or not.

Quadratic assignment problem

Computational complexity of IP problems

• If the size of an instance of a certain problem is denoted by n, the worst case complexity function will be expressed as a function of n denoted by f(n).

• This function may have different forms depending on the used algorithm, it can be polynomial function, exponential function or either logarithmic.

• This function can be defined by studying the basic operations involved in an algorithm, such as comparisons, mathematical operations and functions, and conditional operations.


• The order of this function is denoted by the Big-Oh notation, for example If f(n) = n3-2n+1, the order of the corresponding algorithm is denoted by O(n3).

• The order of the worst case complexity function is taken as a major characteristic of the algorithm as it has the significant effect for the function at higher problem sizes.


• An algorithm is called a polynomial algorithm or polynomial-time algorithm if its worst-case complexity f(n) is O(nk) for some fixed positive integer k.

• Thus an algorithm with complexity n log n is a polynomial algorithm because n log n is O(n2).

• An algorithm whose complexity violates all polynomial bounds is referred to as an exponential algorithm.

• Some examples of rates of growth of exponential algorithms are kn (k > 1), n!, nn and nlog n


• Problems have been categorized into classes according their complexity levels.

• A problem belongs to class P if there is a polynomial algorithm to solve every instance of it.

• A problem that is found to have at least one exponential algorithm to solve it, but no one has proved so far that every algorithm to solve it is necessarily exponential, belongs to class NP.

• The acronym NP is for “non-deterministic polynomial”.


• So, for any problem that belongs to the class NP, there are three mutually exclusive alternatives: – a polynomial algorithm for it will be discovered

– it will be proved that it will not have a polynomial algorithm

– the status of it will never be settled

• The easy problems in NP are in P.

• Since it is not known whether P = NP or not, it is natural to collect all the “hard” problems of NP in one class.

• Donald Knuth of Stanford University and his colleagues named this class as the NP-complete or NP-hard class.


• The following IP problems have been proven to be NP-hard problems:

– Set covering problem

– Set partitioning problem

– Set packing problem

– Knapsack problem

– Traveling salesman problem (TSP)

– Quadratic assignment problem (QAP)

Non-linear programming (NLP)

• In linear programming models, all the functions that constitute the objective function and constraints are assumed to be linear.

• In many applications this assumption may not hold.

• For instance it’s commonly known by economists that the price-demand relationship is linear. When the total revenue is calculated as TR = price demand, the total revenue function becomes non-linear function of the demand as

TR = (a-bD)D = aD – bD2.

Non-linear programming (NLP)

• It is very common in engineering applications to have non-linear objective function and/or constraints.

• If at least one of the relationships in the model is nonlinear with respect to the variables, the mathematical programming model is said to be non-linear programming (NLP).

• A NLP is said to be integer non-linear programming (INLP) if a subset of the variables has integer restrictions.

DefinitionsConvex set: a set S in En is said to be convex if the line segment joining any two points of the set also belongs to the set. i.e if x1

and x2 belong to S then x1 + (1-) x2 belongs to S for all [0, 1].

Convex sets Non-convex sets

Definitions

Convex function: Let f: S → E1, where S is a nonempty convex set in En.

The function f is said to be convex on S if:

f( x1 + (1-) x2) f(x1) + (1-) f(x2) for all [0, 1]

x1 x2x1 + (1-)x2

Convex function

x1 x2x1 + (1-)x2

concave function

x1 x2

neither

Definitions

Convex programming: a mathematical programming model is said to be convex if it involves the minimization of a convex function over a convex feasible region.

Minimize x12 – 4 x1 – 2 x2

s.t. x1 + x2 42 x1 + x2 5- x1 + 4 x2 2

x1, x2 0

Example:

-9

-10

-11

Definitions

Example of a non-convex programming model:

Minimize – 4 x13 + 3 x1 – 6 x2

s.t. x1 + x2 42 x1 + x2 5- x1 + 4 x2 2

x1, x2 0

-19

-24

-32

The problem in this case is that there is no guarantee for finding the global optimal solution due to the shape of the objective function. On the other hand, the convex programming case will guarantee finding the global optimal. This makes the non-convex objective functions difficult to solve.

Non-convex programming

• The case of non-convex programming generally appears in situations where there are economies of scale.

• This condition appears in manufacturing industries as when larger lot size is ordered, the unit cost will be less.

• It also appears in transportation applications as when small quantities are transported between two locations, transportation will be more frequent resulting in higher costs compared to the case when larger trucks are used.

Separable programming

Separable function: is a function that can be expressed as the sum of functions each of a single variable.

For instance the following function is separable: x12 – 4 x1 – 2 x2

While this function is not separable: x1 x2 + x2 / (x1 + 2) + x2

The benefit of having separable functions is that they can be approximated to piecewise linear functions and hence the model can be approximated to a linear programming one.


To illustrate the idea of the piecewise linear approximation consider the following example

Minimize x12 – 4 x1 – 2 x2

s.t. x1 + x2 42 x1 + x2 5- x1 + 4 x2 2x1, x2 0

The only non-linear term is found in the objective function (x1

2). Since the decision variable x1 will not take a value greater than 2.5 or less than zero (from the feasible region defined by the set of constraints), we can only study the portion of this function from 0 to 2.5.


The following figure shows a proposed piecewise linear approximation of the objective function.

This piecewise linear approximation can be done mathematically by introducing a new variable, y, where x1 and y are related by the following equations.

x1 = 0 1 + 1 2 + 2 3 + 2.5 4

y = 0 1 + 1 2 + 4 3 + 6.25 4

1 + 2 + 3 + 4 = 11, 2, 3, 4 0

the ’s are new variables introduced to the model as well. They can be viewed as weights given to the vertices of the proposed linear segments.


The model now becomes an LP model as follows:

Minimize y – 4 x1 – 2 x2

s.t. x1 + x2 42 x1 + x2 5- x1 + 4 x2 2- x1 + 2 + 2 3 + 2.5 4 = 0- y + 2 + 4 3 + 6.25 4 = 0

1 + 2 + 3 + 4 = 1x1, x2, y, 1, 2, 3, 4 0

Separable programmingIn addition a very important condition must be added to demonstrate the fact that only one line segment must be present at a time (i.e. only one line segment 0A or AB or BC must represent the approximation but not any combination of them). Hence, the following constraint must be added:

At most two adjacent i are non zero

If this condition is not considered we could have a situation in which the line segment 0B for instance is the current representation. In this case, the optimal solution would lie on a point far from the actual curve.


In case of convex programming we need not worry about this case since the structure of the objective function and the feasible region restricts the solution to eventually lie on one of the lines.

In contrast, in the case of non-convex function this result is not guaranteed. However, this piecewise linear approximation for the non-convex functions does not guarantee reaching optimal solutions and the solution algorithm can be easily trapped in a local optima.

Non-separable functionsIn models where non-linear functions appear not to be separable, the introduction of new variables can provide a suitable basis for converting them to separable models. A commonly known case is the multiplication of two decision variables x1 x2. This can be easily converted into separable model by replacing x1 and x2 using two new variables u1 and u2

where the old and the new decision variables are related as follows:

u1 = ½ (x1 + x2)u2 = ½ (x1 - x2)

Accordingly the term x1 x2 can be replaced by u12 – u2

2

Similarly there are other forms of non-separable functions that can be transformed into a separable model by introducing new variables

Non-convex feasible regions

• In non-linear applications, non-convex feasible regions may exist due to the definition of non-linear constraint functions.

• To deal with this situation, the non-convex region is divided into a set of convex sub-regions whose “union” will result in the original non-convex feasible region.

Non-convex regions

• Each convex sub-region can be switched on and off by introducing indicator variables.

• Switching sub-regions on and off is necessary to investigate all possible corner points which may represent a possible optimal solution to the problem.

Non-convex regionsFor example, consider the non-convex region given in the following figure. The original non-convex region bounded by the dashed curves are approximated to a region bounded by straight lines as shown.

1 2 3 4 5

1

2

3

4A

B

C

D

E

FG

H

O

The non-convex feasible region OABCDEO can be represented as the union of the three convex regions OHCGO, OAFO and GDEG

Non-convex regions

1 2 3 4 5

1

2

3

4A

B

C

D

E

FG

H

O

Region OHCGO is represented by the constraints:– x1 + 3 x2 82 x1 – x2 4

Region OAFO is represented by the constraint:x1 + x2 4

Region GDEG is represented by the constraints:x1 + x2 52 x1 – x2 4

Non-convex regions

For each convex region, we introduce a binary variable whose value equals 1 when the constraints representing the region are effective and 0 otherwise

Hence, in this example we have the binary variables y1, y2 and y3.These variables are related to the regions’ constraints by:

y1 = 1 → – x1 + 3 x2 8 and 2 x1 – x2 4y2 = 1 → x1 + x2 4y3 = 1 → x1 + x2 5 and 2 x1 – x2 4

when the indicator variable takes the value of 1, the corresponding constraints will be effective and when it takes the value of 0, the constraint can be set to be redundant.

Non-convex regions

To do that, we look for the maximum values that the decision variables can take and modify the constraints as follows:

– x1 + 3 x2 + 4 y1 122 x1 – x2 + 6 y1 10

x1 + x2 + 4 y2 8

x1 + x2 + 3 y3 82 x1 – x2 + 8 (1-y3) 4

and then add the following constraint to represent the condition that at least one convex region should exist:

y1 + y2 + y3 1

Using LINGO for NLP models

• The default non-linear solver in LINGO uses local search procedures such as gradient search techniques.

• Once the solver can no longer find better solutions to the NLP model, it will terminate in either the "Global Optimum" or "Local Optimum" state.

• If the NLP model does not have any nonlinear constraints, then any locally optimal solution will also be a global optimum (because the feasible region of any set of linear constraints is always convex).

• If, on the other hand, the NLP model has one or more nonlinear constraints, then any locally optimal solution may not be the best solution.

• There may be another "peak" that is better than the current one, but the solver's local search procedure is unable to "see" the better peak.


• There are another two specialized nonlinear solvers that may be invoked in LINGO: global solver and multistart solver. These two solvers can deal with non-convex feasible regions.

• The multistart solver and the global solver are specialized solvers that attempt to find the globally optimal solution to non-convex models.

• Typically, they will converge to a local, sub-optimal point that may be quite distant from the true, globally optimal point.

• There is one drawback to using the global solver; it runs considerably slower than the default nonlinear solver. Therefore, the preferred option is to always try and write smooth, convex nonlinear models.

mk:@MSITStore:C:/Program Files/Lingo11/lingo11.chm::/global_solver.htm

mk:@MSITStore:C:/Program Files/Lingo11/lingo11.chm::/multi_start_attempts.htm


• Generally, on nonlinear models, LINGO can terminate only in the local optimum state.

• LINGO may, in fact, have a globally optimal solution, but, given the nature of nonlinear problems, LINGO is unable to claim it as such.

• Given this fact, it is always preferred to formulate a model using only linear constraints whenever possible.

mdp 631 industrial operations research lecture 9...mdp 631 industrial operations research lecture 9...

Documents