lp notes 906
Post on 04-Jun-2018
227 Views
Preview:
TRANSCRIPT
-
8/13/2019 Lp Notes 906
1/101
ISE 702
Mathematical Programming:Linear
Marc Posner
-
8/13/2019 Lp Notes 906
2/101
MISCELLANEOUS INFORMATION
Computer Rooms: BE 344, 351, 356 (all students should have access tothese rooms)
Practice Problem Answers: On ISE 702 web page
Start at www-iwse.eng.ohio-state.edu
Click on Industrial and Systems Engineering (top center)
Click on Courses (left column)
Click on ISE Course Descriptions
Click on 702 Mathematical Programming
Click on Answers to practice problems
To login to the programs:Login name: [start of osu email address]Password: [last 4 digits of social security number] [rst and last initials]
Ex: name, osu email, ss #John Doe, Doe.1osu.edu, 123-45-6789
Login name: Doe.1Password: 6789jd
1
-
8/13/2019 Lp Notes 906
3/101
LP computer packages:
Accessibility Ease of Use Power
EXCEL A D D
LINDO A- A B+CPLEX B A- A
MATLAB B+
To run EXCEL:
Build appropriate spreadsheet. Then, go to EXCEL solver
To run LINDO (for MATLAB, use the MATLAB icon):Click on Start Programs Applications Lindo 6.0 forWindowsorDouble click on Applications on the desktop, double click onLindo 6.0 for Windows
To run CPLEX:
Click on Start Programs Command PromptIn command prompt, type in the following:
C:\ > F:F: \ > cd cplexF: \ CPLEX > cplex
Frequently Asked Questions (FAQ):http://www-unix.mcs.anl.gov/otc/Guide/faq/linear-programming-faq.html
2
-
8/13/2019 Lp Notes 906
4/101
INTRODUCTION
Why Study Linear Programming?
Many practical applications in a variety of elds - both nding optimalsolutions and as approximation
Production, capital budgeting and resource allocation, blending andmix, portfolio analysis, etc,
Used in health care, food packaging, paper companies, air and waterpollution control, utilities, etc.
Standard tool for analysis of problemsIBMs largest selling commercial package (OSL)Tool in Microsoft Excel
Foundation for many theoretical developments in Operations Research
The Simplex Method is considered to be one of the most importantcomputational algorithms in Engineering and the Sciences (Computing
in Science and Engineering, 2000).
Solution Steps
1. Formulate LP
a. Dene decision variablesb. Determine objective function
c. Determine constraints
2. Solve LP - Different methodologies are available
3. Interpretation of solutions (sensitivity analysis)
3
-
8/13/2019 Lp Notes 906
5/101
Linear Program in Canonical Form:
Maximize z(x) = cx (objective function)
subject to Ax b (structural constraints)x 0 (non-negativity constraints)
Decision variables: x = ( x1 , x2 , . . . , x n ) En IR n column vector
(This is assumed throughout the course unless otherwise noted)
Parameters: c = (1 n) row vector
A = ( m n) matrixb = ( m 1) column vector
Ai (a i 1 , a i 2 , . . . , a in ) (row vector)
Aj (a1j , a 2j , . . . , a mj )t (column vector)
LP Assumptions
Deterministic: Each coefficient known with certainty
Additive: No interaction effects, i.e., contribution from(x i and xj jointly) = x i + x j (linear)
Proportional: Values of costs and contributions increase proportionally tosize (linear - no setup costs, economies of scale or higher orderterms)
Divisible: Any fractional level permissible
4
-
8/13/2019 Lp Notes 906
6/101
Linear Programming Example
Two machines ( M 1 , M 2) can produce goods. M 1 is newer and faster butmore costly. M 1 costs $3 to produce one unit. M 2 costs $2 to produce one
unit. The units sell for $5 each.Machine constraints: Cant produce more than 8 units on M 1 or morethan 6 units on M 2 . Must produce at least 1 unit on M 1 .
Material restrictions: Need 1,000 gallons of water for 1 unit on M 1 and2000 gallons for one unit on M 2 . There is 16,000 gallons.
Inventory restrictions: Cant produce more than a total of 13 units.
x i = number of units produced on M i , i = 1 , 2
Maximize z(x1 , x2) = (5 3)x1 + (5 2)x2 = 2x1 + 3x2subject to x1 8
x2 6
x1 1
x1 + 2x2 16x1 + x2 13
x i 0, i = 1 , 2
68
13
1 8 13 16
5
-
8/13/2019 Lp Notes 906
7/101
Other Problems
1. minimize cx = maximize cx.
2. If xi can be negative, then create variables xi and xi .
Replace each occurrence of xi with xi x i . Include xi 0 and xi 0.
If xi > 0, then xi > 0 and xi = 0.
If xi < 0, then xi = 0 and xi > 0.
3. If Ai x = bi , then we can rewrite the two equations as
Ai x bi Ai x bi .
6
-
8/13/2019 Lp Notes 906
8/101
Other Formulations
1. If bi 0, then
a i 1x1 + + a in xn bi a i 1x1 + a in xn + xs = bi
where xs 0 and cs = 0.
xs is called a slack variable.
2. If bi < 0, then
a i 1x1 + + a in xn bi a i 1x1 a in xn ( bi )
a i 1x1 a in xn x l = ( bi )
where xl 0 and cl = 0.
x l is called a surplus variable.
Together, 1 and 2 give Ax = b and b 0. (Standard Form)
7
-
8/13/2019 Lp Notes 906
9/101
Applications
Product Mix Problems: Minimize cost given demand and resourcecapacity constraints.
Production mix, media buying selections, nancial portfolios, etc.
Typical constraints are
production of product i demand for product i
amount of resource j used for each process amount of j available.
Diet Problems: Mixing of raw materials to achieve certain minimum levelsof attributes.
Mixing of livestock feed for cattle, mixing of foods in school cafeterias,etc.
Typical constraints are
ingredients requirement .
Example: Let xi = amount of food i used. Food i has ti calories. Sincethe minimum daily requirement is 2000 calories,
t1x i + t2x2 + + tn xn 2000.
8
-
8/13/2019 Lp Notes 906
10/101
Blending Problems: Mixing of raw materials to achieve product withcertain minimum qualities.
Blending of crude oil to make various gasoline products, mixing of chemicals, blending of alloys to make steel, mixing of recycled paper tomake various qualities of paper, making of hotdogs, etc.
Typical constraints are
quantity of attributetotal quantity
% of attribute required.
Then, multiply by denominator to linearize.
Example: Crude oil i has octane rating r i per gallon. The octane rating of gas must be at least d per gallon. Let xi = amount of crude oil i that goesinto gas.
r 1x1 + r 2x2 + + r n xnx i
d
r 1x1 + r 2x2 + + r n xn d x i(r 1 d)x1 + ( r 2 d)x2 + + ( r n d)xn 0
9
-
8/13/2019 Lp Notes 906
11/101
Multiperiod Planning Problems: Make decisions over several periods.
Inventory problems, nancial problems, work scheduling, etc.
Typical constraints areDemand and capacity constraints for a given period
Constraints that tie periods together
Example: Letdt = demand in period ts t = production capacity in period t
x t = amount produced in period tyt = inventory in period t 1 carried over to t.
x t stx t + yt dt
yt +1 = xt + yt dt
10
-
8/13/2019 Lp Notes 906
12/101
Goal Programming: Objective is to be as close as possible to a set of desired goals.
Meeting environmental standards, production scheduling (deliver itemson time), product mix, etc.
Objective function is of the form min i wi |G i ci x | whereG i = goal i, wi = weight of goal i.
Nonlinear objective function is transformed into
mini
wi (d+i + di )
sub. to. ci x d+i + di = Gi for all id+i , d
i 0 for all i
where d+i = amount over goal Gi , and di = amount under goal Gi .
Note: Can have different weights for d+i and di .
Network Problems: Shortest path, max ow, min cost ow, etc.
Transportation, communication, shipping, etc.
Typical constraints are ow constraints:
ow into a node = ow out of a node
i
x ik =j
xkj , for each node k
where xij = the ow from node i directly to node j .
11
-
8/13/2019 Lp Notes 906
13/101
Transportation Problem: Ship items from production facilities tocustomers at minimum cost.
cij = cost to ship from facility i to customer j , i = 1 , . . . , m , j = 1 , . . . , n
s i = supply of facility i, i = 1, . . . , mdj = demand of customer j , j = 1, . . . , nx ij = amount shipped from i to j , i = 1 , . . . , m , j = 1 , . . . , n
Minimizem
i=1
n
j =1
cij x ij
subject ton
j =1
x ij si , i = 1 , . . . , m
m
i=1
x ij dj , j = 1 , . . . , n
x ij 0, i = 1, . . . , m, j = 1, . . . , n ,
where mi=1 s i =
nj =1 dj .
Can assume nj =1 x ij = s i and
mi=1 x ij = dj because it is never optimal
to ship extra.
Specialized solution techniques such as Stepping Stone Method. (Seetextbook for further details.)
As network ow problem
s i dj
12
-
8/13/2019 Lp Notes 906
14/101
Assignment Problem: Matching of two types of objects
Jobs to machines, crews to routes, etc.
Minimize n
i=1
n
j =1
cij x ij
subject ton
j =1
x ij = 1, i = 1 , . . . , n
n
i=1
x ij = 1, j = 1, . . . , n
x ij {0, 1}, i = 1 , . . . , n, j = 1 , . . . , n
Special type of Transportation Problem
13
-
8/13/2019 Lp Notes 906
15/101
Denitions
6
8
1 8 16
FS
x
Feasible region (Feasible Set) FS { x | Ax b, x 0}.
Feasible solution x0 FS.
Optimal Set (maximization) OS { x FS | cx cx0 for all x0 FS }.
Optimal Solution (max) Global Optimal Solution x OS z(x) z(x) for all x FS
Extreme point corner point of FS.
Optimal solutions for an LP occur at extreme points of FS.
14
-
8/13/2019 Lp Notes 906
16/101
Ball B (x0 ,) = {x IR n | || x0 x || }
Local Optimal Solution (max) x0 S where there is an > 0 such thatf (x0) f (x) for all x S B (x0 ,)
for some neighborhood around x0 , x0 is the best solution
Local
OptimumGlobal
Optimum
15
-
8/13/2019 Lp Notes 906
17/101
Graphical Solution Method
1. Make picture of the constraint region.
2. Move isoquant lines of objective function in direction c until optimumis reached.
3. Determine coordinates of extreme point(s).
Maximize z(x1 , x2) = (5 3)x1 + (5 2)x2 = 2x1 + 3x2subject to x1 8
x2 6
x1 1
x1 + 2x2 16
x1 + x2 13
x i 0, i = 1 , 2
6
8
13
1 8 13 16
x = (8 , 4), z = 28
z = 2
z = 16
16
-
8/13/2019 Lp Notes 906
18/101
Outcomes of a Linear Program
1. Unique Optimal Solution
2. Multiple Optimal Solutions
3. Unbounded Solution
4. Infeasible Solution
2. Multiple Optimal Solutions
Maximize z(x) = x1
subject to x1 8x2 6
x1 + 2x2 16
x 0
Entire set between optimal extreme points is optimal all points inconvex combination = { i x i | x i is an optimal extreme point,
i = 1, 0 i 1 for all i}.
Example: 14 (8, 4) + 34 (8, 0) = (8 , 1)
(8,0)
(8,4)z
17
-
8/13/2019 Lp Notes 906
19/101
3. Unbounded Solution
Maximize z(x) = 2 x1 + 3x2subject to x2 6
x1 + x2 1
x 0
6
1
Note: Solution unbounded FS unbounded.
FS unbounded solution unbounded.
z(x) = 2x1 3x2
18
-
8/13/2019 Lp Notes 906
20/101
4. Infeasible Solution
Maximize z(x) = 2 x1 + 3x2subject to x1 8
x2 6
x1 + 2x2 16
x1 + x2 15
x 0
6
8
15
8 15 16
19
-
8/13/2019 Lp Notes 906
21/101
CONVEXITY AND POLYHEDRAL SETS
Hyperplane { x IR n | x = k, IR n }.
is normal (perpendicular) to hyperplane.
Halfspace { x IR n | x k, IR n } set of points on one side of hyperplane.
Polyhedral set intersection of a nite number of halfspaces.
FS is a polyhedral set.
Convex combination x is a convex combination of x1 , x2 , . . . x m if thereexists IR m such that x =
mi=1 i x
i for i 0 and mi=1 i = 1.
{x | x = x 1 + (1 )x2 , 0 1} = line segment with endpoints x1 andx2 .
Convex set S is a convex set if for all x1 , x2 S and (0, 1), thenx 1 + (1 )x2 S .
Theorem: FS is a convex set.
Proof: Let x1 , x2 F S . This implies that Ax1 b and Ax2 b.
Thus,
A(x 1 + (1 )x2) = Ax 1 + (1 )Ax2
b + (1 )b = b.
20
-
8/13/2019 Lp Notes 906
22/101
Constraint Types for Ai x bi , i {1, 2, . . . , m }:
Binding or Tight for x0 Ai x0 = bi .
Non-binding or Loose for x0 Ai x0 < b i .
Redundant {x | Ak x bk , k = 1, . . . , i 1, i + 1 , . . . , m ; x 0} = {x | Ax b, x 0}.
Faces of a Polyhedral Set
Face F of a polyhedral set F {x | Ax b, x 0} {x | AI x = bI , x J = 0}, for some
I {1, 2, . . . , m } and J {1, 2, . . . , n }.
0-dimensional face = extreme point1-dimensional face = edge
(n 1)-dimensional face = facet (hyperplane)
n-dimensional face = FS ( I = J = {})
21
-
8/13/2019 Lp Notes 906
23/101
Extreme point of a convex set Let S be a convex set. x0 S is anextreme point of S if there does not exist (0, 1) and x1 , x2 S ,where x1 , x2 = x0 , such that x0 = x 1 + (1 )x2
i.e. x0
is not a convex combination of x1
and x2
.
At extreme point x0 , there are at least n tight constraints from the m + nconstraints Ax b, x 0.
degenerate x0 is a degenerate extreme point if more than n constraintsare tight at x0 .
adjacent extreme points x1 and x2 are adjacent if the line segment joining them is an edge of {x | Ax b, x 0}.
Ray emanating at x0 { x0 + d | 0} for some dIR n .
Direction of ray r d such that r = {x0 + d | 0}.
Direction of convex set S direction of recession dIR n such thatx + d S for all x S and 0.
d may be normalized for recession direction, i.e., di = 1.
22
-
8/13/2019 Lp Notes 906
24/101
-
8/13/2019 Lp Notes 906
25/101
Characterization of Polyhedral Sets
Caratheodorys Theorem. Let x1 , x2 , . . . , x k be the set of extreme points and let d1 , d2 , . . . , d l be the set of extreme directions of {x | Ax b, x 0}. x0 {x | Ax b, x 0} iff there exist , such that x0 =
ki=1 i x
i + lj =1 j dj ,
ki=1 i = 1 , , 0.
Proof sketch (see book for details).
. Suppose x0 = ki=1 i x
i + lj =1 j dj ,
ki=1 i = 1, , 0. Because
X = {x | Ax b, x 0} is convex, x = ki=1 i x
iX . d =
lj =1 j d
j isa recession direction of X . Thus, x0 = x + d X .
. Suppose x0X . Let M be sufficiently large so thatk
i=0nj =1 x
ij M . Since xi X = X {x |
nj =1 x j M }, let
x1 , . . . , x k , xk +1 , . . . , x k + u be the extreme points of X .
We rst show that we can write x0 as a convex combination of theextreme points of X . Suppose that the constraintsG {1, 2, . . . , m + n + 1 } are tight at x0 . If |G| n, then this part iscomplete. Otherwise, select a direction d such that the constraints of G
remain tight, i.e., Gd = 0. Move in direction d until an additionalconstraint is tight at y1 . Repeat in direction d to nd point y2 . x0 is aconvex combination of y1 and y2 . If y1 and y2 are extreme points of X ,then this part is complete. Otherwise, repeat for each point until nconstraints are tight at all of the points. Using these points we can writex0 as a convex combination of the extreme points of X .
If these points are extreme points of X , then we are done. Otherwise,observe that each of xk+1 , xk +2 , . . . , x k+ u are adjacent to some extremepoint of X . Let xk+ i be adjacent to xv( i ) X . Since the constraint
ki=0
nj =1 x
ij M is articial, xk + i xv( i ) is an extreme direction.
Thus, xk+ i = xv( i ) + t ( i ) dt ( i ) for some t(i). This implies thatx0 =
ki=1 i x
i + lj =1 j dj .
24
-
8/13/2019 Lp Notes 906
26/101
Concave function f : S IR is concave if f (x 1 + (1 )x2) f (x1) + (1 )f (x2) for all x1 , x2 S and (0, 1).
Theorem. If x0 is a local maximum of a concave function over a convex set S , then x0 is a global maximum.
Proof: If x0 is a local maximum, then there exists an -neighborhood N around x0 such that f (x0) f (x) for all x N .
Let x be a global maximum. Since S is convex, x+ (1 )x0 S forall (0, 1). Select such that x = x + (1 )x0 N .
By the concavity of f , f (x) = f (x + (1 )x0) f (x)+ (1 )f (x0).
Since x N , f (x0) f (x). Thus,
f (x0) f (x) + (1 )f (x0)
f (x0) f (x)
f (x0) f (x)
and x0 OS .
25
-
8/13/2019 Lp Notes 906
27/101
Linear functional f : IR n IR such that f (x + y) = f (x) + f (y) forall x, y IR n , IR .
The objective function is a linear functional.
Theorem. OS is convex.
Proof: Let x1 , x2 OS, and let x = x 1 + (1 )x2 for [0, 1].Since x1 , x2 FS and FS is convex, x FS.Since z is a linear functional,
z(x) = z(x1
+ (1 )x2
)= z(x1) + (1 )z(x2)
= z(x1).
Convex function f : S IR is convex if f (x 1 + (1 )x2) f (x1) + (1 )f (x2) for all x1 , x2 S and
(0, 1).
Closed set a set that includes its boundary points.
Theorem. There exists a local maximum, x0 , of a convex function over a closed set S such that x0 is on the boundary of S.
LP has an optimal solution on the boundary of {x | Ax b, x 0}.
26
-
8/13/2019 Lp Notes 906
28/101
Theorem. If OS = , then z(x) achieves a maximum at an extreme point of FS.
Proof: For any x F S ,
x =k
i=1
i x i +l
j =1
j dj
k
i=1
i = 1
, 0,
where {x i } are the extreme points and {dj } are the extreme directions of
FS.The LP is equivalent to
maxk
i=1
(cxi ) i +l
j =1
(cdj )j
sub to.k
i=1
i = 1
, 0.
If cdj > 0 for any j , then z = and unbounded optimum.
If cdj 0 for all j , then we can set j = 0 for all j .
27
-
8/13/2019 Lp Notes 906
29/101
Therefore, LP is
maxk
i=1
(cxi ) i
sub to.k
i=1
i = 1
0.
The optimum value is max {cxj }, and the optimal solution is at anextreme point.
Examine neighboring extreme points until no further improvement ispossible.
28
-
8/13/2019 Lp Notes 906
30/101
Characterizing an Extreme Point Solution
Let Ax = b. Add surplus and slack variables if necessary.
To have a solution, rank (A) = rank (A, b).
Assume that the rank (A) = m.
Basis of A B m linearly independent columns of A.
A = ( B | N ) after possible rearrangement of the columns
We use B to represent both the columns of A and the indicescorresponding to the columns. The same is true for N . The meaningfollows from the context.
Ax = ( B | N )xBxN
= b
Bx B + Nx N = b
Bx B = b Nx N
Since rank (B ) = m and B is (m m), B 1 exists.
Therefore, xB = B 1b B 1Nx N .
29
-
8/13/2019 Lp Notes 906
31/101
Theorem. x0 is an extreme point of FS iff there exists B {1, 2, . . . n }such that B has m elements, rank (B ) = m, x0B = B
1b 0, and x0N = 0.
Basic variables xB .Nonbasic variables Zero variables xN .
Basic feasible solution x Bx N =B 1 b
0 .
Each basic feasible solution corresponds to a unique extreme point.
Degenerate solution xj = 0 for some j B .
A degenerate extreme point corresponds to multiple bases.
Each non-degenerate extreme point corresponds to a unique basis.
Example:x1 8 x1 + x3 = 8
x2 6 = x2 + x4 = 6
x1 + 2x2 12 x1 + 2x2 + x5 = 12
x 0 x 0
Point (0 , 6, 8, 0, 0) is degenerate because
B = {1, 2, 3}, N = {4, 5}
B = {2, 3, 4}, N = {1, 5}
B = {2, 3, 5}, N = {1, 4}
(0,6)x1 = 0 x1 = 8
x2 = 0
x2 = 6
x1 + 2x2 = 12
At most nm bases implies at mostnm extreme points.
Two extreme points are adjacent only if the set of basic variables differsby 1 index (assuming no degeneracy).
30
-
8/13/2019 Lp Notes 906
32/101
Let ej = (0 , . . . , 0, 1, 0, . . . , 0) (The 1 is the j th entry).
Theorem. d is an extreme direction of {x | Ax = b, x 0} iff for some j N , B 1N j 0 and d = dB
dN = B
1 N j
e j.
Proof sketch: ( B | N ) dBdN = 0 (necessary condition for d to be an extremedirection) implies that ( I | B 1N ) dBdN = 0.
Thus, dB = B 1NdN .
Let dN = ej . Then dBdN = B 1 N j
e j .
Example:
x1 + x2 1 x1 + x2 + x3 = 1
x1 + 2x2 3 = x1 + 2x2 + x4 = 3
x 0 x 0
At (1 , 2, 0, 0), B = {1, 2}, N = {3, 4},
B = 1 1 1 2 , B 1 =
2 1 1 1 , N 3 =
10 , N 4 =
01
B 1N 3 = 2 1 0. Thus, d = (2 , 1, 1, 0)t
increase x3 , keep x4 = 0
B 1N 4 = 11 > 0. Thus, no extreme direction for this j .
31
-
8/13/2019 Lp Notes 906
33/101
THE SIMPLEX METHOD
We now assume that the objective is to minimize (to follow book)
The Algorithm
1. Convert problem to appropriate form (minimize and equalityconstraints).
2. Find an initial basic feasible solution.
3. Check for optimality.
If optimal, then stop. The current extreme point is optimal.
Otherwise, nd a non-basic variable that will improve (reduce) thesolution if increased.
4. Move to an adjacent extreme point.
a. Increase the non-basic variable as much as feasible.
b. Adjust the set of basic and non-basic variables.
Go to 3.
32
-
8/13/2019 Lp Notes 906
34/101
Recognizing an Optimal Solution
min z(x) = cB xB + cN xN
At each extreme point, xB = B 1b, xN = 0.
Thus, z(x) = cB B 1b = z0 = current value.
But xB = B 1b B 1Nx N . Therefore,
z(x) = cB (B 1b) cB B 1Nx N + cN xN = z0 (cB B 1N cN )xN = z0
j N
(cB B 1N j cj )x j
= z0 j N
(zj cj )x j
where zj = cB B 1N j = opportunity cost.
If zj cj 0 for all j N , then we are at an optimal extreme point.
Otherwise, we can improve the solution by increasing xj .
33
-
8/13/2019 Lp Notes 906
35/101
Which Nonbasic Variable to Increase
1. Steepest ascent ( max j N {zj cj }): Dantzigs rule
2. Largest improvement rule - Largest leaving and entering pairExperimental results are discouraging.
3. Partial pricing - Steepest ascent from a small collection of choices
Useful for large scale problems.
4. Lexicographical - Smallest index where zj cj > 0
Useful for avoiding degeneracy.
Amount of Solution Improvement
xB 0
B 1b B 1Nx N 0
B 1b B 1N j x j 0B 1b B 1N j x j , j N
We can increase xj until ( B 1b)i = ( B 1N j ) i x j for some i B .
Minimum ratio test: min iB {(B 1b)i / (B 1N j )i | (B 1N j )i > 0}
book denes ( B 1b)i bi and (B 1N j )i yi
This implies that xi becomes 0.x j is entering variable (the variable that enters the basis).
x i is leaving variable (the variable that leaves the basis).
Since only one variable in basis changes, the extreme points are adjacent.
34
-
8/13/2019 Lp Notes 906
36/101
Resolving for B 1 each iteration is a lot of work.
We keep the current basis and associated values in a tableau to reduce thework and make current information accessible.
The Simplex Tableau
Initial:decision vars. slack vars. RHS
z c 0 0
xs A I b
Later:xB xN RHS
z 0 cB B 1N cN cB B 1b
xB I B 1N B 1b
35
-
8/13/2019 Lp Notes 906
37/101
Example:
max z(x) = 2 x1 + 3x2 min z(x) = 2x1 3x2sub. to x1 8 sub. to x1 + x3 = 8
x2 6 = x2 + x4 = 6
x1 + 2x2 16 x1 + 2x2 + x5 = 16
x 0 x 0
B = {3, 4, 5}, N = {1, 2}
x3 = 8 x1
x4 = 6 x2x5 = 16 x1 2x2
z = 0 2x1 3x2
x1 x2 x3 x4 x5 RHS
z 2 3 0 0 0 0
cB B 1N cN 0 cB B 1b
x3 1 0 1 0 0 8x4 0 1 0 1 0 6
x5 1 2 0 0 1 16
B 1N I B 1b
B = B 1 =1 0 0
0 1 0
0 0 1
, N =1 0
0 1
1 2
,
cB = (0 , 0, 0), cN = ( 2, 3)
Increase x2 (steepest ascent rule)
36
-
8/13/2019 Lp Notes 906
38/101
B 1b =
8
6
16
B 1N 2x2 =1 0 0
0 1 0
0 0 1
0
1
2
x2
= x2 = min {6, 8} = 6, x2 is entering variable, x4 is leaving variable
B = {2, 3, 5}, N = {1, 4}
x3 = 8 x1x2 = 6 x4x5 = 16 x1 2x2 = 16 x1 2(6 x4) = 4 x1 + 2x4
z = 0 2x1 3(6 x4) = 18 2x1 + 3x4
Use elementary row operations to get I for {x3 , x2 , x5}. Note equivalenceto algebra.
x1 x2 x3 x4 x5 RHS
z 2 0 0 3 0 18
x3 1 0 1 0 0 8
x2 0 1 0 1 0 6x5 1 0 0 2 1 4
B =
1 0 0
0 1 0
0 2 1
, B 1 =
1 0 0
0 1 0
0 2 1
, N =
1 0
0 1
1 0
,
cB = (0 , 3, 0), cN = ( 2, 0)B 1 can be found from the columns of the original slack variables because
B 1(A | I ) = ( I | B 1N | B 1).
Increase x1
37
-
8/13/2019 Lp Notes 906
39/101
8
6
4
B 1N 1x1 =1 0 0
0 1 0
0 2 1
1
0
1
x1
= x1 = min {8, 4} = 4, x1 is entering variable, x5 is leaving variable
B = {1, 2, 3}, N = {4, 5}
x3 = 8 x1 = 8 (4 + 2 x4 x5) = 4 2x4 + x5x2 = 6 x4x1 = 4 + 2 x4 x5
z = 18 2(4 + 2 x4 x5) + 3 x4 = 26 x4 + 2x5
x1 x2 x3 x4 x5 RHS
z 0 0 0 1 2 26
x3 0 0 1 2 1 4
x2 0 1 0 1 0 6
x1 1 0 0 2 1 4
B =
1 0 1
0 1 0
0 2 1
, B 1 =
1 2 1
0 1 0
0 2 1
, N =
0 0
1 0
0 1
,
cB = (0 , 3, 2), cN = (0 , 0)
Increase x4
38
-
8/13/2019 Lp Notes 906
40/101
4
6
4
B 1N 4x4 =1 2 1
0 1 0
0 2 1
0
1
0
x4
= x4 = min {2, 6} = 2, x4 is entering variable, x3 is leaving variable
B = {1, 2, 4}, N = {3, 5}
x4 = 2 0.5x3 + 0 .5x5x2 = 4 + 0 .5x3 0.5x5x1 = 8 x3
z = 28 + 0 .5x3 + 1 .5x5
x1 x2 x3 x4 x5 RHS
z 0 0 1/ 2 0 3/ 2 28
cB B 1b
x4 0 0 1/ 2 1 1/ 2 2
x2 0 1 1/ 2 0 1/ 2 4
x1 1 0 1 0 0 8B 1 B 1b
Solution is optimal.
B =0 0 1
1 1 0
0 2 1
, B 1 =1/ 2 1 1/ 2
1/ 2 0 1/ 2
1 0 0
, N =1 0
0 0
0 1
,
cB = (0 , 3, 2), cN = (0 , 0)
39
-
8/13/2019 Lp Notes 906
41/101
Multiple Optimal Solutions
At optimal solution, some nonbasic variable has zj cj = 0. Thus, canincrease variable at no additional cost.
Example:
min z(x) = 32 x1 3x2sub. to x1 + x3 = 8
x2 + x4 = 6
x1 + 2x2 + x5 = 16
x 0
At optimal solution (4 , 6, 4, 0, 0) B = {1, 2, 3}, N = {4, 5} the tableau is
x1 x2 x3 x4 x5 RHS
z 0 0 0 0 3/ 2 24
x3 0 0 1 2 1 4
x2 0 1 0 1 0 6x1 1 0 0 2 1 4
Increase x4 by 2 to get optimal solution (8 , 4, 0, 2, 0)
x1 x2 x3 x4 x5 RHS
z 0 0 0 0 3/ 2 24
x4 0 0 1/ 2 1 1/ 2 2
x2 0 1 1/ 2 0 1/ 2 4x1 1 0 1 0 0 8
OS = set of convex combinations of optimal extreme points ={(4, 6, 4, 0, 0) + (1 )(8 , 4, 0, 2, 0) | [0, 1]}.
40
-
8/13/2019 Lp Notes 906
42/101
Unbounded Solution
At some basic feasible solution, some nonbasic variable has zj cj > 0and has B 1N j 0. Thus, nonbasic variable can go to and continue to
improve solution.
Example:
min z(x) = 2x1 3x2sub. to x2 + x3 = 6
x1 + x2 + x4 = 1
x 0
A last basic feasible solution is when B = {1, 2}, N = {3, 4}
x1 = 1 + x2 + x4 = 1 + (6 x3) + x4 = 5 x3 + x4x2 = 6 x3
z = 2(5 x3 + x4) 3(6 x3) = 28 + 5x3 2x4
x1 x2 x3 x4 RHSz 0 0 5 2 28
x1 1 0 1 1 5
x2 0 1 1 0 6
Increase x4
56
B 1N 4x4 = 1 11 0
01
x4 = 10
x4
= increase x4 indenitely
41
-
8/13/2019 Lp Notes 906
43/101
STARTING SOLUTION AND CONVERGENCE
Finding an Initial Basic Feasible Solution
May not have a basic feasible solution at start.
Example:
max z(x) = 2 x1 + 3x2 min z(x) = 2x1 3x2sub. to x1 8 sub. to x1 + x3 = 8
x2 6 = x2 + x4 = 6
x1 1 x1 x5 = 1
x1 + 2x2 = 16 x1 + 2x2 = 16
x 0 x 0
x3 = 8 x1x4 = 6 x2
x5 = 1 + x1 ? = 16 x1 2x2 solving with x1 basic makes x3 < 0
For , subtract surplus variable and add articial variable.
For =, add articial variable.
x1 + x
3 = 8
x2 + x4 = 6
x1 x5 + xa = 1
x1 + 2x2 + xb = 16
where B = {3, 4,a ,b}
42
-
8/13/2019 Lp Notes 906
44/101
Two Phase Simplex Method
Phase I: Solve
min xartsub. to Ax = b
x 0
If not all xart = 0, then problem is infeasible. Stop.
Otherwise, pivot until all xart are nonbasic (if possible). Drop articial
variables (columns) if desirable, or keep nonbasic.If some articial variables remain in basis, then remove thecorresponding rows (they are redundant), or keep variables at 0.
Phase II: Solve
min cx
sub. to Ax = b
x 0
starting at current basic feasible solution.
43
-
8/13/2019 Lp Notes 906
45/101
Phase I:
x1 x2 x3 x4 x5 xa xb RHS
z 0 0 0 0 0 1 1 0
z 2 2 0 0 1 0 0 17
x3 1 0 1 0 0 0 0 8
x4 0 1 0 1 0 0 0 6
xa 1 0 0 0 1 1 0 1
xb 1 2 0 0 0 0 1 16
x1 = entering variable, xa = leaving variable
z 0 2 0 0 1 2 0 15
x3 0 0 1 0 1 1 0 7
x4 0 1 0 1 0 0 0 6
x1 1 0 0 0 1 1 0 1
xb 0 2 0 0 1 1 1 15
x2 = entering variable, x4 = leaving variable
z 0 0 0 2 1 2 0 3
x3 0 0 1 0 1 1 0 7
x2 0 1 0 1 0 0 0 6
x1 1 0 0 0 1 1 0 1
xb 0 0 0 2 1 1 1 3
44
-
8/13/2019 Lp Notes 906
46/101
x5 = entering variable, xb = leaving variable
x1 x2 x3 x4 x5 xa xb RHS
z 0 0 0 0 0 1 1 0
x3 0 0 1 2 0 0 1 4
x2 0 1 0 1 0 0 0 6
x1 1 0 0 2 0 0 1 4
x5 0 0 0 2 1 1 1 3
A basic feasible solution has been reached. End of Phase I.
Phase II:
x1 x2 x3 x4 x5 xa xb RHS
z 2 3 0 0 0 0 0 0
z 0 0 0 1 0 0 2 26
x3 0 0 1 2 0 0 1 4
x2 0 1 0 1 0 0 0 6
x1 1 0 0 2 0 0 1 4
x5 0 0 0 2 1 1 1 3
x4 = entering variable, x3 = leaving variable
z 0 0 1/ 2 0 0 0 3/ 2 28
x4 0 0 1/ 2 1 0 0 1/ 2 2x2 0 1 1/ 2 0 0 0 1/ 2 4
x1 1 0 1 0 0 0 0 8
x5 0 0 1 0 1 1 0 7
45
-
8/13/2019 Lp Notes 906
47/101
Infeasible Solution
Example:
max z(x) = 2 x1 + 3x2 min z(x) = 2x1 3x2sub. to x1 8 sub. to x1 + x3 = 8
x2 6 = x2 + x4 = 6
x1 15 x1 x5 + xa = 15
x1 + 2x2 16 x1 + 2x2 + x6 = 16
x 0 x 0
Phase I:
x1 x2 x3 x4 x5 x6 xa RHS
z 0 0 0 0 0 0 1 0
z 1 0 0 0 1 0 0 15
x3 1 0 1 0 0 0 0 8
x4 0 1 0 1 0 0 0 6
xa 1 0 0 0 1 0 1 15
x6 1 2 0 0 0 1 0 16
x1 = entering variable, x3 = leaving variable
z 0 0 1 0 1 0 0 7
x1 1 0 1 0 0 0 0 8
x4 0 1 0 1 0 0 0 6
xa 0 0 1 0 1 0 1 7
x6 0 2 1 0 0 1 0 8
Optimal, but cant get xa out of basis
46
-
8/13/2019 Lp Notes 906
48/101
Articials Cannot be Removed From Basis
Example:
max z(x) = 2 x1 + 3x2 min z(x) = 2x1 3x2sub. to x1 = 8 sub. to x1 + xa = 8
x2 = 6 = x2 + xb = 6
x1 + x2 15 x1 + x2 + x3 = 15
x1 + 2x2 = 20 x1 + 2x2 + xc = 20
x 0 x 0
End of Phase I:x1 x2 x3 xa xb xc RHS
z 0 0 0 2 3 0 0
x1 1 0 0 1 0 0 8
x2 0 1 0 0 1 0 6
x3 0 0 1 1 1 0 1
xc 0 0 0 1 2 1 0
Phase II:
x1 x2 x3 xa xb xc RHS
z 2 3 0 0 0 0 0
z 0 0 0 2 3 0 34
Notice 0 in z row for xc . If xa and xb dont enter basis, then xc will neverchange
47
-
8/13/2019 Lp Notes 906
49/101
Big M Method
Add a large value to articial variables to make them leave the basis.
If they cant, then solution infeasible.
Method is similar to Two Phase method.
Advantages:
1. In case of ties, move direction of good LP solution
Disadvantages:
1. Causes rounding errors
2. How big is big enough must take coefficient and RHS intoaccount
.000001x1 1 large value for x1
Methods almost equivalent
Commercial codes use Two Phase method
Improvement is Little , i.e., make original variables small. Then roundingerrors dont affect Phase I.
48
-
8/13/2019 Lp Notes 906
50/101
Example of move in good direction:
max z(x) = 2 x1 + 3x2 min z(x) = 2x1 3x2 + Mx a + Mx bsub. to x1 8 sub. to x1 + x3 = 8
x2 6 = x2 + x4 = 6
x1 1 x1 x5 + xa = 1
x1 + 2x2 = 16 x1 + 2x2 + xb = 16
x 0 x 0
x1 x2 x3 x4 x5 xa xb RHS
z 2 3 0 0 0 M M 0
z 2M + 2 2M + 3 0 0 M 0 0 17M
x3 1 0 1 0 0 0 0 8
x4 0 1 0 1 0 0 0 6
xa 1 0 0 0 1 1 0 1
xb 1 2 0 0 0 0 1 16
Final Tableau
z 0 0 1/ 2 0 0 M M 3/ 2 28
x4 0 0 1/ 2 1 0 0 1/ 2 2
x2 0 1 1/ 2 0 0 0 1/ 2 4
x1 1 0 1 0 0 0 0 8
x5 0 0 1 0 1 1 0 7
Same as Two Phase method, except for articial columns in z row
49
-
8/13/2019 Lp Notes 906
51/101
Degeneracy
Degeneracy too many hyperplanes go through an extreme point (at
least n m + 1)Thus, some basic variable has 0 value
Example:
max z(x) = 2 x1 + 3x2 min z(x) = 2x1 3x2sub. to x1 8 sub. to x1 + x3 = 8
x2
6 = x2 + x
4 = 6
2x1 + 5x2 38 2x1 + 5x2 + x5 = 38
x1 + 2x2 16 x1 + 2x2 + x6 = 16
x 0 x 0
After rst pivot:
x1 x2 x3 x4 x5 x6 RHS
z 2 0 0 3 0 0 18x3 1 0 1 0 0 0 8
x2 0 1 0 1 0 0 6
x5 2 0 0 5 1 0 8
x6 1 0 0 2 0 1 4
(4,6)
x1 is entering variablex5 or x6 is leaving variable. Before degeneracy, more than one variable
can enter basis.
50
-
8/13/2019 Lp Notes 906
52/101
Degenerate Tableau:
x1 x2 x3 x4 x5 x6 RHS
z 0 0 0 2 1 0 26
x3 0 0 1 5/ 2 1/ 2 0 4x2 0 1 0 1 0 0 6
x1 1 0 0 5/ 2 1/ 2 0 4
x6 0 0 0 1/ 2 1/ 2 1 0
x4 enters and x6 leaves for no progress
Cycling Simplex method cycles between bases at a degenerate point andnever leaves extreme point
Fake example: alternate between ( x4 , x5), (x4 , x6), and ( x5 , x6)
Real example appears in book p 165 - 166
Stalling Simplex method cycles between bases at a degenerate pointand takes a long time to leave
Extreme point can be degenerate without cycling or stalling occurring
51
-
8/13/2019 Lp Notes 906
53/101
Methods to Prevent Cycling
Basic idea is to create a rule to break ties in an ordered way so that old
basis is not repeated
1. Lexicographical
2. Bland
Suppose x1 , x2 , . . . , x k have 0 value at a degenerate point. Make it moredesirable to have xi in the basis over any of xi+1 , x i+2 , . . . , x k . Thus,
never return to a previously visited basis because would get worse.
Methods to prevent cycling are not employed in most commercial codes
1. Cost It is too time consuming to check
2. Unlikely It seldom happens in practice because computerrounding errors breaks ties
Maybe in network problems where ow conservationequations have lots of equations at 0 value
52
-
8/13/2019 Lp Notes 906
54/101
SPECIAL IMPLEMENTATIONS
Revised Simplex Method
Keeps all information in smaller array
All commercial Simplex codes use this method
The Simplex Tableau:
xB xN RHS
z 0 cB B 1N cN cB B 1b
xB I B 1N B 1b
Revised Simplex Tableau:
z cB B 1 cB B 1b
xb B 1 B 1b
Revised Simplex Method
1. For each nonbasic variable calculate zj cj = cB B 1N j cj .Select index k = argmax j N {zj cj }.If zk ck 0, then stop.
2. Entering variable is k.Leaving variable isr = argmin
iB{(B 1b)i / (B 1N k )i | (B 1N k )i > 0}.
3. Update tableau using elementary row operations that turn N r intoer . Go to 1.
Standard Simplex array size is ( m + 1)( n + 1) while Revised Simplex arraysize is (m + 1) 2
53
-
8/13/2019 Lp Notes 906
55/101
Example:
min z(x) = 2x1 3x2sub. to x1 + x3 = 8
x2 + x4 = 6x1 + 2x2 + x5 = 16
x 0
B = {3, 4, 5}, N = {1, 2}
z 0 0 0 0
cB B
1 cB B
1bx3 1 0 0 8
x4 0 1 0 6
x5 0 0 1 16
B 1 B 1b
N =1 0
0 1
1 2
, cB = (0 , 0, 0), cN = ( 2, 3)
cB B 1N 1 c1 = (0 , 0, 0)1
0
1
( 2) = 2
cB B 1N 2 c2 = (0 , 0, 0)01
2
( 3) = 3
Increase x2 (steepest ascent rule)
54
-
8/13/2019 Lp Notes 906
56/101
B 1b =
8
6
16
B 1N 2x2 =1 0 0
0 1 0
0 0 1
0
1
2
x2
= x2 = min {6, 8} = 6, x2 is entering variable, x4 is leaving variable
B = {2, 3, 5}, N = {1, 4}
Since B 1N 2 =
0
1
2
=
0
1
0
, add -2(row 2) to row 3
z 0 3 0 18
x3 1 0 0 8
x2 0 1 0 6
x5 0 2 1 4
N =1 00 1
1 0
, cB = (0 , 3, 0), cN = ( 2, 0)
cB B 1 = (0 , 3, 0)1 0 0
0 1 0
0 2 1
= (0 , 3, 0)
55
-
8/13/2019 Lp Notes 906
57/101
cB B 1N 1 c1 = (0 , 3, 0)1
0
1
( 2) = 2
cB B 1N 4 c4 = (0 , 3, 0)0
1
0
0 = 3
Increase x1
8
64
B 1
N 1x1 =
1 0 0
0 1 00 2 1
1
01
x1
= x1 = min {8, 4} = 4, x1 is entering variable, x5 is leaving variable
B = {1, 2, 3}, N = {4, 5}
Since B 1N 1 =
1
01
=
0
01
, add -(row 3) to row 1
z 0 1 2 26
x3 1 2 1 4
x2 0 1 0 6
x1 0 2 1 4
N =0 0
1 0
0 1
, cB = (0 , 3, 2), cN = (0 , 0)
56
-
8/13/2019 Lp Notes 906
58/101
cB B 1 = (0 , 3, 2)1 2 1
0 1 0
0 2 1
= (0 , 1, 2)
cB B 1N 4 c4 = (0 , 1, 2)0
1
0
0 = 1
cB B 1N 5 c5 = (0 , 1, 2)0
0
1
0 = 2
Increase x4
4
6
4
B 1N 4x4 =1 2 1
0 1 0
0 2 1
0
1
0
x4
= x4 = min {2, 6} = 2, x4 is entering variable, x3 is leaving variableB = {1, 2, 4}, N = {3, 5}
57
-
8/13/2019 Lp Notes 906
59/101
Since B 1N 4 =
2
1
2
=
1
0
0
, multiply row 1 by 1/2, add 12 (row
1) to row 2, add row 1 to row 3
z 1/ 2 0 3/ 2 28
cB B 1 cB B 1b
x4 1/ 2 1 1/ 2 2
x2 1/ 2 0 1/ 2 4
x1
1 0 0 8
B 1 B 1b
N =1 0
0 0
0 1
, cB = (0 , 3, 2), cN = (0 , 0)
cB B 1 = (0 , 3, 2)1/ 2 1 1/ 2
1/ 2 0 1/ 2
1 0 0
= ( 1/ 2, 0, 3/ 2)
cB B 1N 3 c3 = ( 1/ 2, 0, 3/ 2)1
0
0
0 = 1/ 2
cB B 1N 5 c5 = ( 1/ 2, 0, 3/ 2)0
0
1
0 = 3/ 2
Therefore, optimal solution
58
-
8/13/2019 Lp Notes 906
60/101
Product Form of the Inverse
Used in some commercial codes uses sparsity, reduces rounding errors,keeps tableau small
To obtain new B 1 (but not needed), for entering variable k at row r,add multiple of row r to other rows to make column k equal to er premultiply B 1 by matrix
E =
1 0 0 0 (B 1N k )1 / (B 1N k )r 0 0
0 1 0 0 (B 1N k )2 / (B 1N k )r 0 0...
0 0 0 1 (B 1N k )r 1 / (B 1N k )r 0 0
0 0 0 0 1/ (B 1N k )r 0 0
0 0 0 0 (B 1N k )r +1 / (B 1N k )r 1 0...
0 0 0 0 (B 1N k )m / (B 1N k )r 0 1
Let B 1i be B 1 at the ith simplex iteration. Then,
B 11 = I,
B 12 = E 1B 13 = E 2E 1
...
B 1t = E t 1E t 2 E 1
Only need to store the column and position for each E
If number of E s become large, reinvert basis and restart at new basis
59
-
8/13/2019 Lp Notes 906
61/101
-
8/13/2019 Lp Notes 906
62/101
cB B 1 = cB E t 1E t 2 E 1
Postmultiply by E
If column r of E is g = ( g1 , g2 , . . . , gm )t, then
cE = ( c1 , c2 , . . . , c m )
1 0 0 g1 0 0
0 1 0 g2 0 0...
0 0 1 gr 1 0 0
0 0 0 gr 0 00 0 0 gr +1 1 0
...
0 0 0 gm 0 1
= ( c1 , c2 , . . . , c r 1 ,cg,cr +1 , . . . , cm )
61
-
8/13/2019 Lp Notes 906
63/101
Example:
min z(x) = 2x1 3x2sub. to x1 + x3 = 8
x2 + x4 = 6x1 + 2x2 + x5 = 16
x 0
xB =x3x4x5
, N = {1, 2}
B 1 =1 0 0
0 1 0
0 0 1
, N =1 0
0 1
1 2
, cB = (0 , 0, 0), cN = ( 2, 3)
cB B 1N 1 c1 = (0 , 0, 0)1
0
1
( 2) = 2
cB B 1N 2 c2 = (0 , 0, 0)0
1
2
( 3) = 3
Increase x2
62
-
8/13/2019 Lp Notes 906
64/101
B 1b =
8
6
16
B 1N 2x2 =0
1
2
x2
= x2 = min {6, 8} = 6, x2 is entering variable, x4 is leaving variable
xB =x3x2x5
Store for E 1 ,(column of E ; entering row r) = ( 0/ 1, 1/ 1, 2/ 1;2) = (0 , 1, 2;2)
xN =x1x4
, N =1 0
0 1
1 0
, cB = (0 , 3, 0), cN = ( 2, 0)
cB B 1 = cB E 1 = 0, (0, 3, 0)0
1
2
, 0 = (0 , 3, 0)
(cB E 1)N 1 c1 = (0 , 3, 0)1
0
1
( 2) = 2
(cB E 1)N 4 c4 = (0 , 3, 0)0
1
0
0 = 3
Increase x1
63
-
8/13/2019 Lp Notes 906
65/101
B 1b = E 1b =8
0
16
+0 6
1 6
2 6
=8
6
4
E 1N 1x1 =1
0
1
+0 0
1 0
2 0
x1 =1
0
1
x1
= x1 = min {8, 4} = 4, x1 is entering variable, x5 is leaving variable
xB =x3x2x1
Store for E 2 , ( 1/ 1, 0/ 1, 1/ 1;3) = ( 1, 0, 1;3)
xN = x4x5
, N =
0 0
1 0
0 1
, cB = (0 , 3, 2), cN = (0 , 0)
cB B 1 = cB E 2E 1 = 0, 3, (0, 3, 2) 1
0
1
E 1
= 0, (0, 3, 2)01
2
, 2 = (0 , 1, 2)
64
-
8/13/2019 Lp Notes 906
66/101
(cB E 2E 1)N 4 c4 = (0 , 1, 2)0
1
0
0 = 1
(cB E 2E 1)N 5 c5 = (0 , 1, 2)0
0
1
0 = 2
Increase x4
B 1b = E 2
8
6
4
=8
6
0
+ 1 4
0 4
1 4
=4
6
4
E 2E 1N 4x4 = E 2
0
0
0
+0 1
1 1
2 1
x4 = E 2
0
1
2
x4
=0
1
0
+( 1)( 2)
0( 2)
1( 2)
x4 =2
1
2
x4
= x4 = min {2, 6} = 2, x4 is entering variable, x3 is leaving variable
xB =x4x2x1
, N = {3, 5}
Store for E 3 , (1/ 2, 1/ 2, 2/ 2;1) = (1 / 2, 1/ 2, 1;1)
65
-
8/13/2019 Lp Notes 906
67/101
N =
1 0
0 0
0 1
, cB = (0 , 3, 2), cN = (0 , 0)
cB E 3E 2E 1 = (0, 3, 2)1/ 2
1/ 2
1
, 3, 2 E 2E 1
= 1/ 2, 3, ( 1/ 2, 3, 2) 1
0
1
E 1
= 1/ 2, ( 1, 3, 3/ 2)0
1
2
, 3/ 2
= ( 1/ 2, 0, 3/ 2)
(cB E 3E 2E 1)N 3 c3 = ( 1/ 2, 0, 3/ 2)10
0
0 = 1/ 2
(cB E 3E 2E 1)N 5 c5 = ( 1/ 2, 0, 3/ 2)0
0
1
0 = 3/ 2
Therefore, optimal solution
66
-
8/13/2019 Lp Notes 906
68/101
LU Factorization
Improvement over product form of inverse because dont have to complete
entire Gaussian EliminationUsed in many commercial codes
To use an upper triangular matrix to solve problem
B11 B12 B1m0 B22 B2m
.
..0 0 Bmm
x1
x2...
xm
=
b1b2...
bm
xm = bm /B mm
xm 1 = ( bm 1 Bm 1,m xm )/B m 1,m 1 . Substitute for xm and solve.
Repeat for xm 2 , xm 3 , . . . , x 1
To get B in upper triangular form1. Move rows around to get approximately upper triangular
(equivalent to switching two rows of I )
2. Perform partial Gaussian Elimination below main diagonal(equivalent to premultiplying by a lower triangular matrix)
Thus, RB = U , where R is lower triangular and U is upper triangular
Thus, B = R 1U = LU and R 1 is lower triangular
Store partial permutations of I and columns of R required to get U
Updating is similar to Product Form of Inverse, but only partial columnsare needed
67
-
8/13/2019 Lp Notes 906
69/101
Bounded Variables
Upper and lower bounds on the xs
To put all bounds in tableau adds 2 n constraints, n slack variables, nsurplus variables and n articial variables
Treat bounds similar to way Simplex Method does for nonnegativityconstraints
Each nonbasic variable k is at lower bound lk or upper bound uk
Partition A = ( B | N l | N u ) where xN l = lN l and xN u = uN u
(Book uses N 1 and N 2)Working basis B
xB = B 1b B 1N l lN l B 1N u uN u
z = cB xB + cN l xN l + cN u xN u
= cB (B 1b B 1N l xN l B 1N u xN u ) + cN l xN l + cN u xN u
= cB B 1b (cB B 1N l cN l )xN l (cB B 1N u cN u )xN u
Bounded Variable Simplex Tableau:
xB xN l xN u RHS
cB B 1b
z 0 cB B 1N l cN l cB B 1N u cN u (cB B 1N l cN l )lN l (cB B 1N u cN u )uN u
xB I B 1N l B 1N u B 1b B 1N l lN l B 1N u uN u
68
-
8/13/2019 Lp Notes 906
70/101
Improving z:
If there is j N l such that zj cj > 0, then we can improve z
If there is j N u such that zj cj < 0, then we can improve z
Find max max j N l {zj cj }, max j N u { zj + cj } , and let k be theindex associated with the maximum.
If max value is 0, then stop. We have an optimal solution.
Otherwise, change the value xk .
Let b = current RHS and z = current value
Change in Tableau:
If k N l , then xk increases
basic variable reaches lower bound
1 = min iB {(b lB )i / (B 1N k )i | (B 1N k )i > 0}
basic variable reaches upper bound
2 = min iB { (uB b/ (B 1N k )i | (B 1N k )i < 0}
Increase xk by = min { 1 , 2 , uk lk }
Update z = z ( zk ck )
Update RHS for pivot row = lk +
Update other RHS = b N k
If < u k lk , then do usual Simplex update for remaining tableau.
Otherwise, do nothing
69
-
8/13/2019 Lp Notes 906
71/101
If k N u , then xk decreases
basic variable reaches lower bound
1 = min iB { (b lB )i / (B 1N k )i | (B 1N k )i < 0}
basic variable reaches upper bound
2 = min iB {(uB b)i / (B 1N k )i | (B 1N k )i > 0}
Decrease xk by = min { 1 , 2 , uk lk }
Update z = z + ( zk ck )
Update RHS for pivot row = uk
Update other RHS = b + N k
If < u k lk , then do usual Simplex update for remaining tableau.Otherwise, do nothing
To nd an initial starting solution:
1. Set each nonbasic variable to bound of choice
2. Adjust RHS and z value to account for bounds
3. Multiply by -1 if necessary so that bi 0 for all i
4. If necessary, add articials and use Two Phase Method
70
-
8/13/2019 Lp Notes 906
72/101
-
8/13/2019 Lp Notes 906
73/101
Adjust z value and RHS
z = 2 18
x3 = 15 12
x4 = 10 6
l u
x1 x2 x3 x4 RHS
z 2 3 0 0 20
x3 1 2 1 0 3
x4 1 1 0 1 4
Increase x1 (Note that x2 must be negative to be considered)
= min {(3 0)/ 1, (4 0)/ 1, 8 1} = 3
= x3 is leaving variable
Adjust RHS and z value
z = 20 6
x1 = 1 + 3
x4 = 4 3
u l
x1 x2 x3 x4 RHS
z 0 1 2 0 26
x1 1 2 1 0 4
x4 0 1 1 1 1
Decrease x2
72
-
8/13/2019 Lp Notes 906
74/101
= min {(8 4)/ 2, (1 0)/ 1, 6 0} = 1
= x4 is leaving variable
Adjust RHS and z value
z = 26 1
x1 = 4 + 2 1
x2 = 6 1
l l
x1 x2 x3 x4 RHS
z 0 0 1 1 27x1 1 0 1 2 6
x2 0 1 1 1 5
Solution is optimal
73
-
8/13/2019 Lp Notes 906
75/101
DUALITY
Associated with each LP is another LP. Solving one is equivalent tosolving the other
Original problem is the Primal Problem. The associated problem is theDual Problem.
min cx max wb
sub. to Ax b sub. to wA c
x 0 w 0
Example:
min 2x1 3x2 max 8w1 6w2 16w3sub. to x1 8 sub. to w1 w3 2
x2 6 = w2 2w3 3
x1 2x2 16 w 0
x 0
Primal problem has n variables and m constraints
Dual problem has m variables and n constraints
Generally better to use one with smaller number of constraints (smallerB ) or one with constraints
74
-
8/13/2019 Lp Notes 906
76/101
Other Forms
Since Ax b Ax b
min cx max wb
sub. to Ax b sub. to wA c
x 0 w 0
max wb
sub. to wA c
w 0
Since Ax = b Ax b,Ax b,
min cx max w1b + w2b
sub. to Ax = b sub. to w1A + w2A c
x 0 w1 0
w2 0max wb
sub. to wA c
w unrestricted
75
-
8/13/2019 Lp Notes 906
77/101
MINIMIZATION MAXIMIZATION
Variables Constraints
0 0
unrestricted =
Constraints Variables
0
0
= unrestricted
76
-
8/13/2019 Lp Notes 906
78/101
max 2x1 + 3x2 min 8w1 + 6w2 + 16w3sub. to x1 8 sub. to w1 + w3 2
x2 6 = w2 + 2w3 3x1 + 2x2 16 w 0
x 0
Final Tableaus:
x1 x2 x3 x4 x5 RHS
z 0 0 1/ 2 0 3/ 2 28
x1 1 0 1 0 0 8
x2 0 1 1/ 2 0 1/ 2 4
x4 0 0 1/ 2 1 1/ 2 2
w1 w2 w3 w4 w5 RHS
z 0 2 0 8 4 28
w1 1 1/ 2 0 1 1/ 2 1/ 2
w3 0 1/ 2 1 0 1/ 2 3/ 2
77
-
8/13/2019 Lp Notes 906
79/101
Theorem. Dual of the Dual is the Primal.
min cx max wb
sub. to Ax b sub. to AT wT c
x 0 w 0
min cx
sub. to ( AT )T x b
x 0
Weak Duality Theorem. Let x0 be a feasible solution to the primal problem (min) and w0 be a feasible solution to the dual problem (max).Then, cx0 w0b.
Proof: Let x0 {x | Ax b, x 0} and w0 {w | wA c, w 0}. Then,cx0 (w0A)x0 = w0(Ax0) w0b.
78
-
8/13/2019 Lp Notes 906
80/101
Shadow price of constraint i The marginal value of relaxing constraint i
At optimality z = cB B 1b j N (zj cj )x j = wb j N (zj cj )x j
Suppose that the optimal solution is nondegenerate
Then, z/b i = wi = shadow price of constraint i = rate of change of the optimal value with a unit increase in bi
Alternative optimal degenerate bases in primal Alternative optimalextreme point solutions in dual
* With degeneracy, + z
b i = max {wji | wj is an optimal dual vertex }
z
b i = min {wji | w
jis an optimal dual vertex } *
Complementary Slackness Theorem. Let x be an optimal solution min{x | Ax b, x 0} and w be an optimal solution tomax {w | wA c, w 0}. Then, w(Ax b) = 0 and (c wA)x = 0 .
Proof. Follows from the Karush-Kuhn-Tucker optimality conditions.
Either constraint is tight or the shadow price is 0
79
-
8/13/2019 Lp Notes 906
81/101
Strong Duality Theorem. If either primal or dual has an optimal solution, then they both do. Further, the objective values are equal.
Proof: From Complementary Slackness Theorem,cx = wAx = wb.
Theorem. With regards to the Primal and Dual problems, exactly one of the following is true
1. Both have optimal solutions x and w respectively, and cx = wb.
2. One problem is unbounded and the other is infeasible.
3. Both problems are infeasible.
Proof. 1. follows from Strong Duality Theorem
2. follows from Weak Duality Theorem
3. follows from the following example
min x1 x2 max w1 + w2
sub. to x1 x2 1 sub. to w1 w2 1 x1 + x2 1 = w1 + w2 1
x 0 w 0
80
-
8/13/2019 Lp Notes 906
82/101
Economic Interpretation of the Dual
Primal
min total cost = j (unit cost of activity j ) (level of activity j )sub. to j (production of commodity i as result of 1 unit of j ) (levelof activity j ) demand for commodity iDual
max income =
i (unit price of commodity i) (demand for commodity i)sub. to j (production of commodity i as result of 1 unit of j ) (unit priceof commodity i) unit cost of activity jTotal charge for goods produced by 1 unit of j
Constraint implies fairness in pricing
81
-
8/13/2019 Lp Notes 906
83/101
Dual Simplex Method
Start with dual feasible problem: zj cj 0 for j N , some of the bi for
i M may be negativePivot keeping zj cj 0 for j N until bi 0 for all i
Very useful when new constraints are added to problem. The objectivefunction remains the same, but the current solution probably violatesthe new constraint, i.e., RHS is negative.
82
-
8/13/2019 Lp Notes 906
84/101
SENSITIVITY ANALYSIS
Interested in the sensitivity of the solution to changes in the parametervalues
Sensitivity Report (EXCEL)
Decision Final Reduced Objective Allowable Allowable
Variables Value Cost Coefficient Increase Decrease
changing cells zj cj c
Final Shadow Constraint Allowable Allowable
Constraints Value Price RHS Increase Decrease
zj cj b
Outputs from most other packages are similar
I. Objective Function: If coefficient changes by , does the primalsolution change?
Nonbasic Variable j N :
Cost change is 0 since the variable is nonbasic
zj cj 0 value of the z row in the nal tableau
Basic Variable t B :
Cost change is et B 1b = z + bt
Since zj cj = cB B 1N j cj for all j N ,
zj cj + et B 1N j 0 for some j N
83
-
8/13/2019 Lp Notes 906
85/101
Example:
min z(x) = 2x1 3x2sub. to x1 + x3 = 8
x2 + x4 = 6x1 + 2x2 + x5 = 16
x 0
x1 x2 x3 x4 x5 RHS
z 0 0 1/ 2 0 3/ 2 28
x4 0 0 1/ 2 1 1/ 2 2
x2 0 1 1/ 2 0 1/ 2 4x1 1 0 1 0 0 8
Nonbasic variables: New basis if c3 1/ 2 or if c5 3/ 2
Basic variables:
zN cN = ( 1/ 2, 3/ 2) et B 1
N = et
1/ 2 1/ 2
1/ 2 1/ 21 0
For x1 , cost change = 8/unit: 1/ 2 and 3/ 2 0 (e3)
1/ 2 = optimal for c1 2 + 1 / 2
For x2 , cost change = 4/unit: 1/ 2 1/ 2 and 3/ 2 1/ 2 ( e2)
1 3 = 3 1 c2 3 + 3
For x4 , cost change = 2/unit: 1/ 2 1/ 2 and 3/ 2 1/ 2 ( e1)
3 1 = 3 c4 1
84
-
8/13/2019 Lp Notes 906
86/101
II. RHS: If a RHS coefficient changes by , does the dual solution change(the primal basis change)?
Slack constraints: 0 cost
move until tight (-value of slack or surplus variable)
Tight constraints: Cost is shadow price
move until constraint reaches new extreme point basic variable becomes 0 (B 1b + B 1ej ) i 0 for some i B
Example:
For x2 6, change in cost = 0: x4 = 2 = 6 2 b2
For x1 8, change in cost = -1/2 per unit:
2
4
8
+1/ 2
1/ 2
1
0 4 8 = 4 b1 16
For x1 + 2x2 16, change in cost = -3/2 per unit:
2
4
8
+ 1/ 2
1/ 2
0
0 8 4 = 8 b3 20
85
-
8/13/2019 Lp Notes 906
87/101
Simultaneous Change of Several Parameters
For objective function coefficients or RHS
Let j = proposed change in coefficient j ,
Let j = maximum change in the same direction of j that does notviolate j s optimality range
If nj =1 j / j 1, then change does not violate optimality range
100% Rule: Basis doesnt change as long as total change is 100%
Objective Coefficient Example:
For N 3 , the rst column gives 1 / 2 1 and 1 2
If we take 0.50 of each change, thenc = ( 2 + 14 , 3
12 , 0, 0, 0) = ( 1
34 , 3
12 , 0, 0, 0)
RHS Example:
For x1 8, the second row gives 1 8For x1 + 2x2 16, the second row gives 8 3
If we take 0.75 of 1 and 0.25 of 3 , then b = (8 + 6 , 6, 16 2) = (14 , 6, 14)
86
-
8/13/2019 Lp Notes 906
88/101
III. Constraint Matrix A:
Nonbasic column:
Since zj cj = cB B 1N j cj for all j N ,
zj cj + cB B 1( ej ) 0 for j N
Example:
If change x1 + (1 + ) x3 = 8,
z3 c3 = 1/ 2 cB B 1 e3
= (0, 3, 2)
1/ 2 1 1/ 2
1/ 2 0 1/ 21 0 0
00
= (1 / 2, 0, 3/ 2)
0
0
1 If (1 + ) < 0, then increase x3 to make constraint slack
ex: If = 2, then x1 x3 = 8
Thus, (10 , 3, 2, 0, 0) FS and z = 29
For x2 + x3 + x4 = 6, 1/ 2 0 ,any value of is OK because constraint is slack
87
-
8/13/2019 Lp Notes 906
89/101
For x1 + 2x2 + x3 + x5 = 16
1/ 2 (1/ 2, 0, 3/ 2)0
0
1/ 3
If < 1/ 3, then increase x3 to make constraint slack
ex: If = 1/ 3, then x1 + 2x2 x3 / 3 + x5 = 16
Thus, (5 , 6, 3, 0, 0) FS and z = 28
Basic column:
Possible but complicated, B 1 changes which affects entries in everycolumn
88
-
8/13/2019 Lp Notes 906
90/101
Parametric Analysis
Find set of optimal solutions and values over a range of a parameters value
Objective function: Set cost coefficients to c + c where 0
RHS: Set RHS to b+ b where 0
Procedure:
1. Find optimal solution for = 0.
2. Use sensitivity results to determine how much must increase to moveto a new extreme point. If there is no such point, then stop.
3. Move to the new extreme point. Go to Step 2.
89
-
8/13/2019 Lp Notes 906
91/101
DECOMPOSITION
Dantzig-Wolf Decomposition
Large scale problem with special structure
2 types of constraints
general complicating constraints
constraints with special structure
Method: solve 2 separate types of LPsMaster problem (for the general constraints)
Subproblems (one for each special structure)
Master problem passes cost coefficients to Subproblems
Subproblems send trial solutions (columns) to Master problem
Iterate between the problems until Subproblems have no solution to send.At this point, an optimal solution is reached
Method is called a column generation procedure
90
-
8/13/2019 Lp Notes 906
92/101
min zP (x) = ( c1c2 . . . cT )x(P )
sub. to
A1 A2 . . . AT B1
B2. . .
BT
x1
x2...
xT
=
b0
b1
b2...
bT
For Subproblem i = 1 , 2, . . . , T , P i = {x i | B i x i = bi , x i 0}
Let xij = extreme points of P i for j = 1 , 2, . . . , t i anddik = extreme directions of P i for k = 1 , 2, . . . , l i
From Caratheodorys Theorem, each feasible x satisesx i =
t ij =1 ij x
ij + l ik =1 ik dik
where t ij =1 ij = 1, and , 0
Therefore, the Master Problem can be written as:
min zM (, ) =T
i=1
t i
j =1
(ci x ij ) ij +l i
k =1
(ci dik )ik
(M )
sub toT
i=1
t i
j =1
(Ai x ij ) ij +l i
k =1
(Ai dik )ik = b0
t i
j =1
ij = 1
, 0
91
-
8/13/2019 Lp Notes 906
93/101
zP = z
M when all extreme points and directions are included
Otherwise, zP zM
Each xij
or dik
represents a column in (M)
Suppose we have an optimal extreme point solution ( , ) to (M) for agiven set of xij and dik
Let the dual solution be(w1 , . . . , w n , 1 , . . . , T ) = ( ci x ij , ci dik )B B 1 = ciB B
1
w are dual multipliers of the constraints, are from convexity
constraints ij = 1Thus, zij cij = wAi x ij + i ci x ij for any new ij N zik cik = wAi dik ci dik for any new ik N
Solve each subproblem max( wAi ci )x i + i , sub to xi P i
If wi Ai x i + i ci x i > 0, then can improve Master Problem by addingeither the optimal extreme point (if one exists) or an extreme direction
(if unbounded). Go back and resolve Master Problem.If no candidate, then optimal
92
-
8/13/2019 Lp Notes 906
94/101
COMPLEXITY ISSUES ANDPOLYNOMIAL TIME ALGORITHMS
Problem Class Set of all instances of same type of problem.
Example: Linear Programming Problems
Purpose
Classify problem classes according to the computational effort required forsolution
Generally concerned with worst case performance
Size of Problem Amount of information necessary to represent aninstance
number of variables and constraints, and size of data
Computational Effort Steps required to solve problem
Good approximation is number of elementary operations, i.e. +, -, , , comparisons, etc.
93
-
8/13/2019 Lp Notes 906
95/101
Problem classes are categorized by computational effort in terms of input size using term with largest order, and ignoring constants and lower order terms.
Example: 3 n2 + 1000 n = O(n2) = Order n2 , where n is size of input
Example: Solving 0-1 Integer Program by total enumeration
1. Select value for x {0, 1}n
2. Check if x is feasible and nd value of solution
3. Repeat for all values of x
Step 2 requires O(mn ) effort
Process is repeated 2 n times
Complexity is O(2n mn )
Complexity representation is only concerned with asymptotic behavior asn . (Otherwise, lower order terms may dominate.)
Example: 10 10 n is O(n) while 10 10 n2 is O(n2)
The rst is smaller only when n 1020
94
-
8/13/2019 Lp Notes 906
96/101
Problem Classes
P Set of problem classes which can be solved in a time that is boundedby a polynomial function of the input size
Decision, Recognition or Threshold problem a problem that can beanswered with yes or no
Example: Does there exist an LP solution where cx K ?
Nondeterministic Procedure for recognition problem
1. Guess a solution
2. Check for feasibility
N P Set of problem classes that can complete NondeterministicProcedure in a time bounded by a polynomial function of the input size
95
-
8/13/2019 Lp Notes 906
97/101
Proposition. P N P
The MAJOR open question in Complexity Theory: P = N P ?
We assume not!
Reducible Problem class P 1 is reducible to P 2 if P 1 can be solved bysolving P 2
If P 1 is reducible to P 2 in polynomial time, then P 2 is at least as hard asP 1
N P -hard Problem class P 2 N P -hard if for every problem classP 1 N P -hard, P 1 transforms to P 2
Problems in N P -hard are at least as hard as those in N P
Frequently optimization versions of N P problems
N P -complete ( N PC ) Problem class P N PC if P N P andP N P -hard
Some N PC problem classes: General Integer Programming, TravelingSalesman Problem
To show problem class P N PC , must show that some N PC problemclass can be transformed to P in polynomial time
96
-
8/13/2019 Lp Notes 906
98/101
Simplex Method May Take Exponential Time
Variant of Klee-Minty (1972):
maximizen
j =1
10n j x j
sub. to 2i 1
j =1
10i j x j + x i 100i 1 i = 1 , 2, . . . , n
x 0
x = (0 , . . . , 0, 100n 1) z = 100 n 1
Steepest ascent rule requires 2 n 1 iterations
Largest increase rule requires 1 iteration (There exist problems wherelargest increase rule is exponential)
Example: n = 3
max 100x1 + 10x2 + x3sub. to x1 1
20x1 + x2 100
200x1 + 20x2 + x3 10000
x 0
Add slacks s1 , s2 and s3
97
-
8/13/2019 Lp Notes 906
99/101
max 100x1 + 10x2 + x3sub. to x1 + s1 = 1
20x1 + x2 + s2 = 100200x1 + 20x2 + x3 + s3 = 10000
x 0
s 0
Basis z
s1 s2 s3 0
x1 s2 s3 100
x1 x2 s3 900
s1 x2 s3 1000
s1 x2 x3 9000
x1 x2 x3 9100
x1 s2 x3 9900
s1 s2 x3 10000
All 23 = 8 extreme points are examined
However, Simplex Method takes O(n3) on average
98
-
8/13/2019 Lp Notes 906
100/101
Khachians Method (1979)
Finding a feasible solution is as hard as nding an optimal solution (use
binary search)
Ellipsoid Algorithm: nds a feasible solution if one exists by movingaround feasible region
1. Cover feasible region with an ellipsoid (make it sufficient large)
2. If center of ellipsoid is in feasible region, then STOP
Otherwise, nd a hyperplane through center that does notintersect feasible region
3. Construct a new ellipsoid based on where old ellipsoid intersectsthe hyperplane
Go to 2
Let E k = kth ellipsoidV ol(E k +1 )/V ol(E k ) 2 1/ 2( n +1) volume reduces exponentially
converges to a feasible solution in polynomial time
Never worked well in practice - invert large matrices to nd ellipsoid
99
-
8/13/2019 Lp Notes 906
101/101
Karmarkars Method (1984)
Affine Scaling procedure: Moves through feasible region
1. Start at an interior point
2. Transform space so that current point is approximately at centerof feasible region
3. Move in gradient direction
Go to 2 until near an extreme point
Large improvement using gradient if at centerConverge to an optimal extreme point in a log number of steps
Beats Simplex Method for certain classes of sparse matrices
top related