Lecture 9 : Simplex method and duality

(Chapter 16, 17 from the textbook)Xiaoqun Zhang

Shanghai Jiao Tong University

Last updated: November 21, 2018

1/19

Simplex method

Outline

Simplex method

Duality

2/19

Simplex method

Canonical form

min cTxs.t. Ax = b

x ≥ 0⇒

min cTB xB + cT

D xD

s.t. [B,D][xBxD

]= b

xB, xD ≥ 0

Augmented matrix[

B D bcT

B cTD 0

]Elementary row operations and transform to canonical form equivalent

to multiply[B−1 0

0 1

]and

[Im 0−cT

B 1

], we obtain the canonical form

corresponding to B[Im B−1D B−1b0 cT

D − cTB B−1D −cT

B B−1b

]xB = B−1b values of the variable corresponding to the basis B;cT

D − cTB B−1D: reduced costs; cT

B B−1b: the value of the objectivefunction.

3/19

Simplex method

Simplex method

Choose Initial basis: A = [B,D], x = [xB, xD]T , c = [cB, cD]T .Former a canonical form:

Represent xB in terms of xD: Ax = [B, D][

xBxD

]thus

xB = B−1b − B−1DxD;Represent the objective value in terms of xD:

z = cT x = cTB B−1b + (cT

D − cTB B−1D)xD , z0 + (cT

D − yT D)xD.

Here z0 = cTB B−1b, yT = cT

B B−1 is called simplex multiplier,rT

D = cTD − yT D is called the reduced cost coefficients corresponding to

the nonbasic variables.Optimality test/selecting entering variables: z = z0 + rT

D xDIf rD ≥ 0, then the basic feasible solution corresponding to B is optimal,stop.If there is a component of rD is negative, then choose the most negativerq, xq is the entering variable.

4/19

Simplex method

Simplex method (cont...)

Decide the step length by ratio test:xB = B−1b − B−1Dx , b − xqAq for b = B−1b, Aq = B−1aq.

If Aq ≤ 0 for all the column, then unbounded (choose xq →∞, xBremains feasible and the cost z = z0 + rqxq → −∞).If Aq � 0, let p = arg min

1≤i≤m{ bi

Aiq, Aiq > 0} and the leaving variable is xp.

Row operations: remove xp from the basis and add xq to the basis.Update B and xB correspondingly (update the canonical matrix bypivoting about (p, q) element.)Repeat the process until optimal or unbounded.

5/19

Simplex method

In tabular formmax 7x1 + 6x2

s.t. 2x1 + x2 ≤ 3x1 + 4x2 ≤ 4x1, x2 ≥ 0

a1 a2 a3 a4 b2 1 1 0 31 4 0 1 4

cT -7 -6 0 0 0

⇒

a1 a2 a3 a4 b1 1/2 1/2 0 3/20 7/2 -1/2 1 5/2

cT 0 -5/2 7/2 0 21/2

⇒

a1 a2 a3 a4 b1 0 4/7 -1/7 8/70 1 -1/7 2/7 5/7

cT 0 0 22/7 5/7 86/7

z∗ = 86/7 and x∗ = [8/7, 5/7]T .

6/19

Simplex method

Two-phrase simplex method

In simplex method, one needs to start from an initial BFS.Transferring the problem into canonical form A = [B,D]⇒ [Im,B−1D]is not always easy unless for Ax ≤ b constraints, naturally there is asubmatrix Im.For standard form Ax = b, x ≥ 0. Consider an artificial problem:

min y1 + y2 + · · ·+ ym

s.t. [A, Im][xy

]= 0[

xy

]≥ 0

7/19

Simplex method

PropsitionThe original LP problem has a basic feasible solution if and only if theassociated artificial problem has an optimal feasible solution with objectivefunction value zero.

Proof:⇒ If the original problem has a BFS x, then the vector

[x0

]is a BFS to the

artificial problem and this solution has an objective function value at zeroand thus optimal.⇐ Suppose that the artificial problem has an optimal feasible solution with

value 0. Then the solution must have the form[x0

]for x ≥ 0. It is a feasible

solution of the original problem and there exists a BFS by the fundamentaltheorem of LP.

8/19

Simplex method

Example

min 2x1 + 3x2

s.t. 4x1 + 2x2 ≥ 12x1 + 4x2 ≥ 6x1, x2 ≥ 0

min 2x1 + 3x2 + 0x3 + 0x4

s.t. 4x1 + 2x2 − x3 = 12x1 + 4x2 − x4 = 6x1, x2, x3, x4 ≥ 0

9/19

Simplex method

SolutionPhase I: canonical form corresponding to basis {x5, x6}Tableau 1:

a1 a2 a3 a4 a5 a6 b4 2 -1 0 1 0 121 4 0 -1 0 1 6-5 -6 1 1 0 0 -18

Tableau 2:

a1 a2 a3 a4 a5 a6 b7/2 0 -1 1/2 1 -1/2 91/4 1 0 -1/4 0 1/4 3/2-7/2 0 1 -1/2 0 3/2 -9

Tableau 3:

a1 a2 a3 a4 a5 a6 b1 0 -2/7 1/7 2/7 -1/7 18/70 1 1/14 -2/7 -1/14 2/7 6/70 0 0 0 1 1 0

Both of the artificial variables have been driven out of the basisand the current basic feasible solution is optimal.

Phase II: Remove the artifi-cial variable and update theobjective function coefficients

a1 a2 a3 a4 b1 0 -2/7 1/7 18/70 1 1/14 -2/7 6/7

cT 2 3 0 0 0In canonical form:

a1 a2 a3 a4 b1 0 -2/7 1/7 18/70 1 1/14 -2/7 6/70 0 5/14 4/7 -54/7

The optimal solution is x∗ =[18/7, 6/7, 0, 0]T and the optimalcost is 54/7.

10/19

Simplex method

Degeneracy

max 2.3x1 + 2.15x2 − 13.55x3 − 0.4x4

s.t. 0.4x1 + 0.2x2 − 1.4x3 − 0.2x4 ≤ 0− 7.8x1 − 1.4x2 + 7.8x3 + 0.4x4 ≤ 0x1, · · · , x4 ≥ 0

a1 a2 a3 a4 a5 a6 b0.4 0.2 -1.4 -0.2 1 0 0-7.8 -1.4 7.8 0.4 0 1 0

cT -2.3 -2.15 13.55 0.4 0 0 0a1 a2 a3 a4 a5 a6 b1 0.5 -3.5 -0.5 2.5 0 00 2.5 -19.5 -3.5 19.5 1 0

cT 0 -1 5.5 -0.75 5.75 0 0a1 a2 a3 a4 a5 a6 b1 0 0.4 0.2 -1.4 -0.2 00 1 -7.8 -1.4 7.8 0.4 0

cT 0 0 -2.3 -2.15 13.55 0.4 0

Tableau 3 is a simple circular shiftof the first one. After 6 iterations,we will get the initial tableau!Cycling.Bland’s rule for anti-cycling:choose the leaving variable q andthe entering variable p:

q = min{i : ri < 0}

p = min{i :bi

Aiq: Aiq > 0}

11/19

Simplex method

Other issues

Revised simplex: Useful for m � n. Does not need to update B−1Dand maintain [B−1,B−1b, yj ].

Sensitivity analysis; answer what if c, b,A change slightly, how it willaffect the solution?

Big-M method: another way to obtain BFS or check feasibility.

12/19

Duality

Outline

Simplex method

Duality

13/19

Duality

Duality

Associated with every linear programming problem is a correspondingdual linear programming problem.Solving an LP problem via its dual may be simpler in certain cases andprovides further insight.

14/19

Duality

Example

Primal problem:max 2x1 + 5x2 + x3

s.t 2x1 − x2 + 7x3 ≤ 6x1 + 3x2 + 4x3 ≤ 93x1 + 6x2 + x3 ≤ 3x1, x2, x3 ≥ 0

Dual problem:

min 6λ1 + 9λ2 + 3λ3

s.t 2λ1 + λ2 + 3x3 ≥ 2− λ1 + 3λ2 + 6λ3 ≥ 57λ1 + 4λ2 + λ3 ≥ 1λ1, λ2, λ3 ≥ 0

15/19

Duality

Properties of dual problems

Lemma (Weak duality)

Suppose that x and λ are feasible solutions to primal and dual LP problems,respectively (either in the symmetric or asymmetric form), then cTx ≥ λTb.

Proof:Asymmetric form: For x primal feasible, Ax = b, x ≥ 0. For λ dualfeasible. λTA ≤ cT . Then λTAx ≤ cTx which leads to λTb ≤ cTx.Symmetric form: similar idea.

16/19

Duality

Remarks

The optimal cost of the dual is less than or equal to the optimal cost ofthe primal, i.e. maximum≤minimum.One problem is unbounded, then the other problem has no feasiblesolution.

Example:

min xs.t. x ≤ 1

max λ

s.t. λ = 1λ ≤ 0

17/19

Duality

Strong duality

TheoremSuppose that x0,λ0 are feasible solutions to the primal and dual respectively.If cTx0 = λT

0 b, then x0 and λ0 are optimal solutions to their respectivelyproblems.

TheoremIf the primal problem (either in symmetric or asymmetric form) has anoptimal solution, then so does the dual and the optimal values of the theirrespective objective functions are equal.

Remarks: The dual solution λ∗ = cTB B−1 can be obtained from the final

tableau.

18/19

Duality

Complementary slackness conditionTheoremThe feasible solutions x and λ to a dual pair of problems (either insymmetric or asymmetric form) are optimal if and only if

(cT − λTA)x = 0λT(Ax − b) = 0

min − (2x1 + x − 2 + 7x3 + 4x4)s.t. x1 + x2 + x3 + x4 = 26

x1, x2, x3, x4 ≥ 0

max 26λs.t. λ ≤ −2

λ ≤ −1λ ≤ −7λ ≤ −4

The solution of the dual is clearly λ = −7. By the complementary slacknesscondition:

[1, 1, 1, 1]x = 26; x ≥ 0; [5, 6, 0, 3]x = 0Thus the primal solution x = [0, 0, 26, 0]T .

19/19