1
Design and Analysis of Algorithms
Yoram MosesLecture 7
April 22, 2010
http://www.ee.technion.ac.il/courses/046002
2
Linear Programming
3
Standard Form
Maximization All constraints are ≤ inequalities All variables are non-negative
Objective function
Constraints
Non-negativity constraints
4
Slack Form Maximization n+m variables and m constraints n “regular variables” and m “slack variables” All variables are non-negative All constraints, except for non-negativity, are equalities Slack variables appear only on LHS of equality constraints Each equality constraint has a single unique slack variable
5
Conversion to Slack Form: Example
Standard LP
Slack LP
6
Basic vs. Non-basic Variables
Slack variables basic variables Regular variables non-basic variables B = set of basic variables, N = set of non-basic variables
Ex: B = { 4,5,6 }, N = { 1,2,3 } |B| = m, |N| = n For a solution x, xB = basic part of x, xN = non-basic part of x
7
Short Matrix Form
|B| = m, |N| = n Constraints can be indexed by members of B For a solution x,
xB = basic part of x (i.e., xi for all i B) xN = non-basic part of x (i.e., xj for all j N)
8
Basic Solutions
Every slack form is associated with a basic solution: All non-basics are set to 0 (i.e., xN = 0)
All basics are set to corresponding free coefficients (i.e., xB = b)
Ex: x1 = 0, x2 = 0, x3 = 0, x4 = 30, x5 = 24, x6 = 36
9
Tight Constraints & Constraint Violation
Tight constraint: One in which the basic variable is forced to 0 in the basic solutionThe corresponding free coefficient is 0.
Violated constraint: One in which the basic variable is forced to be negative in the basic solution. the corresponding free coefficient is negative.
10
Basic Feasible Solutions
Basic Feasible Solution (BFS): A basic solution, which is feasible
Easy fact: A basic solution is feasible if and only if all free coefficients are non-negative.
Lemma: Every BFS corresponds to a vertex of the feasible region polytope.
11
Simplex Algorithm: Overview
Works in iterations At each iteration: transform one slack form
P into an equivalent slack form P’ Objective value of basic solution of P’ is
always at least as good as that of P Stop when reaching a local optimum
12
Moving to an Equivalent Slack Form How to increase the objective value of the BFS?
Increase from 0 the value of some non-basic variable, whose coefficient in the objective function is positive.
By how much? As much as possible without violating any of the constraints.
objective: 0
Can increase x1 by at most 9. Objective increases to 27.
13
Switching Basic with Non-basic Suppose that we increase a non-basic variable xi until
some constraint j becomes tight xj, the basic variable of the constraint j, becomes 0
We can thus switch between xi and xj
xi will become the basic variable of constraint j
xj will become a non-basic
When increasing x1 to 9, x6 becomes 0.
We switch between x1 and x6.
14
Switching Basic with Non-basic
We write x1 as a function of other non-basics and x6:
15
Switching Basic with Non-basic
We rewrite the objective function:
16
Switching Basic with Non-basic
We rewrite all the constraints as well, and obtain the following equivalent linear program:
17
Switching Basic with Non-basic
The new LP is equivalent to the previous LP We just rewrote x1 in terms of other variables
Basic solution: x1 = 9, x2 = 0, x3 = 0, x4 = 21, x5 = 6, x6 = 0 New objective value: 27
18
Example continued
Choose x3
First constraint to become tight is constraint 3.
objective: 27
objective: 27.75
19
Example continued
Choose x2
First constraint to become tight is constraint 2.
objective: 27.75
objective: 28
20
Example continued
No more non-basics whose coefficient in the objective function is positive We stop and output basic solution as the optimal solution Solution: x1 = 8, x2 = 4, x3 = 0, x4 = 18, x5 = 0, x6 = 0. Value: 28.
objective: 28
21
Back to the Standard Form
Solution: x1 = 8, x2 = 4, x3 = 0. Value: 28.
objective: 28
22
Pivoting
Pivot: a single iteration of the simplex algorithm1. Choose a non-basic variable xi whose coefficient in the
objective function is > 0 xi is called the “entering variable” If more than one exists, choose one according to some pivoting
rule
2. Find the first constraint j that will be violated when we increase the value of xi from 0 If more than one exists, choose one according to some pivoting
rule
3. Make xi the basic variable of constraint j, and make xj a non-basic variable xj is called the “leaving variable”
4. Write xi as a function of xj and the other non-basics5. Rewrite the objective function and the constraints
23
Pivoting: Geometric Intuition
Lemma: Pivoting corresponds to moving from one vertex of the feasible region to a neighbor vertex, whose objective value is at least as good.
24
Unbounded Programs
Sometimes it is possible to increase the value of the entering value unboundedly, without violating any constraint
In this case the optimal solution of the LP is unbounded Pivot will return “unbounded”
25
The Simplex Algorithm
1. find an initial BFS2. while there is a non-basic variable whose coefficient in
the objective function is > 03. run pivot4. if pivot returns “unbounded” return “unbounded”5. return BFS of current slack form as the optimal solution
Geometric view: Repeatedly move from a vertex of the feasible region to a better neighbor vertex, until a local maximum is reached.
Initial BFS is found by solving an auxiliary linear program (read section in book)
26
Simplex Analysis: Correctness
If LP is infeasible, S’x will fail to find an initial BFS If LP is unbounded, Pivot will return “unbounded” If LP has a bounded optimal solution, it has one at
a vertex Simplex will reach a local maximum vertex Local maximum vertex must be a global maximum Hence, Simplex will output an optimal solution
27
Simplex Analysis: Running Time
We have not specified the two “pivoting rules” For choosing the entering variable For choosing the leaving variable
Degeneracy: objective value of BFS does not improve in an invocation of Pivot
Unwise pivoting rules may lead to infinite loops (i.e., everlasting degeneracy)
28
Degeneracy
objective: 0 entering: x1
leaving: x4
objective: 8 entering: x3
leaving: x5
objective: 8 entering: x2
leaving: x1
29
Pivoting Rules
Bland’s rule: choose entering/leaving variable with smallest index.
Lemma: If Simplex uses Bland’s rule, it never cycles.
Conclusion: Simplex has at most iterations. Theorem [Kalai]: There is a randomized pivoting
rule, with which Simplex runs for a sub-exponential number of iterations in expectation.
Open problem: Is there a pivoting rule with which Simplex runs in polynomial time?
30
End of Lecture 8