lift-and-project cuts in cplex 12.5.1
DESCRIPTION
This presentation given at INFORMS in November 2014 explains how Lift-and-Project cuts are implemented in CPLEX 12.5.1, and their performance impact.TRANSCRIPT
Decision OptimizationDecision Optimization
Lift-and-Project Cuts in CPLEX 12.5.1Andrea TramontaniCPLEX Optimization, IBM
INFORMS Fall Conference 2013
2 © 2013 IBM Corporation
Decision OptimizationDecision Optimization
Disclaimer
IBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM’s sole discretion. Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision. The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract. The development, release, and timing of any future features or functionality described for our products remains at our sole discretion.
Performance is based on measurements and projections using standard IBM®benchmarks in a controlled environment. The actual throughput orperformance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.
3 © 2013 IBM Corporation
Decision OptimizationDecision Optimization
Literature review– The Cut Generating Linear Program (CGLP)– L&P cuts à la Balas-Perregaard– L&P cuts and Gomory Mixed Integer (GMI) cuts– L&P cuts à la Bonami
L&P cuts in CPLEX 12.5.1: Performance Impact
Outline
4 © 2013 IBM Corporation
Decision OptimizationDecision Optimization
Lift and Project Cuts Beginnings
Invented by Balas in the ’70s:– E. Balas, Disjunctive programming. Ann. Discrete. Math. 5, 3–51, 1979.
Succesfully exploited in practice in the ’90s:– E. Balas, S. Ceria, and G. Cornuejols, A lift-and-project cutting plane algorithm for
mixed-integer programs. Math. Progr. 58, 295-324, 1993.– E. Balas, S. Ceria, and G. Cornuejols: Mixed 0-1 programming by lift-and-project in a
branch-and-cut framework. Man. Science 42, 1229-1246, 1996.
5 © 2013 IBM Corporation
Decision OptimizationDecision Optimization
Lift and Project Cuts: The Separation Problem
Given:
The LP relaxation of a MILP, defined by the underlying polyhedron P:
A fractional solution x* in P (typically, a vertex of P) to cut off
A binary fractional variable xj*: xj in {0,1}, 0 < xj* < 1
Find a violated cut
α x ≥ β
using the simple split disjunction
xj ≤ 0 OR xj ≥ 1
(P) A x ≥ bx ≥ 0
6 © 2013 IBM Corporation
Decision OptimizationDecision Optimization
The Cut Generating LP (CGLP) of Balas, Ceria and Cornuejols
1. Consider the polyhedra P0 and P1 induced by the disjunction:
2. Separate a cut valid for the disjunctive polyhedron
Q = conv (P0 ⋃ P1)
by solving the following CGLP:
3. Strengthen the cut by Monoidal Strengthening (Balas and Jeroslow, Eur. J. Oper. Res. 1980)
P0
(u) A x ≥ b(w) x ≥ 0(u0) - xj ≥ 0
P1
(v) A x ≥ b(z) x ≥ 0(v0) xj ≥ 1
α = u A + w – u0 ejβ = u b
u, w, u0 ≥ 0
α = v A + z + v0 ejβ = v b + v0
v, z, v0 ≥ 0
min α x* - β
P0
P1
x*
αx ≥ β
7 © 2013 IBM Corporation
Decision OptimizationDecision Optimization
CGLP: The Role of Normalization
The CGLP is unbounded by construction (if a violated cut exists)
Can be normalized in many different ways: see, e.g.,– Balas, Ceria and Cornuejols (Math. Prog.1993, Man. Science 1996)– Ceria and Soares (Tech. Report, 1997)
The normalization is the real objective function and makes the difference: see, e.g., – Fischetti, Lodi and T. (Math. Prog. 2011)
Two popular normalization constraints are:– The weak normalization (actually, the one used in CPLEX):
u0 + v0 = 1 (1)
– The strong normalization (Balas, Math. Prog. 1997):
1 u + 1 v + 1 w + 1 z + u0 + v0 = 1 (2)
8 © 2013 IBM Corporation
Decision OptimizationDecision Optimization
Lift and Project Cuts à la Balas-Perregaard (BP L&P cuts)
Balas and Perregaard (Math. Prog. 2003) adopt normalization (2) and:
1. Provide a precise correspondence between feasible CGLP bases and (possibly infeasible) bases of the tableau of the original problem
2. Characterize Gomory Mixed Integer (GMI) cuts from the tableau as CGLP bases
3. Develop an efficient approach for solving the CGLP by a sequence of pivoting in the tableau of the original space.
Different variants of the idea avaliable in commercial and open-source solvers:
FICO XPRESS, see Balas and Perregaard (Math Progr. 2003)
CBC (COIN-OR B&C), see Balas and Bonami (Math. Prog. Comp. 2009)
Warmstart CGLP with GMI basis
Pivots in the spaceA x – I s = b
Optimal CGLPsolution
9 © 2013 IBM Corporation
Decision OptimizationDecision Optimization
L&P Cuts à la Balas-Perregaard vs. Tableau GMI cuts
GMI cuts vs. Balas-Perregaard L&P cuts in Cgl 0.57.1. Test set: 52 MIPLIB3.0 problems.
Results taken from Pierre Bonami‘s talk in Bologna (Italy), 2012
%gap closed: 100 x (zcut - zinitial)/(zopt - zinitial), arithmetic means
Sep. Time: geometric means
Sep. Time
%gap closed # cuts # active density k
GMI - 1 round 0.01 28.79 20.60 9.08 95.95 4.53 x 1e+4
L&P - 1 round 0.07 30.11 25.33 11.90 79.44 5.19 x 1e+4
GMI - 10 rounds 0.06 49.33 212.52 20.62 216.12 5.64 x 1e+6
L&P - 10 rounds 2.31 60.77 310.49 38.87 170.40 2.37 x 1e+6
L&P closure 1.08 60.80 309.49 60.98 86.75 6.02 x 1e+5
10 © 2013 IBM Corporation
Decision OptimizationDecision Optimization
L&P Cuts à la Balas-Perregaard: why better than tableau GMI cuts?
No dominance relationships between tableau GMI cuts and BP L&P cuts
But just a matter of normalization:
Empirical observation from Fischetti, Lodi and T. (Math. Prog. 2011):
Normalization (2) is much better than (1) because it:– Enforces the separation of cuts with sparse dual and primal support– Tends to produce low-rank inequalities – Breaks the correlation between optimization and separation
Tableau GMI cuts are Optimal CGLP cuts with u0 + v0 = 1
11 © 2013 IBM Corporation
Decision OptimizationDecision Optimization
Why normalization (2) better than (1)?
Normalization (2):
Enforces the separation of sparse cuts:
Solving CGLP is equivalent to fix the violation as a constraint and use normalization (2) as the objective function: i.e., minimize the L-1 norm of the multipliers vector.
α = u A + w – u0 ejβ = u b
u, w, u0 ≥ 0
α = v A + z + v0 ejβ = v b + v0
v, z, v0 ≥ 0α x* - β= -1
min 1 u + 1 v + 1 w + 1 z + u0 + v0
12 © 2013 IBM Corporation
Decision OptimizationDecision Optimization
1 u + 1 v + 1 w + 1 z + u0 + v0 = 1
α = u A + w – u0 ejβ = u b
u, w, u0 ≥ 0
α = v A + z + v0 ejβ = v b + v0
v, z, v0 ≥ 0
Why normalization (2) better than (1)?
Normalization (2):
Introduce a natural scaling that penalizes the separation of high rank inequalities
1. Each constraint is penalized in the normalization, but the penalty is constant and therefore affected by the scaling of the constraint.
The cut obtained using ai x >= bi with ui could be obtained using K ai x >= K bi with 1/K uithe smaler the norm, the larger the penalization
2. Cuts are obtained by aggregating constraints with multipliers that sum to 1:Each cut is automatically scaled down w.r.t. the constraints used to generate itIt will be penalized in the subquent separation
minα x* - β
13 © 2013 IBM Corporation
Decision OptimizationDecision Optimization
Normalization (2) vs. Normalization (1): four pictures on one MIPLIB3.0 model
Average density of (u,v)
Average cut density
Dual bound evolution
Cut rank
14 © 2013 IBM Corporation
Decision OptimizationDecision Optimization
L&P Cuts à la Balas-Perregaard vs. Tableau GMI cuts (Cont.d)
GMI cuts vs. Balas-Perregaard L&P cuts in Cgl 0.57.1. Test set: 52 MIPLIB3.0 problems.
Results taken from Pierre Bonami‘s talk in Bologna (Italy), 2012
%gap closed: 100 x (zcut - zinitial)/(zopt - zinitial), arithmetic means
Sep. Time, # cuts, # active, density: geometric means
k: condition number of the final optimal basis
Sep. Time
%gap closed # cuts # active density k
GMI - 1 round 0.01 28.79 20.60 9.08 95.95 4.53 x 1e+4
L&P - 1 round 0.07 30.11 25.33 11.90 79.44 5.19 x 1e+4
GMI - 10 rounds 0.06 49.33 212.52 20.62 216.12 5.64 x 1e+6
L&P - 10 rounds 2.31 60.77 310.49 38.87 170.40 2.37 x 1e+6
L&P closure 1.08 60.80 309.49 60.98 86.75 6.02 x 1e+5
15 © 2013 IBM Corporation
Decision OptimizationDecision Optimization
Lift and Project Cuts: Our Idea
One more trivial empirical observation:
Separating rank-1 cuts:– Tends to produce cleaner and sparser cuts– Enforces the separation of low-rank inequalities (not a big surprise)– Breaks the correlation between optimization and separation
Sounds like normalization (2)
Separate some more rank-1 L&P cuts with u0 + v0 = 1
Keep tableau GMI cuts as they are (if your filter likes them enough), and
16 © 2013 IBM Corporation
Decision OptimizationDecision Optimization
Lift and Project Cuts à la Bonami (this is what we did!)
Bonami (Math. Prog. Comp., 2012):
1. Adopts normalization u0 + v0 = 1
2. Applies some algebra to get rid of u0 and v0 in the CGLP
3. Takes the dual of the simplified CGLP and gets the so called Membership LP (MLP):
4. Provides promising computational results on MIPLIB 3.0 and MIPLIB 2003 models
The (strengthened) L&P closure:– Can be computed in practice– Provides a good approximation of the split closure
(MLP) max yj - δ*
(v) b δ* ≤ A y ≤ b δ* + s* (u)(z) 0 ≤ y ≤ x* (w)
s* := A x* - bδ* := xj*
17 © 2013 IBM Corporation
Decision OptimizationDecision Optimization
Balas-Perregaard L&P cuts, Tableau GMI cuts, Rank-1 L&P cuts
GMI cuts vs. BP L&P cuts in Cgl 0.57.1 vs. Rank-1 L&P closure. Test set: 52 MIPLIB3.0 problems.
Results taken from Pierre Bonami‘s talk in Bologna (Italy), 2012
%gap closed: 100 x (zcut - zinitial)/(zopt - zinitial), arithmetic means
Sep. Time, # cuts. # active, density: geometric means
k: condition number of the final optimal basis
Sep. Time
%gap closed # cuts # active density k
GMI - 1 round 0.01 28.79 20.60 9.08 95.95 4.53 x 1e+4
L&P - 1 round 0.07 30.11 25.33 11.90 79.44 5.19 x 1e+4
GMI - 10 rounds 0.06 49.33 212.52 20.62 216.12 5.64 x 1e+6
L&P - 10 rounds 2.31 60.77 310.49 38.87 170.40 2.37 x 1e+6
L&P closure 1.08 60.80 309.49 60.98 86.75 6.02 x 1e+5
18 © 2013 IBM Corporation
Decision OptimizationDecision Optimization
Lift and Project Cuts in CPLEX 12.5.1
Bonami MLP to separate rank-1 cuts only, plus some tricks:
Heuristic reduction of MLP to– Speed up the separation– Get rid of constraints you don‘t want to use to produce the cut
Clever selection of the disjunctions to use
Not explored but worthy to mention:
Dash and Goycoolea (Math. Prog. Comp., 2010)
Fischetti and Salvagnin (Math. Prog. Comp., 2011)
19 © 2013 IBM Corporation
Decision OptimizationDecision Optimization
0.95 0.
990.
95 0.98
0.98
0.93 0.97
0.92 0.96
0.95
0.87 0.
930.
86 0.92
0.92
0
0.2
0.4
0.6
0.8
1
>1s >10s >100s
seed 1
seed 2
seed 3
seed 4
seed 5
L&P Cuts in CPLEX 12.5.1: Performance Impact All ModelsAll Models
≈≈ 18641864modelsmodels
≈≈ 10891089modelsmodels
≈≈ 567567modelsmodels
1.03x1.03x 1.05x1.05x 1.10x1.10x
Date: 23 May 2013Testset: 3243 models Machine: Intel X5650 @ 2.67GHz, 24 GB RAM, 12 threadsTimelimit: 10,000 sec
20 © 2013 IBM Corporation
Decision OptimizationDecision Optimization
0.91 0.
98
0.91 0.96
0.95
0.87 0.
95
0.86 0.
93
0.92
0.80 0.
88
0.79 0.
88
0.87
0
0.2
0.4
0.6
0.8
1
>1s >10s >100s
seed 1
seed 2
seed 3
seed 4
seed 5
0.95 0.
990.
95 0.98
0.98
0.93 0.97
0.92 0.96
0.95
0.87 0.
930.
86 0.92
0.92
0
0.2
0.4
0.6
0.8
1
>1s >10s >100s
seed 1
seed 2
seed 3
seed 4
seed 5
L&P Cuts in CPLEX 12.5.1: Performance Impact
≈≈ 948 (51%)948 (51%)modelsmodels
≈≈ 617 (57%)617 (57%)modelsmodels
≈≈ 362 (64%)362 (64%)modelsmodels
All ModelsAll Models
Affected ModelsAffected Models
≈≈ 18641864modelsmodels
≈≈ 10891089modelsmodels
≈≈ 567567modelsmodels
Date: 23 May 2013Testset: 3243 models Machine: Intel X5650 @ 2.67GHz, 24 GB RAM, 12 threadsTimelimit: 10,000 sec
1.03x1.03x
1.10x1.10x1.06x1.06x 1.16x1.16x
1.05x1.05x 1.10x1.10x