APPLIED INTEGER PROGRAMMING,

MODELING AND SOLUTION

CHAPTERS 14.4-14.15

Sara Gestrelius

Der-San Chen, Robert G. Batson, Yu Dang

www.sics.se

CHAPTER 14.5: LAGRANGIAN

DUAL

www.sics.se

LAGRANGIAN RELAXATION

(IP)

Complicated constraints.

Easy constraints.

nZ

ts

x

bxA

bxA

cx

22

11..

min

www.sics.se

LAGRANGIAN RELAXATION

(IP) (LRIP)

nZ

ts

x

bxA

bxA

cx

22

11..

min

nZ

ts

x

bxA

xAbucx

22

11

..

)(min

www.sics.se

LAGRANGIAN RELAXATIONNOTE: Less than or equal

to zero if original IP

feasible!

(IP) (LRIP)

nZ

ts

x

bxA

bxA

cx

22

11..

min

nZ

ts

x

bxA

xAbucx

22

11

..

)(min

www.sics.se

LAGRANGIAN RELAXATION

nZ

ts

x

bxA

bxA

cx

22

11..

min

nZ

ts

x

bxA

xAbucx

22

11

..

)(min

NOTE: Less than or equal

to zero if original IP

feasible!

So the Lagrangian relaxation optimal value can be used

as a lower bound to the original IP. But we want a tight

bound…

www.sics.se

LAGRANGIAN DUAL PROBLEM

22

11Zxu

bxA

xAbucx

..

)(minmax0

ts

n

…and to find the largest lower bound we solve the

Lagrangian dual problem.

www.sics.se

LAGRANGIAN DUAL PROBLEM

22

11Zxu

bxA

xAbucx

..

)(minmax0

ts

n

…and to find the largest lower bound we solve the

Lagrangian dual problem.

Dual variable aka

Lagrange multiplier

www.sics.se

LAGRANGIAN DUAL PROBLEM

22

11Zxu

bxA

xAbucx

..

)(minmax0

ts

n

…and to find the largest lower bound we solve the

Lagrangian dual problem.

Dual variable aka

Lagrange multiplier

22

1Z

1

bxA

xuAcub

..

)(minmax0

ts

nxu

www.sics.se

LAGRANGIAN DUAL PROBLEM

Proposition 14.8: Given a specific , if the

following two conditions are satisfied, then the

optimal vector of LRIP(u) is optimal for IP:

1.

2. whenever

11 )(* bxA u

iiu 111 )(* bbxA 0iu

(u)*x

1mRu

www.sics.se

EQUALITY INSTEAD OF

INEQUALITY

22

1Z

1

bxA

xuAcub

..

)(minmax

ts

nxu

nZ

ts

x

bxA

bxA

cx

22

11..

minNo sign restriction on u!

www.sics.se

LP DUAL OF IP

Proposition 14.5: The integer program (IP) and the dual of its

linear programming relaxation (Relaxed IP) form a weak dual

pair.

IP

Relaxed IP

Primal (minimization)

Dual (maximization)

www.sics.se

CHAPTER 14.6: PRIMAL –DUAL

SOLUTION VIA BENDERS’

PARTITIONING

www.sics.se

PRIMAL-DUAL SOLUTION VIA

BENDERS’ PARTITIONING

integerand0,0

..

min

2

21

yx

bDy

byAxA

ycxc

2

11ts

MIPs are hard. Let’s try to partition it.

www.sics.se

PRIMAL-DUAL SOLUTION VIA

BENDERS’ PARTITIONING

integerand0,0

..

min

2

21

yx

bDy

byAxA

ycxc

2

11ts

integerand0

..

)(min 2

y

bDy

yyc

2ts

q

0

..

min)(

21

1

x

yAbxA

xc

1ts

yq

MASTER SUB-PROBLEM

www.sics.se

PRIMAL-DUAL SOLUTION VIA

BENDERS’ PARTITIONING

integerand0,0

..

min

2

21

yx

bDy

byAxA

ycxc

2

11ts

integerand0

..

)(min 2

y

bDy

yyc

2ts

q

0

..

min)(

21

1

x

yAbxA

xc

1ts

yq

It’d be great if the y’s weren’t in the constraints of q(y).

DUAL!

MASTER SUB-PROBLEM

www.sics.se

DUAL OF SUB-PROBLEM

0

..

)'(max

11

2

u

cuA

yAbu 1

ts

0

..

min)(

21

1

x

yAbxA

xc

1ts

yq

SUB-PROBLEM

www.sics.se

DUAL OF SUB-PROBLEM

0

..

)'(max

11

2

u

MuE

cuA

yAbu 1

ts0

..

min)(

21

1

x

yAbxA

xc

1ts

yq

SUB-PROBLEM

Avoid unboundedness.

www.sics.se

PRIMAL-DUAL SOLUTION VIA

BENDERS’ PARTITIONING

0

..

)'(max

11

2

u

MuE

cuA

yAbu 1

ts

1. For every y’ we will pick out a vertex ui of the dual

feasible solution space. From weak duality:

)()( 2yAbuy 1 iq

Dual of sub-problem

www.sics.se

PRIMAL-DUAL SOLUTION VIA

BENDERS’ PARTITIONING

0

..

)'(max

11

2

u

MuE

cuA

yAbu 1

ts)()( 2yAbuy 1 iq

)()( 2yAbuycyyc 122 iqz

2. So, we have that the master objective

Dual of sub-problem1. For every y’ we will pick out a vertex ui of the dual

feasible solution space. From weak duality:

www.sics.se

PRIMAL-DUAL SOLUTION VIA

BENDERS’ PARTITIONING

0

..

)'(max

11

2

u

MuE

cuA

yAbu 1

ts)()( 2yAbuy 1 iq

)()( 2yAbuycyyc 122 iqz

2. So, we have that the master objective

3. ADD CONSTRAINT TO MASTER!

integerand0

..

)(min 2

y

bDy

yyc

2ts

q

Dual of sub-problem

Master

1. For every y’ we will pick out a vertex ui of the dual

feasible solution space. From weak duality:

www.sics.se

PRIMAL-DUAL SOLUTION VIA

BENDERS’ PARTITIONING

0

..

)'(max

11

2

u

MuE

cuA

yAbu 1

ts)()( 2yAbuy 1 iq

)()( 2yAbuycyyc 122 iqz

2. So, we have that the master objective

3. ADD CONSTRAINT TO MASTER!

integerand0

...1)(..

min

22

y

bDy

yAbuyc

2

1 Tizts

z

i integerand0

..

)(min 2

y

bDy

yyc

2ts

q

Dual of sub-problem

Master

For every y’ we will pick out a vertex ui of the dual feasible

solution space. From weak duality:

www.sics.se

THE MASTER

integerand0

...1)(..

min

22

y

bDy

yAbuyc

2

1 Tizts

z

i

The number of vertices (T) can be very large,

so instead of solving the Master with all T, start

with one and then generate more constraints

as needed.

www.sics.se

BENDERS’ ALGORITHM FOR MIP

Step 0: Initialization

Set 𝑡 = 1,𝐵𝑢 = +∞ and select some 𝜀 (convergence criterion). Select some u1

that is feasible for the dual sub-problem.

Step1: Solve master in iteration t

Solve the relaxed pure integer program:

Let zt and yt be the solution. If z is unbounded from below, take yt to be some

value that gives zt some arbitrarily large negative value.

integerand0

...1)(..

min

22

y

bDy

yAbuyc

2

1 tizts

z

i

www.sics.se

BENDERS’ ALGORITHM FOR MIP

Step 2: Solve sub-problem in iteration t

Generate the most violated constraint of IP by solving the linear program:

Let the solution of this LP be optimal value 𝑢0𝑡+1 at 𝒖𝑡+1.

Step 3: Optimality check

Check convergence criterion. Set 𝐵𝑢 = 𝑚𝑖𝑛 𝐵𝑢, 𝑢0𝑡+1 + 𝒄1𝒚

𝑡 . If 𝑧𝑡 > 𝐵𝑢 − 𝜀, stop; the optimal solution has been reached. Otherwise, add the constraint 𝑧 ≥𝒄2𝒚 + 𝒖𝑡+1(𝒃1 − 𝑨2𝒚) to the master problem. Set 𝑡 = 𝑡 + 1. Return to step 1.

0

..

)'(max

11

2

u

MuE

cuA

yAbu 1

ts