Notes 5 IE 312 1

Knapsack Model Intuitive idea: what is the most

valuable collection of items that can be fit into a backpack?

Notes 5 IE 312 2

Race Car Features

1 2 3 4 5 6Cost (thousand) 10.2$ 6.0$ 23.0$ 11.1$ 9.8$ 31.6$ Speed increase (mph) 8 3 15 7 10 12

Proposed Feature (j )

Budget of $35,000

Which features should be added?

Notes 5 IE 312 3

Decision variables

ILP

Formulation

otherwise0

added is feature if1 jx j

1or 0

356.318.91.110.230.62.10s.t.121071538max

654321

654321

jx

xxxxxxxxxxxx

Notes 5 IE 312 4

LINGO FormulationMODEL:

SETS:

FEATURES /F1,F2,F3,F4,F5,F6/: INCLUDE,SPEED_INC,COST;

ENDSETS

DATA:

SPEED_INC = 8 3 15 7 10 12;

COST = 10.2 6.0 23.0 11.1 9.8 31.6;

BUDGET = 35;

ENDDATA

MAX = @SUM( FEATURES: SPEED_INC * INCLUDE);

@SUM( FEATURES: COST * INCLUDE) <= BUDGET;

@FOR( FEATURES: @BIN( INCLUDE));

END

Specifyindex sets

All theconstants

Objective

Constraints

Variables indexed bythis set

Decision variablesare binary

Note ; to end command: to begin an environment

Notes 5 IE 312 5

Solve using Branch & Bound

5 0x 5 1x

Solution?

Candidate Problem

Relaxed Problem

1or 0

356.311.110.230.62.10s.t.1271538max

64321

64321

jx

xxxxxxxxxx

1 0

356.318.91.110.230.62.10s.t.121071538max

654321

654321

jx

xxxxxxxxxxxx

Notes 5 IE 312 6

What is the Relative Worth?

1

2

3

4

5

6

8: 0.7810.23: 0.5615: 0.65237: 0.6311.110: 1.029.812: 0.3831.6

x

x

x

x

x

x

Want to add thisfeature first

Want to add thisfeature second

Notes 5 IE 312 7

Solve Relaxed Problem

5 0x 5 1x

Solution:

Relaxed Problem

Objective <= 24.8

8 1 15 1 7 0.257 1 0

356.318.91.110.230.62.10s.t.121071538max

654321

654321

jx

xxxxxxxxxxxx

money ofout ,78.1

$1.8 $23.0 - $24.8 have still ,1

$24.8 $10.2 - $35 have still ,1

4

3

1

x

x

x

Notes 5 IE 312 8

Now the other node…

5 0x 5 1x

Relaxed Problem

Solution:

1

3

1 Still have $25.2 $10.2 $15

15 0.65223

x

x

Objective <= 27.8

1 0

8.9356.311.110.230.62.10s.t.121071538max

64321

64321

jx

xxxxxxxxxx

Notes 5 IE 312 9

Next Step?

5 0x 5 1x

Objective <= 24.8 Objective <= 27.8

Notes 5 IE 312 10

Rule of Thumb: Better Value

5 0x 5 1x

Obj <= 24.8

1 0x 1 1x Solution:

315 0.65223x

Obj. <= 27.8

3

4

1 Still have $25.2 $23 $2.2

2.2 0.19811.1

x

x

Obj. <=26.4

Solution:

Relaxed Problem

1 0

2.256.311.110.230.6s.t.12715310max

6432

6432

jx

xxxxxxxx

Relaxed Problem

1 0

2.156.311.110.230.6s.t.12715318max

6432

6432

jx

xxxxxxxx

Notes 5 IE 312 11

Next Level

5 0x 5 1x

Obj <= 24.8

1 0x 1 1x

Obj. <=26.4

3 0x 3 1x

InfeasibleObj. = 25

Now What?

Notes 5 IE 312 12

Next Steps …

5 0x 5 1x

Obj <= 24.8

1 0x 1 1x

Obj. <= 26.43 0x 3 1x

InfeasibleObj. = 25

Still need to continue branching here.

Finally we will have

accounted for every solution!

Notes 5 IE 312 13

Capital Budgeting Multidimensional knapsack

problems are often called capital budgeting problems

Idea: select collection of projects, investments, etc, so that the value is maximized (subject to some resource constraints)

Notes 5 IE 312 14

NASA Capital Budgeting2000- 2005- 2010- 2015- 2020- Not Depends

j Mission 2004 2009 2014 2019 2024 Value With On1 Communication satellite 6 2002 Orbital microwave 2 3 33 Io lander 3 5 204 Uranus orbiter 2020 10 50 5 35 Uranus orbiter 2010 5 8 70 4 36 Mercury probe 1 8 4 20 37 Saturn probe 1 8 5 38 Infrared imaging 5 10 119 Ground-based SETI 4 5 200 14

10 Large orbital structures 8 4 15011 Color imaging 2 7 18 8 212 Medical technology 5 7 813 Polar orbital platform 1 4 1 1 30014 Geosynchronous SETI 4 5 3 3 185 9

Budget 10 12 14 14 14

Budget Requirements

Notes 5 IE 312 15

Formulation Decision variables

Budget constraints

otherwise0

chosen is mission if1 jx j

1 2 3 7 9 126 2 3 1 4 5 10 (year 1)x x x x x x

Notes 5 IE 312 16

Formulation Mutually exclusive choices

Dependencies

1

1

1

149

118

54

xx

xx

xx

34

211

xx

xx

Notes 5 IE 312 17

Assignment Problems Assignment problems deal with

optimal pairing or matching of objects in two distinct sets

Decision variable

Let A be the set of allowed assignments and cij be the cost of assigning i to j.

otherwise0

toassigned is if1 jixij

Notes 5 IE 312 18

Formulation

A(i,jx

Njx

Nix

xc

ij

ij

Ajii

ij

Ajij

ijij

Aji

in ) allfor 1,0

allfor ,1

allfor ,1s.t.

min

2

in ),(:

in ),(:

in ),(

Notes 5 IE 312 19

Example We must determine how jobs should be

assigned to machines to minimize setup times, which are given below:

Job 1 Job 2 Job 3 Job 4

Machine 1

14 5 8 7

Machine 2

2 12 6 5

Machine 3

7 8 3 9

Machine 4

2 4 6 10

Notes 5 IE 312 20

Hungarian Algorithm Step 1: (a) Find the minimum

element in each row of the cost matrix. Form a new matrix by subtracting this cost from each row. (b) Find the minimum cost in each column of the new matrix, and subtract this from each column. This is the reduced cost matrix.

Notes 5 IE 312 21

Example: Step 1(a)Job 1 Job 2 Job 3 Job 4

Machine 1

14 5 8 7

Machine 2

2 12 6 5

Machine 3

7 8 3 9

Machine 4

2 4 6 10Job 1 Job 2 Job 3 Job 4

Machine 1

9 0 3 2

Machine 2

0 10 4 3

Machine 3

4 5 0 6

Machine 4

0 2 4 8

Notes 5 IE 312 22

Example: Step 1(b)

Job 1 Job 2 Job 3 Job 4

Machine 1

9 0 3 0

Machine 2

0 10 4 1

Machine 3

4 5 0 4

Machine 4

0 2 4 6

Job 1 Job 2 Job 3 Job 4

Machine 1

9 0 3 2

Machine 2

0 10 4 3

Machine 3

4 5 0 6

Machine 4

0 2 4 8

Notes 5 IE 312 23

Hungarian Algorithm Step 2: Draw the minimum

number of lines that are needed to cover all the zeros in the reduced cost matrix. If m lines are required, then an optimal solution is available among the covered zeros in the matrix. Otherwise, continue to Step 3.

Notes 5 IE 312 24

Example: Step 2

Job 1 Job 2 Job 3 Job 4

Machine 1

9 0 3 0

Machine 2

0 10 4 1

Machine 3

4 5 0 4

Machine 4

0 2 4 6We need 3<4 lines, so continue to Step 3

Notes 5 IE 312 25

Hungarian Algorithm Step 3: Find the smallest nonzero

element (say, k) in the reduced cost matrix that is uncovered by the lines. Subtract k from each uncovered element, and add k to each element that is covered by two lines. Return to Step 2.

Notes 5 IE 312 26

Example: Step 3Job 1 Job 2 Job 3 Job 4

Machine 1

9 0 3 0

Machine 2

0 10 4 1

Machine 3

4 5 0 4

Machine 4

0 2 4 6Job 1 Job 2 Job 3 Job 4

Machine 1

10 0 3 0

Machine 2

0 9 3 0

Machine 3

5 5 0 4

Machine 4

0 1 3 5

Notes 5 IE 312 27

Example: Step 2 (again)

Job 1 Job 2 Job 3 Job 4

Machine 1

10 0 3 0

Machine 2

0 9 3 0

Machine 3

5 5 0 4

Machine 4

0 1 3 5Need 4 lines, so we have the optimal assignment and we stop

Notes 5 IE 312 28

Example: Final SolutionJob 1 Job 2 Job 3 Job 4

Machine 1

10 0 3 0

Machine 2

0 9 3 0

Machine 3

5 5 0 4

Machine 4

0 1 3 5Optimal assignment

1,1,1,1 24413312 xxxx

Notes 5 IE 312 29

Travelling Salesman Problem (TSP)

Ames

Fort Dodge

BooneCarrollMarshalltown

West Des Moines

Waterloo

What is the shortest route,starting in Ames, that visitseach city exactly ones?

Notes 5 IE 312 30

TSP Solution

Ames

Fort Dodge

BooneCarrollMarshalltown

West Des Moines

Waterloo

Notes 5 IE 312 31

Not a TSP Solution

Ames

Fort Dodge

BooneCarrollMarshalltown

West Des Moines

Waterloo

Notes 5 IE 312 32

Applications

Routing of vehicles (planes, trucks,

etc.)

Routing of postal workers

Drilling holes on printed circuit boards

Routing robots through a warehouse,

etc.

Notes 5 IE 312 33

Formulating TSP A TSP is symmetric if you can go

both ways on every arc

otherwise0

and between leg includes route theif1,

jix ji

i ij

jiji xc ,,min

2, ij

jix

Notes 5 IE 312 34

Example1

5

2

6

3 4

10

10

10

11

1

1

1 1

Formulate a TSP

Notes 5 IE 312 35

Subtours It is not sufficient to have two arcs

connected to each node Why?

Must eliminate all subtours Every subset of points must be

exited

Notes 5 IE 312 36

How do we eliminate subtours?

1

5

2

6

3 4

10

10

10

11

1

1

1 1

Notes 5 IE 312 37

Asymmetric TSP Now we have decision variables

Constraints

otherwise0

to from goes route theif1,

jix ji

)(enter 1

) (leave 1

,

,

ix

ix

ijij

ijji

Notes 5 IE 312 38

Asymmetric TSP (cont.) Each tour must enter and leave

every subset of points

Along with all variables being 0 or 1, this is a complete formulation

1, Si Sj

jix

Notes 5 IE 312 39

Example1

5

2

6

3 4

10

10

10

11

1

1

1 1

Assume a two unit penalty for passing from a high to lowernumbered node.

This is now an asymmetric TSP. Why?

Notes 5 IE 312 40

Subtour Elimination Making sure there are no subtours

involves a very large number of constraints

Can obtain simpler constraints if we go with a nonlinear objective function

otherwise0

is tedpoint visith theif1,

iky ik

Notes 5 IE 312 41

Quadratic Assignment Formulation

1,0

1

1

min

,

,

,

,1,,

ik

kik

iik

i j kjkikji

y

y

y

yyd

Notes 5 IE 312 42

Example: reformulate1

5

2

6

3 4

10

10

10

11

1

1

1 1

Notes 5 IE 312 43

Solving TSP We can use branch-and-bound to get

an exact solution to the TSP problem As always, the key to implementing

branch-and-bound is to relax the problem so that we can easily solve the relaxed problem, but we still get good bounds

How can we relax the TSP?

Notes 5 IE 312 44

Relaxing the TSP

)(enter 1

) (leave 1

,

not

,

not

ix

ix

ij

ij

ji

ij

1,

in not in

ji

SiSi

x

1,0, jix

jiji

iji

xc ,,

not

min

Reduces to the assignment problem

Notes 5 IE 312 45

Branching

0, jix 0, ijx

Which cities should be selected?

What is the most important variable?

Notes 5 IE 312 46

Example: Solution to Assignment Problem

Ames

DSM

CR

IC

1,,,, CRICICCRAmesDSMDSMAmes xxxx

Notes 5 IE 312 47

Branching

0, AmesDSMx 0, DSMAmesx

Always branch to split up the subtours!

Solving the assignment problem will hopefully yield a feasible solution soon.

Notes 5 IE 312 48

Solution to New Assignment Problem (Left Branch)

Ames

DSM

CR

IC

This makes both subtours impossible, so we get

Thus, optimal solution is found by solving a total of three assignment problems!

Notes 5 IE 312 49

Poor Branching Examples

0, AmesCRx 0, CRAmesx

0, AmesCRx 1, AmesCRx

Notes 5 IE 312 50

Set Packing, Covering, and Partitioning

Notes 5 IE 312 51

Select Locations

Notes 5 IE 312 52

Ways of Splitting the Set Set covering constraints

Set packing constraints

Set partitioning constraints

Jj

jx 1

Jj

jx 1

Jj

jx 1

Notes 5 IE 312 53

Example: Choosing OR Software

Algorithm 1 2 3 4LP Yes Yes Yes YesIP Yes YesNLP Yes YesCost 3 4 6 14

Software Package (j )

Formulate a set covering problem to acquire the minimum costsoftware with LP, IP, and NLP capabilities.

Formulate set partitioning and set packing problems. What goalsdo they meet?

Notes 5 IE 312 54

Maximum Coverage Perhaps the budget only allows $9000 What can we then do

Maximum coverage How do we now formulate the

problem? Need new variables

otherwise0

providednot ALG if1ALGy

Notes 5 IE 312 55

Useful Formulation Tricks! Integer variables can be used to

model numerous things that seem difficult Fixed-charge IP Either-or constraints If-then constraints

All of these involve introducing a (dummy) binary variable and a “very large number”

Notes 5 IE 312 56

Fixed-Charge Constraint We need to determine how many Gadgets to make

We have variable cost of $7/ Gadget made, but also a fixed cost of $500 if we make any Gadgets (but we could also choose not to make any, in which case we would have no cost

How do we represent the cost?

Here, M is a very large number, e.g. M = 1000

yMx

yxCost

y

5007

otherwise0

Gadgets make weif1

made Gadgets ofNumber x

Notes 5 IE 312 57

What-If Constraints

1,0

)1(100

100or 0

y

yMx

Myx

xx

Need to model (continuous variable):

Trick: add a binary variable y, and a very large M

Here, M = 200 would do nicely

Notes 5 IE 312 58

If-Then Constraints We want to represent

Similar to before:

Here, M=3 would be big enough!

0 then 1 4321 xxxx

1,0

)1(1

432

y

yMx

yMxxx