the unbounded knapsack problem and the generalized cordel ... · the unbounded knapsack problem and...

Knapsack Problems Penalty Method GeCoP Experimental Results Open Questions

The Unbounded Knapsack Problemand the

Generalized Cordel Property

Lisa Schreiber

Friedrich-Schiller-Universitat Jena,Institut fur Angewandte Mathematik

November the 26th, 2010

Outline

1 Knapsack Problems

2 A Penalty Method for the Unbounded Knapsack Problem

3 The Generalized Cordel Property (GeCoP)

4 Experimental Results

5 Open Questions

Outline

1 Knapsack Problems

5 Open Questions

Outline

1 Knapsack Problems

5 Open Questions

Outline

1 Knapsack Problems

5 Open Questions

Outline

1 Knapsack Problems

5 Open Questions

Outline

1 Knapsack Problems

5 Open Questions

The Knapsack Problem

Given n items and a knapsack with capacity C.

Every item has a value v(i) and a weight w(i).Question: Which items shall be packed into the knapsacksuch that the capacity C is not injured and the total value isas large as possible?

Definition: Knapsack Problem

maxx∈Gn

f (x) :=n

∑i=1

v(i)xi subject ton

∑i=1

w(i)xi ≤ C

G = {0,1}: 0-1 Knapsack ProblemG = N: Unbounded Knapsack Problem

Given n items and a knapsack with capacity C.Every item has a value v(i) and a weight w(i).

Question: Which items shall be packed into the knapsacksuch that the capacity C is not injured and the total value isas large as possible?

maxx∈Gn

f (x) :=n

∑i=1

v(i)xi subject ton

∑i=1

w(i)xi ≤ C

Given n items and a knapsack with capacity C.Every item has a value v(i) and a weight w(i).Question: Which items shall be packed into the knapsacksuch that the capacity C is not injured and the total value isas large as possible?

maxx∈Gn

f (x) :=n

∑i=1

v(i)xi subject ton

∑i=1

w(i)xi ≤ C

maxx∈Gn

f (x) :=n

∑i=1

v(i)xi subject ton

∑i=1

w(i)xi ≤ C

maxx∈Gn

f (x) :=n

∑i=1

v(i)xi subject ton

∑i=1

w(i)xi ≤ C

G = {0,1}: 0-1 Knapsack Problem

G = N: Unbounded Knapsack Problem

maxx∈Gn

f (x) :=n

∑i=1

v(i)xi subject ton

∑i=1

w(i)xi ≤ C

Outline

1 Knapsack Problems

5 Open Questions

The Penalty Method

Aim: Compute not only an optimal solution but also one (ormore) alternative solutions.

two Criteria:1 An alternative should be good with respect to the objective

function.2 An alternative should not be too similar to the optimal

solution.

One good method: the Penalty Methodexamined by Schwarz (2003), Sameith (2005) andDornfelder (2009)

The Penalty Method

Aim: Compute not only an optimal solution but also one (ormore) alternative solutions.two Criteria:

1 An alternative should be good with respect to the objectivefunction.

2 An alternative should not be too similar to the optimalsolution.

The Penalty Method

One good method: the Penalty Method

examined by Schwarz (2003), Sameith (2005) andDornfelder (2009)

The Penalty Method

The Penalty Method for the Knapsack Problem

The goal is to compute a second solution, which shalldiffer from the optimal solution B0.have a good function value.

Main idea: Punish the items used in the optimal solutionby reducing their values.

Important properties:

ε increases→ punishment gets higherB0(i) > B0(j)→ punishment of item i is higher

vε (i) =

{v (i) , if B0(i) = 0v (i) · [1− ε · B0(i)] , if B0(i) > 0

vε (i) =

Important properties:ε increases→ punishment gets higher

B0(i) > B0(j)→ punishment of item i is higher

vε (i) =

Important properties:ε increases→ punishment gets higherB0(i) > B0(j)→ punishment of item i is higher

vε (i) = v (i) · [1− ε · B0(i)]

A first example

C = 13values v =[6,8,3,1]

weights w=[5,7,3,6]

optimal solution: B0 = (2,0,1,0) with f (B0) = 15

e.g. ε = 0.7. This leads to the following punished values:

vε(2) = v(2) = 8 andvε(4) = v(4) = 1

vε(1) = v (1) · [1− ε · B0(1)] = 6 · [1− 0.7 · 2] = −2.4

vε(3) = v (3) · [1− ε · B0(3)] = 3 · [1− 0.7 · 1] = 0.9

C, vε = [−2.4, 8, 0.9, 1] and w provide Bε = (0,1,2,0) asbest solution and penalty alternative

A first example

C = 13values v =[6,8,3,1]

weights w=[5,7,3,6]

vε(2) = v(2) = 8 andvε(4) = v(4) = 1

vε(1) = v (1) · [1− ε · B0(1)] = 6 · [1− 0.7 · 2] = −2.4

vε(3) = v (3) · [1− ε · B0(3)] = 3 · [1− 0.7 · 1] = 0.9

A first example

C = 13values v =[6,8,3,1]

weights w=[5,7,3,6]

vε(2) = v(2) = 8 andvε(4) = v(4) = 1

vε(1) = v (1) · [1− ε · B0(1)] = 6 · [1− 0.7 · 2] = −2.4

vε(3) = v (3) · [1− ε · B0(3)] = 3 · [1− 0.7 · 1] = 0.9

A first example

C = 13values v =[6,8,3,1]

weights w=[5,7,3,6]

vε(2) = v(2) = 8 andvε(4) = v(4) = 1

vε(1) = v (1) · [1− ε · B0(1)] = 6 · [1− 0.7 · 2] = −2.4

vε(3) = v (3) · [1− ε · B0(3)] = 3 · [1− 0.7 · 1] = 0.9

A first example

C = 13values v =[6,8,3,1]

weights w=[5,7,3,6]

vε(2) = v(2) = 8 andvε(4) = v(4) = 1

vε(1) = v (1) · [1− ε · B0(1)] = 6 · [1− 0.7 · 2] = −2.4

vε(3) = v (3) · [1− ε · B0(3)] = 3 · [1− 0.7 · 1] = 0.9

A first example

C = 13values v =[6,8,3,1]

weights w=[5,7,3,6]

vε(2) = v(2) = 8 andvε(4) = v(4) = 1

vε(1) = v (1) · [1− ε · B0(1)] = 6 · [1− 0.7 · 2] = −2.4

vε(3) = v (3) · [1− ε · B0(3)] = 3 · [1− 0.7 · 1] = 0.9

A first example

C = 13values v =[6,8,3,1]

weights w=[5,7,3,6]

vε(2) = v(2) = 8 andvε(4) = v(4) = 1

vε(1) = v (1) · [1− ε · B0(1)] = 6 · [1− 0.7 · 2] = −2.4

vε(3) = v (3) · [1− ε · B0(3)] = 3 · [1− 0.7 · 1] = 0.9

Properties of Penalty Alternatives

Properties of Penalty Alternatives (Schwarz 2003)B penalty-alternative⇒ B is optimal for all parameters ε inan optimality interval IB = [ε1, ε2] and nowhere else

P1,P2 penalty alternatives with optimality intervals I1 and I2⇒ three possible cases

1 I1 = I22 I1 ∩ I2 = {ε}, intersection contains only a single epsilon3 I1 ∩ I2 = ∅, empty intersection

∞0ε0 ε1 ε2

P0 = B0 P1 P2 . . .

This provides an algorithm, how to compute all penaltyalternatives P0,P1, . . . ,Pk !

Properties of Penalty Alternatives (Schwarz 2003)B penalty-alternative⇒ B is optimal for all parameters ε inan optimality interval IB = [ε1, ε2] and nowhere elseP1,P2 penalty alternatives with optimality intervals I1 and I2⇒ three possible cases

∞0ε0 ε1 ε2

P0 = B0 P1 P2 . . .

1 I1 = I2

2 I1 ∩ I2 = {ε}, intersection contains only a single epsilon3 I1 ∩ I2 = ∅, empty intersection

∞0ε0 ε1 ε2

P0 = B0 P1 P2 . . .

1 I1 = I22 I1 ∩ I2 = {ε}, intersection contains only a single epsilon

3 I1 ∩ I2 = ∅, empty intersection

∞0ε0 ε1 ε2

P0 = B0 P1 P2 . . .

∞0ε0 ε1 ε2

P0 = B0 P1 P2 . . .

∞0ε0 ε1 ε2

P0 = B0 P1 P2 . . .

∞0ε0 ε1 ε2

P0 = B0 P1 P2 . . .

vε (i) = v (i) · [1− ε · B0(i)]

⇒ fε(B) =n

∑i=1

vε(i) · B(i) =n

∑i=1

v(i) · [1− ε · B0(i)] · B(i)

∑i=1

v(i) · B(i)︸︷︷︸=f (B)

−εn

∑i=1

v(i) · B0(i) · B(i)︸︷︷︸=:p(B)

vε (i) = v (i) · [1− ε · B0(i)]

⇒ fε(B) =n

∑i=1

vε(i) · B(i) =n

∑i=1

v(i) · [1− ε · B0(i)] · B(i)

∑i=1

v(i) · B(i)︸︷︷︸=f (B)

−εn

∑i=1

v(i) · B0(i) · B(i)︸︷︷︸=:p(B)

· · · ∞0ε0 ε1 ε2 εk

threshold parameters

f (P0) ≥ f (P1) > . . . > f (Pk)p (P0) ≥ p (P1) > . . . > p (Pk)

vε (i) = v (i) · [1− ε · B0(i)]

⇒ fε(B) =n

∑i=1

vε(i) · B(i) =n

∑i=1

v(i) · [1− ε · B0(i)] · B(i)

∑i=1

v(i) · B(i)︸︷︷︸=f (B)

−εn

∑i=1

v(i) · B0(i) · B(i)︸︷︷︸=:p(B)

· · · ∞0ε0 ε1 ε2 εk

threshold parameters

f (P0) ≥ f (P1) > . . . > f (Pk)p (P0) ≥ p (P1) > . . . > p (Pk)

Threshold Parameters

Let ε be the threshold parameter between the optimalityintervals of the two penalty alternatives Bl and Br .e.g.: Bl = Pi and Br = Pi+1Then we can compute ε the following way.

fε (Bl) = fε (Br )

⇔ f (Bl)− ε · p (Bl) = f (Br )− ε · p (Br )

⇔ ε =f (Bl)− f (Br )

p (Bl)− p (Br )

An Algorithm for Computing all Penalty Alternatives(Schwarz)

1 Initialization: Compute B0 (optimal solution) and B∞.Go to step 2 with [Bl = B0,Br = B∞].

2 Compute the possible threshold parameter ε between Bland Br and the penalty alternative Bε. Then we considerthe following two cases:fε (Bε) = fε (Bl) = fε (Br ):

bagagagsadgsagsadgdsaglabla

No further branching. ε is the real thresholdparameter between Bl and Br .

fε (Bε) 6= fε (Bl) = fε (Br ):

bagagagsadgsagsadgdsaglabla

With Bε we found a new penalty alternative,so we have to branch. Go to step 2 with[Bl = Bl ,Br = Bε] and [Bl = Bε,Br = Br ]

Outline

1 Knapsack Problems

5 Open Questions

From Chess history

Siegbert Tarrasch(1862−1934)In every chess positionthere exists exactly onebest move!

Emanuel Lasker(1868−1941)In every chess positionthere exist as manyappropriate moves asthere are different players!

Oskar Cordel’s (1843−1913) “Three moves law”In every chess position there exists either

exactly one best moveor at least three equally best moves.

This assumption does not hold in every chess position!

From Chess history

Generalized Cordel Property (GeCoP)

Generalizing Cordel: examine differences betweencandidates

moves a1 and a2 are equally good, if |f (a1)− f (a2)| isvery small

Definition: Generalized Cordel Property (GeCoP)

a1,a2, and a3 solutions of a given maximization problemf (a1) ≥ f (a2) ≥ f (a3)

define differences di := f (ai)− f (ai+1)

We say that a1,a2, and a3 fulfill the Generalized CordelProperty (GeCoP), iff the following holds:

d1 ≥ d2 (GeCoP)

Generalizing Cordel: examine differences betweencandidatesmoves a1 and a2 are equally good, if |f (a1)− f (a2)| isvery small

d1 ≥ d2 (GeCoP)

Definition: Generalized Cordel Property (GeCoP)a1,a2, and a3 solutions of a given maximization problem

f (a1) ≥ f (a2) ≥ f (a3)

d1 ≥ d2 (GeCoP)

Definition: Generalized Cordel Property (GeCoP)a1,a2, and a3 solutions of a given maximization problemf (a1) ≥ f (a2) ≥ f (a3)

d1 ≥ d2 (GeCoP)

Connection between Cordel and GeCoP

as a reminder: Cordel said there are never exactly twobest moves!

best moves in chess position: a1,a2 and a3

assume a1 and a2 are equally good (two best moves)

⇒ d1 = f (a1)− f (a2) will be very small⇒ d2 = f (a2)− f (a3) has to be very small too,

because of GeCoP (d1 ≥ d2)⇒ at least three best moves!

My thesis: analyzing the validity of GeCoP for severalkinds of optimization problems andkinds of solutions (e.g. three best solutions,penalty-alternatives)

assume a1 and a2 are equally good (two best moves)⇒ d1 = f (a1)− f (a2) will be very small

⇒ d2 = f (a2)− f (a3) has to be very small too,because of GeCoP (d1 ≥ d2)

⇒ at least three best moves!

assume a1 and a2 are equally good (two best moves)⇒ d1 = f (a1)− f (a2) will be very small⇒ d2 = f (a2)− f (a3) has to be very small too,

because of GeCoP (d1 ≥ d2)

⇒ at least three best moves!

GeCoP for Penalty Alternatives

0ε0 ε1 ε2P0 P1 P2 . . .

d1 := f(P0)− f(P1) d2 := f(P1)− f(P2)

0ε0 ε1 ε2P0 P1 P2 . . .

d1 := f(P0)− f(P1) d2 := f(P1)− f(P2)≥?

Outline

1 Knapsack Problems

5 Open Questions

Random instances

according to Martello, Toth: “Knapsack Problems” (1990)two parameters v , r , e.g. v = 1000, r = 100w(i) uniformly randomly generated in the int range [1, v ]

C = 0.5 ·∑ni=1 w(i)

Three different types of instances:

Uncorrelated: v(i) uniformly randomly generatedin the range [1, v ].

Weakly correlated: v(i) uniformly randomly generated inthe range [w(i)− r ,w(i) + r ]with respect to v(i) > 0.

Strongly correlated: v(i) = w(i) + r .

Random instances

according to Martello, Toth: “Knapsack Problems” (1990)two parameters v , r , e.g. v = 1000, r = 100w(i) uniformly randomly generated in the int range [1, v ]C = 0.5 ·∑n

i=1 w(i)

Three different types of instances:

Random instances

i=1 w(i)Three different types of instances:

Random instances

Comparison of P (d1 ≥ d2) and P (d2 ≥ d3)

w(i), v(i) ∈ [1,100], v(i) ∈ [w(i)− 10,w(i) + 10], v(i) = w(i) + 10

●● ●

●●

● ●

5 10 20 50 100

●● ● ●

●● ●

● ●

●●

● ●

●● ●

Prob(d1 ≥ d2)

uncorrelatedweakly correlatedstrongly correlated

●● ●

●●

● ● ● ●●

5 10 20 50 100

●● ● ●

● ●● ● ●

● ● ● ● ●

● ● ●

Prob(d2 ≥ d3)

w(i), v(i) ∈ [1,1 000], v(i) ∈ [w(i)− 100,w(i) + 100], v(i) = w(i) + 100

●●

●● ●

● ●

●●

● ●

5 10 20 50 100 200 500

● ●●●

●●

● ●

●●

5 1000n =

Prob(d1 ≥ d2)

● ●●●●●

●●

●● ●

●●

5 10 20 50 100 200 500

● ●●●

●●● ●

●● ●

●●

● ●

●●

●● ●●

●●

● ●

5 1000n =

Prob(d2 ≥ d3)

w(i), v(i) ∈ [1,100 000], v(i) ∈ [w(i)− 10 000,w(i) + 10 000], v(i) = w(i) + 10 000

●●

● ●

●● ● ● ● ●

● ●●

5 10 20 50 100 200 500

●●●

●● ●

● ● ●● ● ●

●●

●● ●

● ● ●●

●●

5 1000n =

Prob(d1 ≥ d2)●

●●●

● ● ●

● ●●

5 10 20 50 100 200 500

●●●

●●

● ●

●●

5 1000n =

Prob(d2 ≥ d3)●

Results

1 Prob (d1 ≥ d2) �Prob (d2 ≥ d3)

2 The behavior changes heavily with the rangesize v .

The first phenomenon is unique according to our wider studieson other optimization problems!

Results

1 Prob (d1 ≥ d2) �Prob (d2 ≥ d3)

Results

1 Prob (d1 ≥ d2) �Prob (d2 ≥ d3)

Average number of penalty alternatives

w(i), v(i) ∈ [1,100], v(i) ∈ [w(i)− 10,w(i) + 10], v(i) = w(i) + 10

●●●●●

● ● ●

● ●

●●

20 40 60 80 100

●●●●●●

●●

●●●●● ●

w(i), v(i) ∈ [1,1 000], v(i) ∈ [w(i)− 100,w(i) + 100], v(i) = w(i) + 100

●●●●●●

●●●

●●

● ●

0 200 400 600 800 1000

●●●●●

●●●●

●●

●●●

●●

w(i), v(i) ∈ [1,100 000], v(i) ∈ [w(i)− 10 000,w(i) + 10 000], v(i) = w(i) + 10 000

●●●●●

●●

● ● ●

●●

0 200 400 600 800 1000

●●●●●●

●●

● ●●

● ●

●●●

●●

● ●

●●

Outline

1 Knapsack Problems

5 Open Questions

Open Questions

1 Do these phenomenona also occur at other optimizationproblems?

2 Is there a better method to generate alternatives for theunbounded knapsack problem?Is there a good method, that generates more than only 6alternatives in average?

3 How can the results presented be used in practice?Is Cordel more than a Gedanken-Experiment?

Open Questions

additional slides

vε (i) = v (i) · [1− ε · B0(i)]

additional slides

The goal is to compute a second solution, which shalldiffer from a given reference solution Br .have a good function value.

vε (i) = v (i) · [1− ε · Br (i)]

additional slides

Reference Solutions

Possible reference solutions are:1 An optimal solution.

2 A Greedy solution

sort items in decreasing order of value per unit of weightput every item in sequence with maximal frequency in theknapsack such that the capacity is not exceeded

Example: v = [6,8,3,1], w = [5,7,3,6], C = 13

optimal solution: B0 = (2,0,1,0)

additional slides

Reference Solutions

Possible reference solutions are:1 An optimal solution.2 A Greedy solution

Example: v = [6,8,3,1], w = [5,7,3,6], C = 13vw ≈ [1.2, 1.14, 1, 0.17]→ order: 1,2,3,4

(2,0,0,0)→ (2,0,0,0)→ (2,0,1,0)

additional slides

Reference Solutions

Example: v = [6,8,3,1], w = [5,7,3,6], C = 13vw ≈ [1.2, 1.14, 1, 0.17]→ order: 1,2,3,4(2,0,0,0),C ′ = 3

additional slides

Reference Solutions

Example: v = [6,8,3,1], w = [5,7,3,6], C = 13vw ≈ [1.2, 1.14, 1, 0.17]→ order: 1,2,3,4(2,0,0,0),C ′ = 3→ (2,0,0,0),C ′ = 3

additional slides

Reference Solutions

Example: v = [6,8,3,1], w = [5,7,3,6], C = 13vw ≈ [1.2, 1.14, 1, 0.17]→ order: 1,2,3,4(2,0,0,0),C ′ = 3→ (2,0,0,0),C ′ = 3→ (2,0,1,0),C ′ = 0

additional slides

How often is the Greedy Solution Optimal?

w(i), v(i) ∈ [1,1 000], v(i) ∈ [w(i)− 100,w(i) + 100], v(i) = w(i) + 100

●●●●●●● ● ● ● ● ● ●

0 20 40 60 80 100

●●●

●●●●

●●

● ●

●●●

●●●● ● ● ● ●

P(greedy optimal)

additional slides

Optimal Solution as Reference Solution

w(i), v(i) ∈ [1,1 000], v(i) ∈ [w(i)− 100,w(i) + 100], v(i) = w(i) + 100

●●

●● ●

● ●

●●

● ●

5 10 20 50 100 200 500

● ●●●

●●

● ●

●●

5 1000n =

Prob(d1 ≥ d2)

● ●●●●●

●●

●● ●

●●

5 10 20 50 100 200 500

● ●●●

●●● ●

●● ●

●●

● ●

●●

●● ●●

●●

● ●

5 1000n =

Prob(d2 ≥ d3)

additional slides

Greedy Solution as Reference Solution

w(i), v(i) ∈ [1,1 000], v(i) ∈ [w(i)− 100,w(i) + 100], v(i) = w(i) + 100

●●

● ●● ●

● ●

5 10 20 50 100 200 500

●●●

●●

●● ●

●●

● ●●

●●●

● ●

5 1000n =

Prob(d1 ≥ d2)

●●

● ●

● ●● ● ●

5 10 20 50 100 200 500

●●●●●

●● ● ●

●●

●●●●

● ●●

●●

5 1000n =

Prob(d2 ≥ d3)

additional slides

Other Studied Optimization-Problems

Shortest Path ProblemBinary Knapsack ProblemMinimal Spanning TreesAssignment Problemsome theoretical problem types

the unbounded knapsack problem and the generalized cordel ... · the unbounded knapsack problem and...

Documents

copiar, distribuir e transmitir a - cárlisson galdino ·...

lecture knapsack bb

dynprog knapsack

multiple knapsack problem

ga knapsack

merkle-hellman knapsack

knapsack piegel -...

knapsack problems

top k knapsack joins - paris dauphine universitylitwin/top k...

algoritma kriptografi knapsack

kasus knapsack

zero one knapsack

a hybrid algorithm for the unbounded knapsack problem

greedy algorithms csc 4520/6520 fall 2013 problems...

pode ir dizendo oxente! cordel cordel autor: arnaud amorim

no reino do cordel do cordel

merkle-hellman knapsack...

the 0/1 knapsack problemthe 0/1 knapsack problem … · 0/1...

palestra cordel

catalogue supports de knapsack et sacs knapsack