lecture 9. greedy algorithm - skkumonet.skku.edu/wp-content/uploads/2016/05/algorithm_09.pdf ·...
TRANSCRIPT
![Page 1: Lecture 9. Greedy Algorithm - SKKUmonet.skku.edu/wp-content/uploads/2016/05/Algorithm_09.pdf · 2016-05-31 · Algorithms Greedy Algorithms A greedy algorithm always makes the choice](https://reader037.vdocuments.net/reader037/viewer/2022100223/5f0ea8457e708231d4404be6/html5/thumbnails/1.jpg)
Copyright 2000-2016 Networking Laboratory
Lecture 9. Greedy Algorithm
T. H. Cormen, C. E. Leiserson and R. L. RivestIntroduction to Algorithms, 3rd Edition, MIT Press, 2009
Sungkyunkwan University
Hyunseung [email protected]
![Page 2: Lecture 9. Greedy Algorithm - SKKUmonet.skku.edu/wp-content/uploads/2016/05/Algorithm_09.pdf · 2016-05-31 · Algorithms Greedy Algorithms A greedy algorithm always makes the choice](https://reader037.vdocuments.net/reader037/viewer/2022100223/5f0ea8457e708231d4404be6/html5/thumbnails/2.jpg)
Algorithms
Greedy Algorithms A greedy algorithm always makes the choice that
looks best at the moment My everyday examples
Playing cards Invest on stocks Choose a university
The hope A locally optimal choice will lead to a globally optimal solution
Networking Laboratory 2/45
![Page 3: Lecture 9. Greedy Algorithm - SKKUmonet.skku.edu/wp-content/uploads/2016/05/Algorithm_09.pdf · 2016-05-31 · Algorithms Greedy Algorithms A greedy algorithm always makes the choice](https://reader037.vdocuments.net/reader037/viewer/2022100223/5f0ea8457e708231d4404be6/html5/thumbnails/3.jpg)
Algorithms
Similar to Dynamic Programming It applies to Optimization Problem When we have a choice to make, make the one that
looks best right now Make a locally optimal choice in hope of getting a globally
optimal solution
Greedy algorithms don’t always yield an optimal solution, but sometimes they do For many problems, it provides an optimal solution much more
quickly than a dynamic programming approach
Introduction
Networking Laboratory 3/45
![Page 4: Lecture 9. Greedy Algorithm - SKKUmonet.skku.edu/wp-content/uploads/2016/05/Algorithm_09.pdf · 2016-05-31 · Algorithms Greedy Algorithms A greedy algorithm always makes the choice](https://reader037.vdocuments.net/reader037/viewer/2022100223/5f0ea8457e708231d4404be6/html5/thumbnails/4.jpg)
Algorithms
The problem of scheduling several competing activities that require exclusive use of a common resource n activities require exclusive use of a common resource For example, scheduling the use of a classroom
Set of activities S={a1, …. , an}. ai needs resource during period [ si , fi )
[ ) is a half-open interval si = start time and fi = finish time
Goal Select the largest possible set of non-overlapping
(mutually compatible) activities Note: Could have many other objectives
Schedule room for longest time Maximize income rental fees
An Activity Selection Problem
Networking Laboratory 4/45
![Page 5: Lecture 9. Greedy Algorithm - SKKUmonet.skku.edu/wp-content/uploads/2016/05/Algorithm_09.pdf · 2016-05-31 · Algorithms Greedy Algorithms A greedy algorithm always makes the choice](https://reader037.vdocuments.net/reader037/viewer/2022100223/5f0ea8457e708231d4404be6/html5/thumbnails/5.jpg)
Algorithms
Here are a set of start and finish times What is the maximum number of activities that can be completed?
An Activity Selection Problem
Networking Laboratory 5/45
![Page 6: Lecture 9. Greedy Algorithm - SKKUmonet.skku.edu/wp-content/uploads/2016/05/Algorithm_09.pdf · 2016-05-31 · Algorithms Greedy Algorithms A greedy algorithm always makes the choice](https://reader037.vdocuments.net/reader037/viewer/2022100223/5f0ea8457e708231d4404be6/html5/thumbnails/6.jpg)
Algorithms
What is the maximum number of activities that can be completed? {a3, a9, a11} can be completed But so can {a1, a4, a8, a11} which is a larger set
But it is not unique, consider {a2, a4, a9, a11}
An Activity Selection Problem
Networking Laboratory 6/45
![Page 7: Lecture 9. Greedy Algorithm - SKKUmonet.skku.edu/wp-content/uploads/2016/05/Algorithm_09.pdf · 2016-05-31 · Algorithms Greedy Algorithms A greedy algorithm always makes the choice](https://reader037.vdocuments.net/reader037/viewer/2022100223/5f0ea8457e708231d4404be6/html5/thumbnails/7.jpg)
Algorithms
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Networking Laboratory 7/45
![Page 8: Lecture 9. Greedy Algorithm - SKKUmonet.skku.edu/wp-content/uploads/2016/05/Algorithm_09.pdf · 2016-05-31 · Algorithms Greedy Algorithms A greedy algorithm always makes the choice](https://reader037.vdocuments.net/reader037/viewer/2022100223/5f0ea8457e708231d4404be6/html5/thumbnails/8.jpg)
Algorithms
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Networking Laboratory 8/45
![Page 9: Lecture 9. Greedy Algorithm - SKKUmonet.skku.edu/wp-content/uploads/2016/05/Algorithm_09.pdf · 2016-05-31 · Algorithms Greedy Algorithms A greedy algorithm always makes the choice](https://reader037.vdocuments.net/reader037/viewer/2022100223/5f0ea8457e708231d4404be6/html5/thumbnails/9.jpg)
Algorithms
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Networking Laboratory 9/45
![Page 10: Lecture 9. Greedy Algorithm - SKKUmonet.skku.edu/wp-content/uploads/2016/05/Algorithm_09.pdf · 2016-05-31 · Algorithms Greedy Algorithms A greedy algorithm always makes the choice](https://reader037.vdocuments.net/reader037/viewer/2022100223/5f0ea8457e708231d4404be6/html5/thumbnails/10.jpg)
Algorithms
S ij = {a k ∈ S : f i ≤ s k < f k ≤ s j } = activities that start after a i finishes and finish before a j starts
Activities in S i j are compatible with all activities that finish by f i, and all activities that start no earlier than s j
To represent the entire problem, add fictitious activities: a0 = [ -∞, 0 )
an+1 = [∞, “∞+1” ) We don’t care about - ∞ in a 0 or “∞+1” in a n+1
Then S = S 0, n+1
Range for S i j is 0 ≤ i, j ≤ n + 1
An The Optimal Substructure of the A.-S. Problem
Networking Laboratory 10/45
![Page 11: Lecture 9. Greedy Algorithm - SKKUmonet.skku.edu/wp-content/uploads/2016/05/Algorithm_09.pdf · 2016-05-31 · Algorithms Greedy Algorithms A greedy algorithm always makes the choice](https://reader037.vdocuments.net/reader037/viewer/2022100223/5f0ea8457e708231d4404be6/html5/thumbnails/11.jpg)
Algorithms
Assume that activities are sorted by monotonically increasing finish time f 0 ≤ f 1 ≤ f 2 ≤ · · · ≤ f n < f n+1
Then i ≥ j ⇒ S i j = ∅
If there exists ak ∈ S i j : f i ≤ sk < f k ≤ s j < f j ⇒ f i < f j
But i ≥ j ⇒ f i ≥ f j . Contradiction
So only need to worry about S i j with 0 ≤ i < j ≤ n + 1 All other S i j are ∅
An The Optimal Substructure of the A.-S. Problem
Networking Laboratory 11/45
![Page 12: Lecture 9. Greedy Algorithm - SKKUmonet.skku.edu/wp-content/uploads/2016/05/Algorithm_09.pdf · 2016-05-31 · Algorithms Greedy Algorithms A greedy algorithm always makes the choice](https://reader037.vdocuments.net/reader037/viewer/2022100223/5f0ea8457e708231d4404be6/html5/thumbnails/12.jpg)
Algorithms
Suppose that a solution to S i j includes ak. Have 2 subproblems: S i k (start after a i finishes, finish before ak starts) S k j (start after ak finishes, finish before a j starts)
Solution to S i j { solution to Sik } ∪ {ak } ∪ { solution to S k j }
Since ak is in neither subproblem the subproblems are disjoint | solution to S | = | solution to S ik | + 1 + | solution to S kj |
An The Optimal Substructure of the A.-S. Problem
Networking Laboratory 12/45
![Page 13: Lecture 9. Greedy Algorithm - SKKUmonet.skku.edu/wp-content/uploads/2016/05/Algorithm_09.pdf · 2016-05-31 · Algorithms Greedy Algorithms A greedy algorithm always makes the choice](https://reader037.vdocuments.net/reader037/viewer/2022100223/5f0ea8457e708231d4404be6/html5/thumbnails/13.jpg)
Algorithms
If an optimal solution to S ij includes ak , then the solutions to S ik and S kj used within this solution must be optimal as well
Let A ij = optimal solution to S ij So A i j = A i k ∪ {ak} ∪ A k j, assuming:
S i j is nonempty we know ak
An The Optimal Substructure of the A.-S. Problem
Networking Laboratory 13/45
![Page 14: Lecture 9. Greedy Algorithm - SKKUmonet.skku.edu/wp-content/uploads/2016/05/Algorithm_09.pdf · 2016-05-31 · Algorithms Greedy Algorithms A greedy algorithm always makes the choice](https://reader037.vdocuments.net/reader037/viewer/2022100223/5f0ea8457e708231d4404be6/html5/thumbnails/14.jpg)
Algorithms
c[i, j] : size of maximum-size subset of mutually compatible activities in S ij i ≥ j ⇒ S i j = ∅ ⇒ c[i, j] = 0
If S ij ≠ ∅, suppose we know that a k is in the subset c [i, j] = c [i, k] + 1 + c [k, j] .
But, we don’t know which k to use, and so
An The Optimal Substructure of the A.-S. Problem
Networking Laboratory 14/45
![Page 15: Lecture 9. Greedy Algorithm - SKKUmonet.skku.edu/wp-content/uploads/2016/05/Algorithm_09.pdf · 2016-05-31 · Algorithms Greedy Algorithms A greedy algorithm always makes the choice](https://reader037.vdocuments.net/reader037/viewer/2022100223/5f0ea8457e708231d4404be6/html5/thumbnails/15.jpg)
Algorithms
Early Finish Greedy
Select the activity with the earliest finish Eliminate the activities that could not be scheduled Repeat!
Networking Laboratory 15/45
![Page 16: Lecture 9. Greedy Algorithm - SKKUmonet.skku.edu/wp-content/uploads/2016/05/Algorithm_09.pdf · 2016-05-31 · Algorithms Greedy Algorithms A greedy algorithm always makes the choice](https://reader037.vdocuments.net/reader037/viewer/2022100223/5f0ea8457e708231d4404be6/html5/thumbnails/16.jpg)
Algorithms
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Networking Laboratory 16/45
![Page 17: Lecture 9. Greedy Algorithm - SKKUmonet.skku.edu/wp-content/uploads/2016/05/Algorithm_09.pdf · 2016-05-31 · Algorithms Greedy Algorithms A greedy algorithm always makes the choice](https://reader037.vdocuments.net/reader037/viewer/2022100223/5f0ea8457e708231d4404be6/html5/thumbnails/17.jpg)
Algorithms
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Networking Laboratory 17/45
![Page 18: Lecture 9. Greedy Algorithm - SKKUmonet.skku.edu/wp-content/uploads/2016/05/Algorithm_09.pdf · 2016-05-31 · Algorithms Greedy Algorithms A greedy algorithm always makes the choice](https://reader037.vdocuments.net/reader037/viewer/2022100223/5f0ea8457e708231d4404be6/html5/thumbnails/18.jpg)
Algorithms
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Networking Laboratory 18/45
![Page 19: Lecture 9. Greedy Algorithm - SKKUmonet.skku.edu/wp-content/uploads/2016/05/Algorithm_09.pdf · 2016-05-31 · Algorithms Greedy Algorithms A greedy algorithm always makes the choice](https://reader037.vdocuments.net/reader037/viewer/2022100223/5f0ea8457e708231d4404be6/html5/thumbnails/19.jpg)
Algorithms
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Networking Laboratory 19/45
![Page 20: Lecture 9. Greedy Algorithm - SKKUmonet.skku.edu/wp-content/uploads/2016/05/Algorithm_09.pdf · 2016-05-31 · Algorithms Greedy Algorithms A greedy algorithm always makes the choice](https://reader037.vdocuments.net/reader037/viewer/2022100223/5f0ea8457e708231d4404be6/html5/thumbnails/20.jpg)
Algorithms
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Networking Laboratory 20/45
![Page 21: Lecture 9. Greedy Algorithm - SKKUmonet.skku.edu/wp-content/uploads/2016/05/Algorithm_09.pdf · 2016-05-31 · Algorithms Greedy Algorithms A greedy algorithm always makes the choice](https://reader037.vdocuments.net/reader037/viewer/2022100223/5f0ea8457e708231d4404be6/html5/thumbnails/21.jpg)
Algorithms
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Networking Laboratory 21/45
![Page 22: Lecture 9. Greedy Algorithm - SKKUmonet.skku.edu/wp-content/uploads/2016/05/Algorithm_09.pdf · 2016-05-31 · Algorithms Greedy Algorithms A greedy algorithm always makes the choice](https://reader037.vdocuments.net/reader037/viewer/2022100223/5f0ea8457e708231d4404be6/html5/thumbnails/22.jpg)
Algorithms
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Networking Laboratory 22/45
![Page 23: Lecture 9. Greedy Algorithm - SKKUmonet.skku.edu/wp-content/uploads/2016/05/Algorithm_09.pdf · 2016-05-31 · Algorithms Greedy Algorithms A greedy algorithm always makes the choice](https://reader037.vdocuments.net/reader037/viewer/2022100223/5f0ea8457e708231d4404be6/html5/thumbnails/23.jpg)
Algorithms
Assumes activities already sorted by monotonically increasing finish time If not, then sort in O(n lg n) time Return an optimal solution for Si,n+1
Initial call: REC-ACTIVITY-SELECTOR(s, f, 0, n) Time: Θ(n) — each activity examined exactly once
A Recursive Greedy Algorithm
Networking Laboratory 23/45
![Page 24: Lecture 9. Greedy Algorithm - SKKUmonet.skku.edu/wp-content/uploads/2016/05/Algorithm_09.pdf · 2016-05-31 · Algorithms Greedy Algorithms A greedy algorithm always makes the choice](https://reader037.vdocuments.net/reader037/viewer/2022100223/5f0ea8457e708231d4404be6/html5/thumbnails/24.jpg)
Algorithms
An Iterative Greedy Algorithm
Time Θ(n)Networking Laboratory 24/45
![Page 25: Lecture 9. Greedy Algorithm - SKKUmonet.skku.edu/wp-content/uploads/2016/05/Algorithm_09.pdf · 2016-05-31 · Algorithms Greedy Algorithms A greedy algorithm always makes the choice](https://reader037.vdocuments.net/reader037/viewer/2022100223/5f0ea8457e708231d4404be6/html5/thumbnails/25.jpg)
Algorithms
A globally optimal solution can be arrived at by making a locally optimal (greedy) choice
Dynamic programming Make a choice at each step Choice depends on knowing optimal solutions to subproblems Solve subproblems first Bottom-up manner
Greedy algorithm Make a choice at each step Make the choice before solving the subproblems Top-down fashion
Greedy-Choice Property
Networking Laboratory 25/45
![Page 26: Lecture 9. Greedy Algorithm - SKKUmonet.skku.edu/wp-content/uploads/2016/05/Algorithm_09.pdf · 2016-05-31 · Algorithms Greedy Algorithms A greedy algorithm always makes the choice](https://reader037.vdocuments.net/reader037/viewer/2022100223/5f0ea8457e708231d4404be6/html5/thumbnails/26.jpg)
Algorithms
The Knapsack Problem
The famous knapsack problem: A thief breaks into a museum.
Fabulous paintings, sculptures, and jewels are everywhere. The thief has a good eye for the value of these objects, and knows that each will fetch hundreds or thousands of dollars on the clandestine art collector’s market. But, the thief has only brought a single knapsack to the scene of the robbery, and can take away only what he can carry. What items should the thief take to maximize the haul?
Networking Laboratory 26/45
![Page 27: Lecture 9. Greedy Algorithm - SKKUmonet.skku.edu/wp-content/uploads/2016/05/Algorithm_09.pdf · 2016-05-31 · Algorithms Greedy Algorithms A greedy algorithm always makes the choice](https://reader037.vdocuments.net/reader037/viewer/2022100223/5f0ea8457e708231d4404be6/html5/thumbnails/27.jpg)
Algorithms
The knapsack problem Good example of the difference
0-1 knapsack problem: not solvable by greedy n items Item i is worth $vi , weighs wi pounds Find a most valuable subset of items with total weight ≤ W Have to either take an item or not take it
can’t take part of it
Greedy Algorithm vs. Dynamic Programming
Networking Laboratory 27/45
![Page 28: Lecture 9. Greedy Algorithm - SKKUmonet.skku.edu/wp-content/uploads/2016/05/Algorithm_09.pdf · 2016-05-31 · Algorithms Greedy Algorithms A greedy algorithm always makes the choice](https://reader037.vdocuments.net/reader037/viewer/2022100223/5f0ea8457e708231d4404be6/html5/thumbnails/28.jpg)
Algorithms
Fractional knapsack problem: Solvable by greedy Like the 0-1 knapsack problem, but can take fraction of an item Both have optimal substructure But the fractional knapsack problem has the greedy-choice
property, and the 0-1 knapsack problem does not To solve the fractional problem, rank items by value/weight
vi / wi
Let v i / wi ≥ vi+1 / wi+1 for all i
Greedy Algorithm vs. Dynamic Programming
Networking Laboratory 28/45
![Page 29: Lecture 9. Greedy Algorithm - SKKUmonet.skku.edu/wp-content/uploads/2016/05/Algorithm_09.pdf · 2016-05-31 · Algorithms Greedy Algorithms A greedy algorithm always makes the choice](https://reader037.vdocuments.net/reader037/viewer/2022100223/5f0ea8457e708231d4404be6/html5/thumbnails/29.jpg)
Algorithms
i 1 2 3vi 60 100 120wi 10 20 30
vi /wi 6 5 4 W = 50 Greedy solution
Take items 1 and 2 value = 160, weight = 30 Have 20 pounds of capacity left over
Optimal solution Take items 2 and 3 value = 220, weight = 50 No leftover capacity
0-1 Knapsack Problem
Networking Laboratory 29/45
![Page 30: Lecture 9. Greedy Algorithm - SKKUmonet.skku.edu/wp-content/uploads/2016/05/Algorithm_09.pdf · 2016-05-31 · Algorithms Greedy Algorithms A greedy algorithm always makes the choice](https://reader037.vdocuments.net/reader037/viewer/2022100223/5f0ea8457e708231d4404be6/html5/thumbnails/30.jpg)
Algorithms
Knapsack Problem
Networking Laboratory 30/45
![Page 31: Lecture 9. Greedy Algorithm - SKKUmonet.skku.edu/wp-content/uploads/2016/05/Algorithm_09.pdf · 2016-05-31 · Algorithms Greedy Algorithms A greedy algorithm always makes the choice](https://reader037.vdocuments.net/reader037/viewer/2022100223/5f0ea8457e708231d4404be6/html5/thumbnails/31.jpg)
Algorithms
0-1 KnapsackDynamic Programming
P(i, w) – the maximum profit that can be obtained from items 1 to i, if the knapsack has size w
Case 1: thief takes item i P(i, w) =
Case 2: thief does not take item i P(i, w) =
vi + P(i - 1, w-wi)
P(i - 1, w)
Networking Laboratory 31/45
![Page 32: Lecture 9. Greedy Algorithm - SKKUmonet.skku.edu/wp-content/uploads/2016/05/Algorithm_09.pdf · 2016-05-31 · Algorithms Greedy Algorithms A greedy algorithm always makes the choice](https://reader037.vdocuments.net/reader037/viewer/2022100223/5f0ea8457e708231d4404be6/html5/thumbnails/32.jpg)
Algorithms
0 0 0 0 0 0 0 0 0 0 0
0
0
0
0
0
0
0:
n
1 w - wi W
i-1
0
first
P(i, w) = max {vi + P(i - 1, w-wi), P(i - 1, w) }
Item i was taken Item i was not taken
i
w
second
0-1 KnapsackDynamic Programming
Networking Laboratory 32/45
![Page 33: Lecture 9. Greedy Algorithm - SKKUmonet.skku.edu/wp-content/uploads/2016/05/Algorithm_09.pdf · 2016-05-31 · Algorithms Greedy Algorithms A greedy algorithm always makes the choice](https://reader037.vdocuments.net/reader037/viewer/2022100223/5f0ea8457e708231d4404be6/html5/thumbnails/33.jpg)
Algorithms
P(i, w) = max {vi + P(i - 1, w-wi), P(i - 1, w) }
0 0 0 0 0 0
0
0
0
0
Item Weight Value
1 2 12
2 1 10
3 3 20
4 2 150 1 2 3 4 5
1
2
3
4
W = 5
0
12 12 12 12
10 12 22 22 22
10 12 22 30 32
10 15 25 30 37
P(1, 1) =
P(1, 2) =
P(1, 3) =
P(1, 4) =
P(1, 5) =
P(2, 1)=
P(2, 2)=
P(2, 3)=
P(2, 4)=
P(2, 5)=
P(3, 1)=
P(3, 2)=
P(3, 3)=
P(3, 4)=
P(3, 5)=
P(4, 1)=
P(4, 2)=
P(4, 3)=
P(4, 4)=
P(4, 5)=
max{12+0, 0} = 12
max{12+0, 0} = 12
max{12+0, 0} = 12
max{12+0, 0} = 12
max{10+0, 0} = 10
max{10+0, 12} = 12
max{10+12, 12} = 22
max{10+12, 12} = 22
max{10+12, 12} = 22
P(2,1) = 10
P(2,2) = 12
max{20+0, 22}=22
max{20+10,22}=30
max{20+12,22}=32
P(3,1) = 10
max{15+0, 12} = 15
max{15+10, 22}=25
max{15+12, 30}=30
max{15+22, 32}=37
0
P(0, 1) = 0
Example:
Networking Laboratory 33/45
![Page 34: Lecture 9. Greedy Algorithm - SKKUmonet.skku.edu/wp-content/uploads/2016/05/Algorithm_09.pdf · 2016-05-31 · Algorithms Greedy Algorithms A greedy algorithm always makes the choice](https://reader037.vdocuments.net/reader037/viewer/2022100223/5f0ea8457e708231d4404be6/html5/thumbnails/34.jpg)
Algorithms
Reconstructing the Optimal Solution
0 0 0 0 0 0
0
0
0
0
0 1 2 3 4 5
1
2
3
4
0
12 12 12 12
10 12 22 22 22
10 12 22 30 32
10 15 25 30 37
0
Start at P(n, W) When you go left-up
item i has been taken
When you go straight up item i has not been taken
• Item 4
• Item 2
• Item 1
Networking Laboratory 34/45
![Page 35: Lecture 9. Greedy Algorithm - SKKUmonet.skku.edu/wp-content/uploads/2016/05/Algorithm_09.pdf · 2016-05-31 · Algorithms Greedy Algorithms A greedy algorithm always makes the choice](https://reader037.vdocuments.net/reader037/viewer/2022100223/5f0ea8457e708231d4404be6/html5/thumbnails/35.jpg)
Algorithms
Define P[k,w] to be the best selection from Sk with weight at most w
Since P[k,w] is defined in terms of P[k-1,*], we can use two arrays of instead of a matrix
Running time: O(nW) Not a polynomial-time algorithm
since W may be large This is a pseudo-polynomial time
algorithm
Algorithm 0-1Knapsack(S, W):Input: set S of n items with benefit vi
and weight wi; maximum weight WOutput: benefit of best subset of S with
weight at most Wlet A and B be arrays of length W + 1for w ← 0 to W do
B[w] ← 0for k ← 1 to n do
copy array B into array A for w ← wk to W do
if A[w−wk] + vk > A[w] thenB[w] ← A[w−wk] + vk
return B[W]
−+−−>−
= else]},1[,],1[max{
if ],1[],[
wkPvwwkPwwwkP
wkPkk
k
0-1 KnapsackDynamic Programming
Networking Laboratory 36/45
![Page 36: Lecture 9. Greedy Algorithm - SKKUmonet.skku.edu/wp-content/uploads/2016/05/Algorithm_09.pdf · 2016-05-31 · Algorithms Greedy Algorithms A greedy algorithm always makes the choice](https://reader037.vdocuments.net/reader037/viewer/2022100223/5f0ea8457e708231d4404be6/html5/thumbnails/36.jpg)
Algorithms
Self-Study
Section 16.3 Huffman codes
Networking Laboratory 37/45
![Page 37: Lecture 9. Greedy Algorithm - SKKUmonet.skku.edu/wp-content/uploads/2016/05/Algorithm_09.pdf · 2016-05-31 · Algorithms Greedy Algorithms A greedy algorithm always makes the choice](https://reader037.vdocuments.net/reader037/viewer/2022100223/5f0ea8457e708231d4404be6/html5/thumbnails/37.jpg)
Algorithms
Huffman Codes
Huffman codes compress data very effectively savings of 20% to 90% are typical
Data compression by assigning binary codes to characters Huffman’s greedy algorithm uses a table giving how often each
character occurs (i.e., its frequency) It builds up an optimal way of representing each character as a
binary string.
Example: Storing a data file of 100,000 characters contains only six different characters: a, b, c, d, e ,f
300,000 bits224,000 bits
Networking Laboratory 38/45
![Page 38: Lecture 9. Greedy Algorithm - SKKUmonet.skku.edu/wp-content/uploads/2016/05/Algorithm_09.pdf · 2016-05-31 · Algorithms Greedy Algorithms A greedy algorithm always makes the choice](https://reader037.vdocuments.net/reader037/viewer/2022100223/5f0ea8457e708231d4404be6/html5/thumbnails/38.jpg)
Algorithms
Prefix Codes (1/2) In a prefix code, no code is also a prefix of some other code
Any optimal character code has an equally optimal prefix code Example: “abc” 0.101.100 = 0101100
Easy encoding and decoding Use a binary tree for decoding Once a string of bits matches a character code, out put that character
with no ambiguity (no need to look ahead)
Networking Laboratory 39/45
![Page 39: Lecture 9. Greedy Algorithm - SKKUmonet.skku.edu/wp-content/uploads/2016/05/Algorithm_09.pdf · 2016-05-31 · Algorithms Greedy Algorithms A greedy algorithm always makes the choice](https://reader037.vdocuments.net/reader037/viewer/2022100223/5f0ea8457e708231d4404be6/html5/thumbnails/39.jpg)
Algorithms
Prefix Code (2/2)
An optimal code for a file is always represented by a full binary tree in which every non-leaf node has two children
Given a tree T corresponding to a prefix code, we can easily compute the number of bits required to encode a file Let C = set of unique characters in file Let f(c) = frequency of character c in file Let dT(c) = depth of c’s leaf node in T The number of bits required to encode a file (cost of the tree T)
)()(][ cdcfTB TCc∑∈
=
Networking Laboratory 40/45
![Page 40: Lecture 9. Greedy Algorithm - SKKUmonet.skku.edu/wp-content/uploads/2016/05/Algorithm_09.pdf · 2016-05-31 · Algorithms Greedy Algorithms A greedy algorithm always makes the choice](https://reader037.vdocuments.net/reader037/viewer/2022100223/5f0ea8457e708231d4404be6/html5/thumbnails/40.jpg)
Algorithms
Constructing a Huffman Code (1/2) Build a tree T by “merging” |C| nodes in a bottom-up manner Use a min-priority queue Q to keep nodes ordered by
frequency
Networking Laboratory 41/45
![Page 41: Lecture 9. Greedy Algorithm - SKKUmonet.skku.edu/wp-content/uploads/2016/05/Algorithm_09.pdf · 2016-05-31 · Algorithms Greedy Algorithms A greedy algorithm always makes the choice](https://reader037.vdocuments.net/reader037/viewer/2022100223/5f0ea8457e708231d4404be6/html5/thumbnails/41.jpg)
Algorithms
Constructing a Huffman Code (2/2)
Networking Laboratory 42/45
![Page 42: Lecture 9. Greedy Algorithm - SKKUmonet.skku.edu/wp-content/uploads/2016/05/Algorithm_09.pdf · 2016-05-31 · Algorithms Greedy Algorithms A greedy algorithm always makes the choice](https://reader037.vdocuments.net/reader037/viewer/2022100223/5f0ea8457e708231d4404be6/html5/thumbnails/42.jpg)
Algorithms
Correctness of Huffman’s Algorithm (1/3)
Huffman exhibits the greedy choice property Given a tree T representing an arbitrary optimal prefix code Let a and b be two characters that are sibling leaves of maximum
depth in a tree T Assume x and y have lowest frequencies
There exists an optimal code in which x and y are at the maximum depth (greedy choice)
Prove that moving x to the bottom (similarly, y to the bottom) yields a better (optimal) solution
Networking Laboratory 43/45
![Page 43: Lecture 9. Greedy Algorithm - SKKUmonet.skku.edu/wp-content/uploads/2016/05/Algorithm_09.pdf · 2016-05-31 · Algorithms Greedy Algorithms A greedy algorithm always makes the choice](https://reader037.vdocuments.net/reader037/viewer/2022100223/5f0ea8457e708231d4404be6/html5/thumbnails/43.jpg)
Algorithms
Correctness of Huffman’s Algorithm (2/3) Assume f(x) ≤ f(y) and f(a) ≤ f(b). We know f(x) ≤ f(a) and f(y) ≤ f(b)
B[T] ≥ B[T’] (similarly, B[T] ≥ B[T”])
)()()()(][][ '' cdcfcdcfTBTB T
CcT
Cc∑∑∈∈
−=−
0))()())(()(()()()()()()()()(
)()()()()()()()( ''
≥−−=+−+=
+−+=
xdadxfafxdafadxfadafxdxfadafxdxfadafxdxf
TT
TTTT
TTTT
Networking Laboratory 44/45
![Page 44: Lecture 9. Greedy Algorithm - SKKUmonet.skku.edu/wp-content/uploads/2016/05/Algorithm_09.pdf · 2016-05-31 · Algorithms Greedy Algorithms A greedy algorithm always makes the choice](https://reader037.vdocuments.net/reader037/viewer/2022100223/5f0ea8457e708231d4404be6/html5/thumbnails/44.jpg)
Algorithms
Correctness of Huffman’s Algorithm (3/3)
Huffman exhibits optimal substructure Consider the optimal tree T for characters C Let x and y be lowest frequency characters in C Consider the optimal tree T’ for C’=C-{x,y}U{z}, where f(z)=f(x)+f(y) If there is a better tree for C’, call it T”, then we could use T” to build
a better original tree by adding in x and y under z The optimal tree T is optimal, so this is a contradiction Thus T’ is the optimal tree for C’
Huffman produces optimalprefix code Immediate from above claims
Networking Laboratory 45/45