COMP-311: Algorithms and Data StructuresProblem Set 4

Juan Duchimaza

November 3, 2010

Contents

1 Book Problems 31.1 Problem 5.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.2 Problem 5.16 . . . . . . . . . . . . . . . . . . . . . . . . . . . 61.3 Problem 6.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61.4 Problem 6.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71.5 Problem 6.10 . . . . . . . . . . . . . . . . . . . . . . . . . . . 81.6 Problem 7.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81.7 Problem 7.31 . . . . . . . . . . . . . . . . . . . . . . . . . . . 91.8 Problem 9.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101.9 Problem 9.5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111.10 Problem 9.7 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131.11 Problem 9.10 . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

2 Appendix 162.1 Problem 6.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

2.1.1 Part a . . . . . . . . . . . . . . . . . . . . . . . . . . . 162.1.2 Part b . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

2.2 Problem 6.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

1

This page intentionally left blank.

2

Part 1

Book Problems

1.1 Problem 5.1

Given input {4371, 1323, 6173, 4199, 4344, 9679, 1989} and a hashfunction h(x) = x(mod 10), show the resulting:

a. Separate chaining hash table.

A separate chaining hash table simply keeps a list of all elements thathash to the same value, almost like a 2-D array. There is no need to worryabout collisions. Figure 1.1 shows the resulting hash table.

Figure 1.1: Separate chaining hash table for Problem 5.1 a.

3

b. Hash table using linear probing.

Using linear probing, the numbers that experience collision are 6173 (with1323), 4344 (with 6173), 9679 (with 4199), and 1989 (with 4199 and thenwith 9679). Figure 1.2 shows the resulting hash table.

After 4371 After 1323 After 6173 After 4199 After 4344 After 9679 After 19890 9679 96791 4371 4371 4371 4371 4371 4371 43712 19893 1323 1323 1323 1323 1323 13234 6173 6173 6173 6173 61735 4344 4344 43446789 4199 4199 4199 4199

Figure 1.2: Hash table using linear probing. A red color means the numberexperienced one or more collisions when inserted.

c. Hash table using quadratic probing.

With quadratic probing, after each successive probe you take a step equiv-alent to s = i2, where i is the number of probes you have done so far. Inthis hash table, most of the collisions require only one probe, so the tablefor quadratic probing in Figure 1.3 resembles the table for linear probing inFigure 1.2. The main issue arises when 1989 is inserted. After i = 1 probes,it looks at index 0, which is already full. After i = 2 probes, it looks atindex (0 + 22) mod 10 = 4, which is already full. After i = 3 probes, itlooks at index (4 + 32) mod 10 = 3. After i = 4 probes, it looks at index(3+42) mod 10 = 9. After i = 5 probes, it looks at index (9+52) mod 10 = 4again. After i = 6 probes, it looks at index (4 + 62) mod 10 = 0. After i = 7probes, it looks at index 9. After i = 8 probes, it looks at index 3. Afteri = 9 probes, it looks at index 4. After i = 10 probes, it looks at index 4again. After i = 11 probes, it looks at index 5. After i = 12 probes, it looksat index 9. Finally, after i = 13 probes, it looks at index 8, which it findsis empty. By this point, quadratic probing has taken a ridiculous amount ofsteps (818 steps!) to find a good place to hash 1989 into.

4

After 4371 After 1323 After 6173 After 4199 After 4344 After 9679 After 19890 9679 96791 4371 4371 4371 4371 4371 4371 437123 1323 1323 1323 1323 1323 13234 6173 6173 6173 6173 61735 4344 4344 4344678 19899 4199 4199 4199 4199

Figure 1.3: Hash table using quadratic probing. A red color means thenumber experienced one or more collisions when inserted.

d. Hash table with second hash function h2(x) = 7− (x mod 7).

With a second hash function, after the hashing function h(x) = x(mod 10)finds a collision, it continues to use h2(x) to properly hash the value. Thus,for example, when h(x) finds a collision when the attempt to insert 6173 ismade, h2(x) moves ahead 7− (6173 mod 7) = 7− 6 = 1 index spots.

The problem comes in when 1989 is hashed into the table. When itfinds that index spot 9 is taken, the second hash function moves ahead 7 −(1989 mod 7) = 7 − 1 = 6 index spots, to index number 5. From there, itmoves ahead another 6 index spots to 1, which is already occupied. Again,it will move ahead another 6 slots to 7, and then to slot number 3, 9, and onto 5 again. Because the second hashing function will keep tying to hash inthis cycle, the heap will at some point be deemed full and 1989 will not behashed.

After 4371 After 1323 After 6173 After 4199 After 4344 After 9679 After 198901 4371 4371 4371 4371 4371 4371 437123 1323 1323 1323 1323 1323 13234 6173 6173 6173 6173 61735 9679 967967 4344 4344 434489 4199 4199 4199 4199

Figure 1.4: Hash table using second hashing. A red color means the numberexperienced one or more collisions when inserted.

5

1.2 Problem 5.16

Under certain assumptions, the expected cost of an insertion intoa hash table with secondary clustering is given by 1/(1 − λ)− λ−ln(1− λ). Unfortunately, this formula is not accurate for quadraticprobing. However, assuming that it is, determine the following:

a. The expected cost of an unsuccessful search.

In a hash table with secondary clustering, the expected cost of an unsuc-cessful search is the same as the expected cost of an insertion. Thus,

E(c) =1

(1 − λ)− λ− ln(1− λ)

b. The expected cost of a successful search.

The expected cost of a successful search is:

E =1

λ

∫ λ

0

1

(1 − λ)− λ− ln(1− λ)

E =1

λ(

∫ λ

0

1

(1 − λ)−∫ λ

0

λ−∫ λ

0

ln(1− λ))

E =1

λ[ln(1 − λ)− λ2

2− (−(1− λ)(ln(1− λ)− 1))]λ0

E =1

λ(ln(1 − λ)− λ2

2+ (1− λ)(ln(1− λ)− 1) + 1)

1.3 Problem 6.2

a. Show the result of inserting 10, 12, 1, 14, 6, 5, 8, 15, 3, 9, 7, 4,11, 13, and 2, one at a time, into an initially empty binary heap.

The result of building the binary heap one by one is shown in Figure 1.5.A step-by-step solution is provided in the Appendix.

b. Show the result of using the linear-time algorithm to build abinary heap using the same input.

The result of building the binary heap by using the linear-time algorithmis shown in Figure 1.6. A step-by-step solution is provided in the Appendix.

6

1

3

6

15 14

7

12 9

2

5

10 11

4

13 8

Figure 1.5: Answer to problem 6.2a.

1

3

12

15 14

6

9 7

2

4

5 11

8

13 10

Figure 1.6: Answer to problem 6.2b.

1.4 Problem 6.3

Show the result of performing three deleteMin operations in theheap of the previous exercise.

The result of deleting 1, 2, and 3 from the tree in Figure heap is shownin Figure 1.7 on the next page. A step-by-step solution is provided in theAppendix.

1.5 Problem 6.10

a. Give an algorithm to find all nodes less than some value, X, ina binary heap. Your algorithm should run in O(K), where K is thenumber of nodes output.

The best approach to find all the nodes less than X in a binary heap isto recursively check each node, starting with the root and following with the

7

4

6

13

15 14

7

12 9

5

10

11

8

Figure 1.7: Answer to problem 6.3.

children, and comparing the value in that node to X. If the value is greaterthan or equal to X, that node’s children will have a greater value, and thatis as far as the search needs to go. Additionally, if it encounters a node thatdoes not have children (ie, a leaf), then start collapsing the recursive calls.The proposed O(K) algorithm is listed below.

1 void lessThan (Node curr )2 i f ( curr i s nu l l )3 re turn4 i f (X < curr )5 \\ add curr to queue , stack , p r in tout s t r i ng , array , e t c6 lessThan ( curr . l e f t )7 lessThan ( curr . r i g h t )

Figure 1.8: Answer to problem 6.10.

1.6 Problem 7.3

Suppose we exchange elements a[i] and a[i+k], which were origi-nally out of order. Prove that at least 1 and at most 2k−1 inversionsare removed.

In any list of n elements where there exist x inversions, exchanging ele-ments i and i+k, where a[i] > a[i+k], you will at least get rid of that oneinversion, especially in the case where k = 1 (a[i] and a[i+k] are consecu-tive elements).

In the worst case where the list sorted in a descending order (or anorder that is completely opposite to how it should be sorted), you have

8

x =n(n+ 1)

2inversions. By exchanging the elements, you will get rid of the

inversions coming from a[i] up until a[i+k], a total of k inversions:

{(a[i],a[i+1]), (a[i], a[i+2]), (a[i], a[i+3]), . . . , (a[i], a[i+k])}.

Additionally, by replacing a[i+k] with a larger value a[i], you will beremoving the inversions from all elements from a[i+1] through a[i+k-1]

for a total of k − 1 inversions:

{(a[i+1],a[i+k]), (a[i+2], a[i+k]), (a[i+3], a[i+k]), . . . , (a[i+(k-1)],a[i+k])}.

Thus, you remove k + k − 1 = 2k − 1 inversions by exchanging elementsa[i] and a[i+k].

1.7 Problem 7.31

A sorting algorithm is stable if elements with equal keys are leftin the same order as they occur in the input. Which of the sortingalgorithms in this chapter are stable and which are not? Why?

Insertion sort is stable because it checks a key in a specific element withthe keys in neighboring elements before swapping, and the only reasons forwhich it swaps elements is if it finds a difference (ie, compareTo < 0). If thekeys are the equal, then the sort does not swap elements.

Mergesort is stable because, given the routine from the book, sortingbegins from the left (earliest input) and move right. It stores an element inthe final array if it finds that the current element from the left array has akey that is lower than or equal to the current element from the right array(the array that is input later). Thus, the element that was input earlier isgiven preference.

Bucket sort is stable. As it creates the new, sorted array, it traversesthe input array beginning at the end. Meanwhile, the sort also populatesthe new array beginning from the last index spot at which an element witha given key may be found in the new array. Thus, by the end of sorting, theinput order is maintained.

Shellsort is not stable. As a counterexample, consider an array a ={5, 4, 31, 32, 33, 34}, where the subscript is used simply to distinguish the el-ements. After a 3-sort, the array will be a = {32, 33, 31, 5, 4, 34}. After thesubsequent 1-sort, the sorted array will be a = {32, 33, 31, 34, 4, 5}. Thus, itis clear that the order of input elements is not maintained.

9

Heapsort is not stable because by performing a deleteMax, it tacks onthe first element with a given key to a position later in the array. In theevent that elements have equal keys, heapsort will actually reverse the orderof the elements, where the first input will actually be the last output.

Quicksort can be either, stable or unstable, depending on where thepivot lays in relation to elements with similar keys. If the pivot happensto be the first or last instance of elements with that key, quicksort will bestable. If the pivot happens to be somewhere in the middle, quicksort maybe unstable.

1.8 Problem 9.1

Find a topological ordering for the graph in Figure 1.9

s D E F t

A B C

G H I

4

1

6

2

2

2

4

3 3

2

3

3

1 3

2 1

6

2

6

1 4

Figure 1.9: Graph used in Exercise 9.1

s → A → D → G → /A → B → E → /B → C → /C → t → /D → A → E → /E → C → F → I → /F → C → t → /G → D → E → H → /H → E → I → /I → F → t → /t → /

Table 1: Corresponding representation of the graph in Figure 1.9.

10

s A B C D E F G H I t Enqueue Dequeue

0 2 1 3 2 4 2 1 1 2 3 s s0 1 1 3 1 4 2 0 1 2 3 G G0 1 1 3 0 3 2 0 0 2 3 D, H D0 0 1 3 0 2 2 0 0 2 3 A H0 0 1 3 0 1 2 0 0 1 3 A0 0 0 3 0 0 2 0 0 1 3 B, E B0 0 0 2 0 0 2 0 0 1 3 E0 0 0 1 0 0 1 0 0 0 3 I I0 0 0 1 0 0 0 0 0 0 2 F F0 0 0 0 0 0 0 0 0 0 1 C C0 0 0 0 0 0 0 0 0 0 0 t t0 0 0 0 0 0 0 0 0 0 0

Table 2. Application of topological sort algorithm to find a potential topo-logical order of the graph in Figure 1.9.

A possible topological order for this graph is:

• s, G, D, H, A, B, E, I, F, C, t

1.9 Problem 9.5

a. Find the shortest path from A to all other vertices for the graphin Figure 1.10.

Applying Dijkstra’s Algorithm to the graph:

Visited A B C D E F GA 0 5 3 ∞ ∞ ∞ ∞A, C 0 5 3 10 10 ∞ ∞A, C, B 0 5 3 10 8 ∞ 6A, C, B, G 0 5 3 10 7 ∞ 6A, C, B, G, E 0 5 3 9 7 8 6A, C, B, G, E, F 0 5 3 9 7 8 6A, C, B, G, E, F, D 0 5 3 9 7 8 6

Turning this table into a graph, Figure 1.10 becomes:

11

A C E

B

G

D F

5

3

1

23

1

7

7

2 12

6

Figure 1.10: Graph used in problem 9.5.

b. Find the shortest unweighted path from B to all other ver-tices for the graph in the Figure 1.10.

In this case, you simply add nodes to the path as you encounter them.Starting with B, you can instantly get to C, E, and G. From there, you cantake one more step to reach D from C, and F from E. Finally, you can takethe last step to connect to A from D, and all the nodes are now included inthe path.

1.10 Problem 9.7

a. Give an example where Dijkstra’s algorithm gives the wrong an-swer in the presence of a negative edge but no negative-cost cycle.

Take for example a tree as illustrated in Figure 1.10. Dijkstra’s algorithm,starting at A, gives a path from A to C with cost of 5, when path A-B-C hasa cost of 7 + (−4) = 3, meaning that it is indeed the shortest path to C.

b. Show that the weighted shortest-path algorithm suggestedin Section 9.3.3 works if there are negative-weight edges, but nonegative-cost cycles, and that the running time of this algorithm

12

A C E

B G

D F

5

3

1

1

2 1

is O(|E| · |V |).

This algorithm will go through each vertex for each edge in the graph,giving it an O(|E| · |V |) asymptotic complexity. For simplicity, this algorithmwill be used on Figure 1.10. This shows that you do get a constantly updatedvalue for the distance to the node you are looking at, and may only updateit if you find a less costly path to it.

v v.dist w w.dist cvw w.dist update? (value)A 0 C ∞ 5 T (5)A 0 B ∞ 7 T (7)B 7 C 5 -4 T (3)

1.11 Problem 9.10

a. Explain how to modify Dijkstra’s algorithm to produce a countof the number of different minimum paths from v to w.

Use an array to maintain the counter of paths from v to any other nodein the graph, and associate each index with a given node. While you checkadjacent nodes to nodes that have been taken off the queue (as you expandthe visibility cloud), compare w.dist with v.dist + cvw. If the quantities areequal, add 1 to the element in the corresponding array index. If you findthat v.dist + cvw is less that w.dist, reset that element to 0 because youhave found a new “shortest” path.

13

A C E

B G

D F

1

1 1

2 23

Figure 1.11: Answer to problem 9.5b

b. Explain how to modify Dijkstra’s algorithm so that if there ismore than one minimum path from v to w, a path with the fewestnumber of edges is chosen.

Similar to maintaining a count of different minimum paths, another arraycan be used to keep a count of minimum edges. This will also require avariable that will keep a count of how many edges the path contains, assumingthat all edges lead to a node, which will be incremented as a node is dequeuedand decremented when w.dist is less than v.dist + cvw (you start going backto find other, cheaper paths). Each time something is enqueued as a newpath, you store this value in the respective array index.

14

A

B

C

5

7

-4

Figure 1.12: Answer to problem 9.7

15

Part 2

Appendix

2.1 Problem 6.2

2.1.1 Part a

10 10

12

1

12 10

1

12

14

10

1

6

14 12

10

1

6

14 12

5

10

1

6

14 12

5

10 8

1

6

14

15

12

5

10 8

1

3

6

15 14

12

5

10 8

16

1

3

6

15 14

9

12

5

10 8

1

3

6

15 14

7

12 9

5

10 8

1

3

6

15 14

7

12 9

4

5

10

8

1

3

6

15 14

7

12 9

4

5

10 11

8

1

3

6

15 14

7

12 9

4

5

10 11

8

13

1

3

6

15 14

7

12 9

2

5

10 11

4

13 8

Figure 2.1: Full solution to problem 6.2a.

In Figure 2.1, the blue nodes are the nodes that are added at each step.

17

2.1.2 Part b

10

12

14

15 3

6

9 7

1

5

4 11

8

13 2

10

12

3

15 14

6

9 7

1

4

5 11

2

13 8

10

3

12

15 14

6

9 7

1

4

5 11

2

13 8

1

3

12

15 14

6

9 7

2

4

5 11

8

13 10

Figure 2.2: Full solution to problem 6.2b.

In Figure 2.2, the green subtree is the current tree you’re“heapifying,”the blue nodes are the nodes that are smaller than the absolute root in theminimum tree.

18

2.2 Problem 6.3

1

3

6

15 14

7

12 9

2

5

10 11

4

13 8

3

6

15 14

7

12 9

2

5

10 11

4

13 8

2

3

6

15 14

7

12 9

4

5

10 11

8

13

1 is deleted 2, 4, and 8 percolate up to fill up hole

3

6

15 14

7

12 9

4

5

10 11

8

13

3

6

13

15 14

7

12 9

4

5

10 11

8

3 is deleted 3, 6 percolate up, 13 moves to fill hole

6

13

15 14

7

12 9

4

5

10 11

8

4

6

13

15 14

7

12 9

5

10

11

8

3 deleted 4, 5, 10 percolate up, 11 moves

Figure 2.3: Full solution to problem 6.3.

19