MPATE-GE 2618: C Programming for Music Technology

Unit 4.4

Exhaustive searches •  An exhaustive search (aka “brute force”) algorithm produces the

entire solution space for the problem (typically represented as a tree data structure).

•  Searching a tree: –  Breadth-!rst search - proceeds by generating and testing each node

that is reachable from a parent node before it expands any of these children.

–  Depth-!rst search - the tree is examined to maximum possible depth before another path is considered

–  ese search are said to be exhaustive because they are guaranteed to generate all reachable states before terminating with failure.

•  Exhaustive sorting algorithm: produces a permutation of the array, checks to see if it is sorted, and continues until it reaches the !rst permutation that is sorted, giving the algorithm a running time of O(n!).

•  Exhaustive searches are also known as backtracking algorithms, although not all backtracking algorithms are exhaustive.

3

2 6

1 8 5 9 1 2 3 4

5 6

7

3

2 6

1 8 5 9 4 5 6 7

2 3

1

Breadth-!rst (numbers below nodes indicate order searched).

Depth-!rst

Greedy algorithms •  A greedy algorithm is de!ned by making choices

that are locally optimal at each stage with the hope of !nding a globally optimal solution.

•  e choice it makes is the one that seems best at the moment––it may depend on choices made so far but not on future choices or all the solutions to the subproblem. It iteratively makes one greedy choice after another, reducing each given problem into a smaller one. In other words, a greedy algorithm never reconsiders its choices.

•  For most problems, greedy algorithms mostly (but not always) fail to !nd the globally optimal solution, because they usually do not operate exhaustively on all the data. ey can make commitments to certain choices too early which prevent them from !nding the best overall solution later.

Shortest path problem (Dijkstra’s algorithm)

V0 V1

V3

V5 V6

V2 V4

1

2 3

2 2

1

5

10

1

4 8 6

Start

Problem: given a directed graph with weighted, non-negative edges (e.g. distances), !nd the shortest path from V0 to all other nodes. is algorithm is often used for network routing; also useful for !nding the shortest path between two points on a map (e.g. when you Google for driving directions from one location to another).

Shortest path problem (Dijkstra’s algorithm)

V0 V1

V3

V5 V6

V2 V4

1

2 3

2 2

1

5

10

1

4 8 6

0

Dijkstra’s algorithm is an example of a greedy algorithm because it just chooses the closest “frontier” vertex at every step

Shortest path problem (Dijkstra’s algorithm)

V0 V1

V3

V5 V6

V2 V4

1

2 3

2 2

1

5

10

1

4 8 6

0 2

1

Shortest path problem (Dijkstra’s algorithm)

V0 V1

V3

V5 V6

V2 V4

1

2 3

2 2

1

5

10

1

4 8 6

0 2

1

Shortest path problem (Dijkstra’s algorithm)

V0 V1

V3

V5 V6

V2 V4

1

2 3

2 2

1

5

10

1

4 8 6

0 2

1

Shortest path problem (Dijkstra’s algorithm)

V0 V1

V3

V5 V6

V2 V4

1

2 3

2 2

1

5

10

1

4 8 6

0

3 1

2

3

5 9

Shortest path problem (Dijkstra’s algorithm)

V0 V1

V3

V5 V6

V2 V4

1

2 3

2 2

1

5

10

1

4 8 6

0

3 1

2

3

5 9

Shortest path problem (Dijkstra’s algorithm)

V0 V1

V3

V5 V6

V2 V4

1

2 3

2 2

1

5

10

1

4 8 6

0

3 1

2

3

5 9

Shortest path problem (Dijkstra’s algorithm)

V0 V1

V3

V5 V6

V2 V4

1

2 3

2 2

1

5

10

1

4 8 6

0

3 1

2

3

5 9

Shortest path problem (Dijkstra’s algorithm)

V0 V1

V3

V5 V6

V2 V4

1

2 3

2 2

1

5

10

1

4 8 6

0

3 1

2

3

5 9

Shortest path problem (Dijkstra’s algorithm)

V0 V1

V3

V5 V6

V2 V4

1

2 3

2 2

1

5

10

1

4 8 6

0

3 1

2

3

5 9

Shortest path problem (Dijkstra’s algorithm)

V0 V1

V3

V5 V6

V2 V4

1

2 3

2 2

1

5

10

1

4 8 6

0

3 1

2

3

5 9

Shortest path problem (Dijkstra’s algorithm)

V0 V1

V3

V5 V6

V2 V4

1

2 3

2 2

1

5

10

1

4 8 6

0

3 1

2

3

5 8

Shortest path problem (Dijkstra’s algorithm)

V0 V1

V3

V5 V6

V2 V4

1

2 3

2 2

1

5

10

1

4 8 6

0

3 1

2

3

5 8

Shortest path problem (Dijkstra’s algorithm)

V0 V1

V3

V5 V6

V2 V4

1

2 3

2 2

1

5

10

1

4 8 6

0

3 1

2

3

5 8

Shortest path problem (Dijkstra’s algorithm)

V0 V1

V3

V5 V6

V2 V4

1

2 3

2 2

1

5

10

1

4 8 6

0

3 1

2

3

5 6

Shortest path problem (Dijkstra’s algorithm)

V0 V1

V3

V5 V6

V2 V4

1

2 3

2 2

1

5

10

1

4 8 6

0

3 1

2

3

5 6

Shortest path problem (Dijkstra’s algorithm)

V0 V1

V3

V5 V6

V2 V4

1

2 3

2 2

1

5

10

1

4 8 6

0

3 1

2

3

5 6

Shortest path problem (Dijkstra’s algorithm)

V0 V1

V3

V5 V6

V2 V4

1

2 3

2 2

1

5

10

1

4 8 6

0

3 1

2

3

5 6

Fibonacci sequence revisited 0, if n = 0 f(n) = 1, if n = 1

f(n-1) + f(n-2) otherwise

int fibonacci(int n) { if(n == 0) { return 0; } if(n == 1) { return 1; } else { return fibonacci(n - 1) + fibonacci(n - 2); } }

Fibonacci: Computations

See !les fs1.c, fs2.c and fs3.c

f(5)

f(4) f(3)

f(3) f(2) f(2) f(1)

f(2) f(1) f(1) f(0) f(1) f(0)

f(1) f(0)

5

3 2

2 1 1 1

1 0 0 1 1

0

1

1

Memoization •  In order to avoid recalculating something we’ve

already calculated before, the solutions are saved. •  en, if the same problem needs to be solved later,

we can retrieve and reuse the already-computed solution.

•  Memoization lowers function’s time cost in exchange for space cost; that is, memoized functions become optimized for speed in exchange for higher memory usage.

Dynamic programming •  Memoization is an optimization technique used in dynamic programming. •  e word “programming” in this case has no particular connection to

computer programming at all, and instead comes from the term “mathematical programming,” a synonym for optimization. us, the “program” is the optimal plan for action that is produced.

•  Often, dynamic programming algorithms are visualized as “!lling an array” where each element of the array is the result of a subproblem that can later be reused rather than recalculated.

•  In order for dynamic programming to be applicable, there has to be two ingredients that an optimization problem must have: –  Optimal substructure –  Overlapping subproblems

Optimal substructure •  Optimal substructure means an optimal solution to a

problem contains optimal solutions to subproblems. •  e.g. a subpath of a shortest path is also a shortest path. •  Example: Find the shortest path between two vertices on a

graph: !rst compute the shortest path to the goal from all vertices adjacent to the start, and then use this to pick the best overall path.

A straight line indicates a single connector (edge); a wavy line indicates a shortest path between the two vertices it connects (other nodes on these paths are not shown); the bold line is the overall shortest path from start to goal.

Overlapping subproblems •  To say that a problem has overlapping subproblems is to say that the same

subproblems are used to solve many different larger problems. •  In other words, subproblems share subsubproblems. A dynamic-

programming algorithm solves every subsubproblem just once and then saves its answer in a table, thereby avoiding the work of recomputing the answer every time the subsubproblem is encountered.

•  Example from before: for Fibonacci, f3 = f1 + f2 and f4 = f2 + f3 — computing each number involves computing f2. Because both f3 and f4 are needed to compute f5, a naive approach to computing f5 would end up computing f2 twice or more. –  A recursive solution contains a “small” number of distinct subproblems

repeated many times.

Greedy vs. dynamic programming •  Because the optimal-substructure property is exploited by both greedy

and dynamic-programming strategies, they can be misused. •  Example: the 0-1 knapsack program vs. the fractional knapsack problem:

–  0-1 knapsack: ief robbing a store !nds n items; the ith item is worth vi dollars and weighs wi pounds. He wants to take as valuable a load as possible, but can carry at most W pounds in his knapsack. What items should he take?

–  Fractional knapsack: same setup as above, but the thief can take fractions of items, rather than having to make a binary (0-1) choice. How would you solve this problem?

–  Both exhibit the optimal-substructure property •  0-1: If item j is removed from an optimal packing, the remaining

packing is an optimal packing with weight at most W - wj

•  Fractional: If w pounds of item j is removed from an optimal packing, the remaining packing is an optimal packing with weight at most W - w that can be taken from other n-1 items plus wj – w of item j

Fractional knapsack problem

•  Fractional knapsack can be solvable by the greedy strategy –  Compute the value per pound vi/wi for each item –  Obeying a greedy strategy, take as much as possible of the

item with the greatest value per pound. –  If the supply of that item is exhausted and there is still

more room, take as much as possible of the item with the next value per pound, and so forth until there is no more room.

0-1 knapsack problem

•  0-1 knapsack cannot be solved by the greedy strategy –  Unable to !ll the knapsack to capacity, and the empty space lowers

the effective value per pound of the packing –  We must compare the solution to the sub-problem in which the item

is included with the solution to the sub-problem in which the item is excluded before we can make the choice

•  Take the case where there are 3 items, the knapsack can hold 50 pounds. Item 1 weighs 10 pounds and is worth $60; item 2 weighs 20 pounds and is worth $100; item 3 weighs 30 pounds and is worth $120.

0-1 knapsack continued

•  e greedy strategy does not work for the 0-1 knapsack problem. (a) e thief must select a subset of the three items shown whose weight must not exceed 50 pounds. (b) e optimal subset includes items 2 and 3. Any solution with item 1 is suboptimal, even though item 1 has the greatest value per pound. (c) For the fractional knapsack problem, taking the items in order of greatest value per pound yields an optimal solution. (Figure taken from Cormen, Leiserson & Rivest).