Download - Evaluating & Improving Algorithms
Evaluating & Improving Algorithms
Analyzing US Change
While M>0 c <- value of largest coin with value <= M give c coin to customer M <- M-c• Three statements per loop • Loop executes k times, where k is…• Therefore, running time is 3*k
Generalized US Change
for i <- 0 to d-1 Calculate and give the max number of
this coin (c[i])• How many times does this loop run? (k)• How many operations are in calculate &
give max? (x)• Resulting time is k*x
General Rules for Calculating Time
• Steps in sequence are added• A function is replaced by its # steps (or
count each call as 1 step)• Total # of steps in a loop gets multiplied
by the # of times the loop executes
Summary so far…
• Original US change– O(N) where N is # coins given
• Generalized change algorithm– O(N) where N is # different coins in system– But this doesn’t work for all cases!
Brute Force* Change
N <- 1While (not done) construct a combination of N coins if it adds up to M return it (done=true) else N <- N+1
*Brute Force = try all combinations
How many loops?
• One loop for every combination with too few coins 1! + 2! + …
• One loop for every combination with the right number of coins before the correct one– Best case for this part is 0– Worst case for this part is k! where k is the
number of coins in the solution
Summary and Analysis
• Original US change– O(N) where N is # coins given (linear)
• Generalized change algorithm– O(N) where N is # different coins – O(1) for a fixed coin system (constant)
• Brute force change algorithm– O (N * N!) where N is # coins given– (Worse than 2N)!
What We’ve Learned
• Brute force algorithms (try all possibilities) are guaranteed to work, but…
• Brute force algorithms (usually) take too long
• We need ways to speed up the work
Detour: Recursion
• Specify a solution to a problem in terms of solutions to one or more smaller problems
• Mathematically elegant
Recursive Fibonacci
• Mathematical definition– F(0) = 0– F(1) = 1– F(N) = F(N-1) + F(N-2)
• Implemented directly:– F(4) = F(3)+F(2) = F(2)+F(1)+F(2) =
F(1)+F(0)+F(1)+F(2) = 1+1+1+F(2) = 3+F(1)+F(0) = 3+1+1 = 5
A Tree of Recursive CallsF(4)
F(3) F(2)
F(2) F(0)
1F(1) F(1)
F(1) F(0)1 1
1 1
Dynamic Programming
• Save intermediate results from which the final result can be computed
• Order the computations so you have all saved results before you need them
• Example: our Fibonacci Algorithm
Dynamic Programming For Fibonacci
• Generate answers from smallest to largest• Use previous answers in calculationsfibonacci (n) O(n) where n is input f[0] <- 1 f[1] <- 1 for i <- 2 to n-1 n times f[i] <- f[i-1]+f[i-2]1 step return f[n-1]
Recursion Isn’t Always Slower
• Recursive US Change (M)if (M>0)
c <- value of largest coin with value <= M give c coin to customer Recursive US Change(M-c)• Only one recursive call, at end• This is “tail recursion”, just as fast as the
original loop
Greedy Algorithms
• Order your algorithm to solve the “biggest chunk” first– Example: always give the biggest coin
possible in the change problem– Not every greedy algorithm is correct (e.g.
generalized change)
Branch and Bound
• Consider a brute force problem solution as a series of choices
• When a single choice is avoided (pruned), all future choices that depend on it are also avoided
• AI calls this “heuristic search”
Randomized
• Make each choice randomly until the problem is solved
• Not guaranteed to solve the problem• Technically not even algorithms, if not
guaranteed to halt (e.g. by time limit)• Sometimes we can prove that a “good
enough” solution is “likely enough” and these are simple to implement
Find the Phone (Brute Force)
Find the phone: if the phone is at your location return true (phone is found) else for each direction (left, right, forw, back) take a step if (find the phone) return true return false (phone not found here)
Find the Phone (Branch & Bound)
Find the phone: if the phone is at your location return true (phone is found) else listen to the phone & rule out opposite direction for each remaining direction take a step if (find the phone) return true return false (phone not found here)
Find the Phone (Greedy)Find the phone: if the phone is at your location return true (phone is found) else listen & pick best direction (left, right, forw, back) take a step if (find the phone) return true return false (phone not found here)
• What if there’s an irreplaceable Ming vase between you and the phone?
Find the Phone (Randomized)Find the phone: if the phone is at your location return true (phone is found) else repeat forever pick a random direction (left, right, forw, back) take a step if (find the phone) return true
Divide and Conquer
• If the problem is small enough, solve it directly
• Else– Break the problem down into 2 or more
subproblems– Solve each one separately (recursive step)– Combine the solutions
• Works best when subproblems are equal size
Divide & Conquer Example
• Quick Sort– Pick a value from the list (k)– Divide the input into “<k” list and “>k” list– Sort each list separately– Build the result:
• Sorted “<k” list followed by k followed by sorted “>=k” list
Machine Learning
• Given: many instances of input and output• Discover: a function that relates them• Generalize: to previously unseen inputs
• Generally need large amounts of representative data
• ML algorithms are fairly time consuming
Summary of Techniques
• Brute Force– Try every possibility– Rarely practical for reasonable size
problems• Dynamic programming
– Solve subproblems in the right order and save solutions to build the final answer
Summary (cont’d)
• Greedy Algorithms– Take the “biggest step” first each time. Not always
correct.• Branch and Bound
– Rule out impossible choices and their successors, but try everything else
• Divide and Conquer– Combine results from smaller subproblems
Summary (cont’d)
• Machine learning– Use large amounts of data to
“automatically” generate an appropriate solution algorithm
• Randomized algorithms– Try solutions at random until one is found
(or “good enough”)– Not always completely random (e.g.
genetic algorithms)