1
RecursionAlgorithm Analysis
Standard Algorithms
Chapter 7
2
Recursion
• Consider things that reference themselves– Cat in the hat in the hat in the hat …– A picture of a picture– Having a dream in your dream!!
• Recursion has its base in mathematical induction
• Recursion always has– an anchor (or base or trivial) case– an inductive case
3
Recursion
• A recursive function will call or reference itself.
• Considerint R(int x){ return 1 + R(x); }
• What is wrong with this picture?– Nothing will stop repeated recursion– Like an endless loop, but will eventually cause
your program to run out of memory
The problem is that this function has no anchor.
The problem is that this function has no anchor.
4
Recursion
A proper recursive function will have• An anchor or base case
– the function’s value is defined for one or more values of the parameters
• An inductive or recursive step – the function’s value (or action) for the current
parameter values is defined in terms of …– previously defined function values (or
actions) and/or parameter values.
5
Recursive Example
int Factorial(int n){ if (n == 0) return 1; else return n * Factorial(n - 1);}
• Which is the anchor?• Which is the inductive or recursive part?• How does the anchor keep it from going forever?
6
A Bad Use of Recursion
• Fibonacci numbers1, 1, 2, 3, 5, 8, 13, 21, 34f1 = 1, f2 = 1 … fn = fn -2 + fn -1
– A recursive functiondouble Fib (unsigned n){ if (n <= 2) return 1; else return Fib (n – 1) + Fib (n – 2); }
• Why is this inefficient?– Note the recursion tree on pg 327
7
Uses of Recursion
• Easily understood recursive functions are not always the most efficient algorithms
• "Tail recursive" functions– When the last statement in the recursive function is a
recursive invocation.– These are much more efficiently written with a loop
• Elegant recursive algorithms– Binary search (see pg 328)– Palindrome checker (pg 330)– Towers of Hanoi solution (pg 336)– Parsing expressions (pg 338)
8
Comments on Recursion
• Many iterative tasks can be written recursively– but end up inefficient
However• There are many problems with good
recursive solutions• And their iterative solutions are
– not obvious– difficult to develop
9
Algorithm Efficiency
• How do we measure efficiency– Space utilization – amount of memory
required– Time required to accomplish the task
• Time efficiency depends on :– size of input– speed of machine – quality of source code– quality of compiler
These vary from one platform to another
These vary from one platform to another
10
Algorithm Efficiency
• We can count the number of times instructions are executed– This gives us a measure of efficiency of an
algorithm
• So we measure computing time as:
T(n) = computing time of an algorithm for input of size n = number of times the instructions are executed
11
Example: Calculating the Mean
Task # times executed
1. Initialize the sum to 0 12. Initialize index i to 0 13. While i < n do following n+14. a) Add x[i] to sum n5. b) Increment i by 1 n6. Return mean = sum/n 1 Total 3n + 4
12
Computing Time Order of Magnitude
• As number of inputs increases T(n) = 3n + 4 grows at a rate proportional to n
• Thus T(n) has the "order of magnitude" n
• The computing time of an algorithm on input of size n, T(n) said to have order of magnitude f(n), written T(n) is O(f(n))
if … there is some constant C such that T(n) < Cf(n) for all sufficiently large values of n
13
Big Oh Notation
Another way of saying this:• The complexity of the algorithm is
O(f(n)).
• Example: For the Mean-Calculation Algorithm:
T(n) is O(n)
• Note that constants and multiplicative factors are ignored.
14
Big Oh Notation
• f(n) is usually simple:
n, n2, n3, ...2n
1, log2nn log2nlog2log2n
15
Big-O Notation
• Cost function– A numeric function that gives performance of
an algorithm in terms of one or more variables– Typically the variable(s) capture number of
data items
• Actual cost functions are hard to develop
• Generally we use approximating functions
16
Function Dominance
• Asymptotic dominance– g dominates f if there is a positive constant c
such that
• Example: suppose the actual cost function is
• Both of these will dominate T(n)
( ) ( )c g x f x
2( ) 1.01T n n
2
3
2( ) 1.5
3( )
T n n
T n n
for sufficiently large values of n
17
Estimating Functions
Characteristics for good estimating functions
• It asymptotically dominates the actual time function
• It is simple to express and understand
• It is as close an estimate as possible
2
2
( ) 5 100
( )
actual
estimate
T n n n
T n n
Because any constant c > 1 will make n2 larger
18
Estimating Functions
• Note how the c*n2 dominates
Function Dominance
020000400006000080000
0 100 200 300
Number of data items
Tim
e
Actual
n^2
1.5n^2
Thus we use n2 as an estimate of the
time required
19
Order of a Function
• To express time estimates concisely we use the concept “order of a function”
• Definition:Given two nonnegative functions f and g, the order of f is g, iff g asymptotically dominates f
• Stated – “f is of order g”– “f = O(g)” big-O notation
O stands for “Order”
20
Order of a Function
• Note the possible confusion– The notation does NOT say “the order of g is f”
nor does it say “f equals the order of g”
– It does say “f is of order g” ( )f O g
21
Big-O Arithmetic
• Given f and g functions, k a constant
1. ( ) ( )
2. ( ) ( ) ( )
( / ) ( ) / ( )
3. ( ) ( ) dominates
4. ( ) [ ( ), ( )]
O k f O f
O f g O f O g and
O f g O f O g
O f O g f g
O f g Max O f O g
22
Example: Calculating the Mean
Task # times executed
1. Initialize the sum to 0 12. Initialize index i to 0 13. While i < n do following n+14. a) Add x[i] to sum n5. b) Increment i by 1 n6. Return mean = sum/n 1 Total 3n + 4
Based on Big-O arithmetic this
algorithm has O(n)
Based on Big-O arithmetic this
algorithm has O(n)
23
Worst-Case Analysis
• The arrangement of the input items may affect the computing time.
• How then to measure performance?– best case not very informative – average too difficult to calculate– worst case usual measure
• Consider Linear search of the list a[0], . . . , a[n – 1].
24
Worst-Case Analysis
Algorithm:1. found = false.2. loc = 0.3. While (loc < n && !found )4. If item = a[loc] found = true // item found 5. Else Increment loc by 1 // keep searching
• Worst case: Item not in the list: TL(n) is O(n)
• Average case (assume equal distribution of values) is O(n)
Linear search of a[0] … a[n-1]
Linear search of a[0] … a[n-1]
25
Binary Search
1. found = false.2. first = 0.3. last = n – 1.4. While (first < last && !found )
5. Calculate loc = (first + last) / 2.6. If item < a[loc] then7. last = loc – 1.// search first half8. Else if item > a[loc] then9. first = loc + 1. // search last half
10. Else found = true. // item found
• Each pass cuts the list in half• Worst case : item not in list TB(n) = O(log2n)
Binary search of a[0] … a[n-1]
Binary search of a[0] … a[n-1]
26
Common Computing Time Functions
log2log2n log2n n n log2n n2 n3 2n
--- 0 1 0 1 1 2
0.00 1 2 2 4 8 4
1.00 2 4 8 16 64 16
1.58 3 8 24 64 512 256
2.00 4 16 64 256 4096 65536
2.32 5 32 160 1024 32768 4294967296
2.58 6 64 384 4096 262144 1.84467E+19
3.00 8 256 2048 65536 16777216 1.15792E+77
3.32 10 1024 10240 1048576 1.07E+09 1.8E+308
4.32 20 1048576 20971520 1.1E+12 1.15E+18 6.7E+315652
For our binary searchFor our binary search
27
Computing in Real Time• Suppose each instruction can be done in 1
microsecond
• For n = 256 inputs how long for various f(n)Function Time
log2log2n 3 microseconds
Log2n 8 microseconds
n .25 milliseconds
n log2n 2 milliseconds
n2 65 milliseconds
n3 17 seconds
2n 3.7+E64 centuries!!
28
Conclusion
• Algorithms with exponential complexity– practical only for situations where number of
inputs is small
• Bubble sort has O(n2)– OK for n < 100– Totally impractical for large n
29
Computing Times Of Recursive Functions
// Towers of Hanoivoid Move(int n, char source, char destination, char
spare){if (n <= 1) // anchor (base) case cout << "Move the top disk from " << source << " to " << destination << endl; else { // inductive case Move(n-1, source, spare, destination); Move(1, source, destination, spare); Move(n-1, spare, destination, source); }}
T(n) = O(2n)T(n) = O(2n)