first questions for algorithm analysis
DESCRIPTION
First Questions for Algorithm Analysis. Space Input magnitude (the number of bits needed to represent the input Number of inputs Space required to store the inputs, perform intermediate computations, and store outputs Time. Measurement Units. Definition of g(n) ε O(f(n)). - PowerPoint PPT PresentationTRANSCRIPT
FIRST QUESTIONS FOR ALGORITHM ANALYSIS
WHAT IS AN ALGORITHM?
From the text (p. 3):“An algorithm is a sequence of unambiguous instructions for solving a problem, i.e., for obtaining a required output for any legitimate input in a finite amount of time.”
From the preceding line:“Although there is no universally agreed-on wording to describe this notion, there is general agreement about what the concept means:”
PROBLEMS WITH THE WORDING
What do you see as problematic???
Some might include:
• Unambiguous• Instructions• Solving• Problem• Finite time
Is this the only discipline with difficult primitives?
• Geometry: Point, line, and plane; axioms (Euclid, Reimann, others)
• Set Theory: Set
YET ANOTHER PERSPECTIVE
Set and its members
Set operations:
• union, • intersection, • complement, (problem of having
universe)• cross product of two sets
Relation
Function
• Special type of relation• Sometimes expressed by a “rule” of
relationship
Could algorithms simply be functions?
COMPARE ALGORITHMS AND FUNCTIONS
Sequentiality
Lack of ambiguity
Instructions
Solving
Problem
Input – Output relationship
Finiteness of time
Uniqueness of expression
Computation
SOME COMMON PROBLEMS
???
Sorting
• What does it mean formally?
Searching
String processing
• Matching• Translation
Graph
• TSP• Connectedness
Combinatorial
• Optimization: TSP, QAP• Constraint satisfaction: SAT, Queen Placement,
Knight’s Tour
Geometric
• Path planning
Numerical (CSC 340)
Etc.
DESCRIBING ALGORITHMIC BEHAVIOR
Strategic
• Greedy• Brute force
• Enumeration (exhaustive)
• Graph• DFS, BFS• Connectedness, planarity, etc.
• Divide and conquer• Dynamic programming• Backtracking
• Branch and bound
• Recursive• Domain transformation
Operational
• Deterministic versus randomized• Exact (find an optimum or find a compliant)• Approximate (estimate pi)• Heuristic (choose quickly, WTA)
Efficiency
• Space• Time
http://en.wikipedia.org/wiki/List_of_algorithms
MEASUREMENT UNITS
Space
• Input magnitude (the number of bits needed to represent the input
• Number of inputs• Space required to store the inputs, perform intermediate
computations, and store outputs
Time
BIG O(.)
Definition of g(n) ε O(f(n)).
The function g(n) is in O(f(n)) if there exist non negative constants c and such that
g(n) ≤ c f(n) for all n ≥ .
FIND ALGORITHMS THAT ARE O(F(N))
DO THINGS EVER GET WORSE?
Is the O() = Φ?
Are there decision questions that cannot be computed by any algorithm?
OTHER BOUNDS
Definition: g(n) ε Ω(f(n)). What does it mean to be bounded below?
• The function g(n) is in Ω(f(n)) if there exist constants c>0 and nonnegative such that
g(n) ≥ c f(n) for all n ≥ .
Definition: g(n) ε Θ(f(n)). What does it mean to be bounded above and below?
• The function g(n) is in Θ(f(n)) if there exist non negative constants c1, c2 and such that
c1 f(n) ≤g(n) ≤ c2 f(n) for all n ≥ .
EXAMPLE WITH F(N)=N*N
1 6 11 16 21 26 31 36 41 46 51 56 61 66 71 76 81 86 91 96 101 106 111 116 121 1260
5000
10000
15000
20000
25000
g(n)
c1*f(n)
c2*f(n)
ANALYZING NON-RECURSIVE ALGORITHMS
How should input size be measured?
• Number of items in a list• Value of a parameter or the number of bits needed to store
the parameter
What is the program’s most frequently occurring operation?
• Look inside loops
Are there other factors than input size that affect the frequency of the most frequent operation?
• Separate best, worst and average case analyses
How to tally the frequency?
• Summations• Profiling
EXAMPLE 1
Find max
Given a list containing n>0 integers (It is a separate question to ask the time for set-up. What would it be for this example?)
Pseudo-code
max = list[0]
for i = 0 to n-1
if list[i]>max
max = list[i]
Analysis?
EXAMPLE 1 ANALYSIS
max = list[0]
• O(1) = load + store
if…
• Evaluate condition (a difference)• Conditional transfer (a jump, branch or skip depending upon
flag values)• Assignment = load + store• O(??)
for…
• Iterates n times• Total accumulated time = ?
ANALYZING RECURSIVE ALGORITHMS
To understand recursion one must first understand recursion
Questions
• How will one measure input size?• What is the core operation?• Do other factors than input size influence the number of
repetitions?• What is a recurrence expressing the repetition of the core
operation?• Solve the recurrence?
EXAMPLE 2:A RECURSIVE ALGORITHM
import random
import math
c = [0, 0]
alist=[]
n = int(input('Number of items for list = ' ))
for i in range(n):
x = random.randint(1, 1000)
alist.append(x)
##print(len(alist))
##print('The initial list = ', alist)
def recursfindmax(alis):
print(alis)
if len(alis) == 1:
c[0]+=1
return alis[0]
else:
c[0]+=1
amax = max(alis[0], recursfindmax(alis[1:len(alis)]))
return(amax)
def findmax(alis):
amax = alis[0]
for i in range(len(alis)):
c[1]+=1
if amax < alis[i]:
amax = alis[i]
return (amax)
print('Recursive max = ', recursfindmax(alist))
print('Iterative max = ', findmax(alist))
print('Count recursive, iterative ', c)
ANALYSISt(n) = 1 + t(n-1)
t(n-1) = 1 + t(n-2)
t(n-2) = 1 + t(n-3)
t(n-3) = 1 + t(n-4)
…
t(n-(n-2)) = 1 + t(n-1)
t(n-n-1)) = t(1) = 1; this is the anchor
t(n) = 1 + t(n-1) = 2 + t(n-2) = 3 + t(n-3) = …= (n-1) + t(n-(n-1))= n
COMPARISON
Both methods are O(n), where n is the number of items in the list
How do the algorithms compare with respect to space utilization?
FUN FACT
Given a homogeneous, second order, linear recurrence with constant coefficients (a≠0) of the form
a y[n] + b y[n-1] + c y[n-2] = 0
The recurrence has a closed form solution depending upon the roots and of the characteristic equation
and the initial conditions for y[0] and y[1]
FUN FACT (CONT’D)If the roots are real and distinct
If there is a repeated root
If the roots are imaginary
, and
CONSIDER THE FIBONACCI NUMBER RECURRENCE
Fib(0) = 0
Fib(1) = 1
Fib(n) = Fib(n-1) + Fib(n-2)
Produces the homogeneous recurrence:
Fib(n) – Fib(n-1) – Fib(n-2) = 0
With characteristic equation:
Where a = 1, b = -1 and c = -1
SOLVING THE EQUATION
Thus,
The roots are real and distinct so that the recurrence is expressed by
APPLYING INITIAL CONDITIONS
Recall that Fib(0) = 0, Fib(1) = 1, and
Applying the first condition yields:
Applying the second condition yields:
Since , the second becomes:
Thus, and so that
NOTES
Given
Recall and note that,
Thus, and
COUNTING ADDITIONS IN RECURSIVE FIBONACCI
A(n) = 1 (addition)+1 A(n-1)+ 1A(n-2)=1 + 1 +A(n-2)+A(n-3)+ A(n-2)= 2 + 2 A(n-2) + 1 A(n-3)= 2 + 2(1+A(n-3)+A(n-4))+A(n-3)= 4 + 3 A(n-3) + 2 A(n-4)= 4 + 3(1+A(n-4)+A(n-5))+2A(n-4)= 7 + 5 A(n-4)+3 A(n-5)= 7 + 5(1 + A((n-5)+A(n-6))+ 3 A(n-5)= 12 + 8 A(n-5) + 5 A(n-6)= …= (Fib(i+2) -1)+Fib(i+1)A(n-i)+Fib(i)A(n-(i+1))
Letting i = n-1, yields A(n) = Fib(n+1)-1,since A(0)=A(1)=0
EMPIRICAL VERFICIATIONc=[0,0,0,0,0]
def fib(n):
c[0]+=1
if n>1:
c[1]+=1
return fib(n-1)+fib(n-2)
elif n==0:
c[2]+=1
return 0
else:
c[3]+=1
return 1
c[4]+=1
print(fib(int(input("Supply a number: "))))
print(c)