meljun cortes algorithm analysis of algorithm efficiency
TRANSCRIPT
-
8/21/2019 MELJUN CORTES ALGORITHM Analysis of Algorithm Efficiency
1/37
Analysis of Algorithm efficiency
Design and Analysis of Algorithm
* Property of STI
Page 1 of 37
Anal ysi s o f A lgo r i t hm
Eff ic iency
c Definition of Analysis of Algorithm
c Time and Space Efficiency
c Asymptotic Order of Growth
c Algorithm Case Efficiencies
c Asymptotic Notations
c Basic Efficiency Classes
c Mathematical Analysis of RecursiveAlgorithms
c Mathematical Analysis of Non-Recursive
Algorithms
c Empirical Analysis of Algorithm
-
8/21/2019 MELJUN CORTES ALGORITHM Analysis of Algorithm Efficiency
2/37
Analysis of Algorithm efficiency
Design and Analysis of Algorithm
* Property of STI
Page 2 of 37
An alysi s Fr am ew o r k
c Analysis of algorithm
q used in a narrower technical sense to mean an
investigation of algorithms efficiency withrespect to the two resources running time andmemory space
c main considerations:
q simplicity, generality and efficiency of algorithmq speed and memory of computer nowadays
-
8/21/2019 MELJUN CORTES ALGORITHM Analysis of Algorithm Efficiency
3/37
Analysis of Algorithm efficiency
Design and Analysis of Algorithm
* Property of STI
Page 3 of 37
An alysi s Fr am ew o r k
c Space efficiency
q amount of memory required by an algorithm torun the application
c Time efficiency
q indicates how fast an algorithm is processedin the machine
c Measuring an Input’s Sizeq examine an algorithm’s efficiency as a
function of some parametern indicating thealgorithm’s input size
c Techniques in Measuring Running Time
q Using standard unit of time measurement
q Counting the number of times eachalgorithm’s operations is executed
-
8/21/2019 MELJUN CORTES ALGORITHM Analysis of Algorithm Efficiency
4/37
Analysis of Algorithm efficiency
Design and Analysis of Algorithm
* Property of STI
Page 4 of 37
An alysi s Fr am ew o r k
cDisadvantages of using techniques in analysisof algorithm:
q Dependence of the computer’s speed
q Dependence on the quality of the programimplementing the computer
q Dependence on the compiler used ingenerating the machine code
q Difficulty of clocking the actual running time ofthe program
-
8/21/2019 MELJUN CORTES ALGORITHM Analysis of Algorithm Efficiency
5/37
Analysis of Algorithm efficiency
Design and Analysis of Algorithm
* Property of STI
Page 5 of 37
An alysi s Fr am ew o r k
cOrder of Growth
q how do time or space requirements grow as
the problem size increases
Approximate value of different important functionsfor algorithm analysis
-
8/21/2019 MELJUN CORTES ALGORITHM Analysis of Algorithm Efficiency
6/37
Analysis of Algorithm efficiency
Design and Analysis of Algorithm
* Property of STI
Page 6 of 37
An alysi s Fr am ew o r k
c Worst-case efficiency
c Best-case efficiency
c Average-case efficiency
c Amortized efficiency
-
8/21/2019 MELJUN CORTES ALGORITHM Analysis of Algorithm Efficiency
7/37
Analysis of Algorithm efficiency
Design and Analysis of Algorithm
* Property of STI
Page 7 of 37
Asym p t o t i c No tat i o n
c Asymptotic complexity
q way of expressing the main component of thecost of an algorithm using idealized units of
computational work
c Notation used in comparing and ranking theorder of growth
q
Big-0h Notation (0
)
q Big-0mega Notation (? )
q Big-Theta Notation (T)
-
8/21/2019 MELJUN CORTES ALGORITHM Analysis of Algorithm Efficiency
8/37
Analysis of Algorithm efficiency
Design and Analysis of Algorithm
* Property of STI
Page 8 of 37
Asym p t o t i c No tat i o n
c Big-0 Notation (0)
q introduced in 1854 by Paul Bachmann
q most commonly used notation for specifying
asymptotic complexity
c Big-O means “on the order of”
q formal method of expressing the upper boundof an algorithm's running time.
c Writef(n) = O(g(n)) if there exist constantsc > 0 , n0 > 0 such that0 = f(n) = cg(n) for alln = n 0
-
8/21/2019 MELJUN CORTES ALGORITHM Analysis of Algorithm Efficiency
9/37
Analysis of Algorithm efficiency
Design and Analysis of Algorithm
* Property of STI
Page 9 of 37
Asym p t o t i c No tat i o n
c Practical Example:
q O (n)
• printing a list of n items to the screen, lookingat each item once
q O (log n )
• taking a list of items, cutting it in half
repeatedly until there's only one item left
q O (n2 )
• taking a list of n items, and comparing everyitem to every other item
-
8/21/2019 MELJUN CORTES ALGORITHM Analysis of Algorithm Efficiency
10/37
Analysis of Algorithm efficiency
Design and Analysis of Algorithm
* Property of STI
Page 10 of 37
Asym p t o t i c No tat i o n
c Theory Example:
q Given:
f(n) = 2 n + 8 , and g(n) = n2
q Find c , so that 2n + 8
-
8/21/2019 MELJUN CORTES ALGORITHM Analysis of Algorithm Efficiency
11/37
Analysis of Algorithm efficiency
Design and Analysis of Algorithm
* Property of STI
Page 11 of 37
Asym p t o t i c No tat i o n
c The Big-O notation can be derived from f(n)using the following steps:
q set the coefficient of the term to 1
q keep the largest term in the function anddiscard the others
q terms are ranked from lowest to highest
log 2 n, n, nlog 2 n, n 2,n 3 . . . .n k 2 n n!
-
8/21/2019 MELJUN CORTES ALGORITHM Analysis of Algorithm Efficiency
12/37
Analysis of Algorithm efficiency
Design and Analysis of Algorithm
* Property of STI
Page 12 of 37
Asym p t o t i c No tat i o n
c Example#1:
q remove all coefficients
• n2 + n
q remove the smaller factors• n2
q Big-O notation:
• o(f(n) = o(n2 )
-
8/21/2019 MELJUN CORTES ALGORITHM Analysis of Algorithm Efficiency
13/37
Analysis of Algorithm efficiency
Design and Analysis of Algorithm
* Property of STI
Page 13 of 37
Asym p t o t i c No tat i o n
c Example#2:
f(n) = a j n k + a j-1 n
k-1 + …+a 2 n 2 +a 1 n+a 0
q remove all coefficients
• f(n) = n k +n k -1 + …+n 2 +n+1
q the order of a polynomial expression is:
• o(f(n) = o(n k )
-
8/21/2019 MELJUN CORTES ALGORITHM Analysis of Algorithm Efficiency
14/37
Analysis of Algorithm efficiency
Design and Analysis of Algorithm
* Property of STI
Page 14 of 37
Asym p t o t i c No tat i o n
c Big-0 Measures of Efficiency
-
8/21/2019 MELJUN CORTES ALGORITHM Analysis of Algorithm Efficiency
15/37
Analysis of Algorithm efficiency
Design and Analysis of Algorithm
* Property of STI
Page 15 of 37
Asym p t o t i c No tat i o n
c Big-0mega Notation (? )
q t(n) ? (g(n))
q Sample formal proof wheren 3 ? (n 2 ) • B 3 = n 2 for all n = 0
q selectc =1 and n 0 = 0
-
8/21/2019 MELJUN CORTES ALGORITHM Analysis of Algorithm Efficiency
16/37
Analysis of Algorithm efficiency
Design and Analysis of Algorithm
* Property of STI
Page 16 of 37
Asym p t o t i c No tat i o n
c Big-Theta Notation (T)
q applied in tight bound
q function t(n) is said to be in T(g(n)), denotedt(n) T(g(n))
• if t(n) is bounded both above and below by somepositive constant multiples of g(n) for all large n
• If there exist some positive constant c1 and c2 and
some nonnegative integer n0 such that:
c 2 g (n) = t (n) = c 1 g(n) for all n = n 0
-
8/21/2019 MELJUN CORTES ALGORITHM Analysis of Algorithm Efficiency
17/37
Analysis of Algorithm efficiency
Design and Analysis of Algorithm
* Property of STI
Page 17 of 37
Asym p t o t i c No tat i o n
c For example, let us prove that right inequality(the upper bound):
q First, we prove the right inequality (the upperbounds)
-
8/21/2019 MELJUN CORTES ALGORITHM Analysis of Algorithm Efficiency
18/37
Analysis of Algorithm efficiency
Design and Analysis of Algorithm
* Property of STI
Page 18 of 37
Asym p t o t i c No tat i o n
c Continuation:
q Second, we prove the left inequality (the lowerbounds).
q Hence:
-
8/21/2019 MELJUN CORTES ALGORITHM Analysis of Algorithm Efficiency
19/37
Analysis of Algorithm efficiency
Design and Analysis of Algorithm
* Property of STI
Page 19 of 37
Basi c Eff i c i en cy Cl asses
c Algorithm’s efficiency
q a function of the number of elements to beprocessed
q general format:
f (n) = efficiency
-
8/21/2019 MELJUN CORTES ALGORITHM Analysis of Algorithm Efficiency
20/37
Analysis of Algorithm efficiency
Design and Analysis of Algorithm
* Property of STI
Page 20 of 37
Basi c Eff i c i en cy Cl asses
c Linear Loops
q Example:• identify how many times the body of the loop is
repeated:
q the number of iterations is directlyproportional to the loop factor
q the higher the factor, the higher the number of
loopsq efficiency is directly proportional to the
number of iterations
1 x = 02 loop (x >= 1000)
1 application code2 x = x plus 1
3 end loop
f (n) = n
-
8/21/2019 MELJUN CORTES ALGORITHM Analysis of Algorithm Efficiency
21/37
Analysis of Algorithm efficiency
Design and Analysis of Algorithm
* Property of STI
Page 21 of 37
Basi c Eff i c i en cy Cl asses
c Linear Loops
q Example:
q the number of iterations is half the loop factor
q Efficiency is proportional to half the factor
1 x = 02 loop (x >= 1000)
1 application code2 x = x plus 2
3 end loop
f (n) = n/2
-
8/21/2019 MELJUN CORTES ALGORITHM Analysis of Algorithm Efficiency
22/37
Analysis of Algorithm efficiency
Design and Analysis of Algorithm
* Property of STI
Page 22 of 37
Basi c Eff i c i en cy Cl asses
c Logarithmic Loops
q a loop in which the controlling variable ismultiplied or divided in each loop
q determine how many times the body of theloops is repeated:
1 x = 1
2 loop (x >= 1000)1 application code2 x = x multiply 2
3 end loop
1 x = 10002 loop (x >= 1)
1 application code2 x = x divide 2
3 end loop
-
8/21/2019 MELJUN CORTES ALGORITHM Analysis of Algorithm Efficiency
23/37
Analysis of Algorithm efficiency
Design and Analysis of Algorithm
* Property of STI
Page 23 of 37
Basi c Eff i c i en cy Cl asses
c Logarithmic Loops
f (n) = [log2n]
Analysis of Multiplication and
Division Loops
-
8/21/2019 MELJUN CORTES ALGORITHM Analysis of Algorithm Efficiency
24/37
Analysis of Algorithm efficiency
Design and Analysis of Algorithm
* Property of STI
Page 24 of 37
Basi c Eff i c i en cy Cl asses
c Nested Loop
q loop within a loop
q includes Linear Logarithmic, Dependent
Quadratic, and Quadratic
Iterations = outer loop iterations x
inner loop iterations
-
8/21/2019 MELJUN CORTES ALGORITHM Analysis of Algorithm Efficiency
25/37
Analysis of Algorithm efficiency
Design and Analysis of Algorithm
* Property of STI
Page 25 of 37
Basi c Eff i c i en cy Cl asses
c Linear logarithmic:
c number of iteration in the inner loop is [log210]
1 x = 1
2 loop (x
-
8/21/2019 MELJUN CORTES ALGORITHM Analysis of Algorithm Efficiency
26/37
Analysis of Algorithm efficiency
Design and Analysis of Algorithm
* Property of STI
Page 26 of 37
Basi c Eff i c i en cy Cl asses
c Dependent quadratic :
c inner loop depends on the outer loop for one of itsfactors
c the number of iteration is 55
1 x = 10
2 loop (x
-
8/21/2019 MELJUN CORTES ALGORITHM Analysis of Algorithm Efficiency
27/37
Analysis of Algorithm efficiency
Design and Analysis of Algorithm
* Property of STI
Page 27 of 37
Basi c Eff i c i en cy Cl asses
c average is 5.5
-
8/21/2019 MELJUN CORTES ALGORITHM Analysis of Algorithm Efficiency
28/37
Analysis of Algorithm efficiency
Design and Analysis of Algorithm
* Property of STI
Page 28 of 37
Basi c Eff i c i en cy Cl asses
c Quadratic:
1 x = 10
2 loop ( x
-
8/21/2019 MELJUN CORTES ALGORITHM Analysis of Algorithm Efficiency
29/37
Analysis of Algorithm efficiency
Design and Analysis of Algorithm
* Property of STI
Page 29 of 37
SEATWORK
c Algorithm efficiency application:Statewhether the statement below isTrueorFalse.
Defend your answer.
1. 20n2+ 2n + 5 = O(n2)
2. 100n + 5 = O(n2)
-
8/21/2019 MELJUN CORTES ALGORITHM Analysis of Algorithm Efficiency
30/37
Analysis of Algorithm efficiency
Design and Analysis of Algorithm
* Property of STI
Page 30 of 37
M at h em at i cal An alysi s o f
Recu r si ve A lgo r i t hm s
c General plan of analyzing Time Efficiency ofrecursive algorithm
1. Decide on a parameter (or parameters)indicating an input’s size.
2. Identify the algorithm’s basic operation.
3. Check whether number of times the basicoperation is executed can vary on differentinputs of the same size
4. Set up recurrence relation, with appropriateinitial condition, for the number of times thebasic operation is executed.
5. Solve the recurrence or at least ascertain the
order of growth of its solution.
-
8/21/2019 MELJUN CORTES ALGORITHM Analysis of Algorithm Efficiency
31/37
Analysis of Algorithm efficiency
Design and Analysis of Algorithm
* Property of STI
Page 31 of 37
M at h em at i cal An alysi s o f
Recu r si ve A lgo r i t hm s
c Compute the factorial function f(n) = n! for anarbitrary nonnegative integer n. Since:
n! = 1 , ….(n-1) . n = (n-1)! . n for n =1
c With 0 != 1 by definition, we can compute F(n) =F(n-1) . n with the following recursive algorithm.
c F(n) = F(n-1) . n for everyn > 0
c M(n) = M (n-1 ) + 1 for everyn > 0
1 if n = 0 return 12 else
3 return F(n-1) * n
-
8/21/2019 MELJUN CORTES ALGORITHM Analysis of Algorithm Efficiency
32/37
Analysis of Algorithm efficiency
Design and Analysis of Algorithm
* Property of STI
Page 32 of 37
M ath em at i cal An alysi s o f
N o n -Recu r si ve A l go r i t h m s
c General Plan for Analyze Time Efficiency
1.
Decide on a parameter (or parameters)indicating an input’s size.
2. Identify the algorithms basic operation.
3. Check whether the number of times the basicoperation is executed depends only on thesize of an input.
4. Set up a sum expressing the number of timesthe algorithm’s basic operation is executed.
5. Using standard formulas and rules of summanipulation, either find a close-form formulafor the count, or the very least, establish itsorder of growth.
-
8/21/2019 MELJUN CORTES ALGORITHM Analysis of Algorithm Efficiency
33/37
Analysis of Algorithm efficiency
Design and Analysis of Algorithm
* Property of STI
Page 33 of 37
M ath em at i cal An alysi s o f
N o n -Recu r si ve A l go r i t h m s
c Example:
c find a formula in expressingC as a functionof size n
maxValue = A[0]loop index = n
if A[index ] > maxValue
return maxValue
-
8/21/2019 MELJUN CORTES ALGORITHM Analysis of Algorithm Efficiency
34/37
Analysis of Algorithm efficiency
Design and Analysis of Algorithm
* Property of STI
Page 34 of 37
Em p i r i cal An al ysi s o f
A l g o r i t hm c Empiricalanalysis
q alternative of the mathematical analysis ofalgorithms efficiency
c General Plan for Empirical Analysis ofAlgorithm Time Efficiency
1. Understand the experiments purpose.
2. Decide on efficiency metric M to be measure
and the measurement unit (an operation’scount vs. a time unit).
3. Decide on characteristics of the input sample(its range, size and so on).
4. Prepare a program implementing thealgorithm ( or algorithms) for the
experimentation.
5. Generate a sample of inputs.
6. Run algorithms on the sample inputs andrecord the data observed.
7. Analyzed the data obtained.
-
8/21/2019 MELJUN CORTES ALGORITHM Analysis of Algorithm Efficiency
35/37
Analysis of Algorithm efficiency
Design and Analysis of Algorithm
* Property of STI
Page 35 of 37
Em p i r i cal An al ysi s o f
A l g o r i t hm
c Limitations of Empirical Analysis
q Quality of implementation
q Extraneous factors
q Requires a working program
-
8/21/2019 MELJUN CORTES ALGORITHM Analysis of Algorithm Efficiency
36/37
Analysis of Algorithm efficiency
Design and Analysis of Algorithm
* Property of STI
Page 36 of 37
SEATWORK
c Calculate the run-time efficiency of thefollowing pseudocode for program
segments.(10 points each)
x = 0
loop( x
-
8/21/2019 MELJUN CORTES ALGORITHM Analysis of Algorithm Efficiency
37/37
Design and Analysis of Algorithm
ASSIGNM ENT
c Research for the following topics:
1. Definition of brute force algorithm techniquedesign technique
2. Applications of brute force algorithm designtechnique
c Note: The research work should be in aprinted copy and compiled in a folder with atitle page.