a l i f al ith ianalysis of algorithms iyxie/courses/cs2210b-2011/htmls/notes/02-analy… ·...
TRANSCRIPT
![Page 1: A l i f Al ith IAnalysis of Algorithms Iyxie/courses/cs2210b-2011/htmls/notes/02-analy… · Characters of Running Time Most algorithms transform input objects into output objects](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce80207d266a24582f9a9c/html5/thumbnails/1.jpg)
A l i f Al ith IAnalysis of Algorithms I
Outline• running timeg
• pseudocode
• primitive operationsp p
• asymptotic notation
M t di li hMore case studies: linear search, insertion-sort,matrix multiplication
© 2010 Goodrich, Tamassia 1
p
Instructor: Yuzhen Xie
![Page 2: A l i f Al ith IAnalysis of Algorithms Iyxie/courses/cs2210b-2011/htmls/notes/02-analy… · Characters of Running Time Most algorithms transform input objects into output objects](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce80207d266a24582f9a9c/html5/thumbnails/2.jpg)
MotivationMotivation
Very similar problem can have very different solutions. How to compare between different algorithms in order to select the best one for a particular application?
CorrectnesslSimplicity
SecurityMaintainability…Response timeSpace usage
In this course, we focus on the running time, with space usagebeing also of interest in some occasions.
2
![Page 3: A l i f Al ith IAnalysis of Algorithms Iyxie/courses/cs2210b-2011/htmls/notes/02-analy… · Characters of Running Time Most algorithms transform input objects into output objects](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce80207d266a24582f9a9c/html5/thumbnails/3.jpg)
Characters of Running TimeMost algorithms transform input objects into output objects. best case
average caseRunning time typically grows with input size, n
Fi d i ti till 100
120
average caseworst case
Fixed n, running time may still varies on different inputs: best case, average case, worst caseE li h
60
80
100
ing
Tim
e
Ex. linear searchinsertion-sort
We focus on the worst case20
40
Run
n
We focus on the worst case running time.
A guarantee to the userCrucial to applications such as
01000 2000 3000 4000
Input Size
3
Crucial to applications such as games, finance and robotics
![Page 4: A l i f Al ith IAnalysis of Algorithms Iyxie/courses/cs2210b-2011/htmls/notes/02-analy… · Characters of Running Time Most algorithms transform input objects into output objects](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce80207d266a24582f9a9c/html5/thumbnails/4.jpg)
Linear Search
Algorithm LinearSearch(v, S[0, …, n-1]) Ex. Input: array S of n elements, value vOutput: position i if S[i] = v; -1 if v ∉S
[3, 6, 1, 5, 8]
Running timei ← 0while (i < n) and (S[i] != v) do
i = i +1
Running time(fixed n)
Best case?if i < n
return ielse
Average case?Worst case?
elsereturn -1
4
![Page 5: A l i f Al ith IAnalysis of Algorithms Iyxie/courses/cs2210b-2011/htmls/notes/02-analy… · Characters of Running Time Most algorithms transform input objects into output objects](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce80207d266a24582f9a9c/html5/thumbnails/5.jpg)
Experimental Evaluation of Running TimeExperimental Evaluation of Running Time
Write a program 9000Write a program implementing the algorithm
h h7000
8000
9000
Run the program with inputs of varying size and composition
4000
5000
6000
me
(ms)
Use a method like System.currentTimeMillis() to get an accurate measure 2000
3000
4000
Tim
gof the actual running time
Plot the results 0
1000
0 50 100
5
Input Size
![Page 6: A l i f Al ith IAnalysis of Algorithms Iyxie/courses/cs2210b-2011/htmls/notes/02-analy… · Characters of Running Time Most algorithms transform input objects into output objects](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce80207d266a24582f9a9c/html5/thumbnails/6.jpg)
Limitations of ExperimentsLimitations of Experiments
It is necessary to implement the algorithm whichIt is necessary to implement the algorithm, which may be difficult
Results may not be indicative of the running time onResults may not be indicative of the running time on other inputs not included in the experiment.
In order to compare two algorithms, the sameIn order to compare two algorithms, the same hardware and software environments must be used
6
![Page 7: A l i f Al ith IAnalysis of Algorithms Iyxie/courses/cs2210b-2011/htmls/notes/02-analy… · Characters of Running Time Most algorithms transform input objects into output objects](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce80207d266a24582f9a9c/html5/thumbnails/7.jpg)
Theoretical Analysis of Running TimeTheoretical Analysis of Running Time
Uses a high level description of the algorithmUses a high-level description of the algorithm instead of an implementation
Characterizes running time as a function of theCharacterizes running time as a function of the input size, n.
Takes into account all possible inputsTakes into account all possible inputs
Allows us to evaluate the speed of an algorithm independent of the ha d a e/soft a e en i onmentindependent of the hardware/software environment
7
![Page 8: A l i f Al ith IAnalysis of Algorithms Iyxie/courses/cs2210b-2011/htmls/notes/02-analy… · Characters of Running Time Most algorithms transform input objects into output objects](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce80207d266a24582f9a9c/html5/thumbnails/8.jpg)
PseudocodePseudocode
High-level description of Example: find max an algorithm
More structured than English prose Algorithm arrayMax(A, n)
pelement of an array
English prose
Less detailed than a program
Input: array A of n integersOutput: maximum element of A
Preferred notation for describing algorithms
currentMax ← A[0]for i ← 1 to n − 1 do
if A[i] > currentMax thenHides program design issues
[ ]currentMax ← A[i]
return currentMax
8
![Page 9: A l i f Al ith IAnalysis of Algorithms Iyxie/courses/cs2210b-2011/htmls/notes/02-analy… · Characters of Running Time Most algorithms transform input objects into output objects](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce80207d266a24582f9a9c/html5/thumbnails/9.jpg)
Pseudocode DetailsPseudocode Details
Control flow Method callif … then … [else …]while … do …repeat until
var.method (arg [, arg…])
Return valuerepeat … until …for … do …Indentation replaces braces
return expression
Expressions← Assignment
Method declarationAlgorithm method (arg [, arg…])
← Assignment(like = in Java)
= Equality testing(like == in Java)g ( g [, g ])
Input: …Output: …
( )n2 Superscripts and other
mathematical formatting allowed
9
![Page 10: A l i f Al ith IAnalysis of Algorithms Iyxie/courses/cs2210b-2011/htmls/notes/02-analy… · Characters of Running Time Most algorithms transform input objects into output objects](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce80207d266a24582f9a9c/html5/thumbnails/10.jpg)
The Random Access Machine (RAM) ModelThe Random Access Machine (RAM) Model
RAM model is independent
A CPU
of hardware architectures.
+ - * / < > =?
A potentially unbounded bank 2A potentially unbounded bank of memory cells, each of which can hold an arbitrary number or character
012
number or character
Memory cells are numbered and i ll i t k
10
accessing any cell in memory takes unit time.
![Page 11: A l i f Al ith IAnalysis of Algorithms Iyxie/courses/cs2210b-2011/htmls/notes/02-analy… · Characters of Running Time Most algorithms transform input objects into output objects](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce80207d266a24582f9a9c/html5/thumbnails/11.jpg)
Primitive OperationspBasic computations performed by an algorithm
Examples:by an algorithm
Identifiable in pseudocodeEvaluating an expression, e.g, + - * /, <, >, == Assigning a value to a
Largely independent from the programming language
E t d fi iti t i t t
g gvariableIndexing into an arrayCalling a methodExact definition not important
(we will see why later)
Assumed to take a constant
Calling a methodReturning from a method
Assumed to take a constant amount of time in the RAM model
11
![Page 12: A l i f Al ith IAnalysis of Algorithms Iyxie/courses/cs2210b-2011/htmls/notes/02-analy… · Characters of Running Time Most algorithms transform input objects into output objects](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce80207d266a24582f9a9c/html5/thumbnails/12.jpg)
Counting Primitive OperationsCounting Primitive OperationsBy inspecting the pseudocode, we can determine the y p g p ,maximum number of primitive operations executed by an algorithm, as a function of the input size.
Algorithm arrayMax(A, n) # operationscurrentMax ← A[0] 2for i ← 1 to n − 1 do n - 1
if A[i] > currentMax then 2(n − 1)currentMax ← A[i] 2(n − 1)currentMax ← A[i] 2(n 1)
return currentMax 1Total 5n − 2
12
![Page 13: A l i f Al ith IAnalysis of Algorithms Iyxie/courses/cs2210b-2011/htmls/notes/02-analy… · Characters of Running Time Most algorithms transform input objects into output objects](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce80207d266a24582f9a9c/html5/thumbnails/13.jpg)
Estimating Running TimeEstimating Running Time
l hAlgorithm arrayMax executes 5n − 2 primitive operations in the worst case. Define:
Time taken by the fastest primitive operationa = Time taken by the fastest primitive operationb = Time taken by the slowest primitive operation
Let T(n) be worst-case time of arrayMax. Thena (5n − 2) ≤ T(n) ≤ b(5n − 2)
Hence, the running time T(n) is bounded by two linear functions
13
![Page 14: A l i f Al ith IAnalysis of Algorithms Iyxie/courses/cs2210b-2011/htmls/notes/02-analy… · Characters of Running Time Most algorithms transform input objects into output objects](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce80207d266a24582f9a9c/html5/thumbnails/14.jpg)
Growth Rate of Running TimeGrowth Rate of Running Time
Changing the hardware/ softwareChanging the hardware/ software environment
Affects T(n) by a constant factor, butAffects T(n) by a constant factor, butDoes not alter the growth rate of T(n)
The linea g o th ate of the nning timeThe linear growth rate of the running time T(n) of arrayMax is an intrinsic property of the algorithmthe algorithm
14
![Page 15: A l i f Al ith IAnalysis of Algorithms Iyxie/courses/cs2210b-2011/htmls/notes/02-analy… · Characters of Running Time Most algorithms transform input objects into output objects](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce80207d266a24582f9a9c/html5/thumbnails/15.jpg)
Seven Important FunctionsSeven Important Functions
Seven functions that ft i l ith 1E+30often appear in algorithm
analysis:Constant ≈ 1L ith i l 1E+22
1E+241E+261E+281E+30
Cubic
Quadratic
LiLogarithmic ≈ log nLinear ≈ nN-Log-N ≈ n log nQ d ti 2
1E+141E+161E+181E+201E 22
T(n
)
Linear
Quadratic ≈ n2
Cubic ≈ n3
Exponential ≈ 2n
1E+41E+61E+8
1E+101E+12
In a log-log chart, the slope of the line corresponds to the
1E+01E+21E+4
1E+0 1E+2 1E+4 1E+6 1E+8 1E+10n
15
corresponds to the growth rate
n
![Page 16: A l i f Al ith IAnalysis of Algorithms Iyxie/courses/cs2210b-2011/htmls/notes/02-analy… · Characters of Running Time Most algorithms transform input objects into output objects](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce80207d266a24582f9a9c/html5/thumbnails/16.jpg)
Growth Rates of some Important Functionsp
n lg n n n lg n n2 n3 2n
8 3 8 24 64 512 256
16 4 16 64 256 4 096 65 53616 4 16 64 256 4,096 65,536
32 5 32 160 1,024 32,768 4,294,967,296
64 6 64 384 4,096 262,144 1.84 x 1019
128 7 128 896 16,384 2,097,152 3.40 x 1038, , ,
256 8 256 2,048 65,536 16,777,216 1.15 x 1077
16
512 9 512 4,608 262,144 134,217,728 1.34 x 10154
![Page 17: A l i f Al ith IAnalysis of Algorithms Iyxie/courses/cs2210b-2011/htmls/notes/02-analy… · Characters of Running Time Most algorithms transform input objects into output objects](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce80207d266a24582f9a9c/html5/thumbnails/17.jpg)
Growth Rates of some Important Functionsp
17
![Page 18: A l i f Al ith IAnalysis of Algorithms Iyxie/courses/cs2210b-2011/htmls/notes/02-analy… · Characters of Running Time Most algorithms transform input objects into output objects](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce80207d266a24582f9a9c/html5/thumbnails/18.jpg)
Why Growth Rate Mattersy
if runtime time for n+1 time for 2n time for 4nis... time for n+1 time for 2n time for 4n
c lg n c lg (n+ 1) c (lg n + 1) c(lg n + 2)
c n c (n + 1) 2c n 4c n
~ c n lg n 2c n lg n + 4c n lg n +runtime
d lc n lg n c n lg n+ c n
2c n lg n + 2cn
4c n lg n + 4cn
c n2 ~ c n2 + 2c n 4c n2 16c n2
quadrupleswhen problemsize
c n3 ~ c n3 + 3c n2 8c n3 64c n3
size doubles
18
c 2n c 2 n+1 c 2 2n c 2 4n
![Page 19: A l i f Al ith IAnalysis of Algorithms Iyxie/courses/cs2210b-2011/htmls/notes/02-analy… · Characters of Running Time Most algorithms transform input objects into output objects](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce80207d266a24582f9a9c/html5/thumbnails/19.jpg)
Growth Rates of some Important Functionsp
n lg n n n lg n n2 n3 2n
8 3 8 24 64 512 256
16 4 16 64 256 4 096 65 53616 4 16 64 256 4,096 65,536
32 5 32 160 1,024 32,768 4,294,967,296
64 6 64 384 4,096 262,144 1.84 x 1019
128 7 128 896 16,384 2,097,152 3.40 x 1038, , ,
256 8 256 2,048 65,536 16,777,216 1.15 x 1077
19
512 9 512 4,608 262,144 134,217,728 1.34 x 10154
![Page 20: A l i f Al ith IAnalysis of Algorithms Iyxie/courses/cs2210b-2011/htmls/notes/02-analy… · Characters of Running Time Most algorithms transform input objects into output objects](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce80207d266a24582f9a9c/html5/thumbnails/20.jpg)
Constant FactorsConstant Factors
Th h i1E+26
Q d tiThe growth rate is not affected by
constant factors or 1E+181E+201E+221E+24 Quadratic
Quadratic
LinearLinear
105n2 + 108n
2lower-order terms
Examples 1E+101E+121E+141E+16
T(n
)102n + 105
n2
Examples102n + 105 is a linearfunction105 2 108 i 1E+2
1E+41E+61E+8
1E+10
n105n2 + 108n is a quadratic function 1E+0
1E+2
1E+0 1E+2 1E+4 1E+6 1E+8 1E+10n
20
n
![Page 21: A l i f Al ith IAnalysis of Algorithms Iyxie/courses/cs2210b-2011/htmls/notes/02-analy… · Characters of Running Time Most algorithms transform input objects into output objects](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce80207d266a24582f9a9c/html5/thumbnails/21.jpg)
The “Big Idea”: Big-Oh NotationGiven functions f(n) and g(n), we say that f(n) is O(g(n)) if there are positive constants c and n0 such that f(n) ≤ cg(n) for n ≥ n0
21
![Page 22: A l i f Al ith IAnalysis of Algorithms Iyxie/courses/cs2210b-2011/htmls/notes/02-analy… · Characters of Running Time Most algorithms transform input objects into output objects](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce80207d266a24582f9a9c/html5/thumbnails/22.jpg)
Big-Oh Notationg
Example: 10,0003n
2n+10 is O(n)2n + 10 ≤ cn(c − 2) n ≥ 10
1,000
3n
2n+10
(c 2) n ≥ 10n ≥ 10/(c − 2)Pick c = 3 and
10
100
n
n0 = 10
How about 10
(c = 4, n0 = 5) ?(c = 12, n0 = 1)?
11 10 100 1,000
n
22
n
![Page 23: A l i f Al ith IAnalysis of Algorithms Iyxie/courses/cs2210b-2011/htmls/notes/02-analy… · Characters of Running Time Most algorithms transform input objects into output objects](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce80207d266a24582f9a9c/html5/thumbnails/23.jpg)
Big-Oh Exampleg p
Example: 1,000,000^2Example:
n2 is not O(n)n2 ≤ cn ?
100,000
n^2
100n
10n
n ≤ c ?The above inequality cannot
1,000
10,000 n
inequality cannot be satisfied since c must be a constant
100
constant
A contradiction! 1
10
23
A contradiction!1 10 100 1,000
n
![Page 24: A l i f Al ith IAnalysis of Algorithms Iyxie/courses/cs2210b-2011/htmls/notes/02-analy… · Characters of Running Time Most algorithms transform input objects into output objects](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce80207d266a24582f9a9c/html5/thumbnails/24.jpg)
More Big-Oh ExamplesMore Big Oh Examples
5n-25n-2 is O(n)need c > 0 and n0 ≥ 1 such that 5n-2 ≤ c•n for n ≥ n0
this is true for c = 5 and n0 = 1this is true for c 5 and n0 1
3n3 + 20n2 + 53n3 + 20n2 + 5 is O(n3)( )need c > 0 and n0 ≥ 1 such that 3n3 + 20n2 + 5 ≤ c•n3 for n ≥ n0
this is true for c = 4 and n0 = 21. (how about c = 28, n0 = 1?)
3 l 53 log n + 53 log n + 5 is O(log n)need c > 0 and n0 ≥ 1 such that 3 log n + 5 ≤ c•log n for n ≥ n0
24
need c > 0 and n0 ≥ 1 such that 3 log n + 5 ≤ c•log n for n ≥ n0
this is true for c = 8 and n0 = 2
![Page 25: A l i f Al ith IAnalysis of Algorithms Iyxie/courses/cs2210b-2011/htmls/notes/02-analy… · Characters of Running Time Most algorithms transform input objects into output objects](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce80207d266a24582f9a9c/html5/thumbnails/25.jpg)
Big-Oh and Growth Rateg
The big-Oh notation gives an upper bound on the th t f f tigrowth rate of a function
The statement “f(n) is O(g(n))” means that the growth ate of f( ) is no mo e than the g o th ate of ( )rate of f(n) is no more than the growth rate of g(n)
We can use the big-Oh notation to rank functions according to their growth rateaccording to their growth rate
f(n) is O(g(n)) g(n) is O(f(n))f( ) (g( )) g( ) (f( ))
g(n) grows more Yes Nof(n) grows more No Yes
25
f( ) gSame growth Yes Yes
![Page 26: A l i f Al ith IAnalysis of Algorithms Iyxie/courses/cs2210b-2011/htmls/notes/02-analy… · Characters of Running Time Most algorithms transform input objects into output objects](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce80207d266a24582f9a9c/html5/thumbnails/26.jpg)
Big-Oh RulesBig Oh Rules
If f(n) is a polynomial of degree dIf f(n) is a polynomial of degree d, then f(n) is O(nd), i.e.,
1. Drop lower-order termsp2. Drop constant factors
Use the smallest possible class of functionsUse the smallest possible class of functionsSay “2n is O(n)” instead of “2n is O(n2)”
Use the simplest expression of the classUse the simplest expression of the classSay “3n + 5 is O(n)” instead of “3n + 5 is O(3n)”
“b ” i Oh h i i
26
“best” Big-Oh characterization
![Page 27: A l i f Al ith IAnalysis of Algorithms Iyxie/courses/cs2210b-2011/htmls/notes/02-analy… · Characters of Running Time Most algorithms transform input objects into output objects](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce80207d266a24582f9a9c/html5/thumbnails/27.jpg)
More Big-Oh RulesMore Big Oh Rules
If d(n) is O(f(n)) and e(n) is O(g(n)) thenIf d(n) is O(f(n)) and e(n) is O(g(n)) thend(n) + e(n) is O(f(n) + g(n)) d(n) e(n) is O(f(n) g(n))( ) ( ) (f( ) g( ))
If d(n) is O(f(n)) and f(n) is O(g(n)) then d(n) is O(g(n))
If p(n) is a polynomial in n then log p(n) is O(log(n))
27
![Page 28: A l i f Al ith IAnalysis of Algorithms Iyxie/courses/cs2210b-2011/htmls/notes/02-analy… · Characters of Running Time Most algorithms transform input objects into output objects](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce80207d266a24582f9a9c/html5/thumbnails/28.jpg)
Asymptotic Algorithm Analysisy p g y
The asymptotic analysis of an algorithm determines the running time in big Oh notationdetermines the running time in big-Oh notation
To perform the asymptotic analysisWe find the worst case number of primitive operationsWe find the worst-case number of primitive operations executed as a function of the input sizeWe express this function with big-Oh notation
Example:Example:We determine that algorithm arrayMax executes at most 5n − 2 primitive operationsWe say that algorithm arrayMax “runs in O(n) time”We say that algorithm arrayMax runs in O(n) time
Since constant factors and lower-order terms are eventually dropped anyhow we can disregard them
28
eventually dropped anyhow, we can disregard them when counting primitive operations
![Page 29: A l i f Al ith IAnalysis of Algorithms Iyxie/courses/cs2210b-2011/htmls/notes/02-analy… · Characters of Running Time Most algorithms transform input objects into output objects](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce80207d266a24582f9a9c/html5/thumbnails/29.jpg)
Computing Prefix AveragesComputing Prefix Averages
We further illustrate asymptotic analysis with two algorithms for prefix averages 25
30
35XA
g
The i-th prefix average of an array X is the average of th fi t (i 1) l t f X 15
20
25
the first (i + 1) elements of X:A[i] = (X[0] + X[1] + … + X[i])/(i+1)
5
10
15
Computing the array A of prefix averages of another array X has applications to
01 2 3 4 5 6 7
29
y ppfinancial analysis
![Page 30: A l i f Al ith IAnalysis of Algorithms Iyxie/courses/cs2210b-2011/htmls/notes/02-analy… · Characters of Running Time Most algorithms transform input objects into output objects](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce80207d266a24582f9a9c/html5/thumbnails/30.jpg)
Prefix Averages (Quadratic)Prefix Averages (Quadratic)
The following algorithm computes prefix averages in g g p p gquadratic time by applying the definition
Algorithm prefixAverages1(X, n)g p f g ( , )Input array X of n integersOutput array A of prefix averages of X #operationsA ← new array of n integers nA ← new array of n integers nfor i ← 0 to n − 1 do n
s ← X[0] nfor j ← 1 to i do 1 + 2 + …+ (n − 1)
s ← s + X[j] 1 + 2 + …+ (n − 1)A[i] ← s / (i + 1) n
30
[ ] ( )return A 1
![Page 31: A l i f Al ith IAnalysis of Algorithms Iyxie/courses/cs2210b-2011/htmls/notes/02-analy… · Characters of Running Time Most algorithms transform input objects into output objects](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce80207d266a24582f9a9c/html5/thumbnails/31.jpg)
Arithmetic ProgressionArithmetic Progression
The running time of 7The running time of prefixAverages1 isO(1 + 2 + …+ n)
5
6
7
The sum of the first nintegers is n(n + 1) / 2 3
4
5
There is a simple visual proof of this fact
Th l i h 1
2
3
Thus, algorithm prefixAverages1 runs in O(n2) time
0
1
1 2 3 4 5 6
31
( )
![Page 32: A l i f Al ith IAnalysis of Algorithms Iyxie/courses/cs2210b-2011/htmls/notes/02-analy… · Characters of Running Time Most algorithms transform input objects into output objects](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce80207d266a24582f9a9c/html5/thumbnails/32.jpg)
Prefix Averages (Linear)g ( )The following algorithm computes prefix averages in linear time by keeping a running sumy p g g
Algorithm prefixAverages2(X, n)Input: array X of n integersInput: array X of n integersOutput: array A of prefix averages of X #operationsA ← new array of n integers n
0 1s ← 0 1for i ← 0 to n − 1 do n
s ← s + X[i] nA[i] ← s / (i + 1) n
return A 1
32
Algorithm prefixAverages2 runs in O(n) time
![Page 33: A l i f Al ith IAnalysis of Algorithms Iyxie/courses/cs2210b-2011/htmls/notes/02-analy… · Characters of Running Time Most algorithms transform input objects into output objects](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce80207d266a24582f9a9c/html5/thumbnails/33.jpg)
Relatives of Big-OhRelatives of Big Oh
big Omegabig-Omegaf(n) is Ω(g(n)) if there is a constant c > 0 and an integer constant n0 ≥ 1 such that g 0
f(n) ≥ c•g(n) for n ≥ n0
big-Thetabig-Thetaf(n) is Θ(g(n)) if there are constants c’ > 0 and c’’ > 0 and an integer constant n0 ≥ 1 such that ’ ( ) ≤ f( ) ≤ ’’ ( ) f ≥c’•g(n) ≤ f(n) ≤ c’’•g(n) for n ≥ n0
33
![Page 34: A l i f Al ith IAnalysis of Algorithms Iyxie/courses/cs2210b-2011/htmls/notes/02-analy… · Characters of Running Time Most algorithms transform input objects into output objects](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce80207d266a24582f9a9c/html5/thumbnails/34.jpg)
Intuition for Asymptotic NotationIntuition for Asymptotic Notation
Big-OhBig Ohf(n) is O(g(n)) if f(n) is asymptotically less than or equal to g(n)
big-Omegaf( ) i Ω( ( )) if f( ) i t ti llf(n) is Ω(g(n)) if f(n) is asymptotically greater than or equal to g(n)
big-Thetaf(n) is Θ(g(n)) if f(n) is asymptotically
l t ( )
34
equal to g(n)
![Page 35: A l i f Al ith IAnalysis of Algorithms Iyxie/courses/cs2210b-2011/htmls/notes/02-analy… · Characters of Running Time Most algorithms transform input objects into output objects](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce80207d266a24582f9a9c/html5/thumbnails/35.jpg)
Example Uses of the Relatives of Big-OhExample Uses of the Relatives of Big Oh
5n2 is Ω(n2)f(n) is Ω(g(n)) if there is a constant c > 0 and an integer constant n0 ≥ 1
such that f(n) ≥ c•g(n) for n ≥ n0.Let c = 5 and n0 = 1
f(n) is Ω(g(n)) if there is a constant c > 0 and an integer constant n0 ≥ 1 such that f(n) ≥ c•g(n) for n ≥ n0
5n2 is Ω(n)
5n2 is Θ(n2)
such that f(n) ≥ c g(n) for n ≥ n0.Let c = 1 and n0 = 1
f(n) is Θ(g(n)) if it is Ω(n2) and O(n2). We have already seen the former, for the latter recall that f(n) is O(g(n)) if there is a constant c > 0 and an integer constant n0 ≥ 1 such that f(n) < c•g(n) for n ≥ n0 .
35
Let c = 5 and n0 = 1
![Page 36: A l i f Al ith IAnalysis of Algorithms Iyxie/courses/cs2210b-2011/htmls/notes/02-analy… · Characters of Running Time Most algorithms transform input objects into output objects](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce80207d266a24582f9a9c/html5/thumbnails/36.jpg)
Cautions about Big-Oh
The constant factors do
g
The constant factors do matter when they are large.Ex.T ( ) 30000
Running time
TA(n) = 30000nTB(n) = 3n2
A t ti ll T i li
A
Asymptotically, TA is linear and TB is quadratic.
In practice if n is no more
B
In practice, if n is no more than 10000, algorithm B should be used.
10000n
36
![Page 37: A l i f Al ith IAnalysis of Algorithms Iyxie/courses/cs2210b-2011/htmls/notes/02-analy… · Characters of Running Time Most algorithms transform input objects into output objects](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce80207d266a24582f9a9c/html5/thumbnails/37.jpg)
Comparison of two Sorting Algorithms insertion sort is n2 / 4merge sort is 2 n lg n
sort a million items?insertion sort takes
hl 70 hroughly 70 hourswhile
merge sort takesroughly 40 secondsroughly 40 seconds
This is a slow machine, but if100x faster then it’s 40 minutesversus less than 0 5 seconds
by Matt Stallmann
versus less than 0.5 seconds.
sort 50 items?
37
by Matt Stallmann
![Page 38: A l i f Al ith IAnalysis of Algorithms Iyxie/courses/cs2210b-2011/htmls/notes/02-analy… · Characters of Running Time Most algorithms transform input objects into output objects](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce80207d266a24582f9a9c/html5/thumbnails/38.jpg)
Math You need to Review
Summations
Math You need to Review
properties of logarithms:Logarithmslogb(xy) = logbx + logbylogb (x/y) = logbx - logbyLogbxa = a logbxgb gb
logba = logxa/logxb
Exponents properties of exponentials:pa(b+c) = aba c
abc = (ab)c
ab /ac = a(b-c)
b l b
38
b = a logab
bc = a c*logab
![Page 39: A l i f Al ith IAnalysis of Algorithms Iyxie/courses/cs2210b-2011/htmls/notes/02-analy… · Characters of Running Time Most algorithms transform input objects into output objects](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce80207d266a24582f9a9c/html5/thumbnails/39.jpg)
E iExercises: Analysis of linear search, insertion-sort and classic matrix multiplication
39
![Page 40: A l i f Al ith IAnalysis of Algorithms Iyxie/courses/cs2210b-2011/htmls/notes/02-analy… · Characters of Running Time Most algorithms transform input objects into output objects](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce80207d266a24582f9a9c/html5/thumbnails/40.jpg)
Insertion-sort
Algorithm InsertionSort(A[1, …, n]) Ex. Input: array A of n elementsOutput: A, s.t. A[1] ≤ A[2] … ≤ A[n]
[8, 2, 4, 9, 3, 6]
Running timefor j ← 2 to n do
key ← A[j]i ← j - 1
(fixed n)Best case?Average case?j
while (i > 0) and (A[i] > key) doA[i+1] ← A[i]i ← i - 1
Average case?Worst case?
i ← i - 1A[i+1] ← key
40