lecture 03 growth functions · 2013. 2. 14. ·...
TRANSCRIPT
Advanced Algorithmics (6EAP)
MTAT.03.238 Order of growth… maths
Jaak Vilo 2013 Spring
1 Jaak Vilo
Program execuGon on input of size n
• How many steps/cycles a processor would need to do
• Faster computer, larger input?
• But bad algorithm on fast computer will be outcompeted by good algorithm on slow…
• How to relate algorithm execuGon to nr of steps on input of size n? f(n)
• e.g. f(n) = n + n*(n-‐1)/2 + 17 n + n*log(n)
Big-‐Oh notaGon classes
Class Informal Intui5on Analogy
f(n) ∈ ο ( g(n) ) f is dominated by g Strictly below < f(n) ∈ O( g(n) ) Bounded from above Upper bound ≤ f(n) ∈ Θ( g(n) ) Bounded from
above and below “equal to” =
f(n) ∈ Ω( g(n) ) Bounded from below Lower bound ≥ f(n) ∈ ω( g(n) )
f dominates g Strictly above >
MathemaGcal Background JusGficaGon
• As engineers, you will not be paid to say: Method A is be#er than Method B
or Algorithm A is faster than Algorithm B
• Such descripGons are said to be qualita-ve in nature; from the OED:
qualita5ve, a. a RelaGng to, connected or concerned with, quality or qualiGes. Now usually in implied or expressed opposiGon to quanGtaGve.
MathemaGcal Background JusGficaGon
• Business decisions cannot be based on qualitaGve statements: – Algorithm A could be be#er than Algorithm B, but Algorithm A would require three person weeks to implement, test, and integrate while Algorithm B has already been implemented and has been used for the past year
– there are circumstances where it may beneficial to use Algorithm A, but not based on the word be#er
MathemaGcal Background JusGficaGon
• Thus, we will look at a quan-ta-ve means of describing data structures and algorithms
• From the OED:
quan5ta5ve, a. RelaGng to or concerned with quanGty or its measurement; that assesses or expresses quanGty. In later use freq. contrasted with qualita-ve.
Program Gme and space complexity
• Time: count nr of elementary calculaGons/operaGons during program execuGon
• Space: count amount of memory (RAM, disk, tape, flash memory, …) – usually we do not difference between cache, RAM, …
– In pracGce, for example, random access on tape impossible
• Program 1.17 float Sum(float *a, const int n) { float s = 0; for (int i=0; i<n; i++) s += a[i]; return s; } – The instance characterisGc is n. – Since a is actually the address of the first element of a[], and n is passed by value, the space needed by Sum() is constant (Ssum(n)=1).
• Program 1.18 float RSum(float *a, const int n) { if (n <= 0) return 0; else return (RSum(a, n-‐1) + a[n-‐1]); }
– Each call requires at least 4 words
• The values of n, a, return value and return address. – The depth of the recursion is n+1.
• The stack space needed is 4(n+1).
n=997
n=1000
n=999
n=998
Input size = n
• usually input size denoted by n • Time complexity funcGon = f(n)
– array[1..n] – e.g. 4*n + 3
• Graph: n verGces, m edges f(n,m)
• So, we may be able to count a nr of operaGons needed
• What do we do with this knowledge?
n2
n log n
n2
100 n log n
n2
100 n log n
n = 1000
0.00001*n2
100 n log n
n = 10,000,000
0.00001*n2
100 n log n
n = 2,000,000,000
Logscale y
logscale x logscale y
• plot [1:10] 0.01*x*x , 5*log(x), x*log(x)/3
Algorithm analysis goal
• What happens in the “long run”, increasing n
• Compare f(n) to reference g(n) (or f and g )
• At some n0 > 0 , if n > n0 , always f(n) < g(n)
Θ , O, and Ω
Figure 2.1 Graphic examples of the Θ , O, and Ω notations. In each part, the value of n0 shown is the minimum possible value; any greater value would also work.
Dominant terms only…
• EssenGally, we are interested in the largest (dominant) term only…
• When this grows large enough, it will “overshadow” all smaller terms
Theorem 1.2
).()( Thes, .for ,)(
have we,1 ,let Therefore,
.1for ,
)(
:Proof).()( then ,...)( If
0
00
0
0
00
01
mm
m
ii
m
ii
m
m
i
mii
m
m
i
ii
m
i
ii
mmm
nOnfnncnnf
nac
nan
nan
nananf
nOnfanananf
=≥≤
==
≥≤
≤
≤=
=+++=
∑
∑
∑
∑∑
=
=
=
−
==
AsymptoGc Analysis • Given any two funcGons f(n) and g(n), we will restrict
ourselves to: – polynomials with posiGve leading coefficient – exponenGal and logarithmic funcGons
• These funcGons as • We will consider the limit of the raGo:
)g()f(limnn
n ∞→
∞→n∞→
AsymptoGc Analysis • If the two funcGon f(n) and g(n) describe the run Gmes of two
algorithms, and
that is, the limit is a constant, then we can always run the slower algorithm on a faster computer to get similar results
∞<<∞→ )g(
)f(lim0nn
n
AsymptoGc Analysis • To formally describe equivalent run Gmes, we will say that
f(n) = Θ(g(n)) if
• Note: this is not equality – it would have been bezer if it said f(n) ∈ Θ(g(n)) however, someone picked =
∞<<∞→ )g(
)f(lim0nn
n
AsymptoGc Analysis • We are also interested if one algorithm runs either
asymptoGcally slower or faster than another
• If this is true, we will say that f(n) = O(g(n))
∞<∞→ )g(
)f(limnn
n
AsymptoGc Analysis • If the limit is zero, i.e.,
then we will say that f(n) = o(g(n)) t • This is the case if f(n) and g(n) are polynomials where f has a
lower degree
0)g()f(lim =
∞→ nn
n
AsymptoGc Analysis
• To summarize:
0)g()f(lim >
∞→ nn
n
))(g()f( nn O=
))(g()f( nn Θ=
))(g()f( nn Ω=
∞<∞→ )g(
)f(limnn
n
∞<<∞→ )g(
)f(lim0nn
n
AsymptoGc Analysis • We have one final case:
∞=∞→ )g(
)f(limnn
n
))(g()f( nn o=
))(g()f( nn Θ=
))(g()f( nn ω=
0)g()f(lim =
∞→ nn
n
∞<<∞→ )g(
)f(lim0nn
n
AsymptoGc Analysis
• Graphically, we can summarize these as follows:
We say if
AsymptoGc Analysis
• All of n2 100000 n2 – 4 n + 19 n2 + 1000000 323 n2 – 4 n ln(n) + 43 n + 10 42n2 + 32 n2 + 61 n ln2(n) + 7n + 14 ln3(n) + ln(n) are big-‐Θ of each other
• E.g., 42n2 + 32 = Θ( 323 n2 – 4 n ln(n) + 43 n + 10 )
AsymptoGc Analysis • We will focus on these Θ(1) constant Θ(ln(n)) logarithmic Θ(n) linear Θ(n ln(n)) “n–log–n” Θ(n2) quadraGc Θ(n3) cubic 2n, en, 4n, ... exponenGal
O(1)<O(logn)<O(n)<O(nlogn)<O(n2)<O(n3)<O(2n)<O(n!)
Growth of funcGons
• See Chapter “Growth of FuncGons” (CLRS)
• hzp://net.pku.edu.cn/~course/cs101/resource/Intro2Algorithm/book6/chap02.htm
Logarithms
• Algebra cheat sheet: hzp://tutorial.math.lamar.edu/pdf/Algebra_Cheat_Sheet.pdf
• hzp://en.wikipedia.org/wiki/Logarithm
Change of base a → b
xb
x aa
b loglog1log =
)(loglog xx ab Θ=
Big-‐Oh notaGon classes
Class Informal Intui5on Analogy
f(n) ∈ ο ( g(n) ) f is dominated by g Strictly below < f(n) ∈ O( g(n) ) Bounded from above Upper bound ≤ f(n) ∈ Θ( g(n) ) Bounded from
above and below “equal to” =
f(n) ∈ Ω( g(n) ) Bounded from below Lower bound ≥ f(n) ∈ ω( g(n) )
f dominates g Strictly above >
Family of Bachmann–Landau nota5ons
!
Notation Name Intuition As , eventually... Definition
Big Omicron; Big O; Big Oh
f is bounded above by g(up to constant factor) asymptotically
for some k
or
(Note that, since the beginning of the 20th century, papers in number theory have been increasingly and widely using this notation in the weaker sense that f = o(g) is false)
Big Omega
f is bounded below by g(up to constant factor) asymptotically
for some positivek
Big Theta f is bounded both above and below bygasymptotically for some positive k1, k2
Small Omicron; Small O; Small Oh
f is dominated by gasymptotically for every !
Small Omega f dominates gasymptotically for every k
on the order of; "twiddles"
f is equal to gasymptotically
How much Gme does sorGng take?
• Comparison-‐based sort: A[i] <= A[j] – Upper bound – current best-‐known algorithm – Lower bound – theoreGcal “at least” esGmate – If they are equal, we have theoreGcally opGmal soluGon
Lihtne sorteerimine
for i=2..n
for j=i ; j>1 ; j-‐-‐ if A[j] < A[j-‐1] then swap( A[j], A[j-‐1] ) else next i
The divide-‐and-‐conquer design paradigm
1. Divide the problem (instance) into subproblems.
2. Conquer the subproblems by solving them recursively.
3. Combine subproblem solu>ons.
Merge sort
Merge-‐Sort(A,p,r) if p<r then q = (p+r)/2
Merge-‐Sort( A, p, q ) Merge-‐Sort( A, q+1,r) Merge( A, p, q, r )
It was invented by John von Neumann in 1945.
Example
• Applying the merge sort algorithm:
Wikipedia / viz. Va
lue
Pos in array
Quicksort
Wikipedia / “video”
Kui palju võtab sorteerimine aega?
• Võrdlustel põhinev: A[i] <= A[j] – Ülempiir – praegu parim teadaolev algoritm – Kas saame hinnata alampiiri?
• Aga kui palju kahendotsimine? • Aga kuidas lisada elemente tabelisse? • (kahend-‐otsimis) Puu ?
n2.3727
n2.376
Conclusions
• Algorithm complexity deals with the behavior in the long-‐term – worst case -‐-‐ typical – average case -‐-‐ quite hard – best case -‐-‐ bogus, “chea-ng”
• In pracGce, long-‐term someGmes not necessary – E.g. for sorGng 20 elements, you don’t need fancy algorithms…