j. elder cosc 3101n thinking about algorithms abstractly lecture 2. relevant mathematics:...
TRANSCRIPT
COSC 3101N
J. Elder
Thinking about Algorithms Abstractly
Lecture 2. Relevant Mathematics: Classifying Functions
Input Size
Tim
ef(i) = n(n)
COSC 3101N
J. Elder
Assumed Knowledge
Read J. Edmonds, How to Think About Algorithms (HTA):
• Section 1.1 Existential and Universal Quantifiers
• Section 1.2 Logarithms and Exponentials
Loves( , )
Loves( , )
g b b g
g b b g
COSC 3101N
J. Elder
Classifying Functions
Giving an idea of how fast a function grows without going into too much detail.
COSC 3101N
J. Elder
Which are more alike?
COSC 3101N
J. Elder
Which are more alike?
Mammals
COSC 3101N
J. Elder
Which are more alike?
COSC 3101N
J. Elder
Which are more alike?
Dogs
COSC 3101N
J. Elder
Classifying Animals
Vertebrates
Birds
Mam
mals
Reptiles
Fish
Dogs
Giraffe
COSC 3101N
J. Elder
Classifying Functions
T(n) 10 100 1,000 10,000
log n 3 6 9 13
n1/2 3 10 31 100
10 100 1,000 10,000
n log n 30 600 9,000 130,000
n2 100 10,000 106 108
n3 1,000 106 109 1012
2n 1,024 1030 10300 103000
Note: The universe contains approximately 1050 particles.
n
COSC 3101N
J. Elder
Which are more alike?
n1000 n2 2n
COSC 3101N
J. Elder
Which are more alike?
Polynomials
n1000 n2 2n
COSC 3101N
J. Elder
Which are more alike?
1000n2 3n2 2n3
COSC 3101N
J. Elder
Which are more alike?
Quadratic
1000n2 3n2 2n3
COSC 3101N
J. Elder
Classifying Functions?
Functions
COSC 3101N
J. Elder
Classifying Functions
Functions
Constant
Logarithm
ic
Poly L
ogarithmic
Polynom
ial
Exponential
Exp
Double E
xp
(log n)5 n5 25n5 log n5 2n5 25n
2
COSC 3101N
J. Elder
Classifying Functions?
Polynomial
COSC 3101N
J. Elder
Classifying Functions
Polynomial
Linear
Quadratic
Cubic
?
5n25n 5n3 5n4
COSC 3101N
J. Elder
Logarithmic
• log10n = # digits to write n
• log2n = # bits to write n = 3.32 log10n
• log(n1000) = 1000 log(n)
Differ only by a multiplicative constant.
COSC 3101N
J. Elder
Poly Logarithmic
(log n)5 = log5 n
COSC 3101N
J. Elder
Logarithmic << Polynomial
For sufficiently large n
log1000 n << n0.001
COSC 3101N
J. Elder
Linear << Quadratic
For sufficiently large n
10000 n << 0.0001 n2
COSC 3101N
J. Elder
Polynomial << Exponential
For sufficiently large n
n1000 << 20.001 n
COSC 3101N
J. Elder
Ordering Functions
Functions
Constant
Logarithm
ic
Poly L
ogarithmic
Polynom
ial
Exponential
Exp
Double E
xp
(log n)5 n5 25n5 log n5 2n5 25n
2
<< << << << << <<
<< << << << << <<
COSC 3101N
J. Elder
Which Functions are Constant?
• 5• 1,000,000,000,000• 0.0000000000001• -5• 0• 8 + sin(n)
YesYesYes
No
YesYes
COSC 3101N
J. Elder
Which Functions are “Constant”?
• 5• 1,000,000,000,000• 0.0000000000001• -5• 0• 8 + sin(n)
YesYesYesNo
NoYes
Lies in between
7
9
The running time of the algorithm is a “Constant” It does not depend significantly
on the size of the input.
COSC 3101N
J. Elder
Which Functions are Quadratic?
• n2
• … ?
COSC 3101N
J. Elder
Which Functions are Quadratic?
• n2
• 0.001 n2
• 1000 n2
Some constant times n2.
COSC 3101N
J. Elder
Which Functions are Quadratic?
• n2
• 0.001 n2
• 1000 n2
• 5n2 + 3n + 2log n
COSC 3101N
J. Elder
Which Functions are Quadratic?
• n2
• 0.001 n2
• 1000 n2
• 5n2 + 3n + 2log n
Lies in between
COSC 3101N
J. Elder
Which Functions are Quadratic?
• n2
• 0.001 n2
• 1000 n2
• 5n2 + 3n + 2log n
Ignore low-order termsIgnore multiplicative constants.
Ignore "small" values of n. Write θ(n2).
COSC 3101N
J. Elder
Which Functions are Polynomial?
• n5
• … ?
COSC 3101N
J. Elder
Which Functions are Polynomial?
• nc
• n0.0001
• n10000
n to some constant power.
COSC 3101N
J. Elder
Which Functions are Polynomial?
• nc
• n0.0001
• n10000
• 5n2 + 8n + 2log n • 5n2 log n • 5n2.5
COSC 3101N
J. Elder
Which Functions are Polynomial?
• nc
• n0.0001
• n10000
• 5n2 + 8n + 2log n • 5n2 log n • 5n2.5
Lie in between
COSC 3101N
J. Elder
Which Functions are Polynomials?• nc
• n0.0001
• n10000
• 5n2 + 8n + 2log n • 5n2 log n • 5n2.5
Ignore low-order termsIgnore power constant.
Ignore "small" values of n. Write nθ(1)
COSC 3101N
J. Elder
Which Functions are Exponential?
• 2n
• … ?
COSC 3101N
J. Elder
Which Functions are Exponential?
• 2n
• 20.0001 n
• 210000 n
n times some constant powerraised to the power of 2.
COSC 3101N
J. Elder
Which Functions are Exponential?
• 2n
• 20.0001 n
• 210000 n
• 8n
• 2n / n100 •2n · n100
too small?too big?
COSC 3101N
J. Elder
Which Functions are Exponential?
• 2n
• 20.0001 n
• 210000 n
• 8n
• 2n / n100 •2n · n100
= 23n
> 20.5n
< 22n
Lie in between
COSC 3101N
J. Elder
Which Functions are Exponential?
• 2n
• 20.0001 n
• 210000 n
• 8n
• 2n / n100 •2n · n100
= 23n
> 20.5n
< 22n
20.5n > n100
2n = 20.5n · 20.5n > n100 · 20.5n
2n / n100 > 20.5n
COSC 3101N
J. Elder
Which Functions are Exponential?
Ignore low-order termsIgnore base.
Ignore "small" values of n. Ignore polynomial factors.
Write 2θ(n)
• 2n
• 20.0001 n
• 210000 n
• 8n
• 2n / n100 •2n · n100
COSC 3101N
J. Elder
Classifying Functions
Functions
Constant
Logarithm
ic
Poly L
ogarithmic
Polynom
ial
Exponential
Exp
Double E
xp
(log n)θ(1) nθ(1) 2θ(n)θ(log n)θ(1) 2nθ(1) 2θ(n)
2
COSC 3101N
J. Elder
Notations
Theta f(n) = θ(g(n)) f(n) ≈ c g(n)
BigOh f(n) = O(g(n)) f(n) ≤ c g(n)
Omega f(n) = Ω(g(n)) f(n) ≥ c g(n)
Little Oh f(n) = o(g(n)) f(n) << c g(n)
Little Omega f(n) = ω(g(n)) f(n) >> c g(n)
COSC 3101N
J. Elder
Definition of Theta
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
f(n) = θ(g(n))
COSC 3101N
J. Elder
Definition of Theta
f(n) is sandwiched between c1g(n) and c2g(n)
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
f(n) = θ(g(n))
COSC 3101N
J. Elder
Definition of Theta
f(n) is sandwiched between c1g(n) and c2g(n)
for some sufficiently small c1 (= 0.0001)
for some sufficiently large c2 (= 1000)
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
f(n) = θ(g(n))
COSC 3101N
J. Elder
Definition of Theta
For all sufficiently large n
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
f(n) = θ(g(n))
COSC 3101N
J. Elder
Definition of Theta
For all sufficiently large n
For some definition of “sufficiently large”
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
f(n) = θ(g(n))
COSC 3101N
J. Elder
Definition of Theta
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
3n2 + 7n + 8 = θ(n2)
COSC 3101N
J. Elder
Definition of Theta
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
3n2 + 7n + 8 = θ(n2)
c1·n2 3n2 + 7n + 8 c2·n2? ?
COSC 3101N
J. Elder
Definition of Theta
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
3n2 + 7n + 8 = θ(n2)
3·n2 3n2 + 7n + 8 4·n23 4
COSC 3101N
J. Elder
Definition of Theta
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
3n2 + 7n + 8 = θ(n2)
3·12 3·12 + 7·1 + 8 4·121False
COSC 3101N
J. Elder
Definition of Theta
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
3n2 + 7n + 8 = θ(n2)
3·72 3·72 + 7·7 + 8 4·727False
COSC 3101N
J. Elder
Definition of Theta
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
3n2 + 7n + 8 = θ(n2)
3·82 3·82 + 7·8 + 8 4·828True
COSC 3101N
J. Elder
Definition of Theta
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
3n2 + 7n + 8 = θ(n2)
3·92 3·92 + 7·9 + 8 4·929True
COSC 3101N
J. Elder
Definition of Theta
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
3n2 + 7n + 8 = θ(n2)
3·102 3·102 + 7·10 + 8 4·10210True
COSC 3101N
J. Elder
Definition of Theta
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
3n2 + 7n + 8 = θ(n2)
3·n2 3n2 + 7n + 8 4·n2?
COSC 3101N
J. Elder
Definition of Theta
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
3n2 + 7n + 8 = θ(n2)
3·n2 3n2 + 7n + 8 4·n2n 88
COSC 3101N
J. Elder
Definition of Theta
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
3n2 + 7n + 8 = θ(n2)
3·n2 3n2 + 7n + 8 4·n2
Truen 8
COSC 3101N
J. Elder
Definition of Theta
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
3n2 + 7n + 8 = θ(n2)
3·n2 3n2 + 7n + 8 4·n2
7n + 8 1·n2
7 + 8/n 1·n
n 8
True
COSC 3101N
J. Elder
Definition of Theta
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
3n2 + 7n + 8 = θ(n2)
3·n2 3n2 + 7n + 8 4·n23 4 8 n 8
True
COSC 3101N
J. Elder
Definition of Theta
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
n2 = θ(n3)
COSC 3101N
J. Elder
Definition of Theta
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
n2 = θ(n3)
c1 · n3 n2 c2 · n3? ?
COSC 3101N
J. Elder
Definition of Theta
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
n2 = θ(n3)
0 · n3 n2 c2 · n30True, but ?
COSC 3101N
J. Elder
Definition of Theta
c c n n n1 0 2 0 0 0, , , , . . .
n2 = θ(n3)
0 Constants c1 and c2 must be positive!
False
COSC 3101N
J. Elder
Definition of Theta
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
3n2 + 7n + 8 = θ(n)
COSC 3101N
J. Elder
Definition of Theta
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
3n2 + 7n + 8 = θ(n)
c1·n 3n2 + 7n + 8 c2·n? ?
COSC 3101N
J. Elder
Definition of Theta
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
3n2 + 7n + 8 = θ(n)
3·n 3n2 + 7n + 8 100·n3 100
COSC 3101N
J. Elder
Definition of Theta
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
3n2 + 7n + 8 = θ(n)
3·n 3n2 + 7n + 8 100·n?
COSC 3101N
J. Elder
Definition of Theta
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
3n2 + 7n + 8 = θ(n)
3·100 3·1002 + 7·100 + 8 100·100100
False
COSC 3101N
J. Elder
Definition of Theta
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
3n2 + 7n + 8 = θ(n)
3·n 3n2 + 7n + 8 10,000·n3 10,000
COSC 3101N
J. Elder
Definition of Theta
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
3n2 + 7n + 8 = θ(n)
3·10,000 3·10,000 2 + 7·10,000 + 8 100·10,00010,000
False
COSC 3101N
J. Elder
Definition of Theta
3n2 + 7n + 8 = θ(n)
What is the reverse statement?
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
COSC 3101N
J. Elder
Understand Quantifiers!!!
Sam Mary
Bob Beth
JohnMarilyn Monroe
Fred Ann
Sam Mary
Bob Beth
JohnMarilyn Monroe
Fred Ann
b, Loves(b, MM)
b, Loves(b, MM)
b, Loves(b, MM)]
b, Loves(b, MM)]
COSC 3101N
J. Elder
Definition of Theta
c c n n n c g n f n or f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( ) ( )
3n2 + 7n + 8 = θ(n)
The reverse statement
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
COSC 3101N
J. Elder
Definition of Theta
c c n n n c g n f n or f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( ) ( )
3n2 + 7n + 8 = θ(n)
3n2 + 7n + 8 > 100·n3 100 ?
COSC 3101N
J. Elder
Definition of Theta
c c n n n c g n f n or f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( ) ( )
3n2 + 7n + 8 = θ(n)
3·1002 + 7·100 + 8 > 100·1003 100
True100
COSC 3101N
J. Elder
Definition of Theta
c c n n n c g n f n or f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( ) ( )
3n2 + 7n + 8 = θ(n)
3n2 + 7n + 8 > 10,000·n3 10,000 ?
COSC 3101N
J. Elder
Definition of Theta
c c n n n c g n f n or f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( ) ( )
3n2 + 7n + 8 = θ(n)
3·10,0002 + 7·10,000 + 8 > 10,000·10,0003 10,000
True10,000
COSC 3101N
J. Elder
Definition of Theta
c c n n n c g n f n or f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( ) ( )
3n2 + 7n + 8 = θ(n)
3n2 + 7n + 8 > c2·nc1 c2 ?
COSC 3101N
J. Elder
Definition of Theta
c c n n n c g n f n or f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( ) ( )
3n2 + 7n + 8 = θ(n)
3·c22 + 7 · c2 + 8 > c2·c2c1 c2
Truec2
COSC 3101N
J. Elder
Definition of Theta
c c n n n c g n f n or f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( ) ( )
3n2 + 7n + 8 = θ(n)
3n2 + 7n + 8 > c2·nc1 c2 ?n0
COSC 3101N
J. Elder
Definition of Theta
c c n n n c g n f n or f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( ) ( )
3n2 + 7n + 8 = θ(n)
3·c22 + 7 · c2 + 8 > c2·c2c1 max(c2,n0 )
Truec2 n0
True
COSC 3101N
J. Elder
Definition of Theta
3n2 + 7n + 8 = g(n)θ(1)
c1, c2, n0, n n0 g(n)c1 f(n) g(n)c2
3n2 + 7n + 8 = nθ(1)
COSC 3101N
J. Elder
Definition of Theta
3n2 + 7n + 8 = nθ(1)
nc1 3n2 + 7n + 8 nc2
c1, c2, n0, n n0 g(n)c1 f(n) g(n)c2
? ?
COSC 3101N
J. Elder
Definition of Theta
3n2 + 7n + 8 = nθ(1)
n2 3n2 + 7n + 8 n3
c1, c2, n0, n n0 g(n)c1 f(n) g(n)c2
2 3
COSC 3101N
J. Elder
Definition of Theta
3n2 + 7n + 8 = nθ(1)
n2 3n2 + 7n + 8 n3
c1, c2, n0, n n0 g(n)c1 f(n) g(n)c2
2 3 ? n ?
COSC 3101N
J. Elder
Definition of Theta
3n2 + 7n + 8 = nθ(1)
n2 3n2 + 7n + 8 n3
c1, c2, n0, n n0 g(n)c1 f(n) g(n)c2
2 3 5 n 5
COSC 3101N
J. Elder
Definition of Theta
3n2 + 7n + 8 = nθ(1)
n2 3n2 + 7n + 8 n3
True
True
c1, c2, n0, n n0 g(n)c1 f(n) g(n)c2
2 3 5 n 5
COSC 3101N
J. Elder
Order of Quantifiers
n n n c c c g n f n c g n0 0 1 2 1 2, , ( ) ( ) ( ), ,
f(n) = θ(g(n))
?
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
COSC 3101N
J. Elder
g b loves b g, , ( , ) b g loves b g, , ( , )
Understand Quantifiers!!!
One girl Could be a separate girl for each boy.
Sam Mary
Bob Beth
JohnMarilin Monro
Fred Ann
Sam Mary
Bob Beth
JohnMarilin Monro
Fred Ann
COSC 3101N
J. Elder
Order of Quantifiers
No! It cannot be a different c1 and c2
for each n.
n n n c c c g n f n c g n0 0 1 2 1 2, , ( ) ( ) ( ), ,
f(n) = θ(g(n))
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
COSC 3101N
J. Elder
Other Notations
Theta f(n) = θ(g(n)) f(n) ≈ c g(n)
BigOh f(n) = O(g(n)) f(n) ≤ c g(n)
Omega f(n) = Ω(g(n)) f(n) ≥ c g(n)
Little Oh f(n) = o(g(n)) f(n) << c g(n)
Little Omega f(n) = ω(g(n)) f(n) >> c g(n)
COSC 3101N
J. Elder
BigOh Notation
• n2 = O(n3)
• n3 = O(n2)
COSC 3101N
J. Elder
BigOh Notation
• O(n2) = O(n3)
• O(n3) = O(n2)Odd Notation
f(n) = O(g(n)) Standard f(n) ≤ O(g(n)) Stresses one function dominating another.
f(n)O(g(n)) Stress function is member of class.