j. elder cosc 3101n thinking about algorithms abstractly lecture 2. relevant mathematics:...

of 95 /95
COSC 3101N J. Elder Thinking about Algorithms Abstractly Lecture 2. Relevant Mathematics: Classifying Functions Input Size Time f(i) = n (n)

Upload: helen-carter

Post on 31-Dec-2015

213 views

Category:

Documents


0 download

TRANSCRIPT

COSC 3101N

J. Elder

Thinking about Algorithms Abstractly

Lecture 2. Relevant Mathematics: Classifying Functions

Input Size

Tim

ef(i) = n(n)

COSC 3101N

J. Elder

Assumed Knowledge

Read J. Edmonds, How to Think About Algorithms (HTA):

• Section 1.1 Existential and Universal Quantifiers

• Section 1.2 Logarithms and Exponentials

Loves( , )

Loves( , )

g b b g

g b b g

COSC 3101N

J. Elder

Classifying Functions

Giving an idea of how fast a function grows without going into too much detail.

COSC 3101N

J. Elder

Which are more alike?

COSC 3101N

J. Elder

Which are more alike?

Mammals

COSC 3101N

J. Elder

Which are more alike?

COSC 3101N

J. Elder

Which are more alike?

Dogs

COSC 3101N

J. Elder

Classifying Animals

Vertebrates

Birds

Mam

mals

Reptiles

Fish

Dogs

Giraffe

COSC 3101N

J. Elder

Classifying Functions

T(n) 10 100 1,000 10,000

log n   3 6 9 13

n1/2 3 10 31 100

10 100 1,000 10,000

n log n 30 600 9,000 130,000

n2 100 10,000 106 108

n3 1,000 106 109 1012

2n 1,024 1030 10300 103000

Note: The universe contains approximately 1050 particles.

n

COSC 3101N

J. Elder

Which are more alike?

n1000 n2 2n

COSC 3101N

J. Elder

Which are more alike?

Polynomials

n1000 n2 2n

COSC 3101N

J. Elder

Which are more alike?

1000n2 3n2 2n3

COSC 3101N

J. Elder

Which are more alike?

Quadratic

1000n2 3n2 2n3

COSC 3101N

J. Elder

Classifying Functions?

Functions

COSC 3101N

J. Elder

Classifying Functions

Functions

Constant

Logarithm

ic

Poly L

ogarithmic

Polynom

ial

Exponential

Exp

Double E

xp

(log n)5 n5 25n5 log n5 2n5 25n

2

COSC 3101N

J. Elder

Classifying Functions?

Polynomial

COSC 3101N

J. Elder

Classifying Functions

Polynomial

Linear

Quadratic

Cubic

?

5n25n 5n3 5n4

COSC 3101N

J. Elder

Logarithmic

• log10n = # digits to write n

• log2n = # bits to write n = 3.32 log10n

• log(n1000) = 1000 log(n)

Differ only by a multiplicative constant.

COSC 3101N

J. Elder

Poly Logarithmic

(log n)5 = log5 n

COSC 3101N

J. Elder

Logarithmic << Polynomial

For sufficiently large n

log1000 n << n0.001

COSC 3101N

J. Elder

Linear << Quadratic

For sufficiently large n

10000 n << 0.0001 n2

COSC 3101N

J. Elder

Polynomial << Exponential

For sufficiently large n

n1000 << 20.001 n

COSC 3101N

J. Elder

Ordering Functions

Functions

Constant

Logarithm

ic

Poly L

ogarithmic

Polynom

ial

Exponential

Exp

Double E

xp

(log n)5 n5 25n5 log n5 2n5 25n

2

<< << << << << <<

<< << << << << <<

COSC 3101N

J. Elder

Which Functions are Constant?

• 5• 1,000,000,000,000• 0.0000000000001• -5• 0• 8 + sin(n)

YesYesYes

No

YesYes

COSC 3101N

J. Elder

Which Functions are “Constant”?

• 5• 1,000,000,000,000• 0.0000000000001• -5• 0• 8 + sin(n)

YesYesYesNo

NoYes

Lies in between

7

9

The running time of the algorithm is a “Constant” It does not depend significantly

on the size of the input.

COSC 3101N

J. Elder

Which Functions are Quadratic?

• n2

• … ?

COSC 3101N

J. Elder

Which Functions are Quadratic?

• n2

• 0.001 n2

• 1000 n2

Some constant times n2.

COSC 3101N

J. Elder

Which Functions are Quadratic?

• n2

• 0.001 n2

• 1000 n2

• 5n2 + 3n + 2log n

COSC 3101N

J. Elder

Which Functions are Quadratic?

• n2

• 0.001 n2

• 1000 n2

• 5n2 + 3n + 2log n

Lies in between

COSC 3101N

J. Elder

Which Functions are Quadratic?

• n2

• 0.001 n2

• 1000 n2

• 5n2 + 3n + 2log n

Ignore low-order termsIgnore multiplicative constants.

Ignore "small" values of n. Write θ(n2).

COSC 3101N

J. Elder

Which Functions are Polynomial?

• n5

• … ?

COSC 3101N

J. Elder

Which Functions are Polynomial?

• nc

• n0.0001

• n10000

n to some constant power.

COSC 3101N

J. Elder

Which Functions are Polynomial?

• nc

• n0.0001

• n10000

• 5n2 + 8n + 2log n • 5n2 log n • 5n2.5

COSC 3101N

J. Elder

Which Functions are Polynomial?

• nc

• n0.0001

• n10000

• 5n2 + 8n + 2log n • 5n2 log n • 5n2.5

Lie in between

COSC 3101N

J. Elder

Which Functions are Polynomials?• nc

• n0.0001

• n10000

• 5n2 + 8n + 2log n • 5n2 log n • 5n2.5

Ignore low-order termsIgnore power constant.

Ignore "small" values of n. Write nθ(1)

COSC 3101N

J. Elder

Which Functions are Exponential?

• 2n

• … ?

COSC 3101N

J. Elder

Which Functions are Exponential?

• 2n

• 20.0001 n

• 210000 n

n times some constant powerraised to the power of 2.

COSC 3101N

J. Elder

Which Functions are Exponential?

• 2n

• 20.0001 n

• 210000 n

• 8n

• 2n / n100 •2n · n100

too small?too big?

COSC 3101N

J. Elder

Which Functions are Exponential?

• 2n

• 20.0001 n

• 210000 n

• 8n

• 2n / n100 •2n · n100

= 23n

> 20.5n

< 22n

Lie in between

COSC 3101N

J. Elder

Which Functions are Exponential?

• 2n

• 20.0001 n

• 210000 n

• 8n

• 2n / n100 •2n · n100

= 23n

> 20.5n

< 22n

20.5n > n100

2n = 20.5n · 20.5n > n100 · 20.5n

2n / n100 > 20.5n

COSC 3101N

J. Elder

Which Functions are Exponential?

Ignore low-order termsIgnore base.

Ignore "small" values of n. Ignore polynomial factors.

Write 2θ(n)

• 2n

• 20.0001 n

• 210000 n

• 8n

• 2n / n100 •2n · n100

COSC 3101N

J. Elder

Classifying Functions

Functions

Constant

Logarithm

ic

Poly L

ogarithmic

Polynom

ial

Exponential

Exp

Double E

xp

(log n)θ(1) nθ(1) 2θ(n)θ(log n)θ(1) 2nθ(1) 2θ(n)

2

COSC 3101N

J. Elder

Notations

Theta f(n) = θ(g(n)) f(n) ≈ c g(n)

BigOh f(n) = O(g(n)) f(n) ≤ c g(n)

Omega f(n) = Ω(g(n)) f(n) ≥ c g(n)

Little Oh f(n) = o(g(n)) f(n) << c g(n)

Little Omega f(n) = ω(g(n)) f(n) >> c g(n)

COSC 3101N

J. Elder

Definition of Theta

c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )

f(n) = θ(g(n))

COSC 3101N

J. Elder

Definition of Theta

f(n) is sandwiched between c1g(n) and c2g(n)

c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )

f(n) = θ(g(n))

COSC 3101N

J. Elder

Definition of Theta

f(n) is sandwiched between c1g(n) and c2g(n)

for some sufficiently small c1 (= 0.0001)

for some sufficiently large c2 (= 1000)

c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )

f(n) = θ(g(n))

COSC 3101N

J. Elder

Definition of Theta

For all sufficiently large n

c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )

f(n) = θ(g(n))

COSC 3101N

J. Elder

Definition of Theta

For all sufficiently large n

For some definition of “sufficiently large”

c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )

f(n) = θ(g(n))

COSC 3101N

J. Elder

Definition of Theta

c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )

3n2 + 7n + 8 = θ(n2)

COSC 3101N

J. Elder

Definition of Theta

c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )

3n2 + 7n + 8 = θ(n2)

c1·n2 3n2 + 7n + 8 c2·n2? ?

COSC 3101N

J. Elder

Definition of Theta

c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )

3n2 + 7n + 8 = θ(n2)

3·n2 3n2 + 7n + 8 4·n23 4

COSC 3101N

J. Elder

Definition of Theta

c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )

3n2 + 7n + 8 = θ(n2)

3·12 3·12 + 7·1 + 8 4·121False

COSC 3101N

J. Elder

Definition of Theta

c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )

3n2 + 7n + 8 = θ(n2)

3·72 3·72 + 7·7 + 8 4·727False

COSC 3101N

J. Elder

Definition of Theta

c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )

3n2 + 7n + 8 = θ(n2)

3·82 3·82 + 7·8 + 8 4·828True

COSC 3101N

J. Elder

Definition of Theta

c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )

3n2 + 7n + 8 = θ(n2)

3·92 3·92 + 7·9 + 8 4·929True

COSC 3101N

J. Elder

Definition of Theta

c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )

3n2 + 7n + 8 = θ(n2)

3·102 3·102 + 7·10 + 8 4·10210True

COSC 3101N

J. Elder

Definition of Theta

c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )

3n2 + 7n + 8 = θ(n2)

3·n2 3n2 + 7n + 8 4·n2?

COSC 3101N

J. Elder

Definition of Theta

c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )

3n2 + 7n + 8 = θ(n2)

3·n2 3n2 + 7n + 8 4·n2n 88

COSC 3101N

J. Elder

Definition of Theta

c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )

3n2 + 7n + 8 = θ(n2)

3·n2 3n2 + 7n + 8 4·n2

Truen 8

COSC 3101N

J. Elder

Definition of Theta

c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )

3n2 + 7n + 8 = θ(n2)

3·n2 3n2 + 7n + 8 4·n2

7n + 8 1·n2

7 + 8/n 1·n

n 8

True

COSC 3101N

J. Elder

Definition of Theta

c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )

3n2 + 7n + 8 = θ(n2)

3·n2 3n2 + 7n + 8 4·n23 4 8 n 8

True

COSC 3101N

J. Elder

Definition of Theta

c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )

n2 = θ(n3)

COSC 3101N

J. Elder

Definition of Theta

c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )

n2 = θ(n3)

c1 · n3 n2 c2 · n3? ?

COSC 3101N

J. Elder

Definition of Theta

c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )

n2 = θ(n3)

0 · n3 n2 c2 · n30True, but ?

COSC 3101N

J. Elder

Definition of Theta

c c n n n1 0 2 0 0 0, , , , . . .

n2 = θ(n3)

0 Constants c1 and c2 must be positive!

False

COSC 3101N

J. Elder

Definition of Theta

c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )

3n2 + 7n + 8 = θ(n)

COSC 3101N

J. Elder

Definition of Theta

c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )

3n2 + 7n + 8 = θ(n)

c1·n 3n2 + 7n + 8 c2·n? ?

COSC 3101N

J. Elder

Definition of Theta

c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )

3n2 + 7n + 8 = θ(n)

3·n 3n2 + 7n + 8 100·n3 100

COSC 3101N

J. Elder

Definition of Theta

c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )

3n2 + 7n + 8 = θ(n)

3·n 3n2 + 7n + 8 100·n?

COSC 3101N

J. Elder

Definition of Theta

c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )

3n2 + 7n + 8 = θ(n)

3·100 3·1002 + 7·100 + 8 100·100100

False

COSC 3101N

J. Elder

Definition of Theta

c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )

3n2 + 7n + 8 = θ(n)

3·n 3n2 + 7n + 8 10,000·n3 10,000

COSC 3101N

J. Elder

Definition of Theta

c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )

3n2 + 7n + 8 = θ(n)

3·10,000 3·10,000 2 + 7·10,000 + 8 100·10,00010,000

False

COSC 3101N

J. Elder

Definition of Theta

3n2 + 7n + 8 = θ(n)

What is the reverse statement?

c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )

COSC 3101N

J. Elder

Understand Quantifiers!!!

Sam Mary

Bob Beth

JohnMarilyn Monroe

Fred Ann

Sam Mary

Bob Beth

JohnMarilyn Monroe

Fred Ann

b, Loves(b, MM)

b, Loves(b, MM)

b, Loves(b, MM)]

b, Loves(b, MM)]

COSC 3101N

J. Elder

Definition of Theta

c c n n n c g n f n or f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( ) ( )

3n2 + 7n + 8 = θ(n)

The reverse statement

c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )

COSC 3101N

J. Elder

Definition of Theta

c c n n n c g n f n or f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( ) ( )

3n2 + 7n + 8 = θ(n)

3n2 + 7n + 8 > 100·n3 100 ?

COSC 3101N

J. Elder

Definition of Theta

c c n n n c g n f n or f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( ) ( )

3n2 + 7n + 8 = θ(n)

3·1002 + 7·100 + 8 > 100·1003 100

True100

COSC 3101N

J. Elder

Definition of Theta

c c n n n c g n f n or f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( ) ( )

3n2 + 7n + 8 = θ(n)

3n2 + 7n + 8 > 10,000·n3 10,000 ?

COSC 3101N

J. Elder

Definition of Theta

c c n n n c g n f n or f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( ) ( )

3n2 + 7n + 8 = θ(n)

3·10,0002 + 7·10,000 + 8 > 10,000·10,0003 10,000

True10,000

COSC 3101N

J. Elder

Definition of Theta

c c n n n c g n f n or f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( ) ( )

3n2 + 7n + 8 = θ(n)

3n2 + 7n + 8 > c2·nc1 c2 ?

COSC 3101N

J. Elder

Definition of Theta

c c n n n c g n f n or f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( ) ( )

3n2 + 7n + 8 = θ(n)

3·c22 + 7 · c2 + 8 > c2·c2c1 c2

Truec2

COSC 3101N

J. Elder

Definition of Theta

c c n n n c g n f n or f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( ) ( )

3n2 + 7n + 8 = θ(n)

3n2 + 7n + 8 > c2·nc1 c2 ?n0

COSC 3101N

J. Elder

Definition of Theta

c c n n n c g n f n or f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( ) ( )

3n2 + 7n + 8 = θ(n)

3·c22 + 7 · c2 + 8 > c2·c2c1 max(c2,n0 )

Truec2 n0

True

COSC 3101N

J. Elder

Definition of Theta

3n2 + 7n + 8 = g(n)θ(1)

c1, c2, n0, n n0 g(n)c1 f(n) g(n)c2

3n2 + 7n + 8 = nθ(1)

COSC 3101N

J. Elder

Definition of Theta

3n2 + 7n + 8 = nθ(1)

nc1 3n2 + 7n + 8 nc2

c1, c2, n0, n n0 g(n)c1 f(n) g(n)c2

? ?

COSC 3101N

J. Elder

Definition of Theta

3n2 + 7n + 8 = nθ(1)

n2 3n2 + 7n + 8 n3

c1, c2, n0, n n0 g(n)c1 f(n) g(n)c2

2 3

COSC 3101N

J. Elder

Definition of Theta

3n2 + 7n + 8 = nθ(1)

n2 3n2 + 7n + 8 n3

c1, c2, n0, n n0 g(n)c1 f(n) g(n)c2

2 3 ? n ?

COSC 3101N

J. Elder

Definition of Theta

3n2 + 7n + 8 = nθ(1)

n2 3n2 + 7n + 8 n3

c1, c2, n0, n n0 g(n)c1 f(n) g(n)c2

2 3 5 n 5

COSC 3101N

J. Elder

Definition of Theta

3n2 + 7n + 8 = nθ(1)

n2 3n2 + 7n + 8 n3

True

True

c1, c2, n0, n n0 g(n)c1 f(n) g(n)c2

2 3 5 n 5

COSC 3101N

J. Elder

Order of Quantifiers

n n n c c c g n f n c g n0 0 1 2 1 2, , ( ) ( ) ( ), ,

f(n) = θ(g(n))

?

c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )

COSC 3101N

J. Elder

g b loves b g, , ( , ) b g loves b g, , ( , )

Understand Quantifiers!!!

One girl Could be a separate girl for each boy.

Sam Mary

Bob Beth

JohnMarilin Monro

Fred Ann

Sam Mary

Bob Beth

JohnMarilin Monro

Fred Ann

COSC 3101N

J. Elder

Order of Quantifiers

No! It cannot be a different c1 and c2

for each n.

n n n c c c g n f n c g n0 0 1 2 1 2, , ( ) ( ) ( ), ,

f(n) = θ(g(n))

c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )

COSC 3101N

J. Elder

Other Notations

Theta f(n) = θ(g(n)) f(n) ≈ c g(n)

BigOh f(n) = O(g(n)) f(n) ≤ c g(n)

Omega f(n) = Ω(g(n)) f(n) ≥ c g(n)

Little Oh f(n) = o(g(n)) f(n) << c g(n)

Little Omega f(n) = ω(g(n)) f(n) >> c g(n)

COSC 3101N

J. Elder

BigOh Notation

• n2 = O(n3)

• n3 = O(n2)

COSC 3101N

J. Elder

BigOh Notation

• O(n2) = O(n3)

• O(n3) = O(n2)Odd Notation

f(n) = O(g(n)) Standard f(n) ≤ O(g(n)) Stresses one function dominating another.

f(n)O(g(n)) Stress function is member of class.