cse 326 asymptotic analysis david kaplan dept of computer science & engineering autumn 2001

CSE 326Asymptotic Analysis

David Kaplan

Dept of Computer Science & EngineeringAutumn 2001

Asymptotic Analysis

CSE 326 Autumn 20012

Housekeeping Homework 1 status Join cse326@cs Poll:

Slide format(s)? Office hours today?

Asymptotic Analysis


Analyzing AlgorithmsAnalyze algorithms to gauge:

Time complexity (running time) Space complexity (memory use)

Input size is indicated by a number n sometimes have multiple inputs, e.g. m

and n

Running time is a function of nn, n2, n log n, 18 + 3n(log n2) + 5n3

Asymptotic Analysis


RAM ModelRAM (random access machine)

Ideal single-processor machine (serialized operations)

“Standard” instruction set (load, add, store, etc.)

All operations take 1 time unit (including, for our purposes, each C++ statement)

Asymptotic Analysis


Order Notation (aka Big-O)

Big-O

T(n) = O( f(n) )Exist positive constants c, n0 such that

T(n) cf(n) for all n n0

Upper bound

Omega

T(n) = ( f(n) )Exist positive constants c, n0 such that

T(n) cf(n) for all n n0

Lower bound

ThetaT(n) = θ( f(n) )T(n) = O(f(n)) AND T(n) = (f(n))

Tight bound

little-oT(n) = o( f(n) )T(n) = O(f(n)) AND T(n) != θ(f(n))

Strict upper bound

Asymptotic Analysis


Simplifying with Big-OBy definition, Big-O allows us to:

Eliminate low order terms 4n + 5 4n 0.5 n log n - 2n + 7 0.5 n log n

Eliminate constant coefficients 4n n 0.5 n log n n log n log n2 = 2 log n log n log3 n = (log3 2) log n log n

But when might constants or low-order terms matter?

Asymptotic Analysis


Big-O Examplesn2 + 100 n = O(n2)

follows from … ( n2 + 100 n ) 2 n2 for n 10

n2 + 100 n = (n2)follows from …( n2 + 100 n ) 1 n2 for n 0

n2 + 100 n = (n2)by definition

n log n = O(n2)n log n = (n log n)n log n = (n)

Asymptotic Analysis


Big-O UsageOrder notation is not symmetric:

we can say 2n2 + 4n = O(n2) … but never O(n2) = 2n2 + 4nRight-hand side is a crudification of the left

Order expressions on left can produce unusual-looking, but true, statements:

O(n2) = O(n3)(n3) = (n2)

Asymptotic Analysis


Big-O Comparisons

Function A

n3 + 2n2

n0.1

n + 100n0.1

5n5

n-152n/100

82log n

Function #2

100n2 + 1000

log n

2n + 10 log n

n!

1000n15

3n7 + 7n

vs.

Asymptotic Analysis


Race 1100n2 + 1000vs.n3 + 2n2

Asymptotic Analysis


Race 2n0.1 log nvs.

In this one, crossover point is very late! So, which algorithm is really better???

Asymptotic Analysis


Race Cn + 100n0.1 2n + 10 log nvs.

Is the “better” algorithm asymptotically better???

Asymptotic Analysis


Race 45n5 n!vs.

Asymptotic Analysis


Race 5n-152n/100 1000n15vs.

Asymptotic Analysis


Race VI82log(n) 3n7 + 7nvs.

Asymptotic Analysis


Big-O Winners (i.e. losers)

Function A

n3 + 2n2

n0.1

n + 100n0.1

5n5

n-152n/100

82log n

Function #2

100n2 + 1000

log n

2n + 10 log n

n!

1000n15

3n7 + 7n

vs.

Winner

O(n2)

O(log n)

O(n) TIE

O(n5)

O(n15)

O(n6) why???

Asymptotic Analysis


Big-O Common Namesconstant: O(1)logarithmic: O(log n) linear: O(n)log-linear: O(n log n)superlinear: O(n1+c) (c is a constant

> 0)quadratic: O(n2)polynomial: O(nk) (k is a constant)exponential: O(cn) (c is a constant

> 1)

Asymptotic Analysis


Kinds of AnalysisRunning time may depend on actual input,

not just length of input

Distinguish Worst case

Your worst enemy is choosing input Average case

Assume probability distribution of inputs Amortized

Average time over many runs Best case (not too useful)

Asymptotic Analysis


Analyzing CodeC++ operations

Consecutive stmts

ConditionalsLoops

Function callsRecursive functions

constant timesum of timeslarger branch plus

testsum of iterationscost of function bodysolve recursive

equation

Asymptotic Analysis


Nested Loops

2

11

1

1 nnn

i

n

j

n

i

for i = 1 to n do

for j = 1 to n do

sum = sum + 1

Asymptotic Analysis


Dependent Nested Loops

for i = 1 to n do

for j = i to n do

sum = sum + 1

ininn

i

n

i

n

i

n

ij

n

i 1111

)1()1(1

2

2

)1(

2

)1()1( n

nnnnnn

Asymptotic Analysis


Recursion A recursive procedure can often be

analyzed by solving a recursive equation

Basic form:T(n) =

base case: some constantrecursive case: T(subproblems) + T(combine)

Result depends upon how many subproblems how much smaller are subproblems how costly to combine solutions

(coefficients)

Asymptotic Analysis


Sum of QueueSumQueue(Q) if (Q.length == 0 ) return 0 else return Q.dequeue() + SumQueue(Q)

One subproblem

Linear reduction in size (decrease by 1)

Combining: constant (cost of 1 add)

T(0) b

T(n) c + T(n – 1) for n>0

Asymptotic Analysis


Sum of Queue Solution

T(n) c + c + T(n-2) c + c + c + T(n-3) kc + T(n-k) for all k nc + T(0) for k=n cn + b = O(n)

T(0) b

T(n) c + T(n – 1) for n>0

Equation:

Solution:

Asymptotic Analysis


Binary SearchBinarySearch(A, x) Search A, a sorted array, for item x

One subproblem, half as large

7 12 30 35 75 83 87 90 97 99

T(1) b

T(n) T(n/2) + c for n>1

Equation:

Asymptotic Analysis


Binary Search: Solution

T(n) T(n/2) + c T(n/4) + c + c T(n/8) + c + c + c T(n/2k) + kc T(1) + c log n where k = log n b + c log n = O(log n)

T(1) b

T(n) T(n/2) + c for n>1

Equation:

Solution:

cse 326 asymptotic analysis david kaplan dept of computer science & engineering autumn 2001

Documents