chapter 01tadavis/cs303/f16/ch02m.pdfgeneral rules (cont.) 40 even worse! โˆ’extremely inefficient,...

13
9/8/2016 1 Chapter 2 Algorithm Analysis Introduction 2 โˆ’ algorithm โˆ’ set of simple instructions to solve a problem โˆ’ analyzed in terms, such as time and memory, required โˆ’too long (minutes, hours, years) โ€“ no good โˆ’too much memory (terabytes) โ€“ no good โˆ’ we can estimate resource usage using a formal mathematical framework to count basic operations โˆ’ big-O (()) โˆ’ big-omega ฮฉ (()) โˆ’ big-theta ฮ˜ ( ) Asymptotic Notation 3 โˆ’ formal definitions = (()) = ฮฉ (()) = ฮ˜( ) and () are positive for >1 when we say that is true when is sufficiently large, we mean there exists such that is true for all > we are comparing relative rates of growth Asymptotic Notation: Big- 4 = (()) if there exist constants and such that โ‰ค ยท () for all > loosely speaking, = is an analog of โ‰ค example: if = 54 2 + 42 and = 2 then = () since โ‰ค 55() for all > 42 Asymptotic Notation: ฮฉ 5 = ฮฉ(()) if there exist constants and such that โ‰ฅ ยท () for all > loosely speaking, =ฮฉ is an analog of โ‰ฅ Asymptotic Notation: ฮ˜ 6 = ฮ˜(()) if there exist constants 1 , 2 , and such that 1 ยท < < 2 ยท () for all > loosely speaking, =ฮ˜ is an analog of โ‰ˆ

Upload: others

Post on 19-May-2020

6 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Chapter 01tadavis/cs303/f16/ch02m.pdfGeneral Rules (cont.) 40 even worse! โˆ’extremely inefficient, especially for values > 40 โˆ’analysis โˆ’for = 0or 1, ๐‘‡ : ;is constant: =

9/8/2016

1

Chapter 2

Algorithm Analysis

Introduction

2

โˆ’algorithm

โˆ’set of simple instructions to solve a problem

โˆ’analyzed in terms, such as time and memory, required

โˆ’too long (minutes, hours, years) โ€“ no good

โˆ’too much memory (terabytes) โ€“ no good

โˆ’we can estimate resource usage using a formal

mathematical framework to count basic operations

โˆ’big-O ๐‘‚(๐‘“(๐‘))

โˆ’big-omega ฮฉ (๐‘“(๐‘))

โˆ’big-theta ฮ˜ (๐‘“ ๐‘ )

Asymptotic Notation

3

โˆ’ formal definitions

๐‘“ ๐‘› = ๐‘‚(๐‘”(๐‘›)) ๐‘“ ๐‘› = ฮฉ (๐‘”(๐‘›)) ๐‘“ ๐‘› = ฮ˜(๐‘” ๐‘› )

๐‘“ ๐‘› and ๐‘”(๐‘›) are positive for ๐‘› > 1

when we say that ๐‘‹ is true when ๐‘› is sufficiently large, we

mean there exists ๐‘ such that ๐‘‹ is true for all ๐‘› > ๐‘

we are comparing relative rates of growth

Asymptotic Notation: Big-๐‘‚

4

๐‘“ ๐‘› = ๐‘‚(๐‘”(๐‘›)) if there exist constants ๐‘ and ๐‘ such that

๐‘“ ๐‘› โ‰ค ๐‘ ยท ๐‘”(๐‘›) for all ๐‘› > ๐‘

loosely speaking, ๐‘“ ๐‘› = ๐‘‚ ๐‘” ๐‘› is an analog of ๐‘“ โ‰ค ๐‘”

example: if ๐‘“ ๐‘› = 54๐‘›2 + 42๐‘› and ๐‘” ๐‘› = ๐‘›2 then ๐‘“ = ๐‘‚(๐‘”) since

๐‘“ ๐‘› โ‰ค 55๐‘”(๐‘›)

for all ๐‘› > 42

Asymptotic Notation: ฮฉ

5

๐‘“ ๐‘› = ฮฉ(๐‘”(๐‘›)) if there exist constants ๐‘ and ๐‘ such that

๐‘“ ๐‘› โ‰ฅ ๐‘ ยท ๐‘”(๐‘›) for all ๐‘› > ๐‘

loosely speaking, ๐‘“ ๐‘› = ฮฉ ๐‘” ๐‘› is an analog of ๐‘“ โ‰ฅ ๐‘”

Asymptotic Notation: ฮ˜

6

๐‘“ ๐‘› = ฮ˜(๐‘”(๐‘›)) if there exist constants ๐‘1, ๐‘2, and ๐‘ such

that

๐‘1 ยท ๐‘” ๐‘› < ๐‘“ ๐‘› < ๐‘2 ยท ๐‘”(๐‘›) for all ๐‘› > ๐‘

loosely speaking, ๐‘“ ๐‘› = ฮ˜ ๐‘” ๐‘› is an analog of ๐‘“ โ‰ˆ ๐‘”

Page 2: Chapter 01tadavis/cs303/f16/ch02m.pdfGeneral Rules (cont.) 40 even worse! โˆ’extremely inefficient, especially for values > 40 โˆ’analysis โˆ’for = 0or 1, ๐‘‡ : ;is constant: =

9/8/2016

2

Relationship of ๐‘‚,ฮฉ, and ฮ˜

7

โˆ’note that

๐‘“ ๐‘› = ๐‘‚ ๐‘” ๐‘› โ‡” ๐‘” ๐‘› = ฮฉ(๐‘“ ๐‘› )

since

๐‘“ ๐‘› < ๐‘๐‘” ๐‘› โ‡” ๐‘” ๐‘› >1

๐‘๐‘“ ๐‘›

โˆ’also

๐‘“ ๐‘› = ฮ˜ ๐‘” ๐‘› โ‡” ๐‘“ ๐‘› = ๐‘‚ ๐‘” ๐‘› and ๐‘“ ๐‘› = ฮฉ ๐‘” ๐‘›

since ๐‘“ ๐‘› = ฮ˜(๐‘”(๐‘›)) means that there exist ๐‘1, ๐‘2, and ๐‘

such that

๐‘1 ยท ๐‘” ๐‘› < ๐‘“ ๐‘› < ๐‘2 ยท ๐‘”(๐‘›) for all ๐‘› > ๐‘

Asymptotic Notation: First Observation

8

โˆ’ for sufficiently large ๐‘›,

๐‘“ ๐‘› = ๐‘‚(๐‘”(๐‘›)) gives an upper bound on ๐‘“ in terms of ๐‘”

๐‘“ ๐‘› = ฮฉ (๐‘”(๐‘›)) gives a lower bound on ๐‘“ in terms of ๐‘”

๐‘“ ๐‘› = ฮ˜(๐‘” ๐‘› ) gives both an upper and lower bound on ๐‘“

in terms of ๐‘”

Asymptotic Notation: Second Observation

9

โˆ’ it is the existence of constants that matters for the purposes of asymptotic complexity analysis โ€“ not the exact values of the constants

โˆ’example: all of the following imply ๐‘›2 + ๐‘› = ๐‘‚(๐‘›2):

๐‘›2 + ๐‘› < 4๐‘›2 for all ๐‘› > 1

๐‘›2 + ๐‘› < 2๐‘›2 for all ๐‘› > 1

๐‘›2 + ๐‘› < 1.01๐‘›2 for all ๐‘› > 100

๐‘›2 + ๐‘› < 1.000001๐‘›2 for all ๐‘› > 1000

Asymptotic Notation: Third Observation

10

โˆ’we typically ignore constant factors and lower-order terms, since all we want to capture is growth trends

โˆ’example:

๐‘“ ๐‘› = 1,000๐‘›2 + 10,000๐‘› + 42

โ‡’ ๐‘“ ๐‘› = ๐‘‚(๐‘›2)

but the following are discouraged:

๐‘“ ๐‘› = ๐‘‚(1,000๐‘›2) true, but bad form

๐‘“ ๐‘› = ๐‘‚(๐‘›2 + ๐‘›) true, but bad form

๐‘“ ๐‘› < ๐‘‚(๐‘›2) < built into ๐‘‚

Asymptotic Notation: Fourth Observation

11

๐‘‚(๐‘”(๐‘›)) assures that you canโ€™t do any worse than ๐‘”(๐‘›), but

it can unduly pessimistic

ฮฉ (๐‘”(๐‘›)) assures that you canโ€™t do any better than ๐‘”(๐‘›), but

it can unduly optimistic

ฮ˜(๐‘” ๐‘› ) is the strongest statement: you canโ€™t do any better

than ๐‘”(๐‘›), but you canโ€™t do any worse

A Shortcoming of ๐‘‚

12

an ๐‘“ ๐‘› = ๐‘‚(๐‘”(๐‘›)) bound can be misleading

example: ๐‘› = ๐‘‚(๐‘›2), since ๐‘› < ๐‘›2 for all ๐‘› > 1

however, for large ๐‘›, ๐‘› โ‰ช ๐‘›2 โ€’ the functions ๐‘“ ๐‘› = ๐‘› and

๐‘” ๐‘› = ๐‘›2 are nothing alike for large ๐‘›

we donโ€™t write something like ๐‘“ ๐‘› = ๐‘‚(๐‘›2) if we know, say,

that ๐‘“ ๐‘› = ๐‘‚(๐‘›)

upper bounds should be as small as possible!

Page 3: Chapter 01tadavis/cs303/f16/ch02m.pdfGeneral Rules (cont.) 40 even worse! โˆ’extremely inefficient, especially for values > 40 โˆ’analysis โˆ’for = 0or 1, ๐‘‡ : ;is constant: =

9/8/2016

3

A Shortcoming of ฮฉ

13

an ๐‘“ ๐‘› = ฮฉ(๐‘”(๐‘›)) bound can be misleading

example: ๐‘›2 = ฮฉ(๐‘›), since ๐‘›2 > ๐‘› for all ๐‘› > 1

however, for large ๐‘›, ๐‘›2 โ‰ซ ๐‘› โ€’ the functions ๐‘“ ๐‘› = ๐‘› and

๐‘” ๐‘› = ๐‘›2 are nothing alike for large ๐‘›

we donโ€™t write something like ๐‘“ ๐‘› = ฮฉ(๐‘›) if we know, say,

that ๐‘“ ๐‘› = ฮฉ(๐‘›2)

lower bounds should be as large as possible!

ฮ˜ is the Most Informative

14

a ๐‘“ ๐‘› = ฮ˜(๐‘”(๐‘›)) relationship is the most informative:

๐‘1 ยท ๐‘” ๐‘› < ๐‘“ ๐‘› < ๐‘2 ยท ๐‘”(๐‘›) for all ๐‘› > ๐‘

example: 2๐‘›2 + ๐‘› = ฮ˜(๐‘›2)

2๐‘›2 < 2๐‘›2 + ๐‘› < 3๐‘›2

for all ๐‘› > 1

for large values of ๐‘›, the ratio (2๐‘›2 + ๐‘›)/๐‘›2 tends to 2;

in this sense,

2๐‘›2 + ๐‘› โ‰ˆ 2๐‘›2

Typical Growth Rates

15

https://expcode.wordpress.com/2015/07/19/big-o-big-omega-and-big-theta-notation/

Additional Rules

16

Rule 1

if ๐‘‡1 ๐‘› = ๐‘‚(๐‘“(๐‘›)) and ๐‘‡2 ๐‘› = ๐‘‚(๐‘”(๐‘›)) then

(a) ๐‘‡1 ๐‘› + ๐‘‡2 ๐‘› = ๐‘‚(๐‘“ ๐‘› + ๐‘”(๐‘›)) (or just

๐‘‚(max (๐‘“ ๐‘› , ๐‘”(๐‘›)))

(b) ๐‘‡1 ๐‘› โˆ— ๐‘‡2 ๐‘› = ๐‘‚(๐‘“ ๐‘› โˆ— ๐‘”(๐‘›))

Rule 2

if ๐‘‡(๐‘›) is a polynomial of degree ๐‘˜, then ๐‘‡ ๐‘› = ฮ˜(๐‘›๐‘˜)

Rule 3

logk๐‘› = ๐‘‚ ๐‘› for any constant ๐‘˜ (logarithms grow very slowly)

Typical Growth Rates

17

Function Name

๐‘ constant

log ๐‘› logarithmic

๐‘› linear

๐‘› log ๐‘›

๐‘›2 quadratic

๐‘›3 cubic

2๐‘› exponential

Typical Growth Rates

18

http://www.cosc.canterbury.ac.nz/cs4hs/2015/files/complexity-tractability-julie-mcmahon.pdf

Page 4: Chapter 01tadavis/cs303/f16/ch02m.pdfGeneral Rules (cont.) 40 even worse! โˆ’extremely inefficient, especially for values > 40 โˆ’analysis โˆ’for = 0or 1, ๐‘‡ : ;is constant: =

9/8/2016

4

Complexity and Tractability

19

assume computer speed of 1 billion ops/sec

T(n)

n n n log n n2

n3

n4

n10

2n

10 .01ms .03ms .1ms 1ms 10ms 10s 1ms

20 .02ms .09ms .4ms 8ms 160ms 2.84h 1ms

30 .03ms .15ms .9ms 27ms 810ms 6.83d 1s

40 .04ms .21ms 1.6ms 64ms 2.56ms 121d 18m

50 .05ms .28ms 2.5ms 125ms 6.25ms 3.1y 13d

100 .1ms .66ms 10ms 1ms 100ms 3171y 4ยด1013

y

103

1ms 9.96ms 1ms 1s 16.67m 3.17ยด1013

y 32ยด10283

y

104

10ms 130ms 100ms 16.67m 115.7d 3.17ยด1023

y

105

100ms 1.66ms 10s 11.57d 3171y 3.17ยด1033

y

106

1ms 19.92ms 16.67m 31.71y 3.17ยด107y 3.17ยด10

43y

What to Analyze

20

in order to analyze algorithms, we need a model of

computation

โˆ’standard computer with sequential instructions

โˆ’each operation (+, -, *, /, =, etc.) takes one time unit, even

memory accesses

โˆ’ infinite memory

โˆ’obviously unrealistic

Complex Arithmetic Operations

21

โˆ’examples: ๐‘ฅ๐‘› log2 ๐‘ฅ ๐‘ฅ

โˆ’count the number of basic operations required to execute

the complex arithmetic operations

โˆ’ for instance, we could compute ๐‘ฅ๐‘› as

๐‘ฅ๐‘› โ† ๐‘ฅ โˆ— ๐‘ฅ โˆ— โ‹ฏโˆ— ๐‘ฅ โˆ— ๐‘ฅ

there are ๐‘› โˆ’ 1 multiplications, so there are ๐‘› โˆ’ 1 basic

operations to compute ๐‘ฅ๐‘› plus one more basic operation

to assign or return the result

What to Analyze

22

different types of performance are analyzed

โˆ’best case

โˆ’may not represent typical behavior

โˆ’average case

โˆ’often reflects typical behavior

โˆ’difficult to determine what is average

โˆ’difficult to compute

โˆ’worst case

โˆ’guarantee on performance for any input

โˆ’typically considered most important

โˆ’should be implementation-independent

Maximum Subsequence Sum Problem

23

example: Maximum Subsequence Sum Problem

โˆ’given (possibly negative) integers, A1, A2, โ€ฆ, AN, find the

maximum value of

โˆ’e.g, for input -2, 11, -4, 13, -5, -2, the answer is 20 (A2

through A4)

j

ik

kA

Maximum Subsequence Sum Problem

24

many algorithms to solve this problem

โˆ’we will focus on four with run time in the table below

Page 5: Chapter 01tadavis/cs303/f16/ch02m.pdfGeneral Rules (cont.) 40 even worse! โˆ’extremely inefficient, especially for values > 40 โˆ’analysis โˆ’for = 0or 1, ๐‘‡ : ;is constant: =

9/8/2016

5

Maximum Subsequence Sum Problem

25

notes

โˆ’computer actually run on not important since weโ€™re looking

to compare algorithms

โˆ’ for small inputs, all algorithms perform well

โˆ’ times do not include read time

โˆ’algorithm 4: time to read would surpass time to compute

โˆ’reading data is often the bottleneck

โˆ’ for algorithm 4, the run time increases by a factor of 10 as

the problem size increases by a factor of 10 (linear)

โˆ’algorithm 2 is quadratic; therefore an increase of input by

a factor of 10 yields a run time factor of 100

โˆ’algorithm 1 (cubic): run time factor of 1000

Maximum Subsequence Sum Problem

26

Maximum Subsequence Sum Problem

27

Running Time Calculations

28

at least two ways to estimate the running time of a program

โˆ’empirically

โˆ’as in previous results

โˆ’realistic measurement

โˆ’analysis

โˆ’helps to eliminate bad algorithms early when several

algorithms are being considered

โˆ’analysis itself provides insight into designing efficient

algorithms

โˆ’pinpoints bottlenecks, which need special coding

A Simple Example

29

calculate ๐‘–3๐‘๐‘–=1

A Simple Example

30

analysis

โˆ’declarations count for no time; ignore call and return costs

โˆ’ lines 1 and 4 count for 1 time unit each

โˆ’ line 3 counts for 4 time units

โˆ’2 multiplications

โˆ’1 addition

โˆ’1 assignment

โˆ’ line 2 has hidden costs

โˆ’ initializing i (1 time unit)

โˆ’testing i โ‰ค n (๐‘ + 1)

โˆ’ incrementing i (๐‘)

โˆ’ total: 6๐‘ + 4 or ๐‘‚(๐‘)

Page 6: Chapter 01tadavis/cs303/f16/ch02m.pdfGeneral Rules (cont.) 40 even worse! โˆ’extremely inefficient, especially for values > 40 โˆ’analysis โˆ’for = 0or 1, ๐‘‡ : ;is constant: =

9/8/2016

6

A Simple Example

31

analysis

โˆ’ too much work to analyze all programs like this

โˆ’since weโ€™re dealing with ๐‘‚, we can use shortcuts to help

โˆ’ line 3 counts for 4 time units, but is ๐‘‚(1)

โˆ’ line 1 is insignificant compared to the for loop

General Rules

32

concentrate on loops and recursive calls

โˆ’Rule 1 โ€“ FOR loops

โˆ’running time of the statements inside (including tests)

times the number of iterations

โˆ’Rule 2 โ€“ Nested loops

โˆ’analyze from the inside out

โˆ’running time of the statements times the product of the

sizes of all the loops

โˆ’watch out for loops that

โˆ’contribute only a constant amount of computation

โˆ’contain break statements

Single Loops

33

for (int i = 0; i < n; i++)

a[i]+= i;

for (int i = 5; i < n; i++)

a[i]+= i;

for (int i = 0; i < n; i++)

{

if (i >= 7) break;

a[i]+= i;

}

Nested Loops

34

for (int i = 0; i < n; ++i)

for (int j = 0; j < n; ++j)

++k;

โˆ’constant amount of computation inside nested loop

โˆ’must multiply number of times both loops execute

โˆ’๐‘‚(๐‘›2)

for (int i = 0; i < n; ++i)

for (int j = i; j < n; ++j)

++k;

โˆ’even though second loop is not executed as often, still

๐‘‚(๐‘›2)

Nested Loops

35

for (int i = 0; i < 7; ++i)

for (int j = 0; j < n; ++j)

++k;

โˆ’outer loop is executed a fixed number of times

โˆ’๐‘‚(๐‘›)

General Rules (cont.)

36

concentrate on loops and recursive calls

โˆ’Rule 3 โ€“ Consecutive statements

โˆ’simply add the running times

for (i = 0; i < n; ++i)

a[i] = 0

for (i = 0; i < n; ++i)

for (j = 0; j < n; ++j)

a[i] += a[j] + i + j

โˆ’๐‘‚(๐‘›) work followed by ๐‘‚(๐‘›2) work, so ๐‘‚(๐‘›2)

Page 7: Chapter 01tadavis/cs303/f16/ch02m.pdfGeneral Rules (cont.) 40 even worse! โˆ’extremely inefficient, especially for values > 40 โˆ’analysis โˆ’for = 0or 1, ๐‘‡ : ;is constant: =

9/8/2016

7

General Rules (cont.)

37

concentrate on loops and recursive calls

โˆ’Rule 4 โ€“ if/else

if (test)

S1

else

S2

โˆ’total running time includes the test plus the larger of the

running times of S1 and S2

โˆ’could be overestimate in some cases, but never an

underestimate

General Rules (cont.)

38

concentrate on loops and recursive calls

โˆ’ function calls analyzed first

โˆ’recursion

โˆ’ if really just a for loop, not difficult (the following is ๐‘‚(๐‘›))

long factorial (int n)

{

if (n <= 1)

return 1;

else

return n * factorial (n โ€“ 1);

}

General Rules (cont.)

39

previous example not a good use of recursion

โˆ’ if recursion used properly, difficult to convert to a simple

loop structure

what about this example?

long fib (int n)

{

if (n <= 1)

return 1;

else

return fib(n โ€“ 1) + fib (n โ€“ 2);

}

General Rules (cont.)

40

even worse!

โˆ’extremely inefficient, especially for values > 40

โˆ’analysis

โˆ’for ๐‘ = 0 or ๐‘ = 1, ๐‘‡(๐‘) is constant: ๐‘‡(0) = ๐‘‡(1) = 1

โˆ’since the line with recursive calls is not a simple

operation, it must be handled differently

๐‘‡ ๐‘ = ๐‘‡ ๐‘ โˆ’ 1 + ๐‘‡ ๐‘ โˆ’ 2 + 2 for ๐‘ โ‰ฅ 2

โˆ’we have seen that this algorithm <5

3

๐‘

โˆ’similarly, we could show it is >3

2

๐‘

โˆ’exponential!

General Rules (cont.)

41

with current recursion

โˆ’ lots of redundant work

โˆ’violating compound interest rule

โˆ’ intermediate results thrown away

running time can be reduced substantially with simple for loop

Maximum Subsequence Sum Problem Revisited

42

example: Maximum Subsequence Sum Problem

โˆ’given (possibly negative) integers, A1, A2, โ€ฆ, AN, find the

maximum value of

โˆ’e.g, for input -2, 11, -4, 13, -5, -2, the answer is 20 (A2

through A4)

โˆ’previously, we reviewed running time results for four

algorithms

j

ik

kA

Page 8: Chapter 01tadavis/cs303/f16/ch02m.pdfGeneral Rules (cont.) 40 even worse! โˆ’extremely inefficient, especially for values > 40 โˆ’analysis โˆ’for = 0or 1, ๐‘‡ : ;is constant: =

9/8/2016

8

Maximum Subsequence Sum Problem Revisited

43

many algorithms to solve this problem

โˆ’we will focus on four with run time in the table below

Maximum Subsequence Sum Problem Revisited

44

Algorithm 1

45

Algorithm 1

46

exhaustively tries all possibilities

โˆ’ first loop iterates ๐‘ times

โˆ’second loop iterates ๐‘ โˆ’ ๐‘– times, which could be small,

but we must assume the worst

โˆ’ third loop iterates ๐‘— โˆ’ ๐‘– + 1 times, which we must also

assume to be size ๐‘

โˆ’ total work on line 14 is ๐‘‚(1), but itโ€™s inside these nested

loops

โˆ’ total running time is therefore ๐‘‚ 1 โˆ™ ๐‘ โˆ™ ๐‘ โˆ™ ๐‘ = ๐‘‚(๐‘3)

โˆ’what about lines 16-17?

Algorithm 1

47

more precise calculation

โˆ’use better bounds on loops to compute how many times

line 14 is calculated

โˆ’compute from the inside out

โˆ’ then

Algorithm 1

48

more precise calculation (cont.)

โˆ’ to complete the calculation

Page 9: Chapter 01tadavis/cs303/f16/ch02m.pdfGeneral Rules (cont.) 40 even worse! โˆ’extremely inefficient, especially for values > 40 โˆ’analysis โˆ’for = 0or 1, ๐‘‡ : ;is constant: =

9/8/2016

9

Algorithm 2

49

to speed up the algorithm, we can remove a for loop

โˆ’unnecessary calculation removed

โˆ’note that

โˆ’new algorithm is ๐‘‚(๐‘›2)

Algorithm 2

50

Algorithm 3

51

recursive algorithm runs even faster

โˆ’divide and conquer strategy

โˆ’divide: split problem into two roughly equal subproblems

โˆ’conquer: merge solutions to subproblems

โˆ’maximum subsequence can be in one of three places

โˆ’entirely in left half of input

โˆ’entirely in right half of input

โˆ’ in both halves, crossing the middle

โˆ’ first two cases solved recursively

โˆ’ last case solved by finding the largest sum in the first half that includes the last element of the first half, plus the largest sum in the second half that includes the first element in the second half

Algorithm 3

52

consider the following example

โˆ’maximum subsequence for the first half: 6

โˆ’maximum subsequence for the second half: 8

โˆ’maximum subsequence that crosses the middle

โˆ’maximum subsequence in the first half that includes the

last element of the first half: 4

โˆ’maximum subsequence in the second half that includes

the first element in the second half: 7

โˆ’total: 11

Algorithm 3

53

Algorithm 3 (cont.)

54

Page 10: Chapter 01tadavis/cs303/f16/ch02m.pdfGeneral Rules (cont.) 40 even worse! โˆ’extremely inefficient, especially for values > 40 โˆ’analysis โˆ’for = 0or 1, ๐‘‡ : ;is constant: =

9/8/2016

10

Algorithm 3

55

notes

โˆ’driver function needed

โˆ’gets solution started by calling function with initial

parameters that delimit entire array

โˆ’ if left == right

โˆ’base case

โˆ’recursive calls to divide the list

โˆ’working toward base case

โˆ’ lines 18-24 and 26-32 calculate max sums that touch the

center

โˆ’max3 returns the largest of three values

Algorithm 3

56

analysis

โˆ’ let ๐‘‡(๐‘) be the time it takes to solve a maximum

subsequence problem of size ๐‘

โˆ’ if ๐‘ = 1, program takes some constant time to execute

lines 8-12; thus ๐‘‡ 1 = 1

โˆ’otherwise, the program must perform two recursive calls

โˆ’ the two for loops access each element in the subarray,

with constant work; thus ๐‘‚(๐‘)

โˆ’all other non-recursive work in the function is constant

and can be ignored

โˆ’recursive calls divide the list into two ๐‘/2 subproblems if

๐‘ is even (and more strongly, a power of 2)

โˆ’time: 2๐‘‡(๐‘/2)

Algorithm 3

57

total time for the algorithm is therefore

๐‘‡ ๐‘ = 2๐‘‡๐‘

2+๐‘ where ๐‘‡ 1 = 1

โˆ’note that

๐‘‡ ๐‘› = ๐‘‡ ๐‘› โˆ’ 1 + ๐‘ is ๐‘‡ ๐‘› = ๐‘๐‘› or just ๐‘‚(๐‘›)

and

๐‘‡ ๐‘› = ๐‘‡ ๐‘› โˆ’ 1 + ๐‘๐‘› is ๐‘‡ ๐‘› = ๐‘๐‘›(๐‘› + 1)/2

or just ๐‘‚(๐‘›2)

Algorithm 3

58

we can solve the recurrence relation directly (later)

๐‘‡ ๐‘ = 2๐‘‡๐‘

2+ ๐‘ where ๐‘‡ 1 = 1

โˆ’ for now, note that

๐‘‡ 1 = 1

๐‘‡ 2 = 4 = 2 โˆ— 2 ๐‘‡ 4 = 12 = 4 โˆ— 3

๐‘‡ 8 = 32 = 8 โˆ— 4 ๐‘‡ 16 = 80 = 16 โˆ— 5

thus, if ๐‘ = 2๐‘˜ ,

๐‘‡ ๐‘ = ๐‘ โˆ— ๐‘˜ + 1 = ๐‘ log๐‘ +๐‘ = ๐‘‚(๐‘ log๐‘)

Algorithm 4

59

linear algorithm to solve the maximum subsequence sum

โˆ’observation: we never want a negative sum

โˆ’as before, remember the best sum weโ€™ve encountered so

far, but add a running sum

โˆ’ if the running sum becomes negative, reset the starting

point to the first positive element

Algorithm 4

60

Page 11: Chapter 01tadavis/cs303/f16/ch02m.pdfGeneral Rules (cont.) 40 even worse! โˆ’extremely inefficient, especially for values > 40 โˆ’analysis โˆ’for = 0or 1, ๐‘‡ : ;is constant: =

9/8/2016

11

Algorithm 4

61

notes

โˆ’ if a[i] is negative, it cannot possibly be the first element of

a maximum sum, since any such subsequence would be

improved by beginning with a[i+1]

โˆ’similarly, any negative subsequence cannot possible be a

prefix for an optimal subsequence

โˆ’this allows us to advance through the original sequence

rapidly

โˆ’correctness may be difficult to determine

โˆ’can be tested on small input by running against more

inefficient brute-force programs

โˆ’additional advantage: data need only be read once (online

algorithm) โ€“ extremely advantageous

Logarithms in the Running Time

62

logarithms will show up regularly in analyzing algorithms

โˆ’we have already seen them in divide and conquer

strategies

โˆ’general rule: an algorithm is ๐‘‚(๐‘ log๐‘) if it takes constant

๐‘‚(1) time to cut the problem size by a fraction (typically 1

2)

โˆ’ if constant time is required to merely reduce the problem

by a constant amount (say, smaller by 1), then the

algorithm is ๐‘‚(๐‘)

โˆ’only certain problems fall into the ๐‘‚(log๐‘) category

since it would take ฮฉ(๐‘) just to read in the input

โˆ’for such problems, we assume the input is pre-read

Binary Search

63

binary search

โˆ’given an integer X and integers A0, A1, โ€ฆ, AN-1, which are

presorted and already in memory, find I such that Ai = X, or

return i = -1 if X is not in the input

โˆ’obvious solution: scan through the list from left to right to

find X

โˆ’runs in linear time

โˆ’does not take into account that elements are presorted

โˆ’better solution: check if X is in the middle element

โˆ’ if yes, done

โˆ’ if smaller, apply same strategy to sorted left subarray

โˆ’ if greater, use right subarray

Binary Search

64

Binary Search

65

analysis

โˆ’all the work done inside the loop takes ๐‘‚(1) time per

iteration

โˆ’ first iteration of the loop is for ๐‘ โˆ’ 1

โˆ’subsequent iterations of the loop halve this amount

โˆ’total running time is therefore ๐‘‚(log๐‘)

โˆ’another example using sorted data for fast lookup

โˆ’periodic table of elements

โˆ’118 elements

โˆ’at most 8 accesses would be required

Euclidโ€™s Algorithm

66

Euclidโ€™s algorithm

โˆ’computes gcd (greatest common divisor) of two integers

โˆ’ largest integer that divides both

Page 12: Chapter 01tadavis/cs303/f16/ch02m.pdfGeneral Rules (cont.) 40 even worse! โˆ’extremely inefficient, especially for values > 40 โˆ’analysis โˆ’for = 0or 1, ๐‘‡ : ;is constant: =

9/8/2016

12

Euclidโ€™s Algorithm

67

Euclidโ€™s Algorithm

68

notes

โˆ’algorithm computes gcd (M, N) assuming ๐‘€ โ‰ฅ ๐‘

โˆ’ if ๐‘ > ๐‘€, the values are swapped

โˆ’algorithm works by continually computing remainders until

0 is reached

โˆ’ the last non-0 remainder is the answer

โˆ’ for example, if ๐‘€ = 1,989 and ๐‘ = 1,590, the sequence of

remainders is 399, 393, 6, 3, 0

โˆ’therefore gcd = 3

โˆ’good, fast algorithm

Euclidโ€™s Algorithm

69

analysis

โˆ’need to determine how many remainders will be

computed

โˆ’log๐‘ is a good guess, but as can be seen from the

example, the values do not decrease in a uniform way

โˆ’we can prove that the remainder is at most half of its

original value after two iterations

โˆ’this would show that the number of iterations is at most

2log๐‘ = ๐‘‚(log๐‘)

Euclidโ€™s Algorithm

70

analysis (cont.)

โˆ’Show: if ๐‘€ > ๐‘, then ๐‘ < ๐‘€/2

โˆ’Proof: 2 cases

โˆ’ if ๐‘ โ‰ค ๐‘€/2, then since the remainder is smaller than ๐‘,

the theorem is true

โˆ’ if ๐‘ > ๐‘€/2, then ๐‘ goes into ๐‘€ once with a remainder

of ๐‘€โˆ’๐‘ < ๐‘€/2, and the theorem is true

โˆ’ in our example, 2log๐‘ is about 20, but we needed only 7

operations

โˆ’the constant can be refined to 1.44log๐‘

โˆ’the average case (complicated!) is

(12 ln2 ln๐‘)/๐œ‹2 +1.47

Exponentiation

71

exponentiation

โˆ’raising an integer to an integer power

โˆ’results are often large, so machine must be able to

support such values

โˆ’number of multiplications is the measurement of work

โˆ’obvious algorithm: for ๐‘‹๐‘ uses ๐‘ โˆ’ 1 multiplications

โˆ’recursive algorithm: better

โˆ’๐‘ โ‰ค 1 base case

โˆ’otherwise, if ๐‘ is even, ๐‘‹๐‘ = ๐‘‹๐‘/2 โˆ™ ๐‘‹๐‘/2

โˆ’ if ๐‘ is odd, ๐‘‹๐‘ = ๐‘‹(๐‘โˆ’1)/2 โˆ™ ๐‘‹(๐‘โˆ’1)/2 โˆ™ ๐‘‹

โˆ’example: ๐‘‹62 = ๐‘‹31 2

๐‘‹3 = ๐‘‹2 ๐‘‹,๐‘‹7 = ๐‘‹3 2๐‘‹, ๐‘‹15 = ๐‘‹7 2๐‘‹, ๐‘‹31 = ๐‘‹15 2๐‘‹

Exponentiation

72

Page 13: Chapter 01tadavis/cs303/f16/ch02m.pdfGeneral Rules (cont.) 40 even worse! โˆ’extremely inefficient, especially for values > 40 โˆ’analysis โˆ’for = 0or 1, ๐‘‡ : ;is constant: =

9/8/2016

13

Exponentiation

73

analysis

โˆ’number of multiplications: 2log๐‘

โˆ’at most, 2 multiplications are required to halve the

problem (if ๐‘ is odd)

โˆ’some additional modifications can be made, but care must

be taken to avoid errors and inefficiencies

Limitations of Worst-Case Analysis

74

analysis can be shown empirically to be an overestimate

โˆ’analysis can be tightened

โˆ’average running time may be significantly less than worst-

case running time

โˆ’often, the worst-case is achievable by very bad input,

but still a large overestimate in the average case

โˆ’unfortunately, average-case analysis is extremely

complex (and in many cases currently unsolved), so

worst-case is the best that weโ€™ve got