Chapter 10Algorithm Analysis

Introduction Generalizing Running Time Doing a Timing Analysis Big-Oh Notation Analyzing Some Simple Programs – no Subprogram calls Best-case, Worst-Case and Average Case Analysis Analyzing Programs Exercise

Outline:

Algorithms

We only analyze correct algorithms An algorithm is correct

◦ If, for every input instance, it halts with the correct output Incorrect algorithms

◦ Might not halt at all on some input instances◦ Might halt with other than the desired answer

Analyzing an algorithm◦ Predicting the resources that the algorithm requires◦ Resources include

Memory Communication bandwidth Computational time (usually most important)

Algorithm Analysis

Factors affecting the running time◦ computer ◦ compiler◦ algorithm used◦ input to the algorithm

The content of the input affects the running time typically, the input size (number of items in the input) is the main

consideration E.g. sorting problem the number of items to be sorted E.g. multiply two matrices together the total number of

elements in the two matrices Machine model assumed

◦ Instructions are executed one after another, with no concurrent operations Not parallel computers

Algorithm Analysis (con’t)

InputSize: n

(1) log n n n log n n²

n³ 2ⁿ

5 1 3 5 15 25 125 32

10 1 4 10 33 100 10³ 10³

100 1 7 100 664 104 106 1030

1000 1 10 1000 104 106 109 10300

10000 1 13 10000 105 108 1012 103000

Generalizing Running Time

Comparing the growth of the running time as the input grows to the growth of known functions.

Analyzing Running Time: Example 1

1. n = read input from user2. sum = 03. i = 04. while i < n 5. number = read input from user6. sum = sum + number7. i = i + 18. mean = sum / n

T(n), or the running time of a particular algorithm on input of size n, is taken to be the number of times the instructions in the algorithm are executed. Pseudo code algorithm illustrates the calculation of the mean (average) of a set of n numbers:

Statement Number of times executed1 12 13 14 n+15 n6 n7 n8 1

The computing time for this algorithm in terms on input size n is: T(n) = 4n + 5.

Analyzing Running Time: Example 2 Calculate

Lines 1 and 4 count for one unit each Line 3: executed N times, each time four units Line 2: (1 for initialization, N+1 for all the tests,

N for all the increments) total 2N + 2 total cost: 6N + 4 O(N)

N

i

i1

3

1

2

3

4

1

2 +2N

4N

1

Worst/Best/Average - Case

Big O Notation : Introduction

• Big O notation is used in Computer Science to describe the performance or complexity of an algorithm.

• Big O specifically describes the worst-case scenario, and can be used to describe the execution time required or the space used (e.g. in memory or on disk) by an algorithm.

Big O Notation : Example O(1)

O(1)

O(1) describes an algorithm that will always execute in the same time (or space) regardless of the size of the input data set.

Big O Notation : Example O(N)

O(N)

O(N) describes an algorithm whose performance will grow linearly and in direct proportion to the size of the input data set.

Big O Notation : Example O(N2)O(N2)

O(N2) represents an algorithm whose performance is directly proportional to the square of the size of the input data set. This is common with algorithms that involve nested iterations over the data set. Deeper nested iterations will result in O(N3), O(N4) etc.

Big O Notation : Example O(2N)

O(2N)

O(2N) denotes an algorithm whose growth will double with each additional element in the input data set. The execution time of an O(2N) function will quickly become very large.

Algorithm Analysis: Example 1

Suppose f(n) = n2 + 3n - 1. We want to show that f(n) = O(n2). f(n) = n2 + 3n - 1

< n2 + 3n (subtraction makes things smaller so drop it) <= n2 + 3n2 (since n <= n2 for all integers n)

= 4n2

Therefore, if C = 4,

we have shown that f(n) = O(n2).

Algorithm Analysis: Example 2

Show: f(n) = 2n7 + 6n5 + 10n2 – 5

We want to show that f(n) = O(n7).

f(n) < 2n7 + 6n5 + 10n2

<= 2n7 + 6n7 + 10n7

= 18n7

thus, with C = 18 and we have shown that f(n) = O(n7)

Exercise 1:Find the Big O for this equation? T(n) = (n) + 4(n-1) + n(n+1)/2 – 1 + 3[n(n-1) / 2]

Summary

• Algorithm Analysis with some examples• Calculate Running Time , T(n)• Analysis of Big O notation