daa - notes

Algorithm Analysis and Design (R 606) 1

Department of Computer Science & Engineering SJCET, Palai



ALGORITHM ANALYSIS AND DESIGN R606 3+1+0

Module 1 Introduction and Complexity

What is an algorithm Properties of an Algorithm, Difference between Algorithm,

Computational Procedure and Program, Study of Algorithms; Pseudo-code

Conventions; Recursive Algorithms Space and Time Complexity Asymptotic

Notations

Relations and Recurrence Trees for Complexity Calculations; Profiling.

Deterministic and non - deterministic algorithms.

Module 2 Divide and Conquer

Control Abstraction, Finding Maximum and Minimum, Binary Search, Divide and

Sort.

Module 3 Greedy Strategy

Control Abstraction, General Knapsack Problem, Optimal Storage on Tapes,

Minimum Cost Spanning Trees Job

sequencing with deadlines.

Module 4 Dynamic Programming

Principle of Optimality, Multi-stage Graph, All -Pairs Shortest Paths, Travelling

Salesman Problem.

Lower Bound Theory - Comparison Trees for Searching and Sorting, Oracles and

Adversary Arguments th

Smallest Element.

Module 5 Backtracking

Control Abstraction - Bounding Functions, Control Abstraction, N-Queens Problem,

Sum of Subsets, Knapsack problem.

Branch and Bound Techniques FIFO, LIFO, and LC Control Abstractions, 15-

puzzle, Travelling Salesman Problem.



Text Book

1. Fundamentals of Computer Algorithms - Horowitz and Sahni, Galgotia

References

1. Computer Algorithms Introduction to Design and Analysis - Sara Baase & Allen Van Gelder, Pearson Education

2. Data Structures algorithms and applications - Sahni, Tata McGrHill

3. Foundations of Algorithms - Richard Neapolitan, Kumarss N., DC Hearth & Company

4. Introduction to algorithm- Thomas Coremen, Charles, Ronald Rivest -PHI



TABLE OF CONTENTS

Module 1

7

Properties of an Al ................ .. 7

Difference between Algorithm Computational

.................... 8

Study of Algorithms........................................................................ 10

Pseudocode con 10

....................... 12

................ 19

Time Complexity............................................................................. 20

A ....... 25

Oh .................................... 26

Omega........................................................................................... ......... 28

Theta............................................................................................ .......... 30

Common Complexity Fu 33

Recurrence Relations....................................................................... 35

Recurrence Tree ................................ 39

............................... 41

Deterministic and non - determ 41



Module 2

44

Finding Maximum a 45

..... 51

................. 54

56

... 60

Quick 69

Module 3

Greedy Strategy

87

88

Optimal Storag 90

Minimum Cost Spanning Tr 91

............. 93

99

100

Module 4

Dynamic Programming

Principle of O 105

Multi - ...... 106



All -Pairs Shorte 118

Lower Bound Theory

Comparison Trees for Searching and Sorting .................................. 127

Oracles and Adversary Arguments.................................................. 134

............................................................... 135

................................................................... 136

............................................................... 142

th ....................... 144

Module 5

Backtracking

Control Ab 149

...... 153

Control Abstra 149

N-Queens Problem.......................................................................... 153

Sum of Subsets ...................................................................... 158

....................................................... 164

Branch and Bound Techniques

FIFO ........................................................................................... 172

.......................................................................................... ............ 177

LC Control Abstractions..................................................................... 181

15- .................................................................................... .......... 182

Trave ...................................................... 184



MODULE 1



WHAT IS AN ALGORITHM

Definition : An algorithm is a finite set of instructions that, if followed, accomplishes a

particular task.

PROPERTIES OF AN ALGORITHM

All algorithms must satisfy the following criteria:

1. Input. Zero or more quantities are externally supplied.

2. Output. At least one quantity is produced.

3. Definiteness. Each instruction is clear and unambiguous.

4. Finiteness. If we trace out the instructions of an algorithm, then for all cases, the

algorithm terminates after a finite number of steps.

5. Effectiveness. Every instruction must be very basic so that it can be carried out, in

principle, by a person using only pencil and paper. It is not enough that each

operation be definite as in criterion 3; it also must be feasible.

An algorithm is composed of a finite set of steps, each of which may require one or more

operations. The possibility of a computer carrying out these operations necessitates that

certain constraints be placed on the type of operations an algorithm can include.

Criteria 1 and 2 require that an algorithm produce one or more outputs and have zero or more

inputs that are externally supplied. According to criteria 3, each operation must be definite,

meaning that it must be perfectly clear what should be done.

The fourth criterion for algorithms is that they terminate after a finite number of operations.

A related consideration is that the time for termination should be reasonably short.

Criteria 5 requires that each operation be effective; each step must be such that it can, at least

in principle, be done by a person using pencil and paper in a finite amount of time.

Performing arithmetic on integers is an example of an effective operation, but arithmetic with

real numbers is not, since some values may be expressible only by infinitely long decimal

expansion.



DIFFERENCE BETWEEN ALGORITHM, COMPUTATIONAL

PROCEDURE AND PROGRAM

COMPUTATIONAL PROCEDURE

Algorithms that are definite and effective are also called computational procedures. One

important example of computational procedures is the operating system of a digital computer.

This procedure is designed to control the execution of jobs, in such a way that when no jobs

are available, it does not terminate but continues in a waiting state until a new job is entered.

PROGRAM

To help us achieve the criterion of definiteness, algorithms are written in a

programming language. Such languages are designed so that each legitimate sentence has a

unique meaning. A program is the expression of an algorithm in a programming language.

Sometimes words such as procedure, function and subroutine are used synonymously for

program.

The study of algorithms includes many important and active areas of research. There

are four distinct areas of study:

1. How to devise algorithms: - Creating an algorithm is an art which may never

be fully automated. There are several techniques with which you can devise

new and useful algorithms. Dynamic programming is one such technique.

Some of the techniques are especially useful in fields other than computer

science such as operations research and electrical engineering.

2. How to validate algorithms: - Once an algorithm is devised, it is necessary to

show that it computes the correct answer for all possible legal inputs. This

process is referred to as algorithm validation. It is sufficient to state the

algorithm in any precise way and need not be expressed as a program. The

purpose of validation is to assure us that this algorithm will work correctly

independently of the issues concerning the programming language it will

eventually be written in. once the validity of the method has been shown, a

program can be written and a second phase begins. This phase is referred to as

program proving or program verification. A proof of correctness requires that



the solution be stated in two forms. One form is usually as a program which is

annotated by a set of assertions about the input and output variables of the

program. These assertions are often expressed in predicate calculus. The

second form is called a specification, and this may also be expressed in the

predicate calculus. A proof consists of showing that these two forms are

equivalent in that for every given legal input, they describe the same output. A

complete proof of program correctness requires that each statement of the

programming language be precisely defined and all basic operations be proved

correct.

3. How to analyze algorithms: - this field of study is called analysis of

processing unit (CPU) to perform operations and its memory (both immediate

and auxiliary) to hold the program and data. Analysis of algorithms or

performance analysis refers to the task of determining how much computing

time and storage an algorithm requires. An important result of this study is that

it allows you to make quantitative judgments about the value of one algorithm

over another. Another result is that it allows you to predict whether the

software will meet any efficiency constraints that exist. Questions such as how

well an algorithm performs in the best case, in the worst case, or on the

average are typical.

4. How to test a program: - testing a program consists of two phases: debugging

and profiling (or performance measurement). Debugging is the process of

executing programs on sample data sets to determine whether faulty results

occur and, if so to correct them. In cases in which we cannot verify the

correctness of output on sample data, the following strategy can be employed:

let more than one programmer develop programs for the same problem, and

compare the outputs produced by these programs. If the outputs match, then

there is a good chance that they are correct. A proof of correctness is much

more valuable than a thousand tests, since it guarantees that the program will

work correctly for all possible inputs. Profiling or performance measurement

is the process of executing a correct program on data sets and measuring the

time and space it takes to compute the results. These timing figures are useful

in that they may confirm a previously done analysis and point out logical

places to perform useful optimization.



STUDY OF ALGORITHM

An algorithm is a finite set of instructions that , if followed , accomplishes a particular task.

The study of algorithms includes many important and active areas of research. There are four

distinct areas of study

1. How to devise algorithms: Creating an algorithm is an art which may never be

fully automated.To study various design techniques that have proven to be useful in

that they have often yielded good algorithms.Dynamic programming is one such

technique.some important design techniques are linear,nonlinear and integer

programming .

2. How to validate algorithms: Once an algorithm is devised , it is necessary to show

that it computes the correct answer for all possible legal inputs.This process is

algorithm validation.The purpose of the validation is to assure us this algorithm will

work correctly independent of the issues concerning the programming language it will

eventually be written in.

3. How to analyze algorithms: This field of study is called analysis of algorithms. As

processing unit to hold the

program and data.Analysis of algorithms refers to the task of determining how much

computing time and storage an algorithm requires.It allows you to predict whether

the software will meet any efficiency constraints that exits.

4. How to test a program: Testing a program consists of two phases, debugging and

profiling.Debugging is the process of executing programs on sample data sets to

determine whether faulty results occur and , if so, to correct them. Profiling is the

process of executing a correct program on data sets and measuring the time and space

it takes to compute the results.

PSEUDOCODE CONVENTIONS

We can describe an algorithm in many ways.We can use a natural language like

English, although I we select this option,we must make sure that the resulting instructions are

definite.

We can present most of our algorithms using a pseudocode that resembles c

1. Comments begin with // and continueuntill the end of line

Eg: count :=count+1;//count is global ;It is initially zero.



2. Blocks are indicated with matching braces: { and } .A compound statement

can be represent as a block.The body of a procedure also forms a block.Statements

are delimited by ;

Eg: for j:= 1 to n do

{

Count:=count+1;

C[i,j]:=a[i,j]+b[i,j];

Count:=count +1; }

3. An identifier begins with a letter. The data types of variables are not explicitly

declared. The types will be clear from the context .Whether a variable is global or

local to a procedure will also be evident from the context. Compound data types

can be formed with records.

Eg: node=record

{

datatype_1 data_1;

:

datatype_n data_n;

node *link;

}

4. Assignment of values to variables is done using the assignment statement

:= ;

Eg: count:= count+1;

5. There are two Boolean values true and false. In order to produce these values, the

logical operators and , or , and not and the relational relational operators

are provided.

Eg: if (j>1) then k:=i-1; else k:=n-1;

6. Elements of multidimentional arrays are accessed using [ and ].

For eg: if A is a two dimentional array , the (I,j) th element of the array is denoted as A[I,j].

Array indicates start at zero.

7. The following looping statements are employed: for,while and repeat until. The

while loop takes the following form.

While (condition) do

{



:

:

}

8. A conditional statement has the following forms:

If < condition > then

If then else

Here < condition > is a Boolean expression and ,, and

< statement 2> are arbitrary statements.

9. Input and output are done using the instructions read and write. No format is used

to specify the size of input or output quantities.

10. There is only one type of procedure: Algorithm. An algorithm consists of a

heading and a body. The heading takes the form

Algorithm Nmae ()

RECURSIVE ALGORITHMS

A function that calls itself repeatedly, satisfying some condition is called a Recursive

Function. The algorithm that does this is called a recursive algorithm. Using recursion, we

split a complex problem into its single simplest case. The recursive function only knows how

to solve that simplest case.

TYPES OF RECURSION:

Linear Recursion

A linear recursive function is a function that only makes a single call to itself each time the

function runs (as opposed to one that would call itself multiple times during its execution).

The factorial function is a good example of linear recursion. Another example of a linear

recursive function would be one to compute the square root of a number using Newton's

method (assume EPSILON to be a very small number close to 0):

double my_sqrt(double x, double a)



{

double difference = a*x-x;

if (difference < 0.0) difference = -difference;

if (difference < EPSILON) return(a);

else return(my_sqrt(x,(a+x/a)/2.0));

}

Tail recursive

Tail recursion is a form of linear recursion. In tail recursion, the recursive call is the last thing

the function does. Often, the value of the recursive call is returned. As such, tail recursive

functions can often be easily implemented in an iterative manner; by taking out the recursive

call and replacing it with a loop, the same effect can generally be achieved. In fact, a good

compiler can recognize tail recursion and convert it to iteration in order to optimize the

performance of the code.

A good example of a tail recursive function is a function to compute the GCD, or Greatest

Common Denominator, of two numbers:

int gcd(int m, int n)

{

int r;

if (m < n) return gcd(n,m);

r = m%n;

if (r == 0) return(n);

else return(gcd(n,r));

}

Binary Recursive

Some recursive functions don't just have one call to themself, they have two (or more).

Functions with two recursive calls are referred to as binary recursive functions. The

mathematical combinations operation is a good example of a function that can quickly be



implemented as a binary recursive function. The number of combinations, often represented

as nCk where we are choosing n elements out of a set of k elements, can be implemented as

follows:

int choose(int n, int k)

{ if (k == 0 || n == k) return(1);

else return(choose(n-1,k) + choose(n-1,k-1));

}

Exponential recursion

An exponential recursive function is one that, if you were to draw out a representation of all

the function calls, would have an exponential number of calls in relation to the size of the

data set (exponential meaning if there were n elements, there would be O(an) function calls

where a is a positive number). A good example an exponentially recursive function is a

function to compute all the permutations of a data set. Let's write a function to take an array

of n integers and print out every permutation of it.

void print_array(int arr[], int n)

{

int i;

for(i=0; i



To run this function on an array arr of length n, we'd do print_permutations(arr, n, 0) where

the 0 tells it to start at the beginning of the array.

Nested Recursion

In nested recursion, one of the arguments to the recursive function is the recursive function

itself! These functions tend to grow extremely fast. A good example is the classic

mathematical function, "Ackerman's function. It grows very quickly (even for small values of

x and y, Ackermann(x,y) is extremely large) and it cannot be computed with only definite

iteration (a completely defined for() loop for example); it requires indefinite iteration

(recursion, for example).

Ackerman's function

int ackerman(int m, int n)

{

if (m == 0) return(n+1);

else if (n == 0) return(ackerman(m-1,1));

else return(ackerman(m-1,ackerman(m,n-1)));

}

Mutual Recursion

A recursive function doesn't necessarily need to call itself. Some recursive functions work in

pairs or even larger groups. For example, function A calls function B which calls function C

which in turn calls function A. A simple example of mutual recursion is a set of function to

determine whether an integer is even or odd.

int is_even(unsigned int n)

{

if (n==0) return 1;

else return(is_odd(n-1));

}

int is_odd(unsigned int n)

{

return (!iseven(n));

}



EXAMPLES OF RECURSIVE ALGORITHMS:

The Towers of Hanoi

The Towers of Hanoi puzzle (TOH) was first posed by a French professor, Edouard Lucas,

in 1883. Although commonly sold today discussed in discrete

mathematics or computer science books because it provides a simple example of recursion. In

addition, its analysis is straightforward and it has many variations of varying difficulty.

The object of the Towers of Hanoi problem is to specify the steps required to move the disks

or, as we will sometimes call them, rings) from pole r (r = 1, 2, or 3) to pole s (s = 1, 2, or 3;

s _= r), observing the following rules:

i) Only one disk at a time may be moved.

ii) At no time may a larger disk be on top of a smaller one.

The most common form of the problem has r = 1 and s = 3.

FIGURE

.

The Towers of Hanoi problem

Solution The algorithm to solve this problem exemplifies the recursive paradigm. We

imagine that we know a solution for and then we

use this solution to solve the problem for n disks. Thus to move n disks from pole 1 to pole 3,

we would:

1. Move 1 disks (the imagined known solution) from pole 1 to pole 2.

However we do this, the nth disk on pole 1 will never be in our way because any valid

sequence of moves with only 1 disks will still be valid if there is an nth (larger) disk

always sitting at the bottom of pole 1 (why?).

2. Move disk n from pole 1 to pole 3.

3. Use the same method as in Step 1 to move the 1 disks now on pole 2 to



pole 3.

Algorithm:

Input num [Number of disks]

Pinit [Initial pole; 1 3]

Pfin [Final pole; 1 3, Pinit _= Pfin]

Output The sequence of commands to the robot to move the disks from

pole Pinit to pole Pfin

Algorithm Hanoi

procedure H(in n, r, s) [Move n disks from pole r to pole s]

if n = 1 then robot( )

else H( 1, r, 6 )

robot( )

H( 1, 6 )

endif

endpro

H(num, Pinit, Pfin) [Main algorithm]

ALGORITHM DESIGN TECHNIQUES

For a given problem, there are many ways to solve them. The different methods are listed

below.

1. Divide and Conquer.

2. Greedy Algorithm.

3. Dynamic Programming.

4. Branch and Bound.

5. Backtracking Algorithms.

6. Randomized Algorithm.

Now let us discuss each method briefly.

1. Divide and Conquer



Divide and conquer method consists of three steps.

a. Divide the original problem into a set of sub-problems.

b. Solve every sub-problem individually, recursively.

c. Combine the solutions of the sub-problems into a solution of the whole original

problem.

2. Greedy Approach

Greedy algorithms seek to optimize a function by making choices which

are the best locally but do not look at the global problem. The result is a good solution but not

necessarily the best one. Greedy Algorithm does not always guarantee the optimal solution

however it generally produces solutions that are very close in value to the optimal solution.

3.Dynamic Programming.

Dynamic Programming is a technique for efficient solution. It is a

method of solving problems exhibiting the properties of overlapping sub problems and

optimal sub-structure that takes much less time than other methods.

4. Branch and Bound Algorithm.

In Branch and Bound Algorithm a given Algorithm

which cannot be bounded has to be divided into at least two new restricted sub-problems.

Branch and Bound Algorithm can be slow, however in the worst case they require efforts that

grows exponentially with problem size. But in some cases the methods converge with much

less effort. Branch and Bound Algorithms are methods for global optimization in non-convex

problems.

5.Backtracking Algorithm.

They try each possibility until they find the right one. It is a depth

first search of a set of possible solution. During the search i

search backtrack to the choice point, the place which presented different alternatives and tries

the next alternative. If there are no more choice points the search fails.

6.Randomized Algorithm.

A Randomized Algorithm is defined as an algorithm that is

allowed to access a source of independent, unbiased, random bits, and it is then allowed to

use these random bits to influence its computation.



ALGORITHMIC COMPLEXITY

The time complexity of an algorithm is given by the number of steps taken by the

algorithm to compute the function it was written for. The number of steps is itself a function

of the instance characteristics. Althogh any specific instance may have several characteristics,

the number of steps computed as a function of some subset of these. Usually, we might wish

to know how the computing time increases as the number of inputs increase. In this case the

number of steps will be computed as a function of the number of inputs alone. For a different

algorithm , we might be interested in determining how the computing time increases as the

magnitude of one of the input increases. In this case the number of steps will be computed as

a function of the magnitude of this input alone. Thus before the step count of an algorithm

can be determined, we need to know which characteristics of the problem instance are to be

used. These define the variables in the expression for the step count. In the case of sum, we

chose to measure the time complexity as a function of the number n of elements being added.

For algorithm Add, the choice of characteristics was the number m of rows and the number n

of columns in the matrices being added.

SPACE COMPLEXITY

Algorithm abc computes a+b+b*c+(a+b-c)/(a+b)+4.0;

RSum is a recursive algorithm that computes Ei n =1 a [i]

The space needed by each of these algorithms is seen to be the sum of the following

components:

1. A fixed part that is independent of the characteristics of the inputs and ouputs . This

part typically includes the instruction space, space for simple variables and fixed-size

component variables, space for constants.

2. A variable part that consists of the space needed by component variables whose size is

dependent on the particular problem instance being solved, the space needed by

reference variables and he recursion stack space.

The space requirement s(p) of any algorithm p may thereore be written as s(p) =c+Sp,

Where c is a constant.



TIME COMPLEXITY

COMPLEXITY OF SIMPLE ALGORITHMS

Time complexity of an algorithm is given by the number of steps taken

by the algorithm to compute the function it was written for. The time T(P) taken by a

program P is the sum of compile time and run time. The compile time does not depends the

instance characteristics. A compiled program will run several times without recompilation.

So we have to concern with the run time of the program which is denoted by tp (instance

characteristics).

If we know the characteristics of the compiler to be used ,we could proceed to determine the

number of additions, subtractions, multiplications, divisions, compares, loads, stores and so

on,that would be made the code for P. So we could obtain an expression for tp (n) of the

form

tp(n)=ca ADD(n) + cs SUB(n) + cm MUL(n) + cd

Where n denotes the instance characteristics, and ca, cs , cm, cd, and so on, respectively, denote

the time needed for an addition, subtraction, multiplication, division ,and so on, and

ADD,SUB,MUL, DIV, and so on, are functions whose values are the numbers of additions,

subtractions, multiplications, divisions, and so on ,that are performed when the code for P is

used on an instance with characteristic n. The value of tp(n) for any n can be obtained only

experimentally.The program is typed,compiled and run on a particular machine.The

execution time is physically blocked,and tp(n) obtained.In a multi user system ,execution time

depends on factors such as system load and number of other programs running on the

computer at the time P is running,the characteristics of these other programs and so on.

A propgram step is loosely defined as a syntactically or semantically meaningful segment of

a program that has an execution time independent of the instance characteristics. For

example,consider the entire stataement

return a+b+b*c+(a+b-c)/(a+b)+4.0 of the program given below



Algorihm abc(a,b,c)

{

return a+b+b*c+(a+b-c)/(a+b)+4.0

}

The above line could be considered as a step since its execution time is independent of the the

instance characteristics.The number of steps any program statement is assigned depends on

the kind of statement.For example comments count as zero steps;an assignment statement

which does not involve any calls to other algorithms is counted as one step;in an iterative

statement such as the for ,while,and repeat until statements ,we consider the step count only

for the control part of the the statement.The control parts for for and while statements have

the following forms:

for i=(expr) to (expr1) do

while(expr) do

Each execution of the control part of a while statement is given by a step count equal to the

number of step counts assignable to(expr).The step count for each execution of the

controlpart of a for statement is one,unless the counts attributable to (expr) and (expr1) are

the functions of the instance characteristics. In the latter case,the first execution of the

control part the for has a step count equal to the sum of counts for (expr)

and (expr1).Remaining executions of the for statement have a step count of one;and o on.

We can determine the number of steps needed by a program to solve a

particular program instance in one of the two ways.In the first method we introduce a new

variable,count into the problem.This is a global variable with initial value equals

zero.Statement to increment count by appropriate amount are introduced into the

program.This is done so that each time a statement in the original program is executed,count

is incremented by the step count of that statement.



EXAMPLE FOR T IIME COMPLEXITY CALCULAT ION

Each new term is obtained by taking the sum of two previous terms.If we call the first term

of the sequence f0,then f0=0,f1=1,and in general

fn=fn-1+fn-2, n>=2

Algorithm Fibonacci(n)

// Compute the nth Fibonacci number

{

if (n1.When n=0 or 1 lines 4and 5 get executed once each.Since each line has

an s/e of 1,total step count for this case is 2.When n>1,lines 4, 8 and 14 are each executed

once.Line 9 gets executed n times ,and lines 11 and 12 get executed n-1 times each.Line 8 has

an s/e of 2,line 12 has an s/e of 2 and line 13 has s/e of 0.The remaining lines that get



Example 1 Sum of n numbers

Algorithm with count statements added

Algorithm sum(a,n)

{ S:=0.0;

Count := count+1;//count is global; it is initially zero.

For i:=1 to n do

{ Count :=count + 1;//For for

S:= s+a[i];count:=count+1;//For assignment

}

Count :=count+1;//For last time of for

Count := count+1;//For the return

Return s;

}

Simplified version of algorithm

Algorithm Sum(a,n)

{ For i:=1 to n do count:= count+2;

Count:= count+3;}

Complexity calculation

tRSum(n)=2+tRSum(n-1) =2+2+tRSum(n-2)

=2(2)+tRSum(n-2)

:

=n(2)+tRSum(0) =2n+2, n>=0

Example 2 :Complexity of Fibonacci series



Algorithm Fibonacci(n)

//compute the nth Fibonacci number

{

If(n1.When n=0 or 1 , lines 4 and 5 get executed once each. Since each line has an

s/e of 1, the total step count for this case is 2.When n>1, lines 4,8 and 14 are each executed

once. Line 9 gets executed n times and lines 11 and 12 get executed n-1 times each.Line 8 has

an s/e of 2, line 12 has an s/e of 2,and line 13 has an s/e of 0. The remaning lines that get

executed have s/e of 1. The total steps for the case n>1 is therefore 4n+1.



ASYMPTOTIC NOTATION

Introduction

A problem may have numerous algorithmic solutions. In order to choose the best algorithm

for a particular task, you need to be able to judge how long a particular solution will take to

run. Or, more accurately, you need to be able to judge how long two solutions will take to

run, and choose the better of the two. You don't need to know how many minutes and

seconds they will take, but you do need some way to compare algorithms against one another.

Asymptotic complexity is a way of expressing the main component of the cost of an algorithm,

using idealized units of computational work. Consider, for example, the algorithm for sorting

a deck of cards, which proceeds by repeatedly searching through the deck for the lowest card.

The asymptotic complexity of this algorithm is the square of the number of cards in the deck.

This quadratic behavior is the main term in the complexity formula, it says, e.g., if you

double the size of the deck, then the work is roughly quadrupled.

The exact formula for the cost is more complex, and contains more details than are needed to

understand the essential complexity of the algorithm. With our deck of cards, in the worst

case, the deck would start out reverse-sorted, so our scans would have to go all the way to the

end. The first scan would involve scanning 52 cards, the next would take 51, etc. So the cost

formula is 52 + 51 + ... + 1. generally, letting N be the number of cards, the formula is 1 + 2

+ ... + N, which equals (N + 1) * (N / 2) = (N2 + N) / 2 = (1 / 2)N

2 + N / 2. But the N^2 term

dominates the expression, and this is what is key for comparing algorithm costs. (This is in

fact an expensive algorithm; the best sorting algorithms run in sub-quadratic time.)

Asymptotically speaking, in the limit as N tends towards infinity, 1 + 2 + ... + N gets closer

and closer to the pure quadratic function (1/2) N^2. And what difference does the constant

factor of 1/2 make, at this level of abstraction. So the behavior is said to be O(n2).

Now let us consider how we would go about comparing the complexity of two algorithms.

Let f(n) be the cost, in the worst case, of one algorithm, expressed as a function of the input

size n, and g(n) be the cost function for the other algorithm. E.g., for sorting algorithms, f(10)

and g(10) would be the maximum number of steps that the algorithms would take on a list of

10 items. If, for all values of n >= 0, f(n) is less than or equal to g(n), then the algorithm with



complexity function f is strictly faster. But, generally speaking, our concern for

computational cost is for the cases with large inputs; so the comparison of f(n) and g(n) for

small values of n is less significant than the "long term" comparison of f(n) and g(n), for n

larger than some threshold.

Note that we have been speaking about bounds on the performance of algorithms, rather than

giving exact speeds. The actual number of steps required to sort our deck of cards (with our

naive quadratic algorithm) will depend upon the order in which the cards begin. The actual

time to perform each of our steps will depend upon our processor speed, the condition of our

processor cache, etc., etc. It's all very complicated in the concrete details, and moreover not

relevant to the essence of the algorithm.

BIG-O NOTATION

Definition

Big-O is the formal method of expressing the upper bound of an algorithm's running time. It's

a measure of the longest amount of time it could possibly take for the algorithm to complete.

More formally, for non-negative functions, f(n) and g(n), if there exists an integer n0 and a

constant c > 0 such that for all integers n > n0, f(n) cg(n), then f(n) is Big O of g(n). This is

denoted as "f(n) = O(g(n))". If graphed, g(n) serves as an upper bound to the curve you are

analyzing, f(n).

O-Notation (Upper Bound)

This notation gives an upper bound for a function to within a constant factor. We

write f(n) = O(g(n)) if there are positive constants n0 and c such that to the right of n0,

the value of f(n) always lies on or below cg(n).



Theory Examples

So, let's take an example of Big-O. Say that f(n) = 2n + 8, and g(n) = n2. Can we find a

constant c, so that 2n + 8



BIG-OMEGA NOTATION

For non-negative functions, f(n) and g(n), if there exists an integer n0 and a constant c > 0

such that for all integers n > n0, f(n) cg(n), then f(n) is omega of g(n). This is denoted as

"f(n) = ".

This is almost the same definition as Big Oh, except that "f(n) cg(n)", this makes g(n) a

lower bound function, instead of an upper bound function. It describes the best that can

happen for a given data size.

-Notation (Lower Bound)

This notation gives a lower bound for a function to within a constant factor. We write

f(n g(n)) if there are positive constants n0 and c such that to the right of n0, the

value of f(n) always lies on or above cg(n).

How asymptotic notation relates to analyzing complexity

Temporal comparison is not the only issue in algorithms. There are space issues as well.

Generally, a trade off between time and space is noticed in algorithms. Asymptotic notation

empowers you to make that trade off. If you think of the amount of time and space your

algorithm uses as a function of your data over time or space (time and space are usually

analyzed separately), you can analyze how the time and space is handled when you introduce

more data to your program.



This is important in data structures because you want a structure that behaves efficiently as

you increase the amount of data it handles. Keep in mind though that algorithms that are

efficient with large amounts of data are not always simple and efficient for small amounts of

data. So if you know you are working with only a small amount of data and you have

concerns for speed and code space, a trade off can be made for a function that does not

behave well for large amounts of data.

A few examples of asymptotic notation

Generally, we use asymptotic notation as a convenient way to examine what can happen in a

function in the worst case or in the best case. For example, if you want to write a function

that searches through an array of numbers and returns the smallest one:

function find-min(array a[1..n])

let j :=

for i := 1 to n:

j := min(j, a[i])

repeat

return j

end

Regardless of how big or small the array is, every time we run find-min, we have to initialize

the i and j integer variables and return j at the end. Therefore, we can just think of those parts

of the function as constant and ignore them.

So, how can we use asymptotic notation to discuss the find-min function? If we search

through an array with 87 elements, then the for loop iterates 87 times, even if the very first

element we hit turns out to be the minimum. Likewise, for n elements, the for loop iterates n

times. Therefore we say the function runs in time O(n).

What about this function:

function find-min-plus-max(array a[1..n])

// First, find the smallest element in the array

let j := ;

for i := 1 to n:

j := min(j, a[i])

repeat



let minim := j

// Now, find the biggest element, add it to the smallest and

j := ;

for i := 1 to n:

j := max(j, a[i])

repeat

let maxim := j

// return the sum of the two

return minim + maxim;

end

What's the running time for find-min-plus-max? There are two for loops, that each iterate n

times, so the running time is clearly O(2n). Because 2 is a constant, we throw it away and

write the running time as O(n). Why can you do this? If you recall the definition of Big-O

notation, the function whose bound you're testing can be multiplied by some constant. If

f(x) = 2x, we can see that if g(x) = x, then the Big-O condition holds. Thus O(2n) = O(n). This

rule is general for the various asymptotic notations.

THETA

Definition :

positive constants c1,c2, and n0 such that c1g(n) =3n for all n>=2 and 3n+2=2, so c1=3,c2=4 and n0=2. 3n+3= theta(n), 10n2+4n+2=theta(n2),6*2n+n2=theta(2n),

and 10* log n+4= theta (log n). 3n+2 !=theta(1), 3n+3 != theta(n2), 10n2+4n+2 !=theta(n),

10n2+4n+2 !=theta(1), 6*2n+n2 !=theta(n2), 6*2n +n2 !=theta(n100), and 6 * 2n +n2

!=theta(1).

The theta notation is more precise than both the big oh and big omega

notations. The function f(n)=theta(g(n)) iff g(n) is both lower and upper bound of f(n).



Little oh

Definition:

Lim f(n)/g(n)=0

n->infinity

Example:

The function 3n+2=o(n2) since lim n->infinity (3n+2)/n2=0. 3n+2=o(n log

n). 3n+2= o(n log log n). 6*2n+n2=o(3n). 6*2n+n2=o(2n log n). 3n+2 !=o(n). 6*2n+n2

!=o(2n).



Little omega

Definition:

Lim g(n)/f(n)=0

n->infinity

Example: Algorithm Sum(a,n)

{ S:=0.0;

for i=1 to n do

s:=s+a[i];

return s;

}

Alg 1. Iterative function for sum

For this algorithm Sum wec determined that tSum(n)=

2n+3. So, tSum(n)=theta(n)

Asymptotic Notation Properties

Let f(n) and g(n) be asymptotically positive functions. Prove or disprove each of the

following conjectures.



a. f(n)=O(g(n)) implies g(n)=O(f(n)).

b. f(n)+g(n)= (min(f(n),g(n))).

c. f(n)=O(g(n)) implies lg(f(n))=O(lg(g(n))),where lg(g(n))>=1 and f(n)>=1 for all

sufficiently large n.

d. f(n)=O(g(n)) implies 2^f(n)=O(2^g(n)).

e. f(n)=O((f(n))^2).

f.

g. f(n)= (f(n/2)).

h. f(n)+o(f(n))= (f(n)).

COMMON COMPLEXITY FUNCTIONS

Comparison of functions

Many of the relational properties of real numbers apply to asymptotic comparisons as well.

For the following, assume that f(n) and g(n) are asymptotically positive.

Transitivity:

f(n)= (g(n)) and g(n)=(h(n)) imply f(n)= (h(n)),

f(n)=O(g(n)) and g(n)=O(h(n)) imply f(n)=O(h(n)),

(g(n)) an

f(n)=o(g(n)) and g(n)=o(h(n)) imply f(n)=o(h(n)),

Reflexivity:

f(n)= (f(n)),

f(n)=O(f(n)),

Symmetry:

f(n)= (g(n)) if and only if g(n)= (f(n)).



Transpose symmetry

Because these properties hold for asymptotic notations, one can draw an analogy between the

asymptotic comparison of two functions f and g and the comparison of two real numbers a

and b:

f(n)=O(g(n)) similar to a=b,

f(n)= (g(n)) similar to a=b,

f(n)=o(g(n)) similar to ab.

We say that f(n) is asymptotically smaller than g(n) if f(n)=o(g(n)), and f(n) is asymptotically

One property of real numbers, however, does not carry over to asymptotic notations:

Trichotomy : For any two real numbers a and b, exactly one of the following must hold:

ab.

Although any two real numbers can be compared, not all functions are asymptotically

comparable. That is, for two functions f(n) and g(n), it may be the case that neither

compared using asymptotic notation, since the value of the exponent in n^(1+sin n) oscillates

between 0 and 2, taking on all values in between.



RECURRENCE RELATION S

The recurrence relation is an equation or inequality that describes a function in terms of its

values of smaller inputs. The main tool for analyzing the time efficiency of a recurrence

algorithm is to setup a sum expressing the number executions of its basic operation and

To solve a recurrence relation means ,to obtain a function defined on natural numbers that

satisfies the recurrence.

Examble: tn = 2tn-1

Different methods for solving recurrence relation are:-

Substitution method

Itration method

Changing variables method

Recurrence tree

Characteristic equation method

Master theorem

Solving recurrences by Substitution method

Idea: Make a guess for the form of the solution and prove by induction.

Can be used to prove both upper bounds O() and lower bounds

T(n) = 2T(n/2) + n using substitution

Guess log n for some constant c (that is, T(n) = O(n log n))

Proof:

Base case: we need to show that our guess holds for some base case (not

necessarily

n = 1, some small n is ok). Ok, since function constant for small constant n.

Assume holds for n/2:

Prove that holds for n: og n

T(n)= 2T(n/2)+n



= cn log n/2 +n

=cn log n-cn log 2+ n

=cn log n-cn + n

Similarly it can be shown that

Similarly it can be shown that

The hard part of the substitution method is often to make a good guess.

Solving Recurrences with the Iteration

The iteration method does not require making a good guess like the substitution

method (but

it is often more involved than using induction).

Example: Solve T(n) = 8T(n/2) + n (T(1) = 1)

T(n) = n + 8T(n/2)

= n + 8(8T( n/2 ) + (n/2))

= n + 8T( n/2 ) + 8(n/4))

= n+ 2n + 8T( n/2 )

= n + 2n + 8(8T( n/2 ) + ( n/2 ))

= n + 2n + 8T( n/2 ) + 8(n/4 ))

= n + 2n + 2n+ 8T( n/2 )

= . . .

= n + 2n + 2n+ 2n + . . .

Masters theorem

The master method is used for solving the following type of recurrence

T(n)=aT(n/b)+f(n),a 1 and b>1.In above



time.The cost of split the problem or combine the solutions of subproblems is given by

function f(n).It should be note that the number of leaves in the recursion tree is where

E=

THEOREM

Let T(n) be defined on the non-negative integers by the recurrence

T(n)=aT(n/b)+f(n) where a 1,b>1 are constants and f(n) be a function.Then T(n) can be

asymtotically as,

CASE 1:

If f (n) O( for some e>0,then ( )

CASE 2:

If ( ) then (f(n).log n)

CASE 3:

If for some e>0 and for some e then T(n) (f(n))



RECURRENCE PROBLEMS

Case 1 problem:

1. T(n)=4T(n/2)+n

E=log 4/log2=2

O(

O( =n

( )

Case 2 problem:

2.T(n)=4T(n/2)+

a=4, b=2

f(n)=

E=log 4/log 2=2

( )

( .log n)

Case 3 problem:

3.T(n)=4T(n/2)+

a=4, b=2

f(n)=

E=log 4/log 2=2

)

)

)

O( )

= O( )

= O( ( )



RECURSION TREES FOR COMPLEXITY CALCULATIONS

A different way to look at the iteration method: is the recursion-tree

we draw out the recursion tree with cost of single call in each node running time is

sum of

costs in all nodes

if you are careful drawing the recursion tree and summing up the costs, the recursion

tree is

a direct proof for the solution of the recurrence, just like iteration and substitution

Example: T(n) = 8T(n/2) + n (T(1) = 1)

T(n)=n+ 2n+ 2n+ 2n+ .....+2^(log n-1) n+8^(log n)



Changing variables

Sometimes reucurrences can be reduced to simpler ones by changing variables

Example: Solve

Let

T(2^m) = 2T(2^m/2) + m

Let S(m) = T(2^m)

T(2^m) = 2T(2^m/2) + m ) S(m) = 2S(m/2) + m)

S(m) = O(mlogm))

T(n) = T(2^m) = S(m) = O(mlogm) = O(log n log log n)

Other recurrences

Some important/typical bounds on recurrences not covered by master method:

Logarithmic:

Recurrence: T(n) = 1 + T(n/2)

Typical example: Recurse on half the input (and throw half away)

Variations: T(n) = 1 + T(99n/100)

Linear:

Recurrence:

Typical example: Single loop

Variations: T(n) = 1+2T(n/2), T(n) = n+T(n/2), T(n) = T(n/5)+T(7n/10+6)+n

Quadratic:

Recurrence:

Typical example: Nested loops

Exponential:

Recurrence:



PROFILING

Profiling or performance measurement is the process of executing a correct program on data

set and measuring the time and space it takes to compute the result.

NONDETERMINISTIC ALGORITHMS

The notation of algorithm that we have been using has the property that the result of every

operation is uniquely defined.Algoithm with this property are termed deterministic

algorithms.

Such algorithms agree with the programs are executed on computer. In a theoretical

framework we can remove this rustication on the outcomes of every operation. We can allow

algorithms to be containing operations whose outcomes of every operation. We can allow

algorithm to contain operations whose outcomes are not uniquely defined but are limited to

specified set of possibilities.

The machine executing such operations is allowed to choose any one of these outcomes

subject to be a defined later. This leads to the concept of nondeterministic algorithms, we

introduce three functions.

1. Choice (S) arbitrarily chooses one of the elements of element of set S.

2. Failure () signals an unsuccessful completion.

3. Success () signals a successful completion.

The assignment statement x=Choice (1, n) could result in x being assigned any

one of the integers in the range [1,n].there is the rule of specifying how this choice is to be

made. The Failure () and Success () signals are used to define a computation of the algorithm.

These statements cannot be used to effect a return. Whenever there is a set of choices that

leads to a successful completion, then one such set of choices is always made and the

algorithm terminates successfully. A nondeterministic algorithm terminates unsuccessfully if

and only if there exists no set choices leading to a success signal. The computing times for.

Choice, Failure, and Success are taken to be O(1).A machine capable of executing a non



nondeterministic algorithm in this way is called a nondeterministic machine. Although

nondeterministic machines do not exist in practice, we see that they provide strong intuitive

reasons to conclude that certain problems cannot be solved by fast deterministic algorithms.

Example

Consider the problem of searching for an element x in a given set of elements A[1:n],n>1.We

are required to determine an index j such that A[j]=x or j=0 if x is not in A.

1. J:=Choice(1.n);

2. If A[j]=x then {write (j);Success();}

3. Write (0); Failure();

From the way a nondeterministic computation is defined, it follows that no is 0 can be the

output if and only if there is no j such that A[j]=x.

Complexity of nondeterministic search algorithms =O(1).

DETERMINISTIC ALGOITHM for n>p

The deterministic algorithm for a selection whose run time is O(n/p log log p+ log

n)The basic idea of this algorithm is same to sequential algorithm.

The sequential algorithm partitions the input into groups(of size,says,5),finds the median of

each group, and output recursively the median(call it M)of these group meadians.Then the

rank

rM of M in the input is computed, and as a result, all element from the input that are either

Respectively. Finally an appropriate selection is performed from the remaining keys

recursively.

We showed that run time of this algorithm was O(n).



MODULE 2



DIVIDE AND CONQUER

Given a function to compute on n inputs the divide and conquer strategy suggests splitting the

inputs into k distinct subsets,1



FINDING THE MAXIMUM AND MINIMUM

This is a simple problem that can be solved by the divide and-conquer

n

ALGORITHM -1

Algorithm StraightMaxMin (a, n, max, min)

min

{

max:=min:=a[1];

for i:=2 to n do

{

if (a[i]>max) then max:= a[i];

if a[i]



ALGORITHM -2

Algorithm MaxMin ( i, j, max, min )

//a[1:n] is a global array. Parameters I and j are integers, 1



Eg:- Suppose we simulate MaxMin on the following nine elements:

a: [1] [2] [3] [4] [5] [6] [7] [8] [9]

22 13 -5 -8 15 60 17 31 47

A good way of keeping track of recursive calls is to build a tree by adding a

node each time anew call is made. For this algorithm each node has four items of

information: i, j, max, and min. On the array a[] above, the tree is produced.

Trees of recursive calls of MaxMin

We see that the root node contains 1 and 9 as the values of i and j corresponding to

the initial call to MaxMin. This execution produces new call to MaxMin where i and j have

the values 1 ,5 and 6,9, respectively, and thus split the set into two subsets of approximately

the same size. From the tree, we can immediately see that the maximum depth of recursion is

four(including the first call).

The order in which max and min are assigned values are follows: [1,2,22,13] [3,3,-5,-5]

[1,3,22,-5] [4,5,15,-8] [1,5,22,-8] [6,7,60,17] [8,9,47,31] [6,9,60,17] [1,9,60,-8].

Number of element comparisons needed for MaxMin:

If T(n) represents the number of element comparisons needed, then the

resulting recurrence relation is:

T ([n/2]) + T ([n/2]) +2 n>2

T (n) = 1 n=2

0 n=1

1, 9,60,-8

6,9,60,17

1, 3, 22, -5 4, 5, 15,-8 6,7,60,17

8, 9, 47, 31

1,2,22,13 3,3,-5,-5

1, 5, 22, -8



When n is a power of 2, n=2k we can solve this equation by successive substitutions:

T(n) = 2T(n/2)+2

= 2(2T(n/4)+2) +2

= 4T(n/4)+ 4 + 2

.

.

= 2k-1

T(2) + 1< = i < = k-1 2i

= 2k-1

+ 2k

-2

= 3n/2 - 2

When n is a power of 2, the number of comparisons in the best , average and worst case is

3n/2-2.

COMPLEXITY ANALYSIS OF FINDING MAXIMUM&MINIMUM

Consider the following nine elements to simulate MaxMin

22 13 -5 -8 15 60 17 31 47

Fig below shows the tree constructed for the algorithm. Each node has four items of

information: i, j, max and min.

9

5 8

3 4 6 7

1 2

Consider the total number of element comparison needed for MaxMin? If T(n) represents this

number, then the resulting recurrence relation is

T (n)= 1 n=2

0 n=1

1, 9, 60,-8

1, 5, 22,-8

1, 3, 22,-5

6, 9, 60,17

4, 5, 15,-8 6, 7, 60,17 1, 9, 60,-8

1, 2, 22, 13 3, 3, -5,-5



When n is a power of 2, n=2k for some positive integer k, then

T(n) = 2T( n/2 ) + 2

= 2(2T (n/4) +2) +2

= 4T (n/4) +4+2

:

:

= 2k-1

1



RECURSION TREE

DEFINITION:

A recursion tree is a tree that depicts the entire recursion process.

Consider the following recurrence relation.

T (n) =3T (n/4)+ 2)

= 3T (n/4) + cn2

cn2

c(n/4)2 c(n/4)

2 c(n/4)

2

:

T(n/16) T(n/16) T(n/16) T(n/16) T(n/16) T(n/16) :

: : : :

c(n/16)2

c(n/16)2

c(n/16)2

: : : :

: : : : : :

T(n/16) T(n/16) T(n/16) : : : : : :

: : : : : : : : :

T(1) T(1) T(1) T(1) T(1) T(1) T(1) T(1) T(1)

Sub problem size at depth i=n/4i

Boundary condition (n=1) :=(n/4i)=1 ie i=log4n

Depth of the tree=log4

Cost of each node at level i=c(n/4)i

No: of node at level log4n=3log4n

Cost of each node=3log4 4 43)

Total cost=cn2+3/16cn

2+ (3/16)

2cn

2+ (3/16)

log4n-1cn

2

= (3/16)I cn

24 43)



= (3/16)I cn

243)

= 1/(1-3/16)cn2

43)

= 16/13cn2

43)

= O (n2)

BINARY SEARCH ALGORITHMS

The Binary search requires an ordered list.

Iterative Algorithm

int find (const list, int target)

// pre: list is sorted in ascending order

//post: ITERATIVE binary search will return the index of the target element, else -1

{

int mid;

int first = 0;

int last = list.length( ) -1;

while ( first target )

last = mid - 1;

else first = mid + 1;

}

return -1;

}

Recursive Algorithm

int find (const list, int target, int first, int last)

// pre: list is sorted in ascending order

//post: RECURSIVE binary search will return the index of the target element, else -1



{

if (first > last)

return -1;

int mid = (first + last) / 2;

if (list[mid] == target)

return mid;

if (list[mid] < target)

return find(list, target, mid+1, last);

return find(list, target, first, mid-1);

}

Complexity Analysis of Binary Search

For binary search, storage is required for the n elements of the array plus the variables

low, high, mid and x.

A binary search tree can be created to explain the binary search where the mid value

of each node is the value of the node. For example, if n=14, the resultant tree will be like the

one shown below.

3

7

11

1 5 9 13

2 4 6 8 10 12 14



The first comparison is s with a[7].if xa[7], then the next comparison is with a[11]. If x is present, then the algorithm

will end at one of the circular nodes tat lists the index into the array where x was found. If x

is not present the algorithm will terminate at one of the square nodes.

Theorem:

If n is in the range [2k-1

,2k), then binary search makes at most k element comparison

for a successful search and either k-1 or k comparison for an unsuccessful search. In other

words the time for a successful search is O (log n) and for an unsuccessful search is (log n).

E=I+2n

Where E=External path length (sum of the distance of all external nodes from the root)

I=Internal path length (sum of the distance of all internal nodes from the root).

Let As (n) is the average number of comparison in a successful search and Au (n) is

the average number of comparison in an unsuccessful search.

The number of comparison to find an element represented by an internal node is one

more than the distance of this node from the root.

As (n) =1+I/n

The number of comparisons on any path from the root to an external node is equal to

the distance between the root and the external node. Since every binary tree with n internal

nodes has n+1 external nodes

Au (n) =E/ (n+1)

As (n) =I/n+1

= (E-2n/n) + 1

=E/n-1

= (Au (n). (n+1))/n -1

= (n. Au (n) + Au (n))/n -1

= Au (n) + Au (n)/n -1

As (n) = Au (n) (1 +1/n) -1



Successful searches

Best- (1) (only one element is compared)

Average- (log n)

Worst- (log n)

Unsuccessful searches

Best, Average and Worst- (log n)

DIVIDE -AND-CONQUER MATRIX MULTIPLICATION ALGORITHM

The product of two n x n matrices X and Y is a third , n x n matrix Z=XY , with (i,j)th

entry

n

Zij ik Ykj

k=1

In general XY, not the same as YX; matrix multiplication is not commutative.

The formula above implies an O (n3)

algorithm for matrix multiplication: there are n

2 entries

to be computed, and each takes linear time. For quite a while, this was widely believed

to be the best running time possible, and it was even proved that no algorithm which used

just additions and multiplications could do better. It was therefore a source of great

excitement when in 1969, Strassen announced a signi_cantly more ef_cient algorithm, based

upon divide-and-conquer.

Matrix multiplication is particularly easy to break into subproblems, because it can be

performed blockwise. To see what this means, carve X into four n/2 x n/2 blocks, and also Y:

X = A B

Y = E F

C D ,

G H

Then their product can be expressed in terms of these blocks, and is exactly as if the blocks

were single elements.

XY= A B E F

= AE+BG AF+BH



C D G H CE+DG CF+DH

We now have a divide-and-conquer strategy: to compute the size-n, product XY,

recursively compute eight size-n/2 products AE,BG,AF,BH,CE,DG,CF,DH and then do a

few O(n2) time additions. The total running time is described by the recurrence relation

T(n)= 8T(n/2) + O(n2),

which comes out to O(n3), the same as for the default algorithm. However, an improvement

in the time bound is possible, and as with integer multiplication, it relies upon algebraic

tricks. It turns out that XY can be computed from just seven sub problems.

P1 = A(F H)

P2 = (A + B)H

P3 = (C + D)E

P4 = D(G E)

P5 = (A + D) (E + H)

P6 = (B - D) (G + H)

P7 = (A - C) (E + F)

XY = P

5+P

4 P

2 +P

6 P1+P

2

P3+P

4 P1+P

5 P

3 P

7

This translates into a running time of

T(n)= 7T(n/2) + O(n2),

which by the result is O(nlog

2 7

2.81

).



Strassen showed that 2x2 matrix multiplication can be accomplished in 7 multiplication and

18 additions or subtractions. This reduce can be done by Divide and Conquer

Approach.Divide the input data S in two or more disjoint subsets S1, S2. Solve the

subproblems recursively. Combine the solutions for S1, S2 S. The base

case for the recursion are subproblems of constant size.Analysis can be done using recurrence

equations.Divide matrices in sub-matrices and recursively multiply sub-matrices

Let A, B be two square matrices over a ring R. We want to calculate the matrix product C as

If the matrices A, B are not of type 2n x 2

n we fill the missing rows and columns with zeros.

We partition A, B and C into equally sized block matrices

with

then

With this construction we have not reduced the number of multiplications. We still need 8

multiplications to calculate the Ci,j matrices, the same number of multiplications we need

when using standard matrix multiplication.

Now comes the important part. We define new matrices
http://en.wikipedia.org/wiki/Square_matrixhttp://en.wikipedia.org/wiki/Ring_%28mathematics%29http://en.wikipedia.org/wiki/Block_matrix



which are then used to express the Ci,j in terms of Mk. Because of our definition of the Mk we

can eliminate one matrix multiplication and reduce the number of multiplications to 7 (one

multiplication for each Mk) and express the Ci,j as

We iterate this division process n-times until the submatrices degenerate into numbers (group

elements).

Practical implementations of Strassen's algorithm switch to standard methods of matrix

multiplication for small enough submatrices, for which they are more efficient. The particular

crossover point for which Strassen's algorithm is more efficient depends on the specific

implementation and hardware. It has been estimated that Strassen's algorithm is faster for

matrices with widths from 32 to 128 for optimized implementations and 60,000 or more for

basic implementations.

Numerical analysis

The standard matrix multiplications takes approximately 2n3 arithmetic operations (additions

and multiplications); the asymptotic complexity is O(n3).
http://en.wikipedia.org/wiki/Submatrices



The number of additions and multiplications required in the Strassen algorithm can be

calculated as follows: let f(k) be the number of operations for a matrix. Then by

recursive application of the Strassen algorithm, we see that f(k) = 7f(k l4k, for some

constant l that depends on the number of additions performed at each application of the

algorithm. Hence f(k) = (7 + o(1))k, i.e., the asymptotic complexity for multiplying matrices

of size n = 2k using the Strassen algorithm is

.

The reduction in the number of arithmetic operations however comes at the price of a

somewhat reduced numerical stability.

Algorithm

void matmul(int *A, int *B, int *R, int n)

{

if (n == 1)

{

(*R) += (*A) * (*B);

}

else

{

matmul(A, B, R, n/4);

)()( Thus 3

1 1

3

1

,

1

,,

NOcNcNT

baC

N

i

N

j

N

k

jk

N

k

kiji
http://en.wikipedia.org/wiki/Numerical_stability



matmul(A, B+(n/4), R+(n/4), n/4);

matmul(A+2*(n/4), B, R+2*(n/4), n/4);

matmul(A+2*(n/4), B+(n/4), R+3*(n/4), n/4);

matmul(A+(n/4), B+2*(n/4), R, n/4);

matmul(A+(n/4), B+3*(n/4), R+(n/4), n/4);

matmul(A+3*(n/4), B+2*(n/4), R+2*(n/4), n/4);

matmul(A+3*(n/4), B+3*(n/4), R+3*(n/4), n/4);

}

}



MERGE SORT

Merge sort is an example of divide and conquer algorithm. Its worst case complexity is

O(nlog n). To understand merge sort assume that elements are arranged in the non decreasing

order. Assume a sequence of n elements a[1],a[2],...a[n]. Split them into two sets like

a[1],a[2],..a[n/2] and a[(n/2)+1],...a[n]. Each set is individually sorted, and the resulting

sorted sequences are merged to produce a single sorted sequence of n elements. Merge-sort is

based on the divide-and-conquer paradigm.

The Merge-sort algorithm can be described in general terms as consisting of the following

three steps:

1. Divide Step: If given array A has zero or one element,it is already sorted. Otherwise,

divide A into two arrays, each containing about half of the elements of A.

2. Recursion Step: Recursively sort array A1 and A2.

3. Conquer Step: Combine the elements back in A by merging the sorted arrays A1 and A2

into a sorted sequence.

Consider the following example:

Consider the array of ten elements a [1:10] = (310,285,179,652,351,423,861,254,450,520)

The merge sort algorithm first divides the array as follows

a[1:5] and a[6:10] and then

a[1:3] and a[4:5] and a[6:8] and a[9:10] and then into

a[1:2] and a[3:3] and a[6:7] and a[8:8] and finally it will look like

a[1:1],a[2:2],a[3:3],a[4:4],a[5:5],a[6:6],a[7:7],a[8:8],a[9:9],a[10:10]

Pictorially the file can now be viewed as

(310|285|179|652,351|423,861,254,450,520)

Elements a[1] and a[2] are merged as

(285,310|179|652,351|423,861,254,450,520)



And then a[3] is merged with a[1:2]

(179,285, 310 |652,351|423,861,254,450,520) next a[4] and a[5]

(179,285, 310 |351,652|423,861,254,450,520)and then a[1:3] and a[4:5]

(179,285, 310,351,652|423,861,254,450,520)

Repeated recursive calls are invoked producing the following array

(179,285, 310 ,351,652|254,423, 450,520,861) and finally

(179,254,285,310, 351,423, 450,520,652,861)

At this point there are two sorted subarrays and final merge produces the fully sorted result

If the time for the merging operation is proportional to n, then the computing time for merge

sort is described by the recurrence relation

a n= 1a a constant

1, 10

1, 5

1, 3 4, 5

6, 10

6, 8 9, 10

1, 1

3, 3 1, 2 4, 4 6, 7 5, 5

2, 2

8, 8 9, 9 10, 10

7, 7 6, 6



T(n)=

2T(n/2)+cn n>1, c a constant

When n is a power of 2, n=2k we can solve this equation by successive substitutions:

T(n) = 2(2T(n4)+cn/2)+cn

= 4T(n/4)+2cn

= 4(2T(n/8)+cn/4) + 2cn

.

.

= 2kT(1) + kcn

= an+cn log n

It is easy to see that if 2k < n



//Find where to split the set .

mid:=[(low+high)/2];

//Solve the subproblems.

MergeSort(low,high);

MergeSort(mid+1,high);

//Combine the solutions.

Merge(low,mid,high);

}

}

Algorithm Merge(low,mid,high)

//a[low:high] is a global array containing two sorted subsets in

//a[low:mid] and in a[mid+1:high].The goal is to merge these two

// sets into single set residing in a[low:high].b[] is an auxiliary global

//array.

{

h:=low; i:=low; j:=mid+1;

whil

{

{

b[i]:=a[h]; h:=h+1;

}

else



{

b[i]:=a[j]; j:=j+1;

}

i:=i+1;

}

if(h>mid) then

for k:=j to high do

{

b[i]:=a[k]; i:=i+1;

}

else

for k:=h to mid do

{

b[i]:=a[k]; i:=i+1;

}

for k:=low to high do a[k]:=b[k];

}



COMPLEXITY ANALYSIS OF MERGE SORT

Merge sort is an ideal example of the divide and conquer strategy in which the given set of

elements is split into two equal-sized sets and the combining operation is the merging of two

sorted sets into one.

Given a sequence of n elements a[1],.....,a[n], the general idea is to imagine them to split into

two sets a[1],...a[n/2] and a[n/2+1],...a[n].Each set is individually sorted and the resulting

sorted sequences are merged to produce a single sorted sequence of n elements.

MergeSort describes this process using recursion and a function Merge which merges two

sorted sets.

Example:

Consider an array of ten elements a[1:10] = (310,285,179,652,351,423,861,254,450,520).

Algorithm MergeSort begins by splitting a[] into two subarrays each of size five (a[1:5] and

a[6:10]). The elements in a[1:5] are then split into two subarrays of size three (a[1:3]) and

two (a[4:5]). Then the items in a[1:3] are split into subarrays of size two (a[1:2]) and one

(a[3:3]). The two values in a[1:2] are split final time into one-element subarrays , and now the

merging begins. A record of the subarrays is implicitly maintained by the recursive

mechanism.

Pictorially the file can be viewed as

( 310 | 285 | 179 | 652 , 351 | 423 , 861 , 254 , 450 , 520 ) where vertical bars indicate the

boundaries of subarrays. Elements a[1] and a[2] are merged to yield

( 285 , 310 | 179 | 652 , 351 | 423 , 861 , 254 , 450 , 520 )

Then a[3] is merged with a[1:2] and

( 179 , 285 , 310 | 652 , 351 | 423 , 861 , 254 , 450 , 520 ) is produced. Next elements a[4]

and a[5] are merged:

( 179 , 285 , 310 | 351 , 652 | 423 , 861 , 254 , 450 , 520 )

And then a[1:3] and a[4:5]:

( 179 , 285 , 310 , 351 , 652 | 423 , 861 , 254 , 450 , 520 )



At this point the algorithm has returned to the first invocation of MergeSort and is about to

process the second recursive call. Repeated recursive calls are invoked producing the

following subarrays:

( 179 , 285 , 310 , 351 , 652 | 423 | 861 | 254 | 450 , 520 )

Elements a[6] and a[7] are merged. Then a[8] is merged with a[6:7]:

( 179 , 285 , 310 , 351 , 652 | 254 , 423 , 861 | 450 , 520 )

Next a[9] and a[10] are merged and then a[6:8] and a[9:10]:

( 179 , 285 , 310 , 351 , 652 | 254 , 423 , 450 , 520 , 861 )

At this point there are two sorted subarrays and the final merge produces the fully sorted

result

( 179 , 254 , 285 , 310 , 351 , 423 , 450 , 520 , 652 , 861 )

Figure (1.1) is a tree that represents the sequence of recursive calls that are produced by

MergeSort when it is applied to ten elements. The pair of values in each node are the values

of parameters low and high. The splitting continues until the sets containing a single element

are produced.

Figure (1.2) is a tree representing the calls to produce Merge by MergeSort. For example,

the node containing 1 , 2 and 3 represents the merging of a[1:2] with a[3].



Tree of calls of Merge

( Fig 1.1 )

Tree of calls of MergeSort(1,10)

( Fig 1.2 )

1,10

1,5 6,10

1,3 4,5 6,8 9,10

1,2 3,3 9,9 10,10 8,8 6,7 4,4 5,5

1,1 2,2 6,6 7,7

1,1,2 6,6,7

1,2,3 6,7,8 9,9,10 4,4,5

1,3,5 6,8,10

1,5,10



If the time for the merging operation is proportional to n, then the computing time for merge

sort is described by the recurrence relation

T(n) = { a n=1 , a a constant

{ 2T(n/2) + cn n>1 , c a constant

When n is a power of 2, n = 2^k, we can solve this equation by successive substitutions:

T(n) = 2(2T(n/4)+ cn/2) +cn

= 4T(n/4) + 2cn

= 4(2T(n/8) + cn/4) + 2cn

.

.

.

= 2^kT(1) + kcn

= an + cn logn

It is easy to see that if 2^k < n < 2^(k+1) , then T(n) < T(2^k+1).

T(n) = O(n logn)



QUICK SORT

Quicksort is one of the fastest and simplest sorting algorithms . It works recursively by a

divide-and-conquer strategy.

Idea

First, the sequence to be sorted a is partitioned into two parts, such that all elements of the

first part b are less than or equal to all elements of the second part c (divide). Then the two

parts are sorted separately by recursive application of the same procedure (conquer).

Recombination of the two parts yields the sorted sequence (combine). Figure 1 illustrates this

approach.

Figure 1: Quicksort(n)

The first step of the partition procedure is choosing a comparison element x

daa - notes

Documents

palai algorithm analysis

palai module

algorithm computational

algorithm properties

deterministic algorithms

foundations of algorithms

algorithm thomas coremen

design r606