2

Contents

• Introduction to Data Structures

• Classification and Operations on Data Structures

• Preliminaries of Algorithm

• Algorithm Analysis and Complexity

• Recursive Algorithms

• Searching Techniques - Linear, Binary, Fibonacci

• Sorting Techniques- Bubble, Selection, Insertion,

Quick and Merge Sort

• Comparison of Sorting Algorithms

3

Introduction to Data Structures

• A data structure is a way of storing data in a

computer so that it can be used efficiently and

it will allow the most efficient algorithm to be

used.

• A data structure should be seen as a logical

concept that must address two fundamental

concerns.

I. First, how the data will be stored, and

II. Second, what operations will be performed on it.

Classification of Data Structures

• Data structures can be classified as i. Simple data structure

ii. Compound data structure

iii. Linear data structure

iv. Non linear data structure

Fig. Classification of Data Structures 4

Simple and Compound Data Structures

• Simple Data Structure: Simple data structure can be

constructed with the help of primitive data structure.

A primitive data structure used to represent the

standard data types of any one of the computer

languages. Variables, arrays, pointers, structures,

unions, etc. are examples of primitive data structures.

Compound Data structure: Compound data

structure can be constructed with the help of any one

of the primitive data structure and it is having a

specific functionality. It can be designed by user. It

can be classified as

i. Linear data structure

ii. Non-linear data structure 5

6

Linear and Non-linear Data Structures

• Linear Data Structure: Linear data structures can be

constructed as a continuous arrangement of data

elements in the memory. It can be constructed by

using array data type. In the linear Data Structures the

relationship of adjacency is maintained between the

data elements.

• Non-linear

structure can

Data Structure: Non-linear data

be constructed as a collection of

randomly distributed set of data item joined together

by using a special pointer (tag). In non-linear Data

structure the relationship of adjacency is not

maintained between the data items.

7

Operations on Data Structures

i. Add an element

ii. Delete an element

iii. Traverse

iv. Sort the list of elements

v. Search for a data element

8

Abstract Data Type

• An abstract data type, sometimes abbreviated

ADT, is a logical description of how we view

the data and the operations that are allowed

without regard to how they will be

implemented.

• By providing this level of abstraction, we are

creating an encapsulation around the data. The

idea is that by encapsulating the details of the

implementation, we are hiding them from the

user‘s view. This is called information hiding.

9

Algorithm Definition

• An Algorithm may be defined as a finite

sequence of instructions each of which has a clear

meaning and can be performed with a finite

amount of effort in a finite length of time.

• The word algorithm originates from the Arabic

word Algorism which is linked to the name of the

Arabic Mathematician AI Khwarizmi.

• AI Khwarizmi is considered to be the first

algorithm designer for adding numbers.

10

Structure of an Algorithm

• An algorithm has the following structure:

– Input Step

–Assignment Step

–Decision Step

–Repetitive Step

–Output Step

11

Properties of an Algorithm

• Finiteness:- An algorithm must terminate after

finite number of steps.

• Definiteness:-The steps of the algorithm must be

precisely defined.

• Generality:- An algorithm must be generic

enough to solve all problems of a particular

class.

• Effectiveness:- The operations of the algorithm

must be basic enough to be put down on pencil

and paper.

Input-Output:- The algorithm must have certain

initial and precise inputs, and outputs that may be

generated both at its intermediate and final steps

13

Algorithm Analysis and Complexity

• The performances of algorithms can be

measured on the scales of Time and Space.

• The Time Complexity of an algorithm or a

program is a function of the running time of

the algorithm or a program.

• The Space Complexity of an algorithm or a

program is a function of the space needed by

the algorithm or program to run to completion.

14

Algorithm Analysis and Complexity

• The Time Complexity of an algorithm can be computed either by an – Empirical or Posteriori Testing

– Theoretical or Apriori Approach

• The Empirical or Posteriori Testing approach calls for implementing the complete algorithm and executes them on a computer for various instances of the problem.

• The Theoretical or Apriori Approach calls for mathematically determining the resources such as time and space needed by the algorithm, as a function of parameter related to the instances of the problem considered.

15

Algorithm Analysis and Complexity

• Apriori analysis computed the efficiency of the

program as a function of the total frequency

count of the statements comprising the

program.

• Example: Let us estimate the frequency count

of the statement x = x+2 occurring in the

following three program segments A, B and C.

16

Total Frequency Count of Program

Segment A

• Program Statements

..…………………

x = x+ 2

….……………….

Total Frequency Count

• Frequency Count

1

1

Time Complexity of Program Segment A is O(1).

17

Total Frequency Count of Program

Segment B

• Program Statements

..…………………

for k = 1 to n do

x = x+ 2;

end

….……………….

Total Frequency Count

• Frequency Count

(n+1)

n

n

……………………

3n+1

Time Complexity of Program Segment B is O(n).

18

Total Frequency Count of Program

Segment C • Program Statements

..…………………

for j = 1 to n do for k = 1 to n do

x = x+ 2;

end

end

….……………….

Total Frequency Count

• Frequency Count

(n+1) (n+1)n n2

n2

n …………………………

3n2 +3n+1

Time Complexity of Program Segment C is O(n2).

19

Asymptotic Notations

• Big oh(O): f(n) = O(g(n)) ( read as f of n is big

oh of g of n), if there exists a positive integer

n0 and a positive number c such that |f(n)| ≤ c

|g(n)| for all n ≥ n0.

• Here g(n) is the upper bound of the function

f(n).

f(n) g(n)

16n3 + 45n2 + 12n n3 f(n) = O(n3 )

34n – 40 n f(n) = O(n)

50 1 f(n) = O(1)

20

Asymptotic Notations

• Omega(Ω): f(n) = Ω(g(n)) ( read as f of n is

omega of g of n), if there exists a positive

integer n0 and a positive number c such that

|f(n)| ≥ c |g(n)| for all n ≥ n0.

• Here g(n) is the lower bound of the function

f(n).

f(n) g(n)

16n3 + 8n2 + 2 n3 f(n) = Ω (n3 )

24n +9 n f(n) = Ω (n)

Asymptotic Notations

• Theta(Θ): f(n) = Θ(g(n)) (read as f of n is

theta of g of n), if there exists a positive

integer n0 and two positive constants c1 and c2

such that c1 |g(n)| ≤ |f(n)| ≤ c2 |g(n)| for all n ≥

n0.

• The function g(n) is both an upper bound and a

lower bound for the function f(n) for all values

of n, n ≥ n0 20

f(n) g(n)

16n3 + 30n2 – 90 n2 f(n) = Θ(n2 )

7. 2n + 30n 2n f(n) = Θ (2n)

22

Asymptotic Notations

• Little oh(o): f(n) = O(g(n)) ( read as f of n is

little oh of g of n), if f(n) = O(g(n)) and f(n) ≠

Ω(g(n)).

f(n) g(n)

18n + 9 n2 f(n) = o(n2) since f(n) = O(n2 ) and f(n) ≠ Ω(n2 ) however f(n) ≠ O(n).

23

Time Complexity

Complexity Notation Description

Constant O(1) Constant number of operations, not depending on the

input data size.

Logarithmic O(logn) Number of operations proportional of log(n) where n

is the size of the input data.

Linear O(n) Number of operations proportional to the input data

size.

Quadratic O(n2 ) Number of operations proportional to the square of

the size of the input data.

Cubic O(n3 ) Number of operations proportional to the cube of the

size of the input data.

Exponential O(2n) Exponential number of operations, fast growing.

O(kn )

O(n!)

Time Complexities of various

Algorithms

24

25

Recursion Examples

Factorial Function

Factorial(Fact, N)

1. If N = 0, then set Fact :=1, and return.

2. Call Factorial(Fact, N-1)

3. Set Fact := N* Fact

4. Return

26

Fibonacci Sequence

Fibonacci(Fib, N)

1. If N=0 or N=1, then: Set Fib:=N, and return.

2. Call Fibonacci(FibA, N-2)

3. Call Fibonacci(FibB, N-1)

4. Set Fib:=FibA + FibB

5. Return

27

Towers of Hanoi

Tower(N, Beg, Aux, End)

1. If N=1, then:

(a) Write: Beg -> End

(b) Return

2. [Move N-1 disks from peg Beg to peg Aux]

Call Tower(N-1, Beg, End, Aux)

3. Write: Beg -> End

4. [Move N-1 disks from peg Aux to peg End]

Call Tower(N-1, Aux, Beg, End)

5. Return

28

Basic Searching Methods

• Search: A search algorithm is a method of

locating a specific item of information in a

larger collection of data.

• There are three primary algorithms used for

searching the contents of an array:

1. Linear or Sequential Search

2. Binary Search

3. Fibonacci Search

29

Linear Search

• Begins search at first item in list, continues

searching sequentially(item by item) through

list, until desired item(key) is found, or until

end of list is reached.

• Also called sequential or serial search.

• Obviously not an

searching ordered

efficient method for

lists like phone

directory(which is ordered alphabetically).

30

Linear Search contd..

• Advantages

1. Algorithm is simple.

2. List need not be ordered in any particular

way.

• Time Complexity of Linear Search is O(n).

31

Recursive Linear Search Algorithm

def linear_Search(l,key,index=0):

if l:

if l[0]==key:

return index

s=linear_Search(l[1:],key,(index+1))

if s is not false:

return s

return false

32

Binary Search

• List must be in sorted order to begin with

Compare key with middle entry of list

For lists with even number of entries, either

of the two middle entries can be used.

• Three possibilities for result of comparison

Key matches middle entry --- terminate

search with success

Key is greater than middle entry ---

matching entry(if exists) must be in upper

part of list (lower part of list can be

discarded from search)

33

Binary Search contd…

Key is less than middle entry ---matching

entry (if exists) must be in lower part of list (

upper part of list can be discarded from

search)

• Keep applying above 2 steps to the

progressively ―reduced‖ lists, until match is

found or until no further list reduction can be

done.

• Time Complexity of Binary Search is O(logn).

34

Fibonacci Search

• Fibonacci Search Technique is a method of

searching a sorted array using a divide and

conquer algorithm that narrows down possible

locations with the aid of Fibonacci numbers.

• Fibonacci search examines locations whose

addresses have lower dispersion, therefore it

has an advantage over binary search in slightly

reducing the average time needed to access a

storage location.

35

Fibonacci Search contd…

search has a complexity of • Fibonacci

O(log(n)).

• Fibonacci search was first devised by

Kiefer(1953) as a minimax search for the

maximum (minimum) of a unimodal function

in an interval.

36

Fibonacci Search Algorithm

• Let k be defined as an element in F, the array

of Fibonacci numbers. n = Fm is the array size.

If the array size is not a Fibonacci number, let

Fm be the smallest number in F that is greater

than n.

• The array of Fibonacci numbers is defined

where Fk+2

= Fk+1 + Fk, when k ≥ 0, F1 = 1, and F0 = 0.

• To test whether an item is in the list of ordered

numbers, follow these steps:

Set k = m.

If k = 0, stop. There is no match; the item is

not in the array.

38

Fibonacci Search Algorithm Contd…

• Compare the item against element in Fk−1.

• If the item matches, stop.

• If the item is less than entry Fk−1, discard

the elements from positions Fk−1 + 1 to n.

Set k = k − 1 and return to step 2.

• If the item is greater than entry Fk−1,

discard the elements from positions 1 to

Fk−1. Renumber the remaining elements

from 1 to Fk−2, set k = k − 2, and return to

step 2.

39

Basic Sorting Methods :-Bubble Sort • First Level Considerations

• To sort list of n elements in ascending order

Pass 1 :make nth element the largest

Pass 2 :if needed make n-1th element the 2nd

largest

Pass 3 :if needed make n-2th element the 3rd

largest Pass n-2: if needed make 3rd n-(n-3)th element the (n-2)th largest Pass n-1 :if needed make 2nd n-(n-2)th element the (n-1)th largest

• Maximum number of passes is (n-1).

40

Bubble Sort

Second Level Considerations

• Pass 1: Make nth element the largest.

Compare each successive pair of elements

beginning with 1st 2nd and ending with n-1th

nth and swap the elements if necessary.

• Pass 2 : Make n-1th element the 2nd largest.

Compare each successive pair of elements

beginning with 1st 2nd and ending with n-2th

n-1th and swap the elements if necessary

Pass n-1:Make 2nd n-(n-2)th element the (n-1)th

largest.

Compare each successive pair of elements

beginning with 1st 2nd and ending with n-(n-1)

th n-(n-2)th 1st 2nd and swap the elements if

necessary.

List is sorted when either of the following occurs

No swapping involved in any pass

Pass n-1:the last pass has been executed

Bubble Sort Example

42

43

Selection Sort

First Level Considerations

• To sort list of n elements in ascending order

• Pass 1: make 1st element the smallest

• Pass 2: make 2nd element the 2nd smallest

• Pass 3: make 3rd element the 3rd smallest

• Pass n-2: make (n-2)th element the (n-2)th smallest

• Pass n-1: make (n-1)th element the (n-1)th smallest

• Number of passes is (n-1).

44

Selection Sort

Second Level Considerations • Pass 1: Make 1st element the smallest

Examine list from 1st to last element locate element with smallest value and swap it with the 1st element where appropriate .

• Pass 2: Make 2nd element the 2nd smallest Examine list from 2nd to last element locate element with smallest value and swap it with the 2nd element where appropriate.

• Pass n-1: Make (n-1)th element the (n-1)th smallest Examine list from (n-1)th to last element locate element with smallest value and swap it with the n-1th element where appropriate.

Selection Sort Example

45

46

Insertion Sort

First Level Considerations

To sort list of n items (stored as 1D array) in ascending order

• NOTE: 1-element sub-array (1st) is always sorted

• Pass 1: make 2-element sub-array (1st 2nd) sorted

• Pass 2 :make 3-element sub-array (1st 2nd 3rd) sorted

• Pass 3 :make 4-element sub-array (1st 4th) sorted

• Pass n-2: make n-1-element sub-array (1st (n-1)th) sorted

• Pass n-1: make entire n-element array (1st nth) sorted

• Number of passes is (n-1)

Insertion Sort Example

47

48

Quick Sort

It uses Divide-and-Conquer approach:

1. Divide: partition A[p..r] into two sub-arrays A[p..q-1] and A[q+1..r] such that each element of A[p..q-1] is ≤ A[q], and each element of A[q+1..r] is ≥ A[q]. Compute q as part of this partitioning.

2. Conquer: sort the sub-arrays A[p..q-1] and

A[q+1..r] by recursive calls to QUICKSORT.

3. Combine: the partitioning and recursive sorting leave us with a sorted A[p..r] – no work needed here.

The Pseudo-Code of Quick Sort

49

Quick Sort Contd…..

50

51

Quick sort Analysis

• Best case running time: O(n log2n)

• Worst case running time: O(n2)!!!

Quick Sort Example

52

53

Merge Sort • Merge-sort is a sorting algorithm based on the divide-and-

conquer paradigm

• Like heap-sort

It uses a comparator

It has O(n log n) running time

• Unlike heap-sort

It does not use an auxiliary priority queue

It accesses data in a sequential manner (suitable to sort data

on a disk)

• Merge-sort on an input sequence S with n elements consists of three steps: Divide: partition S into two sequences S1 and S2 of about n/2 elements each

Recur: recursively sort S1 and S2

Conquer: merge S1 and S2 into a unique sorted sequence

54

Merge Sort Algorithm

Algorithm mergeSort(S, C)

Input sequence S with n elements, comparator C

Output sequence S sorted according to C

if S.size() > 1

(S1, S2) partition(S, n/2)

mergeSort(S1, C)

mergeSort(S2, C)

S merge(S1, S2)

Merge Sort Example

55

Merge Sort Example Contd…

56

57

Analysis of Merge Sort

• The height h of the merge-sort tree is O(log n)

– at each recursive call we divide in half the sequence,

• The overall amount or work done at the nodes of depth

i is O(n)

– we partition and merge 2i sequences of size n/2i

– we make 2i+1 recursive calls

• Thus, the total running time of merge-sort is O(n log n)

Comparison of Sorting Algorithms

58