1 cscd 326 data structures i algorithm analysis. 2 algorithms and program design is it enough if an...

1

CSCD 326 Data Structures IAlgorithm Analysis

2

Algorithms and Program Design

Is it enough if an algorithm implementation "just works" ?

A working algorithm that is poorly designed could execute slowly and thus make one part of a software system sluggish and unresponsive to users.Thus, a central part of computer science is using tools to compare the execution speed of algorithms.

This is usually done before an algorithm is implemented in a programming language.

3

Algorithm Analysis

Mathematical tools and techniques that allow comparison of different solutions to the same problem.

These techniques analyze algorithms independently of specific implementations, computers, or data.These techniques do not provide fine details about the speed of a solution but allow determination of whether one solution is much (order of magnitude) slower or faster than another.

4

The Growth Rate Function

The simplest way to analyze an algorithm is to count the operations it performs.

This is usually not practical since the number of operations depends on the number of data elements to be handled.So - derive a function of n (the data set size) that describes the number of operations as a function of n.This function f(n) is called the growth rate function of the algorithm.

5

Finding f(n)

for(i = 0 ; i < n ; i++) for(j = 0 ; j < n ; j++) { System.out.println("i: "+i); System.out.println("j: "+j); }

Involves analysisof the number of loop iterationsThe outer loop executes:

one setup statement -- i = 0 once.one comparison on each of n iterationsone auto-increment n timesone final comparison to end the loopone for statement (the inner for) n times what loop does.so f(n) = 2n + n*inner + 2

6

Finding f(n)

for(i = 0 ; i < n ; i++) for(j = 0 ; j < n ; j++) { System.out.println("i: "+i); System.out.println("j: "+j); }

The inner loop executes:one setup statement -- j = 0 once.one comparison on each of n iterationsone auto-increment n timesone final comparison to end the looptwo output statements n times.so f(n) = 4n + 2

7

Finding f(n)

for(i = 0 ; i < n ; i++) for(j = 0 ; j < n ; j++) { System.out.println("i: "+i); System.out.println("j: "+j); }

But the inner loop is done once on each iteration of the outer loop so the final f(n) is:

f(n) = 2n +n*(4n + 2) + 2 f(n) = 4n2 + 4n + 2

8

Big - O Notation

Full definition of Big-O notation:Let f and g be functions on the non-negative integers.Then f(n) is of order at most g(n) :

denoted f(n) = O(g(n))This means that there exists a positive constant C such that:

|f(n)| <= C(g(n))for all sufficiently large positive integers n.

9

Big - O Notation

What it really means:for f(n) = 4n2 + 4n + 2 - g(n) is the dominant term of the polynomial - n2 in this case and thus,4n2 + 4n + 2 <= C (n2) is true if n >= 1 and C

= 10.Note that the definition of Big-O notation only holds true for sufficiently large values of n.So an algorithm whose growth rate function is:f(n) = 4n2 + 4n + 2 is said to have time

complexity O(n2).Note that Big-O notation provides only a rough categorization of algorithm speed.

10

Comparison of Algorithms with Different Time Complexities

Execution times assume:1 sec execution time per instructionExactly f(n) instructions will be carried out.n = 104 = 10000

log2n

Execution Timef(n)

nn log2nn2

n3

2n

1.32877 * 10-5 sec

0.1 sec0.13 sec100 sec — 108 µsec106 sec — 11 days 13:46:40>102992 years

11

Graph of Growth Rate Functions

12

Algorithm Time Complexities

Complexity NameBig-O

O(log n)

O(n)O(n log n)O(n2)

O(n3)O(2n)

Logarithmic

LinearLog-Linear

QuadraticCubicExponential

O(1) Constant

Example

Swap

Binary Search

Linear SearchThe “Good” Sort

Selection SortMatrix MultiplicationTowers of Hanoi

13

Algorithms and Analysis

In the following section we will re-examine some basic searching and sorting algorithms and analyze them.

14

Search and Sort Testing Programpublic class SortSearchTest {

public static void main(String[] args) {

Integer[] values = new Integer[10];

values[0] = new Integer(15); values[1] = new Integer(9);

values[2] = new Integer(13); values[3] = new Integer(20);

values[4] = new Integer(5); values[5] = new Integer(0);

values[6] = new Integer(7); values[7] = new Integer(10);

values[8] = new Integer(3); values[9] = new Integer(2);

printem(values);

System.out.println("2 found at: "+

linearSearch(values, new Integer(2)));

bubbleSort(values, values.length);

printem(values);

}

public static void printem(Comparable[] anArray){

for(int i = 0; i < anArray.length; i++) {

System.out.println("array["+i+"]: "+anArray[i]);

}

}

15

Linear (Sequential) Search

public static int linearSearch(Comparable[ ] anArray, Comparable target) { int index = 0;

while(index < anArray.length) { if (target.compareTo(anArray[index]) == 0 ) return index; index++; } return -1;}

The algorithm basically steps through the array until it finds its target element (using target.compareTo()) or it steps through the entire array - returns the index of target or -1 if target is not found

16

Analysis of Linear Search

Worst Case: target is not foundf(n) = 3n +1 +2 so the algorithm is O(n)

Average case: harder to analyzetarget will be found, on average, after n/2 loop iterationsso f(n) = 3(n/2) + 3 and the algorithm is still O(n)

Best case: target is the first item in the array

only one loop iteration so algorithm is O(1)

17

Binary Search

public static int binarySearch(Comparable[ ] anArray, Comparable target) { int index, cmp; int left = 0, right = anArray.length - 1;

while( left <= right ) { index = (left + right) / 2; cmp = target.compareTo(anArray[index]; if( cmp < 0)

right = index - 1; if( cmp > 0)

left = index + 1; else

return index; } return -1; // target not found}

18

Binary Search (2)

Binary Search depends on two things:1) Random access memory (an array) is used so we can go directly to the center of the array.2) The array must be sorted.

19

Binary Search Trace

0 2 3 5 7 9 10 13 15 20[0] [1] [2] [3] [4] [5] [6] [7] [8] [9]

Pass 1 -left = 0 right = 9 index = 4 target = 3

0 2 3 5 7 9 10 13 15 20[0] [1] [2] [3] [4] [5] [6] [7] [8] [9]

Pass 2 -left = 0 right = 3 index = 1

0 2 3 5 7 9 10 13 15 20[0] [1] [2] [3] [4] [5] [6] [7] [8] [9]

Pass 3 -left = 2 right = 3 index = 2

return 2 (successful search)

20

Binary Search Trace 2

0 2 3 5 7 9 10 13 15 20[0] [1] [2] [3] [4] [5] [6] [7] [8] [9]

Pass 1 -left = 0 right = 9 index = 4 target = 1

0 2 3 5 7 9 10 13 15 20[0] [1] [2] [3] [4] [5] [6] [7] [8] [9]

Pass 2 -left = 0 right = 3 index = 1

0 2 3 5 7 9 10 13 15 20[0] [1] [2] [3] [4] [5] [6] [7] [8] [9]

Pass 3 -left = 0 right = 0 index = 0

Pass 4 -left = 1 right = 0return -1 -- unsuccessful search

21

Binary Search Analysis

Worst case - target is not found while( left <= right ) { index = (left + right) / 2;

On each loop iteration the array is (ideally) divided in half.To determine f(n) consider number of items left in the array on each iteration:

32 16 8 4 2 01

Pass 1 Pass 2 Pass 3 Pass 4 Pass 5 Pass 6 Pass 7

22

Binary Search Analysis

32 16 8 4 2 01

Pass 1 Pass 2 Pass 3 Pass 4 Pass 5 Pass 6 Pass 7

nNumber of compares

32168421

765432

log2n

54321...

23

Binary Search Analysis

The relation between n and number of comparisons for n = 2k is:f(n) = log2n + 2

Or for non-integer powers of 2:f(n) = log2n + 2

And the time complexity is:O(log2n)

24

Bubblesortpublic static void bubbleSort(Comparable[] theArray, int n) {// ---------------------------------------------------

// Sorts the items in an array into ascending order.

// Precondition: theArray is an array of n items.

// Postcondition: theArray is sorted into ascending order.

//NOTE: n specifies size and is only needed if you aren’t sorting whole array

// ---------------------------------------------------

for (int pass = 1; pass < n; ++pass) { // Invariant: theArray[n+1-pass..n-1] is sorted

// and > theArray[0..n-pass]

for (int index = 0; index < n-pass; ++index) { // Invariant: theArray[0..index-1] <= theArray[index]

int nextIndex = index + 1;

if (theArray[index].compareTo(theArray[nextIndex])>0){ // exchange items

Comparable temp = theArray[index];

theArray[index] = theArray[nextIndex];

theArray[nextIndex] = temp;

} // end if

} // end for // Assertion: theArray[0..n-pass-1] < theArray[n-pass]

} // end for

} // end bubbleSort

25

Bubblesort Trace

15 9 13 20 5 0 7 10 3 2[0] [1] [2] [3] [4] [5] [6] [7] [8] [9]

Pass = 1

After index = 0 9 15 13 20 5 0 7 10 3 2[0] [1] [2] [3] [4] [5] [6] [7] [8] [9]

After index = 1 9 13 15 20 5 0 7 10 3 2[0] [1] [2] [3] [4] [5] [6] [7] [8] [9]

After index = 2 9 13 15 20 5 0 7 10 3 2[0] [1] [2] [3] [4] [5] [6] [7] [8] [9]

After index = 3 9 13 15 5 20 0 7 10 3 2[0] [1] [2] [3] [4] [5] [6] [7] [8] [9]

After index = 4 9 13 15 5 0 20 7 10 3 2[0] [1] [2] [3] [4] [5] [6] [7] [8] [9]

26

Bubblesort Trace (2)

9 13 15 5 0 20 7 10 3 2[0] [1] [2] [3] [4] [5] [6] [7] [8] [9]

Pass = 1

After index = 5 9 13 15 5 0 7 20 10 3 2[0] [1] [2] [3] [4] [5] [6] [7] [8] [9]

After index = 6 9 13 15 5 0 7 10 20 3 2[0] [1] [2] [3] [4] [5] [6] [7] [8] [9]

After index = 7 9 13 15 5 0 7 10 3 20 2[0] [1] [2] [3] [4] [5] [6] [7] [8] [9]

After index = 8 9 13 15 5 0 7 10 3 2 20[0] [1] [2] [3] [4] [5] [6] [7] [8] [9]

27

Bubblesort Trace (3)

After Pass = 1 9 13 15 5 0 7 10 3 2 20[0] [1] [2] [3] [4] [5] [6] [7] [8] [9]

After Pass = 2 9 13 5 0 7 10 3 2 15 20[0] [1] [2] [3] [4] [5] [6] [7] [8] [9]

After Pass = 3 9 5 0 7 10 3 2 13 15 20[0] [1] [2] [3] [4] [5] [6] [7] [8] [9]

After Pass = 4 5 0 7 9 3 2 10 13 15 20[0] [1] [2] [3] [4] [5] [6] [7] [8] [9]

After Pass = 5 0 5 7 3 2 9 10 13 15 20[0] [1] [2] [3] [4] [5] [6] [7] [8] [9]

After Pass = 6 0 5 3 2 7 9 10 13 15 20[0] [1] [2] [3] [4] [5] [6] [7] [8] [9]

28

Bubblesort Trace (4)

After Pass = 7

After Pass = 8

After Pass = 9

0 3 2 5 7 9 10 13 15 20[0] [1] [2] [3] [4] [5] [6] [7] [8] [9]

0 2 3 5 7 9 10 13 15 20[0] [1] [2] [3] [4] [5] [6] [7] [8] [9]

0 2 3 5 7 9 10 13 15 20[0] [1] [2] [3] [4] [5] [6] [7] [8] [9]

29

Bubblesort Analysis

Basic loop structure:for ( int pass = 1 ; pass < n ; ++pass ) ... for ( int index = 0 ; index < n-pass ; ++index ) ...The outer loop iterates n - 1 times.Inner loop iterations depends on which iteration of the outer loop.So to derive the growth rate function f(n) we must look at each outer loop iteration.

30

Bubblesort Analysis (2)

Make a table of inner loop iterations for each outer loop iteration:

Value of passNumber of innerloop iterations

1234...n -1

n -1n -2n -3n -4...1

31

BubbleSort : Analysis (3)

Now sum the column containing the number of inner loop iterations:

(n -1) + (n -2) + (n -3) + . . . + 1 = n (n -1) / 2

And thus the growth rate function f(n) is:

f(n) = n2 / 2 - n / 2and the dominant term is n2

So BubbleSort has time complexity O (n2)Are there any special cases (best, worst, average)?

32

Improved Bubblesortpublic static void bubbleSort(Comparable[] theArray, int n) {// Sorts the items in an array into ascending order.

// Precondition: theArray is an array of n items.

// Postcondition: theArray is sorted into ascending order.

boolean sorted = false; // false when swaps occur

for (int pass = 1;!sorted; ++pass) { // Invariant: theArray[n+1-pass..n-1] is sorted and > theArray[0..n-pass]

sorted = true; // assume sorted

for (int index = 0; index < n-pass; ++index) { // Invariant: theArray[0..index-1] <= theArray[index]

int nextIndex = index + 1; //avoid multiple addition operations

if (theArray[index].compareTo(theArray[nextIndex]) > 0) {

Comparable temp = theArray[index];

theArray[index] = theArray[nextIndex];

theArray[nextIndex] = temp;

sorted = false; // signal exchange

} // end if

} // end for // Assertion: theArray[0..n-pass-1] < theArray[n-pass]

} // end for

} // end bubbleSort

33

Improved Bubblesort (2)

The boolean variable (sorted) will stay true if no swaps are made while iterating the inner loop.

If this ever happens the sort is complete and there is no need to iterate further.Consider the following array:

Here the 20 will move to the bottom of the array on the first pass in the outer loop and after the second pass the outer loop exits.

20 0 2 3 5 7 9 10 13 15[0] [1] [2] [3] [4] [5] [6] [7] [8] [9]

34

Improved Bubblesort Analysis

Does this change the O(n2) time complexity?

No, but it does force a case by case analysisWorst case:

The array is in reverse order as in the following:

20 15 13 10 9 7 5 3 2 0[0] [1] [2] [3] [4] [5] [6] [7] [8] [9]

35

Improved Bubblesort Analysis

Worst case: Here each value is bubbled to the bottom of the array and the full n2 / 2 - n / 2 iterations must be done.Time complexity is O(n2)

20 15 13 10 9 7 5 3 2 0[0] [1] [2] [3] [4] [5] [6] [7] [8] [9]

36

Improved Bubblesort Analysis

Best case: Values are already sorted.Here the outer loop will iterate only once, since no swaps will be done sorted will never get set false and the outer loop will exit.Time complexity for this case is O(n)

0 2 3 5 7 9 10 13 15 20[0] [1] [2] [3] [4] [5] [6] [7] [8] [9]

37

Improved Bubblesort Analysis

Average case: Values are random.Outer loop iteratations depend on the specific data and although it may exit early it is just as likely not to.Time complexity for this case is still O(n2)

15 9 13 20 5 0 7 10 3 2[0] [1] [2] [3] [4] [5] [6] [7] [8] [9]

38

Selection Sortpublic static void selectionSort(Comparable[ ] theArray, int n) {// Sorts the items in an array into ascending order.

// Precondition: theArray is an array of n items.

// Postcondition: theArray is sorted into ascending order.

// Calls: indexOfLargest.

// ---------------------------------------------------

// last = index of the last item in the subarray of items yet to be sorted

// largest = index of the largest item found

for (int last = n-1; last >= 1; last--) { // Invariant: theArray[last+1..n-1] is sorted and > theArray[0..last]

// select largest item in theArray[0..last]

int largest = indexOfLargest(theArray, last+1);

// swap largest item theArray[largest] with theArray[last]

Comparable temp = theArray[largest];

theArray[largest] = theArray[last];

theArray[last] = temp;

} // end for

} // end selectionSort

39

Selection Sort : indexOfLargestprivate static int indexOfLargest(Comparable[ ] theArray, int size) {// ---------------------------------------------------

// Finds the largest item in an array.

// Precondition: theArray is an array of size items; size >= 1.

// Postcondition: Returns the index of the largest item in the array.

// --------------------------------------------------

int indexSoFar = 0; // index of largest item found so far

// Invariant: theArray[indexSoFar]>=theArray[0..currIndex-1]

for (int currIndex = 1; currIndex < size; ++currIndex) {

if (theArray[currIndex].compareTo( theArray[indexSoFar] ) > 0) {

indexSoFar = currIndex;

} // end if

} // end for

return indexSoFar; // index of largest item

} // end indexOfLargest

--NOTE: some versions of Selection Sort include this code as part of the Selection Sort method itself

40

Selection Sort : Trace

15 9 13 20 5 0 7 10 3 2[0] [1] [2] [3] [4] [5] [6] [7] [8] [9]

last = 9largest = 3

15 9 13 2 5 0 7 10 3 20[0] [1] [2] [3] [4] [5] [6] [7] [8] [9]

last = 8largest = 0

3 9 13 2 5 0 7 10 15 20[0] [1] [2] [3] [4] [5] [6] [7] [8] [9]

last = 7largest = 2

3 9 10 2 5 0 7 13 15 20[0] [1] [2] [3] [4] [5] [6] [7] [8] [9]

last = 6largest = 2

3 9 7 2 5 0 10 13 15 20[0] [1] [2] [3] [4] [5] [6] [7] [8] [9]

last = 5largest = 1

3 0 7 2 5 9 10 13 15 20[0] [1] [2] [3] [4] [5] [6] [7] [8] [9]

last = 4largest = 2

41

Selection Sort : Trace

3 0 5 2 7 9 10 13 15 20[0] [1] [2] [3] [4] [5] [6] [7] [8] [9]

last = 3largest = 2

3 0 2 5 7 9 10 13 15 20[0] [1] [2] [3] [4] [5] [6] [7] [8] [9]

last = 2largest = 0

2 0 3 5 7 9 10 13 15 20[0] [1] [2] [3] [4] [5] [6] [7] [8] [9]

last = 1largest = 0

0 2 3 5 7 9 10 13 15 20[0] [1] [2] [3] [4] [5] [6] [7] [8] [9]

outer loopexits

42

Selection Sort : Analysis

The outer loop:

for (int last = n-1; last >= 1; last--)

Iterates n -1 times and each iteration calls indexOfLargest:

int largest = indexOfLargest(theArray, last+1);

The inner loop (in indexOfLargest):

Number of iterations depends on which outer loop call (last + 1 is passed into size :

for (int currIndex = 1; currIndex < size; ++currIndex)

43

Selection Sort : Analysis (2)

Make a table of inner loop iterations for each outer loop iteration:

Value of lastNumber of innerloop iterations

n -1n -2n -3n -4...1

n -1n -2n -3n -4...1

44

Selection Sort : Analysis (3)

Now sum the column containing the number of inner loop iterations:

(n -1) + (n -2) + (n -3) + . . . + 1 = n (n -1) / 2And thus the growth rate function f(n) is:

f(n) = n2 / 2 - n / 2and the dominant term is n2

So Selection Sort has time complexity O (n2)Are there any special cases (best, worst, average)?

45

Insertion Sortpublic static void insertionSort(Comparable[] theArray, int n) {// ---------------------------------------------------// Sorts the items in an array into ascending order.// Precondition: theArray is an array of n items.// Postcondition: theArray is sorted into ascending order.// --------------------------------------------------- // unsorted = first index of the unsorted region, // loc = index of insertion in the sorted region, // nextItem = next item in the unsorted region

for (int unsorted = 1; unsorted < n; ++unsorted) { // Invariant: theArray[0..unsorted-1] is sorted

Comparable nextItem = theArray[unsorted]; int loc = unsorted; while ((loc > 0) && (theArray[loc-1].compareTo(nextItem) > 0)) { // shift theArray[loc-1] to the right

theArray[loc] = theArray[loc-1]; loc--; //correction from book’s code (both printed and electronic)

} // end while // insert nextItem into sorted region

theArray[loc] = nextItem; } // end for

} // end insertionSort

46

Insertion Sort : Trace

15 9 13 20 5 0 7 10 3 2[0] [1] [2] [3] [4] [5] [6] [7] [8] [9]unsorted = 1

nextitem = 9loc = 0

9 15 13 20 5 0 7 10 3 2[0] [1] [2] [3] [4] [5] [6] [7] [8] [9]unsorted = 2

nextitem = 13loc = 1

9 13 15 20 5 0 7 10 3 2[0] [1] [2] [3] [4] [5] [6] [7] [8] [9]unsorted = 3

nextitem = 20loc = 3

9 13 15 20 5 0 7 10 3 2[0] [1] [2] [3] [4] [5] [6] [7] [8] [9]unsorted = 4

nextitem = 5loc = 0

5 9 13 15 20 0 7 10 3 2[0] [1] [2] [3] [4] [5] [6] [7] [8] [9]unsorted = 5

nextitem = 0loc = 0

47

Insertion Sort : Trace (inner loop)

5 9 13 15 20 0 7 10 3 2[0] [1] [2] [3] [4] [5] [6] [7] [8] [9]unsorted = 5

nextitem = 0loc = 5

5 9 13 15 20 20 7 10 3 2[0] [1] [2] [3] [4] [5] [6] [7] [8] [9]unsorted = 5

nextitem = 0loc = 4

5 9 13 15 15 20 7 10 3 2[0] [1] [2] [3] [4] [5] [6] [7] [8] [9]unsorted = 5

nextitem = 0loc = 3

5 9 13 13 15 20 7 10 3 2[0] [1] [2] [3] [4] [5] [6] [7] [8] [9]unsorted = 5

nextitem = 0loc = 2

[0] [1] [2] [3] [4] [5] [6] [7] [8] [9]

5 9 9 13 15 20 7 10 3 2unsorted = 5nextitem = 0loc = 1

5

0

9 13 15 20 7 10 3 2[0] [1] [2] [3] [4] [5] [6] [7] [8] [9]unsorted = 5

nextitem = 0loc = 0

5

48

Insertion Sort : Analysis

The outer loop:for (int unsorted = 1; unsorted < n; ++unsorted)

Iterates n -1 times and each iteration selects nextItem to be moved into position.

The inner loop:Iterates until nextItem is in its correct position or until the postion into which nextItem will be placed (loc) is at the top of the array:

while ((loc > 0) && (theArray[loc-1].compareTo(nextItem) > 0))

49

Insertion Sort : Analysis (2)

The number of inner loop iterations depends on the value of nextItem:

To analyze use worst case, average case, best case.Worst case: array data is in reverse order.

The inner loop will always iterate until nextItem is placed at the top of the array since nextItem is always smaller than the values above.

50

Insertion Sort : Analysis (2)

Worst case inner loop iterations:

Value of unsortedNumber of innerloop iterations

1234...n -1

1234...n -1

51

Insertion Sort : Analysis (3)

Now sum the column containing the number of inner loop iterations:

(n -1) + (n -2) + (n -3) + . . . + 1 = n (n -1) / 2

And thus the growth rate function f(n) is:

f(n) = n2 / 2 - n / 2and the dominant term is n2

So Insertion Sort has time complexity O (n2) in the worst case.

52

Insertion Sort : Analysis (4)

The number of inner loop iterations depends on the value of nextItem:

To analyze use worst case, average case, best case.Best case: array data is in sorted order.

The inner loop will never iterate since nextItem is always larger than the values above.

53

Insertion Sort : Analysis (5)

Now the outer loop controls time complexity:

The outer loop still executes n -1 times.So Insertion Sort has time complexity O (n) in the best case.The average case lies somewhere between O(n) and O(n2) and depends on the specific data values in the array.So - as the array is more nearly sorted the time complexity of Insertion Sort approaches O(n).

54

Shell Sort - Introduction

More properly, Shell’s SortCreated in 1959 by Donald ShellLink to a local copy of the article:Donald Shell, “A High-Speed Sorting Procedure”, Communications of the ACM Vol 2, No. 7 (July 1959), 30-32 Originally Shell built his idea on top of Bubble Sort (link to article flowchart), but it has since been transported over to Insertion Sort.

55

Shell Sort -General Description

Essentially a segmented insertion sortDivides an array into several smaller non-contiguous segments The distance between successive elements in one segment is called a gap.Each segment is sorted within itself using insertion sort.Then resegment into larger segments (smaller gaps) and repeat sort.Continue until only one segment (gap = 1) - final sort finishes array sorting.

56

Shell Sort -Background

General Theory:Makes use of the intrinsic strengths of Insertion sort. Insertion sort is fastest when:

The array is nearly sorted.The array contains only a small number of data items.

Shell sort works well because:It always deals with a small number of elements.Elements are moved a long way through array with each swap and this leaves it more nearly sorted.

57

Shell Sort - example

80 93 60 6812 8542 30 10

Initial Segmenting Gap = 4

10 30 60 6812 8542 93 80

58

Shell Sort - example (2)

10 30 60 6812 8542 93 80

Resegmenting Gap = 2

10 12 42 6830 9360 85 80

59

Shell Sort - example (3)

10 12 30 8042 8560 68 93

10 12 42 6830 9360 85 80

Resegmenting Gap = 1

60

Gap Sequences for Shell Sort

The sequence h1, h2, h3,. . . , ht is a sequence of increasing integer values which will be used as a sequence (from right to left) of gap values.

Any sequence will work as long as it is increasing and h1 = 1.

For any gap value hk we have A[i] <= A[i + hk]

An array A for which this is true is hk sorted.

An array which is hk sorted and is then hk-1 sorted remains hk sorted.

61

Shell Sort - Ideal Gap Sequence

Although any increasing sequence will work ( if h1 = 1):

Best results are obtained when all values in the gap sequence are relatively prime (sequence does not share any divisors).Obtaining a relatively prime sequence is often not practical in a program so practical solutions try to approximate relatively prime sequences.

62

Shell Sort - Practical Gap Sequences

Three possibilities presented:1) Shell's suggestion - first gap is N/2 - successive gaps are previous value divided by 2.Odd gaps only - like Shell method except if division produces an even number add 1.

better performance than 1) since all odd values eliminates the factor 2.

2.2 method - like Odd gaps method (add 1 to even division result) but use a divisor of 2.2 and truncate.

best performance of all - most nearly a relatively prime sequence.

63

Shell Sort - Added Gap Sequence

Donald Knuth, in his discussion of Shell’s Sort, recommended another sequence of gaps.h0 = 1

hj+1 = hj * 3 + 1

Find the hj > n, then start with hj/3

64

Link to the Java program that generated the above data.

65

Shell Sort - Time Complexity

Time complexity: O(nr) with 1 < r < 2This is better than O(n2) but generally worse than O(n log2n).

66

Shellsort - Code

public static voidshellSort( Comparable[ ] theArray, int n ) {// shellSort: sort first n items in array theArray

for( int gap = n / 2; gap > 0; gap = gap / 2 ) for( int i = gap; i < n; i++ ) { Comparable tmp = theArray[ i ]; int j = i;

for( ; j >= gap && tmp.compareTo(theArray[ j - gap ]) < 0 ; j -= gap ) theArray[ j ] = theArray[ j - gap ]; theArray[ j ] = tmp; }}

67

ShellSort -Trace (gap = 4)

for( int gap = n / 2; gap > 0; gap = gap / 2 ) for( int i = gap; i < n; i++ ) { Comparable tmp = theArray[ i ]; int j = i; for( ; j >= gap && tmp.compareTo(theArray[ j - gap ]) < 0 ; j -= gap ) theArray[ j ] = theArray[ j - gap ]; theArray[ j ] = tmp; }

80 93 60 6812 8542 30 10

[0] [1] [2] [3] [4] [5] [6] [7] [8]

n: 9gap: 4

i:j:

theArray

68

ShellSort -Trace (gap = 2)

for( int gap = n / 2; gap > 0; gap = gap / 2 ) for( int i = gap; i < n; i++ ) { Comparable tmp = theArray[ i ]; int j = i; for( ; j >= gap && tmp.compareTo(theArray[ j - gap ]) < 0 ; j -= gap ) theArray[ j ] = theArray[ j - gap ]; theArray[ j ] = tmp; }

[0] [1] [2] [3] [4] [5] [6] [7] [8]

n: 9gap: 2

i:j:

theArray 10 30 60 6812 8542 93 80

69

ShellSort -Trace (gap = 1)

for( int gap = n / 2; gap > 0; gap = gap / 2 ) for( int i = gap; i < n; i++ ) { Comparable tmp = theArray[ i ]; int j = i; for( ; j >= gap && tmp.compareTo(theArray[ j - gap ]) < 0 ; j -= gap ) theArray[ j ] = theArray[ j - gap ]; theArray[ j ] = tmp; }

[0] [1] [2] [3] [4] [5] [6] [7] [8]

n: 9gap: 1

i:j:

theArray 10 12 42 6830 9360 85 80