searching and sorting searching: sequential, binary sorting: selection, insertion, shell
DESCRIPTION
Searching an Unsorted List n Finding out whether an item is in a list contains method n Linear search: loop/recur thru list looking at each item »stop when item found or no more list to look at to iterLinS(arr, item): for i = 0..arr.len-1: if arr[i]=item: return true return false to recLinS(arr, len, item): if len = 0: return false if arr[len-1] = item: return true; return recLinS(arr, len-1, item)TRANSCRIPT
Searching and SortingSearching and Sorting
Searching: Sequential, BinarySearching: Sequential, BinarySorting: Selection, Insertion, ShellSorting: Selection, Insertion, Shell
OutlineOutline
SearchingSearching unsorted listsunsorted lists sorted listssorted lists Java’s Comparable interfaceJava’s Comparable interface
SortingSorting insertioninsertion selectionselection shellshell
Searching an Unsorted ListSearching an Unsorted List
Finding out whether an item is in a listFinding out whether an item is in a list contains methodcontains method
Linear search:Linear search: loop/recur thru list looking at each itemloop/recur thru list looking at each item
» stop when item found or no more list to look atstop when item found or no more list to look at
to iterLinS(arr, item): for i = 0..arr.len-1: if arr[i]=item: return true return false
to recLinS(arr, len, item): if len = 0: return false if arr[len-1] = item: return true; return recLinS(arr, len-1, item)
Linear SearchLinear Search
Worst case? It’s not there!Worst case? It’s not there! total of N comparisons total of N comparisons O(N) O(N)
But if it is there?But if it is there? best case: it’s first (1 comparison): O(1)best case: it’s first (1 comparison): O(1) worst case: it’s last (N comparisons): O(N)worst case: it’s last (N comparisons): O(N) average case: any position equally likely!average case: any position equally likely!
» 1 comparison, 2 comparisons, 3, 4, …, N1 comparison, 2 comparisons, 3, 4, …, N» average = (1 + 2 + 3 + … + N) / Naverage = (1 + 2 + 3 + … + N) / N
= (N(N+1)/2) / N = (N+1)/2: O(N)= (N(N+1)/2) / N = (N+1)/2: O(N)
Best, Worst, AverageBest, Worst, Average
Number of operations often variableNumber of operations often variable depends on specific values being useddepends on specific values being used no single formula that states amount of work!no single formula that states amount of work!
Formulas for best, worst may be easy to getFormulas for best, worst may be easy to get Formula for average usually a bit harderFormula for average usually a bit harder Most interested in worst and averageMost interested in worst and average
best is best is nicenice, but we don’t expect/worry about it, but we don’t expect/worry about it expectexpect average, average, prepare for prepare for worstworst
Searching a Sorted ListSearching a Sorted List
Improved linear search:Improved linear search: stop when we pass where it should have beenstop when we pass where it should have been
should have been before 17should have been before 17so it must not be thereso it must not be there
Saves very little time, actuallySaves very little time, actually best & worst still the samebest & worst still the same average still O(N)average still O(N)
5 7 9 12 13 17 22 25 27 28 42
15
Cutting in HalfCutting in Half
Binary search better for sorted lists:Binary search better for sorted lists: look at middle itemlook at middle item if it is bigger than what we’re looking for, then if it is bigger than what we’re looking for, then
we only need to look in the lower halfwe only need to look in the lower half if it’s smaller than what we’re looking for, then if it’s smaller than what we’re looking for, then
we only need to look in the upper halfwe only need to look in the upper half if it’s what we’re looking for – return the indexif it’s what we’re looking for – return the index
Binary SearchBinary Search
Find midpoint, compare it to the itemFind midpoint, compare it to the item too big too big search lower part of array search lower part of array too small too small search upper part of array search upper part of array otherwise otherwise just right! just right!
5 7 9 12 13 17 22 25 27 28 42
9
Binary SearchBinary Search
Repeat until found…Repeat until found… too big too big search lower part of array search lower part of array too small too small search upper part of array search upper part of array otherwise otherwise just right! just right!
5 7 9 12 13 17 22 25 27 28 42
9
Binary SearchBinary Search
Find midpoint, compare it to the itemFind midpoint, compare it to the item too big too big search lower part of array search lower part of array too small too small search upper part of array search upper part of array otherwise otherwise just right! just right!
5 7 9 12 13 17 22 25 27 28 42
4
Binary SearchBinary Search
Repeat until found…Repeat until found… too big too big search lower part of array search lower part of array too small too small search upper part of array search upper part of array otherwise otherwise just right! just right!
5 7 9 12 13 17 22 25 27 28 42
4
Binary SearchBinary Search
Repeat until found…Repeat until found… or until nowhere left to lookor until nowhere left to look (return fail)(return fail)
5 7 9 12 13 17 22 25 27 28 42
4
Binary SearchBinary Search
int binaryFind(T item, T v[], int lo, int hi)int binaryFind(T item, T v[], int lo, int hi)if (lo > hi)if (lo > hi)
return –1;return –1; // not found// not foundint mid = lo + (hi – lo)/2;int mid = lo + (hi – lo)/2;if (item<v[mid])if (item<v[mid])
return binaryFind(item, v, lo, mid–1);return binaryFind(item, v, lo, mid–1);if (v[mid]<item)if (v[mid]<item)
return binaryFind(item, v, mid+1, hi);return binaryFind(item, v, mid+1, hi);return mid;return mid; // found// found
Binary Search in JavaBinary Search in Java
Want it to work for any kind of arrayWant it to work for any kind of array need a generic methodneed a generic method
» similar to generic class: public class Pair<T>similar to generic class: public class Pair<T> add <T> before return typeadd <T> before return typepublic static <T> int binaryFind(T item, T[] v, int lo, int public static <T> int binaryFind(T item, T[] v, int lo, int
hi)hi) problem: doesn’t compileproblem: doesn’t compileif (if (item < v[mid]item < v[mid]))bad operand types for binary operator ‘<’!
Comparison in JavaComparison in Java
The < operator only works for numbers:The < operator only works for numbers:if (if (“this string” < “that string”“this string” < “that string”))
Need to use compareTo methodNeed to use compareTo methodif (“this string”.compareTo(“that string”) < 0)if (“this string”.compareTo(“that string”) < 0)
The T for binaryFind must The T for binaryFind must alsoalso have the have the compareTo methodcompareTo method and must have told Java that it doesand must have told Java that it does
» recall: interfacesrecall: interfaces
bad operand types for binary operator ‘<’!
The Comparable InterfaceThe Comparable Interface
Tells Java that this object can be comparedTells Java that this object can be compared generic interface: what can it be compared generic interface: what can it be compared toto?? usually to other objects of same classusually to other objects of same classpublic class Person implements Comparable<Person>public class Person implements Comparable<Person>
Only method required is compareToOnly method required is compareTopublic int compareTo(Person other)public int compareTo(Person other)
» returns < 0 if this < otherreturns < 0 if this < other» returns 0 if this == otherreturns 0 if this == other» returns > 0 if this > otherreturns > 0 if this > other
The Comparable InterfaceThe Comparable Interface
Suppose sorting by height (short to tall)Suppose sorting by height (short to tall)public int compareTo(Person other) {public int compareTo(Person other) { return this.height – other.height;return this.height – other.height;}}
» returns < 0 if this.height < other.heightreturns < 0 if this.height < other.height» returns 0 if this.height == other.heightreturns 0 if this.height == other.height» returns > 0 if this.height > other.heightreturns > 0 if this.height > other.height
Suppose sorting by nameSuppose sorting by namepublic int compareTo(Person other) {public int compareTo(Person other) { return this.name.compareTo(other.name);return this.name.compareTo(other.name);}}
Binary Search in JavaBinary Search in Java
Problem was Problem was item < v[mid]item < v[mid] needed to change it to needed to change it to item.compareTo(v[mid]) < 0item.compareTo(v[mid]) < 0 item must compare itself to objects of type Titem must compare itself to objects of type T item’s type must be item’s type must be Comparable<T>Comparable<T>public static <T> int binaryFind(Comparable<T> item,public static <T> int binaryFind(Comparable<T> item,
T[] v, int lo, int hi)T[] v, int lo, int hi) That worksThat works
works for any type that can compare itself to Tworks for any type that can compare itself to T» so we could look for a Student in a Person[]so we could look for a Student in a Person[]
binaryFind in JavabinaryFind in Java
public static <T> int binaryFind(Comparable<T> item, public static <T> int binaryFind(Comparable<T> item, T[] v, int lo, int hi) {T[] v, int lo, int hi) {
if (lo > hi) { return -1; }if (lo > hi) { return -1; } int mid = lo + (hi - lo) / 2;int mid = lo + (hi - lo) / 2; if (item.compareTo(v[mid]) < 0) {if (item.compareTo(v[mid]) < 0) { return binaryFind(item, v, lo, mid - 1);return binaryFind(item, v, lo, mid - 1); } else if (item.compareTo(v[mid]) > 0) {} else if (item.compareTo(v[mid]) > 0) { return binaryFind(item, v, mid+1, hi);return binaryFind(item, v, mid+1, hi); } else { return mid; }} else { return mid; } // found it!// found it! }}
Complexity of Binary SearchComplexity of Binary Search
Worst case, we get half the list every time!Worst case, we get half the list every time! assume N is a power of 2: N == 2assume N is a power of 2: N == 2kk
» 1 comparison reduces list from 21 comparison reduces list from 2k k to 2to 2k-1 k-1
» 1 more comparison reduces from 21 more comparison reduces from 2k-1k-1 to 2 to 2k-2k-2
» ……» 1 more comparison reduces from 21 more comparison reduces from 211 to 2 to 200 (i.e. 1) (i.e. 1)» 1 more comparison reduces from 21 more comparison reduces from 200 to 0 to 0
• #comparisons = k+1 = 1 + log N = O(log N)#comparisons = k+1 = 1 + log N = O(log N) shorter lists: 1 + ceiling(log N) = O(log N)shorter lists: 1 + ceiling(log N) = O(log N)
Sorted ListsSorted Lists
Sorting a list gives Sorting a list gives bigbig performance benefit performance benefit O(N) vs. O(log N) for N = 1,000,000?O(N) vs. O(log N) for N = 1,000,000?
» 1,000,000 vs. 201,000,000 vs. 20 Sorting lists is a very high prioritySorting lists is a very high priority
lots of different sorting methodslots of different sorting methods simplest methods not usually very goodsimplest methods not usually very good
» bubble sort, anybody?bubble sort, anybody? fastest methods usually hard to explainfastest methods usually hard to explain
Some Simple Sorting MethodsSome Simple Sorting Methods
Selection sortSelection sort ““select” items and place in their final positionselect” items and place in their final position
Insertion sortInsertion sort ““insert” items into a sorted sub-vectorinsert” items into a sorted sub-vector
Shell sortShell sort improved insertion sort moves items fasterimproved insertion sort moves items faster
Selection SortSelection Sort
Based on “copying” a list in orderBased on “copying” a list in order select the smallest item in the listselect the smallest item in the list add it to end of new listadd it to end of new list cross it off the old listcross it off the old list
But do it all in one arrayBut do it all in one array front part of list is sortedfront part of list is sorted won’t need to be changed again – everything won’t need to be changed again – everything
left “unsorted” is biggerleft “unsorted” is bigger
–23
Selection SortSelection Sort
Start at low endStart at low end
For each position in the list…For each position in the list… find the smallest unsorted elementfind the smallest unsorted element swap it into positionswap it into position
6 100 3 –2 8 686 –2 1003 100 6 1001008
6 100 3 –2 8
Selection Sort Top-Level TraceSelection Sort Top-Level Trace
lst lst [6, 100, 3, -2, 8]; [6, 100, 3, -2, 8];SelectionSort( lst );SelectionSort( lst ); 00 1 1 2 2 3 3 4 4 lst == lst == 6 6 100 3 -2 8100 3 -2 8
1st pass:find smallest item in listswap it with first item in list
i.e. swap -2 with 6
Selection Sort Top-Level TraceSelection Sort Top-Level Trace
lst lst [6, 100, 3, -2, 8]; [6, 100, 3, -2, 8];SelectionSort( lst );SelectionSort( lst ); _0_0 1 1 2 2 3 3 4 4 lst == lst == 6 6 100 3 -2 8100 3 -2 8lst == lst == [-2] [-2] 100100 3 6 83 6 8
2nd pass:find smallest remaining item in rest of listswap it into the second position
i.e. swap 3 with 100
Selection Sort Top-Level TraceSelection Sort Top-Level Trace
lst lst [6, 100, 3, -2, 8]; [6, 100, 3, -2, 8];SelectionSort( lst );SelectionSort( lst ); 00 1 1 2 2 3 3 4 4 lst == lst == 6 6 100 3 -2 8100 3 -2 8lst == lst == [-2] [-2] 100100 3 6 83 6 8lst == lst == [-2 3] [-2 3] 100 6 8100 6 8
Selection Sort Top-Level TraceSelection Sort Top-Level Trace
lst lst [6, 100, 3, -2, 8]; [6, 100, 3, -2, 8];SelectionSort( lst );SelectionSort( lst ); 00 1 1 2 2 3 3 4 4 lst == lst == 6 6 100 3 -2 8100 3 -2 8lst == lst == [-2] [-2] 100100 3 6 83 6 8lst == lst == [-2 3] [-2 3] 100 6 8100 6 8lst == lst == [-2 3 6] [-2 3 6] 100 8100 8
Selection Sort Top-Level TraceSelection Sort Top-Level Trace
lst lst [6, 100, 3, -2, 8]; [6, 100, 3, -2, 8];SelectionSort( lst );SelectionSort( lst ); 00 1 1 2 2 3 3 4 4 lst == lst == 6 6 100 3 -2 8100 3 -2 8lst == lst == [-2] [-2] 100100 3 6 83 6 8lst == lst == [-2 3] [-2 3] 100 6 8100 6 8lst == lst == [-2 3 6] [-2 3 6] 100 8100 8lst == lst == [-2 3 6 8] [-2 3 6 8] 100100
ExerciseExercise
Show the evolution of the following array Show the evolution of the following array under selection sort. Show the result after under selection sort. Show the result after each (top-level) passeach (top-level) pass [15, 3, 21, 45, 7][15, 3, 21, 45, 7]
Finding the SmallestFinding the Smallest
Each pass requires finding the smallest item Each pass requires finding the smallest item remaining unsortedremaining unsorted need its location in the array, so we can swap itneed its location in the array, so we can swap it on on iith pass search locations th pass search locations ii–1 .. N–1–1 .. N–1 can start assuming first of those is smallestcan start assuming first of those is smallest
Selection SortSelection Sort
to SelectionSort( List a )to SelectionSort( List a )for i for i 0 .. Length(a) – 2 0 .. Length(a) – 2
p p i; i;for j for j i + 1 .. Length(a) – 1 i + 1 .. Length(a) – 1
if (a[j] < a[p])if (a[j] < a[p])p p j; j;
Swap(a[p], a[i]);Swap(a[p], a[i]);
ExerciseExercise
List all the comparisons made List all the comparisons made between between array elementsarray elements while selection-sorting the while selection-sorting the following list:following list: [2, 8, 12, 4, 9][2, 8, 12, 4, 9]
Insertion SortInsertion Sort
Based on inserting into an ordered listBased on inserting into an ordered list For when you’re creating a listFor when you’re creating a list Keep it sorted right from the startKeep it sorted right from the start Insert each item into its proper place...Insert each item into its proper place... ...shifting other items up as required...shifting other items up as required
But all items already in the arrayBut all items already in the array ““split” array into sorted list part and input partsplit” array into sorted list part and input part
Insertion SortInsertion Sort
Start at low endStart at low end
For each unsorted element…For each unsorted element… find its placefind its place make spacemake space insert itinsert it
6 100 3 –2 8
61006 100 3 –2 86 100 36 100 3 –23 6 100 8–2 1008
Insertion Sort, Top Level LoopInsertion Sort, Top Level Loop
Multiple passes thru the listMultiple passes thru the list At start of At start of iith pass, first th pass, first ii items are in order items are in order
list of length 1 always “in order”list of length 1 always “in order” At end of At end of iith pass, first th pass, first ii+1 items in order+1 items in order
thus need only N – 1 passes in allthus need only N – 1 passes in all On On iith pass, “insert” th pass, “insert” ii+1+1stst item into sorted item into sorted
part of listpart of list
Insertion Sort Top-Level TraceInsertion Sort Top-Level Trace
lst lst [6, 100, 3, -2, 8]; [6, 100, 3, -2, 8];InsertionSort( lst );InsertionSort( lst ); _0_0 _1 _1 2 2 3 3 4 4 lst == lst == [ 6] [ 6] 100 3 -2 8100 3 -2 8
1st pass:list from 0 to 0 is sortedinsert item 1 into the sorted part of the list
i.e. insert 100 into [6]
Insertion Sort Top-Level TraceInsertion Sort Top-Level Trace
lst lst [6, 100, 3, -2, 8]; [6, 100, 3, -2, 8];InsertionSort( lst );InsertionSort( lst ); _0_0 _1 _1 2 2 3 3 4 4 lst == lst == [ 6] [ 6] 100 3 -2 8100 3 -2 8lst == lst == [ 6 100] [ 6 100] 3 -2 83 -2 8
2nd pass:list from 0 to 1 is sortedinsert item 2 into the sorted part of the list
i.e. insert 3 into [6, 100]
Insertion Sort Top-Level TraceInsertion Sort Top-Level Trace
lst lst [6, 100, 3, -2, 8]; [6, 100, 3, -2, 8];InsertionSort( lst );InsertionSort( lst ); _0_0 _1 _1 2 2 3 3 4 4 lst == lst == [ 6] [ 6] 100 3 -2 8100 3 -2 8lst == lst == [ 6 100] [ 6 100] 3 -2 83 -2 8lst == lst == [ 3 6 100] [ 3 6 100] -2 8-2 8
Insertion Sort Top-Level TraceInsertion Sort Top-Level Trace
lst lst [6, 100, 3, -2, 8]; [6, 100, 3, -2, 8];InsertionSort( lst );InsertionSort( lst ); _0_0 _1 _1 2 2 3 3 4 4 lst == lst == [ 6] [ 6] 100 3 -2 8100 3 -2 8lst == lst == [ 6 100] [ 6 100] 3 -2 83 -2 8lst == lst == [ 3 6 100] [ 3 6 100] -2 8-2 8lst == lst == [-2 3 6 100] [-2 3 6 100] 88
Insertion Sort Top-Level TraceInsertion Sort Top-Level Trace
lst lst [6, 100, 3, -2, 8]; [6, 100, 3, -2, 8];InsertionSort( lst );InsertionSort( lst ); _0_0 _1 _1 2 2 3 3 4 4 lst == lst == [ 6] [ 6] 100 3 -2 8100 3 -2 8lst == lst == [ 6 100] [ 6 100] 3 -2 83 -2 8lst == lst == [ 3 6 100] [ 3 6 100] -2 8-2 8lst == lst == [-2 3 6 100] [-2 3 6 100] 88lst == lst == [-2 3 6 8 100][-2 3 6 8 100]
ExerciseExercise
Show the evolution of the following array Show the evolution of the following array under insertion sort. Show the result after under insertion sort. Show the result after each (top-level) passeach (top-level) pass [15, 3, 21, 45, 7][15, 3, 21, 45, 7] [17, 4, 8, 3][17, 4, 8, 3]
Insertion on Pass Insertion on Pass ii
Move Move ii+1+1stst item into temporary storage item into temporary storage Starting at the “end” of the sorted part…Starting at the “end” of the sorted part… ……move items up until you find where move items up until you find where
“new” item goes“new” item goes find a smaller itemfind a smaller item fall off the front of the listfall off the front of the list
Insertion on Pass 2 TraceInsertion on Pass 2 Trace
lst == [6, 100], 3, -2, 8lst == [6, 100], 3, -2, 8Insert 3 into [6, 100]Insert 3 into [6, 100] 00 1 1 2 2 temptemplst == lst == [ 6 [ 6 100100 33]] 3 3lst == lst == [ [ 66 100 100 100] 100] 3 3lst == lst == [[ 66 6 100] 6 100] 3 3lst == lst == [ 3 6 100] [ 3 6 100] 3 3
Insertion on Pass 2 TraceInsertion on Pass 2 Trace
lst == [-2, 3, 6, 100], 8lst == [-2, 3, 6, 100], 8Insert 8 into [-2, 3, 6, 100]Insert 8 into [-2, 3, 6, 100] 00 1 1 2 2 3 3 4 4 temptemp[ -2 3 6 [ -2 3 6 100100 88] ] 8 8[ -2 3 [ -2 3 66 100100 100] 100] 8 8[ -2 3 6 8 100][ -2 3 6 8 100] 8 8
Insertion Sort Pseudo-CodeInsertion Sort Pseudo-Code
to InsertionSort( List a )to InsertionSort( List a )for i for i 0 .. Length(a) – 2 0 .. Length(a) – 2
p p i + 1 i + 1temp temp a[p]; a[p];while (p > 0 && a[p – 1] > temp)while (p > 0 && a[p – 1] > temp)
a[p] a[p] a[p – 1]; a[p – 1];p p p – 1; p – 1;
a[p] a[p] temp; temp;
ExerciseExercise
Show the evolution of the following array Show the evolution of the following array under insertion sort. Show the result after under insertion sort. Show the result after each assignment into the array (that is, inner each assignment into the array (that is, inner as well as outer loops):as well as outer loops): [17, 4, 8, 3][17, 4, 8, 3]
Work in Selection SortWork in Selection Sort
to SelectionSort( List a )to SelectionSort( List a )for i for i 1 .. Length(a) – 1 1 .. Length(a) – 1
p p i; i;for j for j i + 1 .. Length(a) i + 1 .. Length(a)
if (a[j] < a[p])if (a[j] < a[p]) p p j; j;temp temp a[p]; a[p];a[p] a[p] a[i]; a[i];a[i] a[i] temp; temp;
Assignment
Comparison
Counting Ops in Selection SortCounting Ops in Selection Sort
Outer loop runs i from 1 to N – 1Outer loop runs i from 1 to N – 1 inner runs j from i+1 to N (inner runs j from i+1 to N (alwaysalways))
i=1i=1N–1N–1 (3 + (3 + j=i+1j=i+1NN 1) 1)
= = i=1i=1N–1N–1 (3 + (3 + j=1j=1NN 1 – 1 – j=1j=1ii 1) 1) = = i=1i=1N–1N–1 (3 + N – i) (3 + N – i) = (3 + N)(N – 1) – (N – 1)(N)/2= (3 + N)(N – 1) – (N – 1)(N)/2 = N= N22 + 2N – 3 – (N + 2N – 3 – (N22 – N)/2 = (N – N)/2 = (N22 + 5N – 6)/2 + 5N – 6)/2
Work in Insertion SortWork in Insertion Sort
to InsertionSort( List a )to InsertionSort( List a )for i for i 1 .. Length(a) – 1 1 .. Length(a) – 1
p p i + 1 i + 1temp temp a[p]; a[p];while (p > 1 && a[p – 1] > temp)while (p > 1 && a[p – 1] > temp)
a[p] a[p] a[p – 1]; a[p – 1];p p p – 1; p – 1;
a[p] a[p] temp; temp;Assignment
Comparison
Counting Ops in Insertion SortCounting Ops in Insertion Sort
Let N be the size of the array/vectorLet N be the size of the array/vector number of elements in the listnumber of elements in the list
Outer loop iterates N – 1 timesOuter loop iterates N – 1 times contains two assignmentscontains two assignments
Inner loop iterates Inner loop iterates at most at most i timesi times list was in reverse order – need to compare alllist was in reverse order – need to compare all contains one comparison & one assignmentcontains one comparison & one assignment
Counting Ops in Insertion SortCounting Ops in Insertion Sort
Outer loop runs i from 1 to N – 1Outer loop runs i from 1 to N – 1 inner runs p from i+1 down to 2 (“worst case”)inner runs p from i+1 down to 2 (“worst case”)
i=1i=1N–1N–1 (2 + (2 + p=2p=2i+1i+1 2) = 2) = i=1i=1N–1N–1 (2 + 2i) (2 + 2i)
= 2(N – 1) + 2(N – 1)(N)/2= 2(N – 1) + 2(N – 1)(N)/2 = 2N – 2 + N= 2N – 2 + N22 – N – N = N= N22 + N – 2 + N – 2
Comparing Selection & InsertionComparing Selection & Insertion
TTSelection sortSelection sort(N) = (N(N) = (N22 + 5N – 6)/2 + 5N – 6)/2 TTInsertion sortInsertion sort(N) = N(N) = N22 + N – 2 + N – 2 N = 1000 gives:N = 1000 gives:
Selection sort: 502,987 operationSelection sort: 502,987 operation Insertion sort: 1,000,998 operationsInsertion sort: 1,000,998 operations
Selection looks fasterSelection looks faster But this is worst-case timeBut this is worst-case time
Worst Worst vs.vs. Average Average
When we did Insertion sort, we said “at When we did Insertion sort, we said “at most” for the inner loopmost” for the inner loop the most work it could do – “worst case”the most work it could do – “worst case” usuallyusually will do less will do less
For Selection, we said “always”For Selection, we said “always” never does any lessnever does any less
Measuring ComplexityMeasuring Complexity
Worst caseWorst case imagine that everything goes badly for usimagine that everything goes badly for us longest possible time for this algorithm to runlongest possible time for this algorithm to run
Average caseAverage case how much work would we how much work would we expectexpect to do to do
Best caseBest case everything goes well – rarely consideredeverything goes well – rarely considered
Average Case AnalysisAverage Case Analysis
Sometimes hard to doSometimes hard to do not this timenot this time
On On averageaverage, we expect inner loop of , we expect inner loop of insertion to only go ½ way to the frontinsertion to only go ½ way to the front
i=1i=1N–1N–1 (2 + ½(2i)) = 2(N–1) + (N–1)(N)/2 (2 + ½(2i)) = 2(N–1) + (N–1)(N)/2
= 2N – 2 + (N= 2N – 2 + (N22 – N)/2 – N)/2 = (N= (N22 + 3N – 4)/2 + 3N – 4)/2
Average Selection & InsertionAverage Selection & Insertion
TTSelection averageSelection average(N) = (N(N) = (N22 + 5N – 6)/2 + 5N – 6)/2 TTInsertion averageInsertion average(N) = (N(N) = (N22 + 3N – 4)/2 + 3N – 4)/2 N = 1000 gives:N = 1000 gives:
Selection sort: 502,987 operationSelection sort: 502,987 operation Insertion sort: 501,498 operationsInsertion sort: 501,498 operations
They do about the same amount of workThey do about the same amount of work and both are O(Nand both are O(N22))
ExerciseExercise
Show the evolution of each of the following Show the evolution of each of the following arrays under both insertion and selection arrays under both insertion and selection sortssorts [99, 14, 21, 12, 5][99, 14, 21, 12, 5] [3, 7, 14, 5, 2, 8][3, 7, 14, 5, 2, 8] [9, 8, 7, 6, 5, 4, 3, 2, 1][9, 8, 7, 6, 5, 4, 3, 2, 1]
Shell SortShell Sort
Sort far apart items firstSort far apart items first then sort the ones that are closer togetherthen sort the ones that are closer together
Sort items separated by a given “gap”Sort items separated by a given “gap” every fifth element, for exampleevery fifth element, for example
Use any sorting methodUse any sorting method insertion sort, insertion sort, e.g.e.g.
Reduce the gap size & repeatReduce the gap size & repeat stop when you’ve sorted with a gap of 1stop when you’ve sorted with a gap of 1
Shell Sort: Gap = 5Shell Sort: Gap = 5
17241232119481 15757741182829
2481
8124Swap
1794
9417Swap
2911 OK
2832
3228Swap
1812 OK
Start at location 6Up by 1 each timeUse insertion sort
Shell Sort: Gap = 5 (cont)Shell Sort: Gap = 5 (cont)
94811228111724 15757741183229
4181
8141Swap
7794
9477Swap
7529 OK
1532
3215Swap
24 OK
17 OK
28
2815Swap
Shell Sort: Gap = 5 (cont)Shell Sort: Gap = 5 (cont)
77411215111724 32759481182829
List is now “5-sorted”List is now “5-sorted”
7717 94
4124 81
12 18
15 3228
11 7529
Shell Sort: Gap = 3Shell Sort: Gap = 3
77411215111724 32759481182829
Continue with a smaller gapContinue with a smaller gap77
41
12
15
11
17
24
32
75
94
81
18
28
29
SwapSwap
OKOK
OKSwap (41)
Shell Sort: Gap = 3 (cont)Shell Sort: Gap = 3 (cont)
77411215111724 32759481182829
List is now 3-sortedList is now 3-sorted
Note: it’s still 5-sortedNote: it’s still 5-sorted
24
28
17
18
11
12
15
81
77
94
32
75
41
29
2815 322412 94
24281718111215 81779432754129
Shell Sort: Gap = 1Shell Sort: Gap = 1
Now it’s just normal insertion sortNow it’s just normal insertion sort but everything’s “pretty close” to where it’s but everything’s “pretty close” to where it’s
going to end upgoing to end up
Number of assignments:Number of assignments: shell sort:shell sort: 13 + 13 + 15 = 4113 + 13 + 15 = 41 insertion sort:insertion sort: 5757
24281718111215 81779432754129
15 1512 151211 18 1817 28 2824 4129 75 947532 9477 9481
Gap SizesGap Sizes
Any descending sequence will doAny descending sequence will do so long as it ends at 1so long as it ends at 1
Some sequences:Some sequences: N/2, N/4, N/8, …, 1 N/2, N/4, N/8, …, 1 not especially goodnot especially good A = 2A = 2log Nlog N – 1, A/2, A/4, …, 1 – 1, A/2, A/4, …, 1 N/3, N/9, N/27, …, 1 N/3, N/9, N/27, …, 1 …… other sequences may be even betterother sequences may be even better
Gap SizesGap Sizes
N = 13N = 13 insertion sort = 52 comps, 65 storesinsertion sort = 52 comps, 65 stores 11stst gap sequence: 6, 3, 1 gap sequence: 6, 3, 1 41 comps, 77 stores41 comps, 77 stores 22ndnd gap sequence: 7, 3, 1 gap sequence: 7, 3, 1 46 comps, 81 stores46 comps, 81 stores 33rdrd gap sequence: 4, 1 gap sequence: 4, 1 37 comps, 63 stores37 comps, 63 stores
N = 130 (random values)N = 130 (random values) 4487c, 4618s4487c, 4618s 11stst: 65, 32, 16, 8, 4, 2, 1: 65, 32, 16, 8, 4, 2, 1 1383c, 2236s1383c, 2236s 22ndnd: 127, 63, 31, 15, 7, 3, 1: 127, 63, 31, 15, 7, 3, 1 1059c, 1792s1059c, 1792s 33rdrd: 43, 14, 4, 1: 43, 14, 4, 1 1082c, 1592s1082c, 1592s
17241232119481 157741182829
Shell SortShell Sort
to ShellSort( List a )to ShellSort( List a )gap gap Length(a) Length(a) 2; 2; /* or … *//* or … */while (gap while (gap 1) 1)if (gap % 2 == 0) ++gap;if (gap % 2 == 0) ++gap; /* much better! *//* much better! */for i for i gap .. Length(a)–1 gap .. Length(a)–1p p i, i, temp temp a[p]; a[p];while (p while (p gap && a[p–gap] > temp) gap && a[p–gap] > temp)a[p] a[p] a[p–gap], p a[p–gap], p p–gap; p–gap;a[p] a[p] temp; temp;gap gap gap gap 2; 2; /* or … *//* or … */
ExerciseExercise
Show the evolution of the following arrays Show the evolution of the following arrays under shell sort. Use length/3 as the under shell sort. Use length/3 as the starting gap:starting gap: [15, 3, 21, 45, 7, 17, 4][15, 3, 21, 45, 7, 17, 4] [13, 11, 20, 15, 16, 6, 5, 8][13, 11, 20, 15, 16, 6, 5, 8]
Complexity of Shell SortComplexity of Shell Sort
Depends on the gap sequenceDepends on the gap sequence O(NO(N22) for ) for N/2 versionN/2 version
» O(NO(N3/23/2) if we do the only-odd-numbers version) if we do the only-odd-numbers version O(NO(N3/23/2) for 2) for 2log Nlog N – 1 version – 1 version
» ……, 63, 31, 15, 7, 3, 1, 63, 31, 15, 7, 3, 1» some others have this as wellsome others have this as well
some versions have O(Nsome versions have O(N4/34/3)) one version has O(None version has O(N1+sqrt(8ln(5/2)/ln(N))1+sqrt(8ln(5/2)/ln(N)))) others we don’t even know!others we don’t even know!
CommentsComments
All O(NAll O(N22) in worst case) in worst case Shell sort can be O(NShell sort can be O(N3/23/2), but still not good enuf), but still not good enuf doubling size of array quadruples timedoubling size of array quadruples time
Want something better!Want something better! Recursion offers a way upRecursion offers a way up
next time: merge sort, quick sortnext time: merge sort, quick sort