Download - Chapter 9 sorting. Insertion Sort I The list is assumed to be broken into a sorted portion and an unsorted portion The list is assumed to be broken into

Chapter 9Chapter 9

sortingsorting

Insertion Sort IInsertion Sort I The list is assumed to be broken into a The list is assumed to be broken into a

sorted portion and an unsorted portionsorted portion and an unsorted portion Keys will be inserted from the unsorted Keys will be inserted from the unsorted

portion into the sorted portion.portion into the sorted portion.

Sorted

Unsorted

Insertion Sort IIInsertion Sort II For each new key, search backward For each new key, search backward

through sorted keysthrough sorted keys Move keys until proper position is foundMove keys until proper position is found Place key in proper positionPlace key in proper position

Moved

Insertion Sort: CodeInsertion Sort: Codetemplate <class Comparable> void insertionSort( vector<Comparable> & a ) { for( int p = 1; p < a.size( ); p++ ) { Comparable tmp = a[ p ]; int j; for( j = p; j > 0 && tmp < a[ j - 1 ]; j-- ) a[ j ] = a[ j - 1 ]; a[ j ] = tmp; } }

Fixed n-1 iterations

Worst case i-1 comparisons

Move current key to rightInsert the new key to its proper position

Searching for the proper position for the new key

Moved

Insertion Sort: AnalysisInsertion Sort: Analysis Worst Case: Keys are in reverse orderWorst Case: Keys are in reverse order Do i-1 comparisons for each new key, Do i-1 comparisons for each new key,

where i runs from 2 to n.where i runs from 2 to n. Total Comparisons: 1+2+3+ … + n-1Total Comparisons: 1+2+3+ … + n-1

i n n ni

n

1

121

2( )

Comparison

Optimality Analysis IOptimality Analysis I To discover an optimal algorithm we need To discover an optimal algorithm we need

to find an upper and lower asymptotic to find an upper and lower asymptotic bound for a bound for a problemproblem..

An algorithm gives us an upper bound. The An algorithm gives us an upper bound. The worst case for sorting cannot exceed worst case for sorting cannot exceed ((nn22) ) because we have Insertion Sort that runs because we have Insertion Sort that runs that fast.that fast.

Lower bounds require mathematical Lower bounds require mathematical arguments.arguments.

Optimality Analysis IIOptimality Analysis II Making mathematical arguments Making mathematical arguments usuallyusually

involves assumptions about how the involves assumptions about how the problem will be solved.problem will be solved.

Invalidating the assumptions invalidates the Invalidating the assumptions invalidates the lower bound.lower bound.

Sorting an array of numbers requires at least Sorting an array of numbers requires at least ((nn) time, because it would take that much ) time, because it would take that much time to rearrange a list that was rotated one time to rearrange a list that was rotated one element out of position.element out of position.

Rotating One ElementRotating One Element

2nd3rd4th

nth1st

2nd3rd

n-1stnth

1stn keys mustbe moved

(n) time

Assumptions:

Keys must be movedone at a time

All key movements takethe same amount of time

The amount of timeneeded to move one keyis not dependent on n.

Other AssumptionsOther Assumptions The only operation used for sorting The only operation used for sorting

the list is swapping two keys.the list is swapping two keys. Only adjacent keys can be swapped.Only adjacent keys can be swapped. This is true for Insertion Sort and This is true for Insertion Sort and

Bubble Sort.Bubble Sort.

1 61 62 3 45 7 8 910

Inversion Inversion Inversion

Not an Inversion

InversionsInversions Suppose we are given a list of Suppose we are given a list of

elements L, of size n.elements L, of size n. Let i, and j be chosen so 1Let i, and j be chosen so 1i<ji<jn.n. If L[i]>L[j] then the pair (i,j) is an If L[i]>L[j] then the pair (i,j) is an

inversion.inversion.

Maximum InversionsMaximum Inversions The total number of pairs is:The total number of pairs is:

This is the maximum number of This is the maximum number of inversions in any list.inversions in any list.

Exchanging adjacent pairs of keys Exchanging adjacent pairs of keys removes at most one inversion.removes at most one inversion.

2)1(

2

nnn

Swapping Adjacent PairsSwapping Adjacent Pairs

Swap Red and Green

The relative position of the Red and blue areas has not changed. No inversions between the red key and the blue area have been removed. The same is true for the red key and the orange area. The same analysis can be done for the green key.

The only inversionthat could be removedis the (possible) onebetween the red andgreen keys.

Lower Bound ArgumentLower Bound Argument A sorted list has no inversions.A sorted list has no inversions. A reverse-order list has the maximum A reverse-order list has the maximum

number of inversions, number of inversions, ((nn22) inversions.) inversions. A sorting algorithm must exchange A sorting algorithm must exchange ((nn22) )

adjacent pairs to sort a list.adjacent pairs to sort a list. A sort algorithm that operates by A sort algorithm that operates by

exchanging adjacent pairs of keys must exchanging adjacent pairs of keys must have a time bound of at least have a time bound of at least ((nn22).).

Lower Bound For Average ILower Bound For Average I There are n! ways to rearrange a list of There are n! ways to rearrange a list of

n elements.n elements. Recall that a rearrangement is called a Recall that a rearrangement is called a

permutationpermutation.. If we reverse a rearranged list, every If we reverse a rearranged list, every

pair that used to be an inversion will no pair that used to be an inversion will no longer be an inversion.longer be an inversion.

By the same token, all non-inversions By the same token, all non-inversions become inversions.become inversions.

Lower Bound For Average IILower Bound For Average II There are n(n-1)/2 inversions in a There are n(n-1)/2 inversions in a

permutation and its reverse.permutation and its reverse. Assuming that all n! permutations Assuming that all n! permutations

are equally likely, there are n(n-are equally likely, there are n(n-1)/4 inversions in a permutation, 1)/4 inversions in a permutation, on the average.on the average.

The average performance of a The average performance of a “swap-adjacent-pairs” sorting “swap-adjacent-pairs” sorting algorithm will be algorithm will be ((nn22).).

Shell SortShell Sort With insertion sort, each time we insert an With insertion sort, each time we insert an

element, other elements get nudged element, other elements get nudged one stepone step closer to where they ought to becloser to where they ought to be

What if we could move elements a What if we could move elements a much longer much longer distancedistance each time? each time?

We could move each element:We could move each element: A long distanceA long distance A somewhat shorter distanceA somewhat shorter distance A shorter distance stillA shorter distance still

This approach is what makes shellsort so much This approach is what makes shellsort so much faster than insertion sortfaster than insertion sort

Sorting nonconsecutive Sorting nonconsecutive subarrayssubarrays

Consider just the red locationsConsider just the red locations Suppose we do an insertion sort onSuppose we do an insertion sort on just these just these

numbers, numbers, as if they were the only ones in the array?as if they were the only ones in the array? Now consider just the yellow locationsNow consider just the yellow locations We do an insertion sort on just these numbersWe do an insertion sort on just these numbers Now do the same for each additional group of numbersNow do the same for each additional group of numbers The resultant array is sorted within groups, but not

overall

• Here is an array to be sorted (numbers aren’t important)

Doing the 1-sortDoing the 1-sort In the previous slide, we compared numbers that In the previous slide, we compared numbers that

were spaced every 5 locationswere spaced every 5 locations This is a 5-sortThis is a 5-sort

Ordinary insertion sort is just like this, only the Ordinary insertion sort is just like this, only the numbers are spaced 1 apartnumbers are spaced 1 apart We can think of this as a 1-sortWe can think of this as a 1-sort

Suppose, after doing the 5-sort, we do a 1-sort?Suppose, after doing the 5-sort, we do a 1-sort? In general, we would expect that each insertion would In general, we would expect that each insertion would

involve moving fewer numbers out of the wayinvolve moving fewer numbers out of the way The array would end up completely sortedThe array would end up completely sorted

Example of shell sortExample of shell sortoriginoriginalal

8181 9494 1111 9696 1212 3535 1717 9595 2828 5858 4141 7575 1515

5-5-sortsort

3535 1717 1111 2828 1212 4141 7575 1515 9696 5858 8181 9494 9595

3-3-sortsort

2828 1212 1111 3535 1515 4141 5858 1717 9494 7575 8181 9696 9595

1-1-sortsort

1111 1212 1515 1717 2828 3535 4141 5858 7575 8181 9494 9595 9696

Diminishing gapsDiminishing gaps For a large array, we don’t want to do a For a large array, we don’t want to do a

5-sort; we want to do an N-sort, where 5-sort; we want to do an N-sort, where N depends on the size of the arrayN depends on the size of the array N is called the gap size, or interval sizeN is called the gap size, or interval size

We may want to do several stages, We may want to do several stages, reducing the gap size each timereducing the gap size each time For example, on a 1000-element array, we For example, on a 1000-element array, we

may want to do a 364-sort, then a 121-may want to do a 364-sort, then a 121-sort, then a 40-sort, then a 13-sort, then a sort, then a 40-sort, then a 13-sort, then a 4-sort, then a 1-sort4-sort, then a 1-sort

Why these numbers?Why these numbers?

Increment sequenceIncrement sequence No one knows the optimal sequence of diminishing No one knows the optimal sequence of diminishing

gapsgaps This sequence is attributed to Donald E. Knuth:This sequence is attributed to Donald E. Knuth:

Start with h = 1Start with h = 1 Repeatedly compute h = 3*h + 1Repeatedly compute h = 3*h + 1

1, 4, 13, 40, 121, 364, 10931, 4, 13, 40, 121, 364, 1093 This sequence seems to work very wellThis sequence seems to work very well Another increment sequence mentioned in the Another increment sequence mentioned in the

textbook is based on the following formula:textbook is based on the following formula: start with h = the half of the container’s sizestart with h = the half of the container’s size hhii = floor (h = floor (hi-1i-1 / 2.2) / 2.2)

It turns out that just cutting the array size in half It turns out that just cutting the array size in half each time does not work out as welleach time does not work out as well

Analysis Analysis What is the What is the realreal running time of running time of

shellsort?shellsort? Nobody knows!Nobody knows! Experiments suggest something Experiments suggest something

like like O(nO(n3/23/2) ) or or O(nO(n7/67/6)) Analysis isn’t always easy!Analysis isn’t always easy!

Merge SortMerge Sort If List has only one If List has only one

Element, do nothingElement, do nothing Otherwise, Split List in HalfOtherwise, Split List in Half Recursively Sort Both ListsRecursively Sort Both Lists Merge Sorted ListsMerge Sorted Lists

Mergesort(A, l, r) if l < r then q = floor((l+r)/2) mergesort(A, l, q) mergesort(A, q+1, r) merge(A, l, q, r)

The Merge AlgorithmThe Merge AlgorithmAssume we are merging lists A and B into list C.Ax := 1; Bx := 1; Cx := 1;while Ax n and Bx n do if A[Ax] < B[Bx] then C[Cx] := A[Ax]; Ax := Ax + 1; else C[Cx] := B[Bx]; Bx := Bx + 1; endif Cx := Cx + 1;endwhile

while Ax n do C[Cx] := A[Ax]; Ax := Ax + 1; Cx := Cx + 1;endwhilewhile Bx n do C[Cx] := B[Bx]; Bx := Bx + 1; Cx := Cx + 1;endwhile

auxiliary array

smallest smallest

A G L O R H I M S T

MergingMergingMerge.Merge. Keep track of smallest element in each sorted half.Keep track of smallest element in each sorted half. Insert smallest of two elements into auxiliary array.Insert smallest of two elements into auxiliary array. Repeat until done.Repeat until done.

A

auxiliary array

smallest smallest

A G L O R H I M S T

A G


auxiliary array

smallest smallest

A G L O R H I M S T

A G

MergingMergingMerge.Merge. Keep track of smallest element in each sorted Keep track of smallest element in each sorted

half.half. Insert smallest of two elements into auxiliary Insert smallest of two elements into auxiliary

array.array. Repeat until done.Repeat until done.

H

auxiliary array

smallest smallest

A G L O R H I M S T

A G H

MergingMergingMerge.Merge. Keep track of smallest element in each sorted half.Keep track of smallest element in each sorted half. Insert smallest of two elements into auxiliary Insert smallest of two elements into auxiliary


I

auxiliary array

smallest smallest

A G L O R H I M S T

A G H I


L

auxiliary array

smallest smallest

A G L O R H I M S T

A G H I L




M

auxiliary array

smallest smallest

A G L O R H I M S T

A G H I L M




O

auxiliary array

smallest smallest

A G L O R H I M S T

A G H I L M O




R

auxiliary array

first halfexhausted smallest

A G L O R H I M S T

A G H I L M O R


S

auxiliary array

first halfexhausted smallest

A G L O R H I M S T

A G H I L M O R S


T

auxiliary array

first halfexhausted

second halfexhausted

A G L O R H I M S T

A G H I L M O R S T

MergingMergingMerge.Merge. Keep track of smallest element in each sorted half.Keep track of smallest element in each sorted half. Insert smallest of two elements into auxiliary Insert smallest of two elements into auxiliary


Merging numerical exampleMerging numerical example

1 2 24 27 38 13 15 26 50 52

Merge Sort: AnalysisMerge Sort: Analysis Sorting requires no comparisonsSorting requires no comparisons Merging requires Merging requires nn-1 comparisons in the worst case, -1 comparisons in the worst case,

where where nn is the total size of both lists ( is the total size of both lists (nn key movements key movements are required in are required in allall cases) cases)

Recurrence relation:Recurrence relation:

W( ) W( / ) lgn n n n n 2 2 1

Merge Sort: SpaceMerge Sort: Space Merging cannot be done in placeMerging cannot be done in place In the simplest case, a separate list In the simplest case, a separate list

of size of size nn is required for merging is required for merging It is possible to reduce the size of the It is possible to reduce the size of the

extra space, but it will still be extra space, but it will still be (n)(n)

Quick Sort IQuick Sort I Split List into “Big” and

“Little” keys Put the Little keys first, Big

keys second Recursively sort the Big and

Little keys

Little BigPivot Point

Quicksort( A, l, r) if l < r then q = partition(A, l, r) quicksort(A, l, q-1) quicksort(A, q+1, r)

Quicksort IIQuicksort II Big is defined as “bigger than the Big is defined as “bigger than the

pivot point”pivot point” Little is defined as “smaller than the Little is defined as “smaller than the

pivot point”pivot point” The pivot point is chosen “at random”. The pivot point is chosen “at random”.

In the following example, we pick up In the following example, we pick up the middle element as the pivot.the middle element as the pivot.

PartitioningPartitioning

2 97 17 39 12 37 10 55 80 42 46

Pick pivot == 37


2 97 17 39 12 46 10 55 80 42 37

Step 1: Move pivot to end of array


2 97 17 39 12 46 10 55 80 42 37

Step 2: set i == 0 and j == array.length - 1

i j


2 97 17 39 12 46 10 55 80 42 37

Step 3: move i right until value larger than the pivot is found

i j


2 97 17 39 12 46 10 55 80 42 37

Step 4: move j left until value less than the pivot is found

i j


2 10 17 39 12 46 97 55 80 42 37

Step 5: swap elements at positions i and j

i j


2 10 17 39 12 46 97 55 80 42 37

i j

Step 6: move i right until value larger than the pivot is found


2 10 17 39 12 46 97 55 80 42 37

i j

Step 7: move j left until value less than the pivot is found


2 10 17 12 39 46 97 55 80 42 37

i j

Step 8: swap elements at positions i and j


2 10 17 12 39 46 97 55 80 42 37

i

j

Step 9: move i left until it hits j


2 10 17 12 37 46 97 55 80 42 39

i

j

Step 10: put pivot in correct spot

Quicksort IIIQuicksort III Pivot point may not be the exact Pivot point may not be the exact

medianmedian Finding the precise median is Finding the precise median is hardhard If we “get lucky”, the following If we “get lucky”, the following

recurrence applies (n/2 is recurrence applies (n/2 is approximate)approximate) )lg(1)2/(2)( nnnnQnQ

Quicksort IVQuicksort IV If the keys are in order, “Big” portion will have n-1 keys, “Small” If the keys are in order, “Big” portion will have n-1 keys, “Small”

portion will be empty.portion will be empty. T(N) = T(N-1) + O(N) = O(NT(N) = T(N-1) + O(N) = O(N22))

N-1 comparisons are done for first keyN-1 comparisons are done for first key N-2 comparisons for second key, etc.N-2 comparisons for second key, etc. Result:Result:

i n n ni

n

1

121

2( )

A Better Lower BoundA Better Lower Bound The The ((nn22) time bound does not apply ) time bound does not apply

to Quicksort, Mergesort.to Quicksort, Mergesort. A better assumption is that keys can A better assumption is that keys can

be moved an arbitrary distance.be moved an arbitrary distance. However, we can still assume that the However, we can still assume that the

number of key-to-key comparisons is number of key-to-key comparisons is proportional to the run time of the proportional to the run time of the algorithm. algorithm.

Download - Chapter 9 sorting. Insertion Sort I The list is assumed to be broken into a sorted portion and an unsorted portion The list is assumed to be broken into

Top Related