sorting algorithms and analysis robert duncan. refresher on big-o o(2^n)exponential ...
Post on 22-Dec-2015
231 views
TRANSCRIPT
![Page 1: Sorting Algorithms and Analysis Robert Duncan. Refresher on Big-O O(2^N)Exponential O(N^2)Quadratic O(N log N)Linear/Log O(N)Linear O(log N)Log](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d795503460f94a5c196/html5/thumbnails/1.jpg)
Sorting
Algorithms and Analysis
Robert Duncan
![Page 2: Sorting Algorithms and Analysis Robert Duncan. Refresher on Big-O O(2^N)Exponential O(N^2)Quadratic O(N log N)Linear/Log O(N)Linear O(log N)Log](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d795503460f94a5c196/html5/thumbnails/2.jpg)
Refresher on Big-ORefresher on Big-O
O(2^N) ExponentialO(N^2) QuadraticO(N log N) Linear/LogO(N) LinearO(log N) LogO(1) Constant
Hierarchy of Big-O functions from slowest to fastest
![Page 3: Sorting Algorithms and Analysis Robert Duncan. Refresher on Big-O O(2^N)Exponential O(N^2)Quadratic O(N log N)Linear/Log O(N)Linear O(log N)Log](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d795503460f94a5c196/html5/thumbnails/3.jpg)
Generic running times Generic running times
N O(log N) O(N) O(N log N) O(N^2) O(2^N)
1 0 1 0 1 2
8 3 8 24 64 256
128 7 128 896 16384 3.4x10^38
1024 10 1024 10240 1048576 1.8x10^308
![Page 4: Sorting Algorithms and Analysis Robert Duncan. Refresher on Big-O O(2^N)Exponential O(N^2)Quadratic O(N log N)Linear/Log O(N)Linear O(log N)Log](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d795503460f94a5c196/html5/thumbnails/4.jpg)
O(N log N) vs. O(N^2)O(N log N) vs. O(N^2)
0
5000
10000
15000
20000
0 1000
O(N log N) O(N^2)
![Page 5: Sorting Algorithms and Analysis Robert Duncan. Refresher on Big-O O(2^N)Exponential O(N^2)Quadratic O(N log N)Linear/Log O(N)Linear O(log N)Log](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d795503460f94a5c196/html5/thumbnails/5.jpg)
Two Common CategoriesTwo Common Categories
Sorting Algorithms of O(N^2)
Bubble SortSelection SortInsertion Sort
Sorting Algorithms of
O(N log N)Heap SortMerge SortQuick Sort
![Page 6: Sorting Algorithms and Analysis Robert Duncan. Refresher on Big-O O(2^N)Exponential O(N^2)Quadratic O(N log N)Linear/Log O(N)Linear O(log N)Log](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d795503460f94a5c196/html5/thumbnails/6.jpg)
For small values of NFor small values of N
It is important to note that all algorithms appear to run equally as fast for small values of N.
For values of N from the thousands to the millions, The differences between O(N^2) and O(N log N) become dramatically apparent
![Page 7: Sorting Algorithms and Analysis Robert Duncan. Refresher on Big-O O(2^N)Exponential O(N^2)Quadratic O(N log N)Linear/Log O(N)Linear O(log N)Log](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d795503460f94a5c196/html5/thumbnails/7.jpg)
O(N^2) SortsO(N^2) Sorts
Easy to program
Simple to understand
Very slow, especially for large values of N
Almost never used in professional software
![Page 8: Sorting Algorithms and Analysis Robert Duncan. Refresher on Big-O O(2^N)Exponential O(N^2)Quadratic O(N log N)Linear/Log O(N)Linear O(log N)Log](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d795503460f94a5c196/html5/thumbnails/8.jpg)
Bubble SortBubble Sort
The most inefficient of the O(n^2) algorithms
Simplest sorting algorithm available
Works by comparing sequential items, and swapping them if the first one is larger than the second. It makes as many passes through an array as are needed to complete the sort
![Page 9: Sorting Algorithms and Analysis Robert Duncan. Refresher on Big-O O(2^N)Exponential O(N^2)Quadratic O(N log N)Linear/Log O(N)Linear O(log N)Log](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d795503460f94a5c196/html5/thumbnails/9.jpg)
Bubble Sort – Pass 1
2
3
6
4
1
5
2
3
6
4
1
5
2
3
4
6
1
5
2
3
4
1
6
5
2
3
4
1
5
6
![Page 10: Sorting Algorithms and Analysis Robert Duncan. Refresher on Big-O O(2^N)Exponential O(N^2)Quadratic O(N log N)Linear/Log O(N)Linear O(log N)Log](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d795503460f94a5c196/html5/thumbnails/10.jpg)
Bubble Sort – Pass 2
2
3
4
1
5
6
2
3
4
1
5
6
2
3
1
4
5
6
2
3
1
4
5
6
2
3
1
4
5
6
![Page 11: Sorting Algorithms and Analysis Robert Duncan. Refresher on Big-O O(2^N)Exponential O(N^2)Quadratic O(N log N)Linear/Log O(N)Linear O(log N)Log](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d795503460f94a5c196/html5/thumbnails/11.jpg)
Bubble Sort – Pass 3
2
3
1
4
5
6
2
1
3
4
5
6
2
1
3
4
5
6
2
1
3
4
5
6
2
1
3
4
5
6
![Page 12: Sorting Algorithms and Analysis Robert Duncan. Refresher on Big-O O(2^N)Exponential O(N^2)Quadratic O(N log N)Linear/Log O(N)Linear O(log N)Log](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d795503460f94a5c196/html5/thumbnails/12.jpg)
Bubble Sort – Pass 4
1
2
3
4
5
6
1
2
3
4
5
6
1
2
3
4
5
6
1
2
3
4
5
6
1
2
3
4
5
6
![Page 13: Sorting Algorithms and Analysis Robert Duncan. Refresher on Big-O O(2^N)Exponential O(N^2)Quadratic O(N log N)Linear/Log O(N)Linear O(log N)Log](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d795503460f94a5c196/html5/thumbnails/13.jpg)
Selection SortSelection Sort
More efficient than Bubble Sort, but not as efficient as Insertion Sort
Works by finding the largest element in the list and swapping it with the last element, effectively reducing the size of the list by 1.
![Page 14: Sorting Algorithms and Analysis Robert Duncan. Refresher on Big-O O(2^N)Exponential O(N^2)Quadratic O(N log N)Linear/Log O(N)Linear O(log N)Log](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d795503460f94a5c196/html5/thumbnails/14.jpg)
Selection Sort – Pass 1-3
6 8 10 2 4
6 8 4 2 10
6 2 4 8 10
![Page 15: Sorting Algorithms and Analysis Robert Duncan. Refresher on Big-O O(2^N)Exponential O(N^2)Quadratic O(N log N)Linear/Log O(N)Linear O(log N)Log](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d795503460f94a5c196/html5/thumbnails/15.jpg)
Selection Sort – Pass 4-5
4 2 6 8 10
2 4 6 8 10
2 4 6 8 10
![Page 16: Sorting Algorithms and Analysis Robert Duncan. Refresher on Big-O O(2^N)Exponential O(N^2)Quadratic O(N log N)Linear/Log O(N)Linear O(log N)Log](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d795503460f94a5c196/html5/thumbnails/16.jpg)
Insertion SortInsertion Sort
One of the most efficient of the O(n^2) algorithms
Roughly twice as fast as bubble sort
Works by taking items from unsorted list and inserting them into the proper place.
![Page 17: Sorting Algorithms and Analysis Robert Duncan. Refresher on Big-O O(2^N)Exponential O(N^2)Quadratic O(N log N)Linear/Log O(N)Linear O(log N)Log](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d795503460f94a5c196/html5/thumbnails/17.jpg)
Insertion Sort
22
22 35
22 35 48
![Page 18: Sorting Algorithms and Analysis Robert Duncan. Refresher on Big-O O(2^N)Exponential O(N^2)Quadratic O(N log N)Linear/Log O(N)Linear O(log N)Log](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d795503460f94a5c196/html5/thumbnails/18.jpg)
Insertion Sort
13 22 35 48
8 13 22 35 48
8 13 22 27 35 48
![Page 19: Sorting Algorithms and Analysis Robert Duncan. Refresher on Big-O O(2^N)Exponential O(N^2)Quadratic O(N log N)Linear/Log O(N)Linear O(log N)Log](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d795503460f94a5c196/html5/thumbnails/19.jpg)
O(N log N) SortsO(N log N) SortsFastEfficientComplicated, not easy to understandMost make extensive use of recursion and
complex data structures
![Page 20: Sorting Algorithms and Analysis Robert Duncan. Refresher on Big-O O(2^N)Exponential O(N^2)Quadratic O(N log N)Linear/Log O(N)Linear O(log N)Log](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d795503460f94a5c196/html5/thumbnails/20.jpg)
Heap SortHeap Sort
Slowest O(N log N) algorithm.
Although the slowest of the O(N log N) algorithms, it has less memory demands than Merge and Quick sort.
![Page 21: Sorting Algorithms and Analysis Robert Duncan. Refresher on Big-O O(2^N)Exponential O(N^2)Quadratic O(N log N)Linear/Log O(N)Linear O(log N)Log](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d795503460f94a5c196/html5/thumbnails/21.jpg)
Heap SortHeap Sort
Works by transferring items to a heap, which is basically a binary tree in which all parent nodes have greater values than their child nodes. The root of the tree, which is the largest item, is transferred to a new array and then the heap is reformed. The process is repeated until the sort is complete.
![Page 22: Sorting Algorithms and Analysis Robert Duncan. Refresher on Big-O O(2^N)Exponential O(N^2)Quadratic O(N log N)Linear/Log O(N)Linear O(log N)Log](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d795503460f94a5c196/html5/thumbnails/22.jpg)
Forming the heap from an unsorted array
Forming the heap from an unsorted array
11 26 14 2 19 32 7
32
214711
1926
![Page 23: Sorting Algorithms and Analysis Robert Duncan. Refresher on Big-O O(2^N)Exponential O(N^2)Quadratic O(N log N)Linear/Log O(N)Linear O(log N)Log](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d795503460f94a5c196/html5/thumbnails/23.jpg)
Populating the new arrayPopulating the new array
32
214711
1926
![Page 24: Sorting Algorithms and Analysis Robert Duncan. Refresher on Big-O O(2^N)Exponential O(N^2)Quadratic O(N log N)Linear/Log O(N)Linear O(log N)Log](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d795503460f94a5c196/html5/thumbnails/24.jpg)
Reforming the heapReforming the heap
32
19
2711
1426
![Page 25: Sorting Algorithms and Analysis Robert Duncan. Refresher on Big-O O(2^N)Exponential O(N^2)Quadratic O(N log N)Linear/Log O(N)Linear O(log N)Log](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d795503460f94a5c196/html5/thumbnails/25.jpg)
Reforming the heapReforming the heap
32
26
2711
1419
![Page 26: Sorting Algorithms and Analysis Robert Duncan. Refresher on Big-O O(2^N)Exponential O(N^2)Quadratic O(N log N)Linear/Log O(N)Linear O(log N)Log](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d795503460f94a5c196/html5/thumbnails/26.jpg)
Repeat the processRepeat the process
32 26
2711
1419
![Page 27: Sorting Algorithms and Analysis Robert Duncan. Refresher on Big-O O(2^N)Exponential O(N^2)Quadratic O(N log N)Linear/Log O(N)Linear O(log N)Log](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d795503460f94a5c196/html5/thumbnails/27.jpg)
Repeat the processRepeat the process
32 26
14
711
219
![Page 28: Sorting Algorithms and Analysis Robert Duncan. Refresher on Big-O O(2^N)Exponential O(N^2)Quadratic O(N log N)Linear/Log O(N)Linear O(log N)Log](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d795503460f94a5c196/html5/thumbnails/28.jpg)
Repeat the processRepeat the process
32 26
19
711
214
![Page 29: Sorting Algorithms and Analysis Robert Duncan. Refresher on Big-O O(2^N)Exponential O(N^2)Quadratic O(N log N)Linear/Log O(N)Linear O(log N)Log](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d795503460f94a5c196/html5/thumbnails/29.jpg)
Repeat the processRepeat the process
32 26 19
711
214
![Page 30: Sorting Algorithms and Analysis Robert Duncan. Refresher on Big-O O(2^N)Exponential O(N^2)Quadratic O(N log N)Linear/Log O(N)Linear O(log N)Log](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d795503460f94a5c196/html5/thumbnails/30.jpg)
Merge SortMerge SortUses recursion. Slightly faster than heap, but
uses twice as much memory from the 2nd array.
Sometimes called “divide and conquer” sort.
Works by recursively splitting an array into two equal halves, sorting the items, then re-merging them back into a new array.
![Page 31: Sorting Algorithms and Analysis Robert Duncan. Refresher on Big-O O(2^N)Exponential O(N^2)Quadratic O(N log N)Linear/Log O(N)Linear O(log N)Log](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d795503460f94a5c196/html5/thumbnails/31.jpg)
11 26 14 2 32 18 7 21
32 18 7 2111 26 14 2
11 26 14 2 32 18 7 21
![Page 32: Sorting Algorithms and Analysis Robert Duncan. Refresher on Big-O O(2^N)Exponential O(N^2)Quadratic O(N log N)Linear/Log O(N)Linear O(log N)Log](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d795503460f94a5c196/html5/thumbnails/32.jpg)
11 26 2 14 18 32 7 21
![Page 33: Sorting Algorithms and Analysis Robert Duncan. Refresher on Big-O O(2^N)Exponential O(N^2)Quadratic O(N log N)Linear/Log O(N)Linear O(log N)Log](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d795503460f94a5c196/html5/thumbnails/33.jpg)
18 32 7 2111 26 2 14
11 26 2 14 18 32 7 21
![Page 34: Sorting Algorithms and Analysis Robert Duncan. Refresher on Big-O O(2^N)Exponential O(N^2)Quadratic O(N log N)Linear/Log O(N)Linear O(log N)Log](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d795503460f94a5c196/html5/thumbnails/34.jpg)
7 18 21 322 11 14 26
![Page 35: Sorting Algorithms and Analysis Robert Duncan. Refresher on Big-O O(2^N)Exponential O(N^2)Quadratic O(N log N)Linear/Log O(N)Linear O(log N)Log](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d795503460f94a5c196/html5/thumbnails/35.jpg)
2 11 14 26 7 18 21 32
7 18 21 322 11 14 26
![Page 36: Sorting Algorithms and Analysis Robert Duncan. Refresher on Big-O O(2^N)Exponential O(N^2)Quadratic O(N log N)Linear/Log O(N)Linear O(log N)Log](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d795503460f94a5c196/html5/thumbnails/36.jpg)
2 11 14 26 7 18 21 32
![Page 37: Sorting Algorithms and Analysis Robert Duncan. Refresher on Big-O O(2^N)Exponential O(N^2)Quadratic O(N log N)Linear/Log O(N)Linear O(log N)Log](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d795503460f94a5c196/html5/thumbnails/37.jpg)
2 7 11 14 18 21 26 32
![Page 38: Sorting Algorithms and Analysis Robert Duncan. Refresher on Big-O O(2^N)Exponential O(N^2)Quadratic O(N log N)Linear/Log O(N)Linear O(log N)Log](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d795503460f94a5c196/html5/thumbnails/38.jpg)
Quick SortQuick Sort
The most efficient O(N log N) algorithm available
Works by first randomly choosing a pivot point which is hopefully a median element in the array. All elements less than the pivot are transferred to a new array, and all elements greater are transferred to a second array.
![Page 39: Sorting Algorithms and Analysis Robert Duncan. Refresher on Big-O O(2^N)Exponential O(N^2)Quadratic O(N log N)Linear/Log O(N)Linear O(log N)Log](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d795503460f94a5c196/html5/thumbnails/39.jpg)
Quick SortQuick Sort
These new arrays are recursively quick sorted until they have been split down to a single element. The sorted elements smaller than the pivot are placed in a new array, then the pivot is placed after, and finally the elements greater than the pivot. These elements are now sorted.
![Page 40: Sorting Algorithms and Analysis Robert Duncan. Refresher on Big-O O(2^N)Exponential O(N^2)Quadratic O(N log N)Linear/Log O(N)Linear O(log N)Log](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d795503460f94a5c196/html5/thumbnails/40.jpg)
Quick SortQuick Sort
Note: Although it is the quickest sorting algorithm, a badly chosen pivot may cause Quick Sort to run at O(N^2).
Different versions of quick sort address this problem in different ways.
For example, one way is to randomly choose 3 elements, then use the median element as the pivot.
![Page 41: Sorting Algorithms and Analysis Robert Duncan. Refresher on Big-O O(2^N)Exponential O(N^2)Quadratic O(N log N)Linear/Log O(N)Linear O(log N)Log](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d795503460f94a5c196/html5/thumbnails/41.jpg)
11 26 14 2 32 18 7 21 5
Pivot
![Page 42: Sorting Algorithms and Analysis Robert Duncan. Refresher on Big-O O(2^N)Exponential O(N^2)Quadratic O(N log N)Linear/Log O(N)Linear O(log N)Log](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d795503460f94a5c196/html5/thumbnails/42.jpg)
11 26 14 2 32 18 7 21 5
11 2 7 5
Pivot
![Page 43: Sorting Algorithms and Analysis Robert Duncan. Refresher on Big-O O(2^N)Exponential O(N^2)Quadratic O(N log N)Linear/Log O(N)Linear O(log N)Log](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d795503460f94a5c196/html5/thumbnails/43.jpg)
11 2 7 5
Pivot
11 26 14 2 32 18 7 21 5
![Page 44: Sorting Algorithms and Analysis Robert Duncan. Refresher on Big-O O(2^N)Exponential O(N^2)Quadratic O(N log N)Linear/Log O(N)Linear O(log N)Log](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d795503460f94a5c196/html5/thumbnails/44.jpg)
11 26 14 2 32 18 7 21 5
11 2 7 5
2 5
Pivot
![Page 45: Sorting Algorithms and Analysis Robert Duncan. Refresher on Big-O O(2^N)Exponential O(N^2)Quadratic O(N log N)Linear/Log O(N)Linear O(log N)Log](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d795503460f94a5c196/html5/thumbnails/45.jpg)
11 26 14 2 32 18 7 21 5
2 5 711 2 7 5
2 5 2 5
Pivot
![Page 46: Sorting Algorithms and Analysis Robert Duncan. Refresher on Big-O O(2^N)Exponential O(N^2)Quadratic O(N log N)Linear/Log O(N)Linear O(log N)Log](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d795503460f94a5c196/html5/thumbnails/46.jpg)
11 26 14 2 32 18 7 21 5
2 5 7 1111 2 7 5
2 5 2 5
![Page 47: Sorting Algorithms and Analysis Robert Duncan. Refresher on Big-O O(2^N)Exponential O(N^2)Quadratic O(N log N)Linear/Log O(N)Linear O(log N)Log](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d795503460f94a5c196/html5/thumbnails/47.jpg)
What’s the pointWhat’s the point
![Page 48: Sorting Algorithms and Analysis Robert Duncan. Refresher on Big-O O(2^N)Exponential O(N^2)Quadratic O(N log N)Linear/Log O(N)Linear O(log N)Log](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d795503460f94a5c196/html5/thumbnails/48.jpg)
Binary Searching Binary Searching Binary searches achieve O(log N) efficiency on
sorted dataSimilar to High-Low gameEach execution eliminates half of the elements
to search forAlthough hashing offers a quicker search of
O(1), binary searches are simpler, and use much less memory.
![Page 49: Sorting Algorithms and Analysis Robert Duncan. Refresher on Big-O O(2^N)Exponential O(N^2)Quadratic O(N log N)Linear/Log O(N)Linear O(log N)Log](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d795503460f94a5c196/html5/thumbnails/49.jpg)
Binary SearchingBinary Searching
2 7 11 14 16 23 26 29 32 37 43
14?
![Page 50: Sorting Algorithms and Analysis Robert Duncan. Refresher on Big-O O(2^N)Exponential O(N^2)Quadratic O(N log N)Linear/Log O(N)Linear O(log N)Log](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d795503460f94a5c196/html5/thumbnails/50.jpg)
Binary SearchingBinary Searching
2 7 11 14 16 23 26 29 32 37 43
14?
![Page 51: Sorting Algorithms and Analysis Robert Duncan. Refresher on Big-O O(2^N)Exponential O(N^2)Quadratic O(N log N)Linear/Log O(N)Linear O(log N)Log](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d795503460f94a5c196/html5/thumbnails/51.jpg)
Binary SearchingBinary Searching
2 7 11 14 16 23 26 29 32 37 43
14
![Page 52: Sorting Algorithms and Analysis Robert Duncan. Refresher on Big-O O(2^N)Exponential O(N^2)Quadratic O(N log N)Linear/Log O(N)Linear O(log N)Log](https://reader036.vdocuments.net/reader036/viewer/2022062320/56649d795503460f94a5c196/html5/thumbnails/52.jpg)
Conclusion…Conclusion…O(N^2) algorithms are almost never
used in professional software
Quick sort is generally considered to be the best overall sorting algorithm currently available.