algorithm design and analysisudel.edu/~amotong/teaching/algorithm design/lectures/(lec...radix sort...
TRANSCRIPT
Algorithm Design and Analysis
Version: Summer 2018 Amo G. Tong 1
Lecture 1-1Non-comparison Sorting• Lower bound of comparison Sorting
• Counting Sort
• Radix Sort
Version: Summer 2018 Amo G. Tong 2
• Comparison Sort• In a comparison sort, we use only comparisons between
elements to gain order information.• Insertion sort, merge sort, heapsort, quicksort,…
• How does each comparison help us to find the correct permutations?• There are totally n! permutations.
Sorting
Version: Summer 2018 Amo G. Tong 3
• Comparison Sort• In a comparison sort, we use only comparisons between
elements to gain order information.• Insertion sort, merge sort, heapsort, quicksort,…
• How does each comparison help us to find the correct permutations?• There are totally n! permutations
• Each comparison eliminates half of the current feasible permutations.
Sorting
Version: Summer 2018 Amo G. Tong 4
• Comparison Sort• How does each comparison help us to find the correct
permutations?• There are totally n! permutations
• Each comparison eliminates half of the current feasible permutations.
Sorting
Version: Summer 2018 Amo G. Tong 5
Example:
Input: 0, 9, 2, 1𝑎1, 𝑎2, 𝑎3, 𝑎4
Group1={permutations with 𝑎1 < 𝑎2}; Group2={permutations with 𝑎1 > 𝑎2}.After comparing 𝑎1 and 𝑎2 , we know 𝑎1 < 𝑎2 and thus only have to consider
Group1.
• Comparison Sort• How does each comparison help us to find the correct
permutations?• There are totally n! permutations
• Each comparison eliminates half of the current feasible permutations.
• So the number of comparisons needed in the worst-case is 𝐥𝐨𝐠𝒏!, and thus 𝑻(𝒏) = 𝛀(𝐥𝐨𝐠𝒏!)
Sorting
Version: Summer 2018 Amo G. Tong 6
Some Math: 𝑛! ≥𝑛
2
𝑛/2⇒ 𝑇 𝑛 = Ω(𝑛 log 𝑛)
• Comparison Sort• How does each comparison help us to find the correct
permutations?• There are totally n! permutations
• Each comparison eliminates half of the current feasible permutations.
• So the number of comparisons needed in the worst-case is log 𝑛!, and thus 𝑇(𝑛) = Ω(log𝑛!)
• We cannot do better than 𝚯(𝒏 𝐥𝐨𝐠𝒏) by comparing elements.
Sorting
Version: Summer 2018 Amo G. Tong 7
Some Math: 𝑛! ≥𝑛
2
Τ𝑛 2⇒ 𝑇 𝑛 = Ω(𝑛 log 𝑛)
• Counting Sort
Counting Sort
Version: Summer 2018 Amo G. Tong 8
• Counting Sort• Input: a sequence 𝑎1, … , 𝑎𝑛 of integers in the range 0,… , 𝑘 − 1.• For each value 𝑥 ∈ {0,… , 𝑘 − 1}, count the number of 𝑥 in
𝑎1, … , 𝑎𝑛 .
• List the output.
Counting Sort
Version: Summer 2018 Amo G. Tong 9
• Counting Sort
Counting Sort
Version: Summer 2018 Amo G. Tong 10
Example:
Input: 0, 9, 2, 1 with 𝑘 = 10;Step 1: 𝒄[𝒙] = 𝟎 for 𝒙 ∈ {𝟎,… , 𝒌 − 𝟏}.
• Counting Sort
Counting Sort
Version: Summer 2018 Amo G. Tong 11
Example:
Input: 0, 9, 2, 1 with 𝑘 = 10;Step 1: 𝑐[𝑥] = 0 for 𝑥 ∈ {0, … , 𝑘 − 1}.Step 2: 𝒄 𝟎 = 𝒄 𝟎 + 𝟏;
𝒄 𝟗 = 𝒄 𝟗 + 𝟏;𝒄 𝟐 = 𝒄 𝟐 + 𝟏;𝒄 𝟏 = 𝒄 𝟏 + 𝟏;
• Counting Sort
Counting Sort
Version: Summer 2018 Amo G. Tong 12
Example:
Input: 0, 9, 2, 1 with 𝑘 = 10;Step 1: 𝑐[𝑥] = 0 for 𝑥 ∈ {0, … , 𝑘 − 1}.Step 2: 𝑐 0 = 𝑐 0 + 1;
𝑐 9 = 𝑐 9 + 1;𝑐 2 = 𝑐 2 + 1;𝑐 1 = 𝑐 1 + 1;
Step 3: 𝒄 𝟎 =1, 0;𝒄 𝟏 =1, 0, 1;𝒄 𝟐 =1, 0, 1, 2;𝒄 𝟑 =0, 0, 1, 2;….𝒄 𝟗 =1, 0, 1, 2, 9;
• Counting Sort
Counting Sort
Version: Summer 2018 Amo G. Tong 13
Pseudocode:
COUNTING-SORT(𝐴, 𝑛, 𝑘)
1 for 𝑖 = 0 to 𝑘 − 12 𝑐 𝑖 = 03 end4 for 𝑖 = 1 to 𝑛5 𝑐 𝐴[𝑖] ++6 end7 𝑖𝑛𝑑𝑒𝑥 = 1;8 for 𝑖 = 0 to 𝑘 − 19 for 𝑗 = 0 to 𝑐 𝑖10 𝐵 𝑖𝑛𝑑𝑒𝑥 = 𝑖11 𝑖𝑛𝑑𝑒𝑥++12 end13 end14 Output B;
• Counting Sort
Counting Sort
Version: Summer 2018 Amo G. Tong 14
Pseudocode:
COUNTING-SORT(𝐴, 𝑛, 𝑘)
1 for 𝑖 = 0 to 𝑘 − 12 𝑐 𝑖 = 0 //initialize 3 end4 for 𝑖 = 1 to 𝑛5 𝑐 𝐴[𝑖] ++ //count 6 end7 𝑖𝑛𝑑𝑒𝑥 = 1;8 for 𝑖 = 0 to 𝑘 − 19 for 𝑗 = 0 to 𝑐 𝑖10 𝐵 𝑖𝑛𝑑𝑒𝑥 = 𝑖 //copy the output to B11 𝑖𝑛𝑑𝑒𝑥++12 end13 end14 Output B;
• Counting Sort
Counting Sort
Version: Summer 2018 Amo G. Tong 15
Pseudocode:
COUNTING-SORT(𝐴, 𝑛, 𝑘)
1 for 𝑖 = 0 to 𝑘 − 12 𝑐 𝑖 = 0 Θ(𝑘)3 end4 for 𝑖 = 1 to 𝑛5 𝑐 𝐴[𝑖] ++ Θ(𝑛)6 end7 𝑖𝑛𝑑𝑒𝑥 = 1;8 for 𝑖 = 0 to 𝑘 − 19 for 𝑗 = 0 to 𝑐 𝑖10 𝐵 𝑖𝑛𝑑𝑒𝑥 = 𝑖 Θ(𝑐[𝑖] + 1)11 𝑖𝑛𝑑𝑒𝑥++12 end13 end14 Output B;
Θ 𝑐 0 + 𝑐 1 +⋯+ 𝑐 𝑖 − 1 + 𝑘= Θ(𝑛 + 𝑘)
Totally, Θ(𝑛 + 𝑘)
• Counting Sort• Input: a sequence 𝑎1, … , 𝑎𝑛 of integers in the range 0,… , 𝑘 − 1.• For each value 𝑥 ∈ {0,… , 𝑘 − 1}, count the number of x in
𝑎1, … , 𝑎𝑛 .
• List the output.
• Totally, 𝚯(𝒏 + 𝒌), beating 𝚯(𝒏 𝐥𝐨𝐠𝒏). WHY?• Essentially, we have more information about the input.
Counting Sort
Version: Summer 2018 Amo G. Tong 16
• Counting Sort• Input: a sequence 𝑎1, … , 𝑎𝑛 of integers in the range 0,… , 𝑘 − 1.• For each value 𝑥 ∈ {0,… , 𝑘 − 1}, count the number of x in
𝑎1, … , 𝑎𝑛 .
• List the output.
• Totally, 𝚯(𝒏 + 𝒌), beating 𝚯(𝒏 𝐥𝐨𝐠𝒏). WHY?• Essentially, we have more information about the input.
Counting Sort
Version: Summer 2018 Amo G. Tong 17
• A sort is stable if it preserves the input order among equal elements.
Stable Sort
Version: Summer 2018 Amo G. Tong 18
0 9 2 2 1
𝑎1 𝑎2 𝑎3 𝑎4 𝑎5input
output
output
𝑎1 𝑎5 𝑎2 𝑎3 𝑎4
𝑎1 𝑎5 𝑎2 𝑎4 𝑎3
stable
not stable
• A sort is stable if it preserves the input order among equal elements.
• Counting sort can be implemented to be stable.(Exercise)
Stable Sort
Version: Summer 2018 Amo G. Tong 19
• Sort the elements on each digit from the lowest digit position to the highest digit position.
• Use a stable sort for each sorting.
Radix Sort
Version: Summer 2018 Amo G. Tong 20
020911291299
020911291299
• Sort the elements on each digit from the lowest digit position to the highest digit position.
• Use a stable sort for each sorting.
Radix Sort
Version: Summer 2018 Amo G. Tong 21
020911291299
020911291299
911020291299
• Sort the elements on each digit from the lowest digit position to the highest digit position.
• Use a stable sort for each sorting.
Radix Sort
Version: Summer 2018 Amo G. Tong 22
020911291299
020911291299
911020291299
020291299911
• Why a stable sort?
Radix Sort
Version: Summer 2018 Amo G. Tong 23
20119199
20119199
• Why a stable sort?
Radix Sort
Version: Summer 2018 Amo G. Tong 24
20119199
20119199
20119199
20119991
stable
not stable
• Correctness:
• Suppose that the numbers are sorted by their low-order 𝑡 – 1 digits.
• They will be correctly sorted by the low-order 𝑡 digits.
Radix Sort
Version: Summer 2018 Amo G. Tong 25
• Correctness:
• Suppose that the numbers are sorted by their low-order 𝑡 – 1 digits.
• They will be correctly sorted by the low-order 𝑡 digits.
• Two numbers that differ in digit t are correctly sorted, because digit 𝑡 is the most significant so far.
• Two numbers have the same digit t are correctly sorted, because we use a stable sort.
Radix Sort
Version: Summer 2018 Amo G. Tong 26
• Correctness:
• Suppose that the numbers are sorted by their low-order 𝑡 – 1 digits.
• They will be correctly sorted by the low-order 𝑡 digits.
• Two numbers that differ in digit t are correctly sorted, because digit 𝑡 is the most significant so far.
• Two numbers have the same digit t are correctly sorted, because we use a stable sort.
• The correctness follows inductively.
Radix Sort
Version: Summer 2018 Amo G. Tong 27