1 algorithms cs/apma 202 rosen section 2.1 aaron bloomfield
TRANSCRIPT
11
AlgorithmsAlgorithms
CS/APMA 202CS/APMA 202
Rosen section 2.1Rosen section 2.1
Aaron BloomfieldAaron Bloomfield
22
What is an algorithm?What is an algorithm?
An algorithm is “a finite set of precise An algorithm is “a finite set of precise instructions for performing a computation or for instructions for performing a computation or for solving a problem”solving a problem” A program is one type of algorithmA program is one type of algorithm
All programs are algorithmsAll programs are algorithms
Not all algorithms are programs!Not all algorithms are programs! Directions to somebody’s house is an algorithmDirections to somebody’s house is an algorithm A recipe for cooking a cake is an algorithmA recipe for cooking a cake is an algorithm The steps to compute the cosine of 90The steps to compute the cosine of 90°° is an is an
algorithmalgorithm
33
Some algorithms are harder than Some algorithms are harder than othersothers
Some algorithms are easySome algorithms are easy Finding the largest (or smallest) value in a listFinding the largest (or smallest) value in a list Finding a specific value in a listFinding a specific value in a list
Some algorithms are a bit harderSome algorithms are a bit harder Sorting a listSorting a list
Some algorithms are very hardSome algorithms are very hard Finding the shortest path between Miami and SeattleFinding the shortest path between Miami and Seattle
Some algorithms are essentially impossibleSome algorithms are essentially impossible Factoring large composite numbersFactoring large composite numbers
In section 2.2, we’ll see how to rate how “hard” In section 2.2, we’ll see how to rate how “hard” algorithms arealgorithms are
44
Algorithm 1: Maximum elementAlgorithm 1: Maximum element
Given a list, how do we find the maximum Given a list, how do we find the maximum element in the list?element in the list?
To express the algorithm, we’ll use To express the algorithm, we’ll use pseudocodepseudocode Pseudocode is kinda like a programming Pseudocode is kinda like a programming
language, but not reallylanguage, but not really
55
Algorithm 1: Maximum elementAlgorithm 1: Maximum element
Algorithm for finding the maximum Algorithm for finding the maximum element in a list:element in a list:
procedureprocedure max ( max (aa11, , aa22, …, , …, aann: integers): integers)
maxmax := := aa11
forfor i := 2 i := 2 toto n n
ifif max < max < aaii thenthen maxmax := := aaii
{{maxmax is the largest element} is the largest element}
66
procedureprocedure max ( max (aa11, , aa22, …, , …, aann: integers): integers)
maxmax := := aa11
forfor ii := 2 := 2 toto n n
ifif max < max < aaii thenthen maxmax := := aaii
maxmax := := aa11
forfor ii := 2 := 2 toto n n
ifif max < max < aaii thenthen maxmax := := aaii
Algorithm 1: Maximum elementAlgorithm 1: Maximum element
44 11 77 00 55 22 99 33 66 88
aa11 aa22 aa33 aa44 aa55 aa66 aa77 aa88 aa99 aa1010
maxmax
ii 2345678910
479
77
Maximum element running timeMaximum element running time
How long does this take?How long does this take?
If the list has If the list has nn elements, worst case elements, worst case scenario is that it takes scenario is that it takes nn “steps” “steps” Here, a step is considered a single step Here, a step is considered a single step
through the listthrough the list
88
Properties of algorithmsProperties of algorithms
Algorithms generally share a set of properties:Algorithms generally share a set of properties: Input: what the algorithm takes in as inputInput: what the algorithm takes in as input Output: what the algorithm produces as outputOutput: what the algorithm produces as output Definiteness: the steps are defined preciselyDefiniteness: the steps are defined precisely Correctness: should produce the correct outputCorrectness: should produce the correct output Finiteness: the steps required should be finiteFiniteness: the steps required should be finite Effectiveness: each step must be able to be Effectiveness: each step must be able to be
performed in a finite amount of timeperformed in a finite amount of time Generality: the algorithm Generality: the algorithm shouldshould be applicable to all be applicable to all
problems of a similar formproblems of a similar form
99
Searching algorithmsSearching algorithms
Given a list, find a specific element in the Given a list, find a specific element in the listlist
We will see two typesWe will see two types Linear searchLinear search
a.k.a. sequential searcha.k.a. sequential search Binary searchBinary search
1010
Algorithm 2: Linear searchAlgorithm 2: Linear search
Given a list, find a specific element in the listGiven a list, find a specific element in the list List does NOT have to be sorted!List does NOT have to be sorted!
procedureprocedure linear_search ( linear_search (xx: integer; : integer; aa11, , aa22, …, , …, aann: : integers)integers)
ii := 1 := 1whilewhile ( ( ii ≤ ≤ nn and and xx ≠ ≠ aaii ) )
ii := := ii + 1 + 1ifif ii ≤≤ nn thenthen locationlocation := i := ielseelse locationlocation := 0 := 0
{{locationlocation is the subscript of the term that equals x, or it is the subscript of the term that equals x, or it is 0 if x is not found}is 0 if x is not found}
1111
procedureprocedure linear_search ( linear_search (xx: integer; : integer; aa11, , aa22, …, , …, aann: integers): integers)
ii := 1 := 1
whilewhile ( ( ii ≤ ≤ nn and and xx ≠ ≠ aaii ) )
ii := := ii + 1 + 1
ifif ii ≤≤ nn thenthen locationlocation := i := i
elseelse locationlocation := 0 := 0
ii := 1 := 1
whilewhile ( ( ii ≤ ≤ nn and and xx ≠ ≠ aaii ) )
ii := := ii + 1 + 1
ifif ii ≤≤ nn thenthen locationlocation := i := i
elseelse locationlocation := 0 := 0
Algorithm 2: Linear search, take 1Algorithm 2: Linear search, take 1
44 11 77 00 55 22 99 33 66 88
aa11 aa22 aa33 aa44 aa55 aa66 aa77 aa88 aa99 aa1010
ii 23456781
xx 3
locationlocation 8
1212
procedureprocedure linear_search ( linear_search (xx: integer; : integer; aa11, , aa22, …, , …, aann: integers): integers)
ii := 1 := 1
whilewhile ( ( ii ≤ ≤ nn and and xx ≠ ≠ aaii ) )
ii := := ii + 1 + 1
ifif ii ≤≤ nn thenthen locationlocation := i := i
elseelse locationlocation := 0 := 0
ii := 1 := 1
whilewhile ( ( ii ≤ ≤ nn and and xx ≠ ≠ aaii ) )
ii := := ii + 1 + 1
ifif ii ≤≤ nn thenthen locationlocation := i := i
elseelse locationlocation := 0 := 0
Algorithm 2: Linear search, take 2Algorithm 2: Linear search, take 2
44 11 77 00 55 22 99 33 66 88
aa11 aa22 aa33 aa44 aa55 aa66 aa77 aa88 aa99 aa1010
ii 23456789101
xx 11
locationlocation 0
11
1313
Linear search running timeLinear search running time
How long does this take?How long does this take?
If the list has If the list has nn elements, worst case elements, worst case scenario is that it takes scenario is that it takes nn “steps” “steps” Here, a step is considered a single step Here, a step is considered a single step
through the listthrough the list
1414
Algorithm 3: Binary searchAlgorithm 3: Binary search
Given a list, find a specific element in the listGiven a list, find a specific element in the list List MUST be sorted!List MUST be sorted!
Each time it iterates through, it cuts the list in halfEach time it iterates through, it cuts the list in half
procedureprocedure binary_search ( binary_search (xx: integer; : integer; aa11, , aa22, …, , …, aann: increasing integers): increasing integers)ii := 1 := 1 { { ii is left endpoint of search interval } is left endpoint of search interval }jj := := nn { { jj is right endpoint of search interval } is right endpoint of search interval }whilewhile ii < < jjbeginbegin
m := m := ((ii++jj)/2)/2 { { mm is the point in the middle } is the point in the middle }if if x > x > aamm thenthen ii := := mm+1+1elseelse j j := := mm
endendifif xx = = aaii thenthen locationlocation := := iielseelse locationlocation := 0 := 0
{{locationlocation is the subscript of the term that equals x, or it is 0 if x is not found} is the subscript of the term that equals x, or it is 0 if x is not found}
1515
Algorithm 3: Binary search, take 1Algorithm 3: Binary search, take 1
22 44 66 88 1010 1212 1414 1616 1818 2020
aa11 aa22 aa33 aa44 aa55 aa66 aa77 aa88 aa99 aa1010
ii jjmm
ii := 1 := 1
jj := := nn
procedureprocedure binary_search ( binary_search (xx: integer; : integer; aa11, , aa22, …, , …, aann: increasing integers): increasing integers)
whilewhile ii < < jj
beginbegin
m := m := ((ii++jj)/2)/2if if x > x > aamm thenthen ii := := mm+1+1
elseelse j j := := mm
endend
ifif xx = = aaii thenthen locationlocation := := ii
elseelse locationlocation := 0 := 0
ii := 1 := 1
jj := := nn
whilewhile ii < < jj
beginbegin
m := m := ((ii++jj)/2)/2if if x > x > aamm thenthen ii := := mm+1+1
elseelse j j := := mm
endend
ifif xx = = aaii thenthen locationlocation := := ii
1
xx 14
1056 8 87 767
locationlocation 7
1616
Algorithm 3: Binary search, take 2Algorithm 3: Binary search, take 2
22 44 66 88 1010 1212 1414 1616 1818 2020
aa11 aa22 aa33 aa44 aa55 aa66 aa77 aa88 aa99 aa1010
ii jjmm
ii := 1 := 1
jj := := nn
procedureprocedure binary_search ( binary_search (xx: integer; : integer; aa11, , aa22, …, , …, aann: increasing integers): increasing integers)
whilewhile ii < < jj
beginbegin
m := m := ((ii++jj)/2)/2if if x > x > aamm thenthen ii := := mm+1+1
elseelse j j := := mm
endend
ifif xx = = aaii thenthen locationlocation := := ii
elseelse locationlocation := 0 := 0
ii := 1 := 1
jj := := nn
whilewhile ii < < jj
beginbegin
m := m := ((ii++jj)/2)/2if if x > x > aamm thenthen ii := := mm+1+1
elseelse j j := := mm
endend
ifif xx = = aaii thenthen locationlocation := := II
elseelse locationlocation := 0 := 0
1
xx 15
1056 8 87
locationlocation 0
8
1717
Algorithm 3: Binary searchAlgorithm 3: Binary search
A somewhat alternative view of what a A somewhat alternative view of what a binary search does…binary search does…
18
Quick surveyQuick survey
I felt I understood the search I felt I understood the search algorithms…algorithms…
a)a) Very wellVery well
b)b) With some review, I’ll be goodWith some review, I’ll be good
c)c) Not reallyNot really
d)d) Not at allNot at all
1919
Today’s demotivatorsToday’s demotivators
2020
Binary search running timeBinary search running time
How long does this take (worst case)?How long does this take (worst case)?
If the list has 8 elementsIf the list has 8 elements It takes 3 stepsIt takes 3 steps
If the list has 16 elementsIf the list has 16 elements It takes 4 stepsIt takes 4 steps
If the list has 64 elementsIf the list has 64 elements It takes 6 stepsIt takes 6 steps
If the list has If the list has nn elements elements It takes logIt takes log22 nn steps steps
2121
Sorting algorithmsSorting algorithms
Given a list, put it into some orderGiven a list, put it into some order Numerical, lexicographic, etc.Numerical, lexicographic, etc.
We will see two typesWe will see two types Bubble sortBubble sort Insertion sortInsertion sort
2222
Algorithm 4: Bubble sortAlgorithm 4: Bubble sort
One of the most simple sorting algorithmsOne of the most simple sorting algorithms Also one of the least efficientAlso one of the least efficient
It takes successive elements and “bubbles” them It takes successive elements and “bubbles” them up the listup the list
procedureprocedure bubble_sort ( bubble_sort (aa11, , aa22, …, , …, aann))
for for ii := 1 := 1 to to nn-1-1forfor jj := 1 := 1 toto nn--ii
ifif aajj > > aajj+1+1
then then interchange interchange aajj and and aajj+1+1
{ { aa11, …, , …, aann are in increasing order } are in increasing order }
2424
Algorithm 4: Bubble sortAlgorithm 4: Bubble sort
An example using physical objects…An example using physical objects…
2525
Bubble sort running timeBubble sort running time
Bubble sort algorithm:Bubble sort algorithm:for for ii := 1 := 1 to to nn-1-1
forfor jj := 1 := 1 toto nn--ii
ifif aajj > > aajj+1+1
then then interchange interchange aajj and and aajj+1+1
Outer for loop does n-1 iterationsOuter for loop does n-1 iterationsInner for loop does Inner for loop does
nn-1 iterations the first time-1 iterations the first time nn-2 iterations the second time-2 iterations the second time …… 1 iteration the last time1 iteration the last time
Total: (Total: (nn-1) + (-1) + (nn-2) + (-2) + (nn-3) + … + 2 + 1 = (-3) + … + 2 + 1 = (nn22--nn)/2)/2 We can say that’s “about” We can say that’s “about” nn22 time time
2626
Algorithm 5: Insertion sortAlgorithm 5: Insertion sort
Another simple (and inefficient) algorithmAnother simple (and inefficient) algorithmIt starts with a list with one element, and inserts new elements into It starts with a list with one element, and inserts new elements into their proper place in the sorted part of the listtheir proper place in the sorted part of the list
procedureprocedure insertion_sort ( insertion_sort (aa11, , aa22, …, , …, aann))forfor jj := 2 := 2 toto nnbeginbegin
ii := 1 := 1whilewhile aajj > > aaii
ii := := ii +1 +1mm := := aajj
forfor kk := 0 := 0 toto jj--ii-1-1aajj--kk := := aajj--kk-1-1
aaii := := mmendend { { aa11, , aa22, …, , …, aann are sorted } are sorted }
take successive elements in the list
find where that element should be in the sorted portion of the list
move all elements in the sorted portion of the list that are greater
than the current element up by one
put the current element into it’s proper place in the sorted portion of the list
2727
Insertion sort running timeInsertion sort running time
forfor jj := 2 := 2 toto n n beginbeginii := 1 := 1whilewhile aajj > > aaii
ii := := ii +1 +1mm := := aajj
forfor kk := 0 := 0 toto jj--ii-1-1aajj--kk := := aajj--kk-1-1
aaii := := mmendend { { aa11, , aa22, …, , …, aann are sorted } are sorted }Outer for loop runs Outer for loop runs nn-1 times-1 timesIn the inner for loop:In the inner for loop:
Worst case is when the while keeps Worst case is when the while keeps ii at 1, and the for loop runs lots of at 1, and the for loop runs lots of timestimes
If If ii is 1, the inner for loop runs 1 time ( is 1, the inner for loop runs 1 time (kk goes from 0 to 0) on the first goes from 0 to 0) on the first iteration, 1 time on the second, up to iteration, 1 time on the second, up to nn-2 times on the last iteration-2 times on the last iteration
Total is 1 + 2 + … + Total is 1 + 2 + … + nn-2 = (-2 = (nn-1)(-1)(nn-2)/2-2)/2 We can say that’s “about” We can say that’s “about” nn22 time time
28
Quick surveyQuick survey
I felt I understood the sort I felt I understood the sort algorithms…algorithms…
a)a) Very wellVery well
b)b) With some review, I’ll be goodWith some review, I’ll be good
c)c) Not reallyNot really
d)d) Not at allNot at all
2929
Comparison of running timesComparison of running times
SearchesSearches Linear: Linear: nn steps steps Binary: logBinary: log22 nn steps steps Binary search is about as fast as you can getBinary search is about as fast as you can get
SortsSorts Bubble: Bubble: nn22 steps steps Insertion: Insertion: nn22 steps steps There are other, more efficient, sorting techniquesThere are other, more efficient, sorting techniques
In principle, the fastest are heap sort, quick sort, and merge In principle, the fastest are heap sort, quick sort, and merge sortsortThese each take take These each take take nn * log * log22 nn steps stepsIn practice, quick sort is the fastest, followed by merge sortIn practice, quick sort is the fastest, followed by merge sort
3030
Greedy algorithmsGreedy algorithms
Ignore this part…Ignore this part…
31
Quick surveyQuick survey
I felt I understood the material in this I felt I understood the material in this slide set…slide set…
a)a) Very wellVery well
b)b) With some review, I’ll be goodWith some review, I’ll be good
c)c) Not reallyNot really
d)d) Not at allNot at all
32
Quick surveyQuick survey
The pace of the lecture for this The pace of the lecture for this slide set was…slide set was…
a)a) FastFast
b)b) About rightAbout right
c)c) A little slowA little slow
d)d) Too slowToo slow
33
Quick surveyQuick survey
How interesting was the material in How interesting was the material in this slide set? Be honest!this slide set? Be honest!
a)a) Wow! That was SOOOOOO cool!Wow! That was SOOOOOO cool!
b)b) Somewhat interestingSomewhat interesting
c)c) Rather bortingRather borting
d)d) ZzzzzzzzzzzZzzzzzzzzzz