SEARCHING AND SORTING ALGORITHMS (download slides and .py files and follow along!)

6.0001 LECTURE 12

6.0001LECTURE12 1

SEARCH ALGORITHMS §searchalgorithm–methodforfindinganitemorgroupofitemswithspecificproperAeswithinacollecAonofitems§collecAoncouldbeimplicit◦ example–findsquarerootasasearchproblem◦ exhausAveenumeraAon◦ bisecAonsearch◦ Newton-Raphson

§collecAoncouldbeexplicit◦ example–isastudentrecordinastoredcollecAonofdata?

6.0001LECTURE12 2

SEARCHING ALGORITHMS §linearsearch• bruteforcesearch(akaBriAshMuseumalgorithm)• listdoesnothavetobesorted

§bisecAonsearch• listMUSTbesortedtogivecorrectanswer• sawtwodifferentimplementaAonsofthealgorithm

6.0001LECTURE12 3

LINEAR SEARCH ON UNSORTED LIST: RECAP def linear_search(L, e): found = False for i in range(len(L)):

if e == L[i]:found = True

return found

§mustlookthroughallelementstodecideit’snotthere

§O(len(L))fortheloop*O(1)totestife==L[i]

§overallcomplexityisO(n)–wherenislen(L)

6.0001LECTURE12 4

LINEAR SEARCH ON SORTED LIST: RECAP def search(L, e): for i in range(len(L)): if L[i] == e: return True if L[i] > e: return False return False

§ mustonlylookunAlreachanumbergreaterthane

§ O(len(L))fortheloop*O(1)totestife==L[i]§ overallcomplexityisO(n)–wherenislen(L)

6.0001LECTURE12 5

USE BISECTION SEARCH: RECAP 1. Pickanindex,i,thatdivideslistinhalf2. AskifL[i] == e3. Ifnot,askifL[i] islargerorsmallerthane4. Dependingonanswer,searchle_orrighthalfof L fore

Anewversionofadivide-and-conqueralgorithm§ Breakintosmallerversionofproblem(smallerlist),plus

somesimpleoperaAons§ Answertosmallerversionisanswertooriginalproblem

6.0001LECTURE12 6

def bisect_search2(L, e): def bisect_search_helper(L, e, low, high):

if high == low:return L[low] == e

mid = (low + high)//2if L[mid] == e:

return Trueelif L[mid] > e:

if low == mid: #nothing left to searchreturn False

else:return bisect_search_helper(L, e, low, mid - 1)

else:return bisect_search_helper(L, e, mid + 1, high)

if len(L) == 0:return False

else:return bisect_search_helper(L, e, 0, len(L) - 1)

BISECTION SEARCH IMPLEMENTATION: RECAP

6.0001LECTURE12 7

COMPLEXITY OF BISECTION SEARCH: RECAP §bisect_search2anditshelper• O(logn)bisecAonsearchcalls• reducesizeofproblembyfactorof2oneachstep• passlistandindicesasparameters• listnevercopied,justre-passedaspointer• constantworkinsidefuncAon• àO(logn)

6.0001LECTURE12 8

SEARCHING A SORTED LIST -- n is len(L) §usinglinearsearch,searchforanelementisO(n)

§usingbinarysearch,cansearchforanelementinO(logn)• assumesthelistissorted!

§whendoesitmakesensetosortfirstthensearch?• SORT+O(log n)<O(n) àSORT<O(n)–O(log n)• whensorAngislessthanO(n)

•NEVERTRUE!• tosortacollecEonofnelementsmustlookateachoneatleastonce!

6.0001LECTURE12 9

AMORTIZED COST -- n is len(L) § whybothersorAngfirst?§ insomecases,maysortalistoncethendomanysearches

§ AMORTIZEcostofthesortovermanysearches

§ SORT+K*O(log n)<K*O(n) àforlargeK,SORTEmebecomesirrelevant,if

costofsorAngissmallenough

6.0001LECTURE12 10

SORT ALGORITHMS §Wanttoefficientlysortalistofentries(typicallynumbers)

§Willseearangeofmethods,includingonethatisquiteefficient

6.0001LECTURE12 11

MONKEY SORT §akabogosort,stupidsort,slowsort,permutaAonsort,shotgunsort

§tosortadeckofcards• throwthemintheair• pickthemup• aretheysorted?• repeatifnotsorted

6.0001LECTURE12 12

COMPLEXITY OF BOGO SORT def bogo_sort(L): while not is_sorted(L): random.shuffle(L)

§ bestcase:O(n)wherenislen(L)tocheckifsorted§ worstcase:O(?)itisunboundedifreallyunlucky

6.0001LECTURE12 13

BUBBLE SORT § compareconsecuEvepairsofelements§ swapelementsinpairsuchthatsmallerisfirst§ whenreachendoflist,startoveragain§ stopwhennomoreswapshavebeenmade§ largestunsortedelementalwaysatenda_erpass,so

6.0001LECTURE12 14

atmostnpassesCC-BYHydrargyrumhttps://commons.wikimedia.org/wiki/File:Bubble_sort_animation.gif�

COMPLEXITY OF BUBBLE SORT def bubble_sort(L): swap = False while not swap:

swap = Truefor j in range(1, len(L)):

if L[j-1] > L[j]:swap = Falsetemp = L[j]L[j] = L[j-1]L[j-1] = temp

§innerforloopisfordoingthecomparisons§outerwhileloopisfordoingmulEplepassesunAlnomoreswaps§O(n2)wherenislen(L) todolen(L)-1comparisonsandlen(L)-1passes

6.0001LECTURE12 15

SELECTION SORT § firststep•  extractminimumelement•  swapitwithelementatindex0

§ subsequentstep•  inremainingsublist,extractminimumelement•  swapitwiththeelementatindex1

§ keepthele_porAonofthelistsorted•  ati’thstep,firstielementsinlistaresorted•  allotherelementsarebiggerthanfirstielements

6.0001LECTURE12 16

ANALYZING SELECTION SORT §loopinvariant◦ givenprefixoflistL[0:i]andsuffixL[i+1:len(L)],thenprefixissortedandnoelementinprefixislargerthansmallestelementinsuffix1. basecase:prefixempty,suffixwholelist–invariant

true2. inducAonstep:moveminimumelementfromsuffix

toendofprefix.Sinceinvarianttruebeforemove,prefixsorteda_erappend

3. whenexit,prefixisenArelist,suffixempty,sosorted

6.0001LECTURE12 17

COMPLEXITY OF SELECTION SORT

def selection_sort(L): suffixSt = 0 while suffixSt != len(L):

for i in range(suffixSt, len(L)):if L[i] < L[suffixSt]:

L[suffixSt], L[i] = L[i], L[suffixSt]suffixSt += 1

§outerloopexecuteslen(L)Ames

§innerloopexecuteslen(L)–iAmes

§complexityofselecAonsortisO(n2)wherenislen(L)

6.0001LECTURE12 18

MERGE SORT §useadivide-and-conquerapproach:

1. iflistisoflength0or1,alreadysorted2. iflisthasmorethanoneelement,splitintotwolists,

andsorteach3. mergesortedsublists

1. lookatfirstelementofeach,movesmallertoendoftheresult

2. whenonelistempty,justcopyrestofotherlist

6.0001LECTURE12 19

MERGE SORT § divideandconquer

§ splitlistinhalfunAlhavesublistsofonly1element

unsorted

unsorted unsorted

unsorted unsorted unsorted unsorted

unsorted

unsorted

unsorted

unsorted

unsorted

unsorted

unsorted

unsorted

merge merge merge merge merge merge merge merge

6.0001LECTURE12 22

MERGE SORT § divideandconquer

§ mergesuchthatsublistswillbesortedaQermerge

unsorted

unsorted unsorted

unsorted unsorted unsorted unsorted

sort sort sort sort sort sort sort sort

merge merge merge merge

6.0001LECTURE12 23

MERGE SORT §divideandconquer

§mergesortedsublists

§sublistswillbesorteda_ermerge

unsorted

unsorted unsorted

sorted sorted sorted sorted

merge merge

6.0001LECTURE12 22

MERGE SORT § divideandconquer

§mergesortedsublists

§sublistswillbesorteda_ermerge

unsorted

sorted sorted

merge

6.0001LECTURE12 23

MERGE SORT §divideandconquer–done!

sorted

6.0001LECTURE12 24

EXAMPLE OF MERGING Le_inlist1Le_inlist2CompareResult[1,5,12,18,19,20][2,3,4,17]1,2[][5,12,18,19,20][2,3,4,17]5,2[1][5,12,18,19,20][3,4,17]5,3[1,2][5,12,18,19,20][4,17]5,4[1,2,3][5,12,18,19,20][17]5,17[1,2,3,4][12,18,19,20][17]12,17[1,2,3,4,5][18,19,20][17]18,17[1,2,3,4,5,12][18,19,20][]18,--[1,2,3,4,5,12,17][][][1,2,3,4,5,12,17,18,19,20]

6.0001LECTURE12 25

MERGING SUBLISTS STEP def merge(left, right): result = [] i,j = 0,0 while i < len(left) and j < len(right):

if left[i] < right[j]:result.append(left[i])i += 1

else:result.append(right[j])j += 1

while (i < len(left)):result.append(left[i])i += 1

while (j < len(right)):result.append(right[j])j += 1

return result

6.0001LECTURE12 26

COMPLEXITY OF MERGING SUBLISTS STEP §gothroughtwolists,onlyonepass

§compareonlysmallestelementsineachsublist

§O(len(le_)+len(right))copiedelements

§O(len(longerlist))comparisons

§linearinlengthofthelists

6.0001LECTURE12 27

MERGE SORT ALGORITHM -- RECURSIVE def merge_sort(L): if len(L) < 2:

return L[:] else:

middle = len(L)//2left = merge_sort(L[:middle])right = merge_sort(L[middle:])return merge(left, right)

§dividelistsuccessivelyintohalves

§depth-firstsuchthatconquersmallestpiecesdownonebranchfirstbeforemovingtolargerpieces

6.0001LECTURE12 28

84165920

8416

84

8

basecase

4

basecase

16

1

basecase

6

basecase

Merge48

Merge48&161468

Merge16

5920

59

5

basecase

9

basecase

20

2

basecase

0

basecase

Merge59

Merge59&020259

Merge02

Merge1468&025901245689

6.0001LECTURE12 29

COMPLEXITY OF MERGE SORT §atfirstrecursionlevel• n/2elementsineachlist• O(n)+O(n)=O(n)wherenislen(L)

§atsecondrecursionlevel• n/4elementsineachlist• twomergesàO(n)wherenislen(L)

§eachrecursionlevelisO(n)wherenislen(L)§dividinglistinhalfwitheachrecursivecall• O(log(n))wherenislen(L)

§overallcomplexityisO(nlog(n))wherenislen(L)

6.0001LECTURE12 30

SORTING SUMMARY -- n is len(L) §bogosort• randomness,unboundedO()

§bubblesort• O(n2)

§selecAonsort• O(n2)• guaranteedthefirstielementsweresorted

§mergesort• O(nlog(n))

§O(nlog(n))isthefastestasortcanbe

6.0001LECTURE12 31

WHAT HAVE WE SEEN IN 6.0001?

6.0001LECTURE12 32

KEY TOPICS §representknowledgewithdatastructures

§iteraEonandrecursionascomputaAonalmetaphors

§abstracEonofproceduresanddatatypes

§organizeandmodularizesystemsusingobjectclassesandmethods

§differentclassesofalgorithms,searchingandsorAng

§complexityofalgorithms

6.0001LECTURE12 33

OVERVIEW OF COURSE §learncomputaAonalmodesofthinking

§begintomastertheartofcomputaAonalproblemsolving

§makecomputersdowhatyouwantthemtodo

6.0001LECTURE12 34

HopewehavestartedyoudownthepathtobeingabletothinkandactlikeacomputerscienAst

WHAT DO COMPUTER SCIENTISTS DO? §theythinkcomputaAonally◦ abstracAons,algorithms,automatedexecuAon

§justlikethethreer’s:reading,‘riting,and‘rithmeAc–computaAonalthinkingisbecomingafundamentalskillthat everywell-educatedpersonwillneed

35

I6.0001

AdaLovelaceAlanTuring

6.0001LECTURE12

Image in the PublicDomain, courtesy ofWikipedia Commons.

Image in the PublicDomain, courtesy ofWikipedia Commons.

THE THREE A’S OF COMPUTATIONAL THINKING

§abstracAon◦ choosingtherightabstracAons◦ operaAnginmulAplelayersofabstracAonsimultaneously

◦ definingtherelaAonshipsbetweentheabstracAonlayers

§automaAon◦ thinkintermsofmechanizingourabstracAons◦ mechanizaAonispossible–becausewehavepreciseandexacAngnotaAonsandmodels;andbecausethereissome“machine”thatcaninterpretournotaAons

§algorithms◦ languagefordescribingautomatedprocesses◦ alsoallowsabstracAonofdetails◦ languageforcommunicaAngideas&processes

36 6.0001LECTURE12

Person

MITPerson

Student

UG Grad

ASPECTS OF COMPUTATIONAL THINKING §howdifficultisthisproblemandhowbestcanIsolveit?◦ theoreAcalcomputersciencegivesprecisemeaningtotheseandrelatedquesAonsandtheiranswers

§thinkingrecursively◦ reformulaAngaseeminglydifficultproblemintoonewhichweknowhowtosolve

◦ reduc�tion,embedding,transformation,� simulaAon

37

O(logn);O(n);O(nlogn);O(n2);O(cn)

6.0001LECTURE12

Image Licensed CC-BY, Courtesy of Robson# on Flickr.

MIT OpenCourseWare https://ocw.mit.edu

6.0001 Introduction to Computer Science and Programming in Python Fall 2016

For information about citing these materials or our Terms of Use, visit: https://ocw.mit.edu/terms.