30.08.16

1

AdvancedAlgorithmics(6EAP)courses.cs.ut.eeMTAT.03.238Introduction

JaakVilo2016Fall

1JaakVilo

Programmingononeslide

• TRUE/FALSE,boolean operations• Data:3,7,2.47,.333e-02,‘a’,“textdata”, address…• Operations:+- */><=…• Variables:i,j,sum,A[5],B[7,8],a[i].surname,…

• if-then-else,for/while,…• function()

• Input/output• Standardlibraries:java.lang.Math.sqrt()

Goals

• Tolearnthemainconceptsandtechniquesofthealgorithmanddatastructuredesignandanalysis– thepracticalskillsandbasictheoreticalbasis

• Tobeabletochoose,(design,)analyzeandcomparealgorithmsanddatastructures

• Tolearntolearn,usewisely,solve,read,write,andpresent

Algorithms=basicsofCSeducation

• InyourfutureprofessionallifepeopleEXPECTyoutoknowelementarydatastructures,algorithms,andbeabletothinkinhighercategories.

• Weneedtodevelopthelanguagetotalkabouthigh-levelgoals,andyouneedtobeabletoimplementthem– ifsoneeded

• Atworkno-oneisgoingtoteachyoumathematicsorfundamentalsanymore

Algorithms

• Al-Khwārizmī

• AbūʿAbdallāhMuḥammadibnMūsāal-Khwārizmī[1](Arabic:) (أبو عبد الله محمد بن موسى الخوارزمي c.780,Khwārizm[2][3][4] – c.850)wasaPersian[5][2][6]mathematician,astronomer andgeographer,ascholarintheHouseofWisdom inBaghdad.

Example

• Windhasblownawaythe+,*,(,)signs• What’sthemaximalpossiblevalue?• Minimal?

2 1 7 1 4 3

30.08.16

2

• 217143

• (2+1)*7*(1+4)*3=21*15=315• 2*1+7+1*4+3=16

• Q:Howtomaximizethevalueofanyexpression?

2451981219872441123=?

Schoolmathematics:11*13 13x11

O( n2 )

is the same as:

Analysisofalgorithms

• Theoreticalstudyofcomputer-programperformanceandresourceusage.

• What elseis importantthanperformance?•Modularity•Correctness•Maintainability•Functionality•Robustness

•User-friendliness•Programmer time•Simplicity•Extensibility•Reliability

30.08.16

3

Whystudyalgorithmsandperformance?• Algorithmshelpustounderstandscalability.• Performanceoftendrawsthelinebetweenwhatisfeasibleandwhatisimpossible.

• Algorithmicmathematicsprovidesalanguagefortalkingaboutprogrambehavior.

• Performanceisthecurrency ofcomputing.• Thelessonsofprogramperformancegeneralize toothercomputingresources.

• Speedisfun!

SteveJürvetson onSkype:

• "Irememberwondering:howcantheybesogood?"hetoldme,speakingabouttheEstoniancoreofSkype."Howcansuchasmallgroupcandosomuchsoquickly,comparedtotypicaldevelopmenteffortsin,forexample,Microsoft?IhadtheimpressionthatmaybecomingoutofatimeofSovietoccupation,whencomputerswereunderpowered,youhadtoknowhowtoreallyprogram,effectively,parsimoniously,beingveryelegantinsculptingtheprogrammingcodetobetight,effective,andfast.[That's]notlikeinMicrosoft,whichhasaverylazyprogrammingenvironment,whereprogramsarecreatedthathavememoryleaksandallsortsofproblems,thatcrashallthetimeandnoonereallycares—becauseit'sMicrosoft!"

• http://arstechnica.com/business/2013/09/skypes-secrets/

Contents• AlgorithmsandDatastructures– basics• Abstractdatatypes (+STL,LEDA,…)• Lists,trees,heaps,graphs(TSP;shortestpaths,diameter, …)

• Analysisofalgorithmcomplexity– O( nlogn),o(nlogn),Θ(nlogn),Ω(nlogn)– recurrences T(n)=2T(n/2)+n– Turingmachines,NP,NP-complete,P-SPACE,…– Verytough (NP)problems (“Monkeyproblem”;unsolvableproblems?)

Contents(2)

• Basicdesigns: Divideandconquer,dynamicprogramming,randomised algorithms,…

• Deterministicornon-deterministic• Exactorapproximate• Heuristics(approximations):

– GreedyAlgorithms(packing,setcover,weightedsetcover,…)localsearch,tabusearch

– Simulated annealing(e.g.TSP),– Genetic algorithms(?),…– MonteCarlo,LasVegas

Contents(3)

• Sorting,search,dictionary,…• Textalgorithms• Numericalalgorithms• Computationalgeometry

• Onlinealgorithmsvsoffline• Parallelalgorithms

Skills/goals

• Bit-levelunderstandingoflow-levelimplementation

• Creativeuseofexistingimplementations• Formalanalysis,complexitytheory• Typicalheuristicsandalgorithmdesignelements

30.08.16

4

vs

• 67+56=? SUM(1079882846662946257362655369837393771079882846662946257362655369837393111771079882846662946257362655361983111173937710798828466629462573626553698373937710798182666211946257362655369837319377107988284666294625736265536173937710798828466629462573165536983739377)=?

Answer

• 13,000

• Ischecking fasterthanfinding thesolution?

SATisfiability

• Canaboolean formulawithvariablesinstantiatedtobecome“TRUE”– (AorB)and(notAorB)

• Checkif(some)answersatisfiestheformula• veryquickascomparedtofindingananswer

30.08.16

5

Subsetsum

• Doessome subsetofasetsumupto0?– Verifier/witness/certificate– Easytocheckifsomeonepointsout…– 2,-4,3,8,-1,9,-6

• IfverificationinPtime,thenprobleminNP

• Complementary:provethatnoneexists!– co-NP– Everysubsetisnon-0

Maindatamodels

…Linear

Tree

Graph

Relational data model

30.08.16

6

Allfittedonlinear RAM,disk,tape

…Linear

Tree

Graph

Relational

Linear storage: address of a “slot”

Programminglanguages:

• Pseudocode– primarilythiscoursewillusepseudocode

• Pointers,arrays,memoryhandling– Low- andhighlevelabstractions

• C/C++• Java,Python,perl,go,…

– Explicitdatastructuresandalgorithms

• Whicheverchoice:youmustbeabletoexplainit• Clarityandsimplicityisakey

“TheTextbook”• IntroductiontoAlgorithms,SecondEditionThomasH.Cormen,CharlesE.Leiserson,RonaldL.Rivest andCliffordStein

• http://mitpress.mit.edu/algorithms/• http://projects.csail.mit.edu/clrs/• http://books.google.com/books?id=NLngYyWFl_YC

• FoundationsofComputerScience:CEdition

• AlfredV.Aho,JeffreyD.Ullman

• W.H.Freeman(October15,1994)

• Algorithmics:TheSpiritofComputing(3rdEdition)

• DavidHarel,YishaiFeldman• AddisonWesley;3edition(June11,2004)

• http://books.google.com/books?id=txxLovFWkCUC• http://www.wisdom.weizmann.ac.il/~dharel/algorithmic

s.html

• TheArtofComputerProgramming(TAOCP)• DonaldE.Knuth.• http://www-cs-faculty.stanford.edu/~knuth/taocp.html

30.08.16

7

• TheAlgorithmDesignManual– StevenS.Skiena

• Algorithms[ILLUSTRATED]– SanjoyDasgupta,ChristosPapadimitriou,UmeshVazirani

Mathematics

• ConcreteMathematics:AFoundationforComputerScience(2ndEdition)(Hardcover)

• RonaldL.Graham,DonaldE.Knuth, OrenPatashnik

• Hardcover: 672pages• Addison-WesleyProfessional;• 2edition(March10,1994)• ISBN-10: 0201558025ISBN-13: 978-0201558029

• BundleofAlgorithmsinJava,ThirdEdition,Parts1-5:Fundamentals,DataStructures,Sorting,Searching,andGraphAlgorithms(3rdEdition)(Pts.1-5)

• RobertSedgewick

• SafariBookshelf!(UT)

Morebooks

• Parallelalgorithms– AnIntroductiontoParallelAlgorithms JosephJaja

• Heuristics– HowtoSolveit:ModernHeuristics. ByZbigniewMichalewicz,DavidB.Fogel– Edition:2,illustratedPublishedbySpringer,2004– http://www.springer.com/computer/foundations/book/978-3-540-22494-5

• Randomizedalgorithms• Complexitytheory

– Garey,M.R.;Johnson,D.S. (1979).ComputersandIntractability:AGuidetotheTheoryofNP-Completeness.NewYork:W.H.Freeman.ISBN0-7167-1045-5.Thisbookisaclassic,developingthetheory,thencataloguingmanyNP-Completeproblems.

Wikipedia:

• http://en.wikipedia.org/wiki/Algorithm• http://en.wikipedia.org/wiki/List_of_algorithm_general_topics• http://en.wikipedia.org/wiki/List_of_algorithms• http://en.wikipedia.org/wiki/List_of_data_structures• http://en.wikipedia.org/wiki/Computational_complexity_theory• http://en.wikipedia.org/wiki/List_of_NP-complete_problems• http://en.wikipedia.org/wiki/Randomized_algorithm• http://en.wikipedia.org/wiki/Parallel_algorithm• …