CS 2110 Final Review

CS 2110 Final ReviewTopicsWill be coveredImportant Programming ConceptsTypes, Recursion/Induction, Asymptotic ComplexityThe Toolbox (Data Structures)Arrays, Linked Lists, Trees (BSTs, Heaps), HashtablesPractical ThingsSearching, Sorting, Abstract Data TypesGraphsThreads and ConcurrencyTopicsSort of Important Dont have a panic attack over these topics!GUISJust do the problems on old prelims/finalsSoftware engineeringDont write horrible code on coding questionsBut they are both important in the real world!TopicsWill Not Be CoveredLectures 24 and aboveJava Virtual MachineDistributed Systems and Cloud ComputingBalancing Trees (e.g.: AVL trees)But do know what a balanced and unbalanced tree isRecurrencesNetwork FlowImportant Programming ConceptsPrimitive Typesboolean, int, double, etcTest equality with == and !=Compare with =Typingvoid f(int x) { x--;}

int x = 10;f(x);// x == 10Pass by Valuef10f9main10Reference typesActual object is stored elsewhereVariable contains a reference to the object == tests equality for the referenceequals() tests equality for the objectx == y implies that x.equals(y)x.equals(y) does not imply x == yHow do we compare objects of type T?Implement the Comparable interfaceTypingvoid f(ArrayList l) { l.add(2); l = new ArrayList();}

ArrayList l = new ArrayList();l.add(1);f(l);// l contains 1, 2Pass by Reference{}{}flmainl{1}{1,2}We know that type B can implement/extend AB is a subtype of A; A is a supertype of BThe real type of the object is its dynamic typeThis type is known only at run-timeObject can act like a supertype of its dynamic typeCannot act like a subtype of its dynamic typeVariables/arguments of type A accept any subtype of AA is a supertype of the static typeStatic type is a supertype of the dynamic type

TypingStatic type is compile-time, determined by codeDynamic type might be a subtype of the static typeCasting temporarily changes the static typeUpcasts are always safeAlways cast to a supertype of the dynamic typeDowncasts may not be safeCan downcast to a supertype of the dynamic typeCannot downcast to a subtype of the dynamic typeTypingIf B extends A, and B and A both have function foo()Which foo() gets called?Answer depends on the dynamic typeIf the dynamic type is B, Bs foo will() be calledEven if foo() is invoked inside a function of AException: static functionsStatic functions are not associated with any objectThus, they do not have any typeTypingInduction and RecursionRecursionBasic examplesFactorial : n! = n(n-1)!Combinations Pascals triangle

Recursive structureTree (tree t = root with right/left subtree)Depth first searchDont forget base case (in proof, in your code)

Induction and RecursionInductionCan do induction on previous recursive problemsAlgorithm correctness proof (DFS)Math equation proof

Induction and RecursionStep 1Base caseStep 2suppose n is the variable youre going to do induction on. Suppose the equation holds when n=k Strong induction: suppose it holds for all n xBinary search tree is not guaranteed to be balancedIf it is balanced, we can find a node in O(log n) timeBinary Search Trees

Binary Search Trees

Completely unbalanced, but still a BSTNot a Binary Search Tree

8 is in the left subtree of 5Not a Binary Search Tree

3 is the left child of 2Adding to a Binary Search TreeAdding 4 to the BSTStart at root (5)4 < 5Go left to 24 > 2Go right to 34 > 3Add 4 as right child of 3

Tree TraversalsConverts the tree into a listWorks on any binary treeDo not need a binary search treeTraverse node and its left and right subtreesSubtrees are traversed recursivelyPreorder: node, left subtree, right subtreeInorder: left subtree, node, right subtreeProduces a sorted list for binary search treesPostorder: left subtree, rightsubtree, nodeTree TraversalsInorder TraversalIn(5)In(2), 5, In(7)1, 2, In(3), 5, In(7)1, 2, 3, 4, 5, In(7)1, 2, 3, 4, 5, 6, 7, 9

Binary HeapsWeaker condition on each nodeA node is smaller than its immediate childrenDont care about entire subtreeDont care if right child is smaller than left childSmallest node guaranteed to be at the rootNo guarantees beyond thatGuarantee also holds for each subtreeHeaps grow top-to-bottom, left-to-rightShrink in opposite directionBinary Heaps

Binary HeapsAdding to a heapFind lowest unfilled level of treeFind leftmost empty spot, add node thereNew node could be smaller than its parentSwap it up if its smallerIf swapped, could still be smaller than its new parentKeep swapping up until the parent is smaller

Binary HeapsRemoving from a heapTake out element from the top of the heapFind rightmost element on lowest levelMake this the new rootNew root could be larger than one/both childrenSwap it with the smallest childKeep swapping downBinary HeapsCan represent a heap as an arrayRoot element is at index 0For an element at index iParent is at index (i 1)/2Children are at indices 2i + 1, 2i + 2n-element heap takes up first n spots in the arrayNew elements grow heap by 1Removing an element shrinks heap by 1MotivationSort n numbers between 0 and 2n 1Sorting integers within a certain rangeMore specific than comparable objects with unlimited rangeGeneral lower bound of O(n log n) may not applyCan be done in O(n) time with counting sortCreate an array of size 2nThe ith entry counts all the numbers equal to iFor each number, increment the correct entryCan also find a given number in O(1) timeHashtablesCannot do this with arbitrary data typesInteger alone can have over 4 billion possible valuesFor a hashtable, create an array of size mHash function maps each object to an array index between 0 and m 1 (in O(1) time)Hash function makes sorting impossibleQuality of hash function is based on how many elements map to same index in the hashtableNeed to expect O(1) collisionsHashtablesDealing with collisionsIn counting sort, one array entry contains only element of the same valueThe hash function can map different objects to the same index of the hashtableChainingEach entry of the hashtable is a linked listLinear ProbingIf h(x) is taken, try h(x) + 1, h(x) + 2, h(x) + 3, Quadratic probing: h(x) + 1, h(x) + 4, h(x) + 9, HashtablesTable SizeIf too large, we waste spaceIf too small, everything collides with each otherProbing falls apart if number of items (n) is almost the size of the hashtable (m)Typically have a load factor 0 < 1Resize table when n/m exceeds Resizing changes mHave to reinsert everything into new hashtableHashtablesTable doublingDouble the size every time we exceed our load factorWorst case is when we just doubled the hashtableConsider all prior times we doubled the tablen + n/2 + n/4 + n/8 + < 2nInsert n items in O(n) timeAverage O(1) time to insert one itemSome operations take O(n) timeThis also works for growing an ArrayListHashtablesJava, hashCode(), and equals()hashCode() assigns an object an integer valueJava maps this integer to a number between 0 and m 1If x.equals(y), x and y should have the same hashCode()Insert object with one hashCode()Wont find it if you look it up with a different hashCode()If you override equals(), you must also override hashCode()Different objects can have the same hashCode()If this happens too often, we have too many collisionsOnly equals() can determine if they are equalHashtablesPractical ThingsSearchingUnsorted listsElement could be anywhere, O(n) search timeSorted lists binary searchTry middle elementSearch left half if too large, right half if too smallEach step cuts search space in half, O(log n) stepsBinary search requires random accessSearching a sorted linked list is still O(n)SortingMany ways to sortSame high-level goal, same resultWhats the difference?Algorithm, data structures used dictates running timeAlso dictate space usagesEach algorithm has its own flavorOnce again, assume random access to the listSortingSwap operation: swap(x, i, j)temp = x[i]x[i] = x[j]x[j] = tempMany sorts are a fancy set of swap instructionsModifies the array in place, very space-efficientNot space efficient to copy a large arrayInsertion Sortfor (i = 0n-1)Take element at index i, swap it back one spot untilyou hit the beginning of the listprevious element is smaller than this oneEx.: 4, 7, 8, 6, 2, 9, 1, 5 swap 6 back twiceAfter k iterations, first k elements relatively sorted4 iterations: 4, 6, 7, 8, 2, 9, 1, 5O(n2) time, in-place sortO(n) for sorted lists, good for nearly sorted listsSelection Sortfor (i = 0n-1)Scan array from index i to the endSwap smallest element into index iEx.: 1, 3, 5, 8, 9, 6, 7 swap 6, 8After k iterations, first k elements absolutely sorted4 iterations: 1, 3, 5, 6, 9, 8, 7O(n2) time, in-place sortAlso O(n2) for sorted listsMerge SortCopy left half, right half into two smaller arraysRecursively run merge sort each halfBase case: 1 or 2 element arrayMerge two sorted halves back into original arrayEx. (1, 3, 6, 9), (2, 5, 7, 8) (1, 2, 3, 5, 6, 7, 8, 9)Running Time: O(n log n)Merge takes O(n) timeSplit the list in half about log(n) timesAlso uses O(n) extra space!Quick SortRandomly pick a pivot Partition into two (unequal) halvesLeft partition smaller than pivot, right partition largerRecursively run quick sort on both partitionsExpected Running Time: O(n log n)Partition takes O(n) timePivot could be smallest, largest element (n partitions)Expected to split list O(log n) timesIn-place sort: partition can be done in-placeHeap SortUse a max-heap (represented as an array)Can move items up/down the heap with swapsfor (i = 0n-1)Add element at index i to the heapFirst pass builds the heapsfor (i = n-10)Remove largest element, put it in spot iO(n log n), in-place sort

Abstract Data TypesListsStacksLIFOQueuesFIFOSetsDictionaries (Maps) Priority Queues Java APIE.g.: ArrayList is an ADT list backed by an arrayAbstract Data TypesPriority QueueImplemented as heappeek() - look at heap root : O(1)poll() - heap delete op : O(log n)add() - heap add op : O(log n)GraphsA graph has verticesA graph has edges between two verticesn number of vertices; m number of edgesDirected vs. undirected graphDirected edges can only be traversed one wayUndirected edges can be traversed both wayWeighted vs. unweighted graphEdges could have weights/costs assigned to themWhat is a graph?What makes a graph special?Cycles!!!What is a graph without a cycle?Undirected graphsTreesDirected graphsDirected acyclic graph (DAG)What is a graph?Topological sort is for directed graphsIndegree: number of edges entering a vertexOutdegree: number of edges leaving a vertexTopological sort algorithmDelete a vertex with an indegree of 0Delete its outgoing edges, tooRepeat until no vertices have an indegree of 0Topological SortTopological SortABDECABEDCWhat cant topological sort cannot delete?Cycles!!!Every node in a cycle has an indegree of 1Need to delete another node in the cycle firstA graph is DAG iff a topological sort deletes itiff - if and only ifTopological SortWorks on directed and undirected graphsComponents set of vertices connected to edgesComponent starts as a single vertexNo edges are selected yetTravel edges to add vertices to this componentFrom a vertex in the component to a new vertexGraph SearchingImplementation detailsWhy is choosing a random path risky?Cycles!!!Could traverse a cycle foreverNeed to keep track of vertices in your componentNo cycles if you do not visit a vertex twiceGraph search algorithms need:A set of vertices in the component (visited)A collection of vertices to visit (toVisit)Graph SearchingAdd the start vertex to toVisitPick a vertex from toVisitIf the vertex is in visited, do nothingCould add same vertex to toVisit twice before visiting itIf the vertex is not in visited:Add vertex (and the edge to it) to visitedFollow its edges to neighboring verticesAdd each neighboring vertices to toVisit if it is not in visitedRepeat until there are no more vertices to visit

Graph SearchingRunning time analysisCan check if vertex is in visited in O(1) timeUse an array or HashSetEach edge added to toVisit once in a directed graphAlgorithm has at most m iterationsRunning time determined by ADT used for toVisitStack, queue: O(1) to add, delete; O(m) totalPriority queue: O(log m) to add, delete; O(m log m) totalGraph SearchingDepth-first searchUses a stackGoes as deep as it can before taking a new pathBreadth-first searchUses a queue

Graph SearchingGraph Searching: DFSABDEC-AA-BB-CB-EE-DABEDCGraph Searching: BFSABDEC-AE-DA-BC-DB-CB-EABEDCMSTs apply to undirected graphsTake only some of the edges in the graphSpanning all vertices connected togetherTree no cycles connectedFor all spanning trees, m = n 1All unweighted spanning trees are MSTsNeed to find MST for a weighted graphMinimum Spanning TreeInitially, we have no edges selectedWe have n 1-vertex componentsFind edge between two componentsDont add edge between vertices in same componentCreates a cyclePick smallest edge between two componentsThis is a greedy strategy, but it somehow worksIf edge weights are distinct, only one possible MSTDoesnt matter which algorithm you chooseMinimum Spanning TreeKruskals algorithmProcess edges from least to greatestEach edge eitherConnects two vertices in two different components (take it)Connects two vertices in the same component (throw it out)O(m log m) running timeO(m log m) to sort the edgesNeed union-find to track components and merge themDoes not alter running timeMinimum Spanning TreesMinimum Spanning TreesABCDEGF572831012419Prims algorithmGraph search algorithmLike BFS, but it uses a priority queuePriority is the edge weightSize of heap is O(m); running time is O(m log m)Minimum Spanning TreesMinimum Spanning TreesABCDEGF572831012419-A0A-C2A-B5C-B4C-D3D-E8D-F10B-E7D-G12E-G9G-F1For directed and undirected graphsFind shortest path between two verticesStart vertex is the sourceEnd vertex is the sinkUse Dijkstras algorithmAssumes positive edge weightsGraph search algorithm using a priority queueWeight is now entire path lengthCost of path to node + cost of edge to next vertex

Shortest Path AlgorithmShortest Path AlgorithmABCDEGF5728310154190525121516-A0A-B5A-C2C-B6C-D5B-E12D-E13D-F15D-G20F-G16E-G21Threads and ConcurrencyMotivation for ThreadsSplitting the WorkMulti-core processors can multi-taskCant multi-task with a single thread!Operating system simulates multi-taskingObvious: Multi-tasking is fasterLess Obvious: Not all threads are activeWaiting for a movie to download from DC++Multi-task: Study for CS 2110 while you wait!Operating system runs another thread while one waitsThread BasicsCode Set of instructions, stored as dataA paper with instructions does not do anythingCode by itself does not run or executeThread Runs/executes codeMultiple people can follow the same instructionsMultiple threads can run the same code

Thread BasicsMaking the CodeWrite a class implementing the Runnable interfaceOr use an anonymous inner class on the flyCode to execute goes in the run() methodRunning a ThreadThread thread = new Thread(Runnable target);Multiple threads could be constructed with the same targetStart a thread: thread.start();Wait until a thread finishes: thread.join();ConcurrencyTwo or more active threadsHard to predict each threads speedNondeterministic behaviorTwo active threads, one data structureBoth perform writes, or one reads while another writesNo problems if both read dataUpdates are not atomic!E.g.: One heap operation (such as add) takes multiple stepsWait until update completes to read/write againData in a bad state until this happensConcurrencyAvoiding race conditions (write/write, read/write)Imagine a room with a keyMust acquire the key before entering the roomEnter room, do many things, leave room, drop off keyMutual exclusion, or mutexesOnly one thread can acquire a mutexOthers threads wait until mutex is releasedSynchronizationCould use Javas Semaphore class with a count of 1Primitive, many corner cases like exceptionsBetter: synchronized (x) { /* body*/ }Thread 1 acquires a lock on objectx is a reference type; lock acquired on its memory locationThread 2 must wait if it wants a lock on xAlso waits if it wants a lock on y if x == ySynchronized functions: synchronized (this)Only one synchronized function running per objectSynchronizationOnly synchronize when you have toGet key, enter room, chat with BFF on phone for 1 hourPeople waiting for key are very angry!Deadlock via bugsThread 1 never releases lock on A, Thread 2 waiting for ADeadlock via circular waitingThread 1 has lock on A, waiting for lock on BThread 2 has lock on B, waiting for lock on ASolution: always acquire locks in same order