![Page 1: LECTURE 5: DYNAMIC PROGRAMMING ADVANCED ALGORITHMStheory.cs.utah.edu/fall18/algorithms/slides/lecture5.pdf · LECTURE 5 LAST CLASS 3 Divide and conquer Break instance into smaller](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce3ea3d0648972b440dfdc/html5/thumbnails/1.jpg)
ADVANCED ALGORITHMS LECTURE 5: DYNAMIC PROGRAMMING
�1
![Page 2: LECTURE 5: DYNAMIC PROGRAMMING ADVANCED ALGORITHMStheory.cs.utah.edu/fall18/algorithms/slides/lecture5.pdf · LECTURE 5 LAST CLASS 3 Divide and conquer Break instance into smaller](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce3ea3d0648972b440dfdc/html5/thumbnails/2.jpg)
LECTURE 5
ANNOUNCEMENTS
▸ Homework 1 due on Monday Sep 10, 11:59 PM
▸ Instructor office hours Thursday -> Friday (1 - 2 pm)
▸ Comment on notes — JeffE’s material
▸ Solutions to HW0 on course page
▸ Pecking order problem of HW1
�2
vertices birdsO O
could be cycles
O O
![Page 3: LECTURE 5: DYNAMIC PROGRAMMING ADVANCED ALGORITHMStheory.cs.utah.edu/fall18/algorithms/slides/lecture5.pdf · LECTURE 5 LAST CLASS 3 Divide and conquer Break instance into smaller](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce3ea3d0648972b440dfdc/html5/thumbnails/3.jpg)
LECTURE 5
LAST CLASS
�3
▸ Divide and conquer
▸ Break instance into smaller pieces, solve pieces separately, combine solutions (*)
▸ Merge-sort, multiplying n digit numbers (time << n2)
▸ Median (and k’th smallest element) in O(n) time (no sorting!)
▸ Key idea: approximate median to break array into “nearly equal” pieces
▸ All analyses by writing recursion, solving (tree, plug-and-chug, …)
n58
1Master theorem
Notes onhomepageAkira Bazzitheorem Jeff E's
notes
![Page 4: LECTURE 5: DYNAMIC PROGRAMMING ADVANCED ALGORITHMStheory.cs.utah.edu/fall18/algorithms/slides/lecture5.pdf · LECTURE 5 LAST CLASS 3 Divide and conquer Break instance into smaller](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce3ea3d0648972b440dfdc/html5/thumbnails/4.jpg)
LECTURE 5
OVERLAP IN RECURSIVE CALLS
�4
▸ Divide & conquer: usually sub-problems don’t interact “much”
▸ What if they do? Can run into wastefulness — solving same sub-problem many times…
▸ Motivation for dynamic programming
![Page 5: LECTURE 5: DYNAMIC PROGRAMMING ADVANCED ALGORITHMStheory.cs.utah.edu/fall18/algorithms/slides/lecture5.pdf · LECTURE 5 LAST CLASS 3 Divide and conquer Break instance into smaller](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce3ea3d0648972b440dfdc/html5/thumbnails/5.jpg)
LECTURE 5
TODAY’S PLAN
�5
▸ Continue example — subset sum
▸ Knapsack problem
▸ Traveling salesman
![Page 6: LECTURE 5: DYNAMIC PROGRAMMING ADVANCED ALGORITHMStheory.cs.utah.edu/fall18/algorithms/slides/lecture5.pdf · LECTURE 5 LAST CLASS 3 Divide and conquer Break instance into smaller](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce3ea3d0648972b440dfdc/html5/thumbnails/6.jpg)
LECTURE 5
RECAP: SUBSET SUM
�6
Problem: given a set of non-negative integers A[0], A[1], …, A[n-1], and an integer S, find if there is a subset of A[i] whose sum is S.
▸ Trivial algorithm: 2n time — check all subsets
▸ Sometimes better: divide into two halves — check all possibilities for how sum is divided
T(n) = S ⋅ T(n/2) ⟹ T(n) = Slog2 n = nlog2 S
Sn sL s I
Alo All f Aln i
It fl nitlogg2
5 30 i 5 7 4 15 38 221 29
![Page 7: LECTURE 5: DYNAMIC PROGRAMMING ADVANCED ALGORITHMStheory.cs.utah.edu/fall18/algorithms/slides/lecture5.pdf · LECTURE 5 LAST CLASS 3 Divide and conquer Break instance into smaller](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce3ea3d0648972b440dfdc/html5/thumbnails/7.jpg)
LECTURE 5
TRY 2
�7
▸ See if first element is picked or not
Problem: given a set of non-negative integers A[0], A[1], …, A[n-1], and an integer S, find if there is a subset of A[i] whose sum is S.
timeonwhenwehauf.in
Suppose we can write S Afi 1 Afia tafiacorrectnessis
Alo isNOT Alo isused immediate we
usedtowrite fw arepdqkijy.ua.lt
f ESee ifwe canwrite
seeif S Alo can bewritten as
5 using All Aln i using Afi An i
I
![Page 8: LECTURE 5: DYNAMIC PROGRAMMING ADVANCED ALGORITHMStheory.cs.utah.edu/fall18/algorithms/slides/lecture5.pdf · LECTURE 5 LAST CLASS 3 Divide and conquer Break instance into smaller](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce3ea3d0648972b440dfdc/html5/thumbnails/8.jpg)
LECTURE 5
RUNNING TIME
�8
c n runtime when we have
m Afit's
1 n 2T n l I Thi
Th F fintheomania In Fast
2 2 t t 2 t ln
nEE 2 2 almost
as bad as bruteforce
![Page 9: LECTURE 5: DYNAMIC PROGRAMMING ADVANCED ALGORITHMStheory.cs.utah.edu/fall18/algorithms/slides/lecture5.pdf · LECTURE 5 LAST CLASS 3 Divide and conquer Break instance into smaller](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce3ea3d0648972b440dfdc/html5/thumbnails/9.jpg)
LECTURE 5
LOOKING MORE CLOSELY — EXAMPLE
�9
A[] = {1, 2, 3, 5, 7, 9, 10, 11}; S = 20TEt is
3,5 20 E3 B b 19
3 5.3 s 20 3,5 18 3 3,51 17N L I I
sit 17 5,7 s 17
1start index _2Sum needed 17
![Page 10: LECTURE 5: DYNAMIC PROGRAMMING ADVANCED ALGORITHMStheory.cs.utah.edu/fall18/algorithms/slides/lecture5.pdf · LECTURE 5 LAST CLASS 3 Divide and conquer Break instance into smaller](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce3ea3d0648972b440dfdc/html5/thumbnails/10.jpg)
LECTURE 5
AVOIDING MULTIPLE SOLVES ON “SAME” INSTANCE
�10
▸ Store answers!
▸ How much are we storing?
▸ Running time
Howmany differentsub problems
can one have 7 cc 2
if I
any n possibilities possible
of distinct instances is E n sa
![Page 11: LECTURE 5: DYNAMIC PROGRAMMING ADVANCED ALGORITHMStheory.cs.utah.edu/fall18/algorithms/slides/lecture5.pdf · LECTURE 5 LAST CLASS 3 Divide and conquer Break instance into smaller](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce3ea3d0648972b440dfdc/html5/thumbnails/11.jpg)
Modi thm
In each recursive step firstcheck if we have solved thisinstance before If so read offthe answer
If not we store the answer aftercomputation
Answers to computation can be stored in ann Sti array
start index Sam needed
![Page 12: LECTURE 5: DYNAMIC PROGRAMMING ADVANCED ALGORITHMStheory.cs.utah.edu/fall18/algorithms/slides/lecture5.pdf · LECTURE 5 LAST CLASS 3 Divide and conquer Break instance into smaller](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce3ea3d0648972b440dfdc/html5/thumbnails/12.jpg)
LECTURE 5
IS THIS POLYNOMIAL?
�11
subtle point
when we say polytime algorithm we mean
polynomial in INPUT SIZE go
01 all an i S S 2
og att flog aft flog.amif ilgRunning time n.SE e udaiistn
Input size an Cfznz.sn andsaySERunning time n I fan n 2
![Page 13: LECTURE 5: DYNAMIC PROGRAMMING ADVANCED ALGORITHMStheory.cs.utah.edu/fall18/algorithms/slides/lecture5.pdf · LECTURE 5 LAST CLASS 3 Divide and conquer Break instance into smaller](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce3ea3d0648972b440dfdc/html5/thumbnails/13.jpg)
LECTURE 5
“KNAPSACK” / COIN CHANGE
�12
Problem: suppose we are given coins of denominations 1c, 5c, 10c, 20c, 25c, 50c. Make change for $1.90 using the fewest # of coins.
▸ Heuristics?
a
206 201C
50 50d 50k 254,104,5
GreedyAtways use the largest coin that youCAN use
E
![Page 14: LECTURE 5: DYNAMIC PROGRAMMING ADVANCED ALGORITHMStheory.cs.utah.edu/fall18/algorithms/slides/lecture5.pdf · LECTURE 5 LAST CLASS 3 Divide and conquer Break instance into smaller](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce3ea3d0648972b440dfdc/html5/thumbnails/14.jpg)
LECTURE 5
RECURSIVE ALGORITHM
�13
denominations do d dn
target sum S
Son'tused use do once The dotwice
1eachis a recursive call
s using d idnB and
7 do using it 3 tankandHans 2 S 2do 3
i2tans3n
![Page 15: LECTURE 5: DYNAMIC PROGRAMMING ADVANCED ALGORITHMStheory.cs.utah.edu/fall18/algorithms/slides/lecture5.pdf · LECTURE 5 LAST CLASS 3 Divide and conquer Break instance into smaller](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce3ea3d0648972b440dfdc/html5/thumbnails/15.jpg)
LECTURE 5
RUNNING TIME
�14
All recursive calls are of the type
S set of denominationsI1
Sum needed suffix of doD
Space needed to store answers 01ns
Time needed Ofn S Edo
![Page 16: LECTURE 5: DYNAMIC PROGRAMMING ADVANCED ALGORITHMStheory.cs.utah.edu/fall18/algorithms/slides/lecture5.pdf · LECTURE 5 LAST CLASS 3 Divide and conquer Break instance into smaller](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce3ea3d0648972b440dfdc/html5/thumbnails/16.jpg)
LECTURE 5
GENERAL IDEA OF DYNAMIC PROGRAMMING
�15
▸ Write down recursive algorithm
▸ Observe “common sub-problems” in recursion
▸ Capture sub-problems “succinctly”
![Page 17: LECTURE 5: DYNAMIC PROGRAMMING ADVANCED ALGORITHMStheory.cs.utah.edu/fall18/algorithms/slides/lecture5.pdf · LECTURE 5 LAST CLASS 3 Divide and conquer Break instance into smaller](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce3ea3d0648972b440dfdc/html5/thumbnails/17.jpg)
LECTURE 5
TRAVELING SALESMAN PROBLEM
�16
Problem: given an undirected graph with edge weights, find a path that visits every vertex precisely once and returns to start.
![Page 18: LECTURE 5: DYNAMIC PROGRAMMING ADVANCED ALGORITHMStheory.cs.utah.edu/fall18/algorithms/slides/lecture5.pdf · LECTURE 5 LAST CLASS 3 Divide and conquer Break instance into smaller](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce3ea3d0648972b440dfdc/html5/thumbnails/18.jpg)
LECTURE 5
TRAVELING SALESMAN PROBLEM
�17
Problem: given an undirected graph with edge weights, find a path that visits every vertex precisely once and returns to start.
▸ Trivial algorithm?
![Page 19: LECTURE 5: DYNAMIC PROGRAMMING ADVANCED ALGORITHMStheory.cs.utah.edu/fall18/algorithms/slides/lecture5.pdf · LECTURE 5 LAST CLASS 3 Divide and conquer Break instance into smaller](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce3ea3d0648972b440dfdc/html5/thumbnails/19.jpg)
LECTURE 5
TRAVELING SALESMAN PROBLEM
�18
Problem: given an undirected graph with edge weights, find a path that visits every vertex precisely once and returns to start.
▸ Trivial algorithm?
▸ Many heuristics for geometric cases
![Page 20: LECTURE 5: DYNAMIC PROGRAMMING ADVANCED ALGORITHMStheory.cs.utah.edu/fall18/algorithms/slides/lecture5.pdf · LECTURE 5 LAST CLASS 3 Divide and conquer Break instance into smaller](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce3ea3d0648972b440dfdc/html5/thumbnails/20.jpg)
LECTURE 5
RECURSIVE ALGORITHM
�19
![Page 21: LECTURE 5: DYNAMIC PROGRAMMING ADVANCED ALGORITHMStheory.cs.utah.edu/fall18/algorithms/slides/lecture5.pdf · LECTURE 5 LAST CLASS 3 Divide and conquer Break instance into smaller](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce3ea3d0648972b440dfdc/html5/thumbnails/21.jpg)
LECTURE 5
RECURSIVE ALGORITHM — CORRECTNESS
�20
![Page 22: LECTURE 5: DYNAMIC PROGRAMMING ADVANCED ALGORITHMStheory.cs.utah.edu/fall18/algorithms/slides/lecture5.pdf · LECTURE 5 LAST CLASS 3 Divide and conquer Break instance into smaller](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce3ea3d0648972b440dfdc/html5/thumbnails/22.jpg)
LECTURE 5
RECURSIVE ALGORITHM — CORRECTNESS
�21
![Page 23: LECTURE 5: DYNAMIC PROGRAMMING ADVANCED ALGORITHMStheory.cs.utah.edu/fall18/algorithms/slides/lecture5.pdf · LECTURE 5 LAST CLASS 3 Divide and conquer Break instance into smaller](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce3ea3d0648972b440dfdc/html5/thumbnails/23.jpg)
LECTURE 5
SUB-PROBLEMS, RUNNING TIME
�22
![Page 24: LECTURE 5: DYNAMIC PROGRAMMING ADVANCED ALGORITHMStheory.cs.utah.edu/fall18/algorithms/slides/lecture5.pdf · LECTURE 5 LAST CLASS 3 Divide and conquer Break instance into smaller](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce3ea3d0648972b440dfdc/html5/thumbnails/24.jpg)
LECTURE 5
SUB-PROBLEMS, RUNNING TIME
�23
![Page 25: LECTURE 5: DYNAMIC PROGRAMMING ADVANCED ALGORITHMStheory.cs.utah.edu/fall18/algorithms/slides/lecture5.pdf · LECTURE 5 LAST CLASS 3 Divide and conquer Break instance into smaller](https://reader035.vdocuments.net/reader035/viewer/2022071015/5fce3ea3d0648972b440dfdc/html5/thumbnails/25.jpg)
LECTURE 5
CAN WE DO BETTER?
�24
▸ Likely not — NP hard problem
▸ We will see that finding “approximately good” solution possible!