analysis, plug ‘n’ chug, & induction cs16: introduction to data structures & algorithms...
TRANSCRIPT
![Page 1: ANALYSIS, PLUG ‘N’ CHUG, & INDUCTION CS16: Introduction to Data Structures & Algorithms Tuesday, February 3, 2015 1](https://reader036.vdocuments.net/reader036/viewer/2022062304/56649d9d5503460f94a86705/html5/thumbnails/1.jpg)
1
ANALYSIS, PLUG ‘N’ CHUG, & INDUCTION
CS16: Introduction to Data Structures & Algorithms
Tuesday, February 3, 2015
![Page 2: ANALYSIS, PLUG ‘N’ CHUG, & INDUCTION CS16: Introduction to Data Structures & Algorithms Tuesday, February 3, 2015 1](https://reader036.vdocuments.net/reader036/viewer/2022062304/56649d9d5503460f94a86705/html5/thumbnails/2.jpg)
2
Announcements 2/3/15
• Sections have started• Note that rooms this week may be different than
last week!
• If you didn’t receive your graded HW1 and a grade report via email, let us know
• Homework 2 due Thursday 11:59pm• Seamcarve due Monday 11:59pm• Thursday is Python Lab part 2
• Please go to the room you went to last week
Tuesday, February 3, 2015
![Page 3: ANALYSIS, PLUG ‘N’ CHUG, & INDUCTION CS16: Introduction to Data Structures & Algorithms Tuesday, February 3, 2015 1](https://reader036.vdocuments.net/reader036/viewer/2022062304/56649d9d5503460f94a86705/html5/thumbnails/3.jpg)
3
Outline
1) Recurrence Review
2) Recurrence Relations
3) Plug ‘n’ Chug
4) Induction
5) Strong vs. Weak Induction
Tuesday, February 3, 2015
![Page 4: ANALYSIS, PLUG ‘N’ CHUG, & INDUCTION CS16: Introduction to Data Structures & Algorithms Tuesday, February 3, 2015 1](https://reader036.vdocuments.net/reader036/viewer/2022062304/56649d9d5503460f94a86705/html5/thumbnails/4.jpg)
4
Recursion Review
• Recursion is way of decomposing problems into smaller, simpler sub-tasks that are similar to the original.
• Thus, each sub-task can be solved by applying a similar technique.
• The whole problem is solved by combining the solutions to the smaller problems.
• Requires a BASE CASE (A case simple enough to solve without recursion) to end recursion.
Tuesday, February 3, 2015
![Page 5: ANALYSIS, PLUG ‘N’ CHUG, & INDUCTION CS16: Introduction to Data Structures & Algorithms Tuesday, February 3, 2015 1](https://reader036.vdocuments.net/reader036/viewer/2022062304/56649d9d5503460f94a86705/html5/thumbnails/5.jpg)
5
Recursion Example
def factorial(n): if n == 1: return 1 else: return n * factorial(n-1)
• Compute the factorial of a number, n.
Tuesday, February 3, 2015
![Page 6: ANALYSIS, PLUG ‘N’ CHUG, & INDUCTION CS16: Introduction to Data Structures & Algorithms Tuesday, February 3, 2015 1](https://reader036.vdocuments.net/reader036/viewer/2022062304/56649d9d5503460f94a86705/html5/thumbnails/6.jpg)
6
Recursion Simulation
• Calculate 3 factorial• This is a call to factorial(3), sowe put factorial(3) on the call stack
Tuesday, February 3, 2015
def factorial(n): if n == 1: return 1 else: return n*factorial(n-1)
Call Stack
![Page 7: ANALYSIS, PLUG ‘N’ CHUG, & INDUCTION CS16: Introduction to Data Structures & Algorithms Tuesday, February 3, 2015 1](https://reader036.vdocuments.net/reader036/viewer/2022062304/56649d9d5503460f94a86705/html5/thumbnails/7.jpg)
7
Recursion Simulation
• Calculate 3 factorial• This is a call to factorial(3), sowe put factorial(3) on the call stack
Tuesday, February 3, 2015
def factorial(n): if n == 1: return 1 else: return n*factorial(n-1)
Call Stack
factorial(3)
![Page 8: ANALYSIS, PLUG ‘N’ CHUG, & INDUCTION CS16: Introduction to Data Structures & Algorithms Tuesday, February 3, 2015 1](https://reader036.vdocuments.net/reader036/viewer/2022062304/56649d9d5503460f94a86705/html5/thumbnails/8.jpg)
8
Recursion Simulation
• n != 1, so we returnn*factorial(n-1), which includes a call to factorial(2).
• Remember, the call to factorial(3)has not returned yet, so it is stillon the call stack!
Tuesday, February 3, 2015
def factorial(n): if n == 1: return 1 else: return n*factorial(n-1)
Call Stack
factorial(3)
![Page 9: ANALYSIS, PLUG ‘N’ CHUG, & INDUCTION CS16: Introduction to Data Structures & Algorithms Tuesday, February 3, 2015 1](https://reader036.vdocuments.net/reader036/viewer/2022062304/56649d9d5503460f94a86705/html5/thumbnails/9.jpg)
9
Recursion Simulation
• n != 1, so we returnn*factorial(n-1), which includes a call to factorial(2).
• Remember, the call to factorial(3)has not returned yet, so it is stillon the call stack!def factorial(n): if n == 1: return 1 else: return n*factorial(n-1)
Call Stack
factorial(3)
factorial(2)
Tuesday, February 3, 2015
![Page 10: ANALYSIS, PLUG ‘N’ CHUG, & INDUCTION CS16: Introduction to Data Structures & Algorithms Tuesday, February 3, 2015 1](https://reader036.vdocuments.net/reader036/viewer/2022062304/56649d9d5503460f94a86705/html5/thumbnails/10.jpg)
10
Recursion Simulation
• n still != 1, so we returnn*factorial(n-1), which includes a call to factorial(1).
• Neither factorial(2) nor factorial(3)has returned at this point!
def factorial(n): if n == 1: return 1 else: return n*factorial(n-1)
Call Stack
factorial(3)
factorial(2)
Tuesday, February 3, 2015
![Page 11: ANALYSIS, PLUG ‘N’ CHUG, & INDUCTION CS16: Introduction to Data Structures & Algorithms Tuesday, February 3, 2015 1](https://reader036.vdocuments.net/reader036/viewer/2022062304/56649d9d5503460f94a86705/html5/thumbnails/11.jpg)
11
Recursion Simulation
• n still != 1, so we returnn*factorial(n-1), which includes a call to factorial(1).
• Neither factorial(2) nor factorial(3) has returned at this point!
def factorial(n): if n == 1: return 1 else: return n*factorial(n-1)
Call Stack
factorial(3)
factorial(2)
factorial(1)
Tuesday, February 3, 2015
![Page 12: ANALYSIS, PLUG ‘N’ CHUG, & INDUCTION CS16: Introduction to Data Structures & Algorithms Tuesday, February 3, 2015 1](https://reader036.vdocuments.net/reader036/viewer/2022062304/56649d9d5503460f94a86705/html5/thumbnails/12.jpg)
12
Recursion Simulation
• Now n =1, so we return 1!• This is not a recursive call, so factorial(1) returns, and we takeit off of the call stack.
def factorial(n): if n == 1: return 1 else: return n*factorial(n-1)
Call Stack
factorial(3)
factorial(2)
factorial(1)
Tuesday, February 3, 2015
![Page 13: ANALYSIS, PLUG ‘N’ CHUG, & INDUCTION CS16: Introduction to Data Structures & Algorithms Tuesday, February 3, 2015 1](https://reader036.vdocuments.net/reader036/viewer/2022062304/56649d9d5503460f94a86705/html5/thumbnails/13.jpg)
13
Recursion Simulation
• Now n =1, so we return 1!• This is not a recursive call, so factorial(1) returns, and we takeit off of the call stack.
def factorial(n): if n == 1: return 1 else: return n*factorial(n-1)
Call Stack
factorial(3)
factorial(2)
Tuesday, February 3, 2015
![Page 14: ANALYSIS, PLUG ‘N’ CHUG, & INDUCTION CS16: Introduction to Data Structures & Algorithms Tuesday, February 3, 2015 1](https://reader036.vdocuments.net/reader036/viewer/2022062304/56649d9d5503460f94a86705/html5/thumbnails/14.jpg)
14
Recursion Simulation• Now factorial(2) is at the top of the
call stack, so we return to where wewere in factorial(2).
• So we return 2*factorial(1), whichwe now know is 2*1, so factorial(2)returns 2 and is removed from the call stack!def factorial(n): if n == 1: return 1 else: return n*factorial(n-1)
Call Stack
factorial(3)
factorial(2)
Tuesday, February 3, 2015
![Page 15: ANALYSIS, PLUG ‘N’ CHUG, & INDUCTION CS16: Introduction to Data Structures & Algorithms Tuesday, February 3, 2015 1](https://reader036.vdocuments.net/reader036/viewer/2022062304/56649d9d5503460f94a86705/html5/thumbnails/15.jpg)
15
Recursion Simulation• Now factorial(2) is at the top of the
call stack, so we return to where wewere in factorial(2).
• So we return 2*factorial(1), whichwe now know is 2*1, so factorial(2)returns 2 and is removed from the call stack!def factorial(n): if n == 1: return 1 else: return n*factorial(n-1)
Call Stack
factorial(3)
Tuesday, February 3, 2015
![Page 16: ANALYSIS, PLUG ‘N’ CHUG, & INDUCTION CS16: Introduction to Data Structures & Algorithms Tuesday, February 3, 2015 1](https://reader036.vdocuments.net/reader036/viewer/2022062304/56649d9d5503460f94a86705/html5/thumbnails/16.jpg)
16
Recursion Simulation• Now factorial(3) is at the top of the
call stack, so we’re back in factorial(3).• Return 3*factorial(2), which we now know
is 3*2. • Factorial(3) returns 6 and removes itself
from the call stack.
def factorial(n): if n == 1: return 1 else: return n*factorial(n-1)
Call Stack
factorial(3)
Tuesday, February 3, 2015
![Page 17: ANALYSIS, PLUG ‘N’ CHUG, & INDUCTION CS16: Introduction to Data Structures & Algorithms Tuesday, February 3, 2015 1](https://reader036.vdocuments.net/reader036/viewer/2022062304/56649d9d5503460f94a86705/html5/thumbnails/17.jpg)
17
Recursion Simulation• Now factorial(3) is at the top of the
call stack, so we’re back in factorial(3).• Return 3*factorial(2), which we now know
is 3*2. • Factorial(3) returns 6 and removes itself
from the call stack.
def factorial(n): if n == 1: return 1 else: return n*factorial(n-1)
Call Stack
Tuesday, February 3, 2015
![Page 18: ANALYSIS, PLUG ‘N’ CHUG, & INDUCTION CS16: Introduction to Data Structures & Algorithms Tuesday, February 3, 2015 1](https://reader036.vdocuments.net/reader036/viewer/2022062304/56649d9d5503460f94a86705/html5/thumbnails/18.jpg)
18
Recursion Simulation
• Now our call stack is empty,and we know that factorial(3) is 6!
def factorial(n): if n == 1: return 1 else: return n*factorial(n-1)
Call Stack
Tuesday, February 3, 2015
![Page 19: ANALYSIS, PLUG ‘N’ CHUG, & INDUCTION CS16: Introduction to Data Structures & Algorithms Tuesday, February 3, 2015 1](https://reader036.vdocuments.net/reader036/viewer/2022062304/56649d9d5503460f94a86705/html5/thumbnails/19.jpg)
19
Determining Running Times
• For many algorithms, one can investigate code to see how size impacts running time• For seamcarve algorithm, each pixel is only evaluated in the for loops a constant number of times, and therefore the algorithm is O(n)
• But for some algorithms, counting can be tricky!• For example, determining the running time of a recursive algorithm can be complex
Tuesday, February 3, 2015
![Page 20: ANALYSIS, PLUG ‘N’ CHUG, & INDUCTION CS16: Introduction to Data Structures & Algorithms Tuesday, February 3, 2015 1](https://reader036.vdocuments.net/reader036/viewer/2022062304/56649d9d5503460f94a86705/html5/thumbnails/20.jpg)
20
Recurrence Relations
• Recurrence relations describe the runtime of a recursive algorithm in two parts:• base case
• how many instructions are executed in the base case of the recursive function, usually when n=0 or n=1
• general case • how many instructions are executed in the recursive
case
Tuesday, February 3, 2015
![Page 21: ANALYSIS, PLUG ‘N’ CHUG, & INDUCTION CS16: Introduction to Data Structures & Algorithms Tuesday, February 3, 2015 1](https://reader036.vdocuments.net/reader036/viewer/2022062304/56649d9d5503460f94a86705/html5/thumbnails/21.jpg)
21
Recursive array_max# Returns the maximum value of the first n elements in the array# Example: array_max([5,1,9,2], 4) 9
def array_max(array, n): if n == 1: return array[0] else: return max(array[n-1], array_max(array, n-1))
• T(n), the number of instructions executed as a function of the input size, can be expressed as a recurrence relation• T(1) = c0
• constant number of operations to compare and return
• T(n) = c1 + T(n-1)• constant number to do compare and calculate max• plus the operations of the recursive call
• But how do we get a big-O out of this?
Tuesday, February 3, 2015
![Page 22: ANALYSIS, PLUG ‘N’ CHUG, & INDUCTION CS16: Introduction to Data Structures & Algorithms Tuesday, February 3, 2015 1](https://reader036.vdocuments.net/reader036/viewer/2022062304/56649d9d5503460f94a86705/html5/thumbnails/22.jpg)
22
Recursive array_maxdef array_max(array, n): if n == 1: return array[0] else: return max(array[n-1], array_max(array, n-1))
array_max([5,1,9,2], 4)= max(2, array_max([5,1,9], 3))= max(2, max(9, array_max([5,1], 2)))= max(2, max(9, max(1, array_max([5], 1))))= max(2, max(9, max(1, 5)))= max(2, max(9, 5))= max(2, 9)= 9
Tuesday, February 3, 2015
Note: We only show the portion of the list we are working with, because we decrease n by 1 each time. Actually shrinking the list by 1 each time would require creating a copy, which is linear, and thus not optimal.
![Page 23: ANALYSIS, PLUG ‘N’ CHUG, & INDUCTION CS16: Introduction to Data Structures & Algorithms Tuesday, February 3, 2015 1](https://reader036.vdocuments.net/reader036/viewer/2022062304/56649d9d5503460f94a86705/html5/thumbnails/23.jpg)
23
Plug ‘n’ Chug• Given the recurrence relation (the base case T(1) and the general case T(n)),
we can “plug ‘n’ chug” to find a recurrence solution• A recurrence solution is the “answer” to a recurrence relation: it turns the
recursive definition into a simpler, closed-form mathematical expression
• Simplifying, we get: T(n) = c1n – c1 + c0• We can see that T(n) is a linear function, which makes array_max O(n)
T(1) = c0T(2) = c1 + T(2-1) = c1 + T(1) = c1 + c0T(3) = c1 + T(3-1) = c1 + T(2) = c1 + c1 +c0 = 2c1 + c0T(4) = c1 + T(4-1) = c1 + T(3) = c1 + 2c1 + c0 = 3c1 + c0 ⋮T(n) = c1 + T(n-1) = (n-1)c1 + c0
Tuesday, February 3, 2015
![Page 24: ANALYSIS, PLUG ‘N’ CHUG, & INDUCTION CS16: Introduction to Data Structures & Algorithms Tuesday, February 3, 2015 1](https://reader036.vdocuments.net/reader036/viewer/2022062304/56649d9d5503460f94a86705/html5/thumbnails/24.jpg)
24
How can we be sure?
• We just used the plug ‘n’ chug method to posit that the runtime of array_max was O(n)• But are we sure? Can we prove it?
• We observed a pattern in order to reach a recurrence solution, but a “pattern” isn’t a formal proof
• In order to prove that the algorithm has the runtime we think it does, we’ll need to prove that our recurrence solution is correct!
Tuesday, February 3, 2015
![Page 25: ANALYSIS, PLUG ‘N’ CHUG, & INDUCTION CS16: Introduction to Data Structures & Algorithms Tuesday, February 3, 2015 1](https://reader036.vdocuments.net/reader036/viewer/2022062304/56649d9d5503460f94a86705/html5/thumbnails/25.jpg)
25
Induction• Induction is a method of mathematical proof used to establish that a
statement is true for all positive integers• first demonstrate the statement’s truth for a single positive integer• second prove that, given the assumption that the statement is true
for an arbitrary input, the statement is true for the next input
• In other words:• If want to prove something for all n• Prove for:
• n = 1• n = k + 1 if true for n = k
• Celebrate!
• See the handout on website for another example.
Tuesday, February 3, 2015
![Page 26: ANALYSIS, PLUG ‘N’ CHUG, & INDUCTION CS16: Introduction to Data Structures & Algorithms Tuesday, February 3, 2015 1](https://reader036.vdocuments.net/reader036/viewer/2022062304/56649d9d5503460f94a86705/html5/thumbnails/26.jpg)
26
Induction ExampleClaim: The solution for T(1) = c0, T(n) = c1 + T(n-1) is T(n) = (n-1)c1 + c0Base Case• Prove the base case by plugging in n=1 to the recurrence solution:
• T(1) = (1-1)c1 + c0 = c0• We’re given that T(1) = c0 in the base case of the recurrence relation, so recurrence
solution works for the base case!
Inductive Assumption• Assume the solution works for k
• T(k) = (k-1)c1 + c0Inductive Step• Show that it works for k+1 given the assumption that it works for k -- ultimately, we want to
show that T(k+1) = (k)c1 + c0• T(k+1) = c1 + T(k) according to recurrence relation• T(k+1) = c1 + (k-1)c1 + c0 substituting inductive assumption• T(k+1) = (k)c1 + c0 simplifying
Conclusion• Because we’ve proven our claim for the base case n = 1 and shown
truth for n = k implies truth for n = k+1, therefore T(n) = (n-1)c1 + c0 for all positive integers n.
Tuesday, February 3, 2015
![Page 27: ANALYSIS, PLUG ‘N’ CHUG, & INDUCTION CS16: Introduction to Data Structures & Algorithms Tuesday, February 3, 2015 1](https://reader036.vdocuments.net/reader036/viewer/2022062304/56649d9d5503460f94a86705/html5/thumbnails/27.jpg)
27
Induction Example 2• Given:
• A(n) is the number of subsets of a set of size n• A(1) = 2
• empty set and set of one element
• A(n) = A(n-1) + A(n-1) = 2A(n-1)• all the sets in n-1 case with and without nth element included
• By plug ‘n’ chug: 2, 4, 8, 16, 32, …, A(n)• Looks like the recurrence solution is A(n) = 2n• Can we prove it?
• Statement to prove:• A(n) = 2n is the recurrence solution for A(n) = 2A(n-1)
Tuesday, February 3, 2015
![Page 28: ANALYSIS, PLUG ‘N’ CHUG, & INDUCTION CS16: Introduction to Data Structures & Algorithms Tuesday, February 3, 2015 1](https://reader036.vdocuments.net/reader036/viewer/2022062304/56649d9d5503460f94a86705/html5/thumbnails/28.jpg)
28
Induction Example 2• Given:
• A(n) is the number of subsets of a set of size n• A(1) = 2 (empty set and set of one element)• A(n) = 2A(n-1)
Claim: A(n) = 2n is the recurrence solution for A(n) = 2A(n-1)• Proof:
• base case A(1) = 21 = 2 given• inductive assumption A(k) = 2k assume for n = k• inductive step A(k+1) = 2A(k) givenA(k+1) = 2 * 2k substituting
assumption A(k+1) = 2k+1 simplifying
• conclusion: Because we’ve proved our claim for n = 1 and shown that n = k implies n = k+1, therefore A(n) = 2n for all positive integers
Tuesday, February 3, 2015
![Page 29: ANALYSIS, PLUG ‘N’ CHUG, & INDUCTION CS16: Introduction to Data Structures & Algorithms Tuesday, February 3, 2015 1](https://reader036.vdocuments.net/reader036/viewer/2022062304/56649d9d5503460f94a86705/html5/thumbnails/29.jpg)
29
Induction Example 3• Prove P(n) is true for all positive integers, n:
• Base case P(1):
• Assume P(k) is true:
Sometimes called the “predicate”
Tuesday, February 3, 2015
![Page 30: ANALYSIS, PLUG ‘N’ CHUG, & INDUCTION CS16: Introduction to Data Structures & Algorithms Tuesday, February 3, 2015 1](https://reader036.vdocuments.net/reader036/viewer/2022062304/56649d9d5503460f94a86705/html5/thumbnails/30.jpg)
30
Induction Example 3 (2)• Prove
“implies”
Plug in inductive assumption
Multiply by 2/2
Factor out (k+1)
Start with definition of Σ
AWESOME!
Tuesday, February 3, 2015
![Page 31: ANALYSIS, PLUG ‘N’ CHUG, & INDUCTION CS16: Introduction to Data Structures & Algorithms Tuesday, February 3, 2015 1](https://reader036.vdocuments.net/reader036/viewer/2022062304/56649d9d5503460f94a86705/html5/thumbnails/31.jpg)
31
Strong vs. Weak Induction
• This induction example is weak induction• Logic of the inductive step (n = k +1) relies only on
the previous step (n = k)
• This differs from strong induction• Logic of the inductive step relies on a “stronger”
assumption of more than one value (n ≤ k)• Sometimes makes inductive step easier
• The strong and weak refer to the assumptions you have to make to complete the proof, not the strength of the proof
• For much of CS16, weak induction is sufficient
Tuesday, February 3, 2015
![Page 32: ANALYSIS, PLUG ‘N’ CHUG, & INDUCTION CS16: Introduction to Data Structures & Algorithms Tuesday, February 3, 2015 1](https://reader036.vdocuments.net/reader036/viewer/2022062304/56649d9d5503460f94a86705/html5/thumbnails/32.jpg)
32
Readings
• Read the “Induction Handout” on the website! (in the Docs section)• http://cs.brown.edu/courses/cs016/docs/induction.pdf
Tuesday, February 3, 2015