![Page 1: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/1.jpg)
Lecture 2: Asymptotic NotationCSCI 700 - Algorithms I
Andrew Rosenberg
![Page 2: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/2.jpg)
Last Time
Review
Insertion Sort
Analysis of RuntimeProof of Correctness
![Page 3: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/3.jpg)
Today
Asymptotic Notation
Its use in analyzing runtimes.
Review of Proofs by Induction
An important tool for correctness proofs.
![Page 4: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/4.jpg)
Two Approaches
1 Emperical
Generate input.Time the executable.
2 Theoretical
Prove the runtime as a function of the size of the input.This is what we’ll be focusing on.The RAM model.
All information can be held in RAMAll operations take the same amount of resources each timethey’re called.
![Page 5: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/5.jpg)
Asymptotic Notation
Runtime on large inputs is more important than small input.
Sometimes, inefficient, but frequently called subroutines thatoperate on small input can have a dramatic impact on overallperformance. But this isnt the kind of efficiency well beanalyzing here.
It is usually safe to assume that live inputs are larger and lesswell-formed than test input used during development.
![Page 6: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/6.jpg)
Classes of Functions
We want to develop “classes of functions” to compare theruntime of an algorithm across machines.
Criteria
1 We only care about the behavior of the algorithm on inputwith size greater than some value n0.
2 To transfer our analyses across machines, we consider functionsthat differ by at most a constant multiplier to be equivalent.
![Page 7: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/7.jpg)
Definition
Asymptotic Notation is a formal notation for discussing andanalyzing “classes of functions”.
Criteria
1 Capture behavior when n0 ≤ n→∞.2 Functions that differ by at most a constant multiplier are
considered equivalent.
![Page 8: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/8.jpg)
Example
One one machine an algorithm may takeT (n) = 15n3 + n2 + 4
On another, T (n) = 5n3 + 4n + 5
Both will belong to the same class of functions. Namely,“cubic functions of n”.
Function classes can be thought of at “how many times we dosomething to each input”.
![Page 9: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/9.jpg)
O(n) definition
Definition
f (n) ∈ O(g(n)) if there exists constants c > 0 and n0 > 0 suchthat 0 ≤ f (n) ≤ c · g(n) for all n ≥ n0
“Big-O” notation
O(g(n)) is an upper-bound on the growth of a function, f (n).
![Page 10: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/10.jpg)
O(n) proofs
Prove that if T (n) = 15n3 + n2 + 4, T (n) = O(n3).
Proof.
Let c = 20 and n0 = 1.Must show that 0 ≤ f (n) and f (n) ≤ cg(n).0 ≤ 15n3 + n2 + 4 for all n ≥ n0 = 1.f (n) = 15n3 + n2 + 4 ≤ 15n3 + n3 + 4n3
15n3 + n3 + 4n3 = 20n3 = 20g(n) = cg(n)
![Page 11: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/11.jpg)
O(n) proofs
Prove that if T (n) = 15n3 + n2 + 4, T (n) = O(n4).
Proof.
Let c = 20 and n0 = 1.Must show that 0 ≤ f (n) and f (n) ≤ cg(n).0 ≤ 15n3 + n2 + 4 for all n ≥ n0 = 1.f (n) = 15n3 + n2 + 4 ≤ 15n4 + n4 + 4n4
15n4 + n4 + 4n4 = 20n4 = 20g(n) = cg(n)
![Page 12: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/12.jpg)
O(n) recap
T (n) = 15n3 + n2 + 4
T (n) = O(n3).
T (n) = O(n4).
O(n) is an upper bound
“=” is not really equality. It’s used as “set inclusion” ∈ here.
Don’t use O(n) = T (n)
![Page 13: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/13.jpg)
Ω(n) definition
Definition
f (n) ∈ Ω(g(n)) if there exists constants c > 0 and n0 > 0 suchthat 0 ≤ c · g(n) ≤ f (n) for all n ≥ n0
“Big-Omega of n”
Ω(g(n)) is lower-bound on the growth of a function, f (n).
![Page 14: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/14.jpg)
Ω(n) proofs
Prove that if T (n) = 15n3 + n2 + 4, T (n) = Ω(n3).
Proof.
Let c = 15 and n0 = 1.Must show that 0 ≤ cg(n) and cg(n) ≤ f (n).0 ≤ 15n3 for all n ≥ n0 = 1.cg(n) = 15n3 ≤ 15n3 + n2 + 4 = f (n)
![Page 15: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/15.jpg)
Ω(n) proofs
Prove that if T (n) = 15n3 + n2 + 4, T (n) = Ω(n2).
Proof.
Let c = 15 and n0 = 1.Must show that 0 ≤ cg(n) and cg(n) ≤ f (n).0 ≤ 15n3 for all n ≥ n0 = 1.cg(n) = 15n2 ≤ 15n3 ≤ 15n3 + n2 + 4 = f (n)
![Page 16: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/16.jpg)
Ω(n) recap
T (n) = 15n3 + n2 + 4
T (n) = Ω(n3).
T (n) = Ω(n2).
Ω(n) is a lower bound
![Page 17: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/17.jpg)
Θ(n) definition
Definition
f (n) ∈ Θ(g(n)) if there exists constants c1 > 0, c2 > 0 and n0 > 0such that c1 · g(n) ≤ f (n) ≤ c2 · g(n) for all n ≥ n0
“Theta of n”
Θ(g(n)) is tight bound on the growth of a function, f (n).
Can also say, f (n) = Θ(g(n)) iff f (n) = O(g(n)) andf (n) = Ω(g(n))
T (n) = 15n3 + n2 + 4
T (n) = Θ(n3).
![Page 18: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/18.jpg)
Θ(n) proof
Prove that if T (n) = 15n3 + n2 + 4, T (n) = Θ(n3).
Proof.
Let c1 = 15, c2 = 20 and n0 = 1.Must show that c1g(n) ≤ f (n) and f (n) ≤ c2g(n).c1g(n) = 15n3 ≤ 15n3 + n2 + 4 = f (n).f (n) = 15n3 + n2 + 4 ≤ 15n3 + n3 + 4n3 = 20n3 = c2g(n).
![Page 19: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/19.jpg)
Asymptotic Notation Examples
T (n) = 15n3 + n2 + 4 = Θ(n3)
T (n) = 2n3 + 4n + 4 = Θ(n3)
T (n) = 2n2 + 4n + 4 = Θ(n2)
T (n) = 15n3 + n2 + 4 = Θ(2n3 + n2)
While true, this is never done.
Note: f = Θ(g) is a common shorthand for f (n) = Θ(g(n))
![Page 20: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/20.jpg)
Asymptotic Notation Examples
T (n) = n3 + n log n = Θ(n3) ?
If n log n < n3 for all n > n0, then we can use the proofstructure from before.
Let c1 = 1.
c1g(n) = n3 ≤ n3 + n log n.
Let c2 = 2.
n3 + n log n ≤ n3 + n3 = 2n3 = c2g(n)
![Page 21: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/21.jpg)
Runtime at small n
T1(n) = n3 = Θ(n3)
T2(n) = 2n + 100 = Θ(n)
n T1(n) T2(n)
1 1 1022 8 1043 27 1064 64 1085 125 1106 216 112
![Page 22: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/22.jpg)
Asymptotic Notation Properties
f = Θ(h) and g = Θ(h) then f + g = Θ(h)
f = Θ(h) = c1h + slack .g = Θ(h) = c2h + slack .f + g = (c1 + c2)h + slack .
Θ(f + g) = Θ(max(f , g))
f = n3, g = n log n, h = f + g = n3 + n log nΘ(h) = Θ(f + g) = Θ(max(f , g)) = Θ(n3)
![Page 23: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/23.jpg)
Orders of growth
Behavior of a function as n→∞.
if limn→∞f (n)g(n) =∞ then f (n) = Ω(g(n))
“f is bigger than g”, or “f dominates g”
if limn→∞f (n)g(n) = 0 then f (n) = O(g(n))
“f is smaller than g”, or “g dominates f ”
if limn→∞f (n)g(n) = c then f (n) = Θ(g(n))
“f is the same as g”
![Page 24: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/24.jpg)
Orders of growth examples
Show that n2 = Ω(n)
Prove it to yourself first.
n n2 n2 − n
1 1 02 4 23 9 64 16 125 25 196 36 30
![Page 25: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/25.jpg)
Orders of growth examples
Show that n2 = Ω(n).
Evaluate limn→∞f (n)g(n)
limn→∞n2
n
limn→∞ n =∞ so n2 = Ω(n)
![Page 26: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/26.jpg)
Orders of growth examples
Show that n = Ω(log n).
Prove it to yourself first.
n log n n − log n
1 0 14 2 28 3 5
16 4 1232 5 2764 6 58
![Page 27: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/27.jpg)
Orders of growth example.
Show that n2 = Ω(n).
Evaluate limn→∞f (n)g(n)
limn→∞n
log n
l’Hopital’s rule. (from Calculus)
if limn→c f (n) =∞ and limn→cg(n) =∞, then
limn→cf (n)g(n) = limn→c
f ′(n)g ′(n)
limn→∞n
log n = limn→∞1
1/n
limn→∞n
log n = limn→∞ n =∞So, n = Ω(log n)
![Page 28: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/28.jpg)
Order of growth of a summation
S(n) =n∑
i=1
i =n(n + 1)
2
Even if we don’t know this, we can show that S(n) = Θ(n2)
Proof.
Show S(n) = O(n2).
S(n) =n∑
i=1
i ≤n∑
i=1
n = n · n = n2
S(n) ≤ n2
![Page 29: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/29.jpg)
Order of growth of a summation
Proof.
Show S(n) = Ω(n2).
S(n) =n∑
i=1
i ≥n∑
i=n/2+1
i ≥n∑
i=n/2+1
n
2
n∑i=n/2+1
n
2=
n
2· n
2=
n2
4
Thus, S(n) = Ω(n2), and S(n) = Θ(n2)
![Page 30: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/30.jpg)
Arithmetic series
S(n) =n∑
i=1
i
S(6) =6∑
i=1
i
= 1 + 2 + 3 + 4 + 5 + 6
= (1 + 6) + (2 + 5) + (3 + 4)
= 3 ∗ 7
![Page 31: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/31.jpg)
Arithmetic series
S(n) =n∑
i=1
i
S(n) =n∑
i=1
i
= 1 + 2 + . . . + n
= (1 + n) + (2 + (n − 1)) + ...(n
2+
n
2+ 1)
=n
2(n + 1)
![Page 32: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/32.jpg)
Inductive Proof
S(n) =n∑
i=1
i =n(n + 1)
2
Base Case: S(1)
S(1) =1∑
i=1
i
= 1n(n + 1)
2=
1(1 + 1)
2
=2
2= 1
![Page 33: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/33.jpg)
Inductive Step: Assume true for n ≤ n0 = 1. Prove for n+ 1.
S(n + 1) =n+1∑i=1
i
= (n + 1) +n∑
i=1
i
= (n + 1) +n(n + 1)
2
=2(n + 1)
2+
n(n + 1)
2
=2(n + 1) + n(n + 1)
2
=(n + 2)(n + 1)
2
=(n + 1)((n + 1) + 1)
2
![Page 34: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/34.jpg)
Induction over structures
The structure of an inductive proof can be applied tostructures.
Sometimes using the construct of Loop invariants.Initialization = Base CaseMaintenance = Inductive StepTermination
![Page 35: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/35.jpg)
Insertion Sort Pseudocode
InsertionSort(A)
for j ← 2 to size(A) dokey ← A[j ]i ← j − 1while i > 0 and A[i ] > key doSlide the array to the rightA[i + 1]← A[i ]i ← i + 1
end whileA[i + 1]← key
end for
![Page 36: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/36.jpg)
Insertion Sort
Start of the Loop
A[i]: 5 2 4 6 1 3
idx: 1 2 3 4 5 6
j ikey:2
![Page 37: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/37.jpg)
Insertion Sort
A[i]: 5 2 4 6 1 3
idx: 1 2 3 4 5 6
j ikey:2
![Page 38: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/38.jpg)
Insertion Sort
A[i]: 5 5 4 6 1 3
idx: 1 2 3 4 5 6
j ikey:2
![Page 39: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/39.jpg)
Insertion Sort
A[i]: 2 5 4 6 1 3
idx: 1 2 3 4 5 6
j ikey:2
![Page 40: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/40.jpg)
Insertion Sort
Start of the Loop
A[i]: 2 5 4 6 1 3
idx: 1 2 3 4 5 6
j ikey:4
![Page 41: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/41.jpg)
Insertion Sort
A[i]: 2 5 4 6 1 3
idx: 1 2 3 4 5 6
j ikey:4
![Page 42: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/42.jpg)
Insertion Sort
A[i]: 2 5 5 6 1 3
idx: 1 2 3 4 5 6
j ikey:4
![Page 43: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/43.jpg)
Insertion Sort
A[i]: 2 5 5 6 1 3
idx: 1 2 3 4 5 6
j ikey:4
![Page 44: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/44.jpg)
Insertion Sort
A[i]: 2 4 5 6 1 3
idx: 1 2 3 4 5 6
j ikey:4
![Page 45: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/45.jpg)
Insertion Sort
Start of the Loop
A[i]: 2 4 5 6 1 3
idx: 1 2 3 4 5 6
j ikey:6
![Page 46: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/46.jpg)
Insertion Sort
A[i]: 2 4 5 6 1 3
idx: 1 2 3 4 5 6
j ikey:6
![Page 47: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/47.jpg)
Insertion Sort
A[i]: 2 4 5 6 1 3
idx: 1 2 3 4 5 6
j ikey:6
![Page 48: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/48.jpg)
Insertion Sort
Start of the Loop
A[i]: 2 4 5 6 1 3
idx: 1 2 3 4 5 6
j ikey:1
![Page 49: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/49.jpg)
Insertion Sort
A[i]: 2 4 5 6 1 3
idx: 1 2 3 4 5 6
j ikey:1
![Page 50: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/50.jpg)
Insertion Sort
A[i]: 2 4 5 6 6 3
idx: 1 2 3 4 5 6
j ikey:1
![Page 51: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/51.jpg)
Insertion Sort
A[i]: 2 4 5 6 6 3
idx: 1 2 3 4 5 6
j ikey:1
![Page 52: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/52.jpg)
Insertion Sort
A[i]: 2 4 5 5 6 3
idx: 1 2 3 4 5 6
j ikey:1
![Page 53: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/53.jpg)
Insertion Sort
A[i]: 2 4 5 5 6 3
idx: 1 2 3 4 5 6
j ikey:1
![Page 54: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/54.jpg)
Insertion Sort
A[i]: 2 4 4 5 6 3
idx: 1 2 3 4 5 6
j ikey:1
![Page 55: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/55.jpg)
Insertion Sort
A[i]: 2 4 4 5 6 3
idx: 1 2 3 4 5 6
j ikey:1
![Page 56: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/56.jpg)
Insertion Sort
A[i]: 2 2 4 5 6 3
idx: 1 2 3 4 5 6
j ikey:1
![Page 57: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/57.jpg)
Insertion Sort
A[i]: 1 2 4 5 6 3
idx: 1 2 3 4 5 6
j ikey:1
![Page 58: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/58.jpg)
Insertion Sort
Start of the Loop
A[i]: 1 2 4 5 6 3
idx: 1 2 3 4 5 6
j ikey:3
![Page 59: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/59.jpg)
Insertion Sort
A[i]: 1 2 4 5 6 3
idx: 1 2 3 4 5 6
j ikey:3
![Page 60: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/60.jpg)
Insertion Sort
A[i]: 1 2 4 5 6 6
idx: 1 2 3 4 5 6
j ikey:3
![Page 61: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/61.jpg)
Insertion Sort
A[i]: 1 2 4 5 6 6
idx: 1 2 3 4 5 6
j ikey:3
![Page 62: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/62.jpg)
Insertion Sort
A[i]: 1 2 4 5 5 6
idx: 1 2 3 4 5 6
j ikey:3
![Page 63: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/63.jpg)
Insertion Sort
A[i]: 1 2 4 5 5 6
idx: 1 2 3 4 5 6
j ikey:3
![Page 64: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/64.jpg)
Insertion Sort
A[i]: 1 2 4 4 5 6
idx: 1 2 3 4 5 6
j ikey:3
![Page 65: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/65.jpg)
Insertion Sort
A[i]: 1 2 4 4 5 6
idx: 1 2 3 4 5 6
j ikey:3
![Page 66: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/66.jpg)
Insertion Sort
A[i]: 1 2 3 4 5 6
idx: 1 2 3 4 5 6
j ikey:3
![Page 67: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/67.jpg)
Insertion Sort
Termination
A[i]: 1 2 3 4 5 6
idx: 1 2 3 4 5 6
j ikey:3
![Page 68: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/68.jpg)
InsertionSort Loop Invariant
At the start of the for loop, the subarray A[1..j-1] is sorted.
“Sorted” here means contains the initial items of A[1..j-1] inascending order.
![Page 69: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/69.jpg)
InsertionSort Induction Proof
Induction Proof: At the start of the for loop, the subarrayA[1..j-1] is sorted.
Base Case: At initialization j = 2.
The subarray A[1..j-1] contains a single element, A[1].A single element is in sorted order.
![Page 70: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/70.jpg)
InsertionSort Induction Proof
Induction Proof: At the start of the for loop, the subarrayA[1..j-1] is sorted.
Induction Step: Assume true for j > n0. Prove for j + 1.
1 Show that A[j+1] is put in the correct position.2 Show that A[1..j+1] remains sorted.
![Page 71: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/71.jpg)
InsertionSort Induction Proof
Induction Proof: At the start of the for loop, the subarrayA[1..j-1] is sorted.
Induction Step: Assume true for j > n0. Prove for j + 1.1 Show that A[j+1] is put in the “correct;; position.
The index i is used to find the correct position to insertA[j + 1], specifically, the position i such that,A[i ] ≤ A[j + 1] < A[i + 2].At the end of the internal while loop, i points to element withthe greatest index smaller than A[j + 1], or 0 if no suchelement exists.A[j + 1] is moved to the position i + 1.Because A[1..j ] is sorted, every element A[1..i ] is smaller thanA[j ] and every element A[(i + 1)..j ] is larger than A[j + 1].Therefore A[j + 1] is put in the correct position.
![Page 72: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/72.jpg)
InsertionSort Induction Proof
Induction Proof: At the start of the for loop, the subarrayA[1..j-1] is sorted.
Induction Step: Assume true for j > n0. Prove for j + 1.2 Show that A[1..j+1] remains sorted.
At the end of the internal while loop, the elements A[i + 1..j ]have been each been shifted up one position to A[i + 1..j + 1].Therefore A[i+2..j+1] is sorted.A[1..i ] has not been modified. Since A[1..j] was sorted at thestart of the loop, the subarray A[1..i ] remains sorted.Since A[j+1] was put in the position, i+1 such thatA[i ] ≤ A[j + 1] < A[i + 2] the resulting arrayA[1..i ],A[i + 1],A[i + 2..j + 1] is sorted.
![Page 73: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/73.jpg)
InsertionSort Induction Proof
Induction Proof: At the start of the for loop, the subarrayA[1..j-1] is sorted.
Termination: At the termination of the for loop, j = n + 1,therefore A[1..n] is sorted.
![Page 74: Lecture 2: Asymptotic Notation CSCI 700 - Algorithms Ieniac.cs.qc.cuny.edu/andrew/csci700/lecture2.pdfAsymptotic Notation Runtime on large inputs is more important than small input](https://reader031.vdocuments.net/reader031/viewer/2022030500/5aabbe377f8b9a693f8c526a/html5/thumbnails/74.jpg)
Bye
Homework 2 is posted to the course website.
Next time (9/7)
Recursion
For Next Class
Read Sections 2.1, 2.2, 7.1, 7.2, 7.4