introduction to cs 4104courses.cs.vt.edu/~cs4104/murali/fall09/lectures/... · i prerequisites: cs...
TRANSCRIPT
![Page 1: Introduction to CS 4104courses.cs.vt.edu/~cs4104/murali/Fall09/lectures/... · I Prerequisites: CS 2604 or CS 2606, MATH 3134 or MATH 3034 T. M. Murali August 24, 2009 Introduction](https://reader031.vdocuments.net/reader031/viewer/2022040414/5f1a571c428d5c7cbf7bd3b1/html5/thumbnails/1.jpg)
About the Course Data and Algorithm Analysis Proof by Induction
Introduction to CS 4104
T. M. Murali
August 24, 2009
T. M. Murali August 24, 2009 Introduction to CS 4104
![Page 2: Introduction to CS 4104courses.cs.vt.edu/~cs4104/murali/Fall09/lectures/... · I Prerequisites: CS 2604 or CS 2606, MATH 3134 or MATH 3034 T. M. Murali August 24, 2009 Introduction](https://reader031.vdocuments.net/reader031/viewer/2022040414/5f1a571c428d5c7cbf7bd3b1/html5/thumbnails/2.jpg)
About the Course Data and Algorithm Analysis Proof by Induction
Course InformationI Instructor
I T. M. Murali, 2160B Torgerson, 231-8534, [email protected] O�ce Hours: 9:30am�11:30am Mondays and Wednesdays
I Teaching assistantI Chris Poirel, [email protected] O�ce Hours: to be decided
I Class meeting timeI MW 2:30pm�3:45pm, McBryde 134
I Keeping in TouchI Course web site
http://courses.cs.vt.edu/~cs4104/spring2009, updated
regularly through the semesterI Listserv: [email protected]
I Prerequisites: CS 2604 or CS 2606, MATH 3134 or MATH 3034
T. M. Murali August 24, 2009 Introduction to CS 4104
![Page 3: Introduction to CS 4104courses.cs.vt.edu/~cs4104/murali/Fall09/lectures/... · I Prerequisites: CS 2604 or CS 2606, MATH 3134 or MATH 3034 T. M. Murali August 24, 2009 Introduction](https://reader031.vdocuments.net/reader031/viewer/2022040414/5f1a571c428d5c7cbf7bd3b1/html5/thumbnails/3.jpg)
About the Course Data and Algorithm Analysis Proof by Induction
Course InformationI Instructor
I T. M. Murali, 2160B Torgerson, 231-8534, [email protected] O�ce Hours: 9:30am�11:30am Mondays and Wednesdays
I Teaching assistantI Chris Poirel, [email protected] O�ce Hours: to be decided
I Class meeting timeI MW 2:30pm�3:45pm, McBryde 134
I Keeping in TouchI Course web site
http://courses.cs.vt.edu/~cs4104/spring2009, updated
regularly through the semesterI Listserv: [email protected]
I Prerequisites: CS 2604 or CS 2606, MATH 3134 or MATH 3034
T. M. Murali August 24, 2009 Introduction to CS 4104
![Page 4: Introduction to CS 4104courses.cs.vt.edu/~cs4104/murali/Fall09/lectures/... · I Prerequisites: CS 2604 or CS 2606, MATH 3134 or MATH 3034 T. M. Murali August 24, 2009 Introduction](https://reader031.vdocuments.net/reader031/viewer/2022040414/5f1a571c428d5c7cbf7bd3b1/html5/thumbnails/4.jpg)
About the Course Data and Algorithm Analysis Proof by Induction
Course InformationI Instructor
I T. M. Murali, 2160B Torgerson, 231-8534, [email protected] O�ce Hours: 9:30am�11:30am Mondays and Wednesdays
I Teaching assistantI Chris Poirel, [email protected] O�ce Hours: to be decided
I Class meeting timeI MW 2:30pm�3:45pm, McBryde 134
I Keeping in TouchI Course web site
http://courses.cs.vt.edu/~cs4104/spring2009, updated
regularly through the semesterI Listserv: [email protected]
I Prerequisites: CS 2604 or CS 2606, MATH 3134 or MATH 3034
T. M. Murali August 24, 2009 Introduction to CS 4104
![Page 5: Introduction to CS 4104courses.cs.vt.edu/~cs4104/murali/Fall09/lectures/... · I Prerequisites: CS 2604 or CS 2606, MATH 3134 or MATH 3034 T. M. Murali August 24, 2009 Introduction](https://reader031.vdocuments.net/reader031/viewer/2022040414/5f1a571c428d5c7cbf7bd3b1/html5/thumbnails/5.jpg)
About the Course Data and Algorithm Analysis Proof by Induction
Course InformationI Instructor
I T. M. Murali, 2160B Torgerson, 231-8534, [email protected] O�ce Hours: 9:30am�11:30am Mondays and Wednesdays
I Teaching assistantI Chris Poirel, [email protected] O�ce Hours: to be decided
I Class meeting timeI MW 2:30pm�3:45pm, McBryde 134
I Keeping in TouchI Course web site
http://courses.cs.vt.edu/~cs4104/spring2009, updated
regularly through the semesterI Listserv: [email protected]
I Prerequisites: CS 2604 or CS 2606, MATH 3134 or MATH 3034T. M. Murali August 24, 2009 Introduction to CS 4104
![Page 6: Introduction to CS 4104courses.cs.vt.edu/~cs4104/murali/Fall09/lectures/... · I Prerequisites: CS 2604 or CS 2606, MATH 3134 or MATH 3034 T. M. Murali August 24, 2009 Introduction](https://reader031.vdocuments.net/reader031/viewer/2022040414/5f1a571c428d5c7cbf7bd3b1/html5/thumbnails/6.jpg)
About the Course Data and Algorithm Analysis Proof by Induction
Required Course Textbook
I Algorithm DesignI Jon Kleinberg and Éva TardosI Addison-WesleyI 2006I ISBN: 0-321-29535-8
T. M. Murali August 24, 2009 Introduction to CS 4104
![Page 7: Introduction to CS 4104courses.cs.vt.edu/~cs4104/murali/Fall09/lectures/... · I Prerequisites: CS 2604 or CS 2606, MATH 3134 or MATH 3034 T. M. Murali August 24, 2009 Introduction](https://reader031.vdocuments.net/reader031/viewer/2022040414/5f1a571c428d5c7cbf7bd3b1/html5/thumbnails/7.jpg)
About the Course Data and Algorithm Analysis Proof by Induction
Course Goals
I Learn methods and principles to construct algorithms.I Learn techniques to analyze algorithms mathematically for
correctness and e�ciency (e.g., running time and space used).I Course roughly follows the topics suggested in textbook
I Measures of algorithm complexityI GraphsI Greedy algorithmsI Divide and conquerI Dynamic programmingI Network �ow problemsI NP-completeness
T. M. Murali August 24, 2009 Introduction to CS 4104
![Page 8: Introduction to CS 4104courses.cs.vt.edu/~cs4104/murali/Fall09/lectures/... · I Prerequisites: CS 2604 or CS 2606, MATH 3134 or MATH 3034 T. M. Murali August 24, 2009 Introduction](https://reader031.vdocuments.net/reader031/viewer/2022040414/5f1a571c428d5c7cbf7bd3b1/html5/thumbnails/8.jpg)
About the Course Data and Algorithm Analysis Proof by Induction
Required Readings
I Reading assignment available on the website.I Read before class.
T. M. Murali August 24, 2009 Introduction to CS 4104
![Page 9: Introduction to CS 4104courses.cs.vt.edu/~cs4104/murali/Fall09/lectures/... · I Prerequisites: CS 2604 or CS 2606, MATH 3134 or MATH 3034 T. M. Murali August 24, 2009 Introduction](https://reader031.vdocuments.net/reader031/viewer/2022040414/5f1a571c428d5c7cbf7bd3b1/html5/thumbnails/9.jpg)
About the Course Data and Algorithm Analysis Proof by Induction
Lecture Slides
I Will be available on class web site.I Usually posted just before class.I Class attendance is extremely important.
Lecture in classcontains signi�cant and substantial additions to material on theslides.
T. M. Murali August 24, 2009 Introduction to CS 4104
![Page 10: Introduction to CS 4104courses.cs.vt.edu/~cs4104/murali/Fall09/lectures/... · I Prerequisites: CS 2604 or CS 2606, MATH 3134 or MATH 3034 T. M. Murali August 24, 2009 Introduction](https://reader031.vdocuments.net/reader031/viewer/2022040414/5f1a571c428d5c7cbf7bd3b1/html5/thumbnails/10.jpg)
About the Course Data and Algorithm Analysis Proof by Induction
Lecture Slides
I Will be available on class web site.I Usually posted just before class.I Class attendance is extremely important. Lecture in class
contains signi�cant and substantial additions to material on theslides.
T. M. Murali August 24, 2009 Introduction to CS 4104
![Page 11: Introduction to CS 4104courses.cs.vt.edu/~cs4104/murali/Fall09/lectures/... · I Prerequisites: CS 2604 or CS 2606, MATH 3134 or MATH 3034 T. M. Murali August 24, 2009 Introduction](https://reader031.vdocuments.net/reader031/viewer/2022040414/5f1a571c428d5c7cbf7bd3b1/html5/thumbnails/11.jpg)
About the Course Data and Algorithm Analysis Proof by Induction
Homeworks
I Posted on the web site ≈ one week before due date.I Announced on the class listserv.I Prepare solutions digitally but hand in hard-copy.
I Solution preparation recommended in LATEX.
T. M. Murali August 24, 2009 Introduction to CS 4104
![Page 12: Introduction to CS 4104courses.cs.vt.edu/~cs4104/murali/Fall09/lectures/... · I Prerequisites: CS 2604 or CS 2606, MATH 3134 or MATH 3034 T. M. Murali August 24, 2009 Introduction](https://reader031.vdocuments.net/reader031/viewer/2022040414/5f1a571c428d5c7cbf7bd3b1/html5/thumbnails/12.jpg)
About the Course Data and Algorithm Analysis Proof by Induction
Homeworks
I Posted on the web site ≈ one week before due date.I Announced on the class listserv.I Prepare solutions digitally but hand in hard-copy.
I Solution preparation recommended in LATEX.
T. M. Murali August 24, 2009 Introduction to CS 4104
![Page 13: Introduction to CS 4104courses.cs.vt.edu/~cs4104/murali/Fall09/lectures/... · I Prerequisites: CS 2604 or CS 2606, MATH 3134 or MATH 3034 T. M. Murali August 24, 2009 Introduction](https://reader031.vdocuments.net/reader031/viewer/2022040414/5f1a571c428d5c7cbf7bd3b1/html5/thumbnails/13.jpg)
About the Course Data and Algorithm Analysis Proof by Induction
Examinations
I Take-home midterm.I Take-home �nal (comprehensive).I Prepare digital solutions (recommend LATEX).
I Examinations may change to be in class.
T. M. Murali August 24, 2009 Introduction to CS 4104
![Page 14: Introduction to CS 4104courses.cs.vt.edu/~cs4104/murali/Fall09/lectures/... · I Prerequisites: CS 2604 or CS 2606, MATH 3134 or MATH 3034 T. M. Murali August 24, 2009 Introduction](https://reader031.vdocuments.net/reader031/viewer/2022040414/5f1a571c428d5c7cbf7bd3b1/html5/thumbnails/14.jpg)
About the Course Data and Algorithm Analysis Proof by Induction
Examinations
I Take-home midterm.I Take-home �nal (comprehensive).I Prepare digital solutions (recommend LATEX).I Examinations may change to be in class.
T. M. Murali August 24, 2009 Introduction to CS 4104
![Page 15: Introduction to CS 4104courses.cs.vt.edu/~cs4104/murali/Fall09/lectures/... · I Prerequisites: CS 2604 or CS 2606, MATH 3134 or MATH 3034 T. M. Murali August 24, 2009 Introduction](https://reader031.vdocuments.net/reader031/viewer/2022040414/5f1a571c428d5c7cbf7bd3b1/html5/thumbnails/15.jpg)
About the Course Data and Algorithm Analysis Proof by Induction
Grades
I Homeworks: ≈ 8, 60% of the grade.I Take-home midterm: 15% of the grade.I Take-home �nal: 25% of the grade.
T. M. Murali August 24, 2009 Introduction to CS 4104
![Page 16: Introduction to CS 4104courses.cs.vt.edu/~cs4104/murali/Fall09/lectures/... · I Prerequisites: CS 2604 or CS 2606, MATH 3134 or MATH 3034 T. M. Murali August 24, 2009 Introduction](https://reader031.vdocuments.net/reader031/viewer/2022040414/5f1a571c428d5c7cbf7bd3b1/html5/thumbnails/16.jpg)
About the Course Data and Algorithm Analysis Proof by Induction
What is an Algorithm?
Chamber's A set of prescribed computational procedures for solvinga problem; a step-by-step method for solving a problem.
Knuth, TAOCP An algorithm is a �nite, de�nite, e�ective procedure,with some input and some output.
T. M. Murali August 24, 2009 Introduction to CS 4104
![Page 17: Introduction to CS 4104courses.cs.vt.edu/~cs4104/murali/Fall09/lectures/... · I Prerequisites: CS 2604 or CS 2606, MATH 3134 or MATH 3034 T. M. Murali August 24, 2009 Introduction](https://reader031.vdocuments.net/reader031/viewer/2022040414/5f1a571c428d5c7cbf7bd3b1/html5/thumbnails/17.jpg)
About the Course Data and Algorithm Analysis Proof by Induction
What is an Algorithm?
Chamber's A set of prescribed computational procedures for solvinga problem; a step-by-step method for solving a problem.
Knuth, TAOCP An algorithm is a �nite, de�nite, e�ective procedure,with some input and some output.
T. M. Murali August 24, 2009 Introduction to CS 4104
![Page 18: Introduction to CS 4104courses.cs.vt.edu/~cs4104/murali/Fall09/lectures/... · I Prerequisites: CS 2604 or CS 2606, MATH 3134 or MATH 3034 T. M. Murali August 24, 2009 Introduction](https://reader031.vdocuments.net/reader031/viewer/2022040414/5f1a571c428d5c7cbf7bd3b1/html5/thumbnails/18.jpg)
About the Course Data and Algorithm Analysis Proof by Induction
Origin of the word �Algorithm�
1. From the Arabic al-Khwarizmi, a native of Khwarazm, a namefor the 9th century mathematician, Abu Ja'far Mohammed benMusa.
He wrote �Kitab al-jabr wa'l-muqabala,� which evolvedinto today's high school algebra text.
2. From Al Gore, the former U.S. vice-president who invented theinternet.
3. From the Greek algos (meaning �pain,� also a root of�analgesic�) and rythmos (meaning ��ow,� also a root of�rhythm�). �Pain �owed through my body whenever I worked onCS 4104 homeworks.� � former student.
T. M. Murali August 24, 2009 Introduction to CS 4104
![Page 19: Introduction to CS 4104courses.cs.vt.edu/~cs4104/murali/Fall09/lectures/... · I Prerequisites: CS 2604 or CS 2606, MATH 3134 or MATH 3034 T. M. Murali August 24, 2009 Introduction](https://reader031.vdocuments.net/reader031/viewer/2022040414/5f1a571c428d5c7cbf7bd3b1/html5/thumbnails/19.jpg)
About the Course Data and Algorithm Analysis Proof by Induction
Origin of the word �Algorithm�
1. From the Arabic al-Khwarizmi, a native of Khwarazm, a namefor the 9th century mathematician, Abu Ja'far Mohammed benMusa.
He wrote �Kitab al-jabr wa'l-muqabala,� which evolvedinto today's high school algebra text.
2. From Al Gore, the former U.S. vice-president who invented theinternet.
3. From the Greek algos (meaning �pain,� also a root of�analgesic�) and rythmos (meaning ��ow,� also a root of�rhythm�). �Pain �owed through my body whenever I worked onCS 4104 homeworks.� � former student.
T. M. Murali August 24, 2009 Introduction to CS 4104
![Page 20: Introduction to CS 4104courses.cs.vt.edu/~cs4104/murali/Fall09/lectures/... · I Prerequisites: CS 2604 or CS 2606, MATH 3134 or MATH 3034 T. M. Murali August 24, 2009 Introduction](https://reader031.vdocuments.net/reader031/viewer/2022040414/5f1a571c428d5c7cbf7bd3b1/html5/thumbnails/20.jpg)
About the Course Data and Algorithm Analysis Proof by Induction
Origin of the word �Algorithm�
1. From the Arabic al-Khwarizmi, a native of Khwarazm, a namefor the 9th century mathematician, Abu Ja'far Mohammed benMusa.
He wrote �Kitab al-jabr wa'l-muqabala,� which evolvedinto today's high school algebra text.
2. From Al Gore, the former U.S. vice-president who invented theinternet.
3. From the Greek algos (meaning �pain,� also a root of�analgesic�) and rythmos (meaning ��ow,� also a root of�rhythm�).
�Pain �owed through my body whenever I worked onCS 4104 homeworks.� � former student.
T. M. Murali August 24, 2009 Introduction to CS 4104
![Page 21: Introduction to CS 4104courses.cs.vt.edu/~cs4104/murali/Fall09/lectures/... · I Prerequisites: CS 2604 or CS 2606, MATH 3134 or MATH 3034 T. M. Murali August 24, 2009 Introduction](https://reader031.vdocuments.net/reader031/viewer/2022040414/5f1a571c428d5c7cbf7bd3b1/html5/thumbnails/21.jpg)
About the Course Data and Algorithm Analysis Proof by Induction
Origin of the word �Algorithm�
1. From the Arabic al-Khwarizmi, a native of Khwarazm, a namefor the 9th century mathematician, Abu Ja'far Mohammed benMusa.
He wrote �Kitab al-jabr wa'l-muqabala,� which evolvedinto today's high school algebra text.
2. From Al Gore, the former U.S. vice-president who invented theinternet.
3. From the Greek algos (meaning �pain,� also a root of�analgesic�) and rythmos (meaning ��ow,� also a root of�rhythm�). �Pain �owed through my body whenever I worked onCS 4104 homeworks.� � former student.
T. M. Murali August 24, 2009 Introduction to CS 4104
![Page 22: Introduction to CS 4104courses.cs.vt.edu/~cs4104/murali/Fall09/lectures/... · I Prerequisites: CS 2604 or CS 2606, MATH 3134 or MATH 3034 T. M. Murali August 24, 2009 Introduction](https://reader031.vdocuments.net/reader031/viewer/2022040414/5f1a571c428d5c7cbf7bd3b1/html5/thumbnails/22.jpg)
About the Course Data and Algorithm Analysis Proof by Induction
Origin of the word �Algorithm�
1. From the Arabic al-Khwarizmi, a native of Khwarazm, a namefor the 9th century mathematician, Abu Ja'far Mohammed benMusa. He wrote �Kitab al-jabr wa'l-muqabala,� which evolvedinto today's high school algebra text.
2. From Al Gore, the former U.S. vice-president who invented theinternet.
3. From the Greek algos (meaning �pain,� also a root of�analgesic�) and rythmos (meaning ��ow,� also a root of�rhythm�). �Pain �owed through my body whenever I worked onCS 4104 homeworks.� � former student.
T. M. Murali August 24, 2009 Introduction to CS 4104
![Page 23: Introduction to CS 4104courses.cs.vt.edu/~cs4104/murali/Fall09/lectures/... · I Prerequisites: CS 2604 or CS 2606, MATH 3134 or MATH 3034 T. M. Murali August 24, 2009 Introduction](https://reader031.vdocuments.net/reader031/viewer/2022040414/5f1a571c428d5c7cbf7bd3b1/html5/thumbnails/23.jpg)
About the Course Data and Algorithm Analysis Proof by Induction
Problem Example
Find Minimum
INSTANCE: Nonempty list x1, x2, . . . , xn of integers.
SOLUTION: Pair (i , xi) such that xi = min{xj | 1 ≤ j ≤ n}.
T. M. Murali August 24, 2009 Introduction to CS 4104
![Page 24: Introduction to CS 4104courses.cs.vt.edu/~cs4104/murali/Fall09/lectures/... · I Prerequisites: CS 2604 or CS 2606, MATH 3134 or MATH 3034 T. M. Murali August 24, 2009 Introduction](https://reader031.vdocuments.net/reader031/viewer/2022040414/5f1a571c428d5c7cbf7bd3b1/html5/thumbnails/24.jpg)
About the Course Data and Algorithm Analysis Proof by Induction
Algorithm Example
Find-Minimum(x1, x2, . . . , xn)1 i ← 1
2 for j ← 2 to n
3 do if xj < xi4 then i ← j
5 return (i , xi)
T. M. Murali August 24, 2009 Introduction to CS 4104
![Page 25: Introduction to CS 4104courses.cs.vt.edu/~cs4104/murali/Fall09/lectures/... · I Prerequisites: CS 2604 or CS 2606, MATH 3134 or MATH 3034 T. M. Murali August 24, 2009 Introduction](https://reader031.vdocuments.net/reader031/viewer/2022040414/5f1a571c428d5c7cbf7bd3b1/html5/thumbnails/25.jpg)
About the Course Data and Algorithm Analysis Proof by Induction
Running Time of Algorithm
Find-Minimum(x1, x2, . . . , xn)1 i ← 1
2 for j ← 2 to n
3 do if xj < xi4 then i ← j
5 return (i , xi)
I At most 2n − 1 assignments and n − 1 comparisons.
T. M. Murali August 24, 2009 Introduction to CS 4104
![Page 26: Introduction to CS 4104courses.cs.vt.edu/~cs4104/murali/Fall09/lectures/... · I Prerequisites: CS 2604 or CS 2606, MATH 3134 or MATH 3034 T. M. Murali August 24, 2009 Introduction](https://reader031.vdocuments.net/reader031/viewer/2022040414/5f1a571c428d5c7cbf7bd3b1/html5/thumbnails/26.jpg)
About the Course Data and Algorithm Analysis Proof by Induction
Running Time of Algorithm
Find-Minimum(x1, x2, . . . , xn)1 i ← 1
2 for j ← 2 to n
3 do if xj < xi4 then i ← j
5 return (i , xi)
I At most 2n − 1 assignments and n − 1 comparisons.
T. M. Murali August 24, 2009 Introduction to CS 4104
![Page 27: Introduction to CS 4104courses.cs.vt.edu/~cs4104/murali/Fall09/lectures/... · I Prerequisites: CS 2604 or CS 2606, MATH 3134 or MATH 3034 T. M. Murali August 24, 2009 Introduction](https://reader031.vdocuments.net/reader031/viewer/2022040414/5f1a571c428d5c7cbf7bd3b1/html5/thumbnails/27.jpg)
About the Course Data and Algorithm Analysis Proof by Induction
Correctness of Algorithm: Proof 1
Find-Minimum(x1, x2, . . . , xn)1 i ← 1
2 for j ← 2 to n
3 do if xj < xi
4 then i ← j
5 return (i , xi )
I Proof by contradiction: Suppose algorithm returns (k , xk) but thereexists 1 ≤ l ≤ n such that xl < xk and xl = min{xj | 1 ≤ j ≤ n}.
I Is k < l? No. Since the algorithm returns (k , xk), xk ≤ xj , for allk < j ≤ n. Therefore l < k .
I What does the algorithm do when j = l? It must set i to l , since we
have been told that xl is the smallest element.I What does the algorithm do when j = k (which happens after j = l)?
Since xl < xk , the value of i does not change.I Therefore, the algorithm does not return (k , xk) yielding a
contradiction.
T. M. Murali August 24, 2009 Introduction to CS 4104
![Page 28: Introduction to CS 4104courses.cs.vt.edu/~cs4104/murali/Fall09/lectures/... · I Prerequisites: CS 2604 or CS 2606, MATH 3134 or MATH 3034 T. M. Murali August 24, 2009 Introduction](https://reader031.vdocuments.net/reader031/viewer/2022040414/5f1a571c428d5c7cbf7bd3b1/html5/thumbnails/28.jpg)
About the Course Data and Algorithm Analysis Proof by Induction
Correctness of Algorithm: Proof 1
Find-Minimum(x1, x2, . . . , xn)1 i ← 1
2 for j ← 2 to n
3 do if xj < xi
4 then i ← j
5 return (i , xi )
I Proof by contradiction:
Suppose algorithm returns (k , xk) but thereexists 1 ≤ l ≤ n such that xl < xk and xl = min{xj | 1 ≤ j ≤ n}.
I Is k < l? No. Since the algorithm returns (k , xk), xk ≤ xj , for allk < j ≤ n. Therefore l < k .
I What does the algorithm do when j = l? It must set i to l , since we
have been told that xl is the smallest element.I What does the algorithm do when j = k (which happens after j = l)?
Since xl < xk , the value of i does not change.I Therefore, the algorithm does not return (k , xk) yielding a
contradiction.
T. M. Murali August 24, 2009 Introduction to CS 4104
![Page 29: Introduction to CS 4104courses.cs.vt.edu/~cs4104/murali/Fall09/lectures/... · I Prerequisites: CS 2604 or CS 2606, MATH 3134 or MATH 3034 T. M. Murali August 24, 2009 Introduction](https://reader031.vdocuments.net/reader031/viewer/2022040414/5f1a571c428d5c7cbf7bd3b1/html5/thumbnails/29.jpg)
About the Course Data and Algorithm Analysis Proof by Induction
Correctness of Algorithm: Proof 1
Find-Minimum(x1, x2, . . . , xn)1 i ← 1
2 for j ← 2 to n
3 do if xj < xi
4 then i ← j
5 return (i , xi )
I Proof by contradiction: Suppose algorithm returns (k , xk) but thereexists 1 ≤ l ≤ n such that xl < xk and xl = min{xj | 1 ≤ j ≤ n}.
I Is k < l? No. Since the algorithm returns (k , xk), xk ≤ xj , for allk < j ≤ n. Therefore l < k .
I What does the algorithm do when j = l? It must set i to l , since we
have been told that xl is the smallest element.I What does the algorithm do when j = k (which happens after j = l)?
Since xl < xk , the value of i does not change.I Therefore, the algorithm does not return (k , xk) yielding a
contradiction.
T. M. Murali August 24, 2009 Introduction to CS 4104
![Page 30: Introduction to CS 4104courses.cs.vt.edu/~cs4104/murali/Fall09/lectures/... · I Prerequisites: CS 2604 or CS 2606, MATH 3134 or MATH 3034 T. M. Murali August 24, 2009 Introduction](https://reader031.vdocuments.net/reader031/viewer/2022040414/5f1a571c428d5c7cbf7bd3b1/html5/thumbnails/30.jpg)
About the Course Data and Algorithm Analysis Proof by Induction
Correctness of Algorithm: Proof 1
Find-Minimum(x1, x2, . . . , xn)1 i ← 1
2 for j ← 2 to n
3 do if xj < xi
4 then i ← j
5 return (i , xi )
I Proof by contradiction: Suppose algorithm returns (k , xk) but thereexists 1 ≤ l ≤ n such that xl < xk and xl = min{xj | 1 ≤ j ≤ n}.
I Is k < l?
No. Since the algorithm returns (k , xk), xk ≤ xj , for allk < j ≤ n. Therefore l < k .
I What does the algorithm do when j = l? It must set i to l , since we
have been told that xl is the smallest element.I What does the algorithm do when j = k (which happens after j = l)?
Since xl < xk , the value of i does not change.I Therefore, the algorithm does not return (k , xk) yielding a
contradiction.
T. M. Murali August 24, 2009 Introduction to CS 4104
![Page 31: Introduction to CS 4104courses.cs.vt.edu/~cs4104/murali/Fall09/lectures/... · I Prerequisites: CS 2604 or CS 2606, MATH 3134 or MATH 3034 T. M. Murali August 24, 2009 Introduction](https://reader031.vdocuments.net/reader031/viewer/2022040414/5f1a571c428d5c7cbf7bd3b1/html5/thumbnails/31.jpg)
About the Course Data and Algorithm Analysis Proof by Induction
Correctness of Algorithm: Proof 1
Find-Minimum(x1, x2, . . . , xn)1 i ← 1
2 for j ← 2 to n
3 do if xj < xi
4 then i ← j
5 return (i , xi )
I Proof by contradiction: Suppose algorithm returns (k , xk) but thereexists 1 ≤ l ≤ n such that xl < xk and xl = min{xj | 1 ≤ j ≤ n}.
I Is k < l? No. Since the algorithm returns (k , xk), xk ≤ xj , for allk < j ≤ n. Therefore l < k .
I What does the algorithm do when j = l? It must set i to l , since we
have been told that xl is the smallest element.I What does the algorithm do when j = k (which happens after j = l)?
Since xl < xk , the value of i does not change.I Therefore, the algorithm does not return (k , xk) yielding a
contradiction.
T. M. Murali August 24, 2009 Introduction to CS 4104
![Page 32: Introduction to CS 4104courses.cs.vt.edu/~cs4104/murali/Fall09/lectures/... · I Prerequisites: CS 2604 or CS 2606, MATH 3134 or MATH 3034 T. M. Murali August 24, 2009 Introduction](https://reader031.vdocuments.net/reader031/viewer/2022040414/5f1a571c428d5c7cbf7bd3b1/html5/thumbnails/32.jpg)
About the Course Data and Algorithm Analysis Proof by Induction
Correctness of Algorithm: Proof 1
Find-Minimum(x1, x2, . . . , xn)1 i ← 1
2 for j ← 2 to n
3 do if xj < xi
4 then i ← j
5 return (i , xi )
I Proof by contradiction: Suppose algorithm returns (k , xk) but thereexists 1 ≤ l ≤ n such that xl < xk and xl = min{xj | 1 ≤ j ≤ n}.
I Is k < l? No. Since the algorithm returns (k , xk), xk ≤ xj , for allk < j ≤ n. Therefore l < k .
I What does the algorithm do when j = l? It must set i to l , since we
have been told that xl is the smallest element.I What does the algorithm do when j = k (which happens after j = l)?
Since xl < xk , the value of i does not change.I Therefore, the algorithm does not return (k , xk) yielding a
contradiction.T. M. Murali August 24, 2009 Introduction to CS 4104
![Page 33: Introduction to CS 4104courses.cs.vt.edu/~cs4104/murali/Fall09/lectures/... · I Prerequisites: CS 2604 or CS 2606, MATH 3134 or MATH 3034 T. M. Murali August 24, 2009 Introduction](https://reader031.vdocuments.net/reader031/viewer/2022040414/5f1a571c428d5c7cbf7bd3b1/html5/thumbnails/33.jpg)
About the Course Data and Algorithm Analysis Proof by Induction
Correctness of Algorithm: Proof 2
Find-Minimum(x1, x2, . . . , xn)1 i ← 1
2 for j ← 2 to n
3 do if xj < xi
4 then i ← j
5 return (i , xi )
I Proof by induction: What is true at the end of each iteration?
I Claim: xi = min{xm | 1 ≤ m ≤ j}, for all 1 ≤ j ≤ n.I Claim is true ⇒ algorithm is correct (set j = n).I Proof of the claim involves three steps.
1. Base case: j = 1 (before loop). xi = min{xm | 1 ≤ m ≤ 1} is triviallytrue.
2. Inductive hypothesis: Assume xi = min{xm | 1 ≤ m ≤ j}.3. Inductive step: Prove xi = min{xm | 1 ≤ m ≤ j + 1}.
I In the loop, i is set to be j + 1 if and only if xj+1 < xi .I Therefore, xi is the smallest of x1, x2, . . . , xj+1 after the loop
ends.
T. M. Murali August 24, 2009 Introduction to CS 4104
![Page 34: Introduction to CS 4104courses.cs.vt.edu/~cs4104/murali/Fall09/lectures/... · I Prerequisites: CS 2604 or CS 2606, MATH 3134 or MATH 3034 T. M. Murali August 24, 2009 Introduction](https://reader031.vdocuments.net/reader031/viewer/2022040414/5f1a571c428d5c7cbf7bd3b1/html5/thumbnails/34.jpg)
About the Course Data and Algorithm Analysis Proof by Induction
Correctness of Algorithm: Proof 2
Find-Minimum(x1, x2, . . . , xn)1 i ← 1
2 for j ← 2 to n
3 do if xj < xi
4 then i ← j
5 return (i , xi )
I Proof by induction: What is true at the end of each iteration?I Claim: xi = min{xm | 1 ≤ m ≤ j}, for all 1 ≤ j ≤ n.I Claim is true
⇒ algorithm is correct (set j = n).I Proof of the claim involves three steps.
1. Base case: j = 1 (before loop). xi = min{xm | 1 ≤ m ≤ 1} is triviallytrue.
2. Inductive hypothesis: Assume xi = min{xm | 1 ≤ m ≤ j}.3. Inductive step: Prove xi = min{xm | 1 ≤ m ≤ j + 1}.
I In the loop, i is set to be j + 1 if and only if xj+1 < xi .I Therefore, xi is the smallest of x1, x2, . . . , xj+1 after the loop
ends.
T. M. Murali August 24, 2009 Introduction to CS 4104
![Page 35: Introduction to CS 4104courses.cs.vt.edu/~cs4104/murali/Fall09/lectures/... · I Prerequisites: CS 2604 or CS 2606, MATH 3134 or MATH 3034 T. M. Murali August 24, 2009 Introduction](https://reader031.vdocuments.net/reader031/viewer/2022040414/5f1a571c428d5c7cbf7bd3b1/html5/thumbnails/35.jpg)
About the Course Data and Algorithm Analysis Proof by Induction
Correctness of Algorithm: Proof 2
Find-Minimum(x1, x2, . . . , xn)1 i ← 1
2 for j ← 2 to n
3 do if xj < xi
4 then i ← j
5 return (i , xi )
I Proof by induction: What is true at the end of each iteration?I Claim: xi = min{xm | 1 ≤ m ≤ j}, for all 1 ≤ j ≤ n.I Claim is true ⇒ algorithm is correct (set j = n).
I Proof of the claim involves three steps.
1. Base case: j = 1 (before loop). xi = min{xm | 1 ≤ m ≤ 1} is triviallytrue.
2. Inductive hypothesis: Assume xi = min{xm | 1 ≤ m ≤ j}.3. Inductive step: Prove xi = min{xm | 1 ≤ m ≤ j + 1}.
I In the loop, i is set to be j + 1 if and only if xj+1 < xi .I Therefore, xi is the smallest of x1, x2, . . . , xj+1 after the loop
ends.
T. M. Murali August 24, 2009 Introduction to CS 4104
![Page 36: Introduction to CS 4104courses.cs.vt.edu/~cs4104/murali/Fall09/lectures/... · I Prerequisites: CS 2604 or CS 2606, MATH 3134 or MATH 3034 T. M. Murali August 24, 2009 Introduction](https://reader031.vdocuments.net/reader031/viewer/2022040414/5f1a571c428d5c7cbf7bd3b1/html5/thumbnails/36.jpg)
About the Course Data and Algorithm Analysis Proof by Induction
Correctness of Algorithm: Proof 2
Find-Minimum(x1, x2, . . . , xn)1 i ← 1
2 for j ← 2 to n
3 do if xj < xi
4 then i ← j
5 return (i , xi )
I Proof by induction: What is true at the end of each iteration?I Claim: xi = min{xm | 1 ≤ m ≤ j}, for all 1 ≤ j ≤ n.I Claim is true ⇒ algorithm is correct (set j = n).I Proof of the claim involves three steps.
1. Base case: j = 1 (before loop). xi = min{xm | 1 ≤ m ≤ 1} is triviallytrue.
2. Inductive hypothesis: Assume xi = min{xm | 1 ≤ m ≤ j}.3. Inductive step: Prove xi = min{xm | 1 ≤ m ≤ j + 1}.
I In the loop, i is set to be j + 1 if and only if xj+1 < xi .I Therefore, xi is the smallest of x1, x2, . . . , xj+1 after the loop
ends.T. M. Murali August 24, 2009 Introduction to CS 4104
![Page 37: Introduction to CS 4104courses.cs.vt.edu/~cs4104/murali/Fall09/lectures/... · I Prerequisites: CS 2604 or CS 2606, MATH 3134 or MATH 3034 T. M. Murali August 24, 2009 Introduction](https://reader031.vdocuments.net/reader031/viewer/2022040414/5f1a571c428d5c7cbf7bd3b1/html5/thumbnails/37.jpg)
About the Course Data and Algorithm Analysis Proof by Induction
Format of Proof by Induction
I Goal: prove some proposition P(n) is true for all n.I Strategy: prove base case, assume inductive hypothesis, prove
inductive step.
I Base case: prove that P(1) or P(2) (or P(small number)) istrue.
I Inductive hypothesis: assume P(n − 1) is true.I Inductive step: prove that P(n − 1)⇒ P(n).I Why does this strategy work?
T. M. Murali August 24, 2009 Introduction to CS 4104
![Page 38: Introduction to CS 4104courses.cs.vt.edu/~cs4104/murali/Fall09/lectures/... · I Prerequisites: CS 2604 or CS 2606, MATH 3134 or MATH 3034 T. M. Murali August 24, 2009 Introduction](https://reader031.vdocuments.net/reader031/viewer/2022040414/5f1a571c428d5c7cbf7bd3b1/html5/thumbnails/38.jpg)
About the Course Data and Algorithm Analysis Proof by Induction
Format of Proof by Induction
I Goal: prove some proposition P(n) is true for all n.I Strategy: prove base case, assume inductive hypothesis, prove
inductive step.I Base case: prove that P(1) or P(2) (or P(small number)) is
true.I Inductive hypothesis: assume P(n − 1) is true.I Inductive step: prove that P(n − 1)⇒ P(n).
I Why does this strategy work?
T. M. Murali August 24, 2009 Introduction to CS 4104
![Page 39: Introduction to CS 4104courses.cs.vt.edu/~cs4104/murali/Fall09/lectures/... · I Prerequisites: CS 2604 or CS 2606, MATH 3134 or MATH 3034 T. M. Murali August 24, 2009 Introduction](https://reader031.vdocuments.net/reader031/viewer/2022040414/5f1a571c428d5c7cbf7bd3b1/html5/thumbnails/39.jpg)
About the Course Data and Algorithm Analysis Proof by Induction
Format of Proof by Induction
I Goal: prove some proposition P(n) is true for all n.I Strategy: prove base case, assume inductive hypothesis, prove
inductive step.I Base case: prove that P(1) or P(2) (or P(small number)) is
true.I Inductive hypothesis: assume P(n − 1) is true.I Inductive step: prove that P(n − 1)⇒ P(n).I Why does this strategy work?
T. M. Murali August 24, 2009 Introduction to CS 4104
![Page 40: Introduction to CS 4104courses.cs.vt.edu/~cs4104/murali/Fall09/lectures/... · I Prerequisites: CS 2604 or CS 2606, MATH 3134 or MATH 3034 T. M. Murali August 24, 2009 Introduction](https://reader031.vdocuments.net/reader031/viewer/2022040414/5f1a571c428d5c7cbf7bd3b1/html5/thumbnails/40.jpg)
About the Course Data and Algorithm Analysis Proof by Induction
Sum of �rst n natural numbers
P(n) =n∑
i=1
i =
n(n + 1)2
.
T. M. Murali August 24, 2009 Introduction to CS 4104
![Page 41: Introduction to CS 4104courses.cs.vt.edu/~cs4104/murali/Fall09/lectures/... · I Prerequisites: CS 2604 or CS 2606, MATH 3134 or MATH 3034 T. M. Murali August 24, 2009 Introduction](https://reader031.vdocuments.net/reader031/viewer/2022040414/5f1a571c428d5c7cbf7bd3b1/html5/thumbnails/41.jpg)
About the Course Data and Algorithm Analysis Proof by Induction
Sum of �rst n natural numbers
P(n) =n∑
i=1
i =n(n + 1)
2.
T. M. Murali August 24, 2009 Introduction to CS 4104
![Page 42: Introduction to CS 4104courses.cs.vt.edu/~cs4104/murali/Fall09/lectures/... · I Prerequisites: CS 2604 or CS 2606, MATH 3134 or MATH 3034 T. M. Murali August 24, 2009 Introduction](https://reader031.vdocuments.net/reader031/viewer/2022040414/5f1a571c428d5c7cbf7bd3b1/html5/thumbnails/42.jpg)
About the Course Data and Algorithm Analysis Proof by Induction
Recurrence Relation
Given
P(n) =
{P(bn
2c) + 1 if n > 1
1 if n = 1
prove that
P(n) ≤
1 + log2 n.
I Use strong induction: In the inductive hypothesis, assume thatP(i) is true for all i < n.
T. M. Murali August 24, 2009 Introduction to CS 4104
![Page 43: Introduction to CS 4104courses.cs.vt.edu/~cs4104/murali/Fall09/lectures/... · I Prerequisites: CS 2604 or CS 2606, MATH 3134 or MATH 3034 T. M. Murali August 24, 2009 Introduction](https://reader031.vdocuments.net/reader031/viewer/2022040414/5f1a571c428d5c7cbf7bd3b1/html5/thumbnails/43.jpg)
About the Course Data and Algorithm Analysis Proof by Induction
Recurrence Relation
Given
P(n) =
{P(bn
2c) + 1 if n > 1
1 if n = 1
prove that
P(n) ≤ 1 + log2 n.
I Use strong induction: In the inductive hypothesis, assume thatP(i) is true for all i < n.
T. M. Murali August 24, 2009 Introduction to CS 4104
![Page 44: Introduction to CS 4104courses.cs.vt.edu/~cs4104/murali/Fall09/lectures/... · I Prerequisites: CS 2604 or CS 2606, MATH 3134 or MATH 3034 T. M. Murali August 24, 2009 Introduction](https://reader031.vdocuments.net/reader031/viewer/2022040414/5f1a571c428d5c7cbf7bd3b1/html5/thumbnails/44.jpg)
About the Course Data and Algorithm Analysis Proof by Induction
Recurrence Relation
Given
P(n) =
{P(bn
2c) + 1 if n > 1
1 if n = 1
prove that
P(n) ≤ 1 + log2 n.
I Use strong induction: In the inductive hypothesis, assume thatP(i) is true for all i < n.
T. M. Murali August 24, 2009 Introduction to CS 4104