1 problems & algorithmic problem solving lecture 01 its033 – programming & algorithms...
TRANSCRIPT
1
Problems & Algorithmic Problem Solving
Lecture 01 ITS033 – Programming & Algorithms
Asst. Prof. Dr. Bunyarit UyyanonvaraAsst. Prof. Dr. Bunyarit UyyanonvaraIT Program, Image and Vision Computing Lab.
School of Information, Computer and Communication Technology (ICT)
Sirindhorn International Institute of Technology
Thammasat Universityhttp://www.siit.tu.ac.th/bunyarit
[email protected] 5013505 X 2005
2
This lecture’s overview ITS033 course structure
Intro. to Algorithm Why study Algorithm
Problems
Algorithmic Problem solving
Important problem types
3
ITS033 Course Structure & Grading
Lecture 01.1
Asst. Prof. Dr. Bunyarit UyyanonvaraAsst. Prof. Dr. Bunyarit UyyanonvaraIT Program, Image and Vision Computing Lab.
School of Information, Computer and Communication Technology (ICT)
Sirindhorn International Institute of Technology
Thammasat Universityhttp://www.siit.tu.ac.th/bunyarit
[email protected] 5013505 X 2005
ITS033 – Programming & Algorithms
4
Main Text BooksMain Text Books
• • Introduction to The Introduction to The Design & Analysis of Design & Analysis of AlgorithmsAlgorithms, 2nd Edition, , 2nd Edition, Anany Levitin, Addison Anany Levitin, Addison Wesley Publishing, 2006Wesley Publishing, 2006
• • Algorithms DesignAlgorithms Design , 2nd , 2nd Edition, Jon Kleinberg & Edition, Jon Kleinberg & Eva Tardos, Addison Wesley Eva Tardos, Addison Wesley Publishing, 2006Publishing, 2006
• • Algorithm DesignAlgorithm Design, MT , MT Goodrich, R Tamassia, John Goodrich, R Tamassia, John Wiley & Sons, Inc., 2002Wiley & Sons, Inc., 2002
http://www.siit.tu.ac.th/bunyarit/http://www.siit.tu.ac.th/bunyarit/its033.phpits033.php
((การแก�ปั�ญหาด้�วยคอมพิ�วเตอร�การแก�ปั�ญหาด้�วยคอมพิ�วเตอร�))http://www.vcharkarn.com/http://www.vcharkarn.com/vlesson/7vlesson/7
5
TopicsTopic 01Topic 01 -- Problems & Algorithmic Problem SolvingProblems & Algorithmic Problem SolvingTopic 02Topic 02 – Algorithm Representation & Efficiency Analysis – Algorithm Representation & Efficiency AnalysisTopic 03Topic 03 - State Space of a problem - State Space of a problemTopic 04Topic 04 - Brute Force - Brute ForceTopic 05Topic 05 - Divide and Conquer - Divide and ConquerTopic 06Topic 06 -- Decrease and ConquerDecrease and Conquer
Midterm ExamMidterm ExamTopic 07Topic 07 - Dynamics Programming - Dynamics ProgrammingTopic 08Topic 08 -- Transform and ConquerTransform and ConquerTopic 09Topic 09 - Graph Algorithms - Graph AlgorithmsTopic 10Topic 10 - Minimum Spanning Tree - Minimum Spanning TreeTopic 11Topic 11 - Shortest Path Problem - Shortest Path ProblemTopic 12Topic 12 - Coping with the Limitations of Algorithms Power - Coping with the Limitations of Algorithms Power
Final ExamFinal Exam
6
PrerequisiteThis course requiresThis course requires
ITS 050 – Introduction to Computer & Programming ITS 050 – Introduction to Computer & Programming
or or
Some programming skill Some programming skill
7
Grading
100% = 100% = MidtermMidterm 35% + 35% +
+ + FinalFinal 35% 35% + +
+ + Quizes & HomeworksQuizes & Homeworks 30% 30%
+ + BonusBonus (Attendance) 10% (Attendance) 10%
8
Grading Scheme
9
Introduction to Algorithms
Lecture 01.2ITS033 – Programming & Algorithms
Asst. Prof. Dr. Bunyarit UyyanonvaraAsst. Prof. Dr. Bunyarit UyyanonvaraIT Program, Image and Vision Computing Lab.
School of Information, Computer and Communication Technology (ICT)
Sirindhorn International Institute of Technology
Thammasat Universityhttp://www.siit.tu.ac.th/bunyarit
[email protected] 5013505 X 2005
10
Overview Algorithm Definition Why study Algorithm Problem Conceptual flow Problem types
11
Algorithmics
AlgorithmicsAlgorithmics is a study of algorithms is a study of algorithms
how to solve computational problemshow to solve computational problems how to solve it systematically and how to solve it systematically and how to solve it efficiently.how to solve it efficiently.
12
Algorithm
Definition of Algorithm Definition of Algorithm
• An algorithm is a An algorithm is a precise descriptionprecise description of a step-by-step process of a step-by-step process
• that is that is guaranteed to terminateguaranteed to terminate … …
• terminates after terminates after a finite number of stepsa finite number of steps with a ... with a ...
• with a with a correct answercorrect answer for… for…
• for for every particular instanceevery particular instance of an algorithmic problem that may of an algorithmic problem that may occur.occur.
AlgorithmInput Output
13
Algorithm 1: How to bake a Banana CakeAlgorithm 1: How to bake a Banana Cake
Ingredients Ingredients 2 1/2 cups all-purpose flour1 tablespoon baking soda1 pinch salt1/2 cup unsalted butter1 cup white sugar3/4 cup light brown sugar2 eggs4 ripe bananas, mashed2/3 cup buttermilk1/2 cup chopped walnuts
DirectionsDirections
11Preheat oven to 350 degrees F (175 degrees C). Grease and flour 2 - 8 Preheat oven to 350 degrees F (175 degrees C). Grease and flour 2 - 8 inch round pans. In a small bowl, whisk together flour, soda and salt; set inch round pans. In a small bowl, whisk together flour, soda and salt; set aside.aside.
22In a large bowl, cream butter, white sugar and brown sugar until light and In a large bowl, cream butter, white sugar and brown sugar until light and fluffy. Beat in eggs, one at a time. Mix in the bananas. Add flour mixture fluffy. Beat in eggs, one at a time. Mix in the bananas. Add flour mixture alternately with the buttermilk to the creamed mixture. Stir in chopped alternately with the buttermilk to the creamed mixture. Stir in chopped walnuts. Pour batter into the prepared pans.walnuts. Pour batter into the prepared pans.
33Bake in the preheated oven for 30 minutes. Remove from oven, and Bake in the preheated oven for 30 minutes. Remove from oven, and place on a damp tea towel to cool. place on a damp tea towel to cool.
InputInput
AlgorithmAlgorithm
14
What is Algorithm? Algorithm is Algorithm is Not an ANSWERNot an ANSWER to a problem to a problem
It is rather precisely defined It is rather precisely defined PROCEDURESPROCEDURES for for getting answers.getting answers.
Thus specific algorithm design techniques can be Thus specific algorithm design techniques can be interpreted as problem-solving strategiesinterpreted as problem-solving strategies
An attempt to formalize things as algorithms An attempt to formalize things as algorithms leads to a much deeper understandingleads to a much deeper understanding
15
Notion of Algorithm
Programming
16
Why Study Algorithm? To know standard set of important algorithmsTo know standard set of important algorithms
Able to use known algorithms to solve known Able to use known algorithms to solve known problemsproblems
Able to design new algorithms & analyze their Able to design new algorithms & analyze their efficienciesefficiencies
Useful in developing analytical skillUseful in developing analytical skill
17
Analyzing Algorithms Analyzing algorithmsAnalyzing algorithms involves involves
thinking about how their resource thinking about how their resource requirement will scale with increasing input requirement will scale with increasing input size.size.
ResourceResource means means time time and spaceand space
18
Problems & Problem Solving
Lecture 01.3
ITS033 – Programming & Algorithms
Asst. Prof. Dr. Bunyarit UyyanonvaraAsst. Prof. Dr. Bunyarit UyyanonvaraIT Program, Image and Vision Computing Lab.
School of Information, Computer and Communication Technology (ICT)
Sirindhorn International Institute of Technology
Thammasat Universityhttp://www.siit.tu.ac.th/bunyarit
[email protected] 5013505 X 2005
19
Problem 1: Old world puzzleA man finds himself on a riverbank with a wolf, a A man finds himself on a riverbank with a wolf, a
goat, and a head of cabbage. He needs to goat, and a head of cabbage. He needs to transport all three to the other side of the river transport all three to the other side of the river in his boat. However, the boat has room for only in his boat. However, the boat has room for only the man himself and the man himself and oneone other item. other item.
In his absence, the wolf would eat the goat, and the In his absence, the wolf would eat the goat, and the goat would eat the cabbage.goat would eat the cabbage.
Solve this problem, or prove it has no solution.Solve this problem, or prove it has no solution.
* The man is vegetarian who doesn’t like to eat cabbage.* The man is vegetarian who doesn’t like to eat cabbage.
20
Problem 2: New world puzzle Problem 02 - New World Puzzle: Problem 02 - New World Puzzle: There are four people who want to cross a bridge; they all There are four people who want to cross a bridge; they all
begin on the same side. begin on the same side. You have 17 minutesYou have 17 minutes to get them all to get them all across to the other side. It is night, and they have one across to the other side. It is night, and they have one flashlight. A maximum of two people can cross the bridge at flashlight. A maximum of two people can cross the bridge at one time. Any party that crosses, either one or two people, one time. Any party that crosses, either one or two people, must have the flashlight with them. The flashlight must be must have the flashlight with them. The flashlight must be walked back and forth; it cannot be thrown, for example. walked back and forth; it cannot be thrown, for example. Mr. Mr. AA takes 1 minute to cross the bridge, takes 1 minute to cross the bridge, Mr. BMr. B takes 2 minutes, takes 2 minutes, Mr. CMr. C takes 5 minutes, and takes 5 minutes, and Mr. DMr. D takes 10 minutes. A pair takes 10 minutes. A pair must walk together at the rate of the slower person’s pace. must walk together at the rate of the slower person’s pace. For example, if Mr.A and Mr.D walk across first, 10 minutes For example, if Mr.A and Mr.D walk across first, 10 minutes have elapsed when they get to the other side of the bridge. If have elapsed when they get to the other side of the bridge. If Mr.D returns the flashlight, a total of 20 minutes have passed Mr.D returns the flashlight, a total of 20 minutes have passed and you have failed the mission.and you have failed the mission.
Solve this problem, or prove it has no solution.Solve this problem, or prove it has no solution.
21
Problems Problem 03 – 8 Queen Puzzle: Problem 03 – 8 Queen Puzzle: The queen is the most powerful piece in the game of chess. The queen is the most powerful piece in the game of chess.
The queen can be moved in a straight line vertically, The queen can be moved in a straight line vertically, horizontally, or diagonally, any number of unoccupied horizontally, or diagonally, any number of unoccupied squares as shown on the diagram at the right. The queen squares as shown on the diagram at the right. The queen captures by occupying the square on which an enemy captures by occupying the square on which an enemy piece sits. piece sits. Put 8 queens on the same chess board that Put 8 queens on the same chess board that each of them can not capture each other.each of them can not capture each other.
22
Problems Fake Coin ProblemFake Coin Problem
There are 5,000 coins in a bucket that look identical. Just 1 There are 5,000 coins in a bucket that look identical. Just 1 fake coin that is lighter than the others. Luckily, you have fake coin that is lighter than the others. Luckily, you have a balance (which allows them to compare the weights of a balance (which allows them to compare the weights of two groups of coins) to help you with this.two groups of coins) to help you with this.
Design an algorithm to get rid of the fake coin. Design an algorithm to get rid of the fake coin. If you have to pay for each time you use the balance, If you have to pay for each time you use the balance,
design an algorithm to solve the same problem.design an algorithm to solve the same problem.
23
Problems Traveling Salesman problemTraveling Salesman problem
A salesman needs to visit each of the cities in this map once A salesman needs to visit each of the cities in this map once before he returns home. Design an algorithm to take him around before he returns home. Design an algorithm to take him around the map with minimum total ticket cost. (a number on an edge the map with minimum total ticket cost. (a number on an edge indicates the price of the ticket between the two cities)indicates the price of the ticket between the two cities)
24
Problems Optimization problem (Football manager)Optimization problem (Football manager)
25
Problem
26
Problems 8-Queen8-Queen Horse TraversalHorse Traversal Stable MatchingStable Matching Traveling Salesman ProblemTraveling Salesman Problem Knapsack ProblemKnapsack Problem Domino Puzzle (page 371)Domino Puzzle (page 371) Rubik cubesRubik cubes MazeMaze
27
Fundamentals of Algorithmic Problem Solving
Lecture 01.4 ITS033 – Programming & Algorithms
Asst. Prof. Dr. Bunyarit UyyanonvaraAsst. Prof. Dr. Bunyarit UyyanonvaraIT Program, Image and Vision Computing Lab.
School of Information, Computer and Communication Technology (ICT)
Sirindhorn International Institute of Technology
Thammasat Universityhttp://www.siit.tu.ac.th/bunyarit
[email protected] 5013505 X 2005
28
1.2 Fundamentals of Algorithmic Problem Solving
Algorithm = Procedural Solutions to Problem
NOT an answer, BUT rather specific instructions of getting answers.
Therefore, requires steps in designing and analyzing an algorithm
29
Algorithm Design & Analysis Process
30
Step 1: Understand the Problem
the first thing you need to do before designing an the first thing you need to do before designing an algorithm is to algorithm is to understand completely the problem understand completely the problem givengiven. Read the problem’s description carefully and ask . Read the problem’s description carefully and ask questions if you have any doubts about the problem, do a questions if you have any doubts about the problem, do a few small examples by hand, think about special cases, few small examples by hand, think about special cases, and ask questions again if needed.and ask questions again if needed.
An input to an algorithm specifies an An input to an algorithm specifies an instance instance of the of the problem the algorithm solves. It is very important to problem the algorithm solves. It is very important to specify exactly the range of instances the algorithm needs specify exactly the range of instances the algorithm needs to handle.to handle.
Remember that a correct algorithm is not one that works Remember that a correct algorithm is not one that works most of the time but one that works correctly for most of the time but one that works correctly for all all legitimate inputs.legitimate inputs.
31
Step 2: Ascertaining the capabilities of a computational device
Algorithms designed to be executed on machines that executes intstructions one after another are called sequential algorithms.
Algorithms that take advantage of computers that can execute operations concurrently are called parallel algorithms.
32
Step 3: Choosing between Exact & Approximate Problem Solving
Exact algorithms vs Approximation algorithms
Why approximation algorithms?1. Problems cannot be solved exactly2. Available exact algorithms are unacceptably
slow
33
Step 4: Deciding on Appropriate Data Structures In the new world of object-oriented
programming, data structures remain important for both design and analysis of algorithms.
However, we will assume a very basic data structure for now and concentrate on the algorithm side.
Data Structure is in next semester for IT & CS students
34
Step 5: Algorithm Design Techniques
An algorithm design technique (or “strategy” or “paradigm”) is a general approach to solving problems algorithmically that is applicable to a variety of problems from different areas of computing.
Eg. Brute force, Divide-and-Conquer, Transform-and-Conquer
35
Step 6: Methods of Specifying an Algorithm
Pseudocode, Pseudocode, a mixture of a natural language a mixture of a natural language and programming language-likeconstructs.and programming language-likeconstructs.
flowchartflowchart, a method of expressing an algorithm , a method of expressing an algorithm by a collection of connected geometric shapes by a collection of connected geometric shapes containing descriptions of the algorithm’s steps.containing descriptions of the algorithm’s steps.
36
Step 7: Proving an Algorithm’s Correctness
Prove algorithm’s Prove algorithm’s correctnesscorrectness = prove that the = prove that the algorithm yields a algorithm yields a requiredrequired resultresult for for every every legitimate inputlegitimate input in a in a finite amount of timefinite amount of time..
For an approximation algorithm, For an approximation algorithm, correctness correctness means to be able to show that the error produced means to be able to show that the error produced by the algorithm does not exceed a predefined by the algorithm does not exceed a predefined limit.limit.
37
Step 8: Analyzing an Algorithm
1. Efficiency Time efficiency indicates how fast the algorithm runs. space efficiency indicates how much extra memory the
algorithm needs.
2. Simplicity
3. Generality Design an algorithm for a problem posed in more general
terms. Design an algorithm that can handle a range of inputs that
is natural for the problem at hand.
This topic will be fully covered in the next lecture.
38
Step 9: Coding the algorithm
More than implementationMore than implementation
Peril of incorrect & inefficient implementationPeril of incorrect & inefficient implementation
Require testing & debuggingRequire testing & debugging
Require code optimizingRequire code optimizing
39
Important Problem Types
Lecture 01.5
ITS033 – Programming & Algorithms
Asst. Prof. Dr. Bunyarit UyyanonvaraAsst. Prof. Dr. Bunyarit UyyanonvaraIT Program, Image and Vision Computing Lab.
School of Information, Computer and Communication Technology (ICT)
Sirindhorn International Institute of Technology
Thammasat Universityhttp://www.siit.tu.ac.th/bunyarit
[email protected] 5013505 X 2005
40
1.3 Important Problem Types SortingSorting SearchingSearching String processingString processing Graph problemsGraph problems Combinatorial problemsCombinatorial problems Geometric problemsGeometric problems Numerical problemsNumerical problems
41
Sorting The The sorting problem sorting problem asks us to rearrange the asks us to rearrange the
items of a given list in ascending order.items of a given list in ascending order.
we usually need to sort lists of numbers, we usually need to sort lists of numbers, characters from an alphabet, character strings, characters from an alphabet, character strings, and, most important, records similar to those and, most important, records similar to those maintained by schools about their students, maintained by schools about their students, libraries about their holdings, and companies libraries about their holdings, and companies about their employees. In the case of records, about their employees. In the case of records, weneed to choose a piece of information to guide weneed to choose a piece of information to guide sorting.sorting.
42
Searching The The searching problem searching problem deals with finding a deals with finding a
given value, called a given value, called a search keysearch key, in a given set , in a given set (or a multiset, which permits several elements to (or a multiset, which permits several elements to have the same value).have the same value).
43
String Processing A A string string is a sequence of characters from an is a sequence of characters from an
alphabet.alphabet.
String of particular interest: String of particular interest: 1. Text string – comprises letters, numbers, and 1. Text string – comprises letters, numbers, and
special charactersspecial characters2. Bit string – comprises zeros and ones2. Bit string – comprises zeros and ones3. Gene sequence3. Gene sequence
Mainly Mainly string matching problemstring matching problem: searching : searching for a given word in a textfor a given word in a text
44
Graph Problems A A graph graph can be thought of as a collection of can be thought of as a collection of
points called vertices, some of which are points called vertices, some of which are connected by line segments called edges.connected by line segments called edges.
Used for modeling a wide variety of real-life Used for modeling a wide variety of real-life applications.applications.
Basic graph algorithms include: Basic graph algorithms include: 1. 1. Graph traversal algorithmsGraph traversal algorithms - How can one - How can one
visit all the points in a network? visit all the points in a network? 2. 2. Shortest-path algorithmsShortest-path algorithms - What is the best - What is the best
Introduction route between two cities? Introduction route between two cities? 3. 3. Topological sortingTopological sorting for graphs with directed for graphs with directed
edges - Is a set of courses with their edges - Is a set of courses with their prerequisites consistent or self-contradictory?prerequisites consistent or self-contradictory?
45
Combinatorial Problems combinatorial problems:combinatorial problems: problems that ask problems that ask
(explicitly or implicitly) to find a combinatorial (explicitly or implicitly) to find a combinatorial object—such as a permutation, a combination, object—such as a permutation, a combination, or a subset—that satisfies certain constraints or a subset—that satisfies certain constraints and has some desired property (e.g., maximizes and has some desired property (e.g., maximizes a value or minimizes a cost).a value or minimizes a cost).
Are the most difficult problems in computing:Are the most difficult problems in computing:1.1. Combinatorial grows extremely fast with Combinatorial grows extremely fast with
problem sizeproblem size2. No known algorithm solving most such 2. No known algorithm solving most such
problems exactly in an acceptable amount of problems exactly in an acceptable amount of time.time.
46
Geometric Problems Geometric algorithms Geometric algorithms deal with geometric deal with geometric
objects such as points, lines, and polygons.objects such as points, lines, and polygons.
2 class problems:2 class problems: The The closest pair problemclosest pair problem: given : given n n points in the points in the
plane, find the closest pair among them. plane, find the closest pair among them.
The The convex hull problem convex hull problem asks to find the asks to find the smallest convex polygon that would include all smallest convex polygon that would include all the points of a given set. Ifthe points of a given set. If
47
Numerical Problems Numerical problemsNumerical problems, another large , another large
special area of applications, are problems special area of applications, are problems that involve mathematical objects of that involve mathematical objects of continuous nature: solving equations and continuous nature: solving equations and systems of equations, computing definite systems of equations, computing definite integrals, evaluating functions, and so on.integrals, evaluating functions, and so on.
48
Data structure In some cases, obtaining a good running In some cases, obtaining a good running
time bound relies on the use of more time bound relies on the use of more sophisticated data structuressophisticated data structures
However, we assume a very simple data However, we assume a very simple data structure to be used in this course (like structure to be used in this course (like Array)Array)
More complex structure like Heap, More complex structure like Heap, Priority Queue ( will be learned in 2Priority Queue ( will be learned in 2ndnd year year for IT & CS students)for IT & CS students)
49
Conclusions ITS033 course structure Intro. to Algorithm
Why study Algorithm Problems Algorithmic Problem solving Important problem types
http://www.vcharkarn.com/vlesson/showlesson.php?lessonid=7&pageid=1
50
End of Chapter 1
Thank You!