2301520 fundamentals of amcspioneer.netserv.chula.ac.th/~skrung/2301520/lec01x_intro_2up.pdf ·...

2301520 FUNDAMENTALS OF AMCS

Lecture 1: IntroductionLectured by Dr. Krung Sinapiromsaran,

[email protected]

Excerpted from Dr. William Smith, [email protected]

3

• Course plan and text book• Definition of algorithm• Samples of instructions, algorithms, processes• Expressing algorithm• Problem specification• Correctness• Loop invariant• Counter example• Reason for the analysis of the algorithm

Outline

5

• Understand the term and methodology use in this course

• Know the origin of the word “Algorithm” and its informal definition

• Can distinguish between recipes or step-by-step instructions or algorithms

• Identify the specification and correctness of the problem

• Can proof by counter example• Give reason for analysis of algorithm

Objective

7

Three elements:

• Tools necessary to analyse algorithms

• A look at some common algorithms and data structures

• A look at some common algorithm design strategies

Coure plan

9

Three elements:

• Tools necessary to analyse algorithms

• A look at some common algorithms and data structures

• A look at some common algorithm design strategies

Course philosophy:

• Language neutral – English/pseudocode/examples/intuition

• Emphasis important applications of standard algorithms

• Show examples and ideas

• Work in teams on practical problem – explaining an idea to someone helps you solidify your own understanding

• You will be expected to apply your knowledge

Coure plan (extended)

11

Coure text• A lot of examples and theorems

• It is good for a reference.

• Will prove very useful for this course

• Will be useful throughout your degree/career

• A good investment

T.H. Cormen et al. Introduction to Algorithms. MIT Press, 2001.

13

References (1)

K. Mehlhorn and P. Sanders. Algorithms and Data Structures: The Basic Toolbox. Springer, 2008.

• B.R. Preiss. Data Structures and Algorithms with Object-oriented Design Patterns in Java. Wiley, 1999. (+ other Java-specific texts)

• R. Sedgewick. Algorithms. Addison Wesley, 1988.

• S.S. Skiena. The Algorithm Design Manual. Springer, 2008.

15

References (2)

“If you think you’re a really good programmer, read Knuth’s Art of Computer Programming. You should definitely send me a resume if you can read the whole thing.”

Bill Gates

D.E. Knuth. The Art of Computer Programming. Addison Wesley, 1997. (In 3 volumes)

17

What is an algorithm? (1)

Muhammad ibn Mūsā al-Khwārizmī

“algorism” referred to rules of performing arithmetic using Arabic numerals

Evolved into “algorithm” and came to mean a definite procedure for solving a problem or performing a task

“the idea behind any reasonable computer program”

Something like a recipe, process, method, technique, procedure, routine etc

19

What is an algorithm? (2)• There is no formal definition of an algorithm.

• The following features make a sensible informal definition:

1. Finiteness: Must terminate after a (very) finite number of steps.

2. Definiteness: Each step must be precisely defined (rigourous, unambiguous). Programming languages are designed for specifying algorithms – every statement has a definite meaning.

3. Input: Zero or more inputs.

Q: Are there any useful algorithms with no inputs?

4. Output: One or more outputs.

Q: Are there any useful algorithms with no outputs?

5. Effectiveness: Operations must be sufficiently basic that they can be done exactly and in a finite length of time, i.e. computable or do-able.

21










23










25










27










29

Valid algorithm? (1)• According to our criteria, are the following valid algorithms?

Pancake batter

1. Sift the flour into a mixing bowl

2. Add pinch of salt

3. Beat the eggs lightly

4. Add to the flour and mix well in

5. Leave to rest for at least 1 hour or even overnight

31


• Not valid (definiteness)

– “pinch of salt” “beat lightly” “mix well” “even overnight”

• All ambiguous or vague

• As cooks we can interpret approximately what is meant, but not precise enough for a computer to understand

Pancake batter

1. Sift the flour into a mixing bowl

2. Add pinch of salt

3. Beat the eggs lightly

4. Add to the flour and mix well in

5. Leave to rest for at least 1 hour or even overnight

33


Solving Chess

1. Construct the tree of all possible valid game states

2. Mark states as “won” if the player to move can win this turn or “lost” if player to move loses regardless of move chosen

3. Mark states as “lost” if all successors are marked “won”

4. Mark states as “won” if at least one successor is marked “lost”

5. Repeat 3 and 4 until no additional states are marked

6. If first move is marked “won”, then return “white always wins”

35


• Not valid (finiteness)

– ~1043 possible game states – although finite, not realistic to evaluate all

• Evaluate 1 million states per second = 3.2×1029 years!

• (Earth is about 4.5×109 years old)

Solving Chess

1. Construct the tree of all possible valid game states

2. Mark states as “won” if the player to move can win this turn or “lost” if player to move loses regardless of move chosen

3. Mark states as “lost” if all successors are marked “won”

4. Mark states as “won” if at least one successor is marked “lost”

5. Repeat 3 and 4 until no additional states are marked

6. If first move is marked “won”, then return “white always wins”

37


Find the greatest common divisor of positive integers m and n

1. If m<n, exchange m and n

2. Divide m by n and let r be the remainder

3. If r = 0, terminate and return n as the answer

4. Set m = n and n = r, return to 1

39


• Valid

– This is our first algorithm

– One of the oldest known algorithms (300 BC) described by Euclid (in a slightly different form)

Find the greatest common divisor of positive integers m and n

1. If m<n, exchange m and n

2. Divide m by n and let r be the remainder

3. If r = 0, terminate and return n as the answer

4. Set m = n and n = r, return to 1

41


Find the nth prime number

1. Find a function f(n) which computes the nth prime number

2. Apply the function to n and return the result

43


• Not Valid (effectiveness)

– No such function f(n) is known

– Until someone finds such a function, 1 is not an effective operation

Find the nth prime number

1. Find a function f(n) which computes the nth prime number

2. Apply the function to n and return the result

45

Practical algorithmIn practice we want algorithms that are:

• Correct

• Efficient

• Easy to implement

May not be simultaneously achievable.

47

Specification• An algorithm is an implementation of a solution to a problem

• We need a way to specify a problem so we can design algorithms to solve it

• A computer program is a translation of an algorithm into a particular programming language (also an implementation)

• Checking that an implementation correctly executes an algorithm is the realms of software engineering and formal methods

• We are interested in the design and analysis of algorithms

• We don’t care what language it is subsequently implemented in

• But we do need a language to express algorithms

Q: Why don’t we always use English? Why don’t we program computers in English?

49

Expressing Algorithm (1)

Incr

easi

ng e

ase

of e

xpre

ssio

n

Increasing

precision

1. English

2. Pseudocode

3. A programming language

51

Expressing Algorithm (2)• We will use a mixture of English and pseudocode

• Nice features of pseudocode:

– Pseudocode represents a good balance between precision and readability

– Should be straightforward to translate to any programming language

– It’s intuitive - shouldn’t need to learn it

53

Pseudocode of GCD• The algorithm for finding the greatest common divisor given

above can be expressed in pseudocode as follows:

• Exercise: try it out – for different input values, trace the operation of the algorithm keeping track of current values of the variables

Function GCD(m, n:N): N

1. If m < n then

2. (m, n) := (n, m)

3. while(n > 0) do

4. r := m mod n

5. (m, n) := (n, r)

6. Endwhile

7. Return m

55

Specification and correctness (1)• As algorists, we need tools to:

– Specify a problem so we can design algorithms to solve it

– Distinguish correct algorithms from incorrect ones

– Compare algorithms and choose the “best” one

57


– Specify a problem so we can design algorithms to solve it• and strategies to design algorithms which satisfy

specifications

– Distinguish correct algorithms from incorrect ones


59

Problem Specification• A very common (and useful) informal way to specify a problem

is to list its inputs and outputs:

Sorting:

Input:A sequence of n numbers; a1, a

2, ..., a

n

Output:A permutation, i1, i

2, ..., i

n of the input sequence such that

• The formalisation of this is a simple specification.

ai1≤ai2

≤...≤ain

61

Simple Specification• Definition: A simple specification is a pair of predicates, one

describing the starting states, the precondition, and one describing the associated final state, the postcondition.

Factorial

Precondition: n in N; Postcondition: r = n!

Sum

Precondition:n in N; Postcondition: r =

• Preconditions and postconditions are like a contract between the caller and the algorithm:

• Algorithm: “If you ensure the precondition is satisfied, I’ll guarantee that the postcondition is satisfied upon termination”

• Caller: “Ok, and if I don’t ensure the precondition is satisfied I’ll make no assumptions about what your behaviour will be”

∑i=1

n

i

63

GCD Specification• Let’s think up pre/postconditions for the greatest common

divisor algorithm:

Precondition: m, n in N & (m > 0) & (n > 0)

Postcondition: (r|m) & (r|n) & (all i in N; (i|m) & (i|n) → i < r

m and n must be positive integers

Other i that divides both m and n must be less than or equal to r

r must divide both m and n

65

Certifying GCD Algorithm• Sometimes preconditions and postconditions can be easily

checked

• Many programming languages support assertions – good practice to check pre/postconditions (more in PPS), example:

Function GCD(m; n:N):N1. assert (m > 0) & (n > 0)2. if m < n then3. (m; n) := (n, m)4. while n > 0 do5. r := m mod n6. (m; n) := (n; r)7. endwhile8. assert (r|m) & (r|n) & (all i, i|m & i|n → i < r)9. return m

Checking this assertion in a real programming is non-

trivial

67

Certifying Algorithms• This might seem a bit circular – isn’t checking the postcondition

as hard as running the algorithm in the first place?

• In many situations checking assertions can be simplified by computing additional information – a certificate

• A certifying algorithm computes a certificate for the postcondition

• We will see examples of algorithms later where a certificate can be computed without affecting the efficiency of the algorithm

69

Loop invariant• A loop invariant is a statement which is true before and after

each loop iteration

• Used to help prove program correctness

• Helps to clarify purpose of iterative structure

• Loop variant ensures progress is made

Function sum(n:N):N1. assert (n > 0)2. s := 03. for i := 1 to n do4. s := s + i5. invariant s = 6. endfor7. assert s =8. return s

∑j=1

i

j

∑i=1

n

i

71

Specifications and correctness (3)• As algorists, we need tools to:


specifications

– Distinguish correct algorithms from incorrect ones• Check that the algorithm satisfies the problem

specification


73

Reasoning about correctness• We need to know whether our algorithm will always work, i.e.

produce the correct/optimal answer.

• Let’s look at an example:

Robot Tour Optimization

Robot arm must visit a set of contact points on a circuit board to solder each point

Need to choose an order in which the contact points will be visited

We want to minimise time taken, i.e. shortest path

Shorter tour = faster production = cheaper to produce = more profit

• An important algorithm!

75

Nearest neighbor heuristicRobot Tour Optimization

• Simple idea: nearest neighbor tour

• Use a heuristic – pick any starting point, then next move is always to nearest point

Input: A set P of n pointsOutput: The shortest cycle tour that visits each point

77

Tour with nearest neighbor heuristicRobot Tour Optimization:- Nearest neighbor

Function NearestNeighbor(P: Set of points)1. Pick and visit an initial point p[0] in P2. i := 03. while(P is not empty)do4. i := i + 15. P := P – {p[i-1]}6. Let p[i] be the closet point to p[i-1]7. Visit p[i]8. endwhile9. Return to p[0] from p[i]

79

Tour example (1)Robot Tour Optimization:- Nearest neighbour

0

81


01

83


01

2

85


• Look encouraging

01

2

345

6

7

87



• Q:Optimal for this instance?

01

2

345

6

7

89




• Q:Always optimal?

01

2

345

6

7

91




• Q:Always optimal?

• Q:What is needed to prove incorrectness?

01

2

345

6

7

93

Counter tour example (1)Robot Tour Optimization:- Nearest neighbor

An instance where nearest neighbor gives a suboptimal solution:

0

95



0 1

97



0 12

99



0 12 3

101



0 12 34

103



0 12 34 5

105


A better (optimal) solution for the same instance:

• It can be easy to prove that an algorithm is not correct – just find a counter example (the simpler, the better)

• Counter example = instance of problem + optimal solution + demonstration that algorithm produces a less optimal solution

• Lack of a counter example doesn’t prove it works

• We will not study the formal methods to prove correctness of our algorithms

• Instead, we will informally convince ourselves that they work

0 1 2 34 5

107



specifications

– Distinguish correct algorithms from incorrect ones• Check that the algorithm satisfies the problem

specification

– Compare algorithms and choose the “best” one• There may be many alternative algorithms that satisfy a

specification• We want the one that is “best” for our purposes• Often, best = most efficient. Is this always the case?• The subject of much of this course!

109

Analysis of algorithms (1)The theoretical study of computer program performance and resource usages.• Is performance the most important measure of an algorithm?

• What’s more important than performance?

111

Analysis of algorithms (2)The theoretical study of computer program performance and resource usages.• Is performance the most important measure of an algorithm?

• What’s more important than performance?

• Performance is the currency of computation

• We can use it to buy other things: functionality, user-friendliness etc.

correctness security user-friendliness maintainability robustness/reliability

functionality/features simplicity programmer time modularity extensibility

113

Analysis of Algorithms (3)• Why study analysis of algorithms? A few cliches:

– Some algorithms experts are lousy programmers, but no good programmer can be bad at algorithms

– Algorithm design and analysis is the essence of what we do as computer scientists

– With more sophisticated software engineering systems, the demand for mundane programmers will diminish

– Soon all known algorithms will be available in libraries

115

Analysis of Algorithms (4)• Why study analysis of algorithms? A few cliches:

– Great thinkers will always be needed: “I can develop a new algorithm for you”

– “To be a good programmer you can either study programming for 10 years or take an algorithms course and study programming for 2 years”

– Making things faster is fun! Some people try to design faster cars, we try to design faster algorithms!

117

Conclusion• You should now know:

– What an algorithm is

– How to specify a problem

– How to show algorithm incorrectness by counter example

– Some justification for why algorithm analysis is important

2301520 fundamentals of amcspioneer.netserv.chula.ac.th/~skrung/2301520/lec01x_intro_2up.pdf ·...

Documents