Download - Analyzing Algorithms and Problems
Analyzing Algorithms and Analyzing Algorithms and ProblemsProblems
Prof. Sin-Min Lee
Department of Computer Science
Euclid's AlgorithmEuclid's Algorithm
• In Euclid's 7th book in the Elements series (written about 300BC), he gives an algorithm to calculate the highest common factor (largest common divisor) of two numbers x and y where (x < y). This can be stated as:
• 1.Divide y by x with remainder r. • 2.Replace y by x, and x with r. • 3.Repeat step 1 until r is zero. • When this algorithm terminates, y is the
highest common factor.
Euclid's AlgorithmEuclid's Algorithm
• Euclid's Algorithm determines the greatest common divisor of two natural numbers a, b. That is, the largest natural number d such that d | a and d | b.
• GCD(33,21)=3• 33 = 1*21 + 12• 21 = 1*12 + 9• 12 = 1*9 + 3• 9 = 3*3
Basic Algorithm AnalysisBasic Algorithm Analysis
• Questions
How does one calculate the running time of an algorithm?
How can we compare two different algorithms?
How do we know if an algorithm is `optimal'?
Time vs. Input GraphTime vs. Input Graph
How do you know if you have the How do you know if you have the best algorithm?best algorithm?
A lower bound can be established because a basic number of operations must be performed. Thus, any algorithm that performs that number would be optimal, even if no such algorithm has been found that uses that number of steps.
Hanoi TowerHanoi Tower
• Long time ago, there were three towers in Honoi. In the very beginning, there were 64 disks with different sizes stacking on the A tower, small disks always stepping on the bigger disks. A monk had to move all disks to the C tower following rules:
• 1. Each time only one disk could be moved from a tower, to any other;
• 2. At anytime, small disks should step on the bigger disks, not allowing big disks step on smaller disks.
• The Tower of Hanoi (sometimes referred to as the Tower of Brahma or the End of the World Puzzle) was invented by the French mathematician, Edouard Lucas, in 1883. He was inspired by a legend that tells of a Hindu temple where the pyramid puzzle might have been used for the mental discipline of young priests. Legend says that at the beginning of time the priests in the temple were given a stack of 64 gold disks, each one a little smaller than the one beneath it. Their assignment was to transfer the 64 disks from one of the three poles to another, with one important proviso—a large disk could never be placed on top of a smaller one. The priests worked very efficiently, day and night. When they finished their work, the myth said, the temple would crumble into dust and the world
would vanish.
.. CountCount the number of the number of basic operationsbasic operations performed by the algorithm on the performed by the algorithm on the worst-case worst-case inputinput
• A basic operation could be: An assignment
A comparison between two variables
An arithmetic operation between two variables. The worst-case input is that input assignment for which the most basic operations are performed.
Algorithm EfficiencyAlgorithm Efficiency
• Determining the efficiency of an Algorithm is done by counting the number of basic operations that are carried out when the algorithm is executed
• A basic operation is an operation carried out by the algorithm that is language independent, such as addition, subtraction, comparison of two values, etc…
Determination of EfficiencyDetermination of Efficiency
• How to determine the number of basic operations per every input? Impossible!!!
• Instead we devise a way to compare algorithms to each other in a relative way in order to get a “feel” for their efficiency.
• We do this by gauging the algorithm’s Asymptotic Growth Rate
Asymptotic Notation for Big OAsymptotic Notation for Big O
• Big O (Omicron) can be defined as
•Note: the following two conditions must be met:
•The limit must exist
•The limit c <
An Example of solving Big OAn Example of solving Big O
Example:
•Thus c = 35, and n = 10
Big Omega (Ω)Big Omega (Ω)
Definition:
•The limit must be greater than 0
•The limit may be
Definition:
Big Theta(Θ)Big Theta(Θ)
• Theta can be defined as
set is greater than 0
set is smaller than
Comparison of Ω, Θ, and ΟComparison of Ω, Θ, and Ο
• Ο(g): functions that grow no faster than g
• Θ(g): functions grow at the same rate as g
• Ω(g): functions grow at least as fast as g– Together Big O and set establish the lower
and upper boundaries of asymptotic order. (g) = (g) (g)
Notes about Asymptotic NotationNotes about Asymptotic Notation
• Big O notation should be of the smallest order possible
• Lower order and constant terms can be dropped as long as a larger term exists that does not have a coefficient of 0
The Importance of Asymptotic The Importance of Asymptotic OrderOrder
• An algorithm that runs at O(n) time is generally better than an algorithm that runs at O(n ). 2
ClarificationClarification
• When talking about Big O, , , and little o, , we do not talk about specific functions. (that is, there is no function that is exclusively Big O of something else)
• What we mean when we say that function f is big O of function g is that the function f is a member of the set of functions which satisfy the given criteria for big O.
• The proper notation is to say that f is an element in the set of functions which is identified as O(g), or more explicitly
– f O(g),
– Remember, f O(g)
Big O NotationBig O Notation
• Big O (omicron) notation is used to define a set of functions which grow no faster than a given function
• Let g be a function from the nonnegative integers to the positive real numbers
Definition of big ODefinition of big O
• O(g) is the set of all functions f from nonnegative integers into positive real numbers such that the limit of f over g as n approaches is equal to a constant c such
that c <
cng
nf
n )(
)(lim
Example of big O (1)Example of big O (1)
You have three functionsg: y = .5xf1: y = .2xf2: y = 2x
You want to check if f1 and f2 are O of g
0
10
20
30
40
50
60
70
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29
g
f1
f2
Example of big O (2)Example of big O (2)
45.
2lim
2lim
x
x
g
f
nn
• Both values are less than
• Both f1 and f2 are elements of O(g)
4.5.
2.lim
1lim
x
x
g
f
nn
Example of big O (3)Example of big O (3)
• In both cases as the initial value increased, the limit of the ratio between f1, f2 and g remained less than
• f1 and f2 are O(g) or f1 and f2 are O of g.
• In practice, this means that the functions in question (in our case f1 and f2) are growing at a rate that is not faster than g.
When Big O shows a differenceWhen Big O shows a differenceExponential vs Linear growth
0
100000000
200000000
300000000
400000000
500000000
600000000
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
n
f(n
)
• Pink g• g = 1,000,000n
• Yellow f• f = en
n
e
g
f
n
nn 000,000,1limlim
Summing up Big OSumming up Big O
• In order for the following two conditions must be met– The limit must exist
– The limit c <
cng
nf
n )(
)(lim
The Omega setThe Omega set
• The Omega set () is the “big brother” of Big O
of g is the set of functions such that:
0)(
)(lim
ng
nf
n