lecture 1 preliminaries

Lecture 1: Preliminaries 1

Lecture 1Preliminaries

COMP 523: Advanced Algorithmic Techniques

Lecturer: Dariusz Kowalski


Announcements• Textbook: J. Kleinberg, E. Tardos. Algorithm

Design. Addison-Wesley, 2005• Lectures: Monday 13-15 in BROD-405, Thursday

12-13 in BROD-405, Friday 10-11 in MATH-105• Tutorials: Monday 15-16 in ELEC-204• Office hours (room 3.11, Ashton Bldg):

– Monday 15-16– Friday 11-13

• Info and contact:– http://www.csc.liv.ac.uk/~darek/comp523.html– [email protected]

http://www.csc.liv.ac.uk/~darek/

Assessment and feedback

• Assessment: Exam (75%) and 2 tasks (25% total)

• Feedback: – Presentation and discussion of solutions after

assignment deadline– Short individual feedback after assignments, by

email– In case of specific questions, individual

appointments



Algorithms - what does it mean?Algorithm - a list/structure of instructions, which are carried

out in a fixed order – to find the answer to a question– to calculate

Al-Khwarizmi (Muhammed Ibn Musa) - Arabian mathematician who introduced the decimal positional number system to the Western world, the author of Algebra

This module: techniques to design algorithms for problems concerning data commonly processed by computers (written in hard discs, propagated in networks, entered by user or external device, etc.)


Types of algorithms

• Constructive vs. Non-constructive

• Discrete vs. Numerical

• Deterministic vs. non-deterministic (e.g., Randomized, Quantum)

• Sequential vs. Concurrent vs. DNA vs. …

• Exact vs. Approximate

• and many others


Example

Problem: Find if a given word occurs in a given text

Input: Text and word represented as lists

Output: Word found or not

o n c e u p o n a t

o n a


Example - algorithmNaïve solution (exhaustive search):Intuition:• Check letter after letter, starting from the beginning of the lists, if the

corresponding letters are equal• If some corresponding letters are not equal, re-start comparing from the second

letter of the text (and the first letter of the word)• Etc. from the 3rd letter, 4th letter, until the end of text or successful comparison

Implementation:• Initiate three pointers:

– blue_pointer and black_pointer at the beginning of blue_list – yellow_pointer at the beginning of yellow_list

• Initiate Boolean variable successful into false


Naïve solution cont.• Repeat

– stop := false– Move blue_pointer to the next element – Set black_pointer to blue_pointer– Move yellow_pointer to the first element of yellow_list– Repeat

• Move black_pointer and yellow_pointer to the next elements in corresponding lists

• If values pointed by black_pointer and yellow_pointer are different then stop := true

until yellow_pointer or black_pointer is at the end of its list or stop– If yellow_pointer is at the end of yellow_list and not stop then

successful := true

until successful or blue_pointer is at the end of blue_listOutput: successful


Example - how it works

o n c e u p o n a t

o n a

• Repeat• stop := false, • move blue_pointer to the next element, set black_pointer to blue_pointer, • move yellow_pointer to the first element of yellow_list– Repeat

• Move black_pointer and yellow_pointer to the next elements in corresponding lists• If values pointed by black_pointer and yellow_pointer are different then stop := true

until yellow_pointer or black_pointer is at the end of its list or stop– If yellow_pointer is at the end of yellow_list and not stop then successful := true

until successful or blue_pointer is at the end of blue_list


What we expect from algorithms?• Correctness: computes desired output• Termination: stops eventually (or with high probability)• Efficiency: with respect to

– Performance measure• Time (or total number of computation steps)• Size of memory used• Number of messages sent• …

– Methods of measuring• Worst-case• Average-case• Smoothed (a subset of possible inputs, specific distribution of inputs, etc.)• Competitive (comparing to the best solution for particular input)• Expected• …

Analysis depends on

Model!


Example - correctness

• Successful is set to true iff after some execution of the internal loop the yellow_pointer is at the end of the yellow_list

• yellow_pointer is at the end of the yellow_list iff it came through the whole yellow_list without coming back to the beginning

• it happens if the values pointed by black_pointer and yellow_pointer have been the same during all checks of internal loop in the current run (thus a copy of the word exists in the text)

Repeat• stop := false, • move blue_pointer to the next element,

set black_pointer to blue_pointer, • move yellow_pointer to the first

element of yellow_list– Repeat



until yellow_pointer or black_pointer is at the end of its list or stop

– If yellow_pointer is at the end of yellow_list and not stop then successful := true



Example - termination• External Loop Invariant:

At the beginning of the external loop, blue_pointer advances, and it has a finite number of advances to take before reaching the end of the text

• Internal Loop Invariant:

Each run of the internal loop finishes eventually;

It happens because yellow_pointer keeps advancing every iteration (first line of the loop), unless stop condition becomes true or it reaches the end of the word (or black_ pointer reaches the end of the text)

Repeat• stop := false, • move blue_pointer to the next element,

set black_pointer to blue_pointer, • move yellow_pointer to the first

element of yellow_list– Repeat



until yellow_pointer or black_pointer is at the end of its list or stop

– If yellow_pointer is at the end of yellow_list and not stop then successful := true



Example - efficiency

• Complexity measures: time, size of additional memory• Size of additional memory:

– 3 single pointers and 2 binary variables

• Time complexity:– Worst-case: if the sequence is not included in the text then

time is (almost) proportional to the multiplication of sizes of blue and yellow lists, e.g.,

text: ‘aab’ repeated n times

word: aaa

1, 2 or 3 internal loop runs, 3n times, gives about 6n runs


Example - efficiency cont.

• Time complexity (cont.):– Average case: text or sequence are randomly selected;

average short word should be find quickly in the beginning of an average long text, in time proportional to the squared length of the word (exercise)

– Smoothed analysis: text must be randomly selected from “reasonable” set of texts (usually complicated analysis, depends on the family of texts - dictionary)


Efficiency - asymptotic notation• n - integer, size of data, f() - function over integers• Complexity O(f(n)) : there is a positive constant c such

that for every positive integer n complexity is at most cf(n)

• Complexity (f(n)) : there is a positive constant c such that for every positive integer n complexity is at least cf(n)

• Complexity o(f(n)) : complexity divided by f(n) comes to 0 with n going to infinity (strictly smaller than f(n))

• Complexity (f(n)) : complexity divided by f(n) comes to with n going to infinity (strictly bigger than f(n))


Example - asymptotic complexity

• Worst case time: – If size of text is n and size of word is n/2 :

O((n/2)(n/2)) = O(n2) – If size of text is m and size of sequence is k :

O((m-k)k)


Examples

5n3+100 = O(n3) , 5n3+100 ≠ O(n2)5n3+100 = (n3) , 5n3+100 ≠ (n4)log n = o(na) for any positive constant ana = o(cn) for any positive constants a,clog (4n) = log n + log 4 = O(log n)log (n4)= 4 log n = O(log n)(4n)3 = 64n3 = O(n3)(n4)3 = n12 = (n4) (3n)4 = 81n = (3n)Logarithms are to the base 2


Symmetric properties and tight bound

If f(n) = O(g(n)) then g(n) = (f(n)) and vice versa

If f(n) = o(g(n)) then g(n) = (f(n)) and vice versa

Definition:

If f(n) = O(g(n)) and f(n) = (g(n)) then

f(n) = (g(n)) and g(n) = (f(n))

Example:

9n2 + n +7 = (3n2) = (n2)


Transitive properties (order)

If f(n) = O(g(n)) and g(n) = O(h(n)) then

f(n) = O(h(n))

If f(n) = (g(n)) and g(n) = (h(n)) then

f(n) = (h(n))

If f(n) = (g(n)) and g(n) = (h(n)) then

f(n) = (h(n))


Sum and maximum

f1(n) + … + fa(n) = (max(f1(n), … , fa(n)))

for any constant positive integer a

Example:

• 3n2 + n +7 = (3n2) = (n2)

If the range of the summation index is not constant:

in i = n(n+1)/2 = (n2)

(max{1, … , n}) = (n)


Logarithmic time O(log n)

• Reduce input to any fraction of it in constant time

• Example: searching if a given element x is in a sorted array


Linear time O(n)

• It is sufficient to look at each element of the input constant number of times

• Example: finding maximum in an (unsorted) array


Time O(n log n)

• Split the input into two pieces of the similar size and merge both solutions in linear time

• Example: sorting an array (split into halves, sort each part separately, merge sorted parts in linear time)


Quadratic time O(n2)

• It is enough to consider pairs of elements

• Examples: – finding the closest pair of points located in a

plane– sorting by comparison of subsequent elements

(insertion sort)


Polynomial time O(na)

• Algorithm has many nested loops

• Example: are there any two disjoint sets among given family of n sets, each of at most n elements (time O(n3))


Exponential time O(cn)

• Algorithm considers many subsets of the input

• Example: exhaustive search to find the largest clique contained in a given graph (time O(2n))


Beyond exponential time: O(n!), O(nn), …

• Algorithm searching in a large space

• Example: search performed in the set of all permutations (time O(n!))


Graphs• Set of nodes |V| = n• Set of edges |E| = m

– Undirected edges/graph: pairs of nodes {v,w}– Directed edges/graph: pairs of nodes (v,w)

• Set of neighbors of v : set of nodes connected by an edge with v (directed: in-neighbors, out-neighbors)

• Degree of a node: number of its neighbors• Path: a sequence of (non-repeating) nodes such that every

two consecutive nodes constitute an edge• Length of a path: number of nodes minus 1• Distance between two nodes: the length of the shortest

path between these nodes• Diameter: the longest distance in the graph


Lines, cycles, trees, cliques

Line Cycle

Clique

Tree


Conclusions

• Algorithm: list/structure of instructions

• Algorithmic methods for problems arising from computer and communication applications

• Guarantee correctness, termination and efficiency– Model is important!

• Asymptotic analysis of efficiency


Textbook and exercises

READING:• Chapter 2, up to section 2.4

OBLIGATORY EXERCISES: • Exercises 1,2,3 page 67 • Solved exercises 1,2 pages 65,66

OPTIONAL:• Exercises 3,4,5,6 pages 67,68,69• Exercises 7,8 pages 69,70

lecture 1 preliminaries

Documents