CMSC 441: Introduction to Algorithms

Hillol Kargupta, Professorhttp://www.cs.umbc.edu/~hillol/

hillol@cs.umbc.edu

January 27, 2013 CMSC 441 2

Today’s Topics

Administrivia Course Overview

January 27, 2013 CMSC 441 3

Contact Information Instructor:Hillol Kargupta

e-mail: hillol@cs.umbc.edu Office hours: Monday 2:00--3:00pm in ITE 342

Website: http://www.cs.umbc.edu/~hillol/CLASSES/CMSC441Spring2013/

Teaching Assistant: TBA

January 27, 2013 CMSC 441 4

Textbook

Introduction to Algorithms, Cormen, Leiserson, Rivest and Stein, McGraw-Hill, third edition

January 27, 2013 CMSC 441 5

Grading

33%: 11 homework assignments 40%: Exam 1(20%) and 2 (20%) 27%: Final exam

January 27, 2013 CMSC 441 6

Letter Grades

0 <= F < 60 60 <= D < 70 70 <= C < 80 80 <= B < 90 90 <= A <= 100

January 27, 2013 CMSC 441 7

More on Grading

Incomplete grades: Only for medical illness or other such dire circumstances.

January 27, 2013 CMSC 441 8

Homework

Do your homework: Carries a lot of weights.

No late homework Partial marks for serious attempts to

solve a problem So submit your homework in time.

January 27, 2013 CMSC 441 9

Academic Honesty

Maintain academic integrity

Refer to the documentation on UMBC policies for academic honesty (linked from the class web site)

January 27, 2013 CMSC 441 10

Course Purpose

Design of algorithms Analysis of algorithms

Quantifying and comparing algorithmic performance

Proving the correctness of algorithms A taste of different algorithms useful

in computer science.

January 27, 2013 CMSC 441 11

Background

Discrete Math Data Structures Appendix of Text:

Summations Sets Counting and Probability

January 27, 2013 CMSC 441 12

What is an Algorithm?

An algorithm is a precisely defined process, that may take an input and generates an output.

Use of algorithms: From every day life to advanced computing application.

January 27, 2013 CMSC 441 13

An Example

Example: Making a telephone call. Input: Telephone number Process:

(1) pick up the reciever (2) dial (3) talk (4) put down the receiver

Output: Communication

January 27, 2013 CMSC 441 14

Algorithms: Desirable Features

Important aspects of an algorithm: Correctness Resources needed to execute the

algorithm

We need algorithms that are correct and efficient in using resources like memory and time

January 27, 2013 CMSC 441 15

Designing Algorithms:A Naive Approach

Given a particular problem, how do we choose an algorithm?

Empirical Approach: (1) code `em and run `em on ``typical''

inputs, using the same machine; (2) measure their running times; (3) select the ``winner'' (e.g., the faster).

January 27, 2013 CMSC 441 16

A Better Approach

Develop an algorithm. Prove its correctness. Analyze the running time and other

requirements for resources Experiment

January 27, 2013 CMSC 441 17

Sorting: An Example Problem: Sort a list of n numbers in

nondecreasing order.

Input: A sequence of n numbers A(1), A(2),……..,A(n)

Output: A permutation: A’(1), A’(2),…...,A’(n) such that A’(1) A’(2) …... A’(n).

January 27, 2013 CMSC 441 18

Sorting Algorithm: An Example

Insertion sort: A simple but reasonably efficient way to sort small number of elements.

January 27, 2013 CMSC 441 19

Insertion Sort

InsertionSort(int a[], int n) {for (j = 2, n, j++) {

key = A[j]k := j-1

while (k > 0 and A[k]> key) {A[k+1]:= A[k]k := k-1 }

A[k+1] := key}

}

January 27, 2013 CMSC 441 20

Loop Invariants

Sub-array A[1….j] consists of sorted elements.

Plays a key role in understanding why the algorithm is correct.

January 27, 2013 CMSC 441 21

Three Things about Loop Invariants

Initialization: True prior to the first iteration

Maintenance: If it is true before an iteration, it remains true before the next iteration

Termination: When the loop terminates, the invariant gives us useful property for proving the algorithmic correctness

January 27, 2013 CMSC 441 22

Insertion Sort: Analysis

InsertionSort(int a[], int n) {for (j = 2, n, j++) { c1 n

key = A[j] c2 n-1k := j-1 c3 n-1

while (k > 0 and A[k]> key) { c4 A[k+1]:= A[k] c5k := k-1 } c6

A[k+1] := key c7 n-1}

}

n

jjt

2

)1(

n

jjt

2

)1(

n

jjt

2

tj be the number of times the while loop test is executed.

January 27, 2013 CMSC 441 23

Measuring Algorithms

Comparing algorithms Coding environment data set

Find significant differences in performance.

January 27, 2013 CMSC 441 24

Growth Rates of Algorithms

N2 / 5 2 * N + 100 2N - 1

0

200

400

600

800

1000

1200

1 2 3 4 5 6 7 8 9 10

January 27, 2013 CMSC 441 25

Big 0 Notation: The Idea

If some algorithm A requires time proportional to g(n), then it is said to be of order g(n), normally write O(g(n))

g(n) is the growth rate function.

January 27, 2013 CMSC 441 26

Big O Defined

O notation: O(g(n)) = { f(n) : positive constants c, n0 such that 0 f(n) c g(n) for all n n0 }

January 27, 2013 CMSC 441 27

More Definitions

notation: (g(n)) = { f(n) : positive constants c, n0 such that 0 c g(n) f(n) for all n n0 }

notation: (g(n)) = { f(n) : positive constants c1, c2, & n0 such that 0 c1g(n) f(n) c2 g(n) for all n n0 }

January 27, 2013 CMSC 441 28

C g(n)

f(n)

n0

n

O(g(n)): A Graphical Perspective

f(n) = O(g(n))

January 27, 2013 CMSC 441 29

C g(n)

f(n)

n0

n

(g(n)): A Graphical Perspective

f(n) = (g(n))

January 27, 2013 CMSC 441 30

C1 g(n)

f(n)

n0

n

(g(n)): A Graphical Perspective

f(n) = (g(n))

C2 g(n)

January 27, 2013 CMSC 441 31

Theorem

f(n) = (g(n)) if and only if f(n) = O(g(n)) and f(n) = (g(n))

January 27, 2013 CMSC 441 32

Asymptotic Notation in Formulas

n = O(n3) n3 + 2 n + 3 = n3 + O(n) Equality (“=’’) means set membership

January 27, 2013 CMSC 441 33

Transitivity

Transitivity:f(n)= (g(n)) and g(n)= (h(n)) imply f(n)= (h(n))

f(n)= O(g(n)) and g(n)= O(h(n)) imply f(n)= O(h(n))

January 27, 2013 CMSC 441 34

Reflexivity

f(n)= (f(n)) f(n)= O(f(n)) f(n)= (f(n))

January 27, 2013 CMSC 441 35

Symmetry

f(n)= (g(n)) if and only if g(n)= (f(n))

January 27, 2013 CMSC 441 36

What is a Proof?

Given a collection of assumptions or hypotheses statements that are already known to be

correct

Use general principles of logical reasoning for deriving a statement

January 27, 2013 CMSC 441 37

Proof by Direct Construction

Assume p is true to show that q is true.

Example: For any integers a and b, if a and b are odd then ab is odd.

Proof hint: if n is an odd integer n=2x+1

January 27, 2013 CMSC 441 38

Another Example

Show that 2n+1 = O(2n)

Proof by construction

January 27, 2013 CMSC 441 39

Yet Another Example

Show that (n+a)b = (nb)

Hint: n+a n + |a| 2n when |a| n n + a n - |a| n/2 when |a| n/2 0 n/2 n + a 2n when n 2 |a|

January 27, 2013 CMSC 441 40

Proof by Contradiction

Prove p by contradiction

Show that if p is not true then some contradiction arises

p false

Note that p q implies p q

January 27, 2013 CMSC 441 41

Example

To prove: For any sets A, B, and C, if AB= and C B then A C=

Assume if AB= and C B then A C

x xAC; So x AB;

Contradiction AB

January 27, 2013 CMSC 441 42

Another Example

Show that 22n O(2n)

Proof by contradiction.

January 27, 2013 CMSC 441 43

Motivation behind Mathematical Induction

Statements involving a natural number n is often needed to prove.

Example:1+2+3+…… + n = n(n+1)/2

January 27, 2013 CMSC 441 44

Proof by Mathematical Induction

P(n) is a statement involving a integer n. Prove that P(n) is true for every n n0

Show: P(n0) is true For any k n0 if P(k) is true, then P(k+1) is

true.

January 27, 2013 CMSC 441 45

Strong Induction

P(n) is a statement involving a integer n. Prove that P(n) is true for every n n0

Show: P(n0) is true For all n0 n k if P(n) is true, then P(k+1) is

true.

January 27, 2013 CMSC 441 46

Miscellaneous

Monotonocity.

Fibonacci sequence.

f(0) = 1; f(1) = 1;f(n+1) = f(n) + f(n-1)for every n 1

January 27, 2013 CMSC 441 47

Series and Summations

Finite sums: a1 + a2 + a3 + ……. + an

Infinite sums: a1 + a2 + a3 + …….

Diverging series Converging series

January 27, 2013 CMSC 441 48

Arithmetic Series

1+2+……+ n = n(n+1)/2 = O(n2)

Prove it

January 27, 2013 CMSC 441 49

Geometric Series

1 + x +x2 + ….+ xn = (xn+1-1)/(x-1)

Prove it.

January 27, 2013 CMSC 441 50

Harmonic Series

1 + 1/2 + 1/3 + …. + 1/n = ln n + O(1)

Prove it.