September 22, 2003

1

Algorithms and Data StructuresLecture III

Simonas ŠaltenisAalborg Universitysimas@cs.auc.dk

September 22, 2003

2

This Lecture

Divide-and-conquer technique for algorithm design. Example problems: Tiling Searching (binary search) Sorting (merge sort).

September 22, 2003

3

Tiling

A tromino tile:

And a 2nx2n board with a hole:

A tiling of the board with trominos:

September 22, 2003

4

Tiling: Trivial Case (n = 1)

Trivial case (n = 1): tiling a 2x2 board with a hole:

Idea – try somehow to reduce the size of the original problem, so that we eventually get to the 2x2 boards which we know how to solve…

September 22, 2003

5

Tiling: Dividing the Problem

To get smaller square boards let’s divide the original board into for boards

Great! We have one problem of the size 2n-

1x2n-1! But: The other three

problems are not similar to the original problems – they do not have holes!

September 22, 2003

6

Tiling: Dividing the Problem

Idea: insert one tromino at the center to get three holes in each of the three smaller boards

Now we have four boards with holes of the size 2n-

1x2n-1. Keep doing this division,

until we get the 2x2 boards with holes – we know how to tile those

September 22, 2003

7

Tiling: Algorithm

INPUT: n – the board size (2nx2n board), L – location of the hole. OUTPUT: tiling of the board

Tile(n, L) if n = 1 then Trivial case Tile with one tromino return Divide the board into four equal-sized boards Place one tromino at the center to cut out 3 additional

holes Let L1, L2, L3, L4 denote the positions of the 4 holes Tile(n-1, L1) Tile(n-1, L2) Tile(n-1, L3) Tile(n-1, L4)

INPUT: n – the board size (2nx2n board), L – location of the hole. OUTPUT: tiling of the board

Tile(n, L) if n = 1 then Trivial case Tile with one tromino return Divide the board into four equal-sized boards Place one tromino at the center to cut out 3 additional

holes Let L1, L2, L3, L4 denote the positions of the 4 holes Tile(n-1, L1) Tile(n-1, L2) Tile(n-1, L3) Tile(n-1, L4)

September 22, 2003

8

Divide and Conquer

Divide-and-conquer method for algorithm design: If the problem size is small enough to

solve it in a straightforward manner, solve it. Else:

Divide: Divide the problem into two or more disjoint subproblems

Conquer: Use divide-and-conquer recursively to solve the subproblems

Combine: Take the solutions to the subproblems and combine these solutions into a solution for the original problem

September 22, 2003

9

Tiling: Divide-and-Conquer

Tiling is a divide-and-conquer algorithm: Just do it trivially if the board is 2x2, else: Divide the board into four smaller boards

(introduce holes at the corners of the three smaller boards to make them look like original problems)

Conquer using the same algorithm recursively

Combine by placing a single tromino in the center to cover the three introduced holes

September 22, 2003

10

Find a number in a sorted array: Just do it trivially if the array is of one element Else divide into two equal halves and solve each half Combine the results

Binary Search

INPUT: A[1..n] – a sorted (non-decreasing) array of integers, s – an integer. OUTPUT: an index j such that A[j] = s. NIL, if j (1jn): A[j] s Binary-search(A, p, r, s): if p = r then if A[p] = s then return p else return NIL q(p+r)/2 ret Binary-search(A, p, q, s) if ret = NIL then return Binary-search(A, q+1, r, s) else return ret

INPUT: A[1..n] – a sorted (non-decreasing) array of integers, s – an integer. OUTPUT: an index j such that A[j] = s. NIL, if j (1jn): A[j] s Binary-search(A, p, r, s): if p = r then if A[p] = s then return p else return NIL q(p+r)/2 ret Binary-search(A, p, q, s) if ret = NIL then return Binary-search(A, q+1, r, s) else return ret

September 22, 2003

11

Recurrences

Running times of algorithms with Recursive calls can be described using recurrences

A recurrence is an equation or inequality that describes a function in terms of its value on smaller inputs

Example: Binary Search

(1) if 1( )

2 ( / 2) (1) if 1

nT n

T n n

solving_trivial_problem if 1( )

num_pieces ( / subproblem_size_factor) dividing combining if 1

nT n

T n n

September 22, 2003

12

Binary Search (improved)

INPUT: A[1..n] – a sorted (non-decreasing) array of integers, s – an integer. OUTPUT: an index j such that A[j] = s. NIL, if j (1jn): A[j] s Binary-search(A, p, r, s): if p = r then if A[p] = s then return p else return NIL q(p+r)/2 if A[q] s then return Binary-search(A, p, q, s) else return Binary-search(A, q+1, r, s)

INPUT: A[1..n] – a sorted (non-decreasing) array of integers, s – an integer. OUTPUT: an index j such that A[j] = s. NIL, if j (1jn): A[j] s Binary-search(A, p, r, s): if p = r then if A[p] = s then return p else return NIL q(p+r)/2 if A[q] s then return Binary-search(A, p, q, s) else return Binary-search(A, q+1, r, s)

T(n) = (n) – not better than brute force! Clever way to conquer:

Solve only one half!

September 22, 2003

13

Running Time of BS

T(n) = (lg n) !

(1) if 1( )

( / 2) (1) if 1

nT n

T n n

September 22, 2003

14

Merge Sort Algorithm Divide: If S has at least two elements (nothing

needs to be done if S has zero or one elements), remove all the elements from S and put them into two sequences, S1 and S2 , each containing about half of the elements of S. (i.e. S1 contains the first n/2elements and S2 contains the remaining n/2elements).

Conquer: Sort sequences S1 and S2 using Merge Sort.

Combine: Put back the elements into S by merging the sorted sequences S1 and S2 into one sorted sequence

September 22, 2003

15

Merge Sort: Algorithm

Merge-Sort(A, p, r) if p < r then q(p+r)/2 Merge-Sort(A, p, q) Merge-Sort(A, q+1, r) Merge(A, p, q, r)

Merge-Sort(A, p, r) if p < r then q(p+r)/2 Merge-Sort(A, p, q) Merge-Sort(A, q+1, r) Merge(A, p, q, r)

Merge(A, p, q, r) Take the smallest of the two topmost elements of sequences A[p..q] and A[q+1..r] and put into the resulting sequence. Repeat this, until both sequences are empty. Copy the resulting sequence into A[p..r].

Merge(A, p, q, r) Take the smallest of the two topmost elements of sequences A[p..q] and A[q+1..r] and put into the resulting sequence. Repeat this, until both sequences are empty. Copy the resulting sequence into A[p..r].

September 22, 2003

16

MergeSort (Example) - 1

September 22, 2003

17

MergeSort (Example) - 2

September 22, 2003

18

MergeSort (Example) - 3

September 22, 2003

19

MergeSort (Example) - 4

September 22, 2003

20

MergeSort (Example) - 5

September 22, 2003

21

MergeSort (Example) - 6

September 22, 2003

22

MergeSort (Example) - 7

September 22, 2003

23

MergeSort (Example) - 8

September 22, 2003

24

MergeSort (Example) - 9

September 22, 2003

25

MergeSort (Example) - 10

September 22, 2003

26

MergeSort (Example) - 11

September 22, 2003

27

MergeSort (Example) - 12

September 22, 2003

28

MergeSort (Example) - 13

September 22, 2003

29

MergeSort (Example) - 14

September 22, 2003

30

MergeSort (Example) - 15

September 22, 2003

31

MergeSort (Example) - 16

September 22, 2003

32

MergeSort (Example) - 17

September 22, 2003

33

MergeSort (Example) - 18

September 22, 2003

34

MergeSort (Example) - 19

September 22, 2003

35

MergeSort (Example) - 20

September 22, 2003

36

MergeSort (Example) - 21

September 22, 2003

37

MergeSort (Example) - 22

September 22, 2003

38

Merge Sort Summarized To sort n numbers

if n=1 done! recursively sort 2 lists of

numbers n/2 and n/2 elements

merge 2 sorted lists in (n) time

Strategy break problem into similar

(smaller) subproblems recursively solve

subproblems combine solutions to

answer

September 22, 2003

39

Running time of MergeSort

Again the running time can be expressed as a recurrence:

(1) if 1( )

2 ( / 2) ( ) if 1

nT n

T n n n

solving_trivial_problem if 1( )

num_pieces ( / subproblem_size_factor) dividing combining if 1

nT n

T n n

September 22, 2003

40

Repeated Substitution Method

Let’s find the running time of merge sort (let’s assume that n=2b, for some b).

1 if 1( )

2 ( / 2) if 1

nT n

T n n n

2

2

3

lg

( ) 2 / 2 substitute

2 2 / 4 / 2 expand

2 ( / 4) 2 substitute

2 (2 ( /8) / 4) 2 expand

2 ( /8) 3 observe the pattern

( ) 2 ( / 2 )

2 ( / ) lg lg

i i

n

T n T n n

T n n n

T n n

T n n n

T n n

T n T n in

T n n n n n n n

September 22, 2003

41

Example: Finding Min and Max

Given an unsorted array, find a minimum and a maximum element in the array

INPUT: A[l..r] – an unsorted array of integers, l r. OUTPUT: (min, max) such that j (ljr): A[j] min and A[j] max MinMax(A, l, r): if l = r then return (A[l], A[r]) Trivial case q(l+r)/2Divide (minl, maxl) MinMax(A, l, q) (minr, maxr) MinMax(A, q+1, r) if minl < minr then min = minl else min = minr if maxl > maxr then max = maxl else max = maxr return (min, max)

INPUT: A[l..r] – an unsorted array of integers, l r. OUTPUT: (min, max) such that j (ljr): A[j] min and A[j] max MinMax(A, l, r): if l = r then return (A[l], A[r]) Trivial case q(l+r)/2Divide (minl, maxl) MinMax(A, l, q) (minr, maxr) MinMax(A, q+1, r) if minl < minr then min = minl else min = minr if maxl > maxr then max = maxl else max = maxr return (min, max)

Conquer

Combine

September 22, 2003

42

Next Week

Analyzing the running time of recursive algorithms (such as divide-and-conquer)