design and analysis of algorithms – chapter 41 decrease and conquer ii dr. ying lu [email protected]...

Design and Analysis of Algorithms – Chapter 4 1

Decrease and Conquer II

Dr. Ying [email protected]

RAIK 283: Data Structures & RAIK 283: Data Structures & AlgorithmsAlgorithms


Giving credit where credit is due:Giving credit where credit is due:• Most of the lecture notes are based on the slides from Most of the lecture notes are based on the slides from

the Textbook’s companion websitethe Textbook’s companion website

– http://www.aw-bc.com/info/levitinhttp://www.aw-bc.com/info/levitin

• Some examples and slides are based on lecture notes Some examples and slides are based on lecture notes created by Dr. Ben Choi, Louisiana Technical University created by Dr. Ben Choi, Louisiana Technical University and Dr. Chuck Cusack, Hope College and Dr. Chuck Cusack, Hope College

• I have modified many of their slides and added new I have modified many of their slides and added new slides.slides.

RAIK 283: Data Structures & RAIK 283: Data Structures & AlgorithmsAlgorithms


Examples of Decrease and Examples of Decrease and ConquerConquer

Decrease by one:Decrease by one:• Insertion sortInsertion sort• Graph search algorithms:Graph search algorithms:

– DFSDFS– BFSBFS– Topological sortingTopological sorting

• Algorithms for generating permutations, subsetsAlgorithms for generating permutations, subsets

Decrease by a constant factorDecrease by a constant factor• Binary search Binary search • Fake-coin problemsFake-coin problems• multiplication à la russe multiplication à la russe • Josephus problemJosephus problem

Variable-size decreaseVariable-size decrease• Euclid’s algorithmEuclid’s algorithm• Selection by partitionSelection by partition• BSTBST


Fake coin problemFake coin problem

You have 8 coins and a two-pan balance. All the You have 8 coins and a two-pan balance. All the coins weigh the same, except for one, which is coins weigh the same, except for one, which is lighter than all the others. The coins are otherwise lighter than all the others. The coins are otherwise indistinguishable. You may make no assumption indistinguishable. You may make no assumption about how much lighter the fake coin is. What is about how much lighter the fake coin is. What is the minimum number of weighings needed to be the minimum number of weighings needed to be certain of identifying the fake coin?certain of identifying the fake coin?

How about identifying one light fake coin out of n How about identifying one light fake coin out of n coins?coins?


Examples of Decrease and Examples of Decrease and ConquerConquer

Decrease by one:Decrease by one:• Insertion sortInsertion sort• Graph search algorithms:Graph search algorithms:

– DFSDFS– BFSBFS– Topological sortingTopological sorting

• Algorithms for generating permutations, subsetsAlgorithms for generating permutations, subsets

Decrease by a constant factorDecrease by a constant factor• Binary search Binary search • Fake-coin problemsFake-coin problems• multiplication à la russe (read textbook yourself)multiplication à la russe (read textbook yourself)• Josephus problem (read textbook yourself)Josephus problem (read textbook yourself)

Variable-size decreaseVariable-size decrease• Euclid’s algorithmEuclid’s algorithm• Selection by partitionSelection by partition• BST (slides provided for reviewing yourself)BST (slides provided for reviewing yourself)


The selection problemThe selection problem

Input:Input: A set S of n elements A set S of n elements

Output:Output: The k The kthth smallest element of S smallest element of S





To find the To find the median (the middle value)median (the middle value)

To find the smallest elementTo find the smallest element

To find the largest elementTo find the largest element

K =

2

n

K = 1

K = n





The straightforward algorithm: The straightforward algorithm: • step 1:step 1: Sort the n elements Sort the n elements

• step 2:step 2: Locate the k Locate the kth th element in the sorted list.element in the sorted list.

• Time complexity: O(nlogn)Time complexity: O(nlogn)

This algorithm is an overkill!This algorithm is an overkill!


Partition problemPartition problem

Select a Select a pivotpivot (partitioning element), say the left most (partitioning element), say the left most elementelement

Rearrange the list so that all the elements in the positions Rearrange the list so that all the elements in the positions before the pivot are smaller than or equal to the pivot and before the pivot are smaller than or equal to the pivot and those after the pivot are larger or equal to the pivotthose after the pivot are larger or equal to the pivot

p

A[i]≤p A[i]p


Partition problemPartition problem

Select a Select a pivotpivot (partitioning element), say the left most (partitioning element), say the left most elementelement

Rearrange the list so that all the elements in the positions Rearrange the list so that all the elements in the positions before the pivot are smaller than or equal to the pivot and before the pivot are smaller than or equal to the pivot and those after the pivot are larger or equal to the pivotthose after the pivot are larger or equal to the pivot

Design a linear O(n) partition algorithmDesign a linear O(n) partition algorithm Work on problem instance: 7, 8, 10, 2, 9, 6, 5, 4, 11, 13, 0 to Work on problem instance: 7, 8, 10, 2, 9, 6, 5, 4, 11, 13, 0 to

help form the ideahelp form the idea

p

A[i]≤p A[i]p

Partition ExamplePartition Example

• 7, 8, 10, 2, 9, 6, 5, 4, 11, 13, 07, 8, 10, 2, 9, 6, 5, 4, 11, 13, 0

Partition algorithm (I)Partition algorithm (I)



Partition algorithm (II)Partition algorithm (II)


Partition algorithm (II)Partition algorithm (II)

A “sentinel” at A[n] to prevent i advances beyond position n. A “sentinel” at A[n] to prevent i advances beyond position n.


Partition algorithmPartition algorithm

Select a Select a pivotpivot (partitioning element) (partitioning element) Rearrange the list so that all the elements in the positions Rearrange the list so that all the elements in the positions

before the pivot are smaller than or equal to the pivot and before the pivot are smaller than or equal to the pivot and those after the pivot are larger or equal to the pivotthose after the pivot are larger or equal to the pivot

How to solve the selection problem based on the partition How to solve the selection problem based on the partition algorithm?algorithm?

p

A[i]≤p A[i]p


Selection by partitionSelection by partition

Select a Select a pivotpivot (partitioning element) (partitioning element) Partition the list using the pivotPartition the list using the pivot If pivot is in the kIf pivot is in the kthth position, the pivot is the element, done! position, the pivot is the element, done! If pivot’s position is > k, repeat the above process with the If pivot’s position is > k, repeat the above process with the

first sublistfirst sublist If pivot’s position is < k, repeat the above process with the If pivot’s position is < k, repeat the above process with the

second sublistsecond sublist

p

A[i]≤p A[i]>p

Example:Example:

Find the median of the following list of nine Find the median of the following list of nine elements: 4, 1, 10, 9, 7, 12, 8, 2, 15. elements: 4, 1, 10, 9, 7, 12, 8, 2, 15.



Selection by partitionSelection by partition

Select a Select a pivotpivot (partitioning element) (partitioning element) Partition the list using the pivotPartition the list using the pivot If pivot is in the kIf pivot is in the kthth position, the pivot is the element, done! position, the pivot is the element, done! If pivot’s position is > k, repeat the above process with the If pivot’s position is > k, repeat the above process with the

first sublistfirst sublist If pivot’s position is < k, repeat the above process with the If pivot’s position is < k, repeat the above process with the

second sublistsecond sublist

Time efficiencyTime efficiency

p

A[i]≤p A[i]>p


Binary Search Trees (BSTs)Binary Search Trees (BSTs)


Binary Search Trees (BSTs)Binary Search Trees (BSTs)

Binary Search Tree propertyBinary Search Tree property• A binary tree in which the key of an internal node is A binary tree in which the key of an internal node is

greater than the keys in its left subtree and less than or greater than the keys in its left subtree and less than or equal to the keys in its right subtree. equal to the keys in its right subtree.

An inorder traversal of a binary search tree An inorder traversal of a binary search tree produces a sorted list of keys.produces a sorted list of keys.


Variable-size-decrease: Binary Variable-size-decrease: Binary search treessearch trees

Keys are arranged in a binary tree with the Keys are arranged in a binary tree with the binary binary search tree propertysearch tree property::

k

<k k


BST ExamplesBST Examples

Binary Search trees with different degrees of balanceBinary Search trees with different degrees of balance Black dots denote empty treesBlack dots denote empty trees

Size of a search treeSize of a search tree


Variable-size-decrease: Binary Variable-size-decrease: Binary search treessearch trees

Arrange keys in a binary tree with the Arrange keys in a binary tree with the binary binary search tree propertysearch tree property::

k

<k k

Example 1:Example 1: 5, 10, 3, 1, 7, 12, 9

Example 2:Example 2: 4, 5, 7, 2, 1, 3, 6


BST OperationsBST Operations

Find the min/max element (Find the min/max element ()) Search for an elementSearch for an element Find the successor/predecessor of an elementFind the successor/predecessor of an element Delete an elementDelete an element


BST: Min/MaxBST: Min/Max

The minimum element is the left-most nodeThe minimum element is the left-most nodex is a non-empty BST

The maximum element is the right-most nodeThe maximum element is the right-most node


BST: Min/MaxBST: Min/Max

The minimum element is the left-most nodeThe minimum element is the left-most nodex is a non-empty BST

The maximum element is the right-most nodeThe maximum element is the right-most node

(Time efficiency )(Time efficiency )



Find the min/max elementFind the min/max element Search for an element (Search for an element ()) Find the successor/predecessor of an elementFind the successor/predecessor of an element Delete an elementDelete an element


BST Search (Retrieval)BST Search (Retrieval)

Element bstSearch(BinTree bst, Key K)Element bstSearch(BinTree bst, Key K)

1.1. Element foundElement found

2.2. if (bst == nil)if (bst == nil)

3.3. found = null;found = null;

4.4. elseelse

5.5. Element root = root(bst);Element root = root(bst);

6.6. if (K == root.key)if (K == root.key)

7.7. found = root;found = root;

8.8. else if (K < root.key)else if (K < root.key)

9.9. found = bstSearch (leftSubtree(bst), K);found = bstSearch (leftSubtree(bst), K);

10.10. else else

11.11. found = bstSearch(rightSubtree(bst), K);found = bstSearch(rightSubtree(bst), K);

12.12. return found;return found;

(Time efficiency )(Time efficiency )



Find the min/max elementFind the min/max element Search for an elementSearch for an element Find the successor/predecessor of an element (Find the successor/predecessor of an element ()) Delete an elementDelete an element


BST: Successor/PredecessorBST: Successor/Predecessor

Finding the successor of a node Finding the successor of a node xx (if it exists): (if it exists):• If If xx has a nonempty right subtree, then successor(x) is has a nonempty right subtree, then successor(x) is

the smallest element in the tree root at rightSubtree(x)the smallest element in the tree root at rightSubtree(x)



Finding the successor of a node Finding the successor of a node xx (if it exists): (if it exists):• If rightSubtree(If rightSubtree(xx) is empty, then successor() is empty, then successor(xx) is the lowest ) is the lowest

ancestor of ancestor of xx whose left child is also an ancestor of whose left child is also an ancestor of xx..



The predecessor operation is symmetric to The predecessor operation is symmetric to successor.successor.


Why binary search tree?Why binary search tree?

Array: 1 3 4 5 7 8 9 10 13 17 21 23

BST:


BST: advantageBST: advantage

The advantage to the binary search tree approach The advantage to the binary search tree approach is that it combines the advantage of an array--the is that it combines the advantage of an array--the ability to do a binary search with the advantage of ability to do a binary search with the advantage of a linked list--its dynamic size. a linked list--its dynamic size.



Find the min/max elementFind the min/max element Search for an elementSearch for an element Find the successor/predecessor of an elementFind the successor/predecessor of an element Delete an element (Delete an element ())


BST: DeleteBST: Delete

Deleting a node Deleting a node zz is by far the most difficult BST is by far the most difficult BST operation.operation.

There are three cases to considerThere are three cases to consider• If If zz has no children, just delete it. has no children, just delete it.

• If If zz has one child, splice out has one child, splice out zz, That is, link , That is, link zz’s parent ’s parent and childand child

• If If zz has two children, splice out has two children, splice out zz’s successor ’s successor yy, and , and replace the contents of replace the contents of zz with the contents of with the contents of yy

The last case works because if z has two children, The last case works because if z has two children, then its successor has no left childthen its successor has no left child


BST: splice out examplesBST: splice out examples


BST: splice out algorithmBST: splice out algorithm

Only works when a node has at most one child


BST: Deletion AlgorithmBST: Deletion Algorithm

Delete is now simple!Delete is now simple!


BST: one more delete exampleBST: one more delete example


Analysis of BST OperationsAnalysis of BST Operations

All of the BST operations have time complexity All of the BST operations have time complexity ((hh), ), where where hh is the height of the tree is the height of the tree

However, in the worst-case, the height may be However, in the worst-case, the height may be ((nn) ) where where nn is the number of internal nodes is the number of internal nodes• For example, a long chain tree structureFor example, a long chain tree structure

For a tree as balanced as possible, For a tree as balanced as possible, (log(log22 nn)) The objective is to make the tree as balanced as The objective is to make the tree as balanced as

possiblepossible• Technique: Binary Tree Rotations Technique: Binary Tree Rotations

Balanced search tree: AVL tree and 2-3 treeBalanced search tree: AVL tree and 2-3 tree

design and analysis of algorithms – chapter 41 decrease and conquer ii dr. ying lu [email protected]...

Documents

analysis of algorithms

fake coin problem b

subsets b decrease

selection problem b

smallest element b

data structures algorithms

examples of decrease

middle value b