Chapter Five

Recursion and Trees Recursion Divide and conquer Dynamic programming Trees and tree traversals Graphs and graph traversals

Please read chapter 5.

Recursive algorithms What’s the runtime of this algorithm?

int puzzle(int N)

{

if (N == 1) return 1;

if (N % 2 == 0) return puzzle(N/2);

else return puzzle(3 * n + 1);

}

Why is it hard to determine the runtime?

Recursive algorithms need base case and recursive call(s) (preferably smaller!)

Euclid’s algorithm Suppose you write a first-class Fraction class Need to reduce fractions, such as 546/3185 Note that gcd(m,n) = gcd(n,m%n)

gcd(3185,546) gcd(546,455) gcd(455,91) gcd(91,0) return 91

return 91 return 91 return 91

code? int gcd(int m, int n) { if (n == 0) return m; return gcd(n, m%n);}

Fractals What is a fractal? self-similarity at a different level

Koch Snowflake: how to draw?

2D Fractal Star…Fractal flowers…

/kochR { 2 copy ge {dup 0 rlineto } { 3 dif 2 copy kockR 60 rotate 2 copy kochR -120 rotate 2 copy kochR 60 rotate 2 copy kochR } ifelse pop pop} def

0 0 moveto27 81 kochR0 27 moveto9 81 kochR0 54 moveto3 81 kochR0 81 moveto1 81 kochRstroke

Divide and Conquer Split a problem into smaller subproblems Solve the subproblems recursively Merge the subproblem solutions into a

solution to the whole problem

The great point is to learn to solve a problem interms of subproblem solutions.

Divide and Conquer Examples Merge sort Binary Search Towers of Hanoi Parsing HTML, XML pow(x,n) log(n,base) log*(n) Point-in-convex-polygon

Min & Max Given a set of n items, find the min and max. How

many comparisons needed? Simple alg: n-1 comparisons to find min, n-1 for max Recursive algorithm:

void minmax(int i, int j, int& min0, int& max0) if (j<=i+1) [handle base case; return] m=(i+j+1)/2; minmax(i,m-1,min1,max1); minmax(m,j,min2,max2); min0 = min(min1,min2); max0 = max(max1,max2);

Recurrence:T(n)=2T(n/2) + 2; T(2)=1

Solution? 1.5 n - 2

Multiplying How long does it take to multiply two n-bit numbers? Grade school algorithm:

(a 2n/2 + b)(c 2n/2 + d)

ac 2n + (ad + bc)2n/2 + bd

T(n) = 4T(n/2) + c n O(n2)

Improvement:

(a+b)(c+d) = ac + ad + bc + bd

T(n)=3T(n/2) + c n O(nlg 3)

Runtime? O(n2)

Recursive algorithm:How to solve in terms of subproblem solutions?

Recurrence?

Fibonacci Numbers What is the Golden ratio? Can approximate with Fibonacci numbers. Mi to km conversion Pineapple raster, pine cone pattern… Recursive Fibonacci algorithm? Runtime Recurrence? T(1)=1; T(2)=1; T(n) = T(n-1) + T(n-2) + 1

Runtime? not sure…but at least (2n/2)

How to improve runtime? compute bottom-up or just remember values you’ve already computed

Dynamic Programming Solve by combining solutions to overlapping

subproblems Runtime of recursive solution is typically

exponential Make the algorithm efficient:

built a table bottom-up or memoize

DP Example: Making Change Making change

Available coin values: v1, v2, v3, …, vk

How to make change totalling C with fewest coins? Base cases: mincoins(0)=0; mincoins(v1)=1, etc. Recursive step:

mincoins(n)=min[ 1+mincoins(n-v1),

1+mincoins(n-v2), …, 1+mincoins(n-vk) ]

Runtime recurrence? How to make this algorithm efficient?

table-based approach memoization

Algorithm design byDynamic Programming1. Identify subproblem solutions of interest.

Name them.

2. Express solution to whole problem recursively,in terms of subproblem solutions

3. Memoize (or compute bottom-up)

coin values v1, v2, …, vn

mincoins(k)=smallest number of coins to totaling k in value

mincoins(k) = 1 if k = vi, infinity if k < 1mincoins(k) = min[mincoins(k-v1), …, mincoins(k-vn) ] + 1

jot down subproblem solutions as you compute them

DP Example: Weird Dice Given several weird dice. Roll probabilities?

Dice have s1, s2, s3, …, sn sides Count the number of ways you can roll each roll;

divide by sk

Base case: rolling die number 1. What about 2 dice, 3, …? Subproblem solutions of interest? Rolls(n,i) = number of ways of rolling n with first i dice =Rolls(n-1, i-1) + Rolls(n-2,i-1) + … + Rolls(n-

sk,i-1) Rolls(i,1) = 1, i=1..si

Runtime?

O( sk)= O(n) if sk <= const.

Knapsack You are robbing a bank vault. How to maximize

take? Items 1, 2, …, n have values v1, v2, …, vn and integer

weights w1, w2, …, wn How to maximize take, if you can only carry B lbs? Subproblems of interest:

maxval(w,k) = best value with weight w, objects a subset of 1..k

Recursive solution?maxval(w,k) =

max[maxval(w-1,k),maxval(w,k-1), maxval(w-wk,k-1)+vk]

maxval(w,0) = 0want maxval(B,n)

memoize or compute bottom up

Other Dynamic Programming applications Not-one strategies Shortest path Paragraph wrapping Matrix chain multiplication Polygon triangulation lots of optimization problems…

Graphs and Trees Graph:

a set V of vertices (or nodes) and a set E of edges, a subset of VxV

Digraph:a graph with directed edges

Tree:a connected, acyclic graph with one node designated as

root Sibling, parent, ancestor, descendent External node:

“dummy” node or null pointer Leaf:

a node with only external nodes as children

Graphs

Binary Trees Path:

a list of connected vertices Height of a rooted tree:

the length of the longest path from the root to a leaf Binary tree:

a tree where each node has up to 2 children, called “left” and “right”

Complete binary tree:a tree where all the leaves are at the same level

Binary search tree (BST):a binary tree where the key of any node is greater

than the keys of the nodes in its left subtree and less than those of its right subtree.

Tree Properties and Traversals How many edges in a tree of n nodes?

n-1 How many external nodes in a tree of n

nodes?n+1

Height of a binary tree with n nodes?between lg n and n-1

How can I visit every node of a tree? Traversals:

preorder inorder postorder

Parse Trees How would one parse an arithmetic

expression? Parse tree:

a node for each operator leaves for operands

Parse tree traversals: What does inorder traversal give?

infix expression: (3+4)*(5-6) Postorder?

postfix expression: 3 4 + 5 6 – * Preorder?

prefix expression: * + 3 4 – 5 6

Non-recursive Traversals How to implement a traversal non-recursively? What kind of traversal is this:

void traverse(link n, void visit(link)) { STACK<link> s(max); s.push(n); while (!s.empty()) { visit(n = s.pop()); if (n->right != 0) s.push(n->right); if (n->left != 0) s.push(n->left); }}

Graph Traversals How do you find your way

out of a maze, given a large supply of pennies?

Graph traversals Depth-first search Breadth-first search

Other applications: Boggle™, tic-tac-toe, path finding, theorem proving, motion planning, AI, …

Representing a Graph

Two different drawings of the same graph are shown.

What data structure to represent this graph?

Adjacency Matrix A B C D E F G H I J K L M

A 1 1 0 0 1 1 0 0 0 0 0 0

B 0 0 0 0 0 0 0 0 0 0 0

C 0 0 0 0 0 0 0 0 0 0

D 1 1 0 0 0 0 0 0 0

E 1 1 0 0 0 0 0 0

F 0 0 0 0 0 0 0

G 0 0 0 0 0 0

H 1 0 0 0 0

I 0 0 0 0

J 1 1 1

K 0 0

L 1

M

Space required for a graphwith v vertices, e edges?

(e2)

Time to tell if there is anedge from v1 to v2?

(1)

Adjacency ListA: F -> B -> C -> G

B: A

C: A

D: E -> F

E: D -> F -> G

F: D -> E

G: A -> E

H: I

I: H

J: K -> L -> M

K: J

L: M -> J

M: J -> L

Space required for a graph with v vertices, e edges?

(v+e)

Time to tell if there is an edge from v1 to v2?

(v)

Depth-first Search

Main idea: keep traveling to a new, unvisited node until you you get stuck.Then backtrack as far as necessary and try a new

path.

DFS: Recursive How to implement recursively?

void dfs(link n, void visit(link)) { visit(n); visited[n]=1; for (link t=adj[k]; t!=0; t = t->next) if (!visited[t->v]) dfs(t->v, visit); visited[n]=2;}

Runtime? O(e+v) for e edges, v vertices

Space? O(v) – a path may be really long

DFS: Non-recursive How can you implement DFS non-recursively? Use a stack

void dfs(link h, void visit(link)) { STACK<link> s(max); s.push(h); while (!stack.empty()) if (!visited[n = s.pop()]) { visit(n); visited[n]=1; for (link t=adj[n]; t!=0; t=t->next) if (visited[t->v] == 0) s.push(t->v); visited[n]=2; }}

Breadth-first Search Breadth-first search:

Visit all neighbors of the start node(but don’t visit a node you’ve already seen).

Then all neighbors of theirs,and of theirs,

etc.

Time?O(v+e)

Space?O(v)

Breadth-first search How can you implement BFS? Use a queue with the DFS algorithm!

void bfs(link h, void visit(link)) { QUEUE<link> q(max); q.put(h); while (!q.empty()]) if (!visited[n = s.pop()]) { visit(n); visited[n]=1; for (link t=adj[n]; t!=0; t=t->next) if (visited[t->v] == 0) s.push(t->v); visited[n]=2; }}

Aside on AI Will computers ever be intelligent? Really intelligent? Tasks that previously were thought to require

intelligence: adding and subtracting playing chess driving a car recognizing speech or handwriting translating to a foreign language proving mathematical theorems

What does it mean to say that a computer is intelligent? Is that the same as being a person? What is a person? Is a computer program a person? Is a person a computer program?

Achieving “Intelligence” How do AI program achieve “intelligent”

behavior? Currently, three main paradigms:

Symbolic knowledge representation and search Neural Nets Genetic Algorithms

Search in Artificial Intelligence Represent your problem as a graph where nodes are

states and edges are operators that go between states

Define problem states (nodes) Identify start and goal states Define operators (edges) Use DFS or BFS to find goal

Example: Missionaries and cannibals problem states: (3,3,1) 3 missionaries, 3 cannibals, and 1 boat

on left side of river. Operators: one or two people cross the river in the boat,

so that there isn’t a cannibal majority on either side. Goal: get to the other side? Moves? (331)–(220)–(321)–(210)–(221)–(020)–(031)–(010)–(021)–(000)

DFS/BFS Resource Requirements DFS:

Runtime?O(n), n=number of nodes expanded

Space required?

O(d), d = depth of search Can I cut off a search after 5 seconds?

BFS: Runtime? O(n) Space required?

O(breadth of tree) = O(bd), b=branching factor

Can I cut off a search after 5 seconds? Staged DFS: do a DFS of depth 1, 2, 3, … until out of time

Runtime?O(n)

Space required? O(d)

Game Playing We could use DFS but…can’t search whole

tree! limit depth of search and use an evaluation function

We could use DFS but…how do we know which move the opponent will choose?

minimax algorithm: assume the opponent does what looks best.

i.e. at nodes where it is the human’s turn, pick the move that looks best for human. Where computer’s turn, pick the move that looks best for the computer

Mankalah An ancient gamed called Kalah or Mankalah uses stones and

pits: 6 to a side and one on each end. 4 stones are initially placed in each side pit. None are in the

end pits (called Kalahs – a player’s kalah is on her right). A move consists of picking up the stones in a pit and

distributing them, one at a time, in successive pits. If the last stone is placed in your Kalah, you go again If the last stone is placed in an empty pit on your side, you

capture the stones in that pit and the opposite one, on the opponent’s side of the board. These are put into your Kalah.

The game ends when one player has no stones left; the other player puts all the remaining stones on her side into her Kalah.

Whoever ends with more stones in her Kalah wins. See the demo program on holmes at /home/hplantin/kalah.c

Write a smart kalah playing program!

Mankalah minimaxint kalahboard::minimax(depth d): //semi-

pseudocode

if [human won] return –infinity;

if [machine won] return +infinity;

if (d==0) return evaluate();

if (whosemove==HUMAN)

best=+infinity;

for (move=first; move<=last; move++)

kalahboard b=*this; //duplicate board

if (b.board[move]>0)//is move legal?

b.makemove(move);//make the move

v=b.minimax(d-1);//find its value

if (v<best) best=v; //remember if best

else // similarly for MACHINE’s move

return best;