Methods in Computational Linguistics II

Queens College

Lecture 8: Dynamic Programming

2

Today

• Inheritance• Recursion• Dynamic Programming

• Machine Learning Primer

3

Inheritance

• Objects can define relationships between objects.

• Membership operations allow for the representation of “has-a”, “has-many”, and arbitrary property relationships.

• Inheritance relationships allow for the representation of “is-a” relationships.

4

Subclasses and Inheritance

class Base(object): def __init__(self): print “Base init”

class Derived(Base): def __init__(self): super(Derived, self).__init__()

5

Shape Example

Shape

Rectangle Triangle Circle

Square

6

Trees

• Binary Trees are a commonly used data structure.

• The core element of a tree is a node

• These nodes can contain values, and include pointers to one or more children, that are differentiated as “left” and “right”

7

Tree Example

5

a

xyz15

The

8

Binary Search Trees

• Binary Search trees have the properties– the value (or key), of any node is greater than

the value of its left child– the value of any node is less than the value of

its right child.

9

Binary Search Tree Example

5

2

41

9

What does this have to do with Binary Search?

10

Tree class

• What does a tree class require?

11

Graphs

• Graphs are similar to Trees

• Graphs have:– Nodes– Edges

• Nodes can contain values• Edges connect two notes, and can also

have a value.

12

Graph Example

5

2

41

9

a

a

b

c

cd

13

What does a Graph class require?

14

In class coding

• Using an object in a program• Initializing data in an object• Objects that you can iterate over

• Look at the Shape class• Look at the Tree class

15

Recursion

• A function that calls itself is called a recursive function.

• A good recursive function must include – A stopping condition– Modification

16

Example of Recursion

• Print every element of a tree.

• Search for an entry in a tree.

• Print every element of a graph.

• Search in a graph.

17

Dynamic Programming

• Fibonacci Numbers.

• F(x) = F(x-1) + F(x-2)• F(2) = 1• F(1) = 1

18

Recursive Fibonacci

• F(6) = F(5) + F(4)• F(5) = F(4) + F(3)• F(4) = F(3) + F(2)• F(3) = F(2) + F(1)• F(4) = F(3) + F(2)• F(3) = F(2) + F(1)• F(3) = F(2) + F(1)• F(4) = F(3) + F(2)• F(3) = F(2) + F(1)

19

Or do it the smart way

• F(1) = 1• F(2) = 1• F(3) = F(2) + F(1) = 2• F(4) = F(3) + F(2) = 3• F(5) = F(4) + F(3) = 5• F(6) = F(5) + F(4) = 8

20

Dynamic Programming

• Optimal Substructure• Repeated Subproblems

21

Viterbi Decoding

• We know how likely we are to be in a state.

• We know how likely we are to transition from one state to another.

• Find the best state sequence.

22

Lattice Every node is is labeled with P(x|k)

Every edge is labeled with a transition probability aij

23

Next Time

• Machine Learning– A primer on machine learning and its role in

Computational Linguistics.– Classification in NLTK (after break)