1

Week 5• Questions / Concerns• What’s due:

• Lab2 part a due on Sunday • Lab1 check-off by appointment• Test#1

HW#5 next Thursday• Coming up:

• Lab3 Posted.• Discuss grading and homework

• Homework#4• Recursive Descent Parser• Compute First and Follow

2

Revised & Expanded Grammar Example

S -> id = E ;E -> E + T |

E – T |T

T -> T * F |T / F |F

F -> ( E ) |id

S

id(i)

E

E T

T F

= ;

+

*

F

T

id(a)

F

i = a + b * c;

id(b)

id(c)

3

Sample Language• This following language consists of statements in 2 types:

• Function definitions • Function calls

• You don’t have to return values from a function as functions are evaluated to a result value.

• Every function can have up to 2 parameters.(defun add (x y) (+ x y) )(add 10 20)(add 20 20)(set x 10)(negate x)(+ x 10)(print x)(input x)Can you define a grammar for this language?

4

Sample Language Grammar

Program -> Statement StatementsStatements -> Statement Statements | Statement -> FunctionDef | FunctionCallFunctionCall -> (ID Param Param ) | (OP Param Param ) | (BUILTIN Param Param)Param -> ID | Num | BUILTIN -> setx | negate | printOP -> + | - | * | / FunctionDef -> (defun ID (FParam FParam) Statements)FParam -> ID |

5

HW#4• Unit Production

• S-> aX | Yb• X -> S remove X-> S, don’t change anything else• Y -> …. X-> aX | Yb

• Left FactoringS-> a… | a… | a… => S -> aX

6

HW#4• Left Recursion & Unit production

• Remove Left Recursion first• If there are any unit productions left, take care of them afterwards

S-> A | B | Sb | aSb | c

After removal: X -> bX | S -> AX | BX | aSbX | cX

A-> Ab | Bc | Sb |

After removal: Y -> bY | A -> BcY | SbY | Y

A -> BcY | SbY | bY |

Unit production

7

Recursive Descent Parser

A -> aAbb |bB |Bcd

bool A(){ if curToken == a //match aAbb if (A()) if (nextToken == b) if (nextToken == b) return true; else if curToken == b //match bB if (B()) return true; else if (B()) //trying to match Bcd

8

Recursive Descent Parser

A -> aAbb |bB |Bcd

bool A(){ if curToken == a //match aAbb if (A()) if (nextToken == b) if (nextToken == b) return true; else if curToken == b //match bB if (B()) return true; else if (B()) what if B starts with a or b?

9

Recursive Descent Parser

A -> aAbb |bB |Bcd

bool A(){ if curToken == a //match aAbb if (A()) if (nextToken == b) if (nextToken == b) return true; else if curToken == b //match bB if (B()) return true; else if (B()) You don’t have to remove B because it’s not a unit production but you may need to backtrack here if B starts with a or b also

10

Recursive Descent Parser• If possible, try to remove non-terminals as the first symbol

even though they are not unit productions. It reduces some backtracking.

• Example:

• E -> TX E -> idYX | (E)YX• T -> FY T -> idY | (E)Y• F -> id | ( E )

11

Recursive Descent Parser• (Rules without lambdas)

• The rule will return false if• None of the first tokens from the rule matches the current token.

S -> aSb | bS | cd

The current token is not a, b or c

• The first token matches but rest of the rule fails

The input string for above grammar is “cc”.

12

Recursive Descent Parser• (Rules with lambdas)

• What if:• None of the first tokens from the rule matches the current token.

S -> aSb | bS | cd |

The current token is not a, b or c.So the lambda means that S doesn’t really match any token, so just take lambda and return true. It lets the next rule to worry about the current token.

What if the current token is wrong for the next rule?

• The first token matches but rest of the rule failsThe input string for above grammar is “cc”. But lambda could still mean that “cc” is intended for the next rule and that S should just be lambda. We would have to see what the next rule needs if “cc” is wrong here.

13

Recursive Descent Parser• The parser would be much more accurate and efficient if

we know all the first tokens of a rule regardless if they start with a token or a non-terminal.

S -> a S b | b B | C d

• The parser would handle lambdas more accurately if it knows what tokens should come after the rule if lambda is taken.

S -> a S b aaac…S ->

First set

Follow set

You only need Follow set if you have lambdas

14

LL(1)• Another top-down parser• It’s a table-driven parser.• LL(1)

• L – first L, the input is from left to right • L – second L, leftmost derivation (top-down)• 1 – one token look ahead

• Grammar pre-req:• No left recursion• Unit productions are okay, but should minimize• MUST left factor to ensure one-token look ahead.

• Procedure:• Compute First and Follow sets from the grammar

15

First set• Grammar with no lambda S -> A B

S -> cA -> a bAA -> bB -> d BB -> e

First set is just the first token. If the first symbol is a non-terminal, then include all the first

tokens from that non-terminal.

16

First set• S -> A B

S -> cA -> a bAA -> bB -> d BB -> e

S c, First(A) A a, b

B d, e

S c, a, bA a, b

B d, e

17

First set• What about ?

S -> A BS -> cA -> a bAA -> bA -> B -> d BB -> e

S c, First(A) -

A a, b,

B d, e

S c, a, b

A a, b,

B d, e

Only add to S if S itself also goes to as the result of using that rule

18

First set• What about ?

S -> A BS -> cA -> a bAA -> bA -> B -> d BB -> e

S c, First(A)-, First(B)-

A a, b,

B d, e

S c, a, b, d, e

A a, b,

B d, e

Only add to S if S itself also goes to as the result of using that rule

19

First Set

S -> ABcA -> aA -> B -> bB ->

S

A

B

20

Follow set• If any one grammar rule in your grammar has lambda

productions, then you need to compute the follow set for the entire grammar.

• Follow set is just a set of tokens that follow a particular non-terminal.

• There are 4 different cases to consider in computing the Follow set. Not all 4 cases are applicable in every case.

21

Follow Set• Case 1: If S is the start symbol, then add $ (end of input)

to the Follow set of S.Nothing should follow S after S is done parsing.

• Case 2: If you have a token following a non-terminal on the right hand side of the rule, then add that token to the follow set of the non-terminal.

… -> .. Ab - b follows A. Add b to the Follow set of A.

Note: We don’t care what’s on the left hand side of the rule, just that b follows the non-terminal A

22

Follow Set• Case 3: If you have a non-terminal following a non-

terminal on the right hand side of the rule, then add that First set of the second non-terminal to the follow set of the first non-terminal.

… -> .. AB - B follows A. Add First(B) - to the Follow set of A.

Note: We don’t care what’s on the left hand side of the rule, just that B follows the non-terminal A.

We don’t have in the Follow set.

23

Follow Set• Case 4: If you have this following pattern:

A -> ….. B

This could be an unit production or just any rule that ends with a non-terminal.

In this case, anything that follows A also follows B

24

Follow Set• Case 4: If you have this following pattern:

A -> ….. B

A

B

Whatever follows A , also follows B because there is nothing else after B on the lower level of the parser tree

25

Follow set: Case 1• S -> A B c

S -> cA -> a BA -> bB -> d BB -> e

S $

A

B

26

Follow set: Case 2• S -> A B c

S -> cA -> a BA -> bB -> d BB -> e

S $

A

B c

27

Follow set: Case 3• S -> A B c

S -> cA -> a BA -> bB -> d BB -> e

S $

A First(B)

B c

S $

A d, e

B c

28

Follow set: Case 4• S -> A B c

S -> cA -> a BA -> bB -> d BB -> e

S $

A d, e

B c

S $

A d, e

B c, d, e

29

Follow set: Case 1

S -> ABcA -> aA -> B -> bB ->

S $

A

B

30

Follow set: Case 2

S -> ABcA -> aA -> B -> bB ->

S $

A

B c

BUT, B could be a lambda

31

Follow set: Case 2

S -> ABcA -> aA -> B -> bB ->

S $

A c

B c

32

Follow set: Case 3

S -> ABcA -> aA -> B -> bB ->

S $

A c, First(B)

B c

S $

A c, b

B c

No in Follow set

33

Follow set: Case 4

S -> ABcA -> aA -> B -> bB ->

S $

A c, b

B c

No case 4 for this grammar

34

In-Class Exercise #7• Please compute first and follow sets for the following

grammar:

S -> ABCdA -> e | f | B -> g | h | C -> p | q