compiler construction winter 2020 recitation 11: bottom-up … · 2020. 12. 31. · compiler...
TRANSCRIPT
![Page 1: Compiler Construction Winter 2020 Recitation 11: Bottom-Up … · 2020. 12. 31. · Compiler Construction Winter 2020 Recitation 11: Bottom-Up Parsing Conflicts Yotam Feldman Based](https://reader033.vdocuments.net/reader033/viewer/2022060923/60aec7975652b4796a4b79c6/html5/thumbnails/1.jpg)
Compiler ConstructionWinter 2020
Recitation 11:Bottom-Up Parsing Conflicts
Yotam Feldman
Based on slides by the Technion compilers class’ staff and Guy Golan-Gueta
![Page 2: Compiler Construction Winter 2020 Recitation 11: Bottom-Up … · 2020. 12. 31. · Compiler Construction Winter 2020 Recitation 11: Bottom-Up Parsing Conflicts Yotam Feldman Based](https://reader033.vdocuments.net/reader033/viewer/2022060923/60aec7975652b4796a4b79c6/html5/thumbnails/2.jpg)
Lexing & Parsing
Lexical analysis
ParsingJava code
Abstract Syntax Tree
(AST)
Token
sequence
Today
Shift/reduce conflicts
Reduce/reduce conflicts
Precedence
Associativity
![Page 3: Compiler Construction Winter 2020 Recitation 11: Bottom-Up … · 2020. 12. 31. · Compiler Construction Winter 2020 Recitation 11: Bottom-Up Parsing Conflicts Yotam Feldman Based](https://reader033.vdocuments.net/reader033/viewer/2022060923/60aec7975652b4796a4b79c6/html5/thumbnails/3.jpg)
)id(x)*num(7)(+num(5)token stream
Parser
Grammar:E id
E num
E E + E
E E - E
E E * E
E E / E
E - E
E ( E )
+
num(5)
num(7) id(x)
*
Abstract syntax tree
validsyntaxerror
Parsing
num(5)
E
E E+
E * E
( E )
num(7) id(x)
parse tree
![Page 4: Compiler Construction Winter 2020 Recitation 11: Bottom-Up … · 2020. 12. 31. · Compiler Construction Winter 2020 Recitation 11: Bottom-Up Parsing Conflicts Yotam Feldman Based](https://reader033.vdocuments.net/reader033/viewer/2022060923/60aec7975652b4796a4b79c6/html5/thumbnails/4.jpg)
CUP• Constructor of Useful Parsers
• Automatic LALR(1) parser generator
• Input
– Parser specification file
• Output
– Parser written in Java
CUP javacMiniJava.cup Parser
token sequence
AST
Parser.java
![Page 5: Compiler Construction Winter 2020 Recitation 11: Bottom-Up … · 2020. 12. 31. · Compiler Construction Winter 2020 Recitation 11: Bottom-Up Parsing Conflicts Yotam Feldman Based](https://reader033.vdocuments.net/reader033/viewer/2022060923/60aec7975652b4796a4b79c6/html5/thumbnails/5.jpg)
Grammar in CUP
start with expr;
expr ::= expr PLUS expr
| expr MINUS expr
| expr MULT expr
| expr DIV expr
| MINUS expr
| LPAREN expr RPAREN
| NUMBER;
terminal int NUMBER;
terminal PLUS, MINUS, MULT, DIV;
terminal LPAREN, RPAREN;
non terminal expr;
![Page 6: Compiler Construction Winter 2020 Recitation 11: Bottom-Up … · 2020. 12. 31. · Compiler Construction Winter 2020 Recitation 11: Bottom-Up Parsing Conflicts Yotam Feldman Based](https://reader033.vdocuments.net/reader033/viewer/2022060923/60aec7975652b4796a4b79c6/html5/thumbnails/6.jpg)
Lo and Behold
• See how it so majestically compiles!
• But…
![Page 7: Compiler Construction Winter 2020 Recitation 11: Bottom-Up … · 2020. 12. 31. · Compiler Construction Winter 2020 Recitation 11: Bottom-Up Parsing Conflicts Yotam Feldman Based](https://reader033.vdocuments.net/reader033/viewer/2022060923/60aec7975652b4796a4b79c6/html5/thumbnails/7.jpg)
Ambiguity• A grammar is ambiguous
if there exists a string that has two different rightmost derivations
• (A property of the grammar, not necessarily the language)
• Solutions:
– Changing the grammar
– Specifying precedence
a * b + c
a b c
+
*
a b c
*
+
a + b + c
a b c
+
+
a b c
+
+
![Page 8: Compiler Construction Winter 2020 Recitation 11: Bottom-Up … · 2020. 12. 31. · Compiler Construction Winter 2020 Recitation 11: Bottom-Up Parsing Conflicts Yotam Feldman Based](https://reader033.vdocuments.net/reader033/viewer/2022060923/60aec7975652b4796a4b79c6/html5/thumbnails/8.jpg)
LR: Informal Recap
• Start with the final word, work way up to the starting symbol– Replacing a word with a non-terminal deriving it
• Read input from left to right• Finding rightmost derivation, in the inverse order• Stack of the derived word so far, automaton for deciding how to
proceed• Reduce: A prefix in the top of the stack matches the rhs of a
derivation rule. Replace it with the lhs of that rule.• Shift: A prefix in the top of the stack + lookahead is a prefix of the
rhs of a derivation rule. Push the lookahead symbol to the stack
![Page 9: Compiler Construction Winter 2020 Recitation 11: Bottom-Up … · 2020. 12. 31. · Compiler Construction Winter 2020 Recitation 11: Bottom-Up Parsing Conflicts Yotam Feldman Based](https://reader033.vdocuments.net/reader033/viewer/2022060923/60aec7975652b4796a4b79c6/html5/thumbnails/9.jpg)
LR Parsing
|3 + ( 4 )
input stack
3 | + ( 4 )
3 + | ( 4 )
3 + ( | 4 )
operation
shift
shift
shift
shift
3 + ( 4 | ) shift
3 + ( 4 ) | reduce
parse tree
3 + ( 4 ) | (
E
n )
n
n+
n+(
n+(n
n+(n)
n+E
lookahead
n
+
(
n
)
n
E
+
reduce
3 + ( 4 ) | E
![Page 10: Compiler Construction Winter 2020 Recitation 11: Bottom-Up … · 2020. 12. 31. · Compiler Construction Winter 2020 Recitation 11: Bottom-Up Parsing Conflicts Yotam Feldman Based](https://reader033.vdocuments.net/reader033/viewer/2022060923/60aec7975652b4796a4b79c6/html5/thumbnails/10.jpg)
LR Parsing
|( 4 ) + 3
input stack
( |4 ) + 3
( 4 |) + 3
( 4 ) |+ 3
operation
shift
shift
shift
reduce
( 4 ) |+ 3 shift
( 4 ) + |3 shift
parse tree
( 4 ) + 3 |
(
E
n )
(
(n
(n)
E
E+
E+n
lookahead
(
n
)
+
+
n
E
+
reduce
( 4 ) + 3 | E
n
![Page 11: Compiler Construction Winter 2020 Recitation 11: Bottom-Up … · 2020. 12. 31. · Compiler Construction Winter 2020 Recitation 11: Bottom-Up Parsing Conflicts Yotam Feldman Based](https://reader033.vdocuments.net/reader033/viewer/2022060923/60aec7975652b4796a4b79c6/html5/thumbnails/11.jpg)
Shift/Reduce Conflicts
|3 + 4 * 8
input stack
3 | + 4 * 8
3 + | 4 * 8
3 + 4 | * 8
operation
shift
shift
shift
??
parse tree
n
n+
n+n
lookahead
n
+
n
*
![Page 12: Compiler Construction Winter 2020 Recitation 11: Bottom-Up … · 2020. 12. 31. · Compiler Construction Winter 2020 Recitation 11: Bottom-Up Parsing Conflicts Yotam Feldman Based](https://reader033.vdocuments.net/reader033/viewer/2022060923/60aec7975652b4796a4b79c6/html5/thumbnails/12.jpg)
Shift/Reduce Conflicts
|3 + 4 * 8
input stack
3 | + 4 * 8
3 + | 4 * 8
3 + 4 | * 8
operation
shift
shift
shift
reduce
parse tree
n
n+
n+n
E
lookahead
n
+
n
*
*3 + 4 | * 8n
E
+ n
E
* n
Top of the stack matches the rhs of a derivation rule
and the lookahead is possible in a derivation where this rule is applied
![Page 13: Compiler Construction Winter 2020 Recitation 11: Bottom-Up … · 2020. 12. 31. · Compiler Construction Winter 2020 Recitation 11: Bottom-Up Parsing Conflicts Yotam Feldman Based](https://reader033.vdocuments.net/reader033/viewer/2022060923/60aec7975652b4796a4b79c6/html5/thumbnails/13.jpg)
Shift/Reduce Conflicts
|3 + 4 * 8
input stack
3 | + 4 * 8
3 + | 4 * 8
3 + 4 | * 8
operation
shift
shift
shift
shift
3 + 4 * | 8
parse tree
n
n+
n+n
n+n*
lookahead
n
+
n
*
n
n
E
* n
n
E
+
Top of the stack + lookahead is a prefix of the rhs of a derivation rule
![Page 14: Compiler Construction Winter 2020 Recitation 11: Bottom-Up … · 2020. 12. 31. · Compiler Construction Winter 2020 Recitation 11: Bottom-Up Parsing Conflicts Yotam Feldman Based](https://reader033.vdocuments.net/reader033/viewer/2022060923/60aec7975652b4796a4b79c6/html5/thumbnails/14.jpg)
Shift/Reduce Conflicts
![Page 15: Compiler Construction Winter 2020 Recitation 11: Bottom-Up … · 2020. 12. 31. · Compiler Construction Winter 2020 Recitation 11: Bottom-Up Parsing Conflicts Yotam Feldman Based](https://reader033.vdocuments.net/reader033/viewer/2022060923/60aec7975652b4796a4b79c6/html5/thumbnails/15.jpg)
Resolving Ambiguity: Precedence
3+4*8
3 4 8
*
+
3 4 8
+
*
precedence ?? PLUS
precedence ?? MULT
Increasing
precedence
![Page 16: Compiler Construction Winter 2020 Recitation 11: Bottom-Up … · 2020. 12. 31. · Compiler Construction Winter 2020 Recitation 11: Bottom-Up Parsing Conflicts Yotam Feldman Based](https://reader033.vdocuments.net/reader033/viewer/2022060923/60aec7975652b4796a4b79c6/html5/thumbnails/16.jpg)
Precedenceterminal int NUMBER;
terminal PLUS, MINUS, MULT, DIV;
terminal LPAREN, RPAREN;
precedence PLUS, MINUS;
precedence DIV, MULT;
non terminal expr;
expr ::= expr PLUS expr
| expr MINUS expr
| expr MULT expr
| expr DIV expr
| MINUS expr
| LPAREN expr RPAREN
| NUMBER
Increasing
precedence
“When there is a shift/reduce conflict, the parser determines whether the terminal to be shifted has a higher precedence, or if the production to reduce by does. If the terminal has higher precedence, it is shifted”
Rule has precedence of
PLUS
![Page 17: Compiler Construction Winter 2020 Recitation 11: Bottom-Up … · 2020. 12. 31. · Compiler Construction Winter 2020 Recitation 11: Bottom-Up Parsing Conflicts Yotam Feldman Based](https://reader033.vdocuments.net/reader033/viewer/2022060923/60aec7975652b4796a4b79c6/html5/thumbnails/17.jpg)
Resolved via Precedence: Shift
|3 + 4 * 8
input stack
3 | + 4 * 8
3 + | 4 * 8
3 + 4 | * 8
operation
shift
shift
shift
shift
3 + 4 * | 8 shift
3 + 4 * 8 | reduce
parse tree
3 + 4 * 8 |
n
E
* n
n
n+
n+n
n+n*
n+n*n
n+E
lookahead
n
+
n
*
nn
E
+
Reduce: precedence of +Shift: precedence of *
reduce
3 + 4 * 8 | E
![Page 18: Compiler Construction Winter 2020 Recitation 11: Bottom-Up … · 2020. 12. 31. · Compiler Construction Winter 2020 Recitation 11: Bottom-Up Parsing Conflicts Yotam Feldman Based](https://reader033.vdocuments.net/reader033/viewer/2022060923/60aec7975652b4796a4b79c6/html5/thumbnails/18.jpg)
Precedence Beyond Conflict Resolutionterminal int NUMBER;
terminal PLUS, MINUS, MULT, DIV;
terminal LPAREN, RPAREN;
precedence ?? PLUS, MINUS;
precedence ?? DIV, MULT;
non terminal expr;
expr ::= expr PLUS expr
| expr MINUS expr
| expr MULT expr
| expr DIV expr
| MINUS expr
| LPAREN expr RPAREN
| NUMBER
What would happen with- a * b
?
Increasing
precedence
![Page 19: Compiler Construction Winter 2020 Recitation 11: Bottom-Up … · 2020. 12. 31. · Compiler Construction Winter 2020 Recitation 11: Bottom-Up Parsing Conflicts Yotam Feldman Based](https://reader033.vdocuments.net/reader033/viewer/2022060923/60aec7975652b4796a4b79c6/html5/thumbnails/19.jpg)
Precedence Beyond Conflict Resolutionterminal int NUMBER;
terminal PLUS, MINUS, MULT, DIV;
terminal UMINUS;
terminal LPAREN, RPAREN;
precedence ?? PLUS, MINUS;
precedence ?? DIV, MULT;
precedence ?? UMINUS;
non terminal expr;
expr ::= expr PLUS expr
| expr MINUS expr
| expr MULT expr
| expr DIV expr
| MINUS expr %prec UMINUS
| LPAREN expr RPAREN
| NUMBER
Increasing
precedence
Rule has precedence of
UMINUS
UMINUS never returnedby lexer
(used only to define precedence)
(If not specified, of the last terminal in the production rule)
![Page 20: Compiler Construction Winter 2020 Recitation 11: Bottom-Up … · 2020. 12. 31. · Compiler Construction Winter 2020 Recitation 11: Bottom-Up Parsing Conflicts Yotam Feldman Based](https://reader033.vdocuments.net/reader033/viewer/2022060923/60aec7975652b4796a4b79c6/html5/thumbnails/20.jpg)
Resolving Ambiguity: Associativity
precedence ?? PLUS
3+4+8
3 4 8
+
+
3 4 8
+
+
“Associativity rules are also used to resolve shift/reduce conflicts, but only in the case of equal precedence.”
![Page 21: Compiler Construction Winter 2020 Recitation 11: Bottom-Up … · 2020. 12. 31. · Compiler Construction Winter 2020 Recitation 11: Bottom-Up Parsing Conflicts Yotam Feldman Based](https://reader033.vdocuments.net/reader033/viewer/2022060923/60aec7975652b4796a4b79c6/html5/thumbnails/21.jpg)
Shift/Reduce Conflicts: Associativity
|3 + 4 + 8
input stack
3 | + 4 + 8
3 + | 4 + 8
3 + 4 | + 8
operation
shift
shift
shift
??
parse tree
n
n+
n+n
lookahead
n
+
n
+
![Page 22: Compiler Construction Winter 2020 Recitation 11: Bottom-Up … · 2020. 12. 31. · Compiler Construction Winter 2020 Recitation 11: Bottom-Up Parsing Conflicts Yotam Feldman Based](https://reader033.vdocuments.net/reader033/viewer/2022060923/60aec7975652b4796a4b79c6/html5/thumbnails/22.jpg)
|3 + 4 + 8
input stack
3 | + 4 + 8
3 + | 4 + 8
3 + 4 | + 8
operation
shift
shift
shift
reduce
parse tree
n
n+
n+n
E
lookahead
n
+
n
+
+3 + 4 | + 8 n
E
+ n
E
+ n
shift
E+n3 + 4 + | 8 shift
E+E3 + 4 + 8| reduce
E3 + 4 + 8|
precedence left PLUS“If the associativity of the terminal
that can be shifted is left, then a
reduce is performed.
![Page 23: Compiler Construction Winter 2020 Recitation 11: Bottom-Up … · 2020. 12. 31. · Compiler Construction Winter 2020 Recitation 11: Bottom-Up Parsing Conflicts Yotam Feldman Based](https://reader033.vdocuments.net/reader033/viewer/2022060923/60aec7975652b4796a4b79c6/html5/thumbnails/23.jpg)
|3 + 4 + 8
input stack
3 | + 4 + 8
3 + | 4 + 8
3 + 4 | + 8
operation
shift
shift
shift
shift
3 + 4 + | 8 shift
3 + 4 + 8 | reduce
parse tree
3 + 4 + 8 |
n
E
+ n
n
n+
n+n
n+n+
n+n+n
n+E
lookahead
n
+
n
+
nn
E
+
reduce
3 + 4 + 8 | E
precedence right PLUS“If the associativity of the
terminal is right, it is shifted onto the stack. hence, the reductions will take place from right to left.”
![Page 24: Compiler Construction Winter 2020 Recitation 11: Bottom-Up … · 2020. 12. 31. · Compiler Construction Winter 2020 Recitation 11: Bottom-Up Parsing Conflicts Yotam Feldman Based](https://reader033.vdocuments.net/reader033/viewer/2022060923/60aec7975652b4796a4b79c6/html5/thumbnails/24.jpg)
3==4==8
3 4 8
==
==
3 4 8
==
==
precedence nonassoc EQ“If a terminal is declared as
nonassoc, then two consecutive occurrences of equal precedence
non-associative terminals generates an error.”
![Page 25: Compiler Construction Winter 2020 Recitation 11: Bottom-Up … · 2020. 12. 31. · Compiler Construction Winter 2020 Recitation 11: Bottom-Up Parsing Conflicts Yotam Feldman Based](https://reader033.vdocuments.net/reader033/viewer/2022060923/60aec7975652b4796a4b79c6/html5/thumbnails/25.jpg)
Precedence and Associativityterminal int NUMBER;
terminal PLUS, MINUS, MULT, DIV;
terminal UMINUS;
terminal LPAREN, RPAREN;
precedence left PLUS, MINUS;
precedence left DIV, MULT;
precedence left UMINUS;
non terminal expr;
expr ::= expr PLUS expr
| expr MINUS expr
| expr MULT expr
| expr DIV expr
| MINUS expr %prec UMINUS
| LPAREN expr RPAREN
| NUMBER
Increasing
precedence
![Page 26: Compiler Construction Winter 2020 Recitation 11: Bottom-Up … · 2020. 12. 31. · Compiler Construction Winter 2020 Recitation 11: Bottom-Up Parsing Conflicts Yotam Feldman Based](https://reader033.vdocuments.net/reader033/viewer/2022060923/60aec7975652b4796a4b79c6/html5/thumbnails/26.jpg)
Reduce/Reduce Conflicts
|b a a
input
b | a a
operation
shift
reduce 3 or 4??
• 1. S Aaa• 2. S Bab• 3. A b• 4. B b
stack
b
lookahead
b
a
![Page 27: Compiler Construction Winter 2020 Recitation 11: Bottom-Up … · 2020. 12. 31. · Compiler Construction Winter 2020 Recitation 11: Bottom-Up Parsing Conflicts Yotam Feldman Based](https://reader033.vdocuments.net/reader033/viewer/2022060923/60aec7975652b4796a4b79c6/html5/thumbnails/27.jpg)
Debugging CUP
• Getting internal representation– Command line options:
• -dump_grammar
• -dump_states
• -dump_tables
• -dump
– Enabled in the demo’s build.xml CUP ant task
![Page 28: Compiler Construction Winter 2020 Recitation 11: Bottom-Up … · 2020. 12. 31. · Compiler Construction Winter 2020 Recitation 11: Bottom-Up Parsing Conflicts Yotam Feldman Based](https://reader033.vdocuments.net/reader033/viewer/2022060923/60aec7975652b4796a4b79c6/html5/thumbnails/28.jpg)
What With All This Parsing Anyway?
• “Which Parsing Approach?”, September 2020
– “I spent over 20 years assuming that parsing is easy and that I didn’t need to understand it properly in order to use it well. Alas, reality has been a cruel teacher, and in this post I want to share some of the lessons I’ve been forced to slowly learn and acknowledge.”
![Page 29: Compiler Construction Winter 2020 Recitation 11: Bottom-Up … · 2020. 12. 31. · Compiler Construction Winter 2020 Recitation 11: Bottom-Up Parsing Conflicts Yotam Feldman Based](https://reader033.vdocuments.net/reader033/viewer/2022060923/60aec7975652b4796a4b79c6/html5/thumbnails/29.jpg)
Summary
• Bottom-up (LR) parsing
• Shift/reduce conflicts
• Precedence, associativity
• Reduce/reduce conflicts
• Ex 4