lexical and syntax analysis - university of york › fp › lsa › lectures › slideshow6.pdf ·...
TRANSCRIPT
![Page 1: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/1.jpg)
Lexical and Syntax Analysis(of Programming Languages)
Bison, a Parser Generator
![Page 2: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/2.jpg)
Lexical and Syntax Analysis(of Programming Languages)
Bison, a Parser Generator
![Page 3: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/3.jpg)
Bison: a parser generator
Specificationof a parser
Context-free grammarwith a C action for each
production.
Match the input stringand execute the
actions of the productions used.
Bison
C function called
yyparse()
![Page 4: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/4.jpg)
Bison: a parser generator
Specificationof a parser
Context-free grammarwith a C action for each
production.
Match the input stringand execute the
actions of the productions used.
Bison
C function called
yyparse()
![Page 5: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/5.jpg)
Input to Bison
The structure of a Bison (.y) fileis as follows.
/* Declarations */
%%
/* Grammar rules */
%%
/* C Code (including main function) */
Any text enclosed in /* and */is treated as a comment.
![Page 6: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/6.jpg)
Input to Bison
The structure of a Bison (.y) fileis as follows.
/* Declarations */
%%
/* Grammar rules */
%%
/* C Code (including main function) */
Any text enclosed in /* and */is treated as a comment.
![Page 7: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/7.jpg)
Grammar rules
Let α be any sequence ofterminals and non-terminals. Agrammar rule defining non-terminal n is of the form:
n : α1 action1
| α2 action2
| ⋯| αn actionn
;
Each action is a C statement, ora block of C statements of theform {⋯ }.
![Page 8: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/8.jpg)
Grammar rules
Let α be any sequence ofterminals and non-terminals. Agrammar rule defining non-terminal n is of the form:
n : α1 action1
| α2 action2
| ⋯| αn actionn
;
Each action is a C statement, ora block of C statements of theform {⋯ }.
![Page 9: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/9.jpg)
Example 1
/* No declarations */
%%
e : 'x'| 'y'| '(' e '+' e ')'| '(' e '*' e ')'
%%
/* No main function */
expr1.y
/* No actions */
Non-terminal
Terminal
![Page 10: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/10.jpg)
Example 1
/* No declarations */
%%
e : 'x'| 'y'| '(' e '+' e ')'| '(' e '*' e ')'
%%
/* No main function */
expr1.y
/* No actions */
Non-terminal
Terminal
![Page 11: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/11.jpg)
Output of Bison
Bison generates a C function
int yyparse() {⋯
}
Takes as input a stream of tokens.
Returns zero if input conforms togrammar, and non-zero otherwise.
Calls yylex() to get the next token.
Stops when yylex() returns zero.
When a grammar rule is used, thatrule’s action is executed.
![Page 12: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/12.jpg)
Output of Bison
Bison generates a C function
int yyparse() {⋯
}
Takes as input a stream of tokens.
Returns zero if input conforms togrammar, and non-zero otherwise.
Calls yylex() to get the next token.
Stops when yylex() returns zero.
When a grammar rule is used, thatrule’s action is executed.
![Page 13: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/13.jpg)
Example 1, revisted
/* No declarations */
%%
e : 'x'| 'y'| '(' e '+' e ')'| '(' e '*' e ')'
%%
int yylex() {char c = getchar();if (c == '\n') return 0; else return c;
}
void main() {printf(“%i\n”, yyparse());
}
expr1.y
/* No actions */
![Page 14: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/14.jpg)
Example 1, revisted
/* No declarations */
%%
e : 'x'| 'y'| '(' e '+' e ')'| '(' e '*' e ')'
%%
int yylex() {char c = getchar();if (c == '\n') return 0; else return c;
}
void main() {printf(“%i\n”, yyparse());
}
expr1.y
/* No actions */
![Page 15: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/15.jpg)
Running Example 1
At a command prompt '>':
> bison -o expr1.c expr1.y
> gcc -o expr1 expr1.c -ly
> expr1(x+(y*x))0
Input
Important!
Output(0 means successful parse)
![Page 16: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/16.jpg)
Running Example 1
At a command prompt '>':
> bison -o expr1.c expr1.y
> gcc -o expr1 expr1.c -ly
> expr1(x+(y*x))0
Input
Important!
Output(0 means successful parse)
![Page 17: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/17.jpg)
Example 2
Terminals can be declared using a%token declaration, for example,to represent arithmetic variables:
%token VAR
%%
e : VAR| '(' e '+' e ')'| '(' e '*' e ')'
%%
/* main() and yylex() */
expr2.y
![Page 18: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/18.jpg)
Example 2
Terminals can be declared using a%token declaration, for example,to represent arithmetic variables:
%token VAR
%%
e : VAR| '(' e '+' e ')'| '(' e '*' e ')'
%%
/* main() and yylex() */
expr2.y
![Page 19: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/19.jpg)
Example 2 (continued)
int yylex() {int c = getchar();
/* Ignore white space */while (c == ' ') c = getchar();
if (c == '\n') return 0;if (c >= 'a' && c <= 'z')
return VAR;return c;
}
void main() {printf(“%i\n”, yyparse());
}
expr2.y
Return a VAR token
![Page 20: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/20.jpg)
Example 2 (continued)
int yylex() {int c = getchar();
/* Ignore white space */while (c == ' ') c = getchar();
if (c == '\n') return 0;if (c >= 'a' && c <= 'z')
return VAR;return c;
}
void main() {printf(“%i\n”, yyparse());
}
expr2.y
Return a VAR token
![Page 21: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/21.jpg)
Example 2 (continued)
Alternatively, the yylex() functioncan be generated by Flex.
%{#include "expr2.h"%}
%%
" " /* Ignore spaces */\n return 0;[a-z] return VAR;. return yytext[0];
%%
expr2.lex
Generated by Bison
![Page 22: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/22.jpg)
Example 2 (continued)
Alternatively, the yylex() functioncan be generated by Flex.
%{#include "expr2.h"%}
%%
" " /* Ignore spaces */\n return 0;[a-z] return VAR;. return yytext[0];
%%
expr2.lex
Generated by Bison
![Page 23: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/23.jpg)
Running Example 2
At a command prompt '>':
> bison --defines -o expr2.c expr2.y
> flex -o expr2lex.c expr2.lex
> gcc -o expr2 expr2.c expr2lex.c –ly -lfl
> expr2(a + ( b * c ))0
Parser
Output(0 means successful parse)
Lexer
Generate expr2.h
![Page 24: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/24.jpg)
Running Example 2
At a command prompt '>':
> bison --defines -o expr2.c expr2.y
> flex -o expr2lex.c expr2.lex
> gcc -o expr2 expr2.c expr2lex.c –ly -lfl
> expr2(a + ( b * c ))0
Parser
Output(0 means successful parse)
Lexer
Generate expr2.h
![Page 25: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/25.jpg)
Example 3
Adding numeric literals:
%token VAR%token NUM
%%
e : VAR| NUM| '(' e '+' e ')'| '(' e '*' e ')'
%%
void main() {printf(“%i\n”, yyparse());
}
expr3.y
Numeric Literal
![Page 26: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/26.jpg)
Example 3
Adding numeric literals:
%token VAR%token NUM
%%
e : VAR| NUM| '(' e '+' e ')'| '(' e '*' e ')'
%%
void main() {printf(“%i\n”, yyparse());
}
expr3.y
Numeric Literal
![Page 27: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/27.jpg)
Example 3 (continued)
%{#include "expr3.h"%}
%%
" " /* Ignore spaces */\n return 0;[a-z] return VAR;[0-9]+ return NUM;. return yytext[0];
%%
expr3.lex
Numeric Literal
Adding numeric literals:
![Page 28: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/28.jpg)
Example 3 (continued)
%{#include "expr3.h"%}
%%
" " /* Ignore spaces */\n return 0;[a-z] return VAR;[0-9]+ return NUM;. return yytext[0];
%%
expr3.lex
Numeric Literal
Adding numeric literals:
![Page 29: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/29.jpg)
Semantic valuesof tokens
A token can have a semanticvalue associated with it.
A NUM token contains an integer.
A VAR token contains a variable name.
Semantic values are returned viathe yylval global variable.
![Page 30: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/30.jpg)
Semantic valuesof tokens
A token can have a semanticvalue associated with it.
A NUM token contains an integer.
A VAR token contains a variable name.
Semantic values are returned viathe yylval global variable.
![Page 31: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/31.jpg)
Example 3 (revisited)
%{#include "expr3.h"%}
%%
" " /* Ignore spaces */\n return 0;[a-z] { yylval = yytext[0];
return VAR; }[0-9]+ { yylval = atoi(yytext);
return NUM;}
. return yytext[0];
%%
expr3.lex
Returning values via yylval:
![Page 32: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/32.jpg)
Example 3 (revisited)
%{#include "expr3.h"%}
%%
" " /* Ignore spaces */\n return 0;[a-z] { yylval = yytext[0];
return VAR; }[0-9]+ { yylval = atoi(yytext);
return NUM;}
. return yytext[0];
%%
expr3.lex
Returning values via yylval:
![Page 33: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/33.jpg)
Type of yylval
Problem: different tokens mayhave semantic values of differenttypes. So what is type of yylval?
%union{char var;int num;
}
Solution: a union type, whichcan be specified using the%union declaration, e.g.
yylval is eithera char or an int
![Page 34: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/34.jpg)
Type of yylval
Problem: different tokens mayhave semantic values of differenttypes. So what is type of yylval?
%union{char var;int num;
}
Solution: a union type, whichcan be specified using the%union declaration, e.g.
yylval is eithera char or an int
![Page 35: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/35.jpg)
Example 3 (revisted)
%{ #include "expr3.h" %}
%%
" " /* Ignore spaces */\n return 0;[a-z] { yylval.var = yytext[0];
return VAR; }[0-9]+ { yylval.num = atoi(yytext);
return NUM;}
. return yytext[0];
%%
expr3.lex
Returning values via yylval:
![Page 36: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/36.jpg)
Example 3 (revisted)
%{ #include "expr3.h" %}
%%
" " /* Ignore spaces */\n return 0;[a-z] { yylval.var = yytext[0];
return VAR; }[0-9]+ { yylval.num = atoi(yytext);
return NUM;}
. return yytext[0];
%%
expr3.lex
Returning values via yylval:
![Page 37: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/37.jpg)
Tokens have types
The type of token’s semanticvalue can be specified in a %tokendeclaration.
%union{char var;int num;
}
%token <var> VAR;%token <num> NUM;
![Page 38: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/38.jpg)
Tokens have types
The type of token’s semanticvalue can be specified in a %tokendeclaration.
%union{char var;int num;
}
%token <var> VAR;%token <num> NUM;
![Page 39: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/39.jpg)
Semantic valuesof non-terminals
A non-terminal can also have asemantic value associated with it.
$n refers to the semantic value ofthe nth symbol in the rule;
$$ refers to the semantic value of theresult of the rule.
In the action of a grammar rule:
%type <num> e;
The type can be specified in a%type declaration, e.g.
![Page 40: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/40.jpg)
Semantic valuesof non-terminals
A non-terminal can also have asemantic value associated with it.
$n refers to the semantic value ofthe nth symbol in the rule;
$$ refers to the semantic value of theresult of the rule.
In the action of a grammar rule:
%type <num> e;
The type can be specified in a%type declaration, e.g.
![Page 41: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/41.jpg)
Example 4
%{int env[256]; /* Variable environment */
%}%union{ int num; char var; }%token <num> NUM%token <var> VAR%type <num> e
%%
s : e { printf(“%i\n”, $1); }
e : VAR { $$ = env[$1]; }| NUM { $$ = $1; }| '(' e '+' e ')' { $$ = $2 + $4; }| '(' e '*' e ')' { $$ = $2 * $4; }
%%
void main() { env['x'] = 100; yyparse(); }
expr4.y
![Page 42: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/42.jpg)
Example 4
%{int env[256]; /* Variable environment */
%}%union{ int num; char var; }%token <num> NUM%token <var> VAR%type <num> e
%%
s : e { printf(“%i\n”, $1); }
e : VAR { $$ = env[$1]; }| NUM { $$ = $1; }| '(' e '+' e ')' { $$ = $2 + $4; }| '(' e '*' e ')' { $$ = $2 * $4; }
%%
void main() { env['x'] = 100; yyparse(); }
expr4.y
![Page 43: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/43.jpg)
Exercise 1
Modify Example 4 so that yyparse()constructs an abstract syntax tree.
typedef enum { Add, Mul } Op;
struct expr {enum { Var, Num, App } tag;union {
char var;int num;struct {
struct expr* e1; Op op; struct expr* e2;} app;
};};
typedef struct expr Expr;
Consider the following abstract syntax.
![Page 44: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/44.jpg)
Exercise 1
Modify Example 4 so that yyparse()constructs an abstract syntax tree.
typedef enum { Add, Mul } Op;
struct expr {enum { Var, Num, App } tag;union {
char var;int num;struct {
struct expr* e1; Op op; struct expr* e2;} app;
};};
typedef struct expr Expr;
Consider the following abstract syntax.
![Page 45: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/45.jpg)
Precedence and associativity
The associativity of an operatorcan be specified using a %left,%right, or %nonassoc directive.
%left '+’%left '*'%right '&'%nonassoc '='
Operators specified in increasingorder of precedence, e.g. '*' hashigher precedence than '+'.
![Page 46: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/46.jpg)
Precedence and associativity
The associativity of an operatorcan be specified using a %left,%right, or %nonassoc directive.
%left '+’%left '*'%right '&'%nonassoc '='
Operators specified in increasingorder of precedence, e.g. '*' hashigher precedence than '+'.
![Page 47: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/47.jpg)
Example 5
%token VAR%token NUM
%left '+'%left '*'
%%
e : VAR| NUM| e '+' e| e '*' e| ( e )
%%
void main() {printf(“%i\n”, yyparse());
}
expr5.y
![Page 48: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/48.jpg)
Example 5
%token VAR%token NUM
%left '+'%left '*'
%%
e : VAR| NUM| e '+' e| e '*' e| ( e )
%%
void main() {printf(“%i\n”, yyparse());
}
expr5.y
![Page 49: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/49.jpg)
Conflicts
Sometimes Bison cannot deducethat a grammar is unambiguous,even if it is*.
In such cases, Bison will report:
* Not surprising: ambiguity detection isundecidable in general!
a shift-reduce conflict; or
a reduce-reduce conflict.
![Page 50: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/50.jpg)
Conflicts
Sometimes Bison cannot deducethat a grammar is unambiguous,even if it is*.
In such cases, Bison will report:
* Not surprising: ambiguity detection isundecidable in general!
a shift-reduce conflict; or
a reduce-reduce conflict.
![Page 51: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/51.jpg)
Shift-Reduce Conflicts
Bison does not know whether toconsume more tokens (shift) or tomatch a production (reduce), e.g.
stmt : IF expr THEN stmt| IF expr THEN stmt ELSE stmt
Bison defaults to shift.
![Page 52: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/52.jpg)
Shift-Reduce Conflicts
Bison does not know whether toconsume more tokens (shift) or tomatch a production (reduce), e.g.
stmt : IF expr THEN stmt| IF expr THEN stmt ELSE stmt
Bison defaults to shift.
![Page 53: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/53.jpg)
Reduce-Reduce Conflicts
Bison does not know whichproduction to choose, e.g.
expr : functionCall| arrayLookup| ID
functionCall : ID '(' ID ')'
arrayLookup : ID '(' expr ')'
Bison defaults to using the firstmatching rule in the file.
![Page 54: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/54.jpg)
Reduce-Reduce Conflicts
Bison does not know whichproduction to choose, e.g.
expr : functionCall| arrayLookup| ID
functionCall : ID '(' ID ')'
arrayLookup : ID '(' expr ')'
Bison defaults to using the firstmatching rule in the file.
![Page 55: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/55.jpg)
Variants of Bison
There are Bison variants availablefor many languages:
Language Tool
Java JavaCC, CUP
Haskell Happy
Python PLY
C# Grammatica
* ANTLR
![Page 56: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/56.jpg)
Variants of Bison
There are Bison variants availablefor many languages:
Language Tool
Java JavaCC, CUP
Haskell Happy
Python PLY
C# Grammatica
* ANTLR
![Page 57: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/57.jpg)
Summary
Bison converts a context-freegrammar to a parsing functioncalled yyparse().
yyparse() calls yylex() to obtainnext token, so easy to connectFlex and Bison.
![Page 58: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/58.jpg)
Summary
Bison converts a context-freegrammar to a parsing functioncalled yyparse().
yyparse() calls yylex() to obtainnext token, so easy to connectFlex and Bison.
![Page 59: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/59.jpg)
Summary
Each grammar production mayhave an action.
Terminals and non-terminalshave semantic values.
Easy to construct abstractsyntax trees inside actionsusing semantic values.
Gives a declarative (high level)way to define parsers.
![Page 60: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/60.jpg)
Summary
Each grammar production mayhave an action.
Terminals and non-terminalshave semantic values.
Easy to construct abstractsyntax trees inside actionsusing semantic values.
Gives a declarative (high level)way to define parsers.
![Page 61: Lexical and Syntax Analysis - University of York › fp › lsa › lectures › slideshow6.pdf · > bison --defines -o expr2.c expr2.y > flex -o expr2lex.c expr2.lex > gcc-o expr2](https://reader033.vdocuments.net/reader033/viewer/2022060314/5f0baeb87e708231d431b3ce/html5/thumbnails/61.jpg)
Announcement
Summer internship in PLASMAgroup: Compiling Haskell toMicroblaze.
Salary: 250 pounds per week.
Part of the Reduceron project.
Ask if interested.
More details during Monday’spracticals.