courtesy costas busch - rpi1 positive properties of context-free languages
Post on 19-Dec-2015
224 views
TRANSCRIPT
Courtesy Costas Busch - RPI 1
Positive Propertiesof
Context-Free languages
Courtesy Costas Busch - RPI 2
Context-free languages are closed under: Union
1L is context free
2L is context free
21 LL
is context-free
Union
Courtesy Costas Busch - RPI 3
Example
|11 baSS
|| 222 bbSaaSS
Union
}{1nnbaL
}{2RwwL
21 | SSS }{}{ Rnn wwbaL
Language Grammar
Courtesy Costas Busch - RPI 4
In general:
The grammar of the union has new start variableand additional production 21 | SSS
For context-free languageswith context-free grammarsand start variables
21, LL
21, GG
21, SS
21 LL S
Courtesy Costas Busch - RPI 5
Context-free languages are closed under: Concatenation
1L is context free
2L is context free
21LL
is context-free
Concatenation
Courtesy Costas Busch - RPI 6
Example
|11 baSS
|| 222 bbSaaSS
Concatenation
}{1nnbaL
}{2RwwL
21SSS }}{{ Rnn wwbaL
Language Grammar
Courtesy Costas Busch - RPI 7
In general:
The grammar of the concatenation has new start variableand additional production 21SSS
For context-free languageswith context-free grammarsand start variables
21, LL
21, GG
21, SS
21LLS
Courtesy Costas Busch - RPI 8
Context-free languages are closed under: Star-operation
L is context free*L is context-free
Star Operation
Courtesy Costas Busch - RPI 9
|aSbS }{ nnbaL
|11 SSS *}{ nnbaL
Example
Language Grammar
Star Operation
Courtesy Costas Busch - RPI 10
In general:
The grammar of the star operation has new start variableand additional production
For context-free languagewith context-free grammarand start variable
LGS
*L1S
|11 SSS
Courtesy Costas Busch - RPI 11
Negative Propertiesof
Context-Free Languages
Courtesy Costas Busch - RPI 12
Context-free languagesare not closed under: intersection
1L is context free
2L is context free
21 LL
not necessarilycontext-free
Intersection
Courtesy Costas Busch - RPI 13
Example
}{1mnn cbaL
|
|
cCC
aAbA
ACS
Context-free:
}{2mmn cbaL
|
|
bBcB
aAA
ABS
Context-free:
}{21nnn cbaLL NOT context-free
Intersection
Courtesy Costas Busch - RPI 14
Context-free languagesare not closed under: complement
L is context free L not necessarilycontext-free
Complement
Courtesy Costas Busch - RPI 15
}{2121nnn cbaLLLL
NOT context-free
Example
}{1mnn cbaL
|
|
cCC
aAbA
ACS
Context-free:
}{2mmn cbaL
|
|
bBcB
aAA
ABS
Context-free:
Complement
Courtesy Costas Busch - RPI 16
Intersectionof
Context-free languagesand
Regular Languages
Courtesy Costas Busch - RPI 17
The intersection of a context-free language and a regular languageis a context-free language
1L context free
2L regular
21 LL
context-free
Courtesy Costas Busch - RPI 18
1Lfor for 2LNPDA
1M
DFA
2M
Construct a new NPDA machinethat accepts
Machine Machine
M21 LL
context-free regular
M simulates in parallel and 1M 2M
Courtesy Costas Busch - RPI 19
1M 2M
1q 2qcba ,
transition
1p 2pa
transition
NPDA DFA
11, pq cba ,
transition
MNPDA
22 , pq
Courtesy Costas Busch - RPI 20
1M 2M
1q 2qcb,
transition
1p
NPDA DFA
11, pq cb,
transition
MNPDA
12, pq
Courtesy Costas Busch - RPI 21
1M 2M
0q
initial state
0p
initial state
NPDA DFA
Initial state
MNPDA
00 , pq
Courtesy Costas Busch - RPI 22
1M 2M
1q
final state
1p
final states
NPDA DFA
final states
MNPDA
11, pq
2p
21, pq
Courtesy Costas Busch - RPI 23
Example:
, , ,
1, b1, a
1,d1,c
0q 1q 2q 3q
1M
}},{,},{|,|||:{ *2
*121211 dcwbawwwwwL
NPDA
context-free
Courtesy Costas Busch - RPI 24
0p
2M
*2 },{ caL
ca,
DFA
regular
Courtesy Costas Busch - RPI 25
Automaton for: }0:{21 ncaLL nn
, , ,
1, a 1,c
00, pq 01, pq 02, pq 03, pq
NPDA M
context-free
Courtesy Costas Busch - RPI 26
M simulates in parallel and 1M 2M
M accepts string w if and only if
accepts string and w1M
accepts string w2M
)()()( 21 MLMLML
In General:
Courtesy Costas Busch - RPI 27
Therefore:
)()( 21 MLML
is NPDAM
is context-free
21 LL is context-free
Courtesy Costas Busch - RPI 28
Applications of
Regular Closure
Courtesy Costas Busch - RPI 29
The intersection of a context-free language and a regular languageis a context-free language
1L context free
2L regular
21 LL
context-free
Regular Closure
Courtesy Costas Busch - RPI 30
An Application of Regular Closure
Prove that: }0,100:{ nnbaL nn
is context-free
Courtesy Costas Busch - RPI 31
}0:{ nba nn
We know:
is context-free
Courtesy Costas Busch - RPI 32
}{ 1001001 baL is regular
}{}){( 100100*1 babaL is regular
We also know:
Courtesy Costas Busch - RPI 33
}{}){( 100100*1 babaL
regular
}{ nnbacontext-free
1}{ Lba nn context-free
LnnbaLba nnnn }0,100:{}{ 1
is context-free
(regular closure)
Courtesy Costas Busch - RPI 34
Another Application of Regular Closure
Prove that: }:{ cba nnnwL
is not context-free
Courtesy Costas Busch - RPI 35
}:{ cba nnnwL
}{*}**{ nnn cbacbaL
context-free regular context-free
If is context-free
Then
Impossible!!!
Therefore, is not context free L
(regular closure)
Courtesy Costas Busch - RPI 36
Decidable Propertiesof
Context-Free Languages
Courtesy Costas Busch - RPI 37
Membership Question:
for context-free grammarfind if string
G)(GLw
Membership Algorithms: Parsers
• Exhaustive search parser
• CYK parsing algorithm
38
?)(GLw
Courtesy Costas Busch - RPI 39
The CYK Parser
Courtesy Costas Busch - RPI 40
The CYK Membership Algorithm
Input:
• Grammar in Chomsky Normal Form G
• String
Output:
find if )(GLw
w
41
The CYK Membership Algorithm
42
The CYK Membership Algorithm
43
Courtesy Costas Busch - RPI 44
The Algorithm
• Grammar :G
bB
ABB
aA
BBA
ABS
• String : w aabbb
Input example:
Courtesy Costas Busch - RPI 45
a a b b b
aa ab bb bb
aab abb bbb
aabb abbb
aabbb
aabbb
Courtesy Costas Busch - RPI 46
aA
aA
bB
bB
bB
aa ab bb bb
aab abb bbb
aabb abbb
aabbb
bB
ABB
aA
BBA
ABS
Courtesy Costas Busch - RPI 47
a A
a A
b B
b B
b B
aa
ab S,B
bb A
bb A
aab abb bbb
aabb abbb
aabbb
bB
ABB
aA
BBA
ABS
Courtesy Costas Busch - RPI 48
aA
aA
bB
bB
bB
aa abS,B
bbA
bbA
aabS,B
abbA
bbbS,B
aabbA
abbbS,B
aabbbS,B
bB
ABB
aA
BBA
ABS
Courtesy Costas Busch - RPI 49
Therefore: )(GLaabbb
Time Complexity:3||w
The CYK algorithm can be easily converted to a parser(bottom up parser)
Observation:
Courtesy Costas Busch - RPI 50
Empty Language Question:
for context-free grammar find if
G)(GL
Algorithm:
S
1. Remove useless variables
2. Check if start variable is useless
Courtesy Costas Busch - RPI 51
Infinite Language Question:
for context-free grammar find if is infinite
G)(GL
Algorithm:
1. Remove useless variables
2. Remove unit and productions
3. Create dependency graph for variables
4. If there is a loop in the dependency graph then the language is infinite
Courtesy Costas Busch - RPI 52
Example:
cBSC
bbbBB
aaCbA
ABS
|
|
S
A
B
C
Dependency graph Infinite language
Courtesy Costas Busch - RPI 53
cBSC
bbbBB
aaCbA
ABS
|
|
acbbSbbbacBSbBaCbBABS
ii bbbSacbb
bbbSacbbacbbSbbbS
)()(
)()( 22
54
Undecidable CFL Problems
Courtesy Costas Busch - RPI 55
YACC
Yet Another Compiler Compiler
Courtesy Costas Busch - RPI 56
Yacc is a parser generator
Input: A Grammar
Output: A parser for the grammar
Reminder: a parser finds derivations
Courtesy Costas Busch - RPI 57
Example grammar:
The yacc code: expr : '(' expr ')' | expr '+' expr | expr '-' expr | expr '*' expr | expr '/' expr | - expr | INT ;
expr -> ( expr ) | expr '+' expr | expr '-' expr | expr '*' expr | expr '/' expr | - expr | INT ;
Courtesy Costas Busch - RPI 58
Exampe Input:
10 * 3 + 4
Yacc Derivation:
expr => expr + expr => expr * expr + expr => 10*3 + 4
Courtesy Costas Busch - RPI 59
Resolving Ambiguities
%left '+', '-'%left '*', '/'%left UMINUS%%
expr : '(' expr ')' | expr '+' expr | expr '-' expr | expr '*' expr | expr '/' expr | '-' expr %prec UMINUS | INT ;
Courtesy Costas Busch - RPI 60
Actions
%left '+', '-'%left '*', '/'%left UMINUS%%
expr : '(' expr ')' {$$ = $2;} | expr '+' expr {$$ = $1 + $3;} | expr '-' expr {$$ = $1 - $3;} | expr '*' expr {$$ = $1 * $3;} | expr '/' expr {$$ = $1 / $3;} | '-' expr %prec UMINUS {$$ = -$2;} | INT {$$ = $1;} ;
Courtesy Costas Busch - RPI 61
A Complete Yacc program
%union{ int int_val;}
%left '+', '-'%left '*', '/'%left UMINUS
%token <int_val> INT%type <int_val> expr
%start program
%%
Courtesy Costas Busch - RPI 62
program : expr {printf("Expr value = %d \n", $1);} | error {printf("YACC: syntax error near line %d \n", linenum); abort();} ;
expr : '(' expr ')' {$$ = $2;} | expr '+' expr {$$ = $1 + $3;} | expr '-' expr {$$ = $1 - $3;} | expr '*' expr {$$ = $1 * $3;} | expr '/' expr {$$ = $1 / $3;} | '-' expr %prec UMINUS {$$ = -$2;} | INT {$$ = $1;} ;
%%
#include "lex.yy.c"
Courtesy Costas Busch - RPI 63
Execution Example
10 + 20*(3 - 4 + 25)Input:
Output: Expr value = 490
Courtesy Costas Busch - RPI 64
The Lex Code%{int linenum=1;int temp_int;%}%%
\n {linenum++;}
[\t ] /* skip spaces */;\/\/[^\n]* /* ignore comments */;
"+" {return '+';}"-" {return '-';} "*" {return '*';} "/" {return '/';} ")" {return ')';}"(" {return '(';}
Courtesy Costas Busch - RPI 65
[0-9]+ {sscanf(yytext, "%d", &temp_int); yylval.int_val = temp_int; return INT;}
. {printf("LEX: unknown input string found in line %d \n", linenum); abort();}
Courtesy Costas Busch - RPI 66
Compiling:
yacc YaccFilelex LexFilecc y.tab.c -ly -ll -o myparser
Executable: myparser
Courtesy Costas Busch - RPI 67
Another Yacc Program
%union{ int int_val;}
%left '+', '-'%left '*', '/'%left UMINUS
%token <int_val> INT%type <int_val> expr
%start program
%%
Courtesy Costas Busch - RPI 68
program : stmt_list | error {printf("YACC: syntax error near line %d \n", linenum); abort();} ;
stmt_list : stmt_list stmt | stmt ;
stmt : expr ';' {printf("Expr value = %d \n", $1);} ;
Courtesy Costas Busch - RPI 69
expr : '(' expr ')' {$$ = $2;} | expr '+' expr {$$ = $1 + $3;} | expr '-' expr {$$ = $1 - $3;} | expr '*' expr {$$ = $1 * $3;} | expr '/' expr {$$ = $1 / $3;} | '-' expr %prec UMINUS {$$ = -$2;} | INT {$$ = $1;} ;
%%
#include "lex.yy.c"
Courtesy Costas Busch - RPI 70
Execution Example
10 + 20*(30 -67) / 4;
34 * 35 - 123 + -001;
17*8/6;
Input:
Output: Expr value = -175 Expr value = 1066 Expr value = 22
Courtesy Costas Busch - RPI 71
Lex Code
%{int linenum=1;int temp_int;%}%%
\n {linenum++;}
[\t ] /* skip spaces */;\/\/[^\n]* /* ignore comments */;
Courtesy Costas Busch - RPI 72
"+" {return '+';}"-" {return '-';} "*" {return '*';} "/" {return '/';} ")" {return ')';}"(" {return '(';}";" {return ';';}
[0-9]+ {sscanf(yytext, "%d", &temp_int); yylval.int_val = temp_int; return INT;}
. {printf("LEX: unknown input string found in line %d \n", linenum); abort();}
Courtesy Costas Busch - RPI 73
Another Yacc Program%union{ int int_val; char *str_val;}
%left '+', '-'%left '*', '/'%left UMINUS
%token PRINT%token NEWLINE%token <str_val> STRING%token <int_val> INT%type <int_val> expr
%start program%%
Courtesy Costas Busch - RPI 74
program : stmt_list | error {printf("YACC: syntax error near line %d \n", linenum); abort();} ;
stmt_list : stmt_list stmt | stmt ;
stmt : expr ';' {printf("expression found\n");} | PRINT expr ';' {printf("%d", $2);} | PRINT STRING ';' {printf("%s", $2);} | PRINT NEWLINE ';' {printf("\n");} ;
Courtesy Costas Busch - RPI 75
expr : '(' expr ')' {$$ = $2;} | expr '+' expr {$$ = $1 + $3;} | expr '-' expr {$$ = $1 - $3;} | expr '*' expr {$$ = $1 * $3;} | expr '/' expr {$$ = $1 / $3;} | '-' expr %prec UMINUS {$$ = -$2;} | INT {$$ = $1;} ;
%%
#include "lex.yy.c"
Courtesy Costas Busch - RPI 76
Execution Example
Input: print "The value of expression 123 * 25 is ";print 123 * 25; print newline;10 + 5 * 8;print "end of program";print newline;
Output: The value of expression 123 * 25 is 3075expression foundend of program
Courtesy Costas Busch - RPI 77
Lex Code
%{int linenum=1;int temp_int;char temp_str[200];%}%%
\n {linenum++;}
[\t ] /* skip spaces */;\/\/[^\n]* /* ignore comments */;
Courtesy Costas Busch - RPI 78
"+" {return '+';}"-" {return '-';} "*" {return '*';} "/" {return '/';} ")" {return ')';}"(" {return '(';}";" {return ';';}"print" {return PRINT;}"newline" {return NEWLINE;}
Courtesy Costas Busch - RPI 79
[0-9]+ {sscanf(yytext, "%d", &temp_int); yylval.int_val = temp_int; return INT;}
\"[^"\n]*\" {strncpy(temp_str, &(yytext[1]), strlen(yytext)-2); temp_str[strlen(yytext)-2] = (char) 0; yylval.str_val = temp_str; return STRING;}
. {printf("LEX: unknown input string found in line %d \n", linenum); abort();}