simplifications of context-free grammars
DESCRIPTION
Simplifications of Context-Free Grammars. A Substitution Rule. Equivalent grammar. Substitute. A Substitution Rule. Substitute. Equivalent grammar. In general:. Substitute. equivalent grammar. Nullable Variables. Nullable Variable:. Removing Nullable Variables. Example Grammar:. - PowerPoint PPT PresentationTRANSCRIPT
Fall 2004 COMP 335 1
Simplifications of
Context-Free Grammars
Fall 2004 COMP 335 2
A Substitution Rule
bB
aAB
abBcA
aaAA
aBS
Substitute
Equivalentgrammar
aAB
abbcabBcA
aaAA
abaBS
|
|
bB
Fall 2004 COMP 335 3
A Substitution Rule
EquivalentgrammarabaAcabbcabBcA
aaAA
aaAabaBS
||
||
aAB
abbcabBcA
aaAA
abaBS
|
|
Substitute aAB
Fall 2004 COMP 335 4
In general:
1yB
xBzA
Substitute
zxyxBzA 1|equivalentgrammar
1yB
Fall 2004 COMP 335 5
Nullable Variables
:production A
Nullable Variable: A
Fall 2004 COMP 335 6
Removing Nullable Variables
Example Grammar:
M
aMbM
aMbS
Nullable variable
Fall 2004 COMP 335 7
M
M
aMbM
aMbSSubstitute
abM
aMbM
abS
aMbS
Final Grammar
Fall 2004 COMP 335 8
Unit-Productions
BAUnit Production:
(a single variable in both sides)
Fall 2004 COMP 335 9
Removing Unit Productions
Observation:
AA
Is removed immediately
Fall 2004 COMP 335 10
Example Grammar:
bbB
AB
BA
aA
aAS
Fall 2004 COMP 335 11
bbB
AB
BA
aA
aAS
SubstituteBA
bbB
BAB
aA
aBaAS
|
|
Fall 2004 COMP 335 12
Remove
bbB
BAB
aA
aBaAS
|
|
bbB
AB
aA
aBaAS
|
BB
Fall 2004 COMP 335 13
SubstituteAB
bbB
aA
aAaBaAS
||
bbB
AB
aA
aBaAS
|
Fall 2004 COMP 335 14
Remove repeated productions
bbB
aA
aBaAS
|
bbB
aA
aAaBaAS
||
Final grammar
Fall 2004 COMP 335 15
Useless Productions
aAA
AS
S
aSbS
aAaaaaAaAAS
Some derivations never terminate...
Useless Production
Fall 2004 COMP 335 16
bAB
A
aAA
AS
Another grammar:
Not reachable from S
Useless Production
Fall 2004 COMP 335 17
In general:
if wxAyS
then variable is usefulA
otherwise, variable is uselessA
)(GLw
contains only terminals
Fall 2004 COMP 335 18
A production is useless if any of its variables is useless
xA
DC
CB
aAA
AS
S
aSbS
Productions
useless
useless
useless
useless
Variables
useless
useless
useless
Fall 2004 COMP 335 19
Removing Useless Productions
Example Grammar:
aCbC
aaB
aA
CAaSS
||
Fall 2004 COMP 335 20
First: find all variables that can producestrings with only terminals
aCbC
aaB
aA
CAaSS
|| },{ BA
AS
},,{ SBA
Round 1:
Round 2:
Fall 2004 COMP 335 21
Keep only the variablesthat produce terminal symbols:
aCbC
aaB
aA
CAaSS
||
},,{ SBA
aaB
aA
AaSS
|
(the rest variables are useless)
Remove useless productions
Fall 2004 COMP 335 22
Second:Find all variablesreachable from
aaB
aA
AaSS
|
S A B
Use a Dependency Graph
notreachable
S
Fall 2004 COMP 335 23
Keep only the variablesreachable from S
aaB
aA
AaSS
|
aA
AaSS
|
Final Grammar
(the rest variables are useless)
Remove useless productions
Fall 2004 COMP 335 24
Removing All
Step 1: Remove -productions
Step 2: Remove Unit-productions
Step 3: Remove Useless productions
Fall 2004 COMP 335 25
Normal Formsfor
Context-free Grammars
Fall 2004 COMP 335 26
Chomsky Normal Form
Each productions has form:
BCA
variable variable
aAor
terminal
Fall 2004 COMP 335 27
Examples:
bA
SAA
aS
ASS
Not ChomskyNormal Form
aaA
SAA
AASS
ASS
Chomsky Normal Form
Fall 2004 COMP 335 28
Conversion to Chomsky Normal Form
Example:
AcB
aabA
ABaS
Not ChomskyNormal Form
Fall 2004 COMP 335 29
AcB
aabA
ABaS
Introduce variables for terminals:
cT
bT
aT
ATB
TTTA
ABTS
c
b
a
c
baa
a
cba TTT ,,
Fall 2004 COMP 335 30
Introduce intermediate variable:
cT
bT
aT
ATB
TTTA
ABTS
c
b
a
c
baa
a
cT
bT
aT
ATB
TTTA
BTV
AVS
c
b
a
c
baa
a
1
1
1V
Fall 2004 COMP 335 31
Introduce intermediate variable:
cT
bT
aT
ATB
TTV
VTA
BTV
AVS
c
b
a
c
ba
a
a
2
2
1
1
2V
cT
bT
aT
ATB
TTTA
BTV
AVS
c
b
a
c
baa
a
1
1
Fall 2004 COMP 335 32
Final grammar in Chomsky Normal Form:
cT
bT
aT
ATB
TTV
VTA
BTV
AVS
c
b
a
c
ba
a
a
2
2
1
1
AcB
aabA
ABaS
Initial grammar
Fall 2004 COMP 335 33
From any context-free grammar(which doesn’t produce )not in Chomsky Normal Form
we can obtain: An equivalent grammar in Chomsky Normal Form
In general:
Fall 2004 COMP 335 34
The Procedure
First remove:
Nullable variables
Unit productions
Fall 2004 COMP 335 35
Then, for every symbol : a
In productions: replace with a aT
Add production aTa
New variable: aT
Fall 2004 COMP 335 36
Replace any production nCCCA 21
with
nnn CCV
VCV
VCA
12
221
11
New intermediate variables: 221 ,,, nVVV
Fall 2004 COMP 335 37
Theorem:For any context-free grammar(which doesn’t produce )there is an equivalent grammar in Chomsky Normal Form
Fall 2004 COMP 335 38
Observations
• Chomsky normal forms are good for parsing and proving theorems
• It is very easy to find the Chomsky normal form for any context-free grammar
Fall 2004 COMP 335 39
Greibach Normal Form
All productions have form:
kVVVaA 21
symbol variables
0k
Fall 2004 COMP 335 40
Examples:
bB
bbBaAA
cABS
||
GreibachNormal Form
aaS
abSbS
Not GreibachNormal Form
Fall 2004 COMP 335 41
aaS
abSbS
Conversion to Greibach Normal Form:
bT
aT
aTS
STaTS
b
a
a
bb
GreibachNormal Form
Fall 2004 COMP 335 42
Theorem:For any context-free grammar(which doesn’t produce ) there is an equivalent grammarin Greibach Normal Form
Fall 2004 COMP 335 43
Observations
• Greibach normal forms are very good for parsing
• It is hard to find the Greibach normal form of any context-free grammar
Fall 2004 COMP 335 44
The CYK Parser
Fall 2004 COMP 335 45
The CYK Membership Algorithm
Input:
• Grammar in Chomsky Normal Form G
• String
Output:
find if )(GLw
w
Fall 2004 COMP 335 46
The Algorithm
• Grammar :G
bB
ABB
aA
BBA
ABS
• String : w aabbb
Input example:
Fall 2004 COMP 335 47
a a b b b
aa ab bb bb
aab abb bbb
aabb abbb
aabbb
aabbb
Fall 2004 COMP 335 48
aA
aA
bB
bB
bB
aa ab bb bb
aab abb bbb
aabb abbb
aabbb
bB
ABB
aA
BBA
ABS
Fall 2004 COMP 335 49
a A
a A
b B
b B
b B
aa
ab S,B
bb A
bb A
aab abb bbb
aabb abbb
aabbb
bB
ABB
aA
BBA
ABS
Fall 2004 COMP 335 50
aA
aA
bB
bB
bB
aa abS,B
bbA
bbA
aabS,B
abbA
bbbS,B
aabbA
abbbS,B
aabbbS,B
bB
ABB
aA
BBA
ABS
Fall 2004 COMP 335 51
Therefore: )(GLaabbb
Time Complexity:3||w
The CYK algorithm can be easily converted to a parser(bottom up parser)
Observation: