ppcrslidesannotated

Solving Difficult LR Parsing Conflicts byPostponing Them

Luis Garcia-Forte and Casiano Rodriguez-LeonUniversidad de La Laguna

CoRTA2010 September 09-10, 2010

Outline

Introduction

Context Dependent Parsing

Nested Parsing

Conclusions

Outline

Introduction

Nested Parsing

Conclusions

Yacc Limitations

� Yacc-like LR parser generators provide ways to solveshift-reduce mechanisms based on token precedence.

� No mechanisms are provided for the resolution ofreduce-reduce conflicts or difficult shift-reduce conflicts.

� To solve such kind of conflicts the language designer hasto modify the grammar.

Yacc Limitations

Postponed Conflict Resolution Approach

� The strategy presented here extends yacc conflictresolution mechanisms with new ones, supplying ways toresolve conflicts that can not be solved using staticprecedences.

� The algorithm for the generation of the LR tables remainsunchanged, but the programmer can modify the parsingtables during run time.

PPCR Approach: Identify the Conflict

� Identify the conflict: What LR(0)-items/productions andtokens are involved?.

� Tools must support that stage, as for example via the.output file generated by eyapp.

� Suppose we identify that the participants are the twoLR(0)-items

� A→ α↑and

� B → β↑when the lookahead token is@.

� A→ α↑and

� A→ α↑

and� B → β↑

when the lookahead token is@.

� A→ α↑and

� B → β↑

when the lookahead token is@.

� A→ α↑and

� B → β↑when the lookahead token is

� A→ α↑and

� The software must allow the use of symbolic labels to referby name to the productions involved in the conflict.

� In Eyapp names and labels can be explicitly given usingthe directive %name, using a syntax similar to this one:

%name :rA A → α

%name :rB B → β

%name :rA A → α

%name :rB B → β

%name :rA A → α

%name :rB B → β

� Give a symbolic name to the conflict. In this case wechoose isAorB as name of the conflict.

� Inside the body section of the grammar, mark the points ofconflict using the new reserved word %PREC followed bythe conflict name:

%name :rA A → α %PREC IsAorB

%name :rA B → β %PREC IsAorB

Conflict Identification: Shift-Reduce Conflicts

The procedure is similar for shift-reduce conflicts.

� Let us assume we have identified a shift-reduce conflictbetween LR-(0) items

� A→ α↑and

� B → β ↑ γ for some token ’@’.� Again, we must give a symbolic name to A→ α and mark

with the new %PREC directive the places where the conflictoccurs:

B → β %PREC IsAorB γ

The procedure is similar for shift-reduce conflicts.� Let us assume we have identified a shift-reduce conflict

between LR-(0) items

� A→ α↑and

between LR-(0) items� A→ α↑

and� B → β ↑ γ for some token ’@’.

� Again, we must give a symbolic name to A→ α and markwith the new %PREC directive the places where the conflictoccurs:

and� B → β ↑ γ

for some token ’@’.� Again, we must give a symbolic name to A→ α and mark

PPCR Approach: The Conflict Handler

� Introduce a %conflict directive inside the head sectionof the translation scheme to specify the way the conflict willbe solved.

� The directive is followed by some code - known as theconflict handler - whose mission is to modify the parsingtables.

� This code will be executed each time the associatedconflict state is reached.

� This is the usual layout of the conflict handler for areduce-reduce conflict:

%conflict IsAorB {if (is_A) {$self->YYSetReduce(’@’, ’:rA’ );

}else {$self->YYSetReduce(’@’, ’:rB’ );

Inside a conflict code handler the Perl default variable $_refers to the input and $self refers to the parser object.

�� This is the usual layout of the conflict handler for areduce-reduce conflict:

�� This is the usual layout of the conflict handler for ashift-reduce conflict:

}else {$self->YYSetShift(’@’);

� The method YYSetReduce receives a list of tokens and aproduction label, like :rA. The call

$self->YYSetReduce(’@’, ’:rA’ )

sets the parsing action for the state associated with theconflict IsAorB to reduce by the production :rA when thecurrent lookahead is @.

� The method YYSetShift receives a token (or a referenceto a list of tokens). The call

$self->YYSetShift(’@’)

sets the parsing action for the state associated with theconflict IsAorB to shift when the current lookahead is @.

� %conflict IsAorB {if (is_A) { $self->YYSetReduce(’@’, ’:rA’ );}else { $self->YYSetReduce(’@’, ’:rB’ ); }

� The call to is_A represents the context-dependentdynamic knowledge that allows us to take the rightdecision.

� It is usually a call to a nested parser to check that A→ αapplies but it can also be any other contextual informationwe have to determine which one is the right production.

� %conflict IsAorB {if (is_A) { $self->YYSetReduce(’@’, ’:rA’ );}else { $self->YYSetReduce(’@’, ’:rB’ ); }

� The call to is_A represents the context-dependentdynamic knowledge that allows us to take the rightdecision.

� It is usually a call to a nested parser to check that A→ αapplies but it can also be any other contextual informationwe have to determine which one is the right production.

Outline

Introduction

Nested Parsing

Conclusions

A Simple Example: Context Dependant Associativity

$ cat input_for_dynamicgrammar.txt2-1-1 # left: 0 = (2-1)-1

2-1-1 # right: 2 = 2-(1-1)

3-1-1 # left: 1 = (3-1)-1

RIGHT3-1-1 # right: 3 = 3-(1-1)

2-1-1 # right: 2 = 2-(1-1)

3-1-1 # left: 1 = (3-1)-1

RIGHT3-1-1 # right: 3 = 3-(1-1)

2-1-1 # right: 2 = 2-(1-1)

3-1-1 # left: 1 = (3-1)-1

RIGHT3-1-1 # right: 3 = 3-(1-1)

2-1-1 # right: 2 = 2-(1-1)

3-1-1 # left: 1 = (3-1)-1

RIGHT3-1-1 # right: 3 = 3-(1-1)

2-1-1 # right: 2 = 2-(1-1)

3-1-1 # left: 1 = (3-1)-1

RIGHT3-1-1 # right: 3 = 3-(1-1)

2-1-1 # right: 2 = 2-(1-1)

3-1-1 # left: 1 = (3-1)-1

RIGHT3-1-1 # right: 3 = 3-(1-1)

A Simple Example: Bodyp: c * {};

c: $expr { print "$expr\n" }| RIGHT { $reduce = 0}| LEFT { $reduce = 1}

expr: ’(’ $expr ’)’ { $expr }| NUM

| %name :Mexpr.left %PREC leftORright’-’ expr.right %PREC leftORright

{ $left - $right }

A Simple Example: Head Section$ cat dynamicgrammar.eyp

%{my $reduce = 1;%}

%whites /(\s*(?:#.*)?\s*)/%token NUM = /(\d+)/

%conflict leftORright {if ($reduce) { $self->YYSetReduce(’-’, ’:M’) }else { $self->YYSetShift(’-’) }

%expect 1

%{my $reduce = 1;%}

%expect 1

%{my $reduce = 1;%}

%expect 1

%{my $reduce = 1;%}

%expect 1

%{my $reduce = 1;%}

%expect 1

Outline

Introduction

Nested Parsing

Conclusions

Enumerated and Subrange Types in Pascal

� The problem is taken from the Bison manual.

type subrange = lo .. hi;type enum = (a, b, c);

� The Pascal standard allows only numeric literals andconstant identifiers for the subrange bounds (lo and hi),but Extended Pascal and many other implementationsallow arbitrary expressions there.

type subrange = (a) .. b;type enum = (a);

� To aggravate the conflict we have added the C commaoperator to the language

type subrange = (a, b, c) .. (d, e);type enum = (a, b, c);

Enumerated vs Range: Full Grammar%token ID = /([A-Za-z]\w*)/%token NUM = /(\d+)/

%left ’,’%left ’-’ ’+’%left ’*’ ’/’

%%type_decl : ’TYPE’ ID ’=’ type ’;’ ;

type : ’(’ id_list ’)’ | expr ’..’ expr ;

id_list : ID | id_list ’,’ ID ;expr : ID

| NUM | ’(’ expr ’)’| expr ’,’ expr /* new */| expr ’+’ expr | expr ’-’ expr| expr ’*’ expr | expr ’/’ expr

%left ’,’%left ’-’ ’+’%left ’*’ ’/’

Identifying the Conflict$ eyapp -v pascalenumeratedvsrange.eyp2 reduce/reduce conflicts

State 11:id_list -> ID . (Rule 4)expr -> ID . (Rule 12)

’)’ [reduce using rule 12 (expr)]’)’ reduce using rule 4 (id_list)’,’ [reduce using rule 12 (expr)]’,’ reduce using rule 4 (id_list)

’*’ reduce using rule 12 (expr)’+’ reduce using rule 12 (expr)’-’ reduce using rule 12 (expr)’/’ reduce using rule 12 (expr)

Identifying the Conflict$ eyapp -v pascalenumeratedvsrange.eyp2 reduce/reduce conflicts

State 11:id_list -> ID . (Rule 4)expr -> ID . (Rule 12)

’)’ [reduce using rule 12 (expr)]’)’ reduce using rule 4 (id_list)’,’ [reduce using rule 12 (expr)]’,’ reduce using rule 4 (id_list)

’*’ reduce using rule 12 (expr)’+’ reduce using rule 12 (expr)’-’ reduce using rule 12 (expr)’/’ reduce using rule 12 (expr)

Enumerated vs Range: Body

type_decl : ’type’ ID ’=’ type ’;’ ;

type :%name ENUMLookahead ’(’ id_list ’)’

| %name RANGELookahead expr ’..’ expr

;Lookahead: /* empty */

{ $is_range = $_[0]->YYPreParse(’Range’); };

Lookahead: /* empty */{ $is_range = $_[0]->YYPreParse(’Range’); }

;Lookahead: /* empty */{ $is_range = $_[0]->YYPreParse(’Range’); }

Nested Parsing: Future Work

$ cat Range2.eyp

%from pascalnestedeyapp2 import expr%%range: expr ’..’ expr ’;’;%%

Nested Parsing: Current State$ cat Range.eyp%token ID = /([A-Za-z]\w*)/%token NUM = /(\d+)/

%left ’,’%left ’-’ ’+’%left ’*’ ’/’

%%range: expr ’..’ expr ’;’;

id_list :%name ID:ENUMID %PREC rangeORenum

| id_list ’,’ ID;expr : ’(’ expr ’)’ { $_[2] } /* bypass */

| %name ID:RANGEID %PREC rangeORenum

Enumerated vs Range: Bodyid_list :

%name ID:ENUMID %PREC rangeORenum

| id_list ’,’ ID;

expr : ’(’ expr ’)’ { $_[2] } /* bypass */| %name ID:RANGE

Enumerated vs Range: Bodyid_list :

%name ID:ENUMID %PREC rangeORenum

| id_list ’,’ ID;expr : ’(’ expr ’)’ { $_[2] } /* bypass */

| %name ID:RANGEID %PREC rangeORenum

Enumerated vs Range: Header

%{my $is_range = 0;

%}%conflict rangeORenum {if ($is_range){ $self->YYSetReduce([’,’, ’)’], ’ID:RANGE’ ); }else{ $self->YYSetReduce([’,’, ’)’], ’ID:ENUM’ ); }

%token ID = /([A-Za-z]\w*)/%token NUM = /(\d+)/

%left ’,’%left ’-’ ’+’%left ’*’ ’/’

%expect-rr 2%%

Enumerated vs Range: Header%{my $is_range = 0;

%left ’,’%left ’-’ ’+’%left ’*’ ’/’

%expect-rr 2%%

%left ’,’%left ’-’ ’+’%left ’*’ ’/’

%expect-rr 2%%

%left ’,’%left ’-’ ’+’%left ’*’ ’/’

%expect-rr 2%%

%left ’,’%left ’-’ ’+’%left ’*’ ’/’

%expect-rr 2%%

Enumerated vs Range: Execution

$ eyapp -P Range.eyp

$ eyapp -TC pascalnestedeyapp2.eyp

$ ./pascalnestedeyapp2.pm -t -i -m 1 \-c ’type e = (x, y, z);’

typeDecl_is_type_ID_type(TERMINAL[e],ENUM(idList_is_idList_ID(

idList_is_idList_ID(ID(TERMINAL[x]),TERMINAL[y]),

TERMINAL[z])))

Outline

Introduction

Nested Parsing

Conclusions

� The new conflict resolution mechanisms provide ways toresolve conflicts that can not be solved using staticprecedences.

� They also provides finer control over the conflict resolutionprocess than existing alternatives, like GLR andbacktracking LR.

� There are no limitations to PPCR parsing, since the conflicthandler is implemented in a universal language and it thencan resort to any kind of nested parsing algorithm.

� The conflict resolution mechanisms presented here can beintroduced in any LR parsing tools.

� It certainly involves more programmer work than branchingmethods like GLR or backtracking LR.

Conclusions

http://parse-eyapp.googlecode.com/svn/branches/PPCR

Thanks!

http://parse-eyapp.googlecode.com/svn/branches/PPCR

Thanks!

ppcrslidesannotated

Technology