may 21, 2002 the metafront tool aopl, s'02 language transformation: the metafront tool claus...

May 21, 2002 The metafront Tool AoPL, S'02

Language Transformation:The metafront Tool

Claus Brabrand

Michael I. Schwartzbach

BRICS, University of Aarhus, Denmark


Outline

• Introduction

• Specificity parsing

• Language transformation (by example)

• Current metafront limitations

• Exercises


Last Week

• Extend language with macros:• New productions• New nonterminals (via inductive morphings)

• Safety:• Guaranteed termination• Only legal syntax after expansion• Errors caught at macro definition-time


metafront

• Macros are just a special case usage:• A is an extension of B: m: L+ => L

• Make sure only need to write delta: = L+ \ L

metafront

x: A => B

A B

program.a program.b

transformation

input language

input program(s) output program(s)

output language


Lex/Yacc?

• Extensibility:• New productions and nonterminals?• New terminals (lexical structure)?

• Errors:• S/R, R/R: non-local error propagation

• Scanner:• Scanner/Parser state-correspondence


Lex/Yacc?

• Extensibility:• New productions and nonterminals?• New terminals (lexical structure)?

• Errors:• S/R, R/R: non-local error propagation

• Scanner:• Scanner/Parser state-correspondence

Instead…


Specificity Parsing


Specificity Grammar

Definition:

base alphabet

• T Reg( ) set of regular languages over

• N set of nonterminals

• s N start nonterminal : N P(E*) \ Ø, where E = (T U N)

G = (,T,N,s,)G = (,T,N,s,)


Example: While.llanguage While { terminal Num = { [0-9]+ } terminal Id = { [a-zA-Z0-9]+ }

stm[ass] <Id> := <exp> ; [if1] if ( <exp> ) <stm> [if2] if ( <exp> ) <stm> else <stm> [while] while ( <exp> ) <stm> … // block and sequence

exp[add] <base> + <base> // and -,*,/ [base] <base>

base[num] <Num> [id] <Id>}

language While { terminal Num = { [0-9]+ } terminal Id = { [a-zA-Z0-9]+ }

stm[ass] <Id> := <exp> ; [if1] if ( <exp> ) <stm> [if2] if ( <exp> ) <stm> else <stm> [while] while ( <exp> ) <stm> … // block and sequence

exp[add] <base> + <base> // and -,*,/ [base] <base>

base[num] <Num> [id] <Id>}

if(n)x:=0;elsex:=x+1;if(n)x:=0;elsex:=x+1;


Specificity Parsing

• parse(A: P(E*), : *) =1. Calculate first-set (based on production set)

2. Determine applicability (first-set vs. input string)

3. Terminal Selection (among applicable terminals)

4. Production Selection (based on winner terminal)

5. Parse entity (based on winner production)• case t: eat terminal t ’• case n: parse n (recursively): parse((n), ) ’• case : stop!

6. Parse tail: parse(advance(A), ’)








Specificity Parsing

• Challenge rounds:

1. Calculate first-set:• { if, while, <Id> }

2. Determine applicability:• { if: 2, while: -1, <Id>: 2 }

stm[ass] <Id> := <exp> ; [if1] if ( <exp> ) <stm> [if2] if ( <exp> ) <stm> else <stm> [while] while ( <exp> ) <stm>


#1 #2 #3 #4 #5 #6 #7 #8

input string:



Specificity Parsing

3. Terminal selection: { if: 2, while: -1, <Id>: 2 }

– Selection priority:• <length, terminal specificity>

lexicographically– length: prefer longest match;– terminal specificity: prefer most specific match

– In example:

• { if: 2, while: -1, <Id>: 2 } winner: if-terminal

if < id because { if } { [a-zA-Z0-9]+ }if < id because { if } { [a-zA-Z0-9]+ }

t < t’ := regexp(t) regexp(t’)t < t’ := regexp(t) regexp(t’)


Specificity Parsing

4. Production(s) selection: given if-terminal

– Selection priority:• terminal < nonterminal < prefer most specific

– Also, n < n’ := first(n) first(n’)

– In Example:• stm[ass,while] die!; winner: if-entity



#1 #2 #3 #4 #5 #6 #7 #8


Specificity Parsing

5. Parse entity: if

6. Parse tail: advance




#1 #2 #3 #4 #5 #6 #7 #8

stm[if1] ( <exp> ) <stm> [if2] ( <exp> ) <stm> else <stm> stm[if1] ( <exp> ) <stm> [if2] ( <exp> ) <stm> else <stm>


Specificity Parsing














Comments and Whitespace

• Semantics (“omit o;”) :

language While { terminal { Whitespace = { [ \t\n\r]+ } EndOfLineComment = { "//" .. \n } MultiLineComment = { "/*" .. "*/" } Omit = { ( <Whitespace> | <EndOfLineComment> | <MultiLineComment> )* } } omit Omit; // omit defaults to { [ \t\n\r]+ } …}

language While { terminal { Whitespace = { [ \t\n\r]+ } EndOfLineComment = { "//" .. \n } MultiLineComment = { "/*" .. "*/" } Omit = { ( <Whitespace> | <EndOfLineComment> | <MultiLineComment> )* } } omit Omit; // omit defaults to { [ \t\n\r]+ } …}

n e1 e2 … en corresponds to n o e1 o e2 o … en o n e1 e2 … en corresponds to n o e1 o e2 o … en o


Well-formedness

• No left-recursion• As for metamorphisms last week…

• Derivability• As for metamorphisms last week…

• Unique final winner:• Production specificity• Terminal specificity


Lookahead?

• Cannot discern stm[decl] from stm[exp]by looking at <Id>

*** specificity clash: stm[decl vs. exp] round #1 on <Id>*** specificity clash: stm[decl vs. exp] round #1 on <Id>

language JavaSubset { stm[decl] <decl> [exp] <exp> ;

decl[var] <Id> <Id> ;

exp[id] <Id> …}

language JavaSubset { stm[decl] <decl> [exp] <exp> ;


exp[id] <Id> …}


Attractors

• “Try-to-parse” semantics:• false kill• true select

– First([] ) := First() ignored!

language JavaSubset { stm[decl] [<Id> <Id>] <decl> [exp] <exp> ;


exp[id] <Id> …}

language JavaSubset { stm[decl] [<Id> <Id>] <decl> [exp] <exp> ;


exp[id] <Id> …}


The Case Trap

• Consider:

• stms will never stop on input “case…”

because <Id> First(stms) and case First(stms)

• Add “case trap”!:• Recall:

stm[switch] switch ( <exp> ) { <swb> }swb[one] <case> [more] <case> <swb>case[case] case <exp> : <stms>

stm[switch] switch ( <exp> ) { <swb> }swb[one] <case> [more] <case> <swb>case[case] case <exp> : <stms>

First([]) := First() ignored!First([]) := First() ignored!

stm[trap] [case] stm[trap] [case]


The Conjunction Trap

• Consider:

• Cannot parse:

exp[and] will always attempt to steal an ampersand

• Add “conjunction trap”!:

exp1[exp2] <exp2> [andand] <exp2> && <exp1>exp2[exp3] <exp3> [and] <exp3> & <exp2>exp3[id] <Id>

exp1[exp2] <exp2> [andand] <exp2> && <exp1>exp2[exp3] <exp3> [and] <exp3> & <exp2>exp3[id] <Id>

exp2[trap] [&&] exp2[trap] [&&]

x && yx && y

& higher precedence than &&& higher precedence than &&


Specificity Parsing Advantages

• Scanner implicitly derived from grammar

• Deterministic disambiguation

• Local conflict resolution

• Avoids keywordification

• Selection independent of definition-order

• Convenient overloading

• Commit: no explosion; no backtracking


Language Transformation

- by example -


Uminus

• Extend While language’s syntax:

• Specify transformation:

• Shorthand:

language MyWhile extends While { exp[uminus] - <base>}

language MyWhile extends While { exp[uminus] - <base>}

morph MyWhile2While: MyWhile ==> While { metamorph stms,stm,exp,base; exp[uminus](B) base(B)=>xB ==> << 0 - <xB> >>}

morph MyWhile2While: MyWhile ==> While { metamorph stms,stm,exp,base; exp[uminus](B) base(B)=>xB ==> << 0 - <xB> >>} binders inductive

transformations<< body >>

B=>xB short for base(B)=>xBB=>xB short for base(B)=>xB


More Examples

http://www.brics.dk/~brabrand/aopl/



Current metafront Limitations

• metafront (170K Java, 6300 lines):– Current limitations:

• No terminal transformations (micros)• All metamorphs must be explicitly declared• Attractors must be nonterminals (no in-lining)• No metamorph arguments

– Current inefficiencies:• No terminal selection caching• Attractor implementation

http://www.brics.dk/~brabrand/aopl/http://www.brics.dk/~brabrand/aopl/




Future

• Language/transformation calculus:• A idx(A): A => A• x: A => B, source(x): A• x: A => B, target(x): B• x: A => B, y: B => C x;y: A => C• x: A => B, y: A => C x*y: A => B*C• …


Exercises


Exercises (any 3 out of 4)1. Extend Java with a maybe construction:

1. Write Maybe.l, extending Java.l

2. Write Maybe2Java.x

2. Extend Java with a foreach construction:1. Write Foreach.l, extending Java.l

2. Write Foreach2Java.x

3. Extend While with a switch construction:1. Write Switch.l, extending While.l

2. Write Switch2While.x

4. Extend Java with any construction of your own choice:1. Write <any>.l, extending Java.l

2. Write <any>2Java.x


FIN

http://www.brics.dk/~brabrand/aopl/http://www.brics.dk/~brabrand/aopl/



may 21, 2002 the metafront tool aopl, s'02 language transformation: the metafront tool claus...

Documents