may 14, 2002 macro languages aopl, s'02 macro languages claus brabrand michael i. schwartzbach...
Post on 21-Dec-2015
215 views
TRANSCRIPT
May 14, 2002 Macro Languages AoPL, S'02
Macro Languages
Claus Brabrand
Michael I. Schwartzbach
BRICS, University of Aarhus, Denmark
May 14, 2002 Macro Languages AoPL, S'02
Outline
• Introduction
• Macro survey:• Lexical macros: CPP, M4, TEX, (Dylan)
• Syntax macros: C++ templates, Scheme, JTS, MS2
• The <bigwig> macro language
• Metamorphic syntax macros
• Next week: The metafront Tool• Specificity parsing• Language transformation
May 14, 2002 Macro Languages AoPL, S'02
Introduction
May 14, 2002 Macro Languages AoPL, S'02
“Macro”
Webster’s(“macro”) =
Main Entry: 2macroPronunciation: 'ma-(")krOFunction: nounInflected Form(s): plural macrosEtymology: short for macroinstructionDate: 1959“a single computer instruction that stands for a sequence of operations”
Main Entry: 2macroPronunciation: 'ma-(")krOFunction: nounInflected Form(s): plural macrosEtymology: short for macroinstructionDate: 1959“a single computer instruction that stands for a sequence of operations”
May 14, 2002 Macro Languages AoPL, S'02
Motivation (#1)
Abstraction (language extension):
Iterator iterator = list.iterator();while (iterator.hasNext()) { String s = (String) iterator.next(); System.out.println(s);}
Iterator iterator = list.iterator();while (iterator.hasNext()) { String s = (String) iterator.next(); System.out.println(s);}
foreach (String s in list) { System.out.println(s); }
foreach (String s in list) { System.out.println(s); }
vs.
May 14, 2002 Macro Languages AoPL, S'02
Motivation (#2)
Genericity (uniform abstraction mechanism): • Generic abstraction mechanism for all syntactic
categories:
…whereas functions only “take” and “give” expressions:
<exp> <id> ( <exps> )<exp> <id> ( <exps> )
Principle of Abstraction:
“Any semantically meaningful syntactic class can in principle be used as the body of an abstraction”
- Robert Tennent, 1981
Principle of Abstraction:
“Any semantically meaningful syntactic class can in principle be used as the body of an abstraction”
- Robert Tennent, 1981
May 14, 2002 Macro Languages AoPL, S'02
Motivation (#3)
Consistency:
Object[] strings = list.toArray();for (int i=0; i<strings.length; i++) { String s = (String) strings[i]; System.out.println(s); }
Object[] strings = list.toArray();for (int i=0; i<strings.length; i++) { String s = (String) strings[i]; System.out.println(s); }
Iterator iterator = list.iterator();while (iterator.hasNext()) { String s = (String) iterator.next(); System.out.println(s);}
Iterator iterator = list.iterator();while (iterator.hasNext()) { String s = (String) iterator.next(); System.out.println(s);}
vs.
May 14, 2002 Macro Languages AoPL, S'02
Motivation (#4)
Laziness (abbreviation):
vs.
Iterator iterator = list.iterator();while (iterator.hasNext()) { String s = (String) iterator.next(); System.out.println(s);}
Iterator iterator = list.iterator();while (iterator.hasNext()) { String s = (String) iterator.next(); System.out.println(s);}
foreach (String s in list) { System.out.println(s); }
foreach (String s in list) { System.out.println(s); }
May 14, 2002 Macro Languages AoPL, S'02
Motivation (#5)
Encapsulation (hide complexity):
vs.
foreach (String s in list) { System.out.println(s); }
foreach (String s in list) { System.out.println(s); }
Iterator iterator = list.iterator();while (iterator.hasNext()) { String s = (String) iterator.next(); System.out.println(s);}
Iterator iterator = list.iterator();while (iterator.hasNext()) { String s = (String) iterator.next(); System.out.println(s);}
May 14, 2002 Macro Languages AoPL, S'02
Macro Survey
May 14, 2002 Macro Languages AoPL, S'02
Many Macro Languages
CPP
M4
TEX
Scheme
C++templates
MS2
...and many more
Dylan
JTS
<bigwig>
May 14, 2002 Macro Languages AoPL, S'02
Non-Alphabetical Characters
CPP
M4
TEX
Scheme
C++templates
MS2
...and many more
Dylan
JTS
<bigwig>
May 14, 2002 Macro Languages AoPL, S'02
Level of Operation
CPP
M4
TEX
Scheme
C++templates
Dylan
JTS
Lexical Syntactic
...and many more
MS2
<bigwig>
May 14, 2002 Macro Languages AoPL, S'02
Lexical Macros
• Operate on token sequences:• MLEX: (TOKENSEQ)n TOKENSEQ
• Precede actual compilation (conceptually)– i.e. a preprocessor
• Independent of host language syntax
• Languages:• CPP: “The C Preprocessor”• M4: “The Unix Macro Preprocessor”• TEX: TEX’s macro mechanism• Dylan: Dylan’s macro mechanism (hybrid)
May 14, 2002 Macro Languages AoPL, S'02
Lexical Macro Example
• Square (CPP):#define square(X) X * X#define square(X) X * X
May 14, 2002 Macro Languages AoPL, S'02
Lexical Macro Example
• Square (CPP):#define square(X) X * X
square(z + 1)
#define square(X) X * X
square(z + 1)
( )z + 1square
May 14, 2002 Macro Languages AoPL, S'02
Lexical Macro Example
• Square (CPP):#define square(X) X * X
square(z + 1) => z + 1 * z + 1
#define square(X) X * X
square(z + 1) => z + 1 * z + 1
( )z + 1 z + 1 z + 1*=>square
May 14, 2002 Macro Languages AoPL, S'02
Lexical Macro Example
• Square (CPP):#define square(X) X * X
square(z + 1) => z +(1 * z)+ 1
#define square(X) X * X
square(z + 1) => z +(1 * z)+ 1
( )z + 1 z + 1 z + 1*=>square
May 14, 2002 Macro Languages AoPL, S'02
Lexical Macro Example
• Square (CPP):
• Work-around: Explicitly add parentheses
#define square(X) X * X
square(z + 1) => z +(1 * z)+ 1
#define square(X) X * X
square(z + 1) => z +(1 * z)+ 1
#define square(X) (X) * (X)#define square(X) (X) * (X)
( )z + 1 z + 1 z + 1*=>square
May 14, 2002 Macro Languages AoPL, S'02
Lexical Macro Example
• Square (CPP):
• Work-around: Explicitly add parentheses
#define square(X) X * X
square(z + 1) => z +(1 * z)+ 1
#define square(X) X * X
square(z + 1) => z +(1 * z)+ 1
#define square(X) (X) * (X)
square(z + 1) => (z + 1)*(z + 1)
#define square(X) (X) * (X)
square(z + 1) => (z + 1)*(z + 1)
( )z + 1 z + 1 z + 1*=>square
May 14, 2002 Macro Languages AoPL, S'02
Lexical Macro Example
• Square (CPP):
• Work-around: Explicitly add parentheses
– Problem: Independent of host language syntax• Unsafe: parse errors discovered at invocation-time
#define square(X) X * X
square(z + 1) => z +(1 * z)+ 1
#define square(X) X * X
square(z + 1) => z +(1 * z)+ 1
#define square(X) (X) * (X)
square(z + 1) => (z + 1)*(z + 1)
#define square(X) (X) * (X)
square(z + 1) => (z + 1)*(z + 1)
( )z + 1 z + 1 z + 1*=>square
May 14, 2002 Macro Languages AoPL, S'02
Syntactic Macros
• Operate on abstract syntax trees:• MSYN: (AST)n AST
• Integrated with host language– typed with host language nonterminals
• Languages:• C++: The C++ template mechanism• Scheme: Scheme’s define-syntax mechanism• JTS: “Jakarta Tool Suite” macro
mechanism• MS2: “Meta Syntactic Macro System”• <bigwig>: The <bigwig> macro language
May 14, 2002 Macro Languages AoPL, S'02
Syntactic Macro Example
• Square (<bigwig>):
macro <exp> square ( <exp E> ) ::= { <E> * <E>
}
macro <exp> square ( <exp E> ) ::= { <E> * <E>
}
May 14, 2002 Macro Languages AoPL, S'02
Syntactic Macro Example
• Square (<bigwig>):
**
exp
exp
Eexp
E
macro <exp> square ( <exp E> ) ::= { <E> * <E>
}
macro <exp> square ( <exp E> ) ::= { <E> * <E>
}~
May 14, 2002 Macro Languages AoPL, S'02
Syntactic Macro Example
• Square (<bigwig>):
**
exp
exp
Eexp
E
y + 1y
+ 1
exp
macro <exp> square ( <exp E> ) ::= { <E> * <E>
}
macro <exp> square ( <exp E> ) ::= { <E> * <E>
}
square( )
~
May 14, 2002 Macro Languages AoPL, S'02
Syntactic Macro Example
• Square (<bigwig>):
**
exp
exp
Eexp
E
**
exp
y + 1y
+ 1 y + 1y
+ 1
exp
Eexp
Ey
+ 1y + 1
exp
macro <exp> square ( <exp E> ) ::= { <E> * <E>
}
macro <exp> square ( <exp E> ) ::= { <E> * <E>
}
square( ) =>
~
May 14, 2002 Macro Languages AoPL, S'02
Lexical Macros
CPP, M4, TEX, (Dylan)
( )z + 1 z + 1 z + 1*=>square
May 14, 2002 Macro Languages AoPL, S'02
CPP
• CPP (“The C Preprocessor”):
• Also as a stand-alone expander:• gcc –E program• cpp program
• Intercepts preprocessor directives “#”:• #define, #undef, #ifdef, #if, #include, #line, ##, …
#define square(X) (X) * (X)square(z + 1) => (z + 1)*(z + 1)
#define square(X) (X) * (X)square(z + 1) => (z + 1)*(z + 1)
May 14, 2002 Macro Languages AoPL, S'02
Fixed Invocation Syntax#define swap(X,Y) { int t=X; X=Y; Y=t; } #define swap(X,Y) { int t=X; X=Y; Y=t; }
May 14, 2002 Macro Languages AoPL, S'02
Fixed Invocation Syntax#define swap(X,Y) { int t=X; X=Y; Y=t; }
if (a>b) swap(a,b); else b = 0;
#define swap(X,Y) { int t=X; X=Y; Y=t; }
if (a>b) swap(a,b); else b = 0;
May 14, 2002 Macro Languages AoPL, S'02
Fixed Invocation Syntax#define swap(X,Y) { int t=X; X=Y; Y=t; }
if (a>b) swap(a,b); else b = 0;
#define swap(X,Y) { int t=X; X=Y; Y=t; }
if (a>b) swap(a,b); else b = 0;
*** test.c:3: parse error before ‘else’*** test.c:3: parse error before ‘else’
May 14, 2002 Macro Languages AoPL, S'02
Fixed Invocation Syntax#define swap(X,Y) { int t=X; X=Y; Y=t; }
if (a>b) swap(a,b); else b = 0;
#define swap(X,Y) { int t=X; X=Y; Y=t; }
if (a>b) swap(a,b); else b = 0;
*** test.c:3: parse error before ‘else’*** test.c:3: parse error before ‘else’
May 14, 2002 Macro Languages AoPL, S'02
Fixed Invocation Syntax#define swap(X,Y) { int t=X; X=Y; Y=t; }
if (a>b) swap(a,b); else b = 0;
#define swap(X,Y) { int t=X; X=Y; Y=t; }
if (a>b) swap(a,b); else b = 0;
*** test.c:3: parse error before ‘else’*** test.c:3: parse error before ‘else’
#define swap(X,Y) do { int t=X; X=Y; Y=t; } while(0)
if (a>b) swap(a,b); else b = 0;
#define swap(X,Y) do { int t=X; X=Y; Y=t; } while(0)
if (a>b) swap(a,b); else b = 0;
May 14, 2002 Macro Languages AoPL, S'02
Fixed Invocation Syntax
– Problem: fixed invocation syntax• same for exp, stm, … M(x,y,z)
#define swap(X,Y) { int t=X; X=Y; Y=t; }
if (a>b) swap(a,b); else b = 0;
#define swap(X,Y) { int t=X; X=Y; Y=t; }
if (a>b) swap(a,b); else b = 0;
*** test.c:3: parse error before ‘else’*** test.c:3: parse error before ‘else’
#define swap(X,Y) do { int t=X; X=Y; Y=t; } while(0)
if (a>b) swap(a,b); else b = 0;
#define swap(X,Y) do { int t=X; X=Y; Y=t; } while(0)
if (a>b) swap(a,b); else b = 0;
May 14, 2002 Macro Languages AoPL, S'02
Body Expansion
• Consider: #define A 87#define B A#undef A#define A 42
B => ???
#define A 87#define B A#undef A#define A 42
B => ???
May 14, 2002 Macro Languages AoPL, S'02
Body Expansion
• Consider:
• Eager expansion (definition-time):
#define A 87#define B A#undef A#define A 42
B => ???
#define A 87#define B A#undef A#define A 42
B => ???
B => 87B => 87
May 14, 2002 Macro Languages AoPL, S'02
Body Expansion
• Consider:
• Eager expansion (definition-time):
• Lazy expansion (invocation-time):
#define A 87#define B A#undef A#define A 42
B => ???
#define A 87#define B A#undef A#define A 42
B => ???
B => 87B => 87
B => AB => A CPP
May 14, 2002 Macro Languages AoPL, S'02
Body Expansion
• Consider:
• Eager expansion (definition-time):
• Lazy expansion (invocation-time):
#define A 87#define B A#undef A#define A 42
B => ???
#define A 87#define B A#undef A#define A 42
B => ???
B => 87B => 87
B => A => 42B => A => 42 CPP
May 14, 2002 Macro Languages AoPL, S'02
Order of Expansion
• Consider: #define id(X) X#define one(X) id(X)#define two a,b
one(two) => ???
#define id(X) X#define one(X) id(X)#define two a,b
one(two) => ???
May 14, 2002 Macro Languages AoPL, S'02
Order of Expansion
• Consider:
• Inner (aka. “AOR”, call-by-value):
#define id(X) X#define one(X) id(X)#define two a,b
one(two) => ???
#define id(X) X#define one(X) id(X)#define two a,b
one(two) => ???
one(two) => one(a,b) => *** arity error ‘one’one(two) => one(a,b) => *** arity error ‘one’
May 14, 2002 Macro Languages AoPL, S'02
Order of Expansion
• Consider:
• Inner (aka. “AOR”, call-by-value):
• Outer (aka. “NOR”, call-by-name):
#define id(X) X#define one(X) id(X)#define two a,b
one(two) => ???
#define id(X) X#define one(X) id(X)#define two a,b
one(two) => ???
one(two) => one(a,b) => *** arity error ‘one’one(two) => one(a,b) => *** arity error ‘one’
one(two) => id(two) => two => a,bone(two) => id(two) => two => a,b
May 14, 2002 Macro Languages AoPL, S'02
CPP: Order of Expansion
• “Argument prescan”:
one(two) one(two)
#define id(X) X#define one(X) id(X)#define two a,b
#define id(X) X#define one(X) id(X)#define two a,b
CPP
May 14, 2002 Macro Languages AoPL, S'02
CPP: Order of Expansion
• “Argument prescan”:
one(two) => id(a,b)one(two) => id(a,b)
#define id(X) X#define one(X) id(X)#define two a,b
#define id(X) X#define one(X) id(X)#define two a,b
CPP
May 14, 2002 Macro Languages AoPL, S'02
CPP: Order of Expansion
• “Argument prescan”:
one(two) => id(a,b) => *** arity error ‘id’one(two) => id(a,b) => *** arity error ‘id’
#define id(X) X#define one(X) id(X)#define two a,b
#define id(X) X#define one(X) id(X)#define two a,b
CPP
May 14, 2002 Macro Languages AoPL, S'02
CPP: Order of Expansion
• “Argument prescan”:
• For piecing together new macro invocations• partly from the arguments, partly from the body:
one(two) => id(a,b) => *** arity error ‘id’one(two) => id(a,b) => *** arity error ‘id’
#define succ(X) ((X) + 1)#define call7(X) X(7)
call7(succ) => succ(7) => ((7) + 1)
#define succ(X) ((X) + 1)#define call7(X) X(7)
call7(succ) => succ(7) => ((7) + 1)
#define id(X) X#define one(X) id(X)#define two a,b
#define id(X) X#define one(X) id(X)#define two a,b
CPP
May 14, 2002 Macro Languages AoPL, S'02
Recursion
• Consider: #define x 1+x
x => ???
#define x 1+x
x => ???
May 14, 2002 Macro Languages AoPL, S'02
Recursion
• Consider:
• Definition-time (static intercept-and-reject):
#define x 1+x
x => ???
#define x 1+x
x => ???
#define x 1*x *** definition-time error!#define x 1*x *** definition-time error!
May 14, 2002 Macro Languages AoPL, S'02
Recursion
• Consider:
• Definition-time (static intercept-and-reject):
• Invocation-time (non-termination):
#define x 1+x
x => ???
#define x 1+x
x => ???
#define x 1*x *** definition-time error!#define x 1*x *** definition-time error!
x => 1+x => 1+1+x => … // loop at compile-time!x => 1+x => 1+1+x => … // loop at compile-time!
May 14, 2002 Macro Languages AoPL, S'02
CPP: Recursion
• “Dynamic intercept-and-ignore”:• Keep stack of macro invocations• Ignore invocations of already invoked macros:
x => 1+xx => 1+x
int x = 2;#define x 1+x
x => ???
int x = 2;#define x 1+x
x => ???
CPP
May 14, 2002 Macro Languages AoPL, S'02
CPP: Recursion
• “Dynamic intercept-and-ignore”:• Keep stack of macro invocations• Ignore invocations of already invoked macros:
x => 1+x // intercept-and-ignore: (at runtime x 3)x => 1+x // intercept-and-ignore: (at runtime x 3)
int x = 2;#define x 1+x
x => ???
int x = 2;#define x 1+x
x => ???
CPP
May 14, 2002 Macro Languages AoPL, S'02
M4
• M4 (Unix Macro Preprocessor):• Originally called “ratfor”:
– “The Rational Fortran Preprocessor”
• Not tailored for a particular language• universal preprocessor
• 30+ built-in constructions:• macro definition, arithmetic evaluation, string
operations, file and system interfacing, …
May 14, 2002 Macro Languages AoPL, S'02
M4 Example
• Implicit arguments (named: $1, …, $9)• Excess arguments ignored• Missing arguments default to “”
• Quoting:– Expansion removes one layer of quotes
• Controls expansion-time (eager by default):
define(‘square’, ‘eval($1 * $1)’)define(‘square’, ‘eval($1 * $1)’)
May 14, 2002 Macro Languages AoPL, S'02
M4 Example
• Implicit arguments (named: $1, …, $9)• Excess arguments ignored• Missing arguments default to “”
• Quoting:– Expansion removes one layer of quotes
• Controls expansion-time (eager by default):
define(‘square’, ‘eval($1 * $1)’)
square(3)
define(‘square’, ‘eval($1 * $1)’)
square(3)
May 14, 2002 Macro Languages AoPL, S'02
M4 Example
• Implicit arguments (named: $1, …, $9)• Excess arguments ignored• Missing arguments default to “”
• Quoting:– Expansion removes one layer of quotes
• Controls expansion-time (eager by default):
define(‘square’, ‘eval($1 * $1)’)
square(3) => eval(3 * 3)
define(‘square’, ‘eval($1 * $1)’)
square(3) => eval(3 * 3)
May 14, 2002 Macro Languages AoPL, S'02
M4 Example
• Implicit arguments (named: $1, …, $9)• Excess arguments ignored• Missing arguments default to “”
• Quoting:– Expansion removes one layer of quotes
• Controls expansion-time (eager by default):
define(‘square’, ‘eval($1 * $1)’)
square(3) => eval(3 * 3) => 9
define(‘square’, ‘eval($1 * $1)’)
square(3) => eval(3 * 3) => 9
May 14, 2002 Macro Languages AoPL, S'02
TEX
• Flexible invocation syntax:• Designed by macro programmer
\def \vector #1[#2..#3]{ // header $({#1}_{#2},\ldots,{#1}_{#3})$ // body}
\def \vector #1[#2..#3]{ // header $({#1}_{#2},\ldots,{#1}_{#3})$ // body}
May 14, 2002 Macro Languages AoPL, S'02
TEX
• Flexible invocation syntax:• Designed by macro programmer
\def \vector #1[#2..#3]{ // header $({#1}_{#2},\ldots,{#1}_{#3})$ // body}
\vector x’[0..n-1]
\def \vector #1[#2..#3]{ // header $({#1}_{#2},\ldots,{#1}_{#3})$ // body}
\vector x’[0..n-1]
May 14, 2002 Macro Languages AoPL, S'02
TEX
• Flexible invocation syntax:• Designed by macro programmer
\def \vector #1[#2..#3]{ // header $({#1}_{#2},\ldots,{#1}_{#3})$ // body}
\vector x’[0..n-1] => $({x’}_{0},\ldots,{x’}_{n-1})$
\def \vector #1[#2..#3]{ // header $({#1}_{#2},\ldots,{#1}_{#3})$ // body}
\vector x’[0..n-1] => $({x’}_{0},\ldots,{x’}_{n-1})$
May 14, 2002 Macro Languages AoPL, S'02
TEX
• Flexible invocation syntax:• Designed by macro programmer
\def \vector #1[#2..#3]{ // header $({#1}_{#2},\ldots,{#1}_{#3})$ // body}
\vector x’[0..n-1] => $({x’}_{0},\ldots,{x’}_{n-1})$
\def \vector #1[#2..#3]{ // header $({#1}_{#2},\ldots,{#1}_{#3})$ // body}
\vector x’[0..n-1] => $({x’}_{0},\ldots,{x’}_{n-1})$
(x’0, …, x’n-1)(x’0, …, x’n-1)
May 14, 2002 Macro Languages AoPL, S'02
TEX
• Flexible invocation syntax:• Designed by macro programmer• Parsing ambiguities (chooses shortest invocation)
\def \vector #1[#2..#3]{ // header $({#1}_{#2},\ldots,{#1}_{#3})$ // body}
\vector x’[0..n-1] => $({x’}_{0},\ldots,{x’}_{n-1})$
\def \vector #1[#2..#3]{ // header $({#1}_{#2},\ldots,{#1}_{#3})$ // body}
\vector x’[0..n-1] => $({x’}_{0},\ldots,{x’}_{n-1})$
(x’0, …, x’n-1)(x’0, …, x’n-1)
May 14, 2002 Macro Languages AoPL, S'02
TEX
• Flexible invocation syntax:• Designed by macro programmer• Parsing ambiguities (chooses shortest invocation)
– Implies:
\def \vector #1[#2..#3]{ // header $({#1}_{#2},\ldots,{#1}_{#3})$ // body}
\vector x’[0..n-1] => $({x’}_{0},\ldots,{x’}_{n-1})$
\def \vector #1[#2..#3]{ // header $({#1}_{#2},\ldots,{#1}_{#3})$ // body}
\vector x’[0..n-1] => $({x’}_{0},\ldots,{x’}_{n-1})$
(x’0, …, x’n-1)(x’0, …, x’n-1)
M M x // invokes M(M), not M(M(x))M M x // invokes M(M), not M(M(x))
May 14, 2002 Macro Languages AoPL, S'02
TEX
• Flexible invocation syntax:• Designed by macro programmer• Parsing ambiguities (chooses shortest invocation)
– Implies:
• Recursion permitted:• TC compile-time language to “break the recursion”
\def \vector #1[#2..#3]{ // header $({#1}_{#2},\ldots,{#1}_{#3})$ // body}
\vector x’[0..n-1] => $({x’}_{0},\ldots,{x’}_{n-1})$
\def \vector #1[#2..#3]{ // header $({#1}_{#2},\ldots,{#1}_{#3})$ // body}
\vector x’[0..n-1] => $({x’}_{0},\ldots,{x’}_{n-1})$
(x’0, …, x’n-1)(x’0, …, x’n-1)
M M x // invokes M(M), not M(M(x))M M x // invokes M(M), not M(M(x))
May 14, 2002 Macro Languages AoPL, S'02
Syntax Macros
C++ templates, Scheme, JTS, MS2
May 14, 2002 Macro Languages AoPL, S'02
C++ Templates
• Intended as a genericity mechanism• But often used as a macro language
May 14, 2002 Macro Languages AoPL, S'02
C++ Templates
• Intended as a genericity mechanism• But often used as a macro language
• Syntax types:• Arguments: id, const, all types (e.g. int)• The result is always a declaration
May 14, 2002 Macro Languages AoPL, S'02
C++ Templates
• Intended as a genericity mechanism• But often used as a macro language
• Syntax types:• Arguments: id, const, all types (e.g. int)• The result is always a declaration
• Multiple definitions:template<int X, int Y> struct M { … }template<int X> struct M { … }
template<int X, int Y> struct M { … }template<int X> struct M { … }
May 14, 2002 Macro Languages AoPL, S'02
C++ Templates
• Intended as a genericity mechanism• But often used as a macro language
• Syntax types:• Arguments: id, const, all types (e.g. int)• The result is always a declaration
• Multiple definitions:
• Constant folding:
template<int X, int Y> struct M { … }template<int X> struct M { … }
template<int X, int Y> struct M { … }template<int X> struct M { … }
(1 + 2) 3 // at compile-time(1 + 2) 3 // at compile-time
May 14, 2002 Macro Languages AoPL, S'02
However…template<int X>struct ct_pow<X,0> { static const int res = 1;};
template<int X, int Y>struct ct_pow { static const int res = X * ct_pow<X,Y-1>::res;};
const int z = ct_pow<5,3>::res; // c-time z 125
template<int X>struct ct_pow<X,0> { static const int res = 1;};
template<int X, int Y>struct ct_pow { static const int res = X * ct_pow<X,Y-1>::res;};
const int z = ct_pow<5,3>::res; // c-time z 125
May 14, 2002 Macro Languages AoPL, S'02
However…
• Constant folding + multiple definition =Turing complete (at compile-time)!
template<int X>struct ct_pow<X,0> { static const int res = 1;};
template<int X, int Y>struct ct_pow { static const int res = X * ct_pow<X,Y-1>::res;};
const int z = ct_pow<5,3>::res; // c-time z 125
template<int X>struct ct_pow<X,0> { static const int res = 1;};
template<int X, int Y>struct ct_pow { static const int res = X * ct_pow<X,Y-1>::res;};
const int z = ct_pow<5,3>::res; // c-time z 125
May 14, 2002 Macro Languages AoPL, S'02
Scheme
• Convenient pattern matching
(define-syntax and (syntax-rules () ((and) #t) ((and b) b) ((and b …) (if b (and …) #f)))))))))))))))))) :)
(define-syntax and (syntax-rules () ((and) #t) ((and b) b) ((and b …) (if b (and …) #f)))))))))))))))))) :)
May 14, 2002 Macro Languages AoPL, S'02
Scheme
• Convenient pattern matching
(define-syntax and (syntax-rules () ((and) #t) ((and b) b) ((and b …) (if b (and …) #f)))))))))))))))))) :)
(and a (if x y z) c) => (if ...)
(define-syntax and (syntax-rules () ((and) #t) ((and b) b) ((and b …) (if b (and …) #f)))))))))))))))))) :)
(and a (if x y z) c) => (if ...)
May 14, 2002 Macro Languages AoPL, S'02
Scheme
• Convenient pattern matching• Multiple definitions:
– Selection: first match in order listed
(define-syntax and (syntax-rules () ((and) #t) ((and b) b) ((and b …) (if b (and …) #f)))))))))))))))))) :)
(and a (if x y z) c) => (if ...)
(define-syntax and (syntax-rules () ((and) #t) ((and b) b) ((and b …) (if b (and …) #f)))))))))))))))))) :)
(and a (if x y z) c) => (if ...)
May 14, 2002 Macro Languages AoPL, S'02
Scheme
• Convenient pattern matching• Multiple definitions:
– Selection: first match in order listed
• Ellipsis list constructor: “…”– More on this later…
(define-syntax and (syntax-rules () ((and) #t) ((and b) b) ((and b …) (if b (and …) #f)))))))))))))))))) :)
(and a (if x y z) c) => (if ...)
(define-syntax and (syntax-rules () ((and) #t) ((and b) b) ((and b …) (if b (and …) #f)))))))))))))))))) :)
(and a (if x y z) c) => (if ...)
May 14, 2002 Macro Languages AoPL, S'02
Hygienic Macro Expansion
• Scheme has automatic -conversion:• Identifier renaming to avoid name capture
(define-syntax gen-inc (syntax-rules () ((gen-inc i) (lambda (x) (+ x i))))
(define-syntax gen-inc (syntax-rules () ((gen-inc i) (lambda (x) (+ x i))))
May 14, 2002 Macro Languages AoPL, S'02
Hygienic Macro Expansion
• Scheme has automatic -conversion:• Identifier renaming to avoid name capture
• let ((x 3)) (
(define-syntax gen-inc (syntax-rules () ((gen-inc i) (lambda (x) (+ x i))))
(define-syntax gen-inc (syntax-rules () ((gen-inc i) (lambda (x) (+ x i))))
)
May 14, 2002 Macro Languages AoPL, S'02
Hygienic Macro Expansion
• Scheme has automatic -conversion:• Identifier renaming to avoid name capture
• let ((x 3)) (• Without -conversion
(define-syntax gen-inc (syntax-rules () ((gen-inc i) (lambda (x) (+ x i))))
(define-syntax gen-inc (syntax-rules () ((gen-inc i) (lambda (x) (+ x i))))
((gen-inc x) 5) ((gen-inc x) 5)
)
May 14, 2002 Macro Languages AoPL, S'02
Hygienic Macro Expansion
• Scheme has automatic -conversion:• Identifier renaming to avoid name capture
• let ((x 3)) (• Without -conversion
(define-syntax gen-inc (syntax-rules () ((gen-inc i) (lambda (x) (+ x i))))
(define-syntax gen-inc (syntax-rules () ((gen-inc i) (lambda (x) (+ x i))))
((gen-inc x) 5) => ((lambda (x) (+ x x)) 5) ((gen-inc x) 5) => ((lambda (x) (+ x x)) 5)
)
May 14, 2002 Macro Languages AoPL, S'02
Hygienic Macro Expansion
• Scheme has automatic -conversion:• Identifier renaming to avoid name capture
• let ((x 3)) (• Without -conversion
(define-syntax gen-inc (syntax-rules () ((gen-inc i) (lambda (x) (+ x i))))
(define-syntax gen-inc (syntax-rules () ((gen-inc i) (lambda (x) (+ x i))))
((gen-inc x) 5) => ((lambda (x) (+ x x)) 5) => 10((gen-inc x) 5) => ((lambda (x) (+ x x)) 5) => 10
)
May 14, 2002 Macro Languages AoPL, S'02
Hygienic Macro Expansion
• Scheme has automatic -conversion:• Identifier renaming to avoid name capture
• let ((x 3)) (• Without -conversion
• With -conversion:
(define-syntax gen-inc (syntax-rules () ((gen-inc i) (lambda (x) (+ x i))))
(define-syntax gen-inc (syntax-rules () ((gen-inc i) (lambda (x) (+ x i))))
((gen-inc x) 5) => ((lambda (x) (+ x x)) 5) => 10((gen-inc x) 5) => ((lambda (x) (+ x x)) 5) => 10
((gen-inc x) 5) ((gen-inc x) 5) )
May 14, 2002 Macro Languages AoPL, S'02
Hygienic Macro Expansion
• Scheme has automatic -conversion:• Identifier renaming to avoid name capture
• let ((x 3)) (• Without -conversion
• With -conversion:
(define-syntax gen-inc (syntax-rules () ((gen-inc i) (lambda (x) (+ x i))))
(define-syntax gen-inc (syntax-rules () ((gen-inc i) (lambda (x) (+ x i))))
((gen-inc x) 5) => ((lambda (x) (+ x x)) 5) => 10((gen-inc x) 5) => ((lambda (x) (+ x x)) 5) => 10
((gen-inc x) 5) => ((lambda (x’) (+ x’ x)) 5) ((gen-inc x) 5) => ((lambda (x’) (+ x’ x)) 5) )
May 14, 2002 Macro Languages AoPL, S'02
Hygienic Macro Expansion
• Scheme has automatic -conversion:• Identifier renaming to avoid name capture
• let ((x 3)) (• Without -conversion
• With -conversion:
(define-syntax gen-inc (syntax-rules () ((gen-inc i) (lambda (x) (+ x i))))
(define-syntax gen-inc (syntax-rules () ((gen-inc i) (lambda (x) (+ x i))))
((gen-inc x) 5) => ((lambda (x) (+ x x)) 5) => 10((gen-inc x) 5) => ((lambda (x) (+ x x)) 5) => 10
((gen-inc x) 5) => ((lambda (x’) (+ x’ x)) 5) => 8((gen-inc x) 5) => ((lambda (x’) (+ x’ x)) 5) => 8 )
May 14, 2002 Macro Languages AoPL, S'02
Compile-Time Programming
• Basically compile-time functions on S-exps• Lazy (inv.-time) body expansion ( evaluation):
(define-syntax m (syntax-rules () ((m) (if 1 2 3 4))))
(m) => // *** parse error: too many arg’s to if
(define-syntax m (syntax-rules () ((m) (if 1 2 3 4))))
(m) => // *** parse error: too many arg’s to if
May 14, 2002 Macro Languages AoPL, S'02
Compile-Time Programming
• Basically compile-time functions on S-exps• Lazy (inv.-time) body expansion ( evaluation):
• Non-termination possible:
(define-syntax ct-loop (syntax-rules () ((ct-loop) (ct-loop))))
(define-syntax ct-loop (syntax-rules () ((ct-loop) (ct-loop))))
(define-syntax m (syntax-rules () ((m) (if 1 2 3 4))))
(m) => // *** parse error: too many arg’s to if
(define-syntax m (syntax-rules () ((m) (if 1 2 3 4))))
(m) => // *** parse error: too many arg’s to if
May 14, 2002 Macro Languages AoPL, S'02
Compile-Time Programming
• Basically compile-time functions on S-exps• Lazy (inv.-time) body expansion ( evaluation):
• Non-termination possible:
(define-syntax ct-loop (syntax-rules () ((ct-loop) (ct-loop))))
(ct-loop) => // c-time loop
(define-syntax ct-loop (syntax-rules () ((ct-loop) (ct-loop))))
(ct-loop) => // c-time loop
(define-syntax m (syntax-rules () ((m) (if 1 2 3 4))))
(m) => // *** parse error: too many arg’s to if
(define-syntax m (syntax-rules () ((m) (if 1 2 3 4))))
(m) => // *** parse error: too many arg’s to if
May 14, 2002 Macro Languages AoPL, S'02
JTS
• JTS (“Jakarta Tool Suite”):
macro swap(AST_QualifiedName x, AST_QualifiedName y) local temp // explicit -conversion stm{ // body constructor: stm integer temp = x; x = y; y = temp; }stm
macro swap(AST_QualifiedName x, AST_QualifiedName y) local temp // explicit -conversion stm{ // body constructor: stm integer temp = x; x = y; y = temp; }stm
May 14, 2002 Macro Languages AoPL, S'02
JTS
• JTS (“Jakarta Tool Suite”):• Argument types: name, exp, stm, decl, class, type• Result types: “exp”, “stm”, “mth”, “cls”, “prg”
macro swap(AST_QualifiedName x, AST_QualifiedName y) local temp // explicit -conversion stm{ // body constructor: stm integer temp = x; x = y; y = temp; }stm
macro swap(AST_QualifiedName x, AST_QualifiedName y) local temp // explicit -conversion stm{ // body constructor: stm integer temp = x; x = y; y = temp; }stm
May 14, 2002 Macro Languages AoPL, S'02
JTS
• JTS (“Jakarta Tool Suite”):• Argument types: name, exp, stm, decl, class, type• Result types: “exp”, “stm”, “mth”, “cls”, “prg”
– Fixed invocation syntax:
macro swap(AST_QualifiedName x, AST_QualifiedName y) local temp // explicit -conversion stm{ // body constructor: stm integer temp = x; x = y; y = temp; }stm
macro swap(AST_QualifiedName x, AST_QualifiedName y) local temp // explicit -conversion stm{ // body constructor: stm integer temp = x; x = y; y = temp; }stm
#swap(C.x, y);#swap(C.x, y);
May 14, 2002 Macro Languages AoPL, S'02
JTS
• JTS (“Jakarta Tool Suite”):• Argument types: name, exp, stm, decl, class, type• Result types: “exp”, “stm”, “mth”, “cls”, “prg”
– Fixed invocation syntax: – Safe:
• Guaranteed termination, only generate legal syntax
macro swap(AST_QualifiedName x, AST_QualifiedName y) local temp // explicit -conversion stm{ // body constructor: stm integer temp = x; x = y; y = temp; }stm
macro swap(AST_QualifiedName x, AST_QualifiedName y) local temp // explicit -conversion stm{ // body constructor: stm integer temp = x; x = y; y = temp; }stm
#swap(C.x, y);#swap(C.x, y);
May 14, 2002 Macro Languages AoPL, S'02
MS2
• MS2 (“Meta Syntactic Macro System”):• Turing complete AST programming language
– for computing C parse trees at compile-time
syntax decl myenum[] {| $$id::name { $$+\,id::ids }; |} { return (list(`[enum $name $ids;], `[$(symbolconc(“print_”,name))(arg) { switch (arg) $(map((| @id id; `{ case $id: printf(“%s”, $(pstring(id)));}), ids))}));}
syntax decl myenum[] {| $$id::name { $$+\,id::ids }; |} { return (list(`[enum $name $ids;], `[$(symbolconc(“print_”,name))(arg) { switch (arg) $(map((| @id id; `{ case $id: printf(“%s”, $(pstring(id)));}), ids))}));}
May 14, 2002 Macro Languages AoPL, S'02
MS2
• MS2 (“Meta Syntactic Macro System”):• Turing complete AST programming language
– for computing C parse trees at compile-time
syntax decl myenum[] {| $$id::name { $$+\,id::ids }; |} { return (list(`[enum $name $ids;], `[$(symbolconc(“print_”,name))(arg) { switch (arg) $(map((| @id id; `{ case $id: printf(“%s”, $(pstring(id)));}), ids))}));}myenum fruit { apple, orange }; print_fruit(apple);
syntax decl myenum[] {| $$id::name { $$+\,id::ids }; |} { return (list(`[enum $name $ids;], `[$(symbolconc(“print_”,name))(arg) { switch (arg) $(map((| @id id; `{ case $id: printf(“%s”, $(pstring(id)));}), ids))}));}myenum fruit { apple, orange }; print_fruit(apple);
May 14, 2002 Macro Languages AoPL, S'02
• 8 Languages:• { CPP, M4, TEX, Dylan, C++ Templates, Scheme, JTS, MS2 }
Macro Survey
May 14, 2002 Macro Languages AoPL, S'02
• 8 Languages:• { CPP, M4, TEX, Dylan, C++ Templates, Scheme, JTS, MS2 }
• 31 Properties:• { Level of operation, body expansion, order of expansion, … }
Macro Survey
May 14, 2002 Macro Languages AoPL, S'02
• 8 Languages:• { CPP, M4, TEX, Dylan, C++ Templates, Scheme, JTS, MS2 }
• 31 Properties:• { Level of operation, body expansion, order of expansion, … }
Macro Survey
May 14, 2002 Macro Languages AoPL, S'02
<bigwig>
• Declarative:• Based entirely on simple concepts: grammars and
substitution
• Safe:• Guaranteed termination• Only generate legal (and -converted) syntax
• Flexible:• All (55) nonterminals may be arg and return types• Invocation syntax design (guides parsing)
May 14, 2002 Macro Languages AoPL, S'02
On all (55) Nonterminals
• <floatconst>:macro <floatconst> pi ::= { 3.1415926 }macro <floatconst> pi ::= { 3.1415926 }
May 14, 2002 Macro Languages AoPL, S'02
On all (55) Nonterminals
• <floatconst>:
• <stm>:
macro <floatconst> pi ::= { 3.1415926 }macro <floatconst> pi ::= { 3.1415926 }
macro <stm> maybe <stm S> ::= { if (random(2)==0) <S>
}
macro <stm> maybe <stm S> ::= { if (random(2)==0) <S>
}
May 14, 2002 Macro Languages AoPL, S'02
On all (55) Nonterminals
• <floatconst>:
• <stm>:
• <regexp>:
macro <floatconst> pi ::= { 3.1415926 }macro <floatconst> pi ::= { 3.1415926 }
macro <stm> maybe <stm S> ::= { if (random(2)==0) <S>
}
macro <stm> maybe <stm S> ::= { if (random(2)==0) <S>
}
macro <regexp> plus ( <regexp R> ) ::= { concat(star(<R>),<R>)
}
macro <regexp> plus ( <regexp R> ) ::= { concat(star(<R>),<R>)
}
May 14, 2002 Macro Languages AoPL, S'02
Example: repeat
• Invocation syntax design:• Guides the parser
macro <stm> repeat <exp E> until ( <stm S> ) ; ::= { {
<S>
while (<E>) <S>
}
}
macro <stm> repeat <exp E> until ( <stm S> ) ; ::= { {
<S>
while (<E>) <S>
}
}
May 14, 2002 Macro Languages AoPL, S'02
Example: repeat
• Invocation syntax design:• Guides the parser
• Code duplication: <S>• Worst-case: O(2n)
macro <stm> repeat <exp E> until ( <stm S> ) ; ::= { {
<S>
while (<E>) <S>
}
}
macro <stm> repeat <exp E> until ( <stm S> ) ; ::= { {
<S>
while (<E>) <S>
}
}
May 14, 2002 Macro Languages AoPL, S'02
Example: repeat
macro <stm> repeat <exp E> until ( <stm S> ) ; ::= { {
bool first = true;
while (first || !<E>) {
<S>
first = false;
}
}
}
macro <stm> repeat <exp E> until ( <stm S> ) ; ::= { {
bool first = true;
while (first || !<E>) {
<S>
first = false;
}
}
}
May 14, 2002 Macro Languages AoPL, S'02
Example: repeat
-Conversion:• first => first_87
macro <stm> repeat <exp E> until ( <stm S> ) ; ::= { {
bool first = true;
while (first || !<E>) {
<S>
first = false;
}
}
}
macro <stm> repeat <exp E> until ( <stm S> ) ; ::= { {
bool first = true;
while (first || !<E>) {
<S>
first = false;
}
}
}
May 14, 2002 Macro Languages AoPL, S'02
Multiple Definitions
• Macros with same name: si/si-sinon:
• Parsing:• Which macro to select?
macro <stm> si (<exp E>) <stm S> ::= { if (<E>) <S>
}
macro <stm> si (<exp E>) <stm S> sinon <stm T> ::= { if (<E>) <S> else <T>
}
macro <stm> si (<exp E>) <stm S> ::= { if (<E>) <S>
}
macro <stm> si (<exp E>) <stm S> sinon <stm T> ::= { if (<E>) <S> else <T>
}
May 14, 2002 Macro Languages AoPL, S'02
Specificity Parsing
macro <exp> select <id I> from <exp E> where <exp E2>
macro <exp> select all from <exp E> where <exp E2>
macro <exp> select <id I> from <exp E> where <exp E2>
macro <exp> select all from <exp E> where <exp E2>
May 14, 2002 Macro Languages AoPL, S'02
Specificity Parsing
• Challenge rounds:– Select most specific productions (wrt. FIRST sets)
• Resolves many ambiguities• Independent of definition-order• Overloading• Avoids keywordification• Commit no branch explosion, no backtracking
macro <exp> select <id I> from <exp E> where <exp E2>
macro <exp> select all from <exp E> where <exp E2>
macro <exp> select <id I> from <exp E> where <exp E2>
macro <exp> select all from <exp E> where <exp E2>
May 14, 2002 Macro Languages AoPL, S'02
Pretty Print and Error Reporting
• Pretty Printing:• Terminal printers:
ASCII, LA EX, HTML(+/- expansion)
T
May 14, 2002 Macro Languages AoPL, S'02
Pretty Print and Error Reporting
• Pretty Printing:• Terminal printers:
ASCII, LA EX, HTML(+/- expansion)
• Error Reporting:• stdout, HTML*** symbol errors:*** test.wig:7: Identifier ‘inf’ not declared in macro argument ‘S’ in macro invocation ‘reader’ (test.wig:7) defined in [std.wigmac:44]
*** symbol errors:*** test.wig:7: Identifier ‘inf’ not declared in macro argument ‘S’ in macro invocation ‘reader’ (test.wig:7) defined in [std.wigmac:44]
T
May 14, 2002 Macro Languages AoPL, S'02
Concurrency Stack
May 14, 2002 Macro Languages AoPL, S'02
Representation
macro <ids> MXY ( <ids Is> ) ::= { X, <Is>, Y }
A, MXY(B, C), D A, X, B, C, Y, D
macro <ids> MXY ( <ids Is> ) ::= { X, <Is>, Y }
A, MXY(B, C), D A, X, B, C, Y, D
May 14, 2002 Macro Languages AoPL, S'02
Representation
macro <ids> MXY ( <ids Is> ) ::= { X, <Is>, Y }
A, MXY(B, C), D A, X, B, C, Y, D
macro <ids> MXY ( <ids Is> ) ::= { X, <Is>, Y }
A, MXY(B, C), D A, X, B, C, Y, D
May 14, 2002 Macro Languages AoPL, S'02
Representation
macro <ids> MXY ( <ids Is> ) ::= { X, <Is>, Y }
A, MXY(B, C), D A, X, B, C, Y, D
macro <ids> MXY ( <ids Is> ) ::= { X, <Is>, Y }
A, MXY(B, C), D A, X, B, C, Y, D
• “Weaving” yields transparency!
weave
May 14, 2002 Macro Languages AoPL, S'02
Metamorphic Syntax Macros
May 14, 2002 Macro Languages AoPL, S'02
Argument Structure
• Many variants of an abstraction?
enum { zero };
enum { zero, one };
enum { zero, one, two };
…
enum { zero };
enum { zero, one };
enum { zero, one, two };
…
May 14, 2002 Macro Languages AoPL, S'02
Argument Structure
• Many variants of an abstraction?
– Syntax:• Argument structure?
– Transformation:• Specification?
enum { zero };
enum { zero, one };
enum { zero, one, two };
…
enum { zero };
enum { zero, one };
enum { zero, one, two };
…
May 14, 2002 Macro Languages AoPL, S'02
Argument Structure
• Many variants of an abstraction?
– Syntax:• Argument structure?
– Transformation:• Specification?
enum { zero };
enum { zero, one };
enum { zero, one, two };
…
enum { zero };
enum { zero, one };
enum { zero, one, two };
…
Safety?
May 14, 2002 Macro Languages AoPL, S'02
Multiple Definitionsmacro <decls> enum { <id X> }; ::= { const int <X> = 0;
}
macro <decls> enum { <id X>, <id Y> }; ::= { const int <X> = 0;
const int <Y> = 1;
}
macro <decls> enum { <id X>, <id Y>, <id Z> }; ::= { const int <X> = 0;
const int <Y> = 1;
const int <Z> = 2;
}
macro <decls> enum { <id X> }; ::= { const int <X> = 0;
}
macro <decls> enum { <id X>, <id Y> }; ::= { const int <X> = 0;
const int <Y> = 1;
}
macro <decls> enum { <id X>, <id Y>, <id Z> }; ::= { const int <X> = 0;
const int <Y> = 1;
const int <Z> = 2;
}
May 14, 2002 Macro Languages AoPL, S'02
Multiple Definitionsmacro <decls> enum { <id X> }; ::= {…}
macro <decls> enum { <id X>, <id Y> }; ::= {…}
macro <decls> enum { <id X>, <id Y>, <id Z> }; ::= {…}
macro <decls> enum { <id X> }; ::= {…}
macro <decls> enum { <id X>, <id Y> }; ::= {…}
macro <decls> enum { <id X>, <id Y>, <id Z> }; ::= {…}
May 14, 2002 Macro Languages AoPL, S'02
Multiple Definitions
decls enum { id } ;
enum { id , id } ;
enum { id , id , id } ;
decls enum { id } ;
enum { id , id } ;
enum { id , id , id } ;
macro <decls> enum { <id X> }; ::= {…}
macro <decls> enum { <id X>, <id Y> }; ::= {…}
macro <decls> enum { <id X>, <id Y>, <id Z> }; ::= {…}
macro <decls> enum { <id X> }; ::= {…}
macro <decls> enum { <id X>, <id Y> }; ::= {…}
macro <decls> enum { <id X>, <id Y>, <id Z> }; ::= {…}
• Corresponds togrammar extension:
May 14, 2002 Macro Languages AoPL, S'02
Multiple Definitions
decls enum { id } ;
enum { id , id } ;
enum { id , id , id } ;
decls enum { id } ;
enum { id , id } ;
enum { id , id , id } ;
macro <decls> enum { <id X> }; ::= {…}
macro <decls> enum { <id X>, <id Y> }; ::= {…}
macro <decls> enum { <id X>, <id Y>, <id Z> }; ::= {…}
macro <decls> enum { <id X> }; ::= {…}
macro <decls> enum { <id X>, <id Y> }; ::= {…}
macro <decls> enum { <id X>, <id Y>, <id Z> }; ::= {…}
• Corresponds togrammar extension:
Three unrelated productions
May 14, 2002 Macro Languages AoPL, S'02
Multiple Definitions
decls enum { id } ;
enum { id , id } ;
enum { id , id , id } ;
decls enum { id } ;
enum { id , id } ;
enum { id , id , id } ;
macro <decls> enum { <id X> }; ::= {…}
macro <decls> enum { <id X>, <id Y> }; ::= {…}
macro <decls> enum { <id X>, <id Y>, <id Z> }; ::= {…}
macro <decls> enum { <id X> }; ::= {…}
macro <decls> enum { <id X>, <id Y> }; ::= {…}
macro <decls> enum { <id X>, <id Y>, <id Z> }; ::= {…}
• Corresponds togrammar extension:
• Problems:• Only fixed (finite) extent
Three unrelated productions
May 14, 2002 Macro Languages AoPL, S'02
Multiple Definitions
decls enum { id } ;
enum { id , id } ;
enum { id , id , id } ;
decls enum { id } ;
enum { id , id } ;
enum { id , id , id } ;
macro <decls> enum { <id X> }; ::= {…}
macro <decls> enum { <id X>, <id Y> }; ::= {…}
macro <decls> enum { <id X>, <id Y>, <id Z> }; ::= {…}
macro <decls> enum { <id X> }; ::= {…}
macro <decls> enum { <id X>, <id Y> }; ::= {…}
macro <decls> enum { <id X>, <id Y>, <id Z> }; ::= {…}
• Corresponds togrammar extension:
• Problems:• Only fixed (finite) extent• Highly redundant
Three unrelated productions
May 14, 2002 Macro Languages AoPL, S'02
Lists
• Scheme:• Special list constructor: “...”
decls ( enum id * )decls ( enum id * )
(define-syntax and (syntax-rules () ((and) #t) ((and b) b) ((and b ...) (if b (and ...) #f))))
(define-syntax and (syntax-rules () ((and) #t) ((and b) b) ((and b ...) (if b (and ...) #f))))
(enum x y z)(enum x y z)
May 14, 2002 Macro Languages AoPL, S'02
e-BNF
• MS2:• Regular expressions:
– Optionals “?”, lists “+, *”, tuples “{…}”, t-sep. lists “\+t”
syntax decl myenum[] {| $$id::name { $$+\,id::ids }; |} { return (list(`[enum $name $ids;], `[$(symbolconc(“print_”,name))(arg) { switch (arg) $(map((| @id id; `{ case $id: printf(“%s”, $(pstring(id)));}), ids))}));
syntax decl myenum[] {| $$id::name { $$+\,id::ids }; |} { return (list(`[enum $name $ids;], `[$(symbolconc(“print_”,name))(arg) { switch (arg) $(map((| @id id; `{ case $id: printf(“%s”, $(pstring(id)));}), ids))}));
decls enum { id , } ;decls enum { id
, } ;
myenum fruit { apple, orange }; print_fruit(apple);myenum fruit { apple, orange }; print_fruit(apple);
May 14, 2002 Macro Languages AoPL, S'02
Grammar
• Dylan (and JSE, a Java adaptation):• Describe argument syntax via
user-defined nonterminals
decls enum { id enums } ;
enums , id enums
decls enum { id enums } ;
enums , id enums
May 14, 2002 Macro Languages AoPL, S'02
Grammar
• Dylan (and JSE, a Java adaptation):• Describe argument syntax via
user-defined nonterminals
– Unsafe:• Transformations return lexical token sequences• Return user defined abstract syntax trees?
decls enum { id enums } ;
enums , id enums
decls enum { id enums } ;
enums , id enums
May 14, 2002 Macro Languages AoPL, S'02
Metamorphisms
• <bigwig>:• Attach host nonterminals to user def’d nonterminals• Specify morphing (into host syntax) inductively
metamorph <decls> enums;
macro <decls> … <enums: Ds> … ::= { … <Ds> … }
morph <enums> … ::= { … }
metamorph <decls> enums;
macro <decls> … <enums: Ds> … ::= { … <Ds> … }
morph <enums> … ::= { … }
decls enum { id enums } ;
enums , id enums
decls enum { id enums } ;
enums , id enums
May 14, 2002 Macro Languages AoPL, S'02
Example: enum
• Ideally:
enum { a, b, c };enum { a, b, c };const int a = 0;const int b = 1;const int c = 2;
const int a = 0;const int b = 1;const int c = 2;
=>
May 14, 2002 Macro Languages AoPL, S'02
Example: enum
• Ideally:
• But requires compile-time evaluation: 0, 1, 2, …
enum { a, b, c };enum { a, b, c };const int a = 0;const int b = 1;const int c = 2;
const int a = 0;const int b = 1;const int c = 2;
=>
May 14, 2002 Macro Languages AoPL, S'02
Example: enum
• Ideally:
• But requires compile-time evaluation: 0, 1, 2, …
• Instead generate:
enum { a, b, c };enum { a, b, c };const int a = 0;const int b = 1;const int c = 2;
const int a = 0;const int b = 1;const int c = 2;
=>
enum { a, b, c };enum { a, b, c };const int e = 0;const int a = e++;const int b = e++;const int c = e++;
const int e = 0;const int a = e++;const int b = e++;const int c = e++;
=>
May 14, 2002 Macro Languages AoPL, S'02
Example: enummetamorph <decls> enums;metamorph <decls> enums;
decls enum { id enums } ;
enums , id enums
decls enum { id enums } ;
enums , id enums
May 14, 2002 Macro Languages AoPL, S'02
Example: enummetamorph <decls> enums;
macro <decls> enum { <id X> <enums: Ds> } ; ::= { int e = 0;
const int <X> = e++;
<Ds>
}
metamorph <decls> enums;
macro <decls> enum { <id X> <enums: Ds> } ; ::= { int e = 0;
const int <X> = e++;
<Ds>
}
decls enum { id enums } ;
enums , id enums
decls enum { id enums } ;
enums , id enums
May 14, 2002 Macro Languages AoPL, S'02
Example: enummetamorph <decls> enums;
macro <decls> enum { <id X> <enums: Ds> } ; ::= { int e = 0;
const int <X> = e++;
<Ds>
}
morph <enums> , <id X> <enums: Ds> ::= { const int <X> = e++;
<Ds>
}
metamorph <decls> enums;
macro <decls> enum { <id X> <enums: Ds> } ; ::= { int e = 0;
const int <X> = e++;
<Ds>
}
morph <enums> , <id X> <enums: Ds> ::= { const int <X> = e++;
<Ds>
}
decls enum { id enums } ;
enums , id enums
decls enum { id enums } ;
enums , id enums
May 14, 2002 Macro Languages AoPL, S'02
Example: enummetamorph <decls> enums;
macro <decls> enum { <id X> <enums: Ds> } ; ::= { int e = 0;
const int <X> = e++;
<Ds>
}
morph <enums> , <id X> <enums: Ds> ::= { const int <X> = e++;
<Ds>
}
morph <enums> ::= { }
metamorph <decls> enums;
macro <decls> enum { <id X> <enums: Ds> } ; ::= { int e = 0;
const int <X> = e++;
<Ds>
}
morph <enums> , <id X> <enums: Ds> ::= { const int <X> = e++;
<Ds>
}
morph <enums> ::= { }
decls enum { id enums } ;
enums , id enums
decls enum { id enums } ;
enums , id enums
May 14, 2002 Macro Languages AoPL, S'02
Guaranteed Termination
• Termination proof:• Transition system + termination function
0 1 2 …(0) > (1) > (2) > …
0 1 2 …(0) > (1) > (2) > …
May 14, 2002 Macro Languages AoPL, S'02
Guaranteed Termination
• Termination proof:• Transition system + termination function
– States: ( , ) (T N)* sentential form T* input string
0 1 2 …(0) > (1) > (2) > …
0 1 2 …(0) > (1) > (2) > …
May 14, 2002 Macro Languages AoPL, S'02
Guaranteed Termination
• Termination proof:• Transition system + termination function
– States: ( , ) (T N)* sentential form T* input string
– Transitions:– (t , t ) (, )– (N , ) ( , ) , if N
0 1 2 …(0) > (1) > (2) > …
0 1 2 …(0) > (1) > (2) > …
May 14, 2002 Macro Languages AoPL, S'02
Guaranteed Termination
• Termination proof:• Transition system + termination function
– Termination function:( , ) =
0 1 2 …(0) > (1) > (2) > …
0 1 2 …(0) > (1) > (2) > …
May 14, 2002 Macro Languages AoPL, S'02
Guaranteed Termination
• Termination proof:• Transition system + termination function
– Termination function:( , ) = < | |, || || >
0 1 2 …(0) > (1) > (2) > …
0 1 2 …(0) > (1) > (2) > …
May 14, 2002 Macro Languages AoPL, S'02
Guaranteed Termination
• Termination proof:• Transition system + termination function
– Termination function:( , ) = < | |, || || >
– || t || = 0
0 1 2 …(0) > (1) > (2) > …
0 1 2 …(0) > (1) > (2) > …
May 14, 2002 Macro Languages AoPL, S'02
Guaranteed Termination
• Termination proof:• Transition system + termination function
– Termination function:( , ) = < | |, || || >
– || t || = 0
– || N0 || = k+1
0 1 2 …(0) > (1) > (2) > …
0 1 2 …(0) > (1) > (2) > …
N0 N1 1 N2 2 1 … Nk k … 1N0 N1 1 N2 2 1 … Nk k … 1
longest
May 14, 2002 Macro Languages AoPL, S'02
Guaranteed Termination
• Termination proof:• Transition system + termination function
– Termination function:( , ) = < | |, || || > lexicographically
ordered– || t || = 0
– || N0 || = k+1
0 1 2 …(0) > (1) > (2) > …
0 1 2 …(0) > (1) > (2) > …
N0 N1 1 N2 2 1 … Nk k … 1N0 N1 1 N2 2 1 … Nk k … 1
longest
May 14, 2002 Macro Languages AoPL, S'02
Metamorph Wellformedness
• Check at definition time:• No left-recursion
– guarantees parser termination:
xlist xlist X
xlist xlist X
xlist X xlist
xlist X xlist
May 14, 2002 Macro Languages AoPL, S'02
Metamorph Wellformedness
• Check at definition time:• No left-recursion
– guarantees parser termination:
• Derivability– metamorphisms must derive something finite:
xlist xlist X
xlist xlist X
xlist X xlist
xlist X xlist
xlist X xlist
xlist X xlist
xlist X xlistxlist X xlist
May 14, 2002 Macro Languages AoPL, S'02
Example: switchmetamorph <stm> swb;
metamorph <stm> swb;
stm switch ( exp ) { swb }
swb case exp : stms break; swb
case exp : stms break;
stm switch ( exp ) { swb }
swb case exp : stms break; swb
case exp : stms break;
May 14, 2002 Macro Languages AoPL, S'02
Example: switchmetamorph <stm> swb;
macro <stm> switch ( <exp E> ) { <swb: S> } ::= { { var x = <E>; <S> }}
metamorph <stm> swb;
macro <stm> switch ( <exp E> ) { <swb: S> } ::= { { var x = <E>; <S> }}
stm switch ( exp ) { swb }
swb case exp : stms break; swb
case exp : stms break;
stm switch ( exp ) { swb }
swb case exp : stms break; swb
case exp : stms break;
May 14, 2002 Macro Languages AoPL, S'02
Example: switchmetamorph <stm> swb;
macro <stm> switch ( <exp E> ) { <swb: S> } ::= { { var x = <E>; <S> }}morph <swb> case <exp E> : <stms Ss> break; <swb: S> ::= { if (x == <E>) { <Ss> } else <S>}
metamorph <stm> swb;
macro <stm> switch ( <exp E> ) { <swb: S> } ::= { { var x = <E>; <S> }}morph <swb> case <exp E> : <stms Ss> break; <swb: S> ::= { if (x == <E>) { <Ss> } else <S>}
stm switch ( exp ) { swb }
swb case exp : stms break; swb
case exp : stms break;
stm switch ( exp ) { swb }
swb case exp : stms break; swb
case exp : stms break;
May 14, 2002 Macro Languages AoPL, S'02
Example: switchmetamorph <stm> swb;
macro <stm> switch ( <exp E> ) { <swb: S> } ::= { { var x = <E>; <S> }}morph <swb> case <exp E> : <stms Ss> break; <swb: S> ::= { if (x == <E>) { <Ss> } else <S>}morph <swb> case <exp E> : <stms Ss> break; ::= { if (x == <E>) { <Ss> }}
metamorph <stm> swb;
macro <stm> switch ( <exp E> ) { <swb: S> } ::= { { var x = <E>; <S> }}morph <swb> case <exp E> : <stms Ss> break; <swb: S> ::= { if (x == <E>) { <Ss> } else <S>}morph <swb> case <exp E> : <stms Ss> break; ::= { if (x == <E>) { <Ss> }}
stm switch ( exp ) { swb }
swb case exp : stms break; swb
case exp : stms break;
stm switch ( exp ) { swb }
swb case exp : stms break; swb
case exp : stms break;
May 14, 2002 Macro Languages AoPL, S'02
Example: reserve
=>
stm reserve ( res ) stm
res id res
stm reserve ( res ) stm
res id res
reserve ( a b c ) ...;reserve ( a b c ) ...;
acquire(a); acquire(b); acquire(c); ...; release(c); release(b);release(a);
acquire(a); acquire(b); acquire(c); ...; release(c); release(b);release(a);
May 14, 2002 Macro Languages AoPL, S'02
Example: reserve
• Requires non-local transformations• e.g. “b” must generate both “acquire(b)” and
“release(b)” at different locations
=>
stm reserve ( res ) stm
res id res
stm reserve ( res ) stm
res id res
reserve ( a b c ) ...;reserve ( a b c ) ...;
acquire(a); acquire(b); acquire(c); ...; release(c); release(b);release(a);
acquire(a); acquire(b); acquire(c); ...; release(c); release(b);release(a);
May 14, 2002 Macro Languages AoPL, S'02
Multiple Results
• Extend metamorphisms:• Attach host nonterminals to user def’d nonterminals
metamorph <stms,stms> res;
macro <stm> … <res: As,Rs> … ::= { … <As> … <Rs> … }
morph <res> … ::= { … } { … }
metamorph <stms,stms> res;
macro <stm> … <res: As,Rs> … ::= { … <As> … <Rs> … }
morph <res> … ::= { … } { … }
stm reserve ( res ) stm
res id res
stm reserve ( res ) stm
res id res
May 14, 2002 Macro Languages AoPL, S'02
Example: reserve
metamorph <stms,stms> res;
macro <stm> reserve ( <res: As,Rs> ) <stm S> ::= { { <As> <S> <Rs> }
}
morph <res> <id X> <res: As,Rs> ::= { acquire(<X>); <As>
} {
<Rs> release(<X>);
}
morph <res> ::= { } { }
metamorph <stms,stms> res;
macro <stm> reserve ( <res: As,Rs> ) <stm S> ::= { { <As> <S> <Rs> }
}
morph <res> <id X> <res: As,Rs> ::= { acquire(<X>); <As>
} {
<Rs> release(<X>);
}
morph <res> ::= { } { }
stm reserve ( res ) stm
res id res
stm reserve ( res ) stm
res id res
May 14, 2002 Macro Languages AoPL, S'02
Example: enum (cont’d)
enum { a, b, c };enum { a, b, c };const int e = 0;const int a = e++;const int b = e++;const int c = e++;
const int e = 0;const int a = e++;const int b = e++;const int c = e++;
=>
May 14, 2002 Macro Languages AoPL, S'02
Example: enum (cont’d)
• Suppose no initialization expressions• Instead generate:
– CPS Style:• Send initialization expression to metamorphism
enum { a, b, c };enum { a, b, c };const int e = 0;const int a = e++;const int b = e++;const int c = e++;
const int e = 0;const int a = e++;const int b = e++;const int c = e++;
=>
enum { a, b, c };enum { a, b, c };const int a = 0;const int b = 1;const int c = 1+1;
const int a = 0;const int b = 1;const int c = 1+1;
=>
May 14, 2002 Macro Languages AoPL, S'02
Metamorph Argumentsmetamorph <decls> enums(<exp K>);
macro <decls> enum { <id X> <enums: Ds>({1}) }; … { const int <X> = 0;
<Ds>
}
morph <enums> , <id X> <enums: Ds>({<K>+1}) ::= { const int <X> = <K>;
<Ds>
}
morph <enums> ::= { }
metamorph <decls> enums(<exp K>);
macro <decls> enum { <id X> <enums: Ds>({1}) }; … { const int <X> = 0;
<Ds>
}
morph <enums> , <id X> <enums: Ds>({<K>+1}) ::= { const int <X> = <K>;
<Ds>
}
morph <enums> ::= { }
decls enum { id enums } ;
enums , id enums
decls enum { id enums } ;
enums , id enums
May 14, 2002 Macro Languages AoPL, S'02
Metamorph Argumentsmetamorph <decls> enums(<exp K>);
macro <decls> enum { <id X> <enums: Ds>({1}) }; … { const int <X> = 0;
<Ds>
}
morph <enums> , <id X> <enums: Ds>({<K>+1}) ::= { const int <X> = <K>;
<Ds>
}
morph <enums> ::= { }
metamorph <decls> enums(<exp K>);
macro <decls> enum { <id X> <enums: Ds>({1}) }; … { const int <X> = 0;
<Ds>
}
morph <enums> , <id X> <enums: Ds>({<K>+1}) ::= { const int <X> = <K>;
<Ds>
}
morph <enums> ::= { }
decls enum { id enums } ;
enums , id enums
decls enum { id enums } ;
enums , id enums
enum {…x…}; => … const int x = 1+1+1+1; …enum {…x…}; => … const int x = 1+1+1+1; …
May 14, 2002 Macro Languages AoPL, S'02
Metamorph Advantages
• Flexibility:• Tree structures• Non-local transformations (multiple results)
• Safety:• No parse errors as a conseq. of macro expansion• Guaranteed termination
• Simplicity: • Based entirely on declarative concepts:
grammars and substitution
May 14, 2002 Macro Languages AoPL, S'02
vDSL:very Domain-Specific Languagestudies
course Math101
title “Mathematics 101”
2 point fall term
…
exclusions
Math101 <> MathA
Math102 <> MathB
prerequisites
Math101, Math102 < Math201, Math202, Math203
Math101, CS101 < CS202
studies
course Math101
title “Mathematics 101”
2 point fall term
…
exclusions
Math101 <> MathA
Math102 <> MathB
prerequisites
Math101, Math102 < Math201, Math202, Math203
Math101, CS101 < CS202
May 14, 2002 Macro Languages AoPL, S'02
FIN
May 14, 2002 Macro Languages AoPL, S'02
Next Week: metafront
• Macros are just a special case usage:• A is an extension of B: m: L+ => L
• Make sure only need to write delta: = L+ \ L
metafront
x: A => B
A B
program.a program.b
transformation
input language
input program(s) output program(s)
output language