grammarware memes

75
Grammarware Memes Eelco Visser & Guido Wachsmuth

Upload: eelco-visser

Post on 22-Apr-2015

899 views

Category:

Technology


1 download

DESCRIPTION

Slides for mini-tutorial at the International Conference on Sofware Language Engineering (SLE 2012) in Dresden, Germany, September 27, 2012.

TRANSCRIPT

Page 1: Grammarware Memes

Grammarware MemesEelco Visser & Guido Wachsmuth

Page 2: Grammarware Memes

Background

Page 3: Grammarware Memes

Spoofax Language Workbench

Page 4: Grammarware Memes

WebDSL Web Programming Language

define page post(p: Post, title: String) { init{ p.update(); } title{ output(p.title) } bloglayout(p.blog){ placeholder view { postView(p) } postComments(p) }} define ajax postView(p: Post) { pageHeader2{ output(p.title) } postBodyLayout(p) { postContent(p) postExtendedContent(p) postActions(p) }}

Page 5: Grammarware Memes

Researchr Bibliography Management System

Page 6: Grammarware Memes

WebLab Programming Education on the Web

Page 7: Grammarware Memes

Some Philosophical Considerations

Page 8: Grammarware Memes

Grammarware vs Modelware?

GPCEOOPSLA

MODELSICMT

Page 9: Grammarware Memes

Grammarware vs Modelware?

GPCEOOPSLA

MODELSICMTSLE

Page 10: Grammarware Memes

Functional vs Object-Oriented vs Logic vs ...

Page 11: Grammarware Memes

Remember the Programming Language Wars?

ICFP OOPSLA

Page 12: Grammarware Memes

Remember the Programming Language Wars?

ICFP OOPSLAGPCE

Page 13: Grammarware Memes

Scala: Object-Oriented or Functional?

‘closure’: a purely functional feature?

Page 14: Grammarware Memes

X-ware is a Memeplex

memeplex: selection of memes from the meme pool

Page 15: Grammarware Memes

What is Grammarware?

Page 16: Grammarware Memes

Grammarware is about

(Parsing) Text

module users

imports library

entity User { email : String password : String isAdmin : Bool}

Page 17: Grammarware Memes
Page 18: Grammarware Memes

Grammarware is about

Structure

Module( "users", [ Imports("library") , Entity( "User" , [ Property("email", Type("String")) , Property("password", Type("String")) , Property("isAdmin", Type("Bool")) ] ) ])

Page 19: Grammarware Memes

Grammarware: Text & Structure

Page 20: Grammarware Memes

Grammarware is about Transformation

Page 21: Grammarware Memes

Source to Source Transformation

Page 22: Grammarware Memes

Source to Source Transformation

Page 23: Grammarware Memes

Transformation & Analysis

name resolution

type checking

desugaring

code generation

Page 24: Grammarware Memes

Some Memes from the Spoofax Memeplex

Page 25: Grammarware Memes

EnFun: Entities with Functions

module blog entity String { function plus(that:String): String } entity Bool { } entity Set<T> { function add(x: T) function remove(x: T) function member(x: T): Bool } entity Blog { posts : Set<Post> function newPost(): Post { var p : Post := Post.new(); posts.add(p); } } entity Post { title : String }

Page 26: Grammarware Memes

Structure

Page 27: Grammarware Memes

Signature & Terms

constructors Module : ID * List(Definition) -> Module Imports : ID -> Definition

Module( "application", [Imports("library"), Imports("users"), Imports("frontend")])

Page 28: Grammarware Memes

Entities & Properties

constructors Entity : ID * List(Property) -> Definition Type : ID -> Type New : Type -> Exp

constructors Property : ID * Type -> Property This : Exp PropAccess : Exp * ID -> Exp

Module("users", [ Imports("library") , Entity("User" , [ Property("email", Type("String")) , Property("password", Type("String")) , Property("isAdmin", Type("Bool"))])])

Page 29: Grammarware Memes

Constants & Operators & Variables

constructors String : STRING -> Exp Int : INT -> Exp False : Exp True : Exp

constructors VarDecl : ID * Type -> Stat VarDeclInit : ID * Type * Exp -> Stat Assign : Exp * Exp -> Stat Var : ID -> Exp

constructors Geq : Exp * Exp -> Exp Leq : Exp * Exp -> Exp Gt : Exp * Exp -> Exp Lt : Exp * Exp -> Exp Eq : Exp * Exp -> Exp Mul : Exp * Exp -> Exp Minus : Exp * Exp -> Exp Plus : Exp * Exp -> Exp Or : Exp * Exp -> Exp And : Exp * Exp -> Exp Not : Exp -> Exp

Page 30: Grammarware Memes

Statements & Functions

constructors Exp : Exp -> Stat Block : List(Stat) -> Stat Seq : List(Stat) -> Stat While : Exp * List(Stat) -> Stat IfElse : Exp * List(Stat) * List(ElseIf) * Option(Else) -> Stat Else : List(Stat) -> Else ElseIf : Exp * List(Stat) -> ElseIf

constructors FunDef : ID * List(Arg) * Type * List(Stat) -> Property FunDecl : ID * List(Arg) * Type -> Property Arg : ID * Type -> Arg Return : Exp -> Stat MethCall : Exp * ID * List(Exp) -> Exp ThisCall : ID * List(Exp) -> Exp

Page 31: Grammarware Memes

Transformation

Page 32: Grammarware Memes

Transformation by Strategic Rewriting

rules desugar: Plus(e1, e2) -> MethCall(e1, "plus", [e2]) desugar: Or(e1, e2) -> MethCall(e1, "or", [e2])

desugar : VarDeclInit(x, t, e) -> Seq([VarDecl(x, t), Assign(Var(x), e)])

strategies desugar-all = topdown(repeat(desugar))

Page 33: Grammarware Memes

Return-Lifting Applied

function fact(n: Int): Int { var res: Int; if(n == 0) { res := 1; } else { res := this * fact(n - 1); } return res;}

function fact(n: Int): Int {

if(n == 0) { return 1; } else { return this * fact(n - 1); }

}

Page 34: Grammarware Memes

Return-Lifting Rules

rules lift-return-all = alltd(lift-return; normalize-all) lift-return : FunDef(x, arg*, Some(t), stat1*) -> FunDef(x, arg*, Some(t), Seq([ VarDecl(y, t), Seq(stat2*), Return(Var(y)) ])) where y := <new>; stat2* := <alltd(replace-return(|y))> stat1* replace-return(|y) : Return(e) -> Assign(y, e)

Page 35: Grammarware Memes

Seq Normalization

strategies normalize-all = innermost-L(normalize) rules // seq lifting

normalize : [Seq(stat1*) | stat2*@[_|_]] -> [Seq([stat1*,stat2*])]

normalize : [stat, Seq(stat*)] -> [Seq([stat | stat*])]

normalize : Block([Seq(stat1*)]) -> Block(stat1*)

normalize : FunDef(x, arg*, t, [Seq(stat*)]) -> FunDef(x, arg*, t, stat*)

Page 36: Grammarware Memes

Interpretation

eval

Page 37: Grammarware Memes

Small Step Operational Semantics

rules // driver

steps = repeat(step) step: State([Frame(fn, this, slots, cont)|f*], heap) -> State(stack', heap') where cont' := <eval> cont; stack' := <update-stack (|...)> [Frame(..., cont')|f*]; heap' := <update-heap(|...)> heap

Page 38: Grammarware Memes

Small Step Operational Semantics

eval(|this, slots, heap): Var(x) -> val where val := <lookup> (x, slots)eval(|this, slots, heap): PropAccess(Ptr(p), prop) -> val where Object(_, props) := <lookup> (p, heap); val := <lookup> (prop, props)

Page 39: Grammarware Memes

From Text to Structure

Page 40: Grammarware Memes

Declarative Syntax Definition

Entity("User", [ Property("first", Type("String")), Property("last", Type("String"))])

signature constructors Entity : ID * List(Property) -> Definition Type : ID -> Type Property : ID * Type -> Property

Page 41: Grammarware Memes

Declarative Syntax Definition

entity User { first : String last : String}

Entity("User", [ Property("first", Type("String")), Property("last", Type("String"))])

signature constructors Entity : ID * List(Property) -> Definition Type : ID -> Type Property : ID * Type -> Property

Page 42: Grammarware Memes

Declarative Syntax Definition

entity User { first : String last : String}

Entity("User", [ Property("first", Type("String")), Property("last", Type("String"))])

signature constructors Entity : ID * List(Property) -> Definition Type : ID -> Type Property : ID * Type -> Property

context-free syntax "entity" ID "{" Property* "}" -> Definition {"Entity"} ID -> Type {"Type"} ID ":" Type -> Property {"Property"}

Page 43: Grammarware Memes

Declarative Syntax Definition

entity User { first : String last : String}

Entity("User", [ Property("first", Type("String")), Property("last", Type("String"))])

signature constructors Entity : ID * List(Property) -> Definition Type : ID -> Type Property : ID * Type -> Property

context-free syntax "entity" ID "{" Property* "}" -> Definition {"Entity"} ID -> Type {"Type"} ID ":" Type -> Property {"Property"}

Page 44: Grammarware Memes

Context-free Syntax

constructors True : Exp False : Exp Not : Exp -> Exp And : Exp * Exp -> Exp Or : Exp * Exp -> Exp

context-free syntax "true" -> Exp {"True"} "false" -> Exp {"False"} "!" Exp -> Exp {"Not"} Exp "&&" Exp -> Exp {"And"} Exp "||" Exp -> Exp {"Or"}

Page 45: Grammarware Memes

Lexical Syntax

constructors True : Exp False : Exp Not : Exp -> Exp And : Exp * Exp -> Exp Or : Exp * Exp -> Exp

context-free syntax "true" -> Exp {"True"} "false" -> Exp {"False"} "!" Exp -> Exp {"Not"} Exp "&&" Exp -> Exp {"And"} Exp "||" Exp -> Exp {"Or"}

lexical syntax [a-zA-Z][a-zA-Z0-9]* -> ID "-"? [0-9]+ -> INT [\ \t\n\r] -> LAYOUT

constructors : String -> ID : String -> INT

scannerless generalized (LR) parsing

Page 46: Grammarware Memes

Ambiguity

constructors True : Exp False : Exp Not : Exp -> Exp And : Exp * Exp -> Exp Or : Exp * Exp -> Exp

context-free syntax "true" -> Exp {"True"} "false" -> Exp {"False"} "!" Exp -> Exp {"Not"} Exp "&&" Exp -> Exp {"And"} Exp "||" Exp -> Exp {"Or"}

isPublic || isDraft && (author == principal())

amb([ And(Or(Var("isPublic"), Var("isDraft")), Eq(Var("author"), ThisCall("principal", []))), Or(Var("isPublic"), And(Var("isDraft"), Eq(Var("author"), ThisCall("principal", []))))])

Page 47: Grammarware Memes

Disambiguation by Encoding Precedence

constructors True : Exp False : Exp Not : Exp -> Exp And : Exp * Exp -> Exp Or : Exp * Exp -> Exp

context-free syntax "true" -> Exp {"True"} "false" -> Exp {"False"} "!" Exp -> Exp {"Not"} Exp "&&" Exp -> Exp {"And"} Exp "||" Exp -> Exp {"Or"}

context-free syntax "(" Exp ")" -> Exp0 {bracket} "true" -> Exp0 {"True"} "false" -> Exp0 {"False"} Exp0 -> Exp1 "!" Exp0 -> Exp1 {"Not"} Exp1 -> Exp2 Exp1 "&&" Exp2 -> Exp2 {"And"} Exp2 -> Exp3 Exp2 "||" Exp3 -> Exp3 {"Or"}

Page 48: Grammarware Memes

Declarative Disambiguation

context-free syntax "true" -> Exp {"True"} "false" -> Exp {"False"} "!" Exp -> Exp {"Not"} Exp "&&" Exp -> Exp {"And", left} Exp "||" Exp -> Exp {"Or", left} "(" Exp ")" -> Exp {bracket}context-free priorities {left: Exp.Not} > {left: Exp.Mul} > {left: Exp.Plus Exp.Minus} > {left: Exp.And} > {non-assoc: Exp.Eq Exp.Lt Exp.Gt Exp.Leq Exp.Geq}

isPublic || isDraft && (author == principal())

Or(Var("isPublic"), And(Var("isDraft"), Eq(Var("author"), ThisCall("principal", []))))

Page 49: Grammarware Memes

Generating Text from Structure

Page 50: Grammarware Memes

Code Generation by String Concatenation

to-java: Property(x, Type(t)) -> <concat-strings>[ "private ", t, " ", x, ";\n\n", "public ", t, " get_", x, " {\n", " return ", x, ";\n", "}\n", "public void set_", x, " (", t, " ", x, ") {\n", " this.", x, " = ", x, ";\n", "}" ]

Page 51: Grammarware Memes

String Interpolation Templates

to-java: Property(x, Type(t)) -> $[ private [t] [x]; public [t] get_[x] { return [x]; } public void set_[x] ([t] [x]) { this.[x] = [x]; } ]

Page 52: Grammarware Memes

Code Generation by Model Transformation

Page 53: Grammarware Memes

Generating Structured Representations

to-java: Property(x, Type(t)) -> [ Field([Private()], Type(t), Var(x)), Method([Public()], Type(t), $[get_[x]], [], [ Return(Var(x)) ]), Method([Public()], None(), $[set_[x]], [Param(Type(t), x)], [ Assign(FieldAccess(This(), x), Var(x)) ]) ]

Page 54: Grammarware Memes

Concrete Object Syntax

to-java: Property(x, Type(t)) -> |[ private t x; public t get_#x { return x; } public void set_#x (t x) { this.x = x; } ]|

Page 55: Grammarware Memes

Rewriting with Concrete Object Syntax

module desugar

imports Enfun strategies desugar-all = topdown(repeat(desugar)) rules desugar: |[ !e ]| -> |[ e.not() ]| desugar: |[ e1 && e2 ]| -> |[ e1.and(e2) ]| desugar: |[ e1 || e2 ]| -> |[ e1.or(e2) ]| desugar: |[ e1 + e2 ]| -> |[ e1.plus(e2) ]| desugar: |[ e1 * e2 ]| -> |[ e1.times(e2) ]| desugar: |[ e1 - e2 ]| -> |[ e1.minus(e2) ]|

Page 56: Grammarware Memes

Rewriting with Concrete Object Syntax

lift-return-cs : |[ function x(arg*) : t { stat1* } ]| -> |[ function x(arg*): t { var y : t; ( stat2* ) return y; } ]| where y := <new>; stat2* := <alltd(replace-return(|y))> stat1*

lift-return : FunDef(x, arg*, Some(t), stat1*) -> FunDef(x, arg*, Some(t), Seq([ VarDecl(y, t), Seq(stat2*), Return(Var(y)) ])) where y := <new>; stat2* := <alltd(replace-return(|y))> stat1*

Page 57: Grammarware Memes

Pretty-Printing by Model Transformation

Page 58: Grammarware Memes

Code Formatting

module users entity String { } entity User { first : String last : String }

module users

entity String {}

entity User { first : String last : String }

Module( "demo1", [ Entity("String", None(), None(), []) , Entity( "User" , [ Property("first", Type("String")) , Property("last", Type("String")) ] ) ])

Page 59: Grammarware Memes

Pretty-Printer Specification

prettyprint-Definition : Entity(x, prop*) -> [ H([SOpt(HS(), "0")] , [ S("entity ") , <pp-one-Z(prettyprint-ID)> x , S(" {") ] ) , <pp-indent(|"2")> [<pp-V-list(prettyprint-Property)> p*] , H([SOpt(HS(), "0")], [S("}")] ) ]

Page 60: Grammarware Memes

Structure Editors

Page 61: Grammarware Memes

Discoverability

Page 62: Grammarware Memes

Editors with Syntactic Completion

Page 63: Grammarware Memes

Completion Templates

completions

completion template Definition : "entity ID { }" = "entity " <ID:ID> " {\n\t" (cursor) "\n}" (blank) completion template Type : "ID" = <ID:ID> completion template Property : "ID : ID " = <ID:ID> " : " <ID:Type> (blank)

Page 64: Grammarware Memes

Syntax Templates: Structure + Layout

templates

Definition.Entity = < entity <ID> { <Property*; separator="\n"> } >

Type.Type = <<ID>>

Property.Property = < <ID> : <Type> >

prettyprint-Definition = ?Entity(x, prop*) ; ![ H( [SOpt(HS(), "0")] , [ S("entity ") , <pp-one-Z(prettyprint-ID)> x , S(" {") ] ) , <pp-indent(|"2")> [<pp-V-list(prettyprint-Property)> p*] , H([SOpt(HS(), "0")], [S("}")] ) ]

context-free syntax "entity" ID "{" Property* "}" -> Definition {"Entity"} ID -> Type {"Type"} ID ":" Type -> Property {"Property"}

completions

completion template Definition : "entity ID { }" = "entity " <ID:ID> " {\n\t" (cursor) "\n}" (blank) completion template Type : "ID<>" = <ID:ID> "<" <:Type> ">" completion template Property : "ID : ID " = <ID:ID> " : " <ID:Type> (blank)

signature constructors Entity : ID * List(Property) -> Definition Type : ID -> Type Property : ID * Type -> Property

Page 65: Grammarware Memes

Type Checking

Page 66: Grammarware Memes

Example: Checking Method Calls

module renaming

entity String { function plus(that:String): String { ... } } entity User { firstName : String lastName : String fullName : String function rename(first: String, last: String) { fullName := first.plus(0); // argument has wrong type fullName := first.plus(); // too few arguments fullName := first.plus(last, last); // too many arguments fullName := first.plus(last); // correct } }

Page 67: Grammarware Memes

Constraint Checks

rules // errors type-error : (e, t) -> (e, $[Type [<pp>t] expected instead of [<pp>t']]) where <type-of> e => t'; not(<subtype>(t', t))

rules // functions constraint-error: |[ e.x(e*) ]| -> (x, $[Expects [m] arguments, found [n]]) where <lookup-type> x => (t*, t); <length> e* => n; <length> t* => m; not(<eq>(m, n)) constraint-error: |[ e.x(e*) ]| -> <zip; filter(type-error)> (e*, t*) where <lookup-type> x => (t*, t)

Page 68: Grammarware Memes

Type Analysis

rules // constants type-of : |[ true ]| -> TypeBool() type-of : |[ false ]| -> TypeBool() type-of : Int(i) -> TypeInt() type-of : String(x) -> TypeString()

Page 69: Grammarware Memes

Types of Names

rules // names

type-of : Var(x) -> <lookup-type> x

type-of : |[ e.x ]| -> t where <lookup-type> x => t

type-of : |[ e.x(e*) ]| -> t where <lookup-type> x => (t*, t) type-of : |[ f(e*) ]| -> t where <lookup-type> f => (t*, t)

Page 70: Grammarware Memes

Analysis: Name Resolution

+

Page 71: Grammarware Memes

Definitions and References

test type refers to entity [[ module test entity [[String]] { } entity User { first : [[String]] last : String }]] resolve #2 to #1

definition

reference

Page 72: Grammarware Memes

From Tree to Graph

Module( "test", [ Entity("String", []) , Entity( "User" , [ Property("first", ) , Property("last", ) ] ) ])

Page 73: Grammarware Memes

Unique Names

Module( "users"{[Module(), "users"]} , [ Entity("String"{[Type(), "String", "users"]}, []) , Entity( "User"{[Type(), "User", "users"]} , [ Property( "first"{[Property(), "first", "User", "users"]} , Type("String"{[Type(), "String", "users"]})) , Property( "last"{[Property(), "last", "User", "users"]} , Type("String"{[Type(), "String", "users"]})) ] ) ] )

+ index mapping unique names to values

Page 74: Grammarware Memes

Spoofax Name Binding Language

namespaces Module Type

rules

Entity(c, _) : defines Type c Type(x, _) : refers to Type x Module(m, _) : defines Module m scopes Type Imports(m) : imports Type from Module m

See the full story in SLE talk on Friday

abstract from name resolution algorithmics

Page 75: Grammarware Memes

spoofax.org

http://spoofax.org