Eelco Visserhttp://eelcovisser.org

Software Language Design & Engineering

Microsoft Research, RedmondOctober 20, 2011

bridging the gap between problem domain and solution domain

Software Engineering

ProblemDomain Machine

HLLs reduce gap

High-Level Languages

ProblemDomain HLL Machine

"A programming language is low level when its programs require attention to the irrelevant." Alan J. Perlis (1982)

domain-specific languages support more specialization

Domain-Specific Languages

ProblemDomain HLL MachineDSL

Software Language Design & Engineering

enable software engineers to effectively design, implement, and apply domain-specific

software languages

Research: Software Language Engineering

automatically derive efficient, scalable, incremental compiler +

usable IDE from high-level,

declarative language definition

creating full featured IDEs for domain-specific languages

The Spoofax Language Workbench

Research: Software Language Design

systematically design domain-specific

software languages with optimal tradeoff between

expressivity, completeness, portability, coverage, and maintainability

Software Language Design Case Studies

Mobl: client-side stateful web applications

WebDSL: server-side restful web applications

Two Talks

Declaratively Programming the Mobile Web with Mobl

The Spoofax Language Workbench

Eelco Visserhttp://eelcovisser.org

Declaratively Programming the Mobile Web with Mobl*

* To appear: SPLASH/OOPSLA, October 2011, Portland

Zef Hemelhttp://zef.me

Native Applications not Portable

Divergence in Mobile Platforms

Objective-C Java J2ME/C++

HTML/Javascript Java .NET

Convergence in Mobile Platform

Webkit browser

Webkit browser

Webkit browser

Webkit browser

The Universal Userinterface Engine

Mobile Web Architecture

Rich Applications

WebDatabases

Location information (GPS)

Canvas

Multi-touch

Offline support

Full-screen support

Accelerator support

Audio

Native Applications

Address book

Camera

Compass

File IO

Notifications

MVC, No Integration, No Abstraction, Accidental Complexity

Software Engineering with JavaScript

annotated HTML imperative Javascript

Late Failures in Web Applications

Zef Hemel, Danny M. Groenewegen, Lennart C. L. Kats, Eelco Visser. Static consistency checking of web applications with WebDSL. Journal of

Symbolic Computation, 46(2):150-182, 2011.

declarativetyped

integrated concise

http://www.mobl-lang.org

Web Application with Touch

Portable Applications

Mobl Architecture

tipcalculator.mobl

application tipcalculator

import mobl::ui::generic

screen root() { var amount = 20 var percentage = 10 header("Tip calculator") group { item { numField(amount, label="amount") } item { numField(percentage, label="percentage") } item { "$" label(Math.round(amount * (1 + percentage/100))) } }}

Model-View Pattern

Task Manager

HTML5 Data Persistence

Data Model

entity Task { name : String (searchable) done : Bool due : DateTime category : Category (inverse: tasks) tags : Collection<Tag> (inverse: tasks)}entity Category { name : String tasks : Collection<Task> (inverse: category)}entity Tag { name : String tasks : Collection<Task> (inverse: tags)}

statically typed: catch errors early

Logic

entity Task { ... function postpone(days : Num) { this.due = DateTime.create( this.due.getFullYear(), this.due.getMonth(), this.due.getDate() + days); } function import(user : String, pw : String) { var tasksJSON = httpRequest("/export?user="+ user + "&pw=" + pw); foreach(t in tasksJSON) { add(Task.fromSelectJSON(t)); } }}

screen root() { var phrase = "" header("Tasks") { button("Add", onclick={ addTask(); }) } searchBox(phrase) group { list(t in Task.search(phrase) limit 20){ item { checkBox(t.done, label=t.name) } } }}

Reactive User Interfaces

screen root() { var phrase = "" header("Tasks") { button("Add", onclick={ addTask(); }) } searchBox(phrase) group { list(t in Task.search(phrase) limit 20){ item { checkBox(t.done, label=t.name) } } }}

screen addTask() { var t = Task() header("Add") { button("Done", onclick={ add(t); screen return; }) } textField(t.name) datePicker(t.due)}

Navigation

Continuations

screen root() { button("Ask", onclick={ alert("Hello " + prompt("First name") + " " + prompt("Last name")); })}screen prompt(question : String) : String { var answer = "" header(question) { button("Done", onclick={ screen return answer; }) } textField(answer)}

User Interface Idiom: Tab

Reusable Controls

screen root() { tabSet([("One", tab1), ("Two", tab2)], defaultTab="One")} control tab1() { header("Tab 1") label("This is tab 1")}control tab2() { header("Tab 2") label("This is tab 2")}

increase coverage: developers can create abstractions

Tab Set: Higher-Order Control

control tabSet(tabs : [(String,Control)], activeTab : String) { list((tabName, tabControl) in tabs) { block(onclick={ activeTab = tabName; }, style=activeTab==tabName ? activeTabButton : inactiveTabButton) { label(tabName) } } list((tabName, tabControl) in tabs) { block(activeTab==tabName ? visibleTab : invisibleTab) { tabControl() } }}

User Interface Idiom: Master Detail

screen root() { header("Tasks") masterDetail( Task.all() order by due desc, taskItem, taskDetail)} control taskItem(t : Task) { checkBox(t.done, label=t.name)}control taskDetail(t : Task) { textField(t.name) datePicker(t.due)}

Master Detail: Higher-Order Control

control masterDetail(items : Collection<?>, masterItem : Control1<?>, detail : Control1<?>) { group { list(it in items) { item(onclick={ detailScreen(it,detail); }) { masterItem(it) } } }}screen detailScreen(it : ?, detail : Control1<?>) { header("Detail") { backButton() } detail(it)}

Adaptive Layout

applications

Mobl Applications

GR8 Conference Program

mPodder

Ken

http://itsneh.com/ken/

Mobl Summary

❖ Linguistic integration

★ data model, user interface, styling, application logic

❖ Domain abstractions

★ reduce accidental complexity, platform details

❖ User-defined abstractions

★ data binding

★ reactive programming

★ reusable controls

❖ Model-View

★ automate the controller

Constructed on Spoofax Language Workbench

Mobl IDE: Static Cross-Concern Consistency Checking

Eelco Visserhttp://eelcovisser.org

The Spoofax Language Workbench*Rules for Declarative Specification of Languages and IDEs

* OOPSLA 2010

Lennart Katshttp://www.lclnet.nl

Research: Software Language Engineering

Automatically derive efficient, scalable, incremental compiler +

usable IDE from high-level,

declarative language definition

Language Definitions for Compilers

Syntax definition★ concrete syntax★ abstract syntax

Static semantics★ error checking★ name resolution★ type analysis

Model-to-model transformation★ express constructs in core language

Code generation★ translate core language models to implementation

Editor Services for Modern IDEs

Syntactic Editor Services★ syntax checking ★ bracket matching★ syntax highlighting★ code folding★ outline view

Semantic Editor Services★ error checking★ reference resolving★ hover help★ code completion★ refactoring

Syntax Definition with SDF

character-level grammars: integration of lexical and context-free syntax

Declarative Syntax Definition

lexical syntax [a-zA-Z\_\$][a-zA-Z0-9\_]* -> ID

context-free syntax STRING -> LimitedSetExp {cons("String")} NUMBER -> LimitedSetExp {cons("Num")} LimitedSetExp -> Exp QId -> LimitedExp {cons("Var")} "(" Exp ")" -> LimitedExp {cons("Brackets")} Exp BoolMethodId Exp -> Exp {cons("BinMethodCall"), left} Exp CompareMethodId Exp -> Exp {cons("BinMethodCall"), left} Exp TermOperatorMethodId Exp -> Exp {cons("BinMethodCall"), left} Exp OperatorMethodId Exp -> Exp {cons("BinMethodCall"), left}

develop and use language in same environment

An Interactive Language Workbench

Debugging Ambiguities

Declarative Disambiguation

context-free priorities Exp "." ID -> LimitedExp > Exp "." ID "(" {NamedExp ","}* ")" -> LimitedExp > Exp TermOperatorMethodId Exp -> Exp > Exp OperatorMethodId Exp -> Exp > Exp CompareMethodId Exp -> Exp > Exp BoolMethodId Exp -> Exp > "!" Exp -> Exp > Exp "?" Exp ":" Exp -> Exp > LimitedExp Filter+ -> Exp

Parsing after Disambiguation

permissive grammar + backtracking + region discovery [*OOPSLA & SLE 2009]

Natural and Flexible Error Recovery*

Transformation with Stratego

Syntactic Normalization and Desugaring

rules

normalize : FunctionNoReturnType(manno*, name, farg*, stat*) -> Function(manno*, name, farg*, SimpleType(QId("mobl", "void")), stat*)

normalize : IfNoElse(e, block) -> If(e, block, Block([]))

desugar : ForInferred(lvalue, e, elem*) -> For(lvalue, t, e, elem*) where GenericType(_, [t]) := <type-of> e

CPS Transform

cps-lift-exprs : [Return(e)|stats] -> <concat>[stats2, [Return(e2)|<cps-lift-expressions> stats]] where not(<is-sync> e) with {| Exp : stats2 := <cps-lift-expression> e ; e2 := <Exp> |}

Type Analysis

eq-type-of : String(_) -> SimpleType(QId("mobl", "String"))

eq-type-of : FieldAccess(e, x) -> t where Property(_, _, t, _) := <lookup-property> (<type-of> e, x)

origin tracking

Type Checking

constraint-error : t@SimpleType(_) -> (t, $[Type is not defined: [<pp-mobl-type> t]]) where not(<lookup-type> t)

constraint-error : t@FieldAccess(e, x) -> (t, $[Property [x] not defined]) where <type-of> e where not(type-of)

Semantic Editor Services

editor-analyze: (ast, path, project-path) -> (ast2, errors, warnings, notes) with // ...

editor-complete-proposal : SimpleType(COMPLETION(_)) -> proposals where all-types := <get-all-types>; proposals := <map(type-name-to-proposal); flatten-list> all-types

editor-hover: (t@SimpleType(_), position, ast, path, project-path) -> <get-doc> <lookup-type> t2 where t2 := <lookup-node> (position, ast)

editor-resolve: (t@SimpleType(qid), position, ast, path, project-path) -> target where target := <ensure-origin(lookup-type|qid)> t

Connect transformations to IDE

Editor Services Bindings

module MoBL-Builders

imports MoBL-Builders.generated

builders provider : include/mobl.jar observer : editor-analyze on save : generate-artifacts builder : "Show ATerm" = generate-aterm (openeditor) (realtime) (meta) builder : "Format code" = format-code (openeditor) (realtime) builder : "Desugar" = editor-desugar (openeditor) (realtime) (meta)

Includes testing of the editor services [* OOPSLA 2011]

Integrated Language Definition Testing*

http://researchr.org/search/publication/mobl+spoofax+webdsl

Software Language Design & Engineering

http://spoofax.org

http://mobl-lang.org

http://webdsl.org

http://researchr.org

http://eelcovisser.org

separation of concerns + linguistic integration

cross concern consistency checking

early detection of failures

Linguistic abstraction:capture software knowledge in

domain-specific languages

Language workbench: DSL design and implementation with less effort than

traditional language engineering

behind the scenes

var results = Task.all().list();for(var i = 0; i < results.length; i++) { alert(results[i].name);}

synchronous programming

render page

query database and process

results

...

time

render page

query database and process

results

...

timebrowser freeze

render page

send query

...

process query result

...

time

Task.all.list(function(results) { for(var i = 0; i < results.length; i++) { alert(results[i].name); } });...

asynchronous programming

Task.all().list(function(results) { alert("Hello, "); });alert("world!");

breaks sequential execution assumption

Task.all().list(function(results) { // make changes ... persistence.flush(function() { alert("Changes saved!"); });});

continuation-passing styletransformation

function displayLocationAndReturn() : Coordinates { var position = mobl::location::getPosition(); log("Lat: " + position.latitude); log("Long: " + position.longitude); return position;}

function displayLocationAndReturn(callback) { mobl.location.getPosition(function(position) { console.log("Lat: " + position.latitude); console.log("Long: " + position.longitude); callback(position); });};

function displayLocationAndReturn() : Coordinates { var position = mobl::location::getPosition(); log("Lat: " + position.latitude); log("Long: " + position.longitude); return position;}

function displayLocationAndReturn(callback) { mobl.location.getPosition(function(position) { console.log("Lat: " + position.latitude); console.log("Long: " + position.longitude); callback(position); });};

function displayLocationAndReturn() : Coordinates { var position = mobl::location::getPosition(); log("Lat: " + position.latitude); log("Long: " + position.longitude); return position;}

function displayLocationAndReturn(callback) { mobl.location.getPosition(function(position) { console.log("Lat: " + position.latitude); console.log("Long: " + position.longitude); callback(position); });};