domain-specific languages for composable editor plugins (ldta 2009)
DESCRIPTION
Modern IDEs increase developer productivity by incorporating many different kinds of editor services. These can be purely syntactic, such as syntax highlighting, code folding, and an outline for navigation; or they can be based on the language semantics, such as in-line type error reporting and resolving identifier declarations. Building all these services from scratch requires both the extensive knowledge of the sometimes complicated and highly interdependent APIs and extension mechanisms of an IDE framework, and an in-depth understanding of the structure and semantics of the targeted language. This paper describes Spoofax/IMP, a meta-tooling suite that provides high-level domain-specific languages for describing editor services, relieving editor developers from much of the framework-specific programming. Editor services are defined as composable modules of rules coupled to a modular SDF grammar. The composability provided by the SGLR parser and the declaratively defined services allows embedded languages and language extensions to be easily formulated as additional rules extending an existing language definition. The service definitions are used to generate Eclipse editor plugins. We discuss two examples: an editor plugin for WebDSL, a domain-specific language for web applications, and the embedding of WebDSL in Stratego, used for expressing the (static) semantic rules of WebDSL.TRANSCRIPT
![Page 1: Domain-Specific Languages for Composable Editor Plugins (LDTA 2009)](https://reader034.vdocuments.net/reader034/viewer/2022052618/554f86b3b4c905d25b8b4cee/html5/thumbnails/1.jpg)
Domain-Specific Languages for Composable Editor Plugins
LDTA 2009, York, UK
Lennart Kats (me), Delft University of Technology Karl Trygve Kalleberg, University of BergenEelco Visser, Delft University of Technology
Software Engineering Research Group
March 19, 2009
![Page 2: Domain-Specific Languages for Composable Editor Plugins (LDTA 2009)](https://reader034.vdocuments.net/reader034/viewer/2022052618/554f86b3b4c905d25b8b4cee/html5/thumbnails/2.jpg)
2
The Frameworkis the
Language
![Page 3: Domain-Specific Languages for Composable Editor Plugins (LDTA 2009)](https://reader034.vdocuments.net/reader034/viewer/2022052618/554f86b3b4c905d25b8b4cee/html5/thumbnails/3.jpg)
3
The IDEis the
Language
![Page 4: Domain-Specific Languages for Composable Editor Plugins (LDTA 2009)](https://reader034.vdocuments.net/reader034/viewer/2022052618/554f86b3b4c905d25b8b4cee/html5/thumbnails/4.jpg)
4
Implementing IDEs:The Two Faces of Eclipse
Eclipse platform:• Cross-platform, open-source• People have it• People use it
• Java, ...
![Page 5: Domain-Specific Languages for Composable Editor Plugins (LDTA 2009)](https://reader034.vdocuments.net/reader034/viewer/2022052618/554f86b3b4c905d25b8b4cee/html5/thumbnails/5.jpg)
5
Implementing IDEs:The Two Faces of Eclipse
Huge, low-level API• SWT widgets• synchronization• I/O• regexes
![Page 6: Domain-Specific Languages for Composable Editor Plugins (LDTA 2009)](https://reader034.vdocuments.net/reader034/viewer/2022052618/554f86b3b4c905d25b8b4cee/html5/thumbnails/6.jpg)
6
Implementing IDEs:The Two Faces of Eclipse
Weakly typed interfaces• XML• java.lang.Object, IAdaptable• CoreException "A checked exception representing a failure."
![Page 7: Domain-Specific Languages for Composable Editor Plugins (LDTA 2009)](https://reader034.vdocuments.net/reader034/viewer/2022052618/554f86b3b4c905d25b8b4cee/html5/thumbnails/7.jpg)
7
II.
ComposableLanguages
![Page 8: Domain-Specific Languages for Composable Editor Plugins (LDTA 2009)](https://reader034.vdocuments.net/reader034/viewer/2022052618/554f86b3b4c905d25b8b4cee/html5/thumbnails/8.jpg)
8
DSLs and Language Extensions
Domain-specific• Database queries• Regular expressions• XML processing• Matrices• Complex numbers• ...
void browse() { List<Book> books =
<| SELECT *
FROM books
WHERE price < 100.00
|>;
…
for (int i = 0; i < books.size(); i++)
books.get(i).title =~ s/^The //;
}
![Page 9: Domain-Specific Languages for Composable Editor Plugins (LDTA 2009)](https://reader034.vdocuments.net/reader034/viewer/2022052618/554f86b3b4c905d25b8b4cee/html5/thumbnails/9.jpg)
9
Meta-programming withConcrete Object Syntax
• Program transformation• Stratego with WebDSL, Java, XML
webdsl-action-to-java-method: |[ action x_action(farg*) { stat* } ]| -> |[ public void x_action(param*) { bstm* } ]| with param* := <map(action-arg-to-java)> farg*; bstm* := <statements-to-java> stat*
![Page 10: Domain-Specific Languages for Composable Editor Plugins (LDTA 2009)](https://reader034.vdocuments.net/reader034/viewer/2022052618/554f86b3b4c905d25b8b4cee/html5/thumbnails/10.jpg)
10
III.
IntroducingSpoofax/IMP
![Page 11: Domain-Specific Languages for Composable Editor Plugins (LDTA 2009)](https://reader034.vdocuments.net/reader034/viewer/2022052618/554f86b3b4c905d25b8b4cee/html5/thumbnails/11.jpg)
11
IDE development environments(Or: How to Learn to Stop Worrying and Love Eclipse)
• Abstraction• avoid Eclipse framework complexity
• Modularity• separation of concerns• reuse
• Extensibility and customization• integration with existing compilers, tools
![Page 12: Domain-Specific Languages for Composable Editor Plugins (LDTA 2009)](https://reader034.vdocuments.net/reader034/viewer/2022052618/554f86b3b4c905d25b8b4cee/html5/thumbnails/12.jpg)
12
Introducing Spoofax/IMP
• Three pillars:• SDF grammars• DSLs for service descriptors• Implemented using Spoofax and IMP frameworks (“SAFARI”)
![Page 13: Domain-Specific Languages for Composable Editor Plugins (LDTA 2009)](https://reader034.vdocuments.net/reader034/viewer/2022052618/554f86b3b4c905d25b8b4cee/html5/thumbnails/13.jpg)
13
An IDE plugin created with Spoofax/IMP
![Page 14: Domain-Specific Languages for Composable Editor Plugins (LDTA 2009)](https://reader034.vdocuments.net/reader034/viewer/2022052618/554f86b3b4c905d25b8b4cee/html5/thumbnails/14.jpg)
14
SDF and SGLR (1)
• Unified lexical and context-free syntax
module WebDSLimports MixHQL[HQL] AccessControl ...exports lexical syntax [a-zA-Z][a-zA-Z0-9\_]* → Id ... context-free syntax "module" Id Section* → Unit {cons("Module")} "section" SectionName Def* → Section {cons("Section")} "define" Mod* Id "{" Element* "}" → Def {cons("SimpleDef")} ...
![Page 15: Domain-Specific Languages for Composable Editor Plugins (LDTA 2009)](https://reader034.vdocuments.net/reader034/viewer/2022052618/554f86b3b4c905d25b8b4cee/html5/thumbnails/15.jpg)
15
SDF and SGLR (2)
• Scannerless Generalized-LR Parsing• Full class of context-free grammars• Compositional• Declarative disambiguation filters
module Stratego-WebDSL-Java-XMLimports Stratego-Java-15 Stratego-WebDSL Stratego-XML
![Page 16: Domain-Specific Languages for Composable Editor Plugins (LDTA 2009)](https://reader034.vdocuments.net/reader034/viewer/2022052618/554f86b3b4c905d25b8b4cee/html5/thumbnails/16.jpg)
16
Modular Editor Service Definitions
• Main file• Definition for each service• Generated definitions
![Page 17: Domain-Specific Languages for Composable Editor Plugins (LDTA 2009)](https://reader034.vdocuments.net/reader034/viewer/2022052618/554f86b3b4c905d25b8b4cee/html5/thumbnails/17.jpg)
17
Reuse and Modularity in IDE plugins
Stratego + WebDSL editor = StrategoWebDSL editor
• Declarative specifications• (Backdoor available)
![Page 18: Domain-Specific Languages for Composable Editor Plugins (LDTA 2009)](https://reader034.vdocuments.net/reader034/viewer/2022052618/554f86b3b4c905d25b8b4cee/html5/thumbnails/18.jpg)
18
Creating a brand new IDE plugin
Requires:• Syntax definition• Language name• File extension(s)
Gives you:• Service templates• Generated services• plugin.xml, ...
And:• Basic IDE functionality: Coloring, outline, folding
![Page 19: Domain-Specific Languages for Composable Editor Plugins (LDTA 2009)](https://reader034.vdocuments.net/reader034/viewer/2022052618/554f86b3b4c905d25b8b4cee/html5/thumbnails/19.jpg)
19
In the Beginning: WebDSL.main.esv
module WebDSL.main
imports WebDSL-Analysis WebDSL-Colorer WebDSL-...
language Description name : WebDSL aliases : WebDiesel id : org.strategoxt.imp.generated.webdsl description : "Spoofax/IMP-generated editor for the WebDSL language" url : http://strategoxt.org
language Files and parsing [...]
![Page 20: Domain-Specific Languages for Composable Editor Plugins (LDTA 2009)](https://reader034.vdocuments.net/reader034/viewer/2022052618/554f86b3b4c905d25b8b4cee/html5/thumbnails/20.jpg)
20
module WebDSL.main
imports WebDSL-Analysis WebDSL-...
language Description [...]
language Files and parsing [...]
In the Beginning: WebDSL.main.esv
module WebDSL.main
imports WebDSL-Analysis WebDSL-Colorer WebDSL-...
language Description [...]
language Files and parsing extensions : app table : include/WebDSL.tbl start symbols : Unit
![Page 21: Domain-Specific Languages for Composable Editor Plugins (LDTA 2009)](https://reader034.vdocuments.net/reader034/viewer/2022052618/554f86b3b4c905d25b8b4cee/html5/thumbnails/21.jpg)
21
In the Beginning (2): Generated Services
• Based on heuristics• Rapid prototyping• Starting point
• functioning as an example• self-documenting
module WebDSL-Colorer.generated
// ...documentation...
colorer Default highlighting rules keyword : "Keywords" = magenta bold string : "Strings" = blue number : "Numbers" = darkgreen ...
![Page 22: Domain-Specific Languages for Composable Editor Plugins (LDTA 2009)](https://reader034.vdocuments.net/reader034/viewer/2022052618/554f86b3b4c905d25b8b4cee/html5/thumbnails/22.jpg)
22
Example: The colorer service
module Stratego-WebDSL-Colorer
imports Stratego WebDSL
colorer Variables
_.Var : green italic
colorer Concrete syntax
environment _.ToMetaExpr: _ gray
environment _.FromMetaExpr: _ white
![Page 23: Domain-Specific Languages for Composable Editor Plugins (LDTA 2009)](https://reader034.vdocuments.net/reader034/viewer/2022052618/554f86b3b4c905d25b8b4cee/html5/thumbnails/23.jpg)
23
Example: The folding service
module Java-Folding
imports Java-Folding.generated
folding Customization
CompilationUnit NewInstance QNewInstance
ImportDec* (folded) Block (disable)
![Page 24: Domain-Specific Languages for Composable Editor Plugins (LDTA 2009)](https://reader034.vdocuments.net/reader034/viewer/2022052618/554f86b3b4c905d25b8b4cee/html5/thumbnails/24.jpg)
24
Syntactic Editor Services
• Syntax errors• Code folding• Outline view• Brace matching• Comments• Source code formatting
![Page 25: Domain-Specific Languages for Composable Editor Plugins (LDTA 2009)](https://reader034.vdocuments.net/reader034/viewer/2022052618/554f86b3b4c905d25b8b4cee/html5/thumbnails/25.jpg)
25
Semantic Editor Services
• Error reporting• Reference resolving• Reference info• Occurrence highlighting
![Page 26: Domain-Specific Languages for Composable Editor Plugins (LDTA 2009)](https://reader034.vdocuments.net/reader034/viewer/2022052618/554f86b3b4c905d25b8b4cee/html5/thumbnails/26.jpg)
26
Stratego integration
Stratego:• Rewrite rules• Strategies to control their application• Used for e.g., WebDSL, Stratego, Java [OOPSLA'08]
![Page 27: Domain-Specific Languages for Composable Editor Plugins (LDTA 2009)](https://reader034.vdocuments.net/reader034/viewer/2022052618/554f86b3b4c905d25b8b4cee/html5/thumbnails/27.jpg)
27
Interfacing with Stratego
• Interface based on rewrite rules• Adapted primitives for parsing, caching
editoranalyze: (ast, path, fullpath) > (errors, warnings, infos) with ... (errors, warnings, infos) := <collectallmarkers> ast ...
Offending term + message tuples
[(Var(“auhtor”), “undeclared”), ...]
![Page 28: Domain-Specific Languages for Composable Editor Plugins (LDTA 2009)](https://reader034.vdocuments.net/reader034/viewer/2022052618/554f86b3b4c905d25b8b4cee/html5/thumbnails/28.jpg)
28
Interfacing with Stratego
• Interface based on rewrite rules• Adapted primitives for parsing, caching
referenceresolve: (ast, path, fullpath, reference) > declaration with ... declaration := <finddecl> reference ...
Referenced declaration
Property(“author”, ...)
![Page 29: Domain-Specific Languages for Composable Editor Plugins (LDTA 2009)](https://reader034.vdocuments.net/reader034/viewer/2022052618/554f86b3b4c905d25b8b4cee/html5/thumbnails/29.jpg)
29
Using Stratego:A Global-to-Local Transformation
entity User { username :: String (id) password :: Secret isAdmin :: Bool name :: String manager -> User employees -> Set<User>}
entity User { username :: String (id) password :: Secret isAdmin :: Bool}
extend entity User { name :: String manager -> User employees -> Set<User>}
normalize
![Page 30: Domain-Specific Languages for Composable Editor Plugins (LDTA 2009)](https://reader034.vdocuments.net/reader034/viewer/2022052618/554f86b3b4c905d25b8b4cee/html5/thumbnails/30.jpg)
30
entity User { username :: String (id) password :: Secret isAdmin :: Bool name :: String manager -> User employees -> Set<User>}
extend entity User { name :: String manager -> User employees -> Set<User>}
Term Rewriting with Origin Tracking
entity User { username :: String (id) password :: Secret isAdmin :: Bool}
normalize
[Van Deursen et al 1993]
![Page 31: Domain-Specific Languages for Composable Editor Plugins (LDTA 2009)](https://reader034.vdocuments.net/reader034/viewer/2022052618/554f86b3b4c905d25b8b4cee/html5/thumbnails/31.jpg)
31
Program Object Model (POM) adapter
Interpret term operations as API calls • Using Spoofax interpreter• Intercept applications of rewrite rules in strategies• Override term building, 'all', 'some', and 'one'
for origin tracking
[Kalleberg et al, LDTA'07]
![Page 32: Domain-Specific Languages for Composable Editor Plugins (LDTA 2009)](https://reader034.vdocuments.net/reader034/viewer/2022052618/554f86b3b4c905d25b8b4cee/html5/thumbnails/32.jpg)
32
The Ubiquitous Eclipse
![Page 33: Domain-Specific Languages for Composable Editor Plugins (LDTA 2009)](https://reader034.vdocuments.net/reader034/viewer/2022052618/554f86b3b4c905d25b8b4cee/html5/thumbnails/33.jpg)
33
Dynamic Building and Loading:The Editor Service Builder
• Verifies all service descriptors• Syntax• Existing sorts• ...
• Updates plugin.xml, etc.• Builds parse tables
![Page 34: Domain-Specific Languages for Composable Editor Plugins (LDTA 2009)](https://reader034.vdocuments.net/reader034/viewer/2022052618/554f86b3b4c905d25b8b4cee/html5/thumbnails/34.jpg)
34
Dynamic Building and Loading:Dynamically Loading Editor Services
<<language>>
DynamicRootDynamicColor : ColorerdynamicFolder : Folder...
<<language>>
WebDSL
• IMP provides:• static, XML-based editor declarations• language inheritance
• Base language• Defines default services• May be overridden by dynamic or “backdoor” implementations
![Page 35: Domain-Specific Languages for Composable Editor Plugins (LDTA 2009)](https://reader034.vdocuments.net/reader034/viewer/2022052618/554f86b3b4c905d25b8b4cee/html5/thumbnails/35.jpg)
35
Looking back
LDTA'07: Spoofax
Embedded Java code
![Page 36: Domain-Specific Languages for Composable Editor Plugins (LDTA 2009)](https://reader034.vdocuments.net/reader034/viewer/2022052618/554f86b3b4c905d25b8b4cee/html5/thumbnails/36.jpg)
36
Looking back
LDTA'08: sdf2imp
No:• Semantic services• Dynamic loading• Modular definitions
![Page 37: Domain-Specific Languages for Composable Editor Plugins (LDTA 2009)](https://reader034.vdocuments.net/reader034/viewer/2022052618/554f86b3b4c905d25b8b4cee/html5/thumbnails/37.jpg)
37
Looking forward (to)
• Complete Stratego-based DSL environment• compiler for Java• SDF bundle
• Expansion of editor services• e.g. content completion
![Page 38: Domain-Specific Languages for Composable Editor Plugins (LDTA 2009)](https://reader034.vdocuments.net/reader034/viewer/2022052618/554f86b3b4c905d25b8b4cee/html5/thumbnails/38.jpg)
38
Looking forward (to)
• Integration with Aster [CC 2009]
• Better interactive parser• performance• error handling• content completion
![Page 39: Domain-Specific Languages for Composable Editor Plugins (LDTA 2009)](https://reader034.vdocuments.net/reader034/viewer/2022052618/554f86b3b4c905d25b8b4cee/html5/thumbnails/39.jpg)
DomainSpecific Languages for Composable Editor Plugins.Lennart C. L. Kats, Karl T. Kalleberg, and Eelco Visser. LDTA 2009.
http://www.strategoxt.org/Stratego/SpoofaxIMP
Concluding Remarks
• Declarative DSLs• Avoid Eclipse API complexity• Specialized syntax• Compositionality• Co-evolution of language and IDE