testing model transformations
TRANSCRIPT
![Page 1: Testing Model Transformations](https://reader035.vdocuments.net/reader035/viewer/2022062503/589cae811a28abbe4a8b5633/html5/thumbnails/1.jpg)
PRUEBAS EN EL DESARROLLO DE SOFTWARE DIRIGIDO POR
MODELOS
Juan de Lara
(trabajo conjunto con Esther Guerra)
modelling&software engineeringresearch grouphttp://www.miso.es
![Page 2: Testing Model Transformations](https://reader035.vdocuments.net/reader035/viewer/2022062503/589cae811a28abbe4a8b5633/html5/thumbnails/2.jpg)
PRUEBAS DE TRANSFORMACIONES DE
MODELOS
Juan de Lara
(trabajo conjunto con Esther Guerra)
modelling&software engineeringresearch grouphttp://www.miso.es
![Page 3: Testing Model Transformations](https://reader035.vdocuments.net/reader035/viewer/2022062503/589cae811a28abbe4a8b5633/html5/thumbnails/3.jpg)
Model-Driven Engineering (MDE).• Meta-Modelling and DSMLs.• Model Transformations.
Testing model-to-model transformations
Conclusions and Open lines
AGENDA
3
![Page 4: Testing Model Transformations](https://reader035.vdocuments.net/reader035/viewer/2022062503/589cae811a28abbe4a8b5633/html5/thumbnails/4.jpg)
MODEL DRIVEN ENGINEERINGIncrease the level of abstraction in software development.
Models are actively used to:• Describe the problem…• Simulate/verify/test…• Generate code for the final application.
Move from the solution space (code-centric) to the problem space:
• Higher levels of productivity and quality.• Less accidental details.
4
![Page 5: Testing Model Transformations](https://reader035.vdocuments.net/reader035/viewer/2022062503/589cae811a28abbe4a8b5633/html5/thumbnails/5.jpg)
MODEL DRIVEN ENGINEERINGFor specific domains:
• Avoid coding the same solutions over and over.
• Families of applications.• Domain-Specific Modelling
Languages.
CodeGenerator
• Modelling, validation and automatic generation of telephony services. 5
![Page 6: Testing Model Transformations](https://reader035.vdocuments.net/reader035/viewer/2022062503/589cae811a28abbe4a8b5633/html5/thumbnails/6.jpg)
MODEL DRIVEN ENGINEERINGFor specific domains:
• Avoid coding the same solutions over and over.
• Families of applications.• Domain-Specific Modelling
Languages.
• Code generation from State-Machines for Railway Signaling Systems. 6
Code generator
![Page 7: Testing Model Transformations](https://reader035.vdocuments.net/reader035/viewer/2022062503/589cae811a28abbe4a8b5633/html5/thumbnails/7.jpg)
MetaEdit+
DOMAIN SPECIFIC MODELLING LANGUAGES
Describe the structure of the domain• Relevant primitives and abstractions.• Relations, features.• Explicit expert knowledge.
7
![Page 8: Testing Model Transformations](https://reader035.vdocuments.net/reader035/viewer/2022062503/589cae811a28abbe4a8b5633/html5/thumbnails/8.jpg)
DOMAIN SPECIFIC MODELLING LANGUAGESDSMLs need not be graphical…
xText 8
![Page 9: Testing Model Transformations](https://reader035.vdocuments.net/reader035/viewer/2022062503/589cae811a28abbe4a8b5633/html5/thumbnails/9.jpg)
MODELS AND META-MODELSThe abstract syntax of DSMLs is defined through a meta-model
• Classes, • Attributes, • Relations.
9
Factory meta-model
Machine
Part
Conveyor
Generator Assembler
inps
outs*
*
* parts
Terminator
1..*
1..*
![Page 10: Testing Model Transformations](https://reader035.vdocuments.net/reader035/viewer/2022062503/589cae811a28abbe4a8b5633/html5/thumbnails/10.jpg)
MODELS AND META-MODELSThe abstract syntax of DSMLs is defined through a meta-model
• Classes, • Attributes, • Relations.
10
«conforms to»
c1:Conveyor
g:Generator
a:Assemblerc2:Conveyor
c3:Conveyor t:Terminator
p2: Partp2: Partouts
outs
inps
inps
outs inps
Factory meta-model
Machine
Part
Conveyor
Generator Assembler
inps
outs*
*
* parts
Terminator
1..*
1..*
![Page 11: Testing Model Transformations](https://reader035.vdocuments.net/reader035/viewer/2022062503/589cae811a28abbe4a8b5633/html5/thumbnails/11.jpg)
OCL CONSTRAINTS
11
Object Constraint Language
Well-formedness rules, which every model should satisfy.
Based on First-Order Logic.
g:Generator
«conforms to»
c:Conveyor
Factory meta-model
Machine
Part
Conveyor
Generator Assembler
inps
outs*
*
* parts
Terminator
1..*
1..*
Factory meta-model
Machine
Part
Conveyor
Generator Assembler
inps
outs*
*
* parts
Terminator
1..*
1..*
context Generator inv: self.inps->isEmpty() and self.outs->size()>0
context Generator inv: self.inps->isEmpty() and self.outs->size()>0context Terminator inv: self.outs->isEmpty() and self.inps->size()>0context Assembler inv: self.inps->size()>0 and self.outs.size()>0…
inps
![Page 12: Testing Model Transformations](https://reader035.vdocuments.net/reader035/viewer/2022062503/589cae811a28abbe4a8b5633/html5/thumbnails/12.jpg)
MODELS AND META-MODELSModels are represented using concrete syntax
• Visual • Textual
No need for a 1-1 correspondence between abstract and concrete syntax elements.
12
asse
![Page 13: Testing Model Transformations](https://reader035.vdocuments.net/reader035/viewer/2022062503/589cae811a28abbe4a8b5633/html5/thumbnails/13.jpg)
MODEL TRANSFORMATIONSModels need to be manipulated for:
• Simulation.• Optimization/refactoring.• Generating another model.• Generating code.
in-place transformations
13…
![Page 14: Testing Model Transformations](https://reader035.vdocuments.net/reader035/viewer/2022062503/589cae811a28abbe4a8b5633/html5/thumbnails/14.jpg)
MODEL TRANSFORMATIONSModels need to be manipulated for:
• Simulation.• Optimization/refactoring.• Generating another model.• Generating code.
SourceTarget
14
model-to-modeltransformations
![Page 15: Testing Model Transformations](https://reader035.vdocuments.net/reader035/viewer/2022062503/589cae811a28abbe4a8b5633/html5/thumbnails/15.jpg)
MODEL TRANSFORMATIONSModels need to be manipulated for:
• Simulation.• Optimization/refactoring.• Generating another model.• Generating code.
15
MMsrc MMtar
Msrc Mtar
Transformationdefinition
from to
«conforms to» «conforms to»
Transformationexecution
Transformationdeveloper
Final user
model-to-modeltransformations
![Page 16: Testing Model Transformations](https://reader035.vdocuments.net/reader035/viewer/2022062503/589cae811a28abbe4a8b5633/html5/thumbnails/16.jpg)
MODEL TRANSFORMATIONSModels need to be manipulated for:
• Simulation.• Optimization/refactoring.• Generating another model.• Generating code.
16
Template languages
query andmodel navigation
![Page 17: Testing Model Transformations](https://reader035.vdocuments.net/reader035/viewer/2022062503/589cae811a28abbe4a8b5633/html5/thumbnails/17.jpg)
17
SPECIFYING MODEL TRANSFORMATIONSModel transformations are not normally specified using general purpose languages
Specialized languages for transformation• Declarative/imperative• Visual/textual• Formal/semi-formal• …
LHS RHS
G H
work
![Page 18: Testing Model Transformations](https://reader035.vdocuments.net/reader035/viewer/2022062503/589cae811a28abbe4a8b5633/html5/thumbnails/18.jpg)
18
SPECIFYING MODEL TRANSFORMATIONSModel transformations are not normally specified using general purpose languages
Specialized languages for transformation• Declarative/imperative• Visual/textual• Formal/semi-formal• …
L R
G H
K
D
l r
m k m*
f d
P.O. P.O.
![Page 19: Testing Model Transformations](https://reader035.vdocuments.net/reader035/viewer/2022062503/589cae811a28abbe4a8b5633/html5/thumbnails/19.jpg)
SPECIFYING MODEL TRANSFORMATIONS
19
Model transformations are not normally specified using general purpose languages
Specialized languages for transformation• Declarative/imperative• Visual/textual• Formal/semi-formal• …
operation main() { // Epsilon Object Language …}operation Machine work(){ for (c in self.inps) { var p : Part := c.parts.random(); c.parts.remove(p); delete p; } for (c in self.outs) c.parts.add(new Part); }http://www.eclipse.org/epsilon/
![Page 20: Testing Model Transformations](https://reader035.vdocuments.net/reader035/viewer/2022062503/589cae811a28abbe4a8b5633/html5/thumbnails/20.jpg)
rule Conveyor2Place { from c : FAC!Conveyor to p : PN!Place (tokens <- c.parts->size())}
Model transformations are not normally specified using general purpose languages
Specialized languages for transformation• Declarative/imperative• Visual/textual• Formal/semi-formal• …
SPECIFYING MODEL TRANSFORMATIONS
20
![Page 21: Testing Model Transformations](https://reader035.vdocuments.net/reader035/viewer/2022062503/589cae811a28abbe4a8b5633/html5/thumbnails/21.jpg)
rule Conveyor2Place { from c : FAC!Conveyor to p : PN!Place (tokens <- c.parts->size())}
Model transformations are not normally specified using general purpose languages
Specialized languages for transformation• Declarative/imperative• Visual/textual• Formal/semi-formal• …
SPECIFYING MODEL TRANSFORMATIONS
21
SourceTarget
![Page 22: Testing Model Transformations](https://reader035.vdocuments.net/reader035/viewer/2022062503/589cae811a28abbe4a8b5633/html5/thumbnails/22.jpg)
Model-Driven Engineering (MDE).
Testing model-to-model transformations• Challenges• PaMoMo: A specification language for MTs• Specification-based MT testing
Conclusions and open lines
AGENDA
22
![Page 23: Testing Model Transformations](https://reader035.vdocuments.net/reader035/viewer/2022062503/589cae811a28abbe4a8b5633/html5/thumbnails/23.jpg)
TRANSFORMATION TESTINGCHALLENGES
23
Urgently needed• we have found 1000+ issues in a repository of ~100 ATL model
transformations (http://www.eclipse.org/atl/atlTransformations/)
Generation of input models• (very) costly if done manually• are we testing interesting cases?
Oracle function• is the result correct?
• syntactically (model conformant to target meta-model)• semantically (produces the intended model)
![Page 24: Testing Model Transformations](https://reader035.vdocuments.net/reader035/viewer/2022062503/589cae811a28abbe4a8b5633/html5/thumbnails/24.jpg)
TRANSFORMATION TESTING APPROACHES
24
Black box• Meta-model based:
• Input models derived covering the input meta-model. • SAT solving/model finding
• Oracle? Random testing: just check for crashes/conformance• Do the models allow testing all properties of interest?
• Specification (contract) based:• Input models derived from a high level specification• The specification serves as oracle• Specifications for model transformations are still rarely built (OCL?)
White box• Look at the transformation logic to generate the tests• Oracle?
![Page 25: Testing Model Transformations](https://reader035.vdocuments.net/reader035/viewer/2022062503/589cae811a28abbe4a8b5633/html5/thumbnails/25.jpg)
A SPECIFICATION LANGUAGE FOR MODEL TRANSFORMATIONSPaMoMo: A formal specification language based on algebraic graph transformation
• Pattern-based, declarative• Category of graphs and morphisms• Compilation into OCL for practical purposes
Pre-conditions: (+/-) conditions on input modelsPost-conditions: (+/-) conditions on output modelsInvariants: relations between input/output models
• Constructive (+)• Constraining (-)
25
![Page 26: Testing Model Transformations](https://reader035.vdocuments.net/reader035/viewer/2022062503/589cae811a28abbe4a8b5633/html5/thumbnails/26.jpg)
CONSTRAINT GRAPHS (1)Encoding of (meta-)models as (typed, attributed) graphs
26
p0: Placename=“inp”tokens=2
t: Transition
in
p1: Placename=“outp”tokens=0
name=“tr”out
p0
t
p1
“inp”
2
“tr”
“outp”
0
in
out
tokens
name
name
name
tokens
Place Transitionin
out
intString
tokensname name
type
![Page 27: Testing Model Transformations](https://reader035.vdocuments.net/reader035/viewer/2022062503/589cae811a28abbe4a8b5633/html5/thumbnails/27.jpg)
CONSTRAINT GRAPHS (2)Encoding of (meta-)models as (typed, attributed) graphs
• G= (V, EV, D, ED,
sV: EV V,
tV: EV V,
sD: ED V,
tD: ED D)• Plus algebras for data types
Typing as graph homomorphism• Four commuting functions
27
p0
t
p1
“inp”
2
“tr”
“outp”
0
in
out
tokens
name
name
name
tokens
Place Transitionin
out
intString
tokensname name
type
![Page 28: Testing Model Transformations](https://reader035.vdocuments.net/reader035/viewer/2022062503/589cae811a28abbe4a8b5633/html5/thumbnails/28.jpg)
CONSTRAINT GRAPHS (3)To express conditions on input/output models, we need:
• Two graphs typed w.r.t source/target meta-models• Attributes of graphs are variables• A formula relating variables of both• C=Gs, Gt,
28
p0: Placename=Y
m: Machinename=X
X=Y
![Page 29: Testing Model Transformations](https://reader035.vdocuments.net/reader035/viewer/2022062503/589cae811a28abbe4a8b5633/html5/thumbnails/29.jpg)
SATISFACTION OF CONSTRAINT GRAPHSSatisfaction of constraint graphs is checked by pattern matching
• Finding graph homomorphisms
29
p: Place
p0: Placename=“inp”tokens=2
t: Transitionin p1: Placename=“outp”tokens=0
name=“tr”
out
m1={(p, p0)} m2={(p, p1)}
![Page 30: Testing Model Transformations](https://reader035.vdocuments.net/reader035/viewer/2022062503/589cae811a28abbe4a8b5633/html5/thumbnails/30.jpg)
SATISFACTION OF CONSTRAINT GRAPHSHow to relate constraint graphs and graphs?
30
p: Place
p0: Placename=“inp”tokens=2
t: Transitionin p1: Placename=“outp”tokens=0
name=“tr”
out
m1={(p, p0)}
tokens=XX>0
![Page 31: Testing Model Transformations](https://reader035.vdocuments.net/reader035/viewer/2022062503/589cae811a28abbe4a8b5633/html5/thumbnails/31.jpg)
SATISFACTION OF CONSTRAINT GRAPHS
31
p: Place
p0: Placename=N1tokens=T1
t: Transitionin p1: Placename=N3tokens=T2
name=N2
out
m1={(p, p0)}
tokens=X
1 X>0
2 N1=“inp” T1=2 N2=“tr” N3=“outp” T2=0
A graph can be seen as a constraint graph• Constraint graph homomorphism = graph morphism + (reverse) formula implication (in the
example )2[X/N1]1
![Page 32: Testing Model Transformations](https://reader035.vdocuments.net/reader035/viewer/2022062503/589cae811a28abbe4a8b5633/html5/thumbnails/32.jpg)
Invariants have a bidirectional nature• Forwards/Backwards satisfaction
32
SATISFACTION OF INVARIANTS
p0: Placename=Y
m: Machinename=X
X=Y
![Page 33: Testing Model Transformations](https://reader035.vdocuments.net/reader035/viewer/2022062503/589cae811a28abbe4a8b5633/html5/thumbnails/33.jpg)
Invariants have a bidirectional nature• Forwards satisfaction
33
SATISFACTION OF INVARIANTS
m: Machinename=X
p0: Placename=Y
m: Machinename=X
X=Y
m: Machinename=“m1”
p0: Placename=“m1”
p1: Placename=“m2”
![Page 34: Testing Model Transformations](https://reader035.vdocuments.net/reader035/viewer/2022062503/589cae811a28abbe4a8b5633/html5/thumbnails/34.jpg)
Invariants have a bidirectional nature• Backwards satisfaction
34
SATISFACTION OF INVARIANTS
p0: Placename=Y
m: Machinename=X
X=Y
m: Machinename=“m1”
p0: Placename=“m1”
p1: Placename=“m2”
p0: Placename=Y
![Page 35: Testing Model Transformations](https://reader035.vdocuments.net/reader035/viewer/2022062503/589cae811a28abbe4a8b5633/html5/thumbnails/35.jpg)
EXAMPLES
35
![Page 36: Testing Model Transformations](https://reader035.vdocuments.net/reader035/viewer/2022062503/589cae811a28abbe4a8b5633/html5/thumbnails/36.jpg)
EXAMPLES
36
![Page 37: Testing Model Transformations](https://reader035.vdocuments.net/reader035/viewer/2022062503/589cae811a28abbe4a8b5633/html5/thumbnails/37.jpg)
EXAMPLES
37
![Page 38: Testing Model Transformations](https://reader035.vdocuments.net/reader035/viewer/2022062503/589cae811a28abbe4a8b5633/html5/thumbnails/38.jpg)
EXAMPLES
38
![Page 39: Testing Model Transformations](https://reader035.vdocuments.net/reader035/viewer/2022062503/589cae811a28abbe4a8b5633/html5/thumbnails/39.jpg)
WHAT CAN WE DO WITH AN SPECIFICATION?Generate “reference” implementations [OGLE09]
• Based on graph transformation• Terminating, correct, complete• …but not very efficient
Testing: • Generation of oracles based on QVT [GL13], OCL [GS15]• Generation of input models, combining constraints [GS15]
SAT-based refinement checking w.r.t:• Other specifications, implementations in ATL [BEGL13]
39
[BEGL13] Büttner, Egea, Guerra, de Lara. “Checking Model Transformation Refinement”. ICMT 2013: 158-173[GL13] Guerra, de Lara, et al. “Automated verification of model transformations based on visual contracts”. Autom. Softw. Eng. 20(1): 5-46 (2013)[GS15] Guerra, Soeken. “Specification-driven model transformation testing”. SoSyM 14(2): 623-644 (2015)[OGLE09] Orejas, Guerra, de Lara, Ehrig. “Correctness, Completeness and Termination of Pattern-Based Model-to-Model Transformation”. CALCO 2009: 383-397
![Page 40: Testing Model Transformations](https://reader035.vdocuments.net/reader035/viewer/2022062503/589cae811a28abbe4a8b5633/html5/thumbnails/40.jpg)
40
SPECIFICATION-BASED TESTING
report
selectspecification
coverage criterion
mtunit script
oracle(assertions)
input test models
.xmi
SAT solver model transformation
transformationspecification
.pamomo
transformationimplementation
.etl, .atl…
refers totester mtunit
engine
12
3
designer
developer
Given a specification, generation of:•input models.•OCL assertions. •Test script
![Page 41: Testing Model Transformations](https://reader035.vdocuments.net/reader035/viewer/2022062503/589cae811a28abbe4a8b5633/html5/thumbnails/41.jpg)
FROM BPMN TO PETRI NETS
41
![Page 42: Testing Model Transformations](https://reader035.vdocuments.net/reader035/viewer/2022062503/589cae811a28abbe4a8b5633/html5/thumbnails/42.jpg)
OUR SPECIFICATION LANGUAGE
42
Features of Pamomo:
•Formal, pattern-based, declarative
•Positive and negative requirements
•Compilation into OCL
P(OneStartEvent)
e.outgoing.size()=1
e
Preconditions Postconditions Invariants(conditions on (conditions on (relations between the elementsinput models) output models) in the input and output models)
N(UnconnectedPlaces)
pl.inarcs.size()=0 andpl.outarcs.size()=0
pl
N
t2pl2
+
pl1
t1.name = pl1.name and t2.name = pl2.name
N(ParallelGateway3)
t1
g
t2
+g
![Page 43: Testing Model Transformations](https://reader035.vdocuments.net/reader035/viewer/2022062503/589cae811a28abbe4a8b5633/html5/thumbnails/43.jpg)
43
1. Translation of properties into OCL
GENERATION OF INPUT MODELS
N
t2pl2
+
pl1
t1.name = pl1.name and t2.name = pl2.name
N(ParallelGateway3)
t1
g
t2
+g
Task::allInstances()->exists(t1 | Task::allInstances()->exists(t2 | ParallelGateway::allInstances()->exists(g | t1.outgoing->includes(g) and t1<>t2 and not t2.outgoing->includes(g) )))
pre-1 inv-1
1pos-1
ocli-1oclp-1
![Page 44: Testing Model Transformations](https://reader035.vdocuments.net/reader035/viewer/2022062503/589cae811a28abbe4a8b5633/html5/thumbnails/44.jpg)
44
1. Translation of properties into OCL
2. Select coverage criteria (e.g. generate models that allow testing one property at a time, two properties, no property, etc.)
=> the selected coverage induces a different set of ocl expressions, e.g., property1 and property2, not property 1 and property2, etc.
GENERATION OF INPUT MODELS
pre-1 inv-1
coverage criteria
1
2
pos-1
ocli-1oclp-1
ocli-1 and ocl-j
![Page 45: Testing Model Transformations](https://reader035.vdocuments.net/reader035/viewer/2022062503/589cae811a28abbe4a8b5633/html5/thumbnails/45.jpg)
45
1. Translation of properties into OCL
2. Select coverage criteria (e.g. generate models that allow testing one property at a time, two properties, no property, etc.)
3. Use constraint solver to generate models conforming to the meta-model and satisfying the OCL expressions
GENERATION OF INPUT MODELS
pre-1 inv-1
coverage criteria
1
2
pos-1
ocli-1oclp-1
ocli-1 and ocl-j
constraint solver3
model-1
sourcemeta-model
![Page 46: Testing Model Transformations](https://reader035.vdocuments.net/reader035/viewer/2022062503/589cae811a28abbe4a8b5633/html5/thumbnails/46.jpg)
46
• A test suite is generated from the specification
• It contains a test case for each invariant and postcondition, defining:
• assertion for the invariant or postcondition• input models to test (all but those generated from
expressions negating the invariant, as they satisfy the invariant vacuously)
GENERATION OF ORACLE
pre-1 inv-1
coverage criteria
1
2
pos-1
ocli-1oclp-1
ocli-1 and ocl-j
constraint solver3
model-1
sourcemeta-model
mtUnittest script(+ oracle)
4
transformation
report
![Page 47: Testing Model Transformations](https://reader035.vdocuments.net/reader035/viewer/2022062503/589cae811a28abbe4a8b5633/html5/thumbnails/47.jpg)
TOOL SUPPORT
47
generation
PAMOMO•EMF-based tool.•Textual/visual syntax for patterns.•Generation of input models and test scripts. •Model generation by using the USE solver.
mtUnit•EMF-based tool.•Automates transformation testing.
![Page 48: Testing Model Transformations](https://reader035.vdocuments.net/reader035/viewer/2022062503/589cae811a28abbe4a8b5633/html5/thumbnails/48.jpg)
Model-Driven Engineering (MDE).
Testing model-to-model transformations
Conclusions
AGENDA
48
![Page 49: Testing Model Transformations](https://reader035.vdocuments.net/reader035/viewer/2022062503/589cae811a28abbe4a8b5633/html5/thumbnails/49.jpg)
CONCLUSIONSMDE: increase the level of abstraction in software development by the use of (domain-specific) models.
V&V of the different MDE artefacts (meta-models, transformations, code generators)
Model transformation testing:• Generation of input models• Oracle function
A specification language like PaMoMo can be used for both
49
![Page 50: Testing Model Transformations](https://reader035.vdocuments.net/reader035/viewer/2022062503/589cae811a28abbe4a8b5633/html5/thumbnails/50.jpg)
CURRENT LINESStatic analysis of (ATL) model transformations
• Type inference• Quick fixes
DSLs for model mutation• Primitives to generate mutants of models conformant to
arbitrary meta-models• Transformations are also models
Model transformation reuse• Transformations used with different meta-models they were
initially built
50