sysml model-based testingomgwiki.org/omgsysml/lib/exe/fetch.php?media=sysml...sysml model-based...

31
SysML Model-Based Testing Fabrice Bouquet [email protected] OMG SYMSL - Modelica Group SysML / ALF / OCL / Modelica March 20th 2013 Fabrice Bouquet MBT SysML 1 / 23

Upload: others

Post on 19-Jul-2020

20 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: SysML Model-Based Testingomgwiki.org/OMGSysML/lib/exe/fetch.php?media=sysml...SysML Model-Based Testing Fabrice Bouquet fabrice.bouquet@univ-fcomte.fr OMGSYMSL-ModelicaGroup SysML/ALF/OCL/Modelica

SysML Model-Based TestingFabrice Bouquet

[email protected] SYMSL - Modelica Group

SysML / ALF / OCL / ModelicaMarch 20th 2013

Fabrice Bouquet MBT SysML 1 / 23

Page 2: SysML Model-Based Testingomgwiki.org/OMGSysML/lib/exe/fetch.php?media=sysml...SysML Model-Based Testing Fabrice Bouquet fabrice.bouquet@univ-fcomte.fr OMGSYMSL-ModelicaGroup SysML/ALF/OCL/Modelica

Outline

1 Context and TeamResearch TeamContext

2 Current WorksFrameworkModeling Consistency VerificationModel Transformation Validation

3 Synthesis

Fabrice Bouquet MBT SysML 2 / 23

Page 3: SysML Model-Based Testingomgwiki.org/OMGSysML/lib/exe/fetch.php?media=sysml...SysML Model-Based Testing Fabrice Bouquet fabrice.bouquet@univ-fcomte.fr OMGSYMSL-ModelicaGroup SysML/ALF/OCL/Modelica

Context Works Synthesisx . . . Research Team

Research Team

I 11 People (2 full Prof., 3 Lecturers, 4 PhD, 2 developers)I Model-Based Testing (MBT) Domains:

I Security and Safety i.r. functionalI IT (UML) and Embedded System (SysML)

I Test Coverage Criteria:I Classical:

I Flow control,I Data control,I Definition/use of variables

I Dedicated to SysML:I Communication (Signal) CoverageI Equation coverageI SysML + Marte: time coverage

Fabrice Bouquet MBT SysML 3 / 23

Page 4: SysML Model-Based Testingomgwiki.org/OMGSysML/lib/exe/fetch.php?media=sysml...SysML Model-Based Testing Fabrice Bouquet fabrice.bouquet@univ-fcomte.fr OMGSYMSL-ModelicaGroup SysML/ALF/OCL/Modelica

Context Works Synthesisx . . . Context

Language

Fabrice Bouquet MBT SysML 4 / 23

Page 5: SysML Model-Based Testingomgwiki.org/OMGSysML/lib/exe/fetch.php?media=sysml...SysML Model-Based Testing Fabrice Bouquet fabrice.bouquet@univ-fcomte.fr OMGSYMSL-ModelicaGroup SysML/ALF/OCL/Modelica

Context Works Synthesisx . . . Context

Language

with OCL and ALF

Fabrice Bouquet MBT SysML 4 / 23

Page 6: SysML Model-Based Testingomgwiki.org/OMGSysML/lib/exe/fetch.php?media=sysml...SysML Model-Based Testing Fabrice Bouquet fabrice.bouquet@univ-fcomte.fr OMGSYMSL-ModelicaGroup SysML/ALF/OCL/Modelica

Context Works Synthesisx . . . Context

Language

with OCL and AFL

Fabrice Bouquet MBT SysML 5 / 23

Page 7: SysML Model-Based Testingomgwiki.org/OMGSysML/lib/exe/fetch.php?media=sysml...SysML Model-Based Testing Fabrice Bouquet fabrice.bouquet@univ-fcomte.fr OMGSYMSL-ModelicaGroup SysML/ALF/OCL/Modelica

Context Works Synthesisx . . . Context

Language

with OCL and AFL

Fabrice Bouquet MBT SysML 5 / 23

Page 8: SysML Model-Based Testingomgwiki.org/OMGSysML/lib/exe/fetch.php?media=sysml...SysML Model-Based Testing Fabrice Bouquet fabrice.bouquet@univ-fcomte.fr OMGSYMSL-ModelicaGroup SysML/ALF/OCL/Modelica

Context Works Synthesisx . . . Context

Language

with OCL and AFL

Fabrice Bouquet MBT SysML 5 / 23

Page 9: SysML Model-Based Testingomgwiki.org/OMGSysML/lib/exe/fetch.php?media=sysml...SysML Model-Based Testing Fabrice Bouquet fabrice.bouquet@univ-fcomte.fr OMGSYMSL-ModelicaGroup SysML/ALF/OCL/Modelica

Context Works Synthesisx . . . Context

Language

with OCL and AFL

Fabrice Bouquet MBT SysML 5 / 23

Page 10: SysML Model-Based Testingomgwiki.org/OMGSysML/lib/exe/fetch.php?media=sysml...SysML Model-Based Testing Fabrice Bouquet fabrice.bouquet@univ-fcomte.fr OMGSYMSL-ModelicaGroup SysML/ALF/OCL/Modelica

Context Works Synthesisx . . . Context

Language

with OCL and AFL

Fabrice Bouquet MBT SysML 5 / 23

Page 11: SysML Model-Based Testingomgwiki.org/OMGSysML/lib/exe/fetch.php?media=sysml...SysML Model-Based Testing Fabrice Bouquet fabrice.bouquet@univ-fcomte.fr OMGSYMSL-ModelicaGroup SysML/ALF/OCL/Modelica

Context Works Synthesisx . . . Context

Language

with OCL and AFL

Fabrice Bouquet MBT SysML 5 / 23

Page 12: SysML Model-Based Testingomgwiki.org/OMGSysML/lib/exe/fetch.php?media=sysml...SysML Model-Based Testing Fabrice Bouquet fabrice.bouquet@univ-fcomte.fr OMGSYMSL-ModelicaGroup SysML/ALF/OCL/Modelica

Context Works Synthesisx . . . Context

Context: existing MBT tool chain

VETESS Project: http://lifc.univ-fcomte.fr/vetess

Fabrice Bouquet MBT SysML 6 / 23

Page 13: SysML Model-Based Testingomgwiki.org/OMGSysML/lib/exe/fetch.php?media=sysml...SysML Model-Based Testing Fabrice Bouquet fabrice.bouquet@univ-fcomte.fr OMGSYMSL-ModelicaGroup SysML/ALF/OCL/Modelica

Context Works Synthesisx . . . Context

Context: tool chain analysis

StrengthsI Relevant to generate many test casesI Ensures an optimal coverage of the modelI Reasonable time of test generationI Successful feedback from industrial engineers

WeaknessesI Creation of both SysML model and simulation model

=⇒ Late functional validation of the SysML modelI Discrete model of the system

=⇒ Concretization step very costly (+ 50% of time spent duringconcretization)

Fabrice Bouquet MBT SysML 7 / 23

Page 14: SysML Model-Based Testingomgwiki.org/OMGSysML/lib/exe/fetch.php?media=sysml...SysML Model-Based Testing Fabrice Bouquet fabrice.bouquet@univ-fcomte.fr OMGSYMSL-ModelicaGroup SysML/ALF/OCL/Modelica

Context Works Synthesisx . . . Context

Motivations

Validation of Real-Time and Embedded Systems using MBT

I Improvements of an existing MBT tool chain:I Extension for real-time and embedded systemsI Use of simulation to validate the test modelI Automation of the end-to-end MBT process

I Use of standardized languages and MDE techniques:I SysML/MARTE profilesI Generate element for simulation code (VHDL-AMS, Modelica)I UTP as a pivot language

Fabrice Bouquet MBT SysML 8 / 23

Page 15: SysML Model-Based Testingomgwiki.org/OMGSysML/lib/exe/fetch.php?media=sysml...SysML Model-Based Testing Fabrice Bouquet fabrice.bouquet@univ-fcomte.fr OMGSYMSL-ModelicaGroup SysML/ALF/OCL/Modelica

Context Works Synthesisx . . . Framework

Outline

1 Context and TeamResearch TeamContext

2 Current WorksFrameworkModeling Consistency VerificationModel Transformation Validation

3 Synthesis

Fabrice Bouquet MBT SysML 9 / 23

Page 16: SysML Model-Based Testingomgwiki.org/OMGSysML/lib/exe/fetch.php?media=sysml...SysML Model-Based Testing Fabrice Bouquet fabrice.bouquet@univ-fcomte.fr OMGSYMSL-ModelicaGroup SysML/ALF/OCL/Modelica

Context Works Synthesisx . . . Framework

Current Works

Fabrice Bouquet MBT SysML 10 / 23

Page 17: SysML Model-Based Testingomgwiki.org/OMGSysML/lib/exe/fetch.php?media=sysml...SysML Model-Based Testing Fabrice Bouquet fabrice.bouquet@univ-fcomte.fr OMGSYMSL-ModelicaGroup SysML/ALF/OCL/Modelica

Context Works Synthesisx . . . Framework

Current Works

Fabrice Bouquet MBT SysML 10 / 23

Page 18: SysML Model-Based Testingomgwiki.org/OMGSysML/lib/exe/fetch.php?media=sysml...SysML Model-Based Testing Fabrice Bouquet fabrice.bouquet@univ-fcomte.fr OMGSYMSL-ModelicaGroup SysML/ALF/OCL/Modelica

Context Works Synthesisx . . . Framework

Current Works

Fabrice Bouquet MBT SysML 10 / 23

Page 19: SysML Model-Based Testingomgwiki.org/OMGSysML/lib/exe/fetch.php?media=sysml...SysML Model-Based Testing Fabrice Bouquet fabrice.bouquet@univ-fcomte.fr OMGSYMSL-ModelicaGroup SysML/ALF/OCL/Modelica

Context Works Synthesisx . . . Verification

Outline

1 Context and TeamResearch TeamContext

2 Current WorksFrameworkModeling Consistency VerificationModel Transformation Validation

3 Synthesis

Fabrice Bouquet MBT SysML 11 / 23

Page 20: SysML Model-Based Testingomgwiki.org/OMGSysML/lib/exe/fetch.php?media=sysml...SysML Model-Based Testing Fabrice Bouquet fabrice.bouquet@univ-fcomte.fr OMGSYMSL-ModelicaGroup SysML/ALF/OCL/Modelica

Context Works Synthesisx . . . Verification

Modeling Consistency: contributions

I How to generate correct Modelica code?I How to provide a SysML modeling guideline?I How to automate these issues?

=⇒ Rules definition for SysML model consistency:I Syntactic verificationI Semantic verification

=⇒ Automation based on MDE techniques:I ATL for Model to Model TransformationI Acceleo for code generation

Fabrice Bouquet MBT SysML 12 / 23

Page 21: SysML Model-Based Testingomgwiki.org/OMGSysML/lib/exe/fetch.php?media=sysml...SysML Model-Based Testing Fabrice Bouquet fabrice.bouquet@univ-fcomte.fr OMGSYMSL-ModelicaGroup SysML/ALF/OCL/Modelica

Context Works Synthesisx . . . Verification

Modeling Consistency

Fabrice Bouquet MBT SysML 13 / 23

Page 22: SysML Model-Based Testingomgwiki.org/OMGSysML/lib/exe/fetch.php?media=sysml...SysML Model-Based Testing Fabrice Bouquet fabrice.bouquet@univ-fcomte.fr OMGSYMSL-ModelicaGroup SysML/ALF/OCL/Modelica

Context Works Synthesisx . . . Verification

Modeling Consistency Verification

Problemlocation : EStringdescription : EStringseverity : Severity

<<enumeration>>Severity

warningerrorcritic

Fabrice Bouquet MBT SysML 14 / 23

Page 23: SysML Model-Based Testingomgwiki.org/OMGSysML/lib/exe/fetch.php?media=sysml...SysML Model-Based Testing Fabrice Bouquet fabrice.bouquet@univ-fcomte.fr OMGSYMSL-ModelicaGroup SysML/ALF/OCL/Modelica

Context Works Synthesisx . . . Verification

ATL Verification Rule

1 helper def: reservedWords: Sequence(String) =2 Sequence{’ABS’, ’ACCESS’, ’AFTER’, ’ALIAS’, [...], ’TOLERANCE’};34 helper context MMuml!NamedElement def: isReservedWord() : Boolean =5 if self.name <> OclUndefined then6 thisModule.reservedWords->exists(r | r.toString().toLower() = self.name.toString().toLower())7 else8 false9 endif;1011 rule uncorrectNameReservedWord{12 from umlElement: MMuml!NamedElement(13 umlElement.isReservedWord()14 )15 to problem: MMproblem!Problem(16 severity <- #error,17 description <- umlElement.name + ’ is a reserved word for Modelica’,18 location <- ’root::’ + umlElement.getQualifiedName()19 )20 }

Fabrice Bouquet MBT SysML 15 / 23

Page 24: SysML Model-Based Testingomgwiki.org/OMGSysML/lib/exe/fetch.php?media=sysml...SysML Model-Based Testing Fabrice Bouquet fabrice.bouquet@univ-fcomte.fr OMGSYMSL-ModelicaGroup SysML/ALF/OCL/Modelica

Context Works Synthesisx . . . Model Transformation Validation

Outline

1 Context and TeamResearch TeamContext

2 Current WorksFrameworkModeling Consistency VerificationModel Transformation Validation

3 Synthesis

Fabrice Bouquet MBT SysML 16 / 23

Page 25: SysML Model-Based Testingomgwiki.org/OMGSysML/lib/exe/fetch.php?media=sysml...SysML Model-Based Testing Fabrice Bouquet fabrice.bouquet@univ-fcomte.fr OMGSYMSL-ModelicaGroup SysML/ALF/OCL/Modelica

Context Works Synthesisx . . . Model Transformation Validation

Model Transformation Validation

I How to validate the transformation process?I How to detect bugs during the transformation development?

=⇒ Test-Driven Development strategy:I Use of unit testsI Use of the EUnit framework

Fabrice Bouquet MBT SysML 17 / 23

Page 26: SysML Model-Based Testingomgwiki.org/OMGSysML/lib/exe/fetch.php?media=sysml...SysML Model-Based Testing Fabrice Bouquet fabrice.bouquet@univ-fcomte.fr OMGSYMSL-ModelicaGroup SysML/ALF/OCL/Modelica

Context Works Synthesisx . . . Model Transformation Validation

EUnit Framework

I Epsilon Unit Testing FrameworkI Based on Eclipse EMFI Compares transformation result with expected resultI Can be integrated as an automated task

Fabrice Bouquet MBT SysML 18 / 23

Page 27: SysML Model-Based Testingomgwiki.org/OMGSysML/lib/exe/fetch.php?media=sysml...SysML Model-Based Testing Fabrice Bouquet fabrice.bouquet@univ-fcomte.fr OMGSYMSL-ModelicaGroup SysML/ALF/OCL/Modelica

Context Works Synthesisx . . . Model Transformation Validation

Results

I SysML Consistency verification:I Detects Modelica syntactic/semantic errorsI Allows SysML modeling guideline for Modelica generation

I 31 ATL rulesI 24 ATL helpers

I Model Transformation Validation:I SysML2Problem transformation =⇒ 59 unit testsI SysML2Modelica transformation =⇒ 41 unit tests:

I 34 ATL rulesI 23 ATL helpers

Fabrice Bouquet MBT SysML 19 / 23

Page 28: SysML Model-Based Testingomgwiki.org/OMGSysML/lib/exe/fetch.php?media=sysml...SysML Model-Based Testing Fabrice Bouquet fabrice.bouquet@univ-fcomte.fr OMGSYMSL-ModelicaGroup SysML/ALF/OCL/Modelica

Context Works Synthesis

Outline

1 Context and TeamResearch TeamContext

2 Current WorksFrameworkModeling Consistency VerificationModel Transformation Validation

3 Synthesis

Fabrice Bouquet MBT SysML 20 / 23

Page 29: SysML Model-Based Testingomgwiki.org/OMGSysML/lib/exe/fetch.php?media=sysml...SysML Model-Based Testing Fabrice Bouquet fabrice.bouquet@univ-fcomte.fr OMGSYMSL-ModelicaGroup SysML/ALF/OCL/Modelica

Context Works Synthesis

Overview

Fabrice Bouquet MBT SysML 21 / 23

Page 30: SysML Model-Based Testingomgwiki.org/OMGSysML/lib/exe/fetch.php?media=sysml...SysML Model-Based Testing Fabrice Bouquet fabrice.bouquet@univ-fcomte.fr OMGSYMSL-ModelicaGroup SysML/ALF/OCL/Modelica

Context Works Synthesis

Conception Choice

I SysML and (Modelica) Connector?I Connector as Stereotype for Block Vs Flow SpecificationI Use Stereotype on flow properties

I For a subset with only "Model" can be optional?

I Tool for simulation of Modelica?I Interactive simulationI documented API

Fabrice Bouquet MBT SysML 22 / 23

Page 31: SysML Model-Based Testingomgwiki.org/OMGSysML/lib/exe/fetch.php?media=sysml...SysML Model-Based Testing Fabrice Bouquet fabrice.bouquet@univ-fcomte.fr OMGSYMSL-ModelicaGroup SysML/ALF/OCL/Modelica

Context Works Synthesis

Thank you for your attention

Source - http://model-based-testing.info

Research supported by:I SyVAD Project: http://syvad.univ-fcomte.fr/I Smart Blocks Project: http://smartblocks.univ-fcomte.fr/I Labex Action: http://www.labex-action.fr

Fabrice Bouquet MBT SysML 23 / 23