vetess : mde, testing approaches and sysmlfondement.free.fr › lgl › docs › papers ›...
TRANSCRIPT
VETESS : MDE, Testing approaches and SysML
May 2010
214/01/11
Project organization
Vérification de systèmes embarqués VEhicules par génération automatique de TESts à partir des SpécificationsChecking automotive embedded systems with automatic test case generation from specifications
• Started in 07/2008• En in 08/2010
314/01/11
Plan
Test in system engineering
Model Based Testing
The VETESS tool chain
The front wiper case study
Outlook
414/01/11
Plan
Test in system engineering
Model Based Testing
The VETESS tool chain
The front wiper case study
Outlook
Test in system engineering
514/01/11
Product specifications Product
Requirements
Specifications
Prototyping Unit testing
Fonctionnaltesting
Validation plan, integration
Integration
HIL
IntegrationDesign
Conception
Implementation
Problem: sharing specifications
614/01/11
Product specifications Product
Requirements
Specifications
Prototyping Unit testing
Fonctionnaltesting
Validation plan, integration
Integration
HIL
IntegrationDesign
Conception
Implementation
714/01/11
Project objectives
Checking automotive embedded systems with automatic test case generation from specifications• The purpose is to generate test cases directly
from the models representing system specification
814/01/11
Project objectives
Checking automotive embedded systems with automatic test case generation from specifications• Specification models:
− UML or SysML
• Test cases:− Enter test beds
914/01/11
Plan
Test in system engineering
Model Based Testing
The VETESS tool chain
The front wiper case study
Outlook
1014/01/11
Testing
Verification: checking coherence between• A model as engineered from requirements to
represent the system− Functional behavior− Non-functional properties (performance…)
• The system− At runtime− Alone or in situation
1114/01/11
Testing
Static verification• Code reviews• Model checking / proof
Dynamic verification• System stimulation and behavior control
− Can’t cover all possible cases– Most representative test cases have to be selected
− Often more expensive that the system itself– Execute tests– Produce / maintain tests– Analyze results
Testing
Questions:• Do I have enough tests ?• Are my test covering all possibilities offered
by the specification ?• Do my tests execute in a reasonable time ?
1214/01/11
1314/01/11
Test approaches
Manual approach• Scenarios describe
system stimulation and responses
• Human execute scenarios and check results
RequirementsTestplan
Testcases
Testresults
SUT
1414/01/11
Test approaches
Capture/replay approach• Stimuli and responses
are recorded and can be replayed automatically
SUT
RequirementsTestplan
Testcases Test
results
Recorder TestScripts
1514/01/11
Test approaches
Scripted approach• Scripts are written
− JUnit, TestNG,…
SUT
Requirements
Testplan
TestCases
Testresults
TestScripts
1614/01/11
Test approaches
Keyword-based approach• Scripts are written in
some pseudo-code• Concretization matches
keywords with real-life messages
SUT
Requirements
Testplan
Testcases
Testresults
TestScripts
Keywordenvironment
Adapter
Test Execution Tool
1714/01/11
Test approaches
Model Based Testing• A large number of
keyword-based tests are generated from an executable model
SUT
Requirements Testplan
TestCases
Testresults
Testscripts
Model Requirementstracability
matrixTest casesgenerator
Test scriptgenerator
Advantages
Automatically covers the specification model as much as possible
Test case mutualization• 1 test case for covering different parts of the
model
Predicts system behavior (oracle)
1814/01/11
Model Based Testing
14/01/11
SpécificationSpécification
Test caseTest casegenerationgeneration
tooltoolTest Test
RunnerRunnerInterfaceInterface
Test casesTest cases
1. Modeling 2. Generation 3. Concretization 4. Analysis
Simulation vs. test beds
2014/01/11
SpécificationSpécification
Test caseTest casegenerationgeneration
tooltoolTest Test
RunnerRunnerInterfaceInterface
Test casesTest cases
TestTestBedsBeds
SimulationSimulation
ModeMode
1. Modeling 2. Generation 3. Concretization 4. Analysis
2114/01/11
Plan
Test in system engineering
Model Based Testing
The VETESS tool chain
The front wiper case study
Outlook
2214/01/11
Test Designer
Model-based testing tool for discrete systems• The model is a behavioral specification
of the system− UML model or SysML models
• Test can be exported− Keyword− JUnit− HP Quality Center
Test-In-View
Testing continuous systems
Input and outputs are sampled signals• Expected signals are compared
to actual signals− Specific comparators
(Ignoring threshold, accepting latency…)
2314/01/11
Input
System Comparator
Actual
Expected
Result
Test may be exercised on• A system on a test bed• A simulation model
(Matlab, .NET…)
Test-In-View
2414/01/11
Input
System
ComparatorActual
Expected
Result
Model
Actual
Tool chain
2514/01/11
Test Designer
™ibd [block] PowerSubsystem [Al ternative 1 - Combined Motor Generator]
emg:ElectricMotorGener ator
trsm:Transmission
ice:InternalCombustionEngine
acl:accelerator
ecu:PowerControlUnit
ft:F uelT ankAssy
di f:Di fferential
rfw:Chass isSubsytem.FrontWheel
lfw:Chass is Subsytem.F rontWheel
Port:FuelTankFitting
Port:ICEFuelFi tting
fuelDelivery
torqueOut:T orque
torquein:T orque
spline
fuelSupply:Fuel
epc:ElectricalPowerController
bp:BatteryPack
i1:ElectricCurrent
i2 :ElectricCurrent
fp:F uelPump
fi:FuelInjector
fdistbp:BrakeSubsystem.BrakePedal
<>
<>
<><>
4
fuelReturn:F uel
<>
<>
<>
<>
g1:Torque
t2:T
orqu
e
t1:Torque
ice
ct rl
I_ICECmds
I_ICECmds
ctrl
ctrl
I_I CEData I_ICEData
trsmepc
c3
c2
c1
I_IEPCCmdI_IEPCData
I_IEPCDataI_EPCCmd
I_TRSMData
I_T RSMCmd
I_TRSMCmd
I_T RSMData
<>
<>
<>
rightHalfShaft
<>
<><>
leftHal fShaft
UML4MBTMODEL
SysML4MBTMODEL
UML MODEL(RSM, Topcased, …)
TestTestCasesCases
AdaptationAdaptationLayerLayer
+
SysML MODEL(Topcased)
Tool chain
2614/01/11
Test Designer
™ibd [block] PowerSubsystem [Al ternative 1 - Combined Motor Generator]
emg:ElectricMotorGener ator
trsm:Transmission
ice:InternalCombustionEngine
acl:accelerator
ecu:PowerControlUnit
ft:F uelT ankAssy
di f:Di fferential
rfw:Chass isSubsytem.FrontWheel
lfw:Chass is Subsytem.F rontWheel
Port:FuelTankFitting
Port:ICEFuelFi tting
fuelDelivery
torqueOut:T orque
torquein:T orque
spline
fuelSupply:Fuel
epc:ElectricalPowerController
bp:BatteryPack
i1:ElectricCurrent
i2 :ElectricCurrent
fp:F uelPump
fi:FuelInjector
fdistbp:BrakeSubsystem.BrakePedal
<>
<>
<><>
4
fuelReturn:F uel
<>
<>
<>
<>
g1:Torque
t2:T
orqu
e
t1:Torque
ice
ct rl
I_ICECmds
I_ICECmds
ctrl
ctrl
I_I CEData I_ICEData
trsmepc
c3
c2
c1
I_IEPCCmdI_IEPCData
I_IEPCDataI_EPCCmd
I_TRSMData
I_T RSMCmd
I_TRSMCmd
I_T RSMData
<>
<>
<>
rightHalfShaft
<>
<><>
leftHal fShaft
UML4MBTMODEL
SysML4MBTMODEL
UML MODEL(RSM, Topcased, …)
TestTestCasesCases
AdaptationAdaptationLayerLayer
+
SysML MODEL(Topcased)
UML4MBT / SysML4MBT
UML4MBT:Accepted input data• Formalized as a metamodel• Subset of UML• API (required interface) for Test Designer
SysML4MBT• Subset of SysML• Model transformation
from SysML4MBT to UML4MBT2714/01/11
Other languages can be supported through model transformation
2814/01/11
UML4MBTTest
DesignerModeleur UML+ TD Plug-in
TestCases
SysMLModeling Tool SysML4MBT
SysML4MBTto UML4MBT
Tool chain
2914/01/11
Test Designer
™ibd [block] PowerSubsystem [Al ternative 1 - Combined Motor Generator]
emg:ElectricMotorGener ator
trsm:Transmission
ice:InternalCombustionEngine
acl:accelerator
ecu:PowerControlUnit
ft:F uelT ankAssy
di f:Di fferential
rfw:Chass isSubsytem.FrontWheel
lfw:Chass is Subsytem.F rontWheel
Port:FuelTankFitting
Port:ICEFuelFi tting
fuelDelivery
torqueOut:T orque
torquein:T orque
spline
fuelSupply:Fuel
epc:ElectricalPowerController
bp:BatteryPack
i1:ElectricCurrent
i2 :ElectricCurrent
fp:F uelPump
fi:FuelInjector
fdistbp:BrakeSubsystem.BrakePedal
<>
<>
<><>
4
fuelReturn:F uel
<>
<>
<>
<>
g1:Torque
t2:T
orqu
e
t1:Torque
ice
ct rl
I_ICECmds
I_ICECmds
ctrl
ctrl
I_I CEData I_ICEData
trsmepc
c3
c2
c1
I_IEPCCmdI_IEPCData
I_IEPCDataI_EPCCmd
I_TRSMData
I_T RSMCmd
I_TRSMCmd
I_T RSMData
<>
<>
<>
rightHalfShaft
<>
<><>
leftHal fShaft
UML4MBTMODEL
SysML4MBTMODEL
UML MODEL(RSM, Topcased, …)
TestTestCasesCases
AdaptationAdaptationLayerLayer
+
SysML MODEL(Topcased)
Adaptation Layer
Generated test cases• Have to be concretized• Messages : discrete nature
Test bed inputs• Have to be mapped to reality• I/O Signals: continuous nature (sampled)
Automatic matching• Generate variables• Generate signals
3014/01/11
Adaptation Layer
Discrete messagesgenerate continuous signals• Example: turnLeft(moderately) message
changes the signal to -90° to be senton the SteeringWheelAngle variable
• Curves are smoothed
3114/01/11
turnLeft(moderately)SteeringWheelAngle
Testing systems in 4 steps
Create specification model in SysML• States discrete system behavior
(Could also state environment behavior)
Generate tests from model• Take the shape of sampled signals
sent to / read from variables
Map variables to reality• CAN messages, input pins…• Select comparators
Run the tests
3214/01/11
Refining expected signals
Additional step:provide a continuous mathematical model• Generated test are run against the
mathematical model• Output signals are promoted
to expected signals
3314/01/11
3414/01/11
Plan
Test in system engineering
Model Based Testing
The VETESS tool chain
The front wiper case study
Outlook
3514/01/11
Plan
Test in system engineering
Model Based Testing
The VETESS tool chain
The front wiper case study
Outlook