framework for constructing context- specific migration methods … · 2017-10-16 · framework for...

Framework for Constructing Context-Specific Migration Methods for Test Cases

Presented by Ivan Jovanovikj

© All rights reserved

2 © All rights reserved

1 Challenges in Test Case Reusein Migration Projects

2 Construction of Context-Specific Migration Methods

Agenda

3 Outlook and Conclusion

1. CHALLENGES IN TEST CASE REUSE INMIGRATION PROJECTS


Problem Statement


SystemMigratedSystem

test

Restructuring

ReverseEngineering

ForwardEngineering

MigratedModel

Model

testTest CasesTest Cases

Test the desired functionality of

the source system

Test Case Designis costly and time

consuming

Test Requirements must be considered

Changes from thesoftware migrationmust be considered

Problem StatementHow to systematically

reuse existing test cases in a software migration scenario?

SoftwareMigration

Solution Approach


Solution ApproachDevelopment of aContext-Specific

Test Case Migration Method

In this talk:Framework for

Constructing Context-Specific Migration Methods

for Test Cases


test testMigratedTest Cases

Test Cases

SoftwareMigration

2. Construction of Context-SpecificMigration Methods


Approach Overview


Migration Context

Characterization

Tool Implementation

Migration Method Tailoring

Migration Method

Enactment

ReferenceMigrationMethod

ContextModel

AdaptedMigrationMethod

Context-specific

Tools

Migration Validation

Pre-Migration Migration Post-Migration

Migration Context Characterization


System Migration Context

Testing Context

1. What was migrated?

2. How was it migrated?

3. What is the current state of the test cases?

4. What are the testing requirements?


Tool Implementation


Migration Method Enactment


Migration Context Factors


Migration ContextFactors

Source Environment

Characteristics

Language

Framework

Architecture

TransformationCharacteristics

Degree of Automation

Degree ofFormality

Process Support

Change Type

Target Environment

Characteristics

Language

Framework

Architecture

Source Testing Environment

Characteristics

Testing Framework

Test Case Type

Test Case Anatomy

Test Case Quality

Target Testing Environment

Characteristics

Testing Framework

Test Case Type

Test Case Anatomy

Test Case Quality

System MigrationContext Factors

TestingContext Factors

Tool Implementation





Eclipse Modeling Project

System Migration Context - Case Study


Modelling Framework

Ahead-of-TimeCompilation

(AOT)

Tool Implementation





&

Eclipse Modelling Framework

EMF.CodeGen

EMF EcoreEMF.Edit

Model Development Tools

BPMN2

MoDisco

…

OCL

Just-in-TimeCompilation

(JIT)

ecore

OCL

Testing Context - Case Study


OCL Test Framework

Testing Framework(JUnit)

Language(Java)

Test Cases

OCL Test Framework

Testing Framework(NUnit)

Language(C#)

Test Cases

Source Testing EnvironmentCharacteristics

Target Testing EnvironmentCharacteristics

1Migration of

TestOCL

2Migration of

Test Cases

Tool Implementation





Reference Migration Method


Language Metamodel

Test Framework Metamodel

UTP Meta-Model

instance of

instance of

instance of

Textual Artifact

Model

Model to Model Transformation

Text to Text Transformation

Text to Model / Model to Text

Transformation

Exogeneous Tranformation

Endogeneous Tranformation

Activities

Artifacts

MetaModel


Tool Implementation




ModelDiscovery

CodeGeneration

Platform SpecificTest Model

(Source)

TestCode

Reimplementation

TestAbstraction

TestConcretization

Restructuring

Platform SpecificTest Model(Migrated)

Platform IndependentTest Model

Model of the Code

Language Transformation

Model of the Migrated Code

Test CaseUnderstanding

TestConcretization

FrameworkTransformation

MigratedTestCode

Reductive Tailoring

Method Tailoring


Remove Activity

Remove Artifact


Tool Implementation




ModelDiscovery

CodeGeneration


(Source)

TestCode

Reimplementation

TestAbstraction

TestConcretization

Restructuring



Model of the Code




TestConcretization


MigratedTestCode

Method Tailoring


…

Context C4 specific Context C5 specific Context C6 specific

Context C8 specific Context C9 specific Context Cn specificContext C10 specific

Tailoring


Tool Implementation




Context C1 specific Context C2 specific Context C3 specific

Context C7 specific

ModelDiscovery

CodeGeneration


(Source)

TestCode

Reimplementation

TestAbstraction

TestConcretization

Restructuring



Model of the Code




TestConcretization


MigratedTestCode

Method TailoringTest Case Migration Method


Remove Activity

Remove Artifact

Reductive Tailoring


Tool Implementation




ModelDiscovery

CodeGeneration


(Source)

TestCode

Reimplementation

TestAbstraction

TestConcretization

Restructuring



Model of the Code




TestConcretization


MigratedTestCode

Method Tailoring: Test Case Migration Method


Remove Activity

Remove Artifact

Reductive Tailoring


Tool Implementation




Code

Generation

TestConcretization

+

ModelDiscovery


(Source)

TestCode

Reimplementation


Model of the Code



MigratedTestCode

Method Tailoring: TestOCL Migration Method


Remove Activity

Remove Artifact

Reductive Tailoring


Tool Implementation




ModelDiscovery

CodeGeneration


(Source)

TestOCLCode

Reimplementation

TestAbstraction

TestConcretization

Restructuring



Model of the Code




TestConcretization


MigratedTestOCL

Code

Method Tailoring: TestOCL Migration Method


Remove Activity

Remove Artifact

Reductive Tailoring


Tool Implementation




TestOCLCode

Reimplementation MigratedTestOCL

Code

Method Tailoring


2

1 TestOCLMigration Method

Test CaseMigration Method


Tool Implementation




TestOCLCode


Code

Code

Generation

TestConcretization

+

ModelDiscovery


(Source)

TestCode

Reimplementation


Model of the Code



MigratedTestCode

Tool Implementation


TestCase2TestModel

plugin

JDT Core

TestModel2TesCase

plugin

supports


Tool Implementation




Code

Generation

TestConcretization

+

ModelDiscovery


(Source)

TestCode

Reimplementation


Model of the Code



MigratedTestCode



ToolsMethod Specification

2 Test Case Migration

1 TestOCL Migration

TestOCLCode


Code


Tool Implementation




CodeGeneration

TestConcretization

+

ModelDiscovery


(Source)

TestCode

Reimplementation


Model of the Code



MigratedTestCode

TestCase2TestModel

plugin

TestModel2TesCase

plugin




Tool Implementation




Code

Generation

TestConcretization

+

ModelDiscovery


(Source)

TestCode

Reimplementation


Model of the Code



MigratedTestCode

TestCase2TestModel

plugin

JDT Core

TestModel2TesCase

plugin

AST

ModelDiscovery




xUnit model xUnit model

Reimplementation

Code

Generation

TestConcretization

+



Tool Implementation




TestCase2TestModel

plugin

JDT Core

TestModel2TesCase

plugin


ModelDiscovery

AST



TestModel2TesCase

TestCase2TestModel

plugin

JDT Core

plugin


Reimplementation

Code

Generation

TestConcretization

+



Tool Implementation





xUnit model



TestModel2TesCase

TestCase2TestModel

plugin

JDT Core

plugin

xUnit model

Reimplementation

Code

Generation

TestConcretization

+

ModelDiscovery



Tool Implementation




AST




TestModel2TesCase

TestCase2TestModel

plugin

JDT Core

plugin

xUnit model

Reimplementation

Code

Generation

TestConcretization

+

ModelDiscovery


xUnit model


Tool Implementation




AST



TestModel2TesCase

TestCase2TestModel

plugin

JDT Core

plugin

xUnit model

xUnit model

Reimplementation

Code

Generation

TestConcretization

+

ModelDiscovery




Tool Implementation




AST



TestModel2TesCase

TestCase2TestModel

plugin

JDT Core

plugin


Reimplementation

Code

Generation

TestConcretization

+

ModelDiscovery




Tool Implementation




AST



How to validate a test case migration?

What is a valid test case migration?

Test case migration is a process of transferring test cases into new environmentswithout changing their functionality, i.e., without changing what they test.

How to ensure behavioral equivalence in test case migration ?Migration Context Characterization

Tool Implementation






TestReport

Migration

Migrated systemworks as expectedErrors in the

migrated system

TestCases

MigratedTest

Cases

Execution

PassedTest

Cases(negatives)

FailedTest

Cases(positives) False

positives

Truepositives

Falsenegatives

Truenegatives

Problem: Detects non-existing errorsConsequence: Waste of time by looking for non-existing error

Problem: Hides potential errors Consequence: System malfunctioning/crashes


Tool Implementation






How to avoid/detect false positives and false negatives?

How to ensure behavioral equivalence in test case migration?

Constructive approaches

Analytical Approaches

Bi-simulation Mutation Testing

Cost ofTest Case Migration

Cost ofValidation of Test Case Migration

Cost ofDeveloping New

Test Cases+ <=

Bottom Line


Tool Implementation




OUTLOOK AND CONCLUSION


Overview and Conclusion


Migration Domain

Conceptualsolution

Comprehensive discussion on the

practical application


test testMigratedTest Cases

Test Cases

Software Migration

Test the desiredfunctionality of

the source system

Test Case Designis costly and

time consuming

Changes from thesoftware migrationmust be considered

Problem StatementHow to systematically reuse

existing test cases in a software migration scenario?

Test Case Migration

Test Requirements must be considered

MigrationContext

Characterisation

ToolImplementation

MigrationMethodTailoring

MigrationMethod

Enactment

ReferenceMigrationMethod

ContextModel

AdaptedMigrationMethod

Context-specificTools

MigrationValidation

Pre-Migration Migration Post-Migration

plugin

Tools

TestModel2TesCase

TestCase2TestModel

plugin

Method Specification

ModelDiscovery

CodeGeneration


(Source)

TestCode

Reimplementation MigratedTestCode


Model of the Code

Test CaseUnderstanding Test

Concretization


+

Eclipse Modeling Project

Model Development ToolsEclipse Modelling Framework

EMF.CodeGen

EMF Ecore EMF.Edit

BPMN2

MoDisco

…

Modelling Framework

ecore

OCL

Just-in-TimeCompilation

(JIT)

Ahead-of-TimeCompilation

(AOT)

OCL

&

System Migration Context

OCL Test Framework

Testing Framework(JUnit)

Language(Java)

Test Cases

OCL Test Framework

Testing Framework(NUnit)

Language(C#)

Test Cases

Source Testing EnvironmentCharacteristics

Target Testing EnvironmentCharacteristics

2

1

Testing Context


Thank you for your attention

s-lab – Software Quality Lab

Paderborn University

Zukunftsmeile 1

33102 Paderborn

Ivan Jovanovikj

Tel.: (05251) 54 65-216

[email protected]

http://s-lab.upb.de

mailto:[email protected]

http://s-lab.upb.de/

framework for constructing context- specific migration methods … · 2017-10-16 · framework for...

Documents