guest editorial for the special issue on model-based testing

2
SOFTWARE TESTING, VERIFICATION AND RELIABILITY Softw. Test. Verif. Reliab. 2012; 22:363–364 Published online in Wiley Online Library (wileyonlinelibrary.com). DOI: 10.1002/stvr.1481 Editorial Guest Editorial for the Special Issue on Model-Based Testing Volume 2: Formal Approaches to Model-Based Testing Decades of research on model-based testing have spawned a large number of different techniques and tools to derive test cases from models. One particular advantage embraced by researchers is the fact that model-based testing makes it possible to reuse many of the techniques developed in the formal methods and verification domain. This is exemplified in three very different approaches in this second volume of the Special Issue on Model-Based Testing. The first paper, “Formal Passive Testing of Timed Systems: Theory and Tools” by Cesar Andres, Mercedes Merayo, and Manuel Nunez, concerns passive testing, in which a system is tested by either observing it while it is running or by inspecting traces after a sufficiently long run, which is in contrast to active testing where inputs are sent to the system under test. The paper describes a suite encompassing both formal methodologies, algorithms, and implemented tools for testing systems where timing aspects are central. The properties to be tested are specified as invariants, which can be checked against a specification, if one is available. The PASTE tool then checks these invariants against logs, and mutation analysis is used to estimate the quality of a set of invariants. The second paper, “Scenario-Based Testing using Symbolic Animation of B Models” by Frederic Dadeau, Kalou Cabrera Castillos, and Regis Tissot, considers active testing of systems based on a formal model specified in terms of B machines. This approach is based on scenarios specified in a specialized language, which are used to drive the symbolic execution of the models. The execution step is based on a constraint solver and provides concrete parameters for the operations to reach specific states. Using this in a systematic way to generate scenarios using property patterns allows a property-oriented test generation strategy. The approach has been implemented in a prototype tool based on B models, and this has led to promising initial results, showing the potential compared with more traditional coverage-oriented approaches. The third paper, “On Reducing Test Length for FSMs with Extra States” by Adenilso Simao, Alexandre Petrenko, and Nina Yevtushenko, considers the use of finite state machines (FSMs) for testing. This is one of the oldest scenarios used in model-based testing, yet traditionally, research in this area to date is based on restricting assumptions. One of these assumptions is that the number of states of the implementation is not larger than the number of states of the model. This paper investigates the problem of test generation from FSMs when the implementation contains addi- tional states and presents the SPY algorithm for this setting. This algorithm is analyzed theoretically and experimentally, and it is shown that this algorithm even produces up to 40% shorter tests than existing techniques. Guest editors PAUL AMMANN Information Systems and Software Engineering, George Mason University MS 4A4, Fairfax, VA 22030 USA Copyright © 2012 John Wiley & Sons, Ltd.

Post on 15-Oct-2016

214 views

Category:

Documents


1 download

TRANSCRIPT

SOFTWARE TESTING, VERIFICATION AND RELIABILITYSoftw. Test. Verif. Reliab. 2012; 22:363–364Published online in Wiley Online Library (wileyonlinelibrary.com). DOI: 10.1002/stvr.1481

Editorial

Guest Editorial for the Special Issue on Model-Based Testing

Volume 2: Formal Approaches to Model-Based Testing

Decades of research on model-based testing have spawned a large number of different techniquesand tools to derive test cases from models. One particular advantage embraced by researchers is thefact that model-based testing makes it possible to reuse many of the techniques developed in theformal methods and verification domain. This is exemplified in three very different approaches inthis second volume of the Special Issue on Model-Based Testing.

The first paper, “Formal Passive Testing of Timed Systems: Theory and Tools” by Cesar Andres,Mercedes Merayo, and Manuel Nunez, concerns passive testing, in which a system is tested byeither observing it while it is running or by inspecting traces after a sufficiently long run, which is incontrast to active testing where inputs are sent to the system under test. The paper describes a suiteencompassing both formal methodologies, algorithms, and implemented tools for testing systemswhere timing aspects are central. The properties to be tested are specified as invariants, which canbe checked against a specification, if one is available. The PASTE tool then checks these invariantsagainst logs, and mutation analysis is used to estimate the quality of a set of invariants.

The second paper, “Scenario-Based Testing using Symbolic Animation of B Models” by FredericDadeau, Kalou Cabrera Castillos, and Regis Tissot, considers active testing of systems based on aformal model specified in terms of B machines. This approach is based on scenarios specified in aspecialized language, which are used to drive the symbolic execution of the models. The executionstep is based on a constraint solver and provides concrete parameters for the operations to reachspecific states. Using this in a systematic way to generate scenarios using property patterns allows aproperty-oriented test generation strategy. The approach has been implemented in a prototype toolbased on B models, and this has led to promising initial results, showing the potential comparedwith more traditional coverage-oriented approaches.

The third paper, “On Reducing Test Length for FSMs with Extra States” by Adenilso Simao,Alexandre Petrenko, and Nina Yevtushenko, considers the use of finite state machines (FSMs) fortesting. This is one of the oldest scenarios used in model-based testing, yet traditionally, research inthis area to date is based on restricting assumptions. One of these assumptions is that the numberof states of the implementation is not larger than the number of states of the model. This paperinvestigates the problem of test generation from FSMs when the implementation contains addi-tional states and presents the SPY algorithm for this setting. This algorithm is analyzed theoreticallyand experimentally, and it is shown that this algorithm even produces up to 40% shorter tests thanexisting techniques.

Guest editors

PAUL AMMANN

Information Systems and Software Engineering,George Mason University MS 4A4,

Fairfax, VA 22030 USA

Copyright © 2012 John Wiley & Sons, Ltd.

364 EDITORIAL

GORDON FRASER

Department of Computer Science,University of Sheffield Regent Court,

211 Portobello, S1 4DP, Sheffield UKE-mail: [email protected]

FRANZ WOTAWA

Institute for Software Technology,Graz University of Technology Inffeldgasse 16b/2,

Graz 8010 Austria

Copyright © 2012 John Wiley & Sons, Ltd. Softw. Test. Verif. Reliab. 2012; 22:363–364DOI: 10.1002/stvr