1 Levi Lúcio©

A Test Selection Language for CO-OPN Specifications

Levi Lúcio, Luis Pedro

and Didier Buchs

University of Geneva

2 Levi Lúcio©

Presentation Plan

The CO-OPN Modeling Language CO-OPN as a prototyping tool

Testing Theory Stating hypotheses about the SUT for test selection

A Test Selection Language for CO-OPN Specifications Language Structure Hypotheses

• On Paths

• On Values

Conclusion

3 Levi Lúcio©

Presentation Plan

The CO-OPN Modeling Language CO-OPN as a prototyping tool

Testing Theory Stating hypotheses about the SUT for test selection

A Test Selection Language for CO-OPN Specifications Language Structure Hypotheses

• On Paths

• On Values

Conclusion

4 Levi Lúcio©

The CO-OPN Modeling Language

CO-OPN (Concurrent Object-Oriented Petri Nets) is a formal specification language, with clearly stated semantics. Based on:

Algebraic Data Types: data and associated operations Petri Nets: behavior

It includes object based concepts: Class

• Attributes: places in a Petri Net

• Methods: guarded transitions in a Petri Net

Powerful synchronization mechanism Coordination of the components of a specification

• Objects

• Contexts (stateless coordination units) “All or nothing” semantics (sequence / parallel / choice)

5 Levi Lúcio©

Structure of a CO-OPN Specification

Contexts

(instances of) Classes

Algebraic Data Types

Coordinate

Use

Coordinate

Coordinate

6 Levi Lúcio©

Example specification in CO-OPN

A (simple) mobile phone is composed by: A card containing (among other things) the personal

identification number (PIN) to activate the phone; A phone controller holding (among other things) the state of

the phone, which can be:• Off

• Standby (waiting for the correct PIN)

• Idle (ready to receive or make calls)

In our specification the user can perform two actions: Turn on the phone; Insert a PIN number

7 Levi Lúcio©

Outer ContextCoordinates the Phone Controller and the Card

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

gate (output event)

method(inputevent)

8 Levi Lúcio©

Class PhoneController

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Algebraic Data TypesTyped values (+ operations) inside the Petri Net places…

9 Levi Lúcio©

CO-OPN as a Prototyping Tool

Our tool (CoopnBuilder) is able to produce Java code from a CO-OPN specification. We can then execute the specification…

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

10 Levi Lúcio©

SUT (System Under Test) Verification

We can “run” the specification to make sure the prototype will act as expected, but…

What if the prototype is changed / extended, or simply implemented in a language other than Java?

There is no way of keeping a record of previously tested behaviors (for example for regression testing…)

Solution: devise a testing language to include test modules in CO-OPN specifications:

The test module will use the specification to generate syntactically and semantically meaningful tests;

Prototype execution provides an easy way of deciding what should be the result of a given test case when it is applied to the SUT (System Under Test).

11 Levi Lúcio©

Presentation Plan

The CO-OPN Modeling Language CO-OPN as a prototyping tool

Testing Theory Stating hypotheses about the SUT for test selection

A Test Selection Language for CO-OPN Specifications Language Structure Hypotheses

• On Paths

• On Values

Conclusion

12 Levi Lúcio©

Presentation Plan

The CO-OPN Modeling Language CO-OPN as a prototyping tool

Testing Theory Stating hypotheses about the SUT for test selection

A Test Selection Language for CO-OPN Specifications Language Structure Hypotheses

• On Paths

• On Values

Conclusion

13 Levi Lúcio©

Testing Theory

Problem: How do we select “interesting” tests? In the general case, the amount of tests for a given system

is infinite: Infinite domains for operation’s domains; Infinite paths of execution in the SUT’s state space.

It is necessary to restrict the “exhaustive” test set to a practicable one. This is done by providing hypotheses about the functioning of the SUT:

Reducetestset

Increasehypothesis

strength

H0

H1…

Hk-1Hk

T0

T1…

Tk-1Tk

14 Levi Lúcio©

Testing Process

Does P satisfy SP?

P SP

Test Selection(Hypotheses H Application)

Execution of P using T

OracleP satisfies, or not, T!

P does not satisfy SP Undefined P satisfies SP!

P satisfies, or not, H

Test Set T

no inconclusive yes

no yes

Correction of P

TestProcedure

15 Levi Lúcio©

Presentation Plan

The CO-OPN Modeling Language CO-OPN as a prototyping tool

Testing Theory Stating hypotheses about the SUT for test selection

A Test Selection Language for CO-OPN Specifications Language Structure Hypotheses

• On Paths

• On Values

Conclusion

16 Levi Lúcio©

Presentation Plan

The CO-OPN Modeling Language CO-OPN as a prototyping tool

Testing Theory Stating hypotheses about the SUT for test selection

A Test Selection Language for CO-OPN Specifications Language Structure Hypotheses

• On Paths

• On Values

Conclusion

17 Levi Lúcio©

Test Language Structure

CO-OPN event pairsinput event (method), output event (gate)

(black-box approach)

HML (Hennessy-Milner Logic)next <_> operatornot, and operators

Hypothesesconstraints over theexhaustive test set

18 Levi Lúcio©

Mobile Phone Example Revisited

Let us create a test module for an extended version of the Mobile phone CO-OPN specification:

A (simple) mobile phone is composed by:• A card containing (among other things) the personal

identification number (PIN) to activate the phone. The card gets blocked after 3 wrong attempts at inserting a PIN.

• A phone controller holding (among other things) the state of the phone, which can be:

– Off

– Standby (waiting for the correct PIN)

– Idle (ready to receive or make calls) In our specification the user can perform two actions:

• Turn on the phone;

• Insert a PIN number,

• Reset the card when it becomes blocked

19 Levi Lúcio©

ConstraintSet NatelBehaviors;

Interface

Constraints insertPins; block; reachOn;

Body

Constraints nWrongPins;

Use Boolean; Pin;

Axiom [] in nWrongPins; f in nWrongPins => f . HML(<insertPin(newPin(1 1 1 1)),pinResult(false)>) in nWrongPins; f in nWrongPins, depth(f) <= 3 => HML(<turnOn,null>) . f . HML(<m,g>) in insertPins; depth(f) <= 3 => HML(<turnOn,null>) . f . HML(<resetCard,null>) in block; depth(f) <= 3 => HML(<turnOn,null>) . f . HML(<insertPin(newPin(1 2 3 4)),phoneReady>) in reachOn;

Variables f : HML; m : method; g : gate;

End NatelBehaviors;

Test Module for the Mobile Phone

Exported constraints

Locally defined constraints

Imported Algebraic Data Types

Variables used in constraint definition Constraints’Semantics

20 Levi Lúcio©

Hypotheses on Paths

The language allows defining the shape of sequences of input/output pairs that will be used to generate tests:

[] in nWrongPins;f in nWrongPins => f . HML(<insertPin(newPin(1 1 1 1)),pinResult(false)>) in nWrongPins;f in nWrongPins, depth(f) <= 3 => HML(<turnOn,null>)} . f . HML(<m,g>) in insertPins;

nWrongPins (local constraint) defines an (infinite) sequence of insertPin inputs with a wrong PIN;

insertPins defines a sequence of operations starting by turnOn, followed by a sequence of at most three insertPin inputs with a wrong PIN and finishing by any input/output pair.

HML(<turnOn,null>,<insertPin(newPin(1 1 1 1)),pinResult(false)>,<resetCard,null>),falseHML(<turnOn,null>,<insertPin(newPin(1 1 1 1)),pinResult(false)>, <insertPin(newPin(1 1 1 1)), pinResult(false)>,<insertPin(newPin(1 1 1 1),pinResult(false)>,<resetCard,null>),trueHML(<turnOn,null>,<insertPin(newPin(1 2 3 4)),pinResult(true)>,true...

Examples of tests generated by the insertPin constraint:

21 Levi Lúcio©

Hypotheses on Values

By default, the tests generated by the pinTest will correspond to all values of type Pin - large amount

Solution 1:

Axioms HML(<turnOn,null>,<insertPin(p), g>) in pinTest;

Variables p : Pin; g : gate;

uniformity(p) => HML(<turnOn,null>,<insertPin(p), g>) in pinTest

Choose one value from the domain - equivalent to random testing…

Corresponds to the hypotheses: “if insertPin works well for one PIN, it will work well for all PINs”.

22 Levi Lúcio©

Hypotheses on Values (2)

Solution 2:

subUniformity(p) => HML(<turnOn,null>,<insertPin(p), g>) in pinTest

Choose one value from each equivalence class of the behavior of the insertPin operation.

insertPin(p)P is the good PIN => Send pinResult(true)

P is a wrong PIN => Send pinResult(false)

In order to decompose the behavior of an operation we need to analyze its choice points…

This is done by analyzing the CO-OPN specification. Feature under development, using Prolog for symbolic execution and constraint

resolution.

23 Levi Lúcio©

Managing Non-Determinism in the SUT

We would like to manage behaviors of the SUT which cannot be calculated in test generation time…

Example: an e-banking application proposes a challenge as part of the authentication mechanism. How do we calculate the reply to the challenge in order to login successfully during the test?

Solution: Introduce the possibility to have “external” functions that will perform calculations in testing time.

In the e-banking case there would be a function to calculate the answer to the challenge in function of the proposed value.

24 Levi Lúcio©

Conclusion

We have presented a test selection language intended at testing software (and possibly hardware) prototypes specified in CO-OPN;

The test selection is done by the test engineer, assisted by the tool while decomposing the behavior of the operations in a specification;

The tool is currently half implemented. We envisage on the short term a case study to fully validate our ideas, also in terms of scalability.