best test cases selection approachconf-scoop.org/acse-2014/1.a.alhroob_acse.pdfthe executable and...

Best Test Cases Selection Approach

Aysh Alhroob

Department of Software Engineering, Isra University

Amman- Jordan

[email protected]

Abstract— This paper proposes an approach for selecting best

testing scenarios using Genetic Algorithm. Test cases generation

approach using UML sequence diagrams, class diagrams and

Object Constraint Language (OCL) as software specifications

sources. There are three main concepts: Edges Relation Table

(ERT), test scenarios generation and test cases generation used in

this work. The ERT is used to detect edges in sequence diagrams,

identifies their relationships based on the information available

in sequence diagrams and OCL information. ERT is also used to

generate the Testing Scenarios Graph (TSG). The test scenarios

generation technique concerns the generation of scenarios from

the testable model of the sequence diagram. Path coverage

technique is proposed to solve the problem of test scenario

generation that controls explosion of paths which arise due to

loops and concurrencies. Furthermore, GA used to generates test

cases that covers most of message paths and most of combined

fragments (loop, par, alt, opt and break), in addition to some

structural specifications.

Keywords— Test Cases; Software Specifications; UML

Diagram; Genetic Algorithm.

I. INTRODUCTION

There are many forms of testing a software system. One of

them is structural specifications testing [1]. On the other hand,

testing of behavioural specifications of a software system is

very important to evaluate the system interaction and for

testing scenarios. A critical component of testing is the

construction of test cases. Test cases may be used to detect any

faults and problems that exist in the software design even

before the program is implemented. A test case contains test

input values, expected output and the constraints of

preconditions and post conditions for the input values.

Generation and selection of the effective test cases from UML

models is one of the most challenging tasks [3].

Sequence diagram is one of the main UML design

diagrams. It is used to represent the behavioural details and

specifications of a software system. However, sequence

diagram alone does not express liveness/progress properties or

can differentiate between necessary and possible behaviour [2].

To address these limitations, UML class diagram and OCL are

also used in this work to model the software and generate test

cases more accurately. This paper aims to generate test cases

that are able to cover the interaction faults and scenario faults.

Figure 1 represents the proposed approach in this work and

introduces three main components: Edges Relation Table

(ERT), Test Scenarios Generation and Test Cases generation.

The scenarios generation technique concerns the generation of

scenarios from the testable model of the sequence diagram.

Coverage of all paths is a major problem in generating test

scenarios as explosion of paths which arise due to loops and

concurrencies is to be properly dealt with. Each combined

fragment could be one scenario or combined testing scenarios

(interaction). Information Table (InfT) is proposed to enrich the

testing scenarios with necessary information being extracted

from class diagram [1].

Fig. 1. Test cases generation model

This paper is organized as follows: A related work is

described in Section 2. Decomposing Sequence Diagram

into Edges is described in Section 3. Edge relationship table

is discussed in Section 4. Section 5 describes Test cases

generation approach.. Finally, Section 6 draws the

conclusions and proposes future work.

Scientific Cooperations International Workshops on Electrical and Computer Engineering Subfields 22-23 August 2014, Koc University, ISTANBUL/TURKEY

26

II. PREVIOUS WORK

The previous works provide automatic and manual

methodologies to extract test cases from one primary UML

diagram. These methods also use some other UML diagrams to

obtain additional information. These approaches do not de ne a

precise model for the generation of test paths which lead to

generation of test cases. Many fault types may exist in different

combinations of paths / statements. Detection of such faults in

all combinations is a hard problem as the number of statements

and faults increases with the size of a software system.

Systematic Test cases generation technique is needed to

overcome this combinatorial problem [14].

The research in [15] considered reusing of common

activities in model based testing and model based development.

The authors presented a case study (microwave oven) in which

the executable and translatable UML system models were used

for automatically generating test models in the

QTronicModeling Language using horizontal model

transformations.

Genetic Algorithms are used in [16] to generate and

optimize test cases from UML State Chart diagram. Crossover

method of GA is applied to generate the test sequence and the

Mutation Analysis is used for evaluating the efficiency of the

test sequence. The proposed algorithm was tested using a case

study of driverless train. The authors concluded that GA can be

used for test case generation; however, it can only be used

effectively if there are a large number of test cases and large

number of possible test sequences. Research in [17] also used

GA for improved test case generation. The authors presented

an approach that combine information from UML state chart

diagram used as finite state machines with GA. Test cases were

generated by transforming EFSMs into extended control flow

graph. GA is applied to generate feasible test sequence and test

data. The work is primarily motivated by three factors. First,

state-based testing is widely used in protocol software and

embedded system software. Second, generation of test

sequences using brute force method is very expensive. Third,

generation of test sequences using random test cases may not

always lead to feasible test paths, these needs to satisfy the

guard condition.

The authors noted that superior results were reported by GA

based technique.

The authors in [18] consider the problem of automatically

generating test cases for dynamic specification mining that

observes program execution to infer models of normal

behaviour. If a sufficient number of tests are not available, the

resulting specification may be too incomplete to be useful,

therefor necessitating the requirement for systematic test case

generating. The novelty of their approach lies in combining

systematic test case generation and typestate mining.

TAUTOKO, a typestate miner is used to generate test cases

that cover previously unobserved behaviour, systematically

extending the execution space, and enriching the specification.

Researchers in [19] considered user-driven test case

generation aimed at efficiently testing multimedia applications

concerning Digital TV (DTV) receivers and Set-Top Boxes

(STB). The proposed approached was experimentally tested

for correlation between the detected DTV functional failures

using the proposed user-driven test-case generation method and

subjectively perceived failures. Improvement in testing

efficiency and reduction compared to referent approaches.

The research work in [20] tackled the problem of detecting

and improving the test cases after mutation a problem where

artificial bugs referred to as mutants are injected into software

and the resulting test cases are executed on these fault injected

version. They proposed an approach, µTEST that automatically

generates unit tests for object-oriented classes based on

mutation analysis.

S. Ali et. Al. published a review of the application and

empirical investigation of search-based test generation. Readers

are referred to [21] for a detailed overview of studies

concerning search-based software testing and their empirical

evaluation.

III. DECOMPOSING SEQUENCE DIAGRAM INTO EDGES

(NODES)

The simplicity of sequence diagrams makes it easier for the

customers to express the requirements and understand them.

Unfortunately, shortage or lack of semantic content in

sequence diagrams makes them ambiguous and therefore

difficult to interpret [7]. Examination of the informal

documentation almost resolves the ambiguity but, in some

cases, these ambiguities may go undetected leading to costly

software errors. To overcome this problem, two solutions could

be used. Firstly, the user may provide messages with complete

semantic information. Secondly, if sufficient semantic

information is not available, one may interpret sequence

diagrams based on some heuristics. In the proposed approach

in this paper, a compromise has been done, whereby messages

in a sequence diagram may be annotated with a pre/post-

condition style specification expressed in OCL as shown in

Figure 3. Note that this is only a small additional burden on the

user since the amount of information required by our

methodology in this paper is actually very small.

The specifications should include the declaration of global state

variables where a state variable represents some important

aspect of the system, e.g., whether or not the user has inserted

valid coin into the coffee machine mentioned in Figure 2. The

pre/post-conditions should then include references to these

variables.

Now, there are two kinds of constraints on a sequence diagram:

constraints on the state vector given by an OCL specification,

and the constraints on the ordering of the messages given in the

sequence diagram [4]. Edges structure generation is based on

the edges vector as shown in Notation 1; edges vector is


27

Fig. 2 Sequence Diagram for a coffee machine

responsible for merging a proper object with its message. Thus,

the sequence diagram edges will be presented as follows:

𝑠0 → 𝑚1 → 𝑠0′ , 𝑠1 → 𝑚2 → ⋯ → 𝑚𝑟−1 → 𝑠𝑟−1

′ , 𝑠𝑟 → 𝑚𝑟 → 𝑠𝑟 (1)

Where, the 𝑚𝑖 is a message between objects and 𝑠𝑖 , 𝑠𝑖 , are

the edges state vector immediately before and after message

𝑚𝑖 is executed. The source and destination object of message

mi are denoted by 𝑚𝑖𝑠𝑜𝑢𝑟𝑐𝑒 and 𝑚𝑖

𝑑𝑒𝑠𝑡 , respectively.

𝑆𝑖 denotes either𝑠𝑖or 𝑠𝑖 .𝑆𝑖 ′ is the jth element of the vector𝑆𝑖 .

Let 𝑣𝑗 denote the name of the variable associated with position

j in the edge vector.

The initial edges vectors are generated directly from the

message specifications: if 𝑚𝑖has precondition 𝑣𝑗 = 𝑦 then let

𝑆𝑖 𝑗 : = 𝑦 , and if 𝑚𝑖has a post condition𝑣𝑗 = 𝑦, let𝑆𝑖 𝑗 : = 𝑦.

Identifying edges is not enough to establish a coherent testing

scenario. These edges are connected by many types of

relationships in a sequence diagram. These relations must be

identified to build an integrated testing scenario graph.

Fig. 3 OCL style specifications.

IV. EDGES RELATIONSHIPS TABLE (ERT)

Alhroob, Dahal and Alamgir [1] proposed an automatic

approach to generate hierarchy table to set the relationships

among classes in a systematic way. In this paper, this hierarchy

table is enhanced to the Edges Relationships Table (ERT) to set

the relationships between the edges automatically.

A. Testing Scenarios Graph (TSG)

In this section, the TSG is defined and then we present the

proposed methodology to generate TSG from a sequence

diagram. TSG is defined as follows (Notation 2):

TSG

TSGTSGTSGTSGTSGTSG BRSELPARFqNTSG ,,,,0, (2)

Where, NTSG is the set of all nodes of testing scenarios; each

node basically represents an event.

∑TSG is the set of edges representing transitions from one state

to another. q0TSG is the initial node representing a state from

which an operation begins. FTSG is the set of final nodes

representing states where an operation terminates.

PARTSG is the set of parallel edges and it represents states where

one operation red other operations at the same time.

SELTSG is the set of either alt or opt edges.

BRTSG is the set of edges that cause a sequence operation

termination.

The proposed methodology in this paper identities the set of

all testing scenarios where scni = scn1, scn2 , scn3 …… scnm

and the set of all nodes are also identified. Initially

TSGcontains only the StartState; then each node of all scni ∈scn should be followed by its corresponding NextState, and the

duplicates, if any, are removed. The StartState for different

scenarios as illustrated in [1] is StateA and five

FinalStatesare B, C, D, E and F. An operation starts with an

InitialState and undergoes a number of intermediate states due

to the occurrence of various edges. Initial edge(s) can be easily

detected from ERT. The algorithm of the generating TSG

from a sequence diagram uses the ERT as input to get the


28

sequence diagram testing scenario graph as the output. This

process goes through the following steps:

Find out all edges.

1. Detect the initial edge after InitialState and check if

it was visited before.

2. Detect the pre/post conditions of initial edge.

3. If it was visited before but it affects more than one

edge, it is revisited again.

4. If it was visited before and does not affect more

than one edge, go to next edge.

5. If the edge leads to FinalState, consider the path

from InitialState to FinalState as one scenario. 6. If two or more edges are related by parallel

relationship, connect all parallel edges by one circle notation at the beginning of process and in the end as well.

If two edges are related by option relationship, connect option

edges by one decision notation in the beginning of process.

Now, the TSG represents all of the sequence diagram

combined fragments and sequence messages. The initial edge

is detected by identifying its preconditions and post conditions.

If the precondition of an edge is not emerging from post

condition of another edge, it can be said that the former is an

initial edge. E1 in ERT is initial edge because it affects the

other edges but it is not affected by any other edges except the

loop edge. In contrast, the final edge is affected by others but it

cannot affect others except the loop edges.

B. Testing Scenarios

Testing scenarios generation is the main step towards the

generation of correct test cases. The proposed technique

enumerates all possible paths from the start edge to the final

edge of the TSG to generate test scenarios which cover all

edges. Each path then is visited to determine which Combined

Fragment represents it. Figure 2 represents 25 messages

mj(j = 1,2, … 25)between two objects with guard conditions

from which 25 corresponding edges Ei(i = 1,2, … 25) are

obtained. The proposed methodology in this paper generates 19

testing scenarios automatically to cover all paths and

operations as shown in Table 1. The variety of testing scenarios

is a main challenge in this phase. Five main novel features

(loop, par, alt, opt and break) are available in sequence diagram

as shown in Figure 2. This causes a large number of Combined

Fragments. The number of Combined Fragments directly

affects the number of scenarios. Furthermore, the increase in

the number of guard conditions (constraints) causes an increase

in the number of testing scenarios. Each testing scenario starts

with StartState and ends with FinalState. Each FinalState may

be the end of one or more scenario. For example, StateB

(forscn1), StateC (forscn2), StateD (forscn3), StateE (for

scn4andscn5) and StateF (for scn6, , andscn7 , scn8, scn9) are

FinalStates.

V. TEST CASES GENERATION APPROACH

Test cases are used to detect faults in a software system. A

software system is a combination of behavioural and structural

information. Sequence diagram provides behavioural

information. OCL is used to determine the pre/post conditions

of the edges but even then, some other additional information

is missing in a sequence diagram especially in the description

of method constraint and type information. This additional

information may be derived from the class diagram and then

can be appended to each message. Therefore, the structure of

each variable v in the method (e.g., name, type, value,

constraint) is obtained. Here, type and constraint refer to the

data type and the attribute constraints associated to a variable

v:name. The type information is used to map each variable

v:name to a range of values (min, max). This makes sure that

each variable on a path from the initial edge to the final edge

is mapped to a range of values. Furthermore, the constraint

information is used to appropriately set the boundary values

min and max. For example, the type information such as int to

a variable X sets(𝑚𝑖𝑛𝑥 , 𝑚𝑎𝑥𝑥) = (𝑚𝑖𝑛𝐼𝑛𝑡 , 𝑚𝑎𝑥𝐼𝑛𝑡 ).

Where 𝑚𝑖𝑛𝐼𝑛𝑡 and 𝑚𝑎𝑥𝐼𝑛𝑡 are the boundary values. Let us

consider𝑥 ≥ 5 as the constraint associated to a variable x. In

that case, (𝑚𝑖𝑛𝑥 , 𝑚𝑎𝑥𝑥) will be set to 5, 𝑚𝑎𝑥𝐼𝑛𝑡 ) which will

be recognized as the initial domain. Figure 1 represents the

class diagram determination to enrich the TSG by additional

information.

Now, each testing scenario must be represented by one test

case. For example, 𝑠𝑐𝑛1is presented by Test Case 1 in Figure

8. In this case, the variable coin has to be checked if an input

is a coin or not. Actually, in this case, no variable type can be

checked to ensure that. In other case studies, such as student

registration system, the type of variable plays an important

role to test the model. For example, for a message

enterStudentAge (age: integer), if the user used decimal

number to represent age, the test case must detect that error. If

the testing scenarios in Table 1 are mapped directly to test

cases, then we get 19 test cases to cover all model

specifications but these many cases will not be sufficient to

cover all sequence diagram features or all Combined

Fragments behaviour. Recall that a test scenario is a sequence

of operations starting with first edge and ending into final one,

whereas a test case must examine the Combined Fragment and

test whether it works well or not. The following sub-section

presents the combination of testing scenarios to generate

legitimate test cases.

A. Combining Testing Scenarios

The combination process is based on the similarity of functionality of scenarios. For example, the test scenarios 4, 5, 6 and 7 are similar in behavioural point of view but after E10, the edges 11, 12, 13 and 14 are fired in parallel. The work in [1] shows that each pair of edges (11, 12 AND 14, 15) are fired separately and are connected with selection node. To avoid confusion in how theses edges are related, SP node and EP


29

node are used in this work. Subsequently, the combination of test scenarios is specified for parallel operations. The proposed algorithm, in this paper, identifies the parallel edges in each scenario, and then combines all of the scenarios that are same but different in parallel edges. Thus, the scenarios 4

, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18 and 19 can be

combined as shown in Table 2. Therefore, the numbers of test

cases are reduced to 11 test cases instead of 19. This reduction

is important and necessary to produce legitimate test cases

because parallel operations can be covered only after

combining the relevant testing scenarios. In contrast, the

selection operations (alt, opt and break) and the loop testing

scenarios need not be combined because these scenarios

depend on the selection condition and, thus, these scenarios

are independent.

B. Legitimate Test Cases

After combining testing scenarios, the focus is on

generating legitimate test cases. The last step of test case

generation is to convert the testing scenarios (after

combinations) to actual test cases. The test scenarios which

are already populated with the necessary information from two

important design artefacts namely, sequence diagram and

OCL. Third design artefact the class diagram has provided the

information about types and constrains of parameters in a

message. In sequence diagram, the method m represents the

event from a sender class C1 to a receiver class C2. The

method m in class C2 has signature which is defined in the

class diagram. The method signature represents the name of

the method, types of the parameter and the return type while

the class attributes represent information about the instance

variables such as their names and types. In addition, there may

also be OCL constraints which are used to express invariants

on the class attributes [8]. These invariants identify attribute

constraints that are true for all instances of the class.

Information Table (InfT) represents the relationships between

the methods in sequence diagram and its signature in class

diagram. This technique completes the full image of test cases.

For each edge Ej of scenario scni the edge method m is

identified. For the purpose of discussion, two cases are used in

this paper: a method having a guard condition (see Notation 3)

and another method without guard condition. If the method m

has no guard condition, the method is represented in test case

as follows: 𝑇𝐶 = 𝑝𝑟𝑒𝐶, 𝐼 𝑎1, 𝑎2 …… . 𝑎𝑛 , 𝑂 𝑑1 , 𝑑2 …… . 𝑑𝑚 𝑝𝑜𝑠𝐶 (3)

where, TCis a Test Case.

preC = precondition of the method m.

𝐼 𝑎1 , 𝑎2 … … . 𝑎𝑛 = set of input values for the method n.

𝑂 𝑑1 , 𝑑2 … … . 𝑑𝑚 = set of resultant values in the object when

the method is executed.

posC= the post condition of the method m.

In contrast, when the method has a guard condition the test

case uses this condition to show the state of processing. For

example, the loop Combined Fragment in Figure 2 is affected

by a guard condition that forces the user to enter valid coins

and the coinAmount is set to 50. This operation repeats until

the suitable condition is reached that is acceptable according

to the given guard condition and then it goes to the next step.

TABLE I. COFFEE MACHINE TESTING SCENARIOS

coinAmount<50 will be appearing in test cases until the event

satisfied the guard condition. The test case with guard

condition semantic in Notation 4 is similar in Notation 3, but

the guard conditions (g(v)) appeared in test case.

C),g(v),pos,...,d,d),O(d,...,a,apreC,I(aTC mn 2121

Figure 4 shows the first 4 test case generated automatically to

test the coffee machine model. We need to analyse whether all

these test cases are legitimate.

Scn

ID

Testing Scenarios

Scn1 E1E2E3

Scn2 E1E2E4E5

Scn3 E1E2E4E6E7E8

Scn4 E1E2E4E6E7E8E9E10E11E13E17

E18E19


E18E19


E18E19


E18E19


E19E20

E21E22

Scn9 E1E2E4E6E7E8E9E10E12E13E17E18

E19E20E21E22


E19E20E21E22


E19E20E21E22


E19E20E21E23E25


E19E20E21E24E25


E19E20E21E23E25


E19E20E21E24E25


E19E20E21E23E25


E19E20E21E24E25


E19E20E21E23E25


E19E20E21E24E25


30

Legitimate set of test cases refers to the ability of test case

to achieve the best testing in minimum time and cost.

Removing redundant test cases is one of the major challenges

to obtain legitimate set of test cases. Redundant test case is

one, which if removed, will not affect effectiveness of fault

detection of the suite of remaining test cases [9]. In this paper,

redundant test cases cannot be present because redundancy is

detected automatically in the stage of TSG construction

before generating testing scenarios. On other hand, if test

cases are generated from only edges and nodes from sequence

diagrams, the resultant suite may contain redundancy. For

example, TC (4,5) and TC(6,7) contain similar edges except

[E11||E12] and [E14||E15]. So, the difference between two

Combined Testing Scenarios (CTS) occurs in the branches

that are affected by alt. Furthermore, before and after the

branches, the similarity between the CTSs can be seen.

Unfortunately, test cases based on the scenarios are not able to

eliminate redundant edges or nodes because each scenario

must be tested separately. To minimise this problem, we

propose an approach to select best testing scenario in the next

sub-section.

C. Best Testing Scenario Selection Approach Using Genetic

Algorithm

Test cases are derived from testing scenarios. An important

question is what is meant by the best testing scenario in the

sequence diagram? Coverage criteria play a major in selection

of the best scenario. In this section, the coverage criteria that

are used in this paper are listed. These criteria are used to

evaluate the test cases that have been generated previously

(see Figure 4). A total of 11 test cases satisfied our criteria.

We propose the following coverage criterion which is

expressed by the following condition.

Loop adequacy criterion: Test cases must contain at least

one scenario test case in which control reaches the loop and

then check the non-executable loop (zero iteration). Another

test case for testing the body of the loop that is executed at

least once before control leaves the loop (more than zero

iteration) [10]. Concurrent (parallel) coverage criterion: For each parallel

node in TSG, test cases must include one test case

corresponding to every valid interleaving of message

sequences.

All messages coverage criterion: For all messages in sequence

diagram, test cases must execute all message sequence paths

of the sequence diagram [11].

Branch coverage: Each decision outcome within the diagram

must be covered by at least one start-to-end path [26].

Selection coverage criterion: For each selection fragment

(alt, opt and break), test cases must include one test case

corresponding to each evaluation of the constraint.

The criteria above are covered by test cases that are generated

in this paper, but what is the best test case, which covers all

criteria?. When we say "best test case" we mean the one which

has the highest percentage of coverage. To select the best test

case, the criteria mentioned above are categorised into two

evaluation steps: first one includes all of the above coverage

criteria except the ―All message coverage criterion‖ and we

call it Combine Fragments Coverage Criterion (CFCC), and

the second one is specified for ―All messages coverage

criterion‖. Figure 5 shows the uses of these two criteria to

select the best CTS.

TABLE II. COMBINED TESTING SCENARIOS

To achieve this target, the proposed approach using GA

aims to generate testing scenarios to cover maximum Edges

using genetic algorithms technique. The ―All message

coverage criterion‖ depends on the weights of the edges. The

weight of the edges is calculated by the following Equation:

𝐸𝑖𝑤 =𝐸𝑝

𝑁 (1)

𝐸𝑖𝑤is the weight of each edge.

𝐸𝑝 is the edge presence probability in whole testing scenarios.

N is the total weights of all edges in all testing scenarios.

For example, the repeated use of the E10 is 8 times over all

combined testing scenarios in Table 2. The weight of E10 is

calculated by Equation 1 as follows,

54421769.0147

810 wE

.

GA is iterative procedures which work with chromosomes.

The chromosomes are a population of candidate solutions that

are maintained by the GAs throughout the solution process

[9]. At first a population of chromosomes is generated

CTSj Testing Scenario

CTS1 E1E2E3

CTS2 E1E2E4E5

CTS3 E1E2E4E6E7E8

CTS4 (4,5) E1E2E4E6E7E8E9E10[E11||E12]

E13E17E18E19

CTS5 (6,7) E1E2E4E6E7E8E9E10[E14||E15]

E16E17E18E19

CTS6 (7,8) E1E2E4E6E7E8E9E10[E11||E12]

E13E17E18E19 E20E21E22

CTS7(10,11) E1E2E4E6E7E8E9E10[E14||E15]

E16E17E18E19 E20E21E22

CTS8(12,14) E1E2E4E6E7E8E9E10[E11||E12]

E13E17E18E19 E20E21E23E25

CTS9 (13,15) E1E2E4E6E7E8E9E10[E11||E12]

E13E17E18E19 E20E21E24E25

CTS10 (16,18) E1E2E4E6E7E8E9E10[E14||E15]

E16E17E18E19 E20E21E23E25

CTS11(17,19) E1E2E4E6E7E8E9E10[E14||E15]

E16E17E18E19 E20E21E24E25


31

randomly. A selection operator is used to choose two

solutions from the current population. The selection process

used the measured goodness of the solutions (Fitness

Function). The crossover operator swaps sections between

these two selected solutions with a defined crossover

probability. One of the chromosomes solutions is then chosen

for application of the mutation process. The algorithm is

ended, when a defined stopping criterion is reached. Since our

approach focuses on finding a set of transitions by triggers

firing, a chromosome in our approach is a sequence of triggers

itself. Testing scenarios will be the first population from all

possible cases. The fitness value for each chromosome is

calculated from the number of Edges which is covered.

GAs operators used in proposed technique are the two

point crossover and random mutation. Based on some

experimentation and previous knowledge on GAs application

with other problem [20], the parameters of genetic operation

are set as follow:

• The crossover probability is 0.5.

• The mutation probability is 0.05.

• The size of population in each generation is 10.

Fig. 4 First 4 test cases for Coffee Machine system.

The All messages coverage criterion uses the weight of the

edges to calculate the weight of a CTS. This weight is used to

identify the best scenario. The intuition is that such a CTS

makes best messages coverage. Equation 2 shows that the

weight of a scenario is the sum of weights of all edges in that

scenario. The proposed methodology shows the combined

testing scenarios CTS8, CTS9, CTS10 and CTS11 have the

highest weight which means that these four CTSs are the best

from message coverage point of view.

n

i

ii EwwCTS0

1

(2)

Note that we have not adopted the All messages coverage

criterion as the only factor to select the best test scenario.

Second factor (CFCC) is also used to check the Combined

Fragments coverage. Based on this factor, the best CTS is that

one which is able to cover the highest number of Combined

Fragment's edges.

Fig. 5. Best test case selection methodology.

By referring to Figure 2, it can be noted that the Combined Fragments (loop, alt, par, opt and break) contain 13 edges (E1, E2, E3, E4, E5, E11, E12, E13, E14, E15, E16, E22 and E24). The best CTS is the one that covers highest number of these edges. Thus, the proposed approach in this paper examines the presence of Combined Fragment edges in each testing scenario. CTS10 covers 6 Combined Fragments edges (E1, E2, E4, E14, E15 and E16) out of 13. The CTSs with the best Combined Fragments coverage are CTS6, CTS7, CTS9 and CTS11.

The best combined testing scenario (s) is that one which

achieves highest coverage in both the criteria CFCC and All messages coverage criterion. CTS9 or CTS11 can be selected as best CTS because these two have the highest coverage in both these criteria. Let us say CTS11 is the best test scenario and the corresponding test case covers 88% of All messages coverage criterion and 55% of CFCC. As noted, the CTSis unable to capture the coverage of enough edges in the Combined Fragment as just 55% coverage may not be satisfactory.

To improve the coverage of the Combined Fragments edges,

we propose to select the second best combined testing scenario

as well to support the first one. This technique is similar to

that used in [1] to select second testing path to improve unit

coverage.

The selection method for the second best CFCC scenario

depends on the non-similarity of edges contained in the best

CFCC scenarios. In [1], the non-similarity criterion is used to

select the second best testing path, and the same is used in this

paper as well. Based on the non-similarity degree between the

best CFCC scenario and others, the scenarios which are

eliminated have the biggest similarity degree. The CTS6 is the

highest non-similar scenario in comparison with CTS11

because it has 4 different Combined Fragment's edges (E11,

E12, E13 and E22) compared to CTS11. Consequently, both scenarios CTS11 and CTS6 cover 85%

of Combined Fragments edges and 97% of All messages coverage criterion. Together they are considered as best testing scenarios.


32

VI. CONCLUSION AND FUTURE WORK

An approach is proposed to generate test cases automatically

from UML sequence diagram, class diagram and OCL. The

approach generates efficient test cases that meet required

coverage criteria. Furthermore, the relationship between edges

has been recognised by the edge relationships table. This table

is generated automatically to reveal the relationship of the

edges as specified in the sequence diagram. The proposed

methodology covers loop, parallel, alternative, option, break

and sequence relationships and a number of structural

specifications as well. This work provides an efficient

technique to generate testing scenarios graph that is used to

represent the testing scenarios of the events in a sequence

diagram. The testing scenario graph represents combined

fragments and shows the relationships among scenarios. The

proposed methodology decomposes testing scenarios graph to

combined test scenarios to achieve minimum number of

testing scenarios. Information Table (InfT) technique is

proposed to provide the sequence diagram with exact method

signatures that are obtained from the class diagram. Despite

this, presence of the redundant edges in each testing scenario

led us to develop a new technique to select best testing

scenario. To achieve this, we have used All messages coverage

criterion and Combined Fragments coverage criterion to

evaluate the best testing scenario (and the best test case) using

GA. The weakness that appeared in selecting best testing

scenario is a variety of the coverage percentage that the best

testing scenario can cover. This variety comes from the variety

of the model designs. Generally, this technique almost covers

more than 85% of Combined Fragment's edges and more than

94% of the sequence diagram messages.

The test cases that are generated automatically in this paper

meet the coverage criteria. However, UML sequence diagrams

do not contain all information related to verification and non-

functional parameters of the software. This limitation comes

from the semi-formal characteristics of UML diagrams [12],

especially during the first cycle of the software production. In

order to avoid its semi-formal characteristics, we propose to

transform sequence diagram and class diagram to HLPN, a

type of Petri Net in future. Petri Net (PN) is a graphical

diagram for the formal description of the flow of activities in

complex systems [13].

REFERENCES

[1] A. Alhroob, K. Dahal, and H. Alamgir, ―Automatic Test Cases

Generation from Software Specifications Modules," in Proceedings of

the 4th IFIP TC2 Central and East European. Conference on Software

Engineering Techniques CEE-SET. Springer, 2009, pp. 130-142.

[2] A. Cavarra and J. Kuster-Filipe, ―Combining sequence diagrams and OCL for liveness," Electronic Notes in Theoretical Computer Science, vol. 115, pp. 19-38, 2005.

[3] A. O utt, Z. Jin, and J. Pan, ―The dynamic domain reduction procedure

for test data generation," Software: Practice and Experience, vol. 29, no. 2, pp. 167{193, 1999.

[4] [4]B. Li, Z. Li, L. Qing, and Y. Chen, ―Test Case Automate Generation

from UML Sequence Diagram and OCL Expression," in Proceedings of

the 2007 International Conference on Computational Intelligence and

Security. IEEE Computer Society, 2007, pp. 1048-1052. [5] G. Booch, J. Rumbaugh, and I. Jacobson, UniedModeling Language

User Guide, The (Addison-Wesley Object Technology Series), 2005. [6] D. Aredo, ―A framework for semantics of UML sequence diagrams in

PVS," Journal of Universal Computer Science, vol. 8, no. 7, pp. 674{697, 2002.

[7] X. Li, Z. Liu, and H. Jifeng, ―A Formal Semantics of UML Sequence Diagram," in Proceedings of the 2004 Australian Software Engineering Conference. IEEE Computer Society, 2004, p. 168.

[8] OMG, ―Object constraintlanguage, version 2.2,"

http://www.omg.org/spec/OCL/2.2.htm [Last accessed in August 10, 2010],2010. [Online]. Available: http://www.omg.org/spec/OCL/2.2.htm

[9] N. Koochakzadeh and V. Garousi, ―A Tester-Assisted Methodology for Test Re-dundancy Detection," Journal on Information and Software Technology, vol. 52, no. 5, pp. 625-640, 2010.

[10] J. Edvardsson, ―A survey on automatic test data generation," in Proceedings of the 2nd Conference on Computer Science and Engineering, 1999, pp. 21{28.

[11] A. Andrews, R. France, S. Ghosh, and G. Craig, ―Test adequacy criteria for UML design models," Software Testing Verication and Reliability, vol. 13, no. 2, pp. 95-127, 2003.

[12] H. Motameni, A. Movaghar, I. Daneshfar, and H. Zadeh, ―Mapping to Convert Activity Diagram in Fuzzy UML to Fuzzy Petri Net," World Applied Sciences Journal, vol. 3, no. 3, pp. 514-521, 2008.

[13] Bobbio, ―System modelling with Petri nets," in Systems reliability

assessment: proceedings of the Ispra course held at the EscuelaTecnica

Superior de IngenierosNavales, Madrid, Spain, September 19-23, 1988,

in collaboration with Universidad Politecnica de Madrid. Springer, 1990,

p. 103.

[14] DebasishKundu, DebasisSamanta, Rajib Mall "Automatic code

generation from unified modelling language sequence diagrams", IET

Software, Volume 7, Issue 1, February 2013, p. 12 – 28.

[15] Federico Ciccozzi, Antonio Cicchetti, Toni Siljamäki, "Automating Test

Cases Generation: From xtUML System Models to QML Test Models",

ACM proceeding in MOMPES ’10, September 20, 2010, Antwerp,

Belgium

[16] PreetiGulia, R. S. Chillar, "A New Approach to Generate and Optimize

Test Cases for UML State Diagram". ACM SIGSOFT Software

Engineering Notes archive, Volume 37 Issue 3, May 2012, Pages 1-5

[17] Mahesh Shirole, AmitSuthar, Rajeev Kumar, Generation of Improved

Test Cases from UML State Diagram Using Genetic Algorithm,

Proceeding of the 4th India Software Engineering Conference, ISEC

2011, Pages 125-134

[18] ValentinDallmeier, Nikolai Knopp, Christoph Mallon, Gordon Fraser,

Sebastian Hack, and Andreas Zeller, Member, "Automatically

Generating Test Cases for Specification Mining", IEEE

TRANSACTIONS ON SOFTWARE ENGINEERING, VOL. 38, NO.

2, MARCH/APRIL 2012

[19] TarkanTekcan, Vladimir Zlokolica, VukotaPekovic, Nikola Teslic and

Mustafa Gündüzalp, "User-driven Automatic Test-case Generation for

DTV/STB Reliable Functional Verification", IEEE Transactions on

Consumer Electronics, Vol. 58, No. 2, May 2012. Pages: 587-595

[20] Gordon Fraser and Andreas Zeller, "Mutation-Driven Generation of Unit

Tests and Oracles", IEEE TRANSACTIONS ON SOFTWARE

ENGINEERING, VOL. 38, NO. 2, MARCH/APRIL 2012, Pages: 287-

292

[21] Shaukat Ali, Lionel C. Briand, HadiHemmati, and Rajwinder K.

Panesar-Walawege "A Systematic Review of the Application and

Empirical Investigation of Search-Based Test Case Generation", IEEE

TRANSACTIONS ON SOFTWARE ENGINEERING, VOL. 36, NO.

6, NOVEMBER/DECEMBER 2010


33

best test cases selection approachconf-scoop.org/acse-2014/1.a.alhroob_acse.pdfthe executable and...

Documents