20041113 a test generation tool for specifications in the form of state machine
DESCRIPTION
A Test Generation Tool for Specifications in the Form of State MachineTRANSCRIPT
A Test Generation Tool
for Specifications in the
Form of State Machine
Zheng-Wen Shen
2004/11/23
References
A Test Generation Tool for Specifications in the
Form of State Machines
Q. M. Tan, A. Petrenko and G. v. Bochmann
To appear in IEEE International Conference on
Communications -- 96, Dallas, USA, 1996.
Outline
1. Introduction
2. Using the tool 1. An Example
2. Functions provided by the Tool
3. The FSM specification
3. Test derivation methods 1. Preambles
2. Postambles
3. Transition Cover
4. State Identification Sequences
4. Experiments and Applications
5. Conclusion
1. Introduction
2. Using the tool 1. An Example
2. Functions provided by the Tool
3. The FSM specification
3. Test derivation methods 1. Preambles
2. Postambles
3. Transition Cover
4. State Identification Sequences
4. Experiments and Applications
5. Conclusion
1. Introduction
TAG tool - Test Automatic Generation
Automatically generates test case for an FSM
specification.
Transition identification approach for test derivation
Output test cases in the form of an SDL skeleton.
FSM has been extensively used in the testing
phases of system developments.
E.g. conformance testing of communication
protocols.
Communication protocols are the rules that
govern the communication between the
different components within a distributes system.
7-layer OSI model
Protocol conformance testing
A protocol specification generally can lead to
several implementations in software and/or
hardware.
Assure the compatibility with other
implementations of the same protocol.
The existing test derivation methods
Check each transition in an FSM specification at
least once (branch coverage criteria.)
Verify the tail state of the transition to obtain
high fault coverage and to guarantee
conformance in the context of a more general
fault model.
Using state identification techniques
Defect the fault that the FSM enters a different state than
specified.
Most of the existing protocols are not
completely specified.
Not all the sequences of interactions are foreseen.
The existing test derivation methods for
protocols are limited to completely specified
specifications.
TAG working directly for deterministic, partially
specified FSM specifications.
1. Introduction
2. Using the tool 1. An Example
2. Functions provided by the Tool
3. The FSM specification
3. Test derivation methods 1. Preambles
2. Postambles
3. Transition Cover
4. State Identification Sequences
4. Experiments and Applications
5. Conclusion
2. Using the tool
A complete test suite that guarantees full fault
coverage may be derived
A set of test cases that cover a given test
purpose may be derived.
Output format
Mnemonic format
SDL skeleton
2.1 An example
To achieve a particular test purpose which is a certain
transition to be tested:
1. Bring the FSM from its initial state to the starting state of
the transition under test using the shortest input sequence
possible. – a preamble of the test case
2. Execute the transition and check the observed output
3. Check a tail state of the transition by observing its reaction
to a pre-selected set of state identification sequences.
4. Apply an input sequence to return to the initial state of the
FSM. – a postamble of the test case
State cover
The set of all preambles.
Transition cover
The set of sequences used to execute all specified
transitions.
State identification sequences
Distinguish states by their output reactions.
TAG implements the HSI method
HSI method
Similar to the widely used W-method
A characterization set is used for state identification.
A tuple of subsets of a W set.
Can be applied to partially specified FSMs.
The INRES responder
Input alphabet
1- CR; 2- IDISreq; 3- ICONrsp; 4- DT0; 5- DT1
Output alphabet
0- NULL; 1- ICONind; 2- DR; 3- CC; 4- ACK0;
5- ACK0,IDATind; 6- ACK1; 7- ACK1,IDATind
States
S0- Closed (the initial state); S1- Opening; S2-
Waiting_DT0; S3- Waiting_DT1
Test suite and intermediate results
State Identifier = {{41}, {41}, {4}, {4}}
Preamble = {ε, 1, 13, 135}
Transition Cover = {1, 4, 5, 11, 12, 13, 14, 15,
131, 132, 134, 135, 1351, 1352, 1354, 1355}
Postamble = {ε, 2, 2, 2}
Test Suite = {41, 441, 541, 1241, 13241, 135241,
141, 1141, 1441, 1541, 134, 1314, 13514, 13514,
13554}
2.2 Functions provided by the tool
A text file containing the FSM specification with suffix “.fsm”
Symbol table (.tbl)
FSM structure (.cpl)
Complete test derivation
A complete test suite is generated.
Selective test derivation
Give the test purpose, which specifies one transition in the FSM.
The test output is written in a text file.
Mnemonic format (.mnc)
SDL skeleton (.sdl)
2.3 the FSM specification
1. the state definitions
2. The input definitions
3. The output definitions
4. The transition definitions
5. The variable declaration (optional)
6. The homing sequence definition (optional)
7. “end;”
The state definitions are a list of state names.
The input definition are a list of input names.
The output definitions are list of output names.
The transition definitions define the FSM state table
itself by a list of transition specifications.
Current state name, input name, output name and next state
name.
May be followed by a set of the comments.
The variable definitions define the variables in
parameters.
Integer, Charstring, Octetstring, Boolean
Using Keyword “homing” to give a sequence of
input names as a homing sequence.
1. Introduction
2. Using the tool 1. An Example
2. Functions provided by the Tool
3. The FSM specification
3. Test derivation methods 1. Preambles
2. Postambles
3. Transition Cover
4. State Identification Sequences
4. Experiments and Applications
5. Conclusion
3.1 Preambles
A tree with the initial state as its root is constructed such that the tail states of the outgoing transitions from the state corresponding to a current node.
All nodes in this tree must become a current node once and only once in the order that they enter this tree.
The path from the root to a given node is preamble for the corresponding state.
S0
S1
S2 S3
1
3
5
Preamble = {ε, 1, 13, 135 }
3.2 Postambles
A tree with given state as its root is constructed.
Once the initial state has been added to the tree,
the procedure stops.
The path from the root to the last added node is
a postamble from the given state.
Maybe no postamble from the given state.
S0
S1
2
Preamble = {ε, 2, 2, 2 }
S0
S2
2
S0
S3
2
S0
3.3 Transition Cover
The transition cover can be obtained by appending
each outgoing transition from this state to its preamble.
Preamble = {ε, 1, 13, 135 }
S0:ε::1, ε::4, ε::5
S1: 1::1, 1::2, 1::3, 1::4, 1::5
S2: 13::1, 13::2, 13::4, 13::5
S3: 135::1, 135::2, 135::4, 135::5
Transition Cover = {1, 4, 5, 11, 12, 13, 14, 15, 131, 132,
134, 135, 1351, 1352, 1354, 1355}
3.4 State Identification Sequences.
The characterization set W is a set of input
sequence.
Harmonized state identification sets (HSI sets),
subsets of W.
{D0, D1, …, Dn-1}
Di is a set of prefixes of sequences in W
n is the number of states of S.
For any two distinguishable states Si and Sj of S, there
exists a sequence σ that is a prefix of both σi ∈ Di and
σj ∈ Dj such that σ can be accepted by these two states
and produces different outputs.
How is Optimal ?
1. The number of sequences in W is minimal.
2. The sum of their lengths is minimal.
To obtain a minimal characterization set for a
give FSM maybe an NP-Hard problem.
A heuristic solution
A1: A set P0 of all the state pairs that are distinguishable for the given FSM is produced.
A2: With P0, a characterization set W is obtained by forming a search tree from the input alphabet to distinguish the state pairs in P0.
A3: From this W, some HSI set Di is selected for each state i, such that the number of distinguishable state pairs and the length of a sequence are traded off.
The result of INRES responder
The result P0 of the algorithm A1 is the set of all
possible state pairs.
The characterization set W from the algorithm
A2 is {41}
The HSI sets {{41}, {41}, {4}, {4}}
The algorithm A1
The algorithm A1 is obtained by adapting the
FSM minimization algorithm give in
[ G. J. Holzmann. Design and Validation of
Computer Protocols, 1991 ]
The algorithm A2
Forms the root as the current node to probe
For each input word, a son is built if it can lead
to a better solution than other build nodes.
If it is estimated that a subtree with a minimal
number of branches and with a minimal average
length that distinguishes a maximal number of
state pairs could be formed by probing an
unprobed son of ti.
The search tree in A2
The algorithm A3
A sequence σk in W is selected such that weighted sum of its length and the number of the state pairs that it can not distinguish in the left state pairs is minimal.
For each remaining state pair (l, m) that can be distinguished by σk, find a prefix of σk such that (l, m) is distinguished.
Then the prefix is put into the HSI sets Dl and Dm.
W = {σ0, σ1,… σn-1}
Remain state pairs
(l, m)
σk :
σ :
HSI Sets: {D0, D1, D2, .. Dl, …, Dm, …Dn-1}
distinguish ?
1. Introduction
2. Using the tool 1. An Example
2. Functions provided by the Tool
3. The FSM specification
3. Test derivation methods 1. Preambles
2. Postambles
3. Transition Cover
4. State Identification Sequences
4. Experiments and Applications
5. Conclusion
4. Experiments and Applications
State num. 10 50 100 150 200
Transition num. 100 2500 10000 23500 40000
CPU time (sec.) 0.11 2.96 40.87 197.06 760.89
1. Introduction
2. Using the tool 1. An Example
2. Functions provided by the Tool
3. The FSM specification
3. Test derivation methods 1. Preambles
2. Postambles
3. Transition Cover
4. State Identification Sequences
4. Experiments and Applications
5. Conclusion
5. Conclusion
Proposed a heuristic solution to derive near-
optimal harmonized state identification sets.