![Page 1: Software Component Substitutability through Black-Box Testing Andrés Flores aflores@uncoma.edu.ar Alarcos Group Escuela Superior de Informática Universidad](https://reader030.vdocuments.net/reader030/viewer/2022032704/56649d795503460f94a5d0ec/html5/thumbnails/1.jpg)
Software Component Substitutabilitythrough Black-Box Testing
Andrés [email protected]
Alarcos GroupEscuela Superior de Informática
Universidad de Castilla-La Mancha Ciudad Real, España
Macario [email protected]
GIISCo GroupDepartment of Computer Science
University of ComahueNeuquén, Argentina
![Page 2: Software Component Substitutability through Black-Box Testing Andrés Flores aflores@uncoma.edu.ar Alarcos Group Escuela Superior de Informática Universidad](https://reader030.vdocuments.net/reader030/viewer/2022032704/56649d795503460f94a5d0ec/html5/thumbnails/2.jpg)
Andres Flores - Macario Polo --- STV'07 - December, 5th 2007 2
Motivation
Upgrades of Components Updates of Component Systems
Replacement: upgrades or other OTS
Maintain stability of current systems
Evaluate replacement components
Easy the identification of compatibility
![Page 3: Software Component Substitutability through Black-Box Testing Andrés Flores aflores@uncoma.edu.ar Alarcos Group Escuela Superior de Informática Universidad](https://reader030.vdocuments.net/reader030/viewer/2022032704/56649d795503460f94a5d0ec/html5/thumbnails/3.jpg)
Andres Flores - Macario Polo --- STV'07 - December, 5th 2007 3
Approach
Complement regular compatibility analysis by means of Component Testing Criteria
based on the Observability testing metric
Given a component C and a candidate upgrade K, they should match in domain and range, and also on each mapping pair (input,output)
![Page 4: Software Component Substitutability through Black-Box Testing Andrés Flores aflores@uncoma.edu.ar Alarcos Group Escuela Superior de Informática Universidad](https://reader030.vdocuments.net/reader030/viewer/2022032704/56649d795503460f94a5d0ec/html5/thumbnails/4.jpg)
Andres Flores - Macario Polo --- STV'07 - December, 5th 2007 4
Approach – Three Phases
![Page 5: Software Component Substitutability through Black-Box Testing Andrés Flores aflores@uncoma.edu.ar Alarcos Group Escuela Superior de Informática Universidad](https://reader030.vdocuments.net/reader030/viewer/2022032704/56649d795503460f94a5d0ec/html5/thumbnails/5.jpg)
Andres Flores - Macario Polo --- STV'07 - December, 5th 2007 5
Sample: Java Calculator
C JCalculator K JCalc from http://sourceforge.net
![Page 6: Software Component Substitutability through Black-Box Testing Andrés Flores aflores@uncoma.edu.ar Alarcos Group Escuela Superior de Informática Universidad](https://reader030.vdocuments.net/reader030/viewer/2022032704/56649d795503460f94a5d0ec/html5/thumbnails/6.jpg)
Andres Flores - Macario Polo --- STV'07 - December, 5th 2007 6
Component Behaviour TS
TS goal: to check that a candidate component K
coincides on behaviour with an original component C.
each test case: a set of calls to C's services, with an
oracle to determine acceptance or refusal
Component Integration Testing Criteria:
all-Interfaces (all-Methods)
all-Events (including all-exceptions)
all–Context-dependence
intra-/inter-component dependence
1st1st
![Page 7: Software Component Substitutability through Black-Box Testing Andrés Flores aflores@uncoma.edu.ar Alarcos Group Escuela Superior de Informática Universidad](https://reader030.vdocuments.net/reader030/viewer/2022032704/56649d795503460f94a5d0ec/html5/thumbnails/7.jpg)
Andres Flores - Macario Polo --- STV'07 - December, 5th 2007 7
Component Behaviour TS (cont.)
Component Behaviour TS:
intra-component dependence
operational sequences RegEx
Alphabet: names of components services.
RegEx general pattern
protocol of use for a component interface
By reflection Java component interfaces are collected to
automate Test Case generation
1st1st
![Page 8: Software Component Substitutability through Black-Box Testing Andrés Flores aflores@uncoma.edu.ar Alarcos Group Escuela Superior de Informática Universidad](https://reader030.vdocuments.net/reader030/viewer/2022032704/56649d795503460f94a5d0ec/html5/thumbnails/8.jpg)
Andres Flores - Macario Polo --- STV'07 - December, 5th 2007 8
Sample: TS for JCalculator
We use the testoojtestooj tool some settings:
1st1st
Protocol of use:
Service sequences: length of expressions,
minimum 8, derives 20 Test templates:
4 expressions with 1 math service and
16 with an extra iteration for the '+' operator.
![Page 9: Software Component Substitutability through Black-Box Testing Andrés Flores aflores@uncoma.edu.ar Alarcos Group Escuela Superior de Informática Universidad](https://reader030.vdocuments.net/reader030/viewer/2022032704/56649d795503460f94a5d0ec/html5/thumbnails/9.jpg)
Andres Flores - Macario Polo --- STV'07 - December, 5th 2007 9
Sample: TS for JCalculator1st1st
ExceptionsExceptions
![Page 10: Software Component Substitutability through Black-Box Testing Andrés Flores aflores@uncoma.edu.ar Alarcos Group Escuela Superior de Informática Universidad](https://reader030.vdocuments.net/reader030/viewer/2022032704/56649d795503460f94a5d0ec/html5/thumbnails/10.jpg)
Andres Flores - Macario Polo --- STV'07 - December, 5th 2007 10
Sample: TS for JCalculator
test values are combined with 20 test templates (services sequences) and constraints files (pre/postcode).
testoojtestooj provides four algorithms: each choice, antirandom, pairwise, and all-combinations.
In this case we have decided on all-combinations each combination is a test case a testing method
into a test driver file. 468 test cases were generated into a class called
TestJCalculator1
1st1st
![Page 11: Software Component Substitutability through Black-Box Testing Andrés Flores aflores@uncoma.edu.ar Alarcos Group Escuela Superior de Informática Universidad](https://reader030.vdocuments.net/reader030/viewer/2022032704/56649d795503460f94a5d0ec/html5/thumbnails/11.jpg)
Andres Flores - Macario Polo --- STV'07 - December, 5th 2007 11
Sample: TS for JCalculator1st1st
Pre-code(setAdd)
Pre-code(evalExpression)
Post-code(evalExpression)
![Page 12: Software Component Substitutability through Black-Box Testing Andrés Flores aflores@uncoma.edu.ar Alarcos Group Escuela Superior de Informática Universidad](https://reader030.vdocuments.net/reader030/viewer/2022032704/56649d795503460f94a5d0ec/html5/thumbnails/12.jpg)
Andres Flores - Macario Polo --- STV'07 - December, 5th 2007 12
Interface Compatibility
To compare interfaces syntactically, three cases for services:
1)1) Exact matchExact match: two services must have identical signature, i.e. return type, service name, and for parameters and exceptions: amount, type and order.
2)2) Near-Exact matchNear-Exact match: similar to previous, butorder is relaxed for parameters and exceptions.
3)3) Soft matchSoft match: similar to previous, butservice name is ignored and for exceptionsit is relaxed to existence of any.
Outcome: a matching list with characterizedservices correspondences
2nd2nd
![Page 13: Software Component Substitutability through Black-Box Testing Andrés Flores aflores@uncoma.edu.ar Alarcos Group Escuela Superior de Informática Universidad](https://reader030.vdocuments.net/reader030/viewer/2022032704/56649d795503460f94a5d0ec/html5/thumbnails/13.jpg)
Andres Flores - Macario Polo --- STV'07 - December, 5th 2007 13
Sample: JCalculator-JCalc2nd2nd
![Page 14: Software Component Substitutability through Black-Box Testing Andrés Flores aflores@uncoma.edu.ar Alarcos Group Escuela Superior de Informática Universidad](https://reader030.vdocuments.net/reader030/viewer/2022032704/56649d795503460f94a5d0ec/html5/thumbnails/14.jpg)
Andres Flores - Macario Polo --- STV'07 - December, 5th 2007 14
Test-based Semantic Compatibility
Purpose: to find services from candidate K exposing similar behaviour with the original component C.
Implies to run the Component Behaviour TS for C (1st phase), against K
Syntactic matching list (2nd phase),is used to build a set of wrappers for component K.
Wrappers are adapters (adapter pattern) Interface mutation:
operators to change service invocationsand parameter values
To reduce amount of wrappers, starts with stronger matching cases (exact-match).
3rd3rd
![Page 15: Software Component Substitutability through Black-Box Testing Andrés Flores aflores@uncoma.edu.ar Alarcos Group Escuela Superior de Informática Universidad](https://reader030.vdocuments.net/reader030/viewer/2022032704/56649d795503460f94a5d0ec/html5/thumbnails/15.jpg)
Andres Flores - Macario Polo --- STV'07 - December, 5th 2007 15
Sample: run JCalculator TS on JCalc
Semantic compatibility: JCalculator–JCalc
Set of Wrappers: would be 286 in size.
we decided to initially analyzed specific cases of Interface Mutation to manually create 30 wrappers:
3rd3rd
1 wrapper with the true services correspondence (like those marked with '*' for putInBuffer service.
2 wrappers with variation on putInBuffer service (the rest of soft-matches).
3 wrappers with variation on the math services and based on the three previous wrappers.
Four groups of 6 wrappers varying the setExpression service and taking the three previous groups.
![Page 16: Software Component Substitutability through Black-Box Testing Andrés Flores aflores@uncoma.edu.ar Alarcos Group Escuela Superior de Informática Universidad](https://reader030.vdocuments.net/reader030/viewer/2022032704/56649d795503460f94a5d0ec/html5/thumbnails/16.jpg)
Andres Flores - Macario Polo --- STV'07 - December, 5th 2007 16
Conclusions Focused on Maintenance: Component systems are updated by
integrating upgraded components.
Testing criteria used to describe components behaviour to analyse compatibility
Similar to Regression testing: TS reduction strategies to expose faults on targeted pieces.Need to identify changes on versions (upgrades).
We assume only information collected by reflection.
Candidate components could be other OTS
Purpose: not finding faults, but analyse compatibility. TS= valid configurations.
![Page 17: Software Component Substitutability through Black-Box Testing Andrés Flores aflores@uncoma.edu.ar Alarcos Group Escuela Superior de Informática Universidad](https://reader030.vdocuments.net/reader030/viewer/2022032704/56649d795503460f94a5d0ec/html5/thumbnails/17.jpg)
Andres Flores - Macario Polo --- STV'07 - December, 5th 2007 17
Future Work
Important to get a reduced TS: depends on the combination
algorithm, and alternative strategies, like Mutation.
Other issue: subtyping (built-in) now was added, changing the
cases for Interface compatibility.
Wrappers: need to enhance the syntactic Interface Matching,
we are experimenting with other aspects to reduce the set size.
Wrappers analysis: being adjusted to set a proper threshold,
not always possible 100% successful on tests.
More experimentation to properly validate de whole approach.
![Page 18: Software Component Substitutability through Black-Box Testing Andrés Flores aflores@uncoma.edu.ar Alarcos Group Escuela Superior de Informática Universidad](https://reader030.vdocuments.net/reader030/viewer/2022032704/56649d795503460f94a5d0ec/html5/thumbnails/18.jpg)
Andrés [email protected]
Alarcos GroupEscuela Superior de Informática
Universidad de Castilla-La Mancha Ciudad Real, España
Macario [email protected]
GIISCo GroupDepartment of Computer Science
University of ComahueNeuquén, Argentina
Software Component Substitutabilitythrough Black-Box Testing