programming language support for automated testing
DESCRIPTION
Programming Language Support for Automated Testing. Roy Patrick Tan Virginia Tech. Software tools help us write bug-free programs. Compiler Syntax errors Type checking Formal Verification Need to write formal specifications Proofs usually need human intervention Unit testing - PowerPoint PPT PresentationTRANSCRIPT
Programming Language Support for Automated
TestingRoy Patrick Tan
Virginia Tech
Software tools help us write bug-free programs
• Compiler– Syntax errors– Type checking
• Formal Verification– Need to write formal specifications– Proofs usually need human intervention
• Unit testing– Need to write unit tests – However, running unit tests can be automatic
We can use formal specs to generate unit tests
• Design-by-contract style specs (i.e. “executable” specifications)
• Generate test-case candidate using sequences of method calls
• Use postconditions as test-oracle
• Use preconditions to filter invalid tests
Sample Spec: Stackconcept Stack( Item ) {
model method sequence(): concept Sequence( Item ); method push( element: Item ) ensures sequence().equals( \old(sequence().insertLast( element.mathObject() ));
method pop( element: Item ) requires sequence().notEmpty() ensures \old(sequence()).equals( sequence().insertLast( element.mathObject() ));
method length(): Int ensures { \old(sequence()).equals(sequence()); length.equals(sequence().length()); };}
Model object lifetime as walks through a graph
• Every walk from init to finalize is an object lifetime -- can use as a test-case
• Some walks may be infeasible
• Generating all possible walks may be impossible
• Reseach problem: what set of walks will be good test-cases?
Automated-testing: what PL features do we need?
• Design-by-contract formal specs– Preconditions, postconditions, invariants should be
executable
• Enough features to be realistic, but not too much to be overwhelming– Encourage language features that make it easier
to write specs (separate specs and implementation)
– Eliminate language features that make it difficult to write specs (aliasing, null values)
Sulu• Main influences: RESOLVE, JML• Main differences from RESOLVE:
– object-oriented notation– spec language geared toward DBC
• Separation between specification (concepts) and implementation (realizations)
• Every object has a default initial state (no null values)
• Swapping ‘:=:’ as the main data movement operator
Digression: What’s wrong with assignment?
• Consider this java snippet:int a, b;a = 1;b = a;b = b + 1;
• After these operations what is the value of a?
• What is the value of b?
Object assignment is different from scalar assignment
• Now consider this java snippetjava.util.Stack a, b;
a = new java.util.Stack();
b = new java.util.Stack();
b = a;
b.push(“Hello”);
• What does b contain after these statements?• What about a?
Aliasing is evil• Scalar assignment in Java creates a
copy of the value• Object assignment creates an alias to
the same object, not a copy.• Aliasing is a big source of bugs!• Simple example, squaring a matrix:
matrix_multiply(a,a);
• Aliasing breaks modularity of reasoning
Is there a different way?• Copying preserves modularity of
reasoning but can be inefficient• Aliasing is efficient, but breaks
modularity of reasoning• Swapping!
– a :=: b– The pre-value of a becomes the post-value
of b, and vice versa
Swapping as an alternative to assignment?
• Swapping is efficient– Internally, compiler can make every swap
constant time.
• Swapping does not break modularity of reasoning– No aliases
Sulu Example:Using the Stack componentclass StringStack extends concept Stack(String) realization LinkedList();
x.push("Hello");x.push("World");x.push("!");
var c: Console;
x.pop(z);c.println(z);x.pop(z);c.println(z);x.pop(z);c.println(z);
Output:
!
World
Hello
Stack implementationrealization LinkedList() implements Stack( Item ) {
/* A linked-list node is a pair containing the item to store and a pointer to another node*/
class Node extends concept Pair( Item, concept ChainPointer(Node) realization Builtin() ) realization Obvious();
var top: concept ChainPointer(Node) realization Builtin(); var count: Int;
method pop( element: Item ) { //”dereference” the top pointer var topNode: Node; top.swapEntry( topNode );
//get the value at the top of //the stack and put in element topNode.swapFirst( element );
//set the value of top to the //next-pointer topNode.swapSecond( top );
count := count.minus( 1 );}
Stack implementationrealization LinkedList() implements Stack( Item ) {
/* A linked-list node is a pair containing the item to store and a pointer to another node*/
class Node extends concept Pair( Item, concept ChainPointer(Node) realization Builtin() ) realization Obvious();
var top: concept ChainPointer(Node) realization Builtin(); var count: Int;
method pop( element: Item ) { //”dereference” the top pointer var topNode: Node; top.swapEntry( topNode );
//get the value at the top of //the stack and put in element topNode.swapFirst( element );
//set the value of top to the //next-pointer topNode.swapSecond( top );
count := count.minus( 1 );}
Pointers are just another component,No special language constructs for pointers
Stack implementationrealization LinkedList() implements Stack( Item ) {
/* A linked-list node is a pair containing the item to store and a pointer to another node*/
class Node extends concept Pair( Item, concept ChainPointer(Node) realization Builtin() ) realization Obvious();
var top: concept ChainPointer(Node) realization Builtin(); var count: Int;
method pop( element: Item ) { //”dereference” the top pointer var topNode: Node; top.swapEntry( topNode );
//get the value at the top of //the stack and put in element topNode.swapFirst( element );
//set the value of top to the //next-pointer topNode.swapSecond( top );
count := count.minus( 1 );}
Swapping is the main data-movement operator
Stack implementationrealization LinkedList() implements Stack( Item ) {
/* A linked-list node is a pair containing the item to store and a pointer to another node*/
class Node extends concept Pair( Item, concept ChainPointer(Node) realization Builtin() ) realization Obvious();
var top: concept ChainPointer(Node) realization Builtin(); var count: Int;
method pop( element: Item ) { //”dereference” the top pointer var topNode: Node; top.swapEntry( topNode );
//get the value at the top of //the stack and put in element topNode.swapFirst( element );
//set the value of top to the //next-pointer topNode.swapSecond( top );
count := count.minus( 1 );}
Assignment is only allowed with methodreturn values.
What’s next?
• Fully implement specification language
• Implement test-case generation algorithms (all-pairs, all-triples, others?)
• Run experiments
• Graduate