Testing Object Oriented Programs

CSE 111

04/20/23 1

2

Traditional Stages in Testing

• Unit testing Test individual classes or small groups of classes during development

• Integration testing Test larger groups of classes and subsystems

• System testing Test the whole system. Use test scripts that describe a sequence of user-program interactions and expected resulting behavior

04/20/23

Traditional Testing (and Analysis) Methods

• Functional or black box testing: test with normal and oddball cases. – normal: expected functionality– oddball: invalid inputs, empty arrays, etc.

• Coverage: makes sure all the program (e.g. branches, variables and data structures) has been used on a test

• Code reading: read the code, mentally simulating what it is supposed to be doing

04/20/23 3

Specific Object-Oriented Techniques

• Self-testing classes• Test case classes• Stubs, drivers and mock objects• Class design features that facilitate testing– i.e. design for testability

• Subsystem testing strategy for 3-tier system• Inheritance and abstract test cases

04/20/23 4

5

Self-Testing Classes

• Each class has a main method that is used for testing

• Except for the class with the “real” main method that is used to start the program

• Testing a class A with its main() method:• Create an instance of the class A• Call its methods, checking the results• Report results of tests

04/20/23

6

Test Case Class

• Special class T for testing another class or classes (the CUT = class under test)

• T contains a collection of special test and verify methods

• main() method in test case class T– Creates an instance of the CUT, performs

initialization, runs its test methods

04/20/23

7

Tests and Test Suites

• Test = TestCase or TestSuite• TestSuite = Collection of Tests

TestCase

TestSuite

TestCase TestSuite

TestSuite

TestCase TestCase

TestCase

04/20/23

8

TestRunners

• Run the tests for a set of classes• Needs to be able to know which methods in

a class are the test methods it needs to run• If test methods use special assert statements

to check results, then results are automatically reported

04/20/23

J-Unit

• Originally was far more complex and flexible, but was difficult to use

• JUnit 4– Special annotation is used to indicate which

method is a test method• may be in the CUT or in separate test case classes

– Supports specialized kinds of assert statements and other advanced, simplifying features

04/20/23 9

Sample JUnit Test Case

import static org.junit.Assert.assertEquals; public class AdditionTest { private int x = 1;

private int y = 1;

@Test public void addition() { int z = x + y;

assertEquals(2, z); }

}

04/20/23 10

Stubs and Drivers

• Suppose we want to test a class C which supports some responsibility, but is called by other classes, and calls classes itself

• How to isolate C during testing?– Create a “driver” test case class whose test methods call

methods in C. The test methods contain assert statements that check the results of the tests

– Create dummy “stub” objects that emulate the classes called by C. Do not need to be a full implementations, just have enough logic to handle the calls resulting from the tests

04/20/23 11

12

Class Design - Testability

• Designing classes to assist in testing– Mock objects and parameterized class

constructors

04/20/23

13

Mock Objects and Parameterized Constructors 1

• O/O equivalent of a stub is called a “mock object”– Suppose the Class Under Test (CUT) uses instances of

another, “auxiliary” class– Suppose that the auxiliary class is not available, or want to

test CUT in isolation– Need to a create an instance of a mock object class M

which stands in for or simulates instances of the actual class

04/20/23

14

Mock Objects and Parameterized Constructors 2

• Suppose a constructor for class A takes an instance of class B– Define the constructor parameter using an

interface C, which B implements • (“interface” in the Java sense)

– This means that instances of alternative classes can be passed, as long as the classes implement interface C. • e.g. a matching mock object class M

04/20/23

15

Mock Objects and Parameterized Constructors 3

• Suppose that objects of class A send messages to an object of class B.

• In the design, could use attribute visibility, in which, for example, the constructor for A creates an instance of B and assigns it to a class variable in A

• Facilitate testing with a design that uses parameter visibility instead where the constructor for A has a parameter for passing an instance of B which is created “outside” of A

• Now follow the approach of the previous slide, using an interface to declare the parameter type so that either instances of B or mock objects can be passed

04/20/23

DS Example• The Domain Logic subsystem has to send

messages to the DataBase subsystem (i.e. to the DB Java interface class)

• Instead of creating the DataBase subsystem instance “inside of” DomainLogic, “pre-create” the DataBase (interface) instance and pass it as an interface-typed parameter in the DomainLogic constructor

• Permits easy testing of DomainLogic with a stub/mock object for the DataBase, and seamless transference to use of a real DataBase instance

04/20/23 16

17

JUnit and Mock Objects

• Suppose class A uses an object of class B, and this is passed in the constructor for A, with a parameter of type C

• In a test case class which tests A– inside its test method(s)

• create a mock object of type C• create an instance of the CUT (in this case B) under test, and

pass the mock object in the constructor for B

– requires that the C be an interface type

04/20/23

Test Strategies for Our Three-Tier System

• Integration testing– Separate testing of GUI, DomainLogic and DataBase

subsystems – Subsystems small, so may simply do integration

testing for each subsystem, with no unit testing• DataBase

– test DB directly, test DB through the DL• DomainLogic

– test DL directly, using mock DB, use test cases simulate the GUI• GUI

– manual testing with DL mock object– possible separate testing of GUI classes using “information

objects”

04/20/23 18

19

DataBase Testing

• Why not simply test the DB in the context of the whole system?

• Typical test case would– Set up or check DB status– Perform application methods– Check effects by accessing DB

04/20/23

20

Problems with Whole System - Strategy for Testing the DataBase

• Want to test DB directly to observe effects of DB method calls

• Want directly examine the effects of “complex” sequences of DB method calls that would occur as the result of DL activity

• Need to directly test the correctness of the interfaces between DB methods and the DL methods that call them

04/20/23

21

Testing the DB Directly- Individual Method Applications

• Test the DB by writing test cases that execute methods in the DB interface

• Use black box test selection, backed up by coverage analysis, for each DB interface method

• DB interface may have only a small number of methods, which we can focus on individually

04/20/23

Testing the DB Directly- Method Sequences

• Problems may occur due to sequences of DB method applications

• BET (bounded exhaustive testing)– test exhaustively over all inputs with a “small

version” of the application– e.g. DS. Only has 2-3 members, sequences of user

interactions limited to length, say, 3

04/20/23 22

Sample DB Bug in DS Example

• Sequence of interactions used to reveal bug1. initialize() /*reads in files from DB, examples with 0,1,2

entries2. delete(x) where x is not a member of the DB

• Note: this defect is not discoverable if a deletion of a non-existent member is performed after a deletion of an existing member has been performed. So simple testing of a non-existing member may not work. BET picks up the strange short sequence that causes the failure

04/20/23 23

24

Testing theDomainLogic/DB Interfaces

• Purpose is to test the links from one to the other– ignore functionality for now– write test cases that call methods in the DL (Java)

interface, which result in calls from DomainLogic class methods to the DB (Java interface)

– focus on testing each of these DL/DB interfaces for consistency, as simply and directly as possible

• Note: you may find interface inconsistencies during focused test construction

04/20/23

25

Test Case Classes for Testing the DB Directly

• TestCase contents• use a class variable for DB class instance• setup() method would create a DB instance and assign

to the DB instance variable (this is an empty DB)• tearDown() method deletes everything from DB• testx() methods would create and retrieve entries from

data base, using DataBase (Java interface) methods and method sequences. Assertions in the test methods will reference data returned from DB accesses

04/20/23

Test Case Classes for Testing DomainLogic/DB Interfaces

• Test Case contents– use a class variable for DB class instance– setup() method would create an instance dB of the DB

(Java interface) and possibly use dB methods to populate the data base

– testx() methods • would create an instance of DomainLogic, passing dB as a

constructor method parameter• execute DomainLogic methods• use DB methods directly to check results

– teardown() removes effects of tests

04/20/23 26

Domain Logic Testing

• Test the methods in the DL interface using black box and coverage testing measures– include coverage testing of called methods in the

DL – focus on DL (Java interface) methods

• Simulate the GUI– test methods would have (sequences of) calls on

DL that simulate a GUI action

04/20/23 27

28

Test Case Classes for Domain Logic Testing

1) setUp() method in the test case could: 1) create an instance of the mock DB, 2) create an instance of the DL, passing the mock DB as a

parameter.

2) testx() method could1) execute DL methods, checking results with an assert()

04/20/23

29

GUI Testing

• How do we test the GUI?– We will use manual GUI testing only– Test cases will create an instance of a mock DL

and pass it as a parameter in the constructor for the GUI

– Need to construct scripts that document tester input and expected results for each test case• Checking results: tester inspection via GUI

04/20/23

Separate Testing of GUI Classes

• Suppose that we want to test a GUI class more directly, by itself, but it calls on/creates other GUI classes?– If other GUI classes simply present results, keep

them because they are the test output– Suppose that the other GUI classes accept input

and return information • How to delink these subordinate GUI objects?

04/20/23 30

31

Delinking GUI Classes

• Suppose that a window A creates a dialog B for gathering some information

• Instead • create a Java interface class Info with a method getInfo()

– getInfo() returns an information object

• for test use, define an implementation of Info where getInfo() sets and returns an information object directly

• for production use, define an implementation of Info where getInfo() creates an instance of B, reads the information, and then returns it in an information object

04/20/23

Setting the Info object

• Two possibilities– The GUI object A has a constructor with a parameter

for passing in the Info object– requires an external class, to construct the subclass

implementation instances of Info for test or production runs

– The GUI class A can be abstract with an abstract factory method create()– Subclass A with a definition of create() that uses the correct

implementation constructor, depending on whether A is being used for testing or production use

04/20/23 32

Test Strategies for Our Three-Tier System

• System Testing– Combine DomainLogic and with a real DB

instances, simulate actions of GUI• can also be classified as a higher level of integration

testing

– Test whole system through the GUI– apply black box, BET• coverage and code reading are not usually done at the

system level

04/20/23 33

DS System Testing Example

• When doing BET sequence testing included the operations of terminating and starting a new DS session

• In the DS if you logon as a member, and then ask to edit your data, and do not enter any new properties, and then terminate a session, you will get a crash when you try to restart the system with a new session

04/20/23 34

35

New Topic: Abstract Test Cases

• Suppose that a class C’ subclasses a class C– Can we develop a test case T for C that we can re-

use for C’?– Consider the case where C is an interface class– In general, only some tests be reused, and there is

obvious way to determine which

04/20/23

36

Creating Abstract Test Cases

• To create an abstract test, construct an abstract test class T so that it uses an abstract factory method createC() to create instances of C

• If C’ is a subclass of C, which implements the interface, then create a subclass T’ of T in which the implementation of the abstract factory method createC() uses the constructor for C’ to create an concrete instance of C (i.e. instance of C’)

04/20/23

37

Polymorphism and TestCases

• Polymorphic design pattern:– A set of classes corresponds to related alternative

forms of behavior– Can capture alternatives in a set of subclasses of a

class– Subclasses have a common set of methods, but

exhibit different behavior depending on the subclass in which they are defined

04/20/23

38

Creating Abstract Polymorphic Test Cases

• Use approach similar to above, for inheritance • Usefulness? Depends on how much logic in

the abstract test case methods is reusable for the subclasses

04/20/23

39

Additional Test Frameworks: HttpUnit

• HttpUnit emulates the relevant portions of browser behavior, including form submission, JavaScript, basic http authentication, cookies and automatic page redirection, and allows Java test code to examine returned pages either as text, an XML DOM, or containers of forms, tables, and links. When combined with a framework such as JUnit, it is fairly easy to write tests that very quickly verify the functioning of a web site.

04/20/23

Assignment 8

• Use the principles in the lecture to design tests for your project– Use J-Unit to construct and document 2 test cases

for 2 classes. Use normal and oddball tests.– Construct system level tests as per earlier system

testing slide (33)• for user inputs use normal and oddball cases• use BET to test short sequences of operations

04/20/23 40