1

Software testing

2

Testing ObjectivesTesting Objectives

Testing is a process of executing a program with the intent of finding an error.

A good test case is in that has a high probability of finding an as-yet-undiscovered error

A successful test is one that uncover an as-yet-undiscovered error.

3

If testing is conducted successfully. It will uncover errors in the software.

Testing demonstrates that software function appear to be working according to specification that behavioral and performance requirements appear to have been met.

Data collected as testing is conducted provide a good indication of software reliability and some indication of software quality as a whole

4

The testing process Component testing

• Testing of individual program components;

• Usually the responsibility of the component developer (except sometimes for critical systems);

• Tests are derived from the developer’s experience. System testing

• Testing of groups of components integrated to create a system or sub-system;

• The responsibility of an independent testing team;

• Tests are based on a system specification.

5

Testing phases

Componenttesting

Systemtesting

Software developer Independent testing team

6

Defect testing

The goal of defect testing is to discover defects in programs

A successful defect test is a test which causes a program to behave in an anomalous way

Tests show the presence not the absence of defects

7

Testing process goals

Validation testing• To demonstrate to the developer and the system

customer that the software meets its requirements;• A successful test shows that the system operates as

intended. Defect testing

• To discover faults or defects in the software where its behaviour is incorrect or not in conformance with its specification;

• A successful test is a test that makes the system perform incorrectly and so exposes a defect in the system.

8

The software testing process

Design testcases

Prepare testdata

Run programwith test data

Compare resultsto test cases

Testcases

Testdata

Testresults

Testreports

9

Only exhaustive testing can show a program is free from defects. However, exhaustive testing is impossible,

Testing policies define the approach to be used in selecting system tests:• All functions accessed through menus should be tested;• Combinations of functions accessed through the same

menu should be tested;• Where user input is required, all functions must be tested

with correct and incorrect input.

Testing policies

10

System testing

Involves integrating components to create a system or sub-system.

May involve testing an increment to be delivered to the customer.

Two phases:• Integration testing - the test team have access

to the system source code. The system is tested as components are integrated.

• Release testing - the test team test the complete system to be delivered as a black-box.

11

Testing guidelines

Testing guidelines are hints for the testing team to help them choose tests that will reveal defects in the system• Choose inputs that force the system to generate

all error messages;• Design inputs that cause buffers to overflow;• Repeat the same input or input series several

times;• Force invalid outputs to be generated;• Force computation results to be too large or too

small.

12

Integration testing

Involves building a system from its components and testing it for problems that arise from component interactions.

Top-down integration• Develop the skeleton of the system and populate it

with components. Bottom-up integration

• Integrate infrastructure components then add functional components.

To simplify error localisation, systems should be incrementally integrated.

13

Incremental integration testing

T3

T2

T1

T4

T5

A

B

C

D

T2

T1

T3

T4

A

B

C

T1

T2

T3

A

B

Test sequence 1 Test sequence 2 Test sequence 3

14

Performance testing

Part of release testing may involve testing the emergent properties of a system, such as performance and reliability.

Performance tests usually involve planning a series of tests where the load is steadily increased until the system performance becomes unacceptable.

15

Stress testing

Exercises the system beyond its maximum design load. Stressing the system often causes defects to come to light.

Stressing the system test failure behaviour.. Systems should not fail catastrophically. Stress testing checks for unacceptable loss of service or data.

Stress testing is particularly relevant to distributed systems that can exhibit severe degradation as a network becomes overloaded.

16

Component testing

Component or unit testing is the process of testing individual components in isolation.

It is a defect testing process. Components may be:

• Individual functions or methods within an object;• Object classes with several attributes and

methods;• Composite components with defined interfaces

used to access their functionality.

17

Object class testing

Complete test coverage of a class involves• Testing all operations associated with an object;• Setting and interrogating all object attributes;• Exercising the object in all possible states.

Inheritance makes it more difficult to design object class tests as the information to be tested is not localised.

18

Objectives are to detect faults due to interface errors or invalid assumptions about interfaces.

Particularly important for object-oriented development as objects are defined by their interfaces.

Interface testing

19

Interface testing

B

C

Testcases

A

20

Test case design

Involves designing the test cases (inputs and outputs) used to test the system.

The goal of test case design is to create a set of tests that are effective in validation and defect testing.

Design approaches:• Requirements-based testing;• Partition testing;• Structural testing.

21

Requirements based testing

A general principle of requirements engineering is that requirements should be testable.

Requirements-based testing is a validation testing technique where you consider each requirement and derive a set of tests for that requirement.

22

Sometime called white-box testing. Derivation of test cases according to

program structure. Knowledge of the program is used to identify additional test cases.

Objective is to exercise all program statements (not all path combinations).

Structural testing

23

White box testing is a test case design approach that employs the control architecture of the procedural design to produce test cases.

Using white box testing approaches, the software engineering can produce test cases that

(1) guarantee that all independent paths in a module have been exercised at least once,

(2) exercise all logical decisions,

(3) execute all loops at their boundaries and in their operational bounds,

(4) exercise internal data structures to maintain their validity.

White Box TestingWhite Box Testing

24

Structural testing

Componentcode

Testoutputs

Test data

DerivesTests

25

Path testing

The objective of path testing is to ensure that the set of test cases is such that each path through the program is executed at least once.

The starting point for path testing is a program flow graph that shows nodes representing program decisions and arcs representing the flow of control.

Statements with conditions are therefore nodes in the flow graph.

26

Although basis path testing is simple and highly effective, it is not enough in itself. So we consider variations on control structure testing that broaden testing coverage and improve the quality of white box testing.

1. Condition Testing

2. Data Flow Testing

3. Loop Testing

Control Structure TestingControl Structure Testing

27

Condition testing is a test case design approach that exercises the logical conditions contained in a program module. A simple condition is a Boolean variable or a relational expression.

The data flow testing method chooses test paths of a program based on the locations of definitions and uses of variables in the program. Various data flow testing approaches have been examined. For data flow testing each statement in program is allocated a unique s6atement number and that each function does not alter its parameters or global variables.

Control Structure TestingControl Structure Testing

28

Loops are the basis of most algorithms implemented using software. However, often we do consider them when conducting testing. Loop testing is a white box testing approach that concentrates on the validity of loop constructs.

Four loops can be defined:

Simple loops

Concatenate loops

Nested loops and

Unstructured loops.

Control Structure TestingControl Structure Testing

29

Black box testing approaches concentrate on the fundamental requirements of the software.

Black box testing allows the software engineer to produce groups of input situations that will fully exercise all functional requirements for a program.

Black box testing is not an alternative to white box techniques. It is a complementary approach that is likely to uncover a different type of errors that the white box approaches.

Black Box TestingBlack Box Testing

30

Black-box testing

IeInput test data

OeOutput test results

System

Inputs causinganomalousbehaviour

Outputs which revealthe presence ofdefects

31

Black box testing tries to find errors in the following categories:

(1)incorrect or missing functions(2)interface errors(3)errors in data structures or external database

access(4)performance errors, and (5)initialization and termination errors.By applying black box approaches we produce a set

of test cases that fulfill requirements: (1)test cases that reduce the number of test cases to

achieve reasonable testing(2)test cases that tell use something about the

presence or absence of classes of errors.

Black Box TestingBlack Box Testing

32

Graphical user interfaces present interesting challenges for software engineers. Because of reusable components provided as part of GUI development environments, the creation of the user interface has become less time consuming and more precise. But, at the same time, the complexity of GUIs has grown, leading to more difficulty in the design and execution of test case.

Due to large number of permutations associated with GUI operations, testing should be approached using automated tools.

Testing GUI’sTesting GUI’s

33

Tests are designed to answer the following questions:

•How is functional validity tested?•How is system behavior and performance tested?•What classes of input will make good test cases?•Is the system particularly sensitive to certain input values?•How are the boundaries of a data class isolated?•What data rates and data volume can the system tolerate?•What effect will specific combinations of data have on system operations?

34

Overall Approach Unit testing (module) Integration testing

Validation testing

System testing

White / Black box

Black box

Black box

requirements -verification

user desires - validation Black box - performance