lecture 2 (chapter 2)

8/3/2019 Lecture 2 (Chapter 2)

1/36

CHAPTER 2

BLACK BOX TESTING

By Esubalew A.Cseg5325


2/36

Contents

Testing Techniques

Types of Testing Techniques

What is black-box testing?

Black-box testing: process, applicability, prose & Cons

Types of black-box testing

Random Testing

Equivalence partitioning ECP: Example, how to select classes, Applicability and

Limitations

Boundary value analysis

BVA: Examples1,2 & 3, Applicability and Limitations


3/36

Testing Techniques

Testing, however creative and seemingly complete, cannot

guarantee the absence of all errors.

Test-case design is so important because complete testing

is impossible; the obvious strategy, then, is to try to maketests as complete as possible.

Given constraints on time and cost, the key issue of testing

becomes

What subset of all possible test cases has the

highest probability of detecting the most errors?

The study of test-case-design methodologies supplies

answers to this question.


4/36

Testing Techniques

The main objective of test case design techniques is to

design an effective test cases.

If test cases are effective there is

a greater probability of detecting/revealing defects a more efficient use of organizational resources

a higher probability for test reuse

closer adherence to testing and project schedules and

budgets, and

the possibility for delivery of a higher-quality software

product.

Techniques are the synthesis of best practicenot


5/36

Testing Techniques

Test case design techniques are the heart of testing.

There are many advantages of using techniques to

design test cases.

Support systematic and meticulous work and make the

testing specification effective and efficient;

Extremely good for finding possible faults.

the design of the test cases may be repeated by othersthat it is possible to explain how test cases have been

designed using techniques.


6/36

Testing Techniques

Test case design techniques have a few pitfalls that we

need to be aware of.

Even if we could obtain 100% coverage of what we set

out to cover (be it requirements, or statements, or paths),faults could remain after testing simply because the code

does not properly reflect what the users and customers

want.

Validation of the requirements before we start thedynamic testing can mitigate this risk.

value sensitivity. Even if we use an input value that gives

us the coverage we want it may be a value for which

incidental correctness applies.


7/36

Types Testing Techniques

What are the approaches a tester should use to designeffective test cases?

There are two basic strategies that can be used to designtest cases.

Black box /functional/behavioural / specification-driven/data-driven / inputoutput driven test strategy

treats the system as a "black-box", so it doesn't explicitlyuse knowledge of the internal structure, to design test

cases

White box /clear box/ glass box /structural test strategy

allows one to peek inside the "box", and it focusesspecifically on using internal knowledge of the software to

guide the selection of test data.


8/36

What is black-box testing?

These techniques focus on the functionality.

Involves deriving tests from external descriptions of the

software, including specifications, requirements, and

design.Unlike its complement, white box testing, black box testing

requires no knowledge of the internal paths, structure, or

implementation of the software under test (SUT).

Aims at finding cases in which the software does not

behave according to its specifications or requirements

The general black box testing process is:

The requirements or specifications are analyzed.


9/36

Black-box testing: Process

Valid inputs are chosen based on the specification to

determine that the software under test (SUT) processes them

correctly.

Invalid inputs must also be chosen to verify that the SUTdetects them and handles them properly.

Expected outputs for those inputs are determined.

Tests are constructed with the selected inputs.

The tests are run.

Actual outputs are compared with the expected outputs.

A determination is made as to the proper functioning of the

SUT.


10/36

Black-box testing: Applicability

These test case design techniques can be used in all stages andlevels of testingunit, integration, system, and acceptance.

Though the size increases from module to subsystem and then

to system the box gets larger, with more complex inputs andmore complex outputs, the approach remains the same.

Also, as we move up in size, we are forced to the black boxapproach;

there are simply too many paths through the SUT to performwhite box testing.


11/36

Black-box testing: Pros & Cons

Advantages Tester can be non-technical.

Used to verify contradictions in actual system and thespecifications.

Test cases can be designed as soon as the functionalspecifications are complete

directs the tester to choose subsets of tests that are both efficientand effective in finding defects - helps maximize the return onour testing investment

Disadvantages The tester can never be sure of how much of the system under

test has been tested. i.e. chances of having unidentified pathsduring this testing

The test inputs needs to be from large sample space.

It is difficult to identify all possible inputs in limited testing time.


12/36

Types of Black box testing

The goal of smart tester is to effectively use the resources available by developing a set of

test cases that gives the maximum yield of defects for the timeand effortspent

To achieve this goal there are various functional test casedesign techniques. Random Testing

Equivalence partitioning


Decision tables

Cause-effect graph

State transition testing

Very often combinations of the methods are used to detect

different types of defects.

Reading Assignments

Error Guessing

Classification tree method

Pair-wise testing

Use case testing

Domain analysis


13/36

Random Testing

Each software module or system has an input domainfrom which test input data is selected.

If a tester randomly selects inputs from the domain, this is

called random testing. E.g. If input domain for a module is 1-100 a tester may

choose 24, 53 & 7 as input values. But

Are the three values adequate to show that the module

meets its specification when the tests are run?Are there any input values, other than those selected,

more likely to reveal defects?

Should any values outside the valid domain be used as

test inputs?


14/36

Random Testing

Use of random test inputs may save some of the time and

effort that more thoughtful test input selection methods

require.

However, selecting test inputs randomly has very littlechance of producing an effective set of test data.

There are tools that generate random test data for stress

tests.

This type of testing can be very useful especially at the

system level.

Usually the tester specifies a range for the random value

generator, or the test inputs are generated according to a


15/36

Equivalence class partitioning

Involves partitioning the input or output domain intoequivalence classes. Though not always this is possible most

of the times.

Partitioning is splitting the domain into classes such that all

members of the domain belong to exactly one class i.e. no

member belongs to more than one class and no member falls

outside the classes.

Class is a portion of the domain.

Equivalence refers to the assumption that all the members in a

class behave in the same way.

The reason for the equivalence partitioning is that all

members in an equivalence class will either fail or pass the

same test.


16/36

Equivalence class partitioning

When SUT domain is partitioned into equivalence classes,

usually both valid and invalid classes are obtained.

Test cases should be designed for both the valid and the invalid

classes

but sometimes it is not possible to execute test cases based on

the invalid equivalence classes.

Partitioning of the input domain for the SUT using this

technique has the following advantages: It eliminates the need for exhaustive testing, which is not feasible.

It guides a tester in selecting a subset of test inputs with a high

probability of detecting a defect.

It allows a tester to cover a larger domain of inputs/outputs with a


17/36

Equivalence class partitioning:

Example

Look at the following taxation table

Based on the above table 3 valid and 3 invalid equivalent

classes can be found

Valid Equivalent Classes

Values between 0 to 500, 500 to 1000 and 1000 to 5000

Invalid Equivalent Classes

Values less than 0, greater than 5000 and inputs containing

letters

Income TaxPercentage

Up to and including 500 0

More than 500, but less than1,300

30

1,300 or more, but less than

5,000

40


18/36

ECP: How to select classes

The following conditions can be used as guidelines forselecting input equivalence classes

If an input condition for the SUT is specified as

a range of values, select one valid equivalence class that covers

the allowedrange and two invalid equivalence classes, one

outside each end of the range.

a numberof values, then select one valid equivalence class that

includes the allowed number of values and two invalid

equivalence classes that are outside each end of the allowed

number.

a setof valid input values, then select one valid equivalence

class that contains all the members of the set and one invalid

equivalence class for any value outside the set


19/36


If the input specification or any other information leads to

the belief that an element in an equivalence class is not

handled in an identical way by the SUT, then the class

should be further partitioned into smaller equivalence

classes

Consider the specification for square root function


20/36


The inputconditions are that the variable x must be a realnumber and be equal to or greater than 0.0.

The conditions for the output variabley are that it must be areal number equalto or greater than 0.0, whose square is

approximately equal tox. Ifx is not equal to or greater than 0.0, then an exception is

raised.

From this information the tester can easily generate both

invalid and valid equivalence classes and boundaries.

For example, input equivalence classes for the specificationin previous slide is

EC1. The input variablex is real, valid

EC2. The input variablex is not real, invalid


21/36


EC3. The value ofx is greater than 0.0, valid.

EC4. The value ofx is less than 0.0, invalid.

After the equivalence classes have been identified in thisway, the next step in test case design is the development

of the actual test cases

A good approach includes the following steps.

Assigned a unique identifier for each equivalent classes

Develop test cases for all valid & invalid equivalence classesuntil all have been covered by a test case

The equivalence partition coverage is measured as thepercentage of equivalence partitions that have beenexercised by a test


22/36

ECP: Applicability and Limitations

can significantly reduce the number of test cases that must becreated and executed.

most suited to systems in which much of the input data takes

on values within ranges or within sets.

makes the assumption that data in the same equivalence class

is, in fact, processed in the same way by the system.

The simplest way to validate this assumption is to ask the

programmer about their implementation. Equivalence class testing is equally applicable at the unit,

integration, system, and acceptance test levels.

All it requires are inputs or outputs that can be partitioned

based on the system's requirements.


23/36


A boundary value is the value on a boundary of an

equivalence class.

Boundary value analysis is hence strongly related to equivalence

class partitioning.

Equivalence classes ofintervals have boundaries, but those of

lists do not.

Boundary value analysis is the process of identifying the

boundary values. The boundary values require extra attention because defects

are often found on or immediately around these.

Choosing test cases based on boundary value analysis insures

that the test cases are effective.


24/36


The rules-of-thumb described below are useful for gettingstarted with boundary value analysis.

If an input condition for the SUT is specified as a range ofvalues, develop valid test cases for the ends of the range,

and invalid test cases for possibilities just above and belowthe ends of the range.

For interval classes with precise boundaries, it is not difficultto identify the boundary values.

If a class has an imprecise boundary (> or


25/36


If the input or output of the software-under-test is an

ordered set, such as a table or a linear list, develop tests that

focus on the first and last elements of the set.

The boundary value coverage is measured as thepercentage of boundary values that have been exercised

by a test.

Note that

Equivalence class partitioning and boundary value analysis

apply to testing both inputs and outputs of the software-

under-test

Conditions are not combined for equivalence classartitionin or boundar value anal sis.


26/36

Example 1 on ECP and BVA

Given the following user requirements find test conditions

and test cases for the testing of the requirement.

[UR 631] The system shall allow shipments for which the price is

less than or

equal to 100.

Lets do this using a template designed by Carsten

Jrgensen

The design of the test conditions based on equivalence classpartitioning and boundary value analysis can be captured in

a table such as the following one


27/36


The fields in the table are: Test design item number: Unique identifier of the test design item

Traces: References to the requirement(s) or other descriptionscovered by this test design

Based on: Input/Output: Indication of which type of domain thedesign is based on

Assumptions: Here any assumption must be documented.

Type: Must be one of

VCValid class

ICInvalid classDescription: The specification of the test condition

Tag: Unique identification of the test condition

BT = Belongs to: Indicates the class a boundary value belongs to.This can be used to cross-check the boundary values.

VBValid boundary value

IBInvalid boundary value


28/36


To design the test cases

The first thing well

do is fill in the header

of the design table. The next thing is identifying the valid class(es).


29/36


If we want 100% equivalence partition coverage and twovalue boundary value coverage for the requirement, assuming

that invalid values are rejected, we get the following test cases:

We can omit some of the test cases, especially since there are

actually 5 different test cases covering the same equivalence class.


30/36

Example 2 on BVA

Generate test cases for the following user requirement[UR 627] The system shall allow the packing type to be specified

as either Box or Wrapping paper.

The test design table looks like this after the analysis.

This type of equivalence class does not have boundaries.

The test cases could be:


31/36


Suppose we want to test a module that allows a user to enter

new student identifiers into a student data base.

The input specification for the module states that a widget

identifier should consist of 315 alphanumeric characters ofwhich the first two must be letters

Three separate conditions that apply to the input

1. It must consist of alphanumeric characters

2. The range for the total number of characters is between 3 and

15

3. The first two characters must be letters

First identify input equivalence classes and give them each

an identifier


32/36


Condition 1: alphanumeric

EC1. Part name is alphanumeric, valid.

EC2. Part name is not alphanumeric, invalid.

Condition 2: length b/n 3 and 15 EC3. The widget identifier has between 3 and 15 characters,

valid.

EC4. The widget identifier has less than 3 characters, invalid.

EC5. The widget identifier has greater than 15 characters, invalid.

Condition 3

EC6. The first 2 characters are letters, valid.

EC7. The first 2 characters are not letters, invalid.


33/36


Boundary value analysis is now used to refine the results ofequivalence class partitioning.

The following bounds groups are identified and represented

by abbreviations.

BLBa value just below the lower bound (e.g. 2 )

LBthe value on the lower boundary (e.g. 3 )

ALBa value just above the lower boundary (e.g. 4 )

BUBa value just below the upper bound (e.g. 14 ) UBthe value on the upper bound (e.g. 15 )

AUBa value just above the upper bound (e.g. 16 )


34/36


Then we will augment these with the results from boundary

value analysis.

If the tester does not believe in them he or she may omit

bounds groups except LB & UB, as they are included ineither valid or invalid equivalent classes

Finally, the test case design process is to select a set of

actual input values that covers all the equivalence classes

and the boundaries.The table in the next slide is used to organize the results.

The table only describes the tests for the module in terms

of inputs derived from equivalence classes and

boundaries.


35/36


BVA A li bilit d


36/36

BVA: Applicability and

Limitations

Boundary value testing can significantly reduce the

number of test cases that must be created and executed.

It is most suited to systems in which much of the input

data takes on values within ranges or within sets. Boundary value testing is equally applicable at the unit,

integration, system, and acceptance test levels.

All it requires are inputs that can be partitioned and

boundaries that can be identified based on the system's

requirements.

lecture 2 (chapter 2)

Documents