DKT311 Software EngineeringDr. Rozmie Razif bin Othman

Software Testing

Why Testing a Software?Software Failure

Loss of Life Loss of Money Loss of Business Reputation

Error, Defect and Failure … What’s the difference?

Error •In programmer/developer mind•Can lead to Defect

Defect•Inside the code•Produce when programmer/developer has error in mind•Can cause Failure

Failure •When running code with defect, failure will be produced•Not all defect will cause failure.

Objectives of Testing

Finding defects

Gaining confidence about the level of quality

Providing information for decision-making

Preventing defects

7 Testing Principles

1. Testing shows presence of defects

Testing can show that defects are present, but cannot prove that there are no defects. Testing reduces the probability of undiscovered defects remaining in the software but, even if no defects are found, it is not a proof of correctness.

2. Exhaustive testing is impossible

Testing everything (all combinations of inputs and preconditions) is not feasible except for trivial cases. Instead of exhaustive testing, risk analysis and priorities should be used to focus testing efforts.

3. Early testing

To find defects early, testing activities shall be started as early as possible in the software or system development life cycle, and shall be focused on defined objectives

4. Defect clustering

Testing effort shall be focused proportionally to the expected and later observed defect density of modules. A small number of modules usually contains most of the defects discovered during release testing, or is responsible for most of the operational failures.

7 Testing Principles

5. Pesticide paradox

If the same tests are repeated over and over again, eventually the same set of test cases will no longer find any new defects. To overcome this “pesticide paradox”, test cases need to be regularly reviewed and revised, and new and different tests need to be written to exercise different parts of the software or systems to find potentially more defects.

6. Testing is context dependent

Testing is done differently in different contexts. For example, safety-critical software is tested differently from an e-commerce site.

7. Absence-of-errors fallacy

Finding and fixing defects does not help if the system built is unusable and does not fulfil the users needs and expectations.

Fundamental Test Process

Test Planning & Control

Test Analysis & Design

Test Implementation & Execution

Evaluating Exit Criteria & Reporting

Test Closure Activities

V-Model (Sequential Development Model)

VBusiness Requirement User Acceptance Testing

System Requirement System Testing

High Level Definition Integration Testing

Low Level Definition Unit Testing

Code

Component Testing (Unit Testing)

Searches for defects in, and verifies the functioning of, software modules, programs, objects, classes, etc.

Using stubs or drivers if necessary.

May include testing of functional or non-functional requirements.

Tester have access to the code (or test carried out by developer/programmer) with the support of development environment.

One approach in component testing is to prepare and automate test cases before coding (example tool that can be used is JUnit).

Integration Testing

Testing interface between component. Interactions with different parts of a system, such as the operating system, the file system and hardware, and interfaces between the system.

May be more than one level of integration testing

Component integration testing – Interaction between component done after component testing.

System integration testing – Interaction between different system or between hardware and software and may be done after system testing. Cross-platform issues may be significant.

System Testing

Concern with the behaviour of a whole system/product.

Test environment should correspond to the final target or production environment as much as possible in order to minimize the risk of environment-specific failures not being found in testing.

System testing includes:-

Tests based on risks and/or on requirement specifications.

Business process

Use Cases

High level text descriptions

Models of system behaviour.

Acceptance TestingUser Acceptance Testing

•Verifies the fitness for use of the system.•Done by business users.

Operational (acceptance) Testing

•Done by the system administrator.

Contract and Regulation Acceptance Testing

•Done based on the contract previously signed.

Field Testing

•Alpha Testing•Testing done at the developing organization’s site but not by the developing team.

•Beta Testing•Performed by customer or potential customers at their own location.

Types of Testing

Testing

Functional Testing

Non-Functional

Testing

Change Related Testing

Re-Testing Regression Testing

Maintenance Testing

Testing Technique

Testing Technique

Static Testing

Static Analysis Reviews

Dynamic Testing

Specification-based or Black-box Technique

Structure-based or

White-box Technique

Experience-based

Technique

Specification-based or Black-box Technique

Equivalent Partitioning

Inputs to the software or system are divided into groups (partitions) that are expected to exhibit similar behaviour.

Need to consider both valid and invalid input data.

Tests can be designed to cover all partitions

Applicable for all level of testing

Boundary Value Analysis

Behaviour at the edge of partition is more likely to be incorrect than behaviour within the partition.

When designing test cases, a test for each boundary value is chosen.

Example

You are testing a credit-card only, unattended gasoline pump. Once credit card is validated, the customer selected the desired grade, and the pump is ready to pump, the customer may cancel the transaction and owe nothing; however, once the pumping starts, gasoline will be sold in hundredths (0.01) of a gallon. The pump continues to pump gasoline until the user stops or a maximum of 50.00 gallons has been dispersed.

a) Derive the test input using Equivalent Partitioning technique.

b) Derive the test input using Boundary Value Analysis.

Solution

PartitionCan be sold

PartitionCannot be sold 1

PartitionCannot be sold 2

0.00 0.01 50.00 50.01….. …..…..

Equivalent Partition Technique:-

Select one value from each partition1. Partition Cannot be sold 1 (e.g. - 2.50)2. Partition Can be sold (e.g. 10.00)3. Partition Cannot be sold 2 (e.g. 75.00)

Test input using Equivalent Partition Technique are – 2.50, 10.00 and 75.00 gallon

Solution1st Boundary

0.00 0.01 50.00 50.01….. …..…..

Boundary value analysis:-

Choose 2 values (right next to the left and right) for every boundary1. For the 1st boundary is 0.00 and 0.012. For the 2nd boundary is 50.00 and 50.01

Test input using Boundary value analysis are 0.00, 0.01, 50.00 and 50.01 gallon

2nd Boundary

Specification-based or Black-box Technique

Decision Table Testing

Used when the requirements contain logical conditions.

Decision tables are a precise yet compact way to model complicated logic. Decision tables, like if-then-else and switch-case statements, associate conditions with actions to perform.

But, unlike the control structures found in traditional programming languages, decision tables can associate many independent conditions with several actions in an elegant way.

State Transition Testing

Self-study

Use Case Testing

Self-study

Structure-based or White-box Testing

Statement Testing and Coverage

Is the assessment of the percentage of executable statements that have been exercised by a test suite.

Test cases are designed to increase the statement coverage (to 100% coverage)

Decision Testing and Coverage

Related to branch testing

Stronger that statement coverage. (i.e. 100% decision coverage guarantees 100% statement coverage BUT NOT vice versa)

Experience-based Testing

Tests are derived from the tester’s skill and intuition and their experience with similar applications or technologies.

Useful in identifying special tests which is not easily captured by formal technique.

Most common approach call error guessing – testers anticipate defects based on experience.

More systematic approach is to list all possible defects and to design tests to attack these defects. This technique called fault attack.

Exploratory testing:-

“A style of software testing that emphasizes the personal freedom and responsibility of the individual tester to continually optimize the quality of his/her work by treating test-related learning, test design, test execution, and test result interpretation as mutually supportive activities that run in parallel throughout the project.” (Cem Kaner, 1983)