ISBN 0-13-146913-4Prentice-Hall, 2006

Program Testing + Integration Testing

Zeeshan Rana

Types of faults and how to clasify them The purpose of testing Unit testing Integration testing strategies Test planning When to stop testing

Chapter 8 Objectives

Wrong requirement: not what the customer wants

Missing requirement Requirement impossible to implement Faulty design Faulty code Improperly implemented design

8.1 Software Faults and FailuresWhy Does Software Fail?

Objective of testing: discover faults A test is successful only when a fault is

discovered◦ Fault identification is the process of determining

what fault caused the failure◦ Fault correction is the process of making changes

to the system so that the faults are removed

8.1 Software Faults and Failures Objective of Testing

Purpose Input Expected Output Actual Output Sample Format:

Elements of a Test Case

Algorithmic fault Computation and precision fault

◦ a formula’s implementation is wrong Documentation fault

◦ Documentation doesn’t match what program does Capacity or boundary faults

◦ System’s performance not acceptable when certain limits are reached

Timing or coordination faults Performance faults

◦ System does not perform at the speed prescribed

8.1 Software Faults and FailuresTypes of Faults

An algorithmic fault occurs when a component’s algorithm or logic does not produce proper output◦ Branching too soon◦ Branching too late◦ Forgetting to initialize variable or set loop

invariants◦ Comparing variables of inappropriate data types

8.1 Software Faults and FailuresTypical Algorithmic Faults

Module testing, component testing, or unit testing

Integration testing System Testing

◦ Function testing◦ Performance testing

Acceptance testing Installation testing

Levels of Testing

Levels of TestingTesting Organization Illustrated

Egoless programming: programs are viewed as components of a larger system, not as the property of those who wrote them

8.2 Testing IssuesAttitude Toward Testing

Independent test team◦ avoid conflict◦ improve objectivity◦ allow testing and coding concurrently

8.2 Testing IssuesWho Performs the Test?

Closed box or black box: functionality of the test objects◦ Equivalence Class, Boundary Value Analysis,

Scenario-based, Decision Table based, State Machine based…

Clear box or white box: structure of the test objects ◦ Control Flow

Basis Path, Branch, Statement, Decision…◦ Data Flow

Du Path, All-uses Path

8.2 Testing IssuesViews of the Test Objects

Black box: external behavior description State box: black box with state information White box: state box with a procedure

8.2 Testing IssuesSidebar 8.2 Box Structures

Code walkthrough Code inspection

8.3 Unit TestingCode Review

Testing the unit for correct functionality Testing the unit for correct execution

8.3 Unit Testing

Determining test objectives Selecting test cases Executing test cases

8.3 Unit TestingSteps in Testing

Statement testing Branch testing Path testing …

8.3 Unit TestingWhitebox Test Thoroughness

Whitebox TestingControl Flow Graph

Cyclomatic Complexity = E – N + 2 = 9 – 9 + 2 = 2

8.3 Unit TestingComparing Techniques

• Effectiveness of fault-discovery techniques

Requirements Faults Design Faults

Code Faults

Documentation Faults

Reviews Fair Excellent Excellent Good

Prototypes Good Fair Fair Not applicable

Testing Poor Poor Good Fair

Correctness Proofs Poor Poor Fair Fair

Big-bang Bottom-up Top-down Sandwich testing

8.4 Integration Testing

Component Driver: a routine that calls a particular component and passes a test case to it

Stub: a special-purpose program to simulate the activity of the missing component

8.4 Integration TestingTerminology

System viewed as a hierarchy of components

8.4 Integration TestingView of a System

All components integrated at once Locating faults?

8.4 Integration Testing Bing-Bang Integration Example

Only A is tested by itself Stubs of B, C and D are used at first level N-1 stubs required (N=Number of nodes) Locating faults?

8.4 Integration TestingTop-Down Integration Example

Drivers are used to call the child functions Drivers are relatively intelligent N-leaves drivers Locating faults?

8.4 Integration TestingBottom-Up Integration Example

Viewed system as three layers Employ BU where writing drivers is not costly

Employ TD wherestubs are easier toWrite Locating faults?

8.4 Integration Testing Sandwich Integration Example

Integration TestingCall-Graph Based Testing

• Adjacency Matrix• NxN matrix that tells which components call

the other components• Pairwise Integration

• Test each pair (i.e. each edge)• E testing sessions

• Neighborhood based Integration• Integrate each neighborhood

• The nodes at one edge distance from the node to be integrated

• N-sink nodes sessions

Establish test objectives Design and Write test cases Test test cases Execute tests Evaluate test results

8.6 Test Planning

Test plan explains◦ who does the testing◦ why the tests are performed◦ how tests are conducted◦ when the tests are scheduled

8.6 Test PlanningPurpose of the Plan

What the test objectives are How the test will be run What criteria will be used to determine

when the testing is complete

8.6 Test PlanningContents of the Plan

Automated Testing Tools Testing Management Tools Bug Tracking/Configuration Management

Tools

Software Testing Tools

No time left No money left Statistical Criteria

◦ Number of defects found per week becomes lower than a set threshold

8.8 When to Stop Testing

The Ariane-5’s flight control system was tested in four ways◦ equipment testing◦ on-board computer software testing◦ staged integration◦ system validation tests

The Ariane-5 developers relied on insufficient reviews and test coverage

8.10 Real-Time ExampleThe Ariane-5 System

It is important to understand the difference between faults and failures

The goal of testing is to find faults, not to prove correctness

8.11 What this Chapter Means for You

UCF Slides Software Testing, A Craftsman’s Approach

by Jorgensen Software Testing Tools by Prasad

References