module for software testing

1

Software Testing

and

Quality Assurance

Lab module

Mohd Khanapi Abd Ghani Mohd Suffian Sulaiman

Juliana Jaafar

Department of Software Engineering Faculty of Information and Communication Technology

Universiti Teknikal Malaysia Melaka 2010

2

TABLE OF CONTENTS

LAB 1: PROJECT PLANNING .............................................................................. 5

1.1 INTRODUCTION............................................................................................. 5 1.2 STEP-BY-STEP/ EXAMPLES ......................................................................... 8 1.3 EXERCISES ................................................................................................... 13 1.3 SELF REVIEW QUESTION .......................................................................... 17

LAB 2: REQUIREMENT ANALYSIS.................................................................. 18

2.1 INTRODUCTION........................................................................................... 18 2.2 STEP-BY-STEP/ EXAMPLES ....................................................................... 19 2.3 SELF REVIEW QUESTION .......................................................................... 25

LAB 3: SOFTWARE DESIGN .............................................................................. 26

3.1 INTRODUCTION........................................................................................... 26 3.2 STEP-BY-STEP/ EXAMPLES ....................................................................... 27 3.3 SELF REVIEW QUESTION ........................................................................... 29

LAB 4: SOFTWARE TEST PLAN ....................................................................... 30


LAB 5: SOFTWARE TEST DESCRIPTION....................................................... 36


LAB 6: STUB AND DRIVER ................................................................................ 44


LAB 7: RECORDING THE TEST RESULT ....................................................... 50


LAB 8: UNIT TESTING ........................................................................................ 55


LAB 9: INTEGRATION / SYSTEM TEST .......................................................... 61

9.1 INTRODUCTION........................................................................................... 61 9.2 STEP-BY-STEP/ EXAMPLES ....................................................................... 62

3

9.3 SELF REVIEW QUESTION ........................................................................... 63 LAB 10: USER ACCEPTANCE TEST................................................................... 64


LAB 11: QUALITY PLAN....................................................................................... 67

11.1 INTRODUCTION........................................................................................... 67 11.2 STEP-BY-STEP/ EXAMPLES ....................................................................... 68

LAB 12: REVIEW MEETING ................................................................................ 74


4

Faculty of Information and Communication Technology, Universiti Teknikal Malaysia Melaka, Karung Berkunci 1200, 75450 Melaka, Malaysia. © UTeM 2010 FIRST EDITION 2005 All rights reserved. No part of this publication may be reproduced stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, without either the prior written permission of the publisher. The programs in this book have been included for their instructional value. They have been tested with care but are not guaranteed for any particular purpose. The author or publisher does not offer any warranties or representations nor does it accept any liabilities with respect to the programs.

SOFTWARE TESTING &

QUALITY ASSURANCE

Mohd Khanapi Abd Ghani Mohd Suffian Sulaiman

Juliana Jaafar

5

LAB 1: PROJECT PLANNING

By the end of this section, students should be able to: Form a group and develop the team hierarchy with dedicated designation.

Describe task for each dedicated designation.

Identify the deliverables that should be produced for this lab project.

Produce a Software Development Plan (SDP) which focuses mainly on formal

qualification testing and software product evaluation.

1.1 INTRODUCTION

Project is a temporary task that should be finished in a certain period. The total

number of period to complete the project shall be agreed by both parties – contractor and

customer. In other words, project is a specialized task force within specific timeline and

budget which only finish once the task given has been accomplished. A project usually

starts in response to a broad objective or initiates through business requirement. In order

to make sure the project will be successes; the project should have a clear definition and

goal, good teamwork and sufficient resources, facilities, budget and time. Otherwise, it

will be difficult to achieve the targeted goal or fulfill the client’s needs.

Along this course, students will be exposed to an actual environment of software

development life cycle. The student will be assigned with a project and the project will be

done or execute during along twelve sessions of lab. To execute a project, the student will

follow each phases of software engineering process starting from requirement analysis,

analysis modeling, design modeling, implementation and testing to get the gist of running

a real software project. However, this lab module will only emphasize on testing phase

6

and any of testing related activities as the rest of other phases are already covered in

Requirement Analysis and Design subject. Nevertheless, te student shall deliver the

common software engineering documents including Software Development Plan (SDP),

Software Requirements Analysis (SRA), Software Requirements Specification (SRS),

Software Design Description (SDD), Software Test Plan (STP) and Software Test Result

(STR). It was noted that the same documents also being produced in most of the software

development project in industry.

As in real environment, student will need to run the project in team. They have to

form a hierarchical group organization, identify roles such as team leader, programmer,

system analyst and document’s writer, and assign each role and specific task to an

individual. Team leader will be the key person in managing the project starting from

capturing the requirement until the delivering of the product. Whilst, team members will

support the team leader by giving their full commitment and cooperation.

Communication is one of the key factors contributed to the success of the software

project. It is a good practice to have a frequent discussion or meeting in order to discuss

and resolve any arising project issues.

1.1.2 SOFTWARE DEVELOPMENT PLAN (SDP)

Software Development Plan (SDP) is a crucial document to software project

development groups as a means of recording and monitoring the planned project.

Basically, SDP contains chapters such as the Software Development Management,

Software Engineering, Formal Qualification Testing, Software Product Evaluations and

Software Configuration Management.

Software Development Management chapter focuses on the project organization

and resources which includes the contractor facilities and organizational structure. A

detail schedule and key milestones should also be included in the chapter. Each of project

activities can be represented using Gantt chart while the activity network can be

7

represented using Pert chart. In addition, this chapter also cover the risk management

aspect, security, formal review, software development library, corrective action process

and problem change report.

Software Engineering chapter consists of the organization and resources of the

development team, the software standard and the procedures used in software

development phases. The qualification of each team members will be detailed out to

present the capabilities of each personnel in software development. It also describes the

software development environment such as operating system, compilers and code

auditors in order to execute the software development activities. On top of that, it also

documents the software development techniques and methodologies together with the

design and coding standards. In summary, this chapter records all the major and

significance activities in software development life cycle of the project.

The next two remaining chapters are important and will be the major focus in this

subject. They are Formal Qualification Testing and Software Product Evaluation. The

Formal Qualification Testing describes the test organization team, the resources, the test

approach or philosophy planned, test assumption made and any constraints (if applicable)

in the testing phase. An external quality manager will be appointed to audit all the

compliance of the documents.

Lastly, Software Product Evaluation chapter describes the type of review will be

performed along the software project development process such as software specification

review, preliminary design review and test readiness review. Some of common criteria

which will be focused on the review sessions are the internal consistency,

understandability, traceability and consistency with other documents, appropriate use of

analysis, design or coding technique and the requirement covered by appropriate tests.

8

1.2 STEP-BY-STEP/ EXAMPLES

1. Student will form a group of 5 to 6 persons and develop the project organization

hierarchy. Then, propose the group name and the group motto.

2. Student will delegate the task among the team members and describe each role

and its responsibilities.

3. To ensure that each student experience the different role in project development,

the role will be rotated among the team members in the group.

4. The requirements for the project development will be explained accordingly.

5. In managing the project, student will use the filing system to record all the

software project development process by using softcopy and hardcopy.

6. To depict the real software development environment, key project milestones will

be given and for that each project activity needs to be monitored and controlled.

7. Team leader has to ensure all given tasks is distributed accordingly.

8. Each group should deliver the following:

a. Software Development Plan (SDP)

• The document describes the manner how software engineering

activities are carried out from planning stage until the stage of

software configuration management. Particularly, in this course

we only focus on the quality plan and testing plan.

b. Software Requirement Analysis (SRA)

• The document describes the user requirement and the required

characteristics of the ‘to be’ system.

c. Software Requirement Specification (SRS)

• The document describes the details specification of the ‘to be’

system.

d. Software Design Document (SDD)

• The document describes the detail architecture of the software. It

explains how objects, algorithms and data are used in building the

system. In this course we only focus on the design specification.

9

e. Software Test Plan (STP)

• The document tells the manner how the software testing is carried

out.

f. Software Test Description (STD)

• The document focuses on the detail description of each test case.

g. Software Test Result (STR)

• The document records the test result and any arising issues during the implementation of testing phase.

1.2.1 SOFTWARE DEVELOPMENT PLAN (SDP) FRAMEWORK

Cover

Document Approval

Table of Contents

List of Figure

List of Table

1.0 Introduction (Scope)

1.0 Identification

1.2 System Overview (Purpose of the system / CSCI)

1.3 Document Overview (Explain briefly each chapter)

2.0 Referenced Document

2.1 Government Document

2.2 Non Government Document

3.0 Software Development Management

3.1 Organizational Structure (Working Level)

3.2 Personnel

3.3 Project facilities

3.4 Schedule and Milestone

3.5 Risk Management (Not Applicable)

4.0 Software Engineering

4.1 Software / Hardware Items

10

4.2 Design Standards

4.3 Coding Standards

5.0 Formal Qualification Testing

5.1 Organizational Structure (Formal Qualification Testing)

5.2 Personnel (Formal Qualification Testing)

5.3 Test Approach/ Philosophy

6.0 Software Product Evaluation

6.1 Organizational Structure (Software Product Evaluation)

6.2 Personnel (Software Product Evaluation)

6.3 Software Product Evaluations Procedure & Tools

6.3.1 Procedures

6.3.2 Tools

6.4 Activity (Dependent Product Evaluation)

6.4.1 Software Development Plan Review (SDPR)

6.4.2 Software Specification Review (SSR)

6.4.3 Software Design Review (SDR)

6.4.4 Test Readiness Review (TRR)

7.0 Software Configuration Management (Not Applicable)

8.0 Notes (acronym & abbreviation)

9.0 Exhibit (Annex / Appendix)

11

1.2.3 SOFTWARE DEVELOPMENT MODEL In building a software product, it is essential to go through a sequence of phases

which helps in building a timely, budgetary, high quality product. This series of

processes is called software development process. There are several variations of

software development models to be followed such as Waterfall model, Spiral Model,

Incremental Model and RAD model. However, the typical phases of software

development are conceptual, requirements gathering, design and analysis,

implementation, testing and operation and maintenance. The selection of the model

mainly based on the background of the project, the organization and the people involved

in the project. Ones need to wisely select the software development model. Moreover,

the chosen software development model will affect the approach taken in testing. This is

indeed important in planning testing activities of the project.

1.2.2 CODING STANDARDS AND GUIDELINES

In formal reviews, the inspectors will look for problems and omissions in the

code. There are classic bugs where something just won’t work as expected. These can be

found by careful analysis of the code – senior programmers and testers are great at this.

There are also problems where the code may operate properly but may not

comply to the specific standard and guideline. It’s equivalent to writing words that can be

understood and get a point across but don’t meet the grammatical and syntactical rules of

the English language. Standards are the established, fixed, have-to-follow-them rules-the

do’s and don’ts. Guidelines are the suggested best practices, the recommendations and

the preferred way of doing things. Standards are rules that need to comply, have no

exceptions and no short of a structured waiver process. Whilst, guidelines can be a bit

loose, which are not mandatory, yet something valuable to practice.

12

1.2.3 EXAMPLE OF PROGRAMMING STANDARDS

TOPIC 3.05 Control-Restriction on control structures

STANDARD The go to statement (and hence labels as well) should not be used.

The while loop should be used instead of the do-while loop, except where the logic of the problem explicit requires doing the body at least once regardless of the loop condition

If a single if-else can replace a continue, an if-else should be used

JUSTIFICATION The go to statement is prohibited for the empirical reason that its use is highly correlated with errors and hare-to-read code, and for the abstract reason that algorithms should be expressed in structures that facilitate checking the program against the structure of the underlying process.

The standard has four main parts:

i. Title : Describes the topic that the standards covers.

ii. Standard (or guideline) : Describes the standard or guideline explaining

exactly what is allowed and what is not.

iii. Justification : Provides the reasoning behind the standard so that the

programmer understands why it’s good programming practice.

iv. Example: Shows the simple programming samples of how to apply the

standards.

13

1.3 EXERCISES Before you start your project assignment, form a project group consist of 3-4 persons.

Develop organisation structure including project personnel and responsibilities.

a) Below are templates for project title and motto, and basic project organisation. Fill in

the empty boxes or develop your own project organisation.

___________________

[Team Leader]

__________________

[System Analyst]

__________________

[Technical Writer]

__________________

[Programmer]

__________________

[Software Tester / Quality Auditor]

Project:_____________________________________

Motto:____________________________________

14

b) Below is a sample table on how to list down the required human resources and

their job description. Fill in the empty boxes or create your own format for

determining the human resources requirement in your project.

Role Description Quantity _________________

[Team Leader]

_________________

[System Analyst]

_________________

[Technical Writer]

_________________

[Software Tester / Quality Auditor]

_________________

[Programmer]

c) Choose an appropriate software development model for your project and justify your

answer.

15

d) Identify two programming standards and three programming guidelines for your project.

STD 001 <Topic> Standard :

Justification :

Example :

STD 002 <Topic> Standard :

Justification :

Example :

GDL 001 <Topic> Standard :

Justification :

Example :

16

GDL 002 <Topic> Guideline :

Justification :

Example :

GDL 003 <Topic> Guideline :

Justification :

Example :

17

1.3 SELF REVIEW QUESTION

1. What’s wrong with just testing that a program works as expected?

2. How much more does it cost to fix a bug found after the product is released

compared to a bug found at the very start of the project?

3. What’s the goal of a software tester?

4. List reasons why the product specification is usually the largest source of bugs in

a software product?

5. Name several tasks that should be performed before a programmer starts writing

the first line of code.

Comment [MK1]: rephrase

18

LAB 2: REQUIREMENT ANALYSIS

By the end of this section, students should be able to:

Produce the Software Requirement Analysis (SRA) which captures the

functional requirements.

Produce the Software Requirement Specification (SRS) which describes detail

requirements.

2.1 INTRODUCTION

Software is written to fulfill specific needs that a person or a group of people has.

Let’s call them the customer. To ensure that we are building the right software product,

the development team must find out what exactly the customer wants. Some teams

simply guess, but most collect detailed information in the form of surveys, feedback from

previous versions of the software, competitive product information, magazine reviews,

focus groups, and numerous other methods, some are formal, and some are not. The

gathered information is then studied and analyzed, condensed and interpreted to define

exactly what features the software product should have.

The result of the customer requirements studies is just raw data. It doesn’t

describe the detail of the proposed product; it just confirms on what to be created and

what are the features required by customer. The detail will be specified in requirement

specifications. It will utilize all the information and any unstated but mandatory

requirements and define what the product will be, what it will do and how it will look.

The format of specification varies greatly. Some companies-especially those

developing products for the government, aerospace, financial and medical industries –

use a very rigorous process with many checks and balances. The result is an extremely

detailed and through specification that’s locked down, meaning that it can’t change

19

except under very extreme conditions. Everyone on the development team knows exactly

what they are creating.

There are development teams, usually ones who create software for less-critical

applications, who produce specifications on cocktail napkins, if they create them at all.

This has distinct advantage of being very flexible, but there’s lots of risk than not

everyone is “on the same page.” And, what the product finally becomes isn’t known until

it’s released.


1. For each project, construct the Software Requirement Analysis (SRA) which

records all the requirements with its dedicated identification.

2. Conduct an internal review session to review the SRA document. The main

objective of the review session is to identify any bugs in the document. List

all bugs found during the session and update the SRA document accordingly.

3. From the 11 factors in Mc Call’s Quality model, identify the main quality factors

that you think important to be included in your project. Then, match each

functional requirement to the identified quality criteria.

4. Based on the SRA, constructs the Software Requirement Specification (SRS)

which describes in detail the description of each requirements. Specifically in

this course, the SRS should be described using use cases.

5. 3. Once the Software Development Plan (SDP) baseline has been created, the

plan can be sent for a Software Development Plan Review (SDPR). Any

findings/issues found by internal Software Tester / Quality Auditor will be

recorded in Review Report Form (RRF).

2.2.1 SOFTWARE REQUIREMENT ANALYSIS (SRA) FRAMEWORK

Cover

Document Approval

Table of Contents

List of Figure

20

List of Table

1.0 Introduction

1.1 Identification

1.2 System Overview

1.3 Document Overview




3.0 Functional Requirement

4.0 Non Functional Requirement (Not Applicable)

5.0 Exhibit

Note: Diagram involved is use case diagram and activity diagram

2.2.2 EXAMPLE OF FUNCTIONAL REQUIREMENT IN SRA

FR_No Requirement Description Phase

BDC_1 Format Date Rules - Ability to define the type of dates

-

1

…

21

2.2.3 EXAMPLE OF BUGS IN SOFTWARE REQUIREMENT ANALYSIS

(SRA)

Section Number

Problematic Phrase or Section Bug and corrective actions

X.X.X “Wrong IC format should not be migrated”

• Avoid negative phrases. – To rephrase ““The system should only migrate a correct IC format. ”

• Ambiguous statement - Should states clearly the correct IC format.

X.X.X “The calculation of total price must include service fees.”

• Unclear statement – Should states the amount of the service fees and the calculation of the service fees (if applicable)

X.X.X Abbreviation of IC has been specified under “Terms Acronyms and Abbreviations” but some of the sections still use the word “Identification Card”.

• Inconsistency – Should either remove the abbreviation of IC from the list or make sure all sections use the specified abbreviation.

X.X.X “The browser should respond within acceptable time”

• Not testable – Should specified the duration of acceptable time. E.g “The browser should response within 10 seconds”

• •

22

2.2.4 QUALITY FACTORS AND REQUIREMENTS

ID Version Section Description Priority

000 Suitability 000-001 1.0 3.1 The site is suitable for IE version 5 and

higher. 1

000-001 1.1 3.6 The system shall accept payment in local currencies, US dollars and Pounds.

2

010 Security 010-001 2.0 6.0 Action should be taken within 1 hour

for all reports regarding high priority security event.

1

020 Interoperability 020-001 1.0 1.1 The system should be deployed in any

operating systems. 3

2.2.5 SOFTWARE REQUIREMENT SPECIFICATION (SRS) FRAMEWORK

Cover

Document Approval

Table of Contents

List of Figure

List of Table

1.0 Introduction

1.1 Identification

1.2 System Overview


1.4 Summary of use case




23

3.0 Use Case Specification

4.0 Notes (acronym & abbreviations)

5.0 Exhibit / Annex / Appendix

note: diagram involved is sequence diagrams and GUI. 2.2.6 EXAMPLE OF ENGINEERING DOCUMENTS IN SRS

2.1.1 Use Case “Format Rule Date” [SRS_REQ_100]

2.1.1.1 Brief Description

The use case provides the capability to display the format of date rule

according the user needs.

2.1.1.2 Characteristic of Activation

The use case starts when the user runs the format of date rule function

from the main menu.

Pre-Condition(s)

Not applicable

Flow of Events

2.1.1.4.1 Basic Flow

1. This use case begin when the user select the menu which is :

a. Ability to define type of dates [A1]

b. ____________________________

c. ____________________________

2. The use case ends.

2.1.1.4.2 Alternative Flow

24

[A1] Types of dates [SRS_REQ_101]

1. The user key in the type of date based on the format given.

2. The system will display the type of dates [E1]

3. The use case ends.

[A2] ________________________[ ]

1. _________________________

2. _________________________

3. _________________________

[A3] _________________________[ ]

1. __________________________

2. __________________________

3. __________________________

2.1.1.4.3 Exception flow

[E1] – Invalid input

If invalid input accounted, the system should display the following

message ‘Invalid input’ on the message panel

2.1.2.5 Post-Condition(s)

The business date is message is displayed

2.1.2.6 Rule(s)

Not applicable

2.1.2.7 Constraints(s)

Not applicable

25


1. Why can the waterfall methodology method be difficult to use? 2. Why would a software tester like the spiral model better than others? 3. Can a software tester perform white-box testing on a specification? 4. Explain what’s wrong with this specification: When the user selects the Compact

Memory option, the program will compress the mailing list data as small as possible using a Huffman-sparse-matrix approach.

5. Explain what a tester should worry about with this line from a specification: The

software will allow up to 100 million simultaneous connections, although no more than 1 million will normally be used.

26

LAB 3: SOFTWARE DESIGN

By the end of this section, students should be able to: Produce the Software Design Document (SDD).

The students have a basic understanding regarding the configuration

identification.

3.1 INTRODUCTION One common misconception is that when a programmer creates a program, he

simply sits down and starts writing code. That may happen in some small, informal

software project, but for anything other than the smallest programs, there must be a

design process to plan out how the software will be written. Think about this book, which

required an outline before the first words were typed, or a building, which needs

blueprints drawn before the first concrete is poured. The same process should also be

implemented in building software.

The documents produced in design phase vary greatly depending on the company,

the project, and the team, but the main purpose is to plan and organize the codes that are

to be written.

27


1. Student will develop the detail design in Software Design Document (SDD).

2. Student will construct the configuration identification in order to develop the

traceability among design documents.

3. Students should already complete the Software Requirement Analysis (SRA) and

Software Requirement Specification (SRS) baseline and able to establish the

Software Specification Review (SSR) and the internal Software Tester / Quality

Auditor and will state all the arising matter in the Review Report Form (RRF).

3.2.1 SOFTWARE DESIGN DOCUMENT (SDD) FRAMEWORK

Cover

Document Approval

Table of Contents

List of Figure

List of Table

1.0 Introduction

1.1 Identification

1.2 System Overview





3.0 Preliminary Design (Not Applicable)

3.1 CSCI Overview

3.2 CSCI Architecture

3.3 CSCI Design Description

4.0 Detail Design (algorithm)

5.0 Requirement Traceability


28

7.0 Exhibit / Annex / Appendix

note : diagram involved is component diagram, package diagram, class diagrams.

3.2.2 EXAMPLE OF CONFIGURATION IDENTIFICATION

i. Example of document identification

SDD / BITP 3213 / S1 / G1 / 2004

ii. Example of requirement identification

2.1.1 Use case “Format Date Rule” [SRS_REQ_100]

…

2.1.1.4.2 Alternative flow

[A1] Types of dates [SRS_REQ_101]

iii. Example of traceability identification

SRS SDD Description

SRS_BDC_001 SDD_FDR_101 Ability to define types of dates

SRS_BDC_002 SDD_FDR_101 Ability to define types of holiday

SRS_BDC_003 SDD_FDR_101 Ability to configure an observable

holiday

29


1. Due to time constraints, it would be impossible to execute a thorough and complete test. Thus, what information do you think should be considered when deciding to stop the testing?

2. Is it possible to have a high-quality and low-reliability product? Provide an

example to support your answer.

3. Why is it impossible to test a program completely?

4. If you were testing a feature of your software on Monday and finding a new bug every hour, at what rate would you expect to find bugs on Tuesday?

5. If you’re testing a program’s ability to print a document to a printer, what would

be appropriate generic test-to-fail test cases?

30

LAB 4: SOFTWARE TEST PLAN

By the end of this section, students should be able to:

Develop the Software Test Plan (STP) document.

Understand the concept and the significance of Software Test Plan in software

development life cycle.

4.1 INTRODUCTION

The testing process can’t operate in isolation. Performing your testing tasks would

be very difficult if the programmers wrote their code without telling you what it does,

how it works, or when it will be complete. Likewise, if you and the other software testers

don’t communicate what you plan to test, what resources you need, and what your

schedule is, your project will have little chance of succeeding. The software test plan is

the primary means by which software testers communicate their testing schedules,

activities, approaches and measurement to the rest of software development team.

The IEEE Standard 829-1998 for Software Test Documentation states that the

purpose of the software test plan is as follows:

To prescribe the scope, approach, resources, and schedule of the testing

activities. To identify the items being tested, the features to be tested, the testing

tasks to be performed, the personnel responsible for each task, and the risks

associated with the plan.

Given that definition and the rest of the IEEE standard, you will notice that the form the

test plan takes is a written document. That shouldn’t be too surprising, but it’s an

important point because although the end result is a piece of paper (or online document or

web page), that paper isn’t what the test plan is all about.

The test plan is a by product of the detailed planning process that’s undertaken to

create it. It’s the planning process that matters, not the resulting document.

31

The Software Test Plan is used as the basis of project management control

throughout the testing process. It is a living document that guides test execution through

to conclusion, and it must be updated to reflect to any changes.

Depending on your project background and characteristic, you may have multiple

test plans. You may consider having multiple test plans when your test have:-

• Different phases and levels (e.g. unit testing, integration testing, beta

testing, etc)

• Different test types (e.g. performance testing, functional testing, regression

testing, etc)

• Different audience (e.g. hardware and software test)

However, to reduce the risk of redundancy and overlap, ones may have one master plan

which describes the common elements for the overall testing phase.

Basically, a STP should cover information specifying the approach to testing, test

organization, test environment, test strategy and test schedule.

• Test organization

Test organization specifies the staff members who will be expected to involve in the

testing phase, the role that they will perform and their responsibilities.

• Test environment

Test environment describes the setting and context in which the test will be executed.

It also covers the software and hardware to be used in order to run the testing.

• Test strategy

Test strategy specifies the approaches and techniques used in testing software

product.

• Test schedule

32

Test schedule outlines the time frame together with the individual who responsible for

each activity in the testing process. The time frame must be synchronized with the

project plan. Any changes in the schedule might affect the project plan as a whole.

• Test cycle

Test cycle records number of iteration in running the testing. Test cycle can be

counted based on happy or unhappy scenarios, modules, users or locations. It will

ensure the result can be verified.


1. Identify level and the type of testing to be executed for the project. Describes the

objective and goal for each of the testing level or testing type.

2. Based on the types and level of testing identified, each group will develop the

testing plan(s) as accordance to the framework given.

3. Students should already complete the Software Design Document (SDD) baseline

and able to establish the Software Design Review (SDR) and will state all the

arising matter in the Review Report Form (RRF).

4.2.1 SOFTWARE TEST PLAN (STP) FRAMEWORK

Cover

Document Approval

Table of Contents

List of Figure

List of Table

1.0 Introduction

1.1 Test Plan Identifier

1.2 Purpose – Describes the main objective and goal of the test plan

33

1.3 Background – Briefly describes the overall project testing background

information

1.4 System Overview – An overview of the testing and how it relates with

the critical/high risk project functions

1.5 Applicable documents – List all documents pertinent to test program

1.6 Document Overview - Explain briefly on each chapter

2.0 Scope of Testing

2.1 Test Items – Specifies the components or modules together with their

versions that need to be tested

2.2 Features to be tested and Risk issues

2.3 Features not to be tested and Risk issues

3.0 Testing Strategy and Approach

3.1 Overview – Describes the strategies, approaches or methodologies

used for the testing

3.2 Testing Criteria

3.2.1 Entry criteria – A list of criteria to measure that the system

is ready for a particular test phase.

3.2.2 Exit criteria – A list of criteria to measure that the system is

ready to end the testing phase.

3.2.3 Suspension or resumption criteria – A list of criteria to

measure that the system needs to be suspended / resumed.

3.3 Configuration Management – E.g. Version controls, release

management

3.4 Problem / Changed Management

3.4.1 Inspection and Analyze procedures

3.4.2 Problem reporting/tracking procedures – Includes the

procedures for notifications and escalations

3.4.3 Administrative procedures

3.4.4 Review/Meeting procedures

3.5 Testing tools – Describes any testing tools or supporting tools that

will be used in the testing.

34

3.6 Test Measurements and Metrics

4.0 Testing Deliverables – E.g. reports, charts

5.0 Testing Resources, Schedule and Tasks

5.1 Testing Roles and Responsibilities

5.2 Staffing and Training needs

5.3 Test Task Structure – List of all test activities, work breakdown

structure, responsible persons and key milestones

6.0 Environmental and Infrastructure Needs - Includes any configurations

or setup needed for the environment or infrastructure

7.0 Risk and Contingencies

8.0 Approval



note : exhibit shall include the test cases for unit testing, integration & system

testing & user acceptance test.

35


1. What is the purpose of a test plan?

2. Why is it the process of creating the plan that matters, not the plan itself?

3. Names a few typical testing resources that should be considered in test planning phase?

4. What are the four test cases planning?

5. What is ad hoc testing?

36

LAB 5: SOFTWARE TEST DESCRIPTION

By the end of this section, the students should be able to:

Design the test cases and produce Software Test Description (STD).

Understand the concept and the significance of Software Test Description

(STD)

5.1 INTRODUCTION

The overall project test plan is written at very high level. It breaks out the

software into specific features and testable items and assigns them to individual testers,

but it doesn’t specify exactly how those features will be tested. There may be a general

mention of using automation or black-box or white-box testing techniques, but the test

plan doesn’t get into the details of exactly where and how they will be used. This next

level of detail that defines the testing approach for individual software features is the test

design specification.

IEEE 829 states that the test design specification “refines the test approach

[defined in the test plan] and identifies the feature to be covered by the design and its

associated tests. It also identifies the test cases and test procedures, if any, required to

accomplish the testing and specifies the feature pass/fail criteria”

The purpose of the test design specification is to organize and describe the testing

that needs to be performed on a specific feature. It doesn’t, however, give the detailed

cases or the steps to execute to perform the testing. The following topics, adapted from

the IEEE 829 standard, address this purpose and should be part of the test design specs

that you create:

Identifiers. A unique identifier that can be used to reference and locate the

test design spec. The spec should also reference the overall test plan and

contain pointers to any other plans or specs that it references.

37

Features to be tested. A description of the software feature covered by the

test design spec – for example. “the addition function of calculator,” “font size

selection and display in WordPad,” and “Video card configuration testing of

QuickTime.”

Approach. A description of the general approach that will be used to test the

features. It should expand on the approach, if any, listed in the test plan,

describe the technique to be used, and explain how the results will be verified.

For example – A testing tool that used in the testing.

Test case identification. A high level description and references to the

specific test cases that will be used to check the feature. It should list the

selected equivalence partitions and provide references to the test cases and test

procedures used to run them. For example – “Check the highest possible value

– Test Case ID# 15326”.

Pass/Fail Criteria – Describes exactly what constitutes a pass and a fail of

the tested feature. What is acceptable and what is not? This may be very

simple and clear – a pass is when all the test cases are run without finding a

bug. It can also be fuzzy – a failure is when 10 percent or more of the test

cases fail. There should be no doubt, though, what constitutes a pass or a fail

of the feature.

5.1.1 TEST CASES

IEEE 829 states that the test case specification “documents the actual values used

for input along with the anticipated outputs. A test also identifies any constrains on the

test procedure resulting from use of that specific test case”.

Essentially, the details of the test case should explain exactly what values or

conditions will sent to the software and what result is expected. It can be referenced by

one or more test design specs and may reference more than one test procedure. The IEEE

829 standard also lists some other important information that should be included:

Identifiers. A unique identifier is referenced by the test design specs and the

test procedure specs.

38

Test item. Describes the detailed feature, code module and so on that’s being

tested. It should be more specific than the features listed in the test design plan

or specification. If the test design spec said “the addition function of

Calculator,” the test case spec would say “upper limit overflow handling of

addition calculations.” It should also provide references to product

specifications or other design docs from which the test case was based.

Input specification. Specifies a list all inputs or conditions given to the

software to execute the test case. If you’re testing calculator, this may be as

simple as 1+1. If you’re testing cellular telephone switching software, there

could be hundreds or thousands of input conditions. If you’re testing a file

based product, it would be the name of the file and a description of its

contents.

Output specification. Describes the result you expect from executing the test

case. Did 1+1 equal 2? Were the thousands of output variables set correctly in

the cell software? Did all the contents of the file load as expected?

Environmental needs. Environmental needs are the hardware, software, test

tools, facilities, staff, and so on that are necessary to run the test case.

Special procedural requirements. This section describes anything unusual

that must be done to perform the test. Testing WordPad probably doesn’t need

anything special, but testing nuclear power plant software might.

5.1.2 TEST PROCEDURES

After you document the test designs and test cases, what remains are the

procedures that need to be followed to execute the test cases. IEEE 829 states that the test

procedure specification “identifies all the steps required to operate the system and

exercise the specified test cases in order to implement the associated test design”

The test procedure or test script specification defines the step-by-step details of exactly

how to perform the test cases. Here’s the information that needs to be defined:

39

Identifier. A unique identifier that ties the test procedure to the associated test

cases and test design.

Purpose. The purpose of the procedure and reference to the test cases that it will

execute.

Special requirements. Other procedures, special testing skills or special

equipment needed to run the procedure.

Procedure steps. Detailed description of how the tests are to be run:

o Log. Tells how and by what method the results and observations will be

recorded.

o Setup. Explains how to prepare for the test.

o Start. Explains the steps used to run the test.

o Procedure. Describes the steps used to run the test.

o Measure. Describes how the results are to be determined-for example.

With a stopwatch or visual determination.

o Shut down. Explain the steps for suspending the test for unexpected

reasons.

o Restart. Tells the testers how to pick up the test at a certain point if

there’s a failure or after shutting down.

o Stop. Describes the steps for an orderly halt to the test.

o Wrap up. Explains how to restore the environment to its pre-test

condition.

o Contingencies. Explains what to do if things don’t go as planned.

40

5.1.3 TEST DESIGN APPROACH AND TECHNIQUES

Software test description, test cases and test procedures are the result from the

test analysis and design phase. In the activity, different approaches and

techniques will be used to transform the general testing objective defined during

test planning phase into tangible test conditions and test cases. Below are the

summarization of the common approach and common techniques used in

designing the test cases (Dustin, et al., 2000).

White Box Approach Black Box Approach

A structural testing approach focuses on internal structure of the system. Program based.

Functional testing approach focuses on application externals. Specification based.

Type of Test Unit testing Code level Integration testing

Code level Integration Testing System level Integration Testing System Testing User acceptance testing Usability testing

Techniques Fault Insertion String Test Error Handling Statement Coverage Decision Coverage Path Coverage Condition Coverage Data Flow Coverage Memory Leak

Boundary Value Analysis Cause-Effect Graphing Error Guessing Path Analysis Equivalence Partitioning Decision Table Testing State Transition Testing Use Case Testing

Personnel Involved Developers and/or Test Engineers

Developers Test Engineers End Users

41


Prerequisite: The SRA, SRS and SDD should be completed before we proceed

with test design.

1. Within a group, students will discuss and develop the Software Test

Description (STD) based on the framework given. For each testing level/type,

identifies the approach and techniques used.

• Students will utilize the SDD in constructing the unit test cases

• Students will utilize the SDD and/or SRS in constructing the

integration testing

• Students will utilize the SRS in constructing the system testing

• Students will utilize the SRA in constructing the user acceptance

testing

2. The students should start the implementation or able to develop the source

code for the system that had been analyzed and designed. The students should

delegate each functions to develop for each team member.

5.2.1 SOFTWARE TEST DESCRIPTION (STD) FRAMEWORKS

Cover

Document Approval

Table of Contents

List of Figure

List of Table

1.0 Introduction ( Scope )

1.1 Identification

1.2 System Overview ( Purpose of the system / CSCI)

1.3 Document Overview ( Explain briefly each chapter)


42



3.0 Scope of the tests

3.1 The Software package/item to be tested (name, version and

revision)

3.2 The documents providing the basis for the designed tests (name

and version for each document)

4.0 Test environment (for each test)

4.1 Test identification (the test details are documented in the STP)

4.2 Detailed description of the operating system and hardware

configuration and the required switch settings for the tests

4.3 Instructions for software loading

5.0 Testing process

5.1 Instructions for input, detailing every step of the input process

5.2 Data to be recorded during the tests

6.0 Test cases (for each case)

6.1 Test case identification details

6.2 Input data and system settings

6.3 Expected intermediate results (if applicable)

6.4 Expected results (numerical, message, activation of equipment,

etc)



43


1. What is the purpose of a test design specification?

2. What is a test case specification?

3. Other than a traditional document, what other means of resources can you use in developing you test cases?

4. What is the purpose of a test procedure specification?

5. At what level of detail should test procedures be written?

44

LAB 6: STUB AND DRIVER


Understand the concept and the significance of the stub and driver.

Create the test driver and stub in order to perform unit test.

6.1 INTRODUCTION

Unit testing usually runs in development environment which require the

access to the code. It involves the testing of each function, module, or class and also

the integration of those units. For the integration of the units ones often greatly use

either of these two strategies:-

• Big bang testing

To test all units at one go, once the completed package is available.

• Incremental Testing

To test units in modules, as they are completed (unit tests); then to test

groups of tested modules integrated with newly completed modules

(integration tests). This process continues until all the package modules

have been tested. Once this phase is completed, the entire package is

tested as a whole (system test).

They are three strategies in which to construct the hierarchy of the

integration modules:

o Top-down

Starts testing at the highest level module in the software structure

and the last modules to be tested are the lowest level modules.

45

o Bottom-up

This strategy is the reverse of Top-down. It starts at the lowest

level modules and ends with the highest modules.

o Backbone

Starts with critical modules (according to risks order) and ends

with the least critical modules.

In implementing the above categories, ones should construct stubs and drivers. Stubs

and drivers are software replacement simulators required for modules not available when

performing a unit or an integration test.

• Driver

A driver is a substitute module of the upper level module. The driver

passes the test data on to the tested module and accepts the results from

the tested module. Drivers are required in bottom up testing until the top

level modules are developed.

• Stub

A stub or “dummy module” replaces an unavailable lower level module,

subordinate to the module tested. Stubs are required for top down testing

of incomplete system. In this case, stub provides the results to the module

being tested.


1. Student will create stubs and drivers to perform the unit test based on the

example given. For the unit integration testing, student can choose either to

use the top-down or bottom-up approach.

2. At this stage, students should complete the baseline of Software Test Plan

(STP).

46

6.2.1 EXAMPLE FOR STUB AND DRIVER

The objective of this example is to combine three words into one word called

“SAYA MINAT BELAJAR”. The project manager divides the task into three classes and

each class will be assigned a programmer for coding work. The distributions of task are

arranged as follows:

1. Hafiz will perform task1, which consists of three methods – saya, NamaSaya

and NamaKeluarga

2. Kiah will perform task2, which consists of one method - Minat

3. Radzi will perform task3, which consists of the method - Belajar

Problem:

Unfortunately, Kiah could not complete the task2 as planned in the schedule. The

tester has to complete the unit testing for the whole module in order to perform

the next stage of testing.

Instruction:

Supposed you are the software tester and requested by your manager to run the

test even though the task2 is still not completed.

Problem solving:

1. Create a stub names stubtask2.java to replace the uncompleted unit of task2.

2. In order to test the data contains in the task1’s class, create a driver called

driverTask1.java

3. By using the stubtask2.java, create the main class names motto.java

47

The decomposition view for the example module is

Motto

Task1 Task2 Task3

/* * file name : task1.java * The purpose of this class is to display name * */ public class task1 { public static String saya(String mesg) { mesg = "Saya "; return mesg; } public static String namaSaya(String mesg) { mesg = "Mohd Suffian"; return mesg; } public static String namaKeluarga(String mesg) { mesg = "Sulaiman "; return mesg; } }

/* * file name : stubtask2.java * The purpose of this class is to display Minat */ public class stubTask2 { public static String minat(String mesg) { mesg = "Minat"; return mesg; } }

48

/* * file name : task3.java * the purpose of this class is to display Belajar */ public class task3 { public static String belajar(String mesg) { mesg = "Belajar "; return mesg; } }

/* * file name : task1Driver.java * The purpose of this class is to test the spesific unit * software of task1 */ class task1Driver { public static void main(String[] args) { String mesg = " "; task1 displayName = new task1(); System.out.println(displayName.saya(mesg)+ " " +displayName.namaSaya(mesg)+ displayName.namaKeluarga(mesg)); } }

49


1. What is the difference between a test stub and a test driver?

2. Always design your black box test cases first. True or False?

3. If your product development is in a hurry, you can skip module testing and proceed directly to integration testing. True or False?

4. Provide two reasons why testing in a big bang software development model is

nearly impossible? How can these be addressed?

5. What is the difference between white box testing and debugging?

/* * file name : motto.java * The purpose of this class is to call all related classes * to display the combined words */ class motto { public static void main (String[] args) { String mesg = " "; task1 displayTask1 = new task1(); stubTask2 displayTask2 = new stubTask2(); task3 displayTask3 = new task3(); System.out.println(displayTask1.saya(mesg)+ " "+displayTask1.namaSaya(mesg)+displayTask1.namaKeluarga(mesg)+ " " + displayTask2.minat(mesg)+ " "+displayTask3.belajar(mesg)); } }

50

LAB 7: RECORDING THE TEST RESULT


Write and prepare the Software Test Result (STR).

Understand the concept and the significance of Software Test Result (STR) in

testing phase.

7.1 INTRODUCTION

If you stand back and look at the big picture of software testing, you’ll see that it

has four main phases. They are test planning, test analysis and design, test execution and

test evaluation and reporting, which will be covered by this topic.

On the surface, it may seem as though reporting the problems you discover would

be the easiest compared to the other phases. The skills to efficiently find bugs and telling

the world about the bugs would surely be simpler and less time-consuming job. However,

in reality, it might be the most important - and sometimes most difficult – task for the

software tester.

There some fundamental principles for reporting bug:

Report bugs as soon as possible. The earlier you find the bug, the more time it

remains in the schedule to get it fixed. Suppose that you find an embarrassing

misspelling in a Help file a few months before the software is released. That bug

has a high likelihood of being fixed. If you find the same bug a few days before

the release, odds are it won’t be fixed. This may seem strange – the bug is still the

same bug whether you find it today or three months from now. Ideally, it

shouldn’t matter when it is found, just what the bug is. In reality, however, the

risks of fixing the bugs go up over time and increasingly weigh on the decision-

making process.

Effectively describe the bugs. Suppose you were a programmer, and receiving

the following bug report from a tester: “Whenever I type a bunch of random

51

character in the login box, the software starts to do weird stuff.” How would you

even begin to fix this bug without knowing what the random characters were, how

big a bunch is, and what kind of weird stuff was happening?. The bugs report

should provide detailed information about the incident and it should lead to

development and process improvement. Tester must be aware that bugs reports

will be read by different type of readers for different goals (e.g Developer – who

need to fix the bugs, Managers- who need to make resource allocation,

prioritization decisions about the problem, Technical support – who need to be

aware of deferred/unfixed issues at ship time, Testers – who need to know the

current state of the system). Thus, bugs report must be clear, straight-forward,

condense and focus.

Be nonjudgmental in reporting a bug. It’s easy for testers and programmers to

form an adversarial relationship. Bug reports can be viewed by the programmers

and others on the development team as the tester’s “report card” on their work.

Therefore, the bugs report must be nonjudgmental, non-personal and non-

inflammatory. A bug report that says, “Your code for controlling the printer is

terrible, it just plain doesn’t work. I can’t believe that you even checked it in for

testing,” should be out of line. Bug reports should be written against the product,

not the person, and states only the facts. No gloating, no grandstanding, no

personalizing, no egos, no accusations. Tact and diplomacy are what matters.

Follow up on your bug reports. One thing worse than not finding an important

bug is finding a bug, reporting it, and then forgetting about it or losing track of it.

You’ve learned that testing software is hard work, so don’t let the results of your

labor, the bugs you find, become neglected. From the moment you find the bug,

it’s your responsibility to make sure that it’s reported properly and given the

attention it needs. A good tester finds and logs lots of bugs. A great tester finds

and logs lots of bugs but also continues to monitor them through the process of

getting them fixed.

52


1. Students will construct the Software Test Result (STR) based on the framework

given.

2. Once done, students may proceed to the coding phase.

3. The blank template for Test Incident report for unit testing, integration/system

testing and the user acceptance test should be completed.

7.2.1 SOFTWARE TEST RESULT (STR) FRAMEWORK

Cover

Document Approval

Table of Contents

List of Figure

List of Table

1. Introduction ( Scope )

1.1. Test incident report identifier

1.2. Summary – Describes the summary of the bugs report in one line,

especially impact on stakeholders

1.3. System Overview (Purpose of the system / CSCI)

1.4. Document Overview (Explain briefly each chapter)

2. Test Overview

2.1. The documents providing the basis for the tests (name and version for

each document)

2.2. Test site

2.3. Initiation and concluding times for each testing session

2.4. Test team members

2.5. Other participations

2.6. Hours invested in performing the tests

3. Test environment

53

3.1. Software and Hardware configurations

3.2. Preparations and training prior to testing

4. Summary tables for testing

4.1. Summary of current test

4.2. Summary tables for total number of errors

5. Incident

5.1. Events and unpredicted responses of the software during testing

5.1.1. Inputs

5.1.2. Expected results

5.1.3. Actual results

5.1.4. Anomalies

5.1.5. Date and time of the incident

5.1.6. Reproducibility

5.1.7. Reported by

5.1.8. Priority

5.1.9. Severity of the impact on the system

5.1.10. Risk, costs, opportunities and benefits of fixing/not fixing the bugs

5.1.11. Other details

5.2. Proposals

5.2.1. Changes in the test environment, including test preparations.

5.2.2. Changes or corrections in test procedures and test case files.

6. Notes (acronym & abbreviations)

7. Exhibit (Annex / Appendix)

note: exhibit shall include the test cases for unit testing, integration & system

testing & user acceptance test.

54


1. What is software test result?

2. What is the difference between software test result and software test description? 3. What is test incident report?

4. Basically how to classify the bug?

5. What are the basic principles can you apply in your bug reports to give them the

best chance of getting the bug fixed?

55

LAB 8: UNIT TESTING


To prepare, execute and conduct the unit test.

To record the unit test in the bugs report framework.

8.1 INTRODUCTION

The objective of unit testing is to ensure that reliable program units are produced

meet the specified requirements. The process is also intended to identify errors in

program logic. Typically the software engineer or programmer who coded the unit will

design and run a series of test to verify those units meet their requirements.

The precise definition of a “unit” will depend on the approach to design of testing

process and the implementation technology employed in its development, for example:

• A unit in an application developed using a procedural programming language could

be represented by a function or procedure, or closely related group of functions or

procedures.

• A unit in an application developed using an object-oriented programming language

could be represented by a class or an instance of a class or the functionality

implemented by a method

• A unit in a component based development environment could be a predefined

reusable component.

56

Unit testing is typically a white box testing approach, based primarily on the

functional and data requirements expressed in the design documentation. Example of area

typically covered in unit testing includes:

• Correctness of calculations/manipulations performed by the unit.

• Low-level performance issues (such as performance bottlenecks observed

under repeated invocation of the unit and its functionality)

• Low level reliability issues (such as memory leaks observed under repeated

and extended invocation of the unit and its functionality)

• Screen/window and navigation (for graphical user interface (GUI) units)

including consistency of the “look and feel” of the window, use of “hot keys”,

and function keys and keyboard shortcuts

• File/record creation, update, and deletion.

• Report contents, layout and calculations

• Communication between interoperating units.

Besides than executing the tests, it is also essential to have a walkthrough session to

review the developed code. The source code can be compared against the low level

design description described in the SDD. In addition, we may also compare the source

code with the programming standard/guideline document. Below are few examples of

bugs that can be found during code inspection/review:

• Data reference errors – e.g. uninitialized variables

• Data declaration errors – e.g similar variable names

• Computation error – e.g possibly of null pointer exception

• Control flow error – e.g. possible premature loop exit

• Source code does not comply with standard programming guidelines

57


1. Select a module from your project and perform a code review session to identify

bugs in the code. List all bugs found and rectify the bugs accordingly.

2. Student will analyze the Software Design Document (SDD) and produce the test

cases for unit testing. Students may apply several different techniques learnt

during the lecture sessions. Accordingly, the test cases will then be updated in

Software Test Description (STD).

3. Student will execute the unit testing and report any bugs in the bugs report

document.

4. Students will fix any bugs founds and may proceed with the next coding phases.

8.2.1 EXAMPLE OF UNIT TESTING TECHNIQUES

The specification for a software system for validating the salary payment

according to salary scale as stated in a piece of code below:

Switch (salaryScale) {

Case 45 : basicSalary = 2000;

break;


break;


break;

Default : system.out.println (“Wrong salary

scale”);

System.exit(0);

}

58

1. Boundary Analysis technique

What is boundary analysis? A boundary value analysis is when you look at a condition to

understand its limitations. It's important to recognize that boundaries have lower limits

as well as upper limits. I point out lower boundaries because I often see testers focusing

on maximum capacity but forgetting about lower boundaries. A simple boundary analysis

would be the review of a single data entry field to determine its limits. In such an

example, a lower limit might be that the data entry field is a required field so tests might

include attempting to leave the field blank or using the space bar to create an entry. The

upper limit tests might include attempting to enter more characters than the field accepts

and attempting to exceed the field value in through any method available.

Figure 8.1: Boundary analysis for salary scale

Test Id Test cases for

salary scale

Boundary Analysis Expected Result

1 44 “wrong salary scale”

2 45 2000

3 46

45

“wrong salary scale”

4 48 49 “wrong salary scale”

5 49 2500

6 50


7 51 “wrong salary scale”

8 52 3000

9 53

52


Figure 8.2: Test cases/scripts for salary scale

45 49 52

salary scale 44 46 48 50 51 53

59

2. Path Coverage technique

Path coverage is where the different paths in a software module are created by the choice

in conditional statements such as IF-THEN-ELSE, SWITCH and DO WHILE. The flow

chart is created and the test cases are created based on the conditional statement. Based

on flow chart created in Figure 8.3, there are four paths discovered including:

Path 1 : (1,2,9); Path 2 : (3,4,9); Path 3 : (5,6,9);Path 4 : (7,8,9). Figure 8.4 depicts the

possible test cases to be used for testing the salary scale.

Figure 8.3: Path coverage analysis for salary scale

Test ID Test Case Action Expected Result

1 Path 1 Input salary scale = 45 Basic salary = 2000



4 Path 4 Input salary scale = ▲ “wrong salary scale”

Figure 8.4: Test cases for salary scale

60

3. Equivalence Partition technique What is equivalence partition? Equivalence partitioning is when you examine multiple values to determine where tests overlap or additional tests would not likely provide more value. Well, equivalence partitioning helps you group tests in order to select a manageable number of tests (TechTarget, 2010).

Test ID Salary scale Partition Tested Expected output 1 43 Salary scale < 44 “wrong salary scale” 2 47 46 < salary scale < 49 “wrong salary scale” 3 50 49 <salary scale < 52 “wrong salary scale” 4 53 Salary scale > 52 “wrong salary scale”


1. What is unit testing? 2. What is the biggest problem of white box testing? 3. White box testing can find missing items as well as problems. True or False? 4. If a programmer was told that he could name his variables with only eight

characters and the first character had to be capitalized, would that be a standard or a guideline?

5. What are entrance and exit criteria?

45 49 52

salary scale< 44 46 < salary scale < 49 49 < salary scale < 52 salary scale >52

61

LAB 9: INTEGRATION / SYSTEM TEST


Prepare, execute and conduct the integration / system test.

Record the integration / system test in the test cases framework.

9.1 INTRODUCTION

The purpose of integration/system testing is to demonstrate that the module

interact together in a correct, stable and coherent manner prior to system testing. Testing

should be performed against the functional requirements by using black box testing

techniques. Test case design should demonstrate correct interfacing and interaction

between modules, but should avoid any duplication of unit testing effort.

The precise definition of a module will depend on the approach to the design and

the technology employed, for example:

• A module in an application developed using a procedural programming language

could be represented by a closely related group functions or procedures that perform a

well-defined service and communicate with other component module via strictly

defined interfaces.

• A module with other application developed using an object oriented programming

language could be represented by a collection of objects that perform a well-defined

service and communicate with other component module via strictly defined

interfaces.

62

Integration/system testing is typically a black box testing activity, based primarily

on the functional and data requirements expressed in the specification documentation.

Example of area typically covered in integration/system testing includes:

• Invocation of one module from another interoperating module.

• Correct transmission of data between interoperating modules.

• Nonfunctional issues such as the reliability of interfaces between modules.


1. Student will analyze the Software Requirement Specification (SRS) and produce

the test cases for integration/system testing. Students may apply several different

techniques learnt during the lecture sessions. Accordingly, the test cases will then

be updated in Software Test Description (STD).

2. Student will execute the integration/system testing and report any bugs in the bugs

report document.

3. The students will proceed to coding phase with addition of one functions is

completed.

9.2.1 EXAMPLE TEST CASES FORM OF INTEGRATION/SYSTEM TEST

System : Date Component Cycle : 1(happy scenario)

Module/Unit : Format Date Rule [SRS_REQ_100] Date : 10.11.2005 Tester : Abd. Rahim Bin Ahmad

Test No Action Expected Result Actual Result (OK/NOK)

SRS_REQ_101

To test the types of date

Test_BDC_101

Key in the type of date according the format given.

The format of date is accepted and proceed to

63

-eg. dd/mm/yy dd-mm-yy dd.mm.yyyy

the next process

Test_BDC_102

Display the type of date - eg. - Birthday of Yang dipertuan Agong, - Hari Raya Puasa, - Tahun Baru Cina - Deepavali

The system is able to display the selected type of date.

SRS_REQ_102

… … …

…

…

… …

9.3 SELF REVIEW QUESTION 1. What is integration testing? 2. What is the hidden strategy behind the integration testing? 3. Name several advantages of executing the white box testing? 4. What is code coverage? 5. All software has a user interface and therefore must be tested for usability. True

or False?

64

LAB 10: USER ACCEPTANCE TEST


Prepare, execute and conduct the user acceptance test.

Record the user acceptance test in the bugs report framework.

10.1 INTRODUCTION

Software is written to be used. That sound pretty obvious, but it’s sometimes be

forgotten with the rush to design, developing and testing a complex product. More time

and effort are utilized on the technology aspects of writing the code that the development

team ignores the most important aspect of software – that someone will eventually use

the software product. It really doesn’t matter whether the software is embedded in a

microwave oven, a telephone switching station, or an Internet stock trading website.

Eventually the bits and bytes bubble up to where a live person will interact with it.

Usability is how appropriate, functional and effective that interaction is.

The purpose of user acceptance testing is to confirm that the testing process meets

its business requirements as an indicator that the system works correctly and is useable

before it is formally “delivered” to the end user. User Acceptance Test is conducted by

one or more user representative with the assistance of the Test Team.

65

Typically the user acceptance testing involves the test analyst selecting a

representative subset of the system and asking the user representative to execute them.

The selection of these Test Cases normally based upon:

• Discussion with the user representative

• Review of the requirements to identify any particularly important areas or

functions that should be verified.

User Acceptance Test should employ the Black Box approach and should make

use of thread testing techniques to verify the high level business requirements of the

system. In practice, the user representative will involve in testing process by performing

typical tasks that they would perform during their normal usage of the system.


1. Student will analyze the Software Requirement Analysis (SRA) and produce the

test cases for user acceptance testing. Students may apply several different

techniques learnt during the lecture sessions. Accordingly, the test cases will then

be updated in Software Test Description (STD).

2. Student will execute the user acceptance testing and report any bugs in the bugs

report document.

3. The students will proceed to coding phase with addition of one more functions is

completed.

66

10.2.1 EXAMPLE TEST CASES FORM OF USER ACCEPTANCE TEST

System : Date Component Cycle : 1(happy scenario)

Module/Unit : Format Date Rule [BDC_1] Date : 10.4.2006 Tester : Syamin Bin Hamzi

Test No Action Expected Result Actual Result (OK/NOK)

To test ability to define the types of dates

Test_BDC_101

Key in the type of date according the format given. -eg. dd/mm/yy dd-mm-yy dd.mm.yyyy

The date is accepted and proceed to the next process

Test_BDC_102

Display the type of date - eg. - Birthday of Yang dipertuan Agong, - Hari Raya Puasa, - Tahun Baru Cina - Deepavali

The system is able to display the selected type of date.

…

…

…

…


1. Why does knowing how the software works influence how and what you should test?

2. What are a few advantages of having a beta test program for your software? 3. What are a few cautions to be considered with a beta test program? 4. You can perform the black box testing without a product specification or

requirements document? True or False? 5. Describe the pesticide paradox problem and how does the problem be resolved by

bringing in new people to look at the software?

67

LAB 11: QUALITY PLAN


Understand the concept and the significance of software quality checklist.

Understand and able to prepare for review session

11.1 INTRODUCTION

The purpose of quality plan is to ensure the needed preparation have been

established before the actual implementation of the review session is conducted. An

invitation letter will be sent to all parties involved in the review session. The letter will

specifically state the date, time and venue of the session. Beforehand, we have to identify

the responsible individual and assign the role (e.g the moderator, recorder, reader, author

and inspector to handle the review meeting) accordingly.

Basically there are two types of review session. They are technical and

quality/formal review. The technical review focuses on the functionality of requirement,

architecture and design while the quality review focuses on the software project

documents and how they were written.

The formal review (also called inspection) has a six step process as stated below:

Planning

Identifies work product to be inspected and sets the inspection schedule.

Overview

Optional phase where team members who are not familiar with the work

product will receive an orientation session.

68

Individual Preparation

Team member inspect the work individually looking for detects in the

work product.

Review Meeting

Inspection team members met to discuss possible defects in the work

product.

Rework

The work product is revised to conform to requirements and specifications

Follow-up

The rework is verified, final inspection data is collected and summarized

and the inspection is officially closed.


1. Discuss among the group members to a quality plan.

a. Prepare and produce the invitation letter for conducting a review sessions.

b. Identify the responsible person to involve in the review session.

c. Produce a standard form to record the discussion matter in the review

session.

2. Students should already complete the Software Test Description (STD) and

Software Test Result (STR) baseline and are able to establish the Test Readiness

Review (TRR) and the internal Software Tester / Quality Auditor will state all the

arising matters in the Review Report Form (RRF).

69

11.2.1 EXAMPLE OF CHECKLIST FOR SOFTWARE REQUIREMENT INSPECTIONS

1. Clarity Are the requirements clear and unambiguous? 2. Standards Have all the requirements standards been followed? 3. Completeness Are all the requirements complete? 4. Level Of Detail Are the requirements free from design? 5. Consistency Are requirements consistent?

Are data structures and functions named and used consistently?

6. Functionality Are functions correctly specified? Are input and outputs clearly specified? Are functions logically independent and highly cohesive?

7. Performance Are the performance requirements for timing, memory and resource utilization clearly defined?

8. Testability Are requirements testable and verifiable? 9. Feasibility Can each item be implemented with the available technique,

tools, resources, and personnel and with it the specified cost and schedule constraints?

10. Tractability Are the requirements uniquely identified that they will be able to be traced back from code?

11. Modifiability Are the requirements uniquely structured such that any necessary changes to the requirements can be made easily, completely and consistenly?

70

11.2.2 EXAMPLE OF INVITATION LETTER FOR REVIEW SESSION

Name of writer and address Name of the recipient and address Date salutation, Title of letter (invitation for review) State the purpose of invitation 2. State the venue, date and time of review will be held. 3. State the closing remarks briefly. Yours sincerely, _________________ ( Name of writer ) position and company

71

11.2.3 EXAMPLE OF REVIEW FORM

QUALITY/TECHNICAL/TESTING REVIEW REPORT FORM

Section 1 - Project/Deliverable Details

System Name: Business Date Component Ref. No: (If any)

SDD/BITP3213/S2/G1/2004

Deliverable Name: Software Design Document (SDD) Version: 0.1

Author Id/Name: (Group Name)

Pyan Dot Com Sdn Bhd

Section 2 - Reviewer/Review Details

Reviewer Id/Name: Reviewer’s Role:

Imatera Software Corporation

Quality √ Technical √ Testing

Review Number Review Date Review Outcome

17 Okt 2005 Issues Found (Refer to Review Issues Form)

No Issues Found or Outstanding

1 SDD - format, template - description

6

2

3

4

Section 3 - Sign Off Details (Only when all issues have been resolved)

Signature: Date: 22 Okt 2005

Position: Project Manager

72

PROJECT REVIEW ISSUES FORM

System Name: Business Date Component (BDC)

Deliverable Name: Software Design Document (SDD)

Vers Nbr: 0.1

Date/Nbr of Review: 22 Okt 2005 Page Nbr:

Reviewer Id/Name: Imatera Software Corporation

Review Type: Quality & Technical (Qual, Tech, User, Test or CW)

If there are no issues found during the review, please note accordingly.

Nbr Page/Ref Nbr

DESCRIPTION OF PROBLEMS Outcome A - Accepted R - Rejected

Check Sign.

1, 2, 3 Page numbers should be in roman

4 1.3 whole statement out of content.

6 Figure 1.0 Use case are noun

8 Format Date Rules – Font size different from other paragraph

10 – 21 4.1.1 Algorithm should be simple english

2 Numbering for table of contents is not accordingly

73


1. Why are there testing costs associated with the costs of conformance? 2. The test team is responsible for quality. True or False? 3. Why would being called a QA Engineer be a difficult title to live up to? 4. Why is it good for a test or quality assurance group to report independently to

senior management? 5. Name three ways that you become involved in testing computer software before it

is released to the public.

74

LAB 12: REVIEW MEETING


Conduct the document review meeting.

Record all the issues rose in minutes meeting.

12.1 INTRODUCTION

The review session is very significance to software developer in order to ensure

that they follow the standard stated in product evaluation plan SDP. The purpose of

review is to detect the problems in software work products as close to their insertion

point as possible.

As defined by IEEE (1990), a review process is:

“A process or meeting during which a work product, or set of work products, is

presented to project personnel, managers, users, customers or other interested parties for

comment or approval”

Objective of reviews can be divided into two categories, a direct objective and an

indirect objective. Direct objective is to detect the analysis and design errors as well as

subjects where corrections, changes and completions are required with respect to the

original specifications and approved changes, to identify new risks and to approve the

analysis and design product. Approval allows the team to continue to the next

development phase.

Indirect objective is to provide an informal meeting place for exchange of

professional knowledge about development methods, tools and techniques.

75

The traceability document is to classify the correspondent between the test and the

system requirements. We will able to keep track the similarity and scope of requirement

as guidance to produce the test case.


1. Students will form in group to conduct the review meeting session together with

external Software Tester / Quality Auditor.

2. The external Software Tester / Quality Auditor will audit all the documents

provided which is from the Software Development Plan (SDP), Software

Requirement Analysis (SRA), Software Requirement Specification (SRS),

Software Design Document (SDD), Software Test Plan (STP), Software Test

Description (STD) and the Software Test Result (STR) and the completed system.

3. The software tester / quality auditor will list all the issues in the Review Report

Form (RRF).

4. Students will fix the problems and update the system to ensure that all

requirements are fulfilled before final release of the product.

76

12.2.1 EXAMPLE MINUTE OF MEETING FOR DOCUMENTATION

REVIEWED

KOLEJ UNIVERSITI TEKNIKAL KEBANGSAAN MALAYSIA

FACULTY OF INFORMATION AND COMMUNICATION TECHNOLOGY

MINUTES OF MEETING FOR DOCUMENTATION REVIEW

NO. 1 2005

Date : _____________________

Time : _____________________

Venue : _____________________

Member’s name : Position

1. ____________________ ____________

2. ____________________ ____________

3. ____________________ ____________

Agenda Action

1. Intro from chairman

…

Name of person who

will take action

2. Confirmation Minutes of Meeting

…

3. Matter arising

…

4. Action plan made to improve the documentation

…

5. Other matter arising

…

Delayed of Meeting

Prepared by: Approved by:

…………… …………….

Name and position Lecturer name

77


1. What are key elements that make the formal reviews work? 2. Beside being more formal, what is the big difference between inspection and other

types of reviews? 3. Should you adopt the code review checklist from this chapter as your team’s

standard to verify the code? 4. Why is defining the software’s quality and reliability goals an important part of

test planning? 5. Why quality is free?

78

REREFENCE

Dr. Daniel Gallin, 2004, Software Quality Assurance, From Theory to Implementation.

England : Addison Wesley

Edward Kit. 1995. Software Testing in the Real World. New York : Addison Wesley.

Elfriede Dustin, 2003, Effective Software Testing, 50 Specific Ways To Improve Your

Testing. United States : Pearson Education, Inc.

Elfriede Dustin, Jeff Rashaka, and John Paul. 2000. Automated Software Testing: Introduction, Management and Performance. Maryland : Addison-Wesley.

Ernest Wallmuller, 1994, Software Quality Assurance. Great Britain : Prentice Hall

Glenford J. Myers, 2004, The Art Of Software Testing. Canada : John Willey & Sons Inc.

Hung Q. Nguyen, 2003, Testing Applications on the Web. Indianapolis : Wiley

Publishing, Inc.

Ron Patton, 2006, Software Testing, Second Edition. United States : Sams Publishing

Thomas Muller et. al., (2007) ISTQB Certified Tester: Foundation Course in Software Testing. International Software Testing Qualification Board.

79

BIOGRAPHY

Dr. Mohd Khanapi Abd Ghani

• PhD (Biomedical Computing) Coventry University U.K.

• MSc (Computer Science – Real-Time Software Engineering) Universiti Teknologi Malaysia

• BSc (Computer Science)

Universiti Teknologi Malaysia • Diploma (Computer Science)

Mara Institute of Technology

Dr. Mohd Khanapi Abd Ghani has been in software industry for 16 years and has a

variety of experiences in software projects. He begins its career as a programmer until

appointed as a Project Manager in several companies such as Heitech Padu Berhad,

Permodalan Nasional Berhad, Bank Simpanan Nasional, Dewan Bahasa & Pustaka which

had experienced in developing the software project with Government – Ministry of

Health, PNB and Subsidiaries, BERNAS, Lembaga Pertubuhan Peladang, Universiti

Malaysia Sabah, HCL info System (India) and Accounting Firm – Razalee & Co. The last

attachment with software industry was the Medical Online Sdn. Bhd. as a Software

Development Manager (development of MSC Telemedicine Flagship Application

project) before change the profession into academic line as a Senior Lecturer at UTeM.

Mohd Suffian Sulaiman

• MSc (Computer Science – Real-Time Software Engineering) Universiti Teknologi Malaysia – Thompson Universite

• BSc (Computer Science)

Universiti Teknologi Malaysia

80

Juliana Jaafar

• MSc in Advanced Computer Science and IT Management

University of Manchester, UK

• BSc in Computer Scince, Minor in Mathematics

University of North Carolina at Charlotte, USA

Juliana Jaafar has been in web based software development industry for almost 8 years

and has moved up the career ladder as a Senior Software Engineer. She has profound

background in Object Oriented Analysis/Design and Java development and its

technologies, with experience in full life cycle of software development, and skilled in

problem statements to well-documented designs.

She started her career as an academician in UTEM, Melaka in 2010 and currently pursues

her career as a Lecturer in UiTM, Shah Alam.

module for software testing

Documents

self review question

table of contents lab

software testing

software design

software test plan

software test description

test result

project planning