Testing : A Roadmap

Mary Jean Harrold

Georgia Institute of Technology

Presented by : Navpreet Bawa

Introduction

What is Testing ?

Form of Verification

Why is testing Done ?

To do QA, helps construct high quality systems?

How is it done ?

Gathering info ,designing test-cases, executing s/w with those cases, examining results .

Introduction (contd.)

Some Statistics: Testing takes > 50%(Dev Time).

Types of Verification :

Dynamic Analysis: Execution of S/W (Testing)

Static Analysis: Model Checking

Advantages/Disadvantages

Advantages:

1)Can be done with ease.

2)S/W executed in target environment .

Disadvantages :

1)Doesn’t show absence of faults.

2)Can’t show certain qualities.

3)Results of test cases can’t be generalized.

Roadmap Where we are and where we need to go

Testing Component Based System

Component Provider :

The developer , he has access to source code.

Component User :

How component behaves in his configuration

Issue: Availability of source code

What needs to be done :

1.Provider gives coverage information with the component.

2.Develop tools that can use this information.

Testing Based on Precode Artifacts

What are precode artifacts?

Knowledge about the software before we

write code.

Artifacts

Specifications of

Requirement Design Architecture

S/W Architecture as a precode artifact

S/W Architecture :

1)Information about the components of the system

2)Interaction and the constrains on their interaction.

3)Helps us generalize family of systems

Advantages :

Helps to identify problems early

What needs to be done :

Make techniques which help in testing the

testability of the architecture.

Testing of evolving softwareRegression Testing

1)It ensures that the new software being added does not

break the existing code.

2)Used for testing an evolving software.

What needs to be done (Manage Test Suite)

Retest able S/W Optimize test cases

Write New Code Prioritize test cases

Redundant Test Cases

Demonstrating Effectiveness of Testing Techniques

1)Identify which testing technique is the most effective .2)Existing Techniques select test cases, based on behavior without regard to implementation of code.

What Needs to be Done: Research for

1)Criteria to select effective test cases .2) Techniques which select test cases based on faults. 3) Ways to show the analytical ,statistical and empirical

effectiveness of a test case.

Establishing Effective Processfor Testing

Testing process : Make test plan during requirementsgathering phase , execute during s/w dev phase.

Example : Microsoft testing plan

What needs to be done :1)Models to determine the effectiveness of the testingprocesses. (E.g.. Microsoft's test and build every nighttechnique, does this save costs later on?)2)Authors View : Combine various testing process and quality tools.

Using Testing Artifacts

Testing Artifacts : Residue of testing.

Types :

Execution traces, Coverage information, Variable

Values, Pass fail status .

What needs to be done :

1)Research to help use these testing artifacts.

2)Identify information needed at various steps of s/w dev. and see how to derive that from testing artifacts.

Other Testing Techniques

Testing input :

Today there are techniques to generate test

input for small systems.

What needs to be done :

We need semi-automatic / automatic techniques

which can do the same for large systems.

Methods and ToolsToday : Techniques take a long time to be

transferred to from theory to practice.

Prime Reasons : Absence of methods or tools

which help transfer technology.

What needs to be done :

1)Automatic generation of tools (w/o s/w engineer).

2)Need tools and methods which scale beyond small models and can be deployed in the industry.

Thank You