sw testing mam seminar

7/31/2019 SW Testing MAM Seminar

1/34

Mohammed A. Ramadan

Computer Science Department

Modern Academy in Maadi

Software Testing


2/34

Agenda

22-Apr-12Software Testing2

Testing -What Is it ?

Testing lifecycle.

Types of Testing

Test Strategy

Orthogonal Arrays Testing Strategy

(OATS)


3/34

Testing What Is it ???

22-Apr-12Software Testing

An activity in which a system or component is executedunder specified conditions, the results are observed orrecorded, and an evaluation is made of some aspect of thesystem or component

From IEEE Std 610 Software Engineering Terminology:test

The process of executing computer software in order to

determine whether the results it produces are correct

Glass 79

The process of executing a program with the intent of findingerrors, Myers 79

Testing is the measure of software quality, Hetzel 85

3


4/34

Objectives of Testing


What are we looking for?

Errors, faults, failures and incidents. Defect, Problem,Incident, Anomaly, Variance, Inconsistency, Feature,Bugs !

A test is the act of exercising software with test cases withan objective of Finding failure Demonstrate correct execution

Goal To find if the product is

Not operating as Intended


5/34

Matrix of Specification andSoftware


Specification Software

Should do Doesnt

Shouldnt do Does

No mention but should Doesnt

No mention Does

Non-Functional Hard to use,Complex, slow


6/34

Source of Defects


56%27%

7%

10%

Requirements

OtherCode

Design

Source: Software Testing in the Real

World Edward Kit, 1995


7/34

Cost of Repair


The sooner you find bugs, the cheaper

Defec

tsremoved

PROJECT PHASES

Coding Testing Maintenance

Repair

Cost 1

Repair

Cost 4

Repair

Cost 10


8/34

Testing lifecycle


Development Lifecycle

ProjectPlanning

RequirementsCapture

Implementation

Build 0 Build 1 Build 2

Test Lifecycle Plan

Test

Analysis &Design

Design

Test

Setup Test

Bed

ExecuteTest

EvaluateTest

Defect Tracking/Enhancement Request

Testing in the V Model Approach


9/34

Testing lifecycle ,contd


Requirements

Capture

Analysis

Test planning &Scenario Design

Test CaseDevelopment

Test Execution

Test ResultAnalysis

Test CycleClosure

Defect FixingCycle

Defects

New Version

The Scenarios Design and the Test Case Developmentcould normally start in parallel with the DevelopmentCycle.

Test Execution Synchs up with the Development Cycleduring the functional Testing Phases


10/34

Testing in the V Model Approach


Verification Validation

Requirements

(Business Needs)

Requirements

(Business

Functions)Design

Code

AcceptanceTesting

System Testing

IntegrationTesting

Unit Testing

Writetests

Run

tests

The test phase of the lifecycle which ensures thatthe end product meets theusers needs.

All QC activities through outthe life cycle that ensure thatinterim deliverables meet theirinput specification.


11/34

Types of Testing


SoftwareSoftware

Black Box Testing

Static

Dynamic

White Box Testing

Static

Dynamic


12/34

Types of Testing


Black Box

White Box

Static

Dynamic

Dynamic

Static

Specifications

Test to Pass

Test to Fail

Regression

EquivalencePartitioning

Boundary Conditions

Invalid Data

State Transitions

Data Testing

Data Coverage

Data Flow

Code Coverage

Data Reference Error

Computation Errors

Control Flow Errors

Input/Output Errors

Review & Analysis

Formal Review

Peer Review

Walkthrough

Inspections

Coding Standards

Generic Code ReviewLine Coverage

Branch Coverage

Condition Coverage

Unit Testing

Structure Testing


13/34

Black Box Testing


Static Black Box TestingTesting the specification

Dynamic Black Box TestingBehavioral Testing

Testing a running program without anyknowledge of how the software behaves.

Software


14/34

White Box Testing


Static White Box Testing

Code review and analysis

(1)Formal Reviews (2)Peer Reviews (3) Walkthroughs

Dynamic Black Box Testing Structural testing

using information from what the code does and how it worksto determine what to test, what not to test.

Unit Testing

Data Coverage

Divide the code into data and its states

Data flow

Track a piece of data completely through the software

Code coverage

Program and Line Coverage

Branch Coverage

Condition Coverage

Software


15/34

Other Types of Testing


User Interface Testing Test the User Interface for operability, ease of use

Performance

Testing the performance of the computations

Load Testing the performance of the system under varying load

conditions

Stress

Testing the performance and accuracy of system under

resource crunch or any other kind of stress Volume

Testing the accuracy of system when system has a highvolume of data


16/34

Other Types of Testing ,contd


Configuration Testing the system under various hardware/software configurations

Compatibility

Testing the system for ability to interact with other system

Installation Testing the installation of system under one or more configurations

Documentation

Testing the system for documentation clarity and ease ofapprehension

Ad-Hoc Testing

Randomly testing the system for errors with no background on thesystem


17/34

Why Have a Test Strategy?


Test Strategy Describes the overall approach that the test team will

use to test, in each phase and in all.

A Test Plan The game plan to implement

what is to be tested, when, how, how long and till what happens?

Effort Estimation Test Strategy helps the overall test effort estimation


18/34

What to Consider?


Product Requirements

Nature of product

Product Lifecycle

Milestones

Repeatability considerations

Environment

Feedback

Cost


19/34

Optimal Testing


Number ofMissed Bugs

Optimal Amountof Testing

Cost of Testing

Over Testing

Under Testing

Amount of Testing


20/34



21/34

Orthogonal Arrays Testing Strategy(OATS)


(OATS) is a systematical, statistical way of testing pair-wise interactions by deriving suitable small set of testcases from a large number of scenarios.

can be used to reduce the number of test combinationsand provide maximum coverage with a minimum numberof test cases.

(OATS) utilizes an array of values representing variablefactors that are combined pair-wise rather thanrepresenting all combinations of factors and levels.


22/34

Overview


Orthogonal arrays are two dimensional arrays of

numbers by choosing any two columns in the array youreceive an even distribution of all the pair-wisecombinations of values in the array.


23/34

Overview


An L9

(34) orthogonal array with 9 runs,

4 factors, 3 levels, and strength of 2.

An L9 (34 )

orthogonal arraywith 9 runs,

4 factors, 3levels, andstrength of 2.

Runs: the number of rows in the array. This directly translates to the number of testcases that will be generated by the OATS technique.

Factors: the number of columns in an array. This directly translates to the maximumnumber of variables that can be handled by this array.

Levels: the maximum number of values that can be taken on by any single factor. An

orthogonal array will contain values from 0 to Levels-1.

Strength: the number of columns it takes to see each of the Levelstrengh possibilitiesequally often.

Orthogonal arrays are most often named following the pattern L runs (LevelsFactors ).


24/34

Simple OAT Example


Consider three parameters : A, B, and C.

Parameters A, B, and C each have positive values :1, 2, and 3.

Testing all combinations of the three parameters would

involve executing a total of 27 test cases.


25/34

Simple OAT Example (contd)


With the way programming works, a fault will most likelydepend on the values of two parameters, not three.

The fault might occur for each of these three test cases:

A=1, B=1, C=1, A=1, B=1, C=2, and A=1, B=1, C=3.

Not necessary to run all 27 scenarios, 9 test cases issuffice.

The 9 scenarios outlined in Table 1 (on the next screen)address all possible pairs within the three parameters.


26/34

-Parameters


All possible pair-wise combinations between parameters Aand B, B and C, and C and A .


27/34

Example 2


Orthogonal Array has 3 columns, representing 3 variables.

Utilize a three level-alphabet Red, Green, and Blue.

Next, organize combinations into a table (next slide).


28/34

Example 2 (contd)


All Combinations for Three Variables of Three Levels Each

A B C

1 Red Red Red

2 Red Red Green3 Red Red Blue

4 Red Green Red

5 Red Green Green

6 Red Green Blue

7 Red Blue Red

8 Red Blue Green

9 Red Blue Blue

10 Blue Red Red

11 Blue Red Green

12 Blue Red Blue

13 Blue Green Red

14 Blue Green Green

15 Blue Green Blue

16 Blue Blue Red

17 Blue Blue Green

18 Blue Blue Blue

19 Green Red Red20 Green Red Green

21 Green Red Blue

22 Green Green Red

23 Green Green Green

24 Green Green Blue

25 Green Blue Red

26 Green Blue Green

27 Green Blue Blue


29/34

Example 2 (contd)


For each pair of columns, AB, AC, and BC, each pair of colorsappears exactly three times.

To save testing effort, the appearance of each pair has been reducedto once.

All-Pairs Array, Three Variables of Three Levels Each

A B C

2 Red Red Green

4 Red Green Red

9 Red Blue Blue

12 Blue Red Blue

14 Blue Green Green

16 Blue Blue Red

19 Green Red Red

24 Green Green Blue

26 Green Blue Green


30/34

Advantages of OATS


Utilizes variable pair combination

Provides uniformly distributed coverage

Useful for integration testing of software components

Useful for testing configurable options on an application

web page allowing font style

background color

page layout selections


31/34

Advantages of OATS


Conventional Test Cases:

Variables:3Input: 4

Possible cases: 81=34

Variables: 3

Input: 5Possible Cases: 243 = 35

Variables: 5

Input: 5

Possible Cases: 3125 = 55

Orthogonal Test Cases

Variables:3Input: 4

Possible cases: 9

Variables: 3

Input: 5

Possible Cases: 11

Variables: 5Input: 5

Possible Cases: 21


32/34

Disadvantages of OATS


Can only be applied at the initial stage of the product/processdesign system.

There are some situations whereby OA techniques are not

applicable, such as a processes involving influencing factorsthat vary in time and cannot be quantified exactly.

Can be overwhelming to use the first time; improves with use

on each project.

Arrays can be difficult to construct.


33/34


Questions


34/34

Thank You

22 A 1234

sw testing mam seminar

Documents