Product Quality?

What is product quality?

• Quality, simplistically, means that a product

should meet its specification

–The software product should deliver the

required functionality (functional

requirements) with the required quality

attributes (non–functional requirements)

Problematic for software systems– Some quality requirements are difficult to specify in an

unambiguous way

– Software specifications are usually incomplete and often

inconsistent

– Quality attributes are frequently conflicting and increase

development costs, so there is a need for weighting and

balancing.

– Different users may have different opinions about what

constitutes software quality.

Quality attributes

• Correctness : the degree to which the software product

performs the required functions accurately.

• Reliability is the degree to which the software behaves well

and in the way the users expect.

• Robustness implies that the software is unlike to fail or

break.

• performance relates to the response time of the software

system.

• Usability relates to the user friendness of the software.

Quality attributes

• Maintainbility :Correcting software errors and deficiencies

• Scalibility : Ease with which the software can be scaled up or

evolved in response to the growth in demand for its functionality.

• Reusability : Defines the level to which the software components

can be used for construction of other products.

• Portability : Can software run on various hardware/software

platforms without any modifications or after undergoing minor

customization or parameterization ?

Quality control

Quality control is mostly about testing the quality of a product ( to eliminate problems )

Quality assurance is about building quality into the product.

Testing – V model

“V” ModelEach phase has corresponding test or validation

counterpart

Requirements Analysis

System Design

Program Design

Implementation

Unit Test

Integration Test

Acceptance Test

Sawtooth Model (Brugge)

Requirements Analysis

System Design

Program Design

Implementation

Unit Test

Integration Test

Acceptance TestDemo Prototype 1 Demo Prototype 2

Quality is guaranteed at each project stage.

Unit testing The most ‘micro’ scale of Testing

A unit = smallest testable software component Objects and methods Procedures / functions

Performed by Programmer A tester can help. Requires detailed knowledge of the internal program design

and code.

The units are tested in isolation.

Ensures the component is working according to the detailed design/build specifications of the module.

Also known as component, module, or program testing.

Unit Test ( white box)

• White-box testing– Sometime called structural testing– Tester studies the code and decide on data inputs to exercise

all program statements (not all path combinations). • Test coverage measures ensure that all statements have

been executed at least once– Derivation of test cases according to program structure.

Knowledge of the program is used to identify additional test cases.

Also suitable for design models and specification documents (walkthrough and inspections)

Integration Testing

Testing of more than one (tested) unit together to determine if they function correctly.

Focus on interfaces Communication between units

It is done using the integration test design prepared during the architecture design phase.

Helps assembling incrementally a whole system, ensuring the correct ‘flow’ of data from the first through the final component.

Done by developers/designers and testers in collaboration

Also called Interface Testing or Assembly Testing.

System testing

Testing the system as a whole - Black-box type testing that is based on overall requirements specifications; covers all combined parts of a system.

Ensures that system meets all functional and business requirements.

Focus Verifying that specifications are met Validating that the system can be used for the intended

purpose

The system test design is derived from the system design documents and is used in this phase.

It can involve a number of specialized types of tests to check performance, stress, documentation etc. Sometimes testing is automated using testing tools.

Done by Independent testing group

System testing

• Black-box testing (testing to specifications)

– An approach to testing where the program is considered as a ‘black-box’

, taking some inputs and produces some outputs.

– Tester does not know or choose to ignore the internal workings of the

program.

– The program test cases are based on the system specification

– Test planning can begin early in the software process

– Testing is done by feeding the test unit with data inputs and verifying

that the expected output is produced.

– Also applicable to constraint testing ( performance and security ) and

missing functionalities.

Acceptance testing

To determine whether a system satisfies its acceptance criteria and business requirements or not.

Similar to System testing in that the whole system is checked, but the important difference is the change in focus.

Done by real business users.

It enables the customer to determine whether to accept the system or not.

Also called as Beta Testing, Application Testing or End User Testing.

Approach Should be performed in real operating environment . Customer should be able to perform any test based on their

business processes. Final Customer sign-off.

Testing techniquesIt is never possible to test for all possible data inputs or code execution paths.Testing techniques can be classified according to 5 criteria :

• Visibility• Automation• partitioning• Coverage• scripting

Testing techniques

VisibilityAutomation

Partitioning -Equivalence partitioning

• Groups data inputs (and implicitly , data outputs)into partitions constituing homogenoues test targets ( testing with one member implies test with other member in the same partition )

• Supported by Black box testing

Boundary value

• Additional data analysis technique• Bounadary values are extreme cases withing

equivalence partitions.

– Ex: Partition of integer from 1 to 100. Bounadry value analysis recommends tests to be done on the values on the edges that is : -1 , 0 , +1 as well as for 99,100, 101

Coverage

Determine how much code is going to be exercised by a white box

test

– Operation coverage

Ensure that each operation in the code is exercised at least once by the

white box test

– Path coverage• Numbering possible execution paths ( infinite in large program ) in the

program• Exercising them one by one.• For large program , choose the most critical and frequently used ones.• Testing can be manual or automatic

Manual testing• Human tester interracts with the application

under test conducts , according to a predefined test script and observe the results.

• Test script defines step-by-step testing actions and expected outcomes.

• Use cases are used to write test scripts. Problems:

Freuqently output is not presented to the creenLive data are not predefinedExpensive

Automated testing

• Use software testing tools to execute large

volumes of test without human participation

• Tools can produce post test reports

• Automated testing can be divided into :

Regression testing

Exercising testing

Regression testingRepetitive execution of the same test scripts on the

same data to be sure that the system has not been broken by successive changes to the code (changes not related to the tested functionality) .

Execution of the script at scheduled test times.

Exercising testing

• Tool generates automatically and randomly

various possible actions instead of the user.

• Mad user hitting any possible key on keyboard ,

selecting any possible menu item , ….etc.

Test planning• Part of the quality management plan.• Defines testing schedule, budget , tasks (test

cases) and resources.• Test Plan include code and other project

artifacts testings.• Specify which test cases should be conducted• Human and material ressources should be

allocated.• Test database created and test software tools

installed

26

Testing Approaches

• We will look at a small sample of approaches for testing

• White-box testing– Control-flow-based testing– Loop testing– Data-flow-based testing

• Black-box testing– Equivalence partitioning

27

Control-flow-based Testing

• A traditional form of white-box testing• Step 1: From the source, create a graph describing

the flow of control– Called the control flow graph– The graph is created (extracted from the source code)

manually or automatically• Step 2: Design test cases to cover certain elements of

this graph– Nodes, edges, paths

28

Example of a Control Flow Graph (CFG)

if (x+y < 100) s:=s+x+y; else d:=d+x-y;}

1

2

3

45 6

7

8

s:=0;d:=0;while (x<y) {

x:=x+3;y:=y+2;

29

Statement Coverage

• Basic idea: given the control flow graph define a “coverage target” and write test cases to achieve it

• Traditional target: statement coverage– Need to write test cases that cover all nodes in

the control flow graph• Intuition: code that has never been executed

during testing may contain errors– Often this is the “low-probability” code

30

Example

• Suppose that we write and execute two test cases

• Test case #1: follows path 1-2-exit (e.g., we never take the loop)

• Test case #2: 1-2-3-4-5-7-8-2-3-4-5-7-8-2-exit (loop twice, and both times take the true branch)

• Do we have 100% statement coverage?

1

2

3

45 6

7

8

T F

31

Branch Coverage

• Target: write test cases that cover all branches of predicate nodes– True and false branches of each IF– The two branches corresponding to the condition

of a loop– All alternatives in a SWITCH statement

• In modern languages, branch coverage implies statement coverage

32

Branch Coverage

• Statement coverage does not imply branch coverage– Can you think of an example?

• Motivation for branch coverage: experience shows that many errors occur in “decision making” (i.e., branching)– Plus, it subsumes statement coverage.

33

Example

• Same example as before• Test case #1: follows path 1-

2-exit • Test case #2: 1-2-3-4-5-7-8-

2-3-4-5-7-8-2-exit • What is the branch

coverage?

1

2

3

45 6

7

8

T F

34

Black-box Testing

• Unlike white-box testing, here we don’t use any knowledge about the internals of the code

• Test cases are designed based on specifications– Example: search for a value in an array • Postcondition: return value is the index of some

occurrence of the value, or -1 if the value does not occur in the array• We design test cases based on this spec

Test cases Design Project Name:

Test Case Template

Test Case ID: Fun_10 Test Designed by: <Name>

Test Priority (Low/Medium/High): Med Test Designed date: <Date>

Module Name: Google login screen Test Executed by: <Name>

Test Title: Verify login with valid username and password Test Execution date: <Date>

Description: Test the Google login page

Pre-conditions: User has valid username and passwordDependencies:

Step Test Steps Test Data Expected Result Actual Result Status (Pass/Fail) Notes

1 Navigate to login page User= example@gmail.com User should be able to login

User is navigated to

Pass

2 Provide valid username Password: 1234 dashboard with successful

3 Provide valid password login

4 Click on Login button

Post-conditions: