guidelines for improved testing in immediate releases

8/14/2019 Guidelines for Improved Testing in Immediate Releases

1/9

Guidelines for Improved Testing in Immediate Releases

Paper Title: Guidelines for Improved Testing in Immediate Releases. Name of the Author: Basil DiasEmail Address: [email protected]: 91-80-8520408 # 6228Location: Electronic City Tower 6Domain: E&IA (Interops)Account: HP-ODCArea: Process Improvement

Page 1 of 9


2/9


Abstract:

This article talks about the guidelines on improving the effectiveness of testing in the caseof stiff time-lined releases where the customer expects a very low response time to deliver a high quality product. This was a result of six-sigma initiative undertaken to identify the

pain areas felt in the testing areas by practitioners while working with customers having extremely tight schedules and to come up with a simple process to follow.

Introduction:

Testing has been a pain area and less focused phase in the software development life cycle.Today with teams working on many projects from client having extreme schedule pressure,sufficient time is not budgeted for testing. This could be because of having promo demos, productrollouts and to-be first in the market scenes. In these circumstances, it was becoming difficult tomeet the tight schedule in meeting these customer needs to the best of his expectations. Testingwas hampered due to low time for planning and the priority focus would be on to developingthose requirements thereby falling short on the quality front.

In this scenario, following a detailed process defined at the project initiation would affect thedelivery cycle time due to its elaborate procedures. The aim of this activity was to ensure that theshortcomings in the Testing area are arrested and also to examine or suggest any processimprovements especially in the Testing cycle, which can be adopted if feasible.

This document suggests a set of guidelines to be followed when implementing new requirementsfrom the customers with extremely tight schedule.

The process described in this document should be used under the following conditions.

Customer request with very high priority to implement a new feature that has to bedelivered within a short span of time (less than 4 weeks)

Because of the extreme time pressure, following the complete project cycle as specifiedin the process defined in the project plan is not feasible.

Customer requirements are unclear or requirements may change

Incomplete testing environment

The activity covered by this process can be a new enhancement or feature, a bug fix (for maintenance project) or a porting work or a change request.

This document describes the step-by-step procedure to be followed in the various phases startingfrom receiving the customer request till the release is made.

Page 2 of 9


3/9


Process:

Requirements Discussion & Customer Sign-off

After doing the preliminary analysis of the customer requirement, the project manager has to identify the developer(s) and the test engineer(s) depending on the complexity of the work and the amount of the effort that might be involved. If there is no dedicated testteam available for the project, any other developer should be chosen to act as the testengineer for this particular requirement. The developer SHOULD NOT do the testingactivity also. He/She should limit to only Unit Testing.

The test engineer should be involved during requirement analysis, schedule discussionand any other discussion related to the requirement till he/she gets enough clarity andunderstanding on the following things

Features to be testedTest EnvironmentBoundary conditionsRegression & Stress testing

The developer should share all the information he/she gathers related to the requirementand help the test engineer understand any critical things.

Indicate the customer the effort that might be involved for modifications in the design andimplementation if any requirement changes so that final and clear requirement can beobtained at the beginning of the implementation itself.

Maintain a Release Tracking Sheet for each of this activity starting from the requirementsdiscussion to the post-release phase. This could cover the following:

Feature Description.Testing Expectations.Schedule details.Test Cases.Open Issues.

Before starting with the implementation, a clear customer sign-off should be arrived at thefollowing things.

Features to be implementedFeatures to be tested (if complete testing environment is not available)Performance expectationsScheduleLimitations (if any)

There are three key track tables designed in a smarter way to ease the job of finding critical testcases and to perform minimal testing in case of time crunch.

As an example, let us take development of password protection of word document.

Page 3 of 9


4/9


Feature Table:Describes the features to be tested as per the discussion with the customer. Detailsregarding the functionality and the usage of the feature are to be described.

No Abstract of feature Description Completion date(dd/mm/yy)

Mode of Implementation i

1 Password protection Document saved in a local disk shouldbe password protected. Passwordshould be encrypted and stored in thelocal disk.Word should prompt the user for password before opening, if thedocument is password protected.[01/11/01]Private key encryption should be usedto store the password. [21/11/01]

10/01/02 Software,freeware for encryption/decryption.

Product Matrix :Discuss with the team members and expertise to identify the critical functionality of theproduct and the modules. Further this table can even be broken up for each module.

Functionality VsModule

File Embedded Networ k

Graphics Macros

New file creation 9 3 3Open existing file 9 9 9Save 9 3 3 9 3Cut & Paste 1 3 1 9OLE 9 9Help 3 3 3 3 3Font 9 3

Priority 40 21 19 42 6

Impact Matrix:Give the impact of the new requirement on the existing functionality and prioritize. Thiswill help to find the minimal test cases to be executed.

Password protection Priority

New file creation 9 1Open existing file 9 2Save 9 3Cut & PasteOLE

Page 4 of 9


5/9


Help 3 2Font

Implementation

The implementation of the requirement as well as the development of test cases shouldhappen simultaneously.

The design should be done in such way so that any requirement changes in future can beeasily accommodated in the code. Even the test engineer should design the test casesin such way that test cases can be modified easily to take care of the requirementchanges.

The test engineer and the developer should be in sync with respect to the test casesbeing designed. All the test cases (manual/automatic as applicable) should be designedin such way that it is possible to add new cases without much effort if there are anychanges in the requirements.

The project manager and the developer should review the test cases. If possible, it isrecommended to get the test cases reviewed by the customer also.

Reviewer should come up with the following table when the developer is developing thecode. Reviewer should have the product in mind and fill up the table. This table is usedas a tool to make the code more robust.

Mode of failure Severity Design consideration(yes/no)

For the example considered above, the above table can be developed as follows:

Mode of failure Severity Design consideration(yes/no)

Not able to decrypt the password 9 YesCorruption of password file 9 NoHacking the password when accessing the file fromnetwork

3 No

Password lost when file transferred from PC tofloppy/MyBriefcase

3 Yes

Testing

During the testing phase, the test engineer should try to add more test cases or improvethe test cases to find as many defects as possible. Using a good code coverage tool, itshould be ensured that all the critical code paths are tested completely.

All the test defects found (whether minor or major) should be logged.

Page 5 of 9


6/9


After the cycle of finding defects and fixing them is over within the available time, a testlog should be prepared which should contain the following things.

o Features tested and NOT testedo Test environment usedo Test case result for both automatic and manual type of test caseso Open issues/Known limitations

Release

The following items should be prepared to send to the customer.

Release work item.

Release note with the following detailso Features implementedo Open issues/defects

Test log with the details as mentioned in the previous section

Using the checklist it should be ensured that nothing is missed out. The checklist couldprimarily cover the following list of checks:

Has the signoff from the Customer been taken on the requirement? Have all testing limitation been conveyed to the customer? Have the test cases been reviewed by the PM/Developer? Have the test cases been reviewed by the Customer? Have all the test cases identified been executed? Has the Test Log Data been captured? Does the test result meet the testing expectations as decided in the sign-off? Have all open issues (if any) been identified and documented? Have all major defects identified been closed?

Post-Release

Once the release is made, if there is opportunity to improve the quality of the code as wellas the test cases, a review should be scheduled. After the required changes are doneand all the issues/defects are resolved, one more release can be made to the customer.

If the new requirement is to develop a prototype, after the release, the design and thecode should be reviewed. Especially, if lot of work-around solutions were consideredduring the development of the prototype because of the shortage of time, they SHOULDNOT be included for the final product without resolving all the issues and arriving atproper solutions.

Process Improvement Measure

At the end of each release, there is a need identifying and measuring the effectiveness of testing carried out using the guidelines as mentioned above.

Page 6 of 9


7/9


8/9


Page 8 of 9


9/9

guidelines for improved testing in immediate releases

Documents