chapter 7c csta cstp test execution

SYSTEMS TESTING LIFE CYCLE – Test Execution

MANUAL

TEST EXECUTIONTest Planning

Test Preparation

Unit Testing

Integration Testing

System Testing

Acceptance Testing

Start

Test Closure

Test Execution

Test ExecutionContents

Build Verification Process Test Execution

Verify Analyze

Report Defect Bug Life Cycle

Test Execution Test execution procedures should be well

documented and clearly covered to each member of the test team prior to any scheduled test run.

TEST DESIGN PHASE (already discussed)

TEST EXECUTION PHASE

BUILD VERIFICATION PHASE

TEST CASE/SCRIPT EXECUTION –Log results, and log defects

DEFECT REPORTING

Re-TESTING ,REGRESSION TESTING

TEST EXECUTION STATUS REPORT



Verify Analyze


Build Verification Process

Build Manager adds the release notes

Developers build the complied code into Software

Testers perform BVT using Smoke/Sanity Test Testing team will accept the

build and proceed with Test Execution

Fails with critical defects , test team will not proceed with testing and build is rejected



Verify Analyze


TEST EXECUTION ACTIVITIES Prepare Test Infrastructure

Approving and signing-off of the test cases identified for the specific level by stakeholders

Setup test environment or test lab Test Data , Test Management Tool setup(optional) and

deploying the Build Task allocation Prepare the execution run plan based on priority,

functionality dependency and complexity. Availability of mitigation plan for the revalidated risks need

to be ensured.

TEST BED

TEST EXECUTION ACTIVITIES Actual Execution Process: Execute test cases using test data identified for the

specific test cases Analyze test results (check if ER==AR) Establish the course of action i.e if test pass then log

results with proof of execution and Update Traceability Matrix. If test fails then log defect and update status of defect.

Issue problem report and update test case log Planning the reruns Update the test execution log Update the test planning/execution status form Update the testing status reports

Testing

Static Dynamic

Functional Structural(Functionality of the program)

(Logic of the Program)

Black Box Testing

White Box Testing

Execute Tests step by step

Test step PassedRe-Test

Implement Resolution

Record Results

Record Defect

Resolve Defect

Record Results

Update End User Documentation

(as needed)

Test Execution Process

Change Control Board (as needed)

Test step Failed

Execute the next step of same test case

Can we execute the next step of the testcase?

YES

Execute the next TEST CASE

NOTE: Resolving defect and Implementing the Resolution will be one by the development team



Verify Analyze


Execute Test Cases i.e. Run Tests

TEST LOGS

(Pass/Fail)

VERIFY AND ANALYZE RESULT

TEST PROBLEM REPORT

BUG TRACKING

FOR EACH TEST RUN

Verify the existence of defect Verify that the element under test, the execution

environment and the test case implementation are consistent with the test case design.

Verify and confirm the discrepancy between the ER and AR



Verify Analyze


Analyze the Defect The purpose of the task is to analyze and resolve

variances encountered during test execution.

Each variance is analyzed to determine the cause.

Analysis may indicate the cause of variance is a defect, or that a change request is required.

Analyze the Defect Investigate further and gather as much

information as possible about the symptoms.

Analyze failures by varying conditions , options and settings to gauge the severity of the defect.

Analyze failures by varying conditions, options and settings

Change the conditions under which an error occurred by changing what you do. As an example, if the problem seems related to entering data very quickly, try slowing your rate of input.

Change the options and settings of the program under test. As an example, if the problem seems related to desktop screen-resolution settings, try varying the settings.

Change the software and hardware environment. For example, if the defect seems related to a low-end hardware configuration, try using the lowest configuration specified in the requirements.

Change the data used by the program. As an example, if the defect seems related to strings exceeding a certain number of characters in length, create a second set of test data to check the hypothesis.



Verify Analyze


Incident Summary• Incident Summary Report Summarize the incident. Identify the test items

involved, indicating their version and revision level. Supply references to the appropriate test-procedure specification, test-case specification and test log.

SAMPLE TEST PROBLEM DOCUMENTATION OR INCIDENT REPORT

Reporting Defect For each project , the defect methodology

defined would be different. Testers need to follow that methodology defined

in the Test Plan for that project Defect can be reported either in excel sheet or in

any other defect management tool if incorporated in the project

IEEE Standard for Test Defect Report Defect ID

Defect Description Build Version ID Feature/Module Test Case ID Reproducible Severity Priority Test Environment Status

Tester Name Detected On Reported To Reviewed By Suggested Fix(Optional)

Defect Severity Impact of defect in the software Defined in the Test Plan

SEVERITY TYPESEVERITY TYPE EXPLANATIONEXPLANATION EXAMPLEEXAMPLEBlocker Blocks development and/or testing work On clicking on “submit”

button system gets hangedCritical/Major Crashes, loss of data ,severe memory

leak, major loss of function1. The program runs for an extended time and consumes additional memory over time2. In HMS, Reports function missing

Minor Minor loss of function, or other problem where easy workaround is present

Hot keys not working

Trivial Cosmetic problem like misplelled words or misaligned text

“Enter your Nme”

Enhancement Request for enhancement

BUGS???



Verify Analyze


BUG LIFE CYCLESTATUS

UNCONFIRMED NEW ASSIGNED RESOLVED REOPENED VERIFIED CLOSED DEFERED REJECTED

BUG LIFE CYCLEUNCONFIRMED

NEW

ASSIGNED

RESOLVED

REOPEN VERIFIED

CLOSED

Bug confirmed or receives enough votes

Rectified

QA verifies that solution works

Bug is closed

Fixed, Duplicate, Wont Fix, Invalid…

Bug recently been added to database. Not yet Validated

Bug not yet resolved, but assigned to a person

Resolution was deemed incorrect. REJECTED AND DEFERRED

SAMPLE LOG SHEET

Test Types• Smoke• Sanity• Unit• Integration• Usability• System (Functionality)• Performance

Test Types• Adhoc Testing• Negative Testing• Monkey Testing• Regression Testing• Recovery Testing• Alpha Testing• Beta Testing• User Acceptance Test

Test Types• Installation Testing• Configuration Testing• Compatibility Testing• Benefits Realization Testing• Vendor Validation Testing

chapter 7c csta cstp test execution

Documents