cs48717-1 /30 illinois institute of technology
TRANSCRIPT
CS48717-1/30
Illinois Institute of Technology
CS487
Software Engineering
Testing - Part II
Software Testing Strategies
Mr. David A. Lash
© Carl J. Mueller, 2000
CS48717-2/30
Why Software Testing Strategy?
Some Test Strategy Issues:Should develop a formal test plan?
Test entire program as a whole or various pieces?
When should customer be included? Testing often accounts for more effort then developmentUsually strategy is defined in an overall testing strategy and procedure document (often by independent test group)
CS48717-3/30
Elements Of Software Test Strategy
Unit Testing - Tests the implemented source code of the component. Uses White Box.
Integration Testing - focuses on the design, testing and integration of components. Black box and limited White Box.
Validation Testing - Looks at requirements a validates software against it. Black Box Testing.
System Testing - Looks at software and other system elements tested as a whole. (Other hardware, people, databases, processes.)
CS48717-4/30
Unit Testing
Unit Testing - Module test – interfaces are tested to ensure information
flows into and out of the program correctly. – Data structures are tested. – Independent paths (basis paths) are
exercised)– Error handling paths - A mistake to miss these.
Use drivers (accepts data
and tests and passes
to module) and stubs
(replace other modules)
Driver
Stub
Module
Stub
TestCases
Results
CS48717-5/30
Integration A systematic technique to uncover problems
while putting the modules together. Composed of two activities:
– Building Compile and link all of the modules together
– Testing Verify interface function as specified
Avoid the “big bang” approach A.K.A. non-incremental integration
– can yield chaotic results.
CS48717-6/30
Implementation Process
ImplementUnit 1
O bject
ImplementUnit n
O bject
In tegration
Executable
Verification
Report
CS48717-7/30
Top-down (depth-first or breadth-first)– Test higher level modules first– Require stubs (not just returns)
Bottom-up– Test lower-level modules first– Require drivers (require logic)
Sandwich– Test highest level modules in conjunction with
lowest level meeting in the middle– Use stubs and drivers
Approaches to Integration Testing
CS48717-8/30
Example System
M 1
M 2 M 3 M 4
M 5 M 6
M 8
M 7
CS48717-9/30
Top Down (depth) Example
Depth First
– Integrate M1, use stubs for M2, M3 and M5 .
Test all M1 interfaces
– Next would test M2 and use operational stubs
for M3, M4, M5, M6 .Test all M2 interfaces
– Repeat for M5, then M8, M6, and M3...M 1
M 2 M 3 M 4
M 5 M 6
M 8
M 7
CS48717-10/30
Top Down (depth) Example
Integrate M1, use stubs for M2, M3 and M5 . Test all M1 interfaces
Next would test M2 and use operational stubs for M3, M4, M5, M6 .Test all M2 interfaces
Replace stub M5 with module M5. Use operational stubs for M3, M4, M6, M8. Test all the interfaces in M5
Repeat for M8, M6, and M3...
M 1
M 2 M 3 M 4
M 5 M 6
M 8
M 7
CS48717-11/30
Top Down (breath) Example Breath First Order
– M1, M2, M3, M4,
– Next level would be M5, M6, ...
Next Top down testing can have problems when low level
modules are needed (delay full tests or develop better stubs
or go bottom up)
M 1
M 2 M 3 M 4
M 5 M 6
M 8
M 7
CS48717-12/30
Bottom Up
Lower Items first using drivers :
– Using an M8 driver test module M8
– Using an M7 driver test module M7
– Using an M6 driver test module M6
– Using an M5 driver test module M5M 1
M 2 M 3 M 4
M 5 M 6
M 8
M 7
CS48717-13/30
Bottom Up Example II
In reality lower items are clustered together into sub-functions (aka builds).
M2
Driver1 Diver2 Driver3
M1
M3
CS48717-14/30
Sandwich Example
First Round
– Top-down Use operational stubs for M2, M3, M4
Test all the interfaces in M1
– Bottom-up Using an M8 driver test module M8
Using an M7 driver test module M7
Using an M6 driver test module M6M 1
M 2 M 3 M 4
M 5 M 6
M 8
M 7
CS48717-15/30
Regression Testing
The reexecution of some subset of tests to ensure changes have not propagated errors.
Tests conducted when a previously tested software is delivered
– Continuing testing after repair
– New Version
Partial testing old features when testing a maintenance release.
Might include tests of features/functions that were unchanged doing the current release.
Capture/playback tools enable re-execute of tests previously run.
CS48717-16/30
Verification
(1) The process of determining whether or not the products of a given phase of the software development cycle fulfill the requirements established during the previous phase
(2) Formal proof of program correctness
(3) The act of reviewing, inspecting, testing, checking, auditing, or otherwise establishing and documenting whether or not items, processes, services and documents conform to specified requirements
CS48717-17/30
Functional Verification
Verification of the Functional Design Specification
Should be done by developers Normally done by user or testing group
Can be combined with integration testing
CS48717-18/30
Validation
Are we building the right product?Does it meet the requirements?
CS48717-19/30
Validation PhaseRequirem ents
Specification
Design
Specification
Im plem entation
Software
Validation
Report
CS48717-20/30
Validation
The process of evaluating software at the end of the software development process to ensure compliance with software requirements.
Test plans define classes of test to be completed
CS48717-21/30
Validation Activities
System A process that attempts to demonstratethat a program or system does notmeet its original requirements andobjectives as stated in theRequirements
Acceptance The process of comparing the end-product to the current needs of its endusers.
Regression The re-execution of some subset oftest that have already been conducted.
CS48717-22/30
Validation Process
Regression
System
Acceptance
New Version
Re-startingValidation or
Next Interation
Defect
Defect
Defect
Deploy
CS48717-23/30
System Tests
FunctionalRequirements
The process of attempting todetect discrepancies between aprogram’s function and it’sspecification.
Volume Determines whether the programcan handle the required volumesof data requests, etc.
Load/Stress Identify the peak load conditions atwhich the program will fail tohandle required processing loadswithin required time spans.
Security Show that the program’s securityrequirements can be subverted.
CS48717-24/30
System Tests
Recovery Determine whether the system orprogram meets its requirements forrecovery
Performance Determines whether the programmeets the performancerequirements.
ResourceUsage
Determines whether the programuses resources at levels, whichexceed requirements.
Configuration Determine whether the programoperates properly when thesoftware or hardware is configured.
CS48717-25/30
System Tests
Compatibility Determine whether the compatibility objects of the program have been.
Installability Identify the ways in which the installation procedures lead to incorrect results.
Serviceability Identify conditions whose serviceability needs won’t meet requirements.
Reliability Determines whether the system meets its reliability and availability requirements.
CS48717-26/30
System Tests
Usability Identify those operations that willdifficult or inconvenient for theusers. Pubs, Doc and manualprocedures.
CS48717-27/30
Rules for High-Level Tests
1. Test are run on operational production hardware and software.
2. Tests must stress the system significantly
3. All interfaced systems must be in operation throughout the test.
4. Test duration should run a full cycle
5. Tests should exercise the system over a full range of inputs, both legal and illegal
6. All major functions and interfaces should be exercised.
7. If running the entire test cannot be completed, a new run should include a complete restart.
CS48717-28/30
Acceptance Testing
Depends on the type of product and the situation.
When working with an a non-technical user, the following are recommended.
– First, train user on system with a number of test cases.
– Use a comparison test. Re-processing Parallel to existing system
CS48717-29/30
Acceptance Testing
Product Development– Alpha testing
Performed by a selected end-users usually in side the development environment.
– Beta testing Performed by a subset of actual customers
outside the company, before the software is made available to the customer.
– Field Trial Large number of users, but not general
release.
CS48717-30/30
Acceptance Testing
Work for Hire– Training Programming
Evaluate based on comments users general reaction to system.
Usability Test– Parallel Operation
Comparison Testing New software used after old system
process.– Controlled deployment
Representative monitors software at site.