slides chapters 13-14
DESCRIPTION
Software EngineeringBy Roger PressmanTRANSCRIPT
1
Chapter 13 & 14Software Testing
Strategiesand Techniques
Chapter 13 & 14Software Testing
Strategiesand Techniques Software Engineering: A Practitioner’s Approach, 6th edition
by Roger S. Pressman
2
Software Testing
Testing is the process of exercising aTesting is the process of exercising aprogram with the specific intent of findingprogram with the specific intent of findingerrors prior to delivery to the end user.errors prior to delivery to the end user.
3
What Testing Shows
errorserrors
requirements conformancerequirements conformance
performanceperformance
an indicationan indicationof qualityof quality
4
Who Tests the Software?
developerdeveloper independent testerindependent tester
Understands the system Understands the system
but, will test "gently"but, will test "gently"
and, is driven by "delivery"and, is driven by "delivery"
Must learn about the system,Must learn about the system,but, will attempt to break itbut, will attempt to break itand, is driven by qualityand, is driven by quality
5
Validation vs VerificationValidation vs Verification
Verification – Are we building the product right? Is the code correct with respect to its
specification?
Validation – Are we building the right product? Does the specification reflect what it should?
Verification – Are we building the product right? Is the code correct with respect to its
specification?
Validation – Are we building the right product? Does the specification reflect what it should?
6
Testing Strategy
unit testunit test integrationintegrationtesttest
validationvalidationtesttest
systemsystemtesttest
7
Testing StrategyTesting Strategy Begin with unit testing and work your way up to
system testing. Unit testing – test individual components (modules in
procedural languages; classes in OO languages) Integration testing – test collections of components
that must work together Validation testing – test the application as a whole
against user requirements System testing – test the application in the context of
an entire system
Begin with unit testing and work your way up to system testing.
Unit testing – test individual components (modules in procedural languages; classes in OO languages)
Integration testing – test collections of components that must work together
Validation testing – test the application as a whole against user requirements
System testing – test the application in the context of an entire system
8
Unit Testing
modulemoduleto beto betestedtested
test casestest cases
resultsresults
softwaresoftwareengineerengineer
9
Unit Testing
interface interface local data structureslocal data structures
boundary conditionsboundary conditionsindependent pathsindependent pathserror handling pathserror handling paths
modulemoduleto beto betestedtested
test casestest cases
10
Unit Test Environment
ModuleModule
stubstub stubstub
driverdriver
RESULTSRESULTS
interface interface
local data structureslocal data structures
boundary conditionsboundary conditions
independent pathsindependent paths
error handling pathserror handling paths
test casestest cases
11
Integration Testing Strategies
Options:Options:•• the “big bang” approachthe “big bang” approach•• an incremental construction strategyan incremental construction strategy
12
Top Down Integration
top module is tested with top module is tested with stubsstubs
stubs are replaced one at stubs are replaced one at a time, "depth first"a time, "depth first"
as new modules are integrated, as new modules are integrated, some subset of tests is re-runsome subset of tests is re-run
AA
BB
CC
DD EE
FF GG
13
Bottom-Up Integration
drivers are replaced one at a drivers are replaced one at a time, "depth first"time, "depth first"
worker modules are grouped into worker modules are grouped into builds and integratedbuilds and integrated
AA
BB
CC
DD EE
FF GG
clustercluster
14
Regression TestingRegression Testing
The selective retesting of a modified system to help ensure that no bugs have been introduced during modification.
Fixing one part of the code can break another
The selective retesting of a modified system to help ensure that no bugs have been introduced during modification.
Fixing one part of the code can break another
15
High Order TestingHigh Order Testing Validation testing
Focus is on software requirements System testing
Focus is on system integration Alpha/Beta testing
Focus is on customer usage Recovery testing
forces the software to fail in a variety of ways and verifies that recovery is properly performed Security testing
verifies that protection mechanisms built into a system will, in fact, protect it from improper penetration
Stress testing executes a system in a manner that demands resources in abnormal quantity, frequency, or volume
Performance Testing test the run-time performance of software within the context of an integrated system
Validation testing Focus is on software requirements
System testing Focus is on system integration
Alpha/Beta testing Focus is on customer usage
Recovery testing forces the software to fail in a variety of ways and verifies that recovery is properly performed
Security testing verifies that protection mechanisms built into a system will, in fact, protect it from improper
penetration Stress testing
executes a system in a manner that demands resources in abnormal quantity, frequency, or volume Performance Testing
test the run-time performance of software within the context of an integrated system
16
What is a “Good” Test?What is a “Good” Test?
A good test is one that has a high probability of finding an error.
A good test is one that has a high probability of finding an error.
17
Test Case Design
"Bugs lurk in corners "Bugs lurk in corners and congregate at and congregate at boundaries ..."boundaries ..."
Boris BeizerBoris Beizer
OBJECTIVEOBJECTIVE
CRITERIACRITERIA
CONSTRAINTCONSTRAINT
to uncover errorsto uncover errors
in a complete mannerin a complete manner
with a minimum of effort and timewith a minimum of effort and time
18
Exhaustive Testing
loop < 20 Xloop < 20 X
There are 10 possible paths! If we execute oneThere are 10 possible paths! If we execute onetest per millisecond, it would take 3,170 years totest per millisecond, it would take 3,170 years totest this program!!test this program!!
1414
19
Selective Testing
loop < 20 Xloop < 20 X
Selected pathSelected path
20
Software Testing
Methods
Strategies
white-boxmethods
black-box methods
21
White-Box Testing
... our goal is to ensure that all ... our goal is to ensure that all statements and conditions have statements and conditions have been executed at least once ...been executed at least once ...
22
Why Cover?
logic errors and incorrect assumptions logic errors and incorrect assumptions are inversely proportional to a path's are inversely proportional to a path's execution probabilityexecution probability
we often we often believebelieve that a path is not that a path is not likely to be executed; in fact, reality is likely to be executed; in fact, reality is often counter intuitiveoften counter intuitive
typographical errors are random; it's typographical errors are random; it's likely that untested paths will contain likely that untested paths will contain some some
23
Basis Path TestingFirst, we compute the cyclomatic
complexity:
number of simple decisions + 1
or
number of enclosed areas + 1
In this case, V(G) = 4
24
Cyclomatic Complexity
A number of industry studies have indicated A number of industry studies have indicated that the higher V(G), the higher the probability that the higher V(G), the higher the probability or errors.or errors.
V(G)V(G)
modulesmodules
modules in this range are modules in this range are more error pronemore error prone
25
Basis Path TestingNext, we derive the Next, we derive the
independent paths:independent paths:
Since V(G) = 4,Since V(G) = 4,there are four pathsthere are four paths
Path 1: 1,2,3,6,7,8Path 1: 1,2,3,6,7,8Path 2: 1,2,3,5,7,8Path 2: 1,2,3,5,7,8Path 3: 1,2,4,7,8Path 3: 1,2,4,7,8Path 4: 1,2,4,7,2,4,...7,8Path 4: 1,2,4,7,2,4,...7,8
Finally, we derive testFinally, we derive testcases to exercise these cases to exercise these paths.paths.
11
22
3344
55 66
77
88
26
Basis Path Testing Notes
you don't need a flow chart, you don't need a flow chart, but the picture will help when but the picture will help when you trace program pathsyou trace program paths
count each simple logical test, count each simple logical test, compound tests count as 2 or compound tests count as 2 or moremore
basis path testing should be basis path testing should be applied to critical modulesapplied to critical modules
27
Black-Box Testing
requirementsrequirements
eventseventsinputinput
outputoutput
28
Equivalence Partitioning
useruserqueriesqueries
mousemousepickspicks
outputoutputformatsformats
promptsprompts
FKFKinputinput
datadata
29
Sample Equivalence Classes
user supplied commandsuser supplied commandsresponses to system promptsresponses to system promptsfile namesfile namescomputational datacomputational data physical parameters physical parameters bounding valuesbounding values initiation valuesinitiation valuesoutput data formattingoutput data formattingresponses to error messagesresponses to error messagesgraphical data (e.g., mouse picks)graphical data (e.g., mouse picks)
data outside bounds of the program data outside bounds of the program physically impossible dataphysically impossible dataproper value supplied in wrong placeproper value supplied in wrong place
Valid dataValid data
Invalid dataInvalid data
30
Boundary Value Analysis
useruserqueriesqueries
mousemousepickspicks
outputoutputformatsformats
promptsprompts
FKFKinputinput
datadata
outputoutputdomaindomaininput domaininput domain
31
OOT Methods: Behavior TestingOOT Methods: Behavior Testing
emptyacctopen setup Accnt
set upacct
deposit(initial)
workingacct
withdrawal(final)
deadacct close
nonworkingacct
deposit
withdrawbalance
creditaccntInfo
Figure 14.3 State diagram for Account class (adapted from [ KIR94])
The tests to be The tests to be designed designed should achieve should achieve all state all state coverage coverage [KIR94]. That [KIR94]. That is, the is, the operation operation sequences sequences should cause should cause the Account the Account class to make class to make transition transition through all through all allowable allowable statesstates