Download - ISEB Software Testing Presentation
-
8/8/2019 ISEB Software Testing Presentation
1/414
SIM Group Ltd., SQS Group AG, 2002
ISEB Foundation Certificate in Software Testing
Testing TerminologyTesting Terminology
-
8/8/2019 ISEB Software Testing Presentation
2/414
SIM Group Ltd., SQS Group AG, 2002
Testing Terminology
IT is an industry riddled with mysterious words, terms andIT is an industry riddled with mysterious words, terms andTLATLAss
Software testing is no exceptionSoftware testing is no exception
Historically there has not been a generally accepted set ofHistorically there has not been a generally accepted set of
testing definitionstesting definitions
-
8/8/2019 ISEB Software Testing Presentation
3/414
SIM Group Ltd., SQS Group AG, 2002
Testing Terminology
To resolve this issue, the BCS SIGIST created a StandardTo resolve this issue, the BCS SIGIST created a StandardGlossary of Testing TermsGlossary of Testing Terms
British Computer SocietyBritish Computer Society
Special Interest Group In Software TestingSpecial Interest Group In Software Testing
This is now a British StandardThis is now a British Standard
BS7925BS7925--11
-
8/8/2019 ISEB Software Testing Presentation
4/414SIM Group Ltd., SQS Group AG, 2002
Testing Terminology
You will need to read the document and start to learn theYou will need to read the document and start to learn theterminology used in software testingterminology used in software testing
You will be exposed to the terminology and customers willYou will be exposed to the terminology and customers willassume you know what they meanassume you know what they mean
-
8/8/2019 ISEB Software Testing Presentation
5/414SIM Group Ltd., SQS Group AG, 2002
ISEB Foundation Certificate in Software Testing
Why Testing is NecessaryWhy Testing is Necessary
-
8/8/2019 ISEB Software Testing Presentation
6/414SIM Group Ltd., SQS Group AG, 2002
Why Testing is Necessary
In this session we willIn this session we will
Understand what testing is and why it is necessaryUnderstand what testing is and why it is necessary
Define some of the words used in software testingDefine some of the words used in software testing
Look at the cause and cost of errorsLook at the cause and cost of errors
-
8/8/2019 ISEB Software Testing Presentation
7/414SIM Group Ltd., SQS Group AG, 2002
Why Testing is Necessary
Why Test?Why Test?
To find bugsTo find bugs
What is a bug?What is a bug?
-
8/8/2019 ISEB Software Testing Presentation
8/414
SIM Group Ltd., SQS Group AG, 2002
Why Testing is Necessary
What is Testing?What is Testing?
the process of executing a program with the intent to certify itthe process of executing a program with the intent to certify itss
QualityQuality
MillsMills
the process of executing a program with the intent of findingthe process of executing a program with the intent of finding
failures / faultsfailures / faults
MyersMyers
the process of exercising software to detect bugs & to verify ththe process of exercising software to detect bugs & to verify thatat
it satisfies specified functional & nonit satisfies specified functional & non--functional requirementsfunctional requirements
-
8/8/2019 ISEB Software Testing Presentation
9/414
SIM Group Ltd., SQS Group AG, 2002
Why Testing is Necessary
Why is Testing becoming more important?Why is Testing becoming more important?
The Y2K problemThe Y2K problem
EMUEMU
e Commercee Commerce
Increased user baseIncreased user base
Increased complexityIncreased complexity
Speed to MarketSpeed to Market
-
8/8/2019 ISEB Software Testing Presentation
10/414
SIM Group Ltd., SQS Group AG, 2002
Definitions
-
8/8/2019 ISEB Software Testing Presentation
11/414
-
8/8/2019 ISEB Software Testing Presentation
12/414
SIM Group Ltd., SQS Group AG, 2002
Definitions
FAULTFAULT
A Fault, if encountered, may cause a failureA Fault, if encountered, may cause a failure
FAILUREFAILUREA deviation of the software from its expected delivery orA deviation of the software from its expected delivery or
serviceservice
-
8/8/2019 ISEB Software Testing Presentation
13/414
SIM Group Ltd., SQS Group AG, 2002
Definitions
DEFECTDEFECT
The departure of a quality characteristic from its specified valThe departure of a quality characteristic from its specified valueue
that results in a product or service not satisfying its normalthat results in a product or service not satisfying its normal
usage requirementsusage requirements
-
8/8/2019 ISEB Software Testing Presentation
14/414
SIM Group Ltd., SQS Group AG, 2002
Definitions
RELIABILITYRELIABILITY
The probability that software will not cause the failure of aThe probability that software will not cause the failure of a
system for a specified period of time under specifiedsystem for a specified period of time under specified
conditionsconditions
-
8/8/2019 ISEB Software Testing Presentation
15/414
SIM Group Ltd., SQS Group AG, 2002
Definitions
QUALITYITY
The totality of the characteristics of an entity that bear on itThe totality of the characteristics of an entity that bear on itss
ability to satisfy stated or implied needsability to satisfy stated or implied needs
T i & Q li
-
8/8/2019 ISEB Software Testing Presentation
16/414
SIM Group Ltd., SQS Group AG, 2002
Testing & Quality
Does testing improve quality?Does testing improve quality?
Testing does not build Quality into the softwareTesting does not build Quality into the software
Testing is a means of determining the Quality of the SoftwareTesting is a means of determining the Quality of the Software
under Testunder Test
D fi iti
-
8/8/2019 ISEB Software Testing Presentation
17/414
SIM Group Ltd., SQS Group AG, 2002
Definitions
QUALITY ASSURANCEQUALITY ASSURANCE
All those planned actions used to fulfil the requirements forAll those planned actions used to fulfil the requirements for
qualityquality
Wh d ?
-
8/8/2019 ISEB Software Testing Presentation
18/414
SIM Group Ltd., SQS Group AG, 2002
Why do errors occur?
Wh do errors occ r?
-
8/8/2019 ISEB Software Testing Presentation
19/414
SIM Group Ltd., SQS Group AG, 2002
Why do errors occur?
How Errors OccurHow Errors Occur
No one is perfect!No one is perfect!
We all make mistakes or omissionsWe all make mistakes or omissions
The more pressure we are under the more likely we are toThe more pressure we are under the more likely we are to
make mistakesmake mistakes
In IT development we have time and budgetary deadlines toIn IT development we have time and budgetary deadlines to
meetmeet
Poor TrainingPoor Training
Why do errors occur?
-
8/8/2019 ISEB Software Testing Presentation
20/414
SIM Group Ltd., SQS Group AG, 2002
Why do errors occur?
How Errors OccurHow Errors Occur
Poor CommunicationPoor Communication
Requirements not clearly definedRequirements not clearly defined
Requirements change & are not properly documentedRequirements change & are not properly documented
Data specifications not completeData specifications not complete
ASSUMPTIONS!ASSUMPTIONS!
Why do errors occur?
-
8/8/2019 ISEB Software Testing Presentation
21/414
SIM Group Ltd., SQS Group AG, 2002
Why do errors occur?
The Cost Of ErrorsThe Cost Of Errors
A single failure may incur little costA single failure may incur little cost -- or millionsor millions
Report layouts may be wrongReport layouts may be wrong -- little true costlittle true cost
Or a significant error may cost millionsOr a significant error may cost millions
ArianeAriane V, Venus Probe, Mars Explorer and Polar LanderV, Venus Probe, Mars Explorer and Polar Lander
Why do errors occur?
-
8/8/2019 ISEB Software Testing Presentation
22/414
SIM Group Ltd., SQS Group AG, 2002
Why do errors occur?
The Cost Of ErrorsThe Cost Of Errors
In extreme cases a software or systems error may costIn extreme cases a software or systems error may cost
LIVESLIVES
Usually safety critical systems are tested exhaustivelyUsually safety critical systems are tested exhaustively
Aeroplane, railway, nuclear power systems etcAeroplane, railway, nuclear power systems etc
Unfortunately there are exceptions to thisUnfortunately there are exceptions to this
London Ambulance ServiceLondon Ambulance Service
Why do errors occur?
-
8/8/2019 ISEB Software Testing Presentation
23/414
SIM Group Ltd., SQS Group AG, 2002
Why do errors occur?
The Cost Of ErrorsThe Cost Of Errors
The cost of defects increases proportionately (tenfold?) withThe cost of defects increases proportionately (tenfold?) with
the passing of each successive stage in the systemthe passing of each successive stage in the systemdevelopment process before they are detecteddevelopment process before they are detected
To correct a problem at requirements stage may costTo correct a problem at requirements stage may cost 11
To correct the problem postTo correct the problem post--implementation may costimplementation may cost 000000ss
Why Testing is Necessary
-
8/8/2019 ISEB Software Testing Presentation
24/414
SIM Group Ltd., SQS Group AG, 2002
Why Testing is Necessary
Why Testing is Necessary
-
8/8/2019 ISEB Software Testing Presentation
25/414
SIM Group Ltd., SQS Group AG, 2002
Why Testing is Necessary
Why Test?Why Test?
Identifies faultsIdentifies faults
Reduces live defectsReduces live defects
Improves the quality of the users applicationImproves the quality of the users application
Increases reliabilityIncreases reliability
To help ensure live failures donTo help ensure live failures dont impact costs & profitabilityt impact costs & profitability
To help ensure requirements are satisfiedTo help ensure requirements are satisfied
To ensure legal requirements are metTo ensure legal requirements are met
To help maintain the organisationTo help maintain the organisations reputations reputation
Provides a measure of qualityProvides a measure of quality
Why Testing is Necessary
-
8/8/2019 ISEB Software Testing Presentation
26/414
SIM Group Ltd., SQS Group AG, 2002
Why Testing is Necessary
Quality MeasurementsQuality Measurements
Quality can be measured by testing forQuality can be measured by testing for
correctnesscorrectness
reliabilityreliability
usabilityusabilitymaintainabilitymaintainability
testabilitytestability
reusabilityreusability
Why Testing is Necessary
-
8/8/2019 ISEB Software Testing Presentation
27/414
SIM Group Ltd., SQS Group AG, 2002
Why Testing is Necessary
How much is enough?How much is enough?
How do we know when to stop?How do we know when to stop?
There are many factors to considerThere are many factors to consider
Why Testing is Necessary
-
8/8/2019 ISEB Software Testing Presentation
28/414
SIM Group Ltd., SQS Group AG, 2002
Why Testing is Necessary
Exhaustive TestingExhaustive Testing
Find all faults by testing everything?Find all faults by testing everything?
To test everything is rarely possibleTo test everything is rarely possible
The time required makes it impracticalThe time required makes it impractical
The resource commitment required makes it impracticalThe resource commitment required makes it impractical
Why Testing is Necessary
-
8/8/2019 ISEB Software Testing Presentation
29/414
SIM Group Ltd., SQS Group AG, 2002
y est g s ecessa y
If we canIf we cant test everything, what can we do?t test everything, what can we do?
Managing and reducing RiskManaging and reducing Risk
Carry out a Risk Analysis of the applicationCarry out a Risk Analysis of the application
Prioritise tests to focus on the main areas of riskPrioritise tests to focus on the main areas of risk
Apportion time relative to the degree of risk involvedApportion time relative to the degree of risk involved
Understand the risk to the business of the software notUnderstand the risk to the business of the software not
functioning correctlyfunctioning correctly
Why Testing is Necessary
-
8/8/2019 ISEB Software Testing Presentation
30/414
SIM Group Ltd., SQS Group AG, 2002
y g y
Should you stop testing when the product goes live?Should you stop testing when the product goes live?
Continuing testing beyond the implementation date should beContinuing testing beyond the implementation date should be
consideredconsidered
Better that you find the errors than the usersBetter that you find the errors than the users
Why Testing is Necessary
-
8/8/2019 ISEB Software Testing Presentation
31/414
SIM Group Ltd., SQS Group AG, 2002
y g y
SummarySummary
The purpose of testing is to find faultsThe purpose of testing is to find faults
Faults can be fixed, making better softwareFaults can be fixed, making better softwareBetter software is more reliable, less prone to failuresBetter software is more reliable, less prone to failures
Establish the relationship between the software and itsEstablish the relationship between the software and its
specificationspecification
Testing enables us to measure the quality of the softwareTesting enables us to measure the quality of the software
This enables us to understand & manage the risk to theThis enables us to understand & manage the risk to the
businessbusiness
-
8/8/2019 ISEB Software Testing Presentation
32/414
SIM Group Ltd., SQS Group AG, 2002
ISEB Foundation Certificate in Software Testing
The Test ProcessThe Test Process
The Test Process
-
8/8/2019 ISEB Software Testing Presentation
33/414
SIM Group Ltd., SQS Group AG, 2002
In this session we willIn this session we will
Look at the steps involved in the test processLook at the steps involved in the test process
Look in detail at each stepLook in detail at each step
The Test Process
-
8/8/2019 ISEB Software Testing Presentation
34/414
SIM Group Ltd., SQS Group AG, 2002
What is the objective of a test?What is the objective of a test?
AA successfulsuccessful test is one thattest is one that doesdoes detect a faultdetect a fault
The Test Process
-
8/8/2019 ISEB Software Testing Presentation
35/414
SIM Group Ltd., SQS Group AG, 2002
COMPLETION
E
XECUTIO
N
TEST MANAGEMENT
TEST ENVIRONMENT MANAGEMENT
TEST ASSET MANAGEMENT
R
ECORDIN
G
PLANNING
SPECIFICAT
ION
The Test Process
-
8/8/2019 ISEB Software Testing Presentation
36/414
SIM Group Ltd., SQS Group AG, 2002
Five steps in the Test Process areFive steps in the Test Process are
Test PlanningTest Planning
Test Analysis and SpecificationTest Analysis and Specification
Test ExecutionTest Execution
Test Recording (Verification)Test Recording (Verification)
Checking for CompletionChecking for Completion
Test Planning
-
8/8/2019 ISEB Software Testing Presentation
37/414
SIM Group Ltd., SQS Group AG, 2002
The Test Plan describes howThe Test Plan describes howthe Test Strategy isthe Test Strategy is
implementedimplemented
Test Planning
-
8/8/2019 ISEB Software Testing Presentation
38/414
SIM Group Ltd., SQS Group AG, 2002
Contents of a Test Plan includeContents of a Test Plan include
BackgroundBackground
Reference documentsReference documents ApproachApproach
MethodMethod
TimetableTimetable
ResourcesResources
DependenciesDependencies
ReportingReporting
Test Asset IdentificationTest Asset Identification
Exit CriteriaExit Criteria
Test Planning
-
8/8/2019 ISEB Software Testing Presentation
39/414
SIM Group Ltd., SQS Group AG, 2002
The most critical stage of theThe most critical stage of theprocessprocess
Effort spent now will beEffort spent now will be
rewarded laterrewarded later
The foundation on whichThe foundation on which
testing is builttesting is built
Test Specification
-
8/8/2019 ISEB Software Testing Presentation
40/414
SIM Group Ltd., SQS Group AG, 2002
Three step processThree step process
Preparation & analysisPreparation & analysis
Building test casesBuilding test cases
Define expected resultsDefine expected results
Test Specification
-
8/8/2019 ISEB Software Testing Presentation
41/414
SIM Group Ltd., SQS Group AG, 2002
Test preparationTest preparation
Analyse the ApplicationAnalyse the Application
Identify good test conditionsIdentify good test conditions
Identify test casesIdentify test cases
Document thoroughlyDocument thoroughly
CrossCross--referencingreferencing
Test Specification
-
8/8/2019 ISEB Software Testing Presentation
42/414
SIM Group Ltd., SQS Group AG, 2002
Build Test CasesBuild Test Cases
Test cases comprise:Test cases comprise:
Standing DataStanding Data
Transaction DataTransaction Data
ActionsActions
Expected ResultsExpected Results
Test Specification
-
8/8/2019 ISEB Software Testing Presentation
43/414
SIM Group Ltd., SQS Group AG, 2002
Expected ResultsExpected Results
The outcome of each actionThe outcome of each action
The state of the applicationThe state of the application
during & afterduring & after
The state of the data duringThe state of the data during& after& after
Test Specification
-
8/8/2019 ISEB Software Testing Presentation
44/414
SIM Group Ltd., SQS Group AG, 2002
CrossCross--Referencing &Referencing &ClassificationClassification
Enables maintainability ofEnables maintainability of
Test AssetsTest Assets
Allows testing to beAllows testing to be
performed in a focussedperformed in a focussedmanner directed at specificmanner directed at specific
areasareas
Test Execution
-
8/8/2019 ISEB Software Testing Presentation
45/414
SIM Group Ltd., SQS Group AG, 2002
Test Execution checklistTest Execution checklist
Test execution schedule / logTest execution schedule / log
Identify which tests are to be runIdentify which tests are to be run
Test environment primed & readyTest environment primed & ready
Resources ready, willing & ableResources ready, willing & able BackBack--up & recovery procedures in placeup & recovery procedures in place
Batch runs planned & scheduledBatch runs planned & scheduled
Then we are ready to run the testsThen we are ready to run the tests
Test Recording
-
8/8/2019 ISEB Software Testing Presentation
46/414
SIM Group Ltd., SQS Group AG, 2002
Test verificationTest verification
If planning and preparation is sufficiently detailed this is theIf planning and preparation is sufficiently detailed this is the
easy part of testingeasy part of testing
The test is run to verify the application under testThe test is run to verify the application under test
The test itself either passes or fails!The test itself either passes or fails!
Test Recording
-
8/8/2019 ISEB Software Testing Presentation
47/414
SIM Group Ltd., SQS Group AG, 2002
The test log should recordThe test log should record
Software and test versionsSoftware and test versions
Specifications used as test baseSpecifications used as test base
Test timingsTest timings
Test resultsTest results Actual resultsActual results
Expected resultsExpected results
Defect details for erroneous testsDefect details for erroneous tests
Test Completion
-
8/8/2019 ISEB Software Testing Presentation
48/414
SIM Group Ltd., SQS Group AG, 2002
Test Exit CriteriaTest Exit Criteria
Used to determine when toUsed to determine when to
implement the softwareimplement the software
Key Functionality testedKey Functionality tested
Test coverageTest coverage
Budget used?Budget used?
Defect detection rateDefect detection rate
Performance satisfactoryPerformance satisfactory
The Test Process
-
8/8/2019 ISEB Software Testing Presentation
49/414
SIM Group Ltd., SQS Group AG, 2002
SummarySummary
There are five steps in the Test ProcessThere are five steps in the Test Process
Test PlanningTest Planning
Test Analysis and SpecificationTest Analysis and Specification
Test ExecutionTest Execution Test Recording (Verification)Test Recording (Verification)
Checking for CompletionChecking for Completion
-
8/8/2019 ISEB Software Testing Presentation
50/414
SIM Group Ltd., SQS Group AG, 2002
ISEB Foundation Certificate in Software Testing
The Psychology Of TestingThe Psychology Of Testing
The Psychology Of Testing
-
8/8/2019 ISEB Software Testing Presentation
51/414
SIM Group Ltd., SQS Group AG, 2002
In this session we willIn this session we will
Understand what qualities make good testersUnderstand what qualities make good testers
Look at a testersLook at a testers relationship with developersrelationship with developers
Look at a testersLook at a testers relationship with managementrelationship with management
Understand the issues with testing independenceUnderstand the issues with testing independence
What makes a Tester?
-
8/8/2019 ISEB Software Testing Presentation
52/414
SIM Group Ltd., SQS Group AG, 2002
Testing is primarily to find faultsTesting is primarily to find faults
Can be regarded asCan be regarded as destructivedestructive
Development isDevelopment is constructiveconstructive
Testing asks questionsTesting asks questions
Testers need to ask questionsTesters need to ask questions
A tester needs many qualities...A tester needs many qualities...
What makes a Tester?
-
8/8/2019 ISEB Software Testing Presentation
53/414
SIM Group Ltd., SQS Group AG, 2002
Intellectual qualitiesIntellectual qualities
Can absorb incomplete factsCan absorb incomplete facts
Can work with incomplete factsCan work with incomplete facts
Can learn quickly on many levelsCan learn quickly on many levels
Good verbal communicationGood verbal communication Good written communicationGood written communication
Ability to prioritiseAbility to prioritise
SelfSelf--organisationorganisation
-
8/8/2019 ISEB Software Testing Presentation
54/414
What makes a Tester?
-
8/8/2019 ISEB Software Testing Presentation
55/414
SIM Group Ltd., SQS Group AG, 2002
More skills to acquireMore skills to acquire
How to find bugsHow to find bugs -- planning, preparation & executionplanning, preparation & execution
How to understand systemsHow to understand systems
How to read specificationsHow to read specifications
How to extract testable functionalityHow to extract testable functionality How to work efficientlyHow to work efficiently
How to focus on essentialsHow to focus on essentials
Reporting Defects
-
8/8/2019 ISEB Software Testing Presentation
56/414
SIM Group Ltd., SQS Group AG, 2002
Defects need to be reported toDefects need to be reported to
Developers to enable them to fix themDevelopers to enable them to fix them
Management so they can track progressManagement so they can track progress
Communication to both groups is vitalCommunication to both groups is vital
Communication with Developers
-
8/8/2019 ISEB Software Testing Presentation
57/414
SIM Group Ltd., SQS Group AG, 2002
A good relationship is vitalA good relationship is vital
Developers need to keep testers up to date with changes toDevelopers need to keep testers up to date with changes to
the applicationthe application
Testers need to inform developers of defects to allow fixes toTesters need to inform developers of defects to allow fixes to
be appliedbe applied
Communication with Management
-
8/8/2019 ISEB Software Testing Presentation
58/414
SIM Group Ltd., SQS Group AG, 2002
Managers need progress reportsManagers need progress reports
The best way is through metricsThe best way is through metrics
Number of tests planned & preparedNumber of tests planned & prepared
Number of tests executed to dateNumber of tests executed to date
Number of defects raised & fixedNumber of defects raised & fixed How long planning, preparation and execution stages takeHow long planning, preparation and execution stages take
-
8/8/2019 ISEB Software Testing Presentation
59/414
SIM Group Ltd., SQS Group AG, 2002
The Psychology of Testing
Testing IndependenceTesting Independence
Testing Independence
-
8/8/2019 ISEB Software Testing Presentation
60/414
SIM Group Ltd., SQS Group AG, 2002
It is important that testing is separate fromIt is important that testing is separate fromdevelopmentdevelopment
The developer is likely to confirm adherence not deviationThe developer is likely to confirm adherence not deviation
The developer will make assumptionsThe developer will make assumptions -- the same whenthe same when
testing as developingtesting as developing
Testing Independence
-
8/8/2019 ISEB Software Testing Presentation
61/414
SIM Group Ltd., SQS Group AG, 2002
Levels of IndependenceLevels of Independence
LowLow -- Developers write their own testsDevelopers write their own tests
MediumMedium -- Tests are written by another developerTests are written by another developer
HighHigh -- Tests written by an independent bodyTests written by an independent body Tests written by another sectionTests written by another section
Tests written by another organisationTests written by another organisation
UtopiaUtopia -- Tests generated automaticallyTests generated automatically
The Psychology Of Testing
-
8/8/2019 ISEB Software Testing Presentation
62/414
SIM Group Ltd., SQS Group AG, 2002
SummarySummary
Testers require a particular set of skillsTesters require a particular set of skills
The desire to break thingsThe desire to break things The desire to explore and experimentThe desire to explore and experiment
CommunicationCommunication
QuestioningQuestioning
Testing requires a different mentality to developmentTesting requires a different mentality to development
DestroyingDestroying things rather than creating themthings rather than creating them
Testing should be separate from developmentTesting should be separate from development
-
8/8/2019 ISEB Software Testing Presentation
63/414
SIM Group Ltd., SQS Group AG, 2002
ISEB Foundation Certificate in Software Testing
ReRe--Testing & Regression TestingTesting & Regression Testing
Re-Testing & Regression Testing
-
8/8/2019 ISEB Software Testing Presentation
64/414
SIM Group Ltd., SQS Group AG, 2002
In this session we willIn this session we will
Understand how fault fixing and reUnderstand how fault fixing and re--testing are key to thetesting are key to the
overall testing processoverall testing process
Look at how test repeatability helpsLook at how test repeatability helps
Understand what regression testing is and where it fits inUnderstand what regression testing is and where it fits in
Understand how to select test cases for regression testingUnderstand how to select test cases for regression testing
Re-Testing
-
8/8/2019 ISEB Software Testing Presentation
65/414
SIM Group Ltd., SQS Group AG, 2002
Testing is directed at finding faultsTesting is directed at finding faults
These faults will need to be corrected and a new version ofThese faults will need to be corrected and a new version of
the software releasedthe software released
The tests will need to be run again to prove that the fault isThe tests will need to be run again to prove that the fault is
fixedfixed
This is known as ReThis is known as Re--TestingTesting
-
8/8/2019 ISEB Software Testing Presentation
66/414
Regression Testing
-
8/8/2019 ISEB Software Testing Presentation
67/414
SIM Group Ltd., SQS Group AG, 2002
ReRe--testing of a previously tested program followingtesting of a previously tested program followingmodification to ensure that FAULTS have not beenmodification to ensure that FAULTS have not been
introduced or uncovered as a result of the changesintroduced or uncovered as a result of the changes
mademade
Regression Testing
Module 93 v1 3 5Module 93 v1.3.5
H fi thiHow can you fix this
-
8/8/2019 ISEB Software Testing Presentation
68/414
SIM Group Ltd., SQS Group AG, 2002
Module 14 v1.4.2
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~Declare String x =Date();~~~~~~~~~~~~~~~~
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Module 14 v1.4.2
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~Declare String x =Date();~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Module 41 v1.8.9
~~~~~~~~~~~~~~~~Declare Integer x = 0;~~~~~~~~~~~~~~~~
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~For x = 1 to 5~~~~~~~~~~~~~~~~
Module 41 v1.8.9
~~~~~~~~~~~~~~~~
Declare Integer x = 0;~~~~~~~~~~~~~~~~
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~For x = 1 to 5~~~~~~~~~~~~~~~~
Module 93 v1.3.5
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Print (x);~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Module 93 v1.3.5
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Print (x);~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Module 14 v1.4.3
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~Global String x = Date();~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Module 14 v1.4.3
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Global String x = Date();~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
How can you fix thisbut break that?
How can you fix thisbut break that?
Regression Testing
T t ill l d t b h h ki
-
8/8/2019 ISEB Software Testing Presentation
69/414
SIM Group Ltd., SQS Group AG, 2002
Tests will also need to be reTests will also need to be re
--run when checkingrun when checking
software upgradessoftware upgrades
Regression tests should be run whenever there is a changeRegression tests should be run whenever there is a changeto the software or the environmentto the software or the environment
Regression tests are executed to prove aspects of a systemRegression tests are executed to prove aspects of a systemhave not changedhave not changed
Regression Testing is a vital testing techniqueRegression Testing is a vital testing technique
-
8/8/2019 ISEB Software Testing Presentation
70/414
Regression Testing
Regression Testing is the ideal foundation for
-
8/8/2019 ISEB Software Testing Presentation
71/414
SIM Group Ltd., SQS Group AG, 2002
Regression Testing is the ideal foundation forRegression Testing is the ideal foundation for
AutomationAutomation
Selecting test cases is vital, and requires a degree ofSelecting test cases is vital, and requires a degree ofknowledge of the applications and their expected evolutionknowledge of the applications and their expected evolution
Re-Testing & Regression Testing
Summary
-
8/8/2019 ISEB Software Testing Presentation
72/414
SIM Group Ltd., SQS Group AG, 2002
SummarySummary
Once a fault has been fixed the software MUST be reOnce a fault has been fixed the software MUST be re--testedtested
New faults may have been introduced by the fixNew faults may have been introduced by the fix
Existing faults may have been uncovered by the fixExisting faults may have been uncovered by the fix
Tests need to be written to enable their reTests need to be written to enable their re--useuse
ReRe--testing is the rerunning of failed tests once a fix has beentesting is the rerunning of failed tests once a fix has been
implemented to check that the fix has workedimplemented to check that the fix has worked
Regression testing is running of a wider test suite to check forRegression testing is running of a wider test suite to check for
unexpected errors in unchanged codeunexpected errors in unchanged code
-
8/8/2019 ISEB Software Testing Presentation
73/414
Expected Results
In this session we willIn this session we will
-
8/8/2019 ISEB Software Testing Presentation
74/414
SIM Group Ltd., SQS Group AG, 2002
In this session we willIn this session we will
Understand the need to define expected resultsUnderstand the need to define expected results
Understand where expected results can be foundUnderstand where expected results can be found
Expected Results
Expected results = expectedExpected results = expected
-
8/8/2019 ISEB Software Testing Presentation
75/414
SIM Group Ltd., SQS Group AG, 2002
Expected results = expectedExpected results expected
outcomesoutcomes
Identifying required behaviourIdentifying required behaviour
If the expected outcome of aIf the expected outcome of a
test is not defined the actualtest is not defined the actual
output may be misinterpretedoutput may be misinterpreted
Expected Results
Running a test with only aRunning a test with only a
-
8/8/2019 ISEB Software Testing Presentation
76/414
SIM Group Ltd., SQS Group AG, 2002
Running a test with only ag y
general concept of thegeneral concept of the
outcome is fataloutcome is fatal
Expected Results
It is vital the Expected ResultsIt is vital the Expected Results
-
8/8/2019 ISEB Software Testing Presentation
77/414
SIM Group Ltd., SQS Group AG, 2002
pp
are defined with the tests,are defined with the tests,
before they are usedbefore they are used
You cannot decide whether aYou cannot decide whether a
test has passed just becausetest has passed just because
it looks rightit looks right
-
8/8/2019 ISEB Software Testing Presentation
78/414
Expected Results
Consult the Oracle!Consult the Oracle!
-
8/8/2019 ISEB Software Testing Presentation
79/414
SIM Group Ltd., SQS Group AG, 2002
Expected Results
SummarySummary
-
8/8/2019 ISEB Software Testing Presentation
80/414
SIM Group Ltd., SQS Group AG, 2002
Su a yy
Without defining expected results how do you know if a testWithout defining expected results how do you know if a test
has passed or failed?has passed or failed?
Expected results can be found in the system specification andExpected results can be found in the system specification and
by asking experienced business usersby asking experienced business users
-
8/8/2019 ISEB Software Testing Presentation
81/414
SIM Group Ltd., SQS Group AG, 2002
ISEB Foundation Certificate in Software Testing
Prioritisation of TestsPrioritisation of Tests
Prioritisation of Tests
In this session we willIn this session we will
-
8/8/2019 ISEB Software Testing Presentation
82/414
SIM Group Ltd., SQS Group AG, 2002
Understand why we need to prioritise testsUnderstand why we need to prioritise tests
Understand how we decide the priority of individual testsUnderstand how we decide the priority of individual tests
Prioritisation of tests
It is not possible to testIt is not possible to test
-
8/8/2019 ISEB Software Testing Presentation
83/414
SIM Group Ltd., SQS Group AG, 2002
everythingeverything
Therefore faults will getTherefore faults will get
through to the live systemthrough to the live system
We must do the best testingWe must do the best testing
possible in the time availablepossible in the time available
This means we mustThis means we must
prioritise and focus testingprioritise and focus testing
on the prioritieson the priorities
Aspects to Consider
SeveritySeverity Frequency of ChangeFrequency of Change
-
8/8/2019 ISEB Software Testing Presentation
84/414
SIM Group Ltd., SQS Group AG, 2002
ProbabilityProbability
VisibilityVisibility
Priority of RequirementsPriority of Requirements
Customer RequirementsCustomer Requirements
Vulnerability to errorVulnerability to error
Technical criticalityTechnical criticality
ComplexityComplexity
Resource availabilityResource availability
Business Criticality
What elements of theWhat elements of the
-
8/8/2019 ISEB Software Testing Presentation
85/414
SIM Group Ltd., SQS Group AG, 2002
application are essential toapplication are essential to
the success of thethe success of the
organisation?organisation?
-
8/8/2019 ISEB Software Testing Presentation
86/414
Technical Factors
How complex is it?How complex is it?
-
8/8/2019 ISEB Software Testing Presentation
87/414
SIM Group Ltd., SQS Group AG, 2002
How likely is an error?How likely is an error?
How often does this change?How often does this change?
Prioritisation of Tests
SummarySummary
-
8/8/2019 ISEB Software Testing Presentation
88/414
SIM Group Ltd., SQS Group AG, 2002
Due to the last minute nature of testing, it is necessary toDue to the last minute nature of testing, it is necessary to
prioritise testsprioritise tests
There will never be enough time to complete all testsThere will never be enough time to complete all tests
Therefore those tests that cover those areas deemed mostTherefore those tests that cover those areas deemed most
important (to the business, highest risk etc) must be testedimportant (to the business, highest risk etc) must be tested
firstfirst
-
8/8/2019 ISEB Software Testing Presentation
89/414
Models for Testing
In this session we willIn this session we will
-
8/8/2019 ISEB Software Testing Presentation
90/414
SIM Group Ltd., SQS Group AG, 2002
Look at two of the most commonly accepted testing modelsLook at two of the most commonly accepted testing models
-
8/8/2019 ISEB Software Testing Presentation
91/414
-
8/8/2019 ISEB Software Testing Presentation
92/414
-
8/8/2019 ISEB Software Testing Presentation
93/414
A simple V-Model
high level
-
8/8/2019 ISEB Software Testing Presentation
94/414
SIM Group Ltd., SQS Group AG, 2002
low level time
requirements
design
code unittest
system
test
UAT
TT
TT
TT
-
8/8/2019 ISEB Software Testing Presentation
95/414
Models for Testing
V-Model - where do we start testing?
-
8/8/2019 ISEB Software Testing Presentation
96/414
SIM Group Ltd., SQS Group AG, 2002
level
time
RequirementsRequirements
DesignDesign
CodeCode Unit TestUnit Test
System TestSystem Test
UATUATTT
TT
TT
Models for Testing
V-Model - Start Testing Early
-
8/8/2019 ISEB Software Testing Presentation
97/414
SIM Group Ltd., SQS Group AG, 2002
Reqs
level
time
TT
TT
TT
TT
TT
RR
RR
RR
RR
RR
PP
PP
RRPP TT
Spec
Code
UAT
System Test
Unit Test
Models for Testing
SummarySummary
-
8/8/2019 ISEB Software Testing Presentation
98/414
SIM Group Ltd., SQS Group AG, 2002
The VThe V--Model is the most common approaches model forModel is the most common approaches model for
testingtesting
There are other models used for testing, but these are lessThere are other models used for testing, but these are less
widely usedwidely used
Inc. V,V & TInc. V,V & T
Other models have been createdOther models have been created
-
8/8/2019 ISEB Software Testing Presentation
99/414
SIM Group Ltd., SQS Group AG, 2002
ISEB Foundation Certificate in Software Testing
Economics of TestingEconomics of Testing
-
8/8/2019 ISEB Software Testing Presentation
100/414
Economics of Testing
The earlier a fault is found, the cheaper it is to remedyThe earlier a fault is found, the cheaper it is to remedy
-
8/8/2019 ISEB Software Testing Presentation
101/414
SIM Group Ltd., SQS Group AG, 2002
Faults in the requirements can lead to major reFaults in the requirements can lead to major re--engineering ofengineering of
the entire systemthe entire system
Many faults can be found using reviewsMany faults can be found using reviews
Reviews of documentation and / or codeReviews of documentation and / or code
Allows testing to find moreAllows testing to find more substantialsubstantial faultsfaults
-
8/8/2019 ISEB Software Testing Presentation
102/414
Economics of Testing
In initial testing, defect fix times will average betweenIn initial testing, defect fix times will average between
10 to 20 minutes10 to 20 minutes
-
8/8/2019 ISEB Software Testing Presentation
103/414
SIM Group Ltd., SQS Group AG, 2002
10 to 20 minutes10 to 20 minutes
In integration testing each defect can cost an hour orIn integration testing each defect can cost an hour ormoremore
In system test each defect can cost 10 to 40 or moreIn system test each defect can cost 10 to 40 or more
engineer hoursengineer hours
Watts HumphreyWatts Humphrey
Economics of Testing
Early test design can prevent fault multiplicationEarly test design can prevent fault multiplication
-
8/8/2019 ISEB Software Testing Presentation
104/414
SIM Group Ltd., SQS Group AG, 2002
Analysis of specification during test preparation often bringsAnalysis of specification during test preparation often brings
faults in the specification to lightfaults in the specification to light
If faults in documentation are not found then the system mayIf faults in documentation are not found then the system may
be developed incorrectlybe developed incorrectly
-
8/8/2019 ISEB Software Testing Presentation
105/414
SIM Group Ltd., SQS Group AG, 2002
Economics of testing
An exampleAn example
Economics of Testing
Finding the error in the requirementsFinding the error in the requirements
-
8/8/2019 ISEB Software Testing Presentation
106/414
SIM Group Ltd., SQS Group AG, 2002
Change requirements documentChange requirements document
Economics of Testing
Finding the error in the specificationsFinding the error in the specifications
-
8/8/2019 ISEB Software Testing Presentation
107/414
SIM Group Ltd., SQS Group AG, 2002
Change requirements documentChange requirements document
Change specificationsChange specifications
-
8/8/2019 ISEB Software Testing Presentation
108/414
Economics of Testing
Finding the error in system testingFinding the error in system testing
-
8/8/2019 ISEB Software Testing Presentation
109/414
SIM Group Ltd., SQS Group AG, 2002
Change requirements documentChange requirements document
Change specificationsChange specifications
Change the codeChange the code
Change the tests, reChange the tests, re--test & regression testtest & regression test
Economics of Testing
Finding the error in UATFinding the error in UAT
-
8/8/2019 ISEB Software Testing Presentation
110/414
SIM Group Ltd., SQS Group AG, 2002
Change requirements documentChange requirements document
Change specificationsChange specifications
Change the codeChange the code
Change the tests, reChange the tests, re--test & regression testtest & regression test
ReRe--do UATdo UAT
Economics of Testing
Finding the error in LiveFinding the error in Live
Ch i t d tChange req irements doc ment
-
8/8/2019 ISEB Software Testing Presentation
111/414
SIM Group Ltd., SQS Group AG, 2002
Change requirements documentChange requirements document
Change specificationsChange specifications
Change the codeChange the code
Change the tests, reChange the tests, re--test & regression testtest & regression test
ReRe--do UATdo UAT
Plus...Plus...
Economics of Testing
Finding the error in LiveFinding the error in Live
-
8/8/2019 ISEB Software Testing Presentation
112/414
SIM Group Ltd., SQS Group AG, 2002
User to report and log faultUser to report and log fault
Users must be advised of fix, workUsers must be advised of fix, work--around etcaround etc
The live database may need to be fixedThe live database may need to be fixed
Deploying the new build may take hoursDeploying the new build may take hours
Loss of businessLoss of business
Economics of Testing
AdditionallyAdditionally
F l f d i di l i h h d
-
8/8/2019 ISEB Software Testing Presentation
113/414
SIM Group Ltd., SQS Group AG, 2002
Faults found in coding or later require changes to the codeFaults found in coding or later require changes to the code
librarylibrary
Documentation and help files will have to be changedDocumentation and help files will have to be changed
Every fault requires investigating and loggingEvery fault requires investigating and logging
Administration costsAdministration costs
Economics of Testing
The cost of not TestingThe cost of not Testing
Wh i h f i ?Wh t i th t f t ti ?
-
8/8/2019 ISEB Software Testing Presentation
114/414
SIM Group Ltd., SQS Group AG, 2002
What is the cost of testing?What is the cost of testing?
Which is cheaper?Which is cheaper?
Testing and finding faults before releaseTesting and finding faults before release
Not testing and finding faults once the system is liveNot testing and finding faults once the system is live
Companies do not typically have figures for eitherCompanies do not typically have figures for either
Economics of Testing
SummarySummary
Th f t ti i t fi d f ltTh f t ti i t fi d f lt
-
8/8/2019 ISEB Software Testing Presentation
115/414
SIM Group Ltd., SQS Group AG, 2002
The purpose of testing is to find faultsThe purpose of testing is to find faults
The earlier a fault is found the cheaper (and easier) it is to fThe earlier a fault is found the cheaper (and easier) it is to fixix
Starting testing early will help find faults quickerStarting testing early will help find faults quicker
-
8/8/2019 ISEB Software Testing Presentation
116/414
SIM Group Ltd., SQS Group AG, 2002
ISEB Foundation Certificate in Software Testing
High Level Test PlanningHigh Level Test Planning
High Level Test Planning
In this session we willIn this session we will
Look at how a test plan is put togetherLook at how a test plan is put together
-
8/8/2019 ISEB Software Testing Presentation
117/414
SIM Group Ltd., SQS Group AG, 2002
Look at how a test plan is put togetherLook at how a test plan is put together
Understand how it should be used and maintainedUnderstand how it should be used and maintained
Understand why they are so important to a testing projectUnderstand why they are so important to a testing project
High Level Test Planning
What is a test plan?What is a test plan?
A project plan for testingA project plan for testing
-
8/8/2019 ISEB Software Testing Presentation
118/414
SIM Group Ltd., SQS Group AG, 2002
A project plan for testingA project plan for testing
Covering all aspects of testingCovering all aspects of testing
-
8/8/2019 ISEB Software Testing Presentation
119/414
The Different Sections of a Test Plan
Test plan identifierTest plan identifier
IntroductionIntroduction
T t itT t it
Test deliverablesTest deliverables
Testing tasksTesting tasks
EnvironmentEnvironment
-
8/8/2019 ISEB Software Testing Presentation
120/414
SIM Group Ltd., SQS Group AG, 2002
Test itemsTest items
Features to be testedFeatures to be tested
Features not to be testedFeatures not to be tested
ApproachApproach
Item pass / fail criteriaItem pass / fail criteria
Suspension criteria &Suspension criteria &
resumption criteriaresumption criteria
EnvironmentEnvironment
ResponsibilitiesResponsibilities
Staffing and training needsStaffing and training needs
SchedulesSchedules
Risks and contingenciesRisks and contingencies ApprovalsApprovals
Based upon IEEE 829Based upon IEEE 829--1998 standard for1998 standard forsoftware test documentationsoftware test documentation
-
8/8/2019 ISEB Software Testing Presentation
121/414
-
8/8/2019 ISEB Software Testing Presentation
122/414
Test Items
The various items to be used in the testsThe various items to be used in the tests
The software itemsThe software items
-
8/8/2019 ISEB Software Testing Presentation
123/414
SIM Group Ltd., SQS Group AG, 2002
The software itemsThe software items
Their versions numbers / identifiersTheir versions numbers / identifiers
How they will be handed over to testingHow they will be handed over to testing
References to relevant documentsReferences to relevant documents
Features to Be Tested
List all features of the SUT that will be tested under thisList all features of the SUT that will be tested under this
planplan
-
8/8/2019 ISEB Software Testing Presentation
124/414
SIM Group Ltd., SQS Group AG, 2002
Features Not to Be Tested
List all features of the SUT that will not be tested underList all features of the SUT that will not be tested under
this planthis plan
-
8/8/2019 ISEB Software Testing Presentation
125/414
SIM Group Ltd., SQS Group AG, 2002
Between the test items, features to be tested and features not tBetween the test items, features to be tested and features not t
o beo be
tested we have scope of the projecttested we have scope of the project
-
8/8/2019 ISEB Software Testing Presentation
126/414
-
8/8/2019 ISEB Software Testing Presentation
127/414
Suspension criteria & resumption requirements
Reasons that would cause testing to be suspendedReasons that would cause testing to be suspended
Steps necessary to resume testingSteps necessary to resume testing
-
8/8/2019 ISEB Software Testing Presentation
128/414
SIM Group Ltd., SQS Group AG, 2002
p y gp y g
Test Deliverables
Everything that goes to make up the testsEverything that goes to make up the tests
All documentationAll documentation
-
8/8/2019 ISEB Software Testing Presentation
129/414
SIM Group Ltd., SQS Group AG, 2002
e.g. Specification, test plans, procedures, reportse.g. Specification, test plans, procedures, reports
Code releasesCode releases
Testing toolsTesting tools
Test management tools, automation tools, excel, word etcTest management tools, automation tools, excel, word etc
Test systemsTest systems
Manual and automated test casesManual and automated test cases
Testing Tasks
Preparation to perform testingPreparation to perform testing
Test case identificationTest case identification Test case designTest case design
-
8/8/2019 ISEB Software Testing Presentation
130/414
SIM Group Ltd., SQS Group AG, 2002
Test data storageTest data storage
Baseline applicationBaseline application
Special skills neededSpecial skills needed
Spreadsheet skills, test analysis, automation etcSpreadsheet skills, test analysis, automation etc
InterInter--dependenciesdependencies
Environment
Requirements for test environmentRequirements for test environment
Hardware & softwareHardware & software
-
8/8/2019 ISEB Software Testing Presentation
131/414
SIM Group Ltd., SQS Group AG, 2002
PCs, servers, routers etcPCs, servers, routers etc
SUT, interfaced applications, databasesSUT, interfaced applications, databases
ConfigurationConfiguration
Maybe operating systems or middleware to test againstMaybe operating systems or middleware to test against
FacilitiesFacilities
Office space, desks, internet accessOffice space, desks, internet access
-
8/8/2019 ISEB Software Testing Presentation
132/414
Staffing and Training Needs
Staff requiredStaff required
Test managers, team leaders, testers, test analystsTest managers, team leaders, testers, test analysts
-
8/8/2019 ISEB Software Testing Presentation
133/414
SIM Group Ltd., SQS Group AG, 2002
Skill levels requiredSkill levels required
Automation experienceAutomation experience
Spreadsheet skills, etcSpreadsheet skills, etc
Training requirementsTraining requirements
Tools specific trainingTools specific training
Refresher courses, etc.Refresher courses, etc.
Overall project resource requirementsOverall project resource requirements
Schedule
Timescales, dates and milestonesTimescales, dates and milestones
Resources required to meet milestonesResources required to meet milestones
-
8/8/2019 ISEB Software Testing Presentation
134/414
SIM Group Ltd., SQS Group AG, 2002
Availability of software and environmentAvailability of software and environment
DeliverablesDeliverables
Risks and Contingencies
What might go wrong?What might go wrong?
Actions for minimising impact on testing should things goActions for minimising impact on testing should things go
-
8/8/2019 ISEB Software Testing Presentation
135/414
SIM Group Ltd., SQS Group AG, 2002
wrongwrong
Approvals
Who has approved the test planWho has approved the test plan
Names and dates of approvalNames and dates of approval
-
8/8/2019 ISEB Software Testing Presentation
136/414
SIM Group Ltd., SQS Group AG, 2002
Why is it so importantWhy is it so important
Evidence that the document has been viewedEvidence that the document has been viewed
Shows that the approach has been agreed and has the backingShows that the approach has been agreed and has the backing
of those who matterof those who matter
You have commitment, now make them stick to it!You have commitment, now make them stick to it!
-
8/8/2019 ISEB Software Testing Presentation
137/414
ISEB Foundation Certificate in Software Testing
-
8/8/2019 ISEB Software Testing Presentation
138/414
SIM Group Ltd., SQS Group AG, 2002
ISEB Foundation Certificate in Software Testing
Acceptance TestingAcceptance Testing
Acceptance Testing
In this session we willIn this session we will
Understand what acceptance testing isUnderstand what acceptance testing is
Wh ld d i
-
8/8/2019 ISEB Software Testing Presentation
139/414
SIM Group Ltd., SQS Group AG, 2002
Why you would want to do itWhy you would want to do it
How you would plan it and prepare for itHow you would plan it and prepare for it
What you need to actually do itWhat you need to actually do it
Understand the different types of acceptance testingUnderstand the different types of acceptance testing
Introduction
What is Acceptance Testing ?What is Acceptance Testing ?
"Formal testing conducted to enable a user, customer or other"Formal testing conducted to enable a user, customer or other
authorised entity to determine whether to accept a system orauthorised entity to determine whether to accept a system or
-
8/8/2019 ISEB Software Testing Presentation
140/414
SIM Group Ltd., SQS Group AG, 2002
authorised entity to determine whether to accept a system orauthorised entity to determine whether to accept a system or
component."component."
User Acceptance Testing
What is User Acceptance Testing (UAT)?What is User Acceptance Testing (UAT)?
Exactly what it says it is!!Exactly what it says it is!!
-
8/8/2019 ISEB Software Testing Presentation
141/414
SIM Group Ltd., SQS Group AG, 2002
Users of end product conducting the testsUsers of end product conducting the tests
The setThe set--up will represent a working Environmentup will represent a working Environment
Covers all areas of a project, not just the systemCovers all areas of a project, not just the system
A.K.A Business Acceptance Testing or Business ProcessA.K.A Business Acceptance Testing or Business Process
TestingTesting
UnitTest
Where does it fit in?
-
8/8/2019 ISEB Software Testing Presentation
142/414
SIM Group Ltd., SQS Group AG, 2002
SystemsTest
UAT
Alpha /
Beta
Service LevelTesting
Operational User Acceptance
BusinessRequirements
A typical development life-cycle V-Model
-
8/8/2019 ISEB Software Testing Presentation
143/414
SIM Group Ltd., SQS Group AG, 2002
level
time
Operational
Requirements
DesignSpecifications
FunctionalSpecification
ComponentTesting
SystemTesting
IntegrationTesting (large)
User Acceptance
Testing
ProgramSpecification
-
8/8/2019 ISEB Software Testing Presentation
144/414
Why Plan?
If you donIf you dont then how do you know you have achievedt then how do you know you have achieved
what you set out to do!what you set out to do!
Avoids repetitionAvoids repetition
-
8/8/2019 ISEB Software Testing Presentation
145/414
SIM Group Ltd., SQS Group AG, 2002
Avoids repetitionAvoids repetition
Test according to code releasesTest according to code releases
Makes efficient and effective use of time and resourcesMakes efficient and effective use of time and resources
Things to consider
TimescalesTimescales
ResourcesResources
-
8/8/2019 ISEB Software Testing Presentation
146/414
SIM Group Ltd., SQS Group AG, 2002
AvailabilityAvailability
Preparing your tests
Take a logical approachTake a logical approach
Identify the business processesIdentify the business processes
-
8/8/2019 ISEB Software Testing Presentation
147/414
SIM Group Ltd., SQS Group AG, 2002
Separate into everyday business scenariosSeparate into everyday business scenarios
How the business fits together...
The Business
-
8/8/2019 ISEB Software Testing Presentation
148/414
SIM Group Ltd., SQS Group AG, 2002
Process Process Process
Task Task Task Task Task Task Task TaskTask
-
8/8/2019 ISEB Software Testing Presentation
149/414
Running the Tests
Order of TestsOrder of Tests
Confidence ChecksConfidence Checks
-
8/8/2019 ISEB Software Testing Presentation
150/414
SIM Group Ltd., SQS Group AG, 2002
Automated and Manual test runsAutomated and Manual test runs
Contract Acceptance
A demonstration of the acceptance criteriaA demonstration of the acceptance criteria
Acceptance criteria will have been defined in the contractAcceptance criteria will have been defined in the contract
-
8/8/2019 ISEB Software Testing Presentation
151/414
SIM Group Ltd., SQS Group AG, 2002
Before the software is accepted it is necessary to show thatBefore the software is accepted it is necessary to show thatits matches itsits matches its specification as defined in the criteriaspecification as defined in the criteria
Alpha Testing
Letting your customers do your testingLetting your customers do your testing
Requires a stable version of the softwareRequires a stable version of the software
-
8/8/2019 ISEB Software Testing Presentation
152/414
SIM Group Ltd., SQS Group AG, 2002
People who represent your target market use the product inPeople who represent your target market use the product inthe same way(s) as if they had bought the finished versionthe same way(s) as if they had bought the finished version
Alpha testing is conducted inAlpha testing is conducted in--house (at the developers site)house (at the developers site)
Beta Testing
As Alpha testing but users perform tests at their siteAs Alpha testing but users perform tests at their site
-
8/8/2019 ISEB Software Testing Presentation
153/414
SIM Group Ltd., SQS Group AG, 2002
Acceptance Testing
SummarySummary
You can UAT anything!You can UAT anything!
-
8/8/2019 ISEB Software Testing Presentation
154/414
SIM Group Ltd., SQS Group AG, 2002
Every Deliverable should pass through UATEvery Deliverable should pass through UAT
User representation is VITALUser representation is VITAL
If the product does not pass UAT then a decision aboutIf the product does not pass UAT then a decision about
implementation needs to be madeimplementation needs to be made
ISEB Foundation Certificate in Software Testing
-
8/8/2019 ISEB Software Testing Presentation
155/414
SIM Group Ltd., SQS Group AG, 2002
Integration Testing in the LargeIntegration Testing in the Large
-
8/8/2019 ISEB Software Testing Presentation
156/414
Integration Testing in the Large
Integration testingIntegration testing Testing performed to expose faults in the interfaces and in theTesting performed to expose faults in the interfaces and in the
interaction between integrated componentsinteraction between integrated components
-
8/8/2019 ISEB Software Testing Presentation
157/414
SIM Group Ltd., SQS Group AG, 2002
Integration testing in the largeIntegration testing in the large
Integration Testing in the Large is testing performed to exposeIntegration Testing in the Large is testing performed to expose
faults in the interfaces and in the interaction between systemsfaults in the interfaces and in the interaction between systems
Why do we need to do integration testing in the largeWhy do we need to do integration testing in the large
Just because the SUT works in the test lab doesnJust because the SUT works in the test lab doesnt mean it willt mean it willwork in the real worldwork in the real world
-
8/8/2019 ISEB Software Testing Presentation
158/414
Env
EnvEnv
EnvEnv
Env
SUT
SUT SUT
SUT SUT
SUT
A typical system
Data Model
Data Model
-
8/8/2019 ISEB Software Testing Presentation
159/414
SIM Group Ltd., SQS Group AG, 2002
DatabasesDatabases
DataData Ext apps.Ext apps.
Integration Testing in the Large
Integration Testing on a Large ScaleIntegration Testing on a Large Scale
Test the way the SUT interfaces with external systemsTest the way the SUT interfaces with external systems
-
8/8/2019 ISEB Software Testing Presentation
160/414
SIM Group Ltd., SQS Group AG, 2002
The elements may run on the same and / or differentThe elements may run on the same and / or differentplatformsplatforms
They may be located in separate locations and useThey may be located in separate locations and usecommunication protocols to pass data back & forthcommunication protocols to pass data back & forth
Integration Testing in the Large
Typical approachTypical approach
Test the systemTest the system
In a test environmentIn a test environment
Use stubs & drivers where external systems arenUse stubs & drivers where external systems arent availablet available
-
8/8/2019 ISEB Software Testing Presentation
161/414
SIM Group Ltd., SQS Group AG, 2002
Use stubs & drivers where external systems arenUse stubs & drivers where external systems aren t availablet available
Test the systemTest the system in situin situ
In a replica of the production / live environmentIn a replica of the production / live environment
Use stubs & drivers where external systems arenUse stubs & drivers where external systems arent availablet available
Test the system and itsTest the system and its interfaces with other systemsinterfaces with other systems
In a replica of the production / live environmentIn a replica of the production / live environment
Access test versions of the external systemsAccess test versions of the external systems
Integration Testing in the Large
Fundamentally we are still looking atFundamentally we are still looking at
DATADATA
-
8/8/2019 ISEB Software Testing Presentation
162/414
SIM Group Ltd., SQS Group AG, 2002
& how that data is obtained, processed, manipulated, stored& how that data is obtained, processed, manipulated, stored
and made available or transported for use elsewhere within orand made available or transported for use elsewhere within or
outside theoutside the organisationorganisation
Integration Testing in the Large
Additional challenges posed by large scale:Additional challenges posed by large scale:
Multiple PlatformsMultiple Platforms
Communications between platformsCommunications between platforms
-
8/8/2019 ISEB Software Testing Presentation
163/414
SIM Group Ltd., SQS Group AG, 2002
Management of the environmentsManagement of the environments
Coordination of changesCoordination of changes
Different technical skills requiredDifferent technical skills required
The approach used
-
8/8/2019 ISEB Software Testing Presentation
164/414
SIM Group Ltd., SQS Group AG, 2002
Integration Testing
Planning testingPlanning testing
We need to determine when and at what levels IntegrationWe need to determine when and at what levels Integration
Testing will be performedTesting will be performed
-
8/8/2019 ISEB Software Testing Presentation
165/414
SIM Group Ltd., SQS Group AG, 2002
Identify the boundariesIdentify the boundaries
A similar process will need to be followed for all elementsA similar process will need to be followed for all elements Once these elements have been tested individually they areOnce these elements have been tested individually they are
further integrated to support the business processfurther integrated to support the business process
Integration Testing
Approaches to integration testingApproaches to integration testing
IncrementalIncremental
Top downTop down
-
8/8/2019 ISEB Software Testing Presentation
166/414
SIM Group Ltd., SQS Group AG, 2002
BottomBottom--upup FunctionalFunctional
NonNon--incrementalincremental
Big bangBig bang
Integration Testing in the Large
SummarySummary
Integration testing is needed to test how components interactIntegration testing is needed to test how components interact
with each other and how the system under test works withwith each other and how the system under test works with
other systemsother systems
-
8/8/2019 ISEB Software Testing Presentation
167/414
SIM Group Ltd., SQS Group AG, 2002
y
Large scale integration testing is needed to test how theLarge scale integration testing is needed to test how the
system under test works with other systemssystem under test works with other systems
Just because a system works in a test environment doesnJust because a system works in a test environment doesntt
mean it will work in a live environmentmean it will work in a live environment
ISEB Foundation Certificate in Software Testing
-
8/8/2019 ISEB Software Testing Presentation
168/414
SIM Group Ltd., SQS Group AG, 2002
NonNon--Functional System TestingFunctional System Testing
Non-Functional System Testing
In this session we willIn this session we will
Understand what NonUnderstand what Non--Functional System Testing isFunctional System Testing is
-
8/8/2019 ISEB Software Testing Presentation
169/414
SIM Group Ltd., SQS Group AG, 2002
Understand the need for nonUnderstand the need for non--functional system testingfunctional system testing
Look at the various types of nonLook at the various types of non--functional system testingfunctional system testing
Non-Functional System Testing
NonNon--Functional System Testing is defined as:Functional System Testing is defined as:
Testing Of Those requirements that do not relate to theTesting Of Those requirements that do not relate to the
Functionality e.g. Performance, Load, usability, security etcFunctionality e.g. Performance, Load, usability, security etc
-
8/8/2019 ISEB Software Testing Presentation
170/414
SIM Group Ltd., SQS Group AG, 2002
Users, generally, focus on the Functionality of the systemUsers, generally, focus on the Functionality of the system
-
8/8/2019 ISEB Software Testing Presentation
171/414
Security Testing
Testing whether the system meets specified securityTesting whether the system meets specified security
requirementsrequirements
How easy is it for an unauthorised user to access theHow easy is it for an unauthorised user to access the
system?system?
-
8/8/2019 ISEB Software Testing Presentation
172/414
SIM Group Ltd., SQS Group AG, 2002
How easy is it for an unauthorised person to access the data?How easy is it for an unauthorised person to access the data?
Usability Testing
Testing the ease with which users can learn and use theTesting the ease with which users can learn and use the
productproduct
Employ people to use the application as a user and studyEmploy people to use the application as a user and study
how they use ithow they use it
-
8/8/2019 ISEB Software Testing Presentation
173/414
SIM Group Ltd., SQS Group AG, 2002
A betaA beta--test may be a cheap way of doing Usability testingtest may be a cheap way of doing Usability testing
There is no simple way to examine HOW people use theThere is no simple way to examine HOW people use the
productproduct
Storage Testing
Testing whether the system meets its specified storageTesting whether the system meets its specified storage
objectivesobjectives
Study how memory and storage is used by the productStudy how memory and storage is used by the product
-
8/8/2019 ISEB Software Testing Presentation
174/414
SIM Group Ltd., SQS Group AG, 2002
Predict when extra capacity may be neededPredict when extra capacity may be needed
Installability Testing
Testing concerned with the installation procedures for theTesting concerned with the installation procedures for the
systemsystem
Does the installation work?Does the installation work?
-
8/8/2019 ISEB Software Testing Presentation
175/414
SIM Group Ltd., SQS Group AG, 2002
Is it easy to carry out?Is it easy to carry out?
Does it uninstall correctly?Does it uninstall correctly?
Documentation Testing
Testing concerned with the accuracy of the documentationTesting concerned with the accuracy of the documentation
-
8/8/2019 ISEB Software Testing Presentation
176/414
SIM Group Ltd., SQS Group AG, 2002
Recovery Testing
Testing aimed at verifying the systemTesting aimed at verifying the systems ability to recover froms ability to recover from
varying degrees of failurevarying degrees of failure
Should include recovery from system backShould include recovery from system back--upsups
-
8/8/2019 ISEB Software Testing Presentation
177/414
SIM Group Ltd., SQS Group AG, 2002
-
8/8/2019 ISEB Software Testing Presentation
178/414
Stress Testing
Assesses individual components by exercising them to andAssesses individual components by exercising them to and
beyond the limits of expected usebeyond the limits of expected use
-
8/8/2019 ISEB Software Testing Presentation
179/414
SIM Group Ltd., SQS Group AG, 2002
Performance Testing
Tests the efficiency of individual components of an applicationTests the efficiency of individual components of an application
-
8/8/2019 ISEB Software Testing Presentation
180/414
SIM Group Ltd., SQS Group AG, 2002
Volume Testing
Testing where the system is subjected to large volumes ofTesting where the system is subjected to large volumes of
datadata
-
8/8/2019 ISEB Software Testing Presentation
181/414
SIM Group Ltd., SQS Group AG, 2002
Non-Functional System Testing
SummarySummary
Just because a systemJust because a systems functions have been tested doesns functions have been tested doesntt
mean that testing is completemean that testing is complete
-
8/8/2019 ISEB Software Testing Presentation
182/414
SIM Group Ltd., SQS Group AG, 2002
There are a range of nonThere are a range of non--functional tests that need to befunctional tests that need to be
performed upon a systemperformed upon a system
ISEB Foundation Certificate in Software Testing
-
8/8/2019 ISEB Software Testing Presentation
183/414
SIM Group Ltd., SQS Group AG, 2002
Functional System TestingFunctional System Testing
-
8/8/2019 ISEB Software Testing Presentation
184/414
Functional System Testing
What is System Testing?What is System Testing?
Testing of the complete systemTesting of the complete system
May be the last step in integration testing in the smallMay be the last step in integration testing in the small
May be the first time that enough of the system has been putMay be the first time that enough of the system has been puttogether to make a working systemtogether to make a working system
-
8/8/2019 ISEB Software Testing Presentation
185/414
SIM Group Ltd., SQS Group AG, 2002
y g y py g y ptogether to make a working systemtogether to make a working system
Ideally done by an independent test teamIdeally done by an independent test team
Two typesTwo types -- functional and nonfunctional and non--functionalfunctional
Functional System Testing
A Functional Requirement isA Functional Requirement is
A requirement that specifies a function that a system or systemA requirement that specifies a function that a system or system
component must performcomponent must perform
-
8/8/2019 ISEB Software Testing Presentation
186/414
SIM Group Ltd., SQS Group AG, 2002
Functional System testing is geared to checking the functionFunctional System testing is geared to checking the function
of the system against specificationof the system against specification
May be requirements based or business process basedMay be requirements based or business process based
Functional System Testing
Testing Based on RequirementsTesting Based on Requirements
Requirements specification used to derive test casesRequirements specification used to derive test cases
System is tested to ensure the requirements can be metSystem is tested to ensure the requirements can be met
-
8/8/2019 ISEB Software Testing Presentation
187/414
SIM Group Ltd., SQS Group AG, 2002
y qy q
Functional System Testing
Testing carried out against Business processesTesting carried out against Business processes
Based on expected use of the systemBased on expected use of the system
Builds use casesBuilds use cases -- test cases that reflect actual or expectedtest cases that reflect actual or expecteduse of the systemuse of the system
-
8/8/2019 ISEB Software Testing Presentation
188/414
SIM Group Ltd., SQS Group AG, 2002
ppuse of the systemuse of the system
-
8/8/2019 ISEB Software Testing Presentation
189/414
SIM Group Ltd., SQS Group AG, 2002
-
8/8/2019 ISEB Software Testing Presentation
190/414
ISEB Foundation Certificate in Software Testing
-
8/8/2019 ISEB Software Testing Presentation
191/414
SIM Group Ltd., SQS Group AG, 2002
Integration Testing in the SmallIntegration Testing in the Small
Integration Testing in the Small
In this session we willIn this session we will
Understand whatUnderstand what Integration Testing in the SmallIntegration Testing in the Small isis
Look at how & why we doLook at how & why we do Integration Testing in the SmallIntegration Testing in the Small
-
8/8/2019 ISEB Software Testing Presentation
192/414
SIM Group Ltd., SQS Group AG, 2002
Integration Testing in the Small
Integration testingIntegration testing
Testing performed to expose faults in the interfaces and in theTesting performed to expose faults in the interfaces and in the
interaction between integrated componentsinteraction between integrated components
Integration testing in the smallIntegration testing in the small
-
8/8/2019 ISEB Software Testing Presentation
193/414
SIM Group Ltd., SQS Group AG, 2002
g g
Testing performed to expose faults in the interfaces and in theTesting performed to expose faults in the interfaces and in the
interaction between componentsinteraction between components
A sample system
DatabasesDatabases
-
8/8/2019 ISEB Software Testing Presentation
194/414
SIM Group Ltd., SQS Group AG, 2002
Externalsystems
Externalsystems
The SUT
- made up of 000s of units
The SUT
- made up of 000s of units
Integration Testing in the Small
All components of a computer application work on onlyAll components of a computer application work on only
one thingone thing
DATADATA
Data is passed from one component to anotherData is passed from one component to another
-
8/8/2019 ISEB Software Testing Presentation
195/414
SIM Group Ltd., SQS Group AG, 2002
Data is passed from one component to anotherData is passed from one component to another
Data is passed from one system to anotherData is passed from one system to another
Software components will be required to add, amend, &Software components will be required to add, amend, &
delete information, validate the information and report on thedelete information, validate the information and report on theinformationinformation
Integration Testing in the Small
Integration testingIntegration testing
Testing should start with the smallest, definable componentTesting should start with the smallest, definable component
Building tested components up to the next level of definitionBuilding tested components up to the next level of definition
-
8/8/2019 ISEB Software Testing Presentation
196/414
SIM Group Ltd., SQS Group AG, 2002
Eventually the complete business processes is testedEventually the complete business processes is tested End toEnd to
EndEnd
-
8/8/2019 ISEB Software Testing Presentation
197/414
Integration Testing in the Small
Once components have been tested we can link themOnce components have been tested we can link them
together to see if they can work togethertogether to see if they can work together
This is Integration Testing in the SmallThis is Integration Testing in the Small
Linking components together to ensure that they communicateLinking components together to ensure that they communicatecorrectlycorrectly
-
8/8/2019 ISEB Software Testing Presentation
198/414
SIM Group Ltd., SQS Group AG, 2002
Gradually linking more and more components togetherGradually linking more and more components together
Prove that the data is passed between components as requiredProve that the data is passed between components as required
Steadily increase the number of components and create & testSteadily increase the number of components and create & test
subsub--systems and then the complete systemsystems and then the complete system
The approach used
-
8/8/2019 ISEB Software Testing Presentation
199/414
SIM Group Ltd., SQS Group AG, 2002
Integration Testing
Planning testingPlanning testing
We need to determine when and at what levels IntegrationWe need to determine when and at what levels Integration
Testing will be performedTesting will be performed
Identify the boundariesIdentify the boundaries
-
8/8/2019 ISEB Software Testing Presentation
200/414
SIM Group Ltd., SQS Group AG, 2002
A similar process will need to be followed for all elementsA similar process will need to be followed for all elements Once these elements have been tested individually they areOnce these elements have been tested individually they are
further integrated to support the business processfurther integrated to support the business process
Integration Testing
Stubs and driversStubs and drivers
Used to emulate theUsed to emulate the missing bitsmissing bits
External systemsExternal systems
SubSub--systemssystems Missing componentsMissing components
-
8/8/2019 ISEB Software Testing Presentation
201/414
SIM Group Ltd., SQS Group AG, 2002
May need to be written specifically for the SUTMay need to be written specifically for the SUT
Integration Testing
Approaches to integration testingApproaches to integration testing
IncrementalIncremental
Top downTop down
BottomBottom--upup FunctionalFunctional
-
8/8/2019 ISEB Software Testing Presentation
202/414
SIM Group Ltd., SQS Group AG, 2002
NonNon--incrementalincremental Big bangBig bang
Integration Testing in the Small
SummarySummary
Integration testing is needed to test how components interactIntegration testing is needed to test how components interact
with each other and how the system under test works withwith each other and how the system under test works with
other systemsother systems
I i i i h ll i d i h hI t ti t ti i th ll i d ith th
-
8/8/2019 ISEB Software Testing Presentation
203/414
SIM Group Ltd., SQS Group AG, 2002
Integration testing in the small is concerned with theIntegration testing in the small is concerned with the
interaction between the various components of a systeminteraction between the various components of a system
ISEB Foundation Certificate in Software Testing
-
8/8/2019 ISEB Software Testing Presentation
204/414
SIM Group Ltd., SQS Group AG, 2002
Component TestingComponent Testing
Component Testing
In this session we willIn this session we will
Understand what Component Testing isUnderstand what Component Testing is
Look at the Component Testing ProcessLook at the Component Testing Process
-
8/8/2019 ISEB Software Testing Presentation
205/414
SIM Group Ltd., SQS Group AG, 2002
Look at the myriad types of Component TestingLook at the myriad types of Component Testing
Component Testing
What is component testing?What is component testing?
The testing of an individual software componentThe testing of an individual software component
What is a component?What is a component?
-
8/8/2019 ISEB Software Testing Presentation
206/414
SIM Group Ltd., SQS Group AG, 2002
A minimal software item for which a separate specification isA minimal software item for which a separate specification is
availableavailable
Component Testing
Component testing is the lowest form of testingComponent testing is the lowest form of testing
At the bottom of the VAt the bottom of the V--modelmodel
Each component is tested in isolationEach component is tested in isolation Prior to being integrated into the systemPrior to being integrated into the system
-
8/8/2019 ISEB Software Testing Presentation
207/414
SIM Group Ltd., SQS Group AG, 2002
Involves testing the code itselfInvolves testing the code itself Test the code in the gr