test analysis & design – good practices
DESCRIPTION
- PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Test Analysis & Design – good practices](https://reader037.vdocuments.net/reader037/viewer/2022110210/56812bc9550346895d901bfe/html5/thumbnails/1.jpg)
Test Analysis & Design
– good practices
Raluca Gagea2013, October
17th
We all know the importance of doing the right things from the very beginning and of putting the right questions when the impact on the product under development is still minor. From here to test analysis and test cases design is just a small step, but one that needs special attention, experience, intuition and creativity and many good practices.
I won't "teach" you how to do it best, I'll share with you some tips that helped me during testing and we'll try to cover:& some vocab stuff we usually
don't use or we do it wrong& specific activities and their
benefits& ways to measure progress and
report the benefits& test cases writing/designing
styles - advantages and disadvantages
& important properties for test cases
& few mistakes we all do :)& importance of tools and the
testing structures we choose
![Page 2: Test Analysis & Design – good practices](https://reader037.vdocuments.net/reader037/viewer/2022110210/56812bc9550346895d901bfe/html5/thumbnails/2.jpg)
There is an A for everything
Theoretical side of things
Definitions Vocab ISTQB
Good practices
Things to remember
Lessons to learn
Practical (realistic ) side of things
Activities and their benefits
Examples
![Page 3: Test Analysis & Design – good practices](https://reader037.vdocuments.net/reader037/viewer/2022110210/56812bc9550346895d901bfe/html5/thumbnails/3.jpg)
Fundamental Test Process
CONTROL
PLANNING
ANALYSIS DESIGN IMPLEME
NTCLOSU
RE
EXECUTION
EVALUATING EXIT CRITERIA AND REPORTING
![Page 4: Test Analysis & Design – good practices](https://reader037.vdocuments.net/reader037/viewer/2022110210/56812bc9550346895d901bfe/html5/thumbnails/4.jpg)
Test Analysis & Design
Test Analysis & Design is the activity where general testing objectives are transformed into tangible test conditions and test designs.
Test Analysis
Process of looking at
something that can be used to
derive test information
Test Design
Process of identifying the
associated high-level test cases for a test item
![Page 5: Test Analysis & Design – good practices](https://reader037.vdocuments.net/reader037/viewer/2022110210/56812bc9550346895d901bfe/html5/thumbnails/5.jpg)
Test Analysis & Design Vocab
Use casesFunction
al specificationsNon-
functional specifications
Business Scenarios
Emails
Technical documents
Test BasisTest
Object 1
Test Object
2
Test Object
nTest
Item 1Test
Item nTest conditio
n 1
Test conditio
n nTest case
1
Test case
nEffective test case
1
Test case design
techniques Effective test case n
Test suite
Input & Output
Test data
TEST
OBJECTIVES
![Page 6: Test Analysis & Design – good practices](https://reader037.vdocuments.net/reader037/viewer/2022110210/56812bc9550346895d901bfe/html5/thumbnails/6.jpg)
Test Analysis & Design Vocab
User Name
Age
City
Postal CodeSubmit
Subscription Form
Input constraintsUser Name must be between 6 and 12 characters long, must start with a letter and include only digits.Age must be a number greater or equal to 18 and less than 65City must be one of Ottava, Toronto, Montreal or Halifax
Test Object
Test Items
Test Conditions
![Page 7: Test Analysis & Design – good practices](https://reader037.vdocuments.net/reader037/viewer/2022110210/56812bc9550346895d901bfe/html5/thumbnails/7.jpg)
Test Analysis & Design – Why?
Test Basis
Review test basis
Examine specifications
Evaluate testability
Select relevant documents only; Identify gaps and ambiguities in the specifications because we are trying to identify precisely what happens at each point in the system
Prevents defects appearing in the code
At execution moment, all the requirements are translated in terms of testable items
![Page 8: Test Analysis & Design – good practices](https://reader037.vdocuments.net/reader037/viewer/2022110210/56812bc9550346895d901bfe/html5/thumbnails/8.jpg)
Test Analysis & Design – Why?
Test Conditions
Analysis of test items
Identify test conditions Gives us a high-level
list of what we are interested in testing
We can start identifying the type of generic test data we might need
![Page 9: Test Analysis & Design – good practices](https://reader037.vdocuments.net/reader037/viewer/2022110210/56812bc9550346895d901bfe/html5/thumbnails/9.jpg)
Test Analysis & Design – Why?
Test Cases
Design the tests
Use test design techniques The high risks areas
will be covered by tests before the actual execution phase starts
![Page 10: Test Analysis & Design – good practices](https://reader037.vdocuments.net/reader037/viewer/2022110210/56812bc9550346895d901bfe/html5/thumbnails/10.jpg)
Test Analysis & Design – Why?
Test Data
Identify test data
At execution moment, the test cases will be executed using the most closest data than the one in production
![Page 11: Test Analysis & Design – good practices](https://reader037.vdocuments.net/reader037/viewer/2022110210/56812bc9550346895d901bfe/html5/thumbnails/11.jpg)
Test Analysis & Design – Why?T
est Environment
Design the environment set-up
Identify any infrastructure and tools
Availability
At execution moment, everything we need to carry out our work is in place
![Page 12: Test Analysis & Design – good practices](https://reader037.vdocuments.net/reader037/viewer/2022110210/56812bc9550346895d901bfe/html5/thumbnails/12.jpg)
Test Analysis & Design – Why?
Test Basis & Test Cases
Create traceability At every moment, we can
calculate the requirements testing coverage
![Page 13: Test Analysis & Design – good practices](https://reader037.vdocuments.net/reader037/viewer/2022110210/56812bc9550346895d901bfe/html5/thumbnails/13.jpg)
Requirements
A requirement is a singular documented physical and functional need that a particular product or service must be or perform.
Functional reqsNon-functional reqsArchitectural reqsDesign reqsStructural reqsConstraint reqs
Test Basis
Test Oracle
≠ Emails≠ Skype
![Page 14: Test Analysis & Design – good practices](https://reader037.vdocuments.net/reader037/viewer/2022110210/56812bc9550346895d901bfe/html5/thumbnails/14.jpg)
When can we really have them all in place?
![Page 15: Test Analysis & Design – good practices](https://reader037.vdocuments.net/reader037/viewer/2022110210/56812bc9550346895d901bfe/html5/thumbnails/15.jpg)
Lessons I’ve learnt – First Requirements
Piece of Requirements coming
inAre they testable
?Test
Basis
YES
NO
Some documents coming in
Are they
addressing reqs or
additional info for me?
Reqs
Info
Test
Basis
AddtoAd
d to
if relevantKT
pack
Add to
if relevant
Perform static analysis
![Page 16: Test Analysis & Design – good practices](https://reader037.vdocuments.net/reader037/viewer/2022110210/56812bc9550346895d901bfe/html5/thumbnails/16.jpg)
Lessons I’ve learnt – First Test Cases “Design”
Analyze test basis,
& test oracle Identify
critical functionalit
ies
Which kind of test suites will be needed (e.g. smoke, regression, per functionality, per component)?
How can we prioritize them?
What’s the impact in terms of effort when changing the priority (review test cases, execute them, track execution status, revert to initial priority)
Identify needed testing types
Do we need to have separate test suites / cases for each testing type (manual vs automation, functional vs non-functional, etc.)
Do we need to have different design & writing styles for the test cases?
Identify automation
need
Create separate test suites for various automated test cases for different purposes.
Choose an appropriate design and writing style for these test cases.
Identify ways to
keep traceability
![Page 17: Test Analysis & Design – good practices](https://reader037.vdocuments.net/reader037/viewer/2022110210/56812bc9550346895d901bfe/html5/thumbnails/17.jpg)
Lessons I’ve learnt – Testing Structure
Delivery
Model
Test Levels
Number of
Testing
Cycles
Time
Estimated
level of change
Testing Structu
re
Risk
![Page 18: Test Analysis & Design – good practices](https://reader037.vdocuments.net/reader037/viewer/2022110210/56812bc9550346895d901bfe/html5/thumbnails/18.jpg)
Testing Structures – some examples
Traditional Waterfallrelease cycles are
typically several weeks to several months long, and usually have multiple phases of testing (Functional, System test, Performance, User Acceptance Test, etc) during any given release cycle
execution cycles are planned and scheduled based on these phases and metrics are tracked for each phase as well
Organize by Test Phase
Organize by Functionality or System and then by Phase
![Page 19: Test Analysis & Design – good practices](https://reader037.vdocuments.net/reader037/viewer/2022110210/56812bc9550346895d901bfe/html5/thumbnails/19.jpg)
Testing Structures – some examples
Agile release products in
shorter and frequent release cycles, each consisting of multiple Sprints in which one or more User Stories are targeted for development completion
Organize by Sprints and then by User Stories or functionality-> when having large number of releases with shorter Sprint cycles and many overlapping release cycles
![Page 20: Test Analysis & Design – good practices](https://reader037.vdocuments.net/reader037/viewer/2022110210/56812bc9550346895d901bfe/html5/thumbnails/20.jpg)
Testing Structures – some examples
Agile release products in
shorter and frequent release cycles, each consisting of multiple Sprints in which one or more User Stories are targeted for development completion
Organize by moving Sprints at project level as releases-> you have larger release cycles with many Sprints
![Page 21: Test Analysis & Design – good practices](https://reader037.vdocuments.net/reader037/viewer/2022110210/56812bc9550346895d901bfe/html5/thumbnails/21.jpg)
Testing Structures – some examples
Testing of System of systems large, complex platforms that may contain multiple
systems or sub-systems each with its own development and QA tracks, and there may be a need to track testing progress on a per system bases followed by System wide testingOrganize
releases as projects, followed by System level testing. This is useful when different teams are tracking progress for different systems.
![Page 22: Test Analysis & Design – good practices](https://reader037.vdocuments.net/reader037/viewer/2022110210/56812bc9550346895d901bfe/html5/thumbnails/22.jpg)
Testing Structures – some examples
Testing of System of systems large, complex platforms
that may contain multiple systems or sub-systems each with its own development and QA tracks, and there may be a need to track testing progress on a per system bases followed by System wide testing
Organize releases under the same project, followed by systems. This is useful when the platform has larger number of frequent releases and a single PM over all
![Page 23: Test Analysis & Design – good practices](https://reader037.vdocuments.net/reader037/viewer/2022110210/56812bc9550346895d901bfe/html5/thumbnails/23.jpg)
Test Cases Design – some good practices
Why do we write test cases?
The test cases are more than some sentences used to test various flows. They are our way to proof the level of confidence in what
we deliver by: measuring the requirements
coverage and their status at every point in
the development process: (if the requirements are covered by enough test cases, if the testing execution status at some point is the desired or planned one)
![Page 24: Test Analysis & Design – good practices](https://reader037.vdocuments.net/reader037/viewer/2022110210/56812bc9550346895d901bfe/html5/thumbnails/24.jpg)
Test Cases Design – some good practices
Write Test Cases before the implementation of the requirementsWrite Test Cases for all the requirements
Test Cases should map precisely to the requirements and not be an enhancement to the requirement
![Page 25: Test Analysis & Design – good practices](https://reader037.vdocuments.net/reader037/viewer/2022110210/56812bc9550346895d901bfe/html5/thumbnails/25.jpg)
Test Cases Design – some good practices
Use same naming convention for all the test cases in a project
Create unique names for your test cases (use “TC” + identifier + title)Use “Action - Target – Scenario” method to formulate the title
![Page 26: Test Analysis & Design – good practices](https://reader037.vdocuments.net/reader037/viewer/2022110210/56812bc9550346895d901bfe/html5/thumbnails/26.jpg)
Test Cases Design – some good practices
Test Case TitleAction a verb that describes what you are doing (create, delete,
ensure, edit, open,
populate, login)
Targetthe focus of
your test (screen, object entity,
program)
Scenariothe rest of what your
test is about and how you distinguish
multiple test cases for the same Action and Target
![Page 27: Test Analysis & Design – good practices](https://reader037.vdocuments.net/reader037/viewer/2022110210/56812bc9550346895d901bfe/html5/thumbnails/27.jpg)
Test Cases Design – some good practicesAction – Target –
ScenarioCreate – Task – title is not suppliedCreate – Task – title is the maximum allowable length
![Page 28: Test Analysis & Design – good practices](https://reader037.vdocuments.net/reader037/viewer/2022110210/56812bc9550346895d901bfe/html5/thumbnails/28.jpg)
Test Cases Design – some good practices
Write detailed description of every step of execution.
Define one single action per execution step.
Write clear and precise expected results
![Page 29: Test Analysis & Design – good practices](https://reader037.vdocuments.net/reader037/viewer/2022110210/56812bc9550346895d901bfe/html5/thumbnails/29.jpg)
Test Cases Design – some good practices
The Expected Result states: "Verify if error message is displayed." Issue: As executing the Test Case, what if the error message says, “Please provide postcode ", while it should say, "Your postcode is invalid"? Solution: The Expected Results states: "Verify that the error message about an invalid postcode is displayed.”
![Page 30: Test Analysis & Design – good practices](https://reader037.vdocuments.net/reader037/viewer/2022110210/56812bc9550346895d901bfe/html5/thumbnails/30.jpg)
Test Cases Design – some good practices
Each Test Case checks only one testing idea, but two or more expected results are totally acceptable if there is a need to perform several verifications for that testing idea.
Testing idea: “Payment can be performed by MasterCard credit card." Expected results:
1) In DB, cc_transaction table, in MasterCard column, value 1 is registered.
2) Credit card balance is reduced by the amount equal to the amount of the payment.
![Page 31: Test Analysis & Design – good practices](https://reader037.vdocuments.net/reader037/viewer/2022110210/56812bc9550346895d901bfe/html5/thumbnails/31.jpg)
Test Cases Design – some good practices
Expected results should met the test case purpose. Additional steps should be specified separately.
![Page 32: Test Analysis & Design – good practices](https://reader037.vdocuments.net/reader037/viewer/2022110210/56812bc9550346895d901bfe/html5/thumbnails/32.jpg)
Test Cases Design – some good practices
TC ID Execution Steps Expected Results
TC01.01 Verify Customer User is able to create keyword
1.Login as Customer User
2.Navigate to Create Keyword page.
3.Complete all fields with valid data. Submit data.
4.Navigate to Keywords List page.
5.Verify that the created keyword is present in the keywords list.
1. Successful login.
2. Create Keyword page is displayed.
3. Info message is displayed that the keyword is successfully created.
4. Keywords list page is displayed.
5. Newly created keyword is displayed in the keywords list.
Wrong – The Login and Navigate to steps are not required, as the purpose of the test is to verify that the user is able to successfully create keywords. Login and page displaying should be verified in separate Test Cases.
TC ID Execution Steps Expected Results
TC01.01 Verify Customer User is able to create keyword
Login as Customer User
Navigate to Create Keyword page.
1.Complete all fields
with valid data. Submit data.
2.Navigate to Keywords List page. Verify that the created keyword is present in the keywords list.
1.Info message is displayed that the keyword is successfully created.
2.Newly created keyword is displayed in the keywords list.
![Page 33: Test Analysis & Design – good practices](https://reader037.vdocuments.net/reader037/viewer/2022110210/56812bc9550346895d901bfe/html5/thumbnails/33.jpg)
Test Cases Design – important attributes
Test cases
Effective
Non-redundant
Clear
Detailed
AccurateShort and simple
language
Evolvable
Complete
Traceable
Repeatable
Self cleaning
Have a high probability of
detecting errors
Practical and low redundancy. Any
feature under test should not be
repeated in different test cases. Two test
cases should not find the same
defect.
Clear flow of events, correspondence
between execution steps and expected
results; unambiguously
defined execution steps and expected
results
Contain detailed steps needed to test a particular function; no missing execution
steps; no unnecessary
execution steps
No drawbacks like spelling mistakes;
use the system exact functionality / GUI
names
Short rather than lengthy; written in
simple language, so that any person is
able to understand the scope of each
test case
Well structured and maintainable,
neither too simple nor too complex;
separated test cases for positive and
negative scenarios; limit to 15 execution
steps
Should cover all the features/functionaliti
es that have to be tested
Each test case can be traced back to a requirement/use
case
The result of the test case should be
always the same, no matter how many times it has been executed before
Returns the test environment to the clean state
![Page 34: Test Analysis & Design – good practices](https://reader037.vdocuments.net/reader037/viewer/2022110210/56812bc9550346895d901bfe/html5/thumbnails/34.jpg)
Test Cases Cascading vs Independent design
Test Cases built on each other Simpler and smaller The output of one Test Case becomes the input of the next Test Case. Arranging Test Cases in a right order saves time during test execution If one Test Case fails, the subsequent tests may be invalid or blocked
Cascading style
Independent styleEach Test Case is self contained, does not rely on any other Test Cases Any number of Test Cases can be executed in any order Larger and more complex Harder to design, create and maintain
![Page 35: Test Analysis & Design – good practices](https://reader037.vdocuments.net/reader037/viewer/2022110210/56812bc9550346895d901bfe/html5/thumbnails/35.jpg)
Test Cases High-Level vs Low-Level writing style
Test Cases defining what to test in general terms, without specific values for input data and expected results. less time to write greater flexibility in execution more appropriate when tests are executed by testers with a vast knowledge of the application
High-level style
Low-level styleTest Cases with specific values defined for both input and expected results. repetitive it can be executed even by a tester that is just learning the application easier to determine pass or fail criteria easier to automate
![Page 36: Test Analysis & Design – good practices](https://reader037.vdocuments.net/reader037/viewer/2022110210/56812bc9550346895d901bfe/html5/thumbnails/36.jpg)
Test Cases Design – some good practices
Test cases must evolve during the entire software development lifecycle.
![Page 37: Test Analysis & Design – good practices](https://reader037.vdocuments.net/reader037/viewer/2022110210/56812bc9550346895d901bfe/html5/thumbnails/37.jpg)
Test Cases Design – some good practices
As requirements change, the testers must adjust test cases accordingly
Test cases must be modified to accommodate the additional information obtained from other phases
Each test case modified upon a change request should have in the description the record that describes the change (email, meeting minutes, use case ID)
As defects are found and corrected, test cases must be updated to reflect the changes and additions to the system
When a new scenario is encountered, it must be evaluated, assigned a priority and added to the set of test cases
Due to changes in requirements, design or implementation, test cases become often obsolete, out-of-date. Given the pressures of having to complete the testing, testers continue their tasks without ever revisiting the test cases. The problem is that if the test cases become outdated, the initial work creating these tests is wasted and additional manual tests executed without having a test case in place cannot be repeated.
![Page 38: Test Analysis & Design – good practices](https://reader037.vdocuments.net/reader037/viewer/2022110210/56812bc9550346895d901bfe/html5/thumbnails/38.jpg)
Test Analysis & Design – Metrics & Measurements
Percentage of requirements or quality (product) risks covered by test conditions
Percentage of test conditions covered by test casesNumber of defects found during test analysis and design
What cannot be measured cannot be managed.
![Page 39: Test Analysis & Design – good practices](https://reader037.vdocuments.net/reader037/viewer/2022110210/56812bc9550346895d901bfe/html5/thumbnails/39.jpg)
Thanks for attending this session!
Questions? Debates?Thoughts?