test case attributes
Post on 30-Jun-2015
2.346 Views
Preview:
TRANSCRIPT
Empowering the Test Case
Introduction
A pass rate is an abstraction of the actual result of a set of test cases
This abstraction is done to facilitate for stakeholders
A 50 % pass rate says that half the test cases passed – but what value does this
information actually hold?
I most cases, not much!
We have gone to far in our abstraction and the information now holds almost no real
value to our stakeholders, even if they might still think so
Test Case Attributes
How can the result of a set of test cases be more valuable?
By assigning standardized attributes to test cases, the value of the test result will be
much higher
The test case will provide more information that can be aggregated when generating the
test report to the stakeholders
What information depends on the attributes – this presentation outlines one set of
attributes
Test Case Attributes Overview
Test
Case
Software
Quality
Attributes
Priority
Priority
Each test case is assigned a priority based on several factors
What severity will a defect found by the test case most likely have?
How important is the functionality to stakeholders?
With what frequency will the test case most likely find defects?
There can be many factors that set the priority
The priority can be a scale between 1 and a number which is set based on the
granularity that is deemed to be needed by stakeholders
Software Quality Attributes
By listing what sort of software quality attribute the test case covers the test result will
reveal much more detail
You can use any set of software quality attributes
IEC 9126-1 is one example of software quality attributes
Functionality
Maintainability
Efficiency
Usability
Reliability
Portability
Test Suite Overview
Performance
Priority 1
Test Case A
Functionality
Priority 3
Test Case B
Efficiency
Priority 1
Test Case C
Performance
Priority 2
Test Case D
Functionality
Priority 2
Test Case E
Functionality
Priority 1
Test Case F
Test Results without Test Case Attributes
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Pass Rate
Test Results
Blocked
Failed
Passed
Test Results with Software Quality Attributes
0%10%20%30%40%50%60%70%80%90%
100%
Blocked
Failed
Passed
Test Results with Priority
0%10%20%30%40%50%60%70%80%90%
100%
Priority1
Priority2
Priority3
Blocked
Failed
Passed
Test Case Attributes Added Value
Imagine that the three above charts show the same test results
Initial indications show that 85 % are passed – Quite good results
The second chart however shows that almost all of the failed test cases are located in
the reliability area – This can for example indicate that we have severe stability
problems
The third chart shows that almost all of our failed test cases have high priority – also an
increased risk
Even if the initial indications were positive, analyzing the test results with test case
attributes further reveals more information and additional risks
Conclusion
By adding test case attributes to the test cases, the test result can hold much more
value to the stakeholder
It can also facilitate for the tester during scope selection
The cost is very small, and the potential gain can be very large if stakeholders base
critical decisions on the test results
What test case attributes to add depends on stakeholder needs – what information can
be valuable to the stakeholders?
Reference
IEC 9126
http://en.wikipedia.org/wiki/ISO/IEC_9126
top related