the force awakens by joseph ours - qaiquest.org · the force awakens by joseph ours. if measure it...
TRANSCRIPT
NOTICE: Proprietary and Confidential
This material is proprietary to Centric Consulting, LLC. It contains trade secrets and information which is solely the property of Centric Consulting, LLC. This
material is solely for the Client’s internal use. This material shall not be used, reproduced, copied, disclosed, transmitted, in whole or in part, without the express
consent of Centric Consulting, LLC.
© 2013 Centric Consulting, LLC. All rights reserved
The Force Awakens
By Joseph Ours
If measure it you
cannot,
improve it you
cannot.
Examples of Software Testing Metrics
• Test Case Counts by Execution Status
• Test Case Percentages by Execution Status
• Test Case Execution Status Trend
• Test Case Status Counts vs Test Case Status Counts
• Test Case Coverage
• Test Case Status vs Coverage
• Test Case First Run Failure Counts
• Test Case Re-Run Counts
Test Cases
• Automation Index (Percent Automatable)
• Automation Progress
• Automation Test Coverage
Automation extras
Examples of Software Testing Metrics
• Defect Counts by Status
• Defect Counts by Priority
• Defect Status Trend
• Defect Density
• Defect Remove Efficiency
• Defect Leakage
• Average Defect Response Time
Defects
• Requirements Volatility Index
• Testing Process Efficiency
Other
The product of
measuring,
quantities are
0
5
10
15
20
25
30
35
40
45
Test Cases Executed
Each Day of Testing
0
50
100
150
200
250
300
350
Test Cases Executed
Each Day of Testing
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Test Cases Executed
Each Day of Testing
0
50
100
150
200
250
300
350
Test Cases Executed
Each Day of Testing
On Everything
Labels, We put
Important, Not
Everything Is
1-Critical
2-High
3-Medium
4-Low
Defects By Severity
1-Critical
2-High
3-Medium
4-Low
1-Critical4-Low
3-Medium2-High
Defects By Severity
1-Critical
4-Low
3-Medium
2-High
1-Critical
2-High 3-Medium
4-Low
0
5
10
15
20
25
30
35
40
1-Critical 2-High 3-Medium 4-Low
Defects By Severity
1-Critical
2-High
3-Medium
4-Low
Truncating
introduces
distortion.
Yeesssssss.
0
20
40
60
80
100
120
Defect Open and Close Rate
Open
Closed
Each Day of Testing
0
5
10
15
20
25
Defect Open and Close Rate
Open
Closed
Each Day of Testing
10
11
12
13
14
15
16
17
18
19
20
Defect Open and Close Rate
Open
Closed
Each Day of Testing
Win the race, the
turtle will, but the
we'd rather
watch the hare
run. Yes,
hmmm.
0
50
100
150
200
250
300
350
Test Cases Executed vs Planned
Planned
Executed
Each Day of Testing
1
10
100
1000
Test Cases Executed vs Planned
Planned
Executed
Each Day of Testing
Seldom weighed
in the same
scales, defects
we find and tests
we run are.
0
50
100
150
200
250
300
350
Test Cases Executed vs Defects
Defects Opened
Executed Tests
Each Day of Testing
0
20
40
60
80
100
120
0
50
100
150
200
250
300
350
Test Cases Executed vs Defects
Executed Tests
Defects Opened
Each Day of Testing
An issue of mind
over matter,
defect age is. If
mind you not, it
matters not.
Hmmmmmm.
0
5
10
15
20
25
0
20
40
60
80
100
120
Defects Opened vs Average Defect Age
Defects Opened
Defect Age
Each Day of Testing
0
5
10
15
20
250
20
40
60
80
100
120
Defects Opened vs Average Defect Age
Defects Opened
Defect Age
Each Day of Testing
Common Ways to Manipulate Graphs
Plot Cumulative vs Actual
Distort Perspective with 3D
Truncate AxisPlot Secondary Axis
Logarithmic Axis Invert Axis
Purpose of Metrics
Measure of Performance
Conformance to a Stated Practice
Deviation from Goal
Changing the World
How to Leverage Metrics
Explicitly link metrics to goals
Use trends over absolute numbers
Use shorter tracking periods
Change metrics when they stop driving change
Q&A
Joseph Ours
Email: [email protected]
Company Website:https://centricconsulting.com/technology-solutions/software-quality-assurance-and-testing/
Twitter:@justjoehere
LinkedIN:www.linkedin.com/josephours
Personal Blog:http://josephours.blogspot.com