agile metrics and quality
DESCRIPTION
TRANSCRIPT
Agile Metrics and Quality
Metrics -> Improvements
Use metrics to• Improve quality• Shorten product cycle time• Decrease costs• Decrease risks• Increase predictability and reliability
Control Limits and Variances
We can have limits for• WIP• Defects• Team size• Velocity• Time-boxes• Others?We can analyze variances to
understand the special causes of variation
Focus on Trends• Trends matter more than values• Trends and spikes are only the symptoms• Trends are leading indicators
Images from MountainGoat Software
Improve quality◦Quality is measured by passed tests
and customer acceptance◦Quality is defined by Definition of
Done and Acceptance Criteria◦Defect trends to use in Agile projects
Number of open defects Escaped defects Defect Cycle Time Defect spill-over
◦Test Automation is critical!
Defect Injection and Resolution
What is the trend of total Active Defects?Image from MountainGoat Software
Analyzing and Preventing Defects
Fishbone / 5 Why Technique
Pareto Chart
Escaped Defects
Calculation:
◦ For all releases, find all defects related to the release found after the release date
◦ Add up all the defects
◦ Can be captured per day / week / month / … or per sprint / release
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec0
1
2
3
4
5
6
7
8
9
• Used to measure the quality of delivered code
Escaped Defects over Time
Defect Cycle Time• Rapid resolution of defects is important• Average bug-fix time can also be tracked for
different priority bugs
1 2 3 4 5 6 7 8 9 100
2
4
6
8
10
12
14
16
18
20
Defect Cycle Time
Sprint No.
Hours
Desired Threshold
Defect Spill-over• Resolution of defects for a sprint story within the
sprint is important• Definition of Done can include a criteria like “No P1
defects”• We can also tracked whether spilled-over defects are
being closed in the next sprint
1 2 3 4 5 6 7 8 9 100
1
2
3
4
5
6
7
8
9
Defect Spill Over
Sprint No
No o
f defe
cts
Test AutomationCritical factor for regression testingCoverage should be as high as
possible
Shorten product cycle time
Cumulative Flow Diagrams◦Lead Time◦Cycle Time
CFD – An ElaborationFe
atu
res
Time
What does this CFD say?
Not startedStartedCompleted
Too much Work-In-Progress!
Image from MountainGoat Software
Identifying bottlenecks with a CFD
Where is the bottleneck?In DB Procs – After the widening Analysis phase
1 2 3 4 5 6 7 8 8 10 11 12 13Day
Image from http://www.soliantconsulting.com
Decrease costs• Earned Value Management• Wastage
Cost Variance (CV) = EV – ACSchedule Variance (SV) = EV – PVCost Performance Index (CPI) = EV / ACSchedule Performance Index (SPI) = EV / PVEstimate at Completion (EAC) = BAC / CPI or
= AC + BAC – EVEstimate To Complete (ETC) = EAC – ACVariance at Completion (VAC) = BAC – EAC
PV – Planned ValueEV - Earned ValueAC - Actual costBAC - Budget @ CompletionEAC - Estimate @ CompletionETC- Estimate to CompletionVAC- Variance @ Completion
Earned Value Analysis
EVM - InterpretationPV EV AC Schedule Cost
$1 $1 $1 On schedule On cost
$2 $2 $1 On schedule Under cost
$1 $1 $2 On schedule Over cost
$1 $2 $2 Ahead of schedule On cost
$1 $2 $3 Ahead of schedule Over cost
$1 $2 $1 Ahead of schedule Under cost
$3 $2 $1 Behind schedule Under cost
$2 $1 $3 Behind schedule Over cost
$2 $1 $1 Behind schedule On cost
EVM - ProgressPV ACEV
CV
SV
Ref: Earned Value and Agile ReportingAnthony Cabri, Mike Griffiths
WastageAny time not dedicated to working on
the highest priority story / task on the sprint backlog
Should be tracked over time and by cause
1 2 3 4 5 6 7 8 9 10 11 120
10
20
30
40
50
60
Wastage (in hrs)
Prod
issu
es
Trai
ning
Test
aut
omat
ion
Other
team
s
Defec
ts
Other
s0
40
80
120
160
Wastage (in hrs)
Decrease risks• WIP Limits• Small Iterations• More predictability
Increase Predictability / Reliability
• Velocity• Lead Time• Cycle Time
Metrics to Avoid• Velocity comparisons• Productivity e.g. Story Points per day• Effort Estimation accuracy• Number of stories completed• LOC/Function Points• Code quality
