when agile becomes a quality game changer; what data says...
TRANSCRIPT
10/30/2013
1
Michael Mah
Managing Partner
QSM Associates, Inc.
75 South Church Street
Pittsfield, MA USA 01201
413-499-0988
Fax 413-447-7322
e-mail: [email protected]
Website: www.qsma.com Blog: www.optimalfriction.com
When Agile Becomes a Quality Game Changer; What Data Says from Recent
Agile Benchmark Research
(#2)
10/30/2013
2
(#3)
Background
“Agile projects can be considered more successful in the sense that they deliver more functionality with fewer defects.”
- Kent Beck
(#4)
10/30/2013
3
(#5)
(#6)
10/30/2013
4
(#7)
Rayleigh Curve Defect Rate
(#8)
10/30/2013
5
(#9)
(#10)
Background
“We don’t need no stinking metrics” - Jim Highsmith
“Without metrics, you’re just someone with another opinion” - Michael Mah
10/30/2013
6
(#11)
(#12)
10/30/2013
7
(#13)
The QSM SLIM Database
QSM maintains the world’s largest benchmarking database of 10,000+ completed software projects collected worldwide. We put industry productivity statistics on the desktop.
The QSM SLIM database contains projects in all industries, waterfall, Agile, offshore/outsourced, in-house, new development, and maintenance.
SLIM tools enable managers to measure and estimate Agile and/or waterfall projects, and determine ROI.
(#14)
British Telecom
SAP
Microsoft
Intel
AT&T/BellSouth
Nationwide
Motorola
VerizonWireless
Roche Diagnostics
Fiserv Corp
IBM Global
Misys Healthcare
JPMorganChase
Boeing
Bank of New York Mellon
Lockheed Martin
Progressive Insurance
DirecTV
The QSM SLIM Database
10/30/2013
8
(#15)
(#16)
10/30/2013
9
(#17)
(#18)
10/30/2013
10
(#19)
(#20)
Agile Metrics Capture – Velocity etc.
10/30/2013
11
(#21)
Agile vs Waterfall - Schedule
C&T Duration (Months) vs Effective SLOC
10 100 1000
Effective SLOC (thousands)
1
10
100
C&
T D
uration (Months)
Agile 1
Agile 3
Agile 2
Traditional 1
Traditional 2
Agile 1
Agile 3
Agile 2
Traditional 1
Traditional 2
All Sy stems Special Project QSM 2002 Scientific Av g. Line Sty le 1 Sigma Line Sty le
Elan 3.2 Release
6 mos
Industry Average
10 mos
FASTER
(#22)
Agile vs Waterfall - Quality
FEWER BUGS
10/30/2013
12
(#23)
This Data Says: Kent was CORRECT
“Agile projects can be considered more successful in the sense that they deliver more functionality with fewer defects.”
- Kent Beck
(#24)
10/30/2013
13
(#25)
Case Study: Co-Located XP - Follett Software
Team size
24 Developers 7 Testers 3 Customers 3 Project
Leaders Code Base
1,000,000 lines of code
7,000 automated unit test
10,000 automated acceptance test
(#26)
10/30/2013
14
(#27)
(#28)
10/30/2013
15
(#29)
Project Sketch – Core Metrics
(#30)
Input to SLIM
Size
Time
Defects
Effort
10/30/2013
16
(#31)
Industry
Average
Current
Performance
Delta
Project Cost
$3.5 Million
$2.2 Million
-$1.3M
Schedule
12.6 months
7.8 months
-4.8 mos
QA Defects
242
121
-50%
Staffing
35
35
n/a
Follett vs. Industry Average
(#32)
Distributed SCRUM – BMC Software
10/30/2013
17
(#33)
Project Sketch – Core Metrics
(#34)
Industry
Average
Current
Performance
Delta
Project Cost
$5.5 Million
$5.2 Million
-$.3M
Schedule
15 months
6.3 months
-8.7 mos
QA Defects
713
635
-11%
Staffing
40
92
+52
BMC vs. Industry Average
10/30/2013
18
(#35)
Agile Assessment — Schedule
SCHEDULE
10 100 1,000
PROJECT SIZE (thousands)
1
10
100
C&
T D
ura
tion (M
onth
s)
Agile Companies Company B SCRUM Company A XP QSM 2005 Business Avg. Line Style 1 Sigma Line Style
Faster Schedules
(#36)
Agile Assessment – Quality
BUGS
10 100 1,000
Effective SLOC (thousands)
1
10
100
1,000
10,000
Erro
rs (S
ysIn
t-Del)
Agile Companies Company B SCRUM Company A XP QSM 2005 Business Avg. Line Style 1 Sigma Line Style
Fewer Defects
10/30/2013
19
(#37)
This Data also Says: Kent was CORRECT
“Agile projects can be considered more successful in the sense that they deliver more functionality with fewer defects.”
- Kent Beck
(#38)
10/30/2013
20
(#39)
The Columbus Agile Benchmark Study
(Columbus vs the World)
(#40) footer
SLIM-
Estimate:
Size, Schedule,
Cost & Quality
Estimating
SLIM-DataManager
Software Project Metrics Repository
SLIM-Control:
Variance
Analysis
&
Adaptive
Forecasting SLIM-Metrics:
Industry
Benchmarking
& Process
Improvement
SLIM-MasterPlan: Incremental
Development & Project
Aggregation
10/30/2013
21
(#41)
(#42)
10/30/2013
22
(#43)
(#44)
10/30/2013
23
(#45)
(#46)
Agile Captures the Right Metrics for SLIM
Velocity/Burndown
Headcount
Stories and Point Sizing
Bugs
10/30/2013
24
(#47)
(#48)
10/30/2013
25
(#49)
Speed
Time-to-Market
1 10 100 1,000
New + Modified Size (thousands)
1
10
100
Month
s
All Sy stems QSM Business Av g. Line Sty le 1 Sigma Line Sty le
Faster Schedules
(#50)
10/30/2013
26
(#51)
Bugs
Bugs During QA
1 10 100 1,000
New + Modified Size (thousands)
1
10
100
1,000
10,000
Defe
cts
All Sy stems QSM Business Av g. Line Sty le 1 Sigma Line Sty le
Fewer Defects
(#52)
Short Feedback Loops
Paired programmers
Instantaneous code reviews
Accelerated learning and execution
Face to face communication channel
10/30/2013
27
(#53)
Transparency
“Transparency is a great floodlight. People who thrive in political maneuvering hate SCRUM…”
- Ken Schwaber
(#54)
Avoiding Burnout
XP = Sustainable pace
40 Hour Work Weeks
Prevent productivity collapse for overworked teams
10/30/2013
28
(#55)
High-bandwidth Communication
The best teams have “wide-open pipes”
Domain knowledge moves among the team
Information flows rapidly and accurately
(#56)
10/30/2013
29
(#57)
But wait, there’s more…
(#58)
10/30/2013
30
(#59)
(#60)
New Agile Benchmark Trends
Agile Trends - Iterations/Build Phase
Time-to-Market
10 100 1,000
New + Modified Size (thousands)
1
10
100
Mo
nth
s
Effort
10 100 1,000
New + Modified Size (thousands)
1
10
100
1,000
10,000
Pe
rson
-Mo
nth
s
Average Staff
10 100 1,000
New + Modified Size (thousands)
1
10
100
1,000
He
ad
cou
nt
Bugs During QA
10 100 1,000
New + Modified Size (thousands)
1
10
100
1,000
10,000
De
fects
10/30/2013
31
(#61)
(#62)
10/30/2013
32
(#63)
For Additional Information
Michael Mah Managing Partner QSM Associates email: [email protected] website: www.qsma.com blog: www.optimalfriction.com twitter: @michaelcmah Tel: 1 413-499-0988 Andrea Gelli QSM Associates Switzerland 8032 Zurich T +41 44 555 9126 email: [email protected]