smart metrics
DESCRIPTION
TRANSCRIPT
Agile On The Beach September 2011 � Smart Metrics
1Copyright © 2011 – Mike Griffiths - www.LeadingAnswers.com. All Rights Reserved
September 15th, 2011
Agile On The Beach
Mike Griffiths, PMPLeading Answers Inc.
Smart Agile Metrics
www.LeadingAnswers.com
Presenter Background� Project Manager and Trainer
� >25 years IT experience on utilities, defense, & finance� 10 years PMO Agile-to-Traditional Integration
� Agile Project Management� Helped create Agile method DSDM in 1994� 17 years agile project experience (DSDM, XP, Scrum, FDD)� Board director of Agile Alliance and APLN� Author, trainer, and presenter Agile Conference 2001-9
� Traditional Project Management� PMP, PRINCE2 certifications� PMBOK v3 and v5 contributor and reviewer� Trainer for PMI SeminarsWorld 2005-2011� Presenter PMI Global Congress 2004-2011� Steering Committee PMI-ACP Exam
Agile On The Beach September 2011 � Smart Metrics
2Copyright © 2011 – Mike Griffiths - www.LeadingAnswers.com. All Rights Reserved
� 1 <Star>� 2� 3� 4� 5
Agenda
4) Measuring Up
3) �Design Factory� Metrics
2) The Hawthorne Effect
1) Measurement Troubles
6) Smart Metric examples
5) Leading vs. Lagging Metrics
Metrics are like fire�
Agile On The Beach September 2011 � Smart Metrics
3Copyright © 2011 – Mike Griffiths - www.LeadingAnswers.com. All Rights Reserved
MeasurementNot all observations are useful: � The sun rises up in the sky in the morning and then falls down again at night� Planets revolve around the earth� Stars come out at night� Heavier objects fall faster than lighter objects
�There are so many possible measures in a software process that some random selection of metrics will not likely turn up something of value� � Watts Humphrey
Not all useful things are easily observable: � Spouse�s mood � Sponsor Confidence� Team Commitment
"Not everything that can be counted counts, and not everything that counts can be counted� � Albert Einstein.
The Hawthorne Effect
You will influence what you measure...
Agile On The Beach September 2011 � Smart Metrics
4Copyright © 2011 – Mike Griffiths - www.LeadingAnswers.com. All Rights Reserved
Metrics Should be:1. Simple, self-generating2. Relevant to the end-goal3. Ideally, leading, future-focused
Design Factory Metrics
Traditional Metrics ExaminedDesirable Characteristics:ü The Hawthorne Effect is positiveü Simple, self generatingü Relevant to the end-goalü Leading, future-focused
Traditional Metrics:� Lines of Code Written � poor, does not reward
simplification, leads to code bloat
� Function Points Delivered � poor, effort to generate, not relevant to the end-goal of project
� Hours Worked � poor, leads to long hours, burn-out, defects, consumed budgets
Agile On The Beach September 2011 � Smart Metrics
5Copyright © 2011 – Mike Griffiths - www.LeadingAnswers.com. All Rights Reserved
Smarter Project Metrics
Since you will influence what you measure�
Choose Smarter Metrics:� Features Accepted� Sponsor Confidence� User Satisfaction� Defect Cycle Times
Smarter Metrics� Features Accepted
Not StartedIn ProgressAccepted
Scope (points) IPS Project Progress Spend
0
500
1000
1500
2000
2500
3000
1/10/0
6
1/12/0
6
1/2/07
1/4/07
1/6/07
1/8/07
1/10/0
7
1/12/0
71/2
/081/4
/081/6
/081/8
/08
1/10/0
8
1/12/0
8
1/2/09
1/4/09
1/6/09
1/8/09
1/10/0
9
1/12/0
9
Scope BuiltAccepted
Agile On The Beach September 2011 � Smart Metrics
6Copyright © 2011 – Mike Griffiths - www.LeadingAnswers.com. All Rights Reserved
Smarter Metrics� Features Accepted
050
100150200250300350400450
Jan Feb Mar Apr May Jun Jul Aug
Time
Feat
ures
Not StartedIn ProgressCompleted
Not StartedIn ProgressAccepted
Feature Based Reporting Example 2: Parking lot Diagrams
Feature Set Status: Enter Order Details
(15)
24%Percentage Complete:
Not StartedWork in ProgressCompletedAttention (i.e. behind)
Percentage Complete Bar
Sep 2011Target Completion Month:Not Complete Complete
FB Chief Programmers Initials
Feature Set Name
Number of Features in the Feature Set
The Feature Set called Enter Order Details has 15 features and is currently 24% complete. It is due to be completed by September 2011
Agile On The Beach September 2011 � Smart Metrics
7Copyright © 2011 – Mike Griffiths - www.LeadingAnswers.com. All Rights Reserved
Create New Order
(5)
100%
CaptureCustomer
Details(9)
75%
Enter OrderDetails
(15)
24%
ProcessPayment
(11)
Create NewCustomer
(8)
55%
AmendCustomer
Details(6)
20%
ArchiveCustomer
(4)
StockSearch
(6)
95%
ItemDetails
(12)
75%
Order Processing
Customer Management
Inventory Management
Jul 2011
Sep 2011
Sep 2011
Oct 2011
Aug 2011
Sep 2011
Sep 2011Oct 2011
Nov 2011
CM DH LF RS NC
AW
KB
SW SW
Feature Based Reporting
Smarter MetricsUser Satisfaction
Agile On The Beach September 2011 � Smart Metrics
8Copyright © 2011 – Mike Griffiths - www.LeadingAnswers.com. All Rights Reserved
Smarter Metrics� Sponsor Confidence
Smarter Metrics� Defect Cycle Time
Agile On The Beach September 2011 � Smart Metrics
9Copyright © 2011 – Mike Griffiths - www.LeadingAnswers.com. All Rights Reserved
� �You get what you measure�
� �You get only what you measure, nothing else�
� �You tend to loose the things that you can�t measure: insight, collaboration, creativity�
� Nucor Steel - From startup in 1970 to North America�s largest and most profitable steel company
� Incentive pay based on productivity
� Measure one level above where expected / directly controllable
Measuring Up
� Defects could be traced back to individual developers, but may be environmental
� Rolling up to an entire team and get testers involved earlier to provide more timely and valuable feedback to developers
�Instead of making sure that people are measured within their span of control. It is more effective to measure people one level above their span of control. This is the best way to encourage teamwork, collaboration, and global, rather than local optimization� � Mary Poppendieck
Measuring Up in Software
Agile On The Beach September 2011 � Smart Metrics
10Copyright © 2011 – Mike Griffiths - www.LeadingAnswers.com. All Rights Reserved
� For an accountant, a perfect view of the past might be useful
� For a project manager, a perfect or even an imperfect view of the future is far more useful
Leading vs. Lagging Metrics
Pay less attention to Lagging Metrics� Actual values
Pay more attention to Leading Metrics� Trends� Impacts of projections
Trends
CR = ChangeRequests
Clarif. = Clarifications
Observations Mar 3 Mar 10 Mar 17 Mar 24 Mar 31Defects Opened 5 25 30 20 10Defects Closed 1 16 35 22 15Defects Remaining 4 13 8 6 1CR Opened 0 18 20 23 12CR Closed 0 11 21 16 9CR Remaining 0 7 6 13 16Clarif. Opened 9 12 14 8 2Clarif. Closed 6 14 11 9 5Clarif. Remaining 3 1 4 3 0Total Observations 7 21 18 22 17
Observation Trends
02468
1012141618
Mar 3 Mar 10 Mar 17 Mar 24 Mar 31
Defects RemainingCR RemainingClarif. Remaining
Agile On The Beach September 2011 � Smart Metrics
11Copyright © 2011 – Mike Griffiths - www.LeadingAnswers.com. All Rights Reserved
Trends - Risks ReducedJan Feb Mar Apr
ID Risk Short Name Imp Prob Sev Imp Prob Sev Imp Prob Sev Imp Prob Sev1 JDBC driver performance 3 2 6 3 0 0 3 0 0 3 0 02 Calling Oracle Stored Procs via web service 2 2 4 2 0 0 2 0 0 2 0 03 Remote app distribution to PDA's 3 2 6 3 1 3 3 0 0 3 0 04 Oracle Warehouse Builder stability 2 2 4 2 3 6 2 2 4 2 0 05 Source system availability 2 1 2 2 1 2 2 0 0 2 0 06 Access to user community 2 1 2 2 2 4 2 2 4 2 1 27 Availability of Architect 2 2 4 2 3 6 2 2 4 2 0 08 Server Upgrade necessary 1 2 2 1 1 1 1 0 0 1 0 09 Oracle Handheld Warehouse Browser Launch 3 1 3 3 1 3 3 3 9 3 1 310 PST changes for BC 0 0 0 0 0 0 2 2 4 2 1 2
33 25 25 7
Project Risk Profile
0
5
10
15
20
25
30
35
Jan Feb Mar Apr
PST changes for BCOracle Handheld Warehouse Browser LaunchServer Upgrade necessaryAvailability of ArchitectAccess to user communitySource system availabilityOracle Warehouse Builder stability Remote app distribution to PDA'sCalling Oracle Stored Procs via web serviceJDBC driver performance
Analyst UI Designer Code & UT Sys. Test
45 30 40 35
� Identifying constraints� Buffering� Managing flow
Cycle Times
Agile On The Beach September 2011 � Smart Metrics
12Copyright © 2011 – Mike Griffiths - www.LeadingAnswers.com. All Rights Reserved
Cycle Times
Construction Cumulative Flow
020406080
100120140
20/02
/2004
27/02
/2004
05/03
/2004
12/03
/2004
19/03
/2004
26/03
/2004
02/04
/2004
09/04
/2004
16/04
/2004
23/04
/2004
30/04
/2004
07/05
/2004
14/05
/2004
21/05
/2004
28/05
/2004
Total FeaturesAnalysisUI DesignCode & UTSys Test
} Widening Area
Bottleneck Activity = �UI Design� (activity below widening area)
Little�s Law
Little�s Law:Cycle times are proportional to queue lengths.
(We can predict completion times based on queue size)
Agile On The Beach September 2011 � Smart Metrics
13Copyright © 2011 – Mike Griffiths - www.LeadingAnswers.com. All Rights Reserved
Task Boards & Kanban
Summary
1. Use metrics wisely
2. Leverage the Hawthorne Effect
3. Measure Up
4. Favour Leading Metrics
More info and slides: www.LeadingAnswers.comEmail: [email protected]