measuring the success of your agile transformation
DESCRIPTION
The key to a successful agile journey is to identify concrete, measurable goals. Whether your challenge is to improve software quality, time to market, productivity, customer satisfaction, innovation, employee engagement, or some combination of these, agile metrics are crucial to your success. How do you use agile metrics early and often to know that you’re going in the right direction? And how do you know when your goals have been met?TRANSCRIPT
Measuring the Success of Your Agile TransformationPart 1
June 19, 2014
#AgileLIVE
© 2014 VersionOne, Inc. 2
A few logistics…
#AgileLIVE
© 2014 VersionOne, Inc. 3
Moderator
#AgileLIVE
Aaron SmithEditorial DirectorProjectsAtWork
www.agile42.com | All rights reserved. Copyright © 2007 - 2014.agile42 | The Agile Coaching Company
Brad SwansonSenior Agile Coach
@bradswanson
www.agile42.com | All rights reserved. Copyright © 2007 - 2014.agile42 | The Agile Coaching Company
The Enterprise Transition
Framework™
(ETF) for empirical change
1. Assess2. Strategy
3. Pilot4. Rollouthttp://www.agile42.com/en/agile-transition/etf/
www.agile42.com | All rights reserved. Copyright © 2007 - 2014.agile42 | The Agile Coaching Company
Goal PSF NC
Agile Strategy Map™
Define the Goal we want to achieve
Possible Success Factor (PSF) represents a hypothesis about a key factor that would enable us to achieve the Goal
Necessary Condition (NC): what is needed in order to fulfill the Success Factor
http://www.agile42.com/en/agile-transition/agile-strategy-map/
Agile Strategy Map™ - Framework for Strategy
www.agile42.com | All rights reserved. Copyright © 2007 - 2014.agile42 | The Agile Coaching Company
Goal
Agile Strategy Map™
PSF#1
PSF#1: we believe that by consolidating all projects in one Portfolio, we will be able to make better business decisions
NC#1.1
NC#1.2
NC#1.1.1
To validate the hypothesis, we define what is needed to run a pilot which will measure the validity of the PSFNC#1.1: we need to list at least 3 projects per product areaNC#1.1.1: we need to choose projects and define comparable dimensions such as: business value, strategic alignment, cost...NC#1.2: we need to define a sample Portfolio funnel, and check that Teams can deliver with the provided information
What conditions are necessary to achieve the success factor (PSF)?
www.agile42.com | All rights reserved. Copyright © 2007 - 2014.agile42 | The Agile Coaching Company
Success: we can compare the
value of projects easily (using defined KPIs) and
make consistent business decisions.
Failure: we still have endless
discussions about priority and it’s costly to prepare
project info for the portfolio processIndicators (measures):
•< 1 day to prepare 1 project business case
•Allows objective comparison of projects
•< 8 hrs to select priorities each quarter
•No ‘exec overrides’ on priority decisions
How will we measure PSF success and failure?
PSF#1: we believe that by consolidating all projects in one Portfolio, we will be able to make
better business decisions
www.agile42.com | All rights reserved. Copyright © 2007 - 2014.agile42 | The Agile Coaching Company
Goal CSF#1NC#1.
1
NC#1.2
Agile Strategy Map™
CSF#1: we must consolidate all projects in one Portfolio, so that will be able to make better business decisions
NC#1.1.1
The Pilot was successful...
The Potential Success Factor becomes a Critical Success Factor (CSF). Update the strategy to roll out and institutionalize the change.
Pilot success! Now what?
www.agile42.com | All rights reserved. Copyright © 2007 - 2014.agile42 | The Agile Coaching Company
Sustainably faster
delivery of solutions that
delight customers
Faster time to market
Alignment & sup-port across the
org
Engaged & joy-ful people
Transparent empirical view
of org performance
Grow and sus-tain agile prac-
tices
Culture of continuous learning & improve-
ment
Org priorities aligned with
Customer value
Agile TransformationExample Goal + Success Factors (PSFs /
CSFs)
www.agile42.com | All rights reserved. Copyright © 2007 - 2014.agile42 | The Agile Coaching Company
Sustainably faster delivery of solutions
that delight customers
Faster time to market
Balance work-load & capacity:
limit WIP
Alignment & support across
the org
Engaged & joy-ful people
Transparent empirical view of org
performance
Grow and sustain agile
practices
Culture of continuous learn-ing & improve-
ment
Org priorities aligned with
customer valueDedicated Guiding Coalti-
ion
Pilot projects
Clearly communicated vision & strat-
egy
Training & coach-ing
Build internal expertise
Collaborative prioritization
with stakehold-ers
Collaborative cross-functional
teams Automated testing &
deploymentIncremen-tal deliv-
ery
Visible met-rics
Leaders to remove
fearEmpower-ment
Clear, realistic goals
Servant leader-
ship
Action taken
to improve
Foster innova-
tion Celebrate failed experi-ments
Example: Necessary Conditions for the PSFs
Clear career path
GoalSuccess fac-
torsNecessary condi-
tions
Legend:
www.agile42.com | All rights reserved. Copyright © 2007 - 2014.agile42 | The Agile Coaching Company
A few metrics (KPIs) for the goalSustainably faster delivery of solutions that delight customers
These are lagging indicators; some leading indicators are described soon…
www.agile42.com | All rights reserved. Copyright © 2007 - 2014.agile42 | The Agile Coaching Company
Leading & lagging indicators for each PSF / CSF
PSF Leading Indicators Lagging Indicators
• # total concurrent projects• Avg # concurrent
projects/person• Program burn-up chart
• Lead time: idea to 1st delivery• Cycle time: Dev start to 1st
delivery• Deployment frequency post
1st delivery (MVP/V1)
• Frequency & effectiveness of stakeholder collaboration on prioritization
• Standard way to compare project value
• Portfolio priorities highly visible
• All teams working on top portfolio priorities; no ‘side projects’
• Delivered value is measured and meets expectations
• Customer satisfaction survey• Sales and/or usage statistics
1. Faster time to market
2. Org priorities aligned
with cus-
tomer value
www.agile42.com | All rights reserved. Copyright © 2007 - 2014.agile42 | The Agile Coaching Company
CSF 1: Time to Market
Lead time: from request to 1st deploy
Cycle time: dev start to 1st de-ploy
www.agile42.com | All rights reserved. Copyright © 2007 - 2014.agile42 | The Agile Coaching Company
CSF 1: Time to Market – leading indicator
agile42 | The Agile Coaching Company www.agile42.com | All rights reserved. Copyright © 2007 - 2014.
PSF 2: Org priorities aligned with customer value –
leading indicators
Churn = % change af-ter commit
point
agile42 | The Agile Coaching Company www.agile42.com | All rights reserved. Copyright © 2007 - 2014.
Compare actuals
with forecast
PSF 2: Org priorities aligned with customer value –
lagging indicators
www.agile42.com | All rights reserved. Copyright © 2007 - 2014.agile42 | The Agile Coaching Company
Leading & lagging indicators for each PSFPSF Leading Indicators Lagging Indicators
• Sr. Leadership consensus on strategy
• Frequency of comms on vision & strategy; # of different forums
• Short-term wins visibly celebrated
• # hours spent on Guiding Coalition activities
• Poll: % who agree with strategy
• Poll: % prefer new to old approach
• # of teams asking to ‘go agile’
• # people trained on Agile• # people in coached
projects• # people in the ‘coach the
coach’ program
• Agile team practices maturity level
• # people completing ‘coach the coach’
• # people achieving 3rd party Agile certifications
• Monthly attendance in Agile Community of Practice (CoP) mtgs
4. Grow & sustain
agile prac-tices
3. Align-
ment & support across the org
agile42 | The Agile Coaching Company www.agile42.com | All rights reserved. Copyright © 2007 - 2014.
PSF 3: Alignment & support
agile42 | The Agile Coaching Company www.agile42.com | All rights reserved. Copyright © 2007 - 2014.
PSF 4: Grow & sustain agile practices
agile42 | The Agile Coaching Company www.agile42.com | All rights reserved. Copyright © 2007 - 2014.
Leading & lagging indicators for each PSF
*Success ratio over 70% may indicate fear of failure
PSF Leading Indicators Lagging Indicators
• Poll: things getting better or worse?
• Career path defined for all roles
• Poll: employee job satisfaction
• Attrition rate• # employee referrals for
hiring
• Avg frequency of team retrospectives
• # of experimental learnings celebrated (fast safe failures)
• Poll: Do you feel safe to experiment & innovate?
• # team-initiated process changes
• # team-initiated product innovations
• Success ratio of experiments*
5. Engaged & joyful people6. Cul-ture of continu-
ouslearning
& Improve-
ment
www.agile42.com | All rights reserved. Copyright © 2007 - 2014.
agile42 - The Agile Coaching Company
agile42 | The Agile Coaching Company
Measuring Team Progress on Agile Practices
www.agile42.com | All rights reserved. Copyright © 2007 - 2014.agile42 | The Agile Coaching Company
Agile Practices Roadmap – Team Level
www.agile42.com | All rights reserved. Copyright © 2007 - 2014.agile42 | The Agile Coaching Company
Agile Practices Roadmap – Team Level (p.2)
agile42 | The Agile Coaching Company www.agile42.com | All rights reserved. Copyright © 2007 - 2014.
Team Practices Dashboard
agile42 | The Agile Coaching Company www.agile42.com | All rights reserved. Copyright © 2007 - 2014.
Team Practices Dashboard
Highest level achieved across all four di-mensions
www.agile42.com | All rights reserved. Copyright © 2007 - 2014.
agile42 - The Agile Coaching Company
agile42 | The Agile Coaching Company
Attack of the Killer Metrics!
When metrics go bad…
agile42 | The Agile Coaching Company www.agile42.com | All rights reserved. Copyright © 2007 - 2014.
Questions to ask about every metric1. What decisions will be made based on this
metric?2. Is this the RIGHT thing to measure or an easy-to-
measure proxy?3. What might be the unintended consequences of
this metric?4. How could this metric be ‘gamed’? 5. What can we do to guard against unintended
consequences and ‘gaming’?
agile42 | The Agile Coaching Company www.agile42.com | All rights reserved. Copyright © 2007 - 2014.
Compensating / Balancing Metrics• What unintended consequences might happen if you measure speed of delivery?
• What is a compensating metric to counteract it?
• Measure speed AND quality!
www.agile42.com | All rights reserved. Copyright © 2007 - 2014.agile42 | The Agile Coaching Company
Management vs. Team Metrics
https://creativecommons.org/licenses/by-nc-sa/2.0/
https://www.flickr.com/photos/joeshlabotnik/ https://www.flickr.com/photos/jeroen020/
www.agile42.com | All rights reserved. Copyright © 2007 - 2014.
agile42 - The Agile Coaching Company
agile42 | The Agile Coaching Company
Thank you!
Brad SwansonSenior Agile Coach
@bradswanson
© 2014 VersionOne, Inc. 32
Q&AJoin us for Part 2, June 25, Noon-1 PM ET
Thank you for joining us today!
Join us for Part 2, June 25, Noon-1 PM ET
#AgileLIVE