integrative keynotev2

©2015 Cutter Consortium

One Size Does Not Fit AllDr. Murray Cantor, Senior Consultant [email protected]

mailto:[email protected]

http://www.murraycantor.com/


Things I have heard over the years “I have no idea.”

• Developers, when asked about how long will it take?

“We tried agile, but it didn't work for us.”• Development Managers

“Measures are a waste, they are costly, oppressive, and interfere with the real work.” • Some Methodologists

“Trust the (my) process. If the process is not working for you, you are doing it wrong.” • Some (of the same) Methodologists


Does one process ever fit all organizations? Over the years, we have seen

many ‘one true’ processes:• Water Fall• Boehm Spiral• Extreme Programming (XP)• Controlled Iteration, Rational Unified

Process• Software Factories• (Flavors of) Scaled Agile• DevOps

©2015 Cutter Consortium 4

Each of these have generated lots of heated disagreements


The development leader’s choice

Follow ‘the one true method’• Advantage: It is prescriptive• Disadvantage: It is prescriptive in that it

may be blindly applied – there is enough variation in software development that blindly following even a sound process will often, but not always work.

Roll your own• You are likely to ask too much of the

practitioners – software developers want to develop software, not become experts in all these fields, so they can pick and apply the right principle.

• Relearn the old lessons, e.g .Brooks law, Conway’s law, iteration management, role of design, …

There is always a process. Is it what you intend?


So, What to Do.Start by understanding the work you do.


Choosing your methods needs to align

With your organization level and goals

With the mix of work you do


Achieving goals requires sense and respond loops Key principles

– Kelvin’s Principle: “To measure is to know. If you can not measure it, you can not improve it”

• Measures are part of feedback loops

– The converse principle: “Don’t bother to measure what you do not intend to improve”

• Find a small set of measures, not a long laundry list

– Einstein’s Principle: “The best solution is as simple as possible, but not simpler.”

• Pick the right, not overly simple, statistic

(re)Set Goal

Take action

(practices)

Measure progress

(analytics)

React


Adapting your organization


Meeting goals requires analytics

Work item, artifact completionStaff member Commits to

Project, product deliveryProject manager, team lead

Commits to

Efficiency, value deliverySenior manager Commits to

Profit, return on investment, mission fulfillment Line of business executive Commits to

Before


Aligning goals For each level to meet its goal, the

leader is dependent on the lower level.

So, the leader seeks commitments from that layer. Meeting those commitments becomes the goal of the next layer.

Hence, the analytics serve to integrate the organization


Adapting to your mix


Kinds of Development Efforts: What is your mix?

1. Low innovation/high certainty• Detailed understanding

of the requirements• Well understood code

2. Some innovation/some uncertainty• Architecture/Design in

place• Some discovery required

to have confidence in requirements

• Some refactoring/ evolution of design might be required

3. High innovation/Low Uncertainty• Requirements not fully

understood, some experimentation might be required

• May be alternatives in choice of technology

• No initial design/architecture


The methods landscape

Kanban

Lean startup: MVP

Agile, Scrum

Product Development Flow

Systems/Software Engineering

Lean Software

Podular Org.Liminal Thinking.

Technical Debt Management

Iterative learning: Updating estimates and plans in the face of evidence

DevOps/Continuous Delivery


1. Low innovation - high certainty: Statistics of• Cycle, lead times• Backlogs size,

growth• Time in process• Utilization• Non-value added

effort

1 2 3

2. Some innovation - some uncertainty• Time, cost to delivery• Velocity • Burn down• Cumulative flow

diagrams

3. High innovation: Low certainty• Time to pivot• Value of learning• Business canvas• Time, cost to delivery

Apply measures in accord with project characterization

Predictive/Bayesian

Descriptive


Example: Fitting analytics and practices to routine efforts For low innovation efforts (continuous delivery, not “real”

projects), pick product flow practices and analytics• Uncertainty is low: you have already carried out similar projects many

times• The only thing that matters is how quickly or efficiently you can carry

out the project

• Suitable for lean/VSM measures

• Tradeoff between speed/efficiency(utilization)

• The principles described by Don Reinertsen in his book Flow apply in this bucket

http://www.amazon.com/Principles-Product-Development-Flow-Generation/dp/1935401009/ref=sr_1_1?ie=UTF8&qid=1429219228&sr=8-1&keywords=Flow+lean


Artifact-centricity is the appropriate process model for this (routine efforts) bucket Unlike activity-centric processes, artifact-centric processes

focus on describing how business data is changed/updated, by a particular action or task, throughout the process

Specifically, in the routine-effort bucket apply value stream models and flow measures (as described in the previous couple of slides) to state transitions of work products (artifacts)• Two state types:

– In process (undergoing state transitions)– In backlog (awaiting state transition)

If you consider this is a departure from traditional Agile methods, you are right:• One size does not fit all


Semantics of artifact-centric value stream maps


Example: A Value Stream model for routine effortsControl challenges

• Random arrival intervals• Variation of effort to address work items (unlike standardized

manufacturing)


Descriptive example: Cycle times

20

These will be described in more detail in next webinar


To Visualize the data, use a histogram

21

80% point is about 105 days


Insights and Actions Insights

• Both teams performing comparably: Not obvious skills issue

• Backlogs too large• The teams seem to be focusing on the

easier, not the most critical

Actions• With team investigate reason for backlog

size• Discovered the governance process

(decision to update statuses) is overly cumbersome leaving staff free to work elsewhere

• In response, the governance process was: – Streamlined (an approval eliminated)– Automated (less time spent finding e-mails)

• Work with teams to set and track cycle time 80% goal by priority

22


This is what improvement looks like

23


Example 2: Fitting analytics and practices to high innovation projects For high innovation projects, pick probabilistic methods and

the corresponding set of practices:• You really do not know what the solution would look like – you must

experiment in order to find it

Not knowing what the solution would look like, your intuition is a poor guide for estimating and scheduling under systemic uncertainty:• You must experiment in an affordable manner• The results of the experimentation need to be bi-directionally

propagated– Forward, and– Backward


Estimating effort remaining

+ … + =

l e hNo

probability less than

No probability

greater than

Most probable

value

For remaining epics:• Estimate size

with triangular distributions

• Sum using forward propagation (aka Monte Carlo)


Bayesian Example: What improvement looks like: Estimate of weeks late

26


Parting Thoughts: Putting It All Together


The ‘Secret Sauce’ of the Integrative Framework Break your portfolio to the three

buckets

Use the right kind of analytics for each of the three buckets:• Analytics ensure on-going alignment

between projects, programs and portfolios

• In particular, Bayesian analytics enables us to incrementally and iteratively put newly accrued data into consideration:

– In other words, Baysian methods enable iteratively quantified learning

This iteratively quantified learning ensures on-going alignment, hence empowerment


The Virtuous Cycle of the Integrative Framework

Up-to-Date Shared Goals Framework

Based on the Three Buckets and Analytics

Initial Alignment

Empowered Pods

Learning through

Analytics

Realignment


Some things I have learned over the years

To steal ideas from one person is plagiarism; to steal from many is research.

William Mizner

Human beings, who are almost unique in having the ability to learn from the experience of others, are also remarkable for their apparent disinclination to do so.

Douglas Adams

The beginning of wisdom is calling things by their right names.

Chinese Proverb


Questions?


Murray CantorEmail: [email protected]

Contact Me

.




Murray Cantor Areas of research & consulting:

• Agile management• Lean software development• Development intelligence• Systems engineering• Software development analytics• Software governance• Development management due diligence

Major products delivered:• AIX 3.X Graphics subsystem

– Founding member OpenGL ARB• AIX 3.X multimedia subsystem• Top secret system for USAF Space

Command• RUPSE (Systems extension for Rational

Unified Process)

Books:• Object Oriented Project Management• Software Leadership

Sample accolades:• IBM Distinguished Engineer• IBM Plateau 4 Inventor• Software Leadership received 4.7/5

star rating on amazon.com

http://www.amazon.com/Object-Oriented-Project-Management-Murray-Cantor/dp/0471253030/ref=sr_1_1?s=books&ie=UTF8&qid=1429221580&sr=1-1&keywords=Object+Oriented+Murray+Cantor

http://www.amazon.com/Software-Leadership-Guide-Successful-Development/dp/0201700441/ref=sr_1_1?s=books&ie=UTF8&qid=1429221505&sr=1-1&keywords=Software+leadership+Murray+Cantor

http://www.amazon.com/Software-Leadership-Guide-Successful-Development/dp/0201700441/ref=sr_1_1?s=books&ie=UTF8&qid=1429221505&sr=1-1&keywords=Software+leadership+Murray+Cantor

http://www.amazon.com/Software-Leadership-Guide-Successful-Development/product-reviews/0201700441/ref=cm_cr_dp_qt_see_all_top?ie=UTF8&showViewpoints=1&sortBy=byRankDescending


This is joint work with Dr. Israel Gat Areas of research & consulting:

• Agile methods• Devops• Software as a Service (SaaS)• Software development analytics• Software governance• Technical debt & technical due diligence

Major products delivered:• BMC Performance Manager/PATROL• Microsoft Operations Manager• Tivoli Smart Handheld Device Manage

r• EMC Cellera• Digital’s NetView • Nixdorf 8890

Books:• The Concise Executive Guide to Agile

Sample accolades:• Winner, 2006 Innovator Award

[Application Development Trends, May 2006]• “Nearly three times faster time to

market than industry average… … one quarter the expected number

of defects based on team sizes and schedules.” [QSM Study, August 2007]

• “The change you brought to BMC with Agile is the single largest change to the development model that I have ever witnessed in my almost 20 years at BMC.” [Director, BMC Software]

• "When I deal with technical debt issues, I refer to Israel Gat regularly. His approach is the only one I've found that actually works…” [Director, Verisk Health]

http://www.servicetech.com.hk/bmc/patrol.html

http://www.microsoft.com/systemcenter/operationsmanager/en/us/default.aspx

http://www-01.ibm.com/common/ssi/cgi-bin/ssialias?subtype=ca&infotype=an&appname=iSource&supplier=897&letternum=ENUS200-452

http://www-01.ibm.com/common/ssi/cgi-bin/ssialias?subtype=ca&infotype=an&appname=iSource&supplier=897&letternum=ENUS200-452

http://www.emc.com/microsites/unified/unifiedstorage.htm?CMP=KNC-UNIFIED_VNX-Celerra-NAM&activity_id=62225&division=core

http://www.emc.com/microsites/unified/unifiedstorage.htm?CMP=KNC-UNIFIED_VNX-Celerra-NAM&activity_id=62225&division=core

http://www-01.ibm.com/software/tivoli/products/netview/

http://www-01.ibm.com/software/tivoli/products/netview/

http://de.wikipedia.org/wiki/Nixdorf_Computer

http://www.computer.org/portal/web/store?product_id=RN0000008&category_id=ReadyNotes


More granular project characterizations After first level characterization by

novelty, you might want to get into more granular characterization:• Size of the project• Criticality• Timeliness of the project• Quality level • Physical dispersion of the

organization• Logical dispersion of the

organization

A more granular characterization will enable you to:• Fine-tune your project portfolio to more than three buckets; and,• Better balance project needs vis-à-vis organizational needs

Guidance for so doing is provided in the fully-fledged version of this workshop

Source: Alistair Cockburn, Agile Software Development: The Cooperative Game

http://www.amazon.com/Agile-Software-Development-Cooperative-Game/dp/0321482751/ref=sr_1_1?s=books&ie=UTF8&qid=1429220428&sr=1-1&keywords=agile+software+development+the+cooperative+game

http://www.amazon.com/Agile-Software-Development-Cooperative-Game/dp/0321482751/ref=sr_1_1?s=books&ie=UTF8&qid=1429220428&sr=1-1&keywords=agile+software+development+the+cooperative+game


High level view of the framework Capture goals by organization

level based on• Stakeholder needs, business

context– E.g. Organization productivity,

reduced cycle times, more innovation…

• Alignment of levels• Mixture of efforts

At each level, specify sense and respond loops to achieve the goals• Analytics• Practices


Appendix: A quick note on carrying out more granular project characterizations


Appendix B: Auxiliary SlidesBackup slides

38


W. Edwards Demming Quotes “It is a mistake to assume that if

everybody does his job, it will be all right. The whole system may be in trouble.”

“It is management's job to know.”

“I am not reporting things about people. I am reporting things about practices.”


A Pragmatic Note No need to start by characterizing all programs/projects in

your portfolio unless you do so as an integral part of your annual (?) planning and budgeting process• Bootstrapping by starting with a few projects and increasing the

number of projects gradually is quite appropriate and effective in implementing the Integrative Framework

• At a certain point in time, you will need to reengineer both your organization and processes

– No need to rush with so doing before you have a critical mass of teams


Conceptual map for tying it all together


Pert Estimation• Pert Estimation: For set of a tasks (T1, …,Tn), say user

stories, set the sizes (user stories)– (l1, …, ln) be the best cases (lower bounds)– (e1,…,en) be the expected cases– (h1,…,hn) be the worse cases (upper bounds)

• Let L = l1 + …+ ln, E = e1 + …+ en, H = h1 + …+ hn

• Then– mean of the sum of triangulars ~ (L+4E+H)/6– Standard deviation ~ ( H-L)/6

42


We need to get a view of the data to understand team performance and set goals We will analyze the time (in hours) it took

to go from ‘created’ to ‘closed’.

We get a first look at the data with a scatter gram.

Each dot is a closed defect. The height is the number of hours.

Note that there are outliers with values as much as 1150 hours. This is typical of real data sets and may be caused by mistakes in the data capture or some abandoned defect.

With customer agreement we trim the data at 400 hours.

43


We need to get a view of the data to understand team performance and set goals

Since the curve is not a bell-shaped curve, we choose percentiles as a way to measure performance and set goals.

With this data set, we see that • Half of the defects were closed in 93

hours or less.• 80% of the defects were closed in 179

hours or less.• 90% of the defects were closed in 267

hours or less.

They set an initial goal of completing 80% of the defects in 100 hours or less.

44


What improvement looks like By revisiting the value stream

map, the team applied flow principles to reduce the cycle time.

They get a new data set.

45


We will describe the date as a probability distribution

• Height is the likelihood of the value• Area under curve is the probability of value falling within range• Mean is expected value• Standard deviation is measure of uncertainty


Reducing standard deviation improves probability

Sd = 10• Prob(10,30) =68%

Sd = 5• Prob(10,30) =95%

Sd = 1• Prob(10,30) = 100%


Planning to ship on the expected date is a 50/50 bet


Option: Move out the date


Option: Reduce content

Mean = 9.5, standard deviation = 2


Reducing the standard deviation improves the odds

Mean = 11, standard deviation = 2

Mean = 11, standard deviation = 1


Steps for Managing Innovation: Identifying Risk• Do enough design to break the new deliverable into separate smaller

deliverables that can be partitioned to team or individuals:

• For each deliverable, elicit the 3 cases.– Lowest case, l, highest case, h, and expected case, e, for the time to complete

their tasks.

• When h is much larger than l, that constitutes uncertainty and risk– Ask why?– What would they need to know to reduce the uncertainty?

• User needs,• Usability• Technical performance• …

– How would they learn what they needed

• Prioritize work that helps them learn what is needed to reduce the uncertainty


Managing Uncertainty• Use a prediction tool to roll up the 3 cases into a probability of

completion– There are a set of techniques, beyond the scope of this workshop.

• Carry out the risk reduction tasks

• Update the task set with the new information gained

• Measure and track to narrowing of the shape of the distribution


Monte Carlo is used for doing arithmetic with distributions

54

l e h

No probability less than

No probability greater than

Most probable value

+ … + =


Bayes is the way for development teams and management to deal with uncertainties

In types II and III development, quantities such as time, cost to complete, and velocity are not known for certain. • There is not enough known to make exact predictions• You need to utilize the actual data you produce sprint by

sprint

Bayesian analysis is the centuries old method for rigorously dealing with with uncertain quantities.

Bayesian analytics allows everyone on the team to learn together.

Attributes of Bayes: Uncertain quantities are specified probabilities The probabilities capture both the best/worst estimates and the level of

uncertainty The probabilities/beliefs are updated as information, evidence comes in. The probability distributions can be “added,” “multiplied,” etc.


Now that you abolished the “One Size Fits All” myth at the individual project level… …. you need to determine how you would like your project

and program management organizations and processes to support the individual projects…

….without repeating the “One Size Fits All” error all over again

The key for so doing has two parts to it:1. The Aligned Autonomy principle: attain empowerment through on-

going alignment2. The Lean principle, “Does a specific flow of either data or decisions

through a silo really add value, or is it a source of waste in the context of the company’s business design?”


On-going alignment through integrated analytics


Project, product deliveryProject manager, team lead

Commits to


Profit, return on investment, mission fulfillmentLine of business executive Commits to

Com

mitm

ents Ana

lytic

s

After


Different disciplines apply to different parts of the landscape

Lean Manufacturing

Innovation Management

Queuing Theory

Systems Theory

Toyota Management System

Agile Management

Bayesian Nets

Analytics

Quality Control


Part III: Fitting analytics and practices to moderate innovation projects For moderate innovation augment one of the readily available

Agile software methods with some innovative methods for dealing with uncertainty, e.g. • Triangular estimation forces discipline that will protect you against

overly optimistic or overly pessimistic estimates

Use (L+4E+H)/6 to estimate realistically• (L=∑l; E=∑e; H=∑h)

Add some second generation lean methods for dealing with throughput• Backlog management (aka Kanban)• Eliminate NVA activities• Streamline decisions process

l e h


Analytics-based, menu-driven framework


Project, product delivery

Project manager, team lead

Commits to


Profit, return on investment

Line of business executive Commits to

Concerns

com

mitm

ents

The Integrative Framework

Up-to-Date Shared Goals

Framework

Initial Alignment

Empowerment

Learning through

Analytics

Realignment


A tough predicament On the one hand, we must adopt and apply the lessons

learned from the adjacencies• There is too much wisdom there to ignore

On the other hand, the need for prescriptive guidance will not go away• Absent prescriptive guidance, an Agile project team often amounts to

little more than a budgeted Kumbaya


Start by Abolishing the “One Size Fits All” Myth Tempting that this myth is on multiple levels, the fact of the

matter is software development is not a controllable process which can be modeled through a single set of parameters

Rather, different kinds of projects have different goals:

The different goals determine different kinds of:• Analytics; and,• Practices

Examples:– Production bug; – Add a new feature; – Develop a new framework


The only decision you need to make Do the increases in

• Unleashing the creativity of your teams;• Generating value for your customers;• Increased resiliency in your operations (pod networks instead of

chains);• Learning as you go along;• Reporting and governing through Bayesian Networks; and,• Improving business precision

Trump industrial age Taylor-type “scientific management”?• Including:

– Organizational optimization towards very long term stability– The false sense of predictability it gives you/your superiors/your stakeholders


Example: To meaningfully measure cycle times, consider the data set of times between nodes in the VSM

Take a look at a histogram (bar chart) of the data.

The x-axis is the age and the y-axis is the number of defects that had that age at closure.

Notice the shape does not look like a bell-shaped curve. This is always the case for real data.

64

Before After

Improvement


Examples of backlog size fluctuations due to the variation of effort


Examples of backlog size fluctuations due to the variation of effort

Implications

1. Small changes have big impact

2. Operations require constant monitoring and adjusting

integrative keynotev2

Documents