stc15 tutorial technical debt part 3 - usu€¦ · managing technical debt for ipdm projects the...

10/6/15

1

STC 2015 Tutorial Part 3 Managing Technical Debt for

Software and Systems Development Projects

IEEE Software Technology Conference Long Beach, CA October 12, 2015

STC2015 Tutorial Managing Technical Debt Copyright © R.E. Fairley 2015

Tutorial agenda

Part 1: introductions and concepts of technical debt Part 2: Planning and organizing to manage technical debt Part 3: Managing technical debt

2

10/6/15

2


Part 3 topics

Part 3 topics include: •  The life cycle continuum of software

development models •  Risk of and opportunities for managing

technical debt within the continuum

3


The continuum of software development process models

Highly Predictive: linear, one pass phases Incremental Predictive: incremental phases Iterative-Adaptive: iterative cycles Highly Adaptive: highly iterative cycles

Highly Predictive

Highly Adaptive

Incremental Predictive

Iterative Adaptive

see The Software Extension to the PMBOK® Guide Fifth Edition http://marketplace.pmi.org/Pages/ProductDetail.aspx?GMProduct=00101457501

4

10/6/15

3


Some factors in choosing a software development process

Factor to be considered include: Requirements clarity and volatility Level of stakeholder involvement Complexity and size of project and product Criticality of quality attributes Flexibility in allocation of schedule and budget Regulatory and policy constraints o e.g., safety certification or security

accreditation

5


Selecting the development process

Each development process encounters risks of accumulating technical debt o And provides opportunities to manage

technical debt Selecting the wrong development process is a primary factor that contributes to accumulation of technical debt o and failure to mitigate technical debt

6

10/6/15

4


Risk and opportunity (1)

Risk is characterized by probability and cost Risk involves the probability that a negative event or situation will occur o For example, the probability of accumulating

technical debt And the cost to mitigate a risk if it becomes a problem o The cost of paying technical debt that has

been accumulated

7


Risk and opportunity (2)

Opportunity is the probability that a positive event or situation will occur o The probability that technical debt will be

detected before significant debt is incurred o The reduced cost of mitigating the debt when

detected, as compared to letting it accumulate o Plus the opportunity to improve the

development processes to reduce the probability and cost of future technical debt

8

10/6/15

5


The Software Development Process Continuum

Highly Predictive: linear, one pass phases Incremental-Predictive: incremental phases Iterative-Adaptive: iterative cycles Highly Adaptive: highly iterative cycles

Highly Predictive

Highly Adaptive


Iterative Adaptive


9


A Highly Predictive Development Model (HPDM)

Analyze

Architect

Design

Construct

Integrate

time

Elicit

Test

Accept

10

10/6/15

6


HPDM Goals

The goals for HPDMs are: 1.  To initially develop requirements that are

complete, correct, and stable And to develop a suitable software architecture

2.  To accurately predict (initially) effort, cost, resources, skills, and technology needed to successfully complete the project

3.  To complete each project phase, completely and correctly, before proceeding to the next phase

11


Opportunities for managing HPDM technical debt

Opportunities for managing HPDM technical debt include:

1.  Establishing and maintaining traceability links

2.  Maintaining baselines 3.  Practicing version control 4.  Conducting milestone reviews 5.  Iteratively developing the software

during the construction phase

12

10/6/15

7


http://www.engineering.com/portals/0/BlogFiles/Systems_Engineering_V_diagram.jpg

The Vee HPDM

System development

Discipline-specific component development

13


The Vee model and technical debt

The Vee model includes development of test plans as the work products are being developed o Which can identify risk factors for

accumulation of technical debt o Because of issues that are incorrect,

incomplete, inconsistent, or vague

14

A shortcoming of HPDMs: problems created first are detected last

10/6/15

8


Risk factors for HPDM technical debt

Risk factors that increase the probability of incurring HPDM technical debt include: o  Infrequent milestones o  Ineffective milestone reviews o  Infrequent stakeholder interactions o Lack of initial traceability o Failure to maintain traceability o Failure to maintain a project balance o  Inadequate change control o End-of-project V&V

15


Traceability links and technical debt Bi-directional traceability provides the following checklist of opportunities to detect and mitigate technical debt

Are all requirements present in the design? Are all design elements present in the code modules? Are integration and test plans traced to requirements and design? Other links?

16

Are there any missing elements? Are there any extraneous elements?

10/6/15

9


Risk factors

Risk factors for managing HPLC technical debt include: o  Infrequent milestones o  Infrequent stakeholder interactions o Lack of traceability o Failure to maintain a project balance o  Inadequate change control o End-of-project V&V

17


Technical debt: maintaining a project balance

Impact analysis is concerned with assessing the impact of proposed changes o  In-scope changes versus out-of-scope changes

18

Product Scope & Quality attributes

•  Schedule •  Budget •  Resources •  Technology

Failure to maintain a balance is a primary source of technical debt for HPDM projects

10/6/15

10


Risks of HPDM change control

Insufficient impact analysis An ineffective CCB An un-empowered CCB Lagging CCB responses Failure to maintain a project balance Failure to verify and validate changes Failure to communicate changes to all affected parties Other risks?

19


A Highly Predictive Life Cycle (HPDM)

Analyze

Architect

Design

Construct

Integrate

time

Elicit

Test

Accept

20

10/6/15

11


Reducing the risk of HPDM technical debt during software construction

Iterative software construction can reduce the risk of accumulated technical debt for HPDM projects Iterative software construction involves: o Constructing the software in incremental stages

§ With incremental V&V o And with frequent demonstrations of progress (or

lack thereof) §  Internal demonstrations for the software

developers § External demonstrations for the stakeholders

21


The risk of external demonstrations during software construction

Stakeholders may request extensive changes based on demonstrations of the evolving software o This is a symptom of earlier accumulated

technical debt §  In the requirements and design phases

o And may indicate that a HPDM is the wrong approach for the project

22

10/6/15

12


Factors that can reduce the risk of accumulating HPDM technical debt

Stable, well-defined requirements Staged, cross-functional resources Familiar system Familiar customer Short duration project Adequate time, resources, skills, and technology Iterative software construction

23




Highly Predictive

Highly Adaptive


Iterative Adaptive


24

10/6/15

13


An Incremental-Predictive Development Model (IPDM)

Demo Working System

I1

Demo Working System I1+I2

Demo Working System

I1+I2+I3

V&V Design

Construct &

Integrate

V&V

V&V

Feature Set 1

Design Construct

& Integrate

Design Construct

& Integrate

time

Feature Set 2

Feature Set 3

Analyze

Architect

. . . .

25


The IPDM

Periodic demonstrations of system increments involve stakeholders o  and may result in revisions to requirements

§ plus redesign and reallocation of feature sets o  and reallocation of resources and schedule

And can provide objective evidence of progress o Or lack thereof: accumulated technical debt

Early deliveries of subset capabilities for evaluation by users are possible, if desired

26

10/6/15

14


IPLC Stakeholder Involvement

Stak

ehol

der I

nvol

vem

ent

Project Duration

Increment V&V & Demos

Initiation and Planning

System Acceptance

Increment demo intervals may be unequal

27


Some IPDM Capability Prioritization Criteria

allocate the system requirements to increments of growing system subsets some of which are to be periodic delivered OR allocate the system requirements to architectural layers that incrementally result in a succession of layered virtual machines OR establish the architectural skeleton first and incrementally add capabilities OR establish interfaces to external components first OR incrementally incorporate components to be reused

28

10/6/15

15


How many feature sets are needed?

•  The number of features per feature set and the duration of the incremental build phases for the feature sets may vary from feature set to feature set o But the durations of each incremental phase should

not exceed one month •  Limiting the duration of each incremental phase to one

month will determine the number of feature sets and incremental phases needed •  And may require modifying the overall schedule and

budget •  Or dropping some of the lower priority requirements

29


Managing technical debt for IPDM projects

The opportunities and risk for managing IPDMs technical debt are similar to those for HPDMs

With the added benefit of o  Incremental verification and validation o Periodic stakeholder demonstrations of the

evolving product o And the opportunity to make some periodic

revisions to requirements and design o And to restructure the remaining feature sets o And delivery of subset capabilities to users

for evaluation, if desired

30

10/6/15

16


Managing IPDM technical debt

IPDM technical debt can be successful managed when:

Requirements can be mostly defined initially Some flexibility in managing prioritized requirements will be permitted o  plus dynamic allocation of resources and schedule

to increments Periodic V&V and stakeholder involvement provides feedback o  as a basis for adding, deleting, and revising features o  and to support management of technical debt

31




Highly Predictive

Highly Adaptive


Iterative Adaptive


32

10/6/15

17


An Itera)ve-‐Adap)ve Development Model (IADM)

Product Vision

Product Feature Set

Product Planning

Itera4on Planning

Itera4on Feature Set

Product Increment

Demo Working So9ware

Demonstra4on and Review

Daily standup mee4ngs & frequent demos

frequent internal iteration

Demo Test

Design Construct

Ini4al Product Planning

External Itera4ons

Internal Itera4ons

33


Internal Itera)ons by Adap)ve Teams

select next feature(s)

specify requirement(s)

prepare test cases

add code for new features, test, refactor

demonstrate capabilities

frequent iterations & daily

stand-ups

Team members

TDD Internal demonstra4ons of evolving capabili4es may occur on a daily basis

34

10/6/15

18


Test-driven development (TDD)

Internal demonstrations, test-driven development, and refactoring are three iterative approaches to controlling technical debt Internal demonstrations provide evidence of progress TDD involves writing test scenarios and test cases before writing code o May include both static and dynamic tests

Refactoring involves restructuring the software without altering existing behavior o Why not alter behavior?

35


Adaptive Software Development

Technical debt can be successfully managed when pursuing an adaptive approach by: o  developing an initial understanding of the nature

of the desired product to be delivered, o  adopting an architectural style or establish a

shared design metaphor o  determining the constraints on schedule, budget,

resources, and technology that must be observed o  developing a continuous, ongoing relationship with

a knowledgeable user representative o  periodically demonstrating the evolving software o Concurrently evolving the requirements and the

software 36

10/6/15

19


AND BY observing the 10 attributes of agility

37

The 10 attributes of agility are included at the end of these slides

Itera4ve-‐Adap4ve projects are “agile” to the extent that they observe the 10 aOributes of agility


Controlling technical debt by tracking IADM progress

Burndown charts and velocity charts can be used to track progress and control technical debt for IADM projects

38

10/6/15

20


A Burndown Chart for an IALC Iteration Cycle

Iteration Cycle in Work-Days 963 12 15

Num

ber o

f Rem

aini

ng T

asks

0

5

10

15

20

25

30 Planned tasks remaining Actual tasks remaining

39


Tracking Progress Using a Velocity Chart

Iteration #

Num

ber

of C

ompl

eted

Tas

ks p

er It

erat

ion

0

5

10

15

20

25

30

35

1 2 3 4 5 6 7 8 9 10

Series2

40

10/6/15

21


Carrying Tasks Forward

Retrospective meeting: o Why didn’t we complete everything?

Planning meeting: o How are we going to handle the carry-forward?

§ While controlling technical debt

•  The retrospective meeting and planning meeting can be combined

•  unless there is a time lag between iterations •  as in staged procurement or maintenance upgrades

41


Handling carry-forward of IADM tasks

Persistent carry-forward of tasks accumulates technical debt Desirable options for paying IADM technical debt o Use occasional iteration cycles to clear accumulated

technical debt o Re-scope the number of tasks that can be completed

during each time-boxed iteration cycle o Deliver fewer prioritized requirements o Extend the schedule o Add an IADM mentor to the team o Spin off some of the work to another team

42

10/6/15

22


Handling carry-forward of IALC tasks

Undesirable options for paying IALC technical debt Add more team members Replace team members (except in extreme cases) Reduce TTD, refactoring, and verification testing Reduce external demos and interactions with the customer Require overtime

43




Highly Predictive

Highly Adaptive


Iterative Adaptive


44

10/6/15

23


A Highly Adaptive Life Cycle

potential system delivery

Backlog of

use cases or customer stories

team selects use case or

hears customer

story

team discusses

options with customer

team writes test cases

team refactors,

adds capability, and

verifies

team demonstrates capabilities

frequent iterations

Customer

TDD

45


Attributes of HADM projects

Customer controls the backlog of use cases or stories o  and is continuously “in the loop” with the software

developers Demonstrations of progress occur on a daily basis o Or multiple times per day

The “development team” may be 1 or 2 persons

46

10/6/15

24


Controlling technical debt for HADM projects

There are no issues of technical debt because progress is continually demonstrated o And is under the control of the customer

47




Highly Predictive

Highly Adaptive


Iterative Adaptive


48

10/6/15

25


Tutorial Summary

49


Goals for this tutorial

To present the nature of and consequences of technical debt and To present techniques for anticipating, detecting, and mitigating technical debt and to

Address your questions and concerns And hear your recommendations for improving the processes, methods, tools, and techniques for avoiding and managing technical debt

50

10/6/15

26


Tutorial agenda

Part 1: Concepts of technical debt Part 2: Planning and organizing to manage technical debt Part 3: Managing technical debt

51


Part 1 topics

Concepts of debt Accumulation of technical debt for software development Interest rates for technical debt Reasons for knowingly and unknowingly accumulating technical debt Passing it on

52

10/6/15

27


Session 2 topics

•  Excuses for not planning •  Making a plan for the technical work (SwTMP)

53


Part 3 topics

•  The life cycle continuum of software development models

•  Opportunities and risks for managing technical debt across the continuum

54

10/6/15

28


Questions? Comments?

55


10 Attributes of Agility for Iterative-Adaptive projects

56

10/6/15

29


A9ributes of Agility for Adap)ve SDLCs

1.  Increments of working deliverable so9ware are produced on a periodic basis

2.  Adap4ve itera4on cycles are of the same dura4on (i.e., are “4me boxed”) for a project but some cycles may be of longer or shorter dura4on by excep4on o  dura4ons of the 4me-‐boxed itera4on cycles may be

weekly, bi-‐weekly, or monthly but not more than monthly 3.  Increments of working deliverable so9ware are not

necessarily produced by each itera4on o  increments and itera4ons are dis4nct

4.  Requirements, design, and the so9ware product emerge as the project evolves

57


A9ributes of Agility (2)

5.  A representa4ve customer, customer’s representa4ve, and/or knowledgeable user is involved on a con$nuing basis

o  involvement includes observing periodic demonstra4ons of working, deliverable so9ware at the end of each itera4ve development cycle

o  In addi4on, a representa4ve customer or customer’s representa4ve accepts shared responsibility for further product development based on demonstra4ons of working deliverable so9ware

o and accept responsibility for managing the constraints on project scope (schedule, budget, and resources)

58

10/6/15

30


Notes The customer representa4ve must commit to: a.  Being con4nuously available b.  Providing updates to the requirements based on periodic

demonstra4ons of the incrementally evolving product that involve the customer and other key stakeholders

c.  Discussing proposed changes to the requirements with the so9ware developers to assess technical risk and the impact of proposed changes on the product, schedule, and needed resources

d.  Accep4ng responsibility for maintain a balance among schedule, budget, and requirements to be implemented

59



6.  Adap4ve so9ware development teams are small (i.e., 10 or fewer members) and are self-‐organizing

o  large projects include mul4ple small teams

7.  Short daily standup mee4ng, retrospec4ve mee4ngs, and planning mee4ngs are used

8.  Each so9ware development team is cross-‐func4onal o  includes the combina4on of skills needed to accomplish

the work ac4vi4es

o  func4onal experts may be involved occasionally as needed

60

10/6/15

31



9.  All members of each so9ware development team are assigned to one project at a 4me

10.  Over4me is not required

o  Except on rare occasions of limited dura4ons

o  So9ware developers should be able to maintain a steady pace indefinitely o  Without burnout

o  And without sacrificing their personal lives

61

stc15 tutorial technical debt part 3 - usu€¦ · managing technical debt for ipdm projects the...

Documents