1

Introduction to Agile

22

Agenda

Introduction to Agile

Early examples of agile projects

Workin

g D

raft - La

st Mod

ified

10

/20

/20

10

7:5

7:2

4 A

MPrin

ted

5/1

8/2

01

0 8

:28

:55

AM

Software engineering originated together with the first computers in the 1940s

Early days▪ Experimentation and

research▪ Programming by wiring▪ WWII communication

encryption and decoding

Advent of programming▪ Management by schedule impossible▪ Everything open source▪ Programs rewritten with each new

computer

1941 1945 1957 1960 1964

First computer

Algorithmic programming

language

Compiler and

magnetic storage

First key-

board

Fortran COBOL Basic

1952 1956

Workin

g D

raft - La

st Mod

ified

10

/20

/20

10

7:5

7:2

4 A

MPrin

ted

5/1

8/2

01

0 8

:28

:55

AM

A high rate of critical failures led to the “software crisis” 1965-1985

Budget overruns

▪ Most large IT projects went significantly over schedule and budget

▪ IBM’s OS/360 project in the 60’s many years and multi million dollars over schedule

Critical quality defects

▪ Bugs and quality defects in commercial software applications caused critical/fatal incidents

▪ F18 plane crashed due to missing exception condition

Examples

▪ Colorado River flooding in 1983, due to faulty weather data and/or faulty model; too much water was kept dammed prior to spring thaws

Workin

g D

raft - La

st Mod

ified

10

/20

/20

10

7:5

7:2

4 A

MPrin

ted

5/1

8/2

01

0 8

:28

:55

AM

Waterfall model was embraced as the way out of the crisis

Publication Key messages

Managing the development of large software systems: Concepts and Techniques - Dr. Winston Royce, 1970

▪ Software development is stepwise process

1. Requirements2. Design3. Implementation4. Integration5. Acceptance testing

▪ Detailed requirements specification is essential

Mythical man-month– Fred Brooks, 1975

▪ Invest a lot of time up front to build a coherent architectural design

▪ Adding more developers late in the project will only slow things down

▪ Create a detailed milestone schedule, and manage hard towards it

“In order to procure $5 million dollars of software I would estimate a 1,500 page specification is about right ...”

“I made a multi-million dollar mistake by not developing a coherent architecture for OS/360 before we started developing”

“How does a large software project get to be one year late? One day at a time!”

Workin

g D

raft - La

st Mod

ified

10

/20

/20

10

7:5

7:2

4 A

MPrin

ted

5/1

8/2

01

0 8

:28

:55

AM

And it was widely accepted as the standard way of doing software development

Based on Winston Royce’s paper in 1970

Waterfall

Developed by the German Ministry of Defense in 1986

V-Model

Developed by the US Dept of Defense in 1988

DoD-STD-2167

Workin

g D

raft - La

st Mod

ified

10

/20

/20

10

7:5

7:2

4 A

MPrin

ted

5/1

8/2

01

0 8

:28

:55

AM

But it gradually became apparent that waterfall development does not always produce the desired results

45

19

16

13

7

Never

Rarely

Sometimes

Often

Always

…which leads to a very big part of the delivered functionality being redundant

SOURCE: J. Johnson, Standish Group, Keynote speech at XP 2002 conference in Sardinia, Italia 2002

Traditional IT projects have scarce business and user involvement…

Requirement specifications

Detailed design

Implementation

Acceptance testing

Unit testing and integration

Business and user involvement

Only IT involvement

Features actually being used by the customerPercent

Workin

g D

raft - La

st Mod

ified

10

/20

/20

10

7:5

7:2

4 A

MPrin

ted

5/1

8/2

01

0 8

:28

:55

AM

In the 1990’s a number of alternative, “light-weight” approaches emerged

Methodology Inventor/origin First implementationPitched as

▪ 1994▪ Advanced Development Methods -

process automation software. 8 developers. VMARK - OO software development environments

Scrum ▪ "Management and control process that cuts through complexity"

▪ Jeff Sutherland, Ken Schwaber, Mike Beedle

▪ USA

Extreme programming

▪ Addressing “the specific needs of software development conducted by small teams in the face of vague or changing requirements”

▪ Kent Beck▪ USA

▪ Pre-2000, ▪ C3 project Chrysler; 8 developers

Feature-driven development

▪ Blends a number of industry-recognized best practices into a cohesive whole

▪ Jeff De Luca▪ Australia

▪ 1997▪ 15 month, 50 person software

development project at a large Singapore bank

Source: Press search

▪ December 1994▪ Most large US IT defense projects

1994 - 2000

▪ “If a system is developed in multiple builds, its requirements may not be fully defined until the final build”

▪ Department of Defense

▪ USA

Mil-Std-498

Dynamic systems development method

▪ “A framework of controls and best practice for rapid application development”

▪ DSDM Consortium

▪ UK

▪ 1995▪ Been used by BT, Orange, Dept. of

Health, Syndeco/Boston Globe, Sema Group, Logica and British Midlands

Workin

g D

raft - La

st Mod

ified

10

/20

/20

10

7:5

7:2

4 A

MPrin

ted

5/1

8/2

01

0 8

:28

:55

AM

And in 2001, 17 prominent developers gathered in Snowbird Utah to declare the “Agile Manifesto”

“We are uncovering better ways of developing software by doing it and helping others do it. Through the work we have come to value:▪ Individuals and interactions over processes and tools▪ Working software over comprehensive documentation▪ Customer collaboration over contract negotiation▪ Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.”- The Agile Manifesto, February 2001

Individuals and Interactions

Working software

Customer collaboration

Responding to change

Processes and tools

Comprehensive documentation

Contract negotiation

Following a plan

Workin

g D

raft - La

st Mod

ified

10

/20

/20

10

7:5

7:2

4 A

MPrin

ted

5/1

8/2

01

0 8

:28

:55

AM

The 12 principles behind the Agile manifesto

Our highest priority is to satisfy the customer through early and continuous delivery of valuable software

Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done

Continuous attention to technical excellence and good design enhances agility

Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage

The most efficient and effective method of conveying information to and within a development team is face-to-face conversation

Simplicity--the art of maximizing the amount of work not done--is essential

Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale

Working software is the primary measure of progress

The best architectures, requirements, and designs emerge from self-organizing teams

Business people and developers must work together daily throughout the project

Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely

At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly

Workin

g D

raft - La

st Mod

ified

10

/20

/20

10

7:5

7:2

4 A

MPrin

ted

5/1

8/2

01

0 8

:28

:55

AM

Early indications are that agile is more effective than waterfall

05

10152025303540455055

6 9 12 15 18 21 24 27 30 33 36

Very few large IT projects deliver on time and on budget …

... whereas majority Agile projects are considered to be successful

Months

IT Projects delivering required functionality on budget and on time% projects

SOURCE: “ChAOS: A recipe for success”, “ChAOS in the new millennium”, The Standish Group (1998, 2000), State of Agile development by VersionOne (2008);

24

30

46

50%or less

More than50%

More than 80%

Percentage of Agile projects considered successful% projects

Workin

g D

raft - La

st Mod

ified

10

/20

/20

10

7:5

7:2

4 A

MPrin

ted

5/1

8/2

01

0 8

:28

:55

AM

An adoption of Agile has become widespread

31

69

Proportion of organisations with at least one Agile project in progress, 2008Percent

SOURCE: DDJ 2008 Agile Adoption Survey www.ambysoft.com

Workin

g D

raft - La

st Mod

ified

10

/20

/20

10

7:5

7:2

4 A

MPrin

ted

5/1

8/2

01

0 8

:28

:55

AM

There are several advantages of Agile compared to traditional waterfall development

Visibility

Time

Adaptability

Business value Risk

Workin

g D

raft - La

st Mod

ified

10

/20

/20

10

7:5

7:2

4 A

MPrin

ted

5/1

8/2

01

0 8

:28

:55

AM

But Agile is not a silver bulletFailed projectsCommon pitfalls

▪ Lack of long-term thinking leading to brittle, convoluted architecture

▪ Lack of clarity about requirements

▪ Business not being able to or willing to prioritize requests

▪ Too much testing deferred to the customer, causing irritation

▪ Developers doing what they “feel” is right instead of sticking to the agreed process and standards

Railhead: US department of homeland security spent ~500 mUSD on a IT project to enable sharing, fusing and analysis of terrorist intelligence data across government agencies.

Key shortcomings

Poor architecture: Same data field present in 463 tablesLack of documentation: 295 of these tables not documentedFunctionality not implemented: Poor database searchability. Top priority functionality not being delivered, no reason given

CJEP: Australian Justice department spent 54 mUSD on a IT project to facilitate the flow of documents between police, legal representatives, the courts and other related agencies to improve efficiency of the court system

Poor planning: Severe underestimation of complexity. Poor scoping leading to final deliverable 9 years delayed at 4x cost Failure to commit to standards: Failure to meet UI design standard resulted in rewrite of large part of the program

Workin

g D

raft - La

st Mod

ified

10

/20

/20

10

7:5

7:2

4 A

MPrin

ted

5/1

8/2

01

0 8

:28

:55

AM

And agile is not always appropriate

Stability and understanding of requirements

Low High

Low

High

Degre

e o

f busi

ness

know

ledg

e

and e

xpert

ise r

eq

uir

ed

Waterfall/V-model with

offshore developers

Agile, “light-weight” methods (XP / Scrum) with

On-location, senior developers

More formal, but iterative methods (RUP, Evo)

with offshore developers

Recommended delivery models

Workin

g D

raft - La

st Mod

ified

10

/20

/20

10

7:5

7:2

4 A

MPrin

ted

5/1

8/2

01

0 8

:28

:55

AM

To deliver maximum value, Agile must be adapted to the context, and managed properly

Maximize value from

Agile

Only use Agile when relevant

▪ When requirements are stable and well understood, V-model is faster and safer

▪ Agile is not appropriate for junior developers

Manage stakeholders

▪ Communicate Agile methodology and rationale to business

▪ Ensure representation of key stakeholders on all teams

▪ Facilitate prioritizations▪ Communicate openly

Adapt to situation

▪ Leverage existing tools and infrastructure

▪ Assess existing skill sets, and ensure proper coaching

▪ Have at least one agile “champion” per team

Manage process

▪ Involve all team members in iteration planning sessions

▪ Ensure compliance with key principles:– Continuous testing and

integration– Ongoing attention to

simplification and refactoring

Workin

g D

raft - La

st Mod

ified

10

/20

/20

10

7:5

7:2

4 A

MPrin

ted

5/1

8/2

01

0 8

:28

:55

AM

SCRUM is the most widespread of the Agile methodologies

Agile method used most closely% of organizations

10

4

5

9

12

23

37

Other

Feature driven development

DSDM

Custom/other hybrid

XP

Scrum/XP hybrid

Scrum

Organizations use different methods …

… and decide which practices to use within that method

24

36

38

46

50

55

Pair programming

Collective ownership

Test-driven development

Refactoring

Continuous integration

Daily standup

Use of agile practices% of organizations

Source:Report: “State of Agile Development 2007,” VersionOne

Workin

g D

raft - La

st Mod

ified

10

/20

/20

10

7:5

7:2

4 A

MPrin

ted

5/1

8/2

01

0 8

:28

:55

AM

Early agile projects

Project DescriptionYear Organization

X-15 ▪ Construction of hypersonic jet

1950’s NASA

A major contribution to the X-15 success over the long run was its emphasis on incremental development

The X-15 lessons learnedNASA Dryden research facility

Trident submarine

▪ Command and control system for the first USA Trident submarine

▪ Over one million lines of code

1972 IBM FSDThe project was organized in four time-boxed iterations of 6 months each

Integration engineering perspectiveThe journal of systems and software

Army Site Defence Software project

▪ Software project for ballistic missile defence

1972 TRWThe project was developed in 5 iterations, iteration 1 did the tracking of a single object. By iteration 5 two years later the project was complete

Managing the development of reliable softwareInternational conference on reliable software

Workin

g D

raft - La

st Mod

ified

10

/20

/20

10

7:5

7:2

4 A

MPrin

ted

5/1

8/2

01

0 8

:28

:55

AM

Early agile projects

Project DescriptionYear Organization

LAMPS ▪ USA navy helicopter ship system

▪ 4 year, 200 person software effort

1975 –1979

FSD

Delivered in 45 time boxed iterations. Every one of those deliveries were on time and under budget

Principles of software engineeringHarlan Mills

Primary Avionics Software System

▪ The central piece of NASA’s space shuttle software

1977 –1980

IBM FSD17 iterations over 31 months. Traditional development not suitable because of evolving requirements

Design, development, integration, space shuttle flight software system

Communications of the ACM

Workin

g D

raft - La

st Mod

ified

10

/20

/20

10

7:5

7:2

4 A

MPrin

ted

5/1

8/2

01

0 8

:28

:55

AM

Workin

g D

raft - La

st Mod

ified

10

/20

/20

10

7:5

7:2

4 A

MPrin

ted

5/1

8/2

01

0 8

:28

:55

AM

Introduction to SCRUM

Workin

g D

raft - La

st Mod

ified

10

/20

/20

10

7:5

7:2

4 A

MPrin

ted

5/1

8/2

01

0 8

:28

:55

AM

SCRUM is an iterative development approach with seven key components

Input from end-users, customers, team, and other stakeholders

Product backlog

1 23

45

6

7

Product owner Team

Team selects how much to commit to do by Sprint’s end

Sprint backlog

Product backlog refinement

Scrum masterDaily scrum meeting and artifacts update

Review

Potentially shippable product increment

Retrospective

No changes in duration or goal

Sprint 1-4 weeks

Sprint planning meeting (parts 1 and 2)

Workin

g D

raft - La

st Mod

ified

10

/20

/20

10

7:5

7:2

4 A

MPrin

ted

5/1

8/2

01

0 8

:28

:55

AM

The product owner owns and prioritizes the product backlog

1

Product backlog

FeatureValue to business

Value to customers

1. Proactive notification to EAT changes

▪ Increase C.S.▪ Reduce cost by

automating internal processes

Ranked as “very important” by 73% of customers

2. Instant booking confirmation

3. IT supported issue resolution

Reduce average wait time from 6 hours to 15 seconds

▪ Increase C.S., and thereby market share

1. Ensure product relevance▪ Align with business strategy▪ Review with internal

stakeholders▪ Arrange customer reviews▪ Source company insight

2. Maximize return on investment

▪ Prioritize features using high-level business cases

▪ Revise prioritization based on effort estimates

▪ Generate and gather innovation and improvement ideas

3. Safeguard quality▪ Provide feedback on emerging

solution▪ Approve or reject release

candidates

Product owner responsibilities

30

Effort

85

Faster, more effective and transparent resolution process

▪ Increase C.S.▪ Reduce cost by

automating internal processes

50

4.5.6..…

.

.

.

.

.

.

▪ ▪ ▪

xyz

Workin

g D

raft - La

st Mod

ified

10

/20

/20

10

7:5

7:2

4 A

MPrin

ted

5/1

8/2

01

0 8

:28

:55

AM

The team decides how much they can commit to deliver in the next sprint

2

1.

2.

3.

4.

5.

6.

7.

8.

9.

1.1 1.2

1.3

2.1

2.2

2.3

2.4

3.1

3.2

1.1

1.3

2.1

2.2

3.1

1.2

Detailed user stories

Sprint backlog

User st

orie

s

Epics

Product backlog

Team capacity80 story points Ta

sks

Workin

g D

raft - La

st Mod

ified

10

/20

/20

10

7:5

7:2

4 A

MPrin

ted

5/1

8/2

01

0 8

:28

:55

AM

There are no changes to priorities or scope during the course of the sprint

3

0

50

100

150

200

250

300

350

400

At start 30/03/09 31/03/09 01/04/09 02/04/09 03/04/09 06/04/09 07/04/09 08/04/09 09/04/09 10/04/09

Burn down chart tracks delivery of committed scope

Team has absolute certainty that scope, priorities or deadline will not change during the course of the sprint

▪ Reinforces team focus on ensuring completion of committed scope

▪ Forces product owner to put concentrated effort into prioritization

Workin

g D

raft - La

st Mod

ified

10

/20

/20

10

7:5

7:2

4 A

MPrin

ted

5/1

8/2

01

0 8

:28

:55

AM

26

Agenda

Introduction to Agile

Early examples of agile projects

Workin

g D

raft - La

st Mod

ified

10

/20

/20

10

7:5

7:2

4 A

MPrin

ted

5/1

8/2

01

0 8

:28

:55

AM

The SCRUM master facilitates the delivery process

4

▪ Conduct daily SCRUMs▪ Facilitate removal of

impediments▪ Guide and coach team and

organization on skilful use of SCRUM

▪ Facilitate sprint planning and sprint retrospective

▪ Protect the team from outside disturbances

▪ Remind the team on their commitments

Scrum master responsibilities

Workin

g D

raft - La

st Mod

ified

10

/20

/20

10

7:5

7:2

4 A

MPrin

ted

5/1

8/2

01

0 8

:28

:55

AM

The daily SCRUM ensures team coordination and motivation

5

▪ Entire team is on same page▪ Everybody knows who is working on what▪ Agreed commitment and priorities are reinforced▪ Issues and blocks are escalated early

1. What have I done since the last SCRUM?2. What will I do until the next SCRUM?3. Do I have any blocks?4. Are there any new tasks that need to be

added?5. Have I learned anything that the team

should know about?

5 Questions

1. Standing meeting in front of the visual management board

2. Max 15 minutes3. No problem solving4. Prepared update

reports

Ground rules

Workin

g D

raft - La

st Mod

ified

10

/20

/20

10

7:5

7:2

4 A

MPrin

ted

5/1

8/2

01

0 8

:28

:55

AM

Each sprint should result in a potentially shippable product

6

▪ User acceptance testing can start earlier

▪ Team gets feedback sooner

▪ Issues are uncovered faster

▪ Documentation is written while design decisions are fresh in memory

▪ Attention to quality is constant throughout the development cycle

Documentation, testing, user verification/quality assurance is done within the sprint

Sprint planning estimates must also include hygiene and completion activities

Workin

g D

raft - La

st Mod

ified

10

/20

/20

10

7:5

7:2

4 A

MPrin

ted

5/1

8/2

01

0 8

:28

:55

AM

The sprint retrospective is a vehicle to ensure continuous improvement

7

Purpose

Ensure continuous and sustainable improvement in the way we work

Agenda

Time

Output

▪ Improvement suggestions for team lead retrospective

▪ Improvement tasks for sprint planning

Target outcome

Lead by

10:00 Poker assessment of last iteration’s output▪ Quality▪ Reliability▪ Velocity▪ Customer value

10:40 Identification of issues and root cause analysis▪ Internal team issues▪ External issues

11:20 Identification and prioritization of improvement ideas

Team lead

Team lead

Team lead

Topic

▪ Joint team commitment to improve

▪ Shared understanding for how to improve