introduction to lean and agile

©2008 Protegra Inc. All rights reserved.

Introduction to Lean and Agile Software

Development


Agenda

• What is the problem?

• What is Agile?

- Agile Manifesto

- Agile Principles

• Agile Methodologies

• Agile Software Development Standards

- Iteration Planning and Estimating

- Key Meetings

- Role of Project Management

- Role of Requirements Management

- Tools to assist Project and Requirements Management

- Role of Software Development

- Role of Testing

• Is it Working?

• Customize the Process

• FAQ/Q&A


What is the problem?

• Standish Group – CHAOS Study of IT project success

0

10

20

30

40

50

60

1994 1996 1998 2000 2002 2004

Succeeded

Challenged

Failed


What is the problem?

Cost overruns

+ Schedule overruns

+ Failed or Cancelled projects

+ Rigid processes

+ Change Management (that acts like Change Restriction)

+ Late Learning process

+ Building the wrong thing

= Lack of Trust in IT


Agile Manifesto

Reference: http://www.agilemanifesto.org

Snowbird ski resort - Feb 2001

We are uncovering better ways of developing

software by doing it and helping others do it.

Through this work we have come to value:

Individuals and interactions over processes and tools

Working software over comprehensive documentation

Customer collaboration over contract negotiation

Responding to change over following a plan

That is, while there is value in the items on the

right, we value the items on the left more.

http://www.agilemanifesto.org/


Agile Manifesto: 12 Principles

1. Satisfy the customer through early and continuous delivery

- Shorten the distance between requirements gathering and customer

feedback

2. Welcome changing requirements, even late in development

- Shorten the distance between conceiving and implementing an

important change

3. Deliver working software frequently

- Shorten the distance between system-as-desired and system-as-

built

4. Business people and developers work together daily

- Shorten the distance (time) between a question and its

answer

Reference: Agile in a Flash. Retrieved Sept 2009 from http://agileinaflash.blogspot.com/2009/08/12-principles-for-agile-software.html

http://agileinaflash.blogspot.com/2009/08/12-principles-for-agile-software.html











4. Build projects around motivated individuals

- Shorten the distance between intent and action, issue &

resolution

5. Convey information via face-to-face information

- Shorten the distance (space) between a question and its

answer













7. Working software is the primary measure of

progress

- Shorten the distance between thinking we‟re done and

knowing we‟re done

8. Maintain a constant pace indefinitely

- Shorten the distance between the amount of

unproductive time spent building software and a

satisfying work day

9. Give continuous attention to technical excellence

- Shorten the distance between implementation and ideal













10. Simplify: maximize the amount of work not done

- Shorten the distance between comprehension and completion

11. Teams self-organize

- Shorten the distance between need and action

12. Teams retrospect and tune their behaviour

- Shorten the distance between introspection and adaptation












Agile Methodologies

• Scrum

- founded by Ken Schwaber & Jeff Sutherland

• Extreme Programming (XP)

- founded by Kent Beck, Ward Cunningham, Ron Jeffries

• Crystal

- founded by Alistair Cockburn

• Lean Software Development

- founded by Tom & Mary Poppendieck

• Others…

- FDD

- DSDM












Top Three Methodologies

• Lets look at the characteristics of the top three Agile

Methodologies

- Scrum

- Extreme Programming

- Lean Software Development

11


Scrum

• Scrum is a process skeleton that includes a set of practices

and predefined roles.

- Pigs

- Chickens

• The main roles in Scrum are the ScrumMaster who

maintains the processes and works similarly to a project

manager, the Product Owner who represents the

stakeholders, and the Team which includes the developers.


Scrum

• During each sprint, a 15-30 day period, the team creates an

increment of usable software.

• The set of user stories that go into a sprint come from the product backlog, which is a prioritized set of high level

requirements of work to be done.

• Which backlog user stories go into the sprint is determined

during the sprint planning meeting.

• During a sprint, no one is able to change the sprint backlog,

which means that the requirements are frozen for that sprint.

After a sprint is completed, the team demonstrates the use

of the software.

13


Scrum

• Scrum enables the creation of self-organizing teams by

encouraging co-location of all team members, and verbal

communication across all team members and disciplines

that are involved in the project.

• A key principle of Scrum is its recognition that during a

project the customers can change their minds about what

they want and need (often called requirements churn), and

that unpredicted challenges cannot be easily addressed in a traditional predictive or planned manner.


Extreme Programming

• A discipline of software development that follows a specific

structure that is designed to simplify and expedite the

process of developing new software. Kent Beck developed

Extreme Programming to be used with small teams of

developers who need to develop software quickly in an environment of rapidly-changing requirements.

- Interesting note: Kent Beck‟s first book proposed a rigid form of XP.

This has since been changed and the result is a more Agile XP.


Extreme Programming

• Extreme Programming is based on 12 principles:

- The Planning Process -- The desired features of the software,

which are communicated by the customer, are combined with cost

estimates provided by the programmers to determine what the most

important factors of the software are.

- Small Releases -- The software is developed in small stages that

are updated frequently, typically every two weeks.

- Metaphor -- All members on an XP team use common names and

descriptions to guide development and communicate.

- Simple Design -- The software should include only the code that is

necessary to achieve the desired results.

- Testing -- Programmers design the tests first and then write the

software to fulfill the requirements of the test. The customer also

provides tests at each stage to ensure the desired results are

achieved.


Extreme Programming

- Refactoring -- XP programmers improve the design of the software

through every stage of development instead of waiting until the end of the

development and going back to correct flaws.

- Pair Programming -- All code is written by a pair of programmers working

at the same machine.

- Collective Ownership -- Every line of code belongs to every programmer

working on the project, so there are no issues of proprietary authorship to

slow the project down

- Continuous Integration -- The XP team integrates and builds the

software system multiple times per day to keep all the programmers at the

same stage of the development process at once.

- 40-Hour Week -- The XP team does not work excessive overtime to

ensure that the team remains well-rested, alert and effective.

- On-Site Customer -- The XP project is directed by the customer who is

available all the time to answer questions, set priorities and determine

requirements of the project.

- Coding Standard -- The programmers all write code in the same way.

This allows them to work in pairs and to share ownership of the code.


Extreme Programming Technical

Practices

• Extreme Programming is essentially Scrum with additional

Technical Practices

- keep it simple and never add functionality early

(YAGNI, Last Responsible Moment),

- choose a system metaphor (Domain-Driven

Design),

- use CRC cards,

- create a spike solution (proof of concept),

- create acceptance tests,

- establish coding standards,

18

http://www.extremeprogramming.org/rules/simple.html

http://www.extremeprogramming.org/rules/early.html

http://www.extremeprogramming.org/rules/metaphor.html

http://www.extremeprogramming.org/rules/crccards.html

http://www.extremeprogramming.org/rules/spike.html

http://www.extremeprogramming.org/rules/functionaltests.html

http://www.extremeprogramming.org/rules/standards.html


Extreme Programming Technical

Practices

- code the unit test first and ensure that all code

has and passes all unit tests (Test-Driven

Development),

- program in pairs,

- integrate sequentially and often,

- own the code collectively,

- measure and optimize last, and

- create tests to detect bugs.

19

http://www.extremeprogramming.org/rules/testfirst.html

http://www.extremeprogramming.org/rules/unittests.html

http://www.extremeprogramming.org/rules/pair.html

http://www.extremeprogramming.org/rules/sequential.html

http://www.extremeprogramming.org/rules/integrateoften.html

http://www.extremeprogramming.org/rules/collective.html

http://www.extremeprogramming.org/rules/optimize.html

http://www.extremeprogramming.org/rules/bugs.html


Lean Software Development

• Lean Software Development is the application of lean

principles to the craft of software development. So what is

Lean? According to the National Institute of Standards and

Technology Manufacturing Extensions Partnership’s Lean

Network, Lean is:

- “A systematic approach to identifying and eliminating waste through

continuous improvement, flowing the product at the pull of the

customer in pursuit of perfection.” [1]

- “Lean Software Development reduces defects and cycle times while

delivering a steady stream of incremental business value.” [2]



• Lean Software development is a style of software development that emphasizes customer satisfaction through continuous delivery of functional software. In contrast to traditional software development methods, lean developers liaise continuously with business clients.

• Their objective is to deliver working software as frequently as every two weeks during a project, and welcome changes to the requirements in response to evolving business needs.



• The most crucial aspect of Lean LifeCycle is the execution of the project in iterations and quick feedback loops possible because of these iterations. It is essential to note that these iterations to not just apply to construction, they also apply to the following tasks:

- Project Management and Planning

- Analysis

- Technical Design

- Testing

- Deployment

22


Lean Principles

1. Eliminate Waste

The three biggest wastes in software development are:

a) Extra Features

b) Churn

c) Crossing Boundaries

Reference: Lean Software Development Principles. Retrieved Sept 2009 from http://www.poppendieck.com

Client received:

Client needed:



Lean Principles

2. Create Knowledge

Planning is useful. Learning is essential

a) Use the Scientific Method

b) Standards exist to be Challenged and Improved

c) Predictable performance is Driven by Feedback




Lean Principles

3. Build Quality In

If you routinely find defects in your verification process,

your process is defective.

a) Mistake-Proof Code with Test-Driven Development

b) Stop Building Legacy Code

c) The Big Bang is Obsolete




Lean Principles

4. Defer Commitment

Abolish the idea that it is a good idea to start

development with a complete specification.

a) Break Dependencies

b) Maintain Options

c) Schedule Irreversible Decisions at the Last Responsible

Moment




Lean Principles

5. Deliver Fast

Lists and queues are buffers between

organizations that simply slow things

down.

a) Rapid Delivery, High Quality, and Low Cost

are Fully Compatible

b) Queuing Theory applies to Development, not

just Servers

c) Limit Work to Capacity




Lean Principles

6. Respect People

Engaged, thinking people provide the most sustainable

competitive advantage.

a) Teams Thrive on Pride, Commitment, Trust, Applause

b) Provide Effective Leadership

c) Respect Partners




Lean Principles

7. Improve the System

Brilliant products emerge from a unique combination of

opportunity and technology.

a) Focus on the Entire Value Stream

b) Deliver a Complete Product

c) Measure UP




Agile Software Development Standards

• The following roles and activities are done in a standard way

in all Agile Methodologies:


- Key Meetings

- Role of Project Management

- Role of Requirements Management

- Tools to assist Project and Requirements Management

- Role of Software Development

- Role of Testing

30


Iteration Planning and Estimating

• Iteration planning is ‘the’ key planning initiative.

Iterations need to be planned in conjunction with

the client to accomplish the following: • Deliver functionality to define the pace of the project

• Deliver functionality to deliver value to the client

• Deliver functionality to reduce and minimize risk for the entire project

• Lessons learned from one iteration must feed into subsequent iterations so that we don‟t execute the project in iterations with similar results, but

that we execute the project in iterations with better results.


Initial Planning and Estimating

• Initial Planning and estimating needs to be done

- Need detailed deliverable or task levels estimates

- Resource planning

- Milestone based

- Sometimes even a WBS structure based on project reporting

requirements

• This should be required for Agile or Lean projects.

• Agile or Lean should not be an open cheque where

estimates and milestones are not planned and approved up

front

- Some proponents promote that agile estimates should only be given

after team „Velocity‟ is determined

32


What is this ‘Team Velocity’ I hear about?

• Team Velocity is the sustainable pace that the team can do

work

- This will change based upon the project and team composition

• Velocity should not replace detailed estimating and Project

Planning

- Detailed Estimating and Project Planning are to plan the project

- Velocity measurement and refinement are used to manage an

executing project

- BOTH are required for a successful project.

33


Estimation Levels

• Estimating should be done at multiple levels to gain

confidence and increase estimate reliability

• There are 5 levels of estimating

- Task Level

- Deliverable Level

- Schedule Level

- Monte Carlo Simulation

- Planning Poker Sessions (Wide Band Delphi)

34


What is this ‘Planning Poker ’ I hear about?

• Planning Poker is a process whereby the entire team

estimate objects using card decks.

• Planning Poker sessions should be held for each iteration.

• A baseline is first established estimating an item that there

is high estimating confidence in. (i.e. a simple CRUD screen)

- Subsequent items are then estimated in increments of that first

baseline. (i.e. This screen would take the effort of 3 base screens to

develop)

- Everyone chooses a card and then flips it over the same time.

- The team then discusses the difference in opinions why people

estimated differently. The team then has to come to consensus on

the estimate to be used.

- While time consuming, it addresses one of the main issues for

project success. Proper estimating.

• Planning Poker enables the sharing of estimating expertise and knowledge

35


Key Meetings

• The following key meetings are required:

- Project Kick off

- Iteration kick off

- Daily scrums or huddles

• What did you do yesterday

• What are you doing today

• What issues do you have

• 2 minute limit per person

- Iteration Retrospectives


36


Role of Project Management

• Iteration Planning

• Issue Management

• Risk Management

• Change Management is required even more

- Different focus. The focus is how I can get the client the MOST

functionality by following the Change Management process. May

involve:

• Trading requirements

• Eliminating Requirements

• Change Requests for new budget or schedule if required

• Sometimes the perception exists that Agile or Lean projects

do not require the same level of Change Management. This

is an incorrect perception.

- Sometimes Iterative projects actually have less Change

Management than water fall projects.

- Projects that are doing less Change Management than required are

not „being lean‟, they are being misguided.

37


Role of Project Management

• Increased communication and visibility

- Visual Project Management

- Visual Project Dashboards

- Visual Status Reporting

38


Requirements Management

• Business Analysts are the liaison to the business needs and

desires.

• Requirements Management and traceability is even more important for agile projects as there is the possibility of

increased change

• This makes the role of business and systems analysis

critical to the success of the projects

• The focus is on the ‘right’ amount of documentation.

• Sometimes the perception exists that Agile or Lean projects

do not document requirements. This is an incorrect perception.

- Projects should define what the appropriate level of documentation

is on a case by case basis. (within required guidelines)

- Projects that are doing less documentation than required are not

„being lean‟, they are being misguided.

39


Requirements Management

• Requirements must be Test Case Driven.

- Instead of descriptions as to what the software needs to do, the

requirements should first list what test cases will ensure the software

is operating correctly.

- This should be the first section in every specification or use case

- This will then feed into automated test cases and be a living

document that will define the software

- There is less ambiguity in test cases than textual descriptions

40


Tools to assist Project and Requirements Management

• Many free or affordable tools are available to assist Agile or

Lean Projects.

- www.rallydev.com

- www.targetprocess.com

- www.versionone.com

- www.thoughtworks.com/mingle

• This is due to the fact that MS Project is an ok project

planning tool, but not a great tool to manage a project while it is executing. (especially Agile or Lean Projects)

41

http://www.rallydev.com/

http://www.targetprocess.com/

http://www.versionone.com/

http://www.thoughtworks.com/mingle


Role of Software Development

• The following are important aspects of software

development in an Agile or Lean Project:

- Team-Based Design

- Resources are multi-disciplined

- Pull system for task assignment

- Iterative Development

• Each development should be able to be implemented in Production. If

not, it is not Iterative Development.

- Automated Unit Testing

• NUnit/Junit

- Continuous Builds/Continuous Build Servers

42


Role of Testing

• Testing is a critical component in Agile or Lean project. (like all

Waterfall Projects)

• The Test Phase should be planned very diligently and in a very detailed manner.

- Resist the urge to have the iterations tested in an unstructured way. This

only delays the discovery of defects.

- A Test Charter, Test Strategy, and Test Plans are all required.

- Automated System Integration Tests and User Acceptance Tests are

mandatory

• Continuous Integration is mandatory. This is distinct from

Continuous builds.

- Continuous builds are the daily builds and execution of unit test cases to

ensure ongoing development has not introduced new development

issues

- Continuous integrations are the daily builds and execution of regression

test cases to ensure no major application functionality issues exist.

43


Is it working?

Reference: Dr. Dobb‟s Journal 2008 Agile Adoption Survey. Retrieved from http://www.ambysoft.com/surveys/agileFebruary2008.html

0% 20% 40% 60% 80% 100%

Productivity

Quality

Business StakeholderSatisfaction

Cost of SystemDevelopment

Higher

No Change

Lower

Dr. Dobb‟s Journal – Agile Adoption Survey in 2008

• Effectiveness of agile software development compared

with traditional approaches:

http://www.ambysoft.com/surveys/agileFebruary2008.html


Customize the process

• The most important advice I could provide is to not take any

of these methodologies as correct.

• Instead I would encourage you to read about the different methodologies and evaluate which components make sense

for your company and for the particular project in question.

45


FAQ

Q. Silver Bullet?

A. No

Q. Cowboy Coding?

A. No

Q. How much documentation?

A. Only what adds value

Q. Isn’t this just a blank cheque with no controls?

A. No – disciplined approach

Q. How do you deal with constant change to the code?

A. Technical Excellence. TDD, SOLID

Q. How do you estimate up front?

A. Same as waterfall, but at a higher level


FAQ

Q. Is there a place for analysts?

A. Yes

Q. Isn’t Agile unpredictable?

A. Final outcome is not predictable, but value is

Q. Is this just mini-waterfall in phases?

A. No. The team performs all tasks in one day in parallel, not in sequence

Q. What project sizes is this appropriate for?

A. Survey says…. all sizes

Q. Is this for Greenfield projects only?

A. No.

Q. Can I use Agile practices on waterfall projects?

A. Yes. Example: Once requirements & design are complete, execute

the development in iterations


FAQ

Q. Where to start?

A.

- Pick/Customize a methodology

- Learn it • Read, listen, research, ask questions

• Talk to someone who has done it

- Try it • It won‟t likely be smooth on your first try

• Don‟t ignore practices without understanding the why

- Learn a few more methodologies

- Incorporate your learning into a hybrid that

works for your company, your teams, your

projects


Further Reading • Books

- Lean Software Development. Tom and Mary Poppendieck (2003)

- User Stories Applied. Mike Cohn (2004)

- The Art of Agile Development. James Shore and Shane Warden (2008)

- The Art of Lean Software Development. Curt Hibbs, Steve Jewett, Mike Sullivan (2009)

• Sites - www.poppendieck.com

- www.leanprimer.com

- www.agilemanifesto.org

• Podcast Sites - http://www.hanselminutes.com/

- http://blog.agiletoolkit.com/

• Newsgroups

- [email protected]

• Other links to articles, blogs, videos - www.martinfowler.com/articles/newMethodology.html

- http://agileinaflash.blogspot.com/2009/08/12-principles-for-agile-software.html

- www.infoq.com/Agile2009 (Videos from Agile 2009)

- http://www.netobjectives.com/files/BusinessCaseForAgility.pdf (requires account registration)

- http://groups.google.com/group/agile-developer-skills/web/draft-summary-of-chicago-meeting?hl=en&pli=1

- http://www.agilemanifesto.org/history.html - History of the Manifesto

- http://www.devx.com/architect/Article/32836/0/page/4 - comparison of seven popular agile methodologies

http://www.poppendieck.com/

http://www.leanprimer.com/

http://www.agilemanifesto.org/

http://www.hanselminutes.com/

http://blog.agiletoolkit.com/

http://www.martinfowler.com/articles/newMethodology.html










http://www.infoq.com/Agile2009

http://www.netobjectives.com/files/BusinessCaseForAgility.pdf

http://groups.google.com/group/agile-developer-skills/web/draft-summary-of-chicago-meeting?hl=en&pli=1













http://www.agilemanifesto.org/history.html

http://www.devx.com/architect/Article/32836/0/page/4


Q&A


Thank You!

Name: Terry Bunio

Role: Principal Consultant

E-mail: [email protected]

Main: 204-956-2727

www.protegra.com

51

introduction to lean and agile

Technology

agilesof protegra

agile methodologies

distance time

software development

distance space

it2009 protegra

toolsworking software

scrum scrum