using continuous integration for quality improvement with agile

Copyright 2002, Medical Present Value, Inc. All rights reserved.Copyright © 2010 Texas Education Agency. All rights reserved. TEA confidential and proprietary.

Using Continuous Integration for Quality Improvement with

Agile

Mark Gentzel, Project Manager, PMOManjusha Gokhale, Manager, IV&V Services

Texas Education Agency

July 14, 2010

Copyright 2002, Medical Present Value, Inc. All rights reserved.Copyright © 2010 Texas Education Agency All rights reserved. TEA confidential and proprietary.

Agenda

• Problem Statement• What Is Continuous Integration (CI)?• Comparison Between Waterfall and Agile• Using CI within an Agile Environment• Specific Agile Technique• Waterfall vs. Agile Project Comparison• Quality Acceptance Criteria• TEA CI Process• Benefits for Software Development• Benefits for Independent Testing• Best Practices and Lessons Learned• Conclusion

2


Problem Statement

• Adoption of Agile development methodology requires new processes: Continuous Integration Continuous quality control Tighter integration between development and testing

teams

3


What Is Continuous Integration?

• Integrate development work frequently• Integrate and test deliverables frequently rather than waiting until the

end of the development cycle• Integrations verified by an automated build to detect integration

errors as quickly as possible• Significantly reduced integration problems• Cohesive software developed more rapidly

4


Comparison Between Waterfall and Agile

• Classic Waterfall Long requirements phase before design and development Long development phase Testing done post-development with defects returned to team Rework difficult and complex because of advanced stage of code

• Modified Agile Abbreviated requirements phase to gather 40% of the requirements Abbreviated design phase to develop high-level architecture Short three-week sprints with code delivery to Independent Verification and

Validation (IV&V) post-sprint User acceptance with production deployment Rework for previous sprint included in current sprint Rapid rework because of simplified code at acceptance point Quicker testing because of functional isolation and established code base for

each test delivery

5


Using CI within an Agile Environment

• Solution—Extending and updating our Continuous Integration processes Automated processes where code is subjected to metrics to

determine its validity and stability Metrics vary based on server environment Example of metrics:

• Successful compilation• Successful automated unit testing• Successful regression testing• Low cyclomatic complexity

6


Specific Agile Technique

• Three-week sprints• One major feature or bundle of minor features per sprint• Four days of IV&V testing per sprint• Requirements gathering completed a few sprints ahead• Sprint planning meetings during the week between

sprints• Sprint retrospecitive meetings the week after sprints

7


Waterfall vs. Agile Project Comparison

8


Quality Acceptance Criteria

• Build compiles successfully• Unit tests run successfully and build passes unit tests• Build passes code review• Build passes all automated functional regression tests• Build can be deployed successfully!

9


TEA CI Process

10

Checks for new code

Retrieves new code

Deploys

Polling

CI ProcessorBuilds Code

Source ControlProcessor

Code Repository

Unit TestSuccessful

RegressionTest

Successful

ComplexityTest

Successful

AutomatedTestSuites

ApplicationEnvironment


Benefits for Software Development

• Ensures that compilation errors are immediately discovered and addressed

• Provides unit test coverage, ensuring comprehensiveness• Provides metrics to ensure that code is not excessively

complex• Reduces build and deployment times by ensuring stable code

bases• Ensures that unexpected code variances are detected and

addressed• Reduces repetitive builds and deployments

11


Benefits for Independent Testing

• Continuous code integration• Quality control applied continuously to small pieces of

code, providing immediate feedback• Support of test-driven development• Rapid delivery of stable and cohesive software to

customers• Reduced risk of finding critical bugs later in lifecycle• Reduced regression test costs through automated testing• Multiple stable labels, for flexibility in test deployment

12


Benefits for Testing

13

2007 2008 2009 20100

5

10

15

20

25

30

35

Number of Fatal and High-Severity Bugs before and after Use of Automated Functional Regression Suite

Year

Num

ber o

f SC

r's

Aut

omat

ed S

crip

ts In

trodu

ctio

n


Benefits for Testing

14

2007 2008 2009 20100

1

2

3

4

5

6

7

Number of Days and Resources to Perform Regression before and after Use of Automated Functional Regression Suite

ResourcesDays

Year

Num

ber

Auto

mat

ed S

crip

ts I

ntro

ducti

on


Best Practices and Lessons Learned

• Best Practices Maintain a single source repository Do frequent check-ins Automate the build and deployment Test in a clone of the production environment

• Lessons Learned Make the build self-testing Decide when to integrate to control risk Test at the right level of granularity Refactor regularly

15


Conclusion

• Continuous Integration for Continuous Quality Control• Close collaboration between development and IV&V

groups• Reduced product delivery time• Increase in team productivity leading to reduced costs

16


Thank You!

[email protected]@tea.state.tx.us

17

using continuous integration for quality improvement with agile

Documents