architecture in an agile organization managing our software as a valuable asset while delivering...

Architecture in an Agile OrganizationManaging our Software as a Valuable Asset whileDelivering Incrementally

Chris SterlingPrincipal Consultant, Certified Scrum Trainer, and Agile Coach

SolutionsIQ

Mickey PhoenixSenior Software Development Engineer

SolutionsIQ

Architecture in an Agile Organization

• As companies begin to embrace Agile methods, such as Scrum and Extreme Programming (XP), questions about architecture begin to emerge.– How is architecture implemented in an Agile organization? – What is the software architect's role with Agile development? – How should architecture be discussed with business stakeholders?

• Too often, Agile software development methods are tagged as "architecturally weak" or "disconnected from the realities of delivering large systems." However, leading software architects are pushing through the noise to rapidly evolve the Agile architecture discipline.

• This session will focus on the experiences of two experts and the approaches they have taken to better align businesses with architecture goals.

• Presented by:– Chris Sterling, Principal Consultant and Certified Scrum Trainer at SolutionsIQ– Mickey Phoenix, Senior Software Development Engineer at SolutionsIQ

Topics to Discuss

•Self-Organizing Project Teams

•Practices versus Guidelines

•Managing Software Debt

•Coherent Architecture Strategy

•Where can we start?

Questions for You...

•What does “architecture” mean to you?

•What issues are you having currently with architecture?

•Why do we worry about architecture?

Self-organizing Project Teams

* “The best architectures,

requirements, and designs emerge from

self-organizing teams”

* The Agile Manifesto - http://www.agilemanifesto.org

* Self-organizing Project Teams

•Autonomy: top management provides guidance, money, and morale support

•Self-transcendence: team seems on never-ending quest for “the limit” within guidelines set forth by top management

•Cross-fertilization: team consisting of varying functional specializations, thought processes, and behavior patterns

* From Harvard Business Review Jan/Feb 1986“The New New Product Development Game” by Hirotaka Takeuchi and Ikujira Nonaka

Self-organizing Doesn’t Happen Overnight

Practices versus Guidelines

•Practices: techniques, methodologies, procedures, and processes that are used to get the job done

•Guidelines: aim to streamline particular processes according although, by definition, following a guideline is never mandatory. Guidelines are an essential part of the larger process of governance to make delivery more predictable, and presumably of higher quality

Practices versus Guidelines

Example Practices

•Code Review

•Threat Modeling

• Filling out Document Templates

•Test-Driven Development

•Pair Programming

•Continuous Integration

Practices are not bad but may impede a team’s self-organization.

Example Guidelines

•75% Code Coverage

•Audit Trail for all System User Interactions

•Have all Source Control Check-ins Reviewed

•Each Team Member Checks in an Artifact every Day

•Applicable Features Pass Security Audit Policies

Managing Software Debt

•Software gets more difficult to add features to as it ages

•Business expectations do not lessen as software ages

•Software must remain maintainable and changeable to meet needs of business over time

• Lack of emphasis on quality attributes to keep software maintainable

The Problem...

๏ Like-to-like migrations๏ Limited experts capable to work on a

system๏ Long release stabilization periods๏ Increasing duration of costly regression

test phases๏ Higher costs for people to work on

legacy software

Software Asset Liabilities

Like-to-Like Migrations

• “It will be easy since we worked on the original version” - although we understand the domain we will be fighting with new features, technology, tools, and processes

• “We don’t have any other options” - Refactoring and test automation are potential alternatives to like-to-like migrations.

Limited Expertise

Risk increase as expertise becomes more limited for an aging software system.

Long Stabilization Periods

Costly Regression Test Phases

Higher Costs for People

• Legacy software expenses increase with time

• High demand for limited reserve of experienced programmers (ie. COBOL)

• Specialized tools have a limited following with higher bill rates

Software Debt Creeps In

Shows a relatively new system with little debt accrued.

An aging software system slowly incurs significant debt in multiple functional areas.

An older system has accrued significant debt in all functional areas and components.

Types of Software Debt

•Technical Debt

•Quality Debt

•Configuration Management Debt

•Design Debt

•Platform Experience Debt

Technical Debt

• Issues in software implementation that will impede future development if left unresolved

• Technical Debt includes those internal things that you choose not to do now, but which will impede future development if left undone. This includes deferred refactoring.

• Technical Debt doesn’t include deferred functionality, except possibly in edge cases where delivered functionality is “good enough” for the customer, but doesn’t satisfy some standard (e.g., a UI element that isn’t fully compliant with some UI standard).

* Ward Cunningham - “Technical Debt” - http://c2.com/cgi/wiki?TechnicalDebt

* Technical Debt

Quality Debt

• A lack of quality, either technical or functional, will minimize value per feature added increasingly over time

Accrual of Software Debt

Break/Fix Only Prolongs It

A Fit Case Study

• Cost reduction using Fit for test automation and

• data conversion

•Testing was taking 75 person hours during 2 full test runs consisting of:

Comprehensive manual regression testing

Data conversion and validation

•Cost for testing was $17,000 each iteration

Manual Regression Testing

•After 8 iterations team had introduced healthy amount of Fit fixtures and automated tests

•Reduced 70+ hour test runtime down to 6 hours which now included:

Fit automated regression testing

Data conversion and validation automated with Fit fixtures

•Reduced cost of testing each iteration from $17,000 to $7,000

Introducing Fit into Testing Process

Bug or Enhancement?

In Bug Tracking System

In Product Backlog

• Came from important customer.

• Must be fixed and deployed within 2 weeks.

The Situation: A Bug

• Must be in next release which is 1 month away.

• Involves multiple user stories currently estimated to be delivered in 3 weeks based on velocity of Team.

The Situation: New Features

The Situation: Should we work on bugs or enhancements first?• Fix bug and deploy within 2 weeks

• or• Finish 3 weeks of enhancements to deploy in

next release

• The Catch: The release is due to go out in 1 month and doing both will not fit into this

timeframe.

•Put all work into Product Backlog

•Make tradeoff decisions based on

reality of the situation

Maintain One List of Work

Definition of Done

• Define what is involved in delivering a potentially shippable product increment each iteration

Definition of Done

Capture Resulting Artifacts

•Be careful capturing delivered artifacts in Definition of Done• Should capture resulting artifacts

• Minimize capture of transitory documentation

• Minimize capture of practices

• Different for each Team but may include enterprise guidelines when appropriate

Configuration Management Debt• Debt in this area can be

unpredictable, prone to error, and cause poor decision-making on quality attributes of a system

Traditional Source Control Management

Main BranchMain BranchDebtDebt

Death March

{Debt accrues quickly within stabilization periods

Version 1Branch

Integrate forVersion 2

CodeComplete

Main BranchMain Branch

Version 1 Version 2

{Not Easy! Must have proper

infrastructure to do this.

Flexible Source Control Management

Continuous Integration

Scaling to Multiple Integrations

Design Debt

• Design decays when not attended to so put effort into maintaining your software by designing all the time

•Refactoring: a disciplined technique for restructuring an existing body of code, altering its internal structure without changing its external behavior.*

•Merciless: having or showing no [mercy - showing great kindness toward the distressed]

•Relieve your distressed code through kindness and disciplined restructuring

* From http://www.refactoring.com/

Merciless Refactoring

•Refactoring cannot be done effectively without automated tests surrounding code

•Start by creating automated test which fails

• If difficult to create at unit level look at automated acceptance tests from functional perspective

•Over time look for ways to create automated unit tests

Automate Testing to Support Refactoring

Design Debt• When the cost of adding new

functionality to an existing system is more than writing it from scratch

* Architecture Risk Themes

• Observed that twice as many risk themes are risks of “omission” as are risks of “commission”. That is, risk themes identify decisions or investigations that were never made rather than those that were made and could lead to undesirable consequences.

• No discernible relationship between articulated business and mission goals of a system and risk themes from an ATAM evaluation of that system

• No discernible relationship between domain of system being evaluated and risk themes associated with development of that system

* From “Risk Themes Discovered Through Architecture Evaluations” by Len Bass, Robert Nord, William Wood, and David Zubrow

Proper Architecture Design Provides Value• Software is a business asset and our job is

to produce the greatest Return on Investment (ROI) by delivering high value quickly

* Abuse User Stories

Implement Securityfor User

Information

As a Malicious Hacker I want to steal credit card information so

that I can make fraudulent charges

* From “User Stories Applied” presented by Mike Cohn Agile 2006

Platform Experience Debt• How many of you are writing

Legacy Code?

How to Combat Platform Experience Debt

• Ignore it (I do not suggest this!)

•Make interface to underlying system

•Transfer learning of platform to more people

•Rewrite system on more current platform

•Slice functionality to more current platform over time

Architecture Team Patterns

•Virtual Architect Pattern

• Integration Team Pattern

•Component Shepherd Pattern

•Team Architect Pattern

EnterprisePlanning

Virtual Architect Pattern

•Pros

• Share architecture ideas and needs across teams

• Based on verbal communication

•Cons

• Usually singles out special Team Member role

• Could lead to top-down architecture decisions

• IT may gain extensive influence and begin to run Product Backlog prioritization for architecture needs

Virtual Architect Pattern

Feature Development

Integrate Features

All features are All features are implemented and implemented and integrated every integrated every

iterationiteration

Integration Team Pattern

•Pros

• Reduces complexity on Feature Teams

• Forces delivery from Integration Team instead of interface and deployment designs

•Cons

• Perpetuates specialized roles

• Don’t always work on highest value Product Backlog items

Integration Team Pattern

Component Shepherd Pattern

•Pros

• Share more knowledge within organization to minimize platform experience debt

• Work on highest value Product Backlog items

•Cons

• Not always optimal as using individual knowledge

• Difficult to learn multiple systems across Teams

Component Shepherd Pattern

Team Architect Pattern

•Pros

• Team owns architecture decisions

• Decisions are made close to implementation concerns

•Cons

• May not have appropriate experience on Team

• Team could get “stuck” on architecture decisions

Team Architect Pattern

Coherent Architecture Strategy

Product Domain Model

•Domain-Driven Design

•XP Metaphor

•System of Names

•Glossary of Terms

COTS (Commercial Off-The-Shelf)

•COTS does 80% of what we need usually

•We conduct custom configurations to enable rest of what we need

•Automate functional tests for all customizations

•Automated functional tests can be executed against future upgrades to see what “broke”

Automated Tests

New New FeaturFeatur

SystemSystem

Design ChangesDesign Changes

AutomatAutomateded

RegressiRegressionon

Test RunTest Run

Where do we start?• Knowing areas to improve on is

one thing. How can we make progress from where we are today?

• Focus on guidelines rather than practices to influence quality of team deliveries

• Maintain one list of work

• Emphasize quality

• Evolve tools and infrastructure continually

• Improve system design always

• Check in project artifacts more often (Continuous Integration)

• Automate builds from compile to deploy

• Share knowledge across organization

• Hire the right people to work on your software!

Some Suggestions

Principles of Agile Software Quality

•The system always runs

•No code is written without a failing test

•Zero post-iteration bugs

* From “Flawless Iterations” presented by Alex Pukinskis at Agile 2005

Thank you• Questions and Answers

Chris Sterling

• Principal Consultant, Certified Scrum Trainer and Agile Coach at SolutionsIQ

• Consults on enterprise architecture and Agile software development practices across a spectrum of industries

• Founder of the International Association of Software Architects (IASA) Puget Sound chapter

• Email: CSterling@SolutionsIQ.com• Blog: http://chrissterling.gettingagile.com

architecture in an agile organization managing our software as a valuable asset while delivering...

software ages software

overnight slide

managing software debt

software maintainable

ikujira nonaka slide

agile development

agile architecture discipline

aging software system

Documents

incrementally improving graph wavenet performance on...

how we incrementally improved search

sterling mountain smugglers’ notch hiking€™ notch, vt...

a clustering-based framework for incrementally repairing...

incrementally maintaining classification using an rdbms -...

greedy. build up a solution incrementally, myopically

sterling enterprises, pune, sterling silver jewelry

incrementally updateable honey password vaults

gem varna · designer, trendy ,everyday wear gemstone...

ajax-based report pages as incrementally rendered...

sterling background checks · sterling background checks...

safety controller synthesis for incrementally stable...

northwestern university an incrementally non-linear model

incrementally maintaining run-length encoded...

c:\users\obitonyeaso\documents\sterling bank folder\sterling

i’ve seen “enough”: incrementally improving...

incrementally maintaining run-length encoded attributes in...

incrementally launched prestressed concrete girders ·...

sovereign: an autonomous neural system for incrementally...

sterling secure proxy - sterling b2b...