Quality is Everything

Geoffrey BessinMarket Manager, Software Quality

IBM Rational

2

3

What is “quality”?

FURPS+Functionality

UsabilityReliability

PerformanceSupportability

+ others

Acceptable GOAL RESULT

4

Software “engineering”

Software bugs cost the US economy an estimated $59.5

billion in 2002.1

On average, professional coders make 100 to 150 errors in every thousand lines of code they write.2

In 2002, canceled projects cost firms $67 billion; overruns on

other projects racked up another $21 billion.3

1 NIST2 Watts Humphrey, Carnegie Mellon3 Standish Group

5

Room for improvement

DEFECT COUNT

RESULTING QUALITY

Process Problems

•Waterfall-like development

•Poor asset management and measurement

Product Problems

•Lack of proactive validation techniques

•Data disconnect between IT Operations and Development

People Problems

•No shared responsibility for quality

•Over-reliance on test team

70% of organizati

ons are only CMM

Level 1

50% of total

project cost goes toward

software maintenan

ce25% of

projects are

cancelled; another 25% are

late

6

It’s the other person’s fault!

“….we estimate that 40 percent of unplanned application downtime is caused by application failures; 40 percent by application errors; and only 20 percent by hardware (server and network), environmental factors (heating, cooling and power failures) and disasters”

“70%-80% of project failures are tied to poor analysis”

7

If you’re thinking, “Not my problem” – then you’ve got a problem.

8

Solution Selling 101

SQUEAK!!

9

Solution Selling – Graduate Course

10

Is this problem related to hardware? Is an obvious application failure logged? Is this problem related to memory or performance issues of the software? Which additional data/traces should be captured to accelerate problem determination? What is the severity of the problem?

Is this problem related to hardware? Is an obvious application failure logged? Is this problem related to memory or performance issues of the software? Which additional data/traces should be captured to accelerate problem determination? What is the severity of the problem?

Operations

Is this problem a defect? What scenario reproduces the defect? What is the customer priority? Is this defect related to a specific configuration?

Is this problem a request for enhancement? What is the requirement or use case? What is the customer priority?

Is this problem a defect? What scenario reproduces the defect? What is the customer priority? Is this defect related to a specific configuration?

Is this problem a request for enhancement? What is the requirement or use case? What is the customer priority?

Tech Support

Is it possible to reproduce the defect? Which version does it apply to? Is there a need to configure an environment for defect reproduction? Which use case does this defect relate to?

Is it possible to reproduce the defect? Which version does it apply to? Is there a need to configure an environment for defect reproduction? Which use case does this defect relate to?

Tester

Deployment TransitionConstructionElaborationInceptionDeployment TransitionConstructionElaborationInception

PackagedApplication

RDBMS

WebService

Jservice

JSPs, struts,Servlets

Enterprise ACICS

Maintenance of Existing Systems and Software

App Servers

EJBs

App Servers

WebServer

EJBs

EdgeServer

Enterprise B

Traceability

Test Case

Test Case

Execution Trace

Test Case

Defect

Defect

Code

Code

11

Is it possible to reproduce the defect? Which version does it apply to? Is there a need to configure an environment for defect reproduction? Which use case does this defect relate to?

Is it possible to reproduce the defect? Which version does it apply to? Is there a need to configure an environment for defect reproduction? Which use case does this defect relate to?

Tester

What is the business priority of the defect? What is the impact on existing customers? How quickly is a fix needed? What is the relative importance of this defect compared to high-priority enhancements?

What is the business priority of the defect? What is the impact on existing customers? How quickly is a fix needed? What is the relative importance of this defect compared to high-priority enhancements?

Analyst

What is the impact of this defect? Which components realize the defective scenario? Who should be working on the defect? How much time should be allocated to fix it? Is there any foreseeable architectural impact?

What is the impact of this defect? Which components realize the defective scenario? Who should be working on the defect? How much time should be allocated to fix it? Is there any foreseeable architectural impact?

Architect

InceptionDeployment TransitionConstructionElaboration

PackagedApplication

RDBMS

WebService

Jservice

JSPs, struts,Servlets

Enterprise ACICS

Maintenance of Existing Systems and Software

App Servers

EJBs

App Servers

WebServer

EJBs

EdgeServer

Enterprise B

Traceability

Test Case

Test Case

FEATreq

FEATreqArchitecture

Code

Code

Execution Trace

Test Case

Defect

Defect

Planning

12

What is the impact of this defect? Which components realize the defective scenario? Who should be working on the defect? How much time should be allocated to fix it? Is there any foreseeable architectural impact?

What is the impact of this defect? Which components realize the defective scenario? Who should be working on the defect? How much time should be allocated to fix it? Is there any foreseeable architectural impact?

Architect

Which components are suspect? Are there any clues in the captured traces? Is this defect reproducible in the development or test environment? Is there a need for additional traces?

What is the foreseeable impact and risk? What other components are impacted?How much time is needed to fix the issue?

Which components are suspect? Are there any clues in the captured traces? Is this defect reproducible in the development or test environment? Is there a need for additional traces?

What is the foreseeable impact and risk? What other components are impacted?How much time is needed to fix the issue?

Developer

Is there a significant architectural impact on the system? How many components are impacted? Is there a need to refactor the architecture?

Is there a significant architectural impact on the system? How many components are impacted? Is there a need to refactor the architecture?

Architect

How much regression testing is required? Which use cases are impacted? Which components are impacted? Which tests need to be re-executed?

What is the impact on existing plan? How many resources need to be engaged? Is it possible to stay on original schedule?

How much regression testing is required? Which use cases are impacted? Which components are impacted? Which tests need to be re-executed?

What is the impact on existing plan? How many resources need to be engaged? Is it possible to stay on original schedule?

Tester

What is the business case to fix the defect? How many customers might be impacted? What is the $$ loss associated with the defect? Which features/use cases might be de-scoped due to time spent in fixing the issue? What is the potential revenue impact of these features?

What is the business case to fix the defect? How many customers might be impacted? What is the $$ loss associated with the defect? Which features/use cases might be de-scoped due to time spent in fixing the issue? What is the potential revenue impact of these features?

Analyst

ElaborationTriageDeployment TransitionConstruction

PackagedApplication

RDBMS

WebService

Jservice

JSPs, struts,Servlets

Enterprise ACICS

Maintenance of Existing Systems and Software

App Servers

EJBs

App Servers

WebServer

EJBs

EdgeServer

Enterprise B

Traceability

Architecture

Test Case

Test Case

FEATreq

FEATreq

Code

Code

Execution Trace

Test Case

Defect

Defect

Planning

Driver

13

What is the business case to fix the defect? How many customers might be impacted? What is the $$ loss associated with the defect? Which features/use cases might be de-scoped due to time spent in fixing the issue? What is the potential revenue impact of these features?

What is the business case to fix the defect? How many customers might be impacted? What is the $$ loss associated with the defect? Which features/use cases might be de-scoped due to time spent in fixing the issue? What is the potential revenue impact of these features?

Analyst

What is the impact of these changes to existing components?

Which components are impacted by the changes? Is there any regression on these components?

What is the impact of these changes to existing components?

Which components are impacted by the changes? Is there any regression on these components?

Developer

What needs to be tested now? Which fixes have been delivered? Do they fix the primary use case? What are the additional use cases to verify?

What is the stability of this build? How many regressions were introduced? Does the defect trend match projections?

What needs to be tested now? Which fixes have been delivered? Do they fix the primary use case? What are the additional use cases to verify?

What is the stability of this build? How many regressions were introduced? Does the defect trend match projections?

Tester

ConstructionAnalysisTriageDeployment Transition

PackagedApplication

RDBMS

WebService

Jservice

JSPs, struts,Servlets

Enterprise ACICS

Maintenance of Existing Systems and Software

App Servers

EJBs

App Servers

WebServer

EJBs

EdgeServer

Enterprise B

Traceability

Architecture

Test Case

Test Case

FEATreq

FEATreq

Code

Code

Execution Trace

Test Case

Defect

Defect

Planning

14

What needs to be tested now? Which fixes have been delivered? Do they fix the primary use case? What are the additional use cases to verify?

What is the stability of this build? How many regressions were introduced? Does the defect trend match projections?

What needs to be tested now? Which fixes have been delivered? Do they fix the primary use case? What are the additional use cases to verify?

What is the stability of this build? How many regressions were introduced? Does the defect trend match projections?

Tester

Is the fix delivery vehicle working? Does the install work? Does the fix solve the issue?

Is the fix delivery vehicle working? Does the install work? Does the fix solve the issue?Tester

Does this solve the issue? Does this solve the issue?

Operations

Is the customer satisfied with the fix? Is the customer satisfied with the fix?

Tech Support

TransitionFixAnalysisTriageDeployment

PackagedApplication

RDBMS

WebService

Jservice

JSPs, struts,Servlets

Enterprise ACICS

Maintenance of Existing Systems and Software

App Servers

EJBs

App Servers

WebServer

EJBs

EdgeServer

Enterprise B

Traceability

Architecture

Test Case

Test Case

FEATreq

FEATreq

Code

Code

Execution Trace

Test Case

Defect

Defect

Planning

Driver

15

Don’t be intimidated

16

The IBM Software Development Platform

Analyst Architect Developer Tester

Visually construct, transform,integrate

and generate code

Design, create,

and execute tests

Model, simulate, assemble,

and monitor business

processes

Modelapplications

and data

IndustryStandards

Training & Mentoring

Microsoft

IBM and3rd Party Servers

DB2LotusRationalTivoliWebSphere

Pervasiveand

Embedded

Partners

Follow a common process Track project status Manage requirements

Manage changeand assets

Manage qualityProject Manager

Deployer

Provision, configure, tune and

troubleshoot applications

Develop Iteratively

Focus on Architecture

Manage Changeand Assets

Continuously EnsureQuality

17

Building a quality ecosystem for flexibility and growth

Role-customized software quality

tools and processes

Global deployment awareness

Seamless asset unification

Open, extensible, technology/platform-

independent infrastructure

Execution Trace

UCreq

FEATreq

Architecture

Code

UC Realizations

Test CaseTest Plan

H

H

H

H

HYADES

Analys

t

Teste

r

Architect

Developer

IT Operations

Verify and validate from the beginning

Maintain traceable, actionable cross-lifecycle

linkages

Incorporate post-deployment analysis

18

Application Lifecycle ManagementClosed-loop problem determination, repair, and redeployment

Reduce business downtimewhile accelerating business value

throughput…

1. Quickly discover and understand application-level errors even after deployment

2. Speed Tivoli-aware application fix and (re)build

3. Optimize and accelerate (re)deployment

…by bridging developmentand operations teams

Sample TechnologiesSample Technologies Tivoli Monitoring Tivoli Configuration Management Tivoli Provisioning/Orchestration Rational Test and Runtime Analysis Rational Version and Build Control Rational Modeling

Monitor-readyApplication

Development

Monitor-readyApplication

Development

ProblemDetermination

ProblemDetermination

Operations TeamOperations Team

DevelopmentTeam

DevelopmentTeam

1122

33 Rapid DeploymentRapid Deployment

19

Quality is FREE

- A well-defined process with unanimous buy-in from team members- Focused attention to value-add, detail and quality across the lifecycle- Constant evaluation of end-user feedback to ensure reinvigoration of the product line and satisfied, repeat customers

- Poorly defined process with inconsistent buy-in - Front-loaded requirements gathering without reassessment or validation- Absent or limited architectural and code quality assurance - Late cycle system testing - Limited post-deployment monitoring and assessment

20

Quality enables differentiation

Through 2006, 80 percent of development organizations will not follow strict software

engineering practices and will fail in 70 percent of their application delivery efforts

21

Quality enables innovation

22

An IBM & Gartner “rule of thumb” for the relative costs to fix defects: Requirements 1.5 Design: 1.0 Construct: 1.5 Prior to test: 10.0 Testing: 60.0 Production: 100.0

It’s your choice

Cost of a Defect (when identified)

0

20

40

60

80

100

120

Requir

emen

ts

Design

Constr

uct

Prior t

o te

st

Testin

g

Produ

ction

Development Phase

Re

lati

ve

Co

st

Cost of a Defect

Note: Does not include business implications of lost revenue and long term impact to business image

You can wait…..

23

….or you can act

Take personal responsibilityWork together

Establish a right-sized, iterative processAutomate whenever, whereverNever assume and don’t settle

24