cse 8314 - sw measurement and quality engineering copyright © 1995-2005, dennis j. frailey, all...

version 5.09

Slide 1CSE 8314 - SW Measurement and Quality EngineeringCopyright © 1995-2005, Dennis J. Frailey, All Rights Reserved CSE8314M15

SMU CSE 8314 / NTU SE 762-N

Software Measurement and Quality Engineering

Module 15Six Sigma and Zero Defects - Overview

version 5.09


Contents• Zero Defects Overview• Six Sigma Overview• Summary

version 5.09


Zero Defects and Six Sigma

General Concepts

version 5.09


Zero Defects and Six Sigma• Each of these is a measure of

quality• And also a set of principles and

concepts that can be used to establish a program for quality improvement

version 5.09


GoalsA measure of quality that can be applied to anything you produce and that will result in unmatched quality across the board

• Dimensionless• Fosters & motivates quality improvement• Provides insight into the quality

improvement process• Correlates to our innate sense of quality

version 5.09


Zero Defects

See Schulmeyer in reference list

version 5.09


Average Defects per Product

Zero Defects is a Stretch GoalStretch Goal

No defects in delivered productsExpectation

Number of defects will diminish» (asymptotically approach zero)

time

version 5.09


Experience with Zero Defects

0

2

4

6

8

10

Cost of Non- Conformance Cost of Conformance Net Cost

Many quit about here

version 5.09


Another Experience withZero Defects

Number of Products with Indicated Number of Defects

0100200300

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16Number of Defects

Last Year This Year

version 5.09


How Do You Know How Well You Are Doing Relative to

What Is Possible?• The “Zero Defects” measure is not

directly related to “degree of goodness” or “degree of quality” or “what is possible”

• It only measures defects on an absolute scale

version 5.09


• What is needed for some products is not needed for others -- you need to know what the customer requires

How Do You Correlate the Defect Rate to What is

Possible or to Relative Cost?

version 5.09


How Many Products Must Be Defect Free?

• 99%– would mean 1 typo per 100 words of

course notes which is fairly good– but --- 200,000 wrong drug

prescriptions per year - very bad• 99.9%

– 1 typo per page - good– 500 surgical errors per week

• 99.99%– 2000 mail delivery errors per hour

version 5.09


What about Variance?• Average product vs worst case

product– Which one matters more?– Is it better to have an average

product with .3 defects or to have a worst case product with 2 defects?

• In its simplest form, “zero defects” does not tell us the answer to this kind of question

version 5.09


Other Problems with Zero Defect Programs

• How do you gain insight into the nature of the problems?– The measure says nothing about the

causes of the defects or how to cure them

• How do you justify continuous improvements in defect removal?– There is no good way to know if you

can justify the cost

version 5.09


Six Sigma

version 5.09


Premise of Six Sigma Programs

• You use a process to produce something

• The process can vary as well as the product

• Average number of defects is not an acceptable measure . . .– You need to understand the worst case

and why it happens– You need to control process variance as

well as defects

version 5.09


Product Quality Depends on:1) The design of the product2) The materials used to construct it3) The process used to produce it

1) Design 2) Materials

3) ProductionProcess

(Outputs)(Inputs)Products

version 5.09


Applying to Software• For software, the product is

essentially a design!• So the three factors become:

– Inputs:1) The architecture of the software2) The requirements of the software

– Process:3) The software development process

version 5.09


Software Quality Depends on:

1) The architecture of the software2) The requirements used to

construct it3) The process used to develop it

3) DevelopmentProcess1) Architecture

2) Requirements

(Outputs)(Inputs)Software

version 5.09


Basic Units ofSix Sigma Programs

• Opportunity– Any step of the process, type of

material, or design element that can cause a defect

• Defect– Anything wrong with the product

• Defects per million opportunities (DPMO)– This is the basic measure of quality

version 5.09


DPMO• Defects are problems with the

product• Opportunities are steps in the

process, materials, or design elements where you can make a mistakeDPMODPMO = Number of Defects * 106

Number of Opportunities

version 5.09


To Improve Quality• Decrease the number of defects or• Increase the number of opportunities?

A common mistake when first learning about six sigma concepts is to assume that increasing opportunities is a good

thing. In fact, it is a bad thing.

version 5.09


Count opportunities in such a way as to inflate the number, so that DPMO goes down.

Make the process more complex so as to increase the number of opportunities

Correct the process to reduce defects Simplify the process so that

opportunities go down and total defects go down as well

Uses and Abuses of DPMO

version 5.09


Software Defects & Opportunities

• A defect is a failure to meet requirements

• An opportunity is a process step, architecture element, or requirement where a defect could originate

• In a complex product like software, it is very hard to determine all of the opportunities

version 5.09


Characteristics of an Opportunity

• Independent from other opportunities• Constant total number for the

product/process if there is no change in the process

• Consistent measuring method is more important than exact definition

The real objective is to measure & improve, not to count opportunities

perfectly.

version 5.09


Software Opportunities• One shortcut measure of

opportunities is lines of code (if you measure size in lines of code)

• Another might be steps of the process, if your process description is very detailed down to the individual work stepDo NOT get overly concerned about the

definition of an opportunity or defect

version 5.09


Goals

• This goal is derived from a specific analysis of processes and has specific rationale behind it– Details in later slides and modules

• Other goals (more or less aggressive) can be established, based on understanding the rationale

6 Sigma goal: DPMO < 3.4

version 5.09


Techniques to Achieve 6 Sigma

1) Design the product for producability

2) Improve the quality of the materials

3) Design the process to produce quality products

1) Design 2) Materials

3) ProductionProcess

(Outputs)(Inputs)Products

version 5.09


Techniques to Achieve 6 Sigma Quality for Software

1) Architect the software for ease of development and maintenance

2) Improve the quality of the requirements3) Design the development process to produce quality software

1) Architecture 2) Requirements

3) DevelopmentProcess

(Outputs)(Inputs)Software

version 5.09


Process Analysis forManufactured Products

• Each product or raw material has certain measurable characteristics

• Example: a bolt could be measured in terms of:– Length– Diameter– Spacing of threads– Diameter of head– Weight– Strength of material under specific conditions

version 5.09


For Each Measurable Property

You Establish SpecificationsExample: “Diameter should be 0.4 inches, plus

or minus .002 inches”• Notice that there are two

components:–A target value

• specification–A range of values that are acceptable

• specification limits

version 5.09


Responsibility of Designer• Specify tolerances• Design products with reasonable

tolerances relative to the manufacturing capability

• Create designs that can be produced within the stated tolerances

version 5.09


Responsibility of Software Architect

• Specify bounds of acceptable software behavior

• Architect software with reasonable tolerances relative to the design and programming capabilities of the staff

• Create software architectures that can be produced and maintained

version 5.09


SEI Claim0

5

10

15

20

1 3 5 7 9 11 13 15 17 19

0

5

10

15

20

25

1 3 5 7 9 11 13 15 17 19

0

5

10

15

20

25

30

1 3 5 7 9 11 13 15 17 19

0

5

10

15

20

25

30

1 3 5 7 9 11 13 15 17 19

0

10

20

30

40

1 3 5 7 9 11 13 15 17 19

1

2

3

4

5

As you increase

maturity level, the cost and

schedule decrease and the variance goes down

version 5.09


Problems with Software and Sigma

• Software generally has low product volume compared with manufactured products– But what if we measure units, tests,

objects, screens, functions, etc?• Software development process has

very high variance– Does it need to?– Is that necessarily bad?

version 5.09


Problems with Software (continued)

• We don’t have the statistical data to back up applying these techniques– But maybe that does not matter if the

ideas for improvement still apply• NIH (Not Invented Here)

version 5.09


Summary• Six Sigma and Zero Defects

programs involve– Measures of quality, such as– Goals that minimize quality problems– Methods of quality improvement

• Six Sigma focuses on variability as well as overall qualityThe next module will cover six sigma

principles and applications.

version 5.09


References• Harry, Mikel J. and J. Ronald Lawson, Six Sigma

Producibility Analysis and Process Characterization, Motorola University Press, Addison-Wesley, ISBN 0-201-63412-0

• Schulmeyer, G. Gordon. Zero Defect Software. McGraw Hill, 1990. ISBN 0-07-055663-6.

• Schulmeyer, G. Gordon and James McManus. Handbook of Software Quality Assurance, Second Edition (especially chapter 17). Van Nostrand Reinhold, New York, 1992. ISBN 0-442-00796-5.

version 5.09


END OFMODULE 15

cse 8314 - sw measurement and quality engineering copyright © 1995-2005, dennis j. frailey, all...

Documents