(c) dr. wolfgang j. schneider 2003 - 2006 general testing principles dr. wolfgang j. schneider 2003...

(c) Dr. Wolfgang J. Schneider 2003 - 2006

GENERALGENERALTESTING PRINCIPLESTESTING PRINCIPLES

Dr. Wolfgang J. SchneiderDr. Wolfgang J. Schneider

2003 – 20062003 – 2006

(c) Dr. Wolfgang J. Schneider 2003 - 2006

AcknowledgementAcknowledgementMost of the content has been developed Most of the content has been developed by ISTQBby ISTQB

and it’s affiliates for fair use. The copy and it’s affiliates for fair use. The copy right of those parts is with them.right of those parts is with them.The layout has been developed by us.The layout has been developed by us.

(c) Dr. Wolfgang J. Schneider 2003 - 2006

General Testing PrinciplesGeneral Testing Principles

INTRODUCTIONINTRODUCTIONOver the past 40+ years, a number of Over the past 40+ years, a number of

testing principles have been suggestedtesting principles have been suggested

by different authors.by different authors.

They They offer general guidelines commonoffer general guidelines common

for all testing.for all testing.

(c) Dr. Wolfgang J. Schneider 2003 - 2006

General Testing PrinciplesGeneral Testing Principles

Testing shows Presence of DefectsTesting shows Presence of Defects Exhaustive Testing is ImpossibleExhaustive Testing is Impossible Early TestingEarly Testing Defect ClusteringDefect Clustering Pesticide ParadoxPesticide Paradox Testing is Context DependentTesting is Context Dependent Absence-of-Errors FallacyAbsence-of-Errors Fallacy

(c) Dr. Wolfgang J. Schneider 2003 - 2006

General Testing PrinciplesGeneral Testing Principles

Testing shows Presence of DefectsTesting shows Presence of Defects Exhaustive Testing is ImpossibleExhaustive Testing is Impossible Early TestingEarly Testing Defect ClusteringDefect Clustering Pesticide ParadoxPesticide Paradox Testing is Context DependentTesting is Context Dependent Absence-of-Errors FallacyAbsence-of-Errors Fallacy

(c) Dr. Wolfgang J. Schneider 2003 - 2006

General Testing PrinciplesGeneral Testing Principles

Testing shows Presence of DefectsTesting shows Presence of Defects Exhaustive Testing is ImpossibleExhaustive Testing is Impossible Early TestingEarly Testing Defect ClusteringDefect Clustering Pesticide ParadoxPesticide Paradox Testing is Context DependentTesting is Context Dependent Absence-of-Errors FallacyAbsence-of-Errors Fallacy

(c) Dr. Wolfgang J. Schneider 2003 - 2006

General Testing PrinciplesGeneral Testing Principles

Testing shows Presence of DefectsTesting shows Presence of Defects Exhaustive Testing is ImpossibleExhaustive Testing is Impossible Early TestingEarly Testing Defect ClusteringDefect Clustering Pesticide ParadoxPesticide Paradox Testing is Context DependentTesting is Context Dependent Absence-of-Errors FallacyAbsence-of-Errors Fallacy

(c) Dr. Wolfgang J. Schneider 2003 - 2006

General Testing PrinciplesGeneral Testing Principles

Testing shows Presence of DefectsTesting shows Presence of Defects Exhaustive Testing is ImpossibleExhaustive Testing is Impossible Early TestingEarly Testing Defect ClusteringDefect Clustering Pesticide ParadoxPesticide Paradox Testing is Context DependentTesting is Context Dependent Absence-of-Errors FallacyAbsence-of-Errors Fallacy

(c) Dr. Wolfgang J. Schneider 2003 - 2006

General Testing PrinciplesGeneral Testing Principles

Testing shows Presence of DefectsTesting shows Presence of Defects Exhaustive Testing is ImpossibleExhaustive Testing is Impossible Early TestingEarly Testing Defect ClusteringDefect Clustering Pesticide ParadoxPesticide Paradox Testing is Context DependentTesting is Context Dependent Absence-of-Errors FallacyAbsence-of-Errors Fallacy

(c) Dr. Wolfgang J. Schneider 2003 - 2006

General Testing PrinciplesGeneral Testing Principles

Testing shows Presence of DefectsTesting shows Presence of Defects Exhaustive Testing is ImpossibleExhaustive Testing is Impossible Early TestingEarly Testing Defect ClusteringDefect Clustering Pesticide ParadoxPesticide Paradox Testing is Context DependentTesting is Context Dependent Absence-of-Errors FallacyAbsence-of-Errors Fallacy

(c) Dr. Wolfgang J. Schneider 2003 - 2006

General Testing PrinciplesGeneral Testing Principles

Testing shows Presence of DefectsTesting shows Presence of Defects Exhaustive Testing is ImpossibleExhaustive Testing is Impossible Early TestingEarly Testing Defect ClusteringDefect Clustering Pesticide ParadoxPesticide Paradox Testing is Context DependentTesting is Context Dependent Absence-of-Errors FallacyAbsence-of-Errors Fallacy

(c) Dr. Wolfgang J. Schneider 2003 - 2006

General Testing PrinciplesGeneral Testing Principles

Testing shows Presence of DefectsTesting shows Presence of Defects Exhaustive Testing is ImpossibleExhaustive Testing is Impossible Early TestingEarly Testing Defect ClusteringDefect Clustering Pesticide ParadoxPesticide Paradox Testing is Context DependentTesting is Context Dependent Absence-of-Errors FallacyAbsence-of-Errors Fallacy

(c) Dr. Wolfgang J. Schneider 2003 - 2006

Testing showsTesting shows Presence of Defects Presence of Defects

(c) Dr. Wolfgang J. Schneider 2003 - 2006

Testing showsTesting shows Presence of Defects Presence of Defects

Testing can show you that defects areTesting can show you that defects are

there.there.

Testing cannot prove that there are noTesting cannot prove that there are no

defects. defects.

(c) Dr. Wolfgang J. Schneider 2003 - 2006

Testing showsTesting shows Presence of Defects Presence of Defects

Testing can show you that defects areTesting can show you that defects are

there. there.

Testing cannot prove that there are noTesting cannot prove that there are no

defects. defects.

(c) Dr. Wolfgang J. Schneider 2003 - 2006

Testing showsTesting shows Presence of Defects Presence of Defects

Testing can show you that defects areTesting can show you that defects are

there. there.

Testing cannot prove that there are noTesting cannot prove that there are no

defects. defects.

(c) Dr. Wolfgang J. Schneider 2003 - 2006

Testing showsTesting shows Presence of Defects Presence of Defects

(c) Dr. Wolfgang J. Schneider 2003 - 2006

Testing showsTesting shows Presence of Defects Presence of Defects

Testing reduces the probability of Testing reduces the probability of

undiscovered defects remaining in theundiscovered defects remaining in the

software.software.

But, even if no defects are found, it is notBut, even if no defects are found, it is not

a proof of correctness.a proof of correctness.

(c) Dr. Wolfgang J. Schneider 2003 - 2006

Testing showsTesting shows Presence of Defects Presence of Defects

Testing reduces the probability of Testing reduces the probability of

undiscovered defects remaining in theundiscovered defects remaining in the

software.software.

But, even if no defects are found, it is notBut, even if no defects are found, it is not

a proof of correctness.a proof of correctness.

(c) Dr. Wolfgang J. Schneider 2003 - 2006

Testing showsTesting shows Presence of Defects Presence of Defects

Testing reduces the probability of Testing reduces the probability of

undiscovered defects remaining in theundiscovered defects remaining in the

software.software.

But, even if no defects are found, it is notBut, even if no defects are found, it is not

a proof of correctness.a proof of correctness.

(c) Dr. Wolfgang J. Schneider 2003 - 2006

Testing showsTesting shows Presence of Defects Presence of Defects

Testing cannot prove that there are noTesting cannot prove that there are no

defects. defects.

If no defects are found, it is not a proof of If no defects are found, it is not a proof of

correctness.correctness.

(c) Dr. Wolfgang J. Schneider 2003 - 2006

Exhaustive TestingExhaustive Testing is Impossible is Impossible

(c) Dr. Wolfgang J. Schneider 2003 - 2006

Exhaustive TestingExhaustive Testing is Impossible is Impossible

Testing everything (all combinations of inputs and preconditions) is not feasible except for trivial cases. Instead of exhaustive testing, we use risk and priorities to focus testing efforts.

(c) Dr. Wolfgang J. Schneider 2003 - 2006

Exhaustive TestingExhaustive Testing is Impossible is Impossible

Testing everything (all combinations of inputs and preconditions) is not feasible except for trivial cases. Instead of exhaustive testing, we use risk and priorities to focus testing efforts.

(c) Dr. Wolfgang J. Schneider 2003 - 2006

Exhaustive TestingExhaustive Testing is Impossible is Impossible

Testing everything (all combinations of inputs and preconditions) is not feasible except for trivial cases. Instead of exhaustive testing, we use risk and priorities to focus testing efforts.

(c) Dr. Wolfgang J. Schneider 2003 - 2006

Exhaustive TestingExhaustive Testing is Impossible is Impossible

Testing everything (all combinations of inputs and preconditions) is not feasible except for trivial cases. Instead of exhaustive testing, we use risk and priorities to focus testing efforts.

(c) Dr. Wolfgang J. Schneider 2003 - 2006

Early TestingEarly Testing

(c) Dr. Wolfgang J. Schneider 2003 - 2006

Early TestingEarly TestingTesting activities should start as early asTesting activities should start as early as

possible in the software or systempossible in the software or system

development life cycledevelopment life cycle

They should be focused on definedThey should be focused on defined

objectivesobjectives

(c) Dr. Wolfgang J. Schneider 2003 - 2006

Early TestingEarly TestingTesting activities should start as early asTesting activities should start as early as

possible in the software or systempossible in the software or system

development life cycledevelopment life cycle

They should be focused on definedThey should be focused on defined

objectivesobjectives

(c) Dr. Wolfgang J. Schneider 2003 - 2006

Early TestingEarly TestingTesting activities should start as early asTesting activities should start as early as

possible in the software or systempossible in the software or system

development life cycledevelopment life cycle

They should be focused on definedThey should be focused on defined

objectivesobjectives

(c) Dr. Wolfgang J. Schneider 2003 - 2006

Early TestingEarly TestingTesting activities should start Testing activities should start as early asas early as

possiblepossible in the software or system in the software or system

development life cycledevelopment life cycle

They should be focused on definedThey should be focused on defined

objectivesobjectives

(c) Dr. Wolfgang J. Schneider 2003 - 2006

Early TestingEarly TestingTesting activities should start as early asTesting activities should start as early as

possible in the software or systempossible in the software or system

development life cycledevelopment life cycle

They should be They should be focused on definedfocused on defined

objectivesobjectives

(c) Dr. Wolfgang J. Schneider 2003 - 2006

Defect ClusteringDefect Clustering

(c) Dr. Wolfgang J. Schneider 2003 - 2006

Defect ClusteringDefect ClusteringA small number of modules contain mostA small number of modules contain most

of the defects discovered duringof the defects discovered during

pre-release testing, or showpre-release testing, or show

the most operational failures.the most operational failures.

(c) Dr. Wolfgang J. Schneider 2003 - 2006

Defect ClusteringDefect ClusteringA small number of modules contain mostA small number of modules contain most

of the defects discovered duringof the defects discovered during

pre-release testing, or showpre-release testing, or show

the most operational failures.the most operational failures.

A SMALL NUMBER OF MODULESA SMALL NUMBER OF MODULES

CONTAIN MOST OF THE DEFECTSCONTAIN MOST OF THE DEFECTS

(c) Dr. Wolfgang J. Schneider 2003 - 2006

Pesticide ParadoxPesticide Paradox

(c) Dr. Wolfgang J. Schneider 2003 - 2006

Pesticide ParadoxPesticide ParadoxIf the same tests are repeatedIf the same tests are repeated

over and over again,over and over again,

eventually the same set ofeventually the same set of

test cases will no longer findtest cases will no longer find

any new bugs. any new bugs.

(c) Dr. Wolfgang J. Schneider 2003 - 2006

Pesticide ParadoxPesticide ParadoxMost vintage testers call thisMost vintage testers call this

thethe

Mine Field EffectMine Field Effecthttp://www.testingeducation.org/k04/documents/bbstRegression2005.ppt

(c) Dr. Wolfgang J. Schneider 2003 - 2006

Pesticide ParadoxPesticide ParadoxTo overcome this effect, the test casesTo overcome this effect, the test cases

need to be regularly reviewed andneed to be regularly reviewed and

revised, and new and different tests needrevised, and new and different tests need

to be written to exercise different parts ofto be written to exercise different parts of

the software or system to potentially findthe software or system to potentially find

more defects.more defects.

(c) Dr. Wolfgang J. Schneider 2003 - 2006

Testing is Testing is Context DependentContext Dependent

Testing is done differently in differentTesting is done differently in different

contexts. contexts.

For example, safety-critical software isFor example, safety-critical software is

tested differently from an e-commercetested differently from an e-commerce

site.site.

(c) Dr. Wolfgang J. Schneider 2003 - 2006

Testing is Testing is Context DependentContext Dependent

Testing is done differently in differentTesting is done differently in different

contexts. contexts.

For example, safety-critical software isFor example, safety-critical software is

tested differently from an e-commercetested differently from an e-commerce

site.site.

(c) Dr. Wolfgang J. Schneider 2003 - 2006

Testing is Testing is Context DependentContext Dependent

Testing is done differently in differentTesting is done differently in different

contexts. contexts.

For example, safety-critical software isFor example, safety-critical software is

tested differently from an e-commercetested differently from an e-commerce

site.site.

(c) Dr. Wolfgang J. Schneider 2003 - 2006

Testing is Testing is Context DependentContext Dependent

Testing is done differently in differentTesting is done differently in different

contexts. contexts.

For example, safety-critical software isFor example, safety-critical software is

tested differently from an e-commercetested differently from an e-commerce

site.site.

(c) Dr. Wolfgang J. Schneider 2003 - 2006

Testing is Testing is Context DependentContext Dependent

Testing is done differently in differentTesting is done differently in different

contexts. contexts.

For example, safety-critical software isFor example, safety-critical software is

tested differently from an e-commercetested differently from an e-commerce

site.site.

RISKRISK

(c) Dr. Wolfgang J. Schneider 2003 - 2006

Absence-of-Errors Absence-of-Errors FallacyFallacy

Finding and fixing defectsFinding and fixing defects

does not help if the systemdoes not help if the system

built is unusable and doesbuilt is unusable and does

not fulfill the users’ needsnot fulfill the users’ needs

and expectationsand expectations

(c) Dr. Wolfgang J. Schneider 2003 - 2006

Absence-of-Errors Absence-of-Errors FallacyFallacy

Finding and fixing defectsFinding and fixing defects

does not help if the systemdoes not help if the system

built is unusable and doesbuilt is unusable and does

not fulfill the users’ needsnot fulfill the users’ needs

and expectationsand expectations

(c) Dr. Wolfgang J. Schneider 2003 - 2006

Absence-of-Errors Absence-of-Errors FallacyFallacy

Finding and fixing defectsFinding and fixing defects

does not help if the systemdoes not help if the system

built is built is unusableunusable and does and does

not fulfill the users’ needsnot fulfill the users’ needs

and expectationsand expectations

(c) Dr. Wolfgang J. Schneider 2003 - 2006

GENERALGENERALTESTING PRINCIPLESTESTING PRINCIPLES

(c) Dr. Wolfgang J. Schneider 2003 - 2006

General Testing PrinciplesGeneral Testing Principles

Testing shows Presence of DefectsTesting shows Presence of Defects Exhaustive Testing is ImpossibleExhaustive Testing is Impossible Early TestingEarly Testing Defect ClusteringDefect Clustering Pesticide ParadoxPesticide Paradox Testing is Context DependentTesting is Context Dependent Absence-of-Errors FallacyAbsence-of-Errors Fallacy

(c) Dr. Wolfgang J. Schneider 2003 - 2006

DR. WOLFGANG J. SCHNEIDER GMBHInternational Information Technology Consultants

Professional Testing SolutionsTelecommunications and Utility Billing Systems

Euro Preparation for Companies in New EU Member CountriesEDIFACT – DATANORM – ELDANORM – ETIM InterfacesData Exchange with Banks and Credit Card Companies

Total Quality Management – Good Manufacturing PracticesIT Service Management - IT Infrastructure Library (ITIL)

Knowledge Management SolutionsIT Fitness for GAAP – IAS – FASB

Balanced Scorecard

Take Advantage of 30+ Years of IT Experience !

www.drschneider.eu

Dr. Wolfgang J. Schneider - Start