[email protected] contents introduction requirements engineering project management software...
Post on 20-Dec-2015
213 views
TRANSCRIPT
PVK-HT04 [email protected] 1
ContentsIntroduction Requirements Engineering Project Management Software Design Detailed Design and Coding Quality AssuranceMaintenance
Analysis
Design
Testing
Coding
Operation andMaintenance
Installation
Require-ments
Requirements
Specification
Planning
PVK-HT04 [email protected] 2
Quality Assurance
Introduction Testingo Testing Phases and Approacheso Black-box Testingo White-box Testingo System Testing
PVK-HT04 [email protected] 3
What is Quality Assurance?
Constructive vs analytic approaches to QAQualitative vs quantitative quality standardsMeasuremento Derive qualitative factors from measurable
quantitative factorsSoftware Metrics
QA is the combination of planned and unplanned activities to ensure the fulfillment of predefined quality standards.
PVK-HT04 [email protected] 4
Approaches to QAConstructive Approaches
o Syntax-directed editorso Type systemso Transformational programmingo Coding guidelineso ...
Analytic approaches
o Inspectionso Static analysis tools (e.g. lint)o Testingo ...
Usage of methods, languages, and tools that ensure the fulfillment of some quality factors.
Usage of methods, languages, and tools to observe the current quality level.
PVK-HT04 [email protected] 5
Fault vs Failure
?!human error fault failure
can lead to can lead to
Different types of faultsDifferent identification techniquesDifferent testing techniques
Fault prevention and -detection strategies should be based on expected fault profile
Specification/requirements
Environment/support
Documen-tation OtherDesign Code
Fault origin: WHERE?
Missing Unclear Wrong Changed Better way
Fault mode: WHY?
Fau
lt t
ype:
WH
AT
? Requirementsor
specifications
Functionality
HW interface
SW interface
User interface
Functionaldescription
Test HW
Test SW
Integration SW
Developmenttools
Logic
Computation
Data handling
Moduleinterface/
implementation
Standards
(Inter-)Processcommunications
Data definition
Module design
Logic description
Error checking
Standards
HP´s Fault Classification
PVK-HT04 [email protected] 7
Fault Profile of a HP Division
6%
19%
5%
4%
5%
32%
18%
11%
Data handling
Documentation
Requirements
Hardware
Process/ interprocess
Logic
Computation
Other codeSee [Pfleeger 98].
PVK-HT04 8
Testing Phases
Unittest
Unittest
Unittest
Integrationtest
Functiontest
Performancetest
Acceptance(,) test
Installationtest
Com
ponent
code
Com
ponent
code
.
.
.
Teste
d co
mponent
Test
ed c
om
ponent
Integratedmodules
Functioningsystem
Verifiedsoftware
Accepted,validatedsystem
SYSTEMIN USE!
Designspecifications
Systemfunctional
requirements
Othersoftware
requirements
Customerrequirementsspecification
Userenvironment
Systemtest
Pre-implementationtest
9
Pre-Implementation Testing
Inspectionso evaluation of documents and code prior to technical
review or testingWalkthrougho In teamso Examine source code/detailed design
Reviewso More informalo Often done by document owners
Advantageso Effectiveo High learning effecto Distributing system knowledge
Disadvantageso Expensive
PVK-HT04 [email protected] 10
Testing vs “Proofing” Correctness
Verificationo Check the design/code against the requirements
Are we building the product right?Validationo Check the product against the expectations of the
customerAre we building the right product?
Testing
Testing can neither proof that a program is error free, nor that it is correct
[Myers 76]
Testing can only show the presence of errors, never their absence.
[Dijkstra 69]
Testing is the process in which a (probably unfinished) program is executed with the goal to find errors.
11
Testing PrinciplesConstruction of test suiteso Plan tests under the assumption to find errorso Try typical and untypical inputso Build classes of inputs and choose representatives of each
classCarrying out testso Testers implementerso Define the expected results before running a testo Check for superfluous computationo Check test results thoroughlyo Document test thoroughly
Simplify testo Divide programs in separately testable unitso Develop programs test friendly
Each test must be reproducible
PVK-HT04 [email protected] 12
Test MethodsStructural testing (white-box, glass-box)o Uses code/detailed design is to develop test caseso Typically used in unit testingo Approaches:
• Coverage-based testing• Symbolic execution• Data flow analysis• ...
Functional testing (black-box)o Uses function specifications to develop test caseso Typically used in system testingo Approaches:
• Equivalence partitioning• Border case analysis• ...
time
developblack-boxtest cases
developwhite-boxtest cases
performwhite-box
testing
performblack-box
testing
PVK-HT04 [email protected] 13
Test Preparation
Exhaustive testing is prohibited, because of the combinatorial explosion of test casesChoose representative test data
for i := 1 to 100 do if a = b then X else Y;
i paths to test #tests
1 X, Y 22 XX, XY, YX, YY 43 XXX, XXY, ... 8
...100 2100 2 2100 - 2 > 2,5 1030
With 106 tests/sec this would take 81016 yearsChoose test data (test cases)
PVK-HT04 [email protected] 15
Test Case DevelopmentProblems:o Systematic way to develop test caseso Find a satisfying set of test cases
Test case test dataTest data: Inputs devised to test the systemTest case:o Situation to testo Inputs to test this situationo Expected outputs Test are reproducible
Equivalence partitioning Coverage-based testing
PVK-HT04 [email protected] 16
Equivalence Partitioning
System
Input data
Output data
Inputs causinganomalous behaviour
Outputs which revealthe presence of faults
Input- and output data can be grouped into classes where all members in the class behave in a comparable way.
Define the classes Choose representatives
Typical element Borderline cases
x [25 .. 100] class 1: x < 25class 2: x >= 25 and x <= 100class 3: x > 100
PVK-HT04 17
Equivalence Partitioning Examples
oModule keeps a running total of amount of money receivedoInput: AmountofContribution, specified as being a value from $0.01 to $99,999.99
oModule prints an appropriate response letteroInput: MemberStatus, specified as having a value of {Regular, Student, Retiree, StudioClub}
o 3 equivalence classes:Values from $0.01 to $99,999.99
(valid)Values less than $0.01 (invalid)Values greater than $99,999.99
(invalid)
o 2 equivalence classes:Values of {Regular, Student, Retiree, StudioClub}
(valid)All other input (invalid)
PVK-HT04 [email protected] 18
Statement Coverage
1
3
2 4
5
6 7
8
2 test cases: 12467; 13567
Every statement is at least executed once in some test
PVK-HT04 [email protected] 19
Branch Coverage
1
3
2 4
5
6 7
8
2 test cases: 12467; 1358
For every decision point in the graph, each branch is at least chosen once
PVK-HT04 [email protected] 20
Condition Coverage
Test all combinations of conditions in boolean expressions at least onceif (X or not (Y and Z) and ... then b;c := (d + e * f - g) div op( h, i, j);
PVK-HT04 [email protected] 21
Path Coverage
Assure that every distinct paths in the control-flow graph is executed at least once in some test
PVK-HT04 [email protected] 22
Path Coverage
1
3
2 4
5
6 7
8
Assure that every distinct paths in the control-flow graph is executed at least once in some test
PVK-HT04 [email protected] 23
Path Coverage
1
3
2 4
5
6 7
8
4 test cases: 12467; 1358; 1248; 13567
Assure that every distinct paths in the control-flow graph is executed at least once in some test
PVK-HT04 [email protected] 25
Data-flow testing
Def-use analysis: match variable definitions (assignments) and uses. Example:
x = 5; … if (x > 0) ...
Does assignment get to the use?
PVK-HT04 [email protected] 26
Data Flow CoverageAll-uses coverage
x :=2
x :=1
z := 2*r
x :=3
z := 2*x-y
z :=x+y
y :=2r :=4
Red path covers the defs y :=2; r :=4; x :=1
Blue path covers y :=2; x :=3. Does not cover x :=2
PVK-HT04 [email protected] 27
Coverage-based TestingAdvantageso Systematic way to develop test caseso Measurable results (the coverage)o Extensive tool support
• Flow graph generators• Test data generators• Bookkeeping• Documentation support
Disadvantageso Code must be availableo Does not (yet) work well for data-driven
programs
PVK-HT04 [email protected] 28
Integration TestingHow to integrate & test the system
Top-downBottom-upCritical units firstFunctionality-oriented (threads)
Implications of build order Top-down => stubs; more thorough top-levelBottom-up => drivers; more thorough bottom-levelCritical => stubs & drivers.
A
DCB
GFE Test E Test F Test G
Test B,E,F Test C Test D,G
Test all
PVK-HT04 [email protected] 29
Further phases of testing
System testing: Test the functionality, performance, reliability, security of the entire system.Acceptance testing: Operate system in user environment, with standard user input scenarios. Regression testing: Test of modified versions of previously validated system.
30
Testing Tools / SupportTest data generatorso Input: Program + testing strategyo Output: Sets of input data
Profilerso Instrument code to collect run-time data
• Time spent in operations• Number of calls to operations• Number of loop iterations• ... Find bottle-necks Indicate dead code
Simulatorso Common in hard-/software systems and/or real-time
systemso Emulate critical parts by software
PVK-HT04 [email protected] 31
Testing Tools / Support
Debuggerso Manual code instrumentationo Inspect/trace variableso ...
File comparatorso E.g. for regression testing
Test-stub/-driver generatorso Simulate client or server components, which
are not yet availableClient
Server