chapter 22 developer testing peter j. lane. testing can be difficult for developers to follow ...
Post on 03-Jan-2016
216 Views
Preview:
TRANSCRIPT
Testing can be difficult for developers to follow
Testing’s goal runs counter to the goals of the other development activities. The goal in testing a software is to break it.
It can never completely prove the absence of errors.
Testing alone does not improve the software’s quality. Fix it after you test it.
You are required to assume that you’ll find errors in your code.
If you assume you won’t, then that is more than likely so. Even when there are errors.
Testing During Construction“White/glass – box” testing
Can look into the class while testing
“Black – Box” testing See no further than the interface while testing
Easier to debug if you test a routine or class one at a time vs. several at once.
Recommended approach, cover this ground:Test each relevant requirement to make sure
that the requirements have been implemented.
Test each relevant design concern to make sure that the design has been implemented.
“Basis testing” adds detailed test cases to those that the test requirements and the design.
Make a checklist for past errors.
Test First or Last?Writing test cases before code does not take any more effort
than after.
If first, you can detect defects earlier and you can correct them more easily.
Writing cases first forces you to think a little bit about the requirements and design.
It exposes requirement problems sooner before code is written.
LimitationsDeveloper tests tend to be “clean” tests.
Tries to how it works rather than how it breaks.
Tends to have an optimistic view of test coverage. Most programmers assume they’ve covered more
than they actually did,
Tends to skip more sophisticated kinds of test coverage.
Incomplete TestingPick test cases that are most likely to find
errors.
Concentrate on picking a few that tell you different things instead of a set that says the same thing again and again.
Structured Basis Covering Start with 1 test case for the routine itself.
Add 1 test case for every logic statement (if, while, for, and and or)
Data-Flow TestingDefined – The data has been initialized, but
not used yet.
Used – Data that’s been used for computation.
Killed – Once defined data that has now been undefined.
Entered – Control flow enters routine immediately before the variable is acted upon.
Exited – The control flow leaves the routine immediately after the variable acted upon.
Equivalence Partitioning If two test cases bring out exactly the same
errors, you need only one of them.
Helps when you are not looking at it from source code.
Boundary analysis & Compound BoundariesTest cases that exercise the maximum and
minimum of allowable values.
Classes of Bad Data
Too little data (or no data)
Too much data
Invalid data
Wrong size of data
Uninitialized data
Classes of Good DataNominal cases – middle-of-the-road, expected
values
Min and Max normal configurations
Compatibility with old data
Test Cases that make Hand checks convenientUsing numbers in values that are easy to
calculate by hand for testing.
Which classes contain most errors80%of the errors are found in 20% of a
project’s classes or routines.
50% of the errors are found in 5% of a project’s classes.
Errors by ClassificationScope of most errors is fairly limited
Many errors are outside the domain of construction Thin application domain knowledge, conflicting requirements, bad
communication
Most construction errors are the programmer’s fault 95% programmer, 2% system software, 2% other software, 1%
hardware
Typos are surprisingly common source problems
Misunderstanding the design is a recurring theme in studies of programmers errors
Most are easy to fix
It’s a good idea to measure your own organization’s experience with errors.
Proportion from Faulty ConstructionOn small projects, construction defects make
up the vast bulk of all errors.
Construction defects account for at least 35% of all projects regardless of size.
Although cheaper to fix than requirements and design errors, construction errors are still expensive.
Errors in Testing ItselfCheck your work
Plan test cases as you develop your software
Keep your test cases
Plug unit tests into a test framework
The scientific Method of Debugging1. Stabilize the error
Unpredictable errors are often initialization errors Requires narrowing a test case that produces the error.
2. Locate the source of error Gather data that produces defect Analyze the data that has been gathered, and form a
hypothesis about the defect Determine how to prove/disprove the hypothesis, by testing or
examining the code and do so.
3. Fix defect
4. Test fix
5. Look for similar errors
Tips for finding defectsUse all the data available to make hypothesis
Refine the test case that produce the error
Exercise the code in your unit test suite Test code in isolation
Use available tools
Reproduce the error several different ways
Tips (part 2)Generate more data to generate more
hypotheses
Use results of negative tests
Brainstorm for possible hypotheses
Make list of things to try
Narrow the suspicious region of the code
Tips (part 3)Be suspicious of classes and routines that have had
defects before
Check code that’s changed recently
Expand the suspicious region of the code
Integrate incrementally
Check for common defects
Talk to someone else about the problem
Take a break from the problem
Syntax ErrorsDon’t trust line numbers in compiler messages
Compiler may place error on the line before or after the actual error.
Don’t trust compiler messages May need to read between the lines to understand
Don’t trust compiler’s second message Errors following the first may no mean anything
Divide and conquer Temporarily remove part of code and compile again.
Find misplaced comments and quotation marks
Fixing defects
Understand the problem before you fix it
Understand the program, not just the problem
Confirm the defect diagnosis
Relax
Save original source code
Fix the problem, not the symptom
Change code only for good reasons; be certain that it will work
One change at a time
top related