![Page 1: Software Testing mae. Admin Need schedule of meeting with graders for user test Rubrics for Arch Eval posted Scheduled for next Tuesday will be posted](https://reader035.vdocuments.net/reader035/viewer/2022062619/5518c299550346b31f8b5636/html5/thumbnails/1.jpg)
Software Testing
mae
![Page 2: Software Testing mae. Admin Need schedule of meeting with graders for user test Rubrics for Arch Eval posted Scheduled for next Tuesday will be posted](https://reader035.vdocuments.net/reader035/viewer/2022062619/5518c299550346b31f8b5636/html5/thumbnails/2.jpg)
Admin
• Need schedule of meeting with graders for user test
• Rubrics for Arch Eval posted• Scheduled for next Tuesday will be posted on
Saturday
![Page 3: Software Testing mae. Admin Need schedule of meeting with graders for user test Rubrics for Arch Eval posted Scheduled for next Tuesday will be posted](https://reader035.vdocuments.net/reader035/viewer/2022062619/5518c299550346b31f8b5636/html5/thumbnails/3.jpg)
convention wisdom
95% of errors are in 5% of the code
…or maybe 80% of the errors are in 20% of the code…
Testing: can I find that 5% or 20% via a test suite
![Page 4: Software Testing mae. Admin Need schedule of meeting with graders for user test Rubrics for Arch Eval posted Scheduled for next Tuesday will be posted](https://reader035.vdocuments.net/reader035/viewer/2022062619/5518c299550346b31f8b5636/html5/thumbnails/4.jpg)
Software Testing
• Goals• Types of tests• Levels of tests• Test measures• Test plan
![Page 5: Software Testing mae. Admin Need schedule of meeting with graders for user test Rubrics for Arch Eval posted Scheduled for next Tuesday will be posted](https://reader035.vdocuments.net/reader035/viewer/2022062619/5518c299550346b31f8b5636/html5/thumbnails/5.jpg)
Goals
• Verification: Have we built the software right?
• Validation: Have we built the right software?
Bug-free, meets specs
Meets customers’ needs
Are Verification and Validation different or variation of the same ideaMy argument: “meeting specs” should equal “meeting customer needs”... not generally true (my kitchen, satellite systems)
![Page 6: Software Testing mae. Admin Need schedule of meeting with graders for user test Rubrics for Arch Eval posted Scheduled for next Tuesday will be posted](https://reader035.vdocuments.net/reader035/viewer/2022062619/5518c299550346b31f8b5636/html5/thumbnails/6.jpg)
Software Testing
• Goals• Types of tests• Levels of tests• Test measures• Test plan
most of this discussion focuses on verification
(more specifically bug testing)
![Page 7: Software Testing mae. Admin Need schedule of meeting with graders for user test Rubrics for Arch Eval posted Scheduled for next Tuesday will be posted](https://reader035.vdocuments.net/reader035/viewer/2022062619/5518c299550346b31f8b5636/html5/thumbnails/7.jpg)
Types of tests
• Black box• White box• Gray box
![Page 8: Software Testing mae. Admin Need schedule of meeting with graders for user test Rubrics for Arch Eval posted Scheduled for next Tuesday will be posted](https://reader035.vdocuments.net/reader035/viewer/2022062619/5518c299550346b31f8b5636/html5/thumbnails/8.jpg)
Black box tests
input output
interface
1. Does it perform the specified functions?2.Does it handle obvious errors in input?3.Ariane5 – lousy error handling4.Classic ints vs floats, yards vs meters Black box should catch these if there is adequate “test coverage”
![Page 9: Software Testing mae. Admin Need schedule of meeting with graders for user test Rubrics for Arch Eval posted Scheduled for next Tuesday will be posted](https://reader035.vdocuments.net/reader035/viewer/2022062619/5518c299550346b31f8b5636/html5/thumbnails/9.jpg)
Example: ordered list of ints
class OrdInts
create
getFirst
getNext
insert
delete
L=create()L.insert(5)L.insert(-1)L.insert(-1)p=L.getFirst()print (p)L.delete(p)p=L.getFirst()print(p)p=L.getNext(p)print(p)p=L.getNext(p)
-1-15error
![Page 10: Software Testing mae. Admin Need schedule of meeting with graders for user test Rubrics for Arch Eval posted Scheduled for next Tuesday will be posted](https://reader035.vdocuments.net/reader035/viewer/2022062619/5518c299550346b31f8b5636/html5/thumbnails/10.jpg)
Black box tests• Advantage: black box tester≠developer is
unbiased by implementation details. e.g., Use Case testing, just work through all the Use Cases
• Disadvantage: black box tester is uninformed about implementation details– unnecessary tests – test same thing in different
way– insufficient tests – can miss the extremes,
especially if actual use follows a different pattern
![Page 11: Software Testing mae. Admin Need schedule of meeting with graders for user test Rubrics for Arch Eval posted Scheduled for next Tuesday will be posted](https://reader035.vdocuments.net/reader035/viewer/2022062619/5518c299550346b31f8b5636/html5/thumbnails/11.jpg)
black box tests
Input Code
choose good distribution of input – hope good distribution of code tested
x
x
x x
![Page 12: Software Testing mae. Admin Need schedule of meeting with graders for user test Rubrics for Arch Eval posted Scheduled for next Tuesday will be posted](https://reader035.vdocuments.net/reader035/viewer/2022062619/5518c299550346b31f8b5636/html5/thumbnails/12.jpg)
unnecessary tests
Input Code
large range of input may exercise a small part of codee.g., operator test of satellite control stations, run through each
input and output light/key option. Testing same functions, whereas no one had a test for my map function.
![Page 13: Software Testing mae. Admin Need schedule of meeting with graders for user test Rubrics for Arch Eval posted Scheduled for next Tuesday will be posted](https://reader035.vdocuments.net/reader035/viewer/2022062619/5518c299550346b31f8b5636/html5/thumbnails/13.jpg)
insufficient tests
Input Code
a small range of input may exercise a large range of codebut can you ‘know’ this without knowing the code? Did we miss the 20%
![Page 14: Software Testing mae. Admin Need schedule of meeting with graders for user test Rubrics for Arch Eval posted Scheduled for next Tuesday will be posted](https://reader035.vdocuments.net/reader035/viewer/2022062619/5518c299550346b31f8b5636/html5/thumbnails/14.jpg)
sufficient tests
Input Code
complexcode
a small range of input may exercise a small but important/error-prone region of code
![Page 15: Software Testing mae. Admin Need schedule of meeting with graders for user test Rubrics for Arch Eval posted Scheduled for next Tuesday will be posted](https://reader035.vdocuments.net/reader035/viewer/2022062619/5518c299550346b31f8b5636/html5/thumbnails/15.jpg)
White box tests
Based on code
test 1
test 2
![Page 16: Software Testing mae. Admin Need schedule of meeting with graders for user test Rubrics for Arch Eval posted Scheduled for next Tuesday will be posted](https://reader035.vdocuments.net/reader035/viewer/2022062619/5518c299550346b31f8b5636/html5/thumbnails/16.jpg)
Example: ordered list of ints
class ordInts {public: …
private: int vals[1000];
int maxElements=1000;…
}
![Page 17: Software Testing mae. Admin Need schedule of meeting with graders for user test Rubrics for Arch Eval posted Scheduled for next Tuesday will be posted](https://reader035.vdocuments.net/reader035/viewer/2022062619/5518c299550346b31f8b5636/html5/thumbnails/17.jpg)
Example: ordered list of ints
bool testMax(){ L=create(); num=maxElements; for (int i=0; i<=num; i++)
print iL.insert(i)
print maxElements;}
![Page 18: Software Testing mae. Admin Need schedule of meeting with graders for user test Rubrics for Arch Eval posted Scheduled for next Tuesday will be posted](https://reader035.vdocuments.net/reader035/viewer/2022062619/5518c299550346b31f8b5636/html5/thumbnails/18.jpg)
White box tests
• Advantage:– design tests to achieve good code coverage and
avoid duplication – can stress complicated, error-prone code– can stress boundary values (fault injection)
• Disadvantage:– tester=developer may have bias– if code changes, tests may have to be redesigned
(is this bad?)
![Page 19: Software Testing mae. Admin Need schedule of meeting with graders for user test Rubrics for Arch Eval posted Scheduled for next Tuesday will be posted](https://reader035.vdocuments.net/reader035/viewer/2022062619/5518c299550346b31f8b5636/html5/thumbnails/19.jpg)
Gray box tests
• Look at code to design tests• But test through interface• Best of both worlds - maybe
![Page 20: Software Testing mae. Admin Need schedule of meeting with graders for user test Rubrics for Arch Eval posted Scheduled for next Tuesday will be posted](https://reader035.vdocuments.net/reader035/viewer/2022062619/5518c299550346b31f8b5636/html5/thumbnails/20.jpg)
Example: ordered list of ints
class OrdInts
create
getFirst
getNext
insert
delete
L=create()L.insert(1)L.insert(2)L.insert(3)
L.insert(1001)p=L.getFirst()print(p)p=L.getNext(p)print p
123
…
……
![Page 21: Software Testing mae. Admin Need schedule of meeting with graders for user test Rubrics for Arch Eval posted Scheduled for next Tuesday will be posted](https://reader035.vdocuments.net/reader035/viewer/2022062619/5518c299550346b31f8b5636/html5/thumbnails/21.jpg)
Types of tests• Black box: test based on interface, through interface
• Gray box: test based on code, through interface
• White box: test based on code, through code
Testing strategy can/should include all approaches!
My experience: Black Box = non developer, outside testing idiots in your case who?White Box = developer, part of development process in your case who?Gray Box = hated, non developer within developer organization in your case who?
![Page 22: Software Testing mae. Admin Need schedule of meeting with graders for user test Rubrics for Arch Eval posted Scheduled for next Tuesday will be posted](https://reader035.vdocuments.net/reader035/viewer/2022062619/5518c299550346b31f8b5636/html5/thumbnails/22.jpg)
Levels of tests
• Unit• Integration• System• System integration
white
black
![Page 23: Software Testing mae. Admin Need schedule of meeting with graders for user test Rubrics for Arch Eval posted Scheduled for next Tuesday will be posted](https://reader035.vdocuments.net/reader035/viewer/2022062619/5518c299550346b31f8b5636/html5/thumbnails/23.jpg)
Testing measures (white box)
• Code coverage – individual modules• Path coverage – sequence diagrams• Code coverage based on complexity – test of
the risks, tricky part of code (e.g., Unix “you are not expected to understand this” code)
![Page 24: Software Testing mae. Admin Need schedule of meeting with graders for user test Rubrics for Arch Eval posted Scheduled for next Tuesday will be posted](https://reader035.vdocuments.net/reader035/viewer/2022062619/5518c299550346b31f8b5636/html5/thumbnails/24.jpg)
Code coverage
• How much of code is “tested” by tests?– manually– profiling tools
• Design new tests to extend coverage• Is 100% good?
![Page 25: Software Testing mae. Admin Need schedule of meeting with graders for user test Rubrics for Arch Eval posted Scheduled for next Tuesday will be posted](https://reader035.vdocuments.net/reader035/viewer/2022062619/5518c299550346b31f8b5636/html5/thumbnails/25.jpg)
Path coverage
• How many execution paths have been exercised by tests?
• 100% path coverage is usually impossible• aim to cover common paths and error prone
(complex) paths• aim to break code with tests – good testers
are not liked by developers....
![Page 26: Software Testing mae. Admin Need schedule of meeting with graders for user test Rubrics for Arch Eval posted Scheduled for next Tuesday will be posted](https://reader035.vdocuments.net/reader035/viewer/2022062619/5518c299550346b31f8b5636/html5/thumbnails/26.jpg)
convention wisdom
95% of errors are in 5% of the code
…or maybe 80% of the errors are in 20% of the code…
![Page 27: Software Testing mae. Admin Need schedule of meeting with graders for user test Rubrics for Arch Eval posted Scheduled for next Tuesday will be posted](https://reader035.vdocuments.net/reader035/viewer/2022062619/5518c299550346b31f8b5636/html5/thumbnails/27.jpg)
code complexity measures
• cyclomatic complexity measure (McCabe)– measures number of linearly independent paths through
program– ignores asynchronous interaction, fallibility of services– etc.
• developer’s best guess – what problems are likely and which will be hardest to diagnose
• started with Knuth and others who gather stats on programs.
![Page 28: Software Testing mae. Admin Need schedule of meeting with graders for user test Rubrics for Arch Eval posted Scheduled for next Tuesday will be posted](https://reader035.vdocuments.net/reader035/viewer/2022062619/5518c299550346b31f8b5636/html5/thumbnails/28.jpg)
Test plan
• Collection of tests: unit, integration, system• Rationale for test: why these tests?• Strategy for developing/performing tests
– e.g., test units as developed, test integration at each build, etc.
![Page 29: Software Testing mae. Admin Need schedule of meeting with graders for user test Rubrics for Arch Eval posted Scheduled for next Tuesday will be posted](https://reader035.vdocuments.net/reader035/viewer/2022062619/5518c299550346b31f8b5636/html5/thumbnails/29.jpg)
Testing Strategy
• TDD – test driven development• Regression• Test harness• Bug tracking• User tests
Write test BEFORE your write the code!
![Page 30: Software Testing mae. Admin Need schedule of meeting with graders for user test Rubrics for Arch Eval posted Scheduled for next Tuesday will be posted](https://reader035.vdocuments.net/reader035/viewer/2022062619/5518c299550346b31f8b5636/html5/thumbnails/30.jpg)
TDD – test DrivenUnit tests are written first by the SEs. Then as code is written it
passes incrementally larger portions of the test suites. The test suites are continuously updated as new failure conditions and corner cases are discovered, and integrated with regression tests. Unit tests are maintained like software and integrated into the build process. The goal is to achieve continuous deployment with frequent updates.
![Page 31: Software Testing mae. Admin Need schedule of meeting with graders for user test Rubrics for Arch Eval posted Scheduled for next Tuesday will be posted](https://reader035.vdocuments.net/reader035/viewer/2022062619/5518c299550346b31f8b5636/html5/thumbnails/31.jpg)
Testing Strategy
• TDD – test driven development• Regression• Test harness• Bug tracking• User tests
AutomateBuild up testsRun tests before you check in codeMy mess up....
![Page 32: Software Testing mae. Admin Need schedule of meeting with graders for user test Rubrics for Arch Eval posted Scheduled for next Tuesday will be posted](https://reader035.vdocuments.net/reader035/viewer/2022062619/5518c299550346b31f8b5636/html5/thumbnails/32.jpg)
Testing Strategy
• TDD – test driven development• Regression• Test harness• Bug tracking• User tests
Useful for interactive graphics applicationsautomated test framework, built one for satellite console to run through all console interactions
![Page 33: Software Testing mae. Admin Need schedule of meeting with graders for user test Rubrics for Arch Eval posted Scheduled for next Tuesday will be posted](https://reader035.vdocuments.net/reader035/viewer/2022062619/5518c299550346b31f8b5636/html5/thumbnails/33.jpg)
Test harness
everything else
input output
inject test inputquery state
need to have a way to evaluate output
![Page 34: Software Testing mae. Admin Need schedule of meeting with graders for user test Rubrics for Arch Eval posted Scheduled for next Tuesday will be posted](https://reader035.vdocuments.net/reader035/viewer/2022062619/5518c299550346b31f8b5636/html5/thumbnails/34.jpg)
Strategy
• TDD – test driven development• Regression• Test harness• Bug tracking• User test
Design system for trackingScrupulously record problems,their context, the effect, ideas on the cause,attempts to fix, create new tests as bugs uncovered
![Page 35: Software Testing mae. Admin Need schedule of meeting with graders for user test Rubrics for Arch Eval posted Scheduled for next Tuesday will be posted](https://reader035.vdocuments.net/reader035/viewer/2022062619/5518c299550346b31f8b5636/html5/thumbnails/35.jpg)
Strategy
• TDD – test driven development• Regression• Test harness• User test Validation
Nonfunctional requirementsNot best for discovering bugsNeed to know how they generated bug
![Page 36: Software Testing mae. Admin Need schedule of meeting with graders for user test Rubrics for Arch Eval posted Scheduled for next Tuesday will be posted](https://reader035.vdocuments.net/reader035/viewer/2022062619/5518c299550346b31f8b5636/html5/thumbnails/36.jpg)
Test plan
• Collection of tests: unit, integration, system• Rationale for test: why these tests?• Strategy for developing/performing tests
Be thoughtful in designingBe diligent in executing