ics 52: introduction to software engineeringtaylor/ics_52_wq04/ics52wq04-09.pdf · university of...
TRANSCRIPT
![Page 1: ICS 52: Introduction to Software Engineeringtaylor/ICS_52_WQ04/ICS52WQ04-09.pdf · University of California, Irvine 4 What Do These Have in Common? First launch of space shuttle Airbus](https://reader031.vdocuments.net/reader031/viewer/2022022502/5aab55b57f8b9a8f498bc766/html5/thumbnails/1.jpg)
University of California, Irvine 1
ICS 52: Introduction to SoftwareEngineering
Winter Quarter 2004Professor Richard N. Taylor
Lecture Notes: Quality Assurance
http://www.ics.uci.edu/~taylor/ICS_52_WQ04/syllabus.htmlCopyright 2004, Richard N. Taylor. Duplication of course
material for any commercial purpose without written permission is prohibited.
![Page 2: ICS 52: Introduction to Software Engineeringtaylor/ICS_52_WQ04/ICS52WQ04-09.pdf · University of California, Irvine 4 What Do These Have in Common? First launch of space shuttle Airbus](https://reader031.vdocuments.net/reader031/viewer/2022022502/5aab55b57f8b9a8f498bc766/html5/thumbnails/2.jpg)
University of California, Irvine 2
Implementation Assignment Issues-- notes from a previous quarter
UI development tool versus by scratch– learning curve– transferable knowledge– tar-baby code
Documentation issue: why it is so important– said this before in class
Late-delivery penalty– Similar to contract penalities....sometimes negotiated late in the game– Does procrastination ever result in a good product?
How could you have forecasted your problems better? Changes to requirements and design during the implementation phase
– Not as the result of malfeasance, or ill-will, or un-cooperativeness– It is an intrinsic part of the problem– For the GUI aspects, recall our discussion on where this belongs in the process.
Personnel changes– What would have happened if I had changed TAs during the process?– What if there were two TA's answering questions?
![Page 3: ICS 52: Introduction to Software Engineeringtaylor/ICS_52_WQ04/ICS52WQ04-09.pdf · University of California, Irvine 4 What Do These Have in Common? First launch of space shuttle Airbus](https://reader031.vdocuments.net/reader031/viewer/2022022502/5aab55b57f8b9a8f498bc766/html5/thumbnails/3.jpg)
University of California, Irvine 3
Today’s Lecture
Quality assuranceBack to the future: testing
![Page 4: ICS 52: Introduction to Software Engineeringtaylor/ICS_52_WQ04/ICS52WQ04-09.pdf · University of California, Irvine 4 What Do These Have in Common? First launch of space shuttle Airbus](https://reader031.vdocuments.net/reader031/viewer/2022022502/5aab55b57f8b9a8f498bc766/html5/thumbnails/4.jpg)
University of California, Irvine 4
What Do These Have in Common? First launch of space shuttle Airbus 320 http://catless.ncl.ac.uk/Risks/10.02.html#subj1.1 Audi 5000 Mariner 1 launch: http://catless.ncl.ac.uk/Risks/5.73.html#subj2.1 AT&T telephone network Ariane 5 http://catless.ncl.ac.uk/Risks/18.24.html#subj2.1 Radiation therapy machine
http://courses.cs.vt.edu/~cs3604/lib/Therac_25/Therac_5.html Y2K
![Page 5: ICS 52: Introduction to Software Engineeringtaylor/ICS_52_WQ04/ICS52WQ04-09.pdf · University of California, Irvine 4 What Do These Have in Common? First launch of space shuttle Airbus](https://reader031.vdocuments.net/reader031/viewer/2022022502/5aab55b57f8b9a8f498bc766/html5/thumbnails/5.jpg)
University of California, Irvine 5
Impact of Failures Not just “out there”
– Space shuttle– Mariner 1– Ariane 5
But also “at home”– Your car– Your call to your mom– Your homework– Your hospital visit
Peter Neumann’s Risks Forum: http://catless.ncl.ac.uk/Risks
![Page 6: ICS 52: Introduction to Software Engineeringtaylor/ICS_52_WQ04/ICS52WQ04-09.pdf · University of California, Irvine 4 What Do These Have in Common? First launch of space shuttle Airbus](https://reader031.vdocuments.net/reader031/viewer/2022022502/5aab55b57f8b9a8f498bc766/html5/thumbnails/6.jpg)
University of California, Irvine 6
Verification and Validation Verification
– Ensure software meets specifications– Internal consistency– “Are we building the product right?”
Validation– Ensure software meets customer’s intent– External consistency– “Are we building the right product?”
![Page 7: ICS 52: Introduction to Software Engineeringtaylor/ICS_52_WQ04/ICS52WQ04-09.pdf · University of California, Irvine 4 What Do These Have in Common? First launch of space shuttle Airbus](https://reader031.vdocuments.net/reader031/viewer/2022022502/5aab55b57f8b9a8f498bc766/html5/thumbnails/7.jpg)
University of California, Irvine 7
Software Qualities
CorrectnessReliabilityRobustnessPerformanceUser friendlinessVerifiabilityMaintainabilityRepairabilitySafety
EvolvabilityReusabilityPortabilityUnderstandabilityInteroperabilityProductivitySizeTimelinessVisibility
![Page 8: ICS 52: Introduction to Software Engineeringtaylor/ICS_52_WQ04/ICS52WQ04-09.pdf · University of California, Irvine 4 What Do These Have in Common? First launch of space shuttle Airbus](https://reader031.vdocuments.net/reader031/viewer/2022022502/5aab55b57f8b9a8f498bc766/html5/thumbnails/8.jpg)
University of California, Irvine 8
Quality Assurance Assure that each of the software qualities is met
– Goals set in requirements specification– Goals realized in implementation
Sometimes easy, sometimes difficult– Portability versus safety
Sometimes immediate, sometimes delayed– Understandability versus evolvability
Sometimes provable, sometimes doubtful– Size versus correctness
![Page 9: ICS 52: Introduction to Software Engineeringtaylor/ICS_52_WQ04/ICS52WQ04-09.pdf · University of California, Irvine 4 What Do These Have in Common? First launch of space shuttle Airbus](https://reader031.vdocuments.net/reader031/viewer/2022022502/5aab55b57f8b9a8f498bc766/html5/thumbnails/9.jpg)
University of California, Irvine 9
An Idealized View of QA
Design, in formal notation
Executable machine code
Execution on verified hardware
Code, in verifiable language
Complete formal specificationof problem to be solved
Correctness-preserving transformation
Correctness-preserving transformation
Correctness-preserving transformation
Correctness-preserving transformation
![Page 10: ICS 52: Introduction to Software Engineeringtaylor/ICS_52_WQ04/ICS52WQ04-09.pdf · University of California, Irvine 4 What Do These Have in Common? First launch of space shuttle Airbus](https://reader031.vdocuments.net/reader031/viewer/2022022502/5aab55b57f8b9a8f498bc766/html5/thumbnails/10.jpg)
University of California, Irvine 10
A Realistic View of QA
Design, in mixed notation
Pentium machine code
Execution on commercial hardware
Code, in C++, Java, Ada, …
Mixture of formal and informal specifications
Manual transformation
Manual transformation
Compilation by commercial compiler
Commercial firmware
![Page 11: ICS 52: Introduction to Software Engineeringtaylor/ICS_52_WQ04/ICS52WQ04-09.pdf · University of California, Irvine 4 What Do These Have in Common? First launch of space shuttle Airbus](https://reader031.vdocuments.net/reader031/viewer/2022022502/5aab55b57f8b9a8f498bc766/html5/thumbnails/11.jpg)
University of California, Irvine 11
First Complication
Real needs
ActualSpecification
“Correct”Specification
No matter how sophisticated the QA process, theproblem of creating the initial specification remains
![Page 12: ICS 52: Introduction to Software Engineeringtaylor/ICS_52_WQ04/ICS52WQ04-09.pdf · University of California, Irvine 4 What Do These Have in Common? First launch of space shuttle Airbus](https://reader031.vdocuments.net/reader031/viewer/2022022502/5aab55b57f8b9a8f498bc766/html5/thumbnails/12.jpg)
University of California, Irvine 12
Second Complication Complex data communications
– Electronic fund transfer Distributed processing
– Web search engine Stringent performance objectives
– Air traffic control system Complex processing
– Medical diagnosis system
Sometimes, the software system is extremelycomplicated making it tremendously difficult to perform QA
![Page 13: ICS 52: Introduction to Software Engineeringtaylor/ICS_52_WQ04/ICS52WQ04-09.pdf · University of California, Irvine 4 What Do These Have in Common? First launch of space shuttle Airbus](https://reader031.vdocuments.net/reader031/viewer/2022022502/5aab55b57f8b9a8f498bc766/html5/thumbnails/13.jpg)
University of California, Irvine 13
Third Complication
It is difficult to divide the particular responsibilitiesinvolved when performing quality assurance
ProjectManagement
DevelopmentGroup
Quality AssuranceGroup
![Page 14: ICS 52: Introduction to Software Engineeringtaylor/ICS_52_WQ04/ICS52WQ04-09.pdf · University of California, Irvine 4 What Do These Have in Common? First launch of space shuttle Airbus](https://reader031.vdocuments.net/reader031/viewer/2022022502/5aab55b57f8b9a8f498bc766/html5/thumbnails/14.jpg)
University of California, Irvine 14
Fourth Complication Quality assurance lays out the rules
– You will check in your code every day– You will comment your code– You will…
Quality assurance also uncovers the faults– Taps developers on their fingers– Creates image of “competition”
Quality assurance is viewed as cumbersome– “Just let me code”
Quality assurance has a negative connotation
![Page 15: ICS 52: Introduction to Software Engineeringtaylor/ICS_52_WQ04/ICS52WQ04-09.pdf · University of California, Irvine 4 What Do These Have in Common? First launch of space shuttle Airbus](https://reader031.vdocuments.net/reader031/viewer/2022022502/5aab55b57f8b9a8f498bc766/html5/thumbnails/15.jpg)
University of California, Irvine 15
Available Techniques
Formal program verificationStatic analysis of program properties
–Concurrent programs: deadlock,starvation, fairness
–Performance: min/max response timeCode reviews and inspectionsTesting
Most techniques are geared towards verifying correctness
![Page 16: ICS 52: Introduction to Software Engineeringtaylor/ICS_52_WQ04/ICS52WQ04-09.pdf · University of California, Irvine 4 What Do These Have in Common? First launch of space shuttle Airbus](https://reader031.vdocuments.net/reader031/viewer/2022022502/5aab55b57f8b9a8f498bc766/html5/thumbnails/16.jpg)
University of California, Irvine 16
V-Model of Development and Testing
Develop Acceptance TestsAcceptance Test Review
Requirements ReviewDevelop Requirements Execute System Tests
Develop Integration TestsIntegration Tests Review
Design ReviewDesign Execute Integration Tests
Develop Unit TestsUnit Tests Review
Code ReviewCode Execute Unit Tests
![Page 17: ICS 52: Introduction to Software Engineeringtaylor/ICS_52_WQ04/ICS52WQ04-09.pdf · University of California, Irvine 4 What Do These Have in Common? First launch of space shuttle Airbus](https://reader031.vdocuments.net/reader031/viewer/2022022502/5aab55b57f8b9a8f498bc766/html5/thumbnails/17.jpg)
University of California, Irvine 17
Implementation/Testing Interaction
Implementation(previous lecture)
Testing(this lecture)
![Page 18: ICS 52: Introduction to Software Engineeringtaylor/ICS_52_WQ04/ICS52WQ04-09.pdf · University of California, Irvine 4 What Do These Have in Common? First launch of space shuttle Airbus](https://reader031.vdocuments.net/reader031/viewer/2022022502/5aab55b57f8b9a8f498bc766/html5/thumbnails/18.jpg)
University of California, Irvine 18
Testing
Exercise a module, collection of modules, or system– Use predetermined inputs (“test case”)– Capture actual outputs– Compare actual outputs to expected outputs
Actual outputs equal to expected outputs test case succeeds
Actual outputs unequal to expected outputs test case fails
![Page 19: ICS 52: Introduction to Software Engineeringtaylor/ICS_52_WQ04/ICS52WQ04-09.pdf · University of California, Irvine 4 What Do These Have in Common? First launch of space shuttle Airbus](https://reader031.vdocuments.net/reader031/viewer/2022022502/5aab55b57f8b9a8f498bc766/html5/thumbnails/19.jpg)
University of California, Irvine 19
Testing Terminology Failure
– Incorrect or unexpected output– Symptom of a fault
Fault– Invalid execution state– Symptom of an error– May or may not produce a failure
Error– Defect or anomaly in source code– Commonly referred to as a “bug”– May or may not produce a fault
![Page 20: ICS 52: Introduction to Software Engineeringtaylor/ICS_52_WQ04/ICS52WQ04-09.pdf · University of California, Irvine 4 What Do These Have in Common? First launch of space shuttle Airbus](https://reader031.vdocuments.net/reader031/viewer/2022022502/5aab55b57f8b9a8f498bc766/html5/thumbnails/20.jpg)
University of California, Irvine 20
Testing Goals Reveal failures/faults/errors Locate failures/faults/errors Show system correctness
– Within the limits of optimistic inaccuracy Improve confidence that the system performs as specified (verification) Improve confidence that the system performs as desired (validation)
Program testing can be used to show the presenceof bugs, but never to show their absence [Dijkstra]
![Page 21: ICS 52: Introduction to Software Engineeringtaylor/ICS_52_WQ04/ICS52WQ04-09.pdf · University of California, Irvine 4 What Do These Have in Common? First launch of space shuttle Airbus](https://reader031.vdocuments.net/reader031/viewer/2022022502/5aab55b57f8b9a8f498bc766/html5/thumbnails/21.jpg)
University of California, Irvine 21
Levels of Testing Unit testing
– Testing of a single code unit– Requires use of test drivers
Integration testing– Testing of interfaces among integrated units
» Incremental» “Big bang”
– Often requires test drivers and test stubs Acceptance testing
– Testing of complete system for satisfaction of requirements
![Page 22: ICS 52: Introduction to Software Engineeringtaylor/ICS_52_WQ04/ICS52WQ04-09.pdf · University of California, Irvine 4 What Do These Have in Common? First launch of space shuttle Airbus](https://reader031.vdocuments.net/reader031/viewer/2022022502/5aab55b57f8b9a8f498bc766/html5/thumbnails/22.jpg)
University of California, Irvine 22
Test Tasks Devise test cases
– Target specific areas of the system– Create specific inputs– Create expected outputs
Choose test cases– Not all need to be run all the time
» Regression testing Run test cases
– Can be labor intensive
All in a systematic, repeatable, and accurate manner
![Page 23: ICS 52: Introduction to Software Engineeringtaylor/ICS_52_WQ04/ICS52WQ04-09.pdf · University of California, Irvine 4 What Do These Have in Common? First launch of space shuttle Airbus](https://reader031.vdocuments.net/reader031/viewer/2022022502/5aab55b57f8b9a8f498bc766/html5/thumbnails/23.jpg)
University of California, Irvine 23
Two Approaches White box testing
– Structural testing– Test cases designed, selected, and ran based on structure of the code– Scale: tests the nitty-gritty– Drawbacks: need access to source
Black box testing– Specification-based testing– Test cases designed, selected, and ran based on specifications– Scale: tests the overall system behavior– Drawback: ( less systematic) less thorough
![Page 24: ICS 52: Introduction to Software Engineeringtaylor/ICS_52_WQ04/ICS52WQ04-09.pdf · University of California, Irvine 4 What Do These Have in Common? First launch of space shuttle Airbus](https://reader031.vdocuments.net/reader031/viewer/2022022502/5aab55b57f8b9a8f498bc766/html5/thumbnails/24.jpg)
University of California, Irvine 24
Test Oracles Provide a mechanism for deciding whether a test case execution succeeds
or fails Critical to testing
– Used in white box testing– Used in black box testing
Difficult to automate– Typically relies on humans– Typically relies on human intuition– Formal specifications may help
![Page 25: ICS 52: Introduction to Software Engineeringtaylor/ICS_52_WQ04/ICS52WQ04-09.pdf · University of California, Irvine 4 What Do These Have in Common? First launch of space shuttle Airbus](https://reader031.vdocuments.net/reader031/viewer/2022022502/5aab55b57f8b9a8f498bc766/html5/thumbnails/25.jpg)
University of California, Irvine 25
Example Your test shows cos(0.5) = 0.8775825619 You have to decide whether this answer is correct? You need an oracle
– Draw a triangle and measure the sides– Look up cosine of 0.5 in a book– Compute the value using Taylor series expansion– Check the answer with your desk calculator
![Page 26: ICS 52: Introduction to Software Engineeringtaylor/ICS_52_WQ04/ICS52WQ04-09.pdf · University of California, Irvine 4 What Do These Have in Common? First launch of space shuttle Airbus](https://reader031.vdocuments.net/reader031/viewer/2022022502/5aab55b57f8b9a8f498bc766/html5/thumbnails/26.jpg)
University of California, Irvine 26
Use the Principles — Even in Testing Rigor and formality Separation of concerns
– Modularity– Abstraction
Anticipation of change Generality Incrementality