Download - Cleanroom Software Engineering
CLEANROOM SOFTWARE
ENGINEERING
Andy Moyer
Cleanroom Software Engineering What is it? Goals Properties of Cleanroom Cleanroom Technologies Case Studies Critiques
Cleanroom Software Engineering: What is it? Set of principles and practices for the
specification, development, and certification of software-intensive systems.
Based on the work of Harlan D. Mills who was an employee of IBM in early 1980s.Influenced by Dijkstras structured programming,
Nicholas Wirth on stepwise refinement, and David Parnas on modular design
Methods based on two principles:Programs are rules for mathematical functionsSoftware testing is sampling
Where the Idea Came From
The cleanroom model follows the ideas of manufacturing semiconductors in a “cleanroom” environment.
Goals of the Cleanroom Process
Main goal: Achieve or approach zero defects
Other goals:Prevent defectsCreate correct, verifiable softwareIncremental development
Properties of Cleanroom
Spend a lot of time and money "up-front" preventing defects
Use statistical methods to ensure quality Formally state and “prove” requirement
needs Specification, development, and
certification may be done by separate teams depending on the size of the project
Cleanroom Technologies
1. Incremental development under Statistical Process Control
2. Function-Based Specification, Design, and Verification
3. Correctness Verification
4. Statistical Testing and Software Verification
Incremental Development Under Statistical Process Control Used to reduce the risk associated with
managing large systems by focusing on smaller, manageable subsystems of increments.
Each increment is developed and tested separate from the whole system before integrating into the whole system.
As increments are added, the whole system is tested.
Some amount of end-user function used to obtain customer confirmation or clarification
Incremental Development Under Statistical Process Control
Increment should have these properties:Externally executableContain all functionality of prior increments
Benefits:Specification, design, and certification
activities may be performed oftenTimely feedback can be obtained from
customerIntellectual control over a system
Cleanroom Technologies
1. Incremental development under Statistical Process Control
2. Function-Based Specification, Design, and Verification
3. Correctness verification
4. Statistical Testing and Software Verification
Function-Based Specification, Design, and Verification
Objectives:Expand the specification into
implementation via small stepsMaintain intellectual control by designing
data and control structures at appropriate levels of abstraction
The Three Boxes:Black BoxState BoxClear Box
Function-Based Specification, Design, and Verification
The Box StructuresBegins with an external view – Black BoxTransformed into a state machine – State
BoxFully developed into a procedure – Clear
Box
The Black Box
A view of an object that hides data implementation and process implementation. It describes how a system responds to stimuli, usually in a formal specification language.
No internal state is looked at Our good friend – Z (Zed)
The State Box
A view of an object that shows data implementation but hides process implementation. It describes how "state" information is transformed.
Derived from the black box – first step to implementation
This is represented as a finite state machine
The Clear Box
A view of an object that shows both data implementation and process implementation. The goal is to stepwise refine procedures and to prove them correct.
Derived from the state box May introduce new black boxes defining
major operations Creates the required responses (outputs)
to stimuli (inputs)
Stepwise Refinement of Specifications to Code Steps:
1. Expand clear box into lower level designs
2. Verify the correctness of the expanded designs
3. Expand low level designs into code
Cleanroom Technologies
1. Incremental development under Statistical Process Control
2. Function-Based Specification, Design, and Verification
3. Correctness verification
4. Statistical Testing and Software Verification
Correctness Verification The primary debugging process for
Cleanroom Each box structure subject to correctness
verification Objectives:
Determine that the box structures correctly implements design
Remove any errors that were introduced during development
Completely review the code for completeness, consistency, and correctness
Correctness Verification
So what is verified?If the box structure behaves as definedCorrectness of each refinementResponse mapping defined in one step is
preserved
Cleanroom Technologies
1. Incremental development process model
2. Stepwise refinement of specifications to code
3. Correctness verification of developed code
4. Statistical Testing and Software Verification
Statistical Testing and Software Verification
Each increment of compilable functionality is statistically tested to:Ensure that it meets the quality standards
defined by the development organizationCertify a level of reliability that the product
will deliver in the fieldProvide feedback for quality control and
process improvement
Statistical Testing and Software Verification
Steps taken:1. Plan the test
○ Stratification plan which describes each layer to be developed
○ The sampling plan○ A description of which physical testing
resources are to be devoted to each increment and each stratum
2. Develop usage models required by the test plan
3. Develop and validate the usage distribution
Example Usage Model
NASA Case Study
In March of 1990 NASA preformed a case study of SEL Cleanroom versus their typical process
Each group applied the process to the same project – a Coarse/Fine Attitude Determination Subsystem of the Upper Atmosphere Research Satellite.
Completed system contained about 34,000 Source Lines of Code (SLOC) of primarily FORTRAN
NASA Case Study-SEL Cleanroom Lifecycle
NASA Case Study-Experience Comparison
NASA Case Study
Testers and developers are on completely separate teams
Developers have no access to the mainframe computer for compilation and testing
Developers rely on code reading instead of unit testing to verify correctness
Testers use a statistical testing approach
NASA Case Study - Results
NASA Case Study - Results Failure Rate:
Standard SEL process – 6 errors/KSLOCCleanroom SEL procees – 3.3
errors/KSLOC The inexperience of the cleanroom team
had no effect on the outcome Impact of unstable requirements
lessened by concentrated effort in team reviews
DoD Case Study
The STARS program is a US Department of Defense (DoD) reasearch and development program
Emphasizes:Process drivenRe-use basedIntegrated software engineering
environment
DoD Case Study Typical Process
Productivity measured at 121 loc/person month
Failure rate: 23~27 failures/KLOC
Cleanroom ProcessProductivity
measured at 559 loc/person month
Failure rate: 1 failure/KLOC
DoD Case Study
Important benefits:Improved ProductivityImproved qualityHigh staff morale
Critiques
In 1997, Beizer argued the elimination of unit testing is contradicting “known testing theory and common sense”
Also, is it possible to find bugs without compiling/running code?
References 1. Cleanroom Software Engineering. Retrieved
from University of Texas at Arlington website: http://www.uta.edu/cse/levine/fall99/cse5324/cr/clean/page1.html
2. DACS, The Data and Analysis Center for Software. Retrieved January 5, 2009 Found at: www.dacs.dtic.mil/databases/url/key.php? keycode=64
References 3. Green, Scott et al. (1990). "The
Cleanroom Case Study in the Software Engineering Laboratory: Project Description and Early Analysis". NASA. Retrieved from http://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/19910008271_1991008271.pdf
4. Prowell, Stacy et al. (1993). "Cleanroom Software Engineering: Technology and Process". Reading, MA: Addison Wesley Longman, Inc.