software quality assurance1 lecture notes m. d. dykton 21 april 2003 university of maryland...

Software Quality Assurance 1

Software Quality AssuranceLecture Notes

M. D. Dykton21 April 2003

University of Maryland Baltimore CountyCS 345 Software Engineering

Software Quality Assurance


Outline

• What is software quality?

• What is software quality assurance?

• How do you create software quality?


Software Quality – Why Bother?

• Importance of Software– Ever increasing levels of system functionality are

embedded in software, not hardware• For complex, software-intensive systems

– 1970’s…estimated functionality 20% software/80% hardware– 2000’s…estimated functionality 80%+ software and growing

– Size, complexity and criticality of software is growing rapidly

• Quality is not easily produced by accident anymore

– Quality of work/reputation of organizations frequently depends upon producing quality software


Quality – What is it?

• The Institute of Electrical and Electronics Engineers' (IEEE) Standard Glossary of Software Engineering Terminology defines quality as “(1) the degree to which a system, component, or process meets specified requirements, and (2) customer or user needs or expectations.“

• Kitchenham states quality is "hard to define, impossible to measure, easy to recognize."


• Multi-faceted concept, not simply defined• Fundamentally, quality implies fitness for intended use

– In part, implies software meets a specification or set of requirements

– Also includes many additional attributes which may or may not be explicitly captured in a specification:

Software Quality

• Correctness• Safety• Security• Reliability• Resilience• Robustness• Efficiency• Complexity

• Reusability• Learnability• Usability• Testability• Understandability• Modifiability• Portability• Maintainability


Software Quality Assurance

Three Principal Elements of SQA:

• Software Quality Assurance Policy Establishment• Software Quality Planning• Software Quality Control

Quality Management System• Policy

Quality Guidance• Standards• Practices & Procedures

Quality Controls• Process enforcement

Pro

cess Refin

emen

t

Project 1QA Plan

Project 2QA Plan

Project nQA Plan

…

Feed

backProject-specified plans

developed from QA guidance


Software Quality Assurance Policy Establishment

• High-level organization-based statement of software quality policy– Goals– Rationale– Relationship to other QA policies & standards– Specific policies and procedures– Roles and responsibilities– Organizational changes– Establishment of a Quality Assurance

Management System


Software Quality Assurance Organization – JHU/APL Example

Steering Committee

Software Engineering Process Group (SEPG)

Software Quality Assurance Manager

(SQAM)

Quality Councilor Dept. SEPG

Mini-SQAMs or QA Teams

Laboratory-level management oversight, sets policy

Laboratory-level group that defines policy Implementation and process improvement

Department-level group that defines detailed policies, processes and procedures

Oversees and enforces department-level policies,processes and procedures; handles process tailoring, support infrastructure mgmt, staff training

Group-level QA support


SQA Process Improvement

• Software Engineering Institute (SEI) Software Capability Maturity Model (SW-CMM) – classifies five levels of process

– Level 1 Initial – Ad hoc, unpredictable software process

• Herding Cats….

– Level 2 Repeatable – Focus is on effective software project management processes for cost, schedule and functionality

• Project planning• Project tracking and oversight• Requirements management• Quality Assurance• Configuration management• Subcontract management


SQA Process Improvement(continued)

– Level 3 Defined – Software management and engineering processes are standardized and documented

• Emphasis is on formal procedures to ensure defined process is followed– Organizational product definition and focus– Software product engineering– Integrated software management– Reviews– Intergroup coordination

– Level 4 Managed – Measures of the software process and product quality collected, and software process and products are quantitatively understood and controlled

• Software quality management• Quantitative process management


SQA Process Improvement(continued)

– Level 5 Optimizing – Level 4 process coupled with planned and funded process improvement program

• Process change management

• Technology change management

• Defect prevention


Software Quality Planning

• Software Policy & Procedure considerations

• Risk management

• Sound software management and engineering practices

• Measurement program


• Risk management is an integral part of the process to develop software quality

– Quality is not free…QA activities costs time and money…trade-offs are necessary

– Quality assurance activities are risk reduction efforts– A “one-size-fits-all” quality assurance plan is rarely feasible

• QA process tailoring is necessary• Risk management is an approach to intelligent process tailoring

• Risk Management

– Risk Identification– Risk Analysis and Assessment– Risk Planning & Mitigation– Risk Tracking

SQA and Risk Management


• SEI Software Risk Taxonomy

• Use risk taxonomy as a checklist to identify risks and potential risk mitigation activities (i.e., QA process and products)

SQA and Risk Management(continued)

Product Engineering

• Requirements• Design• Code and Unit Test• Integration and Test• Engineering Specialties

Development Environment

• Development Process• Development System• Management Process• Management Methods• Work Environment

Program Constraints

• Resources• Contract• Project Interfaces

See Managing Risk: Methods for Software Systems Development, Elaine M. Hall, Addison Wesley Longman, Inc., 1998, p.76, Table 4.1


Software Quality Control

• Process Enforcement– Independent agent (i.e., Software Quality

Assurance manager/team)– Quantitative Metrics– Data Collection / Archiving– Review / Audit (process and products)– Authority to act

• Process Assessment• Process Improvement Feedback


• Necessary, underused and hard– Forms one of the pillars of “prove-able” quality– Software and its development is complex and multi-

dimensional, hard to understand and measure

• Attributes of “Good” Metrics– Useful– Meaningful– Quantifiable– Measurable– Repeatable

• Metrics should be linked to risk mitigation activities and T&E program (both product and process)

Software Metrics


Software Metrics(continued - Example)

Entities Attributes

Internal External

Products

Design Size, reuse, modularity, functionality,…

Quality, complexity, maintainability, …

Code Size, reuse, algorithmic complexity

Reliability, usability, ….

Processes

Detailed Design Time, effort, number of specification faults found, …

Cost, cost-effectiveness, …

Testing Time, effort, number of bugs, … Cost, cost-effectiveness, stability, …

Resources

Personnel Age, price, skills,… Productivity

Software Price, size, functionality, … Usability, reliability, …

Hardware Price, speed, memory size, … Reliability, …

See Software Metrics: A Rigorous & Practical Approach, Norman E. Fenton and Shari Lawrence Pfleeger, PWS Publishing Company, 1997, p.76.


Software Metrics(continued)

• Scope of software metrics – process, products or resources– Project management– Cost and level-of-effort estimation– Productivity measures– Quality attributes– Reliability measures– Performance – Defect tracking– Structural and complexity metrics– Software engineering process metrics


Summary

• Software Quality is important – software controls life-and-death decisions, has enormous economic consequences, affects reputations, etc.

• An independent Quality Management System is vital component of an effort to produce quality software - Quality needs a “stakeholder”

• “Complete” requirements determination is a critical first step, include both overt and implied requirements

• Quality assurance is a product of effective risk management – Tailor project management and software engineering practices to

mitigate quality-related risks

• Measurement program – demonstrate progress toward quality objectives using metrics, testing and other measurements


References

1. Managing Risk: Methods for Software Systems Development, Elaine M. Hall, Addison Wesley Longman, Inc., 1998.

2. Software Metrics: A Rigorous & Practical Approach, Norman E. Fenton and Shari Lawrence Pfleeger, PWS Publishing Company, 1997.

3. Software Quality, Theory and Management. Gillies, Alan C., International Thomson, Computer Press, 1997.

4. Risk Mangement Processes for Software Engineering Models, Marian Myerson, Artech House, 1996.5. Software Engineering, Ian Sommerville, 5th Edition, Addison-Wesley Publishing Company, 1996.6. Software Quality Assurance: A Practical Approach, Ernest Wallmuller, Prentice Hall International (UK)

Ltd., 1994.7. ISO 9001 and Software Quality Assurance, Darrel Ince, McGraw-Hill Book Company, 1994.8. Handbook of Software Quality Assurance, Edited by G. Gordon Schulmeyer and James I. McManus,

2nd Edition, Van Nostrand Reinhold Company, 1992.9. Managing the Software Process, Watts S. Humphrey, Addison-Wesley Publishing Company,

1989/1990.10. Applications Strategies for Risk Analysis, McGraw-Hill Software Engineering Series, Robert N.

Charette, Intertext Publications, 1990.11. Software Product Assurance: Techniques for Reducing Software Risk, William L. Bryan and Stanley

G. Siegel, Elsevier Science Publishing Co., Inc., 1988.12. Characteristics of Software Quality, Barry W. Boehm, John R. Brown, Hans Kaspar, Myron Lipow,

Gordon J. MacLeod and Michael J. Merritt, North-Holland Publishing Company, 197813. Kitchenham, Barbara, and Shari Lawrence Pfleeger. "Software Quality: The Elusive Target." IEEE

Software 13, 1, Jan. 1996: 12-21.

software quality assurance1 lecture notes m. d. dykton 21 april 2003 university of maryland...

Documents