Software Quality AssuranceSoftware Quality AssuranceActivitiesActivities

Standard DefinitionStandard Definition

Software Quality - Conformance to: 1. explicitly stated functional and performance

requirements,

2. explicitly documented development standards, and

3. implicit characteristics that are expected of all professionally developed software.

The Big Question

How do we assure quality?How do we assure quality?

SQA Activities

1. Pre-Project Activities

2. Development and Maintenance Activities

3. Error Reduction Infrastructure

4. SQ Management Components

5. SQA System Assessment

6. Human Components

Software Quality Assurance by Galin

SQA Activities

1. Pre-Project Activities1. Contract Review

2. Development Plan

3. Quality Plan

2. Development and Maintenance Activities

3. Error Reduction Infrastructure

4. SQ Management Components

5. SQA System Assessment

6. Human Components

Contract Review ProcessContract Review Process

RFP

1st Draft

Revisions

Final ContractFinal Contract

SOW

Contract

Reality Check...

Q: Why should the software geeks worry about the contract?

A: Because the software team must do the work and assure the product's quality. loosely defined requirements unrealistic budgets unrealistic schedules

A: Contract review is required by ISO 9001

What to look for in 1What to look for in 1stst Draft Draft customer reqs clarified and documented?

alternative approaches examined?

risks identified?

costs and time estimates reasonable?

both customer and creator have capacity?

subcontractor participation clear?

proprietary rights?

relationship between customer and creator specified? Galin section 5.3

Development PlanDevelopment Planthat Includes Quality Assurance

RequirementsSpecification

Design

Coding

UnitTesting

Installation &Training

IntegrationTesting

Maintenance

Review the SRS

Design Reviews

CodingStandards

Validation

Co

nfig

uratio

n C

on

trol

Test Proceduresand Tolerances

Do

cum

entatio

n

Defect T

racking

Components of the Quality Plan Quality Goals

hopefully the SRS is some help

Review Activities schedule, type of reviews, scope, responsible persons

Software Test Plan type of unit tests and coverage, integration plan

Acceptance Tests for Sub-contract Software Configuration Tools and Procedures

Galin section 6.3

SQA Activities

1. Pre-Project Activities

2. Development and Maintenance Activities1. Formal Technical Reviews

2. Code Inspections

3. Unit Testing, Integration Test, etc. - earlier lecture

3. Error Reduction Infrastructure

4. SQ Management Components

5. SQA System Assessment

6. Human Components

Types of EvaluationsTypes of Evaluations Formal Reviews

conducted by senior personnel or outside experts uncover potential problems

Inspections and Walkthroughs done by peers detect problems, adherence to standards, etc.

Verification Unit Test Integration Test Usability Test

SQA Activities

1. Pre-Project Activities

2. Development and Maintenance Activities

3. Error Reduction Infrastructure1. work procedures - more details in a later lecture

2. templates and checklists - later lecture

3. staff training - at end of course

4. preventive actions - later lecture

5. configuration control - earlier lecture

6. document control

4. SQ Management Components

5. SQA System Assessment

6. Human Components

DefinitionsDefinitions

procedure - general methodology

work instructions - details of how to implement those procedures for this project

template - a standard Table of Contents

checklist - list of items that are part of an activity

Example Types of Work InstructionsExample Types of Work Instructions

audit process for subcontractors

design documentation templates

C++ programming instructions

coordination and cooperation with the customer

follow-up of beta site reporting

monthly progress reporting

etc...

Galin: page 316

SQA Activities

1. Pre-Project Activities

2. Development and Maintenance Activities

3. Error Reduction Infrastructure

4. QA Management Components Project Progress

schedules, budgets, risk analysis, … - earlier lectures Quality Metrics - earlier lecture Quality Costs

5. SQA System Assessment

6. Human Components

Statistical SQAStatistical SQA1. Info about defects are collected and categorized.

2. Defects are traced to their root cause.

3. Isolate the vital 20%

4. Move to correct problems that caused defects.

Pareto Principle: 80% of defects can be traced to 20% of causes.

SSQA is similar to"Six Sigma"

How much SQA is cost effective?

Software Quality

Costs

InitialCost of SQA

Cost of Failure

SQA+

Failure

Optimal Quality Level

EventualEventualCost of SQACost of SQA

Real NumbersCost of Software Quality for 15 Projects at Raytheon’s Equipment Division

0

10

20

30

40

50

60

70

Pe

rce

nta

ge

of to

tal p

roje

ct co

st

Year

CMM level 3Start of intiative CMM level 1

TCoSQ

PreventionRework

Appraisal

Cost ofConformance

Rework

87 88 89 90 91 92 93 94 95 96

htt

p:/

/ww

w2

.um

ass

d.e

du

/sw

pi/c

ost

mo

de

ling

/pa

pe

rs/s

coq

pa

p1

.do

c

What are the SQA Costs Contract Reviews SRS Reviews Design Reviews Code Walkthrough Checklists

creating the checklists training people to use the checklists filling out the checklists reviewing the checklists data auditing the checklist process

even more SQA Costs Tools o' Plenty

progress tracking tool estimation and scheduling tools testing tools

Metrics time spent creating forms and gathering the data time spent analyzing performance data

Yadda yadda yadda etc etc etc

but don't forget to ask

What are the costs of not conducting SQA? inaccurate proposal => inaccurate time

estimate => unhappy customer poor SRS => expensive and time

consuming changes to design etc etc etc

SQA Activities

1. Pre-Project Activities

2. Development and Maintenance Activities

3. Error Reduction Infrastructure

4. SQ Management Components

5. SQA System Assessment Quality Management Standards

SEI CMM ISO 9001

6. Human Components

Details o'plenty at end of semester

SQA Activities

1. Pre-Project Activities

2. Development and Maintenance Activities

3. Error Reduction Infrastructure

4. SQ Management Components

5. SQA System Assessment

6. Human Components Management SQA Unit

The SQA GroupThe SQA Group Participates in the development of the project's software

process description. Reviews software engineering activities to verify compliance

with the defined process. Audits designated software work products to verify compliance

with those defined as part of the software process. Ensures that deviations in software work and work products are

documented and handled according to a documented procedure.

Records any noncompliance and reports to senior management.

Coordinates and controls change. Helps collect and analyze metrics.

SummarySummary

"Quality" includes explicit and implied properties as well as the development process.

Reviews are a key part of SQA.

An SQA Group is highly recommended.

QA must be part of the plan and the schedule.

SSQA is a commonly used formal approach to process improvement.