Software Quality Assurance

SEN-269 : Software EngineeringTazeen Muzammil

What is Quality• The term 'quality' is often used in a vague,

blurred way. If someone talks about 'working on quality', they may simply mean activities designed to improve the organization and its services. 

• Quality is about:– Knowing what you want to do and how you want

to do it. Learning from what you do.– Using what you learn to develop your organization

and its services. – Seeking to achieve continuous improvement. – Satisfying your stakeholders - those different

people and groups with an interest in your organization.

Why Quality Matters?

Increasing complexity of software productIncreasing demand for software productsIncreasing lifetime of software productsIncreasing use of "Safety Critical" or "Mission Critical" software

3

Software Quality Software Quality is defined as Conformance to:

1. explicitly stated functional and performance requirements,

2. explicitly documented development standards,3. and implicit characteristics that are expected

of all professionally developed software.

In the context of software engineering, software quality measures how well software is designed (quality of design), and how well the software conforms to that design (quality of conformance).

4

What is Quality Control? Quality Control is the process of developing systems that ensures that products or services are designed and produced to meet or exceed customer requirements, and expectations. Key concept of quality control:

Is to ensure that the products, services, or processes provided meet specific requirements and are dependable, satisfactory, and fiscally sound.Implementation approaches:

- Fully automated - Entirely manual- Combination of automated tools and human interactions

What is Software Quality Assurance?

Software Quality Assurance (SQA) is defined as a planned and systematic approach to the evaluation of the quality of the software product standards, processes, and procedures.

SQA includes the process of assuring that standards and procedures are established and are followed throughout the software acquisition life cycle.

Goal --> provide management with the necessary data about product quality.--> gain the insight and confidence of product quality

Software Quality Assurance

The most popular tool used to determine quality assurance is the Shewhart Cycle, developed by Dr. W. Edwards Deming. This cycle for quality assurance consists of four steps: Plan, Do, Check, and Act. These steps are commonly abbreviated as PDCA.ISO 9001:2000 was developed in line with the long-used Shewhart cycle of process or project management; Plan, Do, Check, Act, referred to in ISO 9004 as Plan, Implement, Check, Review.

7

Plan, Do, Check, Act (PDCA)

Plan; adhering to the company mission and vision, the management sets objectives based on information gathered through the quality system (customer feedback, management reports, etc.) Do; the product is produced or the service delivered in accordance with the customers requirements. Check; was it done right? Obtain information from the customer and the organization to measure and monitor performance. Act; “freeze” successful activities and improve less successful ones. This step is the key to continual improvement.

8

9

10

11

12

13

14

Quality Consulting

15

What is Software Quality Engineering?

Software Quality Engineering is composed of two primary activities

1. Process level quality is normally called quality assurance.

2. Process level quality establishes the techniques, procedures, and tools that help promote, encourage, facilitate, and create a software development environment in which efficient, optimized, acceptable, and as fault-free as possible software code is produced.

Product level quality

1. Product oriented quality is normally called testing

2. Product level quality focuses on ensuring that the software delivered is as error-free as possible, functionally sound, and meets or exceeds the real user’s needs. Testing is normally done for finding errors in order to get rid of them.

Process level quality

Cost of QualityCost of quality includes all costs incurred in the pursuit of quality or in performing quality related activitiesQuality cost includes:Prevention cost:

quality planningformal technical reviewstesting equipment

Training Appraisal cost: Include activities to gain insight into product condition the “first time through” each process.

in-process inspectionequipment calibration and maintenancetesting

17

Cost of QualityFailure cost: internal failure cost: Internal failure cost are incurred when we detect a defect in our product prior to shipment

rework

Repair

failure mode analysis

external failure cost: External failure costs are is associated with defects found after the product has been shipped to customer.

complaint resolution

product return and replacement

help line support

warranty work

The relative cost to find and repair a defect increases dramatically as we go from prevention to detection to internal failure to external failure cost. (See fig: Relative cost of correcting an error)

18

SQA ActivitiesPrepare SQA plan for the project.Participate in the development of the project's software process description.Review software engineering activities to verify compliance with the defined software process.Audit designated software work products to verify compliance with those defined as part of the software process.Ensure that any deviations in software or work products are documented and handled according to a documented procedure.Record any evidence of noncompliance and reports them to management.

19

The plan identifies:•Evaluation to be performed.•Audits and reviews to be performed.•Standards that are applicable to the project.•Procedures for error reporting and tracking.•Documents to be produced by the SQA group.•Amount of feedback provided to the software project team.

The software team selects a process. The SQA group reviews the process description for compliance with organizational policy, internal software standards, externally imposed Standards (E.g. ISO-9001), and other parts of the software project plan.

The SQA group identifies, documents, and tracks deviations from the process and verifies that corrections have beenmade.The SQA group identifies, documents, and tracks deviations from the process and verifies that corrections have beenMade; and periodically reports the result of its work to the project manager.Deviation may be encountered in the project plan, process description, applicable standards, or technical work products.

Non-compliance items are tracked until they are resolved.

Software ReviewsObjective: find errors early before they become defects.Excellent ROI: empirical data show that design activities introduce 50-65 % of errors, but formal reviews can detect up-to 75 % of those.Reviews are made to

Uncover errors.Ensure that software meets requirements and standards.Ensure that software and the process is manageable.

Using formal technical reviews (walkthroughs or inspections) is an effective means for improving software quality.

20

21

Review Roles

Presenter (designer/producer).Coordinator (not person who hires/fires).Recorder

records events of meetingbuilds paper trail

Reviewers maintenance oracle (vision)standards beareruser representativeothers

Reviews (Walkthroughs)

In the review process, the producers present a product (part of the software, e.g. a code module or a design model) which is then reviewed by reviewers.

Records should be kept about the reviews. This includes lists of issues discovered.

A follow-up procedure must be in place to correct issues.

22

Defect Amplification Models

Defect amplification models can be used to illustrate the importance of detecting defects in an early stage. They are a simple mathematical model for describing how defects are propagated through the various steps in the SD process.

23

24

Defect amplification,no review

Defect amplification,Reviews conducted

Defect CausesIES: Incomplete and erroneous specification. Example: customer "forgets" an interface to an existing system.MCC: Misinterpretation of customer communication. Example: Customer means one thing, analyst understands something else.IDS: Intentional derivation from specification. Example: Analyst or programmer things he knows better.VPS: Violation of Programming Standards. Example: Tools are used which rely on naming patterns (e.g. bean tools), but the code does not adhere to those patterns.EDR: Error in Data Representation. Example: wrong container types are used, e.g. containers which support duplicates, but this is not permitted in the business logic. ICI: Inconsistant Component Interface. Example: component interface does not support all parameters needed to customize the component.

25

Defect CausesEDL: Error in design logic. Example: wrong algorithm implemented.IET: Incomplete and erroneous testing. Example: there are errors in the test cases itself, and the code is correct w.r.t. the incorrect test cases.IID: Inaccurate and incomplete documentation. Example: the meaning of an API method is incorrectly (or not) documented, and programmers in a team using this component pass the wrong parameter to this component.PLT: Error in PL translation of design. Example: classes design to be abstract are implemented as concrete classes and instantiated.HCI: Ambiguous or inconsistent human computer interface. Example: different meaning of the same key stroke or menu function in different contexts.MIS miscellaneous. 26

27

Formal Technical ReviewsInvolves 3 to 5 people (including reviewers)Advance preparation (no more than 2 hours per person) requiredDuration of review meeting should be less than 2 hoursFocus of review is on a discrete work productReview leader organizes the review meeting at the producer's request.

28

Formal Technical Reviews Reviewers ask questions that enable the producer to discover his or her own error (the product is under review not the producer) Producer of the work product walks the reviewers through the productRecorder writes down any significant issues raised during the reviewReviewers decide to accept or reject the work product and whether to require additional reviews of product or not.

29

Why do peer reviews?

To improve quality.Catches 80% of all errors if done properly.Catches both coding errors and design errors.Enforce the spirit of any organization standards.Training and insurance.

30

Review Guidelines

Keep it short (< 30 minutes).Don’t schedule two in a row.Don’t review product fragments.Use standards to avoid style disagreements.Let the coordinator run the meeting and maintain order.

31

SQA PlanManagement section

describes the place of SQA in the structure of the organization

Documentation sectiondescribes each work product produced as part of the software process

Standards, practices, and conventions section

lists all applicable standards/practices applied during the software process and any metrics to be collected as part of the software engineering work

32

SQA Plan

Reviews and audits sectionprovides an overview of the approach used in the reviews and audits to be conducted during the project

Test sectionreferences the test plan and procedure document and defines test record keeping requirements

33

SQA PlanProblem reporting and corrective action section

defines procedures for reporting, tracking, and resolving errors or defects, identifies organizational responsibilities for these activities

Othertools, SQA methods, change control, record keeping, training, and risk management