software quality assurance

SQA WorkshopVersion 1.0

By: B. M. Shahrier MajumderMy Profile: http://

www.linkedin.com/in/shahrier

http://www.linkedin.com/in/shahrier

http://www.linkedin.com/in/shahrier

SQA Workshop By: B. M. Shahrier Majumder 2

Workshop Contents Quality and Process

Concept Quality Models SQA Role Audit System Summery


Training Objectives

To prepare participants for effective implementation of SQA role in the organization

To provide some practice in the technique used


Logistics

Timings Flow of Course (lecture, exercises, Quiz) Course Material


Let Us Begin!!


Quality & Process Concepts


Quality

Fit for use

Conforms to the statement of requirement


Two Views of Quality

Producer view of quality

Customer view of quality

Quality Assurance closes the gap


Quality DefinitionOperationally, the word quality refers to products. A product is a quality product if it is defect free.

Producer View of Quality: The producer view of quality has these 4 characteristics. Doing the right thing, Doing it the right way, Doing it right the first time and Doing it on time without exceeding cost.

Customer View of Quality: Meeting requirements is a producer’s view of quality. This is the view of the organization responsibility for the project and process, and the products and services acquired, developed, and maintained by those processes.


Quality Gurus

DR. W. Edwards Deming Philips Corseby DR. Joseph Juran


Total Quality Management

A philosophy A set of guiding principle The foundation for a continuous improving

organization The application of quantitative methods

and human resources– To improve processes– To satisfy customers, now and later


Definition of a Process

A process is a vehicle of communication, specifying the methods used to produce a product or service. It is the set of activities that represent the way work is to be performed.

Procedure: the step-by-step method followed to ensure that standards are met.


Why Process are NeededFrom management perspective, process are needed to:

Explain to workers how to perform work tasks Transfer knowledge from more experienced to less experienced

workers Assure predictability of work activities so that approximately the

same deliverables will be produced with the same resources each time the process is followed

Establish a basic set of work tasks that can be continuously improved Provide a means for involving workers in improving quality,

productivity and customer satisfaction by having workers define and improve their own work process

Free management from their activities associated with “expediting work products” to send more time on activities such as planning, and customer & vendor interaction


Why Process are NeededFrom worker perspective, process are needed to:

Increase the probability that the deliverables produced will be the desired deliverables

Put workers in charge of their own destiny because they know the standards by which their work products will be evaluated

Enable workers to devote their creativity to improving the business instead of having to develop work processes to build products

Enable workers to better plan their workday because of the predictability resulting from work processes


Process ManagementProcess management is a PDCA cycle. Process management processes provide the framework from within which an organization can implement process management on a daily basis.

PLANProcess Inventory – 1Process Mapping – 2Process Planning – 3Enables process definition

CHECKProcess Measurement – 6

Enables process assessment

ACTProcess Improvement – 7

Enables process improvement

DOProcess Definition – 4Process Controls – 5Enables process execution


Quality Models


Industry Quality Models

There are many industry models available against which your organization can establish a baseline. Most commonly used models in the IT industry are:

ISO 9001:2000 CMMI


Quality System ElementsISO 9001

Management Responsibility Quality System Contract review Design Control Document & data control Purchasing Control of customer supplied

product Product identification &

traceability Process control Inspection & testing

Control of inspection, measuring & test equipment

Inspection & test status Control of non-conforming

product Corrective & preventive action Handling, storage, packaging,

presentation & delivery Control of quality records Internal quality audits Training Servicing Statistical techniques


ISO 9001:2000 Released in December, 2000 Consistency with PDCA cycle Based on eight quality management principles

1. Customer focus2. Leadership3. Involvement of people4. Process approach5. System approach to management6. Continual improvement7. Factual approach to decision making8. Mutually beneficial supplier relationships


ISO 9001:2000 Logical grouping of

clauses under the following heads:

1. Management Responsibility

2. Resource Management

3. Product realization4. Measure, Analysis,

Improvement


SEI CMMI Capability Maturity Model Integration (CMMI)

evaluates software process capability Used for

- Where are we today?- Where do we want to be?- How do we get there? (Planning)- Have we reached there? (Measurement)


Some Definitions Software Process

A set of activities, methods, practices, and transformations that people use to develop and maintain software and associated products (e.g. plans, design documents, code, test cases, user manual, etc.)

Software process capabilityDescribes that range of expected results that can be achieved by following a software process.

Software process performanceRepresents the actual results achieved by following a software process.


Some Definitions Software Process Maturity

- The extent to which a specific process is explicitly defined, managed, measured, controlled and effective- Implies a potential for growth in capability and indicates both the richness of an organization’s software process and the consistency with which it is applied in projects throughout the organization.

InstitutionalizationEntails building an infrastructure and a corporate culture that supports the methods, practices, and procedures of the business so that they endure after those who originally defined them have gone.


Some DefinitionsMaturity Level

- A well defined evolutionary plateau towards achieving a mature software process.- Each level provides a layer in the foundation for continuous process improvement.

Process AreaIdentifies a cluster of related activities that, when performed collectively, achieve a set of goals considered important for enhancing process capability


CMMI Levels

Level Process CharacteristicsInitial (1) Process is informal and adhocRepeatable (2) Project management practices are

institutionalizedDefined (3) Technical practices are integrated

with management practices and institutionalized

Managed (4) Product and process quantitatively controlled

Optimizing (5) Process improvement institutionalized


Quality Management

Setting Quality goals / policy / objectives

Building support for QualityPlanning QualityMeasuring qualityControlling Quality / Poor QualityImprove Quality


Process Orientation of Quality Quality has been defined in many different ways but

always to satisfy the ‘customers’ Quality can be measured Quality control detects errors Quality assurance prevent errors Processes determine the quality of the product Product can improve only if process improve

continuously Quality is every person’s responsibility It should be imbibed as a pert of day-to-day work


Definitions

Quality ControlThe operational techniques and activities that are used to fulfill requirements for quality.

Quality AssuranceAll those planned and systematic activities implemented within the quality system and demonstrated as needed to provide adequate confidence that an entity will fulfill requirements for quality.


QC vs QA

QC QA Product Process Reactive Proactive Line function Staff function Find defects Prevent defects


QC vs QA Examples

QC QA Walkthrough Quality Audit Testing Defining process Inspection Selection of tools Checkpoint review Training


Cost of QualityFailure Costs Project rework Overtime Maintenance costs Lost credibility Providing alternate service Lost management time Complaints, rebates &

damage claims Lost assets, opportunity Unrealized savings

Appraisal Costs Reviews Inspections TestingPrevention Costs Quality audit Planning quality

improvement Quality training Installation

- Project selection process- Planning database- Improved programming techniques


SQA Role


Quality of requirements / specification

Good planning Use of trained personnel Usage of pre-defined techniques Use of templates, checklists Through review Requirements sign off A good SRS is

- Unambiguous, complete, correct, verifiable- Helps customers describe what they want to obtain- Helps supplier understand what the customer wants


Quality of Design Good planning Use trained resources Choice of appropriate model, techniques and tools for design

- Top-down vs. Bottom-up approaches selected / mixed to get most suitable approach- Build in the attributes related to reuse of components, product attributes like scalability, interoperability, product performance and so on based on application requirement, clear interfaces

Use of standard templates Review checklist to ensure no major aspect is missed out Review of design documents

- Various specialists / review focus – e.g. optimization, technical feasibility- Traceability of design to requirements- Ensure consistency between low level design


For Better Code

Use coding standardsUse proper code samples and templatesPlan to ensure common libraries are availableConduct code reviewUse checklist for reviewGood planningTrain people


For Successful TestingPlan for the test strategy, test casesReview test plans, test conditions, etc.Have independent testing teamsTest the units before moving to integration

testingUse pre-defined forms for test scripts, test

logs, etc.Use processes for testing and defect

management


For Successful Post Testing Activities

Plan for the acceptance testing in the initial stages of the project

Clearly agree on the acceptance criteria with the customer

Have clear documentation of the product in the form of:- Installation manual- Maintenance manual- User manual

Review the manuals and test before delivery Include exception handling


Quality Pyramid

QualityAssurance

Measurement

Quality Control

Procedure

Standards

Management Policies / Plans

Assure Quality

Control Quality

Define Quality

ObjectiveFunction


Quality Management System

ProcessProceduresGuidelinesStandardsChecklistsFormatsTemplates


Value of Documentation

Assists for conformity to customer requirements and quality improvement

Provides appropriate training Enables repeatability & traceabilityProvision of objective evidenceEvaluate effectiveness of QMS


Audit System


Audit-Definition

Definition:A systematic, independent and documented process for obtaining audit evidence and evaluating it objectively to determine the extent to which audit criteria are fulfilled.

Audit Criteria:Set of policies, procedures or requirements used as reference

Audit Evidence:Records, statement of facts or other information which are relevant to the audit criteria


Purpose of Audits

Management ToolPositive and constructive process Identifies problem areasIncreases process complianceIncreases process effectiveness


Audits

NOT to be used to assign blameDoes NOT replace inspection / testing activities

Should NOT be used as a means to accept or reject products

CANNOT support an ineffective system


Types of Audits

First Party

Second Party

Third Party


First Party Audits

To check compliance with QMSTo find & correct system shortfalls To identify improvements of QMSTo enhance quality awarenessTo increase cross-department understanding

A requirement of ISO 9001


Second Party Audits

To evaluate potential suppliers / subcontractors

To keep an eye on suppliers / subcontractors

To help suppliers improve their QATo improve end products and servicesTo limit costs of external failure


Third Party Audits

To provide objective evidenceTo identify required improvements Part of certification processTo provide credibility to claims of quality


The Players

Auditor- A person who has the qualifications to

perform quality auditsClient

- A person or organization requesting the audit

Auditee- An organization to be audited


The Players in a First Party (Internal) Audit

Auditor- An employee (or sometimes a consultant) who is trained as an auditor and is independent of the

area audited

Client - The senior management of the organization

Auditee- The project / department / unit being audited


The Players in a Second Party Audit

Auditor- An employee (or sometimes a consultant) who is trained as an auditor and is representing a

customer or potential customer

Client - The senior management of the customer

organization

Auditee- The project / department / unit being audited


The Players in a Third Party Audit

Auditor- A recognize auditor belonging to a certifying body

Client - The senior management of the organization

Auditee- The project / department / unit and organization being audited


Basic Purpose

Objective Evidence

Does the Quality System meet the requirements of the relevant standard or contract?

Does the organization do what the QMS requires?

Is the QMS effective for the Organization’s business?


Objective Evidence

A factual statement that can be verified

Not based on opinion or preference Not based on emotion Based on actual observations & statements


Evidence – Quality System

Quality Manual referring to procedures Procedures covering the standard being followed

(ISO / CMMI) Departmental Handbooks Project proposals / Plans Instructions Policy and objectives Responsibilities and authorities


Evidence – Implementation Records

Review records Minutes of meeting Audit reports Testing records Delivery notes Training records


Evidence of Effectiveness

Records / results Measurements / metrics Milestone achievement Management review Customer feedback Timely corrective action Customer complaints


The Audit System

Annual Audit Planning

Audit Cycle Scheduling

Opening Meeting

Audit Investigation

Audit Reporting Corrective Actions


Prepare Long Term Audit Plan

Typically for the whole year Aspects to plan for:

- How many cycles (typically once every 2-3 months)- What units / departments / areas / projects will be covered in every cycle – this would depend on the status and importance of the unit / department and the extent of changes expected


For Every Cycle

Review and revise the list of auditee units / departments / projects

Nominate a lead auditor and audit team Make initial contact with auditees Finalize audit program


Review & Revise Auditee List

Review / revise the list of auditee units / departments / projects based on:- The extent of activities- Changes in structure, personnel, type of work- Findings of previous audit- Proposed changes in the projects


Nominate Audit Team To be done by EPG head or SQA lead or mutually agreed and

planned between the PM and SQA Identify “Lead Auditor” for the audit in case of IA across the

organization Identify all auditors of the audit

- Number of auditors- Assignment to auditees areas

Ensure availability of auditors for:- Preparation, interviews, reporting, follow-up (approx 4-5 hours per project / support group)

Provide training to untrained auditors Check whether the auditee and auditor are independent Confirm that the auditee and auditor have no “issues” that may

impact objectively Set up initial contact between auditor and auditee


Lead Auditor Responsibilities

Manage the team Assist in team selection Preparation of program / checklist Quality control over the team’s work Interfacing with auditees management Preparation / submission of audit report Conduct audit interviews


Auditor Responsibilities

Communicate audit requirements Be active and efficient Document observations Report results Verify corrective action effectiveness Remain with scope Support other team members


Auditee Responsibilities

Inform team members Appoint guides Provide logistical resources Cooperate with auditors Share information, records Agree on non-compliances Propose and implement corrective actions


Finalize Schedule for Audit Cycle

Schedule interview of 1-3 hours for each project / department

1-2 auditors to conduct the interviews (new auditors must go in pairs)

Scheduling to be completed around two weeks before audit cycle start

Circulate and get confirmation from all auditees


Checklist Benefits

Ensures coverage is balanced Assists in preparing audit team Help maintain correct pace Provides a record of the audit for future reference Ensure nothing is forgotten!


Checklist Preparation

Use checklist of the previous audit as a starting point

Study the document QMS, Procedures, guidelines

Read relevant section of the Model Prepare separate lists for each project / support

function Consider time allocated and key areas


Remember Become fully conversant with the area before preparing /

modifying checklists Make separate checklists for different support functions You may have to make different checklists for different project

types With more experience you can make smaller checklists or just

bullet points Checklist is a tool and should be servant to the auditor –

CHECKLIST SHOULD NOT BE ALLOWED TO CONTROL THE AUDITOR

Checklists used in one audit can be used as a starting point in the next audit

Standard checklists may be included in the QMS after 1-2 cycles


The Opening Meeting

Purpose

Scheduling

Agenda

Tips


Purpose

Confirms scope and process of audit Put the auditee at ease Create the “right” atmosphere (In external audit) Give the auditors an insight to

the management commitment to quality


Scheduling

Before the start of audit interviews After the audit schedule is finalized Present in the opening meeting:

- Senior Management / MD- EPG / SQA- Lead Auditor for the audit- Other auditors for the audit- Senior-most representatives of all auditee groups (e.g. PMs, Department Heads)- Others who may interested


Agenda

Make sure all participants are presents Introduction to the audit team (Senior Manager / MD or

Lead Auditor for the audit cycle) Circulation of the attendance record Lead Auditor to explain

- Purpose / scope of the audit cycle- The audit interview process- Need for openness- Confidentiality- Documentation of findings- Reporting


Agenda (contd.)

Circulate / display audit schedule Discuss any logistics related issues Provide clarifications Invite everyone to closing meeting The Sr. Manager / MD can emphasize

- Use the findings will be to improve the process- Need to share information openly


Tips for the Auditors

Keep it short- Schedule 30 minutes- Try to finish in 20 minutes

Be well prepared Conduct meeting in businesslike manager Keep a record (attendance) Do not let the MD hijack the session


Audit Investigations

Approach

Interviewing

Audit Trail

Recording Findings


Approach

The auditor must keep control The auditor must manage his / her time Use prepared checklists as a guide Judgment – is there a problem or not The audit team must keep in touch


Objective Evidence

Records

Document

Statements

Observations

Relevance

Significance

ExistenceAcc

urac

y

Remember: only objective evidence is permitted


Audit Trail

Record the facts Is it on your checklist? Is there time available? Pass to the appropriate Auditor Consult the Lead Auditor

Note: if it is important, someone must look at it


Identifying Problems

Focus on the key matters Decide whether or not the Auditee is the right

person to ask the question Consider if there are further symptoms Where in the process could the root cause lie? Always verify evidence of non-compliance


Purpose of Interview

Elaboration Explanation Work status – what really happens? Basis for evidence Understanding Dialogue / rapport Perspective


Starting the Interview

Find a suitable location near their workplace Introduce yourself Explain the process “Assessing the system – not individuals” Be friendly but polite Dialogue / rapport Perspective

Interviewing is your main tool


The Interview

The auditor must keep control The auditor must manage his/her time Split time between managers and staff Work through the checklist

- If no problems – go quickly to next issue- Problems – investigate to get objective evidence & idea of magnitude- No sense digging until something is found


Useful Types of Questions

Open (STARTING) Follow up Probing Focusing Closed (ENDING)


Examples of Open Questions

Please describe your responsibilities Tell me about …? How does ….? Please explain how ….? Please describe the process ….?


Examples of Probing Questions

Where does ….? When did …? What is ….?


Examples of Closed Questions

Is this ….? Do you …? Does this ….? Please show me ….?


Remember

Interviewing is your main tool Look at the evidence Listen to the auditees Make sure you are asking the right persons Be ready to handle auditee reactions Watch out for auditee reactions Verify details of non-compliance Pass on information to team members Focus on the key matters Take help from other auditors / lead auditor


Non-Compliances

Also called- Non-conformities- Non-conformances- Deficiencies- Discrepancies- Deviations


Types of Non-compliances

Major non-compliances- A consistent, significant breakdown of the quality system

Minor non-compliances- Isolated or one-off failures; localized impact

Observations- Warning about potential non-compliances


Recording Non-compliances What Acknowledged by Auditee At the time they are found Using OBJECTIVE evidence

- Where, when, who, (how) Non-compliance statements must be

- Accurate- Complete- Helpful- Brief

Does it pass the ‘so-what’ test? Anticipate the corrective action


Audit Reporting

Report Contents

Closing Meeting

Audit Records


Purpose

To consolidate the activities and findings related to the Audit cycle

To provide feedback to the audit participants, Senior Management and the auditors

To collect all related records and close the Audit


Audit Report - Contents

The Audit Cycle reference Date of the Audit Cycle Scope of the Audit Cycle Lead Auditor and other auditors Summer of non-compliances Summery of good practices identified Target dates for closing all non-compliances


Audit Report - Contents

Statistics- Total meeting hours- Total areas / project audited- Number of major non-compliance- Number of minor non-compliance- Number of observations- Number of good practices observed- Number of persons in the opening meeting

Appendices- Audit Cycle Schedule- List of the attendees in the opening meeting- Non-compliance list / tracking sheet


Closing Meeting

Introduction and thank you Purpose / objective / scope Statistics Important findings Follow-up actions Any questions Acknowledgement of report


Audit Records

Audit Cycle schedule Opening meeting attendance Audit Report Non-compliance Reports Checklist used Interview notes Closing meeting attendance


Corrective Action Follow-up

Identification

Implementation

Tracking and Closure


Identification of the Non-compliance

The Auditor raises the problem

The facts The non-compliance The department / project responsible


Corrective Action Proposal

Auditee proposes corrective action

Root cause analysis Immediate remedial action Long term corrective action

Auditor evaluates the proposed corrective actionsThe organization’s QA functions provides advice to the responsible manager


Corrective Action Implementation

Auditee implements agreed corrective action

Keeps records of implementation Confirms that there is no other occurrences that

need to be corrected Confirms that the probability of similar

occurrences are considerably reduced


Verification

Performed by Auditor

Are there other similar non-compliance? Has the root cause been addressed? Has the likelihood of recurrence been assessed? Have they followed the CA procedure? Is a track of all corrective actions being

maintained?


Closure

Close non-compliance after verification Raise process improvement proposal if the

corrective action is deemed useful on a wider basis


Tracking of Non-compliances

EPG to track ALL non-compliances to closure, using some tracking sheet / database


Conclusion

Auditor Attributes

Purpose of Audit

Audit System


Auditors Attributes

Positive Pragmatic Professional Prepared Perceptive


Purpose of Audit

Compliance to:- Contract, requirements, proposal- Internal quality management system- A quality / process standard (e.g. CMMI)

Provide confidence to management and stakeholders

Identify process improvements


Audit System

Must be planned / scheduled Conducted by trained auditors Finding based on “objective evidence” Actionable findings must be tracked to closure

software quality assurance

Business