© Sogeti

Quality Assurance vs. Quality Control

The future of Software Quality

Bertrand Cornanguer

© Sogeti

Agenda• Introduction• What do we want, what is testing, Quality control• What is Software Quality Assurance and Total Quality

Management• How SQA Frameworks can help to get a good product• Consequences on the people skills and competences• Conclusion

© Sogeti

Introduction• The context of quality in software development has

changed since new technologies, hardware and software, have allowed multiple varieties of development by varieties of people.

• After standalone programs, the idea of Information System, connected objects oblige organization to set up processes to allow the quality of software components.

• But today, a lot of organizations are still confident in Quality Control as the last hedge before production when other more advanced develop Quality Assurance

• This presentation will show the differences between these two concepts and the scope of Quality Assurance in Software Development as the future of software Quality

© Sogeti

IntroductionSome figures from the SOGETI/HP World Quality Report

Test plan, charge

ElicitationReviews,

Functional analysis,

ReqM, Traceability

TDD, BDD, ATDD,

Component, integration

tests

Operational testing, Regression testing,

Continuous Integration testing

© Sogeti

What do we want ?• A « good » product

– Satisfying at first• Usability• Functionalities• Performances• Security

– And also• Maintainability, portability…

•A product released at the good moment•Preventing the defects with low costs•When numerous people are involved…•When software development is a single product or is a COTS…

I don’t care of the IHM, but I want very fast transactions because I am a trader…

I prefer a tool because it is WISIWIG, drag and drop, even there are some bugs…

In case of updates, I want to be able to understand and update the code

With constraints

© Sogeti

What is testing• Testing a web application in several browsers

– IE, Chrome, Firefox, Safari, – Mobile versions…– Versions 7..

– What’s happen if we only test the application in the different browsers at the end of the development ?

– That’s the mystery for me ! The game commences– Oups we cannot deploy the software component !

© Sogeti

What is testing• Testing a product:

– Verifying that a product meets specified requirements. – Basically Quality control of the product

=> If defects are discovered then rework and retest– Increase the time to market– Corrections can generate new defects– Create stress for all stakeholders

SpecificationsNeeds analysis Coding

Acceptance

Integration

Test is a final Quality Gate that can stop or delay a project or let the release of a bad product

Are there defects ?, what do we do, postpone or deploy with bugs ?

Consequences !

© Sogeti

Specifications

What is testing• Testing also a process:

– Verifying that a process allows to create a good product the fist time

• A process, a life cycle development has to be set up• Quality gates allow to check that the process is effective• Basically Quality Control of the process

Needs analysis Coding Integration Acceptance…

Quality Gate 1

Quality Gate 2

Quality Gate 3

Quality Gate 4

Are the non conformity repaired and their status updated ?Is there a test report after a test cycle ?

© Sogeti

What is Quality Control• Quality Control: The operational techniques and

activities, part of quality management, that are focused on fulfilling quality requirements.

Check the product

Check the processes

• Informing developers, customers, testers about

•Reviews•Indicator•Testing entry criteria

•Functionality•Usability…

Test is Quality Control

© Sogeti

What is Quality Assurance• Quality Assurance

– Part of Quality Management focused on providing confidence that quality requirements will be fulfilled

Plan

DoCheck

Act

Quality Control

• Setting up processes• Implementing them• Checking products and processes• Proposing improvements for the processes• Use Methodologies, standards like Lean• Standardize

Is there a test report after a test cycle ? Is the test report read and useful ?

A test report template is created and used by the projects

We need a test report for testing projects and deploy it on several projectsThis will allow to take decisions about deployment

Retrospective say that the report has to change to integrate metrics that we need to take decisions

The plan can be a Quality Assurance Plan

© Sogeti

How Agile environments integrate Software Quality Assurance• Agile practices

– Continuous feedback, continuous improvement– Time estimation– Test depending on quality attributes defined with the User Story– Needs comprehensiveness– Communication– Automated delivery, and configuration management

But is not always well implemented… and often limited to a team, at a development level, not well integrated to the whole organization

© Sogeti

How Agile environments integrate Software Quality Assurance• Agile practices

– In a large organization you need an Extended Agile Framework to prevent the risks of implementing Agile

– Sprint 0 with Global Test Strategy– Communication outside the Agile Team

© Sogeti

Quality Assurance brings a global vision integrating the Agile teams and tests

PortfolioInvestmentsProduction Plan

Program – ARTRelease management TeamProduct managers

ProjectAgile Teams

VisionArchitectureThemes

Release Themes and goals

Components and features

Trace

abilit

y

Epics (High level requirements)

Release (product) Backlog

Iteration(Stories)Backlog

Stories

Tasks

FeatureProgram backlog

Test types

© Sogeti

What is Total Quality Management

• Improving the whole SDLC allows to reach 0 defect

For each activity

•Plan and develop methodologies and tools•Document, train•Review•Control implementation•Set & follow indicators•Get feedback, Improve•Communicate

Needs Analyzing

SpecificationsCoding

TestingDelivery

Deployment

Project Management

Configuration Management

Training

Requirements Management People

Management

Plan

DoCheck

Act

MBT, UMLFunctional analysis

Tools

PMI/Prince2ISTQB/REQB…

ToolsHay Methodology

TPI,TMMI

How to reach 0 defects if needs are not clear ?

© Sogeti

How QA Frameworks help to Reach Quality

Capability Maturity Model Integration (CMMI) Core Process AreasAbbreviation Name Area Maturity Level

CAR Causal Analysis and Resolution Support 5CM Configuration Management Support 2DAR Decision Analysis and Resolution Support 3IPM Integrated Project Management Project Management 3MA Measurement and Analysis Support 2OPD Organizational Process Definition Process Management 3OPF Organizational Process Focus Process Management 3

OPM Organizational Performance Management Process Management 5

OPP Organizational Process Performance Process Management 4OT Organizational Training Process Management 3PMC Project Monitoring and Control Project Management 2PP Project Planning Project Management 2PPQA Process and Product Quality

Assurance Support 2

QPM Quantitative Project Management Project Management 4REQM Requirements Management Project Management 2RSKM Risk Management Project Management 3SAM Supplier Agreement Management Support 2

1-Initial

2-Managed

3-Defined

4-Quantatively Managed

5-Optimizing

Capability Maturity ModelIntegration

For each process, define activities and goals..

© Sogeti

How QA Framework helps to Reach Quality • Validation (VAL)• An Engineering process area at Maturity Level 3• Purpose• The purpose of Validation (VAL) is to demonstrate that a product or product component fulfills its intended use when

placed in its intended environment.• Specific Practices by Goal• SG 1 Prepare for Validation

– SP 1.1 Select Products for Validation– SP 1.2 Establish the Validation Environment– SP 1.3 Establish Validation Procedures and Criteria

• SG 2 Validate Product or Product Components– SP 2.1 Perform Validation– SP 2.2 Analyze Validation Results

Verification (VER)An Engineering process area at Maturity Level 3PurposeThe purpose of Verification (VER) is to ensure that selected work products meet their specified requirements.Specific Practices by GoalSG 1 Prepare for Verification

SP 1.1 Select Work Products for VerificationSP 1.2 Establish the Verification EnvironmentSP 1.3 Establish Verification Procedures and Criteria

SG 2 Perform Peer ReviewsSP 2.1 Prepare for Peer ReviewsSP 2.2 Conduct Peer ReviewsSP 2.3 Analyze Peer Review Data

SG 3 Verify Selected Work ProductsSP 3.1 Perform VerificationSP 3.2 Analyze Verification Results

But CMMI is very generic and need to be detailed.

To improve the testing processes, TPI and TMMI are more useful.

They integrate also the training of people

© Sogeti

How QA Frameworks help to Reach Quality

TMMI

A TestMaturity Model

Extracted from TMMI framework 3.1

•This model • Respects the maturity levels

of CMMI• Focus on testing activities

Initial

Test Policy and StrategyTest PlanningTest Monitoring and ControlTest Design and executionTest Environment

Test OrganizationTest Training ProgramTest Lifecycle and IntegrationNon-Functional TestingPeer Reviews

3 Defined

2 Managed

4 Measured

5 Optimization

Test MeasurementSoftware Quality EvaluationAdvanced Peer Reviews

Defect PreventionTest Process OptimizationQuality Control of Test Processes

But do wee need to follow the model of improvement ?

© Sogeti

Continuous improvement with TPI

• TPI allows to develop the key areas you want independently of the others.

© Sogeti

Consequences on the people skill and competences

• For each activity, people must be trained– On methodologies– On tools

• Tools are needed to be able – To get back– To standardize the

processes– To automate reports and

indicators– To focus on added values

tasks

-Testers must understand processes and QA- People must accept Quality Assurance and processes-People must have a global vision of QA and of the SDLC-Managers must invest in training and tools-QA engineers must be efficient and not only checking the progression of activities

© Sogeti

Conclusion• Quality Control is a

reactive approach

• Quality Assurance prevents the defects

More confident in quality, Release Dates

But•Resistance to Change • feeling of power loss for

some managers •ROI difficult to be evaluated

Stressing ApproachUnknown release dates and Quality

When

© Sogeti

Conclusion• The future of the Software Testing is Software Quality

Assurance– Improving the processes to reach the 0 defect

• ISO9001• CMMI

– Deploying the Consumer Product Quality Methodologies to software development

– The Quality Engineers and Quality Department ensure the quality of the products

– QA and Testing budget must increase, driven by transformational projects• Organizations need to be helped to organize the changes (IDEAL)• A Kaizen spirit has to be insufflated in the organization

– Quality control still exist, but budgets will decrease for Quality Assurance