SE-381 Software Engineering

BEIT-VLecture no. 14

Software Quality Assurance ModelsISO, CMM and TickIT

Software Quality Assurance– Software Development (Product Engineering)

and Process Management be so managed that ultimate Product is of high quality and has been developed within time and budget constraints

– Both Product Engineering and Process management processes are to be refined and matured so that all products produced are of high quality

– SQA processes for Software development ensure that this objective is attained, different organizations, ISO, SEI, ISO/IEC have approved standards that can provide desired Quality.

ISO 9000 & CMM

– Are the two widely used standards by the industry

– ISO 9000 is a series of standards applicable for the certification of industrial processes, ISO 9000-3 refers to software specifically

– CMM is exclusively for Software Industry and is to evaluate and improve the Process, initiated to evaluate US DoD vendors and ensure their processes are mature to yield quality products

– CMM is becoming a de-facto standard for measuring the capability and maturity of software companies

Mature and Immature Organization• Mature Organizations

Planned & Controlled Activities

• Immature Organizations

Crisis Management, Chaos, Always Fire

fighting

Mature and Immature Organization

• Immature Organizations– reactionary

– often busy in fire fighting

– always late & over budget

– unrealistic estimates

– product functionality and quality are often compromised to meet the schedule

– no basis for• judging product quality• solving product or process

problems

• Mature Organizations– organization-wide

managed software processes

– the software process is accurately communicated to both existing staff and new employees

– work activities are carried out according to the planned process

– objective & quantitative basis

• for judging product quality• for analyzing problems

with the product and process

• CMM– Capability Maturity Model– Initiated by Software

Engineering Institute and in turn funded by US DoD

– Initially to identify which of the US DoD Sw vendors have best ‘mature’ Sw process

– Process improvement emphasized through FIVE Maturity levels

– US Centered and getting international de-facto standard

• ISO 9000– Compendium of 4

standards, applicable to wide variety of Industrial systems having stages

• Design• Development• Production• Installation and • Servicing

– Initiated by International Standards Organization

– Document driven – text and Graphical documents maintained to achieve comprehensibility

– Europe centered

ISO 9000 & CMM

CMM Maturity Levels

5 Levels of CMM

CMM Maturity Levels

Software CMM

Process Maturity Framework

Process, Capability & Maturity

– As process matures, it becomes better defined and more consistently implemented

– Software process capability• the range of expected results that can be achieved by

following a software process• means of predicting the most likely outcomes to be expected

from the next software project

– Software process performance• actual results achieved by following a software process.

– Software process maturity• extent to which a specific process is explicitly defined,

managed, measured, controlled, and effective

Process Capability & Maturity

– Maturity• a potential for growth in capability

• indicates both the richness of an organization's software process and the consistency with which it is applied in projects throughout the organization

• the software process is well understood– through documentation and training– continually being monitored and improved

• capability of a mature software process is known

• productivity and quality resulting can be improved over time through consistent gains in the discipline achieved by using its software process

Process Capability & Maturity

– A Mature software organization• institutionalizes its software processes via

– policies– standards– and organizational structures

– Institutionalization• building an infrastructure and a corporate culture that

supports the methods, practices, and procedures of the business so that they endure after those who originally defined them have gone

Objectives for Software CMM®

• Provide a model that is – based on actual practices

– reflects the best of the state of the practice

– reflects the needs of individuals performing software process improvement, software process assessments, or software capability evaluations

– Documented and publicly available from www.sei.cmu.edu

CMMI – CMM Integrated

• Is the integration of Capability Maturity Model into the process, and CMMI is now international standard

Use of SW-CMM®

• To understand the activities necessary to plan and implement a software process improvement program

• To define and improve the software process

• To identify strengths and weaknesses in the organization

• To identify the risks of selecting among different contractors for awarding business and to monitor contracts

Software Process Maturity (SPM)

– Continuous process improvement is based on many small, evolutionary steps rather than revolutionary innovations

– The CMM provides a framework for organizing these evolutionary steps into five maturity levels that lay successive foundations for continuous process improvement

– These five maturity levels define an ordinal scale for measuring the maturity of an organization's software process and for evaluating its software process capability

Maturity Level

– A well-defined evolutionary plateau toward achieving a mature software process

– Each maturity level provides a layer in the foundation for continuous process improvement

– Each level comprises a set of process goals that, when satisfied, stabilized an important component of the software process

– Achieving each level of the maturity framework establishes a different component in the software process, resulting in an increase in the process capability of the organization.

Five Levels of SPM

Initial

(1)

Disciplined Process

Repeatable

(2)

Standard Consistent Process

Defined

(3)

Predictable Process

Managed(4)

Continuously Improving Process

Optimizing(5)

Unpredictable

Disciplined

Consistent

Predictable

Continuously improving

Five Levels of CMM

Initial Level - Level 1• The software process is characterized as

– ad hoc, and occasionally even chaotic– few processes are defined– success depends on individual effort– Every software house starts as a Level-1

CMM company

CMM Levels (cont.)

Repeatable Level - Level 2• Basic project management processes are

established to track– cost– schedule– and functionality

• The necessary process discipline is in place to repeat earlier successes on projects with similar applications

CMM Levels (cont.)

Defined Level - Level 3• The software process for both management and

engineering activities is – documented– standardized– and integrated into a standard software process

• All projects use an approved version of the organization's standard software process for developing and maintaining software

CMM Levels (cont.)

Managed Level - Level 4• Detailed measures/metrics are collected and

controlled for– software process– product quality

• Both the software process and products are quantitatively / statistically controlled and understood by using these detailed measures

CMM Levels (cont.)

Optimizing Level - Level 5• Continuous process improvement is enabled by

– quantitative feedback from the process– piloting innovative ideas and – technologies

Most of the processes are automated. Change in Process and Technology is managed and Errors prevented

CMM® StructureMaturity Levels

Process Capability

Indicate

Key Process Areas

Contain

Goals

Achieve

Common Features

Organized by

Implementation orInstitutionalization

Address

Key Practices

Contain

Infrastructure orActivities

Describe

Key Process Areas (KPAs)– The areas an organization should focus on to improve

its software process

– Key process areas identify the issues that must be addressed to achieve a maturity level

– Each KPA identifies a cluster of related activities that, when performed collectively, achieve a set of goals considered important for enhancing process capability

– When the goals of a key process area are accomplished on a continuing basis across projects, the organization can be said to have institutionalised the process capability characterized by the key process area

– 18 KPAs have been defined for different levles and these are additive

Key Process Areas

Initial (1)

Repeatable (2)Software Configuration ManagementSoftware Quality AssuranceSoftware Subcontract ManagementSoftware Project Tracking & OversightSoftware Project PlanningRequirement Management

Defined (3)Peer ReviewsInter-group CoordinationSoftware Product EngineeringIntegrated Software ManagementTraining ProgramOrganization DefinitionOrganization Process Focus

Managed (4)Software Quality ManagementQuantitative Process Management

Optimizing (5)Process Change ManagementTechnology Change ManagementDefect Prevention

Common Features

– The key process areas are organized by common features

– The common features are attributes that indicate whether the implementation and institutionalization of a key process area is effective, repeatable, and lasting

– The five common features are listed below• Commitment to perform• Ability to perform• Activities performed• Measurement & Analysis• Verifying Implementation

Key Practices

– Each key process area is described in terms of the key practices that contribute to satisfying its goals

– The key practices describe the infrastructure and activities that contribute most to the effective implementation and institutionalization of the key process area.

Building the CMM® Structure

DisciplinedProcess

Indicates

Goal 1:Software estimates are documented

for use in planning & tracking the software project

Achieve

Implementation

Addresses

Level 2: Repeatable

Maturity Level:

Contains

Software Project PlanningKey Process Area:

Organized by

Activities Performed

Common Feature:

Contains

Activity 9: Estimates for the size of software work products (or changes to the size of software work products)are derived according to a documentedprocedure.

Key Practice:

Activity

Describes

Key Process ChallengesLevel Characteristics Key Problem Areas Productivity &

Quality

5 – OptimizingImprovement fed backto process

Automation

4 – Managed(Quantitative)Measured Process

Changing technologyProblem analysisProblem prevention

3 – Defined(Qualitative)

Process defined &institutionalized

Process measurementProcess analysisQuantitative quality plans

2 – Repeatable(Intuitive)

Process dependenton individuals

TrainingTechnical Practices* reviews, testingProcess focus*standards, process groups

1 – Initial(Ad-hoc / Chaotic) Project management

Project planningConfiguration managementSoftware quality assurance

Visibility into Software ProcessIn Out

5

In Out

4

In Out

3

In Out

2

In Out1

CMM® Based Appraisals

“CMM® Based Appraisals” (CBA) use the CMM® as the basis for appraising software processes

They are multi-phased, team-based, site-visit focused appraisal processes

Examples of Appraisal Families

Internal Process Improvement (IPI)• Self improvement

Acquisition• Selecting supplier

Teaming or Joint Improvement • Customer & supplier together

Third Party • Certification

Appraisal Process Key Points

• Planning the appraisal is based on sponsors business goals & appraisal objectives

• Preparing for the appraisal includes training

• Collecting the data is focused on the model

• Validating the implementation of process is essential

• Reporting is done for each process area investigated

Reading Assignment

1. Jalote Pankaje (1997); An Integrated Approach to SE, 2nd Ed, Narosa Publishing House, New Delhi; Ch – 1 Introduction

2. Douglas Bell (2005); Software Engineering for Students; Pearson Education Limited; Ch-1 Problems and Prospects, pp 3-22

3. Rajib Mall (2005); Fundamentals of Software Engineering, 2nd Ed, Printice-Hall of India, New Delhi; Ch – 11 Software Reliability and Quality Management

4. Stephen Schach (2007); software Engineering, 7th Ed, Tata McGraw-Hill Publishing Company, New Delhi; Ch – 4 The Software Process