eva trosborgslide no.: 1quality management quality management fall 2005 agenda what is quality ?...

Eva Trosborg Slide no.: 1Quality Management

Quality ManagementFall 2005

Agenda• What is quality ? Process-, product- and need based qualityDefinitions of qualityQuality review: What and how?Example: Review of a requirement specificationMeasurement from DK and US

by

Eva Trosborg


After this lesson you should :

Be able to discuss quality management as well as plan and implement quality reviews in a smaller project.


Quality Definition

• Quality is fitness for use – J.M.Juran• Quality is meeting or exceeding customer

expectations at a cost that represent a value to them – H. James Harrington

• Quality should be defined as surpassing customer needs and expectations throughout the life of the product – Howard Gitlow and Shelley Gitlow

• Quality is conformance with requirements Philip Crossby


We want quality!

• Because we want to make a good piece of work• Because we are tired of making errors • Because it is boring to search for errors• Because it is cheaper and quicker to do it right

first time• Because quality make our customers happy


Process based QualityProcess based Quality

Product based QualityProduct based Quality

Need based kvalitetNeed based kvalitet

Value based QualityValue based Quality

Four types of quality


Wish

Wish

Wish

Need

Need

Need

Interpretation

Requirementspecification

Communication

Developmentprocess

Product

time1 time2

New requirement

Core


Wish

Wish

Wish

Need

Need

Need

Interpretation

Requirementspecification

Commu-nication

Developmentprocess

Product

time1 time2

Newneed

Core

Process basedquality

Product basedquality

Need basedquality


What is the quality work in a project ?

What is the quality work in a project ?

• Use of defined methods and tools (process- quality)

• Running reviews (process- and product-quality)

• Quality Measurements (non-functional requirements)

• Managing changes from a quality measurement point of view

• Measurement on the way (process) and final (product and need)

• Systematic reporting of quality (based on measurements)


Do quality payoff?

0.1

1

10

100

Krav Kode Systemtest

Pris for at rette


Pareto AnalysisThis is based on the notion that, in many situations, 80% of incidents are

accounted for by 20% of the causes. It gets its name from an Italian statistician who studied the phenomenon in detail.

• Also known as ‘The 80:20 Rule’, it is not always exactly split 80 to 20, but the general principle is remarkably common. What it does is encourage the circle members to find out which of the causes have the greatest effect.

• For example, you could find in your organisation that:• 80% of the turnover comes from 20% of the products • 80% of the complaints come from 20% of the customers • 80% of the absenteeism is accounted for by 20% of the workforce • 80% of the errors are produced on 20% of the machines

As you probably found, it is not a clean 80:20 split in each case, but there is frequently a relationship between the majority of the problems and a small minority of the causes.


Quality Management

• Managing quality from the first idea through development, marketing and implementation to the final product

• Quality management is a top-management

responsibility. It consists of a decision of Quality Level and features – and assuring that the IT products lives up to the agreed


Quality Management

Quality Management includes:

• Agreement of quality goals / quality policy (strategic decision) and

• Which are fulfilled in practice by means of Quality Management

Kilde:(DS/EN ISO 9004-1: 1994, Dansk introduktion, side 2)


PLAN - DO – CHECK – ACT

Plan,

specify

& decide

do

registre/measure

evaluate


Quality Management =

Quality Planning

+

Quality measuring

+

Quality control


Quality Planning

1. Specification of requirement for the required quality for the IT-products

2. Decision of IT-processes, plans, procedures concerning time, place, phases, people etc.


Quality control

• Measuring and registration• Evaluating aiming at corrective actions

Activities aiming at evaluating if the requirement specification conforms to specifications.

The Quality Control consists of:


Quality Assurance

• Measurements and registration• Evaluation, compare and work to secure

corrections are performed if needed

Activities performed with the objective to evaluate if IT-process are performed with respect to the decided plans and procedures.

It consist of:


ReviewReviewA review is a meeting where the quality of a product are evaluated.

The goal is:– To point out improvements– To accept a product– To secure equal quality in each part – To educate the participants

The purpose is to point out problems not to solve them


Some says about Review’s

• A method involving a structured encounter in which a group of technical personnel analyzes or improves the quality of the original work product as well as the quality of the method (Johnson 1998)

• An inspection intended to expose defects in the product (Kendall 1992)

• A “filter” for the software engineering process (Pressman 2000)

• Gives the analyst or programmer an honest appraisal of the system (Edwards 1993)


Phase’ and roles in a reviewPhase’ and roles in a review

Six Phases:

• Planning

• Information meeting

• Preparation

• Review meeting

• Correction

• Ending

Roles:

• Chairman

• writer

• note taker

• Critics


6 phases in a general review process

Choose a group, material and date

Present the product, process and

target

Check product, note findings

Present and consolidate findings

Correct the finding

Verify corrections, end the process

Prepare

Inform

Plan

Review meeting

Correct

End


Planning

• Objectives– Put together the package to be reviewed: product, checklist,

references, data, standards– Decides who to attend– Set a date and place

• Procedure– Chair

• Decides group and review package• Fits in checklists (if needed)• Plans for dates and meetings (information and review)• Checks if the product is ready for review? • Helps the writer to prepare information

Plan Inform Prepare Review Correct End


Information

• Objectives– Writer gives an overview– The ones to review gets the “package”– Objective and preparation fixed– The ones to review accepts

• Procedure– The chair gives out the review package– The writer informs if needed– The role a secretary (who) decided – Chair informs how the preparation should happen



Prepare

• Objectives– Find as many errors a possible

• Procedure– Allocate time for preparation– Each participant makes a review individually Use

checklists, references and standards as focus – Note everything found



The review meeting

• Objective– To create a list containing all findings – To create a forum where (2+2=5)– To improve the ability to review by watching what others do – Create a understanding of the product

• Procedure– Chairman leads the meeting page for page paragraph for paragraph– Findings presented– Secretary notes findings – if possible in a manner so participants

can follow the process. – At the end the findings are categorized on type and how serious



Examples of categories

• Critical – Defects which gives some wrong results or dangerous. Hang-ups or break

downs, no known work around.

• Serious– Defect which leads to ambiguous results or wrong behavior. Several

requirements not fulfilled but required a reprogramming. Something in a program behaving wrongly but where there is a (”work around”) to omit the error.

• Moderate– Defects only concerning a single requirements or a small part of the

system. The defect can be ignored and do not have much consequences

• Minor– Defects which makes it difficult to understand small thing or cosmetics


Example categorizing depending on TYPE

• Example (need) – Missing example to understand• Figure (need) - Missing example to understand• Logical (error) – non conformance to previous work• Missing – Something missing in the document/program • To much – Something which is written twice• Reference (need) – Source • Standard (not respected) – A standard is not respected• back – Defect in previous document/program, which is a

prerequisite for what being reviewed. • Not exact enough – can be misunderstood


Correct

• Objective– Evaluate each findings (if this was not done on

the review meeting). If an important finding correct or remove

– For less important findings (moderate or minor) notes that is was found, but it might wait for a letter release.



End

• Objective– Evaluate the quality of the corrected product– Evaluate the review process– Accept or discard the product

• Procedure – Receive the corrected product– Check– Accept or discard– Inform the participant from the review meeting– Keep data regarding #, types and time (quality data)



Example: Review of requirement specification

Example: Review of requirement specification

• To validate goal for the change (Why should the project be developed)

• To validate that the non-functional requirements are well-defined and documented.

• To validate that the described functions and data correspond to the requirements.


Are the customers requirements?

• SMART– Specific

– Measurable

– Achievable

– Reasonable

– Time constrained

• RUMBA– Reasonable

– Understandable

– Measurable

– Believable

– Achievable

There can be invalid requirements


FURPS+The FURPS+ can be used as checklist for requirement coverage, to

reduce the risk of not considering some important facets of the system

• Functional – features, capabilities, security• Usability – human factors, help, documentation• Reliability – frequency of failure, recoverability, predictability• Performance – response times, throughput, accuracy, availability, resource

usage• Supportability – adaptability, maintability, internationalization,

configurability• The +

– Implementation – resource limitations, languages and tools, hardware, …– Interface – constraints imposed by interfacing with external systems– Operations – systemmanagement in its operational setting– Packaging – for example, physical box– Legal – licensing and so forth

Some of these requirements are collectively called quality attributes / quality requirements

Larman p56 +


Supplementary Specifications

• Non-functional requirements should after the use case be moved to the Special Requirement Section – to keep all non-functional requirements in one place, and not duplicated.

• Quality Attributes – URPS+ usability, reliability, performance,

supportability … these are qualities of the system, not of the attributes themselves. I.e supportability intended low due to …

– Architectural analysis and design largely concerned with identification and resolution of quality attributes in the context of functional requirements

Larman p 107 +


Quality Scenarios

• Recommended as they define measurable (or at least observable) responses to be verified. (easy to modify needs some measure of what it means)

• Quantifying some things, a performance goals and mean time between failure, are well known practices, quality scenarios extend this idea to recording most factors as measurable statements.


Pick your battles

• Quality scenarios are short statements <stimulus> <measurable response>

• When the completed sale is sent to the remote tax calculator to add the taxes, the result is returned within 2 seconds “most” of the time, measured in a production environment under “average” load conditions.

“most” and “average” need further investigation and definition. A quality scenario is not really valid untill it is testable, which implies fully specified.

Will anyone really test them? How and by whom?


Sample factor table

Factor Measures and Quality scenarios

Variability (current flexibility and future evaluation)

Impact of factor (and its variability) on stakeholders, architecture and other factors

Priority for success

Difficulty or Risk

Larman 547


Critical make-or-break quality scenario

• Which are the important battles in your project?

• Write those quality scenarios and follow through with a plan for their evaluation.


Service Level Agreement (SLA) / Service Level Management (SLM)

• A Service Level Agreement (SLA) is a formal written agreement made between two parties: the service provider and the service recipient. The SLA itself defines the basis of understanding between the two parties for delivery of the service itself. The document can be quite complex, and sometimes underpins a formal contract. The contents will vary according to the nature of the service itself, but usually includes a number of core elements, or clauses.

• Generally, an SLA should contain clauses that define a specified level of service, support options, incentive awards for service levels exceeded and/or penalty provisions for services not provided. Before having such agreements with customers the IT services need to have a good quality of these services, Quality management will try to improve the Quality of Service, whereas the SLAs will try to keep the quality and guarantee the quality to the customer.

http://en.wikipedia.org/wiki/Service_Level_Agreement


SLA Management is the key for making sure you get what you paid for.

• SLO (service level objectives) and KPI (key performance indicators).

• SLO is then valued against the agreed-upon service levels, and the result is reported to the service provider and/or consumer.


Information Technology Infrastructure Library (ITIL)

• is a set of best practices used to deliver high quality IT services. The best practices described in ITIL represent the consensus derived from over a decade of work by thousands of IT and data processing professionals’ world-wide, including hundreds of years of collective experience. Because of its depth and breadth, the ITIL has become the defacto world standard for IT best practices.

http://www.itilsurvival.com/


Example: Review of requirement specification – Sources of errorsExample: Review of requirement specification – Sources of errors

– Misunderstandings in the communication– Requirement and measurements are not

quantified – Missing formal information or documentation – Lack in understanding of where are we going– Misunderstanding of existing data– Missing understanding of relationships in data– Misunderstanding of examples and diagrams


Example - Review of requirement specification - documentation

Example - Review of requirement specification - documentation

• Revised requirement specification

• Descriptions of use (Use Cases)

• Minutes of the meetings

• Documentation of conflicts and solutions

• List of demands and wishes being refused

• List of riscs


Measurements from Lyngsø in Denmark

• One manhour a page – including preparation, free-meeting and planning included. It is possible to review 7-9 pages an hour

• Review meetings in the morging is most effective – this means more errors and ommisions are found an hour

Kilde: Bjørn Runge: Erfa-møde i Datateknisk forum om ”Lyngsøs erfaringer med reviews”


Measurements from IBM, USA

• Review meeting lasting more than 2 hours are less effective

• If more than 20 pages are to be reviewed then it is better to split the meeting.

Kilde: Michael Fagan: ”Design og Code Inspections to reduce Errors in program development” IBM Systems Jn., vol. 15, nr. 3, 1976, side 182-211


The DieHards 12:30 – 13:15 Carsten & Eva N/A

The Outsiders 13:30 – 14:15 Carsten & Eva RecievedThe LuckyOnes + The Tiny Group

14:30 – 15:15 Carsten & Eva n/aThe Favorites 15:30 – 16:15 Carsten & Eva n/a

Review of requirements with customer the 7th October in room 3A18 if review material

received before the 4th October

Two presentations: one for us as customers and one for us a part of the organization


See you again the 28th. OctoberObject oriented design and more

patterns chapter 22 - 26

Eva Trosborg

eva trosborgslide no.: 1quality management quality management fall 2005 agenda what is quality ?...

Documents