metrics casestudy

8/6/2019 Metrics Casestudy

1/15

Nagaraj Yekkala


2/15

Topic #1 : Overview of Metrics

Aim

Metrics ?

Metrics Classification

Topic # 2 : What affects software

Topic # 3 : Quality Model

Topic # 4: Metrics cont.

Topic # 5 : Translating Business needs

Topic # 6 : Frameworks

Topic # 7 : Process Improvement


3/15

While metrics are important means to quantify performance of a process, we often

define metrics to comply with processstandards, overlooking actual alignment with

businessfactors.We may put substantial effort and discipline in getting metrics

deployed, but we might not be using the ones that really matter. How can we define

metrics that remain anchored to business needs? In other words,how can we translate

business requirements to the right set of metrics? Once the right metrics are defined,

how can we monitor and benchmark it to control performance and make

breakthroughimprovements?


4/15

Metrics are measurements,collections of data about project activities, resources and

deliverables.Metricscan be used to help estimate projects, measure project progress and

performance, and quantify product attributes.

Software metricscan be classified into three categories:

Product metrics (size,complexity, performance)

Process metrics (used to improve development and maintenance)

Project metrics (cost,schedule, productivity)


5/15

Software metrics are numerical data related to software development.Metricsstronglysupport

software project management activities. They relate to the four functions of management as

follows:

1. Planning Metricsserve as a basis ofcost estimating, training planning, resource planning,

scheduling, and budgeting.

2. OrganizingSize and schedule metricsinfluence a project's organization.

3. ControllingMetrics are used to status and track software development activitiesfor compliance to

plans.

4. ImprovingMetrics are used as a tool for processimprovement and to identify where

improvement effortsshould be concentrated and measure the effects of processimprovementefforts.


6/15

People

Process Technology


7/15

product

operation revision transition

reliability efficiency usability maintainability testability portability reusability

Metrics


8/15

Gathering Data

Gather both quantitative and qualitative data about organizations. Use commonly

accepted practices for collecting and interpreting qualitative and quantitative data.

Employ the technique of ethnological study to software organizations. Ethnography

consists of looking at what people do, what they say, and the tension between what they

really do and what they ought to do, as well as what they make artifacts

Quantitative Data

Gather up as much quantitative data as possible. These are hard facts about an

organization. I gather data at the project and application levels and it is rolled up to the

organizational level. The following is a list of some examples are provided for the

quantitative data to be gathered :

Project Size (measure in function points)Application Size (measured in function points)Use Cases

HoursDefectsStaffSalary FiguresTest CasesTesting ProblemsProduction DefectsProject DurationLines of CodePages of Documentation

Metrics cont.Metrics cont.


9/15

Metrics and Ratios

Develop a set of metrics (ratios) using the quantitative data. One of the metrics to calculate is organizational cost per function

point. This metricis based upon the total organizational cost divided by the total number offunction points. Also calculate

project cost per function point or hours per function point. Often, there is a large gap between project costs per function point

and organizational cost per function point.

Too often the onlycosts that get reported and recorded are programming costs not total organizational costs. Actualprogramming

costs represent between ten and twenty percent of total software development costs.

Qualitative Data

The qualitative scores are based upon actual artifacts where possible. It isimportant to have supporting detail especially

artifacts.When working with a company do not makes requestssimilar to, Rate your project management skills on a

scale of1 to 5 because everyone knows the right answer. Instead, rely on artifacts where possible. If a companyis a five in

project planning,it will have project plans to examine.

GatherQualitative DataConduct formal interviewsTalk to people, and ask a lot of questions

Qualitative Scores And Productivity

There is a strong correlation between high qualitative scores and productivity. It is not by accident that use the word correlateand not the word causes. Correlate and causation are not the same thing at all. It issimilar to figuring out that wealthy people

listen to classical music and concluding that ifyou listen to classical music,you too will be wealthy. Listening to classical music

is a characteristic of the wealthy;it is not what made them wealthy.

Project Success and Project Failures

Large projects withhigh qualitative scoresin the area of project management skills were more likely to be successful than

projects with low scores. Those large software projects with more than 10,000function pointsare more likely to be successful whenthey brought in skilled project managers from the outside.

Metrics cont.Metrics cont.


10/15

Establish measurementgoals aligned withbusiness needsi.e. whatwe want to achieve

Conceptual Level

Generate a set ofquestions that can beused to determineachievement of goals

Operation Level

Establish the set of data,objective or subjective,that will answer eachquestion in aquantitative way

Quantitative Level


11/15

Goal Approach Question Metric

ImproveEstimation

MinimizeDefects

Reduce Costs

Estimating effortbased on requirementsPlanning based onestimation

Detecting defects asearly as possibleMinimize acceptancetesting defects

Detecting defects asearly as possibleMinimize acceptancetesting defects

How good iseffortestimation ?

What arecurrent defectlevels?

What isRework Effort ?

EffortSlippage

DefectDensity

ReworkEffort

Translating business needsTranslating business needs


12/15

Classifying software measures

Three types ofsoftware entities to measure

Processes collections ofsoftware related activities

Products

Resources entities required by a process activity

Within eachclass, we have

Internal attributes measured purelyin terms of the entityitself

External attributes measured with respect to how entity relates to its environment. Behaviorof the entityisimportant

Managers want to be able to measure and predict external attributes

However, external attributes are more difficult to measure than internal ones, and aremeasured late in the development process

Desire is to predict external attributesin terms of more easily-measured internal attributes


13/15

Determining what to measure

Measurement is useful onlyifit helps understand the underlying process or one ofits resultant products

Goal-Question-Metric (GQM) has been proven to be effective in selecting andimplementing metrics

List the major goals of the development project

Derive from each goal the questions that must be answered to determine if

goals are being met

Decide what must be measured in order to be able to answer the questions

adequately

13


14/15

14

GQM example goal is to evaluate effectiveness ofcoding standard

Goal

Who is using standard? What iscoder productivity? What iscode quality?

Proportion ofcoders Using standard

Using language

Experience ofcoders Withstandard

With language With environment, etc.

Code size(lines ofcode,

functionpoints, etc)

Effort Errors

Goal

Questions

Metrics


15/15

Qualityimprovement starts withidentifying the right metrics.We apply techniquessuch ascontrol

charts to improve the stability of the process. Thereafter, we embark on a continuousimprovement journey by

increasing the capability of the process.

Business

objectives

ProcessChange

Piloting

Sub-processSelection

Measure

StatisticalAnalysis

(OPP)

RemovalSpecialCauses

RemoveCommon

Causes

Process/Technology

Change

ProcessStable ?

ProcessCapable ?

NO

OrgChange

Project -LevelChange

NO

Yes Yes

metrics casestudy

Documents