November 2, 1997 1Chapter 4 -- R. A. Volz -- assistance -- Mingjuan Cui

Software Process and Project Metrics

Outline:

In the Software Metrics Domain:product metricsproject metricsprocess metrics

Software Measurementsize-oriented metricsfunction-oriented metrics

Metrics for Software Quality

November 2, 1997 2Chapter 4 -- R. A. Volz -- assistance -- Mingjuan Cui

Measure, Metrics, and Indicator

Measure -- Provides a quantitative indication of the extent, amount, dimensions, capacity, or size of some product or process attribute.

Metrics -- A quantitative measure of the degree to which a system, component, or process possesses a given attribute.

Software Metrics -- refers to a broad range of measurements for computer software.

Indicator -- a metric or combination of metrics that provide insight into the software process, a software project, or the product itself.

November 2, 1997 3Chapter 4 -- R. A. Volz -- assistance -- Mingjuan Cui

In the Process and Project Domains

Process Indicator

enable insight into the efficacy of an existing process to assess the current work status Goal -- to lead to long-term software process improvement

Project Indicator assess the status of an ongoing project track potential risks uncover problem areas before they “go critical” evaluate the project team’s ability to control the product quality

November 2, 1997 5Chapter 4 -- R. A. Volz -- assistance -- Mingjuan Cui

Measurement

What to measure? errors uncovered before release defects delivered to and reported by end users work products delivered human effort expended calendar time expended schedule conformance

At what level of aggregation? By team? Individual? Project?

November 2, 1997 6Chapter 4 -- R. A. Volz -- assistance -- Mingjuan Cui

Privacy Issues

Should they be used for personnel evaluation?

Some issues?

Privacy? Is total assignment being measured? Are the items being measured the same as for other

individuals being measured? Are the conditions of measurement the same across

individuals? However, they can be useful for individual improvement.

November 2, 1997 7Chapter 4 -- R. A. Volz -- assistance -- Mingjuan Cui

Use of Software Metrics

Use common sense and organizational sensitivity. Provide regular feedback to individuals and teams. Don’t use metrics to appraise individuals. Set clear goal and metrics. Never use metrics to threaten individuals or teams Problems != negative. These data are merely an indicator

for process improvement. Don’t obsess on a single metric to the exclusion of other important

metrics. Do not rely on metrics to solve your problems. Beware of people performing to metrics rather than product quality or safety.

November 2, 1997 8Chapter 4 -- R. A. Volz -- assistance -- Mingjuan Cui

Typical Causes of Product Defects

November 2, 1997 10Chapter 4 -- R. A. Volz -- assistance -- Mingjuan Cui

Project Metrics Software Project Measures Are Tactical

used by a project manager and a software team to adapt project work flow and technical activities

The Intent of Project Metrics Is Twofold to minimize the development schedule to avoid delays and

mitigate potential problems and risks to assess project quality on an ongoing basis and modify

the technical approach to improvement quality Production Rates

pages of documentation review hours function points delivered source lines errors uncovered during SW engineering

November 2, 1997 11Chapter 4 -- R. A. Volz -- assistance -- Mingjuan Cui

Software Metrics

Direct measures Cost and effort applied (in SEing process) Lines of code(LOC) produced Execution speed CPU utilization Memory size Defects reported over certain period of time

Indirect Measures Functionality, quality, complexity, efficiency, reliability,

maintainability.

November 2, 1997 12Chapter 4 -- R. A. Volz -- assistance -- Mingjuan Cui

Software Measurement

Size-Oriented Metrics are derived by normalizing quality and/or productivity

measures by considering the “size” of the software that has been produced.

lines of code often as normalization value.

project LOC effort $(000) pp.doc errors defects people

alpha 12,100 24 168 365 134 29 3beta 27,200 62 440 1224 321 86 5

gamma 20,200 43 314 1050 256 64 6

. . . . . .. . . . .

. . . . .

November 2, 1997 13Chapter 4 -- R. A. Volz -- assistance -- Mingjuan Cui

Typical Size-Oriented Metrics

Errors per KLOC Defects per KLOC Dollars per KLOC Pages of documentation per KLOC Errors per person month LOC per person month Dollars per page of documentation

November 2, 1997 14Chapter 4 -- R. A. Volz -- assistance -- Mingjuan Cui

Software Measurement

Function-Oriented Metrics use “functionality” to measure derived from “function point” using an empirical relationship based on countable (direct) measure of SW information

domain and assessments of software complexity Use of Function-Oriented Metrics

Measuring scale of a project Normalizing other metrics, e.g., $/FP, errors/FP

November 2, 1997 15Chapter 4 -- R. A. Volz -- assistance -- Mingjuan Cui

Function Point Calculation

Weighting Factormeasurement parameter count simple average complex

number of user outputs * 4 5 7 =# of user inquiries * 3 4 6 =number of files * 7 10 15 =# of external interfaces * 5 7 10 =

count_total

number of user inputs * 3 4 6 =

November 2, 1997 16Chapter 4 -- R. A. Volz -- assistance -- Mingjuan Cui

Function Point Calculation

Computing function pointsRate each factor on a scale of 0 to 5

no influence incidental moderate average significant essential

1 2 3 4 5 6

1. does the system require reliable backup and recovery?

2. are data communications required?

3. are there distributed processing functions?

4. is performance critical?

........

14. is the application designed to facilitate change and ease of use by the user?

November 2, 1997 17Chapter 4 -- R. A. Volz -- assistance -- Mingjuan Cui

Function-Oriented Metrics

FP = count_total * [0.65 + 0.01 * sum of Fi]

Outcome:

errors per FP

defects per FP

$ per FP

page of documentation per FP

FP per person_month

November 2, 1997 20Chapter 4 -- R. A. Volz -- assistance -- Mingjuan Cui

Reconciling Different Metrics

November 2, 1997 22Chapter 4 -- R. A. Volz -- assistance -- Mingjuan Cui

Measures of Software Quality

Correctness is the degree to which the software performs its required

function. the most common measure for correctness is defects per KLOC

Maintainability the ease that a program can be corrected adapted if the environment changes enhanced if the customer desires changes in requirements based on the time-oriented measure mean time to change.

November 2, 1997 23Chapter 4 -- R. A. Volz -- assistance -- Mingjuan Cui

Measures of Software Quality (Cont’d)

Integrity to measure a system’s ability to withstand attacks (both

accidental and intentional) on its security threat and security are defined

integrity = sum [ 1 - threat * (1- security)] Usability - an attempt to quantify “user friendliness”

physical/intellectual requirement to learn time required to become moderately efficient the net increase in productivity user attitudes toward system

November 2, 1997 24Chapter 4 -- R. A. Volz -- assistance -- Mingjuan Cui

Defect Removal Efficiency

A Quality Metric That Provides Benefit at Both the Project and Process Level

DRE = E / ( E + D )

E = # of errors found before delivery of the software to the end user

D = # of defects found after delivery

More generally,

DREi = Ei / ( Ei + Ei+1 )

Ei = # of errors found during SE activity i

November 2, 1997 27Chapter 4 -- R. A. Volz -- assistance -- Mingjuan Cui

METRICS

CLCS Metrics PhilosophyPhase 1: Provide a mandatory, nearly automated, metrics foundation to

track lines of code and errors.

Phase 2: Provide additional high-return metrics with recognized value.

Schedule metrics (milestones)Additional S/W Problem metrics (actuals, trends,

prediction)Defect correction metricsRun-time analysis metrics (McCabe tools, automated,

COTS)Phase 3: Be driven to additional metrics only by absolute need.

November 2, 1997 28Chapter 4 -- R. A. Volz -- assistance -- Mingjuan Cui

METRICSSystem Software

Milestones Redstone CIT

Complete

Thor CIT Complet

e

Atlas CIT

Complete

Month/Year Sep-97 Oct-97 Nov-97 Dec-97 Jan-98 Feb-98 Mar-98 Apr-98 May-98 Jun-98 Jul-98 Aug-98Software Size (KSLOC) 377.2 377.2 383.3 388.1 450.2 554.3

Actual Size of Executable Code 214.1 214.1 218.2 221.3 250.6 319.3 0.0 0.0 0.0 0.0 0.0 0.0

Code Delivered Comments 163.1 163.1 165.1 166.8 199.6 242.1 0.0 0.0 0.0 0.0 0.0 0.0Razor Issue Closure TOTAL

Issues Opened Urgent 9 3 2 0 4 5 0 0 0 0 0 0 23

(this month) Critical 54 19 8 1 16 53 0 0 0 0 0 0 151

Major 60 16 20 5 28 26 0 0 0 0 0 0 155

Minor 57 17 6 0 13 37 0 0 0 0 0 0 130Total: 180 55 36 6 61 121 0 0 0 0 0 0 459

Issues Closed Urgent 6 6 1 1 3 2 0 0 0 0 0 0 19

(this month) Critical 36 26 11 0 13 12 0 0 0 0 0 0 98

Major 39 24 12 4 14 10 0 0 0 0 0 0 103

Minor 27 17 19 5 2 16 0 0 0 0 0 0 86Total: 108 73 43 10 32 40 0 0 0 0 0 0 306

Current Issues Open: Urgent 3 0 1 0 1 4 0 0 0 0 0 0

Critical 18 11 8 9 12 53 0 0 0 0 0 0

Major 21 13 21 22 36 52 0 0 0 0 0 0

Minor 30 30 17 12 23 44 0 0 0 0 0 0

Total: 72 54 47 43 72 153 0 0 0 0 0 0

Error Density: Issues / KSLOC 0.84 1.10 1.24 1.25 1.35 1.44