metrics based scheduling

8/2/2019 Metrics Based Scheduling

1/53

DACS Gold PracticeTTMM

Document Series GP 25 V 1.0Metrics-Based Scheduling Last Updated: 1/30/07

SOFTWARE ACQUISITION GGGOOOLLLDDD PPPRRRAAACCCTTTIIICCCEEETMTTMM

METRICS-BASED SCHEDULING

FOCUS AREA: MANAGEMENT - MEASUREMENT

DESCRIPTION OF THE PRACTICE:

SUMMARY DESCRIPTIONMetrics-based schedulingis about establishing realistic software development or maintenanceschedules based on accurate estimates of software size and effort. The practice necessitatesuse of a minimum set of four metrics (namely, software size, effort, time/schedule and quality)coined by the Software Engineering Institute (SEI) as the core metrics for softwaredevelopment, along with a productivity metric that is derived from the core metrics and is oftenreferenced as the fifth core metric [Putnam, 2003].

This practice is applicable to both new software development and maintenance projects. Itsscope is very broad, in terms of both content and implementation, since realistic scheduling iscontingent upon the successful integration of several other principles, processes and practicessupported by the appropriate infrastructure and data collection activities. These include:

Establishing a goal driven metrics program that supports the organizations goals as wellas individual project goals

Implementing good requirements engineering practices

Getting accurate estimates of product size early for use in developing other estimatesand tasks that will comprise the project baseline and verifying the feasibility of the projectgiven the environmental constraints

Storing estimates, along with project metadata in a project repository for use in futureestimating and planning

Capturing and quantifying process productivity

Implementing risk management early

It is seldom successful when implemented in an intermittent fashion. It requires a managementcommitment at the organization level that crosses project boundaries, although it may be startedwith one or two pilot projects.

It is important to have valid core metric data (definitions and details provided in the body of thisdocument) from a few completed projects, or access to appropriate benchmark data, in order tobegin to develop confidence in the use of metrics for planning and decision-making. This takestime (the development cycle of the projects) unless such data from previous projects alreadyexists in an accessible form. Thus, a two or three-year window is a reasonable timeframe forestablishing and institutionalizing this practice if you are starting from scratch (with no historical

data). Global repositories of project information exist and can be used to jump-start the processin cases where there is little historical project data within the organization.

Estimation of software size and effort plays a significant role in metrics-based schedulingbecause these metrics are generally used for estimating the cost and schedule as well asplanning the development and establishing a baseline. Estimation accuracy can thus have amajor impact on the ability of the organization to establish and then meet its contractualobligations of delivering a quality product on time and within budget. Much of the success of thispractice stems from managers and other decision-makers developing confidence in the validity

1


2/53



and accuracy of the core metrics they are using. The data needs to be strong enough to allowthem to make critical decisions relating to the feasibility of what is being proposed and tounderstand and communicate the risks associated with such decisions. Accurate project data,from past and current projects, provides the significant evidence they need to support theirdecisions or recommendations.

Another important concept of metrics-based scheduling is understanding the interdependenciesof the core metrics. These interdependencies are often referred to as the software equation, amathematical equation supported by actual project data collected from thousands of projects for aperiod of more than 25 years. Refinements and variations (including constants, factors andexponents) of the software equation form the basis for most of the parametric software estimationmodels used today. Each model makes a set of assumptions, and requires some initial input thatis related to the subject project, or the development environment in which the project will occur.Therefore, it is important to understand these aspects of each model in order to determine whichmodels are appropriate for your particular situation. The body of this document contains a moredetailed discussion of the software equation and its various implementations in parametricmodeling.

The literature cites the following benefits from implementing metrics-based scheduling. Increased likelihood of project success (quality product, on time, and within budget)

Satisfied customers

Improved communication among stakeholders

The capability to acquire software on the basis of cost per delivered functional unit (incontrast to a fixed price contract)

Some of the key practices, identified in this document are summarized as follows: Do not let the lack of a formal scope specification stop you from doing an initial project

estimate.

Merely having core metrics does not guarantee success. You must use them forguidance and decision-making.

The core metrics are interdependent, and should always be addressed together. Every software project has a minimum development time but there is no maximum

development time.

Work to realistic project expectations

Recognize the impossible; be prepared to stop projects

Recognize and use industry specialists such as estimation (software sizing and costestimating) and domain experts

Consider acquiring software on the basis of cost-per-unit-delivered

Establish a metric mindset

Monitor estimation accuracy

Define measurements in a top-down fashion

Involve the stakeholders (participants to the measurement process)

Inform/communicate with the team

2


3/53



DETAILED DESCRIPTION

In 1998, the Software Program Managers Network [SPMN 1998] identified Metrics-basedScheduling and Managementas one of 16 practices critical to successful software developmentand provided the following definition:

Metrics-based scheduling and management is concerned with making cost and schedule

estimates based on empirical data and then monitoring the project for process control as itprogresses using both product and process metrics. An important goal of the practice is toidentify problems early while there is still time to resolve them or make adjustments.

Thus, the scope of the practice, as defined by the SPMN, is very broad, in terms of both contentand implementation, since it encompasses both planning for and management of the projectthroughout the software development life cycle. This practice has dependencies with many othereffective practices, which are the essence of project management, as illustrated in Figure 1.

The DACS has elected to break the content of the original practice, defined by SPMN, into twoseparate but related Gold Practices. One practice, the subject of this Gold Practice, which iscalled Metrics-based Scheduling, focuses on the estimating and planning activities that arenecessary to create a realistic schedule and baseline for a project. The other, called Metrics-

based Management, focuses on the project monitoring and control activities that are part ofproject execution.

All of the project management activities identified in Figure 1 are interrelated (as noted by thedotted connecting lines) but the focus (marked as 1 in Figure 1) ofMetrics-based Schedulingisprimarily on the relationship of measurement and analysis to the estimating and planning

Figure 1: Context Focus for Metrics-based Scheduling

3


4/53



activities that occur in the initial stages of the project and produce the project schedule.Measurement and analysis, along with the outputs of estimating and planning, (see primaryrelationships marked two and three in Figure 1), in turn, affect project monitoring and controlduring execution, which are the focus ofMetrics-based Management. Other relationships andinfluences, such as requirements management (engineering) and risk management, arediscussed as appropriate later on in this document.

Metrics-based schedulingis scheduling based on accurate estimates of effort, which are, in turn,based on estimates of software size (the amount and complexity of the software work product)and an understanding of organizational productivity. It is knowing when a proposed completiondate is not possible and having appropriate data to prove it. It is forward scheduling based onreliable size and effort estimates rather than backward scheduling driven solely by a desiredcompletion date or other constraint. It is reputation building; it is a mindset that approachesscheduling based on the size and complexity of what is to be built, while considering therequested completion date as a constraint. It is being able to predict with confidence the impact ofdecisions made with respect to product or process. It requires having confidence in estimates ofthe core metrics (size, effort, quality and schedule) and the productivity measures that are derivedfrom the core metrics of past projects.

Perhaps the best way to describe metrics-based scheduling is to contrast it with time-based

scheduling (sometimes called backward scheduling or reverse scheduling) as presented in Table1. Some may argue that all scheduling is, by definition, metrics-based, because it involvesscheduling activities over a period of time. Although time is a metric, when time is not derivedfrom or related to the size, or the effort required for developing the software product, then suchscheduling is not metrics-based.

Table 1 Comparison of Time-based and Metrics-based Scheduling

Time-based scheduling Metrics-based schedulingScenarios

The customer wants the project completed by aspecific date. Developer commits to thecompletion date. Scheduling progresses back

from the finish date, theoretically, along acritical path. The result is that the scheduledictates how much time is allowed fordeveloping each of various functions orcomponents of the system, rather than anyanalysis of effort required based on thefunctionality to be delivered. The developerexpects to add more resources as necessary toensure that each component is developedwithin the scheduled time period. Managersmay expect staff to work additionalundocumented hours to ensure adherence tothe schedule during development.

Typically, this type of scheduling is associatedwith a fixed cost that includes a reserve toaccommodate unforeseen needs such asadditional resources.

The customer wants the project completed by aspecific date. Contractor analyzes the productrequirements to determine an estimate of size.

Then, using process productivity data, or otherdata from past projects, the contractorestimates the effort (person hours) required tocomplete the project. Once the effort is known,the contractor develops the critical path andvarious scheduling approaches to see if it isfeasible to complete the project by the daterequested by the customer, and at what costand risk. If yes, contractor reviews with thecustomer the various approaches with respectto risk and cost and they reach agreementabout the scheduling strategy. If not, thecontractor goes back to the customer to

present the reality of the situation and perhapsnegotiates less functionality or an incrementalrelease strategy instead.

4


5/53



Table 2 Comparison of Time-based and Metrics-based Scheduling continued

Time-based scheduling Metrics-based schedulingEffects

May work with small projects of shortduration where accelerated work

schedules will be acceptable to thestaff for short periods Provides no basis for determining

overall productivity unless theorganization tracks actual size andeffort (including uncompensated effort)

Breaks down as the product size andcomplexity increases

Is often perceived as necessary tosustain business

May result in loss of expertise overtime because skilled software peopledo not wish to remain in such anenvironment

Effectiveness depends on havingaccurate size and effort metrics for

past projects, or benchmarking againsta repository of similar projects withinthe same domain, in order to determineprocess productivity

Works for projects of all sizes Provides early identification of

schedule problems Provides realistic time period for

accomplishing specific tasks of theproject

Establishes good customerrelationships

Builds a reputation for delivering quality

products on time Reduces project risks (cost and quality) Supports reduction of staff turnover

Metric- based scheduling is part of a management strategy developed with a metric mindset thatis committed to using appropriate metrics for management decision-making and control ofprojects of all sizes.

ACTIVITIES OF METRICS-BASED SCHEDULING

Metrics-based schedulingconsists of planning and estimating activities, which involve earlycalculation (estimation) of size and projection of costs and schedules from empirical patterns, inorder to verify the feasibility of the project and to establish a realistic project baseline and plan for

moving forward.

These planning and estimating activities represent a subset of the activities described in theProject Planning Process Area of the CMMI [Chrissis, 2003], which are listed in Table 2.

Table 3: Estimating and Planning Activities Identified in the CMMI Project Planning Process Area

[Chrissis, 2003]

SG 1 - Establish Estimates

SP 1.1-1 Estimate the scope of the project

SP 1.2-1 Establish estimates of work product and task attributes

SP 1.3-1 Define the project life cycle

SP 1.4-1 Determine estimates of effort and cost

SG 2 - Develop a Project Plan

SP 2.1-1 Establish a budget and schedule

SP 2.2-1 Identify project risks

SP 2.3-1 Plan for data management

SP 2.4-1 Plan for Project Resources

SP 2.5-1 Plan for needed knowledge and skills

5


6/53



SG 2 - Develop a Project Plan- continued

SP 2.6-1 Plan stakeholder involvement

SP 2.7-1 Establish the Project Plan

SG 3 - Obtain commitment to the Plan

SP 3.1-1 Review the plans that affect the project

SP 3.2-1 Reconcile work and resource levels

SP3.3-1 Obtain plan commitment

Although all of the activities identified in Table 2 are important to project planning, the activities ofestimating project scope (SP 1.1), establishing estimates of work product (SP 1.2), establishingestimates of effort and cost (SP1.4), schedule and budget (SP 2.1), and developing the projectplan (SP 2.7) are the focus of this Metrics-based Schedulingpractice because they involveestimating the core metrics and using them to develop the resulting project baseline.

Figure 2 presents a detailed process flow of these planning and estimating activities to show thesequence in which they are performed along with their inputs and outputs.

Figure 2: Planning and Estimating Process Flow

6


7/53


8/53



Table 4: Comparisons of Physical and Functional Size Measures

Physical Size Measures Functional Size Measures

Physical size metrics count some physicalentity associated with the software productsuch as Source Lines of Code (SLOC), numberof requirements statements, number ofsoftware configuration items (CSCI), pages ofdocumentation or other entities. SLOC aresometimes used as a measure of the lengthof the software.

Functional size metrics quantify thefunctionality and complexity of the softwareproduct based on a standard or widely usedconsistent set of counting guidelines. Thebase measure is a Functional Size Unit (FSU)and each specific functional sizingmethodology identifies its FSU with a uniquename. For example, IFPUG, the predominantfunctional sizing methodology for businessapplications, calls its base measure a FunctionPoint (FP). More recently, a methodologycalled COSMIC FFP has emerged that wasdesigned for quantifying the functionality of realtime and embedded systems. Its basemeasure is called COSMIC FSU, ( CFSU). Eventhough the FSU serves as a standard unit ofmeasure, this does not mean that a single FSUmaps one-to-one with a specific softwarefunction, or that the FSU of one method isequivalent to the FSU of another method.

Because we have evolved the way we build software, the physical size measures (particularlySLOC) may not be as useful as they once were in many organizations. The boundaries of asoftware product have expanded beyond the organization. We

Use multiple programming languages in the same project

Use code generators rather than write the code

Develop pieces of software systems in concert with other organizations

Reuse code

Integrate existing software components or systems rather than develop software fromscratch

Many organizations are migrating to Functional Size Measurementfor the following reasons:

It works for all types of software (scientific, business apps, web portals, embeddedsystems, etc.)

It works for all types of projects (new development, enhancements, maintenance, etc.) It is language independent

It is technology independent It is repeatable (consistent) two analysts independently estimating the same software

product, using the same resources and the same sizing methodology, will arrive at thesame size within a few percentage points ( 5%)

It produces statistically significant results It can be applied early in the development life cycle

Although the concept seems simple, estimating size is really quite difficult and often requires theengagement of a skilled professional who is experienced in using the selected sizingmethodology.

8


9/53



Size is the primary input to many models used to estimate effort, cost, and schedule, but it is alsoan estimate. Therefore, achieving accuracy of the size estimate is critical to the estimationprocess since so many attributes are based on it. Size is also an important part of the functionalbaseline because the product is affected by requirements changes. Tracking size throughout theproject, particularly when there are major requirements changes, provides visibility into andheightens awareness of scope creep. Awareness can trigger better control over the project byadjusting effort, schedule, and/or cost before the project reaches a crisis state.

Having an accurate size estimate enables the developer to better estimate effort and cost earlierand with fewer resources. However, the size estimate, by itself, is of little value. We must use ittogether with other project attributes and historical data from previous projects. Projects of similarsize will require similar effort when other project attributes match. The correlation betweenproduct size and effort might be strong or weak depending on the match of other projectattributes, but as an organization builds its repository of project metadata, it is possible to infer thestrength of the relationship based on key project characteristics. Although it takes time to achievethis state, the benefit is that of achieving sound effort and cost estimates based on accurate sizeestimates in a minimal amount of time. In addition, the project repository can be used to estimatethe size of a new project based on comparisons with previous projects.

The critical elements in choosing the size metrics your organization should use are: Accuracy Over time, your estimates should come closer and closer to the actual size

determined at project completion. Accuracy should get better with each new project.

Consistency Choose size measures that can be applied consistently across allprojects, now and in the future. If you are no longer using Ada, having size estimates interms of the number of SLOC for Ada is not very useful unless you can convert them toan equivalent size relating to the language you are currently using. If you are usingmultiple languages, or doing COTS integration, SLOC is probably not as useful as afunctional size metric. If you will never have a need to benchmark against externalorganizations, then focus on achieving consistency within your organization. The key isto identify the size metric(s) that provide the best correlation with effort within yourorganizational culture and development process.

Predictability Having an accurate size metric should enable you to predict the effortand cost when used in conjunction with other project data, and to know the degree ofcorrelation. Track the correlation as you proceed. Functional size may be a goodpredictor in your particular development environment, but not in another organization.You have to focus on finding what works for you. It may be that you need more than onesize metric.

Earliest usage -- Choose a size metric that you can use early in the project withreasonable accuracy. This allows you to assess the feasibility or reality of completing theproject and meeting customer expectations before you have invested significantresources in the project. Having a size estimate is of little value for project planning if wehave to wait until after the design phase is complete, or after testing before we canmeasure size.

Unfortunately, in many organizations, developers often skip the sizing activity during initialplanning. They may develop the WBS asserting that it reflects the product size through the taskcomponents that identify software functionality to be developed. Many developers tend to equatesize with effort or cost. In fact, they seldom refer to product size at all. If one asks, How big isthe product?, both the developer and the acquirer are likely to answer that it is a 3-year effort, ora $2 million dollar project. They are not likely to say that it is estimated to be 600,000 SLOC or2,000 functional size units (FSUs). Many developers perceive that a quantitative size estimate isnot obtainable in the initial stages of project planning. Acquirers concerned with the cost to meetthe requirements rather than how functionality is quantified, may perceive early sizing of the

9


10/53



product as a waste of time. The developer focuses on building the WBS and using that toestimate effort and schedule. Many organizations capture a product size estimate only uponproject completion. The mindset described in this paragraph needs to be overcome if you elect toimplement metrics-based scheduling. Although the entire project team needs to be aware of theproduct size and how it might be changing, the project should use skilled professionals who aretrained in the appropriate size estimating techniques to do the estimating functions. This resultsin estimates that are more consistent over time. Details about specific estimating techniques andestimating practices are addressed under the Estimating Techniques section of this document.

Step 3 - Estimate Effort

Effort is the amount of work expended in order to complete the software; it is typically measuredin person hours (PH), person months (PM) or person years (PY) depending on the scope of theproject and the granularity of information that is available.

Note, at this point in the process, our effort estimate will be a nominal estimate, an estimate thatis in the ball park but not exact; it assumes variations of staff experience and skill exist. Itspurpose is to assist with determining both schedule and cost to assess the feasibility of theproject. We may know that if we used all highly experienced staff, they might do the work in lesshours than an inexperienced staff would take, but we also know that they might cost a great dealmore. That is not the intended purpose of this initial effort estimate. The distribution and

allocation of effort among staff resources is addressed as part of scheduling.There are several ways to estimate effort and the methods you choose will depend on theavailability of information and its accuracy. You should not rely on a single method but insteadestimate using at least two methods and compare the results. Major differences betweenestimates should trigger investigation as to why they differ, and some resolution before goingforward with using the effort estimates for deriving a schedule. Effort estimation approaches,described in the literature, typically fit into one of the four groups identified in Table 4. The bestway to determine the effectiveness of your chosen method(s) is to monitor the estimationaccuracy as you move forward with projects. You may find that what works best for you is totailor one of these approaches, or create a composite of these approaches. Always keep in mindthat the effort estimate, like any other estimate, is not free. Resources must be expended toderive the estimate and those resources translate into time and cost. It may not be efficient toexpend significant time and resources to get an estimate that is extremely accurate, at a time

when you do not need that degree of accuracy. The intended use of the effort estimate shouldalways drive the decision regarding the appropriate estimating methods. It is noteworthy that withthe exception of the bottom-up method, these approaches depend on having some data and/orknowledge of previous projects. The accuracy of the effort estimate depends on the accuracy ofthe historical data to some extent.

Table 5: Effort Estimation Approaches

Approach Description Usage Considerations

Top-down Comparing high-level projectcomposition with previous orsimilar projects and using theeffort from those projects as astarting point. Adjust the effortup or down based on expert

judgment of the risks anddifferences between the currentproject and the selectedcomparison projects.

Easiest and perhaps most acceptablefrom the perspective of the technicalstaff, but its value depends on theamount and accuracy of the metadata forprojects in the comparison, and theability of the estimator to makeadjustments consistently.

Subjective since it depends on thedomain knowledge of the estimator andthe organizational culture.

Breaks down as projects increase in sizeand complexity

10


11/53



Table 6: Effort Estimation Approaches contined

Approach Description Usage Considerations

Bottom-up Detail is added to the WBS to

decompose the project into tasksof anticipated short duration.Then individual developersestimate the effort to completeeach task. The rule of thumb isthat if a task takes longer thantwo weeks it should be furtherdecomposed. The aggregate ofthe individual task estimates isused as the project effortestimate.

Use when historical data from past

projects is not available.

Use as a reality check against any of theother methods.

Studies show that developers tend tounderestimate when using this or anymanual approach.

The organizational culture can impacthow individuals estimate, and thus, thevalidity of the estimate (see EstimationGames)

Parametric

Modeling

Formal implementation of the

Software Equation using a toolthat accepts project size,complexity and othercharacteristics, and calculatesthe effort based on a historicalrelationship among thecharacteristics of past projectsfrom many organizations.

The tool must be calibrated for the

unique characteristics of thisorganization.

Requires skilled estimation specialist withexperience using the tool and a thoroughknowledge of the metric data collected bythe organization

DerivationfromEffectivenessParameters

Applying a version of thesoftware equation to calculate theeffort using the size estimate ofthe current project, and anestablished productivity rate (orcomparable effectivenessmeasure) from previous projectswith similar characteristics

Requires the least time and resources ofall approaches

Usefulness depends on the strength ofcorrelation between size and effort

Depends on the validity of theproductivity rate, which is derived fromaccurate measures of actual effort forpast projects and consistent sizemeasures gathered on completion ofeach project

Confidence in the estimates is based on the rationale for the estimation model selection and thenature of the data. Sometimes, historical data does not apply, such as where efforts areunprecedented, the type of task does not fit available models, the project leadership haschanged, or the development environment has changed.

There are now some worthwhile methods to derive accurate effort estimates easily and quickly ifyour organization collects functional size measures on its completed projects along with projectmetadata. It is important to identify the risks associated with using the selected estimation modeland the uniqueness of the project to ensure a common understanding of any assumptions thatare made in the planning stages of the project.

Ideally, when the organization has sufficient history to derive organizational productivity rates forprojects based on specific attributes, the effort estimate can be calculated from the size, but thispresumes a consistent mature software development process is in place.

11


12/53



If I estimate the size of this software product to be 1000 FSUs and, historically, my team hasproduced at a rate of 8 FSUs per person month in this environment, then, with simple arithmetic(1000/8) I can calculate a realistic effort estimate in a matter of minutes. As a starting point, I canestimate it will take 125 person months to develop this system. This estimate can be used as aconsistency check (reality check) against other effort estimating techniques. If a bottom-upestimate of effort results in only 75 person months, then it should trigger the need for furtherinvestigation before committing to a baseline since the difference in the two estimates issignificant and a significant difference in effort results in a significant difference in costs as well.

Studies have shown that we (practitioners) tend to underestimate the effort when we are usingmanual (top down or bottom up) approaches.

If you have been involved in tracking effort, but not estimating it initially, there is a tendency tomistrust the effort estimate, perceiving that perhaps it does not account for the variation ofindividuals within the team. You might tend to think that if we used more of person A and less ofperson B the effort would be less because person A is more experienced than person B is.Remember that the productivity rate you chose to use was based on actual past projects, withsimilar or same staffing scenarios and the same environment. Therefore, that variation is alreadyaccommodated in the productivity rate used for the estimate.

You may find that the reality of your situation is that you (the project manager) are given the

funding amount and instructed to make the effort estimate fit the funding. You allocate the fundingamong the tasks of the WBS and adjust the effort in each case. Then, during project execution,effort is tracked in accordance with this baseline. The project team may be working manyadditional hours to complete the project but the extra effort is never recorded and not visible oncethe project is complete. There is a big difference between the actual effort and the documentedeffort. If the documented effort is stored in the project repository and not the actual effort, it canskew the project team and organizational productivity metrics and render them useless.

There must be a commitment across the organization to capture and record real/actual effort dataif your project repository is to have any real value. Your metrics repository is not a showcase ofmarketing tidbits. It is a tool for decision-making. Your organization needs to take theresponsibility for ensuring that the data posted there is accurate and factual, not inflated.

Step 4 - Estimate Schedule

Scheduling involves distributing an effort estimate among staff resources and across a calendartimeframe to ensure that both product and performance or operational requirements will be met.It is also the process of planning the appropriate level of staffing. If our scheduling analysisdemonstrates that we need a minimum of eight developers, but we only have five, we know earlyon that we must adjust the staffing allocation or risk failure. Schedule is typically expressed incalendar weeks, months, or years, depending on the scope of the project and the desired level ofgranularity.

One can use the following technique as a reality check against schedules derived by other meansto alert the planner to major inconsistencies early in the planning stages before committing to aparticular schedule baseline.

The metrics-based effort estimation process (based on product size) provides us with a nominaleffort estimate, perhaps bounded by a meaningful range based on the level of risk, or the

uniqueness of project attributes (125 person months + or 10%).

Assuming a size-related effort estimate of 125 person months, if our team consists of five people,and they each work the same amount of time concurrently, then the minimum calendar schedulewould be 25 months. This is typically the minimum schedule because it assumes that (a) thesoftware critical path can be accomplished by this scheduling scenario and that (b) theexperience mix of the staff can address all of the project tasks. In addition, if any one personworks less than full time, the schedule would have to be stretched to offset the additional time ofanother person. This is the starting point for scheduling, but more importantly, it provides a realitycheck against any specified time constraints against the project such as a customer requirement

12


13/53



or expectation that the project be completed within 12 months. This tells us immediately that it isnot possible for us to complete the project within the expected timeframe, given the currentstaffing plan. Having just three metrics (realistic size estimate, valid project team productivityrates and the number of staff) enables us to get the schedule estimate in a matter of minutes. Itis not intended to be exact. Its purpose is to let us know the minimum schedule and provide astarting point for refinement. We would next have to ask, If we double the staff could weaccomplish the project within the 12-month window? That answer would come only from analysisof the software critical path and the skills and experience needed to perform the tasks.

It is extremely important to define for your organization exactly what constitutes a person hour,person-week and person month. That information needs to be standardized across the broadestpossible range of groups within the organization. Assuming a person hour is defined as oneperson working on the project for one hour, then a person-week is likely to be viewed as a personworking 40 hours on the project, and a person-month is one person working 160 hours on aproject (40 X4). However, is the person who is full time on the project really working on theproject for 40 hours? Are they spending perhaps one hour a day reading email or othermiscellaneous tasks that are not totally tied to the project? What happens to the person monthconcept when holidays occur? Long weekends? Vacations? How many person hours of eachmonth are realistic. Uncompensated overtime is also a big problem. In many organizationssoftware professionals are expected to work overtime (uncompensated) whenever there is

pressure to meet a deadline or milestone. Because it is uncompensated, it is usually not trackedor recorded. Thus, it is never reflected in the effort data reported for a project. The assumptionsrelated to the schedule metrics need to be documented and stored with the data. If yourdepartment uses 160 hours as the standard person month, but another department uses 176hours as the standard person month, then there is an impact on scheduling caused by thevariation of metric data used to derive the schedule.

Many parametric models have been developed to aid in estimating cost and schedule; they arebased on historical data from many projects that may or may not be pertinent to your project.Because of this, it is important to use multiple models and compare results to ensure a higherlevel of confidence in the estimate. Ultimately, any such models should be calibrated using yourown repository of historical data. This is an area where it may be appropriate to bring in anindustry specialist, trained to work with such models to maximize their value for decision makers.

Obviously, there is more to scheduling than the simplistic perspective presented here but thefocus of this practice is in knowing when and how to use the core metrics to derive the initialschedule and to create this valuable comparison point for assessing the validity of any schedulethat might be derived from other methods.

The larger the project (product plus related tasks) the more important it is to establish this initialmetrics-based schedule as the starting point for further schedule refinement.

Step 5 - Estimate Costs

In metrics-based scheduling and management, costs are derived from effort and scheduleestimates which, in turn, are based on software product size estimates combined with historicalproject data.

Note that in this planning context, the planner wants to estimate as accurately as possible whatthe actual costs will be. This is not necessarily the same as a cost estimate performed for biddingpurposes. The cost drivers that apply to bidding estimates tend to address a particular businessstrategy. Hopefully, the two types of cost estimates are well aligned, but they serve differentpurposes. The purpose of the estimate should be carried along with the estimate in the projectrepository.

Just as with effort and schedule, an accurate cost estimate enables the planner to determine thefeasibility of meeting cost constraints imposed for the project. If the cost estimate is $500,000and the acquirer has set a ceiling of $200,000 then it is highly unlikely that the developer will beable to bring the project in on time for $200,000 without sacrificing quality and/or functionality.

13


14/53



Having this awareness early in a project, and having supporting evidence ( based on the body ofhistorical data which influence the estimates) enables the planner to communicate the reality ofthe situation to the leaders, and the leaders to make informed decisions about whether toproceed, and if so, how to proceed. Typically, when there is sound evidence and confidence inthe estimating process, the acquirer and developer work together to de-scope such a project toestablish a more realistic set of requirements, or to adjust the development cycle in some way.

There are many cost estimating models and approaches to consider and some are most effectivewhen used by a skilled professional. Generally, the larger the project, the more important it is toemploy a skilled professional to do the estimating. References for cost estimating techniques areprovided later in this document along with additional reading.

Step 6 - Do a Reality Check

The reality check essentially asks the following types of questions:

Are the estimates we have developed realistic?

How do our estimates align with budget availability?

Do we have more information now that should trigger re-estimating of some elements?

Have we considered all the risks and can we live with the level of risk that remains?

This is a critical decision point in the process and will often uncover the very elements that arelikely to cause a project to fail at a later point in time. It is far less costly to perform the realitycheck than to wait in anticipation of resolving issues during project execution.

Note that you may need to negotiate functionality and repeat iterations of steps 1 5 as neededto establish the estimates and WBS that comprise the Project Baseline. When the baseline isestablished you can proceed with developing the Project Plan, or, you can evolve the ProjectPlanas you proceed through the necessary iterations.

One should note that the acquisition process and the contracting policies may actually be barriersto metrics-based scheduling in that they might not have provision for making adjustments basedon the reality check. Consider alternative acquisition strategies and contractual arrangementssuch as performance-based contracting that provide greater flexibility for making realistic

adjustments.Step 7 - Develop the Project Plan

The Project Planis a formal, approved document that the developer uses to manage and controlthe execution of the project. It is based on the project requirements and the establishedestimates.

It documents and ties together all activities that are needed to ensure successful development ofthe product. It serves as a guide for all project participants and promotes a commonunderstanding of how the project will progress and what processes are to be applied.

The project plan includes the initial budget and schedule that is based on the developed

estimates. Sometimes the schedule is event-driven rather than calendar based. Schedule-driven

is most useful for projects that you have done before (for another client, for example), andeveryone on your team knows exactly what to do to get the job done. Schedule-driven tends to

ensure that things get done according to a known schedule, on time, and on budget, but it is

problematic when a project involves a lot of uncertainty.

Event-driven schedule is required for projects that have never been attempted before or that

involve a lot of uncertainty. Attempting to stick to a pre-defined calendar based schedule on such

a project is very difficult, as new and unforeseen obstacles tend to pop up and slow your

14


15/53



progress, leading to continual adjusting of the schedule, and confusion among your client(s).

Event-driven methodology breaks the project into sub-tasks, and delineates the order in which

sub-tasks must be accomplished before moving on to the next (subsequent tasks are usually

dependent on the completion of preceding tasks). An event-driven methodology does not provide

a fixed schedule or completion date, but it does tend to ensure that a complex job with a lot of

uncertainties is done right the first time. It is effective in dealing with project risk, especially

when resources are limited.

The plan thus details schedule dependencies as well as budget. It identifies major milestones,

schedule assumptions, constraints and task dependencies.

The schedule and budget are tied together because the budget must support the resources thatare allocated to complete the tasks in the schedule. Establishing the projects budget andschedule includes the following:

Defining the expected availability of resources and facilities

Determining time phasing of activities

Determining a breakout of subordinate schedules

Defining the dependencies between activities

Defining the schedule activities and milestones to support accuracy in progressmeasurement

Identifying milestones for delivery of product to the customer

Defining activities of appropriate duration

Defining milestones of appropriate time separation

Defining a management reserve based on the confidence level in meeting the scheduleand budget

Using appropriate historical data to verify the schedule

Defining incremental funding requirements

Documenting project assumptions and rationale

An important part of planning, which sets the stage for effective Metrics-based Managementactivities as the project progresses, is to allocate an appropriate portion of the budget to eachwork unit identified in the WBS. This allows tracking of progress by the accumulated value ofcompleted work units, thus integrating actual work performed with planned and actual spending.This is called Earned Value Management. Although tracking earned value occurs during projectexecution, it cannot be accomplished if appropriate project planning and budget allocation has notoccurred up front. This concept is discussed in detail in a separate Gold Practice titled Track

Earned Value available on the DACS gold practice website (see www.goldpractices.com).Additionally, the planner should incorporate a risk management plan, configuration managementplan and quality assurance plan into the project plan and obtain commitment to the plan. Forfurther details regarding these activities refer to the project planning process area of the CMMIguidelines [Chrissis 2003], and to the DACS Gold Practice on Formal Risk Management.

Another recent scheduling methodology, which is applicable for large complex projects, is calledCritical Chain Scheduling. It is innovative and, in many ways, contrary to the techniquesdescribed in the CMMI. It does not specifically address the use of the core metrics, but that doesnot preclude its usefulness in the execution of the project. The DACS topic area on Project

15
http://www.goldpractices.com/http://www.goldpractices.com/


16/53



Management (see www.thedacs.com) has a subtopic on Critical Chain Management thatprovides some informative articles about what it is, how it differs from more traditional schedulingtechniques, and how it should be implemented.

CONTRIBUTING PRACTICES

As indicated in the definition of the practice, Metrics-based Schedulingis, in reality, acompendium of the body of knowledge relating to the implementation of many other effectivepractices. In the context of the preceding pages, these other practices were identified asmechanisms that contributed to effective metrics-based scheduling. Now, in the remaining pageswe will address the salient points of those key practices and provide you with additionalreferences in each case. They are:

Estimating Techniques

Knowledge of the Software Equation (Core Metrics)

Measurement Program

Requirements Engineering

Risk Management

ESTIMATING TECHNIQUES

Note: The explanation of Estimating Techniques provided in the following paragraphs is derivedprimarily from John McGarrys book titled Practical Software Measurement: ObjectiveInformation for Decision Makers [McGarry 2002].

Estimation produces quantitative projections of key project attributes, which are the basis forinitial project plans; it is usually the first type of measurement analysis performed on mostprojects, often occurring early in the project before the project team is even selected. Itestablishes the overall funding and schedule commitments for the project, even though theseearly estimates are often imprecise. Poor estimates and misconceptions about the estimatingprocess often contribute to failed projects by leading to planning targets that are impossible toachieve.

Poor estimation can be attributed to the following factors:

Lack of estimating experience

Lack of historical data on which to base estimates

Lack of a systematic estimation process, sound techniques, or models suited to the projectsneeds

Failure to include essential project activities and products within the scope of the estimates

Unrealistic expectations or assumptions

Failure to recognize and address the uncertainty inherent in project estimates

Estimation involves using one measure to predict another; product functionality to quantify

product size, product size to predict effort; effort to predict schedule; effort to predict cost; productsize to predict the number of defects (quality).

There are four basic types of estimation approaches: parametric models, activity-based models,analogy, simple estimating relationships (see Table 5).

16
http://www.thedacs.com/http://www.thedacs.com/


17/53



Table 7: Types of Estimation Approaches

Parametric models Parametric models assume that a specific mathematical relationshipexists between size, effort, schedule, and quality and that therelationships are affected by measurable performance factors

(parameters) and are based on theoretical reasoning or analysis ofhistorical data. They take the form:

Effort =A x (Size)Bx C

Where Effort is expressed in hours or person months, Size representsthe amount of software functionality or product, A is a constant, B is anaggregation of performance factors that have a non-linear effect on theoutput, and C is an aggregation of performance factors that affectEffort in a linear manner. The non-linear nature of B causes thefactors to have more influence on larger software products than thosewith smaller products. Performance factors generally fall into twocategories, process and product factors. Estimation models provideas many as 100 factors, but organizations find that only a few factorssubstantially affect their performance.

Parametric models must be calibrated to data from the localdevelopment environment for best results. Calibrating all of theparameters takes a great deal of data, however, only a few data pointsmay be required to calibrate a models constants to local developmentconditions, with a resulting improvement in model accuracy.

Most parametric models assume that schedule is a function of effort.Few models support both effort/schedule estimation and qualityestimation.

Activity-BasedModels

These models are sometimes referred to as activity-based costingorbottom-up estimatingbecause they are derived from engineeringestimates of size, effort, and/or schedule for all products and activities

in a project. The low-level estimates for each individual activity orwork product are derived from (1) engineering judgment, (2) historicaldata, or (3) a combination of the two. Estimates are then aggregatedto produce a project-level estimate.

This approach requires detailed knowledge of the product andprocess. It should include effort for all products whether internal orexternal.

AnalogyApproach This is based on a comparison of the characteristics of the proposedproject with other previously completed projects. Data from similar, oranalogous, projects is the basis for the proposed projects estimates.Differences are identified and appropriate changes are made to adjustthe size, effort, schedule, and quality to fit the new situation. Anestimate can be generated from just one similar project; however, itrequires a detailed understanding of project contexts andcharacteristics for both projects.

17


18/53



Table 8: Types of Estimation Approaches continued

Simple EstimatingRelationships

(Simplified parametric approach) are based on use of local historicaldata instead of a comprehensive mathematical model. An examplewould be calculating Effort as the product of size and productivity,where the productivity rate is determined from past projects. Thesesimple relationships assume that all of the factors that affect effort aresimilar among past and future projects; therefore, they are seldomappropriate outside of the organization and domain that provided thedata.

Considerations for selecting an estimation approach are presented in Table 6.

Table 9: Considerations for Selecting an Estimation Approach

EstimationApproach

Assumptions andUnderstandings

Data Required MathematicalComplexity

Applicability

ParametricModels

General understanding ofperformance factors

Might not include all projectactivities (e.g. requirementsanalysis)

May assume minimum level ofdocumentation

Model is based on historicaldata from many organizations

Historical datafor calibration

Estimate of size

Complexstatisticaltechniques

Early in lifecycle

NewDevelopment

Large projects

Activity-BasedModels

Detailed process and productinformation is available (work

breakdown structure

Very detailedhistorical data

for a fewprojects

Arithmetic Afterrequirements

phase

Small andmediumprojects

Analogy Detailed project knowledge ofboth projects

Historical datafor at least onesimilar project

Arithmetic Small andmediumprojects

Stable staffingenvironments

SimpleEstimatingRelationships

General descriptive information Historical datafrom multipleprojects within

the organization

Simplestatisticaltechniques

Maintenanceprojects

Similar projects

Estimating Size, Effort and Schedule

Estimates are typically sequenced as follows: size, effort, schedule, and then quality.

The size metric relates to functional or physical size and sometimes both. The most prominentunit of measure for functional size is function points. Function points consider the software froman external perspective while source lines of code (typically used to address physical size)

18


19/53



consider software from an internal structural perspective. Function points can be counted early;source lines of code (SLOC) require knowledge of the potential software design. SLOC worksbest with the analogy and activity-based estimation methods. SLOC should be applied to thelowest degree of resolution that is economically possible.

Size estimates should be updated throughout the project because they, in turn, impact effort andschedule estimates.

When an organization has a significant project repository of valid data from past projects, it ispossible to calculate an effort estimate easily and quickly using the size estimate and aproductivity rate from past projects with similar characteristics. Effort is a function of size;schedule estimation is a function of effort combined with project constraints.

A primary mistake in estimating schedule is to ignore critical path dependencies. Paralleldevelopment and schedule compression have their limits. Although sound from a mathematicalperspective, they may be infeasible in reality. Scheduling involves estimating optimum effortdistribution or staffing levels but this is relevant only for large projects; many short-durationprojects and maintenance organizations operate with a fixed staff.

Table 7 describes some key resources for use in learning about estimating techniques.

Table 10: Some Resources for Learning about Estimating Techniques

Resource Description and Location

DACS Topic Area onSoftware Cost Estimation

This web page, which is continuously updated by the DACS,identifies a multitude of resources relating to cost estimating andestimating size, effort and schedule.

http://www.thedacs.com/databases/url/key.php?keycode=4

DACS Topic Area onMeasurement

This web page addresses measurement in general but hassubtopics focused on specific items such as functional sizing andvarious measurement methodologies that may be utilized tocollect the right data for estimating.


The Common SoftwareMeasurement InternationalConsortium (COSMIC) WebSite

This site provides the body of knowledge relating to the COSMIC-FFP method, which was designed for deriving the functional sizeof non-business software such as real-time and embeddedsystems.

http://www.cosmicon.com/

International Function PointUsers Group (IFPUG)

Web Site

This site provides the authoritative body of knowledge relating tofunction point counting and analysis

http://www.ifpug.org/

International Society ofParametric Analysts (ISPA)Web Site

ISPA is dedicated to the improvement and promotion of:

parametric cost modeling techniques and methodologies

risk analysis

econometricsdesign-to-cost

technology forecasting

cost management

http://www.ispa-cost.org/

19
http://www.thedacs.com/databases/url/key.php?keycode=4http://www.thedacs.com/databases/url/key.php?keycode=3http://www.cosmicon.com/http://www.ifpug.org/http://www.ispa-cost.org/http://www.ispa-cost.org/http://www.ifpug.org/http://www.cosmicon.com/http://www.thedacs.com/databases/url/key.php?keycode=3http://www.thedacs.com/databases/url/key.php?keycode=4


20/53



Table 11: Some Resources for Learning about Estimating Techniques continued

Resource Description and Location

International Software

Benchmarking StandardsGroup (ISBSG)

This group provides education and training, tools and other

resources relating to software estimation, benchmarking,productivity, risk analysis and cost and maintains a database ofproject data for more than 3000 projects. Developers can use thisdatabase to jumpstart their estimation activity when a local projectrepository is not available.

http://www.isbsg.org/

SEI Sage Web Site This web site provides specific definitions relating to size metricsinvolving source lines of code (SLOC)

Estimating is a broad discipline that goes far beyond the scope of this practice, and subsequentlythere are many good resources. Refer to the resources section of this document for additionalmaterial.

KNOWLEDGE OF THE SOFTWARE EQUATION (CORE METRICS)

The software equation, as defined by Putnam and Meyers [Putnam 2003], identifies the genericrelationships among some key metrics coined by the Software Engineering Institute (SEI) andothers as the core metrics. Specifically, people, working at some level ofproductivity, producea quantity of function (size) or a work product at a level ofquality(reliability) by the expenditureofeffortover a timeinterval. The interconnectedness means that when one is adjusted, theothers are affected. To understand the value of the software equation we must attempt tounderstand the core metrics and their interconnectedness.

Figure 5 identifies and illustrates the relationships among these important metrics. Some sourcesassert that there are only four core metrics, size, quality, effort, and schedule, but Putnam[Putnam, 2002] claims that productivity is also one of the core metrics.

Putnam defines these five core metrics as follows:

Size - Quantity of work or functionality (software product), usually measured in terms of sizesuch as source lines of code or functional size measures, that ultimately executes on thecomputers; a way of measuring the amount of functionality a project/product represents ---before we embark on it.

Quality Level of quality or reliability required for the product, quantified by measures suchas the defect rate (or its reciprocal, mean time to defect)

Time (Schedule) - the duration of the project, typically measured in calendar months or

weeks for shorter projects (often referenced as the schedule)

Effort the amount of work expended, typically measured in person-months or person-hours

Productivity the rate of progress, expressed in terms of the functionality produced for thetime and effort expended. The conventional measure of productivity has been source linesof code per person-month (SLOC/PM), calculated from past projects. In recent years withincreased reuse, and code generation, this measure has become less accurate and is beingset aside in favor of Functional Size Units per person-month.

20
http://www.isbsg.org/http://www.seisage.com/SoftwareSizeDefinitions.pdfhttp://www.seisage.com/SoftwareSizeDefinitions.pdfhttp://www.isbsg.org/


21/53



Figure 3: Core Metrics and Their Relationships

As illustrated in Figure 3 the basic tenets of the relationships among these core metrics are:

Size and quality estimates represent what must be done (the work) and therefore drivethe estimation of effort

Productivity (derived from performance on past projects) together with size, can facilitateestimating effort and can be used to identify the minimum possible schedule inscenarios where the number of staff is fixed

Effort estimates together with known project constraints drive the development of

schedule

Knowledge of organizational productivity enhances realistic scheduling

Cost is derived from both effort and schedule

Knowing something about the size (functionality) of the work product, together with the requiredlevel of quality supports the developer in estimating the effort that will be required. Quantifyingthat functionality in terms of a common metric such as source lines of code or functional sizemeasures facilitates estimating the effort, because the developer can compare it to past projects,or can use parametric modeling to get an estimate of the effort. In the absence of such historicaldata from past projects, the developer must rely on functional decomposition that is sufficientlydetailed to enable the technical staff to provide realistic estimates of the effort needed tocomplete each task. However, such estimates typically fall below what is actually needed for the

project.The software equation establishes a formal mathematical relationship among the core metrics.Table 8 presents the evolution of the equation from its generic concept to its formalrepresentation as defined by Putnam [Putnam 2003].

21


22/53



Table 12: Evolution of the Software Equation

Work Product (at a Quality level) = Effort over a Time interval at a Productivity level

Size(at implied Quality) = Effort x Time x Productivity

OrProductivity = Size/(Effort x Time)

Process Productivity = Size/[(Effort/)(1/3)

x Time(4/3)

]

Note that this definition of productivity is different from the conventional definition, P= size/effort.Therefore, we need to distinguish it in name as well. Putnam calls it Process Productivity, theproductivity of a project full of all kinds of software people operating over the period of the project.

Putnam [Putnam 2003] has refined the equation over time to reflect what is evidenced by dataspanning more than 25 years for thousands of projects from hundreds of companies, many ofwhom were not aware that such a formula existed when their data was provided.

However, it is not a law of nature; the exponents are approximations, not physical constants.Beta, , is a size-dependent parameter that has the effect of giving greater weight to the effortfactor in very small systems. The exponent for the effort term, (1/3), is necessarily less than one;the exponent of the time term (4/3) is greater than one, reflecting the relationship between timeand size. The major difference between conventional productivity and Putnams processproductivity is that he asserts (through mathematical derivation) that schedule is a factor inproductivity; this is not the case for conventional productivity.

Variations of this equation form the basis for most of the parametric models used for costestimating.

According to Putnam [Putnam, 2003], managers can use the software equation to carry out sixbasic functions relating to project and process control. They are:

1. Estimating time and effort2. Setting dates3. Supplying resources4. Monitoring work progress5. Assuring product quality6. Measuring process improvement

Some key concepts revealed by the software equation are as follows [Putnam, 2003]:

To get more work product (more functionality, measured by some metric of size) we haveto (1) increase the amount of effort, or (2) lengthen the development time, or (3) improveproductivity, or (4) augment some combination of effort, time and productivity

To get a product of higher quality (reliability) we must (1) increase effort, or (2) lengthendevelopment time, or (3) improve process productivity, or, (4) implement somecombination of these increases

To reduce effort and/or time, we must limit the functionality of the product (reduce itssize), accept less quality, or increase productivity

Improving productivity enables us to achieve more functionality or quality at the samelevel of effort and time

With improved productivity we can reduce the effort and time and still achieve the originalfunctionality or quality

22


23/53



With significant increases in productivity, we can achieve higher levels of the other fourmetrics ( reduced time and effort, greater functionality and quality)

Here are some of the key facts about the software equation, the core metrics and theirrelationships:

Acquire core measures from completed projects where the values of time, effort and size

are known (or can be collected). Productivity is thus obtained through simplecalculation. Because it is derived from measured attributes, it is as precise as thosemeasurements. Productivity should be calculated at the end of every project, but mayalso be calculated as soon as enough work has been completed to make the metricsvalid.

Productivity (derived from the software equation) stands for the productivity of the entireproject organization (not just the productivity of the code writers) during the time periodthe measurements cover

There is also a relationship between quality and the other core metrics but it is notnecessarily simple math. Although defects tend to occur at a rate derived from historicdata, the core metrics tend to influence the defect rate (typical measure of quality).Larger projects generate proportionally more defects; high productivity results in fewerdefects; allowing adequate time for development reduces defects.

Planning: Given size and process productivity, we can estimate project time and effortfrom the relationship between size and process productivity

Project Control: given the planning projection of time and effort, management cancompare actual achievement against what was projected, and act to harness an out-of-control project

Defect Tracking: Given a projection of the defect rate, derived from the other coremetrics, management can track defects found against those expected

Process Improvement: Given the productivity value on a series of completed projects,management can assess its progress toward increasing capability and determinewhether its improvement efforts are succeeding.

Merely having core metrics does not guarantee success. You must use them forguidance and decision-making.

The core metrics are interdependent, and should always be addressed together.

Every software project has a minimum development time. There is always a limit to howmuch the schedule can be compressed. It varies from project to project depending onsize, domain, developer skill, and developer environment. Determining that time, for anew project, involves keeping the core metrics on past projects and having at least arough size estimate. Extending development time beyond that elusive minimum allowsone to reduce the effort and realize improved quality, but the amount of reduction orimprovement is based on the knowledge provided by the basic metrics. A reasonableamount is somewhere in the middle.

There is no maximum development time,but Putnams data shows that beyond 130% ofminimum development time the trade-off between time and effort lessens sharply,therefore, there is little economic point in extending development beyond 130% range.

Productivity seldom improves in the short term; data shows that it typically stays thesame within the time range of a project. Productivity gains come slowly; for businessprojects, the best has been 8% per year, while for engineering and real-time software ithas seldom exceeded 5% per year, with many organizations reporting no gains at all.

Productivity relates to the project/process as a whole, never to the efforts of individualswithin the project. Individual productivity is not one of the core metrics.

23


24/53



As new development practices emerge, new measures of functionality may be needed.

We should choose a size (functionality) metric for which we have adequate data frompast projects.

We calibrate the estimate of a product by using the size metrics from past projects toestimate the size of the new project.

Some (but not all) software activity is of a research nature (requires creativity); in thosecases the project should not be constrained by a firm schedule (and fixed price bid).

Standardize the five core metrics within the scope of a single project and preferably,within the organization. Solid definitions make cross comparisons possible and facilitateeducation, training and the mobility of people.

Integrate the metrics within the software development process

There are a multitude of articles and resources that relate to the core metrics and theirrelationships. Table 9 identifies a few resources to get started with. Others are listed in theresources section of this document.

Table 13: Some Resources for Learning about the Software Equation and core MetricsResource Description and Location

DACS TopicArea onSoftware CostEstimation

This webpage, continuously updated by the DACS, identifies a multitude ofresources relating to cost estimating.


DACS TopicArea onMeasurement

This webpage addresses measurement in general but has subtopics focused onspecific items such as functional sizing and various measurement methodologiesthat may be utilized to collect the right data for estimating.


Crosstalk

Article, Feb.2006

Software Estimating Models: Three Viewpoints

This article compares the approaches taken by three widely used models forsoftware cost and schedule estimation. The comparisons illustrate significantdifferences between the models, and show significant differences in theapproaches used by each of the model classes.http://www.stsc.hill.af.mil/crosstalk/2006/02/0602JensenPutnamRoetzheim.html

CrosstalkArticle ,

Aug. 2002

Control the Software Beast With Metrics-Based Management

The five core metrics enable managers to estimate and bid software projects,and control progress during the project. Higher executives and clients need tounderstand the relationships among them because, otherwise, they are theprime source of unrealistic expectations and the resulting failures.

http://www.stsc.hill.af.mil/Crosstalk/2002/08/putnam.pdf

MEASUREMENT PROGRAM

Throughout this document, we have focused on the need to collect and use the core metrics. Inorder to do this it is necessary to establish a measurement program. The program is necessaryto ensure consistency and accuracy of the data. Gopal [Gopal, 2002] notes that many companiesfind software measurement to be a complex and difficult undertaking, citing studies in the 1990sshowing that fewer than 10 percent of the industry classified metrics programs as positive and

24
http://www.thedacs.com/databases/url/key.php?keycode=4http://www.thedacs.com/databases/url/key.php?keycode=3http://www.stsc.hill.af.mil/crosstalk/2006/02/0602JensenPutnamRoetzheim.htmlhttp://www.stsc.hill.af.mil/Crosstalk/2002/08/putnam.pdfhttp://www.stsc.hill.af.mil/Crosstalk/2002/08/putnam.pdfhttp://www.stsc.hill.af.mil/crosstalk/2006/02/0602JensenPutnamRoetzheim.htmlhttp://www.thedacs.com/databases/url/key.php?keycode=3http://www.thedacs.com/databases/url/key.php?keycode=4


25/53



that two out of three metrics initiatives did not last beyond the second year. Metrics programimplementation is a long and complex journey confronted with dilemmas stemming fromcontradictory demands.

Metrics can help guide the organization toward informed decisions by providing indicatorsregarding the quality, adequacy and progress of projects, processes and products and byenabling the organization to recognize the sum of its collective capabilities, which can lead to

consistently realistic and achievable plans for providing and delivering products and services.

Metrics also promote teamwork and improve team morale by linking efforts of individual teammembers to the overall success of the project and ultimately, the success of the organization.

They can also serve as a clear and objective basis for communication with stakeholders.

Metrics, in and of themselves do not impart any value. The ROI from metrics is the value of theaction a project professional takes, with the assistance of the metrics, to manage the issue athand. People make the decisions; metrics simply provide the foundation and rationale for suchdecisions.

McGarry [McGarry, 2002] asserts that fundamental software measurement analysis concepts canbe categorized as follows:

Estimation: Producing estimates of size, effort, schedule, and quality

Feasibility Analysis: Assessing the feasibility of project plans during initial planning and re-planning activities

Performance analysis: Assessing the projects actual performance against plansthroughout the project

Levin [Levin 2004] describes a measurement framework comprised of three sets of projectrelated issues: things, people, and the organization.

Things metrics describe the deliverable of the project and the efficiency with which it is beingproduced. This involves the WBS and metrics dealing with element cost, project cost andproject schedule. Delivery issues include risk management, product quality, and contracts.It includes metrics to assess and track scope and quality and contractor performance withrespect to cost, schedule and quality.

People metrics address the team members relationships with one another and attempt toquantify or characterize the behavioral attributes of people. They also measure thefriendliness of the organization toward the project team and the team toward itself.

Organization-Oriented Metrics address the environment in which the project team mustoperate. Metrics are tailored to the organizations important issues and strategic objectives.The organization provides the environment for the project and, at the same time, is the directbeneficiary of the success of the project.

It is important to strike a balance between the organizational culture and the best practices ofproject management and establish metrics that cover all three areas.

Gopal [Gopal 2002] conducted some research into factors that affect metrics program success.Success is measured with two variables ---- the use of metrics information in decision-making and

improved organizational performance.Gopal identifies technical factors influencing use of metrics in decision-making:

The frequency with which metrics are collected has a positive and significant influence ontheir use in decision-making.

The presence of a systematic process for identifying and collecting metrics across theorganization increases the use of the metrics in decision-making.

25


26/53



The benefits from collecting and using the more advanced metrics are greater than thebenefits from using the basic metrics; however, the basic metrics must be captured and usedfirst.

The rigor and usefulness of the analyses done on the raw metrics data increases the use ofmetrics in decision-making.

Better communication processes for dissemination of metrics analysis showed a positiveeffect on use of metrics in decision-making but was not statistically significant.

The other variables (technical factors) were not statistically significant, preventingstatements asserting that they, in fact, positively influenced use of metrics in decision-making.

Gopal also identifies organizational factors influencing organizational performance:

Involvement of upper management and project managers as well as metrics personnel insetting goals and identifying metrics increases organizational performance. The institutionalbelief factor is positively associated with organizational performance and is statisticallysignificant. The organization that believes that a metrics program is beneficial has higherorganizational performance than an organization without such a set of beliefs.

Companies with software development as their primary activity are more likely to reportimproved performance resulting from their metrics program than companies where softwaredevelopment is only a small percentage of their business.

Resource sufficiency and maturity level were not statistically significant.

In general, Gopals research results highlight the need to cultivate a culture within theorganization to promote metrics usage to ensure eventual success of the metrics program.Software managers first need to focus on the technical factors and provide incentives fordevelopers to use the metric information in daily operation. Once metrics data are used at theproject level, managers can then focus on the organizational performance improvement in thesecond stage.

There are several measurement methodologies (developed in the 1980s and1990s but still valid)that one can follow when implementing a measurement program. There is no need to start from

scratch. The two predominant methodologies are:

1. Goal Question Metric (GQM)2. Practical Software Measurement (PSM) recently renamed as Practical Software and

Systems Measurement

GQMis a methodology for identifying the right metrics by a process of specifying goals that arealigned with organizational goals, asking questions to refine and clarify the goals, and identifyingthe metrics that result from responding to the questions posed. The generic methodology can beapplied at various levels, but it is intended to help the organization establish a metrics programthat is closely aligned with the goals of the organization. In most cases, the core metrics wouldbe identified as a result of applying this process, but it might reveal other metrics as well that areunique to your project and/or organizational culture. GQM relies on participation of all

stakeholders for its success. This is consistent with Gopals findings. For further details, refer tothe DACS Gold practice on GQM. See http://www.goldpractices.com/practices/gqm/index.php

PSMis an information-driven, issue-based approach to software measurement for projectmanagement [McGarry 2002]. This measurement process initiative, established in 1994, isfunded by the DOD and is freely available athttp://www.psmsc.com. It has been endorsed bygovernment Acquisition Executives, and is a key element of the OUSD (A&T) software initiative.The methodology is also being adopted by government and industry organizations.

26
http://www.goldpractices.com/practices/gqm/index.phphttp://www.psmsc.com./http://www.psmsc.com./http://www.goldpractices.com/practices/gqm/index.php


27/53



Highlights of the PSM process are: Its focus is on cost, schedule, and technical objectives

It identifies project-specific issues

Issues are grouped into common software issue areas

Measurement categories correspond to issue areas

Each measurement category has a candidate set of proven measures Measures are selectedbased on availability, environment, and other factors.

The PSM process defines and uses seven common information categories to facilitate theidentification and prioritization of a projects specific information needs [McGarry 2002]. They are:

Schedule and Progress

Product Size and Stability

Product Quality

Process Performance

Resources and Cost

Technical Effectiveness

Customer Satisfaction

PSM differs from GQM in the following ways:

PSM is information-driven while GQM is goal-driven. PSM does not ignore goals; ratherit focuses on issues that result when striving to achieve a goal. Many of the broad goalsrelating to cost, schedule and quality are assumed.

PSM focuses on using existing measures or metrics rather than creating new ones. Itdoes this by providing a pick list of measures for each of the various informationcategories, which is based on best measurement practices of DoD, government andindustry programs. GQM perceives each project as unique and therefore asserts thatone should not arbitrarily adopt another projects plan or measures. PSM, however,advocates tailoring the templates and plan to the specific environment.

PSM is primarily a tool for the project managers who work with a measurement specialistto establish the measurement program that meets their needs. Other stakeholders, suchas the development team are not directly involved in the planning although they arecalled upon to provide data and implement data collection procedures. GQM seeks toinvolve the stakeholders in the entire process and focuses on the human and culturalissues involved in implementing a metrics program, operating on the principle thatinvolvement at all levels is critical to a successful measurement program.

PSM is designed to help put measurement into practice at the project level, therebyproviding the data for addressing enterprise level performance, process improvement,and business-related questions. GQM has a more generic application in that it can beused wherever there is a need to assess satisfaction of goals, irrespective of the subjectdomain or the level.

PSM was the basis for the development of ISO/IEC 15939 and has influenced several otherstandards as noted in Figure 4 (extracted from the PSM website in June 2006):

The purpose and outcomes of the measurement process from ISO/IEC 15939 have beenadded to the revision to ISO/IEC 12207, Software Life Cycle Processes, within a newsupporting process, entitled Measurement.

Measurement concepts have also been added to ISO/IEC 15288, System Life CycleProcesses.

27


28/53



The new measurement terminology has also been coordinated with the revisions toISO/IEC 9126, Software Product Quality, and IOS/IEC 14598, Evaluation of SoftwareProducts, so that all these standards will use a common set of measurement terminology,once the revisions are complete.

In addition, the purpose and outcomes of the measurement process have been added toISO 9000-3: Application of ISO 9001:2000 to Software.

Figure 4: The Influence of PSM on Various Measurement Related Standards

The draft international standard ISO/IEC 15393, in turn, was used as an input to theMeasurement and Analysis (MA) process area of the CMMIsm. The MA process area provides amethodology for assessing whether a project's measurement program is compliant with theinternational standard, in addition to providing relevant information on CMMI-based process

improvement activities. Overall, the CMMI helps organizations to institutionalize theirmeasurement and analysis activities, rather than addressing measurement as a secondaryfunction. In the MA process area, the activities of "Plan Measurement" and "PerformMeasurement" are detailed in two specific goals that must be implemented and eight specificpractices that are considered important in achieving the associated specific goals. The activitiesof "Evaluate Measurement" and "Establish and Sustain Commitment" are considered through thegeneric goals, with elaborations specific to the MA process area.

The coordination of these documents means that the software and systems engineeringcommunities have a consistent set of information-driven standards and guidance forimplementing project and process measurement.

Some common themes or lessons learned that have emerged from the literature aboutmeasurement programs include:

Establish a metrics mindset

Create and foster an atmosphere conducive to the acceptance of metrics technology byproviding ways to monitor cost effectively and efficiently

Find meaningful incentives (with short-term payoffs) to motivate managers to establish ametrics program

Create awareness: build confidence through success stories

28


29/53



Emphasize continuous data collection and awareness

Address the question of organizational culture. Can a metrics program be effective, orwill change management be needed?

In initial planning for the metrics program, assess whether the skills exist internally andwhether there is time to initiate the program. Alternatively, are outside consultantsneeded?

Involve practitioners through feedback

Key measures need to be clearly defined

Determine up front how to evaluate the success of the metrics program. According t

metrics based scheduling

Documents