Metrics BasedMetrics BasedProjectProject

GovernanceGovernance

“Without objective data you are justanother person with an opinion”

Presented by : Pam MorrisTOTAL METRICS

ACOSM 2003

Department of Innovation, Industry and Regional Development Victoria Online

“harnessing the“harnessing thecapabilities of yourcapabilities of your

metrics team tometrics team tominimise project risk”minimise project risk”

3

Victoria Online

Summary SlideStandish Group Chaos Report : 1994 -2002

4

Victoria Online

Standish Group Chaos Report : 1994 - 2002

0%10%20%30%40%50%60%70%80%90%

100%

1994 1996 1998 2002

ChallengedFailedSucceeded

•Successful - on timeand on budget with allfeatures and functions•Challenged - overbudget, over time, not allfunctions and features•Failed - cancelled

Only 34 % achieved Business Intent

Governance – “balancing risk against opportunity”

5

Victoria Online

Consequences of Failure� Private / public embarrassment

� Missed business opportunities

� Disenchanted clients

� Legal repercussions

� De-moralised staff

6

Victoria Online

August 1999 - INCIS - New ZealandPolice System “NZ Finance Minister confirmed that INCIS' cost to thetaxpayer so far was $106 million. Others have estimated thecost at up to $130 million, but in any case, costs have run wellbeyond the $98 million police budgeted for the entire project in1995”.Computerworld - 10th August 1999

“the New Zealand police said to have made more than 900variations to the original 10,000-page contract and IBM saying ithad already lost $40 million on the ill-fated application”The Age 24 June 2003

7

Victoria Online

Feb 2001 - Citylink – Melbourne’s Tolling andTunnel System- “Trouble is, it's been plagued with problems from

the first sod - delays technical problems withcomputer tolling systems” 7:30 Report 22nd Feb 2001

- “Transurban claimed liquidated damages of$251.5 million from TOJV for the late completionof the Melbourne CityLink”. Citylink Media Release - 27th November 2001

8

Victoria Online

Feb 2003 - Victorian Public TransportTicketing System

- “In February, the Victorian Governmentcompleted payment of a $65 million settlementto ERG subsidiary OneLink after changes madeto the project design at a late stage blew out thebudget of OneLink's Melbourne public transportticketing system rollout. ” The Age June 24th 2003

-

9

Victoria Online

August 2003 - RMIT - Academic ManagementSystem

- “The AMS project displayed a lack of essentialrigour and a lack of effective risk management ”Victorian Auditor Generals Report 2003

- “Original estimate was $12.6 million it is nowanticipated to cost $47.3 million by the end of thisYear” The Australian – August 5th 2003

Department of Innovation, Industry and Regional Development Victoria Online

Attributes ofAttributes of“Scope Manager”“Scope Manager”

Excellent skills in:Business analysisMetrics and Function pointsCommunicationManagement Reporting

Report to:Project Board / Project Sponsor

Paid by:Project Budget/client/supplier

Independence

11

Victoria Online

Governance

Board of Directors

CFO CEO CIO

Project Board

IT Project Director

Business ProjectManager

DevelopmentProject Manager

Project Team

“ScopeManager”

Project Governance

IT Governance

Project Sponsor

Corporate Governance

Project Activities – Metrics Based Governance

Business CaseFPA Size “Estimate”

Effort, Costand DurationEstimate

ISBSGIndustryProductivityData

Approval

©Total Metrics

BENEFIT - Independent project estimatesbased Industry data from ‘best practice’

experience or reality check on clientbudget.

Feasibility Study

Budget allocation

13

Victoria Online

Will the software be delivered in time and beworth the Business Value?

300 ~ 500 function points

Delivery ~ 7 to 9 months

Cost ~ $390k to $580k

Develop Requirements / RFT

Budget Approval

Project Activities – Metrics Based GovernanceRequirementsSpecification

Functional Analysis

RequirementsFunctionalSize

RequirementsAssessment &Refinement

BENEFIT – Independently reviewed,auditable, quantified, traceable list of

requirementsRFT

©Total Metrics

RequirementsFunction &DataCataloguedList + Profiles

15

Victoria Online

Baseline List of Individual Requirements

16

Victoria Online

Profile and Quantify Requirements

Core Functions selectivelyidentified and selectively sized= 300fps of total of 450 fps

17

Victoria Online

Re-Estimate – Java , J2EE

Minimum MaximumFunction Points 400 550PDR hrs/fp 17 21Effort hours 5,442 7,197Duration months 9.0 11.4Cost $642,108 $849,255Cost $/fp $1,167 $2,123

Project Activities– Metrics Based GovernanceTenderResponseEvaluation

SuppliersFunctionalSolution

BENEFIT – Independent evaluationsuppliers price and schedule

estimates. Quantitative match ofsupplier functional proposal

Select Supplier

©Total Metrics

Compare againstInitial Effort, Cost andDuration Estimate

Suppliers totalcost andschedule

SuppliersPrice Dollarsper FP

Estimates Reality Check

%Match against RFTFunction & DataCatalogue Lists

19

Victoria Online

Suppliers Solution Mapped to Requirements

Quantify % Match = 205out of core 300 functionpoints can be deliveredwithout change by theSuppliers PackageSolution

20

Victoria Online

Evaluate Suppliers Responses forReasonableness

Estimates Minimum MaximumDuration months 9.0 11.4Cost $/fp $1,167 $2,123Supplies Bids Cost $/fp DurationSupplier A $1,892 12Supplier B $1,563 10Supplier C $987 6⌧

Project Activities– Metrics Based Governance

FunctionalSpecification

BENEFIT – Independent assessment ofcompleteness of Functional Requirements.Quantify different functional solutions to fitbudget / schedule. Expert advice on Pricing &Change Control Model and Project Metrics.

Build

©Total Metrics

SpecificationAssessment &Refinement

FunctionalAnalysis

Refine BaselineFunctional Sizeand Catalogue

Quantify FRS matchto RequirementsRFT

EstimatesReality Check

Budget andscheduleconstraints

Advice on Cost & Change Control Model and Metrics

Project Activities– Metrics Based GovernanceFunctionalChanges

Implement Change

©Total Metrics

SpecificationAssessment &Refinement

FunctionalAnalysis

Update BaselineFunctional Size

Quantify sizeof Change forPricing

EstimatesReality Check

Budget andscheduleconstraints

BENEFIT – Independent objective assessmentof the extent of change and assistance with

pricing. Quantify impact on plannedschedules and budgets.

23

Victoria Online

Quantified Traceable Changes AgainstRequirements

Project Activities– Metrics Based GovernanceProjectProgressReporting

BENEFIT – Independent assessment ofproject status by quantifying percent

delivered.

ReviseStrategy

©Total Metrics

Earned Value -FPs deliveredversus plannedschedule.

Map User AcceptanceTesting Test Cases againstFunctional Catalogue

ISBSGIndustryProjectActivityProfile

Identify and quantify Test Casecoverage, hot spots for defects

25

Victoria Online

Independent ProgressRecording Project Team

record Actualtime againstfunction

Completed %Status ofFunctionpredicts EffortConsumed

TodaysD 25-Apr-02Week # 8

BFCs

1.1Create Assignment 2.4 Specified 26% 6 14.4 3.0 3.7 11.4

1.2 Modify Assignment 2.4 Specified 26% 6 14.4 5.0 3.7 9.4

1.3View / Print Assignment Detail 2.4 Specified 26% 3 7.2 5.0 1.9 2.2

1.4.1Assign Contractor to Assignmnt 2.4 Specified 26% 4 9.6 4.0 2.5 5.6

1.4.2Remove Contractor Assignmnt 2.4 Specified 26% 3 7.2 7.0 1.9 0.2

1.4.3List Assignment Contractors 2.4 Specified 26% 4 9.6 4.0 2.5 5.6

1.5 Quotation Success 2.4 Built 74% 4 9.6 6.0 7.1 3.6

1.6List Assignments Date Range 2.4 Built 74% 3 7.2 1.5 5.3 5.7

2.1Create Assignment Type 2.4 Built 74% 4 9.6 6.5 7.1 3.1

2.2Modify Assignment Type 2.4 Built 74% 4 9.6 5.5 7.1 4.1

2.3Delete Assignment Type 2.4 Specified 26% 3 7.2 1.5 1.9 5.7

2.4View Assignment Type 2.4 Specified 26% 3 7.2 1.5 1.9 5.7

2.5List /Print Assignment Type 2.4 Tested 91% 3 7.2 4.7 6.6 2.5etc…………… etc……

TOTAL 119 285.6 105.9 139.5 178.2

Completed Stage

Current Completion %

Function Points

Predicted PDR

(hrs/fp)Actual Hours Remaining

Predicted Total Hours

FP Predicted

Hours Consumed

Actual Hours Consumed

26

Victoria Online

Progress Reporting

start 01-Mar-02 Week Number 8today 25-Apr-02 Actual Original Plan

1.8 2.4

58.1 46.5

105.9 132.8

178.2 161.8

10.5 9.607-Jul-02 01-Jul-02

PDR

Actual Value Calculation based on:

Hours consumed for FPs delivered

Function Points Delivered

Effort Hours ConsumedRecorded by team for work against a function

FPs by Percentage completion

Effort Hours Remaining

Predicted by Remaining hours

Relationship between Effort and Duration

Total hours predicted minus hours consumed

Weeks RemainingDue Completion Date

27

Victoria Online

Progress Reportingstart 01-Mar-02 Week Number 8today 25-Apr-02 Actual Original Plan

48.9% 39.1%

40.2% 48.9%

45.1% 36.0%

% Effort Consumed

%Effort Consumed of total compared to Effort predicted to be consumed for FPs delivered

%Schedule Consumed

%Schedule Consumed of total compared to predicted to be consumed for the effort expended

% Product Delivered

%FPs Delivered of total compared to that predicted to be delivered for effort consumed

Calculation based on:

Project Activities– Metrics Based GovernanceProjectImplementation

BENEFIT – Independent assessmentproject productivity and process. Expert to

assist in the collation of data forsubmission to the ISBSG Repository and

interpretation of results.

Contract Signoff

©Total Metrics

Validation,collation,analysis andsubmission ofProject Metrics.

Project Productivity andQuality AssessmentReport

ISBSGIndustryProjectDataRepository

Department of Innovation, Industry and Regional Development Victoria Online

Case StudiesCase Studies

Case 1- Win / Win

Complete functional requirementsSuccessful implementation, on

time, on budget and all requirementsGood Productivity ratesOngoing good relationship

Department of Innovation, Industry and Regional Development Victoria Online

Case StudiesCase StudiesCase 2 – Conflictive

Supplier estimates unrealisticEarly warning of potential over

runs

Supplier status reporting notreflective true situation

Quantitative assessment offunctionality delivered to UAT

Requirements not all implemented Quantitative assessment of

functionality omitted for pricenegotiations

Department of Innovation, Industry and Regional Development Victoria Online

Which ProjectsWhich ProjectsNeedNeed

Governance?Governance?High risk

New / inexperienced projectmanagement

High profile

Cost / BenefitCost / BenefitAnalysisAnalysis

Cost ? = 1 – 3 % of software cost•depending on :

•level of activities•Size of project•Internal or external consultant

Cost of not having it ?•0 – >100% of project cost

Key SuccessKey SuccessFactors?Factors?

FPA is a standardised, structuredmethod to identify, classify andquantify functionality to be deliveredIndependence of Scope ManagerCertification of Scope ManagerReporting at Project Board LevelCommitment both Client and SupplierCustomisation of method to suitProjectTransparency of the processAvailability of relevant ISBSG data

Thank YouThank You

Total MetricsTotal Metricswww.totalmetrics.comwww.totalmetrics.com

03 9882 761103 9882 7611admin@totalmetrics.comadmin@totalmetrics.com