defending australia and its national interests session 16 – cost methodology framework presenter...
TRANSCRIPT
Defending Australia and its National Interests
Session 16 – Cost Methodology Framework
Presenter – Mick Doherty
COST ESTIMATION CONFERENCE
Outline
– Cost Methodology Framework• Generic Cost Estimating Process;• Cost Estimating Techniques;• Which Technique to Use & When• Costing Exemplars ; and• Measures of Cost Estimate Quality.
Generic Cost Estimation Process
Define Project Scope
Define key Assumptions
Analyse, Adjust & Maintain the
Estimate
Develop WBS & Cost Elements
Investigate & Obtain Cost Data
Develop Workforce Plan
Prepare Cost Estimate for each
Cost Element
Techniques* Analogy* Parametric* Engineering* Actual Costs* Price Data* Expert Opinion
Review Cost Model & Compare
Against DCP Provision
Measures of Quality* Accuracy* Repeatability* Traceability* Comprehensiveness
Publish, Present & Use the Estimate
* Reasonableness* Sensitivity Analysis* Cost Benefit Analysis* Cost Risk Analysis
No Yes
Generic Cost Estimation Process
Step 1 – Define the Project Scope
Step 2 – Define Key Assumptions• Document key assumptions in the scoping
directive & Basis of Estimate
The quality of an estimate is only as a good as the maturity of the project definition and the
appropriateness of the assumptions made
Generic Cost Estimation Process
Step 3 – Develop WBS & Cost Elements • Developed off a skeleton WBS• A dynamic process• WBS & Cost Elements will develop and expand over time.
Step 4 – Investigate and Obtain Cost Data • Where to find cost data?
– From Contractors– From other projects– Defence Library, Janes, Forecast International etc– US President’s Budgets
Step 5 – Develop the Workforce Plan• Understand the Govt workforce to:
– Undertake Capability Development– Acquire the Capability & setup the Support Systems– Crew and Maintain the Capability
• Needs to be balanced against the White Paper Provisions
Estimating Principles - cont
Step 6 - For each cost element:– Use best available cost data. – Seek input from Subject Matter Experts, to assist you in
obtaining the cost data. – Use as many costing sources and techniques as possible to
validate and test the range of estimates.– Record your assumptions and document the basis of the
estimate.– Regularly review the estimate
Step 7– Review Cost Model– Compare against the DCP Provision
Hint: If possible, obtain an Independent Cost Estimate (ICE) and compare the ICE against the cost model for completeness and accuracy.
Estimating Principles - cont
Step 8 - Test the Model– Reasonableness– Sensitivity Analysis– Cost Benefit Analysis– Cost Risk Analysis
Step 9 - Analyse, Adjust and Maintain the Estimate
Step 10 - Publish, Present and Use the Estimate
This is an Iterative Process and these steps will be repeated on a regular and frequent basis as the project is further defined and the cost model develops.
Generic Cost Estimation Process
Define Project Scope
Define key Assumptions
Analyse, Adjust & Maintain the
Estimate
Develop WBS & Cost Elements
Investigate & Obtain Cost Data
Develop Workforce Plan
Prepare Cost Estimate for each
Cost Element
Techniques* Analogy* Parametric* Engineering* Actual Costs* Price Data* Expert Opinion
Review Cost Model & Compare
Against DCP Provision
Measures of Quality* Accuracy* Repeatability* Traceability* Comprehensiveness
Publish, Present & Use the Estimate
* Reasonableness* Sensitivity Analysis* Cost Benefit Analysis* Cost Risk Analysis
No Yes
Estimating Techniques
AnalogyParametric Engineering
Bottom up
Price Data
ExpertOption
Extrapolationfrom Actuals
Cost Estimation Methods
Why include ‘Price Data’?
• In the Australian context, we place a high reliance on ‘tender quality’ costs for projects seeking 2nd Pass Approval.
• Rather than obtaining ‘costs’ from tenderers, we are in fact being quoted the ‘price’ we are expected to pay for the goods or service.
• Price Data varies in quality, from ROM quotes to tender quality.
Cost Estimating Techniques
Analogy• direct comparison with historical data for similar item,
eg Weight Based Analogy
Parametric• Relates cost to some physical attributes or performance
characteristics in a cost-estimating relationship (CER)
Engineering (Bottom up)• Examination of separate work segments in detail and a
synthesis of many detailed estimates into a total.
Which Technique to use, when
Parametric Price Data Engineering
Actuals
Analogy
DC
P E
ntr
y
1st P
ass
App
rova
l
2nd P
ass
App
rova
l
In-ServiceDetermineDefence Needs
Capability Requirements
DefinitionAcquisition
Cost Estimation Technique and Project Maturity
Which Technique to use, when
Costing Methodology
DCP Entry DCP Update 1st Pass Approval
2nd Pass Approval
Real Cost Increase (RCI)
Analogy
Parametric
Engineering
Actual Costs
Price Data
Expert Opinion
Legend - Primary - Secondary - Not recommended
Applicability of Cost Estimating Technique
Which Technique to use, whenPrinciples
• Each cost element should be costed using the most appropriate method
• Use as many techniques as possible to validate the estimate before settling on the most appropriate method and the level of risk
• Document results from other methods in the basis of estimate
• Early on, when information is scarce, the primary techniques are:– Analogy– Parametric Estimation
Estimating at DCP EntryCosting Exemplars
• Make assumptions about systems that might fill the capability gap and choose an “exemplar” system to cost– Something, that if brought today, would meet the
capability– Must be technically feasible– Doesn’t mean this is what will be delivered
• Develop cost estimates based on the Exemplar
Costing ExemplarsCost to Circumnavigate the Moon
• In his 1865 novel "From the Earth to the Moon" Jules Verne described as his exemplar a "moon gun" located in Florida.
• 2.74m bore Gun poured into a 274 m deep well.
• First 61m of barrel filled with 122 tons of guncotton.
• When fired, a 2.74 m diameter shell was propelled to a velocity of 16.5 km/second.
• After deceleration through the earth's atmosphere, the shell would have a residual velocity of 11 km/second, sufficient to reach the moon.
• Estimated the cost to be US$5,446,675 in 1865 prices.
Cost to Circumnavigate the Moon
• In 1969, Apollo 8 circumnavigated the moon.
• Up to the Apollo 8 mission, the cost of the Apollo program was US$14.4 billion in then year dollars.
• Recall Jules Verne’s estimate of US $5,446,675 in 1865 prices which equates to US$12.1 billion in 1969 prices.
• Within 15%
Not bad for an estimate made 104 years earlier!!!
Estimation at DCP EntryOutline Approach
• Define mission system & obtain indicative costs
• Make adjustments to mission system for:– Cost growth & cost escalation– Procurement & integration of additional equipment
• Communications equipment• Electronic Warfare & mission support systems• Carried Equipment, (helos, weapons, UAVs etc)
– Adjust for Quantity (learning curve, LRIP v Full Rate)
• Apply metrics to– Capability Development– Mission System Development– Support System– Contractor Support System– Productions Support– DMO Project Support
Estimation at DCP EntryMetrics - Rules of Thumb
Cost Element Metric
1st to 2nd Pass Activities 0.5% to 1.5%
Detailed Design Activities 3.0% to 5.0%
Mission System
ILS & Support 20.0% to 50.0%
Project Management 2.0% to 5.0%
Contingency 20.0% to 50.0%
As a percentage of Mission System Cost
Estimation at DCP EntryMetrics - Rules of Thumb
Cost Element Metric Comments
PM, SE& ILS Management 10% to 15%
Test & Evaluation 2% to 5% Depends upon many factors
Initial Spares 7.5% to 15% Depends on ROE, BOP, system
Data & Documentation 2% to 5% IP, language, MIL-SPEC vs COTs.
Facilities 2% to 5%
Training 2% to 5%
Support Equipment 1% to 3%
PHS&T 1% to 3% Depends on delivery point
DMO PO 5% to 15%
FMS 2.5% to 5%
Australian Prime 10% to 15%
NPOC
Hardware Maintenance 5% of acquisition cost per annum
Software Maintenance 15% of acquisition cost per annum
Estimation at DCP EntryPrice Data Example
• Replenishment Ship based on Berlin Class– Cost US$141.3m 1998 (0.74 exchange rate)– 1.42 Inflator to 2010/11 prices– Current Price = A$271m 2011 prices
Metric Total Cont Total
1st to 2nd Pass Activities 0.50% $1.89 $0.50 $2.39
Acquisition
Mission System $271.10 $81.33 $352.43
Premium to build in Australia 30.00% $81.33 $24.40 $105.73
Weapons (Self Protection) $15.00 $4.50 $19.50
Communication Equip $10.00 $3.00 $13.00
Sub Total $377.43 $113.23 $490.66
ILS & Support 30.00% $113.23 $33.97 $118.10
Project Management 5.00% $18.87 $5.66 $19.70
Total Acquisition Cost $511.42 $153.36 $630.85
Estimation at DCP EntryOutline Approach
• Progressively replace “metric estimate” with detailed estimate that takes account of:– Engineering Development & Non Recurring Engineering– Support System
• incl training and simulation• Global Supply Chain?
– Facilities & Infrastructure– Ordnance – initial load out & warstock– Production Systems if equipment to be manufactured or
assembled in Australia
Estimation at DCP EntryWeight Based Analogy
• Process– Obtain cost & technical data for reference system
• Eg: Displacement, Gross Weight– Estimate scaling factor for new system, Displacement– Prorata
• CostNew = CostOld * DispNew / DispOld
• Advantages– Easy to use– Scaleable
• Disadvantages– First order approximation only– Scaling factor may not be constant– Error Band unknown– Needs to be confirmed by other costing methods
Weight Based Analogy Joint Light Tactical Vehicle
M-ATV Joint Light Tactical Vehicle
HMMWV
How to Estimate the Cost of the New Vehicle?
Estimate off HMMWV & M-ATV using a Weight Based Analogy
Weight Based Analogy Joint Light Tactical Vehicle
Capability M-ATV HMMWV JLTV
Gross vehicle weight (GVW) (pounds)
32,500 12,100 19,950
23,950
Estimated base price per system (without GFE and other costs)
$445,000 $186,000 ????
????
Calculation $445/32.5 $186/12.1
Cost per GVW ($/pound) $13.69 $15.37
Estimated Cost Range $13.69*19.95 $273,000
Based on M-ATV $13.69*23.95 $328,000
Estimated Cost Range $15.37*19.95 $307,000
based on HMMWV $15.37*23.95 $368,000
Note: US Army Estimate $306,000 to $332,000
Weight Based Analogy Joint Light Tactical Vehicle
• Do you think the estimate is reasonable?
Capability M-ATV HMMWV JLTV
Turning radius (feet) 54 25 25
Payload (pounds) 4,000 1,800 - 3,500 -
3,950 5,100
Gross vehicle weight (GVW) (pounds)
32,500 12,100 - 19,95 -
14,800 23,950
Maximum speed at GVW (miles per hour)
65 63.8 70
Minimum range (miles) 300 250 400
Reliability (mean miles between operational mission failure)
600 800 - 4,500 -
2,250 5,170
Operational availability 80% 90% 95%
Power generation-on-board/exportable (kilowatt)
12.7/0 1.2/0 20/10
Estimated base price per system (without GFE and other costs)
$445,000 $186,000 $306,000 -
$332,000
Weight Based Analogy Joint Light Tactical Vehicle
Weight Based Analogy Joint Light Tactical Vehicle
• Now, do you think the estimate is reasonable?
• Potential problems with scaling include:– Increased reliability (doubling of mean miles between
failure)– Increased Operational Availability (from 90% to 95%)– Increased power generation– Faster 65mph to 70mph– No allowance for cost growth
• Where might cost growth occur?
Estimation at DCP EntryOutline Approach - NPOC
• Understand the operating profile:– Level of preparedness– Rate of Effort– Basis of Provisioning– Ratio of crew to equipment
• Scale off existing equipment using a “weight based analogy”
Weight Based Analogy SEA 1180 – Multirole Vessel
Operating CostsExemplar HMNZS Otago• Length overall – 85m• Beam – 14.0m• Draught – 3.6m• Disp – 1626 tonnes
• How can we determine operating for the new ship?
What about extrapolating using a weight based analogy from RAN Streaming Day Rates for similar ships?
Weight Based Analogy SEA 1180 – Multirole Vessel
Operating Costs
STEAMING DAY RATES $
Class Sea
days
Rations Fuel Weapons DMO Stores
DMO Refit Suppliers
FFG 93 $3,347 $54,516 $12,688 $59,596 $245,637 $4,506
FFH 122 $2,591 $31,904 $6,916 $86,382 $150,452 $5,285
ACPB 215 $241 $5,492 $210 $461 $8,878 $148
SML 122 $194 $2,505 $19 $6,315 $9,727 $136
AGS 173 $718 $7,374 $169 $16,242 $42,502 $577
MHC 118 $630 $4,372 $833 $17,678 $61,044 $732
• Step 1 – Obtain daily steaming day rates for exemplar vessels
What next?
Weight Based Analogy SEA 1180 – Multirole Vessel
Operating Costs
• Step 2 – Annualise steaming day rates• Annual Cost = Seadays x $daily_rate
• Step 3 – Obtain full load displacement for each ship
Annual Cost $m
Class Disp(tonnes)
Rations Fuel Weapons DMO Stores
DMO Refit Suppliers Total
FFG 4,267 $0.31 $5.06 $1.18 $5.54 $22.82 $0.42 $35.33
FFH 3,759 $0.32 $3.91 $0.85 $10.58 $18.43 $0.65 $34.73
ACPB 300 $0.05 $1.18 $0.05 $0.10 $1.91 $0.03 $3.32
SML 325 $0.02 $0.31 $0.00 $0.77 $1.19 $0.02 $2.30
AGS 2,205 $0.12 $1.28 $0.03 $2.81 $7.36 $0.10 $11.70
MHC 732 $0.07 $0.51 $0.10 $2.08 $7.19 $0.09 $10.05
Sea 1180 1,626 ?? ?? ?? ?? ?? ?? ??
Weight Based Analogy SEA 1180 – Multirole Vessel
Operating Costs
• Step 4 – Determine scaling factor
• Disp_Ratio = Dispnew / Dispold
• Step 5 – Multiply annual cost by Displacement Ratio Annual Cost $m
Class Disp(tonnes)
Rations Fuel Weapons DMO Stores
DMO Refit Suppliers Total
FFG 0.38 $0.12 $1.93 $0.45 $2.11 $8.70 $0.16 $13.46
FFH 0.43 $0.14 $1.69 $0.37 $4.58 $7.97 $0.28 $15.02
ACPB 5.42 $0.28 $6.40 $0.24 $0.54 $10.35 $0.17 $17.99
SML 5.00 $0.12 $1.53 $0.01 $3.85 $5.93 $0.08 $11.52
AGS 0.74 $0.09 $0.94 $0.02 $2.07 $5.43 $0.07 $8.63
MHC 2.22 $0.16 $1.14 $0.22 $4.63 $15.97 $0.19 $22.31
Sea 1180 1.00 $0.15 $2.27 $0.22 $2.96 $9.06 $0.16 $14.82
Scaling up or down on the basis of the Displacement Ratio suggests an annual cost of around $15m pa
What are the Expected Cost Quality Standards?
• Cost estimates are expected at each and every stage of the process and for key approvals from DCP Entry to 2nd Pass Approval.
• 1st Pass Cost Estimates– Can be budgetary quotes, market surveys, info from
Allies etc for implementation costs (for FMS cases P&A data)
– However, funds to be expended during CapDev (or 1st to 2nd Pass activities) are to be of tender quality
• 2nd Pass Cost Estimates– Tender Quality Costs is the aim (for FMS cases draft
LOA)
• DCP Entry – more then one data point for big ticket items.
• First Pass – Survey the market and ask the right questions (non competitive).
• Second Pass – Tender data (lowest possible price).
• Vigilantly document the assumptions and calibrations at every level.
Cost Quality Standards
Questions
?