Download - Acosm 2010 Harold Van Heeringen V3
1
Challenges from a supplier point of view
Metrics questions in Request for Proposals
Harold van Heeringen
Sogeti Nederland
Senior metrics consultant / Cost engineer
Sizing, Estimating & Control
metrieken.sogeti.nl
@haroldveendam
ACOSM 2010ACOSM 201022
Company name and Presentation Title hereCompany name and Presentation Title here
Agenda
Request for proposals
Software estimation using function points
Typical questions in request for proposals (RFP’s)
Supplier challenges
Recommendations for the client organization.
Harold van Heeringen - Metrics Questions in RFPs
ACOSM 2010ACOSM 201033
Company name and Presentation Title hereCompany name and Presentation Title hereHarold van Heeringen - Metrics Questions in RFPs
Trends in outsourcing?
Dutch government wastes billions of euros every year
on failed ICT projects
ACOSM 2010ACOSM 201044
Company name and Presentation Title hereCompany name and Presentation Title here
Request for Proposal (RFP) An invitation for suppliers, through a bidding process, to
submit a proposal on a specific product or service (source: wikipedia)
Information provided: Corporate information
Schedule of bidding process
Project Summary
Detailed overview of the project
Decision criteria
Harold van Heeringen - Metrics Questions in RFPs
ACOSM 2010ACOSM 201055
Company name and Presentation Title hereCompany name and Presentation Title here
Demand issues (RFP sender) Provide the right information
Detailed
Up-to-date
Ask the right questions
Build a good decision model
Evaluate the proposals
Choose wisely
Harold van Heeringen - Metrics Questions in RFPs
ACOSM 2010ACOSM 201066
Company name and Presentation Title hereCompany name and Presentation Title here
Supplier issues (RFP responder)
Can we:
Deliver the required functionality ?
Meet the technical and quality requirements ?
Within the time limits required ?
Answer all RFP questions ?
Estimate the project costs accurately ?
Score the best on the clients decision model ?
Support our claims with proof ?
Harold van Heeringen - Metrics Questions in RFPs
ACOSM 2010ACOSM 201077
Company name and Presentation Title hereCompany name and Presentation Title here
Sogeti Sizing, Estimating & Control Division MD – fixed price/fixed date projects
Sizing, Estimating & Control 11 (COSMIC) Function Point Analysts
2 metrics consultants
Responsible for metrics part of a quotation. Size: FPA/COSMIC
Estimation: QSM / Sogeti tool / ISBSG
Scenarios !
Product: Methodical Estimation Report
Harold van Heeringen - Metrics Questions in RFPs
ACOSM 2010ACOSM 201088
Company name and Presentation Title hereCompany name and Presentation Title here
Agenda
Request for proposals
Software estimation using function points
Typical questions in request for proposals (RFP’s)
Supplier challenges
Recommendations for the client organization.
Harold van Heeringen - Metrics Questions in RFPs
ACOSM 2010ACOSM 201099
Company name and Presentation Title hereCompany name and Presentation Title here
Software estimation using function points Function point analysis (NESMA / IFPUG / COSMIC)
Objective (ISO/IEC)
Repeatable
Verifiable
Quantifying the size om the functional user requirements Technology independent
Independent of implementation
Is a product measure, not a project measure
Doesn’t take into account any ‘non-functionals’
Harold van Heeringen - Metrics Questions in RFPs
ACOSM 2010ACOSM 20101010
Company name and Presentation Title hereCompany name and Presentation Title here
Project estimation
Size objectively measured Size = xxx function points
Estimation: Effort (hours) per activity
Duration (months) and milestones
Team size (average staff, peak staff)
Tools QSM SLIM suite
ISBSG repository release 11
Sogeti Estimating wizard
Harold van Heeringen - Metrics Questions in RFPs
ACOSM 2010ACOSM 20101111
Company name and Presentation Title hereCompany name and Presentation Title here
QSM SLIM Estimate
ACOSM 2010ACOSM 20101212
Company name and Presentation Title hereCompany name and Presentation Title hereHarold van Heeringen - Metrics Questions in RFPs
Generic Estimation Model
Need Software
Energy
Software
development
process
Waste
Time
Size Size
Defects
Effort
Duration
Defects
Productivity
Metric: Size
Function points
Metric: Size
Function Points
Metric: Effort
Number of hours
Manpower buildup
Peak staff
Metric: Duration
Number of weeks
Metric: Quality
Number of defects
Metric: Process productivity
Skills and experience team
Development environment
Complexity
Quality Management System
External influences
ACOSM 2010ACOSM 20101313
Company name and Presentation Title hereCompany name and Presentation Title here
Agenda
Request for proposals
Software estimation using function points
Typical questions in request for proposals (RFP’s)
Supplier challenges
Recommendations for the client organization.
Harold van Heeringen - Metrics Questions in RFPs
ACOSM 2010ACOSM 20101414
Company name and Presentation Title hereCompany name and Presentation Title here
RFP – Typical Metric Questions Q1 - What is your productivity for Java projects?
Q2 - How long do you need to build a .Net application of 500 FP?
Q3 - What is your price per function point for a 500 FP Oracle system?
Are these the right questions ??
Is the client organization capable of making the right decision based on the answers to these questions?
Harold van Heeringen - Metrics Questions in RFPs
ACOSM 2010ACOSM 20101515
Company name and Presentation Title hereCompany name and Presentation Title here
Agenda
Request for proposals
Software estimation using function points
Typical questions in request for proposals (RFP’s)
Supplier challenges
Recommendations for the client organization.
Harold van Heeringen - Metrics Questions in RFPs
ACOSM 2010ACOSM 20101616
Company name and Presentation Title hereCompany name and Presentation Title here
time
Concept
Definition
High
Level
Design
Low level
Design
Realization
Requirements: complete and detailed?
Idea
Why What How
Size: Function Points
RFP
4x
3x
2x
1x
0.8x
0.5x
Project Rate1 42 33 14 15 16 27 48 49 510 5Average 3
Harold van Heeringen - Metrics Questions in RFPs
ACOSM 2010ACOSM 20101717
Company name and Presentation Title hereCompany name and Presentation Title here
time
Size
Size always grows !
RFP
Concept
Definition
High level
design
Low level
designRealisation
Idea
Challenge: Which size should we use in our estimation
and which size will the competitors use in their
estimation?
Harold van Heeringen - Metrics Questions in RFPs
Why What How
ACOSM 2010ACOSM 20101818
Company name and Presentation Title hereCompany name and Presentation Title here
The Software equation (Putnam)E
ffo
rt (
ma
nh
ou
rs)
Duration
Plan A: 6 months, 4.500 manhours
Plan B: 7 months, 2.400 manhours
Size/productivity
= effort 1/3 * duration 4/3
Harold van Heeringen - Metrics Questions in RFPs
ACOSM 2010ACOSM 20101919
Company name and Presentation Title hereCompany name and Presentation Title here
Same project, different durationE
ffo
rt (
ma
nh
ou
rs)
/ Pe
ak s
ize
Duration
Plan A
Duration: 6 months
Effort: 4.500 manhours
Peak staff : 5,8 fte
MTTD: 1,764 daysPlan B
Duration: 7 months
Effort: 2.400 manhours
Peak staff: 2,7 fte
MTTD: 2,816 days
Harold van Heeringen - Metrics Questions in RFPs
ACOSM 2010ACOSM 20102020
Company name and Presentation Title hereCompany name and Presentation Title here
Scenarios
Estimate / Business Case
Cost depending on Time-to-market
Example Scenario 1:
Duration: 5,5 months
Effort: 5.000 Mhr
Team size: 6,7 fte
Cost: € 430.000
Example Scenario 2:
Duration: 5,2 months
Effort: 5.500 Mhr
Team size: 7,5 fte
Cost: € 480.000
Example Scenario 3:
Duration: 4,8 months
Effort: 5.900 Mhr
Team size: 8,3 fte
Cost: € 530.000
Example Scenario 4:
Duration: 4,5 months
Effort: 6.300 Mhr
Team size: 9,4 fte
Cost: € 620.000
Example Scenario 5:
Duration: 5,8 months
Effort: 5.200 Mhr
Team size: 6,2 fte
Cost: € 400.000
Example Scenario 6:
Duration: 6,1 months
Effort: 4.900 Mhr
Team size: 5,8 fte
Cost: € 380.000
Example Scenario 7:
Duration: 6,3 months
Effort: 4.700 Mhr
Team size: 5,5 fte
Cost: € 360.000
Eff
ort
(m
an
ho
urs
)
Duration
Harold van Heeringen - Metrics Questions in RFPs
ACOSM 2010ACOSM 20102121
Company name and Presentation Title hereCompany name and Presentation Title here
Challenge supplierP
ric
e p
er
fun
cti
on
po
int
Duration
Plan A: 767 €/FP
Plan B: 452 €/FP
Answer: 380 €/FP ??
Client expectation
What is your price per function point for a 500 FP Oracle system?
Harold van Heeringen - Metrics Questions in RFPs
ACOSM 2010ACOSM 20102222
Company name and Presentation Title hereCompany name and Presentation Title here
Professionalism en realism
Expertise Use of function point analysis
Database with experience data
Repository with Benchmark data / tooling
Realisme Opportunism: ‘buying projects’
Commercial interests
A non-realistic bid is not in anyones interest !!
Harold van Heeringen - Metrics Questions in RFPs
ACOSM 2010ACOSM 20102323
Company name and Presentation Title hereCompany name and Presentation Title here
Cost of high and low estimates
Non- Lineair extra costs
Planning errors
Enlarging team more expensive but almost not
faster
Extra management attention / overhead
Stress: More defects, lower maintainability
Lineair extra Costs
Extra hours will be spent
Harold van Heeringen - Metrics Questions in RFPs
ACOSM 2010ACOSM 20102424
Company name and Presentation Title hereCompany name and Presentation Title hereHarold van Heeringen - Metrics Questions in RFPs
A
Re
aliz
ati
on
(h
ou
rs)
5.000
15.000
CB
10.000
5.000 hours3.000 hours 7.000 hours
7.000
Estimation Result
B: Realistic
5.000 hours
7 months
Successful !
Efficient!
Successful !
Not efficient !
7.000 hours
11 months
A: Optimistic
3.000 hours
5 months
C: Pessimistic
7.000 hours
11 months
High and low estimates in practice
FAILS !!
10.000 hours
9 months
5.000 hours
7 months
Realistic estimation is the key to
succesful projects
ACOSM 2010ACOSM 20102525
Company name and Presentation Title hereCompany name and Presentation Title here
Agenda
Request for proposals
Software estimation using function points
Typical questions in request for proposals (RFP’s)
Supplier challenges
Recommendations for the RFP sender
Harold van Heeringen - Metrics Questions in RFPs
ACOSM 2010ACOSM 20102626
Company name and Presentation Title hereCompany name and Presentation Title here
Recommendations
Ask the right questions objective comparison, keeping other factors equal.
Assess the reality value of the quotation Establish a range in which the answer has to fall
Tools: for instance Galorath SEER, QSM SLIM or the ISBSG database
Demand objective evidence Experience data of the supplier
Harold van Heeringen - Metrics Questions in RFPs
ACOSM 2010ACOSM 20102727
Company name and Presentation Title hereCompany name and Presentation Title here
RFP Question should include
Metric to compare between competitors, for instance Productivity (hours/FP, Function points/hour, PI)
Costs (Price/FP)
Quality (defects per function point, Mean-time-to-defect (MTTD))
Technology for instance Java, Cobol, Oracle or MS.NET
Size (in Function Points or COSMIC FP)
Technical/ Functional Complexity for instance high/mediate/low
Phases/Activities to include for instance Technical Design, Coding, Unit testing, systems testing.
Duration required !!
Harold van Heeringen - Metrics Questions in RFPs
ACOSM 2010ACOSM 20102828
Company name and Presentation Title hereCompany name and Presentation Title here
Example of a good RFP Question
‘What is your productivity rate (hours/FP) for a moderately complex Java project of 500 function points and a duration of 20 weeks? Phases to include are technical design, coding, unit testing, systems testing and support of the user organization during the user acceptance test.’
Harold van Heeringen - Metrics Questions in RFPs
ACOSM 2010ACOSM 20102929
Company name and Presentation Title hereCompany name and Presentation Title here
Assess reality with ISBSG
ISBSG database R11 International Software Benchmarking Standards Group
R11: >5.200 projects ‘Best in Class’
Realistic range: 7,2 uur/FP – 11,6 uur/FP
Realistic range: 4,5 mnd - 9,5 mnd
ISBSG R11 hours/FP DurationNUMBER OF PROJECTS 24 24PERCENTILE 10% (P10) 3,5 3,3 mndPERCENTILE 25% (P25) 7,2 4,5 mndMEDIAN 8,4 6,0 mndPERCENTILE 75% (P75) 11,6 9,5 mndPERCENTILE 90% (P90) 19,6 12,2 mnd
Harold van Heeringen - Metrics Questions in RFPs
ACOSM 2010ACOSM 20103030
Company name and Presentation Title hereCompany name and Presentation Title here
Conclusions
Ask the right questions: Size, Cost, Productivity, Duration and Quality are highly
inter dependent
State the questions as specific as possible!
Evaluate bids Assess reality of proposals
Demand evidence of the supplier
Choose wisely Don’t go automatically for the cheapest supplier!
Harold van Heeringen - Metrics Questions in RFPs
ACOSM 2010ACOSM 20103131
Company name and Presentation Title hereCompany name and Presentation Title here
Next steps IWSM, Stuttgart (Nov. 10-12, 2010)
Workshop on proposing a standard framework for the use of software metrics in RFP management
Harold van Heeringen - Metrics Questions in RFPs
ACOSM 2010ACOSM 20103232
Company name and Presentation Title hereCompany name and Presentation Title here
staat voor resultaat
0031 88 6606600 +3165
metrieken.sogeti.nl
@haroldveendam
Harold van Heeringen - Metrics Questions in RFPs