using the isbsg data to improve your organization success - van heeringen (metrikon 2013)
DESCRIPTION
This presentation gives an introduction into the ISBSG organization and shows how the ISBSG products and data can help the software industry to become more mature. Three cases from personal experience are showed: a reality check of an expert estimate (that turned out to be optimistic), the analysis of the outgoing bids of a software supplier and the measurement of the performance of a single supplier, providing target productivity levels they should reach.TRANSCRIPT
![Page 1: Using the ISBSG data to improve your organization success - van Heeringen (Metrikon 2013)](https://reader033.vdocuments.net/reader033/viewer/2022042814/555d5123d8b42a52368b49a7/html5/thumbnails/1.jpg)
Harold van Heeringen,
ISBSG president
![Page 2: Using the ISBSG data to improve your organization success - van Heeringen (Metrikon 2013)](https://reader033.vdocuments.net/reader033/viewer/2022042814/555d5123d8b42a52368b49a7/html5/thumbnails/2.jpg)
� We measured 350 FP, so... What is the estimate?
� � Effort, duration, quality, risk?
� � Price per FP?
� Why are we losing most of the bids we do?
� Competitive advantage? Or not?
Size does matter, but….
� Competitive advantage? Or not?
� What is our productivity compared to the industry?
� We are going to send out a request for bid based on price/FP
� How can we compare the proposals?
� How can we exclude unrealistic (overoptimistic) ones?
� What should the market average bid approximately look like?
� I did a project and the client thinks it was too expensive
� How did I perform against similar projects?
![Page 3: Using the ISBSG data to improve your organization success - van Heeringen (Metrikon 2013)](https://reader033.vdocuments.net/reader033/viewer/2022042814/555d5123d8b42a52368b49a7/html5/thumbnails/3.jpg)
Historical data of completed projects!!
The answer…
Historical data of completed projects!!
![Page 4: Using the ISBSG data to improve your organization success - van Heeringen (Metrikon 2013)](https://reader033.vdocuments.net/reader033/viewer/2022042814/555d5123d8b42a52368b49a7/html5/thumbnails/4.jpg)
� International Software Benchmarking Standards Group
� Independent and not-for-profit
� Full Members are non-profit organizations, like DASMA, IFPUG, FiSMA, QESP and NESMA
Historical data: ISBSG repositories
� Grows and exploits two repositories of software data:
� New development projects and enhancements (> 6000 projects)
� Maintenance and support (> 1000 applications)
� Everybody can submit project data
� DCQ on the site
� Anonymous
� Free benchmark report in return
![Page 5: Using the ISBSG data to improve your organization success - van Heeringen (Metrikon 2013)](https://reader033.vdocuments.net/reader033/viewer/2022042814/555d5123d8b42a52368b49a7/html5/thumbnails/5.jpg)
� Mission: “To improve the management of IT resources by both business and government, through the provision and exploitation of public repositories of software engineering knowledge that are standardized, verified, recent and representative of current technologies”.
ISBSG
� All ISBSG data is � validated and rated in accordance with its quality guidelines
� current
� representative of the industry
� independent and trusted
� captured from a range of organization sizes and industries
![Page 6: Using the ISBSG data to improve your organization success - van Heeringen (Metrikon 2013)](https://reader033.vdocuments.net/reader033/viewer/2022042814/555d5123d8b42a52368b49a7/html5/thumbnails/6.jpg)
Website and portal
![Page 7: Using the ISBSG data to improve your organization success - van Heeringen (Metrikon 2013)](https://reader033.vdocuments.net/reader033/viewer/2022042814/555d5123d8b42a52368b49a7/html5/thumbnails/7.jpg)
� ISBSG wishes to help restore the confidence in IT, by giving transparency of what is realistic
� Improved project success rates
� Better customer satisfaction
ISBSG � IT Confidence
� Better customer satisfaction
� Shorter time to market
� Better alignment with the business/customer
� Easier contracting, resulting in better relationships between customer and supplier
![Page 8: Using the ISBSG data to improve your organization success - van Heeringen (Metrikon 2013)](https://reader033.vdocuments.net/reader033/viewer/2022042814/555d5123d8b42a52368b49a7/html5/thumbnails/8.jpg)
3 Cases from my experience
� Case 1: Telecom project reality check on the expert estimate
� Case 2: Assessing the competitive position of an organization
� Case 3: Supplier Performance Measurement
![Page 9: Using the ISBSG data to improve your organization success - van Heeringen (Metrikon 2013)](https://reader033.vdocuments.net/reader033/viewer/2022042814/555d5123d8b42a52368b49a7/html5/thumbnails/9.jpg)
Case 1� A telecom company wished to develop a new Java
system for the maintenance of subscription types;
� A team of experts studied the requirements documents and filled in the WBS-based estimation documents and filled in the WBS-based estimation calculation (bottom-up estimate);
� They decide that an estimate of 5.500 hours and a duration of 6 months should be feasible;
� The project manager decided not to believe the experts ‘on their blue eyes’ only and wished to carry out a reality check.
![Page 10: Using the ISBSG data to improve your organization success - van Heeringen (Metrikon 2013)](https://reader033.vdocuments.net/reader033/viewer/2022042814/555d5123d8b42a52368b49a7/html5/thumbnails/10.jpg)
ISBSG Reality Check: Effort� An estimated FPA comes up with the expected size:
� Min: 550 FP, likely 850 FP, Max 1300 FP
� Selecting the most relevant projects in the ISBSG D&E repository show the next results:
Functional SizeFunctional Sizeshow the next results:
PDR (h/FP)
Min. 3,2
Percentile 10% 4,3
Percentile 25% 6,2
Median 8,9
Percentile 75% 12,9
Percentile 90% 19,8
Max. 34,2
N 89
PDR (h/FP)
Min. 3,2
Percentile 10% 4,3
Percentile 25% 6,2
Median 8,9
Percentile 75% 12,9
Percentile 90% 19,8
Max. 34,2
N 89
550 850 1300
3.410 5.270 8.060
4.895 7.565 11.570
7.095 10.965 16.770
Functional Size
550 850 1300
3.410 5.270 8.060
4.895 7.565 11.570
7.095 10.965 16.770
Functional Size
5.500 hours seems optimistic
![Page 11: Using the ISBSG data to improve your organization success - van Heeringen (Metrikon 2013)](https://reader033.vdocuments.net/reader033/viewer/2022042814/555d5123d8b42a52368b49a7/html5/thumbnails/11.jpg)
ISBSG Reality Check: Duration� Same analysis is possible
� Also, formulas have been published in the Practical Project Estimation book
� For instance:� For instance:
table C-2.2 Project Duration, estimated from software size only
Functionele omvang 550 FP
C uit tabel 0,507
E1 uit tabel 0,429
Duration = C * Size^E1
Duration = 7,6 elapsed months
550 850 1300
Duration 7,6 9,2 11,0
![Page 12: Using the ISBSG data to improve your organization success - van Heeringen (Metrikon 2013)](https://reader033.vdocuments.net/reader033/viewer/2022042814/555d5123d8b42a52368b49a7/html5/thumbnails/12.jpg)
Result� Expert estimate was assessed optimistic
� Adjusted Estimate:
� Effort: 8.000 hours
Duration: 10 months� Duration: 10 months
� This turned out to be quite accurate!
� The project manager now always carries out a reality check and is ‘spreading the word’.
![Page 13: Using the ISBSG data to improve your organization success - van Heeringen (Metrikon 2013)](https://reader033.vdocuments.net/reader033/viewer/2022042814/555d5123d8b42a52368b49a7/html5/thumbnails/13.jpg)
Case 2� Senior management of a software company wondered how
competitive they were when it comes to productivity.
� Many bids for projects were lost and they wished to improve, especially their Microsoft .Net department.
� Analysis of the bids by department showed the next figures:
Nr. of bids 23
Average PDR in bid 16,3 h/FP
Average Size (FP) 230 FP
Average teamsize 6 fte
PDR (h/FP)
Min. 3,2
Percentile 10% 3,8
Percentile 25% 5,9
Median 7,6
Percentile 75% 12,9
Percentile 90% 18,9
Max. 34,2
N 35
ISBSG data analysis
![Page 14: Using the ISBSG data to improve your organization success - van Heeringen (Metrikon 2013)](https://reader033.vdocuments.net/reader033/viewer/2022042814/555d5123d8b42a52368b49a7/html5/thumbnails/14.jpg)
Case 2� Analysis of the bid phase showed a number of issues:
� Estimates were extremely pessimistic due to severe penalties in case of overruns;
� In a number of stages, risk surcharges were added;� In a number of stages, risk surcharges were added;
� They wished to work in fixed team of 6 fte, but ISBSG data shows that the project size was usually to small for this teams size to be efficient;
� Because of the knowledge that the department bids were not market average (or better), the bid process was redesigned, making the company more successful!
![Page 15: Using the ISBSG data to improve your organization success - van Heeringen (Metrikon 2013)](https://reader033.vdocuments.net/reader033/viewer/2022042814/555d5123d8b42a52368b49a7/html5/thumbnails/15.jpg)
Case 3� An organization outsourced all of their IT work to one
supplier;
� Therefore, the competition was gone and potentially the supplier could charge whatever they wish;the supplier could charge whatever they wish;
� The ISBSG data was used to construct the productivity targets for the supplier;
� Bonus/malus arrangements were agreed upon based on these targets.
![Page 16: Using the ISBSG data to improve your organization success - van Heeringen (Metrikon 2013)](https://reader033.vdocuments.net/reader033/viewer/2022042814/555d5123d8b42a52368b49a7/html5/thumbnails/16.jpg)
Case 3: Supplier targets
7
8
9
10
11
12
13
PD
R (
h/
FP
)
500
600
700
800
900
1000
1100
1200
1300
EU
R/
FP
Target PDR PDR (h/FP)
500
Target PCR PCR (EUR/FP)
0
5
10
15
20
25
30
35
40
45
jan-13 feb-13 mrt-13 apr-13 mei-13 jun-13 jul-13 aug-13 sep-13 okt-13
Target Defects/1000 FP Defects/1000 FP
The supplier is measured continuously and still has to make his target for the first time!
The organization is happy that the trends show improvement and they feel in control.
![Page 17: Using the ISBSG data to improve your organization success - van Heeringen (Metrikon 2013)](https://reader033.vdocuments.net/reader033/viewer/2022042814/555d5123d8b42a52368b49a7/html5/thumbnails/17.jpg)
Other useful purposes� Analyze the difference in productivity or quality
between two (or more) types of projects:
� Traditional vs. Agile
� Outsourced vs. In-house� Outsourced vs. In-house
� Government vs. Non-government
� One site, multi site
� Reuse vs. no reuse
� � ISBSG Special Analysis reports
![Page 18: Using the ISBSG data to improve your organization success - van Heeringen (Metrikon 2013)](https://reader033.vdocuments.net/reader033/viewer/2022042814/555d5123d8b42a52368b49a7/html5/thumbnails/18.jpg)
Traditional vs. Agile� Agile productivity: 10 – 20% gain after 1 year of
adoption
� Agile cost: 20-40% lower after 1 year of adoption
� Agile time-to-market: 10-60% less� Agile time-to-market: 10-60% less
� Agile quality (post production defects): 2-8% higher
� �10 Take Aways from Reifer Agile Report (www.isbsg.com)
![Page 19: Using the ISBSG data to improve your organization success - van Heeringen (Metrikon 2013)](https://reader033.vdocuments.net/reader033/viewer/2022042814/555d5123d8b42a52368b49a7/html5/thumbnails/19.jpg)
Special reports� Impact of Software Size on Productivity
� Government and Non-Government Software Project Performance
� ISBSG Software Industry Performance report� ISBSG Software Industry Performance report
� ISBSG The Performance of Business Application, Real-Time and Component Software Projects
� Estimates – How accurate are they?
� Planning Projects – Role Percentages
� Team size impact on productivity
� Manage your M&S environment – what to expect?
� Many more
![Page 20: Using the ISBSG data to improve your organization success - van Heeringen (Metrikon 2013)](https://reader033.vdocuments.net/reader033/viewer/2022042814/555d5123d8b42a52368b49a7/html5/thumbnails/20.jpg)
![Page 21: Using the ISBSG data to improve your organization success - van Heeringen (Metrikon 2013)](https://reader033.vdocuments.net/reader033/viewer/2022042814/555d5123d8b42a52368b49a7/html5/thumbnails/21.jpg)
Country reportWhere is Germany??
![Page 22: Using the ISBSG data to improve your organization success - van Heeringen (Metrikon 2013)](https://reader033.vdocuments.net/reader033/viewer/2022042814/555d5123d8b42a52368b49a7/html5/thumbnails/22.jpg)
USA
![Page 23: Using the ISBSG data to improve your organization success - van Heeringen (Metrikon 2013)](https://reader033.vdocuments.net/reader033/viewer/2022042814/555d5123d8b42a52368b49a7/html5/thumbnails/23.jpg)
Government vs. Non-government
![Page 24: Using the ISBSG data to improve your organization success - van Heeringen (Metrikon 2013)](https://reader033.vdocuments.net/reader033/viewer/2022042814/555d5123d8b42a52368b49a7/html5/thumbnails/24.jpg)
Role percentages
![Page 25: Using the ISBSG data to improve your organization success - van Heeringen (Metrikon 2013)](https://reader033.vdocuments.net/reader033/viewer/2022042814/555d5123d8b42a52368b49a7/html5/thumbnails/25.jpg)
� Everybody wants to use data
� But nobody wants to submit data… Why not?
� Is it hard?
Is there a risk?
We need data!
� Is there a risk?
� Is the reward not big enough?
� Why not try it? You’ll get a free Benchmark report and 100 portal credits in return!!
� Are there any factors preventing you?
� WWW.ISBSG.ORG
![Page 26: Using the ISBSG data to improve your organization success - van Heeringen (Metrikon 2013)](https://reader033.vdocuments.net/reader033/viewer/2022042814/555d5123d8b42a52368b49a7/html5/thumbnails/26.jpg)
Conference offer
DASMA members always get a 10% discount, using the code provided earlier this year.
H.S.1
![Page 27: Using the ISBSG data to improve your organization success - van Heeringen (Metrikon 2013)](https://reader033.vdocuments.net/reader033/viewer/2022042814/555d5123d8b42a52368b49a7/html5/thumbnails/27.jpg)
Dia 26
H.S.1 van Heeringen; 15-11-2013
![Page 28: Using the ISBSG data to improve your organization success - van Heeringen (Metrikon 2013)](https://reader033.vdocuments.net/reader033/viewer/2022042814/555d5123d8b42a52368b49a7/html5/thumbnails/28.jpg)
Thank you!
![Page 29: Using the ISBSG data to improve your organization success - van Heeringen (Metrikon 2013)](https://reader033.vdocuments.net/reader033/viewer/2022042814/555d5123d8b42a52368b49a7/html5/thumbnails/29.jpg)
NEtherlands Software Metrics users Association
![Page 30: Using the ISBSG data to improve your organization success - van Heeringen (Metrikon 2013)](https://reader033.vdocuments.net/reader033/viewer/2022042814/555d5123d8b42a52368b49a7/html5/thumbnails/30.jpg)
To celebrate our anniversary NESMAwill organise the IWSM Mensura
October 6-8, 2014
2014.iwsm-mensura.org
October 6-8, 2014
![Page 31: Using the ISBSG data to improve your organization success - van Heeringen (Metrikon 2013)](https://reader033.vdocuments.net/reader033/viewer/2022042814/555d5123d8b42a52368b49a7/html5/thumbnails/31.jpg)
HistoricalHistorical landmarkslandmarks
![Page 32: Using the ISBSG data to improve your organization success - van Heeringen (Metrikon 2013)](https://reader033.vdocuments.net/reader033/viewer/2022042814/555d5123d8b42a52368b49a7/html5/thumbnails/32.jpg)
Modern Modern landmarkslandmarks
![Page 33: Using the ISBSG data to improve your organization success - van Heeringen (Metrikon 2013)](https://reader033.vdocuments.net/reader033/viewer/2022042814/555d5123d8b42a52368b49a7/html5/thumbnails/33.jpg)
EuropeanEuropean CulturalCultural CapitalCapital 20012001
![Page 34: Using the ISBSG data to improve your organization success - van Heeringen (Metrikon 2013)](https://reader033.vdocuments.net/reader033/viewer/2022042814/555d5123d8b42a52368b49a7/html5/thumbnails/34.jpg)
The city of modern The city of modern architecturearchitecture
![Page 35: Using the ISBSG data to improve your organization success - van Heeringen (Metrikon 2013)](https://reader033.vdocuments.net/reader033/viewer/2022042814/555d5123d8b42a52368b49a7/html5/thumbnails/35.jpg)
LargestLargest soccerstadiumsoccerstadium in the in the NetherlandsNetherlands
![Page 36: Using the ISBSG data to improve your organization success - van Heeringen (Metrikon 2013)](https://reader033.vdocuments.net/reader033/viewer/2022042814/555d5123d8b42a52368b49a7/html5/thumbnails/36.jpg)
WelcomeWelcome toto
RotterdamRotterdam
![Page 37: Using the ISBSG data to improve your organization success - van Heeringen (Metrikon 2013)](https://reader033.vdocuments.net/reader033/viewer/2022042814/555d5123d8b42a52368b49a7/html5/thumbnails/37.jpg)
BusiestBusiest port in Europeport in Europe
![Page 38: Using the ISBSG data to improve your organization success - van Heeringen (Metrikon 2013)](https://reader033.vdocuments.net/reader033/viewer/2022042814/555d5123d8b42a52368b49a7/html5/thumbnails/38.jpg)
2014.iwsm-mensura.org
I hope to see you next year
October 6-8, 2014