joel semeniuk ceo, imaginet resources microsoft regional director microsoft mvp – team system...
TRANSCRIPT
Overcoming Project Estimation Madness
Joel SemeniukCEO, Imaginet ResourcesMicrosoft Regional DirectorMicrosoft MVP – Team SystemDPR205
What Is an Estimate?
Estimate <> Target or Commitment
Estimate
• Prediction
Target
• Desire
Commitment
• Promise
Estimate <> Planning
Estimation• Analytical Unbiased
Process• Goal = Accuracy
Planning• Biased Goal-Seeking
Process• Goal = Seek Result
Plans Need Good Estimates
Estimation
Scheduling
Identifying Critical Path
Breaking Down WorkPrioritization
Defining Iterations
Common ScenarioHow long do you think this project
should take? We need to have it ready in 3 months with these
features….
My estimates say we can do this in 5 months
5 Months? Didn’t you hear me? I need to have this in 3 months for
the trade show!
The manager wasn’t asking for an estimate but a plan to solve a problem…
Tip: When asked for an estimate – determine if it should be an estimate or a plan to hit a target
Single Point Estimates
Rarely is a single point estimate an estimateEstimates should include some level of probability
Tip: When you see a single point estimate – question to see if this is really a target
Tip: When you see a single point estimate – you should question the probability
Tip: Don’t use a planning tool to make estimates (e.g., MS Project)
Using MS Project and Single Point Estimates
ID Task Name Work Baseline Variance Actual Remaining % W. Comp.
0 New Product 14,272 hrs 0 hrs 14,272 hrs 0 hrs 14,272 hrs 0%1 New Product Development Template14,272 hrs 0 hrs 14,272 hrs 0 hrs 14,272 hrs 0%2 Initial New Product Screening Stage72 hrs 0 hrs 72 hrs 0 hrs 72 hrs 0%3 New product opportunity identified0 hrs 0 hrs 0 hrs 0 hrs 0 hrs 0%4 Describe new product idea (1-page written disclosure)16 hrs 0 hrs 16 hrs 0 hrs 16 hrs 0%5 Gather information required for go/no-go decision48 hrs 0 hrs 48 hrs 0 hrs 48 hrs 0%6 Convene opportunity of screening committee (decision to pursue idea or not)8 hrs 0 hrs 8 hrs 0 hrs 8 hrs 0%7 Decision point - go/no-go to preliminary investigation0 hrs 0 hrs 0 hrs 0 hrs 0 hrs 0%
Really?
Estimation ChallengesProvide a 90% Confident Estimate for the following:
How many people are at this conference? What is the size of the largest whale ever discovered? How many bugs will you have in your next software project? What is the distance between Mars and the Sun? What is the surface temperature of the sun? What is the total volume of the Great Lakes?
Question: What makes you think you are 90% confident?Question: Did you use an artificially narrow range?Question: Where does the pressure to use a narrow range come from?
Two Important Laws of Nature
Parkinson’s LawIf you give a person 5 days to deliver a task that could be delivered in 4, the person will find something to do with that extra day
The Student SyndromeIf given too much time, people procrastinate until late in the project, at which point they rush to complete the work, and likely won’t finish on time
These are two common arguments against Overestimation
Uncertainty and Software Estimation
Not all requirements are known until development time in most cases
Narrowing the Cone
Tip: The cone does not narrow itself; it narrows when you continually make decisions that narrow variability
Sources of Error1. Chaotic Development Process2. Unstable Requirements3. Omitted Activities4. Unfounded Optimism5. Subjectivity6. Off-the-Cuff Estimates
Things that Influence Estimates
Project SizeType of Software Being DevelopedPersonnel FactorsProgramming Languagesand LOTS of others such as:
• Complexity• Constraints• Turnover• Experience• Tools• Ego
• Team Cohesion• Process maturity• Architecture Risk• Technology Maturity• I’m sure you can name 10 more…
Estimation PracticesCount, Compute, Judge
Calibration and Historical Data
Expert Judgment
Decomposition and Recomposition
Estimation by Analogy
Proxy-Based Estimates
Expert Judgment in Groups
Use of Multiple Approaches
Count, Compute, Judge
Count, Compute, Judge
If you can count something – count it Use “expert” judgment when there’s nothing to countWhat to count?
Something correlated with size of softwareSomething we can count sooner rather than laterSomething that will produce a statistically meaningful averageSomething that you understandSomething you can count with minimal effort
Count, Compute, Judge
Examples of things you can count…
List of things More things
Requirements Web Pages
Features Bugs
Use cases Reports
Stories Dialog boxes
Function Points Database Tables
Change Requests Classes
Tests Cucumbers
Calibration and Historical Data
Helps to transform Count into EstimateBest way to predict today’s weather is to look at yesterday’sWays to calibrate:
Industry DataHistorical DataProject Data
Industry Data
Historic Project Data Example
Team Size Average Stories Delivered per Sprint
1 5
2–3 12
4–5 22
6–7 31
8 No Data
Tip: Using Time and “Actual Effort” as historical data is difficult
Current Project Data
Use Data from the Current Project to Predict the Rest of the Project
Data for Sprint 1
27 story points (sp) delivered
12 staff weeks (sw)
3 calendar weeks (cw)
Preliminary Calibration
Effort = 27sp / 12sw = 2.25 sp per sw
Schedule = 27sp / 3w = 9 sp per cw
Remainder of Project
Effort = 2.25 sp per staff week (spsw)
Schedule = 9 sp per calendar week (spcw)
Project size = 180 sp
Preliminary Whole-Project Estimate
Effort = 180sp/2.25 spsw = 80 staff weeks
Schedule = 180sp / 9 spcw = 20 calendar weeks
Individual Expert JudgmentTwo Types
Intuitive Expert Judgment (Spidey Senses) – not very accurateStructured Expert Judgment – almost as accurate as model-based estimates
Who Estimates?People who are doing the work – most accurate
GranularityEnd up with estimates that are around ¼–1 day large
Use RangesBest Case, Most Likely Case, Worst Case
Use FormulasOptimistic PERT = [BestCase+(4XMostLikely)+WorstCase]/6Pessimistic PERT = [BestCase+(3xMostLikely)+(2xWorstCase)]/6
Decomposition and RecompositionAKA – Bottom UpThe Law of Large Numbers
Single Large Estimates are usually Largely wrong in a single direction (+/-)Breaking into smaller pieces – some are above, some are lower
The error at the detail level balances out the error on the whole
How Small to Break Things Down?You need 5–10 smaller things to start seeing benefits
Use Common Work Breakdown to Avoid Missing Activities
Category Create/Do Plan Manage Review Rework DefectIntegration X X X X
Documentation X X X X X
Etc X X X
Big Problem with Bottom UP
Aggregating Estimates Doesn’t Work WellUse complex standard deviation formulas to aggregate when you have more than 10 tasksThis can get VERY complicated if you want to be accurateTake a statistics course for more information
Estimation by Analogy
Check for congruent assumptions
Create an effort based on the size of the new project compared to old
Build up estimate of new project as a % of the old project size
Compare size of new project piece by piece
Get details of previous project decomposed by feature area, work breakdown or some other scheme
Proxy-based Estimates
Find a Proxy that correlates to what you are estimating that is easier to estimate/count
E.g., Estimation of # of Test Cases is correlated to # of Requirements
Some use Proxies to estimate lines of codeWhat do lines of code really give you?
Some great techniques Story PointsT-Shirt Sizing
Good for estimating test cases, defects, documentation, etc.
Story Points
Assign a size to each story you are building
Story Point Scale Specific Points to the Scale
Powers of 2 1, 2, 4, 8, 16, 32
Fibonacci 1, 2, 3, 5, 8, 13, 21
Story Points ExampleStory Points
Story 1 2
Story 2 1
Story 3 4
Story 4 8
…
Story 60 2
Total 180
Need to Calibrate an IterationData for Iteration 1
27 Story Points Delivered
12 Staff weeks
3 Calendar Weeks
Calibration:
Effort = 27 story points / 12 staff weeks = 2.25 story points/staff week
Schedule = 27 story points / 3 calendar weeks = 9 story points/calendar week
Projecting Further
Assumptions from Calibration
Effort = 2.25 story points/staff week
Schedule = 9 story points/calendar week
Project size = 180 story points
Preliminary Whole-Project Estimate
Effort = 180 story points / 2.25 story points/staff week = 80 staff weeks
Schedule = 180 story points / 9 story points/calendar week = 20 calendar weeks
T-Shirt Sizing
How do Sales people and Delivery people work together?
Sales people need to know relative size to help determine priorityEstimator won’t estimate until further decomposedEarly in the Cone of Uncertainty
Remember – goal isn’t accuracy, but accurate enough to support control and decisionsCan use T-shirt sizing to help
T-Shirt SizingFeature Business Value Dev Cost
Feature A Large Small
Feature B Small Large
Feature C Large Large
Feature D Medium Medium
Using for PrioritizationBusiness Value Extra Large Large Medium Small
Extra Large 0 4 6 7
Large -4 0 2 3
Medium -6 -2 0 1
Small -7 -3 -1 -
Feature Bus. Value Dev Cost Net Value
Feature A L S 3
Feature B L M 2
Feature C L L 0
Feature D M M 0
Expert Judgment in Groups
Group reviews can be funHave each team estimate pieces individually, compare resultsDon’t average estimatesArrive at consensus
TechniquesWideband Delphi Technique (quite formal)Estimation Poker (quite fun)
Mixing Things Up
E.g., Expert Proxy Judgment in Groups
Quick Demos
Feature Prioritization Worksheet – T-Shirt Sizing (kinda)Feature Point WorksheetMulti-Point WorksheetEstimation Poker
Some Parting Thoughts
Keep Estimation SimpleTime Boxing is your friendDo things that minimize uncertaintyDon’t be afraid to re-estimate and thus, re-planEstimation via consensus is harder than you thinkSometimes you have to commit; I’m sorryBe careful when recording historic data – time-based data is all a lie and lines of code are meaningless when technology changesFind a method that works for you – then continually improve
Reference
[email protected] & answer
www.microsoft.com/teched
Sessions On-Demand & Community
http://microsoft.com/technet
Resources for IT Professionals
http://microsoft.com/msdn
Resources for Developers
www.microsoft.com/learningMicrosoft Certification and Training Resources
www.microsoft.com/learning
Microsoft Certification & Training Resources
Resources
Complete an evaluation on CommNet and enter to win!
© 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS,
IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.