statistical analysis of new product development (npd) cycle-time data
DESCRIPTION
Alpha Technologies recently concluded a comprehensive study of historical NPD performance data (cost, cycle-time, resource requirements, program type and complexity, plans versus actuals, etc.) and has successfully applied the results of this study to make measureable improvements to their NPD process. Probability-based summary data is used to aide in planning and budgeting, set statistically valid continuous improvement targets for performance scorecards, and develop a visual resource management tool for allocation of human resources to present and future NPD programs.TRANSCRIPT
Quality and Business Excellence
Celebration
ASQ Vancouver 25th Anniversary
Statistical Analysis of New Product Development (NPD) Cycle-time Data Including Applications of Results
Steve Pratt, MEng., PE, CSSBB Director of Engineering, Alpha Technologies
Alpha Technologies
Company Background
Alpha is a Full Service Power
Systems Provider to Multiple Markets
Pratt – Slide 1
Power Modules
Indoor Power Systems
Outdoor Power Systems
Alpha’s Power Solutions Products
Pratt – Slide 2
Phase-Gate New Product
Development (NPD) Process
Deliverables: • Product Concept • Business Case • Preliminary Plan
Deliverables: •High-Level Design •Requirements •Complete Project Plan
Deliverables: • Design Outputs • DFx Reviews • Preliminary Test
Reports
Deliverables: • Pilot Doc Pack • Pilot Build • Sales Forecasts
Deliverables: • Compliance
Certifications • Training • MarCom docs
Deliverables: • Sustaining Eng • Repair/Support • Cost Reduction
Phase 2 Planning
Phase 1 Concept
Phase 3 Development
Phase 4 Qualification and Pre-Production
Phase 5 LA and Production Ramp-up
Phase 6 GA and MOL
Pratt – Slide 3
Concurrent Engineering via Cross-
Functional Core Teams
Service Supply Chain
Product Management
Program Manager
Quality
Engineering
Manufacturing
Pratt – Slide 4
Motivation for Study
Continuous Improvement:
• general perception that NPD takes too long
• lack of proper management of resources
• desire to establish performance benchmarks
Pratt – Slide 5
NPD Data Collected and Analyzed
Schedule Data
• Recorded Dates: Start, Gate 1, Gate 2, Gate 3, Gate 4 (LA) & Gate 5 (GA)
• Program Plans: estimated time to LA, estimated time to GA
• Calculated: total time, time per phase, actual vs. plan
Cost Data
• Timecard System: actual total effort (man-hours)
• Program Plans: estimated total effort
• Calculated: effort by function, effort by phase, remaining effort, actual vs. plan
Pratt – Slide 6
Tools Used in the Study
JMP – Statistical Discovery Software Analyze-Distribution Platform: • Calculations: Quantiles, Moments • Plots: Histogram, Quantile Box, Normal Quantile • Fit Distribution: Normal, Beta, Goodness of Fit Tests
Fit Y by X Platform: Bivariate Analysis • Fit Line, Fit Polynomial, Summary of Fit
Fit Y by X Platform: One-way Analysis • Calculations: Means and Standard Deviations • Plots: Mean Diamonds • Analysis of Variance (ANOVA)
Fit Model Platform • Calculations: Standard Least Squares, Summary of Fit, Effect Tests • Plots: Actual by Predicted, Effect Leverage, Residual by Predicted • Two-way ANOVA with interactions
Microsoft Excel and PowerPoint
Pratt – Slide 7
External Benchmarking Data – PDMA
Best Practices Research
Provides industry-average NPD Cycle-Time based on complexity of programs:
• new-to-the world • new-to-the firm • next-gen improvements • incremental improvements
Pratt – Slide 8
Source: Griffin A., “Product development cycle time for business-to-business products,” Industrial Marketing Management 2002;31: 291-304 *
Average NPD Cycle-Time Benchmarking
Pratt – Slide 9
Planned vs. Actual NPD Cycle-Time
Pratt – Slide 10
ANOVA to Identify Significant Factors
for Schedule Data
Pratt – Slide 11
ANOVA to Identify Significant Factors
for Effort Data
Pratt – Slide 12
Categorizing NPD Programs into 9
Buckets
Product Types:
Modules (power modules, shelves, controllers) Indoor Systems (rack-based systems, racks, distribution) OSP (outside plant power/battery systems)
Program Complexities:
A – major program requiring advanced development B – completely new, but no advanced development C – new with incremental development
Pratt – Slide 13
NPD Summary Data –
Schedule (Gate 2 to LA)
Pratt – Slide 14
NPD Summary Data –
Schedule (Gate 2 to GA)
Pratt – Slide 15
NPD Summary Data –
Effort (Total Development Hours)
Pratt – Slide 16
NPD Schedule & Effort Continuous
Probability Distributions
Pratt – Slide 17
Additional Analyses
• plan vs. actual – schedule and effort
• schedule and effort variance by phase
• time spent per phase
• total effort by function and phase
• default team size and composition
Pratt – Slide 18
Applications
Better Estimating
Pratt – Slide 19
More Accurate Gate 2 Point Estimates
of Schedule and Effort
Previous estimates were:
- subject to negotiation
- overly optimistic and aggressive
- “rose-colored” recollections of past
Pratt – Slide 20
Alpha’s Historical Schedule
Estimation Accuracy
Median MRE = 44%
Pratt – Slide 21
NPD Schedule Point-Estimation via
Historical Mean Values
Median MRE = 19%
Pratt – Slide 22
Basic Schedule Equation Using Historical Mean Effort Values
Median MRE = 21%
Pratt – Slide 23
Informal Comparisons Equation Using
NPD Mean Values
Median MRE = 20%
Pratt – Slide 24
Mean Magnitude of Relative Error –
Schedule Estimates
Pratt – Slide 25
Alpha’s Historical Effort
Estimation Accuracy
Median MRE = 36%
Pratt – Slide 26
NPD Effort Point-Estimation via
Historical Mean Values
Median MRE = 18%
Pratt – Slide 27
Mean Magnitude of Relative Error –
Effort Estimates
Pratt – Slide 28
Determine the “Cone of Uncertainty”
Pratt – Slide 29
Cone of Uncertainty for NPD Project
Estimates
Pratt – Slide 30
Cone of Uncertainty for NPD Project
Schedule Data
Pratt – Slide 31
Cone of Uncertainty Applied to the 9
NPD Project Buckets
Pratt – Slide 32
Schedule Cone of Uncertainty Quantified by Program Phase
Pratt – Slide 33
Effort Cone of Uncertainty Quantified
by Program Phase
Pratt – Slide 34
Effort Estimate Ranges by Program
Phase
Pratt – Slide 35
Applications
Top-Down Project Planning
Pratt – Slide 36
Creation of Top-Down Program
Schedules
Pratt – Slide 37
NPD – Time Spent per Phase
Pratt – Slide 38
How Much Time Should Be Spent on
Planning?
Pratt – Slide 39
Applications
Performance Goal Setting
Pratt – Slide 40
SMART Performance Improvement
Goals
• fact-based and statistically valid
• objective and quantifiable
• deterministic at the start of a program
• consistent across all types of programs
• an enabler of continuous improvement
• unambiguous and easy to calculate
An ideal performance improvement measure is:
Pratt – Slide 41
RSD: a Normalized Measure of
Dispersion
Pratt – Slide 42
Program Scoring: RSD as Performance
Unit-of-Measure
Pratt – Slide 43
2013 NPD Cycle-Time Performance
Pratt – Slide 44
Earned-Value Milestones to Score
WIP NPD Programs
Pratt – Slide 45
Applications
Better Project Selection
Pratt – Slide 46
Deterministic Business Case Analysis
“Figures of Merit”
Pratt – Slide 47
Probabilistic Business Case Analysis
Beta Distribution
Pratt – Slide 48
Monte Carlo Simulation of
Cash Flows ($000)
Trial #1 Trial #2 Trial #3 Trial #4 Trial #5000
NPV $556 $461 $681 $447 $621
NRE 562 560 597 530 664
GM$ year1
348 274 241 220 274
GM$ year2
604 653 407 579 635
GM$ year3
451 651 644 447 558
Pratt – Slide 49
Probability-Based Figures of Merit
Pratt – Slide 50
Applications
Resource Management
Pratt – Slide 51
Default Team Size and Composition
Effort divided by Schedule represents number of individuals
Pratt – Slide 52
Roadmap and Budget Planning
Pratt – Slide 53
Program Resource Management
Engineering Pokémon board
• visual resource management tool
• 9 program type/complexity buckets
• default team size/composition
Pratt – Slide 54
NPD Programs are Represented by a
Series of Magnetic Trays
Pratt – Slide 55
Program Trays Incorporate Default
Team Size/Composition
Pratt – Slide 56
NPD Team Members are Represented
by a Series of Cards
Pratt – Slide 57
Engineering Pokémon – Additional
Pieces
Program Status Indicators
Bonus Targets Add-on Resource Trays “Ghost” Cards
Important Date Indicators
Clear Plastic Overlays
Pratt – Slide 58
The Board in Action
Pratt – Slide 59
Resulting Improvements
Pratt – Slide 60
Improved New Product Development
Better planning and management of resources…
• more stability and focus / less chaos
• increased efficiency
• enhanced organizational understanding
• faster NPD cycle-time
Pratt – Slide 61
In Conclusion:
It is never too late to start recording data!
- even simple data such as key dates and time spent on activities can facilitate powerful analyses
- capture data in real-time; don’t rely on memory or post mortem activities
Pratt – Slide 62
Questions???
Pratt – Slide 63
Thank You!
STEVEN PRATT, MEng., PE
Director of Engineering
www.alpha.ca
Pratt – Slide 64