statistical analysis of new product development (npd) cycle-time data

Quality and Business Excellence

Celebration

ASQ Vancouver 25th Anniversary

Statistical Analysis of New Product Development (NPD) Cycle-time Data Including Applications of Results

Steve Pratt, MEng., PE, CSSBB Director of Engineering, Alpha Technologies

Alpha Technologies

Company Background

Alpha is a Full Service Power

Systems Provider to Multiple Markets

Pratt – Slide 1

Power Modules

Indoor Power Systems

Outdoor Power Systems

Alpha’s Power Solutions Products

Pratt – Slide 2

Phase-Gate New Product

Development (NPD) Process

Deliverables: • Product Concept • Business Case • Preliminary Plan

Deliverables: •High-Level Design •Requirements •Complete Project Plan

Deliverables: • Design Outputs • DFx Reviews • Preliminary Test

Reports

Deliverables: • Pilot Doc Pack • Pilot Build • Sales Forecasts

Deliverables: • Compliance

Certifications • Training • MarCom docs

Deliverables: • Sustaining Eng • Repair/Support • Cost Reduction

Phase 2 Planning

Phase 1 Concept

Phase 3 Development

Phase 4 Qualification and Pre-Production

Phase 5 LA and Production Ramp-up

Phase 6 GA and MOL

Pratt – Slide 3

Concurrent Engineering via Cross-

Functional Core Teams

Service Supply Chain

Product Management

Program Manager

Quality

Engineering

Manufacturing

Pratt – Slide 4

Motivation for Study

Continuous Improvement:

• general perception that NPD takes too long

• lack of proper management of resources

• desire to establish performance benchmarks

Pratt – Slide 5

NPD Data Collected and Analyzed

Schedule Data

• Recorded Dates: Start, Gate 1, Gate 2, Gate 3, Gate 4 (LA) & Gate 5 (GA)

• Program Plans: estimated time to LA, estimated time to GA

• Calculated: total time, time per phase, actual vs. plan

Cost Data

• Timecard System: actual total effort (man-hours)

• Program Plans: estimated total effort

• Calculated: effort by function, effort by phase, remaining effort, actual vs. plan

Pratt – Slide 6

Tools Used in the Study

JMP – Statistical Discovery Software Analyze-Distribution Platform: • Calculations: Quantiles, Moments • Plots: Histogram, Quantile Box, Normal Quantile • Fit Distribution: Normal, Beta, Goodness of Fit Tests

Fit Y by X Platform: Bivariate Analysis • Fit Line, Fit Polynomial, Summary of Fit

Fit Y by X Platform: One-way Analysis • Calculations: Means and Standard Deviations • Plots: Mean Diamonds • Analysis of Variance (ANOVA)

Fit Model Platform • Calculations: Standard Least Squares, Summary of Fit, Effect Tests • Plots: Actual by Predicted, Effect Leverage, Residual by Predicted • Two-way ANOVA with interactions

Microsoft Excel and PowerPoint

Pratt – Slide 7

External Benchmarking Data – PDMA

Best Practices Research

Provides industry-average NPD Cycle-Time based on complexity of programs:

• new-to-the world • new-to-the firm • next-gen improvements • incremental improvements

Pratt – Slide 8

Source: Griffin A., “Product development cycle time for business-to-business products,” Industrial Marketing Management 2002;31: 291-304 *

Average NPD Cycle-Time Benchmarking

Pratt – Slide 9

Planned vs. Actual NPD Cycle-Time

Pratt – Slide 10

ANOVA to Identify Significant Factors

for Schedule Data

Pratt – Slide 11

ANOVA to Identify Significant Factors

for Effort Data

Pratt – Slide 12

Categorizing NPD Programs into 9

Buckets

Product Types:

Modules (power modules, shelves, controllers) Indoor Systems (rack-based systems, racks, distribution) OSP (outside plant power/battery systems)

Program Complexities:

A – major program requiring advanced development B – completely new, but no advanced development C – new with incremental development

Pratt – Slide 13

NPD Summary Data –

Schedule (Gate 2 to LA)

Pratt – Slide 14

NPD Summary Data –

Schedule (Gate 2 to GA)

Pratt – Slide 15

NPD Summary Data –

Effort (Total Development Hours)

Pratt – Slide 16

NPD Schedule & Effort Continuous

Probability Distributions

Pratt – Slide 17

Additional Analyses

• plan vs. actual – schedule and effort

• schedule and effort variance by phase

• time spent per phase

• total effort by function and phase

• default team size and composition

Pratt – Slide 18

Applications

Better Estimating

Pratt – Slide 19

More Accurate Gate 2 Point Estimates

of Schedule and Effort

Previous estimates were:

- subject to negotiation

- overly optimistic and aggressive

- “rose-colored” recollections of past

Pratt – Slide 20

Alpha’s Historical Schedule

Estimation Accuracy

Median MRE = 44%

Pratt – Slide 21

NPD Schedule Point-Estimation via

Historical Mean Values

Median MRE = 19%

Pratt – Slide 22

Basic Schedule Equation Using Historical Mean Effort Values

Median MRE = 21%

Pratt – Slide 23

Informal Comparisons Equation Using

NPD Mean Values

Median MRE = 20%

Pratt – Slide 24

Mean Magnitude of Relative Error –

Schedule Estimates

Pratt – Slide 25

Alpha’s Historical Effort

Estimation Accuracy

Median MRE = 36%

Pratt – Slide 26

NPD Effort Point-Estimation via

Historical Mean Values

Median MRE = 18%

Pratt – Slide 27

Mean Magnitude of Relative Error –

Effort Estimates

Pratt – Slide 28

Determine the “Cone of Uncertainty”

Pratt – Slide 29

Cone of Uncertainty for NPD Project

Estimates

Pratt – Slide 30

Cone of Uncertainty for NPD Project

Schedule Data

Pratt – Slide 31

Cone of Uncertainty Applied to the 9

NPD Project Buckets

Pratt – Slide 32

Schedule Cone of Uncertainty Quantified by Program Phase

Pratt – Slide 33

Effort Cone of Uncertainty Quantified

by Program Phase

Pratt – Slide 34

Effort Estimate Ranges by Program

Phase

Pratt – Slide 35

Applications

Top-Down Project Planning

Pratt – Slide 36

Creation of Top-Down Program

Schedules

Pratt – Slide 37

NPD – Time Spent per Phase

Pratt – Slide 38

How Much Time Should Be Spent on

Planning?

Pratt – Slide 39

Applications

Performance Goal Setting

Pratt – Slide 40

SMART Performance Improvement

Goals

• fact-based and statistically valid

• objective and quantifiable

• deterministic at the start of a program

• consistent across all types of programs

• an enabler of continuous improvement

• unambiguous and easy to calculate

An ideal performance improvement measure is:

Pratt – Slide 41

RSD: a Normalized Measure of

Dispersion

Pratt – Slide 42

Program Scoring: RSD as Performance

Unit-of-Measure

Pratt – Slide 43

2013 NPD Cycle-Time Performance

Pratt – Slide 44

Earned-Value Milestones to Score

WIP NPD Programs

Pratt – Slide 45

Applications

Better Project Selection

Pratt – Slide 46

Deterministic Business Case Analysis

“Figures of Merit”

Pratt – Slide 47

Probabilistic Business Case Analysis

Beta Distribution

Pratt – Slide 48

Monte Carlo Simulation of

Cash Flows ($000)

Trial #1 Trial #2 Trial #3 Trial #4 Trial #5000

NPV $556 $461 $681 $447 $621

NRE 562 560 597 530 664

GM$ year1

348 274 241 220 274

GM$ year2

604 653 407 579 635

GM$ year3

451 651 644 447 558

Pratt – Slide 49

Probability-Based Figures of Merit

Pratt – Slide 50

Applications

Resource Management

Pratt – Slide 51

Default Team Size and Composition

Effort divided by Schedule represents number of individuals

Pratt – Slide 52

Roadmap and Budget Planning

Pratt – Slide 53

Program Resource Management

Engineering Pokémon board

• visual resource management tool

• 9 program type/complexity buckets

• default team size/composition

Pratt – Slide 54

NPD Programs are Represented by a

Series of Magnetic Trays

Pratt – Slide 55

Program Trays Incorporate Default

Team Size/Composition

Pratt – Slide 56

NPD Team Members are Represented

by a Series of Cards

Pratt – Slide 57

Engineering Pokémon – Additional

Pieces

Program Status Indicators

Bonus Targets Add-on Resource Trays “Ghost” Cards

Important Date Indicators

Clear Plastic Overlays

Pratt – Slide 58

The Board in Action

Pratt – Slide 59

Resulting Improvements

Pratt – Slide 60

Improved New Product Development

Better planning and management of resources…

• more stability and focus / less chaos

• increased efficiency

• enhanced organizational understanding

• faster NPD cycle-time

Pratt – Slide 61

In Conclusion:

It is never too late to start recording data!

- even simple data such as key dates and time spent on activities can facilitate powerful analyses

- capture data in real-time; don’t rely on memory or post mortem activities

Pratt – Slide 62

Questions???

Pratt – Slide 63

Thank You!

STEVEN PRATT, MEng., PE

Director of Engineering

www.alpha.ca

Pratt – Slide 64