eindhoven university of technology master npd portfolio ... · npd portfolio management: the...

Eindhoven University of Technology

MASTER

NPD portfolio managementthe development of a reference model for Kalypso

De Mulder, M.L.W.

Award date:2011

Link to publication

DisclaimerThis document contains a student thesis (bachelor's or master's), as authored by a student at Eindhoven University of Technology. Studenttheses are made available in the TU/e repository upon obtaining the required degree. The grade received is not published on the documentas presented in the repository. The required complexity or quality of research of student theses may vary by program, and the requiredminimum study period may vary in duration.

General rightsCopyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright ownersand it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.

• Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain

https://research.tue.nl/en/studentthesis/npd-portfolio-management(936e491c-63f3-4479-8a11-0bbb5838ab11).html

Eindhoven, January 2011

BSc Mechanical Engineering

Student identity number 0597664

In fulfilment of the requirements for the degree of

Master of Science

in Innovation Management

Eindhoven University of Technology

First Supervisor: Prof. dr. F. Langerak, TU/e, IE & IS

Second Supervisor: dr. ir. I. Reymen, TU/e, IE & IS

Kalypso

First Supervisor: Laurens Broekhof

NPD Portfolio Management

The Development of a Reference Model

for Kalypso

by

Maarten De Mulder

NPD Portfolio Management: The Development of a Reference Model For Kalypso

M.L.W. De Mulder I

Preface This master thesis is the result of my graduation project conducted at Kalypso. The

project is the conclusion of the master program Innovation Management at the

Eindhoven University of Technology (TU/e). The project was supervised by the

department of Industrial Engineering & Innovation Sciences (IE&IS).

For me, this journey has come to an end. I look back with great excitement on my

master thesis project and, in particular, the period spent at Kalypso. It has been a

period of great personal development both academically and professionally. None of

this would have been possible without the help of so many people.

First of all, I would like to thank my university supervisors, Prof. dr. Fred Langerak

and dr. Ir. Isabelle Reymen for their critique and support. Especially the guidance of

Fred Langerak has been very helpful throughout the project and the preliminary

literature study.

I am also very grateful to Laurens Broekhof, my supervisor at Kalypso, for his

guidance and fast response to my many questions. In addition, I want to thank Mick

Broekhof, he has been a very insightful source of knowledge and experience. From

the TU Delft I would like to thank Ir. Linda Kester for her very thorough critique and

comments.

Finally, I am grateful to everybody who supported me – intellectually, financially, and

mentally – during my graduation project and the rest of my study. These people

include my family, who had so much patience with me, my girlfriend and my great

friends, who were always there for distraction and memorable moments.

Maarten De Mulder

Amsterdam, January 2011


M.L.W. De Mulder II

Management summary

Introduction Developing the “right” new products is critical to a firm’s success and is often cited as

the key to a sustainable competitive advantage (Chao and Kavadias, 2007). New

Product Development (NPD) portfolio management enables organizations to decide

which projects to pursue and which to abandon. There are currently various NPD

portfolio management methods, and each method has its own advantages and

disadvantages. Consequently, none of these methods are suited to be used as the

single and only method in an organization.

Goal Kalypso developed an NPD portfolio management model which is specifically

tailored to one of their clients. The model is designed around client specific metrics

and criteria which makes it difficult to implement the model in other organizations.

The goal of this master thesis is:

To develop a reference NPD portfolio management model that enables users to make

the best possible NPD portfolio decisions (1). This reference model should be

scientifically substantiated and practically applicable (2). The NPD portfolio

reference framework should enable Kalypso to implement NPD portfolio management

solutions at any client (3).

NPD portfolio management assessment tool Based on scientific literature, a list of criteria for NPD portfolio management is

composed (table 2.9, page 23). These criteria are subdivided into seven categories.

The list of criteria forms the basis of the assessment tool. An NPD portfolio

management method receives a score of 1 (poor) to 7 (excellent) for each criterion.

The overall score is calculated by the average of each criterion score. The assessment

tool is used to assess various NPD portfolio management methods e.g., financial

methods, scoring models and strategic methods. None of these methods received a

score high enough to be considered the single and only NPD portfolio management

method in an organization.


M.L.W. De Mulder III

Table I Average scores various NPD portfolio management models Assessment score Financial methods 3,09 Probabilistic financial methods 3,48 Strategic Methods 3,98 Checklist and Scoring methods 3,73 Mapping approaches or bubble diagrams 3,50 Poor 1 2 3 4 5 6 7 Excellent

The Original Kalypso Model In the original Kalypso model, projects are assessed individually on 3 value drivers;

strategic fit, financial and risk. Strategic fit, risk and, in the first two stages of the

development process financial, are assessed with qualitative criteria. In later stages

the financial assessment is based on a quantitative metric. Based on these 3 variables

a single score is calculated. This score enables users to rank and prioritize projects.

The original Kalypso model is assessed with the NPD portfolio management tool;

table II shows the results of this assessment.

Table II Average scores Kalypso model per category

1. General Criteria 5.71

2. Usability Criteria 5.17

3. Process Criteria 4.71

4. Balance Criteria 3.75

6. Financial Criteria 3.11

5. Risk Criteria 3.00

7. Strategic Criteria 2.75

Overall score 4.03 Poor 1 2 3 4 5 6 7 Excellent

The Kalypso model has a high score on several criteria such as: general, usability and

process criteria. E.g., the Kalypso model uses multiple types of methods for the

valuation of projects. However, the assessment of the various value drivers is flawed.

• Due to the lack of a clear strategy accompanied by well-defined goals and

objectives, it is very difficult to determine the strategic fit of individual

projects. Because of the absence of strategic buckets it is very problematic to

implement a strategy top-down.

• The quantitative financial metric is not suited for NPD portfolio management

decisions.


M.L.W. De Mulder IV

• The risk assessment is not sufficient; projects are assessed on a too limited

number of risks. The risks that are reviewed are not assessed on the three

dimensions of risk.

• The value of the charts produced by the original Kalypso model is limited due

to the lack of a clear definition of what balance is.

The Kalypso model v2 The assessment of the original Kalypso model provides the starting point for the

redesign. The basis of the model remains the same, the Kalypso model v2 uses the

same three variables (risk, strategic fit and financial) as the original model. However,

the determination and assessment of these variables is done in a more detailed

manner. Risks are now assessed on three dimensions (occurrence, impact and the

ability to influence the outcome). This assessment is based on the Risk Diagnosing

Methodology (RDM) and the risks used are based on the risk reference framework

and organization specific risks.

A strategy should be able to answer the questions proposed by Markides (1999):

1. Who should the organization target as customers?

2. What products or services should the organization offer the targeted

customers?

3. How, with what core competences and processes?

4. Why, what is the value proposition to the customers and its competitive

advantage?

5. What are the major threats in the environment?

Based on these questions, various strategic criteria can be formulated. In addition, the

Kalypso model v2 introduces strategic buckets. A strategic bucket is a collection of

NPD projects which are aligned with a particular innovation strategy or strategic

objective (Chao and Kavadias, 2007).

In the first two stages of the development process the financial variable is assessed

with qualitative criteria. In later stages the financial variable is based on the Expected


M.L.W. De Mulder V

Commercial Value (ECV). This metric is able to incorporate probabilities,

development costs, launch costs and Net Present Value.

A project receives a score on each variable (financial, risk and strategic fit),

depending on the development stage of the project, a final score can be calculated.

The weight of each variable varies per development stage. The Kalypso model v2

provides a framework for determining the weight of each variable depending on the

development stage. The final NPD portfolio score is determined with:

!"# !"#$%"&'" !"#$% = !"!! ∗ !! + !" ∗ !! + !"! ∗ !! Where: ai = weight of Strategic Fit Score (SFS) 1 < ai <100 bi = weight of Risk Score (RS) 1 < bi <100 ci = weight of Financial Score (FS) 1 < ci <100 i = stage project is in ∈ (0,1,..,4)

The Kalypso model v2 is also assessed with the NPD portfolio assessment tool. The

results of the assessment are shown in table III.

Table III Average scores per category Original Kalypso

model Kalypso model v2

General Criteria 5,7 6,4 Process Criteria 4,7 5,4 Strategic Criteria 2,8 5,9 Financial Criteria 3,1 6,1 Balance Criteria 3,8 5,0 Risk Criteria 3,0 6,8 Usability Criteria 5,2 5,6

Overall score 4,0 5,9 Poor 1 2 3 4 5 6 7 Excellent

The Kalypso model v2 is a significant improvement over the original model. In all

categories the Kalypso model receives a higher score. Consequently, the Kalypso

model v2 enables users to make the best possible NPD portfolio decisions.

The Kalypso model v2 requires significant commitment from its users. Because many

people are involved in the process, organizations could be hesitant to implement the

Kalypso model v2. In addition, the Kalypso model v2 requires a significant amount of

data, this could limit the usability of the model.


M.L.W. De Mulder VI

Scientific Contribution This research contributes to the NPD portfolio management body of knowledge by

developing a model that combines the NPD portfolio management process developed

by Archer and Ghasemzadeh (1999), research conducted by Cooper et al., (1990,

1993, 1995, 1998, 1999, 2001, 2002, 2006, 2008) and the Risk Diagnosing

Methodology by Keizer et al. (2002). In addition an NPD portfolio management

method assessment tool is developed. This tool is an effective way to assess NPD

portfolio methods.

Limitations There are some limitations to this research. First, the Kalypso model v2 is not tested

in practice, it is a theoretical model. Second, the experts consulted for the

development and assessment of the Kalypso model v2 have a consulting and science

background. No practitioners were involved in the development of the model.


M.L.W. De Mulder VII

Table of Contents

Preface .................................................................................................................... I

Management summary .......................................................................................... II Introduction ......................................................................................................................................................... II Goal .......................................................................................................................................................................... II NPD portfolio management assessment tool ......................................................................................... II The Original Kalypso Model ........................................................................................................................ III The Kalypso model v2 ..................................................................................................................................... IV Scientific Contribution .................................................................................................................................... VI Limitations ........................................................................................................................................................... VI Chapter 1: Introduction ....................................................................................... 1 1.1 Problem description ............................................................................................................................. 1 1.2 Goal of Master Thesis ........................................................................................................................... 2 1.3 Scope ............................................................................................................................................................ 3 1.4 Approach .................................................................................................................................................... 3 1.5 Outline ......................................................................................................................................................... 4 Chapter 2: NPD Portfolio Management ............................................................... 6 2.1 What is NPD Portfolio Management? ............................................................................................ 6 2.2 NPD Portfolio Process .......................................................................................................................... 8 2.3 NPD portfolio management goals, propositions and criteria ........................................... 12 2.3.1 Goal One: Maximize the value of the portfolio .................................................................... 12 2.3.2 Goal Two: The portfolio must be aligned with the strategy .......................................... 15 2.3.3 Goal Three: The portfolio must be balanced ........................................................................ 17 2.3.4 Risk ......................................................................................................................................................... 20 2.3.5 Usage of NPD Portfolio Management Methods .................................................................. 21 2.3.6 Conclusion: Many criteria ............................................................................................................ 23

2.4 Portfolio methods ................................................................................................................................ 25 2.4.1 Financial methods ........................................................................................................................... 26 2.4.2 Strategic methods ............................................................................................................................ 27 2.4.3 Checklists & Scoring models ........................................................................................................ 28 2.4.4 Bubble diagrams .............................................................................................................................. 29 2.4.5 Conclusion: Which Method? ........................................................................................................ 30

Chapter 3: Description the Original Kalypso model ............................................ 33 3.1 Goal Kalypso model ............................................................................................................................. 33 3.2 Process ...................................................................................................................................................... 35 3.2.1 Input ...................................................................................................................................................... 36 3.2.2 Processing & Output ....................................................................................................................... 42 3.2.3 Charts and Diagrams ..................................................................................................................... 44

3.3 Usability ................................................................................................................................................... 45 3.4 Conclusion ............................................................................................................................................... 47 Chapter 4: Evaluation of the Original Kalypso Model ......................................... 48 4.1.1 General Criteria ................................................................................................................................ 48 4.1.2 Process criteria ................................................................................................................................. 50 4.1.3 Strategic criteria .............................................................................................................................. 51 4.1.4 Financial Criteria ............................................................................................................................. 53 4.1.5 Balance criteria ................................................................................................................................ 54 4.1.6 Risk Criteria ........................................................................................................................................ 54 4.1.7 Usability Criteria .............................................................................................................................. 55


M.L.W. De Mulder VIII

4.2 Conclusion ............................................................................................................................................... 56 Chapter 5: Improving the Original Kalypso model .............................................. 58 5.1.1 Strategic criteria .............................................................................................................................. 58 5.1.2 Risk criteria ........................................................................................................................................ 64 5.1.3 Financial criteria ............................................................................................................................. 68 5.1.4 Balance criteria ................................................................................................................................ 72 5.1.5 Process criteria ................................................................................................................................. 73 5.1.6 Usability criteria .............................................................................................................................. 76 5.1.7 General criteria ................................................................................................................................. 78

5.2 Evaluation of the Kalypso Model v2 ............................................................................................. 80 5.2.1 General Criteria ................................................................................................................................ 81 5.2.2 Process Criteria ................................................................................................................................. 82 5.2.3 Strategic Criteria ............................................................................................................................. 83 5.2.4 Financial Criteria ............................................................................................................................. 84 5.2.5 Balance Criteria ................................................................................................................................ 85 5.2.6 Risk Criteria ........................................................................................................................................ 86 5.2.7 Usability Criteria .............................................................................................................................. 86

5.3 Conclusion ............................................................................................................................................... 87 Chapter 6: Conclusion ....................................................................................... 88 6.1 Scientific Implications ........................................................................................................................ 88 6.2 Scientific Limitations .......................................................................................................................... 88 6.3 Practical Implications ......................................................................................................................... 89 6.4 Practical Limitations ........................................................................................................................... 91 6.5 Recommendation for further research ....................................................................................... 92 References ........................................................................................................... 94

Appendices .......................................................................................................... 98 Appendix 1: Description of the different stages of the Stage Gate® process ........................ 98 Appendix 2: Description of different levels of PML maturity .................................................... 100 Appendix 3: Risk Reference tool including risk issues taken from literature and issues resulting from case studies ....................................................................................................................... 101 Appendix 4: Assessment of the five different NPD portfolio management methods ...... 105 Appendix 5: Overview of Kalypso model performance per criteria ranked by score ..... 108 Appendix 6: Decision rules for classification into risk classes .................................................. 110

List of Figures and Tables

Preface .................................................................................................................... I

Management summary .......................................................................................... II Table I Average scores various NPD portfolio management models ........................................ III Table II Average scores Kalypso model per category ...................................................................... III Table III Average scores per category ...................................................................................................... V Chapter 1: Introduction ....................................................................................... 1 Figure 1.1 Design master thesis project ................................................................................................... 3 Figure 1.2 Document outline ........................................................................................................................ 5 Chapter 2: NPD Portfolio Management ............................................................... 6 Table 2.1 Difficulties in NPD portfolio management .......................................................................... 7 Figure 2.1 NPD portfolio process ................................................................................................................ 8


M.L.W. De Mulder IX

Table 2.2 Overview of NPD portfolio management process criteria ......................................... 12 Table 2.3 Requirements are for an ideal financial metric. ............................................................. 13 Table 2.4 An overview of criteria for achieving strategic alignment ........................................ 17 Table 2.5 Example of parameters used for X or Y-‐axis .................................................................... 18 Table 2.6 Criteria for NPD portfolio management for reaching balance in the portfolio . 20 Table 2.7 Overview of risk criteria ........................................................................................................... 21 Table 2.8 Overview of NPD portfolio management model usability criteria ......................... 23 Table 2.9 Overview of criteria for NPD portfolio management ................................................... 23 Exhibit 2.1 Determination of expected commercial value of project ........................................ 27 Table 2.10 Criteria used to Rank Projects ............................................................................................. 28 Figure 2.3 Risk-‐Reward Bubble diagram ............................................................................................... 30 Table 2.11 overview of different NPD portfolio methods with their assessment score, advantages and disadvantages .................................................................................................................. 31 Chapter 3: Description the Original Kalypso model ............................................ 33 Exhibit 3.1 implemented Stage Gate process ....................................................................................... 34 Exhibit 3.2 Kalypso model process diagram ........................................................................................ 35 Table 3.1 Overview of value categories ................................................................................................. 36 Table 3.2 Overview of value drivers ........................................................................................................ 37 Table 3.3 Qualitative criteria for assessing the probability of financial success ................. 38 Table 3.4 Criteria for determining level of strategic fit ................................................................... 39 Exhibit 3.3 Criteria for determining project risk level .................................................................... 40 Table 3.5 Distribution of points per value driver .............................................................................. 43 Table 3.6 Weighting of each value driver for each phase ............................................................... 43 Table 3.7 List of charts and diagrams the Kalypso Europe model produces ......................... 44 Exhibit 3.4 Responsibilities in the Kalypso Model ............................................................................ 45 Table 3.8 Role and Responsibilities ......................................................................................................... 46 Chapter 4: Evaluation of the Original Kalypso Model ......................................... 48 Table 4.1 Kalypso model general criteria evaluation ....................................................................... 48 Table 4.2 Kalypso model process criteria evaluation ...................................................................... 50 Table 4.3 Kalypso model strategic criteria evaluation .................................................................... 51 Table 4.4 Criteria for determining level of strategic fit ................................................................... 52 Table 4.5 Kalypso model financial criteria evaluation .................................................................... 53 Table 4.6 Kalypso model balance criteria evaluation ...................................................................... 54 Table 4.7 Kalypso model risk criteria assessment ............................................................................ 54 Table 4.8 Kalypso model usability criteria evaluation .................................................................... 55 Table 4.9 Average scores Kalypso model per category ................................................................... 56 Chapter 5: Improving the Original Kalypso model .............................................. 58 Figure 5.1 Stage Gate® process by Cooper ............................................................................................ 58 Figure 5.2 Strategy development .............................................................................................................. 59 Table 5.1 Various consideration for the development of strategic buckets ........................... 60 Figure 5.3 Example of cascading down strategic Buckets ............................................................. 61 Exhibit 5.1 Process of strategic buckets ................................................................................................. 62 Figure 5.4 Strategy process ......................................................................................................................... 63 Figure 5.5 Outline of risk diagnosing methodology .......................................................................... 64 Exhibit 5.2 Example of risk questionnaire (for two risks) ............................................................. 66 Exhibit 5.3 Example of a risk profile (for two risks) ........................................................................ 66 Figure 5.6 Risk process ................................................................................................................................. 67 Exhibit 5.4 Determination of expected commercial value of project ........................................ 68 Table 5.2 Example of distribution of risk .............................................................................................. 70 Figure 5.7 Financial process ........................................................................................................................ 71 Figure 5.7 Balance process .......................................................................................................................... 72 Figure 5.8 Kalypso model v2 process diagram ................................................................................... 74


M.L.W. De Mulder X

Figure 5.9 Responsibilities in the Kalypso v2 model ....................................................................... 77 Table 5.2 Role and Responsibilities Kalypso model v2 ................................................................... 78 Exhibit 5.5 Weighting of the various variables depending on development phase ............ 79 Table 5.3 KE model general criteria evaluation ................................................................................. 81 Table 5.4 KE model process criteria evaluation ................................................................................. 82 Table 5.5 Kalypso model v2 strategic criteria evaluation .............................................................. 83 Table 5.6 Kalypso model v2 Financial criteria evaluation ............................................................. 84 Table 5.7 Kalypso model v2 Balance criteria evaluation ................................................................ 85 Table 5.8 Kalypso model v2 Risk criteria evaluation ....................................................................... 86 Table 5.9 Kalypso model v2 Usability criteria evaluation .............................................................. 86 Table 5.10 Average scores per category ................................................................................................ 87 Chapter 6: Conclusion ....................................................................................... 88 Table 6.1 Scientific Limitations .................................................................................................................. 89 Figure 6.1 OPMP3tm maturity levels ........................................................................................................ 90 Table 6.2 Practical Limitations .................................................................................................................. 92


M.L.W. De Mulder 1

Chapter 1: Introduction 1.1 Problem description New products are vital to the success and future prosperity of modern organizations

(Cooper et al., 1998). Organizations that fail to excel at developing new products will

invariably disappear or be gobbled up by the winners (Cooper et al., 1998). A vital

question in new product development is: How should organizations invest their R&D

and new product resources most effectively? The goal of New Product Development

(NPD) portfolio management is to answer this question. An NPD portfolio is a group

of NPD projects carried out under the sponsorship of a particular organization. It

often occurs that an organization can’t develop all its projects due to limited R&D

resources; they have to choose between projects. NPD portfolio management refers to

the selection and support of project investments or program investments that are

guided by the organization’s strategic plan and available resources (in PMBOK.

2000). Hence, NPD portfolio management is about resource allocation in the firm.

Which projects, out of often many opportunities should an organization fund, which

project should receive top priority? Cooper et al. (1993, 1998) found that failing to

conduct NPD portfolio management could lead to:

1. An overloaded NPD pipeline; there are too many project for the resource

available.

2. Consequently, the time to market of projects increases significantly because

resources are spread too thin.

3. When resources are spread too thin the quality of execution starts to suffer.

4. There will be a significant increase in failure rates.

5. Failing to conduct NPD portfolio management will result in a lack of “stellar

new product winners” because good projects are starved for resources.

6. Without NPD portfolio management the wrong projects are often selected

based on the erroneous criteria. Instead of decisions based on facts and

objective criteria decisions are based on politics, opinioneering and emotion.

7. An NPD portfolio composition with projects that are not aligned with the

corporate strategy.


M.L.W. De Mulder 2

Consequently, a lot of research has been directed toward NPD portfolio management.

However, surveys have shown that the solutions provided by academic research have

found little use in practice (Shane and Ulrich, 2004). Scientific models often require a

lot of data: e.g., information on the financial results, probabilities of completion and

success for all projects. Practitioners often don’t have the required data and when

they do get results, practitioners doubt them due to lack of transparencies and

robustness (Loch et al., 2002). Managers perceive solutions provided by science to be

too difficult to understand and use (Cooper et al., 1999). Practitioners often use

multidimensional decision making tools or ranking methods. A drawback of these

tools is that they rely on an ad hoc list of dimensions and their decision makers often

manipulate the methods to generate desired outcomes instead of using them as true

decision support tools (Chao and Kavadias, 2007 and Loch and Kavadias, 2002).

These tools do help management “think trough” a problem but they lack theoretical or

empirical basis (Loch and Kavadias, 2002). Senior managers, R&D managers, and

project managers are forced to make resource allocation decisions based primarily on

intuition or heuristic rules (Chao and Kavadias, 2007). Hence there is the need for an

NPD management model that is both based on scientific research and is practically

applicable by organizations. Currently, there is not a single NPD portfolio method

which is able to fulfill these needs. This brings us to the goal of this master thesis

research project.

1.2 Goal of Master Thesis Kalypso developed an NPD portfolio management model that tried to overcome the

difficulties mentioned above. They developed an NPD portfolio management model

that is specifically tailored for one of their clients. The model is designed around

client specific metrics and criteria, which makes it difficult to implement the model in

other organizations. The goal of this master thesis is:

To develop a reference NPD portfolio management model that enables users to make

the best possible NPD portfolio decisions (1). This reference model should be

scientifically substantiated and practically applicable (2). The NPD portfolio

reference framework should enable Kalypso to implement NPD portfolio management

solutions at any client (3).


M.L.W. De Mulder 3

With supporting sub questions:

1. What is NPD portfolio management?

2. Which requirements are needed for an NPD portfolio management tool?

3. Which NPD portfolio management methods are there?

4. What are the advantages and disadvantages of these NPD portfolio

management methods?

5. How does the Kalypso NPD portfolio management model work?

6. What are the strengths and weaknesses of the Kalypso NPD portfolio

management model?

7. How can the Kalypso NPD portfolio management model be improved in order

to comply with the answer to question 2?

1.3 Scope NPD portfolio management can be conducted at various levels, from entire NPD

programs to specific projects. This research is limited to the NPD portfolio

management of individual projects.

1.4 Approach This master thesis project will be relevant to both scientists and practitioners by

contributing to both streams of knowledge. The design for this master thesis project is

shown in figure 1.1.

Figure 1.1 Design master thesis project

Literature

Original model developed by Kalypso

List Criteria for NPD Portfolio Management

Science Practice

!"#$%#&'

New model

Experts

!"#$%#&'

()*+%**),-

Knowledge of NPD Portfolio Management

('"'$,.


M.L.W. De Mulder 4

Currently there is a large body of knowledge about NPD portfolio management. Both

scientist and practitioners contribute to this body of knowledge. Scientists contribute

by conducting research and publish their findings. Practitioners use and contribute to

this knowledge by applying it in real business environments. In this master thesis a

list of criteria for NPD portfolio management is composed. This list of criteria is the

result of a comprehensive review of the scientific literature concerning NPD portfolio

management. Kalypso developed an NPD portfolio management model (hereafter:

original model) for one of its clients based on their experience and knowledge of NPD

portfolio management. The list of criteria will be used to evaluate the original

Kalypso model and is conducted by the author and experts. This evaluation will give

insight into the strong and weak aspects of the original model. The new model is built

on the strong points of the original model while overcoming the weak. In addition, the

new model is based on scientific literature as well as on contributions by experts. In

the end, the new model will be evaluated based on the same criteria as the original

model. By comparing both evaluations it will become clear if the new model is an

improvement over the original model. If this is the case, the new model will

contribute to the existing body of knowledge by adding new knowledge about NPD

portfolio management models.

1.5 Outline The goal and supporting sub questions form the outline of this master thesis report. A

graphical depiction of the outline is shown in figure 1.2. In chapter two, the subject

of NPD portfolio management will be researched and answers to what NPD portfolio

management is and why it is so difficult will be sought. In the second part of chapter

two a comprehensive list of criteria for NPD portfolio management, found in the

literature, is composed. In addition, various existing NPD portfolio management

methods are discussed. These methods are assessed on the criteria found in the

previous part of this chapter. This assessment provides a good insight in the strong

and weak aspects of the various NPD portfolio management methods. What will

become clear is that none of these methods is suited to be used as a stand-alone NPD

portfolio management method. In chapter three the original Kalypso model, the

process and various in and outputs will be described. In addition, a running example

is added to clarify numerous aspects of the original Kalypso model. In chapter four

the original Kalypso model is evaluated on the criteria for NPD portfolio management


M.L.W. De Mulder 5

found in chapter two. This assessment is done together with Laurens Broekhof, a

Kalypso employee who was involved in the development of the current model. Each

score is explained and substantiated. Chapter four provides insight in the strong and

weak aspects of the original Kalypso model. These strong and weak aspects provide

an excellent starting point for the redesign in chapter five. This chapter describes in

detail the various improvements in the Kalypso model v2 over the original Kalypso

model. These improvements are validated with experts from Kalypso (Mick

Broekhof) and outside Kalypso (Linda Kester). Chapter five ends with an assessment

of the Kalypso model v2 on the criteria for NPD portfolio management found in

chapter two. This assessment answers the question if the Kalypso model v2 is better

suited for NPD portfolio management than the original Kalypso model and the other

models covered in this research. Chapter six starts with an overall conclusion

followed by a description of the consequences of this research for science and

practice. Chapter six ends with the limitations of this research, recommendations for

further research and usage of the Kalypso model v2.

Figure 1.2 Document outline

Chapter 2

Chapter 3

Chapter 4

Chapter 5

Chapter 6

Literature Review

Description of OriginalModel

Evaluation of OriginalModel

Development of NewModel

Conclusion

-What is NPD portfolio management?-Why is it so difficult?-What are the criteria for NPD Portfolio management?-What are the current NPD Portfolio management methods?-Evacuation of the current NPD portfolio management methods.

-Description of the goal of the original model.-Description of the process.-Description of the various inputs.-Description of the output.-Running example of original model.

-Evaluation of the original model based on the criteria from chapter 2.-Evaluation conducted by author and experts.-Conclusion of evaluation, what are the strong and weak points?

-Redesign of the original model prioritized by the results of chapter 4.-Evaluation of the new model based on the criteria from chapter 2.-Evaluation conducted by author and experts.-Conclusion of evaluation.

Chapter 1

-Statement of goal master thesis project-Statement of sub questions.-Definition of scope master thesis project.-Description of master thesis approach.-Description of master thesis outline.

Introduction, and Statementof Goal Master Thesis Project

-Is the master thesis goal attained?-Consequences for science and practitioners-Recommendations-Limitations-Further research


M.L.W. De Mulder 6

Chapter 2: NPD Portfolio Management

2.1 What is NPD Portfolio Management? A project portfolio is a group of projects carried out under the sponsorship of a

particular organization. It is possible that an organization has multiple projects that

meet all the criteria set, however it often occurs that an organization can’t continue

with all projects due to limited resources, they have to choose between projects. NPD

Portfolio management is the management of different NPD projects and NPD

programs (depending on the size of the NPD portfolio) in an NPD portfolio, it is

about deciding which projects to pursuit, continue, recycle or to abandon. It is very

important for a organization to have the right product development mix, failing to do

so could lead to loss of a organization’s competitive advantage (Chao and Kavadias,

2004, 2007). Such a mix could consist of long and short term, radical and incremental

and low and high-risk projects. Success requires a fundamental trade off: short-term

benefits accrued through incremental improvements versus long term benefits

achieved through radical or new to the world products and services (Tushman and

O’Reilly, 1996). NPD portfolio management enables organizations to make these

decisions. Cooper et al. (1999) define NPD portfolio management as a dynamic

decisions process, whereby a organization’s collection of active new product (and

R&D) projects is constantly updated and revised. In this process, new projects may be

accelerated, terminated, or de-prioritized; and resources are allocated and reallocated

to the active projects.

An NPD portfolio allows firms to operate and implement their innovation strategy

over time. This implies that the NPD portfolio problem entails a large component of

ambiguity and complexity, since the determinants of organizational success and their

interactions are rarely known (Loch and Tapper, 2002). Another difficulty of NPD

portfolio management is resource scarcity, an organization doesn’t have limitless

resources and therefore can’t pursue every opportunity. In many cases (76%)

organizations have too many projects for the resources available (Cooper and Edgett,

2006). This results in resources that are spread too thin over too many projects and

there is not the time or ability to do many of the key project activities proficiently

(Cooper et al., 2002). In multi-project environments, scarce resources render the

resource allocation decision a critical factor for success (Adler et al., 1995). It often


M.L.W. De Mulder 7

occurs that organizations develop multiple projects in closely related markets. When

these projects interact, synergies or incompatibilities can exist in their technical

aspects. Products may substitute or complement each other in different markets.

Interactions between success determinants play a critical role in the resource

allocation decision (Chao and Kavadias, 2007). Outcome uncertainty is another aspect

which makes NPD portfolio management difficult. Management faces uncertainty

along the dimensions of the potential market value and the technical output of any

given project (Chao and Kavadias, 2007, Pich et al., 2002). Decision makers must

allocate resources over time and NPD programs evolve over time. Not all projects

evolve in the same manner over time. It is often difficult to quantify the potential of

promising ideas or to precisely measure the risks involved (Chao and Kavadias,

2007). Table 2.1 summarizes the difficulties of NPD portfolio management.

Table 2.1 Difficulties in NPD portfolio management

• NPD portfolio problem entails a large component of ambiguity and complexity (Loch and Tapper, 2002) • NPD portfolio management is constrained by resource scarcity (Cooper and Edgett, 2006) • NPD portfolio management transcends multi-‐project environments (Adler et al., 1995) • NPD portfolio management deals with outcome uncertainty (Chao and Kavadias, 2007, Pich et al.,

2002) • Projects are interdependent (Chao and Kavadias, 2007, Pich et al., 2002) • Programs and projects often don’t evolve in the same manner (Pich et al., 2002) • Difficult to quantify promising ideas (Chao and Kavadias, 2007)

In remaining part of this chapter we will discuss NPD portfolio management in detail.

There are two important streams of literature. The first one is by Cooper et al. (1998,

1999) and the second one is by Archer and Ghasemzadeh (1999). Cooper et al. (1998,

1999) formulated three goals for NPD portfolio management. Archer and

Ghasemzadeh (1999) developed an NPD portfolio process model and formulated

various propositions for NPD portfolio management. What will become clear is that

the two streams of literature overlap but most of all complement each other. In this

chapter a list of criteria for NPD portfolio management will be developed, this list

will be the basis for the assessment of the Kalypso model in chapter 4. However, first

we will describe the NPD portfolio management process model developed by Archer

and Ghasemzadeh (1999). What will become clear is that goals of Cooper et al. (1998,

1999) and the propositions of Archer and Ghasemzadeh (1999) do not cover all

aspects involved in NPD portfolio management, these missing aspects will be covered


M.L.W. De Mulder 8

in paragraph 2.3.4. The rest of this chapter will provide insight in current NPD

portfolio methods.

2.2 NPD Portfolio Process We will start with the description of the NPD portfolio management process

developed by Archer and Ghasemzadeh (1999). The NPD portfolio management

process consists of various stages which progress from the initial broad strategy

considerations towards the final solution. The NPD portfolio management process is

shown in figure 2.1.

Figure 2.1 NPD portfolio process

Source: Archer and Ghasemzadeh (1999) page 211

The portfolio selection process proposed by Archer and Ghasemzadeh (1999, 2007) is

divided in three different phase; the preprocess phase, process phase and post process

phase. The first phase, preprocess phase, consists out of two parts. The first part is

methodology selection, this is a strategic process that should be done in advance of

any other activities in NPD portfolio selection. Methodology selection needs to be

done once for all time, with minor adjustments from time to time if other

methodology choices appear to be better matches for the tasks at hand before each

cycle of portfolio selection (Archer and Ghasemzadeh, 2007). It is critical that

common measures (e.g., NPV, scoring attributes, valuation of risk etc.) are chosen so

Strategy Development

Proposed Projects

PrescreeningIndividual

ProjectAnalysis

Screening PortfolioSelection

PortfolioAdjustment

ProjectDevelopment

ProjectEvaluation

PortfolioCompletion

MethodologySelection

ProjectDatabase

Guidelines Resource allocation

MainStages

Pre ProcessActivity

Pre ProcessActivity

Post Processstages


M.L.W. De Mulder 9

they can be calculated separately for each project under consideration, allowing an

equitable comparison of the projects (Archer and Ghasemzadeh, 1999). The second

part of the first phase is strategy development. The NPD Portfolio enables an

organization to implement its strategy, to do so, the strategy must be developed before

hand. Strategy can be used to develop a focused objective for an NPD portfolio and

the level of resources needed for its support (Cooper et al., 1998, 1999, Chao and

Kavadias, 2007).

The process phase consists out of 5 (main) stages. The first stage is prescreening. The

goal of this stage is to ensure that any project being considered for the portfolio fits

the strategy. Projects should be classified in advance of the portfolio selection,

according to criteria that can override other considerations (Archer and Ghasemzadeh,

2007).

The second stage of the process phase is the individual project analysis. Each project

is individually assessed on the common measures selected in the methodology

selection stage. The output of this stage is a common set of parameter estimates for

each project. Individual project analysis can be done with a Stage Gate® process

developed by Cooper (1990, 2008). The stage gate process is depicted in figure 2.2.

Figure 2.2 Stage Gate® process with funnel

Source: combination of Docherty (2006) and Cooper (2008) page 215 and 231

Stage 1 Stage 4Stage 3Stage 2 Stage 5Gate3

Gate2

Gate4

Gate5

Stage 0 Gate1

Post launchReview

$

Project i

2

1

3

12

8

14

11

13

11

9

166 5

1110

4

18

19

24

21

i

Discovery Scoping BuildBusiness

Case

Development Testing &Validation

Launch

Idea ScreenSecondScreen

Go ToDevelopment

Go ToTesting

Go To Launch


M.L.W. De Mulder 10

An effective Stage-Gate® system is essential to sound portfolio management (Cooper,

2008). The Stage-Gate® process is a conceptual and operational map for moving new

product projects form idea to launch and beyond. Numerous projects enter the funnel

but their number decreases at every step. A Stage Gate® process is a process with

different stages divided by gates. Each stage has the goal to gather information and

reduce risk of a project. The costs concerning a project increase significantly at each

consecutive stage. Teams are cross-functional and work parallel on activities in each

stage. After each stage, at the gate a go/kill/hold/recycle decision is made along with

an approved action plan for the next stage. These decisions are made based on

deliverables and criteria. Deliverables are visible, based on a standard menu for each

gate and are determined at the output of the previous gate. Every gate has criteria

against which the project is judged, these criteria are divided in must meet criteria and

should meet criteria. Must meet criteria are criteria that a project must meet, if they

are not met the project will be terminated. Should meet criteria are highly desirable

characteristics that are rated in a point-count scheme. Most best practice organizations

have adopted a Stage Gate® system (Griffin, 1997, Cooper, 2008). The different

stages and gates of the Stage Gate® process are described in appendix 1. Each project

should be assessed individually on its (financial) value (Archer and Ghasemzadeh,

1999). Consequently the portfolio decision process encompasses and/or overlaps a

number of decision-making processes within the Stage Gate® process. Information

and criteria used to review a single project at each gate can also be used to make

decision regarding the entire portfolio.

Screening is the next stage, the common set of parameter estimates are examined in

advance of the actual selection process in order to eliminate any projects or

interrelated families of projects that do not meet present criteria (e.g., NPV), except

for those projects that are mandatory or required to support other projects still being

considered (Archer and Ghasemzadeh, 2007).

In the following stage, Portfolio selection, interactions among various projects are

considered, including interdependencies, competition for resources and timing, with

the value of each project determined from a common set of parameters that were

estimated for each project in the previous state (individual project analysis) (Archer

and Ghasemzadeh, 1999). It often occurs that projects are interdependent (e.g. project


M.L.W. De Mulder 11

C can be undertaken when projects A and B are completed since their outputs feed

project C) (Chao and Kavadias, 2007). An NPD portfolio tool should take into

account these interdependencies. Many portfolio selection techniques do not consider

the time-dependent resource requirements of projects (Martino, 1995), and most

implicitly assume that all projects selected will start immediately. This does not fit the

reality of project management, where projects compete for limited resources. Projects

ought to be scheduled in such a way that they use resources as smoothly as possible in

time, and projects should be completed within some planned interval (Archer and

Ghasemzadeh, 1999). Hence an NPD portfolio method should take into account the

time dependent nature of project resource selection.

The last main stage is portfolio adjustment. The end result of this process is a

portfolio that meets the objectives of the organization optimally or near-optimally.

This stage provides an overall view, where the characteristics of projects, critical for

an optimized portfolio (e.g., risk, time-to-complete), can be represented. An important

aspect of this stage is achieving some form of balance (Archer and Ghasemzadeh,

2007).

Cooper and Kleinschmidt (1995) found that a high quality new product development

process is the most important factor for new product success. The NPD portfolio

management process shown in figure 2.2.1 enables decision makers to move logically

towards an integrated consideration of projects most likely to be selected, based on

sound theoretical models (Archer and Ghasemzadeh, 1999). The product development

process consists of many stages (see appendix 1), each time a project leaves one stage

to enter another one, the project should be re-evaluated at the same time as new

projects being considered for selection. Note that the NPD portfolio management

process by Archer and Ghasemzadeh (1999) has a lot of similarities with the Stage

Gate® process by Cooper (1990, 2008). Table 2.2 shows an overview of the criteria

for NPD portfolio management mentioned in this paragraph.


M.L.W. De Mulder 12

Table 2.2 Overview of NPD portfolio management process criteria • The NPD portfolio management model should be divided in various stages (Archer and Ghasemzadeh,

1999) • The NPD portfolio methodology and common measures should be selected before the NPD portfolio

process starts (Archer and Ghasemzadeh, 1999) • The strategy should be defined before the NPD portfolio management process starts (Archer and

Ghasemzadeh, 1999) • Projects should be classified in advance of the before the NPD portfolio selection starts (Archer and

Ghasemzadeh, 1999) • Projects should be assessed individually, based on a Stage Gate® system (Archer and Ghasemzadeh,

1999) • Projects should be individually assesses based on a common set of parameters (Archer and

Ghasemzadeh, 1999, Cooper 1990, 2008) • The NPD portfolio model should consider the interdependencies between projects (Chao and Kavadias,

2007) • The NPD portfolio model should consider the time dependent resource requirements of projects

(Martino, 1995) • The NPD portfolio model should provide an overview of the NPD portfolio where the most important

project characteristics are represented (Archer and Ghasemzadeh, 1999)

2.3 NPD portfolio management goals, propositions and criteria Different authors have formulated various goals and criteria. In the next part we will

discuss the three goals proposed by Cooper et al. (1998, 1999) and various

propositions of Archer and Ghasemzadeh (1999). What will become clear is that both

streams complement and reinforce each other and form an excellent basis for NPD

portfolio management tool development.

2.3.1 Goal One: Maximize the value of the portfolio We will start with the description of the three goals proposed by Cooper et al., (1998,

1999). The first goal Cooper et al. (1998, 1999) formulated is to maximize the NPD

portfolio’s value against one or more objective (such as profitability, strategic,

acceptable risk, etc.). You can achieve this by selecting projects with the highest

value for an organization. The highest value is often determined with financial

metrics. Various financial metrics can be used to determine the value of a project. The

Net Present Value (NPV) method involves discounting expected cash flows at a

discount rate that is set to reflect the perceived riskiness of a project. The Return On

Investment (ROI) is the ratio of money gained (or lost) on an investment relative to

the amount of money invested. In the literature various requirements of financial

metrics are mentioned. Financial metrics should incorporate the cost of development

and launch (Cooper et al., 1998). In order to correctly compare projects one must

look at what the project yields and at what the project costs. Another reason is that

organizations don’t have limitless resources, each critical resources must be optimally


M.L.W. De Mulder 13

used. Consequently, a financial metric should therefore be constrained by a critical

resource (Cooper et al., 1998, 1999).

Most financial metrics disfavor advanced technology projects due to the long-term

payoff and high likelihood of failure (Chao and Kavadias, 2007). The ideal financial

metrics should have no bias towards high-risk projects. The NPD process is a staged

and flexible process (projects can be abandoned, paused, and stages could be done

over again, etc. (Cooper, 1990, 2008)) a financial metrics should take this into

account. These options can add value to a project (Newton et al., 2004). In order to

incorporate these options one must look at the probability of these options occurring

(Pennings and Lint, 1997, Cooper et al., 1998). Sunk costs are not relevant for the

decision process and should therefore not be included into the financial metric

(Cooper et al., 1998).

Remer and Nieto (1995) studied 25 different (financial) project evaluation techniques,

they tested each method by calculating the value of two projects (projects A and B

where project A is more valuable than project B). Remer and Nieto (1995) found that

methods that did not use the Minimum Acceptable Rate of Return (MARR) and Time

Value of Money (TVM) selected the wrong project (project B over A). As mentioned

before there are various financial metrics, these metrics differ in complexity and

applicability. It is important that the financial metric used is understandable plus

usable for practitioners and doesn’t appear to be a black box (Faulkner, 1996, Cooper

et al., 1999, Kester et al., 2009). An overview of the requirements for a financial

metric are shown in table 2.3.

Table 2.3 Requirements are for an ideal financial metric.

• The financial metric should incorporate the cost of development and launch (Cooper et al., 1998) • The financial metric should have no bias towards long term radical projects (Chao and Kavadia, 2007) • The financial metric should capture the flexibility of the development process (Newton et al., 2004) • The financial metric should be constrained by a critical resource (Cooper et al., 1998) • The financial metric should ignore sunk cost (Cooper et al., 1998) • The financial metric should incorporate the probability of success or failure (Cooper et al., 1998,

Pennings and Lint, 1997) • The financial metric should be user friendly and not too complex or appear as a black box (Faulkner,

1996, Cooper et al., 1999, Kester et al., 2009) • The financial metric should use the Minimum Acceptable Rate of Return (MARR) and Time Value of

Money (TVM) (Remer and Nieto, 1995)


M.L.W. De Mulder 14

There are alternatives to quantitative financial metrics. Another method for

maximizing the value of the NPD portfolio are scoring models (Cooper et al. 1998,

1999). Scoring models, can take the weight of different criteria into account when

calculating an overall score. Determining the weighting and scores is a subjective

process that depends as much on personal opinion as hard numbers (Coldrick et al.,

2003). Projects are ranked according to the overall score. Projects can be added or

deleted without affecting the scores of the remaining projects. Scoring models are less

appropriate when there is a high degree of interdependence between projects; when

activities and the results of one project depend on the activities and results of a

different project (Henriksen et al., 1999). The advantage of scoring models is that

they do not place too heavy an emphasis on (quantitative) financial criteria, whose

reliability is often doubtful in the early stages of a project. Another advantage of

scoring models is that they can capture multiple goals. The goal of an NPD portfolio

management method is to assist management in making sound portfolio selection

decisions. This is only possible if the NPD portfolio management methods enable

objective comparison of projects (Archer and Gasemzadeh, 1999), which means that

scoring models should produce score data (interval or ratio scale) (Graziano and

Raulin, 2005).

There are two methods for maximizing the value of the NPD portfolio, so which one

is best? Cooper et al. (1999) found that scoring models result in a portfolio with high

value, profitable, high return projects with solid economic prospects. The use of only

financial metrics for selecting NPD projects results in a portfolio with poor value

projects and a portfolio with too many projects for the resources available (Cooper et

al., 1999). In order to maximize the value of the NPD portfolio Cooper et al. (1999)

recommend that both methods are used. Scoring models because of their good

performance and financial metrics because they are dominant and widely used in

business. What applies for both NPD portfolio methods is that the criteria’s for

valuating each project should be determined in advance (Archer and Gasemzadeh,

1999). In addition, competing projects should be assessed on common criteria and

metrics (Archer and Gasemzadeh, 1999).


M.L.W. De Mulder 15

2.3.2 Goal Two: The portfolio must be aligned with the strategy The second goal formulated by Cooper et al. (1998) is that the portfolio should be

aligned with the organization’s strategy. Cooper and Kleinschmidt (1995) found that

the new product strategy is the second most important success factor for an NPD

program. The essence of strategy is selecting one position that an organization can

claim as its own (Markides, 1999). According to Markides (1999) a strategic positions

is the sum of an organization’s answers to the following questions:



customers?



advantage?


The most common source of strategic failure is the inability to make clear and explicit

choices on these five dimensions (who, what, how, why, threats) (Markides, 1999). In

order to assess the alignment of the projects in a portfolio one must translate the

business strategy into a technical strategy (Loch and Tapper, 2002). This means that

the technology strategy must specify what technologies to master, and how these

technologies relate to the business: what products and segments these technologies

address, what they contribute to competitive advantage and to hedging against

environmental threats (Loch and Tapper, 2002). According to Boyer and McDermott

(1999) strategic alignment is the level of agreement within an organization regarding

the relative importance of cost, quality, delivery and flexibility to the organization’s

operational goals as well as the relationships between competitive priorities and

policies. Strategic alignment can be classified under two categories, internal fit and

external fit. Internal fit is the consistency between the task and policies and practices

(Skinner, 1974). External fit is the alignment of the operational strategy with the

business and corporate strategies and to examine the alignment-performance

relationship (Joshi et al., 2003).


M.L.W. De Mulder 16

Kaplan and Norton (1996) developed the Balanced Scorecard (BSC) method. The

first step of the BSC process is to translate the vision. In this step the vision of the

organization is translated into agreed upon integrated set of objectives measures that

describe the long-term drivers for success. The vision and strategy is translated into

four perspectives; a financial perspective, a customer perspective, an internal business

process perspective and a learning and growth perspective. “To succeed financially,

how should we appear to our share holders?” is the main question for the financial

perspective. For the customer perspective a organization must ask itself “To achieve

our vision, how should we appear to out customer?” The financial and customer

perspective form the basis of the internal business process perspective. Here the

question “To satisfy our shareholders and customers, what business process must we

excel at?” should be asked. For the learning and growth perspective the question is

“To achieve our vision, how will we sustain our ability to sustain and improve?” By

asking these questions objectives, measures, targets and initiatives will be formulated

for each perspective.

NPD portfolio management should be a top-down and bottom-up approach (Cooper et

al., 1998, Archer and Ghasemzadeh, 1999). The top down part is described above, top

management compose a vision and strategy and sets aside resources for achieving it.

Where money is spend mirrors the business’s strategy. The bottom-up approach

begins with a review of specific projects and focuses on selecting the best (Cooper et

al., 1998). Strategic criteria are built into the project selection tools: thus, strategic fit

is achieved by incorporating numerous strategic criteria into the Go/Kill and

prioritization methods (Cooper et al., 1998). Scoring models are excellent tools for

incorporating and assessing these criteria.

In order to reach the second goal, align the portfolio with the strategy, it is very

important that there is a strategy. Cooper et al. (1995) identified that a clear well

communicated new product strategy for the organization is a driver for new product

performance. The strategy of an organization is the basis of NPD portfolio

management and should therefore be defined before the NPD portfolio is selected

(Archer and Gasemzadeh, 1999). You can only align a portfolio with the strategy if

the strategy is clearly defined in advanced. This strategy must be translated in to clear

goals and objectives and should be able to answer the question proposed by Kaplan


M.L.W. De Mulder 17

(1996) and Markides (1999). In the end, the mission, vision and strategy of the

business must be operationalized in term of where the business spends money

(Cooper et al., 1998). It must become clear where an organization wants to spend its

resources, which types of projects, across which markets and products lines, etc.

Resource decisions should be made corresponding to the strategy (Archer and

Gasemzadeh, 1999). This can be done with strategic buckets. A strategic bucket is a

collection of NPD projects or NPD programs that are aligned with a particular

innovation strategy (Chao and Kavadias, 2007). The NPD programs in a strategic

bucket may involve process improvements and cost reductions, minor product

modifications, radical next-generation technological research, or groundbreaking

R&D initiatives, etc. (Chao and Kavadias, 2004, 2007). While the definition of each

strategic bucket is top-down, the assignment of the projects to buckets is a bottom-up

approach. Projects are assigned to only one bucket. The criteria for placing a project

in a bucket can differ per bucket. Projects are assigned until the resources of a bucket

are depleted. Barczak et al. (2009) found that top performing organizations use the

strategic buckets methods more often than the rest is. Strategic buckets prevent non-

related projects from competing for the same resources. Table 2.4 shows an overview

of criteria for achieving the second goal for NPD portfolio management methods

Table 2.4 An overview of criteria for achieving strategic alignment

• There must be a clear mission and vision present and corresponding goals and objectives (Kaplan and Norton 1996, Cooper et al., 1998, Markides, 1999, Loch and Tapper, 2002)

• A clear strategy consists of the who, what, how, why and threats (Markides, 1999, Loch and Tapper, 2002) or consist out of four perspectives (financial, customer perspective, internal business process and learning and growth perspective) (Kaplan and Norton, 1996)

• The NPD portfolio management tool should be able review projects based bullet one and two • The NPD portfolio management tool should be top-‐down and bottom-‐up (Cooper et al., 1998). • The strategy must make clear where and how many resources should be spent (Cooper et al., 1998). • The NPD portfolio management tool must assess projects on strategic fit (Cooper and Kleinschmindt,

1995, Cooper et al., 1998) • Projects should be assigned to buckets Barczak et al. (2009)

2.3.3 Goal Three: The portfolio must be balanced The last goal formulated by Cooper et al. (1998, 1999) is that the portfolio must be

balanced. As mention before success requires a fundamental trade off: short-term

benefits accrued through incremental improvements versus long term benefits

achieved through radical or new to the world products and services (Tushman and

O’Reilly, 1996). Consequently you need to balance short-term projects with long-


M.L.W. De Mulder 18

term projects or radical projects with incremental projects. What balance is, is

determined by senior management and should be based on the strategy of an

organization. It is hard to capture balance into a single metric, the most effective way

to asses the balance of a portfolio is trough visual tools (Cooper et al., 1998, Cooper

and Edgett 2001). The current state of a portfolio can be visually depicted with bubble

diagrams where projects are plotted on an X-Y plot or map. The most popular bubble

diagram depicts risk-reward (Cooper et al., 1998, Cooper and Edgett 2001). With this

bubble diagram one axis is some measure of the reward and the other axis represents

the probability of success. On the diagram, projects are depicted based on their

resource requirements (the bigger the resource requirement, the larger the diameter of

the circle). Consequently management is forced to consider the resource issues. Often

there are a finite number of resources available, therefore the sum of the areas of the

circles must be a constant. Of course there are many more parameter that can be used

for the X or Y-axis, table 2.5 give some example of them.

Table 2.5 Example of parameters used for X or Y-‐axis

• Fit with the business or corporate strategy (low, medium, high) • R&D cost to completion (e.g. dollars) • Time to completion (Years, months) • Competitive impact of technologies (base, key, pacing and embryonic technologies) • Durability of the competitive advantage (short-‐term, medium, long-‐term) • Etc.

Source Cooper et al., 1998 page 56

There are some risks involved in using bubble diagrams. A bubble diagram gives an

overview of the entire portfolio, this means that all the projects are shown on the

diagram. However, these projects are often not in the same development phase, this

means that the information used for depiction projects differs in reliability. Projects in

early stages lack reliable (financial) data while projects in later stages do have reliable

data. Consequently there is the probability that a bubble diagram incorrectly depicts

the portfolio due to unreliable data. There is also the risk of information overload

(Cooper et al., 1998), due to the almost limitless number of variants of visual

depictions where all kinds of variable are used. An organization should only use

bubble diagrams that depict the most important information. In addition a bubble

diagram does not rank projects! Therefore they can’t be used to directly make NPD

portfolio decisions, bubble diagrams are a starting point for discussion only (Cooper


M.L.W. De Mulder 19

et al., 1998, Cooper and Edgett, 2001). Management still has to translate these data

into actionable decisions (Cooper et al., 1998). The most important question of all is

what is the right balance? Management has to clearly define what balance is and what

imbalance is. If imbalance occurs what actions should be taken, should management

immediately start cancelling projects and approving others?

A balanced portfolio results in diverse a composition of projects. Not all NPD

portfolio tools are practical for all project types e.g., hard financial tools are less

applicable for radical projects while they are easy to use for incremental projects.

Archer and Ghasemzadeh (1999) propose that the NPD portfolio method must be

flexible enough so that stake holders can choose in advance a particular techniques or

methodologies with which they are comfortable in analyzing relevant data and

making choices of the type of projects at hand. Cooper and Edgett (1999) found that

the average organization uses 2,4 methods for their portfolio selection. It must be

noted that for each type of project (or strategic bucket) the same methodology should

be chosen because the only way to objectively compare projects is when they are

assessed on common measures. This will allow an equitable comparison of projects

during the selection process (Archer and Ghasemzadeh, 1999). Which (financial)

criterion is used for valuating each project should also be determined before the

portfolio selection process starts.

The third goal is a difficult goal to attain because balance cannot be captured in a

single metric. The most effective method for achieving balance is by using multiple

NPD portfolio methods that compensate each individual method’s biases towards a

certain type of project and/or development stage. The most effective method to

evaluate balance is with a visual depiction of the NPD portfolio. The major difficulty

is how to define a visual depiction as balanced or imbalanced. Table 2.6 shows an

overview of what a NPD portfolio management method must do in order to reach the

third goal set by Cooper et al. (1998, 1999).


M.L.W. De Mulder 20

Table 2.6 Criteria for NPD portfolio management for reaching balance in the portfolio • The NPD portfolio tool must be able to visually depict the portfolio (Cooper et al., 1998) • Management must define what balance and imbalance is (Cooper et al., 1998) • The NPD portfolio should only show the for management most important diagrams (Cooper et al.,

1998) • The Bubble diagram X and Y-‐axis should be based on a organization’s strategic goals and objectives

(Cooper et al., 1998) • The NPD portfolio management tool should use multiple assessment methods for valuating the

projects (Cooper and Edgett, 1999)

2.3.4 Risk There is one important aspect of NPD portfolio management that is barely covered

but has a great influence on the value of a project and thus portfolio selection: risk. As

mentioned before, risk should be taken into account in the financial metric (Cooper et

al., 1998, Pennings and Lint, 1997), risk is also an important input for bubble

diagrams. All NPD projects deal with risk! Risk is most commonly conceived as

reflecting variation in the distribution of possible outcomes, their likelihoods, and

their subjective values (March and Shapira, 1987). Another definition of risk is the

exposure to losses in a project (Webb, 1994) or the probability of losses in a project

(Remenyi, 1996). The impact of a risk factor is usually described as depending on its

likelihood of occurring and the consequence if it does occur (Turner, 1993).

Disappointment and failures in the realization of product innovation are often due to a

naïve but optimistic attitude towards the available and required knowledge and skills

(Halman and Keizer, 1994). These gaps are directly linked to risk, if there is a large

knowledge gap within a project there is also a high risk involved. There are numerous

kinds of risks, Abetti and Stuart (1988) found three categories of risk, market,

function and technology risk. Halman and Keizer (1994) also describe three kinds of

risk, technical, organizational and commercial. Abetti and Stuart (1988) conclude that

product risk increases with the degree of product newness (as viewed by the supplier)

and decreases with innovation uniqueness (as viewed by users and market place).

During the NPD process risk should be assessed repeatedly but what is the right

moment to do so? When it is done too early the evaluation may kill promising ideas

or otherwise hamper the creative process since risk in the early stage will probably

evaluated as high or very high (Abetti and Stuart, 1988). Abetti and Stuart (1988)

argue that when an NPD project’s risk assessment is done too late organizations may

realize that the risk of a project is too high and that the organization has wasted time,

money and human resources. According to Abetti and Stuart (1988) the optimum


M.L.W. De Mulder 21

timing for performing risk assessments is after the concept testing and concept

development phase at the end of the initiation phase before product development,

when significant technical and marketing efforts will be made. Across all categories,

uncertainty tends to be high at the initial planning stage and to be reduced over the

course of the project (Sommer et al., 2007, chapter 17).

According to Halman and Keizer (1994) risk has three dimensions: occurrence,

impact and control. A particular NPD activity is seen as risky if (1) the likelihood of a

bad result is considerable, (2) the impact on the success of the NPD project is great

and (3) the ability of the team to influence it within the time and resource limits of the

project is small (Keizer et al., 2005). Each risk should be assessed on these three

dimensions, but which risks should be assessed? Keizer et al. (2005) developed a risk

reference tool for NPD based on a meta-study on risk that initially identified 325 risks

that could be reduced to 142 (see appendix 3). These risks where divided into 12 main

risks categories. Not all risks identified came from the meta-study, Keizer et al.

(2005) also conducted interviews where they discovered additional risks. Hence, 142

risks seems to be too many for effective project assessment but Keizer et al. (2005)

argue that too many projects have failed because less frequently occurring risks were

not taken seriously. This argument applies for radical new projects, although it can be

assumed that in the case of incremental innovation an organization already has a fair

overview of the risks involved. Consequently in the case of incremental innovative

project a less throughout risk assessment should be sufficient. Hence it is important

that a project is assessed on a sufficient number of risks depending on the project

type. What “sufficient” is, is a consideration that each organization has to make on its

own.

Table 2.7 Overview of risk criteria

• The NPD portfolio management tool should assess projects on risk (Halman and Keizer, 1994. Keizer et al., 2005

• Risk should be assessed on three dimensions; likelihood, impact, and the ability to do something about it (Halman and Keizer, 1994)

• Risk should be assessed on a sufficient number of criteria (Halman and Keizer, 1994)

2.3.5 Usage of NPD Portfolio Management Methods In the previous paragraphs we have mainly looked at the technical side of NPD

portfolio management. However usability is as important as technical soundness. If a


M.L.W. De Mulder 22

model is technical sound but practically useless the value of this model is limited.

This paragraph describes what the difficulties are for using NPD portfolio

management models. The NPD portfolio process has a lot of similarities with the

Stage Gate® process. Consequently, it is assumed that the usability criteria are similar.

Defining governance roles and responsibilities is an important facet of Stage-Gate®

process (Cooper, 2008). The same applies for NPD portfolio management, it must be

clear who the project leader is, who assess the project on what criteria and how many

people are involved in project assessment. The project leader is responsible for the

project and leads the project through the various development stages. However,

(senior) management makes the NPD portfolio decisions. They oversee and finance

the project. Project leaders should not make portfolio decisions because they can get

emotionally involved and become very reluctant to terminate the project, even if there

are many clear signals that the project is not going to be successful (Balachandra and

Friar, 1997).

It must also be known when and how often an NPD portfolio is assessed. Cooper

(2008) recommends doing this once a month and setting one day aside due to the

often busy schedule of senior management. The goal of an NPD portfolio

management tool is to assess the entire portfolio and make decisions on which

projects to continue, pause or abandon. Consequently a decision must be made at the

end of the NPD portfolio management meeting. This is only possible if the NPD

portfolio management method is user friendly and not too complex or appears as a

black box (Faulkner, 1996, Cooper et al., 1999, Kester et al., 2009). Cooper (2008)

identified deliverables overkill as a major threat to effective use of the Stage-Gate® system. Too much bureaucracy could also hamper the effectiveness of an NPD

portfolio management method. This can be overcome by clearly defining what the

deliverables are for each portfolio assessment. One must also be aware of overly

detailed templates. According to Cooper (2008) you must keep the deliverables and

their templates limited to the essential information needed to make the gate decisions.

Users should not be overloaded with unneeded data, but should be able to access

relevant data when it’s needed (Archer and Ghazemzadeh, 1998). Table 2.8

summarizes the criteria mentioned in this paragraph.


M.L.W. De Mulder 23

Table 2.8 Overview of NPD portfolio management model usability criteria

• Governance roles and responsibilities must be defined (Cooper, 2008) • The NPD portfolio should be periodically assessed (Cooper, 2008) • The NPD portfolio management method is user friendly and not too complex or appears as a black box

(Faulkner, 1996, Cooper et al., 1999, Kester et al., 2009) • Deliverables must be clearly defined for each portfolio assessment (Cooper, 2008) • Deliverables should be focused on the essential information needed for NPD portfolio management

decisions (Cooper, 2008) • Templates should not be overly detailed and be limited to the essential information needed for NPD

portfolio management decisions (Archer and Ghazemzadeh, 1998, Cooper, 2008)

2.3.6 Conclusion: Many criteria This chapter described what portfolio management is and why it is so difficult. At the

same time various criteria for NPD portfolio management were presented. These

criteria range from general goals to specific properties of a financial metric or from

technical criteria to criteria concerning usability. Table 2.9 shows an overview of all

criteria, which are subdivided into 7 sub categories. These criteria will be used to

evaluate the original Kalypso model.

Table 2.9 Overview of criteria for NPD portfolio management General Criteria

1.1. The NPD portfolio management tool should be usable and understandable for its users (Kester et al., 2009, Archer and Chasemzadeh, 1999,2007, Loch and Kavadias 2002)

1.2. The NPD portfolio management tool should use multiple types of methods and criteria (Literature review)

1.3. The NPD portfolio management tool should be able to rank projects based on pre-‐determined criteria (Archer and Chasemzadeh, 1999,2007)

1.4. The selection methodology should be determined before the NPD portfolio selection starts (Archer and Chasemzadeh, 1999,2007)

1.5. The NPD portfolio management tool should use a common measure which enable an equitable comparison (Archer and Chasemzadeh, 1999,2007)

1.6. The NPD portfolio management tool should not solely depend on financial tools (Cooper and Kleinschmidt, 1999)

1.7. The NPD portfolio management tool should incorporate qualitative criteria (Cooper et al. 1998, 1999) Process Criteria

2.1. The NPD portfolio method should be divided in phases (Archer and Chasemzadeh, 1999,2007) 2.2. A project should be re-‐evaluated each time it transfers to another stage (Cooper, 1990, 2008, Archer

and Chasemzadeh, 1999,2007) 2.3. Individual project development and evaluation should be based on the Stage Gate® process (Cooper,

1990, 2008, Archer and Chasemzadeh, 1999,2007) 2.4. The NPD portfolio should be reviewed periodically (Cooper et al., 1998, 1999, Archer and

Chasemzadeh, 1999, 2007, Chao and Kavadias, 2007) 2.5. Each project should be assessed individually on its (financial) value (Archer and Ghasemzadeh, 1999). 2.6. The NPD portfolio management tool should take into account the interdependencies between projects

(Archer and Ghasemzadeh, 1999) 2.7. The NPD portfolio management tool should take into account the time dependent nature of project

resource selection (Archer and Ghasemzadeh, 1999)


M.L.W. De Mulder 24

Table 2.9 Overview of criteria for NPD portfolio management (continued) Strategic Criteria

3.1. The NPD portfolio management tool must assess projects on strategic fit (Cooper and Kleinschmindt, 1995, Cooper et al., 1998)

3.2. The strategy should be determined before the NPD portfolio selection process starts (Archer and Chasemzadeh, 1999,2007)

3.3. A clear strategy consists of the who, what, how, why and threats (Markides, 1999, Loch and Tapper, 2002) or consist out of four perspectives (financial, customer perspective, internal business process and learning and growth perspective) (Kaplan and Norton, 1996)

3.4. The NPD portfolio management tool should be able to align the portfolio to the strategy (Cooper et al. 1998, 1999)

3.5. The NPD portfolio management tool should be top-‐down and bottom-‐up (Cooper et al., 1998) 3.6. The strategy must make clear where and how many resources should be spent (Cooper et al., 1998) 3.7. Projects should be classified in advance of the portfolio selection (e.g. strategic Buckets) (Cooper et al.,

1998, 1999, Archer and Chasemzadeh, 1999, 2007, Chao and Kavadias, 2007) 3.8. Projects should be assigned to buckets (Barczak et al., 2009)

Financial Criteria 4.1. The NPD portfolio management tool should strive to maximize the value of the portfolio (Cooper et al.,

1998, 1999) 4.2. The financial metric should incorporate the cost of development and launch (Cooper et al., 1998) 4.3. The financial metric should have no bias towards long term radical projects (Chao and Kavadias, 2007) 4.4. The financial metric should capture the flexibility of the development process (Newton et al., 2004) 4.5. The financial metric should be constrained by a critical resource (Cooper et al., 1998) 4.6. The financial metric should ignore sunk cost (Cooper et al., 1998) 4.7. The financial metric should incorporate the probability of success or failure (Cooper et al. ,1998,

Pennings and Lint, 1997) 4.8. The financial metric should be user friendly and not too complex or appear as a black box (Faulkner,

1996, Cooper et al., 1999, Kester et al., 2009). 4.9. The financial metric should use the Minimum Acceptable Rate of Return (MARR) and Time Value of

Money (TVM) (Remer and Nieto, 1995) Balance Criteria

5.1. The NPD portfolio tool must be able to visually depict the portfolio (Cooper et al., 1998) 5.2. Management must define what balance and imbalance is (Cooper et al., 1998) 5.3. The NPD portfolio should only show the for management most important diagrams (Cooper et al.,

1998) 5.4. The Bubble diagram X and Y-‐axis should be based on a organization’s strategic goals and objectives

Cooper et al., 1998) Risk Criteria

6.1. The NPD portfolio management tool should assess projects on risk (Halman and Keizer, 1994. Keizer et al., 2005

6.2. Risk should be assessed on three dimensions; likelihood, impact, and the ability to do something about it (Halman and Keizer, 1994)

6.3. Risk should be assessed on a sufficient number of criteria (Keizer et al., 1994) Usability Criteria

7.1. Governance roles and responsibilities must be defined (Cooper, 2008) 7.2. The NPD portfolio should be periodically assessed (Cooper, 2008) 7.3. The NPD portfolio management method is user friendly and not too complex or appears as a black box

(Faulkner, 1996, Cooper et al., 1999, Kester et al., 2009) 7.4. Deliverables must be clearly defined for each portfolio assessment (Cooper, 2008) 7.5. Deliverables should be focused on the essential information needed for NPD portfolio management

decisions (Cooper, 2008) 7.6. Templates should not be overly detailed and be limited to the essential information needed for NPD

portfolio management decisions (Archer and Ghazemzadeh, 1998, Cooper, 2008)


M.L.W. De Mulder 25

The NPD portfolio management criteria listed in table above for are not mutually

exclusive and some even conflict. Having a clearly formulated strategy is a

prerequisite for many other criteria (e.g., what bubble diagrams to use). Some criteria

conflict; the need for a usable and understandable NPD portfolio management method

conflicts with the criteria that the NPD portfolio management should incorporate

probabilities, interdependencies between projects and the time dependent nature of

projects. Consequently, for each NPD portfolio management method, a well-

considered trade-off has to be made between these criteria (usability versus

complexity).

2.4 Portfolio methods Now that we have discussed the criteria for NPD portfolio management we move on

to the actual NPD portfolio management methods available. As described in the

introduction, surveys have shown that the solutions provided by academic research

have found little use in practice (Shane and Ulrich, 2004). Scientific models often

require lots of data: e.g., information on the financial results, resources needs timing

plus probabilities of completion and success for all projects. Practitioners often don’t

have the required data and when they do get results, practitioners doubt them due to

lack of transparencies and robustness (Loch et al., 2002). Managers often perceive

solutions provided by scientists to be too difficult to understand and use (Cooper et

al., 1999). Practitioners often use multidimensional decision making tools or ranking

methods. A drawback of these tools is that they rely on an ad hoc list of dimensions

and their decision makers often manipulate these methods to generate desired

outcomes instead of using them as true decision support tools (Chao and Kavadias,

2007 and Loch and Kavadias, 2002). These tools do help management “think

through” a problem but they lack theoretical or empirical basis (Loch and Kavadias,

2002). Senior managers, R&D managers, and project managers are forced to make

resource allocation decisions based primarily on intuition or heuristic rules (Chao and

Kavadias, 2007). Several NPD portfolio management methods and tools have been

briefly mentioned during the description of the various criteria for NPD portfolio

management. These and additional methods are discussed in more detail in the next

part. A more extensive and comprehensive review of different NPD portfolio

management methods is in the literature review. What becomes clear is that there is

not a one-tool solution. In order to develop an NPD portfolio method that meets all


M.L.W. De Mulder 26

criteria set in the previous part the NPD portfolio management tool most be composed

out of multiple methods.

2.4.1 Financial methods Financial methods are the most frequent used NPD portfolio decision methods

(Cooper et al., 1999, 2001). Projects are selected based on financial indexes.

Different criteria can be used like Return on Investment (ROI), Internal Rate of

Return (IRR) or Net Present Value (NPV). The application of financial analysis to

projects or product lines is conceptual appealing because it allows management to use

the same criteria to evaluate each component of a business which stockholders use to

evaluate the organization as a whole (Cardozo et al., 1983). Senior management

formulates the financial criteria for NPD projects. Cooper and Kleinschmidt (1999)

found that organizations that only use financial tools perform less; financial methods

produce portfolios with poor-value projects, too many projects for the resources

available and gridlock in the pipeline. Another drawback of financial methods is that

the outcome can be unreliable due to the fact that NPD project data is often

speculative until market launch, there is no accurate data (Linton et al., 2002).

Financial methods also tend to disfavor advanced technology projects due to the long-

term payoffs and high likelihood of failure (Chao and Kavadias, 2008). For example

NPV often valuates inaccurately long-term and risky projects. This is because NPV

makes the assumption of managerial inflexibility, projects will proceed as planned

regardless of what happens in the future. The NPV doesn’t consider the option of

expansion, abandoning or modification of an NPD project although these options can

add value to the project (Newton et al., 2004). The NPV also assumes an immediate

initiation of the capital project (i.e. management has no leeway about the timing of the

investment).

2.4.1.1 Probabilistic financial methods Cooper et al. (1998) recommend the Excepted Commercial Value (ECV) method for

maximizing the value of the portfolio. The ECV seeks to maximize the value or

commercial worth of the portfolio, subject to certain budget constraints. The ECV

considers the future stream of earning from a project, the probabilities of both

commercialization and development cost (Cooper et al. 1998).


M.L.W. De Mulder 27

Exhibit 2.1 Determination of expected commercial value of project

!"# = !"# ∗ !!" − ! ∗ !!" − !

$ECV= Pts= Pcs= $D= $C= $NPV=

Expected Commercial Value of the project Probability of technical success Probability of commercial success (given technical success) Development cost remaining Commercialization (Launch cost) Net present value of the project’s future earnings (discounted to the present)

Source: Cooper et al. (1998) page 24

Advantages of the ECV over other financial metrics is that the ECV recognizes, that if

a project is halted partway through, certain expenses are not incurred and that the

development process is step-wise. The ECV method acknowledges that management

has GO/Kill options along the way, these options reduce the risk of a project, and that

the “correct value” of a project could be determined via a decision tree approach

(Cooper et al., 1998, Faulkner, 1996). The ECV penalizes projects that are years away

from launch by discounting all dollar amounts today (not just from launch date).

Both financial and probabilistic financial methods are most effective in later stages of

the development process. Stage 0 and 1 of the development process are characterized

by high uncertainty and consequently lack reliable information for the determination

of a quantitative financial metric. In later stages (from stage 2, development onward)

the information used to determine the quantitative financial metric become more

reliable. Hence, quantitative financial methods are best suited for the assessment of

projects from gate 3 onwards.

2.4.2 Strategic methods Strategic methods split resources into smaller strategic buckets. A strategic bucket is a

collection of NPD programs that are aligned with a particular innovation strategy

(Chao and Kavadias, 2007). The role of the strategic buckets method is to translate

the business’s strategy into clearly defined areas (buckets) and to decide resource

allocation to each (Cooper et al., 1998). Strategic buckets is a top down approach, the

number and type of buckets are determined by senior management and are based on

an organization’s strategy. Projects are assigned to only one bucket until the resources

of the bucket are depleted. The criteria for placing a project in a bucket can differ per

Development $D

Launch$C

TechnicalSucces

CommercialSucces $NVP

Commercial Failure

Technical Failure

Pts

Pcs

Yes

No

Yes

No

$ECV


M.L.W. De Mulder 28

bucket. Barczak et al. (2009) found that top performing organizations use the strategic

buckets methods more often than the rest is. An advantage of strategic buckets is that

it prevents totally different projects from competing for the same (scarce resources).

Project selection inside a bucket can be based on financial metrics. Note that the

strategic buckets method doesn’t have a tool for assigning projects to the buckets,

strategic buckets method decrease the scope of the NPD portfolio problem but doesn’t

resolve it.

Strategic methods are best used in the early stage of the development process (stage 0,

1 or 2). Strategic methods help organizations to implement their strategy by

dedicating resources to a specific strategic goal. After stage 2 the cost of a project

increase significant during the project development in stage 3. Therefore projects

have to be assigned to a specific bucket before stage 3. Consequently projects that

don’t fit any strategy should be filtered out at latest at gate 3. After gate 3 strategic

methods loose some of their importance.

2.4.3 Checklists & Scoring models Checklists and scoring models are examples of a bottom-up approach. Checklists are

used for examining whether or not a product met the requirements set. Projects are

ranked according to number of criteria’s met. Every criteria has an equal weight and

is consequently equally important. It occurs often that not all criteria are equally

important. Scoring models, opposite to checklist, can take the weight of different

criteria into account for calculating an overall score. Scoring models are used to

determine, various criteria, table 2.10 shows which criteria are used by organizations

rank by popularity.

Table 2.10 Criteria used to Rank Projects

1. Strategic fit/ leverages core-‐ competencies

2. Financial reward / pay-‐off 3. Risk and probability of success 4. Timing 5. Technological capability 6. Commercialization capability 7. Protectability 8. Synergy between projects



M.L.W. De Mulder 29

Determining the weighting and scores is a subjective process that depends as much on

personal opinion as on hard numbers (Coldrick et al., 2003). Projects are ranked

according to the overall score and can be added or deleted without affecting the scores

of the remaining projects. An advantage of scoring methods is that it is possible to

rate projects on multiple criteria from economic to non-economic. Scoring models are

effective analytical tools when data requirements and complexity of more sophisticate

approaches are considered not justified (Dean et al., 1965). According to Paolini and

Glaser (1977) scoring models are most applicable during basic research and

exploratory development. Another advantage of scoring models is that they are

quantitative enough to possess a certain degree of rigor but not so complex to mystify

and discourage potential users (Henriksen et al., 1999). Cooper (1981) states that

scoring models are often seen as oversimplifications, they attempt to reduce a

complicated decision situation to a simple equation yielding a composite score. A

drawback of scoring models is that criteria should be mutually exclusive, in real life,

criteria are often not independent (Cooper, 1981). Scoring models are less appropriate

when there is a high degree of interdependence between projects, when activities and

results of one project depend on the activities and results of a different project

(Henriksen et al., 1999). However, Cooper and Kleinschmidt (1999) found that

scoring models yield high performance results in terms of producing a portfolio of

high value products.

Checklists and scoring models are usable during the entire development process

depending on the criteria used. These criteria should increase in depth and scope

going from gate to gate during the development process. In early stages, the criteria

should be broad while later in stages the development process, when more detailed

information is available, projects should be assessed on specific and detailed criteria.

2.4.4 Bubble diagrams A bubble diagram is a diagram which represents information visually in the form of a

series of bubbles. Projects are plotted on an X-Y plot or map. The axes vary, the most

popular chart is the risk-reward bubble diagram, where the NPV is plotted versus

probability of technical success (an example of a bubble diagram is shown in figure

2.3).


M.L.W. De Mulder 30

Figure 2.3 Risk-‐Reward Bubble diagram

Pro

babi

lity

of T

echn

ical

Suc

cess

Reward (NPV)

1256 3 078

High

Low

Pearls

Oysters White Elephants

Bread and Butter

Circle size = resources annual Source: Cooper et al. (2001), page 396

Other types of bubble diagrams are; technical newness by market newness, technical

feasibility by market attractiveness and competitive position by project attractiveness

(Cooper et al, 2001). Bubble diagrams should not be used for making portfolio

decisions (Cooper and Edgett, 2001). They depict the current state of the portfolio and

where resources are going. A visual depiction portrays balance much better than

numbers or lists (Cooper and Edgett, 2001). Bubble diagrams provide a start point for

discussions on what should be and how your resources should be allocated (Cooper

and Edgett, 2001). Hence bubble diagrams are not recommended for making

decisions because they only depict the current situation, rather, they should be used

for information display.

Bubble diagram are only used for depicting the current state of the portfolio. Bubble

diagrams can be used during the entire development process. Users should be aware

that the information used in diagrams differs in reliability depending on the

development stage a project is in. E.g., a risk-reward bubble diagram of the entire

NPD portfolio will depict projects of all stages of the development process, the

information used for depicting a project in stage 0 will be far less reliable than the

information for depicting a project in stage 4.

2.4.5 Conclusion: Which Method? Every method has its advantages and disadvantages, some methods are better suited in

the beginning of the NPD process while other are more suited during the end.


M.L.W. De Mulder 31

Organizations use different criteria at each gate, some methods are more useful than

other depending on the criteria used, hence organizations must use multiple methods.

In the second part of this chapter a list with criteria was formulated for an NPD

management model, this list is used to evaluate the five methods mentioned in the

previous paragraph. Each method received a score from 1 to 7 point scale for each

criteria. This is assessment is done by the author. If a method receives a score of 1 the

NPD portfolio management method doesn’t meet the criteria at all. When a project

receives a score of 7 the NPD portfolio management method complies totally with the

criteria. In this simple assessment the average score determines the overall score of

the method. The results and substantiation (when deemed necessary) of the

assessment are shown in appendix 4. A summary of this assessment is shown in table

2.11 accompanied with the main advantages and disadvantages of each method.

Table 2.11 overview of different NPD portfolio methods with their assessment score, advantages and disadvantages

Assessment score Advantage Disadvantage

Best used at gate

Financial methods 3,09

Conceptual appealing, popular metric in business

Disfavors high risk, very innovative projects. Less useful in the early stages of the development process

3, 4 or 5

Probabilistic financial methods 3,48

It doesn’t disfavor high risk projects, popular metric in business

The validity of the outcomes of these probabilistic methods depends on the estimates made. Less useful in the early stages of the development process

3, 4 or 5

Strategic Methods

3,98

Makes it possible to align portfolio to strategy, doesn’t disfavor high risk projects

How to determine what projects are put into the bucket? It does decrease the scope of the NPD portfolio problem but doesn’t solve it.

1, 2 or 3

Checklist and Scoring methods

3,73

Ability to rate projects on multiple factors from economic to non-‐economic. Effective analytical tool when data requirements and complexity of more sophisticate approaches are considered not justified

Can be seen as oversimplifications. Less appropriate when there is a high degree of interdependence between projects

All gates*

Mapping approaches or bubble diagrams

3,50

Depict the current state of the portfolio and where resources are going

Should not be used for making decision, only for information

All gates

Poor 1 2 3 4 5 6 7 Excellent *Criteria differ per gate

The first thing that strikes is that each individual method receives a relatively low

score on the assessment. Strategic methods receive the highest score, a 3,98 out of 7.

These scores are based on the how well each method meets the criteria set for an NPD


M.L.W. De Mulder 32

portfolio management method. Hence none of these NPD portfolio management

methods is sufficient enough to be used as the only NPD portfolio management tool

of an organization! This is not surprising, each method is best suited for assessing

projects in a specific stage. When a model only uses one method you expose a project

to the limitation and biases of such a method. There is no single criteria or value

driver that is able to correctly represent the value of a project throughout the entire

development process. Financial methods for example are less suited for assessing

projects in early development stages due to the lack of reliable data, when more

reliable data becomes available during later development stages financial methods

become more usable. Cooper and Kleinschmidt (1999) found that organizations that

only use financial tools perform less; financial methods produce portfolios with poor-

value projects, too many projects for the resources available and gridlock in the

pipeline. Organizations use more than one NPD portfolio method to compensate the

drawbacks the different methods (Cooper et al., 1999, 2001, 2008). The problem with

current portfolio methods is that most of them use different measurements to rank

projects, it impossible to compare projects ranked with different methods How to

compare a financial ranked project with a scored project or a project selected based on

consensus? The only way to objectively compare projects is that they are assessed on

common measures. This will allow an equitable comparison of projects during the

selection process (Archer and Ghasemzadeh, 1999). Even if one uses the same

method for different phases it is still very difficult to compare because more data is

available near the end of the process.

In the next chapter the NPD portfolio management tool developed by Kalypso is

described. Whether this tool performs better than the current NPD portfolio

management methods will become clear in chapter 5.


M.L.W. De Mulder 33

Chapter 3: Description the Original Kalypso model This chapter describes the NPD portfolio model developed by Kalypso. First the goal

of the model is explained. Next we dive into the process, various inputs are discussed

and the output is described. Accompanied are various text boxes, in which two

(coded) projects and their journey through the model is described. One project is the

development of a desert and the other project concerns the development of a cereal

bar. This chapter ends with a conclusion.

3.1 Goal Kalypso model Kalypso developed an NPD portfolio management model for one of its clients. This

model enables the client to ranked and prioritize NPD ideas, concepts and projects.

Projects that are more attractive to the client should be ranked higher and should be

better resourced than less attractive projects. This model should also be able to give

insight into the composition of the portfolio e.g. how many project are in what

development stage, what is the distribution of risk in the portfolio with respect to the

earning, etc. The client operates in the food processing industry, they operate globally

although most of the products are developed locally for local markets. The client has

currently a wide spectrum of approaches to prioritize initiatives, ranging from no

prioritization to a complex prioritization matrix. The client has a Product Lifecycle

Management (PLM) maturity level 0, which is described by Batenburg et al. (2005) as

a level where project investments are done on an ‘ad-hoc’ basis only. Nobody is

responsible for PLM and there is no vision available for PLM. Therefore, there are no

consistent PLM processes and supporting systems. At this level, information about a

product is scattered throughout the organization, which hinders strategic decision-

making. PLM maturity refers to the cumulative nature of the deployment process, the

organization has to go through different stages of growth before PLM is implemented

at all levels and connects all managerial aspects (for a description of the different

maturity levels see appendix 2) (Batenburg et al., 2005). The NPD portfolio

management tool developed by Kalypso is the first step towards a higher PML

maturity. However NPD portfolio is tailored to the client’s current maturity level, this

has led too some “strange design” choices that will be discussed later.

One of the objectives of the Kalypso Model is to establish a standard prioritization

matrix for all of the client’s R&D initiatives around the world. The Kalypso model is


M.L.W. De Mulder 34

constrained by the metrics used at the client, e.g. Net Sales Value and the Gross Profit

minus Marketing expenses are the internal financial metrics and must be used. At the

client there are three types of R&D projects, innovation, range maintenance and cost

out. Innovative projects are projects that are new to the client, range maintenance are

changes in existing products while cost out projects concerns value engineering of

products. The main difference between range maintenance and cost out is that the first

one is done reactively to market and supply changes and the latter one is done

proactively. Range maintenance and cost out projects are often incremental of nature.

Consequently, there is already al lot of information available for cost-out and range

maintenance projects. The client has implemented a stage gate process, which is

depicted in exhibit 3.1.

Exhibit 3.1 implemented Stage Gate process

The last stage, stage 6 does not have a direct influence on the NPD portfolio selection

process and shall be ignored from now on. The portfolio is reviewed once a quarter by

senior management.

Develop ideas into consumer validated concepts

Con

cept

Qua

lifica

tion

Develop R&D prototype and marketing mix trough sequential recyclingPr

oduc

t Qua

lifica

tion

Develop, test and optimize product: finalize marketing and sales plans

Mar

ket Q

ualifi

catio

n

Full implementationof launch plans. Monitor inmarket performance

Laun

ch

Project closed

Post

-Lau

nch

Aud

it

Deep dive into the opportunity

Idea

tion

Stage 1 Stage 3 Stage 4Stage 2 Stage 5 Stage 6


M.L.W. De Mulder 35

3.2 Process The process diagram of the Kalypso model is depicted in exhibit 3.2, there are various

inputs which are used to calculate the project’s rank. In this paragraph the whole

Kalypso model process will be described in detail. Next we describe the various

inputs.

Exhibit 3.2 Kalypso model process diagram

*!" = !"# − !"#$ !" !""#$ !"#$ −!"#$%&'() !"#$%$%

Projecttype

Product category

Up-aging

Geography

Area of focus

Risk Category

Project Stage

ProjectStatus

Urgency

Capability

Portability and

scalability

Stage 3,4 & 5

Project priority

Financial value driver *

Strategic fit value driver

Risk value driver

Stage 1 & 2

Project priority

Financial value driver


Risk value driver

10 %

80%

10%

60%

10%

30%

High - 15Medium - 5Low -1

High - 15Medium -10Low - 1

High - 1Medium - 5Low - 15

High - 15Medium -10Low - 1

High - 1Medium - 5Low - 15

Value Category Value Driver

OutputProcessInput

Project priority

Project priority

Ranked Projects based on Priority

Project Information

Nametype

Project Owner

Work Country

Project Country

LaunchDate

GPConstant ×15


M.L.W. De Mulder 36

3.2.1 Input There are three kinds of input for the Kalypso model, project information, value

categories and value drivers. Project information contains regular project information

such as project name, project description and project number. Value categories are

input variables that can be used to identify or specify a project (they are used to

describe a project). An overview of all the value categories used in the Kalypso model

is shown in table 3.1.

Table 3.1 Overview of value categories Value Category Description 1. Project type The kind of project, innovation, range maintenance or cost out 2. Project category To which main product group does the project belong 3. Up-‐aging A project that aims to extended the age range of customers 4. Geography Where is the project launched, Europe, North America, Asia Pacific or the rest of the world 5. Project stage In which stage is a project 6. Area of focus Product, packaging, process conditions and scientific activities 7. Project status Project status indicates the overall state of a project and general point in the project

lifecycle: active, completed, on hold or canceled Only one value category has an influence on the ranking of projects; project stage.

The stage where a project is in determines what weight is allocated for each value

driver.

Value drivers are the third type of input. The overall project value is based on these

value drivers. Value drivers can and should be reduced to a number or a choice that

can be put in a table or plotted in a chart. There are 6 different value drivers identified

during the Kalypso model development project. Table 3.2 gives an overview of the

different value drivers.

Running Example Project Name Desert X Cereal Bar Y Project Owner Mr S. Mr. T. Project Country Italy India Work Country Italy India Launch Date Feb 2010 March 2010 Basic project information is entered.


M.L.W. De Mulder 37

Table 3.2 Overview of value drivers Value driver Description 1. Financial Metrics that are used to capture the fiancial value of the project 2. Strategic Fit Represents the alignment of the project with the global strategy 3. Risk The likelihood and impact of an adverse event resulting form a 4. Urgency Projects that are urgent must be completed regardless of value drivers 5. Capability Does the development of the project fit in whith the capabilities of the

organization 6. Portability and scalability Can the product be launched elsewhere

Only three of the six value drivers are used for ranking a project; financial, strategic

fit and risk. The weight of each value driver differs per stage. In early stages the

financial and risk value drivers weigh less because the information on which these

values drivers are based are often vague and unreliable. Strategic fit on the other hand

is an important consideration in the early stages of the development process. Value

drivers are also used to visually depict the composition of the portfolio e.g., risk and

financial. Each value driver is described in the next part.

3.2.1.1 Financial value driver There are two kind of financial value drivers, a qualitative and a quantitative one. The

qualitative value driver is used for projects in the first two stages of the development

process. In these stages there is no reliable financial data available. The qualitative

financial value driver is based on the probability of financial success. Market size,

marked need and competition are important indicators for financial success and are

therefore used for assessing the probability of financial success (Montoya-Weiss and

Running Example Continued Desert X Cereal Bar Y Project Type Innovation Innovation Product category Infant Nutrition Area of focus Product Product Up-‐aging No Yes Geography Europe Asia Pacific Project stage Stage 3 Stage 2 Project status Active Active Both projects are new and thus belong to the innovation product type. Desert x belongs to the infant product category, which are products for babies (0-‐3 year). The two products are new to the organization, hence the focus lies on the product (not packaging or process). Cereal Bar Y is in the Nutrition product category which focuses on specific markets (special diets e.g., diabetics or gluten free). Cereal Bar Y could also be targeted at customers of various ages, Desert X not (only babies). Desert X is targeted at the European market while Cereal Bar Y is aimed at the Asia Pacific market. Desert X is development stage 3 whereas Cereal bar Y is in stage 2. Both projects are actively in development.


M.L.W. De Mulder 38

Calantone, 1994, Cooper et al., 1995, Henard and Szymanski, 2001, Pattikawa et al.,

2006). In the Kalypso model there are three probability levels for financial success,

high, medium and low. Table 3.3 shows on what criteria the qualitative financial

value driver is based.

Table 3.3 Qualitative criteria for assessing the probability of financial success • High The market is large and growing, the idea or concept could be highly profitable, and the

need for the product is significant, competition is vulnerable. • Medium The market is moderate but growing, the idea or concept could be moderately profitable,

the need for the product has a slightly above average advantage over what is available, competition is moderate and sustainability of competitive advantage is 1-‐2 years.

• Low The market is slight and stable, the idea or concept has low-‐medium profitability, the need for the product is as yet uncertain, and competition is strong.

The quantitative financial value driver is used if a project is in stage 3, 4, or 5. In

these stages reliable financial is available because of market research. The metric used

is the Gross Profit Margin minus Total Marketing Expenses (GP). The GP is

determined for the 2nd full fiscal year after the launch year. For example, if a product

was launched in October 2010, the GP should be for 2012. The idea is to reflect

market penetration and defrayed marketing costs. The calculation of the GP is as

follows:

!" = !"# − !"#$ !" !""#$ !"#$ −!"#$%&'() !"#$!!"!

Where NSV stands for Net Sales Value. The NSV is revenue generated form the sales

of a specific project. The GP and NSV are standard metrics used at the client and are

therefore used in the Kalypso model.

Running Example Continued Desert X Cereal Bar Y Financial value driver $5.250.000 Low Because Desert X is in stage 3 a quantitative prediction of the Gross Profit Minus the Total Marketing Expenses can be made. Cereal Bar Y on the other hand is only in stage 2, therefore a qualitative assessment is made. Cereal Bar Y is targeted at a stable market where the margins are low. Competition is the Cereal Bar market is strong and the need for the product is not known. Hence the financial value driver for Cereal Bar Y is low.


M.L.W. De Mulder 39

3.2.1.2 Strategic fit value driver The second value driver used to rank a project is strategic fit. This value driver

indicates how well a project is aligned with the strategy of the client. The assessment

of strategic fit is based upon qualitative criteria. There are three levels of strategic fit,

high, medium, and low. The criteria and the corresponding levels of fit are shown in

table 3.4.

Table 3.4 Criteria for determining level of strategic fit High The idea or concept keeps the client on track with areas of performance (AOP)/strategic plan

objectives Medium • The idea or concept keeps emerging markets growing at current rate.

• The idea of concept strengthens competitive position in developed markets • The idea or concept produces a competitive "game changing" advantage • Leverages global scale

Low The project does not fit with the client’s strategic

In the Kalypso model is strategic fit the most important value driver when a project is

in stage 1 or 2. In later stages of the project development process the weight of

strategic fit decreases because the other value drivers become more reliable.

3.2.1.3 Risk value driver The last value driver used for ranking projects is risk. Risk is the likelihood and

impact of an adverse event resulting from a project. Risk is divided into three

subcategories; technical risk, commercial risk and regulatory risk. The subcategory

with the highest risk determines the overall risk of a project. For the determination of

technical risk three factors are important; familiarity, time and resources. Familiarity

implies how much expertise the client has in developing a project, is it a totally new

venture or does the client has a lot of expertise in this field. The more familiar the

client is with a project the more likely it is that the product is successfully developed.

Time entails the amount of time available for the development of the project. Hence,

if there is plenty of time available for development the risk will be low and visa versa.

The third factor that determines technical risk is resources. The availability of key

Running Example Continued Desert X Cereal Bar Y Strategic fit value driver High Medium The strategic fit value driver of Desert X is high since it keeps Italy on track while the strategic fit of Cereal Bar Y is medium because it keeps emerging markets growing at current rate (The author does not know the precise considerations of this assessment).


M.L.W. De Mulder 40

resources is an important factor for the successful development of a project. The lack

of available resources poses a risk for the development.

The second subcategory is commercial risk. Commercial risk is about the probability

of unfavorable events due to poor market response and price point. Another part of

commercial risk is product introduction speed and method, are there distribution

channels available or need they be build? Proprietary position and patent defensibility

are also part of commercial risk. The last part of commercial risk is the probability of

a response from the competition with a similar or better product.

The last category is regulatory risk. Two factors are looked at when assessing

regulatory risk, regulatory and label claim. Regulatory is about the existence of

regulatory issues and how much effort is needed to comply. Label claims are specific

risks for the industry the client operates in.

Each subcategory is assessed on a five-point scale where 5 represents the highest and

1 the lowest risk. Qualitative criteria are used for determining the risk level of each

subcategory. The highest risk score determines the overall risk level of a project.

There are three levels of risk, high, medium and low. The weight of the risk value

driver is low when a project is in stage 1 or 2. Risk increases in importance when a

project is in stage 3, 4 or 5 because the information used to assess the risk of a project

becomes more reliable during later stages. The risk assessment tool is shown in

exhibit 3.3.

Exhibit 3.3 Criteria for determining project risk level Risk Level Technical Risk Commercial Risk Regulatory Risk

5 (Highest)

• Familiarity – This has not been not done before and is new to the world.

• Consumer – Little or no information on the consumers or price point is available.

• Regulatory – Major regulatory issues exist. It is likely the product will need radical effort to comply.

• Time – There is extremely little time to complete the project.

• Customer – Channels required are new to the world.

• Label Claims – It is very likely the product label claims cannot be communicated or will require extremely large investments in scientific research. These label claims are critical to competitive position.

• Resources – Key resources are highly likely to be unavailable.

• Competitor – Simultaneous or pre-‐emptive competitor response likely.


M.L.W. De Mulder 41

Exhibit 3.3 Criteria for determining project risk level (continued)

4

• Familiarity – This is new to the client, but not new to the world.

• Consumer – The need is unarticulated by consumers, but is inferred from research. There is little data on price point.

• Regulatory – Regulatory issues exist. It is likely the product will require significant effort to comply.

• Time – There is significantly less time to complete the project than desired.

• Customer – New channels are required, but these channels have been used by others.

• Label Claims – It is likely the product label claims cannot be communicated or will require significant investment in scientific research. These label claims are important to competitive position.

• Resources – There is a strong chance that key resources are unavailable.

• Competitor – Immediate post-‐launch response likely to be substantial and vigorous.

3

• Familiarity – This is not new to the client, but it is not one of our present skills.

• Consumer – The need has been articulated by consumers though not validated. Price point data is available but not validated.

• Regulatory – Regulatory issues exist. The product may require some effort to comply.

• Time – The time period to complete the project is less than desired. However, this is not likely to severely impact the project outcome.

• Customer – New channels or customer relationships are required, although existing channels can be used as a foundation.

• Label Claims – There is a possibility that product label claims cannot be communicated as desired. These label claims have a moderate impact on competitive position.

• Resources – There is a moderate chance that key resources are unavailable.

• Competitor – Immediate post-‐launch response likely to be limited.

2

• Familiarity -‐ This is a good fit with the client core competencies, but a similar project has not been done before.

• Consumer -‐ The need has been articulated by consumers and validated. Price point has been validated.

• Regulatory -‐ Minor regulatory issues exist. It is unlikely that the product will require effort to comply.

• Time -‐ There is slightly less time to complete the project than desired. However, this is very unlikely to negatively impact the project.

• Customer -‐ Need to modify existing channels or customer relationships to accommodate new product.

• Label Claims -‐ There is a small possibility that the product label claims cannot be communicated as desired. These label claims are not crucial to the competitive position

• Resources -‐ There is a small chance that key resources are unavailable.

• Competitor -‐ Delayed competitive response likely.


M.L.W. De Mulder 42

Exhibit 3.3 Criteria for determining project risk level (continued)

1 (Lowest)

• Familiarity -‐ The client has experts in this and have done this before.

• Consumer -‐ The need has been highly articulated by consumers, and validated. Price point has been extensively validated.

• Regulatory -‐ There are no known regulatory issues related to the product.

• Time -‐ There is sufficient time to complete the project.

• Customer -‐ Existing channels and customer relationships are in place and can be used.

• Label Claims -‐ There are no known issues related to product label claims.

• Resources -‐ There are no foreseen issues related to resource availability.

• Competitor -‐ No significant competitive response anticipated

3.2.1.4 Portability and Scalability, Urgency and Capability The value drivers portability and scalability, urgency and capability are not used for

ranking projects. The Kalypso model is used locally, the value driver portability and

scalability is better assessed globally, a local R&D unit isn’t able to assess the global

considerations that are needed for the portability and scalability assessment.

Consequently, the portability and scalability value driver is not used in the model. The

value driver capability is not included because this value driver is covered in the risk

assessment. Urgency is not included but it does impact the ranking of the project.

3.2.2 Processing & Output Once all value categories and value drivers are entered into the model the processing

of this data begins. The final ranking is based on three value drivers, financial,

strategic fit and risk. A project can receive minimal 1 point and maximal 15 points for

Running Example Continued Desert X Cereal Bar Y Risk value driver High Medium

Desert X: There is less time available for the development as desired although it is not likely that this will severely impact the outcome (risk level 3). The risk lies in the competition, there is a high probability that the competition will response vigorously (with competing products) to the launch of Desert X. (risk level 4) There are no known regulatory issues, it is very unlikely new regulations are created that will impact the product and there are no public relations risks (risk level 4). The overall risk ranking of Desert X is high due to the high probability of competitor response. Cereal Bar Y: There is sufficient time for the development of Cereal bar Y and the client is familiar with this kind of project (risk level 2). The customer needs have been articulated and validated (hence the customer group is very specific) (risk level 2). A low chance exists that new regulations will be introduced that impact the project (risk level 2). The overall risk level of Cereal Bar Y is medium (level 2).


M.L.W. De Mulder 43

each value driver. There is one exception if a project is in stage 3, 4, or 5, in this case

the amount of points is based on the Gross Profit minus the Marketing Expenses (GP).

The GP is divided by a constant and multiplied by 15. The distribution of points per

category is shown in table 3.5.

Table 3.5 Distribution of points per value driver Value driver Stage Level Score Financial Stage 1 or 2 High 15 points

Medium 5 points Low 1 point

Stage 3, 4 or 5 GP!"#$%&#%

∗ 15

Strategic fit Stage 1, 2, 3, 4 or 5 High 15 points Medium 10 points Low 1 point

Risk Stage 1, 2, 3, 4 or 5 High 15 points Medium 5 points Low 1 point

As mentioned before not all value drivers are weighted equal, the weight of each

value driver depends on the stage a project is in. In stages 1 and 2, is strategic fit the

most important value driver. The financial and risk value driver are less important

because they are difficult to determine due to the lack of reliable data. Strategic fit

becomes less important after stage 1 and 2 because projects that don’t fit the

organization should be filtered out. In addition the information used for the other two

variables becomes much more reliable in stage 3, 4 or 5. If a project is in stage 3, 4 or

5 the financial value driver weighs most. Table 3.6 shows an overview of the different

weights of each value driver per development stage.

Table 3.6 Weighting of each value driver for each phase Stage Financial Strategic Fit Risk Stage 1 10% 80% 10% Stage 2 10% 80% 10% Stage 3 60% 10% 30% Stage 4 60% 10% 30% Stage 5 60% 10% 30%


M.L.W. De Mulder 44

For the calculation of the NPD portfolio score each value driver score is dived by 15

and summed up.

!"# !"#$%"&'" !"#$% =!"!! ∗ !!15 +

!" ∗ !!15 +

!"! ∗ !!15

Where: ai = weight of Strategic Fit Score (SFS) 1 < ai <100 bi = weight of Risk Score (RS) 1 < bi <100 ci = weight of Financial Score (FS) 1 < ci <100 i = stage project is in ∈ (0,1,..,4)

The project with the highest NPD portfolio score suits the client best and should

therefore be resourced first and best. Ranked projects are divided into three

categories; gold, silver and bronze.

3.2.3 Charts and Diagrams The Kalypso model produces various charts and diagrams. These charts and diagrams

provide a visual depiction of the NPD portfolio. Table 3.7 lists various charts and

diagrams the Kalypso model produces. Most of the charts and diagrams consist out of

a value driver compare to a value category.

Table 3.7 List of charts and diagrams the Kalypso Europe model produces

• Number of and types of projects diagram • Number of urgent projects diagram • Number of gold, silver and bronze projects diagram • NSV by Launch Month chart • Average priority by launch chart • Number of projects by category (incl. gold, silver and bronze) (infant or nutrition) chart • Number of projects by type (incl. gold, silver and bronze) (e.g. innovation, cost out) chart • Number of projects by area of focus (e.g. product, packaging etc.) (incl. gold, silver and bronze) chart

Running Example Continued Desert X Weighting Cereal Bar Y Weighting Financial value driver 10534000 à !.!"#.!!!

!""""""∗ 15 =

39.4 points

60% Low à 1 point 10%


High à 15 points 10% Medium à 10 points 80%

Risk value driver High à 5 30 % Medium à 5 10% Overall Score 1,78 0,57 The weighting per stage (Desert X, stage 3 and Cereal Bar Y, stage 2) is based on table 3.6. Each weighted value driver is dived by 15, the sum of these three weighted value drivers is the final score. Desert X has a higher score than Cereal Bar Y, hence when management has to choose between the two projects, project Desert X is the better option.


M.L.W. De Mulder 45

Table 3.7 List of charts and diagrams the Kalypso Europe model produces (continued)

• Number of projects by stage (incl. gold, silver and bronze) chart • NSV by risk chart • Gross profit by risk chart • Stage by NSV chart • Stage by gross profit chart • Stage by priority chart • NSV by project cost chart • Gross profit by project cost chart • Average project priority by category (infant, nutrition) chart • Average project priority by project type (e.g. innovation, cost out, etc.) chart • Average priority by area of focus (e.g. product, packaging) chart • NSV by strategic fit chart • Gross profit by strategic fit chart • Percentage of stage 1-‐2 projects by market attractiveness chart

3.3 Usability In the Kalypso model responsibilities are clearly defined. Exhibit 3.4 gives insight in

the task and responsibilities in different levels of management at the client.

Exhibit 3.4 Responsibilities in the Kalypso Model

The president of the organization is responsible for defining a global strategy and for

monitoring the high-level key performance indicators essential for the strategy. The

global strategy is the basis for the R&D strategy developed by the global VP of R&D

and global marketing. The regional R&D strategy is developed by the regional VP

and is based on the global R&D strategy. In the end the global strategy is cascaded

down until country level where the country president, R&D director, marketing

!"#$%&#'(

)*+,-*./!.012

)*+,-*.3-"4#5'6

)*+,-*.!!37"+8#$$.9:'#"

0#6%+'-*./!

;+<'("=.!"#$%$#'(

012.2%"#8(+"

3-"4#5'6.2%"#8(+"

><77*=.;?-%'.2%"#8(+"

@%'-'8#.2%"#8(+"

>#(.)*+,-*.>("-(#6=

>#(.0#6%+'-*.012.>("-(#6=

>#(.;+<'("=.012.

>("-(#6=

/-*%&-(#.;+<'("=.012.

>("-(#6=

2#A#*+7.012.

!+"B+*%+

/-*%&-(#.012.

!+"B+*%+

CD#8<(#.012.

!+"B+*%+

;+'&<8(!30.

E>(-6#.6-(#."#A%#:$FE3+'(?*=F

;+'$+*%&-(#!30.2-(-

;+'&<8(.!+"B+*%+.

0#A%#:.-'&.!"+A%&#.@##&,-84EG<-"(#*=

0#A%#:.-'&.>#'&.

@##&,-84

0#A%#:.-'&.>#'&.

@##&,-84

975H%I#.-'&.

CD#8<(#.+'.012.7+"B+*%+

>#(.)*+,-*.012

>("-(#6=


M.L.W. De Mulder 46

director, supply chain director and the finance director formulate together a country

specific R&D strategy. The regional VP validates this strategy. The R&D director,

marketing director, supply chain director and the finance director develop an NPD

portfolio which is validated by the country president. The R&D director, marketing

director, supply chain director and the finance director are responsible for the

execution of the R&D portfolio. Each month individual projects are assessed and

valuated by the country president, R&D director, marketing director, supply chain

director and the finance director. The global ppm process owner consolidates all the

individual assessments from different countries. These collected individual

assessments are reviewed (quarterly) in total by the president and global VP of R&D

and provide feedback. Subsequently this feedback is reviewed by the regional VP

who sends it thereafter to the country director. The country director also reviews the

feedback and sends his feedback towards the R&D director, marketing director,

supply chain director and the finance director who then optimize the R&D portfolio

according the feedback. Table 3.8 gives an overview of the responsibilities of the

different people in the process.

Table 3.8 Role and Responsibilities Role Responsibility President • Set global strategy for Infant & Nutrition.

• Monitor high-‐level KPIs to ensure strategy is adhered to. Global VP R&D • Translate global strategy into global R&D strategic goals.

• Monitor the global R&D project portfolio to ensure strategic R&D goals are accomplished.

Global Marketing • Help translate global strategy into global R&D strategic goals. • Monitor the global R&D project portfolio to ensure strategic goals are

accomplished. Global PPM Process Owner

• Act as an owner of the overall PPM process. • Facilitate the flow of data and prepare global portfolio views.

Regional VP • Translate global R&D strategy into a regional R&D strategy. • Provide guidance on local R&D strategy to countries. • Monitor regional performance.

Country President • Translate regional R&D strategy into actionable country R&D specific strategic goals.

• Monitor the country R&D project portfolio and ensure local R&D portfolio achieves the country’s goals.

Local R&D Director • Execute on projects in the portfolio. • Maintain the correct mix of projects to meet country goals. • Provide input on the feasibility of projects in the portfolio.

Marketing Director • Provide input on the feasibility of projects in the portfolio. Supply Chain Director • Provide input on the feasibility of projects in the portfolio. Finance Director • Provide input on the feasibility of projects in the portfolio.


M.L.W. De Mulder 47

3.4 Conclusion The Kalypso model enables the client to assess each project individually on three

different value drivers (financial, strategic fit and risk). The weight of each value

driver is determined by the stage a project is in. The financial value driver is

determined in the first two stages of the development process with a qualitative

assessment. In later stages the financial value drivers is calculated based on the NSV.

Strategic fit and risk are determined via qualitative criteria. Based on these weighted

value drivers a general score is calculated that enables users to rank projects based on

their individual score. The Kalypso model produces various visual depictions of the

NPD portfolio after each project is ranked. Various people throughout the

organization are involved in the NPD portfolio process who each has his/her

individually responsibilities. In the next chapter the Kalypso model will be evaluated

with the criteria formulated in chapter 4 and will it become clear how well the

Kalypso model is able to fulfill its purpose.


M.L.W. De Mulder 48

Chapter 4: Evaluation of the Original Kalypso Model In the previous chapter the Kalypso model is described. In this chapter the Kalypso

model is evaluated. The evaluation is based on the criteria discussed in chapter 2. The

Kalypso model will first be evaluated based on the general criteria. Thereafter the

Kalypso model will be evaluated based on process, strategic, financial, balance finally

risk and usability criteria. The Kalypso model receives a score of 1 (poor) to 7

(excellent). Having seven points tends to be a good balance between having enough

points of discrimination without having to maintain too many response options. This

chapter ends with a conclusion. The evaluation is conducted by the author in

cooperation with Laurens Broekhof, a Kalypso employee who was involved in the

development of original model. The author and Laurens Broekhof separately

evaluated the original Kalypso model. Afterwards we compared the assessments,

when the scores differed we discussed our consideration and reached consensus on

the score.

4.1.1 General Criteria Table 4.1 Kalypso model general criteria evaluation

1.1. The NPD portfolio management tool is usable and understandable for its users (Kester et al., 2009, Archer and Chasemzadeh, 1999,2007, Loch and Kavadias 2002) 5

1.2. The NPD portfolio management tool uses multiple types of methods and criteria (see paragraph 2.4.5) 7

1.3. The NPD portfolio management tool is able to rank projects based on pre-‐determined criteria (Archer and Chasemzadeh, 1999,2007) 5

1.4. The selection methodology is determined before the NPD portfolio selection starts (Archer and Chasemzadeh, 1999,2007) 7

1.5. The NPD portfolio management tool uses a common measure which enable an equitable comparison (Archer and Chasemzadeh, 1999,2007) 7

1.6. The NPD portfolio management tool does not solely depend on financial tools (Cooper and Kleinschmidt, 1999) 3

1.7. The NPD portfolio management tool incorporates qualitative criteria (Cooper et al. 1998, 1999) 6

Average 5.7 Poor 1 2 3 4 5 6 7 Excellent

The first criterion mentioned is that the NPD portfolio tool is usable and

understandable for its users (Kester et al., 2009, Archer and Chasemzadeh,

1999,2007, Loch and Kavadias 2002). The Kalypso model is understandable (1.1), the

input consist out of three variables, the financial value driver, the strategic fit value

driver and the risk value driver. These three variables can be rooted back to the goals

formulated by Cooper et al. (1998, 1999). There are no complex calculations involved

in calculation of the final score, hence the Kalypso model is not a black box. However


M.L.W. De Mulder 49

the transition from the Gross Profit minus Marketing Expenses to the financial value

driver (see paragraph 3.2.1.1) seems very pragmatic, how is the constant determined?

The same goes for the weighting of the different value drivers at the various

development stages, where are these based on?

There is not a single method or metric for effective NPD portfolio management. An

NPD portfolio management tool should use more than one method to compensate the

strength and weak points of each individual method or metric (1.2). The Kalypso

model uses multiple metrics, it uses a financial, risk and strategic fit metrics. The

metrics used support each other e.g. high risk can be compensated by a high financial

reward. A radical innovative project can still be valuable due to high financial

estimates despite high risks. Although multiple methods are used the emphasis,

especially in later stages, lays on the financial value driver. The weight of each value

driver differs per stage, the distribution of each value driver seems to be

pragmatically/ad hoc determined and not based on scientific research. The weighting

should at least be determined via consensus between various users of the model.

Archer and Chasemzadeh (1999, 2007) state that the NPD portfolio tool should be

able to rank projects based on pre-determined criteria. This is not the entirely case in

the Kalypso model. It is known on which criteria a project is assessed (financial,

strategic fit and risk), however, there are no minimum values set, e.g. a project must

have a minimal NPV of …, there are no thresholds formulated (1.3). The

methodology is known in advance (1.4).

Each project is assessed on the same three value drivers independent of the

development stage the project is in. These three value drivers are used to calculate a

common measure and thus enable equitable comparison (1.5). The Kalypso model

doesn’t rely solely on financial tools, it also uses risk and strategic fit. However there

is a very large emphasis on the financial tool in later stages. This could lead to an over

exposure to the biases of financial tools (see paragraph 2.3.1) (1.6).

In the early stages there is too little information available to make reliable quantitative

financial predictions. For this reason a qualitative assessment of the probability of

financial success is made. The qualitative criteria used to assess the financial value


M.L.W. De Mulder 50

driver (see table 3.3) correspond to marketing proficiency. Marketing proficiency is a

driver for new product success (Montoya-Weiss and Calantone, 1994, Henard and

Szymanski, 2001, Pattikawa et al., 2006). Cooper (1979), describes marketing

proficiency including knowledge about market size, understanding of the competition

and understanding user needs. Hence, the qualitative criteria used are appropriate for

assessing the financial value driver in the early stages in the development process.

The financial value driver in the early stages is a very rough approximation of

possible future performance, therefore the weight of the financial value driver in the

Kalypso model is low in the early stages. The qualitative approach enables the

Kalypso model to assess a project on a financial value driver despite the early stage a

project is in. The other value drivers (risk and strategic fit) are assessed entirely based

on qualitative criteria with the use of scoring models (1.7)

4.1.2 Process criteria Table 4.2 Kalypso model process criteria evaluation

2.1. The NPD portfolio method is divided in phases (Archer and Chasemzadeh, 1999,2007) 6 2.2. A project is re-‐evaluated each time it transfers to another stage (Cooper, 1990, 2008, Archer

and Chasemzadeh, 1999,2007) 7

2.3. Individual project development and evaluation is based on the Stage Gate® process (Cooper, 1990, 2008, Archer and Chasemzadeh, 1999,2007) 4

2.4. The NPD portfolio is reviewed periodically (Cooper et al., 1998, 1999, Archer and Chasemzadeh, 1999, 2007, Chao and Kavadias, 2007) 7

2.5. Each project is assessed individually on its (financial) value (Archer and Ghasemzadeh, 1999). 7

2.6. The NPD portfolio management tool takes into account the interdependencies between projects (Archer and Ghasemzadeh, 1999) 1

2.7. The NPD portfolio management tool takes into account the time dependent nature of project resource selection (Archer and Ghasemzadeh, 1999) 1


The Kalypso model is divided in stages and has some resemblance to the NPD

portfolio process presented by Archer and Chasemzadeh, (1999, 2007) (2.1). The first

step of the Kalypso model is entering project data and classify the project (range

maintenance, cost out, innovation) this matches to some degree the prescreening stage

of process presented by Archer and Chasemzadeh, (1999, 2007). In this stage projects

are also classified, in the process of Archer and Chasemzadeh (1999, 2007) this

classification is partly based on the strategy of an organization. The next stage of the

NPD portfolio process is the individual project analysis. In this stage projects are

individually assessed bases on a common set of criteria (2.5). The output of this stage

is a common set of parameter estimates for each project. Projects in the Kalypso


M.L.W. De Mulder 51

model are individually assessed on the financial value driver, strategic value driver

and the risk value driver. A project receives a score for each variable. Based on these

scores an overall fit value is calculated. Because the development stage of the project

has an influence on the weight of the different value drivers each time the project

enters a new stage the individual assessment is done over (2.2). The Kalypso model

differs with the NPD portfolio process in these later stages. The NPD portfolio

process has an additional screening stage. In the screening stage, projects that

obviously doesn’t belong in the portfolio are eliminated, hence it is a threshold. The

final portfolio selection in the NPD process is based on the set of common parameters

found at the individual project analysis. The Kalypso model produces after the

individual project assessment and calculation of a project score a variety of visual

depictions of the portfolio. This matches the last phase of the NPD portfolio process,

portfolio adjustment. What can be concluded is that the Kalypso model and the NPD

portfolio process of Archer and Chasemzadeh, (1999, 2007), share a lot of resembles.

The Kalypso model thus enables decision makers to move logically towards an

integrated consideration of projects most likely to be selected. The Kalypso model

also uses the Stage Gate® process (2.3), the stage a project is in determines the weight

of the value categories for calculating the final score. However the execution of the

Stage Gate® process is flawed, often project skip the first two stages and when they

are screened the screening is weak. The portfolio is reviewed periodically, once a

quarter (2.4). The Kalypso model doesn’t take into account interdependencies

between projects and/or the time dependent nature of projects (2.6 and 2.7).

4.1.3 Strategic criteria Table 4.3 Kalypso model strategic criteria evaluation

3.1. The NPD portfolio management tool assesses a projects on strategic fit (Cooper and Kleinschmindt, 1995, Cooper et al., 1998) 7

3.2. The strategy is determined before the NPD portfolio selection process starts (Archer and Chasemzadeh, 1999,2007) 2

3.3. Does the clear strategy consists of the who, what, how, why and threats (Markides, 1999, Loch and Tapper, 2002) or consist out of four perspectives (financial, customer perspective, internal business process and learning and growth perspective) (Kaplan and Norton, 1996)

1

3.4. The NPD portfolio management tool aligns the portfolio to the strategy (Cooper et al. 1998, 1999) 3

3.5. The NPD portfolio management tool is top-‐down and bottom-‐up (Cooper et al., 1998) 3 3.6. The strategy makes clear where and how many resources should be spent (Cooper et al.,

1998) 1

3.7. Projects are classified in advance of the portfolio selection (e.g. strategic Buckets) (Cooper et al., 1998, 1999, Archer and Chasemzadeh, 1999, 2007, Chao and Kavadias, 2007) 4

3.8. Projects are assigned to buckets (Barczak et al., 2009) 1



M.L.W. De Mulder 52

The Kalypso model pursuits the second goal of Cooper et al. (1998, 1999), the

portfolio must be aligned with the strategy. This is achieved by assessing each project

on strategic fit (3.1). A big drawback of the Kalypso model is the way in which

strategic fit is assessed. A clear strategy should be determined before the NPD

portfolio selection process starts (3.2). This is not the case in the Kalypso model, the

little strategic criteria available are too vague. Strategic fit is measured on six criteria

(see table 4.4), when a project matches a criterion the corresponding level of strategic

fit is determined (high, medium or low).

Table 4.4 Criteria for determining level of strategic fit High The idea or concept keeps the client on track with areas of performance (AOP)/strategic plan

objectives Medium • The idea or concept keeps emerging markets growing at current rate.

• The idea of concept strengthens competitive position in developed markets • The idea or concept produces a competitive "game changing" advantage • Leverages global scale

Low The project does not fit with the client’s strategic

The distribution of fit in the Kalypso model is questionable, what if a project matches

all criteria for a medium strategic fit? In this case it seems more sensible to rank a

project with a high degree of strategic fit. The transition from medium to low appears

big, a project that has a medium strategic fit receives 10 point, a project with a low

strategic fit receives 1 point. This difference is too large and leaves no room for

nuance. What if there is a project where 10 or 1 doesn’t represent the strategic fit but

should receive a value in between?. Hence the strategic fit assessment is hampered

due to the lack of a clearly formulated strategy and corresponding goals and

objectives (3.3 and 3.4). Due to the absence of a clear strategy top management isn’t

able implement (or cascade down) the strategy by setting strategic objectives and

criteria. Because of this the effectiveness of the input from the bottom is also limited

(3.5). E.g., when you assess a project on vague criteria the value of this assessment is

limited. Where money is spend mirrors the business’s strategy. It must be clear from

the strategy where and how many resources are allocated, this is currently not the case

(3.6). Projects are classified in advance however this classification has no influence

on specific spending (3.7). All projects in the Kalypso model compete for the same

resources, there is no differentiation between projects. Hence, the Kalypso model

does not use buckets (3.8). This can lead to a skewed composition of the NPD


M.L.W. De Mulder 53

portfolio. Cost out projects or range maintenance have fewer risk and have a more

predicable future performance than innovative projects. Consequently cost out and

range maintenance projects are (on short term) more interesting to develop. When all

projects are competing for the same resources there is a good chance that innovative

projects are chosen last.

4.1.4 Financial Criteria Table 4.5 Kalypso model financial criteria evaluation

4.1. The NPD portfolio management tool strives to maximize the value of the portfolio (Cooper et al., 1998, 1999) 5

4.2. The financial metric incorporates the cost of development and launch (Cooper et al., 1998) 1 4.3. The financial metric has no bias towards long term radical projects (Chao and Kavadias,

2007) 4

4.4. The financial metric captures the flexibility of the development process (Newton et al., 2004) 1

4.5. The financial metric is constrained by a critical resource (Cooper et al., 1998) 1

4.6. The financial metric ignores sunk cost (Cooper et al., 1998) 7 4.7. The financial metric incorporates the probability of success or failure (Cooper et al., 1998,

Pennings and Lint, 1997) 1

4.8. The financial metric uses the Minimum Acceptable Rate of Return (MARR) and Time Value of Money (TVM) (Remer and Nieto, 1995) 1

4.9. The financial metric should be user friendly and not too complex or appear as a black box (Faulkner, 1996, Cooper et al., 1999, Kester et al., 2009). 7


Cooper et al. (1998, 1999) propose financial tools and scoring models to maximize

the value of the NPD portfolio. The Kalypso model uses both a financial metric

(Gross Profit Margin minus Total Marketing expenses) and scoring models (for

strategic fit and risk) and thus strives to maximize the value of the NPD portfolio

(4.1). However this attempt is hampered by the financial metric used: Gross Profit

Margin minus Total Marketing Expenses (GP) generated by a product in the 2nd fiscal

year. This financial metrics has been the dominant decision metric at the client and is

for this reason used in the Kalypso model. A major drawback of the GP is that it

doesn’t take into account the investment necessary for development (4.2). Hence, it is

impossible to determine which project developers most “bang for buck”. All GP

predictions have a time horizon of 2 years. A highly innovative project needs more

time to perform well than an incremental innovative project. Is it fair to demand the

same performance development of all projects? The GP doesn’t has a total bias

towards radical projects. Radical project are often more costly to develop, the current

financial metric as mentioned before doesn’t consider the development cost which

limits the bias towards radical projects. The GP is not flexible (4.4), it does ignore


M.L.W. De Mulder 54

sunk cost (4.6), nevertheless the GP also ignores probabilities of success or failure

(4.7) plus MARR and TVM (4.8). What can be concluded is that the GP is not suited

to make NPD portfolio decisions upon. The current financial metric doesn’t meet

most of the criteria set and thus isn’t able to maximize the value of the NPD portfolio.

On the other hand is the GP not a complex metric and is easy to comprehend (4.9).

4.1.5 Balance criteria Table 4.6 Kalypso model balance criteria evaluation

5.1. The NPD portfolio tool is able to visually depict the portfolio (Cooper et al., 1998) 7

5.2. Management has defined what balance and imbalance is (Cooper et al., 1998) 1 5.3. The NPD portfolio shows only the, for management, most important diagrams (Cooper et

al., 1998) 3

5.4. The Bubble diagram’s X and Y-‐axis are based on a organization’s strategic goals and objectives Cooper et al., 1998) 4


The Kalypso model is able to visually depict the portfolio and produce various charts

(5.1). These chart provide insight into the portfolio e.g., types of projects in the

portfolio, number of project by strategic fit etc. However it is not clear what balance

or imbalance is (5.2)? Due to the lack of a definition of balance it is difficult to

determine what the most important diagrams and charts are. The Kalypso model

produces a vast amount (23) of charts and diagrams, this increases the chance of

information overload (5.3). Because of the lack of a clear strategy and corresponding

goals, the Kalypso model can’t depict the portfolio based on these goals and

objectives (however if there were a clear strategy available the Kalypso model should

be able to do so).

4.1.6 Risk Criteria Table 4.7 Kalypso model risk criteria assessment

6.1. The NPD portfolio management tool assesses projects on risk (Halman and Keizer, 1994. Keizer et al., 2005 5

6.2. Risk is assessed on three dimensions; likelihood, impact, and the ability to do something about it (Halman and Keizer, 1994) 1

6.3. Risk is assessed on a sufficient number of criteria (Keizer et al., 2005) 3


Risk is assessed in the Kalypso model (6.1). Risk is assessed on three subcategories;

technical, commercial and regulatory risk. The category with the highest risk

determines the overall risk of the project. However the risk assessment of the Kalypso

model is limited compared to the risk reference framework proposed by Keizer et al.


M.L.W. De Mulder 55

(2005) where risk is assessed on 12 subcategories. The weight of the risk value driver

differs per stage the project is in. If a project is in stage 1 or 2 the risk score is

weighted at 10% and for later stages 30%, but according to Sommer et al, 2007 risk

decreases during the development process, hence the distribution of weight doesn’t

make much sense. Risk should be assessed on three dimensions; (1) likelihood, (2)

impact and (3) the ability to do something about it. Currently risk is only assessed on

occurrence and impact trough qualitative criteria linked to a 5-point risk scale. The

Kalypso model doesn’t use the third criteria proposed by Halman and Keizer (1994),

control (6.2). High risk isn’t necessary bad; it depends on the reward. In the Kalypso

model this link isn’t very explicit. Furthermore the number of criteria used is very

limited compared to the reference framework developed by Keizer et al. (2005) (6.3).

4.1.7 Usability Criteria Table 4.8 Kalypso model usability criteria evaluation

7.1. Governance roles and responsibilities are defined (Cooper, 2008) 6

7.2. The NPD portfolio is periodically assessed (Cooper, 2008) 7 7.3. The NPD portfolio management method is user friendly and not too complex or appears as a

black box (Faulkner, 1996, Cooper et al., 1999, Kester et al., 2009) 4

7.4. Deliverables are clearly defined for each portfolio assessment (Cooper, 2008) 6 7.5. Deliverables are focused on the essential information needed for NPD portfolio

management decisions (Cooper, 2008) 3

7.6. Templates are not overly detailed and be limited to the essential information needed for NPD portfolio management decisions (Archer and Ghazemzadeh, 1998, Cooper, 2008) 5


In the Kalypso model it is clear who is responsible for what (7.1). There are a many

people at different levels of the organization involved in the NPD portfolio

management process. This makes the execution of the Kalypso model difficult

especially for high-level employees who often have a full schedule. This can be

overcome by conducting the NPD portfolio management process periodically. In the

Kalypso model, Stage Gate® (or product management review) reviews are done

monthly and NPD portfolio reviews are on a quarterly basis (7.2). The Kalypso model

is a user-friendly model, it is not complex and does not require huge amounts of data,

and consequently does not appear to be a black box. However, the weighting of the

various value driver and the constant in the calculation of the financial value driver

(from stage 3 onwards, see paragraph 3.2.2) seems to be very pragmatic and ad hoc

(7.3). Because the project assessment methods and criteria are known in advance it is

also clear what (data) has to be delivered for each NPD portfolio assessment (7.4).


M.L.W. De Mulder 56

However it is not clear what the essential information is due to the lack of a clear

strategy (7.5). This could lead to templates which are overly detailed although this

seems not the case with Kalypso model. As mentioned before, the data requirements

of the Kalypso model are limited but lack purpose due to the absence of a clear

strategy (7.6).

4.2 Conclusion The Kalypso model has been assessed on various criteria in seven categories. Table

4.9 shows the average score the Kalypso model received in each category.

Table 4.9 Average scores Kalypso model per category

1. General Criteria 5.7

2. Usability Criteria 5.2

3. Process Criteria 4.7

4. Balance Criteria 3.8

6. Financial Criteria 3.1

5. Risk Criteria 3.0

7. Strategic Criteria 2.8

Overall score 4.0 Poor 1 2 3 4 5 6 7 Excellent

The Kalypso model received an overall score of 4.0, this score is the average score of

the category scores. The Kalypso model receives a higher score than the alternative

models described in paragraph 2.4. What becomes clear is that there is an opportunity

for improvement. Table 4.9 provides insight in the strong and weak points of the

Kalypso model (appendix 5 gives and overview of Kalypso model performance per

criteria ranked by score). The Kalypso model scores well on general, usability and

process criteria. E.g. the Kalypso model uses multiple types of methods for valuating

projects (1.2). However, the Kalypso model fails in the details. The assessment of the

various value drivers is flawed. Due to the lack of clear strategy accompanied by

well-defined goals and objectives it is very difficult to determine the strategic fit of

individual projects (3.3). Because of the absence of strategic buckets it is very

problematic to top-down implement a strategy while totally different projects are

competing for the same resources (3.4, 3.6). The quantitative financial metric is not

suited for NPD portfolio management decisions. In addition, many projects have

failed because less frequently occurring risks were not taken seriously (Keizer et al.,

2005). The risk assessment is not sufficient, projects are assessed on too few risks


M.L.W. De Mulder 57

(6.3). The risks that are reviewed are not assessed on the three dimensions of risk

(6.2). Due to the absence of a clearly formulated strategy the graphs and charts lack

purpose, 23 charts are just too many (5.3). In addition the lack of a clear definition of

what balance is also hampers the value of the charts (5.2).

The basis of the Kalypso model is good, the model uses relevant variables for

determining the value of a project. The model also uses multiple methods for

assessing the value of projects. However the model lacks detail and rigor in the

assessment these individual value drivers. Because of this, the Kalypso model

produces a sub-optimal portfolio.

In this chapter the Kalypso model was evaluated, many strong and weak points were

identified. There is a great opportunity for improvement and improvement will be the

main focus of the next chapter. In chapter 5 a modified Kalypso model will be

presented, this model continues to build on the strong points and tries to overcome the

weak points of the original Kalypso model.


M.L.W. De Mulder 58

Chapter 5: Improving the Original Kalypso model In chapter 4 various strong and weak points of the original Kalypso model were

identified. These points were found by assessing the Kalypso model on the criteria

listed in chapter 2 (see table 2.9). In this chapter, an improved Kalypso model will be

presented; the Kalypso model v2. Table 4.9 provides insight in what the weakest

aspects of the original Kalypso model are. The modifications of the Kalypso model

are prioritized on the results shown in table 4.9. Scientific literature and experts were

consulted for the development of the Kalypso model v2. During expert discussions

with, Mick Broekhof, Laurens Broekhof from Kalypso and Linda Kester of the TU

Delft, valuable input was attainted. We will first focus on the strategic criteria,

thereafter risk, financial, balance, process, usability and last general criteria. Various

references to the Stage Gate® process are made in this chapter. Figure 5.1 shows the

Stage Gate® process used for the Kalypso model v2.

Figure 5.1 Stage Gate® process by Cooper

Source: Cooper (2008), page 215

5.1.1 Strategic criteria The Kalypso model received the lowest score on the strategic criteria assessment. The

Kalypso model failed to capture the strategy and consequently failed to assess

projects on their fit with the strategy (3.3). This is partly due to the fact that the client

doesn’t have a clearly defined strategy with corresponding goals and objectives.

Another big drawback of the strategic assessment was the absence of a statement of

where and how many resources should be spent (3.6), no strategic buckets were

defined (3.8).

We will start with inability of the original Kalypso model to effectively assess the fit

of a project with an organization’s strategy. You can only assess a project on strategic

fit if there is a clear strategy present. The strategy must originate in the top of the

organization (Kaplan and Norton, 1996, Markides, 1999, Loch and Tapper, 2002).

Stage 0 Discovery

Stage 1Scoping

Stage 2Build business

caseStage 3

DevelopmentStage 4

Testing & validation

Stage 5Launch

Gate1

Idea Screen

Gate2

Second Screen

Gate3

Go To Development

Gate4

Go To Testing

Gate5

Go To Launch


M.L.W. De Mulder 59

The CEO or president of an organization is responsible for the formulation of the

organization’s strategy. This responsibility is already clearly defined in the original

Kalypso model (see paragraph 3.3). However, where a strategy consists of must be

defined more clearly in the modified model. The Kalypso model v2 must make clear

what the deliverables for a strategy are. It is important that the strategy is clear and

explicit (Markides, 1999). Objectives and goals must be derived from the strategy in

order to assess projects on strategic fit as shown in figure 5.2.

Figure 5.2 Strategy development

An organization has to define when a goal is attained by setting various criteria. Every

organization has its own unique strategy, consequently each organization has to

define its own goals, objectives and corresponding conditions and criteria.

Subsequently the Kalypso model v2 cannot present a list of reference criteria and

objectives. As described in chapter 2 a good strategy must be able to answer the

following questions:



customers?



advantage?


Based on these questions organizations should be able to formulate numerous

strategic criteria. Andrews (1995) states that one critical requirement for the

successful implementation of a strategy is to ensure that decisions made by managers

Strategy

Goal 1

Objective 1.1 Objective 1.2

Criteria 1.1.1 Criteria 1.1.2 Criteria 1.2.1 Criteria 1.2.2

Goal 2



Goal 3



Broader

Narrower


M.L.W. De Mulder 60

and senior managers are consistent with the organization’s goals and objectives. The

strategy must identify tradeoffs and priorities among conflicting goals (Loch and

Tapper, 2002). An effective way to prioritize goals is by assigning resources to each

goal. This brings us to the next improvement of the original Kalypso model.

The second (strategic) limitation of the original Kalypso model is the inability to

focus resources. The original Kalypso model doesn’t use strategic buckets. As

described in chapter 2, a strategic bucket is a collection of NPD projects that are

aligned with a particular innovation strategy or strategic objective (Chao and

Kavadias, 2007). These innovation strategies and objectives are the result of the

strategy development described in the previous part. The goal of the strategic buckets

method is to translate the organization’s strategy into clearly defined arenas (buckets)

and to dedicate (a finite amount of) resources to each (Cooper et al, 1998).

Management determines the amount of resources assigned to each bucket. Cooper et

al. (1998) identified various considerations for the development of strategic buckets,

which are shown in table 5.1.

Table 5.1 Various consideration for the development of strategic buckets

Strategic goals Management spilt resources across the specific strategic goals Product lines Resources are spilt across product lines. Market segments The business may operate in several different market segments, with varying

degrees of attractiveness and potential for the future. Hence management splits resources across market segments

Technology types Resources could be split across technologies, technology types or platforms Project types What percent of resources should go to new product development? To platform

development? To process improvement? Etc. Geography Resources could be split based on geography. E.g. what proportion goes to Europe,

North America, Asia, etc.? Source: Cooper et al. (1998), page 87-‐89

Based on, for an organization relevant considerations, various strategic buckets

should be defined. As with the strategy, the buckets are cascaded down. If necessary

lower level management could further specify or split buckets (this depends on the

autonomy of lower level management). Figure 5.3 illustrates the notion of cascading

down the strategic buckets.


M.L.W. De Mulder 61

Figure 5.3 Example of cascading down strategic Buckets

A (global) strategy is formulated by top management. In this example three goals are

formulated. For each goal various criteria and preconditions are formulated, which

give insight in how and when the goal is obtained. The total available resources are

divided into three buckets (one for every goal). Each goal mirrors a specific part of

the (global) strategy. Regions A and B are involved in attaining goal III. Hence the

bucket for goal III must be divided into buckets 3.1 and 3.2 (one for each region).

This division is done at the highest organizational levels. Bucket 3.2 is cascaded down

to the management of region B. There are three technologies necessary for achieving

goal III. The selection of these technologies is based on the initial criteria and

preconditions set by top management. Consequently, the management of region B

must divide the resources of bucket 3.2 into three smaller buckets; 3.2.1, 3.2.2, 3.2.3.

They must define (additional) conditions for each bucket. Each technology bucket can

be filled with individual projects based on the assessment on the conditions

formulated by regional management and by local technology management. The

criteria for assigning, markets, technologies, projects, etc. become more specific with

each lower level. Exhibit 5.1 shows the process of strategic buckets

Goal I --- Bucket 1Goal II --- Bucket 2Goal III --- Bucket 3Region A --- Bucket 3.1

Region B --- Bucket 3.2

Technology X---

Bucket 3.2.1

Technology Y---

Bucket 3.2.2

Technology Z---

Bucket 3.2.3

Proj

ect 1

Proj

ect 2

Proj

ect 3

Proj

ect 4

Proj

ect 5

Proj

ect 6

Proj

ect 7

Proj

ect 8

Proj

ect 8

Dev

elop

ed G

loba

lly

Dev

elop

ed R

egio

nally

Dev

elop

ed L

ocal

ly

Dev

elop

ed L

ocal

ly

Dev

elop

ed L

ocal

ly

Company Strategy


M.L.W. De Mulder 62

Exhibit 5.1 Process of strategic buckets

x ∈ (Top management level,.., Project management level)

*Project or bucket depends on the management level

Project interdependencies should be assessed during the strategic fit assessment.

Verma and Sihna (2002) classified three kinds of interdependencies; resource

interdependencies, technology interdependencies, and market interdependencies. The

resource allocation for each project is inversely related to resources for other

concurrent projects; an increase in the resource level for one project would lead to a

decrease in the resource level of another project. Technology interdependencies result

from leveraging common technology across multiple projects (Verma and Sihna,

2002). Market interdependencies, according to Verma and Sihna (2002), stem from

(1) a new product’s diffusion into an already existing product market and (2) utilizing

a current product’s market knowledge (e.g. how to manage a dealer network) for

development of a new product for an entirely different market. Therefore, both

technology and market interdependencies are what Thompson (1967) refers to as

“sequential interdependencies”: outcome of a project affects the resource allocation of

other projects that follow. Especially technology market interdependencies are of

strategic importance. Which technologies to uses and which markets to target are

strategic considerations. Consequently, technology market interdependencies should

be assessed during strategic fit assessments. E.g. “does project x built on the

technology of project y?” Technology interdependencies could be basis for assigning

project to a specific bucket (see table 5.1).

A scoring method is the most effective way to assess strategic fit due to the qualitative

nature of variable (Cooper and Kleinschmidt, 1995, Cooper et al., 1998). In order to

determine the strategic fit of a project it must be assessed on various criteria. These

Define BucketsManagement level x

Define Criteria

Project/Buckets*Management level x-1

Fill Bucket!"#$%&'(

)*++$%&,-

.$+$/0$%&,-

Strategy

!"#$%&'(!"#$%&'(


M.L.W. De Mulder 63

criteria should be directly linked to the strategy and must address the various facets of

a strategy. Often a strategy consists of several goals, every goal is pursued with

different projects. Hence each goal has its own criteria for the strategic fit assessment

of projects. Each criterion is ranked on a seven-point scale (1 (poor) to 7 (excellent)).

The overall strategic fit score is calculated by:

!"# =S!!!"

!!

! ∗ 7 ∗ 100

Where: SCF = Strategic Fit Score ci = Strategic criteria i ∈ (1,2,..,N) N = Number of criteria used Sci = Received score (1 to 7) on strategic criteria i

Figure 5.4 depicts the strategy process in the Kalypso model v2. The strategy is

developed by answering the who, what, how, why and threats proposed by Markides

(1999). Based on the strategy clear goals are formulated accompanied by well-defined

criteria. Resources are dedicated to each goal (buckets are defined), the amount of

resources is based on the strategy. Projects are appointed to one bucket based on the

criteria formulated for each goal. Projects in each bucket are ranked (and prioritized)

based on strategic fit, risk and financial. The strategic fit assessment is based on the

criteria defined for each goal. A detailed description of the project ranking process

can be found in paragraph 5.4.

Figure 5.4 Strategy process

*Not part of the strategy process

Develop strategy

Define Goals

Define criteria for each goal

Dedicate resources to each goal (define buckets)

Appoint projects to each bucket

Based on

Projects

Rank projects*

Based on

Risk*

Financial*Based on

Based on

Based on

Who,What, How, Why and Threats?Based On


M.L.W. De Mulder 64

5.1.2 Risk criteria The risk assessment is the second weakest aspect of the original Kalypso model.

Especially the number of risks that are assessed and the manner or depth of this

assessment is lacking. Risk should be assessed on three dimensions (occurrence,

impact and control) instead of two (occurrence and impact) (Halman and Keizer,

1994). In the Kalypso model v2 the assessment of risk is done more rigorously.

According to Keizer et al. (2002) a risk assessment approach would:

• Evaluate each potential risk on its likelihood, its controllability and its relative

importance to project performance (the three dimensions).

• Take a cross-functional perspective by identifying and evaluating technological,

market, financial and operational risks.

• Conduct the risk assessment at the end of the feasibility phase (stage 2) and

periodically reassess the project for unforeseen risks and deviations from the risk

management plan.

• Identify and evaluate the product innovation risks individually and generate,

evaluate and select alternative solutions in subgroups and plenary sessions.

The Kalypso model v2 risk assessment will be based on the Risk Diagnosing

Methodology (RDM) developed by Keizer et al. (2002) which aims to identify

various risks. RDM consists of three stages with various steps that are shown in figure

5.5.

Figure 5.5 Outline of risk diagnosing methodology

Source: Keizer et al. (2002), page 216

Risk IdentificationStep 1: Initial briefing between project manager and risk facilitatorStep 2: Kick-Off meeting: project manager & team and risk facilitatorStep 3: Individual interviewing of participants by risk facilitator

Risk AssessmentStep 4: Development of risk questionnaire by risk facilitatorStep 5: Answering of risk questionnaire by participantsStep 6: Constricting of risk profile by risk facilitator

Risk Response Development and ControlStep 7: Preparing of risk management session by project manager & risk facilitator Step 8: Risk management session: project manager & team and risk facilitator Step 9: Drawing up & execution of risk management plan


M.L.W. De Mulder 65

The most relevant part of RDM for the Kalypso model v2 is the risk assessment

because it covers the actual assessment of the various risks. The risk identification

and risk response development and control stages should be done before and after the

NPD portfolio selection process (see paragraph 5.1.5). However these stages are

important for the success of RDM.

The numbers of risks used in the original Kalypso model are not sufficient, a too

limited number of risks are assessed (see paragraph 4.1.6). Appendix 3 lists the

various risks identified by Keizer et al. (2005). These risks are based on risks found in

literature (e.g. the meta-study by Montoya-Weiss and Calantone (1994) of success

factors for new product success) and case interviews. These risks are generic. A

thorough risk assessment must also include context specific risks. Most organizations

have their own shortlist of critical mishaps that management hopes future projects

will escape (Keizer et al., 2002). Therefore, the risks used for the assessment of risk in

the Kalypso model v2 should be based on both the risk reference framework and

context specific risks found at the organization. This approach should help team

members to think of less obvious issues (Keizer et al., 2002). In addition,

interdependencies should be assessed during the risk assessment. E.g., “abandoning

this project has no influence on other projects in the portfolio” can be asked.

The risks identified are assessed on three dimensions: (1) likelihood, (2) impact and

(3) the ability to influence the outcome. Each risk must be positively framed because

Kahneman and Tversky (1979) found that negative framing of risks induces more

positive perceptions opposed to positive framing. People who are confronted with

negative framing tend to think: “Its not that bad”. While confronted with positive

framing people tend to respond: “They may suppose so, but I am not so sure”.

Positive framing leads to the most critical assessment, therefore all risks in the

Kalypso model v2 should be framed positively. The typical number of risks in a risk

assessment is, according to Keizer et al. (2002), 50-60 and should take 45-60 minutes

to complete. Exhibit 5.2 shows an example of a risk assessment (for two risks).


M.L.W. De Mulder 66

Exhibit 5.2 Example of risk questionnaire (for two risks) What is the level of

certainty that the statement will be true? (likelihood)

Ability of the team to influence course of actions within time &resource limits (ability to influence the outcome)

Relative importance of statement for obtaining project success (impact)

Very Low

Very High

Very Low

Very High

Very High

Very Low

Risk statements: 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1. The new product will be safe to

use for people with sensitive skin.

. ✔ . . . . ✔ . . . . ✔ . . .

2. With the trade customer clear after sales arrangements have been agreed.

. . . ✔ . . ✔ . . . . . . . ✔


All project team members should preferably conduct the risk assessment. Each risk

receives for each dimension a score in range of 1 to 5. After each team member has

conducted his or her assessment a risk profile is made. This risk profile presents both

the degrees of risk perceived by the majority of the respondents and the distribution of

their perceptions (Keizer et al., 2002). Exhibit 5.3 shows a risk profile for two risks.

Exhibit 5.3 Example of a risk profile (for two risks) What is the level of certainty

that the statement will be true?

Ability of the team to influence course of actions within time &resource limits

Relative importance of statement for obtaining project success

Score for each dimension of risk

Risk class

Very Low

Very High Very Low

Very High

Very High

Very Low

Risk statements: 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 C A I 1. The new product will be safe to use for people with sensitive skin.

N. resp. 0 0 1 3 5 0 0 1 3 5 0 2 5 2 0

0 0 m L Cum % 0 0 11 44 100 0 0 11 44 100 0 22 78 100 100

50% � � �

2. With the trade customer clear after sales arrangements have been agreed.

N. resp. 3 2 4 0 0 0 2 5 2 0 3 5 1 0 0

* m * H Cum % 33 56 100 100 100 0 22 78 100 100 33 89 100 100 100

50% � � �

Decision rules: N resp.: Number of team members who scored in certain

column “*”: At least 50 % of scores are 1 or 2 on the five point

scale and there are no scores of 5 Cum %: Cumulative percentage of team respondents “0”: At least 50 % of scores are 4 or 5 on the five point

scale and there are no scores of 1 50%: Column in which at least 50 response is first met “m”: At least 50 % of scores are 3 on the five point

scale and there are no scores of 1 or 5 “?”: Lack of consensus: there is a wide distribution of

opinions Risk Classes used: S = Safe; L = Low risk; M = Medium Risk; H = High Risk; F = Fatal Risk



M.L.W. De Mulder 67

Every risk is classified along three dimensions into four groups (“*”, “0”, “m” and

“?”) by the decision rules shown in exhibit 5.3. Each risk is classified next, RDM uses

five risks classes; S = Safe; L = Low risk; M = Medium Risk; H = High Risk and F =

Fatal Risk. This classification is based in the combination of scores (e.g. “*,*,*”) for

each risk. There are 64 combinations, each combination has its own risk class (shown

in appendix 6). It is possible that, due to a lack of consensus, multiple risk classes

apply for a single risk. A combination of classes means that the risk team should work

out whether the disagreement can be resolved and a single classification can be

achieved. If consensus can’t be achieved the worst possible case should be assumed.

After each risk is assessed and classified an overall risk score must be attained. This is

different from the RDM where no overall score is calculated. This is because a project

can fail because of one risk, consequently, risks shouldn’t be put on a heap. However

in order to compare different projects on risk, one score must be determined in the

Kalypso model v2. Not all risks are equal, one fatal risk should have a larger impact

on the risk score than a medium risk. What is acceptable differs per organization and

should be determined by the each individual organization. The strategy should be

used as a guide for what acceptable risk levels are. E.g., for some parts of the strategy

(specific buckets) a higher risk is acceptable than in other parts of the strategy. The

risk process described in this paragraph is shown in figure 5.6.

Figure 5.6 Risk process

*Not part of the risk process

Rank projects*

Strategy*

Financial*Based on

Based on

Individual risk assessment

Generic risks(Risk reference list) Context specific risks

Relevant riskBased on

Risk score Based on

Project team members

Identified by

Conducted byRisk facilitator (NPD process

manager)

Determined by

Determined by


M.L.W. De Mulder 68

5.1.3 Financial criteria Paragraph 4.1.4 gave insight in the strong and weak points of the original Kalypso

model regarding its financial assessment. Paragraph 4.1.4 made clear that the current

financial metric is not suited to base NPD portfolio decisions upon. In this paragraph

a new financial metric will be presented.

In the Kalypso model v2 the Expected Commercial Value (ECV) will be used. The

ECV overcomes most of the limitations of the current quantitative financial metric.

As described in chapter 2.3.1.1, the ECV seeks to maximize the value or commercial

worth of the portfolio.

Exhibit 5.4 Determination of expected commercial value of project

!"# = !"# ∗ !!" − ! ∗ !!" − !

$ECV= Pts= Pcs= $D= $C= $NPV=

Expected Commercial Value of the project Probability of technical success (range 0.0 -‐ 1.0) Probability of commercial success (given technical success) (range 0.0 -‐ 1.0) Development cost remaining Commercialization (Launch cost) Net present value of the project’s future earnings (discounted to the present)


The calculation of the ECV is based on the development costs, launch costs, Net

Present Value (NPV) and the probability of technical and commercial success. The

NPV represents the future earnings of a project where all cash flows are moved to the

present. The NPV is calculated with the following formula:

!"# =!"#!(1+ !)!

!

!!!

Where: NPV = Net present value NCF = Net cash flow of year t ∈ (1,2,..,n) i = time value of money or minimum attractive rate of return (MARR)

Development $D

Launch$C

TechnicalSucces

CommercialSucces $NVP

Commercial Failure

Technical Failure

Pts

Pcs

Yes

No

Yes

No

$ECV


M.L.W. De Mulder 69

The Minimum Attractive Rate of Return (MARR) is the minimum rate of return on a

project a manager or organization is willing to accept before starting a project, given

its risk and the opportunity cost of forgoing other projects (Park, 2007). Hence this

rate is different for each organization and should be based on an organization’s

strategy. Criterion 4.5 (2.3.6) notes that the financial metric should be constrained by

a critical resource (e.g. development cost). Constraining the ECV by a critical

resource requirement enables users to get insight in which projects deliver most “bang

for buck”. Hence the calculations:

!"#$#%"$& !"#$% =!"#

!"#$%&'(#% !"#$%!&" =!"# ∗ !!" − ! ∗ !!" − !!"#$%&'i!" !"#o!"#$

Where: ECV = Expected Commercial Value of the project NPV = Net present value Pts = Probability of technical success (range 0.0 -‐ 1.0) Pcs = Probability of commercial success (given technical success) (range 0.0 -‐ 1.0) D = Development cost remaining C = Commercialization (Launch cost)

The variables Pts (probability of technical success) and Pcs (probability of commercial

success) should be determined with qualitative and established criteria (Cooper et al.,

1998). Keizer et al. (2005) developed a risk reference framework (shown in appendix

3). Some of the risk identified by Keizer et al. (2005) could also be used for the

assessment of these two variables. Hence, if there is a high risk there will be a low

probability of success and visa versa. Not all risks are relevant for each stage of the

development process. E.g. it doesn’t make much sense to assess a project in the first

stage of the development process on product format meets functional requirement

because there is no reliable information about this specific risk. Consequently each

risk is best suited from a specific development stage onward. Each organization has to

develop its own list of relevant technical and commercial risks for each stage of the

development process. The risks identified (especially, product technology risks and

manufacturing technology risks categories for Pts, and consumer acceptance and

marketing risks and commercial viability risks categories for Pcs) in the risk reference

framework (appendix 3) are a good starting point for developing a list of relevant

risks for each variable. Risk should be assessed only once for each development stage.

This means that the risks used to determine Pts and Pcs are assessed during the risk

assessment (described in 5.1.2). The transformation of project risk scores (one for


M.L.W. De Mulder 70

technical and one for commercial success) into probabilities should be based on a

predetermined distribution (e.g., table 5.2). Each organization should determine its

own distribution.

Table 5.2 Example of distribution of risk Risk class Probability of success

S 90% L 70% M 50% H 30% F 10%

One strong aspect of the original Kalypso model is that it endorsed the notion that

quantitative financial metrics are not reliable in the early stages of the development

process. Consequently, the original Kalypso model assessed the financial metric in the

early stages with a qualitative assessment. The Kalypso model v2 will continue with

this approach. In stages 0 and 1 is the financial score solely based on qualitative

criteria. These criteria are similar to the criteria used to determine the probabilities of

technical and commercial success. Each organization has to compose its own list of

risks that are relevant for financial success. Again these risks should be assessed in

the same manner as described in paragraph 5.1.2. However at gates 1 or 2 the

classification is not transformed into a probability but into a financial score.

There are two ways for obtaining a financial score, at gates 1 and 2 a qualitative one

and at the later gates a (partly) quantitative one. Each project receives a score of 1 up

to a 100. If a project is at gates 1 or 2 the score depends on the risk profile obtained

via the assessment of relevant risk for financial success. Again, each organization

should determine its own distribution of scores depending on this assessment. This

distribution could differ per bucket. If a project is at gates 3, 4, and 5, the financial

score is determined via a quantitative metric. In the Kalypso model v2 financial score

computation differs from the formula suggest by Cooper et al. (1998) shown on the

previous page.


M.L.W. De Mulder 71

!"#$#%"$& !"#$% = 1 −!"#$%&'(#% !"#$%!&"

!"#∗ 100

= 1 −

Constrained resource!"# ∗ !!" − ! ∗ !!" − !

∗ 100

The formula is rewritten because this enables a better comparison between the

financial assessments of project at gates 1or 2 with later gates. Figure 5.7 depicts the

financial process as described in this paragraph.

Figure 5.7 Financial process

*Not part of the financial process

Rank projects*

Strategy*

Risk*Based on

Individual financial assessment

Technical risks

Financial score Based on

Gates 1 or 2 Gates 3,4 or 5

Commercial risks

Probability of technical success

Probability of commercial success

NPV

ECV Development cost

Launch cost

Technical risks

Commercial risks

Additional risks

Based on

Based on

Distribution of score per risks

Contrained resource

Individual risk assessment*


M.L.W. De Mulder 72

5.1.4 Balance criteria The original Kalypso model is able to depict numerous graphical representations of

the NPD portfolio based on various criteria (criteria 5.1). This leaves little room for

improvement. What does need improvement is the definition of what balance is and

what the most important metrics for an organization are. In the Kalypso model v2 the

number of visual depictions will be limited, consequently, an organization has to

choose. Cooper et al. (1998) found that most organizations where able to reduce the

number of (relevant) diagrams to three. However, this was refuted during expert

discussions. It was stated that limiting the number of depictions was not realistic. The

information needs are different for each level of the organization. A CEO has

different information needs for instance than a regional manager. Consequently, the

number of depictions that the Kalypso model v2 produces will depend on the

organizational level of its end users. Every organizational level has to determine the

number of relevant depictions. The risk that too many depictions are chosen is, from

experience, low. Every organization has to define what balance is. The strategy of an

organization is an important aid for determining what balance is and which diagrams

should be depicted.

Figure 5.7 Balance process

*Not part of the balance process

Definition of Balance

Rank projects*

Risk*

Financial*

Based on Strategy*

Based on

Based on

X amount of Graphical depictions

Based on

Organizational level of end users

amount depends on


M.L.W. De Mulder 73

5.1.5 Process criteria The Kalypso scored well on many of the process criteria (see paragraph 4.1.2).

Consequently, the Kalypso model v2 will have a lot of similarities with current

model. The process of the Kalypso model v2 will be based on the NPD portfolio

process developed by Archer and Chasemzadeh (1999, 2007). The process of the

Kalypso model v2 is shown in figure 5.8 on the next page. In the new model various

aspects have to be determined before the process starts. The most important pre-

process activity is the determination of a clear strategy and corresponding goals. In

addition, resources must be dedicated to each goal by creating buckets. Another pre-

process activity is defining what balance is using the strategy as starting point.

In the next stage, prescreening, new NPD projects enter the process. These projects

are in the first stage of the development process. In this stage, each project is assigned

to a specific bucket. This assignment is based on criteria formulated for each goal

(bucket) (see paragraph 5.1.1). For each project a list of potential risk is prepared, this

list is based on the risks identified in the risk reference framework and context

specific risks identified by project members as described in paragraph 5.1.2.

In the next stage projects are assessed individually. Each project is assessed on risk,

strategic fit and financial criteria (for detailed descriptions see PARAGRAH 5.1.1,

5.1.2 and 5.1.3). Using the Risk Diagnosing Methodology (RDM) every project is

assessed on relevant risks defined in the previous stage (prescreening). During the

strategic assessment projects are assessed on the strategic criteria formulated during

the strategy development. This assessment is done with a scoring model where each

project is ranked on a 7-point scale (see paragraph 5.1.1). The financial assessment

depends on the development stage a project is in. If a project is in stage 0 or 1 the

individual assessment is based on qualitative criteria. These criteria are based on

relevant risks. A project is assessed on technical, commercial and (if necessary)

additional risks. The RDM is used for the assessment of these risks. If a project is in

stages 2,3 or 4 a more quantitative assessment is done. The NPV, development and

launch costs are estimated for each project.


M.L.W. De Mulder 74

Figure 5.8 Kalypso model v2 process diagram

Pres

cree

ning

Port

folio

A

djus

tmen

tPo

rtfo

lio S

elec

tion

Scre

enin

gIn

divi

dual

Pro

ject

Ana

lysi

sPr

e Pr

oces

s A

ctiv

ity

Deve

lop

stra

tegy

Defin

e G

oals

Defin

e cr

iteria

for e

ach

goal

Dedi

cate

reso

urce

s to

eac

h go

al (d

efine

buc

kets

)

Who

,Wha

t, H

ow, W

hy a

nd

Thre

ats?

Defin

ition

of B

alan

ce

Appo

int p

roje

cts

to e

ach

buck

et

Proj

ects

Gen

eric

risks

(Risk

refe

renc

e lis

t)

Cont

ext s

pecifi

c ris

ks

Rele

vant

risk

Indi

vidua

l risk

ass

essm

ent

Indi

vidua

l stra

tegi

c as

sess

men

t

Indi

vidua

l fina

ncia

l as

sess

men

t

Risk

sco

re

(1-1

00)

Fina

ncia

l sco

re

(1-1

00)

Stra

tegi

c sc

ore

(1-1

00)

Stag

e 0

or 1

Tech

nica

l risk

s

Com

mer

cial

risks

Addi

tiona

l ris

ks

Dist

ribut

ion

of

scor

e pe

r risk

s

ECV

Cont

rain

ted

reso

urce

NPV

Deve

lopm

ent

cost

Laun

ch c

ost

Tech

nica

l risk

sCo

mm

ercia

l ris

ks

Prob

abilit

y of

te

chni

cal s

ucce

ssPr

obab

ility

of

com

mer

cial s

ucce

ss

or

Dist

ribut

ion

of

scor

e pe

r risk

s

NPD

Portf

olio

sco

re

GAT

E

Stag

e

Rank

ed P

roje

cts

Gra

phica

lde

pict

ions

Proj

ect

Dev

elop

men

tPr

ojec

tD

atab

ase

Risk

Dia

gnos

ing

Met

hodo

logy

or

Stag

e 0

or 1

Stag

e 2,

3 o

r 4

Base

d on

Base

d on

Base

d on Base

d on

Base

d on

Base

d on

Base

d on

Base

d on

GAT

EC

riter

ia

depe

nd o

n

Port

folio

C

ompl

etio

n

!"#$%"&'()*"+,-.

"'-/'0#,"

In

!""!

!"#$#%"$&!!"#$%

!

!!!"#$%&'(#%!!"#$%!&"

!"#

!!""

!

!!

!"#$%&'(#%!!"#$%!&"

!"#

!! !"!!!! !"!!

!!""

The

form

ula

is r

ewrit

ten

beca

use

this

ena

bles

a b

ette

r co

mpa

rison

bet

wee

n th

e

finan

cial

ass

essm

ents

of

proj

ect

at g

ates

1or

2 o

r la

ter

gate

s. F

igur

e X

dep

icts

the

finan

cial

pro

cess

as

desc

ribed

in th

is p

arag

raph

.

!"#$

%&'('!")*)

+"*,'-%.+&//'

!

!! !"

#$%&'($%#

)%$*+%),-'

-.,'/%&(#.+00%

Ran

k pr

ojec

ts*

Stra

tegy

*

Ris

k*Ba

sed

on

Indi

vidu

al fi

nanc

ial

asse

ssm

ent

Tech

nica

l ris

ks

Fina

ncia

l sco

reBa

sed

on

Gat

es 1

or 2

Gat

es 3

,4 o

r 5 Com

mer

cial

ris

ks

Prob

abilit

y of

te

chni

cal s

ucce

ssPr

obab

ility

of

com

mer

cial

suc

cess

NPV

ECV

Dev

elop

men

t co

st

Laun

ch c

ost

Tech

nica

l ris

ks

Com

mer

cial

ris

ks

Addi

tiona

l ris

ks

Base

d on

Base

d on

Dis

tribu

tion

of

scor

e pe

r ris

ks

Con

train

t re

sour

ce

Indi

vidu

al ri

sk a

sses

smen

t*

Org

aniza

tiona

l leve

l of

end

use

rs

Dep

end

on


M.L.W. De Mulder 75

The probabilities of technical and commercial success have an influence on the

(financial) value of a project. These probabilities are determined via the assessment of

technical and commercial risk (see paragraph 5.1.3).

After the individual project analysis it is time for the project screening. In this phase a

project receives a score on risk, strategic fit and financial based on the previous stage.

The risk score depends on individual risk assessment and the predetermined

distribution of score per risk (what risks are acceptable differ per organization or even

projects, depending on the strategy). A strategic fit score is calculated based on the

individual strategic assessment (see paragraph 5.1.1). The determination of the

financial score depends on the stage the project is in. If a project is in one the first two

stages the financial score is based on a predefined distribution of score per risks

(found in the previous stage). If a project is in stage 2, 3 or 4, the financial score is

calculated with the formula on page 71. Every project receives a score of 1 to 100 for

each category.

After the screening phase the projects reach a gate (part of the Stage Gate process), at

this gate projects are continued or abandoned. This selection process should be

conducted on predetermined criteria (see paragraph 2.2 and appendix 1), the criteria

differ depending on the stage a project is in. Next, each project receives and overall

NPD portfolio score. This score is based on the three individual scores (risk, strategy

and a financial score), the weight of each score depends on the development stage a

project is in. Every organization determines the specific weighting, however

paragraph 5.1.7 describes some rules for determining the various weights for each

development stage. The next step is to rank each project based on the NPD portfolio

score. Remember, since the prescreening phase projects belong to a specific bucket.

Therefore, projects only compete for resources with projects in the same bucket.

Consequently projects are ranked “within” a bucket. E.g. if there are five buckets you

will have five stacks of ranked projects.

The next phase is the portfolio adjustment, a number of graphical depictions (for each

bucket) show the “state” of the NPD portfolio. Based on the definition of balance

(defined in the preprocess phase), the NPD portfolio can be adjusted if necessary. The

next stage is development, in this stage the selected projects are developed until they


M.L.W. De Mulder 76

reach the next development stage (for a stage gate assessment) or for the next

periodical NPD portfolio assessment, which starts at the individual assessment.

5.1.6 Usability criteria The original Kalypso model scored well on the usability criteria. As described in the

previous paragraph, the NPD portfolio should be reviewed periodically. This period is

organization specific; it depends on the average development time, the amount of new

projects started, etc. Cooper (2008) recommends doing this once a month. The

original Kalypso model clearly defined who is responsible for what (see paragraph

3.3). In the Kalypso model v2 there are additional people, tasks and responsibilities

involved (see figure 5.9 on the next page). The president and senior management set

the global strategy and define goals. In addition, resources have to be committed to

each goal. Hence, they have to split the portfolio into smaller buckets, each created to

serve a specific goal of the strategy. Specific criteria are formulated for each bucket.

The management level below senior management (intermediate level) adjust the

strategy to their level of management, hence, the strategy is cascaded down. Based on

their strategy and the criteria set by higher management, intermediate level

management formulates buckets with corresponding criteria for low-level

management. The size and goals of these buckets are constrained by the size and

goals of higher-level buckets. E.g., a lower level bucket can’t pursuit totally different

goals or be larger in size than the bucket it consists out of (as described in paragraph

5.1.1). Intermediate level management has the responsibility to review the project

assessments made low-level management, and fill buckets with these projects based

on the criteria set by senior level management. This cycle is repeated by lower level

management. However, the criteria set for individual projects are also based on input

from project manager and members. E.g., project manager and project members are

able to identify project specific risks that management couldn’t identify (due to lack

of project specific knowledge). Project managers are responsible for delivering the

necessary information for low-level management to make their NPD portfolio

decisions. Project assessments are done by project manager and team member but the

final decision is made by low level management. All relevant departments are

involved in the criteria formulation: e.g., R&D, marketing, supply chain and financial

under the supervision of NPD process manager. The NPD process manager is

responsible for collecting of relevant risks, this is done in cooperation with the project


M.L.W. De Mulder 77

manager. The NPD process manager is not part of the project team and ensures that

no criteria are overlooked. The NPD process manager is the “neutral” party. Figure 5.9 Responsibilities in the Kalypso v2 model

Note: this is an universal diagram, reposibilites could differ per organizion

The individual assessment of each project is the responsibility of the project leader

and is done by the project leader and the team members under the supervision of the

NPD process manager. Projects that are not sufficiently assessed will not be admitted

to the NPD portfolio reviews, and consequently lose their possibility for additional

funding! Low-level management in correspondence with the NPD process manager

and project leader make gate decisions. Hence, buckets are defined from the top-

down and are filled bottom-up. NPD process manager monitor the definition of

criteria and check of each criteria is assessed correctly. Table 5.2 shows the

responsibilities for each level.

Bucket Y.y

Bucket Y

Projects

Develop Criteria for Bucket Y

Develop Criteria for Bucket Y.y

Develop Criteria for Projects

Develop Strategy Portfolio

Senior Management level

Intermediate Management level

Low Management Level

Based on

Based on

Project Level

Review and Fill

Review and Fill

Feedback

Feedback

Feedback

NPD Process Manager

NPD Process Manager

NPD Process Manager

Project Manager

Project Member

Monitor

Monitor

Monitor

Collect

Collect

Collect

Review and Fill Based on

Develop & Assess Provide Input

Develop Intermediate Level Strategy

Develop Low Level Strategy

CascadedDown

CascadedDown


M.L.W. De Mulder 78

Table 5.2 Role and Responsibilities Kalypso model v2 Role Responsibility President & Senior Management

• Set global strategy for the organization. • Define goals. • Dedicate resources to each goal by defining buckets. • Monitor high-‐level KPIs to ensure strategy is adhered to.

Intermediate level management

• Translate global strategy into intermediate level management specific actionable strategy

• Dedicate resources to each R&D goal by defining (sub) buckets. • Fill buckets with program/buckets • Monitor the R&D project portfolio to ensure strategic R&D goals are

accomplished. Low level management

• Translate intermediate level strategy into low level management specific actionable strategy

• Dedicate resources to each R&D goal by defining (sub sub) buckets. • Fill buckets with projects • Monitor the R&D project portfolio to ensure strategic R&D goals are

accomplished. NPD Process Manager

• Collect projects/programs/buckets • Monitor criteria development (risk, strategic fit and financial) • Supervise project assessment.

Project Leader • Identify risks. • Collect project data (risk assessment, strategy assessment, financial assessment). • Assess project

Project Team Members

• Identify risks. • Assess project

5.1.7 General criteria The original Kalypso model scored best on the general criteria. Consequently the

Kalypso model v2 has the most similarities with the original Kalypso model regarding

general criteria. Similar to the original Kalypso model, the Kalypso model v2

produces an overall score based on multiple variables (strategic fit, risk and financial

score). Paragraph 2.4 explained that not all variables are equally important and

reliable in all stages. As in the original Kalypso model, variables will be weighed in

the Kalypso model v2. The exact weight assigned to each variable differs per

development stage and is determined by each organization. Exhibit 5.5 presents some

rules regarding the weighting of each variable per development stage.


M.L.W. De Mulder 79

Exhibit 5.5 Weighting of the various variables depending on development phase

Source: Cooper (2008), page 215

SFS1>RS1>FS1 SFS2>RS2>FS2 RS3>FS3>SFS3 RS4>FS4>SFS4

(RS4<RS3 and FS4 >FS3)

FS5>RS5>SFS5

100 = !"! + !" + !" SFS = Strategic Fit Score RS = Risk Score FS = Financial Score

At gate 1 (idea screen) strategic fit is the most important variable. The primary goal of

this gate is to assign projects in buckets (see appendix 1). The assignment of projects

to buckets is primary based on strategic criteria (Cooper et al. 1998, Chao and

Kavadias, 2004). There is little reliable financial information available at gate 0

(Linton et al., 2002), hence the financial variable should have the least impact. At the

second gate a similar (although more rigorous) assessment is conducted.

Consequently the weighting of the variables is the same. After gate 2 all projects have

to be assigned to a bucket. Gate 3 is the gate before a project enters full development.

According to Abetti and Stuart (1988) and Keizer et al. (2002) the risk assessment is

most important at this gate. After gate 3 the project’s costs increase significantly.

Consequently, you want to know how risky each project is before you invest in it.

Therefore risk is the most important variable at gate 3. Due to the detailed

investigation of markets conducted during stage 2 a more reliable financial

assessment is possible. The financial variable should have the second largest impact at

gate 3. Projects that have no fit with the strategy should be filtered out since the

previous two gates. Accordingly, strategic fit has the least impact at gate 3. The

distribution of weights at the gate 4 is similar to the previous stage. However the

financial metric increases in impact while the risk variable decreases in impact. After

testing most risks are identified and dealt with accordingly. Across all categories,

uncertainty tends to be reduced over the course of the project (Sommer et al., 2007,

chapter 17). Consequently there is reliable financial data available. In the last stage

the financial variable is has the greatest impact followed by risk. For the same reasons

mentioned earlier the strategic fit variable has the least impact.

Stage 0 Discovery

Stage 1Scoping

Stage 2Build business

caseStage 3

DevelopmentStage 4

Testing & validation

Stage 5Launch

Gate1

Idea Screen

Gate2

Second Screen

Gate3

Go To Development

Gate4

Go To Testing

Gate5

Go To Launch


M.L.W. De Mulder 80

The final NPD portfolio project score is calculated by multiplying each variable score

with its weight and add them as is shown in the following formula:

!"# !"#$%"&'" !"#$% = !"!! ∗ !! + !" ∗ !! + !"! ∗ !! Where: ai = weight of Strategic Fit Score (SFS) 1 < ai <100 bi = weight of Risk Score (RS) 1 < bi <100 ci = weight of Financial Score (FS) 1 < ci <100 i = stage project is in ∈ (0,1,..,4)

The NPD portfolio score enables users to compare projects independent of the

development stage. The project with the highest NPD portfolio score should receive

top priority. An organization should strive to maximize the sum of the NPD portfolio

scores constrained by the available resources. You can only compare projects that are

in the same bucket! This is because projects, in different buckets, are often assessed

with other criteria and the variables could be weighted differently.

It is possible that projects, critical for an organization, receive a low NPD portfolio

score. E.g., the development of a (underperforming) project for a specific client that,

if not developed, could lead to the loss of the client. If an organization values this

client the project has to be undertaken, hence the NPD portfolio score doesn’t have

influence the decisions. This is similar with urgent project in the original Kalypso

model. The NPD portfolio score is an aid, in the end, managers have to decide which

projects to pursuit and which one to abandon.

5.2 Evaluation of the Kalypso Model v2 The Kalypso model v2 is evaluated by the same criteria as the original model (chapter

4) and alternative NPD portfolio management models (paragraph 2.4). The author,

Mick Broekhof and Linda Kester conducted the assessments. Mick Broekhof is the

founding and managing partner of Kalypso Europe and has over 30 years of

professional experience in business management and consulting. He has extensive

expertise within the process industries. Linda Kester is a PhD student in product

portfolio management (since November 2006) at the department of Industrial Design

Engineering, Delft University of Technology (the Netherlands). The assessments were

conducted independent of each other. The final score of each criterion is average

score of each assessment. In the next part the assessment is discussed in more detail.


M.L.W. De Mulder 81

5.2.1 General Criteria Table 5.3 KE model general criteria evaluation

M. B

roekho

f

M. D

e Mulde

r

L. Kester

Average

1.1. The NPD portfolio management tool should be usable and understandable for its users (Kester et al., 2009, Archer and Ghsemzadeh, 1999,2007, Loch and Kavadias 2002)

6 5 2 4,3

1.2. The NPD portfolio management tool should use multiple types of methods and criteria (see paragraph 2.4.5)

7 7 7 7,0

1.3. The NPD portfolio management tool should be able to rank project based on pre-‐determined criteria (Archer and Ghsemzadeh, 1999,2007)

7 7 6 6,7

1.4. The selection methodology should be determined before the NPD portfolio selection starts (Archer and Ghsemzadeh, 1999,2007)

7 7 6 6,7

1.5. The NPD portfolio management tool should use a common measure which enable an equitable comparison (Archer and Ghsemzadeh, 1999,2007)

7 7 6 6,7


7 7 6 6,7

1.7. The NPD portfolio management tool should incorporate qualitative criteria (Cooper et al. 1998, 1999)

7 7 6 6,7

Average 6,9 6,7 5,6 6,4 Poor 1 2 3 4 5 6 7 Excellent

Linda Kester is of the opinion that the Kalypso model is very complex, this hampers

the understandability and limits the usability of the model. She based this opinion on

the process diagram presented in paragraph 5.1.5. Mick Broekhof and the author

disagree, they perceive the Kalypso model v2 to be understandable, each project is

assessed on three value drivers; financial, strategic fit and risk. Not all value drivers

are equally relevant in each development stage. Consequently, weights are assessed to

each value driver depending on the development stage a project is in. The size of each

weight is determined by predefined rules. The project with the highest score should

have the highest priority for the organization. Every variable (risk, financial and

strategic fit) is thoroughly assessed, which is a time consuming process. This could

limit the usability of the Kalypso model v2. Users might be hesitant to use the

Kalypso model v2 because of these resource requirements (time and people). The

Kalypso model v2 uses multiple criteria and is able to rank projects on predetermined

criteria (1,2, 1.3 and 1.4). The final NPD portfolio score enables users to compare

projects independent of their development phase and allows an equitable comparison

(1.5). The Kalypso model v2 does not solely depend on financial tools and

incorporates qualitative criteria (1.6 and 1,7).


M.L.W. De Mulder 82

5.2.2 Process Criteria Table 5.4 KE model process criteria evaluation

M. B

roekho

f

M. D

e Mulde

r

L. Kester

Average

2.1. The NPD portfolio method should be divided in phases (Archer and Ghsemzadeh, 1999,2007)

7 7 4 6,0

2.2. A project should be re-‐evaluated each time it transfers to another stage (Cooper, 1990, 2008, Archer and Ghsemzadeh, 1999,2007)

7 7 5 6,3

2.3. Individual project development and evaluation should be based on the Stage Gate® process (Cooper, 1990, 2008, Archer and Ghsemzadeh, 1999,2007)

7 7 4 6,0

2.4. The NPD portfolio should be reviewed periodically (Cooper et al., 1998, 1999, Archer and Ghsemzadeh, 1999, 2007, Chao and Kavadias, 2007)

7 7 5 6,3

2.5. Each project should be assessed individually on its (financial) value (Archer and Ghasemzadeh, 1999).

7 7 7 7,0

2.6. The NPD portfolio management tool should take into account the interdependencies between projects (Archer and Ghasemzadeh, 1999)

7 5 2 4,7

2.7. The NPD portfolio management tool should take into account the time dependent nature of project resource selection (Archer and Ghasemzadeh, 1999)

1 1 2 1,3

Average 6,1 5,9 4,1 5,4

Poor 1 2 3 4 5 6 7 Excellent

The Kalypso model v2 process is divided in phases (see paragraph 5.1.5) where each

project is re-evaluated each time a project transfers to another stage (2.1 and 2.2).

This process is based on the Stage Gate® process (2.3). The portfolio is reviewed

periodically based on individual project assessments (2.4 and 2.5). Project

interdependencies are treated during the risk and strategic fit assessments. However,

these assessments only look at interdependencies in a very limited manner. Linda

Kester is of the opinion that this assessment of interdependencies is too limited. This

is a design choice, complexity would increase drastically if a more rigorous

assessment of project interdependencies were implemented (2.6). This would be at the

cost of the criteria 1.1 (usability and understandable) and would limit the practical

applicability of the model (page 2). The same goes for implementing an assessment of

the time depended nature of project resource selection (2.7).


M.L.W. De Mulder 83

5.2.3 Strategic Criteria Table 5.5 Kalypso model v2 strategic criteria evaluation

M. B

roekho

f

M. D

e Mulde

r

L. Kester

Average

3.1. The NPD portfolio management tool must assess projects on strategic fit (Cooper and Kleinschmindt, 1995, Cooper et al., 1998)

7 7 4 6,0

3.2. The strategy should be determined before the NPD portfolio selection process starts (Archer and Ghsemzadeh, 1999,2007)

7 7 4 6,0

3.3. A clear strategy consists of the who, what, how, why and threats (Markides, 1999, Loch and Tapper, 2002) or consist out of four perspectives (financial, customer perspective, internal business process and learning and growth perspective) (Kaplan and Norton, 1996)

7 7 3 5,7

3.4. The NPD portfolio management tool should be able to align the portfolio to the strategy (Cooper et al. 1998, 1999)

7 7 3 5,7

3.5. The NPD portfolio management tool should be top-‐down and bottom-‐up (Cooper et al., 1998)

7 7 3 5,7

3.6. The strategy must make clear where and how many resources should be spent (Cooper et al., 1998)

7 7 3 5,7

3.7. Projects should be classified in advance of the portfolio selection (e.g. strategic Buckets) (Cooper et al., 1998, 1999, Archer and Ghsemzadeh, 1999, 2007, Chao and Kavadias, 2007)

7 7 4 6,0

3.8. Projects should be assigned to buckets Barczak et al. (2009) 7 7 6 6,7

Average 7,0 7,0 3,8 5,9


The Kalypso model v2 assesses projects on strategic fit (3.1). This assessment only

works if the strategy is determined beforehand (3.2). The strategy in the Kalypso

model v2 must be able to answer the who, what, how, why and threats (3.3). By using

strategic buckets it is clear where and how many resources are spend (3.6, 3.7 and

3.8). Top management defines a strategy, including goals and objectives. In addition,

top management formulates buckets. Each bucket is accompanied by a set of criteria.

Assessed on the criteria formulated by higher management, lower level management

fills the buckets with projects. (The projects are individually assessed by a

consecutive lower level.) Consequently, the model is a top down and bottom up

approach (3.5). Projects with the highest (overall) score, will probably score high on

strategic fit. Hence, by selecting projects with the highest score, users are able to align

the portfolio to an organization’s strategy. This all results in a model that is able to

align the portfolio to an organization’s strategy (3.4). Linda Kester argues that the

Kalypso model v2 aids users too little in the development of the strategy. She fears

that this will be the weak point of the Kalypso model v2, consequently she scored the

Kalypso model v2 low on other criteria.


M.L.W. De Mulder 84

5.2.4 Financial Criteria Table 5.6 Kalypso model v2 Financial criteria evaluation

M. B

roekho

f

M. D

e Mulde

r

L. Kester

Average

4.1. The NPD portfolio management tool should strive to maximize the value of the portfolio (Cooper et al., 1998, 1999)

7 7 5 6,3

4.2. The financial metric should incorporate the cost of development and launch (Cooper et al., 1998)

7 7 6 6,7

4.3. The financial metric should have no bias towards long term radical projects (Chao and Kavadias, 2007)

4 4 3 3,7

4.4. The financial metric should capture the flexibility of the development process (Newton et al., 2004)

7 6 6 6,3

4.5. The financial metric should be constrained by a critical resource (Cooper et al., 1998)

7 7 6 6,7

4.6. The financial metric should ignore sunk cost (Cooper et al., 1998) 7 7 7 7,0

4.7. The financial metric should incorporate the probability of success or failure (Cooper et al. ,1998, Pennings and Lint, 1997)

7 7 6 6,7

4.8. The financial metric should use the Minimum Acceptable Rate of Return (MARR) and Time Value of Money (TVM) (Remer and Nieto, 1995)

7 7 7 7,0

4.9. The financial metric should be user friendly and not too complex or appear as a black box (Faulkner, 1996, Cooper et al., 1999, Kester et al., 2009).

4 5 4 4,3

Average 6,3 6,3 5,6 6,1


The Kalypso model v2 strives to maximize the value of an NPD portfolio by using

both financial (qualitative (stages 0 and 1) and quantitative (stages 2,3 and 4)) and

strategic criteria to determine the value of projects (4.1). The quantitative metric used

(ECV), incorporates the costs of development and launch and is constrained by a

critical resource (4.2 and 4.5) (see paragraph 5.1.3). The ECV has a limited bias

towards long-term radical projects because it captures the flexibility of the

development process by incorporating the probabilities of success or failure into the

calculations (4.3, 4.4 and 4.7). These probabilities are determined by assessment of

relevant risks (see paragraph 5.1.3). ECV uses the NPV for its calculations, the NPV

uses the MARR and TVM (4.8). The use of the ECV and probabilities increase the

complexity and decreases user-friendliness (4.9).


M.L.W. De Mulder 85

5.2.5 Balance Criteria Table 5.7 Kalypso model v2 Balance criteria evaluation

M. B

roekho

f

M. D

e Mulde

r

L. Kester

Average

5.1. The NPD portfolio tool must be able to visually depict the portfolio (Cooper et al., 1998)

7 7 6 6,7

5.2. Management must define what balance and imbalance is (Cooper et al., 1998) 5 5 5 5,0

5.3. The NPD portfolio should only show the for management most important diagrams (Cooper et al., 1998)

4 5 1 3,3

5.4. The Bubble diagram X and Y-‐axis should be based on a companies strategic goals and objectives Cooper et al., 1998)

7 7 1 5,0

Average 5,8 6,0 3,3 5,0


The Kalypso model v2 can produce various visual depictions (5.1). E.g., the model

uses risk and financial data, consequently, it can depict a risk reward diagram. It uses

Stage Gate® data, hence it can depict which projects are in what development stage.

Etc. The Kalypso model v2 addresses the notion that balance has to be defined,

however it doesn’t aide users in determining what balance is (5.2). Balance is defined

by management and has to be based on the strategy. E.g., an organization wants to

spend 70% of their R&D resources on radical new projects. When a “type of project –

development cost diagram” shows that only 40% of the budget is spend on radical

projects while 60% is spend on incremental innovation one can conclude that the

portfolio is imbalanced. Another possibility is to look at the risk profiles of each

project. The number of acceptable high, medium and low risk projects could be

predetermined. However, the Kalypso model v2 can't define what balance is, its

unique to each organization. The same applies to determining what the most

important diagrams are (5.3). Linda Kester, argues that this is a management decision

and, consequently, isn’t a function of the Kalypso model v2. The author and Mick

Broekhof disagree, if an organization decides what the most important diagrams are,

the Kalypso model is able to depict them. The Kalypso model v2 helps organizations

to implement their strategy, this enables the model to produce diagrams that are based

on an organization’s goals and objectives (5.4).


M.L.W. De Mulder 86

5.2.6 Risk Criteria Table 5.8 Kalypso model v2 Risk criteria evaluation

M. B

roekho

f

M. D

e Mulde

r

L. Kester

Average


7 7 7 7,0

6.2. Risk should be assessed on three dimensions; likelihood, impact, and the ability to do something about it (Halman and Keizer, 1994)

7 7 7 7,0

6.3. Risk is assessed on a sufficient number of criteria (Keizer et al., 2005) 6 6 7 6,3

Average 6,7 6,7 7,0 6,8


The Kalypso model v2 assesses projects on risk (6.1). Each project is assessed on

three dimensions; occurrence, impact and ability to do something about it (6.2). The

Kalypso model advocates using a sufficient number of risks (60) for the assessment of

each project (6.3) (see paragraph 5.1.2).

5.2.7 Usability Criteria Table 5.9 Kalypso model v2 Usability criteria evaluation

M. B

roekho

f

M. D

e Mulde

r

L. Kester

Average

7.1. Governance roles and responsibilities must be defined (Cooper, 2008) 6 6 6 6,0

7.2. The NPD portfolio should be periodically assessed (Cooper, 2008) 7 7 5 6,3

7.3. The NPD portfolio management method is user friendly and not too complex or appears as a black box (Faulkner, 1996, Cooper et al., 1999, Kester et al., 2009)

7 5 2 4,7

7.4. Deliverables must be clearly defined for each portfolio assessment (Cooper, 2008)

7 6 5 6,0

7.5. Deliverables should be focused on the essential information needed for NPD portfolio management decisions (Cooper, 2008)

7 7 3 5,7

7.6. Templates should not be overly detailed and be limited to the essential information needed for NPD portfolio management decisions (Archer and Ghazemzadeh, 1998, Cooper, 2008)

7 5 2 4,7

Average 6,8 6,0 3,8 5,6


There are more people involved in the Kalypso model v2 compared to the original

Kalypso model. Consequently extra roles and responsibilities are defined in the

Kalypso model v2. It is clear who is responsible for what and why (see paragraph

5.1.6) (7.1). The NPD portfolio is periodically assessed (7.2), however this

assessment period differs per organization. Mick Broekhof and the author find the

overall Kalypso model v2 not complex and easy to understand because all variables


M.L.W. De Mulder 87

and their weight are clearly defined. Consequently the deliverables are also clearly

defined (it is clear on what criteria a project is assessed), however each organization

has to define its own deliverables (7.3). As mentioned before, using the Kalypso

model v2 requires resources (especially time). It takes time to collect all information

and conducted the necessary assessments. The usability of the Kalypso model v2

could be limited when users are not prepared to commit these resources (7.3). By

defining on what variables (risk, strategic fit and financial) a project is assessed, it

becomes easy to focus the deliverables (7.5). The Kalypso model v2 risks having too

detailed templates (7.6). This is especially the case for the risk assessment when too

many risks are assessed.

5.3 Conclusion The results of the assessment of the Kalypso model v2 are shown in table 5.10. Table

5.10 also shows the results of the assessment of the original Kalypso model.

Table 5.10 Average scores per category Original Kalypso

model Kalypso model v2

General Criteria 5,7 6,4

Process Criteria 4,7 5,4

Strategic Criteria 2,8 5,9 Financial Criteria 3,1 6,1 Balance Criteria 3,8 5,0

Risk Criteria 3,0 6,8

Usability Criteria 5,2 5,6

Overall score 4,0 5,9 Poor 1 2 3 4 5 6 7 Excellent

What become clear is that the Kalypso model v2 is a significant improvement over

the original Kalypso model. The Kalypso model v2 receives a higher score on all

criteria. Big improvements are made on strategic, risk and financial criteria. These

criteria were the weakest aspects of the original Kalypso model. Based on the results

of the assessment it be concluded is that the Kalypso model v2 can be used as the

single and only NPD portfolio management model in an organization. Hence, the

Kalypso model v2 enables organizations to make the best NPD portfolio decisions

possible.


M.L.W. De Mulder 88

Chapter 6: Conclusion The goal of this master thesis was to develop an NPD portfolio management reference

model that enables users to make the best possible NPD portfolio decisions. The

result of this master thesis project is the Kalypso model v2. By assessing each project

on three relevant variables (strategic fit, financial and risk) and using these

assessments to conduct an overall score, the Kalypso model v2 enables users to make

the best NPD portfolio decisions possible. The overall NPD portfolio score allows

users to compare projects independent of their development phase. The Kalypso

model v2 is the result of combining various processes and models developed by

scientists, while building on experience of practitioners and experts. The developed

NPD portfolio reference framework enables Kalypso to implement NPD portfolio

management solutions at any client. In the next paragraphs, the scientific and practical

implications are discussed in more detail, accompanied by the model’s limitations.

This chapter ends with recommendations for further research.

6.1 Scientific Implications The Kalypso model contributes to the body of knowledge by combining existing

models with practical knowledge. The Kalypso model v2 is the first model that

combines the NPD portfolio management process developed by Archer and

Ghasemzadeh (1999), research conducted by Cooper et al. (1998, 2001) and the Risk

Diagnosing Methodology by Keizer et al. (2002).

During this research, a tool was developed for evaluating NPD portfolio management

methods. Based on the scientific literature, a list of relevant criteria was composed

(table on which an NPD portfolio management tool can be assessed. The assessment

score provides an indication of usability of an NPD portfolio method. This NPD

portfolio management method assessment tool will also indicate what the strong and

weak points of the method are.

6.2 Scientific Limitations A limitation of this research is that the model presented in this master thesis report is

not empirically tested. Empirical testing provides insight in how the model works in

practice. It will highlight what is applicable and what is not. Another limitation

concerns the validation of the model. Experts were involved in the validation of the

new model. Their views and comments were used to adjust and improve the new


M.L.W. De Mulder 89

model. However, the number of experts involved is limited. The background of these

experts is consulting and academic research. Hence, no people who actually conduct

portfolio management were involved. This hampers the validity of the Kalypso model

v2.

Table 6.1 Scientific Limitations

• The Kalypso model v2 is not tested • Limited number of experts involved • No practitioners who actual conduct NPD portfolio management were involved

6.3 Practical Implications The first goal of this master thesis is to develop an NPD portfolio model that can be

used by practitioners. The Kalypso model v2 receive a 5,9 out of 7,0 in the

assessment and therefore can be used as the only and primary tool in an organization

(see paragraph 5.3). The Kalypso model v2 scored significantly higher on the

assessment then the original model and the various other methods. The new model

can help practitioners overcome many of the difficulties encountered in NPD portfolio

management today. A recent (2010) benchmark study, commissioned by Planview®,

with 922 respondent, found that companies have too many projects for their

resources. This can partly be explained by the reluctance of organizations to abandon

under-performing projects (Planview®, 2010). In the Kalypso model v2 projects are

assessed thoroughly and the results are translated into a universal score. This enables

organizations to objectively compare different projects independent of the

development phase. Consequently, organizations have a better insight in the

performance of various projects. Due to this “ranking”, deciding to abandon or

continue projects becomes more rational; there is less room for subjectivity. Hence,

using the Kalypso model v2 will decrease the probability of an overloaded NPD

portfolio. In addition, the implementation of buckets will lead to more focused

resource allocations, this will also results in a decreased risk of a portfolio overload.

Every bucket has its own committed resources. The number of projects in one bucket

should have no effect on the resource availability in another bucket.

The benchmark study, commissioned by Planview®, found that the most important

criteria for a company is strategic alignment followed by revenue and margin (hence,

the last two are financial criteria). Risk wasn’t indicated as a top priority, but still


M.L.W. De Mulder 90

important. The Kalypso model v2 uses the same variables (strategic, financial and

risk) to conduct an overall score for each project. The Kalypso model v2 uses criteria

which are relevant for practitioners.

Not all organizations are able to directly implement the Kalypso model v2. As

described earlier, every organization can be ranked on a maturity scale. Dooley et al.

(2001) define maturity as “the extent to which a process is explicitly defined,

managed, measured and continuously improved”. There are various levels of

organizational maturity and not all levels are compatible with the Kalypso model v2.

Figure 6.1 shows the various maturity levels defined by the Organization Project

Management Maturity Model (OPMP3tm) accompanied by some of their properties.

Figure 6.1 OPMP3tm maturity levels

Source: based on P3M3 Public Consultation Draft v2.0 http://www.ogc.gov.uk/documents/p3m3.pdf

Only organizations with a maturity level 3 or higher are able to implement the

Kalypso model v2. There are many people involved in the Kalypso model v2. E.g. all

project team members, project managers, process managers, high level management

etc. Consequently, an organization should be committed to provide these people to the

NPD portfolio management process. This can only be achieved if top management is

Level 1 - Initial Process- Ad-hoc- No proces defined- No standards- Limited formal checking or review- Information outdated- No formalization

Level 2 - Repeatable Process- Partly, localized, defined process- Localized standards- Estimation is more "guestimation" and does not use standard techniques-Inconsistent business objectives-lack of fully integrated risk management

Level 3 Defined Process- Centrally defined process- Organization wide standards-Strategies transformed into portfolio initiatives- Redundant initiatives are eliminated-Roles are defined- High level management involvement

Level 4 - Managed Process-Robust framework in place-High level organizational involvement- Information is current and extensively referenced for better decision making-Quantitative and qualitative measure are used

Level 5 - Optimized Process-Portfolio management processes proven and optimized-Key Performance Indicators aligned to the organization's strategic objectives


M.L.W. De Mulder 91

involved. NPD portfolio management requires various departments to cooperate and

provide information. Therefore organization wide standards and processes have to be

defined, otherwise collaboration and communication will be a cumbersome effort.

For organizations that have a maturity level of 1 or 2, the Kalypso model v2 can

provide a starting point or reference guide from which they can improve their NPD

portfolio management process. The Kalypso reference model can provide

practitioners various insights concerning NPD portfolio management like:

• The Kalypso model provides insight in the application of strategic buckets

(paragraph 5.1.1).

• The Kalypso model v2 provides insight in which variables should be assessed

and how an overall score can be conducted (paragraph 5.1.7)

• Risk should be assessed on three dimensions occurrence, impact and control

(paragraph 5.1.2)

• Although financial criteria are the second most important criteria for

practitioners (Planview®, 2010), quantitative financial metrics are less

appropriate in the early stages of the development process. However, via a

qualitative assessment it is possible to determine a financial score (paragraph

5.1.3).

• The Kalypso model v2 provides insight in which variable is important in

which development phase (paragraph 5.1.7)

• Responsibilities throughout an organization with regards to NPD portfolio

management (paragraph 5.1.6)

6.4 Practical Limitations There are some possible risks involved in using the Kalypso model v2 . During one of

the expert discussions, the concern was raised that the implementation of the Kalypso

model could result in a loss of organizational flexibility. This concern was based on

the formal process proposed in the Kalypso model v2. Formalization comes at the

cost of flexibility (Kester et al., 2009).

Another possible limitation is the level of information and commitment required in

order to effectively use the Kalypso model v2. Many people throughout the


M.L.W. De Mulder 92

organization are involved in the new model. Consequently, it takes up precious

resources. In order to make thoroughly considered NPD portfolio decisions a

significant number of people are required in the Kalypso model v2. They collect

information, conduct assessments and monitor the process (e.g. the project risk

assessment is conducted by project team members and is monitored by a NPD process

manager). By involving various people throughout the organization, the risk of

continuing underperforming projects is reduced. In the Kalypso model v2, employees

who are directly involved in the development of a project do not make the final

portfolio decisions. By providing objective scores, decision makers are enabled to

make a sound decision without being influenced by various biases. However, this is at

the cost of including numerous employees in the process.

The Kalypso model v2 is a reference model. Consequently, it needs to be tailored to

every organization. This takes time and effort, e.g., the determination of project

specific risks is a cumbersome process. The same applies to the development of the

strategy, corresponding objectives and translating them to NPD portfolio management

criteria. Table 6.2 lists the practical limitations of the Kalypso model v2.

Table 6.2 Practical Limitations

• The Kalypso model v2 could lead to loss of flexibility • High commitment is required of the organization • Many people are involved • The Kalypso model v2 can’t directly be applied

6.5 Recommendation for further research There are still many areas concerning NPD portfolio management that are not covered

in this research. In this paragraph various recommendations for further research are

proposed.

• Project interdependencies are complex to measure and manage.

Interdependencies play a small part in the Kalypso model v2. Future research

could be aimed at investigating the role of interdependencies with regard to

NPD portfolio management.


M.L.W. De Mulder 93

• The implementation of the Kalypso model is dependent on the maturity of an

organization. Future research could focus on which steps organizations must

take in order to implement the Kalypso model.

• Future research could be aimed at testing the Kalypso model in practice. The

current model is theoretical and the next step could be testing the Kalypso

model in practice.

• The focus of this research was NPD portfolio management at the level of

individual projects. At program or business unit level, the scope of variables

change from specific to more general (Chao and Kavadias, 2007). This brings

a different dynamic into the NPD portfolio process. Further research could be

conducted regarding different levels in NPD portfolio management.

• Not all industries are alike, e.g. pharmaceutical manufactures differ

substantially from electronics manufactures with regard to the NPD process.

These differences could affect the applicability of the Kalypso model v2.

Further research is recommend to address these possible influences.


M.L.W. De Mulder 94

References Abetti, P.A., Stuart, R.W., 1988, Evaluating new product risk, Research Technology Management, vol. 31, pp. 40-

44

Adler, P.S., Mandelbaum A., Nguyen V., Schwerer, E., 1995, From project to process management: an

empirically-developed framework for analyzing product development time, Management Science, vol. 41, pp. 458-

484.

Andrews K. R. 1995. The concept of corporate strategy. The strategy process. H. Mintzberg, J. B. Quinn, S.

Ghoshal. Prentice Hall London, from K. R. Andrews. 1980. The concept of corporate strategy. Copyright Richard

D. Irvin, Inc.

Archer, N., Ghasemzadeh, F., 2007, Project portfolio selection and management, in Morris, P.W.G., Pinto, J.K.,

The Wiley Guide to Project Program & Portfolio Management, pp.94-112, New Jersey: John Wiley & sons

Barczak, G., Griffin, A., Kahn, K.B., 2009. Perspective: trends and drivers of success in NPD practices: results of

the 2003 PDMA best practices study, Journal of Product Innovation Management, vol. 26, pp. 3–23

Balachandra, R., Friar, J.H., 1997, Factors for success in R&D projects and new product innovation: a contextual

framework. IEEE Transactions on Engineering Management, vol. 44, pp. 276–287.

Batenburg, R., Helms, R., Versendaal, J., 2005, The maturity of product lifecycle management in Dutch

organizations—A strategic alignment perspective, Institute of Information and Computing Sciences, Utrecht

University, Technical report UU- CS-2005-009

Boyer, K.K., McDermott, C., 1999, strategic consensus in operations strategy, Journal of Operations

Management, vol. 17, pp. 289-305

Chao, R., Kavadias, S., 2004, A theoretical framework for managing the new product development portfolio: when

and how to use strategic buckets, Management Science, vol. 54, pp. 907-921

Chao, R., Kavadias, S., 2007, Resource allocation and new product portfolio management, Handbook of New

Product Development Research, Oxford: Elsevier/Butterworth

Coldrick, S., Longhurst, P. Ivey, P., Hannis, J., 2003, An R&D options selection model for investment decisions,

Technovation, vol. 25, pp. 185-193

Cooper, R.G., 1990, Stage-gate systems: a new tool for managing new products, Business Horizons, vol. 33, pp.

44-54

Cooper, R.G., 1993, Winning at new products, 2nd edition, Addison-Wesley, Reading, Ma

Cooper, R.G., 2008, Perspective: the Stage Gate® idea-to-launch process- update, what’s new, and nexgen

systems, Product Innovation Management, vol. 24, pp. 213-132


M.L.W. De Mulder 95

Cooper, R.G., Edgett, S.J., 2001, Portfolio management for new products: picking the winners, Product

Development institute, http://www.stage-gate.net/downloads/working_papers/wp_11.pdf

Cooper, R.G., Edgett, S.J., 2006, 10 ways to make better portfolio and project selection decisions, Stage Gate®

International, http://www.stage-gate.net/downloads/working_papers/wp_24.pdf

Cooper, R.G., Edgett, S.J., Kleinschmidt, E.J., 1998, Portfolio management for new products, Perseus Books,

Cambridge, Ma

Cooper, R.G., Edgett, S.J., Kleinschmidt, E.J., 1999, New product portfolio management: practices and

performance, Journal of Product innovation Management, vol. 16, pp. 333-391

Cooper, R.G., Edgett, S.J., Kleinschmidt, E.J., 2001. Portfolio management for new product development: results

of an industry practices study, R&D Management, vol. 31, pp. 361- 380

Cooper, R.G., Edgett, S.J., Kleinschmidt, E.J., 2002, Optimizing the stage-gate process: What best –practice

companies do- II, Research Technology Management, vol. 2, pp 43-49

Cooper, R.G., Kleinschmidt, E.J., 1995, Benchmarking the firm’s critical success factors in new product

development, Journal of Product innovation Management, vol. 12, pp. 374-351

Docherty, M., 2006, Primer on "Open Innovation ": Principles and Practice, Vision, vol.2, pp 13-17

Faulkner, T.W., 1996, Applying “option thinking” to R&D valuation, Research Technology Management, vol. 39,

pp. 50-56

Graziano, A.M., Raulin, M.L., 2005, Research methods, A process of Inquiry – Fifth edition, Pearson Education

Group, Boston, MA

Griffin, A., 1997, PDMA research on new product development practices: updating trends and benchmarking best

practices, Journal of Product Innovation Management, vol. 14, pp. 429-458

Henard, D.H., Szymanski, D.M., 2001, Why some new products are more successful than others, Journal of

Marketing Research, vol.38, pp. 362-275

Joshi, M.P., Kathuria, R., Porth, S.J., 2003, Alignment of strategic priorities and performance: an integration of

operation and strategic management perspectives, Journal of Operations Management, vol. 21, pp. 353-369

Kahneman, D., Tversky, A., 1979, Prospect theory: an analysis of decision under risk, Econometrica, vol. 47, pp.

263-292

Kaplan, R.S., Norton, D.P., 2001, Transforming the balanced scorecard form performance measurement to

strategic management: Part I, Accounting Horizons, vol. 15, pp. 87-104


M.L.W. De Mulder 96

Keizer, J.A., Halman, J.I.M., Song, M., 2002, From experience: applying the risk diagnosing methodology,

Journal of Product Innovation Management, vol. 19, pp. 213-232

Keizer, J.A., Vos, J.P., Halman, J.I.M., 2005, Risks in new product development: devising a reference tool, R&D

Management, vol. 35, pp. 297-307

Kester, L., Griffin, A., Hultink,E.J., Lauche, K., 2009, Towards a general model of portfolio decision making

effectiveness, Accepted for Acadeny of Management Annual Meeting

Kester, L., Hultink, E.J., Lauche, K., 2009, Portfolio decision-making genres: a case study, Journal of Engineering

and Technology Management, vol. 26, pp. 327-341.

Loch, C. H., Tapper, S., 2002, Implementing a strategy-driven performance measurement system for an applied

research group, Journal of Product Innovation Management, vol. 19, pp 185-198.

March, J.G., Shapira, Z., 1987, Managerial perspectives on risk and risk taking, Management Science, vol. 33, pp.

1404-1418

Markides, C.C., 1999, A dynamic view of strategy, Sloan Management Review, vol. 40, pp. 55-64

Martino, J.P. 1995, R&D project selection, Wiley, New York, NY

Montoya-Weiss, M.M., Calantone, R., 1994, Determinants of new product performance: a review and meta

analysis, Journal of Product Innovation Management, vol. 11, pp. 397-417.

Newton, D.P., Paxson, D.D., Widdicks, M., 2004, Real R&D options, International Journal of Management

Reviews, vol. 5/6, pp. 112-130

Park, C.S., 2007, Contemporary Engineering Economics (4th Edition). Prentice Hall.

Pattikawa, L.H., Verwaal, E., Commandeur, H.R., 2006, Understanding new product project performance,

European Journal of Marketingi, vol. 40, pp. 1178-1193

Pennigs, E., Lint, O., 1997, The option value of advanced R&D, European Journal of Operations Research, vol.

103, pp. 83-94

Pennings, E., Lint, O., 1997, The option value of advanced R&D, European Journal of Operational Research, vol.

103, pp. 83-94

Pich, M.T., Loch, C.H., De Meyer, A., 2002, On Uncertainty, Ambiguity, and Complexity in Project Management,

Management Science, vol. 48, pp. 1008-1023

Planview®, 2010, 2nd Annual benchmark study: Product portfolio management, www.planview.com


M.L.W. De Mulder 97

Remenyi, D. and Heafield, A. (1996), “Business process re-engineering: some aspects of how to evaluate and

manage the risk exposure”, International Journal of Project Management, Vol. 14 No. 6, pp. 349-57

Remer, D.S., Nieto, A.P., 1995, A compendium and comparison of 25 project evaluation techniques. Part 2 Ratio,

payback and accounting methods, International Journal of Production Economics, vol. 42, pp. 101-129

Skinner, W., 1974, The focused factory, Harvard Business Review, vol. 3, pp. 113-119

Sommer, S.C., Loch, C.H., Pich, M.T., 2008, Project risk management in new product development, Loch, C.H.,

Kavadias, L., Handbook of New Product Development Management, Elsevier Linacre House, Oxford

Stark, J., 2004, Product lifecycle management – 21st century paradigm for product realization, Springer, London

Thompson, J.D., 1967, Organizations in Action, McGraw-Hill, New York.

Turner, J R, 1993, The Handbook of Project Based Management, Improving the Processes for Achieving Strategic

Objectives, McGraw-Hill, UK

Tushman, M.L., O’Reilly, C.A., 1996, Ambidextrous organizations: Managing evolutionary and revolutionary

change, California Management Review, vol. 38, pp. 8–30

Verma, D., Sinha, K.K., 2002, Toward a theory of project interdependencies in high tech R&D environments,

Journal of Operations Management, vol. 20, pp. 451-468

Webb, A., 1994, Managing Innovative Projects, Chapman & Hall, London


M.L.W. De Mulder 98

Appendices

Appendix 1: Description of the different stages of the Stage Gate® process

Description of the different stages of the Stage Gate® process Stage 0: Discovery The process starts with the first stage discovery phase, in this phase ideas are generated freely.

Gate 1: Initial Screen The ideas are submitted to the first gate idea screen. At this gate the first decisions to commit resources to the project. Project are selected with a set of must meet and should meet criteria. These criteria are cover strategic alignment, project feasibility, magnitude of the opportunity differential advantage, synergy with the firm’s core business and resources and market attractiveness. Financial criteria are not part in this gate.

Stage 1: Preliminary Assessment When a project passes the first gate it enters a stage called scoping. This is an inexpensive stage which has the objective to determine a project’s technical and marketplace merits. This stage is about gathering market and technical information at low cost and in a short time.

Gate 2: Second Screen At gate 2 the project undergoes a second screening. The project is reevaluated based on the information gathered during the previous stage. The project is screened with the same criteria as gate 1. There are also extra criteria about dealing with sales force and customer reaction to the proposed product. A quick and simple financial return is calculated but a checklist and scoring model facilitate the gate decision.

Stage 2: Building business case During the next stage a business case is build. The attractiveness of a project is verified, market research studies are undertaken to determine the customer’s needs, wants and preferences. A detailed technical assessment focused on the “do-‐ability” of the project must be made. Customer needs and wants must be translated into technically and economically feasible solutions. A detailed financial analysis is conducted involving a discounted cash flow approach, complete with sensitivity analysis.

Gate 3: Decision on Business Case The next gate is the final gate prior to the development stage. After this gate the spending on the project increases significant. The project is evaluated on qualitative and quantitative criteria. The qualitative evaluation involves a review of each of the activities in stage 2. The quantitative evaluation involves a review of the financial analysis made in the previous stage. At gate 3 agreements must be reached on a number of key items before the project proceeds into development. These items include target market definition, definition of the product concept, specification of a product positioning strategy, definition of the product benefits to be delivered and agreement on essential and desired product features attributes and specification. If a project passes gate 3; development is started.

Stage 3: Development During this stage the product, detailed tests, marketing and operational plans are developed. The financial analysis is updated with new and more accurate data.

Gate 4: Post Development Review Gate 4 is a check on the progress and the continued attractiveness of the product and project. Development work is reviewed and checked, ensuring that the work has been completed in a quality fashion. This gate revisits the economic question via a revised financial analysis based on new and more accurate data.

Stage 4: Validation This stage tests the entire viability of the project: the product itself, the production process, customer acceptance and the economics of the project.


M.L.W. De Mulder 99

Description of the different stages of the Stage Gate® process (continued)

Gate 5: Pre-‐Commercialization Decision Gate 5 is the final gate between testing and validation and launch. Financial projections play a key role in the decision to move ahead. Operations and marketing plans are reviewed an approved for implementation. Gate 5 is the final point at which the project can still be killed. Stage 5: Launch The final stage of the Stage Gate® process is the launch stage. This stage involves the implementation of both marketing and launch plan and the operations plan. Source: Cooper (1990), page 52-‐53


M.L.W. De Mulder 100

Appendix 2: Description of different levels of PML maturity Description of different levels of PLM maturity No PLM investment, or on ‘ad-‐hoc’ basis only (level 0). Nobody is responsible for PLM and there is no vision available for PLM. Therefore, there are no consistent PLM processes and supporting systems. At this level, information about a product is scattered throughout the organization, which hinders strategic decision-‐making. On departmental level (‘silo’ orientation) (level 1). PLM is seen as a data management problem that should be dealt with on departmental level, but there is no overall vision to coordinate local initiatives. Often the development or engineering department is the first department that starts to implement PLM systems. At this level, at least all information regarding the early stages of a product is stored in a central system. On the organizational level (cross-‐departments) (level 2). PLM is interpreted as a business problem that requires a corporate vision and an integral approach. Besides engineering and development also other departments are heavily involved. PLM processes are defined that cross-‐departmental borders and organization wide PLM systems are implemented to support these processes. Moreover, PLM systems are integrated with other major enterprise systems, such as ERP. At this level, all product information within the organization is stored in a central system and there is control information available regarding PLM processes. On the inter-‐organizational level (cross supply chain partners) (level 3). PLM is seen as a business problem that spans the complete product lifecycle. Therefore, the supply chain should be involved in defining a PLM vision. PLM processes are defined that cross-‐organizational borders and PLM systems are integrated with those of the suppliers to enable collaboration. At this level, all product information across the product lifecycle is stored in a central system making the product lifecycle become transparent enabling proper decision-‐ making concerning a product. Source: Batenburg et al. (2005) page 5



Appendix 3: Risk Reference tool including risk issues taken from literature and issues resulting from case studies Risk Reference Framework including risk issues taken from literature and issues resulting from case studies

Product Family and Brand positioning Risks

1 Literature Contribution to category’s business strategy

2 Literature Contribution to project portfolio

3 Cases Contribution to brand name position

4 Cases Global roll out potential and schedule

5 Cases Fit within existing brand

6 Cases Cannibalization of existing portfolio

7 Cases Fit with brand image

8 Literature Family development potential

9 Cases Platform deployment

10 Cases Organization reputation

11 Cases Brand recovery potential

12 Cases Brand development potential

13 Cases Consumer acceptance of platform

Product technology risks

1 Literature Intended functions known and specified

2 Literature New product fulfills intended functions

3 Cases In-‐use conditions known and specified

4 Cases Interactions of product in-‐use with sustaining materials, tools etc.

5 Cases Components’ properties, function and behavior

6 Cases Correct balance between product components

7 Cases Product format meets functional requirements

8 Literature Safety and technical requirements for assembled product

9 Literature Alternatives to realize intended product functions

10 Cases Parity in performance compared with other products

11 Cases Stability of product while in storage (factory, shop/warehouse, transportation, at home)

Manufacturing technology risks

1 Cases Raw materials meeting technical requirements

2 Cases Process steps to realize the new product

3 Literature Conditions (temperature, energy, safety requirements) to guarantee the processing of a good product quality

4 Literature Production means (equipment and tools) necessary to guarantee good product quality

5 Literature Scale up potential according to production yield standards

6 Literature Production system requirements (quality and safety standards, training of human resources, facilities etc.)

7 Cases Product packaging implications

8 Cases Alternative approaches to process the required product

9 Cases Efficiency of production

10 Literature Adequate production capacity available

11 Literature Adequate Production Start Up

12 Cases Reusability of rejects in production

Intellectual property position

1 Literature Protection of original know-‐how

2 Literature Dependency on third party development



Risk Reference Framework including risk issues taken from literature and issues resulting from case studies (continued) 3 Literature Availability of required external licenses or know-‐how

4 Literature Relation to legal and patent rights of competitors

5 Literature Knowledge of relevant patent issues

6 Cases Patent crossing potential

7 Literature Trade mark registration potential

Supply chain and sourcing risks

1 Cases Constant and predictable product quality

2 Cases Capacity to meet peak demands

3 Literature Appropriate after sales services

4 Cases Contingency options for each of the selected suppliers

5 Cases Reliability of each supplier in delivering according to requirements

6 Cases Financial position of each supplier ensuring a long-‐term supply performance

7 Cases Past experiences with each of the suppliers

8 Cases Suppliers’ readiness to accept modifications if required

9 Cases Possibility to cancel supply contracts

10 Cases Ability to produce required quantities against acceptable prices

11 Cases Appropriate contract arrangements with suppliers

Consumer acceptance and Marketing risks

1 Literature Product specifications meeting consumer standards and demands

2 Cases Fit of new product with consumer habits and/or user conditions

3 Literature Consumer benefits of new product: unique features or attributes

4 Literature Consumers’ conviction that they get value for money, compared with competitive products

5 Literature Product’s appeal to generally accepted values (e.g. health, safety, nature, environment)

6 Literature Product’s easy-‐in-‐use advantages, compared with competitive products

7 Literature Product offering additional enjoyment, compared with competitive products

8 Literature Product reducing consumer’s costs, compared with competitive products

9 Cases Non-‐intended product use by consumers

10 Cases Stability of target consumer’s attitudes during the development period

11 Literature Communicating the product with target consumers

12 Literature Knowledge of primary consumer requirements

13 Literature Target consumers accepting key product ingredients

14 Literature Niche marketing capabilities

15 Cases Realistic product claim

16 Literature Efficacy of advertising

17 Literature Product claims stimulating target consumers to buy

18 Literature Repeat sales potential of new product

Trade customer acceptance risks

1 Cases Product specifications meeting trade customer standards and demands

2 Cases Trade customers welcoming the new product from the perspective of potential sales

3 Cases Trade customers welcoming the new product from the perspective of profit margin

4 Cases Trade customers welcoming the new product given required surface and volume on shelf and storage facilities

5 Cases Trade customer’s attitude remaining unchanged during the development period

6 Cases Communicating the product with trade customers

7 Cases Use of right distribution channels



Risk Reference Framework including risk issues taken from literature and issues resulting from case studies (continued) 8 Cases Appropriate product care by trade

9 Cases Trade supporting persons endorsing the new product

10 Cases Meeting stock demands

Competitor risks

1 Literature Implications of being technology leader or follower for this project

2 Literature Product providing clear competitive advantages

3 Literature Impact of the introduction of the new product on market share positions

4 Literature Impact of the introduction of the new product on market prices

5 Literature Competitive product being launched before launch of new product

6 Literature Response actions towards public and media expected from competitors

7 Literature New product enabling the creation of potential barriers for competitors

8 Literature Acting effectively to competitor’s actions

9 Literature Foreseeing future competitor’s challenges

Commercial viability risks

1 Literature The market target clearly defined and agreed

2 Literature Market target based on convincing research data

3 Literature Capital cost projection for the product’s life cycle feasible and based on convincing data

4 Literature Delays in product launch affecting the commercial viability of the product

5 Literature Sales perspectives being realistic

6 Literature Profit margin based on convincing research data

7 Literature Profit margin meeting the organization’s standards

8 Literature The return on investment (R.O.I.) projection meeting the organization’s standards

9 Cases Clear and reliable volume estimates

10 Cases Product viability because of repeat sales

11 Cases Attractive purchasing agreements with suppliers

12 Cases Knowledge of pricing sensitivity

13 Cases Adequate investments to secure safety in production

14 Cases Long term market potential

15 Cases Financing of capital investment being secured

16 Cases Fall back potential to use prior product concept

17 Cases Product viability in spite of market restrictions

Organizational and Project Management Risks

1 Literature Internal political climate being in favor of this project

2 Literature Top management’s priority and support for the project

3 Literature Sponsor’s interest for the project

4 Literature Project mission and project goals being clearly specified and feasible

5 Literature Project team being sufficiently authorized and qualified for the project

6 Literature Project team utilizing the knowledge and experience of (internal) experts effectively

7 Literature Roles, tasks and responsibilities of all team members defined and appropriate

8 Literature Organization and management of the project

9 Literature Decision making process

10 Literature Communication between members in the project team

11 Literature Reliable and feasible estimation of required money, time and (human) resources

12 Literature Required money, time and (human) resources being available when needed

13 Literature Project team being timely informed about project progress



Risk Reference Framework including risk issues taken from literature and issues resulting from case studies (continued) 14 Literature External parties’ ability and willingness to deliver in time, within budget and conform

technical specifications

15 Literature Contingency options for the parts in the project that have been contracted out to external parties

16 Literature Collaboration within the project team

17 Literature Collaboration with external parties

18 Literature Collaboration between project team and the parent organization

19 Literature Motivated and committed project team

20 Literature Project team paying attention to the right issues

21 Literature Project planning and contingency planning

22 Literature Learning from past experiences

External risks

1 Cases Clearness about who is responsible for PR of this project

2 Cases Clearness abut who are key opinion formers

3 Cases Anticipating effectively to possible negative external reactions

4 Cases Anticipating effectively to legal and political restrictions for markets where the product will be launched

5 Cases Environmental issues

6 Cases Safety issues

8 Cases In case of new technology: checking prior (external) experience

9 Cases Support of key opinion formers

Screening and appraisal

1 Cases Testing and measuring of new product performance targets

2 Cases Testing and measuring trade customer appreciation

3 Cases Testing and measuring consumer appreciation

4 Cases Testing and measuring adverse properties as a consequence of the technological change

5 Cases Credibility of the (internal) measures to external agencies

6 Cases Tests providing reliable evidence

Source: Keizer et al., 2005



Appendix 4: Assessment of the five different NPD portfolio management methods

Assessment of the five different NPD portfolio management methods

General criteria evaluation Fina

ncial

metho

ds

Prob

abilistic

finan

cial

Strategic

Metho

ds

Checklist and

Scoring Mod

els

Map

ping

approa

ches

1.1. The NPD portfolio management tool is usable and understandable for its users (Kester et al., 2009, Archer and Chasemzadeh, 1999,2007, Loch and Kavadias 2002)

6 3 7 6 7

1.2. The NPD portfolio management tool uses multiple types of methods and criteria (see paragraph 2.4.5) 1 2 7 7 7

1.3. The NPD portfolio management tool is able to rank projects based on pre-‐determined criteria (Archer and Chasemzadeh, 1999,2007) 7 7 7 7 1

1.4. The selection methodology is determined before the NPD portfolio selection starts (Archer and Chasemzadeh, 1999,2007) 7 7 7 7 7

1.5. The NPD portfolio management uses a common measure which enable an equitable comparison (Archer and Chasemzadeh, 1999,2007) 7 7 7 7 7

1.6. The NPD portfolio management tool does not solely depend on financial tools (Cooper and Kleinschmidt, 1999) 1 1 7 7 7

1.7. The NPD portfolio management tool incorporates qualitative criteria (Cooper et al. 1998, 1999) 1 2 7 7 7

Most methods are easy to understand, only the determination of the probabilities of probabilistic financial methods could be a cumbersome enterprise. Strategic methods and scoring model assess projects on multiple criteria and mapping approaches and diagrams can depict multiple criteria. However financial and probabilistic financial methods only assess a project only on a single (financial) metric. When every method is applied on the same manner it is possible to equitable compare different projects.

Process criteria evaluation 2.1. The NPD portfolio method is divided in phases (Archer and Chasemzadeh,

1999,2007) 1 1 1 1 1

2.2. A project is re-‐evaluated each time it transfers to another stage (Cooper, 1990, 2008, Archer and Chasemzadeh, 1999,2007) 4 4 4 4 4

2.3. Individual project development and evaluation are based on the Stage Gate® process (Cooper, 1990, 2008, Archer and Chasemzadeh, 1999,2007)

4 4 4 4 4

2.4. The NPD portfolio is reviewed periodically (Cooper et al., 1998, 1999, Archer and Chasemzadeh, 1999, 2007, Chao and Kavadias, 2007) 4 4 4 4 4

2.5. Each project is assessed individually on its (financial) value (Archer and Ghasemzadeh, 1999). 7 7 1 1 1

2.6. The NPD portfolio management takes into account the interdependencies between projects (Archer and Ghasemzadeh, 1999) 1 1 1 1 1

2.7. The NPD portfolio management tool takes into account the time dependent nature of project resource selection (Archer and Ghasemzadeh, 1999)

1 1 1 1 1

Financial and probabilistic methods are not able to align the portfolio to the strategy because these methods only assess a project on a single metric while a strategic assessment involves multiple criteria. Strategic methods and scoring models are able to align the strategy, and they are both bottom-‐up and top-‐down. Poor 1 2 3 4 5 6 7 Excellent



Assessment of the five different NPD portfolio management methods (continued)

Strategic criteria evaluation Fina

ncial

metho

ds

Prob

abilistic

finan

cial

Strategic

Metho

ds

Checklist and

Scoring mod

els

Map

ping

approa

ches

3.1. The NPD portfolio management tool assesses projects on strategic fit (Cooper and Kleinschmindt, 1995, Cooper et al., 1998) 1 1 7 5 4

3.2. The strategy is determined before the NPD portfolio selection process starts (Archer and Chasemzadeh, 1999,2007) 1 1 7 5 4

3.3. Does the strategy consists of the who, what, how, why and threats (Markides, 1999, Loch and Tapper, 2002) or consist out of four perspectives (financial, customer perspective, internal business process and learning and growth perspective) (Kaplan and Norton, 1996)

1 1 7 5 4

3.4. The NPD portfolio management tool is able to align the portfolio to the strategy (Cooper et al. 1998, 1999) 1 1 7 5 4

3.5. The NPD portfolio management tool is top-‐down and bottom-‐up (Cooper et al., 1998) 3 3 7 4 4

3.6. The strategy must make clear where and how many resources should be spent (Cooper et al., 1998) 1 1 1 1 1

3.7. Projects should are classified in advance of the portfolio selection (e.g. strategic Buckets) (Cooper et al., 1998, 1999, Archer and Chasemzadeh, 1999, 2007, Chao and Kavadias, 2007)

1 1 1 1 1

3.8. Projects are assigned to buckets (Barczak et al., 2009) 1 1 1 1 1

Financial criteria evaluation

4.1. The NPD portfolio management tool strives to maximize the value of the portfolio (Cooper et al., 1998, 1999) 3 3 6 6 1

4.2. The financial metric incorporates the cost of development and launch (Cooper et al., 1998) 7 7 1 1 1

4.3. The financial metric has no bias towards long term radical projects (Chao and Kavadias, 2007) 1 5 7 7 4

4.4. The financial metric captures the flexibility of the development process (Newton et al., 2004) 1 7 4 5 1

4.5. The financial metric is constrained by a critical resource (Cooper et al., 1998) 7 7 1 1 1

4.6. The financial metric ignores sunk cost (Cooper et al., 1998) 7 7 7 7 7 4.7. The financial metric incorporates the probability of success or failure

(Cooper et al., 1998, Pennings and Lint, 1997) 1 7 1 1 1

4.8. The financial metric uses the Minimum Acceptable Rate of Return (MARR) and Time Value of Money (TVM) (Remer and Nieto, 1995) 4 4 1 1 1

4.9. The financial metric is user friendly and not too complex or appear as a black box (Faulkner, 1996, Cooper et al., 1999, Kester et al., 2009). 7 4 1 1 1

As mentioned in chapter X (goal 1) financial methods are often used of maximizing the value of the portfolio they seldom succeed in it. Scoring models on the other hand do succeed in composing an NPD portfolio with high value projects.

Balance criteria evaluation

5.1. The NPD portfolio tool is able to visually depict the portfolio (Cooper et al., 1998) 1 1 1 1 7

5.2. Management has defined what balance and imbalance is (Cooper et al., 1998) 1 1 1 1 4

5.3. The NPD portfolio only shows, the for management, most important diagrams (Cooper et al., 1998) 1 1 1 1 7

5.4. The Bubble diagram’s X and Y-‐axis are based on a organization’s strategic goals and objectives Cooper et al., 1998) 1 1 1 1 7




Assessment of the five different NPD portfolio management methods (continued)

Risk criteria assessment Fina

ncial

metho

ds

Prob

abilistic

finan

cial

Strategic

Metho

ds

Checklist and

Scoring mod

els

Map

ping

approa

ches

6.1. The NPD portfolio management tool assesses projects on risk (Halman and Keizer, 1994. Keizer et al., 2005 1 4 7 7 1

6.2. Risk is assessed on three dimensions; likelihood, impact, and the ability to do something about it (Halman and Keizer, 1994) 1 4 5 5 1

6.3. Risk is assessed on a sufficient number of criteria (Keizer et al., 2005) 3 3 3 3 3

Probabilistic financial methods, strategic methods and scoring models are all able to assess a project on risk. Because they are able to assess risk, they also should be able to assess each risk on the three dimensions.

Usability criteria evaluation

7.1. Governance roles and responsibilities are defined (Cooper, 2008) 1 1 1 1 1 7.2. The NPD portfolio is periodically assessed (Cooper, 2008) 7 7 7 7 7 7.3. The NPD portfolio management method is user friendly and not too

complex or appears as a black box (Faulkner, 1996, Cooper et al., 1999, Kester et al., 2009)

5 4 4 4 4

7.4. Deliverables are clearly defined for each portfolio assessment (Cooper, 2008) 6 6 6 6 6

7.5. Deliverables are focused on the essential information needed for NPD portfolio management decisions (Cooper, 2008) 5 5 4 4 4

7.6. Templates are not overly detailed and be limited to the essential information needed for NPD portfolio management decisions (Archer and Ghazemzadeh, 1998, Cooper, 2008)

3 3 2 2 2

There is the risk with all these methods that they are too detailed, this risk is especially high with strategic methods, scoring models and diagrams (see paragraph 2.3.5)

Overall score 3,09 3,48 3,98 3,73 3,50




Appendix 5: Overview of Kalypso model performance per criteria ranked by score

Overview of Kalypso model performance per criteria ranked by score 1.4. The selection methodology should be determined before the NPD portfolio selection starts (Archer

and Chasemzadeh, 1999,2007) 7

1.5. The NPD portfolio management tool should use a common measure which enable an equitable comparison (Archer and Chasemzadeh, 1999,2007)

7

2.2. A project is re-‐evaluated each time it transfers to another stage (Cooper, 1990, 2008, Archer and Chasemzadeh, 1999,2007)

7

2.4. The NPD portfolio is reviewed periodically (Cooper et al., 1998, 1999, Archer and Chasemzadeh, 1999, 2007, Chao and Kavadias, 2007)

7

2.5. Each project is assessed individually on its (financial) value (Archer and Ghasemzadeh, 1999). 7 3.1. The NPD portfolio management tool must assess projects on strategic fit (Cooper and Kleinschmindt,

1995, Cooper et al., 1998) 7

4.6. The financial metric should ignore sunk cost (Cooper et al., 1998) 7 4.9. The financial metric is user friendly and not too complex or appear as a black box (Faulkner, 1996,

Cooper et al., 1999, Kester et al., 2009). 7

5.1. The NPD portfolio tool must be able to visually depict the portfolio (Cooper et al., 1998) 7 7.2. The NPD portfolio is periodically assessed (Cooper, 2008) 7 1.7. The NPD portfolio management tool should incorporate qualitative criteria (Cooper et al. 1998, 1999) 6 2.1. The NPD portfolio method is divided in phases (Archer and Chasemzadeh, 1999,2007) 6 7.1. Governance roles and responsibilities must be defined (Cooper, 2008) 6 7.4. Deliverables must be clearly defined for each portfolio assessment (Cooper, 2008) 6 1.1. The NPD portfolio management tool is usable and understandable for its users (Kester et al., 2009,

Archer and Chasemzadeh, 1999,2007, Loch and Kavadias 2002) 5

1.3. The NPD portfolio management tool is able to rank project based on pre-‐determined criteria (Archer and Chasemzadeh, 1999,2007)

5

4.1. The NPD portfolio management tool should strive to maximize the value of the portfolio (Cooper et al., 1998, 1999)

5


5

7.6. Templates should not be overly detailed and be limited to the essential information needed for NPD portfolio management decisions (Archer and Ghazemzadeh, 1998, Cooper, 2008)

5

2.3. Individual project development and evaluation are based on the Stage Gate® process (Cooper, 1990, 2008, Archer and Chasemzadeh, 1999,2007)

4

3.7. Projects are classified in advance of the portfolio selection (e.g. strategic Buckets) (Cooper et al., 1998, 1999, Archer and Chasemzadeh, 1999, 2007, Chao and Kavadias, 2007)

4

4.3. The financial metric should have no bias towards long term radical projects (Chao and Kavadias, 2007)

4

5.4. The diagram’s X and Y-‐axis are based on a organization’s strategic goals and objectives Cooper et al., 1998)

4

7.3. The NPD portfolio management method is user friendly and not too complex or appears as a black box (Faulkner, 1996, Cooper et al., 1999, Kester et al., 2009)

4


3

3.4. The NPD portfolio management tool is able to align the portfolio to the strategy (Cooper et al. 1998, 1999)

3

3.5. The NPD portfolio management tool is top-‐down and bottom-‐up (Cooper et al., 1998) 3 5.3. The NPD portfolio should only show the for management most important diagrams (Cooper et al.,

1998) 3

6.3. Risk is assessed on a sufficient number of criteria (Keizer et al., 2005) 3 7.5. Deliverables are focused on the essential information needed for NPD portfolio management

decisions (Cooper, 2008) 3

3.2. The strategy is determined before the NPD portfolio selection process starts (Archer and Chasemzadeh, 1999,2007)

2

2.6. The NPD portfolio management tool should take into account the interdependencies between projects (Archer and Ghasemzadeh, 1999)

1

2.7. The NPD portfolio management tool should take into account the time dependent nature of project resource selection (Archer and Ghasemzadeh, 1999)

1



Overview of Kalypso model performance per criteria ranked by score (continued) 3.3. A clear strategy consists of the who, what, how, why and threats (Markides, 1999, Loch and Tapper,

2002) or consist out of four perspectives (financial, customer perspective, internal business process and learning and growth perspective) (Kaplan and Norton, 1996)

1

3.6. The strategy must make clear where and how many resources are spent (Cooper et al., 1998) 1 3.8. Projects are assigned to buckets (Barczak et al., 2009) 1 4.2. The financial metric should incorporate the cost of development and launch (Cooper et al., 1998) 1 4.4. The financial metric should capture the flexibility of the development process (Newton et al., 2004) 1 4.5. The financial metric is constrained by a critical resource (Cooper et al., 1998) 1 4.7. The financial metric should incorporate the probability of success or failure (Cooper et al., 1998,

Pennings and Lint, 1997) 1

4.8. The financial metric should use the Minimum Acceptable Rate of Return (MARR) and Time Value of Money (TVM) (Remer and Nieto, 1995)

1

5.2. Management must define what balance and imbalance is (Cooper et al., 1998) 1 6.2. Risk is assessed on three dimensions; likelihood, impact, and the ability to do something about it

(Halman and Keizer, 1994) 1




Appendix 6: Decision rules for classification into risk classes Decision rules for classification into risk classes

Certainty Ability of team to influence the course of action

Relative importance to project success

RISK CLASS

Certainty Ability of team to influence the course of action

Relative importance to project success

RISK CLASS

1. * * * F

28. * * ? M-‐F 2. * * 0 L

29. * ? * H-‐F

3. * 0 * M

30. ? * * M-‐F 4. 0 * * H

31. * ? ? L-‐F

5. 0 0 * L

32. ? * ? L-‐F 6. 0 * 0 L

33. ? ? * L-‐F

7. * 0 0 L

34. ? ? ? S-‐F 8. 0 0 0 S

35. ? 0 0 L

9. * * m H

36. 0 ? 0 L 10. * m * H

37. 0 0 ? L

11. m * * H

38. ? ? 0 S-‐M 12. * m m M

39. ? 0 ? S-‐H

13. m * m M

40. 0 ? ? S-‐M 14. m m * M

41. * ? 0 L-‐M

15. m m m M

42. * 0 ? L-‐H 16. 0 * m M

43. 0 * ? L-‐M

17. * 0 m M

44. 0 ? * L-‐M 18. 0 m * M

45. ? 0 * L-‐H

19. * m 0 M

46. ? * 0 L-‐M 20. m * 0 M

47. * ? m M-‐H

21. m 0 * M

48. * m ? M-‐H 22. 0 0 m L

49. m ? * M-‐H

23. 0 m 0 L

50. m * ? M-‐H 24. m 0 0 L

51. ? m * M-‐H

25. 0 m m M

52. ? * m M-‐H 26. m m 0 M

53. m ? 0 L-‐M

27. m 0 m M

54. m 0 ? L-‐M F = Fatal risk 55. 0 ? m L-‐M H = High risk 56. 0 m ? L-‐M M = Medium risk 57. ? 0 m L-‐M L = Low risk 58. ? m 0 L-‐M S = Safe, no risk 59. ? m m L-‐M A combination of classes means that the risk team should work out whether the disagreement can be resolved and hence a single classification can be achieved. If consensus can’t be achieved the worst possible case should be assumed.

60. m ? m M

61. m m ? M

62. ? ? m L-‐H

63. ? m ? L-‐H

64. m ? ? L-‐H Source: Keizer et al. (2002), page 230

eindhoven university of technology master npd portfolio ... · npd portfolio management: the...

Documents