series editors technica a scientiae rerum naturalium...

ABCDEFG

UNIVERS ITY OF OULU P.O.B . 7500 F I -90014 UNIVERS ITY OF OULU F INLAND

A C T A U N I V E R S I T A T I S O U L U E N S I S

S E R I E S E D I T O R S

SCIENTIAE RERUM NATURALIUM

HUMANIORA

TECHNICA

MEDICA

SCIENTIAE RERUM SOCIALIUM

SCRIPTA ACADEMICA

OECONOMICA

EDITOR IN CHIEF

PUBLICATIONS EDITOR

Professor Mikko Siponen

University Lecturer Elise Kärkkäinen

Professor Hannu Heusala

Professor Olli Vuolteenaho

Senior Researcher Eila Estola

Information officer Tiina Pistokoski

University Lecturer Seppo Eriksson


Publications Editor Kirsti Nurkkala

ISBN 978-951-42-9147-0 (Paperback)ISBN 978-951-42-9148-7 (PDF)ISSN 0355-3213 (Print)ISSN 1796-2226 (Online)

U N I V E R S I TAT I S O U L U E N S I SACTAC

TECHNICA


TECHNICA

OULU 2009

C 324

Pekka Belt

IMPROVING VERIFICATION AND VALIDATION ACTIVITIES IN ICT COMPANIES—PRODUCT DEVELOPMENT MANAGEMENT APPROACH

FACULTY OF TECHNOLOGY,DEPARTMENT OF INDUSTRIAL ENGINEERING AND MANAGEMENT,UNIVERSITY OF OULU

C 324

ACTA

Pekka Belt

C324etukansi.fm Monday, May 18, 2009 10:08 AM

A C T A U N I V E R S I T A T I S O U L U E N S I SC Te c h n i c a 3 2 4

PEKKA BELT

IMPROVING VERIFICATION AND VALIDATION ACTIVITIESIN ICT COMPANIES—PRODUCT DEVELOPMENT MANAGEMENT APPROACH

Academic dissertation to be presented with the assent ofthe Faculty of Technology of the University of Oulu forpublic defence in Raahensali (Auditorium L10), Linnanmaa,on 17 June 2009, at 12 noon

OULUN YLIOPISTO, OULU 2009

Copyright © 2009Acta Univ. Oul. C 324, 2009

Supervised byProfessor Harri Haapasalo

Reviewed byProfessor Hannu KärkkäinenProfessor Asko Miettinen

ISBN 978-951-42-9147-0 (Paperback)ISBN 978-951-42-9148-7 (PDF)http://herkules.oulu.fi/isbn9789514291487/ISSN 0355-3213 (Printed)ISSN 1796-2226 (Online)http://herkules.oulu.fi/issn03553213/

Cover designRaimo Ahonen

OULU UNIVERSITY PRESSOULU 2009

Belt, Pekka, Improving verification and validation activities in ICTcompanies—product development management approachFaculty of Technology, Department of Industrial Engineering and Management, University ofOulu, P.O.Box 4610, FI-90014 University of Oulu, Finland Acta Univ. Oul. C 324, 2009Oulu, Finland

AbstractThe main motive for this research arises from the fact that the research has been scarce onverification and validation (V&V) activities from the management viewpoint, even though V&Vhas been covered from the technical viewpoint. There was a clear need for studying themanagement aspects due to the development of the information and communications technology(ICT) sector, and increased significance of V&V activities.

ICT has developed into a turbulent, high clock-speed sector and the importance of V&Vactivities has increased significantly. As a consequence, companies in the ICT sector require ideasfor improving their verification and validation activities from the product developmentmanagement viewpoint.

This study approaches the above mentioned goal from four perspectives: current V&Vmanagement challenges, organisational and V&V maturities, benchmarking another sector, anduncertainty during new product development (NPD). This dissertation is qualitative in nature andis based on interviewing experienced industrial managers, reflecting their views against scientificliterature. The researcher has analysed the obtained material and made conclusions.

The main implications of this doctoral dissertation can be concluded as a need to overcome thecurrent tendency to organise through functional silos, and low maturity of V&V activities.Verification and validation activities should be viewed and managed over the entire NPD process.This requires new means for cross-functional integration. The maturity of the overall managementsystem needs to be adequate to enable higher efficiency and effectiveness of V&V activities.There are pressures to shift the emphasis of V&V to early NPD and simultaneously delay decision-making in NPD projects to a stage where enough information is available. Understandingenhancing V&V methods are a potential way to advance towards these goals.

Keywords: ICT, new product development, testing, validation, verification

5

Acknowledgements

My preconception for dissertation work for doctoral degree was that with good guidance and hard work, it could be done in a couple of years. The greatest disappointment was the sluggish start, as there was no ready template for the work. In addition, I had to split my time and work on a number of matters aside my dissertation research. At the end, it took me almost three years to come up with the doctoral dissertation, which, however, is not terribly bad result.

Luckily, the project team had good guys to complement each others know-how. From the beginning, we knowingly discussed among each other, how to advance our individual dissertations. These discussions gradually transformed into review sessions, where different issues were taken forward, and written ideas were commented. Continuous efforts to write, combined with regularly held reviews, proved to be the key to advance the doctoral dissertations. Verbal communication is necessary, but not sufficient, hard work and written communication are a necessity between discussions. Fortunately, Janne and Matti proved to be true work horses, they had qualities that complemented my own, enabling successful team efforts.

I would like to sincerely thank Professor Harri Haapasalo for his comments on my Journal articles and the Introduction of this dissertation. His sharp academic views helped me perceiving, especially how to fulfill the academic requirements for theory. In addition, Harri gave numerous comments, leading me to gradually understand what it means to write academic articles. I would also like to thank Professor Pekka Kess, the head of the department, for giving me the opportunity to do this work, and for his contribution. Pekka has proved a true online person, who immediately gives his comments. Both Harri and Pekka have made significant efforts to build up finance from small pieces, truly enabling the realisation of this dissertation. I also send my gratitude to Dr Tauno Jokinen for obtaining the initial finance for the Tekes project, under which this work was initiated.

Writing a compilation dissertation alone is a fairly laborious way to write a doctoral dissertation, not to mention contributing to three simultaneous compilation dissertations. It might have been possible to obtain a doctoral degree with less effort, should things be done differently. On the other hand, this way I have obtained greater academic merits and I have been a co-author in fourteen Journal articles, thirteen of which deal with verification and validation. For the successful completion of the project, I would like to thank you Janne for being

6

hardworking, systematic and goal oriented, complemented with your excellent command of English language. These skills have been vital for the successful realisation of this project. I would like to thank you, Matti, for your incomprehensible ability to understand fuzzy materials that are often written in indigestible English. Your capability to organise different elements to form a logical continuation, together with your hard work with the illustrations and formats, has greatly helped us. In addition, Matti, you have had the humbleness to complete relevant, sometimes routine project work that many lazier complainers would try to push to others.

I would like to thank our industrial partners for providing the required access and data. Also, many thanks to Dr Pekka J. Heinonen of Nokia for providing his views over verification and validation. Many thanks to Tekes, Tauno Tonning Foundation, and Riitta & Jorma Takanen Foundation for their financial support.

I would also like to thank the pre-examiners of this dissertation, Professor Emeritus Asko Miettinen and Professor Hannu Kärkkäinen for their valuable comments and recommendations.

Many thanks to my wife and my children for your support and patience. Without this support I would have been unable to carry out this work.

Oulu, May 2009 Pekka Belt

7

List of abbreviations and definitions

B2B Business to Business B2C Business to Customer CE Chief Engineer CMM Capability Maturity Model CMMI Capability Maturity Model® Integration ICT Information and Communication Technology IEM Industrial Engineering and Management NPD New Product Development OEM Original Equipment Manufacturer P-CMM People Capability Maturity Model PD Product Development R&D Research and Development RQ Research Question SW-CMM Software Capability Maturity Model TEKES National Technology Agency of Finland TMM Testing Maturity Model TNPD Toyota’s New Product Development UE V&V Understanding Enhancing Verification and Validation V&V Verification and Validation V2M2 Verification Validation Maturity Model

The terminology used in this dissertation includes concepts of verification, validation and testing. Verification is widely understood as a method to prove the compliance with the specifications. The aim of validation is to prove that the user is satisfied. Testing is the best known V&V method to check a complete system or its part. Testing include, functional, environmental, and reliability tests. Testing can include, for example, module, integration, system, and acceptance tests. However, the term testing is often understood, especially in software development, broadly to cover both verification and validation. As this dissertation is managerial in nature, it is not critical for the reader to attempt to make a distinction between these terms.

9

List of original publications

This dissertation is based on the following publications:

I Belt P, Oiva-Kess A, Harkonen J, Mottonen M & Kess P (2009) Organisational maturity and functional performance. International Journal of Management and Enterprise development 6(2): 147–164. DOI: 10.1504/IJMED.2009.022624.

II Belt P, Harkonen J, Mottonen M, Kess P & Haapasalo H (2008) Improving the efficiency of verification and validation. International Journal of Services and Standards 4(2): 150–166. DOI: 10.1504/IJSS.2008.016630.

III Belt P, Haapasalo H, Harkonen J, Mottonen M & Kess P (in press) Improving product development in different types of ICT companies. International Journal of Innovation and Learning.

IV Belt P, Harkonen J, Mottonen M, Kropsu-Vehkapera H & Haapasalo H (in press) Technological uncertainty and verification & validation activities. International Journal of Innovation and Learning.

The first two articles are published, and the other two have gone through a double blind review process and have been accepted for Journal publication in a forthcoming issue. The author of this dissertation has been the primary author in all of the original publications. The researcher has been responsible for formulating the research problems, theoretical base, devising research questions, coordinating the collection of empirical material, analysing the material, drawing conclusions, and finally being the primary author for all four articles. With regards to article one, the author combined a co-author’s previous research to the topic of this dissertation. The role of the co-authors has mainly been commenting and giving valuable feedback to the articles.

11

Table of contents

Abstract Acknowledgements 5 List of abbreviations and definitions 7 List of original publications 9 Table of contents 11 1 Introduction 13

1.1 Background and research environment ................................................... 13 1.2 Objectives and scope............................................................................... 15 1.3 Research approach .................................................................................. 17 1.4 Research realisation and dissertation structure ....................................... 19

2 Theoretical foundation 21 2.1 Product development management ......................................................... 22 2.2 Quality management in the context of verification and validation ......... 25 2.3 Testing in electronics hardware, software and embedded systems ......... 26

2.3.1 Testing in electronics development .............................................. 26 2.3.2 Testing in software development .................................................. 27 2.3.3 Testing of systems with HW and SW integrated .......................... 28

2.4 Maturity models ...................................................................................... 30 2.5 Managing technological uncertainty ....................................................... 32

2.5.1 Simultaneous technological alternatives during product development ................................................................................. 32

2.5.2 Project portfolio management and technological uncertainty.................................................................................... 33

2.5.3 Platform development .................................................................. 34 2.5.4 Verification & validation and NPD uncertainty............................ 34 2.5.5 Synthesis on managing technological uncertainty........................ 35

3 Research contribution 37 3.1 Organisational maturity and functional performance.............................. 37 3.2 Improving the efficiency of verification and validation.......................... 38 3.3 Improving product development in different type of ICT

companies ............................................................................................... 41 3.4 Technological uncertainty and verification & validation activities......... 42

4 Discussion 47 4.1 Theoretical implications.......................................................................... 47 4.2 Managerial implications.......................................................................... 48

12

4.3 Reliability and validity ............................................................................ 52 4.4 Recommendations for further research ................................................... 54

5 Summary 55 References 59 Original publications 71

13

1 Introduction

1.1 Background and research environment

This dissertation is positioned within the field of industrial engineering and management (IEM) that deals with enterprise management and efficient & effective product/ service development, production and supply chain. The need for this dissertation was initiated based on the needs of Finnish information and communications technology (ICT) companies, operating in telecommunications, electronics and related areas.

The ICT industry has expanded rapidly during the past decades and has changed and witnessed a transformation in terms of frequent technology changes, increase in complexity, time-to-market pressures and the decrease in physical dimensions (Moore 1999, D’Aveni 1995). Today’s ICT business sector is characterised by technology and market uncertainties and an inability to predict the future (Deszca et al. 1999, Suikki 2007, Leeman & Winer 1997). One way to gain profit and increase market share is to launch new high quality and innovative products earlier than competitors, resulting in continuous demands for shortening design cycles (e.g. Costanzo 2004). In addition, fierce price erosion forces companies to reduce costs. (e.g. Helo 2004)). Shorter development times have resulted in an increased need to engage customers and subcontractors already during product development (Perttula 2007). Another challenge for ICT companies is that requirements for products typically change during product development (e.g. Hsu et al. 2008, Nuseibeh & Easterbrook 2000, Gupta et al. 2003). Additionally, requirements are interpreted differently in different parts of organisations (Weber & Weisbrod 2003, Skander et al. 2008).

The current trend in the ICT industry is that the increasing number of functionalities is created through software. The nature of software development is moving from traditional waterfall models towards incremental software development process (e.g. Haapasalo & Ylihoikka 2004).

Technical superiority of products and new functionalities are not enough for companies to succeed, but one must also invest in developing processes and people. (e.g. Pisano & Wheelwright 1995, Cooper et al. 2004, Drejer 2008, Nonaka & Takeuchi 1995, Gupta et al. 2007). Products being more complicated than ever, and customer segments more fragmented, addressing the needs of both internal and external customers has become a challenge for product development

14

and production. Successful product development requires multidisciplinary approaches and necessitates the integration of engineers, industrial designers and marketing personnel. (Cooper et al. 2004, Gupta et al. 2007).

Verification and validation (V&V) is a critical issue for modern high technology product development. It has been claimed that it is impossible to design and manufacture products without defects (e.g. Woodward & Hennell 2005, Black 2004). Rumney (2003) states that “the more complex a system becomes, the more resources are required to test it, but clearly from the perspective of the entire telecommunications industry, the growing complexity and cost of test can only be considered as painful”. In the literature, V&V activities have been estimated to take between 30 to 60 percent of the entire costs in high-technology product development. V&V has become a critical factor (e.g. Kung et al. 1998, Gilb 2005, Andersson & Runeson 2002). However, the significance of V&V activities is not always understood.

The cost of fixing errors increases towards the field use, where the costs can be up to 1000 times higher than in the requirements phase (Boehm 1981), which could just be fatal for a business, considering the size of some of today’s development projects. Changes in requirements, which are often made late in the development, cause a need to repeat testing, causing unnecessary waste of resources. Unfortunately, the traditional system development models often understand V&V as a separate error detection process that occurs just after the product integration, close to the product launch. (e.g. Pressman 2004, Mooz et al. 2003, Salustri & Parmar 2003, Forsberg & Mooz 1992).

The technical aspects of V&V are well documented, as well as V&V in more specific areas such as pure software or hardware. The literature shows that the management aspects of V&V are insufficient, even though verification and validation is seen as a significant matter for companies (e.g. Perttula 2007, Maatta et al. 2009, Andersson & Runeson 2002). The literature is scarce on comprehensive V&V management viewpoint, giving justification for this dissertation. This dissertation aims to find ideas for different type ICT companies to improve their verification and validation activities from the product development management viewpoint.

15

1.2 Objectives and scope

The main motive for this research arises from the fact that the research has been insufficient on verification and validation activities from the management viewpoint, even though V&V has been covered from the technical viewpoint. There is a clear need for studying the management aspects due to the development of the ICT sector, and increased significance of V&V activities. ICT sector is a multifaceted business sector, comprising of areas, including manufacturing industry, services, telecommunications, and digital communication. V&V and testing are also discussed in a function specific manner in the literature, such as software, testing of integrated circuits. This dissertation aims to take a managerial approach on V&V activities, allowing viewing these issues more broadly outside their traditional silos. However, where required, these issues are studied area specifically. Thus this dissertation does not concentrate solely on hardware, software, or system products, but is adopting a more general approach.

The research problem attempted to address in this dissertation is stated as follows:

Companies in the ICT sector require research efforts and new knowledge for improving their verification and validation activities from the product development management viewpoint.

This problem is studied from four complementary perspectives – organisational and V&V maturities, current V&V management challenges, benchmarking another sector, and uncertainty during NPD – based on which four research questions (RQ) are formed (Table 1). Both the research problem and the research questions are industry based, and have been derived from discussions with industry representatives. The research problem has remained the same during the dissertation project, but the research questions have been refined along the progress. The selected viewpoints could be different, however, these are the ones selected by the researcher, as they seemed logical at the decision times.

16

Table 1. Research questions.

RQ # Research question

RQ1 How does the organisational maturity reflect the V&V maturity?

RQ2 What is the development path of V&V activities and what are the challenges and possible

solutions?

RQ3 How can ICT companies benefit from benchmarking the NPD management practices against a

mature business sector?

RQ4 How are different types of companies coping with uncertainty in high tech NPD, and how can

V&V aid this?

Each research question is answered with a Journal article. Each article provides a partial solution to the research problem. The contributions of these articles are combined in this dissertation summary.

Table 2 lists the articles and combines them with the research questions.

Table 2. Research papers overview.

Article RQ # Title Publication

I RQ1 Organisational maturity and

functional performance

International Journal of Management and

Enterprise development

II RQ2 Improving the efficiency of

verification and validation

International Journal of Services and Standards

III RQ3 Improving product development in

different type of ICT companies

International Journal of Innovation and Learning

IV RQ4 Technological uncertainty and

verification & validation activities

International Journal of Innovation and Learning

Figure 1 illustrates the positioning of the four research articles into managerial and functional context. The first article clarifies managerial maturity and functional V&V maturity, and their interdependence. The second paper studies V&V challenges and proves that addressing them cannot be done adequately at the functional level. The third article complements the managerial viewpoint by comparing the NPD practices of the ICT industry against those of a mature business sector. V&V is an integral part of product development in the ICT sector. Justification for this article is that V&V management must be seen together with other NPD practices. The fourth article partially returns to the functional level by giving recommendations on the required V&V mode in different NPD stages, and giving recommendations on tackling NPD uncertainty in general.

17

Fig. 1. Research framework.

1.3 Research approach

The researcher faces ontological, epistemological and ethical questions, when he considers scientific research from a philosophical viewpoint: how can he trust that reality is based on scientific research, how is this scientific knowledge obtained and whether it is scientific, and whether the researcher acts unethically against scientific community and abuses his research (e.g. Lancaster 2005).

Ontology can be interpreted as a reality where studied phenomena are understood to locate and the way the studied phenomena relate to this reality. Scientific research typically makes ontological pre-conceptions on the nature of studied issues. Ontology answers questions relating to objectivity and subjectivity of the studied reality. Ontological foundation has an influence over the selection of theory and concepts. (e.g. Anttila 2005, Harisalo 2008).

This dissertation research is qualitative in nature and utilises the principles of empiricism, applying mainly inductive reasoning. The research is mainly descriptive with a slight touch of normative approach in giving recommendations (see Figure 2). All the research papers are based on semi-structured interviews. The number of interviews for each article varied between 22 and 53, the total number of interviews being close to one hundred. The main motivations for the selected research approach were the facts that the topic has not been adequately covered by previous studies, and that the researcher had both a need and an access to the views of experienced industrial managers through qualitative interviews.

Managerial levelManagerial level

FunctionallevelFunctionallevel

1. Managerial & functional

maturity

2.V&V

challenges

3. Benchmarking

managerial practices

4. Uncertainty

& V&V

18

Fig. 2. Research approach (modified from Kasanen et al. 1993).

This research utilises qualitative approach in order to enable interviewees to express their viewpoints, opinions and experiences without limitations as far as possible. Qualitative research aims to understand a certain phenomenon, to describe its functioning and to provide a theoretically sound interpretation of this phenomenon. The goal is to describe true life realities. (Denzin & Lincoln 2005, Eskola & Suoranta 1998, Hirsjärvi et al. 2008)

Qualitative research gives a certain degree of freedom to the researcher in relation to planning and executing his research. However, on the other hand, the researcher is not totally free from his own values and limitations. Consequently, it is not possible for the research to be fully objective as the researcher and the studied phenomenon are connected. True objective knowledge does not exist, but all research is subjective as the researcher’s understanding has an influence over the obtained results. (Hirsjärvi et al. 2008, Tuomi & Sarajärvi 2006, Eskola & Suoranta 2008)

Different methods and paradigms have been developed to support scientific approaches. It is imperative to select a method that supports the research problem in question. Descriptive research aims to describe phenomena by describing objects and processes in an attempt to increase understanding. Normative research aims for solutions that can be utilised for developing current activities, or even creating something new. (e.g. Olkkonen 1993).

Qualitative research aims to understand a studied phenomenon, to clarify its meaning and significance. Samples used in qualitative research typically are discretionary to the researcher, and objects studied may not be high in numbers, but are studied thoroughly. This highlights the quality of input material.

19

Nevertheless, the sample size must be covering enough in relation to the intended analysis and interpretation (Siggelkow 2007, Patton 2002, Marshall & Rossmann 1998, Eskola & Suoranta 1998, Hirsjarvi & Huttunen 1995).

Empiricism is a theory of knowledge emphasising the role of experience, especially experience that is based on perceptual observations. Knowledge is seen possible to obtain through inductive reasoning. Inductive reasoning is typical for qualitative analysis, aiming to make generalisations and conclusions based on factors arising from the material under scrutiny. Material is attempted to analyse in a multifaceted manner in adequate detail, bringing up important themes (Eskola & Suoranta 1998, Manktelow 1999).

1.4 Research realisation and dissertation structure

The research was initiated during a large project, involving industrial companies and university researchers. It was found relevant during working sessions to collect, analyse and summarise the views of experienced industry managers. The project initiated a series of interview studies. These interviews, totalling almost one hundred, formed the foundation for this research. This material was then complemented by literature reviews and workshops. The researcher has participated in collecting the research material, but more importantly has been responsible for analysing and drawing conclusions.

The research process typical to all the four articles, this dissertation is founded on is described in Figure 3. The chosen topic is first studied through literature to familiarise with the subject and to understand what other authors have written previously. Based on the obtained understanding the research focus was defined. Interview structure and questionnaire were then formulated. Potential interview candidates were identified with the help of industry representatives to assure adequate coverage. Industrial interviews were carried out in the ICT sector, in order to clarify the issues under study. The interviews followed a semi-structured thematic interview approach (see e.g. Merton et al. 1990) and were conducted informally, in a qualitative manner, allowing the interviewees to explain and clarify the cases and topics as entities. Interviews were recorded and transcribed in order to assure full utilisation of the views of experienced industrial managers. All the individual interview results were analysed according to chosen focus, and when appropriate, compared to the literature. Finally, conclusions were drawn based on the analysis. Individual research processes have been described in more detail in each article (see the Appendix).

20

Fig. 3. Research process typical to all the articles.

Table 3 presents the number of industrial interviews (equals to the number of interviewees) and the number of companies for each research article.

Table 3. Number of industrial interviews for each article.

Article # Title Number of

interviews

Number of

companies

I How does the organisational maturity reflect

functional performance?

60 17

II What is the development path of V&V activities and

what are the challenges and possible solutions?

22 7

III How can ICT companies benefit from

benchmarking the NPD management practices

against a mature business sector?

53 11

IV How are different types of companies coping with

uncertainty in high tech NPD, and how can V&V

aid this?

50 10

This dissertation consists of four individual articles and this summary. The summary is organised as follows: Chapter 2 presents the theoretical foundation for the research. Chapter 3 summarises the research contribution of the four articles attached as an appendix. In Chapter 4 the overall findings of the research are concluded. Finally, Chapter 5 summarises the research.

21

2 Theoretical foundation

Managing new product development can be considered contain numerous issues, such as: accounting, project management, finance and marketing, product design, law and contracts, integrated product development, human resource management, innovation management, information systems, systems engineering, quality management etc. (e.g. Mantel et al. 2007, Cross 2008, Tidd & Bessant 2009, Blanchard 2008, Dale et al. 2007). Also other management concepts can be seen to be connected integrally to NPD management. These concepts include: product life-cycle management, requirements management, technology management, design for excellence, knowledge management and many others. However, this dissertation does not address each of these issues individually, but concentrates on issues that are relevant for managing verification and validation activities.

This dissertation bases it theoretical foundation on product development management, quality management, V&V in software, hardware and embedded environments, maturity models, and management of technological uncertainty. Selected theories are applied to an extent as required for the purpose of improving verification and validation activities from product development management viewpoint. For practical reasons some relevant theories may have been excluded, which, however, could have been included should the scale be wider than a single doctoral dissertation.

Figure 4 gives a simplified illustration of positioning the dissertation to theory.

22

Fig. 4. Positioning the dissertation to theory.

2.1 Product development management

Common management methods typical to any type of business are functional also for NPD management to a certain degree. However, managing NPD has certain characteristics that are distinct. These characteristics include: dealing with people with a variety of different type of skills, specialists often have a deeper know-how on specific issues than their managers, project nature of NPD, high levels of uncertainty and long feedback times on success or failure. Results are often visible a long time after efforts, once knowledge over market success is available, making any feedback mechanisms very complex. In this type of environment, NPD managers are having to pay special attention on functional communication. (Paasivaara 2005, Leonard-Barton 1998).

NPD management is complex and contains a high number of issues to be considered. Leonard-Barton (1998) simplifies NPD management to include, management systems, technical systems, skills & knowledge base and values & norms. Morgan & Liker (2006) divide NPD management into a socio-technical system of processes, people and tools & technologies.

The agony caused by the complexity of NPD management can be alleviated by utilising different types of models, from the ones describing the NPD process

Requirements engineering

Knowledge management

Systems

engineering

Product development

Product development

Verification & validation

Organisational performance

Uncertainty

Technology

management

Quality management

Human resource

management

V&V

management

23

to different type of system development models. (e.g. Ulrich & Eppinger 1995, Ulrich & Eppinger 2007, Cooper 2001, Pressman 2004, Mooz et al. 2003). System development models include different ones from ad-hoc and waterfall models to V-models and W-models.

Typically NPD process is described as a sequential process. Ulrich & Eppinger (1995) have presented a well-known NPD process including five phases: concept development, system-level design, detail design, testing and refinement, and production ramp-up. Cooper’s (2001) stage-gate process is a conceptual and operational model for new product projects covering issues from idea to launch. The stage-gate model is seen to break the innovation process into a predetermined set of stages. Generally this type of models focus on developers’ interests to verify specifications, not to adjust features to the customers’ even undefined needs (Suikki 2007).

Company size is seen to influence the manner product development is organised. The larger the organisation, the more formal control is needed (e.g. Grimpe 2007, Arvidsson et al. 2003). In general, company size is seen to correlate to innovation rate, supplier involvement, and also with managing development (e.g. Laforet 2008, Wynstra et al. 2001).

Large companies typically have adequate resources and competences to divide NPD activities into smaller tasks, based on either time or specialisation. Bureaucracy, inertia, and communication needs may also cause challenges in large companies. However, when all other things are equal, large companies may generate more new products than small companies (Leenders & Wierenga 2008).

Small companies have fewer resources in their disposal, forcing them to focus their NPD activities, and thus utilise simpler methods. Small companies may also have difficulties in developing parallel alternatives simultaneously, and do not have specialised personnel for all the activities. On the other hand, coordination of activities is easier in small companies with less need for formal methods (e.g. Haapasalo & Ekholm 2004, Anderson & Tushman 1991). Senior management involvement is seen to occur earlier in the small firms (Ledwith et al. 2006).

Different new product development frameworks (e.g. McGrath 1995, McGrath 2001) are typically more suited for large companies, as they are large enough to consider issues, such as product platforms, and involve the full enterprise. Nevertheless, in small companies, product lines are more limited, and it is not appropriate for them to strictly follow the same frameworks as the large ones. Instead, it is seen to better suit them to base product development on

24

derivatives of a core technology (e.g. Ledwith et al. 2006). Figure 5 gives a simplified illustration of the structural level of NPD practices in small and large companies.

Considering supply chain from the perspective of NPD provides new viewpoints. Involving suppliers early and extensively in product development is seen to be one way to enhance product development performance relating to productivity, speed, and quality (van Echtelt et al. 2008). Nevertheless, increased supplier involvement can be challenging, requiring unlearning old behaviour and adjustments in order to successfully integrate and benefit from new resource configurations (van Echtelt et al. 2008). Company size is seen to influence even the company-supplier relationships (e.g. Redondo & Fierro 2007). Also, the resource/competence based view is seen to guide companies in developing and handling relationships with technology suppliers, instead of a traditional product oriented view (Moller et al. 2003).

This dissertation, however, does not view suppliers from the supply chain perspective, but approaches them as independent companies.

Fig. 5. Relative level of NPD structures in small and large companies.

Large

companies

Structured

NPD

Small

companies

Unstructured

NPD

Systemised

NPD

Creativity Under structured

Over structured

Large

companies

Structured

NPD

Small

companies

Unstructured

NPD

Systemised

NPD

Creativity Under structured

Over structured

25

2.2 Quality management in the context of verification and validation

During the quality revolution of the 1980s thousands of people received quality training, for the first time, in a wider scope. The message was mostly as follows: “Do everything right the first time, every time, and there will be no need for testing”. (Davis 1994). However, this statement talks about quality in general, with regard to all kinds of manufacturing and service industries. At the same time, the ICT industry is notably different in terms of frequent technology change, increase in complexity, time-to-market pressures and the decrease in physical dimensions in electronics. These factors set enormous challenges for design, manufacturing, maintenance and procurement, which are unmatched by the challenges of any other industry. In the ICT industry it is impossible to design and manufacture products without defects (Davis 1994, Black 2004, Pol & Veenendaal 1998, Woodward & Hennell 2005). As a consequence, a target of 100% quality is not realistic (Crosby 1980). Unrealistic quality thinking may have contributed to the undervaluation of V&V activities. (e.g. Davis 1994, Black 2004, Pol & Veenendaal 1998).

Traditionally, quality costs have been divided according to Feigenbaum’s Prevention, Appraisal, and Failure (PAF) model (Feigenbaum 1961) into three separate categories:

1. Prevention costs (costs of doing it right the first time) 2. Appraisal costs (costs of checking it is right) 3. Failure costs (costs of getting it wrong).

Failure costs have typically been divided further into internal and external failure costs. The PAF model has been criticised because of an impractical classification of quality costs, as often one specific quality cost can be understood as any of the following: prevention, appraisal, or failure cost. Technically this is not seen to get support from cost accounting systems (Moen 1998, Oakland 2000, Porter & Rayner 1992). The PAF model assumes that there is a certain point of optimal quality level, above which there is a trade-off between investments in prevention and failure costs. This is sometimes criticised for not encouraging continuous improvement. (Sippola 2000, Feigenbaum 1961). In practice, the resources of any given company are limited and the prevention activities should focus on the most beneficial areas of improvement.

26

Testing is required under certain circumstances to mitigate the risks related to quality. Naturally, the ultimate objective is to prevent faults, instead of – as above and often in an expensive way – detecting and fixing the faults at different stages of the product process. However, in order to stay competitive, organisations must make quality related investments, including investing in testing, to meet the expectations and requirements set by customers. (Jahangiri & Abercrombie 2005, Kessler 2000).

As Davis (1994) has stated: “Testing has always been a cost-avoidance activity. We test because it is cheaper to test than not to test”. This statement can be justified by the fact that the costs incurred in testing are lower than the costs of fixing and repairing the faults later in the field. In addition, faulty products would eventually lead to the loss of customers, which is difficult to quantify directly as the value of testing. Another interesting question is as to, when the total quality costs reach the minimum, i.e. where the sum of testing costs and failure costs are minimised (Black 2004).

2.3 Testing in electronics hardware, software and embedded systems

2.3.1 Testing in electronics development

The dimensions of electronic components has been decreasing based on the well-known Moore’s law (Moore 1998, Moore 1965), while the number of product components has increased and made the managing and testing of components even more difficult. Nowadays, products have an increasing quantity of features spoiling customers with variety. Higher number of different components is challenging for power consumption setting requirements for batteries, causing thermal challenges. The wireless applications in particular have high frequency elements creating phenomena that are complicated to model mathematically and difficult to control (Goh et al. 2006). Difficulties to simulate these phenomena create needs for practical experimenting (Li & Kececioglu 2006). In addition to all this, there are also problems with uncertainties over the responsibilities on testing activities relating to product development, in-house production, outsourcing, and suppliers (e.g. Cheng et al. 2007). Testing is applied for a physical hardware object at different levels, including component, board, and system levels. Consequently, HW testing is an issue with multiple dimensions.

27

DFT (design for testability) (Abramovici et al. 1990), BIST (Built in self test) (e.g. Abramovici et al. 1990, Gizopoulos 2006), embedded testing (Gizopoulos 2006), and modularity (e.g. Gershenson et al. 2003) are among different ways that can be used to ease the problems relating to complexity. Modularisation can be seen to shift testing to earlier product development phases, where testing is considered for the entire product, allowing product variants and a degree of flexibility. Simulation on the other hand allows anticipating possible issues early (O'Flynn et al. 2007). In addition, the choice of materials, components, architecture and techniques are the keys for designing a product in electronics. Another challenge is to assure that testing is adequately conducted during the manufacturing process, even with inevitable component variation. When using sourced components for products, tuning is required to obtain an ideal end result. Component variation is more of an issue for manufacturing and for sourcing than it is for product development, and must be taken into account already before manufacturing.

2.3.2 Testing in software development

The global competition sets severe time-to-market pressures for software development, including external demands for greater functionality and rapid delivery. All this has a profound impact on product design, construction, and testing (e.g. Hersleb & Moitra 2001). Typically large companies emphasise documentation, while smaller ones lean on key persons on their procedures (Andersson & Runeson 2002).

There have been different wisdoms and approaches at different times to aid in rectifying problems with the quality of software, formal methods in the 1980s, software process improvement in the 1990s, and yet something else for the current decade (Whittaker & Voas 2002, Tsai 2005). Different approaches can be used for alleviating the risks relating to software development and for minimising the spent resources (e.g. Dey et al. 2007). Should one try to design out the risks and costs would require getting it right the first time. In reality, human errors are unavoidable, and specifications are likely to change during the development. On the other hand, modularity could ease some of the complexity related issues and allow quicker development of product variants (Boehm & Sullivan 2000), possibly shifting some of the problems to integration phase. In addition, paying attention to software architecture is one potential approach for mitigating risks.

28

SW test automation is another contemporary topic for addressing testing issues during development.

System development models (V & W Models, and others) provide frameworks for development activities at different levels, but these models tend to see testing mainly as an error correction activity that is conducted at the end of the design cycle. (see for example, Mooz et al. 2003). However, different NPD philosophies (e.g. Hsieh & Chen 2005) position development activities so that they leave options open for changing situations (Perttula 2007), potentially alleviating current problems. After all, it is difficult to replace software testing, as changes in the processes simply transform testing into a new form, or shifts it to later stages. Instead, deliberately converting testing from a set of retrospective activities into an active flow conducted alongside development could allow meeting business requirements better.

New process models for software development emphasise iterativity, incrementality and agility (Larman & Basili 2003). Rapid application development, rational unified process, feature driven development and scrum can be mentioned as examples of these process models. The importance of proactive checking and testing is highlighted in all of these models, opposed to the traditional waterfall model. Test automation, testing in product families, recognition of repair responsibility (root cause analysis) in long supply chains, and collection of relevant measurement data from the development process are also issues in testing of software-based products over products life, making testing more challenging than before.

Fixing every defect as soon as it is discovered, a simple rule of agility, has several advantages helping to avoid ballooning problems. The defects take far less time to fix in agile approach, and the development time is faster due to cleaner and stable software code (Talby et al. 2006). This way, prioritising defects is avoided, easing the bottlenecks. Also, situations where test priority issues need to be negotiated with customers can be avoided. In agile SW development testers are fully integrated into development teams (e.g. Talby et al. 2006, Juristo et al. 2006).

2.3.3 Testing of systems with HW and SW integrated

The significance and the challenges of verification and validation activities have been noted when considering HW and SW together (e.g. Perttula 2007), but are yet to be discussed more extensively. According to Neuvo (2004) design,

29

verification and testability are among the main challenges of the ICT industry. In practice, adequate testability means more work in software and hardware development. However, there would be better possibilities to find also financial justification for the increased R&D work should all test needs over a product’s life were integrally seen together.

Literature considers embedded systems as ones with the code being stored with the system, typically without hard discs, where adding additional code dynamically is often an exception (e.g. Marwedel 2006). In general, the literature views testing as a function conducted by a separate team after development. Practices typical to software development, used for managing the risks and producing quality software, have not yet become prevalent for embedded systems (Karlesky et al. 2006).

The literature also uses terms co-synthesis and co-design, when discussing designing hardware and software together. This appears in the context of system design (e.g. De Micheli & Ernst 2001, Broy 2006). Co-design addresses the issue of concurrently designing software, and hardware on which the software runs, aimed at gaining some benefits when designing complex systems (e.g. De Micheli & Ernst 2001), but has not been created from the testing perspective.

The ICT industry was originally a HW oriented field, but this has changed as the share of SW has been increasing constantly. For instance, in mobile phone development, the number of SW designers outnumbered the HW designers during the 1990s. Additionally, hardware and software have become more interrelated. However, HW and SW being equally important, it is the software that really determines the functionality and can be seen as more dominating.

Organisational boundaries are one of the main obstacles limiting the development of efficient processes. The development of communication and information management practices is often a challenge, but may provide an effective tool for achieving the goals and objectives of testing. Jablokow & Booth (2006) have noted how communication between separate product development phases provides a distinct business advantage by combining the diverse cognitive attributes necessary for solving complex problems in high performance organisations. On the other hand, Patrashkova & McComb (2004) state that highly qualified personnel may compensate for less communication and that performance may improve when the focus is on exchanging only the vital pieces of information.

Antila (2006) has introduced a Lifetime Testability V model (LTV model), which is an interesting tool for managing and developing testing activities over a

30

product’s life. The LTV model is a procedure for managing the testability requirements of complex systems. The model also helps in identifying re-use possibilities for tests and test solutions over the product’s life. Standardisation can reduce the amount of overlapping work, but can also have an enormous impact on the definition of products life (e.g. Kumar & Fullenkamp 2005).

2.4 Maturity models

The capability of industrial processes can be measured by utilising various types of maturity models. Processes cannot be effectively improved before understanding them. The term maturity can be seen as an indication of how well organisations or employees cope with a given process (Leonard 2004). The maturity approach to assess organisational abilities has its roots in quality management (Tiku et al. 2007). Crosby’s (1996) Quality Management Maturity Grid describes the typical company behaviour, which evolves through five phases in their ascent to excellence in quality management.

Humphrey (1989) presented the first complete formulation of the process maturity framework. This was later used as the basis, when the Software Engineering Institute from Carnegie Mellon University developed the first maturity model, specifically designed for software development (SW-CMM, Software Capability Maturity Model) (Paulk et al. 1993). The original model has been further developed into Capability Maturity Model Integration (CMMI) to cover both software and system development (e.g., CMU/SEI 2006, Lee & Chang 2006). Nevertheless, even now CMMI does not cover all the activities that may be relevant for different practitioners, or explicitly address all function specific issues (see e.g., Beecham et al. 2005, Gottschalk & Solli-Saether 2006, Jokela et al. 2006, McCaffery & Coleman 2007).

As SW-CMM has gradually become a de facto standard for assessing and improving software processes (e.g., Gillies & Howard 2003, Leung et al. 2007). Nevertheless, it was later discovered that SW-CMM concentrates too heavily on technical processes (e.g. Yoo et al. 2006). There has been a need for a model that would include also people and their skills and competencies. On the basis of the original model, Curtis et al. (1995) created the People Capability Maturity Model (P-CMM) for assessing processes and practices for managing competences. The model highlights the readiness of an organisation to execute its strategy. Curtis at al. have later developed the P-CMM model further (Curtis et al. 2003a, Curtis et al. 2003b).

31

Operational functions, such as testing are other issues that the original SW-CMM did not address sufficiently. This was the reason for Burnstein (1996) to develop the Testing Maturity Model (TMM). TMM was specifically developed for guiding software-testing processes (Burnstein 1996). Nevertheless, TMM is seen to overlook the fact that improvement actions at higher levels cannot be carried out without considering organisational aspects (e.g., Jacobs & Trienekens 2002). This was the motivation for Ham et al. (2001) to develop the Metric Based Testing Maturity Model using TMM and CMMI as the basis. Jacobs & Trienekens (2002) developed the model further and created the Metric Based Verification and Validation Maturity Model (MB-V2M2). The V2M2 model is seen to be comprehensive and is not geared towards any specific type of business, such as software engineering. V2M2 has a CMMI-like structure, allowing better compatibility with enterprise-wide maturity assessments. In addition, V2M2 takes the organisational aspects into consideration at higher levels.

Figure 6 describes the development of CMM-based maturity models used in this dissertation. The development path of these models has departed into two directions, some having overall management specific scope and others covering single functions. Previous studies have identified the combination of different maturity frameworks as the best way to develop operational functions (e.g. Vakaslahti 1997, Jacobs et al. 2007). The assessment with these models, such as P-CMM have, however, been found laborious. There seems to be a need for further development of these models.

Fig. 6. The development of CMM-based maturity models used in this study.

32

2.5 Managing technological uncertainty

Decision-making is challenged by technological uncertainty especially together with market and revenue uncertainties (e.g. Sengupta 2005, Chen et al. 2006, Murto 2007). Gressgard & Stensaker (2006) see technological development to influence markets, and consequently technology is not only an internal NPD issue. The literature considers different methods as means for coping with technological uncertainty, including, simultaneous development, project portfolio management, platforms, and verification and validation. (e.g. Lakemond & Berggren 2006, Chen et al. 2005, Bstieler 2005). Project portfolio management is among the most unambiguous ways to describe and manage technological uncertainty. In addition, commonly discussed product development funnel has been developed further to address this ambiguity. (e.g. Feland et al. 2004, Wheelwright & Clark 1992a, Wang et al. 2008, Zhang & Doll 2001, Matthews 1991).

Sequential NPD models do not apply to products based on discontinuous innovations in process and product technology. The discontinuities are caused by both market and technological uncertainties. Technological uncertainty is a vital issue for companies having an influence on product development (e.g. Magnusson et al. 2003, Magnusson & Berggren 2001). Uncertainty can be understood as management’s challenge relating to decision-making, when all the relevant information is not available (e.g. Song & Montoya-Weiss 2001). Lynn & Akgun, (2001) argue that it is practically impossible to predict the final product, its price, and its target market during the early NPD process.

2.5.1 Simultaneous technological alternatives during product development

Quick solutions are attempted to obtain in the traditional design practices, which are later modified until the objectives are met. In this approach the chosen solution is followed from start to finish throughout the entire NPD process. This approach is also called point-based product development, where the initial choices determine success (e.g. Joglekar & Rosenthal 2003, Zhang et al. 2002, Zhang et al. 2008).

Traditionally decision-making in NPD is based on schedules, as the phase review models, and similar methods suggest (e.g. McGrath 1996, McGrath 2004). Companies such as Toyota, on their behalf, aim to solve problems continuously based on available information, instead of fixed date milestones (e.g. Morgan &

33

Liker 2006). This type of just-in-time decision making utilises an information-based approach, opposed to the schedule-based approach, and decisions are made when all necessary information is available. Decisions can thus be made earlier than artificial decision gates, preventing wasting time, or later when there is not enough information available to make an optimal decision. (e.g. Laurindo & de Carvalho 2005, Holman 2003, Moultrie et al. 2007, Perttula 2005).

The literature lists different models addressing gradual selection from a large pool of alternatives. These models include funnel and set-based approaches (e.g. Wheelwright & Clark 1992a, Cooper 2008, Liker et al. 1996, Drejer 2008, Yassine et al. 2008). The set-based approach means keeping multiple options open during early product development. These approaches aid in coping with uncertainty by delaying final decision-making. For example, Ford & Sobek (2005) argue how longer time spent in early NPD enables organisations to have more alternatives and potentially better solutions later during the development process.

For managing NPD flexibly, set-based product conceptualisation has been highlighted as an important issue, enabling multiple options, and timely feedback (e.g. Hines et al. 2006, Ford & Sobek 2005, Zhang et al. 2008). This type of approach allows anticipating changes, and facilitating operational manoeuvrability swiftly. These needs for adjustments are typically caused by altered customer requirements, technical, or engineering problems. The set-based approach is seen to allow offering a variety of products with reasonable costs. According to Morgan & Liker (2006) the use of this virtually inefficient approach has made, for example, Toyota’s development fastest in its sector.

2.5.2 Project portfolio management and technological uncertainty

Addressing uncertainty can also be considered from the viewpoint of project portfolio management, instead of a single project’s perspective. Matthews (1991) has categorised research and development (R&D) projects into three different groups based on allocated resources and uncertainty. Long-term research for developing future technologies forms the first group. The level of uncertainty is high, and the projects are seen as obligatory overhead costs needed to secure future success of a company. The purpose of the second group is to evaluate technologies and reduce associated uncertainty, and finally to select the most potential ones for further development. The projects of the third group are seen as investments, while the markets and the development and manufacturing costs are known well enough to calculate return on investment. The degree of uncertainty

34

is low at this stage. Portfolio management encourages companies to initiate second-group projects, with a sole purpose to decrease uncertainty before committing into large scale development projects. (Suikki 2007, Suomala & Jokioinen 2003, Suomala 2004, Matthews 1991)

2.5.3 Platform development

When tackling uncertainty during product development, platform development is a significant R&D tool, potentially helping to cope with the situation (Gershenson et al. 2003, Zhang & Doll 2001, Meyer & Utterback 1995, Robertson & Ulrich 1998, Salonen et al. 2008). Platforms are often considered broadly as planning constructs instead of addressing the needs of a single product (e.g. Yang & Jiang 2006). Platforms can be seen as the basic structure with a common set of components, modules, and parts from which derivate products can be generated, while maintaining the core technology (e.g. Yakob & Tell 2007, Meyer & Lehnerd 1997, Lee 2008, Koufteros et al. 2005). The platform approach reduces the time used for new product planning, while useful elements already exist (e.g. Kim et al. 2005, Robertson & Ulrich 1998). Technical and marketing uncertainties are lower, and product changes can be made more rapidly (e.g. Koufteros et al. 2005). According to Ratamaki (2004), tailoring for customer needs can be conducted fairly late during the development process when utilising platforms. Nevertheless, platforms can fail in many ways as they can be complicated to design (e.g. Wheelwright & Clark 1992b, Robertson & Ulrich 1998).

2.5.4 Verification & validation and NPD uncertainty

Verification and validation is an effective way for collecting information, and thus potentially reducing uncertainty. Nevertheless, conventional pass/fail type of V&V for checking whether requirements are met or not does not increase thorough understanding. Additionally, V&V is typically considered too late in the development (e.g. Harkonen et al. 2009, Hsieh & Chen 2005, Host & Johansson 2000, Mahanti & Antony 2006, Perttula 2005). Errors and misunderstood requirements are more expensive to fix the later they are addressed (e.g. Boehm 1981, Butcher et al. 2002, Firesmith 2007, Chun 2006).

Perttula (2005) has introduced an information-based approach, which is especially beneficial for a situation with changing requirements. In this approach

35

more information is collected from the product through verification and validation, than is necessary for pass/fail decisions. When requirements later change, it is possible to come back to the information collected earlier, to check whether the product meets the new requirements, without having to repeat the physical tests. Perttula’s approach strongly relates to the methods utilised by Toyota, where all the relevant information of a design is attempted to collect during early product development. Toyota’s approach includes methods, such as simulation, modelling, measuring, analysis, and ijiwaru testing (e.g. Morgan & Liker 2006, Fan & Yu 2004). It is then possible to refer to this information when requirements change.

Both software and hardware are typically included in modern gadgets. However, performance of a product, or its component, is the determining factor, not whether this is achieved through hardware of software. Perttula (2007) highlights the significance of this type V&V in module/ platform based product development enabling later reuse without carrying out re-tests.

Requirements typically stabilise during the NPD process and the role of V&V changes. Pass/fail verification is seen more effective during later stages, when uncertainty diminishes and efficient execution is crucial. (e.g. Beecham et al. 2005, Ebert 2007).

2.5.5 Synthesis on managing technological uncertainty

The different viewpoints on managing technological uncertainty in high tech NPD have been summarised in Table 4. This synthesis highlights the key findings from the theory for the purpose of this dissertation.

36

Table 4. Synthesis on managing technological uncertainty.

Potential methods Key findings

Developing simultaneous alternatives Developing, and studying, simultaneous alternatives reduce

risks, and aids in gaining understanding.

Enables choosing several directions, and reacting to changes.

Enables delayed decision making

Requires resources and time.

Project portfolio management Promotes the establishment of projects for strategic options

that have a goal of better clarifying few technological

alternatives for increased understanding.

Categorises R&D projects in two dimensions of uncertainty

and resources.

Requires resources and time

Platform development Solutions that have once been sufficiently considered can be

later utilised for a number of purposes.

Enables rapid moves, using existing building blocks.

Final decisions can be delayed.

Not always the most customer oriented approach.

Platforms can be complicated to design.

Requires resources and time when platforms are developed.

Information based approach on V&V A set of methods, assisting to enhance understanding,

especially during early development.

Accelerates, and clarifies activities towards the end of NPD

process.

Significance for platform based product development enabling

effective reuse.

Requires resources and time during the early NPD.

37

3 Research contribution

This chapter presents the individual research contributions of the research papers.

3.1 Organisational maturity and functional performance

This article answers the research question 1. In this article, maturity models are utilised for assessing the management systems of companies together with analysing V&V as an operational function. The main focus is on the interdependence and the balance of managerial and operational aspects. This article utilises two different maturity models. The People Capability Maturity Model (P-CMM) (Curtis et al 2001) has been used to analyse the level of overall management systems in companies. The Verification and Validation Maturity Model (V2M2) (Jacobs & Trienekens 2002) has been utilised to analyse V&V. The interdependence of these two aspects is also scrutinised. The article clarifies whether the bottleneck for performance improvement lies with the operational function itself or with the maturity of the overall management system.

Figure 7 summarises the key finding of the maturity analyses on both the management system and the operational function. The difference between the maturity of the management system and that of the studied operational function was found to be the greatest in the top company, a major Scandinavian telecom company.

The article presents results that indicate a clear connection between the maturity of the general management system and the studied operational function. Noteworthy is that there was significant differences between different companies. Verification and validation activities in the identified top company were at a surprisingly low level despite of its excellent results for the overall management system. A company with such a management system should be able to develop its V&V activities to a much higher level of maturity.

The article shows that a company being superior with their management system maturity does not automatically mean that they are superior with all operational functions. Nevertheless, the management of such a company should have the capability of reacting into the rising significance of new matters affecting their business success. V&V activities seem to be one of the increasingly important factors. It is not realistic for companies with lower maturity to gain superiority in new and difficult matters without first improving their management system maturity.

38

Fig. 7. Summary of the P-CMM and V2M2 maturity analyses.

This article points out that investing in a single function is only productive, if the company is capable of utilising the results of such an investment. There should be a balance between the investments in a single activity and the overall organisational maturity. When analysing the maturity of a single function, according to any CMM-based maturity model, it must be noted that it can be developed only to level 3 without investing in the improvement of the entire management system. In practice this means that an organisation needs to develop the maturity of its management system, should it desire to raise the maturity of any single function to higher levels of 4 and 5 in the scale of the model.

3.2 Improving the efficiency of verification and validation

This article answers the research question 2 and discusses testing from the management perspective, giving a description on testing in the ICT sector. Testing is presented in relation to pure electronics hardware (HW), pure software (SW) and embedded SW & HW. In addition, the article presents methods that can be used for reducing test costs in different phases of the product development process.

Figure 8 presents the development of testing activities. The magnitude of testing is especially large in software development. This is where SW developers first discovered that old, hardware related, testing practices did not fit in with

39

expanding testing needs and new practices had to be developed. The same challenges discovered in the SW industry have later also emerged in HW. The application of the ICT technologies is continuously spreading to other industries, for example to the automotive industry. Growing complexity of products and increasing need for integration result in challenges for testing. Sectors that have earlier been considered as solid quality sectors are currently facing new problems with reliability. The management of these more traditional sectors has found it difficult to leave the old sector-specific mindset and learn from the SW and ICT sectors. The most developed testing practices can probably be found in SW and other sectors may find it beneficial to benchmark against these software practices.

Fig. 8. Development of testing activities.

Table 5 lists the challenges preventing testing becoming more effective, together with potential solutions attempted by the ICT industry. Testing activities are typically not been managed sufficiently and has been seen as an independent set of actions. Sub-optimisation of testing activities can cause inefficiency and repetition of tests. Regardless of it being worthwhile to improve testing in each product development phase, the best results can be obtained by analysing the entire chain of testing. The article shows how better results can be obtained by closer integration of HW and SW. A general learning is that the emphasis of

40

testing activities should be shifted towards earlier part of NPD. Each product development phase should consider not only testing within the own phase but also the following phases as well in order to remove unnecessary overlapping. One common problem for many companies identified in this article is a general lack of personnel with good and versatile competences in testing. In addition, systematic management practices are in a key position in improving the efficiency of testing.

Table 5. V&V challenges and possible solutions.

Phase Challenges/Problems Solutions

Applied

research

Uncertainty on what to research

Lack of competent personnel

Lack of systematic practices

Simulation

Product

development

High personnel costs


Large amount of tests

Overlapping tests

Unreliability of new test systems

Balancing the number of designers and test personnel

Personnel training

Test automation especially on software testing

Production High investment costs & short

life-time of equipment


Functional testing is expensive

& bottleneck

Large amount of tests

Inaccuracy of test equipment

Avoid overlapping tests with R&D

Developing tests methods of functional testing (e.g.

sampling)

Removal of unnecessary tests

Emphasis shift from functional to process testing

After Sales Large number of test equipment

Non-standardised products

Entire

development

chain

Overlapping tests between

different phases

Lack of understanding the

entire chain

Understanding the entire value chain

Shifting the emphasis to earlier stages

Each phase should consider the following phase

Utilisation of test management systems to control test

coverage and eliminate overlapping

Standardisation of product interfaces

Maximised utilisation of test equipment

Better synchronisation of HW & SW projects

Better utilisation of the experiences of earlier projects

Increasing Embedded testing & BIST

41

This article points out how testing must be optimised in order to reduce costs and to shorten the development cycles. This optimisation must start with considering the entire development chain, and continuing through every phase of product development to reduce overlapping. Internal cooperation is required between different product development phases and functions. Testing is a challenging support function requiring versatile competences and knowledge on issues, such as test methods, product structure & purpose, specifications, and technological limitations. Software testing can be seen as a forerunner in the development of testing activities and management practices in the ICT industry.

3.3 Improving product development in different type of ICT companies

This article answers the research question 3. The article utilises Toyota’s NPD model (TNPD) as a benchmarking tool for obtaining potential improvement ideas for ICT companies. The research covers three different company types: large, small, and suppliers. The main benefits of TNPD are its impact on issues, such as time-to-market, productivity, quality, the number of changes at the end of development, product cost, R&D capacity, rapid learning, among others (see e.g. Radeka & Sutton 2007). Another motivation for using Toyota as a source for ideas is their procedures being well documented in the literature (e.g. Morgan & Liker 2006, Ward 2007, Ford & Sobek 2005). Selecting automotive industry as a benchmark was logical as ICT has grown in volume and its nature has become more similar to more traditional industries. In addition, the car sector is mature has streamlined it business practices, including NPD, to a very high level.

Considerations potential for ICT companies include development of several parallel alternatives, the role similar to Toyota’s chief engineer for managing product development, company driven competence development and rigorous standardisation including people skill-set standardisation. Smaller companies, having limited resources, should only consider parallel options and delayed decision-making with technologies critical for their NPD. Toyota’s NPD, also gives good examples for supplier integration from both, supplier’s and client’s perspectives. Simple visual communication is an example of TNPD that all the company types can learn from.

All-inclusive and direct Toyota benchmark better serves the needs of large companies. For small companies, it may be more beneficial to benchmark Toyota directly only partially and to include indirect analyses through top companies in

42

one’s own business sector. Suppliers may also obtain ideas for improving their product development by including their main clients as an indirect way for benchmarking. Table 6 present potential learnings for the three studied company types discussed in the article.

Table 6. Summary of potential learnings for large & small companies and suppliers.

Sub-category Large Small Suppliers

Processes Delay decision making

Separate value-added from waste

Indentifying and communication needs

of internal customers better

Standardise more people skill-set

Deploy delayed

decision-making

selectively

Separate value-added

from waste

Aim to integrate into

clients’ processes

People CE type solutions to integrate NPD

processes

Consider more company driven career

path planning

Improve communication between

suppliers

Systematic continuous improvement

Pay more attention on

developing functional

expertise

mprove

communication to/from

customers

Tools &

technologies

Idolise technology less and strive for

customer orientation and efficiency

Process data for individual needs

Use simple and visual communication

Pull-based technology

acquisitions

Adopt pull-based

technology

acquisitions

Use simple and visual

communication

3.4 Technological uncertainty and verification & validation activities

This article answers the research question 4. The article considers how different organisations cope with uncertainties in their product development, and specifically how ICT companies utilise verification and validation activities for tackling these uncertainties. The article analyses differences among three company categories: independent actors with products of their own, original equipment manufacturers (OEM), and subcontractors providing services for the two first mentioned categories.

The article shows that ICT companies tend to make the technological decisions as early as possible leading to great technological uncertainty. In the

43

case of the initial decisions proving wrong, significant costs are created in form of wasted resources and delayed development. Maximisation of understanding in the early part of product development allows companies reacting to surprises when requirements change, without entering a panic mode, or causing excessive delays. The tendency of companies to make decision early is due to the mindset dominated by the schedule-based view on product development, whereas company management should consider more information-based approaches. Decisions should be made when required information is available, not based on pre-set schedules. The challenge is accumulated, the studied companies typically being more technology, than business oriented in their NPD.

The study indicated that different methods are utilised to cope with uncertainty in high tech NPD process. Simultaneous development, project portfolio management, platform development, understanding enhancing V&V, agile methods, prototyping, and standardisation are among the methods companies use to address uncertainty. All these methods, discussed in this article, have two common nominators, enhancement of information during early NPD, and intentionally building options for the management to have choices for later product development. Both of these allow companies to make their final decisions later when they have more information and deeper understanding on used technologies.

Verification and validation activities have traditionally been seen as a pass/fail decision, indicating whether specifications have been met. Nevertheless, an information-based approach can be especially beneficial for situations with changing requirements. In this type of approach more information is collected from the product through V&V, than is necessary for pure pass/fail decisions. When requirements later change, it is possible to come back to the information collected earlier, to check whether the product meets the new requirements, without repeating the physical tests. This ‘understanding enhancing V&V’ (UE V&V), strongly relates to the methods utilised by for example Toyota, where all the relevant information of a design is attempted to collect during early NPD. UE V&V include methods:

– Simulation – Modelling – Measuring – Analysis – Ijiwaru testing.

44

Once requirements change it is then possible to refer to back to this collected information. This is one of the reasons why UE V&V approach is beneficial for situations with high uncertainty typical for product development in ICT.

V&V activities are seen inherent to all the interviewed companies, but a deeper utilisation of V&V as a means for tackling uncertainty has not been internalised. In the early part of product development, V&V are typically used for finding the performance limits of new technologies, for example, through simulations and modelling. Uncertainty is greatest in the beginning of NPD and UE V&V is more beneficial. Towards the end of NPD, relevant understanding exists and as a consequence uncertainty decreases making pass/fail V&V more effective.

The article shows that the three studied company categories proved to be different in the manner they handle V&V activities. Independent actors tend to utilise UE V&V the most, and within the category, the most advanced companies utilise UE V&V more than smaller and less developed ones. Original equipment manufacturers utilise UE V&V, but significantly less than large advanced companies with products of their own. In the subcontractors’ category, verification and validation almost exclusively means pass/fail testing as the requirements are set by their customers. UE V&V is utilised by subcontractors only when they develop platforms of their own. Figure 9 summarises the relative divisions of the two types of V&V efforts among the company categories studied in the article.

45

Fig. 9. Relative divisions of UE V&V and pass/fail V&V in the studied categories.

The study shows that even the most advanced companies should consider possibilities to increase the amount of UE V&V, and to shift the emphasis to earlier NPD. Especially smaller companies and OEMs could learn from the experience of more successful ones. Nevertheless, it is not necessarily beneficial for subcontractors to significantly increase the amount of UE V&V. However, it might be beneficial to analyse the issue from their customers’ standpoint, as understanding customer behaviour is always worthwhile. In the case of product development cooperation, a client can attempt to build UE V&V into the contract, should they find it desirable for their business.

47

4 Discussion

4.1 Theoretical implications

In this dissertation the research problem is approached from four complementary perspectives – organisational and V&V maturities, current V&V management challenges, benchmarking another sector, and uncertainty during NPD – based on which four research questions (RQ) were formed. Table 7 summarises the theoretical implications of the four articles and how they reflect the chosen perspectives.

Table 7. Summary of theoretical implications.

Article Title Implications

I

Organisational maturity and functional

performance

Development of maturity models for V&V

purposes

Comparison of functional and management

system maturity

II Improving the efficiency of verification and

validation

Documenting the development of V&V activities

Documenting challenges and potential solutions

Indentified state-of-art on V&V management

III Improving product development in different

type of ICT companies

Description of NPD in different type of companies

Differences in NPD practices in ICT and

automotive sectors

IV Technological uncertainty and verification &

validation activities

Synthesis on managing technological uncertainty

Understanding enhancing V&V

Organisational and V&V maturities: This paper clarifies the interdependence of the overall management system maturity and the maturity of verification and validation activities. The article also describes the development of maturity models. This study provides new input to the scientific community by utilising two different maturity models, one for assessing the management system and the other for V&V, and compares their results against each other. V&V activities cannot be separately developed to a high level of maturity, without also considering the management system maturity.

Current V&V challenges: This article clarifies V&V challenges in ICT companies and presents the development of testing methods as a function of time.

48

The article also describes the current status of testing activities separately in hardware, software and embedded environments. This article complements the studies by previous authors by updating and expanding the understanding of V&V problems. Opposed to previous studies, this paper does not focus on specific areas of V&V, such as HW, SW or production testing, but instead attempts to find an inclusive management perspective.

Benchmarking another sector: This article demonstrates how development ideas can be obtained for the purpose of different type ICT companies by benchmarking company practices against a mature business sector. Benchmarking the automotive industry for the purpose of V&V activities in the ICT sector is new to the scientific community. This article clarifies whom should different level actors benchmark for a maximum benefit.

Uncertainty: This article clarifies how ICT companies cope with uncertainty, and challenges, caused by the changing environment, focusing particularly on the role of verification and validation (V&V) activities. Uncertainty is widely studied in the literature, however, this article provides new scientific information by combining V&V activities and inevitable NPD uncertainty in the ICT sector. The article clarifies the benefits of understanding enhancing V&V in tackling uncertainty.

This research as a whole provides examples for different type of companies on how to cope with the current V&V challenges encountered in modern ICT product development. Optimisation on V&V activities has a direct impact to time-to-market, cost reduction and product quality. One has to keep in mind that the examples presented are only possibilities and each company should make the final decisions based on their own business perspective.

4.2 Managerial implications

The purpose of this dissertation is to find ideas for ICT companies to improve their verification and validation activities from the product development management viewpoint. This purpose is addressed through the four individual research articles.

Article 1 (“Organisational maturity and functional performance”) points out how an organisation should assess both, the maturity of its entire management system as well as that of its individual activities, such as V&V. It is not profitable for all companies to invest in gaining superiority in every single activity, rather

49

the investment should be in line with the company strategy and its overall maturity.

Investing in a single function is only productive, if the company is capable of utilising the results of such an investment. There should be a balance between the investments in a single activity and the overall organisational maturity. When analysing the maturity of a single function, according to any CMM-based maturity model, it must be noted that it can be developed only to level 3 without investing in the improvement of the entire management system. In practice this means that an organisation needs to develop the maturity of its management system, if it desires to raise the maturity of any single function to higher levels of 4 and 5.

The capability, or maturity, of the management system of a company does not automatically mean that all important functions are of high maturity. This study revealed that not even the identified top company has reached a V&V specific maturity level higher than 3, even though the management system maturity would allow this. It would be possible for the top company to raise the V&V maturity, should they decide to invest in the matter. However, already now the top company does utilise the information from the after-sales phase to improve the performance of the earlier phases. Also, the V&V maturity of the product development is clearly at a higher level in the top company than is the case for the industry average.

Article 2 (“Improving the efficiency of verification and validation”) points out how testing has become a very important factor in creating ICT products, and is responsible for a significant part of the overall costs. Testing should be understood as an internal service function supporting the development and production of high quality products. Testing is not an end in itself, but rather a part of a bigger goal. In order to make this internal service function as efficient as possible, it must be integrated into the company strategy and into the entire product development process.

To reduce costs, and to shorten development cycles, testing must be optimised. The optimisation must start with consideration of the entire development chain, and continue through every phase of the NPD to minimise overlapping. Internal cooperation is needed between different phases and functions. Testing is a challenging service function requiring versatile knowledge and competences e.g. on test methods, product structure and purpose, specifications, and technological limitations. SW testing is a forerunner in the development of testing activities and management practices.

50

One way of rationalising testing is to outsource it. Testing services are, however, still underdeveloped, but specialised testing services are likely to increase in the future, and outsourcing testing should be an option to device manufactures.

When formulating their research strategies companies should connect their researchers to the standardisation work as technologies and products are later tested against these standards. During the product development phase the interfaces between various modules should be standardised to make it easier to use the same testing methods.

Article 3 (“Improving product development in different type of ICT companies”) highlights how ICT companies can benefit from Toyota’s learnings. According to the study, the strategy of using parallel technological alternatives and delayed decision-making is not widely applied. Companies need to better address the needs of internal customers, and optimise their NPD processes over the entire product life-cycle. Large companies could potentially learn from Toyota’s CE type solutions in their cross-functional integration. Especially large companies may wish to consider including people skill-sets in their standardisation efforts by company driven competence development. Data overflow is currently a significant challenge and companies need to address this issue by further processing the data into a meaningful format, and selectively tailor it for different purposes.

The managers of different type of companies may need to consider which methods utilised by Toyota are realistic for their realities and selectively consider them. Smaller companies having limited resources could selectively consider delayed decision-making in their NPD projects, better separation of added value from waste, and pay more attention on developing functional expertise. Supplier type companies should obtain ideas directly from Toyota, and indirectly from both top companies in their own field, and from their advanced clients.

Article 4 (“Technological uncertainty and verification & validation activities”) indicates that ICT companies tend to make the technological decisions as early as possible leading to great technological uncertainty. Should the initial decisions prove wrong, significant costs are created in form of wasted resources and delayed development. Maximisation of understanding in the early part of NPD allows companies reacting to surprises when requirements change, without entering a panic mode, or causing excessive delays. The inclination of companies to make decision early is due to the mindset dominated by the schedule-based view on NPD, whereas the management should consider more information-based

51

approaches. Decisions should be made when required information is available, not based on pre-set schedules. The challenge is accumulated, as the studied companies are typically more technology, than business oriented.

A central finding for company managers is the potential of understanding enhancing V&V (UE V&V) for addressing uncertainty in NPD. UE V&V methods suit the purposes of hardware and embedded products, not necessarily pure software code. Currently, the utilisation of V&V for enhancing understanding is not sufficient. Building adequate understanding is, however, vital for continuous improvement. Even the most advanced companies should consider possibilities to increase the amount of UE V&V, and to shift the emphasis to earlier NPD. Especially smaller companies and OEMs could learn from the experience of more successful ones. However, it is not necessarily beneficial for subcontractors to significantly increase the amount of UE V&V. Nevertheless, it might be worthwhile to analyse the issue from their customers’ standpoint, as understanding customer behaviour is always essential. In the case of product development cooperation, a client can attempt to build UE V&V into the contract, should they find it desirable.

The summary of managerial implications of each article is presented in Table 8.

Table 8. Summary of managerial implications.

Article Title Contribution

I Organisational maturity and functional

performance

Understanding the balance between functional and

management system maturity

Identifying areas for development

II Improving the efficiency of verification and

validation

Learning from the practices of other companies

Enabling managers to perceive the bigger picture

III Improving product development in different

type of ICT companies

Benefits of benchmarking for V&V

What type of companies should different actors

benchmark

IV Technological uncertainty and verification &

validation activities

Understanding methods for tackling uncertainty

Understanding the relevance of UEV&V for different

company types

52

The main managerial implications of this doctoral dissertation can be concluded as a need to fully understand the importance of V&V activities, realisation of the current low maturity and appreciate the need to overcome the existing tendency to organise through functional silos. Verification and validation activities should be viewed and managed over the entire NPD process. This requires new means for cross-functional integration. The maturity of the overall management system needs to be adequate to enable higher efficiency and effectiveness of V&V activities. There are pressures to shift the emphasis of V&V to early NPD and simultaneously delay decision-making in NPD projects to a stage where enough information is available. Understanding enhancing V&V methods are a potential way to advance towards these goals.

4.3 Reliability and validity

This research proceeded gradually searching for ever new potential solutions to the research problem. This research is qualitative and descriptive in nature, applying mainly inductive reasoning.

Yin (2003) proposes four tests to establish the quality of any empirical social research: construct validity, internal validity, external validity, and reliability.

To meet the test of construct validity, a researcher must be sure to cover two steps: (1) select the specific types of issues that are to be studied and (2) demonstrate that the selected measures on these issues actually reflect the specific types of issues that have been selected. (Yin 2003). Industry has had a significant role in formulating the research problem and topics to be covered. In addition, the main source for empirical data has been the experienced industrial managers that have been interviewed. Consequently, one could assume that the research topics have been relevant for the industry, and thus increasing the validity. The research problem was viewed from four complementary perspectives, through four Journal articles. Additionally, each perspective was reflected against existing literature. However, should the industrial managers consulted when defining the research areas to be studied be different, or should the studied industry be different, the result could vary to some degree. Also, should the number of selected perspectives or themes be different, this could also influence the obtained results to a degree.

Internal validity is a concern when a researcher attempts to determine whether event A leads to event B. Should the researcher incorrectly conclude that there is a causal relationship between A and B without knowing that a third factor,

53

C, may actually have caused B, the research design has failed to ensure internal validity (Yin 2003). The articles included in this dissertation have undergone a double-blind review process, and have thus been subject to critical judgement by the scientific community. According to Airala & Pekkanen (2002), publicity and critical judgement by the scientific community are integral parts of scientific research. In addition the researcher has extensive industrial experience from the business sector in question. However, should the scientific articles be sent to a completely different Journal, the comments made by the reviewers could vary to some degree, resulting in slightly different end result. Should the prior knowledge of the researcher be different, the analyses could have provided different results.

External validity deals with the problem of knowing whether the research findings can be generalised beyond the immediate context of the study. (Yin 2003). The data analysed in this dissertation has included the views of experienced managers from tens of different companies. However, the direct validity of the results is limited to the studied business sector. Nevertheless, this research has studied different company types within the sector and discussed the applicability of the learnings.

The objective of reliability is to establish the quality of the research. The purpose is to ensure that other researchers can repeat research and obtain similar results by following the described procedures. (Yin 2003). In qualitative research the most critical aspect for reliability is the researcher (Eskola & Suoranta 1998). Relying excessively on the researcher’s observations may lead to invalid research findings. It is vital for the researcher to recognise his or her biases and values. Qualitative research focuses on a few cases and aims to analyse them thoroughly. The criteria for reliability rely on the quality, not on quantity. In addition, the research has been conducted at a certain time, by certain individuals, making the research somewhat unique. Collecting data in a qualitative manner has its limitations in the form of different interviewees potentially reacting differently to certain issues, potentially influencing the obtained results. The research methodology and process used in this dissertation are documented and described for each individual Journal article, making it possible to repeat the research and compare the findings. However, no researcher is perfect and incorrect conclusions are possible.

54

4.4 Recommendations for further research

Managing new product development and V&V activities can be considered contain numerous issues. However, for practical reasons some relevant issues have been excluded from this dissertation, potentially leaving room for further research.

The previous research has been scarce on studying V&V from the perspective of NPD process phases. This could include splitting the NPD process into distinct smaller increments and analyse V&V activities in detail for each of these phases. This type of approach would enable managers to allocate resources to optimise with a more comprehensive perspective.

Another aspect identified important during the interviews conducted during the dissertation process is the fact that requirements do change during the NPD process and thus requirements management has potentially a great influence on V&V activities. System development models do not adequately address changing requirements. Potential further research could include studying the needs of internal and external customers in relation to requirements management. Some of the interviewed companies utilise design for excellence (DFX) and concurrent engineering in an attempt to improve requirements management. A deeper analysis of V&V management and DFX could be a potential topic for further research.

Modern ICT products are complicated, offering a vast variety of features for customers. As a consequence, it is important that product data management is analysed in conjunction with product structure. Product life-cycle management must acknowledge product structures and related information from product idea through product development, manufacturing and service to disposal. Implications of these issues to V&V would be an interesting topic for further study.

55

5 Summary

The ICT industry has expanded rapidly during the past decades and has changed and witnessed a transformation in terms of frequent technology changes, increase in complexity, time-to-market pressures and the decrease in physical dimensions. Technical superiority of products and new functionalities are not enough for companies to succeed, but one must also invest in developing processes.

Verification and validation (V&V) is a critical issue for modern high technology product development. It has been claimed that it is impossible to design and manufacture products without defects. Even though verification and validation is seen as a significant matter for companies, studies that deal with the management aspects of V&V are scarce. This was the motivation for this dissertation. The research problem addressed in this dissertation was stated as follows:

Companies in the ICT sector require research efforts and new knowledge for improving their verification and validation activities from the product development management viewpoint.

To be able to give a solution to the above problem, this research was approached from complementary perspectives with four research questions, each of which is discussed in an individual research article. The four Journal articles formed a logical chain, which was refined during the research process. Table 9 summarises the research questions and their contributions.

56

Table 9. Research questions and contributions.

Article Research question Implications

I How does the organisational maturity

reflect functional performance?

Development of maturity models for V&V

purposes

Comparison of functional and management

system maturity

Understanding the balance between functional

and management system maturity

Identifying areas for development

II What is the development path of V&V

activities and what are the challenges

and possible solutions?

Documenting the development of V&V activities

Documenting challenges and potential solutions

State-of-art on V&V management

Learning from the practices of other companies

Enabling managers to perceive the bigger picture

III How can ICT companies benefit from

benchmarking the NPD management

practices against a mature business sector?

Description of NPD in different type of companies

Differences in NPD practices in ICT and

automotive sectors

Benefits of benchmarking for V&V

What type of companies should different actors

benchmark

IV How are different types of companies coping

with uncertainty in high tech NPD, and how

can V&V aid this?

Synthesis on managing technological uncertainty

Understanding enhancing V&V

Understanding methods for tackling uncertainty

Understanding the relevance of UEV&V for

different company types

The research questions are interrelated, even though their focus is different. Each article covers a large area and would be worth further study. However, this scope was chosen based on initiatives by the industry. The research questions – from one to four – cover both the managerial and practical levels.

The main implications of this doctoral dissertation can be concluded as a need to overcome the current tendency to organise through functional silos, and low maturity of V&V activities. Verification and validation activities should be viewed and managed over the entire NPD process. This requires new means for cross-functional integration. The maturity of the overall management system needs to be adequate to enable higher efficiency and effectiveness of V&V activities. There are pressures to shift the emphasis of V&V to early NPD and

57

simultaneously delay decision-making in NPD projects to a stage where enough information is available. Understanding enhancing V&V methods are a potential way to advance towards these goals.

The contributions of this research benefit ICT companies in the form of describing and documenting the challenges and by providing potential solutions. This dissertation forms a base for understanding V&V activities from managerial viewpoint. It is, however, vital to understand that different type of companies require solutions that suit their individual business environment, internal resources and competences.

59

References Abramovici M, Breuer MA & Friedman AD (1990) Digital systems testing and testable

design, Computer Science Press, NY: 653. Airala M & Pekkanen M. (2002) Tekniikan alan vaitoskirjaopas, Teknillinen korkeakoulu,

Espoo, Finland: 72. Anderson P & Tushman ML (1991) Managing through cycles of technological change,

Research Technology Management 34: 26–31. Andersson C & Runeson P (2002) Verification and validation in industry – a qualitative

survey on the state of practice. Empirical Software Engineering, Proceedings, IEEE, International Symposium.

Antila J (2006). Lifetime Testability V-model, EBTW'06 European Board Test Workshop, Southampton, 2006.

Anttila P (2005) Ilmaisu, teos, tekeminen ja tutkiva toiminta, (in Finnish) Akatiimi Oy, Hamina, Finland.

Arvidsson M, Gremyr I & Johansson P (2003) Use and knowledge of robust design methodology: a survey of Swedish industry, Journal of Engineering Design, 14(2): 129–143

Beecham S, Hall T & Rainer A (2005) Defining a requirements process improvement model, Software Quality Journal. 13(3): 247–279.

Black R (2004) Critical Testing Process – Plan, Prepare, Perform, Perfect, Boston: Addison-Wesley.

Blanchard BS (2008) System Engineering Management, 4th Edition, Wiley: 560. Boehm BW (1981) Software engineering economics. Prentice Hall: 767. Boehm BW & Sullivan KJ. (2000) Software economics: a roadmap. International

Conference on Software Engineering. Proceedings of the Conference on the Future of Software Engineering, Limerick, Ireland: 319–343.

Broy M (2006) Challenges in automotive software engineering. Proceeding of the 28th international conference on Software engineering. ACM Press, NY: 33–42.

Bstieler I (2005) The Moderating Effect of Environmental Uncertainty on New Product Development and Time Efficiency. Journal of Product Innovation Management, 22(3): 267–284.

Burnstein I (1996) Developing a Testing Maturity Model. Illinois Institute of Technology. Butcher M, Munro H & Kratschmer T (2002) Improving software testing via ODC: Three

case studies. IBM Systems Journal, 41(1): 31–44. Chen J, Reilly RR & Lynn GS (2005) The impacts of speed-to-market on new product

success: the moderating effects of uncertainty. IEEE Transactions on Engineering Management, 52(2): 199- 212.

Chen HH, Lee AHI & Tong Y (2006) Analysis of new product mix selection at TFT-LCD technological conglomerate network under uncertainty. Technovation, 26(11): 1210–1221.

Cheng S-R, Hsu B-M & Shu M-H (2007) Fuzzy testing and selecting better processes performance. Industrial Management and Data Systems 107(6) 862–881.

60

Chun YH (2006) Estimating the number of undetected software errors via the correlated capture–recapture model. European Journal of Operational Research 175(2): 1180–1192.

CMU/SEI (2006) CMMI® for Development. Version 1.2. Carnegie Mellon University, Software Engineering Institute.

Cooper RG (2001) Winning at New Products: Accelerating the Process from Idea to Launch. 3rd edition, Perseus Publishing: 416.

Cooper RG (2008) Perspective: The Stage-Gate® Idea-to-Launch Process—Update, What is New, and NexGen Systems. Journal of Product Innovation Management 25 (3): 213–232.

Cooper RG, Edgett SJ & Kleinschmidt EJ (2004) Benchmarking best NPD practices – III. Research Technology Management 47 (6): 43–55.

Costanzo LA (2004) Strategic foresight in a high-speed environment. Futures 36: 219–235. Crosby PB (1980) Quality Is Free. Mentor, U.S: 270. Crosby PB (1996) Quality Is Still Free: Making Quality Certain in Uncertain Times.

McGraw-Hill, New York, NY. Cross N (2008) Engineering Design Methods: Strategies for Product Design. 4th Edition,

Wiley: 230. Curtis B, Hefley WE & Miller SA (1995a) Overview of the People Capability Maturity

Model., Software Engineering Institute, Carnegie Mellon University, Pennsylvania. Curtis B, Hefley WE & Miller SA (1995b) People Capability Maturity Model. CMU/SEI –

95 MM-02, Software Engineering Institute, Carnegie Mellon University, Pennsylvania. Curtis B, Hefley WE & Miller SA (2001) People Capability Maturity Model® (P.CMM®),

version 2.0 CMU/SEI-2001-MM-01, Software Engineering Institute, Carnegie Mellon University, Pennsylvania.

Curtis B, Hefley WE & Miller SA (2002) The People Capability Maturity model. Guidelines for Improving the Workforce. Carnegie Mellon. The SEI Series in Software Engineering. Addison- Wesley. USA.

Curtis B, Hefley WE & Miller SA (2003a) Experiences Applying the People Capability Maturity Model. Software Engineering Institute, Carnegie Mellon University, Pennsylvania.

Curtis B, Hefley WE & Miller SA (2003b) Introduction to the People CMM, Version 2. Carnegie Mellon, Software Engineering Institute.

Dale BG, van der Wiele T & van Iwaarden J (2007) Managing Quality, 5th Edition, Wiley: 640.

D’Aveni R (1995) Hypercompetitive Rivalries, Competing in Highly Dynamic Environments. The Free Press, New York.

Davis B (1994) Economic modeling of board test strategies. Journal of Electronic Testing 5 (2–3): 157–169.

De Micheli G & Ernst R (2001) Readings in Hardware/Software Co-Design, Morgan Kaufmann: 697.

Denzin NK & Lincoln YS (2005) Handbook of qualitative research, 3rd Edition, Thousands Oaks, Sage Publications: 1232.

61

Deszca G, Munro H & Noori H (1999) Developing breakthrough products: challenges and options for market assessment. Journal of Operation Management 17: 613–630.

Dey PK, Kinch J & Ogunlana SO (2007) Managing risk in software development projects: a case study. Industrial Management and Data Systems 107(2): 284–303.

Drejer A (2008) Are you innovative enough? International Journal of Innovation and Learning 5(1): 1–17.

Ebert C (2007) The impacts of software product management. Journal of Systems and Software 80(6): 850–861.

van Echtelt FEA, Wynstra F, van Weele AJ & Duysters G (2008) Managing Supplier Involvement in New Product Development: A Multiple-Case Study. Journal of Product Innovation Management 25(2): 180–201.

Eskola J & Suoranta J (1998) Johdatus laadulliseen tutkimukseen. Tampere, Vastapaino. Eskola J & Suoranta J (2008) Johdatus laadulliseen tutkimukseen. (in Finnish) Vastapaino,

Tampere, Finland: 266. Fan C-F & Yu Y-C (2004) BBN-based software project risk management. Journal of

Systems and Software 73(2): 193–203. Feigenbaum AV (1961) Total quality control. McGraw-Hill, New York. Feland JM, Leifer LJ & Cockayne WR (2004) Comprehensive Design Engineering:

Designers Taking Responsibility. International Journal of Engineering Education 20(3): 416–423.

Firesmith DG (2007) Common Requirements Problems, Their Negative Consequences, and Industry Best Practices to Help Solve Them. Journal of Object Technology 6(1): 17–33.

Ford DN & Sobek DK II (2005) Adapting real options to new product development by modeling the second Toyota paradox. IEEE Transactions on Engineering Management 52(2): 175–185.

Forsberg K & Mooz H (1992) The Relationship of Systems Engineering to the Project Cycle. Engineering Management Journal 4(3): 36–43.

Gershenson JK, Prasad GJ & Zhang Y (2003) Product modularity: definitions and benefits. Journal of Engineering Design 14(3): 295–313.

Gilb T (2005) Competitive Engineering: A Handbook for Systems Engineering, Requirements Engineering, and Software Engineering Using Planguage.

Gillies A & Howard J (2003) Managing change in process and people: combining a maturity model with a competency-based approach. TQM & Business Excellence 14(7): 779–787.

Gizopoulos D (2006) Advances in Electronic Testing: Challenges and Methodologies. Springer.

Goh TJ, Chiu C-P, Seetharamu KN, Quadir GA & Zainal ZA (2006) Test chip and substrate design for flip chip microelectronic package thermal measurements. Microelectronics International 23(2): 3–10.

Gottschalk P. & Solli-Saether H (2006) Maturity model for IT outsourcing relationships. Industrial Management and Data Systems 106(2): 200–212.

62

Gressgard LJ & Stensaker I (2006) The mobile service industry: strategic challenges and future business models. International Journal of Mobile Communications 4(5): 509–531.

Grimpe C (2007) Successful Product Development after Firm Acquisitions: The Role of Research and Development. Journal of Product Innovation Management 24(6): 614–628.

Gupta AK, Brockhoff K & Weisenfeld U (2003) Making Trade-Offs in the New Product Development Process: A German/US Comparison. Journal of Product Innovation Management 9(1): 11–18.

Gupta A, Pawara KS & Smart P (2007) New product development in the pharmaceutical and telecommunication industries: A comparative study. International Journal of Production Economics 106(1): 41–60.

Haapasalo H & Ylihoikka M (2004) Incremental Project Model in Software Development and Testing – Practical Experiences of the Applicability. Project Perspectives 27(1): 14–27.

Haapasalo H & Ekholm T (2004) A profile of European incubators: a framework for commercialising innovations. International Journal of Entrepreneurship and Innovation Management 4(2–3): 248–270.

Ham M, Jacobs J, Swinkels R & Van Veenendaal E (2001) Metric based testing maturity model framework v1.1. URI: http://tmitwww.tm.tue.nl/research/v2m2/.

Harisalo R (2008) Organisaatioteoriat. (in Finnish) University press, University of Tampere, Tampere, Finland.

Harkonen J, Belt P, Mottonen M, Kess P. & Haapasalo H. (2009) Maturity of verification and validation in ICT companies. International Journal of Innovation and Learning 6(1): 33–50.

Helo P (2004) Managing agility and productivity in the electronics industry. Industrial Management & Data Systems 104(7): 567–577.

Hersleb JD & Moitra D (2001) Global software development. IEEE, Software 18(2): 16–20.

Hines P, Francis M and Found P (2006) Towards lean product lifecycle management: A framework for new product development. Journal of Manufacturing Technology Management 17(7): 866–887.

Hirsjarvi S & Huttunen J (1995) Johdatus kasvatustieteeseen. 4th Edition, Helsinki, WSOY.

Hirsjärvi S, Remes P & Sajavaara P (2008) Tutki ja kirjoita. (in Finnish) 13th-14th Edition, Tammi, Helsinki, Finland: 448.

Holman R, Kaas H-W & Keeling D (2003) The future of product development. The McKinsey Quarterly, 3: 29–40.

Host M & Johansson C (2000) Evaluation of code review methods through interviews and experimentation. Journal of Systems and Software 52(2–3): 113–120.

Hsieh L-F, & Chen SK 2005. Incorporating voice of the consumer: does it really work? Industrial Management & Data Systems 105(6): 769–785.

63

Hsu J.S-C, Chan C-L, Liu JY-C & Chen H-G (2008) The impacts of user review on software responsiveness: Moderating requirements uncertainty. Information & Management 45(4): 203–210.

Humphrey WS (1989) Managing the Software Process. Reading, MA. Addison-Wesley. Jablokow KW & Booth DE (2006) The impact and management of cognitive gap in high

performance product development organizations. Journal of Engineering and Technology Management 23(4): 313–336.

Jacobs J & Trienekens J (2002) Towards a Metrics Based Verification and Validation Maturity Model. Lecture Notes in Computer Science 2434: 133–148.

Jacobs J, van Moll J, Kusters R, Trienekens J & Brombacher A (2007) Identification of factors that influence defect injection and detection in development of software intensive products. Information and Software Technology 49(7): 774–789.

Jahangiri J & Abercrombie D (2005) Value-added defect testing techniques. Design & Test of Computers, IEEE 22(3).

Joglekar NR & Rosenthal SR (2003) Coordination of Design Supply Chains for Bundling Physical and Software Products. Journal of Product Innovation Management 20(5): 374–390.

Jokela T, Siponen M, Hirasawa N & Earthy J (2006) A survey of usability capability maturity models: implications for practice and research. Behaviour & Information Technology 25(3): 263–282.

Juristo, N., Moreno, A. M. & Strigel, W. (2006) 'Software Testing Practices in Industry', Software, IEEE 23(4): 19–21.

Karlesky MJ, Bereza WI & Erickson CB (2006) Effective Test Driven Development for Embedded Software. Electro/information Technology, IEEE: 382–387.

Kasanen E, Lukka K & Siitonen A (1993) The Constructive Approach in Management Accounting Research. Journal of Management Accounting Research 5: 241–264.

Kessler EH (2000) Tightening the belt: methods for reducing development costs associated with new product innovation. Journal of Engineering and Technology Management 17(1): 59–92.

Kim J-Y, Wong V & Eng T-Y (2005) Product variety strategy for improving new product development proficiencies. Technovation 25(9): 1001–1015.

Koufteros X, Vonderembse M & Jayaram J (2005) Internal and External Integration for Product Development: The Contingency Effects of Uncertainty, Equivocality, and Platform Strategy. Decision Sciences 36(1): 97–133.

Kumar S & Fullenkamp J (2005) Analysis of European Union environmental directives and producer responsibility requirements. International Journal of Services and Standards 1(3): 379–398.

Kung DC, Hsia P & Gao J (1998) Testing object-oriented software. IEEE Computer Society Press: 280.

Laforet S (2008) Size, strategic, and market orientation affects on innovation. Journal of Business Research 61(7): 753–764.

Lakemond N & Berggren C (2006) Co-locating NPD? The need for combining project focus and organizational integration. Technovation 26(7): 807–819.

64

Lancaster G (2005) Research Methods in Management: A concise introduction to research in management and business consultancy', Butterworth-Heinemann,: 240.

Larman C & Basili VR (2003) Iterative and incremental developments; a brief history. IEEE Computer 36(6): 47–56.

Laurindo FJB & de Carvalho MM (2005) Changing product development process through information technology: a Brazilian case. Journal of Manufacturing Technology Management 16(3): 312–327.

Ledwith A, Richardson I & Sheahan A (2006) Small firm-large firm experiences in managing NPD projects. Journal of Small Business and Enterprise Development 13(3): 425–440.

Lee LT-S (2008) The effects of team reflexivity and innovativeness on new product development performance. Industrial Management & Data Systems 108(4): 548–569.

Lee M-C & Chang T (2006) Applying TQM, CMM and ISO 9001 in knowledge management for software development process improvement. International Journal of Services and Standards 2(1): 101–115.

Leeman DR & Winer RS (1997) Product Management. Irwin, Chicago, Il. Leenders MAAM & Wierenga B (2008) The effect of the marketing–R&D interface on

new product performance: The critical role of resources and scope. International Journal of Research in Marketing 25(1): 56–68.

Leonard A (2004) Focusing on maturity levels for the effective management of end users involved in software project teams. South African Journal of Business Management 35(1): 17–25.

Leonard-Barton D (1998) Wellsprings of Knowledge: Building and Sustaining the Sources of Innovation. Boston, MA. Harvard Business School Press: 334.

Leung HKN, Liao L & Qu Y (2007) Automated support of software quality improvement. International Journal of Quality & Reliability Management 24(3): 230–243.

Li Q & Kececioglu DB (2006) Design of an optimal plan for an accelerated degradation test: a case study. International Journal of Quality and Reliability Management 23(4): 426–440.

Liker JK, Sobek DK, Ward AC & Cristiano JJ (1996) Involving Suppliers in Product Development in the U.S. and Japan: Evidence for Set-Based Concurrent Engineering. IEEE Transactions on Engineering Management 43(2): 165–177.

Lynn GS & Akgun AE (2001) Project visioning: It's components and impact on new product success. Journal of Product Innovation Management 18(6): 374–387.

Maatta J, Harkonen J, Jokinen T, Mottonen M, Belt P, Muhos M. & Haapasalo H (2009) Managing testing activities in telecommunications: A case study. Journal of Engineering and Technology Management 26(1–2): 73–96.

Magnusson T, Lindstrom G & Berggren C (2003) Architectural or modular innovation? Managing discontinuous product development in response to challenging environmental performance targets. International journal of innovation management 7(1): 1–26.

65

Magnusson T & Berggren C (2001) Environmental innovation in auto development – managing technological uncertainty within strict time limits. International Journal of Vehicle Design 26(2–3): 101–115.

Mahanti R & Antony J, (2006) Six Sigma in software industries: some case studies and observations. International Journal of Six Sigma and Competitive Advantage 2(3): 263–290.

Matthews WH (1991) Kissing technological frogs: Managing technology as a strategic resource. European Management Journal 9(2): 145–148.

Manktelow KI (1999) Reasoning and Thinking. Psychology Press, 1st edition, Hove, UK: 259.

Mantel SJ, Meredith JR, Shafer SM & Sutton MM (2007) Project Management in Practice. 3rd Edition, Wiley: 336.

Marshall C & Rossman GB (1998) Designing Qualitative Research. 3rd edition, Thousand Oaks, Sage Publications:240.

Marwedel P (2006) Embedded System Design. Springer: 236. McCaffery F & Coleman G (2007) Developing a configuration management capability

model for the medical device industry. Int. Journal of Information Systems and Change Management 2(2): 139–154.

McGrath ME (1995) Product Strategy for High-technology Products. Irwin Professional, Publishing, New York, NY.

McGrath ME (1996) Setting the PACE in Product Development: A Guide to Product And cycle-time excellence., Butterworth-Heinemann, Boston, MA: 240.

McGrath ME (2001) Product Strategy for High Technology Companies. McGraw-Hill, 2nd edition, New York, NY.

McGrath ME (2004) Next Generation Product Development: How to Increase Productivity, Cut Costs, and Reduce Cycle Times’, McGraw-Hill Professional: 352.

Meyer MH & Utterback JM (1995) Product development cycle time and commercial success. IEEE Transactions on Engineering Management 42(4): 297–304.

Meyer MH, Lehnerd AP (1997) The Power of Product Platforms: Building Value and Cost Leadership. The Free Press, New York: 288.

Merton R, Fiske M & Kendall P (1990) The Focused Interview: A Manual of Problems and Procedures, 2nd ed. New York: The Free Press.

Moen RM (1998) New Quality Cost Model Used as a Top Management Tool. The TQM Magazine 10(5).

Moller MM, Johansen J & Boer H (2003) Managing buyer-supplier relationships and inter-organisational competence development. Integrated Manufacturing Systems 14(4): 369–379.

Moore GA (1999) Crossing the chasm. Marketing and selling technology products to mainstream customers. Capstone Publishing Limited, Oxford, United Kingdom.

Moore GE (1965) Cramming more components onto integrated circuits. Electronics Magazine 19.

Moore GE (1998) Cramming More Components onto Integrated Circuits. Proceedings of the IEEE 86(1): 82–85.

66

Mooz H, Forsberg K & Cotterman H (2003) Communicating project management: the integrated vocabulary of project management and systems engineering. John Wiley & Sons: 384.

Morgan JM & Liker JK (2006) The Toyota product development system: integrating people, process, and technology. Productivity Press: 377.

Moultrie J, Clarkson PJ & Probert D (2007) Development of a Design Audit Tool for SMEs. Journal of Product Innovation Management 24(4): 335–368.

Murto P (2007) Timing of investment under technological and revenue-related uncertainties. Journal of Economic Dynamics and Control 31(5): 1473–1497.

Neuvo Y (2004) Cellular phones as embedded systems. Solid-State Circuits Conference, Digest of Technical Papers. ISSCC. IEEE International: 32–37.

Nonaka I & Takeuchi H (1995) The Knowledge-creating Company: How Japanese Companies Create the Dynamics of Innovation. Oxford University Press: 284.

Nuseibeh B & Easterbrook S (2000) Requirements engineering: a roadmap. Proceedings of the Conference on The Future of Software Engineering, Limerick, Ireland: 35–46.

Oakland J 2000. TQM – Text with Cases. 2nd Edition, Butterworth-Heinemann, Oxford. O'Flynn B, Laffey D, Buckley J, Barton J & O'Mathuna SC (2007) Simulation, design,

development and test of antennas for wireless sensor network systems. Microelectronics International 24(2): 3–6.

Olkkonen, T. (1993) Johdatus teollisuustalouden tutkimustyöhön.(in Finnish), University of Technology, Helsinki, Industrial Economics and Industrial Psychology, Report No. 152: 114.

Paasivaara M (2005) Communication Practices In Interorganisational Product Development. TKK Dissertations 17, Helsinki, University of Technology, Espoo, Finland: 240.

Patrashkova RR & McComb SA (2004) Exploring why more communication is not better: insights from a computational model of cross-functional teams. Journal of Engineering and Technology Management 21(1–2): 83–114.

Patton MQ (2002) Qualitative Research and Evaluation Methods. 3rd edition, Thousand Oaks, Sage Publications: 598.

Paulk MC, Weber CV, Garcia SM, Chrisis MB & Bush M (1993) Key Practices of the Capability Maturity ModelSM, Version 1.1. Technical Report CMU/SEI-93-TR-025, ESC-TR-93–178, Software Engineering Institute, Carnegie Mellon University, Pittsburgh, PA.

Perttula AO (2005) Changing Focus of Verification at Different Phases of Fast Cycle Electronics Product Development. Stevens Institute of Technology, Proceedings CSER, March 23–25, Hoboken, NJ, USA.

Perttula A (2007) Challenges and Improvements of Verification and Validation Activities in High Volume Electronics Product Development. PhD dissertation, Tampere University of Technology, Finland. Publication 650.

Pisano GP & Wheelwright SC (1995) The new logic of high-tech R&D. Harvard Business Review 73(5): 93–104.

67

Pol M, & Veenendaal E (1998) Structured Testing of Information Systems – an Introduction to TMap. Kluwer Bedrijfs Informatie, Deventer.

Porter LJ & Rayner P (1992) Quality Costing for Total Quality Management. International Journal of Production Economics, 27. Elsevier Science Publisher B. V.

Pressman R (2004) Software Engineering: A Practitioner's Approach. 6th Edition, McGraw-Hill: 880.

Radeka K & Sutton T (2007) What is “lean” about product development? An overview of Lean Product Development. PDMA Visions 31(2): 11–15.

Ratamaki K (2004) Product Platform Development from the Product Lines Perspective: Case of Switching Platform. Acta Universitatis Lappeenrantaensis, publication No. 201.

Redondo YP & Fierro JJC (2007) Importance of company size in long-term orientation of supply function: an empirical research. Journal of Business & Industrial Marketing 22(4): 236–248.

Robertson D & Ulrich K (1998) Planning for Product Platforms. Sloan Management Review 39(4): 19–31.

Rumney M (2003) Position statement by panellist Moray Rumney of Agilent Technologies. International Test Conference, ITC 2003 Panel Discussion Testing 3G-controlled systems: time to rejoice or time to feel pain? URI: http://csdl2.computer.org/comp/ proceedings/itc/2003/2063/00/20631322.pdf.

Salonen M, Holtta-Otto K & Otto K (2008) Effecting product reliability and life cycle costs with early design phase product architecture decisions. International Journal of Product Development 5(1–2): 109–124.

Salustri FA & Parmar J (2003) Visualising early product development information, CDEN: 1–9.

Sengupta JK (2005) Nonparametric efficiency analysis under uncertainty using data envelopment analysis. International Journal of Production Economics 95(1): 39–49.

Siggelkow N (2007) Persuasion with case studies. Academy of Management Journal 50(1): 20–24.

Sippola K (2000) Quality Cost Measurement as a Tool for Top Management in Contemporary Software Business Environment. University of Oulu.

Skander A, Roucoules L & Meyer JSK (2008) Design and manufacturing interface modelling for manufacturing processes selection and knowledge synthesis in design. The International Journal of Advanced Manufacturing Technology 37(5–6): 443–454.

Song M & Montoya-Weiss MM (2001) The effect of perceived technological uncertainty on Japanese new product development. The Academy of Management Journal 44(1): 61–80.

Suikki R (2007) Changing business environment—effects of continuous innovations and disruptive technologies. Acta Universitatis Ouluensis Technica, C 265: 90.

Suomala P (2004) The Life Cycle Dimension of New Product Development Performance Measurement. International Journal of Innovation Management. 8(2): 193–221.

Suomala P & Jokioinen I (2003) The patterns of success in product development: a case study. European Journal of Innovation Management 6(4): 213–227.

68

Talby D, Keren A, Hazzan O & Dubinsky Y (2006) Agile software testing in a large-scale project. Software, IEEE 23(4): 30–37.

Tidd J & Bessant J (2009) Managing Innovation: Integrating Technological, Market and Organizational Change. 4th Edition, Wiley: 640.

Tiku S, Azarian M & Pecht M (2007) Using a reliability capability maturity model to benchmark electronics companies. International Journal of Quality & Reliability Management 24(5): 547–563.

Tsai B-Y (2005) A novel hybrid object-oriented class testing method. International Journal of Services and Standards 1(4): 512–524.

Tuomi J & Sarajärvi A (2006) Laadullinen tutkimus ja sisällön analyysi. (in Finnish), Tammi, Helsinki, Finland: 158.

Ulrich KT & Eppinger SD (1995) Product Design and Development. McGraw-Hill, New York.

Ulrich KT & Eppinger SD (2007) Product Design and Development. 4th edition, McGraw-Hill: 384.

Vakaslahti P (1997) Process Improvement frameworks – a Small Case Study with People Capability Maturity Model. Software Process – Improvement and Practice 3: 225–234. TeraQuest Metrics. USA.

Wang C-H, Lu I-Y & Chen C-B (2008) Evaluating firm technological innovation capability under uncertainty. Technovation, Vol. 28, No. 6, pp. 349–363.

Ward AC (2007) Lean Product and Process Development. Lean Enterprise Institute, Cambridge, MA: 208.

Weber M & Weisbrod J (2003) Requirements engineering in automotive development: Experiences and challenges. IEEE Software 20(1): 16–24.

Wheelwright SC & Clark KB (1992a) Revolutionizing Product Development. The Free Press, New York.

Wheelwright SC & Clark KB (1992b) Creating Project Plans to Focus., Harvard Business Review 70(2): 70–82.

Whittaker JA & Voas JM (2002) 50 years of software: key principles for quality. IT Professional 4(6): 28–35.

Woodward MR & Hennell MA (2005) Strategic benefits of software test management: a case study. Journal of Engineering and Technology Management 22(1–2):113–140.

Wynstra F, van Weele A & Weggemann M (2001) Managing supplier involvement in product development: Three critical issues. European Management Journal 19(2): 157–167.

Yakob R & Tell F (2007) Managing near decomposability in complex platform development projects. International Journal of Technology Intelligence and Planning 3(4): 387–407.

Yang C & Jiang S (2006) Strategies for Technology Platforms. Research-Technology Management 49(3): 48–57.

Yassine AA, Sreenivas RS & Zhu J (2008) Managing the exchange of information in product development. European Journal of Operational Research 184(1): 311–326.

69

Yin RK (2003) Case Study Research: Design and Methods. 3rd Edition. Sage Publications, Thousand Oaks: 200.

Yoo C, Yoon J, Lee B, Lee C, Lee J, Hyun S & Wu C (2006) A unified model for the implementation of both ISO 9001:2000 and CMMI by ISO-certified organizations. Journal of Systems and Software 79(7): 954–961.

Zhang Q, Vonderembse MA & Lim J-S (2002) Value chain flexibility: a dichotomy of competence and capability. International Journal of Production Research 40(3): 561–583.

Zhang Q & Doll WJ (2001) The fuzzy front end and success of new product development: a causal model. European Journal of Innovation Management 4(2): 95–112.

Zhang Q, Vonderembse MA & Cao M (2008) Product concept and prototype flexibility in manufacturing: Implications for customer satisfaction. European Journal of Operational Research 194(1): 143–154.

71

Original publications

This dissertation is based on the following publications:

I Belt P, Oiva-Kess A, Harkonen J, Mottonen M & Kess P (2009) Organisational maturity and functional performance. International Journal of Management and Enterprise development 6(2): 147–164. DOI: 10.1504/IJMED.2009.022624.

II Belt P, Harkonen J, Mottonen M, Kess P & Haapasalo H (2008) Improving the efficiency of verification and validation. International Journal of Services and Standards 4(2): 150–166. DOI: 10.1504/IJSS.2008.016630.

III Belt P, Haapasalo H, Harkonen J, Mottonen M & Kess P (in press) Improving product development in different types of ICT companies. International Journal of Innovation and Learning.

IV Belt P, Harkonen J, Mottonen M, Kropsu-Vehkapera H & Haapasalo H (in press) Technological uncertainty and verification & validation activities. International Journal of Innovation and Learning.

Reprinted with permission from Inderscience (I–IV). Inderscience retains the copyright to the original articles.

Original publications are not included in the electronic version of the dissertation.


Distributed byOULU UNIVERSITY LIBRARY

P.O. Box 7500, FI-90014University of Oulu, Finland

Book orders:OULU UNIVERSITY PRESSP.O. Box 8200, FI-90014University of Oulu, Finland

S E R I E S C T E C H N I C A

307. Kuismanen, Kimmo (2008) Climate-conscious architecture—design and windtesting method for climates in change

308. Kangasvieri, Tero (2008) Surface-mountable LTCC-SiP module approach forreliable RF and millimetre-wave packaging

309. Metsärinta, Maija-Leena (2008) Sinkkivälkkeen leijukerrospasutuksen stabiilisuus

310. Prokkola, Jarmo (2008) Enhancing the performance of ad hoc networking bylower layer design

311. Löytynoja, Mikko (2008) Digital rights management of audio distribution in mobilenetworks

312. El Harouny, Elisa (2008) Historiallinen puukaupunki suojelukohteena jaelinympäristönä. Esimerkkeinä Vanha Porvoo ja Vanha Raahe. Osa 1

312. El Harouny, Elisa (2008) Historiallinen puukaupunki suojelukohteena jaelinympäristönä. Esimerkkeinä Vanha Porvoo ja Vanha Raahe. Osa 2

313. Hannuksela, Jari (2008) Camera based motion estimation and recognition forhuman-computer interaction

314. Nieminen, Timo (2009) Detection of harmful microbes and their metaboliteswith novel methods in the agri-food production chain

315. Marjala, Pauliina (2009) Työhyvinvoinnin kokemukset kertomuksellisinaprosesseina–narratiivinen arviointitutkimus

316. Ahola, Juha (2009) Reaction kinetics and reactor modelling in the design ofcatalytic reactors for automotive exhaust gas abatement

317. Koskimäki, Heli (2009) Utilizing similarity information in industrial applications

318. Puska, Henri (2009) Code acquisition in direct sequence spread spectrumsystems using smart antennas

319. Saari, Seppo (2009) Knowledge transfer to product development processes. Amultiple case study in two small technology parks

320. Komulainen, Mikko (2009) Bandwidth enhanced antennas for mobile terminalsand multilayer ceramic packages

321. Ronkanen, Anna-Kaisa (2009) Hydrologic and hydraulic processes in northerntreatment peatlands and the significance for phosphorus and nitrogen removal

322. Liedes, Toni (2009) Improving the performance of the semi-active tuned massdamper

C324etukansi.fm Page 2 Monday, May 18, 2009 10:08 AM

ABCDEFG

UNIVERS ITY OF OULU P.O.B . 7500 F I -90014 UNIVERS ITY OF OULU F INLAND


S E R I E S E D I T O R S

SCIENTIAE RERUM NATURALIUM

HUMANIORA

TECHNICA

MEDICA

SCIENTIAE RERUM SOCIALIUM

SCRIPTA ACADEMICA

OECONOMICA

EDITOR IN CHIEF

PUBLICATIONS EDITOR

Professor Mikko Siponen

University Lecturer Elise Kärkkäinen

Professor Hannu Heusala


Senior Researcher Eila Estola

Information officer Tiina Pistokoski

University Lecturer Seppo Eriksson


Publications Editor Kirsti Nurkkala

ISBN 978-951-42-9147-0 (Paperback)ISBN 978-951-42-9148-7 (PDF)ISSN 0355-3213 (Print)ISSN 1796-2226 (Online)


TECHNICA


TECHNICA

OULU 2009

C 324

Pekka Belt

IMPROVING VERIFICATION AND VALIDATION ACTIVITIES IN ICT COMPANIES—PRODUCT DEVELOPMENT MANAGEMENT APPROACH

FACULTY OF TECHNOLOGY,DEPARTMENT OF INDUSTRIAL ENGINEERING AND MANAGEMENT,UNIVERSITY OF OULU

C 324

ACTA

Pekka Belt

C324etukansi.fm Page 1 Monday, May 18, 2009 10:08 AM

series editors technica a scientiae rerum naturalium...

Documents