Studies in Fuzziness and Soft Computing

Volume 303

Series Editor

Janusz Kacprzyk, Warsaw, Poland

For further volumes:http://www.springer.com/series/2941

The series ‘‘Studies in Fuzziness and Soft Computing’’ contains publications onvarious topics in the area of soft computing, which include fuzzy sets, rough sets,neural networks, evolutionary computation, probabilistic and evidential reasoning,multi-valued logic, and related fields. The publications within ‘‘Studies in Fuzzi-ness and Soft Computing’’ are primarily monographs and edited volumes. Theycover significant recent developments in the field, both of a foundational andapplicable character. An important feature of the series is its short publication timeand world-wide distribution. This permits a rapid and broad dissemination ofresearch results. Contact the series editor by e-mail: kacprzyk@ibspan.waw.pl.

Ajeet Kumar Pandey • Neeraj Kumar Goyal

Early Software ReliabilityPrediction

A Fuzzy Logic Approach

123

Ajeet Kumar PandeyAECOMHyderabad, Andhra PradeshIndia

Neeraj Kumar GoyalReliability Engineering CentreIndian Institute of Technology KharagpurKharagpur, West BengalIndia

ISSN 1434-9922 ISSN 1860-0808 (electronic)ISBN 978-81-322-1175-4 ISBN 978-81-322-1176-1 (eBook)DOI 10.1007/978-81-322-1176-1Springer New Delhi Heidelberg New York Dordrecht London

Library of Congress Control Number: 2013940762

� Springer India 2013This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part ofthe material is concerned, specifically the rights of translation, reprinting, reuse of illustrations,recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission orinformation storage and retrieval, electronic adaptation, computer software, or by similar or dissimilarmethodology now known or hereafter developed. Exempted from this legal reservation are briefexcerpts in connection with reviews or scholarly analysis or material supplied specifically for thepurpose of being entered and executed on a computer system, for exclusive use by the purchaser of thework. Duplication of this publication or parts thereof is permitted only under the provisions ofthe Copyright Law of the Publisher’s location, in its current version, and permission for use mustalways be obtained from Springer. Permissions for use may be obtained through RightsLink at theCopyright Clearance Center. Violations are liable to prosecution under the respective Copyright Law.The use of general descriptive names, registered names, trademarks, service marks, etc. in thispublication does not imply, even in the absence of a specific statement, that such names are exemptfrom the relevant protective laws and regulations and therefore free for general use.While the advice and information in this book are believed to be true and accurate at the date ofpublication, neither the authors nor the editors nor the publisher can accept any legal responsibility forany errors or omissions that may be made. The publisher makes no warranty, express or implied, withrespect to the material contained herein.

Printed on acid-free paper

Springer is part of Springer Science+Business Media (www.springer.com)

Late Prof. R. B. Misra

Preface

Due to continuous growth in size and complexity of software system, reliabilityassurance becomes tough for developers while satisfying user expectations.Applicability of software keeps on increasing for products/systems ranging frombasic home appliances to safety critical business applications. Development ofreliable software with acceptable level of quality within available time frame andbudget has become a challenging objective. This objective could be achieved tosome extent through early prediction of number of faults present in the software.Early prediction of faults provides an opportunity to make early corrections duringdevelopment process and reduces the cost of development. Further it also helps inbetter planning and utilization of development resources. This book is an upshot ofour research at Reliability Engineering Center IIT Kharagpur, India. During ourresearch we struggled a lot to find the way to predict software reliability at earlystages of software development.

The book is intended to serve three associated audiences: software profes-sionals, project managers, and researchers. For software professionals, this bookserves as a means to understand reliability of their product, the potential factorsaffecting it, and adoption of best practices to reduce number of errors in thesoftware. For software project manager, this book serves as a collection of tools,which can be easily modeled and used for monitoring reliability of softwarethroughout the development process. Further, the reliability information obtainedcan be used for reducing software development time and cost. For software reli-ability and quality researcher, this book provides important references in the area.Besides, the models proposed in this book are basic framework. This frameworkcan be put to critical investigation to find better methods which remain easy tounderstand, implement, and use. We hope that this book will help software pro-fessionals and researchers to solve complicated problems in the area of earlysoftware reliability with ease.

The book is divided into seven chapters. Chapter 1 starts with a discussion onthe need of reliability and quality software. By providing various standard defi-nitions of software reliability, error, fault, and failure, chapter has discussed twoapproaches for measuring reliability of a software system. First, a developer’sview focusing on software faults; if the developer has grounds for believing thatthe software is relatively fault-free, then the system is assumed to be reliable.

vii

Second, a user-based view which defines software reliability as ‘‘the probabilityfailure-free software operation for a specified period of time in a specified envi-ronment’’. Irrespective of these viewpoints, software reliability measurementincludes two types of activities: estimation and prediction. This chapter also dis-cusses about the software reliability growth model (SRGMs) and its limitations.Chapter has highlighted the need of early software reliability prediction, itspractical challenges, and solutions. The reliability relevant metrics and CapabilityMaturity Model (CMM) are also discussed in the chapter.

Chapter 2 provides background on software reliability and quality predictions.This chapter has focused on the literature surveys on software reliability models,reliability relevant software metrics, software capability maturity models, softwaredefect prediction model, software quality prediction models, regression testing,software, and operational profile.

Chapter 3 proposes an early fault prediction model using software metrics andprocess maturity. For this, various reliability relevant software metrics applicableto different phase of software development are collected along with processmaturity. Model approach is modular in the sense that it can be applied at eachphase of Software Development Life Cycle (SDLC) and suitable for all type ofsoftware projects.

Chapter 4 has focused on fault density which seems to be a more desirablemeasure than number of fault for reliability and quality indicator. Fault densitymetric can be found for requirement document, design document, code document,and also for test report. The chapter proposes a multistage fault prediction modelwhich predicts the number of residual faults present in the software as softwareindustries are more interested to know residual faults present in their software.

Chapter 5 presents an approach to classify software modules as fault-prone (FP)and not fault-prone (NFP). Besides, the approach also predicts degree of faultproneness on a scale of zero to one, and ranks the module accordingly. ID3algorithm along with fuzzy inference system is used for classifying softwaremodules as FP or NFP. ID3 algorithm provides the required rules by analyzinghistory data for proposed fuzzy inference system.

Chapter 6 discusses an integrated and cost-effective approach to test prioriti-zation that increases the test-suite’s fault detection rate. The proposed approachconsiders the three important factors, program change level (PCL), test-suitechange level (TCL) and test-suite size (TS), before applying any techniques toexecute test-cases. These factors can be derived by using the information from themodified program version. A cost-effective reliability centric test case prioritiza-tion is also discussed in this chapter.

Chapter 7 discusses the way to develop the operational profile of a softwaresystem, to compute the number of test case required, and the way to allocate them.The chapter explains how the reliability of a software-based product depends on itsoperational profile. If developed early, an operational profile may be used toprioritize the development process, so that more resources are put on the mostimportant operations. A case study using the traditional and operational profile-based testing is also discussed.

viii Preface

Appendices containing the data set used in various chapters of this are given theend. Readers are advised to synchronize with these dataset while utilizing themodel for reliability prediction of any software or module. Any forms of con-structive suggestions to improve our existing models are welcome. Readers canmail their queries/suggestions at ajeet.mnnit@gmail.com.

A. K. PandeyN. K. Goyal

Preface ix

Acknowledgments

Writing the acknowledgment is the loveliest and toughest part. This book has beenan outcome of our research at IIT Kharagpur and Cognizant Technology Solutions(CTS), Hyderabad, India. I am indebted to many persons from these organizationswho directly or indirectly helped us during the preparation of the manuscript. It ismy pleasure to acknowledge their help.

First, I would like to thank my professors, Prof. V. N. Naikan, Prof. S. K.Chaturvedi, and Prof. N. K. Goyal who not only introduced the subject to me buthelped me a lot to explore the new opportunities and challenges of reliabilityengineering. I express my special thanks to Late Prof. R. B. Misra; he motivatedme to start the journey of software reliability.

I acknowledge the help and cooperation received from Prof. P. K. J. Mahapatra,Prof. V. N. Giri, and Prof. U. C. Gupta during my stay at IIT Kharagpur. Thanksare also due to our EMS CoE coworkers of CTS, Hyderabad. I express my specialthanks to Dr. Phanibhushan Sistu, Vivek Diwanji, and Smith Jessy, for theirconstant motivation and support. Other friends, who reviewed the manuscript andmade helpful suggestions, are Abhisek Gupta, Ravi Kumar Mutya, Srinivas Pan-changam, and Nitin Jain of CTS Hyderabad, Rakesh Chandra Verma of TCSChennai, Prof. Rajesh Mishra of Gautam Buddha University, Greater Noida, Prof.Vivek Srivastava of NIT Delhi. Thanks are due to Prof. D. K. Yadav of NITJamshedpur, Prof. P. K. Kapur of Amity University, Noida, and Prof. M. M. Goreof MNNIT, Allahabad. I wish to thank Praveen Goyal, Director (Technical) atAECOM, Hyderabad; Dr. V. S. K. Reddy and Dr. D. Ramesh of MRCET,Hyderabad for their decent support during the last hours to finalize the manuscripton time. I also wish to thank Aninda Bose of Springer for his enthusiasm, patient,and support since beginning of the book, as well as Suguna Ramalingam,Production Editor (Springer), and his staff for their meticulous effort regardingproduction. I gratefully acknowledge the anonymous reviewers for theirconstructive comments and suggestions to improve this book significantly.

Special acknowledgment to MHRD, Government of India and administration ofCTS, Hyderabad, India, for providing financial and administrative support duringthe draft of this book.

xi

Finally, I wish to express my sincere gratitude to my parents Dr. D. P. Pandeyand Kalawati Pandey, my parents-in-law Dr. M. L. Trivedi and Santosh Trivedi fortheir AASHIRVAD. I would like to acknowledge the continuous support providedby my wife Shweta and two kids Avichal and Khyati, whose laughter used toabsorb my tiredness and let me complete this manuscript successfully.

Finally, I would like to wind up by paying my heartfelt thanks and prayers tothe Almighty, for his love and grace.

A. K. Pandey

xii Acknowledgments

Contents

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1 Need for Reliable and Quality Software . . . . . . . . . . . . . . . . . 11.2 Software Reliability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.2.1 Software Error, Fault, and Failure . . . . . . . . . . . . . . . . 21.2.2 Measuring Software Reliability . . . . . . . . . . . . . . . . . . 31.2.3 Limitation of Software Reliability Models . . . . . . . . . . 4

1.3 Why Fault Prediction? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.4 Software Fault Prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

1.4.1 Software Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61.4.2 Capability Maturity Model Level. . . . . . . . . . . . . . . . . 61.4.3 Limitation of Early Reliability Prediction Models . . . . . 71.4.4 Early Software Fault Prediction Model. . . . . . . . . . . . . 7

1.5 Residual Fault Prediction Model . . . . . . . . . . . . . . . . . . . . . . 81.5.1 Software Development Life Cycle and Fault Density . . . 81.5.2 Software Metrics and Fault Density Indicator . . . . . . . . 10

1.6 Quality of Large Software System . . . . . . . . . . . . . . . . . . . . . 101.6.1 Fault-Prone and Not Fault-Prone Software Modules. . . . 111.6.2 Fault-Proneness Factors . . . . . . . . . . . . . . . . . . . . . . . 111.6.3 Need for Software Module Classification . . . . . . . . . . . 111.6.4 Limitations with Earlier Module Prediction Models . . . . 12

1.7 Regression Testing and Software Reliability . . . . . . . . . . . . . . 121.8 Software Reliability and Operational Profile . . . . . . . . . . . . . . 131.9 Organization of the Book . . . . . . . . . . . . . . . . . . . . . . . . . . . 14References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

2 Background: Software Quality and Reliability Prediction . . . . . . . 172.1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172.2 Software Reliability Models . . . . . . . . . . . . . . . . . . . . . . . . . 17

2.2.1 Failure Rate Models. . . . . . . . . . . . . . . . . . . . . . . . . . 182.2.2 Failure or Fault Count Models . . . . . . . . . . . . . . . . . . 192.2.3 Error or Fault-Seeding Models . . . . . . . . . . . . . . . . . . 192.2.4 Reliability Growth Models . . . . . . . . . . . . . . . . . . . . . 20

xiii

2.3 Architecture-based Software Reliability Models. . . . . . . . . . . . 212.4 Bayesian Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222.5 Early Software Reliability Prediction Models . . . . . . . . . . . . . 222.6 Reliability-Relevant Software Metrics. . . . . . . . . . . . . . . . . . . 232.7 Software Capability Maturity Models . . . . . . . . . . . . . . . . . . . 242.8 Software Defects Prediction Models . . . . . . . . . . . . . . . . . . . . 252.9 Software Quality Prediction Models . . . . . . . . . . . . . . . . . . . . 262.10 Regression Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282.11 Operational Profile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292.12 Observations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

3 Early Fault Prediction Using Software Metricsand Process Maturity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353.1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353.2 Brief Overview of Fuzzy Logic System . . . . . . . . . . . . . . . . . 363.3 Proposed Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

3.3.1 Description of Metrics Considered for the Model . . . . . 393.4 Implementation of the Proposed Model . . . . . . . . . . . . . . . . . 46

3.4.1 Information Gathering Phase . . . . . . . . . . . . . . . . . . . . 463.4.2 Information Processing Phase . . . . . . . . . . . . . . . . . . . 493.4.3 Defuzzification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 493.4.4 Fault Prediction Phase . . . . . . . . . . . . . . . . . . . . . . . . 50

3.5 Case Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 513.6 Results and Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 543.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

4 Multistage Model for Residual Fault Prediction . . . . . . . . . . . . . . 594.1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 594.2 Research Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

4.2.1 Software Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . 604.2.2 Fault Density Indicator . . . . . . . . . . . . . . . . . . . . . . . . 62

4.3 Overview of the Proposed Model. . . . . . . . . . . . . . . . . . . . . . 624.3.1 Description of Software Metrics and their Nature . . . . . 64

4.4 Model Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 674.4.1 Independent and Dependent Variables . . . . . . . . . . . . . 674.4.2 Development of Fuzzy Profiles . . . . . . . . . . . . . . . . . . 674.4.3 Development of Fuzzy Rules . . . . . . . . . . . . . . . . . . . 694.4.4 Information Processing . . . . . . . . . . . . . . . . . . . . . . . . 734.4.5 Residual Fault Prediction . . . . . . . . . . . . . . . . . . . . . . 74

xiv Contents

4.5 Case Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 754.5.1 Dataset Used . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 754.5.2 Metrics Considered in ‘‘qqdefects’’ Dataset. . . . . . . . . . 754.5.3 Conversion of Dataset . . . . . . . . . . . . . . . . . . . . . . . . 75

4.6 Results and Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 764.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

5 Prediction and Ranking of Fault-Prone Software Modules. . . . . . . 815.1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 815.2 Research Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

5.2.1 Data Mining . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 835.2.2 Software Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . 845.2.3 Fuzzy Set Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

5.3 Proposed Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 865.3.1 Assumptions and Architecture of the Model . . . . . . . . . 86

5.4 Model Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 865.4.1 Training Data Selection . . . . . . . . . . . . . . . . . . . . . . . 875.4.2 Decision Tree Construction. . . . . . . . . . . . . . . . . . . . . 885.4.3 Estimating Classifier Accuracy . . . . . . . . . . . . . . . . . . 895.4.4 Module Ranking Procedure . . . . . . . . . . . . . . . . . . . . . 905.4.5 An Illustrative Example . . . . . . . . . . . . . . . . . . . . . . . 91

5.5 Proposed Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 925.6 Case Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

5.6.1 Dataset Used . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 935.6.2 Converting Data in Appropriate Form . . . . . . . . . . . . . 955.6.3 Resultant Decision Tree . . . . . . . . . . . . . . . . . . . . . . . 97

5.7 Results and Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 975.8 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103

6 Reliability Centric Test Case Prioritization . . . . . . . . . . . . . . . . . . 1056.1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1056.2 Earlier Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1066.3 Test Case Prioritization. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107

6.3.1 Test Case Prioritization Techniques . . . . . . . . . . . . . . . 1086.3.2 APFD Metric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109

6.4 Proposed Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1106.5 Results and Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1146.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115

Contents xv

7 Software Reliability and Operational Profile . . . . . . . . . . . . . . . . . 1177.1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1177.2 Backgrounds and Related Works . . . . . . . . . . . . . . . . . . . . . . 118

7.2.1 Embedded Systems’ Testing . . . . . . . . . . . . . . . . . . . . 1187.2.2 Function Point Metric . . . . . . . . . . . . . . . . . . . . . . . . 1187.2.3 Operational Profile. . . . . . . . . . . . . . . . . . . . . . . . . . . 119

7.3 Proposed Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1217.3.1 Premise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1217.3.2 Model Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . 121

7.4 Case Study: Automotive Embedded ECU . . . . . . . . . . . . . . . . 1237.4.1 Fog Light ECUs . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1247.4.2 Functional Complexity of Fog Light ECU . . . . . . . . . . 1247.4.3 Operational Profile of Fog Light ECU . . . . . . . . . . . . . 1267.4.4 Test Case Generation . . . . . . . . . . . . . . . . . . . . . . . . . 1287.4.5 Test Case and Transition Probability . . . . . . . . . . . . . . 128

7.5 Results and Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1287.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130

Appendix A. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131

Appendix B . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135

Appendix C. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139

About the Author . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153

xvi Contents

Acronyms

LOC McCabe’s line count of codeCC McCabe’s cyclomatic complexityEC McCabe’s essential complexityDC McCabe’s design complexityEL Halstead’s executable line countCL Halstead’s comments lineBL Halstead’s blank linesCCL Code and comment linen1 unique operatorsn2 unique operandN1 Total operatorsN2 Total operandsBC Branch countsFP Fault-prone (goal metric)

xvii

Notation and Abbreviations

APFD Average percentage of faults detectedCM Coding metricsCMM Capability maturity modelDM Design metricsFCP Number of faults at the end of coding phaseFDC Fault density indicator at the end of coding phaseFDD Fault density indicator at the end of design phaseFDP Number of faults at the end of design phaseFDR Fault density indicator at the end of requirement phaseFDT Fault density indicator at the end of testing phaseFIS Fuzzy inference systemFC Functional complexityFP Function pointFP Module Fault-prone moduleFRP Number of faults at the end of requirement phaseID3 Interactive Dichotomizer 3NFP Module Not fault-proneOP Operational profileOPBT Operational profile-based testingOEM Original equipment manufacturerPCL Program change levelRC Requirement complexityRTS Regression test selectionRTM Regression test metricsRM Requirements metricsRRSMs Reliability relevant software metricsSRGMs Software reliability growth modelsTCP Test case prioritizationTSM Test-suite minimizationTCL Test-suite change levelTS Test-suite sizeTFN Triangular fuzzy numbersTM Testing metrics

xix