study of empirical approaches to analyze the software metrics

Study of Empirical Approaches to Analyze the Software Metrics

Adesh Kr. Pandey Department of Information Technology

Krishna Institute of Engg. & Tech., Ghaziabad, India

[email protected]

C.P Agrawal Department of Computer Application

M.C.N.U.J.C., Bhopal, India

[email protected]

Arun Sharma Department of Computer Science & Engg.

Krishna Institute of Engg. & Tech., Ghaziabad, India

[email protected]

P. Sasikala Department of New Media Technology

M.C.N.U.J.C., Bhopal, India

[email protected]

ABSTRACT Software affects nearly every aspect of human lives. Software functional quality is a key to achieve industrial and business relevance, in particular to industrial development and growth.

Software metrics are important indicators to improve the processes and products in all organizations. They define baselines of quality and productivity and enable comparisons against industry averages that help in identifying opportunities for improvement. In addition, Software metrics design and analysis are major activities in the software development life cycle. Software metrics play a vital role in software cost, quality, scheduling, reliability and maintenance. There are various methods to decide which metrics should be used for which purposes. Attributes of a metric may be independent; or attributes may depend on one another.

Analytical Hierarchy Process (AHP) is used to assign weights to various parameters of a decision model when they are related to each other in a particular hierarchy. Analytical Network Process (ANP) and Fuzzy ANP are used to solve the decision problem, where attributes of decision parameters form dependency networks. The objective of this paper is to explore the possibilities of using empirical approaches like AHP, ANP and Fuzzy ANP to analyze the software metrics by measuring the weights of different attributes.

Categories and Subject Descriptors D.2.8 [software engineering]: Metrics/Measurement

Keywords Software metrics, measurement, attributes, AHP, ANP, fuzzy ANP,

dependence.

1. INTRODUCTION Empirical approaches like AHP and ANP provide efficient mechanisms to check consistency of the evaluation measures and alternatives to reduce bias in decision making [15, 24]. They also assist to compute qualitative measures for specific attributes of a software project. AHP and ANP allow organizations to minimize common pitfalls of decision making such as lack of focus, planning, participation and ownership. These techniques are useful when the decision-making is complex and unstructured. Any complex situation, in which the decision cycle involves a variety of criteria and requires structuring, measurement and synthesis, is a good candidate for AHP and ANP [13]. Broad areas where AHP and ANP have been successfully employed include: selection of one alternative from many, resource allocation, forecasting,

total quality management, business process re-engineering, quality function deployment, and the balanced scorecard.

Now a days, software development is iterative and incremental due to continual changes in customer’s requirements. This paper explores the different empirical approaches, which may be used to reduce the dimensionality of software metrics, by measuring the weights of different attributes too.

The paper is categorized into sections like AHP, ANP, AHP versus ANP, fuzzy set theory, fuzzy AHP and ANP, adopted methodology, motivation of using empirical approaches, analysis of the proposed concept and conclusion.

1.1 Analytical Hierarchy Process (AHP) Studies in the literature identify the multi-criteria decision technique, known as Analytical Hierarchy Process (AHP) to be the most appropriate for solving complicated problems. AHP was proposed by Saaty [24] as a method of solving socioeconomic decision-making problems. AHP is a comprehensive framework that is designed to make multi-objective, multi-criteria and multi-factor decisions with or without certainty. In AHP we arrange the factors of decision making, once selected, in a hierarchical structure descending from an overall goal to criteria, sub-criteria and alternatives in successive levels. The basic assumptions of AHP are that it can be used independently of an upper part or cluster of the hierarchy.

AHP is a decision analysis technique that reduces dimensionality of problems. Decisions are determined by a single number for the best outcome or by a vector of priorities that gives an ordering of the different possible outcomes.

1.1.1 Applications of AHP Today AHP is widely used in industry. The Xerox Corporation uses AHP for R&D decisions in portfolio management, technology implementation, and engineering design selection. It helps in making marketing decisions regarding market segment prioritization, product-market matching, and customer requirement structuring [14]. Now a days, AHP is applied in the selection of products, suppliers and consultants [14]. It has been used to select the new management structure for the Edgewood Research Development and Engineering Center (ERDEC) [18]. AHP was applied to perform a study to select a propulsion system for the Lunar Lander [19]. The concept of AHP effectively evaluates the stream suitability of the habitat for rainbow trout, including reproduction considerations and over-winter survival [17]. Best diagnostic management of acute upper gastrointestinal bleeding was decided by using AHP [20].

ACM SIGSOFT Software Engineering Notes July 2013 Volume 38 Number 4

DOI: 10.1145/2492248.2492270 http://doi.acm.org/10.1145/ 2492248.2492270

Don Petkov and his colleagues [21] have pinpointed many of the factors involved in software productivity and laid out their relationships in an AHP hierarchy for prioritizing software development productivity factors. The Air Products and Chemicals company uses a systematic project-selection process based on AHP to improve the continuous progress and completion of its projects [23].

AHP was applied to benchmark IBM’s computer integrated manufacturing processes against the other best-of-breed companies throughout the world [22]. AHP was used to estimate the complexity of the structure of business sectors [3]. A methodology has been proposed to evaluate public complaints using the AHP technique, which is generally described as multi-criteria Decision Making (MCDM) [13]. Sharma et. al [8] applied the AHP technique to assign the weight values to the software quality attributes. These weight values are then used to evaluate the quality contribution of sub-characteristics, characteristics and then finally the overall quality of the component by using the appropriate metrics.

1.2 Analytical Network Process (ANP) Analytic Network Process (ANP) is a generalization of AHP by considering the dependence between the elements of the hierarchy [15]. Many decision making problems cannot be structured hierarchically. Therefore, ANP is represented by a network rather than a hierarchy. ANP is a comprehensive decision-making technique that captures the outcome of the dependence and feedback within and among clusters of elements [15]. ANP is a coupling of two parts: the first consists of a control hierarchy or network of criteria and sub-criteria that control the interactions while the second is a network of influences among the elements and clusters. Unlike a hierarchy, ANP uses a network without a need to specify levels. Some of the fundamental ideas in support of ANP are as follows [15]:

ANP is built on the widely used AHP. ANP allows for interdependency; therefore, ANP goes

beyond AHP. ANP deals with dependence within a set of elements (inner

dependence) and among different sets of elements (outer dependence).

In the loose network structure of the ANP, problems from any field are represented without concern of criteria since AHP can resolve hierarchically structured problems.

ANP is a non-linear structure that deals with sources, cycles and sinks having a hierarchy of linear forms, with goals in the top level and the alternatives in the bottom level.

ANP is a suitable technique to portray a real-world representation of the problem under consideration by prioritizing not only the elements but also the groups or clusters of elements.

ANP utilizes a control hierarchy or a control network to deal with different criteria and eventually provide an opportunity to analyze the benefits, costs and risks.

1.2.1 Applications of ANP ANP uses a network structure to represent the problem [9]. ANP was used to determine the personal selection factors and building the personal selection model to recruit the staff [5]. ANP was applied to identify the preferable bank loan quality for reducing non-performing loans (NPL) [1]. ANP has also been used by researchers to study the commodity market [6].

1.3 AHP Versus ANP AHP represents a framework with a unidirectional hierarchical relationship, ANP allows for more complex interrelationships among decision making levels and attributes. The ANP feedback approach replaces hierarchies with networks in which the relationships among the levels are not easily represented as higher or lower, dominated or being dominated, directly or indirectly. For example, the importance of the

criteria determines the importance of the alternatives, as it would in a hierarchy, but the importance of the alternatives may, in turn, have an impact on the importance of the criteria [15]. Therefore a hierarchical structure with a linear, top-to-bottom form is not applicable in a complex system. A system with feedback can be represented by a

Figure 1. (a) Structure of AHP (b) Structure of ANP

network where nodes correspond to the levels or components [15]. The structural difference between a hierarchy and a network is depicted in Figure 1. The elements in a node may influence some or all of the elements of any other node. In a network there can be source nodes, intermediate nodes and sink nodes. Relationships in a network are represented by arcs and the directions of these arcs signify dependence. Interdependency between two nodes, termed outer dependence, is represented by a two-way arrow and inner dependence between elements in a node is represented by a looped arc.

1.4 Fuzzy Set Theory Zadeh introduced the Fuzzy set theory to handle uncertainty due to imprecision and vagueness. Fuzzy set theory deals with linguistic variables of natural languages. Fuzzy set theory is capable of representing vague data and provides a decision making method in fuzzy environments [25].

If X is a collection of objects, then a fuzzy set A in X is defined as a set of ordered pairs A = {(x, µA(x)) | x ϵ X }, where, µA(x) is called the membership function for the fuzzy set A. The membership function maps each element of X to a membership grade (or a membership value) between 0 and 1.

The definition of a fuzzy set is a simple extension of the definition of a classical (crisp) set in which the characteristic function is permitted to have any values between 0 and 1. If the value of the membership function is restricted to either 0 or 1 then A is reduced to a crisp set.

1.5 Fuzzy AHP and ANP There are many fuzzy AHP/ANP methods proposed by various researchers [2, 3, 7, 12, 16]. The Fuzzy AHP/ANP approach to study the relationship between parameters of a particular problem provides an opportunity to handle vagueness in the dependence of the parameters. Theses approach gives a chance to researchers to deal with the uncertainty (fuzziness) of human decision-making. These techniques are widely used to solve problems in a fuzzy environment as follows:

The Fuzzy analytic network process (FANP) model was used to measure the sectoral competition level for an organization [3]. This study helps to do a SWOT analysis of an organization. The evaluation model based on FANP is used to determine evaluation criteria of material support plans [12].

On the basis of our literature survey, we conclude that empirical approaches like AHP, ANP and fuzzy ANP are widely used to solve the problems that have various dependent parameters, which are related to each other either in hierarchical or network structures. Such wide

(a) (b)



applications of these techniques motivate us to explore their applicability to analyze software metrics.

2. ADOPTED OUTLINES The paper outlines a step wise method to implement the conducted survey in order to ensure better understanding. The critical analysis of each relevant paper has been done by the following process: Step-1: Searching the online/offline database of existing literature using relevant keywords like AHP, ANP, Fuzzy ANP and software metrics. Step-2: Segregation and analysis of papers as per their relevance to the objective of the proposed study. Step-3: Study the abstract of the relevant papers, which were segregated in step-2. Step-4: Further segregation of papers was done by matching the abstract with the objective of the proposed study. Step-5: Generating an output document through the critical study of each paper which helped to formulate the introduction of this paper. Step-6: Critical analysis of various software metrics to understand the use and importance of the metrics. Step-7: In this step, we studied the possible relations between various metrics and also the relations between different attributes of the same metric. Step-8: Study of possibilities to apply AHP, ANP and Fuzzy ANP to study the relation between different metrics and between their attributes.

3. MOTIVATION OF USING EMPIRICAL APPROACHES In this section, a scenario of a software metric is used to explore the possibility of using empirical concepts for analysis.

Let us assume that M is a software metric as follows:

Where a1, a2, a3, a4, a5 and a6 are the attributes of metric M. We can easily analyze the relation between a1, a2, a3, a4, a5 and a6 by doing a survey

among some domain experts or work done by other researchers. The dependence of attribute and is denoted by → . In general set theory, dependence can be defined as follows: For , an attribute depends on attribute with certain degree of dependence. This relation is denoted by → , where d represents the degree of dependence. The four possible relationships among attributes of metric M are independency, hierarchical dependency, network dependency and fuzzy network dependency. These relationships are defined as follows: a) Independency: All attributes of M are independent 0). b) Hierarchical Dependency: Some attributes may be hierarchically dependent and remaining may be independent. Consider Figure 2, where a2, a3, a4 are dependent on a1 and a5 is dependent on a2, a3, and a4. Further three cases may arise as degree of dependence (d) may vary as follows:

Case 1: Where the degree of dependence is one ( 1 . It is the case of total dependence, i.e. , , are totally dependent on and is totally dependent on , , .

Case 2: Where the degree of dependence is partial 0 1 . It is the case of partial hierarchical dependence.

Case 3: Where the degree of dependence is linguistically defined. It is the case of fuzzy hierarchal dependence.

c) Network Dependency: Some or all attributes may be either self-dependent or may be dependent on two or more attributes. Consider Figure 3, where is an independent attribute. The network dependence of attributes can be represented by network dependency matrix M(d). d) Fuzzy Network Dependency: The degree of dependence can be a linguistic term, for instance poorly dependent, strongly dependent and

very strongly dependent.

Figure 2. Hierarchal relationship between a1, a2, a3, a4 and a5.

M(d) =

a1 a2 a3 a4 a5 a6 a1 0 0 0 0 0 0 a2 0 0 d4 0 0 0 a3 d1 0 0 0 0 0 a4 0 0 0 0 0 0 a5 d3 0 d2 0 d8 d5 a6 0 d6 d7 0 0 0

Figure 3. Network relationship between attributes

Consider Figure 4, in which degree of dependence is linguistically defined like poorly , strongly , and very strongly dependant. Attributes and are independent attributes. The fuzzy dependency matrix M(dF) for the above scenario may be written as follows:

M(dF) =

a1 a2 a3 a4 a5 a6 a1 0 0 0 0 0 0 a2 0 0 0 0 0 0 a3 pd 0 0 0 0 0 a4 0 0 0 0 0 0 a5 vsd 0 sd 0 0 pd a6 0 0 sd 0 0 0

4. ANALYSIS OF PROPOSED CONCEPT A metric is a quantitative measure of the degree to which a system, component or process possesses a given attribute. The motivations for using software metrics are as follows:

Estimates the cost & schedule of future projects Evaluates the productivity impacts of new tools and

techniques

d1

d2

d4 d5

d6

d8d7

a1

a2

a5

a3

a6

a4

d3

a1

a3

a5

a2 a4



Establishes productivity trends over time Improves software quality Forecasts future staffing needs Anticipates and reduces future maintenance needs

Figure 4. Linguistic dependency relationship network Here we use quality metrics for concept analysis. ISO (International Standard Organization) defines six main attributes for software product evaluation [7]. These attributes are shown with their sub characteristics in Table 1. The relationship between attributes of quality metrics can be established through the survey of some domain experts in software quality. Various possible scenarios of relationships may arise as follows:

Table1. Quality Metrics defined by ISO

(i) There may be hierarchical relationships between attributes of quality metrics. In this scenario, we may apply AHP to get the weights of different attributes in the metrics.

(ii) Hierarchical relationships between attributes of quality metric may be defined linguistically. In this scenario we may apply fuzzy AHP to get the weights of different attributes in the metrics.

(iii) An attribute may depend on two or more attributes with an absolute degree of dependence. In this scenario we may apply ANP to get the weights of different attributes in the metrics.

(iv) An attribute may depend on two or more attributes and degree of dependence may be defined linguistically. In this scenario we may apply fuzzy ANP to get the weights of different attributes in the metrics.

The same relationship scenario may arise between sub characteristics of an attribute or between sub characteristics of different attributes.

5. CONCLUSION Accurate and effective metric design is an important factor that affects the success and the realization of software development. The attributes of software metrics have different importance. A weighted list of attributes by their importance can be achieved by using empirical approaches like AHP, ANP and fuzzy AHP/ANP.

The objective of the present study is to find out the possible relationships between attributes of particular metrics. The dimensionality of a metric can be reduced by selecting the important attributes as per their weight values. The dimensionality reduction may lead towards cost and schedule optimization in the software development process.

This paper explores the use of empirical approaches to analyze software metrics. These techniques may be effective to find out the weights of different attributes as per their effectiveness in particular metrics.

6. REFERENCES [1] Yi-Shan Chen, Chin-Tsai Lin and Jung-Ho Lu, “The analytic

network process for the banking sector: An approach to evaluate the creditability of emerging industries”, African Journal of Business Management, vol. 5, pp: 1343-1352, 2011.

[2] Arijit De , “A Fuzzy Ordered Weighted Average (OWA) Approach to Result Merging for Metasearch using the Analytical Network Process”, Second International Conference on Emerging Applications of Information Technology, Kolkata, pp: 17-20, 2011.

[3] M. Dagdeviren, İhsan Yüksel et.al, “A fuzzy analytical network process (ANP) model for measurement of the sectoral competition level(SCL)”, Expert system with application, vol. 37, pp: 1005-‘ 1014, 2010.

[4] Alen Jakupovic, Mile Pavlic, Sanja Candrlic, “Application of Analytic Hierarchy Process (AHP) to Measure the Complexity of the Business Sector and Business Software”, in Conference on Computer Science and Software Engineering, New York, pp: 35-42, 2010.

[5] Hung-Tso Lin, “Personnel selection using analytic network process and fuzzy data envelopment analysis approaches”, Computers and Industrial Engineering, voll.59, pp: 937-944, 2010.

[6] Erika Neira, Diana Lesmes, “Analytic Network Process (ANP): An Approach to Estimate the Colombian Baby Diapers Market Share”, in proceedings of the international symposium on the analytical hierarchical process/network process multi criteria Decision Making, Pittsburgh, Pennsylvania, 2009.

[7] Cakir, O., Canbolat, M.S,“A web-based decision support system for multi criteria inventory classification using fuzzy AHP methodology ”, Expert system with Applications, vol.35, pp: 1367-1378, 2008

[8] Arun Sharma, Rajesh Kumar, P S Grover, “Estimation of quality for software components: an empirical approach”, ACM SIGSOFT Software Engineering Notes, Volume 33 Issue 6, November 2008

[9] Patrizia L. Lombardi, Isabella M. Lami, Marta Bottero, Cinzia Grassl, “Application of the Analytical Network Process and the Multi-model frame work to and urban upgrading case study”, in International Conference on Whole Life Urban Sustainability and its Assessment, Glasgow, 2007.

[10] Saaty T.L and Vargas L.G, Decision Making with the Analytical Network Process, New York: Springer Science, 2006

[11] Kim, Dong Jun, CHUNG, Sung Bong, SONG, Ki Han, HONG, Sang Yeon, “Development Of An Assessment Model Using AHP Technique For Railroad Projects Experiencing Severe Conflicts In Korea”, in Proceedings of the Eastern Asia Society for Transportation Studies, vol. 5, pp. 2260 - 2274, 2005.

[12] Lili Qu, Rui Kang, Jun Long, “A Fuzzy ANP Model for Evaluating Material Support Plan in Development Phase”, in Proceedings of the Eastern Asia Society for Transportation Studies, vol. 5, pp: 2260 - 2274, 2005



[13] Saaty T.L, “Decision Making-the analytical hierarchical and network process (AHP/ANP),” Journal of Systems Science and Systems Engineering, vol. 13, pp: 1-35, 2004

[14] Ernest H. Forman, Saul I. Gass, “The Analytic Hierarchy Process – An Exposition”, Operations Research, vol. 49, pp: 469-486, 2001.

[15] Saaty T.L. Decision Making with Dependence and Feedback: The Analytical Network Process. Pittsburgh: RWS Publications, 1996.

[16] Chag D.Y.,“Application of extent analysis method on fuzzy AHP”, European Journal of Operational Research, vol.95,pp: 649-655, 1996

[17] Margaret F. Merritt, “Ranking Selected Streams in Interior Alaska on the Basis of Suitability for Sustaining an Introduced Rainbow Trout Population”, in Times International Conference, Anchorage, Alaska. 1994.

[18] http://www.dtic.mil/dtic/tr/fulltext/u2/a266875.pdf, John H. Heitz and Miles C. Miller, 1993.

[19] Robert J. Moreland and Jerry B. Sanders, “Lunar Lander and Return Propulsion System Trade Study: Methodology with

Results”, in 29th Joint Propulsion Conference, Monterey, California, 1993.

[20] James G. Dolan, Donald R. Bordley, Heidi Miller, "Diagnostic Strategies in the Management of Acute Upper Gastrointestinal Bleeding: Patient and Physician Preferences", Journal of General Internal Medicine, vol. 8, pp: 525-529, 1993

[21] Gavin R. Finnie, Gerhard E. Wittig, Doncho I. Petkov, "Prioritizing Software Development Productivity Factors using the Analytic Hierarchy Process”, Journal of Systems and Software, vol. 22, pp: 129-139,1993.

[22] Eyrich, H.G., "Benchmarking to Become the Best of Breed," Manufacturing Systems magazine, April 1991.

[23] Forman E. H., “AHP is Intended for More than Expected Value Calculations", Decision Sciences, vol. 21, pp: 670-672, 1990.

[24] Saaty T.L. The Analytical Hierarchical Process. New York: McGraw-Hill, 1980.

[25] L. A. Zadeh, “The concept of a linguistic variable and its application to approximate reasoning”, Information Science, vol. 8, pp: 199-249, 1975.



study of empirical approaches to analyze the software metrics

Documents