evaluation of a business application framework using complexity and functionality metrics

1

Software Engineering Research Group, Graduate School of Engineering Science, Osaka University

Evaluation of a Business Application Framework Using Complexity and

Functionality Metrics

Hikaru Fujiwara†, Shinji Kusumoto†, Katsuro Inoue†, Toshifusa Ootsubo‡, Katsuhiko Yuura‡

†Graduate School of Engineering Science,

Osaka University, Japan. ‡Business Solution Systems Division, Hitachi Ltd.

This research is partially supported

by International Information Science Foundation

2


1. Background

2. Approach 2.1 Purpose

2.2 Reuse

3. Evaluation Method

4. Case Study 4.1 Case Study 1

4.2 Case Study 2

5. Conclusion

Contents

1. Background

3


1. Background (1/4)

• It becomes important to develop high-quality software cost-effectively.

• Reuse is one of the most famous techniques to attain it.• In object-oriented software development, developers

reuse a particular library called framework.• A framework is a collection of classes that provide a set

of services for a particular domain.

4


1. Background (2/4)

• A department of Hitachi Ltd. develops application software for many local governments.

• The developers have been using the original reuse technique based on a conventional module-based reuse.

5


1. Background (3/4)

• 47 prefectures in Japan.

• 3229 local governments, 671 cities, 1991 towns, 567 villages.

• A department of Hitachi Ltd. develops application software for many local governments.

• The requirement of the application is different depending on local ordinance.

6


1. Background (4/4)

• The department is going to introduce the framework in order to improve the efficiency.

• It is difficult to transfer the new framework to the development.

• To motivate the developers, we need to show the benefit of using framework quantitatively.

7


1. Background


2.2 Reuse



4.2 Case Study 2

5. Conclusion

Contents

1. Background

8


2. Approach2.1 Purpose

• We evaluate the usefulness of the framework quantitatively from two viewpoints.– saving cost– improving software quality

• We conducted two case studies to evaluate it.– Application : Applications for local governments– Language : Java

9


2.2 ReuseConventional module-based reuse (1/2)

• An application consists of a main module and some screen control modules.

• The main module controls business logics and screen transitions of the application.

• Each screen control module corresponds to one screen.

A ： Database renewal program

X1 ： Data InquiryY1 ： Showing the

Inquiry ResultZ1 ： Updating the

Record

(a) Health insurance application for a local government A

Screen transition controller

10


2.2 ReuseConventional module-based reuse (2/2)

A ： Database renewal program



Record

(a) Health insurance application for a local government A




Record

(b) Health insurance application for a local government B

B ： Database renewal program


11


2.2 ReuseFramework-based reuse

• The framework is intended to reuse the processing of the transition of the screens in addition to module-based reuse. – Typical transitions of screens are prepared.

• data inquiry, data renewal, data addition, data removal, etc.

Database Renewal Program A

Database renewal Framework

Data InquiryShowing the

Inquiry ResultUpdating the

Record

F1 ：　 Framework

Specific Parameters to a Local

Government A

Specific Parameters to a Local

Government B

Database Renewal Program B

12


1. Background


2.2 Reuse



4.2 Case Study 2

5. Conclusion

Contents

1. Background

13



• We compare the framework-based reuse with the conventional module-based reuse from the viewpoints of cost and quality.(Case Study 1) Developing applications that have the same

functions.

Specification of function fa

framework-based

reuse

conventional

reuse

　

Ca

FW

Pa

FW ： framework

14



(Case Study 2) Adding functions to the existing application.


　

Ca

FW

Pa

framework-based

reuse

conventional

reuse

Specification of function fb

　

Ca+b

FW

Pa+b

Specification of function fc

　

Ca+b+c

FW

Pa+b+c

15


3. Evaluation Method Metrics used in the case studies

• We could not collect the actual effort and the number of faults.

• We used following metrics to indirectly evaluate the productivity and quality.– OOFP (Object-Oriented Function Point) measures

functionality.

productivity.– C&K metrics (Chidamber and Kemerer’s metrics) measure

complexity.

quality.

16


3. Evaluation Method OOFP （ Object Oriented Function Points§)

• OOFP is an adaptation of FP(Function Point) to enable the measurement of object-oriented analysis and design specifications.– FP is measured from logical files(Internal Logical

Files:ILFs, External Interface Files:EIFs) and transactions (inputs, outputs, inquiries).

– OOFP is measured from logical files (ILF, EIF) and transactions (Service Requests:SRs).

• Classes correspond to logical files• Methods correspond to transactions

§:G.Caldiera, G.Antoniol, R.Fiutem, C.Lokan, “Definition and Experimental Evaluation of Function Points for Object-Oriented Systems”, IEEE, Proc. of METRICS98, pp.167-178 (1998).

17


3. Evaluation Method C&K metrics (Chidamber and Kemerer’s metrics)

viewpoints metrics how to calculate metrics of a class C

inheritance DIT the depth of C in the inheritance tree

NOC the number of immediate sub-classes subordinated to C

coupling CBO the number of couplings between C and any other class

RFC (the number of the methods in C) ＋ (the number of the methods called by C)

method WMC the sum of the complexity of the methods in C

LCOM Assume that methods M1...Mn ∈ C,

Ii is a set of instance variables used in Mi,

(the number of pairs (Mk, Ml) such that Ik∩Il ＝ φ)

－ (the number of pairs (Mk, Ml) such that Ik∩Il≠φ)

18


1. Background


2.2 Reuse



4.2 Case Study 2

5. Conclusion

Contents

1. Background

19


4. Case Study4.1 Case Study 1

• Four applications Ca, Cb, Cc, Cd are developed using the framework-based reuse.

• Four applications Pa, Pb, Pc, Pd are developed using the conventional reuse.

• Ci and Pi (i = a,b,c,d) implement the same function fi.

framework-based conventional

function

fa Ca Pa

fb Cb Pb

fc Cc Pc

fd Cd Pd

20


4.1 Case Study 1

• We compared Ci with Pi from the viewpoints of productivity and quality.

• We measured OOFP and C&K metrics from newly developed part of each application.

FW ： framework : newly developed


　

Ca

FW

Pa

framework-based

reuse

conventional

reuse

FW ： framework

21


4.1 Case Study 1Result of case study 1

number of classes

OOFP CBO RFC WMC LCOM

Ca

Cb

Cc

Cd

5

5

11

8

176

180

418

252

3.8

5.8

5.4

4.1

14.4

18.2

33.1

17.8

7.4

7.6

8.1

6.4

21.4

23.2

28.7

13.8

Pa

Pb

Pc

Pd

25

25

29

28

526

526

671

672

2.1

2.3

3.0

2.4

5.8

5.9

7.4

8.6

3.3

3.3

3.4

4.3

2.1

2.1

2.0

15.7

C&K metrics are the average values per class

22


4.1 Case Study 1Analysis of OOFP

framework-based (C)

conventional (P)

function

fa 176 526

fb 180 526

fc 418 671

fd 252 672

OOFP(Ci) < OOFP(Pi)

Application development using framework reduces the effort of development.

23



• To develop the framework, initial investment (effort) is needed.

• The OOFP of the framework FW is 1298.• The framework-based reuse is about 2.5 times more

effective than the conventional reuse.• The department develops the similar applications

repeatedly.• It will save the effort after three or four applications have

been developed, whereas the investment for the framework was spent.

24


4.1 Case Study 1Analysis of CBO and RFC

average CBO average RFC

framework-

based (C)

conventional

(P)

framework-

based (C)

conventional

(P)

function

fa 3.8 2.1 14.4 5.8

fb 5.8 2.3 18.2 5.9

fc 5.4 3.0 33.1 7.4

fd 4.1 2.4 17.8 8.6CBO(Ci) > CBO(Pi), RFC(Ci) > RFC(Pi)

All of the methods called by Ci are included in the framework. If the framework is high quality, the complexity does not affect the quality of the overall application program.

25


4.1 Case Study 1Analysis of WMC and LCOM

average WMC average LCOM

framework-

based (C)

conventional

(P)

framework-

based (C)

conventional

(P)

function

fa 7.4 3.3 21.4 2.1

fb 7.6 3.3 23.2 2.1

fc 8.1 3.4 28.7 2.0

fd 6.4 4.3 13.8 15.7WMC(Ci) > WMC(Pi), LCOM(Ci) > LCOM(Pi)

There are many simple methods, that set/get the values of the attribute.

The complexity does not affect the quality of the overall application program.

26


1. Background


2.2 Reuse



4.2 Case Study 2

5. Conclusion

Contents

1. Background

27


4.2 Case Study 2• At first, application Ca and Pa are developed.

• Function fb, fc, fd are continuously added to Ca and Pa.

• Ci and Pi (i = a,a+b,a+b+c,a+b+c+d) implement the same function fi.

framework-based

conventional

function

fa Ca Pa

fa+b Ca+b Pa+b

fa+b+c Ca+b+c Pa+b+c

fa+b+c+d Ca+b+c+d Pa+b+c+d

28


4.2 Case Study 2• We compared the differences between the two successive

applications from the viewpoint of productivity and quality.• We measured OOFP and C&K metrics from newly developed

part of each application.


　

Ca

FW

Pa

framework-based

reuse

conventional

reuse

Specification of function fb

　

Ca+b

FW

Pa+b

Specification of function fc

　

Ca+b+c

FW

Pa+b+c

29


4.2 Case Study 2Result of case study 2

number of classes

OOFP CBO RFC WMC LCOM

Ca

Ca+b

Ca+b+c

Ca+b+c+d

5

7

15

20

176

251

578

743

3.8

5.0

5.9

5.8

14.4

16.7

30.1

28.3

7.4

7.6

8.3

7.9

21.4

20.1

28.6

25.6

Pa

Pa+b

Pa+b+c

Pa+b+c+d

25

29

39

46

526

615

849

1084

2.1

2.8

3.7

4.0

5.8

6.9

8.6

10.5

3.3

3.3

3.3

3.9

2.1

2.0

1.8

10.0

C&K metrics are the average values per class

30



framework-based (C)

conventional (P)

function

fa 176 526

fa+b 251 615

fa+b+c 578 849

fa+b+c+d 743 1084

75

327

165

89

234

235

OOFP(Ca+b+c - Ca+b) > OOFP(Pa+b+c - Pa+b)

The adaptability of the framework to the function fc is not good. It is necessary to add new components for the functions like fc to the framework.

31



OOFP(Ca+b+c+d) < OOFP(Pa+b+c+d)

framework-based (C)

conventional (P)

function

fa 176 526

fa+b 251 615

fa+b+c 578 849

fa+b+c+d 743 1084

The framework-based reuse is more effective to reduce the effort of development than the conventional reuse.

32


4.2 Case Study 2Analysis of CBO and RFC

average of CBO average of RFC

framework-based (C)

conventional (P)

framework-based (C)

conventional (P)

function

fa 3.8 2.1 14.4 5.8

fa+b 5.0 2.8 16.7 6.9

fa+b+c 5.9 3.7 30.1 8.6

fa+b+c+d 5.8 4.0 28.3 10.5

1.2

0.9

-0.1

0.9

0.9

0.3

2.3

13.4

-1.8

1.1

1.7

1.9

RFC(Ca+b+c - Ca+b) > RFC(Pa+b+c - Pa+b)

Ca+b+c calls a lot of methods included in the framework, in order to handle many data items in the function fc. If the framework is high quality, the complexity does not affect the overall application program.

33


1. Background


2.2 Reuse



4.2 Case Study 2

5. Conclusion

Contents

1. Background

34


5. Conclusions• We have experimentally evaluated the usefulness of the

framework quantitatively from the viewpoints of quality and saving cost.

• As the result of the case studies, the framework-based reuse is more effective than the conventional reuse.

• In order to show the usefulness of the framework, we are going to apply the framework to many software development projects in future.

35


36


Appendix How to measure OOFP(1)

OOFP=OOFPILF+OOFPEIF+OOFPSR

• For each ILF, OOFPILF is measured using DETs (Data Element Types) and RETs (Record Element Types).

• For each EIF, OOFPEIF is measured using DETs and RETs.

• For each SR, OOFPSR is measured using DETs and FTRs (File Types Referenced).

• Finally, OOFP is calculated by summing up OOFPILF, OOFPEIF and OOFPSR.

37


AppendixHow to measure OOFP (2)

• To calculate OOFP from Java source code, we suppose that the concepts in Java correspond to ones in OOFP as follows.

OOFP Java

Logical files ILFs Classes within the application.

EIFs Classes outside of the application.

DETs Number of simple attributes (such as integers, strings)

RETs Number of complex attributes (such as objects)

Transactions SRs Methods within the application.

DETs Number of simple arguments, instance variables and class variables.

FTRs Number of complex arguments, instance variables, class variables and objects.

38


AppendixCase Study 2; Analysis of WMC and LCOM (1)

average of WMC average of LCOM

framework-based (C)

conventional (P)

framework-based (C)

conventional (P)

function

fa 7.4 3.3 21.4 2.1

fa+b 7.6 3.3 20.1 2.0

fa+b+c 8.3 3.3 28.6 1.8

fa+b+c+d 7.9 3.9 25.6 10.0

0.2

0.7

-0.4

0.0

0.0

0.6

-1.3

8.5

-3.0

-0.1

-0.2

8.2

LCOM(Ca+b+c - Ca+b) > LCOM(Pa+b+c - Pa+b)There are a lot of set/get methods in Ca+b+c , in order to handle many data items in the function fc.

39




framework-based (C)

conventional (P)

framework-based (C)

conventional (P)

function

fa 7.4 3.3 21.4 2.1

fa+b 7.6 3.3 20.1 2.0

fa+b+c 8.3 3.3 28.6 1.8

fa+b+c+d 7.9 3.9 25.6 10.0

0.2

0.7

-0.4

0.0

0.0

0.6

-1.3

8.5

-3.0

-0.1

-0.2

8.2

LCOM(Ca+b+c+d - Ca+b+c) < LCOM(Pa+b+c+d - Pa+b+c)There are complex transitions of the screens in function fd. In Pa+b+c+d , the LCOM values of the classes implement these transactions are high.

40




framework-based (C)

conventional (P)

framework-based (C)

conventional (P)

function

fa 7.4 3.3 21.4 2.1

fa+b 7.6 3.3 20.1 2.0

fa+b+c 8.3 3.3 28.6 1.8

fa+b+c+d 7.9 3.9 25.6 10.0

WMC(Ci) > WMC(Pi), LCOM(Ci) > LCOM(Pi)

We examined the classes of Ci that have high WMC and LCOM values. It is found that there are many simple methods, that set/get the values of the attribute.

41


AppendixProcess of Case Study

• Development of the framework• Case Study1

– Two developers

– Develop Ca, Cb, Cc, Cd

– Develop Pa, Pb, Pc, Pd

• Case Study2– Two developers

– Develop Ca, Ca+b, Ca+b+c, Ca+b+c+d

– Develop Pa, Pa+b, Pa+b+c, Pa+b+c+d

• Apply the metrics to the applications.• Analysis

evaluation of a business application framework using complexity and functionality metrics

Documents