evaluating uml using a generic quality framework

7/31/2019 Evaluating UML Using a Generic Quality Framework

1/6

1459

E

Category: Software & Systems Design

Evaluating UML Using a Generic QualityFramework

John Krogstie IDI, NTNU, SINTEF, Norway

Copyright 2009, IGI Global, distributing in print or electronic forms without written permission of IGI Global is prohibited.

IntroductIon

According to Booch, Rumbaugh, and Jacobson (2005),developing a model for an industrial strength softwaresystem before its construction is increasingly regarded asa necessary activity in information systems development.The use of object-oriented modeling in analysis and designstarted to become popular in the late 80s, producing a largenumber of different languages and approaches. Over the last10 years, UML (OMG, 2006a) has taken a leading positionin this area.

In this chapter, we give an overview assessment of UMLusing a generic evaluation framework. We will rst presentthe evaluation framework. We will then evaluate the languagequality of UML before pointing to the future direction and

potential of UML.

Background

Earlier, we developed a framework for understanding andassessing quality of models and modeling languages (Krogstie& Slvberg, 2003; Krogstie, Sindre, & Jrgensen, 2006).

The main concepts of the framework and their relation-ships are shown in Figure 1 and are explained next. Qualityhas been de ned referring to the correspondence betweenstatements belonging to the following sets:

G, the goals of the modeling task L, the language extension (i.e., the set of all statements

that are possible to make according to the graphemes,vocabulary, and syntax of the modeling languages

used) D, the domain (i.e., the set of all statements that can

be stated about the situation at hand) M, the externalized model itself Ks, the relevant explicit knowledge of those being

involved in modeling. A subset of these is activelyinvolved in modeling, and their explicit knowledgeis indicated by K M.

I, the social actor interpretation (i.e., the set of allstatements that the audience thinks that an externalizedmodel consists of)

T, the technical actor interpretation (i.e., the statementsin the model as interpreted by modeling tools)

Figure 1. Framework for discussing the quality of models

Empiricalquality

Socialquality

Physicalquality

Socialpragmaticquality

Semanticquality

Syntacticquality

Technicalpragmatic

quality

Modelexternalization

M

Socialactor

interpretationI

Technicalactor

interpretationT

Modelingdomain

D

Languageextension

L

Modellerexplicit

knowledgeK M

Perceivedsemantic

quality

Social actorexplicit

knowledgeK S

Goals ofmodelling

G

Organizationalquality


2/6

1460

Evaluating UML Using a Generic Quality Framework

The main quality types are indicated by solid lines betweenthe sets and are described brie y next:

Physical quality: The basic quality goals on thephysical level is that the knowledge K of the domainD has been externalized, and internalizeability, thatthe externalized model M is available.

Empirical quality deals with predictable error frequen-cies when a model is read or written by different users,coding (e.g., shapes of boxes) and HCI-ergonomicsfor documentation and modeling-tools. For instance,graph layout to avoid crossing lines in a model is amean to address the empirical quality of a model.

Syntactic quality is the correspondence between themodel M and the language extension L.

Semantic quality is the correspondence between themodel M and the domain D. This includes validity andcompleteness.

Perceived semantic quality is the similar correspon-dence between the audience interpretation I of a modelM and his or hers current knowledge K of the domainD.

Pragmatic quality is the correspondence betweenthe model M and the audiences interpretation andapplication of it (I). We differentiate between socialpragmatic quality (to what extent people understandand are able to use the models) and technical pragmaticquality (to what extent tools can be made that interpretthe models).

The goal de ned for social quality is agreement amongaudience members interpretations I.

The organizational quality of the model relates to thatall statements in the model contribute to ful lling the goalsof modeling (organizational goal validity), and that all thegoals of modeling are addressed through the model (orga-nizational goal completeness).

Language quality relates the modeling language usedto the other sets. Six quality areas for language quality areidenti ed with aspects related to both the language meta-model and the notation as illustrated in Figure 2.

Domain appropriateness: This relates the languageand the domain. Ideally, the conceptual basis must bepowerful enough to express anything in the domain,not having what Wand and Weber (1993) term con -struct de cit. On the other hand, you should not beable to express things that are not in the domain (i.e.,what is termed construct excess) (Wand et al., 1993).Domain appropriateness is primarily a mean to achievephysical quality, and through this, to achieve semanticquality.

Participant language knowledge appropriateness relates the social actors explicit knowledge to thelanguage. Participant language knowledge appropri-ateness is primarily a mean to achieve physical andpragmatic quality.

Knowledge externalizability appropriateness: Thisarea relates the language extension to the participantknowledge. The goal is that there are no statementsin the explicit knowledge of the modeler that cannotbe expressed in the language. Knowledge externaliz-ability appropriateness is primarily a mean to achievephysical quality.

Figure 2. Language quality in the quality framework


3/6

1461


E Comprehensibility appropriateness relates the

language to the social actor interpretation. The goalis that the participants in the modeling effort usingthe language understand all the possible statementsof the language. Comprehensibility appropriateness isprimarily a mean to achieve empirical and pragmaticquality.

Technical actor interpretation appropriateness relates the language to the technical audience interpreta-tions. For tool interpretation, it is especially importantthat the language lend itself to automatic reasoning.This requires formality (i.e., both formal syntax andsemantics being operational and/or logical), but for -mality is not necessarily enough, since the reasoningmust also be ef cient to be of practical use. This iscovered by what we term analyzability (to exploit anymathematical semantics) and executability (to exploitany operational semantics). Different aspects of tech -nical actor interpretation appropriateness are a meanto achieve syntactic, semantic, and pragmatic quality(through formal syntax, mathematical semantics, andoperational semantics).

Organizational appropriateness relates the languageto standards and other organizational needs within theorganizational context of modeling. These are meansto support organizational quality.

Whereas knowledge externalizability appropriateness andorganizational appropriateness is closely linked to particularorganizations and modelers involved, for a more generalevaluation, we can use the other four categories.

oVerVIeW oF eValuatIon

The evaluation is structured according to the four areas of language quality previously presented. Before presentingthe evaluation, we will position UML in relation to the setsof the quality framework.

Domain: According to OMG (2006a), UML is alanguage for specifying, visualizing, constructing,and documenting the artifacts of software systems, aswell as for business modeling and other non-softwaresystems. In other words, UML is meant to be used inanalysis of business and information, requirementspeci cation, and design. UML is meant to supportthe modeling of (object-oriented) transaction systems,real-time, and safety critical systems. As illustrated inFavre (2003), UML is used and adapted for a number of different speci c areas.

Language: We have based the evaluation on UML(version 2.0, (Booch et al., 2005, OMG, 2006a)).

The sets Knowledge, Model, and Interpretationmust be judged from case to case in the practical applicationof a modeling language and tools. Also when it comes toweighting the different criteria against each other, this must

be done in the light of the speci c modeling task, such as has been done by (e.g., Krogstie & Arnesen, 2004; Nysetvold& Krogstie, 2006).

Due to the limitation on the length of a paper of this kindand the breadth of the evaluation, we will only have room for

presenting some of the major results. See Krogstie (2003)for a more detailed description of using the framework forevaluating UML.

d m i app p i ss

Looking brie y on the coverage of the seven main model -ing-perspectives in information systems modeling (Krogstieet al., 2003), we nd:

UML has a very good support for modeling accord -ing to an object-oriented perspective, especially fordesign.

The structural perspective is also well supported, al-though not as well as in languages made speci callyfor this purpose.

The behavioral perspective is supported particularlythrough statecharts.

The functional (process) perspective is supported on ahigh level through use case modeling, a language thathas been highly criticized for not being well-de ned(Hitz & Kappel, 1998). Whereas use-cases are meantfor requirements modeling, activity diagrams can beused for simple procedural description by showingcontrol ow and the production of data or objects ina process ow. Changes to the activity diagrams areintroduced in UML2.0 to improve the modeling of

business processes. The lack of traditional data ow inactivity diagrams has been noted as a problem. Notethat another upcoming standard in this area, BPMN(BPMN 2006) has been taken over by OMG, althoughthere are no immediate plans for including BPMN inUML.

The actor-role perspective is partly covered usingthe collaboration diagrams. Using roles in sequencediagrams or swimlanes in activity diagrams, we alsoget a role-oriented view, but there is no intuitive wayto represent organizational and group-structures andrelationships in UML.

There are fundamental problems with expressingcertain constraints in any OO modeling framework (Hydalsvik & Sindre, 1993). Temporal and deonticconstraints are hard to express. Whereas these kindof concepts are not included in UML, a new initiativewithin OMG, SBVR (OMG, 2006b), includes such


4/6

1462


things as deontic operators. For the moment, thereare no plans for including this in UML. There is nosupport for modeling of goal-hierarchies (Mylopoulos,Chung, & Tu, 1999).

The language-action perspective, which is most usefulfor the analysis of businesses, is not supported.

A meta-model of UML is de ned (using UML), and thereexist extension mechanisms to make the language more ap-

plicable in speci c domains, although when creating such pro les, it is not possible to describe the semantics of theextensions in a formal manner.

Most of UML is rst useful during design. These partsof the language should not be used in analysis and require-ments speci cation, even in areas where the transition fromanalysis to design is seamless. (There is much evidence,especially for business systems, that this transition is farfrom seamless even when using object-oriented modeling inall domains (Davis, 1995; Hydalsvik et al., 1993). Proper guidelines for avoiding this are not consistently provided,and there is limited support for avoiding using analysis anddesign concepts in the same model.

There are also mismatches between underlying basis andexternal representation. In sequence diagrams, for instancea number of concepts are semantically vacant (Morris &Spanoudakis, 2001). Some of these problems are addressedin UML 2.0. We also nd examples of concrete constructs inthe UML meta-model, with no representation in the notation(e.g., namespace and model).

p r i i n l ngu ge know edgea ro ri eness

It can be argued that for those being familiar with the mainOO-modeling concepts and main OO modeling-languages,the core of UML should not represent a too steep learningcurve. Almost all CS and IS-degrees now include courseswhere UML is lectured and used. On the other hand, wehave noted the complexity of the language above. The largenumber of constructs in UML is partly due to its diversediagramming techniques (Siau & Cao, 2001). Constructs inuse case diagrams are very different from constructs in classdiagrams. Class diagrams are also very different from activitydiagrams or Statecharts. This diversity causes problems. First,there are more constructs to learn. Second, the knowledge andexperience gained in using one diagramming technique can-not easily be transferred to another diagramming techniquein UML. The de nition of language units and incrementswithin units introduced in UML 2.0 will hopefully help inpractical application of the language.

p r i i n com rehensibi i ya ro ri eness

UML can be argued to be overly complex with 233 differ -

ent concepts (in UML 1.4) (Castellani, 1999). A similar number of concepts can be found in UML 2.0. In Siau etal. (2001) a detailed comparison between UML and other modeling approaches are presented. Although UML hasmore diagramming techniques when compared to object-oriented methods such as OMT, OOAD, and Shlaer/Mellor,each of the diagramming techniques in isolation is no morecomplex than techniques found in these methods. On theoverall method level on the other hand, UML stands outnoticeably, being most complex according to most of themetrics. On the other hand, Erickson and Siau (2004) pointout that the actual usage of a large number of the conceptswithin UML is limited.

With so many concepts, it is not surprising that someredundancy and overlap is witnessed.Examples of lacking symbol differentiation is found,

for example, that both classes and objects are shown usingrectangles. On the other hand, since the text-style of the objectgives an indication of this, this is a minor problem.

UML contains many possibilities of adding (often small)adornments to the models. Such small adornments are oftendif cult to see and comprehend.

A uniform use of symbols is not adhered to. An exampleis that different symbols are used for a role if it is externalto the system (pin-man) or internal (rectangle).

te hni a or In er re iona ro ri eness

The UML-syntax is rigorously de ned and the language isdescribed through a meta-model made in the structural modelof UML with accompanying descriptions of the semantics.Using UML to model UML means that some of the de ni -tions are circular, and this leaves UML (formally speaking)unde ned. This would be unproblematic if most practitionersalready understood the meaning of the core concepts (classes,inheritance, and associations) that are involved. In UML 2.0a formal (operational) action language have been includedto support a wider repertoire of modeling techniques. Only

some parts of UML have been provided a formal (operational)semantics. So-called semantic variation points are includedin UML 2.0 to clearly indicate where the semantics is notrigorously de ned. In any case, the UML meta-model onlydescribe the structural aspects of the language, and not thebehavioral aspects.


5/6

1463


EFuture trends

Modeling as a general technique within information sys -tems development has received increasing interest over the

last years, and will continue to be of high importance, forexample, in connection to the OMG MDA approach and business process support. Due to its strong support, UMLis probably the best general modeling language to adoptas a basis for object-oriented development if one is notalready using another language with good tool support thatone is satis ed with. Most of the accidental problems suchas inconsistencies in the language-descriptions found inearlier version of UML seem to be addressed in UML 2.0,

but there are still concerns. UML 3.0 is already planned, butit is dif cult to judge what this will look like. Judging fromthe time it has taken to nalize UML 2.0, we expect to haveto relate to this for a number of years.

conclusIon

UML has been developed and re ned over the last 10 years,but there are still some major weaknesses with the approach.Even if it has not been possible to agree on a standard process,outline process guidelines need to be included--even if thebest that can be done is to describe a number of alternatives.Particularly problematic is the logical/physical confusion inthe UML-de nition. As discussed by Davis (1995), thereare fundamental differences between the models related toanalysis, design, and requirement speci cation. What our

investigation has also illustrated is that although there is aperceived need to extend the expressiveness and formality of the language, the language has several weaknesses regardingcomprehensibility appropriateness, and is already lookedupon as dif cult to comprehend. The distinction betweeninfrastructure and superstructure provided for UML 2.0tries to address this. Looking at the accepted 1000-plus-page

proposal for UML2.0 superstructure alone does not give usmuch hope that the whole of UML will be particularly mucheasier to learn and use, as pointed out also in France, Ghosh,Dinh-Trong, and Solberg (2006).

reFerences

Booch , G., Rumbaugh, J., & Jacobson, I. (2005). The uni ed modeling language: User guide (2nd ed.). Addison-Wesley.

BPMN. (2006). Business process modeling notation . Re-trieved from http://www.bpmn.org

Castellani, X. (1999). Overview of models de ned withcharts of concepts. In E. Falkenberg, K. Lyytinen, & A.Verrijn-Stuart (Eds.), Proceedings of the IFIP8.1 Working

Conference On Information Systems Concepts (ISCO4) (pp.235-256); An Integrated Discipline Emerging September 20-22 , Leiden, The Netherlands.

Davis, A. (1995). Object-oriented requirements to object-

oriented design: An easy transition? Journal of Systems and Software , 30(1/2), 151-159.

Erickson, J., & Siau, K. (2004). Theoretical and practicalcomplexity of uni ed modeling language: Delphi study andmetrics analyses. In Proceedings of International Conferenceon Information Systems (ICIS) (pp. 183-194).

Favre, L. (2003). UML and the uni ed process . IRMPress.

France, R. B., Ghosh, S., Dinh-Trong, T., & Solberg, A.(2006). Model-driven development using UML2.0: Promisesand pitfalls. IEEE Computer , February, 59-66.

Hitz, M., & Kappel, G. (1998). Developing with UML--Some pitfalls and workarounds. In J. Bzivin, & P. A. Muller (Eds.), UML98Beyond the notation (pp. 9-20) . Mulhouse,France: Springer-Verlag.

Hydalsvik, G. M., & Sindre, G. (1993, September ). On the purpose of object-oriented analysis. In A. Paepcke (Ed.), Proceedings of the Conference on Object-Oriented Program -ming Systems, Languages, and Applications (OOPSLA93) (pp. 240-255). ACM Press.

Krogstie, J. (2003). Evaluating UML using a generic qualityframework. In L. Favre (Ed.), UML and the uni ed process (pp. 1-22). IRM Press.

Krogstie, J., & Arnesen, S. (2004). Assessing enterprisemodeling languages using a generic quality framework.In J. Krogstie, K. Siau, & T. Halpin (Eds.), Informationmodeling methods and methodologies. Hershey, PA: IdeaGroup Publishing.

Krogstie, J., Sindre, G., & Jrgensen, H. (2006). Processmodels representing knowledge for action: A revised qual-ity framework. European Journal of Information Systems ,15, 91-102.

Nysetvold, A. G., & Krogstie, J. (2006). Assessing busi -ness process modeling languages using a generic qualityframework. In K. Siau (Ed.), Advanced topics in databaseresearch. Hershey, PA: Idea Group Publishing.

Morris, S., & Spanoudakis, G. (2001). UML: An evalua -tion of the visual syntax of the language. In Proceedingsof HICSS 34 .

Mylopoulos, J., Chung, L., & Tu, E. (1999). From object-oriented to goal-oriented requirements analysis. Communi-cations of the ACM , 42 (1), 31-37.


6/6

1464


OMG. (2006a). Uni ed modeling language v 2.0 . Retrievedfrom http://www.omg.org

OMG. (2006b). Semantics of business vocabulary and rulesinterim speci cation . Retrieved June 3, 2002, from http://

www.omg.org/cgi-bin/doc?dtc/

Siau, K., & Cao, Q. (2001). Uni ed modeling language(UML)A complexity analysis. Journal of Database Man-agement, 26-34, Jan-Mar.

Wand, Y., & Weber, R. (1993). On the ontological expressive -ness of information systems analysi s and design grammars.

Journal of Information Systems , 3(4), 217-237.

key terMs

Analysis Model: A model developed to learn all aspectsof a problem domain to determine the best way to solve aspeci c set of user needs.

Design Model: A model developed to represent theoptimal technical solution of a speci ed user need (as rep -resented in a requirements model).

Model: An abstraction represented in a modeling lan-

guage.

Modeling Language: A language (i.e., a set of symbols,and rules for how to combine these symbols) to representknowledge relevant in (information systems) develop -ment.

Requirements Model: A model to represent the externalrequirement to a system without taking into account howthe system looks inside.

UML (Unifed Modeling Language): A general-purposevisual modeling language that is used to specify, visualize,construct and document the artifacts of a software system.

Visual Modeling Language: A diagrammatic modelinglanguage (i.e., where the models made in the language are2-dimensional diagrams).

evaluating uml using a generic quality framework

Documents