Jambalaya: The Closest Visualisation Fit for the Protégé Ontology Conceptual-Relationship Tracer

Muthukkaruppan Annamalai1 and Hamid Reza Mohseni2

Faculty of Computer and Mathematical Sciences Universiti Teknologi MARA

40450 Shah Alam, Selangor, Malaysia mk@tmsk.uitm.edu.my1, hrmohseni@gmail.com2

Abstract — Retrieving the relevant background information about the conceptual-relationships defined in an ontology is a prerequisite for evaluating the competency of the ontology. For this purpose, a Conceptual-Relationship Tracer (CRT) for Protégé ontologies has been developed in a previous work. Protégé ontologies are web-ontologies developed using the Protégé ontology editor. The CRT’s textual output is sometimes hard to trace, making the intended information difficult to understand. We regard, a distinct visualisation of the output can help to better convey the intended information, thus improving its understandability. The candidates are TGViz, OntoViz and Jambalaya, the three independently developed Protégé visualiser plugins. In this paper, we provide a set of visualisation factors to qualitatively compare the utility and the usability potentials of the candidates, and to decide which one is best at providing the requisite visualisation support for the CRT. The results of the analyses show that Jambalaya is the closest visualisation fit.

Keywords – Qualitative comparison; Ontology Visualisation factors; Ontology engineering; Competency evaluation

I. INTRODUCTION An ontology is an explicit specification of a shared

conceptualisation [1]. It is formally represented and organised into classes of entities as a way of structuring and defining the meaning of the concepts that feature in a domain. A web-ontology is portable on the web, and is represented using Frames and Description logic [2]. The Frame representation allows the specification of relations and value restricted properties attributed to a concept, helping to constrain its interpretation. Description logic is a restrictive binary logic to express the relationships between concepts. Instances are extensions of the defined concepts used in the description of knowledge base contents.

Ontologies are often developed with the aid of ontology editors such as Protégé [3], which supports the construction and management of the ontologies. We refer to the web-ontologies developed using Protégé as Protégé ontologies.

Ontology evaluation is a necessary part of an ontology life cycle. Because ontologies are developed to serve as a semantic backbone of knowledge based systems [4], Annamalai [5] argues that the evaluation of the competency of an ontology is crucial for sake of the ontology’s successful use. An accepted course to gauge the competency of an ontology is by means of

competency questions [6], which a knowledge base founded on an ontology should be able to answer. For example, a competency question to check the contents of a knowledge base instantiated using concepts in a Family ontology (illustrated in Figure 1) is “Who are the female members related to Adam [man]?” As a result, the query answerer is expected to list all the female members in Adam’s family who are directly and indirectly related to him.

Figure 1. A graphically reproduced segment of an example Family ontology

It should be noted that that the evaluation in the manner of [6] is performed on a knowledge base after the related ontology has been fully developed and implemented, i.e., summatively. On the other hand, we are concerned with the competency evaluation of an ontology under development, i.e., its formative evaluation. In this case, the evaluation needs to corroborate the applicability of the conceptual definitions with respect to the competency questions by checking the evolving ontology, and not the knowledge base.

One possibility is to check the ontological commitment is by examining whether the ontology has the necessary concepts, relationships and properties to formulate the predefined competency questions, as well as their corresponding answers. Referring to the competency question for the Family ontology mentioned earlier, the ontology engineer needs to check whether the two key concepts in the question, i.e., Man and Female, are defined in the ontology, and are related in such way to respond to this question. For this purpose, a Conceptual-Relationship Tracer (CRT) for Protégé ontologies has been developed in a previous work [7] to assist with the competency evaluation.

A main limitation of the CRT is that its output is represented in textual form that are sometimes long winded and bogged with unwanted redundancies. It makes the output hard to trace and sometimes the represented information is difficult to understand. For example, look at the CRT’s output for the request to trace the relationships between the Man and the Female concepts defined in the Family ontology shown in Figure 2.

Man has-brother(brother-of) Man : Man has-father(father-of) Man : Man has-son(son-of) Man : Man father-of(has-father) Man : Man father-of(has-father) Woman : Woman has-daughter (daughter-of) Woman : Woman has-mother(mother-of) Woman : Woman has-sister(sister-of) Woman : Woman has-gender(gender-of) Female

Man has-brother(brother-of) Man : Man has-father(father-of) Man : Man has-son(son-of) Man : Man father-of(has-father) Man : Man has-daughter (daughter-of) Woman : Woman has-daughter (daughter-of) Woman : Woman has-mother(mother-of) Woman : Woman has-sister(sister-of) Woman : Woman has-gender(gender-of) Female

Man has-brother(brother-of) Man : Man has-father(father-of) Man : Man has-son(son-of) Man : Man father-of(has-father) Man : Man has-mother(mother-of) Woman : Woman has-daughter (daughter-of) Woman : Woman has-mother(mother-of) Woman : Woman has-sister(sister-of) Woman : Woman has-gender(gender-of) Female

Man has-brother(brother-of) Man : Man has-father(father-of) Man : Man has-son(son-of) Man : Man father-of(has-father) Man : Man has-wife(has-husband) Woman : Woman has-daughter (daughter-of) Woman : Woman has-mother(mother-of) Woman : Woman has-sister(sister-of) Woman : Woman has-gender(gender-of) Female

Man has-brother(brother-of) Man : Man has-father(father-of) Man : Man has-son(son-of) Man : Man father-of(has-father) Man : Man has-sister(sister-of) Woman : Woman has-daughter (daughter-of) Woman : Woman has-mother(mother-of) Woman : Woman has-sister(sister-of) Woman : Woman has-gender(gender-of) Female

Figure 2. The CRT’s output tracing the relationships between Man and Female in the Family ontology

The output indicates that the concepts Man and Female are related through an intermediary concept Woman in five different ways, i.e., through each of the five parallel relations defined between Man and Woman (printed in bold), namely father-of, has-daughter, has-mother, has-wife and has-sister, and their respective inverse relations in brackets. However, there are redundancies in the output that makes the information appear long winded. Note the self relations between Man (e.g. brother-of), the self relations between Woman (e.g. has-daughter) and the direct relation between Woman and Female (has-gender) are repeated in all the five text blocks.

We believe the visualisation of the CRT’s output can express the relevant information about the ontology being evaluated more simply and clearly, and free from redundancies. After all, visualisation is mainly designed to enhance browsing, exploring and interacting with information spaces [8] in order to promote more intuitive and easy understanding of the investigated information. To some extent, this is evident in Figure 3, which graphically delineates the CRT’s textual output shown in Figure 2. The concepts are represented as labelled nodes while the arcs denote the direct relationships between a pair of concepts. Note: The relation names (arc labels) are not shown for the sake of visibility of the relationships.

Figure 3. A graphical delineation of the CRT’s output tracing the relationships

between Man and Female in the Family ontology

To this end, we propose to adapt and link an available visualiser to the CRT in order produce a distinct graphical representation of its textual output with an intention of improving the traceability and the understandability of the analytic information.

The paper describes the investigative approach we have taken to identify a suitable visualisation support for the CRT. In section II, we discuss the related works, which includes the candidate visualisers. In section III, we detail the visualisation factors that are used to analyse the visualisers. The results of the analyses are presented in section IV. Finally, in the concluding section, we summarise the contribution of this paper and point to the direction of future work.

II. RELATED WORKS This section presents the CRT. Then, it quickly reviews the

visualisation support for engineering web-ontologies before discussing about the Protégé visualisers in brief.

A. Conceptual-Relationship Tracer (CRT) The CRT is designed to help an ontology engineer to

interactively query and analyse the competency of a Protégé ontology under development [7]. A user can make a request for the background information about the terms defined in an ontology by invoking one of the six pre-defined query options portrayed in Figure 4. The concept, relation and property terms are represented using rectangle, arrow and diamond shapes, respectively. The solid shapes indicate the terms in focus, while the outlined shapes denote the terms associated with the term(s) in focus.

Figure 4. An outline of the test cases for each of the CRT’s six query options

For option I, the CRT lists the properties associated with the named concept, and all relations that originate from it or directed to it. For option II, the CRT lists all the concepts that are defined using the named property p. For option III, the CRT lists all the concept pairs that are directly related through the named relation r. For option IV, all the concepts immediately succeeding the named predecessor concept through the named relation r are listed. For option V, all the concepts immediately preceding the named successor concept through the named relation r are listed. Options IV and V are

actually subset of option I, but centred on specific relation and its orientation. Finally, for option VI, the CRT lists all the direct as well as the indirect relationships between the named source concept S and the target concept T. When the source and target concepts are adjacent, i.e., directly related, the result is somewhat similar to option IV. However, when they are distant, i.e., indirectly related, the association between these concepts must be inferred, and there can be multiple ways and intermediate relations to consider (as demonstrated by the example in Section I).

The concept, property and relation terms defined in an ontology are extracted from the Protégé ontology data file, and stored in three distinct vectors. In addition, the conceptual- relationships in the ontology are captured separately as a directed graph and represented in an adjacency matrix that facilitates easy access and manipulation. As a result, the processing of query options I to V can be implemented quite straightforwardly by systematically searching the data in the term vectors and the adjacency matrix to retrieve the relevant background information.

On the other hand, the implementation of query option VI is rather challenging because of the need to find not only the direct relationships between two concepts, but also all the indirect relationships that exist between them. For this, the Warshall’s algorithm [9] has been adapted to compute the transitive closure of the binary relationships in the ontology. The original Warshall’s algorithm simply manipulates Boolean values, and so can only indicate the presence or the absence of path connecting any two nodes in a graph. Therefore, the algorithm has been modified to allow the manipulation of textual information in the adjacency matrix. Besides, a marking scheme has been employed to represent the tracking information created during the computation of the transitive closure, which is later utilised to construct the output.

B. Visualisation Support for Web-ontology Editors The commonly referenced web-ontology editors are Protégé

[10], OntoEdit [11], WebODE [12] and Hozo [13]. Protégé is an extendable ontology engineering platform whose core functionalities are knowledge acquisition and ontology creation. Hozo specifically supports the development of ontologies that deal with “role concepts”. More recently, the functionalities of OntoEdit and WebODE have been consolidated and developed into a comprehensive extendable ontology engineering environment called NeON [14]. While, NeON aims to provide ontology engineering lifecycle capabilities in an integrated manner, its commercial version called OntoStudio [15] focuses on axioms and axiom based information integration.

The ontology editors are equipped with some form of visualisation capabilities to support the ontology construction and management. In general, the visualisation concerns with the display of concepts, properties, relations and instances. The displays are usually based on nested and graph views. Users can exploit the graphical support to create, browse, edit, navigate and debug the ontologies.

The concept hierarchies rendered by these ontology editors support the visualisation of dependencies, which facilitates the

inspection of the subtype relationships. Most of the editors also support the visualisation of the instances according to the concept structure in the ontology.

Since the visualisation of dense ontologies leads to cluttered views, functions for optimised view of the graph such as zoom, rotation, and alternative layouts such as clustered graphs, hyperbolic tree and fish-eye view are provided.

In the context of evaluation, the ontology editors commonly emphasise consistency checking helps to highlight specific violations. Sometimes the ontological commitments are tested by querying over a set of instances. Incidentally, the queries offer a view onto the knowledge base.

C. Protégé Visualiser plugins The visualisation environment in extendable ontology

editors such as Protégé is supported using plugins. There are currently three open source visualiser plugins for Protégé that supports the visualisation of Frame based ontologies, namely, OntoViz [16], Jambalaya [17] and TGViz [18].

TGViz integrates the TouchGraph [19] technology to create the visualisation of the concepts and their corresponding instances in a Protégé ontology as a directed graph. It creates the impression of the taxonomic structure of the concepts by making use of an animated spring embedder algorithm [20]. The nodes representing the concepts can be expanded in order to view the details. Users can also tweak the ontology visualisation requirements, such as handling specific types of relations or labelling of the arcs representing the relations.

OntoViz uses GraphViz [21], a graph drawing software to visualise Protégé ontologies. OntoViz is found to be useful for graphically representing small ontologies or parts of a large ontology. The types of visualisations are configurable in term of picking a set of concepts or their instances, and can be made to display the nodes and arcs using a number of colours.

Jambalaya visualises Protégé ontologies with SHriMP [22], which was originally used for software visualisation. Its zooming feature is developed using animated zooming techniques [23] and uses interactive environment, animated panning, multiple colours and other functionalities to visualise the concepts and instances. It simultaneously supports two other visualisation layouts: tree and nested view.

Since we propose to find a visualisation support for the CRT that has been developed as a front-end application for Protégé, it would seem logical and less demanding to adapt one of these existing Protégé visualisers.

III. TGVIZ, ONTOVIZ OR JAMBALAYA: HOW TO CHOOSE? Utility and usability testing are the common basis of tool

evaluation. We used this approach to assess the strength and weakness of the visualisers. In practice, these tests are performed post-implementation by involving real users. However, since the CRT has no visualisation support at this stage, we will substitute the practical user testing with informal analyses to qualitatively compare the utility and usability potentials of the visualisers for the CRT. We describe our investigative approach, giving details about the synthesised visualisation factors in the following sub-sections.

A. Approach First, we carried out a general inquiry by reviewing the

background research on the visual display of information, followed by a contextual inquiry on the visualisation of ontologies, and their supporting tools (see Section II). We also surveyed the discussion about good and bad visualisation in online forums (e.g. visualisation course forum at the University of Berkeley [24]). Backed by our experience in competency evaluation of ontologies and literature reviews, we recognised a number of factors that are concerned with the utility and usability potentials of the CRT’s visualisation extension.

Next, we compared the three Protégé visualisers to gauge their capabilities in terms of the identified visualisation factors. With regards to utility analysis, we assessed the visualisers’ effective support for competency evaluation, i.e., we compared their ability to visually express the CRT’s output and the flexible support available to view the visualisation. As for usability analysis, we assessed the visualisers’ efficient and user-friendly support by comparing their user interface design against a set of basic usability principles.

B. Visualisation Factors Centred on Utility We view the factors of visualisation from the perspective of

an ontology engineer performing the competency evaluation task. In the process, we studied the needs of the visual content and the supporting visual form.

Factors of Visual Content

Intuitiveness. Making sense of a visual content will be effortless if it maps to the world view of the user. In our case, the visual content describes the conceptual-relationships in an ontology, which in essence is a concept model. Since, graphs are pervasive in conceptual modelling, the graphical representation of the content would make the displayed information intuitively understandable by users. Therefore, we check whether the visualiser supports graphical layout.

Expressiveness is a necessary quality to support cognitive abstraction in ontology tools [25]. We check the expressiveness of the visual content by testing whether the visualiser is able to effectively visualise the output generated for each of the CRT’s query options, i.e., it is able to express all and only the relevant facts related to the query

Clarity. A visualisation should clearly communicate its content in order to facilitate its interpretation. We check whether the content displayed by a visualiser is perceptually obvious and easy to grasp, and explanation is provided where necessary. However, too much information can sometimes detract clarity from the visualisation. For example, while labels increase the readability of the visual representation, cluttered labels would contribute to the opposite.

Factors of Visual Form

Display characteristics. Users should have the flexibility to change the visual effects to suit their needs. For example, the nodes and arcs in a graph could be represented using specific shapes and colours. In fact, colours help to distinguish the different arcs in parallel relations. We check whether the visualiser provides the means for a user to set the desired

visualisation format such as the size of the display space, the types of shapes and colours, and the scale encodings.

Optimised view. Concepts and relations in dense and large graphs tend to be mixed up, and tracing the conceptual-relationships becomes difficult, unless the user can focus on a part of the ontology, or relocate the position of the related concepts. This optimised view of the visualisation is usually supported by interactive reformatting features such as drag, rotation and zoom. We check whether the visualiser supports the facilities to rearrange, reposition and expand the areas of interest in the visualisation.

Multiple layouts. As in the previous case, users can gain better insights about the represented information, if the content can be visualised from different perspectives or layout. We check whether the visualiser allows the automatic reorganisation of the visual content and also enables a user to rapidly switch between the different layouts.

C. Visualisation Factors Centred on Usability In the absence of usability testing, we propose to make use

of Nielsen’s Usability Heuristics (UH) for interface design [26] in order to informally assess the usability of the visualisers. The ten factors are associated with usability attributes as learnability, efficiency, memorability, errors and satisfaction.

UH1. Visibility of system status: The system should always keep users informed about what is going on, through appropriate feedback within reasonable time.

UH2. Match between system and the real world: The system should speak the users' language, with words, phrases and concepts familiar to the user, rather than system-oriented terms. It should make information appear in natural and logical order.

UH3. User control and freedom: Users need a clearly marked "emergency exit" to leave the unwanted state without having to go through an extended dialogue. Undo and redo facilities should also be supported.

UH4. Consistency and standards: Users should not have to wonder whether different words, situations, or actions mean the same thing. Platform conventions should be followed.

UH5. Error prevention: The system should either eliminate error-prone conditions or check for them and present users with a confirmation option before they commit to the action.

UH6. Recognition rather than recall: The system should minimise the user's memory load by making objects, actions, and options visible. Instructions should be visible or easily retrievable whenever appropriate.

UH7. Flexibility and efficiency of use: Accelerators may often speed up the interaction for the expert user such that the system can cater to both inexperienced and experienced users. System should allow users to tailor frequent actions.

UH8. Aesthetic and minimalist design: Dialogues should not contain information which is irrelevant or rarely needed.

UH9. Help users recognise, diagnose, and recover from errors: Messages should be expressed in plain language, precisely indicate the problem, and constructively suggest a solution.

UH10. Help and documentation: It is necessary to provide help and documentation that are easy to search, focused on the user's task, and list concrete steps to be carried out.

IV. WHY JAMBALAYA IS CHOSEN? We present the results of our analyses based on the

visualisation factors detailed in the previous section, which clearly point to Jambalaya as the choice visualiser plugin for the CRT.

A. Visualisation Factors Centred on Utility Figure 5 gives an overview of the analysis about the utility

potentials of the CRT extended by each of the three Protégé visualiser plugins.

Figure 5. Result of the analysis about the utility potentials of the visualisers

Intuitiveness. All the visualisers support graphical layout.

Expressiveness. All the visualisers lack effective support for query expressiveness in one way or another. A more detailed description of the analysis is shown in Figure 6.

Figure 6. The visualisers’ capacity to support the CRT’s query expressiveness

Jambalaya does not support query option I well because it could not display the properties of the queried concept. None of the visualisers support query option II. TGViz does not support query option III, and the remaining two fail on the side of minimality. The inability to present only the relevant facts is apparent in the visualisers’ responses to query options IV and V. Jambalaya and TGViz do not support query option VI, and even OntoViz’s support for this option is rather weak, where it tends to display extraneous concepts and relations, or misses out the relevant ones.

Clarity. It is quite tricky to trace the conceptual-relationships on the cluttered view of TGViz. Further, parallel relations between a pair of concepts are indistinct because the multiple relations are combined and represented using a single arc.

Display characteristics. TGViz only allows two colours in its graph, one for the nodes and the other for the arcs. OntoViz also provides limited colour options, but allows users to select the shapes of nodes and arcs. In contrast, Jambalaya demonstrates more efficient use of screen and provides a wider range of colours and shapes.

Optimised view. All the visualisers provide some form of reformatting facilities such as scaling and zooming. However, OntoViz restricts the relocation of a particular node to a desired position. Although TGViz supports relocation, the entire graph

moves together. By comparison, Jambalaya offers a more varied support to interactively reformat the displayed content.

Multiple layouts. TGVizTab and OntoViz only supports graphical layout. On the contrary, Jambalaya supports additional predefined layouts such as tree and nested views of the graph, and also allows users to quickly switch between these layouts.

The result of the analysis centred on the utility potentials of the CRT favours an extension of visualisation support proffered by Jambalaya. Although OntoViz and Jambalaya are better than TGViz in terms of the visual content, Jambalaya edges over the rest when the factors of the visual form are taken into account.

B. Visualisation Factors Centred on Usability Figure 7 shows the result of our analysis about the usability

potentials of the visualisers based on Nielsen’s usability heuristics.

Figure 7. Result of the analysis about the usability potentials of the visualisers

UH1. TGVizTab and OntoViz do not give the information about the status of the running process. When it takes a long time to generate the graph, it is uncertain as to whether the visualiser is functioning or has stalled.

UH2. The labels such as slx, slt and sle in the visualisation configuration section of the OntoViz’s interface is not clear. We did not come across such doubtful problems in TGViz and Jambalaya.

UH3. Jambalaya provides a mechanism to abort the running process but TGVizTab and OntoViz do not

UH4. We are generally satisfied to note the consistent use of instructions and messages in all the visualisers.

UH5. It is unlikely for users to get into an erroneous system state due to user input in all the visualisers because the inputs are largely restricted to list boxes, check boxes and radio buttons. Moreover, mistaken selections will not lead to grave damage for the users because the visualisers do not tamper with the visual information; they merely display it.

UH6. The instructions and messages of the visualisers are noticeably highlighted and always appear in the appropriate areas on the interfaces.

UH7. Unlike Jambalaya, TGViz and OntoViz do not provide multiple layout facility. However they offer some options to configure the visualisation to suit a user’s need. Nonetheless, the configuration is not easy to do when the labels are doubtful as in the case of OntoViz. Jambalaya additionally allows users to select and unselect the concepts and relations to be displayed. Further, Jambalaya relatively requires much less time to generate its visual display.

UH8. We have not noticed the presence of irrelevant or redundant information in the system dialogues.

UH9. While testing the visualisers, we did not encounter errors to recover from. This could be attributed to the restricted user inputs mentioned earlier, which prevent a problem from occurring in the first place.

UH10. TGViz and OntoViz provide online instructions. Jambalaya additionally provides a quick start guide and has an online help page.

The result of the analysis centred on the usability potentials of the CRT clearly favours an extension of visualisation support afforded by Jambalaya.

V. CONCLUSIONS The CRT has been developed as a front-end application for

the Protégé ontology editor. In our effort to provide a visualisation support for the CRT, we considered adapting one of the three existing Protégé visualiser plugins: TGViz, OntoViz and Jambalaya. The question is which visualiser is best at providing the requisite visualisation support for the CRT?

In the absence of a definable benchmark to be used as the reference for comparison, we deliberated on a set of visualisation factors to qualitatively compare the strengths and weaknesses of the three visualisers. The factors are based on the utility and the usability potentials of the CRT extended by a visualisation support.

With regards to utility analysis, we assessed the visualisers’ effective support for competency evaluation, i.e., we compared their ability to visually express the CRT’s output clearly and provide the users the flexibility to view the visualisation. Consequently, intuitiveness, expressiveness, clarity, display characteristics, optimised view and multiple layouts have been identified as important utility centred visualisation factors.

As for usability analysis, we assessed the visualisers’ efficient and user-friendly support by comparing their user interface design against a Nielsen’s ten usability heuristics factors.

The results of the utility and usability analyses show that Jambalaya is the closest visualisation fit for the CRT.

Visualisation support for the CRT’s query expressiveness is a quintessential quality, yet Jambalaya lacks effective support for query expressiveness on many cases (see Figure 6). Our future work will be devoted to: 1) addressing the Jambalaya’s expressiveness deficiencies, and 2) integrating the enhanced visualiser support into the CRT’s functions.

REFERENCES [1] T. Gruber, “A translation approach to portable ontologies”. Knowledge

Acquisition, vol. 5, no. 2, pp. 199 – 220, 1993. [2] H. Wang, N. Noy, A. Rector, M. Musen, and T. Redmond, “Frame and

OWL side-by-side”, International Protégé Conference, Stanford, USA, 2006.

[3] N. Noy, R. Fergerson, and M. Musen, “The knowledge model of Protégé-2000: combining interoperability and flexibility”, International Conference on Knowledge Engineering and Knowledge Management, Juan-les-Pins, France, 2000.

[4] A. Gangemi, C. Catenacci, M. Ciaramita, and J. Lehmann, “A theoretical framework for ontology evaluation and validation”, Second Italian Semantic Web Workshop, Trento, Italy, 2005.

[5] M. Annamalai, “Formative evaluations of ontologies for information agents”, Conference in Computer Science, Technology and Networking, Shah Alam, Malaysia, 2005.

[6] M. Gruninger, and M. Fox, “Methodology for the design and evaluation of ontologies”, International Joint-Conference in Artificial Intelligence Workshop on Basic Ontological Issues in Knowledge Sharing, Montreal, Canada, 1995.

[7] M. Annamalai, and N. Teo, “Tool support for competency evaluation of web-ontologies”, International Conference on Informatics, Petaling Jaya, Malaysia, 2007.

[8] E. Freedman, and P. Shah, “Toward a model of knowledge-based graph comprehension”. International Conference on Diagrammatic Representation and Inference, Callaway Gardens, Georgia, USA, 2002.

[9] S. Warshall, “A theorem on boolean matrices”. Journal of the ACM, vol. 9, pp. 11 – 12, 1962.

[10] N. Noy, et al., “Creating semantic web contents with Protégé 2000”. IEEE Intelligent Systems, vol. 16, pp. 60-71, 2001.

[11] Y. Sure, et al., “OntoEdit: Collaborative ontology development for the semantic web”, International Semantic Web Conference, Sardinia, Italy, 2002.

[12] Ó. Corcho, M. Fernández-López, A. Gómez-Pérez, and Ó. Vicente, “WebODE: An integrated workbench for ontology representation, reasoning, and exchange”, International Conference on Knowledge Engineering and Knowledge Management. Ontologies and the Semantic Web, Siguenza, Spain, 2002.

[13] K. Kozaki, Y. Kitamura, M. Ikeda, and R. Mizoguchi, “Hozo: An environment for building/using ontologies based on a fundamental consideration of role and relationship”, International Conference on Knowledge Engineering and Knowledge Management. Ontologies and the Semantic Web, Siguenza, Spain, 2002.

[14] P. Hasse, H. Lewen, R. Studer and M. Erdmann, “The NeON ontology engineering toolkit”. WWW2008, Beijing, China, 2008.

[15] M. Weiten, “OntoSTUDIO as a ontology engineering environment”. Semantic Knowledge Management, J. Davies, M. Grobelnik and D. Mladenic (eds), Springer-Verlag, Berlin, pp. 51-60, 2009.

[16] M. Sintek, “OntoViz”, 2001. Available online from: http://protegewiki.stanford.edu/wiki/OntoViz

[17] M. Storey, N. Noy, M. Musen, C. Best and R. Fergerson, “Jambalaya: an interactive environment for exploring ontologies”, International Conference on Intelligent User Interfaces, San Francisco, California, USA, pp. 239-, 2002.

[18] H. Alani, “TGVizTab: An ontology visualization extension for Protégé”, Knowledge Capture Workshop on Visualization Information in Knowledge Engineering, Sanibel Island, Florida, USA, 2003.

[19] “TouchGraph”, 2001. Available online from: http://www.touchgraph.com

[20] P. Mutton, and P. Rodgers, “Spring embedder preprocessing for www visualization”, International Conference on Information Visualization, London, UK, 2002.

[21] J. Ellson, E. Gansner, L. Koutsofios, S. North, and G. Woodhull, “GraphViz: open source graph drawing tools”, International symposium on Graph Drawing, Vienna, Austria, 2001.

[22] M. Storey, K. Wong, F. Fracchia and H. Muller, “On integrating visualization techniques for effective software exploration”, IEEE Symposium on Information Visualization, Phoenix, Arizona, USA, 1997.

[23] B. Bederson, J. Meyer, and L. Good, “Jazz: an extensible zoomable user interface graphics toolkit in Java”, ACM Symposium on User Interface Software and Technology, pp. 171-180, 2000.

[24] “Visualisation forum at the University of Berkeley”, 2007. Available online from: http://vis.berkeley.edu/courses/cs294-10-fa07/wiki/ index.php

[25] M. Lanzenberger, J. Sampson and M. Rester, “Visualization in ontology tools”, International Conference on Complex, Intelligent and Software Intensive Systems, Fukuoka, Japan, 2009.

[26] J. Nielsen, “Ten Usability Heuristics”, 2005. Available online from: http://www.useit.com/papers/heuristic/heuristic_list.html