a survey on designing metrics suite to asses the quality of ontology

6
  Abstract---With the persistent growth of the World Wide Web, the difficulty is increased in the retrieval of relevant information for a user’s query. Present search engines offer the user with several web pages, but different levels of relevancy. To overcome this, the Semantic Web has been proposed by various authors to retrieve and utilize additional semantic information from the web. As the Semantic Web adds importance for sharing knowledge on the internet this has guide to the development and publishing of several ontologies in different domains. Using the database terminology, it can be said that the web-ontology of a semantic web system is schema of that system. As web ontology is an integral aspect of semantic web systems, hence, design quality of a semantic web system can be deliberated by measuring the quality of its web- ontology. This survey focuses on developing good ontologies. This survey draws upon semiotic theory to develop a suite of metrics that assess the syntactic, semantic, pragmatic, and social aspects of ontology quality. This research deliberates about the metrics that may contribute in developing a high quality semantic web system.  Keywords--- Quality Metrics, Web ontology, Semiotic Metrics, Semantic Quality, Domain modularity.  I. INTRODUCTION EMANTIC Web is nothing but the extension of the present web in which the web resources are prepared with formal semantics about their interpretation for the machines. These web resources are combined in the form of web information systems, and their formal semantics are usually characterized in the form of web-ontologies. By means of the database terminology, it can be said that the web-ontology of a semantic web system is representation of that system [11]. Design quality of a semantic web system can be calculated by computing the quality of its web-ontology because web ontology is the integral element of semantic web systems [25]. The main concern is that when the design of a web-ontology is completed, it is suitable time to assess its quality so that in case, the design is of low quality, it can be enhanced before its instantiation. This helps in saving of considerable amount of cost and effort for developing high quality semantic web systems. Metrics are considered as the appropriate tools for estimating quality. This survey focuses on several metrics for web ontology quality evaluation. II. LITERATURE SURVEY Ahluwalia et al., [1] presented a Semiotic Metrics Suite for Assessing the Quality of Ontologies. Table 1 shows some of the metrics for quality evaluation [1, 3]. As a decisive construct, overall quality (Q) is a subjective function of its syntactic (S), semantic (E), pragmatic (P), and social (O) qualities [1] (i.e., Q = b1×S + b2×E + b3×P + b4×O). The addition of weight is equal to 1. In the absence of pre-specifi ed weights, the weights are assigned to be equal. Syntactic Quality (S) evaluates the quality of the ontology according to the way it is written. Lawfulness is the extent to which an ontology language’s rules have been obeyed. Not every ontology editors have error-checking capabilities; however, without correct syntax, the ontology cannot be read and used. Richness is nothing but the proportion of features in the ontology language that have been used in ontology (e.g., whether it includes terms and axioms, or only terms). Richer ontologies are more valuable to the user (e.g., agent). Semantic Quality (E) estimates the meaning of terms in the ontology library. Three attributes are used here are interpretability, consistency, and clarity. Interpretability deals with the meaning of terms (e.g., classes and properties) in the ontology. In the real world, the knowledge provided by the ontology can map into meaningful concepts. This is accomplished by checking that the words used by the ontology be present in another independent semantic source, such as a domain-specific lexical database or a comprehensive, generic lexical database such as WordNet. Consistency is nothing but whether terms having a consistent meaning in the ontology. For example, if an ontology claims that X is a subclass of Y, and that Y is a property of X, then X and Y have incoherent meanings and are of no semantic value. For example, ontological terms such as IS-A is often used inconsistently. Clarity is the term which determines whether the context of terms is clear. For example, if ontology claims that class “Chair” has the property “Salary,” an agent must know that this illustrate academics, not furniture. Pragmatic Quality (P) deals with the ontology’s usefulness for users or their agents, irrespective of syntax or semantics. Three criteria are used for determining P. Accuracy is whether the claims on ontology makes are ‘true.’ This is very tricky to determine automatically without a learning mechanism or truth maintenance system. Currently, a domain expert evaluates accuracy. The measure of the size of the ontology is A Survey on Designing Metrics suite to Asses the Quality of Ontology K.R Uthayan G.S.Anandha Mala, Professor & Head, Department of Information Technology, Department of Computer Science & Engineering SSN College of Engineering St.Joseph’s College of Engineering, Chennai, India Chennai, India [email protected] [email protected] S (IJCSIS) International Journal of Computer Science and Information Security, Vol. 8, No. 8, November 2010 179 http://sites.google.com/site/ijcsis/ ISSN 1947-5500

Upload: ijcsis

Post on 09-Apr-2018

220 views

Category:

Documents


0 download

TRANSCRIPT

8/8/2019 A Survey on Designing Metrics suite to Asses the Quality of Ontology

http://slidepdf.com/reader/full/a-survey-on-designing-metrics-suite-to-asses-the-quality-of-ontology 1/6

 

 Abstract---With the persistent growth of the World Wide Web,

the difficulty is increased in the retrieval of relevant information for a

user’s query. Present search engines offer the user with several web

pages, but different levels of relevancy. To overcome this, the

Semantic Web has been proposed by various authors to retrieve and

utilize additional semantic information from the web. As the

Semantic Web adds importance for sharing knowledge on the internet

this has guide to the development and publishing of several

ontologies in different domains. Using the database terminology, it

can be said that the web-ontology of a semantic web system is

schema of that system. As web ontology is an integral aspect of 

semantic web systems, hence, design quality of a semantic web

system can be deliberated by measuring the quality of its web-

ontology. This survey focuses on developing good ontologies. This

survey draws upon semiotic theory to develop a suite of metrics that

assess the syntactic, semantic, pragmatic, and social aspects of 

ontology quality. This research deliberates about the metrics that may

contribute in developing a high quality semantic web system.

 Keywords---  Quality Metrics, Web ontology, Semiotic Metrics,

Semantic Quality, Domain modularity. 

I.  INTRODUCTION 

EMANTIC Web is nothing but the extension of the

present web in which the web resources are prepared with

formal semantics about their interpretation for the machines.

These web resources are combined in the form of web

information systems, and their formal semantics are usually

characterized in the form of web-ontologies. By means of the

database terminology, it can be said that the web-ontology of a

semantic web system is representation of that system [11].

Design quality of a semantic web system can be calculated by

computing the quality of its web-ontology because web

ontology is the integral element of semantic web systems [25].

The main concern is that when the design of a web-ontology is

completed, it is suitable time to assess its quality so that in

case, the design is of low quality, it can be enhanced before its

instantiation. This helps in saving of considerable amount of 

cost and effort for developing high quality semantic web

systems. Metrics are considered as the appropriate tools for

estimating quality. This survey focuses on several metrics for

web ontology quality evaluation.

II.  LITERATURE SURVEY 

Ahluwalia et al., [1] presented a Semiotic Metrics Suite for

Assessing the Quality of Ontologies. Table 1 shows some of 

the metrics for quality evaluation [1, 3].

As a decisive construct, overall quality (Q) is a subjective

function of its syntactic (S), semantic (E), pragmatic (P), and

social (O) qualities [1] (i.e., Q = b1×S + b2×E + b3×P +

b4×O). The addition of weight is equal to 1. In the absence of 

pre-specified weights, the weights are assigned to be equal.

Syntactic Quality (S) evaluates the quality of the ontology

according to the way it is written. Lawfulness is the extent to

which an ontology language’s rules have been obeyed. Not

every ontology editors have error-checking capabilities;

however, without correct syntax, the ontology cannot be read

and used. Richness is nothing but the proportion of features in

the ontology language that have been used in ontology (e.g.,

whether it includes terms and axioms, or only terms). Richer

ontologies are more valuable to the user (e.g., agent).

Semantic Quality (E) estimates the meaning of terms in the

ontology library. Three attributes are used here are

interpretability, consistency, and clarity. Interpretability dealswith the meaning of terms (e.g., classes and properties) in the

ontology. In the real world, the knowledge provided by the

ontology can map into meaningful concepts. This is

accomplished by checking that the words used by the ontology

be present in another independent semantic source, such as a

domain-specific lexical database or a comprehensive, generic

lexical database such as WordNet. Consistency is nothing but

whether terms having a consistent meaning in the ontology.

For example, if an ontology claims that X is a subclass of Y,

and that Y is a property of X, then X and Y have incoherent

meanings and are of no semantic value. For example,

ontological terms such as IS-A is often used inconsistently.

Clarity is the term which determines whether the context of terms is clear. For example, if ontology claims that class

“Chair” has the property “Salary,” an agent must know that

this illustrate academics, not furniture.

Pragmatic Quality (P) deals with the ontology’s usefulness

for users or their agents, irrespective of syntax or semantics.

Three criteria are used for determining P. Accuracy is whether

the claims on ontology makes are ‘true.’ This is very tricky to

determine automatically without a learning mechanism or

truth maintenance system. Currently, a domain expert

evaluates accuracy. The measure of the size of the ontology is

A Survey on Designing Metrics suite to Asses the

Quality of OntologyK.R Uthayan G.S.Anandha Mala, Professor & Head,

Department of Information Technology, Department of Computer Science & Engineering

SSN College of Engineering St.Joseph’s College of Engineering,Chennai, India Chennai, India

[email protected] [email protected]

S

(IJCSIS) International Journal of Computer Science and Information Security,

Vol. 8, No. 8, November 2010

179 http://sites.google.com/site/ijcsis/ISSN 1947-5500

8/8/2019 A Survey on Designing Metrics suite to Asses the Quality of Ontology

http://slidepdf.com/reader/full/a-survey-on-designing-metrics-suite-to-asses-the-quality-of-ontology 2/6

called as Comprehensiveness. Larger ontologies are more

probable to be complete representations of their domains, and

provide more knowledge to the agent. Relevance indicates

whether the ontology satisfies the agent’s specific

requirements.

 TABLE 1: DETERMINATION OF METRIC VALUES 

Attributes Determination

Overall Quality (Q) Q = b1.S + b2.E + b3.P + b4.O

Syntactic Quality (S) S = bs1.SL + bs2.SR

Lawfulness (SL)Let X be total syntactical rules. Let Xb be total breached rules. Let NS

be the number of statements in the ontology. Then SL = Xb / NS.

Richness (SR)

Let Y be the total syntactical features available in ontology language.

Let Z be the total syntactical features used in this ontology.

Then SR = Z/Y.

Semantic Quality (E) E = be1.EI + be2.EC + be3.EA

Interpretability (EI)

Let C be the total number of terms used to define classes and properties

in ontology.Let W be the number of terms that have a sense listed in WordNet. Then

EI = W/C.

Consistency (EC)

Let I = 0. Let C be the number of classes and properties in ontology.

∀Ci, if meaning in ontology is inconsistent, I+1. Therefore, I = number

of terms with inconsistent meaning. Ec = I/C.

Clarity (EA)Let Ci = name of class or property in ontology. ∀ Ci, count Ai, (the

number of word senses for that term in WordNet). Then EA = A/C.

Pragmatic Quality (P) P = bp1.PO + bp2.PU + bp3.PR

Comprehensiveness (PO)Let C be the total number of classes and properties in ontology. Let V

be the average value for C across entire library. Then PO = C/V.

Accuracy (PU)

Let NS be the number of statements in ontology. Let F be the number of 

false statements. PU = F/NS. Requires evaluation by domain expert

and/or truth maintenance system.

Relevance (PR)

Let NS be the number of statements in the ontology. Let S be the type of 

syntax relevant to agent. Let R be the number of statements within NS

that use S. PR = R / NS.

Social Quality (O) O = bo1.OT + bo2.OH

Authority (OT)

Let an ontology in the library be OA. Let the set of other ontologies in

the library be L. Let the total number of links from ontologies in L to

OA be K. Let the average value for K across ontology library be V.

Then OT = K/V.

History (OH) Let the total number of accesses to an ontology be A. Let the averagevalue for A across ontology library be H. Then OH = A/H.

Cohesion (Coh)Coh=|SCC|

Where SCC is separate connected components

Fullness (F)  

Readability (Rd)    

For the purpose of evaluation, it needs some knowledge of 

the agent’s requirements. This metric is coarse as it verifies for

the type of information the agent uses by ontology (e.g.,

property, subclass, etc), rather than the semantics needed for

(IJCSIS) International Journal of Computer Science and Information Security,

Vol. 8, No. 8, November 2010

180 http://sites.google.com/site/ijcsis/

ISSN 1947-5500

8/8/2019 A Survey on Designing Metrics suite to Asses the Quality of Ontology

http://slidepdf.com/reader/full/a-survey-on-designing-metrics-suite-to-asses-the-quality-of-ontology 3/6

specific tasks (e.g., the particular subclasses needed to

interpret a user’s specific query).

Social quality (O) imitates the fact that agents and

ontologies exist in communities. The authority of an ontology

is nothing but the number of other ontologies that link to it

(define their terms using its definitions). More authoritative

ontologies indicate that the knowledge they provide is

accurate or useful. The history indicates the number of times

the ontology is accessed. Ontologies are more dependablewhen they are with longer histories.

The cohesion (Coh) of a KB is nothing but the number of 

separate connected components (SCC) of the graph

representing the KB.

The fullness (F) of a class Ci is defined as the actual number

of instances that belong to the subtree rooted at Ci (Ci(I))

compared to the expected number of instances that belong to

the subtree rooted at Ci (Ci`(I)).

The readability (Rd) of a class C i is defined as the total of 

the number attributes that are comments and the number of 

attributes that are labels the class has.

Amjad et al., [2] provided the Web-Ontology Design

Quality Metrics. The author proposes design metrics for web-

ontology [21] by maintaining certain recommended principles

like a metric may reach its highest value for perfect quality for

excellent case and vice versa that is it may reach its lowest

level for worst case. It is supposed to be monotonic, clear, and

intuitive. It must correlate well with human decisions and it

should be automated if possible. The proposed metrics may

give notification about how much knowledge can be derived

from a given webontology; how much it is relevant to a user’s

specific necessities and how much it is effortless to reuse,

manage, trace and adapt. The metrics provided by the author

are Knowledge Enriched (KnE), Characteristics Relevancy

(ChR) and Domains modularity (DoM).

Knowledge Enriched metric

The reasoning capability of a web-ontology is determined

by Knowledge Enriched (KnE) metric, and it is based on two

sub-metrics so-called Isolated Axiom Enriched (IAE) metric

and Overlapped Axiom Enriched (OAE) metric. There are

three parts in this axiom namely, predicate, resource and

object. If none of these is similar with any other axiom of 

identical domain then that axiom is termed as isolated axiom.

If the two axioms have some similar parts, it is said to be

overlapped. There may be more than a few transitively

overlapped axioms in any domain. This metric determines the

percentage of IAE and OAE, and if the former is greater thanthe later one, then the web-ontology can be regarded as less

knowledge enriched. IAE is officially defined as the ratio of 

total number of isolated axioms (tIAs) to the total number of 

domain axioms (tDAs).

 

   (1)

In the above equation, n is total number of sub-domains of 

web-ontology. Similarly, the OAE metric is officially defined

as ratio of total number of overlapped axioms (tOAs) to the

total number of domain axioms. It can be written as follows:

 

 (2)

In the equation given above, n is total number of sub-

domains of web-ontology. Lastly, the KnE metric is the

difference of total number of overlapped axioms and the total

number of isolated axioms. It may be written as follows:

 

(3)

If the resultant KnE value is positive, then the web-ontology

is more knowledge enriched, if it is zero, then the web-

ontology is average knowledge enriched, and if it is negative,then the web-ontology is less knowledge enriched.

Characteristics Relevancy metric

Characteristics Relevancy (ChR) metric gives us the

suggestion about how much a given web-ontology is close to a

user’s specific necessities and the degree of reusability of the

web-ontology. Formally, it is termed as the ratio of the

number of relevant attributes (nRAs) in a class to the total

number of attributes (TnAs) of that class. It can be written as

follows:

 

   (4)

where n in above equation represents the total number of 

classes in the provided web-ontology. ChR metric reveals the

proportion of relevant attributes in the web-ontology, and this

number gives insights how much a web-ontology is relevant.

 Domain Modularity metric

Domain modularity (DoM) metric denotes the component-

orientation feature of a web-ontology. This metric specifies

the grouping of knowledge in different components of web-ontology. The webontology is best manageable, traceable,

reusable and adaptable, if it is designed in components

(subdomains). Formally, the DoM metric is given as the

number of sub-domains (NSD) contained in a webontology.

This metric also depends on the coupling and cohesion [25]

levels of sub-domains, and it is directly proportional to its

cohesion level and inversely proportional to its coupling level.

(IJCSIS) International Journal of Computer Science and Information Security,

Vol. 8, No. 8, November 2010

181 http://sites.google.com/site/ijcsis/

ISSN 1947-5500

8/8/2019 A Survey on Designing Metrics suite to Asses the Quality of Ontology

http://slidepdf.com/reader/full/a-survey-on-designing-metrics-suite-to-asses-the-quality-of-ontology 4/6

 

 

(5)

In the above equation, DCoh indicates the level of domain

cohesion and DCoup represents the level of coupling among

sub-domains of web-ontology domain. DoM metric is a realnumber indicating the degree of partial reusability of a given

web-ontology.

Samir et al., [3] given the OntoQA: Metric-Based Ontology

Quality Analysis. The metrics presented can highlight key

characteristics of an ontology schema and also its population

and facilitate users to make an informed judgment easily. The

metrics used by the author here are not 'gold standard'

measures of ontologies. Instead, the metrics are projected to

estimate several aspects of ontologies and their potential for

knowledge representation. Rather than describing ontology as

merely effective or ineffective, metrics describe a certain

aspect of the ontology because, in most cases, the way the

ontology is built is largely dependent on the domain in whichit is designed. The metrics defined here are Schema Metrics

and Instance Metrics. The following are metrics considered by

the author:

The following are some of Schema Metrics:

  Relationship Richness: The diversity of relations and

placement of relations in the ontology is defined by this

metrics. An ontology that has many relations further than

class-subclass relations is better than taxonomy with no more

than class-subclass relationships. The relationship richness

(RR) is defined as the ratio of the number of relationships (P)

defined in the schema to the sum of the number of subclasses

(SC) plus the number of relationships.

 

 Attribute Richness: The attribute richness (AR) is defined as

the average number of attributes (slots) per class. It is given as

the ratio of number attributes for all classes (att) to the number

of classes (C).

   

  Inheritance Richness: The inheritance richness of the

schema (IRs) is defined as the average number of subclasses

per class. The number of subclasses (C1) for a class Ci is

defined as |HC (C1, Ci)|.

 

The following are some of Instance Metrics:

Class Richness: The class richness (CR) of a knowledge

base is defined as the ratio of the number of classes used in the

base (C`) to the number of classes defined in the ontology

schema (C).

 

 Average Population: Formally, the average population (P)of classes in a knowledge base is defined as the number of 

instances of the knowledge base (I) to the number of classes

defined in the ontology schema (C).

 

 Importance: The importance (Imp) of a class Ci is defined

as the number of instances that belong to the subtree rooted at

Ci in the knowledge base (Ci(I)) compared to the total number

of instances in the knowledge base (I).

 

Werner [4] provided a Realism-Based Metric for Quality

Assurance in Ontology Matching. There are three levels

introduced to the methodology for the measurement of quality

improvements in single ontologies. These levels are:

•    Level 1:  reality, consisting of both instances and

universals and also the various relations that acquire

between them;

•    Level 2: the cognitive representations of this reality

personified in observations and interpretations;

•   Level 3: the   publicly accessible concretizations of thecognitive representations in representational artifacts of a

range of sorts, of which ontologies are examples.

Harith et al., [5] defined the metrics for Ranking

Ontologies. In this paper AKTiveRank, a prototype system for

ranking ontologies is proposed based on the analysis of their

structures. This paper describes the metrics used in the ranking

system. The ranking measures used are described below:

Class Match Measure

The Class Match Measure (CMM) is intended to estimate

the coverage of ontology for the provided search terms.

AKTiveRank looks for classes in every ontology that havelabels matching a search term either exactly (class label

identical to search term) or partially (class label “contains” the

search term).

 Density Measure

Density Measure (DEM) is deliberated to approximate the

representational-density or information-content of classes and

accordingly the level of knowledge detail. DEM considers

how well the concept is additionally specified (the number of 

(IJCSIS) International Journal of Computer Science and Information Security,

Vol. 8, No. 8, November 2010

182 http://sites.google.com/site/ijcsis/

ISSN 1947-5500

8/8/2019 A Survey on Designing Metrics suite to Asses the Quality of Ontology

http://slidepdf.com/reader/full/a-survey-on-designing-metrics-suite-to-asses-the-quality-of-ontology 5/6

subclasses), the number of attributes related with that concept,

number of siblings, etc.

Semantic Similarity Measure

Similarity measures have often been used in information

retrieval systems to afford enhanced ranking for query results.

Ontologies can be analyzed as semantic graphs of concepts

and relations, and hence similarity measures can be applied to

explore these conceptual graphs. This helps in resolvingambiguities.

Henry [7, 23] described a Measurement Ontology

Generalizable for Emerging Domain Applications on the

Semantic Web. The semantic Web is considered as the next

generation Web of structured data that are automatically

shared by software agents, which apply definitions and

constraints structured in ontologies to correctly process data

from contrasting sources. One aspect needed to develop

semantic Web ontologies of emerging domains is creating

ontologies of concepts that are common to those domains.

These general ontologies can be used as building blocks to

develop more domain-specific ontologies. However most

measurement ontologies focus on representing units of 

measurement and quantities, and not on other measurement

concepts such as sampling, mean values, and evaluations of 

quality based on measurements. In this paper, the author

elaborates on a measurement ontology that represents all these

concepts. This paper presents the generality of the ontology,

and describes how it is developed, used for analysis and

validated.

Fensel et al., [8] provided OIL (Ontology Interchange

Language): an ontology infrastructure for the Semantic Web.

Initially, Researchers in artificial intelligence motivate the

development of ontologies [14] to facilitate knowledge sharingand reuse. Ontologies [15] play a key role in supporting

information exchange across different networks. A

prerequisite for such a role lead to the development of a joint

standard for specifying and exchanging ontologies. The

authors present OIL which satisfies such standards.

Carlos et al., [9] presented an Ontology-based Metrics

Computation for Business Process Analysis.  Business Process

Management (BPM) aims to support the whole life-cycle

required to deploy and maintain business processes in

organizations. Analyzing business processes have a need of 

computing metrics that can facilitate determining the health of 

business activities and thus the whole enterprise. However, the

degree of automation currently achieved cannot maintain thelevel of reactivity and adaptation demanded by businesses. In

this paper the author argue and show how the use of Semantic

Web technologies can enhance to an important extent the level

of automation for analyzing business processes. The author

presents a domain-independent ontological framework for

Business Process Analysis (BPA) with support for

automatically computing metrics. In particular, a set of 

ontologies for specifying metrics are defined in this paper. The

domain-independent metrics computation engine is defined

that can interpret and compute them.

Orme et al., [10] described Coupling Metrics for Ontology-

Based Systems. XML has grown to be frequent in Internet-

based application domains such as business-to-business and

business-to-consumer applications. It has moreover produced

a basis for service-oriented architectures such as Web services

and the Semantic Web, mainly because ontology data

employed in the Semantic Web [16, 17] are stored in XML.

Measuring system coupling is a generally accepted software

engineering practice connected with producing high-qualitysoftware products. In many application domains, coupling can

be assessed in ontology-based systems before system

development by measuring coupling in ontology data. A

proposed set of metrics determines coupling of ontology data

in ontology-based systems [22] represented in the Web

Ontology Language (OWL), a derivative of XML.

Andrew et al., [1] define a semiotic metrics suite for

assessing the quality of ontologies. A suite of metrics

proposed here is to assess the quality of the ontology. The

metrics evaluate the syntactic, semantic, pragmatic, and social

aspects of ontology quality according to the semiotic theory.

The author operationalizes the metrics and employs them in a

prototype tool called the Ontology Auditor. A primary

validation of the Ontology Auditor on the DARPA Agent

Markup Language (DAML) library of domain ontologies

represents that the metrics are feasible and highlights the wide

variation in quality between ontologies in the library. The

contribution of the research is to afford a theory-based

framework that developers can utilize to develop high quality

ontologies and that applications can exploit to choose

appropriate ontologies for a given task. Zhe et al., [24]

provides some Evaluation Metrics for Ontology Complexity

and Evolution Analysis.

Ying et al., [12] discusses about semantic web. Presently,

computers are shifting from single, isolated devices into doorpoints to a worldwide network of information exchange and

business transactions called the World Wide Web (WWW).

For this cause, support in data, information, and knowledge

exchange has become a key issue in current computer

technology. The achievement of the WWW has made it

increasingly hard to find, access, present, and maintain the

information required by a wide variety of users. In answer to

this problem, many new research initiatives and commercial

enterprises have been provided to enhance available

information with machine processable semantics. This

semantic web will offer intelligent access to heterogeneous,

distributed information, enabling software products (agents)

[20] to intervene between user needs and the informationsources available. This paper reviews ongoing research in the

area of the semantic web [19], focusing especially on ontology

technology.

Anthony et al., [13, 18] put forward the Complexity and

coupling metrics for ontology based information. Ontologies

are greatly used in bioinformatics and genomics to

characterize the structure of living things. This research

focuses on complexity metrics for ontologies. These

complexity metrics are obtained from semantic relationships

in an ontology. These metrics will assist for selecting the best

(IJCSIS) International Journal of Computer Science and Information Security,

Vol. 8, No. 8, November 2010

183 http://sites.google.com/site/ijcsis/

ISSN 1947-5500

8/8/2019 A Survey on Designing Metrics suite to Asses the Quality of Ontology

http://slidepdf.com/reader/full/a-survey-on-designing-metrics-suite-to-asses-the-quality-of-ontology 6/6

ontologies in several application areas, including

bioinformatics and genomics.

Dimitris N. Kanellopoulos [22] described ODELO: an

ontology-driven model for the evaluation of learning

ontologies. Trying out or renewing an existing learning

ontology [6] and providing evaluation tools to assess its

quality are fundamental steps before putting an e-learning

system online. Ontology [21, 23] evaluation is a central task 

and it is typically the output of an automatic process. Thispaper put forwards an ontology-driven model, called

Ontology-Driven model for the Evaluation of Learning

Ontologies (ODELO), for the estimation of ontologies

representing learning resources with respect to several metrics.

Syntax contracts with the proper relations between signs (e.g.,

words, phrases, sentences) and the construction of new ones.

Social metrics imitate the fact that software agents and

ontologies coexist and communicate in communities.

Ontology cohesion metrics indicates the degree of relatedness

of ontology classes. ODELO is a deductive valuation model

that identifies the elements of ontological quality for learning

ontologies. In this paper the author propose a framework for

assessing the quality of learning ontologies that constitute the

basis for intelligent educational Adaptive Hypermedia (AH)

systems.

III.  CONCLUSION 

In this survey, the different suite of metrics for evaluating

ontologies based upon the semiotic-web is analyzed. It is

better to assess the quality of web-ontology after the design is

completed. This helps in neglecting the low quality ontology

before its development by enhancing those defects in

ontology. This helps in saving of considerable amount of cost

and effort [2] for developing high quality semantic websystems. Several metrics such as Knowledge Enriched metric,

Characteristics Relevancy metric, Domain Modularity metric,

Richness, Instance Metrics, Semantic Similarity Measure,

Density Measure, etc., are analyzed for assessing the quality

of ontology. This survey helps in choosing the best suited

metrics for assessing the quality of ontology. The different

metrics which involve different criteria for ontology are

analyzed in this survey.

REFERENCES 

[1]  Punit Ahluwalia, Andrew Burton-Jones, Veda C. Storey and Vijayan

Sugumaran “A Semiotic Metrics Suite for Assessing the Quality of 

Ontologies”,  journal of Data & Knowledge Engineering,vol 55, pp 84-102, 2005.

[2]  Amjad Farooq, Syed Ahsan, and Abad Shah, “Web-Ontology DesignQuality Metrics”, Journal of American Science, 2010.

[3]  Samir Tartir, I. Budak Arpinar, Michael Moore, Amit P. Sheth and

Boanerges Aleman-Meza, “OntoQA: Metric-Based Ontology Quality

Analysis”, IEEE Workshop on Knowledge Acquisition from Distributed,

Autonomous, Semantically Heterogeneous Data and Knowledge

Sources, 2005.[4]  Werner Ceusters,” Towards a Realism-Based Metric for Quality

Assurance in Ontology Matching”,  Proceeding of the 2006 conference

on Formal Ontology in Information Systems: Proceedings of the FourthInternational Conference (FOIS ) ,pp 321-332, 2006.

[5]  Harith Alani and Christopher Brewster,” Metrics for Ranking

Ontologies”, WWW2006, pp 22–26, 2006.

[6]  Maedche and Steffen Staab"Ontology Learning for the Semantic Web",

IEEE Intelligent Systems, Vol 16, pp 72-79, 2001

[7]  Henry M. Kim," A Measurement Ontology Generalizable for EmergingDomain Applications on the Semantic Web", Journal of Database

Management, Vol 18, pp 20-42, 2006.

[8]  D. Fensel., F.V. Harmelen, I. Horrocks., D.L. McGuinness, and P.F.Patel-Schneider, “Oil: An ontology infrastructure for the semantic web”,

IEEE Intelligent Systems, pp 38-45, 2001.

[9]  Carlos Pedrinaci and John Domingue,”  Ontology-based Metrics

Computation for Business Process Analysis”.

[10]  Orme, Anthony M., Yao, Haining, and Etzkorn, Letha, “Coupling

Metrics for Ontology-Based Systems,” IEEE Software, Vol. 23, No. 2,pp. 102-108, 2006.

[11]  Chen, Y., Zhou, L., & Zhang, D. “Ontology-Supported Web Service

Composition: An Approach to Service-Oriented KnowledgeManagement in Corporate Services.” Journal of Database Management,

17(1),pp 67-84, 2006.

[12]  Ying Ding, Dieter Fensel , Michel Klein and Borys Omelayenko, “Thesemantic web: yet another hip?”, Data & Knowledge Engineering,

Vol.41 n.2-3, pp 205-227, 2002.

[13]  Anthony Mark Orme , Haining Yao and Letha H. Etzkorn , "Complexitymetrics for ontology based information", International Journal of 

Technology Management, 2009.

[14]  Thomas R. Gruber, “A translation approach to portable ontology

specifications”, Knowledge Acquisition, Vol.5 no.2, pp.199-220, 1993.

[15]  Martin Doerr, Jane Hunter and Carl Lagoze,"Towards a Core Ontologyfor Information Integration", Journal of Digital Information, Vol 4,

No.1, 2003.

[16]  Jeff Heflin, James Hendler, “A Portrait of the Semantic Web in Action”,IEEE Intelligent Systems, Vol.16 no.2, pp.54-59, 2001.

[17]  James Hendler, “Agents and the Semantic Web”, IEEE Intelligent

Systems, Vol.16 n.2, pp.30-37, 2001.[18]  Anthony M. Orme, Haining Yao, Letha H. Etzkorn, "Coupling Metrics

for Ontology-Based Systems," IEEE Software, Vol. 23, no. 2, pp. 102-

108, 2006.[19]  Alexander Maedche, Steffen Staab, “Ontology Learning for the

Semantic Web”, IEEE Intelligent Systems, Vol.16 no.2, pp.72-79, 2001.

[20]  Larry M. Stephens, Michael N. Huhns, “Consensus Ontologies:

Reconciling the Semantics of Web Pages and Agents”, IEEE Internet

Computing, Vol.5 n.5, pp.92-95, 2001.

[21]  Vijayan Sugumaran , Veda C. Storey, “Ontologies for conceptualmodeling: their creation, use, and management, Data & Knowledge

Engineering”, Vol.42 no.3, pp.251-271, 2002.[22]  Dimitris N. Kanellopoulos , "ODELO: an ontology-driven model for the

evaluation of learning ontologies",International Journal of Learning

Technology archive ,Vol 4, pp 73-99, 2009.

[23]  Henry M. Kim, Arijit Sengupta and Joerg Evermann, “MOQ: Web

services ontologies for QoS and general quality evaluations”.

[24]  Zhe Yang, Dalu Zhang and Chuan YE, “Evaluation Metrics for

Ontology Complexity and Evolution Analysis”,  ICEBE '06. IEEEInternational Conference, 2006.

[25]  Yinglong Maa, Beihong Jinb and Yulin Fengb, “Semantic oriented

ontology cohesion metrics for ontology-based systems”,  Journal of Systems and Software archive, Vol 83, pp 143-152, 2010. 

(IJCSIS) International Journal of Computer Science and Information Security,

Vol. 8, No. 8, November 2010

184 http://sites.google.com/site/ijcsis/

ISSN 1947-5500