A Survey on Designing Metrics suite to Asses the Quality of Ontology

Download A Survey on Designing Metrics suite to Asses the Quality of Ontology

Post on 09-Apr-2018




0 download

Embed Size (px)


<ul><li><p>8/8/2019 A Survey on Designing Metrics suite to Asses the Quality of Ontology</p><p> 1/6</p><p>Abstract---With the persistent growth of the World Wide Web,the difficulty is increased in the retrieval of relevant information for a</p><p>users query. Present search engines offer the user with several web</p><p>pages, but different levels of relevancy. To overcome this, the</p><p>Semantic Web has been proposed by various authors to retrieve and</p><p>utilize additional semantic information from the web. As the</p><p>Semantic Web adds importance for sharing knowledge on the internet</p><p>this has guide to the development and publishing of several</p><p>ontologies in different domains. Using the database terminology, it</p><p>can be said that the web-ontology of a semantic web system is</p><p>schema of that system. As web ontology is an integral aspect of</p><p>semantic web systems, hence, design quality of a semantic web</p><p>system can be deliberated by measuring the quality of its web-</p><p>ontology. This survey focuses on developing good ontologies. This</p><p>survey draws upon semiotic theory to develop a suite of metrics that</p><p>assess the syntactic, semantic, pragmatic, and social aspects of</p><p>ontology quality. This research deliberates about the metrics that may</p><p>contribute in developing a high quality semantic web system.</p><p>Keywords--- Quality Metrics, Web ontology, Semiotic Metrics,</p><p>Semantic Quality, Domain modularity.</p><p>I. INTRODUCTIONEMANTIC Web is nothing but the extension of the</p><p>present web in which the web resources are prepared with</p><p>formal semantics about their interpretation for the machines.</p><p>These web resources are combined in the form of web</p><p>information systems, and their formal semantics are usually</p><p>characterized in the form of web-ontologies. By means of the</p><p>database terminology, it can be said that the web-ontology of a</p><p>semantic web system is representation of that system [11].</p><p>Design quality of a semantic web system can be calculated by</p><p>computing the quality of its web-ontology because web</p><p>ontology is the integral element of semantic web systems [25].</p><p>The main concern is that when the design of a web-ontology is</p><p>completed, it is suitable time to assess its quality so that in</p><p>case, the design is of low quality, it can be enhanced before its</p><p>instantiation. This helps in saving of considerable amount of</p><p>cost and effort for developing high quality semantic web</p><p>systems. Metrics are considered as the appropriate tools for</p><p>estimating quality. This survey focuses on several metrics for</p><p>web ontology quality evaluation.</p><p>II. LITERATURE SURVEYAhluwalia et al., [1]presenteda Semiotic Metrics Suite for</p><p>Assessing the Quality of Ontologies. Table 1 shows some of</p><p>the metrics for quality evaluation [1, 3].</p><p>As a decisive construct, overall quality (Q) is a subjective</p><p>function of its syntactic (S), semantic (E), pragmatic (P), and</p><p>social (O) qualities [1] (i.e., Q = b1S + b2E + b3P +</p><p>b4O). The addition of weight is equal to 1. In the absence of</p><p>pre-specified weights, the weights are assigned to be equal.</p><p>Syntactic Quality (S) evaluates the quality of the ontology</p><p>according to the way it is written. Lawfulness is the extent to</p><p>which an ontology languages rules have been obeyed. Not</p><p>every ontology editors have error-checking capabilities;</p><p>however, without correct syntax, the ontology cannot be read</p><p>and used. Richness is nothing but the proportion of features in</p><p>the ontology language that have been used in ontology (e.g.,</p><p>whether it includes terms and axioms, or only terms). Richer</p><p>ontologies are more valuable to the user (e.g., agent).</p><p>Semantic Quality (E) estimates the meaning of terms in the</p><p>ontology library. Three attributes are used here are</p><p>interpretability, consistency, and clarity. Interpretability dealswith the meaning of terms (e.g., classes and properties) in the</p><p>ontology. In the real world, the knowledge provided by the</p><p>ontology can map into meaningful concepts. This is</p><p>accomplished by checking that the words used by the ontology</p><p>be present in another independent semantic source, such as a</p><p>domain-specific lexical database or a comprehensive, generic</p><p>lexical database such as WordNet. Consistency is nothing but</p><p>whether terms having a consistent meaning in the ontology.</p><p>For example, if an ontology claims that X is a subclass of Y,</p><p>and that Y is a property of X, then X and Y have incoherent</p><p>meanings and are of no semantic value. For example,</p><p>ontological terms such as IS-A is often used inconsistently.</p><p>Clarity is the term which determines whether the context ofterms is clear. For example, if ontology claims that class</p><p>Chair has the property Salary, an agent must know that</p><p>this illustrate academics, not furniture.</p><p>Pragmatic Quality (P) deals with the ontologys usefulness</p><p>for users or their agents, irrespective of syntax or semantics.</p><p>Three criteria are used for determining P. Accuracy is whether</p><p>the claims on ontology makes are true. This is very tricky to</p><p>determine automatically without a learning mechanism or</p><p>truth maintenance system. Currently, a domain expert</p><p>evaluates accuracy. The measure of the size of the ontology is</p><p>A Survey on Designing Metrics suite to Asses the</p><p>Quality of OntologyK.R Uthayan G.S.Anandha Mala, Professor &amp; Head,</p><p>Department of Information Technology, Department of Computer Science &amp; Engineering</p><p>SSN College of Engineering St.Josephs College of Engineering,Chennai, India Chennai, India</p><p>uthayankr@yahoo.com gs.anandhamala@gmail.com</p><p>S</p><p>(IJCSIS) International Journal of Computer Science and Information Security,</p><p>Vol. 8, No. 8, November 2010</p><p>179 http://sites.google.com/site/ijcsis/ISSN 1947-5500</p></li><li><p>8/8/2019 A Survey on Designing Metrics suite to Asses the Quality of Ontology</p><p> 2/6</p><p>called as Comprehensiveness. Larger ontologies are more</p><p>probable to be complete representations of their domains, and</p><p>provide more knowledge to the agent. Relevance indicates</p><p>whether the ontology satisfies the agents specific</p><p>requirements.</p><p>TABLE 1:DETERMINATION OF METRIC VALUES</p><p>Attributes Determination</p><p>Overall Quality (Q) Q = b1.S + b2.E + b3.P + b4.O</p><p>Syntactic Quality (S) S = bs1.SL + bs2.SR</p><p>Lawfulness (SL)Let X be total syntactical rules. Let Xb be total breached rules. Let NS</p><p>be the number of statements in the ontology. Then SL = Xb / NS.</p><p>Richness (SR)</p><p>Let Y be the total syntactical features available in ontology language.</p><p>Let Z be the total syntactical features used in this ontology.</p><p>Then SR = Z/Y.</p><p>Semantic Quality (E) E = be1.EI + be2.EC + be3.EA</p><p>Interpretability (EI)</p><p>Let C be the total number of terms used to define classes and properties</p><p>in ontology.Let W be the number of terms that have a sense listed in WordNet. Then</p><p>EI = W/C.</p><p>Consistency (EC)</p><p>Let I = 0. Let C be the number of classes and properties in ontology.</p><p>Ci, if meaning in ontology is inconsistent, I+1. Therefore, I = number</p><p>of terms with inconsistent meaning. Ec = I/C.</p><p>Clarity (EA)Let Ci = name of class or property in ontology. Ci, count Ai, (the</p><p>number of word senses for that term in WordNet). Then EA = A/C.</p><p>Pragmatic Quality (P) P = bp1.PO + bp2.PU + bp3.PR</p><p>Comprehensiveness (PO)Let C be the total number of classes and properties in ontology. Let V</p><p>be the average value for C across entire library. Then PO = C/V.</p><p>Accuracy (PU)</p><p>Let NS be the number of statements in ontology. Let F be the number of</p><p>false statements. PU = F/NS. Requires evaluation by domain expert</p><p>and/or truth maintenance system.</p><p>Relevance (PR)</p><p>Let NS be the number of statements in the ontology. Let S be the type of</p><p>syntax relevant to agent. Let R be the number of statements within NS</p><p>that use S. PR = R / NS.</p><p>Social Quality (O) O = bo1.OT + bo2.OH</p><p>Authority (OT)</p><p>Let an ontology in the library be OA. Let the set of other ontologies in</p><p>the library be L. Let the total number of links from ontologies in L to</p><p>OA be K. Let the average value for K across ontology library be V.</p><p>Then OT = K/V.</p><p>History (OH) Let the total number of accesses to an ontology be A. Let the averagevalue for A across ontology library be H. Then OH = A/H.</p><p>Cohesion (Coh)Coh=|SCC|</p><p>Where SCC is separate connected components</p><p>Fullness (F) Readability (Rd) </p><p>For the purpose of evaluation, it needs some knowledge of</p><p>the agents requirements. This metric is coarse as it verifies for</p><p>the type of information the agent uses by ontology (e.g.,</p><p>property, subclass, etc), rather than the semantics needed for</p><p>(IJCSIS) International Journal of Computer Science and Information Security,</p><p>Vol. 8, No. 8, November 2010</p><p>180 http://sites.google.com/site/ijcsis/</p><p>ISSN 1947-5500</p></li><li><p>8/8/2019 A Survey on Designing Metrics suite to Asses the Quality of Ontology</p><p> 3/6</p><p>specific tasks (e.g., the particular subclasses needed to</p><p>interpret a users specific query).</p><p>Social quality (O) imitates the fact that agents and</p><p>ontologies exist in communities. The authority of an ontology</p><p>is nothing but the number of other ontologies that link to it</p><p>(define their terms using its definitions). More authoritative</p><p>ontologies indicate that the knowledge they provide is</p><p>accurate or useful. The history indicates the number of times</p><p>the ontology is accessed. Ontologies are more dependablewhen they are with longer histories.</p><p>The cohesion (Coh) of a KB is nothing but the number of</p><p>separate connected components (SCC) of the graph</p><p>representing the KB.</p><p>The fullness (F) of a class Ci is defined as the actual number</p><p>of instances that belong to the subtree rooted at Ci (Ci(I))</p><p>compared to the expected number of instances that belong to</p><p>the subtree rooted at Ci (Ci`(I)).</p><p>The readability (Rd) of a class C i is defined as the total of</p><p>the number attributes that are comments and the number of</p><p>attributes that are labels the class has.</p><p>Amjad et al., [2] provided the Web-Ontology Design</p><p>Quality Metrics. The author proposes design metrics for web-</p><p>ontology [21] by maintaining certain recommended principles</p><p>like a metric may reach its highest value for perfect quality for</p><p>excellent case and vice versa that is it may reach its lowest</p><p>level for worst case. It is supposed to be monotonic, clear, and</p><p>intuitive. It must correlate well with human decisions and it</p><p>should be automated if possible. The proposed metrics may</p><p>give notification about how much knowledge can be derived</p><p>from a given webontology; how much it is relevant to a users</p><p>specific necessities and how much it is effortless to reuse,</p><p>manage, trace and adapt. The metrics provided by the author</p><p>are Knowledge Enriched (KnE), Characteristics Relevancy</p><p>(ChR) and Domains modularity (DoM).</p><p>Knowledge Enriched metric</p><p>The reasoning capability of a web-ontology is determined</p><p>by Knowledge Enriched (KnE) metric, and it is based on two</p><p>sub-metrics so-called Isolated Axiom Enriched (IAE) metric</p><p>and Overlapped Axiom Enriched (OAE) metric. There are</p><p>three parts in this axiom namely, predicate, resource and</p><p>object. If none of these is similar with any other axiom of</p><p>identical domain then that axiom is termed as isolated axiom.</p><p>If the two axioms have some similar parts, it is said to be</p><p>overlapped. There may be more than a few transitively</p><p>overlapped axioms in any domain. This metric determines the</p><p>percentage of IAE and OAE, and if the former is greater thanthe later one, then the web-ontology can be regarded as less</p><p>knowledge enriched. IAE is officially defined as the ratio of</p><p>total number of isolated axioms (tIAs) to the total number of</p><p>domain axioms (tDAs).</p><p> (1)</p><p>In the above equation, n is total number of sub-domains of</p><p>web-ontology. Similarly, the OAE metric is officially defined</p><p>as ratio of total number of overlapped axioms (tOAs) to the</p><p>total number of domain axioms. It can be written as follows:</p><p> (2)</p><p>In the equation given above, n is total number of sub-</p><p>domains of web-ontology. Lastly, the KnE metric is the</p><p>difference of total number of overlapped axioms and the total</p><p>number of isolated axioms. It may be written as follows:</p><p>(3)</p><p>If the resultant KnE value is positive, then the web-ontology</p><p>is more knowledge enriched, if it is zero, then the web-</p><p>ontology is average knowledge enriched, and if it is negative,then the web-ontology is less knowledge enriched.</p><p>Characteristics Relevancy metric</p><p>Characteristics Relevancy (ChR) metric gives us the</p><p>suggestion about how much a given web-ontology is close to a</p><p>users specific necessities and the degree of reusability of the</p><p>web-ontology. Formally, it is termed as the ratio of the</p><p>number of relevant attributes (nRAs) in a class to the total</p><p>number of attributes (TnAs) of that class. It can be written as</p><p>follows:</p><p> (4)</p><p>where n in above equation represents the total number of</p><p>classes in the provided web-ontology. ChR metric reveals the</p><p>proportion of relevant attributes in the web-ontology, and this</p><p>number gives insights how much a web-ontology is relevant.</p><p>Domain Modularity metric</p><p>Domain modularity (DoM) metric denotes the component-</p><p>orientation feature of a web-ontology. This metric specifies</p><p>the grouping of knowledge in different components of web-ontology. The webontology is best manageable, traceable,</p><p>reusable and adaptable, if it is designed in components</p><p>(subdomains). Formally, the DoM metric is given as the</p><p>number of sub-domains (NSD) contained in a webontology.</p><p>This metric also depends on the coupling and cohesion [25]</p><p>levels of sub-domains, and it is directly proportional to its</p><p>cohesion level and inversely proportional to its coupling level.</p><p>(IJCSIS) International Journal of Computer Science and Information Security,</p><p>Vol. 8, No. 8, November 2010</p><p>181 http://sites.google.com/site/ijcsis/</p><p>ISSN 1947-5500</p></li><li><p>8/8/2019 A Survey on Designing Metrics suite to Asses the Quality of Ontology</p><p> 4/6</p><p>(5)</p><p>In the above equation, DCoh indicates the level of domain</p><p>cohesion and DCoup represents the level of coupling among</p><p>sub-domains of web-ontology domain. DoM metric is a realnumber indicating the degree of partial reusability of a given</p><p>web-ontology.</p><p>Samir et al., [3] given the OntoQA: Metric-Based Ontology</p><p>Quality Analysis. The metrics presented can highlight key</p><p>characteristics of an ontology schema and also its population</p><p>and facilitate users to make an informed judgment easily. The</p><p>metrics used by the author here are not 'gold standard'</p><p>measures of ontologies. Instead, the metrics are projected to</p><p>estimate several aspects of ontologies and their potential for</p><p>knowledge representation. Rather than describing ontology as</p><p>merely effective or ineffective, metrics describe a certain</p><p>aspect of the ontology because, in most cases, the way the</p><p>ontology is built is largely dependent on the domain in whichit is designed. The metrics defined here are Schema Metrics</p><p>and Instance Metrics. The following are metrics considered by</p><p>the author:</p><p>The following are some of Schema Metrics:</p><p> Relationship Richness: The diversity of relations and</p><p>placement of relations in the ontology is defined by this</p><p>metrics. An ontology that has many relations further than</p><p>class-subclass relations is better than taxonomy with no more</p><p>than class-subclass relationships. The relationship richness</p><p>(RR) is defined as the ratio of the number of relationships (P)</p><p>defined in the schema to the sum of the number of subclasses</p><p>(SC) plus the numbe...</p></li></ul>