visualizing consensus with online ontologies to support quality in ontology development

Visualizing Consensus with Online Ontologies to Support Quality in Ontology Development

Mathieu d’Aquin and Enrico Motta

Knowledge Media Institute, The Open University,UK

Agreement and Disagreement in Ontologies

• Ontologies are knowledge artifacts, they express opinions and beliefs and contradict each others

• Assessing (dis)agreement in ontologies is very useful to understand how to combine knowledge from different sources

• In [1], we defined measures of agreement, disagreement, consensus and controversy between ontologies and ontological statements

• Here, we show how such measures can be useful in providing an overview of an ontology in development, with respect to the “opinion” of other ontologies on the Web

[1] d’Aquin, M.: Formally measuring agreement and disagreement in ontologies. In: International Conference on Knowledge Capture - K-CAP. (2009)

Agreement and Disagreement - Measures

• Agreement(st, O) [0..1] and Disagreement(st, O) [0..1] where st is a statement <subject, predicate, object> and O is an ontology

• Based on extracting the part of the ontology that expresses a relation between subject and object

• (Dis)agreement between ontologies:

• Global (dis)agreement in a repository

• Consensus:

• Controversy:

What can we do with that?

Consensus and controversy in statements about SeaFood in Watson

Agreements and disagrements between ontologies containing SeaFood in Watson

SeaFood disjointWith Meat

SeaFood subClassOf Meat

Vegan subClass Vegetarian

Agreement, disagreement, consensus and controversy for ontology engineering

• Making use of these measures to support ontology engineering

• By providing an overview of the developed ontology introducing the “perspective of other ontologies”

• I.e., using Watson to measure the global agreement/disagrement and consensus/controversy with online ontologies.

• And show that in an easy to use visualization.

VisualizationWe show the values of the 4 measures wrt to Watson ontologies using

color coding on the edges of a graph representing the statements in the ontology

Node colors correspond to the average color of their incoming and outgoing edges

blue=agreement red=disagreement green=don’t know

bright blue=+consensus bright red=-consensus dark=controversy

Color=agreement/disagreement

Brightness=consensus/controversy

Example: AKT SupportBright blue = high positive consensusExample: Duration subClassOf IntangibleThing

Bright red= low negative consensusExample: Duration subClassOf PhysicalQuantity(Cyc, SUMO and ATO_Ontology disagree as they do not declare such a relation, or even say these two classes disjoint)

Example: Local Drama Ontology

Green = we don’t know as no other ontology declares anything about two entities

Character subClassOf Person

Node color = aggregation of edges colorsExample: Organization where there is agreement (Company is a subClass) and disagreement (Archive is a subClass)

Using consensus to assess an ontology(a new NeOn toolkit plugin

AKT Portal

The brighter the blue the higher the positive consensus (higher agreement)The brighter the red the lower the negative consensus (higher disagreement) Dark = controversy: no clear cut between disagreement and agreement

Example: The statement attached to the class Employee are controversial: some ontologies agree, others disagree (often due to alternative representations of roles)

NeO

n To

olki

t Plu

gin

Conclusion and Future Work

• Visualizing agreement/disagreement and consensus/controversy can be very useful to identify parts of the ontology where a stronger focus is needed in ontology engineering

• Initial results show that, even in well designed ontologies questions can emerge from such visualizations

• The tool need a lot of technical work to make it more usable in terms of:– Performance (the measures are hard to compute and depend on

internet connection)– Interaction with the user (e.g., give better explanations for the

results)• A complete user evaluation showing how this can guide the

“refinement” of an ontology and improve its quality

Thank You!

m.daquin@open.ac.uk

@mdaquin