acquisition of axioms in ontology learningarios/docs/seminario/presentacion2... · by contrast, an...

Acquisition of axioms in ontology learning

Ana B. Rios-Alvarado

Information Technology LaboratoryCINVESTAV - Tamaulipas, Mexico

October, 31th 2012

Supervisor:Dr. Ivan Lopez-Arevalo


Outline

1 Introduction

2 The research problem

3 Objectives

4 The proposed approach

5 Experiments and Results

6 Conclusions

7 References

2 / 54


Introduction

Motivation• Ontologies allow the construction of more “intelligent” appli-

cations: semantic search, automated reasoning, among others.

3 / 54


Introduction

Some definitions...

What is an ontology?

• “... an element that defines the basic terms and relationscontained into the vocabulary of a topic area as the rules forcombining terms and relations to define extensions of a con-ceptualization” [Neches et al., 1991]

What is ontology learning?

• “Ontology learning is the set of methods and techniques usedfor building an ontology from scratch, enriching, or adaptingan existing ontology in a semi-automatic fashion using severalknowledge and information sources” [Gomez-Perez, 2003]

4 / 54


Introduction

Some definitions ...

What is ontology learning from text?

• It is the process of deriving high-level concepts and relations aswell as the axioms from unstructured information to form anontology

5 / 54


Introduction

Some definitions ...

What are the elements of an ontology?

Element DescriptionConcepts (classes) Ideas to formalizeTaxonomic relationships Relation is-a or subClassOfNon-taxonomic Interaction between elementsrelationshipsInstances “real-world” objectsAxioms Theorems on relations

to be satisfied by elements

6 / 54


Introduction

Example: Travel ontology

• Taxonomic relation: Beach is a Destination• Non-taxonomic relation: Activity isOfferedAt Destination• Axiom: disjointWith(UrbanArea, RuralArea)• Instance: “Atlanta” is an instance of City

7 / 54


Introduction

Ontology learning from text

The aspects and tasks in ontology learning from text are structuredas a set of layers [Cimiano, 2006]

8 / 54


Introduction

Some techniques for obtaining the vocabulary

• Linguistic analysis

• Statistical analysis• Term weightning TD-IDF• Mutual Information

• Based on WordNet

• Latent Semantic Analysis

• Text clustering

The use of clustering techniques relies on the assumption thatsimilar terms share similar syntactic contexts

9 / 54


Introduction

Some techniques for obtaining taxonomic relations

• Lexical databases (for example, WordNet)

• Linguistic approaches

• Co-ocurrence analysis

• Lexico-syntactic patterns

The lexical patterns occur frequently in many text genders

• Use of web search and Wikipedia as knowledge source

The web search uses as knowledge base the whole Web

10 / 54


Introduction

Some techniques for obtaining taxonomic relationsLexical patterns

• An expression A is a hyponym of an expression B if the meaningof B is part of the meaning of A and A is a subordinate of B.By contrast, an expresion B is a hypernym of A if B includesthe meaning of A and B is a superior to A.

Hearst’s PatternsA, and other BA, or other B

A is a BB, such as AB, including AB, specially A

11 / 54


Introduction

Some techniques for obtaining taxonomic relationsQuerying the Web

Given a query, it is possible to do the analysis of the number ofhits retrieved by a search engine.

12 / 54


Introduction

Some the techniques for automatic axiom extraction

What is an axiom?

Assertions (including rules) in a logical form that together comprisethe overall theory that the ontology describes in its domain of appli-cation.

Axiom DL Syntax ExamplesubClassOf C1 v C2 Human v Animal u Biped

equivalentClass C1 ≡ C2 Man ≡ Human u MaledisjointWith C1 v ¬ C2 Male v ¬ Female

sameIndividualAs {x1} ≡ {x2} {President Bush} ≡ {G. W. Bush}

differentFrom {x1} v ¬ {x2} { john} v ¬ {peter}

13 / 54


Introduction

Some the techniques for automatic axiom extraction

Kind of axioms Description ExampleClass expression Allow relationships subClassOf(Car,CompactCar)

to be establishedbetween class

Object property Characterise and establish subPropertyOf(hasMother,

relationships between hasParent)

object property expressions

Assertion Axioms about sameIndividualAs(President Bush,

individuals G. W. Bush)

• Lexical patterns [Del Vasto Terrientes, et al., 2010]

• Transforming rules [Volker, et al., 2007a] and statistical analysis [Volker,et al., 2007b][Volker and Rudolph, 2008]

• Inductive logic programming [Lisi and Straccia, 2011]

14 / 54


Introduction

Related work

15 / 54


Introduction

Related workOntology learning from text approaches

[Cimiano et al., [Jiang and Tan, [Ochoa et al.,2005] 2010] 2011]

Domain Tourism and Sport Event and Oncology andFinance Terrorism Finance

(Spanish texts)

Phase Technique

Concepts Formal concept Statistical Linguisticanalysis analysis patterns

Taxonomic Formal concepts are Rule based Semanticrelations partially ordered algortihm roles

Non-taxonomic - Rule mining Semanticrelations algorithm roles

Axioms - - -

16 / 54


The research problem

The problem

• The most of learned ontologies are limited to non-taxonomicrelationships

• Limitations in previous works:

• Assignment a label to a group with a semantic context• Dependency on corpus or linguistic databases for a specific

domain• Limited contexts into corpus for the lexical pattern matching• Ontology learning process without the axiom layer• The axiom learning approaches start from seed concepts and

structures well defined

17 / 54


The research problem

Research questions...

• Is it possible to build an ontology from textual resources withhigh level of expressiveness?

• Is it possible to extract the vocabulary and their relationshipsby text clustering and web search?

• Is it possible to extract axioms using natural language process-ing techniques?

Hypothesis

Considering that text clustering can get the vocabulary of a corpus,web search can obtain taxonomic relationships, and natural languageprocessing techniques allows discover axioms, it is possible adapt andintegrate these techniques for ontology learning.

18 / 54


Objectives

Objectives

General objective

Obtain an approach for ontology learning getting the vocabulary,taxonomic relationships, and key axioms from textual resources

Particular objectives

• Obtain a method to get the vocabulary from text using a clusteringalgorithm

• Obtain a method to extract taxonomic relationships between concepts

• Obtain an unsupervised method for axiom learning

19 / 54


The proposed approach


20 / 54



The proposed approach: main tasks

1 Pre-processing• collect corpus, POS tagging, lemmatization, delete stopwords

2 Extract topics• obtain the representation model of input corpus’s content,

feature selection (nouns/verbs)

3 Discovering taxonomic relations• build querys, execute web search

4 Axiom learning• linguistic analysis, named entity recognition

5 Evaluation of the obtained ontology• improvement the obtained ontology, evaluation and

comparison with “gold standard” ontologies

21 / 54



Topic extraction

22 / 54



Topic extraction

This phase involves the construction of a representation model from corpus andthe implementation of clustering algorithm

22 / 54



Topic extraction: the representation modelFor obtaining the vocabulary:

• Using linguistic analysis the pairs <verb, subject> and <verb,object> are considered for building a pair-term matrix

• The Mutual Information is the measure used for the associationstrength between two words, a verb (vi ) and a noun (nj )

mi(vi , nj ) = logp(vi , nj )

p(vi )p(nj )(1)

23 / 54



Topic extractionFor topic extraction:

• Clustering by Committee can assign words to different clustersusing sets of representative elements committees) for discove-ring unambiguous centroids that describe the members of apossible class

• This method only creates clusters of terms, but it does notcreate a hierarchical structure

24 / 54



Discovering taxonomic relations

25 / 54



Discovering taxonomic relations

This phase comprises the identification of hypernymy/hyponymy relationshipsapplying a method that combines lexical patterns and web search

25 / 54



Discovering taxonomic relationsQuerying the Web

For finding hypernym relations between elements in each topic:

• A set of queries is built considering the following:• Lexical patterns + contextual information• Lexical patterns + related information

• Each query set is executed on a web search

• Candidate hypernyms are extracted from the retrieved pages

• A score is computed using:

ScoreCandHypernym =hits(LexicalPattern(term,CandHypernym))

hits(CandHypernym)

• The hypernym with greater score is selected

26 / 54



Discovering taxonomic relationsQuerying the Web

• Using lexical patterns + contextual information

term + lexical pattern + terms with the higher frequencies in theinput corpus

• Example:museum + and other + cash + travel + product

• Using lexical patterns + related information

term + lexical pattern + the most representative terms in theWordNet synset

• Example:museum + and other + collection + object + display

27 / 54



Axiom learning approach

28 / 54




Considerations:

1 High evidence of named entities

28 / 54



Axiom learning approachConsiderations:


2 An instance level and class level into hierarchical structure

28 / 54



Axiom learning approachConsiderations:


2 An instance level and class level into hierarchical structure

3 Contextual information

28 / 54




Definition 1

A class is a set of individuals which share similar characteristics orproperties. It is denoted by C .

Definition 2

An instance x is an object given for a specific class.

Definition 3

A instanceOf relation associates an instance as a member of aspecific class. It is denoted by instanceOf (x ,C ).

Example:• C = “river” and x = “Nile”

• instanceOf(“Nile”, “river”)

29 / 54




Definition 4

Let A and B be classes, B is a subclass of A if all the instances ofB are a subset of A.

Definition 5

Let A and B be classes, A and B are disjoint classes if the set ofinstances of A is different to set of instances of class B.

Definition 6

Let A and B be classes, A and B are equivalent classes if the set ofinstances of A is the same to set of instances of class B.

30 / 54



Axiom learning approachExample of disjoint classes

31 / 54



Axiom learning approachExample of equivalent classes

Case 1:

Case 2:

32 / 54




33 / 54


Experiments and Results

Test Corpus

• Tourism: “Lonely Planet”a

• manually constructed• 1801 HTML files• 96 concepts• 103 taxonomic relations• 278 named entities

ahttp://olc.ijs.si/lpReadme.html

• Sport Events: “SmartWebProject”b

• manually constructed butindependent of the corpus

• 400 files• 235 concepts• 185 taxonomic relations

bhttp://www.dfki.de/sw-lt/olp2 dataset

34 / 54



Evaluation measures

• Precision (P)

P =CorrectlySelectedEntities

TotalSelectedEntities(2)

• Recall (R)

R =CorrectlySelectedEntities

TotalDomainEntities(3)

• F-Measure (F)

F =2 ∗ P ∗ RP + R

(4)

35 / 54



Experiments

Phases evaluted in the experiments:

36 / 54



Phase: Topic extractionObjective: Selection of syntactic-dependency parser

Test corpus: one file sample of “Lonely Planet”

Tool Number of words Multi-wordsMinipar 36 12

StanfordParser 45 0LinkGrammar 52 0

Minipar parser can identify multi-words as:

• Salt Lake City

• Majestic Taj Mahal hotel

• St. Mary Magdeline Church

• geographical region

• olympic games

37 / 54



Phase: Topic extractionObjective: Selection of similarity measure

Test corpus: “Lonely Planet” - 10856 words

Measure Number of groups F MeasureHindle 10 0.1302

Jaccard 11 0.1176PMIa 25 0.1136

Cosine 13 0.1052NGDb 1 0.0555

aPointwise Mutual Information

bNormalized Google Distance

Examples of groups using Hindle measure in CBC algorithm:

• {accomodation, business, traveler, hotel, food}• {July, March, August, week, day, time, year, month}

38 / 54



Phase: Discovering taxonomic relationsObjective: Evaluation the taxonomy relation extraction

Example:

• Query with contextual information:• region + lexical pattern + cash + travel +

product

• Query with related information:• region + lexical pattern + extended + spatial

+ location

39 / 54



Phase: Discovering taxonomic relationsObjective: Evaluation the taxonomy relation extraction

Example:

“gold standard”

learned ontology

The obtained precision is 53.3%

40 / 54



Phase: Axiom learningObjective: Evaluation of “instanceOf” relation by classes

Evaluation of instanceOf relation for the classes: City, Countryand Holiday

• 12 relations of City class

• 35 relations of Country class

• 10 relations ofHoliday class

Class Tool Precision Recall F MeasureCity AlchemyAPI 0.4000 0.5000 0.4444

OpenCalais 0.3529 0.5000 0.4137

Country AlchemyAPI 0.7631 0.8285 0.7945OpenCalais 0.7000 0.8000 0.7486

Holiday AlchemyAPI 0.4285 0.3000 0.3529OpenCalais 0.4000 0.4000 0.4000

41 / 54



Phase: Axiom learningObjective: Evaluation of “subClassOf” relation

Test corpus: 30 files of “Lonely Planet” manually annotatedNER tool: AlchemyAPI

The obtained precision was 70.37%

42 / 54



Phase: Axiom learningObjective: Evaluation of “disjointWith” relation

Test corpus: 30 files of “Lonely Planet” manually annotatedNER tool: AlchemyAPI

The obtained precision was 83.80%43 / 54



Phase: Axiom learningObjective: Evaluation of “equivalentClass” relation

Test corpus: 30 files of “Lonely Planet” manually annotatedNER tool: AlchemyAPI and OpenCalais

The obtained precision was 67.50%

44 / 54



Results

Previous works:

• Cimiano et al., 2005• Learning concept hierarchies from text corpora using formal

concept analysis• Tourism and Finance domain

• Jiang and Tan, 2010• A semantic-based domain ontology learning system from text• Terrorism and Sport event domain

45 / 54



Results

Comparison with Cimiano et al.’s work (“Lonely Planet dataset”):

Precision (%)```````````Element

Author(s)Cimiano et al., 2005 Rios-Alvarado et al., 2012

Concept extraction 29.33 67.00Taxonomic relation 50.00 55.00Axioms:instanceOf - 46.67subClassOf - 70.37disjointWith - 83.80equivalentClass - 92.00

46 / 54



Results

Comparison with Jiang and Tan’s work (“Sport Event dataset”):

Precision (%)```````````Element

Author(s)Jiang and Tan, 2010 Rios-Alvarado et al., 2012

Concept extraction 45.00 34.01

Taxonomic relation 74.00 77.50Non-taxonomic relation 69.40 -

Axioms:instanceOf - 23.52subClassOf - 64.44disjointWith - 85.00equivalentClass - 93.33

47 / 54


Conclusions

Publications

• Ana B. Rios-Alvarado, Ivan Lopez-Arevalo, and Victor Sosa-Sosa. “Discovering hypernyms using linguisticpatterns on web search”. In the Proceedings of International Conference on Next Generation Web ServicesPractices (NWeSP‘11), Salamanca, Spain, October 19-21, 2011, pp. 302-307, IEEE Computer Society, doi:10.1109/NWeSP.2011.6088195.

• Ana B. Rios-Alvarado, Ivan Lopez-Arevalo, and Victor Sosa-Sosa. “Structuring taxonomies by using linguisticpatterns and Wordnet on web search”. In the Proceedings of the International Conference on KnowledgeEngineering and Ontology Development (KEOD 2011), Paris, France, October 26-29, 2011, pp. 273-278.

• Ana B. Rios-Alvarado, Ivan Lopez-Arevalo and Victor Sosa-Sosa. “A taxonomy construction approach su-pported by web content”. In the Proceedings of the 9th International Symposium on Distributed Computingand Artificial Intelligence (DCAI 2012), Salamanca, Spain. March 28-30 2012, LNCS Springer, pp. 461-468.

• Ana B. Rios-Alvarado, Ivan Lopez-Arevalo, and Victor Sosa-Sosa. “An overview on ontology learning me-thods from textual resources towards the acquisition of axioms”, Innovative Ways of Knowledge Represen-tation and Management, Universidad de Medellın, Colombia, 2012. (accepted chapter)

in review ....• Ana Rios-Alvarado, Ivan Lopez-Arevalo, and Victor Sosa-Sosa. “A taxonomy construction approach sup-

ported by web content”. Journal of Information Science and Engineering

48 / 54


Conclusions

Schedule of activities

49 / 54


Conclusions

Conclusions I

• An approach for ontology learning from scratch has been pre-sented, the source of knowledge is a set of textual resources ona specific domain

• Our proposal is inspired by layer cake model and it has beenworked three main layers:• topic extraction• discovering taxonomic relations• learning axioms

• In the topic extraction layer, the text clustering combined withlinguistic analysis is a good technique to obtain the represen-tative vocabulary in a specific domain

50 / 54


Conclusions

Conclusions II

• For the discovering taxonomic relations, the use of additional in-formation and lexical-patterns into query sets executed on websearch showed a good evidence to identify hypernymy/hyponymyrelations

• The method for axiom learning is based on identifying namedentities as class instances and comparing their context for build-ing the taxonomic structure, and so establish axiomatic rela-tions such as:• instanceOf• subClassOf• disjointWith• equivalentClass

• In two corpus, our approach has shown promising results

51 / 54


References

References I

[Neches et al., 1991] Neches, R., Fikes, R., Finin, T., Gruber, T., Patil R., Swartout W. R. “Enabling

technology for knowledge sharing”. AI Magazine, 12, September 1991, 36-56 pp.

[Gomez-Perez, 2003] Gomez-Perez, A., Manzano-Macho, D. “A survey of ontology learning methods and

techniques”. Deliverable 1.5 OntoWeb, Universidad Politecnica de Madrid. 2003.

[Cimiano et al., 2005] Cimiano, P., Hotho, A., Staab, S. “Learning concept hierarchies from text corpora

using formal concept analysis”. Journal Artificial Intelligence Research, 24(1). 2005. 305-339 pp.

[Cimiano, 2006] Cimiano, P. “Ontology Learning and Population from Text: Algorithms, Evaluation and

Applications”, Springer-Verlag New York, Inc., Secaucus, NJ, USA, 2006. pag. 23

[Jiang and Tan, 2010] Jiang, X., Tan, A. “CRCTOL: A semantic-based domain ontology learning system”.

Journal American Society Information Science Technologies, 61(1). 2010. 150-168 pp.http://dx.doi.org/10.1002/asi.v61:1

[Ochoa et al., 2011] Ochoa, J.L., Hernandez-Alcazar, M.L., Valencia-Garcıa, R., and Martınez-Bejar, R. “A

semantic role-based methodology for knowledge acquisition from Spanish documents”. International Journalof the Physical Sciences, 6(7). 2011. 1755-1765 pp.

[Lisi and Straccia, 2011] Lisi, F. A., Straccia, U. “An Inductive Logic Programming Approach to Learning

Inclusion Axioms in Fuzzy Description Logics”. In Proceedings of the 26th Italian Conference onComputational Logic, 810, 2011, 57-71 pp.

52 / 54


References

References II

[Del Vasto Terrientes, et al., 2010] Del Vasto Terrientes, L., Moreno, A., Sanchez A. “Discovery of relation

axioms from the Web”. In Proceedings of the 4th International Conference on Knowledge Science,Engineering and Management, 2010, 222-233 pp.

[Volker, et al., 2007a] Volker, J., Hitzler, P., Cimiano, P. Acquisition of owl dl axioms from lexical resources,

The Semantic Web: Research and Applications, LNCS, 4519, 2007, 670-685 pp.

[Volker, et al., 2007b] Volker, J., Vrandedic, D., Sure, Y., Hotho, A. Learning disjointness. In Proceedings of

the 4th European Conference on the Semantic Web: Research and Applications, LNCS, Springer, 2007,175-189 pp.

[Volker and Rudolph, 2008] Volker, J., Rudolph, S. Lexico-logical acquisition of OWL DL axioms. In

Proceedings of the International Conference on Formal Concept Analysis, LNAI, Springer, 4933, 2008, 62-77pp.

53 / 54

Thanks a lot for your kind attention!

Ana B. [email protected]

acquisition of axioms in ontology learningarios/docs/seminario/presentacion2... · by contrast, an...

Documents