Ontology Summit 2017:AI, Learning, Reasoning, and Ontologies

Track A: "Using Automation and Machine Learning to Extract Knowledge and Improve Ontologies"

Champion: Gary Berg-CrossOntolog Board Member

1

Feb. 22, 2017 2

Motivation: Knowledge Acquisition/Evolution Bottleneck Context: Building & maintaining knowledge bases & ontologies is hard work

and could use some automated help.

Costly to construct & time-consuming including updates/revisions

Significant coverage of domain is needed

Meaningful and consistent generalizations are required

Perception: Advances in several areas (e.g. NLP, information retrieval, ML, data mining, & knowledge rep) are helping to advance our quest to automate the process of conceptualizing, designing, populating & making sense of ever growing bodies of “Big” digital information, KBs & Ontologies.

Motivation: bring together various researchers to discuss the issues and state of the art.

Ontology Evolution....

Feb. 22, 2017 3

“While there has been some recent work on developing methodologies allowing us to estimate the cost of knowledge engineering projects, it is legitimate to assume that not all the relevant knowledge can be encoded manually. Techniques that can extract and discover knowledge by analyzing human behaviour and data produced as a result thereof can offer an important contribution...”

Foreword by Philipp Cimiano to Perspectives on Ontology Learning*

Jens Lehmann and Johanna Volker (Eds.)

* Ontology learning is a term coined by Alexander Mädche and Steffen Staab in 2001

Feb, 22, 2017 4

KE & OE Resource Bottleneck: Several reasons

4

Domain experts needed for knowledge & knowledge engineers for formalization

Feb. 2017 Ontology Learning 5

Community Inventory of EarthCube Resources for Geosciences Interoperability (CINERGI) Atmosphere & Geo Connections

Soil microbial

communities are an

important component

of ecosystem response

to environmental

change

Abiotic soil properties

are measured by

observing soil moisture,

temperature, pH, and

inorganic N

Gas flux?

Scenario: examine the relationship between soil microbial communities & trace gas fluxes across the soil-atmosphere interface. Collect all co-located measurements of these 2 kinds of data. I want to limit my DB to only those that were collected in the same ecosystem or vegetation type within 1 mile of each other, and within 2 years of each other.

Are there ontologies & other resources that let me do this?

Feb, 22, 2017 6

The objective of this Ontology Summit 2017 track is to:

Explore work that bridges the realm from symbols to concepts & examine the current status/role of automated and machine learning (ML) to support the development and improvement for ontologies - populating them with instantiations of both concepts and relations, often broadly called ontology learning (Lehmann & Volker, 2014).

Machine learning (ML) technology, both symbolic and now non-symbolic, seems to be advancing rapidly as a diverse and interdisciplinary research field. What are the major challenges in extracting, and developing knowledge bases/ontologies from a variety of forms?

Can we, for example, in the face of big, but noisy data, expect that machine learning will increasingly be employed for building axiom rich ontologies of quality as well as adding light semantics for such related things as metadata annotation

Can we harmonize rationalized ontologies with data-driven concept heterogeneity reflecting the “idiosyncrasies” of particular datasets and vocabularies?

Mission Statement

Feb. 22, 2017 7

Approach / Track Plan We have enlisting a variety of practitioners to discuss ontological and

application issues and problems, and present their efforts and experiences and to stimulate forum discussion within the broader Ontology Summit community.

We reference/build on past Ontology Summits (for example, session or reuse and bottlenecks) as well as connect to other tracks as part of the Summit.

Promote discussion of track session topics on the Ontolog/Summit forum both before and after sessions and leading up to the face-to-face meeting

Work with our speakers and the attending community to distill the virtual meeting topics to a useful integration and set up material for the face-to-face Symposium and Communique.

Help draft material for the final communique.

Feb. 22, 2017 8

Sessions Plan

Overview: March 1 Paul Buitelaar (National University of Ireland, Galway (NUIG), vice-director of the Insight Centre for Data Analytics at NUIG and head of the Insight Unit for Natural Language Processing) Ontology Learning - Use of mapping rules...

OntoLT is a plug-in for Protégé with which concepts (Protégé classes) and relations (Protégé slots) can be extracted automatically from a linguistically annotated text collection. For this purpose, OntoLT provides mapping rules, defined by use of a precondition language, that allow for a mapping between linguistic entities in text and class/slot candidates in Protégé.

Feb. 22, 2017 9

● Estevam Hruschka (Associate Professor at Federal University of Sao Carlos DC-UFSCar. Adjunct Professor at Carnegie Mellon University) Never-Ending Language Learning (NELL) and learning from text. Example: Toward an architecture for never-ending language learning. In Proceedings of the 24th Conference on Artificial Intelligence(AAAI), 2010.

● Alessandro Oltramari (Research Scientist at Bosch, working on intelligent systems for the Internet of Things) - "From machines that learn to machines that know: the role of ontologies in machine intelligence"

● 3rd TBD.....

Session 1 – 8 March 2017 Speakers & Plan:

http://www.slideshare.net/AmazonWebServices/bdt311-deep-learning-going-beyond-machine-learning

Forum/Email conversations to flesh out possible alternative approaches

Feb. 22, 2017 10

Session 2 5th April 2017 Speakers & Plan

•Michael Yu (UCSD) - "Inferring the hierarchical structure and function of a cell from millions of biological measurements". 

•Francesco Corcoglioniti (Fondazione Bruno Kessler)- “Rule-based ontology population process: Frame-Based Ontology Population with PIKES”.

•Evangelos Pafilis (Hellenic Center Marine Research [HCMR]) - “EXTRACT: interactive extraction of environment metadata and term suggestion for metagenomic sample annotation”.

Feb. 22, 2017 11

Paul Buitelaar, Philipp Cimiano & Bernardo Magnini (Editors)

Information Sources:● Relevant text (Web documents

mainly) ● Web document schemata (XML,

DTD, RDF) ● Ontologies● Databases on the Web

Dictionaries ● Semi-structured documents

Wikis, …..

Feb. 22, 2017 12

Scoping 6-8 Ontology Learning/Automation Tasks: A Layer Cake of Ontological Primitives (Buitelaar, Cimiano)

Learn concepts & relations from text etc. Select text fragments and assign them to an ontological concept...learn appropriate domain and ranges...associating terms, construct hierarchies & label relations....from Buitelaar, P., Cimiano, P., & Magnini, B. (2005). Ontology learning from text:An overview. In Buitelaar, P., Cimmiano, P., and Magnini, B., editors, Ontology Learning from Text: Methods, Evaluation and Applications. IOS Press.

Organizing, inductive techniques

HC

set of non-taxonomic relations — R

HR

A

Strings for lexical entries L

Feb. 22, 2017 13

What Methods are Used to Learn “Ontologies”? 1. Natural Language Processing2. Dictionary Parsing 3. Statistical Analysis: extract rules and patterns from Big Data in a supervised or

unsupervised manner based on extensive statistical analysis– Term Weighting (term frequency–inverse document frequency - TF-IDF) – Co-occurrence analysis (Common clustering method applied in Text Mining)...

4. Machine Learning & Hybrid– Ex; tries to find an optimal combination of techniques using supervised ML

5. Hierarchical Concept Clustering 6. Formal Concept Analysis (Lattices) ….

Examples of Linguistic Methods: Simple Part-of-speech tagging: Identify syntactic class (Intension, instance)

Ex: Noun -> Class, Verb -> Relation Stemming Ex: (ize/ization/ized/izing) Head-modifier analysis

Ex: radical ion, the catalyst of the reaction Grammatical function analysis • Ex: “explosion during heating glacial acetic acid treated with chromium trioxide” ->

cause (heat,explosion)

“Ontology Learning” Briefing by Ícaro Medeiros (2009)http://www.slideshare.net/icaromedeiros/slidesontolearning

Feb 22, 2017 Ontology Learning 14

Concept LearningJens LEHMANN. Nicola FANIZZI, Lorenz BÜHMANN and

Claudia D’AMATO

● Strategy: adopt algorithms from inductive logic programming (basics like downward/upward refinement operators -searching a space for a solution [D] starting from the approximation [D'], decision trees and information gain can suggest new intermediate concepts not defined in a starter ontology.)

● Relevant Operator factors considered: completeness, weak completeness, properness, finiteness, & non-redundancy

● Reduce the effort of a domain expert

● i.e. she labels individual resources as instances of the target concept.

● From those labels, axioms can be induced, which can then be confirmed by the knowledge engineer.

Feb. 22, 2017 15

NELL: Never-Ending Language Learner (2014)Semi-Supervised Bootstrap Learning

Inputs:• initial ontology (categories and relations)• dozen examples of each ontology predicate• the web• occasional interaction with human trainers

The task: run 24x7, forever

• each day:1. extract more facts from the web to populate the ontology2. learn to read (perform #1) better than yesterday

Result:• knowledge base with 90+ million candidate beliefs• learning to read• learning to reason• extending ontology

politician of arg1live in arg1

Feb. 22, 2017 16

Various Extraction Tools & Methods have been developed. For example, OntoLearn http://lcl.uniroma1.it/tools.jsp

The OntoLearn methodology is composed of four basic steps:1. discovering new terms, 2.finding definitions, 3.extraction of taxonomic information, and 4. ontology updating.

Human specialists may review the output after each of these steps, or at the end.

OntoLearn has been successfully experimented in several domains (art, tourism, economy and finance, web learning, interoperability).

Feb 22, 2017 Ontology Learning 17

Example: Knowledge Extraction based on Discourse Representation Theory and Linguistic Frames

Valentina Presutti, Francesco Draicchio, & Aldo Gangemi (STLab)

They have implemented a novel approach for robust ontology design from natural language texts by combining:

Discourse Representation Theory (DRT), linguistic frame semantics, and ontology design patterns.

They argue that DRT-based frame detection is feasible by conducting a comparative evaluation of their approach and existing tools.

Semantic Technology Laboratory

http://wit.istc.cnr.it/stlab-tools/

Feb 22, 2017 Ontology Learning 18

Limited quality: Automatically learned ontologies might be noisy, non-perfect, incomplete or even logically inconsistent. Are these ontologies useful for just confusing?

Do we have a proper Ontology Framework within which to work?Standard ontology definition (including L; C; HC ; R; HR; A; F; G)

Extraction:● Things like Relation Extraction are open research problem. (Karoui L. 2007).

● A trend is to combine several different approaches as a verb centered● method, lexical analyses, syntactic and statistic ones, or machine learning to achieve

higher precision, recall, or F-measure. Machine Learning● Learning axioms and rules is difficult and less often performed

● notion of axiom impact is used to determine a beneficial order of axiom evaluation in order to further increase the effectiveness of ontology revision (Nikitina et al, 2012)

Ontology Evolution:● Ontology enrichment: Need selective attention to fill gaps while extending ontology through the addition

of new concepts, relations and rules.● remains a semi-automatic procedure (Petasis et al.,2007).

● Consistency checking and elimination of contracting information.● maximal consistent sub-ontology or the minimal inconsistent sub-ontology

● Evaluation according to some predefined criteria.● Lexical, structural, task functional etc. (after (Gangemi 2006)

● Ontology Population: Eliminating Redundancy and entity disambiguation are main problems....

Examples of Issues

Feb. 22, 2017 19

ReferencesBuitelaar, Paul, Philipp Cimiano, and Bernardo Magnini. "Ontology learning from text: An overview." Ontology learning from text: Methods, evaluation and applications 123 (2005): 3-12.

Cimiano, P, Ontology Learning and Population from Text: Algorithms, Evaluation and Applications, Springer, 2006.

Cimiano, P. and Völker, J. (2005). Text2onto - a framework for ontology learning and data-driven change discovery. In Montoyo, A., Munoz, R., and Metais, E., editors, Proceedings of the 10th International Conference on Applications of Natural Language to Information Systems (NLDB), volume 3513 of Lecture Notes in Computer Science, pages 227–238, Alicante, Spain.

Gangemi, A., et al. (2006). Modelling ontology evaluation and validation. In The Semantic Web:

Research and Applications, pp. 140–154.

Gómez-Pérez, Asunción, and David Manzano-Macho. "An overview of methods and tools for ontology learning from texts." The knowledge engineering review 19.03 (2004): 187-212.

Haase, P. and Völker, J. (2005) Ontology learning and reasoning - dealing with uncertainty and inconsistency. In In Proceedings of the Workshop on Uncertainty Reasoning for the Semantic Web (URSW, pages 45–55

Ivanova, T. "Ontology Learning Technologies-Brief survey, trends and problems." proceedings of the International Conference on Information Technologies. 2012.

19

Feb. 22, 2017 20

● Karoui L. et al, (2007). Analyses and Fundamental ideas for a Relation Extraction Approach.Proceedings of the 2007 IEEE 23rd International Conference on Data EngineeringWorkshop, pp. 880-887.

● Lehmann, Jens, and Johanna Voelker. "An introduction to ontology learning." Perspectives on ontology learning. AKA/IOS Press, Heidelberg (2014): 9-16.

● Nikitina, Nadeschda, Sebastian Rudolph, & Birte Glimm. "Interactive ontology revision." Web Semantics: Science, Services & Agents on the World Wide Web 12 (2012): 118-130.

● Maedche, Alexander, and Steffen Staab. "Discovering conceptual relations from text." Ecai. Vol. 321. No. 325. 2000.

● Maedche, Alexander, and Steffen Staab. "Semi-automatic engineering of ontologies from text." Proceedings of the 12th international conference on software engineering and knowledge engineering. 2000.

● Maedche, Alexander. Ontology learning for the semantic web. Vol. 665. Springer Science & Business Media, 2012.

● Petasis G., et al. (2007). D4.3: Ontology Population and Enrichment: State of the Art. BOEMIE Project FP-027538 .

● Simperl, Elena , Tobias Buerger, Simon Hangl, Stephan Woelger, and Igor Popov. Ontocom: A reliable cost estimation method for ontology development projects. Web Semantics: Science, Services and Agents on the World Wide Web, 16(0):1 – 16, 2012.

References- Cont

Feb. 22, 2017 21

● Springer. Faure, D. and Edellec, C. N. (1998). A corpus-based conceptual clustering method for verb frames and ontology acquisition. In In LREC workshop on, pages 5–12.

● Wong, Wilson, Wei Liu, and Mohammed Bennamoun. "Ontology learning from text: A look back and into the future." ACM Computing Surveys (CSUR) 44.4 (2012): 20.

References- Cont

Feb. 22, 2017 22

Feb 22, 2017 Ontology Learning 23

Refinement OperatorsThe solution of the learning problem stated above can be cast as a search for a correct

concept definition in an ordered space (,). In such a setting, one can define suitable

operators to traverse the search space. Refinement operators can be formally defined as: