Change paths in reasoning!

Raphael VolzFZI Forschungszentrum Informatik

Universität Karlsruhe (TH)Karlsruhe, Germany

11.11.2007

Proposition

1. We need a consensus benchmark

2. Approximate is better than nothing

3. Tractable languages make speed

4. Incremental reasoning is smart

We need a consensus benchmark (1)Recent Performance Benchmark @ FZI

Note: Joint work with Jürgen Bock, Qiu Ji, Peter Haase, Pellet performance close to Racer, Sesame preformance close to OWLIM

Is this evaluation

representative???

We need a consensus benchmark (2)

Shortcomings of various benchmarks

Source: Timo Weithörner et al., What‘s wrong with owl benchmarks, Proc. of 2nd int. Workshop on Scalable Semantic Web Knowledge Base Systems (SSWS 2006)

Need for a consensus benchmark (3)Improvements Achieved with TREC

Source: E. M. Voorhees, TREC: Improving Information Access through Evaluation,American Society for Information Science and Technology   Vol. 32, No. 1  Oct/Nov 2005

ConsensusMetric

ConsensusData Sets

Approximate is better than nothing (1)

Ontologies in WATSON Corpus

Source: Mathieu d‘Acquin et al., Characterizing Knowledge on the Semantic Web with Watson,EON 2007 Workshop, Busan, Korea

We need to approximatewith OWL DL

reasoners

Approximate is better than nothing (2)

Idea derived from FOL work

Source: Hitzler and Vrandecic, Resolution-based approximate reasoning for OWL DL, in Y. Gil et al. (Eds.): Proc. of ISWC 2005, LNCS 3729, pp. 383–397, 2005.

Approximate is better than nothing (3)

Idea derived from FOL work

Source: Hitzler and Vrandecic, Resolution-based approximate reasoning for OWL DL, in Y. Gil et al. (Eds.): Proc. of ISWC 2005, LNCS 3729, pp. 383–397, 2005.

Tractable languages make speed (1)

Dez 2003 - DAML.ORG Corpus

Source: Mathieu d‘Acquin et al., Characterizing Knowledge on the Semantic Web with Watson,EON 2007 Workshop, Busan, Korea

Jul 2007 - WATSON Corpus

Source: Raphael Volz, Web Ontology Reasoning with logic databases, dissertation, university of karlsruhe, 2004

Tractable Languages dominate (and will continue to do so)

Tractable languages make speed (2)

Source: Markus Krötzsch, Sebastian Rudolph, and Pascal Hitzler; Complexity of Horn Description LogicsTechnical Report, Institute AIFB, University of Karlsruhe, 2007

Combined complexity of various DLs

11

1

1

3

#

AvailableReasoners

(known to me)

Incremental reasoning is smart (1)

Incremental answers is what we expect on the web

Standardexpectation

for queryanswering on

the web

Incremental reasoning is smart (2)

Incremental reasoning saves work

Alternate possible interpretations of incremental reasoning

1. Maintain state information from previous reasoning cycles when dealing with change to the KB (Pellet interpretation)

2. Provide answers / results incrementally (anytime behaviour)

Source: C. Halaschek-Wiener et al. Description Logic Reasoning for Dynamic A-Boxes, 2006 DL Workshop, CEUR WS 189

The ideal Semantic WEB WEB reasoning approach

time

Qualityas a combination

of soundness and completness

Functional Qualities• Measurable quality• Recognizable quality• Monotonicity• Consistency• Diminishing returns• Interruptability• Preemtability

Source: Shlomo Zilberstein, Using Anytime Algorithms in Intelligent Systems, AI Magazine, Fall 1996

HAPPY TODISCUSS

WITH YOU !!!