December 2, 2013Thessaloniki, Greece

GNORASI WORKSHOP

Charalampos Doulaverakis

CERTH/ITI

Knowledge and processing algorithms for remote sensing data

Reasoning and semantic interpretation of visual data in GNORASI

2

Goals of semantic interpretation

Reasoning and representation of knowledge for semantic-enabled image analysis Expert knowledge and visual information

processing data are represented through ontologies

Development of a reasoning process for the knowledge assisted interpretation of images

Reasoning methods Fuzzy inference support

3

Definition on ontology Representation and querying Land use/Land cover ontologies

Ontologies and reasoning

4

Ontology definition

An ontology defines a set of representational primitives with which to model a domain of knowledge or discourse (Gruber, 1995) Classes: which represent a set of objects Properties: which express attributes and

relations between classes and objects Constraints: for expressing logical

consistency Individuals: atoms (objects) which are

members of a class Abstraction level of data models

Analogous to hierarchical and relational models

Ontologies, through inference, can provide us with implicitly defined information Reasoning engines

Thomas R. Gruber. Toward Principles for the Design of Ontologies Used for Knowledge Sharing. International Journal of Human-Computer Studies, vol. 43, no. 5-6, pages 907-928, 1995

5

Ontology languages

Several languages have been proposed RDF(S)

RDF: Uses URI for expressing relationships between objects (triples) RDF: Allows structured and semi-structured data to be mixed,

exposed, and shared across different applications. RDFS: Allows the definition of classes, the relations between them and

semantic constraints on RDF OWL

Based on Description Logics. Designed to represent rich and complex knowledge 3 types of increasing expressivity

Lite, DL, Full

OWL2 Logical extension of OWL Deals with weaknesses in OWL expression and integrates features

requested by users 3 variations

EL, QL, RL

6

Rule languages

They add expressive extensions to ontology languages E.g: SandArea(?x), SeaArea(?y),

isAdjacent(?x,?y)-> Beach(?x) Standard languages have been defined

RuleML, SWRL Other rule languages are offered by

reasoning engines Such as Jena, OWLIM, Pellet, Hermit, …

7

Query languages

Query languages have been proposed for retrieving information from ontology repositories SPARQL, SeRQL, RDQL

Most have similarities with SQL SELECT * WHERE {?X rdf:type gn:LandCover}

Retrieve all instances of class gn:LandCover (SPARQL) Query languages can be used for deriving new facts

CONSTRUCT {?region gn:depicts gn:Vegetation} WHERE {

?region gn:hasNDVI ?value . FILTER (?value > 0.5)

} A type of rule expression

Such approaches are proposed e.g. in SPIN (SPARQL Inference Notation)

8

Land Use/Land Cover systems Available land classifications correspond to specific

applications, e.g. crop or vegetation characterization As such, they cannot be used for generic telesensing

applications Most important of them are

CORINE Organized in a 3 level hierarchy: 5 categories of the 1st level

are broken down to 15 categories on the 2nd level which in turn are broken down to 44 3rd level categories

Land Cover Classification System (LCCS) It doesn’t specify predefined land cover categories, it rather

defines general classification criteria for characterizing land covers

GNORASI classification10

Challenges Solutions

Usage examples

Development details

11

Challenges

Object-based image analysis produces a large number of objects (thousands)

Probabilistic inference for class membership

Classification is based on user-defined rules Feature-based Geospatial restrictions

Classes can appear as premises in rules

12

Solutions

Numerical computations are executed outside the ontological framework Fuzzy membership values are computed using

membership functions Ontological inference is used for the assignment

of objects to classes According to user rules

SPARQL Update and GeoSPARQL are used to define the rules in ontological terms

Development decision The ontological classification is implemented as an

external web service

13

Rule definition UI

Hierarchy

Class ruleset

Rule definition

14

Fuzzy values

The outcome of the rule definition processor are the fuzzy values of all objects for the features present in the rules

These arithmetic values have to be assigned to semantic entities, i.e. the defined classes

Object id

Fuzzy Band1 Mean

Fuzzy Roundness

Fuzzy NDVI

Fuzzy Band3 Kurtosis

1 0.335 0.843 0.776 0.094

2 0.598 0.321 0.705 0.544

3 0.101 0.202 0.777 0.000

4 0.000 0.101 1.000 0.333

15

Ontology data

The fuzzy values along with the class hierarchy and rule definitions are sent to the ontology classification service

The following are performed by the service The class hierarchy is added to the core

ontology The fuzzy values are transformed to

ontological data properties Rules are translated to SPARQL Update

queries Rules are iteratively executed

16

Example SPARQL Update

Example queries for assigning an object to classes Sidewalk and Vegetation with confidences ?conf1 and ?conf2Sidewalk

INSERT {?object gn:depicts gn:depiction1. gn:depiction1 gn:depictsClass gn:Sidewalk. gn:depiction1 gn:withConfidence ?conf.} WHERE {?object rdf:type gn:Object.

?object gn:fuzzyNDVIMean ?conf.}

VegetationINSERT {?object gn:depicts gn:depiction2. gn:depiction2 gn:depictsClass gn:Vegetation. gn:depiction2 gn:withConfidence ?conf2.} WHERE {?object rdf:type gn:Object. ?object gn:fuzzyNDVIMean ?conf2.}

INSERT {?object gn:depicts gn:depiction1. gn:depiction1 gn:depictsClass gn:Sidewalk. gn:depiction1 gn:withConfidence ?conf.} WHERE {?object rdf:type gn:Object . ?object gn:fuzzyNDVIMean ?conf. ?filterObject rdf:type gn:Object. ?filterObject gn:depicts ?gn:filterDepiction. ?filterDepiction gn:depictsClass gn:Road FILTER (geof:sfTouches(?object, ?filterObject))}

Sidewalk rule with geospatial restriction (adjacent to Road)

17

Classification

In the example, Sidewalk depends on Road definition. Objects assigned to Road must exist

Iterative rule execution until convergence (no changes in the repository)

In the end, every object will be assigned to classes with different membership values

INSERT {?object gn:depicts gn:depiction1. gn:depiction1 gn:depictsClass gn:Sidewalk. gn:depiction1 gn:withConfidence ?conf.} WHERE { ?object rdf:type gn:Object.

?object gn:fuzzyNDVIMean ?conf.?filterObject rdf:type gn:Object.?filterObject gn:depicts ?

gn:filterDepiction.?filterDepiction gn:depictsClass

gn:RoadFILTER (geof:touches(?object, ?

filterObject))}

18

Classification

In a hierarchy, objects will try to match the deepest classes

19

Classification

Object are assigned to classes with the highest membership value

A minimum threshold is applied The service returns a list

[<object id> <class id> <confidence>]* This is the classification result

20

Development

Java based REST communication Server

Grizzly project1

Reasoner employed OpenRDF Sesame2 backend, OWLIM-lite3

reasoner Geospatial library

uSeekM IndexingSail41 Grizzy, https://grizzly.java.net/2 OpenRDF Sesame, http://www.openrdf.org3 OWLIM Lite, http://www.ontotext.com/owlim4 IndexingSail, https://dev.opensahara.com/projects/useekm/wiki/IndexingSail

21

Concluding

Efficient ontology-based classification Employ both feature-based classification

and geospatial restrictions Handling of fuzzy membership values Built as a web service

22

Thank you!

Questions?