a novel approach to multimedia ontology engineering for … · 2016. 3. 15. · a novel approach to...

A Novel Approach to Multimedia Ontology Engineering for

Automated Reasoning over Audiovisual LOD Datasets

Leslie F. Sikos, Ph.D. Flinders University, Australia ACIIDS 2016, 14 March 2016

Đà Nẵng, Vietnam

Reasoning over Video

• Vocabularies? Ontologies?

• Formalism with Description Logics

• ABox+TBox+RBox+DL-Safe Ruleset

• VidOnt, the most expressive video

ontology to date

• Reasoning over audiovisual datasets

Automated Reasoning over Audiovisual LOD Datasets ACIIDS’16

Outline

Ontology EngineeringVocabularies? Ontologies?

Thesaurus

Controlled

vocabulary

Ontology

Knowledge base

Dataset


Ontology Engineering

• Description Logics (DLs): formal

knowledge representation languages

(decidable fragments of FOL)

DLs are suitable for the expressive

formalization of multimedia contents and the

semantic refinement of video segmentation.

Formalism with Description Logics


Ontology EngineeringFormalism with Description Logics

FOL DL OWL

unary predicate concept class

binary predicate role property

constant individual individual

formula with two

free variables

role

expression

property

expression


Ontology EngineeringDescription Logics


Ontology EngineeringAvailable Constructors

Symbol Constructors Example

ALC ⊑ ≡ ⊓ ⊔ ¬ ∀∃ ⊤ ⊥

cartoon ≡ animatedMovie

⊓ ¬(liveAction ⊔computerAnimation)

S

+ transitivity

partOf ◦ partOf ⊑ partOf


ALC



R Limited complex

role inclusion

axioms, reflexivity

and irreflexivity,

role disjointness

partOf ◦ starredIn ⊑starredIn




O Nominals AnimationStudios ≡ {

DreamWorks, Walt Disney

Animation Studios, Pixar }

I Inverse roles directedBy ≈ directorOf-




Q Qualified

cardinality

restrictions

Actor ⊑= 1hasBirthplace.⊤



Symbol Example(D) runningTime(Zambezia, 83)



copyrightHolder ≡ Organization ⊔ Person

copyrightHolder(DreamWorks)

DreamWorks ≡ Organization ⊔ Person

Reasoning Potential ( )


ALC


co-starredWith(JohnWayne, ClintEastwood) ⊑ ⊥starredIn(ClintEastwood, Unforgiven)

starredIn(JohnWayne, Unforgiven) ⊑ ⊥



SROIQ(D)


basedOn ◦ basedOn ⊑ basedOn

basedOn(LastManStanding, AFistfulOfDollars) ⊑ ⊤basedOn(AFistfulOfDollars, Yojimbo) ⊑ ⊤

basedOn(LastManStanding, Yojimbo) ⊑ ⊤



SROIQ(D)


releasedIn.PaleRider(1985)

retiredIn.JohnWayne(1979)

swrl:greaterThan(?releaseDate, ?retiredIn)

→ (inverse starredIn)(?a, ?m)

starredIn(JohnWayne, PaleRider) = ⊥

Reasoning Potential (Rules)



Example. The Visual Descriptor Ontology

(VDO) was published as an “ontology for

multimedia reasoning” while it features very

limited description logic expressivity

(corresponding to ALH) and reasoning potential

Description Logic Expressivity


Multimedia Ontologies

Ontology Language DL

LinkedMDB RDFS AL

LSCOM OWL AL

M3O OWL SHIQ(D)

COMM OWL SHOIN(D)

Limited DL Expressivity



LinkedMDB RDFS AL

LSCOM OWL AL

M3O OWL SHIQ(D)

COMM OWL SHOIN(D)

Multimedia OntologiesFull DL Expressivity


VidOnt OWL 2 SROIQ(D)

http://vidont.org


http://vidont.org/

Ontology EngineeringTBox Axioms

DL Syntax Turtle Syntax

liveAction ⊑ Movie :liveAction rdfs:subClassOf

:Movie .

Narrator ≡ Lector :Narrator owl:equivalentClass

:Lector .


Ontology EngineeringABox Axioms


actor(JackieChan) :JackieChan a :actor .

房仕龍 ≈ JackieChan :房仕龍 owl:sameIndividualAs

:JackieChan .


Bridging the Semantic GapThe Missing Piece

• T TBox: terminological knowledge

• AABox: assertional knowledge


Bridging the Semantic GapThe Missing Piece

• T TBox: terminological knowledge

• AABox: assertional knowledge

• R RBox: complex role inclusion

axioms + transitive roles


Ontology EngineeringRBox Axioms


partOf ◦ starredIn ⊑starredIn

:starredIn

owl:propertyChainAxiom

(:partOf :starredIn) .

basedOn ◦ basedOn ⊑basedOn

:basedOn a

owl:TransitiveProperty .



[ rdf:type swrl:Imp ; swrl:body [ rdf:type

swrl:AtomList ; rdf:rest [ rdf:type swrl:AtomList ;

rdf:first [ rdf:type swrl:DatavaluedPropertyAtom ;

swrl:argument2 "1080p" ; swrl:propertyPredicate:

videoMode ; swrl:argument1 :m ] ; rdf:rest rdf:nil ]

DL-Safe Ruleset


…

…

Multimedia Reasoning

Actor(a), Actor(b), Actor(c), Actor(d)

Movie(m), Sequel(s), partOf(m, s)

partOf ◦ starredIn ⊑ starredIn

starredIn(a, m), starredIn(b, m), starredIn(c, m),

starredIn(d, s)

starredIn(?x, m) → co-starredWith(?x, d)

RBox and Rule-Based Reasoning


Multimedia Reasoning

New object property assertions:

co-starredWith(a, d), co-starredWith(b, d),

co-starredWith(c, d)

starredIn(a, s), starredIn(b, s), starredIn(c, s)

Automatically Inferred Assertions


Multimedia ReasoningReasoning Complexity


DL Worst-Case Combined Complexity

EXPTIME

EXPTIME

NEXPTIME

N2EXPTIME

ALC

SHIF

SHOIN

SROIQ

VidOnt’s Knowledge DomainsVideo Production

Knowledge-Driven Video Information Retrieval with LOD ESAIR’15

VidOnt’s Knowledge DomainsBroadcasting

Knowledge-Driven Video Information Retrieval with LOD ESAIR’15

Reasoning over Audiovisual DataConclusions

• VidOnt, the first SROIQ(D) video ontology

• Multimedia ontology

engineering best practices

• Automatically infer new statements

with reasoners from audiovisual

LOD datasets


Thank you.

Questions?

More on Description Logics

• Sikos, L. F. Description Logics in Multimedia

Reasoning. Springer (forthcoming)

• Sikos, L. F. Mastering Structured

Data on the Semantic Web: From

HTML5 Microdata to Linked Open

Data. Apress, New York, USA, 2015

http://www.lesliesikos.com/

a novel approach to multimedia ontology engineering for … · 2016. 3. 15. · a novel approach to...

Documents