hyunjang kong,myunggwun hwang,kwansang na,pankoo kim
DESCRIPTION
The Study on the Semantic Image Retrieval Using the Cognitive Spatial Relationships in the Semantic Web. Hyunjang Kong,Myunggwun Hwang,Kwansang Na,Pankoo Kim. Contents. Introduction Related Works Our Approach Test and Experimental Results Evaluations Conclusion. Introduction. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Hyunjang Kong,Myunggwun Hwang,Kwansang Na,Pankoo Kim](https://reader036.vdocuments.net/reader036/viewer/2022062408/568144fc550346895db1c767/html5/thumbnails/1.jpg)
CHOSUN UNIV.
The Study on the Semantic Image Retrieval Using the Cognitive Spatial Relationships in the Semantic Web
Hyunjang Kong,Myunggwun Hwang,Kwansang Na,Pankoo Kim
![Page 2: Hyunjang Kong,Myunggwun Hwang,Kwansang Na,Pankoo Kim](https://reader036.vdocuments.net/reader036/viewer/2022062408/568144fc550346895db1c767/html5/thumbnails/2.jpg)
CHOSUN UNIV.
Contents
• Introduction• Related Works• Our Approach• Test and Experimental Results• Evaluations• Conclusion
![Page 3: Hyunjang Kong,Myunggwun Hwang,Kwansang Na,Pankoo Kim](https://reader036.vdocuments.net/reader036/viewer/2022062408/568144fc550346895db1c767/html5/thumbnails/3.jpg)
CHOSUN UNIV.
Introduction
• Huge number of data in the web• Image data is rapidly increasing• Object Based Spatial Relationships
VS Cognitive Spatial Relationships• Building a Spatial Relationships
Ontology
![Page 4: Hyunjang Kong,Myunggwun Hwang,Kwansang Na,Pankoo Kim](https://reader036.vdocuments.net/reader036/viewer/2022062408/568144fc550346895db1c767/html5/thumbnails/4.jpg)
CHOSUN UNIV.
Related Works
• Information Retrieval System– Keyword Matching – Very important technique on the web
environment– Process the various information items
• Text Documents, Images, Sounds and etc.
– Generally, accuracy is low
![Page 5: Hyunjang Kong,Myunggwun Hwang,Kwansang Na,Pankoo Kim](https://reader036.vdocuments.net/reader036/viewer/2022062408/568144fc550346895db1c767/html5/thumbnails/5.jpg)
CHOSUN UNIV.
Related Works
• Ontology based Image Retrieval– Try to solve the heterogeneous between
the terminologies– Need the extra works
• Creating and Maintaining the ontologies
– It is still unsuitable for the image retrieval system• Because it doesn’t consider the features of
the images
![Page 6: Hyunjang Kong,Myunggwun Hwang,Kwansang Na,Pankoo Kim](https://reader036.vdocuments.net/reader036/viewer/2022062408/568144fc550346895db1c767/html5/thumbnails/6.jpg)
CHOSUN UNIV.
Related Works
• The Spatial Description Logic– Region Connection Calculus : RCC-8– Spatial representation is regular subsets of the
topological space– Elementary binary relationships between the
regions• PO, NTPP, TPP, EQ, TPP-1, NTPP-1, EC, DC
X Y X Y YX XY
X Y X Y Y XXY
DC(X, Y) EC(X, Y) TPP(X, Y) TPP- 1(X ,Y)
PO(X, Y) EQ(X, Y) NTPP(X, Y) NTPP- 1(X, Y)
![Page 7: Hyunjang Kong,Myunggwun Hwang,Kwansang Na,Pankoo Kim](https://reader036.vdocuments.net/reader036/viewer/2022062408/568144fc550346895db1c767/html5/thumbnails/7.jpg)
CHOSUN UNIV.
Our Approach
• Background Knowledge of the Cognitive Spatial Relationships
![Page 8: Hyunjang Kong,Myunggwun Hwang,Kwansang Na,Pankoo Kim](https://reader036.vdocuments.net/reader036/viewer/2022062408/568144fc550346895db1c767/html5/thumbnails/8.jpg)
CHOSUN UNIV.
Our Approach
• Building Process of the Spatial Relationships Ontology– Defining the Cognitive Spatial
Relationships– User Research– Using WordNet and Dictionary– OWL Representation
![Page 9: Hyunjang Kong,Myunggwun Hwang,Kwansang Na,Pankoo Kim](https://reader036.vdocuments.net/reader036/viewer/2022062408/568144fc550346895db1c767/html5/thumbnails/9.jpg)
CHOSUN UNIV.
A B
A B
A B
BA
BA
AB
AB
A(B)
C(A,B)
DC(A,B)
PO(A,B)
TPP(A,B)
NTPP(A,B)
TPP- 1(A,B)
NTPP- 1(A,B)
EQ(A,B)
A B
A B
A B
C(A,B)
DC(A,B)
PO(A,B)
RCC- 8
Cognitive Spatial Relationships defined
in our study
Our Approach
• Definition of the Cognitive Spatial Relationships
![Page 10: Hyunjang Kong,Myunggwun Hwang,Kwansang Na,Pankoo Kim](https://reader036.vdocuments.net/reader036/viewer/2022062408/568144fc550346895db1c767/html5/thumbnails/10.jpg)
CHOSUN UNIV.
Our Approach
• User Research– 200 images– 10 people– Clustering the spatial words
![Page 11: Hyunjang Kong,Myunggwun Hwang,Kwansang Na,Pankoo Kim](https://reader036.vdocuments.net/reader036/viewer/2022062408/568144fc550346895db1c767/html5/thumbnails/11.jpg)
CHOSUN UNIV.
Our Approach
• Architecture of the spatial ontology– Upper level– Basic spatial
words level– Instance level
Cognitives_r
connect partof disconnect
verb
v_c v_p v_d
proposition
p_c p_p p_d
kiss lie run swim ride look jump fall
Bussosculatexxxx
Restxxxx
Speed
hurryzipxxxx
Gomovetravelxxxx
Sitdrivexxxx
Frontfaceseexxxx
Leapboundspringxxxx
Pursuenearxxxx
onacrossthroughalongxxxx
atin
aroundroundxxxx
overunderbesidenearxxxx
Upper_Level Cognitive Spatial Relationships
Spatial VerbsBased on the Research
Spatial Verbs Based on the WordNetSpatial Propositions Based on
the OXFORD Dictionary
![Page 12: Hyunjang Kong,Myunggwun Hwang,Kwansang Na,Pankoo Kim](https://reader036.vdocuments.net/reader036/viewer/2022062408/568144fc550346895db1c767/html5/thumbnails/12.jpg)
CHOSUN UNIV.
Our Approach
• WordNet and Dictionary Matching
Cognitive spatial relationships
Research words WordNet matching words
connect Attach Connect, link, tie, link up, fasten, touch, adjoin, meet, contact
connect Kiss Buss, osculate
disconnect Chase Chase after, trail, tail, tag, give chase, god, go after, pursue, follow
disconnect Jump Leap, bound, spring
partof Float Drift, be adrift, blow, swim, transport
partof Hide Conceal, shroud, enshroud, cover, obscure, blot out, obliterate, veil
The spatial propositions based on OXFORD Dictionary
connect On, along, across, through
disconnect Over, under, above, below, by, beside, near, before, behind
partof At, in, around, round
![Page 14: Hyunjang Kong,Myunggwun Hwang,Kwansang Na,Pankoo Kim](https://reader036.vdocuments.net/reader036/viewer/2022062408/568144fc550346895db1c767/html5/thumbnails/14.jpg)
CHOSUN UNIV.
Test and Experimental Results
• System Architecture– Contents provider interface– Ontology part– User interface
![Page 15: Hyunjang Kong,Myunggwun Hwang,Kwansang Na,Pankoo Kim](https://reader036.vdocuments.net/reader036/viewer/2022062408/568144fc550346895db1c767/html5/thumbnails/15.jpg)
CHOSUN UNIV.
Test and Experimental Results
• Test Environment– Queries
1. Only one word query – e.g. swan2. Two words query – e.g. swan and lake3. Query containing the spatial relationships –
e.g. swan in the lake4. Natural Language query containing the
spatial verbs – e.g. swimming swan5. Natural Languages query containing the
spatial verbs and proposition – e.g. swan swims in the lake
![Page 16: Hyunjang Kong,Myunggwun Hwang,Kwansang Na,Pankoo Kim](https://reader036.vdocuments.net/reader036/viewer/2022062408/568144fc550346895db1c767/html5/thumbnails/16.jpg)
CHOSUN UNIV.
Test and Experimental Results
• Accuracy Measurement
• Experimental Results
Accuracy = All images searched throughout the system
Correct images matched with the query
![Page 19: Hyunjang Kong,Myunggwun Hwang,Kwansang Na,Pankoo Kim](https://reader036.vdocuments.net/reader036/viewer/2022062408/568144fc550346895db1c767/html5/thumbnails/19.jpg)
CHOSUN UNIV.
Conclusion and Future Works
• Definition of the Cognitive Spatial Relationships
• Applying them to the Image Retrieval System
• Still have Limitation : Semi-Automatic• Our study presents the vision of the
semantic image retrieval and natural language query processing