relationship web: trailblazing, analytics and computing ... · wright state university core scholar...

Wright State UniversityCORE Scholar

Kno.e.sis Publications The Ohio Center of Excellence in Knowledge-Enabled Computing (Kno.e.sis)

10-20-2008

Relationship Web: Trailblazing, Analytics andComputing for Human ExperienceAmit P. ShethWright State University - Main Campus, amit.sheth@wright.edu

Follow this and additional works at: http://corescholar.libraries.wright.edu/knoesis

Part of the Bioinformatics Commons, Communication Technology and New Media Commons,Databases and Information Systems Commons, OS and Networks Commons, and the Science andTechnology Studies Commons

This Presentation is brought to you for free and open access by the The Ohio Center of Excellence in Knowledge-Enabled Computing (Kno.e.sis) atCORE Scholar. It has been accepted for inclusion in Kno.e.sis Publications by an authorized administrator of CORE Scholar. For more information,please contact corescholar@www.libraries.wright.edu.

Repository CitationSheth, A. P. (2008). Relationship Web: Trailblazing, Analytics and Computing for Human Experience. .http://corescholar.libraries.wright.edu/knoesis/264

Knowledge Enabled Information and Services Science 1

Knowledge Enabled Information and Services Science

Kno.e.sis Center -- Labs

Bioinformatics Lab (Dr Raymer)

Semantic Sciences Lab (Dr Sheth)

Metadata and Languages Lab (Dr Prasad)

Semantic Web Lab (Dr Sheth + Dr. S.Wang)

Data Mining Lab (Dr Dong)

Service Research Lab (Dr Sheth)

Sensor Networking Bin Wang

Kno.e.sis Members – a subset

Evolution of the Web

1990sWeb of databases

- dynamically generated pages- web query interfaces

Web of pages- text, manually created links- extensive navigation

Web of services- data = service = data, mashups- ubiquitous computing

Web of people- social networks, user-created content- GeneRIF, Connotea

Web as an oracle / assistant / partner- “ask to the Web”- using semantics to leveragetext + data + services + people

Computing for Human Experience

Relationship Web: Trailblazing , Analytics and Computing for Human Experience

27th International Conference on Conceptual Modeling (ER 2008)Barcelona, Oct 20-23 2008

Amit ShethKno.e.sis Center, Wright State University, Dayton, OH

This talk also represents work of several members of Kno.e.sis team, esp. the Semantic Discovery and Semantic Sensor Data. http://knoesis.wright.edu

Thanks, K. Godam, C. Henson, M. Perry, C. Ramakrishnan, C. Thomas. Also thanks to sponsors: NSF (SemDis and STT), NIH, AFRL, IBM, HP, Microsoft.

A Compelling Vision: Trailblazing

“(Human mind works) by association. With one item in its grasp, it snaps instantly to the next that is suggested by the association of thoughts, in accordance with some intricate web of trails carried by the cells of the brain.“

Dr. Vannevar Bush, As We May Think, 1945

Documents, Objects, Events, Insight, Experience

“An object by itself is intensely uninteresting”. – Grady Booch, Object Oriented Design with Applications, 1991

Keywords

Search(data)

Entities

Integration(information)

Relationships,Events

Analysis,Insight(knowledge)

Semantics and Relationships on a Web scale

Increasing depth and sophistication in dealing with semantics–From searching to integration to analysis, insight, discovery

and decision making–Relationships

• Heart of semantics• Allows to continue progress from syntax and structure to

semantics

A. Sheth, et al. Relationships at the Heart of Semantic Web: Modeling, Discovering, and Exploiting Complex Semantic Relationships, in Enhancing the Power of the Internet (Studies in Fuzziness and Soft Computing, V. 139). SpringerVerlag., 2004. http://knoesis.wright.edu/library/resource.php?id=00190

Relationship Web takes you away from “which document” could have data/information I need, to interconnecting Web of information embedded in the resources to give knowledge, insights and answers I seek.

Amit P. Sheth, Cartic Ramakrishnan: Relationship Web: Blazing Semantic Trails between Web Resources. IEEE Internet Computing July 2007.

Structured text (biomedical literature)

Informal Text (Social Network

chatter) Multimedia Content and Web data

Web Services

Metadata Extraction

Patterns / Inference / Reasoning

Domain Models

Meta data / Semantic Annotations

Relationship Web

Search

Integration

Analysis

Discovery

Question Answering

Issues - Relationships

Understanding and modeling relationshipsIdentifying Relationship (extraction)Discovering and Exploring Relationships (reasoning) Hypothesizing and Validating Complex RelationshipsUsing/exploiting Relationships for Semantic Applications (in search, querying, analysis, insight, discovery)

Data, Information, and Insight

What: Which thing or which particular one

Who: What or which person or persons

Where: At or in what place

When: At what time

How: In what manner or way; by what means

Why: For what purpose, reason, or cause; with what intention, justification, or motive

EventWeb [Jain]RelationshipWeb [Sheth]

A Web in which each node is an event or object, and connected to other nodes using–Linguistic Relationships–Referential Links: hypertext links to related information

(HREF); links with metadata (MREF), links referring to model (model reference in SAWSDL, SA-REST)

–Structural Links: showing spatial and temporal relationships–Causal Links: establishing causality–Relational Links: giving similarity or any other relationship

Adapted/extended from Ramesh Jain

Supporting Relationships in the Real World

Objects and event represent or model real world– More complex relationships

• Living organisms: Saprotropism, Antagonism, Exploitation, Predation, Ammensalism or Antibiosis, Symbiosis

• Relationships between humans: …

Karthik Gomadam

AmitSheth

Attended Google IO

Moscone Center, SFOMay 28-29, 2008

Is advised by

Ph.D Student Researcheris_advised_by

Assistant Professor Professor

Research Paper

Journal Conference

Location

published_in published_in

has_location

Image Metadata

Causal

kno.e.sis

DirectsRelational

Identifying & Representing Relationships

Implicit Relationships– Statistical representation of interactions between entities,

co-occurrence of terms in the same cluster, tag cloud...

Linking of one document to another via a hyperlink…

– Two documents’ belonging to categories that are siblings in a concept hierarchy.

Explicit Linguistic Relationships

“He beat Randy Johnson” converted into the triple

Dontrelle WillisbeatRandy Johnson

Formal Relationships

–Subsumption, partonomy, ….

Domain specific vs domain-independent relationships

–Example of domain independent relationships: time, space/location

Complex entity and relationships (example later)

Why is This a Hard Problem?

Are objects/entities equivalent/equal(same)?How (well) are they related? –Implicit vs explicit:

• Statistical representation of interactions between entities, co-occurrence of terms in the same cluster, tag cloud...

• formal/assertional vs social consensus based• powerful (beyond FOL): partial, probilistic and fuzzy match

–Degrees of relatedness and relevance: semantic similarity, semantic proximity, semantic distance, …• [differentiation, disjointedness]• related in a “context”

Why is This a Hard Problem?

Semantic ambiguity When information (knowledge) is –Incomplete– inconsistent –Approximate

–Even is-a link involves different notions: identify, unity, essense (Guarino and Wetley 2002)

A.Sheth, et al, “Semantics for The Semantic Web: the Implicit, the Formal and the Powerful,” International Journal on Semantic Web & Information Systems, 1 (no. 1), 2005, pp. 1–18.

Faceted Search and Semantic Analytics –Some Early Work where relationships played key role

InfoHarness/VisualHarness (1993-1998)InfoQuilt (1996-2000)MREF (1996-1998)OBSERVER (1996-2000)Taalee Semantic (Faceted) Search, Semantic Directory and Semagix Freedom enabled analytics (1999-2006)

InfoQuilt (1996-2000): Using metadata PatchQuilt and user models/ontologies to support queries and analytics over globally distributed heterogeneous media repositories

Physical Link to Relationship

<TITLE> A Scenic Sunset at Lake Tahoe </TITLE><p>Lake Tahoe is a popular tourist spot and<A HREF = “http://www1.server.edu/lake_tahoe.txt”>some interesting facts</A> are available here. The scenic beauty of Lake Tahoe can be viewed in this photograph:<center><IMG SRC=“http://www2.server.edu/lake_tahoe.img”></center>

Traditionally, correlation is achieved by using physical linksDone manually by user publishing the HTML document

MREF (1996-1998)Metadata Reference Link -- complementing HREF

Creating “logical web” through Media independent metadata based on correlation

Metadata Reference Link (<A MREF …>)

<A HREF=“URL”>Document Description</A>physical link between document (components)

<A MREF KEYWORDS=<list-of-keywords>; THRESH=<real>>Document Description</A>

<A MREF ATTRIBUTES(<list-of-attribute-value-pairs>)>Document Description</A>

Correlation Based on Content-Based Metadata

Some interesting information on dams is available here“information on dams” defined by MREF to keywords and metadata (may be used for a query)

height, width and size

water.gif (Data) Metadata Storage

water.gif……mpeg……ppm

Major component(RGB)Blue

Content based Metadata

ContentDependentMetadata

Abstraction Layers

METADATA

ONTOLOGYNAMESPACE

MREFin RDF

MREF (1998)

Model for LogicalCorrelation using Ontological Terms and Metadata

Framework for Representing MREFs

Serialization(one implementation choice)

K. Shah and A. Sheth, "Logical Information Modeling of Web-accessible Heterogeneous Digital Assets",Proc. of the Forum on Research and Technology Advances in Digital Libraries," (ADL'98),Santa Barbara, CA, May 28-30, 1998, pp. 266-275.

M R E F

Domain Specific Correlation

Potential locations for a future shopping mall identified by all regions having a population greater than 500 and area greater than 50 sq meters having an urban land cover and moderate relief <A MREF ATTRIBUTES(population < 500; area < 50 & region-type = ‘block’ & land-cover = ‘urban’ & relief = ‘moderate’)>can be viewed here</A>

Domain Specific Correlation

=> media-independent relationships between domain specific metadata: population, area, land cover, relief

=> correlation between image and structured data at a higher domain specific level as opposed to physical “link-chasing” in the WWW

TIGER/Line DB

Population: Area:

Boundaries:

Land cover:Relief:

Census DB

Map DB

Regions(SQL)

Boundaries

Image Features(IP routines)

Repositories and the Media Types

Complex Relationships

Some relationships may not be manually asserted, but according to statistical analyses of text, experimental data, etc.– allow association of provenance data with classes,

instances, relationship types and direct relationships or statements

Complex Relationships

Relationships (mappings) are not always simple mathematical / string transformationsExamples of complex relationships–Associations / paths between classes–Graph based / form fitting functions–Probabilistic relational

E 1 :Reviewer

E 6 :Person

E 5 :Person

E 2 :Paper

E 4 :Paper

E 7 :Submission

E 3 :Person

author _ ofauthor _ of

author _of

author _ of

A Simple Relationship ?

Smoking CancerCauses

Graph based / form fitting functions

Complex Relationships Cause-Effects & Knowledge Discovery

VOLCANO

LOCATIONASH RAIN

PYROCLASTICFLOW

ENVIRON.

LOCATION

PEOPLE

WEATHER

BUILDING

DESTROYS

COOLS TEMP

DESTROYS

Knowledge Discovery - Example

Earthquake Sources Nuclear Test Sources

Nuclear Test May Cause Earthquakes

Is it really true?

Complex Relationship:

How do you model this?

Complex Relationships – Several Challenges

–Probabilistic relations

Number of earthquakes with magnitude > 7 almost constant. So if at all, then nuclear tests only cause earthquakes with magnitude < 7

Entity, Relationship, Event Extraction and Semantic Annotation

From content with structure –Web pages–Deep Web

From well-formed text (edited for rules of grammar)From informal or casual text–social networking sites

From digital media

Knowledge Enabled Information and Services Science© Semagix, Inc.

Semagix Freedom for building ontology-driven information system

Extracting Semantic Metadata from Semistructured and Structured Sources

WWW, EnterpriseRepositories

METADATA

EXTRACTORS

Digital Maps

NexisUPIAPFeeds/

Documents

Digital Audios

Data Stores

Digital Videos

Digital Images. . .

. . . . . .

Create/extract as much (semantics) metadata automatically as possible;

Use ontlogies to improve and enhance extraction

Information Extraction for Metadata Creation

Automatic Semantic Metadata Extraction/Annotation

Limited tagging(mostly syntactic)

COMTEX Tagging

Content‘Enhancement’

Rich Semantic Metatagging

Value-added Voquette Semantic Tagging

Value-addedrelevant metatagsadded by Voquetteto existing COMTEX tags:

• Private companies • Type of company• Industry affiliation• Sector• Exchange• Company Execs• Competitors

Semantic Annotation (Extraction + Enhancement)

Video withEditorialized Text on the WebAuto

Categorization

Semantic Metadata

Automatic Classification & Metadata Extraction (Web Page)

Extraction Agent

Enhanced Metadata AssetWeb Page

Ontology-Directed Metadata Extraction (Semi-Structured Data)

Some real-world or commercial semantic applications

Taalee’s Semantic (Facted) Search (1999-2001):Highly customizable, precise and fresh A/V search

Delightful, relevant information,exceptional targeting opportunity

Semantic Targeting

BLENDED BROWSING & QUERYING INTERFACE

ATTRIBUTE & KEYWORDQUERYING

uniform view of worldwide distributed assets of similar type

SEMANTIC BROWSING

Targeted e-shopping/e-commerce

assets access

VideoAnywhere and Taalee Semantic Querying and Browsing (1998-2001)

Semantic/Interactive Targeting (1999-2001)

Buy Al Pacino VideosBuy Russell Crowe VideosBuy Christopher Plummer VideosBuy Diane Venora VideosBuy Philip Baker Hall VideosBuy The Insider Video

Precisely targeted through the use of Structured Metadata and integration from multiple sources

Keyword, Attribute and Content Based Access

Blended Semantic Browsing and Querying (Intelligence Analyst Workbench) 2001-2003

Search for company

‘Commerce One’

Links to news on companies that compete against

Commerce One

Links to news on companies Commerce One competes

against

(To view news on Ariba, click on the link for Ariba)

Crucial news on Commerce One’s

competitors (Ariba) can be accessed easily

and automatically

Semantic Browsing/Directory (2001-….)

System recognizes ENTITY & CATEGORY

Relevant portionof the Directory is automatically presented.

Semantic Directory

Users can exploreSemantically related

Information.

Semantic Directory

Semantic Relationships

Focused relevantcontent

organizedby topic

(semantic categorization)

Automatic ContentAggregation

from multiple content providers and feeds

Related relevant content not explicitly asked for (semantic

associations)

Competitive research inferred

automatically

Automatic 3rd party content

integration

Semantic Application Example – Research Dashboard (Voquette/Semagix: 2001-2004)

Watch list Organization

Company

WorldCom

FBI Watchlist

Ahmed Yaseer

appears on Watchlistmember of organization

works for Company

Ahmed Yaseer:• Appears on Watchlist ‘FBI’

• Works for Company ‘WorldCom’

• Member of organization ‘Hamas’

Early Semantic Association: An Application in Risk & Compliance (Semagix 2004-2006)

Global Investment Bank

Example of Fraud

prevention applicationused in financial services

User will be able to navigate the ontology using a number of different interfaces

World Wide Web content

Public Records

BLOGS,RSS

Un-structure text, Semi-structured Data

Watch ListsLaw

Enforcement Regulators

Semi-structured Government Data

Scores the entity based on the content and entity relationships

EstablishingNew Account

demostration

How Background Knowledge helps Informal Content Analysis

A Community’s Pulse is often Informal

•Wealth of information available in blogs, social networks, chats etc.

•Free medium of self-expression makes mass opinions / interests available

•Polling for popular culture opinions is easier

•Social Production undeniably affects markets• geo-specific retail ads, demographic interests in music

Background Knowledge Improves Content Analysis

Metadata creation Example – music comments Spot Artist, Track names and associated sentiments

Example comment “Keep your smile on Lil.”

Smile here is a track from Artist Lilly Allen’s album Background knowledge from Music Brainz taxonomy provides

evidence Annotate ‘smile’ as Track ‘Lil’ as Lilly Allen

Background knowledge from Urban Dictionary for understanding Slang I say: “Your music is wicked” What I really mean: “Your music is good”

Results - Pulse of a Music Community

•Mining artist popularity from chatter on MySpace- Lists close to listeners preferences vs. Bill Boards

BB User Comments: May 07

User Comments: Jun 07

Rihanna

Biffy Clyro

Maroon 5

McCartney

Winehouse

Rascal

Rihanna

Winehouse

Maroon 5

Mccartney

Biffy Clyro

Rascal

Rihanna

Winehouse

Maroon 5

Mccartney

Biffy Clyro

Rascal

830.9570 194.9604 2580.2985 0.3592688.3214 0.2526779.4759 38.4939784.3607 21.7736

1543.7476 1.38221544.7595 2.99771562.8113 37.47901660.7776 476.5043

parent ion m/z

fragment ion m/z

ms/ms peaklist data

fragment ionabundance

parent ionabundance

parent ion charge

Mass Spectrometry (MS) Data

Semantic Extraction/Annotation of Experimental Data

Semantic Sensor Web

Semantically Annotated O&M

<swe:component name="time"><swe:Time definition="urn:ogc:def:phenomenon:time" uom="urn:ogc:def:unit:date-time">

<sa:swe rdfa:about="?time" rdfa:instanceof="time:Instant"><sa:sml rdfa:property="xs:date-time"/>

</sa:swe></swe:Time></swe:component><swe:component name="measured_air_temperature"><swe:Quantity definition="urn:ogc:def:phenomenon:temperature“

uom="urn:ogc:def:unit:fahrenheit"><sa:swe rdfa:about="?measured_air_temperature“

rdfa:instanceof=“senso:TemperatureObservation"><sa:swe rdfa:property="weather:fahrenheit"/><sa:swe rdfa:rel="senso:occurred_when" resource="?time"/><sa:swe rdfa:rel="senso:observed_by" resource="senso:buckeye_sensor"/>

</sa:sml></swe:Quantity></swe:component>

<swe:value name=“weather-data">2008-03-08T05:00:00,29.1</swe:value>

Semantic Sensor Web @ Kno.e.sis74

Person

Company

Coordinates

Coordinate System

Time Units

Timezone

SpatialOntology

DomainOntology

TemporalOntology

Mike Botts, "SensorML and Sensor Web Enablement," Earth System Science Center, UAB Huntsville

Semantic Sensor ML – Adding Ontological Metadata

Semantic Query

Semantic Temporal Query

Model-references from SML to OWL-Time ontology concepts provides the ability to perform semantic temporal queries

Supported semantic query operators include:– contains: user-specified interval falls wholly within a sensor reading

interval (also called inside)– within: sensor reading interval falls wholly within the user-specified interval

(inverse of contains or inside)– overlaps: user-specified interval overlaps the sensor reading interval

Example SPARQL query defining the temporal operator ‘within’

demo of Semantic Sensor Webhttp://knoesis.wright.edu/research/semsci/application_domain/sem_sensor/

Relationship/Fact Extraction from Text

Knowledge Engineering approach–Manually crafted rules

• Over lexical items <person> works for <organization>• Over syntactic structures – parse trees

–GATE

Relationship/Fact Extraction from Text

Machine learning approaches–Supervised–Semi-supervised –Unsupervised

Schema-Driven Extraction of Relationships from Biomedical Text

Cartic Ramakrishnan, Krys Kochut, Amit P. Sheth: A Framework for Schema-Driven Relationship Discovery from Unstructured Text. International Semantic Web Conference 2006: 583-596 [.pdf]

Method – Parse Sentences in PubMed

SS-Tagger (University of Tokyo)

SS-Parser (University of Tokyo)

(TOP (S (NP (NP (DT An) (JJ excessive) (ADJP (JJ endogenous) (CC or) (JJ exogenous) ) (NN stimulation) ) (PP (IN by) (NP (NN estrogen) ) ) ) (VP (VBZ induces) (NP (NP (JJ adenomatous) (NN hyperplasia) ) (PP (IN of) (NP (DT the) (NN endometrium) ) ) ) ) ) )

• Entities (MeSH terms) in sentences occur in modified forms• “adenomatous” modifies “hyperplasia”• “An excessive endogenous or exogenous stimulation” modifies “estrogen”

• Entities can also occur as composites of 2 or more other entities• “adenomatous hyperplasia” and “endometrium” occur as “adenomatous

hyperplasia of the endometrium”

Method – Identify entities and Relationships in Parse Tree

induces

NPINof

NNendometrium

JJadenomatous

NNhyperplasia

NP PPINby

NNestrogenDT

excessive ADJP NNstimulation

JJendogenous

JJexogenous

ModifiersModified entitiesComposite Entities

Resulting Semantic Web Data in RDF

ModifiersModified entitiesComposite Entities

estrogen

An excessive endogenous or

exogenous stimulation

modified_entity1composite_entity1

modified_entity2

adenomatous hyperplasia

endometrium

hasModifier

hasPart

induces

hasPart

hasModifier

hasPart

Blazing Semantic Trails in Biomedical Literature

Cartic Ramakrishnan, Extracting, Representing and Mining Semantic Metadata from Text: Facilitating Knowledge Discovery in Biomedicine, PhD Thesis, Wright State University, August 2008.

Amit Sheth and Cartic Ramakrishnan, “Relationship Web: Blazing Semantic Trails between Web Resources,” IEEE Internet Computing, July–August 2007, pp. 84–88.

Relationships -- Blazing the Trails

“The physician, puzzled by her patient's reactions, strikes the trail established in studying an earlier similar case, and runs rapidly through analogous case histories, with side references to the classics for the pertinent anatomy and histology.

The chemist, struggling with the synthesis of an organic compound, has all the chemical literature before him in his laboratory, with trails following the analogies of compounds, and side trails to their physical and chemical behavior.”

[V. Bush, As We May Think. The Atlantic Monthly, 1945. 176(1): p. 101-108. ]

Semantic Browser

demo of Semantic Browserhttp://knoesis.wright.edu/research/semweb/projects/textMining/iswc2008/

Original Documents

PMID-10037099

PMID-15886201

Semantic Trail

“Everything's connected, all along the line. Cause and effect.

That's the beauty of it. Our job is to trace the connections and reveal them.”

Jack in Terry Gilliam’s 1985 film - “Brazil”

How are Harry Potter and Dan Brown related?

Leonardo Da Vinci

The Da Vinci code

The Louvre

Victor Hugo

The Vitruvian man

Santa Maria delle Grazie

Et in Arcadia EgoHoly Blood, Holy Grail

Harry Potter

The Last Supper

Nicolas Poussin

Priory of Sion

The Hunchback of Notre Dame

The Mona Lisa

Nicolas Flammel

painted_by

member_of

written_by

mentioned_in

displayed_at

cryptic_motto_of

displayed_at

mentioned_in

How are Harry Potter and Dan Brown related?

Semantic Trails Over All Types of Data

Semantic Trails can be built over a Web of Semantic (Meta)Data extracted (manually, semi-automatically and automatically) and gleaned from –Structured data (e.g., NCBI databases)–Semi-structured data (e.g., XML based and semantic

metadata standards for domain specific data representations and exchanges)

–Unstructured data (e.g., Pubmed and other biomedical literature)

and–Various modalities (experimental data, medical images, etc.)

Discovering Complex Connection Patterns

Discovering informative subgraphsGiven a pair of end-points (entities) produce a subgraph with relationships connecting them such that the subgraph is small enough to be visualized and contains relevant “interesting” connections

Cartic Ramakrishnan, William H. Milnor, Matthew Perry, Amit P. Sheth. (2005). Discovering informative connection subgraphs in multi-relational graphs. SIGKDD Explorations, 7(2), pp. 56-63.

Discovering Complex Connection Patterns

We defined an interestingness measure based on the ontology schema –In future biomedical domain the scientist will control this

with the help of a browsable ontology–Our interestingness measure takes into account

• Specificity of the relationships and entity classes involved• Rarity of relationships etc.

Cartic Ramakrishnan, William H. Milnor, Matthew Perry, Amit P. Sheth. (2005). Discovering informative connection subgraphs in multi-relational graphs. SIGKDD Explorations, 7(2), pp. 56-63.

Heuristics

Two factor influencing interestingness

Discovery Algorithm

• Bidirectional lock-step growth from S and T• Choice of next node based on interestingness measure• Stop when there are enough connections between

the frontiers• This is treated as the candidate graph

Discovery Algorithm

Model the Candidate graph as an electrical circuit–S is the source and T the sink–Edge weight derived from the ontology schema are treated as

conductance values–Using Ohm’s and Kirchoff’s laws we find maximum current

flow paths through the candidate graph from S to T–At each step adding this path to the output graph to be

displayed we repeat this process till a certain number of predefined nodes is reached

Discovery Algorithm

Results–Arnold Schwarzenegger, Edward Kennedy

Other related work–Semantic Associations

Results

Hypothesis Driven Retrieval Of Scientific Text

Spatial

Temporal

Thematic

Events: 3 Dimensions – Spatial, Temporal and Thematic

Events and STT dimensions

Powerful mechanism to integrate content–Describes the Real-World occurrences–Can have video, images, text, audio all of the same event–Search and Index based on events and STT relations

Many relationship typesSpatial: –What events happened near this event? –What entities/organizations are located nearby?

Temporal: –What events happened before/after/during this event?

Thematic:–What is happening?–Who is involved?

Going furtherCan we use –What? Where? When? Who?

To answer –Why? / How?

Use integrated STT analysis to explore cause and effect

High-level Sensor (S-H)Low-level Sensor (S-L)

A-H E-HA-L E-L

• How do we determine if A-H = A-L? (Same time? Same place?)

• How do we determine if E-H = E-L? (Same entity?)

• How do we determine if E-H or E-L constitutes a threat?

Example Scenario: Sensor Data Fusion and Analysis

Sensor Data Pyramid

Raw Sensor (Phenomenological) Data

Feature Metadata

Entity Metadata

Relationship

Metadata

Information

Semantics/Understanding/Insight

Data Pyramid

Collection

Feature Extraction

Entity Detection

Semantic Analysis

• Raw Phenomenological Data

Fusion

Ontologies

Information

• Entity Metadata

• Feature Metadata

Knowledge

• Object-Event Relations

• Spatiotemporal Associations

• Provenance Pathways

Sensors (RF, EO, IR, HIS, acoustic)

• Object-Event Ontology

• Space-Time Ontology

Analysis Processes

Annotation Processes

Sensor Data Architecture

Current Research Towards STT Relationship Analysis

Modeling Spatial and Temporal data using SW standards (RDF(S))1

–Upper-level ontology integrating thematic and spatial dimensions

–Use Temporal RDF3 to encode temporal properties of relationships

–Demonstrate expressiveness with various query operators built upon thematic contexts

1. Matthew Perry, Farshad Hakimpour, Amit Sheth. "Analyzing Theme, Space and Time: An Ontology-based Approach", Fourteenth International Symposium on Advances in Geographic Information Systems (ACM-GIS '06), Arlington, VA, November 10 - 11, 2006

2. Matthew Perry, Amit Sheth, Farshad Hakimpour, Prateek Jain. “Supporting Complex Thematic, Spatial and Temporal Queries over Semantic Web Data", Second International Conference on Geospatial Semantics (GeoS ‘07), Mexico City, MX, November 29 –30, 2007

3. Claudio Gutiérrez, Carlos A. Hurtado, Alejandro A. Vaisman. “Temporal RDF”, ESWC 2005: 93-107

Current Research Towards STT Relationship Analysis

Graph Pattern queries over spatial and temporal RDF data2

–Extended ORDBMS to store and query spatial and temporal RDF

–User-defined functions for graph pattern queries involving spatial variables and spatial and temporal predicates

–Implementation of temporal RDFS inferencing

1. Matthew Perry, Farshad Hakimpour, Amit Sheth. "Analyzing Theme, Space and Time: An Ontology-based Approach", Fourteenth International Symposium on Advances in Geographic Information Systems (ACM-GIS '06), Arlington, VA, November 10 - 11, 2006

2. Matthew Perry, Amit Sheth, Farshad Hakimpour, Prateek Jain. “Supporting Complex Thematic, Spatial and Temporal Queries over Semantic Web Data", Second International Conference on Geospatial Semantics (GeoS ‘07), Mexico City, MX, November 29 –30, 2007

3. Claudio Gutiérrez, Carlos A. Hurtado, Alejandro A. Vaisman. “Temporal RDF”, ESWC 2005: 93-107

Upper-Level Ontology Modeling Theme and Space

OccurrentContinuant

Named_PlaceSpatial_OccurrentDynamic_Entity

Spatial_Region

Occurrent: Events – happen and then don’t existContinuant: Concrete and Abstract Entities – persist over time

Named_Place: Those entities with static spatial behavior (e.g. building)

Dynamic_Entity: Those entities with dynamic spatial behavior (e.g. person)

Spatial_Occurrent: Events with concrete spatial locations (e.g. a speech)

Spatial_Region: Records exact spatial location (geometry objects, coordinate system info)

occurred_at located_at

occurred_at: Links Spatial_Occurents to their geographic locationslocated_at: Links Named_Places to their geographic locations

rdfs:subClassOfproperty

Spatio-Temporal-Thematic Query Processing @ Kno.e.sis

OccurrentContinuant

Named_Place

Spatial_OccurrentDynamic_Entity

PersonCity

Politician

SoldierMilitary_Unit

BattleVehicle

Bombing

Speech

Military_Eventassigned_to

on_crew_ofused_in

participates_in

trains_at

Spatial_Region

located_at occurred_at

Upper-level Ontology

Domain Ontology

rdfs:subClassOf used for integrationrdfs:subClassOfrelationship type

Temporal RDF Graph: Platoon Membership

E1:Soldier

E3:Platoon E5:Soldier

E2:Platoon

E4:Soldierassigned_to [1, 10]

assigned_to [11, 20]

assigned_to [5, 15]

E1 is assigned to E2 from time 1 to 10 and then assigned to E3 from time 11 to 20

Also need to handle inferencing:

(x rdf:type Grad_Student):[2004, 2006] AND(x rdf:type Undergrad_Student):[2000, 2004]

(x rdf:type Student):[2000, 2006]

Time interval represents valid time of the relationship

ORDBMS Implementation: DB Structures

Unlike thematic relationships which are explicitly stated in the RDF graph, many spatial and temporal relationships (e.g., distance) are implicit and require additional computation

Sample STT Query

Scenario (Biochemical Threat Detection): Analysts must examine soldiers’ symptoms to detect possible biochemical attackQuery specifies a relationship between a soldier, a chemical agent and a battle location (graph pattern 1)a relationship between members of an enemy organization and their known locations (graph pattern 2)a spatial filtering condition based on the proximity of the soldier and the enemy group in this context (spatial Constraint)

Going places …

Formal

Social,InformalImplicit

Powerful

looking into my crystal ball

TECHNOLOGY THAT FITS RIGHT IN

“The most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it.

Machines that fit the human environment instead of forcing humans to enter theirs will make using a computer as refreshing as a walk in the woods.” Mark Weiser, The Computer for the 21st Century (Ubicomp vision)

Get citation117

imagine

imagine when

Farm Helper

with this

Latitude: 38° 57’36” NLongitude: 95° 15’12” WDate: 10-9-2007Time: 1345h

that is sent to

Geocoder

Farm Helper

Services Resource

Sensor Data ResourceStructured Data Resource

Agri DBSoil

Survey

Lat-Long

Soil InformationPest information …

Weather Resource

LocationDate /Time

Weather Data

Six billion brains

imagination today

impacts our experience tomorrow

Computing For Human Experience

Prof. Amit P. Sheth,LexisNexis Eminent Scholar and

Director, kno.e.sis centerWright State University

http://knoesis.org

relationship web: trailblazing, analytics and computing ... · wright state university core scholar...

Documents

trailblazing shorter paths to plm value

trailblazing timeline

web and complex systems lab @ kno.e.sis

linked sensor data harshal patni, cory henson, amit p. sheth...

krishnaprasad thirunarayan, pramod anantharam, cory a....

trailblazing, complex hypothesis evaluation, abductive...

trailblazing in the wilderness of data management

herrenknecht trailblazing tunnelling technology for the...

trailblazing summer 2013(b)

exciting, trailblazing start-ups and established and...

prateek jain dissertation defense, kno.e.sis, wright state...

analyzing the social media footprint of street gangs 1 ohio...

new year’s message: a trailblazing blueprint

trailblazing special libraries

trailblazing with the mi volunteer registry

changing focus on interoperability in information systems -...

trailblazing summer 2013

home sale will support trailblazing musicians

introduction to kno.e.sis center - march 2011

trailblazing in entrepreneurship - oapen