Semantic Knowledge and Privacy in

the Physical WebPRAJIT KUMAR DAS, ABHAY KASHYAP,

GURPREET SINGH, CYNTHIA MATUSZEK, TIM FININ, ANUPAM JOSHI

UMBC ebiquity and IRAL Labs

Motivation

Our goal is to provide contextually aware information, using the IoT, that is privacy preserving and ubiquitously helpful

Image courtesy Batman Wikia

CARLTON

Slide 2 of 44

IoT by Volume

Slide 3 of 44

IoT by Domain

Slide 4 of 44

IoT by Privacy Concerns

Slide 5 of 44

Salient features CARLTON: A context-aware, NL question-

answer BOT Context derived from the Physical Web

(IoT) Under development, prototype system Simple NLP using tools like Stanford

CoreNLP Mobile app and Kiosk for front-end ABAC privacy model, Privacy rules using

SWRL Hierarchical context ontology Optional authentication for UMBC people

Slide 6 of 44

Concretization of IoT

Small, quick seamless

interactions with

physical objects and

locations with your

device

Physical web: What?

Slide 7 of 44

Physical web: What? Everything is a tap

away

Slide 8 of 44

Physical web: What? See what’s useful

around you

Slide 9 of 44

Physical web: What? Any object or place

can broadcast

content

Slide 10 of 44

Physical web: How?Three main techniques

Nearby Connections

Slide 11 of 44

Physical web: How?Three main techniques

Nearby Connections

Nearby Notifications

Slide 12 of 44

Physical web: How?Three main techniques

Nearby Connections

Nearby Notifications

Nearby Messages

Slide 13 of 44

System Overview

Slide 14 of 44

System Overview

Slide 15 of 44

System Overview

Slide 16 of 44

Who is Tim Finin?

System Overview

Slide 17 of 44

“Tim Finin”: Person Entity type

“Who”: WH query type

Text to Semi-Structured Text

Intent

Who is Tim Finin?

System Overview

Slide 18 of 44

“Tim Finin”: Person Entity type

“Who”: WH query type

Text to Semi-Structured Text

Intent

SPARQL query generatorContext

Who is Tim Finin?

System Overview

Slide 19 of 44

“Tim Finin”: Person Entity type

“Who”: WH query type

Text to Semi-Structured Text

Intent

SPARQL query generatorContext

Who is Tim Finin?

Inference EngineOntologyKnowledge base

System Overview

Slide 20 of 44

“Tim Finin”: Person Entity type

“Who”: WH query type

Text to Semi-Structured Text

Intent

SPARQL query generatorContext

Who is Tim Finin?

Inference EngineResponse: JSON{“text”: “He’s a Professor in the

Computer Science department!”}

OntologyKnowledge base

System Overview

Slide 21 of 44

Example query

Slide 22 of 44

Is this room booked from

2PM-3PM?

Example query

Slide 23 of 44

User is a faculty and is in front of Conf. room 1.

Is this room booked from

2PM-3PM?

Example query

Slide 24 of 44

Conf. room 1 calendar has no events during that time.

Is this room booked from

2PM-3PM?

Example query

Slide 25 of 44

Is this room booked from

2PM-3PM? No, would you like me to book it from

2PM – 3PM?

Example query

Slide 26 of 44

Is this room booked from

2PM-3PM? No, would you like me to book it from

2PM – 3PM?Yes, please!

Example query

Slide 27 of 44

Okay, the room has been booked

in your name from 2PM – 3PM

Is this room booked from

2PM-3PM? No, would you like me to book it from

2PM – 3PM?Yes, please!

Example query

Slide 28 of 44

Example query

Slide 29 of 44

Is Dr. Joshi here?

Example query

Slide 30 of 44

Is Dr. Joshi here?

User could mean Dr. A. Joshi or Dr. K. Joshi.

Example query

Slide 31 of 44

Is Dr. Joshi here?

But user is in front of Dr. A. Joshi’s office.

Example query

Slide 32 of 44

Is Dr. Joshi here?

User is an advisee of Dr. A. JoshiExample query

Slide 33 of 44

Is Dr. Joshi here?

Dr. Joshi is in a meeting till

3PM

Example query

Slide 34 of 44

Example query

Slide 35 of 44

Example query

Where is Dr. Finin’s office?

Slide 36 of 44

Example queryUser is in CSEE building

Where is Dr. Finin’s office?

Slide 37 of 44

Example query User is unknown to system

Where is Dr. Finin’s office?

Slide 38 of 44

Example query

Where is Dr. Finin’s office?

Please see CSEE front desk for

required information

Slide 39 of 44

Example1.@prefix crltn:<https://www.ebiquity.org/ontologies/carlton/0.1>.@prefix swrlb:<http://www.w3.org/2003/11/swrlb>.crltn: student(?requester)∧(

crltn: supervises(“Xavier”,?requester)∨(crltn: affiliatedWith(?requester,?labName) crltn: leads(“Xavier”,?labName))∧

)∧crltn: hasCurrentLocation(?requester,?aBldgLocation)∧crltn: room(?aBldgLocation) crltn: sitsIn(“Xavier”,?aBldgLocation)∧ ∧crltn: currentTime(?currTime) swrlb: Exists(?anEvent) crltn: speakingAt(“Xavier”,?anEvent)∧ ∧ ∧(

(crltn: startTime(?anEvent,?eventStartTime) swrlb: greaterThan(?eventStartTime,?∧currTime))∨

(crltn: endTime(?anEvent,?eventEndTime) swrlb: greaterThan(?currTime,?eventEndTime))∧) crltn: hasCurrentLocation(“Xavier”,?aLocation) crltn: Location(?aLocation)∧ ∧ ∧crltn: requestLocation(“Xavier”)⇒shareLocation(?aLocation)

Policy Example

Slide 40 of 44

supervises(“Xavier”,?requester)OR(

affiliatedWith(?requester,?labName)ANDleads(“Xavier”,?labName)

)

Policy Example

Slide 41 of 44

hasCurrentLocation(?requester,?aBldgLocation) ANDroom(?aBldgLocation) AND sitsIn(“Xavier”,?aBldgLocation)=>shareLocation(?aLocation)

Policy Example

Slide 42 of 44

Future work Prototype system constantly adding

conversations Beacons on robots Reason over robots near you How robots respond to instructions?

“Can you take me to Prof. Matuszek now?”

“Show me the way to the ITE 346 conference room”

Slide 43 of 44

Summary We presented CARLTON A context-aware, NL question-answer BOT Context derived from the Physical Web (IoT) Semantic web technologies used to preserve

data privacyThanks to NSF for the travel grant!

and Thanks to Google for the gift of

beacons! Slide 44 of 44