towards(stream-based(reasoning(and( …endler/talks/intellisys-2017.pdf · intellisys 2017 . 19...
TRANSCRIPT
Laboratory for Advanced Collaboration
L A C
Towards Stream-‐based Reasoning and Machine Learning for IoT Applica<ons
Markus Endler1, Jean-Pierre Briot2, Francisco Silva e Silva3, Vitor P. Almeida1 , Edward H. Haeusler1
1 PUC-Rio, Rio de Janeiro, Brazil
2 LIP6, Paris, France 3 UFMA, São Luiz, Brazil
2
Towards Real-‐<me Intelligence for IoT
General goals: • Get a better understanding of what is happening in the physical world; • Correlate apparently unrelated sensed events; • Predict events that have not been sensed yet; • Identify consequences of sensed events. • Send out correct and timely actuation commands or notifications to humans.
www.iotglobalnetwork.com/iotdir/2017/05/08/
4
Challenges of IoT Intelligence
§ Sensors are multi-modal, distributed and heterogeneous § Their data is noisy and incomplete § Associated with time and space (Context) § Availability and reliability are dynamic § Must be processed fast – several IoT applications require
“soft” real-time analysis § Privacy and security are important issues
But data analysis alone is not sufficient! IoT demands ac7onable informa7on based on domain knowledge.
IntelliSys 2017
6
Outline
§ The ESMOCYP Approach § Example of Scenario § Implemented Prototype § Machine Learning Prospects § Conclusion
IntelliSys 2017
7
Project ESMOCYP(1): Merging three disciplines
& Data Stream Processing
(Complex Event Processing)
Cyber-Physical Systems (IoT)
Semantic Web & Reasoning
IntelliSys 2017 (1) Efficient Semantic MOdels and Fault-tolerant Middleware for CYber-Physical Systems
10
The ESMOCYP Approach
Goal: achieve online reasoning as a combination of CEP and Deduction Logic reasoning Main steps: 1. Semantic annotation: Identify the type of entities (smart
objects) and the action happening with it 2. Generate RDF streams (Semantic Event stream) from
annotated sensor data 3. Use these Semantic Events to generate facts (Fact
stream) using the Knowledge Base (KB), and also input new facts into the KB
4. In offline mode, process the KB and derive new Event Processing rules for online analysis
ESMOCYP relies on the ContextNet middleware [Endler et al, Middleware 2011], that has cloud and mobile components.
IntelliSys 2017
11
M-EPA: Mobile Event Processing Agent
ContextNet services in the Cloud
§ Mobile Hub [Rios/Endler:2015] is a middleware component for the Internet of Mobile Things (IoMT)
§ M-EPA [Rios/Endler,2016] allows the Mobile Hub to pre-process “local” sensor data streams and send to the cloud just relevant information
§ Performs Complex Event Processing (CEP) using ESPER engine • Can identify relevant event patterns and generate complex events • Can receive and deploy new CEP/EPL rules on the fly
12
Semantic Annotation of Simple Sensor Events/Data § The M-EPA (Mobile Event Processing Agent) identifies the entity/
object and the action performed with the smart object § The entity is identified by a mapping of UUID (Universal Unique
Identifier) to ontology concepts § The action is identified by the analysis of sensor data signal (e.g.
value)
(pr,sub)
UUID Mapping
Sensor data events
UUID is cell phoneY or
Action on entity: e.g. kicking
Predicate: moved Subject: cell phone
UUID value
UUID value
UUID value
Mobile EPA
IntelliSys 2017
UUID is temp. by sensorW
13
From Data Event Stream to Semantic Event Stream
(pr,sub) (pr,obj) (pr,obj) (pr,sub)
Context mapper A
(sub,pr,obj) (sub,pr,obj) (sub,pr,obj)
Subject +Action Object +Action
Context-specific RDF triples Semantic Event Stream
Annotated Sensor Event Stream
Annotated Sensor Data Context
mapper B Spatio-
temporal correlations
IntelliSys 2017
Main Idea: An action (predicate) of a subject (sub) on an object (obj) occur at a unique instant of time and in a same place.
14
From Data Event Stream to Semantic Event Stream
(pr,sub) (pr,obj) (pr,obj) (pr,sub)
Annotated Data Event Stream
Context mapper CEP rules link subjects and objects that have same/related predicates (actions), same timestamp and same location. This correlation is done by the M-EPA (close to the sensors)
< , t , location > < , t, context>
RDF +Timestamp + Conetxt Attributes
(pr,sub)
< , t , location > (pr,obj)
Time window
≅ ≅ (sub, pr, obj)
IntelliSys 2017
15
(sub,pr,obj) (sub,pr,obj) (sub,pr,obj)
Semantic Event Rule
A
FACT A FACT C FACT B
(sub,pr,obj) (sub,pr,obj) (sub,pr,obj)
(sub,pr,obj) (sub,pr,obj) (sub,pr,obj) Sub
stream 1
Sub stream 2
Sub stream 3
Knowledge facts
…
Semantic Event Stream (RDF)
Fact Stream
Semantic Event Rule
B Knowledge
Base
From Semantic Event Stream to Deduced Fact Stream
IntelliSys 2017
16
Example of Scenario § Mr. Silva carries a smartphone running MobileHub and
travels by bus with air conditioning and smart sensors § MobileHub automatically discovers and connects to
nearby smart sensors (with temperature and accelerometers)
§ Analyzed data (from sensor location, accelerometer...): ambient condition (temperature), movement pattern...
Deduced information: § Silva is riding on Bus Line #435 § Silva is in an air-conditioned ambient
IntelliSys 2017
17
Example of Scenario RDF events in the seman<c stream: § (cell-‐phoneY, connectedTo, sensorX), § (sensorX, reads, Abs(Accelerometer) > 10), § (sensorX, reads, temp=20) Knowledge Base facts: § (Silva, carries, cell-‐phoneY) § (sensorX,locatedIn, BusKKZ8674), § (BusKKZ8674, serves, BusLine 435). DL Rules: § (cell-‐phone, connectedTo, sensor) ∧ (sensor, in, ambient) => (cell-‐phone,
in, ambient) § (person, has, cell-‐phone) ∧ (cell-‐phone, in, ambient) => (person, in,
ambient) § (person, in, ambient) ∧(ambient, isA, bus) ∧(bus, servers, line) =>
(person, riding, line) § (person, in, ambient) ∧ (sensor, in, ambient) ∧ (sensor, reads, data) =>
(person, experiences, data)
IntelliSys 2017
18
Prototype
Level 1 uses Esper: receives the UUID of the Bluetooth sensor/beacon and the temperature values. By matching a sequence of data it will generate RDF triples with the entities’ type information.
Level 2: runs C-SPARQL (a continuous stream reasoning engine) that uses the KB. i.e., new facts can be deduced doing reasoning on their type hierarchy and their relationship to other objects in the application domain
KB: Ontholgy has relations between entities (people, roles, devices, rooms, reachability, and time)
IntelliSys 2017
19
Technologies used in the Prototype
§ ESPER • ESPER Event Processing Language (EPL) • EPL: declarative (SQL-like) language with pattern language and sliding
window • Queries on Java objects and their attributes
§ C-SPARQL • Continuous/Stream Extension of SPARQL (SPARQL Protocol and RDF
Query Language) • Queries on stream of RDF triples
§ Mobile Hub • Android-based middleware for opportunisctically detecting, connecting and
retrieving sensor data from Bluetooth Smart SensorTags
§ M-EPA • Android-based ESPER CEP engine where EPL rules can be downloaded
and managed remotely (from the cloud)
IntelliSys 2017
20
Machine Learning Prospects
Machine Learing is more appropriate for sta7s7cal analysis, and therefore should be employed in the early phases of the reasoning, e.g. to detect which phenomenon is occuring with the subjects/objects. Some ideas are:
1. Extract Features from Sensors Data § Autoencoders, RBMs
2. Extract Paaerns from Features § k-‐Means
3. Learn Temporal Series of Paaerns § Recurrent Networks, Markov Chains…
Ex: SensorSax [Ganz et al. 2016]: 1; 2; 3 4. Deduc7on of abstract Facts and Rules
§ Induc7ve Logic Programming
IntelliSys 2017
22
Conclusion
Project ESMOCYP is exploring real-‐7me reasoning for IoT systems, by combining Complex Event Processing and knowledge-‐based reasoning. § So far, we are just exploring the technological feasibility § Our prototype implementa7on has shown promising results
But… many problems remain to be solved: § How to handle facts that are only transient? § What happens if the deduc7on of new facts is made on stale
knowledge § Knowledge base should have a no7on of 7me (for rela7ons and
rules) § How to deal with large ontologies? § It is possible to “clip/cut off” the relevant part of the ontology? § How to merge the modified clipped part into the whole
ontology?
23
Thank you!
Questions? Acknowledgement: CAPES-DAAD PROBRAL Cooperation Program For more information please concact:
[email protected] http://www.lac.inf.puc-rio.br
IntelliSys 2017