Download - 20131031 giis 2013 keynote r.giaffreda
Cognitive Management of Objects and
Applications for the Internet of Things
Raffaele Giaffreda (CREATE-NET)Raffaele Giaffreda (CREATE-NET)
Keynote at GIIS conference
Trento, 31 Oct 2013
Outline
• Introduction, IoT vs. the Internet
• object virtualisation – separation between object data and object mgmt concerns
• overview of IoT standardisation activities
• interoperability and objects as services – IoT reliability and resilienceresilience
• a top-down perspective on the IoT – user friendliness, wide adoption
• Real World Knowledge modelling and use of cognitive technologies in IoT
• examples of ongoing trials
• conclusions
transistor density / space efficiency
Turing’s Pilot ACE: Automatic
Computing Engine
bandwidth / spectral efficiency
a bit of IoT infographics...
BOSCH
7 bln connected devices by 2015
SAP
24 bln connected devices by 2020
INTEL
31 bln connected devices by 2020
CISCO
37-50 bln connected devices by 2020
others...
Source: IDATE
some initial considerations
• IoT will be BIG
• problems
• human in the loop
• configuring, using, maintaining• configuring, using, maintaining
• handling huge amounts of data produced
the Internet parallel
• imagine the Internet with no browser, no
plugins
• collection of bespoke, non interoperable
content specific applications enabling access content specific applications enabling access
and visualisation of connected files
The Internet parallel
HTTP/WWW
search engines
connect your info
TCP/IP
HTML
represent info / aggregate info
WWWpersonalised knowledge
collections, blogs...
The Semantic Web
find info
VALUE!
The Internet parallelearly stages for the IoT...
HTTP/WWW
The Semantic Web
VALUE!
personalised knowledge
collections, blogs...
represent info / aggregate info
search engines
connect your info
TCP/IP
HTML
WWW
find infoobject
today
Internet vs. Internet of Things
• files vs. objects
• static memory cells vs. energy standalone
units
• need to separate data source from data mgmt • need to separate data source from data mgmt
and operations
• objects virtualisation
Introducing Virtual Objects
VO SW agent
VO APIs
(remote) proprietary API calls
VO SW agent host
the VO concept
• VO exposes several APIs to the upper
layers
– Features, functionalities and resources
can be re-used
• Cognitive control enabled by exposing
APIs which can be used to optimize the
behaviour of the ICT object
VO SW agent may or may not be
Exposed APIs
ICT
object processes
ICT APIs
API calls
ICT object
A simple example: VO SW agent host = laptop with Zigbee dongle
ICT object = Zigbee temperature meternon-ICT object = room
• VO SW agent may or may not be
installed on the ICT object
– Depends on ICT object capabilities
• Association management between ICT
and non-ICT is a real challenge!
• RESILIENCE ASPECTS
Association
non-ICT object17
what do VOs achieve: logical level
iCore FW
Application: pure function
VO Front VO Front
End
VO Front VO Front
End
VO Front VO Front
End
18
Gateway
VO Back
End:
Net Driver
VO Back
End:
Net Driver
VO Back
End:
RWO Driver
VO Back
End:
Net Driver
RWO1
fostering automation - discovery
• description associated with an IoT Object, it better be
machine readable
• i.e. semantic enrichment based on info model for semantic-
based selection
• what is this good for? • what is this good for?
– selection “by relevance”: performance and “selection quality” is
dependent upon combination of enrichment + algorithm that exploits
it...
– assessment of “proximity” is a prerequisite in achieving more
automatic and scalable solutions
VO Information Model – semantic
search
Examples (energy efficiency for
sensors)
• besides discovering and selecting
• virtual representative “takes the heat off” real sensors becoming their actual “manager”– energy efficiency
– reuse– reuse
– resilience
– self-x for constrained resource devices
• conflict resolution (actuators)
• Examples– compression algorithms, data caching, pub/sub
schemes, rules for self-x management
HUMANMACHINE
added value besides sensing efficiency
what happens when it becomes
easier and easier to tap into
object produced data?
cars increasingly more
complex
increasing competition
for owner’s attention
OBD
On Board Diagnostics
HUMANMACHINE
added value besides sensing
efficiency – Innovation potential
“Innovation”: one
can focus on apps!!!
we make “machines” step-in, assisting us!
MACHINEHUMAN
OBD
On Board Diagnostics
the story so far...
• increasing number of objects
• discovery and self-management of objects
• connect and virtualise your objects, unlock
valuevalue
• no mention of application domains...
siloed and bespoke IoT applications
APPS
HO
US
E
APPS
FR
IDG
E
APPS
PA
TIE
NT
APPS
PA
TIE
NT
APPSAPPS
APPSAPPS
APPSAPPS
DATA / INFORMATION OVERLOAD, BUT...
CA
R
SENSORS
HO
US
E
SENSORSF
RID
GE
SENSORS
PA
TIE
NT
SENSORS
PA
TIE
NT
SENSORS
PA
TIE
NT
SENSORS
PA
TIE
NT
SENSORS
PA
TIE
NT
SENSORS
PA
TIE
NT
SENSORS
APPS
PA
TIE
NT
SENSORS
APPS
TR
UC
K
SENSORS
APPS
IF A WELL-DEFINED INTERFACE INTO CAR
SENSORS BRINGS SUCH POTENTIAL...
APPS
HO
US
E
APPS
FR
IDG
E
APPS
PA
TIE
NT
APPS
PA
TIE
NT
APPSAPPS
APPSAPPS
APPSAPPS
CA
R
HO
US
E
FR
IDG
E
PA
TIE
NT
PA
TIE
NT
SENSORS
PA
TIE
NT
SENSORS
PA
TIE
NT
SENSORS
PA
TIE
NT
SENSORS
PA
TIE
NT
SENSORS
APPS
PA
TIE
NT
SENSORS
APPS
TR
UC
K
SENSORS
APPS
SENSORS SENSORS SENSORS SENSORS
of course that’s a dream far from
becoming true...
http://readwrite.com/2013/06/14/whats-holding-up-the-internet-of-things
the IoT standardisation jungle
Real-World Information DB
Au
then
tica
tio
n
Au
then
tica
tio
n
Real-World Knowledge Model (RDF Concepts & Facts)
GUI
Service Requester (Technology Agnostic)
Situation Awareness
Situation Projection
Situation Classification
Situation Recognition
Situation Detection
Queried Fact Collector
Natural Language Processing
(NLP)
Service Request
(SPARQL)
API
Service Request Analysis
RDF Rules
Inference
Engine
Semantic
Query
MatcherLearning
Statistics
Service Analysis
User Characterisation
Learning
Mechanisms
Intent Recognition
Service Templates
Repository
CVO FactoryAP
I
Domain
Expert /
Knowledge
Engineer
AP
I
Ad
min
istr
atio
n &
Man
agem
ent
I/F
Authentication
Authentication
CVO Management Unit
Service Execution Request
M2M
M2MM2M
M2M
M2M
CLOUD
P1723
EPCGlobal
SES
W3C PROV
PROV-DM /PROV-O /
PROV-AQ / PROV-XML
W3C PROV
PROV-DM / PROV-O /
PROV-AQ / PROV-LINKW3C PROV
PROV-DM /PROV-O /
PROV-AQ / PROV-XML
W3C PROV
PROV-DM / PROV-O / PROV-
AQ / PROV-CONSTRAINT
ITU-T
IoT-GSI
ITU-T
FG M2M
ITU-T
FG DistractionISO/IEC
JTC1 WG7
CVO Container (Execution)
Au
then
tica
tio
n
Au
then
tica
tio
nA
uth
enti
cati
on
VO Factory
VO Templates Repository
CVOCVO
CVOCVO
CVO
CVO
Situation ObserverCVO
Situation ObserverCVO
Situation ObserverCVO
Situation Observer
System Knowledge Model
CVO Registry
CVO Templates Repository
VO Registry
VO Management Unit
VO Lifecycle
ManagerCoordination
Data Manipulation
/ Reconciliation
Resource
OptimisationVO Container (WS host)
VOVOVOVOVOVOVOVOVOVO
VO Front End
VO Back End: RWO Driver
Approximation &
Reuse
Opportunity
Detection
CVO Composition
Engine
Orchestration /
Workflow
Management
Learning
Mechanisms
Device
manufactu
rer
API
Resource SensorActuator
GTW/Controller
Data
Processing
Domain
Expert /
Developer
Access
Control
Access
Control
Ad
min
istr
atio
n &
Man
agem
ent
I/F
Installer/
User
Installs
Sensor/Ac
tuator
Devices
Resource SensorActuator
GTW/Controller …………..
….. …..
CVO Lifecycle
ManagerCoordination
Quality
Assurance
Performance
Management
CVO Execution Request
M2M
M2M
M2M
CLOUD
CLOUD
CoAP
CoAP
SSN-XG
MQTT
MQTT
3GPP
3GPP
3GPP 3GPP
EPCGlobal
SPS
SOSSES SAS
SPS
SOS
SPS
SASWNS
SPSSOS
SOS
SOR SIR
CSW
O&M
SensorML
W3C PROV
PROV-DM /PROV-O /
PROV-AQ / PROV-XML
W3C PROV
PROV-DM / PROV-O / PROV-
AQ / PROV-CONSTRAINT
W3C PROV
PROV-DM /PROV-O /
PROV-AQ / PROV-XML
EPCGlobal
ITU-T
IoT-GSI
ITU-T
IoT-GSI
ITU-T
IoT-GSI
ITU-T
IoT-GSI
ITU-T
FG M2M
ITU-T
FG M2M
ITU-T
FG Distraction
ISO/IEC
JTC1 WG7
ISO/IEC
JTC1 WG7
ISO/IEC
JTC1 WG7
LWM2M
LWM2M
LWM2M
LWM2M
LWM2M
LWM2M
LWM2M
courtesy of Panagiotis Vlacheas and Vera Stavroulaki (Piraeus University )
some (good) candidates
• imagine the Internet with no browser, no plugins
• collection of bespoke, non interoperable content
specific applications enabling access and
visualisation of connected filesvisualisation of connected files
an IP based web services view from Sensinode
http://www.iot-week.eu/presentations/thursday/02_Shelby-IoT-Smart-Cities.pdf
Courtesy of Zach Shelby (Sensinode)
fostering interoperability
• at service level (ESBs)
• at communication level (PUB/SUB MQTT bus)
• at device level (GSN)
• no silver bullet...a lot of it will depend on
application context...
useful ingredients?
• common interfaces to interact with
objects (i.e. REST)
• + extra containers for metadata
• let the systems know what the object • let the systems know what the object
is good for, its location (“I am a Temp
sensor in Room A”), its accuracy, its
energy levels etc.
take inspiration from HTML and the Semantic Web
“I am a webpage and I talk about Paris (city of France) history”
Integration at “application level” with all pros and cons associated with it
the story so far...
• increasing number of objects
• discovery and self-management of objects
• connect and virtualise your objects, unlock
valuevalue
• interoperability across application domains
and reliability still big issues...
once achieved the means to access
an objects as a service...
• object redundancy would allow me to cope with resource constraint nature of objects as well as with the diversity of interfaces– if I had a bunch of VO temp objects to chose from I would be much
more likely to tell you what the temperature is...
• semantic enrichment allows me to find alternatives, to foster object reuse and achieve service approximation conceptsobject reuse and achieve service approximation concepts
• here we start entering more the “cognitive-inside” IoT object management territory
• having a logic for choosing the appropriate Virtual Objects according to the application expectations
• having the means to easily connect objects together in a more or less complex graph (CEPs, PUB/SUB channels)
• features of Composite Virtual Objects and associated “CVO Templates”
cognitive mash-ups of semantically interoperable VOs (and their offered
services) which render services matching the application requirements
Introducing the CVO
CVO concept allows for approximate services...
CA
R
APPS
HO
US
E
APPS
FR
IDG
E
APPS
PA
TIE
NT
APPS
HO
US
E
FR
IDG
E
PA
TIE
NT
SENSORS SENSORS SENSORS SENSORS
PATIENT is near the FRIDGE
CAR is near the HOUSE
PATIENT is driving the CAR
objects reuse
across domains
KitchenPresDetect PatientStatusDetect
CVOs allow Automatic Composition
CVOType 1
VOType :: Temp sensor
getTemp()
Subject to constraints:
- Dist (Pos, myPos) < 10m
CVO 1
VOx
FIND
getTemp()
VOType :: Press sensor
getPressure()
- Dist (Pos, myPos) < 10m
- Not already allocated
Logic:
If getTemp() > 20° and getPressure() > 2bar
then NiceWeather
VOy
CP Solver to find VO
allocations that satisfies
all constraints and
minimizes network traffic
USE
leveraging on System Knowledge (i.e. VOx is good and fully
charged) to maintain IoT-based services...
CVO templates
• factoring “smart logic algorithms” out of users /
developers concerns
– IF “crash” THEN “alertRSA”
– “crash” (IF VO_x = TRUE THEN crash := TRUE)
– (IF VO_x = TRUE AND VO_y = TRUE THEN crash := TRUE)
TAG:
crash
detect
VO_x
TAG:
crash
detect
VO_yIF (VO_x = TRUE) AND (VO_y = TRUE)
THEN crash := TRUE
IF VO_x = TRUE
THEN crash := TRUE
IF (VO_x > TH_x) AND (VO_y > TH_y)
THEN crash := TRUE
factor out cognitive technologies
• “ready meals” for IoT apps
workflow-based SEP for CVOs
Car’s sensors/actuators
Open Data (Web)
courtesy of Michele Stecca (M3S)
more info: http://www.slideshare.net/steccami/ieee-icin-2011
Event based CVO execution
CVO ContainerCVO Container
CVOCVO CVO CVO
CEP engineCEP engine
Observer
Observer
Machine Learning
extensions
Machine Learning
extensions
Event / (C)VO Bus (pub/sub based on MQTT)
Event / (C)VO Bus (pub/sub based on MQTT)
CEP engineCEP engine
VO ContainerVO Container
Sensor
VO
Sensor
VO
Sensor
VO
Sensor
VO
Sensor
VO
Sensor
VO
Actuator
VO
Actuator
VO
Actuator
VO
Actuator
VO
courtesy of Walter Waterfeld (Software AG)
more info: http://terracotta.org/downloads/universal-messaging
Internet vs. IoT
• a page + a page + a page...connect info
• represent info – HTML
• aggregate info – hyperlink
• a (sensor) feed + a feed + a feed...
• represent feeds – VO
• aggregate feeds – CVO
the story so far...bottom-up
what’s in here?
user friendliness and user friendliness and
wide adoption...
the story so far...
• increasing number of objects
• discovery and self-management of objects
• connect and virtualise your objects, unlock value
• exploit redundancy pick the most suitable / interoperable / reliable objects interoperable / reliable objects
• VO / CVO services like Lego bricks fostering innovation from IoT makers
• cognitive inside? so far only application-driven match-making
• ultimate goal: user-friendly IoT services fostering wide adoption
a ‘top-down’ view
• routine jobs: water the plants, feed the fish, take my pills, track sent items etc.
• there are objects, sensors, actuators
• there are people (busy lives, forgetful patients, green fingers vs. fingers that “kill every plant they look after”)every plant they look after”)
• objects can be connected
• objects can be mashed-up
• create your own IoT apps (this is what IoTmakers do) vs. provide some input and have this interpreted so the right actions are set to achieve your goals
• make the IoT easy to use and rely upon...
unlocking a huge potential
data
data datada
ta
da
ta
datapresence
patterns exist ...
CVOs
derive patterns of ...
interpret
data
da
ta
data data
data
da
ta data
data
da
ta
datadata
data
data
H/WSE
NS
ING
Real World Objects
(RWO)
VOs
data goldmine
and lots of
siloed
applications
presence
derive patterns of ...
it’s a complex IoT world...
the need for cognitive technologies
• rather than for the selection of appropriate templates,
here focus is on refinement of selected one according
to observed system-reality matching
• Real-World-Knowledge “growing”
• Learning and adaptation to the users preferences• Learning and adaptation to the users preferences
TAG:
crash
detect
VO_x
TAG:
crash
detect
VO_yIF (VO_x > TH_x) AND (VO_y > TH_y)
THEN crash := TRUE
assess
QUALITY of
PREDICTION
REFINE
TH_x, and TH_y
Real World and System Knowledge
models
Real World Knowledge
(RWK)
Models
derive patterns of ...
interpret
data
System Knowledge
(SK)
Models
presence
derive patterns of ...
What are these
good for?
Cognitive Inside where and why...
• Service Level: gather data relate to actions /
situations
• support users (OBSERVE – LEARN – REPLACE)
– routine jobs (watering plants, feeding the fish, taking – routine jobs (watering plants, feeding the fish, taking
pills, switch on/off lights)
– non-expert alerts (a fire, a leak, a fault)
• provide feedback
– improvement of system performance
Some examples please?
• tracking cars in a smart city
• medical equipment tracking and asset
management
tracking cars in a smart city
Best demo
award at
FuNeMS 2013
courtesy of Marc Roelands (Bell Labs – Alcatel Lucent)
more info: http://www.iot-icore.eu/attachments/article/66/iCore_FuNeMS%2713_ALU.pdf
tracking medical equipment
Validate
Train
Execute
RWO parameter reconfiguration
recommendations to improve energy
efficiency of location sensorsIn the demo implementation,
Database of location
information(spatial &
temporal) of objects
3
4
5
Train efficiency of location sensorsIn the demo implementation,
location data of objects is
simulated
1
2
2a4a
6
7
Trento Hospital S. Chiara
Trilogis + ZIGPOS
IoT, Cloud and Big Data
the challenges ahead...
• Big data: “big” relates to the huge number of data sources – have data, patterns exist
– Need to purposefully aggregate data
– scaling-up use of machine learning is a challenge...
• Cloud: constrained devices and limited scope for data • Cloud: constrained devices and limited scope for data processing– dynamic deployment of data-processing resources on the
data-source � data-consumer path is a challenge...
• IoT Networking: delivery of “object-produced” data– M2M traffic and dynamic deployment of connectivity
resources is a challenge...
Conclusions
• increasing number of objects
• discovery and self-management of objects
• connect and virtualise your objects, unlock value
• exploit redundancy pick the most suitable / interoperable /
reliable objects reliable objects
• VO / CVO services like Lego bricks fostering innovation from
IoT makers
• cognitive inside: the importance of modelling the Real World
• Cognitive IoT: user-friendly services fostering wide adoption
• implementations exploiting iCore project results in real trial
settings
• challenging times ahead!
Further info / links
[REF2] P. Vlacheas, R. Giaffreda et al. "Enabling Smart Cities Through
a Cognitive Management Framework for the Internet of Things“,
IEEE Communications Magazine - Special Issue on Smart Cities (June
2013)
[REF1] IERC April 2013 Newsletter – Foreword (see THIS LINK)
[REF3] iCore website (www.iot-icore.eu/latest-news) [REF3] iCore website (www.iot-icore.eu/latest-news)
Best Demo Award at FuNeMS 2013
Thank you!
Raffaele GiaffredaSmart IoT (RIoT) Research Area Head
(CREATE-NET)
EU FP7 iCore Project CoordinatorEU FP7 iCore Project Coordinator
Websites:
www.create-net.org/research/research-areas/riot
www.iot-icore.eu
Backup slides
CVO Management Unit
Real World Knowledge/Model
API
Service Request Analysis
iCore User
Preferences
Service Templates
Repository
System Knowledge/Model
Service Execution Request
CVO
Domain Expert
/ Knowledge
Engineer
AP
I
iCore User
Service Request
RW
K
De
sig
n
& S
tore
AP
I
CV
O T
em
pla
te
De
sig
n &
Sto
reUser Profiling
Sys
tem
Ad
min
istr
ati
on
&
Ma
na
ge
me
nt
RWK Update
Situation
Modelling
People
modelling
Natural Language
Processing
the iCore Architecture
VO FactoryVO Template
Repository
CVO
Registry
CVO Templates
Repository
VO
Registry
VO
Container VO OVOVO VO VO Front End
VO Back End: RWO Driver
CVO Factory
Device
manufac
turer
AP
I
Data
Processing
Domain
Expert /
Developer
AP
I
Device
Install
er
VO Management Unit
AP
I
VO Data Session
iCore
System
Operator
CV
O T
em
pla
te
De
sig
n &
Sto
re
AP
IA
PI
Sys
tem
Ad
min
istr
ati
on
&
Ma
na
ge
me
ntVO Execution Request
CVO Container (Execution)CVOCVOCVOCVOCVO CVO
Situation Observer
Real World Objects
(RWO)
Cognitive Inside – take-away messages
(RWK)
Models
(SK)
Models
more dependable IoT
support users of future
Smart Cities applications
(routine + alerts)
more reliable and
interoperable IoT
Models
more resilient IoT
IoT reliability and durability
through VOs
IoT resilience and fault
tolerance
Dublinked initiative
build models and predict!
mash-up data across domains
IBM Research Ireland
personalised journey tips
throughout the execution
iCore ID
3 yrs EU FP7 Integrated Project
(started 1st Oct 2011)
20 Partners with strong industrial
representation
8.7mEur EU Funding
ID Card
Japan
8.7mEur EU Funding
EU + China and Japan