university of applied sciences citypulse: reliable …...authority technical certificate authority...

IoT Week, 2nd June 2016 Ralf Tönjes 1

University of Applied Sciences

Osnabrück

Satelliten- und Mobilfunk Prof. Dr.-Ing. Ralf Tönjes 1

Ralf Tönjes


Osnabrück, Germany

CityPulse:

Reliable Information Processing

in Smart City Frameworks



Osnabrück

Content

1. Introduction

2. Framework for Smart City Data Analysis

3. QoI Monitoring

4. Spatial Reasoning

5. Conclusion



Osnabrück

Smart Services are Context-aware

Personal Digital Assistant

Recommender System

Advertisements Context-aware Traffic Management

Augmented

Reality



Osnabrück



Osnabrück

Smart City Data

Data is multi-modal and heterogeneous

Requires (near-) real-time analysis

Noisy and incomplete

Time and location dependent

Dynamic and varies in quality

Crowd sourced data can be unreliable

Data alone may not give a clear picture

we need contextual information,

background knowledge,

multi-source information and

obviously better data analytics solutions…

t0

t0+5min.

t0++10min.



Osnabrück

Content

1. Introduction


3. QoI Monitoring


5. Conclusion



Osnabrück

An Integrated Approach

Re-usable

components



Osnabrück

• Virtualisation

• Heterogeneous data sources

• Overcome silo architectures and

provide common abstract interface

• Assigning semantic annotations

to data streams

• Federation (Sensor Fusion)

• Combines heterogeneous data

streams to one unified view

• Aggregation (Data Fusion)

• Reduce amount of data:

• Clustering

• Filtering

• Pattern recognition

• Complex event processing

• Smart Adaptation

• Higher level information processing

• Real-time reasoning

• Enables adaptation of the

data processing pipeline

CityPulse Framework



Osnabrück

• User Centric Decision Support

• Goal: provide optimal configuration

of smart city applications

• Social and context analysis

• Matchmaking and discovery

mechanisms

• Match data according to users

preferences and context

• Reliable Information Processing

• Challenge: Dynamic environments,

changes and prone to errors

• Reliable data processing requires

accuracy and trust (reputation)

• Cope with

• Malfunctions

• Disappearing sensors

• Conflicting data by monitoring

of streams (runtime)

• Smart City Applications

CityPulse Framework



Osnabrück

Content

1. Introduction


3. QoI Monitoring


5. Conclusion



Osnabrück

Unreliable, outdated,

temporarily unavailable data

Contradicting data

Single data sources could provide faulty

information

• Example

– Travel planning application that needs

current traffic information

– Traffic sensors deliver contradictory information

Malfunctioning sensor which delivers false information?

or Local traffic jam?

Provenance of Data

Trust in social media data

Jam

!

Ok

Problem: Unreliable Data



Osnabrück

Modelling Trustworthiness and QoI

• Identification of application independent information quality

parameters and metrics

• Definition of an explicit semantic model for quality annotation of

smart city data streams

• Result: 5 Categories, 23 Parameters



Osnabrück

Quality of Information subcategory

abstraction

levelMeasurementunit

information Probabilitythatinformationiswithintherangeofprecisionandcompleteness

Resolution information absolutevalueinsensingunit

Deviation(max) information maximumdeviationpercentage

informationProbabilitythatallstreamdatasetscontainthedefinedvaluesandareupdated

intheirdefinedfrequency

PacketLoss technical Ratio/ErrorRate

Bandwith(Bitrate) technical Bitspersecond

Latency technical (mili,micro)seconds

Jitter technical (Milli)Seconds

Throughput technical Bitspersecond

QueuingType technical QueueType

Ordered technical Probabilitythatdatasetsarriveinthedefinedqueuingorder

technical Definedperinformationorperoperatingtime

operational Definedperinformationorperoperatingtime

technical Definedperinformationorperoperatingtime

licencedef. operational ReferencetoLicenceclass,e.g.http://creativecommons.org/ns#Licence

maybeused operational ReferencetoPermissionclass,e.g.http://creativecommons.org/ns#Permission

maybepublished operational ReferencetoPermissionclass,e.g.http://creativecommons.org/ns#Permission

operational Encryptionmethod,authorityforkeymanagement

authority technical Certificateauthority

publickey technical Keytodecryptsignatures

information maximumtimebetweenmeasurementandpublication

information AverageDurationhowlongtheinformationisusable,measuredinseconds

technical Maximumtimespanbetweentwodatasets

Communication

MonetaryConsumption

Frequency

Confidentiality(reuseofrightsontology,e.g.http://creativecommons.org/ns)

Parameter-name

Signing

Queuing

Precision

Completeness

Correctness

EnergyConsumption

Accuracy

Cost

Timeliness

Age

Volatility

NetworkPerformance

NetworkConsumption

Encryption

Security

Qu

alit

y o

f In

form

atio

n (

Qo

I)



Osnabrück

Atomic Monitoring: Rating

Current Implementation for:

•Frequency: (based on t(x)virt – t(x-1)virt)

•Age: (based on tnow – t(x-1)sample)

•Latency: (based on t(x)virt – t(x)sample)

•Completeness: (completeness of tuple)

•Correctness: sanity check derived from

stream annotation (value range, data format,

etc.)



Osnabrück

Atomic Monitoring – QoI Explorer



Osnabrück

Atomic Monitoring Evaluation – QoI Explorer



Osnabrück

Atomic Monitoring: Traffic Frequency



Osnabrück

Where are the bad sensors?



Osnabrück

Find

Correlated

Streams

Determine

Temporal

Distance

Determine

Temporal

Distance

Determine

Temporal

Distance

Compute

Partial

Correctness

Compute

Partial

Correctness

Compute

Partial

Correctness

Compute

Composite

Correctness

Event

. . .

. . .

Which streams can

be used to validate

event?

How long does it

take for the event to

reach the sensor?

Does the other

stream agree?

Do all other streams

agree?

Composite Monitoring: Correlation



Osnabrück

Composite Monitoring

Time series analysis

Sensors 179202 and 179228 detecting

slow traffic at event time

assumption that event is plausible

20



Osnabrück

Content

1. Introduction


3. QoI Monitoring


5. Conclusion



Osnabrück

Distance Sight Way Track/Vehicle

Propagation Radial

Radial with blocking

Distinct Grid Restricted Layer on base Grid

Example Pollution Light Street System Subway Ride

Feasibility Simple Complex Medium Medium

Improve QoI by

Finding Correlated Streams



Osnabrück

Euclidean Distance Does not Reflect

Data for Infrastructure (Like Streets)

The nearest traffic sensor does not reflect the traffic status.

Voronoi diagram - depicting the nearest traffic sensor (labelled with a number)

and traffic condition value for every street segment inside a Voronoi cell:



Osnabrück

Example: Misleading Distances

• How far is the next hospital?



Osnabrück

●●●

●● ●

● ●

●

●

0.0

0.2

0.4

0.6

890 900 910 920 930

Euclidean Distance(m)

Bra

y C

urt

is D

iss

am

ilir

ity

TrafficSensor

●

●

●

●

●

●

●

●

●

●

178955

178983

181060

181088

181114

181142

188172

188225

189941

190126

●●●

● ●●

● ●

●

●

0.0

0.2

0.4

0.6

1200 1300 1400 1500

Shortest Path Distance(m)

Bra

y C

urt

is D

iss

am

ilir

ity

Optimisation by Distance Metric

• Correlating similarities between sensor time series against

their distance to each other

• Better regressions when using shortest path distance

• Convincing model (less corellation with higher

distance)

• Smaller variance

Comparing 1 Parking Garage Sensor against

10 Traffic Sensors:

449 Traffic Sensors

in Aarhus Denmark



Osnabrück

Correlation of Distance Metrics

pearson spearman kendall pearson_offset spearman_offset kendall_offset

●

●●●

●

●

●

●●

●

● ●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●●●

●

●

●

●

●

●

●●●●

●

●●●●●

●

●

●

●●

●

●●●●●

●●●●●●

●●●●

●

●

●●

●●

●

●

●

●

●

●●●

●

●●●

●●●●●

●●●●

●

●

●●

●

●●

●

●●●

●

●●

●

●●●●

●

●●●

●

●

●

●

●

●

●●●●

●

●●●

●●

●

●

●

●

●●●●●

●

●●

●●●●

●

●●

●●●

●

●

●

●●

●●

●●●●●●●

●

●

●

●

●

●

●●

●●

●

●

●●

●

●

●

●

●●

●

●●●

●

●●●●●●●●●●

●

●

●●

●

●●

●

●●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●●

●

●●●

●●●

●

●

●●

−0.5

0.0

0.5

eucl

idea

n_ds

t

shor

test

_pat

h_ds

t

dura

tion

num

ber_

of_s

teps

eucl

idea

n_ds

t

shor

test

_pat

h_ds

t

dura

tion

num

ber_

of_s

teps

eucl

idea

n_ds

t

shor

test

_pat

h_ds

t

dura

tion

num

ber_

of_s

teps

eucl

idea

n_ds

t

shor

test

_pat

h_ds

t

dura

tion

num

ber_

of_s

teps

eucl

idea

n_ds

t

shor

test

_pat

h_ds

t

dura

tion

num

ber_

of_s

teps

eucl

idea

n_ds

t

shor

test

_pat

h_ds

t

dura

tion

num

ber_

of_s

teps

Metric

Valu

e

• Pairwise correlation of 449 traffic sensors.

• Resulting correlation values (Pearson correlation) have been correlated

against different distance models.

=> The utilisation of matching metrics and a time shift

of the time series shows a significant effect on the correlation value.

Time Offset: modells propagation speed



Osnabrück

Conclusion

Objective: Enable uptake of context-aware Smart City applications

Approach

Make Raw Data Meaningful

Semantic annotation for knowledge based machine interpretation

Processing Capabilites for Unreliable Data

Modelling and processing trustworthiness and QoI

Reasoning in the city depends heavily on spatial context

Appropriate distance measures are required by spatial reasoning,

e.g. shortest path

Multiple information coverage of the same spatiotemporal boundaries is

needed

Individual distance calculations help finding correlation partners

(Euclidean dist. is not sufficient, but can be first iteration step)

Cross domain re-usable tools

To overcome silo architectures and

ease service creation



Osnabrück

• Thank you!

• EU FP7 CityPulse Project:

http://www.ict-citypulse.eu/

@ictcitypulse

[email protected]

university of applied sciences citypulse: reliable …...authority technical certificate authority...

Documents