IOT BIG DATA: REALISING THE FULL POTENTIAL OF THE IoTETSI PresentationNovember 2016

THE IOT BIG DATA ECOSYSTEM

IOT TODAY

Disparate systemsSilos of heterogeneous dataValuable data, proprietary implementationsWidespread innovation by device manufacturers

VISION FOR IOT

Sharing of dataStandardisationData monetisationIncreased innovation across all tiersIncreased protections: privacy, security etc

VISION FOR AN IOT BIG DATA ECOSYSTEM

Creation of new business models, products and services by applying various capabilities, such as mash-up and analytics, to IoT and context data captured from multiple sources“

“- Enable an ecosystem through which data can be harmonised and shared from

multiple sources

- Develop new commercial products and services by applying capabilities to harmonised IoT and context data

“Water, water, every where,Nor any drop to drink.”

Expect we will be flooded in data but how usable will it be?

Some of the challenges Device protocols/ APIs Context data protocols (JSON/

CSV/ XML etc) Knowing the accuracy of devices Consistency of units Device reliability Device stability Finding the data

4

THE CHALLENGE OF DATA SET DIVERSITY

5

Source: Internet of Things Study 2016, Evans Data Corporation

IOT DEVELOPER INSIGHTS

KEY AREAS OF GSMA FOCUS 2016/2017

Standardisation - data & APIs Data privacy Ecosystem collaboration and

innovation

Agreement of de-facto standards: Harmonised data sets for key

market sectors Common framework for delivery

of IoT Big Data ecosystem

Common API for sharing of harmonised data & control

Directory of harmonised operator IoT Big Data

GSMA position on Privacy and Big Data

Big Data Guidelines incorporating IoT Big Data published

Assessment of candidate API technologies

Ecosystem input into de facto standards

Innovation and showcases

7

IOT BIG DATA USE CASES

Smart Home Utility metering Home appliance

Agriculture Farms Greenhouses

Environment Air quality Water quality Weather

Smart Cities Roads Parking

Automotive Vehicle Faults

The potential scale of data being stored, processed or queried given

The number of IoT devices and update frequency

Combining current / historical IoT data with external data

Managing privacy issues including consent, aggregation, anonymisation

The large variety of devices in the field where each vendor/ device type might have its own specific interface

Differences in device attributes – names, units etc - even where the device is reasonably generic

8

PARTICULAR CHALLENGES FOR IOT/BIG DATA

GSMA has published three technical documents defining the approach for delivery of IoT Big Data services to the general third party application developer ecosystem1. Technical Architecture Framework2. API Specification for Data & Control exposure towards 3. Initial set of Harmonised Entity Definitions

Documents available from http://www.gsma.com/connectedliving/iot-big-data/

9

GSMA Technical Documents

TECHNICAL ARCHITECTURE FRAMEWORK

Logical architecture provides flexibility so organisations can follow different business strategies‒ Analytics on the left‒ Direct data & control on the

right Common elements:

‒ Harmonised data entity definitions

‒ Common APIs for direct data & control

FIT WITH ONEM2M

Initial proprietary interfaces (I2) will be augmented with OneM2M

Substantial simplification of interfacing layers

Gateways handle local data acquisition & control

Dedicated IoT applications delivered using oneM2M stack

Hybrid architecture supports richer applications stack including advanced analytics, visualisation, machine learning

Applications which span broader set of devices and context data

Data transformation/ storage to be used in rich analytics/ machine learning

AE

CSE

NSE

Mca

Mcn

oneM2M Functional

Architecture

I2 = Mca

FIT WITH FIWARE CONTEXT BROKER

FIWARE (Telefonica, Orange, NEC and others) have specified NGSIv2

FIWARE have also implemented a generic ’context broker’ component ‘Orion’ which can be used for Big Data service delivery

Flexible environment for communicating harmonised entity data throughout the architecture

13

HTTP Verb: POSTResource URL: /v2/entities?options=options

Request body example{

"id": "a01b4849-2eca-4518-b5a3-230f9749147e","type": "AgriSoil","source": "www.samplefarmproduct.com","dataProvider": "OperatorA","name": "Sand"

}

EXAMPLE NGSIv2 – CREATE ENTITY

Create an entity including one or more attributes;

Retrieve entity (/specified attribute`s) by entity identifier;

Update one or more attributes of an existing identity;

Remove an entity; Get an attribute value; Update an attribute value; Remove an attribute of an entity; List entities matching specified criteria;

List known entity types; Retrieve entity type information; Create a subscription; List subscriptions; Retrieve details of a subscription; Update details of a subscription; Delete a subscription; Batch update; Batch query.

14

NGSIv2 OPERATIONS

15

Agriculture AgriCrop

AgriGreenHouse

AgriParcel

AgriParcelRecord

AgriParcelOperation

AgriPest

AgriProduct

AgriProductType

AgriSoil

HARMONISED DATA ENTITIES

Environment AirQuality

EnvironmentObserved

PointOfInterest

WaterQuality

WeatherForecast

WeatherObserved

Smart Home Building

BuildingType

BuildingRecord

BuildingRecordType

Connected Car /Smart Cities

Vehicle

VehicleType

VehicleFault

Road

RoadSegment

General IoT Device

DeviceRecord

DeviceRecordType

Machine

MachineType

Subscriber

SubscriptionService

Entity definitions published at : https://github.com/GSMADeveloper/HarmonisedEntityDefinitions

16

EXAMPLE ENTITY DEFINITION - AGRISOIL

Attribute Name Attribute Type

Description Mandatory/Optional

May be Null

name Text The name of this soil type. M N

alternateName Text Alternative name of this soil type. O Y

description Text A description of this soil. O Y

refAgriProduct Array of Reference

An array containing a JSON encoded sequence of characters referencing the unique ids of the recommended AgriProduct fertiliser (or other) product(s).

O Y

Attribute Name Attribute Type

Description Mandatory/Optional

May be Null

id Text Unique id of this instance of this entity. M N

type Text Must be equal to "AgriSoil". M N

dateCreated DateTime Entity creation timestamp. M N

dateModified DateTime Timestamp of the last modification of the entity.

M Y

source Text A sequence of characters giving the source of the entity data as a URL.

M Y

dataProvider Text A sequence of characters identifying the originator of the harmonised entity.

M Y

<AgriSoil><Generic Attributes>

<AgriSoil><Entity Specific Attributes>

17

EXAMPLE ENTITY CODING (JSON) - AGRISOIL{"id": "789363b4-c771-43d6-8505-ca582efe8fcd","type": "AgriSoil","dateCreated": {"value": "2016-08-08T10:18:16Z", "type": "DateTime"

},"dateModified": {"value": "2016-08-08T10:18:16Z", "type": "DateTime"

},"source": {"value": "http://www.samplefarmproduct.com", "type": "URL"

},"dataProvider": {"value": "OperatorA", "type": "Text"

},"name": {"value": "Clay", "type": "Text"

},"alternateName": {"value": "Heavy soil", "type": "Text"

},"description": {"value": "Fine grained, poor draining soil. Particle size less than

0.002mm", "type": "Text"},"refAgriProduct": ["ea54eedf-d5a7-4e44-bddd-50e9935237c0","275b4c08-5e52-4bb7-8523-74ce5d0007de"

]}

Analytics & Visualisation Better understanding of data relationships

E.g. weather to crop yields Traffic to pollution

Acting on that information Decision to harvest crops based on

actual/ forecast weather Changing traffic signalling to improve

pollution levels

Machine Learning Use of AI/ Deep Learning

Establishing new/ non obvious relationships between data and outcomes

Advanced image/ video processing applications e.g. traffic monitoring, smart parking

18

NEXT STEPS FOR IOT BIG DATA