designing and implementing your iot solutions with open source

Designing & Implementing your IOT Solutions with Open Source

Sunil Patil Solutions Architect, MapRSridhar Reddy Director, Solutions Engineering, MapR

Agenda• Use Case• Demo• Layers

– Sensor – Controller– Protocol– Messaging– Computation– Persistence– Dashboard

Use Case – Temperature Monitoring

Use Case – Temperature monitoring dashboard

Do I need big data for IOT ?Sensor reads temperature once every second

• Number of readings per year3600*24*365 = 31,536,000 (31 million)

• Every reading is 100 bytes 31,536,000 * 100 = 3,153,600,000 (3.1 GB)• Every store has 50 aisel(4 sensors) and 6000+ stores

200 * 6000 = 1,200,000 (1.2 million sensors)3 * 200 * 6000 = 3,600,000 (3.6 PetaBytes)

Demo – Data Flow

IOT Big Data Reference Architecture

Sensors/Actuators

Why do we need Sensor/Actuator Layer ?Sensor/actuator will be responsible for1. Interacting with physical world

1. Sensor – Measures or detects physical property

1. Thermistor2. Temperature Oscillators

2. Actuator – Move or control mechanism or system

1. LED2. Servo Motor

Temperature Sensors

DS18B20 DHT22

Sensor Pin Layout

DS18B20 DHT22

Actuator Pin Layout

Home Automation

IOT Home AutomationYou can build home automation1. Temp Monitoring – For AC2. Auto turning of lights on and off3. Sprinkler System4. Primitive home security5. Gas detection

Device

Why do we need Device Layer ?The controller layer is responsible for1. Reading data from sensor2. Transform sensor data into

message3. Communicate with IOT Gateway

using suitable protocol4. Listening to message from IOT Gateway5. Activating with actuators based on message from Gateway

Arduino

Arduino Uno NodeMCU V2 –ESP8266

Arduino Application

Install Sensor Library

Temperature sensor – Sensor Library

Temperature sensor – Start Sensor

Arduino – Read temperature

Upload code to Arduino

Raspberry Pi

Raspberry pi - Logic for reading temperature

Raspberry Pi – Development Life Cycle

IOT Device comparison

Arduino Raspberry Pi

Architecture Micro controller Micro Processor

Operating System No Yes

Power Can go to sleep Continuous 5V

Network No Yes

Sensor Analog + digital Digital

Development Environment

Arduino IDE – C code Java, Python,… etc

Protocols

Why do we need Protocol Layer ?The protocol layer is responsible for 1. Communication between IOT Device and IT infrastructure2. The IOT Devices operate in constrained environment

1. Limited processing power2. Limited amount of memory (Less than 100K)3. Network usage

1. Less Reliable network2. Optimized usage of network

4. Operate on battery1. Sleep mode

MQTT – Message Queue Telemetry TransportMQTT is publish/subscribe messaging protocol designed for lightweight M2M communication.• Open: MQTT is OASIS standard created by IBM • Lightweight Client: Client libraries less than 30 KB• Optimized Network usage: Small packet size 2 bytes (header)

– TCP Based. MQTT-SN is UDP based– Easy to parse packets

• Reliable: Three QoS patterns to avoid packet loss on client disconnect– At-most-once – does not survive failures, but is never duplicated– At-least-once – survives connection loss and other failures, but can be duplicated– Exactly-once – survives connection loss and other failures, and is never

duplicated

MQTT vs. HTTPS 1. Performance is 80 times better2. Battery usage 10 times less

http://stephendnicholas.com/archives/1217

Device 1

Device 2

Device 3

MQTTBroker

Subscriber 1

Subscriber 2

/store/fremont/aisel1

/store/sanjose/aisel1

/store/fremont/#

/store/#

MQTT – Server & Broker

Subscriber 1

Subscriber 2

/store/sanjose/aisel1

/store/fremont/#

/store/#

MQTT – Include MQTT client library

MQTT – Setup MQTT Connection

Arduino – Send Receive Message

Arduino – Memory usage

CoAP – Constrained Application ProtocolThe CoAP is request/response protocol similar to HTTP + REST• Open: CoAP is IETF standard - RFC 7252 IETF• Lightweight Client: Client libraries less than 30 KB• Optimized Network usage

– Based on UDP (6LowPAN), SMS in addition to TCP– Compact 4 bytes header, compressed– CoAP uses familiar HTTP stuff like Method (Get, Put, Post, Put, Delete), status

codes, URIs, content types/mapping– Can be used to carry different type of data. It uses content type to identify type

of data being carried– HTTP Proxy can be used to map COAP to HTTP

• Reliable: The HTTP proxy takes care of reliability

Arduino – Read Temp. + Publish MQTT Message

http://slideplayer.com/slide/9365664/

Device1

Device 2

Device 3

HTTPProxy

Subscriber 1

Subscriber 2

/store/sanjose/aisel1 coap://192.168.1.2:5683:/store/sanjose/aisel1

http://192.168.1.2:5683:/store/fremont/store1

Arduino – Read Temp. + Publish COAP Message

Messaging

Why do we need Messaging Layer ?The Messaging layer is responsible for 1. Highly available MQTT

1. Scale MQTT layer horizontally2. Provide high availability

2. Decouple IOT layer from the computation layer

3. Allow multiple consumers to process the IOT data to meet different business requirements

MQTT load balancing – No Kafka

Device

MQTT load balancing - Kafka

Device

MQTT Kafka Bridge

MQTT Kafka

Bridge 1

Bridge 2

Computation

Why do we need computation layer ?The computation layer is responsible for 1. Process huge amount of data in linear

scalable manner1. Read streaming data 2. Transform/Enrich/Analyze data3. Store the incoming data into persistence storage

2. Analyze IOT data1. Business Rules2. Predictive model

Spark Streaming

Business rules

Storage

Why do we need Storage Layer ?The storage layer is responsible for 1. Providing persistent store for IOT

data2. Store data in time series format

1. Efficient storage of data2. Ingest data at high speed3. Retrieve/Query data

Time Series Database• Data Point

(time, temperature)

• Metricssensor.temp.1s

• TagssensorId=2, location=bed1

• Metrics + Tags = Time Series

InfluxDB

Dashboard

Why do we need Dashboard Layer ?The dashboard layer is responsible for 1. Interact with TSDB2. Visualize TSDB data in easy to

understand format

Configure InfluxDB connection

Temperature Variation Graph

Resources • Streaming Processing for Real-Time Analytics and Dashboard• MapR Streams• MapR DB• Eclipse IOT • COAP Tutorial• GitHub For source code

Questions ?

Sunil Patil, MapR Technologiessunilpatil@mapr.com

Arduino - Logic for reading temperature

designing and implementing your iot solutions with open source

Technology

iot-a project: designing a reference architectural for...

module 4 designing and implementing views

designing and implementing branding strategies

designing and implementing payments for …

designing and implementing household targeting systems

18 designing and implementing innovative...

designing, implementing and facilitating communities of

designing and implementing conservation agriculture of

designing and implementing corporate social responsibility

designing algorithms for industrial iot analytics

designing and implementing a destination-based corporate...

designing and implementing mixed method research

designing implementing honeypot scada network 35252

e-learning, designing and implementing

designing, implementing, and using reactive apis

designing and implementing electronic monitoring systems

designing and implementing workforce planning process

designing & implementing branding strategies

4. designing and implementing successful grp

designing, implementing and evaluating parenting...