designing and implementing your iot solutions with open source

© 2016 MapR Technologies 1© 2016 MapR Technologies

Designing & Implementing your IOT Solutions with Open Source

Sunil Patil Solutions Architect, MapRSridhar Reddy Director, Solutions Engineering, MapR

2016

© 2016 MapR Technologies 2

Agenda• Use Case• Demo• Layers

– Sensor – Controller– Protocol– Messaging– Computation– Persistence– Dashboard


Use Case – Temperature Monitoring


Use Case – Temperature monitoring dashboard


Do I need big data for IOT ?Sensor reads temperature once every second

• Number of readings per year3600*24*365 = 31,536,000 (31 million)

• Every reading is 100 bytes 31,536,000 * 100 = 3,153,600,000 (3.1 GB)• Every store has 50 aisel(4 sensors) and 6000+ stores

200 * 6000 = 1,200,000 (1.2 million sensors)3 * 200 * 6000 = 3,600,000 (3.6 PetaBytes)


Demo


Demo – Data Flow


IOT Big Data Reference Architecture


Sensors/Actuators


Why do we need Sensor/Actuator Layer ?Sensor/actuator will be responsible for1. Interacting with physical world

1. Sensor – Measures or detects physical property

1. Thermistor2. Temperature Oscillators

2. Actuator – Move or control mechanism or system

1. LED2. Servo Motor


Temperature Sensors

DS18B20 DHT22


Sensor Pin Layout

DS18B20 DHT22


Actuator Pin Layout


Home Automation


IOT Home AutomationYou can build home automation1. Temp Monitoring – For AC2. Auto turning of lights on and off3. Sprinkler System4. Primitive home security5. Gas detection


Device


Why do we need Device Layer ?The controller layer is responsible for1. Reading data from sensor2. Transform sensor data into

message3. Communicate with IOT Gateway

using suitable protocol4. Listening to message from IOT Gateway5. Activating with actuators based on message from Gateway


Arduino

Arduino Uno NodeMCU V2 –ESP8266


Arduino Application


Install Sensor Library


Temperature sensor – Sensor Library


Temperature sensor – Start Sensor


Arduino – Read temperature


Upload code to Arduino


Raspberry Pi


Raspberry pi - Logic for reading temperature


Raspberry Pi – Development Life Cycle


IOT Device comparison

Arduino Raspberry Pi

Architecture Micro controller Micro Processor

Operating System No Yes

Power Can go to sleep Continuous 5V

Network No Yes

Sensor Analog + digital Digital

Development Environment

Arduino IDE – C code Java, Python,… etc


Protocols


Why do we need Protocol Layer ?The protocol layer is responsible for 1. Communication between IOT Device and IT infrastructure2. The IOT Devices operate in constrained environment

1. Limited processing power2. Limited amount of memory (Less than 100K)3. Network usage

1. Less Reliable network2. Optimized usage of network

4. Operate on battery1. Sleep mode


MQTT – Message Queue Telemetry TransportMQTT is publish/subscribe messaging protocol designed for lightweight M2M communication.• Open: MQTT is OASIS standard created by IBM • Lightweight Client: Client libraries less than 30 KB• Optimized Network usage: Small packet size 2 bytes (header)

– TCP Based. MQTT-SN is UDP based– Easy to parse packets

• Reliable: Three QoS patterns to avoid packet loss on client disconnect– At-most-once – does not survive failures, but is never duplicated– At-least-once – survives connection loss and other failures, but can be duplicated– Exactly-once – survives connection loss and other failures, and is never

duplicated


MQTT vs. HTTPS 1. Performance is 80 times better2. Battery usage 10 times less

http://stephendnicholas.com/archives/1217


MQTT

Device 1

Device 2

Device 3

MQTTBroker

Subscriber 1

Subscriber 2

/store/fremont/aisel1


/store/sanjose/aisel1

/store/fremont/#

/store/#


MQTT – Server & Broker

Subscriber 1

Subscriber 2



/store/sanjose/aisel1

/store/fremont/#

/store/#


MQTT – Include MQTT client library


MQTT – Setup MQTT Connection


Arduino – Send Receive Message


Arduino – Memory usage


CoAP – Constrained Application ProtocolThe CoAP is request/response protocol similar to HTTP + REST• Open: CoAP is IETF standard - RFC 7252 IETF• Lightweight Client: Client libraries less than 30 KB• Optimized Network usage

– Based on UDP (6LowPAN), SMS in addition to TCP– Compact 4 bytes header, compressed– CoAP uses familiar HTTP stuff like Method (Get, Put, Post, Put, Delete), status

codes, URIs, content types/mapping– Can be used to carry different type of data. It uses content type to identify type

of data being carried– HTTP Proxy can be used to map COAP to HTTP

• Reliable: The HTTP proxy takes care of reliability


Arduino – Read Temp. + Publish MQTT Message

http://slideplayer.com/slide/9365664/


COAP

Device1

Device 2

Device 3

HTTPProxy

Subscriber 1

Subscriber 2



/store/sanjose/aisel1 coap://192.168.1.2:5683:/store/sanjose/aisel1

http://192.168.1.2:5683:/store/fremont/store1


Arduino – Read Temp. + Publish COAP Message


Messaging


Why do we need Messaging Layer ?The Messaging layer is responsible for 1. Highly available MQTT

1. Scale MQTT layer horizontally2. Provide high availability

2. Decouple IOT layer from the computation layer

3. Allow multiple consumers to process the IOT data to meet different business requirements


MQTT load balancing – No Kafka

Device

Device

Device

Device

MQTT

MQTT

MQTT

Spark

Spark

Spark


MQTT load balancing - Kafka

Device

Device

Device

Device

MQTT

MQTT

MQTT

Spark

Spark

Spark

Kafka


MQTT Kafka Bridge

MQTT Kafka

Bridge 1

Bridge 2


Computation


Why do we need computation layer ?The computation layer is responsible for 1. Process huge amount of data in linear

scalable manner1. Read streaming data 2. Transform/Enrich/Analyze data3. Store the incoming data into persistence storage

2. Analyze IOT data1. Business Rules2. Predictive model


Spark Streaming


Business rules


Storage


Why do we need Storage Layer ?The storage layer is responsible for 1. Providing persistent store for IOT

data2. Store data in time series format

1. Efficient storage of data2. Ingest data at high speed3. Retrieve/Query data


Time Series Database• Data Point

(time, temperature)

• Metricssensor.temp.1s

• TagssensorId=2, location=bed1

• Metrics + Tags = Time Series


InfluxDB


Dashboard


Why do we need Dashboard Layer ?The dashboard layer is responsible for 1. Interact with TSDB2. Visualize TSDB data in easy to

understand format


Configure InfluxDB connection


Temperature Variation Graph


Resources • Streaming Processing for Real-Time Analytics and Dashboard• MapR Streams• MapR DB• Eclipse IOT • COAP Tutorial• GitHub For source code

https://www.mapr.com/solutions/quickstart/stream-processing-for-real-time-analytics-dashboards

https://www.mapr.com/products/mapr-streams

http://iot.eclipse.org/


https://community.arm.com/servlet/JiveServlet/previewBody/8633-102-2-15471/ARM%20CoAP%20Tutorial%20April%2030%202014.pdf



Questions ?

Sunil Patil, MapR [email protected]


Arduino - Logic for reading temperature

designing and implementing your iot solutions with open source

Technology