iot data as service with hadoop
TRANSCRIPT
Internet Of Things & Big Data
As Service
About me
Sriram ReddyArchitectwww.Ammeon.com
Software Architect with a passion for
• Linked Devices• Device Multi Tenancy• Low Power Wide Area Networks• Distributed Device Management • Low Power Device Security
• Distributed Data Persistence• Distributed Data Processing• Analytics
Index
1. Introduction2. Internet Of Things
• Micro Controller & Processor• Power Management
3. IOT Networks• Long Range• Short Range
4. IOT Data Types• Binary (MQTT & Proto buff)• Jason
5. IOT Data Acquisition• MQTT• CoAP• Kafka
5. IOT Data processing• Spark• Nifi• StreamSets
6. Data Persistence• Hadoop• Cassandra• OrentDB• Titan• Mongo
7.Reporting• BIRT Reporting• Eclipse RCP
8. Analytics9. Market Vertices 10. Q&A
Introduction
Nothing is new. There are devices, there is network and there is data.
Challenge is Lot Of Devices, Network Everywhere, Lot Of Data
IOT Triad
Communications
AnalyticsDevice
EnablementHW-ClinetContainerModeling
Dashboard
EthernetGSM/3G/4G/MB-IOT BLE/LORA /WIFI
AVR/PIC/Psoc (MCU)SOC(MCP)Boards (Pis)
MQTTCoAPHTTPKAFKA
HadoopCassandraIgniteKafkaSpark/Flink/DrillNIFI/Streamsets/Camel
An IOT deployment architecture demands right device design, right communications and data management & analytics.
IOT Devices
Microprocessor Based Devices
Microcontroller Based Devices
Raspberry Pi, Orange Pi, Banana Pi, Nano Pi,Bagel Bone, Panda board, Onion, Chip etc.
• SOCs from Boradcom, Rock chip, All winner, Intel etc • Runs full Linux/Windows • Powerful • Power-hungry
Arduino, Wiring, Sypris etc • Micro-chip, Atmel, Nordic etc • SOC with memory and other peripherals• Low power• Less power consumption
IOT Communications
Long Range
Short Range
Networks to span full metros to countries
• Traditional operator provided 2G/3G/4G• LORA• SIGFOX
With in 100 meter range
• WIFI• BLE
IOT Data Types
Binary Data
JSON
Binary data provides size advantage but adds more processing.
• MQTT• Protobuffs
JSON or any markup payloads offer human readability and existing processing infrastructure.
IOT Data Acquisition
MQTT
CoAP
Kafka
• Lot of queues (millions)• Light Weight• Open source brokers• Well Integrated • Well supported
• HTTP like• Matured with telco industry• Human readable
• Light server • Not very heavy client• Client controlled• Fast• Easy to scale• Easy to replicate
IOT Data Processing
Spark Ignite
NifiStreamSets
Beam
IOT Data Persistence
Hadoop Cassandra
Oriented DB
IOT Market
IOT Market
Q&A