From the Internet of Things toIntelligent Systems:

A Developer's Primer

Rick G. GaribayDistinguished Engineer, Neudesic

MVP, Microsoft Azure@rickggaribay

Level: Intermediate

About Me

• Distinguished Engineer, Neudesic working on IoT,Intelligent Transportation and Hospitality & Gaming

• Microsoft MVP, Microsoft Azure• Co-Author, “Windows Server AppFabric Cookbook”

by Packt Pub.• Chairman, Co-Founder Phoenix Connected

Systems User Group (PCSUG.org)• twitter: @rickggaribay• blog: http://rickgaribay.net• email: rick.garibay@neudesic.com | b-rigari@microsoft.com

What is the Internet of Things?

The Internet of Things: By theNumbers

B5075212

This change is happeningmore rapidly than anyone

imagined.

This change could bringtremendous opportunity to yourbusiness, industry and you as

a technologist.

The Internet of Things is thenetwork of physical objects thatcontain embedded technology tocommunicate and interact with

their internal states or the externalenvironment.

OEM Revenue Opportunity |Market Forecast CY17

Auto & Trans Retail Manufacturing Healthcare Energy Computing Telecom Consumer

$7 B $16 B $197 B $3 B $27 B $908 B $179 B $356 B System Revenue

IntelligentSystems

1.7T$

SmartProducts

Grid

Renewables

Oil/Gas/CoalRecovery and

Distribution

Pointsof Sale

Restaurants

Hotels

FuelStations

Patients

Clinics

Hospitals

NursingHomes

MobileCare

SafetySecurity

ComfortLighting

Automation

ManufacturingIntegration and

AutomationRemoteServicing

Predictive andReactive

Maintenance

Water

Waste

PollutionControl

Fire

Emergency

PublicSafety

LawEnforcement

LettersPackages

Containers

Tanks Bulkware

Games

Events

Sports

TelevisionStreaming

Traffic Buses

Cars

Trucks

Trains

Vessels

Aircraft

BikesSmartEnergy

SmartRetail

SmartMobilitySmart

Mobility

SmartLogisticsSmart

Logistics

SmartFactorySmart

Factory

SmartCitiesSmartCities

SmartEntertain-

ment

SmartEntertain-

ment

SmartHealth-

care

SmartBuildingHome

SmartBuildingHome

IoT Device Taxonomy

Large

Mobile

Micro

Small

POS terminal, ATM, MRIx86, PC-like, apps

Industry handheld, POS tabletARM and x86, shell experience, apps

Gateways, wearables, panels, carsARM and x86, diverse hardware, no shell

Controllers, fixed-use, sensors, actuatorsARM, constrained hardware, headless

Raspberry Pi

Intel Galileo

Gadgeteer Fez Spider

5mm 5mm

Arduino Uno

Netduino Plus 2

Spark

Beyond the garage, the truesignificance of IoT is the foundation itprovides for enabling new business

capabilities.

FromInformation Technology

toOperational Technology

ITServers,

Applications,Systems

ITServers,

Applications,Systems

OTDevices,

Telemetry,Command &

Control

OTDevices,

Telemetry,Command &

Control

Data-Driven Insight + Action at a Distance

Data-Driven Insight

• Data –> Information –> Insight ($+)–Make more efficient use of resources

(reduce cost, environmental impact)Example: Power management in buildings and data centers

–Provide more targeted products andservices (increase revenue, socialimpact)Example: Preventive maintenance, optimal usage analytics forexpensive machines

• “Things” = a rapidly expanding sourceof raw material for the Insight pipeline

Action at a Distance

• Data isn’t the only raw material being unlocked by the IoT– The ability to act remotely – automatically and intelligently– Remote control is a source of efficiency– Enables new forms of customer interaction and engagement

• IoT extends customer engagement opportunities to physicalproducts

• Taking engagement with customers beyond the point of sale– Preventive maintenance– Best practices guidance– Proactive sales– Remote servicing

• From CRM to PRM – “Product Relationship Management”

From IoT to Intelligent Systems

Large

Mobile

Micro

Small

M2M/Device to

Cloud

Various Communication Protocols

2G2G

Wi-Fi

Wi-Fi

Bluetooth/BLE

Bluetooth/BLE

RFIDRFID

GPRSGPRS SMSSMS

3G3G

LTELTE

WiMaxWi

Max

ZigBeeZigBee

Various Application Protocols

AMQPAMQP MQTTMQTT

CoAPCoAP

CustomCustom

HTTPHTTP

……

MQ Telemetry Tranport (MQTT)

• Born out of IBM MQ Series messaging middleware product• Compact binary protocol – min. 7 byte overhead per message

sent• No structured message – message bodies are byte arrays• Simple topic name based pub/sub messaging model

– Send to topic name, e.g., “/a/b/c/d” or “/a/b/e/f”– Subscribe to topic name, e.g., “/a/b/c/d” or use wildcard, e.g., “/a/b/#”

• Reliable – fire-and-forget to reliable, exactly-once delivery• Two innovative, device-oriented features:

– Retain – mark a message to be delivered to new subscribers onconnection

– Last will and testament – register message to be sent on abruptdisconnect

• Not general purpose – lacking key features, e.g., flow control• Standardization in progress through OASIS

Constrained Application Protocol(CoAp)

• Embedded web transfer protocol (coap://)

• Asynchronous transaction model

• UDP binding with reliability and multicast support

• GET, POST, PUT, DELETE methods

• URI support

• Small, simple 4 byte header

• DTLS based PSK, RPK and Certificate security

• Subset of MIME types and HTTP response codes

• Built-in discovery

• Optional observation and block transfer

Advanced Message QueuingProtocol 1.0 (AMQP)

• Efficient – binary connection-oriented protocol• Reliable – fire-and-forget to reliable, exactly-once delivery• Portable data representation and structured message

definition• Flexible – peer-peer, client-broker, and broker-broker

topologies• Broker-model independent – no requirements on broker

internals• Rich flow control – multiplex multiple data streams over a

connection• OASIS Standard (Oct 2012); International Standardization in

progress– Somewhat controversial…

Message Types

Voluntaryinformation flowfrom device to

another system.

Requests forinformation fromdevice to other

systems.

Instructionsfrom other

systems to adevice.

Information flowfrom other

systems to thedevice.

Telemetry Inquires Commands Notifications

Default Connectivity Model

• Connectivity (IPv6 + VPN)– Give every device a routable IP address– Devices expose services for control/query

operations– Command Source is either on premise or remote,

enabled by a bridge of some sort.– Remote access is enabled within the VPN’s routing

domain

Default Connectivity Model

Connections arecommand source

initiated.

Connections arecommand source

initiated.

Device exposes aservice/API

Device exposes aservice/API

CommandSource

CommandSource

DEMO 1Device Commands with the Default IoT Connectivity Model

Default Connectivity ModelChallenges

• Addressability– Requires network-layer intervention– Doesn’t work for devices that are loosely connected (roaming,

frequently offline)

• Security– By default, every protocol that can be routed over Ethernet can flow –

and between any two nodes– SSL/TLS is not an option on many small devices.– VPN controls access to IP addresses and ports, not application

endpoints (lack of granular authorization)– Many devices are not VPN-capable due to resource/bandwidth

constraints

• Efficient scale– VPN infrastructure is expensive and costly to maintain– Does not address device management.

Think 1K, 10K, 100K+ devices

On-Premise Brokered DeviceCommunications

• Connectivity (IPv6 + VPN)– Give every device a routable IP address.– Devices participate in pub-sub messaging on-prem or

via VPN using industry standard protocol like MQTT.– Command Source is either on premise or remote,

enabled by a bridge of some sort.– Remote access is enabled within the VPN’s routing

domain.

On-Premise Brokered DeviceCommunications

Device subscribes tobroker via TCP, etc.

Device subscribes tobroker via TCP, etc.

Device BrokerDevice Broker

Typically a socketconnection.

Typically a socketconnection.

Messaging happenson premise, attacksurface minimized.

Messaging happenson premise, attacksurface minimized.

MQTT, etc.CommandSource

CommandSource

Must be on premiseor somehow bridged.Must be on premise

or somehow bridged.

DEMO 2Brokered Commands with MQTT & RabbitMQ

On-Premise Brokered DeviceCommunications Challenges

• Addressability– Device and broker are intimately connected.– Doesn’t work for devices that are loosely connected (roaming,

frequently offline).

• Security– SSL/TLS is not an option on many small devices.– Many devices are not VPN-capable due to resource/bandwidth

constraints.

• Efficient scale– VPN infrastructure is expensive and costly to maintain.– External commands require some kind of a gateway service.– Does not address device management.

Think 1K, 10K, 100K+ devices

Service Assisted Communications

• Devices connect via open standard protocols– AMQP 1.0 and HTTP supported natively by the Service Bus– MQTT, CoAP and others can be implemented via custom gateway/adapter model– Sockets secured via TLS (or a lightweight variant)

• Each device has a dedicated Inbox/Outbox on the Gateway– Device sends telemetry/alerts and routes service invocations via its Outbox– Device receives commands and queries from its Inbox– Correlated request/reply patterns can be implemented on top of these two messaging channels– The device knows, and has access to, only its own specific inbox/outbox endpoints (URI’s)

BackendComponents

BackendComponents

Cloud GatewayCloud Gateway

InboxInbox

OutboxOutbox

Com

man

d AP

IC

omm

and

API

Prot

ocol

Hea

dPr

otoc

ol H

ead

Service-Assisted Communications

Connections aredevice-initiated and

outbound

Connections aredevice-initiated and

outbound

NAT/FirewallDevice (Router)

NAT/FirewallDevice (Router)

IP NAT

CloudGateway

CloudGateway

CommandSource

CommandSource

Port mapping isautomatic, outbound

Port mapping isautomatic, outbound

Device does notlisten for unsolicited

traffic

Device does notlisten for unsolicited

traffic

No inbound portsopen, attack surface

is minimized

No inbound portsopen, attack surface

is minimized

Access-controlledcommand API

Secure, managedhosting platform

Access-controlledcommand API

Secure, managedhosting platform

DNSmyapp.cloudapp.net

IoT Cloud Platform “Stack” –Abstract Model

Non-IPCapableDevices

IPCapableDevices C

loud

Gat

eway

CustomCode

Clo

ud P

latfo

rmSe

rvic

es

EnterpriseSystems

Third-Party Dataand Services

A B C D E F

Fiel

dG

atew

ay

Azure Hosting Options

Non-IPCapableDevices

IPCapableDevices C

loud

Gat

eway

CustomCode

Clo

ud P

latfo

rmSe

rvic

es

EnterpriseSystems

Third-Party Data

and Services

A B C D E F

Web SitesWeb Sites

Mobile ServicesMobile Services Cloud ServicesCloud Services

External CodeExternal Code

VM RolesVM Roles

Fiel

dG

atew

ay

Azure Platform Services

Non-IPCapableDevices

IPCapableDevices C

loud

Gat

eway

CustomCode

Clo

ud P

latfo

rmSe

rvic

es

EnterpriseSystems

Third-Party Dataand Services

A B C D E F

HD InsightHD Insight

AzureDatabases

AzureDatabases

Table/BlobStorage

Table/BlobStorage

BizTalk ServicesBizTalk Services

Service BusService Bus

Fiel

dG

atew

ay

Media ServicesMedia Services

Azure – IoT Cloud Gateway

Non-IPCapableDevices

IPCapableDevices C

loud

Gat

eway

CustomCode

Clo

ud P

latfo

rmSe

rvic

es

EnterpriseSystems

Third-Party Dataand ServicesFi

eld

Gat

eway

A B C D E F

Serv

ice

Bus

A/B

Serv

ice

Bus

A/B

Cus

tom

GW

Rol

e

Pattern 1: Device Direct Pattern 2: Custom Gateway

Telemetry Routing with the AzureService Bus

Split the stream Enable parallel processing

Implement different Q QoS levels Level and balance the load

Topic SubsFilters

Service Bus

Device 2

Receiver 2b

Device 1

Device 3Receiver 2a

Alerts

Data

Receiver 1 AlertProcessor

StoragePre-processor

Routing Commands with theAzure Service Bus

TopicSubs Filters

Service Bus

Device 2

Device 1

Device 3Sender 2

Model A

Device 3

Sender 1

Model TModel T

Model A

Target individuals or groups Set delivery timeouts (TTL)

Deal with spotty connectivity Traverse NATs/firewalls

securely

DEMO 3Service-Assisted Device-Direct Commands over Azure Service Bus

Service Assisted Custom/CloudGateway Challenges

• Connectivity– Addressability (non-IP devices, firewalls/NATs, online/offline,

roaming)– Heterogeneity (OS/firmware, power/network constraints,

protocols)– Security (identity, authorization, privacy, data integrity)– Efficient Scale (millions of devices per tenant, at a reasonable

cost)

• Messaging– Telemetry (collection, filtering, routing, throughput, per-

message QoS)– Notifications (targeting devices/device groups within large

populations)– Command/Query and Inquiries (correlation, sessions/batching)

Additional Key Considerations

• Device Provisioning• Security• Performance• Scale• Redundancy

Service Bus MessagingService Bus Messaging

Device Gateway Accelerator –Reference Architecture

(Reykjavík)

1. Custom ProtocolGateway

2. Telemetry Pump andAdapters

3. Command Gateway

4. Provisioning Serviceand Metadata Store

Custom Protocol Gateway HostCustom Protocol Gateway Host

MQTTMQTT CoAPCoAP ……

Telemetry/RequestRouter

Telemetry/RequestRouter

Notification/CommandRouter

Notification/CommandRouter

AdaptersAdapters Command API HostCommand API Host

ProvisioningService

DeviceMetadataand Key

StoreH

DIn

sigh

tH

DIn

sigh

t

Biz

Talk

Biz

Talk

Orle

ans

Orle

ans

Azur

e St

orag

eAz

ure

Stor

age

Azur

eD

bsAz

ure

Dbs

Serv

ice

Bus

Serv

ice

Bus HTTP

HTTP

DevicesDevices

AMQP

11

22 33

44

ConfigurationConfiguration

HTTP

Your

Pro

cess

Your

Pro

cess

Device Gateway – PartitionTopology

• The “Partition” is a set of resources dedicated to a specificdevice population (or subset thereof).

• The “Master” role manages partition deployment and deviceprovisioning into the partitions.

PartitionMaster

PartitionRepo

PartitionRepo

Command TopicsCommand Topics

Service Bus Standard ProtocolService Bus Standard Protocol Custom ProtocolCustom Protocol

Device RepoDevice Repoin0000in0000 inFFFFinFFFF…in0001in0001 in0002in0002

AMQPAMQP HTTPHTTP MQTTMQTT Custom Protocol HostCustom Protocol HostProtocol AdaptersProtocol Adapters

diagdiagallall diagdiagallall diagdiagallall diagdiagallall

Telemetry Pump/RouterTelemetry Pump/RouterN Instances

TelemetryAdapter

TelemetryAdapter

TelemetryAdapter

TelemetryAdapter

TelemetryAdapter

TelemetryAdapter

DeploymentRuntime

DeploymentRuntime

out0000out0000 outFFFFoutFFFF…out0001out0001 out0002out0002

s000

1s0

001

s000

2s0

002

s03E

7s0

3E7

s000

1s0

001

s000

2s0

002

s03E

7s0

3E7

s000

1s0

001

s000

2s0

002

s03E

7s0

3E7

s000

1s0

001

s000

2s0

002

s03E

7s0

3E7

g0000/rte0000g0000/rte0000

g0000/rte0001g0000/rte0001

out0

out0

out1

out1

out2

out2

n Groups of m Routers

out0

out0

out1

out1

out2

out2

g0001/rte0000g0001/rte0000

g0001/rte0001g0001/rte0001

out0

out0

out1

out1

out2

out2

out0

out0

out1

out1

out2

out2

ProvisioningRuntime

ProvisioningRuntime

Ingestion Topics (Telemetry)Ingestion Topics (Telemetry)

CommandAPI Host

CommandAPI Host

Device Gateway – CustomerTopology

• Global coverage achieved by spreading partitions across multiple Azureregions

• Reference architecture supports up to 1000 distinct partitions• Number and distribution of partitions driven by data volumes, business

continuity, legal and proximity considerations

DEMO 4Provisioning and Exercising Reykjavik

Device

Device

EventHub Azure

EventProcessing

AzureStorage

Azure

Customer Apps

HDInsight

BI Systems

3rd Party Solutions

Data Flow

SQLAzure

EventHub

BasicDevice

Registry

Command & Control

Microsoft Azure Stack for IoT

Device(Non-ISS)

Device(Non-ISS)

EventHub

AzureStorage

Rich Device Registry & Object Model of “Things”Rich Device Registry & Object Model of “Things”

Azure

ISS

Customer Apps

HDInsights

BI Systems

3rd Party Solutions

Data Flow

ISS Solution built on Azure

SQLAzure

EventHub

BasicDevice

Registry

ISS

Sec

urity

,P

rivac

y &

Sha

ring

Con

trols

IoT RuleTemplatesIoT Rule

Templates

Natural LanguageQuery

Natural LanguageQuery

ISSAgents

ISSAgents

ISSAgents

ISS Solution

Sin

gle

Acc

ount

, Per

dev

ice

Bill

ing,

etc.

Sin

gle

Acc

ount

, Per

dev

ice

Bill

ing,

etc.

Command & Control

AzureEventProcessing

ISS

Por

tal

ISS

Por

tal

More on ISS

• //build 2014: Windows and the Internet of Things:http://bit.ly/1ijTeyW

• Internetofyourthings.com

More on Reykjavik/DeviceGateway

• //build 2014: Internet of Things with Azure Service Bus:http://bit.ly/1m4MMME

• Neudesic is currently offering industry-specificbriefings on IoT.

• The Azure M2M team is very interested in working withearly adopters.

• If you or your organization think you’re a candidate forDevice Gateway and are interested in learning moreconnect with us:

http://neudesic.com/iotInvitation code: VSLChicago

References

• Internet of Things with Azure Service Bus:http://bit.ly/1m4MMME

• Windows and the Internet of Things:http://bit.ly/1ijTeyW

• Subscribe!: http://channel9.msdn.com/Blogs/Subscribe• Service Assisted Communications:

http://vasters.com/clemensv/CategoryView,category,Architecture.aspx

• Internet of Things & Azure Service Bus:http://bit.ly/1jFf5k5 and http://bit.ly/1jFf5k5

• M2MQTT Library for .NET MF:http://m2mqtt.codeplex.com/

• Special thanks to Clemens Vaster, Markus Horsemanand Todd Holmquist-Sutherland on the Microsoft AzureM2M team.

About Me

• Distinguished Engineer, Neudesic working on IoT,Intelligent Transportation and Hospitality & Gaming

• Microsoft MVP, Microsoft Azure• Co-Author, “Windows Server AppFabric Cookbook”

by Packt Pub.• Chairman, Co-Founder Phoenix Connected

Systems User Group (PCSUG.org)• twitter: @rickggaribay• blog: http://rickgaribay.net• email: rick.garibay@neudesic.com | b-rigari@microsoft.com