microservices and internet of things - meetupfiles.meetup.com/2479462/bluemetal-azureiot.pdf ·...
TRANSCRIPT
Microservices and Internet of ThingsBob FamiliarPractice Direct, Cloud & Services
BlueMetal, Inc.
Blog: TheUndocumentedAPI.com
Follow: @bobfamiliar
Email: [email protected]
BlueMetal At A Glance
Practices Areas Industry Specializations Locations
Cloud & Services
Collaboration & Social
Data Platforms & Analytics
Devices & Mobility
Strategy & Design
Managed Services
Health Care Life Sciences
Financial Services
Professional Services
Boston
New York
Chicago
Modern technology, craftsman quality. We’re an interactive design and technology
architecture firm matching the most experienced consultants in the industry to the most
challenging business and technical problems facing our clients. We seek to understand
your business strategy and technical foundation to craft modern applications that
holistically blend strategic vision, creative design, architecture, and innovation, to exactly
meet your needs and ensure your success.
6 3 3
What does IoT mean to you?
•
•
•
•
•
•
•
How many PCs on planet earth?
•
How many mobile phones?
•
By 2020 how many devices connected to the Internet?
•
Device Management
Event Ingestion & Transformation
Status & Notifications
Analytics & Visualization
Integration
Components of an IoT Solution
Microservice
Microservices are…
• A microservice is a self-contained unit of functionality with
loosely coupled dependencies on other services. Autonomous
• A microservice is a unit of deployment that can be
modified, tested and deployed as a unit without impacting
other areas of a solutionIsolated
• A microservice is stateless therefore it can be horizontally
scaled up and down as neededElastic
• A microservice is fault tolerant and highly available Resilient
• A microservice responds to request in a reasonable amount
of timeResponsive
Microservices are… (cont.)
• The intelligence in a system is found in the endpoints not on the
wire. ESB is an anti-pattern to Microservices. Intelligent
• Microservices rely on asynchronous message-passing to establish a
boundary between componentsMessage Oriented
• Microservices provide API’s for access by developers and
administratorsProgrammable
• Microservices provide an API and/or a console that provides access
to administrative operationsConfigurable
• The lifecycle of a microservice is managed through automation
that includes dev, build, test, staging, production and distributionAutomated
Benefits of a Microservices Approach
• Can be developed alongside existing monolithic applications
providing a bridge to a future stateEvolutionary
• Language agnostic APIs
• Highly decoupledOpen
• No monolith to fall over
• Designed for failureResilient
• Adding, updating and maintaining services and can be done at
velocitySpeed of Development
• Reusable and Composable Reuse
Benefits of a Microservices Approach (cont.)
• Services are deployed independentlyDeployment Governance
• On-demand scaling of smaller services leads to better cost controlScale Governance
• Services can be rewritten and replaced with minimal downstream
impactReplaceable
• New API’s can be released without impacting clients that are using
previous API’sVersioned
• Microservices are typically owned by one team from development
through deploymentOwned
Microservice
Microservice Logical Architecture
• Any client Experience
• SDK for accessing serviceSDK
• Communication protocol such as
REST/HTTPProtocol
• On the wire data modelsModels
• Business Rules/LogicService
• Data AccessDAC
• Model persistent store Store
• Continuous DeliveryAutomation
Protocol
Protocol
DAC
Store
Any Client
Models
Service
SDK
Au
tom
ati
on
Microservice
Microservice Logical Architecture
• Any client / Administrators ConsoleExperience
• Public/Private SDKSDK
• Public / Private APIAPI
• Rules, validations, calculationsService
• On the wire data modelsModels
• Rest, DAC, Model ManagementFrameworks
• Model persistent store Store
• Continuous DeliveryAutomation
Public API Private API
Frameworks
Frameworks
Store
Any Client Admin Console
Models
Public Service
Public SDK Private SDK
Au
tom
ati
on
Private Service
IoT Microservices
1
6
Event Hub Overview
• An event ingestor service that provides event and telemetry ingress to the cloud at massive scale, with low latency and high reliability
• Particularly useful in application instrumentation, user experience or workflow processing, and Internet of Things (IoT) scenarios
• Provides a message stream handling capability that has very different characteristics than traditional enterprise messaging
• EH differs from Enterprise messaging scenarios which commonly require a number of sophisticated capabilities such as sequencing, dead-lettering, transaction support, and strong delivery assurances
• The dominant concern for event ingestion is high throughput and processing flexibility for event streams
Partitioned Consumer Pattern
• A partition is an ordered sequence of events that is held in an Event Hub.
• As newer events arrive, they are added to the end of this sequence.
• A partition can be thought of as a “commit log.”
• Each partition can support 1MB ingress and 2MB egress per second
Partitioned Consumer Pattern
• An Event Hub contains multiple partitions.
• Each partition is independent and contains its own sequence of data.
• As a result, partitions often grow at different rates.
• The default setting for partitions is 16
• This can be lowered to 8 and increased to 32 in the portal
Stream Analytics Overview
• Stream Analytics is a fully managed, real-time stream computation service hosted in Azure
• It provides highly resilient, low-latency, and scalable complex event processing of streaming data
• It enables developers to combine streams of data with historic records or reference data to derive business insights easily and quickly
• Stream Analytics connects to Azure Event Hubs for stream ingestion and the Azure Blob service for reference data
• Data can be written from Stream Analytics to Blobs, Tables, SQL Database or sent to another Event Hub for further downstream processing
Integration with Event Hubs
• Stream Analytics is capable of handling high event throughput of up to 1GB/second.
• Integration with Azure Event Hubs allows the solution to ingest millions of events per second coming from connected devices, clickstreams, and log files, to name a few.
• In order to achieve this, SA leverages the partitioning capability of Event Hubs, which can yield 1MB/s per partition.
• Stream Analytics provides support for partitioning while processing these ingested events along both horizontal and vertical axes.
• A user can partition the computation into a number of logical steps within the query definition, each with the ability to be further partitioned into data-parallel execution elements.
• Over time, Stream Analytics will automatically scale based on the event ingestion rate, complexity of processing, and expected latencies to allow users to customize their workload appropriately.
Home Biomedical Demo
Home Biometrics Pharmaceutical Trial
Walkthrough
Thank you