creating the internet of your things
TRANSCRIPT
Creating the Internet of Your ThingsFai [email protected] Global Technology Specialist – IoTMicrosoft Cloud + Enterprise – Enterprise Partner Group
Simplifying IoTAnalytics &
Operationalized InsightsPresentation &
Business ConnectivityDevice Connectivity &
Management
• Azure IoT Hub
• Azure IoT Device SDK w/Cross Platform OS Support
• Azure IoT Gateway SDK w/Cross Platform OS Support
• Azure IoT Hub Device Management
• Cortana Intelligence Suite
• Azure Data Lake
• Azure Machine Learning
• Azure Stream Analytics
• Azure HDInsight Storm
• Power BI
• Azure App Service
• Azure Mobile Services
• Azure API Management
• Azure Logic Apps
Simplifying IoTWorld Class Foundation
Hybrid & On-PremisesAzure Stack will enable you to run IoT
workloads wherever you want
Hyperscale Cloud30 regions worldwide
More than any other hyperscale cloud vendor
Enterprise ProvenLargest Number of Compliance Certifications
ISV and System Integrator Partnerships
End to End Security & PrivacyIndustry Standard Support
Rich Security Services
Data & Analytics EcosystemCortana Intelligence Suite
World Class Support
Rich, Open EcosystemAzure Certified for IoTAzure IoT OSS SDKs
Simplifying IoTOpenness & Interoperability
Open Source SoftwareOPC-UA Support for Windows 10 and Azure IoT
Eclipse Foundation / Kura ConnectorsAzure IoT Device SDK
Azure IoT Gateway SDKOpenT2T
Open Standards & ConsortiumsOpen Mobile Alliance (OMA) / LWM2M
OASIS / MQTTIndustrial Internet Consortium (IIC)
Internet Engineering Task Force (IETF)Open Connectivity Foundation (OCF)
Open Fog ConsortiumEclipse Foundation
OPC Foundation
Connected device Azure IoT Hub
Device-to-Cloud telemetryCloud-to-Device commands
Demo
Analyzing data with Spark StreamingJulian [email protected] Global Technology Specialist – Advanced AnalyticsMicrosoft Cloud + Enterprise – Enterprise Partner Group
Apache SparkGreat momentum in the industry
Active and large communitySupported by all major big data vendorsFast release cadence
Upcoming in Spark 2.0Dataset is the new unifying APITungsten Phase 2 (3-10x speedup)Structured Streams [ALPHA]
Spark on HDInsightFully Managed Service
100% open source Apache Spark and Hadoop bitsLatest releases of Spark (currently 1.6.1, 1.6.2 and 2.0 are coming)Fully supported by Microsoft and Hortonworks99.9% Azure Cloud SLACertifications: PCI, ISO 27018, SOC, HIPAA, EU-MC
Optimized for data exploration, experimentation and development
Jupyter Notebooks (scala, python, automatic data visualizations)IntelliJ and Eclipse plugins (job submission, remote debugging)ODBC connector for Power BI, Tableau, Qlik, SAP, Excel, etc
Integrated with Azure Data PipelinesBatch pipelines
Azure Data Factory orchestrates Spark ETL jobsPowerBI connector for SparkHA of job submission service
Streaming pipelinesHigh Availability for Spark Streaming using YarnAzure connectors: EventHub, Power BI, Azure SQL, DW, Data Lake
Github: https://github.com/hdinsight/spark-powerbi-connectorBlog:
https://blogs.msdn.microsoft.com/azuredatalake/2016/03/09/saving-spark-dataframe-to-powerbi/
Brief Introduction to Spark 2.0• Unifying DataFrames and Datasets in Scala/Java. DataFrame is just a
type alias for Dataset of Row. • SparkSession - a new entry point that supersedes SQLContext and
HiveContext. • DataFrame-based Machine Learning API emerges as the primary ML
API. Original spark.mllib preserved but development on DataFrame-based API• Machine learning pipeline persistence – save and load ML pipelines• Updated support in R for GLMs, Naïve Bayes, Survival Regression,
Kmeans
Source: Databricks
Some Benchmarks
Source: Databricks
Spark Streaming
Source: Apache Spark
Spark Streaming
Source: Apache Spark
Connected device Azure IoT Hub
Device-to-Cloud telemetryCloud-to-Device commands
Demo