creating the internet of your things

Creating the Internet of Your ThingsFai Laihoongfai@microsoft.com Global Technology Specialist – IoTMicrosoft Cloud + Enterprise – Enterprise Partner Group

Simplifying IoTAnalytics &

Operationalized InsightsPresentation &

Business ConnectivityDevice Connectivity &

Management

• Azure IoT Hub

• Azure IoT Device SDK w/Cross Platform OS Support

• Azure IoT Gateway SDK w/Cross Platform OS Support

• Azure IoT Hub Device Management

• Cortana Intelligence Suite

• Azure Data Lake

• Azure Machine Learning

• Azure Stream Analytics

• Azure HDInsight Storm

• Power BI

• Azure App Service

• Azure Mobile Services

• Azure API Management

• Azure Logic Apps

Simplifying IoTWorld Class Foundation

Hybrid & On-PremisesAzure Stack will enable you to run IoT

workloads wherever you want

Hyperscale Cloud30 regions worldwide

More than any other hyperscale cloud vendor

Enterprise ProvenLargest Number of Compliance Certifications

ISV and System Integrator Partnerships

End to End Security & PrivacyIndustry Standard Support

Rich Security Services

Data & Analytics EcosystemCortana Intelligence Suite

World Class Support

Rich, Open EcosystemAzure Certified for IoTAzure IoT OSS SDKs

Simplifying IoTOpenness & Interoperability

Open Source SoftwareOPC-UA Support for Windows 10 and Azure IoT

Eclipse Foundation / Kura ConnectorsAzure IoT Device SDK

Azure IoT Gateway SDKOpenT2T

Open Standards & ConsortiumsOpen Mobile Alliance (OMA) / LWM2M

OASIS / MQTTIndustrial Internet Consortium (IIC)

Internet Engineering Task Force (IETF)Open Connectivity Foundation (OCF)

Open Fog ConsortiumEclipse Foundation

OPC Foundation

Connected device Azure IoT Hub

Device-to-Cloud telemetryCloud-to-Device commands

Demo

Analyzing data with Spark StreamingJulian Leejulian.lee@microsoft.com Global Technology Specialist – Advanced AnalyticsMicrosoft Cloud + Enterprise – Enterprise Partner Group

Apache SparkGreat momentum in the industry

Active and large communitySupported by all major big data vendorsFast release cadence

Upcoming in Spark 2.0Dataset is the new unifying APITungsten Phase 2 (3-10x speedup)Structured Streams [ALPHA]

Spark on HDInsightFully Managed Service

100% open source Apache Spark and Hadoop bitsLatest releases of Spark (currently 1.6.1, 1.6.2 and 2.0 are coming)Fully supported by Microsoft and Hortonworks99.9% Azure Cloud SLACertifications: PCI, ISO 27018, SOC, HIPAA, EU-MC

Optimized for data exploration, experimentation and development

Jupyter Notebooks (scala, python, automatic data visualizations)IntelliJ and Eclipse plugins (job submission, remote debugging)ODBC connector for Power BI, Tableau, Qlik, SAP, Excel, etc

Integrated with Azure Data PipelinesBatch pipelines

Azure Data Factory orchestrates Spark ETL jobsPowerBI connector for SparkHA of job submission service

Streaming pipelinesHigh Availability for Spark Streaming using YarnAzure connectors: EventHub, Power BI, Azure SQL, DW, Data Lake

Github: https://github.com/hdinsight/spark-powerbi-connectorBlog:

https://blogs.msdn.microsoft.com/azuredatalake/2016/03/09/saving-spark-dataframe-to-powerbi/

Brief Introduction to Spark 2.0• Unifying DataFrames and Datasets in Scala/Java. DataFrame is just a

type alias for Dataset of Row. • SparkSession - a new entry point that supersedes SQLContext and

HiveContext. • DataFrame-based Machine Learning API emerges as the primary ML

API. Original spark.mllib preserved but development on DataFrame-based API• Machine learning pipeline persistence – save and load ML pipelines• Updated support in R for GLMs, Naïve Bayes, Survival Regression,

Kmeans

Source: Databricks

Some Benchmarks

Source: Databricks

Spark Streaming

Source: Apache Spark

Spark Streaming

Source: Apache Spark

Connected device Azure IoT Hub

Device-to-Cloud telemetryCloud-to-Device commands

Demo