Deep Dive on the Data Management Gateway in Power BI for connectivity to on premise and hybrid scenarios

Majdi BadarinPrincipal Program ManagerMicrosoft

SPC397

Matt Masson Senior Program ManagerMicrosoft

Session Objectives And TakeawaysOverview of Power BI for Office 365Data Management Gateway explained Role of the “IT Admin” in Power BIPower BI Admin CenterHow to enable consistent way for data accessHow to enabling refresh of Excel workbooks in Sharepoint Online

The World of Data is Changing

“By 2015, organizations integrating high-value, diverse, new information types and sources into a coherent information management infrastructure will outperform their industry peers financially by more than 20%.”

– Gartner, Regina Casonato et al., “Information Management in the 21st Century”

Consumerization of IT

10x increase every five years

85% from new data types

Dataexplosion

4.3 connected devices per adult

27%using social media input

Discover Combine Refine

Relational Non-relational Streaming

immersive data

experiences

connecting with worlds data

any data, any

size, anywhere

Self-Service Collaboration Corporate Apps Devices

Analytical

End-To-End Business Analytics Platform

Microsoft Power BI for Office 365

Collaborate in Office 3651 in 4 enterprise customers on Office 365

Insights in Excel1 Billion Office Users

Analyze Visualize Share Find

Q&A

MobileDiscover

Scalable | Manageable | Trusted

Self-service BI with the familiarity of Office and the power of the service

Extend with Hybrid Cloud Solutions

Enable ease of discovery with data search across on premise data

Monitor data usage across your organization to optimize system investments

Enterprise Data Search

Data Usage & Telemetry

Enable manual or scheduled data refresh for reports in Office 365 to on-premise sources

22

Extend with Hybrid Cloud Solutions

Enable ease of discovery with data search across on premise data

Monitor data usage across your organization to optimize system investments

On-Premise Data Access

Data Usage & Telemetry

Enable manual or scheduled data refresh for reports in Office 365 to on-premise sources

22

Extend with Hybrid Cloud Solutions

Enable ease of discovery with data search across on premise data

Monitor data usage across your organization to optimize system investments

On-Premise Data Access

Enterprise Data Search

Enable manual or scheduled data refresh for reports in Office 365 to on-premise sources

22

Power BI – Conceptual Architecture

O365 Power BI

Corporate Data Sources

IT Admin

Data Management Gateway

Corporate Data Catalog

Power BI Admin Center

Data Management Portal

SPO team sites, BI Sites, Q&A

Cloud Data Sources

PublicData Catalog

Data Steward

IW (Author)

IW (Consumer)

Manage Data

Publish raw data sources or expose them as OData feeds

Publish queries

Data Refresh

Data Refresh

Power QueryPower PivotPower ViewPower Map

Publish reports & create featured questions, configure scheduled refresh

Import data, Publish queries

Excel,Tablet/Mobile,Browser – Silverlight, HTML5

Consume interactive reports, ask questions

Configure

Import DataImport Data

Import Data

Import Data

Public Cloud Data Sources

Index

IT Infrastructure Services for Power BIGovern usage, secure access and grant authority Provides a single pane of glass to manage and monitor all services and serversEnabling IT to expose and protect data sources and complex feedsRespond to incidents - service KPIs, SLA metrics, audit logs

Role of the IT Admin in Power BIEnabler of Self Service BIVarying levels of control across data sources, departmentsOversight and monitoring of cloud data accessAbility to make corporate data sources easier to discover, and easier to access

Power BI Admin CenterAccess via link in O365 admin portalConfigure Power BI rolesInstall and monitor the Data Management Gateways for your organizationConfigure access to cloud enabled data sourcesExpose OData feeds to corporate data sources

Demo

Power BI Admin Center and the Data Management Gateway

Data Management Gateway

Enabling Corporate OData Feeds

Enabling Excel Workbook Data Refresh using SharePoint Online

Enabling Discovery in Power Query

capabilities

Power BI Admin CenterData Management Gateway

Data Management Gateway - Conceptual

Power BI Admin CenterAllows IT to configure, manage and monitor access to corporate data sources.

Data Management GatewayConnects to corporate data sources and sends data to Microsoft cloud services through a secure channel (Service Bus).

Corporate Data SourcesThe Gateway can connect to

a variety of data sources.

Secure Credential StoreAll credentials used by the gateway are stored on-premises. Encrypted backup of credentials can also be stored in the cloud to enable business continuity scenarios.

Data Management Gateway - ODataExpose SQL tables or views as OData feedsRequires a primary key or unique indexAutomatically registered in the Data Catalogue

Creates a cloud endpointCurrently only used for discovery and authenticationActual OData results are served from connection to Gateway

Consumable from Power QueryOrg ID authentication, and redirection from cloud to on-premises

Gateway Configuration DetailsCommunicates with cloud services Requires outgoing ports (9350-9354)

Falls back to 443/80 if the other ports aren’t openNo incoming ports from the internet (messages received via Service Bus)

Receives credentials from ClickOnce applicationServes OData feeds to on-prem clientsRequires incoming port

default: 8050 (and 8051 if HTTP is used)Recommended to use HTTPS with signed certificate

Must be able to resolve DNS names for data sources

Data SourcesAvailable system wideUsers and groups configured through the portalStatic credentials used for refresh

Support for SQL Server and Oracle More data sources coming soon

Complex Data Feed functionalityExpose SSIS data flow as SQL view

Corporate OData Feeds

20

Data Management Gateway

Power Query

(1) Using Power Query Anna connects to OData feed

(3) The Data Management Gateway connects to SQL Server using either Windows account or Database account setup by IT when creating the feed

(4) Returns Result

(5) Returns OData feed

Power BI Cloud Services

(2) Redirect to gateway

Demo

Corporate OData Feeds and the Data Catalogue

Data Refresh from Power BI Sites

Data Management

Gateway

Excel Workbook in Sharepoint

OnlineGateway

Cloud Service

(1) Data refresh is triggered from Power BI Sites

(2) Connects to Gateway Cloud Service

(3) Checks whether user is authorized to perform a refresh

(4) Sends command (SQL statement, connection string) to on-premise Gateway

(5) Sends SQL to SQL Server

(6) Return Results

(7) Compress, chunk and return the data

(8) Returns data to Excel Workbook

Current LimitationsNot all data sources can be refreshedCloud: Windows Azure SQL Database, SQL Server (in Azure VM), ODataOn-Prem: SQL Server (2005+), Oracle (10g, 11g, and 11gR2)Power Query support coming soon!

Uses pre-defined credentialsCloud: Embedded in workbook, or read from SharePoint Secure Store On-Prem: Configure through the Power BI Admin Center

Demo

Scheduled Refresh through SharePoint Online

Data Management Gateway

Enabling Corporate OData Feeds

Enabling Excel Workbook Data Refresh using SharePoint Online

Enabling Discovery in Power Query

capabilities

Power BI Admin CenterData Management Gateway

Getting Stated with Power BI for Office 365 previewhttp://office.microsoft.com/en-us/excel/power-bi-download-add-in-FX104087144.aspxPower BI blog http://blogs.msdn.com/b/powerbi/Introduction to Power BI Admin CenterPower BI Admin Center HelpRestore a Data Management GatewayMonitor System Health using the System Health OData FeedPublish SSIS Packages as OData Feed SourcesSupported Data Sources and Data Types

Resources

MySPCSponsored by

connect. reimagine. transform.

Evaluate sessionson MySPC using yourlaptop or mobile device:myspc.sharepointconference.com

© 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.