power bi dataflows bi dataflows… · dataflow capability power bi pro power bi premium scheduled...
TRANSCRIPT
Power BI DataFlowsJulio Granados - [email protected]
Session Goals
• What are the Power BI DataFlows?• Where is the Data Stored?
How did we use to do it?
Decision making
Data PreparationData ModelingData DiscoveryData Insights
Decision MakingData Discovery
Data PreparationData Modeling
Data PreparationData Insights and Predictions
How did we use to do it?
Why were dataflows born?
Data PreparationData Modeling
Decision MakingData DiscoverySome Data Preparation
Data PreparationData Insights and Predictions
Dataflows?
DataFlow
DataSets
Reports And Dashboards
Entity Entity
Metadata
QueryDefintion
Data
Each
Manual or
scheduledrefresh
Workspaces
Dataflows Sources
No Live Query No Direct Query No Streaming Data
DEMO
Create a Simple DataFlow
Demo
Questions
Q: Do dataflows replace Power BI datasets?
DEMO
Dataflow as Source
Demo
Questions Q: Do dataflows replace Power BI datasets?
Sources
Datasets
Reports
Questions
Q: Can I reuse dataflows defined in other workspaces?
DEMO
Using Linked and Computed Entities
Demo
Question
Q: Can I reuse dataflows defined in other workspaces?
Entity Entity
Entity
EntityEntity
Link
Compute• Reference• Join• Append• Incremental• Calculate tables
Question Q: Can I reuse dataflows defined in other workspaces?
Entity Entity
Entity
EntityEntity
Link
Compute
Physical folder
• Reference• Join• Calculate tables
Where is my Dataflow?
Global Or PremiumStorage
Default
Data Lake StoragePreview
Data Lake Storage Gen2 for Power BI
Data Lake Storage Gen2 and Power BI Dataflow Integration
Data Lake Storage gen2(Storage Account)
Create Blob container(Inside Storage Account)
Namedpowerbi
Admin Portal
Data Lake Storage gen2
(Storage Account)
Read Role
Access
Associate
All Write and Execute
How Data Lake Storage Gen2 and Power BI Dataflow Works ?
Blob Container
DataFlow
Data Lake Storage gen2
(Storage Account)
CDM Folder
Refresh Data
Load ProcessCreate Process
What is the Common Data Model Defintion?Shared data language and folder structure for business and analytical applications to use
Common Data Model Schema
Common Data Model Structure
DataFlow Folder
Entity Folder
SnapshotsFolderModel
Data
Blob Container
N Entities
N DataFlows
Definition:• Query• Refresh policie• Data Location• Entity• Entity Type• Columns• Relathionships
Common Data Model And Azure ServicesShared data language and folder structure for business and analytical applications to use
DEMO
Using DataBricks and DataFactoryReading from CMD
Demo
• Q: Dataflows replace Azure Data Factory?
Dataflows replace Data Factory?
Dataflows replace Data Factory?• Q: Dataflows replace Azure Data Factory?
DataFlowData Factory
• Data Engineers• All Sources• All Destinations• Complex transformation• drag and drop• Manually-Orchestated• Scalled ETL(SSIS+DTU)
• Business Users• Bach Sources• Just PBI Dataset destination• Complex transformation by code• Auto-Orchestated• Scalled depends of the PBI node
DEMO
Auto-ML & AI in Dataflows
Demo
Dataflows Pro vs Premium
Dataflow Capability Power BI Pro Power BI PremiumScheduled refresh 8 per day 48 per dayTotal Storage 10 GB/user 100 TB/nodeDataflow Authoring with Power Query Online + +
Dataflow Management within Power BI + +
Dataflows Data Connector in the Power BI Desktop + +
Integration with Azure + +Computed Entities (in-storage transformations via M) +New connectors + +Dataflow incremental refresh +Running on Power BI Premium capacity / Parallel execution of transforms
+
Dataflow linked entities +Standardized Schema / Built-In Support for the Common Data Model + +
AutoML and AI +
When and Why Dataflows?
• User Friendly (Power Query)• Reuse Logic• Auto Orchestration• Integration with Data Lake+CMD• Auto-ML & AI
• Data Discovery• Simple transformations• Quick Action Scenario
Why were dataflows born?
BI Team ML & AI Team
Business team
Development Cycle
Summary Power BI DataFlows Dataflows Components Differences between DataFlows and DataSets Computed and Linked Entities Azure Data Lake Data Lake + Power BI DataFlows Integration Common Data Model Power BI DataFlow and Common Data Model Dataflow with Databricks and DataFactory DataFlow vs Data Factory DataFlow Pro vs Premium AutoML- and AI
Resources
https://go.microsoft.com/fwlink/?linkid=2011419&clcid=0x409 https://docs.microsoft.com/en-us/power-bi/service-dataflows-overview https://docs.microsoft.com/en-us/business-applications-release-
notes/October18/intelligence-platform/power-bi-service/self-service-data-prep https://myignite.techcommunity.microsoft.com/sessions/65437 https://www.microsoft.com/en-us/businessapplicationssummit/video/BAS2018-
2117 https://ssbipolar.com/category/power-bi/dataflows/ https://blog.crossjoin.co.uk/category/dataflows/ https://github.com/Azure-Samples/cdm-azure-data-services-
integration/blob/master/Tutorial/CDM-Azure-Data-Services-Integration-Tutorial.md