building the modern analytics architecture
TRANSCRIPT
![Page 1: Building the Modern Analytics Architecture](https://reader034.vdocuments.net/reader034/viewer/2022051303/58a1ab8b1a28abcf5a8ba614/html5/thumbnails/1.jpg)
![Page 2: Building the Modern Analytics Architecture](https://reader034.vdocuments.net/reader034/viewer/2022051303/58a1ab8b1a28abcf5a8ba614/html5/thumbnails/2.jpg)
Building the Modern Analytics Architecture
Karen López
DBI-B210
![Page 3: Building the Modern Analytics Architecture](https://reader034.vdocuments.net/reader034/viewer/2022051303/58a1ab8b1a28abcf5a8ba614/html5/thumbnails/3.jpg)
Karen Lopez
InfoAdvisors.com @datachick
Senior Project Manager
& Architect
Love Your Data
![Page 4: Building the Modern Analytics Architecture](https://reader034.vdocuments.net/reader034/viewer/2022051303/58a1ab8b1a28abcf5a8ba614/html5/thumbnails/4.jpg)
Modern Analytics Architectural ComponentsArchitecting versus buildingCost, benefit, risk
Building the Architecture in Microsoft AzureAnalytics
Importing external dataUsing Power Query & HDInsight for analysisUsing Power View
Q&A and Call to Action
What we will be doing
![Page 5: Building the Modern Analytics Architecture](https://reader034.vdocuments.net/reader034/viewer/2022051303/58a1ab8b1a28abcf5a8ba614/html5/thumbnails/5.jpg)
Modern Analytics Architectural Components &Concepts
![Page 6: Building the Modern Analytics Architecture](https://reader034.vdocuments.net/reader034/viewer/2022051303/58a1ab8b1a28abcf5a8ba614/html5/thumbnails/6.jpg)
Why Hadoop?BIG DATA
Unstructured data
Cloud
Analytics
Scale
Data LakeData Reservoir
Variable Data
Data versus ProcessingSchemaless
![Page 7: Building the Modern Analytics Architecture](https://reader034.vdocuments.net/reader034/viewer/2022051303/58a1ab8b1a28abcf5a8ba614/html5/thumbnails/7.jpg)
Hadoop Ecosystem (a Zoo)
Distributed Storage(HDFS)
Query(Hive)
Distributed Processing
(MapReduce)
Scripting(Pig)
NoSQL Database(HBase)
Metadata(HCatalog)
Data Integration( ODBC / SQOOP/ REST)
Relational
(SQL Server)
Machine Learning(Mahout)
Graph(Pegasus
)
Stats processin
g(RHadoo
p)
Event Pipeline(Flum
e)
Active Directory (Security)
Monitoring & Deployment
(System Center)
C#, F#, .NET
JavaScript
Pipeline / Workflow
(Oozie)
Azure Storage Vault (ASV)
PDW Polybase
Business Intelligence
(Excel, Power View,
SSAS)World's Data (Azure Data Marketplace)
Event Driven
Processing
LegendRed = Core HadoopBlue = Data processingPurple = Microsoft integration points and value addsYellow = Data MovementGreen = Packages
Via Hadoop Ecosystem pptx, Cindy Gross. Used with permission
![Page 8: Building the Modern Analytics Architecture](https://reader034.vdocuments.net/reader034/viewer/2022051303/58a1ab8b1a28abcf5a8ba614/html5/thumbnails/8.jpg)
Shuffle and sort dataGet from large data to smaller dataParallel processing
MapReduce
![Page 9: Building the Modern Analytics Architecture](https://reader034.vdocuments.net/reader034/viewer/2022051303/58a1ab8b1a28abcf5a8ba614/html5/thumbnails/9.jpg)
SQL-like query language Abstraction on top of MapReduceMetastructure on top of HDFS
Hive
![Page 10: Building the Modern Analytics Architecture](https://reader034.vdocuments.net/reader034/viewer/2022051303/58a1ab8b1a28abcf5a8ba614/html5/thumbnails/10.jpg)
Traditional DW ArchitectureOLTP DB
OLTP DB
OLTP DB
EDWstaging
On Premises
Data
Mart
Data
Mart
![Page 11: Building the Modern Analytics Architecture](https://reader034.vdocuments.net/reader034/viewer/2022051303/58a1ab8b1a28abcf5a8ba614/html5/thumbnails/11.jpg)
Modern DW ArchitectureOLTP DB
OLTP DB
OLTP DB
EDWstaging
On Prem
Microsoft Azure
Analytics Mart
Data
Mart
On Prem
![Page 12: Building the Modern Analytics Architecture](https://reader034.vdocuments.net/reader034/viewer/2022051303/58a1ab8b1a28abcf5a8ba614/html5/thumbnails/12.jpg)
Scale up versus scale out Nodes, not beefier servers Data stays, clusters come and go Quick Create for playing Custom Create for architecting a solution
Scale and Nodes
![Page 13: Building the Modern Analytics Architecture](https://reader034.vdocuments.net/reader034/viewer/2022051303/58a1ab8b1a28abcf5a8ba614/html5/thumbnails/13.jpg)
“Every design decision should include cost, benefit and risk”
- Karen Lopez
![Page 14: Building the Modern Analytics Architecture](https://reader034.vdocuments.net/reader034/viewer/2022051303/58a1ab8b1a28abcf5a8ba614/html5/thumbnails/14.jpg)
Building the Architecture in Microsoft Azure
![Page 15: Building the Modern Analytics Architecture](https://reader034.vdocuments.net/reader034/viewer/2022051303/58a1ab8b1a28abcf5a8ba614/html5/thumbnails/15.jpg)
Blob StorageStorage
Container HDInsight
Cluster
HDInsight Demo Components
PowerShell Azure SQL DB
![Page 16: Building the Modern Analytics Architecture](https://reader034.vdocuments.net/reader034/viewer/2022051303/58a1ab8b1a28abcf5a8ba614/html5/thumbnails/16.jpg)
Demo: HDInsight Components….Let’s get started
![Page 17: Building the Modern Analytics Architecture](https://reader034.vdocuments.net/reader034/viewer/2022051303/58a1ab8b1a28abcf5a8ba614/html5/thumbnails/17.jpg)
Let’s talk billing….Per hourPer node
head nodesecurity nodedata nodes
Data egress, not ingressGeo-redunancy
![Page 18: Building the Modern Analytics Architecture](https://reader034.vdocuments.net/reader034/viewer/2022051303/58a1ab8b1a28abcf5a8ba614/html5/thumbnails/18.jpg)
HDInsight AdvantagesSeparate Blob Storage from ClusterBilling
Pay as you go6 or 12 month plans
Tools you knowSkills you haveSkills users have
![Page 19: Building the Modern Analytics Architecture](https://reader034.vdocuments.net/reader034/viewer/2022051303/58a1ab8b1a28abcf5a8ba614/html5/thumbnails/19.jpg)
Analysis
![Page 20: Building the Modern Analytics Architecture](https://reader034.vdocuments.net/reader034/viewer/2022051303/58a1ab8b1a28abcf5a8ba614/html5/thumbnails/20.jpg)
The Data Story
*http://www.dsireusa.org/incentives/incentive.cfm?Incentive_Code=US37F
http://www.irs.gov/file_source/pub/irs-soi/zipcode08flds.xls
IRSEnergy Tax
Credit Data*
Retail Data Warehouse
HDInsight
Power Query & Power
View
![Page 21: Building the Modern Analytics Architecture](https://reader034.vdocuments.net/reader034/viewer/2022051303/58a1ab8b1a28abcf5a8ba614/html5/thumbnails/21.jpg)
Power BI Stack
Power Query
Power View
Power Pivot
Power Map
![Page 22: Building the Modern Analytics Architecture](https://reader034.vdocuments.net/reader034/viewer/2022051303/58a1ab8b1a28abcf5a8ba614/html5/thumbnails/22.jpg)
Demo: Analysis
Bringing it all together
![Page 23: Building the Modern Analytics Architecture](https://reader034.vdocuments.net/reader034/viewer/2022051303/58a1ab8b1a28abcf5a8ba614/html5/thumbnails/23.jpg)
HDInsight + SQL Server + External Data + Power BI
Wrap up
![Page 24: Building the Modern Analytics Architecture](https://reader034.vdocuments.net/reader034/viewer/2022051303/58a1ab8b1a28abcf5a8ba614/html5/thumbnails/24.jpg)
Breakout Sessions DBI-B213 Details of Microsoft SQL Server 2012 Parallel Data Warehouse Appliance Update 1
DBI-B328 From Zero to Data Insights Using HDInsight on Microsoft Azure
DEV-B341 Using Big Data to Advance Your Cloud Service + Device Solutions
Related content
LabsDBI-H306 What's New in Microsoft SQL Server 2014 for DBAs
Find Me Later At MSE Data Platform
![Page 25: Building the Modern Analytics Architecture](https://reader034.vdocuments.net/reader034/viewer/2022051303/58a1ab8b1a28abcf5a8ba614/html5/thumbnails/25.jpg)
Track resourcesDownload Microsoft SQL Server 2014
http://www.trySQLSever.com
Try out Power BI for Office 365! http://www.powerbi.com
Sign up for Microsoft HDInsight today! http://microsoft.com/bigdata
![Page 26: Building the Modern Analytics Architecture](https://reader034.vdocuments.net/reader034/viewer/2022051303/58a1ab8b1a28abcf5a8ba614/html5/thumbnails/26.jpg)
ResourcesLearning
Microsoft Certification & Training Resourceswww.microsoft.com/learning
msdnResources for Developers
http://microsoft.com/msdn
TechNetResources for IT Professionals
http://microsoft.com/technet
Sessions on Demandhttp://channel9.msdn.com/Events/TechEd
![Page 27: Building the Modern Analytics Architecture](https://reader034.vdocuments.net/reader034/viewer/2022051303/58a1ab8b1a28abcf5a8ba614/html5/thumbnails/27.jpg)
Complete an evaluation and enter to win!
![Page 28: Building the Modern Analytics Architecture](https://reader034.vdocuments.net/reader034/viewer/2022051303/58a1ab8b1a28abcf5a8ba614/html5/thumbnails/28.jpg)
Evaluate this session
Scan this QR code to evaluate this session.
![Page 29: Building the Modern Analytics Architecture](https://reader034.vdocuments.net/reader034/viewer/2022051303/58a1ab8b1a28abcf5a8ba614/html5/thumbnails/29.jpg)
© 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.