sql server 2014 faster insights from any data -level 300 presentation from atidan
TRANSCRIPT
![Page 1: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/1.jpg)
SQL Server 2014 Faster Insights from Any Data(Level 300 Deck)
![Page 2: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/2.jpg)
SQL Server 2014 Faster Insights from Any Data(Level 300 Deck)
![Page 3: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/3.jpg)
SQL Server 2014 Faster Insights from Any Data(Level 300 Deck)
![Page 4: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/4.jpg)
SQL Server 2014 Faster Insights from Any Data(Level 300 Deck)
![Page 5: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/5.jpg)
![Page 6: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/6.jpg)
![Page 7: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/7.jpg)
Easy Access to Data, Big and Small
![Page 8: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/8.jpg)
Microsoft Power BI for Office 365
1 in 4 enterprise customers on Office 3651 Billion Office Users
Analyze Visualize Share Find
Q&A
MobileDiscover
Scalable | Manageable | Trusted
![Page 9: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/9.jpg)
Power Query
![Page 10: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/10.jpg)
Power Query
![Page 11: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/11.jpg)
Powerful Self-Service BI with Excel 2013
![Page 12: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/12.jpg)
Power QueryEnable self-service data discovery, query, transformation and mashup experiences for Information
Workers, via Excel and PowerPivot
Discovery and connectivity to a wide range of data sources, spanning volume as well as variety of data.
Highly interactive and intuitive experience for rapidly and iteratively building queries over any data source, any size.
Consistency of experience, and parity of query capabilities over all data sources.
Joins across different data sources; ability to create custom views over data that can then be shared with team/department.
![Page 13: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/13.jpg)
Power QueryDiscover, combine, and refine Big Data, small data, and any data with Data
Explorer for Excel.
![Page 14: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/14.jpg)
S
Data Sources
Windows Azure
Marketplace
Windows Active
Directory
Azure SQL
DatabaseAzure HDInsight
![Page 15: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/15.jpg)
PowerPivot
![Page 16: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/16.jpg)
PowerPivot
![Page 17: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/17.jpg)
Powerful Self-Service BI with Excel 2013
![Page 18: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/18.jpg)
Introducing PowerPivot
![Page 19: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/19.jpg)
PowerPivot for SharePoint
![Page 20: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/20.jpg)
Power View
![Page 21: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/21.jpg)
Power View
![Page 22: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/22.jpg)
Powerful Self-Service BI with Excel 2013
![Page 23: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/23.jpg)
Introducing Power View
![Page 24: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/24.jpg)
Power View in ExcelExcel Database
server
SQL AS
(Tabular)
Power View
SQL RS
ADOMD.NET
SQL AS
(PowerPivot)
![Page 25: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/25.jpg)
Power View in SharePointBrowser SharePoint
web server
Database
server
SharePoint
app server
SQL AS
(PowerPivot)
SQL AS
(Tabular)
SQL RS
Add-InSQL RS
Power View
![Page 26: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/26.jpg)
Power View for Multidimensional Models• Power View on Analysis Services via BISM
• Native support for DAX in Analysis Services
• Better flexibility: Choice of DAX on Tabular or Multidimensional (cubes)
![Page 27: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/27.jpg)
Architecture
Internet Explorer
Analysis Services
BI Semantic Model
Tabular
SharePoint
(2010 or 2013)
Reporting
Services
Power ViewAnalysis Services
BI Semantic Model
Multidimensional
SQL Server Data Tools
SQL Server Data Tools
1
2
35
6
4
![Page 28: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/28.jpg)
BI Semantic Model: ArchitectureThird-party
applications
Reporting Services
(Power View) Excel PowerPivot
Databases LOB Applications Files OData Feeds Cloud Services
SharePoint
Insights
![Page 29: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/29.jpg)
BISM-MD Object Tabular ObjectCube Model
Cube Dimension Table
Attributes (Key(s), Name) Columns
Measure Group Table
Measure Measure
Measure without MeasureGroup Within Table called “Measures”
MeasuregroupCube Dimension
relationship
Relationship
Perspective Perspective
KPI KPI
User/Parent-Child Hierarchies Hierarchies
Multidimensional-Tabular Mapping
![Page 30: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/30.jpg)
Query Execution ArchitectureClient Application
Analysis Services
Query Parser
Query Processor
sFE Caches
MDX Formula Engine
Storage
Engines
SE Caches Partition Data
Query
SE Evaluation Engine
Tabular Metadata
Metadata
Layer DAX Query Processor
Multi-dimensional
Metadata1
DISCOVER_CSDL_METADATA 2
3
DAX Query 1
2
45
6
7
9
8
3
![Page 31: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/31.jpg)
Power Map
![Page 32: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/32.jpg)
Power Map
![Page 33: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/33.jpg)
Powerful Self-Service BI with Excel 2013
![Page 34: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/34.jpg)
Power Map for Microsoft Excel enables information workers to discover and share new insights
from geographical and temporal data through three-dimensional storytelling.
What Is Power Map?
![Page 35: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/35.jpg)
Map Data
• Data in Excel
• Geo-Code
• 3D and 3 Visuals
Discover Insights
• Play over Time
• Annotate points
• Capture scenes
Share Stories
• Cinematic Effects
• Interactive Tours
• Share Workbook
Power Map: Steps to 3D insights
![Page 36: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/36.jpg)
Map Data
•
![Page 37: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/37.jpg)
Discover Insights
•
•
•
•
![Page 38: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/38.jpg)
Share Stories
•
•
•• Export to Video for Viral!
![Page 39: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/39.jpg)
Power MapExcel Add-in to Enhance Data Visualization
![Page 40: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/40.jpg)
Power BI Site
![Page 41: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/41.jpg)
Power BI Site
![Page 42: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/42.jpg)
Collaborate and Stay Connected with Office 365
Q&A
![Page 43: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/43.jpg)
Q & A
![Page 44: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/44.jpg)
Q & A
![Page 45: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/45.jpg)
Collaborate and Stay Connected with Office 365
Q&A
![Page 46: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/46.jpg)
Mobile BI
![Page 47: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/47.jpg)
Mobile BI
![Page 48: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/48.jpg)
Collaborate and Stay Connected with Office 365
Q&A
![Page 49: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/49.jpg)
Engage customers with smart,
contextual mobile experiences
Boost agility with real-time access to
apps and data from anywhere
Enable Deep Business and Customer ConnectionsVirtually Anytime, Anywhere
![Page 50: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/50.jpg)
Excite and Engage UsersDeliver Immersive, Connected Customer Experiences
Optimize for discovery and reach
Connect with social apps and networks
Real time content and updates
Beautiful experiences + security and performance
![Page 51: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/51.jpg)
Stay Productive on the GoDeliver Familiar, Connected Experiences to a Mobile Workforce
…while ensuring enterprise security, manageability, and compliance
![Page 52: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/52.jpg)
Mobile BI Capabilities Available Today
Browser-based corporate BI solutions on iOS, Android and Windows:
• SharePoint Mobile enhancements
• PerformancePoint Services
• Excel Services
• SQL Server Reporting Services
“Ultimately, the new Microsoft mobile BI solution leads to more revenue for Recall
and gives us deeper customer insight, helping us stay ahead of our competitors.”
Recall Records Management Company Gets Real-Time BI, Boosts Sales with Mobile Solution case study. Full Case study.
![Page 53: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/53.jpg)
SharePoint Mobile 2013
Mobile Browser
Support across
different mobile
devices, including
touch for tablets and
phone
Native Apps
Rich native apps
experience to business
social interactions and
collaboration
Office Hub
A hub to all your doc
storage services and Office
rich editing experience
Learn more: http://blogs.office.com/b/sharepoint/archive/2013/03/06/out-and-about-new-sharepoint-mobile-offerings.aspx
![Page 54: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/54.jpg)
Tablet Touch Experience
![Page 55: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/55.jpg)
Office Web AppsNever be without the tools you need.
Access and share with confidence.
![Page 56: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/56.jpg)
Excel Web App
![Page 57: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/57.jpg)
Excel Web App
Quick Explore
![Page 58: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/58.jpg)
Mobile-Friendly Apps for Office
![Page 59: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/59.jpg)
Extend with Hybrid Cloud Solutions
![Page 60: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/60.jpg)
Extend with Hybrid Cloud Solutions
![Page 61: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/61.jpg)
Extend with Hybrid Cloud Solutions
![Page 62: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/62.jpg)
HDInsight
![Page 63: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/63.jpg)
HDInsight
![Page 64: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/64.jpg)
Key Trends
![Page 65: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/65.jpg)
Big Data Analytics
![Page 66: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/66.jpg)
Internet of things
Audio / Video
Log Files
Text/Image
Social Sentiment
Data Market Feeds
eGov Feeds
Weather
Wikis / BlogsClick Stream
Sensors / RFID / Devices
Spatial & GPS Coordinates
WEB 2.0Mobile
Advertising CollaborationeCommerce
Digital Marketing
Search Marketing
Web Logs
Recommendations
ERP / CRM
Sales Pipeline
Payables
Payroll
Inventory
Contacts
Deal Tracking
Terabytes
(10E12)
Gigabytes
(10E9)
Exabytes
(10E18)
Petabytes
(10E15)
Velocity - Variety - variability
Vo
lum
e
1980
190,000$
2010
0.07$
1990
9,000$
2000
15$Storage/GB
ERP / CRM WEB
2.0
Internet of things
What Is Big Data?
![Page 67: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/67.jpg)
Modern Data Warehousing
![Page 68: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/68.jpg)
Hadoop Distributed Architecture
![Page 69: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/69.jpg)
MapReduce: Move Code to the Data
![Page 70: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/70.jpg)
So How Does It Work?
![Page 71: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/71.jpg)
Distributed Storage
(HDFS)
Query
(Hive)
Distributed Processing
(MapReduce)
OD
BC
Legend
Red = Core
Hadoop
Blue = Data
processing
Gray= Microsoft
integration
points and
value adds
Orange = Data
Movement
Green =
Packages
HDInsight and Hadoop Ecosystem
![Page 72: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/72.jpg)
Record
readerMap Combiner
Partitioner
Shuffle
and sort
ReduceOutput
format
![Page 73: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/73.jpg)
![Page 74: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/74.jpg)
MapReduce: Driver
![Page 75: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/75.jpg)
MapReduce: Mapper
![Page 76: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/76.jpg)
MapReduce: Reducer
![Page 77: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/77.jpg)
MapReduce Summary
77
![Page 78: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/78.jpg)
Programming HDInsight
Hive, Pig, Mahout, Cascading, Scalding, Scoobi, Pegasus…
C#, F# Map/Reduce, LINQ to Hive, Microsoft .NET
management clients
JavaScript Map/Reduce, browser hosted console, Node.js
management clients
PowerShell, cross-platform CLI tools
![Page 79: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/79.jpg)
Authoring Jobs App Integration
Building Developer Experiences
Authoring frameworks and languages
Connectivity
Programmability
Security
Loosely coupled
Lightweight
Low cost to extend
Scenario oriented
Innovation flows
upward
New compute models
Performance
enhancements
Extend breadth & depth
Enable new scenarios
Integrate with current tool
chains
![Page 80: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/80.jpg)
RDBMS vs. Hadoop
![Page 81: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/81.jpg)
Microsoft Hadoop VisionInsights to all users by activating new types of data
![Page 82: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/82.jpg)
Polybase
![Page 83: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/83.jpg)
Polybase
![Page 84: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/84.jpg)
Polybase
84
DBHDFS
SQL Server PDW querying HDFS data, in-situ
=
![Page 85: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/85.jpg)
Polybase in PDW V2
85
Hadoop
HDFS DB
(a) PDW query in, results out
Hadoop
HDFS DB
(b) PDW query in, results stored in HDFS
![Page 86: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/86.jpg)
Sensor
& RFID
Web
Apps
Unstructured data Structured data
Traditional schema-
based DW applications
RDBMSHadoop
Social
Apps
Mobile
Apps
How to overcome the
“impedance mismatch”
Increasingly massive amounts of unstructured data driven by new sources
At the same time, vast amounts of corporate data and data sources, and the bulk of their data analysis
Polybase addresses this challenge for advanced data analytics by allowing native query across PDW and Hadoop, integrating structured and unstructured data
Native Query Across Hadoop and PDW
![Page 87: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/87.jpg)
• Querying data in Hadoop from PDW using regular SQL queries, including
• Full SQL query access to data stored in HDFS, represented as ‘external tables’ in PDW
• Basic statistics support for data coming from HDFS
• Querying across PDW and Hadoop tables (joining ‘on the fly’)
• Fully parallelized, high performance import of data from HDFS files into PDW tables
• Fully parallelized, high performance export of data in PDW tables into HDFS files
• Integration with various Hadoop distributions: Hadoop on Windows Server, Hortonwork and Cloudera.
• Supporting Hadoop 1.0 and 2.0
Native Query Across Hadoop and PDWPolybase Features in SQL Server PDW
![Page 88: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/88.jpg)
Native Query Across Hadoop and PDWCreating “External Tables”
• Internal representation of data residing in Hadoop/HDFS (delimited text files only)
• High-level permissions required for creating external tables
• ADMINISTER BULK OPERATIONS & ALTER SCHEMA
• Different than ‘regular SQL tables’: essentially read only (no DML support)
CREATE EXTERNAL TABLE table_name ({<column_definition>} [,...n ])
{WITH (LOCATION =‘<URI>’,[FORMAT_OPTIONS = (<VALUES>)])}
[;]
Indicates
“External” Table
1
Required location of
Hadoop cluster and file
2
Optional Format Options associated
with data import from HDFS
3
![Page 89: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/89.jpg)
Native Query Across Hadoop and PDWQuerying Unstructured Data
1. Querying data in HDFS and displaying results in table form (using external tables)
2. Joining data from HDFS with relational PDW data
Example – Creating external table ‘ClickStream’:
CREATE EXTERNAL TABLE ClickStream(url varchar(50), event_date date, user_IP
varchar(50)), WITH (LOCATION =‘hdfs://MyHadoop:5000/tpch1GB/employee.tbl’,
FORMAT_OPTIONS (FIELD_TERMINATOR = '|'));
Text file in HDFS with | as field delimiter
SELECT top 10 (url) FROM ClickStream where user_IP = ‘192.168.0.1’ Filter query against data in
HDFS
SELECT url.description FROM ClickStream cs, Url_Description url
WHERE cs.url = url.name and cs.url=’www.cars.com’;
Join data coming from files in
HDFS (Url_Description is a second text file in HDFS)
Query Examples
1
2
SELECT user_name FROM ClickStream cs, Users u WHERE
cs.user_IP = u.user_IP and cs.url=’www.microsoft.com’;
3Join data from HDFS
with relational PDW table(Users is a distributed PDW table)
![Page 90: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/90.jpg)
Native Query Across Hadoop and PDWParallel Data Import from HDFS into PDW
Persistently storing data from HDFS in PDW tablesFully parallelized via CREATE TABLE AS SELECT (CTAS) with external tables as source table and PDW tables (either distributed or replicated) as destination
CREATE TABLE ClickStream_PDW WITH DISTRIBUTION = HASH(url)
AS SELECT url, event_date, user_IP FROM ClickStream
Retrieval of data in HDFS “on-the-fly”
Enhanced
PDW query
engine
CTAS Results
External Table
DMS
Reader
1
DMS
Reader
N
…
HDFS bridge
Parallel
HDFS Reads
Parallel
Importing
Sensor
& RFID
Web
Apps
Unstructured data
Hadoop
Social
Apps
Mobile
Apps
Structured data
Traditional DW
applications
PDW
![Page 91: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/91.jpg)
Sensor
& RFID
Web
Apps
Unstructured data
Social
Apps
Mobile
Apps
HDFS data nodes
Native Query Across Hadoop and PDWParallel Data Export from PDW into HDFS• Fully parallelized via CREATE EXTERNAL TABLE AS SELECT (CETAS) with external tables as
destination table and PDW tables as source
• ‘Round-trip of data’ possible with first importing data from HDFS, joining it with relational data, and then exporting results back to HDFS
CREATE EXTERNAL TABLE ClickStream (url, event_date, user_IP)
WITH (LOCATION =‘hdfs://MyHadoop:5000/users/outputDir’, FORMAT_OPTIONS
(FIELD_TERMINATOR = '|')) AS SELECT url, event_date, user_IP FROM ClickStream_PDW
Enhanced
PDW query
engine
CETAS Results
External Table
DMS
Writer
1
DMS
Writer
N
…
HDFS bridge
Parallel
HDFS Writes
Parallel
Reading
Structured data
Traditional DW
applications
PDW
![Page 92: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/92.jpg)
In-Memory for big data analyticsInteractive Analytics over “Big Data”
92
• SQL Server Analysis Services scaled out to very
large data volumes
• Sourced from “Big Data” sources, e.g.
• Hadoop, Isotope, etc.
• Enterprise data sources (SQL Server, Oracle, SAP,
etc.)
• Built upon the In-Memory Analytics engine
• In-memory, column-store, 10x compression
• Deployment vehicles: Box, Appliance, Cloud
• Customers:
• Skype, Klout, Halo 4, UBS, AdCenter, Windows
Update
XMLAWeb services
External
Data Sources
GW
Mgmt
Deploy
Monitor
AS
Instance
AS
Instance
AS
Instance
Reliable Persistent Storage
Excel, PV
3rd party apps,
tools, etc.
![Page 93: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/93.jpg)
StreamInsight
![Page 94: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/94.jpg)
StreamInsight
![Page 95: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/95.jpg)
StreamInsightManaging Streaming Data In-Memory
•
•
•
Customer benefits
•
•
•
•
95
Event
Output
streamInput
stream
![Page 96: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/96.jpg)
Analyzing Streaming Data in the CloudStreamInsight on Windows Azure
96
StreamInsight on Azure is a cloud scale service for
complex event-processing
Ideal for analyzing streaming data either born on the
cloud, or globally distributed
Customer benefits
• Insights from data in motion
• Elastic scale out in the cloud
• Simplified management via built-in connectivity
• Low TCO of cloud service
![Page 97: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/97.jpg)
Complete and Consistent Data Platform
![Page 98: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/98.jpg)
Call to actionDownload SQL Server 2014 CTP2
Stay tuned for availabilitywww.microsoft.com/sqlserver
![Page 99: SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from Atidan](https://reader030.vdocuments.net/reader030/viewer/2022032514/55d4fa94bb61eb36428b45f7/html5/thumbnails/99.jpg)
© 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other
countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond
to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date
of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION