big data. small data. all data.download.microsoft.com/download/8/8/1/881a3c2b-343b-48d2... ·...
TRANSCRIPT
![Page 1: Big Data. Small Data. All Data.download.microsoft.com/download/8/8/1/881A3C2B-343B-48D2... · 2018-10-15 · Big Data. Small Data. All Data. Stéphane D’Avril-Favreau Technology](https://reader033.vdocuments.net/reader033/viewer/2022053000/5f04706b7e708231d40df8db/html5/thumbnails/1.jpg)
Big Data. Small
Data. All Data.
Stéphane D’Avril-Favreau
Technology Specialist Dataplatform & BI
1
![Page 2: Big Data. Small Data. All Data.download.microsoft.com/download/8/8/1/881A3C2B-343B-48D2... · 2018-10-15 · Big Data. Small Data. All Data. Stéphane D’Avril-Favreau Technology](https://reader033.vdocuments.net/reader033/viewer/2022053000/5f04706b7e708231d40df8db/html5/thumbnails/2.jpg)
Enabling Familiar, Powerful Business Intelligence
Big Data. Small Data. All Data
![Page 3: Big Data. Small Data. All Data.download.microsoft.com/download/8/8/1/881A3C2B-343B-48D2... · 2018-10-15 · Big Data. Small Data. All Data. Stéphane D’Avril-Favreau Technology](https://reader033.vdocuments.net/reader033/viewer/2022053000/5f04706b7e708231d40df8db/html5/thumbnails/3.jpg)
3
Data sources
![Page 4: Big Data. Small Data. All Data.download.microsoft.com/download/8/8/1/881A3C2B-343B-48D2... · 2018-10-15 · Big Data. Small Data. All Data. Stéphane D’Avril-Favreau Technology](https://reader033.vdocuments.net/reader033/viewer/2022053000/5f04706b7e708231d40df8db/html5/thumbnails/4.jpg)
4
Data sourcesNon-Relational Data
![Page 5: Big Data. Small Data. All Data.download.microsoft.com/download/8/8/1/881A3C2B-343B-48D2... · 2018-10-15 · Big Data. Small Data. All Data. Stéphane D’Avril-Favreau Technology](https://reader033.vdocuments.net/reader033/viewer/2022053000/5f04706b7e708231d40df8db/html5/thumbnails/5.jpg)
Microsoft’s technologycan uniquely speed people to insights and action
![Page 6: Big Data. Small Data. All Data.download.microsoft.com/download/8/8/1/881A3C2B-343B-48D2... · 2018-10-15 · Big Data. Small Data. All Data. Stéphane D’Avril-Favreau Technology](https://reader033.vdocuments.net/reader033/viewer/2022053000/5f04706b7e708231d40df8db/html5/thumbnails/6.jpg)
Data sources Non-Relational Data
![Page 7: Big Data. Small Data. All Data.download.microsoft.com/download/8/8/1/881A3C2B-343B-48D2... · 2018-10-15 · Big Data. Small Data. All Data. Stéphane D’Avril-Favreau Technology](https://reader033.vdocuments.net/reader033/viewer/2022053000/5f04706b7e708231d40df8db/html5/thumbnails/7.jpg)
![Page 8: Big Data. Small Data. All Data.download.microsoft.com/download/8/8/1/881A3C2B-343B-48D2... · 2018-10-15 · Big Data. Small Data. All Data. Stéphane D’Avril-Favreau Technology](https://reader033.vdocuments.net/reader033/viewer/2022053000/5f04706b7e708231d40df8db/html5/thumbnails/8.jpg)
![Page 9: Big Data. Small Data. All Data.download.microsoft.com/download/8/8/1/881A3C2B-343B-48D2... · 2018-10-15 · Big Data. Small Data. All Data. Stéphane D’Avril-Favreau Technology](https://reader033.vdocuments.net/reader033/viewer/2022053000/5f04706b7e708231d40df8db/html5/thumbnails/9.jpg)
![Page 10: Big Data. Small Data. All Data.download.microsoft.com/download/8/8/1/881A3C2B-343B-48D2... · 2018-10-15 · Big Data. Small Data. All Data. Stéphane D’Avril-Favreau Technology](https://reader033.vdocuments.net/reader033/viewer/2022053000/5f04706b7e708231d40df8db/html5/thumbnails/10.jpg)
10
Scale out technologies in SQL Server Parallel Data Warehouse
![Page 11: Big Data. Small Data. All Data.download.microsoft.com/download/8/8/1/881A3C2B-343B-48D2... · 2018-10-15 · Big Data. Small Data. All Data. Stéphane D’Avril-Favreau Technology](https://reader033.vdocuments.net/reader033/viewer/2022053000/5f04706b7e708231d40df8db/html5/thumbnails/11.jpg)
PDW 2012 Appliance design
Server types
– Control
– Passive Failover
– Active Compute
Control Server
Passive Failover Server
Active Compute Server
Active Compute Server
Each server is
configured with two
8-core processors
Storage JBOD
Representative ¼ Rack Configuration
The Parallel Data Warehouse (PDW)
Appliance is a single unit made up of two or
more compute nodes all controlled by a single
PDW control machine
![Page 12: Big Data. Small Data. All Data.download.microsoft.com/download/8/8/1/881A3C2B-343B-48D2... · 2018-10-15 · Big Data. Small Data. All Data. Stéphane D’Avril-Favreau Technology](https://reader033.vdocuments.net/reader033/viewer/2022053000/5f04706b7e708231d40df8db/html5/thumbnails/12.jpg)
Smallest (0TB) To Largest (5PB)
• Start small with a few Terabyte warehouse
• Add capacity up to 5 Petabytes
0TB 5 PB
Add
Capacity
Add
Capacity
Largest Warehouse
PB
Start Small And Grow
No Downtime
Start Small With A Few TB and Linearly Scale OUT
![Page 13: Big Data. Small Data. All Data.download.microsoft.com/download/8/8/1/881A3C2B-343B-48D2... · 2018-10-15 · Big Data. Small Data. All Data. Stéphane D’Avril-Favreau Technology](https://reader033.vdocuments.net/reader033/viewer/2022053000/5f04706b7e708231d40df8db/html5/thumbnails/13.jpg)
13
![Page 14: Big Data. Small Data. All Data.download.microsoft.com/download/8/8/1/881A3C2B-343B-48D2... · 2018-10-15 · Big Data. Small Data. All Data. Stéphane D’Avril-Favreau Technology](https://reader033.vdocuments.net/reader033/viewer/2022053000/5f04706b7e708231d40df8db/html5/thumbnails/14.jpg)
![Page 15: Big Data. Small Data. All Data.download.microsoft.com/download/8/8/1/881A3C2B-343B-48D2... · 2018-10-15 · Big Data. Small Data. All Data. Stéphane D’Avril-Favreau Technology](https://reader033.vdocuments.net/reader033/viewer/2022053000/5f04706b7e708231d40df8db/html5/thumbnails/15.jpg)
![Page 16: Big Data. Small Data. All Data.download.microsoft.com/download/8/8/1/881A3C2B-343B-48D2... · 2018-10-15 · Big Data. Small Data. All Data. Stéphane D’Avril-Favreau Technology](https://reader033.vdocuments.net/reader033/viewer/2022053000/5f04706b7e708231d40df8db/html5/thumbnails/16.jpg)
![Page 17: Big Data. Small Data. All Data.download.microsoft.com/download/8/8/1/881A3C2B-343B-48D2... · 2018-10-15 · Big Data. Small Data. All Data. Stéphane D’Avril-Favreau Technology](https://reader033.vdocuments.net/reader033/viewer/2022053000/5f04706b7e708231d40df8db/html5/thumbnails/17.jpg)
![Page 18: Big Data. Small Data. All Data.download.microsoft.com/download/8/8/1/881A3C2B-343B-48D2... · 2018-10-15 · Big Data. Small Data. All Data. Stéphane D’Avril-Favreau Technology](https://reader033.vdocuments.net/reader033/viewer/2022053000/5f04706b7e708231d40df8db/html5/thumbnails/18.jpg)
18
Scale out non-relational data in HDInsight (for Azure or PDW)
![Page 19: Big Data. Small Data. All Data.download.microsoft.com/download/8/8/1/881A3C2B-343B-48D2... · 2018-10-15 · Big Data. Small Data. All Data. Stéphane D’Avril-Favreau Technology](https://reader033.vdocuments.net/reader033/viewer/2022053000/5f04706b7e708231d40df8db/html5/thumbnails/19.jpg)
19
SQL Result setPolyBase
![Page 20: Big Data. Small Data. All Data.download.microsoft.com/download/8/8/1/881A3C2B-343B-48D2... · 2018-10-15 · Big Data. Small Data. All Data. Stéphane D’Avril-Favreau Technology](https://reader033.vdocuments.net/reader033/viewer/2022053000/5f04706b7e708231d40df8db/html5/thumbnails/20.jpg)
20
![Page 21: Big Data. Small Data. All Data.download.microsoft.com/download/8/8/1/881A3C2B-343B-48D2... · 2018-10-15 · Big Data. Small Data. All Data. Stéphane D’Avril-Favreau Technology](https://reader033.vdocuments.net/reader033/viewer/2022053000/5f04706b7e708231d40df8db/html5/thumbnails/21.jpg)
![Page 22: Big Data. Small Data. All Data.download.microsoft.com/download/8/8/1/881A3C2B-343B-48D2... · 2018-10-15 · Big Data. Small Data. All Data. Stéphane D’Avril-Favreau Technology](https://reader033.vdocuments.net/reader033/viewer/2022053000/5f04706b7e708231d40df8db/html5/thumbnails/22.jpg)
![Page 23: Big Data. Small Data. All Data.download.microsoft.com/download/8/8/1/881A3C2B-343B-48D2... · 2018-10-15 · Big Data. Small Data. All Data. Stéphane D’Avril-Favreau Technology](https://reader033.vdocuments.net/reader033/viewer/2022053000/5f04706b7e708231d40df8db/html5/thumbnails/23.jpg)
![Page 24: Big Data. Small Data. All Data.download.microsoft.com/download/8/8/1/881A3C2B-343B-48D2... · 2018-10-15 · Big Data. Small Data. All Data. Stéphane D’Avril-Favreau Technology](https://reader033.vdocuments.net/reader033/viewer/2022053000/5f04706b7e708231d40df8db/html5/thumbnails/24.jpg)
![Page 25: Big Data. Small Data. All Data.download.microsoft.com/download/8/8/1/881A3C2B-343B-48D2... · 2018-10-15 · Big Data. Small Data. All Data. Stéphane D’Avril-Favreau Technology](https://reader033.vdocuments.net/reader033/viewer/2022053000/5f04706b7e708231d40df8db/html5/thumbnails/25.jpg)
![Page 26: Big Data. Small Data. All Data.download.microsoft.com/download/8/8/1/881A3C2B-343B-48D2... · 2018-10-15 · Big Data. Small Data. All Data. Stéphane D’Avril-Favreau Technology](https://reader033.vdocuments.net/reader033/viewer/2022053000/5f04706b7e708231d40df8db/html5/thumbnails/26.jpg)
![Page 27: Big Data. Small Data. All Data.download.microsoft.com/download/8/8/1/881A3C2B-343B-48D2... · 2018-10-15 · Big Data. Small Data. All Data. Stéphane D’Avril-Favreau Technology](https://reader033.vdocuments.net/reader033/viewer/2022053000/5f04706b7e708231d40df8db/html5/thumbnails/27.jpg)
![Page 28: Big Data. Small Data. All Data.download.microsoft.com/download/8/8/1/881A3C2B-343B-48D2... · 2018-10-15 · Big Data. Small Data. All Data. Stéphane D’Avril-Favreau Technology](https://reader033.vdocuments.net/reader033/viewer/2022053000/5f04706b7e708231d40df8db/html5/thumbnails/28.jpg)
![Page 29: Big Data. Small Data. All Data.download.microsoft.com/download/8/8/1/881A3C2B-343B-48D2... · 2018-10-15 · Big Data. Small Data. All Data. Stéphane D’Avril-Favreau Technology](https://reader033.vdocuments.net/reader033/viewer/2022053000/5f04706b7e708231d40df8db/html5/thumbnails/29.jpg)
![Page 30: Big Data. Small Data. All Data.download.microsoft.com/download/8/8/1/881A3C2B-343B-48D2... · 2018-10-15 · Big Data. Small Data. All Data. Stéphane D’Avril-Favreau Technology](https://reader033.vdocuments.net/reader033/viewer/2022053000/5f04706b7e708231d40df8db/html5/thumbnails/30.jpg)
30
![Page 31: Big Data. Small Data. All Data.download.microsoft.com/download/8/8/1/881A3C2B-343B-48D2... · 2018-10-15 · Big Data. Small Data. All Data. Stéphane D’Avril-Favreau Technology](https://reader033.vdocuments.net/reader033/viewer/2022053000/5f04706b7e708231d40df8db/html5/thumbnails/31.jpg)
![Page 32: Big Data. Small Data. All Data.download.microsoft.com/download/8/8/1/881A3C2B-343B-48D2... · 2018-10-15 · Big Data. Small Data. All Data. Stéphane D’Avril-Favreau Technology](https://reader033.vdocuments.net/reader033/viewer/2022053000/5f04706b7e708231d40df8db/html5/thumbnails/32.jpg)
![Page 33: Big Data. Small Data. All Data.download.microsoft.com/download/8/8/1/881A3C2B-343B-48D2... · 2018-10-15 · Big Data. Small Data. All Data. Stéphane D’Avril-Favreau Technology](https://reader033.vdocuments.net/reader033/viewer/2022053000/5f04706b7e708231d40df8db/html5/thumbnails/33.jpg)
![Page 34: Big Data. Small Data. All Data.download.microsoft.com/download/8/8/1/881A3C2B-343B-48D2... · 2018-10-15 · Big Data. Small Data. All Data. Stéphane D’Avril-Favreau Technology](https://reader033.vdocuments.net/reader033/viewer/2022053000/5f04706b7e708231d40df8db/html5/thumbnails/34.jpg)
A Definition of Big Data – 4Vs
Big data: techniques and
technologies that make handling
data at extreme scale economical.
Volume Velocity
Variety Variability
![Page 35: Big Data. Small Data. All Data.download.microsoft.com/download/8/8/1/881A3C2B-343B-48D2... · 2018-10-15 · Big Data. Small Data. All Data. Stéphane D’Avril-Favreau Technology](https://reader033.vdocuments.net/reader033/viewer/2022053000/5f04706b7e708231d40df8db/html5/thumbnails/35.jpg)
35
MapReduce (Job Scheduling/Execution System)
HDFS (Hadoop Distributed File System)
HBase (Column DB)
Hive Mahout
Oozie
Sqoop
HBase/Cassandra/Couch/
MongoDB
Avro
Zo
okeep
er
Pig
Hadoop = MapReduce + HDFS
FlumeCascad-
ingR
Am
bari
HCatalog
![Page 36: Big Data. Small Data. All Data.download.microsoft.com/download/8/8/1/881A3C2B-343B-48D2... · 2018-10-15 · Big Data. Small Data. All Data. Stéphane D’Avril-Favreau Technology](https://reader033.vdocuments.net/reader033/viewer/2022053000/5f04706b7e708231d40df8db/html5/thumbnails/36.jpg)
Web app
optimization
Smart meter
monitoring
Equipment
monitoring
Advertising
analysis
Life sciences
research
Fraud
detection
Healthcare
outcomes
Weather
forecasting
Social network
analysis
Churn
analysis
Traffic flow
optimization
IT infrastructure
optimization
Legal
discovery
Natural resource
exploration
![Page 37: Big Data. Small Data. All Data.download.microsoft.com/download/8/8/1/881A3C2B-343B-48D2... · 2018-10-15 · Big Data. Small Data. All Data. Stéphane D’Avril-Favreau Technology](https://reader033.vdocuments.net/reader033/viewer/2022053000/5f04706b7e708231d40df8db/html5/thumbnails/37.jpg)
37
Windows Azure
![Page 38: Big Data. Small Data. All Data.download.microsoft.com/download/8/8/1/881A3C2B-343B-48D2... · 2018-10-15 · Big Data. Small Data. All Data. Stéphane D’Avril-Favreau Technology](https://reader033.vdocuments.net/reader033/viewer/2022053000/5f04706b7e708231d40df8db/html5/thumbnails/38.jpg)
38
![Page 39: Big Data. Small Data. All Data.download.microsoft.com/download/8/8/1/881A3C2B-343B-48D2... · 2018-10-15 · Big Data. Small Data. All Data. Stéphane D’Avril-Favreau Technology](https://reader033.vdocuments.net/reader033/viewer/2022053000/5f04706b7e708231d40df8db/html5/thumbnails/39.jpg)
![Page 40: Big Data. Small Data. All Data.download.microsoft.com/download/8/8/1/881A3C2B-343B-48D2... · 2018-10-15 · Big Data. Small Data. All Data. Stéphane D’Avril-Favreau Technology](https://reader033.vdocuments.net/reader033/viewer/2022053000/5f04706b7e708231d40df8db/html5/thumbnails/40.jpg)
Easy Access to Data, Big and Small
![Page 41: Big Data. Small Data. All Data.download.microsoft.com/download/8/8/1/881A3C2B-343B-48D2... · 2018-10-15 · Big Data. Small Data. All Data. Stéphane D’Avril-Favreau Technology](https://reader033.vdocuments.net/reader033/viewer/2022053000/5f04706b7e708231d40df8db/html5/thumbnails/41.jpg)
41
![Page 42: Big Data. Small Data. All Data.download.microsoft.com/download/8/8/1/881A3C2B-343B-48D2... · 2018-10-15 · Big Data. Small Data. All Data. Stéphane D’Avril-Favreau Technology](https://reader033.vdocuments.net/reader033/viewer/2022053000/5f04706b7e708231d40df8db/html5/thumbnails/42.jpg)
![Page 43: Big Data. Small Data. All Data.download.microsoft.com/download/8/8/1/881A3C2B-343B-48D2... · 2018-10-15 · Big Data. Small Data. All Data. Stéphane D’Avril-Favreau Technology](https://reader033.vdocuments.net/reader033/viewer/2022053000/5f04706b7e708231d40df8db/html5/thumbnails/43.jpg)
Freedom of deployment options
and hybrid solutions
![Page 44: Big Data. Small Data. All Data.download.microsoft.com/download/8/8/1/881A3C2B-343B-48D2... · 2018-10-15 · Big Data. Small Data. All Data. Stéphane D’Avril-Favreau Technology](https://reader033.vdocuments.net/reader033/viewer/2022053000/5f04706b7e708231d40df8db/html5/thumbnails/44.jpg)
Enabling Familiar, Powerful Business Intelligence
Big Data. Small Data. All Data
![Page 45: Big Data. Small Data. All Data.download.microsoft.com/download/8/8/1/881A3C2B-343B-48D2... · 2018-10-15 · Big Data. Small Data. All Data. Stéphane D’Avril-Favreau Technology](https://reader033.vdocuments.net/reader033/viewer/2022053000/5f04706b7e708231d40df8db/html5/thumbnails/45.jpg)
![Page 46: Big Data. Small Data. All Data.download.microsoft.com/download/8/8/1/881A3C2B-343B-48D2... · 2018-10-15 · Big Data. Small Data. All Data. Stéphane D’Avril-Favreau Technology](https://reader033.vdocuments.net/reader033/viewer/2022053000/5f04706b7e708231d40df8db/html5/thumbnails/46.jpg)