analytics platform system - neudesic...analytics platform system audie wright, dw & big data...
TRANSCRIPT
Analytics Platform System
Audie Wright, DW & Big Data Specialist
Ofc 425-538-0044, Cell 303-324-2860
Big data. Small data. All data.
Sean Mikha, DW & Big Data Architect
Cell 818-203-1136,
Taking an End-to-End Approach toBI and Analytics
Modernizing Your Data Warehouse for Hadoop
Unlock Insightson Any Data
The traditional data warehouse
“…data warehousing has reached the most significant
tipping point since its inception.
The biggest, possibly most elaborate data
management system in IT is changing.”
– Gartner, “The State of Data Warehousing in 2012”
The traditional data warehouse
Real time data2
Increasing datavolumes
1
Cloud-borndata
4
Increasing datavolumes
1 New data sourcesand types
3
The modern data warehouse
Microsoft’s modern data warehouse
Data Platform
Analytics Platform System
SQL Server 2014
Microsoft Azure HDInsight
Scale out technologies
in Analytics Platform System
0TB 6PB
APS /
HDInsight
APS
APS /
HDInsight
APS /
HDInsight
APS /
HDInsight
APS /
HDInsight
APS /
HDInsight
From terabytes to multi-petabytesScale out relational data to petabytes
Scale Out non-relational data
Scale out non-relational data
in HDInsight
(for Microsoft Azure or APS)
Scale out big data
In-memory performanceIn-memory Columnstore for next-generation performance
Columnstore
index representation
Concurrency and mixed workloadsGreat performance for mixed workloads
Query
Results
Near real-time insightsReal-time with complex event processing
Event Targets
Event Sources
Data complexity: variety and velocity
Petabytes
What is big data?
Hadoop Cluster
What is Hadoop?Distributed, scalable system on commodity HW
Core Services
Operational services Data services
HDFS
SQOOP
FLUME
NFS
LOAD & EXTRACT
WebHDFS
OOZIE
AMBARI
YARN
MAP REDUCE
HIVE &HCATALOG
PIG
HBASEFALCON
compute
&
storage
. . .
. . .
. . compute
&
storage
.
.
Hadoop clusters provide scale-out storage and distributed data processing on commodity hardware
Web app
optimization
Smart meter
monitoring
Equipment
monitoring
Advertising
analysis
Life sciences
research
Fraud
detection
Healthcare
outcomesWeather forecasting
Social network
analysis
Churn
analysis
Traffic flow
optimization
IT infrastructure
optimization
Legal
discovery
Natural resource
exploration
Hadoop offerings on-premise and cloudReal-time with complex event processing
Microsoft Azure
Integrate relational data and HadoopIntegrated query with PolyBase in SQL APS
Analytics
Platform
System
Hortonworks
(Windows, Linux),
Cloudera
Microsoft Azure
HDInsight
Microsoft
HDInsight
Result set
PolyBase
Select…
Microsoft’s modern data warehouse
Data Platform
Analytics Platform System
SQL Server 2014
Microsoft Azure HDInsight
Freedom of deployment options
and hybrid solutions
Appliance vs. Reference Architecture
Buying a Reference Architecture
• Order hardware from a BOM
• Customer builds & configures
• Installs software, drivers, firmware, etc.
• Customer manages multiple support channels
Buying an appliance• Order SKU from a list of configuration
options• Factory builds & tests• Hardware vendor installs & connects• Microsoft validates function &
performance• Hands over the keys to the customer• Microsoft is the single point of contact for
support
Sign up for a free architectural design session for APS with your Microsoft rep
Visit Analytics Platform System at http://www.microsoft.com/aps
Try HDInsight at http://www.windowsazure.com/bigdata
Try SQL Server for data warehousing in Microsoft Azure VMs athttp://www.windowsazure.com
Try Hortonworks Data Platform for Windows at http://www.hortonworks.com/products/hdp-windows/
Try SQL Server 2014 at http://www.microsoft.com/sql/sql-server-2014.aspx
Growth Topology PDW Region Only
Base Unit Scale UnitExtension Base Unit
Growth Topologies Hadoop Region
Min
Extend
.