pact 2010 big data workloads an architect’s perspective lizy k. john university of texas at austin...
TRANSCRIPT
PACT 2010
Big Data WorkloadsAn Architect’s Perspective
Lizy K. JohnUniversity of Texas at Austin
BPOE 2014
PACT 2010
The Buzz with Big Data
3/1/2014
BPOE 2014
Lizy K. John
PACT 2010
BIG DATA - Seeing things we could not see before
3/1/2014
BPOE 2014
Lizy K. John
Analyze massive amounts of data
Derive Insights
Business
Medicine
World Economy
PACT 2010
3/1/2014
BPOE 2014
An Architect would like to know
Lizy K. John
What kind of cores, memory organizations and clustering support needed to support big data
Performance metrics to guide workload partitioning strategies other than use available/affordable nodes
Partitioning considering performance, power, energy
Scaling of computation and communication depending on partitions
Becomes important to understand big data workloads
PACT 2010
Common Definition
“Data that is too large and complex to classify using traditional relational database methods”
-Wikipedia
3/1/2014
BPOE 2014
What is “Big Data”?
Lizy K. John
1 Terabyte?
– Yesterdays “Big Data”
Petabytes? Exabytes?
– Today’s “Big Data”
Zettabytes?
– Tomorrow’s “Big Data”
What does complex mean??
Need a more complete definition
PACT 2010
3/1/2014
BPOE 2014
Some examples
Lizy K. John
Combined Space of all hard drives in 2006
– 160 exabytes
All hard drives sold by Seagate in 2011
– 300 exabytes
The world wide web in 2013
– 4 zettabytes
NSA Utah Data Center in Snowden leaks
– 5 zettabytes (some claimed it to be 1 YB)
Exa = 2^60
Zetta = 2^70
Yotta = 2^80
PACT 2010
3/1/2014
BPOE 2014
Characteristics of Big Data
Lizy K. John
* Not always included in taxonomy
PACT 2010
3/1/2014
BPOE 2014
Big Data Analytics = I got this in the mail the very same week my son turned 16
Lizy K. John
PACT 2010
3/1/2014
BPOE 2014
What’s the Problem?
Lizy K. John
Deriving insights from data NOT a new problem
– Traditional relational databases that contain carefully pruned and organized data
But storage is relatively cheap these days
– Possible to store more data in unstructured form
Need intelligent ways to distill large amounts of data in different formats to actionable KNOWLEDGE
Many different levels to approach this problem…..
PACT 2010
3/1/2014
BPOE 2014
Big Data Stack
Lizy K. John
Algorithms
– PageRank, Genetic Algorithms, SVM, etc.
Frameworks and Implementations
– Map/Reduce (Hadoop), MySQL, NoSQL (Cassandra), etc
Hardware
– SMT, Accelerator Nodes (Intel Phi, GPU), etc
How does workload analysis fit in?
– EVERYONE BENEFITS FROM A DEEP UNDERSTANDING OF A WORKLOAD AND ITS CHARACTERISTICS!
PACT 2010
3/1/2014
BPOE 2014
Are New Benchmarks Needed?
Lizy K. John
Already have industry standard benchmarks!
Critical Question
– Do Big Data workloads have different characteristics than these “traditional” Benchmarks?
– Yes they do!
• TLB Behavior [Wang et al]• I-Cache Behavior [Ferdman et al, Zhen et al, Wang et al]• SMT [Ferdman et al]• Operation Intensity [Wang et al]• Data Volume [Wang et al]
PACT 2010
3/1/2014
BPOE 2014
Why New Benchmarks?
Lizy K. John
I-Cache behavior from Cloudsuite [Ferdman et al]
– Much higher miss rate than traditional benchmarks
– Significant OS contribution to cache behavior
PACT 2010
3/1/2014
BPOE 2014
Why New Benchmarks?
Lizy K. John
OS Activity [Zhen et al]
– Shows percentage of instructions
– Significant variation in kernel/application dynamic instructions
PACT 2010
3/1/2014
BPOE 2014
Why New Benchmarks?
Lizy K. John
I-TLB Behavior from BigDataBench [Wang et al]
– Once again, more misses than traditional benchmarks
PACT 2010
3/1/2014
BPOE 2014
Big Data Characterization Challenges
Lizy K. John
INPUT GENERATION
Input data is critical!
Couple of approaches
– Synthetic data generation • Questionable Veracity
– Grab data from industry• Not always possible
• CAIDA-like
How much data?
– Feasibility vs accuracy
PACT 2010
3/1/2014
BPOE 2014
Common Big Data Domains
Lizy K. John
Databases
– Structured
• Typically relational data• SQL databases
– Unstructured
• Example: document oriented• Generally no fixed table schema
– Semi-structured
PACT 2010
3/1/2014
BPOE 2014
Common Big Data Domains
Lizy K. John
Common NoSQL Databases
– Cassandra
• Industry leading, ultra scalable
– HBase
• Database built on top of Hadoop and HDFS
– MongoDB
• JSON- database with dynamic schema
PACT 2010
3/1/2014
BPOE 2014
Common Big Data Domains
Lizy K. John
Map/Reduce - Hadoop
Key/ Value computation
– Map and Reduce phase
PACT 2010
3/1/2014
BPOE 2014
Common Big Data Domains
Lizy K. John
Graph Algorithms
– Important for Data Mining and Machine Learning
– Graphlab – essentially Hadoop over large graphs
– GraphChi – web scale graph computation
– Streaming graph changes
– asynchronous changes to the graph (i.e changes written to edges are immediately visible to subsequent computation)
– Partitioning Challenges
PACT 2010
3/1/2014
BPOE 2014
Hierarchical Decomposition of Workloads
Lizy K. John
By dividing into functional blocks - e.g. front end, back end, and database.
By subdividing into tasks, task groups, processes, threads, etc.
By dividing considering hardware modules at microarchitectural level – memory subsystem, CPU, disk, etc. eg: consider AMD APUs
Group together tasks in an application that use data from the same rack.
PACT 2010
3/1/2014
BPOE 2014
Entropy Guided Optimizations
Lizy K. John
Partitioning Graph Workloads
– How do we assign work to nodes?
Important Factors
– Data Locality
– Minimize Communication
– Maximize Resource Utilization
Bisection bandwidth
Entropy Guided Optimization
Entropy = (memory-in, memory out, #computations, …other attributes)
PACT 2010
3/1/2014
BPOE 2014
In-Memory Map/Reduce
Lizy K. John
IBM Main Memory Map Reduce (M3R)
– Eliminates intermediate disk writes for Hadoop Map/Reduce Jobs
– Pros
• Significantly speeds up some workloads– 45x on sparse matrix mult
– Cons
• Data must fit in cluster memory
• No failure resilience
PACT 2010
3/1/2014
BPOE 2014
Big Data Benchmarking Challenges
Lizy K. John
WORKLOAD VARIETY
Ton of software stacks required
– Configuration of software platform sometimes more important than workload (see next slide)
A comprehensive benchmark should feature
– Offline (Batch Style Analytics)
– Online (Real Time Analytics)
Seeing positive momentum here!
TPC-* -> Cloudsuite, BigDataBench, etc
PACT 2010
3/1/2014
BPOE 2014
Hadoop Case Study – Optimal Settings
Lizy K. John
What are the optimal framework settings?
– Workload Dependent?
– Hardware Dependent?
– Just set everything to the maximum value??
– Does it matter?
How do engineers setup clusters for new platforms?
– Some “rules of thumb” available, but imprecise
PACT 2010
3/1/2014
BPOE 2014
Hadoop Case Study
Lizy K. John
Standard Hadoop configuration algorithm ):-
hadoop_options = Google(“Best Hadoop Configuration”)
launch_cluster()
if (!cluster_boots || !clients_happy) {
hadoop_options = Permute(hadoop_options)
launch_cluster()
if(!cluster_boots || !clients_happy) {
options = Lookup_Options(Buddy_at_Other_Company)
launch_cluster()
if(!cluster_boots || !clients_happy) {
options = default_options
launch_cluster()
}
}
PACT 2010
3/1/2014
BPOE 2014
Hadoop Case Study (Mapper-Reducer Slots)
Lizy K. John
16m4r
2m2r
32m4r8m8r
CPU Occupancy of TeraSort for different mapper-reducer slots
– Simple app, but different very different execution profile depending on configuration
64m4r
PACT 2010
3/1/2014
BPOE 2014
Hadoop Case Study (Mapper-Reducer Slots)
Lizy K. John
Memory Utilization of TeraSort for different mapper-reducer slots
– Simple app, but different very different execution profile depending on configuration
16m4r2m2r 32m4r8m8r
64m4r
PACT 2010
3/1/2014
BPOE 2014
Hadoop Case Study (Block Size)
Lizy K. John
TeraSort – Higher block size reduces total number of maps
– Simple app, but different very different execution profile depending on configuration
32MB 64MB 128MB 256MB 512MB
PACT 2010
3/1/2014
BPOE 2014
Big Data Benchmarking Frameworks
Lizy K. John
Management frameworks and harnesses essential
Example: AMD SWAT
– Software platform for automating the…..
• creation, deployment, provisioning, execution, and data gathering of synthetic workloads on scalable clusters
Several benchmarks available
– Cloudsuite
– Hadoop
– Graphlab
– Anything you want to plugin!
PACT 2010
Lizy K. John 3/1/2014
PACT 2010
3/1/2014
BPOE 2014
Big Data Benchmarking Challenges
Lizy K. John
Big Cluster
Lots of cores, lots of memory and disk space
– Hard for non-industry researchers
Prohibitively long runtimes
Can we simulate Big Data?• Requires full system simulation• Cloudsuite on Flexus (EPFL)
PACT 2010
3/1/2014
BPOE 2014
Adaptable Scalable Futuristic Benchmark Proxies
Lizy K. John
Generate Clones by setting knobs to appropriate values
Adaptable
Scalable
Futuristic
Benchmark Synthesizer
Application Behavior Space
‘Knobs’ for Changing Program
Characteristcs
Workload Synthesis Algorithm
Synthetic Benchmark
Pre-silicon ModelHardwareCompile and Execute
Workload Characteristics
PACT 2010
3/1/2014
BPOE 2014
Lizy K. John
Abstract
Workload
Model
No. Metric Category
1 Basic block size Control flow predictability 2 Branch taken rate for each branch
3 Branch transition rate
4 Proportion of INT ALU, INT MUL, INT DIV, FP ADD, FP MUL, FP DIV, FP MOV, FP SQRT, LOAD & STORE
Instruction mix
5 Dependency distance distribution Instruction level parallelism
6 Private stride value per static load/storeData locality
7 Data Footprint of the workload
8 Mean and standard deviation of the MLP Memory Level Parallelism (MLP)9 MLP frequency
10 Number of threads Thread level parallelism
11 Thread class and processor assignment
Shared data access pattern and communication characteristics
12 Percentage loads to private data
13 Percentage loads to read-only data
14 Percentage migratory loads
15 Percentage consumer loads
16 Percentage irregular loads
17 Percentage stores to private data
18 Percentage producer stores
19 Percentage irregular stores
20 Shared stride value per static load/store
21 Data pool distribution based on sharing patterns
22 Number of lock/unlock pairs andSynchronization Characteristics
23 Number of mutex objects
24 Number of Instructions between lock and unlock
PACT 2010
3/1/2014
BPOE 2014
Big Data Synthetics? A Possibility?
Lizy K. John
Given challenges in Big Data workloads, this would be useful
But what are the knob settings for “Big Data”
– Need detailed characterization
Benchmark Synthesizer
Program
LocalityInstru
ction
Mix Control F
low
BehaviorApplication
Behavior Space
‘Knobs’ for Changing Program
Characteristcs
Workload Synthesis Algorithm
Synthetic Benchmark
Pre-silicon ModelHardwareCompile and Execute
Workload Characteristics
Thread Level
ParallelismCommunicatio
n
characteristic
s
Data Sharing
Patterns
PACT 2010
3/1/2014
BPOE 2014
Big Data Workload Clones
Lizy K. John
CLONES WILL AVOID COMPLEX SOFTWARE STACKS:
Clones for Hadoop
Clones for Graph Processing
Clones for DSS
Clones for OLAP
Clones for DSS with materialized views
Need detailed characterization
Benchmark Synthesizer
Program
LocalityInstru
ction
Mix Control F
low
BehaviorApplication
Behavior Space
‘Knobs’ for Changing Program
Characteristcs
Workload Synthesis Algorithm
Synthetic Benchmark
Pre-silicon ModelHardwareCompile and Execute
Workload Characteristics
Thread Level
ParallelismCommunicatio
n
characteristic
s
Data Sharing
Patterns
PACT 2010
3/1/2014
BPOE 2014
Tricks from the Old Treasure Chest
Lizy K. John
Search and Sort –
– age old computer science problems
– new issues raised by scale but
Old OLTP, OLAP and DSS
Combination of HPC and Database Ideas
Old Scatter-Gather
Piece-wise modeling
PACT 2010
3/1/2014
BPOE 2014
Conclusion
Lizy K. John
Big Data is here to stay
Increasingly important
Cloud and Big Data will take
the world in unprecedented ways
Appropriate hardware and software need to be developed
Workload metrics to guide partitioning
Need to act now to develop intelligent benchmarks and workload analysis methodology
PACT 2010
3/1/2014
BPOE 2014
Thank You! Questions?
Laboratory for Computer Architecture (LCA)The University of Texas at Austin
lca.ece.utexas.eduLizy K. John
PACT 2010
[1] M. Ferdman, et. al.. 2012. Clearing the clouds: a study of emerging scale-out workloads on modern hardware.SIGARCH Comput. Archit. News 40, 1 (March 2012), 37-48.
[2] Zhen Jia, Lei Wang, Jianfeng Zhan Lixin Zhang, Chunjie Luo. Characterizing Data Analysis Workloads in Data Centers. In Workload Characterization (IISWC), 2013 IEEE International Symposium on. IEEE.
[3] Lei Wang, Jianfeng Zhan, Chunjie Luo, Yuqing Zhu, Qiang Yang, Yongqiang He, Wanling Gao, Zhen Jia, Yingjie Shi, Shujie Zhang, Cheng Zhen, Gang Lu, Kent Zhan, Xiaona Li, and Bizhu Qiu. The 20th IEEE International Symposium On High Performance Computer Architecture (HPCA-2014), February 15-19, 2014, Orlando, Florida, USA.
[4] Huang, Shengsheng, et al. "The HiBench benchmark suite: Characterization of the MapReduce-based data analysis." Data Engineering Workshops (ICDEW), 2010 IEEE 26th International Conference on. IEEE, 2010.
[5] Cooper, Brian F., et al. "Benchmarking cloud serving systems with YCSB."Proceedings of the 1st ACM symposium on Cloud computing. ACM, 2010.
[6] GridMix [Online]. Available: https://hadoop.apache.org/docs/r1.2.1/gridmix.html. (21.10.2013).
[7] PigMix [Online]. Available: https://cwiki.apache.org/confluence/display/PIG/PigMix.(21.10.2013).
[8] PAVLO, A., PAULSON, E., RASIN, A., ABADI, D.J., DEWITT, D.J., MADDEN, S., and STONEBRAKER, M., 2009. A comparison of approaches to large-scale data analysis. In Proceedings of the 2009 ACM SIGMOD International Conference on Management of data ACM, 165-178.
[9] Transaction Processing Performance Council (Online) http://www.tpc.org/default.asp (02-13-2013)
[10] GHAZAL, A., RABL, T., HU, M., RAAB, F., POESS, M., CROLOTTE, A., and JACOBSEN, H.-A., 2013. BigBench: Towards an Industry Standard Benchmark for Big Data Analytics. In SIGMOD ACM, New York, New York, 2013, 197-1208.
[11] SUMBALY, R., KREPS, J., and SHAH, S., 2013. Linkbench: a database benchmark based on the Facebook social graph In Proceedings of the SIGMOD (New York, New Youk, USA2013), ACM, 1185-1196.
[12] Cloudsuite on Flexus[Online]. http://parsa.epfl.ch/cloudsuite/isca12-tutorial.html (02-13-2013). ISCA 2012 Tutorial
[13] Graphlab [Online]. Available: http://graphlab.com/).
[14] Shinnar, A., Cunningham, D., Saraswat, V., & Herta, B. (2012). M3R: increased performance for in-memory Hadoop jobs. Proceedings of the VLDB Endowment,5(12), 1736-1747.
[15] Nambiar, Raghunath Othayoth, and Meikel Poess. "The making of TPC-DS."Proceedings of the 32nd international conference on Very large data bases. VLDB Endowment, 2006.
[16] Breternitz, Mauricio, et al. "Cloud Workload Analysis with SWAT." Computer Architecture and High Performance Computing (SBAC-PAD), 2012 IEEE 24th International Symposium on. IEEE, 2012.
Lizy K. John 3/1/2014
References