embedded system lab. jung youngjin janus: optimal flash provisioning for cloud storage workloads c....

Click here to load reader

Post on 19-Jan-2018

213 views

Category:

Documents

0 download

Embed Size (px)

DESCRIPTION

정 영 진 Embedded System Lab. Introduction System description Workload characterization Economics and provisioning Optimizing the flash allocation for workloads Optimization with bounded write rates Evaluation Conclusion Contents

TRANSCRIPT

Embedded System Lab. Jung YoungJin Janus: Optimal Flash Provisioning for Cloud Storage Workloads C. Albrecht, A. Merchant, M. Stokely, M. Waliji, F. Labelle, N. Coehlo, X. Shi, and C. E. Schrock. In Proceedings of the annual conference on USENIX Annual Technical Conference, ATC 13, Berkeley, CA, USA, USENIX Association. Embedded System Lab. Embedded System Lab. Introduction System description Workload characterization Economics and provisioning Optimizing the flash allocation for workloads Optimization with bounded write rates Evaluation Conclusion Contents Embedded System Lab. HDD & SSD Disks are slow, even as their capacities grow We can compensate for this by adding flash storage Large cloud environment Many user Many workload Distributing the available flash capacity uniformly between the workloads is not ideal Introduction Embedded System Lab. Janus? Provides flash storage allocation recommendations for workloads in a distributed file system Used in distributed file system, GFS, Colosus Google workload Analyzed workload characterizations Most I/O accesses -> recently created files 28% of read operations -> 1% data Files are placed in the flash upon creation Introduction Embedded System Lab. Recommendation Runs periodically to adjust Many read operation -> flash storage Key input Age of data Read rate of the data by age Janus work step Collect age of data and characterization of how cacheable each workload Allocate flash amongst the workloads Coordination with the distributed file system System description Hybrid Storage Hybrid Storage Colosus or GFS Colosus or GFS Embedded System Lab. Workload A large application have many job Need to define a metric that lets us compare how many read operations would be served Workload characterization Embedded System Lab. Cacheability functions FIFO eviction instance How much data there is of a given age How many reads there are to files of a given age LRU eviction instance Amount of data with a given temporal locality Rate of reads to files with that temporal locality(time gap) Workload characterization SIGELMAN, B. H., ET AL. Dapper, a large-scale distributed systems tracing infrastructure. rep., Google, Inc., 2010. Embedded System Lab. Obtaining instance From file system metadata From trace sample Function input/output Input : size of data Output : the number of read operations Workload characterization Embedded System Lab. Peak IOPS and capacity requirements Economics and provisioning Cost effective to put workloads entirely in flash Cost effective to hot portions of the data on flash Embedded System Lab. Determine the best flash allocation for each workload Primary goal Find maximize the aggregate rate of read operations Instance Workloads with cacheability function Total flash capacity Task Allocate flash to workloads to maximize the weighted flash read rate Optimizing the flash allocation for workloads Embedded System Lab. Secondary goal Bound the flash write rate to reduce flash wear Instance Workloads with cacheability function and write rate Bound on the flash write rate Total flash capacity Task Allocate flash to workloads and determine write probability for each workload to maximize the flash read rate Optimization with bounded write rates Embedded System Lab. Flash hit rate during training Evaluation Embedded System Lab. Flash usage and flash read rate for one workload over time Evaluation Embedded System Lab. Comparison of flash hit rates for alternative allocation methods Evaluation Allocation Method Cell A (low workload variance) Cell B (high workload variance) Optimized28%74% Proportional to read rate 26%64% Single FIFO19%42% Proportional to data size 14%15% Embedded System Lab USENIX Annual Technical Conference, Presentation video Reference Embedded System Lab. Q & A