hypervisor-based prototyping of disaggregated memory and...
Post on 01-Oct-2020
4 Views
Preview:
TRANSCRIPT
Hypervisor-based Prototyping of Disaggregated Memory and Benefits for VM ConsolidationKevin Lim
1, Jichuan Chang
2, Jose Renato Santos
2, Yoshio Turner
2, Trevor Mudge
1, Parthasarathy
Ranganathan2, Steven Reinhardt
1, 3, Thomas Wenisch
1
1University of Michigan, Ann Arbor
2Hewlett-Packard Labs
3Advanced Micro Devices
IntroductionProblem: Impending memory capacity wall for commodity systems!
Opportunity: Optimizing for ensemble-level memory usage.
Proposed Solution: Disaggregated memory to provide memory capacity expansion and dynamic capacity sharing.1
PrototypeGoal: Explore system-level implications of disaggregated memory.
What hypervisor changes are needed? Are new system-level optimizations enabled?
Current Research: Implementing a Xen hypervisor-based prototype of disaggregated memory (Figure B).
The majority of the code changes are in: (1) memory management(2) user-level tools for setting parameters
Functionality: Detects accesses to remote memory, on remote access swaps remote page with local page.
Configuration: 2x Quad-core AMD Opteron 2354, 32 GB DRAM.Large memory workloads: SPEC CPU (perlbench, gcc, bwaves, mcf, zeusmp), PARSEC (blackscholes), quicksort, TPC-H.
ResultsVirtual machines (VM) are setup with 2 or 8 GB of local and remote memory. Percentage and latency of remote memory are varied across runs.Results with the hypervisor-based prototype, Figure C and D, show disaggregated memory provides remote capacity with minimal slowdown.
Virtual Machine ConsolidationDisaggregated memory offers independent memory and CPU scaling, removing the memory bottleneck for VM consolidation. Other benefits are possible through techniques to increase effective memory capacity, such as content-based page sharing (CBPS).
ConclusionsMemory disaggregation addresses the capacity wall while providing greater efficiency and OS-transparency. Our prototype shows the feasibility of implementing disaggregated memory in a widely-used open source hypervisor. Importantly, it enables in-depth research into large-scale workloads and content-based page sharing.
5
A B FIgure A. Disaggregated memory architecture, with memory blade shared by multiple compute blades.
Figure B. Architecture of imple-mented prototype. Modifications are made to hypervisor; part of local memory emulates remote memory.
Future WorkWe plan to use the prototype to model multiple systems sharing a memory blade. We will also explore coordinating page migration with VM scheduling for greater consolidation. One use case is for virtual desktops, where non-shared pages can be evicted when a VM is descheduled (user think time), and the evicted pages are prefeteched when idle VM is rescheduled.
6
0%20%40%60%80%
100%120%140%160%180%
10% 20% 30% 40% 50% 60% 70% 80% 90%% of hot content in shared pages
Over CBPS-only
Over mBlade-only
C D
EFigure E. Potential VM consolidation savings enabled by memory disaggregation with CBPS at the memory blade level.
Figure C. Performance of workloads as percentage of remote memory is varied. SPEC work-loads have 2 GB of memory, all others have 8 GB.
Figure D. Microbenchmark results showing measured remote memory latency versus configured latency.
Hypervisor
OS
AppDIMMDIMM CPUs
DIMMDIMM
Software Stack Server using our hypervisor
1 K. Lim, et al., “Disaggregated memory for expansion and sharing in blade servers,” Proceed-ings of the 36th Annual International Symposium on Computer Architecture, June 2009
Memory blade (enlarged)
Backplane
Protocol engine
Memory controller
Address mapping
CPUsDIMMDIMM
CPUsDIMMDIMM
CPUsDIMMDIMM
CPUsDIMMDIMM
Compute blades
OS
App
Software Stack
Hypervisor
DIM
M
DIM
M
DIM
M
DIM
M
DIM
M
DIM
M
DIM
M
DIM
M
DIM
M
DIM
M
DIM
M
DIM
M
DIM
M
DIM
M
DIM
M
0
5
10
15
20
25
30
35
40
0 µsec 4 µsec 10 µsec 15 µsec 20 µsec
Acc
ess ti
me
per r
emot
e pa
ge, µ
sec
Configured remote memory access latency
0.7
0.8
0.9
1
1.1
1.2
1.3
1.4
1.5
1.6
bwaves gcc mcf perlbench zeusmp blackscholes nsort tpch
Nor
mal
ized
runti
me
0% 10% 20% 30% 40% 50% 90%
1
10
100
1000
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
Relativ
e ca
paci
ty
Year
# of Cores
GB DRAM
Capacity Wall
Time
Per-
serv
er m
em. u
sage
top related