vnuma in xen · 2017. 12. 14. · i hvm vnuma patches posted by andre przywara circa 2010 i elena u...
TRANSCRIPT
vNUMA in Xen
Wei Liu & Elena Ufimtseva
Chicago – August 18, 2014
Agenda Motives History and status Design Problems Benchmark Future Work
Agenda
I Motives
I History and status
I Design
I Problems
I Preliminary benchmark results
I Future work
Chicago – August 18, 2014 vNUMA in Xen 2 / 19
Agenda Motives History and status Design Problems Benchmark Future Work
Motives
I Cross NUMA node memory access is expensiveI Need to avoid cross node memory access
I Xen is NUMA awareI NUMA aware schedulingI NUMA aware guest memory placement
I Operating system like Linux is NUMA awareI NUMA aware schedulingI NUMA aware memory allocation / migration
I The missing bitsI Memory layout informationI CPU topology
Chicago – August 18, 2014 vNUMA in Xen 3 / 19
Agenda Motives History and status Design Problems Benchmark Future Work
History and status
I PV vNUMA presented in Xen Summit 2010 by Dulloor Raohttp://slidesha.re/1AXsFbu
I HVM vNUMA patches posted by Andre Przywara circa 2010
I Elena Ufimtseva has been working on upstreamable PVvNUMA since 2013
Chicago – August 18, 2014 vNUMA in Xen 4 / 19
Agenda Motives History and status Design Problems Benchmark Future Work
Design: PV and PVH
I Toolstack puts enlightenment information in hypervisor
I Guest memory allocation in accordance with enlightenmentinformation
I Guest retrieves enlightenment information via hypercall duringboot up
Chicago – August 18, 2014 vNUMA in Xen 5 / 19
Agenda Motives History and status Design Problems Benchmark Future Work
Design: PV and PVH
Enlightenment information structure
struct vnuma_info
{
nr_vnodes;
vdistance[nr_vnodes * nr_vnodes];
vcpu_to_vnode[nr_vnodes];
vnode_to_pnode[nr_vnodes];
vmemrange[nr_vnodes];
}
Chicago – August 18, 2014 vNUMA in Xen 6 / 19
Agenda Motives History and status Design Problems Benchmark Future Work
Design: HVM
I Toolstack puts enlightenment information in hypervisor
I Toolstack arranges ACPI tables
I Guest memory allocation in accordance with enlightenmentinformation
I Guest retrieves layout information via ACPI tables during bootup
Chicago – August 18, 2014 vNUMA in Xen 7 / 19
Agenda Motives History and status Design Problems Benchmark Future Work
Problems: vNUMA and other features
PV PVH HVMBallooning Y Y* NPoD N/A ? N
Chicago – August 18, 2014 vNUMA in Xen 8 / 19
Agenda Motives History and status Design Problems Benchmark Future Work
Problems: CPU topology
Chicago – August 18, 2014 vNUMA in Xen 9 / 19
Agenda Motives History and status Design Problems Benchmark Future Work
Benchmark
I HostI 2 sockets, 12 PCPUs, HT disabledI 36GB RAM, 2 NUMA nodesI NUMA balancing enabled
I GuestI 12 VCPUsI 16GB RAM, 2 virtual NUMA nodesI vnodes mapped to different pnodes, vcpu pinned to pnodeI NUMA balancing enabled
I Benchmarks to runI AutonumaI SPECJBBI STREAM
Chicago – August 18, 2014 vNUMA in Xen 10 / 19
Agenda Motives History and status Design Problems Benchmark Future Work
Benchmark: Autonuma
Chicago – August 18, 2014 vNUMA in Xen 11 / 19
Agenda Motives History and status Design Problems Benchmark Future Work
Benchmark: Autonuma
Chicago – August 18, 2014 vNUMA in Xen 12 / 19
Agenda Motives History and status Design Problems Benchmark Future Work
Benchmark: Autonuma
Chicago – August 18, 2014 vNUMA in Xen 13 / 19
Agenda Motives History and status Design Problems Benchmark Future Work
Benchmark: SPECJBB
Chicago – August 18, 2014 vNUMA in Xen 14 / 19
Agenda Motives History and status Design Problems Benchmark Future Work
Benchmark: SPECJBB
Chicago – August 18, 2014 vNUMA in Xen 15 / 19
Agenda Motives History and status Design Problems Benchmark Future Work
Benchmark: STREAM
Chicago – August 18, 2014 vNUMA in Xen 16 / 19
Agenda Motives History and status Design Problems Benchmark Future Work
Benchmark: Conclusion
I vNUMA improves performance for PV guest
I vNUMA has weird result in SPECJBB for PVH guest, butother two benchmarks have good results
Chicago – August 18, 2014 vNUMA in Xen 17 / 19
Agenda Motives History and status Design Problems Benchmark Future Work
Future Work
I basic vNUMA support for all guest types
I Dom0 vNUMA
I address vNUMA compatibility issues with PoD and ballooningfor HVM guest
I address performance issue for PVH
Chicago – August 18, 2014 vNUMA in Xen 18 / 19
Thank you!
Chicago – August 18, 2014 vNUMA in Xen 19 / 19