comparison of open source virtualization technologies

8/7/2019 Comparison of Open Source Virtualization Technologies

1/44

Rvolution Linux 2008. Ce document est confdentiel.

Fernando Laudares Camargos

Gabriel Girard

Benoit des Ligneris, Ph. [email protected]

Comparative study of

Open Source virtualization & contextualizationtechnologies


2/44


Context (1)

Introduction

Why virtualize the server infrastructure

Virtualization technologies

The experiments

Explanations & anomalies

Which technology is best for ... you ?


3/44


Context (2)

Research executed by Fernando L. Camargos in pursuitof his Masters degree in Computer Science under thedirection of Gabriel Girard (Universit de Sherbrooke)and Benot des Ligneris (Rvolution Linux)

This being a research work, some questions remainunsolved... maybe you can help !


4/44


Why virtualize the server infrastructure (1) ?

Server consolidation is the most mentionned argument


5/44


Why virtualize the server infrastructure (2) ?

Reduction of the purchase and maintenance costs

Compatibility with legacy applications and OSs

Security : environment to execute untrusty applications

Low cost environment for software development

Centralized control/management

Easy backup/restore procedures

Live migration Quick server fail-over

High availability

Virtual appliances

Controled sharing of ressources

Cloud computing

Hardware abstraction

It's ... cool !


6/44


Full virtualization

Para-virtualization

OS-level virtualization (contextualization)

Hardware emulation

Binary translation

Classic virtualization

Virtualization technologies (1)


7/44


Full virtualization

Para-virtualization

OS-level virtualization

(contextualization/containers)

Hardware emulation

binary translation

Classic virtualization

XenXen

Linux-VServerLinux-VServer

OpenVZOpenVZ

KVMKVM

VirtualBoxVirtualBox

KQEMUKQEMU



8/44


virtualization != emulation

QEMU is an emulatorQEMU is an emulator



9/44



Virtualization technologiesVirtualization technologies

partial emulationpartial emulation no emulationno emulation

KQEMUKQEMU KVMKVM VirtualBoxVirtualBox OpenVZOpenVZ XenXen (Linux)(Linux) Linux-VServerLinux-VServer


10/44


2 types of hypervisors:2 types of hypervisors:

Hypervisors type I: KVM, XenHypervisors type I: KVM, Xen Hypervisors type II: VirtualBox, KQEMUHypervisors type II: VirtualBox, KQEMU



11/44


The experiments (1)

virtualization layervirtualization layer overheadoverhead

But of how much ?But of how much ?

To discover, we need to mesure theTo discover, we need to mesure the efficiencyefficiency ofofthe virtualization technologiesthe virtualization technologies

efficiency = performance + scalabilityefficiency = performance + scalability

where :where :


12/44


1) Performance (overhead): one virtual machine only

2) Scalability: several virtual machines

2 types of experiments:2 types of experiments:

The experiments (2)


13/44


Virt. solution Version Server kernel VMs kernel

Host -- 2.6.22-14-server --

Linux-VServer 2.2.0.5 2.6.22.14 2.6.22.14

Xen 3.1.0 2.6.22-14-xen 2.6.22-14-xen

KVM 58 2.6.22-14-server 2.6.15-26-amd64

OpenVZ 5.1 2.6.22-ovz005 2.6.22-ovz005

KQEMU 1.3.0~pre11-6 2.6.22.14-kqemu 2.6.15-26-amd64

VirtualBox 1.5.4_OSE / 1.5.51_OSE2.6.22-14-server 2.6.22.14

Virtualization solutions evaluated in this study

Chosen OSs:

Host: Ubuntu 7.10 VMs: Ubuntu 6.06

Test bed:

Intel Core 2 Duo 6300, 1.86GHz

(x86_64 / VT-x) 4G Memory

Hard drive SATA 80G

The experiments (3)


14/44


64 bit kernel for all technologies

Use of VT extension for KVM, Xen

32 bit VM for VirtualBox

Identical memory allocation per VM for everytechnology but Vserver : 2039 Mo

Bits & Bytes & VMs :Bits & Bytes & VMs :

The experiment (4)


15/44


7 benchmarks (dierent workloads)

Reference : executed in the Linux host (scale = 1) executed inside the virtual machines

4 execution sets

results = the average of the 3 last sets normalized by

the result obtained by the Linux host

Methodology :Methodology :

The experiments Performance (1)


16/44


An equilibrated workload a little bit of everything withoutstressing one particular ressource too much

Metric: given time for the completion of the compilation

Compilation of the Linux kernelCompilation of the Linux kernel

tar xvzf linux-XXX.tar.gzcd linux-XXXmake defconfig # ("New config with default answer to all options")---date +%s.%N && make && date +%s.%N...

make cleandate +%s.%N && make && date +%s.%N...

3x



17/44


0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Compilation of the Linux kernel

Linux-VServerXenOpenVZ

KVMVirtualBox (-hwvirtex o)KQEMU

VirtualBox (-hwvirtex on)QEMU

Nomalizedtime

(kernelvanilla)



18/44


Software for le compression

Using option that yields maximal compression whichconsiderably increases the memory utilisation per process

Metric: given time for the completion of the compression

Bzip2Bzip2

cd /var/tmpcp /home/fernando/Iso/ubuntu-6.06.1-server-i386.iso .date +%s.%N && bzip2 -9 ubuntu-6.06.1-server-i386.iso && date +%s.%N

rm ubuntu-6.06.1-server-i386.iso.bz2...

4x



19/44


0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

Bzip2

XenLinux-VServerVirtualBox (-hwvirtex on)VirtualBox (-hwvirtex o)KVM

KQEMUOpenVZ

QEMU

Normalizedtime

(kernelvanilla)



20/44


Derived from the Netbench benchmark

Emulates the load imposed in a le server by n Windows 95clients

n(umber of clients)=100, t(ime)=300

Metric: throughput (Mb/sec)

DbenchDbench

/usr/local/bin/dbench -t 300 -D /var/tmp 100 # 4x



21/44

Rvolution Linux 2008. Ce document est confdentiel.* no results for VirtualBox* no results for VirtualBox

0.00

0.20

0.40

0.60

0.80

1.00

1.20

Dbench

Linux-VServer

Xen

OpenVZ

KVM

KQEMU

QEMU

Normalizesthroughput

(kernelvanilla)



22/44


Application for low level (bit by bit) data copy

Mesures the performance of the I/O system (hard driveaccess)

2 tests:

copy of a single big le copy of 60G of /dev/zero to /dev/null

Metric : throughput

dddd

...

dd if=/opt/iso/ubuntu-6.06.1-server-i386.iso of=/var/tmp/out.iso

...dd if=/dev/zero of=/dev/null count=117187560 # 117187560 = 60G...rm -fr /var/tmp/* # between execution sets...



23/44

Rvolution Linux 2008. Ce document est confdentiel.* no results for KQEMU nor VirtualBox* no results for KQEMU nor VirtualBox

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

dd (copy of ISO le)

Linux-VServerXen

KVM

OpenVZ

Normalizedthroughput

(kernelvanilla)



24/44

Rvolution Linux 2008. Ce document est confdentiel.* no results for KQEMU nor VirtualBox* no results for KQEMU nor VirtualBox

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

dd (60G /dev/zero --> /dev/null)

Linux-VServerKVMOpenVZ

Xen

Normalizedthroughput

(kernelvanilla)



25/44


A benchmark that can be used to measure several aspects ofthe network performance

TCP Stream test: measure the speed of the exchange of TCP

packets through the network (10 sec.) Metric: throughput (bits/sec)

NetperfNetperf

netserver # in the server..

netperf -H # in the client, 4x



26/44


0

0.2

0.4

0.6

0.8

1

1.2

Netperf (TCP Stream Test)

Virtual Box (-hwvirtex o)Virtual Box (-hwvirtex on)

Linux-VServerXenOpenVZQEMU (-no-kqemu)

KQEMU

KQEMU (-kernel-kqemu)KVM

Normalizedthroughp

ut

(kernelvanilla)



27/44


Similar to Netperf's TCP Stream Test, measures theperformance of le exchange through the network

2 tests:

ISO le: 1 big le (433M) Linux kernel tree: several small les (294M)

Metric: time (sec.)

RsyncRsync

..date +%s.%N && rsync -av ::kernel /var/tmp && date +%s.%N...date +%s.%N && rsync -av ::iso /var/tmp && date +%s.%N...rm -fr /var/tmp/* # between execution sets...



28/44


0.0

0.2

0.4

0.6

0.8

1.0

1.2

Rsync (kernel tree)

OpenVZVirtualBox (-hwvirtex on)VirtualBox (-hwvirtex o)Linux-VServer

XenQEMUKQEMU (-kernel-kqemu)

KQEMUKVM

Normalizedtime

(kernelvanilla)



29/44


0.0

0.2

0.4

0.6

0.8

1.0

1.2

Rsync (ISO le)

OpenVZ

VirtualBox (-hwvirtex on)KQEMUVirtualBox (-hwvirtex o)

Linux-VServerKQEMU (-kernel-kqemu)

XenQEMUKVM

Normalizedtime

(kernelvanilla)



30/44


Measures the performance of a DB server Workload centered in I/O operations in the le system

Metric: throughput (transactions/sec)

SysbenchSysbench

sysbench --test=oltp --mysql-user=root --mysql-host=localhost --debug=off prepare # (1x)sysbench --test=oltp --mysql-user=root --mysql-host=localhost --debug=off run # (4x)

OOn-LLine TTransaction PProcessing

OLTP test statistics:queries performed:

read: 140000write: 50000other: 20000

total: 210000transactions: 10000 (376.70 per sec.)deadlocks: 0 (0.00 per sec.)read/write requests: 190000 (7157.28 per sec.)other operations: 20000 (753.40 per sec.)



31/44


0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Sysbench

Linux-VServer

Xen

KVM

OpenVZ

VirtualBox (-hwvirtex on)

VirtualBox (-hwvirtex o)KQEMUKQEMU (-kernel-kqemu)

QEMU

Normalizedth

roughput

(kernelva

nilla)



32/44


Conclusion :Conclusion :

Linux-VServer: excellent performance. Has presented minimalto no overhead when compared to Linux.

Xen: has shown a great performance in all but the Dbench

benchmark (I/O bound benchmark).

KVM's performance was fairly good for a full virtualizationsolution but should be avoided to run applications thatstrongly rely on I/O.



33/44


Conclusion (cont) :Conclusion (cont) :

OpenVZ has shown a very variable performance (from weak toexcellent). Certainly because of accounting for I/O andbecause of some network optimization for the Network relatedtests.

VirtualBox has presented a good performance for the lecompression and network based benchmarks. Poorperformance for all the other situations.

KQEMU has shown a poor performance for all benchmarks. It

is clear that this virtualization solution does not make a gooduse of the available ressources and its application inproduction servers should be avoided.


Th i S l bili


34/44


1 benchmark (Sysbench, kernel compilation) executed by nVMs concurrently

n = 1, 2, 4, 8, 16 et 32

4 execution sets:

results = average of the last tree execution sets Memory allocation per VM :

Methodology:Methodology:

n Mmoire (Mb)

1 2039*

2 1622

4 811

8 405

16 202

32 101

* 1536 Mb (KQEMU)

The experiments Scalability (1)

Th i t S l bilit (2)


35/44


1 2 3 4 5 6 7 8 91

01

11

21

31

41

51

61

71

81

92

02

12

22

32

42

52

62

72

82

93

03

13

23

3

0

50

100

150

200

250

300

350

400

Sysbench at scale: average throughput per VM

Linux-VServer

Xen

KVM

OpenVZVirtualBox

KQEMU

Number of VMs (n)

Throughput(tra

nsactions/sec)


Th i t S l bilit (3)


36/44


1 2 3 4 5 6 7 8 910

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

0

100

200

300

400

500

600

700

Sysbench at scale: total throughput

Linux-VServer

Xen

KVM

OpenVZVirtualBox

KQEMU

Number of VMs (n)

Throughput(transactions/sec)




37/44



1 2 4 8 16 320

1000

2000

3000

4000

5000

6000

7000

8000

Kernel compilation (VMs average)

HostVserverXen

number of VMs (n)

time(sec

onds)

Th i S l bili


38/44



1 2 4 8 16 32

0

5

10

15

20

25

Kernel scalability

Host

VserverXen

Number of VM

Kernel/

Hours

Conclusion Scalability (1)


39/44


Conclusion :Conclusion :

The eciency of virtualization solutions is strongly related tothe number of VMs executing concurrently (scalability)

Most of the time, one additional VM helps to get the maximum

performance out of a given server (link with the number ofCPU)

More decreases performance as a bottleneck is limitingperformance (CPU/core number important !)

Linux-VServer has shown the best global performance for upto 5-7 VMs.

Xen has proved to be the best full virtualization basedsolution.

Conclusion - Scalability (1)



40/44


KVM has shown a reasonable performance for a full

virtualization solution.

OpenVZ's performance was not what we would expect of acontextualization solution. Our hypothesis is that theaccounting (beancounter) is the root cause of th overhead

VirtualBox has shown an impressive performance, the totalthroughput has more than doubled when the number of VMshas pass from 8 to 16. However, we were unaible to executethis experiment with 32 VirtualBox VMs executingconcurrently.

KQEMU's performance was weak when compared to all othersolutions, independently of the number of VMs in execution.


Which technology to use in each case ?


41/44


A technologicalpoint of view ...

OpenVZ : pure network related applications, thanks tothe optimizations done in the network layer. Notindicated for I/O applications.

Linux-VServer: all kinds of situations (a priori).

Xen can also be used in all kinds of situations but

requires important modications in the guest OS kernelsOR the use of the VT virtualization instructions.

KVM and VirtualBox have proven to be good options fordevelopment environments.

KQEMU has shown weak performance. It is indicated fordevelopment only.

Conclusion : Which technology to use in each


42/44


Results are very dierent for everybenchmark/technology ,so benchmark the technologyyou plan to use with your own mission criticalapplication BEFORE virtualizing your servers (ex: FileServers benchmark).

Only Xen is actually supported by the industry(RedHat, SuSE, Mandrake, IBM, etc.)

KVM is available in the standard Linux kernel : Yeah!But poor performance overall ;-(

Linux-VServer and OpenVZ: they both need a modiedkernel that is not ocially supported by the aforementioned giants of the industry but . . .

Conclusion : Which technology to use in eachcase ?

Future / contextualisation


43/44


Since last OLS, key players like IBM, Intel, and Google

are working hard to include a containerbasedtechnology in the Linux kernel

Lots of patches from OpenVZ gets integrated into thekernel recently and everyone expects that we will

have Really Soon Now a contextualization in the linuxkernel without the need for any kernel hacking

We strongly believe that the integration of acontextualization/container solution is the best way togo for Linux-on-Linux virtualization needs

It will oer VMWare a very strong and completelyopen-source competition

Future / contextualisation


44/44


comparison of open source virtualization technologies

Documents