openvz performance evaluation

7/29/2019 Openvz Performance Evaluation

1/6

Proceedings of II th International Conference on Computer and Information Technology (ICCIT 2008)25-27 December, 2008, Khulna, BangladeshServer Consolidation Using OpenVZ: Performance Evaluation

Mohuiddin Ahmed, Showayb Zahda, Majed AbbasComputer Science Department

International Islamic University Malaysia

Abstract-Virtualization is abstracting computer resources. It isthe ability to run multiple operating systems simultaneously on asingle physical machine. In this paper we evaluate theperformance of the operating system level virtualization software(OpenVZ). The evaluation will be based on Web hosting, in otherwords, we evaluate the performance of Web servers on twodifferent machines (dubbed as Baru and Lama) and we observethe behavior of OpenVZ and its scalabili ty for Web servers ondifferent architectures (32 bits and 64 bits). The study evaluatesthe CPU, Memory and Throughputs under different loadingconditions.Index Terms - OpenVZ, performance, server consolidation,virtualization.

I. INTRODUCTION"Server consolidation is an approach to the efficient usage

of computer server resources in order to reduce the totalnumber of servers or server locations that an organizationrequires"[1]. Simply put, server consolidation is thecombination of several servers (i.e. Web Server, Mail Server,Java Server, etc) into one physical machine, in order to, savecost and utilize the computer resources.One way to achieve server consolidation is to usevirtualization. Virtualization as a word does not exists in mostEnglish dictionaries. However, the closest word to it inOxford Concise Dictionary is "virtual" which means"Computing: not physically existing as such but made bysoftware to appear to do so" [2]. However, virtual is not a newterm in the computer jargons, it was used before such asvirtual memory, virtual reality, etc.Virtualization in our context according to [3] is defined as"the abstraction of computer resources" and "abstraction is theprocess of separating hardware functionality from theunderlying hardware." In other words, virtualization issharing, and partitioning the resources of computer hardwareamongst more than one running operating systems on a singlephysical machine simultaneously. Each one of the runningoperating systems is called a Virtual Machine (VM).Virtualization offers the ability to run multiple operatingsystems (Linux, Windows, MacOS X) on one physicalmachine at the same time.Currently, virtualization is grabbing the attention of the data

centers as well as the small and medium enterprises due to itsenormous advantages. This led processor manufacturer mainlyIntel and AMD to develop processors to support virtualizationi.e. Intel VT and AMD-V respectively.[11 The corresponding author.1-4244-2136-7/08/$20.00 20081EEE 341

Indeed, virtualization brings a huge economical value to theorganizations. Mainly, it reduces the Total Cost ofOwnership(TCO) by consolidating a bunch of physical servers into onephysical server. For example, lOWeb servers can use onephysical machine instead of using 10 physical machines. Thisreduces a lot of cost -beside the cost of the machines- such aselectricity bills, space, air conditioning, etc. In addition,organizations can couple virtualization with open sourcesoftware solutions (e.g. OpenVZ, Xen) to achieve morereduction in TCO and not to bother anymore about licensingissues. This is handy when multiple virtual machines or virtualmachine backups are kept online for redundancy, restoration,or disaster recovery purposes [4].Virtualization technology can be implemented in several

techniques namely, hardware emulation, full virtualization,paravirtualization, operating system level virtualization,application virtualization, and desktop virtualization.However, the discussion about the differences amongst themis beyond this paper. But, a brief discussion about operatingsystem level virtualization will be provided in OpenVZsection.Besides cost savings, virtualization in general has great

advantages for the organizations, due to the fact that thevirtualization layer isolates and separates the VMs. None ofthe VMs is aware about the existence of other VMs on thesame machine. They seem to each other as if they wererunning on different physical machines. The isolation provideshigher availability and security for the organization. If one ofthe VMs crashes or goes down, it has no effects on otherVMs. Moreover, system administrators can fire up a VM inless than a minute. As for security, running different serviceson different VMs improves the security. For instance, if anintruder manages to hack one server, only that particularserver goes down and others will remain running as if nothinghappened.Live migration of VMs is yet another great advantage of

virtualization. Assume one of the VMs is exhausting thehardware resources, then the administrator can migrate theserver into another physical machine in less than one minute.Or if the physical machine should go down for upgrade ormaintenance, the VMs can be migrated to another machine.Therefore, migration boosts the availability of the runningservers.Software developers and testers use virtualization to run

several platforms to develop and test the products on (i.e.Linux, Windows, MacOS X). It reduces the time of installingthe platforms as well as increasing the number of platforms


2/6

that can be used for testing and development.Virtualization is indeed a very helpful technology for

education purposes. Now, universities and educationalinstitutions no longer need to buy extra machines to cater theirstudent's need of running several operating systems.Moreover, each student can have a root access which allowsthem to have full authority over the system.Virtualization has disadvantages too; a big damage to all the

running servers might occur if the power of the host machinegoes down. Or, if the administrator is upgrading the systemhardware, he/she has to shutdown the machine. However, byusing migration, the system administrators can easily migratethe VMs to another physical machine in less than one second.The rest of the paper is organized as follows: in section IIOpenVZ is briefly described, in section III we describe themethodology of the performance evaluation, section IVdescribes the test bed and its components, in section V weanalyze the test results followed by concluding remarks.

II.OPENVZOpenVZ is a free and open source virtualization solution. It isthe basis of the commercial virtualization solution Virtuozzofrom Parallel (formerly known as SWsoft). OpenVZimplements the operating system level virtualization techniquewhich virtualizes servers on the operating system (kernel)layer [5]. In other words, OS-level Virtualization adds a newlayer on top of the operating system. This layer manages allthe issues related to OpenVZ containers and resourcemanagement.

the containers from each other. However, sharing one kernelintroduces a drawback. In case the kernel crashes all thecontainers are down.The performance of OpenVZ is closed to the native

performance of the machine. The overhead only consumesabout 1-3% of the resources. This indeed makes OpenVZ agood choice for server consolidation especially for webhosting [7].OpenVZ is based on Linux operating system and it onlyruns in Linux distributions. Moreover, OpenVZ can also be

tested without installation using OpenVZ LiveCDI.It takes a few minutes to setup one container using

OpenVZ. This is because of the usage of Operating SystemTemplates. An OS-template is a set of packages from someLinux distribution used to populate a VE. With OpenVZ,different distributions can co-exist on the same hardware boxso multiple OS templates are available. [6] ,Furthermore, changing the configuration of any container

can be done on the fly. There is no need to restart anycontainer in order to give or take from it some resources.OpenVZ comes with a command line control utilities that

allow the administrators to create, start, stop, destroy andmigrate a container. The utilities also allow him/her to set thequota, manage the resources and execute commands on thecontainers from outside the container itself. For instance,"vzsplit" is a command that creates a configuration file whichsplits the resources of the computer equally amongst a numberof containers the user specifies (vzsplit -n 4).

V'Jt'I>V7 O p p nV ? " ' 1. H l. t q" " ,; ' nt f ( ' ol I ' ' : o l l l l d foil .,1,,)(sa- _ 10- History aoolcmarb ]t)01s .tierP . (> ~ . .. ~ f - ~ h t t ' p ; ; i i O c ' a i h o s t : - j Q o o i m e n u ; i i S t _ v P ; : ~ ~ 1 g ~ j ' " _.__..... '--rcl

__, ._- ......:.-, .. - - - - - - - - - - ~ ~ m

. ~ . : :i: ._ . :1 '.... ... .... , ~ _ ~ .

Fig. 2. WebVZ Management ToolWebVZ is a web management tool for OpenVZ and was

developed by the authors and available in [8] under GPLGNU. WebVZ allows system administrators to control thecontainers locally and remotely. Fig 2 shows a screenshot ofWebVZ. Resource allocation among the containers is done onthe basis of demand. If there is need to migrate a container toanother server due to resource hogging, this can easily beattained by migration to a different machine which is lightly

Fig. 1. Operating System level Virtualization [5]

OpenVZ has its own terminologies regarding virtualizationtechnology. The physical machine that is running OpenVZ iscalled a Host System or Hardware Node. A running instanceof the operating system that partitions and shares the resourcesof the hardware node is called a container, VirtualEnvironment (VE), and/or Virtual Private Server (VPS) [6]. Inthis paper we will refer to the running instance as a container.Operating system-level virtualization shares the kernelamongst all running containers. The kernel (OpenVZ kernelcreates the virtualization layer) bears the duty of managing 1 Live CD: is an operating system that runs without the need forand partitioning the resources of the computer, and isolating installation on the hard disk and it does not have any effect on the existingdata. Once the CO is ejected, everything goes back to its original state

342


3/6

Fig 3 vzlist output

HOSTNAMEcentosS._lcentos5 2centos5 .3centos5 4centos5 5centos5 6centosS. 7centosS_8centosS 9centosS 18centosS 11ce.ntosS 12centosS 13centosS 14centosS 15centosS 16centosS 17centosS 18centosS .. 19centosS 28

loaded. Fig 3 shows the output of the command "vzlist" (listall containers)[root@\oca\host -] #[root@localhost - ]# vzlis t

VElD NPROC STATUS IP , .ADDR181 16 runn1ng 192.168.8.1182 16 running 192.168.8.2183 16 running 192.168.8.3184 16 running 192.168.8.4185 16 running 192.168.8.5186 16 running 192.168.8.6187 16 running 192.168.8.7188 16 running 192.168.8.8189 16 running 192.168.8.9118 16 running 192.168.8.18111 16 running 192.168.8.11112 16 runn1ng 192.168.8.12113 9 running 192.168.8.13114 9 running 192.168.8.14115 9 running 192.168.8.15116 9 running 192.168.8.16117 8 running 192.168.8 .17118 7 running 192.168.8.18119 7 running 192.168.8.19128 7 running 192.168.8.28(root@locathost -] #

III. METHODOLOGYIn order to test the migration between containers we

injected http queries from a server and then used a wrapper toincrease the number of requests until the system chokes orcrashes.In order to evaluate the performance of the containers onOpenVZ kernel, first we evaluated the performance of bothmachines (we named them Baru and Lama respectively) as asingle unit without running OpenVZ kernel so as to know howmany requests per second each machine can handle and tofind out the relationships between the containers and the HostSystem. Second, we evaluated the containers by scaling thenumber of containers from 4 to 8, and finally to 12 on the twomachines. All the containers receive the same amount ofrequests per second concurrently. In other words, eachcontainer for example is receiving 100 connections per secondat the same time.

In order to measure the performance of each running Webserver we used a free and open source HTTP performancemeasurement tool from HP called "httperf' version 0.9.0.According to the manual page of httperf "httperf speaks theHTTP protocol both in its HTTP/I.O and HTTP/I.l flavorsand offers a variety of workload generators. While running, itkeeps track of a number of performance metrics that aresummarized in the form of statistics that are printed at the endof a test run. The most basic operation of httperf is to generatea fixed number of HTTP GET requests and to measure howmany replies (responses) came back from the server and atwhat rate the responses arrived."However, httperf is limited to one test at a time which

generates a fixed number of HTTP GET. In order to runhttperf several times and increase the number of HTTP GETseach time, another tool called "autobench" can be used whichautomates web server benchmarking using httperf. Accordingto autobench version 2.1.1 manual page "autobench runshttperf against the specified host or hosts, ramping up thenumber of requested connections, and logging the results inTSV or CSV format f i l e s . ~ ~

343

In order to capture the reading of the CPU and the RAM weused "top -b -d 1" to read the values every second. While thethroughput of each test was reported by "httperf for plottingand display the result, "bench2graph" is a tool comes withautobench, was used to plot the output of the tests.This paragraph explains how autobench runs httperf. (E.g.autobench --single_host --host1 192.168.0.1 --low_rate 50 -high_rate 100 --rate_step 10 --timeout 7 --file ve1.tsv). This

command will run "httperf' 5 times. First time httperf initiates50 connections per second until the total number ofconnections reaches 5000 connections (the default) which isthe total number of connections that are initiated during onetest, and each connection contains 10 requests (the default). Inthe second test httperf ini tiates 60 connections per seconduntil the total number of connections reaches 5000 and so on,each time the number of connections per second increasesaccording to the specified rate step value. The output ofhttperf is shown in Fig 4 which is extracted and tabulated byautobench as shown in Fig 5.

htIpeIf -wmeout-7 --dent-()Il --"''92.168.0.1 --f)Oft-80 ..\lill/ .-rate-5C) -eencHMIer-4096_rectt d t e r a l " ~ ~ 1 0MuillUnCOMICI N I t ~ : 1

Comedlon ..: 50,0 COM'S (20.0msJcoM.


4/6

Fig. 6. The Test Bed architecture

Fig. 7. Configuration of a typical container

V. DISCUSSIONA.OpenVZThere are general observations about OpenVZ that we

would like to talk about in this section.It is easy to fire up a new container using OpenVZ. It would

take about one minute. Moreover, a container occupies only asmall amount of memory. However, this depends on the sizeof the distribution as well as the running services. In our case,we only fired up the web server and the default services thatrun on the Centos 5 template from OpenVZ Website. Wefound that each VE takes around 25 MB of RAM and about400 MB of hard disk. Accordingly only 20 containers can berunning web servers using 450 MB of RAM and about 8 GBof hard disk.

We ran 55 containers on Baru and 23 on Lama at the sametime. This almost filled the memory of each machinerespectively. Moreover, we ran the containers with noapplication overhead within itself.

B. The host machine (Baru)After running "autobench" (from 1000 to 7000 requests persecond), the machine managed to handle about 6000 requests

per second with no error. This amount of concurrent requestshas saturated the CPUs of the machine. However, thethroughput was not saturated due to the size of the files (--80Kilo bytes).Figure 8 shows the results of "autobench" which uses

httperf to benchmark the machines. In Fig 8 the X-axisrepresents the number of requests per second that httperf issupposed to send each test, and Y-axis represents differentvalues that changes based on the figure's keys. In thefollowing paragraph, we will describe what each line means.Sent request per second is the number of requests that was

sent to the machine each second. Average reply rate persecond is the average number of replies from the server persecond. Response time is the reply time of the request inmilliseconds. Throughput is the size of the bandwidth that wasused during each test in Kilo bit per second. And finally weobserved the percentage of errors for each test as packet loss.As for Fig 8 we see that the first test sent 1000 requests persecond and the replies were all intact. In the next test which

was 1500 requests per second also it was fine. Autobenchconducted 14 tests starting from 1000 requests per second andending at 7000 requests per second. Each test autobenchincreases the number of requests per second by 500. So, wesee that until 6000 requests per second everything was fineand no errors occurred. After 6000 requests per second wenotice there is a drop in the number of requests sent to theserver which is a bit steep. And so did every thing. The dropoccurred due to the CPUs saturation which was utilized 100%.Fig 9 shows the CPU usage in relation with the time. X-axisrepresents the time line of the test. We captured the reading ofthe CPU each second for the whole duration of the tests. Yaxis shows the percentage of the idle CPU. Why the idle?Because the percentage of the idle CPU show the percentageof the free portion of CPU, we can see whether the wholesystem including the web server is utilizing the CPU or not.

Clen11

( CII*IW' e-a" .M p. I . _0 I )( ea.-w 2 c..I ....... '. . . .10 ' )~ e : . - I .. .M.. , . t I IOJ )( e : - - t ~ .._ . . ,..u )

The configuration files of all the containers were the sameand a sample is shown in Fig 7.ONBOOT="yes"#UBC parameters (in form of barrier:limit)KMEMSIZE="11055923:11377049"LOCKEDPAGES="256:256"PRIWMPAGES="65536:69632"SHMPAGES="21504:21504"NUMPROC="240:240"PHYSPAGES="0:2147483647"VMGUARPAGES="33792:2147483647"OOMGUARPAGES="26112:2147483647"NUMTCPSOCK="360:360"NUMFLOCK="188:206"NUMPTY="16:16"NUMSIGINFO="256:256"TCPSNDBUF="1720320:2703360"TCPRCVBUF="1720320:2703360"OTHERSOCKBUF="1126080:2097152"DGRAMRCVBUF="262144:262144"NUMOTHERSOCK="360:360"DCACHESIZE="3409920:3624960"NUMFILE="9312:9312"AVNUMPROC="180:180"NUMIPTENT="128:128"#Disk quota parameters (in form of softlimit:hardlimit)DISKSPACE="1 048576:1153024"DISKINODES="200000:220000"QUOTATlME="O"#CPU fair sheduler parameterCPUUNITS="1000"

The following configuration of "autobench" was done tomeasure the performance. When we benchmarked the wholemachine, the total number of connections to be sent to themachine was set to 20,000. The number of requests in eachconnection was set to 10. The client time out was set to 7seconds. However, when we benchmarked the containers, foreach container, 5,000 connections were sent in each test. Andthe rest of the configuration parameters were the same asabove.OpenVZ kernel version 2.6.18, and Centos 5 templates fromOpenVZ website were used and Apache web server version2.2.3 was used as the Web server. The http queries weregenerated by using Httperfversion 0.9.0 and autobench 2.1.2.We conducted various tests by scaling the number ofcontainers from 4 to 8 and finally to 12.The configurations of Apache were left to the default

except the "KeepAlive Off' which we changed to "On" inorder to allow multiple requests in one connection.

344


5/6

100 ,--------,------r------y-----,.------.

In other words, all the containers receive the same load atthe same time. Fig 11 shows the results of autobench on oneof the containers (the other containers have almost an identicalgraphs so we present one of them here). As the graph showsautobench starts from 200 requests and ends at 2000 requests.However, the errors started to occur at about 1600 requestsper second. This means that the machine is receiving about6400 requests per second at the same time. Note that thisnumber is about the same as the performance of machine Baruin section A. Surprisingly, the CPU usage (Fig 12) was almostthe same as in section A and is saturated at the same the erroroccurrences. As for the RAM (Fig 13) it has also the sameusage as in section A.

Il102030

10

Fig 9 shows the CPU idle percentage. We observe that atthe beginning of the test the usage of the CPU was about 10%.This value keeps fluctuating each second and keeps goingdown until it reaches 0% which means that the CPU is fullyused. In this test, this point was reached at approximately at210 seconds and error occurrences were increasing from thispoint and onward (i.e. 6500 requests per second and beyond).Fig 10 shows the usage of the RAM of the whole duration

of this test.The results indicate that Baru machine can handle about

6000 requests per second and the CPU was saturated.However, in reality, usually benchmarking is done up to 80%of the CPU usage.

C. The host machine (Lama)Since we are testing the performance of OpenVZ on twodifferent machines, section A presented the results of the firstmachine (Baru) whereas this section presents the secondmachine (Lama). In this section we will not present the graphsand we will only mention the results.After running "autobench", the machine managed to handle

about 3300 requests per second with no error. This amount ofconcurrent requests has saturated the CPU of the machine.However, the throughput was not saturated due to the size ofthe files ('"'"'80 Kilo bytes).

Time (St.-cond)Fig. 9. CPU Usage ofBaru

O'--------"------'------I...--- ......... " ' - - - - - - - - 'o5000

6(JOOr-----......----...-------,----...,.-----..,_-----.

4000 U5t+06 , . . . . . ----- . , ---- . . , . ------ , . . ---__- -__1.'&+08

2000

1000

0 -....-_...---......-_--+--.....-----11__ -....-----


6/6

TABLE 1Perfonnance of Baru and its containers

Machine Number Requests Total requestsof servers per second per second

Bam 1 -6000 -60004 containers 4 -1600 -64008 containers 8 -750 -600012 containers 12 -500 -6000

As for Lama we mentioned before that it handled about3300 requests per second. So, when we ran 4 containers on itwe found that 700 requests per seconds were handled by eachcontainer which means 700 * 4 = 3200 requests per secondwhich is about the same number of requests that were handledby Lama. Note that we did not manage to run 12 containers onLama due to RAM limitations.

REFERENCES[1] Information Technology @ Johns Hopkins-Glossary ofTechnical Terms and Acronyms. Available

http://it.jhu.edu/glossary/pgrs.html[2] Concise Oxford English Dictionary (Eleventh Edition).Computer Software.[3] WolfC., Halter E. (2005). Virtualization from the desktop to theenterprise. Apress, 2005. p.l.[4] Shields G., The Shortcut Guide to Selecting the RightVirtualization Solution, electronic book. Available:

http://nexus.realtimepublishers.com/SGSRVS.htm[5] OS Virtualization. Available:http://www.parallels.com/en/products/virtuozzo/os/[6] OpenVZWiki. Available:

http://wiki.openvz.org/Category:Definitions[7] Padala P., Xiaoyun Zhu, Zhikui Wang, Singhal S., Shin K.

Performance Evaluation of Virtualization Technologies forServer Consolidation. HP Laboratories Palo Alto, April 11,2007. Available: http://www.hpl.hp.com/techreports/2007/HPL2007-59.pdf

[8] WebVZ. Available: http://webvz.sourceforge.net[9] The Linux HTTP Benchmarking HOWTO. Available:

http://www.xenoclast.org/doc/benchmark/HTTP-benchmarkingHOWTO/

TABLE 2Performance of Lama and its containers

VI.CONCLUSIONWe found that when using OpenVZ, almost no overhead onthe computer resources and it shares the resources of thecomputer fairly amongst the running containers. Moreover,we found that both machines (Baru and Lama) can handleabout 6000 and 3300 requests per second respectively. And sodo containers of OpenVZ when we run all of themconcurrently on the both Baru and Lama. This means thatOpenVZ perfonnance does not change with the change of thearchitecture (32 bits and 64 bits). We can safely say that if wewant to run 15 Web servers using OpenVZ containers, eachVE can handle 400 requests per second at the same time (thisis on Baru). The WebVZ is a very useful tool foradministrating OpenVZ containers both locally and remotely.

Machine Number of Requests per Total requestsservers second per second

Lama 1 -3300 -33004 containers 4 -700 -32008 containers 8 -400 -3200

1400

2000

120000000

throughput KB/spercentage of errors

Fig. 11. Perfonnance of one of the containers

Time (second)Fig. 12. CPU usage of 4 containers

O ~ _ + - -"-_-+--+- --+ - - - l I ~ ~ I - - - - - "200 ..00 800 800 1000 1200 1 80a.U~ so40

302010

Time (second)Fig. 13. RAM usage of the 4 containers

346

openvz performance evaluation

Documents