energy-aware autonomic resource allocation in multitier virtualized environment … ·...

Energy-Aware Autonomic Resource Allocation in Multitier Virtualized Environment (in very Large Service Centers)

Danilo Ardagna

Dipartimento di Elettronica e Informazione, Politecnico di Milano [email protected]

ICEP - London 1st June 2012

This talk is based on….

1.  D. Ardagna, B. Panicucci, M. Trubian, L. Zhang. Energy-aware autonomic resource allocation in multi-tier virtualized environments. IEEE Transactions on Services Computing, 5(1), 2-19, 2012.

2.  B. Addis, D. Ardagna, B. Panicucci, L. Zhang. Autonomic management of cloud service centers with availability guarantees. Cloud 2010 Proceedings, 220-227, 2010.

3.  M. S. Squillante, T. Nowicki, C. Wah Wu. Fundamentals of dynamic decentralized optimization in autonomic computing systems. Self-star Properties in Complex Information Systems, 2005.

2

Motivations

•  Modern cloud infrastructures live in an open world, characterized by continuous changes in the environment

•  Continuous changes occur autonomously and unpredictably, and they are out of control of the cloud provider

•  Cloud providers main challenges •  Costs (energy!) reduction •  Performance levels improvement •  Availability and dependability enhancement

3

Microsoft Service Centers

Google Service Centers

Microsoft: •  Windows LiveID: >1Mld aut/day •  MSN: 550M visitors per month •  Hotmail: 400M users •  Messenger: 320M users •  Bing: >2Bill query/month Overall:

•  210K m2

•  >200,000 servers

Google: •  Number of searches July 2008: 48.7

billions •  Peak number searches: 65 mln/h

Overall:

•  36 Service Centers

•  >500,000 servers

4

Increasing complexity and size of Service Centers




•  210K m2

•  >200,000 servers



Overall:


•  >500,000 servers

Quincy: Ø  Area: 43,600 m2 (10 football fields) Ø  Consumption/Area: 54W/foot Ø  Chilled water pipes: 4.8 km

Largest Google Service Center Ø  Largest service center world wide Ø  45 containers, > 45,000 servers

5





•  210K m2

•  >200,000 servers



Overall:


•  >500,000 servers

Quincy: Ø  Area: 43,600 m2 (10 football fields) Ø  Consumption/Area: 54W/foot Ø  Chilled water pipes: 4.8 km

Largest Google Service Center Ø  Largest service center world wide Ø  45 containers, > 45,000 servers

6


Internet

Web Server2

Web Server1

Web Server3 Physical servers

Autonomic Self-Managment

7

Autonomic Self-Managment

Resource Allocation:

•  Server ON/OFF

•  Applications placement

•  Frequency Scaling (DVFS)

•  Load balancing

•  Capacity allocation

8

Resource Allocation System

Our work

•  Perspective of a Platform as a Service provider which has to manage the applications of its end customers:

•  Providing response time and availability guarantees

•  Minimizing its energy costs

•  We propose a distributed hierarchical solution based on mixed integer non-linear optimization for the management of resources of very large cloud service centres, acting at multiple time-scales

9

Resource Allocation System: Hierarchical Structure

10

•  Central Manager (CM)

•  Partitions service applications and resources into clusters on the basis of workload forecasts for the next day (24h)

•  Application Manager (AM)

•  Determines the optimal resource allocation for a single cluster

•  Server ON/OFF

•  Application placement

•  Frequency Scaling

•  Load Balancing

•  Capacity allocation

•  Actuation mechanisms are raked according to their overhead

Fine grained multi-time scale resource

allocation

...

...

...

Free server pool

Central Manager

ApplicationManager

... ......

Virtual Machine Monitor

OperatingSystem

App

OperatingSystem

App

VM VM

- Load balancing- Capacity allocation - Frequency scaling

- Servers switching - Application placement

- Classes partitioning - Servers partitioning

Application Manager

T1

T2

T3

Cluster Cluster

•  Open queueing network model: heterogeneous service centers and a delay center

•  VMM modelling: Generalized Processor Sharing (GPS) scheduling

•  VMs running the same tiers are replicated on multiple physical servers and work in load-sharing

Hexogenous arrival rate

Session modelling

Service centers model physical servers which support VMs execution

A class k request becomes a request k’ with probability πk,k’ or terminates

Service Center Performance and Availability Models

π

11

Evaluation of request response time

•  GPS bounding technique to approximate the queueing system (Towsley et al. Statistical Analysis of the GPS Scheduling Discipline. IEEE J. Selected Areas in Comm. 1995)

•  The server i capacity devoted to class k requests for tier j at time t is:

•  The average response time for the execution of the process at tier j of class k requests at server i can be evaluated as:

•  The average response time for class k requests is:

12

Interrelationship among the time scales T1, T2, and T3

0"

50"

100"

150"

200"

250"

300"

350"

T1

T2

T3

Time [min]

Req

uest

arr

ival

[req

/sec

]

⇤tk(T2)

⇤tk(T3)

13

Revenues are a function of average response times

Average response time soft-constraint

SLA – Service Level Agreement

⌥k

RkRk

↵k

vk

14

Servers power model

Dispatcher

HTTP servers tier

!k

Think time

"k,k’

DBMS servers tier

Application servers tier

-ak

vk

Rk Rk

Vk

bi,h

(a) (b) (c)Ui

pi,h+ci

15

R. Raghavendra, P. Ranganathan, V. Talwar, Z. Wang, X. Zhu. No ‘Power’ Struggles: Coordinated Multi-Level Power Management for the Data Center. SIGARCH Computer Architecture News, 36(1), 48-59, 2008.

System parameters

K Set of request classes

Nk Set of applications involved in the execution of class k request

I Set of servers available at the cloud service center

C Set of clusters

Hi Set of operating frequencies for server iCi,h Capacity of server i when it is working at frequency h 2 Hi

Ui Server i CPU utilization

RAM i RAM available at server iRAMk,j RAM required for the execution of the application tier j for class kRk Class k average response time

Ri,k,j Average response time of server i while executing the application at tier j for class k requests

T1 Long-term control time horizon

T2 Medium-term control time horizon

T3 Short-term control time horizon

nt2 Number of time bands T2 within the period T1

nt3 Number of time bands T3 within the period T2

16

SLA and Costs Parameters

Ak Class k availability threshold

Rk Class k revenue/penalty region threshold

�k Request class k maximum utility function value

↵k Request class k utility function slope

ai Server i availabilityci Time unit cost for server i when it is in low power sleep state

ci+pi,h Time unit static cost for server i when it is working at frequency h 2 Hi

bi,h Time unit dynamic cost for server i when it is working at frequency h 2 Hi

csi Cost for switching server i from low power sleep to active state

cm Cost associated with moving VMs to a di↵erent server

17

Workload Prediction and Monitoring Parameters

⇤tk(T2) Overall incoming workload for class k request in the time band t at time scale T2

⇤tk(T3) Overall incoming workload for class k request in the time band t at time scale T3

µk,j Maximum service rate of a capacity 1 server for executing application at tier j for class krequests

18

Optimization problems formulation

•  Maximize profits:

•  Revenues - Costs

•  The CM determines one day ahead:

•  The class partitions

•  Server assignment to each cluster c and for every

time band

•  We will consider first the AM solution acting at time scale T2 (medium-term problem)

19

{Kc}Itc

t 2 [1..nt2 ]

Decision variables – T2

20

x

ti 1 if server i is running at the time band t, 0 otherwise

y

ti,h 1 if server i is working at the time band t with frequency h, 0 otherwise

z

ti,k,j 1 if the application tier j for class k is assigned to server i at the time band t, 0 otherwise

�

ti,k,j Rate of execution for class k at tier j on server i at the time band t

�

ti,k,j Fraction of capacity devoted for executing VM at tier j for class k at server i at the time band

t

w

ti,k 1 if at least an application tier j for class k is assigned to server i at the time band t, 0 otherwise

Class k average response time

Utility function evaluation

Static servers cost

Servers switiching cost

VMs migrations cost

Optimization Problem – Objective function

2

66664

X

k2Kc

0

BBBB@�↵k

X

i2Itc,j2Nk

�ti,k,j

Ph2Hi

Ci,h yti,h

!µk,j �t

i,k,j � �ti,k,j

1

CCCCA+X

k2Kc

�k⇤tk(T2)�

X

i2Itc

ci+

�X

i2Itc,h2Hi

0

B@pi,hyti,h + bi,h y

ti,h

Pj2Nk,k2Kc

�

ti,k,j

Ph2Hi

Ci,h µk,j yti,h

1

CA

3

75 T2 �X

i2Itc

csi max(0, x

ti � x

t�1i ) +

�X

i2Itc,k2Kc,j2Nk

cmmax(0, z

ti,k,j � z

t�1i,k,j)

Dynamic servers cost

21

X

i2Itc

�

ti,k,j = ⇤t

k(T2) = ⇤tk,j(T2) 8k 2 Kc, j 2 Nk,

X

k2Kc,j2Nk

�

ti,k,j 1 8i 2 It

c,

X

h2Hi

y

ti,h = x

ti 8i 2 It

c,

z

ti,k,j x

ti 8i 2 It

c, k 2 Kc, j 2 Nk,

�

ti,k,j ⇤k(T2)

tz

ti,k,j 8i 2 It

c, k 2 Kc, j 2 Nk,

All incoming requests have to be served

At most 100% server capacity can be used

Application can be allocated only on servers ON Allocate workload were the corresponding application is running

If a server is running only one operating frequency can be selected

Optimization Problem – Constraints

22

�

ti,k,j

X

h2Hi

Ci,hyti,h

!µk,j�

ti,k,j � ✏ 8i 2 It

c, k 2 Kc, j 2 Nk

X

j2Nk

RAMk,jzti,k,j RAM i 8i 2 It

c, k 2 Kc,

Y

j2Nk

0

@1�Y

i2Itc

(1� ai)wt

i,k

1

A � Ak 8k 2 Kc

NkX

j=1

z

ti,k,j Nkw

ti,k 8i 2 It

c, k 2 Kc

�

ti,k,j ,�

ti,k,j � 0, x

ti, y

ti,h, z

ti,k,j , w

ti,k 2 {0, 1} 8i 2 It

c, k 2 Kc, j 2 Nk.

Queue equilibrium

Optimization Problem – Constraints

Servers RAM is not saturated

Availability is guaranteed

w is raised to 1 if at least one VM of the class k is allocated on server i

23

Optimization Model Analysis

•  The optimization problem is a NP-hard non linear Mixed Integer Programming problem

•  Even if the set of server ON is fixed, the joint capacity allocation and load balancig problem is difficult since the objective function is neither concave and neither convex

24

•  Problem decomposition:

•  Initial solution: Assign VMs to physical servers (problem equivalent to a special case of a CFLP, Capacitated Facility Location Problem)

•  Fixed Point Iteration: Optimum load balancing and capacity allocation

•  A local search iteratively improves the solution

Heuristic solution

25

•  The set of servers ON and frequency of operations are held fixed

•  Iteratively identifies the optimum value of a set of variables (λ-s or φ-s), while the value of the other one (alternatively φ-s or λ-s) is held fixed

•  Based on Karush-Kuhn-Tucker (KKT)‏ conditions

Fixed Point Iteration

Initial solution •  Greedy algorithm which identifies a new solution from the solution of the

previous control time (T2) horizon:

•  Reduce the number of servers switches

•  Reduce the number of VM migrations

•  Improve system stability

•  Reduced optimization time

Heuristic solution

•  Identifies a local optimum with respect to the integer variables x, y, z, and w, while the continuous variables (λ and φ) are updated in a greedy way

•  Diversification of the Neighborhood search is obtained by the relaxation of availability constraints

Local search

26

AM optimization algorithm (T2 time scale)

27

12

Obj 4 pages

In this section we discuss the optimization technique used for solving the medium-term (P T2c ) and the

short (P T3c ) resource allocation problems. Computational results show that only small instances of these

problems can be solved to optimality with commercial non-linear optimization packages. To deal with

realistic problems of reasonable size we propose a heuristic approach (see Algorithm 1).

1 (x,y,z, �,�)=Initial Solution();2 FPI(�,�);3 STOP=false;4 while not STOP do5 while improve do6 while improve do7 Local Search Availability(x,y,z, �,�);

end8 FPI(�,�);

end9 (x,y,z,ˆ�,ˆ�)=Save Best Solution();

10 while iteration limit do11 Local Search withoutAvailability(x,y,z, �,�);

end12 Remove Violations(x,y,z, �,�);13 if not feasible solution then14 STOP=true;15 (x,y,z, �,�)=(x,y,z, ˆ�,ˆ�);

endendreturn (x,y,z, �,�);

Algorithm 1: Resource allocation optimization procedure

The basics of the overall procedure are:

(i) Finding an initial solution: For the medium-term problem we perform an heuristic method, called

Initial Solution() whose output is a set of servers in active state x, their corresponding operating frequencies

y, the assignment of tiers to servers z, the initial capacity allocation � = (�i,k,j)i2I,k2K,j2Nkand the load

balancing � = (�i,k,j)i2I,k2K,j2Nk. The details of the heuristic are in Section V-A1. For the short-term

problem the initial solution is the current one.

(ii) Improving the current solution with a local-search method, which consists on two two main phases:

First, a fixed point iteration procedure (FPI) searches for a local improvement of the capacity allocation and

load balancing decisions. The FPI changes variables �, while keeping � fixed, and vice-versa, by solving

the load balancing and capacity allocation problems, as described in Section V-A2. The current solution is

then enhanced iteratively with a local search procedure, that is a process upon which an exhaustive search

is performed in the neighborhood of the current solution in order to derive a local optima. A neighbour

of the current solution is constructed through changing the current one by using the so called “moves,” as

itemized in Section V-A3.

DRAFT

Initial solution

28

Servers are ordered a.t. non-increasing perf./cost ratio

⇤tk,j(T2)

�ti,k,j

Ui = U

�ti0,k,j

�ti0,k0,j

⇤tk0,j(T2)

⇤tk,j(T2)

Pj2Nk

1µk,j

Applications tiers are ordered a.t. non-increasing value

Fixed Point Iteration Technique

•  The Fixed Point Iteration iteratively solves the load balancing and capacity allocation problems

•  Load balancing and capacity allocation problems are convex

•  We can not guarantee that the procedure converges to a global optimum but the procedure will always converge

29

FPI - The capacity allocation sub-problem

•  The problem is separable and capacity allocation sub-problems (one for each server in each cluster) can be solved independently

•  The objective function is convex, the optimum solution of each sub-problem can be identified by the KKT conditions

minPk2K

↵kP

i2Iactive,j2Nk

�i,k,j

Ciµk,j�i,k,j��i,k,j

Pk2K,j2Nk

�i,k,j 1 8i 2 Iactive,

�i,k,j < Ciµk,j�i,k,j , 8i 2 Iactive, k 2 K, j 2 Nk,

�i,k,j � 0 8i 2 Iactive, k 2 K, j 2 Nk.

|Iactive|

30

φ-iteration based on KKT

(P-CA)

31

minPk2K

↵kP

j2Nk

�k,j

Cµk,j�k,j��k,j

X

k2K,j2Nk

�k,j 1,

�k,j >�k,j

Cµk,j8k 2 K, j 2 Nk.

φ-iteration based on KKT

•  Theorem 1: In the optimal solution of problem (P-CA) let

be the set of indexes (k,j) s.t. , and let an arbitrarily element in . Then:

where

•  The overall complexity for the φ-iteration is:

NJ�k,j > 0 (k, j)

�k,j =1

Cµk,j

✓rakµk,j�k,j

akµk,j�k,j

(Cµk,j�k,j � �k,j) + �k,j

◆8(k, j) 2 NJ � {(k, j)},

�k,j =�k,j

Cµk,j

+1�

Pk2K

Pj2Nk

�k,jCµk,j

Pk2K

Pj2Nk

sakµ

k,j�k,j

akµk,j�k,j

NJ

O

✓|Iactive|

Pk2K

|Nk|◆

32

Theorem 1 Proof

•  The Lagrangian of each sub-problem is:

33

L(�, L, Lk,j) =X

k2K

↵k

X

j2Nk

�k,j

⇥k,j�k,j � �k,j

� L

0

@1�X

k2K

X

j2Nk

�k,j

1

A+

�X

k2K

X

j2Nk

Lk,j

�⇥k,j�k,j � �k,j

�

Theorem 1 Proof

•  The KKT conditions of optimality are:

34

L

0

@1�X

k2K

X

j2Nk

�k,j

1

A = 0

�k,j � 1 0 8k 2 K, j 2 Nk

�k,j < �k,j ⇥k,j 8k 2 K, j 2 Nk

Lk,j

�⇥k,j�k,j � �k,j

�= 0 8k 2 K, j 2 Nk

L � 0 8k 2 K, j 2 Nk

Lk,j � 0 8k 2 K, j 2 Nk

@L(�, L, Lk,j)

@�k,j=

�↵k⇥k,j�k,j

(⇥k,j�k,j � �k,j)2+

+L� Lk,j⇥k,j = 0 8k 2 K, j 2 Nk,

Theorem 1 Proof

•  Since multipliers are zero

•  If L=0, it follows that

and there is no solution

•  Then L>0, and hence

•  For any given (k1,j1), (k2,j2):

35

�k,j < ⇥k,j�k,j , Lk,j

�↵kUk,j�k,j

(Uk,j�k,j��k,j)2= 0

Pk2K,j2Nk

�k,j = 1

�↵k1⇥k1,j1�k1,j1

(⇥k1,j1�k1,j1 � �k1,j1)2=

�↵k2⇥k2,j2�k2,j2

(⇥k2,j2�k2,j2 � �k2,j2)2.

Theorem 1 Proof

•  Then we get:

•  and:

36

�k,j =1

⇥k,j

s↵k⇥k,j�k,j

↵k⇥k,j�k,j

(⇥k,j�k,j � �k,j) + �k,j

!

X

k2K

X

j2Nk

1

⇥k,j

s↵k⇥k,j�k,j

↵k⇥k,j�k,j

(⇥k,j�k,j � �k,j) + �k,j

!= 1

�k,j =�k,j

⇥k,j

+

1�Pk2K

Pj2Nk

�k,j

⇥k,j

Pk2K

Pj2Nk

r↵k⇥k,j�k,j

↵k⇥k,j�k,j

FPI – The load balancing sub-problem

•  The solution of the load balancing sub-problem can be determined similarly

•  The problem is separable and load balancing sub-

problems (one for every tier of every class) can be solved

independently

•  The overall complexity for the λ-iteration is:

37

Pk2K

|Nk|

O

✓|Iactive|2

Pk2K

|Nk|◆

Local search

•  Given a feasible solution the Local Search tries to improve it:

•  Starts with an initial feasible solution: the current solution

•  A set of solutions close to the current one, the neighborhood, is generated

•  An improving solution is searched among the neighbor solutions

•  Remarks:

•  Local search does not build one alternative solution, but a set of solutions

•  The set of neighbor solutions is built by partially modifying the current solution

•  Local search moves from feasible solution to feasible solution

38

Local search

LocalSearch(InitialSolution){

x:= InitialSolution; build N(x); x:=σ(x); //the best solution of the neighborhood σ(x) is selected

while (x!=σ(x)) {

build N(x);

x:=σ(x);

}

39

•  x is the current solution

•  σ(x) is a function which selects the best solution in the neighborhood

The Local Search Algorithm

•  The FPI allows to obtain a local optimum for the joint load balancing and capacity allocation problems, when a service center configuration in terms of servers ON, operating frequencies, and allocation of application tiers to servers is given

•  The neighborhood is defined by moves based on the switching ON or OFF of servers, frequency scaling, and the re-allocation of application tiers on servers

•  We can not run the FPI for every neighborhood candidate solution, it is too time consuming

40

The Local Search Algorithm

•  To overcome this difficulty, during the neighborhood exploration we overestimate the value of each candidate solution by updating the values of a subset of the variables λ-s and φ-s

•  The neighborhood exploration is obtained by applying four different moves:

•  Turning ON a server

•  Turning OFF a server

•  Frequency scaling (increasing and decreasing)

•  Servers swapping

•  Re-allocation of tiers to servers

•  When the local-search stops at a local optimum, then we execute the fixed point iteration again (try to escape from the local optima)

41

AM optimization algorithm (T2 time scale)

42

12

Obj 4 pages

In this section we discuss the optimization technique used for solving the medium-term (P T2c ) and the

short (P T3c ) resource allocation problems. Computational results show that only small instances of these

problems can be solved to optimality with commercial non-linear optimization packages. To deal with

realistic problems of reasonable size we propose a heuristic approach (see Algorithm 1).

1 (x,y,z, �,�)=Initial Solution();2 FPI(�,�);3 STOP=false;4 while not STOP do5 while improve do6 while improve do7 Local Search Availability(x,y,z, �,�);

end8 FPI(�,�);

end9 (x,y,z,ˆ�,ˆ�)=Save Best Solution();

10 while iteration limit do11 Local Search withoutAvailability(x,y,z, �,�);

end12 Remove Violations(x,y,z, �,�);13 if not feasible solution then14 STOP=true;15 (x,y,z, �,�)=(x,y,z, ˆ�,ˆ�);

endendreturn (x,y,z, �,�);

Algorithm 1: Resource allocation optimization procedure

The basics of the overall procedure are:

(i) Finding an initial solution: For the medium-term problem we perform an heuristic method, called

Initial Solution() whose output is a set of servers in active state x, their corresponding operating frequencies

y, the assignment of tiers to servers z, the initial capacity allocation � = (�i,k,j)i2I,k2K,j2Nkand the load

balancing � = (�i,k,j)i2I,k2K,j2Nk. The details of the heuristic are in Section V-A1. For the short-term

problem the initial solution is the current one.

(ii) Improving the current solution with a local-search method, which consists on two two main phases:

First, a fixed point iteration procedure (FPI) searches for a local improvement of the capacity allocation and

load balancing decisions. The FPI changes variables �, while keeping � fixed, and vice-versa, by solving

the load balancing and capacity allocation problems, as described in Section V-A2. The current solution is

then enhanced iteratively with a local search procedure, that is a process upon which an exhaustive search

is performed in the neighborhood of the current solution in order to derive a local optima. A neighbour

of the current solution is constructed through changing the current one by using the so called “moves,” as

itemized in Section V-A3.

DRAFT

Preserving availability constraints limits the solution space exploration

The current solution stabilizes, no improving moves exist

Local search implementation

43

If a local optima is obtained, availability contraints are relaxed and the current solution is unfeasible

The availability is forced again and the move can worsen the current solution value

The fixed point iteration performes load balancing and capacity allocation

If the current solution is improved, the search continues preserving availability constraints until a local optima is identified

Local search implementation

44

Short-term problem solution (T3)

•  Based on the algorithm discussed before

•  The only move allowed in the Local Seach is the frequency scaling

•  This move does not introduce any availability violation

•  Only the Local_Search_Availability() and FPI() procedures are executed (decision variables: y, λ, φ)

45

Experimental Results

•  Instances randomly generated: the number of servers has been varied between 80 and 400. Clusters up to 200 requests classes have been considered and the number of tiers has been varied between 2 and 5

•  Service times were randomly generated and for each test case the load was increased in a way that the utilization the cluster resources varied between 0.2 and 0.8

•  Nk, αk and Rk values have been randomly generated, Rk is proportional to the number of tiers Nk and to the overall demanding time at various tiers of class k

•  Per hour CPU/software revenues between 1 and 10$

•  Cooling overhead 0.7 Time in sec

46

•  Real log traces (Politecnico di Milano Web site), 10 requests classes

•  Comparison with IBM Tivoli resource allocation policies

Scenario 1: users come from the same time zone

Scenario 2: users come from different time zones

IBM Tivoli comparison

47


•  Incoming workload is evenly shared among physical servers and the scheduling policy applied at each server is processor sharing

•  The application placement problem is reduced to a variant of the class constrained multiple-knapsack problem and a very efficient heuristic procedure is proposed (reduce the number of applications starts and stops, while equally balancing the load of physical machines)

•  The number of servers to be adopted at each physical tier is determined by a local search algorithm which moves (homogeneous) servers from one tier to another with the aim at improving bottleneck request response time

48

SCENARIO 1


49

SCENARIO 2


50

Test on a real prototype environment

•  SPECweb2005 JSP implementation and RUBBoS benchmarks

•  The system is based on three physical servers based on a Intel Nehalem dual socket quad-core CPUs with 24GB of RAM which run Xen 3.4.1-rc10 (default credit scheduler). Each core has five P-states, corresponding to frequencies of 2.4, 2.27, 2.13, 2.00, and 1.6 GHz

•  The first sever hosts the load generators, the remaining two are the System Under Tests (SUTs)target of our analyses and hosts the Web and DBMS server instances

•  The Xen hypervisor is assigned to a dedicated core on the first socket of each server, while the VMs running the Web and DBMS instances are assigned to the cores of the second socket by setting CPU affinity

•  The SPECweb2005 back-end simulators are assigned to a dedicated core on the second socket, they are always overprovisioned and their performance will be neglected

51

Test on a real prototype environment

•  The validation in the real test bed is based on two case studies:

•  Case a: Two classes single tier system based on SPECweb2005 e-commerce and banking workloads

•  We set α1=α2=1, ,

•  Case b: Three request classes: Class 1 and 2 run SPECweb2005 e-commerce and banking respectively, while class 3 executes RUBBoS. We set α1=α3=5, α2=10,

52

R1 = 1.4s R2 = 0.7s

R1 = 0.85s, R2 = 0.40s, R3 = 1.20s

Case a

5 10 15 20 2510

12

14

16

18

20

22

24

26

Time [minutes]

Thro

ughput [r

eq/s

ec]

Lambda1

Lambda2

53

Case a

5 10 15 20 250.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

Time [minutes]

Resp

onse

tim

e [s]

R1

R2

R1

R2

54

•  The two VMs share a single core running at the maximum frequency

•  The medium-term algorithm is run

Case a

5 10 15 20 250.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

Time [minutes]

Resp

onse

tim

e [s]

R1

R2

R1

R2

55

•  Another VM hosting class 1 Web server is assigned to a dedicated core of the first SUT server which run also at the maximum available frequency

Case a

5 10 15 20 250.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

Time [minutes]

Resp

onse

tim

e [s]

R1

R2

R1

R2

56

•  φ and λ are updated every 5 minutes running the short-term algorithm

Case a

5 10 15 20 250.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

Time [minutes]

Resp

onse

tim

e [s]

R1

R2

R1

R2

57

•  In the last time interval, the incoming workload is further reduced, the two cores are set to the minimum frequency

Case b

0 50 100 150 200 250 300 350 400 450 5000

100

200

300

400

500

600

700

800

Time [minutes]

Nu

mb

er

of

Co

ncu

rre

nt

Use

rs

N1

N2

N3

58

Case b

0 50 100 150 200 250 300 350 400 450 5000.2

0.4

0.6

0.8

1

1.2

1.4

1.6

Time [minutes]

Resp

onse

tim

e [s]

R1R2R3RT1RT2RT3R1!ModelR2!ModelR3!Model

59

•  Server 1 hosts: •  On a first core three VMs running the Web server for the

three classes •  On a second core a VM running class 3 database •  The two cores are set to the maximum frequency P0

•  Server 2 hosts: •  On two dedicated cores two additional instances of

the Web server for class 1 and 2 which are set to P1

•  φ and λ are updated according to the short-term algorithm

Case b

0 50 100 150 200 250 300 350 400 450 5000.2

0.4

0.6

0.8

1

1.2

1.4

1.6

Time [minutes]

Resp

onse

tim

e [s]


60

•  An additional Web servers is assigned to a dedicated core of server 1, for class 3

•  An additional instance of the Web server is instantiated for class 2 which is assigned to a dedicated core on server 2

Case b

0 50 100 150 200 250 300 350 400 450 5000.2

0.4

0.6

0.8

1

1.2

1.4

1.6

Time [minutes]

Resp

onse

tim

e [s]


61

•  The cores of the two servers are set to P3 and P4

•  The frequency of operation is raised again to P1 and P2

The long-term problem (T1)

•  Each AM represents a subsystem of the global Cloud Service Center, operating on a subset of applications and resources

•  Each AM has to assign the available physical servers to the given subset of applications in an optimal way at time scales T2 and T3

•  The CM determines the different partitions of applications and servers to be assigned to each AM:

•  At the beginning of each day, the whole application classes are partitioned using a heuristic procedure, considering the workload forecast for the entire day

•  The assignment of computing capacity to the application partition is done on hourly basis, considering the workload of the current period

Ø  Mark’s Theorem

62

The long-term problem (T1)

•  Objective function:

•  The aggregation function (the sum) is order preserving, the decentralized hierarchical optimal solution is as good as the centralized optimal solution

63

nT2Pt=1

Pc2C

f tc

•  Include in the same partition applications whose peaks and valleys overlap

Hyerarchical Resource Allocation: Application partitioning

•  Evaluate area of the workload predictions

64

Area ratio criteria example

65

0

0.1

0.2

0.3

0.4

0.5

0.6

1 3 5 7 9 11 13 15 17 19 21 23

Class 1

0

0.1

0.2

0.3

0.4

0.5

0.6

1 3 5 7 9 11 13 15 17 19 21 23

Class 2

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8

1 3 5 7 9 11 13 15 17 19 21 23

Class 3

0

0.1

0.2

0.3

0.4

0.5

0.6

1 3 5 7 9 11 13 15 17 19 21 23

Class 1-2

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1 3 5 7 9 11 13 15 17 19 21 23

Class 1-3

AR=1 AR=0.5

Hyerarchical Resource Allocation: Initial solutions

Ø  Initial assignment: greedy algorithm

Central Manager

Application Manager A

Application Manager B

Application Manager C

Free Server Pool

VM1 VM2

VM3 VM4

F.O. C: 0.3 F.O. B: 0.2 F.O. A: 0.1

VM1 VM2

VM4 VM3

66


Ø  Local Search

Ø  Implements 3 moves

Increase Cluster Capacity : New servers are added to a cluster

Central Manager




VM1 VM2

VM3 VM4

Delta F.O. C: 0.1 Delta F.O. B: 0.2 Delta F.O. A: 0.1

67


Ø  Local Search


Decrease Cluster Capacity : Servers are moved back to the free server pool

Central Manager




VM1 VM2

VM3 VM4

Delta F.O. C: 0 Delta F.O. B: -0.2 Delta F.O. A: -0.1

68

Variazione F.O. A: +0.1 Variazione F.O. C: +0.2 Delta F.O. C: 0


Ø  Local Search


Ø  Capacity migration: servers are moved between two different clusters Ø  Decrease Capacity: the best servers are removed from a cluster Ø  Increase Capacity: the worst servers are added to a cluster

Central Manager




VM1 VM2

VM3 VM4

Variazione F.O. B: -0.2 Delta F.O. A: -0.1 Delta F.O. B: +0.3

69

Experimental Analysis

70

Aims: Analyze the behavior of the hierarchical algorithm working at different timescales with incoming workloads characterized by increasing variability Synthetic traffic generation Test campaign on Large Scale Service Centers

•  2400/4800/7200 servers and 240/480/720 service applications respectively •  5/10/15 minutes timescale •  Spread out parameter equals to [0.1, 0.2, 0.3]

Experimental analysis •  Cluster dimension evaluation •  Analysis of multiple timescale optimization

•  Injection of reference workload (e.g., workload varying at 5 minutes time scale) into Service Center allocated at 1 hour or 15/10/5 minutes

h

h

h

h

Experimental results: Centralized solver comparison

80 classes, 800 servers 100 classes, 1000 servers

71

Hierarchical-10 Hierarchical-10






Centralized Centralized

Centralized Centralized

Centralized

Ø  Physical machine: Ubuntu 10.4, 2 CPU Intel Nehalem 2.4 GHz quad-core, 24 GB RAM

Ø  Percentage error and speedup (average over 10 runs)

Ø  Speedup around 8

Ø  Feasible solutions can be identified under critical workload conditions

Ø  Overhead of the hierarchical algorithm can be neglected for instances larger than 400 servers

Experimental results: Centralized solver comparison

72

(|K|,|I|) % Gap centralized Speedup

(20, 20) 11.00 0.24

(40, 400) 11.00 0.49

(80, 800) 10.81 4.45

(100, 1000) 9.98 6.97

(120, 1200) 7.75 7.61

Experimental Analysis – Low variabiility

73

Hour 5 min 10 min 15 min

Revenues 0.34 0.28 0.31 0.38

Server Switch Mean 150 33 61 85

Mean per time unit(15 min)

- 99 91.5 85

VM Migration Mean 950 160 330 480


- 480 495 480

Response Time [0.46 5.65] [0.21 15,6] [0.20 8.2] [0.21 8.2]

Experimental Analysis – High variabiility

74

Hour 5 min 10 min 15 min

Revenues 0.32 0.25 0.32 0.23

Server Switch Mean 150 76 119 150


- 228 178 150

VM Migration Mean 950 500 760 930


- 1500 1140 930

Response Time [0.63 6.17] [0.30 14,6] [0.27 8.8] [0.31 9.4]

Conclusions

•  This work proposes resource allocation policies for the management of multi-tier virtualized cloud systems maximizing the cloud provider profits

•  A hierarchical heuristic solution based on local-search which provides also availability guarantees for the running applications has been developed

•  Current work is considering the resource allocation problem: •  At very fine-grained time scales (sec) by adopting control theory models

•  At larger time scales allocating resources of multiple service centers exploiting also the availability of green energy sources

75

Additional references

•  G. Pacifici, M. Spreitzer, A. Tantawi, A. Youssef. Performance management for cluster-based web services. IEEE Journal on Selected Areas in Communications 2005.

•  B. Urgaonkar, G. Pacifici, P. Shenoy, M. Spreitzer, A. Tantawi. An analytical model for multi-tier internet services and its applications. SIGMETRICS 2005.

•  B. Urgaonkar, G. Pacifici, P. Shenoy, M. Spreitzer, A. Tantawi. Analytical modeling of multi-tier internet applications. TWEB 2007.

•  M. Tanelli, D. Ardagna, M. Lovera. Identification of LPV state space models for Autonomic Web service systems. IEEE Transactions on Control Systems Technology. 19(1), 93-103, 2011.

76

Thanks !

Any questions ?

77

energy-aware autonomic resource allocation in multitier virtualized environment … ·...

Documents