agile, dynamic provisioning of multitier internet applications bhuvan urgaonkar, prashant shenoy,...

1

AGILE, DYNAMIC PROVISIONING OF MULTITIERINTERNET APPLICATIONSBhuvan Urgaonkar, Prashant Shenoy, Abhishek Chandray, and Pawan Goyal

ACM Transactions on Autonomous Adaptive Systems, 3(1), 2008

2

Agenda

Introduction System Overview Provisioning Algorithm

How much When

Server Switching Evaluation Conclusion Comments

3

Introduction (1/4)

Internet applications employ a multi-tier architecture, with each tier providing a certain functionality Such applications tend to see dynamically

varying workloads that contain long-term variations such as time-of-day effects short-term fluctuations due to flash crowds

Predicting the peak workload of an Internet application and capacity provisioning based on these worst case estimates is notoriously difficult

4

Introduction (2/4)

Since many single-tier provisioning mechanisms have already been proposed a straightforward extension is to employ such

an approach at each tier of the application But….

Use single-tier provisioning mechanisms Bottleneck Shifting

Model all tiers as a black box and allocate servers whenever the observed response time exceed a threshold

Hard to determine how much servers and where the server should be allocated

5

Introduction (3/4)

6

Introduction (4/4)

Research Contributions Predictive and Reactive Provisioning Analytical modeling and incorporating tails

of workload distributions Virtual Machine based provisioning Handling session-based workloads

7

System Overview (1/6) --Multi-tier Internet Application

A tier may be clustered or not the front-end tier can be a clustered Apache server that

runs on multiple machines the backend tier employs a database with shared-

nothing architecture, it cannot be replicated on-demand

Each clustered tier is also assumed to employ a load balancing element responsible for distributing requests to servers If a session is stateful, successive requests will need to

be serviced by the same server at each tier the load balancing element will need account for this server

state when redirecting requests

8


Every application also runs a special component called a sentry polices incoming sessions to an application’s

server pool unlike systems that use per-tier admission control

makes a one-time admission decision when a session arrives

avoids resource wastage resulting from partially serviced requests that may be dropped at later tiers

Once a session has been admitted, none of its requests can be dropped at any intermediate tier

9


10

System Overview (4/6) --Hosting Platform Architecture

The hosting platform is a data center that consists of a cluster of commodity servers interconnected by gigabit Ethernet

Servers Hosting Application Components each application runs on a subset of the servers

and a server is allocated to at most one application at any given time

The component of an application that runs on a server is referred to as a capsule If the capsule is replicable – the server is called Elf If the capsule is non-replicable – the server is called

Ent

11


Nucleus a software component that performs online

measurements of the capsule workload, performance and resource usage

these statistics are periodically conveyed to the control plane

Control Plane responsible for dynamic provisioning of

servers to individual applications

12


13

Provisioning Algorithm --How much (1/3)

Model each server as a G/G/1 queuing model

Request arrival rate to tier i λi : the request arrival rate to tier i di : the mean response time for tier i si : the average service time for a request : the variance of inter-arrival time : the variance of service time

14

=>

=>

=>

=>

=>

Wq : the waiting time in queue

X : the (random) service time

15


Observe that di is known the per-tier service time si the variance of inter-arrival and service

times and can be monitored online in the system. By substituting these values, a lower bound

on request rate λi that can serviced by a single server can be obtained.

16


ηi : The number of servers needed at tier i (output) Z : average session think-time : the rate that a session issues requests λ : the session arrival rate : the average session duration βi: the requests that triggered by a single

incoming request at tier i

17

Provisioning Algorithm –When – Predictive Provisioning for Long Term(1/3) Predictive provisioning is motivated by long-

term variations such as time-of-day or seasonal effects exhibited by Internet workloads the workload seen by an Internet application

typically peaks around noon every day and is minimum in the middle of the night

The predictor uses past observations of the workload to predict peak demand that will be seen over a period of T hours For simplicity of exposition, assume that T = 1 hour

18

Provisioning Algorithm –When – Predictive Provisioning for Long Term(2/3)

19

Provisioning Algorithm –When – Predictive Provisioning for Long Term(3/3) λpred(t): the predicted arrival rate during a

particular hour denoted by t λobs(t): the actual arrival rate seen during

this hour λobs(t) - λpred(t): the prediction error h : the mean prediction error over the

past h hours

20

Provisioning Algorithm –When – Reactive Provisioning for Short Term(1/3) sudden load spikes or flash crowds are

inherently unpredictable phenomena Reactive provisioning is used to swiftly

react to such unforeseen events operates on short time scales—on the order of

minutes—checking for workload anomalies

21

Provisioning Algorithm –When – Reactive Provisioning for Short Term(2/3) Reactive provisioning is invoked once every

few minutes It can also be invoked on-demand by the

application sentry

Two approaches Recompute a new allocation of server for the

various tiers Increase the allocation of all tiers that are at or

near saturation by a constant amount

22

Provisioning Algorithm –When – Reactive Provisioning for Short Term(3/3) If the free pool is empty or has

insufficient servers need to be borrowed from other

underloaded applications running on the hosting platform

An application is said to be underloaded if its observed workload is significantly lower than its provisioned capacity

23

Server Switching (1/2)

assume that each Elf server runs multiple virtual machines and capsules of different applications within it Only one capsule and its virtual machine is

active at any time Other virtual machines are dormant—they

are allocated minimal server resources If the server belongs to the free pool, all of

its resident VMs are dormant

24

Server Switching (2/2)

switching an Elf server from one application to another implies deactivating a VM by reducing its resource allocation to ε ε is a small value such that the VM consumes

negligible resources But, if the server retains state of existing

sessions Fixed rate ramp down

Some long-lived residual session will be forced to terminate

Measurement-based ramp down The server switching time is long

25

Evaluation –Environment (1/3)

a prototype data center a cluster of 40 Pentium servers

An application capsule (2.8GHz, 512MB RAM) Load balancer Control plane (dual-processor 450MHz, 1GB RAM) Sentry (dual-processor 1GHz, 1GB RAM) Workload Generator

connected via a 1Gbps ethernet switch running Linux 2.4.20

Three tiers Apache Web server (2.0.48) Tomcat servlets container (4.1.29) Non-replicable Mysql database server (4.0.18)

26


Virtual Machine Monitor Xen 1.2 …..

Nucleus online measurements of resource usages and request performance real-time processing of logs provided by the application software

components offline measurements to determine various quantities needed by the

control plane Sentry and Load balancer

Use Kernel TCP Virtual Server (ktcpvs) version 0.0.14 for sentry and Apache layer

mod_jk: an Apache module that implement a varient of round robin request distribution for Tomcat layer

Control Plane A daemon running in a dedicated machine Implements the predictive and reactive provisioning

27


two open-source multi-tier applications Rubis

An eBay like auction site Three type of user sessions : selling, browsing, bidding 9 tables in the database 26 interactions that can be accessed from the clients’ Web

browsers Rubbos

A bulletin-board application Two different levels of access : regular user and moderator provides 24 Web interactions

SLA: the 95th percentile of the response time is no greater than 2 seconds

28

Evaluation -- independent per-tier provisioning(1/3)

Use Rubbos application Workload increase every 10 minutes

29


employ dynamic provisioning only at the most compute-intensive tier of the application, since it is the most common bottleneck the Tomcat tier

The capacity of a Tomcat server was determined to be 40 simultaneous sessions, while Apache was configured with a connection limit of 256 sessions

30


Use multi-tier provisioning technique

31

Evaluation -- the black box approach(1/2)

Use Rubis assume that two Tomcat servers and one

Apache server are added to the application every time a capacity increase is signaled

But database is not replicable

33

Evaluation -- the black box approach(2/2)

Use multi-tier provisioning technique

34

Evaluation -- Predictive and Reactive Provisioning(1/4)

Use Rubis Workload

1998 Soccer World Cup Site 8 day period

Compressing the original 24-hr long trace to 1hr Picking every 24th minutes and discarding the

rest Day 6(typical day) Day 7(moderate overload) Day 8(extreme overload)

35


Day 6 Only predictive provisioning

36


Day 7 Predicted with/without recent trand Prediction failed during interval 2 Reactive must trigger after the SLA is

violated

37


Day 8 Prediction is failed The unpredictable workload consumes

all the server Using policing to drop sessions

38

Evaluation –Switching of server resources

Scenario 1: New server taken from free pool; the application must be start

Scenario 2: as 1, but application is already running

Scenario 3: taken from another application, waiting for all residual sessions to finish

Scenario 4: as 3, let two VMs share the CPU equally until the session finish

Scenario 5: as 3, using “fixed rate ramp down”

39

Conclusion

a flexible queuing model to determine how much resources to allocate to each tier of the application

a combination of predictive and reactive methods that determine when to provision these resources, both at large and small time scales

40

Comments(1/2)

A different thinking about resource provisioning Which service should be allocated resource ?

SLA must be violated first How many resources and when to allocate to

services ? The accuracy of prediction is key point

Can the two ways combine together? The evaluation result in the paper seems not so

good The prediction interval and reactive interval is too

long (15 min and few minutes) But frequently checking will make more loading

41

Comments(2/2)

Unpredictable workload is really unpredictable ? Cooperate with news But its not automatic

Queuing theory…………

42

Thanks The End

agile, dynamic provisioning of multitier internet applications bhuvan urgaonkar, prashant shenoy,...

Documents