by tarek f. abdelzaher, john a. stankovic, chenyang lu ...lu/papers/control03.pdf · time-related...

Software systems are becoming larger and morecomplex. At the same time, they are being de-ployed in applications where performance guar-antees are required. Traditional approaches toproviding these performance guarantees arenot effective for a

large class of software sys-tems. Recently, control theoryhas been identified as a prom-ising theoretical foundationfor performance control incomplex software applications, such as real-time scheduling,Web servers, multimedia control, storage managers, powercontrol in CPUs, and routing in computer networks. This arti-cle describes advances in the application of control theory tosoftware systems. We demonstrate the formulation of soft-ware performance assurance problems as those of feedbackcontrol. We describe modeling the software system and pro-vide an example of performance control in contemporary

Web servers. We embody the control-theoretical methodol-ogy for software quality-of-service (QoS) provisioning into amiddleware called ControlWare. This middleware providesa generic interface between the computing and control sub-systems of a software application and automates many

parts of the feedback controldesign and implementationfor software systems. Middle-ware solutions such asControlWare are needed to re-solve the system challenges

and provide analytic foundations to enable efficient soft-ware performance control in next-generation perfor-mance-assured computing systems.

Software Performance ControlAlthough engineered physical systems carefully addressquality assurance, software system design evolved in a

74 IEEE Control Systems Magazine June 20030272-1708/03/$17.00©2003IEEE

By Tarek F. Abdelzaher,John A. Stankovic, Chenyang Lu,

Ronghua Zhang, and Ying Lu

Abdelzaher ([email protected]), Stankovic, Zhang, and Y. Lu are with the Department of Computer Science, University of Virginia,Charlottesville, VA 22904, U.S.A. C. Lu is with Washington University in St. Louis, MO 63130, U.S.A.

©ARTVILLE

more ad hoc fashion, with less rigorous guarantees on per-formance and quality. Most software engineering researchis concerned with tools and paradigms that facilitate the de-velopment of functionally correct software. Functional cor-rectness was implicitly assumed to be an adequate softwarequality metric.

There are several notable exceptions to the assumptionthat functional correctness is adequate. For example, inmost embedded control systems, an important factor affect-ing quality of software is the timeliness of software re-sponse. Here, a delayed, but functionally correct, reactionto physical events in the environment can be as devastatingas a physical equipment malfunction. This observationgives rise to research that attempts to design software withpredictable nonfunctional properties such as timeliness, se-curity, availability, and performance. These properties areoften referred to as software QoS attributes.

Traditional performance analysis of embedded control ap-plications relies on worst-case estimates of load and resourceavailability. A system whose performance is guaranteed inworst-case conditions will not violate its performance spec-ifications under more favorable circumstances. The notionof QoS guarantees, however, is needed today in a muchlarger class of applications, which, unlike closed embeddedsystems, operate in open, unpredictable environments. Theworst-case scenario in such environments is not known apriori or is very pessimistic, rendering traditionalworst-case analysis impractical.

The need for QoS guarantees in open systems is fueled inpart by the emergence of modern global communicationnetworks such as the Internet and by the increasing prolifer-ation of globally accessible digital services such as onlinebanking, trading, and distance learning. Such services rep-resent points of massive aggregation, which may suffer fromunpredictable loads, potential bottlenecks, and securitybreaches such as denial of service attacks. Failure to meetacceptable performance specifications may result in loss ofcustomers, financial damage, or liability violations. Existingapproaches for designing performance-guaranteed comput-ing systems that rely on a priori workload and resourceknowledge are no longer applicable. Instead, the predomi-nant practice for providing QoS assurances today isoverdesign. This practice results in costly systems with un-certain assurances regarding performance. Putting perfor-mance guarantees on a solid analytic foundation in suchsystems is an important challenge.

In this article, we show how classical feedback control of-fers a solution to the problem of achieving performanceguarantees for this new category of applications and dis-cuss the challenges involved in realizing this approach. Themain challenges in implementing a feedback-based QoS con-trol solution in computing systems are i) analyzing the soft-ware architecture to model it as a feedback control system,ii) mapping the particular QoS control problem into a sys-tem of feedback loops, iii) choosing a proper actuator thatcan affect server resource allocation and a monitor that can

measure current performance reliably, and iv) designing acontroller for the server. Solving these challenges permitsthe mathematical basis of control theory to support the per-formance guarantees of software systems. We also demon-strate that a software system can be approximated by alinearized model and describe the needed software actua-tors and sensors. Importantly, we create a taxonomy of themost significant QoS assurance problems in the software lit-erature and describe how they can be translated into a feed-back control formulation.

We also present some experimental results for aWeb-based application that show that the control-theoreticapproach is feasible for software systems. Although our ap-proach can be applied to other services as well (and we pro-vide brief explanations of those), we have chosen to focus oncontrolling the performance of Web applications due to theincreasing importance of the Web, spurred by the phenome-nal growth of the Internet. The Web is the largest and mostvisible client-server application in existence today. It is an ex-cellent vehicle for investigating fundamental problems of dis-tributed client-server computing such as that of overloadprotection and providing performance guarantees.

In the rest of this article, we describe the internal archi-tecture of a typical Web server, present its dynamic model,and elaborate on the equivalents of sensors and actuatorsin this computing system. We then map the most importantsoftware performance assurance problems into those offeedback control and piece these elements together in acase study on Web-server performance control. We also de-scribe briefly other examples of the use of control theory forQoS guarantees. Finally, we introduce a middleware servicethat generalizes the control-theoretic approach, implement-ing a new paradigm for service performance guarantees insoftware systems. The article concludes with a brief discus-sion of important results and remaining challenges.

Inside a QoS-Aware ServiceA successful architecture for controlling the performance ofa computing system begins with an understanding of the dy-namics that affect performance and the mechanisms avail-able to manipulate those dynamics. Here, we are concernedspecifically with performance attributes that involve a no-tion of time, since they lend themselves more naturally to afeedback-control framework. Generally, the most commonof these performance attributes are classified into two cate-gories depending on whether they are directly proportionalor inversely proportional to time. The first category in-cludes performance metrics such as queuing delays, execu-tion latencies, and service response times. The secondcategory includes metrics such as connection bandwidth,service throughput, and packet rate. We call the two catego-ries delay metrics and rate metrics, respectively. There arealso derived metrics defined as ratios between other met-rics, for example, the relative delay of two traffic classes orthe hit ratio (i.e., the fraction of hit rate to the total requestrate) of a cache.

June 2003 IEEE Control Systems Magazine 75

Time-related performance attributes can generally becontrolled by adjusting resource allocation. Queuing theoryhas often been used to predict performance, given a particu-lar resource allocation, or to determine how resourcesshould be allocated to yield a particular performance level[1]. In general, if the service is modeled as a queuing system,allocating more resources to the queue’s server will reducethe mean service time, decrease the mean queuing delays,and increase the average service rate. Unfortunately, queu-ing theory generally requires assumptions about the inputtraffic arrival pattern that are not always accurate, leadingto potentially poor predictions regarding performance. Forexample, a significant body of queuing literature applies to

Poisson arrivals, whereas many common arrival patterns,such as those of Web requests, are known to follow aheavy-tailed distribution [2], [3]. Even when all assump-tions are accurate, queuing theory, by nature, offers onlypredictions on average behavior. In a QoS-aware system,stronger guarantees are often required. For example, it isnot enough that the frame rate of a streaming video be 30frames/s on average. Instead, guarantees are usually re-quired on maximum deviation and recovery time from tran-sient perturbations around the nominal rate.

Service ArchitectureTo design feedback loops for software performance control,we need to understand the basic components of a softwaresystem and how they interact. We focus on an important cat-egory of software systems where feedback control is partic-ularly important, namely, software servers. In this section,we present the typical design of a multithreaded server anddescribe how this design yields a system model suitable forfeedback control.

Consider a distributed client-server system in which asuccession of requests arrives at a server from clientsacross a communication network. The server acts as a pointof aggregation of client requests. The performance ob-served for a request at the server depends on the path therequest takes inside the server until it is served. In a typicalmultithreaded server, independently schedulable entitiescalled worker threads execute the arriving sequence of cli-ent requests. Each thread implements a loop that processesincoming requests and generates responses.

Client requests are first read from an input queue (such asthe server transfer control protocol (TCP) socket queue) by a

kernel entity that hands each request to a different workerthread. Worker threads that have requests to serve becomerunnable and are queued for the CPU. The order in whichthreads get the CPU to execute is determined by the CPUscheduling policy. This policy maintains a priority queuecalled the ready queue. The thread at the top of this queue ex-ecutes for a particular time quantum or until it is blocked. Re-quest processing by a worker thread typically entails accessto one or more auxiliary server resources, the most notablebeing disk input/output (I/O). For example, in a Web server,disk I/O is usually needed to read the requested Web pagefrom disk. Access to auxiliary resources blocks the callingthread, at which time it is queued for I/O until the awaited re-

source becomes available. Each re-source usually has a queue thatdetermines the order in which ac-cesses are served. The resource ismade available to the thread at thetop of the queue, at which time thecorresponding thread becomesrunnable again and reenters the CPUready queue. When request process-ing is done, the worker thread sends a

response back to the client. Sending the response entailsenqueuing data into the outgoing packet queue for transmis-sion on the network.

Figure 1(a) illustrates the aforementioned main com-ponents of a software server, including the input client re-quest queue, the CPU ready queue, the resource I/Oqueue, and the outgoing network queue. Numbered ar-rows depict the progress of requests from one queue toanother. Worker threads are also shown. The black circleon each thread represents the current position of threadexecution. We are especially interested in the case werethe server is operating at a nontrivial load. Hence, re-sponse time is dominated by queuing delays rather thanservice times. For the server to yield acceptable perfor-mance, the order of request service times must be muchsmaller than the order of tolerable server response times.If it takes Ci units of time to serve request i, and if Di is themaximum tolerable server response time, then typically∀ <<i j C Di j, : . We call this condition the liquid task model.The model is representative of high-performance serversthat handle many thousands of requests per second. Forexample, in the case of Web servers, a worst-case tolera-ble response time would be of the order of seconds to tensof seconds, whereas a typical request service time wouldbe of the order of hundreds of microseconds to single mil-liseconds. The liquid task model represents the limitingcase in which tolerable response times are generally fi-nite, yet the service times are practically infinitesimal.More formally, in the liquid task model, ∀ →i Ci: 0 and∀ →i C Di i: / 0. Although this model is an idealization, re-sults based on this model hold well in systems exhibitinga large number of small requests.

76 IEEE Control Systems Magazine June 2003

We demonstrate that a software systemcan be approximated by a linearizedmodel and describe the needed softwareactuators and sensors.

In a high-performance server approximated by a liquidtask model, the progress of requests through the serverqueues resembles a fluid flow. The service rate, dN t dtk( ) / ,of stage k, defines the amount of flow through that stage,where N tk( ) is the total number of requests served by thisstage by time t. The different queues in Figure 1(a) can there-fore be modeled as capacities that accumulate the corre-sponding flows. The number of requests queued at stage k,denoted Vk , is a quantity akin to volume, given byV F F dtk

t

k= −−∞∫ ( )in , where Fk is the service rate of stage k

(i.e., F dN t dtk k= ( ) / ) and Fin is the request arrival rate tothat stage. Queues also offer points at which flows, Fk , canbe manipulated. Figure 1(b) depicts the server from a con-trol perspective where capacities are represented by watertanks. Observe that “valves” in Figure 1(b) represent pointsof control (i.e., actuators, which manipulate theservice rates Fk). We assume in this analogy thatflow through the valve depends only on valveopening and not on the liquid level. Thus, the ar-rangement is perhaps more akin to a pump.

Note that the fluid flow analogy does notmake assumptions on how individual requestsare prioritized. It simply allows us to describethe dynamics of request flows and queue fill lev-els. In contrast, queuing theory and real-timescheduling theory have well-understood foun-dations for relating aggregate metrics such asqueue length, total workload, or total utilizationto delays seen by individual requests under aparticular prioritization policy. Hence, combin-ing control theory with real-time scheduling orqueuing analysis, one can develop feedbackloops to maintain appropriate queue fill levelsthat are guaranteed to produce the desired cli-ent delays under different request prioritizationschemes. Most of today’s servers implementfirst-in, first-out (FIFO) queuing on resources(such as socket queues and semaphore queues)and are composed of pools of same-prioritythreads or processes. Hence, in this article, weassume that queues are FIFO. Multiple clientclasses may exist, however, each with its ownqueue of a different priority.

Service ModelingInternet servers described by the liquid taskmodel have dynamics akin to those of flow dueto their intrinsic queuing structures. This ob-servation motivates the use of difference equa-tions to model Internet servers. Let y k( ) denotethe average performance (e.g., delay orthroughput) reported at the kth sampling in-stant. This variable reflects a measurement car-ried out during the most recent samplinginterval. Let u k( ) denote the control input at

that sampling instant. A server can be modeled by an nthorder ARMA difference equation relating u k( ) to y k( ). Thesystem order n represents the length of history that deter-mines the current server performance. Figure 1(b) suggeststhat the difference equation can be derived from a statespace representation of the server model

x x

x

( ) ( ) ( )

( ) ( ),

k A k bu k

y k C k

= − +=

1

where x( )k is the state vector and A, b, and C describe thesystem model. Representing software system dynamics bydifference equation models has recently gained much popu-larity. For example, in [4] a model of a Web proxy cache is de-rived, and in [5] a model of TCP dynamics is presented in the


WorkerThreads

WorkerThreads

Output to Clients

Output to Clients

Server

Server

OutgoingNetwork I/O

OutgoingNetwork I/O

OS Scheduler

CPU Ready Queue

6

4

3

5

2

1

ResourceAccess

ResourceAccess

RequestDequeuing

RequestDequeuing

I/O Queue

I/O Queue

ClientRequestQueue

ClientRequestQueue

Network I/OQueue

NetworkQueue

(a)

3

1

2

5

V3

4

6V4

V1

(b)

V2

CPU Queue

Figure 1. Server architecture: (a) the computing model and (b) the control-oriented representation.

presence of network active queue management based onRED gateways [6].

Derivation of computing system dynamics can be difficultin the absence of complete knowledge of software systemcode due to hidden interactions between different compo-nents in such systems. For example, a semaphore used forsynchronization can create a waiting queue that affects sys-tem dynamics. Yet the presence of such a semaphore mightnot be discovered without access to system code. Further-more, deriving system dynamics from first principles may beimpractical in some situations for administrative reasons.For example, the model parameters may need to be recom-puted every time the server’s hardware or software configu-rations are changed (e.g., due to a system upgrade), sincesuch changes may affect platform speed and consequentlythe rates and capacities in the system. Moreover, system ad-ministrators usually lack the expertise to perform con-trol-theoretic modeling. Therefore, a more practicalapproach may be to perform automatic parameter estima-tion (e.g., using a least-squares estimator). System identifica-tion requires adding software modules to instrument arunning software system and iteratively estimate the param-eters of difference equation models based on system inputand output. A successful application of the approach is de-scribed in [7] for a Web proxy cache to determine the relationbetween disk allocation and cache hit ratio.

A key prerequisite to system modeling is to define the ac-tuators and their manipulated variables (i.e., the valves inFigure 1(b)). These actuators represent the main interfacebetween the control subsystem and the computing subsys-tem in the feedback control loop. In the next section, we de-scribe the types of actuators in computing systems andtheir principles of operation.

Resource Allocation for QoS GuaranteesMost time-related performance metrics in software systemscan be tied to the state of the queues depicted in Figure 1.Delay metrics such as response time and latency are gener-ally related to the queue (tank) fill levels. Higher fill levelsimply longer delays and vice versa. Rate metrics such as ser-vice throughput are generally related to the dequeue rates(i.e., flows through the valves). Higher rates imply higherthroughput. Both sets of metrics can be affected by control-ling the different flows in the system. The flow (i.e., the ser-vice rate) of a stage in the software service depends on theamount of computing resources allocated to this stage andthe operation being performed. Allocating more resourceswill generally increase the flow the same way opening avalve would. Thus, to control performance, we need actua-tors that manage resource allocation or alter the functional-ity of the server in a way that manipulates the rate at whichwork is done. The feasibility of implementing such actua-tors in a computing system is one basic reason why controltheory is applicable. Actuators in computing systems fallinto three basic types as described below.

Input Flow ActuatorsAn input flow actuator manipulates the input workload of aserver, Fin . In software systems, admission control is a pri-mary input flow actuator that affects queue fill levels in theserver. In the simplest case, admission control limits thenumber of clients who access the server concurrently bydenying service to some of them, hence reducing the loadseen by the server. The actuator accepts as input a controlvariable m that determines the desired average flow. This isanalogous to controlling the opening of input valve V1 in Fig-ure 1(b) and discarding all requests that overflow. A more in-telligent admission control algorithm decides, on arequest-by-request basis, whether service should begranted (request admitted) or denied (request rejected),perhaps depending on client identity, queue fill level, andserver utilization. Input flow actuators are used when someclients are inherently more important than others such thatdenying service to less important clients is acceptable in fa-vor of higher priority clients. Examples of the control-theo-retic formulation of admission control schemes in real-timesystems are described in [8] and [9].

Quality Adaptation ActuatorsA more flexible way of controlling flow is to alter the pro-cessing requirements of the request. For example, reducingthe processing requirements increases service rate (i.e., in-creases F dN dtk k= / ), which is desired when the server ap-proaches overload. To reduce processing requirements,the service offers intermediate “degraded” service levels.In a Web server, a degraded level of service can correspondto providing an abbreviated version of content or a lowerquality version of images and sound. The approach is oftenfollowed manually by editors of various sports and newsWeb sites such as the Cable News Network (CNN),www.cnn.com, upon important breaking news. An examplewas the obvious abbreviation of the CNN front page in thefirst hours following the September 11 events in 2001. Twoimportant questions are i) how much content should be ab-breviated and ii) what fraction of clients need to receive theabbreviated version for the overload to be avoided. An au-tomatic Web content adaptation scheme that answersthese questions by using feedback control theory is de-scribed in [10]. The scheme dynamically selects a suitablecontent version for each client on a per-request basis de-pending on load conditions. Content adaptation has alsobeen discussed at length in multimedia applications. Agood survey is found in [11].

Generally, a quality adaptation actuator offers a flexibletradeoff between delay and quality. Since servers don’t haveunlimited queue space, some compromise is often unavoid-able to prevent server queue overflow at extreme load con-ditions. An advantage of quality adaptation actuators overadmission control schemes is that service is provided to allclients, albeit potentially at a degraded level. For effectiveload reduction, it is important that the actuator control the


bottleneck resource. For example, if the bottleneck re-source is the processor, CPU-intensive operations should bereduced. If the bottleneck is the network, the number of sentbytes should be reduced. The identity of the bottleneck canbe measured dynamically by measuring the utilization ofdifferent resources.

To approximate flow, a quality adaptation actuatorshould provide a continuous range of service rates, Fk , evenwhen the server exports only a finite (small) number of dif-ferent service levels. Consider a general server with M dis-crete service levels that differ in their consumption of thebottleneck resource. Let these levels be numbered 1,...,Mfrom lowest quality to highest quality. The level 0 may beadded to denote the special case of request rejection. Theactuator accepts as input the control variable m in the range[ , ]0 M . An essential challenge in actuator design is that of aunique mapping from m to the manipulated variables;namely, the fraction of clients served at each QoS level toproduce smooth flow control.

To resolve the aforementioned challenge, the authors of[12] propose to decompose the fractional value of input, m,into an integral part I and a fraction F , such that m I F= + .The two nearest integers to m (namely, I and I +1, whereI m I< < +1) determine the two most appropriate servicelevels at which clients must be served under the given loadconditions. The fractional part F determines the fraction ofclients served at each of the two levels. In effect, m is inter-preted to mean that a fraction 1 − F of clients must be servedat level I and a fraction F at level I +1. The scheme workswell when the input average request arrival rate does notchange quickly. Server utilization increases (possiblynonlinearly) when m is increased and vice versa. At the up-per extreme, m M= , all requests are given highest qualityservice. At the lower extreme, m = 0, all requests are re-jected. Hence, the actuator changes the amount of load on aserver with discrete service levels in a continuous fashion,depending on its input m.

Resource Reallocation ActuatorsA different way of controlling flow (other than quality adapta-tion) is to alter the amount of resources available for the pro-cessing of the request queue. This important category ofresource management actuators typically arises in serverswith multiple classes of clients. In such servers, it is often de-sired to maintain some constant ratio between the perfor-mance of different client classes. The computing resources areusually partitioned among these classes such that the sum ofall partitions is constant and equal to the total resource capac-ity of the service. We call this condition the constant-sum in-variant. Individual actuators can alter the partitioning, but thetotal resource allocation must remain constant. We call suchactuators resource reallocation actuators.

In a multiclass server, a separate control loop is often as-sociated with each client class. An actuator in each loop al-ters the resources allocated to it. An important design

objective of this approach is to ensure that individual loopsmay operate in isolation without violating the constant-suminvariant. We demonstrate how the above is achieved withan example. Consider a multiclass service that offers someclasses of requests better performance than others by allo-cating resource space among different request classes ap-propriately. Let the measured performance of request classibe Hi. One can think of Hi as the output of the ith controlloop. A very common model for discrimination in amulticlass environment is the relative differentiated ser-vices model [13], [14]. In this model, a differentiation policyspecifies that the measured performance of differentclasses should be related by the expression: H H Hn1 2: : :K =C C Cn1 2: : ... : , whereCi is the importance weight of class i. In acontrol-theoretic formulation, we define the relative perfor-mance, Ri, of class i as R H H H Hi i n= + + +/( )1 2 L . It deter-mines how the class is performing relative to other classes.The desired relative performance of class i should beR C C C Ci i ndesired

= + + +/ ( )1 2 L . This represents the set pointfor the class. This formulation is akin to ratio control [15].The difference R Ri idesired

− is the performance error ei of classi. Note that the aggregate performance error of the system isalways zero, because

e R R

C

C C C

H

ii n ii n i

ii n

n

1 1

1

1 2

≤ ≤ ≤ ≤

≤ ≤

∑ ∑∑

= −

=+ + +

−

( )desired

L

ii n

nH H H1

1 2

1 1 0

≤ ≤∑+ + +

= − =L

.

The aforementioned problem formulation was used inthe differentiated caching services architecture describedin [4] and [7]. The authors developed a Web proxy cachethat can give some Web content preferential treatment. Inthis example, the performance metric being controlled iscache hit ratio and the resource being allocated is diskspace. There are n content classes in the system. Each classof content i is assigned a different amount of cache storagesi, such that ∑ i is is the total size of the cache. A controller isinvoked at fixed intervals for each class at which it correctsresource allocation to that class based on the measured per-formance error. To compute the correction δs ki[ ]in resourceallocation, the controller uses a linear function f ei( ), wheref ( )0 0= . If the computed correction δs ki[ ] is positive, thespace allocated to class i is increased by| [ ]|δs ki . Otherwise,it is decreased by that amount. Since the function f is linear,∑ = ∑i i i if e k f e k( [ ]) ( [ ]). Since we showed that ∑ =i ie k[ ] 0, itfollows that ∑ = =i if e k f( [ ]) ( )0 0. Hence, the sum of correc-tions across all classes is zero (i.e., the constant-sum invari-ant is maintained).

Many computing systems with multiple client classesuse similar per-class feedback loops [16], [17]. The differ-ence between these systems lies generally in the resourcebeing allocated by the actuators. For example, instead ofmanipulating storage allocation, actuators can be built tomanipulate the number of worker processes (and hence


CPU capacity) allocated to a class [16] or the fraction of net-work link bandwidth allocated to a flow [17].

This section discussed the natural existence of actuatorsin computing systems, which makes it possible to imple-ment the “valves” that appear in Figure 1(b). Another impor-tant cornerstone of applying feedback control to computingsystems is the existence of a natural translation from com-mon QoS assurance problems into those of feedback con-trol. This topic is covered in the next section.

QoS MappingA cornerstone of a control-theoretic paradigm for QoS guar-antees in software systems lies in the ability to convert com-mon resource management and software performanceassurance problems into feedback control problems. Onecan think of each QoS control problem as having a corre-sponding control-loop instantiation that describes how thisparticular QoS control problem is solved using feedbackcontrol. We call such an instantiation a control-loop tem-plate. Here we describe control-loop templates for the main

QoS control problems such as absolute convergenceguarantees, performance isolation, statistical multiplexing,prioritization, relative differentiated service guarantees,and optimization guarantees. The fundamental buildingblock in these templates is one that implements the basic(absolute) convergence guarantee. Interconnecting suchblocks can lead to formulating more complex guaranteessuch as relative guarantees, prioritization, and optimizationas feedback control problems.

The Absolute Convergence GuaranteeSince it is impossible to achieve absolute guarantees in asystem where load and resources are not known a priori, wedefine the absolute guarantee problem as one of conver-gence to a specified performance. The statement of theproblem is to ensure that a performance metric, R, i) con-verges within a specified exponentially decaying envelopeto a fixed value, Rdesired, and that ii) the maximum deviationR Rdesired − is bounded at all times, as shown in Figure 2(a).


R

Rdes.

Specified Maximum Deviation

Actual Performance, R

Time

Specified Decay Envelope

(a)

ApproximateSystem Model

PerformanceError Correction

PerformanceSet Point

ControllerActuator

(Resource Allocator)

ResourceAllocation

SoftwareSystem

ActualPerformance

PerformanceSensorMeasured

Performance

(b)



First-ClassAllocation

Controller

Controller

UnusedCapacity

UnusedCapacity

Correction

AdmittedFirst-Class Clients

AdmittedClientsActuator

(Resource Allocator)

Actuator(Resource Allocator)

SoftwareSystem

SoftwareSystem

ClassResourceConsumption

ResourceConsumption

ResourceConsumption

PerformanceSensor

PerformanceSensor

PerformanceSensor

PerformanceSensor

MeasuredConsumption of First-Class Clients

MeasuredConsumption of First-Class Clients

MeasuredConsumption of Second-Class Clients

MeasuredConsumption of Second-Class Clients

LeftoverCapacity

TotalLeftoverCapacity

LeftoverCapacity

UnusedCapacity

Correction

AdmittedSecond-Class Clients

Controller

Controller



SoftwareSystem

SoftwareSystem

ClassResourceConsumption

Virtual-EstateAllocation

mi

mb

Best-EffortAllocation

Admitted Best-EffortClients

(c) (d)

Figure 2. Control loop templates: (a) the absolute guarantee specification, (b) basic loop, (c) prioritization, and (d) excess capacitymanagement.

The absolute convergence guarantee is translated into thecontrol loop shown in Figure 2(b). The loop samples the mea-sured performance, compares it to the desired value Rdesired,and uses the difference to induce changes in resource alloca-tion via the actuator A R( ). The absolute convergence guaran-tee loop is the elementary building block from which all otherQoS assurances follow. In the context of time-related perfor-mance metrics, it is interesting to classify the convergenceguarantee loops depending on the performance variable be-ing controlled. As is the case with physical plants, the con-trolled output of interest affects the model of the system andwhether or not the control loop is linear.

• Rate and queue-length control: To a first approxima-tion, rate metrics and queue length are easiest to con-trol because they result in linear feedback loops. The(flow) rate can be controlled directly by the actuators.Queue length can be linearly controlled by controllingthe flow. The simplest example of a loop of this cate-gory is a server utilization control loop. Such a loop,for instance, can be used to maintain high server utili-zation while avoiding overload. In this section wemake several uses of utilization control as a basicbuilding block for other types of guarantees.

• Delay control: Delay guarantees are more difficult toprovide. This is because delay is inversely propor-tional to flow. If a request arrives at a queue of lengthQ,with a dequeue rate of r, the queuing delay d of the re-quest is d Q r= / . Generally, since the rate r changesover time, the delay of this request is d dq rQ= ∫0 / . Theinverse relation between the manipulated variable(rate) and the delay makes the control loop nonlinear.Note that, although level control (i.e., queue fill-levelcontrol) is a very common problem in physical plants,one generally does not care how long it takes a particu-lar liquid molecule to leave the tank. Thus, the nonlin-ear delay control problem is less commonlyencountered in physical process control. We believethat an efficient solution to this type of nonlinearity willhave great impact on software QoS-control research.

The Resource Reservation GuaranteeIn some applications, it is necessary to guarantee that cer-tain resources be reserved. The absolute guarantee solu-tion, described above, is particularly useful for resourcereservation for the purpose of isolating applications thatshare common resources. For example, in a Web-hosting ap-plication, the Web server may offer a “virtual estate,” suchas a percentage of server capacity, for sale to hosted Websites. The particular capacity allocation purchased by agiven site can be enforced using the control loop in Figure2(b). The amount of server capacity owned by the site willrepresent the utilization control-loop set point. The actualamount of resources used can be measured dynamically.The difference controls the actuator, which in this case can

be an admission controller for requests for the particularsite. One control loop is needed per Web site.

The Prioritization GuaranteeTo see how prioritization can be implemented using feedbackcontrol, consider a sampled system in which a large numberof new service requests is introduced at each sampling time.Requests are classified by priority and are enqueued in a sep-arate queue depending on their class. The scheduling policydepletes one queue at a time in priority order during the sam-pling interval. By the end of the sampling interval, somequeues will be fully depleted, at most one queue will be par-tially depleted (the one the scheduler was working on whenthe next sampling time arrived), and the rest will remain un-touched. To emulate this effect with control loops, imagineeach queue being a separate pipe with its own control valve.Given server capacity, the valves of queues that can be fullydepleted are saturated in the open position. The valve of thenext queue is controlled to dequeue exactly the right fractionof requests within the sampling interval to fully utilize anyleftover server capacity. All the remaining (lower priority)valves are closed. Below we illustrate a feedback controlscheme that achieves the above. Note that the scheme is un-conventional in that at most one loop is operating at anygiven time outside of saturation limits. The rest are saturatedin either the fully open or the fully closed position as ex-plained above. Integrator antiwindup is used to prevent un-reasonable integral action buildup in the controller duringsaturation periods.

Let there be M priority classes defined within a serversuch that priority 1 is highest and priority M is lowest. Col-lectively, clients of the server are allocated a target utiliza-tionU*. This capacity should be made available to clients inpriority order. A resource allocation loop can be createdthat gives the entire server capacity to the highest priorityclass. The unused capacity of each class is measured andtreated as the set point for the resource allocation to lowerpriority classes. If this capacity is not enough, these clientsare degraded or rejected accordingly by the actuator intheir utilization control loop. The architecture is describedfor a two-class server in Figure 2(c). One control loop isneeded per class.

For illustration, consider the operation of a system withtwo priority classes. When the high-priority class is gettingmore than its resource share, the controller error of thehigh-priority loop is negative. In this case, i) the class is con-trolled to reduce its consumption to the allotted share, andii) the lower priority class has a zero resource allocation.The first condition is ensured by the loop of the higher prior-ity class. The second is ensured by the other loop since itsset point will be negative in this case, causing its actuator(e.g., the admission controller) to saturate at a fully closedposition. Conversely, when the higher priority class doesnot consume all its allocated resources, the error (leftoverresources) is positive, which is the set point to the lower pri-


ority loop. The actuator of the high-priority class is satu-rated at a fully open position, admitting all requests of thisclass. The loop for the lower priority class makes sure onlyas many requests are admitted as permitted by the leftoverresources from the high-priority class. Hence, the loops em-ulate priority semantics.

The Statistical Multiplexing GuaranteeOne shortcoming of resource reservation is that it can leadto unnecessary resource underutilization when some re-source owner does not use all the resources allocated to it.This is objectionable if some other owner is starving for re-sources in the meantime. Hence, a statistical multiplexingscheme is needed whenever spare capacity exists on themachine. Individual resource allocations should be en-forced only when the entire machine is overloaded. Thesetwo requirements can be simultaneously satisfied by thecontrol loop set in Figure 2(d).

To describe how statistical multiplexing is achieved, as-sume that the controller output in the utilization controlloop of a server i is mi. All surplus requests are treated as alower priority server, called the best effort server. Let thecontroller output of the utilization control loop of the besteffort server be mb . Sharing of spare capacity is achieved us-ing a variation of a min-max selector scheme [15] in whichthe objective is to minimize degradation of premium traffic.In this scheme, the service level of a request for a givenfirst-class server i is determined by the actuator of premiumtraffic if m mi b> and with the actuator of best effort trafficotherwise. Thus, the request is handled according to thehigher of mi and mb . When the first-class server is over-loaded while the machine as a whole is not, m mb i> . Conse-quently, incoming requests are served with qualitydetermined by mb , which is higher than that warranted bymi, thus utilizing excess machine capacity. On the otherhand, if the machine is overloaded, m mb i< . Consequently,the quality of content delivered by server i is determined bymi. Thus, the individual server is prevented from exceedingits capacity allocation. The mechanism allows smooth andinformed switching between a mode of operation where anindividual server i is allowed to exceed its capacity alloca-tion and a mode of operation where it is required to staywithin its original capacity.

The Relative GuaranteeIn some applications, it is desirable that the ratio between theperformance of different classes of work be fixed; for exam-ple, it may be that the delays of two traffic classes in a net-work should be fixed at a ratio of 3:1. This fixed ratio is a goodcandidate for the performance set point, R. We can thereforepose relative guarantees as a variation of the ratio controlproblem [15]. Generally, if there are n request classes in thesystem, and if Hi is the measured performance of class i, therelative guarantee specifies that H H H C C Cn n1 2 1 2: : : : : ... :K = ,whereCi is the weight of class i. One way to achieve this guar-

antee is to formulate per-class relative performance setpoints in the form R C C C Ci i ndesired

= + + +/ ( )1 2 L and compareeach against R H H H Hi i n= + + +/ ( )1 2 L . Note that this loop(same as in Figure 2(b), but with the set point defined above)is nonlinear, since the sensor divides outputs of differentloops to provide feedback on the relative performance of in-dividual classes, as well as because flow is inversely propor-tional to delay.

It is possible to formulate the relative guarantee problemin a way that avoids the former nonlinearity. In this case, eachoutput Hi is multiplied by the constant W C C C Ci n i= 1 2K / .Note that when the performance metric is satisfied (i.e., whenH H H C C Cn n1 2 1 2: : : : : ... :K = ), the ratio H Ci i/ is equal for allclasses. Thus, the products H Wi i are equal for each class i.Let the set point be the average of these products (i.e.,R H W ni

ni idesired = ∑ = 1( ) / ). Hence, when the performance met-

ric is satisfied, all H Wi i are equal to the set point. When themetric is not satisfied, the error ei of class i is given bye R H Wi i i= −desired . A control loop is designed per class withthe objective of driving the error of that class to zero. Thecontrol loop in this case is linear, sinceWi for a given class isconstant. However, the control loops of different classes arecoupled through their common set point, which is a weightedsum of the outputs of all loops.

Finally, yet another way of defining the control loops is todefine set points on pairwise relative performance metricsC Ci i/ − 1. Only n −1loops are then needed. Each loop reducesthe corresponding error e C C H Hi i i i i= −− −/ /1 1 to zero. Inthis case, each loop decides on the ratio of resource alloca-tion between two adjacent classes. Resources are then glob-ally reallocated such that i) all pairwise ratios are satisfiedand ii) the sum of allocated resources is equal to the total re-source capacity.

The Utility Optimization GuaranteeAnother type of performance guarantee addressable us-ing a control-theoretic framework is that of utility optimi-zation. Following a microeconomic model [18], consider acomputing service that produces an amount of work w.Let the benefit per unit of work be k. Hence, the total util-ity U produced by the service is U kw= . Let the resourceconsumption of the service be some nonlinear function,g w( ), which represents a measure of cost. It is desired toachieve the maximum net profit (i .e. , maximizekw g w− ( )). Assuming a concave cost function, g w( ), theprofit is maximized when the marginal utility is equal tothe marginal cost, or when ( ( ) / )dg w dw k= . The equationcan be solved for w, which then becomes the control setpoint, R. In a computing example, w may be the desiredserver utilization, the desired workload size, or othermetric, depending on the problem formulation. The re-sulting feedback loop is illustrated in Figure 2(b), wherethe set point is derived as described above.

In summary, we have identified the main types of perfor-mance guarantees that cover the performance requirements


of many software systems. For each type of guarantee, wehave specified a feedback control-loop template that can beused as a basis for providing that type of guarantee. We be-lieve that these templates, and the nonlinearities they ex-hibit, provide an interesting opportunity for exploring newcontrol techniques that target the characteristic couplingsand nonlinearities of loops implementing guarantees on time-related software performance metrics.

Feedback-Based SolutionsIn this section, we bring together the concepts introducedabove by describing several successful applications of thecontrol-theoretic approach in the software domain. We firstgive a detailed example involving Web servers and then brieflyintroduce other applications on Internet servers, CPU sched-uling, networking, and microprocessor architectures. The suc-cess of feedback control theory in these different domainsdemonstrates the generality and strength of the control-theo-retic framework when applied to computing systems.

A Detailed Example of RelativeDelay Guarantees in Web ServersAs the Internet becomes commercialized, it generates an in-centive for e-businesses to provide guaranteed shorter ser-vice delay to premium customers, either because they payhigher fees or because they are more important to the busi-ness than the other customers. Web servers have to handledisturbances caused by highly unpredictable workload. Forexample, the population in each customer class often variesdramatically at runtime, yet offered performance must re-main constant.

In this section, we present an example of applying feed-back control theory to provide relative delay guarantees onWeb servers. Every HTTP request over the Internet belongsto a class i. A desired relative delay Wi is assigned to eachclass i. The service delayC ki( ) of class i is defined as the av-erage delay of established connections of class i measuredwithin the kth sampling period (( ) , )k T kT−1 , where T is theconstant length of the sampling period. A relative delayguarantee requires that C k C k W Wj i j i( ) / ( ) /= for anyclasses j and i. For example, if class 0 has a desired relativeweightW0 1= and class1has a desired relative weightW1 2= ,the relative delay guarantee requires that the delay of class0 be half that of class 1.

Sensors and ActuatorsThe first step in the feedback control design is to identifythe controlled and manipulated variables in Web serversystems. We focus on the Apache server and HTTP 1.1 re-quests. Apache is currently the most popular Web serversoftware. The server runs a pool of server processes listen-ing to a common TCP port. Each server process can onlyhandle one TCP connection at any time instant. The HTTP1.1 protocol works as follows: 1) a client (e.g., a Webbrowser) establishes a TCP connection with a server pro-cess; 2) the client submits an HTTP request to the server

over the TCP connection; 3) the server processes therequest and generates a response; 4) the server sends theresponse back to the client over the TCP connection; 5) toamortize the overhead of TCP connection establishment,the TCP connection is left open after the response has beentransmitted in anticipation that the same TCP connectioncan be reused by a following HTTP request from the sameclient; if a new HTTP request arrives within a configurablekeep alive interval (e.g., 15 s), the TCP connection is keptopen; otherwise the connection is closed. This feature iscalled “persistent connections.”

From the client’s perspective, the delay of an HTTP re-quest includes three components: the connection delay onthe server for establishing a TCP connection with a serverprocess, the processing delay on the server for local compu-tations (network protocol processing, open/read files, andrunning CGI scripts), and the network delays for transmit-ting the TCP connection request, the HTTP request, and theresponse over an established TCP connection. In this exam-ple, we focus on controlling the connection delay. The con-nection delay can be significant in overload conditionswhen all server processes are tied up with existing connec-tions. When the server is highly loaded, all new TCP connec-tion requests are queued in the TCP listen queue until theyare accepted by a server process. The connection delay of aservice class can be controlled by manipulating the processbudget B ki( ) (i.e., the number of server processes allocatedto class k in the sampling period ( ,( ) )kT k T+1 ). Increasingthe process budget of a class leads to a shorter connectiondelay for this class.

A relative guarantee scheme is used to provide relativedelay guarantees in the Apache server. The connection de-lay ratio of two adjacent classes i and i −1 is controlled by acontrollerCRi. ForCRi, the controlled variable is the desiredconnection delay ratio V k C k C ki i i( ) ( ) / ( )= − 1 betweenclasses i and i −1, and the reference is the desired delay ratioW Wi i/ − 1. The manipulated variable is the process budget ra-tio U k B k B ki i i( ) ( ) / ( )= − 1 between class i −1 and i. Note thatthe class of the denominator and the numerator is reversedin our notations ofV ki( )andU ki( )due to the inverse relationbetween service rate and delay. At runtime, the controllerCRi periodically updates the process ratioU ki( )and invokesa resource reallocation actuator to allocate server pro-cesses to different classes according toU ki( ). The goal is tocontrol the delay ratioV ki( ) to remain close to the referenceW k W ki i( ) / ( )− 1 .

System IdentificationAs an instance of the server architecture described earlier,the Web server has dynamics caused by the queuing of TCPconnection requests. To compute the dynamic model be-tween the connection delay ratio and the process budgetratio, we explore the system identification approach. We ap-proximate the server with an ARMA difference equation:


V k a V k j b U k jj

n

jj

n

j( ) ( ) ( )= − + −= =∑ ∑

1 1

.

In an nth-order model, 2n parameters { , | ,..., }a b j nj j =1need to be estimated. We estimate the open-loop dynamicsof the Apache Web server using pseudorandom digitalwhite noise as input. The input signal randomly changes theprocess ratio between two different levels. A least-squaresestimator is invoked periodically to estimate model param-eters at every sampling instant.

We conduct system identification experiments to modela real Apache Web server on five Linux PCs connected with a100-Mb/s Ethernet. A Linux PC runs an Apache server to-gether with our system identification software, and fourLinux PCs run a SURGE workload generator [19] to simulate400 clients. Figure 3(a) shows that the estimated parame-ters of a second-order model at successive sampling in-stants in a 30-min run (the system identification is started at2 min after the run starts to give SURGE time to fully startup). The estimates of the parameters ( , , , )a a b b1 2 1 2 convergeto (0.74, −0.37, 0.95, −0.12).

To verify the accuracy of the model, we reran the experi-ment using a white noise input with a different seed andcompared the actual delay ratio to that predicted by the es-

timated model. Figure 3(b) shows that the prediction of theestimated model is consistent with the actual relative delaythroughout the 30-min run. These results demonstrate thatthe real Apache server can be modeled as a second-orderdifference equation. Experiments also show that an esti-mated first-order model had a larger prediction error (Fig-ure 3(c)) than the second-order model, whereas anestimated third-order model (Figure 3(d)) did not improvethe modeling accuracy. We chose the second-order modelas a tradeoff between the model accuracy and complexity.

Note that although the general server model shown inFigure 1(b) suggests a high-order system, the server is usu-ally dominated by two important bottlenecks. The first isthe client request queue on the server’s input port. Hun-dreds of requests are often queued there until they aredequeued by some server process. The second queue is theCPU ready queue. More than a hundred server processesare usually dequeuing requests concurrently. The readyqueue thus has a large number of entries. Together the twoqueues give rise to a second- order system, as verified bythe above experiment.

Evaluation of the Closed-Loop ServerWe use the digital root-locus method to design a PI control-

ler for the difference equationmodel. The PI controller guar-antees stability and zerosteady-state error and has asettling time of 4.5 min. Thefeedback control loop is im-plemented by modifying theApache server software. Thedetailed system identifica-tion, design, and implementa-tion are described in [16].

Due to the highly unpredict-able Internet client access pat-terns, it is critical for a Webservertoachieverobustrelativedelay guarantees in the face ofdisturbances caused by chang-ing workloads. We run experi-ments on the Linux PC testbedto compare the performance ofthe open-loop server with theclosed-loop server for changingclient populations. Both theopen-loop and the closed-loopserversaretestedwiththesameworkload. Each experimentstarts with the nominal work-load generated by 200 basic(class 1) clients and 100 pre-mium (class 0) clients. To testthe servers’ disturbance rejec-


1

0

–1

a1a2b1b2

Est

imat

edP

aram

eter

s

ActualEstimate

ActualEstimate

ActualEstimate

8

8

8

6

6

6

4

4

4

2

2

2

0

0

0

Del

ayR

atio

Del

ayR

atio

Del

ayR

atio

0

0

0

0

120

120

120

120

240

240

240

240

360

360

360

360

480

480

480

480

600

600

600

600

720

720

720

720

840

840

840

840

960

960

960

960

1080

1080

1080

1080

1200

1200

1200

1200

1320

1320

1320

1320

1440

1440

1440

1440

1560

1560

1560

1560

1680

1680

1680

1680

1800

1800

1800

1800

Time [s](a)

Time [s](b)

Time [s](c)

Time [s](d)

Figure 3. System identification results for an Apache server; (a) estimated model parameters(second-order model), (b) modeling error (second-order model), (c) modeling error (first-ordermodel), and (d) modeling error (third-error model).

tion capabilities, the number of premium clients is suddenly in-creased from 100 to 200 at 870 s.

The open-loop server has a fixed process budget ratiothat is hand-tuned to 0.83 through extensive system profil-ing based on a nominal workload. The process budget ratioof the closed-loop server is arbitrarily initialized to 1 with-out hand tuning. The reference to the controller is set to 3,which requires the connection delay of the basic clients tobe three times that of the premium clients.

The performance of the open-loop server is shown in Fig-ure 4(a). The open-loop server performs well in the beginningof the experiment because its process budget ratio has beenfine-tuned for the nominal workload. However, after the num-ber of premium clients suddenly increases to 200 at 870 s, theconnection delay of the premium clients increases signifi-cantly. Consequently, the connection delay ratio drops fromthe reference and remains below 1 (i.e., the basic clients re-ceive a shorter delay than the premium clients!) in the rest ofthe experiment. This result shows that the open- loop servercannot reject disturbances caused by a changing workload.

The performance of the closed-loop server is shown inFigure 4(b). In the beginning of the experiment, the connec-tion delay ratio of the closed-loop server converges toaround the reference by adjusting its process budget ratio.In response to the population change at 870 s, the feedbackcontrol loop allocates more processes to the premium cli-ents while reducing the number of processes for the basicclients. By 1,140 s, the connection delay ratio settles aroundthe reference and stays close to it for the rest of the run. Thisresult demonstrates that the closed-loop server is able toreject the disturbance caused by instantaneous changes inclient populations. The re-covery time is close to the de-signed settling time of 4.5min. The server remains stablethroughout the run, and theconnection delay ratio re-mains close to the reference atsteady state.

In summary, we have shownthe effectiveness of feedbackcontrol theory in achieving QoSguarantees in a representativeInternet server. We successfullydeveloped effective softwaresensors and actuators toinstantiate the relative guaran-tee control template. We haveshown that the dynamics of acomplex Internet server can bemodeled through system iden-tification. The implementationand evaluation on real serversystems demonstrate that feed-back control loops can provide

robust QoS guarantees and disturbance rejection capabilities inInternet servers.

Other Applications in Computing SystemsFeedback control theory has been successfully applied to abroad spectrum of computing systems. The application do-mains cover different resources that need to be controlled(network bandwidth, CPU cycles, storage spaces, and I/Obandwidth), as well as different metrics and types of QoS guar-antees. We now briefly summarize some recent application ex-amples, providing references for more detailed information.

Internet ServersIn addition to the Web example presented earlier, severalrecent papers [10], [12], [20] present a control-theoreticalapproach to Web server resource management throughWeb-content adaptation. Feedback control loops were de-veloped to achieve absolute convergence guarantees, ca-pacity reservation, and prioritization on request rate anddelivered bandwidth for different service classes. In [21],multi-input, multi-output (MIMO) control was designed toachieve absolute convergence guarantees on both mem-ory and CPU utilization in Web servers. In other work, feed-back control and fuzzy control were applied to control theinput queue length in Lotus e-mail servers with admissioncontrol [22], [23].

CPU SchedulingFeedback control theory has been used for CPU schedulingin real-time systems, such as multimedia and embedded


0

0

120

120

240

240

360

360

480

480

600

600

720

720

840

840

960

960

1080

1080

1200

1200

1320

1320

1440

1440

1560

1560

1680

1680

1800

1800

Time [s](a)

Time [s](b)

5

5

4

4

3

3

2

2

1

1

0

0

Del

ayR

atio

()/

()

Cm

Cm

10

Reference( )( )

V kU k

Reference( )( )

V kU k

Vk

Uk

()

and

()

Figure 4. Comparing the performance of the open-loop and closed-loop servers: connection delayratio V k C k C k( ) ( ) / ( )= 1 0 ; process budget ratio U k B k B k( ) ( ) / ( )= 0 1 ; (a) open-loop server and (b)closed-loop server.

control systems. Steere et al. [24] developed a feed-back-based CPU scheduler that coordinates the CPU cyclesallocated to the consumer and supplier threads to guaran-tee the fill level of buffers. In [9] and [25], feedback controlreal-time scheduling algorithms were developed to providedeadline miss ratio guarantees for real-time applicationswith unknown task execution times. Feedback controlreal-time scheduling has also been extended to handle dis-tributed systems [8]. All the above scheduling algorithmsare designed for absolute convergence guarantees.

Storage ManagementStorage is another critical resource in many server systems.In [4], a relative guarantee template was developed to pro-vide a relative hit ratio guarantee in a Web proxy cachethrough a resource reallocation actuator that dynamicallychanges the disk space allocation for different serviceclasses. Recently, adaptive control was applied to the sameWeb proxy cache to improve its portability and robustnessvia automatic controller tuning [7].

The Aqueduct system [26] featured a feedback controlloop that regulated the speed of background data migrationin enterprise storage servers while bounding the perfor-mance impact on front-end applications. Aqueduct pro-vided absolute convergence guarantees on the I/O latencyof front-end applications during data migration.

Network RoutersAt the network layer, control theory was applied to packetflow control in Internet routers. Hollot et al. [5] appliedcontrol theory to analyze the RED active queue manage-ment on IP routers. Their control analysis provided guid-ance for tuning the RED algorithm, which had previouslybeen a difficult problem. Recently, a feedback-based algo-rithm was developed for quantitative assured forwardingservices [17] to provide absolute and proportional differ-entiation of loss, service rates, and packet delays onInternet routers. Li and Nahrstedt [27] developed a hierar-chical architecture that integrates an upper layer fuzzycontrol and a lower layer packet rate control to achieve ab-solute convergence guarantees on tracking precision indistributed visual tracking systems.

Microprocessor ArchitectureFeedback control theory has also been applied in micro-processor architectures. In [28], absolute convergenceguarantees were given on CPU chip temperatures by apply-ing control-theoretic techniques to microprocessor ther-mal management.

Middleware for QoS ControlAs shown earlier, over the last several years, an increasingnumber of individual feedback control solutions have been de-

veloped for various software performance assurance prob-lems. A natural question is whether any generalization of thesesolutions is possible. Indeed it seems that a small set of QoSguarantee types covers a wide range of applications, and foreach of these classes of guarantees, we can define control-the-oretic templates that are amenable to online solutions. In com-puter systems where many applications can make use ofsimilar services, these services are often provided in terms ofmiddleware. In our work, we have developed a middlewarecalled ControlWare [29] that supports multiple types of QoSguarantees, each based on feedback control.

ControlWareControlWare is a middleware QoS-control architecturebased on control theory, motivated by the needs of perfor-mance-assured Internet services. ControlWare allows theuser to express QoS specifications offline, maps these speci-fications into appropriate feedback control-loop sets, tunesloop controllers analytically to guarantee convergence tospecifications, and connects loops to the right performancesensors and actuators in the application such that the de-sired QoS is achieved. A main novelty of our middleware liesin isolating the software application programmer from con-trol-theoretic concerns while utilizing this theory to achievethe desired QoS guarantees. At the same time, ControlWareisolates the control engineer from the software task of inter-facing the controller to the controlled software system anddesigning software performance sensors and actuators. Aconceptual representation of ControlWare componentsused in the process of designing software performance-as-surance loops is shown in Figure 5(a).

ControlWare contains a library of macros written in our to-pology description language, each formulating a particulartype of QoS guarantee as a feedback control problem. The li-brary is extensible in that a control engineer can transform anew guarantee type into a macro that describes the corre-sponding loop interconnection topology and stores thatmacro in the middleware’s library. Currently, the library in-cludes macros for absolute convergence guarantees, relativedifferentiated service guarantees, prioritization, and optimi-zation guarantees. Each macro, like a block diagram, includescomponents such as sensors, actuators, and controllers.ControlWare contains a library of common sensors and actu-ators that can be used in these software control loops. The li-brary is extensible in that it is possible for an applicationprogrammer to add new sensor and actuator types.

Overall, ControlWare can express many common guaran-tee types required in performance-assurance software bycasting them appropriately as feedback control problems.Once the control loops are instantiated from the QoS specifi-cation, the middleware uses textbook techniques to esti-mate system models and determine appropriate feedbackcontroller parameters for guaranteed convergence of thecontrol loops to the specified performance.


SoftBus: The ControlWare BackboneSeveral implementation challenges had to be overcome to cre-ate ControlWare, including interoperability. To promoteinteroperability, the engineering community standardizedopen-layered interface architectures such as the Fieldbus [30],which greatly simplify the interconnection of sensors, actua-tors, and controllers in a digital control system. Similarly, acrucial step in developing an open middleware layer for soft-ware QoS control is to provide a similar generic applicationprogramming interface (API) and communication backbone tointerface the computing and control subsystems. This back-bone must be appropriate for distributed software rather thanfor a physical bus. We call this backbone a SoftBus.

ControlWare implements a SoftBus that provides a com-mon interface for efficient information exchange betweensoftware performance sensors, actuators, and controllersacross machines and address spaces. The sen-sors, actuators, and controllers are viewed asinter- changeable plug-in modules. Note thatthese modules are not physical devices (suchas a temperature sensor), but rather softwarecomponents that conceptually act like a sensoror actuator (e.g., a software load sensor mightinvoke an operating system call to measure CPUutilization). Modules connected to SoftBusneed not know each other’s locations and neednot worry about distributed communication.Underneath the common API, different informa-tion exchange mechanisms are developed fordifferent situations. This layered architecture isdepicted in Figure 5(b).

Plug-In Sensors and ActuatorsIn Softbus, we support two types of softwaresensors and actuators: passive and active. Apassive sensor or actuator is just a function orsoftware component that returns sample dataor accepts a command when called by the con-troller. An active sensor or actuator, in contrast,is a process or thread that may be running in itsown address space. It is usually awakened peri-odically by the operating system scheduler toperform sensing or actuation. For example, anidle-CPU-time sensor may be implemented as anactive sensor process that runs at the lowestpriority and computes the percentage of time ithas been executing. If a controller needs to com-municate with such a sensor, some kind of IPCshould be used instead of a direct function call.Controllers are designed completely independ-ent of the identity of the sensors and actuators.From the controller’s perspective, the softwaresystem being controlled acts like a regular phys-ical plant.

Sensors typically amount to a modest instrumentation ofapplication code. For example, a sensor measuring the re-quest rate on a particular site can be implemented as a sim-ple counter. A sensor measuring delay can be implementedas a moving average of the difference between two timestamps. Often the measured metric is already available as avariable maintained by the controlled software service (e.g.,queue length) or the operating system (such as CPU utiliza-tion). To implement the sensor, one only needs to pass thevalue to the middleware. The main challenge, however, isthe design of the actuator. To meet this challenge, ourmiddleware includes a generic resource manager (GRM)that can be thought of as an all-purpose actuator or a “ge-neric control valve.” The actuator interfaces to any resourcequeue (i.e., “pipe”) such that resource allocation (i.e.,“flow”) can be controlled in a simple, unified, yet customiz-


QoSContract

ControlConfiguration

File

QoS Mapper Control LoopComposition Sys ID

ControllerDesign

Software QoSControl Loops

Controller Monitor GRM

ControlWare Library

(a)

Application

Middleware

Monitor Actuator Monitor Actuator

SoftBus

GenericResourceManager(GRM)

Control Group Control Group

WhiteNoise

Generator

ModelEstimator

ControllerDesignService

QoSMapper

GroupManagement Timer

System Identification

(b)

ControllerController

Control Group Control Group

Figure 5. ControlWare: (a) development methodology and (b) interfacing tothe SoftBus.

able manner. In the following, we shed some light on the ge-neric resource manager in ControlWare.

Generic Resource ManagerOur GRM is designed for use with Internet servers such asWeb servers, DNS servers, mail servers, and proxy cacheservers. As mentioned earlier, in these applications, it isthe order and quantity of resource consumption that de-cide the quality of the service. Therefore, the actuatormust control the access to the resources quantitatively.The actuator uses quota, which is one of the knobs it ex-ports, to represent the resources allocated to each class. Itcan change the QoS of different classes by changing thequota dynamically at runtime. The definition of “quota” de-pends on the type of resource under consideration. Gener-ally, server resources can be categorized into twocategories according to their access patterns:

• Time-multiplexed resources. Such resources as CPUscannot be accessed by multiple users at the sametime. Instead, access has to be serialized and multi-plexed in time. For such resources, the GRM main-tains per-class queues. A classifier classifies therequests and puts them into the different queues. Thequota refers to the dequeue rate of each class as afraction of server capacity.

• Space-multiplexed resources. Memory and disk spaceare examples of such resources. They can be used bymultiple users at the same time. Typically, each classhas a quota indicating the amount of the resource ded-icated to it. A queue may or may not be necessary inthis scenario. In many cases, requests are admitteduntil the quota is used up, at which point further re-quests are simply rejected. An interesting alternativeis that when some class’s quota is used up and a new

request from this class arrives, a previ-ously allocated resource amount can berevoked from other clients and trans-ferred to this new request. Cache manage-ment is an example of this phenomenon,where newly cached pages replace olderpages in the cache.

In summary, the generic resource managerunderstands the notion of traffic classes and ex-ports the abstraction of resource quota to rep-resent the amount of logical resources allocatedto a particular class. The action of the managerlies in controlling resource quota allocations.

The structure of the generic resource man-ager is shown in Figure 6(a). In the figure, theClassifier and Resource Allocator are providedby the application. The Resource Allocatordoes resource allocation. The Queue Managermaintains one queue for each class, governedby a certain queuing policy. The Quota Managermaintains a resource quota for each class.

To use GRM, the application must exportthree interfaces: allocProc, rejectProc, andrevokeProc. allocProc is called to execute ap-plication-specific resource allocation meth-ods. When a previously arrived request hasto be rejected, rejectProc is called to docleanup work (for example, close the net-work connection). revokeProc is called whena resource previously allocated to some cli-ents needs to be revoked (such as the casewith cache replacement policies).

Figure 6(b) summarizes the interaction be-tween GRM and the application. When some re-source is requested by the application, therequest is first classified by the Classifier. Afterthat, the request is passed to GRM by callinginsertRequest. GRM controls resource allocation


ResourceRequest

ClassifierQueue

Manager

QueuePolicy

Actuator

ResourceAllocator

QuotaManager

GRM

(.....){

If (queue of this class is not empty){buffer this requestreturn

}if (this class still has quota){

(...)update quota usage

} else {buffer this request

}}

InsertRequest

allocProc

resourceAvailable

allocProc

(.....){

update quota usageget requests from classthat still has quota

(...)

}

(a)

(b)

Applicationclass = (request)

(...,class,...)classify

insertRequest

Resource Allocator

allocProc(...){

do resource allocation}

When some resource available(...)resourceAvailable

Figure 6. Generic resource manager: (a) structure of GRM and (b) resourceallocation procedure.

by checking the request against two constraints: the queuelength and the quota constraint. If the queue for the given classis empty and the class has quota, the request is satisfied imme-diately via the function call allocProc to the resource allocator,and the quota is updated accordingly. If the request can’t besatisfied immediately, it will be buffered in its queue. Whensome resource becomes available, the application callsresourceAvailable to notify GRM, which will try to satisfy asmany pending requests as possible.

It is important to mention that quota is a purely logicalconcept. Unlike the traditional resource reservation sys-tem, in our middleware the mapping of quota to physical re-source consumption need not be known. In effect, the GRMis a logical queuing, admission control, and resource alloca-tion policy interface with a backend that is capable of exe-cuting a primitive service function such as assigning arequest to a service process. The GRM generalizes the ex-pression of various resource allocation policies in a com-mon framework and makes it possible to control logicalquota allocations by simple feedback controllers such thatperformance constraints are met. The control loop is guar-anteed to converge because of the way controllers are de-signed, which is the advantage of using a control-theoreticapproach. Most important, the physical mapping of quotato actual resource consumption need not be known for cor-rect operation, which separates this approach from re-source reservation systems.

In summary, ControlWare is novel middleware that sup-ports QoS guarantees based on control theory. Although themiddleware exists, many open questions remain and manyimprovements are possible. For example, control templatesfor distributed systems, for adaptive controllers, and faultconditions are some of the interesting issues that must beanswered and then added to the ControlWare libraries.

ConclusionsIn this article, we have presented the necessary foundationsfor using a control-theoretic framework to achieve QoSguarantees in software systems. With the growing popular-ity of the Internet, providing performance guarantees inopen software systems has become increasingly important.We illustrated the successful application of the controlframework by practical examples in which guarantees wereprovided within a software service. We analyzed the generalarchitecture of software servers for purposes of feedbackcontrol. We have shown that high-performance servers canbe approximated in practice by a liquid task model. Themodel gives rise to a fluid representation of workload inwhich the server is modeled as a cascaded flow through apipeline of tanks of different capacities. We identified typicalactuators that control these flows within the server and il-lustrated software protocols that implement these actua-tors. We also presented a taxonomy of the most importantQoS assurance problems in the software literature and de-scribed how they can be translated into a feedback controlformulation. Having shown the feasibility of a control-theo-

retic formulation of software assurance problems, the feasi-bility of linear modeling, and the feasibility of actuation inpractical application scenarios, we believe that the founda-tion is now in place for applying the wealth of control-theo-retic results in the Internet application domain.

Many remaining issues and challenges warrant furtherresearch; for example, how to model nonlinearities pecu-liar to computing systems when they cannot be adequatelyhandled by the linearization techniques we employed?And how can these nonlinearities be accounted for in con-troller tuning? The most widespread nonlinearity in soft-ware systems is the inverse relation between rate anddelay that complicates the analysis of delay control loops,a key step toward reasoning about and controlling tempo-ral behavior. Although we focused on fixed-parameter con-trollers, of significant interest is the possible application ofadaptive control and robust control techniques to handleparameter variations and load uncertainties. Another in-teresting possibility is the use of a predictive controlframework, or one where queuing-theoretic prediction isintegrated with feedback-based correction to achieve thedesired software performance. Finally, further examples,theoretical foundations, experimental evidence, and prac-tical experience are needed to establish the true potentialand limitations of applying feedback performance controlto different computing systems. This remains an importantfocus for our future research.

AcknowledgmentsThis work was supported, in part, by the National ScienceFoundation under grants ANI-0105873, CCR-0093144, andCCR-0098269, by the DARPA NEST program under grantF33615-01-C-1905, and by the MURI award N00014-01-1-0576.

References[1] T.G. Robertazzi, Computer Networks and Systems: Queuing Theory and Per-formance Evaluation, 3rd ed. New York: Springer Verlag, 2000.[2] V. Paxton and S. Floyd, “Wide-area traffic: The failure of Poisson model-ing,” IEEE/ACM Trans. Networking, vol. 3, pp. 226-244, June 1995.[3] M.E. Crovella and A. Bestavros, “Self-similarity in World Wide Web traffic:Evidence and possible causes,” IEEE/ACM Trans. Networking, vol. 5, pp. 55-79,Dec. 1997.[4] Y. Lu, A. Sexana, and T. Abdelzaher, “Differentiated caching services; Acontrol-thoeretical approach,” in Proc. 2001 Int. Conf. Distributed ComputingSystems, 2001, pp. 615-622.[5] C. V. Hollot, V. Misra, D. Towsley, and W. Gong, “A control theoretic analysisof red,” in Proc. IEEE Infocom, Anchorage, AK, Apr. 2001, pp. 1510-1519.[6] S. Floyd and V. Jacobson, “Random early detection gateways for conges-tion avoidance,” IEEE/ACM Trans. Networking, vol. 1, no. 4, pp. 397-413, Aug.1993.[7] Y. Lu, T. Abdelzaher, C. Lu, and G. Tao, “An adaptive control framework andits application to differentiated caching services,” in Proc. Int. Conf. Quality ofService, Miami Beach, FL, May 2002, pp. 23-32.[8] J.A. Stankovic, T. He, T.F. Abdelzaher, M. Marley, G. Tao, S.H. Son, and C. Lu,“Feedback control scheduling in distributed systems,” in Proc. IEEEReal-Time Systems Symp., London, U.K., Dec. 2001, pp. 59-70.[9] C. Lu, J.A. Stankovic, G. Tao, and S.H. Son, “Feedback control real-timescheduling: Framework, modeling, and algorithms,” J. Real-Time Syst., vol. 23,no. 1/2, May 2002.[10] T.F. Abdelzaher, K.G. Shin, and N. Bhatti, “Performance guarantees forWeb server end-systems: A control-theoretical approach,” IEEE Trans. Paral-lel Distrib. Syst., vol. 13, pp. 80-96, Jan. 2002.[11] C. Aurrecoechea, A. Cambell, and L. Hauw, “A survey of QoS architec-tures,” in Proc. 4th IFIP Int. Conf. Quality of Service, Paris, France, Mar. 1996, pp.138-151.


[12] T.F. Abdelzaher and N. Bhatti, “Web server QoS management by adaptivecontent delivery,” in Proc. Int. Workshop on Quality of Service, London, U.K.,June 1999, pp. 216-225.[13] C. Dovrolis, D. Stiliadis, and P. Ramanathan, “Proportional differentiatedservices: Delay differentiation and packet scheduling,” in Proc. SIGCOMM,1999, pp. 109-120.[14] C. Dovrolis and P. Ramanathan, “Proportional differentiated services,Part II: Loss rate differentiation and packet dropping,” in Proc. Int. WorkshopQuality of Service, Pittsburgh, PA, June 2000, pp. 53-61.[15] K.J. Åström and T. Hägglund, PID Controllers: Theory, Design, and Tuning,2nd ed. Instrument Society of America, 1995.[16] C. Lu, T. Abdelzaher, J. Stankovic, and S. Son, “A feedback control ap-proach for guaranteeing relative delays in Web servers,” in Proc. IEEEReal-Time Technology and Applications Symp., Taipei, Taiwan, June 2001, pp.51-62.[17] N. Christin, J. Liebeherr, and T.F. Abdelzaher, “A quantitative assured for-warding service,” in Proc. IEEE Infocom, New York, NY, June 2002, pp. 864-873.[18] W.A. McEachern, Economics, 5th ed. South-Western College Publishing,1999.[19] P. Barford and M.E. Crovella, “Generating representative Web workloadsfor network and server performance evaluation,” in Proc. Performance‘98/ACM SIGMETRICS ‘98, Madison, WI, 1998, pp. 151-160.[20] T.F. Abdelzaher, “An automated profiling subsystem for QoS-aware ser-vices,” in Proc. IEEE Real-Time Technology and Applications Symp., Washing-ton, D.C., June 2000, pp. 208-217.[21] Y. Diao, N. Gandhi, J.L. Hellerstein, S. Parekh, and D.M. Tilbury, “MIMOcontrol of an Apache Web server: Modeling and controller design,” in Proc.American Control Conf., Anchorage, AK, May 2002, pp. 4922-4927.[22] S. Parekh, N. Gandhi, J. Hellerstein, D. Tilbury, T. Jayram, and J. Bigus,“Using control theory to achieve service level objectives in performancemanagement,” in IFIP/IEEE Int. Symp. Integrated Network Management, Seat-tle, WA, May 2001, pp. 841-854.[23] Y. Diao, J.L. Hellerstein, and S. Parekh, “Using fuzzy control to maximizeprofits in service level management,” IBM Syst. J., vol. 41, no. 3, pp. 403-420,2002.[24] D.C. Steere, A. Goel, J. Gruenberg, D. McNamee, C. Pu, and J. Walpole, “Afeedback-driven proportion allocator for real-rate scheduling,” in OperatingSystems Design and Implementation, 1999, pp. 145-158.[25] C. Lu, J.A. Stankovic, G. Tao, and S.H. Son, “The design and evaluation of afeedback EDF control scheduling algorithm,” in Proc. IEEE Real-Time SystemsSymp., Phoenix, AZ, Dec. 1999, pp. 56-67.[26] C. Lu, G.A. Alvarez, and J. Wilkes, “Aqueduct: Online data migration withperformance guarantees,” in Proc. USENIX Conf. File and Storage Technolo-gies, Monterey, CA, Jan. 2002, pp. 219-230.[27] B. Li and K. Nahrstedt, “A control-based middleware framework for qual-ity of service adaptations,” IEEE J. Select. Areas Commun., vol. 17, pp.1632-1650, Sept. 1999.[28] K. Skadron, T. Abdelzaher, and M. Stan, “Control-theoretic techniquesand thermal RC modeling for accurate and localized dynamic thermalmangement,” in Proc. Int. Symp. High Performance Computer Architecture,Cambridge, MA, Feb. 2002, pp. 17-28.[29] R. Zhang, C. Lu, T.F. Abdelzaher, and J.A. Stankovic, “Controlware: Amiddleware architecture for feedback control of software performance,” inProc. 2002 Int. Conf. Distributed Computing Systems, Vienna, Austria, July 2002,pp. 301-310.[30] A. Chatha, “Fieldbus: The foundation for field control systems,” Contr.Eng., vol. 41, no. 6, pp. 47-50, May 1994.

Tarek F. Abdelzaher received his B.Sc. and M.Sc. degreesin electrical and computer engineering from Ain Shams Uni-versity, Cairo, Egypt, in 1990 and 1994, respectively. He re-ceived his Ph.D. in 1999 from the University of Michigan incomputer science with a specialization in real-time systems.He is an assistant professor in the Department of ComputerScience at the University of Virginia. His current research in-terests include real-time systems, sensor networks, qual-ity-of-service control, networking, multimedia applications,next-generation Web architecture, fault tolerance, and de-pendable computing. He is the author and co-author of morethan 45 refereed publications. He received the Distin-guished Achievement Award in Computer Science and Engi-

neering from the University of Michigan in 1999 and the NSFCAREER Award in 2001. He is a co-editor of IEEE DistributedSystems Online and a guest editor of Computer Communica-tion and the Journal of Real-Time Systems. He has also servedon many conference committees and is the designated in-ventor of a European patent on adaptive Web servers.

John A. Stankovic is the BP America Professor and Chair ofthe Computer Science Department at the University of Vir-ginia. He is a Fellow of both the IEEE and the ACM. He re-ceived the IEEE Real-Time Systems Technical Committee'sAward for Outstanding Technical Contributions and Leader-ship. He also serves as treasurer for the Board of Directorsof the Computer Research Association. Before joining theUniversity of Virginia, he taught at the University of Massa-chusetts, where he won an outstanding scholar award. Hehas also held visiting positions in the Computer Science De-partment at Carnegie Mellon University, at INRIA in France,and Scuola Superiore S. Anna in Pisa, Italy. He was the edi-tor-in-chief for IEEE Transactions on Distributed and ParallelSystems and is a co-editor-in-chief for the Real-Time SystemsJournal. His research interests are in distributed computing,real-time systems, operating systems, and ad hoc networksfor pervasive computing. He received his Ph.D. from BrownUniversity.

Chenyang Lu is an assistant professor in the Department ofComputer Science and Engineering at Washington Univer-sity in St. Louis. He received a B.S. in computer science fromthe University of Science and Technology of China in 1995, aMaster’s in computer science from the Chinese Academy ofSciences in 1997, and a Ph.D. in computer science from theUniversity of Virginia in 2001. His current research interestsinclude real-time systems and middleware, wireless sensornetworks, Internet servers, and network storage. A theme ofhis research is developing adaptive distributed softwaresystems that provide performance-assured services in un-predictable environments.

Ronghua Zhang received a B.S. degree from Huazhong Uni-versity of Science and Technology, Wuhan, China, and anM.S. degree in computer science from the Institute of Soft-ware, Chinese Academy of Sciences, Beijing, China. Cur-rently, he is working toward his Ph.D. in computer science atthe University of Virginia, Charlottesville.

Ying Lu received her B.S. in computer science from South-west Jiaotong University, Chengdu, China, in 1996, and herM.S. in computer science from Jinan University, Guangzhou,China, in 1999. She is currently working on her Ph.D. degreein computer science at the University of Virginia. Her re-search interests include applying control theory toInternet-based services, adaptive QoS architecture design,content distribution networks, and storage systems.


by tarek f. abdelzaher, john a. stankovic, chenyang lu ...lu/papers/control03.pdf · time-related...

Documents