jaws: a framework for high-performance web servers · web serversusing ooapplicationframeworks...

JAWS: A Framework for High-performance Web Servers

James C. [email protected]

�Douglas C. [email protected]

Department of Computer ScienceWashington UniversitySt. Louis, MO 63130

Abstract

Developers of communication software face many challenges.Communication software contains both inherent complexities,such as fault detection and recovery, and accidental complex-ities, such as the continuous re-rediscovery and re-inventionof key concepts and components. Meeting these challengesrequires a thorough understanding of object-oriented appli-cation frameworks and patterns. This paper illustrates howwe have applied frameworks and patterns for communicationsoftware to develop a high-performance Web server calledJAWS.

JAWS is an object-oriented framework that supports theconfiguration of various Web server strategies, such as aThread Pool concurrency model with asynchronous I/O andLRU caching vs.. a Thread-per-Request concurrency modelwith synchronous I/O and LFU caching. Because JAWS is aframework, these strategies can be customized systematicallyand measured both independently and collaboratively to de-termine the best strategy profiles. Using these profiles, JAWScan statically and dynamically adapt its behavior to deploy themost effective strategies for a given software/hardware plat-form and workload. JAWS’ adaptive software features makeit a powerful application framework for constructing high-performance Web servers.

1 Introduction

During the past several years, the volume of traffic on theWorld Wide Web (Web) has grown dramatically. Traffic in-creases are due largely to the proliferation of inexpensive andubiquitous Web browsers such as NCSA Mosaic, NetscapeNavigator, and Internet Explorer. Likewise, Web protocols andbrowsers are increasingly applied to specialized computation-ally expensive tasks, such as image processing servers used by

�The work described here was made possible by funds from Siemens AG,Eastman Kodak, and NSF Research Grant NCR-9628218.

Siemens [1] and Kodak [2] and database search engines suchas AltaVista and Lexis-Nexis.

To keep pace with increasing demand, it is essential to de-velop high-performance Web servers. However, developersface a remarkably rich set of design strategies when config-uring and optimizing Web servers. For instance, developersmust select from among a wide range of concurrency mod-els (such as Thread-per-Requestvs.Thread Pool), dispatchingmodels (such as synchronous vs. asynchronous dispatching),file caching models (such as LRU vs. LFU), and protocol pro-cessing models (such as HTTP/1.0 vs. HTTP/1.1). No sin-gle configuration is optimal for all hardware/software platformand workloads [1, 3].

The existence of all these alternative strategies ensures thatWeb servers can be tailored to their users’ needs. However,navigating through many design and optimization strategies istedious and error-prone. Without guidance on which designand optimization strategies to apply, developers face the her-culean task of engineering Web servers from the ground up,resulting inad hocsolutions. Such systems are often hard tomaintain, customize, and tune since much of the engineeringeffort is spent just trying to get the system operational.

1.1 Definitions

We will frequently refer to the termsOO class library, frame-work, pattern, andcomponent. These terms refer to tools thatare used to build reusable software systems. An OO classlibarary is a collection of software object implemetations thatprovide reusable functionality as the user calls into the objectmethods. A framework is a reusable, “semi-complete” appli-cation that can be specialized to produce custom applications[4]. A pattern represents a recurring solution to a softwaredevelopment problem within a particular context [5]. A com-ponent refers to a “reifiable” object. Both OO class librariesand frameworks are collections of components that are reifiedby instantiation and specialization. A pattern component isreified through codification.

1

1.2 Overview

This paper illustrates how to produce flexible and efficientWeb servers using OOapplication frameworksanddesign pat-terns. Patterns and frameworks can be applied synergisticallyto improve the efficiency and flexibility of Web servers. Pat-terns capture abstract designs and software architectures ofhigh-performance and adaptive Web servers in a systematicand comprehensible format. Frameworks capture the concretedesigns, algorithms, and implementations of Web servers ina particular programming language, such as C++ or Java. Incontrast,OO class librariesprovide the raw materials neces-sary to build applications, but no guidance as to how to put thepieces together.

This paper focuses on the patterns and framework compo-nents used to develop a high-performance Web server calledJAWS [1, 3]. JAWS is both a Web server and a frameworkfrom which other types of servers can be built. The JAWSframework itself was developed using the ACE framework[6, 7]. The ACE framework reifies key patterns [5] in the do-main of communication software. The framework and patternsin JAWS and ACE are representative of solutions that havebeen applied successfully to communication systems rangingfrom telecommunication system management [8] to enterprisemedical imaging [2] and real-time avionics [9].

This paper is organized as follows: Section 2 presents anoverview of patterns and frameworks and motivates the needfor the type of communication software framework providedby JAWS; Section 3 illustrates how patterns and componentscan be applied to develop high-performance Web servers; Sec-tion 4 compares the performance of JAWS over high-speedATM networks with other high-performance Web servers; andSection 5 presents concluding remarks.

2 Applying Patterns and Frameworksto Web Servers

There is increasing demand for high-performance Web serversthat can provide services and content to the growing number ofInternet and intranet users. Developers of Web servers strive tobuild fast, scalable, and configurable systems. This task can bedifficult, however, if care is not taken to avoid common trapsand pitfalls, which include tedious and error-prone low-levelprogramming details, lack of portability, and the wide rangeof design alternatives. This section presents a road-map tothese hazards. We then describe how patterns and frameworkscan be applied to avoid these hazards by allowing developersto leverage both design and code reuse.

2.1 Common Pitfalls of Developing Web ServerSoftware

Developers of Web servers confront recurring challenges thatare largely independent of their specific application require-ments. For instance, like other communication software, Webservers must perform various tasks related to the following:connection establishment, service initialization, event demul-tiplexing, event handler dispatching, interprocess communi-cation, memory management and file caching, static and dy-namic component configuration, concurrency, synchroniza-tion, and persistence. In most Web servers, these tasks areimplemented in anad hocmanner using low-level native OSapplication programming interfaces (APIs), such as the Win32or POSIX, which are written in C.

Unfortunately, native OS APIs are not an effective way todevelop Web servers or other types of communication middle-ware and applications [10]. The following are common pitfallsassociated with the use of native OS APIs:

Excessive low-level details: Building Web servers with na-tive OS APIs requires developers to have intimate knowl-edge of low-level OS details. Developers must carefully trackwhich error codes are returned by each system call and han-dle these OS-specific problems in their server. Such detailsdivert attention from the broader, more strategic issues, suchas semantics and program structure. For example, UNIX de-velopers who use thewait system call must distinguish thebetween return errors due to no child processes being presentand errors from signal interrupts. In the latter case, thewaitmust be reissued.

Continuous rediscovery and reinvention of incompatiblehigher-level programming abstractions: A common rem-edy for the excessive level of detail with OS APIs is to definehigher-level programming abstractions. For instance, manyWeb servers create a file cache to avoid accessing the filesys-tem for each client request. However, these types of abstrac-tions are often rediscovered and reinvented independently byeach developer or project. Thisad hocprocess hampers pro-ductivity and creates incompatible components that are notreadily reusable within and across projects in large softwareorganizations.

High potential for errors: Programming to low-level OSAPIs is tedious and error-prone due to their lack of type-safety. For example, most Web servers are programmed withthe Socket API [11]. However, endpoints of communicationin the Socket API are represented as untyped handles. Thisincreases the potential for subtle programming mistakes andrun-time errors.

Lack of portability: Low-level OS APIs are notoriouslynon-portable, even across releases of the same OS. For in-

2

stance, implementations of the Socket API on Win32 plat-forms (WinSock) are subtly different than on UNIX plat-forms. Moreover, even WinSock implementations on differ-ent versions of Windows NT possess incompatible timing-related bugs that cause sporadic failures when performing non-blocking connections.

Steep learning curve: Due to the excessive level of detail,the effort required to master OS-level APIs can be very high.For instance, it is hard to learn how to program with POSIXasynchronous I/O [12] correctly. It is even harder to learn howto write aportableapplication using asynchronous I/O mech-anisms since they differ widely across OS platforms.

Inability to handle increasing complexity: OS APIs de-fine basic interfaces to mechanisms like process and threadmanagement, interprocess communication, file systems, andmemory management. However, these basic interfaces do notscale up gracefully as applications grow in size and complex-ity. For instance, a typical UNIX process allows a backlog ofonly�7 pending connections [13]. This number is inadequatefor heavily accessed Web servers that must handle hundreds ofsimultaneous clients.

2.2 Overcoming Web Server Pitfalls with Pat-terns and Frameworks

Software reuse is a a widely touted method of reducing de-velopment effort. Reuse leverages the domain knowledge andprior effort of experienced developers. When applied effec-tively, reuse can avoid re-creating and re-validating commonsolutions to recurring application requirements and softwaredesign challenges.

Java’sjava.lang.net and RogueWaveNet.h++ aretwo common examples of applying reusable OO class librariesto communication software. Although class libraries effec-tively support component reuse-in-the-small, their scope isoverly constrained. In particular, class libraries do not capturethe canonical control flow and collaboration among familiesof related software components. Thus, developers who applyclass library-based reuse often re-invent and re-implement theoverall software architecture for each new application.

A more powerful way to overcome the pitfalls describedabove is to identify thepatterns that underlie proven Webservers and to reify these patterns inobject-oriented appli-cation frameworks. Patterns and frameworks help alleviatethe continual re-discovery and re-invention of key Web serverconcepts and components by capturing solutions to commonsoftware development problems [5].

The benefits of patterns for Web servers: Patterns doc-ument the structure and participants in common Web servermicro-architectures. For instance, the Reactor [14] and Active

Object [15] patterns are widely used as Web server dispatch-ing and concurrency strategies, respectively. These patternsare generalizations of object-structures that have proven use-ful to build flexible and efficient Web servers.

Traditionally, these types of patterns have either been lockedin the heads of the expert developers or buried deep withinthe source code. Allowing this valuable information to resideonly in these locations is risky and expensive, however. For in-stance, the insights of experienced Web server designers willbe lost over time if they are not documented. Likewise, sub-stantial effort may be necessary to reverse engineer patternsfrom existing source code. Therefore, capturing and docu-menting Web server patterns explicitly is essential to preservedesign information for developers who enhance and maintainexisting software. Moreover, knowledge of domain-specificpatterns helps guide the design decisions of developers whoare building new servers in other domains.

The benefits of frameworks for Web servers: Knowledgeof patterns helps to reduce development effort and mainte-nance costs. However, reuse of patterns alone is not suf-ficient to create flexible and efficient Web server software.While patterns enable reuse of abstract design and architec-ture knowledge, abstractions documented as patterns do notdirectly yield reusable code [16]. Therefore, it is essential toaugment the study of patterns with the creation and use ofapplication frameworks. Frameworks help developers avoidcostly re-invention of standard Web server components by im-plementing common design patterns and factoring out com-mon implementation roles.

2.3 Relationship Between Frameworks, Pat-terns, and Other Reuse Techniques

Frameworks provide reusable software components for ap-plications by integrating sets of abstract classes and definingstandard ways that instances of these classes collaborate [4].In general, the components are not self-contained, since theyusually depend upon functionality provided by other compo-nents within the framework. However, the collection of thesecomponents forms a partial implementation,i.e., an applica-tion skeleton. This skeleton can be customized by inheritingand instantiating from reusable components in the framework.

The scope of reuse in a Web server framework can be sig-nificantly larger than using traditional function libraries or OOclass libraries of components. In particular, the JAWS frame-work described in Section 3 is tailored for a wide range of Webserver tasks. These tasks include service initialization, errorhandling, flow control, event processing, file caching, concur-rency control, and prototype pipelining. It is important to rec-ognize that these tasks are also reusable for many other typesof communication software.

3

In general, frameworks and patterns enhance reuse tech-niques based on class libraries of components in the followingways.

Frameworks define “semi-complete” applications that em-body domain-specific object structures and functionality:Class libraries provide a relatively small granularity of reuse.For instance, the classes in Figure 1 are typically low-level,relatively independent, and general-purpose components likeStrings, complex numbers, arrays, and bit sets.

INVOCATIONS

SELECTIONS

EVENTLOOP

OO DESIGN

APPLICATIONSPECIFIC LOGIC

Reactor Adapter

State Active Object

Singleton Strategy

DATABASE ADTs

MATH NETWORKING

GRAPHICS GUI

DESIGN PATTERNS

CLASS LIBRARIES

Figure 1: Class Library Component Architecture

In contrast, components in a framework collaborate to pro-vide a customizable architectural skeleton for a family of re-lated applications. Complete applications can be composed byinheriting from and/or instantiating framework components.As shown in Figure 2, frameworks reduce the amount ofapplication-specific code since much of the domain-specificprocessing is factored into generic framework components.

Active Object

Singleton Adapter

State

ReactorMATH

ADTs

GRAPHICS

LOOP

GUI

OO DESIGN

INVOCATIONS

CALLBACKS

DATABASE

EVENT

NETWORKING

LOGIC

APPLICATIONSPECIFIC

Figure 2: Application Framework Component Architecture

Frameworks are active and exhibit “inversion of control”at run-time: Class library components generally behavepassively. In particular, class library components often per-form their processing by borrowing the thread(s) of controlfrom application objects that are “self-directed.” Because ap-plication objects are self-directed, application developers arelargely responsible for deciding how to combine the compo-nents and classes to form a complete system. For instance,the code to manage an event loop and determine the flow ofcontrol among reusable and application-specific componentsis generally rewritten for each new application.

The typical structure and dynamics of applications builtwith class libraries and components is illustrated in Figure 1.This figure also illustrates how design patterns can help guidethe design, implementation, and use of class library compo-nents. Note, that the existence of class libraries, while provid-ing tools to solve particular tasks (e.g., establishing a networkconnection) do not offer explicit guidance to system design. Inparticular, software developers are solely responsible for iden-tifying and applying patterns in designing their applications.

In contrast to class libraries, components in a frameworkare moreactive. In particular, they manage the canonical flowof control within an application via event dispatching patternslike Reactor [14] and Observer [5]. The callback-driven run-time architecture of a framework is shown in Figure 2.

Figure 2 illustrates a key characteristic of a framework: its“inversion of control” at run-time. This design enables thecanonical application processing steps to be customized byevent handler objects that are invoked via the framework’sreactive dispatching mechanism [14]. When events occur,the framework’s dispatcher reacts by invoking hook methodson pre-registered handler objects, which perform application-specific processing on the events.

Inversion of control allows the framework, rather than eachapplication, to determine which set of application-specificmethods to invoke in response to external events (such asHTTP connections and data arriving on sockets). As a re-sult, the framework reifies an integrated set of patterns, whichare pre-applied into collaborating components. This designreduces the burden for software developers.

In practice, frameworks, class libraries, and components arecomplementary technologies. Frameworks often utilize classlibraries and components internally to simplify the develop-ment of the framework. For instance, portions of the JAWSframework use the string and vector containers provided bythe C++ Standard Template Library [17] to manage connec-tion maps and other search structures. In addition, application-specific callbacks invoked by framework event handlers fre-quently use class library components to perform basic taskssuch as string processing, file management, and numericalanalysis.

4

To illustrate how OO patterns and frameworks have beenapplied successfully to develop flexible and efficient commu-nication software, the remainder of this paper examines thestructure, use, and performance of the JAWS framework.

3 The JAWS Adaptive Web Server

The benefits of applying frameworks and patterns to commu-nication software are best illustrated by example. This sectiondescribes the structure and functionality of the JAWS. JAWS isa high-performance and adaptive Web server that implementsthe HTTP protocol. It is also a platform-independent appli-cation framework from which other types of communicationservers can be built.

3.1 Overview of the JAWS Framework

Figure 3 illustrates the major structural components and de-sign patterns that comprise the JAWS Adaptive Web Server(JAWS) framework. JAWS is designed to allow various Web

Cached VirtualFilesystem

Expander

Framework

I/O Strategy

StrategyConcurrency

Protocol PipelineFramework

Framework

Tilde ~/home/...

Event Dispatcher

Acceptor

Asynchronous Completion Token

Reactor/Proactor

Pipes and Filters

Stat

e

Singleton

HandlerProtocol

ProtocolFilter

Adapter

Strategy

Service Configurator

Active Object

Serv

ice

Con

figu

rato

rSt

ate

Mem

ento

Strategy

Figure 3: Architectural Overview of the JAWS Framework

server strategies to be customized in response to environmen-tal factors. These factors includestaticfactors, such as supportfor kernel-level threading and/or asynchronous I/O in the OS,and the number of available CPUs, as well asdynamicfactors,such as Web traffic patterns and workload characteristics.

JAWS is structured as aframework of frameworks. Theoverall JAWS framework contains the following componentsand frameworks: anEvent Dispatcher, Concurrency Strat-egy, I/O Strategy, Protocol Pipeline, Protocol Handlers, andCached Virtual Filesystem. Each framework is structured asa set of collaborating objects implemented using components

in ACE [18]. The collaborations among JAWS componentsand frameworks are guided by a family of patterns, which arelisted along the borders in Figure 3. An outline of the keyframeworks, components, and patterns in JAWS is presentedbelow. A more detailed description of how these patterns havebeen applied to JAWS’ design will be shown in Section 3.2.

Event Dispatcher: This component is responsible for co-ordinating JAWS’Concurrency Strategywith its I/O Strategy.The passive establishment of connections with Web clients fol-lows theAcceptorpattern [19]. New incoming requests areserviced by a concurrency strategy. As events are processed,they are dispatched to theProtocol Handler, which is parame-terized by an I/O strategy. The ability to dynamically bind to aparticular concurrency strategy and I/O strategy from a rangeof alternatives follows theStrategypattern [5].

Concurrency Strategy: This framework implements con-currency mechanisms (such as Single Threaded, Thread-per-Request, or Thread Pool) that can be selected adaptivelyat run-time using theStatepattern [5] or pre-determined atinitialization-time. TheService Configuratorpattern [20] isused to configure a particular concurrency strategy into aWeb server at run-time. When concurrency involves multi-ple threads, the strategy creates protocol handlers that followtheActive Objectpattern [15].

I/O Strategy: This framework implements various I/Omechanisms, such as asynchronous, synchronous and reac-tive I/O. Multiple I/O mechanisms can be used simultaneously.Asynchronous I/O is implemented via theProactor [21] andAsynchronous Completion Token[22] patterns. Reactive I/Ois accomplished through theReactorpattern [14]. ReactiveI/O utilizes theMementopattern [5] to capture and externalizethe state of a request so that it can be restored at a later time.

Protocol Handler: This framework allows system develop-ers to apply the JAWS framework to a variety of Web systemapplications. AProtocol Handleris parameterized by a con-currency strategy and an I/O strategy. These strategies remainopaque to the protocol handler by following theAdapter[5]pattern. In JAWS, this component implements the parsing andhandling of HTTP/1.0 request methods. The abstraction al-lows for other protocols (such as HTTP/1.1 and DICOM) to beincorporated easily into JAWS. To add a new protocol, devel-opers simply write a newProtocol Handlerimplementation,which is then configured into the JAWS framework.

Protocol Pipeline: This framework allows filter operationsto be incorporated easily with the data being processed by theProtocol Handler. This integration is achieved by employingthe Adapter pattern. Pipelines follow thePipes and Filterspattern [23] for input processing. Pipeline components can belinked dynamically at run-time using theService Configuratorpattern.

5

Cached Virtual Filesystem: The component improves Webserver performance by reducing the overhead of filesystem ac-cesses. Various caching strategies, such as LRU, LFU, Hinted,and Structured, can be selected following theStrategypattern[5]. This allows different caching strategies to be profiled foreffectiveness and enables optimal strategies to be configuredstatically or dynamically. The cache for each Web server isinstantiated using theSingletonpattern [5].

Tilde Expander: This component is another cache compo-nent that uses a perfect hash table [24] that maps abbrevi-ated user login names (e.g.,�schmidt ) to user home direc-tories (e.g., /home/cs/faculty/schmidt ). When per-sonal Web pages are stored in user home directories, and userdirectories do not reside in one common root, this componentsubstantially reduces the disk I/O overhead required to accessa system user information file, such as/etc/passwd . Byvirtue of theService Configuratorpattern, the Tilde Expandercan be unlinked and relinked dynamically into the server whena new user is added to the system, for example.

3.2 Overview of the Design Patterns in JAWS

The JAWS architecture diagram in Figure 3 illustrateshowJAWS is structured, but notwhy it is structured this particularway. To understand why JAWS contains frameworks and com-ponents like theConcurrency Strategy, I/O Strategy, ProtocolHandler, andEvent Dispatcherrequires a deeper understand-ing of the design patterns underlying the domain of commu-nication software, in general, and Web servers, in particular.Figure 4 illustrates thestrategicandtacticalpatterns related toJAWS. These patterns are summarized below.

3.2.1 Strategic Patterns

The following patterns arestrategicto the overall software ar-chitecture of Web servers. Their use widely impacts the levelof interactions of a large number of components in the system.These patterns are also widely used to guide the architectureof many other types of communication software.

notifies

Acceptor

accept()peer_acceptor_

Protocol Handlercreate

+ activateProtocol Handlerpeer_stream_open()

ConcurrencyEvent Dispatcher

Task

Figure 5: Structure of the Acceptor Pattern in JAWS

The Acceptor pattern: This pattern decouples passive con-nection establishment from the service performed once theconnection is established [19]. JAWS uses the Acceptor pat-tern to adaptively change its concurrency and I/O strategiesindependently from its connection management strategy. Fig-ure 5 illustrates the structure of the Acceptor pattern in thecontext of JAWS. TheAcceptoris a factory [5]. It creates,accepts, and activates a newProtocol Handlerwhenever theEvent Dispatchernotifies it that a connection has arrived froma client.

The Reactor pattern: This pattern decouples the syn-chronous event demultiplexing and event handler notificationdispatching logic of server applications from the service(s)performed in response to events [14]. JAWS uses the Reactorpattern to process multiple synchronous events from multiplesources of events,withoutpolling all event sources or blockingindefinitely on any single source of events. Figure 6 illustratesthe structure of the Reactor pattern in the context of JAWS.

Timer Queueschedule_timer(h)cancel_timer(h)expire_timers(h)

Reactive IO

InputOutput InputOutput Handler Protocol Pipeline

Reactor Reactive IO Handlerhandle_input()handle_output()

IO Handle

Initiation Dispatcherhandle_events()register_handler(h)remove_handler(h)

Sync

hron

ous

IO

Asy

nchr

onou

s IO

send_file()recv_file()send_data()recv_data()

Filecache Handle

Figure 6: Structure of the Reactor Pattern in JAWS

JAWSReactive IO Handlerobjects register themselves withthe Initiation Dispatcher for events, i.e., input and outputthrough connections established by HTTP requests. TheIni-tiation Dispatcherinvokes thehandle input notificationhook method of theseReactive IO Handlerobjects when theirassociated events occur. The Reactor pattern is used by theSingle-Threaded Web server concurrency model presented inSection 3.3.2.

The Proactor pattern pattern: This pattern decouples theasynchronous event demultiplexing and event handler com-pletion dispatching logic of server applications from the ser-vice(s) performed in response to events [21]. JAWS uses theProactor pattern to perform server-specific processing, such asparsing the headers of a request, while asynchronously pro-cessing other I/O events. Figure 7 illustrates the structure ofthe Proactor pattern in the context of JAWS. JAWSProactiveIO Handler objects register themselves with theCompletion

6

Singleton

Pipes and Filters

Acceptor


Adapter

State



State

Strategy

Memento

Reactor

Adapter

Proactor


Active Object

Concurrency StrategyFramework

Protocol Pipeline StrategyFramework

I/O Strategy FrameworkCached Virtual Filesystem

Framework

Figure 4: Design Patterns Used in the JAWS Framework

Asynch Opopen()cacnel()

expire_timers(h)

Timer Queue

IO Handle

cancel_timer(h)schedule_timer(h)

Proactor

Completion Dispatcherhandle_events()register_handler(h)remove_handler(h)

send_file()recv_file()send_data()recv_data()

Proactive IO Handlerhandle_read_file()handle_write_file()

Filecache Handle

Sync

hron

ous

IO

Rea

ctiv

e IO

InputOutput InputOutput Handler Protocol Pipeline

Asynchronous IO

Figure 7: Structure of the Proactor Pattern in JAWS

Dispatcherfor events (i.e., receipt and delivery of files throughconnections established by HTTP requests).

The primary difference between the Reactor and Proactorpatterns is that theProactive IO Handlerdefinescompletionhooks, whereas theReactive IO Handlerdefinesinitiationhooks. Therefore, when asynchronously invoked operationslike recv file or send file complete, theCompletionDispatcherinvokes the appropriate completion hook methodof theseProactive IO Handlerobjects. The Proactor pattern isused by the asynchronous variant of the Thread Pool in Sec-tion 3.3.2.

The Active Object pattern: This pattern decouples methodinvocation from method execution, allowing methods to runconcurrently [15]. JAWS uses the Active Object pattern to exe-cute client requests concurrently in separate threads of control.Figure 8 illustrates the structure of the Active Object patternin the context of JAWS.

snmp_request()dispatch()

Resource Representation

Protocol Pipeline

ftp_request()

for (;;) {

}

m = aq->remove();dispatch(m);

http_request()

Scheduler

aq->insert(http)

Protocol Handlerhttp_request()

snmp_request()ftp_request()

Activation Queueinsert()remove()

Method Object

delegates

Figure 8: Structure of the Active Object Pattern in JAWS

The Protocol Handler issues requests to theScheduler,which transforms the request method (such as an HTTP re-quest) intoMethod Objectsthat are stored on anActivationQueue. TheScheduler, which runs in a separate thread from

7

the client, dequeues theseMethod Objectsand transformsthem back into method calls to perform the specified protocol.The Active Object pattern is used in the Thread-per-Request,Thread Pool, and Thread-per-Session concurrency models de-scribed in Section 3.3.2.

The Service Configurator pattern: This pattern decouplesthe implementation of individual components in a system fromthetimewhen they are configured into the system. JAWS usesthe Service Configurator pattern to dynamically optimize, con-trol, and reconfigure the behavior of Web server strategies atinstallation-time or during run-time [25]. Figure 9 illustratesthe structure of the Service Configurator pattern in the contextof theProtocol Pipeline FiltersandCaching Strategies.

Service Repositoryservices

LRU Strategy LFU Strategy

Protocol Pipeline

Filter Repository

Cache Strategy Repository

Serviceinit()fini()suspend()resume()info()

Filecache

Protocol Handler

...

FilterRead Request

Parse Request Log Request

DLL Cache Strategy...

Figure 9: Structure of the Service Configurator Pattern inJAWS

The figure depicts how theService Configuratorcan dy-namically manageDLLs, which are dynamically linked li-braries. This allows the framework to dynamically configuredifferent implementations of server strategies at run-time. TheFilter RepositoryandCache Strategy Repositoryinherit func-tionality from theService Repository. Likewise, the strategyimplementations (such asParse Requestand LRU Strategy)borrow interfaces from theServicecomponent of the patternso that they can be managed dynamically by the repository.

3.2.2 Tactical Patterns

Web servers also utilize manytactical patterns, which aremore ubiquitous and domain-independent than the strategicpatterns described above. The following tactical patterns areused in JAWS:

The Strategy pattern: This pattern defines a family of algo-rithms, encapsulates each one, and make them interchangeable

[5]. JAWS uses this pattern extensively to selectively config-ure different cache replacement strategies without affecting thecore software architecture of the Web server.

The Adapter pattern: This pattern transforms a non-conforming interface into one that can be used by a client[5]. JAWS uses this pattern in its I/O Strategy Framework touniformly encapsulate the operations of synchronous, asyn-chronous and reactive I/O operations.

The State pattern: This pattern defines a composite ob-ject whose behavior depends upon its state [5]. TheEventDispatcher component in JAWS uses the State pattern toseamlessly support different concurrency strategies and bothsynchronous and asynchronous I/O.

The Singleton pattern: This pattern ensures a class onlyhas one instance and provides a global point of access to it[5]. JAWS uses a Singleton to ensure that only one copy of itsCached Virtual Filesystem exists in a Web server process.

In contrast to the strategic patterns described earlier, tacticalpattern have a relatively localized impact on a software de-sign. For instance, Singleton is a tactical pattern that is oftenused to consolidate certain globally accessible resources in aWeb server. Although this pattern is domain-independent andthus widely applicable, the problem it addresses does not im-pact Web server software architecture as pervasively as strate-gic patterns like the Active Object and Reactor. A thoroughunderstanding of tactical patterns is essential, however, to im-plement highly flexible software that is resilient to changes inapplication requirements and platform characteristics.

The remainder of this section discusses the structure ofJAWS’ frameworks for concurrency, I/O, protocol pipelining,and file caching. For each framework, we describe the key de-sign challenges and outline the range of alternatives solutionstrategies. Then, we explain how each JAWS framework isstructured to support the configuration of alternative strategyprofiles.

3.3 Concurrency Strategies

3.3.1 Design Challenges

Concurrency strategies impact the design and performance ofa Web system significantly. Empirical studies [3] of exist-ing Web servers, including Roxen, Apache, PHTTPD, Zeus,Netscape and the Java Web server, indicate that a large por-tion of non-I/O related Web server overhead is due to the Webserver’s concurrency strategy. Key overheads include syn-chronization, thread/process creation, and context switching.Therefore, choosing an efficient concurrency strategy is cru-cial to achieve high-performance.

8

3.3.2 Alternative Solution Strategies

Selecting the right concurrency strategy is nontrivial. Thefactors influencing the decision are both static and dynamic.Staticfactors can be determineda priori. These factors includehardware configuration (e.g., number of processors, amount ofmemory, and speed of network connection), OS platform (e.g.,availability of threads and asynchronous I/O), and Web serveruse case (e.g., database interfacing, image server, or HTMLserver). Dynamic factors are those detectable and measur-able conditions that occur during the execution of the system.These factors include machine load, number of simultaneousrequests, dynamic memory use, and server workload.

Existing Web servers use a wide range of concurrencystrategies, reflecting the numerous factors involved. Thesestrategies include single-threaded concurrency (e.g., Roxen),process-based concurrency (e.g., Apache and Zeus), andmulti-threaded concurrency (e.g., PHTTPD, and JAWS). Eachstrategy offers positive and negative benefits, which must beanalyzed and evaluated in the context of both static and dy-namic factors. These tradeoffs are summarized below.

Thread-per-Request: This model handles each requestfrom a client in a separate thread of control. Thus, as eachrequest arrives, a new thread is created to process the request.This design allows each thread to use well understood syn-chronous I/O mechanisms to read and write the requested file.Figure 10 illustrates this model in the context of the JAWSframework. Here, there is anAcceptorthread that iterateson waiting for a connection, creating aProtocol Handler, andspawning a new thread so that the handler can continue pro-cessing the connection.

The advantage of Thread-per-Request is its simplicity andits ability to exploit parallelism on multi-processor platforms.Its chief drawback is lack of scalability,i.e., the number of run-ning threads may grow without bound, exhausting availablememory and CPU resources. Therefore, Thread-per-Requestis adequate for lightly loaded servers with low latency. How-ever, it may be unsuitable for servers that are accessed fre-quently to perform time consuming tasks.

Task

Thread-per-Request

Acceptor

Event Dispatcher

Protocol Handler+ spawns

create

notifies

ProtocolHandler

ProtocolHandler

ProtocolHandler

Figure 10: Thread-per-Request Strategy in JAWS

Thread-per-Session: A sessionis a series of requests thatone client makes to a server. In Thread-per-Session, all ofthese requests are submitted through a connection betweeneach client and a separate thread in the Web server process.Therefore, this model amortizes both thread creation and con-nection establishment costs across multiple requests.

Thread-per-Session is less resource intensive than Thread-per-Request since it does not spawn a separate thread for eachrequest. However, it is still vulnerable to unbounded resourceconsumption as the number of clients grows. Also, the useof Thread-per-Session requires both the client and server tosupport the concept of re-using an established connection withmultiple requests. For example, if both the Web client andWeb server are HTTP/1.1 compliant, Thread-per-Session canbe used between them. However, if either the client or serveronly supports HTTP/1.0, then Thread-per-Session degrades toThread-per-Request [26, 27].

Thread Pool: In this model, a group of threads are pre-spawned during Web server initialization. Each thread fromthe pool retrieves a task from a job queue. While the thread isprocessing the job, it is removed from the pool. Upon finish-ing the task, the thread is returned to the pool. As illustrated inFigure 11, the job being retrieved is completion of theAccep-tor. When it completes, the thread creates aProtocol Handlerand lends its thread of control so that the handler can processthe connection.

Thread Pool has lower overhead than Thread-per-Requestsince the cost of thread creation is amortized through pre-spawning. Moreover, the number of resources the pool canconsume is bounded since the pool size is fixed. However,if the pool is too small, it can be depleted. This causes newincoming requests to be dropped or wait indefinitely. Further-more, if the pool is too large, resource consumption may be nobetter than using Thread-per-Request.

ProtocolHandler

ProtocolHandler

ProtocolHandler

+ completecreate

Protocol Handler

Event Dispatcher Thread Pool notifies

Acceptor Task

Figure 11: Thread Pool Strategy in JAWS

Single-Threaded: In this model, all connections and re-quests are handled by the same thread of control. Simple im-plementations of Single-Threaded servers process requests it-eratively. This is usually inadequate for high volume produc-

9

tion servers since subsequent requests are blocked until theyare reached, creating unacceptable delays.

More sophisticated Single-Threaded implementations at-tempt to process multiple requests concurrently using asyn-chronous or reactive I/O (which are described in Section 3.4).Single-threaded concurrency strategies can perform betterthan multi-threaded solutions on uni-processor machines thatsupport asynchronous I/O [1]. Since JAWS’ I/O frameworkis orthogonal to its concurrency framework, we consider theSingle-Threaded concurrency strategy as a special case of theThread Pool where the pool size is 1.

Experiments in [1] and [3] demonstrate that the choice ofconcurrency and event dispatching strategies significantly af-fect the performance of Web servers that experience varyingload conditions. In particular, no single server strategy pro-vides optimal performance for all cases. Thus, a server frame-work should provide at least two degrees of freedom:

1. Static adaptivity– The framework should allow the Webserver developers to select the concurrency strategy thatbest meets the static requirements of the system. For ex-ample, a multi-processor machine may be more suitedto multi-threaded concurrency than a uni-processor ma-chine.

2. Dynamic adaptivity– The framework should allow itsconcurrency strategy to adapt dynamically to currentserver conditions to achieve optimal performance in thepresence of dynamic server load conditions. For instance,in order to accommodate unanticipated load usage, it maybe necessary to increase the number of available threadsin the thread pool.

3.3.3 JAWS Concurrency Strategy Framework

As discussed above, no single concurrency strategy performsoptimally under for all conditions. However, not all platformscan make effective use of all available concurrency strate-gies. To address these issues, the JAWS Concurrency StrategyFramework supports both static and dynamic adaptivity withrespect to its concurrency and event dispatching strategies.

Figure 12 illustrates the OO design of JAWS’ Concur-rency Strategy framework. TheEvent DispatcherandCon-currency objects interact according to theState pattern.As illustrated, server can be changed to eitherThread-per-Connectionor Thread Poolbetween successive calls toserver->dispatch() , allowing different concurrencymechanisms to take effect. TheThread-per-Connectionstrat-egy is an abstraction over both the Thread-per-Request andThread-per-Session strategies discussed above. Each concur-rency mechanism makes use of aTask. Depending on thechoice of concurrency, aTaskmay represent a singleActive

acceptor->accept()

Event Dispatcherdispatch()

server

server->dispatch()

Concurrencydispatch()

task

Thread Pooldispatch()dispatch()

Thread-per-Connection

task->activate()task->accept()

task->enque()

handleracceptor

accept()

Acceptoropen()

Protocol Handler

Taskaccept()

activate()enque()deque()svc()

Figure 12: Structure of the Concurrency Strategy Framework

Object, or a collection ofActive Objects. The behavior oftheConcurrencyobject follows theAcceptorpattern. This ar-chitecture enables the server developer to integrate alternateconcurrency strategies. With the aid of a strategy profile, theserver can dynamically choose different strategies at run-timeto achieve optimal performance.

3.4 I/O Strategies


Another key challenge for Web server developers is to deviseefficient data retrieval and delivery strategies, collectively re-ferred to asI/O. The issues surrounding efficient I/O can bequite challenging. Often, a system developer must resolvehow to arrange multiple I/O operations to utilize concurrencyavailable from the hardware/software platform. For instance,a high-performance Web server should simultaneously trans-fer files to and from the network, while concurrently parsingnewly obtained incoming requests from other clients.

Certain types of I/O operations have different requirementsthan others. For example, a Web transaction involving mon-etary fund transfer may need to runsynchronously, i.e., toconclude before the user can continue. Conversely, Web ac-cesses to static information, such as CGI-based search enginequeries, may runasynchronouslysince they can be cancelledat any time. Resolving these various requirements have led todifferent strategies for performing I/O.


As indicated above, various factors influence which I/O strat-egy to choose. Web servers can be designed to use severaldifferent I/O strategies, includingsynchronous, reactive, and

10

asynchronousI/O. The relative benefits of using these strate-gies are discussed below.

The synchronous I/O strategy: Synchronous I/O describesthe model of I/O interaction between a Web server processand the kernel. In this model, the kernel does not return thethread of control to the server until the requested I/O operationeither completes, completes partially, or fails. [1] shows thatsynchronous I/O usually performs well for small file transferson Windows NT over high-speed ATM networks.

Synchronous I/O is well known to UNIX server program-mers and is arguably the easiest to use. However, there aredisadvantages to this model. First, if combined with a Single-Threaded concurrency strategy, it is not possible to performmultiple synchronous I/O operations simultaneously. Second,when using multiple threads (or processes), it is still possiblefor an I/O request to block indefinitely. Thus, finite resources(such as socket handles or file descriptors) may become ex-hausted, making the server unresponsive.

The reactive I/O strategy: Early versions of UNIX pro-vided synchronous I/O exclusively. System V UNIX intro-duced non-blocking I/O to avoid the blocking problem. How-ever, non-blocking I/O requires the Web server to poll the ker-nel to discover if any input is available [11]. Reactive I/Oalleviates the blocking problems of synchronous I/O withoutresorting to polling. In this model, a Web server uses anOS event demultiplexing system call (e.g. select in UNIXor WaitForMultipleObjects in Win32) to determinewhich socket handles can perform I/O. When the call returns,the server can perform I/O on the returned handles,i.e., theserverreactsto multiple events occurring on separate handles.

Reactive I/O is widely used by event-driven applications(such as X windows) and has been codified as theReactordesign pattern [14]. Unless reactive I/O is carefully encap-sulated, however, the technique is error-prone due to the com-plexity of managing multiple I/O handles. Moreover, reactiveI/O may not make effective use of multiple CPUs.

The asynchronous I/O strategy: Asynchronous I/O sim-plifies the de-multiplexing of multiple events in one or morethreads of control without blocking the Web server. When aWeb server initiates an I/O operation, the kernel runs the oper-ation asynchronously to completion while the server processesother requests. For instance, theTransmitFile operationin Windows NT can transfer an entire file from the server tothe client asynchronously.

The advantage of asynchronous I/O is that the Web serverneed not block on I/O requests since they complete asyn-chronously. This allows the server to scale efficiently for op-erations with high I/O latency, such as large file transfers. Thedisadvantage of asynchronous I/O is that it is not available onmany OS platforms (particularly UNIX). In addition, writing

asynchronous programs can be more complicated than writingsynchronous programs [21, 22, 28].

3.4.3 JAWS I/O Strategy Framework

Empirical studies in [1] systematically subjected differentserver strategies to various load conditions. The results re-veal that each I/O strategy behaves differently under differ-ent load conditions. Furthermore, no single I/O strategy per-formed optimally under all load conditions. The JAWS I/OStrategy Framework addresses this issue by allowing the I/Ostrategy to adapt dynamically to run-time server conditions.Furthermore, if a new OS provides a custom I/O mechanism(such as, asynchronous scatter/gather I/O) that can potentiallyprovide better performance, the JAWS I/O Strategy frameworkcan easily be adapted to use it.

send_file(fh)Proactor->

io->send_file("index.html")

send_file(fh)Reactor->

write(fh)

Synchronous IOreceive_file()receive_data()

send_data()send_file()

Asynchronous IO

send_data()

receive_file()receive_data()send_file()

Proactor

Reactive IOreceive_file()

send_data()

receive_data()send_file()

Reactor

InputOutput Handlerio

error()complete()

receive_file()

send_data()send_file()receive_data()

iohInputOutput

fh

receive_file()

send_data()

receive_data()send_file()

ioPerform Requestsvc()

Filecache Handle

Figure 13: Structure of the I/O Strategy Framework

Figure 13 illustrates the structure of the I/O Strategy Frame-work provided by JAWS.Perform Requestis a Filter thatderives fromProtocol Pipeline, which is explained in Sec-tion 3.5. In this example,Perform Requestissues an I/O re-quest to itsInputOutput Handler. The InputOutput Handlerdelegates I/O requests made to it to theInputOutputobject.

The JAWS framework providesSynchronous, AsynchronousandReactive IOcomponent implementations derived fromIn-putOutput. Each I/O strategy issues requests using the appro-priate mechanism. For instance, theSynchronous IOcompo-nent utilizes the traditional blockingread andwrite systemcalls; theAsynchronous IOperforms requests according to theProactor pattern [21]; andReactive IOutilizes theReactorpattern [14].

11

An InputOutputcomponent is created with respect to thestream associated by theAcceptorfrom theTaskcomponentdescribed in Section 3.3. File operations are performed withrespect to aFilecache Handlecomponent, described in Sec-tion 3.6. For example, asend file operation sends the filerepresented by aFilecache Handleto the stream returned bytheAcceptor.

3.5 Protocol Pipeline Strategies


Early Web servers, like NCSA’s originalhttpd , performedvery little file processing. They simply retrieved the requestedfile and transfered its contents to the requester. However, mod-ern Web servers perform data processing other than file re-trieval. For instance, the HTTP/1.0 protocol can be used to de-termine various file characteristics, such as the file type (e.g.,text, image, audio or video), the file encoding and compres-sion types, the file size and its date of last modification. Thisinformation is returned to requesters via an HTTP header.

With the introduction of CGI, Web servers have been able toperform an even wider variety of tasks. These include searchengines, map generation, interfacing with database systems,and secure commercial and financial transactions. However,a limitation of CGI is that the server must spawn a newpro-cessto extend server functionality. It is typical for each re-quest requiring CGI to spawn its own process to handle it,causing the server to behave as aProcess-per-Requestserver,which is a performance inhibitor [3]. The challenge for a high-performance Web server framework is to allow developers toextend server functionality without resorting to CGI processes.


Most Web servers conceptually process or transform a streamof data in several stages. For example, the stages of process-ing an HTTP/1.0 request can be organized as a sequence oftasks. These tasks involve (1) reading in the request, (2) pars-ing the request, (3) parsing the request header information, (4)performing the request, and (5) logging the request. This se-quence forms apipelineof tasks that manipulate an incomingrequest, as shown in Figure 14.

The tasks performed to process HTTP/1.0 requests have afixed structure. Thus, Figure 14 illustrates astatic pipelineconfiguration. Static configurations are useful for server ex-tensions that are requested to perform a limited number of pro-cessing operations. If these operations are relatively few andknowna priori, they can be pre-fabricated and used directlyduring the execution of the server. Examples of static process-ing operations include: marshalling and demarshaling data,data demultiplexing through custom Web protocol layers, and

ParseRequest

ParseHeaders

PerformRequest

LogRequest

CONNECTION

USER_AGENT

HOST

ACCEPT

ACCEPT

ACCEPT

Keep-Alive

Mothra/0.1

any.net

image/gif

image/jpeg

*/*

Date: Thu, 09 Oct 1997 01:26:00 GMT

Server: JAWS/1.0

Content-Length: 12345

Content-Type: text/html

<HTML>

<TITLE>Homepage</TITLE>

<H1>Welcome</H1>

...

</HTML>

HTTP/1.0 200 OK

Last-Modified: Wed, 08 Oct 1997 ...

GET /~jxh/home.html HTTP/1.0Connection: Keep-AliveUser-Agent: Mothra/0.1Host: any.netAccept: image/gif, image/jpeg, */*

any.net - -[09/Oct/1997:01:26:00 -0500]"GET /~jxh/home.html HTTP/1.0"

GET

HTTP/1.0

/users/jxh/public_html/home.html

ReadRequest

Figure 14: The Pipeline of Tasks to Perform for HTTP Re-quests

compiling applet code. Incorporating static pipelines into theserver makes it possible to extend server functionality withoutresorting to spawning external processes. The Apache Webserver provides this type of extensibility by allowingmodules,which encapsulate static processing, to be dynamically linkedto the Web server.

However, situations may arise that require a pipeline of op-erations to be configureddynamically. These occur when Webserver extensions involve arbitrary processing pipeline config-urations, so the number is essentially unlimited. This can hap-pen if there is a large number of intermediary pipeline com-ponents that can be arranged into arbitrary sequences. If theoperations on the data are known only during the execution ofthe program, dynamic construction of the pipeline from com-ponents provides an economical solution. There are many ex-amples of when dynamic pipelines may be useful, includingthe following:

Advanced search engines: Search engines may constructdata filters dynamically depending on the provided querystring. For instance, a query such as “(performance AND(NOT symphonic)) ”, requesting for Web pages contain-ing the word “performance” but not the word “symphonic”,could be implemented as a pipeline of a positive match com-ponent coupled with a negative match component.

Image servers: Image servers may construct filters dynami-cally depending on the operations requested by the user. As anexample, a user may request that the image be cropped, scaled,rotated, and dithered.

12

Adaptive Web content: Web content can be dynamicallydelivered dependent on characteristics of the end-user. Forexample, a Personal Digital Assistant (PDA) should receiveWeb content overviews and smaller images, whereas a work-station could receive full multimedia enriched pages. Like-wise, a home computer user may choose to block certain typesof content.

Existing solutions that allow server developers to enhanceWeb server functionality dynamically are either too applica-tion specific, or too generic. For example, Netscape’s NetDy-namics focuses entirely on integrating Web servers with exist-ing database applications. Conversely, Java-based Web servers(such as Sun’s Java Server and W3C’s Jigsaw) allow for arbi-trary server extensibility, since Java applications are capable ofdynamically executing arbitrary Java code. However, the prob-lem of how to provide a dynamically configurable pipeline ofoperations is still left to be resolved by the server developers.Thus, developers are left to custom engineer their own solu-tions without the benefits of an application framework basedon design patterns.

Structuring a server to process data in logical stages isknown as thePipes and Filterspattern [23]. While the patternhelps developers recognize how to organize the componentsof a processing pipeline, it does not provide a framework fordoing so. Without one, the task of adapting a custom serverto efficiently adopt new protocol features may be too costly,difficult, or result in high-maintenance code.

3.5.3 JAWS Protocol Pipeline Strategy Framework

The preceding discussion motivated the need for developers toextend server functionality without resorting to external pro-cesses. We also described how thePipes and Filterspatterncan be applied to create static and and dynamic informationprocessing pipelines. JAWS’Protocol Pipeline Frameworkis designed to simplify the effort required to program thesepipelines. This is accomplished by providingtask skeletonsof pipeline components. When the developer completes thepipeline component logic, the component can then be com-posed with other components to create a pipeline. The com-pleted components are stored into a repository so that they areaccessible while the server is running. This enables the serverframework to dynamically create pipelines as necessary whilethe Web server executes.

Figure 15 provides an illustration of the structure of theframework. The figure depicts aProtocol Handlerthat uti-lizes theProtocol Pipelineto process incoming requests. AFilter component derives fromProtocol Pipeline, and servesas a task skeleton for the pipeline component.

Pipeline implementors derive fromFilter to create pipelinecomponents. Thesvc method of each pipeline component

log()Filter::svc()

perform()Filter::svc()

Parse Requestsvc()parse()

Filter::svc()parse()

Log Requestsvc()log()

Perform Requestsvc()perform()

Parse Headerssvc()parse()

io->receive_data()

Filtersvc() component->svc()

Read Requestsvc()

pipe

component

ioProtocol Pipelinesvc()

InputOutput HandlerProtocol Handler

Figure 15: Structure of the Protocol Pipeline Framework

first calls into the parentsvc , which retrieves the data tobe processed, and then performs its component specific logicupon the data. The parent invokes thesvc method of the pre-ceding component. Thus, the composition of the componentscauses a chain of calls pulling data through the pipeline. Thechain ends at the component responsible for retrieving the rawinput, which derives directly fromProtocol Pipelineabstrac-tion.

3.6 File Caching Strategies


The results in [3] show that accessing the filesystem is a sig-nificant source of overhead for Web servers. Most distributedapplications can benefit from caching, and Web servers are noexception. Therefore, it is not surprising that research on Webserver performance focuses on file caching to achieve betterperformance [29, 30].

A cache is a storage medium that provides more efficientretrieval than the medium on which the desired informationis normally located. In the case of a Web server, the cacheresides in the server’s main memory. Since memory is a lim-ited resource, files only reside in the cache temporarily. Thus,the issues surrounding optimal cache performance are drivenby quantity (how much information should be stored in thecache) andduration(how long information should stay in thecache).

13


Quantity and duration are strongly influenced by the size allo-cated to the cache. If memory is scarce, it may by undesirableto cache a few large files since caching many smaller files willgive better average performance. If more memory is available,caching larger files may be feasible.

Least Recently Used (LRU) caching: This cache replace-ment strategy assumes most requests for cached files havetemporallocality, i.e., a requested file will soon be requestedagain. Thus, when the act of inserting a new file into thecached requires the removal of another file, the file that wasleast recently usedis removed. This strategy is relevant toWeb systems that serve content with temporal properties (suchas daily news reports and stock quotes).

Least Frequently Used (LFU) caching: This cache re-placement strategy assumes files that have been requested fre-quently are more likely to be requested again, another form oftemporal locality. Thus, cache files that have beenleast fre-quently usedare the first to be replaced in the cache. Thisstrategy is relevant to Web systems with relatively static con-tent, such as Lexis-Nexis and other databases of historical fact.

Hinted caching: This form of caching is proposed in [29].This strategy stems from analysis of Web page retrieval pat-terns that seem to indicate that Web pages havespatial local-ity. That is, a user browsing a Web page is likely to browse thelinks within the page. Hinted caching is related topre-fetching,though [29] suggests that the HTTP protocol be altered to al-low statistical information about the links (orhints) to be sentback to the requester. This modification allows the client todecide which pages to pre-fetch. The same statistical informa-tion can be used to allow the server to determine which pagesto pre-cache.

Structured caching: This refers to caches that have knowl-edge of the data being cached. For HTML pages, structuredcaching refers to storing the cached files to support hierarchi-cal browsing of a single Web page. Thus, the cache takesadvantage of the structure that is present in a Web page todetermine the most relevant portions to be transmitted to theclient (e.g., a top level view of the page). This can potentiallyspeed up Web access for clients with limited bandwidth andmain memory, such as PDAs. Structured caching is related tothe use of B-tree structures in databases, which minimize thenumber of disk accesses required to retrieve data for a query.

3.6.3 JAWS Cached Virtual Filesystem Framework

The solutions described above present several strategies forimplementing a file cache. However, employing a fixedcaching strategy does not always provide optimal performance

[31]. The JAWS Cached Virtual Filesystem Framework ad-dresses this issue in two ways. For one, it allows the cachereplacement algorithms and cache strategy to be easily inte-grated into the framework. Furthermore, by utilizing strat-egy profiles, these algorithms and strategies can be dynami-cally selected in order to optimize performance under chang-ing server load conditions.

Figure 16 illustrates to components in JAWS thatcollaborate to make the Cached Virtual Filesystem Frame-

cvf->open(file,fo)fh->open(file)

LRU Strategyinsert()

LFU Strategyinsert()

replace leastrecently used

replace leastfrequently used

htFilecachefind()fetch()remove()create()finish()

rw_lock[]

Cache Strategyinsert()

InputOutputreceive_file()send_file()

filename

fhFilecache Handlehandle()mapaddr()name()size()

open()

fo

cvf

cache

handle()mapaddr()name()size()

Filecache Object

Hashtable

Figure 16: Structure of the Cached Virtual Filesystem

work. TheInputOutputobject instantiates aFilecache Handle,through which file interactions, such as reading and writing,are conducted. TheFilecache Handlereferences aFilecacheObject, which is managed by theFilecachecomponent. TheFilecache Objectmaintains information that is shared acrossall handles that reference it, such as the base address of thememory mapped file. TheFilecachecomponent is the heart ofthe Cached Virtual Filesystem Framework. It managesFile-cache Objectsthrough hashing, and follows theStatepatternas it is choosing aCache Strategyto fulfill the file request. TheCache Strategycomponent utilizes theStrategypattern to al-low different cache replacement algorithms, such as LRU andLFU, to be interchangeable.

3.7 The JAWS Framework Revisited

Section 2 motivated the need for frameworks and patterns tobuild high-performance Web servers. We used JAWS as anexample of how frameworks and patterns can enable program-mers to avoid common pitfalls of developing Web server soft-ware. Together, patterns and frameworks support the reuse ofintegrated components and design abstractions [4].

14

Section 3 has described how the JAWS framework is archi-tected and the strategies it provides. To articulate the organi-zation of JAWS’ design, we outlined the strategic and tacticaldesign patterns that had the largest impact on the JAWS frame-work. TheAcceptor, Reactor, Proactor, Active Object, andService Configuratorpatterns are among the most significantstrategic patterns. TheStrategy, Adapter, State, andSingletonpatterns are influential tactical patterns used in JAWS.

These patterns were chosen to provide the scaffolding ofthe JAWS architecture. The patterns described the necessarycollaborating entities, which took the form of smaller com-ponents. From this, the major components of the framework,(i.e., theConcurrency Strategy, IO Strategy, Protocol Pipeline,and theCached Virtual Filesystemframeworks), were con-structed and integrated into a skeleton application. Developerscan use this skeleton to construct specialized Web server soft-ware systems by customizing certain sub-components of theframework (e.g., Filters andCache Strategies).

Figure 17 shows how all the patterns and components ofthe JAWS framework are integrated. The JAWS frameworkpromotes the construction of high-performance Web serversin the following ways. First, it provides several pre-configuredconcurrency and I/O dispatching models, a file cache offeringstandard cache replacement strategies, and a framework forimplementing protocol pipelines. Second, the patterns used inthe JAWS framework help to decouple Web server strategiesfrom (1) implementation details and (2) configuration time.This decoupling enables new strategies to be integrated eas-ily into JAWS. Finally, JAWS is designed to allow the serverto alter its behavior statically and dynamically. For instance,with an appropriate strategy profile, JAWS can accommodatedifferent conditions that may occur during the run-time exe-cution of a Web server. By applying these extensible strate-gies and components with dynamic adaptation, JAWS simpli-fies the development and configuration of high-performanceWeb servers.

4 Web Server Benchmarking Testbedand Empirical Results

Section 3 described the patterns and framework componentsthat enable JAWS to adapt statically and dynamically to its en-vironment. Although this flexibility is beneficial, the successof a Web server ultimately depends on how well it meets theperformance demands of the Internet, as well as corporate In-tranets. Satisfying these demands requires a thorough under-standing of the key factors that affect Web server performance.

This section describes performance results obtained whilesystematically applying different load conditions on differentconfigurations of JAWS. The results illustrate how no sin-

gle configuration performs optimally for all load conditions.This demonstrates the need for flexible frameworks like JAWSthat can be reconfigured to achieve optimal performance underchanging load conditions.

4.1 Hardware Testbed

Our hardware testbed is shown in Figure 18. The testbed con-

��

Start

��

Start

RequestLifecycle

��ATM Switch

Millennia PRO2 plus Millennia PRO2 plus

Requests

Web Client Web Server

Figure 18: Benchmarking Testbed Overview

sists of two Micron Millennia PRO2 plus workstations. EachPRO2 has 128 MB of RAM and is equipped with 2 Pen-tiumPro processors. The client machine has a clock speed of200 MHz, while the server machine runs 180 MHz. In ad-dition, each PRO2 has an ENI-155P-MF-S ATM card madeby Efficient Networks, Inc. and is driven by Orca 3.01 driversoftware. The two workstations were connected via an ATMnetwork running through a FORE Systems ASX-200BX, witha maximum bandwidth of 622 Mbps. However, due to lim-itations of LAN emulation mode, the peak bandwidth of ourtestbed is approximately 120 Mbps.

4.2 Software Request Generator

We used the WebSTONE [32] v2.0 benchmarking software tocollect client- and server-side metrics. These metrics includedaverage server throughput, andaverage client latency. Web-STONE is a standard benchmarking utility, capable of gener-ating load requests that simulate typical Web server file accesspatterns. Our experiments used WebSTONE to generate loadsand gather statistics for particular file sizes in order to deter-mine the impacts of different concurrency and event dispatch-ing strategies.

The file access pattern used in the tests is shown in Table 1.This table represents actual load conditions on popular servers,based on a study of file access patterns conducted by

15

Protocol Handler

Acceptor

Filter

Event Dispatcher

Parse Headers

Concurrency Task

Protocol Pipeline


Log Request

State

Adapter

Adapter

Perform Request

InputOutputInputOutput Handler

Parse Request

State

Filecache Handle

Pipes and FiltersService ConfiguratorActive ObjectAcceptor Service Configurator

StrategySingleton

Reactor

Service ConfiguratorMementoProactor

I/O Strategy Framework

Concurrency Strategy Framework Protocol Pipeline Strategy Framework

Cached Virtual Filesystem Framework

Thread-per-Connection Thread Pool

Read Request

Filecache Object

LRU Strategy LFU Strategy

HashtableFilecacheCache Strategy

Synchronous IO Asynchronous IO Reactive IO

Figure 17: The JAWS Web Server Framework

Document Size Frequency500 bytes 35%5 Kbytes 50%50 Kbytes 14%5 Mbytes 1%

Table 1: File Access Patterns

SPEC [33].

4.3 Experimental Results

The results presented below compare the performance of sev-eral different adaptations of the JAWS Web server. We dis-cuss the effect of different event dispatching and I/O modelson throughput and latency. For this experiment, three adapta-tions of JAWS were used.

1. Synchronous Thread-per-Request:In this adaptation,JAWS was configured to spawn a new thread to handleeach incoming request, and to utilize synchronous I/O.

2. Synchronous Thread Pool: JAWS was configured topre-spawn a thread pool to handle the incoming requests,while utilizing synchronous I/O.

3. Asynchronous Thread Pool: For this configuration,JAWS was configured to pre-spawn a thread pool to han-dle incoming requests, while utilizingTransmitFilefor asynchronous I/O.TransmitFile is a custom

Win32 function that sends file data over a network con-nection, either synchronously or asynchronously.

Throughput is defined as the average number of bits re-ceived per second by the client. A high-resolution timer forthroughput measurement was started before the client bench-marking software sent the HTTP request. The high-resolutiontimer stops just after the connection is closed at the client end.The number of bits received includes the HTML headers sentby the server.

Latency is defined as the average amount of delay in mil-liseconds seen by the client from the time it sends the requestto the time it completely receives the file. It measures howlong an end user must wait after sending an HTTPGETrequestto a Web server, and before the content begins to arrive at theclient. The timer for latency measurement is started just beforethe client benchmarking software sends the HTTP request andstops just after the client finishes receiving the requested filefrom the server.

The five graphs shown for each of throughput and la-tency represent different file sizes used in each experiment,500 bytes through 5 Mbytes by factors of 10. These files sizesrepresent the spectrum of files sizes benchmarked in our ex-periments, in order to discover what impact file size has onperformance.

4.3.1 Throughput Comparisons

Figures 19-23 demonstrate the variance of throughput as thesize of the requested file and the server hit rate are systemat-

16

ically increased. As expected, the throughput for each con-nection generally degrades as the connections per second in-creases. This stems from the growing number of simultaneousconnections being maintained, which decreases the throughputper connection.

As shown in Figure 21, the throughput of Thread-per-Request can degrade rapidly for smaller files as the connectionload increases. In contrast, the throughput of the synchronousThread Pool implementation degrade more gracefully. Thereason for this difference is that Thread-per-Request incurshigher thread creation overhead since a new thread is spawnedfor eachGETrequest. In contrast, thread creation overhead inthe Thread Pool strategy is amortized by pre-spawning threadswhen the server begins execution.

The results in figures 19-23 illustrate thatTransmitFileperforms extremely poorly for small files (i.e., < 50Kbytes). Our experiments indicate that the performance ofTransmitFile depends directly upon the number of simul-taneous requests. We believe that during heavy server loads(i.e., high hit rates),TransmitFile is forced to wait whilethe kernel services incoming requests. This creates a highnumber of simultaneous connections, degrading server perfor-mance.

As the size of the file grows, however,TransmitFilerapidly outperforms the synchronous dispatching models. Forinstance, at heavy loads with the 5 Mbyte file (shown inFigure 23), it outperforms the next closest model by nearly40%. TransmitFile is optimized to take advantage ofWindows NT kernel features, thereby reducing the number ofdata copies and context switches.

4.3.2 Latency Comparisons

Figures 19-23 demonstrate the variance of latency perfor-mance as the size of the requested file and the server hit rate in-crease. As expected, as the connections per second increases,the latency generally increases, as well. This reflects the ad-ditional load placed on the server, which reduces its ability toservice new client requests.

As before, TransmitFile performs extremely poorlyfor small files. However, as the file size grows, its latencyrapidly improves relative to synchronous dispatching duringlight loads.

4.3.3 Summary of Benchmark Results

As illustrated in the results in this section, there is signif-icant variance in throughput and latency depending on theconcurrency and event dispatching mechanisms. For smallfiles, the synchronous Thread Pool strategy provides bet-ter overall performance. Under moderate loads, the syn-chronous event dispatching model provides slightly better la-

tency than the asynchronous model. Under heavy loads andwith large file transfers, however, the asynchronous model us-ing TransmitFile provides better quality of service. Thus,under Windows NT, an optimal Web server should adapt itselfto either event dispatching and file I/O model, depending onthe server’s workload and distribution of file requests.

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1 5 10 15

Concurrent Clients

Th

rou

gh

pu

t (M

bp

s)

Thread/RequestThread PoolTransmitFile

0

50

100

150

200

250

1 5 10 15

Concurrent Clients

Lat

ency

(m

sec)


Figure 19: Experiment Results from 500 Byte File

0

2

4

6

8

10

12

1 5 10 15

Concurrent Clients

Th

rou

gh

pu

t (M

bp

s)


0

50

100

150

200

250

1 5 10 15

Concurrent Clients

Lat

ency

(m

sec)


Figure 20: Experiment Results from 5K File

0

10

20

30

40

50

60

70

1 5 10 15

Concurrent Clients

Th

rou

gh

pu

t (M

bp

s)


0

50

100

150

200

250

300

1 5 10 15

Concurrent Clients

Lat

ency

(m

sec)



0

10

20

30

40

50

60

70

80

90

100

1 5 10 15

Concurrent Clients

Th

rou

gh

pu

t (M

bp

s)


0

100

200

300

400

500

600

700

800

900

1000

1 5 10 15

Concurrent Clients

Lat

ency

(m

sec)



0

20

40

60

80

100

120

1 5 10 15

Concurrent Clients

Th

rou

gh

pu

t (M

bp

s)


0

1000

2000

3000

4000

5000

6000

7000

8000

9000

1 5 10 15

Concurrent Clients

Lat

ency

(m

sec)


Figure 23: Experiment Results from 5M File

17

4.4 A Summary of Techniques for OptimizingWeb Servers

From our research, we have found that it is possible to improveserver performance with a superior server design (a similar ob-servation was made in [34]). Thus, while it is undeniable that a“hard-coded” server (i.e., one that uses fixed concurrency, I/O,and caching strategies) can provide excellent performance, aflexible server framework, such as JAWS, does not necessarilycorrelate with poor performance.

This section summarizes the most significant determinantsof Web server performance. These observations are based onour studies [1, 3] of existing Web server designs and imple-mentation strategies, as well as our experience tuning JAWS.These studies reveal the primary targets for optimizations todevelop high performance Web servers.

Lightweight concurrency: Process-based concurrencymechanisms can yield poor performance, as seen in [3].In multi-processor systems, a process-based concurrencymechanism might perform well, especially when thenumberof processes are equal to the number of processors. In thiscase, each processor can run a Web server process and contextswitching overhead is minimized.

In general, processes should bepre-forkedto avoid the over-head of dynamic process creation. However, it is preferable touse lightweight concurrency mechanisms (e.g., using POSIXthreads) to minimize context switching overhead. As with pro-cesses, dynamic thread creation overhead can be avoided bypre-spawningthreads into a pool at server start-up.

Specialized OS features: Often times, OS vendors willprovide specialized programming interfaces which may givebetter performance. For example, Windows NT 4.0 pro-vides theTransmitFile function, which uses the WindowsNT virtual memory cache manager to retrieve the file data.TransmitFile allows data to be prepended and appendedbefore and after the file data, respectively. This is particularlywell-suited for Web servers since they typically send HTTPheader data with the requested file. Hence, all the data to theclient can be sent in a single system call, which minimizesmode switching overhead.

Usually, these interfaces must be benchmarked carefullyagainst standard APIs to understand the conditions for whichthe special interface will give better performance. In the caseof TransmitFile , our empirical data indicate that the asyn-chronous form ofTransmitFile is the most efficient mech-anism for transferring large files over sockets on Windows NT,as shown in Section 4.

Request lifecycle system call overhead: The request life-cycle in a Web server is defined as the sequence of instruc-tions that must be executed by the server after it receives an

HTTP request from the client and before it sends out the re-quested file. The time taken to execute the request lifecycledirectly impacts the latency observed by clients. Therefore, itis important to minimize system call overhead and other pro-cessing in this path. The following describes various places inWeb servers where such overhead can be reduced.� Reducing synchronization: When dealing with concur-rency, synchronization is often needed to serialize accessto shared resources (such as the Cached Virtual Filesys-tem). However, the use synchronization penalizes perfor-mance. Thus, it is important to minimize the number of locksacquired (or released) during the request lifecycle. In [3], it isshown that servers that average a lower number of lock oper-ations per request perform much better than servers that per-form a high number of lock operations.

In some cases, acquiring and releasing locks can also resultin preemption. Thus, if a thread reads in an HTTP requestand then attempts to acquire a lock, it might be preempted,and may wait for a relatively long time before it is dispatchedagain. This increases the latency incurred by a Web client.� Caching files: If the Web server does not perform filecaching, at least two sources of overhead are incurred. First,there is overhead from theopen system call. Second, thereis accumulated overhead from iterative use of theread andwrite system calls to access the filesystem, unless the fileis small enough to be retrieved or saved in a single call.Caching can be effectively performed using memory-mappedfiles, available in most forms of UNIX and on Windows NT.� Using “gather-write”: On UNIX systems, thewritevsystem call allows multiple buffers to be written to a device ina single system call. This is useful for Web servers since thetypical server response contains a number of header lines inaddition to the requested file. By using “gather-write”, headerlines need not be concatenated into a single buffer before beingsent, avoiding unnecessary data-copying.� Pre-computing HTTP responses:Typical HTTP requestsresult in the server sending back the HTTP header, which con-tains the HTTP success code and the MIME type of the file re-quested, (e.g., text/plain ). Since such responses are partof the expected case they can bepre-computed. When a fileenters the cache, the corresponding HTTP response can alsobe stored along with the file. When an HTTP request arrives,the header is thus directly available in the cache.

Logging overhead: Most Web servers support features thatallow administrators to log the number of hits on various pagesthey serve. Logging is often done to estimate the load on theserver at various times during the day. It is also commonlyperformed for commercial reasons,e.g., Web sites might basetheir advertising rates on page hit frequencies. However, log-ging HTTP requests produces significant overhead for the fol-lowing reasons:

18

� Filesystem access:A heavily loaded Web server makes asignificant number of I/O calls, which stresses the filesystemand underlying hardware. Writing data to log files increasesthis stress and thus contributes to lower performance. Keepinglog files and the HTTP files on separate filesystems and, ifpossible, on separate physical devices can limit this overhead.� Synchronization overhead:A typical Web server has mul-tiple active threads or processes serving requests. If thesethreads/processes are required to log requests to a commonshared log file, access to this log file needs to be synchronized,i.e., at most one thread/process can write to the shared log fileat any time. This synchronization introduces additional over-head and is thus detrimental to performance. This overheadcan be reduced by keeping multiple independent log files. Ifmemory buffers are used, these should be stored inthread-specific storageto eliminate locking contention.� Reverse hostname lookups:The IP address of the clientis available to a Web server locally. However, the hostnameis typically more useful information in the log file. Thus, theIP address of the client needs to be converted into the corre-sponding host name. This is typically done usingreverse DNSlookups. Since these lookups often involve network I/O, theyare very costly. Therefore, they should be avoided or com-pleted asynchronously (using threads or asynchronous I/O).� Ident lookups: The Ident protocol [35] allows a Web serverto obtain the user name for a given HTTP connection. Thistypically involves setting up a new TCP/IP connection to theuser’s machine and thus involves a round-trip delay. Also, theident lookup must be performed while the HTTP connectionis active and therefore cannot be performed lazily. To achievehigh performance, such lookups must thus be avoided when-ever possible.

Transport layer optimizations: The following transportlayer options should be configured to improve Web server per-formance over high-speed networks:� The listen backlog: Most TCP implementations buffer in-coming HTTP connections on a kernel-resident “listen queue”so that servers can dequeue them for servicing usingaccept .If the TCP listen queue exceeds the “backlog” parameter tothe listen call, new connections are refused by TCP. Thus,if the volume of incoming connections is expected to be high,the capacity of the kernel queue should be increased by givinga higher backlog parameter (which may require modificationsto the OS kernel).� Socket send buffers:Associated with every socket is a sendbuffer, which holds data sent by the server, while it is be-ing transmitted across the network. For high performance,it should be set to the highest permissible limit (i.e., largebuffers). On Solaris, this limit is 64k.� Nagle’s algorithm (RFC 896): Some TCP/IP implementa-tions implement Nagle’s Algorithm to avoidcongestion. This

can often result in data getting delayed by the network layerbefore it is actually sent over the network. Several latency-critical applications (such as X-Windows) disable this algo-rithm, (e.g., Solaris supports theTCP NODELAYsocket op-tion). Disabling this algorithm can improve latency by forcingthe network layer to send packets out as soon as possible.

5 Concluding Remarks

As computing power and network bandwidth have increaseddramatically over the past decade, so has the complexity in-volved in developing communication software. In particu-lar, the design and implementation of high-performance Webserver software is expensive and error-prone. Much of the costand effort stems from the continual rediscovery and reinven-tion of fundamental design patterns and framework compo-nents. Moreover, the growing heterogeneity of hardware ar-chitectures and diversity of OS and network platforms make ithard to build correct, portable, and efficient Web servers fromscratch.

Building a high-performance Web server requires an un-derstanding of the performance impact of each subsystem inthe server,e.g., concurrency and event dispatching, server re-quest processing, filesystem access, and data transfer. Ef-ficiently implementing and integrating these subsystems re-quires developers to meet various design challenges and nav-igate through alternative solutions. Understanding the trade-offs among these alternatives is essential to providing optimalperformance. However, the effort required to developad hocdesigns is often not cost effective as requirements (inevitably)change. As examples, the performance may prove to be inad-equate, additional functionality may be required, or the soft-ware may need to be ported to a different architecture.

Object-oriented application frameworks and design patternshelp to reduce the cost and improve the quality of softwareby leveraging proven software designs and implementationsto produce reusable components that can be customized tomeet new application requirements. The JAWS frameworkdescribed in this paper demonstrates how the developmentof high-performance Web servers can be simplified and uni-fied. The key to the success of JAWS is its ability to capturecommon communication software design patterns and consol-idate these patterns into flexible framework components thatefficiently encapsulate and enhance low-level OS mechanismsfor concurrency, event demultiplexing, dynamic configuration,and file caching. The benchmarking results presented hereserve to illustrate the effectiveness of using frameworks to de-velop high-performance applications. It brings to light thatflexible software is not antithetical to performance.

The source code and documentation for JAWS are availableatwww.cs.wustl.edu/ �schmidt/ACE.html .

19

References[1] J. Hu, I. Pyarali, and D. C. Schmidt, “Measuring the Impact

of Event Dispatching and Concurrency Models on Web ServerPerformance Over High-speed Networks,” inProceedings of the2nd Global Internet Conference, IEEE, November 1997.

[2] I. Pyarali, T. H. Harrison, and D. C. Schmidt, “Designand Performance of an Object-Oriented Framework for High-Performance Electronic Medical Imaging,”USENIX Comput-ing Systems, vol. 9, November/December 1996.

[3] J. Hu, S. Mungee, and D. C. Schmidt, “Principles for Develop-ing and Measuring High-performance Web Servers over ATM,”in Submitted to INFOCOM ’97 (Washington University Techni-cal Report #WUCS-97-10), February 1997.

[4] R. Johnson, “Frameworks = Patterns + Components,”Commu-nications of the ACM, vol. 40, Oct. 1997.

[5] E. Gamma, R. Helm, R. Johnson, and J. Vlissides,Design Pat-terns: Elements of Reusable Object-Oriented Software. Read-ing, MA: Addison-Wesley, 1995.

[6] D. C. Schmidt and T. Suda, “An Object-Oriented Frameworkfor Dynamically Configuring Extensible Distributed Commu-nication Systems,”IEE/BCS Distributed Systems EngineeringJournal (Special Issue on Configurable Distributed Systems),vol. 2, pp. 280–293, December 1994.

[7] D. C. Schmidt, “Applying Design Patterns and Frameworks toDevelop Object-Oriented Communication Software,” inHand-book of Programming Languages(P. Salus, ed.), MacMillanComputer Publishing, 1997.

[8] D. C. Schmidt, “A Family of Design Patterns for Application-level Gateways,”The Theory and Practice of Object Systems(Special Issue on Patterns and Pattern Languages), vol. 2, no. 1,1996.

[9] T. H. Harrison, D. L. Levine, and D. C. Schmidt, “The De-sign and Performance of a Real-time CORBA Event Service,”in Proceedings of OOPSLA ’97, (Atlanta, GA), ACM, October1997.

[10] D. C. Schmidt and C. Cleeland, “Applying Patterns to DevelopExtensible and Maintainable ORB Middleware,”Communica-tions of the ACM, to appear, Dec. 1997.

[11] M. K. McKusick, K. Bostic, M. J. Karels, and J. S. Quarter-man,The Design and Implementation of the 4.4BSD OperatingSystem. Addison Wesley, 1996.

[12] “Information Technology – Portable Operating System Inter-face (POSIX) – Part 1: System Application: Program Interface(API) [C Language],” 1995.

[13] W. R. Stevens,UNIX Network Programming, Second Edition.Englewood Cliffs, NJ: Prentice Hall, 1997.

[14] D. C. Schmidt, “Reactor: An Object Behavioral Pattern forConcurrent Event Demultiplexing and Event Handler Dispatch-ing,” in Pattern Languages of Program Design(J. O. Coplienand D. C. Schmidt, eds.), pp. 529–545, Reading, MA: Addison-Wesley, 1995.

[15] R. G. Lavender and D. C. Schmidt, “Active Object: an Ob-ject Behavioral Pattern for Concurrent Programming,” inPat-tern Languages of Program Design(J. O. Coplien, J. Vlissides,and N. Kerth, eds.), Reading, MA: Addison-Wesley, 1996.

[16] D. C. Schmidt, “Experience Using Design Patterns to DevelopReuseable Object-Oriented Communication Software,”Com-munications of the ACM (Special Issue on Object-Oriented Ex-periences), vol. 38, October 1995.

[17] A. Stepanov and M. Lee, “The Standard Template Library,”Tech. Rep. HPL-94-34, Hewlett-Packard Laboratories, April1994.

[18] D. C. Schmidt, “ACE: an Object-Oriented Framework forDeveloping Distributed Applications,” inProceedings of the6th USENIX C++ Technical Conference, (Cambridge, Mas-

sachusetts), USENIX Association, April 1994.

[19] D. C. Schmidt, “Acceptor and Connector: Design Patterns forInitializing Communication Services,” inPattern Languages ofProgram Design(R. Martin, F. Buschmann, and D. Riehle,eds.), Reading, MA: Addison-Wesley, 1997.

[20] P. Jain and D. C. Schmidt, “Service Configurator: A Patternfor Dynamic Configuration of Services,” inProceedings of the3rd Conference on Object-Oriented Technologies and Systems,

USENIX, June 1997.

[21] T. Harrison, I. Pyarali, D. C. Schmidt, and T. Jordan, “Proac-tor – An Object Behavioral Pattern for Dispatching Asyn-chronous Event Handlers,” inThe 4

th Pattern Languages ofProgramming Conference (Washington University technical re-port #WUCS-97-34), September 1997.

[22] I. Pyarali, T. H. Harrison, and D. C. Schmidt, “AsynchronousCompletion Token: an Object Behavioral Pattern for EfficientAsynchronous Event Handling,” inPattern Languages of Pro-gram Design(R. Martin, F. Buschmann, and D. Riehle, eds.),Reading, MA: Addison-Wesley, 1997.

[23] F. Buschmann, R. Meunier, H. Rohnert, P. Sommerlad, andM. Stal, Pattern-Oriented Software Architecture - A System ofPatterns. Wiley and Sons, 1996.

[24] D. C. Schmidt, “GPERF: A Perfect Hash Function Generator,”in Proceedings of the2nd C++ Conference, (San Francisco,California), pp. 87–102, USENIX, April 1990.

[25] P. Jain and D. C. Schmidt, “Service Configurator: A Patternfor Dynamic Configuration and Reconfiguration of Communi-cation Services,” inThe3rd Pattern Languages of ProgrammingConference (Washington University technical report #WUCS-97-07), February 1997.

[26] T. Berners-Lee, R. T. Fielding, and H. Frystyk, “Hyper-text Transfer Protocol – HTTP/1.0,” Informational RFC1945, Network Working Group, May 1996. Available fromhttp://www.w3.org/.

[27] R. Fielding, J. Gettys, J. Mogul, H. Frystyk, and T. Berners-Lee,“Hypertext Transfer Protocol – HTTP/1.1,” Standards TrackRFC 2068, Network Working Group, January 1997. Availablefrom http://www.w3.org/.

[28] D. C. Schmidt and C. D. Cranor, “Half-Sync/Half-Async: anArchitectural Pattern for Efficient and Well-structured Con-current I/O,” in Pattern Languages of Program Design(J. O.Coplien, J. Vlissides, and N. Kerth, eds.), Reading, MA:Addison-Wesley, 1996.

[29] J. C. Mogul, “Hinted caching in the Web,” inProceedings ofthe Seventh SIGOPS European Workshop: Systems Support forWorldwide Applications, 1996.

20

[30] S. Williams, M. Abrams, C. R. Standridge, G. Abdulla, andE. A. Fox, “Removal Policies in Network Caches for WorldWide Web Documents,” inProceedings of SIGCOMM ’96,(Stanford, CA), pp. 293–305, ACM, August 1996.

[31] E. P. Markatos, “Main memory caching of web documents,” inProceedings of the Fifth International World Wide Web Confer-ence, May 1996.

[32] Gene Trent and Mark Sake, “WebSTONE: The First Gen-eration in HTTP Server Benchmarking.” Silicon Graph-ics, Inc. whitepaper, February 1995. Available fromhttp://www.sgi.com/.

[33] A. Carlton, “An Explanation of the SPECweb96 Bench-mark.” Standard Performance Evaluation Corporation whitepa-per, 1996. Available from http://www.specbench.org/.

[34] H. F. Nielsen, J. Gettys, A. Baird-Smith, E. Prud’hommeaux,H. W. Lie, and C. Lilley, “Network Performance Effects ofHTTP/1.1, CSS1, and PNG,” inTo appear in Proceedings ofACM SIGCOMM ’97, 1997.

[35] M. S. Johns, “Identification Protocol,”Network InformationCenter RFC 1413, Feb. 1993.

21