achieving scalability in highly contentious database systems

23
Informatics and Computer Science NOgI~- ItOUAND Achieving Sealability in Highly Contentious Database Systems ABDELSALAM HELAL* Department of Computer Sciences, Purdue University, W. Lafayette, IN 47907 and JUDSON FORTNER Computer Science and Engineering Department, University of Texas at Arlington, Arlington, TX 76019 ABSTRACT The issue of scalability of a database system is becoming ever more important in today's information age. As the demand for information increases, it is clear that transaction processing was not designed to scale up in a wide open environment. A database faces three dimensions of rapid growth affecting transaction processing: the number of users and transactions (vertical), the geographic span (horizontal), and the volume of data (vertical and horizontal), with this paper focusing on the vertical growth. In particular, we follow a backward-engineering approach to studying the scalability of database systems under very high contention (thousands of transactions per second). We offer a systematic paradigm and its tool for others to use to do the same, based on their prospective application, economics, and environment. As part of this paper, we give details of ScaleTool, a GUI-based simulation tool. This tool is unique in its programmability and usability. It is very usable because of its GUI interface, batch/interactive mode, and help facility. It is very programmable as we have wired in many methods, hardware parameters, database system model, and transaction model parameters and benchmarks. Using our tool, system designers can tailor the database environment closer to their application environment. They can do the same for the prospective query information by tailoring the transaction benchmark. Alternatively, they can opt to use industry standard benchmark, or even synthetic benchmarks. To demonstrate ScaleTool and expand the issue of scale, we describe the results of one scalability study performed. We provide details of the experimental setup, the methods we include in the study, and present the results in succinct way in the form of a performance scalability matrix. * Correspondence should be addressed to this author. INFORMATION SCIENCES 89, 39-61 (1996) © Elsevier Science Inc. 1996 655 Avenue of the Americas, New York, NY 10010 0020-0255/96/$15.00 SSDI 0020-0255(95)00223-C

Upload: abdelsalam-helal

Post on 15-Jul-2016

214 views

Category:

Documents


2 download

TRANSCRIPT

Informatics and Computer Science

NOgI~- ItOUAND

A c h i e v i n g S e a l a b i l i t y in H i g h l y C o n t e n t i o u s D a t a b a s e S y s t e m s

ABDELSALAM HELAL*

Department of Computer Sciences, Purdue University, W. Lafayette, IN 47907

and

JUDSON FORTNER

Computer Science and Engineering Department, University of Texas at Arlington, Arlington, TX 76019

ABSTRACT

The issue of scalability of a database system is becoming ever more important in today's information age. As the demand for information increases, it is clear that transaction processing was not designed to scale up in a wide open environment. A database faces three dimensions of rapid growth affecting transaction processing: the number of users and transactions (vertical), the geographic span (horizontal), and the volume of data (vertical and horizontal), with this paper focusing on the vertical growth. In particular, we follow a backward-engineering approach to studying the scalability of database systems under very high contention (thousands of transactions per second). We offer a systematic paradigm and its tool for others to use to do the same, based on their prospective application, economics, and environment.

As part of this paper, we give details of ScaleTool, a GUI-based simulation tool. This tool is unique in its programmability and usability. It is very usable because of its GUI interface, batch/interactive mode, and help facility. It is very programmable as we have wired in many methods, hardware parameters, database system model, and transaction model parameters and benchmarks. Using our tool, system designers can tailor the database environment closer to their application environment. They can do the same for the prospective query information by tailoring the transaction benchmark. Alternatively, they can opt to use industry standard benchmark, or even synthetic benchmarks.

To demonstrate ScaleTool and expand the issue of scale, we describe the results of one scalability study performed. We provide details of the experimental setup, the methods we include in the study, and present the results in succinct way in the form of a performance scalability matrix.

* Correspondence should be addressed to this author.

INFORMATION SCIENCES 89, 39-61 (1996) © Elsevier Science Inc. 1996 655 Avenue of the Americas, New York, NY 10010

0020-0255/96/$15.00 SSDI 0020-0255(95)00223-C

40 A. HELAL AND J. FORTNER

1. INTRODUCTION

As we progress more heavily into the information age, the location of such information and the speed at which it can be accessed are becoming ever more important. Research has been conducted to evaluate perfor- mance of the hardware platforms, the supporting software, and the appli- cations that utilize this information. Each of these areas has been further decomposed, and each of the components analyzed. The continued ad- vances in hardware development have allowed significant improvement in CPU performance, but software performance has not made comparable improvements [2]. The study of concurrency control algorithms has shown the effect that these algorithms can have on the performance of a database. There are still, however, many questions that need to be an- swered, especially those of scalability [5]. How will the database system scale in performance when the number of users accessing the database increases? What are the requirements to meet a specific Transaction Per Second (TPS) throughput?

In studying scalability, it is clear that transaction processing was not designed to be upwardly scalable in a wide open environment. Current database systems have three directions of rapid growth affecting transac- tion processing (Figure 1). The first direction is the population of users and their transactions, where the workload of concurrent database opera- tions demands a higher degree of multiprogramming (a vertical growth). The second direction is the geographical (nodal) span, where the number of sites comprising the system increases (a horizontal growth). The third

Number of Usen & Transactions Concurrency Control

/ Volume of Data Store

~--~ Data Replication Geographical (Nodal) Span

Fig. 1. Direction of growth in database systems.

SCALABILITY IN CONTENTIOUS DATABASE 41

direction is the volume of data store, where the size of the data and the rate by which it accumulates require higher capacity storage devices and higher I / O and network transfer rates (horizontal and vertical growth). This paper focuses on the first direction of growth, specifically on concur- rency control issues in excessively contentious data systems running thou- sands of TPS. Examples of such systems include On-Line Transaction Processing (OLTP) of financial and international market networks and transaction-oriented telephone and commercial communication networks. The motivation behind this research is to understand how factors such as the speed of the CPU and I / O access times affect the performance of a proposed system under varying degrees of contention. Such information can lend itself to better database system planning and design.

To study the scalability of database systems, the ScaleTool was devel- oped. This tool, a simulator based on a closed-queuing model of a single-site database system, was developed to allow the control of system variables and collect statistical information on system performance. Scale- Tool provides a graphical user interface (GUI) for experiment setup, execution, and the viewing of results. The GUI allows for the definition of the database management system to use, the size and configuration of the database itself, the computer model to simulate, and the transaction composition.

ScaleTool was used to define a series of experiments to evaluate the performance of concurrency control algorithms under varying conditions. A modified version of the TPC Benchmark B [3] was used for uniformity of transactions, and a fixed size database (1000 data elements) was used to increase the contention rate for data access. The experiments were run to evaluate the performance of the database from low contention (multipro- gramming level of 10) to highly contentious (multiprogramming level of 2000) under different CPU ratings (in MIPS) and disk I / O speeds. In order to obtain a high confidence level in the simulation results, each experiment was conducted ten times, and the average of the ten results was used based on confidence interval analysis. At the completion of experiment execution, the results were combined into a scalability matrix. This matrix can be constructed to allow the determination of a least cost solution based on available hardware or to gauge maximum system throughput on existing hardware.

The remainder of this paper contains the procedures and results of the scalability studies. Section 2 describes the ScaleTool used for experiment setup and execution. The procedures used to conduct the experiments are detailed in Section 3, with the results in Section 4. The conclusion of this work is presented in Section 5.

42 A. HELAL AND J. FORTNER

2. SCALETOOL

For the study of database performance and scalability, the ScaleTool, a UNIX-based public domain program utilizing OSF/Moti f , was developed. The foundation for this tool is a simulator designed to support two physical resources, a single CPU and a stack of disks, and two logical resources, the concurrency controller (CC) and Adaptor server (scheduler). This model is a modified version of that used in [1], which in turn had its origins in the model of [8, 9]. The logical queuing model is depicted in Figure 2.

The ScaleTool supports a graphical user interface (GUI) to allow the user an easy method to change, view, and control simulation operation. The layout of the screen allows for observation of all user-changeable parameters and their current values. In addition to the visual portion, the GUI provides validation of entered data to ensure the correct operation of the simulation at execution time. As seen in the main window (Figure 3), four models are supported:

• The Database System--controls the size and granularity of the database, as well as the multiprogramming level.

• The User--defines the simulation as closed or open model.

Terminals

| ~ ~ o a ~ , t I

I",, ¢,,7¢,~ a,, i~ Pro~a~'a$

Fig. 2. Logical queuing model.

SCALABILITY IN CONTENTIOUS DATABASE 43

(O'Jteol: Illdtlnl Follow IIq. IRltlsI ~ : ~ Initial ¥41w: ~ T~lPe: Total Run COunt: !

Grm~uler try: l D It~mlc I.~ltl~J n Gm~lfl-~lottl~ ~ I ~ . Vlfltm: ~ ]n~. ¥4fl~: ~ t~k :

rl I~t'-QIL r*l IdauM,.Vllt A Hit. V i l ~ I ~ i NIl. VI I I : l r~8 S~10dI~llng: P,4Cm'4~ In i'41ble 2: t~0

StII let l lm fslpe: Jun St4~m: $1auletlen Ct~cl~: OW4mt:

XIIC~u Te the ~.41ITCOI COntrol Plnel

InItrU~lon Count F~: Ikl film

N-- -1

~*edleck Iet4cttm Ill:

I td t Servtm Time:

I l l ~ R4tlnl: rr-----1

Nmebor oY I t S :

Fig. 3.

Sdneduling ~et~bt F~: Ile~t~t nl Opw~lm

T~tnk Tim:

TrlnIICt~an Cmpc4ll t ten:

~1~ YrB~Ncttm Size:

I* I lro.t.eet ton Size:

Reld"Jk'l te Re!to:

frlnIlCt ion f.(eWaIt t lan

0~ret,on ld: l'able ld: Act!on:

I Rq4CH

ScaleTool main window.

• The Computer System--allows the specification of the hardware configuration, as well as the instruction count for different database system functions.

• The Transaction--specifies the composition of the transaction itself.

The database system model supports four concurrency control algo- rithms: Static 2-PIy(2PL) [11], Dynamic 2PL, Hotspot Quasi-Dynamic Locking (HS-QDL) [5], and Greedy Quasi-Dynamic Locking (GY-QDL) [5]. In addition to the concurrency control algorithms, five conflict resolu- tion policies are supported to specify the necessary action in the condition of a blocked transaction. These policies include No-Waiting (NW), Gen- eral-Waiting (GW), Cautious-Waiting (CW) [6], Wound-wait (WW), and Wait-Die (WD) [10]. The database model also allows the specification of the degree of multiprogramming in a closed system and the seed used to initialize the random transaction generator.

44 A. HELAL AND J. FORTNER

The database system model also allows the layout of the database itself. On selection of the database setup button (Database), a pop-up window (Figure 4) is displayed, allowing the specification of the number of tables, the number of records in each table, and the granularity of the tables. This allows database table design that fits certain benchmarks, especially user defined. The remaining controls in the database system specify if read locks can be shared, if a restarted transactions should wait or immediately be placed on the ready queue, and if transaction scheduling [4] is enabled.

In addition to parameter setting, the Database System Model allows for the definition of simulation scripts. Under a normal testing environment, a single execution of the simulator does not provide the necessary informa- tion for analysis. Multiple runs are required under varying conditions to properly evaluate the results. The definition of a script allows the simula- tor to run through a predefined set of tests with variations in the schema, blocking policy, multiprogramming level, and seed value. The total number of tests to be executed is displayed in Total Run Count label.

The user model allows the control of the simulation type that specifies the arrival process for users, allowing either an open system, or a closed system with terminals.

The Computer Model controls information related to both the simu- lated hardware platform and the Database Management System in use. The Instruction Count (refer to Figure 3) allows the specification for the cost of different functions within the DBMS in terms of the number of instructions executed. The specified instruction count is used in conjunc-

~ T i ~ COIf'IGURflTION SUII_I~III

Tables tn [ ~ t ~ : l~t.~l~m Size:

~ 1 Td~ie 11000 ]

2 Tables ~t,464~ Granulrit9:

* 3 Tables 1 1 I

@ 4 Ta31e*

S T~i~l~

Rec~d~ tn Table 1:

R.,~o.~ in Table 2:

Ru¢~-~,~ In Table 3:

I ,° I

Fig. 4. Database configuration window.

SCALABILITY IN CONTENTIOUS DATABASE 45

Transaction Composition:

Synthetic

TPC Benchmark B

User Defined Benchmark

Percentage

Fig. 5. Transaction composition pull down.

tion with the CPU MIPS Rating field to determine the cost in time of that function. In addition to instruction count, the Computer Model also defines the cost (in time) for the I / O system and deadlock detection when the general waiting blocking policy is in effect. The variables controlled by the Computer Model affects the calculation of response times, throughput, and various other values.

The transaction model (refer to Figure 3) allows the specification of parameters affecting the composition of the transaction submitted to the simulation. The composition of the transaction defines the list of opera- tions that need to be executed and the tables against which the operations apply. Several options are available in transaction creation (refer to Figure 5):

• Synthetic--where the system randomly creates a transaction accord- ing to probability distributions.

• TPC Benchmark B--an industry standard benchmark [3]. • User-definable benchmarks--where the user defines the exact com-

position of the transaction (refer to Figure 6). • Percentage--where a fixed percentage of the size of the database

composes the number of operations in the transaction.

The Scheduling Weight (refer to Figure 3) variables are used for calculation of the scheduling priority introduced in adaptive transaction scheduling [4]. The Hot Spots Access Ratio and Database Hot Spot Ratio define the ratio of data contention in accessing the database. The Think Time variable specifies the delay between the completion of a transaction and the initiation of a new transaction from a terminal.

With reference to Figure 3, general information is displayed in the right-hand side of the Transaction Model panel. This information is for display only, and will be updated automatically as parameters are changed.

46 A. HELAL AND J. FORTNER

ESICNER ~tli_H~U

Operation C~Jnt:

<>2

<>3

(>5

e6

<>?

~a

~9

lO

[ ] p ~ : ~n It/11 ,*

12 ~ 1 ~ 1

t~ite • O~ret1~ 2

~ite • Operetlon 3

O I]l~retton 4

r l I}l~retion 5

• ' t t e 111 I l l - . i r o n 61

01~eotion I Table Id"

0t~rotton 2 ramie Id:

11 I Operation 3 Table Id:

12 1 Operation 4 T,,,hle Id:

12 I 0W, 4ttcc 5 Table ld:

13 I O~retion 6 Table ld:

J

V-w-q

Fig. 6. Benchmark designer window.

The control panel for the ScaleTool allows the start and stop of experiments, help capabilities, and the ability to generate graphs from the results of an experiment. The graph generation uses gnuplot [12] to read the data stored on disk and create a graphical display or a postscript output for printing.

3. EXPERIMENTAl2 PROCEDURE

To study the scalability, all but three of the simulation parameters were maintained at a consistent value. The three differences were those related to the multiprogramming level, the CPU MIPS rating, and disk access speeds. The multiprogramming level was changed to show the performance of the algorithms under severe conditions. Changing of the disk and CPU speeds revealed performance characteristics under different possible hard-

SCALABILITY IN C O N T E N T I O U S DATABASE 47

ware configurations. For this scalability study, only the Static 2PL and the Dynamic 2PL concurrency control algorithms were used. The study was divided into three experiments, with these experiments referred to as S1, $2, and $3. The S1 experiment is composed of two subgroups, each subgroup testing the concurrency control scalability with two different blocking policies. This experiment will only examine the effect of CPU processing power on throughput; therefore, the disk access rate remained constant. Refer to Table 1 for the parameters used in this experiment.

Experiment $2 is identical to S1, but the CPU speed is maintained at a constant rate and the disk access time is varied (refer to Table 1).

Experiment $3 is a combination of both S1 and $2, where both the CPU speed and the disk access times are being varied in the same study. For

TABLE 1 Scalability Simulation Parameters

Parameter Value

Database Size Table 1 Size Table 2 Size Table 3 Size Granularity Seed mpl Transaction Composition New Transaction Instruction Count Restarted Transaction Instruction Count Transaction Completion Instruction Count Concurrency Control Instruction Count Adaptor Instruction Count Disk IO Processing Deadlock Detection Avg. Disk Service Time

Number of Disks CPU MIPS Rating

Hot Spots Access Ratio Database Hot Spot Ratio Simulation Cycles Simulation Type

1000 total data elements 890 data elements 100 data elements 10 data elements

On a data element basis 1, 3, 7, 11, 13, 17, 19, 29, 31, 37

10, 50, 100, 500, 1000, 1500, 2000 TPC Benchmark B

100000 50000

1000 per operation 5000

500 per operation 25000 per operation

500 only when GW block policy active 20 ms (experiments $1 and $2)

10 ms (experiment $2) 5 ms (experiment $2) 2 ms (experiment $2)

Determined Dynamically 50 (experiments S1 and $2)

75 (experiment S1) 150 (experiment S1) 200 (experiment S1)

0.1 0.1

20,000 Closed

48 A. HELAL AND J. FORTNER

this study, however, the Hotspot Quasi-Dynamic Locking (HS-QDL) algo- rithm [5] was used for concurrency control. The parameters shown in Table 1 were used for this test, but the CPU MIPS Rating and the Avg. Disk Service Time were set such that each CPU MIPS rating value was run with all four of the Avg. Disk Service Time values.

In order to obtain a high confidence level in the simulation results, each experiment was conducted ten times, and the average of the ten results was used. In each of these experiments, a different seed value (as specified in Table 1) was used to initialize the transaction generator. The length of each simulation run is specified by the number of lock requests generated (Simulation Cycles). A large value was chosen for the number of simula- tion cycles to allow the ScaleTooi to ramp up and reach a steady state for a prolonged duration, ensuring more reliable results.

The confidence of each experiment was validated by a 95% confidence interval value [7]. The values represented in the graphs presented in Section 4 are the midpoints of a confidence interval.

3.1. EXTENDED TPC B BENCHMARK

One of the most important considerations for a comparative test among the concurrency control algorithms was the composition of the transaction. To maintain a uniform transaction model, the use of the TPC Benchmark B was selected. This benchmark uses a single, simple, update-intensive transaction to load the system. The TPC Benchmark B is intended to reflect the performance of the database system over which an application will run. This benchmark does not consider outside factors such as termi- nal and network input/output and multiple transaction types of varying complexities [3]. The transaction composition for this benchmark is repre- sented in Table 2.

The modified version of the TPC Benchmark B implemented in the ScaleTool requires the installation of three tables. The first table, repre- senting the Customer Data as defined in the benchmark, is the largest table and presents a minimum amount of data contention. The second table, representing the teller data, is a table with fewer records, and therefore subjects the transaction to a higher rate of data contention. The third table, representing the Branch data, is the smallest table and offers the highest rate of data contention. For each transaction-per-second configured, the sizing requirements [3] for these tables are:

Table 1 (Customer Data) 100,000 records Table 2 (Teller Data) 10 records Table 3 (Branch Data) 1 record.

S C A L A B I L I T Y IN C O N T E N T I O U S D A T A B A S E

TABLE 2

TPC Benchmark B Implementation Comparison

49

ScaleTool TPC Original Definition of TPC Benchmark B Benchmark B Implementation

BEGIN TRANSACTION Update Account where Account__ ID = Aid:

Read Account._Balance from Account

Set Account___ Balance = Account__ Balance + Delta Write AccounLBalance to Account

Write to History Aid, Tid, Bid, Delta, time_stamp

Update Teller where Teller_id = Tid Set Teller_ Balance = Teller_ Balance + Delta Write Teller_Balance to Teller

Update Branch where Branch_ID = Bid Set Branch_Balance = Branch_ Balance + Delta Write Branch_Balance to Branch

COMMI'I"F TRANSACTION Return Account_Balance to driver

BEGIN TRANSACTION Access Table 1

Read from Table 1, data element x

(no action) Write to Table 1,

data element x Not Supported

Access Table 2 (no action) Write to Table 2,

data element y Access Table 3

(no action) Write to Table 3,

data element z COMMITI" TRANSACTION

(no action)

Since the in tent of the Sca leTool is to calcula te the p e r f o r m a n c e of the concur rency cont ro l le r , and because these r equ i r emen t s well exceed the capabi l i t ies o f the ScaleTool , a mod i f i ed a p p r o a c h was taken. A smal le r da t abase o f 1000 records was used to in t roduce a h igher p robabi l i ty o f conflict . In looking at the or iginal specif icat ion, the tab les within the da t abase were cons t ruc ted to al low high confl ict for Tables 2 and 3 and min imal confl ict for Tab le 1. The th ree tables were conf igured as follows:

Tab le 1 ( C u s t o m e r D a t a )

Tab le 2 (Te l le r D a t a )

Tab le 3 (Branch Da ta )

to ta l da t abase size x 89%

tota l da t abase size × 10%

tota l da t abase size x 1%.

A l t h o u g h this table sizing would in t roduce na tu ra l confl ict for the smal le r tables , art if icial confl ict was also in t roduced by dec la r ing 10% of each table as ho t spot areas. In addi t ion , 10% of the t ransac t ions ope ra - t ions were ins t ruc ted to access hot spot e lements . As the t ransac t ion only con ta ined a to ta l of four opera t ions , a r a n d o m mix of hot spot, nonho t spot access evolved with some t ransac t ions de l ibe ra te ly accessing hot spot i tems

50 A. H E L A L A N D J. F O R T N E R

TABLE 3

MIPS Scalability Throughput

Multiprogramming Level

Block Policy Algorithm MIPS 10 50 I00 500 1000 1500 2000

General SL2 50 72.3 110.2 116 .4 104.2 100.0 97.4 96.6 Wait 75 73.2 112.4 118 .8 115.1 103.3 103.3 103.7

150 74.1 112.8 119.8 120.9 117.1 113.6 111.5 200 74.4 113.5 120 .7 122.1 118.7 115.7 114.5

Dynamic 50 115.1 197.5 194 .4 173.4 151.6 144.5 157.6 2PL

Cautious SL2 Wait

Dynamic 2PL

75 116.4 295.4 291.1 258.7 225.9 216.2 235.4 150 117.6 374.2 418.7 411.6 395.1 387.3 397.0 200 118.3 373.4 421.1 431.2 429.3 423.8 421.6

50 72.3 110.1 116 .4 104.2 100.0 97.4 96.6 75 73.2 112.4 118.8 115.1 103.3 103.3 102.7

150 74.1 112.78 119.80 120.9 117.1 113.6 111.5 200 74.4 113.5 120 .7 122.1 118.7 115.7 114.5 50 115.1 187.8 172 .6 115.1 78.8 65.0 63.9

75 116.4 281.3 258 .1 172.8 118.3 94.3 90.8 150 117.9 370.9 419.9 342.0 239.8 195.9 202.1 200 118.2 370.3 420.2 414.9 321.9 270.4 281.8

and some not. Def in ing the ho t spots and the access ra te to these hot spots a reas ensured that a h igher confl ict ra te would be achieved.

4. S C A L A B I L I T Y S T U D Y

The da t a co l lec ted f rom each of the expe r imen t s were analyzed for p e r f o r m a n c e behav io r of the concur rency con t ro l a lgor i thms against var ious ha rdware character is t ics and m u l t i p r o g r a m m i n g levels. The per fo r - mance met r i c used as the t r ansac t ion t h roughpu t ra te m e a s u r e d in Trans- act ions Per Second (TPS). T h e da t a p r e s e n t e d in the fol lowing d iagrams are i n t ended to highl ight the d i f fe rences be tween , the overa l l p e r f o r m a n c e of, each of the a lgor i thms, and not to give abso lu te values. The la t te r in fo rmat ion can be ob t a ined f rom the assoc ia ted tables.

4.1. MIPS SCALABILITY

E x p e r i m e n t $1, which tests the effects of carrying C P U speeds at a cons is ten t disk assess ra te (2 ms), was d iv ided into two subgroups , the first

SCALABILITY IN CONTENTIOUS DATABASE 51

subgroup testing the Static 2PL and Dynamic 2PL concurrency control algorithms against General-Waiting blocking policy, and the second sub- group against Cautious-Waiting blocking policy. The first, General-Wait- ing, is presented in Table 3, with a graphical representation in Figure 7. Because of the nature of Static 2PL, requiring acquisition of all locks needed at the same time without blocking on unavailable locks, the blocking policy in use has no effect on the performance of the algorithm. As can be seen in Figure 7, as the MIPS value is increased, the throughput for both the algorithms improves. For Static 2PL, the improvement is minimal across all speeds of the CPU, ranging from 3% at the lowest multiprogramming level and increasing to 19% at the highest multipro- gramming level. Dynamic 2PL, on the other hand, shows significant improvement with the increase of CPU speeds. At the slowest speed, the improvement is the same as Static 2PL, approximately 3%, but as the CPU

Throughput

I $1.2

Cir. MIPS Rating 150

5O

I0

I

Multi-Programming Level

Fig. 7.

150C

General-Waiting MIPS scalability.

52 A. HELAL AND J. FORTNER

speed increases, the throughput improves up to 167%. It is apparent from the graph that both algorithms hit a thrashing point as the multiprogram- ming level increases and, at such a point, the performance begins to decline, although very slightly. The thrashing points for Static 2PL occur slightly higher than that seen for Dynamic 2PL. At the lower CPU speeds, Static 2PL reaches a maximum throughput at a multiprogramming level of 100, and at faster speeds, at a multiprogramming level of 500. Dynamic, on the other hand, reaches its thrashing point at a multiprogramming level of 50 for the slower CPU speeds and 100 for the faster speeds.

The second part of experiment S1 tested the concurrency control algorithms against the Cautious-Waiting blocking policy. Again, since Static 2PL is an all or nothing algorithm, the blocking policy has no effect

Throughput

m $L2

CPU MIPS Ratiag 150

75

50

5o I00

5OO

Muhi-Progranunmg Level

Fig. 8.

2000

Cautious-Waiting MIPS scalability.

SCALABILITY IN CONTENTIOUS DATABASE 53

on the performance. The results from this experiment can be seen in Table 3 and Figure 8. Like that seen with General-Waiting, as the performance of the CPU increases, the throughput also increases. Unlike General-Wait- ing, when the thrashing point is hit for Dynamic 2PL, the performance degrades rapidly.

In comparing both results, it can be seen that Dynamic 2PL shows the greatest improvement with a more powerful CPU, and shows that the performance of this algorithm is closely coupled with the speed of the CPU executing it. Both Static 2PL and Dynamic 2PL have an optimal throughput at a multiprogramming level of between 50 and 500. At this level, Dynamic 2PL shows a throughput 250% greater than that seen for Static 2PL for both blocking policies. After this point, thrashing occurs and, in particular with Cautious-Waiting, the performance of Dynamic 2PL begins to decline. These experiments also point out that, although Static 2PL generally produces smaller throughput results, the algorithm's perfor- mance is much more predictable, maintaining a much more uniform throughput, and in some instances (slow CPU with high multiprogram- ming), outperforming Dynamic 2PL.

TABLE 4

Disk Access Scalability Throughput

D i s k Mult iprogramming Level

Block Policy Algorithm Speed 10 50 100 500 1000 1500 2000

General SL2 20 ms 72.3 11.02 116.4 104.2 100.0 97 .4 96.6 Waiting 10ms 136.2 193.2 193.8 163.1 161.3 159.4 157.4

5 ms 194.8 199.1 198.2 193.8 191.9 190.5 189.3 l ms 249.4 248.6 248.1 244.7 241.6 238.8 235.9

Dynamic 20 ms 115.1 197.5 194.4 173.4 151.6 144.5 157.6 2PL

Cautious SL2 Waiting

Dynamic 2PL

10ms 196.7 197.6 194.7 173.2 152.1 145.3 158.4 5 ms 199.6 197.9 194.9 173.3 152.6 144.7 157.3 l ms 249.5 247.2 243.5 219.7 212.8 221.1 221.3

20 ms 72.3 110.1 116.4 104.2 100.0 97 .4 96.6 10 ms 136.2 193.2 193.8 163.1 161.3 159.4 157.4 5 ms 194.8 199.1 198.2 193.8 191.9 190.5 189.3 l ms 249.4 248.6 248.1 244.8 241.6 238.8 235.9

20 ms 115.1 187.8 172.6 115.1 78 .8 65 .0 63.9

10 ms 196.6 187.8 173.1 116.1 79.3 64.2 63.2 5 ms 199.3 188.1 173.1 117.5 80 .6 63 .8 62.8 1 ms 249.2 235.2 216.9 153.4 147.9 149.3 150.2

54 A. HELAL AND J. FORTNER

4.2. DISK ACCESS S C A L A B I L I T Y

On conclusion of the $1 experiment group, there was a clear under- standing of the effects CPU speed had on the performance of the two algorithms. The next step was to maintain a constant CPU speed (50 MIPS) and evaluate the performance with varying disk access speeds.

Like S1, experiment $2 was divided into two subgroups, each testing the two concurrency control algorithms against two blocking policies. The first subgroup compares two algorithms using the General-Waiting blocking policy. Table 4 and Figure 9 represent the results of this experiment. As can be seen in these results, the increase in disk access speeds had a significant impact on the performance of Static 2PL. The level of improve- ment ranged from 245% at a multiprogramming level of 10-144% at a multiprogramming level of 2000. Although Dynamic also showed improve-

Throughput

Disk Accc

d ; i p J u

Fig. 9. General-Waiting disk access scalability.

SCALABILITY IN CONTENTIOUS DATABASE 55

ment, it was significantly less than Static 2PL, ranging from 117% at a multiprogramming level of 10-41% at a multiprogramming level of 2000. In all results shown in Figure 9, at the fastest disk speeds, the performance of Statice 2PL was equal to, or better than, that of Dynamic 2PL.

The second subgroup of the $2 experiment evaluates the performance of the two algorithms against the Cautious-Waiting blocking policy. The results are presented in Table 4 and Figure 10. As seen with General- Waiting, the Dynamic 2PL algorithm reacts much less favorably to the increase in disk access speed as compared with CPU speed. With use of the Cautious-Waiting blocking policy, Dynamic 2PL showed considerably less throughput than seen while using General-Waiting, especially at the higher multiprogramming levels.

In comparing the results of these two experiments, it can be seen that the increase in disk access speed helped Dynamic 2PL very little in its

Throughput

Dilk Acc~

L ~

Fig. 10. Cautious-Waiting disk access scalability.

56 A. HELAL AND J. FORTNER

performance, while providing significant benefits to Static 2PL. This fur- ther strengthens the assumption that Dynamic 2PL is more tightly coupled with the speed of the CPU, and indicates a tight coupling between Static 2PL and the disk access speed.

4.3. M1PS/DISK SCALABILITY

To complete the scalability study, experiment $3 compared the effects of manipulating both the CPU rate and the disk access rate simultane- ously. In this set of experiments, the Hotspot Quasi-Dynamic Locking (HS-QDL) [5] algorithm was used with the Cautious-Waiting blocking policy. The results from this experiment can be seen in Figure 11 and Table 5. The performance of the HS-QDL algorithm maintains a fairly consistent level of throughput after reaching a maximum at the lower CPU

2o ~ clink e , : ~ , l ~

~,~

Fig. 11.

2 0 0 U

MIPS/disk access scalability.

SCALABILITY IN CONTENTIOUS DATABASE

TABLE 5

MIPS/Disk Access Scalability Throughput

57

Multiprogramming Level

MIPS Disk Speed 10 50 100 500 1000 1500 2000

50 20 ms 81.2 169.9 191.5 171 .9 165.1 160 .4 164.5 10 ms 153.9 197.2 194.5 179 .8 176.7 178.3 186.1 5 ms 197.6 197.4 195.0 182.9 181.8 184.8 193.5 1 ms 249.2 247 .7 245 .8 243 .9 246 .3 246.1 245.6

75 20 ms 82.5 170 .6 230 .8 249 .3 233.1 216 .5 216.8 10 ms 159.1 289.1 291 .3 264 .6 255 .6 253 .2 263.4 5 ms 273.9 296.1 292 .2 273.1 269 .3 273 .5 285.5 1 ms 249.5 245 .8 241 .9 222.1 206 .3 201 .4 213.3

150 20 ms 83.7 169.5 219.1 385 .4 354.3 322.5 292.8 10 ms 163.8 339 .7 468 .4 496 .9 463 .5 431 .7 429.0 5 ms 318.5 579 .6 582.1 528 .2 511 .9 507 .9 526.0 1 ms 498.9 492 .8 485 .0 441 .2 414 .0 406.3 428.3

200 20 ms 83.9 170.6 220 .7 389 .3 377 .9 344 .3 316.2 10 ms 164.1 336 .4 465 .5 635.1 580.5 522 .7 492.5 5 ms 324.7 676 .7 765 .9 684 .4 657 .9 638 .6 657.2 1 ms 747.1 740.1 729 .3 675.5 658 .4 660.1 706.5

ratings. At the 200 MIPS rating, a maximum throughput is reached at a multiprogramming level of 100, after which performance begins to de- grade. It can be seen that adjusting the CPU or disk speeds improves the performance and, as expected, by adjusting both, the highest throughput was achieved. It is apparent, however, that the most significant factor in the performance of this algorithm is the speed of the CPU.

The results collected during the scalability testing show the characteris- tics of the algorithm under varying hardware conditions. This information can be compiled into a throughput matrix that specifies the hardware requirements necessary for a specific throughput and response time. Table 6 depicts a portion of such a matrix in table format compiled from the $3 test results. In compiling this table, a least-cost approach was taken assuming that higher disk access speeds are less expensive than higher CPU speeds. In reality, the least-cost approach could be a faster CPU and slower disk at one throughput point and a slower CPU and faster disk at another throughput point. The contents of this table recommend the CPU MIPS (e.g., 50MIPS) and disk access speed (e.g., 10Disk) that satisfy the throughput, response time, and multiprogramming level requirements. By using a matrix similar to that shown in Table 6, the throughput rate for each combination can be seen, and from this, the best and most cost-effec- tive solution can be derived.

oo

TA

BL

E 6

Sca

labi

lity

Mat

rix

Tab

le

Mul

ti-

Thr

ough

- pr

ogra

mm

ing

put

Lev

el

50-1

00

100-

150

150-

200

Res

pons

e T

ime

(ms)

200-

250

250-

300

300-

350

350

400

400-

500

0-10

0

100-

200

10

50M

IPS

/20D

isk

50

50M

IPS

/20D

isk

100

50M

IPS

/20D

isk

500

150M

IPS

150M

IPS

75M

IPS

/1

0Dis

k /2

0Dis

k /2

0Dis

k 10

00

200M

IPS

15

0MIP

S/2

0Dis

k /5

Dis

k 15

00

150M

IPS

/10D

isk

2000

20

0MIP

S

/20D

isk

10

50M

IPS

/10D

isk

50

50M

IPS

/20D

isk

100

50M

IPS

/20D

isk

500

150M

IPS

150M

IPS

75M

IPS

/1

0Dis

k /2

10D

isk

/20D

isk

1000

20

0MIP

S

150M

IPS

/20D

isk

/5D

isk

1500

15

0MIP

S /1

0Dis

k 20

00

200M

IPS

/2

0Dis

k

50M

IPS

/20D

isk

75M

IPS

/20D

isk

50M

IPS

/20D

isk

150M

IPS

/20D

isk

75M

IPS

/20D

isk

50M

IPS

/20D

isk

150M

IPS/

2OD

isk

75M

IPS

75

MIP

S/2

0Dis

k 50

MIP

S

/1D

isk

/20D

isk

50M

IPS

/20D

isk

75M

IPS

/20D

isk

50M

IPS

/20D

isk

150M

IPS

/20D

isk

150M

IPS

/20D

isk

75M

IPS

/10D

isk

50M

IPS

/20D

isk

75M

IPS

75M

IPS

/20D

isk

50M

IPS

/1D

isk

/20D

isk

~Z

©

7z

200-

300

300-

400

10

50

100

500

1000

1500

2000

10

50

100

500

1000

1500

2000

50M

IPS

/1D

isk

50M

IPS

/1D

isk

50M

IPS

/1D

isk

150M

IPS

150M

IPS

75M

IPS

50

MIP

S/1

Dis

k /1

0Dis

k /2

0Dis

k /2

0Dis

k 20

0MIP

S

150M

IPS

/20D

isk

75M

IPS

/20D

isk

50M

IPS

/1D

isk

/5D

isk

150M

IPS

150M

IPS

/20D

isk

75M

IPS

/20D

isk

/10D

isk

200M

IPS

15

0MIP

S/2

0Dis

k 75

MIP

S 75

MIP

S/2

0Dis

k /2

0Dis

k /1

Dis

k 15

0MIP

S/5

Dis

k

150M

IPS

/20D

isk

200M

IPS

15

0MIP

S/2

0Dis

k /5

Dis

k 15

0MIP

S 15

0MIP

S/2

0Dis

k /1

0Dis

k 20

0MIP

S

150M

IPS

/20D

isk

/20D

isk

50M

IPS

/1

Dis

k 50

MIP

S

/1D

isk

©

Z

©

60 A. HELA L AND J. F O R T N E R

On a larger scale, a similar matrix can be constructed to include all of the algorithms and blocking policies tested. After each of the algorithms, in combination with the different blocking policies, have been evaluated for performance, an entry corresponding to the throughput, response times, and the related multiprogramming level can be made in the matrix. Several algorithms, or the same algorithm with different blocking policies or required hardware, can share a position in this matrix. This information can then be used to determine the most appropriate concurrency control mechanism based on specific throughput and response time requirements. By traversing through the matrix using either the throughput, response time, or multiprogramming level, a list of concurrency control algorithms can be identified. In addition to the algorithm's name, the associated blocking policy, and hardware requirements necessary to achieved the specified throughput and response times are listed. By comparing the different algorithms and the necessary hardware requirements, a least-cost solution that will work on current available hardware can be determined.

5. CONCLUSION

In this paper, we presented ScaleTool, a GUI-based simulation tool that is based on OSF/Mot i f . We presented the tool, and showed how to program its database, user, hardware, and transaction components. The programmability of ScaleTool allows it to be used to tailor application- specific benchmarks that span the table design and the transaction compo- sition. Such programmability is very useful in assessing the performance of a database system in a particular application domains. Other industry standard benchmarks like TPC-B are also available in ScaleTool. We have demonstrated the objective use of Scale Tool through a performance scalability study that aimed at producing a performance scalability matrix. The latter gives guidelines into which hardware configurations and concur- rency methods are needed to meet certain Transaction Per Second re- quirements.

REFERENCES

1. R. Agrawal, M. Carey, and M. Livny, Concurrency control performance modeling: Alternatives and implications, ACM Trans. Database Sys. 609-654 (Dec. 1987).

2. P. Franaszek, J. Robinson, and A. Thomassian, Concurrency control for high contention environments, ACM Trans. Database Sys. 304-345 (June 1992).

3. J. Gray, The Benchmark Handbook, Morgan Kaufmann, San Mateo, CA, 1991. 4. A. Helal R. Elmasri, T. Ku, and S. Mukherjee, Adaptive transaction scheduling,

University of Texas at Arlington, May 1993.

S C A L A B I L I T Y IN C O N T E N T I O U S D A T A B A S E 61

5. A. Helal and T. Ku, Quasi-dynamic two-phase locking, University of Texas at Arlington, Feb. 1994.

6. M. Hsu and B. Zhang, Performance evaluation of cautious waiting, ACM Trans. Database Sys. 477-512 (Sept. 1992).

7. A. Law and W. Kelton, Simulation Modeling & Analysis, McGraw-Hill, New York, 1991.

8. D. Ries and M. Stonebraker, Effects of locking granulairty on database manage- ment system performance, ACM Trans. Database Sys. 233-246 (Sept. 1977).

9. D. Ries and M. Stonebraker, Locking granularity revisited. ACM Trans. Database Sys. 210-227 (June 1979).

10. D. Rosenkrantz, R. Steams, and P. Lewis, System level concurrency control for distributed database system, ACM Trans. Database Sys. 178-198 (June 1978).

11. Y. Tay., Locking Performance in Centralized Databases, Academic Press, Boston, 1987.

12. J. Yli-Nokari, R. Hartrannft, and R. Lang, GNUPLOT, 1990.

Received 29 June 1994