<Insert Picture Here>

Queuing model-based load testing of large

enterprise applications

Leonid Grinshpan, Ph.D.

Consulting Technical Director, Oracle Corporation

Share to Facebook

Share to LinkedIn

Share toTwitter

Share to SlideShare

The views expressed in this

presentation are author’s own and

do not reflect the views of the

companies he had worked for neither

Oracle Corporation.

All brands and trademarks mentioned

are the property of their owners.

Presentation’s model related

2

Presentation’s model related

considerations

are based on author’s book

“Solving Enterprise Applications

Performance Puzzles: Queuing

Models to the Rescue”

(available in bookstores and from Web

booksellers from March 2012)

http://www.amazon.com/Solving-

Enterprise-Applications-Performance-

Puzzles/dp/1118061578/ref=sr_1_1?ie=

UTF8&qid=1326134402&sr=8-1

What this presentation is about and what it is not

about?

About: The presentation outlines a methodology of queuing

model-based load testing of large enterprise applications

deployed on premises and in the Cloud

Not about: It is not about similarly sounding model based testing

(MBT) that allows a test engineer to automatically generate test

cases from a model of the system under test

3

The presentation’s models are discussed in details in author’s book:

“Solving Enterprise Applications Performance Puzzles:

Queuing Models to the Rescue”

(available in bookstores and from Web booksellers from March 2012)

http://www.amazon.com/Solving-Enterprise-Applications-Performance-

Puzzles/dp/1118061578/ref=sr_1_1?ie=UTF8&qid=1326134402&sr=8-1

Challenges of large enterprise applications load

testing

� High cost of commercial load testing tools enabling emulation of

thousands virtual user

� Deployment of considerable load testing framework with distributed

around the world load generators

� Execution of multiple hours/days-long tests generating gigabytes of

performance data

� Analysis of gigabytes of performance data

4

� Analysis of gigabytes of performance data

� When tests point to a shortage of hardware capacity it is very

problematic or impossible to increase it and retest within time

allocated to load test (challenges in testing what-if scenarios)

Proposed solution in three tweets

1. Limit load test workload to bring down project cost and time

2. Leverage forecasting power of queuing models to estimate

system performance under realistic workload

5

3. Calibrate model using data from limited load test to ensure

modeling results accuracy

Proposed solution in pictures

•Limited workload

Limited (basic)

load test

• Data for model calibration

Model calibration

Calibrated model

6

Calibrated model

•Realistic workload

•What-if scenarios

•Modeling results

Analysis of modeling results

Methodology of queuing model-based load testing (1)

� Execution of minimal scope (limited) load tests in order to collect data for

model calibration (limited numbers of users, load generators, and test

executions). Limited load test can be conducted against two environments:

full or partial system deployments (the latter architecture features

transactions directed to a subset of servers in server farms)

� Gathering transactions profile data (from production system measurements

and log files)

Methodology consists of implementation of the following steps

7

and log files)

� Building application queuing model

� Characterization of realistic full scope workload

� Description of hosting platform (distribution of application’s software

components among servers, per each server specification of number

of cores and CPUs, clock speed, RAM size, operating system)

� Specification of transactions profiles based on data obtained during

transaction measurements

Methodology of queuing model-based load testing (2)

� Model calibration using data collected by load tests (appropriately

calibrated model delivers transaction times and server utilizations close to

the values observed during basic load test)

� Model solving for multiple what-if scenarios

� Analyzing modeling results

Methodology consists of implementation of the following steps

8

Advantages of queuing model-based load testing

� Significant cost saving

� Can be implemented in much shorter time

� Does not requires deployment of large testing framework

� Evaluates multiple what-if scenarios without deployment of

additional hardware

9

additional hardware

� Testing of a particular cloud application has no impact on other

cloud applications and they can work in production mode during test

cycle

Model’s input data to collect during limited load test

1. Workload characterization

� List of business transactions

� Number of users per each business transaction

� Per each transaction a number of executed transactions per user

per hour (transaction rate)

� Per each transaction its 90th percentile response time

Example of workload characterization

10

Model’s input data to collect during limited load test

2. Hosting platform

� Hardware architecture (connections among servers and numbers

of servers on each tier)

� Distribution of application’s software components among servers

(software components hosted by each server)

� Specification of each server

(number of cores and CPUs,

clock speed, RAM size etc)

11

clock speed, RAM size etc)

� Operating system

(Windows, LINUX, etc)

Example server specification

Model’s input data to gather from production

system measurements and log files

3. Transactions profiles

� Profile of each business transaction

Transaction profile is comprised of the time intervals a transaction has spent in

system servers it has visited when application was serving only single request

Example of transactions profiles

12

Model’s input data

4. What-if scenarios

If modeling results point to a shortage of hardware capacity, the following changes can be

quickly evaluated:

– Hardware architecture, specification of servers (number of servers, number of

CPUs on each server, server types).

– Distribution (hosting) of application’s components among servers.

Changing any of the above represents new what-if scenario.

13

Changing any of the above represents new what-if scenario.

Mapping system into model

System under

basic load test

14

System’s model

Solving model

�Author usesTeamQuest software to solve models http://teamquest.com/

� It is possible to solve models using open source software packages. One

of them is Java Modeling Tools (JMT); it is developed by Politecnico di

Milano and can be downloaded from http://jmt.sourceforge.net/. A few

following slides demonstrate its basic functionality.

15

Solving model using opens source package JMT

Workload definition

16

Solving model using opens source package JMT

Specification of hardware servers

17

Solving model using opens source package JMT

Specification of transaction profiles

18

Solving model using opens source package

Modeling results (utilization of servers and transaction times)

19

Model deliverables

DELIVERABLES

� Average transaction response time for each transaction

� Utilization of each hardware server

Transaction time (seconds) Utilization of system servers (%)

20

Model calibration

Appropriately calibrated model delivers transaction times and server utilizations close to

the values observed during basic load test

21

Contact author

Want to know more about

enterprise applications load testing and capacity planning?

Contact Leonid Grinshpan at 101capacityplanning@gmail.com

Share this presentation

22

Share to Facebook

Share to LinkedIn

Share toTwitter

Share to SlideShare