introduction 2 load testing - progress.com

Introduction 2 Load Testing

Gus Björklund, Progress Software

Exchange Australia, October 18, 20, 22, 2010 Melbourne, Brisbane, Sydney

© 2010 Progress Software Corporation. All rights reserved. 2

Load testing your application is well worth the time and effort. By load testing you can determine if your system is sized properly to handle the expected workload, what will happen when you add more users, whether application changes you make will have adverse effects, and prove to prospects that your application can meet their performance and scalability requirements. Now you can learn how in this session.

~40 minutes

Abstract


Please ask questions as we go

sometimes I do not explain something well enough, or

you may want to know more


Load testing an application is

a bit more work, a bit more time consuming, and a bit more expensive

than you might think — but well worth doing.

Summary


What?


What is (software) load testing?

Process of subjecting (your) software to a known workload, observing, and measuring its response

+


Why?


Capacity Planning

0

100

200

300

400

500

600

700

800

900

1000 Tr

ansa

ctio

ns P

er S

econ

d

Number of Users

if you increase workload, can system still do its job ?


System Sizing

0 200 400 600 800

system 1

system 2

system 3

max users

Will your intended hardware be sufficient?


Regression testing

0

100

200

300

400

500

600

version 4.1 version 5.0 beta

TPS

Did your changes cause unintended consequences?


Stress testing

44004

88307

133501

174399

220242

0

0.3 0.3

0.2

0.3

0.4

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0

50000

100000

150000

200000

250000

400 800 1200 1600 2000 3000

Avg,

Res

pons

e (S

econ

ds)

Tran

sact

ions

per

Hou

r

Number of Users

How far can you push it before it breaks?


Sales

scales to 10,000 users !

Prove performance, scalability, and headroom to a prospect


How?


The Basic Idea

  Run the application •  With requisite number of users •  With specified input data •  Each performing some number of operations •  For a suitable amount of time

  Collect data •  Response times •  Transaction rates •  Correct outputs


Q: But where am I going to get 500 people to use the application?



A: You can do that, but you don't have to. There is another way.



A: You can do that, but you don't have to. There is another way.

Use software to simulate the people using your application.


Load Testing Software

  “Virtual users” run the application in place of humans   Parameterized test scripts and scenarios

•  Provide user inputs and randomized data •  Different scripts for different operations

– Order entry –  Shipping –  Payment processing –  Etc.

  Data files •  Contain variable inputs for data values for many transactions


Some Commercial Load Testing Tools

  HP LoadRunner   Silk Performer   Facilita Forecast   Scapa TPP   Microfocus QALoad   IBM Rational Performance Tester

All have similar capabilities. Details and depth differ: Supported operating systems Protocols UI types


1000 User LoadRunner Setup Used by a Customer

LoadRunner Controller

Character LoadRunner Injector (4 Cores)

GUI LoadRunner Injector (32 Cores)

HP UX 11.31 600 Character Clients

Windows 2003 400 GUI Clients

HP UX 11.31 Database Server


LoadRunner Character UI Virtual User

ABL Application

Char UI

Terminal Emulator

LoadRunner Injector


scripts data

A

Virtual

User

results data

LoadRunner Analyzer

reports

scripts


LoadRunner GUI Virtual User

Remote Desktop Protocol

LoadRunner Injector


scripts scripts data

results data

LoadRunner Analyzer

reports ABL

Application

GUI

A

Virtual

User


Windows GUI Script Example

# Type in account number in the Account field account = 100; set_window ("ATM"); edit_set ("Account", account); # Enter the amount to be deposited in the amount field. amount = 50; set_window ("ATM"); edit_set ("Amount", amount); # Press the Deposit button. button_press ("Deposit"); # Close client application. menu_select_item ("File; Exit");


Script Parameterization

# Type in account number in the Account field account = {Account_number}; set_window ("ATM"); edit_set ("Account", account); # Enter the amount to be deposited in the amount field. amount = {Amount}; set_window ("ATM"); edit_set ("Amount", amount); # Press the Deposit button. button_press ("Deposit"); # Close client application. menu_select_item ("File; Exit");

{} values read from file


Preparing


Different Solutions for Different Application Architectures

  What kind of UI do you have? •  Character •  Windows GUI •  .NET GUI •  Web browser •  Etc.

  Can you test back-end without a UI? •  App server calls from 4GL test program •  Batch jobs


What's Involved?

  Planning   Workload model definition   Resource allocation and procurement   Scenario and script development   Data generation   Test execution


Model Your Workload

  What are the important operations performed by your application?

  What is NOT important?   Number of users?   How many of each type?   Nightly batch runs?   Month-end, quarter-end, year-end processing?


Sample Workload Model

Transaction type User mix Transaction time (sec)

Transactions / hour / user

User login/logout 100 % 0.2 OE order entry 45 % 72 30 OE order shipping 10 % 42 15 OE order inquiry 5 % 180 2 AP costing 20 % 72 10 AR cash receipts 15 % 72 35 PO order entry 5 % 72 20 OE pick ticket printing 1 120 4 Batch login/logout 1 4 AI archiving 1 30


What Should We Measure?

  Time required to execute fixed amount of work   Amount of work executed in a fixed time   Response times for various operations   Resource consumption

•  Memory •  CPU •  Disk I/O rate •  Disk space •  Network traffic

  Etc.


Planning, Part 1

  Determine need   Establish goals and success criteria   Determine what data will be needed to establish

success or failure   Allocate resources   Create test plan and procedures   Run small-scale tests

•  Identify problems •  Correct errors •  Repeat as needed


Planning, Part 2

  Procure hardware •  Use own hardware •  Rent or borrow

– Use a vendor's benchmark center o  IBM, HP, Sun, Dell, etc

– Use Amazon EC2   Set up software

•  Save configured, clean VM's if using

Plan for things to go wrong !!! Allow extra time for the unexpected.


Doing


Load Test Execution

  Run the tests •  Multiple times – are results reproducible ?? •  You may have to fix something

  Archive data and test configuration •  Everything needed to do it again to get same results

  Analyze and report results   Post-mortem

•  What went well ? •  What did we do wrong ? •  What should we do better next time ?


2 Quick Examples


example 0:

verify capacity & headroom on new hardware


0: A Customer's Load Test Project

  Goals: •  Can we consolidate servers by using VMware ESX? •  Will proposed new server machine be sufficient? •  Will it handle 3 years’ expected growth?


0: A Customer's Load Test Project

  Goals: •  Can we consolidate servers by using VMware ESX? •  Will proposed new server machine be sufficient? •  Will it handle 3 years’ expected growth?

  Test up to 560 users   Response time 1 sec or less for all transactions   Scenarios for normal load, peak load, month-end load   Must include all

•  High-volume processes •  High business risk processes •  Resource-intensive processes


0: Where was Time Spent?

Activity duration Planning 30 days Proof of concept trial 3 days Create test scripts 10 days Design test scenarios 2 days Prepare test data files 2 days Test environment setup 3 days Run the tests 9 days Archive test environment and data 1 days Analyze the results 2 days Write test report 3 days Update documents to reflect reality 3 days Contingency time (10 %) 7 days

total 75 days


example 1:

prove performance & scalability to a prospect


1: ACE hardware

  IBM Pseries P690+ (32 way) : 1,7 GHz POWER4+ architecture, 128 GB RAM

  AIX 5.2 with dynamic logical partitioning   IBM ESS storage server model 800   8 channel 2 GB fibre connections between server

and disk array (128 disks)

8x 2 GB FC/AL


Application tuning 150 man days Smaller scaled benchmarks 30 man days Loadrunner adapter 50 man days Benchmark scenario 25 man days Benchmark setup 5 man days Actual benchmark (2 weeks) 60 man days + Benchmark centre (machines + infrastructure) 70.000 $

TOTAL: 320 man days

1: ACE Load Test Project Labor Summary


3 Things

0) Takes commitment

1) Requires budget and resources

2) Well worth doing

Sell more stuff !!

Earn more money


Questions


Bonus Material


LoadRunner protocols/interfaces

  FTP, HTTP, HTTPS, LDAP, DNS, SOAP, COM, RMI, IIOP, SAPGUI, IMAP, POP 3, MAPI, SMTP, SNMP

  Terminal emulation   Windows GUI   Windows RDP   Citrix ICA   Sockets   ODBC   JDBC   and others

introduction 2 load testing - progress.com

Documents