free load testing tools for oracle database – which one do i use?

BASEL BERN BRUGG DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. GENEVA

HAMBURG COPENHAGEN LAUSANNE MUNICH STUTTGART VIENNA ZURICH

Free Load Testing Tools for Oracle

Database – Which One Do I Use?Christian Antognini

8 November 2016

@ChrisAntognini

Free Load Testing Tools for Oracle Database – Which One Do I Use?2 08/11/2016

Senior principal consultant, trainer and partner at Trivadis

[email protected]

http://antognini.ch

Focus: get the most out of Oracle Database

Logical and physical database design

Query optimizer

Application performance management

Author of Troubleshooting Oracle Performance (Apress, 2008/14)

OakTable Network, Oracle ACE Director

Agenda


1. Introduction

2. Free Load Testing Tools I Use

3. Summary


Introduction

Common Situations Leading to Database Load

Generation


Perform usability and functionality checks of tools or utilities that require a

non-trivial load to be carried out

Assess the performance of a (new) platform or storage subsystem

Verify whether a set of SQL statements executed on a specific environment

and/or configuration fulfils the expected performance requirements

Golden Rule


Use the right tool

for the right job!

Free Load Testing Tools in My Toolbox


Swingbench

The Silly Little Oracle Benchmark (SLOB)

Apache JMeter


Swingbench

What’s Swingbench?


Load generator designed to stress test Oracle Database

Includes four configurable synthetic benchmarks

OrderEntry, SalesHistory, CallingCircle and StressTest

Extensible with user-defined transactions (StressTest provides an example)

Consists of several CLI and GUI utilities

Developed by Dominic Giles

Available at http://www.dominicgiles.com/swingbench.html

http://www.dominicgiles.com/swingbench.html

What Can Swingbench Do?


Carry out, with minimal effort, all necessary operations to run one of the four

provided benchmarks

Create test schema (incl. data population)

Generate load

Store key performance indicators about test run

Swingbench Key Components – Setup


Three wizards, that also expose a CLI, are available: OrderEntry, SalesHistory

and CallingCircle

StressTest doesn’t have a wizard because it requires no setup

XML file config dataoewizard

shwizard

ccwizard

Swingbench Key Components – Load Generation


swingbench

minibench

charbench

XML file

XML fileresults2pdf

PDF file

config

load KPI

Swingbench: Demo


Swingbench: Example of Utilization (1)


Purpose

Observe the impact of a vMotion

started while a database server

is subject to a non-trivial load

Benchmark

OrderEntry, scale 10

900 transactions/second



0

20

40

60

80

13:5

8:0

8

13:5

8:1

8

13:5

8:2

8

13:5

8:3

8

13:5

8:4

8

13:5

8:5

8

13:5

9:0

8

13:5

9:1

8

13:5

9:2

8

13:5

9:3

8

13:5

9:4

8

13:5

9:5

8

14:0

0:0

8

14:0

0:1

8

14:0

0:2

8

14:0

0:3

8

14:0

0:4

8

14:0

1:0

2

14:0

1:1

3

14:0

1:2

3

14:0

1:3

3

14:0

1:4

3

14:0

1:5

3

14:0

2:0

3

14:0

2:1

3

14:0

2:2

3

14:0

2:3

3

AA

S

Time

vMotion took 79 seconds



Purpose

Quantify the overhead of TDE

Compare encryption algorithms

Benchmark

OrderEntry, scale 50

160 distinct tests (30 minutes each)

– Users: 10..100

– Encryption: None, AES128,

AES192, AES256

– With and without AES-NI

– Buffer cache: 7GB and 70GB



0

500

1000

1500

2000

2500

No Yes

Tra

nsa

cti

on

s/S

eco

nd

AES-NI

AES128

AES192

AES256

None

0

2

4

6

8

10

12

14

No Yes

CP

U/T

ran

sacti

on

[m

s]

AES-NI

AES128

AES192

AES256

None


The Silly Little Oracle Benchmark

(SLOB)

What’s SLOB?


Platform performance and stability-testing framework that

makes Oracle Database execute SQL statements

Synthetic benchmark

Consists of a few scripts

Developed by Kevin Closson

Available at https://kevinclosson.net/slob/

https://kevinclosson.net/slob/

What Can SLOB Do?


Create one or several test schemas (incl. data population)

Generate load

Random PIO and/or LIO

– Generation of sequential/large PIO by forcing a FTS (e.g. invisible index)

Read only or read write

Lite or heavy redo generation

Store DB as well as OS key performance indicators about test run

SLOB Key Components – Setup


slob.conf config datasetup.sh

SLOB Key Components – Load Generation


runit.sh

slob.conf

AWR

reports

iostat

mpstat

vmstat

awr_info.sh

text file

config

load

SLOB: Demo


SLOB: Example of Utilization (1)


Purpose

Find out the maximum number

of random IOPS (and their

average latency) that a storage

subsystem can sustain

Benchmark

100GB

Read-only PIO/LIO



72563

93546

103718 103299 101258

48 64 96 128 160

Ph

ysi

cal

Read

s p

er

Seco

nd

Sessions

2.9 2.8 4.0 5.2 6.3

31 33

44

60

78

48 64 96 128 160

Late

ncy i

n M

illi

seco

nd

s

Sessions

db file sequential reads db file parallel reads



Purpose

Assess disk I/O performance in

a VMware environment

Questions

– What’s the impact of running

several DB per VM?

– What’s the impact of running

several VM per server?

Benchmark

100GB

1/2/4/8 VM

One/several DB per VM

Each server running an increasing

number of sessions

Read-only and read-write tests


Apache JMeter

What’s JMeter?


Application designed to load test functional behavior and measure

performance

No out-of-the box benchmark available DIY

Consists of several CLI and GUI utilities

Open source project supported by a number of contributors

Available at http://jmeter.apache.org/

http://jmeter.apache.org/

What Can JMeter Do?


Many things!

For the intent of this presentation:

Execute SQL statements

through JDBC

Verify whether the executions

return the expected output

and/or in the expected time

Store key performance

indicators

But it’s “only” a framework… it requires

time to “code” a benchmark

Use real schema with data

Use the SQL statements to be

verified

– Bind variables can be either

generated or provided as input

SLOB Key Components – Load Generation


front end

GUI/CLI

test plan

user

defined

output

text file

config

loadKPI

log

JMeter: Demo


JMeter: Example of Utilization (1)


Purpose

Verify that a set of queries

performs acceptably under load

Verify that the DB server

supports the expected load

Benchmark

33 distinct queries, each with each

own set of bind variable values,

randomly picked out

100 queries/minute



0

500

1000

1500

2000

2500

3000

3500

4000

407

408

503

114

303

111

115

112

308

101

301

117

204

113

118

105

306

116

505

120

201

504

304

305

307

404

403

401

501

203

202

502

302A

ve

rag

e E

lap

se

d T

ime

[m

s]

Query



AAS during a period of 13 minutes


Summary

Comparison


Swingbench SLOB JMeter

Can generates load forOracle

Database

Oracle

Database

any DB with

JDBC support

Synthetic benchmark

User-defined benchmark ()

Effort to implement minimal minimal low high

GUI

CLI for automation with scripts

Distributed load generation

Tool of Choice for Common Situations Leading to

Database Load Generation


Perform usability and functionality checks of tools or utilities that require a

non-trivial load to be carried out

Swingbench

Assess the performance of a (new) platform or storage subsystem

SLOB

Verify whether a set of SQL statements executed on a specific environment

and/or configuration fulfils the expected performance requirements

Apache JMeter

Summary


No single tool can cover all

requirements

If used correctly, Swingbench and

SLOB are simple and effective tools

To run a specific set of SQL

statements don’t spend time build

your own script/tool, use JMeter

Questions and Answers

Christian Antognini

Senior Principal Consultant

[email protected]

@ChrisAntognini

08/11/2016 Free Load Testing Tools for Oracle Database – Which One Do I Use?44

free load testing tools for oracle database – which one do i use?

Software