PostgreSQL PortlandPerformance Practice Project

Database Test 2 (DBT-2)How to Use the Kit

Mark Wongmarkwkm@postgresql.org

Portland State University

March 12, 2009

How to use the DBT-2 test kit

Further questions regarding the test kit can be sent to the mailinglist: osdldbt-general@lists.sourceforge.netYou can sign up for the mailing list here:http://lists.sourceforge.net/mailman/listinfo/osdldbt-general

WARNING

This test kit is a work in progress. The last “release” was February11, 2007. The next “release” still needs a bit of work before it’spolished. So let’s work with the source repository and hope it stillworks with these instructions.

Prerequisites for building

◮ PostgreSQL must be installed.

◮ PostgreSQL executables must be in your path.

◮ cmake must be installed, used to build the kit.

◮ A C compiler.

Recommended for running

None of these are required for running, but help characterizedifferent aspects of the system.

◮ Test::Parser perl module for calculating test results.

◮ systat, procps, and iopp for collecting system statistics.

◮ readprofile for Linux kernel profiling.

◮ oprofile for additional Linux kernel profiling, and user spaceprofiling, call graphs, and source code and assemblyannotations.

Prerequisites for analyzing

◮ gnuplot to for making charts.

◮ Test::Presenter perl module for the modules that producethe charts.

Get the kit

git clone git://git.postgresql.org/git/~markwkm/dbt2.git

git is not required, you can download a tarball by clicking on the

snapshot link from here:

http://git.postgresql.org/?p=~markwkm/dbt2.git;a=summary

Relevant documentation

Documentation to supplement or contradict what is in this howtopresentation.Files to note:

◮ INSTALL

◮ README

◮ README-POSTGRESQL

WARNING: Documentation may be incorrect.

Build and Install the Kit

cmake CMakeLists.txt

make

Note: The CMakeLists.txt file is not very polished. The kit will notbuild properly if both MySQL and PostgreSQL are in your path.

make install DESTDIR=/usr/local

Note: Don’t fear, you can uninstall the kit:

xargs rm < install_manifest.txt

Install supporting perl modules

Install from cpan:

cpan install Test::Parser

cpan install Test::Presenter

Install from source1:

svn co https://svn.sourceforge.net/svnroot/crucible/Test-Parser

cd Test-Parser

perl Makefile.PL

make

make install

svn co https://svn.sourceforge.net/svnroot/crucible/Test-Presenter

cd Test-Presenter

perl Makefile.PL

make

make install

1You can use subversion or git to check out the source code.

Build the C stored functions

cd storedproc/pgsql/c

make

make install

Note: The make install command needs to be run as the ownerof the PostgreSQL installation, and the PostgreSQL installation tobe used must in the path.

Setting up the user environment

export DBNAME=dbt2 # Defines the database name to use.

export PGDATA=/var/lib/data # Defines where the

# database will be built.

export DBT2TSDIR=/mnt/dbt2 # Defines where additional

# mount points are for use

# with tables spaces.

Build the database

Build a 1 warehouse database, generating the load file in/tmp/datafile:

dbt2-pgsql-build-db -d /tmp/datafile -g -w 1

Again but use the -b flag to load the tables and build the indexesin parallel: /tmp/datafile.

dbt2-pgsql-build-db -d /tmp/datafile -g -w 1 -b

Yet again but use the -t flag to use the predefined locationspecified to use tablespaces:

dbt2-pgsql-build-db -d /tmp/datafile -g -w 1 -b -t

Note: If you have a database built already, the -r flag is need todrop the database first. Also if data files are already created, dropthe use of the -g flag so they are not generated again.

Tablespace mount points for tables

In the ${DBT2TSDIR} directory, disk devices can be mounted atthe following points:

◮ customer - The customer table.

◮ district - The district table.

◮ history - The history table.

◮ item - The item table.

◮ new order - The new order table.

◮ order line - The order line table.

◮ orders - The orders table.

◮ stock - The stock table.

◮ warehouse - The warehouse table.

Tablespace mount points for indexes

In the ${DBT2TSDIR} directory, disk devices can be mounted atthe following points for the primary key indexes:

◮ pk customer - The customer table primary key index.

◮ pk district - The district table primary key index.

◮ pk item - The item table primary key index.

◮ pk new order - The new order table primary key index.

◮ pk order line - The order line table primary key index.

◮ pk orders - The orders table primary key index.

◮ pk stock - The stock table primary key index.

◮ pk warehouse - The warehouse table primary key index.

For the rest of the indexes:

◮ index1 - An index on the orders table.

◮ index2 - An index on the customer table.

Run a test

Run a 2 minute test (120 seconds) against a 1 warehouse databaseand save the test results to /tmp/test.

dbt2-run-workload -a pgsql -d 120 -w 1 -o /tmp/test

Results!

If the Test::Parser perl module is install and the test ransuccessfully you’ll see results formatted something like:

Response Time (s)

Transaction % Average : 90th % Total Rollbacks %

------------ ----- --------------------- ----------- --------------- -----

Delivery 3.97 11.886 : 13.391 44959 0 0.00

New Order 45.34 10.673 : 11.887 513109 5054 0.99

Order Status 4.01 10.408 : 11.667 45409 0 0.00

Payment 42.66 10.396 : 11.688 482790 0 0.00

Stock Level 4.01 10.278 : 11.539 45352 0 0.00

------------ ----- --------------------- ----------- --------------- -----

8482.31 new-order transactions per minute (NOTPM)

59.3 minute duration

0 total unknown errors

1041 second(s) ramping up

Generating a report

Generate a report with charts2:

dbt2-generate-report --indir /tmp/test \

--outdir /tmp/test/report

An index.html file will be created in/tmp/test/report/index.html

2Test::Presenter and gnuplot must be installed.

Generating a report with tablespaces

To determine how to name your devices in the comma separatedlist for each flag, look to see how the devices are named by iostat.For example:

dbt2-generate-report --indir /tmp/test \

--outdir /tmp/test/report

--log cciss/c0d4,cciss/c0d5

Transaction log device:

◮ --log

Flags for plotting specific i/o data for tables

Table devices:

◮ --customer

◮ --district

◮ --history

◮ --item

◮ --new order

◮ --order line

◮ --orders

◮ --stock

◮ --warehouse

Flags for plotting specific i/o data for indexes

Primary key index devices:

◮ --pkcustomer

◮ --pkdistrict

◮ --pkitem

◮ --pknew order

◮ --pkorder line

◮ --pkorders

◮ --pkstock

◮ --pkwarehouse

Other indexes device:

◮ --index1

◮ --index2

Materials Are Freely Available

PDF

◮ http://www.slideshare.net/markwkm

LATEX Beamer (source)◮ http://git.postgresql.org/?p=~markwkm/performance-tuning.git

Time and Location

When: 2nd Thursday of the monthLocation: Portland State UniversityRoom: FAB 86-01 (Fourth Avenue Building)Map: http://www.pdx.edu/map.html

Coming up next time. . .

Linux file system testing using fio3.

__ __

/ \~~~/ \ . o O ( Thank you! )

,----( oo )

/ \__ __/

/| (\ |(

^ \ /___\ /\ |

|__| |__|-"

3http://brick.kernel.dk/snaps/

Acknowledgements

Haley Jane Wakenshaw

__ __

/ \~~~/ \

,----( oo )

/ \__ __/

/| (\ |(

^ \ /___\ /\ |

|__| |__|-"

License

This work is licensed under a Creative Commons Attribution 3.0Unported License. To view a copy of this license, (a) visithttp://creativecommons.org/licenses/by/3.0/us/; or, (b)send a letter to Creative Commons, 171 2nd Street, Suite 300, SanFrancisco, California, 94105, USA.