db-01 upgrading to openedge ® practices & initial tuning tom harris, managing director, rdbms...

DB-01Upgrading to OpenEdge® Practices & Initial Tuning

Tom Harris, Managing Director, RDBMS Technology

2 DB-01 Upgrading to OpenEdge® Practices© 2005 Progress Software Corporation

Audience Survey

V7-V8.1 Users ? V8.2 or 8.3 Users? V9 Users? OpenEdge 10 Migration done? OpenEdge 10 Migration in the next 6

months?


Why Are You Going to OpenEdge 10?

Start building new Integratable applications? Performance ? Data Types ? New Language Features (JTA, ProDataSet)? High Availability (online operations) Large Databases ?

“yeah, we do that ...”


Agenda

Conceptual Differences V8-V9-OE10 The “5-minute rule” What happens during the 5-minutes ? Staging A Move Moved – What Tuning Should I Know About? Questions for the Developers


Unreleased Feature Information

Caution:Caution:

Features labeled “10.1A” are presently in Features labeled “10.1A” are presently in development and may possibly encounter development and may possibly encounter problems that prevent their availability in the problems that prevent their availability in the release.release.

10.1A work is in red. 10.1A work is in red. If red doesn’t work for you, If red doesn’t work for you, we’ll also underline it.we’ll also underline it.

Until they are available – they are “futures”!Until they are available – they are “futures”!


D I S C L A I M E R

Under Development

This talk includes information about potential future products and/or product enhancements.

What I am going to say reflects our current thinking, but the information contained herein is preliminary and subject to change. Any future products we ultimately deliver may be materially different from what is described here.

D I S C L A I M E R


Conceptual Differences

Release Contention Data Index Schema Chgs

V7

V8

V9

OE10

DB level Mixed Synchronous Off-Line

DB level Mixed Off-Line

Area level by area On-Line

Object level cluster Optimistic On-Line

Schema

Basic 4GL

+SQL Schema

+audit schema in 10.1A

Basic 4GL

Synchronous

Synchronous


Conceptual Differences (cont.)

Release CLOBS

V7

V8

V9

OE10

Add Extents

NO Off-Line

NO

SQL

BLOBS

YES

NO

NO

NO

YES

Off-Line

Off-Line

ON-Line (10.1A)

>2GB files

NO

YES 9.1D

NO

Ent.DB

Datetime

NO

NO

SQL

SQL, 4GL


Conceptual Differences (cont.)

Release Threads

V7

V8

V9

OE10

NONE

SQL-92, dbtool

SQL + utilities

NONE

Replication

triggers,ai logs

triggers, logs, Fathom Repl

utility 10.1A

*

AI Mgt

FailOver Cluster

user script

IntegratedEnt.DB

triggers,ai logs

triggers, logs, Fathom Repl

user script

IntegratedEnt.DB

user script

user script

user script


Hey! I just need to get over to 10 !

So ... let’s take the fast lane


The 5-Minute Rule

CONV runs in-place, in 5-minutes Mostly, we’re upgrading the schema

No changes to record headers No changes to physical structures

“Before-and-After Backups are good”


What Happens after 5-minutes?

You can run the database !

It will very likely run better than before Batch jobs with big temp-tables get faster

Fragmentation still exists Most of your DBA scripts should work fine

Optimize when you can ...


The Alternative

What if I want more performance than I get with the fast convert ?


You could plan to dump and load... Put your schema in its own area Move to Type II Areas

– Group some tables Similar recordsize Similar update rates Similar access modes

– Some tables need own area High create/delete Very Big Tables

Future focus is on Type II Areas...


Block Sizes

bi Cluster size (1024 – 4096) bi & ai need to match (8K-16K) bi file is sequential access – put it by itself Data Cluster Sizes

– Table areas: 64

– Index areas: 8 Records per block

– block size / mean record size

– Or 128 and forget it!

– 1 for index only areas


Staging A Move Backups – still your friend!

Keep dbanyls & promon data for reference

The usual rules still apply for AI & BI disks

Ditto, for SAN setups

Move to 10 (convert or d/l)

Run UPDATE STATISTICS for SQL

Get that backup

Keep sets of promon scripts as you go


Ready to Optimize Further?

Whether You CONV or Dump & Load, there are more gains to be had


Plan for “full production” Database

Useful Parameters: -spin: (5,000 thru 10,000) * # CPUs -B: As big as you can without paging -hash: -B/4 when –B > 400,000 -bibufs: 25 -Bp avoids impact on production -omsize select count(*) from _storageObject


BIW/AIW– Performs writes to bi/ai– Decreases empty buffer waits

APW– Decreases LRU writes– Reduces checkpoint time– Performs DB buffer pool I/O– Increase until 0 blocks flushed at checkpoint– Decrease if partial bi writes increase– Typically need more if running Direct with I/O

Watch Dog– Monitors connections

Plan for “full production” DatabasePlan for “full production” Database


Buffers: -Bt

Temp Tables use Type II Storage Areas– Fast empty/drop

– Use “empty temp-table <name>”

– Mitigate space overhead with –Bt

– -tmpbsize 1 or -tmpbsize 8 Records per block Create/toss limit Possible wasted space Additional block format overhead


Index Rebuild

Index Rebuild by area, table or schema– proutil <db> -C idxbuild table <table>

Typical Uses:– Complete D&L process

– Bring deactivated indexes online Online index activate coming soon

– Fix corruption or Index reorg Always sort when asked (if you have the

temp space)


Idxbuild – Sort Groups

# of sort groups -SG: 8 thru 64 – Default: 48 A larger -SG performs less sort/merge(s) Databases with many indexes per area use

larger SG values Other values

– -TM 32 (merge buffers)

– -TB 24 (temp block size)

– -T specify a different disk


Index Compaction

proutil <db> -C idxcompact <index> [ n ] Can be run online When to run

– %utilization < 60% and Factor > 2

– Suggest value of 80% to 90%

– 100% may cause expensive index block splits


Dump and Load

Dump and Load Indicators– Scatter factor > 4– Fragmentation Factor > 2

proutil <db> -C dump <table> . –index 0– Dump performance over data order– Typically choose index based on read order

proutil <db> -C load <table>.bd build– Load with build indexes– Load to truncated area

(truncate rather than “emptied”)


Promon Scripts

Use promon and sar or perfmon

– I/O rates, CPU usage and system paging

– Buffers flushed at checkpoint

– Time between checkpoints

– Buffer Hit ratio

– Contention (buffers, locks, latches)


Among new 10.1A10.1A DB capabilities...

Logfile scanners: format is changing

– scan for error number – still works OK

– scan for message text – still works OK

Monitor & add extents on-line if needed

Capture important DB events for analysis

– you can enable this capability

– it’s useful for operations review/support

Default Enterprise install performs better


10.1A: Selective Events – logged to table

startup info

shutdown info

abnormal ends

locktable ovfl

bigrow

bulkload

roll forward

force options

dbrpr

<more>

dbtool

probkup

procopy

resolve limbo tx

prorest

prostrct (most)

2pc bgn/end

2pc mod/rcvr

protrace files

<more>

proutil -EnableKeyEvents


Default Parameters: 10.1 A changes

–aibufs number of after image buffers

–B blocks in the database buffer pool-bibufs before image buffers-G seconds before bi cluster reuse-L number of locktable entries-Ma # remote users per database server -Mm max size of TCP/IP message fragment


Default Parameters in 10.1A (cont.)

-semsets number of sets avl. to broker-spin latch tries before pausingDB block sizebi cluster size (on-disk block size for notes)bi block I/O block size writing to bi fileai block I/O block size writing to ai file


Summary

There are important changes in the RDBMS You can get over to OpenEdge10 easily

– There is immediate benefit

– There is also a lot more to gain! It’s easy to stage a move in several steps Have a plan for “full production” Check promon/SAR as you take each step Until we ship it, it’s a Future


Questions?

Storage Engine Crew


Thank you for your time!

Storage Engine Crew

db-01 upgrading to openedge ® practices & initial tuning tom harris, managing director, rdbms...

Documents