db-03: a tour of the openedge™ rdbms storage architecture richard banville technical fellow

DB-03:A Tour of the OpenEdge™ RDBMS Storage Architecture

Richard BanvilleTechnical Fellow

2 DB-03: The OpenEdge™ RDBMS Storage Architecture© 2005 Progress Software Corporation

What’s New:Type II Storage Areas

Block clustering– Alleviates object fragmentation

– Improves I/O efficiency Concurrent Space Allocation

– Area space Allocation

– Object space allocation Improved object manipulation

– Table scan without index

– Fast object deletion


Agenda

Physical Layout

Advantages

A Usage Example

Summary


Type II Area Block Clusters

Block Cluster: 8, 64, or 512 adjacent blocks Configured in .st file Applied via prostrct Fixed size for area Unit of space allocation for objects Blocks within cluster are “non-social”


Type II Area ObjectsObject are made up of one or more block clusters

(Objects: tables, indexes, lobs, area control)


Type II Area ObjectsClusters chained together for fast access

(Objects: tables, indexes, LOBS, area control)

Allows table scan without an indexAllows fast table delete


What Else Is Different w/Type II

Area HWM increased a cluster at a time– More efficient block formatting– Concurrent Space Allocation

Database extend– MAX(64, cluster size)

Other– Reduced fragmentation and scatter– Allows more advanced tools to be designed


Object Block (Type I Area):Space Allocation Chains

MixedObjectBlock

MixedRec

Block

MixedRec

Block

MixedRec

Block

MixedRec

Block

FreeBlock

FreeBlock

FreeBlock

FreeBlock

Record Free Chain

IdxdelBlock

IdxdelBlock

IdxdelBlock

Unique indexes

Only

Total blocks, HWM


Object Block (Type II Area):Area Free Space Allocation Chains

AreaObjectBlock

FreeBlock

FreeBlock

FreeBlock

FreeBlock

Area Control Object

Total blocks, Cluster HWM

Free Cluster List (for Area)


Object Block (Type II Area):Space Allocation Chains

IndexObjectBlock

RecFreeBlock

RecFreeBlock

RecFreeBlock

RecFreeBlock

FreeBlock

FreeBlock

FreeBlock

FreeBlock

IdxdelBlock

IdxdelBlock

IdxdelBlock

Unique indexes

Only

Table 2ObjectBlock

RecFreeBlock

RecFreeBlock

RecFreeBlock

RecFreeBlock

Table 1ObjectBlock

Total blocks,

Cluster HWM

Total blocks,

Cluster HWM

Total blocks,

Cluster HWM


This Is Important Stuff

Indexes– Index delete chain maintenance

– Index blocks are clustered together Mixed Areas

– Vast improvements multi table or multi index areas

– Won’t have index and record blocks intertwined Reduced fragmentation and scatter

– Fewer Dump and Loads! More efficient I/O


Record Space Allocation

Record Packing Factor– Records stored variable length

– Space allocated from record block free chains

– RPF regulates record block free chain content Definitions

– Records per Block: (Blocksize / mean rec size)

– Create limit: rec block free space required for a created record’s expansion. Also the minimum rec fragment size. (75/150)

– Toss limit: free space required for a rec block to remain on the record block free chain (150/300)


Changing RPF

Records Per Block– Changed via .st file

– Granularity Value per area

Create/Toss limits– _proutil <db> C setTableTossLimit <table> value

– Granularity Values per area in Type I storage area Values per object in Type II storage area


RPF Suggestions

Change Create Limit if:– Fragmentation occurs due to record updates

of newly created records

– You expect to see 1 fragment but get 2 Change Toss Limit if fragmentation occurs

due to record updates of existing records– You expect 1 or 2 fragments but get 3 or 4

Coordinate with RPB– (Blocksize / mean rec size)

Don’t change if you have no reason to


Overall Type II Layout

Area

Control

Object

Object #1

Object #2

Cluster Free List

Object #3

Area Data (Free & in use)


Agenda

Physical Layout

Advantages

A Usage Example

Summary


Other Advantages of Type II Storage Areas

Block level Check sum– Identifies corrupt blocks prior to data change

– Allows for larger I/O in future Storage

– Rowids stored as 64 bits

– Variable length block header size Object information stored in block header

– Allows for improved maintenance

– Improves database repair operations


Storage Management Advantages

Improvements through organization Efficient block formatting

– Cluster at a time

– Fewer bi/ai notes written Bottlenecks Resolved

– Concurrent space allocation

– Optimistic buffer and index locking protocols Migration Path

– Can use both Type I & Type II in same database


RootLevel 1

Level 2

Level 3(Leaf)

- Leaf entries contain pointer to record

Records

- Cursor maintains info or last key accessed

Table Scan via B-tree


Select * from Customer;

- Cursor maintains info of last record accessed

- I/O Sequential through cluster

2nd Cluster

1st Cluster

3rd Cluster

4th Cluster


Fast Object Delete

Area

Control

Object

Object #1

Object #2

Cluster Free List

Object #3



Fast Object Delete

Area

Control

Object

Object #2

Cluster Free List

Object #3



OpenEdge 10 Temp tables

Released in 10.0b– Fast delete– Fast delete/create on empty

Enhanced in 10.0b02– Avoid delete/create– Avoid I/O when formatting

Hybrid Type I & II Storage Area– Index Objects Type I– Other Objects Type II– 8 Block Clusters

Suggestions– “empty temp-table <name>”– -tmpbsize 1, -tmpbsize 8– Better performance with increased –Bt


Agenda

Physical Layout

Advantages

A Usage Example

Summary


Best Practices for Use

Physical– Include Striping (RAID or do it yourself)– File extent Location

Schema– Separate index and table data– Multi table area for small, medium & large records– Records per block properly set for each area

Growth– Always have a variable length extent– Enable large files


Location, Location, Location

b /bi/exampleDB.b1 f 1024000


b /bi/exampleDB.b3

#

d "Schema Area":6,64 /db/exampleDB.d1

#

d “Customer Indexes":7,1;8 /db/exampleDB_7.d1 f 512000

d “Customer Indexes":7,1;8 /db/exampleDB_7.d2

#

d “Customer Data":8,128;64 /db/exampleDB_8.d1 f 1024000

d “Customer Data":8,128;64 /db/exampleDB_8.d2


Cluster Size



b /bi/exampleDB.b3

#


#



#




Records Per Block



b /bi/exampleDB.b3

#


#



#




Multi Object Areas

d “Large Record Indexes":9,1;8 /db/exampleDB_9.d1 f 512000

d "Large Record Indexes":9,1;8 /db/exampleDB_9.d2

#

d “Large Record Tables":10,16;64 /db/exampleDB_10.d1 f 1024000

d “Large Record Tables":10,16;64 /db/exampleDB_10.d2

#

d “Small Record Indexes":11,1;8 /db/exampleDB_11.d1 f 512000

d “Small Record Indexes":11,1;8 /db/exampleDB_11.d2

#

d “Small Record Tables":12,256;64 /db/exampleDB_12.d1 f 1024000

d “Small Record Tables":12,256;64 /db/exampleDB_12.d2


Cluster Size:Fast Growing Tables

d “Misc Indexes":13,1;6464 /db/exampleDB_13.d1 f 512000

d “Misc Indexes":13,1;6464 /db/exampleDB_13.d2

#

d “Fast Growing Tables":14,64;512512 /db/exampleDB_14.d1 f 1024000


d “Fast Growing Tables":14,64;512512 /db/exampleDB_14.d3

#

a /ai/exampleDB.a1 f 51200



…


Records Per Block:Fast Growing Tables



#




#




…


AI File Location



#




#




…


In Summary

More efficient layout Better Performance Migration path Foundation for the future


Questions?


Thank you for your time!


OpenEdge 10 RDBMS Advanced Storage Architecture

The following Progress courses cover related subject matter. Please visit: www.progress.com/education for course descriptions and relevant curriculum maps.

Database Administration

db-03: a tour of the openedge™ rdbms storage architecture richard banville technical fellow

Documents