ims05 ims v13 haldb alter and db vers - ims ug june 2013 syndey

© Copyright IBM Corporation 20133.3.1

© Copyright IBM Corporation 2013

IBM Software Group

Rick Long – IMS LVL2 Database Team

&

Solutions Test Team

®

V13 HALDB ALTER and DB Versioning

[email protected]


Disclaimer

2

© Copyright IBM Corporation 2009. All rights reserved.U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.

THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL PURPOSES ONLY. WHILE EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE INFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED “AS IS” WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. IN ADDITION, THIS INFORMATION IS BASED ON IBM’S CURRENT PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO CHANGE BY IBM WITHOUT NOTICE. IBM SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION. NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, NOR SHALL HAVE THE EFFECT OF, CREATING ANY WARRANTIES OR REPRESENTATIONS FROM IBM (OR ITS SUPPLIERS OR LICENSORS), OR ALTERING THE TERMS AND CONDITIONS OF ANY AGREEMENT OR LICENSE GOVERNING THE USE OF IBM PRODUCTS AND/OR SOFTWARE.

IBM, the IBM logo, ibm.com, IMS, IBM High Performance Pointer Checker for z/OS and CICS are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol (® or ™), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at “Copyright and trademark information” at www.ibm.com/legal/copytrade.shtml

Other company, product, or service names may be trademarks or service marks of others.

© Copyright IBM Corporation 2013 3

HALDB Alter

● Background– Customers face the challenge of maintaining online database

availability while making structural changes to a HALDB database– Customers cannot make structural changes to a Full Function HALDB

without taking the database offline to unload/reload● Solution

– Structural changes can be made to DBD segment definitions – Online reorganization command must be issued to apply the structural

change to the HALDB database– Online change command must be issued to activate the new ACBLIB

member● Benefit

– Make structural changes to a database while maintaining database availability


4

HALDB Alter Prerequisites

● Software requirements– CSL

• SCI• OM

– DBRC RECON MINVERS (‘13.1’)● Hardware requirements

– Same as base IMS 13


5

HALDB Alter Overview

●Structural changes can be made to HALDB databases while the database remains online– Add a new field to space at the end of an existing segment– Increase the length of an existing segment– Define new fields that remap existing fields and space in an existing

segment– Change CI size allowed when making a DBD change

●HALDB Alter is only supported for database types: – PHDAM – PHIDAM


6

FIELD 1

FIELD 1

FIELD 2

New SpaceFIELD 2

Increase the length of an existing segment

Example 1

Example 2

FIELD 1

FIELD 1 FIELD 2

FIELD 2

New Space

Space

Space


7

DFS1849E - Reason codes

1. The access method of the database is changed.2. The randomizer changed.3. The exit routine changed.4. The number of segments changed.5. The number of database data set groups changed.6. The length of a logical child or logical parent segment changed.7. The segment level changed.8. The segment prefix changed.9. The segment changed from fixed-length to variable-length or vice versa.10. The insert rule changed.11. The delete rule changed.12. The segment code changed.13. The length of the concatenated key changed.14. A logical relationship was added or changed.15. The segment length was decreased, which caused an existing field to be deleted.16. The field was deleted, moved to another segment, or its name was changed.17. The key length of the field changed.18. The field length changed.19. The value of the TYPE keyword on the FIELD statement changed.20. The new field was added between existing fields in the segment. 21. The parent of the segment changed.22. The segment length was decreased


HALDB Alter DBRC Preparation (cont’d)

CHANGE.PART Command– ALTERSZE(nnnnn) | NOALTRSZ

• Mutually exclusive, optional keywords • ALTERSZE(nnnnn)

– Specifies new OSAM block sizes or VSAM CI sizes for the output partition data sets

– Specify up to 10 values, one for each data set group defined in the DBD• Omitted values remain unchanged• Numeric values must be even and no greater than 32K

– New sizes must be stored in the RECON partition record before the INITIATE OLREORG OPTION(ALTER) command is issued

– New sizes are stored in the RECON partition record until the alter operation is complete and an online change is performed

– After online change is performed, new OSAM block sizes are saved in the OSAM BLOCK SIZE field of the RECON partition record


9

HALDB Alter Overview (cont’d)

●After changes are made to the DBD segment definition (DBDGEN and ACBGEN) – Type-2 Online Reorg command is used to alter a database from the

current structure to the new structure

• Type-1 command /INITIATE OLREORG does not support altering a HALDB

• All partitions are OLR’ed (max 10 at a time) (rate =100%)

– Modified Online Change is used to bring the changed ACBLIB DBD member online

• DBR required to bring in the change (unlike OLC without ALTER)

– Application programs can start using the new database structure


10

Change Timeline

Start

DBDGEN

INIT OLRALTER

Programs use OLD SEGMENT Format

MemberOLC

Part 1complete

Part 2complete

Part Lastcomplete

DBR

OLR runningA – M data sets activeCursor activeStaging ACBLIB dynamically allocated for target definitionAll updates loggedSecondary indexes “healed” as neededALL programs receive “old” segments in IOAREAMEMBER OLC for changed DBDALL programs receive “new” segments in IOAREA

Programs useNEW SEGMENT Format

ACBGEN

Application Unavailable


11

Change Timeline

Start

DBDGEN

INIT OLRALTER


MemberOLC

Part 1complete

Part 2complete

Part Lastcomplete

DBR

Options to ConsiderDB Quiesce to create recovery point for fallbackIC after OLC for recovery point

Recovery would use SLDS only before this, off-site?Secondary indexes rebuild?

Need to recycle MPR for new program load units if segment lengths have changed


ACBGEN


DB QUEISCEDB IC

MPR recycle


HALDB Alter Overview (cont’d)

●Why use HALDB ALTER?– How long does the unload/reload take

• Application down time limited to ACB change• 10 partitions vs 300 partitions

– No time limit from the completion of the last OLR to OLC• Can leave until change window

– Make use of the ALTER CI size if change does not affect segment lengths• Adding FIELD to existing space

– If you define the FIELD statements• IMS will initialize the fields to valid values• Avoid 0C7 abends

12


13

HALDB ALTER Process

● I need to tell you the change process before I can fully explain what you can change when you also use versioning– Need to set the sequence of events so I can explain what is happening

and why

●Extremely powerful when used in conjunction with DB Versioning


14

Current DB fallback process

1. Chg DBD source & DBDGEN to chg.DBDLIB

2. DBR DB from online3. STOP application to avoid abends4. Run IC to create backup for fall back5. Run HD Unload from prod.DBDLIB6. Copy chg.DBDLIB TO prod.DBDLIB7. Run HD Reload (plus extras like IB…)8. Run IC 9. Run ACBGEN and OLC for DBD 10.Recycle MPRs so new progs ready to run11.Start DBD, PROG,TRANS…. For testers12.Run a few trans for verification13.Start whole Application


15

Current DB fallback process

1. Chg DBD source & DBDGEN to chg.DBDLIB

2. DBR DB from online3. STOP application to avoid abends4. Run IC to create backup for fall back5. Run HD Unload from prod.DBDLIB6. Copy chg.DBDLIB TO prod.DBDLIB7. Run HD Reload (plus extras like IB…)8. Run IC 9. Run ACBGEN and OLC for DBD 10.Recycle MPRs so new progs ready to run11.Start DBD, PROG,TRANS…. For testers12.Run a few trans for verification13.Start whole Application

DBR DBRecycle MPRsRecover DB to HEREOLC to inactive ACBLIBRestore prod.DBDLIBStart APPL and Go


16

HALDB Alter Segment Field Fill Values

●For fields added at the end of a segment– If TYPE ‘X’

• Fill with x’00’

– If TYPE ‘P’ • Fill low order byte with x’0C’ and other bytes with x’00’

– If TYPE ‘C’ • Fill with x’40’

●For space added to a segment without field definition(s)– Fill with x’00’


17

FIELD 1 FIELD 2 SpaceNewFIELD 3

Add a new field to space at the end of a segment

FIELD 2 SpaceFIELD 1

Can be done by program Copybook change only

Old programs will initialize fields to blanks or binary zeros

Advanatages of DBD Change:Fields filled with correct values on ISRT

Disadvantages of DBD Change:Requires ACBGEN/change



Database Versioning

FF ,HALDB & DEDBDatabases


Database Versioning Overview

● Database Versioning provides the ability to assign user-defined version identifiers to different versions of a database structure– Enables structural changes to a database while providing multiple

views of the physical IMS data to application programs

● Applications referencing a new physical database structure can be brought online without affecting applications that use previous database structures– Applications which do not require sensitivity to the new physical

structure, do not need to be modified and can continue to access the database


Database Versioning Overview (cont’d)

● Database Versioning supports the following database types– DEDB– HDAM– HIDAM– PHDAM– PHIDAM

● Database Versioning only supports the following database structure changes– For all supported database types

• Increasing the length of a segment

• Adding a new field to space at the end of a segment

– For Full-Function and HALDB database types • Adding new field(s) that remap existing field(s) and space at the end of a segment


● Database Versioning must be enabled

● Versioning is at the DBD level– Users define the version of a database definition on the DBD – Version numbers must be maintained in incremented values

● Database Versioning requires enablement of the IMS catalog– DBD definitions for versioned databases must be in the IMS catalog

● Application programs can elect the desired database version– Using the IMS system default for this IMS instance– Specifying the version number on the PCB of the PSB– Specifying the version number on a DL/I INIT VERSION call

Database Versioning Overview (cont’d)


● DFSDFxxx PROCLIB: new DATABASE section keywords– Specify whether database versioning is enabled and default version

• DBVERSION=• DBLEVEL=

● Database and Program Generation Statements– Specify the database version number

• DBD: DBVER=

• PCB: DBVER=– Override the default versioning specified in DFSDFxxx

• PSBGEN: DBLEVEL=● “INIT VERSION” DL/I Call

– Overrides the version number that is specified on the PCB statement– Must be issued before issuing a DL/I DB call to the database

Database Versioning Implementation


DFSDFxxx PROCLIB Member

● New keywords added to DATABASE section– DBVERSION = Y | N

• Enables database versioning• Database versioning is disabled by default

– DBLEVEL = CURR | BASE• Ignored when DBVERSION=N• CURR (default)

– IMS returns data from all databases using the current DBD version, which is the current physical level, unless a specific version is specified

• BASE– IMS returns data from all databases using the lowest DBD version number

retrieved from IMS Catalog, unless a specific version is specified


Database Generation Statement

● New parameter added to DBD Statement – DBVER=n

• Specifies a DBD version number to be associated with a database structure change • Supports ACCESS types DEDB, HDAM, HIDAM, PHDAM & PHIDAM• Numeric values from 1 – 2147483647 (2 Gigs)

.-,VSAM-. >>-DBD--NAME=(dbname1)--,ACCESS=-(--+--+-HDAM---+--+-,OSAM-+--+--)--+----------+------>

' '-HIDAM--' ' '-,DBVER=n-'' '-PHDAM--' '' '-PHIDAM-' '' ''-----DEDB----------------'

...

DBVER= parameter on the DBD statement for specific ACCESS types


Program Generation Statement

● New parameter added to PCB Statement – DBVER=n

• Specifies the version of the DBD to use when accessing the database• Must match a defined DBD version number stored in the IMS catalog• If multiple PCBs within a PSB refer to the same database, each PCB must specify

the same DBD version number• Numeric values from 0 – 2147483647 (2 Gigs)• If not specified, the DBD version used depends on DBLEVEL= parm in the PSBGEN

statement or the DFSDFxxx PROCLIB member

>>-+-----------+--PCB--TYPE=DB--+-,DBDNAME=-+--name-----------------+----------+------> '-label-----' '-,NAME=----' '-,DBVER=n-'

...

DBVER= parameter on the PCB statement


Program Generation Statement

● New parameter added to PSBGEN Statement – DBLEVEL=CURR | BASE

• Specifies the default database version level returned to programs using this PSB and not requesting specific database versions

– DBLEVEL=CURR• PCBs within the PSB which do not specify a DBVER= parm will access

the database using the current physical structure– DBLEVEL=BASE

• PCBs within the PSB which do not specify a database version on the DBVER parameter will use the lowest base version from the IMS catalog

• Overrides the default setting for DBLEVEL specified in DFSDFxxx PROCLIB member

>>-PSBGEN--PSBNAME=-name--+--------------------+------------------------------------->' ''-,DBLEVEL=-+-CURR-+-‘

'-BASE-'...

DBLEVEL= parameter on the PSBGEN statement


Database Versioning Flow

DBJK21DBJK22DBJK23

Application

IMS

DBJK21 V1DBJK22 V1DBJK23 V1

ACBLI B

DBJK21 V0,V1DBJK22 V0,V1DBJK23 V0,V1

IMS Catalog


DBDLIB

PSBJK

PSBLIBPSBGEN

--------------------------PCB--------------------------DBJK21 V0DBJK22 DBJK23 V1

PSB=PSBJK Source

DBJK21 V1

DBD Source

DBJK22 V1

DBJK23 V1

DBDGEN

PSBGEN


DLI

Retrieve DBJK21 from Catalog


Active

ACBGEN

Version “V1” of DBDs put into ACBLIB & Catalog

Database Versioning enabled -> data returned to app at V0 & V1 levels

DBJK21

DBJK22

DBJK23


DBVER 0 program (unchanged)

1,000Waldrugs55555 3,000,0002,000,000

Customer No Customer name Credit bal Credit Limit VISA BALANCEIMS Buffer area

Customer No Customer name Credit bal Credit Limit

Waldrugs55555 3,000,0002,000,000

Program IOAREA

1,000Waldrugs55555 3,000,0001,000,000


1,000,000


DBVER 0 program Insert call (unchanged)

Customer No Customer name Credit bal Credit Limit

Waldrugs55555 3,000,0002,000,000

Program IOAREA

000Waldrugs55555 3,000,0001,000,000



1,000

Customer Account segment

Waldrugs55555 3,000,0002,000,000

Customer No Customer name Credit bal Credit Limit VISA BALANCE

new program B can gets and updates the Credit balance and the new field VISA BALANCE using DBVER=1.

It adds a value to a field in the extended segment

The new fields might not have much to do with some of the current fields and thus the existing programs don’t need to refer to it.



Database VersioningImplementation



● FF and DEDB– Normal Unload and Reload off line processes apply

● HALDB can make use of the HALDB ALTER

● SET system default to DBVER=BASE – All existing programs will see DBVER=0 version of DB

● Prior to segment length change– Programs can access larger segments without any changes– Modify PSB to have DBVER=x and required for each PSB



● Implications to IMS systems– First access for a DB not the CURR version (each IMS system)

• IMS Catalog accessed (tune catalog DB and buffers) • DBD control blocks built based on old segment structures• Backup and recovery of the catalog not just from staging ACBLIB

● IMS Utilities always access CURR DBD view

● SET system default to DBVER=BASE

● All existing programs will see DBVER=0 version of DB– Prior to segment length change– Programs can access larger segments without any changes

● Modify PSB to have DBVER=x and required for each PSB



● Legacy vs CURR program transactions– Modify 10% of legacy programs to read extended segments– Less 90% using the Catalog versions of DBD

• Real storage to hold control blocks• IO for the Catalog • Complexity

● Catalog and ACBLIB backup and recovery– Can’t rebuild the Catalog from the staging ACBLIB as you lose older

version of DBD?

Database Versioning


35

Change Timeline

Start

DBDGEN

INIT OLRALTER


MemberOLC

Part 1complete

Part 2complete

Part Lastcomplete

DBR

OLR runningA – M data sets activeCursor activeStaging ACBLIB dynamically allocated for target definitionAll updates loggedSecondary indexes “healed” as neededALL programs receive “old” segments in IOAREAMEMBER OLC for changed DBDALL programs receive “new” segments in IOAREA


ACBGEN




Database VersioningMy opinion



Versioning Alternative


38



• Mutually exclusive, optional keywords • ALTERSZE(nnnnn)

– Specifies new OSAM block sizes or VSAM CI sizes for the output partition data sets

– Specify up to 10 values, one for each data set group defined in the DBD• Omitted values remain unchanged• Numeric values must be even and no greater than 32K

– New sizes must be stored in the RECON partition record before the INITIATE OLREORG OPTION(ALTER) command is issued

– New sizes are stored in the RECON partition record until the alter operation is complete and an online change is performed

– After online change is performed, new OSAM block sizes are saved in the OSAM BLOCK SIZE field of the RECON partition record


39



• NOALTRSZ– Clears ALTERSZE values from a RECON partition record

• If size values not set before the next ALTER operation starts ->• output data sets are created with the same Block or CI sizes as

the input partition data sets– If NOALTRSZ is specified and ALTERSZE values have not been set ->

• No action is taken


40

HALDB Alter Online Processing (cont’d)

● Issue Type-2 Online Reorganization command INITIATE OLREORG NAME(masterdb) OPTION(ALTER)

– Initiates the dynamic structural change processing for HALDB partitions

– Only one HALDB master database name can be specified per INIT OLR command

– IMS reads the new definitions for the DBD from the staging ACBLIB • Staging ACBLIB is dynamically allocated• Member info used to construct the OLR output data set • Output data sets for all the HALDB partitions are built

– TERM OLREORG or /TERM command is allowed while database structure change is in progress– INIT OLR with OPTION(ALTER) will restart the HALDB ALTER process where it

left off


41

● INITIATE OLREORG Return and Reason Codes

● INITIATE OLREORG Completion Codes

HALDB Alter Return, Reason and Completion Codes

Return Code Reason Code Meaning

X’00000000’ X’00000000’ Command completed successfully.

X’00000010’ X’00004520’ Another OLR Alter in progress.

Completion Code

Completion Code Text Meaning

0 Command completed successfully1E1 OLR ITASK creation failed OLR internal ITASK can not be created1E2 Incorrect HALDB version detected The ddir version for the database is different than the number

recorded in the RECON DB record for the HALDB

1E3 Partition queued for OLR The partition is being queued for HALDB alter processing

1E4 HALDB alter pending for Online Change An alter request against the same HALDB was done, but an Online Change has not yet done for that HALDB

1E5 No DB structure change detected An alter request is made but there are no database structure changes made

1E6 Insufficient CI/Block size detected During alter processing, the CI/Block size of the database data set is smaller than the database largest segment size

1E7 Unsupported DBD changes detected An alter request is made but the type of DBD changes made are not supported

(many more) … …1EC Logical database error During alter processing, an error related to logical relationship on

the altered DBD is detected


42

HALDB Alter Online Processing (cont’d)

● The new database structure will be available for use after the OLR ALTER and OLC processes have completed successfully– Make necessary updates to affected application programs– Take a backup of the active ACBLIB member(s) being changed – Stop access to the altered HALDB database

• Use /DBR or UPDATE command, do not use UPDATE START(QUIESCE)

– Do OLC to complete the alter process for changed ACBLIB member(s)• Member Online Change is recommended

– Reads directly from the staging ACBLIB– Can process specific ACBLIB members requiring activation

INITIATE OLC TYPE(ACBMBR) NAME(acbmember)

– Start access to the altered HALDB database– Until OLC process is complete, application programs accessing the

database receive the unaltered format of the segment


● Content of several DBRC records have changed– DSPDBHRC – Database record (DB)

• no size increase– DSPPTNRC – Partition record (PART)

• size increases only when ALTER is running– DSPRRGRC – Reorganization record (REORG)

• no size increase

HALDB Alter DBRC RECON Records

Remember to reassemble any programs using the changed DBRC control blocks !

ims05 ims v13 haldb alter and db vers - ims ug june 2013 syndey

Technology

copyright ibm corporation

segment length

field length

ibm logo

ibm trademarked terms

current list of ibm

existing segment example

segment level