blue phoenix idms migration
DESCRIPTION
Migrating IDMS based applicationsTRANSCRIPT
www.bphx.com
BluePhoenix DBMSMigrator for IDMSApril 2010
Avishai Shafir – VP MarketingGil Shalit - IDMS Migration Manager
2
A Full Migration Solution
Remodel the data structures from IDMS to a relational DBMS.
Convert the programs to be compatible with new RDBMS.
Translate DC and ADS/O to a variety of targets.
Migrate data from IDMS to RDBMS.
3
Migration Concept
Fully automated
Automated database remodeling – network DBMSinto normalized relational DBMS
Automated software conversion – rule-based process - with refresh capabilities
Automated data migration – generation of data migration programs, allowing customer-defined integrity checks and data transformations
4
Target Systems
Target PlatformsMainframe
UNIX / LINUX
Windows + .Net
Target RDBMSDB2 (7 8 or 9)
Oracle
SQL Server
Target Online EnvironmentCOBOL CICS
J2EE
.Net
5
TP and Batch Emulation Targets
MicroFocus Server
Clerity UniKix & BPE
IBM TXSeries
Oracle Tuxedo A.R.T. (Metaware)(We have been approached by Oracle to integrate Tuxedo A.R.T API support in our tools,
starting with the IDMS migration toolset)
6
Refresh
Testing & Tuning
Data Migration
Code & JCL Conversion
Detailed Analysis & Remodeling
Global Assessment &
Planning
Acceptance Testing
IDMS Migration Project Methodology
7
Remodeling
The database remodeling is rules based
Transformations are created for each IDMS entity according to the following:
IDMS RDBMS
Records Tables
Items Fields
CALC Keys Primary Keys
Sets Foreign Keys
System Sets Indication Fields
8
BanksId HQ
BranchesId Adrs B-Id
CustomersId Adrs B-Id Br-Id Yng
BankBank-ID
HQ-Address
---
--- BranchBranch-ID
Address
---
---
CustomerCust-ID
SSN
---
---
B-Br Set
Br-Cust S
et
B-C
ust Set
Young-Cust System Set
CALC Key
Record to Table
Record to Table
Set to FK
Set to FK
Set to Indication
Key to PK
Record to Table Remodeling
9
Database Remodeling – Transformations
Database normalization
Flat Array
Array Table
Set to Foreign Key
Date/Time format
REDEFINE rules
Group/Sub-elements
Initial value for columns
Naming convention
Special transformations: split/merge records and fields
RDBMS
Data Model
IDMS
Structure
Mapping Rules:
Remodeling by Transformations
10
Rule Based Repository
Data MigrationPrograms
SoftwareConversion
RDBMS
Data Model
IDMS File
Structure
Mapping Rules:
Remodeling by Transformations
11
Remodeling Workspace
Load Schema
Setting Configurations
Setting Hints
Automatic Remodeling
Editing Transformations
DDL Generation
12
Although the underlying data structures change, the program logic is untouched.
Only I/O commands are rewritten.
Code Conversion: Keep Program Logic
IDENTIFICATION DIVISION
PROGRAM NAME XYZ
---
COPY IDMS REC1
---
IDMS IO
---
IDMS LOGIC
---
IDENTIFICATION DIVISION
PROGRAM NAME XYZ
---
COPY REC1
COPY REC1_I2D
COPY REC1_D2I
---
SQL IO
---
IDMS LOGIC
---
13
Interface Copybooks
IDMS RECORD XY
02 F1
02 F2
04 F21
04 F22
02 F3 OCCURES 10
---
DB2 TABLE T-XY
TF3
TF1 TF21 TF22
TF1 IDX
CHILD TABLE T-XY-3
01 XY
02 F1
02 F2
04 F21
04 F22
02 F3 OCCURES 10
---
01 T-XY
02 TF1
02 TF21
02 TF22
01 XY-3
02 TF1
02 IDX
02 TF3
MOVE F1 TO TF1
MOVE F21 TO TF21
MOVE F22 TO TF22
--
*ARRAY SECTION
PERFORM UNTIL IDX > 10
---
REC1_I2D
MOVE TF1 TO RF1
MOVE TF21 TO RF21
MOVE TF22 TO RF22
--
*ARRAY SECTION
PERFORM UNTIL IDX > 10
---
REC1_D2I
14
Copybooks Workspace
Generate Copybooks
15
Code Conversion – DML Conversion Rules
IDMS RDBMS
OBTAIN/FINDCalcDBKEYUSINGCURRENTWITHIN
CURSOR/SINGLE SELECTWHERE on CALC FIELDSWHERE on PK FIELDSWHERE on SORT FIELDSWHERE on PK FIELDSWHERE on FK FIELDS
16
Code Conversion – DML Conversion Rules Cont.
DBKEY operations are converted to PK operations.
IDMS RDBMSACCEPT
FROM record CURRENCY
FROM set CURRENCY
MOVE PK TO WS-DBKEY
FETCH PK by FK
RETURN Same cursor as for OBTAIN, PK to WS-DBKEY
MOVE DB-KEY MOVE rec-PK to WS-DBKEY
CONNECT OWNER PK to FK
DISCONNECT NULLIFY FK
17
Code Conversion – DML Conversion Rules Cont.
DBKEY operations are converted to PK operations.
IDMS RDBMS
Store Insert
Modify Update Using PK
Erase Delete Using PK
18
MOVE
IDMS I/O layer - Read
WS record (IDMS format)
RDBMS
Host variables
New I/O layer - SELECT
IDMS
IDMS program Relational program
Program logic
Code Conversion - Architecture
19
Software Conversion
Add Program
Automatic Conversion
Manual Fixes
DBFA Graphs
Refresh
20
DB Flow Analysis (DBFA)
A Simple conversion would result in opening un-necessary or incorrect cursors which do not take into account the context of the commands.
The DBFA process analyzes the program flow and decides on the required cursors and on the conversion of each DML command to one of those cursors.
The advantage: Automatic analysis is used to automatically optimize the new IO layer, so that performance is improved.
21
DB Flow Analysis (DBFA)
Cursor opened before the first fetch and closed when it is not needed
Cursors Open and Close operations are moved out of loops where possible.
Connect and Disconnect operations are performed only when needed.
Prefer simple cursors (FK=:pk) over paging cursors (FK=PK AND ORDER>:current-hv)
FIND/OBTAIN CURRENT analyzed to find if the command is necessary for SQL operation
Smart Select is used to list in the SELECT clause only the fields that are used in the program.
22
Inline SQL Vs. IO Modules
Inline SQLStandards based SQL solution
Context based SQL optimization (DBFA)
Not an IDMS emulation
No IDMS skills required to maintain and further develop.
DB access changes are localized – no need for impact analysis and across the board testing.
No maintenance “surprises” after go-liveI/O Modules solution produces a maintenance nightmare
University of Miami could not maintain their migrated IDMS/ADS migration.
The reason was that the IO module based migration produced un-maintainable code with changes made to one part affecting the whole code base.
BluePhoenix does offer an IO module based alternative to customers who prefer a lower price.
23
DECLARE CURSOR
SELECT/FETCH
INSERT
Code Conversion by Fixes
Program logic……
OBTAIN
Program logic……
STORE
Program logic……
Add Fix
Source Program Fix List (internal repository)
Modify Fix
Modify Fix
Enabling Automatic Refresh
24
Loading the new source code into DBMSMigrator
Performing an automated comparison between the original and new source code, which makes all needed adjustments:
If code was moved, the corresponding DBMSMigrator code fix is moved to the new location
If code was deleted, the corresponding DBMSMigrator code fix is deleted
If code was added, DBMSMigrator adds the appropriate “automatic” code fix.
If a fix was not located automatically, it is kept in the repository for possible use.
Automatic Refresh – Freeze Friendly
25
Data Migration
A full set of programs is generated automatically for the migration of IDMS data to the RDBMS database
The remodeling rules are used to automatically generate the programs to unload, validate, transform and upload the data.
Advantage: The automatic creation of the program results in a highly efficient and accurate migration of the data. All the data that is implicit in the IDMS structures (Set Order, Owner Pointers etc.) is preserved automatically, to ensure identical operation of the converted programs.
26
Data Migration
Automatically generate an application to:
Unload – Legacy data to flat files
Validate - all numeric and date fields are valid
Transform - apply the rules defined in the data model repository
Load - Load Cards and JCLs to load the transformed data into the RDBMS
27
Online application – Translation and Migration
Possible Targets:
Web based application with logic in Java and JSF pages. A Java framework replaces all ADS/O functionality, as well as COBOL structure handling.
COBOL CICS application including COBOL framework to replace all ADS/O functionality. Valid both on Mainframe and CICS emulation on open systems
COBOL.Net based application with logic in COBOL.Net DLLs, Web-Service based communication and HTML or VB.Net thin client
All targets are under production (COBOL.Net ending customer testing)None of the solutions involve any black box components
28
Online application – CICS application
ADS Dialogs and applications are translated automatically to COBOL CICS with optimized inline SQL
The translated applications in the CICS environment eliminate the need for the ADS infrastructure: The ADS implicit logic is fully covered in the translated application
The translation generates a COBOL-CICS program in pseudo-conversational mode. Meaning, SEND MAP is always followed by RETURN TRANSID.
Return point logic is generated in order to allow continuing from the same place where the program last left (i.e. in the translation of ADS/O LINK command).
ADS/O response processes are generated as sections, performed according to the user selected key.
Automatic Editing and Error handling (validation) is implemented in a generic Edit Routine
29
ADS/O Components Conversion (CICS)
ADSO/DC CICS
Task code Transaction ID
Dialog
Process
Cobol program Pseuso-converse mode
Section
OLM (Online mapping)
Extended attributes
Automatic editing / Edit &Code tables
BMS
Code
Subroutine
Built in functions Subroutines
Queue / Scratch Management Subroutine (Oracle)
Work / Map records Copybooks
ADSA runtime flow of control Subroutine (navigation rules in Oracle)
30
Online application – WEB-Based
OLM maps converted to JSF pages Keep similarity to original display, with better look and feel
Automatic Editing and Error handling are implemented in a Java Framework
ConversationalNo need for return-point management
Dialog Stack (nesting of running dialogs)Implemented as a link-list of DialogRunner objectsSupervised by a control thread
Queue is implemented as special DB tables
Scratch is implemented as memory Hash Map
31
Generated Components and the Framework
Converted
DialogsJSF Pages
Converted
Work records
Helper
Records
ADS2J FrameworkFacelets
Components
Resources
Generator
32
JSF generation
XHTML pages
Java beans, descendants of PageStatebasefor data binding
JSF generator
OLM Maps
Input
Output
Screen template
Faceletscomponents
+
33
Translated ADS/O Screen to Web Based
34
Case Study – Volvo Finance
Medium Sized IDMS ADS/O application
Migrated to UDB Database on Linux
Web Based Online application
Microfocus COBOL Batch
Good Performance
Reduced TCO
Happy Customer
35
Case Study – Wisconsin DWD
Medium Sized IDMS application
Migrating to COBOL CICS DB2 on Mainframe
Beat IBM on price by more than a factor of 4
Beat Ateras in the technical score by a factor of 2 as well as on price by a small margin
36
Tool-based, repository-driven solution
Rule-based engine promotes consistent, error-free conversion and migration
Powerful and flexible remodeling capabilities.
No change to program logic and business rules
Automatic Optimization (DBFA)
Flexible online environment options
Automated refresh makes for a “freeze friendly” conversion
Strong project management capabilities
Proven approach/methodology that reduces time, cost, and risk
Solution Advantages
