Steps in a Simplified Approach to Data Migration 

This Page describes a series of Steps in a Simplified Approach to Data Migration betweenSource Systems and Target Systems.It covers the basics of Extracting data from a number of Sources, Transforming it andLoading it into a Target Database.

y  Preparation ...

The only Deliverable from this Simpified Approach is a Word document Tableor a Spreadsheet whichdefines the Mappings between Source and Target Data Items.You will start by defining an appropriate Document or Spreadsheet with theseColumns :-

1.  Source Table (or Filename)2.  Source Column (or Fieldname)3.  Source Data Type4.  Validation Rules.5.  Typical Values.6.  Transformation Rules.7.  Target Table8.  Target Column9.  Target Data Type.

y  Setting-up the Simplified Data Migration Process ...   Optionally choose a Data Modelling Tool with Reverse Engineering

Capability,(such as ERWin).  Optionally define and create the Data Dictionary.

2.  Identify all the required Data Sources, and an 'owner' for each Source. (Data

Feeds, Legacy Systems & Operational Data Stores).

3.  If you want to migrate Access Databases to Oracle,MySQL, Sybase orPostgreSQL, then check out MDB Tools, from Brian Burns.

4.  Define the Data Items required, in consultation with the Users.  Optionally create the Data Models for the Source Data.

5.  Define Data Validation Checks (bottom-up, meaning identify available low-level data, such as Invoice Dates,Amounts,etc. ).

6.  Define Validation Rules, (eg Start Date earlier then End Date).7.  Define Clean-Up Business Rules for Source Data, (eg Set default End Date).

  Optionally carry out an Audit of the Data Quality in major Databases,(bottom-up and top-down).

  Optionally evaluate the benefits of a Data Cleansing Product.8.  Define the Staging Area, with 'Source' Tables to store Extract Files, and

'Target' Tables to store data after transformation.I prefer to have Target Tables that exactly reflect the final Database so that the data can be loaded indirectly,in the knowledge that there is no chance of errors in the final load. 

  Optionally create the Business Data Model for the Consolidated

Database9.  Define the Data Mapping between Source and Target Data Items.

  Optionally create a CRUD Matrix to identify the interactions between

Data and Functions.10. Define Acceptance Tests for data in the Integrated Database.

 

y  Executing the Data Migration Process. 1.  Either use SQL or choose a Data Migration Tool. 2.  Create Extract Files for each Data Source in turn.3.  Carry out Validation Checks against Extract Files.4.  Resolve any Errors with the Users.5.  Create Extract and perform Data Clean-Up.6.  Complete the Migration, (Extract, Transform and Load) for each Data Source.7.  Run Acceptance Tests on the Integrated Target Database,(esp. Referential

Integrity

y  Use Case Templatey  Relate use cases and functional Cross reference requirements

y  Frequency How often is the use case performed?

y  Priority How important is the use case?y  Post-conditions Condition that must prevail after executing the use

case

y  Policies Specific rules that must be enforced by the use case

y  Exceptions What might go wrong during the execution of the use casey  Variation Different ways to accomplish use case actions

y  Precondition A condition that must hold before a use case can begin

y  Overview A brief description of the usage of the process

y  Purpose The intention of the use case

y  Actors Role names of people or external entities initiating the use case

y  Use Case Name Name of the use case