methodology for sap data migration using informatica and confidential methodology for sap data...

Internal and Confidential

Methodology forSAP Data Migration using

Informatica

NON LIVE FEED SCENARIO

For live feed scenario, where site/country data keeps changing on ongoing basis even after go live and needs to be incorporated in target SAP/other

system, please refer to our other presentation.

Agenda

Agenda

What's New Now ?

New Conversion platform Design (AIP)

Screen Shoots of Future AIP Platform

Nomenclature

What’s New

• Benefit Highlightsq We present you a architecture model, that will meet your immediate and long term needsq A architecture that has a proven track record of being one of the best. It has been built over few

years of continuous inputs and acts as great referenceq Eliminate try and try approach by inexperienced resources.

Ø This approach can be costly in terms of time and moneyØ Often this is a continuous ongoing patchy work to existing solutionØ Often becomes more and more unmanageable due to patch works

q Delivery of proven platform will deliver great values and provide cost saving on business front tooq This platform gives the country data owners high level of transparency in term of seeing there data

that gets selected, the data that would get loaded along with errors at each check level.q Save multi millions euros over the next 1-2 years in terms of learning curve, avoiding fixes to

existing work, eliminate waste of time by dependent users to get right reports etcØ Shorten delivery of right platformØ Delivery capability to manage / roll-out more sites at the same time within fixed deadlinesØ Shorten each phase time due to better platform and quick turnaround time for checks and

appropriate reports

NEW METHODOLGYA proven methodology and mature technological architecture to

be used for Data Migration projects by big clients with multiplecountries/sites to be migrated in a phased plan over a period

of time.

What’s New

• New Experienced resourcesq We bring on board new resources with proven experience of working on this large scale SAP

Migration projects. These resources has the needed exposure to plan, design and run data conversion in a controlled fashion

q These resources understand the data dependencies and complications across objectsq They understand the pre-checks and SAP data inclusion to be done prior to object loadsq They are highly methodical and capable of crisis management

• Better Controlq Introduction of new template to manage tasks, progress and issues at SAP object levelq Introduction of new template for enhanced SAP configuration values managementq These templates will deliver the appropriate graphs and figures for management reports

• Financial ROIq This approach is a candidate for projects with more then xx million € budget and more then 1 siteq It can assure 10 upto 30% financial savingsq It can guarantee higher quality data migration and thus ensuring further saving of 10%, which

normally is spent to fix poor data quality migration. This is ensured by this platform approach and further supported by our suggested strategy to manage real time data quality validations.

General Client issues at hand

• Tight deadlines ahead• Enhance reconciliation and error logging capabilities to offer desired level of

data records report to client for each country• Certain enhancements needed for most objects implemented up to S2 level• Final touches needed for objects implemented in S3 level (IDOC writing phase)• S3 Mappings need enhancement for

q Inclusion of SAP existing data wherever needed eg. Contact persons, partner functionsq Inclusion of further checks against SAP data

• Standardize all connection parameters to ensure smooth run across DEV, TEST and PROD system load runs

Certain clients already has a ongoing project; in those scenarios too we come across below mentioned issues to resolve on urgent basis apart from building the future

to be ideal solution

Agenda

Agenda

What's New Now ?



Nomenclature

Team Setup Approach

• Based on add-on of new experience in the team and to meet project goals in time with high satisfaction, below is new setup proposal

SAPSystem Load

& Reconciliation

S1) Basic Mappings for extracting Site data and

map it to object templates

S3) SAP Check & Load Mappings

Highly experienced resources with knowledge of different

SAP data objects and there correlations.They will also design the needed templates

and reconcilliation model.

General Informatica Resources

S2) Mappings for Basic checks &

Data accumulation

Informatica Site Process

LegacySystem Mapping

DataCleansing

FreeFormat

FlatFile to Central Team, via FTP

SiteResponsibility

Off-LineQualityChecksLLLL

JJJJ

•Check KDS values•Check Currency formats, values•Check Date formats•Analyse missing Req field values

•Check 0, Site Mapping•Check 1 Mapping

Informatica Cluster/Central Process

SAPProduction

System

ClusterResponsibility

JJJJ

ChecksChk1Chk2Chk3

Online Quality Checks

LegacySystem Mapping

DataCleansing

FreeFormat

FlatFile

SiteResponsibility

Off-LineQualityChecksLLLL JJJJ

LLLL

OverAll Process Flow

LEGACYSYSTEM SAP

R/3

Target/Source

fileSource

fileLoadData

CLUSTERSITE

Check 1

Check 2

Check 4

Load

Check 6&7

Check 1 errors

Check 2 errors

Review comments on checks

Check 3 Duplicates/Load errors

Sourcefile

Check 3

Site MappingCheck 0

Check 1

Check 5

SQL

Excel Template/

KDS

CONFIGREPO

Sourcefile

ExtractData

ExtractData

LKP

Web Portal- One Point Control Center

Informatica Mappings Central (C)- Team

• Check 1 Mappingq Perform Relevance check (quality checks level 1)q Running this mapping to ensure that all the necessary data fields are available from the data

ownersq Generated separate files for good and error records

• Check 2 Mapping : FD business rules checkq Perform Completeness check (quality checks level 2)q It checks if the data conforms to the business rules as specified in the FD. The business rules

cover, among others, data ranges, data formats and value dependencies. Check completeness of local data with cluster scope, identify incomplete data (e.g. no post code on an address) and missing data (e.g. no date of birth or address for permanent employee).

q Examples:Ø Site X can only have Plants Y and Z.Ø Why do we try to load a PIR for a Material which was not created or extended for you plant.Ø Why do we want to load a PIR when the specified Vendor was not in the Vendor file.

•q Perform duplicates check and generates output file with duplicate recordsq Mapping 3 loads all records which passed Chk-2 into a SQL table.q Assess inner duplicate ratio (percentage of duplicated records within site legacy systems) and outer

duplicate ratio (local data already existing in SAP)q A version of this mapping also deletes all the previously loaded data for this delivery for this specific

site

Informatica Mappings C- Team

• Check 4 Mapping (SAP Load)q These mappings writes the data to IDOCsq Some of these mappings also generate text load files in the required format for loading

into SAP using other tools like LSMW, WinRunner, WinShuttel, ABAP etc

• Check 5 Mapping (SAP_SENT_FLAG)q It takes a file/table with the SQL keys back against the SQL table and set the

SAP_SENT_FLAG to 'Y' to indicate the rows which were sent to SAP for loading.Ø This step also helps capture SAP-NO for common objects like HR load, vendor

create, customer create

•q It is used to reconcile the data loaded into SAP. There is no standard layout for this

mapping. The checks are specified in the BII (business check documents).

•q checks the integrity between data objects in SAP. There is no standard layout for this

mapping. The checks are specified in the BII.

•q These mappings downloads the data from SAP tables on daily basis or regular

intervals.

OverAll Process Flow – Error Reporting

SAPR/3

LoadData

CLUSTERSITE

LoadC

heck 1 errors

Check 2 errors

Review comments on checks (Email)

Check 3 D

uplicates/Load errors

CHECKS SQL CHECKS

Informatica Check Reports (Email)

CTQuality

Checks & Feedbacks

Web Portal-One Point Control Center

Business User / Data owners

Checks & Reviews

& Corrections

Online Status and Error Log access

File Server

Informatica –File Server Overview - Central

• It covers the understanding of the location of in & out files used for the mapping and tracing of them on server; these are primarily the I/O files related to mappings.

• Also we cover the following things in context of specifying the sessions for mappings:

q Speciying of Source files locationq Speciying of Target files locationq Locating error files generatedq Locating duplicates/bad files generated

• In brief, we highlight the tracing of following In & out files types of files:q Source, Target Error, Duplicates Files –Chk1q Source, Target Error Files –Chk1 (C)q Source, Target Error Files –Chk2q Duplicates Files –Chk3

• The initial decision for the source, target locations are specified by the Informatica expert OR the person responsible for administration of your informatica Server.

• The location of the files generated is somewhere on a central server, which has restricted access. It is highly organized by SiteCodes and ObjectNames.

The Informatica I/O are usually structured as:INF_ROOT/

SITE_ID/<<Folders>>

Informatica –File Server Overview - Central

• Source Fileq Your source file is with the same name that you used in FTP name.q In above case it is with <object_name>.out (eg. c002_vendor_partner_function.out)

• Target Files –Chk1q Files generated after the automatic running of the CHK1 central mappings are

<object_name>_chk1.out & <object_name>_chk1_error.outq <object_name>_chk1.out represents the file with correct recordsq <object_name>_chk1_error.out represents the file with error recordsq In above eg. the concerned files are:

Ø c002_vendor_partner_function_chk1.out Ø c002_vendor_partner_function_chk1_error.out


<object_name>_chk2.out & <object_name>_chk2_error.outq <object_name>_chk2.out represents the file with correct recordsq <object_name>_chk2_error.out represents the file with error recordsq In above eg. the concerned files are:

Ø c002_vendor_partner_function_chk2.out Ø c002_vendor_partner_function_chk2_error.out


<object_name>_sql.badq <object_name>_sql.bad represents the file with duplicate recordsq In above eg. the concerned files are:

Ø c002_vendor_partner_function_sql.bad

• Each time you FTP your file to central server, a special code and a directory with that code name is created to track all actions on it. Eg. 20090815_1609 on right side. It consist of date and a random number.

Informatica – Disorganised Site Data

• Project Scenario - where we are not organised for Site data?q Not easy to remember last used txt file version of a FDq Not easy to takeover the work of the colleague during his holidays, illness etc scenario‘sq Mappings might pick an old file OR overwrite a similar filename of other object.q Not easy for data super users/Data Owners to update new changes/ correctionsq More stress to locate or navigate through different objects

Informatica Mappings – Organised Site Data

Objective is to map Legacy data to Target Mapping in structured approach.

• Performed by Site/(designated central team resources) Informatica resource, this process involves feeding of the raw data in the mappings and producing a file format as desired for Site Check1 mapping.

q This is a tedious process and involves careful analysis of input data. The Informatica resource has to make the mappings to perform this transformation and it might involve lot of data manipulations etc.

q Please do not perfom complex business checks, which are already infact taken care in the C-Team mappings, to avoid duplication of activities

Chk0 Mapping-• Basic activies in the basic site mappings are :

q Legacy Unit Codes into SAP Unit Codesq Format Codes – eg. Sales Material begin with ‘A’ and have no ‘-’q Set the KDS Valuesq Set Default Valuesq Lookup tables to compare values, return new data values.q Test for small relationships eg. bankcode of vendors not in Banks fileq Filter out any unwanted records Or take only desired ones eg. Same source file contains all

customers and we need to filter out External, Internal etc.

Chk1 Mapping- Use the shortcut to C-Team Chk1 mapping to verify your data

Informatica –File Server Overview - Site

• It covers the understanding of the location of in & out files used for the mapping and tracing of them on server; these are primarily the I/O files related to mappings.

• Also we cover the following things in context of specifying the sessions for mappings:

q Speciying of Source files locationq Speciying of Target files locationq Locating error files generatedq Locating duplicates/bad files generated

• In brief, we highlight the tracing of following In & out files types of files:q Source, Target, Error, Duplicates Files –Chk0q Source, Target Error, Duplicates Files –Chk1

• The initial decision for the source, target locations are specified by the Informatica expert OR the person responsible for administration of your informatica Server.

• It is OK to establish subdirectories under the folders SrcFiles, TgtFiles and BadFiles for better management of your files.

Informatica –File Server Overview - Site

• Site Basic mapping (Chk0 ) Contextq Source files

Ø SrcFiles used as the input to mappings; should be a location on the Informatica serverØ Files with <object name>.txt extension are stored here.

q Target files (under)Ø TgtFiles used to generate the output files, lying at your Informatica serverØ Files with <object name>.txt extension are generated here.

q Error filesØ Basically not generated as such, all targets are output files; in case you generate one, it should

be kept under TgtFiles location. The session run errors are however located under SessLogsq Duplicates/BAD records files (under)

Ø BadFiles directory is used to generate these files by default.Ø A file could be generated in case your target is a DB table and you have implemented some

primary keys that resulted in not loading of some records.

• Site mapping (Chk1) Context The CHK1 mapping you use also make use of the above directories but some times in a different way.

q Source FilesØ This directory SrcFiles is not used for picking the source file for these mappings rather we use

the files from TgtFiles directoryØ Files with <object name>.txt extension lying under TgtFiles are used as source files

q Target FilesØ Files with <object name>.out extension are generated under TgtFiles by these mappingsØ Once these .out files are considered good enough, please activate the FTP option and send

this file to appropriate folder on the central conversion site server.q Error Files

Ø Same as above

Agenda

Agenda

What's New Now ?



Nomenclature

AIP (Automated Informatica Platform)- Main Functions

• Scope Controlq Global Repository for all data objects (documentation)q Site specific scope control

• Manage File Deliveriesq Display error files directly in excelq Rerun – Submit- Follow Up on errors

• Planningq Set Delivery / Load dates

• Statisticsq Follow Up on delayed deliveries / loads

• SAP Downloadsq Manage all table downloads from SAPq Instantly refresh tables – Selective access to end users

• Easy To Maintainq Maintain multiple sites at the same time in the AIP q Adding additional sites will have no lead-time < 1 minute to setup.q Whole solution can be maintained by 2-3 Informatica Developers at the central data

conversion location

Scope Control

File Management

Planning

Statistics

SAP Downloads

Agenda

Agenda

What's New Now ?



Nomenclature

Nomeclature

• P3 Targets, Sourcesq Central Team generates these Targets, Sources.q The structure specified in the FD needs to be built in Informatica as a target. The sites will map their

source to this target in the mapping process.• SAP Extracts

q Central Team Extracts current production SAP Data for Sites to use as reference tables. eg. Vendors,Customers, Materials, Banks.

• Site Mappingsq Used to refer Mappings used for the purpose of extraction/formating of data from legacy systems

into the P2 Target format.q It could be 1 or more mappings for each FD object based on complexity

• Site Chk1 Mapping q A shortcut to the Central teams Chk1 mappingq It checks data for Mandatory Fields and generates a error fileq If no errors during running of this mapping, we can FTP the file to the central team, else analyse the

errors and fix them.q The source and the Target structure is same in this mapping

• Central Chk Mappings q These represent the reusable set of object mappings. They require minimal set of changes to

include new countries• KDS

q It refers to Key Data structure values which are configured inside SAP for each country/site• Localization Matrix

q It refers to country specific exceptions• Site Team

q This is the team of technical and business data owners who focuses on extracting site data, site level checks and reconciliations and delivery it further

• Central Teamq This is the technical and Functional designers who ensures implementation of all site level, project

phase level and SAP level checks and reconciliations

methodology for sap data migration using informatica and confidential methodology for sap data...

Documents