gm interface strategy updated for 7th jan review

InformationTechnology

04/21/23 GM Confidential 1

SAP Interface Strategy



Objectives

• Establish Integration patterns and methodologies for SAP

• Propose best practices for Interfaces including data validation, transformation, translation and error handling

• Integration strategy using PI

• Benefits of Global Interface re-design using PI

• Next steps



Agenda

• Establish Interface scenarios • Establish Interface patterns • Understanding key factors for interface design• Establish integration patterns for SAP• Transformation, Translation & Validation• Best practices for Transformation, Translation & Validations• Analysis of CSIDS interface • Analysis of GPS interface• Advantages in re-design using PI• End to end error handling strategy• Appendix


Establish Interface Scenarios


InformationTechnologyInterfacing Systems

• Legacy SAP Systems• Legacy Non-SAP Systems• Data Warehouse Systems • SAP systems




Interface formats

• File / FTP– SFTP– Text, CSV, Other formats

• IDOC • RFC/BAPI• Plain HTTP • JDBC• SOAP• XI (Proxy)

Best practice: Consider use of standard IDOCs, BAPIs.

Consider Proxy implementation for addressing high volumes

InformationTechnologyInterface format decision matrix


SAPor

Non SAPYes

No

Custom data

No

nS

AP

SAPStandard data

SynchronousProcessing

required

Data requiredin predefinedformat (file)

Yes

No

File

SAP

Proxy

BAPIExists

Yes BAPI

No PROXY

Yes

NoIDOC

System(only File based

processing)

Data BaseSystem

(Eg.DataWarehouse)

AllowsUpdates

to Database

DetermineAppropriate

Adapter(Eg. JDBC)

Application

Exchanges data(Real time -

Synchronous orAsynchronous)

DetermineSOAP / HTTP

Adapter

DetermineSystem

Capabilitiesfor Data

exchange

IDOCExists

Yes

No PROXY

File

InformationTechnologyEstablishing interface scenarios

File to IDOC 1. SAP Standard or extended IDOC available in SAP ECC2. Asynchronous mode of processing and large volumes to be processed3. Greater administrative control required4. SAP Standard Master data (Customer or Vendor master for example or

SAP Transactional data example Sales orders or Purchase orders) where IDOC is available.

PROXY to File1. When Non-SAP standard data is to be exchanged2. When large volumes of data is to be handled3. When a pre-defined file is to be delivered to target system

File to PROXY1. When Non-SAP standard data is to be exchanged2. When large volumes of data is to be handled3. When pre-defined formatted file is provided by source system


InformationTechnologyEstablishing interface scenarios

File to File 1. On source or target systems pre-defined programs generate or processes file in a

pre-defined format.– For example SAP to WPCS interface where a pre-define file is generated @

SAP ECC for target system

2. High volumes of data to be processed

3. SAP or Non-SAP standard data to be exchanged

IDOC to File

1. SAP System generating standard IDOC or extended custom IDOC

2. Target system expects the message in the form of file

3. Minimum development on SAP side for building interface

HTTP to Proxy1. Source system or application needs to connect to SAP PI using native HTTP

interface

2. Synchronous communication required to be supported

3. XML body in the request to be processed in SAP PI



Establish Interface patterns


InformationTechnologyInterface patterns

1. Content enrichment– Access external data source to

augment or enrich a message with additional information.

– Data lookup step helps gain information for enrichment. This is possible by:

• Fixed Value Mapping

• Local Mapping tables

• Unified Key mapping services

– Majority of the GM interfaces will be addressed by this pattern

Sender SystemSAPPI

Receiver System

Fixed ValueMapping

Local MappingTables

Unified KeyMapping Service

2. Canonical– Data received in different formats from

multiple systems processed to one single format in SAP ECC.

– Example Invoices from multiple systems in different formats delivered to SAP

– GM’s existing Interfaces for processing Invoices to SAP system

Invoicing System A

CanonicalData

Model

SAP ECCInvoice

Invoicing System B

Invoicing System C

Mapping_A

Mapping_B

Mapping_C

Mapping




Interface patterns

3. Message splitter– Data received in PI. PI generates

multiple messages in different formats for multiple target systems.

– Example a customer master record containing customer master data as well as address master data is transformed into two IDOCS. ADRMAS (Address master) & DEBMAS (Customer master).

Customer MasterData from MDM

System

Splitter FunctionPI

1:N MappingExecuted

Receiver A

Receiver B

ADRMAS

DEBMAS

RequestingSystem

RequestSAP PI

(ccBPM)ReceivingSystem

Reply

4. Request response– Represents asynchronous de-coupled

message processing solution

– Guaranteed delivery

– Synchronous / Asynchronous design possible

– ccBPM used to orchestrate message processing

– Correlation of messages in PI



Interface patterns

Sender System

Persistence &Acknowledgement &

ConfirmationsPI

Receiver System1

2 4

3

5. Guaranteed delivery– Based on Quality of Service (QoS)

Principle

– Asynchronous message processed Exactly Once (EO) or Exactly Once in Order (EOIO) fashion.

– Message persisted in PI

– Receiver will acknowledge the receipt which can be forwarded to sender

InformationTechnologyInterface patterns summary

• Right choice of Interface scenario & Integration pattern will result in:– Greater re-usability of design time objects

– Common mapping services across multiple interfaces

– Lower TCO in long run

– Greater scope for addressing / improving business processes leveraging functionalities offered by PI



Understanding key factors for Interface Design




Key factors in Interface Design

The following key factors play a vital role in deciding the interface design:

– Message size – Message volume– Processing mode– Frequency



Message size and volume

• Message size plays an important role in Interface design decision making process:– Message sizes briefly classified into following categories

• Greater than 20MB– High system resource requirements

• Less than 20MB– More than normal system resources demanded

• Between 1MB ~ 5MB• Best practice : Limit message sizes ranging from 1MB – 5MB

• Message Volume– Even if the message size is less, high volumes may impact performance– Too many small size messages also impact performance– Bundling of messages to reach ideal message size (1MB – 5MB) will

deliver better performance. • Best practice : Look for scope for bundling of small individual messages• Interface design to be based on realistic volumes



Processing modeReal time

Immediate updates in target systemExpecting a response from target systemEg. Shopping cart application saving a sales order to back end SAP system.

Partner system or application inquiring an order status in SAP ECC.Near real-time

Delayed updates in target systemAsynchronous mode of processingEg. Collecting IDOCs in SAP for processing collectively at a given point of time in a day.

BatchTypically based on time, event or actionMostly asynchronous

Synchronous – Expecting immediate processing – Response / acknowledgement for the message expected

Asynchronous– Executed in background– No user interaction– Implements fire and forget or with or without response

Best Practice: – Given a choice chose Asynchronous & Batch mode of processing to be preferred– Near real time is the next best option – Real time & Synchronous mode of processing to be considered for Business critical

Interfaces

InformationTechnologyFrequency

• Frequency is a factor to be considered as a vital factor for design. • In spite of the message sizes being smaller high frequency may impact performance

• Best practices:– Establish business requirements

• Daily, Weekly, Monthly or on Demand– Establish peak-time usage– Consider realistic volumes for taking design decisions– Look for opportunity to bundle messages / collect messages and process them

together reaching ideal message size– Consider processing 1 time large volume of data in multiple batches




Defining Integration Patterns for SAP

Interface Scenarios

Interface Patterns

Processing mode

Frequency, Size & Volumes

Interfaced Data

Integ

ration

Pattern

s for S

AP

InformationTechnologyIntegration Pattern Identification Decision Matrix

IdentifyInterfaceScenario

IdentifyInterfacePattern

ProcessingMode

Start

Synchronous

Asynchronous

Real Time

BatchFrequency

Volumes

Low

Medium

High

Type of DataSAP

Non SAP(Customer

Data)

1. Content Enrichment2. Canonical3. Splitter4. Request & response5. Guaranteed delivery

X

1. File - to - IDOC2. PROXY - to - File3. FILE - to - PROXY4. FILE - to - FILE5. IDOC - to - FILE6. HTTP- to - PROXY

IDOC

CUSTOM IDOC

BAPI

1

1

3

1

2

1

4

1

5

1

6

4

6

4

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

XX

X

X

X

X

X

X

X

X

1

2

X

X

X

X

X

X

X

5

3

X

X

X

X

X

X

X

IFP_INB_P001

IFP_INB_P002

IFP_OUT_P003

IFP_OUT_P004

Proxy

X

IFP_OUT_P005

IFP_INB_P006

IFP_INB_P007

IFP_INB_P008

IFP_OUT_P009

X X


InformationTechnologyInterface Patterns Identified for SAP




Summarizing key factors


Transformation

Translation

and

Validation


InformationTechnologyTranslations, Transformations & Validations

Validation

Translation

Transformation

PI Message Mappings

Graphical Message Mapping

JAVA Mappings

XSLT Mappings

ABAP Mappings

•Combination of above possible

DB Lookups (Non-SAP systems)

RFC Lookups (SAP systems)

Value Mappings (With-in PI)

Unified mapping services (UKMS)

RFC Lookups & DB Lookups enable validations

Message content based validations

1 : 1 Structural transformations Text-to-xml, xml-to-xml, file-to-IDOC

N : 1 Message transformation

1 : N Message transformation04/21/23 GM Confidential 25

InformationTechnologySAP PI Message mapping positioning

Source Messageformat..

(Eg. Txt File)

SAP IDOC

SOURCESystem

TARGETSystem

SAP PIMessage Mapping

IDOCMetadata

Source MessageData structure

IDOC Adapter

SAP ECC 6.0

Typical file to IDOC scenario

A text file received in SAP PI

Data mapped on file to IDOC structure• Transformations, translations & validations carried out• Runtime configurations determine the target system

IDOC generated in SAP ECC




SAP PI Message mapping

SAP PI Message mapping optionsOrder of choice

1. Message mapping (Graphical Mapping tool)2. Java mapping 3. XSLT mapping (Java Engine)4. ABAP workbench mapping programs

– ABAP Mapping– XSLT Mapping ( ABAP)

Best practicesDecision to be based on – Features & functions offered by mapping options– Performance – Standards / portability– Usability (Ease of maintenance)– Message Mapping to be considered as default choice– Avoid building heavy business logic in integration layer – Consolidate business rules for greater visibility & re-usability and reduced

development time– Consider re-usability factor for design and development across multiple

interfaces



Translation, Transformation and validationTranslation

Changes to values within messages– Value Mapping

• Consider value mapping option to maintain static data for translation of data elements in PI layer. • Ideal candidates for example

– GPS Currency code to SAP Currency code– GPS Incoterms to SAP Incoterms– Legacy country key to SAP Country key

– DB Lookups• RFC / JDBC look ups• Consider usage of RFC / JDBC lookups when dynamic data in SAP is to be referred• Ideal candidates for example

– Transaction like Sales orders or Master data like Customer or Vendor Masters– Is a given Contract exists in SAP ECC

TransformationStructure mappings transform entire message structures

• 1: 1, 1 : N & N : 1 message transformations• Ideal candidates for example

– Message level » A text file to IDOC» A text file to a XML format» Two messages received at different time lines mapped into 1 message for target system

– Data level » Value level Legacy date format to a specific SAP date format» Modification of Legacy material number to SAP material number based on a pre-defined logic» Generate line item numbers for the new contract items on GPS contracts in SAP

Best Practices– Lookups to be developed with re-usability option across multiple interfaces– Use standard mapping functions provided in SAP PI– Custom functions to be developed to address a broader scope



Positioning lookup data and transformation logic


Analysis of existing Interfaces




High Level Re-design considerations to GSFt Interfaces

• An attempt is made to demonstrate Interface design considering the following existing interfaces in GSFt– CSIDS – Supplier data to SAP ECC

– GPS Contracts to SAP ECC



Current situation

• Interface programs to process data in SAP • Transformation, Translation & Validations knowledge spread across

Middle layer and SAP ECC• Two step process of data – Source data to file (SeeBeyond), File to

IDOC in SAP ECC• Minimum re-usability of the development objects if semantically

similar data is to be exchanged with new partner / system• Multiple Interface programs may be resulting due to the approach

over a period of time• Higher cost of development and maintenance• Poor visibility of transformation, translation & validations across

multiple interface exchanging semantically similar data



What is value add ?

• One step process – IDOC created in SAP ECC directly by PI• Transformation, translation & validations achieved in PI and

centralized knowledge in PI• Re-usable design objects in PI • Higher re-usability due to de-couple design in PI• Ease of implementation in rollouts or operational changes to

interfaces during interface life cycle• Greater visibility of development knowledge across interfaces• Lower implementation costs of in long run• Faster turn around times for development activities



Existing Inbound Interface from CSIDS

CSIDS SeeBeyondData Extracted

Data saved to ODS

ODS

GFIMSSAP ECC

Flat file Generatedfor GFIMS

Custom Interface Program/GMC/FMIMM_06183_00_VEND_NAST

New filesDelivered bySeeBeyond

Interface ErrorFiles

Validation /transformationsprior to IDOC

generation

Error

Error data writtento Suspense file

for User validation

OK

IDOC/GMC/ZCREMAS04

Posted

Error handling toolZFMCAZERR

for User to review the error dataand correct the data

InformationTechnologyDesign decisions for CSIDS Interface

• Interface scenario: File to IDOC• Systems: Legacy to SAP ECC• Processing mode: Batch & Asynchronous• Volumes: Medium to High• Interface pattern: Content enrichment• Interfaced data: SAP Standard data (IDOC processing possible)• Pattern Identified: IFP_INB_P001

– Inbound file to IDOC interface with content enrichment, batch and asynchronous processing.

• Error handling:– Standard SAP IDOC error handling tools will be used

– Invalid or Error data segregated to a text file for re-processing in PI

– Interface administrators will be emailed when errors arise with error reasons for resolution




CSIDS Interface using PI

CSIDS

SAP PIMultiple Recipients

DeterminedODS,GFIMS(SAPECC),

SAP SRM

(1)

Map data to IDOC (4)

(2)

JDBC Adapter (3)

PI Mapping

(5)IDOC Adapter

RFC Lookup - forData Validations &

Read Interface Control table

GFIMS(SAP ECC)

• Validations in Interface programs carried out in PI• IDOC delivered to SAP ECC instead of flat file• Error data can be delivered to SAP ECC as flat file for re-processing if required• Pre-validations in /GMC/FMIMM_06183_00_VEND_NAST program will be moved to PI



Value add using PI – CSIDS Interface

Existing functionality To-be functionality with PIFlat file delivered by SeeBeyond to SAP ECC IDOC Created in SAP ECC by PI

Interface program in SAP ECC to do validations on data before mapping it to IDOC PI to do the validations in run time

Extended IDOC of CREMAS04 - /GMC/ZCREMAS04 being currently used Extended version /GMC/ZCREMAS04 will be used

Differential treatment of data based on New or Updates to existing records handled in SAP ECC

IDOCS mapped in PI based on the situation if it is a New record or change to existing record.

Records which failed validation in Interface program are processed by ZFMCAZERR (Correction tool)

Validations in PI will be carried out to determine the invalid data and error handling mechanism will take care of the re-processing of data.

Determination of new or existing data based on DUNS number on record

PI will determine new or existing record with a RFC look up to SAP ECC and populate the IDOC accordingly. Content based enrichment.

Country Key & Purchase order currency are determined Country Key & Purchase order currency are determined during PI runtime

Pre-validations equivalent to validations carried out in SAP Standard IDOC posting function module carried out in Custom interface program

Pre-validations will be carried out in PI in runtime. List of such validations include:

LFA1-KTOKK - Vendor account groupLFA1-PSTLZ - Validation checks will no longer be performed. Value will be made optional.LFA1-KRAUS - Duns, mandatoryLFA1-LAND1 - Country keyLFA1-SPRAS - Language keyLFM1- WAERS - Purchase order currencyLFM1-ZTERM - Terms of payment keyLFM1-INCO1 - Incoterms (part 1)LFM1-INCO2 - Incoterms (part 2), mandatory depending of field TINC-ORTOB

InformationTechnologyExisting Inbound Interface from GPS

Create Contracts/POs for GPS Direct Materials

EDS Filters for relevant Contracts/PO

Transform legacy contract to SAP Contract

IDOC based on the values in Translation

table TranslationTables & rules

Create/Change SAP Contract

Create/Change Price Condition

BLAORD03 COND_A02

SAPCONTRACT

Update Contracts

Update Contracts Condition Record

GPS

See

Beyond

SAP


InformationTechnologyDesign decisions for GPS Interface

• Interface scenario: File to IDOC• Systems: Legacy to SAP ECC• Processing mode: Batch & Asynchronous• Volumes: Medium to High• Interface pattern: Splitter• Interfaced data: SAP Standard data (IDOC processing possible)• Pattern Identified: IFP_INB_P010

– Inbound file to IDOC interface with splitter functionality, batch and asynchronous processing.

– Two IDOCS created in target SAP system

• Error handling:– Standard SAP IDOC error handling tools will be used– Invalid or Error data segregated to a text file for re-processing in PI– Interface administrators will be emailed when errors arise with error

reasons for resolution




GPS Contracts to SAP ECC using PI

• Value Mapping or DB Lookups will be based on the type of data under consideration• Two IDOC’s will be generated

– Pricing condition records – COND_A02– Contract – BLAORD03

EDS

(Extract from GPS &Push to PI)

SAPPI

FileAdapterValueMapping

&DBLookUps

SAP ECC

RF

C

Message transformationto Generate 2 IDOCs

for SAP ECC

PricingRecords

COND_A02

BLAORD03

IDOC Adapter

IDOC Adapter

End to End - Monitoring on RWB

GPS[Contracts]



Value add using PI – GPS contracts to SAP ECC

Existing functionality To-be functionality with PI

Conversion of GPS Currency code to SAP Currency code currently in SAP ECC during IDOC processing Value mapping in PI used to host the static data in PI.

Mapping of DUNS number from Legacy system to SAP Vendor Number in SAP ECC during IDOC processing

RFC look up in SAP ECC 6.0 will be invoked in PI runtime to obtain DUNS number.

Mapping of GPS delivery terms to SAP INCO terms involving a custom (Ztable) for mapping in SAP ECC during IDOC processing Value mapping in PI used to host the static data in PI

Mapping of GPS UOM to SAP UOM – in SAP ECC during IDOC processing Value mapping in PI used to host the static data in PI

SAP Line Item numbering based on Legacy Line item numbering - carried out using User exit implementation

Update or Create situation determined in PI and PI mapping will be utilized to create line item numbers for SAP ECC contract

Determination of account assignment category based on custom table data (Ztable) on SAP side during IDOC processing

RFC look up in SAP ECC 6.0 will be invoked in PI runtime to obtain required data.

Payment terms determination currently on SAP side during IDOC processing


Incoterm determination currently achieved on SAP side during IDOC processing Value mapping in PI used to host the static data in PI

Determination of the contract line item while processing pricing condition records


1. IDOC enhancements & IDOC user exits implemented to achieve the functionality - EXIT_SAPLEINN_001, EXIT_SAPLEINN_002, EXIT_SAPLEINN_003, EXIT_SAPLVKOI_001

2. Interface programs contains validations and translation logic

1.IDOC enhancements are not required to achieve this transformation & translation.2.One single RFC look up will be developed in SAP ECC consolidating the required translation values and data for more than one contract retrieved in one call.

InformationTechnologyIntegration landscape with PI

Inbound ..WPCS

(JV data)

InterfaceCSIDS

(Vendor Master)

InterfaceHeiler PPM

(Material Master) Design tim

e

Run

time

Mapping

Mapping

Mapping

WPCS

CSIDS

IDOCs

SAP PI 7.0SAP ECC

6.0

CREMAS

HeilerIn

MATMAS

Runtime Workbench

Heiler Out

MATMAS

JDBC update

Legacy InvoicingSystem A

Legacy InvoicingSystem B

Legacy InvoicingSystem C

Legacy Systems

Invoice Intf.Invoices

MAP1

MAP2

MAP3

MAP

Invoicing system MAP4



Error Handling




PI out of the box monitoring - RWB

• PI Runtime Work Bench (RWB)– One point for monitoring in PI

• Features available– Component Monitoring

– Message Monitoring

– End-to-End Monitoring

– Performance Monitoring

– Alert Configuration

– Cache Monitoring

– Alert mechanism• Functionality offered

– Missing data in mapping table

– Missing data in source system

– Data constraint (mandatory field or a field required for process missing)

– Runtime errors (Missing PI Configuration)

– Other user defined validations

• Flexibility to program additional features



Error handling – Classification of errors and what can be done

• Technical errors– Errors resulting due to technical problems eg. Connectivity and others

• PI Runtime work bench will be used for monitoring technical errors

• Data Errors– Data errors resulting from:

– Validations in mapping

– Transformations

• Errors in runtime captured and alerts will be raised

• Logical errors– Some of the common logical error situations

– Dependant configuration in SAP ECC missing

– Dependant master data is missing

– Dependant data is not yet in required status for message to be processed

– A look up table entry is missing for the data to be processed successfully

• PI runtime can segregate the data for processing at a later point of time when the situation at target is resolved



Re-processing options

• Inbound– If source system owns the data

• Error data will be routed back to source systems for correction and re-processing

• Error logging & communication to owners for audit purposes

– If source system does not own the data and target has to fix the data• Error data will be persisted at target• Error logging & communication to owners • Re-processing of the data will be based on business requirements

• Outbound– SAP ECC owns the data

• Error logging & communication mechanism will be utilized to handle the situation

• Re-transmission of the data triggered after error resolution



Common error logging & audit mechanism

• Common error logging & audit mechanism will be built with the following features and will be used across all the interfaces– Inbound data to SAP ECC

• Summarization of data counts• Capturing errors with reasons • Communicating the owners for resolution / follow up• Capturing interface run details (eg. Last run date, time )

– Outbound data • Summarization of data counts• Logging interface run details

– The data collected for all interfaces will be accessible to the interface administrators & key users



Next Steps

• Evaluation of the Interfaces• Gather information around key factors

– Influencing interface design & pattern

– Transformation, translation & business rules

– Identify Integration patterns

• Decisions– Design

– DB Look Ups / Value Mappings

• Revisit PI sizing requirements based on any Integration scenarios


Questions ?




Appendix

• Design time Best practices



Design time best practices

• Transaction & Master data– Ex. Sales Orders, Purchase Orders, Release Orders, Material Masters

• Best practice to use BAPI / IDOC for processing data to SAP systems

– Communication• Asynchronous processing for performance

• If no Standard BAPI or IDOC is available– Processing of data using Custom RFC or Proxy

– Real life examples: • Cross referencing data • Legacy system requires custom table data




• Enterprise repository / Design time Objects– Pre-Delivered SAP Data types

• SAP Core data types (typical XSD type)

• SAP Global data types (Contain business semantics)

• Best practices– Aggregated Global data types

– Custom data types developed with• Core data types & SAP Global Data types

– Consider Canonical data models when building custom data types

• Evaluating need to enhance SAP Data types:– SAP’s Aggregated data types can be enhanced.

– Ensure re-usability of GM’s custom data types across landscape by using dependency feature @ Software component level in SLD.

– Improve Re-usability across different interfaces




• External Definitions– SAP PI provides option to import the following standard schema to

Integration repository which reduces development time in PI to a great extent:

• WSDL Definitions

• XSD (XML Schema Definition Language)

• DTDs (Document type Definitions)

– Extracted message schema will be used as:

• Output or Input messages in Message Interfaces

• Source & Target structures in Message Mappings

– Message mappings

• Existing XSLT or JAVA message mappings can be imported into Integration Repository as Archives and used in run time.

gm interface strategy updated for 7th jan review

Documents