line item level data extraction for financial accounting and controlling

Data Extraction (Line Item Level) FI/CO(Financial Accounting and Controlling)Revised 06/05/03

Paula Gist (CPA), BW RIGLothar Schubert, BW RIG

October 3rd, 2002

SAP Labs America 2002, FI / CO Line Item Extraction 2

Content

Business Drivers

FI Extraction

FI line-item extraction

Timestamp mechanism

Customer extensions

Global Settings

Availability and further information

CO-OM Extraction

CO-OM line-item extraction

Delta mechanism (Revised 06/05/03)

Customer extensions (Revised 06/05/03)

Availability and further information


Business drivers

Flexibility:Need for project specific choice of fields.Check on status fields, like open / cleared items.

Drill-down and drill-through (analysis):Drill down capability from summarized data to line-item details.Full through capability, e.g. from Accounts Payable to G/Laccounts information.

Actuality:Access to recent (or even real-time) data.

Scalability:High number / wide variety of users.Large volumes of detailed R/3 transaction data are transferred toBW Daily full loads are inefficient.

In other words: Detailed data and delta capability


FI line-item extraction: Data Sources and Data Targets

BSADCleared Items

BSIDOpen Items

Accts. receivable

0FIAR_C03FIAR: Line item

BSAKCleared Items

Accts. payable

BKPFHeader

BSEGSegment

G/L accounts

ODS -> document detailsoperational queries on line-item data

BW

R/3 DataSources

InfoSources

InfoCubes -> summarized dataanalytical queries on cube data

Drilldown

R/3 Tables

0FIAP_C03FIAP: Line item

0FI_AR_O03FIAR: Line item

0FI_AR_4FIAR: Line item

0FI_AP_O03FIAP: Line item

0FI_AP_4FIAP: Line item

0FI_GL_O02FIGL: Line item

0FI_GL_4FIGL: Line item

Coupled extractionCoupled extraction

BSIKOpen Items


FI line-item extraction: Features

0FI_GL_4 (G/L Accounts: line items)No redundant fields are transferred into BW:

Only fields from the FI document tables (BKPF/BSEG) that arerelevant to general ledger accounting (compare table BSIS),No customer or vendor related fields.

0FI_AP_4 (AP: line items) and 0FI_AR_4 (AR: line items)Vendor / Customer related information (e.g. payment/dunning data).“Coupled” extraction consistent “snapshot” of FI data in BW:

G/L account extraction determines selection criteria (comp.code,fiscal period) and upper time limit of all extracted FI line-items.AP and AR extraction: no further selection criteria necessary /possible.“Uncoupled” extraction possible with PlugIn PI 2002.2, see OSSnote 551044.

Actuality of extracted dataThe most recent FI documents, which are extracted to BW, areposted in R/3 the day before.See OSS note 485958 if you need to circumvent this restriction.


FI line-item extraction: Delta mechanism

0FI_GL_4, 0FI_AP_4, 0FI_AR_4 use an After Image DeltaDelta type “Extractor“:

Delta records are directly selected from the R/3 tables using atimestamp mechanism.Delta records are directly transferred to BW. No record is written tothe BW delta queue.

After Image Delta:FI line items are transferred from the source system in their finalstate (= “After Image“).This delta method is not suitable for direct InfoCube update.ODS object is obligatory to determine the delta for InfoCube update.

Where BW Delta Queue is used:Delta queue and BW scheduler ensure correct serialization of therecords (e.g. inserts must not pass changes)Distribution of delta records to multiple BW systems.Selection criteria of Delta-Init upload are used to “couple” thedatasources logically.


Timestamp Mechanism: New documents

New FI documentsPosted in R/3 since the last line-item extraction.Selection based on the field BKPF-CPUDT.

MANDTBUKRSBELNRGJAHR::

CPUDTCPUTMAEDAT::

Table BKPF(FI document header)

new documents

changed documents

BWOM2_TIMESTLower limit:upper limit of thelast G/L line-itemextraction

Upper limit:previous day

FI document keyTable BWOM2_TIMEST(Timestamp table)


Timestamp Mechanism: Entries in BWOM2_TIMEST

Timestamps are stored as Seconds since 01.01.1990”.

Other DataSources (e.g. CO line-item extraction)also use this table.

Documentation of upload history: Recording of FI datathat has already been extracted into BW.

Well-defined sync-points, if the delta upload has failed.

Table BWOM2_TIMEST (Timestamp table)

OLTPSOURCE AEDAT/AETIM UPD DATE_LOW DATE_HIGH LAST_TS0FI_GL_4 16.05.2000/20:15 Init 01.01.1990 15.05.20000FI_GL_4 24.05.2000/16:59 Delta 16.05.2000 23.05.20000FI_GL_4 02.06.2000/21:45 Delta 24.06.2000 01.06.20000FI_GL_4 15.06.2000/12:34 Delta 02.06.2000 14.06.20000FI_GL_4 21.06.2000/18:12 Delta 15.06.2000 20.06.2000 X0FI_AP_4 18.05.2000/21:23 Init 01.01.1990 15.05.20000FI_AP_4 30.05.2000/12:48 Delta 16.05.2000 23.05.20000FI_AP_4 10.06.2000/13:19 Delta 24.06.2000 01.06.2000 X0FI_AR_4 17.05.2000/18:45 Init 01.01.1990 15.05.20000FI_AR_4 04.06.2000/13:32 Delta 16.05.2000 01.06.20000FI_AR_4 16.06.2000/15:41 Delta 02.06.2000 14.06.2000 X


Timestamp Mechanism: Changed documents

Changed FI documentsChanged in R/3 since the last line-item extractionCannot be selected based on field BKPF-AEDAT:

BKPF-AEDAT is not updated by all BW relevant changes of FIExample: Clearing of an open invoice (by incoming payment) changesthe document state, but does not update BKPF-AEDAT.

SolutionRecording of all changed FI documents (FI document key, date

of last change) in table BWFI_AEDAT.

FI update programs in R/3 had to be changed.Selection of the changed FI documents based on tables

BWFI_AEDAT and (timestamp table) BWOM2_TIMEST.

MANDT BUKRS BELNR GJAHR AEDAT AETIM600 1000 1900000010 2001 21.02.2001 10:23:17600 1000 1000000016 2001 21.02.2001 10:28:34


Timestamp mechanism: Recording of changed docs

function FI_DOCUMENT_CHANGE...update BSEG.

call function OPEN_FI_PERFORM_00005011_P...endfunction.

Business Transaction Events (Tab. TPS31)Process Function00005011 BWFIP_WRITE_AEDAT_POINTER

function BWFIP_WRITE_AEDAT_POINTER...move-corresponding BKPF to I_T_BKPF.

call function BWFIP_INSERT_BWFI_AEDATtables I_T_BPKF.

endfunction.

This entry in table TPS31starts the recording ofchanged FI documents.• It should be inserted before

the delta-init upload.• See OSS note 401646

R/3 transactions that recordchanged FI documents intable BWFI_AEDAT:• Change document• Incoming/outgoing payments• Dunning• ...

Simplified flow chart


Customer Extensions

Extension of extract structures using customer includesData source R/3 table Extract struct. Customer include0FI_GL_4 BSEG DTFIGL_4 CI_BSIS0FI_AP_4 BSIK/BSAK DTFIAP_3 CI_BSIK0FI_AR_4 BSID/BSAD DTFIAR_3 CI_BSID

Case 1: All fields of the customer include are part of the R/3 table.

No additional ABAP-coding is required. The fields of the customerinclude is automatically filled by the extractor.

Case 2: Some fields of the customer include are not part of the R/3 table.

Use Business Transaction Event 00005021.Build a function module that fills the fields of the customer include.Use the following template: SAMPLE_PROCESS_00005021.No modification of SAP standard extractors required.For more information: See OSS note 410799

Clean separation of SAP standard and customer extraction


Global Settings (BWOM_SETTINGS)

BWFILOWLIMHere, you can set the lower time limit of the time stamp selection for data requests in update mode initializationof the delta method. Together with the selection criteria, this parameter is a decisive factor for the data volumewith data requests in update mode initialization of the delta method.

BWFISAFETYHere, you can the upper time limit of the time stamp selection for data requests in all update modes. Value 1 inthis parameter means that the upper time limit of the time stamp selection is the previous day. With largerparameter values, the upper limit of the time stamp selection is correspondingly further (in days) in the past.Note that you cannot choose a parameter value cannot be smaller than 1, as this would prevent the deltadataset being found completely.

ORGSYSONLYIf you SAP R/3 system is part of an ALE scenario, set this parameter to X. This ensures that only those lineitems that were originally posted to the BW system will be transferred there from the SAP R/3 system. Thisprevents line items being transferred several times from various SAP R/3 system.

DELTIMESTWith this parameter, you can set the retention period of entries that are no longer required in time stamp tableBWOM2_TIMEST. The default settings for deletion of entries that are no longer required in the time stamptable is 60 days. The minimum retention period for entries in the time stamp table is 15 days. When entries aredeleted from table BWOM2_TIMEST, the corresponding entries for the changed FI line items are deletedsimultaneously from log table BWFI_AEDAT.


Further information and availability

System requirementsR/3 source system:

Release 4.0B++PlugIn 2001.1++

BW:Release 3.0A++(Release 2.0b / 2.1c: See OSS note 401646)

SAPNet (OSS) notesImplementation of new DataSources: 401646Extension of extract structures: 410799 / 430303Extraction of recent updates: 485958Migration from 0FI_AP_3 / 0FI_AR_3: 410797 / 200576“Coupled” extraction: 428571“Uncoupled” extraction: 551044

Online documentationCheck documentation on business content, BW 3.0b.


CO-OM line-item extraction

COBK

COEP

CO documents

Cost Center0CCA_C11

0CO_OM_CCA_9

BW

R/3

DataSourcesInfoSources

InfoCubes ->summarized dataanalytical queries

R/3 Tables

Internal Order0OPA_C11

0CO_OM_OPA_6

Project0WBS_C11

0CO_OM_WBS_6

Bus. Process0ABC_C06

0CO_OM_ABC_7

ODS (optional) ->document detailsoperational querieson line-item data


CO-OM Delta mechanism

0CO_OM_CCA_9, 0CO_OM_OPA_6, etc. use an Additive DeltaDelta type “Extractor“:

Delta records are directly selected from the R/3 tables using atimestamp mechanism (Timestamp table BWOM2_TIMEST).Field *COEP-TIMESTMP is used as a reliable timestamp for selectingnew CO-OM line-items.

* Reference Field name revised 06/05/03

Delta records are directly transferred to BW. No record is written tothe BW delta queue.

Additive Delta :Only new CO-OM line-items are transferred from the source system(=“Additive Delta“). No changes on CO documents are relevant for BW.This delta method is suitable for direct InfoCube update.ODS object is optional to store document details (for operationalqueries on line-item data).

Where BW Delta Queue is used:Delta queue and BW scheduler ensure correct serialization of therecords (e.g. inserts must not pass changes)Distribution of delta records to multiple BW systems.


CO-OM Customer extensions

Extension of extract structures

* If there is a requirement to extract new fields into BW, use transactionRSA6. The extract structure must be increased with the new fieldsrequired to be extracted. The new data will be written automatically withthe extractor ,but please note, this will not automatically be sent to BW ifthe fields are not in the extract structure.

* The includes CI_COBL e.g. for 0CO_OM_CCA_9 do not have anything to dowith BW but it is normal CO include where the customer can fill in somenew fields he needs in CO.

Data source R/3 tables Extract struct. Customer include0CO_OM_CCA_9 COBK/COEP ICCTRCSTA1 CI_COBL0CO_OM_OPA_6 COBK/COEP ICORDCSTA1 CI_COBL0CO_OM_WBS_6 COBK/COEP ICWBSCSTA1 CI_COBL0CO_OM_ABC_7 COBK/COEP ICBPRCSTA1 CI_COBL

* Please review OSS Note 394491 – Handling Errors from DataSource AppendStructures (released 05/14/03) for further details

*Revised for content correction 06/05/03


CO-OM Further information and availability

System requirementsR/3 source system:

Release 4.0B or higherBW:

Release 2.0B or higher

SAPNet (OSS) notesTechnical background info: 416265Performance optimization: 382329, 387964

Online documentationCheck documentation on business content, BW 3.0b.

line item level data extraction for financial accounting and controlling

Documents