1st Document

Most DataSources within an SAP system are delta capable. However, the delta capability is not available when you connect to external databases using DB connect. By following this process, you will be able to connect your BW system to an external database and make the DataSource delta capable.

Key Concept

Delta-capable extractors deliver fewer records, which means fewer network resources are used, BW transfer and update rules require less work, and fewer records need to be rolled up in aggregates. Delta capability has a great impact on performance when you are loading data into BW.

The open architecture of SAP BW from Release 3.0 on allows you to integrate data from an external database using DB connect. It offers an option for extracting data into BW from tables and views in other external database management systems that are supported by SAP and are connected to BW.

The integration between BW and R/3 is seamless. DataSources and extractors are delivered along with a plug-in that is installed on the source system that makes the extraction of data into BW an easy task. Most of the DataSources are delta capable, which means they extract only the data that is newly created or changed after the last load into BW.

However, all the fun dries up when you use DB connect to extract data from a source system that is not part of SAP. Although BW allows you to connect to external databases such as Oracle, Microsoft SQL Server, and IBM DB2, the delta capability delivered as a part of the Service Application Programming Interface (SAPI) component of the plug-in is not available. Because of the lack of delta capability, system administrators monitoring extraction and system performance have not always exploited DB connect.

In our two-part series, we explain why you do not have to reinvent the wheel to make the external DataSource delta capable. (See the sidebar, “Prerequisites for Configuring DB Connect.”) We’ll use the example of an external Oracle database to take you step by step through the process. This month we’ll show you the first two steps:

Step 1. Connect to the external database using the DB connect Functionality of SAP BW

Step 2. Create and configure the DataSource on the external database system

In the next article, we’ll show you the ABAP code that makes the DataSource delta capable.

Step 1. Create a source system for the external database using DB connect. Go to Administrator Workbench in the modeling area to use the source system creation (menu path Modeling>Administrator Workbench:Modeling>Source Systems).

Use the context menu from the source system and click on the create icon (Figure 1). First click on the Source Systems button and then right-click to get the context menu. In this case the context menu gives only one option

— Create.

Figure 1 Create a source system in Administrator Workbench

Select the button for Database System (Data and Metadata Transfer Using SAP DB Connect). Click on the transfer icon to confirm the selection. Assign a Logical System Name in the pop-up screen (Figure 2). It should

be a meaningful name. We selected DEV_DBCON since we are connecting a development system using DB connect.

Figure 2 Assign a logical system name

Choose the external database system you are connecting to. In this case we are connecting to an external

database on Oracle (Figure 3).

Figure 3 List of available database systems to connect to using DB connect

The system leads you to a screen where you need to provide the logon data for the external database system

(Figure 4). The User name is the user created on the external database application.

Figure 4 Add logon data

The Conn. info is required for the native SQL to open the database connection of the external application. The Permanent check box is useful for the situations in which the database connection is lost — for example, when the network link is broken or the database itself experiences some sort of failure. You should set the flag if the database connection is essential, or when it is accessed very often. In our scenario, this box is blank. Since we load the data to BW every night, it is a periodic job that requires just a temporary connection to the external database.

The database administrator of the external application provides the user name, password, and connection information. You put the connection information into a file called tnsnames.ora on the Oracle database server on

the BW side. The connection information details are shown in Figure 5. You can use an editor to edit the file. The main parameter values in this connection information are the IP address of your external application and port number. The database administrator of the BW system makes those entries in the file. The administrator of the external database provides all the required parameters (e.g., the IP address of the external database, port, and surrogate ID [SID] for the database).

Figure 5 Entry in file tnsnames.ora in the database server for BW

Smartdw is the Conn. info name provided in Figure 4. The .WORLD after smartdw in Figure 5 is the domain. This is defined by means of the entry NAMES.DEFAULT_DOMAIN in SQLNET.ORA. Normally WORLD is used as the default domain name. SDU stands for session data unit, which determines the size of the sent data

packages. The maximum value set for this parameter is 32768.

Note

SAP BW 3.0B with SP 24, DB connect, permits connection to Oracle, Informix, Microsoft SQL Server, DB2/MVS, DB2/RS6000, ADABAS/D, and DB2/OS400. These databases require different parameters for connecting to BW. SAP has released several SAP notes for connecting to specific databases. Refer to the section “Reference to Related Notes” of SAP note 323151 to access the correct SAP note. SAP BW 3.5 goes beyond the extraction capabilities of DB connect. It offers a new feature called UD Connect that allows you to access data from SAP as well as non-SAP platforms, including almost all relational and multidimensional DataSources. However, UD Connect does not provide delta capability automatically.

After entering the logon data, click on the save icon. You now have the new source system in the external

database system list, as shown in Figure 6.

Figure 6 Include external database in the external database system list

You also see the newly created source system in the Source Systems screen (Figure 7).

Figure 7 External database (Oracle) included in Source Systems screen

Step 2. Create and configure the DataSource on the external database system. A prerequisite is that the database administrator has permitted access to the specific table and that the authorization permits editing of the table views. On the selected source system, use the context menu and choose the option Select Database

Tables (Figure 8).

Figure 8 Selection of database tables from the newly created Source System using DB connect

After you select the option Select Database Tables, the system directs you to select the tables/views from the

object catalog on the external database, as shown in Figure 9.

Figure 9 Selection screen to get the object catalog from the connected source system

Click on execute icon or press F8 to display the list of tables and views based on the selection made, either by

entering a specific name or by a generic search, as shown in Figure 10.

Figure 10 List of tables and views from connected system

Select the name of the Table/View from the list of Database Selection — Tables/Views and click on the Edit

DataSource button. As shown in Figure 11, the system allows you to set the different attributes of the DataSource to be created.

Figure 11 Screen to set the attributes of the DataSource

You may find some settings useful in making your DataSource more meaningful — for example, by declaring a field a selection field, you can use it to filter the extraction.

You can change the Application Comp. to your required application component, which makes a search for a DataSource easier. Otherwise the generated DataSource will be available under the NODESNOTCONNECTED application component in BW.

You can change the DataSource type; otherwise, by default it selects the Transaction data type. The DataSources differ based on the type of information/data they are extracting and bringing to BW. The DataSource is likely to provide three types of information/data:

Transaction data

Master data attributes Master data text (language-dependent description)

The Database Fields area has a number of columns. The first three are useful for creating a DataSource.The first column with the information icon indicates whether the field is available to be included as a part of the DataSource. If there is no value, the field is available. If it contains the symbol , then it is not available as a part of the DataSource. DB connect has limitations in including fields of Table/View as a part of the DataSource. Some limitations are common to any source database you use. For example, technical field names cannot be longer than 16 characters and must contain uppercase letters, numbers, and underscores only. You cannot use fields with reserved names used by the system for a specific purpose, such as COUNT. Other limitations specific to the source database are explained in the SAP notes listed below. SAP therefore recommends that you create views that can overcome those limitations. The DataSources for BW should use views instead of tables. Once a table is created in the system and is filled with data, it is difficult to change any attribute (e.g., field name or type of field) of this table. In contrast, a view is a definition and does not contain any data. Views are based on one or more tables. SAP recommends that you create a view on a table to overcome the limitations of creating a DataSource with DB connect. We will explain how to create a view in part 2 of our series. The second column indicates whether the field is available for the extraction and transfer of data into BW. By default all the fields are available for transfer. You can remove the check mark to indicate which fields should not be extracted and moved to BW.The third column indicates whether you want to use selection criteria for that particular field in the InfoPackage. Selection criteria filter the data — for example, filtering for a currency if that is declared as a selection field, as shown in Figure 11. You can mark more than one field for selection to obtain more control over the extraction. Click on the Generate DataSource button when you have made the changes to those three columns. Then click on Yes. The technical name of the table/view created is INCOMING_VIEW. The system adds the prefix 6DB_, so it generates the DataSource with the name 6DB_INCOMING_VIEW.This newly created data source is now available for data extraction from the external database using DB connect. This DataSource will bring data in the full update mode. You will be able to make it delta capable after following the

third step, which will be in a subsequent issue of BW Expert.

Note

The DB connect documentation is subject to change. For this reason, you must also refer to the SAP notes listed in the documentation on the BW pages in the SAP Service Marketplace (http://service.sap.com/bw), under Documentation>Documentation Enhancements. SAP note 512739 provides additional information on DB Connect. SAP note 323151 provides information on several connections with native SQL, and refers to other SAP notes relevant to the topic.

The following database-specific SAP notes have more information on DB connect:

MSS database, SAP note 512739

Oracle database, SAP note 518241 Informix database, SAP note 520496 SAP DB database, SAP note 520647 IBM DB2/390 database, SAP note 523552 BM DB2/400 database, SAP note 523381

IBM DB2 UDB database, SAP note 523622

Prerequisites for Configuring DB Connect

To follow this process, you need experience with the source database tools and database-specific SQL syntax functions. You need to be able to transfer semantically usable data into the BW. The SAP-specific part of the database interface — the Database Shared Library (DBSL) — must be installed for the corresponding source database management system on each of the BW application servers.

In our example, we are using the Oracle database and SAP BW 3.0B with Support Pack (SP) 17 on the BW side. Our external application was built using an Oracle database.

Since the source database and target database are the same, we did not have to install DBSL. Different databases

have different steps for making the connection. See the list of SAP notes that explain details and limitations for different databases earlier in this article.

You also need to ensure you do the following while using DB connect:

Create a special user and give the user the DB_OWNER role on the required database.

Grant this user rights on the required tables. Create a view under this user to the table. Use this user to create the database connection in the BW system.

Use the view to extract the data.

After you have connected your BW system to an external database using DB connect, you can make the DataSource delta capable with ABAP code using the time-stamp technique.

Key Concept

Most databases have a method for identifying new records and changes to the database whereby each new and changed record carries details of the time in year/month/day/hours/ minute/seconds. These details are also known as a time stamp. Oracle databases have a time stamp for all their tables unless they are specifically created otherwise. Using the time-stamp technique, delta records can be identified using the time-stamp details.

SAP BW from Release 3.0 on allows you to extract data from an external database using DB connect. However, delta capability is not automatically available. In part 1 of this series, we explained the first two steps for making an external DataSource delta capable: how to connect to the external database and how to create and configure the DataSource on the external system. In step 3, which we describe in this article, you make the DataSource delta capable with custom ABAP code.

We use the time-stamp technique, in which the delta records are identified using the date and time of creation or change captured by an application on the external system. Technically not a true delta, this method gives the user the delta records. In a true delta, the system tracks records that are already extracted. An overview of the coding and logic we used follows:

Step 1. Find the time of the last successful extraction. You limit the extraction to the records created/changed after the last successful extraction. Using coding, you identify the last successful extraction date/time. The delta mechanism works by comparing the time stamp of the InfoPackage run with all time stamps of the records. It then chooses those records whose time stamps are greater than or equal to the InfoPackage time stamp.

Step 2. Calculate the safety delta time to arrive at the extraction parameters of the time stamp in the InfoPackage.

Step 3. Convert the time details, which are in Universal Time Converted (UTC) format, to the time format of the user.

Before getting to details of these three steps, let’s look at the “view definition” of the database table. You need to

make some changes while creating the view. Figure 1 shows the code for creating a view based on an Oracle table.

CREATE VIEW SMARTBI.INCOMING_VIEW(TRANSACTION_ID,RAW_TXN_ID,TXN_TYPE,DATE_OF_TXN,TIME_OF_TXN,

CURRENCY_ID, TXN_AMT, SETTLEMENT_ID, MOD_DATE, MOD_TIME, MERCHANT_ID, TERMINAL_ID, BATCH_ID,ROC,

CARD_ID)AsSELECT

TRANSACTION_ID,RAW_TRANSACTION_ID as RAW_TXN_ID,TRANSACTION_TYPE as TXN_TYPE ,to_char(DATE_OF_TRANSACTION,'YYYYMMDD') as DATE_OF_TXN,to_char(DATE_OF_TRANSACTION,'HH24MIss') as TIME_OF_TXN,CURRENCY_ID , TRANSACTION_AMOUNT as txn_amt, SETTLEMENT_ID,to_char(MODIFICATION_DATE,'YYYYMMDD') as MOD_DATE,to_char(MODIFICATION _DATE,'HH24MIss') as MOD_TIME,MERCHANT_ID , TERMINAL_ID , BATCH_ID ,ROC , CARD_ID

FROM smartdba.incoming_transactions

Figure 1 Create a view based on a table in Oracle

With the code in Figure 1, we created a view for the table that stores the source data that we want to extract. We did that because the table’s name INCOMING_TRANSACTIONS is longer than 16 characters. The view INCOMING_VIEW overcomes the limitation. Note that these tables and views are not standard SAP tables. Because you are connecting BW to an external application, the tables/views on the external application are developed by the designer of external application. If the table name/field name does not fall within one of the limitations, then you do not need to generate a view.

The application data in our example is stored in a schema called smartdba. incoming_transactions is the table in the Oracle database from which we will extract the data.

Note in Figure 1 that the fields with names that exceed 16 characters — for example, RAW_TRANSACTION_ID — are referred to as RAW_TXN_ID. The date format is changed to YYYYMMDD and the time is in HHMMSS with 24-hour time settings.

Step 1. Identify the Last Successful Extraction

Whenever you execute an InfoPackage to extract data, the details are stored in system tables in SAP BW. This includes details such as the name of the InfoPackage, the request number generated by the system, the date and time when the InfoPackage was executed, the Quality Management (QM) status of this request, and the number of records extracted. One of the useful database tables for our requirement is RSREQDONE, which gives the information required to make the DataSource delta capable. All of the above field values are stored whenever any InfoPackage is executed to extract the data from the source system regardless of the source system.

To implement the process, create an InfoPackage to extract all existing records. This brings all the records from the source system to BW. This InfoPackage is executed without any selection criteria. The details are stored in table RSREQDONE in BW. Make sure to enter the InfoPackage entry into table ZSC_DELTA_IP with the flag as I for initial upload.

Let’s say the InfoPackage for the initial load is ZPAK_3U94F3YC8LY7XAG26PWU2OCQ6. The system generates the technical name of the InfoPackage. We executed this InfoPackage on 06.11.2004 at 10:11:32. All the details of this InfoPackage are stored in the table RSREQDONE along with the QMSTATUS and number of records extracted.

When you extract the delta data from this DataSource, you want only those records that are newly created or changed after the last successful extraction. To get the last successful extraction date and time, query on table RSREQDONE using the technical name of the InfoPackage. You may find many records for the same InfoPackage depending on the number of times you have executed it. Therefore, you need to put those entries in the system

memory and find the last successful run. At this point we expect only one entry since we executed the initial upload InfoPackage only once.

To put the entries into the memory, you create internal tables by writing a small piece of ABAP code. BW has a

facility at the InfoPackage level to write an ABAP routine and create a dynamic selection criteria (Figure 2). This ABAP code fetches the date and time of the last successful extraction, which are passed as selection criterion for the current extraction.

Figure 2 InfoPackage selection screen

We recommend a separate InfoPackage to differentiate between the initial upload and subsequent delta uploads. Table RSREQDONE stores the data for the InfoPackage that was executed. That table retrieves the date and time for the last successful extraction. You create another InfoPackage for subsequent deltas. To differentiate the first delta from the following ones, you need to write some custom code. Create a Z table. We called it ZSC_DELTA_IP. It contains four fields (Figure 3):

ZZ_FIRST_DELTAIP stores the technical name of the InfoPackage used to run the initial upload

ZZ_SUBSQ_DELTAIP stores the technical name of the InfoPackage used to run delta uploads ZZ_FIRST_DELTA indicates by a “yes” or “no” whether the current run is the first delta or a subsequent

upload or initial upload

ZZ_INFOSOURCE indicates the name of the InfoSource to which this DataSource is connected

Figure 3 Structure of table ZSC_DELTA_IP

Since the initial upload does not have an ABAP routine attached to it, it goes to the source system and extracts all the records in the system for that particular DataSource. In the InfoPackage used for the delta upload, first evaluate through the custom ABAP code whether this is the first or a subsequent upload. The one-time activity to support this identification of the initial request is that you enter the technical name of the InfoPackages (Initial Upload and Delta Upload) into this table with a “Y” flag and the name of the InfoSource. The name of the InfoSource does not have any other technical significance in creating this delta technique, but it helps the end user to recognize the pair of InfoPackages since the technical names are generated by the system.

To give an example, the name of the initial InfoPackage is ZPAK_ 3U94F3YC8LY7XAG26PWU2OCQ6 and the name of the subsequent delta InfoPackage is ZPAK_U67PT8H4PCBV9KMZWV94GNZG. You need to create an entry in table ZSC_DELTA_IP with these details:*ZZ_FIRST_DELTAIP =                  ZPAK_3U94F3YC8LY7XAG26PWU2OCQ6ZZ_SUBSQ_DELTAIP =                  ZPAK_3U67PT8H4PCBV9KMZWV94GNZG

ZZ_FIRST_DELTA = I for initial upload

  = Y for first delta upload

  = N for subsequent delta upload

Once you create the above entry, the ABAP code below helps to determine the InfoPackage name, which then helps you get the time-stamp entry from table RSREQDONE. The code below is used in the InfoPackage ZPAK_3U67PT8H4PCBV9KMZWV94GNZG, which extracts the delta upload. CALL FUNCTION 'Z_BW_SC_GET_INFOPACKAGE'

EXPORTINGZZ_IP_INFOPACKAGE = 'ZPAK_3U67PT8H4PCBV9KMZWV94GNZG'

IMPORTINGZZ_OP_INFOPACKAGE = wa_infopackageZZ_FIRST_DELTA = wa_first_delta.

One limitation is that you are unable to use a variable to get the name of the InfoPackage in the ABAP code. Therefore, this is the only hardcoded area of our technique. In the code in Figure 4, we used a custom-built function module. FM Z_BW_SC_GET_ INFOPACKAGE determines which InfoPackage is used to get the time stamp. The extracted time stamp is used to identify the delta records from the source system. FUNCTION Z_BW_SC_GET_INFOPACKAGE.tables : ZSC_DELTA_IP.select single * from ZSC_DELTA_IP where ZZ_SUBSQ_DELTAIP = ZZ_IP_INFOPACKAGE.if sy-subrc = 0. if ZSC_DELTA_IP-ZZ_FIRST_DELTA = 'Y'. " First Delta Load ZZ_OP_INFOPACKAGE = ZSC_DELTA_IP-ZZ_FIRST_DELTAIP. ZZ_FIRST_DELTA = ZSC_DELTA_IP-ZZ_FIRST_DELTA. elseif ZSC_DELTA_IP-ZZ_FIRST_DELTA = 'N'. " Subsequent Delta Load ZZ_OP_INFOPACKAGE = ZSC_DELTA_IP-ZZ_SUBSQ_DELTAIP. ZZ_FIRST_DELTA = ZSC_DELTA_IP-ZZ_FIRST_DELTA. elseif ZSC_DELTA_IP-ZZ_FIRST_DELTA = 'I'. " Initial Load ZZ_FIRST_DELTA = ZSC_DELTA_IP-ZZ_FIRST_DELTA. endif.endif.ENDFUNCTION.

Figure 4 ABAP code of the function module Z_BW_SC_GET_INFOPACKAGE

The input passed to this function module is the technical name of the InfoPackage used to extract the delta upload. Using the entries in table ZSC_DELTA_IP and the flag ZZ_FIRST_DELTA, it determines which InfoPackage is used to get the time-stamp information from the table RSREQDONE. Once you determine the name of the InfoPackage, get the time stamp of the last successful request. To retrieve the time stamp, query on the table RSREQDONE using the technical name of the InfoPackage. Use two more conditions. Records retrieved should be more than 0 (RECORDS > 0) and the technical status of the request should be GREEN (TSTATUS = ‘@08@’ or TSTATUS = ‘@08\’). Since you may have more than one record retrieved from the above query, put them in an internal table using the ABAP code below.select * from rsreqdone

into corresponding fields of table it_rsreqdone where LOGDPID = ZZ_INFOPACKAGE and RECORDS > 0

and ( TSTATUS = '@08@' or TSTATUS = '@08\').You select the latest request from the above records. To do that, sort the internal table IT_RSREQDONE by the field TIMESTAMPBEGIN in descending order. The latest record will be the first record in the internal table. Pick up the values stored in the first record, particularly DATUM , UZEIT, and TIMESTAMPBEGIN.The fields DATUM and UZEIT store the date and time respectively when the extraction started. The field TIMESTAMPBEGIN stores the same information but in a different format. We stored the values in the local variables WA_DATE_RSREQDONE, WA_TIME_RSREQDONE, and WA_RSEQDONE_TS, respectively.Step 2. Calculate the Safety Delta While the extraction is being carried out, documents could be created in the system. These are likely to be missed in the extraction unless you use a safety delta. We use it a little differently than SAP does. Our objective is not to ensure we don’t miss out on any records, but to extract specific delta records.Take the example of a record with a time stamp of 10:00:01 hours that was committed to the database at 10:00:05, a small delay of 00:00:04 hours. Without a safety delta, this record could be lost in extraction. The safety delta time and the decision on the amount of time may differ from application to application. The larger the safety delta, the greater the chances of having duplicate records extracted. With a safety delta, you extract records that carry a time stamp lesser than that of the last successful extraction. An overlap of records in the two successive extractions occurs, and the longer the duration of safety delta, the more duplicate records are extracted.That is why we recommend using an ODS object in the data staging process to eliminate the duplicate records. Normally after extraction the records could be directly uploaded to an InfoCube for multidimensional analysis, but in this case, we recommend that the records first be loaded to an ODS. The ODS subsequently loads the records to an InfoCube. For more detail on employing an ODS to eliminate duplicate records, see our article, “Design Better Custom Extractors for ODS Delta Loads,” in the July/August 2004 issue of BW Expert.Now we’ll show you how to retrieve the value of the safety delta. To provide flexibility in the process, we created

one more Z table — ZSC_SAFETY_DELTA — with only one field (Figure 5).

Figure 5 Z table to designate the default safety delta

This table, which has just one record with the value in hours, is maintained manually. The value in hours is subtracted from the time stamp. The time stamp we are using is the date and time when the extraction started from the source system side. The value is extracted using ABAP code (used in the second function module). If you want to change the value of the safety delta, you do not need to change the ABAP code. If you want to go below one

hour or above 24 hours, you can use the Z table ZSC_SET_EXT_TIME (Figure 6).

Figure 6 Z table for less than one hour or more than 24 hours SAFETY DELTA

If you want to use the safety delta in hours, then the table in Figure 6 should be kept empty. To use the safety delta in seconds or minutes or days, put the appropriate entry in the custom table. The custom module accounts for the rest of the calculation. If table ZSC_SET_EXT_TIME has any record, then the safety delta in hours stored in the table ZSC_SAFETY_DELTA is not used. Table 1 shows how the entries are stored. Depending on the value of the

safety delta, you maintain the table ZSC_SAFETY_DELTA or table ZSC_SET_EXT_TIME.

If you want to subtract the value shown below from TIMESTAMP

Use the following entry in the table

ZSC_SET_EXT_TIME

  ZZ_TIME_FLAG ZZ_HOWMUCH

30 seconds S 30

1 minute M 1

2 days D 2

Table 1 What to fill in the table ZSC_SET_EXT_TIME

Note

All InfoPackages, regardless of the DataSource/source system executed, create a record in the table RSREQDONE. While getting the delta, our aim is to get the delta or time stamp for a specific DataSource/source system combination. Each DataSource has multiple entries in the table RSREQDONE depending on the number of times they were executed in the system. To get the correct time stamp, use INFOPACKAGE ID (as it is the only information available to begin with) to filter the records from the entire record set in the table RSREQDONE. Using this subset, you get the MAX of the time stamp using sort in descending order. Take the first entry of that sorted list.

Note

For the initial upload when you execute the InfoPackage, make sure that the value of the field ZZ_FIRST_DELTA is I. For the first delta extraction, make sure that the value of the field ZZ_FIRST_DELTA is Y for this InfoPackage entry. Once you extract the delta the first time, make sure to change this entry to N. This exercise is required only for the first delta after the full upload initial request. This procedure ensures that the program eliminates the entry for the initial upload InfoPackage while evaluating the InfoPackages for the deltas.

Step 3. Convert the Time DetailsNow that you have the time stamp of the last successful request (using step 1) and the value of the safety delta (using step 2), you need to subtract the safety delta value from the time stamp to start the next extraction.The time the system displays in the SAPGUI (as shown in the monitor) and the time in the database table are different. The database table RSREQDONE stores information in UTC format. That time is converted to local time when displayed in the SAPGUI. When you calculate the time stamp using the safety delta, it does not give the correct time stamp for the next retrieval. To get the correct time you need to convert the UTC time into your local time using the ABAP statement shown below.convert time stamp wa_rseqdone_ts

time zone sy-zonlointo date wa_ts_date

time wa_ts_time.The system variable SY-ZONLO allows the statement to convert the UTC TIMESTAMP into your local time. The above statement also splits the TIMESTAMP into two different local variables — WA_TS_DATE and WA_TS_TIME. You pass the date and time separately since the external application stores the values in two different fields.Our safety delta in the normal case is stored in hours and the last extracted TIMESTAMP is available in the form of date and time. You need to bring these two values in the same unit. We prefer to convert both these values in seconds to complete the calculation, as shown below.wa_safety_delta_in_seconds =

wa_default_safety_delta * 60 * 60.Here, the wa_default_safety_delta refers to the safety delta value picked up from table ZSC_SAFETY_DELTA. The statement converts the default safety delta time in seconds. The code below shows the ABAP code to convert date and time (result of ABAP code above) to time in seconds. wa_ts_time_in_seconds =

( wa_ts_time(2) * 60 * 60 ) + ( wa_ts_time+2(2) * 60 )+ wa_ts_time+4(2).

If the value of safety delta is greater than the time retrieved from the last request, you need to subtract a day with the statement shown below. if wa_ts_time_in_seconds > wa_safety_delta_in_seconds.

wa_ts_time = wa_ts_time — wa_safety_delta_in_seconds.wa_ts_date = wa_ts_date.

else.*** 24 Hours * 60 Minutes * 60 Seconds = 86400 seconds

wa_ts_time =86400 — ( wa_safety_delta_in_seconds -

wa_ts_time_in_seconds ).wa_ts_date = wa_ts_date — 1.

endif.

Note

The ABAP code has been broken into pieces in this article for the purpose of detailed explanation. To see the complete code, go to the Download Section of the BW Expert Web site at www.BWExpertOnline.com.

The calculations provide the time-stamp value required in the InfoPackage selection criteria MOD_TIME and MOD_DATE (parameter for extracting delta records as shown in Figure 2). However, we are providing the value of MOD_DATE as the low range (beginning limit for the date) and 99991231 as the high range (last limit for the date). This is because the records having the time stamp from 00:00:00 hours of the beginning date to the time derived from the safety delta will also be extracted. They will be removed in the start routine before you upload to the data target.

2nd Document

DB Connect

DB Connect was first introduced in BW 3.0 and enables BW to extract data from other relational databases that SAP products run on. Currently the list is Oracle, Microsoft SQL Server, IBM DB2, and platform-specific variants (DB4 and DB6), Informix, and SAP DB/MaxDB.

To make DB Connect work, you typically need your Basis team to set up the initial connection in BW. You also need input from a technical contact responsible for the source database to provide connectivity and logon information.

The main technical prerequisite for using DB Connect is for the BW server to install the relevant client connectivity software and associated SAP database shared library (DBSL). If your BW system runs on Oracle and you want to access another Oracle database within your organization, this is straightforward. However, if you run Oracle and want to access data from an IBM DB2 database, then you need to install the relevant IBM client software and SAP database libraries on your BW server. This can become even more complicated if your BW system is Unix based and you want to access Microsoft SQL Server. Microsoft does not provide a Unix client for SQL Server, so you can only do this by having at least one Windows-based application server.

The actual BW connection to the source database is straightforward to set up. Ideally you should create a dedicated database user on the source database for extraction purposes. This makes it possible to restrict what the BW extract user can access in the source database and also reduces the number of tables and views that you

see in DB Connect (Figure 2).

Figure 2 DB Connect table and view selection in BW

Once you have an active connection, DB Connect behaves similarly to other BW source systems with restrictions and limitations that you need to know. Firstly, table or view names in the source system need to be uppercase and

fewer than 26 characters in length. Secondly, the names of columns in the table or view are limited to 16 characters and can only contain letters, numbers, and underscores.

You might think that this would be a major problem, but the way around this is to create new database views on the source database to shorten table names and rename fields into a BW-friendly format. For example, if the table that you want to extract is customer_purchase_history_table (31 characters), you would create a view called BW_CUST_HIST_TRAN (17 characters, uppercase) to enable BW to successfully access this view. Create this view at the database level in the source database using the relevant SQL statement such as create view BW_CUST_HIST_TRAN as select FIELD1, FIELD 2 FROM customer_purchase_history_table. The exact syntax varies by database. Once the fields have the proper naming convention, you can generate DataSources from

source fields (Figure 3).

Figure 3 Generate DataSources from source fields

Database views are also invaluable for performing basic data manipulation such as converting the contents of a field to uppercase, performing substring operations (to provide the second to sixth characters of a field), and converting data types such as a native date-time field into a date or time format that BW can handle such as YYYYMMDD.

DB Connect performs very fast. In my experience, it’s faster than extracting data from SAP systems, but this obviously depends on where your source database is located and network performance. The majority of problems I’ve experienced with DB Connect usually relate to data quality, such as non-permitted characters and data type conversion errors. You should plan extra time to account for the increased difficulty of working with non-SAP data.