sand cdbms nearline for sap bw - informatica · accelerating the largest user populations, the...

Accelerating the largest user populations, the biggest data, and the most complex analyticsSM

SAND CDBMS Nearline for SAP BW

Backup and Restore Guide (BW 7)

Informatica

Version 3.1 MR2

February 2012

Copyright (c) 1998-2012 Informatica Corporation. All rights reserved.

This software and documentation contain proprietary information of Informatica Corporation and are provided under a license

agreement containing restrictions on use and disclosure and are also protected by copyright law. Reverse engineering of the

software is prohibited. No part of this document may be reproduced or transmitted in any form, by any means (electronic,

photocopying, recording or otherwise) without prior consent of Informatica Corporation. This Software may be protected by U.S.

and/or international Patents and other Patents Pending.

Use, duplication, or disclosure of the Software by the U.S. Government is subject to the restrictions set forth in the applicable

software license agreement and as provided in DFARS 227.7202-1(a) and 227.7702-3(a) (1995), DFARS 252.227-7013©(1)(ii)

(OCT 1988), FAR 12.212(a) (1995), FAR 52.227-19, or FAR 52.227-14 (ALT III), as applicable.

The information in this product or documentation is subject to change without notice. If you find any problems in this product or

documentation, please report them to us in writing.

Informatica, Informatica Platform, Informatica Data Services, PowerCenter, PowerCenterRT, PowerCenter Connect,

PowerCenter Data Analyzer, PowerExchange, PowerMart, Metadata Manager, Informatica Data Quality, Informatica Data

Explorer, Informatica B2B Data Transformation, Informatica B2B Data Exchange Informatica On Demand, Informatica Identity

Resolution, Informatica Application Information Lifecycle Management, Informatica Complex Event Processing, Ultra Messaging

and Informatica Master Data Management are trademarks or registered trademarks of Informatica Corporation in the United

States and in jurisdictions throughout the world. All other company and product names may be trade names or trademarks of

their respective owners.

Portions of this software and/or documentation are subject to copyright held by third parties, including without limitation:

Copyright DataDirect Technologies. All rights reserved. Copyright © Sun Microsystems. All rights reserved. Copyright © RSA

Security Inc. All Rights Reserved. Copyright © Ordinal Technology Corp. All rights reserved.Copyright © Aandacht c.v. All rights

reserved. Copyright Genivia, Inc. All rights reserved. Copyright Isomorphic Software. All rights reserved. Copyright © Meta

Integration Technology, Inc. All rights reserved. Copyright © Intalio. All rights reserved. Copyright © Oracle. All rights reserved.

Copyright © Adobe Systems Incorporated. All rights reserved. Copyright © DataArt, Inc. All rights reserved. Copyright ©

ComponentSource. All rights reserved. Copyright © Microsoft Corporation. All rights reserved. Copyright © Rogue Wave

Software, Inc. All rights reserved. Copyright © Teradata Corporation. All rights reserved. Copyright © Yahoo! Inc. All rights

reserved. Copyright © Glyph & Cog, LLC. All rights reserved. Copyright © Thinkmap, Inc. All rights reserved. Copyright ©

Clearpace Software Limited. All rights reserved. Copyright © Information Builders, Inc. All rights reserved. Copyright © OSS

Nokalva, Inc. All rights reserved. Copyright Edifecs, Inc. All rights reserved. Copyright Cleo Communications, Inc. All rights

reserved. Copyright © International Organization for Standardization 1986. All rights reserved. Copyright © ej-technologies

GmbH. All rights reserved. Copyright © Jaspersoft Corporation. All rights reserved.

This product includes software developed by the Apache Software Foundation (http://www.apache.org/), and other software

which is licensed under the Apache License, Version 2.0 (the "License"). You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0. Unless required by applicable law or agreed to in writing, software distributed

under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either

express or implied. See the License for the specific language governing permissions and limitations under the License.

This product includes software which was developed by Mozilla (http://www.mozilla.org/), software copyright The JBoss Group,

LLC, all rights reserved; software copyright © 1999-2006 by Bruno Lowagie and Paulo Soares and other software which is

licensed under the GNU Lesser General Public License Agreement, which may be found at http:// www.gnu.org/licenses/lgpl.html.

The materials are provided free of charge by Informatica, "as-is", without warranty of any kind, either express or implied,

including but not limited to the implied warranties of merchantability and fitness for a particular purpose.

The product includes ACE(TM) and TAO(TM) software copyrighted by Douglas C. Schmidt and his research group at

Washington University, University of California, Irvine, and Vanderbilt University, Copyright (©) 1993-2006, all rights reserved.

This product includes software developed by the OpenSSL Project for use in the OpenSSL Toolkit (copyright The OpenSSL

Project. All Rights Reserved) and redistribution of this software is subject to terms available at http://www.openssl.org and

http://www.openssl.org/source/license.html.

This product includes Curl software which is Copyright 1996-2007, Daniel Stenberg, <[email protected]>. All Rights Reserved.

Permissions and limitations regarding this software are subject to terms available at http://curl.haxx.se/docs/copyright.html.

Permission to use, copy, modify, and distribute this software for any purpose with or without fee is hereby granted, provided that

the above copyright notice and this permission notice appear in all copies.

The product includes software copyright 2001-2005 (©) MetaStuff, Ltd. All Rights Reserved. Permissions and limitations

regarding this software are subject to terms available at http://www.dom4j.org/ license.html.

The product includes software copyright © 2004-2007, The Dojo Foundation. All Rights Reserved. Permissions and limitations

regarding this software are subject to terms available at http://dojotoolkit.org/license.

This product includes ICU software which is copyright International Business Machines Corporation and others. All rights

reserved. Permissions and limitations regarding this software are subject to terms available at http://source.icu-

project.org/repos/icu/icu/trunk/license.html.

This product includes software copyright © 1996-2006 Per Bothner. All rights reserved. Your right to use such materials is set

forth in the license which may be found at http:// www.gnu.org/software/ kawa/Software-License.html.

This product includes OSSP UUID software which is Copyright © 2002 Ralf S. Engelschall, Copyright © 2002 The OSSP Project

Copyright © 2002 Cable & Wireless Deutschland. Permissions and limitations regarding this software are subject to terms

available at http://www.opensource.org/licenses/mit-license.php.

This product includes software developed by Boost (http://www.boost.org/) or under the Boost software license. Permissions and

limitations regarding this software are subject to terms available at http:/ /www.boost.org/LICENSE_1_0.txt.

This product includes software copyright © 1997-2007 University of Cambridge. Permissions and limitations regarding this

software are subject to terms available at http:// www.pcre.org/license.txt.

This product includes software copyright © 2007 The Eclipse Foundation. All Rights Reserved. Permissions and limitations

regarding this software are subject to terms available at http:// www.eclipse.org/org/documents/epl-v10.php.

This product includes software licensed under the terms at http://www.tcl.tk/software/tcltk/license.html,

http://www.bosrup.com/web/overlib/?License, http://www.stlport.org/doc/ license.html, http://www.asm.ow2.org/license.html,

http://www.cryptix.org/LICENSE.TXT, http://hsqldb.org/web/hsqlLicense.html, http://httpunit.sourceforge.net/doc/ license.html,

http://jung.sourceforge.net/license.txt , http://www.gzip.org/zlib/zlib_license.html,

http://www.openldap.org/software/release/license.html, http://www.libssh2.org, http://slf4j.org/license.html,

http://www.sente.ch/software/OpenSourceLicense.html, http://fusesource.com/downloads/license-agreements/fuse-message-

broker-v-5-3- license-agreement; http://antlr.org/license.html; http://aopalliance.sourceforge.net/;

http://www.bouncycastle.org/licence.html; http://www.jgraph.com/jgraphdownload.html; http://www.jcraft.com/jsch/LICENSE.txt.

http://jotm.objectweb.org/bsd_license.html; . http://www.w3.org/Consortium/Legal/2002/copyright-software-20021231;

http://www.slf4j.org/license.html; http://developer.apple.com/library/mac/#samplecode/HelpHook/Listings/HelpHook_java.html;

http://www.jcraft.com/jsch/LICENSE.txt; http://nanoxml.sourceforge.net/orig/copyright.html; http://www.json.org/license.html;

http://forge.ow2.org/projects/javaservice/, http://www.postgresql.org/about/licence.html, http://www.sqlite.org/copyright.html,

http://www.tcl.tk/software/tcltk/license.html, http://www.jaxen.org/faq.html, http://www.jdom.org/docs/faq.html,

http://www.slf4j.org/license.html; http://www.iodbc.org/dataspace/iodbc/wiki/iODBC/License; and

http://www.keplerproject.org/md5/license.html.

This product includes software licensed under the Academic Free License (http://www.opensource.org/licenses/afl-3.0.php), the

Common Development and Distribution License (http://www.opensource.org/licenses/cddl1.php) the Common Public License

(http://www.opensource.org/licenses/cpl1.0.php), the Sun Binary Code License Agreement Supplemental License Terms, the

BSD License (http:// www.opensource.org/licenses/bsd-license.php) the MIT License (http://www.opensource.org/licenses/mit-

license.php) and the Artistic License (http://www.opensource.org/licenses/artistic-license-1.0).

This product includes software copyright © 2003-2006 Joe WaInes, 2006-2007 XStream Committers. All rights reserved.

Permissions and limitations regarding this software are subject to terms available at http://xstream.codehaus.org/license.html.

This product includes software developed by the Indiana University Extreme! Lab. For further information please visit

http://www.extreme.indiana.edu/.

This Software is protected by U.S. Patent Numbers 5,794,246; 6,014,670; 6,016,501; 6,029,178; 6,032,158; 6,035,307;

6,044,374; 6,092,086; 6,208,990; 6,339,775; 6,640,226; 6,789,096; 6,820,077; 6,823,373; 6,850,947; 6,895,471; 7,117,215;

7,162,643; 7,243,110, 7,254,590; 7,281,001; 7,421,458; 7,496,588; 7,523,121; 7,584,422, 7,720,842; 7,721,270; and 7,774,791,

international Patents and other Patents Pending.

DISCLAIMER: Informatica Corporation provides this documentation "as is" without warranty of any kind, either express or implied,

including, but not limited to, the implied warranties of noninfringement, merchantability, or use for a particular purpose.

Informatica Corporation does not warrant that this software or documentation is error free. The information provided in this

software or documentation may include technical inaccuracies or typographical errors. The information in this software and

documentation is subject to change at any time without notice.

NOTICES

This Informatica product (the “Software”) includes certain drivers (the “DataDirect Drivers”) from DataDirect Technologies, an

operating company of Progress Software Corporation (“DataDirect”) which are subject to the following terms and conditions:

1. THE DATADIRECT DRIVERS ARE PROVIDED “AS IS” WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR

IMPLIED, INCLUDING BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A

PARTICULAR PURPOSE AND NON-INFRINGEMENT.

2. IN NO EVENT WILL DATADIRECT OR ITS THIRD PARTY SUPPLIERS BE LIABLE TO THE END-USER CUSTOMER FOR

ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, CONSEQUENTIAL OR OTHER DAMAGES ARISING OUT OF THE USE

OF THE ODBC DRIVERS, WHETHER OR NOT INFORMED OF THE POSSIBILITIES OF DAMAGES IN ADVANCE. THESE

LIMITATIONS APPLY TO ALL CAUSES OF ACTION, INCLUDING, WITHOUT LIMITATION, BREACH OF CONTRACT,

BREACH OF WARRANTY, NEGLIGENCE, STRICT LIABILITY, MISREPRESENTATION AND OTHER TORTS.

SAND CDBMS Nearline for SAP BW Backup and Restore Guide (BW 7)

Release 3.1 MR2 February 2012

Copyright ©2012 SAND Technology.

All rights reserved. SAND Technology and Nucleus are registered trademarks, and SAND, SAND CDBMS, SAND CDBMS Nearline, SAND/DNA Access, and other SAND- and Nucleus-related terms are trademarks of SAND Technology Inc. SAP, SAP NetWeaver, the SAP logo, and the SAP Partner logo are trademarks or registered trademarks of SAP AG in Germany and in several other countries all over the world. Other trademarks remain the property of their respective owners.

Visit SAND Technology on the Web at www.sand.com. 12/01/26


Table of Contents

1 INTRODUCTION...................................................................................................................... 1

2 GENERAL CONSIDERATIONS .............................................................................................. 1

2.1 THE BACKUP PROCESS ....................................................................................................... 2

2.2 SYNCHRONIZATION OF THE SAP BW DATABASE WITH THE NEARLINE REPOSITORY................ 3

3 OFFLINE BACKUP.................................................................................................................. 3

4 RESTORING AN OFFLINE BACKUP..................................................................................... 3

5 ONLINE BACKUP USING SNAPSHOT TOOLS .................................................................... 4

6 RESTORING AN ONLINE BACKUP PERFORMED WITH SNAPSHOT TOOLS.................. 4

7 STANDARD ONLINE BACKUP .............................................................................................. 4

7.1 VIEWING A REPORT ON STANDARD ONLINE BACKUPS............................................................ 6

8 RESTORING A STANDARD ONLINE BACKUP.................................................................... 8

9 CHECKING CONSISTENCY ................................................................................................. 10

9.1 SAP CONSISTENCY CHECK PROGRAM ............................................................................... 10

9.1.1 Data Archive Process .............................................................................................. 11

9.1.2 Data Archive Request .............................................................................................. 11

9.2 THE SAND/DNA ACCESS METADATA VALIDATION PROGRAM ............................................. 13

9.3 CONSISTENCY CHECK PROCESS FLOW .............................................................................. 16

9.4 CONSISTENCY CHECK EXECUTION ..................................................................................... 17


1 Introduction

This document describes the procedures involved in backing up and recovering SAND CDBMS Nearline for SAP BW. As well, this document describes how to validate the consistency of the SAP/SAND environment after a restore process, and the way to correct inconsistencies if they exist. It is intended for SAP BW system administrators working with the NLS (NearLine Storage) component, and covers general data management considerations as well as the step-by-step procedures for performing backup, restore, and consistency verification in a SAND CDBMS Nearline for SAP BW environment. Familiarity with the use of NLS for SAP BW is assumed.

Notes:

• While the procedures described in this document can be used on both UNIX and Windows platforms, UNIX file-naming conventions have been used throughout for convenience.

• Some of the sample commands assume a UNIX installation, or a Windows installation for all users. Keep in mind that a Windows installation for a specified user will have the user name and domain appended to the names of the Metadata Repository database, ODBC DSNs, related connection and database sections in the nucleus.ini file, and the SNIC service. Such user-specific naming is not reflected in the examples of this document.

2 General Considerations

A system backup should be performed once after the software is installed and again after each change to the system configuration.

After installing, the following directories should be backed up:

bin

dna SAND/DNA Access installation directory

docs

conf

lib

meta

tasks/sand_nlic.dtd

tasks/sand_binli.dtd

1


tasks/XMLFileBWInterface.properties SAP BW 3.x only

install.bat Windows install

install.sh UNIX install

Restoring the system backup will also require recreation of the tmp, logs, and sct directories.

Each time the SAND CDBMS Nearline configuration is changed, the following files should be backed up:

bin/sand_nlic.properties

dna/nucleus.ini

dna/ssa.ini

dna/odbc.ini UNIX only

bin/sand_nlic.properties

tasks/sand_nlic.dtd

tasks/sand_binli.dtd

tasks/XMLFileBWInterface.properties SAP BW 3.x only

When backing up SAND CDBMS Nearline, the contents of the meta (nearline metadata) and sct (compressed nearline data) directories must be backed up. Backup of log files (in the logs directory) is optional, but recommended.

Backups can be performed either online (when SAND CDBMS Nearline is in operation) or offline (after SAND CDBMS Nearline has been stopped). Offline backup involves simply making copies of the relevant SCT files and Metadata Repository database (meta.n00) file. An online backup involves dumping the contents of the online Metadata Repository database using the SAND/DNA Access add_task command.

Because SCT (compressed nearline data) files are read-only and cannot be changed, it is possible to implement an incremental backup strategy, where only the files that have been added since the last backup are affected.

2.1 The Backup Process

From a high-level perspective, the steps involved in performing a backup of an SAP BW system using an NLS component are as follows:

1. Use the SAP NLS interface to execute all nearline requests up to the verification step.

2. Back up the NLS component as described below.

2


3. Back up the SAP BW database.

4. Delete the data from the SAP online database (using the NLS interface).

5. Back up the NLS component a second time (see below).

2.2 Synchronization of the SAP BW Database with the Nearline Repository

A major consideration when restoring an SAP BW NLS repository from backup is the need to keep it synchronized with the SAP BW database. Because the NLS repository is only updated when data is moved to nearline storage from SAP BW, subsequent backups of the online database will not affect the NLS repository until another nearline request has been executed.

As an example, consider a situation where SAP data is moved to nearline storage on Monday October 22, 2007. The NLS component is then backed up using the process described above. If an SAP BW database restore is required on Wednesday October 24, restoring the database to its state on Tuesday October 23, and no other data has been moved to nearline since Monday October 22, only the SAP BW database will need to be restored.

On the other hand, if the database needs to be restored to its state on the previous Friday, October 19, then both the SAP BW database and the nearline repository will need to be restored.

3 Offline Backup

The procedure for performing an offline backup of the SAND CDBMS Nearline for SAP BW NLS component is as follows:

1. Stop the SAND Nearline Integration Controller (SNIC) Server (SAND CDBMS Nearline for SAP BW Installation and Setup Guide (BW 7), chapter 3).

2. Copy the meta.n00 and relevant SCT files to the backup location.

3. Restart the SNIC Server (see SAND CDBMS Nearline for SAP BW Installation and Setup Guide (BW 7), chapter 3).

4. Delete the data from the SAP database when the nearline backup has successfully completed.

5. Repeat steps 1, 2 and 3.

An SAP backup should be taken at the same time as the first SAND CDBMS Nearline backup.

4 Restoring an Offline Backup

To restore the nearline repository from an offline backup:

1. Verify that backups are available.

3


2. Stop the SNIC Server (see SAND CDBMS Nearline for SAP BW Installation and Setup Guide (BW 7), chapter 3).

3. Remove the existing meta.n00 and *.sct files.

4. Restore the meta.n00 and *.sct files from backup.

5. Restart the SNIC Server (see SAND CDBMS Nearline for SAP BW Installation and Setup Guide (BW 7), chapter 3).

Note that restoring a SAND CDBMS Nearline for SAP BW repository from before the deletion step in SAP BW (which usually involves restoring the SAP system as well) also requires rerunning the deletion steps for pending nearline requests in SAP BW.

5 Online Backup Using Snapshot Tools

Many commercial backup software packages provide a facility for performing snapshot backups of file systems. These file system snapshots, which consist of a locally retained, point-in-time image of data usually associated with a mirror image of the file system, can be used to back up SCT files and the metadata repository database for a SAND CDBMS Nearline system. While a standard online backup cannot assure the integrity of the metadata repository without performing a dump of the data it contains, a snapshot backup of the repository will not be subsequently modified and so can be backed up from the mirrored copy. The snapshot of the metadata repository database can then be used to restore the SAND CDBMS Nearline metadata.

In performing an online backup using snapshot tools, snapshots of the sct and meta directory file systems must be taken at the same time. The backup can then be performed on the snapshot. A snapshot backup can be performed at any time, but it is recommended to use this approach within the context of the backup process as described in section 2.1.

6 Restoring an Online Backup Performed with Snapshot Tools

To restore an online backup performed using snapshot tools, follow the steps for the offline backup/restore process described in section 4 of this document.

7 Standard Online Backup

An online backup of nearline data differs from the offline approach in that it involves creating a backup of the meta.n00 metadata database file while the system is running. This is performed by executing the SAND/DNA Access add_task command to export the contents of the metadata database to a tar file, which can subsequently be backed up to tape. SCT files can be backed up at any time by simply copying them to another location, since they are "closed" at the operating system level once they have been successfully created. An online backup of the SAND CDBMS Nearline repository should be performed when no nearline requests are copying data to the SAND CDBMS Nearline repository (this can be controlled by process chains).

4


To perform the online backup, create a location for the backed up metadata, then execute the add_task –b command using the syntax below:

add_task –b –l <metadata backup location> -t meta meta dba

Use the –l flag to specify the backup location and –t to create a tar file with the backup data. The MAX REQUESTID value appearing in the output will increase with each backup task added.

Example:

add_task -b -l /sand_dvd/sand/meta_dump -t meta meta dba

Sample Output:

SAND/DNA Access Add Task Utility ... ...

Getting MAX REQUESTID...52

Inserting a new record into SSA_SERVICE.SSA_TASK TABLE (BACKUP)...Ok

Done!

Once the backup task is added, start the ssamon utility to check the execution of the backup job:

ssamon

Sample Output:

SAND/DNA Access Service Monitor ... ...

Service/Jobs/Steps PID UID PSR CPU MEM ------------------------- ----------- ----------- ----- -------- ---- SSASERVICE 4471 nlsadm - 0.60 9968 JOB.BACKUP_52 4492 nlsadm - 3.14 7307

Once the backup job has finished, it should no longer appear in the ssamon process list:

SAND/DNA Access Service Monitor ... ...

Service/Jobs/Steps PID UID PSR CPU MEM ------------------------- ----------- ----------- ----- ----- ---- SSASERVICE 4471 nlsadm - 0.05 3

5


Check the backup directory to verify that the tar file (ssa_metadata_backup_<timestamp>.tar) has been created.

Finally, query the ssa_service.ssa_task table in the metadata database to verify that the backup job has completed successfully (that is, has a status of "done"). Use a command like the following (note that the REQUESTID specified in the WHERE clause should be the one returned when the add_task command was executed, as described above):

Example:

echo "select * from SSA_SERVICE.SSA_TASK where REQUESTID = 52;" | ssasql meta meta DBA

Sample: Output:

SAND/DNA Access ... Interactive SQL Utility ... ...

SESSION 1: META@meta

SQL:1> select * from SSA_SERVICE.SSA_TASK where REQUESTID = 52;

1 row selected

REQUESTID JOBTYPE STATUS FREQ DAY_NU HOURS SOURCE ----------- ---------- ---------- ---------- ------ ------ ------ 52 BACKUP DONE ad hoc

1 row fetched

The backup is valid only if the status of the job is displayed as DONE. Otherwise, the backup could still be in process (Status=IN PROCESS), finished but did not complete all subtasks (Status=INCOMPLETE), or did not complete successfully (Status=FAILED).

7.1 Viewing a Report on Standard Online Backups

The /SAND/0_UTIL_BACKUP_LIST program can be used to generate a report about all previous and current standard online backup tasks from a specified starting date.

To generate a backup tasks report:

1. Run the /SAND/0_UTIL_BACKUP_LIST program on transaction SA38.

2. Configure the settings for the report:

Nearline Connection: Specify the appropriate nearline connection. Default: current nearline connection.

Wait for file seconds: Specify the timeout period (in seconds) for retrieving the backup task information. Default: 10 seconds.

6


Backup Start Date: Specify the start date for the backup tasks. Backup tasks that were performed before this date will not be included in the report. Default: one year before the current date.

3. Generate the backup tasks report by clicking the Execute button (F8). The report will look something like this:

The fields in the backup tasks report are as follows:

Bckp Req.: The internal request ID for the backup task

BckpStatus: The current or final status of the backup task. The possible backup states are:

• DONE (completed successfully)

• IN PROCESS (currently executing)

• INCOMPLETE (finished, but not all subtasks could be completed)

• FAILED (did not complete successfully)

Backup Start: The timestamp when the backup task was started

Backup Target: The full path of the resulting backup file

7


BckAttemp: The number of backup task attempts

8 Restoring a Standard Online Backup

Online recovery of the SAND CDBMS Nearline component requires stopping the SNIC Server, then restoring the meta.n00 metadata database from the tar file created by the backup task described in section 7 of this document. Most of this recovery procedure is handled automatically by the rest_backup command.

To restore a standard backup of SAND CDBMS Nearline:

First, stop the SNIC Server (see SAND CDBMS Nearline for SAP BW Installation and Setup Guide (BW 7), chapter 3).

Next, execute the rest_backup command, specifying the location and date of the backup:

rest_backup -l <tar file location> -d <restore date> -c meta meta DBA

The restore date must be expressed in the format yyyy-mm-dd.

Example:

rest_backup -l /sand_dvd/sand/meta_dump -d 2007-06-09 –c meta meta DBA

The rest_backup command will automatically perform the following actions in sequence:

a. The existing meta.n00 file (in the meta directory of the SNIC directory structure) will be backed up to meta.n00.<timestamp>.

b. A new meta.n00 file will be created from the mddb.n00 database template file (located in the main SAND/DNA Access directory)

c. The new "empty" metadata database will be started via ssaeng.

d. The metadata database will be restored using the saved metadata from the specified backup.

e. The metadata database will shut down.

Sample Output (excerpt:)

Checking access to the restore database... Database is not mounted. Creating new database... command line : ssaeng mddb mddb Ok Done Starting a metadata cleanup... * Getting a schemas list...Done! *

8


* Getting a views list...Done! * * Getting a table list...Done! * * Getting a views list...Done! * * Getting a table list...Done! * * Getting a domain list...Done! * * Getting a domain list...Done! * * Getting a authorization list...Done! * * Metadata cleaned up * Creating a metadata structure... * 13789_metadata_2007-06-09-14-40-11.ddl...Ok * Metadata structure created * Loading metadata... * 13789_ssa_task_2007-06-09-14-46-46.sct...Ok * 13789_ssa_work_2007-06-09-14-46-47.sct...Ok * 13789_ssa_version_2007-06-09-14-46-47.sct...Ok * 13789_ssa_ini_2007-06-09-14-46-47.sct...Ok * 13789_ssa_backup_2007-06-09-14-46-47.sct...Ok * 13789_ssa_nearline_2007-06-09-14-46-51.sct...Ok * 13789_ssa_nearline_log_2007-06-09-14-46-52.sct...Ok * 13789_sds_version_2007-06-09-14-46-53.sct...Ok * 13789_sds_table_2007-06-09-14-46-53.sct...Ok * 13789_sds_file_2007-06-09-14-46-55.sct...Ok * 13789_sds_file_dom_2007-06-09-14-47-27.sct...Ok * 13789_sds_file_col_2007-06-09-14-48-21.sct...Ok * Metadata loaded successfully * Restoring constraints... * 13789_metadata_constraints_2007-06-09-14-40-11.ddl...Ok * Constraints restored successfully * Stopping METADATA database... Done THE 2007-06-09 METADATA BACKUP HAS BEEN RESTORED

At this point, the metadata backup for 2007-06-09 has been restored. The metadata database will then be shut down automatically.

Once the relevant SCT file backups have been restored (by copying them back to their original location), the SNIC Server can be restarted.

9


9 Checking Consistency

Note: This section does not apply to SAP BW 3.5.

After a successful restore, the consistency between SAP, SAND CDBMS Nearline, and SCT files must be verified. From a detailed perspective, there are five major components that must be inspected to ensure that the data and metadata are correct at all levels. These components are:

• SCT files (the compressed objects containing the nearline data)

• SAND/DNA Access (the application that provides nearline access to the data)

• SAND Services (services that maintain the operations of the SAND/DNA Access application)

• SAND Nearline Interface Controller (SNIC, the interface that bridges SAP and SAND/DNA Access)

• SAP BW

SAND/DNA Access, SAND Services, and SNIC all maintain their control information within separate schemas in the SAND repository.

Consistency checking is comprised of several phases that are performed using the SAP Consistency Check program and the SAND/DNA Access Metadata Validation program. The SAP Consistency Check program is an ABAP application that analyzes and compares the SAP metadata to the SAND CDBMS Nearline metadata, reporting any warnings or errors found. The SAND/DNA Access Metadata Validation program performs similar tasks on the SCT components, SAND/DNA Access, SAND Services, and SNIC. The SAND/DNA Access Metadata Validation program has three modes of operation:

• Report mode that will report all inconsistencies without taking any action

• Fix mode that can correct inconsistencies

• CSV mode that implements changes to coordinate with SAP and then corrects inconsistencies.

9.1 SAP Consistency Check Program

The SAP Consistency Check program will verify the data and object structures between SAP BW and the SAND Nearline Interface Controller. The Data Archive Process (DAP), as well as any requests associated with a DAP, will be analyzed for consistency.

10


9.1.1 Data Archive Process From a Data Archive Process standpoint, the definition of the table stored in nearline must match the structure contained with SAP. At different points in time, this structure might change. The following table displays the various scenarios that will be evaluated.

DAP Definition

SAP NLS

DAPn Not Defined Case 1

DAPn DAPn OK

DAPn DAPx Case 1

Not Defined DAPn Case 2

DAPn is the current state of the Data Archive Process definition. DAPx is a Data Archive Process definition at a different state, either a previous version, a later version, or a completely different definition from what is stored within SAP. The only satisfactory situation is when the SAND CDBMS Nearline version matches that of SAP.

• In Case 1, the resolution is to reactivate the Data Archive Process definition within SAP, which will automatically update the definition within SAND CDBMS Nearline, ensuring both sides are synchronized.

• In Case 2, since the DAP does not exist within SAP, it is a simple matter to drop the DAP defined within SAND CDBMS Nearline and all associated requests.

In either Case 1 or Case 2, the SAND/DNA Access Metadata Validation program must be executed after any modifications to ensure a consistent SAND repository.

9.1.2 Data Archive Request The requests are associated with an existing DAP. Therefore, once the DAP has been validated, any requests within each DAP must be validated. Refer to the following table for inconsistency scenarios involving data archive requests:

11


SNIC State

SAP State not

present created copying closed inactive active invalid cancelled unreg

not present OK Case4 Case4 Case4 Case4 Case4 Case4 Case4 Case4

initial 0 OK Case4 Case4 Case4 Case4 Case4 Case4 Case4 Case4

CREATED 10 OK Case4 Case4 Case4 Case4 Case4 Case4 Case4 Case4

changes 20 OK Case4 Case4 Case4 Case4 Case4 Case4 Case4 Case4

LOCKED 30 OK Case1 Case1 Case1 Case1 Case1 Case1 Case1 Case1

write / active 35, 38 Case1 Case1 Case1 Case1 Case1 Case1 Case1 Case1 Case1

WRITE 40 Case1 Case1 Case1 OK Case1 Case1 Case1 Case1 Case1

verify / active 45 Case1 Case1 Case1 Case1 Case1 Case1 Case1 Case1 Case1

VERIFY 50 Case1 Case1 Case1 Case1 OK Case1 Case1 Case1 Case1

delete / active 55,60,65 Case6 Case6 Case6 Case6 Case5 OK Case1 Case1 Case1

DELETE 70 Case2 Case2 Case2 Case2 Case5 OK Case2 Case2 Case5

reload / active 75 Case2 Case2 Case2 Case2 Case5 OK Case2 Case2 Case7

RESTORE 80 Case3 Case5 Case5 Case5 Case5 Case5 OK Case5 Case5

Nearline delete / active

85 Case3 Case5 Case5 Case5 Case5 Case5 OK OK Case5

NEARLINE DELETE

90 Case3 Case5 Case5 Case5 Case5 Case5 OK OK Case5

CANCEL 99 Case3 Case5 Case5 Case5 Case5 Case5 OK OK Case5

UNREGISTER Case2 Case2 Case2 Case2 Case5 Case5 Case2 Case2 OK

REREG_CHA Case2 Case2 Case2 Case2 Case8 Case8 Case2 Case2 Case8

UNREG_CHA Case2 Case2 Case2 Case2 Case8 Case8 Case2 Case2 Case8

• In Case1, the request must be manually invalidated within SAP. This process will automatically cancel the request within SAND CDBMS Nearline, ensuring that both sides are consistent.

• Case2 is a very serious situation indicating that there has been a loss of data. SAND should be contacted, and the request will have to be manually invalidated within SAP.

• Case3, while exhibiting an inconsistency, is not an issue, as the data or request is not needed within SAND CDBMS Nearline. In this case no error is reported.

12


• Case4 is a scenario where no requests are needed within SAND CDBMS Nearline, and so if they exist, they should be removed within SNIC.

• Case5 represents a mild case of inconsistency where the data exists and is stable. Therefore updating the state within the SNIC metadata will provide a coherent synchronization.

• Case6 is another important situation: if it is not remedied quickly, it could result in Case2 and a loss of data. In this condition, the request must be invalidated within SAP as soon as possible.

• Case7 is actually either Case1 or Case2, depending on the phase:

o Case1 if the restore request is in the copy or verification phase.

o Case2 if the restore request is in the deletion phase.

• Case8 represents a warning to complete the unregistering or registration activity before running the consistency check.

9.2 The SAND/DNA Access Metadata Validation Program

The SAND/DNA Access Metadata Validation program will verify the data and object structures for consistency between the SCT files, SAND/DNA Access, SAND/DNA Access Services, and SNIC.

The first verification is between the SCT files and the metadata for SAND/DNA Access and the Services. These tests can be further broken down to the SCT file, the metadata table, and their correlation. Refer to the following table for inconsistency scenarios involving data/object structures:

SCT Metadata

SSAService meta TableID FileID SCT File Resolution

Present/Missing Present Present Present OK

Present/Missing Present Present Missing Case1

Present/Missing Present Missing Present Case2

Present/Missing Missing Present Present Case3

Present/Missing Missing Present Missing Case4

Present Missing Missing Present OK

Present Missing Missing Present Case2

Missing Missing Missing Present Case2

Missing Present Present Present Case 4

13


From the above table we have the following results:

• Case1 is the scenario where we have missing data. In this case, the SCT file should be restored from a previous backup.

• Case2 indicates that some metadata does not exist in relation to the SCT file. Therefore the SCT file can be deleted from the disk.

• Case3 is the situation where no table is defined for an SCT file that exists and is referenced within the metadata. In this case, the SCT file should be unregistered from SAND/DNA Access, and then the SCT file can be deleted from the disk.

• Case4 indicates that either the Service metadata does not exist for the particular SCT file, or the table and the SCT file do not exist. In this situation, the SCT file should be unregistered from SAND/DNA Access.

Once the SCT files are synchronized with SAND/DNA Access and SAND/DNA Access Services, SNIC must be brought into line. The following table portrays the different situations at this level:

SNIC State SAND/DNA Access Service not

present created copying closed inactive active invalid cancelled unreg rereg

MISSING OK OK Case1 Case1 Case1 Case1 OK OK Case1 Case1

NOT REGISTERED Case2 Case2 Case1 Case1 Case4 Case5 Case2 Case2 Case7 Case5

OFFLINE Case2 Case2 Case2 OK OK Case5 Case2 Case2 Case7 Case5

ONLINE Case2 Case2 Case2 Case6 Case4 OK Case2 Case2 Case7 OK

DELETED Case2 Case1 Case1 Case1 Case1 Case1 OK OK Case1 Case1

UNREG SCT Case2 Case2 Case2 Case6 Case7 Case7 Case2 Case2 OK Case5

The synchronization between the SAND/DNA Access Services and SNIC are categorized by the following cases:

• Case1 involves invalidating the associated SNIC request. The action implies removing all corresponding request information.

• Case2 implies additional information exists within SAND/DNA Access Services. In this case, the action taken is to delete the SAND/DNA Access Service request.

• Case3 would require removing the associated information from the SAND/DNA Access metadata, however this case does not occur in the current implementation.

• In Case4, SNIC thinks the data exists but is not generally available. In this situation, the SAND/DNA Access Service state should changed to an OFFLINE state.

14


• In Case5, SNIC thinks the data exists and is generally available. In this situation the SAND/DNA Access Service state should be altered to an ONLINE state.

• Case6 is an unstable situation. In this case, the associated table should be unregistered, and a cleanup of all related metadata should take place.

• Case7 involves unregistering the SCT file and setting it to the UNREG state.

15


9.3 Consistency Check Process Flow

The following diagram summarizes the consistency check process:

16


9.4 Consistency Check Execution

To execute the consistency check:

1. Verify the existence of a proper backup of the SAP BW and SAND CDBMS Nearline environments. If such a backup does not exist, refer to the previous chapters of this document for information about creating backups at regular intervals.

2. Set the SAND CDBMS Nearline environment to run in “Recovery Mode”. In this mode, to prevent users from making changes to the environment state while consistency is being verified and corrected, only specified users are allowed to connect to the SNIC server. These Recovery Mode users should engage only in consistency check procedures.

Either prior to starting the SAND CDBMS Nearline environment or after shutting down the SAND CDBMS Nearline environment ($SAND_NLIC/bin/snic stop), edit the $SAND_NLIC/bin/sand_nlic.properties file and add or change the following parameters (substitute one or more actual users for “user1”/“user2”):

a. sand.recovery.mode=true

b. sand.recovery.users=user1,user2

3. Restart the SAND CDBMS Nearline environment ($SAND_NLIC/bin/snic start). If the metadata repository database (meta.n00 in the meta directory of the SNIC directory structure) cannot be accessed or is corrupted, contact a SAND Technology support representative as soon as possible.

4. Run the SAP Consistency Check program. Within the SAP environment:

a. Open the transaction SA38.

b. On the initial screen, choose the program “/SAND/0_CONSISTENCY_CHECK” and then click the Execute button (F8). This will bring up the following screen to check the consistency between SAP and SAND CDBMS Nearline:

c. You can now choose to verify all schemas within the SAND CDBMS Nearline environment or a particular schema that is associated with one or more SAND CDBMS Nearline connections.

d. After selecting the schema(s), click the Execute button (F8). You will be presented with the output of the consistency check, which lists the terminations, errors, warnings, and information items associated with consistency messages for the DAP/table and request levels, grouped by schema.

17


Note that only SAP NLS request IDs are reported, not the original SAP archive request IDs.

Each message provides a brief description of the issue and the cause of the issue. This screen provides the ability to view more detailed information for each message, filter and sort the messages, search through the list of messages, and print or export the list.

e. Click on the check mark or “Continue” button (ENTER) to view the SAND SAP Consistency Check report. This report details the steps required to reestablish consistency between the SAP online environment and the SAND CDBMS Nearline environment.

18


If there are any instances where requests were lost in SAND CDBMS Nearline, a warning message will be returned, along with the suggestion that you contact a SAND Technology support representative, which should be done as soon as possible.

5. Run the SAND/DNA Access Metadata Validation program in “Report” mode to determine if there are any instances of possible data loss:

a. From the command prompt, run ssavalidate. The syntax is as follows:

ssavalidate [ <flags> ]

Flags (can be specified in any order):

-h help: display this screen

-c test the integrity of the registered SCT files in the -s locations

-d delete orphan SCT files only

-f fix metadata and delete orphan SCT files automatically

-l <CSV file> fix SNIC server metadata using CSV information generated by the SAP Consistency Check program

19


-m n specify the number of threads used for SCT file integrity checking (the default is 1)

-r n delete only orphan SCT files older than n hours (the default is 0.5, which is 30 minutes)

-s <dir>[,<dir>] SCT files path(s). Multiple comma-separated paths can be specified; if a path contains spaces, it must be enclosed by quotation marks. The default path is "%SSA_INI_DIR%\..\sct" (Windows) or "$SSA_INI_DIR/../sct" (UNIX).

-t <dir> working directory for log and runtime files. The default path is "%SSA_INI_DIR%\..\tmp" (Windows) or "$SSA_INI_DIR/../tmp" (UNIX).

Default mode: validate metadata only

For example:

ssavalidate –c –m 4 –s $SAND_NLIC/sct/ -t $SAND_NLIC/logs/

This will look for metadata issues and orphan SCT files, then test the integrity of any SCT files (-c) listed in the metadata and that exist in the specified file paths (-s) using four threads in parallel (-m). The results are displayed on-screen and in a summary report file in the temporary folder (-t).

b. Verify the output in the generated report_<timestamp>.log file, located in the folder specified by the ssavalidate -t option.

c. If any corrupted SCT files are detected or manual corrections are required, contact a SAND Technology support representative.

6. If the previous step did not present any data-loss situations, it is safe to use the SAND/DNA Access Metadata Validation program to fix inconsistencies:

a. From the command prompt, run ssavalidate in “Fix” mode (the ssavalidate syntax is described in step 5). For example:

ssavalidate -c -f -m 4 –r 720 -s $SAND_NLIC/sct/ -t $SAND_NLIC/logs/

This will look for, and automatically attempt to fix, metadata issues. Orphan SCT files will also be detected. All orphan SCT files older than 720 hours (30 days) in the $SAND_NLIC/sct/ path will be deleted, followed by a test, using four threads in parallel (-m 4), of the integrity of any SCT files (-c) listed in the metadata that remain in the specified file paths (-s). The results are displayed on-screen and in a summary report file in the temporary folder (-t).

b. Verify the output in the generated report_<timestamp>.log file, located in the folder specified by the ssavalidate -t option.

20


c. The summary at the end of the report should display:

MANUAL CORRECTIONS REQUIRED : 0 ERRORS FIXED AUTOMATICALLY : 0

If errors continue to exist, then rerun the ssavalidate command from step a. above. If, after three iterations, errors continue to exist, please contact a SAND Technology support representative.

7. Run the SAP Consistency Check program to generate the appropriate CSV file needed to complete the corrections to any discovered inconsistencies:

a. Follow the procedure from step 3 above.

b. Once you have reached the final report, copy the CSV data into an ASCII text file. The CSV data should look similar to the following:

RequestType,Schema,Table,RequestID,State "table_delete","PUBLIC","/BIC/ONZSND_C00I","","" "request_delete","PUBLIC","/BIC/ONZSND_C01I","115","" "request_update","PUBLIC","/BIC/ONZSND_C26I","208","active" "request_update","PUBLIC","/BIC/ONZSND_C42I","246","invalid"

To save text from the report, there are two options:

i. Save the entire report: From the System menu choose List> Save> Local File. In the popup you should the select “In the Clipboard” and paste the clipboard data into a text file.

ii. Save part of the report: Place the cursor anywhere in the report, press <Ctrl+Y>, and mark the desired text. A red square indicates the cursor, while the purple rectangle indicates marked text.

21


When the desired text is selected, press <Ctrl+C> to copy the text, and then paste it into a text (CSV) file.

8. Use the saved CSV file with the SAND/DNA Access Metadata Validation program:

a. From the command prompt, run ssavalidate in CSV mode (the ssavalidate syntax is described in step 5). For example:

./ssavalidate -f -l scc.csv -r 720 -s $SAND_NLIC/sct/ -t $SAND_NLIC/logs/

First, this command will implement the required alterations from the CSV file (-l). It will then look for metadata issues (which it will automatically try to fix) and orphan SCT files. Any orphan SCT files older than 720 hours (30 days) in the $SAND_NLIC/sct/ path will be deleted. The results are displayed on-screen and in a summary report file in the temporary folder (-t).

b. Verify the output in the generated report_<timestamp>.log file

c. The summary at the end of the report should display:

MANUAL CORRECTIONS REQUIRED : 0 ERRORS FIXED AUTOMATICALLY : 0

If errors continue to exist, rerun the ssavalidate command from step a. above. If, after three iterations, errors continue to exist, please contact a SAND Technology support representative.

9. Run the SAND SAP Consistency Check program to ensure no CSV issues are needed as in step 7. At this point the report should not display any CSV data.

10. Execute the manual actions listed in the SAP Consistency Check final report from the previous step to deal with any final inconsistencies in SAP. For example, the report might state:

manual actions to recover inconsistencies in SCHEMA PUBLIC

22


Please activate the following DAPs ZSND_C44I ZSND_C45I

Please invalidate the following requests 120 of DAP ZSND_C04I 140 of DAP ZSND_C09I 142 of DAP ZSND_C10I

11. Run the SAND SAP Consistency Check program to ensure a consistent environment.

If issues continue to exist, please contact your SAND Technology support representative.

12. Turn off SNIC “Recovery Mode” in the SAND CDBMS Nearline environment:

a. Stop SNIC ($SAND_NLIC/bin/snic stop).

b. Once the SAND CDBMS Nearline environment is stopped, edit the $SAND_NLIC/bin/sand_nlic.properties file and set the parameter sand.recovery.mode to false:

sand.recovery.mode=false

c. Restart the SAND CDBMS Nearline environment ($SAND_NLIC/bin/snic start).

The consistency check process, including verification and correction, should now be complete.

23

sand cdbms nearline for sap bw - informatica · accelerating the largest user populations, the...

Documents