veritasstoragefoundation for oracle® rac release …storage foundation for oracle rac release notes...

Veritas Storage Foundation™for Oracle® RAC ReleaseNotes

AIX

6.0

November 2011

Veritas Storage Foundation™ for Oracle RAC ReleaseNotes

The software described in this book is furnished under a license agreement andmay be usedonly in accordance with the terms of the agreement.

Product version: 6.0

Document version: 6.0.0

Legal NoticeCopyright © 2011 Symantec Corporation. All rights reserved.

Symantec, the Symantec logo, Veritas, Veritas Storage Foundation, CommandCentral,NetBackup, Enterprise Vault, and LiveUpdate are trademarks or registered trademarks ofSymantec corporation or its affiliates in the U.S. and other countries. Other names may betrademarks of their respective owners.

The product described in this document is distributed under licenses restricting its use,copying, distribution, and decompilation/reverse engineering. No part of this documentmay be reproduced in any form by any means without prior written authorization ofSymantec Corporation and its licensors, if any.

THEDOCUMENTATIONISPROVIDED"ASIS"ANDALLEXPRESSORIMPLIEDCONDITIONS,REPRESENTATIONS AND WARRANTIES, INCLUDING ANY IMPLIED WARRANTY OFMERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR NON-INFRINGEMENT,ARE DISCLAIMED, EXCEPT TO THE EXTENT THAT SUCH DISCLAIMERS ARE HELD TOBELEGALLYINVALID.SYMANTECCORPORATIONSHALLNOTBELIABLEFORINCIDENTALOR CONSEQUENTIAL DAMAGES IN CONNECTION WITH THE FURNISHING,PERFORMANCE, OR USE OF THIS DOCUMENTATION. THE INFORMATION CONTAINEDIN THIS DOCUMENTATION IS SUBJECT TO CHANGE WITHOUT NOTICE.

The Licensed Software andDocumentation are deemed to be commercial computer softwareas defined in FAR12.212 and subject to restricted rights as defined in FARSection 52.227-19"Commercial Computer Software - Restricted Rights" and DFARS 227.7202, "Rights inCommercial Computer Software or Commercial Computer Software Documentation", asapplicable, and any successor regulations. Any use, modification, reproduction release,performance, display or disclosure of the Licensed Software andDocumentation by theU.S.Government shall be solely in accordance with the terms of this Agreement.

Symantec Corporation350 Ellis StreetMountain View, CA 94043

http://www.symantec.com

http://www.symantec.com

Technical SupportSymantec Technical Support maintains support centers globally. TechnicalSupport’s primary role is to respond to specific queries about product featuresand functionality. TheTechnical Support group also creates content for our onlineKnowledge Base. The Technical Support group works collaboratively with theother functional areas within Symantec to answer your questions in a timelyfashion. For example, theTechnical Support groupworkswithProductEngineeringand Symantec Security Response to provide alerting services and virus definitionupdates.

Symantec’s support offerings include the following:

■ A range of support options that give you the flexibility to select the rightamount of service for any size organization

■ Telephone and/or Web-based support that provides rapid response andup-to-the-minute information

■ Upgrade assurance that delivers software upgrades

■ Global support purchased on a regional business hours or 24 hours a day, 7days a week basis

■ Premium service offerings that include Account Management Services

For information about Symantec’s support offerings, you can visit our Web siteat the following URL:

www.symantec.com/business/support/index.jsp

All support services will be delivered in accordance with your support agreementand the then-current enterprise technical support policy.

Contacting Technical SupportCustomers with a current support agreement may access Technical Supportinformation at the following URL:

www.symantec.com/business/support/contact_techsupp_static.jsp

Before contacting Technical Support, make sure you have satisfied the systemrequirements that are listed in your product documentation. Also, you should beat the computer onwhich theproblemoccurred, in case it is necessary to replicatethe problem.

When you contact Technical Support, please have the following informationavailable:

■ Product release level

http://www.symantec.com/business/support/index.jsp

http://www.symantec.com/business/support/contact_techsupp_static.jsp

■ Hardware information

■ Available memory, disk space, and NIC information

■ Operating system

■ Version and patch level

■ Network topology

■ Router, gateway, and IP address information

■ Problem description:

■ Error messages and log files

■ Troubleshooting that was performed before contacting Symantec

■ Recent software configuration changes and network changes

Licensing and registrationIf yourSymantecproduct requires registrationor a licensekey, access our technicalsupport Web page at the following URL:

www.symantec.com/business/support/

Customer serviceCustomer service information is available at the following URL:

www.symantec.com/business/support/

Customer Service is available to assist with non-technical questions, such as thefollowing types of issues:

■ Questions regarding product licensing or serialization

■ Product registration updates, such as address or name changes

■ General product information (features, language availability, local dealers)

■ Latest information about product updates and upgrades

■ Information about upgrade assurance and support contracts

■ Information about the Symantec Buying Programs

■ Advice about Symantec's technical support options

■ Nontechnical presales questions

■ Issues that are related to CD-ROMs or manuals

http://www.symantec.com/business/support/

http://www.symantec.com/business/support/

Support agreement resourcesIf youwant to contact Symantec regarding an existing support agreement, pleasecontact the support agreement administration team for your region as follows:

[email protected] and Japan

[email protected], Middle-East, and Africa

[email protected] America and Latin America

DocumentationProduct guides are available on the media in PDF format. Make sure that you areusing the current version of the documentation. The document version appearson page 2 of each guide. The latest product documentation is available on theSymantec Web site.

https://sort.symantec.com/documents

Your feedback on product documentation is important to us. Send suggestionsfor improvements and reports on errors or omissions. Include the title anddocument version (located on the second page), and chapter and section titles ofthe text on which you are reporting. Send feedback to:

[email protected]

About Symantec ConnectSymantec Connect is the peer-to-peer technical community site for Symantec’senterprise customers. Participants can connect and share informationwith otherproduct users, including creating forum posts, articles, videos, downloads, blogsand suggesting ideas, as well as interact with Symantec product teams andTechnical Support. Content is rated by the community, and members receivereward points for their contributions.

http://www.symantec.com/connect/storage-management

mailto:[email protected]





http://www.symantec.com/connect/storage-management

Storage Foundation forOracle RAC Release Notes

This document includes the following topics:

■ About this document

■ Component product release notes

■ About Veritas Storage Foundation for Oracle RAC

■ About Symantec Operations Readiness Tools

■ Important release information

■ Changes introduced in SF Oracle RAC 6.0

■ No longer supported

■ System requirements

■ Fixed issues

■ Known issues

■ Software limitations

■ Documentation

About this documentThis document provides important information aboutVeritas Storage Foundationfor Oracle RAC (SF Oracle RAC) version 6.0 for AIX. Review this entire documentbefore you install or upgrade SF Oracle RAC.

The information in the Release Notes supersedes the information provided in theproduct documents for SF Oracle RAC.

This is Document version: 6.0.0 of the Veritas Storage Foundation for Oracle RACRelease Notes. Before you start, make sure that you are using the latest versionof this guide. The latest product documentation is available on the SymantecWebsite at:


Component product release notesProduct guides are available at the following location on the software media inPDF formats:

/product_name/docs

Symantec recommends copying the files to the /opt/VRTS/docsdirectory on yoursystem.

For information regarding software features, limitations, fixed issues, andknownissues of component products:

■ Veritas Cluster Server (VCS)See Veritas Cluster Server Release Notes (6.0).

■ Storage Foundation (SF)See Veritas Storage Foundation Release Notes (6.0).

■ Storage Foundation Cluster File System High Availability (6.0)See Veritas Storage Foundation Cluster File System High Availability ReleaseNotes (6.0).

About Veritas Storage Foundation for Oracle RACVeritas Storage Foundation™ for Oracle® RAC (SF Oracle RAC) leveragesproprietary storage management and high availability technologies to enablerobust, manageable, and scalable deployment of Oracle RAC on UNIX platforms.The solution uses Veritas Cluster File System technology that provides the dualadvantage of easy file systemmanagement aswell as the use of familiar operatingsystem tools and utilities in managing databases.

The solution stack comprises the Veritas Cluster Server (VCS), Veritas ClusterVolume Manager (CVM), Veritas Oracle Real Application Cluster Support(VRTSdbac), VeritasOracleDiskManager (VRTSodm), Veritas Cluster File System(CFS), and Veritas Storage Foundation, which includes the base Veritas VolumeManager (VxVM) and Veritas File System (VxFS).

Storage Foundation for Oracle RAC Release NotesComponent product release notes

8


Benefits of SF Oracle RACSF Oracle RAC provides the following benefits:

■ Support for file system-basedmanagement. SF Oracle RAC provides a genericclustered file system technology for storing and managing Oracle data filesas well as other application data.

■ Support for high-availability of cluster interconnects.For Oracle RAC 10g Release 2:The combination of LMX/LLT protocols and the PrivNIC/MultiPrivNIC agentsprovides maximum bandwidth as well as high availability of the clusterinterconnects, including switch redundancy.For Oracle RAC 11g Release 1/Oracle RAC 11g Release 2:ThePrivNIC/MultiPrivNIC agents providemaximumbandwidth aswell as highavailability of the cluster interconnects, including switch redundancy.See the following Technote regarding co-existence of PrivNIC/MultiPrivNICagents with Oracle RAC 11.2.0.2:http://www.symantec.com/business/support/index?page=content&id=TECH145261

■ Use of Cluster File System and Cluster Volume Manager for placement ofOracle Cluster Registry (OCR) and voting disks. These technologies providerobust shared block interfaces (for all supportedOracleRACversions) and rawinterfaces (for Oracle RAC 10g Release 2) for placement of OCR and votingdisks.

■ Support for a standardized approach toward application and databasemanagement. Administrators can apply their expertise ofVeritas technologiestoward administering SF Oracle RAC.

■ Increased availability and performance using Veritas Dynamic Multi-Pathing(DMP). DMP provides wide storage array support for protection from failuresand performance bottlenecks in the Host Bus Adapters (HBA), Storage AreaNetwork (SAN) switches, and storage arrays.

■ Easy administration andmonitoring of multiple SF Oracle RAC clusters usingVeritas Operations Manager.

■ VCS OEM plug-in provides a way to monitor SF Oracle RAC resources fromthe OEM console.For more information, see the Veritas Storage Foundation: Storage andAvailability Management for Oracle Databases guide.

■ Improved file system access times using Oracle Disk Manager (ODM).

■ Ability to configureOracleAutomatic StorageManagement (ASM) disk groupsover CVMvolumes to take advantage ofVeritasDynamicMulti-Pathing (DMP).

9Storage Foundation for Oracle RAC Release NotesAbout Veritas Storage Foundation for Oracle RAC

http://www.symantec.com/business/support/index?page=content&id=TECH145261

■ Enhanced scalability and availability with access to multiple Oracle RACinstances per database in a cluster.

■ Support for backup and recovery solutions using volume-level and filesystem-level snapshot technologies, StorageCheckpoints, andDatabaseStorageCheckpoints.For more information, see the Veritas Storage Foundation: Storage andAvailability Management for Oracle Databases guide.

■ Support for space optimization using periodic deduplication in a file systemto eliminate duplicate data without any continuous cost.For more information, see the Veritas Storage Foundation Administrator'sGuide.

■ Ability to fail over applicationswithminimumdowntimeusingVeritas ClusterServer (VCS) and Veritas Cluster File System (CFS).

■ Prevention of data corruption in split-brain scenarios with robust SCSI-3Persistent Group Reservation (PGR) based I/O fencing or Coordination PointServer-based I/O fencing. The preferred fencing feature also enables you tospecify how the fencing driver determines the surviving subcluster.

■ Support for sharing applicationdata, in addition toOracle database files, acrossnodes.

■ Support for policy-managed databases in Oracle RAC 11g Release 2.

■ Fast disaster recoverywithminimal downtime and interruption to users.Userscan transition froma local high availability site to awide-area disaster recoveryenvironment with primary and secondary sites. If a node fails, clients that areattached to the failed node can reconnect to a surviving node and resumeaccess to the shared database.

■ Verification of disaster recovery configuration using fire drill technologywithout affecting production systems.

■ Support for a wide range of hardware replication technologies as well asblock-level replication using VVR.

■ Support for campus clusters with the following capabilities:

■ Consistent detach with Site Awareness

■ Site aware reads with VxVM mirroring

■ Monitoring of Oracle resources

■ Protection against split-brain scenarios

Storage Foundation for Oracle RAC Release NotesAbout Veritas Storage Foundation for Oracle RAC

10

About Symantec Operations Readiness ToolsSymantec Operations Readiness Tools (SORT) is a Web site that automates andsimplifies some of the most time-consuming administrative tasks. SORT helpsyou manage your datacenter more efficiently and get the most out of yourSymantec products.

SORT can help you do the following:

■ List product installation andupgrade requirements, includingoperating system versions, memory, disk space, andarchitecture.

■ Analyze systems to determine if they are ready to install orupgrade Symantec products.

■ Download the latest patches, documentation, and highavailability agents from a central repository.

■ Access up-to-date compatibility lists for hardware, software,databases, and operating systems.

Prepare for your nextinstallation or upgrade

■ Get automatic email notifications about changes to patches,array-specific modules (ASLs/APMs/DDIs/DDLs), and highavailability agents from a central repository.

■ Identify and mitigate system and environmental risks.

■ Display descriptions and solutions for hundreds of Symantecerror codes.

Manage risks

■ Find and download patches based on product version andplatform.

■ List installed Symantec products and license keys.

■ Tune and optimize your environment.

Improve efficiency

Note: Certain features of SORT are not available for all products. Access to SORTis available at no extra cost.

To access SORT, go to:

https://sort.symantec.com

Important release information■ For important updates regarding this release, review the Late-Breaking News

TechNote on the Symantec Technical Support website:http://www.symantec.com/docs/TECH164885

11Storage Foundation for Oracle RAC Release NotesAbout Symantec Operations Readiness Tools

https://sort.symantec.com

https://sort.symantec.com/

http://www.symantec.com/docs/TECH164885

■ For the latest patches available for this release, go to:http://sort.symantec.com/

■ Thehardwarecompatibility list contains informationabout supportedhardwareand is updated regularly. For the latest information on supported hardwarevisit the following URL:http://www.symantec.com/docs/TECH170013Before installing or upgrading Storage Foundation and High AvailabilitySolutions products, review the current compatibility list to confirm thecompatibility of your hardware and software.

Changes introduced in SF Oracle RAC 6.0This section lists the changes in SF Oracle RAC 6.0.

Changes in SF Oracle RACThis section describes the changes to SF Oracle RAC in this release.

Support for policy-managed database environmentsSF Oracle RAC now supports policy-managed database environments in OracleRAC 11g Release 2. In policy-managed database environments, the administratorspecifies the server pool on which the database instances run. Oracle GridInfrastructure determines the server on which the database instances run. Youcan use the VCS agent for Oracle to start, stop, and monitor policy-manageddatabases.

Support for Web-based installation and configurationThis release extends theWeb-based installer capabilities to support the followingtasks:

■ SF Oracle RAC installation and configuration checks

■ I/O fencing configuration

■ Rolling upgrade

■ Preparation, installation, and post-installation tasks of Oracle RAC

■ Adding nodes to a cluster

Web-based installer capabilities for the following tasks already exist and wereintroduced in version 5.1SP1:

■ Installation and configuration of SF Oracle RAC

Storage Foundation for Oracle RAC Release NotesChanges introduced in SF Oracle RAC 6.0

12

http://sort.symantec.com/


■ Uninstallation of SF Oracle RAC

SF Oracle RAC installer enhancementsThis release introduces the following enhancements in theSFOracleRAC installer:

■ Mirroring is now optional for the creation of OCR and voting disk storage.

■ You can now choose to create either single or separate file systems for OCRand voting disk.

■ The application resource for cssd can now be named as preferrable. The namecssd is no longer embedded into the monitor script.

■ The SF Oracle RAC Installation and Configuration Checks option nowincludes a new check that verifies whether or not the user nobody exists onall nodes in the cluster.

CRSResource agent support for Oracle RAC 11g Release 2The CRSResource agent is now supported for Oracle RAC 11g Release 2.

The installer can now detect duplicate VCS cluster IDs and canautomatically generate cluster IDs

The installer can now detect duplicate VCS cluster IDs and prompt you to selectan unused one. It can also generate an unused ID during installation.

The installer can check product versions and hotfixesYou can check the existing product versions using the installer command withthe -version option before or after you install. After you have installed thecurrent version of the product, you can use the showversion script in the/opt/VRTS/install directory to find version information.

You can discover the following information with these commands:

■ The installed version of all released Strorage Foundation andHighAvailabilitySuite of products

■ The missing required filesets or patches as applicable for platform

■ The available updates (including patches or hotfixes) from SORT for theinstalled products

Depending on the product, the script can identify versions from 4.0 onward.

13Storage Foundation for Oracle RAC Release NotesChanges introduced in SF Oracle RAC 6.0

Using the installer's postcheck optionYou can use the installer's postcheck option to diagnose installation-relatedproblems and to provide troubleshooting information.

Rolling upgrade improvementsThe rolling upgrade procedure has been streamlined and simplified.

Allow Response files to change tuning parametersYou can set non-default product and system tunable parameters using a tunablestemplate file. With the file, you can set tunables such as the I/O policy or togglenative multi-pathing during or after the installation procedure.

See the Installation Guide for more information.

Changes related to SFDB toolsThe following sections describe the changes related to Storage Foundation forDatabases (SFDB) tools in SF Oracle RAC 6.0.

Support for space-optimized snapshots for database cloningYoucanuseStorageFoundation forDatabases (SFDB) tools to take space-optimizedsnapshots of yourOracle database and then create database clones by using thosesnapshots. SFDB tools use the underlying features of Storage Foundation for thisoperation.

See Veritas Storage Foundation: Storage And Availability Management for OracleDatabases.

Enhancements to Cached ODM Advisor (dbed_codm_adm)You can use the Cached ODM Advisor command dbed_codm_adm to generate avariety of reports that help youdeterminewhichdata files are suitable for enablingCachedODM. The reports generated by CachedODMAdvisor are enhanced to usethe historical data from Oracle Automatic Workload Repository (AWR).



14

Support for space-optimized snapshots onDRsite for databasecloningYou can use Storage Foundation for Databases (SFDB) tools in a replicatedenvironment to take space-optimized snapshots on a disaster recovery (DR) site.This functionality lets you create clones of your Oracle database on the DR sitein a setup where the database on the primary site is being replicated on the DRsite.


Single CLI for different point-in-time copy operationsYou can use the new SFDB command vxsfadm to perform various point-in-timecopy operations on yourOracle database. vxsfadmprovides the following benefits:

■ Uniform command line for multiple operations

■ Use case based functionality

■ Enhanced error handling


Support for file-level snapshots for database cloningYou can use Storage Foundation for Databases (SFDB) tools to take file-levelsnapshots of your Oracle database and then create one or more clones based onthose snapshots. SFDB tools use the underlying features of Storage Foundationfor this operation.


Enhanced authentication supportThe authentication support for Storage Foundation for Databases (SFDB) tools isenhanced in this release. You can use the sfae_auth_op to set up and configureauthentication for SFDB tools.



SmartTier integration with OEMYou can now view the following SmartTier related information in the OracleEnterprise Manager (OEM) reports:

■ Storage allocation and free space in each tier

■ Space occupied by a data file in each tierThis is usefulwhen a part of a data file ismoved from tier to tierwhen databaseobjects such as table or index are moved.

Packaging updatesThe following lists the package changes in this release.

■ New VRTSsfcpi60 fileset for product installer scripts

The VRTSsfcpi60 fileset is introduced in this release. The VRTSsfcpi60 filesetcontains the installer scripts and libraries that the installer uses to install,configure and upgrade Veritas products.

For more information, see the Installation Guide.

Changes to SF Oracle RAC clusters in secure modeIn this release, the installation and configuration experience of secure cluster isconsiderably simplified. You can easily convert the cluster into secure clusterwith this simplified secure cluster configuration model.

Thenewarchitecture is based on embeddedVxAT,where the security componentsare installed as a part of the SF Oracle RAC package. The root broker is no longera single-point-of-failure in the new architecture. There is no dependency on aseparate VRTSat package. Non-root users who are already logged on SF OracleRAC hosts are now not prompted for password. Additionally, a cluster-level userfeature is introduced to simplify user administration in secure clusters.

See the Installation Guide and Administrator's Guide for more information.

Changes to LLTThis release includes the following new features and changes to LLT:

■ LLT now supports VLAN tagging (IEEE 802.1Q).

■ The lltconfig command includes the following new options:

■ -NYou can use this option to list all the used cluster IDs.

■ -M


16

You can use this option to display the currently loaded LLTmodule versioninformation.

See the lltconfig manual page for more information.

■ Link utilization statistics are enhanced that help in the root cause analysis ofperformance related issues.

■ Periodic flushing of ARP cache is disabled.

See theVeritas Storage Foundation for Oracle RAC Installation and ConfigurationGuide and the Veritas Storage Foundation for Oracle RAC Administrator's Guidefor more details.

Changes to GABThis section covers the new features and changes related to GAB in this release.

Better GAB and I/O fencing integration to ensure applicationavailabilityIn the event of a split-brain situation before VxFEN module implements thedecision, sometimes GAB proceeds with attempting to resolve the join after thesplit-brain. GAB removes all but one joining subcluster. This behavior can causethe entire cluster to shut down. To avoid this scenario, GAB now gives priority tothe fencing module.

With the GAB and I/O fencing integration in this release, if the I/O fencingmodule’s decision is still pending before GAB initiates a join of the subcluster,GAB delays the iofence message. GAB wait depends on the value of the VxFENtunable parameterpanic_timeout_offst based onwhichVxFENcomputes the delayvalue and passes to GAB.

See the Veritas Storage Foundation for Oracle RAC Administrator's Guide formore details.

GAB can now recognize clientswith names in addition to portsWhen kernel clients initialize GAB API, they can now define a client name string.GAB now adds a client name which enables GAB to track the client even beforeGAB port is registered. GAB also passes the client name information to LLTwhenregistering the LLT port. The lltstat -p command also displays the GAB clientnames when providing the status details of the ports in use.

This feature is applicable only to GAB kernel clients, and not applicable foruser-land GAB clients such as HAD.


The gabconfig command has new -C optionThe -C option of the gabconfig command lists the names of the GAB clients thathave registered with GAB. The -C option when used with -a option lists the clientnames along with the port membership details.

Changes to I/O fencingThis section covers the new features and changes related to I/O fencing in thisrelease.

Support for racer node re-election during I/O fencing raceAt the time of a network partition, the VxFEN module elects the lowest node ineach sub-cluster as the racer node to race for the coordination points on behalfof the sub-cluster. The other spectator nodes wait on the racer node to do thefencing.

In the previous releases, the I/O fencing racewas entirely dependent on the singleracer node as follows:

■ If the racer node is not able to reach a majority of coordination points, thenthe VxFEN module on the racer node sends a LOST_RACE message and allnodes in the subcluster also panicwhen they receive the LOST_RACEmessage.

■ If the racer node panics during the arbitration, then the spectator nodes inthe sub-cluster assume that the racer node lost the race and the spectatornodes also panic.

With thenew racer node re-election feature, theVxFENmodule re-elects thenodewith the next lowest node id in the sub-cluster as the racer node. This featureoptimizes the chances for the sub-cluster to continuewith the race for coordinationpoints.

See theVeritas StorageFoundation forOracleRACAdministrator'sGuide formoredetails.

Support for multiple virtual IP addresses in CP serversYou can now configure multiple network paths (virtual IP addresses) to access aCP server. CP server listens on multiple virtual IP addresses. If a network pathfails, CP server does not require a restart and continues to listen on one of theother available virtual IP addresses.



18

Support for Quorum agent in CP serversWith the support for multiple virtual IP addresses, you can now use the Quorumagent to configure CP server service group failover policies. You can specify theminimum number of IP resources that must be online for the Quorum resourceto remain online.


With fencing enabled, GAB can now automatically seed thecluster when some cluster nodes are unavailableIn the earlier releases, if some of the nodes are not up and running in a cluster,then GAB port does not come up to avoid any risks of preexisting split-brain. Insuch cases, you can manually seed GAB using the command gabconfig -x tobring the GAB port up. However, if you have enabled I/O fencing in the cluster,then I/O fencing can handle any preexisting split-brain in the cluster.

In this release, I/O fencing has extended this functionality to be able toautomatically seed GAB as follows:

■ If a number of nodes in a cluster are not up, GAB port (port a) still comes upin all the member-nodes in the cluster.

■ If the coordination points do not have keys from any non-member nodes, I/Ofencing (GAB port b) also comes up.

This new functionality is disabled by default. You must manually enable thisautomatic seeding feature of GAB in clusters where I/O fencing is configured inenabled mode.


You can still use the gabconfig -x command to manually seed the cluster.

Graceful shutdown of a node no longer triggers I/O fencingrace condition on peer nodesIn the earlier releases, a gracefully leaving node clears its I/O fencing keys fromcoordination points. But the remaining sub-cluster races against the gracefullyleavingnode to remove its registrations from the data disks. During this operation,if the sub-cluster loses access to the coordination points, the entire cluster maypanic if the racer loses the race for coordination points.


In this release, this behavior has changed. When a node leaves gracefully, theCVM or other clients on that node are stopped before the VxFEN module isunconfigured. Hence, data disks are already clear of its keys. The remainingsub-cluster tries to clear the gracefully leaving node’s keys from the coordinationpoints but does not panic if it is not able to clear the keys.

Installer support to migrate between fencing configurationsin an online clusterYou can now use the installer to migrate between disk-based and server-basedfencing configurations. You can also replace the coordination points for any I/Ofencing configuration in an online cluster using the same installer option. Theinstaller uses the vxfenswap script internally.

You can also use response files to perform these I/O fencing reconfigurationoperations.


Availability of shared disk group configuration copiesIf the Cluster VolumeManager (CVM)master node loses access to a configurationcopy, CVM redirects the read or write requests over the network to another nodethat has connectivity to the configuration copy. This behavior ensures that thedisk group stays available.

In previous releases, CVM handled disconnectivity according to the disk groupfailure policy (dgfail_policy). This behavior still applies if the disk group versionis less than 170. The dgfail_policy is not applicable to disk groups with a versionof 170 or later.

Enhancements to CVM detach policiesIn this release, the following changes have been made to the detach policies:

■ A failure is considered global only if it affects all nodes in the cluster. Otherwise,a failure is treated as a local failure. Previously, any failure that affectedmorethan one node was considered to be global.

■ When the global detach policy is set, local failure of all plexes at the same timedoesnot trigger plex detach. In this case, the volume remains enabled and I/Osfail on the node.

■ When a node loses local connectivity to a disk, the disk is put in the lfailedstate.


20

Enhancements to master node selection for failoverIf the Cluster Volume Manager (CVM) master node leaves the cluster, CVM failsover the master role to another node in the cluster. In this release, CVM selectsthe node for failover based on the node's connectivity to the disks in the diskgroup. This behavior is an enhancement over previous releases of CVM.

During regular operations, CVM dynamically assigns an offset preference valueto each node. The preference assignment is automatic, and generally does notrequire any intervention from the administrator.

If you need greater control over themaster selection, you can also set customizedpreference values.

When a master failover occurs, CVM uses the custom node preferences togetherwith the offset preference values to select the new master node.

Node join with DGDISABLED disk groupsIn this release, a node can join the cluster even if there is a shared disk group thatis in the DGDISABLED state. In previous releases, the node join would fail.

Support for product upgrades using the Network Installation ManagerAlternate Disk Migration utility on AIX

You can now use the Network Installation Manager Alternate Disk Migration(nimadm) utility to upgrade the operating system and the product.

See the Installation Guide for more information.

Licensing changes in the SFHA Solutions 6.0 releaseStorage Foundation and High Availability Solutions 6.0 introduces the followinglicensing changes:

■ The Cluster File System license is deprecated. CFS customers are entitled tothe Storage Foundation Cluster File System High Availability (SFCFS HA)functionality.

■ TheVVROption is renamed asVeritas ReplicatorOption. This option includesVVR (volume-based replication) and the new file-based replication solution.

■ The VVR Enterprise license is deprecated; you can use Storage FoundationEnterprise and add Veritas Replicator Option to get this functionality. VVREnterprise customers are entitled to Storage Foundation Enterprise withReplicator Option.


■ The VCS license enables full cluster functionality as well as the limitedstart/stop functionality.

■ Storage Foundation Enterprise CFS for Oracle RAC (Linux/x64) customers areentitled to Storage Foundation Enterprise for Oracle RAC (Linux/x64.)

The following functionality is included in the Standard and Enterprise licenses:

■ The Compression feature is available with the Standard license.

■ The SmartTier feature is now available with the Standard license.

■ The Deduplication feature is available with the Enterprise license.

The following products are included in this release:

■ Dynamic Multi-Pathing

■ VirtualStore

■ Storage Foundation Basic

■ Storage Foundation Standard

■ Storage Foundation Enterprise

■ Veritas Cluster Server

■ Veritas Cluster Server HA/DR

■ Storage Foundation Standard HA: Storage Foundation Standard plus VeritasCluster Server

■ StorageFoundationEnterpriseHA: StorageFoundationEnterprise plusVeritasCluster Server

■ Storage Foundation Enterprise HA/DR

■ Storage Foundation Enterprise Cluster File System HA

■ Storage Foundation Enterprise Cluster File System HA/DR

■ Storage Foundation Enterprise for Oracle RAC

■ Storage Foundation Enterprise HA/DR for Oracle RAC

■ Storage Foundation Enterprise for Sybase ASE CE

■ Storage Foundation Enterprise HA/DR for Sybase CE

HA: High Availability

HA/DR: High Availability and Disaster Recovery

Veritas Replicator Option can be added to all Storage Foundation and HighAvailability products, except Dynamic Multi-Pathing and Veritas Cluster Server.


22

Note that products, features, and options may differ by operating system andplatform. Please see the product documentation for information on supportedplatforms.

Enhancements to collecting a VxExplorer troubleshooting archiveThe Symantec Operations Readiness Tools (SORT) data collector containsfunctionality to collect and submit aVxExplorer archive. You can send this archiveto Symantec Technical Support for problem diagnosis and troubleshooting.VxExplorer does not collect customer data.

The legacy VxExplorer script now works differently. When you run the script, itlaunches the SORTdata collector on the specified local hostwith the -vxexploreroption.

To learn more about using the data collector to collect a VxExplorer archive, see:

www.symantec.com/docs/HOWTO32575

Changes related to product documentationThe Storage Foundation and High Availability Solutions 6.0 release includes thefollowing changes to the product documentation.

Table 1-1 lists the documents introduced in this release.

Table 1-1 New documents

NotesNew documents

Installation and upgrade information forStorage Veritas Foundation.

Veritas Storage Foundation InstallationGuide

Administration information for VeritasStorage Foundation.

Veritas Storage Foundation Administrator'sGuide

Release-specific information for VeritasStorage Foundation and High Availabilityusers.

Veritas Storage Foundation and HighAvailability Release Notes

Solutions and use cases for Veritas StorageFoundation andHighAvailability Solutions.

Veritas Storage Foundation and HighAvailability Solutions Solutions Guide

Troubleshooting information for VeritasStorage Foundation and High AvailabilitySolutions.

Veritas Storage Foundation and HighAvailability SolutionsTroubleshootingGuide

Table 1-2 lists the documents that are deprecated in this release.


http://www.symantec.com/docs/HOWTO32575

Table 1-2 Deprecated documents

NotesDeprecated documents

Content now appears in the Veritas StorageFoundationAdministrator'sGuide and in theVeritas Storage Foundation Cluster FileSystem High Availability Administrator'sGuide.

Veritas File System Administrator's Guide

Content now appears in the Veritas StorageFoundationAdministrator'sGuide and in theVeritas Storage Foundation Cluster FileSystem High Availability Administrator'sGuide.

Veritas VolumeManager Administrator'sGuide

Content now appears in the Veritas StorageFoundation and High Availability SolutionsSolutions Guide.

Veritas Storage Foundation AdvancedFeatures Administrator's Guide

Content now appears in the Veritas StorageFoundation and High Availability SolutionsTroubleshooting Guide.

Veritas VolumeManager TroubleshootingGuide

Content now appears in the Veritas ClusterServer Bundled Agents Reference Guide.

Veritas Cluster Server Agents for VeritasVolume Replicator Configuration Guide

Content now appears in the Veritas StorageFoundation and High Availability SolutionsReplication Administrator's Guide.

Veritas Volume Replicator Planning andTuning Guide

Content now appears in the Veritas StorageFoundation and High Availability SolutionsReplication Administrator's Guide.

Veritas Volume Replicator Advisor User'sGuide

Table 1-3 lists documents that are no longer bundled with the binaries. Thesedocuments are now available online.

Table 1-3 Online documents

Document

Veritas Cluster Server Agent Developer's Guide

Veritas File System Programmer's Reference Guide


24

No longer supportedThis section lists software versions and features that are no longer supported.Symantec advises customers to minimize the use of these features.

SF Oracle RAC does not support the following:

■ Several documents are deprecated in this release.See “Changes related to product documentation” on page 23.

■ Oracle RAC 11g Release 1 Clusterware

■ Dissimilar version upgrades of SF Oracle RAC componentsFor example, if youhaveSFOracleRAC4.1 installedwithOracleRAC9iRelease2, you can not upgrade only VCS to version 6.0.

■ Option hawizard -rac for configuring service groups

■ 32-bit Oracle architectures

■ Oracle RAC 9i, Oracle RAC 10g Release 1

■ ASMInst agentThe ASMInst agent is no longer supported in SF Oracle RAC environments.The ASM instances are managed by Oracle Clusterware.

■ Use of crossover cablesOracle does not support the use of crossover cables for cluster interconnectsdue to the possibility of data corruption and other software limitations.

Note: Crossover cables are however known to function without any issues inSFOracle RAC.While the SFOracle RACTechnical support teammay continueto provide support on related issues for existing deployments, this supportmaybe constrained in some respects as it is no longer a supported configurationby Oracle.

The use of crossover cables is discouraged for new deployments.

■ Bunker replication is not supported in a Cluster Volume Manager (CVM)environment.

Veritas Storage Foundation for Databases (SFDB) tools features whichare no longer supported

The following Storage Foundation for Databases (SFDB) tools features are notsupported in this release:

■ FlashSnap reverse resync

25Storage Foundation for Oracle RAC Release NotesNo longer supported

■ Checkpoint policy and Checkpoint quotas

■ Interactive modes in clone and rollback

System requirementsThis section describes the system requirements for this release.

Important preinstallation informationBefore you install SF Oracle RAC, make sure you have reviewed the followinginformation:

■ Hardware compatibility list for information about supported hardware:http://www.symantec.com/docs/TECH170013

■ Latest information on support for Oracle database versions:http://www.symantec.com/docs/TECH44807

■ Oracle documentation for additional requirements pertaining to your versionof Oracle.

Hardware requirementsDepending on the type of setup planned, make sure you meet the necessaryhardware requirements.

See Table 1-4 on page 26.For basic clusters

See Table 1-5 on page 28.For campus clusters

Table 1-4 Hardware requirements for basic clusters

DescriptionItem

Two to sixteen systems with two or more CPUs.

For details on the additional requirements for Oracle, see the Oracledocumentation.

SF Oracle RACsystems

A DVD drive on one of the nodes in the cluster.DVD drive

SF Oracle RAC requires that all shared storage disks support SCSI-3Persistent Reservations (PR).

Note: The coordinator disk does not store data, so configure the diskas the smallest possible LUN on a disk array to avoid wasting space.The minimum size required for a coordinator disk is 128 MB.

Disks

Storage Foundation for Oracle RAC Release NotesSystem requirements

26



Table 1-4 Hardware requirements for basic clusters (continued)

DescriptionItem

You can evaluate your systems for available disk space by runningthe product installation program. Navigate to the product directoryon the product disc and run the following command:

# ./installsfrac -precheck node_name

You can also use the Veritas Web-based installation program todetermine the available disk space.

For details on the additional space that is required for Oracle, see theOracle documentation.

Disk space

Each SF Oracle RAC system requires at least 2 GB.

For Oracle RAC requirements, see the Oracle Metalink document:169706.1

RAM

See the Oracle Metalink document: 169706.1Swap space

Two or more private links and one public link.

Linksmust be 100BaseT or gigabit Ethernet directly linking eachnodeto the other node to form a private network that handles directinter-system communication. These links must be of the same type;you cannot mix 100BaseT and gigabit.

Symantec recommends gigabit Ethernet using enterprise-classswitches for the private links.

Oracle requires that all nodes use the IP addresses from the samesubnet.

You can also configure aggregated interfaces.

Network

At least one additional SCSI or Fibre Channel Host Bus Adapter persystem for shared data disks.

Fiber Channel orSCSI host busadapters

Table 1-5 lists the hardware requirements for campus clusters in addition to thebasic cluster requirements.

27Storage Foundation for Oracle RAC Release NotesSystem requirements

Table 1-5 Hardware requirements for campus clusters

DescriptionItem

■ The storage switch (to which each host on a site connects) must haveaccess to storage arrays at all the sites.

■ Volumesmust bemirroredwith storage allocated fromat least two sites.

■ DWDM links are recommended between sites for storage links.

DWDM works at the physical layer and requires multiplexer andde-multiplexer devices.

■ The storage and networks must have redundant-loop access betweeneach node and each storage array to prevent the links from becoming asingle point of failure.

Storage

■ Oracle requires that all nodes use the IP addresses from the same subnet.

■ Symantec recommends a common cross-site physical infrastructure forstorage and LLT private networks.

Network

I/O fencing requires placement of a third coordinator point at a third site.The DWDM can be extended to the third site or the iSCSI LUN at the thirdsite can be used as the third coordination point. Alternatively CoordinationPoint Server can be deployed at the third remote site as an arbitration point.

I/O fencing

Supported AIX operating systemsThis section lists the supported operating systems for this release of Veritasproducts.

Table 1-6 shows the supported operating systems for this release.

Table 1-6 Supported operating systems

ChipsetsLevelsOperatingsystems

Any chipset that the operating system supportsTL0 or laterAIX 7.1

Power 7, Power 6, or earlierTL5 or laterAIX 6.1

For Storage Foundation for Oracle RAC, all nodes in the cluster must have thesame operating system version and update level.

AIX 7.1 support for virtual processorsVeritas Storage Foundation forOracle RAC supports up to 1024 virtual processorson AIX 7.1.

Storage Foundation for Oracle RAC Release NotesSystem requirements

28

Supported database software

Note: SF Oracle RAC supports only 64-bit Oracle.

The following database versions are supported:

■ Oracle RAC 10g Release 2

■ Oracle RAC 11g Release 1 Database

Note: Oracle 11g Release 1 Clusterware is not supported.

■ Oracle RAC 11g Release 2

For the latest informationon supportedOracle database versions, see the followingTechnical Support TechNote:


Support for minor database versions is also documented in the afore-mentionedTechnical Support TechNote.

Additionally, see the Oracle documentation for information on patches that maybe required by Oracle for each release.

Supported replication technologies for global clustersSF Oracle RAC supports the following hardware-based replication andsoftware-based replication technologies for global cluster configurations:

■ EMC SRDF

■ Hitachi TrueCopy

■ IBM Metro Mirror

■ IBM SAN Volume Controller (SVC)

■ EMC MirrorView

Hardware-based replication

■ Veritas Volume Replicator

■ Oracle Data Guard

Software-based replication

Fixed issuesThis section covers the incidents that are fixed in this release.

29Storage Foundation for Oracle RAC Release NotesFixed issues


See the corresponding Release Notes for a complete list of fixed incidents relatedto that product.

See “Documentation” on page 57.

Issues fixed in SF Oracle RAC 6.0Table 1-7 lists the issues fixed in SF Oracle RAC 6.0.

Table 1-7 Issues fixed in SF Oracle RAC 6.0

DescriptionIncidentnumber

TheCSSDagent embedscssd as the resource name in the cssdmonitor script.

This causes the agent to log the following warning message in the engine logfile if the resource name is not set to cssd:

VCS WARNING V-16-1-10260 Resource does not exist: cssd

2386570

The PrivNIC/MultiPrivNIC agents fail to remove the original IP address aftera failover or failback operation.

2336374

Incorrect ownership assigned to the parent directory ofORACLE_BASE causesOracle Clusterware/Grid Infrastructure installations to fail.

When you use the SF Oracle RAC script-based installation program to installOracle Clusterware/Grid Infrastructure, the ownership of the parent directoryof ORACLE_BASE/GRID_BASE that is created by the installation program isincorrectly set to root. This causes the Oracle Universal Installer to displayerrors when it creates the oraInventory directory.

2212272

Failure to update the DBHome attribute while configuring the VIP ResTypeattribute logs an error message in the engine log file.

2491788

Whenanodepanics on aheavily loaded cluster, theOracle instance onanothernode crashes.

2554974

Failure to set the MTU (Maximum Transmission Unit) size in LLT over UDPenvironments causes issues with the PrivNIC/MultiPrivNIC agents.

If the MTU size field is not set explicitly when you configure thePrivNIC/MultiPrivNIC agents in an LLT over UDP environment, the agentsmay fail to plumb the private IP addresses during their operations or mayconfigure incorrect MTU size on the LLT interfaces.

2555319

Storage Foundation for Oracle RAC Release NotesFixed issues

30

Veritas Storage Foundation for Oracle RAC: Issues fixed in 5.1SP1 RP2Table 1-8 describes the incidents that are fixed in Veritas Storage Foundation forOracle RAC in 5.1 SP1 RP2.

Table 1-8 Veritas Storage Foundation for Oracle RAC fixed issues

DescriptionFixedissues

Oracle instance crashed; failure occurred at: vcsipc_dosnd2374977

memory leak in vcsmm_set_cluster_proto2390892

The cssd agent explicitly uses hard-coded string "cssd" as resource name.2429449

CRSResource agent support for 11gR22374970

Storage Foundation for Databases (SFDB) tools fixed issuesTable 1-9 describes the Veritas Storage Foundation for Databases (SFDB) toolsissues fixed in this release.

Table 1-9 SFDB tools fixed issues

DescriptionFixed issues

In amultiple disk group environment, if the snapshot operation failsthen dbed_vmsnap fails to reattach all the volumes.

1840672

If the database fails over during FlashSnap operations, various errormesages appear.

1469310

Storage Foundation for Databases (SFDB) tools: Issues fixedin 5.1 SP1 RP1There are no SFDB fixed issues in 5.1 SP1 RP1.

Storage Foundation for Databases (SFDB) tools: Issues fixedin 5.1 SP1 RP2Table 1-10 describes the incidents that are fixed in Storage Foundation forDatabases (SFDB) tools in 5.1 SP1 RP2.


Table 1-10 Veritas Storage Foundation for Databases (SFDB) tools fixed issues

DescriptionFixedissues

dbed_update does not work on AIX 7.1 Power 72429359

vxdbed looping doing read/write to IDLE sockets2509867

LLT, GAB, and I/O fencing fixed issuesTable 1-11 lists the fixed issues for LLT, GAB, and I/O fencing.

Table 1-11 LLT, GAB, and I/O fencing fixed issues

DescriptionIncident

[Fencing]Uninstallation ofVRTSvxfenpackage shows errormessagewhenthe GAB port b does not show up even when VxFEN is being used in caseGAB is waiting for seeding.

2563177

[GAB] gabconfig ioctl behaviour changed to return EALREADY if GAB isalready configured.

2515932

[GAB] System crashed when uninstalling the GAB package.2509400

[Fencing] vxfend does not terminate if you run the vxfenswap commandto change the fencing mode from ‘scsi3’ to ‘customized’, and chooses torollback when vxfenswap prompts for confirmation.

2495020

[LLT] Improve LLT over UDP performance.2481098

[LLT] Reduce lltd CPU consumption by reducing the wakeup calls.2442402

[Fencing] Fails to run the vxfenswap command to the same diskgroupwhen the disk policy changed.

2437022

[Fencing] vxfend does not terminate when you run the vxfenswapcommand to migrate from the customized mode to the scsi3 mode.

2426664

[GAB] Add a check inGAB forMAXmessage size of 64KB before enqueuingthe message.

2411652

[Fencing] Fencing configuration fails and vxfenadm prints same serialnumber for all LUNs which have more than 96 bytes of SCSI Inquiry datain page 0x83.

2386325

Storage Foundation for Oracle RAC Release NotesFixed issues

32

Table 1-11 LLT, GAB, and I/O fencing fixed issues (continued)

DescriptionIncident

[Fencing] Once vxfenconfig -c with a particularmode (say customized) hasreturned EFAULT ("1036 Unable to configure..."), all subsequent runs ofvxfenconfig -c with a different mode (say scsi3) fail with error EBADMSG("1050 Mismatched modes...").

2369742

[Fencing] The vxfenswap utility fails to accurately check for the exit statusof the vxfenconfig commands run on the other nodes in the background.This may lead to the vxfenswap utility appearing indefinitely hung if thevxfenconfig process does not succeed for any reason.

2351011

[Fencing] Fencing shutdown script does not retry stopping the fencingmodule if fencing fails to unconfigure because of clients being registered.

2337916

[Fencing] Fencing details are printed in the engine logs every five minutesif fencing is running and the CoordPoint resource is configured.

2311361

[Fencing] VxFEN tunable resets when node comes up after panic.2254947

[Fencing] Fencing fails to start if any of the coordination points isunavailable at the startup time.

2253321

[Fencing] Provide options to force the fencing library to obtain serialnumbers using standard inquiry or extended inquiry using a variety of IDtypes.

2252470

[LLT] Disable the fastpath in LLT and make it optional.2242747

[VxCPS] The cpsadm command fails if LLT is not installed or configuredon a single-node cluster which hosts the CP server.

2218448

[VxCPS] Configuring fencing is successful with three disks even whensingle_cp=1 and the formatting of warning messages aer required invxfend_A.log.

2209664

[Fencing] Failed to configure fencing on a 64-node cluster, fencing comesup only on first 33 nodes.

2203070

[Fencing] Fencingpanics in fp_close() after a considerable timeafter storageis unavailable.

2176097

[Fencing] Preferred fencing does not work as expected for large clustersin certain cases if you have configured system-based or group-basedpreferred fencing policy.

2161816

[VxCPS] Server-based I/O fencing fails to start after configuration onnodeswith different locale settings.

2112742


Table 1-11 LLT, GAB, and I/O fencing fixed issues (continued)

DescriptionIncident

[Fencing] There is failure message even the migration from server-basedto disk-based using vxfenswap succeeded.

2100896

[VxCPS] Coordination point server (CP server) binds only to a single virtualIP and listens on the same. Application clusters cannot access the CP serverif it fails to establish connection to this virtual IP. Therefore, if theconnection fails because of the subnet in which the virtual IP of the CPserver exists, you cannot access the CP server even if there is anothersubnet through which the client can connect to the CP server over adifferent IP.

2085941

[VxCPS]Whenyou reconfigure anapplication cluster that uses server-basedI/O fencing (customized fencing mode), the installer does not remove theapplication cluster information from the CP servers before thereconfiguration. As a result, if you reconfigure the application cluster andchoose to configure I/O fencing in customized mode using the same CPservers, then reconfiguration of server-based fencing for the applicationcluster fails.

2076240

[Fencing] The agent XML files are missing for CP server agent.1973713

[Fencing] Cannot change the VxFEN tunable parameters due to a softwarelimitation.

1863916

Known issuesThis section covers the known issues in this release.

For Oracle RAC issues:

See “Oracle RAC issues” on page 34.

For SF Oracle RAC issues:

See “SF Oracle RAC issues” on page 36.

See the corresponding Release Notes for a complete list of known issues relatedto that product.


Oracle RAC issuesThis section lists the known issues in Oracle RAC.

Storage Foundation for Oracle RAC Release NotesKnown issues

34

Oracle Grid Infrastructure installation may fail with internaldriver errorThe Oracle Grid Infrastructure installation may fail with the following error:

[INS-20702] Unexpected Internal driver error

Workaround:

Perform one of the following steps depending on the type of installer you use forthe installation:

■ Script-based installerExport the OUI_ARGS environment variable, before you run the SF Oracle RACinstallation program:

export OUI_ARGS=-ignoreInternalDriverError

For more information, see the Oracle Metalink document: 970166.1

■ Web-based installerWhenyou run theWeb-based installer, in theEntertheargumentstobepassedtotheOracleinstaller text box, enter the value -ignoreInternalDriverError.

For more information, see the Veritas Storage Foundation for Oracle RACInstallation and Configuration Guide.

During installationor systemstartup,OracleGrid Infrastructuremay fail to startAfter successful installation of Oracle RAC 11g Release 2 Grid Infrastructure,while executing the root.sh script, ohasd may fail to start. Similarly, duringsystemstartup, OracleGrid Infrastructuremay fail to start though theVCS enginelogs may indicate that the cssd resource started Oracle Grid Infrastructuresuccessfully.

Forpossible causesandworkarounds, see theOracleMetalinkdocument: 1069182.1

Oracle VIPConfigurationAssistant failswith an errormessage(1182220)During Oracle RAC 10g Release 2 installation, the VIP Configuration Assistantmay fail with the following error message:

The given interface(s), "en0" is not public.

Public interfaces should be used to configure virtual IPs.

This message appears only when the VIP is not from the regular public IP range(for example, 200.).

35Storage Foundation for Oracle RAC Release NotesKnown issues

Workaround: Invoke the vipca utility manually as the superuser.

# export DISPLAY=nebula:0.0

# $CRS_HOME/bin/vipca

Oracle Cluster Verification utility displays a warning messageDuring the final stage of Oracle RAC 10g Release 2 installation, you may receivea warning message with the Oracle Cluster Verification utility.

For example:

Utility

============================================================

OUI-25031: Some of the configuration assistants failed. It is

strongly recommended that you retry the configuration

assistants at this time. Not successfully running any "

Recommended" assistants means your system will not be correctly

configured.

1. Check the Details panel on the Configuration Assistant Screen

to see the errors resulting in the failures.

2. Fix the errors causing these failures.

3. Select the failed assistants and click the 'Retry' button

to retry them.

============================================================

Workaround: You may safely ignore this message if the cluster is operatingsatisfactorily.

SF Oracle RAC issuesThis section lists the known issues in SF Oracle RAC for this release.

Issues related to installationThis section describes the known issues during installation and upgrade.

SF Oracle RAC installer does not support use of fully qualified domainnames (2585899)

The SF Oracle RAC installer does not support the use of fully qualified domainnames (FQDN). Specifying the fully qualified domain name of a system results inthe following error:

The node galaxy doesn’t seem to be part of the cluster,

or CVM is not running on the node galaxy.


36

Workaround: Use only the host name of the systemwhen you specify the systemname.

Warningmessagesmay be seen during script-based installation (2615500)

When you install SF Oracle RAC using the script-based installer, you may see thefollowing warning message:

interpreter "/opt/VRTSperl/bin/perl" not found

Workaround: You must install perl to resolve the issue.

To install perl

1 Exit the installer.

2 Install the VRTSperl fileset from the product media manually:

# cd /dvd_path/pkgs

# installp -a -d VRTSperl.bff VRTSperl

3 Start the installer again.

Stopping the installer during an upgrade and then resuming the upgrademight freeze the service groups (2591399)

The service groups freeze due to upgrading using the product installer if youstopped the installer after the installer already stopped some of the processesand then resumed the upgrade.

Workaround: You must unfreeze the service groups manually after the upgradecompletes.

To unfreeze the service groups manually

1 List all the frozen service groups

# hagrp -list Frozen=1

2 Unfreeze all the frozen service groups:

# haconf -makerw

# hagrp -unfreeze service_group -persistent

# haconf -dump -makero

Unable to stop some SF Oracle RAC processes (2329580)

If you install and start SF Oracle RAC, but later configure SF Oracle RAC usinginstallsfrac, somedriversmaynot stop successfullywhen the installer attempts


to stop and restart the SFOracleRACdrivers andprocesses. The reason the driversdo not stop is because some dependent SF Oracle RAC processes may be in therunning state.

Workaround: To re-configure the product, use the corresponding installproductcommand to re-configure the product. Otherwise some processesmay fail to stopor start. For example, use installsfrac to re-configure SF Oracle RAC ratherthan using installvcs.

Web installer does not ask for authentication after the first session if thebrowser is still open (2509330)

If you install or configure SF Oracle RAC and then close the Web installer, if youhave other browser windows open, the Web installer does not ask forauthentication in the subsequent sessions. Since there is no option to log out ofthe Web installer, the session remains open as long as the browser is open on thesystem.

Workaround: Make sure that all browser windows are closed to end the browsersession and subsequently log in again.

After finishing a kernel upgrade on amaster node the cvm group on a slavenode does not come online (2439439)

After successfully finishing a kernel upgrade on one node, the cvm group doesnot come online on the second node.

Workaround: Check that your cluster is not in a jeopardy state before you performa rolling upgrade.

After performing a rolling upgrade, make sure the CVM is online on allnodes without errors (2595441)

Make sure that the CVM is online on all nodes without errors after you performthe first phase of a rolling upgrade. The CVM protocol version will not upgradesuccessfully on the nodes where CVM is offline or has errors.

If the CVM protocol version does note upgrade successfully, upgrade the CVMprotocol on the CVM master node.

To upgrade the CVM protocol on the CVM master node

1 Find out which node is the CVM master. Enter the following:

# vxdctl -c mode

2 On the CVM master node, upgrade the CVM protocol. Enter the following:

# vxdctl upgrade


38

Issues related to LLTThis section covers the known issues related to LLT in this release.

LLT may fail to make connections with LLT on peer nodes in virtualenvironment (2343451/2376822)

After you upgrade from 5.0 MP3 or earlier releases to version 6.0, LLT may failto make connections with LLT on the peer nodes in AIX virtual environment.

This is a known IBM VIOS issue. Install APAR IV00776 on your VIOS server.Without this fix, VIOS fails to handle new LLT packet header and drops packets.

Workaround: Disable the largesend attribute of the SEA adapter. Check theproperties of the SEA adapter (on which the virtual links are configured underLLT maps) on each VIOS using the following command:

#lsattr -El SEA

If the largesend is set to 1, then set it to 0 using the following command:

#chdev -l SEA -a largesend=0

Issues related to GABThis section covers the known issues related to GAB in this release.

Cluster panics during reconfiguration (2590413)

While a cluster is reconfiguring, GAB broadcast protocol encounters a racecondition in the sequence request path. This condition occurs in an extremelynarrow window which eventually causes the GAB master to panic.

Issues related to I/O fencingThis section covers the known issues related to I/O fencing in this release.

Installer is unable to split a cluster that is registered with one or more CPservers (2110148)

Splitting a cluster that uses server-based fencing is currently not supported.

You can split a cluster into two and reconfigure SFOracle RAC on the two clustersusing the installer. For example, you can split a cluster clus1 into clus1A andclus1B.

However, if you use the installer to reconfigure the SF Oracle RAC, the installerretains the same cluster UUID of clus1 in both clus1A and clus1B. If both clus1Aand clus1B use the same CP servers for I/O fencing, then the CP server allowsregistration only from the cluster that attempts to register first. It rejects the


registration from the cluster that attempts next. Thus, the installer reports failureduring the reconfiguration of the cluster that uses server-based fencing.

Workaround: None.

Fencing does not come up on one of the nodes after a reboot (2573599)

If VxFEN unconfiguration has not finished its processing in the kernel and in themeantime if you attempt to start VxFEN, you may see the following error in the/var/VRTSvcs/log/vxfen/vxfen.log file:

VXFEN vxfenconfig ERROR V-11-2-1007 Vxfen already configured

However, the output of the gabconfig -a command does not list port b. Thevxfenadm -d command displays the following error:

VXFEN vxfenadm ERROR V-11-2-1115 Local node is not a member of cluster!

Workaround: Start VxFEN again after some time.

The vxfenswap utility does not detect failure of coordination pointsvalidation due to an RSH limitation (2531561)

The vxfenswap utility runs the vxfenconfig -o modify command over RSH orSSH on each cluster node for validation of coordination points. If you run thevxfenswap command using RSH (with the -n option), then RSH does not detectthe failure of validation of coordination points on a node. From this point,vxfenswap proceeds as if the validation was successful on all the nodes. But, itfails at a later stage when it tries to commit the new coordination points to theVxFENdriver. After the failure, it rolls back the entire operation, and exits cleanlywith anon-zero error code. If you runvxfenswapusingSSH (without the-noption),then SSH detects the failure of validation of coordination of points correctly androlls back the entire operation immediately.

Workaround: Use the vxfenswap utility with SSH (without the -n option).

In absence of cluster details in CP server, VxFEN fails with pre-existingsplit-brain message (2433060)

When you start server-based I/O fencing, the node may not join the cluster andprints error messages in logs similar to the following:

In the /var/VRTSvcs/log/vxfen/vxfen.log file:

VXFEN vxfenconfig ERROR V-11-2-1043

Detected a preexisting split brain. Unable to join cluster.

In the /var/VRTSvcs/log/vxfen/vxfen.log file:


40

operation failed.

CPS ERROR V-97-1400-446 Un-authorized user cpsclient@galaxy,

domaintype vx; not allowing action

The vxfend daemon on the application cluster queries the coordination pointserver (CP server) to check if the clustermembers as seen in theGABmembershipare registered with the CP server. If the application cluster fails to contact the CPserver due to some reason, then fencing cannot determine the registrations onthe CP server and conservatively assumes a pre-existing split-brain.

Workaround: Before you attempt to start VxFEN on the application, ensure thatthe cluster details such as cluster name, UUID, nodes, and privileges are addedto the CP server.

The cpsadm command fails if LLT is not configured on the applicationcluster (2583685)

The cpsadm command fails to communicate with the coordination point server(CP server) if LLT is not configured on the application cluster nodewhere you runthe cpsadm command. You may see errors similar to the following:

# cpsadm -s 10.209.125.200 -a ping_cps

CPS ERROR V-97-1400-729 Please ensure a valid nodeid using

environment variable

CPS_NODEID

CPS ERROR V-97-1400-777 Client unable to communicate with CPS.

However, if you run the cpsadm command on the CP server, this issue does notarise even if LLT is not configured on the node that hosts CP server. The cpsadmcommand on the CP server node always assumes the LLT node ID as 0 if LLT isnot configured.

According to the protocol between the CP server and the application cluster, whenyou run the cpsadm on an application cluster node, cpsadm needs to send the LLTnode ID of the local node to the CP server. But if LLT is unconfigured temporarily,or if the node is a single-node VCS configuration where LLT is not configured,then the cpsadm command cannot retrieve the LLT node ID. In such situations,the cpsadm command fails.

Workaround: Set the value of the CPS_NODEID environment variable to 255. Thecpsadm command reads the CPS_NODEID variable and proceeds if the command isunable to get LLT node ID from LLT.


Fencing port b is visible for few seconds even if cluster nodes have notregistered with CP server (2415619)

Even if the cluster nodes have no registration on the CP server and if you providecoordination point server (CP server) information in the vxfenmode file of thecluster nodes, and then start fencing, the fencing port b is visible for a few secondsand then disappears.

Workaround: Manually add the cluster nodes' and users' information to the CPserver to resolve this issue. Alternatively, you can use installer as the installeradds cluster nodes’ and users’ information to the CP server during configuration.

CP server repetitively logs unavailable IP addresses (2530864)

If coordination point server (CP server) fails to listen on any of the IP addressesthat are mentioned in the vxcps.conf file or that are dynamically added usingthe command line, then CP server logs an error at regular intervals to indicatethe failure. The logging continues until the IP address is bound to successfully.

CPS ERROR V-97-51-103 Could not create socket for host

10.209.79.60 on port 14250

CPS ERROR V-97-1400-791 Coordination point server could not

open listening port = [10.209.79.60]:14250

Check if port is already in use.

Workaround: Remove the offending IP address from the listening IP addresseslist using the rm_port action of the cpsadm command.


Server-based fencing comes up incorrectly if default port is notmentioned(2403453)

When you configure fencing in customized mode and do no provide default port,fencing comes up. However, the vxfenconfig -l command output does not listthe port numbers.

Workaround: Retain the "port=<port_value>" setting in the /etc/vxfenmode file,when using customized fencingwith atleast one CP server. The default port valueis 14250.

Secure CP server does not connect from localhost using 127.0.0.1 as theIP address (2554981)

The cpsadm command does not connect to the secure CP server on the localhostusing 127.0.0.1 as the IP address


42

Workaround: Connect the secure CP server using any of the virtual IPs that isconfigured with the CP server and is plumbed on the local node.

Unable to customize the 30-second duration (2551621)

When the vxcpserv process is not able to bind to an IP address during startup, itattempts to bind to that IP address at an interval of 30 seconds. This interval isnot configurable.

Workaround: No workaround.

NIC resource gets created with incorrect name while configuring CPSSGwith the configure_cps.pl script (2585229)

The name of the NIC resource created by the configure_cps.pl script does notcome out correct when, for example, mth VIP is mapped to nth NIC and every m isnot equal to n. In this case, although CPSSG continues to function without anyproblem, when you unconfigure CPSSG using configure_cps.pl, it fails.

Workaround: To unconfigure CPSSG, you must remove the CPSSG configurationfrom the VCS configuration.

The cpsadmcommand fails after upgrading CP server to 6.0 in securemode(2478502)

The cpsadm command may fail after you upgrade coordination point server (CPserver) to 6.0 in secure mode. If the old VRTSat fileset is not removed from thesystem, thecpsadm command loads theold security libraries present on the system.As the installer runs the cpsadm command on the CP server to add or upgrade theSF Oracle RAC cluster (application cluster), the installer also fails.

Workaround : Perform the following steps on all the nodes of the CP server:

■ Rename cpsadm to cpsadmbin.

# mv /opt/VRTScps/bin/cpsadm /opt/VRTScps/bin/cpsadmbin

■ Create a file /opt/VRTScps/bin/cpsadm with the following content:

#!/bin/sh

EAT_USE_LIBPATH=”/opt/VRTScps/lib”

export EAT_USE_LIBPATH

/opt/VRTScps/bin/cpsadmbin "$@"

■ Provide the following permissions to the new file:

# chmod 755 /opt/VRTScps/bin/cpsadm


PrivNIC and MultiPrivNIC agents not supported with OracleRAC 11.2.0.2ThePrivNIC andMultiPrivNIC agents are not supportedwithOracle RAC11.2.0.2.

For more information, see the following Technote:


Node fails to join the SF Oracle RAC cluster if the file systemcontaining Oracle Clusterware is not mounted (2611055)The sequence number of the startup script for Oracle High Availability Servicesdaemon (ohasd) is lower than some of the SF Oracle RAC components such asVXFEN and VCS. During system startup, if the file system containing OracleClusterware does not get mounted before the ohasd startup script is executed,the script continuously waits for the file system to become available. As a result,the other scripts (including those of SFOracle RAC components) are not executedand the node being started does not join the SF Oracle RAC cluster.

Workaround: If the rebooted node does not join the SF Oracle RAC cluster, thecluster can be started manually using the following command:

# installsfrac -start node1 node2

Issue with format of the last 8-bit number in private IPaddresses (1164506)ThePrivNIC/MultiPrivNIC resources fault if theprivate IP addresseshave a leading0 in any of the octets that comprise the IP address, for example X.X.X.01 orX.X.0X.1. or X.0X.X.1 or 0X.X.X.1, where X is an octet of the IP address.

When you configure private IP addresses for Oracle Clusterware, ensure that theIP addresses have a format as displayed in the following two-node example:

■ On galaxy: 192.168.12.1

■ On nebula: 192.168.12.2

Confirm the correct format by viewing the PrivNIC or MultiPrivNIC resource inthe /etc/VRTSvcs/conf/config/main.cf file.

CVMVolDg agent may fail to deport CVM disk groupThe CVM disk group is deported based on the order in which the CVMVolDgresources are taken offline. If the CVMVolDg resources in the disk group containa mixed setting of 1 and 0 for the CVMDeportOnOffline attribute, the disk groupis deported only if the attribute value is 1 for the last CVMVolDg resource taken


44


offline. If the attribute value is 0 for the last CVMVolDg resource taken offline,the disk group is not deported.

Workaround: If multiple CVMVolDg resources are configured for a shared diskgroup, set the value of the CVMDeportOnOffline attribute to 1 for all of theresources.

Long messages run off the screen if the screen width is lessthan 100 charactersMessages that exceed 80 characters escape from view if the screen width of yourterminal window is less than 100 characters. For example, when you run theinstallation and configuration checks using the SF Oracle RAC Installation andConfigurationChecks option in the SFOracle RAC installermenu, longmessagesrun off the side on terminal window sizes less than 100.

Workaround: Set the terminal window size to a value greater than 100.

Messages scroll out of view on clusters with three or morenodesOn clusters with three or more nodes, messages scroll out of view duringinstallation or configuration activities that print a large number of messages onscreen. For example,whenyou run the installation and configuration checks usingtheSFOracleRACInstallationandConfigurationChecks option in the SFOracleRAC installermenu on a three-node cluster, themessages run off the screen afterthe terminal window displays the first page ofmessages. Thesemessages can notbe viewed or retrieved.

Workaround: For any failures that may result during the checks, see the log file/opt/VRTS/install/logs.

Policy-managed Oracle RAC databases fail to come online onsome of the nodes in the server pool (2392741)If the cardinality of a policy-managed Oracle RAC database is set to a numberlesser than the number of nodes in the server pool, and if the Oracle agent triesto bring the database online on all the nodes in the server pool, the operation failson some of the nodes in the server pool. The resource on respective nodes moveto the faulted state.


"Configurationmust beReadWrite : Use haconf -makerw" errormessage appears in VCS engine log when hastop -local isinvoked (2609137)A message similar to the following example appears in the/var/VRTSvcs/log/engine_A.log log file when you run the hastop -local

command on any system in a SF Oracle RAC cluster that has CFSMountresources:

2011/11/15 19:09:57 VCS ERROR V-16-1-11335 Configuration must be

ReadWrite : Use haconf -makerw

The hastop -local command successfully runs and you can ignore the errormessage.

Workaround: There is no workaround for this issue.

Multiple system panics upon unmounting a CFS file system(2107152)There is a systempanicwhenyouunmount amntlock-protectedVxFS file system,if that device is duplicate mounted on different directories.

Workaround: There is no workaround for this issue.

The vxassist maxsize option fails to report the maximum sizeof the volume that can be createdwith given constraints whenthe disk group has the siteconsistent flag set (2563195)The vxassist maxsize option fails to report the maximum size of volume thatcan be created with given constraints when the disk group has the siteconsistentflag set. The following error is reported:

# vxassist -g dgname maxsize

VxVM vxassist ERROR V-5-1-752 No volume can be created within the given

constraints

Workaround:

Specify the size explicitly to the vxassist make command.


46

Application group attempts to come online on primary sitebefore fire drill service group goes offline on the secondarysite (2107386)The application service group comes online on the primary sitewhile the fire drillservice group attempts to go offline at the same time, causing the applicationgroup to fault.

Workaround: Ensure that the fire drill service group is completely offline on thesecondary site before the application service group comes online on the primarysite.

Oracle group fails to come online if Fire Drill group is onlineon secondary cluster [2556835]If a parallel global service group faults on the local cluster and does not find afailover target in the local cluster, it tries to failover the service group to theremote cluster. However, if the firedrill for the service group is online on a remotecluster, offline local dependency is violated and the global service group is notable to failover to the remote cluster.

Workaround: Offline the Firedrill service group and online the service group ona remote cluster.

Oracle service group faults on secondary site during failoverin a disaster recovery scenario [2558903]Oracle service group fails to go online in the DR site when disaster strikes theprimary site. This happens if the AutoFailover attribute on the Service Group isset to 1 and when the corresponding service group's FireDrill is online in the DRsite. Firedrill Service group may remain ONLINE on the DR site.

Workaround: If the service group containing theOracle (or any database) resourcefaults after attempting automatic DR failover while FireDrill is online in the DRsite, manually offline the FireDrill Service Group. Subsequently, attempt theonline of the Oracle Service Group in the DR site.

Removal of SAN cable from any node in a global cluster setuptakes application service groups offline on all nodes (2580393)In a replicated global cluster setup, the removal of SAN cable from any node inthe cluster causes theCFSmount points to fault. As a result, dependent applicationgroups are taken offline and replication to the secondary site is adversely affected.


Veritas Storage Foundation for Databases (SFDB) tools known issuesThe following are known issues in this release of Veritas Storage Foundationproducts.

Database Storage Checkpoint unmount may fail with devicebusy (2591463)In some cases,when adatabase that is clonedusing aDatabase StorageCheckpointis shut down, an error similar to the following may occur:

SFAE Error:0457: Failed to unmount device

/dev/vx/dsk/datadg/datavol:Ckpt_1317707593_rw_1317708154.

Reason: VxFS returned error : umount: /tmp/clonedb/data: device is busy

Workaround

As an Oracle user, force shut down the clone database if it is up and then retrythe unmount operation.

Incorrect error message if wrong host name is provided(2585643)If you provide an incorrect host namewith the -r option of vxsfadm, the commandfails with an error message similar to one of the following:

FSM Error: Can't use string ("") as a HASH ref while "strict refs"

in use at /opt/VRTSdbed/lib/perl/DBED/SfaeFsm.pm line 776.

SFDB vxsfadm ERROR V-81-0609 Repository location is invalid.

The error messages are unclear.

Workaround

Provide the name of a host that has the repository database, with the -r optionof vxsfadm.

FlashSnap validate reports snapshot unsplittable (2534422)The FlashSnap validation operation fails with the following error if the mirrorsfor data volumes and archive log volumes share the same set of disks:

SFAE Error:0642: Storage for diskgroup oradatadg is not splittable.


48

Workaround

Ensure that snapshot plexes for data volumes and snapshot plexes for archivelog volumes reside on separate set of disks.

Attempt to use SmartTier commands fails (2332973)The attempts to run SmartTier commands such as dbdst_preset_policyordbdst_file_move fail with the following error:

fsppadm: ERROR: V-3-26551: VxFS failure on low level mechanism

with message - Device or resource busy

This error occurs if a sub-file SmartTier command such as dbdst_obj_move hasbeen previously run on the file system.

There is no workaround for this issue. You cannot use file-based SmartTier andsub-file SmartTier simultaneously.

dbed_vmclonedb ignores new clone SID value after cloningonce (2580318)After you have done FlashSnap cloning using a snapplan, any further attemptsto create a clone from the same snapplan using the dbed_vmclonedb continue touse the original clone SID, rather than the new SID specified using the new_sidparameter.

This issue is also observedwhen you resynchronize the snapplan, take a snapshotagain without specifying the new clone SID, and then try to clone with the newSID.

Workaround

You can use one of the following workarounds:

■ After the snapshot is resynchronized, delete the snapplan using thedbed_vmchecksnap -o remove command. You can then use a new clone SIDby creating a new snapplan, which may have the same name, and using thesnapplan for taking more snapshots.

■ Use the vxsfadm command to take the snapshot again and specify the cloneSID with the snapshot operation so that the clone operation can be done withthe new clone SID.


Attempt to use certain names for tiers results in error(2581390)If you attempt to use certain names for tiers, the following error message isdisplayed:

SFORA dbdst_classify ERROR V-81-6107 Invalid Classname BALANCE

This error occurs because the followingnames are reserved and are not permittedas tier names for SmartTier:

■ BALANCE

■ CHECKPOINT

■ METADATA

Workaround

Use a name for SmartTier classes that is not a reserved name.

User authentication fails (2579929)The sfae_auth_op -o auth_user command, used for authorizing users, failswith the following error message:

SFDB vxsfadm ERROR V-81-0384 Unable to store credentials for <username>

Reattempting the operation fails with the following error message:

SFDB vxsfadm ERROR V-81-0372 AT broker failed to start:

The authentication setup might have been run with a strict umask value, whichresults in the required files and directories being inaccessible to the non-rootusers.

Workaround

If you have not done authentication setup, set umask to a less strict value beforerunning thesfae_auth_op -o setuporsfae_auth_op -o import_broker_config

commands.

To set umask to a less strict value

◆ Use the command:

# umask 022

If you have already done authentication setup, perform the following steps.


50

To resolve the problem if you have already done authentication setup

1 Shut down the authentication broker, if it is running.

# /opt/VRTSdbed/at-broker/bin/sfaeatd.sh stop

2 Change the permissions for files and directories that are required to bereadable by non-root users.

# chmod o+r /etc/vx/vxdbed/admin.properties

# chmod o+rx /var/vx/vxdba/auth/users

# find /opt/VRTSdbed/at-broker -type d -exec chmod o+rx {} \;

Clone operation failure might leave clone database inunexpected state (2512664)If the clone operation fails, itmay leave the clone database in an unexpected state.Retrying the clone operation might not work.

Workaround

If retrying does not work, perform one the following actions depending on thepoint-in-time copy method you are using:

■ For FlashSnap, resync the snapshot and try the clone operation again.

■ For FileSnap and Database Storage Checkpoints, destroy the clone and createthe clone again.

■ For space-optimized snapshots, destroy the snapshot and create a newsnapshot.

Contact Symantec support if retrying using the workaround does not succeed.

FlashSnap resync fails if there is an existing space-optimizedsnapshot (2479901)If you try a FlashSnap resync operationwhen there is an existing space-optimizedsnapshot, the resync operation fails with the following error:

Error: VxVM vxdg ERROR V-5-1-4597 vxdg join FS_oradg oradg failed

datavol_snp : Record already exists in disk group

archvol_snp : Record already exists in disk group

Workaround

Destroy the space-optimized snapshot first and then perform the FlashSnapresync operation.


Upgrading Veritas Storage Foundation for Databases (SFDB)tools from 5.0x to 6.0 (2184482)The sfua_rept_migratecommand results in an error message after upgradingSFHA or SF for Oracle RAC version 5.0 or 5.0MP3 to SFHA or SF for Oracle RAC6.0.

When upgrading from SF Oracle RAC version 5.0 or 5.0MP3 to SF Oracle RAC 6.0the S*vxdbms3 startup script is renamed to NO_S*vxdbms3. The S*vxdbms3startup script is required by sfua_rept_upgrade. Thuswhen sfua_rept_upgrade

is run, it is unable to find theS*vxdbms3 startup script andgives the errormessage:

/sbin/rc3.d/S*vxdbms3 not found

SFORA sfua_rept_migrate ERROR V-81-3558 File: is missing.

SFORA sfua_rept_migrate ERROR V-81-9160 Failed to mount repository.

Workaround

Before running sfua_rept_migrate, rename the startup script NO_S*vxdbms3to S*vxdbms3.

Clone command fails if PFILE entries have their values spreadacross multiple lines (1764885)If you have a log_archive_dest_1 in single line in the init.ora file, thendbed_vmclonedbwill work but dbed_vmclonebwill fail if you put inmultiple linesfor log_archive_dest_1.

Workaround

There is no workaround for this issue.

Health checkmonitoring is not supported for Oracle database11g R1 and 11g R2 [1985055]Health checkmonitoring is not supported for Oracle database 11g R1 and 11g R2.

Workaround: Set MonitorOption attribute for Oracle resource to 0.

Software limitationsThis section covers the software limitations of this release.

See the corresponding Release Notes for a complete list of software limitationsrelated to that component or product.


Storage Foundation for Oracle RAC Release NotesSoftware limitations

52

Oracle Clusterware/Grid Infrastructure installation fails if the clustername exceeds 14 characters

Setting the cluster name to a value that exceeds 14 characters during theinstallation of Oracle Clusterware/Grid Infrastructure causes unexpected clustermembership issues. As a result, the installation may fail.

Workaround: Restart the Oracle Clusterware/Grid Infrastructure installationand set the cluster name to a value of maximum 14 characters.

Parallel execution of vxsfadm is not supported (2515442)Only one instance of the vxsfadm command can be run at a time. Runningmultipleinstances of vxsfadm at a time is not supported.

Stale SCSI-3 PR keys remain on disk after stopping the cluster anddeporting the disk group

Whenall nodes present in the SFOracle RAC cluster are removed from the cluster,the SCSI-3 Persistent Reservation (PR) keys on the data disks may not getpreempted. As a result, the keysmaybe seen on the disks after stopping the clusteror after the nodes have booted up. The residual keys do not impact data diskfencing as they will be reused or replaced when the nodes rejoin the cluster.Alternatively, the keys can be cleared manually by running the vxfenclearpreutility.

For more information on the vxfenclearpre utility, see the Veritas StorageFoundation for Oracle RAC Administrator's Guide.

Creating point-in-time copies during database structural changes isnot supported (2496178)

SFDB tools do not support creating point-in-time copies while structural changesto the database are in progress, such as adding or dropping tablespaces and addingor dropping data files.

However, once a point-in-time copy is taken, you can create a clone at any time,regardless of the status of the database.

53Storage Foundation for Oracle RAC Release NotesSoftware limitations

Web installation program does not support phased upgrade and nativeoperating system upgrade mechanisms

The Web installation program does not support phased upgrade and nativeoperating system upgrade mechanisms such as Alternate Disk Installation (ADI)and Network Installation Manager Alternate Disk Migration (NIMADM).

Policy-managed databases not supported by CRSResource agentThe CRSResource agent supports only admin-managed database environmentsin this release. Policy-managed databases are not supported.

Health checks may fail on clusters that have more than 10 nodesIf there are more than 10 nodes in a cluster, the health check may fail with thefollowing error:

vxgettext ERROR V-33-1000-10038

Arguments exceed the maximum limit of 10

The health check script uses the vxgettext command, which does not supportmore than 10 arguments.[2142234]

Cached ODM not supported in SF Oracle RAC environmentsCachedODM is not supported for files onVeritas local file systems and on ClusterFile System.

Limitation on upgrading to 6.0 on a Veritas Storage Foundation andHigh Availability cluster

Veritas Storage Foundation (SF) 6.0 requires the AIX operating system to be at6.1 TL5 or above. To upgrade SF to 6.0 from a release prior to 5.0 MP3 RP1, youmust first upgrade SF to the 5.0 MP3 RP1 release. If upgrading to 5.0 MP3 RP1requires an intermediate operating system upgrade, the operating system levelcannot exceed 6.1 TL1. After upgrading to 5.0 MP3 RP1, you must upgrade theoperating system to AIX 6.1 TL5, which is the minimum requirement for the 6.0release. You must upgrade SF to 5.0 MP3 RP1 to avoid a system panic or crashthat can occur when a node running AIX 6.1 TL2 or above with a release prior to5.0MP3RP1 is removed fromtheVeritas StorageFoundationandHighAvailabilitycluster. Removing the node causes file system threads to exit. The panic is causedby a check introduced from AIX 6.1 TL2 that validates the lockcount values whena kernel-thread-call exits.


54

For more information, see the following TechNote:


Limitations related to I/O fencingThis section covers I/O fencing-related software limitations.

Preferred fencing limitationwhenVxFENactivatesRACERnodere-electionThe preferred fencing feature gives preference to more weighted or largersubclusters by delaying the smaller subcluster. This smaller subcluster delay iseffective only if the initial RACERnode in the larger subcluster is able to completethe race. If due to some reason the initial RACER node is not able to complete therace and the VxFEN driver activates the racer re-election algorithm, then thesmaller subcluster delay is offset by the time taken for the racer re-election andthe lessweighted or smaller subcluster couldwin the race. This limitation thoughnot desirable can be tolerated.

Limitation with RDAC driver and FAStT array for coordinatordisks that use raw disksFormultipathing to connected storage,AIXuses theRDACdriver for FAStTarrays.Since it is an active/passive array, only the current active path is exposed toclients. The I/O fencing driver, vxfen, can use only a single active path and hasno foreknowledge of the passive paths to the coordinator disks on an array. If thesingle active path fails, all nodes in the cluster lose access to the coordinator disks.

The loss of the path to the coordinator disks can potentially go unnoticed until areboot, split brain, or any other reason that leads to a clustermembership changeoccurs. In any of these conditions, the cluster cannot form, and all nodes panicto prevent data corruption. No data loss occurs.

Workaround:UseDMPand specify paths to coordinator disks asDMPpaths ratherthan raw disks to avoid this limitation.

Stopping systems in clusters with I/O fencing configuredThe I/O fencing feature protects against data corruption resulting from a failedcluster interconnect, or “split brain.”See theVeritasClusterServerAdministrator'sGuide for a description of the problems a failed interconnect can create and theprotection I/O fencing provides.

In a cluster using SCSI-3 based fencing, I/O fencing implements data protectionby placing the SCSI-3 PR keys on both the data disks and coordinator disks. In a

55Storage Foundation for Oracle RAC Release NotesSoftware limitations


cluster using CP server-based fencing, I/O fencing implements data protectionby placing the SCSI-3 PRkeys on data disks and similar registrations onCP server.The VCS administrator must be aware of several operational changes neededwhenworkingwith clustersprotectedby I/O fencing. Specific shutdownproceduresensure keys are removed from coordination points and data disks to preventpossible difficulties with subsequent cluster startup.

Using the reboot commandrather than theshutdowncommandbypasses shutdownscripts and can leave keys on the coordination points and data disks. Dependingon the order of reboot and subsequent startup events, the cluster may warn of apossible split brain condition and fail to start up.

Workaround: Use the shutdown -r command on one node at a time and wait foreach node to complete shutdown.

UninstallingVRTSvxvmcauses issueswhenVxFEN is configuredin SCSI3 mode with dmp disk policyWhen VxFEN is configured in SCSI3 mode with dmp disk policy, the DMP nodesfor the coordinator disks can be accessed during system shutdown or fencingarbitration. After uninstalling VRTSvxvm fileset, the DMP module will no longerbe loaded in memory. On a system where VRTSvxvm fileset is uninstalled, ifVxFEN attempts to access DMP devices during shutdown or fencing arbitration,the system panics. [Incident 2522069]

Veritas Storage Foundation for Databases (SFDB) tools softwarelimitations

The following are the SFDB tools software limitations in this release.

Oracle Data Guard in an Oracle RAC environmentDatabase snapshots and Database Storage Checkpoints are not supported in aData Guard with Oracle RAC environment.

Upgrading if using Oracle 11.1.0.6If you are running Oracle version 11.1.0.6 and upgrading a Storage Foundationproduct to 6.0: upgrade theOracle binaries and database to version 11.1.0.7 beforemoving to 6.0.


56

DocumentationProduct guides are available in the PDF format on the software media in the/product_name/docs directory. Additional documentation is available online.

Symantec recommends copyingpertinent information, such as installation guidesand release notes to your system's /opt/VRTS/docs directory for reference.

Make sure that you are using the current version of documentation. The documentversion appears on page 2 of each guide. The publication date appears on the titlepage of each document. The latest product documentation is available on theSymantec website.

http://sort.symantec.com/documents

Documentation setTable 1-12 lists the documentation forVeritas Storage Foundation forOracleRAC.

Table 1-12 Veritas Storage Foundation for Oracle RAC documentation

File nameDocument title

sfrac_notes_60_aix.pdfVeritasStorageFoundation forOracleRACReleaseNotes

sfrac_install_60_aix.pdfVeritas Storage Foundation for Oracle RACInstallation and Configuration Guide

sfrac_admin_60_aix.pdfVeritas Storage Foundation for Oracle RACAdministrator's Guide

Table 1-13 lists the documentation for Veritas Storage Foundation Cluster FileSystem High Availability.

Table 1-13 Veritas Storage Foundation Cluster File System High Availabilitydocumentation


sfcfs_notes_60_aix.pdfVeritas Storage Foundation Cluster File SystemHigh Availability Release Notes

sfcfs_install_60_aix.pdfVeritas Storage Foundation Cluster File SystemHigh Availability Installation Guide

sfcfs_admin_60_aix.pdfVeritas Storage Foundation Cluster File SystemHigh Availability Administrator's Guide

57Storage Foundation for Oracle RAC Release NotesDocumentation


Table 1-14 lists the documents for Veritas Cluster Server.

Table 1-14 Veritas Cluster Server documentation

File nameTitle

vcs_install_60_aix.pdfVeritas Cluster Server Installation Guide

vcs_notes_60_aix.pdfVeritas Cluster Server Release Notes

vcs_admin_60_aix.pdfVeritas Cluster Server Administrator’s Guide

vcs_bundled_agents_60_aix.pdfVeritas Cluster Server Bundled Agents ReferenceGuide

vcs_agent_dev_60_unix.pdfVeritas Cluster Server Agent Developer’s Guide

vcs_db2_agent_60_aix.pdfVeritas Cluster Server Agent for DB2 Installationand Configuration Guide

vcs_oracle_agent_60_aix.pdfVeritasClusterServerAgent forOracle Installationand Configuration Guide

vcs_sybase_agent_60_aix.pdfVeritasClusterServerAgent forSybase Installationand Configuration Guide

Table 1-15 lists the documentation for Veritas Storage Foundation.

Table 1-15 Veritas Storage Foundation documentation


sf_notes_60_aix.pdfVeritas Storage Foundation Release Notes

sf_install_60_aix.pdfVeritas Storage Foundation Installation Guide

sf_admin_60_aix.pdfVeritasStorageFoundationAdministrator'sGuide

sf_adv_ora_60_aix.pdfVeritas Storage Foundation: Storage andAvailability Management for Oracle Databases

vxfs_ref_60_aix.pdfVeritasFileSystemProgrammer'sReferenceGuide

Table 1-16 lists the documentation for Veritas Storage Foundation and HighAvailability Solutions products.

Storage Foundation for Oracle RAC Release NotesDocumentation

58

Table 1-16 Veritas Storage Foundation and High Availability Solutions productsdocumentation


sfha_solutions_60_aix.pdfVeritas Storage Foundation andHighAvailabilitySolutions Solutions Guide

sfha_virtualization_60_aix.pdfVeritas Storage Foundation andHighAvailabilitySolutions Virtualization Guide

sf_replication_admin_60_aix.pdfVeritas Storage Foundation andHighAvailabilitySolutions Replication Administrator's Guide

If you use Veritas Operations Manager (VOM) to manage Veritas StorageFoundation and High Availability products, refer to the VOM productdocumentation at:


Manual pagesThemanual pages forVeritas StorageFoundation andHighAvailability Solutionsproducts are installed in the /opt/VRTS/man directory.

Set the MANPATH environment variable so the man(1) command can point to theVeritas Storage Foundation manual pages:

■ For the Bourne or Korn shell (sh or ksh), enter the following commands:

MANPATH=$MANPATH:/opt/VRTS/man

export MANPATH

■ For C shell (csh or tcsh), enter the following command:

setenv MANPATH ${MANPATH}:/opt/VRTS/man

See the man(1) manual page.

59Storage Foundation for Oracle RAC Release NotesDocumentation


Storage Foundation for Oracle RAC Release NotesDocumentation

60

veritasstoragefoundation for oracle® rac release …storage foundation for oracle rac release notes...

Documents