rac 11g on windows

Table of ContentsRac11gR2OnWindows........................................................................................................................................1

1. Introduction.........................................................................................................................................1 1.1. Overview of new concepts in 11gR2 Grid Infrastructure...................................................1

1.1.1. SCAN..................................................................................................................1 1.1.2. GNS....................................................................................................................1 1.1.3. OCR and Voting on ASM storage......................................................................1 1.1.4. Intelligent Platform Management interface (IPMI)............................................1 1.1.5. Time sync............................................................................................................2 1.1.6. Clusterware and ASM share the same Oracle Home..........................................2

1.2. System Requirements....................................................................................................................................2 1.2.1. Hardware Requirements................................................................................................................2 1.2.2. Network Hardware Requirements.................................................................................................2 1.2.3. IP Address Requirements..............................................................................................................3 1.2.4. Installation method.......................................................................................................................3

2. Prepare the cluster nodes for Oracle RAC.......................................................................................................3 2.1. User Accounts..................................................................................................................................3

2.1.1. User Account changes specifically for Windows 2008:..................................................3 2.1.2. Net Use Test.....................................................................................................................4 2.1.3. Remote Registry Connect................................................................................................5

2.2. Networking...................................................................................................................................................5 2.2.1. Network Ping Tests.......................................................................................................................6 2.2.2. Network Interface Binding Order (and Protocol Priorities).........................................................7 2.2.3. Disable DHCP Media Sense.........................................................................................................7 2.2.4. Disable SNP Features...................................................................................................................7

2.3. Stopping Services..........................................................................................................................................7 2.4. Synchronizing the Time on ALL Nodes.......................................................................................................8 2.5. Environment Variables.................................................................................................................................8 2.6. Stage the Oracle Software.............................................................................................................................8 2.7. Cluster Verification Utility (CVU) stage check............................................................................................9 3. Prepare the shared storage for Oracle RAC.....................................................................................................9

3.1. Shared Disk Layout..........................................................................................................................9 3.1.1. Grid Infrastructure Shared Storage..................................................................................9 3.1.2. ASM Shared Storage.....................................................................................................10

3.2. Enable Automount......................................................................................................................................10 3.3. Clean the Shared Disks...............................................................................................................................10 3.4. Create Logical partitions inside Extended partitions..................................................................................11

3.4.1. View Created partitions..............................................................................................................13 3.5. List Drive Letters........................................................................................................................................13

3.5.1. Remove Drive Letters.................................................................................................................13 3.5.2. List volumes on Second node.....................................................................................................15

3.6. Marking Disk Partitions for use by ASM...................................................................................................15 3.7. Verify Grid Infrastructure Installation Readiness.......................................................................................17 4. Oracle Grid Infrastructure Install...................................................................................................................18

4.1. Basic Grid Infrastructure Install (without GNS and IPMI)...........................................................18 5. Grid Infrastructure Home Patching................................................................................................................27 6. RDBMS Software Install...............................................................................................................................27 7. RAC Home Patching......................................................................................................................................31 8. Run ASMCA to create diskgroups................................................................................................................31 9. Run DBCA to create the database.................................................................................................................34

i

Rac11gR2OnWindows

1. Introduction

1.1. Overview of new concepts in 11gR2 Grid Infrastructure

1.1.1. SCAN

The single client access name (SCAN) is the address used by all clients connecting to the cluster. The SCANname is a domain name registered to three IP addresses, either in the domain name service (DNS) or the GridNaming Service (GNS). The SCAN name eliminates the need to change clients when nodes are added to orremoved from the cluster. Clients using SCAN names can also access the cluster using EZCONNECT.

The Single Client Access Name (SCAN) is a domain name that resolves to all the addresses allocatedfor the SCAN name. Allocate three addresses to the SCAN name. During Oracle grid infrastructureinstallation, listeners are created for each of the SCAN addresses, and Oracle grid infrastructurecontrols which server responds to a SCAN address request. Provide three IP addresses in the DNS touse for SCAN name mapping. This ensures high availability.

•

The SCAN addresses need to be on the same subnet as the VIP addresses for nodes in the cluster.• The SCAN domain name must be unique within your corporate network.•

1.1.2. GNS

In the past, the host and VIP names and addresses were defined in the DNS or locally in a hosts file. GNS cansimplify this setup by using DHCP. To use GNS, DHCP must be configured in the subdomain in which thecluster resides.

1.1.3. OCR and Voting on ASM storage

The ability to use ASM diskgroups for the storage of Clusterware OCR and Voting disks is a new feature inthe Oracle Database 11g Release 2 Grid Infrastructure. If you choose this option and ASM is not yetconfigured, OUI launches ASM configuration assistant to configure ASM and a diskgroup.

1.1.4. Intelligent Platform Management interface (IPMI)

Intelligent Platform Management Interface (IPMI) provides a set of common interfaces to computer hardwareand firmware that administrators can use to monitor system health and manage the system.

With Oracle Database 11g Release 2, Oracle Clusterware can integrate IPMI to provide failure isolationsupport and to ensure cluster integrity. You must have the following hardware and software configured toenable cluster nodes to be managed with IPMI:

Each cluster member node requires a Baseboard Management Controller (BMC) running firmwarecompatible with IPMI version 1.5, which supports IPMI over LANs, and configured for remotecontrol.

•

Each cluster member node requires an IPMI driver installed on each node.• The cluster requires a management network for IPMI. This can be a shared network, but Oraclerecommends that you configure a dedicated network.

•

Each cluster node's ethernet port used by BMC must be connected to the IPMI management network.•

Rac11gR2OnWindows 1

If you intend to use IPMI, then you must provide an administration account username and password toprovide when prompted during installation.

1.1.5. Time sync

There is a general requirement for Oracle RAC that the time on all the nodes be the same. With 11gR2 timesynchronization can be performed by the Clusterware using CTSSD (Cluster Time Synchronization ServicesDaemon) or by using the Windows Time Service. If the Windows Time Service is being used, it MUST beconfigured to prevent the time from being adjusted backwards.

1.1.6. Clusterware and ASM share the same Oracle Home

The clusterware and ASM share the same home thus we call it Grid Infrastructure home (prior to 11gR2 ASMcould be installed either in a separate home or in the same Oracle home as RDBMS.)

1.2. System Requirements

1.2.1. Hardware Requirements

Physical memory (at least 1.5 gigabyte (GB) of RAM)• An amount of swap space equal the amount of RAM• Temporary space (at least 1 GB) available in /tmp• A processor type (CPU) that is certified with the version of the Oracle software being installed• At minimum of 1024 x 786 display resolution, so that Oracle Universal Installer (OUI) displayscorrectly

•

All servers that will be used in the cluster have the same chip architecture, for example, all 32-bitprocessors or all 64-bit processors

•

Disk space for software installation locations. You will need at least 4.5 GB of available disk spacefor the Grid home directory, which includes both the binary files for Oracle Clusterware and OracleAutomatic Storage Management (Oracle ASM) and their associated log files, and at least 4 GB ofavailable disk space for the Oracle Database home directory.

•

Shared disk space•

An Oracle RAC database is a shared everything database. All data files, control files, redo log files, and theserver parameter file (SPFILE) used by the Oracle RAC database must reside on shared storage that isaccessible by all the Oracle RAC database instances. The Oracle RAC installation that is described in thisguide uses Oracle ASM for the shared storage for Oracle Clusterware and Oracle Database files. The amountof shared disk space is determined by the size of your database.

1.2.2. Network Hardware Requirements

Each node has at least two network interface cards (NIC), or network adapters.• Public interface names must be the same for all nodes. If the public interface on one node uses thenetwork adapter 'PublicLAN', then you must configure 'PublicLAN' as the public interface on allnodes.

•

You should configure the same private interface names for all nodes as well. If 'PrivateLAN' is theprivate interface name for the first node, then 'PrivateLAN' should be the private interface name foryour second node.

•

For the private network, the end points of all designated interconnect interfaces must be completelyreachable on the network. Every node in the cluster should be able to connect to every privatenetwork interface in the cluster.

•

1.1.4. Intelligent Platform Management interface (IPMI) 2

The host name of each node must conform to the RFC 952 standard, which permits alphanumericcharacters. Host names using underscores ("_") are not allowed.

•

1.2.3. IP Address Requirements

One public IP address for each node• One virtual IP address for each node• Three single client access name (SCAN) addresses for the cluster•

1.2.4. Installation method

This document details the steps for installing a 2-node Oracle 11gR2 RAC cluster on Windows:

The Oracle Grid Infrastructure Home binaries are installed on the local disk of each of the RACnodes.

•

The files required by Oracle Clusterware (OCR and Voting disks) are stored in ASM• The installation is explained without GNS and IPMI (additional Information for installation with GNSand IPMI are explained)

•

2. Prepare the cluster nodes for Oracle RAC

The guides include hidden sections, use the and image for each section to show/hide the section or youcan Expand all or Collapse all by clicking these buttons. This is implemented using the Twisty Plugin whichrequires Java Script to be enabled on your browser.

2.1. User Accounts

The installation should be performed as the Local Administrator, the Local Administrator username andpassword MUST be identical on all cluster nodes.

If a domain account is used, this domain account must be explicitly defined as a member of the LocalAdministrator group on all cluster nodes.

For Windows 2008:

Open Windows 2008 Server Manager• Expand the Configuration category in the console tree• Expand the Local Users and Groups category in the console tree• Within Groups, open the Administrator group• Add the desired user account as a member of the Administrator Group• Click OK to save the changes.•

We must now configure and test the installation user's ability to interact with the other cluster nodes.

2.1.1. User Account changes specifically for Windows 2008:

1. Change the elevation prompt behavior for administrators to "Elevate without prompting" to allow for userequivalence to function properly in Windows 2008:

Open a command prompt and type "secpol.msc" to launch the Security Policy Console managementutility.

•

1.2.2. Network Hardware Requirements 3

http://globaldc.oracle.com/perl/twiki/view/TWiki/JQueryTwistyPlugin

From the Local Security Settings console tree, click Local Policies, and then Security Options• Scroll down to and double-click User Account Control: Behavior of the elevation prompt foradministrators.

•

From the drop-down menu, select: "Elevate without prompting (tasks requesting elevation willautomatically run as elevated without prompting the administrator)"

•

Click OK to confirm the changes.• Repeat the previous 5 steps on ALL cluster nodes.•

2. Ensure that the Administrators group is listed under "Manage auditing and security log":

Open a command prompt and type "secpol.msc" to launch the Security Policy Console managementutility.

•

Click on "Local Policies"• Click on "User Rights Assignment"• Locate and double click the "Manage auditing and security log" in the listing of User RightsAssignments.

•

If the Administrators group is NOT listed in the "Local Security Settings" tab, add the group now.• Click OK to save the changes (if changes were made).• Repeat the previous 6 steps on ALL cluster nodes.•

3. Disable Windows Firewall When installing Oracle Grid Infrastructure and/or Oracle RAC it is required toturn off the Windows firewall. Follow these steps to turn off the windows firewall :

Click Start, click Run, type "firewall.cpl", and then click OK• In the Firewall Control Pannel, click "Turn Windows Firewall on or off" (upper left hand corner ofthe window).

•

Choose the "Off" radio button in the "Windows Firewall Settings" window and click OK to save thechanges.

•

Repeat the previous 3 steps on ALL cluster nodes.•

After the installation is successful, you can enable the Windows Firewall for the public connections. However,to ensure correct operation of the Oracle software, you must add certain executables and ports to the Firewallexception list on all the nodes of a cluster. See Section 5.1.2, "Configure Exceptions for the WindowsFirewall" of OracleÂ® Grid Infrastructure Installation Guide 11g Release 2 (11.2) for Microsoft Windows fordetails: http://download.oracle.com/docs/cd/E11882_01/install.112/e10817/postinst.htm#CHDJGCEHNOTE: The Windows Firewall must be disabled on all the nodes in the cluster before performing anycluster-wide configuration changes, such as:

Adding a node• Deleting a node• Upgrading to patch release• Applying a one-off patch•

If you do not disable the Windows Firewall before performing these actions, then the changes might not bepropagated correctly to all the nodes of the cluster.

2.1.2. Net Use Test

The "net use" utility can be used to validate the ability to perform the software copy among the cluster nodes.

Open a command prompt• Execute the following (replacing C$ with the appropriate drive letter if necessary) repeat thecommand to ensure access to every node in the cluster from the local node replacing with theappropriate nodes in the cluster.

•

2.1.1. User Account changes specifically for Windows 2008: 4

http://download.oracle.com/docs/cd/E11882_01/install.112/e10817/postinst.htm#CHDJGCEH

C:\Users\Administrator>net use \\remote node name\C$ The command completed successfully.

Repeat the previous 2 steps on ALL cluster nodes.•

2.1.3. Remote Registry Connect

Validate the ability to connect to the remote nodes registry(s) as follows:

Open a command prompt and type "regedit"• Within the registry editor menu bar, choose File and select "Connect Network Registry"• In the Select Computer window enter the remote node name.• Click OK and wait for the remote registry to appear in the tree.• Repeat the previous 4 steps for ALL cluster nodes.•

2.2. Networking

NOTE: This section is intended to be used for installations NOT using GNS.

1. Determine your cluster name. The cluster name should satisfy the following conditions:

The cluster name is globally unique throughout your host domain.• The cluster name is at least 1 character long and less than 15 characters long.• The cluster name must consist of the same character set used for host names: single-bytealphanumeric characters (a to z, A to Z, and 0 to 9) and hyphens (-).

•

NOTE: It is a requirement that network interfaces used for the Public and Private Interconnect be consistentlynamed (have the same name) on every node in the cluster. Common practice is to use the names Public andPrivate for the interfaces, long names should be avoided and special characters are NOT to be used.

2. Determine the public host name for each node in the cluster. For the public host name, use the primary hostname of each node. In other words, use the name displayed by the hostname command for example: racnode1.

It is recommended that NIC teaming is configured. Active/passive is the preferred teaming methoddue to its simplistic configuration.

•

For Windows 2008:

Perform the following to rename the network interfaces:

Click Start, click Run, type "ncpa.cpl", and then click OK.• Determine the intended purpose for each of the interfaces (may need to view the IP configuration)• Right click the interface to be renamed and click "rename"• Enter the desired name for the interface.• Repeat the previous 4 steps on ALL cluster nodes ensuring that the public and private interfaces havethe same name on every node.

•

3. Determine the virtual hostname for each node in the cluster. The virtual host name is a public node namethat is used to reroute client requests sent to the node if the node is down. Oracle recommends that youprovide a name in the format <public hostname>-vip, for example: racnode1-vip. The virutal hostname mustmeet the following requirements:

The virtual IP address and the network name must not be currently in use.• The virtual IP address must be on the same subnet as your public IP address.•

2.1.2. Net Use Test 5

The virtual host name for each node should be registered with your DNS.•

4. Determine the private hostname for each node in the cluster. This private hostname does not need to beresolvable through DNS and should be entered in the hosts file (typically located in:c:\windows\system32\drivers\etc). A common naming convention for the private hostname is <publichostname>-priv.

The private IP should NOT be accessible to servers not participating in the local cluster.• The private network should be on standalone dedicated switch(es).• The private network should NOT be part of a larger overall network topology.• The private network should be deployed on Gigabit Ethernet or better.• It is recommended that redundant NICs are configured using teaming. Active/passive is the preferredteaming method due to its simplistic configuration.

•

5. Define a SCAN DNS name for the cluster that resolves to three IP addresses (round-robin). SCAN VIPsmust NOT be in the c:\windows\system32\drivers\etc\hosts file. SCAN VIPs must be resolvable by DNS.

6. Even if you are using a DNS, Oracle recommends that you list the public IP, VIP and private addresses foreach node in the hosts file on each node. Configure thec:\windows\system32\drivers\etc\hosts file so that it is similar to the following example:

NOTE: The SCAN VIP MUST NOT be in the hosts file. This will result in only 1 SCAN VIP for the entirecluster.

#PublicLAN - PUBLIC192.0.2.100 racnode1.example.com racnode1192.0.2.101 racnode2.example.com racnode2#VIP192.0.2.102 racnode1-vip.example.com racnode1-vip192.0.2.103 racnode2-vip.example.com racnode2-vip#PrivateLAN - PRIVATE172.0.2.100 racnode1-priv172.0.2.101 racnode2-priv

After you have completed the installation process, configure clients to use the SCAN to access the cluster.Using the previous example, the clients would use docrac-scan to connect to the cluster.

The fully qualified SCAN for the cluster defaults to cluster_name-scan.GNS_subdomain_name, for exampledocrac-scan.example.com. The short SCANfor the cluster is docrac-scan. You can use any name for the SCAN, as long as it is unique within yournetwork and conforms to the RFC 952 standard.

2.2.1. Network Ping Tests

There are a series of 'ping' tests that should be completed, and then the network adapter binding order shouldbe checked. You should ensure that the public IP addresses resolve correctly and that the private addresses areof the form 'nodename-priv' and resolve on both nodes via the hosts file.

* Public Ping test Pinging Node1 from Node1 should return Node1's public IP address Pinging Node2 from Node1 should return Node2's public IP address Pinging Node1 from Node2 should return Node1's public IP address Pinging Node2 from Node2 should return Node2's public IP address * Private Ping test Pinging Node1 private from Node1 should return Node1's private IP address Pinging Node2 private from Node1 should return Node2's private IP address Pinging Node1 private from Node2 should return Node1's private IP address Pinging Node2 private from Node2 should return Node2's private IP address * VIP Ping test Pinging the VIP address at this point should fail. VIPs will be activated at the end of the Oracle Clusterware install.

If any of the above tests fail you should fix name/address resolution by updating the DNS or local hosts fileson each node before continuing with the installation.

2.2. Networking 6

2.2.2. Network Interface Binding Order (and Protocol Priorities)

It is required that the Public interface be listed first in the network interface binding order on ALL clusternodes.

For Windows 2008:

Perform the follow tasks to ensure this requirement is met:

Click Start, click Run, type "ncpa.cpl", and then click OK.• In the menu bar on the top of the window click "Advanced" and choose "Advanced Settings" (ForWindows 2008, if the "Advanced" is not showing, click 'Alt' to enable that menu item).

•

Under the Adapters and Bindings tab use the up arrow to move the Public interface to the top of theConnections list.

•

Under Binding order for increase the priority of IPv4 over IPv6• Click OK to save the changes• Repeat the previous 5 steps on ALL cluster nodes•

2.2.3. Disable DHCP Media Sense

Media Sense should be disabled. Media Sense allows Windows to uncouple an IP address from a card whenthe link to the local switch is lost. You should disable this activity using the registry editor regedit.Navigate to the Key HKLM\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters andright click to create a new key of type DWORD. make sure that the Key is called DisableDHCPMediaSense,is of type DWORD and has a value of 1. For Windows 2008 we can check the status of DHCP Media Sensewith the command: netsh interface ipv4 show global

2.2.4. Disable SNP Features

On Windows 2003 SP2 and later platforms there are several network issues related to SNP features. Theseissues are described in detail in Microsoft KB article 948496 and 951037. Perform the following tasks to takeproactive action on these potential issues:

Click Start, click Run, type "ncpa.cpl", and then click OK.• Right-click a network adapter object, and then click Properties.• Click Configure, and then click the Advanced tab.• In the Property list, click Receive Side Scaling, click Disable in the Value list, and then click OK.• In the Property list, click TCP/IP Offload, click Disable in the Value list, and then click OK.• Repeat steps 2 through 5 for each network adapter object.•

The same can be accomplished on Windows 2008 by issuing the following commands:

netsh int tcp set global chimney=disabled and netsh int tcp set globalrss=disabled

Validate these changes with the command: netsh interface ipv4 show global

2.3. Stopping Services

There can be issues with some (non-Oracle) services, which may already be running on the cluster nodes.Typically a Microsoft Service: Distributed Transaction Coordinator (MSDTC) can interact with Oraclesoftware during install. It is recommended that this service is stopped and set to 'manual' start usingservices.msc on both nodes.

2.2.2. Network Interface Binding Order (and Protocol Priorities) 7

2.4. Synchronizing the Time on ALL Nodes

There is a general requirement for Oracle RAC that the time on all the nodes be the same. With 11gR2 timesynchronization can be performed by the Clusterware using CTSSD (Cluster Time Synchronization ServicesDaemon) or by using the Windows Time Service. If the Windows Time Service is being used, it MUST beconfigured to prevent the time from being adjusted backwards. Perform the following steps to ensure the timeis NOT adjusted backwards using Windows Time Service:

Open a command prompt and type "regedit"• Within the registry editor locate theHKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W32Time\Config key.

•

Set the value for MaxNegPhaseCorrection? to 0 and exit the registry editor.• Open a command prompt and execute the following to put the change into effect:•

cmd> W32tm /config /update

Repeat steps 1 through 4 for ALL cluster nodes.•

2.5. Environment Variables

Set the TEMP and TMP environment variables to a common location that exists on ALL nodes in the cluster.During installation the Oracle Universal Installer (OUI) will utilize these directories to store temporary copiesof the binaries. If the location is not the same for both variables on ALL cluster nodes the installation will fail.Most commonly these parameters are set as follows:

TMP=C:\temp TEMP=C:\temp For Windows 2008:

To set the TEMP and TMP environment variables:

Log into the server as the user that will perform the installation• Open Computer Properties• Click the Advanced system settings link (on the left under tasks)• Under the Advanced tab, click the Environment Variables button• Modify the TEMP and TMP variables under "User variables for Administrator" to the desired setting.Keep in mind, this path must be identical for both TMP and TEMP and they must be set to the samelocation on ALL cluster nodes.

•

Click OK to save the changes.• Repeat steps 1 through 6 for ALL cluster nodes.•

2.6. Stage the Oracle Software

It is recommended that you stage the required software onto a local drive on Node 1 of your cluster.Important. Ensure that you use only 32 bit versions of the Oracle Software on a 32bit OS and 64 bit versionsof the Oracle Software on a 64bit OS.

For the Grid Infrastructure (clusterware and ASM) software download:

Oracle Database 11g Release 2 Grid Infrastructure (11.2.0.1.0) for Windows

For the RDBMS software download from OTN:

Oracle Database 11g Release 2 (11.2.0.1.0) for Windows

2.4. Synchronizing the Time on ALL Nodes 8

http://globaldc.oracle.com/perl/twiki/edit/RACGuides/MaxNegPhaseCorrection?topicparent=RACGuides.Rac11gR2OnWindows

2.7. Cluster Verification Utility (CVU) stage check

Now you can run the CVU to check the state of the cluster prior to the install of the Oracle Software. Check ifthere is a newer version of CVU available on otn compared to the one that ships on the installation mediahttp://otn.oracle.com/rac

3. Prepare the shared storage for Oracle RAC

This section describes how to prepare the shared storage for Oracle RAC

Shared Disk Layout1. Enable Automounting of disks on Windows2. Clean the Shared Disks3. Create Logical partitions inside Extended partitions4. Drive Letters5. View Disks6. Marking Disk Partitions for use by ASM7. Verify Clusterware Installation Readiness8.

3.1. Shared Disk Layout

It is assumed that the two nodes have local disk primarily for the operating system and the local OracleHomes. Labelled C: The Oracle Grid Infrastructure software also resides on the local disks on each node. The2 nodes must also share some central disks. This disk must not have cache enabled at the node level. i.e. if theHBA drivers support caching of reads/writes it should be disabled. If the SAN supports caching that is visibleto all nodes then this can be enabled.

3.1.1. Grid Infrastructure Shared Storage

With Oracle 11gR2 it is considered a best practice to store the OCR and Voting Disk within ASM and tomaintain the ASM best practice of having no more than 2 diskgroups (Flash Recovery Area and DatabaseArea). This means that the OCR and Voting disk will be stored along with the database related files. If you areutilizing external redundancy for your disk groups this means you will have 1 Voting Disk and 1 OCR.

For those who wish to utilize Oracle supplied redundancy for the OCR and Voting disks you could create aseparate (3rd) ASM Diskgroup having a minimum of 2 fail groups (total of 3 disks). This configuration willprovide 3 multiplexed copies of the Voting Disk and a single OCR which takes on the redundancy of that diskgroup (mirrored within ASM). The minimum size of the 3 disks that make up this diskgroup is 1GB. Thisdiskgroup will also be used to store the ASM SPFILE.

For demonstration purposes within this cookbook, we will be using the more complex of the aboveconfigurations by creating a 3rd diskgroup for storage of the OCR and Voting Disks. Our third disk group willbe normal redundancy allowing for 3 Voting Disks and a single OCR which takes on the redundancy of thatdiskgroup.

Disk Number Volume Size(MB) ASM Label Prefix Diskgroup RedundancyDisk 1 Volume 2 1024 OCR_VOTE OCR_VOTE NormalDisk 2 Volume 3 1024 OCR_VOTE OCR_VOTE NormalDisk 3 Volume 4 1024 OCR_VOTE OCR_VOTE Normal

2.7. Cluster Verification Utility (CVU) stage check 9

http://otn.oracle.com/rac

3.1.2. ASM Shared Storage

It is recommended that ALL ASM disks within a disk group are of the same size and carry the sameperformance characteristics. Whenever possible Oracle also recommends sticking to the SAME (Stripe AndMirror Everything) methodology by using RAID 1+0. If SAN level redundancy is available, externalredundancy should be used for database storage on ASM.

Number of LUNs(Disks) RAID Level Size(GB) ASM Label Prefix Diskgroup Redundancy4 1+0 100 DBDATA DBDATA External4 1+0 100 DBFLASH DBFLASH External

In this document we will use the diskpart command line tool to manage these LUNs. You must create logicaldrives inside of extended partitions for the disks to be used by Oracle Grid Infrastructure and Oracle ASM.There must be no drive letters assigned to any of the Disks1 – Disk10 on any node. For MIcrosoft Windows2003 it is possible to use diskmgmt.msc instead of diskpart (as used in the following sections) to create thesepartitions. For Microsoft Windows 2008, diskmgmt.msc cannot be used instead of diskpart to create thesepartitions.

3.2. Enable Automount

You must enable automounting of disks for them to be visible to Oracle Grid Infrastructure. On each node login as someone with Administrator privileges then Click START->RUN and type diskpart

C:\>diskpart

Microsoft DiskPart version 5.2.3790.3959Copyright (C) 1999-2001 Microsoft Corporation.On computer: WINNODE1

DISKPART>AUTOMOUNT ENABLERepeat the above command on all nodes in the cluster

3.3. Clean the Shared Disks

You may want to clean your shared disks before starting the install. Cleaning will remove data from anyprevious failed install. But see a later Appendix for coping with failed installs. On Node1 from within diskpartyou should clean each of the disks. WARNING this will destroy all of the data on the disk. Do not selectthe disk containing the operating system or you will have to reinstall the OSCleaning the disk ‘scrubs’ every block on the disk. This may take some time to complete.

DISKPART> list disk

Disk ### Status Size Free Dyn Gpt -------- ---------- ------- ------- --- --- Disk 0 Online 34 GB 0 B Disk 1 Online 1024 MB 1024 MB Disk 2 Online 1024 MB 1024 MB Disk 3 Online 1024 MB 1024 MB Disk 4 Online 1024 MB 1024 MB Disk 5 Online 1024 MB 1024 MB Disk 6 Online 100 GB 100 GB Disk 7 Online 100 GB 100 GB Disk 8 Online 100 GB 100 GB Disk 9 Online 100 GB 100 GB

3.1.2. ASM Shared Storage 10

Disk 10 Online 100 GB 100 GBNow you should clean disks 1 – 10 (Not disk0 as this the local C: drive)DISKPART>select disk 1Disk 1 is now the selected disk.DISKPART> clean all

DISKPART> select disk 2Disk 2 is now the selected disk.DISKPART> clean all









3.4. Create Logical partitions inside Extended partitions

Assuming the disks you are going to use are completely empty you must create an extended partition and theninside that partition a logical partition. In the following example, for Oracle Grid Infrastructure, I havededicated LUNS for each device.

DISKPART> select disk 1Disk 1 is now the selected disk.DISKPART> create part extDiskPart succeeded in creating the specified partition.DISKPART> create part logDiskPart succeeded in creating the specified partition.

3.3. Clean the Shared Disks 11


DISKPART> select disk 3Disk 3 is now the selected disk.DISKPART> create part extDiskPart succeeded in creating the specified partition.DISKPART> create part logDiskPart succeeded in creating the specified partition..






DISKPART> select disk 9Disk 9 is now the selected disk.DISKPART> create part extDiskPart succeeded in creating the specified partition.DISKPART> create part log

3.4. Create Logical partitions inside Extended partitions 12

DiskPart succeeded in creating the specified partition.


3.4.1. View Created partitions

DISKPART> list disk

Disk ### Status Size Free Dyn Gpt -------- ---------- ------- ------- --- --- Disk 0 Online 34 GB 0 B Disk 1 Online 1024 MB 0 MB Disk 2 Online 1024 MB 0 MB Disk 3 Online 1024 MB 0 MB Disk 4 Online 1024 MB 0 MB Disk 5 Online 1024 MB 0 MB Disk 6 Online 100 GB 0 MB Disk 7 Online 100 GB 0 MB Disk 8 Online 100 GB 0 MB Disk 9 Online 100 GB 0 MB Disk 10 Online 100 GB 0 MB3.5. List Drive Letters

Diskpart should not add drive letters to the partitions on the local node. The partitions on the other node mayhave drive letters assigned. You must remove them. On earlier versions of Windows 2003 a reboot of the‘other’ node will be required for the new partitions to become visible. Windows 2003 SP2 and Windows 2008do not suffer from this issue.

Using diskpart on Node2

DISKPART> list volume

Volume ### Ltr Label Fs Type Size Status Info ---------- --- ----------- ----- ---------- ------- --------- -------- Volume 0 C NTFS Partition 16 GB Healthy System Volume 1 D RAW Partition 1023 MB Healthy Volume 2 E RAW Partition 1023 MB Healthy Volume 3 F RAW Partition 1023 MB Healthy Volume 4 G RAW Partition 1023 MB Healthy Volume 5 H RAW Partition 1023 MB Healthy Volume 6 I RAW Partition 100 GB Healthy Volume 7 J RAW Partition 100 GB Healthy Volume 8 K RAW Partition 100 GB Healthy Volume 9 L RAW Partition 100 GB Healthy Volume 10 M RAW Partition 100 GB HealthyNotice that the volumes are listed in a completely different order compared to the disk list.

3.5.1. Remove Drive Letters

You need to remove the drive letters D E F G H I J K L M These relate to volumes 1 2 3 4 5 6 7 8 9 10 *DoNOT remove drive letter C which, in this case, is the local disk. This relates to volume 0 (In this example)

3.4.1. View Created partitions 13

DISKPART> select volume 1Volume 1 is the selected volume.DISKPART> removDiskPart successfully removed the drive letter or mount point.










3.5.1. Remove Drive Letters 14

3.5.2. List volumes on Second node

You should check that none of the RAW partitions have drive letters assigned

DISKPART> list vol

Volume ### Ltr Label Fs Type Size Status Info ---------- --- ----------- ----- ---------- ------- --------- -------- Volume 0 C NTFS Partition 16 GB Healthy System Volume 1 RAW Partition 1023 MB Healthy Volume 2 RAW Partition 1023 MB Healthy Volume 3 RAW Partition 1023 MB Healthy Volume 4 RAW Partition 1023 MB Healthy Volume 5 RAW Partition 1023 MB Healthy Volume 6 RAW Partition 100 GB Healthy Volume 7 RAW Partition 100 GB Healthy Volume 8 RAW Partition 100 GB Healthy Volume 9 RAW Partition 100 GB Healthy Volume 10 RAW Partition 100 GB HealthyYou can now exit diskpart on all nodes

3.6. Marking Disk Partitions for use by ASM

The only partitions that the Oracle Universal Installer acknowledges on Windows systems are logical drivesthat are created on top of extended partitions and that have been stamped as candidate ASM disks. Thereforeprior to running the OUI the disks that are to be used by Oracle RAC MUST be stamped using ASM Tool.ASM Tool is available in two different flavors, command line (asmtool) and graphical (asmtoolg). Bothutilities can be found under the asmtool directory within the Grid Infrastructure installation media. For thisinstallation, asmtoolg will be used to stamp the ASM disks. The following table outlines the summary of thedisks that will be stamped for ASM usage:

Number of LUNs(Disks) RAID Level Size(GB) ASM Label Prefix Diskgroup Redundancy3 1+0 1024 OCR_VOTE OCR_VOTE Normal4 1+0 100 DBDATA DBDATA External4 1+0 100 DBFLASH DBFLASH External

Perform this task as follows:

Within Windows Explorer navigate to the asmtool directory within the Grid Infrastructure installationmedia and double click the â–�asmtoolg.exeâ–� executable.

•

Within ASM Tool GUI, select â–�Add or Change Labelâ–� and click â–�Nextâ–�.•

3.5.2. List volumes on Second node 15

On the Select Disks screen choose the appropriate disks to be assigned a label and enter an ASMLabel Prefix to make the disks easily identifiable for their intended purpose, size and/or performancecharacteristics. After choosing the intended disks and entering the appropriate ASM Label Prefix,click â–�Nextâ–� to continue.

•

Review the summary screen and click â–�Nextâ–�.•

3.6. Marking Disk Partitions for use by ASM 16

On the final screen, click â–�Finishâ–� to update the ASM Disk Labels.•

Repeat these steps for all ASM disks that will differ in their label prefix.•

3.7. Verify Grid Infrastructure Installation Readiness

Prior to installing Grid Infrastructure it is highly recommended to run the cluster verification utility(CLUVFY) to verify that the cluster nodes have been properly configured for a successful Oracle GridInfrastructure installation. There are various levels at which CLUVFY can be run, at this stage it should be runin the CRS pre-installation mode. Later in this document CLUVFY will be run in pre dbinst mode to validatethe readiness for the RDBMS software installation.

3.7. Verify Grid Infrastructure Installation Readiness 17

Though CLUVFY is packaged with the Grid Infrastructure installation media, it is recommended to downloadand run the latest version of CLUVFY. The latest version of the CLUVFY utility can be downloaded from:


Once the latest version of the CLUVFY has been downloaded and installed, execute it as follows to performthe Grid Infrastructure pre-installation verification:

Login to the server in which the installation will be performed as the Local Administrator.• Open a command prompt and run CLUVFY as follows to perform the Oracle Clusterwarepre-installation verification:

•

cmd> runcluvfy stage -post hwos -n [â–�verbose]

cmd> runcluvfy stage -pre crsinst -n [-verbose]

If any errors are encountered, these issues should be investigated and resolved before proceeding with theinstallation.

4. Oracle Grid Infrastructure Install

4.1. Basic Grid Infrastructure Install (without GNS and IPMI)

Shutdown all Oracle Processes running on all nodes (not necessary if performing the install on newservers)

•

Start the Oracle Universal Installer (OUI) by running setup.exe as the Local Administrator user fromthe Clusterware (db directory if using the DVD Media, see step 3) directory on the 11g Release 2(11.2.0.1) installation media.

•

If using the DVD installation media (from edelivery.oracle.com) the first screen to appear will be theSelect a Product to Install screen. Choose Oracle Clusterware 11g and click next to continue. If theOTN media is used the RDBMS/ASM and Clusterware are separate downloads and the first screen tobe displayed will be the Welcome screen .

•

On the Select Installation Option Screen, choose â–�Install and Configure Grid Infrastructure for aClusterâ–� and click â–�Nextâ–� to continue.

•

4. Oracle Grid Infrastructure Install 18


* On the Select Installation Type screen, choose â–�Advanced Installationâ–� and click â–�Nextâ–� to continue.

Choose the appropriate language on the Select Product Languages screen and click â–�Nextâ–� tocontinue.

•

4.1. Basic Grid Infrastructure Install (without GNS and IPMI) 19

On the Grid Plug and Play Information screen perform the following:

Enter the desired Cluster Name, this name should be unique for the enterprise and CAN NOTbe changed post installation.

♦

Enter the SCAN Name for the cluster. The SCAN Name must be a DNS entry resolving to 3IP addresses. The SCAN Name MUST NOT be in the hosts file.

♦

Enter the port number for the SCAN Listener, this port defaults to 1521.♦ Uncheck the â–�Configure GNSâ–� checkbox.♦ Click â–�Nextâ–� to continue.♦

•

Add all of the cluster nodes hostnames and Virtual IP hostnames on the Cluster Node InformationScreen. By default the OUI only knows about the local node, additional nodes can be added using theâ–�Addâ–� button.

•


On the Specify Network Interface Usage screen, make sure that the public and private interfaces areproperly specified. Make the appropriate corrections on this screen and click â–�Nextâ–� to continue.

•

NOTEThe public and private interface names MUST be consistent across the cluster, so ifâ–�Publicâ–� is public and â–�Privateâ–� is private, the same MUST be true on all of the othernodes in the cluster.

Choose â–�Automatic Storage Management (ASM) on the Storage Option Information screen andclick â–�Nextâ–� to create the ASM diskgroup for Grid Infrastructure.

•


On the Create ASM Disk Group screen perform the following:•

Enter the Disk Group Name that will store the OCR and Voting Disks (e.g. OCR_VOTE)♦ Choose â–�Normalâ–� for the Redundancy level.♦ Select the disks that were previously designated for the OCR and Voting Disks.♦

•

NOTEIf no Candidate disks are available the disks have not yet been stamped for use byASM. To stamp the disks review the instructions in the Storage Prerequisites sectionof this document and click the â–�Stamp Disksâ–� button on this screen.

Click â–�Nextâ–� to continue.•


Enter the appropriate passwords for the SYS and ASMSNMP users of ASM on the Specify ASMPasswords screen and click â–�Nextâ–� to continue.

•

On the Failure Isolation Support screen choose â–�Do not use Intelligent Platform ManagementInterface (IPMI)â–� and click â–�Nextâ–� to continue.

•

Specify the location for Oracle Base (e.g. d:\app\oracle) and the Grid Infrastructure Installation (e.g.d:\OraGrid) on the Specify Installation Location screen. Click â–�Nextâ–� to allow the OUI to performthe prerequisite checks on the target nodes.

•

NOTEContinuing with failed prerequisites may result in a failed installation; therefore it isrecommended that failed prerequisite checks be resolved before continuing.


After the prerequisite checks have successfully completed, review the summary of the pendinginstallation and click â–�Finishâ–� to install and configure Grid Infrastructure.

•


On the Finish screen click â–�Closeâ–� to exit the OUI.•

Once the installation has completed, check the status of the CRS resources as follows:•

NOTEAll resources should report as online with the exception of GSD and OC4J? . GSD will onlybe online if Grid Infrastructure is managing a 9i database and OC4J? is reserved for use in afuture release. Though these resources are offline it is NOT supported to remove them.

cmd> %GI_HOME%\bin\crsctl stat res -tThe output of the above command will be similar to the following:

--------------------------------------------------------------------------------

NAME TARGET STATE SERVER STATE_DETAILS


http://globaldc.oracle.com/perl/twiki/edit/RACGuides/OC4J?topicparent=RACGuides.Rac11gR2OnWindows

http://globaldc.oracle.com/perl/twiki/edit/RACGuides/OC4J?topicparent=RACGuides.Rac11gR2OnWindows

--------------------------------------------------------------------------------

Local Resources--------------------------------------------------------------------------------

ora.LISTENER.lsnr ONLINE ONLINE ratwin01

ONLINE ONLINE ratwin02

ora.OCR_VOTE.dg ONLINE ONLINE ratwin01


ora.asm ONLINE ONLINE ratwin01 Started

ONLINE ONLINE ratwin02 Started

ora.eons ONLINE ONLINE ratwin01


ora.gsd OFFLINE OFFLINE ratwin01

OFFLINE OFFLINE ratwin02

ora.net1.network ONLINE ONLINE ratwin01


ora.ons ONLINE ONLINE ratwin01


--------------------------------------------------------------------------------

Cluster Resources--------------------------------------------------------------------------------

ora.LISTENER_SCAN1.lsnr 1 ONLINE ONLINE ratwin02



ora.oc4j 1 OFFLINE OFFLINE

ora.ratwin01.vip 1 ONLINE ONLINE ratwin01

ora.ratwin02.vip 1 ONLINE ONLINE ratwin02

ora.scan1.vip 1 ONLINE ONLINE ratwin02




5. Grid Infrastructure Home Patching

This Chapter is a placeholder

6. RDBMS Software Install

Prior to installing the Database Software (RDBMS) it is highly recommended to run the cluster verificationutility (CLUVFY) to verify that Grid Infrastructure has been properly installed and the cluster nodes havebeen properly configured to support the Database Software installation. In order to achieve this CLUVFYmust be run in pre dbinst mode. The following outlines how to achieve this task:

cmd> runcluvfy stage -pre dbinst -n -verbose

Start the OUI by running the setup.exe command as the Local Administrator user from the DBdirectory of the Oracle Database 11g Release 2 (11.2.0.1) installation media and click next to beginthe installation process.

•

If using the DVD installation media (from edelivery.oracle.com) the first screen to appear will be theSelect a Product to Install screen. Choose Oracle Database 11g and click next to continue. If the OTNmedia is used the RDBMS/ASM and Clusterware are separate downloads and the first screen to bedisplayed will be Installation Type screen .

•

The first OUI screen will prompt for an email address to allow you to be notified of security issuesand to enable Oracle Configuration Manager. If you wish NOT to be notified or not to use OracleConfiguration Manager leave the email box blank and uncheck the check box. After entering thedesired information click â–�Nextâ–� to continue.

•

Choose â–�Install database software onlyâ–� on the Select Installation Option screen and clickâ–�Nextâ–� to continue.

•

5. Grid Infrastructure Home Patching 27

On the Grid Installation Options screen, choose â–�Real Application Clusters database installationâ–�and select ALL nodes for the software installation. Click â–�Nextâ–� to continue.

•

Choose the appropriate language on the Select Product Languages screen and click â–�Nextâ–� tocontinue.

•

6. RDBMS Software Install 28

On the Select Database Edition screen choose â–�Enterprise Editionâ–� and click â–�Nextâ–� to continue.•

NOTEIf there is a need for specific database options to be installed or not installed, these optionscan be chosen by clicking the â–�Select Optionsâ–� button.

Specify the location for Oracle Base (e.g. d:\app\oracle) and the database software installation (e.g.d:\app\oracle\product\11.2.0\db_1) on the Specify Installation Location screen. Click â–�Nextâ–� toallow the OUI to perform the prerequisite checks on the target nodes.

•

NOTEIt is recommended that the same Oracle Base location is used for the database installation thatwas used for the Grid Infrastructure installation.


NOTEContinuing with failed prerequisites may result in a failed installation; therefore it isrecommended that failed prerequisite checks be resolved before continuing.

After the prerequisite checks have successfully completed, review the summary of the pendinginstallation and click â–�Finishâ–� to install the database software.

•


On the Finish screen click â–�Closeâ–� to exit the OUI.•

7. RAC Home Patching

This Chapter is a placeholder

8. Run ASMCA to create diskgroups

Prior to creating a database on the cluster, the ASM diskgroups that will house the database must be created.In an earlier chapter, the ASM disks for the database diskgroups were stamped for ASM usage. We will nowuse the ASM Configuration Assistant to create the diskgroups:

7. RAC Home Patching 31

Run the ASM Configuration Assistant (ASMCA) from the ASM Home by executing the following:•

cmd> %GI_HOME%\bin\asmca

After launching ASMCA, click the DiskGroups? tab.•

While on the DiskGroups? tab, click the â–�Createâ–� button to display the Create DiskGroup? window.•

NOTE: To reduce the complexity of managing ASM and its diskgroups, Oracle recommends that generallyno more than two diskgroups be maintained for database storage, a Database Area DiskGroup? and a FlashRecovery Area DiskGroup? . The Database Area Diskgroup will house active database files such as datafiles,control files, online redo logs, and change tracking files (used in incremental backups) are stored. The FlashRecovery Area will house recovery-related files are created, such as multiplexed copies of the current controlfile and online redo logs, archived redo logs, backup sets, and flashback log files

Within the Create DiskGroup? window perform the following:

Enter the desired DiskGroup? Name (e.g. DBDATA or DBFLASH)♦ Choose External Redundancy (assuming redundancy is provided at the SAN level).♦ Select the candidate disks to include in the DiskGroup?♦

•

NOTE: If no Candidate disks are available the disks have not yet been stamped for use by ASM. To stampthe disks review the instructions in the 'Rac11gR2WindowsPrepareDisk' chapter of this document and clickthe â–�Stamp Disksâ–� button on this screen.

8. Run ASMCA to create diskgroups 32

http://globaldc.oracle.com/perl/twiki/edit/RACGuides/DiskGroups?topicparent=RACGuides.Rac11gR2OnWindows


http://globaldc.oracle.com/perl/twiki/edit/RACGuides/DiskGroup?topicparent=RACGuides.Rac11gR2OnWindows






Click â–�OKâ–� to create the DiskGroup? .♦ •

Repeat the previous two steps to create all necessary diskgroups.•

Once all the necessary DiskGroups? have been created click â–�Exitâ–� to exit from ASMCA.•

Note:

It is Oracle's Best Practise to have an OCR mirror stored in a second disk group. To follow thisrecommendation add an OCR mirror. Mind that you can only have one OCR in a diskgroup.

Action:

8. Run ASMCA to create diskgroups 33



1. To add OCR mirror to an Oracle ASM disk group, ensure that the Oracle Clusterware stack is running andrun the following command as administrator:

2. D:\OraGrid\BIN>ocrconfig -add +DBDATA

3. D:\OraGrid\BIN>ocrcheck -config

9. Run DBCA to create the database

To help to verify that the system is prepared to successfully create a RAC database, use the following ClusterVerification Utility command syntax:

cmd> runcluvfy stage -pre dbcfg -n all -d c:\app\11.2.0\grid -verbosePerform the following to create an 11gR2 RAC database on the cluster:

Run the Database Configuration Assistant (DBCA) from the ASM Home by executing the following:•

cmd> %ORACLE_HOME\bin\dbca

On the Welcome screen, select â–�Oracle Real Application Clustersâ–� and click â–�Nextâ–�•

On the Operation screen, select â–�Create Databaseâ–� and click â–�Nextâ–�•

9. Run DBCA to create the database 34

The Database Templates screen will now be displayed. Select the â–�General Purpose or TransactionProcessingâ–� database template and click â–�Nextâ–� to continue.

•

On the Database Identification screen perform the following:

Select the â–�Admin Managedâ–� configuration type.♦ Enter the desired Global Database Name.♦ Enter the desired SID prefix.♦ Select ALL the nodes in the cluster.♦ Click â–�Nextâ–� to continue.♦

•


On the Management Options screen, choose â–�Configure Enterprise Managerâ–�. Once Grid Controlhas been installed on the system this option may be unselected to allow the database to be managed byGrid Control. Click â–�Nextâ–� to continue.

•

Enter the appropriate database credentials for the default user accounts and click â–�Nextâ–� tocontinue.

•


On the Database File Locations screen perform the following:

Choose Automatic Storage Management (ASM).♦ Select â–�Use Oracle-Managed Filesâ–� and specify the ASM Disk Group that will house thedatabase files (e.g. +DBDATA)

♦

Click â–�Nextâ–� and enter the ASMSNMP password when prompted.♦ Click â–�OKâ–� after entering the ASMSNMP password to continue.♦

•


For the Recovery Configuration, select â–�Specify Flash Recovery Areaâ–�, enter +DBFLASH for thelocation and choose an appropriate size. It is also recommended to select â–�Enable Archivingâ–� at thispoint. Click â–�Nextâ–� to continue.

•

On the Database Content screen, you are able create the sample schemas. Check the check box if thesample schemas are to be loaded into the database and click â–�Nextâ–� to continue.

•


Enter the desired memory configuration and initialization parameters on the Initialization Parametersscreen and click â–�Nextâ–� to continue.

•

On the Database Storage screen, review the file layout and click â–�Nextâ–� to continue.•

NOTE: At this point you may want to increase the number of redo logs per thread to 3 and increase the sizeof the logs from the default of 50MB.


On the Last screen, ensure that â–�Create Databaseâ–� is checked and click â–�Finishâ–� to review thesummary of the pending database creation.

•

Click â–�OKâ–� on the Summary window to create the database.•


rac 11g on windows

Documents