rac checklist 11gr2 grid infrastructure for clusters

11gR2 Grid Infrastructure for Clusters

Please note that this cheat sheet is not meant to substitute the official Oracle 11gR2 documentation. If you need more information about 11gR2 you should consult the Oracle 11gR2 documentation.

Table of Contents1)Setup system related tasks............................................................................................................ 2

Create GI group and user etc.......................................................................................................2Setup swap and system limits etc................................................................................................2Setup /etc/hosts for Single Client Access Name (SCAN)...........................................................2Create Grid Infrastructure directories etc....................................................................................3Setup SSH................................................................................................................................... 3Create ASM candidate disks etc..................................................................................................5

2)Prepare for a Grid Infrastructure for Clusters installation............................................................7Install ORCLudlm (SPARC only)...............................................................................................7Install RAC framework resources...............................................................................................7Setup VNC.................................................................................................................................. 7

3)Install Grid Infrastructure for Clusters via a VNC client............................................................. 8Install and configure Grid Infrastructure for Clusters................................................................. 8Display the Grid Infrastructure resources................................................................................. 17Create another ASM disk group................................................................................................18

4)Install and create an 11gR1 database via a VNC client .............................................................19Install the 11gR1 database software..........................................................................................19Create the 11gR1 database using dbca...................................................................................... 21Create the 11gR1 database listener using netca........................................................................ 22Copy some 11gR1 database files to the another node............................................................... 24Verify the 11gR1 database installation......................................................................................26Check the Grid Infrastructure resources................................................................................... 28

5)Install and create an 11gR2 database via a VNC client .............................................................29Install the 11gR2 database software..........................................................................................29Create the 11gR2 database using dbca...................................................................................... 30Copy some 11gR2 database files to the another node............................................................... 31Verify the 11gR2 database installation......................................................................................33Check the Grid Infrastructure resources................................................................................... 34

6)Using SCAN for the 11gR2 single instance database................................................................ 357)Place everything under Oracle Solaris Cluster ......................................................................... 36

Register the resource types........................................................................................................36Register and enable the clustered ASM instance resource........................................................37Register and enable the clustered ASM disk group resource....................................................38Register and enable the 11gR1 database and listener resources .............................................. 38Switch the 11gR1 Resource Group........................................................................................... 40Register and enable the 11gR2 database resource ....................................................................41

Switch the 11gR2 Resource Group...................................................................................... 44Final picture ............................................................................................................................. 47

1

1) Setup system related tasks

Create GI group and user etc.

bash-3.00# vi /etc/nsswitch.conf (remove nis from passwd: and group:)

bash-3.00# groupadd -g 300 oinstallbash-3.00# groupadd -g 301 dba bash-3.00# useradd -g 300 -G 301 -u 300 -d /export/home/oragrid -s /usr/bin/bash oragrid

bash-3.00# mkdir -p /export/home/oragridbash-3.00# chown oragrid:oinstall /export/home/oragrid

bash-3.00# passwd oragridNew Password: Re-enter new Password: passwd: password successfully changed for oragridbash-3.00#

Setup swap and system limits etc.

bash-3.00# mkfile 8g /swapfilebash-3.00# swap -a /swapfile

Add to /etc/vfstab

bash-3.00# cat /etc/vfstab | grep swapfile/swapfile - - swap - no -bash-3.00#

Copy the following entries into /etc/project

bash-3.00# cat /etc/projectsystem:0::::process.max-sem-nsems=(privileged,2048,deny);project.max-shm-memory=(privileged,2199023255552,deny)user.root:1::::noproject:2::::default:3::::process.max-file-descriptor=(priv,65536,deny);project.max-shm-memory=(priv,4294967296,deny);project.max-user-process=(basic,16384,deny)group.staff:10::::bash-3.00#

Setup /etc/hosts for Single Client Access Name (SCAN)

Here, freak-scan-lh will be used as the SCAN name and freak-1 and freak-2 are VIPs for each cluster node that SCAN will use. You will need to determine which IP addresses are free to use for your cluster. bash-3.00# cat /etc/hosts## Internet host table

2

#::1 localhost127.0.0.1 localhost10.11.188.56 pfreak1 loghost10.11.188.57 pfreak2 # Cluster Node10.11.188.60 vzfreak1a freak-scan-lh10.11.188.58 freak-1 10.11.188.59 freak-2bash-3.00#

Create Grid Infrastructure directories etc.

bash-3.00# mkdir -p /u01/app/oragridbash-3.00# mkdir -p /u01/app/oraInventorybash-3.00# mkdir -p /u01/app/11.2.0/grid

bash-3.00# chown -R oragrid:oinstall /u01bash-3.00# chmod -R 775 /u01

Setup SSH.

On pfreak1:

bash-3.00# su - oragridSun Microsystems Inc. SunOS 5.10 Generic January 2005-bash-3.00$ mkdir .ssh-bash-3.00$ chmod 700 .ssh-bash-3.00$ cd .ssh

On pfreak2:

bash-3.00# su - oragridSun Microsystems Inc. SunOS 5.10 Generic January 2005-bash-3.00$ mkdir .ssh-bash-3.00$ chmod 700 .ssh-bash-3.00$ cd .ssh

On pfreak1:

Note: we will also NOT specify a passphrase.

-bash-3.00$ ssh-keygen -t rsaGenerating public/private rsa key pair.Enter file in which to save the key (/export/home/oragrid/.ssh/id_rsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /export/home/oragrid/.ssh/id_rsa.Your public key has been saved in /export/home/oragrid/.ssh/id_rsa.pub.The key fingerprint is:e6:63:c9:71:fe:d1:8f:71:77:70:97:25:2a:ee:a9:33 [email protected]$-bash-3.00$ pwd/export/home/oragrid/.ssh

3

-bash-3.00$ cat id_rsa.pub >> authorized_keys-bash-3.00$ chmod 600 authorized_keys-bash-3.00$ scp authorized_keys pfreak2:/export/home/oragrid/.sshThe authenticity of host 'pfreak2 (10.11.188.57)' can't be established.RSA key fingerprint is 4d:d3:84:72:44:fe:0c:48:a7:76:fb:7d:65:c4:da:a9.Are you sure you want to continue connecting (yes/no)? yesWarning: Permanently added 'pfreak2,10.11.188.57' (RSA) to the list of known hosts.Password: authorized_keys 100% |*****************************************************************************************| 223 00:00 -bash-3.00$

On pfreak2:

Note: we will also NOT specify a passphrase.

-bash-3.00$ ssh-keygen -t rsaGenerating public/private rsa key pair.Enter file in which to save the key (/export/home/oragrid/.ssh/id_rsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /export/home/oragrid/.ssh/id_rsa.Your public key has been saved in /export/home/oragrid/.ssh/id_rsa.pub.The key fingerprint is:be:8f:e6:14:ff:ff:06:b7:c7:64:1b:44:3b:d3:5a:e8 [email protected]$-bash-3.00$ pwd/export/home/oragrid/.ssh-bash-3.00$ cat id_rsa.pub >> authorized_keys-bash-3.00$ chmod 600 authorized_keys-bash-3.00$ scp authorized_keys pfreak1:/export/home/oragrid/.sshThe authenticity of host 'pfreak1 (10.11.188.56)' can't be established.RSA key fingerprint is b8:46:db:d0:88:ed:5c:33:b5:b0:d0:f1:d6:be:a0:38.Are you sure you want to continue connecting (yes/no)? yesWarning: Permanently added 'pfreak1,10.11.188.56' (RSA) to the list of known hosts.Password: authorized_keys 100% |*****************************************************************************************| 446 00:00 -bash-3.00$

Test ssh has been setup correctly.

On pfreak1:

-bash-3.00$ ssh pfreak1 dateThe authenticity of host 'pfreak1 (10.11.188.56)' can't be established.RSA key fingerprint is b8:46:db:d0:88:ed:5c:33:b5:b0:d0:f1:d6:be:a0:38.

4

Are you sure you want to continue connecting (yes/no)? yesWarning: Permanently added 'pfreak1,10.11.188.56' (RSA) to the list of known hosts.Fri May 14 07:30:55 PDT 2010-bash-3.00$ -bash-3.00$ ssh pfreak1 dateFri May 14 07:30:57 PDT 2010-bash-3.00$ ssh pfreak2 dateFri May 14 07:31:02 PDT 2010-bash-3.00$

On pfreak2:

-bash-3.00$ ssh pfreak2 dateThe authenticity of host 'pfreak2 (10.11.188.56)' can't be established.RSA key fingerprint is dc:6e:af:36:f8:c7:74:b1:c6:b1:1c:96:e3:da:11:51.Are you sure you want to continue connecting (yes/no)? yesWarning: Permanently added 'pfreak2,10.11.188.56' (RSA) to the list of known hosts.Fri May 14 07:33:02 PDT 2010-bash-3.00$ -bash-3.00$ ssh pfreak2 dateFri May 14 07:33:04 PDT 2010-bash-3.00$ ssh pfreak1 dateFri May 14 07:33:08 PDT 2010-bash-3.00$

Create ASM candidate disks etc.

bash-3.00# cldev list -vDID Device Full Device Path---------- ----------------d1 pfreak1:/dev/rdsk/c3t500000E0183E1140d0d2 pfreak1:/dev/rdsk/c3t600C0FF0000000000928FF1E6623E60Fd0d2 pfreak2:/dev/rdsk/c3t600C0FF0000000000928FF1E6623E60Fd0d3 pfreak1:/dev/rdsk/c3t600C0FF0000000000928FF1E6623E60Ed0d3 pfreak2:/dev/rdsk/c3t600C0FF0000000000928FF1E6623E60Ed0d4 pfreak1:/dev/rdsk/c3t600C0FF0000000000928FF1E6623E60Dd0d4 pfreak2:/dev/rdsk/c3t600C0FF0000000000928FF1E6623E60Dd0d5 pfreak1:/dev/rdsk/c3t600C0FF0000000000928FF1E6623E60Cd0d5 pfreak2:/dev/rdsk/c3t600C0FF0000000000928FF1E6623E60Cd0d6 pfreak1:/dev/rdsk/c3t600C0FF0000000000928FF1E6623E60Bd0d6 pfreak2:/dev/rdsk/c3t600C0FF0000000000928FF1E6623E60Bd0d7 pfreak1:/dev/rdsk/c3t600C0FF0000000000928FF1E6623E60Ad0d7 pfreak2:/dev/rdsk/c3t600C0FF0000000000928FF1E6623E60Ad0d8 pfreak1:/dev/rdsk/c3t600C0FF0000000000928FF1E6623E609d0d8 pfreak2:/dev/rdsk/c3t600C0FF0000000000928FF1E6623E609d0d9 pfreak1:/dev/rdsk/c3t600C0FF0000000000928FF1E6623E608d0d9 pfreak2:/dev/rdsk/c3t600C0FF0000000000928FF1E6623E608d0d10 pfreak2:/dev/rdsk/c3t500000E016AE6B70d0bash-3.00# bash-3.00# for i in 2 3 4 5 6 7 8 9

5

> do> dd if=/dev/zero of=/dev/did/rdsk/d${i}s0 bs=1025k count=200> chown oragrid:oinstall /dev/did/rdsk/d${i}s0> chmod 660 /dev/did/rdsk/d${i}s0> ls -lHL /dev/did/rdsk/d${i}s0> done200+0 records in200+0 records outcrw-rw---- 1 oragrid oinstall 239, 64 May 17 03:23 /dev/did/rdsk/d2s0200+0 records in200+0 records outcrw-rw---- 1 oragrid oinstall 239, 96 May 17 03:23 /dev/did/rdsk/d3s0200+0 records in200+0 records outcrw-rw---- 1 oragrid oinstall 239, 128 May 17 03:23 /dev/did/rdsk/d4s0200+0 records in200+0 records outcrw-rw---- 1 oragrid oinstall 239, 160 May 17 03:23 /dev/did/rdsk/d5s0200+0 records in200+0 records outcrw-rw---- 1 oragrid oinstall 239, 192 May 17 03:23 /dev/did/rdsk/d6s0200+0 records in200+0 records outcrw-rw---- 1 oragrid oinstall 239, 224 May 17 03:23 /dev/did/rdsk/d7s0200+0 records in200+0 records outcrw-rw---- 1 oragrid oinstall 239, 256 May 17 03:23 /dev/did/rdsk/d8s0200+0 records in200+0 records outcrw-rw---- 1 oragrid oinstall 239, 288 May 17 03:23 /dev/did/rdsk/d9s0bash-3.00#

On pfreak2.

bash-3.00# for i in 2 3 4 5 6 7 8 9> do> chown oragrid:oinstall /dev/did/rdsk/d${i}s0> chmod 660 /dev/did/rdsk/d${i}s0> ls -lHL /dev/did/rdsk/d${i}s0> donecrw-rw---- 1 oragrid oinstall 239, 64 May 17 03:24 /dev/did/rdsk/d2s0crw-rw---- 1 oragrid oinstall 239, 96 May 17 03:24 /dev/did/rdsk/d3s0crw-rw---- 1 oragrid oinstall 239, 128 May 17 03:24 /dev/did/rdsk/d4s0crw-rw---- 1 oragrid oinstall 239, 160 May 17 03:24 /dev/did/rdsk/d5s0crw-rw---- 1 oragrid oinstall 239, 192 May 17 03:24 /dev/did/rdsk/d6s0crw-rw---- 1 oragrid oinstall 239, 224 May 17 03:24 /dev/did/rdsk/d7s0crw-rw---- 1 oragrid oinstall 239, 256 May 17 03:24 /dev/did/rdsk/d8s0crw-rw---- 1 oragrid oinstall 239, 288 May 17 03:24 /dev/did/rdsk/d9s0bash-3.00#

6

2) Prepare for a Grid Infrastructure for Clusters installation

Install ORCLudlm (SPARC only)

bash-3.00# pwd/net/dv5/swlib/oracle/released/db/11gR2/11.2.0.1/sparc-sunos/64bit-installable/grid/udlmbash-3.00# ls -ltotal 1816-rw-r--r-- 1 root root 899375 Nov 6 2009 ORCLudlm.tar.Z-rw-r--r-- 1 root root 7493 Nov 6 2009 README.udlm-rw-r--r-- 1 root root 770 Nov 6 2009 release_notes.33410bash-3.00# cp ORCLudlm.tar.Z /var/tmpbash-3.00# cd /var/tmpbash-3.00# uncompress ORCLudlm.tar.Zbash-3.00# tar -xf ORCLudlm.tarbash-3.00# pkgadd -d . ORCLudlm

Install RAC framework resources

bash-3.00# clrt listSUNW.LogicalHostname:3SUNW.SharedAddress:2bash-3.00# bash-3.00# clrt register SUNW.rac_frameworkbash-3.00# clrt register SUNW.rac_udlm bash-3.00# clrg create -S rac-fmwk-rgbash-3.00# clrs create -g rac-fmwk-rg -t SUNW.rac_framework rac-fmwk-rsbash-3.00# clrs create -g rac-fmwk-rg -t SUNW.rac_udlm -p resource_dependencies=rac-fmwk-rs rac-udlm-rsbash-3.00# clrg online -eM rac-fmwk-rgbash-3.00# clrs status

=== Cluster Resources ===

Resource Name Node Name State Status Message------------- --------- ----- --------------rac-fmwk-rs pfreak1 Online Online pfreak2 Online Online

rac-udlm-rs pfreak1 Online Online pfreak2 Online Online

bash-3.00#

Setup VNC

bash-3.00# cd; vi .profilePATH=${PATH}:/usr/cluster/bin:/usr/X11/bin:/usr/openwin/bin; export PATH

MANPATH=/usr/man:/usr/cluster/man; export MANPATHbash-3.00# . .profile

7

bash-3.00# vncserver :10

You will require a password to access your desktops.

Password:Verify:xauth: creating new authority file //.Xauthority

New 'pfreak1:10 ()' desktop is pfreak1:10

Creating default startup script //.vnc/xstartupStarting applications specified in //.vnc/xstartupLog file is //.vnc/pfreak1:10.log

bash-3.00#

3) Install Grid Infrastructure for Clusters via a VNC client

Install and configure Grid Infrastructure for Clusters

bash-3.00# xhost +bash-3.00# su - oragrid-bash-3.00$ export DISPLAY=:10.0-bash-3.00$ cd /net/dv5/swlib/oracle/released/db/11gR2/11.2.0.1/sparc*/64*/grid-bash-3.00$ ./runInstaller

Select Installation Option • Select Install and Configure Grid Infrastructure for a Cluster Server• Click Next

Select Installation Type• Select Advanced Installation• Click Next

Select Product Languages Type• Click Next

Grid Plug and Play Information• Cluster Name – freak-cluster• SCAN Name – freak-scan-lh• SCAN Port - 1521• Click Next

Cluster Node Information• Edit Virtual IP Name• pfreak1-vip to freak-1• pfreak2-vip to freak-2• Click Next

Specify Network Interface Usage• Ensure main IPMP NIC is public• Ensure clprivnet0 is private• Click Next

Storage Option Information• Click ASM

8

• Click NextCreate ASM Disk Group

• Disk group Name – your choice (CRS)• Select Redundancy – External• Select Candidate Disks • Click Change Discovery Path• Enter /dev/did/rdsk/d* - OK• May take a few minutes to discover all the candidate disks• Select two disks • Click Next

Specify ASM Password• Use same password – manager (click yes to ignore warning)

• Click NextSpecify Operating System Groups

• ASM OSDBA, OSOPER and OSASM are oinstall• Click Next• Click Yes (OSDBA, OSOPER and OSASM are in the same OS Group)

Specify Installation Location• Oracle Base - /u01/app/oragrid• Software Location - /u01/app/11.2.0/grid• Click Next

Create Inventory• Inventory Directory - /u01/app/oraInventory• Click Next

Perform Prerequisite Checks• Ignore max open file descriptors• Ignore Compiler patch 124861-15• Ignore users with same UID• Resolve other issues• Click Ignore all• Click Next

Summary• Click Finish

Execute Configuration Scripts• Click OK once done

Note, please see below, i.e. after the root scripts output for a continuation of the installation.

pfreak1 scripts

bash-3.00# pwd/u01/app/oraInventorybash-3.00#bash-3.00# ./orainstRoot.shChanging permissions of /u01/app/oraInventory.Adding read,write permissions for group.Removing read,write,execute permissions for world.

Changing groupname of /u01/app/oraInventory to oinstall.The execution of the script is complete.

9

bash-3.00# pwd/u01/app/oraInventorybash-3.00#

Repeat above for pfreak2. Then on pfreak1,

bash-3.00# ./root.shRunning Oracle 11g root.sh script...

The following environment variables are set as: ORACLE_OWNER= oragrid ORACLE_HOME= /u01/app/11.2.0/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]: Creating /usr/local/bin directory... Copying dbhome to /usr/local/bin ... Copying oraenv to /usr/local/bin ... Copying coraenv to /usr/local/bin ...

Creating /var/opt/oracle/oratab file...Entries will be added to the /var/opt/oracle/oratab file as needed byDatabase Configuration Assistant when a database is createdFinished running generic part of root.sh script.Now product-specific root actions will be performed.2010-05-17 05:09:57: Parsing the host name2010-05-17 05:09:57: Checking for super user privileges2010-05-17 05:09:57: User has super user privilegesUsing configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_paramsCreating trace directoryLOCAL ADD MODE Creating OCR keys for user 'root', privgrp 'root'..Operation successful. root wallet root wallet cert root cert export peer wallet profile reader wallet pa wallet peer wallet keys pa wallet keys peer cert request pa cert request peer cert pa cert peer root cert TP profile reader root cert TP pa root cert TP peer pa cert TP pa peer cert TP

10

profile reader pa cert TP profile reader peer cert TP peer user cert pa user certAdding daemon to inittabCRS-4123: Oracle High Availability Services has been started.ohasd is startingCRS-2672: Attempting to start 'ora.gipcd' on 'pfreak1'CRS-2672: Attempting to start 'ora.mdnsd' on 'pfreak1'CRS-2676: Start of 'ora.gipcd' on 'pfreak1' succeededCRS-2676: Start of 'ora.mdnsd' on 'pfreak1' succeededCRS-2672: Attempting to start 'ora.gpnpd' on 'pfreak1'CRS-2676: Start of 'ora.gpnpd' on 'pfreak1' succeededCRS-2672: Attempting to start 'ora.cssdmonitor' on 'pfreak1'CRS-2676: Start of 'ora.cssdmonitor' on 'pfreak1' succeededCRS-2672: Attempting to start 'ora.cssd' on 'pfreak1'CRS-2672: Attempting to start 'ora.diskmon' on 'pfreak1'CRS-2676: Start of 'ora.diskmon' on 'pfreak1' succeededCRS-2676: Start of 'ora.cssd' on 'pfreak1' succeededCRS-2672: Attempting to start 'ora.ctssd' on 'pfreak1'CRS-2676: Start of 'ora.ctssd' on 'pfreak1' succeeded

ASM created and started successfully.

DiskGroup CRS created successfully.

clscfg: -install mode specifiedSuccessfully accumulated necessary OCR keys.Creating OCR keys for user 'root', privgrp 'root'..Operation successful.CRS-2672: Attempting to start 'ora.crsd' on 'pfreak1'CRS-2676: Start of 'ora.crsd' on 'pfreak1' succeededCRS-4256: Updating the profileSuccessful addition of voting disk 6f0f679d5b804f04bf7433abb9133ccf.Successfully replaced voting disk group with +CRS.CRS-4256: Updating the profileCRS-4266: Voting file(s) successfully replaced## STATE File Universal Id File Name Disk group-- ----- ----------------- --------- --------- 1. ONLINE 6f0f679d5b804f04bf7433abb9133ccf (/dev/did/rdsk/d2s0) [CRS]Located 1 voting disk(s).CRS-2673: Attempting to stop 'ora.crsd' on 'pfreak1'CRS-2677: Stop of 'ora.crsd' on 'pfreak1' succeededCRS-2673: Attempting to stop 'ora.asm' on 'pfreak1'CRS-2677: Stop of 'ora.asm' on 'pfreak1' succeededCRS-2673: Attempting to stop 'ora.ctssd' on 'pfreak1'CRS-2677: Stop of 'ora.ctssd' on 'pfreak1' succeededCRS-2673: Attempting to stop 'ora.cssdmonitor' on 'pfreak1'CRS-2677: Stop of 'ora.cssdmonitor' on 'pfreak1' succeededCRS-2673: Attempting to stop 'ora.cssd' on 'pfreak1'CRS-2677: Stop of 'ora.cssd' on 'pfreak1' succeeded

11

CRS-2673: Attempting to stop 'ora.gpnpd' on 'pfreak1'CRS-2677: Stop of 'ora.gpnpd' on 'pfreak1' succeededCRS-2679: Attempting to clean 'ora.gpnpd' on 'pfreak1'CRS-2681: Clean of 'ora.gpnpd' on 'pfreak1' succeededCRS-2673: Attempting to stop 'ora.gipcd' on 'pfreak1'CRS-2677: Stop of 'ora.gipcd' on 'pfreak1' succeededCRS-2673: Attempting to stop 'ora.mdnsd' on 'pfreak1'CRS-2677: Stop of 'ora.mdnsd' on 'pfreak1' succeededCRS-2672: Attempting to start 'ora.mdnsd' on 'pfreak1'CRS-2676: Start of 'ora.mdnsd' on 'pfreak1' succeededCRS-2672: Attempting to start 'ora.gipcd' on 'pfreak1'CRS-2676: Start of 'ora.gipcd' on 'pfreak1' succeededCRS-2672: Attempting to start 'ora.gpnpd' on 'pfreak1'CRS-2676: Start of 'ora.gpnpd' on 'pfreak1' succeededCRS-2672: Attempting to start 'ora.cssdmonitor' on 'pfreak1'CRS-2676: Start of 'ora.cssdmonitor' on 'pfreak1' succeededCRS-2672: Attempting to start 'ora.cssd' on 'pfreak1'CRS-2672: Attempting to start 'ora.diskmon' on 'pfreak1'CRS-2676: Start of 'ora.diskmon' on 'pfreak1' succeededCRS-2676: Start of 'ora.cssd' on 'pfreak1' succeededCRS-2672: Attempting to start 'ora.ctssd' on 'pfreak1'CRS-2676: Start of 'ora.ctssd' on 'pfreak1' succeededCRS-2672: Attempting to start 'ora.asm' on 'pfreak1'CRS-2676: Start of 'ora.asm' on 'pfreak1' succeededCRS-2672: Attempting to start 'ora.crsd' on 'pfreak1'CRS-2676: Start of 'ora.crsd' on 'pfreak1' succeededCRS-2672: Attempting to start 'ora.evmd' on 'pfreak1'CRS-2676: Start of 'ora.evmd' on 'pfreak1' succeededCRS-2672: Attempting to start 'ora.asm' on 'pfreak1'CRS-2676: Start of 'ora.asm' on 'pfreak1' succeededCRS-2672: Attempting to start 'ora.CRS.dg' on 'pfreak1'CRS-2676: Start of 'ora.CRS.dg' on 'pfreak1' succeeded

pfreak1 2010/05/17 05:18:24 /u01/app/11.2.0/grid/cdata/pfreak1/backup_20100517_051824.olrConfigure Oracle Grid Infrastructure for a Cluster ... succeededUpdating inventory properties for clusterwareStarting Oracle Universal Installer...

Checking swap space: must be greater than 500 MB. Actual 19161 MB PassedThe inventory pointer is located at /var/opt/oracle/oraInst.locThe inventory is located at /u01/app/oraInventory'UpdateNodeList' was successful.Bash-3.00#

Finally, on pfreak2.

bash-3.00# ./root.shRunning Oracle 11g root.sh script...

The following environment variables are set as:

12

ORACLE_OWNER= oragrid ORACLE_HOME= /u01/app/11.2.0/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]: Creating /usr/local/bin directory... Copying dbhome to /usr/local/bin ... Copying oraenv to /usr/local/bin ... Copying coraenv to /usr/local/bin ...

Creating /var/opt/oracle/oratab file...Entries will be added to the /var/opt/oracle/oratab file as needed byDatabase Configuration Assistant when a database is createdFinished running generic part of root.sh script.Now product-specific root actions will be performed.2010-05-17 05:34:28: Parsing the host name2010-05-17 05:34:28: Checking for super user privileges2010-05-17 05:34:28: User has super user privilegesUsing configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_paramsCreating trace directoryLOCAL ADD MODE Creating OCR keys for user 'root', privgrp 'root'..Operation successful.Adding daemon to inittabCRS-4123: Oracle High Availability Services has been started.ohasd is startingCRS-4402: The CSS daemon was started in exclusive mode but found an active CSS daemon on node pfreak1, number 1, and is terminatingAn active cluster was found during exclusive startup, restarting to join the clusterCRS-2672: Attempting to start 'ora.mdnsd' on 'pfreak2'CRS-2676: Start of 'ora.mdnsd' on 'pfreak2' succeededCRS-2672: Attempting to start 'ora.gipcd' on 'pfreak2'CRS-2676: Start of 'ora.gipcd' on 'pfreak2' succeededCRS-2672: Attempting to start 'ora.gpnpd' on 'pfreak2'CRS-2676: Start of 'ora.gpnpd' on 'pfreak2' succeededCRS-2672: Attempting to start 'ora.cssdmonitor' on 'pfreak2'CRS-2676: Start of 'ora.cssdmonitor' on 'pfreak2' succeededCRS-2672: Attempting to start 'ora.cssd' on 'pfreak2'CRS-2672: Attempting to start 'ora.diskmon' on 'pfreak2'CRS-2676: Start of 'ora.diskmon' on 'pfreak2' succeededCRS-2676: Start of 'ora.cssd' on 'pfreak2' succeededCRS-2672: Attempting to start 'ora.ctssd' on 'pfreak2'CRS-2676: Start of 'ora.ctssd' on 'pfreak2' succeededCRS-2672: Attempting to start 'ora.asm' on 'pfreak2'CRS-2676: Start of 'ora.asm' on 'pfreak2' succeededCRS-2672: Attempting to start 'ora.crsd' on 'pfreak2'CRS-2676: Start of 'ora.crsd' on 'pfreak2' succeededCRS-2672: Attempting to start 'ora.evmd' on 'pfreak2'CRS-2676: Start of 'ora.evmd' on 'pfreak2' succeeded

13

pfreak2 2010/05/17 05:38:36 /u01/app/11.2.0/grid/cdata/pfreak2/backup_20100517_053836.olrConfigure Oracle Grid Infrastructure for a Cluster ... succeededUpdating inventory properties for clusterwareStarting Oracle Universal Installer...

Checking swap space: must be greater than 500 MB. Actual 19148 MB PassedThe inventory pointer is located at /var/opt/oracle/oraInst.locThe inventory is located at /u01/app/oraInventory'UpdateNodeList' was successful.bash-3.00#

After running the root scripts on all node Click OK.

The following tasks then failed:

– Configure Oracle Grid Infrastructure for a Cluster.– Oracle Cluster Verification Utility.

Looking at the installation log, the following errors are seen:

bash-3.00# pwd/u01/app/oraInventory/logsbash-3.00# bash-3.00# tail -200f installActions2010-05-17_04-12-47AM.log

INFO: Checking name resolution setup for "freak-scan-lh"...INFO: ERROR: INFO: PRVF-4664 : Found inconsistent name resolution entries for SCAN name "freak-scan-lh"INFO: ERROR: INFO: PRVF-4657 : Name resolution setup check for "freak-scan-lh" (IP address: 10.11.188.60) failedINFO: ERROR: INFO: PRVF-4663 : Found configuration issue with the 'hosts' entry in the /etc/nsswitch.conf file

Searching metalink for the above, resulting in Metalink note 887471.1, the solution is to verify that “ping freak-scan-lh” works and if it does then ignore the above error.

bash-3.00# ping freak-scan-lhfreak-scan-lh is alivebash-3.00#

INFO: NTP Configuration file check started...INFO: ERROR: INFO: PRVF-5402 : Warning: Could not find NTP configuration file "/etc/inet/ntp.conf" on node "pfreak2"INFO: ERROR:

14

INFO: PRVF-5402 : Warning: Could not find NTP configuration file "/etc/inet/ntp.conf" on node "pfreak1"INFO: PRVF-5405 : The NTP configuration file "/etc/inet/ntp.conf" does not exist on all nodesINFO: pfreak2,pfreak1INFO: PRVF-5414 : Check of NTP Config file failed on all nodes. Cannot proceed further for the NTP testsINFO: Clock synchronization check using Network Time Protocol(NTP) failedINFO: PRVF-9652 : Cluster Time Synchronization Services check failed

Looking at /etc/inet we can see ntp.conf.cluster

bash-3.00# ls -l ntp*-rw-r--r-- 1 root sys 291 Jan 21 2005 ntp.client-rw-r--r-- 1 root sys 4710 Dec 5 2006 ntp.cluster-rw-r--r-- 1 root root 4379 May 17 02:48 ntp.conf.cluster-rw-r--r-- 1 root sys 2809 Jan 21 2005 ntp.server-rw-r--r-- 1 root sys 3632 Jul 2 2009 ntp4.client-rw-r--r-- 1 root sys 4061 Jul 2 2009 ntp4.serverbash-3.00# pwd/etc/inetbash-3.00# ls -l ntp*-rw-r--r-- 1 root sys 291 Jan 21 2005 ntp.client-rw-r--r-- 1 root sys 4710 Dec 5 2006 ntp.cluster-rw-r--r-- 1 root root 4379 May 17 02:48 ntp.conf.cluster-rw-r--r-- 1 root sys 2809 Jan 21 2005 ntp.server-rw-r--r-- 1 root sys 3632 Jul 2 2009 ntp4.client-rw-r--r-- 1 root sys 4061 Jul 2 2009 ntp4.serverbash-3.00#

To overcome this error I created a symlink.

bash-3.00# cd /etc/inetbash-3.00# ls -l ntp*-rw-r--r-- 1 root sys 291 Jan 21 2005 ntp.client-rw-r--r-- 1 root sys 4710 Dec 5 2006 ntp.cluster-rw-r--r-- 1 root root 4379 May 17 02:48 ntp.conf.cluster-rw-r--r-- 1 root sys 2809 Jan 21 2005 ntp.server-rw-r--r-- 1 root sys 3632 Jul 2 2009 ntp4.client-rw-r--r-- 1 root sys 4061 Jul 2 2009 ntp4.serverbash-3.00# ln -s ntp.conf.cluster ntp.conf bash-3.00# ls -l ntp*-rw-r--r-- 1 root sys 291 Jan 21 2005 ntp.client-rw-r--r-- 1 root sys 4710 Dec 5 2006 ntp.clusterlrwxrwxrwx 1 root root 16 May 17 06:09 ntp.conf -> ntp.conf.cluster-rw-r--r-- 1 root root 4379 May 17 02:48 ntp.conf.cluster-rw-r--r-- 1 root sys 2809 Jan 21 2005 ntp.server-rw-r--r-- 1 root sys 3632 Jul 2 2009 ntp4.client-rw-r--r-- 1 root sys 4061 Jul 2 2009 ntp4.serverbash-3.00#

15

After the above click retry, after which the following failed:

INFO: Checking name resolution setup for "freak-scan-lh"...INFO: ERROR: INFO: PRVF-4664 : Found inconsistent name resolution entries for SCAN name "freak-scan-lh"INFO: ERROR: INFO: PRVF-4657 : Name resolution setup check for "freak-scan-lh" (IP address: 10.11.188.60) failedINFO: ERROR: INFO: PRVF-4663 : Found configuration issue with the 'hosts' entry in the /etc/nsswitch.conf fileINFO: Verification of SCAN VIP and Listener setup failed

Which, so long as you can ping freak-scan-lh is OK to ignore.

INFO: NTP daemon slewing option check failed on some nodesINFO: Check failed on nodes: INFO: pfreak2,pfreak1INFO: PRVF-5436 : The NTP daemon running on one or more nodes lacks the slewing option "slewalways yes"INFO: Clock synchronization check using Network Time Protocol(NTP) failedINFO: PRVF-9652 : Cluster Time Synchronization Services check failedINFO: Post-check for cluster services setup was unsuccessful on all the nodes.

For this error and after looking at metalink for

The NTP daemon running on one or more nodes lacks the slewing option

Resulting in Metalink note 1056693.1

I added the following to /etc/inet/ntp.conf and restarted NTP:

slewalways yesdisable pll

bash-3.00# ps -ef | grep ntp root 1021 1 0 02:51:14 ? 0:01 /usr/lib/inet/xntpd -c /etc/inet/ntp.conf.clusterbash-3.00# kill -9 1021bash-3.00# /usr/lib/inet/xntpd -c /etc/inet/ntp.conf.clusterbash-3.00# ps -ef | grep ntp root 19214 1 0 06:27:34 ? 0:00 /usr/lib/inet/xntpd -c /etc/inet/ntp.conf.clusterbash-3.00#

Then Click Retry.

Afterwards the NTP daemon verification passed as did the rest of the Cluster

16

Verification Utility.

INFO: NTP Configuration file check started...INFO: NTP Configuration file check passedINFO: Checking daemon liveness...INFO: Liveness check passed for "xntpd"INFO: NTP daemon slewing option check passedINFO: NTP daemon's boot time configuration check for slewing option passedINFO: NTP common Time Server Check started...INFO: Check of common NTP Time Server passedINFO: Clock time offset check from NTP Time Server started...INFO: Clock time offset check passedINFO: Clock synchronization check using Network Time Protocol(NTP) passedINFO: Oracle Cluster Time Synchronization Services check passedINFO: Post-check for cluster services setup was unsuccessful on all the nodes. Then Click Next/Close.

Display the Grid Infrastructure resources

bash-3.00# su - oragridSun Microsystems Inc. SunOS 5.10 Generic January 2005

SC Lab Install: COMPLETE - install logs are in /var/sadm/system/logs

-bash-3.00$ cat .profileORACLE_HOME=/u01/app/11.2.0/gridPATH=${PATH}:${ORACLE_HOME}/bin

MANPATH=/usr/man:/usr/cluster/man; export MANPATH-bash-3.00$-bash-3.00$ crsctl stat res -t--------------------------------------------------------------------------------NAME TARGET STATE SERVER STATE_DETAILS --------------------------------------------------------------------------------Local Resources--------------------------------------------------------------------------------ora.CRS.dg ONLINE ONLINE pfreak1 ONLINE ONLINE pfreak2 ora.LISTENER.lsnr ONLINE ONLINE pfreak1 ONLINE ONLINE pfreak2 ora.asm ONLINE ONLINE pfreak1 Started ONLINE ONLINE pfreak2 Started ora.eons ONLINE ONLINE pfreak1

17

ONLINE ONLINE pfreak2 ora.gsd OFFLINE OFFLINE pfreak1 OFFLINE OFFLINE pfreak2 ora.net1.network ONLINE ONLINE pfreak1 ONLINE ONLINE pfreak2 ora.ons ONLINE ONLINE pfreak1 ONLINE ONLINE pfreak2 --------------------------------------------------------------------------------Cluster Resources--------------------------------------------------------------------------------ora.LISTENER_SCAN1.lsnr 1 ONLINE ONLINE pfreak1 ora.oc4j 1 OFFLINE OFFLINE ora.pfreak1.vip 1 ONLINE ONLINE pfreak1 ora.pfreak2.vip 1 ONLINE ONLINE pfreak2 ora.scan1.vip 1 ONLINE ONLINE pfreak1 -bash-3.00$

Create another ASM disk group

bash-3.00# xhost +bash-3.00# su - oragrid-bash-3.00$ export DISPLAY=:10.0-bash-3.00$ /u01/app/11.2.0/grid/bin/asmca

ASM Configuration Assistant: Configure ASM: ASM Instances• Select Disk Groups tab• Click Create• Disk Group Name – DATA• Redundancy – External• Select remaining disks• Click OK• Note thaat DATA is MOUNTED (2 of 2)• Click Exit/Yes

Note that a Grid Infrastructure ASM Disk Group resource has been created.-bash-3.00$ crsctl stat res -t--------------------------------------------------------------------------------NAME TARGET STATE SERVER STATE_DETAILS --------------------------------------------------------------------------------

18

Local Resources--------------------------------------------------------------------------------ora.CRS.dg ONLINE ONLINE pfreak1 ONLINE ONLINE pfreak2 ora.DATA.dg ONLINE ONLINE pfreak1 ONLINE ONLINE pfreak2 ora.LISTENER.lsnr ONLINE ONLINE pfreak1 ONLINE ONLINE pfreak2 ora.asm ONLINE ONLINE pfreak1 Started ONLINE ONLINE pfreak2 Started ora.eons ONLINE ONLINE pfreak1 ONLINE ONLINE pfreak2 ora.gsd OFFLINE OFFLINE pfreak1 OFFLINE OFFLINE pfreak2 ora.net1.network ONLINE ONLINE pfreak1 ONLINE ONLINE pfreak2 ora.ons ONLINE ONLINE pfreak1 ONLINE ONLINE pfreak2 --------------------------------------------------------------------------------Cluster Resources--------------------------------------------------------------------------------ora.LISTENER_SCAN1.lsnr 1 ONLINE ONLINE pfreak1 ora.oc4j 1 OFFLINE OFFLINE ora.pfreak1.vip 1 ONLINE ONLINE pfreak1 ora.pfreak2.vip 1 ONLINE ONLINE pfreak2 ora.scan1.vip 1 ONLINE ONLINE pfreak1 -bash-3.00$

4) Install and create an 11gR1 database via a VNC client

Install the 11gR1 database software

Before installing an 11gR1 database we need to pin the cluster nodes. Please refer to Metalink note 949073.1

19

bash-3.00# cd /u01/app/11.2.0/grid/binbash-3.00# ./olsnodes -t -npfreak1 1 Unpinnedpfreak2 2 Unpinnedbash-3.00#bash-3.00# ./crsctl pin css -n pfreak1 pfreak2CRS-4664: Node pfreak1 successfully pinned.CRS-4664: Node pfreak2 successfully pinned.bash-3.00# bash-3.00# ./olsnodes -t -npfreak1 1 Pinnedpfreak2 2 Pinnedbash-3.00#

bash-3.00# mkdir -p /export/home/ora11gbash-3.00# useradd -g oinstall -G dba -u 302 -d /export/home/ora11g -s /usr/bin/bash ora11gbash-3.00# chown ora11g:oinstall /export/home/ora11gbash-3.00#bash-3.00# mkdir /u01/app/ora11gbash-3.00# chown ora11g:oinstall /u01/app/ora11gbash-3.00# chmod 775 /u01/app/ora11gbash-3.00# bash-3.00# passwd ora11gNew Password: Re-enter new Password: passwd: password successfully changed for ora11gbash-3.00# su - ora11gSun Microsystems Inc. SunOS 5.10 Generic January 2005


-bash-3.00$ cat > .profile <<-EOF> ORACLE_BASE=/u01/app/ora11g> EOF-bash-3.00$

From a VNC client on pfreak1.

bash-3.00# xhost +bash-3.00# su – ora11g-bash-3.00$ export DISPLAY=:10.0-bash-3.00$ cd /net/dv5/swlib/oracle/released/db/11gR1/11.1.0.6/sparc*/64*/database-bash-3.00$ ./runInstaller

Welcome• Click Next

Select Installation Type• Select Custom

20

• Click NextSpecify installation Location

• Oracle Base - /u01/app/ora11g• Software Location • Name: OraDb11g_home1• Path: /u01/app/ora11g/product/11.1.0/db_1• Click Next

Specify Hardware Cluster Installation Mode• Select Local Installation• Click Next

Product-Specific Pre-requisite Checks • Click Next

Available Product Components• Click Next

Privileged Operating System groups• OSDBA – dba• OSOPER – oinstall• OSASM - oinstall• Click Next

Create Database• Select Install database Software only• Click Next

Summary• Click Install

Execute Configuration scripts• Execute root.sh and Click OK/Close

Repeat the above to install the database software on pfreak2, however ensure that the Path is the same within “Specify installation Location”, as this install is on local file system.

Create the 11gR1 database using dbca

Create 11gR1 database via dbca on pfreak1 via a VNC client.

bash-3.00# su - ora11gSun Microsystems Inc. SunOS 5.10 Generic January 2005


-bash-3.00$ export DISPLAY=:10.0-bash-3.00$ cd /u01/app/ora11g/product/11.1.0/db_1/bin-bash-3.00$ ./dbca

Welcome to dbca• Click Next• Step 1/15 – Create a Database – Click Next• Step 2/15 – General Purpose – Click Next• Step 3/15 – DB1 – Click Next• Step 4/15 – Click Next• Step 5/15 – Use same password (manager) – Click Next

21

• Step 6/15 – Select ASM Click Next • Specify ASMSNMP password to ASM (manager) – Click OK• Step 7/16 – Select DATA - Click Next• Step 8/16 – Click Next• Step 9/16 – Click Next• Step 10/16 – Click Next• Step 11/16 – memory your choice (1024) – Click Next• Step 12/15 – Click Next• Step 13/15 – Click Next• Step 14/15 – Click Next• Step 15/15 – Create Database - Click Finish/OK• Summary – Click Exit

Create the 11gR1 database listener using netca

Create 11gR1 database listener via netca on pfreak1 via a VNC client.



-bash-3.00$ export DISPLAY=:10.0-bash-3.00$ cd /u01/app/ora11g/product/11.1.0/db_1/bin-bash-3.00$ ./netca

Select the type of Oracle Net Services configuration• Select Single node configuration• Click Next

Welcome• Select Listener configuration• Click Next

Listener_name• Enter LISTENER_DB1• Click Next

Listener proptocol• Click Next

Listener port number• Use another port number 1522• Click Next

More listeners• Select no• Click Next/Next/Finish



-bash-3.00$-bash-3.00$ cd /u01/app/ora11g/product/11.1.0/db_1/network/admin

22

Change listener.ora and tnsnames.ora to use a logical host.

-bash-3.00$ cat listener.ora # listener.ora Network Configuration File: /u01/app/ora11g/product/11.1.0/db_1/network/admin/listener.ora# Generated by Oracle configuration tools.

LISTENER_DB1 = (DESCRIPTION_LIST = (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = vzfreak2a)(PORT = 1522)) (ADDRESS = (PROTOCOL = IPC)(KEY = EXTPROC1522)) ) )

-bash-3.00$ -bash-3.00$ cat tnsnames.oraLISTENER_DB1 = (ADDRESS = (PROTOCOL = TCP)(HOST = vzfreak2a)(PORT = 1522))-bash-3.00$

-bash-3.00$ export ORACLE_HOME=/u01/app/ora11g/product/11.1.0/db_1-bash-3.00$ export ORACLE_SID=DB1-bash-3.00$ $ORACLE_HOME/bin/sqlplus "/as sysdba"

SQL*Plus: Release 11.1.0.6.0 - Production on Wed May 19 03:43:40 2010

Copyright (c) 1982, 2007, Oracle. All rights reserved.

Connected to:Oracle Database 11g Enterprise Edition Release 11.1.0.6.0 - 64bit ProductionWith the Partitioning, OLAP, Data Mining and Real Application Testing options

SQL>SQL> show parameters local_listener

NAME TYPE VALUE------------------------------------ ----------- ------------------------------local_listener stringSQL>SQL> alter system set local_listener='LISTENER_DB1' scope=both;

System altered.

SQL>SQL> show parameters local_listener

NAME TYPE VALUE

23

------------------------------------ ----------- ------------------------------local_listener string LISTENER_DB1SQL> SQL> exitDisconnected from Oracle Database 11g Enterprise Edition Release 11.1.0.6.0 - 64bit ProductionWith the Partitioning, OLAP, Data Mining and Real Application Testing options-bash-3.00$

Copy some 11gR1 database files to the another node

After dbca and netca are finished we need to copy some files from pfreak1 to pfreak2. Note that the ../../.. represents the ora11g home directories, i.e. /export/home/ora11g.On pfreak2.

bash-3.00# uname -npfreak2bash-3.00# su - ora11gSun Microsystems Inc. SunOS 5.10 Generic January 2005


-bash-3.00$ cd /u01/app/ora11g/diag/rdbms-bash-3.00$ ls -ltotal 0-bash-3.00$ /usr/sfw/bin/wget -r -nH --cut-dirs=8 ftp://ora11g:ora11g@pfreak1:/../../../u01/app/ora11g/diag/rdbms/db1<...>-bash-3.00$ ls -ltotal 2drwxr-xr-x 3 ora11g oinstall 512 May 18 08:11 db1-bash-3.00$ cd ../..-bash-3.00$-bash-3.00$ pwd/u01/app/ora11g-bash-3.00$ ls -ltotal 4drwxrwxr-x 11 ora11g oinstall 512 May 18 06:49 diagdrwxr-xr-x 3 ora11g oinstall 512 May 18 06:35 product-bash-3.00$-bash-3.00$ /usr/sfw/bin/wget -r -nH --cut-dirs=6 ftp://ora11g:ora11g@pfreak1:/../../../u01/app/ora11g/admin<...>-bash-3.00$ ls -ltotal 6drwxr-xr-x 3 ora11g oinstall 512 May 18 08:12 admindrwxrwxr-x 11 ora11g oinstall 512 May 18 06:49 diagdrwxr-xr-x 3 ora11g oinstall 512 May 18 06:35 product-bash-3.00$

24

ftp://ora11g:ora11g@pfreak1:/../../../u01/app/ora11g/admin

-bash-3.00$ cd /u01/app/ora11g/product/11.1.0/db_1/dbs-bash-3.00$-bash-3.00$ ls -ltotal 44-rw-r--r-- 1 ora11g oinstall 8385 Sep 11 1998 init.ora-rw-r--r-- 1 ora11g oinstall 12920 May 3 2001 initdw.ora-bash-3.00$-bash-3.00$ ftp pfreak1Connected to pfreak1.220 pfreak1 FTP server ready.Name (pfreak1:root): ora11g331 Password required for ora11g.Password:230 User ora11g logged in.Remote system type is UNIX.Using binary mode to transfer files.ftp> cd /u01/app/ora11g/product/11.1.0/db_1/dbs250 CWD command successful.ftp> lcd /u01/app/ora11g/product/11.1.0/db_1/dbsLocal directory now /u01/app/ora11g/product/11.1.0/db_1/dbsftp> prompt Interactive mode off.ftp> mget *DB1200 PORT command successful.150 Opening BINARY mode data connection for lkDB1 (24 bytes).226 Transfer complete.local: lkDB1 remote: lkDB124 bytes received in 0.014 seconds (1.68 Kbytes/s)200 PORT command successful.150 Opening BINARY mode data connection for orapwDB1 (1536 bytes).226 Transfer complete.local: orapwDB1 remote: orapwDB11536 bytes received in 0.013 seconds (111.88 Kbytes/s)ftp> ftp> cd /u01/app/ora11g/product/11.1.0/db_1/network/admin250 CWD command successful.ftp> lcd /u01/app/ora11g/product/11.1.0/db_1/network/adminLocal directory now /u01/app/ora11g/product/11.1.0/db_1/network/adminftp> mget *ora*200 PORT command successful.150 Opening BINARY mode data connection for listener.ora (337 bytes).226 Transfer complete.local: listener.ora remote: listener.ora337 bytes received in 0.012 seconds (26.59 Kbytes/s)200 PORT command successful.150 Opening BINARY mode data connection for tnsnames.ora (77 bytes).226 Transfer complete.local: tnsnames.ora remote: tnsnames.ora77 bytes received in 0.011 seconds (6.80 Kbytes/s)ftp> bye221-You have transferred 1974 bytes in 4 files.

25

221-Total traffic for this session was 3428 bytes in 6 transfers.221-Thank you for using the FTP service on pfreak1.221 Goodbye.-bash-3.00$

Verify the 11gR1 database installation




SQL*Plus: Release 11.1.0.6.0 - Production on Tue May 18 08:16:41 2010


Connected to:Oracle Database 11g Enterprise Edition Release 11.1.0.6.0 - 64bit ProductionWith the Partitioning, OLAP, Data Mining and Real Application Testing options

SQL> show sga

Total System Global Area 801939456 bytesFixed Size 2092920 bytesVariable Size 260051080 bytesDatabase Buffers 536870912 bytesRedo Buffers 2924544 bytesSQL> SQL> create user homer identified by simpson;

User created.

SQL> alter user homer default tablespace system quota 1m on system;

User altered.

SQL> grant select on v_$sysstat to homer;

Grant succeeded.

SQL> grant select on v_$archive_dest to homer;

Grant succeeded.

SQL> grant create session to homer;

Grant succeeded.

26

SQL> grant create table to homer;

Grant succeeded.

SQL> grant select on v_$database to homer;

Grant succeeded.

SQL> shutdown immediateDatabase closed.Database dismounted.ORACLE instance shut down.SQL>SQL> exitDisconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit ProductionWith the Partitioning, Automatic Storage Management, OLAP, Data Miningand Real Application Testing options-bash-3.00$

And on pfreak2.




SQL*Plus: Release 11.1.0.6.0 - Production on Wed May 19 01:53:13 2010


Connected to an idle instance.

SQL> startupORACLE instance started.

Total System Global Area 801939456 bytesFixed Size 2092920 bytesVariable Size 268439688 bytesDatabase Buffers 528482304 bytesRedo Buffers 2924544 bytesDatabase mounted.Database opened.SQL>SQL> select username from all_users where username='HOMER';

27

USERNAME------------------------------HOMER

SQL>SQL> shutdown immediateDatabase closed.Database dismounted.ORACLE instance shut down.SQL> exitDisconnected from Oracle Database 11g Enterprise Edition Release 11.1.0.6.0 - 64bit ProductionWith the Partitioning, OLAP, Data Mining and Real Application Testing options-bash-3.00$

Check the Grid Infrastructure resources

Note that Grid Infrastructure does not know about DB1 as 11gR1 is not 11gR2 Grid Infrastructure aware. As such there is no Grid Infrastructure resource ora.db1.db.



-bash-3.00$ -bash-3.00$ /u01/app/11.2.0/grid/bin/crsctl stat res -t--------------------------------------------------------------------------------NAME TARGET STATE SERVER STATE_DETAILS --------------------------------------------------------------------------------Local Resources--------------------------------------------------------------------------------ora.CRS.dg ONLINE ONLINE pfreak1 ONLINE ONLINE pfreak2 ora.DATA.dg ONLINE ONLINE pfreak1 ONLINE ONLINE pfreak2 ora.LISTENER.lsnr ONLINE ONLINE pfreak1 ONLINE ONLINE pfreak2 ora.asm ONLINE ONLINE pfreak1 Started ONLINE ONLINE pfreak2 Started ora.eons ONLINE ONLINE pfreak1 ONLINE ONLINE pfreak2

28

ora.gsd OFFLINE OFFLINE pfreak1 OFFLINE OFFLINE pfreak2 ora.net1.network ONLINE ONLINE pfreak1 ONLINE ONLINE pfreak2 ora.ons ONLINE ONLINE pfreak1 ONLINE ONLINE pfreak2 --------------------------------------------------------------------------------Cluster Resources--------------------------------------------------------------------------------ora.LISTENER_SCAN1.lsnr 1 ONLINE ONLINE pfreak1 ora.oc4j 1 OFFLINE OFFLINE ora.pfreak1.vip 1 ONLINE ONLINE pfreak1 ora.pfreak2.vip 1 ONLINE ONLINE pfreak2 ora.scan1.vip 1 ONLINE ONLINE pfreak1 -bash-3.00$

5) Install and create an 11gR2 database via a VNC client

Install the 11gR2 database software

Note that here I will install the database software on each node and afterwards create the database.

From a VNC client on pfreak1.

bash-3.00# xhost +bash-3.00# su – ora11g -bash-3.00$ export DISPLAY=:10.0-bash-3.00$ cd /net/dv5/swlib/oracle/released/db/11gR2/11.2.0.1/sparc*/64*/database-bash-3.00$ ./runInstaller

Configure Security Updates• Deselect I wish to receive security updates...• Click Next/Yes• Select Install database software • Click Next

Node Selection• Select single instance database installation• Click Next

29

Select Product Languages• Click Next

Select Database Edition• Select Enterprise Edition• Click Next

Specify installation Location• Oracle Base - /u01/app/ora11g• Software Location - /u01/app/ora11g/product/11.2.0/dbhome_1• Click Next

Privileged Operating System groups• OSDBA – dba• OSOPER – oinstall• Click Next

Perform Prerequisite Checks• Ignore Compiler patch 124861-15• Ignore users with same UID• Select Ignore All• Click Next

Summary• Click Finish

Execute Configuration scripts• Execute root.sh and Click OK/Close

Repeat the above to install the database software on pfreak2.

Create 11gR2 database via dbca on pfreak1 via a VNC client.

As per Metalink note 983715.1, if creating 11.2.0.1 (fixed with 11.2.0.2) amend the following on each node.

/etc/init.d/ohasd

start(){ $ECHO -n $"Starting $PROG: " ulimit -n 65536 ulimit -l 50000000

Create the 11gR2 database using dbca



-bash-3.00$ export DISPLAY=:10.0-bash-3.00$ cd /u01/app/ora11g/product/11.2.0/dbhome_1/bin-bash-3.00$ ./dbca

Welcome to dbca• Click Next

30

• Step 1/12 – Create a Database – Click Next• Step 2/12 – General Purpose – Click Next• Step 3/12 – DB2 – Click Next• Step 4/12 – Click Next• Step 5/12 – Use same password (manager) – Click Next/yes• Step 6/12 – Storage Type (ASM) Database Area (+DATA)• Specify ASMSNMP password to ASM (manager) – Click OK• Step 7/11 – Click Next• Step 8/11 – Click Next• Step 9/11 – memory your choice (1024) – Click Next• Step 10/11 – Click Next• Step 11/11 – Create Database - Click Finish/OK• Summary – Click Exit

Copy some 11gR2 database files to the another node

After dbca is finished we need to copy some files from pfreak1 to pfreak2. Note that the ../../.. represents the ora11g home directories, i.e. /export/home/ora11g.

On pfreak2.

bash-3.00# uname -npfreak2bash-3.00# su - ora11gSun Microsystems Inc. SunOS 5.10 Generic January 2005


-bash-3.00$ cd /u01/app/ora11g/diag/rdbms-bash-3.00$ ls -ltotal 2drwxr-xr-x 3 ora11g oinstall 512 May 18 08:11 db1-bash-3.00$-bash-3.00$ /usr/sfw/bin/wget -r -nH --cut-dirs=8 ftp://ora11g:ora11g@pfreak1:/../../../u01/app/ora11g/diag/rdbms/db2<...>-bash-3.00$ ls -ltotal 4drwxr-xr-x 3 ora11g oinstall 512 May 18 08:11 db1drwxr-xr-x 3 ora11g oinstall 512 May 18 11:41 db2-bash-3.00$ cd ../../admin-bash-3.00$ ls -ltotal 2drwxr-xr-x 5 ora11g oinstall 512 May 18 08:12 DB1-bash-3.00$-bash-3.00$ /usr/sfw/bin/wget -r -nH --cut-dirs=7 ftp://ora11g:ora11g@pfreak1:/../../../u01/app/ora11g/admin/DB2<...>-bash-3.00$ pwd/u01/app/ora11g/admin

31

ftp://ora11g:ora11g@pfreak1:/../../../u01/app/ora11g/admin

-bash-3.00$ ls -ltotal 4drwxr-xr-x 5 ora11g oinstall 512 May 18 08:12 DB1drwxr-xr-x 5 ora11g oinstall 512 May 18 11:43 DB2-bash-3.00$ -bash-3.00$ cd /u01/app/ora11g/product/11.2.0/dbhome_1/dbs-bash-3.00$-bash-3.00$ ftp pfreak1Connected to pfreak1.220 pfreak1 FTP server ready.Name (pfreak1:root): ora11g331 Password required for ora11g.Password:

230 User ora11g logged in.Remote system type is UNIX.Using binary mode to transfer files.ftp> cd /u01/app/ora11g/product/11.2.0/dbhome_1/dbs250 CWD command successful.ftp> promptInteractive mode off.ftp> mget *DB2*200 PORT command successful.150 Opening BINARY mode data connection for hc_DB2.dat (1544 bytes).226 Transfer complete.local: hc_DB2.dat remote: hc_DB2.dat1544 bytes received in 0.02 seconds (76.25 Kbytes/s)200 PORT command successful.150 Opening BINARY mode data connection for initDB2.ora (33 bytes).226 Transfer complete.local: initDB2.ora remote: initDB2.ora33 bytes received in 0.011 seconds (3.05 Kbytes/s)200 PORT command successful.150 Opening BINARY mode data connection for lkDB2 (24 bytes).226 Transfer complete.local: lkDB2 remote: lkDB224 bytes received in 0.013 seconds (1.83 Kbytes/s)200 PORT command successful.150 Opening BINARY mode data connection for orapwDB2 (1536 bytes).226 Transfer complete.local: orapwDB2 remote: orapwDB21536 bytes received in 0.012 seconds (125.02 Kbytes/s)ftp> bye221-You have transferred 3137 bytes in 4 files.221-Total traffic for this session was 4307 bytes in 5 transfers.221-Thank you for using the FTP service on pfreak1.221 Goodbye.-bash-3.00$

32

Verify the 11gR2 database installation

Note that for an 11gR2 database we do not need to create a user that would normally be used by the HA-Oracle (SUNW.oracle_server) probe, as probing is mainly provided by a Grid Infrastructure database resource, which has been automatically created when the database was created using dbca.

Also note that at this point we cannot verify that the database DB2 can start on the other node as the database is configured, via dbca, to only run on one node. That node is the node where the database was created.

When HA-Oracle (SUNW.oracle_server) is deployed later on, the server pool associated with the database is modified (by SUNW.oracle_server) so that the database can run on another node. This is transparent to the administrator and is managed by SUNW.oracle_server.



-bash-3.00$ export ORACLE_HOME=/u01/app/ora11g/product/11.2.0/dbhome_1-bash-3.00$ export ORACLE_SID=DB2-bash-3.00$ $ORACLE_HOME/bin/sqlplus "/as sysdba"

SQL*Plus: Release 11.2.0.1.0 Production on Wed May 19 02:12:42 2010


Connected to:Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit ProductionWith the Partitioning, Automatic Storage Management, OLAP, Data Miningand Real Application Testing options

SQL> show sga

Total System Global Area 1069252608 bytesFixed Size 2154936 bytesVariable Size 717233736 bytesDatabase Buffers 343932928 bytesRedo Buffers 5931008 bytesSQL> exitDisconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit ProductionWith the Partitioning, Automatic Storage Management, OLAP, Data Miningand Real Application Testing options-bash-3.00$

33

Check the Grid Infrastructure resources

Note that for an 11gR2 database Grid Infrastructure does know about the database. As such there is Grid Infrastructure resource ora.db2.db.

-bash-3.00$ exitlogoutbash-3.00# su - oragridSun Microsystems Inc. SunOS 5.10 Generic January 2005


-bash-3.00$ srvctl status db -d db2Instance DB2 is running on node pfreak1-bash-3.00$ -bash-3.00$ crsctl stat res -t--------------------------------------------------------------------------------NAME TARGET STATE SERVER STATE_DETAILS --------------------------------------------------------------------------------Local Resources--------------------------------------------------------------------------------ora.CRS.dg ONLINE ONLINE pfreak1 ONLINE ONLINE pfreak2 ora.DATA.dg ONLINE ONLINE pfreak1 ONLINE ONLINE pfreak2 ora.LISTENER.lsnr ONLINE ONLINE pfreak1 ONLINE ONLINE pfreak2 ora.asm ONLINE ONLINE pfreak1 Started ONLINE ONLINE pfreak2 Started ora.eons ONLINE ONLINE pfreak1 ONLINE ONLINE pfreak2 ora.gsd OFFLINE OFFLINE pfreak1 OFFLINE OFFLINE pfreak2 ora.net1.network ONLINE ONLINE pfreak1 ONLINE ONLINE pfreak2 ora.ons ONLINE ONLINE pfreak1 ONLINE ONLINE pfreak2 --------------------------------------------------------------------------------Cluster Resources-----------------------------------------------------------------------------

34

---ora.LISTENER_SCAN1.lsnr 1 ONLINE ONLINE pfreak2 ora.db2.db 1 ONLINE ONLINE pfreak1 Open ora.oc4j 1 OFFLINE OFFLINE ora.pfreak1.vip 1 ONLINE ONLINE pfreak1 ora.pfreak2.vip 1 ONLINE ONLINE pfreak2 ora.scan1.vip 1 ONLINE ONLINE pfreak2 -bash-3.00$

6) Using SCAN for the 11gR2 single instance database

In order for the single instance database to use SCAN we need to amend the remote_listener parameter.

-bash-3.00$ srvctl config scanSCAN name: freak-scan-lh, Network: 1/10.11.188.0/255.255.255.0/bge0SCAN VIP name: scan1, IP: /vzfreak1a/10.11.188.60-bash-3.00$ -bash-3.00$ srvctl config scan_listenerSCAN Listener LISTENER_SCAN1 exists. Port: TCP:1521-bash-3.00$

SQL> show parameters listener

NAME TYPE VALUE------------------------------------ ----------- ------------------------------listener_networks stringlocal_listener string (DESCRIPTION=(ADDRESS_LIST=(AD

DRESS=(PROTOCOL=TCP)(HOST=frea k-1)(PORT=1521))))

remote_listener stringSQL>SQL> alter system set remote_listener = 'freak-scan-lh:1521' scope=both;

System altered.



DRESS=(PROTOCOL=TCP)(HOST=frea

35

k-1)(PORT=1521))))remote_listener string freak-scan-lh:1521SQL>

On the client, as we are using /etc/hosts for the SCAN address, you will need an entry within /etc/hosts

10.11.188.60 freak-scan-lh

also the client $ORACLE_HOME/network/admin/tnsnames.ora

DB2 = (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = freak-scan-lh)(PORT = 1521)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = DB2) ) )

7) Place everything under Oracle Solaris Cluster

Register the resource types

bash-3.00# clrt listbash-3.00# clrt listSUNW.LogicalHostname:3SUNW.SharedAddress:2SUNW.rac_framework:5SUNW.rac_udlm:3bash-3.00#bash-3.00# clrt register SUNW.crs_frameworkbash-3.00# clrt register SUNW.scalable_asm_instance_proxybash-3.00# clrt register SUNW.scalable_asm_diskgroup_proxybash-3.00# clrt register SUNW.oracle_serverbash-3.00# clrt register SUNW.oracle_listenerbash-3.00# clrt listSUNW.LogicalHostname:3SUNW.SharedAddress:2SUNW.rac_framework:5SUNW.rac_udlm:3SUNW.crs_framework:3SUNW.scalable_asm_instance_proxySUNW.scalable_asm_diskgroup_proxySUNW.oracle_server:6SUNW.oracle_listener:5bash-3.00#

36

bash-3.00# clrg status

=== Cluster Resource Groups ===

Group Name Node Name Suspended Status---------- --------- --------- ------rac-fmwk-rg pfreak1 No Online pfreak2 No Online

bash-3.00# clrs create -g rac-fmwk-rg -t SUNW.crs_framework -p resource_dependencies=rac-fmwk-rs crs-fmwk-rsbash-3.00# clrs status




crs-fmwk-rs pfreak1 Online Online pfreak2 Online Online

bash-3.00#

Register and enable the clustered ASM instance resourcebash-3.00# clrg create -S asm-inst-rgbash-3.00# clrg set -p rg_affinities=++rac-fmwk-rg asm-inst-rgbash-3.00# clrs create -g asm-inst-rg -t SUNW.scalable_asm_instance_proxy -p oracle_home=/u01/app/11.2.0/grid -p 'oracle_sid{pfreak1}'=+ASM1 -p 'oracle_sid{pfreak2}'=+ASM2 -p crs_home=/u01/app/11.2.0/grid -p resource_dependencies_offline_restart=crs-fmwk-rs -d asm-inst-rsbash-3.00# clrg online -eM asm-inst-rgbash-3.00# clrs status





37

asm-inst-rs pfreak1 Online Online - +ASM1 is UP and ENABLED pfreak2 Online Online - +ASM2 is UP and ENABLED

bash-3.00#

Register and enable the clustered ASM disk group resource

bash-3.00# clrg create -S asm-dg-rgbash-3.00# clrg set -p rg_affinities=++asm-inst-rg asm-dg-rgbash-3.00# clrs create -g asm-dg-rg -t SUNW.scalable_asm_diskgroup_proxy -p asm_diskgroups=DATA -p resource_dependencies_offline_restart=asm-inst-rs -d asm-dg-rsbash-3.00# clrg online -eM asm-dg-rgbash-3.00# clrs status





asm-inst-rs pfreak1 Online Online - +ASM1 is UP and ENABLED pfreak2 Online Online - +ASM2 is UP and ENABLED

asm-dg-rs pfreak1 Online Online - Mounted: DATA pfreak2 Online Online - Mounted: DATA

bash-3.00#

Register and enable the 11gR1 database and listener resources

bash-3.00# clrg create ora-11gr1-rgbash-3.00# clrg set -p rg_affinities=++asm-dg-rg ora-11gr1-rgbash-3.00# clrslh create -g ora-11gr1-rg -h vzfreak2a ora-db1-lhbash-3.00# clrs create -g ora-11gr1-rg -t SUNW.oracle_server -p oracle_home=/u01/app/ora11g/product/11.1.0/db_1 -p oracle_sid=DB1 -p alert_log_file=/u01/app/ora11g/diag/rdbms/db1/DB1/trace/alert_DB1.log -p connect_string=homer/simpson -p resource_dependencies_offline_restart=asm-dg-rs -d ora-db1-rs

38

bash-3.00#bash-3.00# clrg online -eM ora-11gr1-rgbash-3.00# clrs status -g ora-11gr1-rg


Resource Name Node Name State Status Message------------- --------- ----- --------------ora-db1-rs pfreak1 Online Online pfreak2 Offline Offline

ora-db1-lh pfreak1 Online Online - LogicalHostname online. pfreak2 Offline Offline

bash-3.00#



-bash-3.00$-bash-3.00$ export ORACLE_HOME=/u01/app/ora11g/product/11.1.0/db_1-bash-3.00$ /u01/app/ora11g/product/11.1.0/db_1/bin/lsnrctl status LISTENER_DB1

LSNRCTL for Solaris: Version 11.1.0.6.0 - Production on 19-MAY-2010 03:51:57


Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=vzfreak2a)(PORT=1522)))STATUS of the LISTENER------------------------Alias LISTENER_DB1Version TNSLSNR for Solaris: Version 11.1.0.6.0 - ProductionStart Date 19-MAY-2010 03:37:02Uptime 0 days 0 hr. 14 min. 55 secTrace Level offSecurity ON: Local OS AuthenticationSNMP OFFListener Parameter File /u01/app/ora11g/product/11.1.0/db_1/network/admin/listener.oraListener Log File /u01/app/ora11g/diag/tnslsnr/pfreak1/listener_db1/alert/log.xmlListening Endpoints Summary... (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=pfreak1)(PORT=1522))) (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC1522)))Services Summary...

39

Service "DB1" has 1 instance(s). Instance "DB1", status READY, has 1 handler(s) for this service...Service "DB1XDB" has 1 instance(s). Instance "DB1", status READY, has 1 handler(s) for this service...Service "DB1_XPT" has 1 instance(s). Instance "DB1", status READY, has 1 handler(s) for this service...The command completed successfully-bash-3.00$ exitlogoutbash-3.00#bash-3.00# clrs create -g ora-11gr1-rg -t SUNW.oracle_listener -p listener_name=LISTENER_DB1 -p oracle_home=/u01/app/ora11g/product/11.1.0/db_1 -p resource_dependencies=ora-db1-lh -d ora-db1-lsrbash-3.00# clrs enable ora-db1-lsrbash-3.00# clrs status -g ora-11gr1-rg=== Cluster Resources ===


ora-db1-lh pfreak1 Online Online - LogicalHostname online. pfreak2 Offline Offline

ora-db1-lsr pfreak1 Online Online pfreak2 Offline Offline

bash-3.00#

Switch the 11gR1 Resource Group




asm-inst-rg pfreak1 No Online pfreak2 No Online

asm-dg-rg pfreak1 No Online pfreak2 No Online

ora-11gr1-rg pfreak1 No Online pfreak2 No Offline

40

bash-3.00# clrg switch -n pfreak2 ora-11gr1-rgbash-3.00# clrg status





ora-11gr1-rg pfreak1 No Offline pfreak2 No Online

bash-3.00# clrs status -g ora-11gr1-rg


Resource Name Node Name State Status Message------------- --------- ----- --------------ora-db1-rs pfreak1 Offline Offline pfreak2 Online Online

ora-db1-lh pfreak1 Offline Offline - LogicalHostname offline. pfreak2 Online Online - LogicalHostname online.

ora-db1-lsr pfreak1 Offline Offline pfreak2 Online Online

bash-3.00#

Register and enable the 11gR2 database resource

Note that for a single insatnce 11gR2 database we do not need to specify the ALERT_LOG_FILE and CONNECT_STRING extension properties. However, it is still okay to specify those extension properties, they would just be ignored for a single instance 11gR2 database.

bash-3.00# clrg create ora-11gr2-rgbash-3.00# clrg set -p rg_affinities=++asm-dg-rg ora-11gr2-rgbash-3.00# clrs create -g ora-11gr2-rg -t SUNW.oracle_server -p oracle_home=/u01/app/ora11g/product/11.2.0/dbhome_1 -p oracle_sid=DB2 -p db_unique_name=DB2 -p resource_dependencies_offline_restart=asm-dg-rs -d ora-db2-rsbash-3.00#

41

bash-3.00# clrg online -eM ora-11gr2-rgbash-3.00# clrs status -g ora-11gr2-rg



bash-3.00#

Also note that the preferred listener for a single instance 11gR2 database is to use the Grid Infrastructure for Clusters SCAN Listener. Briefly, the SIDB registers itself to a local Oracle listener. Note that local Oracle listeners are each running on all cluster nodes, however the SIDB only registers itself to one local listener at any one time. As the SIDB is switched or failed over between nodes it will unregister itself from the local listener on the node where it was running and register itself with the local listener on the node where it is now running. SCAN provides the ability to present the client with a single name of where the SIDB is running, regardless of where the SIDB is located within the cluster. As such when the client tries to connect to the SIDB the SCAN GI resource determines which local Oracle listener is listening for the SIDB service and returns that address to the client. The client then continues to connect with the SID using the address provided by SCAN.

If the administrator does not want to deploy or leverage SCAN, then the existing SUNW.oracle_listener RT can be deployed and does not require development changes. Nevertheless, the 11gR2 SIDB needs to have it's local_listener system parameter changed to reflect the local listener that is then managed by the SUNW.oracle_listener RT. Appropriate tnsnames.ora and listener.ora files need to be defined to reflect the local_listener definition as described within the SC/OSC HA-Oracle documentation.

By default, when creating the 11gR2 database with dbca, the following local_listener entry is defined. Note that earlier we also modified the remote_listener to use SCAN.






42





remote_listener string freak-scan-lh:1521SQL> exitDisconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit ProductionWith the Partitioning, Automatic Storage Management, OLAP, Data Miningand Real Application Testing options-bash-3.00$ exitlogoutbash-3.00# su - oragridSun Microsystems Inc. SunOS 5.10 Generic January 2005


-bash-3.00$ lsnrctl status



Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER)))STATUS of the LISTENER------------------------Alias LISTENERVersion TNSLSNR for Solaris: Version 11.2.0.1.0 - ProductionStart Date 19-MAY-2010 08:04:37Uptime 0 days 0 hr. 20 min. 16 secTrace Level offSecurity ON: Local OS AuthenticationSNMP OFFListener Parameter File /u01/app/11.2.0/grid/network/admin/listener.oraListener Log File /u01/app/oragrid/diag/tnslsnr/pfreak1/listener/alert/log.xmlListening Endpoints Summary... (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LISTENER))) (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=10.11.188.56)(PORT=1521)))

43

(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=10.11.188.58)(PORT=1521)))Services Summary...Service "+ASM" has 1 instance(s). Instance "+ASM1", status READY, has 1 handler(s) for this service...Service "DB2" has 1 instance(s). Instance "DB2", status READY, has 1 handler(s) for this service...Service "DB2XDB" has 1 instance(s). Instance "DB2", status READY, has 1 handler(s) for this service...The command completed successfully-bash-3.00$

-bash-3.00$ $ORACLE_HOME/bin/lsnrctl status LISTENER_SCAN1

LSNRCTL for Solaris: Version 11.2.0.1.0 - Production on 03-JUN-2010 07:46:01


Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER_SCAN1)))STATUS of the LISTENER------------------------Alias LISTENER_SCAN1Version TNSLSNR for Solaris: Version 11.2.0.1.0 - ProductionStart Date 03-JUN-2010 06:11:13Uptime 0 days 1 hr. 34 min. 47 secTrace Level offSecurity ON: Local OS AuthenticationSNMP OFFListener Parameter File /u01/app/11.2.0/grid/network/admin/listener.oraListener Log File /u01/app/11.2.0/grid/log/diag/tnslsnr/pover2/listener_scan1/alert/log.xmlListening Endpoints Summary... (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LISTENER_SCAN1))) (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=10.11.150.149)(PORT=1521)))Services Summary...Service "DB2" has 1 instance(s). Instance "DB2", status READY, has 1 handler(s) for this service...Service "DB2XDB" has 1 instance(s). Instance "DB2", status READY, has 1 handler(s) for this service...The command completed successfully-bash-3.00$

Switch the 11gR2 Resource Group

Also note that after a switch over the local_listener is updated by Grid Infrastructure to ensure that the database connects to the local listener on the new target node, e.g.

bash-3.00# clrs status -g ora-11gr2-rg


44


bash-3.00# clrg switch -n pfreak2 ora-11gr2-rg bash-3.00# clrs status -g ora-11gr2-rg


Resource Name Node Name State Status Message------------- --------- ----- --------------ora-db2-rs pfreak1 Offline Offline pfreak2 Online Online

bash-3.00#bash-3.00# uname -n pfreak2bash-3.00#bash-3.00# su - oragridSun Microsystems Inc. SunOS 5.10 Generic January 2005









remote_listener string freak-scan-lh:1521SQL> exitDisconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production

45

With the Partitioning, Automatic Storage Management, OLAP, Data Miningand Real Application Testing options-bash-3.00$

bash-3.00# uname -n pfreak2bash-3.00# su - oragridSun Microsystems Inc. SunOS 5.10 Generic January 2005


-bash-3.00$ lsnrctl status



Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER)))STATUS of the LISTENER------------------------Alias LISTENERVersion TNSLSNR for Solaris: Version 11.2.0.1.0 - ProductionStart Date 19-MAY-2010 09:05:45Uptime 0 days 0 hr. 26 min. 53 secTrace Level offSecurity ON: Local OS AuthenticationSNMP OFFListener Parameter File /u01/app/11.2.0/grid/network/admin/listener.oraListener Log File /u01/app/oragrid/diag/tnslsnr/pfreak2/listener/alert/log.xmlListening Endpoints Summary... (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LISTENER))) (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=10.11.188.57)(PORT=1521))) (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=10.11.188.59)(PORT=1521)))Services Summary...Service "+ASM" has 1 instance(s). Instance "+ASM2", status READY, has 1 handler(s) for this service...Service "DB2" has 1 instance(s). Instance "DB2", status READY, has 1 handler(s) for this service...Service "DB2XDB" has 1 instance(s). Instance "DB2", status READY, has 1 handler(s) for this service...The command completed successfully-bash-3.00$ -bash-3.00$ crsctl stat res ora.db2.db -t--------------------------------------------------------------------------------NAME TARGET STATE SERVER STATE_DETAILS --------------------------------------------------------------------------------Cluster Resources

46

--------------------------------------------------------------------------------ora.db2.db 1 ONLINE ONLINE pfreak2 Open -bash-3.00$

Final picture






ora-11gr1-rg pfreak1 No Online pfreak2 No Offline

ora-11gr2-rg pfreak1 No Offline pfreak2 No Online

bash-3.00# clrs list -g ora-11gr1-rgora-db1-lhora-db1-lsrora-db1-rsbash-3.00# clrs list -g ora-11gr2-rgora-db2-rsbash-3.00# clrs status





asm-inst-rs pfreak1 Online Online - +ASM1 is UP and

47

ENABLED pfreak2 Online Online - +ASM2 is UP and ENABLED

asm-dg-rs pfreak1 Online Online - Mounted: DATA pfreak2 Online Online - Mounted: DATA

ora-db1-lh pfreak1 Online Online - LogicalHostname online. pfreak2 Offline Offline - LogicalHostname offline.

ora-db1-lsr pfreak1 Online Online pfreak2 Offline Offline

ora-db1-rs pfreak1 Online Online pfreak2 Offline Offline

ora-db2-rs pfreak1 Offline Offline pfreak2 Online Online

bash-3.00#

48

rac checklist 11gr2 grid infrastructure for clusters

Documents