contentsappsdba.info/docs/rac/install/install_rac_using_oraclevm_on_oel6… · configure scan dns...
Post on 27-Jul-2018
235 Views
Preview:
TRANSCRIPT
Author – A.Kishore
http://www.appsdba.info
CONTENTS
Introduction ............................................................................................................................................................ 2
Oracle RAC Software .......................................................................................................................................... 3
Oracle VirtualBox Software ................................................................................................................................ 4
Oracle Enterprise Linux 6.7 software ................................................................................................................. 4
Oracle VirtualBox Installation ............................................................................................................................. 8
Virtual Machine Setup ........................................................................................................................................ 8
Guest Operating System Installation ................................................................................................................ 23
Prepare Steps........................................................................................................................................................ 46
Edit the “hosts” file of rac1 . ......................................................................................................................... 46
Install Guest Additions .................................................................................................................................. 47
Configure SCAN DNS for RAC 11G RAC using dnsmasq in OEL 6 .................................................................. 51
Disable Firewall ............................................................................................................................................. 56
Disable secure linux ...................................................................................................................................... 56
Install oracle-rdbms-server-11gR2-preinstall using yum .............................................................................. 57
Deconfigure Network Time Protocol ............................................................................................................ 57
Install and Configure Automatic Storage Management and Disks. .............................................................. 61
Verify ASMLib Packages. ............................................................................................................................... 62
Configuring and Loading the ASMLib Packages. ........................................................................................... 62
Backup VIRTUAL MACHINE. .......................................................................................................................... 66
TO add shared disks click on storage ............................................................................................................ 68
Clone the Virtual Machine ............................................................................................................................ 75
Configure the Shared Disks ........................................................................................................................... 93
Create ASM disks. ......................................................................................................................................... 96
Take Snapshot node1 and nodes2 ................................................................................................................ 99
Grid Installation .................................................................................................................................................. 102
NFS sharing of the software ....................................................................................................................... 103
Install the missing package on node1 and node2 ....................................................................................... 105
Setup SSH .................................................................................................................................................... 105
Verify using Cluvfy utility ............................................................................................................................ 106
KICK OFF THE INSTALLATION ...................................................................................................................... 107
Post-Install Actions ..................................................................................................................................... 133
Author – A.Kishore
http://www.appsdba.info
Install Database .................................................................................................................................................. 136
References .......................................................................................................................................................... 149
Problems & Solutions ........................................................................................................................................ 149
INTRODUCTION
Here is my setup, I have 16 GB Desktop with Linux installed and with 250 SSD on which the
installation was done. I have created 2 VMs using Oracle Virtual Machine
If you want to get through all steps of the Oracle RAC installation and your laptop or desktop
computer has 8 GB or more of RAM, then this is entirely feasible using Oracle VirtualBox as
demonstrated in this article. You can get a running RAC system which can host a small test database.
The created system is not, and should not be considered, a production-ready system. It's simply to
allow you to get used to installing and using RAC and test various administration procedures. The
article also explains how to save the images and restore RAC from the images in a matter of minutes.
Even if you break your test system, it will be easy to restore.
We have used dnsmasq for scan configuration, we can now have 3 scan listeners on our 2 node
RAC. I had to refer multiple docs to make it work and appreciate all the people who have contributed.
This article uses the 64-bit versions of Oracle Linux, version 6.7, and Oracle 11g Release 2, version
11.2.0.3. Using VirtualBox you can run multiple Virtual Machines (VMs) on a single server, allowing
you to run both RAC nodes on a single machine. In addition, it allows you to set up shared virtual
disks. The finished system includes two guest operating systems, two sets of Oracle Grid
Infrastructure (Clusterware + ASM) and two Database instances all on a single server.
Linux Desktop
16 GB RAM / 250 SSD
RAC1 - Linux 6.7
3 GB RAM
RAC2 - Linux 6.7
3 GB RAM
Author – A.Kishore
http://www.appsdba.info
This article was originally inspired by the article "Oracle Database 11g Release 2 RAC On Linux
Using VirtualBox" written by Sergei Romanenko and published in his blog. Then it was almost entirely
revised and reworked, now this article seems to have very little resemblance with the original work.
Note. When this article was written, Oracle Database 11g Release 2 (11.2.0.3) for Linux 64-bit (both
clusterware and database) was available through the Oracle support to licensed customers only.
As of now, 11.2.0.3 can be downloaded in Oracle support site, in "Patches & Updates", then select
"Latest Patchsets", then select "Oracle Database", then select "Linux x86-64", then select
"11.2.0.3.0". The number of this patch set is 10404530, it is possible to jump to the download page
using this number. This patch set is a full installation of the Oracle Database software. This means
that you do not need to install Oracle Database 11g Release 2 (11.2.0.1) before installing Oracle
Database 11g Release 2 (11.2.0.3). For installing RAC database you will need only 3 files:
ORACLE RAC SOFTWARE
I have used 11.2.0.3 which comes as a patch 10404530, if you are not having support userid you can
download 11.2.0.1 from otn.oracle.com
11.2.0.3 software
Oracle Database (includes Oracle Database and Oracle RAC), part 1:
p10404530_112030_Linux-x86-64_1of7.zip 1.3G
Oracle Database (includes Oracle Database and Oracle RAC), part 2:
p10404530_112030_Linux-x86-64_2of7.zip 1.1G
Oracle Grid Infrastructure (includes Oracle ASM, Oracle Clusterware):
p10404530_112030_Linux-x86-64_3of7.zip 933M
Author – A.Kishore
http://www.appsdba.info
11.2.0.1 Software from OTN
http://otn.oracle.com
Oracle 11g client on Windows for testing TAF (transaction application failover)
ORACLE VIRTUALBOX SOFTWARE
On Linux host
VirtualBox-5.0-5.0.2_102096_el5-1.x86_64.rpm
ORACLE ENTERPRISE LINUX 6.7 SOFTWARE
http://edelivery.oracle.com/linux
Click Sign In/Register
Author – A.Kishore
http://www.appsdba.info
1.1.2. Enter Username & Password
Author – A.Kishore
http://www.appsdba.info
Accept Terms & Restrictions, Click Continue
Select Oracle Linux from Product Pack & select x86 (64 bit) from Platform, Click Go
Author – A.Kishore
http://www.appsdba.info
Select Oracle Linux Release 6 Update 7 Media Pack for x86 (64 bit), Click Continue.
V77197-01.iso
Note:, you need to burn them using the image option available with your cd-writer software
Refer: http://www.petri.co.il/how_to_write_iso_files_to_cd.htm
Author – A.Kishore
http://www.appsdba.info
ORACLE VIRTUALBOX INSTALLATION
rpm -Uvh VirtualBox-5.0-5.0.2_102096_el5-1.x86_64.rpm
Create below folders on the HOST machine
cd /d05/oracle/RAC
mkdir rac1 rac2 asm
VIRTUAL MACHINE SETUP
Go to System Tools – Oracle VM VirtualBox
Author – A.Kishore
http://www.appsdba.info
Now we must define the two virtual RAC nodes. We can save time by defining one VM, then cloning it when it is installed.
Start VirtualBox and click the "New" button on the toolbar. Click the "Next" button on the first page of the Virtual Machine Wizard.
Author – A.Kishore
http://www.appsdba.info
Enter the name "rac1", OS "Linux" and Version "Oracle (64 bit)", then click the "Next" button.
Author – A.Kishore
http://www.appsdba.info
Set Base Memory to 3 GB or more, then click the “Next” button
Author – A.Kishore
http://www.appsdba.info
Accept the default option to create a new virtual hard disk by clicking the “Create” button
Author – A.Kishore
http://www.appsdba.info
Select the default VMDK type and click the “Next” button on the Create Virtual Hard Drive screen
Select Dynamically Allocated
Author – A.Kishore
http://www.appsdba.info
Provide the location where you want to save the VM files. In my case I have provided path as below. Also
increase the size of your vm from default to say 30GB and Click “Create”
Author – A.Kishore
http://www.appsdba.info
The "rac1" VM will appear on the left hand pane. Scroll down the "Details" tab on the right
Author – A.Kishore
http://www.appsdba.info
Click on General->Advanced and change shared clipboard to Bidirectional and Change Drag ‘n’ Drop to
Bidirectional and click Ok
Author – A.Kishore
http://www.appsdba.info
Add two Adaptors.
Make sure “Adapter 1” is enabled, attached to “Bridged Adapter”. This interface will be used
for public network, for example, for connection to the RAC database from other applications.
Author – A.Kishore
http://www.appsdba.info
Then click on the “Adapter 2” tab. Make sure “Adapter 2” is enabled and attach to “Internal Network”. Name
this network “privnet”:
Author – A.Kishore
http://www.appsdba.info
Click on storage to load OEL6.7 ISO
Author – A.Kishore
http://www.appsdba.info
GUEST OPERATING SYSTEM INSTALLATION
Please note that during installation Virtual Box will be keeping the mouse pointer inside VM area. To exit, press Right Control key on the keyboard.
Click on Start as shown below
Author – A.Kishore
http://www.appsdba.info
After starting we get below screen and select Install or Upgrade an Existing System
Select skip
Author – A.Kishore
http://www.appsdba.info
Select basic Storage and click Next
Select Discard any Data
Author – A.Kishore
http://www.appsdba.info
Change the hostname and Scroll down and click on “Configure Network
Author – A.Kishore
http://www.appsdba.info
Select System eth0 and click on edit
Select Connect Automatically and select IPV4 Setting
Author – A.Kishore
http://www.appsdba.info
Make sure that “Connect automatically” is checked. Select “IPv4 Settings” tab; change Method to “DHCP”;
Press “Add” and fill Address: 192.168.1.6; Netmask: 255.255.255.0; Gateway: 192.168.1.1 Press “Apply” then
done:
In the Network Connections screen select “System eth1” interface, this will be used for private network, then
press “Edit”. Then check the box “Connect automatically”. Select “IPv4 Settings” tab; change Method to
“Manual”. Press “Add” and fill Address: 192.168.0.100; Netmask: 255.255.255.0; Gateway: 0.0.0.0. When
done, press “Apply”:
Author – A.Kishore
http://www.appsdba.info
Finally select “System eth2” interface, this will be used for Internet, then press “Edit”. Check the box “Connect
automatically”. Select “IPv4 Settings” tab make sure the Method is set to “Automatic (DHCP)”.Press “Apply”
button:
Click Next
Author – A.Kishore
http://www.appsdba.info
Give password for root
Select Replace Existing Linux
Author – A.Kishore
http://www.appsdba.info
Select Write Changes to disk
Select Database Server option and select Customize now and Click Next
Author – A.Kishore
http://www.appsdba.info
Click all the options under Desktops and Click Next
Author – A.Kishore
http://www.appsdba.info
Click Yes
Reboot the Guest Machine
Author – A.Kishore
http://www.appsdba.info
PREPARE STEPS
First of all check whether we can connect to the internet or not
ping www.google.com
PING www.google.com (74.125.130.103) 56(84) bytes of data.
64 bytes from sb-in-f103.1e100.net (74.125.130.103): icmp_seq=1 ttl=46 time=63.4 ms
64 bytes from sb-in-f103.1e100.net (74.125.130.103): icmp_seq=2 ttl=46 time=63.5 ms
64 bytes from sb-in-f103.1e100.net (74.125.130.103): icmp_seq=3 ttl=46 time=62.2 ms
EDIT THE “HOSTS” FILE OF RAC1 .
In this chapter we are going to prepare RAC1 and later clone to RAC2
Follow the below steps on rac1
vi /etc/hosts
127.0.0.1 localhost.localdomain localhost
# Public Network - (eth0)
192.168.1.6 rac1.appsdba.info rac1
192.168.1.7 rac2.appsdba.info rac2
# Private Interconnect - (eth1)
192.168.0.100 rac1-priv.appsdba.info rac1-priv
192.168.0.101 rac2-priv.appsdba.info rac2-priv
# Public Virtual IP (VIP) addresses for - (eth0)
192.168.1.200 rac1-vip.appsdba.info rac1-vip
192.168.1.201 rac2-vip.appsdba.info rac2-vip
Author – A.Kishore
http://www.appsdba.info
INSTALL GUEST ADDITIONS
Click Ok
Author – A.Kishore
http://www.appsdba.info
If It fails. We need to update packages
Solution:
To overcome this error update gcc using
yum update gcc
yum update - It will take 20 minutes
yum install kernel-uek-devel
Note: Reboot the Server once above steps are completed
Author – A.Kishore
http://www.appsdba.info
Once again try to install VM tools
Purpose we will be able to view the shared folders in Linux Guest
Author – A.Kishore
http://www.appsdba.info
CONFIGURE SCAN DNS FOR RAC 11G RAC USING DNSMASQ IN OEL 6
Install and configure dnsmasq
1. To install dnsmasq run as root following command
yum install dnsmasq
Loaded plugins: refresh-packagekit, security
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package dnsmasq.x86_64 0:2.48-13.el6 will be updated
---> Package dnsmasq.x86_64 0:2.48-14.el6 will be an update
--> Finished Dependency Resolution
Dependencies Resolved
================================================================================
Package Arch Version Repository Size
================================================================================
Updating:
dnsmasq x86_64 2.48-14.el6 public_ol6_latest 149 k
Transaction Summary
================================================================================
Upgrade 1 Package(s)
Total download size: 149 k
Is this ok [y/N]: y
Downloading Packages:
dnsmasq-2.48-14.el6.x86_64.rpm | 149 kB 00:00
warning: rpmts_HdrFromFdno: Header V3 RSA/SHA256 Signature, key ID ec551f03: NOK EY
Retrieving key from http://public-yum.oracle.com/RPM-GPG-KEY-oracle-ol6
Importing GPG key 0xEC551F03:
Userid: "Oracle OSS group (Open Source Software group) <build@oss.oracle.com>"
From : http://public-yum.oracle.com/RPM-GPG-KEY-oracle-ol6
Author – A.Kishore
http://www.appsdba.info
Is this ok [y/N]: y
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Updating : dnsmasq-2.48-14.el6.x86_64 1/2
Cleanup : dnsmasq-2.48-13.el6.x86_64 2/2
Verifying : dnsmasq-2.48-14.el6.x86_64 1/2
Verifying : dnsmasq-2.48-13.el6.x86_64 2/2
Updated:
dnsmasq.x86_64 0:2.48-14.el6
Complete!
2. Configure dnsmasq
create new file “/etc/racdns” with settings for SCAN
[root@rac1 ~]# vi /etc/racdns
#scan
192.168.1.20 rac-scan rac-scan.appsdba.info
192.168.1.21 rac-scan rac-scan.appsdba.info
192.168.1.22 rac-scan rac-scan.appsdba.info
modifydnsmasq default configuration file “/etc/dnsmasq.conf”. One parameter addn-hosts should be changed to point to file “/etc/racdns”.
cat /etc/dnsmasq.conf | grep addn-hosts
addn-hosts=/etc/racdns
3. Start dnsmasq
service dnsmasq start
chkconfig dnsmasq on
Author – A.Kishore
http://www.appsdba.info
4. Next step is to resolve problem with file “/etc/resolv.conf”
I’m using third network card as NAT with DHCP so each time you restart network card or reboot host the file is overwritten with automatically generated settings. Nameserver points for 192.168.1.1 which is required to resolve internet entries but not enough to resolve SCAN settings via dnsmasq.
cat /etc/resolv.conf
# Generated by NetworkManager
search appsdba.info
nameserver 192.168.1.1
As default dnsmasq is running on ipadress 127.0.0.1 so it requires following settings in “/etc/resolv.conf”.
cat /etc/resolv.conf
# Generated by NetworkManager
nameserver 127.0.0.1
search appsdba.info
nameserver 192.168.1.1
You must protect the file from being automatically overwritten by host reboot, network card restart etc.
[root@rac1 ~]# chattr +i /etc/resolv.conf
To write we use below command
chattr -i /etc/resolv.conf
Author – A.Kishore
http://www.appsdba.info
Verification
It’s just running nslookup to verify all is working fine
nslookup rac-scan
Server: 127.0.0.1
Address: 127.0.0.1#53
Name: rac-scan.appsdba.info
Address: 192.168.1.21
Name: rac-scan.appsdba.info
Address: 192.168.1.22
Name: rac-scan.appsdba.info
Address: 192.168.1.20
[root@rac1 ~]# nslookup rac-scan
Server: 127.0.0.1
Address: 127.0.0.1#53
Name: rac-scan.appsdba.info
Address: 192.168.1.22
Name: rac-scan.appsdba.info
Address: 192.168.1.20
Name: rac-scan.appsdba.info
Address: 192.168.1.21
Author – A.Kishore
http://www.appsdba.info
[root@rac1 ~]# nslookup rac1
Server: 127.0.0.1
Address: 127.0.0.1#53
Name: rac1.appsdba.info
Address: 192.168.1.6
[root@rac1 ~]# nslookup rac1-priv
Server: 127.0.0.1
Address: 127.0.0.1#53
Name: rac1-priv.appsdba.info
Address: 192.168.0.100
[root@rac1 ~]# nslookup rac1-vip
Server: 127.0.0.1
Address: 127.0.0.1#53
Name: rac1-vip.appsdba.info
Address: 192.168.1.200
Author – A.Kishore
http://www.appsdba.info
DISABLE FIREWALL
Now we need to disable the firewall: Linux Main menu | System | Administration | Firewall. Click on “Disable”
icon, then on “Apply”.
Click on "Disable" icon, then on "Apply".
DISABLE SECURE LINUX
Disable secure linux by editing the “/etc/selinux/config” file, making sure the SELINUX flag
is set as follows.
SELINUX=permissive
Author – A.Kishore
http://www.appsdba.info
INSTALL ORACLE-RDBMS-SERVER-11GR2-PREINSTALL USING YUM
yum install oracle-rdbms-server-11gR2-preinstall
DECONFIGURE NETWORK TIME PROTOCOL
Either configure NTP, or make sure it is not configured so the Oracle Cluster Time Synchronization Service
(ctssd) can synchronize the times of the RAC nodes. If you want to deconfigure NTP do the following.
Below steps will disable NTP - we have followed below steps only
# service ntpd stop
Shutting down ntpd: [ OK ]
# chkconfig ntpd off
# mv /etc/ntp.conf /etc/ntp.conf.orig
# rm /var/run/ntpd.pid
Below steps will enable NTP
If you want to use NTP, you must add the “-x” option into the following line in the “/etc/sysconfig/ntpd” file.
OPTIONS="-x -u ntp:ntp -p /var/run/ntpd.pid"
Then restart NTP.
# service ntpd restart
Author – A.Kishore
http://www.appsdba.info
Edit the “sysctl.conf” of rac1 with the following values.
vi /etc/sysctl.conf
kernel.core_uses_pid = 1
kernel.shmmax=4294967295
kernel.shmmni=4096
kernel.sem=256 32000 100 142
fs.file-max = 6815744
kernel.msgmnb = 65535
kernel.msgmni = 2878
net.core.rmem_default = 262144
net.core.rmem_max= 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048576
net.ipv4.tcp_wmem = 262144 262144 262144
net.ipv4.tcp_rmem = 262144 262144 262144
fs.aio-max-nr = 1048576
net.ipv4.ip_local_port_range = 9000 65500
Author – A.Kishore
http://www.appsdba.info
Edit the “limits.conf” file of rac1 with the following values.
vi /etc/security/limits.conf
* hard nofile 327680
* soft nofile 262144
* hard nproc 327680
* soft nproc 262144
* hard memlock 3145728
* soft memlock 3145728
* hard stack 16384
* soft stack 10240
Create "oracle/grid" User and Directories in both nodes.
groupadd -g 1000 oinstall
groupadd -g 1020 asmadmin
groupadd -g 1021 asmdba
groupadd -g 1022 asmoper
groupadd -g 1031 dba
useradd -u 1100 -g oinstall -G asmoper,asmdba,asmadmin,dba grid
userdel oracle
useradd -u 1101 -g oinstall -G dba,asmdba oracle
cd /home
chown -R oracle:dba oracle
Author – A.Kishore
http://www.appsdba.info
Create directories and grant permissions
cd /home
chown -R oracle:oinstall oracle/
mkdir -p /u01/app/grid
chown -R grid:oinstall /u01/app/
chmod -R 775 /u01/app/grid
mkdir -p /u01/app/oracle
chown oracle:oinstall /u01/app/oracle
chmod -R 775 /u01/app/oracle
Change the password of grid and oracle users
passwd grid
passwd oracle
passwd grid
Changing password for user grid.
New password:
BAD PASSWORD: it is too short
BAD PASSWORD: is too simple
Retype new password:
passwd: all authentication tokens updated successfully.
[root@rac1 home]# passwd oracle
Changing password for user oracle.
New password:
[root@rac1 home]# passwd oracle
Changing password for user oracle.
New password:
BAD PASSWORD: it is based on a dictionary word
BAD PASSWORD: is too simple
Author – A.Kishore
http://www.appsdba.info
Retype new password:
passwd: all authentication tokens updated successfully.
INSTALL AND CONFIGURE AUTOMATIC STORAGE MANAGEMENT AND DISKS.
Install Oracleasmlib. I have downloaded file from
http://www.oracle.com/technetwork/server-storage/linux/asmlib/ol6-1709075.html
yum install oracleasm-support
oracleasmlib-2.0.4-1.el6.i686.rpm
rpm -Uvh oracleasmlib-2.0.4-1.el6.i686.rpm
Preparing... ########################################### [100%]
1:oracleasmlib ########################################### [100%]
Author – A.Kishore
http://www.appsdba.info
VERIFY ASMLIB PACKAGES.
This installation needs to be performed on all nodes as the root user account:
rpm -qa |grep asm
oracleasmlib-2.0.4-1.el6.i686
oracleasm-support-2.1.8-1.el6.x86_64
CONFIGURING AND LOADING THE ASMLIB PACKAGES.
Now that we downloaded and installed the ASMLib Packages for Linux, we need to configure and load the ASM kernel module. This task needs to be run on all nodes as root:
$ su –
/etc/init.d/oracleasm configure
/etc/init.d/oracleasm configure
Configuring the Oracle ASM library driver.
This will configure the on-boot properties of the Oracle ASM library
driver. The following questions will determine whether the driver is
loaded on boot and what permissions it will have. The current values
will be shown in brackets ('[]'). Hitting <ENTER> without typing an
answer will keep that current value. Ctrl-C will abort.
Default user to own the driver interface []: grid
Default group to own the driver interface []: asmadmin
Start Oracle ASM library driver on boot (y/n) [n]: y
Author – A.Kishore
http://www.appsdba.info
Scan for Oracle ASM disks on boot (y/n) [y]: y
Writing Oracle ASM library driver configuration: done
Initializing the Oracle ASMLib driver: [ OK ]
Scanning the system for Oracle ASMLib disks: [ OK ]
Login as the grid user and add the following lines at the end of the .bash_profile file.
su - grid
vi .bash_profile
# Oracle Settings
TMP=/tmp; export TMP
TMPDIR=$TMP; export TMPDIR
ORACLE_HOSTNAME=rac1.appsdba.info; export ORACLE_HOSTNAME
ORACLE_BASE=/u01/app/grid; export ORACLE_BASE
GRID_HOME=/u01/app/11.2.0/grid; export GRID_HOME
ORACLE_SID=+ASM1; export ORACLE_SID
ORACLE_HOME=$GRID_HOME; export ORACLE_HOME
PATH=$PATH:$ORACLE_HOME/bin; export PATH
ORACLE_TERM=xterm; export ORACLE_TERM
LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib; export LD_LIBRARY_PATH
CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib; export CLASSPATH
if [ $USER = "oracle" ]; then
if [ $SHELL = "/bin/ksh" ]; then
ulimit -p 16384
Author – A.Kishore
http://www.appsdba.info
ulimit -n 65536
else
ulimit -u 16384 -n 65536
fi
fi
Login as the oracle user and add the following lines at the end of the .bash_profile file.
su – oracle
vi .bash_profile
# User specific environment and startup programs
# database name - RACDB
# instance name 1 - RACDB1
# instance name 2 - RACDB2
PATH=$PATH:$HOME/bin
export PATH
PATH=$PATH:$HOME/bin
export PATH
# Oracle Settings
TMP=/tmp; export TMP
TMPDIR=$TMP; export TMPDIR
DB_HOME=/u01/app/oracle/product/11.2.0/dbhome_1
ORACLE_HOSTNAME=lb.appsdba.info; export ORACLE_HOSTNAME
export ORACLE_UNQNAME=RACDB
ORACLE_SID=RACDB1; export ORACLE_SID
ORACLE_HOME=$DB_HOME; export ORACLE_HOME
PATH=$ORACLE_HOME/bin:$PATH; export PATH
Author – A.Kishore
http://www.appsdba.info
LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib; export LD_LIBRARY_PATH
CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib; export CLASSPATH
ORACLE_TERM=xterm; export ORACLE_TERM
if [ $USER = "oracle" ]; then
if [ $SHELL = "/bin/ksh" ]; then
ulimit -p 16384
ulimit -n 65536
else
ulimit -u 16384 -n 65536
fi
fi
Author – A.Kishore
http://www.appsdba.info
BACKUP VIRTUAL MACHINE.
Now we will shut down the server as we need to add asm disks. Before we add asm disks.
We will backup our vm
Click File export Appliance
Author – A.Kishore
http://www.appsdba.info
Select our VM and click next Save to some location
Click next and click export
Ok We done with the backup now add disk’s for ASM storage.
Author – A.Kishore
http://www.appsdba.info
TO ADD SHARED DISKS CLICK ON STORAGE
Author – A.Kishore
http://www.appsdba.info
CLONE THE VIRTUAL MACHINE
Later versions of VirtualBox allow you to clone VMs, but these also attempt to clone the shared disks, which is not what we want. Instead we must manually clone the VM.
Shut down the "rac1" virtual machine using the following command.
# shutdown -h now
If you are using VirtualBox version 4.0.0 or later, so the following.
VBoxManage clonehd rac1.vdi /d05/oracle/RAC/rac2/rac2.vdi
Create rac2
Author – A.Kishore
http://www.appsdba.info
Change the disks to shared that were created earlier for ASM
Author – A.Kishore
http://www.appsdba.info
Do the same for all the ASMDISK’s
Author – A.Kishore
http://www.appsdba.info
Now once this is done click on RAC2 Vm and Storage (Attach the disks)
Author – A.Kishore
http://www.appsdba.info
Now start the Rac2 server make sure RAC1 IS DOWN
The main purpose of starting the server is that we have to configure Network
If we see here it is still rac1. Hostname has to be changed
Author – A.Kishore
http://www.appsdba.info
Change the hostname and reboot rac2
cat /etc/sysconfig/network
NETWORKING=yes
HOSTNAME=rac2.appsdba.info
Start "Network Connections" tool (Main menu | System | Preferences | Network Connections). Edit eth0 and
eth1 interfaces and set in IPv4 addresses 192.168.1.7 and 192.168.0.101 correspondingly.
Author – A.Kishore
http://www.appsdba.info
Check the MAC address of each of the available network connections. Don't worry that they are listed as
"eth2" and "eth3". These are dynamically created connections because the MAC address of the "eth0" and
"eth1" connections is incorrect.
ifconfig -a | grep eth
eth2 Link encap:Ethernet HWaddr 08:00:27:57:F1:AB
eth3 Link encap:Ethernet HWaddr 08:00:27:64:EC:15
Edit the "/etc/sysconfig/network-scripts/ifcfg-eth0", amending only the IPADDR and HWADDR settings
as follows and deleting the UUID entry. Note, the HWADDR value comes from the "eth2" interface
displayed above.
Author – A.Kishore
http://www.appsdba.info
eth2 Link encap:Ethernet HWaddr 08:00:27:57:F1:AB
eth3 Link encap:Ethernet HWaddr 08:00:27:64:EC:15
Edit the "/etc/sysconfig/network-scripts/ifcfg-eth1", amending only the IPADDR and HWADDR settings as
follows and deleting the UUID entry. Note, the HWADDR value comes from the "eth3" interface displayed
above.
If the adapter names do not reset properly, check the HWADDR in the "/etc/udev/rules.d/70-persistent-
net.rules" file. If it is incorrect, amend it to match the settings described above.
init 0
Verify the adapter settings
Author – A.Kishore
http://www.appsdba.info
CONFIGURE THE SHARED DISKS
Start the "rac1" virtual machine by clicking the "Start" button on the toolbar. When the server has started, log in as the root user so you can configure the shared disks. The current disks can be seen by issuing the following commands.
# cd /dev
# lssd*
sda sda1 sda2 sdbsdcsdd
fdisk –l
Use the "fdisk" command to partition the disks sdb to sdf. The following output shows the expected fdisk output for the sdb disk.
# fdisk /dev/sdb
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF
disklabel
Building a new DOS disklabel. Changes will remain in memory only,
until you decide to write them. After that, of course, the previous
content won't be recoverable.
Author – A.Kishore
http://www.appsdba.info
The number of cylinders for this disk is set to 1305.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)
Warning: invalid flag 0x0000 of partition table 4 will be corrected by
w(rite)
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1305, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-1305, default 1305):
Using default value 1305
Command (m for help): p
Disk /dev/sdb: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
Author – A.Kishore
http://www.appsdba.info
/dev/sdb1 1 1305 10482381 83 Linux
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
#
In each case, the sequence of answers is "n", "p", "1", "Return", "Return", "p" and "w".
Once all the disks are partitioned, the results can be seen by repeating the previous "ls" command.
# cd /dev
# lssd*
sda sda1 sda2 sdb sdb1 sdc sdc1 sdd sdd1
#
Author – A.Kishore
http://www.appsdba.info
CREATE ASM DISKS.
Create the ASM disks on any one node as the root user.
/usr/sbin/oracleasm createdisk DISK1 /dev/sdb1
/usr/sbin/oracleasm createdisk DISK2 /dev/sdc1
/usr/sbin/oracleasm createdisk DISK3 /dev/sdd1
Verify that the ASM disks are visible from every node. (Right now it will not work)
# /etc/init.d/oracleasm scandisks
Scanning system for ASM disks: [ OK ]
We can see the disk are now visible to ASM using the "listdisks" command.
# /usr/sbin/oracleasm listdisks
DISK1
DISK2
DISK3
Author – A.Kishore
http://www.appsdba.info
Edit the "/home/oracle/.bash_profile" file on the "rac2" node to correct the ORACLE_SID and ORACLE_HOSTNAME values.
ORACLE_SID=RACDB2; export ORACLE_SID
ORACLE_HOSTNAME=rac2.appsdba.info; export ORACLE_HOSTNAME
Also, amend the ORACLE_SID setting in the "/home/oracle/db_env" and "/home/oracle/grid_env" files.
ORACLE_SID=+ASM2; export ORACLE_SID
Start the "rac1" virtual machine and restart the "rac2" virtual machine. When both nodes have started, check they can both ping all the public and private IP addresses using the following commands.
ping -c 3 rac1
ping -c 3 rac1-priv
ping -c 3 rac2
ping -c 3 rac2-priv
At this point the virtual IP addresses defined in the "/etc/hosts" file will not work, so don't bother testing them.
The virtual machine setup is now complete.
rac-scan
[root@rac2 ~]# nslookup rac2-vip
Server: 127.0.0.1
Address: 127.0.0.1#53
Name: rac2-vip.appsdba.info
Address: 192.168.1.201
[root@rac2 ~]# nslookup rac2
Server: 127.0.0.1
Address: 127.0.0.1#53
Name: rac2.appsdba.info
Address: 192.168.1.7
Author – A.Kishore
http://www.appsdba.info
[root@rac2 ~]# nslookup rac-scan
Server: 127.0.0.1
Address: 127.0.0.1#53
Name: rac-scan.appsdba.info
Address: 192.168.1.21
Name: rac-scan.appsdba.info
Address: 192.168.1.22
Name: rac-scan.appsdba.info
Address: 192.168.1.20
Author – A.Kishore
http://www.appsdba.info
TAKE SNAPSHOT NODE1 AND NODES2
Shutdown node1 and node2
After shutdown go to Oracle VM
Click Snapshots on the right hand side
Author – A.Kishore
http://www.appsdba.info
GRID INSTALLATION
Clean the old installation (on both nodes, if required)
cd /etc/init.d
rm –rf ohasdinit.ohasd
rm -rf /etc/oracle
rm -rf /etc/oraInst.loc
rm -rf /etc/oratab
cd /u01/app
rm -rf oracle
rm -rf /u01/app/*.*
rm -rf /u01/app/11.2.0/grid/*.*
cd /u01/app/grid
rm -rforaInventory
mkdir -p /u01/app/grid
chown -R grid:oinstall /u01/app/
chmod -R 775 /u01/app/grid
mkdir -p /u01/app/oracle
chownoracle:oinstall /u01/app/oracle
chmod -R 775 /u01/app/oracle
rm -rf /u01/app/grid/*
rm -rf /usr/tmp/.oracle/
rm -rf /u01/app/oraInventory/*
Author – A.Kishore
http://www.appsdba.info
-- On one of the node
-- If there are 3 diskgroups then use the below commands
/usr/sbin/oracleasmdeletedisk DISK1
/usr/sbin/oracleasmdeletedisk DISK2
/usr/sbin/oracleasmdeletedisk DISK3
/usr/sbin/oracleasmcreatedisk DISK1 /dev/sdb1
/usr/sbin/oracleasmcreatedisk DISK2 /dev/sdc1
/usr/sbin/oracleasmcreatedisk DISK3 /dev/sdd1
/etc/init.d/oracleasmlistdisks
Always Verify
Public ip’s
Private ip’s
Date between the both the nodes are same
NFS SHARING OF THE SOFTWARE
The software is available on host machine – 192.168.1.31
On the server (host machine)-
vi /etc/exports
/software/RAC_SOFTWARE/ 192.168.1.10(ro)
Service nfs restart
On the client – node1
mkdir –p /software
Author – A.Kishore
http://www.appsdba.info
mount 192.168.1.31:/d04/oracle/RAC_SOFTWARE /software
Software
p10404530_112030_Linux-x86-64_3of7.zip
unzipp10404530_112030_Linux-x86-64_3of7.zip
Author – A.Kishore
http://www.appsdba.info
INSTALL THE MISSING PACKAGE ON NODE1 AND NODE2
cd /software/11gR11203/grid/rpm
rpm -Uvh cvuqdisk-1.0.9-1.rpm
Preparing... ########################################### [100%]
Using default group oinstall to install package
1:cvuqdisk ########################################### [100%]
SETUP SSH
Connect as grid user
cd /software/11gR11203/grid/sshsetup
./sshUserSetup.sh -user grid -hosts "rac1 rac2" -noPromptPassphrase -advanced -exverify
Verifying SSH connectivity has been setup from rac1 to rac1
------------------------------------------------------------------------
IF YOU SEE ANY OTHER OUTPUT BESIDES THE OUTPUT OF THE DATE COMMAND OR IF YOU ARE PROMPTED FOR A PASSWORD HERE, IT MEANS SSH SETUP HAS NOT BEEN SUCCESSFUL.
Tue Sep 1 05:12:35 EDT 2015
------------------------------------------------------------------------
------------------------------------------------------------------------
Verifying SSH connectivity has been setup from rac1 to rac2
------------------------------------------------------------------------
IF YOU SEE ANY OTHER OUTPUT BESIDES THE OUTPUT OF THE DATE COMMAND OR IF YOU ARE PROMPTED FOR A PASSWORD HERE, IT MEANS SSH SETUP HAS NOT BEEN SUCCESSFUL.
Tue Sep 1 05:12:35 EDT 2015
------------------------------------------------------------------------
-Verification from rac1 complete-
------------------------------------------------------------------------
Verifying SSH connectivity has been setup from rac2 to rac1
------------------------------------------------------------------------
IF YOU SEE ANY OTHER OUTPUT BESIDES THE OUTPUT OF THE DATE COMMAND OR IF YOU ARE PROMPTED FOR A PASSWORD HERE, IT MEANS SSH SETUP HAS NOT BEEN SUCCESSFUL.
Tue Sep 1 05:12:35 EDT 2015
Author – A.Kishore
http://www.appsdba.info
------------------------------------------------------------------------
------------------------------------------------------------------------
Verifying SSH connectivity has been setup from rac2 to rac2
------------------------------------------------------------------------
IF YOU SEE ANY OTHER OUTPUT BESIDES THE OUTPUT OF THE DATE COMMAND OR IF YOU ARE PROMPTED FOR A PASSWORD HERE, IT MEANS SSH SETUP HAS NOT BEEN SUCCESSFUL.
Tue Sep 1 05:12:36 EDT 2015
------------------------------------------------------------------------
-Verification from rac2 complete-
SSH verification complete.
VERIFY USING CLUVFY UTILITY
Prior to 11gR2 we would probably use the "runcluvfy.sh" utility in the clusterware root directory to check the prerequisites have been met. If you are intending to configure SSH connectivity using the installer this check should be omitted as it will always fail. If you want to setup SSH connectivity manually, then once it is done you can run the "runcluvfy.sh" with the following command.
[grid@racnode1 ~]$ cd /software/11gR11203/grid
[grid@racnode1 grid]$ sh runcluvfy.sh stage -pre crsinst -n rac1,rac2 -verbose
If you get any failures be sure to correct them before proceeding.
Author – A.Kishore
http://www.appsdba.info
Every ASM disk is divided into allocation units (AU). An AU is the fundamental unit of allocation within a disk group. A file extent consists of one or more AU. An ASM file consists of one or more file extents. When you create a disk group, you can set the ASM AU size to be between 1 MB and 64 MB in powers of two, such as, 1, 2, 4, 8, 16, 32, or 64. Larger AU sizes typically provide performance advantages for data warehouse applications that use large sequential reads.
Author – A.Kishore
http://www.appsdba.info
Only for 11.2.0.3 grid installation (this is already done)
Author – A.Kishore
http://www.appsdba.info
The results of prerequisite checks are shown on the next screen. You should see two warnings and one failure. The failure was caused
by inability to lookup SCAN in DNS and that should be expected. Check "Ignore All" box and press "Next".
Author – A.Kishore
http://www.appsdba.info
/u01/app/oraInventory/orainstRoot.sh
Changing permissions of /u01/app/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.
Changing groupname of /u01/app/oraInventory to oinstall.
The execution of the script is complete.
[root@rac1 ~]# /u01/app/11.2.0/grid/root.sh
Performing root user operation for Oracle 11g
The following environment variables are set as:
ORACLE_OWNER= grid
ORACLE_HOME= /u01/app/11.2.0/grid
Author – A.Kishore
http://www.appsdba.info
Enter the full pathname of the local bin directory: [/usr/local/bin]:
Copying dbhome to /usr/local/bin ...
Copying oraenv to /usr/local/bin ...
Copying coraenv to /usr/local/bin ...
Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params
Creating trace directory
User ignored Prerequisites during installation
OLR initialization - successful
root wallet
root wallet cert
root cert export
peer wallet
profile reader wallet
pa wallet
peer wallet keys
pa wallet keys
peer cert request
pa cert request
peer cert
pa cert
peer root cert TP
profile reader root cert TP
pa root cert TP
peer pa cert TP
pa peer cert TP
profile reader pa cert TP
profile reader peer cert TP
peer user cert
pa user cert
Author – A.Kishore
http://www.appsdba.info
Adding Clusterware entries to upstart
CRS-2672: Attempting to start 'ora.mdnsd' on 'rac1'
CRS-2676: Start of 'ora.mdnsd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'rac1'
CRS-2676: Start of 'ora.gpnpd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rac1'
CRS-2672: Attempting to start 'ora.gipcd' on 'rac1'
CRS-2676: Start of 'ora.cssdmonitor' on 'rac1' succeeded
CRS-2676: Start of 'ora.gipcd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'rac1'
CRS-2672: Attempting to start 'ora.diskmon' on 'rac1'
CRS-2676: Start of 'ora.diskmon' on 'rac1' succeeded
CRS-2676: Start of 'ora.cssd' on 'rac1' succeeded
ASM created and started successfully.
Disk Group DATA created successfully.
clscfg: -install mode specified
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
CRS-4256: Updating the profile
Successful addition of voting disk 5f5866ae4c0e4fafbffab1ab690a130b.
Successfully replaced voting disk group with +DATA.
CRS-4256: Updating the profile
CRS-4266: Voting file(s) successfully replaced
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE 5f5866ae4c0e4fafbffab1ab690a130b (/dev/oracleasm/disks/DISK1) [DATA]
Located 1 voting disk(s).
CRS-2672: Attempting to start 'ora.asm' on 'rac1'
CRS-2676: Start of 'ora.asm' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.DATA.dg' on 'rac1'
CRS-2676: Start of 'ora.DATA.dg' on 'rac1' succeeded
Configure Oracle Grid Infrastructure for a Cluster ... succeeded
Author – A.Kishore
http://www.appsdba.info
On node2
/u01/app/oraInventory/orainstRoot.sh
Changing permissions of /u01/app/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.
Changing groupname of /u01/app/oraInventory to oinstall.
The execution of the script is complete.
[root@rac2 ~]# /u01/app/11.2.0/grid/root.sh
Performing root user operation for Oracle 11g
The following environment variables are set as:
ORACLE_OWNER= grid
ORACLE_HOME= /u01/app/11.2.0/grid
Enter the full pathname of the local bin directory: [/usr/local/bin]:
Copying dbhome to /usr/local/bin ...
Copying oraenv to /usr/local/bin ...
Copying coraenv to /usr/local/bin ...
Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params
Creating trace directory
User ignored Prerequisites during installation
OLR initialization - successful
Adding Clusterware entries to upstart
CRS-4402: The CSS daemon was started in exclusive mode but found an active CSS daemon on node rac1, number 1, and is terminating
Author – A.Kishore
http://www.appsdba.info
An active cluster was found during exclusive startup, restarting to join the cluster
Configure Oracle Grid Infrastructure for a Cluster ... succeeded
Author – A.Kishore
http://www.appsdba.info
Hurray :) we have 3 scans
On node1
[root@rac1 ~]# ps -ef |grep tns
root 15 2 0 Sep01 ? 00:00:00 [netns]
grid 15436 1 0 00:25 ? 00:00:00 /u01/app/11.2.0/grid/bin/tnslsnr LISTENER_SCAN2 -inherit
grid 15450 1 0 00:25 ? 00:00:00 /u01/app/11.2.0/grid/bin/tnslsnr LISTENER_SCAN3 -inherit
grid 16190 1 0 00:31 ? 00:00:00 /u01/app/11.2.0/grid/bin/tnslsnr LISTENER -inherit
root 18572 3820 0 00:33 pts/0 00:00:00 greptns
On node2
ps -ef |grep tns
root 15 2 0 Sep01 ? 00:00:00 [netns]
grid 13708 1 0 00:29 ? 00:00:00 /u01/app/11.2.0/grid/bin/tnslsnr LISTENER_SCAN1 -inherit
grid 14410 1 0 00:31 ? 00:00:00 /u01/app/11.2.0/grid/bin/tnslsnr LISTENER -inherit
Author – A.Kishore
http://www.appsdba.info
root 16571 11656 0 00:33 pts/0 00:00:00 greptns
[root@orcl1 ~]# su - grid
cluvfy comp scan -verbose
[grid@rac1 ~]$ cluvfy comp scan -verbose
Verifying scan
Checking Single Client Access Name (SCAN)...
SCAN Name Node Running? ListenerName Port Running?
---------------- ------------ ------------ ------------ ------------ ------------
rac-scan.appsdba.info rac2 true LISTENER_SCAN1 1521 true
rac-scan.appsdba.info rac1 true LISTENER_SCAN2 1521 true
rac-scan.appsdba.info rac1 true LISTENER_SCAN3 1521 true
Checking TCP connectivity to SCAN Listeners...
Node ListenerName TCP connectivity?
------------ ------------------------ ------------------------
rac1 LISTENER_SCAN1 yes
rac1 LISTENER_SCAN2 yes
rac1 LISTENER_SCAN3 yes
TCP connectivity to SCAN Listeners exists on all cluster nodes
Checking name resolution setup for "rac-scan.appsdba.info"...
SCAN Name IP Address Status Comment
------------ ------------------------ ------------------------ ----------
rac-scan.appsdba.info 192.168.56.112 passed
rac-scan.appsdba.info 192.168.56.110 passed
rac-scan.appsdba.info 192.168.56.111 passed
Verification of SCAN VIP and Listener setup passed
Verification of scan was successful.
Author – A.Kishore
http://www.appsdba.info
crsctl stat res -t
[grid@rac1 ~]$ crsctl stat res -t
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATA.dg
ONLINE ONLINE rac1
ONLINE ONLINE rac2
ora.LISTENER.lsnr
ONLINE ONLINE rac1
ONLINE ONLINE rac2
ora.asm
ONLINE ONLINE rac1 Started
ONLINE ONLINE rac2 Started
ora.gsd
OFFLINE OFFLINE rac1
OFFLINE OFFLINE rac2
ora.net1.network
ONLINE ONLINE rac1
ONLINE ONLINE rac2
ora.ons
ONLINE ONLINE rac1
ONLINE ONLINE rac2
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE rac2
Author – A.Kishore
http://www.appsdba.info
ora.LISTENER_SCAN2.lsnr
1 ONLINE ONLINE rac1
ora.LISTENER_SCAN3.lsnr
1 ONLINE ONLINE rac1
ora.cvu
1 ONLINE ONLINE rac1
ora.oc4j
1 ONLINE ONLINE rac1
ora.rac1.vip
1 ONLINE ONLINE rac1
ora.rac2.vip
1 ONLINE ONLINE rac2
ora.scan1.vip
1 ONLINE ONLINE rac2
ora.scan2.vip
1 ONLINE ONLINE rac1
ora.scan3.vip
1 ONLINE ONLINE rac1
POST-INSTALL ACTIONS
By default, the Global Services Daemon (GSD) is not started on the cluster. To start GSD, change
directory to the <CRS_HOME> and issue the following commands:
srvctl enable nodeapps –n rac1
srvctl enable nodeapps –n rac2
srvctl start nodeapps
Successful Oracle Clusterware operation can also be verified using the following command:
[grid@racnode1 oraagent_grid]$ crsctl check crs
CRS-4638: Oracle High Availability Services is online
CRS-4537: Cluster Ready Services is online
Author – A.Kishore
http://www.appsdba.info
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
olsnodes -n
racnode1 1
racnode2 2
srvctl status asm -a
ASM is running on rac2,rac1
ASM is enabled.
ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 3
Total space (kbytes) : 262120
Used space (kbytes) : 2676
Available space (kbytes) : 259444
ID : 720927388
Device/File Name : +DATA
Device/File integrity check succeeded
Device/File not configured
Device/File not configured
Device/File not configured
Device/File not configured
Cluster registry integrity check succeeded
Logical corruption check bypassed due to non-privileged user
Author – A.Kishore
http://www.appsdba.info
crsctl query css votedisk
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE 5f5866ae4c0e4fafbffab1ab690a130b (/dev/oracleasm/disks/DISK1) [DATA]
Located 1 voting disk(s).
crsctl check cluster -all
**************************************************************
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE 5f5866ae4c0e4fafbffab1ab690a130b (/dev/oracleasm/disks/DISK1) [DATA]
Located 1 voting disk(s).
[grid@rac1 ~]$ crsctl check cluster -all
**************************************************************
rac1:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
rac2:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
Author – A.Kishore
http://www.appsdba.info
INSTALL DATABASE
Confirm that the oracle user in this session is a member of the required groups
Connect as oracle user setup SSH
cd /software/11gR11203/grid/sshsetup
./sshUserSetup.sh -user grid -hosts "rac1 rac2" -noPromptPassphrase -advanced -exverify
root@rac1 Desktop]#su - oracle
[oracle@rac1 ~]$ cd /tmp/database/
[oracle@rac1 database]$ ls
doc readme.html rpm sshsetup welcome.html
install responserunInstaller stage
[oracle@rac1 database]$ ./runInstaller
Author – A.Kishore
http://www.appsdba.info
On node1 and node2
/u01/app/oracle/product/11.2.0/dbhome_1/root.sh
Performing root user operation for Oracle 11g
The following environment variables are set as:
ORACLE_OWNER= oracle
ORACLE_HOME= /u01/app/oracle/product/11.2.0/dbhome_1
Enter the full pathname of the local bin directory: [/usr/local/bin]:
The contents of "dbhome" have not changed. No need to overwrite.
The contents of "oraenv" have not changed. No need to overwrite.
The contents of "coraenv" have not changed. No need to overwrite.
Entries will be added to the /etc/oratab file as needed by
Author – A.Kishore
http://www.appsdba.info
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Finished product-specific root actions.
Author – A.Kishore
http://www.appsdba.info
REFERENCES
http://balaoracledba.com/2013/12/23/step-by-step-install-of-oracle-rac-11gr2-on-oel-6-5-on-oracle-vm-virtual-box/
http://dbaora.com/configure-scan-dns-for-rac-11g-rac-12c-using-dnsmasq-in-oel5-oel6-2/
http://www.lab128.com/rac_installation_using_vb/article_text.html#install_grid_infrastructure
https://oracle-base.com/articles/11g/oracle-db-11gr2-rac-installation-on-oracle-linux-6-using-virtualbox#clone_the_vm
http://forums.oracle.com/forums/thread.jspa?messageID=4345930
http://www.oracle.com/technology/pub/articles/wartak-rac-vm_3.html
http://wiki.oracle.com/page/11gR2+RAC+on+a+Mac+-+Part+6 –nice one
PROBLEMS & SOLUTIONS
11gR2 Grid: root.sh fails to start the clusterware on the second node (Doc ID 981357.1)
How to Proceed from Failed 11gR2 Grid Infrastructure (CRS) Installation [ID 942166.1]
Issues and solutions
cd /home/oracle/app/11.2.0/grid/log/linux2
cat alertlinux2.log
[ctssd(27935)]CRS-2409:The clock on host linux2 is not synchronous with the mean
cluster time. No action has been taken as the Cluster Time Synchronization Service is
running in observer mode.
2010-07-24 05:26:20.471
[ctssd(27935)]CRS-2412:The Cluster Time Synchronization Service detects that the local
time is significantly different from the mean cluster time. Details in
/home/oracle/app/11.2.0/grid/log/linux2/ctssd/octssd.log.
2010-07-24 05:26:20.471
[ctssd(27935)]CRS-2409:The clock on host linux2 is not synchronous with the mean
cluster time. No action has been taken as the Cluster Time Synchronization Service is
running in observer mode.
http://www.oracle.com/technology/pub/articles/hunter-rac11gr2-iscsi.html
http://arjudba.blogspot.com/2010/03/in-11gr2-grid-rootsh-fails-with-crs.html
Author – A.Kishore
http://www.appsdba.info
How to Proceed from Failed 11gR2 Grid Infrastructure (CRS) Installation [ID 942166.1]
-- If root.sh has failed, run the below commands to clean up
-- first node
cd /home/oracle/app/11.2.0/grid/crs/install
./rootcrs.pl -verbose -deconfig -force
-- second node
cd /home/oracle/app/11.2.0/grid/crs/install
./rootcrs.pl -verbose -deconfig -force –lastnode
To add vip manually
[root@linux2 bin]# ./srvctl add vip -n linux2 -A 192.168.0.201/255.255.255.0/eth0 -k 1
[root@linux2 bin]# ./srvctlconfigvip -n linux2
TIP - to start the RAC services manually
cd /etc/oracle/scls_scr/racnode1/root/
Modify files crsstart and ohasdstr
change enable to disable
top related