2Oracle Clusterware Architecture

2 - 2

Objectives

• Describe the Oracle Clusterware architecture

• Describe the components of Grid Infrastructure– High Availability Services– Clusterware– Automatic Storage Management

• Understand why ASM and Clusterware are combined in GI

2 - 3

Oracle Grid Infrastructure For a Cluster

• Oracle Clusterware:– Platform-independent facility for starting, stopping, and

managing clusterwide resources– Oracle Clusterware comprises two physical stacks:

— Cluster Ready Services stack— Oracle High Availability Services stack

• Oracle Automatic Storage Management (ASM):– Platform-independent shared storage solution providing

storage for the Clusterware, databases, and most other application or system requirements

2 - 4

Oracle Cluster Registry (OCR)

Node1

OCR cache

CRSDprocess

Clientprocess

Node2

OCR cache

CRSDprocess

Node3

OCR cache

CRSDprocess

Clientprocess

OCRprimary

file

Sharedstorage

OCRsecondary

file

2 - 6

CSS Voting Disk Function

Node1 Node2

CSS CSS

Voting disk

Node3

CSS

Node1 Node2

CSS CSS

Voting disk

Node3

CSS

Nodes cansee each other.

Node3 can no longercommunicate throughprivate interconnect.

Others no longer see its heartbeats and

evict that node by using the voting disk.

I do not see 3.

Node1: I see 1&2.Node2: I see 1&2.

=>We shouldevict 3!

I see 1&2.

I see 3.

I’ve beenevicted!

I’d better stop!

I see1, 2, & 3.

I see1, 2, & 3.

I see 1, 2, & 3.

Split-brain

(RMN)

2 - 7

Oracle Local Registry and High Availability

• The Oracle Local Registry (OLR) is a registry similar to OCR that is located on each node in a cluster.– The OLR contains Clusterware manageability information,

including dependencies between services.

• Oracle High Availability Services uses the OLR to maintain Clusterware resources.

• To check the integrity of the Oracle Local Registry:# ocrcheck -localStatus of Oracle Local Registry is as follows : Version : 3 Total space (kbytes) : 262120 Used space (kbytes) : 2644 Available space (kbytes) : 259476 ID : 250248496 Device/File Name : /u01/app/11.2.0/grid/cdata/host01.olr Device/File integrity check succeeded Local registry integrity check succeeded Logical corruption check succeeded

2 - 8

• Oracle Clusterware is started by the OS init daemon.

• Oracle Clusterware installation modifies /etc/inittab to restart ohasd in the event of a crash.

Oracle Clusterware Initialization

/etc/init.d/init.ohasd

ohasd.bin cssdmonitor

octssd.bin crsd.bin

oraagent.bin gipcd.bin

diskmon.bin mdnsd.bin

ocssd.bin gpnpd.bin

evmd.bin evmlogger.bin

cssdagent oraagent.bin

ologgerd.bin osysmond.bin

ons orarootagent.bin

/sbin/initClusterwarestartup script

Oracle Clusterware processes

Operating systeminit daemon

# cat /etc/inittab..h1:35:respawn:/etc/init.d/init.ohasd run >/dev/null 2>&1 </dev/null 

2 - 9

Clusterware Startup Details

init

oraagentcssdagent

ohasd

cssdmonitororarootagent

cssd

2 - 11

Clusterware Startup: The OHASD orarootagent

ohasd

ctssdosysmond diskmonACFS Driversologgerd

oraagentcssdagent cssdmonitororarootagent

crsd

2 - 13

osysmond

Clusterware Startup Details: The CRSD orarootagent

ohasd

ctssddiskmonACFS Driversologgerd

oraagentcssdagent cssdmonitororarootagent

crsd

network GNS vipSCAN vipNode vip

orarootagent oraagent

2 - 14

osysmond

Clusterware Startup Details:The CRSD oraagent

ohasd

ctssddiskmonACFS Driversologgerd

oraagentcssdagent cssdmonitororarootagent

crsd

SCAN Listener

Node Listener

ASM Instances

ons

orarootagent oraagent

Database Instances

2 - 15

Clusterware Startup Details:The OHASD oraagent

ohasd

gpnpdgipcd evmd ASMmdnsd

oraagentcssdagent cssdmonitororarootagent

2 - 16

The crsctl utility is used to invoke certain OHASD functions.

• To start or stop Oracle Clusterware on all nodes:

• To enable or disable Oracle Clusterware for automatic startup on a specific node:

• To check the status of CRS on the local node:

# crsctl start cluster

# crsctl stop cluster

# crsctl enable crs

# crsctl disable crs

Controlling Oracle Clusterware

# crsctl check cluster

2 - 17

Verifying the Status of Oracle Clusterware

• The crsctl utility can be used to verify the status of Oracle Clusterware on all nodes:

$ crsctl check cluster -all***********************************************************host01:CRS-4537: Cluster Ready Services is onlineCRS-4529: Cluster Synchronization Services is onlineCRS-4533: Event Manager is online***********************************************************host02:CRS-4537: Cluster Ready Services is onlineCRS-4529: Cluster Synchronization Services is onlineCRS-4533: Event Manager is online***********************************************************host03:CRS-4537: Cluster Ready Services is onlineCRS-4529: Cluster Synchronization Services is onlineCRS-4533: Event Manager is online***********************************************************

2 - 18

Viewing the High Availability Services Stack

$ crsctl stat res -init -t---------------------------------------------------------------NAME TARGET STATE SERVER STATE_DETAILS ---------------------------------------------------------------Cluster Resources---------------------------------------------------------------ora.asm 1 ONLINE ONLINE host01 Started ora.cluster_interconnect.haip 1 ONLINE ONLINE host01 ora.crf 1 ONLINE ONLINE host01 ora.crsd 1 ONLINE ONLINE host01 ora.cssd 1 ONLINE ONLINE host01 ora.cssdmonitor 1 ONLINE ONLINE host01 ora.ctssd 1 ONLINE ONLINE host01 OBSERVER ora.evmd 1 ONLINE ONLINE host01 ...

2 - 19

How GPnP Works: Client Database Connections

Database client

DNS

SCANlistener

Listener1

Listener2

Listener3SCANlistener

SCANlistener

GNS

2 - 20

• ASM is a volume manager and file system built in to the Oracle Database server.

• ASM provides a complete shared storage solution for:– Clusterware files

— Voting files— Oracle Cluster Registry (OCR)

– Database files

• ASM permits Clusterware to access the shared OCR file. – The Oracle ASM partnership and status table (PST) is

replicated on multiple disks and is extended to store OCR.– Oracle ASM reserves several blocks at a fixed location on

every Oracle ASM disk for storing the voting disk.

Automatic Storage Management

2 - 21

Summary

In this lesson, you should have learned how to:

• Describe the Oracle Clusterware architecture

• Describe the components of Grid Infrastructure– High Availability Services– Clusterware– Automatic Storage Management

• Understand why ASM and Clusterware are combined in GI