d60488gc11_sg1

Oracle 11g: RAC and Grid Infrastructure Administration AcceleratedVolume I - Student Guide

D60488GC11

Edition 1.1

September 2010

D65231

Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Copyright © 2010, Oracle and/or its affiliates. All rights reserved.

Disclaimer

This document contains proprietary information and is protected by copyright and other intellectual property laws. You may copy and print this document solely for your own use in an Oracle training course. The document may not be modified or altered in any way. Except where your use constitutes "fair use" under copyright law, you may not use, share, download, upload, copy, print, display, perform, reproduce, publish, license, post, transmit, or distribute this document in whole or in part without the express authorization of Oracle.

The information contained in this document is subject to change without notice. If you find any problems in the document, please report them in writing to: Oracle University, 500 Oracle Parkway, Redwood Shores, California 94065 USA. This document is not warranted to be error-free.

Restricted Rights Notice

If this documentation is delivered to the United States Government or anyone using the documentation on behalf of the United States Government, the following notice is applicable:

U.S. GOVERNMENT RIGHTSThe U.S. Government’s rights to use, modify, reproduce, release, perform, display, or disclose these training materials are restricted by the terms of the applicable Oracle license agreement and/or the applicable U.S. Government contract.

Trademark Notice

Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners.

AuthorsJames WomackJames Spiller

Technical Contributors

David BrowerMark FullerBarb LundhildMike LeathermanS. Matt TaylorJean-Francois VerrierRick Wessman

Technical Reviewers

Christopher AndrewsChristian BauwensJonathan CreightonMichael CebullaAl FlournoyAndy FortunakJoel GoodmanMichael HazelPete JonesJerry LeeMarkus MichalewiczPeter SharmanRanbir SinghRichard StrohmJanet SternLinda SmalleyBranislav ValnyDoug Williams

Graphics

Satish Bettegowda

Editors

Nita PavitranArijit GhoshAju KumarDaniel Milne

Publishers

Shaik Mahaboob BashaNita BrozowskiJayanthy Keshavamurthy

Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Contents

1 Oracle Grid Infrastructure Architecture Objectives 1-2 Oracle Grid Infrastructure 1-3 Module 1: Oracle Clusterware Concepts 1-4 What Is a Cluster? 1-5 What Is Clusterware? 1-6 Oracle Clusterware 1-7 Oracle Clusterware Architecture and Services 1-8 Goals for Oracle Clusterware 1-9 Oracle Clusterware Networking 1-10 Interconnect Link Aggregation: Single Switch 1-12 Interconnect Link Aggregation: Multiswitch 1-14 Interconnect NIC Guidelines 1-15 Additional Interconnect Guidelines 1-16 Quiz 1-17 Module 2: Oracle Clusterware Architecture 1-18 Oracle Clusterware Startup 1-19 Oracle Clusterware Process Architecture 1-20 Grid Plug and Play 1-22 GPnP Domain 1-23 GPnP Components 1-24 GPnP Profile 1-25 Grid Naming Service 1-26 Single Client Access Name 1-27 GPnP Architecture Overview 1-29 How GPnP Works: Cluster Node Startup 1-31 How GPnP Works: Client Database Connections 1-32 Quiz 1-33 Module 3: ASM Architecture 1-35 ASM and ASM Cluster File System 1-37 ASM Key Features and Benefits 1-39 ASM Instance Designs: Nonclustered ASM and Oracle Databases 1-40 ASM Instance Designs: Clustered ASM for Clustered Databases 1-41 ASM Instance Designs: Clustered ASM for Mixed Databases 1-42 ASM System Privileges 1-43 ASM OS Groups with Role Separation 1-44 Authentication for Accessing ASM Instances 1-45 Password-Based Authentication for ASM 1-46 Managing the ASM Password File 1-47

iii

Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Using a Single OS Group 1-48 Using Separate OS Groups 1-49 ASM Components: Software 1-50 ASM Components: ASM Instance 1-51 ASM Components: ASM Instance Primary Processes 1-53 ASM Components: Node Listener 1-54 ASM Components: Configuration Files 1-55 ASM Components: Group Services 1-56 ASM Components: ASM Disk Group 1-57 ASM Disk Group: Failure Groups 1-58 ASM Components: ASM Disks 1-59 ASM Components: ASM Files 1-60 ASM Files: Extents and Striping 1-61 ASM Files: Mirroring 1-62 ASM Components: ASM Clients 1-63 ASM Components: ASM Utilities 1-64 ASM Scalability 1-65 Summary 1-66

2 Grid Infrastructure Installation Objectives 2-2 Module 1: Preinstallation Planning 2-3 Shared Storage Planning for Grid Infrastructure 2-4 Sizing Shared Storage 2-5 Storing the OCR in ASM 2-6 Oracle Clusterware Repository (OCR) 2-7 Managing Voting Disks in ASM 2-9 CSS Voting Disk Function 2-10 Oracle Clusterware Main Log Files 2-11 Installing ASMLib 2-12 Preparing ASMLib 2-13 Quiz 2-14 Module 2: Grid Infrastructure Preinstallation Tasks 2-15 Oracle Grid Infrastructure 11g Installation 2-16 Checking System Requirements 2-17 Checking Network Requirements 2-18 Software Requirements (Kernel) 2-21 Software Requirements: Packages 2-22 Oracle Validated Configuration RPM 2-23 Creating Groups and Users 2-25 Creating Groups and Users: Example 2-26 Shell Settings for the Grid Infrastructure User 2-27 Quiz 2-29 Module 3: Grid Infrastructure Installation 2-30

iv

Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Installing Grid Infrastructure 2-31 Grid Plug and Play Support 2-32 Cluster Node Information 2-33 Specify Network Interface Usage 2-34 Storage Option Information 2-35 Failure Isolation Support with IPMI 2-39 Privileged Operating System Groups 2-40 Installation and Inventory Locations 2-41 Prerequisite Checks 2-42 Finishing the Installation 2-43 Verifying the Grid Infrastructure Installation 2-44 Module 4: Configuring ASM Disk Groups and ACFS 2-45 Configuring ASM with ASMCA 2-46 Creating an ASM Disk Group with ASMCA 2-47 Creating an ASM Disk Group: Advanced Options 2-48 Creating a Disk Group with Enterprise Manager 2-49 Creating an ASM Disk Group with Command-Line Tools 2-51 Configuring an ASM Volume: ASMCA 2-52 Configuring an ASM Volume: EM 2-53 Configuring an ASM Volume: ASMCMD and SQL 2-54 Configuring the ASM Cluster File System 2-55 Configuring ACFS with EM 2-56 Configuring ACFS with Command Line 2-57 Mounting ACFS with ASMCA 2-58 Mounting an ACFS with EM 2-59 Mounting ACFS with Command-Line Tools 2-60 Quiz 2-61 Summary 2-62 Practice 2 Overview 2-63

3 Administering Oracle Clusterware Objectives 3-2 Managing Oracle Clusterware 3-3 Managing Clusterware with Enterprise Manager 3-4 Controlling Oracle Clusterware 3-5 Verifying the Status of Oracle Clusterware 3-6 Determining the Location of Oracle Clusterware Configuration Files 3-7 Checking the Integrity of Oracle Clusterware Configuration Files 3-8 Backing Up and Recovering the Voting Disk 3-9 Adding, Deleting, or Migrating Voting Disks 3-10 Locating the OCR Automatic Backups 3-11 Changing the Automatic OCR Backup Location 3-12 Adding, Replacing, and Repairing OCR Locations 3-13 Removing an Oracle Cluster Registry Location 3-14

v

Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Migrating OCR Locations to ASM 3-15 Migrating OCR from ASM to Other Shared Storage 3-16 Performing Manual OCR Backups 3-17 Recovering the OCR by Using Physical Backups 3-18 Recovering the OCR by Using Logical Backups 3-19 Oracle Local Registry 3-20 Determining the Current Network Settings 3-22 Changing the Public VIP Addresses 3-23 Changing the Interconnect Adapter 3-25 Managing SCAN VIP and SCAN Listener Resources 3-27 Quiz 3-30 Summary 3-32 Practice 3 Overview 3-33

4 Managing Oracle Clusterware Module 4-1 Adding and Deleting Oracle Clusterware Homes 4-2 Objectives 4-3 Adding Oracle Clusterware Homes 4-4 Prerequisite Steps for Running addNode.sh 4-5 Adding a Node with addNode.sh 4-7 Completing OUI Silent Node Addition 4-8 Removing a Node from the Cluster 4-9 Deleting a Node from the Cluster 4-10 Deleting a Node from a Cluster (GNS in Use) 4-13 Deleting a Node from the Cluster 4-14 Quiz 4-15 Summary 4-16 Module 4-2 Patching Oracle Clusterware 4-17 Objectives 4-18 Out-of-Place Oracle Clusterware Upgrade 4-19 Oracle Clusterware Upgrade 4-20 Types of Patches 4-21 Patch Properties 4-23 Configuring the Software Library 4-24 Setting Up Patching 4-25 Starting the Provisioning Daemon 4-26 Obtaining Oracle Clusterware Patches 4-27 Uploading Patches 4-29 Deployment Procedure Manager 4-31 Reduced Down-Time Patching for Cluster Environments 4-32 Rolling Patches 4-33 Checking Software Versions 4-34 Installing a Rolling Patchset with OUI 4-35 Patchset OUI 4-36

vi

Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Installing a Rolling Patchset with OUI 4-37 OPatch: Overview 4-38 OPatch: General Usage 4-40 Before Patching with OPatch 4-41 Installing a Rolling Patch with OPatch 4-42 Installing a Patch with Minimum Down Time with OPatch 4-44 Quiz 4-45 Summary 4-47

5 Making Applications Highly Available with Oracle Clusterware Objectives 5-2 Oracle Clusterware High Availability (HA) 5-3 Oracle Clusterware HA Components 5-4 Resource Management Options 5-5 Server Pools 5-6 Server Pool Attributes 5-7 GENERIC and FREE Server Pools 5-9 Assignment of Servers to Server Pools 5-11 Server Attributes and States 5-12 Creating Server Pools with srvctl and crsctl 5-14 Managing Server Pools with srvctl and crsctl 5-15 Adding Server Pools with Enterprise Manager 5-16 Managing Server Pools with Enterprise Manager 5-17 Clusterware Resource Modeling 5-18 Resource Types 5-20 Adding a Resource Type 5-21 Resource Type Parameters 5-23 Resource Type Advanced Settings 5-24 Defining Resource Dependencies 5-25 Creating an Application VIP by Using crsctl 5-27 Creating an Application VIP by Using EM 5-29 Managing Clusterware Resources with EM 5-30 Adding Resources with EM 5-31 Adding Resources by Using crsctl 5-36 Managing Resources with EM 5-37 Managing Resources with crsctl 5-40 HA Events: ONS and FAN 5-42 Managing Oracle Notification Server with srvctl 5-43 Quiz 5-44 Summary 5-46 Practice 5 Overview 5-47

6 Troubleshooting Oracle Clusterware Objectives 6-2

vii

Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Golden Rule in Debugging Oracle Clusterware 6-3 Oracle Clusterware Main Log Files 6-5 Diagnostics Collection Script 6-6 Cluster Verify: Overview 6-7 Cluster Verify Components 6-8 Cluster Verify Locations 6-9 Cluster Verify Configuration File 6-10 Cluster Verify Output: Example 6-12 Enabling Resource Debugging 6-13 Dynamic Debugging 6-15 Enabling Tracing for Java-Based Tools 6-17 Preserving Log Files Before Wrapping 6-18 Process Roles for Node Reboots 6-19 Determining Which Process Caused Reboot 6-21 Using ocrdump to View Logical Contents of the OCR 6-22 Checking the Integrity of the OCR 6-23 OCR-Related Tools for Debugging 6-24 Browsing My Oracle Support Knowledge Articles 6-26 Quiz 6-27 Summary 6-29 Practice 6 Overview 6-30

7 Administering ASM Instances Objectives 7-2 ASM Initialization Parameters 7-3 ASM_DISKGROUPS 7-4 Disk Groups Mounted at Startup 7-5 ASM_DISKSTRING 7-6 ASM_POWER_LIMIT 7-8 INSTANCE_TYPE 7-9 CLUSTER_DATABASE 7-10 MEMORY_TARGET 7-11 Adjusting ASM Instance Parameters in SPFILEs 7-12 Starting and Stopping ASM Instances by Using srvctl 7-13 Starting and Stopping ASM Instances by Using SQL*Plus 7-14 Starting and Stopping ASM Instances by Using ASMCA and ASMCMD 7-16 Starting and Stopping ASM Instances Containing Cluster Files 7-17 Starting and Stopping the Node Listener 7-18 ASM Dynamic Performance Views 7-19 ASM Dynamic Performance Views Diagram 7-20 Quiz 7-22 Summary 7-24 Practice 7 Overview: Administering ASM Instances 7-25

viii

Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

8 Administering ASM Disk Groups Objectives 8-2 Disk Group Overview 8-3 Creating a New Disk Group 8-4 Creating a New Disk Group with ASMCMD 8-6 Disk Group Attributes 8-7 V$ASM_ATTRIBUTE 8-9 Compatibility Attributes 8-10 Features Enabled by Disk Group Compatibility Attributes 8-11 Support for 4KB Sector Disk Drives 8-12 Supporting 4 KB Sector Disks 8-13 ASM Support for 4KB Sector Disks 8-14 Using the SECTOR_SIZE Clause 8-15 Viewing ASM Disk Groups 8-17 Viewing ASM Disk Information 8-19 Extending an Existing Disk Group 8-21 Dropping Disks from an Existing Disk Group 8-22 REBALANCE POWER 0 8-23 V$ASM_OPERATION 8-24 Adding and Dropping in the Same Command 8-25 Adding and Dropping Failure Groups 8-26 Undropping Disks in Disk Groups 8-27 Mounting and Dismounting Disk Groups 8-28 Viewing Connected Clients 8-29 Dropping Disk Groups 8-30 Checking the Consistency of Disk Group Metadata 8-31 ASM Fast Mirror Resync 8-32 Preferred Read Failure Groups 8-33 Preferred Read Failure Groups Best Practice 8-34 Viewing ASM Disk Statistics 8-35 Performance, Scalability, and Manageability Considerations for Disk Groups 8-37 Quiz 8-38 Summary 8-40 Practice 8 Overview: Administering ASM Disk Groups 8-41

9 Administering ASM Files, Directories, and Templates Objectives 9-2 ASM Clients 9-3 Interaction Between Database Instances and ASM 9-5 Accessing ASM Files by Using RMAN 9-6 Accessing ASM Files by Using XML DB 9-8 Accessing ASM Files by Using DBMS_FILE_TRANSFER 9-9 Accessing ASM Files by Using ASMCMD 9-10 Fully Qualified ASM File Names 9-11

ix

Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Other ASM File Names 9-13 Valid Contexts for the ASM File Name Forms 9-15 Single File Creation Examples 9-16 Multiple File Creation Example 9-17 View ASM Aliases, Files, and Directories 9-18 Viewing ASM Files 9-20 ASM Directories 9-21 Managing ASM Directories 9-22 Managing Alias File Names 9-23 ASM Intelligent Data Placement 9-24 Guidelines for Intelligent Data Placement 9-25 Assigning Files to Disk Regions 9-26 Assigning Files to Disk Regions with Enterprise Manager 9-27 Monitoring Intelligent Data Placement 9-28 Disk Group Templates 9-29 Viewing Templates 9-31 Managing Disk Group Templates 9-32 Managing Disk Group Templates with ASMCMD 9-33 Using Disk Group Templates 9-34 ASM Access Control Lists 9-35 ASM ACL Prerequisites 9-36 Managing ASM ACL with SQL Commands 9-37 Managing ASM ACL with ASMCMD Commands 9-38 Managing ASM ACL with Enterprise Manager 9-39 ASM ACL Guidelines 9-41 Quiz 9-42 Summary 9-44

10 Administering ASM Cluster File Systems Objectives 10-2 ASM Files and Volumes 10-3 ACFS and ADVM Architecture Overview 10-4 ADVM Processes 10-6 ADVM Restrictions 10-7 ASM Cluster File System 10-8 ADVM Space Allocation 10-9 Striping Inside the Volume 10-10 Volume Striping: Example 10-11 Creating an ACFS Volume 10-13 Create an ASM Dynamic Volume with Enterprise Manager 10-14 Managing ADVM Dynamic Volumes 10-17 Create an ASM Cluster File System with Enterprise Manager 10-18 Manage Dynamic Volumes with SQL*PLUS 10-19 Registering an ACFS Volume 10-20

x

Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Creating an ACFS Volume with ASMCA 10-21 Creating the ACFS File System with ASMCA 10-22 Mounting the ACFS File System with ASMCA 10-23 Managing ACFS with EM 10-24 Extending ASMCMD for Dynamic Volumes 10-25 Linux-UNIX File System APIs 10-26 Linux-UNIX Extensions 10-27 ACFS Platform-Independent Commands 10-28 ACFS Snapshots 10-29 Managing ACFS Snapshots 10-30 Managing ACFS Snapshots with Enterprise Manager 10-32 Creating ACFS Snapshots 10-33 Managing Snapshots 10-34 Viewing Snapshots 10-35 ACFS Backups 10-36 ACFS Performance 10-37 Using ACFS Volumes After Reboot 10-38 ACFS Views 10-39 Quiz 10-40 Summary 10-41 Practice 10 Overview: Managing ACFS 10-42

11 Real Application Clusters Database Installation Objectives 11-2 Installing the Oracle Database Software 11-3 Creating the Cluster Database 11-7 Select Database Type 11-8 Database Identification 11-9 Cluster Database Management Options 11-10 Passwords for Database Schema Owners 11-11 Database File Locations 11-12 Recovery Configuration 11-13 Database Content 11-14 Initialization Parameters 11-15 Database Storage Options 11-16 Create the Database 11-17 Monitor Progress 11-18 Postinstallation Tasks 11-19 Check Managed Targets 11-20 Background Processes Specific to Oracle RAC 11-21 Single Instance to RAC Conversion 11-23 Issues for Converting Single Instance Databases to Oracle RAC 11-24 Single-Instance Conversion Using the DBCA 11-25 Conversion Steps 11-26

xi

Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Single-Instance Conversion Using rconfig 11-29 Quiz 11-31 Summary 11-33 Practice 11 Overview 11-34

12 Oracle RAC Administration Objectives 12-2 Cluster Database Home -3 Cluster Database Instance Home Page 12-5 Cluster Home Page 12-6 Configuration Section 12-7 Topology Viewer 12-9 Enterprise Manager Alerts and RAC 12-10 Enterprise Manager Metrics and RAC 12-11 Enterprise Manager Alert History and RAC 12-13 Enterprise Manager Blackouts and RAC 12-14 Redo Log Files and RAC 12-15 Automatic Undo Management and RAC 12-17 Starting and Stopping RAC Instances 12-18 Starting and Stopping RAC Instances with SQL*Plus 12-19 Starting and Stopping RAC Instances with srvctl 12-20 Switch Between the Automatic and Manual Policies 12-21 RAC Initialization Parameter Files 12-22 SPFILE Parameter Values and RAC 12-23 EM and SPFILE Parameter Values 12-24 RAC Initialization Parameters 12-26 Parameters That Require Identical Settings 12-28 Parameters That Require Unique Settings 12-29 Quiescing RAC Databases 12-30 Terminating Sessions on a Specific Instance 12-31 How SQL*Plus Commands Affect Instances 12-32 Transparent Data Encryption and Wallets in RAC 12-33 Quiz 12-34 Summary 12-36 Practice 12 Overview 12-37

13 Managing Backup and Recovery for RAC Objectives 13-2 Protecting Against Media Failure 13-3 Media Recovery in Oracle RAC 13-4 Parallel Recovery in RAC 13-5 Archived Log File Configurations 13-6 RAC and the Fast Recovery Area 13-7 RAC Backup and Recovery Using EM 13-8

xii

Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Configure RAC Recovery Settings with EM 13-9 Archived Redo File Conventions in RAC 13-10 Configure RAC Backup Settings with EM 13-11 Oracle Recovery Manager 13-12 Configure RMAN Snapshot Control File Location 13-13 Configure Control File and SPFILE Autobackup 13-14 Crosschecking on Multiple RAC Clusters Nodes 13-15 Channel Connections to Cluster Instances 13-16 RMAN Channel Support for the Grid 13-17 RMAN Default Autolocation 13-18 Distribution of Backups 13-19 One Local Drive Shared Storage Backup Scheme 13-20 Multiple Drives Shared Storage Backup Scheme 13-21 Restoring and Recovering 13-22 Quiz 13-23 Summary 13-25 Practice 13 Overview 13-26

14 RAC Database Monitoring and Tuning Objectives 14-2 CPU and Wait Time Tuning Dimensions 14-3 RAC-Specific Tuning 14-4 RAC and Instance or Crash Recovery 14-5 Instance Recovery and Database Availability 14-7 Instance Recovery and RAC 14-8 Analyzing Cache Fusion Impact in RAC 14-10 Typical Latencies for RAC Operations 14-11 Wait Events for RAC 14-12 Wait Event Views 14-13 Global Cache Wait Events: Overview 14-14 2-way Block Request: Example 14-16 3-way Block Request: Example 14-17 2-way Grant: Example 14-18 Global Enqueue Waits: Overview 14-19 Session and System Statistics 14-20 Most Common RAC Tuning Tips 14-21 Index Block Contention: Considerations 14-23 Oracle Sequences and Index Contention 14-24 Undo Block Considerations 14-25 High-Water Mark Considerations 14-26 Concurrent Cross-Instance Calls: Considerations 14-27 Monitoring RAC Database and Cluster Performance 14-28 Cluster Database Performance Page 14-29 Determining Cluster Host Load Average 14-30

xiii

Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Determining Global Cache Block Access Latency 14-31 Determining Average Active Sessions 14-32 Determining Database Throughput 14-33 Accessing the Cluster Cache Coherency -35 Viewing the Cluster Interconnects Page 14-37 Viewing the Database Locks Page 14-39 AWR Snapshots in RAC 14-40 AWR Reports and RAC: Overview 14-41 Active Session History Reports for RAC 14-43 Automatic Database Diagnostic Monitor for RAC 14-45 What Does ADDM Diagnose for RAC? 14-47 EM Support for ADDM for RAC 14-48 Quiz 14-49 Summary 14-51 Practice 14 Overview 14-52

15 Services Objectives 15-2 Oracle Services 15-3 Services for Policy- and Administrator-Managed Databases 15-4 Default Service Connections 15-5 Create Service with Enterprise Manager 15-6 Create Services with SRVCTL 15-7 Manage Services with Enterprise Manager 15-8 Managing Services with EM 15-9 Manage Services with srvctl 15-10 Use Services with Client Applications 15-11 Services and Connection Load Balancing 15-12 Services and Transparent Application Failover 15-13 Use Services with the Resource Manager 15-14 Services and Resource Manager with EM 15-15 Use Services with the Scheduler 15-16 Services and the Scheduler with EM 15-17 Using Distributed Transactions with RAC 15-19 Distributed Transactions and Services 15-20 Service Thresholds and Alerts 15-22 Services and Thresholds Alerts: Example 15-23 Service Aggregation and Tracing 15-24 Top Services Performance Page 15-25 Service Aggregation Configuration 15-26 Service, Module, and Action Monitoring 15-27 Service Performance Views 15-28 Quiz 15-29 Summary 15-31

xiv

Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Practice 15: Overview 15-32

16 Design for High Availability Objectives 16-2 Causes of Unplanned Down Time 16-3 Causes of Planned Down Time 16-4 Oracle’s Solution to Down Time 16-5 RAC and Data Guard Complementarity 16-6 Maximum Availability Architecture 16-7 RAC and Data Guard Topologies 16-8 RAC and Data Guard Architecture 16-9 Data Guard Broker (DGB) and Oracle Clusterware (OC) Integration 16-11 Fast-Start Failover: Overview 16-12 Data Guard Broker Configuration Files 16-14 Real-Time Query Physical Standby Database 16-15 Hardware Assisted Resilient Data 16-16 Database High Availability: Best Practices 16-17 How Many ASM Disk Groups Per Database? 16-18 Which RAID Configuration for High Availability? 16-19 Should You Use ASM Mirroring Protection? 16-20 What Type of Striping Works Best? 16-21 ASM Striping Only 16-22 Hardware RAID–Striped LUNs 16-23 Hardware RAID–Striped LUNs HA 16-24 Disk I/O Design Summary 16-25 Extended RAC: Overview 16-26 Extended RAC Connectivity 16-27 Extended RAC Disk Mirroring 16-28 Achieving Quorum with Extended RAC 16-29 Additional Data Guard Benefits 16-30 Using a Test Environment 16-31 Quiz 16-32 Summary 16-33

Appendix A: Practices and Solutions

Appendix B: DHCP and DNS Configuration For GNS Objectives B-2 GNS Overview B-3 DHCP Service B-4 DHCP Configuration Example B-5 DNS Concepts B-7 DNS Forwarding For GNS B-9 DNS Configuration: Example B-11 DNS Configuration: Detail B-13

xv

Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Summary B-17

Appendix C: High Availability of Connections Objectives C-2 Types of Workload Distribution C-3 Client-Side Load Balancing C-4 Client-Side, Connect-Time Failover C-5 Server-Side Load Balancing C-7 Runtime Connection Load Balancing and Connection Pools C-9 Fast Application Notification: Overview C-11 Fast Application Notification: Benefits C-12 FAN Events C-13 FAN Event Status C-14 FAN Event Reasons C-15 FAN Event Format C-16 Load Balancing Advisory: FAN Event C-17 Implementation of Server-Side Callouts C-18 Server-Side Callout Parse: Example C-19 Server-Side Callout Filter: Example C-20 Configuring the Server-Side ONS C-21 Optionally Configure the Client-Side ONS C-22 JDBC Fast Connection Failover: Overview C-23 Using Oracle Streams Advanced Queuing for FAN C-24 JDBC/ODP.NET FCF Benefits C-25 Load Balancing Advisory C-26 Runtime Connection Load Balancing and Connection Pools C-27 Monitor LBA FAN Events C-28 Transparent Application Failover: Overview C-29 TAF Basic Configuration Without FAN: Example C-30 TAF Basic Configuration with FAN: Example C-31 TAF Preconnect Configuration: Example C-32 TAF Verification C-33 FAN Connection Pools and TAF Considerations C-34 Summary C-35

Appendix D: Oracle RAC One Node The Omotion Utility D-3 Oracle RAC One Node and OVM D-4 Creating and Managing RAC One Node D-5

Appendix E: Cloning Oracle Clusterware Objectives E-2 What Is Cloning? E-3 Benefits of Cloning Oracle Clusterware E-4

xvi

Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Creating a Cluster by Cloning Oracle Clusterware E-5 Preparing the Oracle Clusterware Home for Cloning E-6 Cloning to Create a New Oracle Clusterware Environment E-9 The clone.pl Script E-11 The clone.pl Environment Variables E-12 The clone.pl Command Options E-13 Cloning to Create a New Oracle Clusterware Environment E-14 Log Files Generated During Cloning E-18 Cloning to Extend Oracle Clusterware to More Nodes E-20 Quiz E-25 Summary E-27

Appendix F: Clusterware Concepts Objectives F-2 Oracle Grid Infrastructure F-3 What Is a Cluster? F-4 What Is Clusterware? F-5 Oracle Clusterware F-6 Oracle Clusterware Architecture and Services F-7 Oracle Clusterware Networking F-8 Oracle Clusterware Startup F-10 Oracle Clusterware Process Architecture F-11 Grid Plug and Play F-13 GPnP Domain F-14 GPnP Components F-15 GPnP Profile F-16 Grid Naming Service F-17 Single Client Access Name F-18 GPnP Architecture Overview F-20 How GPnP Works: Cluster Node Startup F-22 How GPnP Works: Client Database Connections F-23 Summary F-24

Appendix G: RAC Concepts Objectives G-2 Benefits of Using RAC G-3 Clusters and Scalability G-4 Levels of Scalability G-5 Scaleup and Speedup G-6 Speedup/Scaleup and Workloads G-7 I/O Throughput Balanced: Example G-8 Performance of Typical Components G-9 Necessity of Global Resources G-10 Global Resources Coordination G-11

xvii

Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Global Cache Coordination: Example G-12 Write to Disk Coordination: Example G-13 Dynamic Reconfiguration G-14 Object Affinity and Dynamic Remastering G-15 Global Dynamic Performance Views G-16 Efficient Internode Row-Level Locking G-17 Parallel Execution with RAC G-18 Summary G-19

xviii

Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Oracle Grid Infrastructure Architecture

Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Objectives

After completing this lesson, you should be able to:• Explain the principles and purposes of clusters• Describe Cluster hardware best practices• Describe the Oracle Clusterware architecture• Describe how Grid Plug and Play affects Clusterware• Describe the Automatic Storage Management (ASM)

architecture• Describe the components of ASM

Oracle 11g: RAC and Grid Infrastructure Administration Accelerated 1 - 2

Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Oracle Grid Infrastructure• ASM and Oracle Clusterware are installed into a single

home directory called Oracle Grid Infrastructure 11gRelease 2.

• This directory is referred to as the Grid Infrastructure home.

Oracle Grid InfrastructureWith Oracle Database 11g Release 2, Automatic Storage Management (ASM) and Oracle Clusterware are installed into a single home directory, collectively called Oracle Grid Infrastructure. This directory is referred to as the Grid Infrastructure home. Configuration assistants start after the Oracle Universal Installer interview process and binary installation that configure ASM and Oracle Clusterware. Although the installation is called Oracle Grid Infrastructure, Oracle Clusterware and Automatic Storage Manager remain separate components.Oracle Grid Infrastructure concepts are presented here in three modules:

• Module 1: Oracle Clusterware Concepts• Module 2: Oracle Clusterware Architecture• Module 3: ASM Concepts and Architecture


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Module 1: Oracle Clusterware Concepts


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


What Is a Cluster?

• A group of independent, but interconnected, computers that act as a single system

• Usually deployed to increase availability and performance or to balance a dynamicallychanging workload

Storage Network

Interconnect

Users

Network

What Is a Cluster?A cluster consists of a group of independent but interconnected computers whose combined resources can be applied to a processing task. A common cluster feature is that it should appear to an application as though it were a single server. Most cluster architectures use a dedicated network (cluster interconnect) for communication and coordination between cluster nodes.A common cluster architecture for data-intensive transactions and computations is built around shared disk storage. Shared-nothing clusters use an alternative architecture where storage is not shared and data must be either replicated or segmented across the cluster. Shared-nothing clusters are commonly used for workloads that can be easily and predictably divided into small units that can be spread across the cluster in parallel. Shared disk clusters can perform these tasks but also offer increased flexibility for varying workloads. Load balancing clusters allow a single application to balance its workload across the cluster. Alternatively, in a failover cluster, some nodes can be designated as the primary host for an application, whereas others act as the primary host for different applications. In a failover cluster, the failure of a node requires that the applications it supports be moved to a surviving node. Load balancing clusters can provide failover capabilities but they can also run a single application across multiple nodes providing greater flexibility for different workload requirements. Oracle supports a shared disk cluster architecture providing load balancing and failover capabilities. In an Oracle cluster, all nodes must share the same processor architecture and run the same operating system.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


What Is Clusterware?

Software that provides various interfaces and services for a cluster. Typically, this includes capabilities that:• Allow the cluster to be managed as a whole• Protect the integrity of the cluster• Maintain a registry of resources across the cluster• Deal with changes to the cluster• Provide a common view of resources

What Is Clusterware?Clusterware is a term used to describe software that provides interfaces and services that enable and support a cluster. Different cluster architectures require clusterware that delivers different services. For example, in a simple failover cluster the clusterware may monitor the availability of applications and perform a failover operation if a cluster node becomes unavailable. In a load balancing cluster, different services are required to support workload concurrency and coordination.Typically, clusterware includes capabilities that:

• Allow the cluster to be managed as a single entity (not including OS requirements), if desired

• Protect the integrity of the cluster so that data is protected and the cluster continues to function even if communication with a cluster node is severed

• Maintain a registry of resources so that their location is known across the cluster and so that dependencies between resources is maintained

• Deal with changes to the cluster such as node additions, removals, or failures• Provide a common view of resources such as network addresses and files in a file system


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Oracle Clusterware

Oracle Clusterware is:• A key part of Oracle Grid Infrastructure• Integrated with Oracle

Automatic Storage Management (ASM)

• The basis for ASM Cluster File System (ACFS)

• A foundation for Oracle Real Application Clusters (RAC)

• A generalized cluster infrastructure for all kinds of applications

Oracle ClusterwareOracle Clusterware is a key part of Oracle Grid Infrastructure, which also includes Automatic Storage Management (ASM) and the ASM Cluster File System (ACFS).In Release 11.2, Oracle Clusterware can use ASM for all the shared files required by the cluster. Oracle Clusterware is also an enabler for the ASM Cluster File System, a generalized cluster file system that can be used for most file-based data such as documents, spreadsheets, and reports. The combination of Oracle Clusterware, ASM, and ACFS provides administrators with a unified cluster solution that is not only the foundation for the Oracle Real Application Clusters (RAC) database, but can also be applied to all kinds of other applications.Note: Grid Infrastructure is the collective term that encompasses Oracle Clusterware, ASM, and ACFS. These components are so tightly integrated that they are often collectively referred to as Oracle Grid Infrastructure. In this lesson, ASM and ACFS will be referred to in specific instances; however, they are covered in greater detail in later lessons.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Oracle Clusterware Architecture and Services

• Shared disk cluster architecture supporting application load balancing and failover

• Services include:– Cluster management– Node monitoring– Event services– Time synchronization– Network management– High availability

Oracle Clusterware Architecture and ServicesOracle Clusterware provides a complete set of cluster services to support the shared disk, load balancing cluster architecture of the Oracle Real Application Cluster (RAC) database. Oracle Clusterware can also be used to provide failover clustering services for single instance Oracle databases and other applications.The services provided by Oracle Clusterware include:

• Cluster Management, which allows cluster services and application resources to be monitored and managed from any node in the cluster

• Node Monitoring, which provides real-time information regarding which nodes are currently available and the resources they support. Cluster integrity is also protected by evicting or fencing unresponsive nodes.

• Event Services, which publishes cluster events so that applications are aware of changes in the cluster

• Time Synchronization, which synchronizes the time on all nodes of the cluster• Network Management, which provisions and manages Virtual IP (VIP) addresses that are

associated with cluster nodes or application resources to provide a consistent network identity regardless of which nodes are available. In addition, Grid Naming Service (GNS) manages network naming within the cluster.

• High Availability, which services, monitors, and restarts all other resources as required


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Goals for Oracle Clusterware

• Easier installation• Easier management• Continuing tight integration with Oracle RAC• ASM enhancements with

benefits for all applications• No additional clusterware

required

Goals for Oracle Clusterware Oracle Clusterware has become the required clusterware for Oracle Real Application Clusters (RAC). Oracle Database 11g Release 2 builds on the tight integration between Oracle Clusterware and RAC by extending the integration with Automatic Storage Management (ASM). The result is that now all the shared data in your cluster can be managed using ASM. This includes the shared data required to run Oracle Clusterware, Oracle RAC, and any other applications you choose to deploy in your cluster.In most cases, this capability removes the need to deploy additional clusterware from other sources, which also removes the potential for integration issues caused by running multiple clusterware software stacks. It also improves the overall manageability of the cluster.Although most of the enhancements to ASM are the subject of later lessons, the next part of this lesson examines a series of additional Oracle Clusterware capabilities and the benefits they provide.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Oracle Clusterware Networking

• Each node must have at least two network adapters.• Each public network adapter must support TCP/IP.• The interconnect adapter must support:

– User Datagram Protocol (UDP) or Reliable Data Socket (RDS) for UNIX and Linux for database communication

– TCP for Windows platforms for database communication• All platforms use Grid Interprocess Communication (GIPc)

Interconnect: Private network

Public network

NIC2NIC1 NIC1

NIC1NIC2

NIC2

Oracle Clusterware NetworkingEach node must have at least two network adapters: one for the public network interface and the other for the private network interface or interconnect. In addition, the interface names associated with the network adapters for each network must be the same on all nodes. For example, in a two-node cluster, you cannot configure network adapters on node1 with eth0 as the public interface, but on node2 have eth1 as the public interface. Public interface names must be the same, so you must configure eth0 as public on both nodes. You should configure the private interfaces on the same network adapters as well. If eth1 is the private interface for node1, then eth1 should be the private interface for node2.Before starting the installation, on each node, you must have at least two interfaces to configure for the public and private IP addresses. You can configure IP addresses with one of the following options:

• Oracle Grid Naming Service (GNS) using one static address defined during installation, which dynamically allocates VIP addresses using Dynamic Host Configuration Protocol (DHCP), which must be running on the network. You must select the Advanced Oracle Clusterware installation option to use GNS.

• Static addresses that network administrators assign on a network domain name server (DNS) or each node. To use the Typical Oracle Clusterware installation option, you must use static addresses.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Oracle Clusterware Networking (continued)For the public network, each network adapter must support TCP/IP. For the private network, the interconnect must support UDP or RDS (TCP for Windows) for communications to the database. Grid Interprocess Communication (GIPc) is used for Grid (Clusterware) interprocess communication. GIPC is a new common communications infrastructure to replace CLSC/NS. It provides a full control of the communications stack from the operating system up to whatever client library uses it. The dependency on network services (NS) prior to 11.2 is removed, but there is still backwards compatibility with existing CLSC clients (primarily from 11.1). GIPC can support multiple communications types: CLSC, TCP, UDP, IPC and of course the communication type GIPC.Use high-speed network adapters for the interconnects and switches that support TCP/IP. Gigabit Ethernet or an equivalent is recommended. Each node in a cluster requires a supported interconnect protocol to support Cache Fusion and TCP/IP to support Clusterware polling. Token Ring is not supported for cluster interconnects on IBM AIX. Your interconnect protocol must be certified by Oracle for your platform. Note: Cross-over cables are not supported for use with Oracle Clusterware interconnects.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Interconnect Link Aggregation: Single Switch

• Link aggregation can be used to increase redundancy for higher availability with an Active/Standby configuration.

• Link aggregation can be used to increase bandwidth for performance with an Active/Active configuration.

eth1bond0

Active Standby Active Active

Interconnect Interconnect

Eachnode

Eachnode

eth2

eth1bond0

eth2

Interconnect Link Aggregation: Single SwitchInterconnect link aggregation involves bonding two or more physical network interface cards into a single logical “bonded” interface. The behavior of the bonded interfaces depends on thesettings, modes, and drivers used to accomplish the aggregation.One strategy often used for highly available configurations is the Active/Standby aggregation, sometimes known as Active/Backup or Active/Passive aggregation. Generally, only one of the network interface cards carry traffic, and the other is available for failover. An example of an Active/Backup setup on Linux as reported with the ifconfig command is as follows:

bond0 Link encap:Ethernet HWaddr 00:C0:F0:1F:37:B4

inet addr:XXX.XXX.XXX.YYY Bcast:XXX.XXX.XXX.255 Mask:255.255.252.0

UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1

RX packets:7224794 errors:0 dropped:0 overruns:0 frame:0

TX packets:3286647 errors:1 dropped:0 overruns:1 carrier:0

collisions:0 txqueuelen:0


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Interconnect Link Aggregation: Single Switch (continued)eth1 Link encap:Ethernet HWaddr 00:C0:F0:1F:37:B4


UP BROADCAST RUNNING NOARP SLAVE MULTICAST MTU:1500 Metric:1




Interrupt:10 Base address:0x1080

eth2 Link encap:Ethernet HWaddr 00:C0:F0:1F:37:B4


UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1




Interrupt:9 Base address:0x1400

The eth1 and eth2 devices are physical network interface cards. The bond0 device is a “virtual” network interface card. Notice that the logical bond0 interface is listed as the MASTER, and the other two interfaces are listed as SLAVE. The interface device without the NOARP (eth2) is the current active SLAVE. Also notice that all three interfaces report the same layer-2 or Media Access Control (MAC) address and have IP addresses. Traffic statistics exist on all Network Interface Card (NIC) devices in this sample output because of extended up time and failures in the past.Note: During the installation of the Oracle Clusterware product, the bond0 interface will be supplied as the value to the prompts for the interconnect interface to be used.Another common strategy for link aggregation involves Active/Active configurations following the IEEE 802.3ad standards. This arrangement involves simultaneous use of both the bonded physical network interface cards in parallel to achieve a higher bandwidth beyond the limit of any one single network card. It is very important that if 802.3ad is used at the NIC layer, the switch must also support and be configured for 802.3ad. Misconfiguration results in poor performance and interface resets or “port flapping.” An alternative is to consider a single network interface card with a higher bandwidth, such as 10Gbit Ethernet instead of 1Gbit Ethernet. Infiniband can also be used for the interconnect.Link aggregation is sometimes known as “NIC teaming,” “NIC bonding,” “port trunking,”“EtherChannel,” “Multi-Link Trunking (MLT),” “Network Fault Tolerance (NFT),” and “link aggregate group (LAG).” Link aggregation is usually limited to a single switch. Multiswitch solutions include “Split Multi-Link Trunking (SMLT),” “Distributed Split Multi-Link Trunking (DSMLT),” and “Routed Split Multi-Link Trunking (RSMLT). ”


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Interconnect Link Aggregation: Multiswitch

• Redundant switches connected with an Inter-Switch Trunk may be used for an enhanced highly available design.

• This is the best practice configuration for the interconnect.

Active Standby

Eachnode

Inter Switch Trunk

Interconnect

eth1

eth2bond0

eth1

eth2bond0

Interconnect Link Aggregation: MultiswitchWith the single switch solutions presented in the previous slide, a failure at the switch level would bring down the entire interconnect. A better highly available (HA) design would be to implement a redundant switch strategy as illustrated in the slide, with an Inter-Switch Trunk connecting the switches. This is the best practice design for the Oracle Clusterware interconnect. Only Active/Standby mode is supported in this configuration.Some of the standard aggregation solutions include:

• Cisco EtherChannel based on 802.3ad• AIX EtherChannel• HPUX Auto Port Aggregation• Sun Trunking, IPMP, GLD• Linux Bonding (only certain modes)• Windows NIC teaming

A virtual LAN (VLAN) is supported in a shared switch environment for the interconnect. The interconnect should be a dedicated nonroutable subnet mapped to a single dedicated, nonshared VLAN.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Interconnect NIC Guidelines

Optimal interconnect NIC settings can vary depending on the driver used. Consider the following guidelines:• Configure the interconnect NIC on the fastest PCI bus.• Ensure that NIC names and slots are identical on all

nodes.• Define flow control: receive=on, transmit=off.• Define full bit rate supported by NIC.• Define full duplex autonegotiate. • Ensure compatible switch settings:

– If 802.3ad is used on NIC, it must be used and supported on the switch.

– The Maximum Transmission Unit (MTU) should be the same between NIC and the switch.

• Driver settings can change between software releases.

Interconnect NIC GuidelinesFailure to correctly configure the network interface cards and switches used for the interconnect results in severe performance degradation and possible node evictions or node fencing. If there is a choice between bus standards such as PCI and PCI express, configure the interconnect NIC on the fastest PCI bus. It is a requirement for the NIC devices and switch to autonegotiate to achieve the highest supported bit rate possible. Flow control should be turned on for receive. Cases have occurred where this setting has been altered between driver software updates and changes. Depending on the mode of link aggregation used, specialized support may be needed at the switch. Synchronization between the switch settings and network interface cards is very important.For Oracle database Real Application Clusters (RAC), the interconnect will be used to transport database block images. An Oracle database block can be sized up to 32k bytes, whereas the typical interconnect communication message averages around 200 bytes.A misconfigured or faulty interconnect can lead to a variety of problems such as:

• Dropped packets and fragments• Buffer overflows• Packet reassembly failures or timeouts• General Tx/Rx errors


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Additional Interconnect Guidelines

UDP socket buffer (rx)• Default settings are adequate for the majority of

customers.• It may be necessary to increase the allocated buffer size

when the:– MTU size has been increased– netstat command reports errors– ifconfig command reports dropped packets or overflow

Jumbo frames:• Are not an Institute of Electrical and Electronics Engineers

(IEEE) standard• Are useful for Network-Attached Storage (NAS)/iSCSI

storage• Have network device interoperability concerns• Need to be configured with care and tested rigorously

Additional Interconnect GuidelinesThe maximum UDP socket receive buffer size varies according to the operating system. The upper limit may be as small as 128k or as large as 1G. In most cases, the default settings are adequate for the majority of customers. This is one of the first settings to consider if you are receiving lost blocks. Consult the My Oracle Support (formerly, MetaLink) Web site for best-practice settings for your platform. Three significant conditions that indicate when it may be necessary to change the UDP socket receive buffer size are when the MTU size has been increased, when excessive fragmentation and/or reassembly of packets is observed, and if dropped packets or overflows are observed.Jumbo frames are not a requirement for Oracle Clusterware and not configured by default. The use of jumbo frames is supported; however, special care must be taken because this is not an IEEE standard and there are significant variances among network devices and switches especially from different manufacturers. The typical frame size for jumbo frames is 9k, but again, this can vary. It is necessary that all devices in the communication path be set to the same value.Note: For Oracle Clusterware, the Maximum Transmission Unit (MTU) needs to be the same on all nodes. If it is not set to the same value, an error message will be sent to the Clusterware alert logs.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Quiz

Each cluster node’s public Ethernet adapter must support UDPor RDS.1. True2. False

Answer: 2This statement is false. Actually, each cluster node’s public Ethernet adapter should support TCP/IP. The private adapter should support UDP or RDS on Linux/UNIX platforms and TCP/IP on Windows platforms.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Module 2: Oracle Clusterware Architecture


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


• Oracle Clusterware is started by the OS init daemon.

• Oracle Clusterware installation modifies the /etc/inittab file to restart ohasd in the event of a crash.

Oracle Clusterware Startup

/etc/init.d/init.ohasd

ohasd.bin oclskd.bin

octssd.bin crsd.bin

oraagent.bin gipcd.bin

diskmon.bin mdnsd.bin

ocssd.bin gpnpd.bin

evmd.bin scriptagent.bin

cssdagent oraagent.bin

orarootagent.bin

init

Clusterwarestartup script

Oracle Clusterware processesOperating systeminit daemon

# cat /etc/inittab..h1:35:respawn:/etc/init.d/init.ohasd run >/dev/null 2>&1 </dev/null

Oracle Clusterware StartupDuring the installation of Oracle Clusterware, the init.ohasd startup script is copied to /etc/init.d. The wrapper script is responsible for setting up environment variables and then starting the Oracle Clusterware daemons and processes.The Oracle High Availability Services daemon (ohasd) is responsible for starting in proper order, monitoring, and restarting other local Oracle daemons, up through the crsd daemon, which manages clusterwide resources. When init starts ohasd on Clusterware startup,ohasd starts orarootagent, cssdagent, and oraagent. These processes then carry out the following tasks:• orarootagent starts crsd.

- crsd starts another orarootagent process responsible for root-owned CRS resources including the SCAN VIPS.

• cssdagent starts cssd (ocssd).• oraagent starts mdnsd, evmd, ASM, ctssd, and gpnpd. oraagent also starts

gsd, Oracle Notification Service (ONS), and the listeners.Some of the high availability daemons will be running under the root user with real-time priority, and others will be running under the Clusterware owner with user-mode priorities after they are started. When a command is used to stop Oracle Clusterware, the daemons will be stopped, but the ohasd process will remain running.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Oracle Clusterware Process ArchitectureClusterware processes are organized into several component groups. They include:

Component Processes OwnerCluster Ready Service (CRS) crsd root

Cluster Synchronization Service (CSS) ocssd,cssdmonitor, grid owner, cssdagent root, root

Event Manager (EVM) evmd, evmlogger grid owner

Cluster Time Synchronization Service (CTSS)

octssd root

Oracle Notification Service (ONS) ons, eons grid ownerOracle Agent oraagent grid owner

Oracle Root Agent orarootagent root

Grid Naming Service (GNS) gnsd root

Grid Plug and Play (GPnP) gpnpd grid owner

Multicast domain name service (mDNS) mdnsd grid owner

Oracle Clusterware Process Architecture Oracle Clusterware comprises several processes that facilitate cluster operations. The Cluster Ready Service (CRS), Cluster Synchronization Service (CSS), Event Management (EVM), and Oracle Notification Service (ONS) components communicate with other cluster component layers in the same cluster database environment. These components are also the main communication links between Oracle Database, applications, and the Oracle Clusterwarehigh availability components. In addition, these background processes monitor and manage database operations. The following list describes some major Oracle Clusterware background processes. The list includes components that are processes on Linux and UNIX, or services on Windows.

• Cluster Ready Service (CRS): Is the primary program for managing high availability operations in a cluster. The CRS process manages two types of CRS resources:

- Cluster resources: A cluster resource is an Oracle Clusterware resource. Cluster resources are viewed, added, modified, or removed using the crsctl command.

- Local resources: A local resource runs on every node in the cluster (no failover) and can be, for example, a listener, ASM, a disk group, or Oracle Notification Service (ONS).

• The CRS daemon (crsd) manages cluster resources based on configuration information that is stored in Oracle Cluster Registry (OCR) for each resource. This includes start, stop, monitor, and failover operations. The crsd process generates events when the status of a resource changes.Oracle 11g: RAC and Grid Infrastructure Administration Accelerated 1 - 20

Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Oracle Clusterware Process Architecture (continued)When you have Oracle RAC installed, the crsd process monitors the Oracle database instance, listener, and so on, and automatically restarts these components when a failure occurs. When a CRS resource fails, the CRS daemon attempts to restart it, if the resource is so configured. CRS fails the resource over to another node (again, if it is configured to do so) after exhausting restart attempts.

• Cluster Synchronization Service (CSS): Manages the cluster configuration by controlling which nodes are members of the cluster and by notifying members when a node joins or leaves the cluster. If you are using certified third-party clusterware, then CSS processes interfaces with your clusterware to manage node membership information. CSS has three separate processes: the CSS daemon (ocssd), the CSS Agent (cssdagent), and the CSS Monitor (cssdmonitor). The cssdagentprocess monitors the cluster and provides input/output fencing. This service formerly was provided by Oracle Process Monitor daemon (oprocd), also known asOraFenceService on Windows. A cssdagent failure results in Oracle Clusterwarerestarting the node.

• Disk Monitor daemon (diskmon): Monitors and performs input/output fencing for Oracle Exadata Storage Server. As Exadata storage can be added to any Oracle RAC node at any point in time, the diskmon daemon is always started when ocssd is started.

• Event Manager (EVM): Is a background process that publishes Oracle Clusterwareevents

• Multicast domain name service (mDNS): Allows DNS requests. The mDNS process is a background process on Linux and UNIX, and a service on Windows.

• Oracle Grid Naming Service (GNS): Is a gateway between the cluster mDNS and external DNS servers. The GNS process performs name resolution within the cluster.

• Oracle Notification Service (ONS): Is a publish-and-subscribe service for communicating Fast Application Notification (FAN) events

• oraagent: Extends clusterware to support Oracle-specific requirements and complex resources. It runs server callout scripts when FAN events occur. This process was known as RACG in Oracle Clusterware 11g Release 1 (11.1).

• Oracle root agent (orarootagent): Is a specialized oraagent process that helps CRSD manage resources owned by root, such as the network, and the Grid virtual IP address

• Cluster kill daemon (oclskd): Handles instance/node evictions requests that have been escalated to CSS

• Grid IPC daemon (gipcd): Is a helper daemon for the communications infrastructure


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Grid Plug and Play• In previous releases, adding or removing servers in a

cluster required extensive manual preparation.• In Oracle Database 11g Release 2, GPnP allows each

node to perform the following tasks dynamically:– Negotiating appropriate network identities for itself– Acquiring additional information from a configuration profile– Configuring or reconfiguring itself using profile data, making

host names and addresses resolvable on the network• To add a node, simply connect the server to the cluster

and allow the cluster to configure the node.

Grid Plug and PlayWith past releases, adding or removing servers in a cluster required extensive manual preparation. With the release of Oracle Database 11g Release 2, Grid Plug and Play (GPnP) reduces the costs of installing, configuring, and managing server nodes by using a Grid Naming Service within the cluster to allow each node to perform the following tasks dynamically:

• Negotiating appropriate network identities for itself• Acquiring additional information it needs to operate from a configuration profile• Configuring or reconfiguring itself using profile data, making host names and addresses

resolvable on the networkAs servers perform these tasks dynamically, adding and removing nodes simply requires an administrator to connect the server to the cluster and to allow the cluster to configure the node. Using Grid Plug and Play, and using best practices recommendations, adding a node to the database cluster is part of the normal server restart, and removing a node happens when a server is turned off. This removes many manual operations, reduces opportunity for error, and encourages configurations that can be changed more easily than those requiring fixed per-node configuration.The best case uses ASM and Automatic Undo Management so there is no particular policy decision to make if an undo tablespace needs to be allocated for a newly identified database instance.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


GPnP Domain

• The GPnP domain is a collection of nodes belonging to a single cluster served by the GPnP service:– Cluster name: cluster01– Network domain: example.com– GPnP domain: cluster01.example.com

• Each node participating in a GPnP domain has the following characteristics:– Must have at least one routable interface with connectivity

outside of the GPnP domain for the public interface– A unique identifier that is unique within the GPnP domain– A personality affected by the GPnP profile, physical

characteristics, and software image of the node

GPnP DomainThe GPnP domain is a collection of nodes served by the GPnP service. In most cases, the size of the domain is limited to the domain served by the multicast domain. The nodes in a multicast domain implicitly know which GPnP domain they are in by the public key of their provisioning authority. A GPnP domain is defined by the nodes in a multicast domain that recognize a particular provisioning authority, as determined by the certificate in their software images. A GPnP node refers to a computer participating in a GPnP domain. It must have the following:IP Connectivity: The node must have at least one routable interface with connectivity outside of the GPnP domain. If the node has varying connectivity (multiple interfaces, subnets, and so on), the required binding (public, private, storage) needs to be identified in the GPnP profile. The physical connectivity is controlled by some outside entity and is not modified by GPnP.Unique identifier: Associated with a node is a unique identifier, established through OSD. The ID is required to be unique within the GPnP domain, but is usefully unique globally within the largest plausible domain. Personality: Characteristics of the node personality include:

• Cluster name• Network classifications (public/private)• Storage to be used for ASM and CSS• Digital signatures• The software image for the node, including the applications that may be 23


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


GPnP Components

Software image• A software image is a read-only collection of software to be

run on nodes of the same type.• At a minimum, the image must contain:

– An operating system– The GPnP software– A security certificate from the provisioning authority– Other software required to configure the node when it starts

up

GPnP ComponentsA software image is a read-only collection of software to be run on nodes of the same type. At a minimum, it must contain an operating system, the GPnP software, the security certificate of the provisioning authority, and other software required to configure the node when it starts up. In the fullest form, the image may contain all application software to be run on the node, including the Oracle Database, iAS, and customer applications. Under GPnP, an image need not be specific to a particular node and may be deployed to as many nodes as needed.An image is created by a provisioning authority through means not defined by GPnP. It may involve doing an installation on an example machine, then “scraping” the storage to remove node-dependent data.An image is distributed to a node by the provisioning authority in ways outside the scope ofGPnP—it may be a system administrator carrying CDs around, a network boot, or any other mechanism.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


GPnP Profile

The profile.xml file:$ cat GRID_HOME/gpnp/profiles/peer/profile.xml

<?xml version="1.0" encoding="UTF-8"?><gpnp:GPnP-Profile Version="1.0" xmlns="http://www.grid-pnp.org/2005/11/gpnp-profile" ... xsi:schemaLocation="http://www.grid-pnp.org/2005/11/gpnp-profile gpnp-profile.xsd" ProfileSequence="4" ClusterUId="2deb88730e0b5f1bffc9682556bd548e" ClusterName="cluster01" PALocation=""><gpnp:Network-Profile><gpnp:HostNetwork id="gen" HostName="*"><gpnp:Network id="net1" IP="192.0.2.0" Adapter="eth0" Use="public"/><gpnp:Network id="net2" IP="192.168.1.0" Adapter="eth1" Use="cluster_interconnect"/></gpnp:HostNetwork></gpnp:Network-Profile><orcl:CSS-Profile id="css" DiscoveryString="+asm" LeaseDuration="400"/><orcl:ASM-Profile id="asm" DiscoveryString="/dev/sd*" SPFile="+data/spfile.ora"/><ds:Signature

<ds:SignedInfo><ds:CanonicalizationMethod Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#"/><ds:SignatureMethod Algorithm="http://www.w3.org/2000/09/xmldsig#rsa-sha1"/><ds:Reference URI="">

<ds:Transforms><ds:Transform Algorithm="http://www.w3.org/2000/09/xmldsig#enveloped-signature"/><ds:Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#">

<InclusiveNamespaces xmlns="http://www.w3.org/2001/10/xml-exc-c14n#" PrefixList="gpnp orcl xsi"/>

...<ds:DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1"/><ds:DigestValue>gIBakmtUNi9EVW/XQoE1mym3Bnw=</ds:DigestValue>

... <ds:SignatureValue>cgw3yhP/2oEm5DJzdachtfDMbEr2RSfFFUlZujLemnOgsM...=</ds:SignatureValue>

GPnP ProfileA GPnP profile is a small XML file that is used to establish the correct global personality for otherwise identical images. A profile may be obtained at boot time from a remote location, or it may be burned permanently in an image. It contains generic GPnP configuration information and may also contain application-specific configuration data. The generic GPnP configuration data is used to configure the network environment and the storage configuration duringbootup. Application parts of the profile may be used to establish application-specific personalities. The profile is security sensitive. It might identify the storage to be used as the root partition of a machine. A profile must be digitally signed by the provisioning authority, and must be validated on each node before it is used. Therefore, the image must contain the public certificate of the authority to be used for this validation. Profiles are intended to be difficult to change. They may be burned into read-only images or replicated to machines that are down when a change is made. Profile attributes defining node personality include:

• Cluster name• Network classifications (public/private)• Storage to be used for ASM and CSS• Digital signature information

Changes made to a cluster and therefore the profile are replicated by the gpnpd daemon during installation, system boot, or when updated. These updates can be triggered by making changes with configuration tools such as oifcfg, crsctl, asmca, and so on.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Grid Naming Service

• GNS is an integral component of Grid Plug and Play.• The only static IP address required for the cluster is the GNS

virtual IP address.• The cluster subdomain is defined as a delegated domain.

• A request to resolve cluster01-scan.cluster01.example.com would be forwarded to the GNS on 192.0.2.155.

• Each node in the cluster runs a multicast DNS (mDNS) process.

[root@my-dns-server ~]# cat /etc/named.conf// Default initial "Caching Only" name server configuration...# Delegate to gns on cluster01cluster01.example.com #cluster sub-domain# NS cluster01-gns.example.com# Let the world know to go to the GNS vipcluster01-gns.example.com 192.0.2.155 # cluster GNS Address

Grid Naming ServiceEmploying Grid Naming Service (GNS) assumes that there is a DHCP server running on the public network with enough addresses to assign to the VIPs and single client access name (SCAN) VIPs. With GNS, only one static IP address is required for the cluster, the GNS virtual IP address. This address should be defined in the DNS domain. GNS sets up a multicast DNS (mDNS) server within the cluster, which resolves names in the cluster without static configuration of the DNS server for other node IP addresses.The mDNS server works as follows: Within GNS, node names are resolved using link-local multicast name resolution (LLMNR). It does this by translating the LLMNR “.local” domain used by the multicast resolution to the subdomain specified in the DNS query. When you select GNS, an mDNS server is configured on each host in the cluster. LLMNR relies on themDNS that Oracle Clusterware manages to resolve names that are being served by that host.To use GNS, before installation, the DNS administrator must establish domain delegation to the subdomain for the cluster. Queries to the cluster are sent to the GNS listener on the GNS virtual IP address. When a request comes to the domain, GNS resolves it using its internalmDNS and responds to the query.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Single Client Access Name

• The single client access name (SCAN) is the address used by clients connecting to the cluster.

• The SCAN is a fully qualified host name located in the GNS subdomain registered to three IP addresses.

• The SCAN provides a stable, highly available name for clients to use, independent of the nodes that make up the cluster.

# dig @192.0.2.155 cluster01-scan.cluster01.example.com...;; QUESTION SECTION:;cluster01-scan.cluster01.example.com. IN A;; ANSWER SECTION:cluster01-scan.cluster01.example.com. 120 IN A 192.0.2.244cluster01-scan.cluster01.example.com. 120 IN A 192.0.2.246cluster01-scan.cluster01.example.com. 120 IN A 192.0.2.245;; AUTHORITY SECTION:cluster01.example.com. 10800 IN A 192.0.2.155;; SERVER: 192.0.2.155#53(192.0.2.155)

Single Client Access NameThe single client access name (SCAN) is the address used by clients connecting to the cluster. The SCAN is a fully qualified host name (host name + domain) registered to three IP addresses. If you use GNS, and you have DHCP support, then the GNS will assign addresses dynamically to the SCAN. If you do not use GNS, the SCAN should be defined in the DNS to resolve to the three addresses assigned to that name. This should be done before you install Oracle Grid Infrastructure. The SCAN and its associated IP addresses provide a stable name for clients to use for connections, independent of the nodes that make up the cluster.SCANs function like a cluster alias. However, SCANs are resolved on any node in the cluster, so unlike a VIP address for a node, clients connecting to the SCAN no longer require updated VIP addresses as nodes are added to or removed from the cluster. Because the SCAN addresses resolve to the cluster, rather than to a node address in the cluster, nodes can be added to or removed from the cluster without affecting the SCAN address configuration.During installation, listeners are created on each node for the SCAN IP addresses. OracleClusterware routes application requests to the cluster SCAN to the least loaded instance providing the service.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Single Client Access Name (continued)SCAN listeners can run on any node in the cluster. SCANs provide location independence for the databases so that client configuration does not have to depend on which nodes run a particular database.Oracle Database 11g Release 2 and later instances register with SCAN listeners only as remote listeners. Upgraded databases register with SCAN listeners as remote listeners, and also continue to register with all other listeners.If you specify a GNS domain during installation, the SCAN defaults to clustername-scan.GNS_domain. If a GNS domain is not specified at installation, the SCAN defaults toclustername-scan.current_domain. Note: dig: Domain Information Groper


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


GPnP Architecture Overview

LL LL LL LL LL

SL SL SL

mDNS mDNS mDNS mDNS mDNS

remote_listener

local_listenerClient

DNS GNS

DHCP

Scan+port+service Node1

Load

bal

acin

g

Register node VIP & SCAN VIP

One name thatresolves to three VIPs

Get

s th

e th

ree

SCA

N V

IPS

Dyn

amic

VIP

add

ress

esob

tain

ed b

y or

aroo

tage

nt Least

loadednodefor

serviceprofile.xml

Use

1 S

CA

N V

IP

GNS Static VIP

GPnP GPnP GPnP GPnP GPnP Profilereplication

Host name to address resolution

Get Node VIP& SCAN VIP

orarootagent

Node & SCAN VIP

agents

GPnPd discovery

VIP name and hostname resolution

VIP resolution

Node2 Node3 Node4 Noden

…

GPnP Architecture OverviewGPnP ServiceThe GPnP service is collectively provided by all the GPnP agents. It is a distributed method of replicating profiles. The service is instantiated on each node in the domain as a GPnP agent. The service is peer-to-peer, there is no master process. This allows high availability because any GPnP agent can crash and new nodes will still be serviced. GPnP requires standard IP multicast protocol (provided by mDNS), to locate peer services. Using multicast discovery,GPnP locates peers without configuration. This is how a GPnP agent on a new node locates another agent that may have a profile it should use.Name ResolutionA name defined within a GPnP domain is resolvable in the following cases:

• Hosts inside the GPnP domain use normal DNS to resolve the names of hosts outside of the GPnP domain. They contact their regular DNS service and proceed. They may get the address of their DNS server by global configuration, or by having been told by DHCP.

• Within the GPnP domain, host names are resolved using mDNS. This requires an mDNSresponder on each node that knows the names and addresses used by this node, and operating system client library support for name resolution using this multicast protocol. Given a name, a client executes gethostbyname, resulting in an mDNS query. If the name exists, the responder on the node that owns the name will respond with the IP address. The client software may cache the resolution for the given time-to-live value.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

GPnP Architecture Overview (continued)• Machines outside the GPnP domain cannot resolve names in the GPnP domain by using

multicast. To resolve these names, they use their regular DNS. The provisioning authority arranges the global DNS to delegate a sub-domain (zone) to a known address that is in the GPnP domain. GPnP creates a service called called GNS to resolve theGPnP names on that fixed address.The node on which the GNS server is running listens for DNS requests. On receipt, they translate and forward to mDNS, collect responses, translate, and send back to the outside client. GNS is “virtual” because it is stateless. Any node in the multicast domain may host the server. The only GNS configuration is global:

- The address on which to listen on standard DNS port 53;- The name(s) of the domains to serviced.

There may be as many GNS entities as needed for availability reasons. Oracle-provided GNS may use CRS to ensure availability of a single GNS provider.

SCAN and Local ListenersWhen a client submits a connection request, the SCAN listener listening on a SCAN IP address and the SCAN port is contracted on the client’s behalf. Because all services on the cluster are registered with the SCAN listener, the SCAN listener replies with the address of the local listener on the least-loaded node where the service is currently being offered. Finally, the client establishes connection to the service through the listener on the node where service is offered. All of these actions take place transparently to the client without any explicit configuration required in the client.During installation, listeners are created on nodes for the SCAN IP addresses. Oracle Net Services routes application requests to the least loaded instance providing the service. Because the SCAN addresses resolve to the cluster, rather than to a node address in the cluster, nodes can be added to or removed from the cluster without affecting the SCAN address configuration.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


How GPnP Works: Cluster Node Startup

1. IP addresses are negotiated for public interfaces using DHCP:– VIPs– SCAN VIPs

2. A GPnP agent is started from the nodes Clusterware home.

3. The GPnP agent either gets its profile locally or from one of the peer GPnP agents that responds.

4. Shared storage is configured to match profile requirements.

5. Service startup is specified in the profile, which includes:– Grid Naming Service for external names resolution– Single client access name (SCAN) listener

How GPnP Works: Cluster Node StartupWhen a node is started in a GPnP environment:

• Network addresses are negotiated for all interfaces using DHCP• The Clusterware software on the starting node starts a GPnP agent• The GPnP agent on the starting node gets its profile locally or uses resource discovery

(RD) to discover the peer GPnP agents in the grid. If RD is used, it gets the profile from one of the GPnP peers that responds.The GPnP agent acquires the desired network configuration from the profile. This includes creation of reasonable host names. If there are static configurations, they are used in preference to the dynamic mechanisms. Network interfaces may be reconfigured to match the profile requirements.

• Shared storage is configured to match the profile requirements• System and service startup is done as configured in the image. In the case of RAC, the

CSS and CRS system will then be started, which will form the cluster and bring up appropriate database instances. The startup of services may run down their own placeholder values, or may dynamically negotiate values rather than rely on fixed-up configurations. One of the services likely to be started somewhere is the GNS system for external name resolution. Another of the services likely to be started is an Oracle SCAN listener.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


How GPnP Works: Client Database Connections

Database client

DNS

SCANlistener

Listener1

Listener2

Listener3SCANlistener

SCANlistener

GNS

How GPnP Works: Client Database ConnectionsIn a GPnP environment, the database client no longer has to use the TNS address to contact the listener on a target node. Instead, it can use the EZConnect method to connect to the database. When resolving the address listed in the connect string, the DNS will forward the resolution request to the GNS with the SCAN VIP address for the chosen SCAN listener and the name of the database service that is desired. In EZconnect syntax, this would look like:scan-name.cluster-name.company.com/ServiceName, where the service name might be the database name. The GNS will respond to the DNS server with the IP address matching the name given; this address is then used by the client to contact the SCAN listener. The SCAN listener uses its connection load balancing system to pick an appropriate listener, whose name it returns to the client in an OracleNet Redirect message. The client reconnects to the selected listener, resolving the name through a call to the GNS.The SCAN listeners must be known to all the database listener nodes and clients. The database instance nodes cross-register only with the known SCAN listeners, also sending them per-service connection metrics. The SCAN known to the database servers may be profile data or stored in OCR.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Quiz

The init.ohasd entry in the /etc/inittab file is responsible for:1. Starting Oracle Clusterware when the node boots2. Mounting shared volumes as required by Oracle

Clusterware3. Managing node evictions4. Restarting ohasd in the event of a crash

Answers: 4The answer is 4. The init.ohasd entry in the /etc/inittab file is responsible for restarting the Oracle high availability daemon (ohasd) in the event of a crash.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Quiz

Which of the following statements regarding Grid Naming Service is not true?1. GNS is an integral component of Grid Plug and Play.2. Each node in the cluster runs a multicast DNS (mDNS)

process.3. The GNS virtual IP address must be assigned by DHCP.4. The cluster subdomain is defined as a delegated domain.

Answers: 3Statement 3 is not correct. The GNS VIP address must be statically defined.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Module 3: ASM Architecture


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


What Is Oracle ASM?

Application

Operating system

Hardware

Logical Volume Manager

File system

Application

Operating system

Hardware

ASM

What Is Oracle ASM?In essence, Oracle Automatic Storage Management (ASM) is a volume manager and file system built into the Oracle Database server. Raw disk volumes are allocated to ASM for management and control in the same way that raw volumes are managed by a volume manager. ASM is highly integrated with, and highly optimized for, the Oracle Database. It has become the best practice standard for Oracle Database storage.Combining volume management functions with a file system allows a level of integration and efficiency that would not otherwise be possible. For example, ASM is able to avoid the overhead associated with a conventional file system and achieve native raw disk performance for Oracle data files and other files types supported by ASM.ASM is engineered to operate efficiently in both clustered and nonclustered environments. Beginning with Oracle Database 11g Release 2, ASM can also be applied as an enterprise class volume manager and file system for non-Oracle data.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


ASM and ASM Cluster File System

• ASM manages Oracle database files.

• ACFS manages other files.

• Spreads data across disks to balance load

• Provides integrated mirroring across disks

• Solves many storage management challenges

ASM/ADVM

3rd Party FS

Operating System

ACFSDatabase

Application

ASM and ASM Cluster File SystemASM provides storage management in a single machine or across multiple nodes of a cluster for Oracle Real Application Clusters (RAC) support. In Oracle Database 11g Release 2, ASM Dynamic Volume Manager (ADVM) is available to support ASM Cluster File System (ACFS) as a general-purpose file system. ASM distributes input/output (I/O) load across all available disk resources within a disk group to optimize performance while removing the need for manual I/O tuning. ASM helps DBAs manage a dynamic database environment by allowing them to increase the database size without having to shut down the database to adjust the storage allocation. ACFS allows storage managers to extend ASM advantages to general file storage needs. ASM can maintain redundant copies of data to provide fault tolerance, or it can be built on top of vendor-supplied reliable storage mechanisms. Data management is done byselecting the desired reliability and performance characteristics for classes of data on a per-file basis. The capabilities of ASM and ACFS save administrators time by automating storage management and thereby increase their ability to manage larger databases (and more of them) and file systems with increased efficiency. The graphic in the slide illustrates where ASM and ADVM reside in the storage stack. The traditional stack consists of an operating system, followed by a logical volume manager, a file system, a database, and then an application. ASM can replace the logical volume manager and file system for an Oracle database. ADVM with ACFS eliminates the need for a separate cluster file system for shared nondatabase files. ASM is a complete solution.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

ASM and ASM Cluster File System (continued)In the slide, the Oracle 11g Release 2 Automatic Storage Management stack is shown. ASM sits on the OS level and includes the ASM Dynamic Volume Manager (ADVM). At the same level as the database, there is the ASM Cluster File System (ACFS) and the possibility of third-party file systems built on ADVM.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


ASM Key Features and Benefits

• Stripes files rather than logical volumes• Provides redundancy on a file basis• Enables online disk reconfiguration and dynamic

rebalancing• Reduces the time significantly to resynchronize a transient

failure by tracking changes while disk is offline• Provides adjustable rebalancing speed• Is cluster-aware• Supports reading from mirrored copy instead of primary

copy for extended clusters• Is automatically installed as part of the Grid Infrastructure

ASM Key Features and BenefitsASM provides striping and mirroring without the need to purchase a third-party Logical Volume Manager (LVM). ASM divides a file into pieces and spreads them evenly across all the disks. ASM uses an index technique to track the placement of each piece. Traditional striping techniques use mathematical functions to stripe complete logical volumes. ASM is unique in that it applies mirroring on a file basis, rather than on a volume basis. Therefore, the same disk group can contain a combination of files protected by mirroring, or not protected at all.When your storage capacity changes, ASM does not restripe all the data. However, in an online operation, ASM moves data proportional to the amount of storage added or removed to evenly redistribute the files and maintain a balanced I/O load across the disks. You can adjust the speed of rebalance operations to increase or decrease the speed and adjust the impact on the I/O subsystem. This capability also enables the fast resynchronization of disks that may suffer a transient failure.ASM supports all Oracle database file types. ASM supports Real Application Clusters (RAC) and eliminates the need for a cluster Logical Volume Manager or a cluster file system. In extended clusters, you can set a preferred read copy ASM is included in the Grid Infrastructure installation. It is available for both the Enterprise Edition and Standard Edition installations.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


ASM Instance Designs:Nonclustered ASM and Oracle Databases

ASM disk group A ASM disk group B

Single-instancedatabase server

ASM instance

Oracle DB A instance

Oracle DB B instance

Oracle DB C instance

ASM Instance Designs: Nonclustered ASM and Oracle DatabasesThis slide illustrates the first of three ASM instance designs. This design uses a nonclusteredASM environment for one or more nonclustered databases. On the bottom of the graphic, there are two disk groups that the ASM instance uses to provide space for the databases. In this environment, a single host machine contains the ASM instance along with each of the three database instances. Each database can store its files in both disk groups, or only a single disk group if desired. The ASM instance manages the metadata and provides space allocations for the ASM files that are created by each database. When a database instance creates or opens an ASM file, it communicates those requests to the ASM instance. In response, the ASM instance provides file extent map information to the database instance. This design is useful for storage consolidation.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


ASM Instance Designs:Clustered ASM for Clustered Databases


Oracle RACservers

ASM instance 3

Oracle DB A instance 1


Oracle DB B instance 1

ASM instance 2ASM instance 1 ASM instance 4


ASM Instance Designs: Clustered ASM for Clustered DatabasesThis slide illustrates the second of three ASM instance designs. It shows an ASM cluster in an Oracle RAC database environment where ASM provides a clustered pool of storage. For each of the four nodes shown, there is one ASM instance for each node that can provide space management for multiple Oracle RAC or single-instance databases in the cluster. Two RAC databases are shown, with Oracle RAC database “A” having instances on the first two nodes, and then Oracle RAC database “B” having instances on the other two nodes. This represents two distinct RAC databases. However, there is only one ASM cluster of four nodes. Each ASM instance on each machine coordinates with each other when provisioning disk space from the two disk groups shown. In this design, ASM provides storage consolidation, and RAC possibly provides database consolidation.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


ASM Instance Designs:Clustered ASM for Mixed Databases


Database servers

ASM instance 3



Oracle DB C instance 1

ASM instance 2ASM instance 1

Oracle DB C instance 2

ASM Instance Designs: Clustered ASM for Mixed DatabasesThis slide illustrates the third of three ASM instance designs. This design shows a three-node ASM clustered design that provides a clustered storage pool consisting of disk group “A” and disk group “B.” A shared ASM storage pool is achieved by using Oracle Clusterware. However, in this design, clustered and nonclustered database environments are used on each of the three nodes. This design is useful in a grid environment where any instance can be moved to any database server node, and multiple instances can reside on any node.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


ASM System Privileges

• An ASM instance does not have a data dictionary, so the only way to connect to ASM is by using these system privileges:

• The SYS user on ASM is automatically created with the SYSASM privilege.

ASM Privilege Privilege Group Privilege

SYSASM OSASM Full administrative privilege

SYSDBA OSDBA for ASM Access to data stored on ASM Create and delete filesGrant and revoke file access

SYSOPER OSOPER for ASM Limited privileges to start and stop the ASM instance along with a set of nondestructive ALTER DISKGROUP commands

ASM System PrivilegesAn ASM instance does not have a data dictionary, so the only way to connect to an ASM instance is by using one of three system privileges: SYSASM, SYSDBA, or SYSOPER. The table in the slide introduces these ASM system privileges.The SYSDBA privilege on the ASM instance grants access to data stored on ASM. To use SQL*Plus commands to manage ASM components associated with the database, connect as SYSDBA to the database instance rather than the ASM instance. Users connected as SYSDBAcan create and delete files, aliases, directories, and templates; examine various ASM instance views; operate on files that were created by this user; access files to which another user has explicitly granted access; grant ASM file access control to other users. Users connected with the SYSDBA privilege cannot create or resize a disk group when connected with the SYSDBAprivilege. Note: By default, ASMCMD attempts to connect as SYSDBA based on the OS group.Users granted the SYSOPER privilege on the ASM instance are allowed to start up, shut down, mount, dismount, and check disk groups (but not repair). Other operations, such as CREATE DISKGROUP and ADD/DROP/RESIZE DISK, require the SYSASM privilege and are not allowed with the SYSOPER privilege. The SYSOPER privilege does not allow access to system views—for example, v$asm_*. During installation, the ASMSNMP user with SYSDBA privileges is created for monitoring the Oracle ASM instance.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


ASM OS Groups with Role Separation

To separate the duties of ASM administrators and DBAs, there aresix OS groups:

Group For Example OS Group PrivilegeOSASM ASM asmadmin SYSASM

OSDBA ASM asmdba SYSDBA

OSOPER ASM asmoper SYSOPER

oraInventory group

Both oinstall

OSDBA DB dba SYSDBA

OSOPER DB oper SYSOPER

ASM OS Groups with Role SeparationIn Oracle Database 11g, a new OS privileged group was introduced—the OSASM group—which is used exclusively for ASM. Members of this group can connect as SYSASM using OS authentication and have full access to ASM. Initially, only the OS user that installs the Grid Infrastructure software is a member of this group. However, other users can be added. This group also owns the ASM disks. The Oracle binaries must setgid to this group, so that Oracle instances and tools can have access to ASM disks without giving users direct access to these devices. For this example, the name of this group is asmadmin.In this example, the OSDBA group for ASM is called asmdba. Members of this group can connect to the ASM instance as SYSDBA. Every database instance owner that will use ASM for storage must be a member of this group. The OSDBA group for ASM could be a different group than the OSDBA group for a database. In this example, the OSOPER group for ASM is called asmoper. Members of this OS group have the SYSOPER privilege, which limits the set of allowable SQL commands to the minimum required for the basic operation of an already configured ASM instance. The OSOPER group for a database would be a different group.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Authentication for Accessing ASM Instances

There are three modes of connecting to ASM instances:• Local connection using operating system authentication

• Local connection using password file authentication

• Remote connection using Oracle Net Services and password authentication

$ sqlplus / AS SYSASM

SQL> CONNECT / AS SYSOPER

$ sqlplus fred/xyzabc AS SYSASM

SQL> CONNECT bill/abc123 AS SYSASM

$ sqlplus bill/abc123@asm1 AS SYSASM

SQL> CONNECT fred/xyzabc@asm2 AS SYSDBA

Authentication for Accessing ASM InstancesThere are three modes of connecting to ASM instances:

• Local connection using operating system authentication. Operating system users that are members of the OSASM, OSDBA for ASM, or OSOPER for ASM groups can connect to ASM without providing any additional credentials. For example, an operating system user who is a member of the OSASM group can connect to ASM with full administrative privileges using:

CONNECT / AS SYSASMNote: A local connection using AS SYSDBA always uses OS authentication even

when using the username/password syntax.• Local connection using password file authentication. The following example shows a

local connection using password file authentication:CONNECT sys/<sys_password> AS SYSASM

• Remote connection by way of Oracle Net Services using password authentication. Password-based authentication is also supported remotely using Oracle Net Services. The following example shows a remote connection by way of Oracle Net Services using password authentication.

CONNECT sys/<sys_password>@<net_services_alias> AS SYSASM


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Password-Based Authentication for ASM

• Password-based authentication:– Uses a password file– Can work both locally and remotely

— REMOTE_LOGIN_PASSWORDFILE must be set to a value other than NONE to enable remote password-based authentication.

• A password file is created initially:– By Oracle Universal Installer when installing ASM– Manually with the orapwd utility– Containing only the SYS and ASMSNMP users

• Users can be added to the password file using:– SQL*Plus GRANT command– ASMCMD orapwuser command

Password-Based Authentication for ASMFor ASM, password-based authentication uses a password file. Password-based authentication works both locally and remotely. Password-based authentication is enabled by default when the ASM instance is created by the Oracle Universal Installer. You can create a password file for ASM manually using the orapwd utility. To enable remote password-based authentication, the password file must exist, and the initialization parameter REMOTE_LOGIN_PASSWORDFILE must have a value other than NONE. The default value is EXCLUSIVE for ASM instances.To add other users to the password file, you can use the SQL CREATE USER and GRANTcommands. For example:

REM create a new user, then grant the SYSOPER privilegeSQL> CREATE USER new_user IDENTIFIED BY new_user_passwd;SQL> GRANT SYSOPER TO new_user;

With ASMCMD, add a user to the password file with the SYSASM privilege: asmcmd orapwusr –-add –-privilege sysasm new_user


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Managing the ASM Password File

For the ASM instance, the password file:• Can be created by a user that owns the ASM software• Holds roles assigned to users• Is required for Oracle Enterprise Manager to connect to

ASM remotely• Can be viewed from

– SQL*Plus SELECT * FROM V$PWFILE_USERS– ASMCMD lspwusr

Managing the ASM Password FileA password file is required to enable Oracle Enterprise Manager to connect to ASM.You can list the users in the password file and their assigned privileges with the SQL*Plus command: SELECT * FROM V$PWFILE_USERS;

or from ASMCMD with:lspwusr

To revoke a privilege, use the SQL*Plus command: REVOKE SYSASM FROM user

In ASMCMD, you can change the privilege with:asmcmd orapwuser --modify --privilege sysasm user

Note: Whatever privilege is named replaces any other privilege previously granted.In ASMCMD, you can remove a user from the password file with:asmcmd orapwuser --delete user

Note: ASMCMD users cannot connect through the password file. Their privileges are determined by the OS group membership.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Using a Single OS Group

Role/Software Software Owner

Groups/Privileges

Oracle ASM administrator/Oracle Grid Infrastructure home

oracle dba/SYSASM, SYSDBA, SYSOPER

Database administrator 1/Database home 1


Database administrator 2/Database home 2


Operating system disk device owner

oracle dba

Using a Single OS GroupOracle Grid Infrastructure and Oracle Database may be installed with a single OS group for all ASM users.For example, the oracle user could install both the Grid Infrastructure and the Database software, and thus become the owner of both sets of software. The recommended groups in this case are the primary group for the oracle user oinstall and the secondary group dba. The disk devices will be owned by the oracle user. The oracle OS user may connect to the databases as SYSDBA, and the ASM instance as SYSASM. There is no separation of duties. The same user can perform all management functions on the ASM instance and both databases.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Using Separate OS Groups

Role/Software Software Owner

Groups/Privileges/OS Group

Oracle ASM administrator Oracle Grid Infrastructure home

grid asmadmin/SYSASM/OSASMasmdba/SYSDBA/OSDBA for ASM asmoper/SYSOPER /OSOPER for ASM

Database administrator 1Database home 1

oracle1 asmdba/SYSDBAdba1/SYSDBA for db1oper1/SYSOPER for db1

Database administrator 2Database home 2

oracle2 asmdba/SYSDBAdba2/SYSDBAoper2/SYSOPER

Operating system disk device owner

grid asmadmin

Using Separate OS GroupsWhen you implement separate administrator privileges, choose an OSDBA group for the ASM instance that is different from the groups that you select for the database instance. For example, dba1 and dba2 are the privileged groups for database1 and database2; asmadminis the privileged group for ASM. Notice that both database owners are members of theasmdba group. This allows both of them privileges to create, access, and manage ASM data files, but not ASM disk groups. In this example, it is assumed that the storage administrators and database administrators are different people and different groups. The database administrators of database1 are different from the administrators of database2.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


ASM Components:Software

For ASM installation of software:• The directories are located by the operating system

environment variables.– ORACLE_BASE is the top-level directory for a particular

software owner.– ORACLE_HOME is used to identify the top-level directory of

the Grid Infrastructure software.• Use a common ORACLE_BASE for all Oracle products

owned by the same user.• Use an isolated ORACLE_HOME location from other Oracle

products even if they are the same version.• Do not place Grid ORACLE_HOME below ORACLE_BASE.• ORACLE_HOME requires 3 GB to 5 GB of disk space.

ASM Components: Software You are required to create two directories for the installation of the software binaries: the Oracle base and the Oracle home directories. The Oracle base directory is a top-level directory for Oracle software installations per software owner. Optimal Flexible Architecture (OFA) guidelines recommend that you use a path similar to /u01/app/<software owner>, where the software owner is oracle if you use a single owner installation for both Database and Grid Infrastructure, and the owner is grid if you use separate owners for Database and Grid Infrastructure software. Create the operating system environment variable $ORACLE_BASE to locate the Oracle base directory. If you have set the environment variable $ORACLE_BASE for the grid user, then the Oracle Universal Installer creates the Oracle Inventory directory in the $ORACLE_BASE/../oraInventory path. The Oracle Inventory directory (oraInventory) stores an inventory of all software installed on the system.The Oracle home directory is the directory where you choose to install the software for a particular Oracle product. You must install different Oracle products, or different releases of the same Oracle product, in separate Oracle home directories. OFA guidelines recommend that you use a path similar to /u01/app/11.2.0/grid.Create the operating system environment variable $ORACLE_HOME to locate the Oracle home directory for the Grid Infrastructure software, which will require between 3 GB and 5 GB of disk space depending on the type of installation.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


ASM Components:ASM Instance

The ASM instance comprises the process and memory components for ASM.

Other misc. processes

System Global Area (SGA) – memory

Sharedpool

Largepool

ASMcache

Freememory

Processes – CPU components

ASM instance

RBAL ARBn GMON Onnn PZ9n

MARK

ASM Components: ASM InstanceEvery time ASM or a database is started, a shared memory area called the System Global Area (SGA) is allocated and Oracle ASM background processes are started. The combination of the background processes and the SGA is called an Oracle ASM instance. The instance represents the CPU and RAM components of a running ASM environment.The SGA in an ASM instance is different in memory allocation and usage than the SGA in a database instance. The SGA in the ASM instance is divided into four primary areas as follows:

• Shared Pool: Used for metadata information• Large Pool: Used for parallel operations• ASM Cache: Used for reading and writing blocks during rebalance operations• Free Memory: Unallocated memory available

The minimum recommended amount of memory for an ASM instance is 256 MB. Automatic memory management is enabled by default on an ASM instance and will dynamically tune the sizes of the individual SGA memory components. The amount of memory that is needed for an ASM instance will depend on the amount of disk space being managed by ASM. The second part of the ASM instance is the background processes. An ASM instance can have many background processes; not all are always present.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

ASM Components: ASM Instance (continued)The background processes specific to ASM functionality are covered in the next slide. Because the ASM instance shares the same code base as an Oracle database instance, all the required background processes of a database instance will exist in the ASM instance. There are required background processes and optional background processes. Some of these processes may include the following:• ARCn: The archiver processes• CKPT: The checkpoint process• DBWn: The database writer processes• DIAG: The diagnosability process• Jnnn: Job queue processes• LGWR: The log writer process• PMON: The process monitor process• PSP0: The process spawner process• QMNn: The queue monitor processes• RECO: The recoverer process• SMON: The system monitor process• VKTM: The virtual keeper of time process• MMAN: The memory manager process

The preceding list of processes is not complete. There can be hundreds of database instance background processes running depending on the database options and configuration of the instance. For the ASM instance, these processes will not always perform the same tasks as they would in a database instance. For example, the LGWR process in a database instance is responsible for copying change vectors from the log buffer section of the SGA to the online redo logs on disk. The ASM instance does not contain a log buffer in its SGA, nor does it use online redo logs. The LGWR process in an ASM instance copies logging information to an ASM disk group.If ASM is clustered, additional processes related to cluster management will be running in the ASM instance. Some of these processes include the following:• LMON: The global enqueue service monitor process• LMDn: The global enqueue service daemons• LMSn: The global cache service processes• LCKn: The lock processes

Additional processes are started when ADVM volumes are configured.• VDBG: The Volume Driver Background process forwards ASM requests to lock or

unlock an extent for volume operations to the Dynamic Volume Manager driver. The VDBG is a fatal background process so the unplanned death of this process brings down the ASM instance.

• VBGn: Volume Background processes wait for requests from the Dynamic Volume Manager driver, which need to be coordinated with the ASM instance. An example of such a request would be opening or closing an ASM volume file when the Dynamic Volume Manager driver receives an open for a volume (possibly due to a file system mount request) or close for an open volume (possibly due to a file system unmountrequest). The unplanned death of any of these processes does not have an effect on the ASM instance.

• VMB: Volume Membership Background (VMB) coordinates cluster membership with the ASM instance.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


ASM Components:ASM Instance Primary Processes

Process Description

RBAL Opens all device files as part of discovery and coordinates the rebalance activity

ARBn One or more slave processes that do the rebalance activity

GMON Responsible for managing the disk-level activities such as drop or offline and advancing the ASM disk group compatibility

MARK Marks ASM allocation units as stale when needed

Onnn One or more ASM slave processes forming a pool of connections to the ASM instance for exchanging messages

PZ9n One or more parallel slave processes used in fetching data on clustered ASM installation from GV$ views

The ASM instance primary processes are responsible for ASM-related activities.

ASM Components: ASM Instance Primary ProcessesThe ASM instance uses dedicated background processes for much of its functionality. The RBAL process coordinates the rebalance activity for disk groups in an Automatic Storage Management instance. It performs a global open on Automatic Storage Management disks. The ARBn processes perform the actual rebalance data extent movements in an Automatic Storage Management instance. There can be many of these at a time, called ARB0, ARB1, and so on. The GMON process maintains disk membership in ASM disk groups. The MARK process marks ASM allocation units as stale following a missed write to an offline disk. The Onnnprocesses represent the server side of a client/server connection. These processes will appear the moment the instance is started, and will disappear after that. They form a pool of connections to the ASM instance for exchanging messages and only appear when needed. The PZ9n processes represent one or more parallel slave processes that are used in fetching data when ASM is running in a clustered configuration on more than one machine concurrently.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


ASM Components:Node Listener

The node listener is a process that helps establish network connections from ASM clients to the ASM instance.• Runs by default from the Grid $ORACLE_HOME/bin

directory• Listens on port 1521 by default• Is the same as a database instance listener• Is capable of listening for all database instances on the

same machine in addition to the ASM instance• Can run concurrently with separate database listeners or

be replaced by a separate database listener• Is named tnslsnr on the Linux platform

ASM Components: Node ListenerThe node listener is a process that helps establish network connections from ASM clients to the ASM instance. The listener process is installed with the ASM software installation. It runs by default from the Grid $ORACLE_HOME/bin directory on port 1521. The node listener is the same as a database listener or Oracle Net Listener. Because ASM is usually installed before installing the database, the node listener is configured and started automatically to provide a gateway to the ASM instance for remote clients. This listener is capable of listening for all instances on that machine, including database instances. After the ASM installation, a database installation is performed into a different ORACLE_HOME location. Each ORACLE_HOME location can provide an additional listener process and a set of configuration files. The node listener and separate database listeners can run concurrently on different port numbers or the node listener can be replaced by a separate database listener. There are two configuration files for the listener process. They are as follows:

$ORACLE_HOME/network/admin/listener.ora$ORACLE_HOME/network/admin/sqlnet.ora (Optional)

The listener process is named tnslsnr on the Linux platform. It can be managed with thesrvctl utility. The syntax to start and stop the listener is as follows:

srvctl start listener -n nodenamesrvctl stop listener -n nodename


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


ASM Components:Configuration Files

The ASM installation of software uses several configuration files to define the environment.• ASM instance configuration files include:

– The server parameter file (SPFILE), which initializes the ASM instance and defines startup parameters

– orapw+ASM which is the binary password file used for remote authentication to the ASM instance

• Node listener configuration files include:– listener.ora, a text file that defines the node listener– sqlnet.ora, an optional text file that provides additional

listener options• Other miscellaneous text configuration files include:

– /etc/oratab, which lists all the instances on the host machine– /etc/oraInst.loc, which defines the Oracle inventory directory

ASM Components: Configuration FilesMost of the metadata that relates to ASM is stored within the ASM disk group and in the memory-based tables of the ASM instance. A few other files are used for configuration. The ASM instance uses two configuration files. The server parameter file that defines initialization parameters at startup is a binary file created at installation by default in a specified ASM disk group at +<disk_group>/<cluster>/ASMPARAMETERFILE/register.253.nnnnnn. The administrator may create a text initialization parameter file by default named init+ASMn.ora. The text and binary versions are mutually exclusive for startup. The other file that is used to configure the instance is the binary password file namedorapw+ASM. This file is used for local and remote authentication from ASM clients. The password file can be found in the $ORACLE_HOME/dbs directory.The node listener also uses two configuration files. The main configuration file for the listener is the text file listener.ora. This defines the protocol along with protocol-specific information for listening connection endpoints. The other file is the optional text filesqlnet.ora. It is used to define the computer’s domain and default resolution method. Both these configuration files can be found in the $ORACLE_HOME/network/admin directory.There are a few other miscellaneous text configuration files. The /etc/oratab text file lists all the instances on the host machine and the /etc/oraInst.loc text file defines the location of the central Oracle inventory.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


ASM Components:Group Services

Group services provided by Oracle Clusterware allow for cooperating applications to communicate in a peer environment.• Group services for the Oracle environment:

– Provide information necessary to establish connections– Provide assistance in doing lock recovery– Guarantee ASM disk group number uniqueness– Monitor node membership, evictions, and cluster locks

• Oracle Clusterware is responsible for:– Starting and stopping ASM instances– Starting and stopping dependent database instances– Mounting and dismounting disk groups – Mounting and dismounting ACFS volumes

ASM Components: Group ServicesASM depends on group services provided by Oracle Clusterware in both clustered andnonclustered environments. Group services allow for cooperating applications, such as the ASM instance and database instances, to communicate in a peer environment and discover the status of other applications. For ASM, group services:

• Enable the database instance to locate the ASM instance along with credentials to the ASM instance for establishing an Oracle Call Interface (OCI) connection

• Provide assistance in doing lock recovery. The ASM instance maintains certain locks on behalf of database instances. If a database instance were to fail, ASM uses the group services of ocssd.bin to confirm that all database processes are terminated before releasing locks.

• Offer guarantee that the ASM disk group number that is assigned to a disk group name at run time is unique

Oracle Clusterware is responsible for node membership and heartbeat monitoring, basic cluster locking, and node evictions.Oracle Clusterware is responsible for the startup and shutdown of the ASM instance along with the dependent database instances and resources, as well as resources on which ASM depends.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


ASM Components:ASM Disk Group

The ASM disk group is the fundamental object that ASM manages; it:• Consists of one or more ASM disks that provide space• Includes self-contained metadata and logging information

for management of space within each disk group• Is the basis for storage of ASM files• Supports three disk group redundancy levels:

– Normal defaults to internal two-way mirroring of ASM files.– High defaults to internal three-way mirroring of ASM files.– External uses no ASM mirroring and relies on external

disk hardware or redundant array of inexpensive disks (RAID) to provide redundancy.

• Supports ASM files from multiple databases

ASM Components: ASM Disk GroupThe ASM disk group consists of one or more ASM disks that provide space and is the fundamental object that ASM manages. Each disk group contains its own metadata and logging information that is required for the management of space within that disk group. ASM files are allocated from disk groups. This is similar to logical volumes or logical units of other disk management solutions. Any ASM file is completely contained within a disk group, but an ASM disk group can contain files from different databases. Although many disk groups can be created, best practices recommend that only two be created. One disk group should be used for all data for multiple databases, and the other disk group should be used for the flash recovery area for multiple databases.When you create a disk group, you specify an ASM disk group type based on one of the following three redundancy levels:• Normal for two-way mirroring• High for three-way mirroring• External to not use ASM mirroring and rely on the external hardware for redundancy

The redundancy level controls how many disk failures are tolerated without dismounting the disk group or losing data. The redundancy level sets a default for number of copies of ASM file extents, but the redundancy is controlled at the file level.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


ASM Disk Group: Failure Groups

A failure group is a subset of the disks in a disk group, which could fail at the same time because of shared hardware.• Failure groups enable the mirroring of metadata and user

data.• The default failure group creation puts every disk in its own

failure group.• Multiple disks can be placed in a singe failure group at disk

group creation.• Failure groups apply only to normal and high redundancy

disk groups.– A normal redundancy disk group requires at least two failure

groups to implement two-way mirroring of files.– A high redundancy disk group requires at least three failure

groups to implement three-way mirroring of files.

ASM Disk Group: Failure GroupsA failure group is a subset of the ASM disks in a disk group, which could fail at the same time because of shared hardware. The failure of common hardware must be tolerated. Four drives that are in a single removable tray, not protected with RAID, should be in the same failure group because the tray could be removed, thus making all four drives fail at the same time. Drives in the same cabinet could be in multiple failure groups if the cabinet has redundant power and cooling so that it is not necessary to protect against failure of the entire cabinet.Failure groups are used to store mirror copies of data when ASM is used for mirroring by declaring the disk group type to be Normal or High at creation time. A normal redundancy disk group requires at least two failure groups to implement two-way mirroring of files. A high redundancy disk group requires at least three failure groups to implement three-way mirroring of files. There are always failure groups even if they are not explicitly created. If you do not specify a failure group for a disk, that disk is placed in its own failure group with the failure group name the same as the disk name. Therefore, if 20 disks were in a single disk group, there could be 20 failure groups as well. Failure groups have meaning only when used with normal and high redundancy disk groups.All failure groups within the same disk group should be created with the same capacity to avoid space allocation problems.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


ASM Components: ASM Disks

ASM disks are the storage devices provisioned to ASM disk groups.• Are formed from five sources as follows:

– A disk or partition from a storage array– An entire physical disk or partitions of a physical disk– Logical volumes (LV) or logical units (LUN)– Network-Attached Files (NFS)– Exadata grid disk

• Are named when added to a disk group using a different name than the operating system device name

• May use different operating system device names on different nodes in a cluster for the same ASM disk

• Are divided into allocation units (AU) with sizes 1, 2, 4, 8, 16, 32, or 64 MB allowed

ASM Components: ASM DisksASM disks are the storage devices that provide space to ASM disk groups. They are not always the same as physical disks. There are five sources for an ASM disk as follows:

• A disk or partition from a storage array• An entire physical disk or individual partitions of a physical disk• Logical volumes (LV) or logical units (LUN)• Network-Attached Files (NFS)• Exadata grid disk

ASM uses the ASM disk name, not the OS name, that is assigned to the disk when it is added to the ASM disk group. ASM disk names allow a common logical naming convention to be used. When ASM is clustered, the ASM disk name must be the same for all nodes in the cluster, but the operating system name can be different. Every ASM disk is divided into allocation units (AU). An AU is the fundamental unit of allocation within a disk group. A file extent consists of one or more AUs. An ASM file consists of one or more file extents. Each file extent is allocated to a single ASM disk. When you create a disk group, you can set the ASM AU size to be between 1 MB and 64 MB in powers of two, such as 1, 2, 4, … .64. ASM spreads files evenly across all the disks in the disk group. This allocation pattern maintains every disk at the same capacity level and the same I/O load. Different ASM disks should not share the same physical drive.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


ASM Components:ASM Files

ASM files are a limited set of file types stored in an ASM disk group.• Some supported file types:

• Are stored as a set or collection of data extents• Are striped across all disks in a disk group• Use names that begin with a plus sign (+), which are

automatically generated or from user-defined aliases

Control files Flashback logs Data Pump dump sets

Data files DB SPFILE Data Guard configuration

Temporary data files RMAN backup sets Change tracking bitmaps

Online redo logs RMAN data file copies OCR files

Archive logs Transport data files ASM SPFILE

ASM Components: ASM FilesASM files are a limited set of file types stored in an ASM disk group. These files are not created directly by ASM, but ASM automatically generates the file names. ASM file names begin with a plus sign (+) followed by a disk group name. In addition, the file name will contain the database name that created it along with a file type qualifier name. A fully qualified file name has the following form:

+group/dbname/file_type/file_type_tag.file.incarnation

You can think of the plus sign (+) as the root directory of the ASM file system, similar to the slash (/) on Linux file systems. An example of a fully qualified ASM file name is:

+dgroup2/prod/controlfile/current.256.541956473

The chart in the slide lists the valid types of files that can be stored in an ASM disk group. Each ASM file must be contained within a single disk group, but a single Oracle database can have files in multiple disk groups. You can specify user-friendly aliases for ASM files by creating a hierarchical directory structure and use the directory names as prefixes to the file names.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


ASM can use variable size data extents to support larger files, reduce memory requirements, and improve performance.• Each data extent resides on an individual disk.• Data extents consist of one or more allocation units.• The data extent size is:

– Equal to AU for the first 20,000 extents (0–19999)– Equal to 4 × AU for the next 20,000 extents (20000–39999)– Equal to 16 × AU for extents above 40,000

ASM stripes files using extents with a coarse method for load balancing or a fine method to reduce latency.• Coarse-grained striping is always equal to the effective AU

size.• Fine-grained striping is always equal to 128 KB.

ASM Files: Extents and Striping

ASM Files: Extents and StripingThe contents of ASM files are stored in a disk group as a set, or collection, of data extents that are stored on individual disks within disk groups. Each extent resides on an individual disk. Extents consist of one or more allocation units (AU). To accommodate increasingly larger files, ASM uses variable size extents.The initial extent size is equal to the allocation unit size and it increases by a factor of 4 and 16 at predefined thresholds. The data extent size increases by a factor of 4 after 20,000 extents. It will increase by a factor of 16 after 40,000 extents, where it will remain at 16 × AU size.ASM uses coarse-grained striping to balance loads across all the disks in a disk group and fine-grained striping to reduce I/O latency. The coarse-grained stripes are always equal to the effective AU size. The effective AU size is the AU size defined by a disk group at creation. It can vary for each disk group. The fine-grained stripe size always equals 128 KB. The fine-grained stripe is taken from within a series of coarse-grained stripes. Control files, online redo logs, and flashback logs use fine-grained striping by default. All other file types use coarse-grained striping. This default striping method of coarse-grained or fine-grained can be changed with custom templates.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


ASM mirroring is specified at the file level.• Two files can share the same disk group with one file

being mirrored while the other is not.• ASM will allocate the extents for a file with the primary and

mirrored copies in different failure groups.• The mirroring options for ASM disk group types are:

ASM Files: Mirroring

Disk Group Type Supported Mirroring Levels

Default Mirroring Level

External redundancy Unprotected (None) Unprotected (None)

Normal redundancy Two-wayThree-wayUnprotected (None)

Two-way

High redundancy Three-way Three-way

ASM Disk Group: MirroringMirroring protects data integrity by storing copies of data on multiple disks that are isolated from a single failure. ASM mirroring is more flexible than traditional RAID mirroring because you can specify the redundancy level for each file within a disk group. Two files can share the same disk group with one file being mirrored while the other is not.When ASM allocates an extent for a normal redundancy file (two-way mirroring), ASM allocates a primary extent and a secondary extent. ASM chooses the disk on which to store the secondary extent in a different failure group other than the primary extent. The simultaneous failure of all disks in a failure group does not result in data loss.The table in the slide lists the default disk group, the supported mirroring levels for files within that disk group, and the default mirroring level for any file created in the disk group unless a custom mirroring level is designated.With an external redundancy disk group, ASM relies on the storage system to provide RAID functionality. Any write errors will cause a forced dismount of the entire disk group. With normal redundancy, a loss of one ASM disk is tolerated. With high redundancy, a loss of two ASM disks in different failure groups is tolerated.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


ASM Components:ASM Clients

Any active database instance that is using ASM storage and currently connected to the ASM instance is an ASM client.• ASM clients are tracked in the v$asm_client dynamic

performance view.• Each file in ASM is associated with a single database.

Oracle DB A instance

Oracle DB B instance

Oracle DB C instance

ASM instance

ADVM Mounted ACFS

OCR

ASM Components: ASM ClientsAny active database instance that is using ASM storage and currently connected to the ASM instance is an ASM client. Each file in ASM is associated with only one database or client at a time. This can be observed with the fully qualified ASM file name. The second field of the fully qualified ASM file name is the database name as follows:


ASM clients are tracked in the v$asm_client dynamic performance view. There will exist one row for each combination of distinct database instance name and disk group number being used.An OCR file stored in ASM is listed as an ASM client with the name +ASM.ADVM volumes are ASM clients and the file system must be dismounted before the instance can be shut down. ASM volumes have a client name of asmvol. Note: Many utilities are capable of connecting to the ASM instance to perform administration. These utilities are sometimes referred to as ASM clients, but they do not appear in the v$asm_client dynamic performance view. These will be referred to as ASM utilities in this course.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


ASM Components:ASM Utilities

Many utilities can be used for ASM administration. These utilities may include:• Oracle Universal Installer (OUI)• ASM Configuration Assistant (ASMCA)• Oracle Enterprise Manager (EM)• SQL*Plus• ASM Command-Line utility (ASMCMD)• Listener controller utility (lsnrctl)• Server controller utility (srvctl)• XML DB (FTP and HTTP)

ASM Components: ASM UtilitiesThere are many utilities that can be used for ASM administration. Some of these utilities connect directly to the ASM instance and others can scan the ASM disks headers directly. The utilities may include:

• Oracle Universal Installer (OUI): Is used to install the ASM software and can create the initial disk groups

• ASM Configuration Assistant (ASMCA): Is used to initially configure the ASM instance and create disk groups. It can be invoked from within the OUI utility.

• Oracle Enterprise Manager (EM): Is used to perform central administration of a grid environment, including the ASM instance using a graphical client interface

• SQL*Plus: Is used to provide command-line SQL language access to the ASM instance. All ASM commands and administration can be performed with this utility.

• ASM Command-Line utility (ASMCMD): Is used for ASM administration from the command line without the SQL language. It uses the UNIX style syntax.

• Listener controller utility: (server controller is preferred) Is used to start, stop, and check the status of the listener

• Server controller utility: Is used to start, stop, and check the status of all the ASM instances in a cluster environment with a single command

• XML DB: Provides FTP and HTTP access to ASM files


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


ASM Scalability

ASM imposes the following limits:• 63 disk groups in a storage system• 10,000 ASM disks in a storage system• Two-terabyte maximum storage for each ASM disk (non-

Exadata)• Four-petabyte maximum storage for each ASM disk

(Exadata)• 40-exabyte maximum storage for each storage system• 1 million files for each disk group• ASM file size limits (database limit is 128 TB):

– External redundancy maximum file size is 140 PB.– Normal redundancy maximum file size is 42 PB.– High redundancy maximum file size is 15 PB.

ASM ScalabilityASM imposes the following limits:

• 63 disk groups in a storage system• 10,000 ASM disks in a storage system• Two-terabyte maximum storage for each ASM disk not on Oracle Exadata storage • Four-petabyte maximum storage for each ASM disk on Oracle Exadata storage• 40-exabyte maximum storage for each storage system• 1 million files for each disk group

The Oracle database supports data file sizes up to 128 TB. ASM supports file sizes greater than 128 TB in any redundancy mode. This provides near-unlimited capacity for future growth. The ASM file size limits are as follows:

• External redundancy: 140 petabytes• Normal redundancy: 42 petabytes• High redundancy: 15 petabytes

Some standard metric prefixes include the following:• 1 gigabyte = 1,000,000,000 bytes or 109 bytes using a decimal prefix• 1 terabyte = 1,000,000,000,000 bytes or 1012 bytes using a decimal prefix• 1 petabyte = 1,000,000,000,000,000 bytes or 1015 bytes using a decimal prefix• 1 exabyte = 1,000,000,000,000,000,000 bytes or 1018 bytes using a decimal prefix


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Summary

In this lesson, you should have learned how to:• Explain the principles and purposes of clusters• Describe Cluster hardware best practices• Describe the Oracle Clusterware architecture• Describe how Grid Plug and Play affects Clusterware• Describe the ASM architecture• Describe the components of ASM


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Copyright © 2010, Oracle and/or its affiliates. All rights reserved..

Grid Infrastructure Installation

Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Objectives

After completing this lesson, you should be able to:• Perform preinstallation tasks for Grid Infrastructure• Install Grid Infrastructure• Verify the installation• Configure Automatic Storage Management (ASM) disk

groups• Configure ASM volumes• Make the ASM cluster file system• Mount ASM Cluster File System (ACFS) volumes


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Module 1: Preinstallation Planning


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Shared Storage Planning for Grid Infrastructure• There are three ways of storing Grid Infrastructure files:

– A supported Cluster File System (CFS)– A certified Network File System (NFS)– Automatic Storage Management (ASM)

• New installations on block or raw devices using DBCA or OUI are not allowed.

• When upgrading an existing RAC database, you can use an existing raw or block device partition and perform a rolling upgrade of the installation.

Storage Option Voting/ OCR Oracle SoftwareAutomatic Storage Manager (ASM) Yes NoASM Cluster File System (ACFS) No YesOracle Cluster File System (OCFS2) Yes YesRed Hat Global File System (GFS) Yes YesNFS (certified only) Yes YesShared disk slices (block or raw devices) No No

Shared Storage Planning for Grid InfrastructureThe Oracle Clusterware configuration files (voting and Oracle Cluster Registry [OCR]) can be stored on a supported cluster file system or network file system. With the introduction of Oracle 11g Release 2, the ASM option can now be used to store OCR and voting files. In conjunction with ASM Cluster File System, ASM now provides a complete shared storage solution for Oracle RAC. Other vendor Storage Area Network (SAN) solutions fall under the category of shared block devices in the chart mentioned in the slide.With the release of Oracle Database 11g Release 2, using DBCA)or the Universal Installer to store Oracle Clusterware or Oracle Database files directly on block or raw devices is not supported although the command line interfaces still accept them.If you intend to upgrade an existing Oracle RAC database, or an Oracle RAC database with ASM instances, then you can use an existing raw or block device partition and perform a rolling upgrade of your existing installation.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Sizing Shared Storage

Minimum sizing for Oracle Clusterware configuration files using External, Normal, or High redundancy:

Redundancy Minimum # of Disks

OCR Files Voting Disk Files

Total

External 1 280 MB 280 MB 560 MB

Normal 3 560 MB 840 MB 1.4 GB

High 5 840 MB 1.4 GB 2.3 GB

Sizing Shared StorageIf you choose to place your configuration files on a shared file system, Oracle recommends that the disk used for the file system be on a highly available storage device such as redundant array of inexpensive disks (RAID), or that two file systems be mounted and OracleClusterware be used to provide redundancy. When you use external redundancy, the minimum requirement is for one Oracle Cluster Registry (OCR) file and one voting disk of 280 MB each. If Oracle Clusterware is used for implementing normal redundancy, two OCR files and three voting disks of 280 MB each should be created for a total of 1.4 GB. Only five OCR files may be created, but Oracle Clusterware allows up to 15 voting disks to be created. Always configure an odd number of voting disks.The user account with which you perform the installation must have write permissions to create the files in the path that you specify during the installation. The OCR file locations should have the root:oinstall ownership and the voting disk should have thecrs:oinstall ownership. All file locations should have 0640 permission patterns. These requirements are only for Oracle Clusterware. Additional space for database and applications need to be planned for.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Storing the OCR in ASM

• The Oracle Cluster Registry (OCR) can be stored in ASM.• It is protected by mirroring based on the redundancy

settings for the ASM disk group that contains it.

Storing the OCR in ASMOracle Clusterware uses the Oracle Cluster Registry (OCR) to store and manage information about the components that Oracle Clusterware controls, such as Oracle RAC databases, services, listeners, and virtual IP addresses (VIPs) along with any other applications. OCR stores configuration information in a series of key-value pairs in a tree structure. To improve Oracle Clusterware storage manageability, the OCR is now configured by default to use Automatic Storage Management (ASM). With the Oracle Clusterware storage residing in an ASM disk group, you can manage both database and clusterware storage using Oracle Enterprise Manager.The OCR is stored like any other data file in ASM as shown in the slide. If you upgrade from a previous version of Oracle Clusterware, you can migrate your OCR location or locations to reside on ASM.Storing the OCR on ASM eliminates the need for third-party cluster volume managers and eliminates the complexity of managing separate disk partitions for the OCR in Oracle Clusterware installations.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Oracle Clusterware Repository (OCR)

Node1

OCR cache

CRSDprocess

Clientprocess

Node2

OCR cache

CRSDprocess

Node3

OCR cache

CRSDprocess

Clientprocess

OCRprimary

file

Sharedstorage OCR

secondaryfile

Oracle Clusterware Repository (OCR)Cluster configuration information is maintained in the OCR. The OCR relies on distributed shared-cache architecture for optimizing queries, and clusterwide atomic updates against the cluster repository. Each node in the cluster maintains an in-memory copy of OCR, along with the CRSD that accesses its OCR cache. Only one of the CRSD processes actually reads from and writes to the OCR file on shared storage. This process is responsible for refreshing its own local cache, as well as the OCR cache on other nodes in the cluster. For queries against the cluster repository, the OCR clients communicate directly with the local CRS daemon (CRSD) process on the node from which they originate. When clients need to update the OCR, they communicate through their local CRSD process to the CRSD process that is performing input/output (I/O) for writing to the repository on disk.The main OCR client applications are OUI, SRVCTL, Enterprise Manager (EM), the Database Configuration Assistant (DBCA), the Database Upgrade Assistant (DBUA), Network Configuration Assistant (NETCA), and the ASM Configuration Assistant (ASMCA). Furthermore, OCR maintains dependency and status information for application resources defined within Oracle Clusterware, specifically databases, instances, services, and node applications.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Oracle Clusterware Repository (OCR) (continued)The installation process for Oracle Clusterware gives you the option of automatically mirroring OCR. This creates a second OCR file, which is called the OCR mirror file, to duplicate the original OCR file, which is called the primary OCR file. Although it is recommended to mirror your OCR, you are not forced to do it during installation.The Oracle Grid Infrastructure installation now defines three locations for the OCR and supports up to five. New installations to raw devices are no longer supported.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Managing Voting Disks in ASM

• Each node must be able to access a majority of voting disks; otherwise, it will be evicted from the cluster.

• Voting disks can be stored on an ASM disk group.– They are not regular ASM files.– Clusterware knows the location in case ASM is unavailable.

• The number of voting disks is determined by the ASM disk group redundancy setting.– 1 for external redundancy disk group– 3 for normal redundancy disk group– 5 for high redundancy disk group

• A separate failure group is required for each voting disk.• Voting disks are managed using the crsctl utility.

Managing Voting Disks in ASMOracle Clusterware uses voting disk files, also called voting disks, to determine which nodes are members of a cluster and to maintain the integrity of the cluster. If you configure voting disks on ASM, then you do not need to manually configure the voting disks. Depending on the redundancy of the specified disk group, a predefined number of voting disks are created.ASM manages voting disks differently from other files that it stores. When you initially configure Oracle Clusterware, you specify the disk group to contain the voting disks. Each voting disk is housed in a separate ASM failure group. You must specify enough failure groups to support the number of voting disks associated with each disk group redundancy setting. For example, you must have at least three failure groups to store your voting disks in a normal redundancy disk group. Voting disks do not appear as regular files within ASM, ratherClusterware records exactly where the voting disk information is located. This arrangement exists so that if ASM is unavailable for any reason, then Cluster Synchronization Services can still access the voting disks and maintain the cluster.One of the benefits of using an ASM disk group, with either normal or high redundancy, is that if a disk containing a voting disk fails, then as long as there is another disk available in the disk group, ASM automatically recovers the voting disk. Voting disks are managed using the crsctl utility. For example, the following command migrates voting disks from their current location to an ASM disk group named VOTE:

# crsctl replace votedisk +VOTE


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


CSS Voting Disk Function

Node1 Node2

CSS CSS

Voting disk

Node3

CSS

Node1 Node2

CSS CSS

Voting disk

Node3

CSS

Nodes cansee each other.

Node3 can no longercommunicate throughprivate interconnect.Others no longer see

its heartbeats and evict that node by

using the voting disk.

I do not see 3.Node1: I see 1&2.Node2: I see 1&2.

=>We shouldevict 3!

I see 1&2.

I see 3.

I’ve beenevicted!

I’d better stop!

I see1, 2, & 3.

I see1, 2, & 3.

I see 1, 2, & 3.

Split-brain

(RMN)

CSS Voting Disk FunctionCSS is the service that determines which nodes in the cluster are available and provides cluster group membership and simple locking services to other processes. CSS typically determines node availability via communication through a dedicated private network with a voting disk used as a secondary communication mechanism. This is done by sending heartbeat messages through the network and the voting disk as illustrated by the top graphic in the slide. The voting disk is a file on a clustered file system that is accessible to all nodes in the cluster. Note that with the current release, the voting disk is supported on raw devices only for backward compatibility and migrating earlier versions to 11g Release 2. Its primary purpose is to help in situations where the private network communication fails. The voting disk is then used to communicate the node state information used to determine which nodes go offline. Without the voting disk, it can be difficult for isolated nodes to determine whether it is experiencing a network failure or whether the other nodes are no longer available. It would then be possible for the cluster to enter a state where multiple subclusters of nodes would have unsynchronized access to the same database files. The bottom graphic illustrates what happens when Node3 can no longer send heartbeats to other members of the cluster. When others can no longer see Node3’s heartbeats, they decide to evict that node by using the voting disk. When Node3 reads the removal message or “kill block,” it generally reboots itself to ensure that all outstanding write I/Os are lost. Oracle Clusterware supports up to 15 redundant voting disks.Note: The voting disk or file is usually known as quorum disk in vendor clusterware.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Oracle Clusterware Main Log Files

<Grid_Home>

<hostname>

crsd cssd evmd

log

alert<nodename>.log

racg

gpnpd mdnsd ohasd

clientadmin ctssdagent

diskmon gipcd gnsd gnsd

Oracle Clusterware Main Log FilesOracle Clusterware uses a unified log directory structure to consolidate the Oracle Clusterwarecomponent log files. This consolidated structure simplifies diagnostic information collection and assists during data retrieval and problem analysis.The slide shows you the main directories used by Oracle Clusterware to store its log files:

• CRS logs are in <Grid_Home>/log/<hostname>/crsd/. The crsd.log file is archived every 10 MB (crsd.l01, crsd.l02, …).

• CSS logs are in Grid_HOME /log/<hostname>/cssd/. The cssd.log file is archived every 20 MB (cssd.l01, cssd.l02, …).

• EVM logs are in <Grid_Home>/log/<hostname>/evmd.• SRVM (srvctl) and OCR (ocrdump, ocrconfig, ocrcheck) logs are in

<Grid_Home>/log/<hostname>/client/ and $ORACLE_HOME/log/<hostname>/client/.

• Important Oracle Clusterware alerts can be found in alert<nodename>.log in the <Grid_Home>/log/<hostname> directory.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Installing ASMLib

To install ASMLib manually:• Determine the correct oracleasm package.

• Download and install the oracleasm package from:http://www.oracle.com/technology/tech/linux/asmlib/index.html

• Download the oracleasmlib and oracleasm-supportpackages for your version of Linux.

• Download the oracleasm package corresponding to your kernel version.

• Log in as root and install the ASM packages.

# uname –rm

2.6.9-5.ELsmp i686

# rpm -Uvh oracleasm-support-version.arch.rpm \oracleasm-kernel-version.arch.rpm \oracleasmlib-version.arch.rpm

Installing ASMLibInstall the Linux ASMLib RPMs to simplify storage administration. ASMLib provides persistent paths and permissions for storage devices used with ASM, eliminating the need for updating udev or devlabel files with storage device paths and permissions. ASMLib 2.0 is delivered as a set of three Linux packages:• oracleasmlib-2.0: The ASM libraries• oracleasm-support-2.0: Utilities needed to administer ASMLib• oracleasm: A kernel module for the ASM library

Each Linux distribution has its own set of ASMLib 2.0 packages, and within each distribution, each kernel version has a corresponding oracleasm package.If you are registered with the Oracle Unbreakable Linux Network, then you can download and install ASMLib packages for your system automatically. To install ASMLib from ULN:

1. Log in as root.2. Run the following command:

# up2date -i oracleasm-support oracleasmlib oracleasm-’uname -r’

To install ASMLib manually, follow the steps in the slide.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Preparing ASMLib

• Configure ASMLib.

• Mark ASM candidate disk partitions.

• Make ASM disks available to the other cluster nodes.

# /usr/sbin/oracleasm configure -i

# /usr/sbin/oracleasm createdisk ASM_DISK_NAME \candidate_disk

# /usr/sbin/oracleasm scandisks

Preparing ASMLibConfiguring ASMLibLog in as root and enter the following command:# oracleasm configure -i

Provide information as prompted for your system. If you enter the oracleasmconfigure command without the -i flag, then you are shown the current configuration. Then execute the oracleasm init command.Marking ASM Candidate Disk PartitionsFor OUI to recognize partitions as ASM disk candidates, you must log in as root and mark the disk partitions that ASM can use. To mark a disk for use by ASM, enter the following command syntax, where ASM_DISK_NAME is the name of the ASM disk group, and candidate_disk is the name of the disk device that you want to assign to that disk group:oracleasm createdisk ASM_DISK_NAME candidate_disk

For example: # oracleasm createdisk data1 /dev/sdf

Making ASM Disks Available to All NodesTo make the disk available on the other nodes in the cluster, enter the following command as root on each node: # /usr/sbin/oracleasm scandisks

This command identifies shared disks attached to the nodes that are marked as ASM disks.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Quiz

The Oracle Cluster Registry (OCR) can now be stored in ASM.1. True2. False

Answer: 1This statement is true.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Module 2: Grid Infrastructure Preinstallation Tasks


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Oracle Grid Infrastructure 11g Installation

1. Check system requirements.2. Check network requirements.3. Install required operating system packages.4. Set kernel parameters.5. Create groups and users.6. Create required directories.7. Configure installation owner shell limits.8. Install Oracle Clusterware.

Oracle Grid Infrastructure 11g InstallationTo successfully install Oracle Grid Infrastructure, it is important that you have an understanding of the tasks that must be completed and the order in which they must occur. Before the installation can begin in earnest, each node that is going to be part of your cluster installation must meet the hardware and software requirements that are covered in this lesson. You must perform step-by-step tasks for hardware and software verification, as well as for platform-specific preinstallation procedures. You must install the operating system patches required by the cluster software and verify that the kernel parameters are correct for your needs.Oracle Grid Infrastructure must be installed by using the graphical OUI. Character-based tool installations are not possible; however, OUI can be run in silent batch mode using response files to supply the values that the installation would need. Ensure that your cluster hardware is functioning normally before you begin this step. Failure to do so results in an aborted ornonoperative installation.If you intend to use Enterprise Manager Grid Control to manage your cluster deployments, you must next install the Enterprise Manager (EM) agent on each cluster node. Note: This lesson provides details about performing an installation, but it should not be used as a substitute for the Installation manual for your platform.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Checking System Requirements

• At least 1.5 GB of physical memory is needed.

• A minimum of 1.5 GB of swap space is required.

• The local /tmp directory should have at least 1 GB free.

• At least 4.5 GB of local storage on each node for the software is needed.

# grep MemTotal /proc/meminfoMemTotal: 2097320 kB

# grep SwapTotal /proc/meminfoSwapTotal: 4194296 kB

# df -h /tmpFilesystem Size Used Avail Use% Mounted on/dev/sda6 9.7G 2.3G 7.0G 25% /

Checking System RequirementsThe system must meet the following minimum hardware requirements:

• The minimum required RAM is 1.5 GB for grid infrastructure for a cluster, or 2.5 GB for grid infrastructure for a cluster and Oracle RAC. To determine the amount of physical memory, enter the following command:

# grep MemTotal /proc/meminfo

• The minimum required swap space is 1.5 GB. Oracle recommends that you set swap space to 1.5 times the amount of RAM for systems with 2 GB of RAM or less. For systems with 2 GB to 16 GB RAM, use swap space equal to RAM. For systems with more than 16 GB RAM, use 16 GB of RAM for swap space. To determine the size of the configured swap space, enter the following command:

grep SwapTotal /proc/meminfo• At least 1 GB of disk space must be available in the /tmp directory (TMP and TMPDIR

variables can force another location to be used). To determine the amount of disk space available in the /tmp directory, enter the following command:

df -h /tmp. • At least 1.4 GB of disk space for the Oracle Clusterware configuration files is needed,

assuming the standard redundancy of two OCR files and three voting disks. This must be the shared storage accessible by every node.

• At least 4.5 GB for the Grid Infrastructure software home is required. Oracle 11g: RAC and Grid Infrastructure Administration Accelerated 2 - 17

Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Checking Network Requirements

• Each node must have at least two network interface cards (NICs).

• Interface names must be the same on all nodes.• Public NIC must support TCP/IP and Private NIC UDP.• Public IP must be registered in the domain name server

(DNS) or the /etc/hosts file.

• IF GNS is used, the Cluster GNS address must be registered in the DNS.

• Prevent public network failures when using NAS devices or NFS mounts by starting the Name Service Cache Daemon.

# cat /etc/hosts##### Public Interfaces – eth0 (odd numbers)####xxx.xxx.100.11 host01.example.com host01xxx.xxx.100.13 host02.example.com host02

# /sbin/service nscd start

Checking Network RequirementsThe following is a list of requirements for network configuration:

• Each node must have at least two network interface cards (NICs): one for the public network and one for the private network.

• The interface names must be the same on all nodes in the cluster.• On the public network, each NIC must support TCP/IP. On the private network, each

NIC must support User Datagram Protocol (UDP). Windows platforms use TCP only.• If domain name servers (DNS) are being used, the public IP addresses should be

registered. • To prevent public network failures when using Network Attached Storage (NAS) devices

or Network File System (NFS) mounts, enter the following command as root to enable the Name Service Cache Daemon (nscd):

# /sbin/service nscd start

• Ensure that each node is properly identified using the hostname and ifconfigutilities.

• If the time stamps among the nodes differ significantly, node evictions and reboots can occur. Network Time Protocol (NTP) can be used to synchronize time stamps between cluster nodes.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Checking Network Requirements (Continued)You can configure IP addresses with one of the following options:

• Oracle Grid Naming Service (GNS) using one static address defined during installation, with dynamically allocated addresses for all other IP addresses, obtained from your organization’s Dynamic Host Configuration Protocol (DHCP) server, and resolved using a multicast domain name server configured within the cluster

• Static addresses that network administrators assign on a network domain name server (DNS) for each node

Please note that the cluster verify utility has been updated to require the -x option for ntpdfor Linux systems. For NTP, the maximum slew rate possible is limited to .5 ms/s as a consequence of the principles on which the NTP protocol and algorithm design are based. As a result, the local clock can take a long time to converge to an acceptable offset, about 2,000s for each second the clock is outside the acceptable range. As a result, an adjustment as much as five minutes (300s) s will take almost 7 days to complete.During this interval the local clock will not be consistent with other network clocks and the system cannot be used for distributed applications that require correctly synchronized network time.As the result of this behavior, once the clock has been set, it very rarely strays more than 128ms, even under extreme cases of network path congestion and jitter. Sometimes, in particular when ntpd is first started, the error might exceed 128ms. This may on occasion cause the clock to be stepped backwards if the local clock time is more than 128s in the future relative to the server. In some applications, this behavior may be unacceptable. If the -xoption is included on the command line, the clock will never be stepped and only slew corrections will be used. In practice this reduces the false alarm rate where the clock is stepped in error to a vanishingly low incidence.If NTP is not configured, Oracle Clusterware installs a cluster time daemon, csstd in observer mode.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Checking Network Requirements (continued)You can configure IP addresses with one of the following options:

• Oracle Grid Naming Service (GNS) using one static address defined during installation, with dynamically allocated addresses for all other IP addresses, obtained from your organization’s Dynamic Host Configuration Protocol (DHCP) server, and resolved using a multicast domain name server configured within the cluster

• Static addresses that network administrators assign on a network domain name server (DNS) for each node


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Software Requirements (Kernel)

Linux x86 (32-bit and 64-bit) operating system kernel requirements

Linux Distribution RequirementsAsianux Asianux 2, kernel 2.6.9 or later

Asianux Server 3, kernel 2.6.18 or laterOracle Enterprise Linux (OEL) OEL 4 Update 7, kernel 2.6.9 or later

OEL 5 Update 2, kernel 2.6.18 or laterRed Hat Enterprise Linux (RHEL) RHEL 4 Update 7, kernel 2.6.9 or later

RHEL 5 Update 2, kernel 2.6.18 or laterSUSE Enterprise Linux SUSE 10, kernel 2.6.16.21 or later

SUSE 11, kernel 2.6.27.19 or later

Software Requirements (Kernel)For installations of Oracle Grid Infrastructure, ensure that you have the kernel versions and packages listed in the table in the slide. If you intend to install Oracle Database or Oracle RAC in addition to Oracle Grid Infrastructure, then refer to the Oracle Grid Infrastructure Installation Guide, 11g Release 2 (11.2) for Linux to determine whether you must install additional packages for the features you plan to use.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Software Requirements: Packages

Linux x86 Oracle 32-bit Grid Infrastructure and Oracle RAC RPM requirements for Asianux Server 3, OEL 5, and RHEL 5

binutils-2.17.50.0.6compat-libstdc++-33-3.2.3elfutils-libelf-0.125elfutils-libelf-devel-0.125glibc-2.5-24glibc-common-2.5glibc-devel-2.5glibc-header-2.5gcc-4.1.2gcc-c++-4.1.2

libaio-0.3.106libaio-devel-0.3.106libgcc-4.1.2libgomp-4.1.2libstdc++-4.1.2libstdc++-devel-4.1.2make-3.81sysstat-7.0.2unixODBC-2.2.11unixODBC-devel-2.2.11

Software Requirements: PackagesIf you intend to install an Oracle Database or Oracle cluster database in addition to Oracle Grid Infrastructure, then refer to the list in the slide to determine whether you must install additional packages for the features you plan to use. If you are running Asianux Server 3, OEL5, or RHEL5, refer to the list in the slide for RPM requirements. To determine whether the required packages are installed, enter a command similar to the following: # rpm –qa | grep package_name

It is best to use the grep command to search for the package name without the version information because newer versions may be installed. Install missing packages and their dependencies with the following syntax: # rpm –ivh package_name


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Oracle Validated Configuration RPM

When installed, the Oracle Validated Configuration RPM does the following:• Installs any additional packages needed for installing

Oracle Grid Infrastructure and Oracle Database• Creates an oracle user and the oraInventory

(oinstall) and OSDBA (dba) groups• Sets sysctl.conf settings, startup

parameters, user limits, and driver parameters to values based on recommendations from the Oracle Validated Configurations program

Oracle Validated Configuration RPMIf your Linux distribution is Oracle Enterprise Linux or Red Hat Enterprise Linux, then you can complete most preinstallation configuration tasks by using the Oracle Validated Configuration RPM, available from the Unbreakable Linux Network. When it is installed, the Oracle Validated Configuration RPM does the following:

• Automatically installs any additional packages needed for installing Oracle Grid Infrastructure and Oracle Database

• Creates an oracle user and the oraInventory (oinstall) and OSDBA (dba) groups• Sets and verifies sysctl.conf settings, startup parameters, user limits, and driver

parameters to values based on recommendations from the Oracle Validated Configurations program

If you are using Oracle Enterprise Linux 4.7 and later or Oracle Enterprise Linux 5.2 and later, then the Oracle Validated Configuration RPM is included on the installation media. Use the following procedure to subscribe to Oracle Unbreakable Linux channels and to add the Oracle Software for Enterprise Linux channel that distributes the Oracle Validated Configuration RPM:

1. Complete a default Oracle Enterprise Linux workstation installation or a default Red Hat Enterprise Linux installation.

2. Register your server with Unbreakable Linux Network (ULN). By default, you are registered for the Enterprise Linux Latest channel for your operating system and hardware.Oracle 11g: RAC and Grid Infrastructure Administration Accelerated 2 - 23

Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Oracle Validated Configuration RPM (continued)3. Log in to ULN at the following URL: https://linux.oracle.com.4. Click the Systems tab, and in the System Profiles list, select a registered server. The

System Details window opens and displays the subscriptions for the server.5. From the Available Channels list, select the Oracle Software for Enterprise Linux

channel that is appropriate for your installation of Linux—for example, “Oracle Software for Enterprise Linux 4 (x86_64).”

6. Click Subscribe.7. From a terminal session, as root, enter the following command:

# up2date --nox --show-channels

You should see output indicating that you have subscribed to the Oracle Software for Enterprise Linux channel. For example:el4_i386_latestel4_i386_oracle

8. Open a terminal session as root and install the Oracle Validated Configuration RPM with up2date, using the following command:# up2date --install oracle-validated

9. Repeat steps 1 through 8 on all other servers in your cluster.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Creating Groups and Users

• Create an Oracle Software inventory group on each node.• Group ID must be consistent on each node.

• Create the Oracle Software owner on each node. • User ID must be consistent on each node and the

inventory group must be the primary group.• Most Oracle products (Grid Infrastructure, database,

Enterprise Manager, and so on) are usually owned by the same user, typically called oracle, but each product can be owned by a different user.

# groupadd –g 501 oinstall

# useradd –u 501 –g oinstall oracleor

# useradd –u 502 –g oinstall grid

Creating Groups and UsersAn operating system group needs to be created that will be associated with Oracle Central Inventory (oraInventory). oraInventory contains a registry of the Oracle home directories from all Oracle products installed on the server. It is designed to be shared among many products. It also contains installation log files and trace files from the installation programs. The suggested name for the operating system group is oinstall. In a cluster installation, it is very important that the group ID be the same on all nodes of the cluster.An operating system user needs to be created to own the Oracle Clusterware installation. Traditionally, all Oracle products installed on the same machine such as clusterware, databases, disk management, and enterprise management tools are owned by the same user called oracle. It is possible for each product to be created under a different operating system account. This may be desired if different job responsibilities are used to manage different components. It is very important that the user ID be the same on all nodes of the cluster.It is required that the Oracle Clusterware software owner and the Oracle Database software owner have the same primary group as the Oracle Inventory group.Note: If this installation of Oracle Clusterware contains a database and other Oracle products, consider creating the following groups: dba and oper.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Creating Groups and Users: ExampleCreate Groups:# /usr/sbin/groupadd -g 1000 oinstall# /usr/sbin/groupadd -g 1100 asmadmin# /usr/sbin/groupadd -g 1200 dba# /usr/sbin/groupadd -g 1201 oper# /usr/sbin/groupadd -g 1300 asmdba# /usr/sbin/groupadd -g 1301 asmoper

Create UsersIf a single software owner is needed:# /usr/sbin/useradd -u 1100 -g oinstall -G asmdba,dba, \asmoper oracle

If a separate owner for Grid Infrastructure is needed:# /usr/sbin/useradd -u 1100 -g oinstall -G asmdba,dba, \oracle# /usr/sbin/useradd -u 1101 -g oinstall -G asmdba, \asmadmin,asmoper grid

Create Directories# mkdir -p /u01/app/grid /u01/app/oracle# chown -R grid:oinstall /u01/app# chown oracle:oinstall /u01/app/oracle# chmod 775 /u01/app

1

2

3

Creating Groups and Users: ExampleThe following is an example of how to create the Oracle Inventory group (oinstall), OSDBA, OSASM, and OSOPER for ASM groups where they are the same group (dba), and how to create the Grid Infrastructure software owner (grid) and one Oracle Database owner (oracle) with correct group memberships. This example shows how to configure an Oracle base path compliant with Optimal Flexible Architecture structure with correct permissions:

1. Log in as root. Enter commands similar to the example in the slide to create theoinstall, asmadmin, and asmdba groups, and if required, the asmoper, dba, andoper groups. Use the -g option to specify the correct group ID for each group.

2. To create the Grid Infrastructure user, enter a command similar to the following (in this example):

# /usr/sbin/useradd -u 1100 -g oinstall -G asmdba,dba,asmoper oracle

If a separate grid owner is needed:# /usr/sbin/useradd -u 1100 -g oinstall -G asmdba,dba, oracle

# /usr/sbin/useradd -u 1101 -g oinstall -G asmdba,dba,asmoper grid

3. Create base directories for Oracle Grid Infrastructure and Database software.# mkdir -p /u01/app/grid /u01/app/oracle# chown -R grid:oinstall /u01/app# chown oracle:oinstall /u01/app/oracle# chmod 775 /u01/app


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Shell Settings for the Grid Infrastructure User

• Add the following lines to the /etc/security/limits.conf file on each node:

• Add the following to the /etc/pam.d/login file on each node:

grid soft nproc 2047grid hard nproc 16384grid soft nofile 1024grid hard nofile 65536oracle soft nproc 2047oracle hard nproc 16384oracle soft nofile 1024oracle hard nofile 65536

session required pam_limits.so

Shell Settings for the Grid Infrastructure UserPluggable Authentication Modules (PAM) for Linux is a flexible mechanism for authenticating users and provides a series of library modules that many programs such as login and su can use for authentication. The advantage is that newer authentication schemes can be plugged into PAM and made available; you do have to rewrite the programs that depend on authentication. PAM provides for account management, authentication management, password management, and session management. The /etc/security/limits.conf file is the configuration file for the pam_limits module, one of several PAM modules.The recommendation for the Oracle Grid Infrastructure owner is to increase the hard limits for the maximum number of processes (nproc) and the maximum number of open files (nofile)to higher values than the default values. The hard limits are set by the superuser and enforced by the kernel. End-user accounts are not allowed to raise their limits above the hard settings. The soft limits are considered to be the default starting values for normal usage, but end users may alter the soft limits.Note: These requirements also exist for the account, typically named oracle, that will be used for the database software. Consider adding this to /etc/security/limits.conf in addition to the Oracle Grid Infrastructure account owner.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Shell Settings for the Grid Infrastructure User

For the Bourne, Bash, or Korn shell, add the following to the /etc/profile file on each node.

if [ $USER = "oracle" ] || [ $USER = "grid" ]; thenif [ $SHELL = "/bin/ksh" ]; then

ulimit -u 16384ulimit -n 65536

elseulimit -u 16384 -n 65536

fiumask 022

fi

Shell Settings for the Grid Infrastructure User (continued)The Linux ulimit command provides control over the available system resources to the shell and to processes started by the shell. To check the current settings, issue the following command: ulimit –a

The maximum number of processes available to a single user is controlled with the –u option, and the maximum number of open file descriptors is set with the –n option. The exact syntax for setting these options depends on the shell that the user account is running under. Oracle recommends that these values be higher than the default values.Note: These requirements also exist for the account, typically named oracle, that will be used for the database software. Consider adding this to /etc/profile in addition to the Oracle Clusterware account owner.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Quiz

Which of the following statements is not true?1. At least 1.5 GB of physical memory is needed.2. A minimum of 1.5 GB of swap space is required.3. The local /tmp directory should have at least 400 MB free.4. At least 4.5 GB of local storage on each node for the

software is needed.

Answer: 3Statement 3 is not true. The local /tmp directory should have at least 1 GB free.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Module 3: Grid Infrastructure Installation


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Installing Grid Infrastructure./runInstaller

Installing Grid InfrastructureRun OUI by executing the runInstaller command as the grid owner from the installation CD or the staged software directory. Click the “Install and Configure Grid Infrastructure For a Cluster” button and then click Next.Choosing an Installation TypeThe OUI has been enhanced to support the new Oracle Real Application Clusters (RAC) improvements found in Oracle Database 11g Release 2. You will immediately notice that the look and feel of the installer is different from past releases. One of the first improvements you are likely to encounter is the Installation Type page. Here, you may choose a Typical Installation employing a basic fixed IP configuration. You may also choose an Advanced Installation incorporating Grid Plug and Play (GPnP) and offering greater flexibility in configuring:

• Shared storage for OCR and voting files• Network options such as Grid Naming Service (GNS)• Failure isolation support employing Intelligent Platform Management Interface (IPMI)

Because the Typical Installation option assumes a basic configuration, the interview process is shortened.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Grid Plug and Play Support

Grid Plug and Play Support – Advanced InstallationAfter you have chosen product languages from the Select Product Languages page, the Grid Plug and Play Information page is displayed. You must supply a cluster name. This name should be unique within your subnet. If Grid Control is used, the cluster name should be unique within the Grid Control management realm.The single client access name (SCAN) is the address used by clients connecting to the cluster. The SCAN is a domain name registered to three IP addresses, either in the domain name server (DNS) or the Grid Naming Service (GNS). The SCAN addresses need to be on the same subnet as the VIP addresses for nodes in the cluster. The SCAN domain name must be unique within your corporate network. The SCAN port should default to 1521 unless you have a conflict at that port.If you specify a GNS subdomain, the SCAN defaults to clustername-scan.GNSdomain. For example, if you start the Oracle Grid Infrastructure installation from the racnode01 server, the cluster name is cluster01 and the GNS domain is cluster01.example.com, then the SCAN is cluster01-scan.cluster01.example.com.If you do not use GNS, the SCAN should be defined in the DNS to resolve to the addresses assigned to that name. Assign three addresses to the SCAN. Finally, you must provide the address of your GNS. This address must be resolvable through your DNS.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Cluster Node Information

Cluster Node Information – Advanced InstallationOn the Cluster Node Information page, the public host names and virtual host names are defined. In the past, the host and VIP names and addresses were defined in the DNS or locally in a hosts file. If you want to configure your cluster in this manner, make sure that the Configure GNS check box is unselected on the previous screen. Click the Add button and enter the Public Node Name and Virtual Host Name for each node in the cluster.If GNS has been selected on the previous screen and DHCP is configured in the subdomain in which the cluster resides, then configuration is simplified. Click the Add button and then add the host name as shown in the graphic in the slide. There is no need to provide a Virtual IP name for each node because Virtual IP names are automatically configured by Clusterwareusing DHCP.Secure Shell (SSH) can be configured for the specified nodes by clicking the SSH Connectivity button. You will be required to provide the software owners password common to all nodes. When SSH connectivity has been established, click Next to continue.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Specify Network Interface Usage

Specify Network Interface Usage – Advanced InstallationOn the Specify Network Interface Usage page, you can select the network interfaces on your cluster nodes to use for internode communication. You must choose one interface for the public network and one for the private network. Ensure that the network interfaces that you choose for the interconnect have enough bandwidth to support the cluster and RAC-related network traffic. A gigabit Ethernet interface is highly recommended for the private interconnect. To configure the interface for private use, click the interface type and choose the proper usage for each network interface. In the example shown in the slide, there are three interfaces: eth0, eth1, and eth2. The eth0 interface is the hosts’ primary network interface and should be marked Public. The eth1 interface is configured for the private interconnect and should be marked Private. The eth2 interface is dedicated for the storage network, which supports this cluster’s shared storage. It should be marked Do Not Use. When you finish, click the Next button to continue.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Storage Option Information

Storage Option Information – Advanced InstallationOn the Storage Option Information page, you can select your preferred method for storingClusterware files. The first option is Automatic Storage Management (ASM). The ability to use ASM disk groups for Clusterware OCR and voting disks is a new feature in Oracle Database 11g Release 2 Grid Infrastructure. If you choose this option and ASM is not yet configured, OUI launches the ASM configuration assistant to configure ASM and a disk group to support the chosen configuration as illustrated in the slide. When using ASM, the redundancy for the OCR and voting files is tied to the redundancy that you define for the disk group and the number of disks you put in the disk group. The voting disk is specifically allocated to three disks in the disk group under normal redundancy and five disks under high redundancy. By default, each disk in the disk group is put in its own failure group and the voting disks are put into different failure groups. The OCR is stored similar to the way database files are stored. The other storage option available is shared file system storage. Suitable file systems include supported cluster file systems such as OCFS2 or Network File System (NFS). If you select Shared File System on the Storage Option Information page, you will be prompted to provide a cluster file system or NFS directory to be used for the OCR and voting disk files as shown in the slide.The next page displayed is the Specify ASM Password page. The new ASM instance requires its own SYS user with SYSASM privileges. Specify a secure password here and click Next to continue.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Specify Cluster Configuration – Typical InstallationIf you choose not to implement GNS, you can choose Typical Installation from the Select Installation Type page. On the Specify Cluster Configuration page, you must enter the SCAN. The SCAN should be unique in your network. In a non-GNS environment, the SCAN should be defined in the DNS or locally in the /etc/hosts file. If the SCAN is defined locally, you are limited to one SCAN. If it is defined in DNS, you can have as many SCANs as you deem necessary, although three SCANs are recommended .Click the Add button to add cluster node information for the remaining nodes that will be included in this installation. You must provide the hostname and VIP name for the additional nodes. Both of these names must be resolvable, either locally from the /etc/hosts file or from your DNS.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Specify Cluster Configuration – Typical InstallationSecure Shell (SSH) can be configured for the software owner on the specified nodes by clicking the SSH Connectivity button. You will be required to provide the software owners password and it should be identical for all nodes. Click the Setup button to initiate the configuration. If SSH has already been configured for the software owner, you could click the Test button to check existing configuration.Click the Specify Network Interfaces button and select the network interfaces on your cluster nodes to use for internode communication. You must choose one interface for the public network and one for the private network. Ensure that the network interfaces that you choose for the interconnect have enough bandwidth to support the cluster and RAC-related network traffic. For each adapter shown, click the interface type and choose the proper usage for each one. In the example shown in the slide, there are four interfaces: eth0, eth1, eth2, and virbr0. The eth2 interface is the hosts’ primary network interface and should be marked Public. The eth1 interface is configured for the private interconnect and should be marked Private. The eth0 interface is dedicated for the storage network, which supports this cluster’s shared storage. It should be marked Do Not Use. The virbr0 is a virtual adapter created to support virtual machines and should also be marked Do Not Use. When finished, click OK and then the Next button to continue.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Install Locations– Typical InstallationOn the Specify Install Locations you must enter a base location for installing Oracle software. A separate Oracle base should be created for each operating system user owning an Oracle software installation. In the example in the slide, that location is /u01/app/grid.In the Software Location field, enter a directory in which to install Oracle Grid Infrastructure. In the example, the grid operating system user will own this installation, so the software location should be /u01/app/11.2.0/grid. Next, you must indicate where the cluster registry and voting disks will be stored. In the example above, Oracle Automatic Storage Management (ASM) is chosen. If you choose ASM, you must also assign the SYSASM password and the OSASM group. If you choose a cluster file system instead of ASM for the Cluster Registry Storage type, you must provide a valid path to that shared storage. When finished click Next to continue. If selected ASM for the Cluster Registry Storage type, you will be prompted to configure an ASM disk group. Click the Change Discovery Path button and provide a path to all disks intended for ASM storage if necessary. Choose a name and desired Redundancy for the disk group. Select the disks to be used for the disk group and click Next to continue.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Failure Isolation Support with IPMI

Failure Isolation Support with IPMI – Advanced InstallationIntelligent Platform Management Interface (IPMI) provides a set of common interfaces to computer hardware and firmware that administrators can use to monitor system health and manage the system. With Oracle Database 11g Release 2, Oracle Clusterware can integrate IPMI to provide failure isolation support and to ensure cluster integrity.You must have the following hardware and software configured to enable cluster nodes to be managed with IPMI:

• Each cluster member node requires a Baseboard Management Controller (BMC) running firmware compatible with IPMI version 1.5, which supports IPMI over local area networks (LANs) and configured for remote control.

• Each cluster member node requires an IPMI driver installed on each node.• The cluster requires a management network for IPMI. This can be a shared network, but

Oracle recommends that you configure a dedicated network.• Each cluster node’s Ethernet port used by BMC must be connected to the IPMI

management network.If you intend to use IPMI, you must provide an administration account username and password to provide when prompted during installation.Note: For Oracle Clusterware to communicate with BMC, the IPMI driver must be installed permanently on each node, so that it is available on system restarts. The IPMI driver is available on the Asianux Linux, Oracle Enterprise Linux, Red Hat Enterprise Linux, and SUSE Enterprise Linux distributions supported with this release.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Privileged Operating System Groups

Privileged Operating System Groups – Advanced InstallationOn the Privileged Operating System Groups page, you must specify the groups that the Grid Infrastructure owner should belong to for proper operating system authentication to ASM. The example in the slide specifies asmdba for the ASM Database Administrator (OSDBA) group,asmoper for the ASM Instance Administration Operator (OSOPER) group, and asmadminfor the ASM Instance Administrator (OSASM) group. Click Next to continue.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Installation and Inventory Locations

Installation and Inventory LocationsNext, OUI displays the Specify Installation Location page. Enter a base location for installing Oracle software. A separate Oracle base should be created for each operating system user owning an Oracle software installation. In the example in the slide, that location is /u01/app/grid.In the Software Location field, enter a directory in which to install Oracle Grid Infrastructure. In the example, the grid operating system user will own this installation, so the software location should be /u01/app/11.2.0/grid. When you have entered proper Oracle Base and Software Location values, click Next to continue.On the Create Inventory page, enter a location under your Oracle Base to store the Oracle Inventory. Click Next to continue.Note: The Create Inventory page is common to both Advanced and Typical installation types as are the remaining installation steps.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Prerequisite Checks

Prerequisite ChecksWith the introduction of Oracle Database 11g Release 2, the OUI provides additional guidance to ensure recommended deployment and to prevent configuration issues. In addition, configuration assistants validate configurations and provide scripts to fix issues that you can choose to use or reject. Clicking the Fix and Check Again button brings up a window instructing you to run a fixup script that must be run as root on all nodes before continuing.The example in the slide shows the Oracle Universal Installer response to the kernel parameter WMEM_MAX, currently set too low for a successful installation. The installer can determine whether the deficiency can be corrected, presenting the user performing the installation with the option of allowing the OUI to correct the situation. By clicking the Fix & Check Again button, a script is generated on the nodes where the deficient condition exists. After executing the scripts as root on the nodes, the kernel parameter is adjusted, allowing the installation to continue uninterrupted. When the prerequisite checks have completed successfully, click Next to continue. It is possible that an installation deficiency cannot be corrected with a generatedfixup script. The installer will indicate this with a “No” in the Fixable column for that item. When the Summary page appears, check the installation information displayed there. If the information is correct, click the Finish button to begin the software setup.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Finishing the Installation

Finishing the InstallationAfter the Grid Infrastructure has been copied to the remote nodes, you will be prompted to run root scripts on all installed nodes. If this is the first Oracle software initiated by the operating system user, you will run the orainstRoot.sh scripts on all nodes as prompted by the installer. Next, you must execute the root.sh script on all nodes. When the root.sh script finishes executing on the last node, click Finish to exit the installer.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Verifying the Grid Infrastructure Installation[root@racnode01 ~]# /u01/app/11.2.0/grid/bin/crsctl stat res -t--------------------------------------------------------------------------------NAME TARGET STATE SERVER STATE_DETAILS--------------------------------------------------------------------------------Local Resources--------------------------------------------------------------------------------ora.DATA.dg

ONLINE ONLINE racnode01ONLINE ONLINE racnode02

ora.LISTENER.lsnrONLINE ONLINE racnode01ONLINE ONLINE racnode02

ora.asmONLINE ONLINE racnode01 StartedONLINE ONLINE racnode02 Started

ora.eonsONLINE ONLINE racnode01ONLINE ONLINE racnode02

...ora.registry.acfs

ONLINE ONLINE racnode01ONLINE ONLINE racnode02

--------------------------------------------------------------------------------Cluster Resources--------------------------------------------------------------------------------ora.LISTENER_SCAN1.lsnr

1 ONLINE ONLINE racnode02ora.LISTENER_SCAN2.lsnr

1 ONLINE ONLINE racnode01ora.LISTENER_SCAN3.lsnr

1 ONLINE ONLINE racnode01ora.gns

1 ONLINE ONLINE racnode01ora.gns.vip

1 ONLINE ONLINE racnode01ora.oc4j

1 OFFLINE ONLINE racnode02ora.racnode01.vip

1 ONLINE ONLINE racnode01 ...

Verifying the Grid Infrastructure InstallationExecute the crsctl command as shown in the slide to confirm that all cluster resources are up and running. In addition, you should confirm that your DNS is properly forwarding address requests for your application and SCAN VIPs to your GNS and that they are resolved properly. You can do this with dig. Execute the dig command with DNS and VIP addresses as shown:# dig @myDNS.example.com racnode01-cluster01.example.com...;; QUESTION SECTION:;racnode01-vip.cluster01.example.com. IN A

;; ANSWER SECTION:racnode01-vip.cluster01.example.com. 120 IN A 192.0.2.103...# dig @myDNS.example.com cluster01-scan.cluster01.example.com...;; QUESTION SECTION:;cluster01-scan.cluster01.example.com. IN A

;; ANSWER SECTION:cluster01-scan.cluster01.example.com. 120 IN A 192.0.2.248cluster01-scan.cluster01.example.com. 120 IN A 192.0.2.253cluster01-scan.cluster01.example.com. 120 IN A 192.0.2.254


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Module 4: Configuring ASM Disk Groups and ACFS


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Configuring ASM with ASMCA

Configuring ASM with ASMCAThe ASM Configuration Assistant (ASMCA) such as ASMCMD, and SQL*Plus, is located in the Grid Infrastructure home directory. All these utilities require that the ORACLE_SIDenvironment variable be set. Using the oraenv script, you can set the ORACLE_SID, ORACLE_HOME, and PATH variables.An example assuming that you are connected as the grid software owner on the first node of your cluster:

$ . oraenvORACLE_SID = [+ASM1] ? +ASM1The Oracle base for ORACLE_HOME=/u01/app/11.2.0/grid is /u01/app/grid

Start the ASM Configuration Assistant (ASMCA) utility with: $ asmca

ASMCA provides a complete set of commands to manage the ASM instances, disk groups, volumes, and ASM Cluster File Systems. The operations needed to manage ASM can also be performed in SQL*Plus, ASMCMD, and Enterprise Manager.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Creating an ASM Disk Group with ASMCA

Creating an ASM Disk Group with ASMCAIn the slide, the ASMCA utility is being used to create a disk group. Redundancy is set to Normal. Four disks are selected to be members of this disk group: ORCL:DATA5, ORCL:DATA6, ORCL:DATA7, and ORCL:DATA8. The first two disks are placed in the failure group ACFS1, and the second disks are in the failure group ACFS2. The header status can show multiple valid values. When this is the first time a disk has been created, the header status will be CANDIDATE. If the disk has been a member of the disk group and has been cleanly dropped, the header status will be FORMER as shown in the slide. The PROVISIONED header status is similar to CANDIDATE except that PROVISIONED implies that an additional platform-specific action has been taken by an administrator to make the disk available for ASM.The disk discovery path is set by an initialization parameter, ASM_DISKSTRING. This parameter can consist of multiple search paths separated by commas. The default ASM_DISKSTRING parameter string is empty. There is an appropriate default discovery path for most OS platforms. A disk discovery string that limits the directories that are searched can reduce the time for discovery, but the disk discovery string must include all disks that are members of existing disk groups. The default string will allow ASM to find disks that have been initialized with the Oracle ASMLib utility, oracleasm. Advanced options for disk groups are shown later in this lesson.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Creating an ASM Disk Group:Advanced Options

Creating an ASM Disk Group: Advanced OptionsAt the bottom of the Create Disk Group page, click Show Advanced Options to see the portion of the page shown in the slide. On this portion of the page, you can set disk group attributes: allocation unit size and compatibility parameters. The field labeled ASM Compatibility sets the COMPATIBLE.ASM attribute. For Oracle ASM in Oracle Database 11g, 10.1 is the default setting for the COMPATIBLE_ASM attribute when using the SQL CREATE DISKGROUP statement, the ASMCMD mkdg command, and the Oracle Enterprise Manager Create Disk Group page. When creating a disk group with ASMCA, the default setting is 11.2.0. When setting the values for the compatibility attributes, specify at least the major and minor versions of a valid Oracle Database release number. For example, you can specify compatibility as “11.1” or “11.2”; Oracle assumes that any missing version number digits are zeros. The Database Compatibility sets the minimum version level for any database instance that is allowed to use the disk group. This is the COMPATIBLE.RDBMS attribute.To use ADVM volumes, ADVM Compatibility must be set to 11.2.0 or higher and the ASM Compatibility must be 11.2.0 or higher. The ADVM Compatibility field sets the COMPATIBLE.ADVM attribute. Note: Advancing the values for disk group compatibility attributes is an irreversible operation.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Creating a Disk Group with Enterprise Manager

Creating a Disk Group with Enterprise ManagerThe same functionality is available with Enterprise Manager. This slide shows the Disk Group page of the Automatic Storage Management target. Note: Enterprise Manager is not part of the Grid Infrastructure installation. As you will see in the practice for this lesson, you must install a database instance to have access to Enterprise Manager Database Control that is shown in the slide.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Creating a Disk Group with Enterprise Manager

Creating a Disk Group with Enterprise Manager (continued)Just as with the ASMCA utility, you can specify the disks and failure groups on the Create Disk Group page in Enterprise Manager. To change the disk discovery string, click the Configuration tab. The Compatibility Options are available on the Create Disk Group page, but that section is not shown in the slide.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Creating an ASM Disk Group with Command-Line Tools

Create a disk group with SQL:CREATE DISKGROUP ACFS NORMAL REDUNDANCY

FAILGROUP ACFS2

DISK 'ORCL:DATA7' SIZE 1953 M ,'ORCL:DATA8' SIZE 2329 M

FAILGROUP ACFS1

DISK 'ORCL:DATA5' SIZE 1953 M ,'ORCL:DATA6' SIZE 1953 M

ATTRIBUTE 'compatible.rdbms' = '11.2.0.0.0',

'compatible.asm' = '11.2.0.0.0',

'compatible.advm' = '11.2.0.0.0' ;

Create a disk group with ASMCMD:asmcmd mkdg disk_config.xml

Creating an ASM Disk Group with Command-Line ToolsYou can create an ASM disk group with command-line tools. The SQL statement shown in the slide performs the same operation that was shown in the previous slides using ASMCA and Enterprise Manager. The asmcmd command shown in the slide performs the same operation if the disk_config.xml file is:<dg name="data" redundancy="normal">

<fg name="fg1">

<dsk string="/dev/disk1"/>

<dsk string="/dev/disk2"/> </fg>

<fg name="fg2">

<dsk string="/dev/disk3"/>

<dsk string="/dev/disk4"/> </fg>

<a name="compatible.asm" value="11.2"/>

<a name="compatible.rdbms" value="11.2"/>

<a name="compatible.advm" value="11.2"/>

</dg>


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Configuring an ASM Volume: ASMCA

Configuring an ASM Volume: ASMCATo create an ASM volume with ASMCA, click the Volumes tab and then click Create. On the Create Volume page, enter the Volume Name, select the Disk Group Name, and enter the Size. These are the basic parameters that must be supplied. The Advanced attributes are Redundancy and Striping. The redundancy is applied at the ASM file level in ASM—in this case, the volume. “Unprotected” means that there is only one copy of the volume file extent in the disk group. Normal specifies that there will be two copies of each volume file extent in separate failure groups, so for any single disk failure there will still be a surviving copy. High Redundancy specifies three copies, so that the file can survive two disk failures. Striping indicates how data chunks are allocated inside the volume. Volumes are allowed to have more granularity in specifying the stripes than do the ASM files. Stripe Columns specify how many volume file extents are allocated for the stripe and Stripe Width specifies how many bytes are allocated in each file extent. The file extents are placed round-robin on the disks in the disk group. Then the data chunks of the stripe width size are allocated round-robin in each file extent. For the example in the slide, there will be four file extents and the stripes will fill the extents in 128K increments. All the file extents will get one data chunk before any file extent gets a second data chunk. This spreads the I/Os across all the disks in the disk group to reduce latency.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Configuring an ASM Volume: EM

Configuring an ASM Volume: EMFrom the ASM instance home page, navigate to the Disk Group page for the disk group where you want to create the volume.

1. On the ASM instance home page, click the Disk Groups tab. 2. On the Disk Groups page, click the disk group name. 3. On the Disk Group: <disk group name> page, click the Volumes tab. 4. Click Create.5. On the Create ASM Volume page, enter the Volume, Size, and Redundancy. Optionally,

select the disk regions for the primary and mirror extents.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Configuring an ASM Volume:ASMCMD and SQL

Create a volume with SQL:ALTER DISKGROUP ACFS ADD VOLUME TESTVOL

SIZE 500M UNPROTECTED

Create a volume using ASMCMD:volcreate -G ACFS -s 500M --redundancy unprotected

Configuring an ASM Volume: ASMCMD and SQLYou can create an ASM volume with the same properties using either SQL or ASMCMD.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Configuring the ASM Cluster File System

Configuring the ASM Cluster File SystemUsing ASMCA, you can format the ASM volume with an ASM cluster file system. On the Create ASM Cluster File System page, you can specify the mount point for a general purpose file system and, optionally, register the mount point. Registration causes the volume to be a managed resource, which will be mounted when the server is started. For an ACFS volume containing an Oracle database home directory, registration is not optional. The database home must be registered as a managed resource to maintain the dependencies between the database, ASM, and ACFS. Dependency means when the database instance is commanded to start using srvctl, ASM will be started, the ACFS volume will be mounted, and then the database will be started. The database cannot be started until the directory containing its executables (ORACLE_HOME) is mounted.Note: ACFS cannot be used for a root file system, boot file system, or a file system containing the executables for Grid Infrastructure. ACFS cannot be used for any files that can be stored in ASM.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Configuring ACFS with EM

Configuring ACFS with EMThis slide shows that you can configure ACFS with the same options using EM as you can with ASMCA


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Configuring ACFS with Command Line

Linux/UNIX• Format the volume with ACFS:/sbin/mkfs -t acfs /dev/asm/testvol-461 "testvol"

• Register the mount point:/sbin/acfsutil registry -a -f /dev/asm/testvol-461

/u01/app/grid/acfsmounts/acfs_testvol

Windows• Format the volume with ACFS: acfsformat asm-volume1-123

• Register the mount point:acfsutil registry -f -a asm-volume1-123 G:/mnt

Configuring ACFS with Command LineOn Linux or UNIX, the Grid Infrastructure software owner can format the volume. The example in the slide places an ACFS file system on the volume /dev/asm/testvol-461and give it a volume label of testvol. The grid software owner can also register this volume and a particular mount point with the Cluster configuration so the volume will be mounted on restart.On Windows, use the command shown to place an ACFS file system on volume asm-volume-123. The Windows command to register the volume has the same syntax as the Linux/UNIX command, with the differences in device and directory naming shown in the slide.Windows has a separate set of commands mainly for use with ACFS file systems and ADVM volumes. On Linux and UNIX, you use the OS commands such as mkfs and fsck, whereas on Windows, you use acfsformat and acfschkdsk. See the Oracle Database Storage Administrator’s Guide: Oracle ACFS Command-Line Tools for the syntax of the ACFS command-line tools for both environments.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Mounting ACFS with ASMCA

Mounting ACFS with ASMCAThe slide shows the steps to mount an ACFS file system from ASMCA.

1. Invoke ASMCA and click the ASM Cluster File Systems tab.2. Right-click the volume that you want to mount and select the Show Mount Command.3. On the Mount ACFS Command page, verify the mount point and click Generate

Command.4. On the Mount ACFS Command page, copy the command into a terminal window on

each node and execute the command as a superuser (root).


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Mounting an ACFS with EM

Mounting an ACFS with EMThe slide shows the steps to mount an ACFS from Enterprise Manager.

1. From the ACFS tab of the ASM instance page, select the volume and click Mount. 2. On the Cluster Select Hosts page, select the hosts where you want to mount the file

system and then click Continue. 3. On the Mount ASM Cluster File System page, verify the mount point and click Generate

Command. 4. On the Show Command page, copy the command into a terminal window on each node

and execute the command as a superuser (root).


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Mounting ACFS with Command-Line Tools

Windows C:> acfsmountvol G:\mnt asm-volume1-123

Linux/UNIX# /bin/mount -t acfs /dev/asm/testvol-461

/u01/app/grid/acfsdata/test

Mounting ACFS with Command-Line ToolsWhether you use Enterprise Manager or ASMCA, the commands are executed by a superuser. The superuser is root on UNIX or Linux and administrator on Windows. The command-line tools are the same commands generated by Enterprise Manager or ASMCA.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Quiz

You must configure GNS to take advantage of automatic VIP configuration.1. True2. False

Answer: 1This statement is true.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Summary

In this lesson, you should have learned how to:• Perform preinstallation tasks for Grid Infrastructure• Install Grid Infrastructure• Verify the installation• Configure ASM disk groups• Configure ASM volumes• Make the ASM cluster file system• Mount ACFS volumes


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Practice 2 Overview

This practice covers the following topics:• Perform Grid Infrastructure preinstallation tasks• Install Oracle Grid Infrastructure• Create an ASM disk group• Create and mount an ASM volume


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Administering Oracle Clusterware

Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Objectives

After completing this lesson, you should be able to:• Display Clusterware management proficiency• Demonstrate Oracle Cluster Registry (OCR) backup and

recovery techniques• Manage network settings


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Managing Oracle Clusterware

• Command-line utilities– crsctl manages clusterware-related operations:

— Starting and stopping Oracle Clusterware— Enabling and disabling Oracle Clusterware daemons— Registering cluster resources

– srvctl manages Oracle resource–related operations:— Starting and stopping database instances and services

• Enterprise Manager– Browser-based graphical user interface– Enterprise Manager cluster management available in:

— Database control—within the cluster— Grid control—through a centralized management server

Managing Oracle ClusterwareOngoing management of Oracle Clusterware is achieved by using the crsctl and srvctlcommand-line utilities installed under the Oracle Grid Infrastructure home directory.Oracle Clusterware components and resources can be monitored and managed from any node in the cluster by using crsctl. The srvctl utility provides similar monitoring and management capabilities for Oracle-related resources such as database instances and database services. Both utilities are provided with Oracle Clusterware. However, most crsctlcommands are available only to clusterware administrators, whereas srvctl commands are available to other groups such as database administrators.Enterprise Manager is Oracle’s browser-based graphical management interface. Enterprise Manager can be used to manage Oracle Clusterware in two ways. Enterprise Manager Database Control can be used within a cluster to manage the functions of that cluster. Enterprise Manager Database Control requires the installation and configuration of an Oracle Database on the cluster before it can be used. Enterprise Manager Grid Control uses a centralized management server to enable the management of many clusters from a unified console.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Managing Clusterware with Enterprise Manager

Example: Relocate a resource.

Managing Clusterware with Enterprise ManagerEnterprise Manager Database Control provides facilities to manage Oracle Clusterware. This includes the ability to register and manage resources. The example in this slide provides a typical illustration of the management interface. It shows how resources can be dynamically relocated from one node to another within your cluster. In this case, my_resource is relocated from host02 to host01.You can see on the Confirmation page the command that will be run to relocate the resource:

crsctl relocate resource my_resource –n host01 –s host02


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Controlling Oracle Clusterware

The crsctl utility can be used to control Oracle Clusterware.• To start or stop Oracle Clusterware on a specific node:

• To enable or disable Oracle Clusterware on a specific node:

# crsctl start crs

# crsctl stop crs

# crsctl enable crs

# crsctl disable crs

Controlling Oracle ClusterwareWhen a node that contains Oracle Clusterware is started, the Oracle Clusterware wrapper script is automatically started by the /etc/init.d/ohasd startup script. When the crsctl utility is used to disable Cluster Ready Services (CRS) from automatically starting, state information related to startup is placed in the SLCS_SRC control files, preventing automatic startup on machine reboot. To check the status of CRS, use the following syntax:# crsctl check crsCRS-4638: Oracle High Availability Services is onlineCRS-4537: Cluster Ready Services is onlineCRS-4529: Cluster Synchronization Services is onlineCRS-4533: Event Manager is online

You may have to manually control the Oracle Clusterware stack while applying patches or during planned outages. You can stop Oracle Clusterware by using the crsctl stop crscommand and start it by using the crsctl start crs command.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Verifying the Status of Oracle Clusterware

The crsctl utility can be used to verify the status of Oracle Clusterware.• To determine the overall health on a specific node:

• To check the viability of Cluster Synchronization Services (CSS) across nodes:

$ crsctl check crsCRS-4638: Oracle High Availability Services is onlineCRS-4537: Cluster Ready Services is onlineCRS-4529: Cluster Synchronization Services is onlineCRS-4533: Event Manager is online

$ crsctl check clusterCRS-4537: Cluster Ready Services is onlineCRS-4529: Cluster Synchronization Services is onlineCRS-4533: Event Manager is online

Verifying the Status of Oracle ClusterwareThe crsctl utility can be used to verify the status of Oracle Clusterware on specific nodes and across nodes. In contrast to the crsctl controlling commands that required the rootaccess (shown in the previous slide), the check commands do not require root and may be executed by the Oracle Clusterware software owner. The overall health of the clusterware on a specific node can be obtained by using the crsctl check crs command. It is possible to target three of the individual daemons by using the crsctl check <daemon> command for the crsd, evmd, and cssd daemons only. These commands are processed only on the node on which they are executed. To check the viability of Cluster Synchronization Services (CSS) across all nodes, use the crsctl check cluster command. The output of the overall health check performed on a specific node is shown in the slide.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Determining the Location of Oracle Clusterware Configuration Files

The two primary configuration file types for Oracle Clusterware are the voting disk and the Oracle Cluster Registry (OCR).• To determine the location of the voting disk:

• To determine the location of the OCR:

# crsctl query css votedisk## STATE File Universal Id File Name Disk group-- ----- ----------------- ---------- ----------1. ONLINE 8c2e45d734c64f8abf9f136990f3daf8 (ASMDISK01) [DATA]2. ONLINE 99bc153df3b84fb4bf071d916089fd4a (ASMDISK02) [DATA]3. ONLINE 0b090b6b19154fc1bf5913bc70340921 (ASMDISK03) [DATA]

Located 3 voting disk(s).

$ cat /etc/oracle/ocr.lococrconfig_loc=+DATAlocal_only=FALSE

Determining the Location of Oracle Clusterware Configuration FilesOracle Clusterware uses two primary configuration file types: the voting disk and the Oracle Cluster Registry (OCR). There can be multiple redundant copies of each. You can determine the location of the voting disk by using the crsctl query css votedisk command on any node. This does not require the CSS daemons to be running, and the command can be executed as the Grid Infrastructure owner. The location of the OCR file can be determined by using the cat /etc/oracle/ocr.loc command. Because these files are always located on shared storage, the command can be executed from any node.Note: The OCR can also be located by using the ocrcheck utility, provided that the path to the utility is known or the path has been added to the PATH environment variable.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Checking the Integrity of Oracle Clusterware Configuration Files

The following techniques are used to validate the integrity of Oracle Cluster configuration files.• Check the ocssd.log for voting disks issues.

• Use the cluvfy utility or the ocrcheck command to check the integrity of the OCR.

$ grep voting <grid_home>/log/<hostname>/cssd/ocssd.log

$ ocrcheck

$ cluvfy comp ocr –n all -verbose

Checking the Integrity of Oracle Clusterware Configuration FilesTo check the integrity of the voting disks, examine ocssd.log. Errors with the voting disks appear in the log. The following is a snippet of the output that indicates what an error may look like:$ grep voting ocssd.log[ CSSD]2008-09-09 10:47:09.711 [100494224] >ERROR:

clssnmvReadFatal: voting device corrupt (0x00000000/0x00000000/1//dev/sda6)

[ CSSD]2008-09-09 10:47:09.711 [3082128272] >ERROR: clssnmvReadFatal: voting device corrupt (0x00000000/0x00000000/2//dev/sda7)

Two commands may be used to check the integrity of the OCR file. They are:$ ocrcheck$ cluvfy comp ocr –n all –verbose


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Backing Up and Recovering the Voting Disk

• In Oracle Clusterware 11g Release 2, voting disk data is backed up automatically in the OCR as part of any configuration change.

• Voting disk data is automatically restored to any added voting disks.

• Using dd to back up and restore a voting disk may result in the loss of the voting disk!

• To add or remove voting disks on non-Automatic Storage Management (ASM) storage, use the following commands:

# crsctl delete css votedisk path_to_voting_disk# crsctl add css votedisk path_to_voting_disk

Backing Up the Voting DiskIn previous releases, backing up the voting disks using a dd command was a required postinstallation task. With Oracle Clusterware 11g Release 2, backing up and restoring a voting disk using the dd command may result in the loss of the voting disk, so this procedure is not supported.Backing up voting disks manually is no longer required because voting disk data is backed up automatically in the OCR as part of any configuration change and voting disk data is automatically restored to any added voting disks.

Recovering Voting DisksIf you have multiple voting disks on non-ASM storage, you can remove the voting disks and add them back into your environment with all the information from the other voting disks using the following commands, where path is the complete path of the location where the voting disk resides:crsctl delete css votedisk path_to_voting_disk

crsctl add css votedisk path_to_voting_disk

Note: You can migrate voting disks from non-ASM storage options to ASM without taking down the cluster. To use an ASM disk group to manage the voting disks, you must set the COMPATIBLE.ASM attribute to 11.2.0.0.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Adding, Deleting, or Migrating Voting Disks

• To add or delete one or more voting disks to non-ASM storage:

• To add a voting disk to ASM:

• To migrate voting disks from non-ASM storage devices to ASM or vice versa, specify the ASM disk group name or path to the non-ASM storage device:

# crsctl add css votedisk path_to_new_voting_disk# crsctl delete css votedisk path_to_old_voting_disk

# crsctl replace votedisk +asm_disk_group

# crsctl replace votedisk {+asm_disk_group | path_to_voting_disk}

Adding, Deleting, or Migrating Voting DisksTo add one or more voting disks to non-ASM storage, run the following command as root:# crsctl add css votedisk path_to_voting_disk [...]

To add a voting disk to ASM:# crsctl replace votedisk +asm_disk_group

To replace voting disk A with voting disk B on non-ASM storage, first add voting disk B and then delete voting disk A:# crsctl add css votedisk path_to_voting_diskB# crsctl delete css votedisk path_to_voting_diskA

Use the crsctl replace votedisk command to replace a voting disk on ASM. You do not have to delete any voting disks from ASM using this command.To remove a voting disk, run the following command as root, replacing the voting_disk_GUID variable with one or more space-delimited, voting disk globally unique identifiers (GUIDs) you want to remove:# crsctl delete css votedisk voting_disk_GUID

To migrate voting disks from non-ASM storage devices to ASM or vice versa, specify the ASM disk group name or path to the non-ASM storage device in the following command:# crsctl replace votedisk {+asm_disk_group | path_to_voting_disk}

You can run this command on any node in the cluster.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Locating the OCR Automatic Backups

• The OCR is backed up automatically. • Only one node performs the backup.• To determine the node and location of the backup:

• Files could be spread across nodes due to outages.• The backup frequency and retention policies are:

– Every four hours: CRS keeps the last three copies.– At the end of every day: CRS keeps the last two copies.– At the end of every week: CRS keeps the last two copies.

$ ocrconfig -showbackup auto

host02 2009/07/28 12:20:42 /u01/app/.../cdata/cluster01/backup00.ocrhost02 2009/07/28 08:20:41 /u01/app/.../cdata/cluster01/backup01.ocrhost02 2009/07/28 04:20:40 /u01/app/.../cdata/cluster01/backup02.ocr

host02 2009/07/27 16:20:37 /u01/app/.../cdata/cluster01/day.ocrhost02 2009/07/28 00:20:39 /u01/app/.../cdata/cluster01/week.ocr

Locating the OCR Automatic BackupsThe information contained in the OCR is much more dynamic in nature than the voting disk. Oracle Clusterware automatically performs routine backups of the OCR file. These are physical backups. Only one node has the responsibility to perform these backups, but that responsibility can transfer to any other node in the cluster when outages occur. The default target location of each automatically generated OCR backup file is the <Grid Home>/cdata/<cluster name> directory. The automatic backup is on a four-hour schedule, but only a limited number of files are retained. Only the last three backups of the routine four-hour intervals are kept, with newer backups overwriting older ones. At the end of the day, a backup is taken and the last two are retained. At the end of the week, a backup is taken and the last two are retained. In conclusion, there should not be more than seven automatic backups that require storage: one four-hours old, one eight-hours old, one 12-hours old, one 24-hours old, one 48-hours old, one seven-days old, and one 14-days old. The four-hour backup interval is not based on the time of the day, but instead on an offset from the time that the clusterware was started.The backup file names cannot be changed and are named as follows: backup00.ocr, backup01.ocr, backup02.ocr, day.ocr, day_.ocr, week.ocr, and week_.ocr.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Changing the Automatic OCR Backup Location

• The automatic backup location should be changed to a location shared by all nodes.

• The backup location will be used for both automatic and manual backups.

• It is recommended that these files be included in routine scheduled backups to an offline location.

• If CRS has been stopped on all nodes, the schedule of backups is suspended.

• On restart, a backup is not immediately taken and the backup timer is reset.

# ocrconfig –backuploc <path to shared CFS or NFS>

Changing the Automatic OCR Backup LocationBecause the automatic backup is performed only by the master node to the local file system by default, it is recommended that you change the OCR automatic backup location to one that is shared by all nodes in the cluster by using the ocrconfig -backuploc <new location> command. This command takes one argument that is the full path of the directory name for the new location. The location will be used for both automatic and manual backups. You cannot customize the backup frequencies, the number of retained copies, or the names of the backup files. If CRS on the master node is shut down, another node becomes the master, and backups will resume on that node. If the backup location has not been changed to a common shared location, backups will exist locally across multiple nodes potentially. If CRS is stopped on all nodes during a scheduled backup, on restart, a backup will not be immediately taken and the backup timer will be reset. This could result in a longer time duration between automatic backups than the standard four-hour interval. Because of the importance of the OCR information, it is also recommended that you manually create copies of automatically generated physical backups. You can use any backup software to copy the automatically generated backup files, and it is recommended that you do that at least once daily to a different device from where the automatic backups are.Note: Do not place your automatic OCR backups on ASM Cluster File System (ACFS)storage.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Adding, Replacing, and Repairing OCR Locations

• Add an OCR location to either ASM or other storage device:

• To replace the current OCR location:

• To repair OCR configuration, run this command on the node on which you have stopped Oracle Clusterware:

You cannot perform this operation on a node on which Oracle Clusterware is running.

# ocrconfig -add +DATA2# ocrconfig -add /dev/sde1

# ocrconfig -replace /dev/sde1 -replacement +DATA2

[root@host03]# ocrconfig -repair -add +DATA1

Adding, Replacing, and Repairing OCR LocationsYou can add an OCR location after an upgrade or after completing an Oracle Grid Infrastructure installation. Oracle Clusterware can manage up to five redundant OCR locations. As root, run the following command to add an OCR location to either ASM or other storage device:# ocrconfig -add +asm_disk_group | file_nameTo replace the current OCR location using either destination_file or +ASM_disk_group to indicate the current and target OCR locations:# ocrconfig -replace destination_file | +ASM_disk_group -replacement destination_file | +ASM_disk_group

It may be necessary to repair an OCR configuration if your configuration changes while a node is stopped. Repairing an OCR configuration involves adding, deleting, or replacing an OCR location. To repair an OCR configuration, run ocrconfig on the node on which you have stopped Oracle Clusterware as root:# ocrconfig -repair -add file_name | -delete file_name | -replace current_file_name -replacement new_file_name

This operation changes the OCR configuration only on the node on which you run this command. For example, if the OCR location is /dev/sde1, use the command syntax ocrconfig -repair -add /dev/sde1 on this node to repair its OCR configuration.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Removing an Oracle Cluster Registry Location

• To remove an OCR location, at least one other OCR must be online.

• Run the following command on any node in the cluster to remove an OCR location from either ASM or another shared location:

• Do not perform an OCR removal unless there is at least one other active OCR location online, or you will get an error.

# ocrconfig -delete +DATA2# ocrconfig -delete /dev/sde1

Removing an Oracle Cluster Registry LocationTo remove an OCR location, at least one other OCR must be online. You can remove an OCR location to reduce OCR-related overhead or to stop mirroring your OCR because you movedOCR to redundant storage such as RAID.Perform the following procedure as the root user to remove an OCR location from your Oracle Clusterware environment:

1. Ensure that at least one OCR location other than the OCR location that you are removing is online.

2. Run the following command on any node in the cluster to remove an OCR location from either ASM or another location:

# ocrconfig -delete +ASM_disk_group | file_name

The file_name variable can be a device name or a file name. This command updates the OCR configuration on all the nodes on which Oracle Clusterware is running.Caution: Do not attempt to perform an OCR removal unless there is at least one other active OCR location online otherwise you will get an error. You cannot remove the last OCR file.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Migrating OCR Locations to ASM

1. Ensure that Oracle Clusterware is upgraded to 11g Release 2.

2. Start ASM on all nodes and create a disk group that has at least 1 GB of space and has at least normal redundancy.

3. To add an OCR location to an ASM disk group, run the following command as root:

4. To remove storage configurations no longer in use, run the following command as root:

# ocrconfig -add +DATA2

# ocrconfig -delete /dev/raw/raw1# ocrconfig -delete /dev/raw/raw2

$ crsctl query crs activeversionOracle Clusterware active version on cluster is [11.2.0.1.0]

Migrating OCR Locations to ASMTo improve Oracle Clusterware storage manageability, OCR is configured, by default, to use ASM in Oracle Database 11g Release 2. With the Oracle Clusterware storage residing in an ASM disk group, you can manage both database and clusterware storage using EM.However, if you upgrade from a previous version of Oracle Database, you can migrate your OCR location or locations to reside on ASM, and take advantage of the improvements in managing Oracle Clusterware storage. To migrate OCR locations to ASM using ocrconfig:

1. Ensure that Oracle Clusterware upgrade to 11g Release 2 is complete. Run the following command to verify the current running version:

$ crsctl query crs activeversion

2. Use ASM Configuration Assistant (ASMCA) to configure and start ASM on all nodes in the cluster, and then create a disk group that has at least 1 GB of space and has at least normal redundancy.

3. To add an OCR location to an ASM disk group, ensure that the Clusterware stack is running and run the following command as root: # ocrconfig -add +new_disk_group

You can run this command more than once if you add more than one OCR location.4. To remove storage configurations no longer in use, run the following command as root:

# ocrconfig -delete old_storage_location

Note: OCR inherits the redundancy of the disk group. If you want high redundancy for OCR, you must configure the disk group with high redundancy when you create it.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Migrating OCR from ASM to Other Shared Storage

1. Ensure that Oracle Clusterware is upgraded to 11g Release 2.

2. Create at least one shared file with the following permissions: root, oinstall, 640 making sure that the mount partition has at least 300 MB of space.

3. To add an OCR location, ensure that the Clusterware stack is running and run the following command as root:

4. To remove storage configurations no longer in use, run the following command as root:

# ocrconfig -add /dev/sde1# ocrconfig -add /dev/sdf1

# ocrconfig -delete +DATA2

$ crsctl query crs activeversionOracle Clusterware active version on cluster is [11.2.0.1.0]

Migrating OCR from ASM to Other Shared Storage To migrate Oracle Clusterware storage from ASM to another storage choice:

1. Ensure that Oracle Clusterware upgrade to 11g Release 2 is complete. Run the following command to verify the current running version: $ crsctl query crs activeversion.

2. Create a shared file with the following permissions: root, oinstall, 640, making sure that the mount partition has at least 300 MB of space.

3. To add the file as an OCR location, ensure that the Oracle Clusterware stack isrunning and run the following command as root:

# ocrconfig -add new_file_location

You can run this command more than once if you add more than one OCR location.4. To remove storage configurations no longer in use, run the following command as root:

# ocrconfig -delete old_storage_location

You can run this command more than once if there are multiple OCR locations configured.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Performing Manual OCR Backups

When significant changes to the configuration have occurred, a manual, on-demand backup is suggested. • To perform a physical backup:

• To display a list of manual backups:

• To perform a logical backup:

# ocrconfig -manualbackup

$ ocrconfig –showbackup manualhost02 2009/07/28 16:59:17 /u01/app/.../cdata/cluster01/backup_20090728_165917.ocr

# ocrconfig -export /home/oracle/ocr.backup

Performing Manual OCR BackupsUnlike the voting disk, the OCR content can be very dynamic in nature, especially with the High Availability framework. If a significant amount of work has been done that would cause modifications to the OCR, it is recommended that a manual backup or export be performed before the routine automatic backup occurs. This on-demand backup can be used to restore the information if the OCR becomes corrupted or lost before the automatic backup occurs.You are not allowed to specify the name used for the manual backup. A file will be created with the name backup_<date>_<time>.ocr and placed into the default backup location. When a manual backup is performed, it does not affect the automatic backup interval. The export command will create a binary file containing a logical backup of the OCR keys and values.Most configuration changes that you make not only change the OCR contents but also cause file and database object creation. Some of these changes are often not restored when you restore the OCR. Do not perform an OCR restore as a correction to revert to previous configurations if some of these configuration changes fail. This may result in an OCR with contents that do not match the state of the rest of your system.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Recovering the OCR by Using Physical Backups

1. Locate a physical backup:

2. Stop the Oracle Clusterware stack on all nodes:

3. Stop Oracle High Availability Services on all nodes:

4. Restore the physical OCR backup:

5. Restart Oracle High Availability Services on all nodes:

6. Check the OCR integrity:

# ocrconfig –restore /u01/app/.../cdata/cluster01/day.ocr

$ ocrconfig –showbackup

# crsctl stop cluster -all

# crsctl stop crs

# crsctl start crs

$ cluvfy comp ocr -n all

Recovering the OCR by Using Physical BackupsUse the following procedure to restore the OCR on UNIX-based systems:

1. Identify the OCR backups by using the ocrconfig -showbackup command. You can execute this command from any node as the oracle user. The output tells you on which node and which path to retrieve both automatically and manually generated backups. Use the auto or manual argument to display only one category.

2. Stop the Oracle Clusterware stack on all nodes by using the crsctl stop cluster -all command.

3. Stop Oracle High Availability Services on all the nodes of your cluster by executing the crsctl stop crs command on all the nodes as the root user.

4. Perform the restore by applying an OCR backup file that you identified in step 1 using the following command as the root user, where file_name is the name of the OCR file that you want to restore: ocrconfig -restore file_name. Ensure that the OCR devices that you specify in the OCR configuration file (/etc/oracle/ocr.loc) exist and that these OCR devices are valid.

5. Restart Oracle High Availability Services on the nodes in your cluster by restarting each node or running the crsctl start crs command as the root user.

6. Check the integrity of the OCR files on all nodes with the cluvfy comp ocr –n all command.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Recovering the OCR by Using Logical Backups

1. Locate a logical backup created using an OCR export.2. Stop Oracle Clusterware on all nodes:

3. Stop Oracle High Availability Services on all nodes:

4. Restore the logical OCR backup:

5. Restart Oracle High Availability Services on all nodes:

6. Check the OCR integrity:

# ocrconfig –import /shared/export/ocrback.dmp

# crsctl stop cluster -all

# crsctl start crs

$ cluvfy comp ocr -n all

# crsctl stop crs

Recovering the OCR by Using Logical BackupsUse the following procedure to import the OCR on UNIX-based systems:

1. Identify the OCR export file that you want to import by identifying the OCR export file that you previously created using the ocrconfig -export file_name command.

2. Stop Oracle Clusterware on all the nodes in your RAC database by executing thecrsctl stop crs command on the nodes as the root user.

3. Perform the import by applying an OCR export file that you identified in step 1 using the following command, where file_name is the name of the OCR file from which you want to import the OCR information:

ocrconfig -import file_name

4. Restart Oracle High Availability Services on all the nodes in your cluster by restarting each node by using the crsctl start crs command.

5. Run the following Cluster Verify Utility (cluvfy) command to verify the OCR integrity, where the -n all argument retrieves a listing of all the cluster nodes that are configured as part of your cluster:

cluvfy comp ocr -n all


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Oracle Local Registry• Each cluster node has a local registry for node-specific

resources, called an Oracle Local Registry (OLR).• The OLR is installed and configured when Oracle

Clusterware is installed.• One of its functions is to facilitate Clusterware startup in

situations where the ASM stores the OCR and voting disks• You can check the status of OLR by using ocrcheck:

$ ocrcheck -localStatus of Oracle Local Registry is as follows :

Version : 3Total space (kbytes) : 262120Used space (kbytes) : 2204Available space(kbytes): 259916ID : 1535380044Device/File Name : /u01/app/11.2.0/grid/cdata/host01.olr

Device/File integrity check succeededLocal registry integrity check succeededLogical corruption check succeeded

Oracle Local RegistryIn Oracle Clusterware 11g Release 2, each node in a cluster has a local registry for node-specific resources, called an Oracle Local Registry (OLR), that is installed and configured when Oracle Clusterware installs OCR. Multiple processes on each node have simultaneous read and write access to the OLR particular to the node on which they reside, regardless of whether Oracle Clusterware is running or fully functional. The OLR provides various Oracle Clusterware processes with access to key configuration information even when Oracle Clusterware is not running on the node. One of its functions is to facilitate the Oracle Clusterware startup process in situations where the ASM stores the OCR and voting disks. During the startup process, the OLR is referenced to determine the exact location of the voting disks. This enables the node to join the cluster. After this initial phase, ASM is started. After ASM is started, processes that require the full OCR can start and the clusterware startup process completes.By default, OLR is located at grid_home/cdata/hostname.olr. You can manage the OLR using ocrcheck, ocrdump, and ocrconfig utilities with the -local option.You can check the status of OLR using the ocrcheck utility, as follows: $ ocrcheck -local


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Oracle Local Registry (continued)You can display the content of OLR to the text terminal that initiated the program using the OCRDUMP utility, as follows:$ ocrdump -local -stdout

You can perform administrative tasks on OLR using the OCRCONFIG utility. To export OLR to a file: $ ocrconfig –local –export file_name

To import a specified file to OLR: $ ocrconfig –local –import file_name

To modify the OLR file on the local node: $ ocrconfig –local –repair olr file_name

The olr keyword used with the -repair option is valid only when -local is used.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Determining the Current Network Settings

• To determine the list of interfaces available to the cluster:

• To determine the public and private interfaces that have been configured:

• To determine the Virtual IP (VIP) host name, VIP address, VIP subnet mask, and VIP interface name:

$ oifcfg iflist –p -n

$ oifcfg getifeth0 192.0.2.0 global publiceth1 192.168.1.0 global cluster_interconnect

$ srvctl config nodeapps -aVIP exists.:host01VIP exists.: /192.0.2.247/192.0.2.247/255.255.255.0/eth0...

Determining the Current Network SettingsTo determine the list of interfaces available to the cluster, use the Oracle Interface Configuration (oifcfg) utility. The oifcfg iflist –p –n command queries the operating system to find out which network interfaces are present on the node. The output lists the network number of each interface, not the IP address along with the netmask if the –noption is used.To determine the public, private, and storage interfaces that have been configured for Oracle Clusterware, use the oifcfg getif command. Virtual IP (VIP) addresses should be associated only with public interfaces. To determine the VIP host name, VIP address, VIP subnet mask, and VIP interface name, use the srvctl config nodeapps –a command.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Changing the Public VIP Addresses

1. Stop all services running on the node whose VIP address you want to change:

2. Confirm the current IP address for the VIP address:

3. Stop the VIP address:

4. Verify that the VIP address is no longer running by running the ifconfig -a command.

$ srvctl stop service -d RDBA -s sales,oltp -n host01

$ srvctl config vip -n host01VIP exists.:host01VIP exists.: /host01-vip/192.168.2.20/255.255.255.0/eth0

$ srvctl stop vip -n host01

Changing the Public VIP AddressesClients configured to use Public VIP addresses for Oracle Database releases before Oracle Database 11g Release 2 can continue to use their existing connection addresses. It is recommended that you configure clients to use single client access names (SCANs), but it is not required. When an earlier version of Oracle Database is upgraded, it is registered with the SCAN, and clients can start using the SCAN to connect to that database, or continue to use VIP addresses for connections.If you continue to use VIP addresses for client connections, you can modify the VIP address while Oracle Database and Oracle ASM continue to run. However, you must stop services while you modify the address. When you restart the VIP address, services are also restarted on the node. Perform the following steps to change a VIP address:

1. Stop all services running on the node whose VIP address you want to change: $ srvctl stop service -d db_name -s service_name_list -n my_node

2. Confirm the current IP address for the VIP address using the srvctl config vipcommand:$ srvctl config vip -n my_node

3. Stop the VIP address using the srvctl stop vip command:$ srvctl stop vip -n mynode

4. Verify that the VIP address is no longer running using the ifconfig -a command.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Changing the Public VIP Addresses

5. Make necessary changes to the /etc/hosts file on all nodes and make necessary domain name server (DNS) changes to associate the new IP address with the old host name.

6. Modify node applications and provide a new VIP address:

7. Start the node VIP.

8. Repeat the steps for each node in the cluster.9. Run cluvfy to verify node connectivity between all the

nodes for which your cluster is configured:

# srvctl start vip -n host01

# srvctl modify nodeapps -n host01 -A \192.168.2.125/255.255.255.0/eth0

$ cluvfy comp nodecon -n all -verbose

Changing the Public VIP Addresses (continued)5. Make any changes necessary to the /etc/hosts files on all nodes and make any

necessary DNS changes to associate the new IP address with the old host name.6. Modify the node applications and provide a new VIP address using the following syntax:

# srvctl modify nodeapps -n node_name -A new_vip_address7. Start the node VIP by running the srvctl start vip command:

$ srvctl start vip -n mynode

8. Repeat the steps for each node in the cluster.Because the srvctl utility is a clusterwide management tool, you can accomplish these tasks for any specific node from any node in the cluster, without logging in to each of the cluster nodes.

9. Run the following command to verify node connectivity between all the nodes for which your cluster is configured. This command discovers all the network interfaces available on the cluster nodes and verifies the connectivity between all the nodes by way of the discovered interfaces. This command also lists all the interfaces available on the nodes, which are suitable for use as VIP addresses.$ cluvfy comp nodecon -n all -verbose


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Changing the Interconnect Adapter

• On a single node in the cluster, add the new global interface specification:

• Verify the changes with oifcfg getif and then stop Clusterware on all nodes by running the following command as root on each node:

• Assign the network address to the new network adapters on all nodes using ifconfig:

• Remove the former adapter/subnet specification and restart Clusterware:

$ oifcfg setif -global eth2/192.0.2.0:cluster_interconnect

# oifcfg getif# crsctl stop crs

# ifconfig eth2 192.0.2.15 netmask 255.255.255.0 \broadcast 192.0.2.255

$ oifcfg delif -global eth1/192.168.1.0# crsctl start crs

Changing the Interconnect Adapter To change the network interface for the private interconnect (for example, eth1), you must perform the change on all nodes (globally). This is because Oracle currently does not support the use of different network interface cards in the same subnet for the cluster interconnect.To change the network interface, perform the following steps:

1. Make sure that the Oracle Clusterware stack is up and running on all cluster nodes.2. Use operating system commands (ifconfig or the command for your system) to ensure

that the new or replacement interface is configured and up on all cluster member nodes.3. On a single node in the cluster, add the new global interface specification:

$ oifcfg setif -global interface_name/subnet:cluster_interconnect4. On a node in the cluster, use ifconfig to ensure that the new IP address exists.5. Add the new subnet, with the following command, providing the name of the interface

and the subnet address. The changes take effect when Oracle Clusterware restarts:$ oifcfg setif -global interface_name/subnet:cluster_interconnect

6. Verify the configuration with the oifcfg getif command.7. Stop Oracle Clusterware by running the following command as root on each node:

# crsctl stop crs


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Changing the Interconnect Adapter (continued)8. Assign the current network address to the new network adapter by using ifconfig.

As the root user, issue the ifconfig operating system command to assign the currently used private network address to the network adapter intended to be used for the interconnect. This usually requires some down time for the current interface and the new interface. See your platform-specific operating system documentation for more information about issuing the ifconfig command.You must update the operating system configuration changes because changes made using ifconfig are not persistent.

10. Remove the former subnet, as follows, providing the name and subnet address of the former interface: oifcfg delif -global interface_name/subnetFor example: $ oifcfg delif -global eth1/10.10.0.0Note: This step should be performed only after a replacement interface is committed into the Grid Plug and Play configuration. Simple deletion of cluster interfaces without providing a valid replacement can result in invalid cluster configuration.

11. Restart Oracle Clusterware by issuing the following command as the root user on all nodes:# crsctl start crsYou must restart Oracle Clusterware after running the oifcfg delif command because Oracle Clusterware, Oracle ASM, and Oracle RAC continue to use the former subnet until they are restarted.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Managing SCAN VIP and SCAN Listener Resources

• To add a SCAN VIP resource:

• To add a SCAN listener resource:

$ srvctl add scan_listener $ srvctl add scan_listener -p 65536 ## using nondefault port number ##

$ srvctl add scan -n cluster01-scan

• To remove Clusterware resources from SCAN VIPs:$ srvctl remove scan [-f]

$ srvctl remove scan_listener [-f]

• To remove Clusterware resources from all SCAN listeners:

Managing SCAN VIP and SCAN Listener ResourcesThe srvctl add scan command is used to add Oracle Clusterware resources for a givenSCAN name. The command creates the same number of SCAN VIP resources as the number of IP addresses that SCAN resolves to, or three when network_number identifies a DHCP network and Oracle GNS configuration. Use the srvctl add scan command with the following syntax:srvctl add scan -n scan_name [-k network_number [-S subnet/ netmask[/if1[|if2|...]] where –n is the domain name–qualified SCAN name and -kis the optional network number where the SCAN IP addresses come from. Otherwise, the SCAN VIPs come from the same network as the standard VIPs. If a network number does not exist, the -S <subnet>/<netmask>/[|if1] clause creates the network number. The –Sclause must be used when <network_number> does not exist.To add the SCAN name new-scan.cluster01.example.com, run the following command:# srvctl add scan -n new-scan.cluster01.example.com

The srvctl add scan_listener command can be used to add resources to the SCAN listeners. The number of SCAN listener resources created is the same as that for SCAN VIP resources. Use the srvctl add scan_listener command with the following syntax:srvctl add scan_listener [-l lsnr_name_prefix] [-p scan_port]where –l is the SCAN listener name prefix and –p is the SCAN listener port number.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Managing SCAN VIP and SCAN Listener Resources

• The srvctl modify scan command modifies the SCAN VIP configuration to match that of another SCAN VIP:

$ srvctl modify scan_listener -u

$ srvctl modify scan –n cluster01-scan

• The srvctl modify scan_listener -u command modifies the configuration information for all SCAN listeners to match the current SCAN VIP configuration:

Managing SCAN VIP and SCAN Listener Resources (continued)The srvctl modify scan command modifies the SCAN VIP configuration to match that of another SCAN VIP specified with scan_name. If scan_name currently resolves to more IP addresses than when it was initially configured, new Oracle Clusterware resources for those additional IP addresses are created. If scan_name currently resolves to fewer IP addresses, Oracle Clusterware resources for SCAN VIP addresses with numerically higher ordinal numbers are removed until the remaining SCAN VIP resources match the number of IP addresses to which scan_name resolves.Use the srvctl modify scan command with the following syntax:srvctl modify scan [-n scan_name]

To modify the cluster01-scan SCAN VIP configuration:$ srvctl modify scan -n cluster01-scan

The srvctl modify scan_listener command modifies the configuration information for all SCAN listeners. Use the srvctl modify scan_listener command with the following syntax:srvctl modify scan_listener { -p scan_port | -u }

where –p is the new scan port and –u updates the SCAN listener configuration to match the current SCAN VIP configuration.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Managing SCAN VIP and SCAN Listener Resources (continued)To change the SCAN listener port for LISTENER_SCAN1 and update the cluster configuration, run the following command:# srvctl modify scan_listener -p 1531# srvctl modify scan_listener –u# srvctl config scan_listener ### To verify changes ###SCAN Listener LISTENER_SCAN1 exists. Port: TCP:1531


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Quiz

Which of the following tools cannot be used to manage Clusterware operations or Oracle resources?1. Enterprise Manager2. srvctl

3. Oracle Universal Installer4. crsctl

Answer: 3The correct answer is 3.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Quiz

Which of the following statements regarding the Oracle Local Registry (OLR) is true?1. Each cluster node has a local registry for node-specific

resources.2. The OLR should be manually created after installing Grid

Infrastructure on each node in the cluster.3. One of its functions is to facilitate Clusterware startup in

situations where the ASM stores the OCR and voting disks.

4. You can check the status of the OLR using ocrcheck.

Answer: 1,3,4The correct answer is 1, 3, and 4.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Summary

In this lesson, you should have learned how to:• Display Clusterware management proficiency• Demonstrate OCR backup and recovery techniques• Manage network settings


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Practice 3 Overview

This practice covers the following topics:• Verifying, Starting, and Stopping Oracle Clusterware• Adding and Removing Oracle Clusterware Configuration

Files• Performing a Backup of the OCR and OLR


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Managing Oracle Clusterware

Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Module 4-1 Adding and Deleting Oracle Clusterware Homes


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Objectives

After completing this module, you should be able to:• Perform the prerequisite steps for extending a cluster• Use addNode.sh to add a node to a cluster• Delete a node from a cluster


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Adding Oracle Clusterware Homes

The addNode.sh shell script is used to add nodes to an existing Oracle Clusterware environment. It:• Runs without a graphical interface• Does not perform the prerequisite operating system tasks

Storage

GRID HOME

Operating system

GRID HOME

Operating system

Adding Oracle Clusterware HomesYou can use a variety of methods to add and delete nodes in an Oracle Clusterwareenvironment:

• Silent cloning procedures: Copy images of an Oracle Clusterware installation to other nodes to create new clusters that have identical hardware by using the clone.pl script.

• Enterprise Manager (EM) Grid Control: Provides a GUI interface and automated wizards to the cloning procedures.

• addNode.sh: Invokes a subset of OUI functionality• Delete node procedure: Removes a node from a cluster

In this module, you examine the use of the addNode.sh and the delete node procedure. Special attention must be given to the procedures because some steps are performed on the existing nodes, whereas other steps are performed on the nodes that are being added or removed. Note: In this lesson, assume that host01 and host02 are the existing nodes, with host03being the node that will be added or removed.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Prerequisite Steps for Running addNode.sh

The following steps assume that you already have a successful Linux and Oracle Clusterware installation.1. Make physical connections: networking, storage, and other2. Install the operating system.3. Perform the Oracle Clusterware installation prerequisite

tasks:A. Check system requirements.B. Check network requirements.C. Install the required operating system packages.D. Set kernel parameters.E. Create groups and users.F. Create the required directories.G. Configure the installation owner’s shell limits.H. Configure Secure Shell (SSH) and enable user equivalency.


The addNode.sh script is used to extend an existing, successful Oracle Clusterwareinstallation to more nodes. Before the addNode.sh script can be run, several prerequisite steps must be performed. For step 1, the new node must be physically connected to the existing cluster network infrastructure to include the public, private, storage, and other connections that may exist. Remember that all nodes must use the same adapter names for the public and private interfaces. For step 2, install a cloned image of the operating system that matches the operating system on the other nodes in the cluster, including the required service patches, drivers, and modifications to configuration files. If a cloned image is not possible, the individual modifications that were performed as prerequisite tasks for installing OracleClusterware will have to be performed on the new node for step 3. The provisioning of the storage prerequisite task is not listed because this step has already been performed by the existing nodes. You need to ensure that Secure Shell (SSH) is configured to operate without prompts for both the fully qualified names and nonqualified host names. This involves updates to the authorized_keys and known_hosts files of the existing nodes in addition to the new nodes being added.Depending on the method used to transfer the operating system to the new node, some of the tasks in step 3 may have been performed and only need to be checked.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED



4. Verify the installation with Cluster Verify Utility (cluvfy) from existing nodes.A. Perform a post-hardware and operating system check.

B. Perform a detailed properties comparison of one existing reference node to the new node.

[grid@host01]$ cluvfy stage –post hwos –n host03

[grid@host01]$ cluvfy comp peer -refnode host01 \-n host03 -orainv oinstall -osdba asmdba -verbose

Prerequisite Steps for Running addNode.sh (continued)You have already seen a variety of methods by which Cluster Verify Utility (cluvfy) can be invoked. A complete reference to cluvfy is provided in the lesson titled “Troubleshooting Oracle Clusterware.” For step 4, you invoke cluvfy from an existing node (host01) to perform a post-hardware and operating system installation check against the new node (host03). Next, you perform a detailed properties comparison with cluvfy by using an existing node as a reference node, and comparing it against the new node to be added. If errors are discovered while performing these checks, they must be corrected before continuing with the addNode.sh script. In the slide examples, host01 is an existing node and host03 is the new node that is being added.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Adding a Node with addNode.sh

1. Ensure that Oracle Clusterware is successfully installed on at least one node.

2. Verify the integrity of the cluster and the node to be added (host03) with:[grid@host01] cluvfy stage -pre nodeadd -n host03

3. If you are not using Grid Naming Service (GNS), run the addNode.sh script by using the following syntax to add host03 to an existing cluster :[grid@host01]$ cd /Grid_Home/oui/bin[grid@host01]$ ./addNode.sh –silent \"CLUSTER_NEW_NODES={host03}" \"CLUSTER_NEW_VIRTUAL_HOSTNAMES={host03-vip}"

Adding a Node with addNode.sh

The addNode.sh script is used to distribute the Oracle Clusterware software from an existing node to the new nodes being added to the existing cluster. Without the silent option, the script requires that the DISPLAY environment variable be set; but without the silent option, no graphical windows are displayed. If you are not using GNS, for an existing cluster containing the host01 and host02 nodes, a new node host03 will be added as follows:[grid@host01]$ ./addNode.sh -silent "CLUSTER_NEW_NODES={host03}" "CLUSTER_NEW_VIRTUAL_HOSTNAMES={host03-vip}"

At the end of the addNode.sh script, instructions are given to run several scripts as the root user on selected nodes. Each script has a different name, is located in a different directory, and is run on a different node. Do not run these scripts in parallel. The instructions look like:The following configuration scripts need to be executed as the root user in each cluster node./u01/app/11.2.0/grid/root.sh #On nodes gr7214

As the root user, execute the scripts from a terminal window on the nodes you are adding as directed.Note: Currently, adding a node in a GNS environment using addNode.sh requires CLUSTER_NEW_VIRTUAL_HOSTNAMES be defined.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Completing OUI Silent Node Addition

4. Perform integrity checks on the cluster.

[grid@host01]$ cluvfy stage –post nodeadd –n host03 -verbose

Completing OUI Silent Node AdditionFor step 4, perform integrity checks on the cluster to verify a successful node addition.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Removing a Node from the Cluster

A procedure of steps is used to remove a node.• You cannot simply remove the node from the cluster.• Oracle Central Inventory on each node has information

about all nodes.• The Oracle Cluster Registry (OCR) contains information

about all nodes.

Storage

GRID HOME

Operating system

GRID HOME

Operating system

GRID HOME

Operating system

host03

host03

host03host03

Removing a Node from the ClusterOn each node in the cluster, Oracle Central Inventory on that node contains information about all the nodes of the cluster. The binary OCR and voting disk also contain information about each node of the cluster. Therefore, to remove a node from the cluster properly, a procedure must be followed. You cannot simply remove the node. The procedure to remove a node from the cluster involves running a set of steps. Deleting a node from the cluster is a multiple-step process. Some commands are run on the node to be deleted and other commands are run on an existing node of the cluster. Some commands are run by the root user and other commands are run by the Oracle Clusterwaresoftware owner’s account. When passing arguments to a command, sometimes the existing node is passed, sometimes the node to be removed is passed, and at other times a complete list of remaining nodes is passed as an argument. This requires special attention to detail to avoid making mistakes during the process.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Deleting a Node from the Cluster

1. Verify the location of the Oracle Clusterware home.2. From a node that will remain, run the following as root to

expire the Cluster Synchronization Service (CSS) lease on the node that you are deleting:

3. Run the rootcrs.pl script as root from the Grid_home/crs/install directory on each node to be deleted:

[root@host01]# crsctl unpin css -n host03

[root@host03]# ./rootcrs.pl -delete -force

Deleting a Node from the ClusterIf you run a dynamic Grid Plug and Play (GPnP) cluster using Dynamic Host Configuration Protocol (DHCP) and GNS, skip to step 3.Step 1: Verify the location of the Oracle Clusterware home. This directory should be consistent on all nodes.Step 2: Expire the CSS lease on the node you are deleting. The crsctl unpin command will fail if Cluster Synchronization Services (CSS) is not running on the node being deleted. Run the olsnodes –s –t command to show whether the node is active or pinned. If the node is not pinned, go to step 3. Note: You cannot unpin a node that has a Real Application Clusters (RAC) instance older than 11.2.0 on that node, if you installed Oracle Clusterware 11g release 2 (11.2) on that node.Step 3: Disable the Oracle Clusterware applications and daemons running on the node. Then if you run a dynamic Grid Plug and Play cluster using DHCP and GNS, continue to step 4.Note: This procedure assumes that the node to be removed can be accessed. If you cannot execute commands on the node to be removed, you must manually stop and remove the VIP resource using the following commands as root from any node that you are not deleting:

# srvctl stop vip -i vip_name -f # srvctl remove vip -i vip_name -f

where vip_name is the Virtual IP (VIP) for the node to be deleted.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED



4. From a node that will remain, delete the node from the cluster with the following command run as root:

5. On the node to be deleted, as the user that installed Oracle Clusterware, run the following command from the Grid_home/oui/bin directory:

[root@host01]# crsctl delete node -n host03

[grid@host03]$ ./runInstaller -updateNodeList ORACLE_HOME=Grid_Home "CLUSTER_NODES={host03}" CRS=TRUE -local

Deleting a Node from the Cluster (continued)Step 4: As the root user, delete the node from the cluster from a node that will remain in the cluster. Then if you run a dynamic Grid Plug and Play cluster using DHCP and GNS, skip to step 7.Step 5: As the user that installed Oracle Clusterware on the node to be deleted, run the command as follows:

[grid@host03]$ cd $Grid_Home/oui/bin[grid@host03]$ ./runInstaller –updateNodeList \ORACLE_HOME=Grid_Home "CLUSTER_NODES={host03}" CRS=TRUE -local


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED



6. On the node that you are deleting, run the runInstallercommand as the user that installed Oracle Clusterware.A. If you have a shared home:

B. For a nonshared home, deinstall the Oracle Clusterware home:

7. On any remaining node, as the Grid software owner, update the node list:

[grid@host03]$ ./runInstaller -detachHome ORACLE_HOME=/u01/app/11.2.0/grid

[grid@host03]# ./deinstall –local

[grid@host01]$ cd /Grid_home/oui/bin[grid@host01]$ ./runInstaller -updateNodeList \ORACLE_HOME=/u01/app/11.2.0/grid \"CLUSTER_NODES={host01,host02}" CRS=TRUE

Deleting a Node from the Cluster (continued)Step 6:

A. For a shared home from the Grid_home/oui/bin directory as the clusterwareowner:

[grid@host03]$ ./runInstaller -detachHomeORACLE_HOME=/u01/app/11.2.0/grid

B. For a nonshared home from the Grid_home/deinstall directory as the Grid software owner:

[grid@host03]# ./deinstall –localWarning: deinstall –local can damage the cluster installation.

Step 7: On any remaining node, update the node list. Run this command as the Grid software owner:

[grid@host01]$ /Grid_Home/oui/bin/runInstaller \-updateNodeList ORACLE_HOME=/u01/app/grid \"CLUSTER_NODES={host01,host02}" CRS=TRUE


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Deleting a Node from a Cluster(GNS in Use)

If your cluster uses GNS, do the following (steps from the previous procedure):3. Run the rootcrs.pl script as root from the

Grid_home/crs/install directory on each node to be deleted.

4. From a node that will remain, delete the node from the cluster.

7. On any remaining node as the Grid software owner, update the node list.

Deleting a Node from a Cluster (GNS in Use)If your cluster is a dynamic Grid Plug and Play cluster using DHCP and GNS, the procedure for deleting a node from the cluster is simplified. Run just steps 3, 4, and 7 from the previous procedure.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED



8. On any remaining node, verify that the specified nodes have been deleted from the cluster.[grid@host01]$ cluvfy stage -post nodedel –n host03 [-verbose]

Deleting a Node from the Cluster (continued)For step 8, run the cluvfy command to verify that specific nodes have been successfully deleted from a cluster. Typically, this command verifies that the node-specific interface configuration details have been removed, the nodes are no longer a part of the cluster configuration, and proper Oracle Automatic Storage Management (ASM) cleanup has been performed.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Quiz

The addNode.sh script can generate fixup scripts to correct prequisites for new nodes for an existing cluster.1. True2. False

Answer: 2No, addNode.sh requires that the new node to be added be already configured properly. Thecluvfy utility can be used to generate fixup scripts.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Summary

In this module, you should have learned how to:• Perform the prerequisite steps for extending a cluster• Use addNode.sh to add a node to a cluster• Delete a node from a cluster


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Module 4-2 Patching Oracle Clusterware


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Objectives

After completing this module, you should be able to:• Describe the types of patches and upgrades available• Design for rolling patches and rolling upgrades• Compare software versions with the active version• Install a patchset with the Oracle Universal Installer (OUI)

utility• Install a patch with the opatch utility


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Out-of-Place Oracle Clusterware Upgrade

• Oracle Clusterware 11g Release 2 supports only out-of-place upgrades.– An in-place upgrade is installed in the existing Clusterware

home and replaces the older software.– An out-of-place upgrade has both Clusterware versions

present on the nodes at the same time, in different Grid homes; but only one is active.

• Install Oracle Clusterware in a separate home before the upgrade reduces the down time for cluster upgrades.

• The active software version and Grid home location are stored in the OCR.

• There is a clusterwide active version that reflects the version of the communication protocols and shared disk data structures that are currently in use on the cluster.

Out-of-Place Oracle Clusterware UpgradeOracle Clusterware 11g Release 2 supports only out-of-place upgrades. Out-of-place upgrades mean the new software is placed in a different Oracle Clusterware home directory. In past versions, in-place upgrades were performed. In-place upgrades (or downgrades) mean that the software being installed is placed in the same home directory as the existing or active software. With the out-of-place upgrade, both versions of the software are present on the nodes at the same time, in different Grid homes; but only one is active.Installing Oracle Clusterware in a separate home before the upgrade reduces the planned outage time required for cluster upgrades, which assists in meeting availability service-level agreements. After the software is installed, you can upgrade the cluster by stopping the previous version of the software and starting the new version node by node (known as a rolling upgrade).Each node maintains a “software version” that indicates the version of the Oracle Clusterwarethat is active on the machine. The software version is stored in the OCR. There is also aclusterwide “active version” for the entire cluster that reflects the version of the communication protocols and shared disk data structures that are currently in use on the cluster.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Oracle Clusterware Upgrade

Oracle Clusterware UpgradeUpgrading Oracle Clusterware using the Oracle Database 11g Release 2 OUI allows you to perform an out-of-place upgrade. The Specify Installation Location page is displayed. The Specify Location field is not read-only, as you can see in the graphic in the slide. All you can enter is a new directory under the current Oracle base in which to install the new Clusterwaresoftware, performing an out-of-place upgrade.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Types of Patches• CRS Patchset:

– Is installed with the OUI utility– Has the naming convention: 10.2.0.4.0,11.1.0.7.0,11.2.0.1– Is cumulative for each base release

• CRS Bundle Patch (BP):– Is installed with the opatch utility– Has the naming convention: 11.1.0.6.0 BP#1, BP#2, BP#3– Is a fully regression tested bundle of around 20–30 patches– Is cumulative for each BP on the same patchset level

• CRS Merge Label Request (MLR) or “One Off” patch:– Is installed with the opatch utility– Generally contains priority 1 fixes not in a bundle yet

• Patch Set Update (PSU)– Is a cumulative patch containing recommended bug fixes released

on a quarterly schedule– Installed with the opatch utility

Types of PatchesThere are different types of patches that are available for Oracle Clusterware. The first type is a patchset. This is a set of patches provided on top of a base release (for example, 10.2.0.1.0 or 11.1.0.2.0). A patchset increments the fourth digit of the release number. A patchset will include updates for Oracle Clusterware and updates for the Oracle RDBMS software. You must update Oracle Clusterware before updating the Oracle RDBMS product. Though the reverse is not true, Oracle Clusterware may be patched without patching the RDBMS.Patchsets are released less frequently than other types of patches. They are cumulative and may contain hundreds of fixes. The Oracle Universal Installer (OUI) utility is always used to install patchsets. The second common type of patch available for Oracle Clusterware is known as a CRS Bundle Patch (BP). A CRS BP is a small grouping of individual fixes, usually around 20 to 30, that is fully regression tested and released more frequently than patchsets. Each CRS bundle patch on the same patchset level is cumulative for that patchset level only. The naming convention is 11.1.0.6.0 BP#1, 11.1.0.6.0 BP#2, and so on. When the patchset level is incremented, the CRS bundle patch numbering will restart. CRS bundle patches are always applied with the opatchutility instead of the OUI utility. BPs are binaries and do not require relinking.Another type of patch that may be applied to Oracle Clusterware is known as a CRS Merge Label Request or a “One Off” patch. These are installed with the opatch utility and are usually priority 1 fixes that are not contained in a bundle yet. One MLR can contain multiple fixes.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Types of Patches (Continued)Patch Set Updates (PSU) are patches containing important updates and fixes accumulated from the last PSU released. The purpose of these types of patches is to provide important updates and fixes in a single, well tested package on a regular schedule. The PSUs generally supersede older bundle patches and establish a new baseline version for easier tracking.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Patch Properties

• All upgrades and patchsets are installed as rolling.• All patch bundles can use the minimum down-time

procedures. Most patch bundles are rolling upgradeable.• Individual patches may be rolling; check the patch.• After 11.2.0.2, patch bundles can be in-place or out-of-

place.

Patch type Tool Method Rolling Upgradable

Patchset OUI Out-of-place Yes

Patch bundle OPatch/Enterprise Manager In-place Most (check)

One-off patch OPatch/Enterprise Manager In-place Most (check)

Patch PropertiesDepending on their type, patches can have different methods and utilities for installation.Patchsets can be installed as a rolling patch, with the Oracle Universal Installer (OUI).Patchsets are installed out-of-place. Patch bundles can be installed with either Enterprise Manager (EM) or OPatch. Even EM Database Control can be used to patch Oracle Clusterware and the RDBMS software. Patch bundles are installed in-place and most bundles can be installed as a rolling patch, but check the README.txt file that comes with the patch, or the patch metadata. One-off patches can only be installed in-place with EM or OPatch. Just like patch bundles, most one-off patches can be installed as a rolling patch, if they were created to be rolling. If the patch or patch bundle were not labeled as rolling, it cannot be used in a rolling manner. Any one-off patch or patch bundle may use the minimum down-time method. The use of Enterprise Manager to install patches is covered in detail in the Oracle Database 2 Day + Real Application Clusters Guide 11g Release 2.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Configuring the Software Library

Use the Provisioning page of Enterprise Manager.• Add a software library location:

Configuring the Software LibraryTo use the patching features of Enterprise Manager, you configure a software library location. You can have one or more library locations. This will be the directory that patches will be stored in when they are transferred to the local cluster. To navigate to the Provisioning page from the Database home page:

1. Click the Software and Support tab.2. In the Deployment Procedure Manager section, click the Deployment and Provisioning

Software Library link.On the Provisioning page, you will find the Software Library Configuration at the bottom of the page. The software library directory location must reference an existing directory.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Setting Up Patching

Setting Up Patching Patching is simplified if the Enterprise Manager is configured to connect to My Oracle Support (formerly, MetaLink). Available patches can be downloaded and deployed through Enterprise Manager. The Patching Setup page can be accessed by a superuser, using the setup button. The patching configuration can also be supplied during installation. To complete the patching configuration, you must supply the credentials to connect to My Oracle Support. This assumes that your cluster has access to the Patch Search URL: http:/updates.oracle.com either directly or through a proxy. The Proxy and Connections Settings tab allows you to configure the Internet connection.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Starting the Provisioning Daemon

The provisioning daemon is started with:$ pafctl start

Enter repository user password :

Enter interval [default 3]:

Provisioning Daemon is Up, Interval = 3

Starting the Provisioning DaemonThe Deployment Procedure Manager allows you to view, edit, run, and monitor deployment procedures. Deployment procedures are best practices provided by Oracle for various provisioning tasks.In order for the Deployment Procedure Manager to function properly, the Provisioning Daemon job should be running. The provisioning daemon monitors the status of running deployment procedures.The pafctl command allows you to start, stop, and view the status of the provisioning daemon.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Obtaining Oracle Clusterware Patches

Obtaining Oracle Clusterware PatchesThe latest patchsets and recommended CRS BPs can be downloaded from the My Oracle Support Web site at the following URL:

http://support.oracle.com/ After signing in to the Web site, click the “Patches & Updates” tab. For the latest patchset, click the “Quick Links to: Latest Patchsets, Mini Packs and Maintenance Packs” link. To locate the latest CRS BP, if there are any available, start from the “Patches & Updates” tab and click the “Oracle Database” link under the Recommended Patches heading. The Recommended Patches Search page will then be displayed. Leave the Product field set to the value of “Oracle Database Family,” change the value for the Release field, change the value of the Patch Target field to “Real Application Clusters,” change the value of the Platform field, and then click Go. The latest CRS BP will be listed if available.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Obtaining Oracle Clusterware Patches

Obtaining Oracle Clusterware Patches (continued)Individual patches that are not included in a BP are usually made available as a result of a service request logged with Oracle Support for a specific problem.Note: This slide queried for the 11.1.0.7 release to show an example for a BP. At the time of writing this course, there were no BPs available for 11.2.0.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Uploading Patches

Uploading PatchesWhen a direct connect to the patch URL is not available, patches can be downloaded to a local machine and then uploaded to the Software Library. To upload patches to the Software Library using Enterprise Manager database control:

1. On the Database Home page, click Software and Support.2. On the Software and Support page, select View Patch Cache under the heading Database

Software Patching.3. On the Patch Cache page, click Upload Patch. The “Create Oracle Software Update

Component” page appears.4. Click the Browse button and then navigate to the directory where you saved the patch file

to disk. If you are accessing DB Control from a browser running on a client computer, then the patch must be available on that client computer. When you have located the patch file, click the file name to select the file, then click Open. This places the file name in the Patch File field.

5. In the Patch Attributes section of the “Create Oracle Software Update Component” page, enter the following information about the patch you are uploading:

- Patch number- The type of patch (individual or patchset)- The date that the patch was created


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Uploading Patches (continued)- The patch description- The product family for the patch (for example, Oracle Database)- The product that the patch applies to (for example, RDBMS Server) - The release version of the product- The platform that your database runs on- The language to use (for example, American English)- (Optional) A description of the patch

You can find most of this information in the patchset’s README file.6. When you have finished entering the information, select “Add Patch File to Patch

Cache” and click Upload. The Patch Cache page appears again, with a confirmation message.

7. The patch now appears in the Patch Cache.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Deployment Procedure Manager

Deployment Procedure ManagerThe “Deployment Procedure Manager” page shows several procedures that can be scheduled to apply patches. To navigate to the “Deployment Procedure Manager” page, start at the database home page, click the Software and Support tab. On the Software and Support page, click Deployment Procedures. Then you can select a procedure and schedule deployment.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Reduced Down-Time Patching for Cluster Environments

Patching Oracle Clusterware and Oracle RAC can be completed without taking the entire cluster down.• OPatch now can apply patches in multinode, multipatch

fashion.• OPatch detects whether the database schema is at an

earlier patch level than the new patch, and runs SQL commands to bring the schema up to the new patch level.

• OUI installs patchsets as out-of-place upgrades, reducing the down time required for patching.

Reduced Down-Time Patching for Cluster EnvironmentsPatching Oracle Clusterware and Oracle RAC can be completed without taking the entire cluster down. In many cases, patching can be performed with zero down time. This also allows for out-of-place upgrades to the cluster software and Oracle Database, reducing the planned maintenance down time required in an Oracle RAC environment.OPatch can now apply patches in multinode, multipatch fashion. OPatch will not start up instances that have a nonrolling patch applied to it if other instances of the database do not have that patch. OPatch also detects whether the database schema is at an earlier patch level than the new patch, and it runs SQL commands to bring the schema up to the new patch level.You can use srvctl to shut down the Oracle software running within an Oracle home, in preparation for patching. Oracle Grid Infrastructure patching is automated across all nodes, and patches can be applied in a multinode, multipatch fashion.Patchsets are now installed as out-of-place upgrades to the Grid Infrastructure software (OracleClusterware and Automatic Storage Management) and Oracle Database. This reduces the down time required for planned outages for patching.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Rolling Patches

A rolling patch allows one node at a time to be patched, while other nodes continue to provide service. It:• Requires distinct software homes for each node• Allows different versions to coexist temporarily• May not be available for all patches

GRID HOMENode1 Node2GRID HOME

Local FS storage Local FS storage

GRID HOMENode1 Node2

Shared CFS storage

Software not shared Software shared

All nodes must be patched at the same time.

Rolling patches may be allowed,depending on the patch.

Rolling PatchesBy default, Oracle supports rolling upgrade for patches for Oracle Clusterware. This is available for patchsets, bundle patches, and most individual patches. A rolling patch allows one node to be patched to the latest version, while other nodes continue to use the older version and provide business service.Rolling patches are enabled by using a locally accessible, nonshared file system to store the software files, known as Grid home. Each node maintains its own copy. With this technique on a 50-node cluster, there would be 50 copies of the Oracle Clusterware software—that is, 50 copies requiring disk space and 50 copies to update. This technique enables rolling patches and rolling upgrades by allowing two versions in the cluster at the same time.Rolling patches cannot be done when the Oracle Clusterware software files are stored in a shared cluster file system in which a single copy of the software is shared among all nodes. A single copy requires much less disk space and a single copy to patch or upgrade. However, to patch or upgrade, the software must be stopped. Stopping the Oracle Clusterware software also requires all databases, applications, and services that depend on Oracle Clusterware to be stopped. This technique requires a complete outage with down time to patch or upgrade.Note: A patchset that can be rolled for the clusterware may not be able to be rolled for the RDBMS.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Checking Software Versions

• With rolling patches, the software version may be temporarily newer than the active version.– To check the software version on a single node:

– To check the active version of the cluster:

• Different versions should exist only while applying a patch.

$ crsctl query crs softwareversion [hostname]

$ crsctl query crs activeversion

Checking Software VersionsWhen a rolling patch or upgrade is being performed, two versions of the software will temporarily coexist in the cluster. The software version is the latest version that is installed on an individual node. You can check the software version registered in the OCR with the following command:

$ crsctl query crs softwareversionOracle Clusterware version on node [host01] is [11.1.0.7.0]

The active version is the lowest version anywhere in the cluster. It applies to the cluster and not an individual node. The active version is not updated until the last node has been updated to the newest version. You can check the active version with the following command:

$ crsctl query crs activeversionOracle Clusterware active version on the cluster is [11.1.0.6.0]

Permanently operating Oracle Clusterware at different versions is not supported. This is allowed only for a short duration—that is, the time it takes to apply the patchset or patch to the cluster.Note: The version of Oracle Clusterware must be greater than the version running other Oracle products such as the Real Application Clusters (RAC) database and ASM software versions.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Installing a Rolling Patchset with OUI

On one node of the cluster, perform the following steps:1. Read the latest release notes for the patchset.2. Shut down all applications that depend on Oracle

Clusterware. 3. Stop the Oracle Clusterware node applications, relocating

the Virtual IP (VIP) to another node.

4. Start the newer OUI located from the patchset.

$ srvctl stop nodeapps -n host01 -r

$ cd patchset_directory/Disk1$ ./runInstaller

Installing a Rolling Patchset with OUIAfter downloading the latest patchset, unzip the distribution into a directory that is accessible by the first node that will be patched. The patchset does not need to be on a shared file system. The installer will distribute the patchset software to the other nodes in the cluster. Step 1: Read the platform-specific release notes. The patchset is specific to an operating system platform and should contain platform-specific release notes. The outline being presented here is generic, and specific operating systems may contain different steps. Step 2: It is necessary to stop all applications on the first node that depend on OracleClusterware such as database instances, database listeners, ASM instances, and any other applications. The database administrator may choose to disable the connection mechanisms to the first node and have users that are currently connected to the node to log out, log back in, and get redistributed to other nodes before stopping the applications. Step 3: Stop the Oracle Clusterware node applications with the srvctl utility as illustrated in the slide. The –r option relocates the Virtual IP (VIP) to other nodes to avoid connection timeouts. If the OCR and voting files are in ASM, use the crsctl stop crs command.Step 4: Always run the latest version of the installer from the unzipped distribution instead of the older version that can be found in the existing clusterware software. OUI is used to installpatchsets and must be invoked from a graphical-capable session. The OUI utility must be invoked by the user account that owns the Oracle Clusterware software, not the root user.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Patchset OUI

Patchset OUINote: The instructions here are general because there were no patchsets available at the time of this writing for Oracle 11g Release 2. Check the instructions that come with the patchset.Invoke OUI from the patchset directory and follow the instructions. In the Welcome window, you may verify a list of all the products that are currently installed on the machine by clicking the Installed Products button. Click the Next button to install the patchset.The Source window may be displayed, so you can specify the directory where the patchsetresides. The Install Location window asks for the Oracle Base directory and the Oracle Home name and directory; the default values depend on the environment variables that are set and the applications currently installed. A product selection window may appear because thepatchset contains both RDBMS and clusterware patches. For an existing cluster, the Specify Hardware Cluster Installation Mode window shows all nodes automatically selected and the option to change this is not allowed. The software will be distributed to each node in the cluster, and no other actions will be taken on those nodes that continue to provide business services.In the Summary window, all nodes are listed, but software distribution is the only action that is being performed to the other nodes. The remaining nodes can continue to run the OracleClusterware stack and applications, providing continuous business service until they are ready to be brought down.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Installing a Rolling Patchset with OUI

As the root user, on one node of the cluster, perform the following steps:5. Shut down the Oracle Clusterware and Oracle daemons.

6. Run the root upgrade script.

7. Start up all dependent applications that were shut down in step 2.

8. Repeat all steps except step 4 (runInstaller) on each node.

# crsctl stop crs

# cd /u01/app/11.2.0/grid/install# ./root*.sh

Installing a Rolling Patchset with OUI On the first node that the installation was run on, log in as the root user and invoke thecrsctl utility to stop the clusterware stack. Before stopping the clusterware stack, the software version will still report the old version.After the clusterware stack has been successfully stopped, invoke the script named in the OUI instructions to perform the update. This script restarts the clusterware stack at the end and the software version then reports the newer version. The active version remains at the older version until all nodes have been updated.After the clusterware stack has been restarted, all the dependent applications that were shut down on this one node can be restarted. Repeat all the steps on the remaining nodes, except step 4 (runInstaller). The software has already been distributed to each node in thecluster. The updates for rest of the nodes need to be completed in a timely manner. Running the nodes of an Oracle Clusterware environment at different software versions for an extended length of time is not supported.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


OPatch: Overview

OPatch is a Java-based utility that allows the application and rolling back of interim patches. It:• Is used for CRS BP and MLR patches• Supports rolling-patch application for Oracle Clusterware• Maintains an inventory of the patches that are installed• Does not require CRS patches to be relinked• Is invoked as the Grid Infrastructure software owner• Requires special prescripts and postscripts to be invoked

for CRS BP:– prerootpatch.sh (as root)– prepatch.sh (as the clusterware owner)– postpatch.sh (as the clusterware owner)– postrootpatch.sh (as root)

OPatch: OverviewOPatch is a Java-based utility that allows the application and rolling back of interim patches to an Oracle product. It is not used to apply patchsets. OPatch is included with the OracleClusterware 11g installation. When patching an Oracle Clusterware installation using the rolling-patch method, the user is prompted for which nodes to patch during the patch installation process. A rolling patch is identified inside the patch and cannot be enabled when invoking the patch tool. OPatch also supports a minimum down-time mode. An inventory of all patches that have been installed is maintained by OPatch. Patching Oracle Clusterware has special considerations: Part of the software is owned by root and part is owned by the account used to install the product. To patch the root-owned components, the clusterware software will need to be unlocked by running special prepatchscripts. After patching, the clusterware software will need to be returned to its previous secure settings by running postpatch scripts. These script can be found in the patch directory. The additional scripts include the following:• prerootpatch.sh:

- Confirms that the script is invoked as the root user- Verifies that Oracle Clusterware is stopped on the local node- Checks that the correct parameters were specified


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

OPatch: Overview (continued)- Unlocks the Oracle Clusterware software, setting the ownership of some files to the

non-root user, and changes permissions• prepatch.sh:

- Checks that the parameters are correct- Changes the permissions on selected files- Searches for variables from the installation included in the existing files- Verifies that the values were obtained for the variables- Saves the variables and values in the Grid_home/install/params.crs file

• postpatch.sh:- Verifies that the Grid_home/install/params.crs file has the correct

format- Parses and replaces the correct values in the files sourcing from params.crs

- Resets the permissions on selected files• postrootpatch.sh:

- Locks the executables and directories by changing the ownership to root- Copies the initialization wrapper scripts to the correct location- Updates /etc/inittab so that the new files are used to start Oracle Clusterware- Waits for Oracle Clusterware to start

Note: With the June 2010 PSU, prerootpatch.sh, prepatch.sh, postpatch.sh andpostrootpatch.sh have been replaced by rootcrs.pl although prepatch.sh andpostpatch.sh are still used for database homes.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


OPatch: General Usage

• To define the ORACLE_HOME or –oh option on all commands:

• To obtain help with the OPatch syntax:

• To check whether a patch supports a rolling application (Run from the patch directory.):

$ export ORACLE_HOME=/u01/app/11.2.0/grid$ opatch command [options]

$ opatch command –oh /u01/app/11.2.0/grid [options]

or

$ opatch command –help

$ opatch query -all | grep –i Rolling

OPatch: General UsageThe OPatch utility requires that the ORACLE_HOME environment variable be defined or that the value of ORACLE_HOME be passed as an argument on the command line with the –ohoption. For the case of Oracle Clusterware, ORACLE_HOME refers to the installation directory for Oracle Clusterware, not the location of other Oracle products that may be installed. In general, ORACLE_HOME refers to the home for the product to be patched.The OPatch documentation can be found in the Grid_home/OPatch/docs directory, which includes the OPatch Users Guide and the OPatch Prerequisite Users Guide. The utility contains help for its syntax by using the –help option as follows:

opatch -helpopatch apply -helpopatch lsinventory -helpopatch rollback -helpopatch prereq -helpopatch util –help

In general, CRS BPs and CRS MLR patches can be applied in a rolling fashion—that is, one node at a time. However, it is still important to check each patch for exceptions to this rule. To verify that a patch supports rolling applications, unzip the downloaded patch into a directory of your choosing and, from that directory, issue the following command:$ORACLE_HOME/OPatch/opatch query -is_rolling_patch <patch_location>


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Before Patching with OPatch

• Check the current setting of the ORACLE_HOME variable.• Back up the directory being patched with an OS utility or

Oracle Secure backup.• Stage the patch to each node.• Update the PATH environment variable for the OPatch

directory.

Before Patching with OPatchThe Oracle Patching utility, OPatch, verifies that the ORACLE_HOME environment variable names an actual directory. You should verify that the ORACLE_HOME variable is set to the Oracle home of the product you are trying to patch.It is best practice to back up the software directory that you are patching before performing any patch operation. This applies to Oracle RAC, ASM, or Oracle Clusterware software installation directories. The backup should include the Oracle Inventory directory as well.If you manually download the patch and use OPatch to install the patch, you must stage the patch on each node. If you use Enterprise Manager to download the patch and you selected all the nodes in your cluster as targets for the patch, then the patch is automatically staged on those nodes. The opatch binary file is located in the $ORACLE_HOME/OPatch directory. You can either specify this path when executing OPatch, or you can update the PATH environment variable to include the OPatch directory. To change the PATH variable on Linux, use:

$ export PATH=$PATH:$ORACLE_HOME/OPatch


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Installing a Rolling Patch with OPatch

1. Verify that Oracle Inventory is properly configured.

2. Stop CRS on the first node to be patched.

3. Unlock the protected files.

4. Save important configuration settings.

[grid]$ opatch lsinventory -detail -oh \/u01/app/11.2.0/grid

[root]$ crsctl stop crs

[root]# cd <patch_location>[root]# ./custom/scripts/prerootpatch.sh -crshome\/u01/app/11.2.0/grid -crsuser grid

[grid]$ ./custom/scripts/prepatch.sh –crshome\/u01/app/11.2.0/grid

Installing a Rolling Patch with OPatchThe OPatch utility can patch all cluster nodes simultaneously with a single invocation of the command. However, this would require Oracle Clusterware to be stopped on all nodes simultaneously. To avoid complete outage, patches can be applied in a rolling fashion to the local node first, and later to each successive node. To apply a rolling patch with OPatch, start in step 1 by verifying that Oracle Inventory can be located and properly configured with the following command:

opatch lsinventory –detail –oh Grid_home

If the ORACLE_HOME environment variable has been defined, it is not necessary to include the –oh option. For step 2, stop the Oracle Clusterware process stack on the local node with the following command:

crsctl stop crs

For step 3, log in to the root user account and unlock the protected files with the following command:

prerootpatch.sh –crshome Grid_home -crsuser <Grid owner>For step 4, save important configuration settings to prevent them from being overwritten by the patch using the following command:

prepatch.sh –crshome Grid_home

The Grid_home/install/params.crs file will have been created.Oracle 11g: RAC and Grid Infrastructure Administration Accelerated 4 - 42

Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Installing a Rolling Patch with OPatch

5. Patch the CRS installation on the first node only.

6. Apply configuration settings to the patched files.

7. Restore the lock to the protected files.

8. Verify patch installation.

9. Repeat steps 1–8 on each node, one at a time.

[grid]$ opatch apply -local -oh Grid_home\patch_location

[grid]$ ./custom/scripts/postpatch.sh -crshome\Grid_home

[root]# ./custom/scripts/postrootpatch.sh\-crshome Grid_home

[grid]$ opatch lsinventory -detail -oh Grid_home

Installing a Rolling Patch with OPatch (continued)For step 5, patch only the local Grid installation on the first node with the following command:

opatch apply –local –oh grid_home patch_location

The patch location is the name of the directory when the patch was unzipped. If a path name is not given, the current directory is used. For step 6, apply the configuration settings that were saved in step 4 to the files that have been overwritten by the patch with the following command:

postpatch.sh –crshome <grid_home>

For step 7, it is necessary to restore the lock of the Oracle Clusterware software by setting the ownership of selected files back to the root user and setting the permissions accordingly by using the following command as the root user:

postrootpatch.sh –crshome Grid_home

For step 8, verify the patch installation with the following command:opatch lsinventory –detail –oh Grid_home

The eight steps can then be repeated on each node of the cluster, one node at a time, to roll the patch throughout the cluster.Note: A patch may contain both Grid and RDBMS patch elements. Refer to the README.txt file in the patch for additional instructions.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Installing a Patch with Minimum Down Time with OPatch

Actions: • The local node is always patched first.• The local node is used as a base to patch the other nodes.• The user is prompted for the first set of nodes to patch.• For each node in the first set:

– Stop the instance.– The patch is propagated.– Patch the node.

• Shut down Clusterware on the remaining nodes.• The patch is propagated to the remaining nodes.• Inventory is updated.• Start up the first set before patching each node of the

second set.

Installing a Patch with Minimum Down Time with OPatchIn minimum down-time patching, the nodes are divided into two sets. One set of nodes is shut down and the patch is applied to those nodes. After the first set of nodes has been patched, the second set of nodes is shut down. The first set of nodes is then restarted and the patch is applied to the second set of nodes. After the patch has been applied to the second set of nodes, those nodes are restarted. This method leads to less down time for Oracle RAC, compared to having all the nodes shut down at the same time.When you use the minimum down-time patching method, the following actions occur:

• The local node is always patched first.• The local node is used as a base to patch the other nodes.• The user is prompted for the first set of nodes to patch.• For each node in this first set, the user is asked to stop the instance and then the patch is

propagated and applied to that node before continuing to the next node. • When the first set of nodes has been patched, the user is asked to shut down Clusterware

on the remaining nodes.• The instances are stopped on the last set of remote nodes. • The patch is propagated to the last set of nodes and the inventory is updated. • You can then start up the patched nodes (the first set of nodes) before patching the

remaining nodes.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Quiz

Which tools can be used to install a patchset?1. Oracle Universal Installer2. OPatch3. Enterprise Manager Database Console4. Database Configuration Assistant

Answer: 1,3In Oracle 11g Release 2, the Oracle Universal Installer or Enterprise Manager Database Console can be used.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Quiz

Patching an Oracle Clusterware environment has special consideration due to file ownerships and permissions.1. True2. False

Answer: 1The Oracle Home directory for Oracle Clusterware and many of the files in the directory are owned by root. The permissions and ownership have to be changed before patching and returned to the original state after patching.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Summary

In this module, you should have learned how to:• Describe the types of patches available• Design for rolling patches• Compare software versions with the active version• Install a patchset with the Oracle Universal Installer (OUI)

utility• Install a patch with the opatch utility


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Making Applications Highly Available with Oracle Clusterware

Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Objectives

After completing this lesson, you should be able to:• Describe the high availability components of Oracle

Clusterware• Contrast policy-managed and administration-managed

databases• Describe the functionality of server pools• Describe application-placement policies• Create an application Virtual IP (VIP)• Manage application resources


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Oracle Clusterware High Availability (HA)

Oracle Clusterware provides HA services to Real Application Clusters (RAC) databases and other applications.• Oracle Clusterware monitors all protected applications

periodically.• Based on the defined failover policy, Oracle Clusterware

can restart failed applications on the same node or relocate them to another node.

• It can protect Oracle-based as well as non-Oracle-based applications.

Oracle Clusterware High Availability (HA) Oracle Clusterware is a portable cluster infrastructure that provides HA services to RAC databases and other applications. Oracle Clusterware makes applications highly available by monitoring the health of the applications, by restarting applications on failure, and by relocating applications to another cluster node when the currently used node fails or when the application can no longer run in the current node. In the case of node failure, certain types of protected applications, such as a RAC database instance, may not be failed over to the surviving nodes.A cluster is a collection of two or more nodes where the nodes share a common pool of storage used by the Oracle Clusterware system files (the Oracle Cluster Registry [OCR] and the voting disk), a common network interconnect, and a common operating system.Oracle Clusterware monitors all protected applications periodically, and based on the defined failover policy, it can restart them either on the same node or relocate them to another node, or it can decide to not restart them at all.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Oracle Clusterware HA Components

Several components are used to implement HA with Oracle Clusterware.Component DefinitionResource An entity that Oracle Clusterware manages for HA such as

an applicationAction Program A developed program that provides the logic for starting,

stopping, and monitoring a resourcePrivileges Access and usage privileges for a resource allowing it to

run as a different user than the Cluster Ready Services (CRS) user

Resource Dependency A relationship among resources or applications that implies operational ordering

Application VIP A VIP that an application has a dependency with

OCR Storage mechanism for resource profiles, policies, and privileges

Oracle Clusterware HA ComponentsSeveral components work together in building a Highly Available (HA) framework for applications using Oracle Clusterware. A resource is an entity that Oracle Clusterwaremanages for HA such as an application. This is different from the context of Resource Manager, a tool inside an Oracle database for managing system-consumption resources such as the CPU and the RAM. Resources for HA are defined with an application profile that describes attributes and policies for the application. The application profile also identifies the action program, which is a program or script responsible for providing logic to start, stop, and check the status of a resource. The application profile also defines the failure policies for an application. The Oracle Clusterware software runs with root or administrator rights. Privileges enable Oracle Clusterware to control the components of an application to include allowing the application to run under the context of a different user from that which Cluster Ready Services (CRS) runs under. Resources can have a dependency on other resources for operation. For example, a database resource may depend on a storage resource to be running. An application Virtual IP (VIP) is a VIP that can fail over to other nodes if policies allow it and is one example of a typical application dependency. The application VIP is a resource. All the information about a resource is stored in the Oracle Clusterware OCR configuration file and is available to each node in the cluster.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Resource Management Options

Oracle Database 11g Release 2 provides two styles of resource management for RAC databases:• Administrator-managed configuration:

– Uses a subpool of the built-in GENERIC server pool– Places resources manually as defined by the database

administrator– Is the management strategy used in previous releases– Is good for a small number of nodes

• Policy-managed configuration:– Divides servers in a cluster into sets of server pools– Controls placement of all services by using server pools– Is Workload Management compliant– Is best for a large number of nodes

Resource Management OptionsThere are two styles of configuring resource management for a RAC database: administrator-managed configuration and policy-managed configuration. Each management style uses the concept of server pools, which are a logical division of a cluster into pools of servers. Administrator-managed configuration is the management strategy used in previous releases. For this style, the database administrators define on which servers a database resource should run and place resources manually as needed. It uses a subpool of the built-in GENERIC server pool. This style is best suited for a small number of nodes due to the manual placement requirements.Policy-managed configuration controls the placement of all resources by using server pools. Database administrators specify in which server pool (excluding GENERIC or FREE ) the database resource will run. Oracle Clusterware is then responsible for placing the database resource on a specific server. It eliminates physically assigning resources to particular nodes in the cluster. This style is workload management compliant and best for a large number of nodes.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Server Pools

Server pools are logical divisions of a cluster into pools of servers or nodes. They:• Distribute a uniform workload over several servers in the

cluster• Are allocated to host databases or other resources• Are managed using the crsctl and srvctl commands• Support parent-child relationships among server pools

– Top-level pools are mutually exclusive.• Include two built-in server pools at Oracle Clusterware

installation:– FREE: For servers that are not assigned to other pools– GENERIC: For administrator-managed fixed configuration

and pre-11g Release 2 databases

Server PoolsServer pools are logical divisions of a cluster into pools of servers or nodes. They usually represent a subset of the total number of nodes a cluster contains. Server pools are allocated to host databases or other resources and determine on which nodes those resources may execute. Each server pool name must be unique within the cluster. Server pools distribute a uniform workload (set of Oracle Clusterware resources) over several servers in the cluster. They are managed using the crsctl and srvctl commands. With role-separated management, you can explicitly grant permission to operating system users to change attributes of certain server pools. Server pools support parent-child relationships among the pools. The top-level server pools are always mutually exclusive, meaning that one server in the cluster can reside only in one particular server pool at a certain point in time. Top-level server pools create a logical division of the cluster into subclusters. When Oracle Clusterware is installed, two server pools are created automatically: GENERICand FREE. All servers in a new installation are assigned to the FREE server pool, initially. Servers move from FREE to newly defined server pools automatically.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Server Pool Attributes

Attribute Description

ACL Access Control List that defines privileges for the server pool

ACTIVE_SERVERS List of servers currently assigned to a server pool

EXCLUSIVE_POOLS Governs whether servers can be shared among other pools

IMPORTANCE Relative importance of server pool ranging from 0 (lowest) to 1000

MAX_SIZE Maximum number of servers a server pool can contain

MIN_SIZE Minimum number of servers a server pool can contain

NAME The name of the server pool

PARENT_POOLS Specifies parent pools when creating nested server pools

SERVER_NAMES List of servers that maybe associated with a server pool

Server Pool AttributesServer pool attributes can be specified when you create a server pool, or they can be modified after a server pool has already been created. The only required attribute is NAME. The optional attributes include:• ACL: Defines the owner of the server pool and what privileges are granted to various

operating system users and groups. The value of this attribute is populated at the time a server pool is created based on the identity of the process creating the server pool, unless explicitly overridden. It uses a string in the format of: owner:user:rwx,pgrp:group:rwx,other::r— to specify allowed privileges for the owner, group, and other users. The privileges allowed for a server pool are read-only (r), the ability to modify attributes or delete the server pool (w), and the ability to assign resources to the pool (x).

• ACTIVE_SERVERS: Is a space-delimited list of servers that are currently assigned to the server pool. This attribute is automatically managed by Oracle Clusterware.

• EXCLUSIVE_POOLS: Governs whether servers assigned to this server pool are shared with other server pools. This attribute is a string value containing any arbitrary string. Any other server pool that has the same value for this string is mutually exclusive with this server pool.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Server Pool Attributes (continued)• IMPORTANCE: Determines the relative importance of the server pool compared to other

server pools, with 0 denoting the lowest level of importance and 1000 the highest level of importance. The default value is 0.

• MAX_SIZE: Determines the maximum number of servers a server pool can contain. A value of –1 for this attribute spans the entire cluster and is the default value.

• MIN_SIZE: Determines the minimum number of servers a server pool can contain. The value of this attribute does not set a hard limit. It governs the priority for server assignment. The default value is 0.

• PARENT_POOLS: Allows the creating of nested server pools. Server pools listed in this attribute are referred to as parent server pools. Multiple parent server pools may be specified by using a comma-delimited list of server pool names.

• SERVER_NAMES: Lists the candidate node names upon which servers reside that may be associated with a server pool. If this attribute is empty, Oracle Clusterware assumes that any server may be assigned to any server pool, to the extent allowed by other attributes, such as PARENT_POOLS.

Note: All attributes of the GENERIC server pool are read-only and cannot be modified. For the FREE server pool, only the IMPORTANCE and ACL attributes can be edited.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


GENERIC and FREE Server Pools

• When upgrading Clusterware, all nodes are placed in the GENERIC pool to ensure compatibility with older releases.

• The GENERIC server pool stores pre-11.2 and administrator-managed databases with fixed configurations.

• FREE server pool attributes are restricted, as follows:– SERVER_NAMES, MIN_SIZE, and MAX_SIZE cannot be

edited by the user.– IMPORTANCE and ACL can be edited by the user.

• Configuration attributes of the GENERIC server pool cannot be edited.

GENERIC and FREE Server PoolsWhen Oracle Clusterware is installed, two server pools are created automatically: GENERICand FREE. All servers in a new installation are assigned to the FREE server pool, initially. Servers move from FREE to newly defined server pools automatically. When you upgrade Oracle Clusterware, all nodes are assigned to the GENERIC server pool to ensure compatibility with database releases before Oracle Database 11g Release 2. The FREE server pool contains servers that are not assigned to any other server pools. The attributes of the FREE server pool are restricted, as follows:

• SERVER_NAMES, MIN_SIZE, and MAX_SIZE cannot be edited by the user.• IMPORTANCE and ACL can be edited by the user.

The GENERIC server pool stores pre-11.2 databases and administrator-managed databases that have fixed configurations. The GENERIC pool contains servers that match either of the following:

• Servers that you specified in the HOSTING_MEMBERS attribute of all resources of the application resource type

• Servers with names that you specified in the SERVER_NAMES attribute of the server pools that list the GENERIC server pool as a parent server pool

The GENERIC server pool’s attributes are restricted, as follows:• Configuration attributes of the GENERIC server pool cannot be edited.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

GENERIC and FREE Server Pools (continued)• When you specify a server name in the HOSTING_MEMBERS attribute, Oracle

Clusterware allows it only if the server is:- Online and exists in the GENERIC server pool- Online and exists in the FREE server pool, in which case Oracle Clusterware

moves the server into the GENERIC server pool- Online and exists in any other server pool and the client is either a cluster

administrator or is allowed to use the server pool’s servers, in which case the server is moved into the GENERIC server pool

- Offline and the client is a cluster administratorWhen you register a child server pool with the GENERIC server pool, Oracle Clusterwareallows it only if the server names pass the same requirements as previously specified for the resources. Servers are initially considered for assignment into the GENERIC server pool at cluster startup time or when a server is added to the cluster, and only after that to other server pools.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Assignment of Servers to Server Pools

NAME IMPORTANCE MIN_SIZE MAX_SIZE PARENT_POOLS EXCLUSIVE_POOLS

sp1 1 1 10

sp2 3 1 6

sp3 2 1 2

sp2_child1 2 1 5 sp2 s123

sp2_child2 1 1 5 sp2 s123

Assume that there are no servers in a cluster, all server pools are empty, and server pools are defined as follows:

host01 joins the cluster and will be assigned to the sp2 and sp2_child1 server pools.

Assignment of Servers to Server PoolsAssume that there are no servers currently in a cluster, all server pools are empty, and server pools are defined according to the chart listed in the slide. When a server named host01joins the cluster, server-to-server pool assignment commences:

1. Oracle Clusterware processes only top-level server pools first (those that have no parent server pools). In this example, the top-level server pools are sp1, sp2, and sp3.

2. Oracle Clusterware lists the server pools in order of their IMPORTANCE attribute as follows: sp2, sp3, sp1.

3. Oracle Clusterware assigns host01 to sp2 because sp2 has the highest IMPORTANCEvalue and its MIN_SIZE value has not yet been met.

4. Oracle Clusterware processes child pools of sp2 (sp2_child1 and sp2_child2). The sizes of both server pools are below their MIN_SIZE attribute.

5. Oracle Clusterware lists the two remaining pools in order of their IMPORTANCEattribute as follows: sp2_child1, sp2_child2.

6. Oracle Clusterware assigns host01 to sp2_child1 but cannot assign host01 to sp2_child2 because sp2_child1 is configured to be exclusive with sp2_child2using the EXCLUSIVE_POOLS string of s123.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Server Attributes and States

Attribute Purpose

NAME The node name of the serverACTIVE_POOLS A space-delimited list of the names of the server pools to

which a server belongsSTATE The state of a server can be: ONLINE, OFFLINE,

LEAVING, JOINING, VISIBLE, and RECONFIGURING.STATE_DETAILS Additional details for STATE attributes

Use the crsctl status server command to check the server status:

Note: Actual command output is oriented vertically.

# crsctl status server -f

NAME=host01 NAME=host02 NAME=host03STATE=ONLINE STATE=ONLINE STATE=ONLINEACTIVE_POOLS=ora.OLTP ACTIVE_POOLS=APP ACTIVE_POOLS=ora.OLTPSTATE_DETAILS= STATE_DETAILS= STATE_DETAILS=

Server Attributes and StatesOracle Clusterware assigns each server a set of attributes as soon as you add a server to a cluster. If you remove the server from the cluster, Oracle Clusterware revokes those settings. These attributes include:NAME: The node name of the server. A server name can contain any platform-supported characters except the exclamation point (!) and the tilde (~). A server name cannot begin with a period or with ora. This attribute is required.ACTIVE_POOLS: A space-delimited list of the names of the server pools to which a server belongs. Oracle Clusterware manages this list, automatically.STATE: A server can be in one of the following states:• ONLINE: The server is a member of the cluster and is available for resource placement.• OFFLINE: The server is not currently a member of the cluster. Subsequently, it is not

available for resource placement.• JOINING: When a server joins a cluster, Oracle Clusterware processes it to ensure that

it is valid for resource placement. Oracle Clusterware also checks the state of resources configured to run on the server. When the validity of the server and the state of the resources are determined, the server transitions out of the state.

• LEAVING: When a planned shutdown for a server begins, the state of the server transitions to LEAVING, making it unavailable for resource placement.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Server Attributes and States (continued)• VISIBLE: Servers that have Oracle Clusterware running, but not the Cluster Ready

Services daemon (crsd), are put into the VISIBLE state. This usually indicates an intermittent issue or failure and Oracle Clusterware trying to recover (restart) the daemon. Oracle Clusterware cannot manage resources on servers while the servers are in this state.

• RECONFIGURING: When servers move between server pools due to server pool reconfiguration, a server is placed into this state if resources that ran on it in the current server pool must be stopped and relocated. This happens because resources running on the server may not be configured to run in the server pool to which the server is moving. As soon as the resources are successfully relocated, the server is put back into the ONLINE state.

STATE_DETAILS: This is a read-only attribute that Oracle Clusterware manages. The attribute provides additional details about the state of a server. Possible additional details about a server state are:

• Server state: ONLINE:- AUTOSTARTING RESOURCES: The resource autostart procedure (performed

when a server reboots or the Oracle Clusterware stack is restarted) is in progress for the server.

- AUTOSTART QUEUED: The server is waiting for the resource autostart to commence. When that happens, the attribute value changes to AUTOSTARTING RESOURCES.

• Server state: RECONFIGURING:- STOPPING RESOURCES: Resources that are restricted from running in a new

server pool are stopping.- STARTING RESOURCES: Resources that can run in a new sever pool are starting.- RECONFIG FAILED: One or more resources did not stop and thus the server

cannot transition into the ONLINE state. At this point, manual intervention is required. You must stop or unregister resources that did not stop. After that, the server automatically transitions into the ONLINE state.

• Server state: JOINING:- CHECKING RESOURCES: Whenever a server reboots, the Oracle Clusterware

stack restarts, or crsd on a server restarts, the policy engine must determine the current state of the resources on the server. While that procedure is in progress, this value is returned.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Creating Server Pools with srvctl and crsctl

Use the crsctl utility or the srvctl utility to create additional server pools.• Specifying attributes on the command line:

• Specifying attributes using a text file to supply them:

$ crsctl add serverpool SP1 –attr "MIN_SIZE=2, MAX_SIZE=5, IMPORTANCE=3"

$ crsctl add serverpool SP1 –file /usr/local/bin/SP1_attributes.txt

$ srvctl add srvpool –g SP1 –l 2 –u 5 –i 3 -n "server1,server2"

Creating Server Pools with srvctl and crsctl

Use the crsctl add serverpool command or the srvctl add srvpool command to add a server pool to Oracle Clusterware. The only required attribute is NAME. The first example in the slide specifies the optional attributes on the command line using the crsctlutility. The attribute name and values must be enclosed in double quotation marks ("") and separated by commas. Do not use the crsctl utility for any server pools with names that begin with ora because these server pools are Oracle server pools. The srvctl utility is also shown. It allows only the MIN_SIZE (-l), MAX_SIZE (-u), IMPORTANCE (-i), and SERVER_NAMES (-n) attributes to be specified. The second example in the slide specifies the attributes by identifying a text file that contains the attributes. Additional options to thecrsctl add serverpool command include:

• (-i) : Causes the command to fail if the crsd process cannot complete the request immediately

• (-f) : Is the force option, which causes the crsd process to stop resources running on a server in another server pool and relocates that server into the server pool that you are adding


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Managing Server Pools with srvctl and crsctl

Use the crsctl utility or the srvctl utility to delete and modify server pools.• To delete server pools:

• To modify server pools:

$ crsctl delete serverpool SP1

$ crsctl modify serverpool SP2 –attr "MIN_SIZE=4, MAX_SIZE=8, IMPORTANCE=7"

$ srvctl remove srvpool –g SP1

$ srvctl modify srvpool -g SP2 –l 4 –u 8 –i 7

Managing Server Pools with srvctl and crsctl

Use the crsctl utility or the srvctl utility to delete and modify server pools. Do not use the crsctl utility for any server pools with names that begin with ora because these server pools are Oracle server pools.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Adding Server Pools with Enterprise Manager

Adding Server Pools with Enterprise ManagerTo add a server pool by using Enterprise Manager, click the Cluster folder tab, and then click the Administration subfolder tab. Locate the Server Pools section and click the Add Server Pool link. On the Add Server Pool page, specify the server pool name and whether it will contain Oracle or non-Oracle resources. In addition, specify the maximum size of the server pool. You can select Entire Cluster or Specify Maximum Size and supply an upper limit of servers for this pool. Expanding the Advanced section allows you to configure Minimum Size, Importance, and Parent Server Pool. In addition, you can directly assign specific servers to the pool. If the pool will support non-Oracle resources, Exclusive Groups can also be defined.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Managing Server Pools with Enterprise Manager

Managing Server Pools with Enterprise ManagerServer pools can be managed from Enterprise Manager’s Server Pools page. To navigate to the page, click the Cluster tab, and then click the Administration subfolder tab. In the Server Pools section, click Manage Server Pools. Here you will find detailed information regarding configured server pools. Possible actions from this page include:

• View server pool attributes.• Edit server pool attributes.• Remove server pools.• Relocate server pools.• Create new server pools.

If any of the top-level server pools contain other pools, they can be managed here as well by expanding the parent tree.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Clusterware Resource Modeling

• When an application is registered, you define how Clusterware manages it by defining its attributes.

• The registration information includes an action script or action program that Clusterware calls to start, stop, check, and clean up the application.– An action script is a shell script that a generic script

agent provided by Oracle Clusterware calls.– An application-specific agent is usually a C or C++

program that calls Clusterware APIs directly.• Oracle Clusterware 11g R2 includes two agents:

– Script agent: CRS_HOME/bin/scriptagent– Application agent: CRS_HOME/bin/appagent

Clusterware Resource ModelingWith the introduction of Oracle Database 11g Release 2, there are now more options for managing all types of applications and creating dependencies among them by using OracleClusterware. When you register an application as a resource in Oracle Clusterware, you define how Oracle Clusterware manages the application by using resource attributes that you assign to the resource. The frequency with which the resource is checked and the number of attempts to restart a resource on the same server after a failure before attempting to start it on another server (failover) are examples of resource attributes. The registration information also includes a path to an action script or application-specific action program that Oracle Clusterware calls to start, stop, check, and clean up the application.An action script is a shell script that a generic script agent provided by Oracle Clusterwarecalls. An application-specific agent is usually a C or C++ program that calls OracleClusterware–provided APIs directly. Oracle Clusterware 11g Release 2 (11.2) includes two script agents that make it possible to use scripts to protect an application. The two agents are:

• Script agent (scriptagent in Linux; scriptagent.exe in Windows): Use this agent to use shell or batch scripts to protect an application. Both the cluster_resource and local_resource resource types are configured to use this agent, and any resources of these types automatically take advantage of this agent.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Clusterware Resource Modeling (continued)Application agent (appagent in Linux; appagent.exe in Windows): This agent automatically protects any resources of the application resource type used in previous versions of Oracle Clusterware. You are not required to configure anything to take advantage of the agent. It invokes action scripts in the manner done with previous versions of OracleClusterware and should be used only for the application resource type.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Resource Types

• Oracle Clusterware uses resource types to organize resources employing similar attributes.

• Benefits from the use of resource types include:– You need to manage only necessary resource attributes.– You can manage all resources based on the resource type.

• All resources registered with Oracle Clusterware must be associated with a resource type.

• There are two resource types predefined in Oracle Clusterware:– Local resource– Cluster resource

• You can define custom resource types with Enterprise Manager.

Resource TypesGenerally, resources are unique but some may have common attributes. Oracle Clusterwareemploys resource types to organize these similar resources. Benefits that resource types provide include:

• Manage only necessary attributes required by the resources.• You can easily manage many resources by managing the common resource type.

Every resource that you register in Oracle Clusterware must have a certain resource type. In addition to two resource types included in Oracle Clusterware, you can define custom resource types to suit your needs. The included resource types are:

• Local resource: These are server-centric resources; the type name is local_resource. These run locally on individual servers of the cluster and are not relevant outside of the scope of the server.

• Cluster resource: Cluster-aware resource types (type name is cluster_resource) are aware of the cluster environment and are subject to cardinality and cross-server switchover and failover. The state of cluster resources is relevant in the context of the cluster.

Previous versions of Clusterware supported only the application resource type. This resource type still exists, but only for backward compatibility. Oracle recommends that you register application-type resources as the cluster_resource type in OracleClusterware 11g Release 2.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Adding a Resource Type

Adding a Resource TypeTo add a new resource type to Oracle Clusterware by using Oracle Enterprise Manager, perform the following steps:

1. Log in to Oracle Enterprise Manager Database Control and click the Cluster tab and then the Administration subfolder tab. Then click Add Resource Type.

2. Enter a cluster administrator username and password to display the Add Resource Type page.

3. Enter a name for the resource type in the Name field.4. Select either cluster_resource or local_resource as the base resource type

from the Base Resource Type drop-down list. Generally, select the cluster_resource type for resources that can reside on any server in a cluster. Select the local_resource type for resources that must be present on each server of a cluster, by definition, such as VIPs, ASM instances, and network resources. Clicking the View button allows you to view the chosen resource type’s characteristics, parameters, and dependencies.

5. The optional parameters in the Placement section define where in a cluster OracleClusterware places the resource. Placement characteristics include:

- Cardinality: This indicates the number of nodes the resource should run on simultaneously.

- Degree: This indicates the number of times a resource can run simultaneously on a single node.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Adding a Resource Type (continued)- Server Pool: This allows the resource to be assigned to a server pool if server

pools have been defined. This would indicate that resources configured using this type will be policy managed.

- Hosting Members: This creates an association between a resource and a group of hosts. This would indicate that resources configured using this type will be administrator managed.

- Placement Policy: A balanced policy allows the resource to be placed on any online node.

- Load: Clusterware interprets this value along with that of the PLACEMENT value. When the value of PLACEMENT is balanced, the value of LOAD determines where best to place a resource.

- Active placement: When set to 1, Clusterware uses this attribute to reevaluate the placement of a resource during addition or restart of a cluster server.

6. In the Action Program section, select from the Action Program drop-down list whether Oracle Clusterware calls an action script, an agent file, or both to manage the resource.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Resource Type Parameters

Resource Type ParametersTo further configure the resource type, click the Parameters tab. Resource parameters can be categorized as start parameters, stop parameters, or status parameters, as shown in the slide. The configurable parameters here include:

• Auto Start: Indicates whether Oracle Clusterware automatically starts a resource after a cluster server restart

• Start Attempts: Represents the number of times that Oracle Clusterware attempts to start a resource on the current server before attempting to relocate it

• Start Timeout: Is the maximum time (in seconds) in which a start action can run. OracleClusterware returns an error message if the action does not complete within the time specified.

• Uptime Threshold: Represents the length of time that a resource must be up beforeOracle Clusterware considers the resource to be stable

• Stop Timeout: Is the maximum time (in seconds) in which a stop action can run• Check Interval: Is the interval in seconds between repeated executions of the check

action• Script Timeout: Is the maximum time (in seconds) for an action script to run. The

timeout applies to all actions (start, stop, check, and clean).• Failure Interval: Is the interval, defined in seconds, during which Oracle Clusterware

applies the Failure Threshold attribute. If the value is zero (0), tracking of failures is disabled.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Resource Type Advanced Settings

Resource Type Advanced SettingsUse the table on the Advanced Settings tab to configure permissions for resource parameters using the resource type that you are creating. A parameter configured to be read-only cannot be modified when a resource is registered using the new resource type. A parameter configured to be mandatory must be defined during resource registration. For a complete explanation of resource parameters, refer to the Oracle Clusterware Administration and Deployment Guide.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Defining Resource Dependencies

Defining Resource DependenciesClick Dependencies to configure start and stop dependencies between resources for the new resource type. Two or more resources are dependent when they either depend on each other or affect other resources. For example, when Oracle Clusterware attempts to start a resource, it is necessary for any resources on which the initial resource depends to be online and in the same location. If Oracle Clusterware cannot bring the resources online, the initial (dependent) resource cannot be brought online either. Resource dependencies are separated into start and stop categories. This separation improves and expands the start and stop dependencies between resources and resource types.Start DependenciesOracle Clusterware considers start dependencies between resources whenever a resource state changes from OFFLINE to ONLINE. Start dependencies displayed in EM include:

• Hard: This defines a hard start dependency for a resource if another resource must be ONLINE before the dependent resource can start.

• Weak: This means that an attempt is made to start the resource on which the resource in question is dependent on if it is not ONLINE.

• Attraction: This indicates that Clusterware will attempt to start a resource on the same node on which the resource it is dependent on is running.

• Pullup: When you specify the pullup start dependency for a resource, this resource starts as a result of starting the named resources.

• Dispersion: This indicates that the resource will not be located on the same server as dependent resources, if possible.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Defining Resource Dependencies

Defining Resource Dependencies (continued)Stop DependenciesOracle Clusterware considers stop dependencies between resources whenever a resource state changes from ONLINE to OFFLINE.Hard means that if a resource has a hard stop on another dependency, then the dependent resource must be stopped when the second transitions into the INTERMEDIATE, OFFLINE, or UNKNOWN state. The two resources may attempt to start or relocate to another server, depending upon how they are configured.Clicking the flashlight icon to the right of the “Dependent On Resource” field brings up a list of registered resources to select from. When you have finished, return to the General folder tab and click the Submit button.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Creating an Application VIP by Using crsctl

• If clients access the application through a network, you must register a VIP on which the application depends.

• To create a type for the new application VIP, run the following command as root or the Clusterware owner:

• After you create an application VIP type, you can add the actual VIP resource to the cluster:

• The appvipcfg script simplifies VIP addition:

# crsctl add type app.appvip.type -basetype ora.cluster_vip.type

# crsctl add resource MyAppsVip -type app.appvip.type -attr "RESTART_ATTEMPTS=2,START_TIMEOUT=100,STOP_TIMEOUT=100, CHECK_INTERVAL=10,USR_ORA_VIP=192.168.10.10,START_DEPENDENCIES=’hard(ora.net1.network) pullup(ora.net1.network)’, STOP_DEPENDENCIES=’hard(ora.net1.network)’"

# appvipcfg create -network=1 -ip=192.168.10.10 \-vipname=MyAppsVIP -user=root

Creating an Application VIP by Using crsctl

If clients of an application access the application through a network, you can register a virtual internet protocol address (VIP) on which the application depends. An application VIP is a cluster resource that Oracle Clusterware manages. (Oracle Clusterware provides a standard VIP agent for application VIPs.) Base new application VIPs on this VIP type to ensure consistent behavior of all VIPs deployed in a cluster.To create a type for the new application VIP, run the crsctl add type command as shown in the slide as root or the Oracle Clusterware installation owner. After you create an application VIP type, you can add the actual VIP resource to the cluster. To register a VIP as a resource with Oracle Clusterware, run the crsctl add resource command as root or the Oracle Clusterware installation owner as shown in the slide.On Linux and UNIX operating systems, an application VIP must run as root. If the VIP was not added as root, you must ensure that the VIP can run as root:

1. Log in as root and run the following command:# crsctl setperm resource MyAppsVip –o root

2. Run this command to enable the Oracle Database installation owner to run this script:# crsctl setperm resource MyAppsVip –u user:oracle:r-x

Note: Currently, the USR_ORA_VIP parameter is ignored in GNS/DHCP-enabled clusters.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Creating an Application VIP by Using crsctl (continued)Oracle recommends using the <Grid_home>/bin/appvipcfg script to create or delete an application VIP. The usage of this script is as follows:appvipcfg create -network=network_number -ip=ip_address-vipname=vip_name -user=user_name [-group=group_name

appvipcfg delete -vipname=vip_name]

where network_number is the number of the network, ip_address is the IP address,vip_name is the name of the VIP, and user_name is the name of the user.The appvipcfg script assumes that the default ora.vip network resource (ora.net1.network) is used as the default. In addition, it is also assumed that a default app.appvip type is used for those purposes.As a rule, the network number for the network on which the hostname resolves (public network) is the first network (1). The other networks used by the cluster are assigned arbitrarily. The networks used by the node can be viewed with ifconfig -a on Linux andipconfig -a on Windows, but he network administrator must tell you which network is used for which purpose: public, private, and storage network.You can find the network number using the crsctl command:$ crsctl stat res -p |grep -ie .network -ie subnet |grep -ie name -iesubnet

NAME=ora.net1.network

USR_ORA_SUBNET=192.0.2.0

The net1 string in NAME=ora.net1.network indicates this is network 1, and the second line indicates the subnet on which the VIP will be created.If you are adding an application VIP to a network other than the default network and it has not been assigned a network number, the network number can be assigned by using the –S option of srvctl add scan command when adding scan resources to the network:srvctl add scan -n scan_name [-k network_number]

[-S subnet/netmask[/if1[|if2|....]]]


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Creating an Application VIP by Using EM

Creating an Application VIP by Using EM1. Log in to Oracle Enterprise Manager Database Control and click the Cluster tab. Click

Administration and then click Manage Resources.2. Enter an administrator name and password and click Add VIP.3. Enter a name for the VIP in the Name field. The VIP is based on the

ora.cluster_vip_net1.type resource type. 4. Select “Start the resource after creation” if you want the VIP to start immediately.5. The optional parameters define where in a cluster the VIP is placed. 6. In the Action Program section, select from the Action Program drop-down list whether

Oracle Clusterware calls an action script, an agent file, or both to manage the VIP. “Use Agent File” is selected by default, along with a path to an executable. You must also specify a path to the script, file, or both, depending on what you select from the drop-down menu. If you select Action Script, you can click Create New Action Script to use the EM action script template.

7. To further configure the VIP, click Parameters. On this page, you can configure start, stop, and status parameters, offline monitoring, and any attributes that you define.

8. Click Advanced Settings to configure the VIP attributes according to instance.9. Click Dependencies to configure start and stop dependencies between the VIP and other

resources. Click Submit when you finish configuring the VIP.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Managing Clusterware Resources with EM

• Enterprise Manager can now be used to: – Create Oracle Clusterware resources– Manage Oracle Clusterware resources– Monitor Oracle Clusterware resources

• Resource Management tasks that can be performed from EM include:– Editing resource attributes– Starting and stopping resources– Relocating a resource

• Resource particulars that can be monitored from EM:– Cardinality– Current and target states– Running hosts

Managing Clusterware Resources with EM With the introduction of Oracle Database 11g Release 2, you can now use Oracle Enterprise Manager to manage Oracle Clusterware resources. You can create and configure resources in Oracle Clusterware and also monitor and manage resources after they are deployed in the cluster. Using Oracle Enterprise Manager to monitor and manage various Oracle Clusterwareresources eases the daily management in High Availability environments.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Adding Resources with EM

Adding Resources with EMTo add resources to Oracle Clusterware by using Oracle Enterprise Manager:

1. Log in to Oracle Enterprise Manager Database Control and click the Cluster tab, then click Administration, and then click Add Resource.

2. Enter a cluster administrator username and password to display the Add Resource page.3. Enter a name for the resource in the Name field.4. Select either cluster_resource or local_resource from the Resource Type

drop-down list. Oracle Clusterware uses resource types to organize these similar resources. Every resource that you register in Oracle Clusterware must have a certain resource type. In addition to two resource types included in Oracle Clusterware, you can define custom resource types. Generally, select the cluster_resource type for resources that can reside on any server in a cluster. Select the local_resource type for resources that must be present on each server of a cluster, by definition, such as VIPs, ASM instances, and network resources. Clicking the View button allows you to view the chosen resource type’s characteristics, parameters, and dependencies.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED



Resource Parameters

Adding Resources with EM (continued)5. Enter a clear and understandable description of the resource in the Description field.6. Select “Start the resource after creation” if you want the resource to start immediately.7. The optional parameters in the Placement section define where in a cluster Oracle

Clusterware places the resource. Placement characteristics include:- Cardinality- Degree- Server Pool- Hosting Members- Placement Policy- Load - Active placement

8. In the Action Program section, select from the Action Program drop-down list whether Oracle Clusterware calls an action script, an agent file, or both to manage the resource. If you select Action Script, you can click Create New Action Script to use the Oracle Enterprise Manager action script template to create an action script for your resource.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED



Resource parameters:

Adding Resources with EM (continued)To further configure the resource, click the Parameters tab. Resource parameters can be categorized as start parameters, stop parameters, or status parameters, as shown in the slide. The configurable parameters here include:

• Auto Start• Start Attempts • Start Timeout• Uptime Threshold • Stop Timeout • Script Timeout • Check Interval • Failure Interval• Failure Threshold • Restart Attempts • Offline Check Interval


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED



Resource dependencies:

Adding Resources with EM (continued)Click Dependencies to configure start and stop dependencies between resources. Two or more resources are dependent when they either depend on each other or affect other resources. In Oracle Clusterware 11g Release 2, resource dependencies are separated into start and stop categories. Start DependenciesOracle Clusterware considers start dependencies between resources whenever a resource state changes from OFFLINE to ONLINE. Start dependencies displayed in EM include:

• Hard • Weak• Attraction• Pullup• Dispersion


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED



Adding Resources with EM (continued)Stop DependenciesOracle Clusterware considers stop dependencies between resources whenever a resource state changes from ONLINE to OFFLINE.

• Hard means that if a resource has a hard stop on another dependency, then the first resource must be stopped when the second transitions into the INTERMEDIATE, OFFLINE, or UNKNOWN state. The two resources may attempt to start or relocate to another server, depending upon how they are configured.

Click Submit when you finish configuring the resource.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Adding Resources by Using crsctl

• Resources can be registered in Oracle Clusterware by using the crsctl add resource command.

• To add the Apache Web server as:– An administrator-managed resource:

– A policy-managed resource:

# crsctl add resource myApache -type cluster_resource -attr "ACTION_SCRIPT='/u01/ogi/scripts/myapache.scr', PLACEMENT=’restricted’, HOSTING_MEMBERS=’host01 host02’, CHECK_INTERVAL=’30’,START_DEPENDENCIES=’hard(appsvip)’,STOP_DEPENDENCIES=’hard(appsvip)’, RESTART_ATTEMPTS=’2’,"

# crsctl add resource myApache -type cluster_resource -attr "ACTION_SCRIPT=’/u01/ogi/scripts/myapache.scr’, PLACEMENT=’restricted’, SERVER_POOLS=’myServerPool’, CHECK_INTERVAL=’30’, START_DEPENDENCIES=’hard(appsvip)’, STOP_DEPENDENCIES=’hard(appsvip)’,RESTART_ATTEMPTS=’2’,"

Adding Resources by Using crsctl

Resources are registered in Oracle Clusterware 11g Release 2 using the crsctl add resource command. The crs_register and crs_profile commands are still available in the Oracle Clusterware home but are provided for backward compatibility only.To register an application as a resource:crsctl add resource resource_name -type resource_type -attr"attribute_name=’attribute_value’, attribute_name=’attribute_value’, ...”

The name of the resource type follows the -type option, and a comma-delimited list of resource attribute-value pairs enclosed in double quotation marks (" ") follows the -attroption. The attribute values must be enclosed in single quotation marks (' ').To unregister a resource, use the crsctl delete resource command. You cannotunregister an application or resource that is ONLINE or required by another resource, unless you use the -force option. The following example unregisters the Apache Web server application:# crsctl delete resource myApache

Run the crsctl delete resource command as a clean-up step when a resource is no longer managed by Oracle Clusterware. Oracle Corporation recommends that you unregisterany unnecessary resources.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Managing Resources with EM

Management actions

Managing Resources with EMTo manage resources with Oracle Enterprise Manager:

1. Log in to Oracle Enterprise Manager Database Control and click the Cluster tab, then click Administration, and then click Manage Resources.

2. Enter a cluster administrator username and password to display the Manage Resources page.

3. Select “Show Oracle Resources” to display a list of Oracle resources or enter the name of a particular resource you want to manage. After a resource has been chosen, click the View button to view the resource’s attributes or the Edit button to modify them.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Managing Resources with EMStart or stop a resource:

Managing Resources with EM (continued)Your resources can be stopped or started from the EM Manage Resources page as shown in the slide. To stop the resource, my resource, select the resource from the list and click the Stop button. Click the Confirm button on the Start/Stop Confirmation page to proceed with the stop action. The Manage Resources page is then displayed, showing the current state of the resource. It should show DOWN (OFFLINE) for both Current State and Target State as shown in the graphic in the slide. To start the resource, simply select the resource from the Manage Resources page and click the Start button. Click the Continue button on the Start/Stop Confirmation page to continue. The Current State and Target State should indicate UP (ONLINE). In addition, the current hosts will be displayed.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Managing Resources with EM

Relocate a resource:

Managing Resources with EM (continued)Resources can be relocated from one node to another within your cluster from the Manage Resources page. To move the my resource resource from host02 to host01, select the resource and click the Relocate button. On the Relocate Resource page, select host02 as the From Host from the drop-down list. The list shows all nodes on which the resource is currently running. Enter or select host01 as the To Host and click the Continue button. On the Relocate Resource: Confirmation page, click the Continue button to proceed with the action. On the Confirmation page, the command that will be run to relocate the resource is shown:crsctl relocate resource my_resource –n host01 –s host02


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Managing Resources with crsctl

• Start or stop resources with the crsctl start|stopresource command:

• Use the crsctl relocate resource command to relocate applications and application resources:

# crsctl stop resource myApacheCRS-2673: Attempting to stop 'myApache' on 'host01'CRS-2677: Stop of 'myApache' on 'host01' succeeded# crsctl start resource myApacheCRS-2672: Attempting to start 'myApache' on 'host01'CRS-2676: Start of 'myApache' on 'host01' succeeded

# crsctl relocate resource myApache -n host02 CRS-2673: Attempting to stop 'myApache' on 'host01'CRS-2677: Stop of 'myApache' on 'host01' succeededCRS-2672: Attempting to start 'myApache' on 'host02'CRS-2676: Start of 'myApache' on 'host02' succeeded


Start and stop resources with the crsctl start|stop resource commands. Manual starts or stops outside of Oracle Clusterware can invalidate the resource status. In addition, OracleClusterware may attempt to restart a resource on which you perform a manual stop operation. Running crsctl start resource on a resource sets the resource target value to ONLINE.Clusterware attempts to change the state to match the target by running the action program with the start parameter. When a resource is running, both the target state and the current state are ONLINE.To start an application resource that is registered with Oracle Clusterware, use the crsctlstart resource command. For example:# crsctl start resource myApache

Use the crsctl relocate resource command to relocate applications and application resources. To relocate the Apache Web server application to a server named rac2, run the following command:# crsctl relocate resource myApache -n host02

To relocate an application and its required resources, use the -f option with the crsctlrelocate resource command.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED



Enter the following command to view information about all applications and resources in tabular format:# crsctl status resourceNAME=ora.DATA.dgTYPE=ora.diskgroup.typeTARGET=ONLINE , ONLINESTATE=ONLINE on host01, ONLINE on host02

NAME=ora.LISTENER.lsnrTYPE=ora.listener.typeTARGET=ONLINE , ONLINESTATE=ONLINE on host01, ONLINE on host02...NAME=ora.rdba.dbTYPE=ora.database.typeTARGET=ONLINE , ONLINESTATE=ONLINE on host01, ONLINE on host02

NAME=ora.scan3.vipTYPE=ora.scan_vip.typeTARGET=ONLINESTATE=ONLINE on host02

Managing Resources with crsctl (continued)To display status information about applications and resources that are on cluster servers, use the crsctl status resource command. The following example displays the status information for the Apache Web server application:

# crsctl status resource myApacheNAME=myApacheTYPE=cluster_resourceTARGET=ONLINESTATE=ONLINE on host01

Enter the following command to view information about all applications and resources in tabular format:

# crsctl status resource

Append a resource name to the preceding command to determine:• How many times the resource has been restarted• How many times the resource has failed within the failure interval• The maximum number of times that a resource can restart or fail• The target state of the resource and the normal status information

Use the -f option with the crsctl status resource resource_name command to view full information about a specific resource.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


HA Events: ONS and FAN

Node1

CRS EMD

ONS

HAevents

HAevents

HAevents

C appONS C

API

ONS

Java appONS Java

API

ONS

Java app

DBcontrol

JDBC

AQ

Proxyapp

Calloutscript

Calloutexec

C app

OCI API

.NET app

ODP.NET

HAevents

HA Events: ONS and FANHA events are generated when resources change state within an Oracle Clusterwareenvironment. Oracle Notification Service (ONS) is a facility that creates a bridge with middle-tier servers or applications to transport these events to application logic for handling or reaction. ONS is part of a larger framework known as Fast Application Notification (FAN). With FAN, applications use these events to achieve very fast detection of failures and rebalancing of connection pools, following failures and recovery. When FAN is used together with an Oracle Database, the Advanced Queuing (AQ) feature allows HA events to be received by external applications such as .NET clients. The easiest way to receive all the benefits of FAN, with no effort, is to use a client that is integrated with FAN, such as:

• Java Database Connectivity (JDBC) Implicit Connection Cache• User-extensible callouts• Connection Manager (CMAN)• Listeners• Oracle Notification Service (ONS) API• OCI Connection Pool or Session Pool• Transparent Application Failover (TAF)• ODP.NET Connection Pool

Note: Not all the preceding applications can receive all types of FAN events.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Managing Oracle Notification Server with srvctl

• To add Oracle Notification Server:

• To start or stop Oracle Notification Server:

$ srvctl start|stop nodeapps

$ srvctl add nodeapps -l 6100 -r 6200

• To determine the current ONS configuration:

$ srvctl config nodeapps -s ONS daemon exists. Local port 6100, remote port 6200

Managing Oracle Notification Server with srvctl

The srvctl utility can be used to create, manage, and remove the Oracle Notification Server (ONS). To add an ONS, run the following command:srvctl add nodeapps [-l ons_local_port][-r ons_remote_port] [-t host[:port],[host[:port]],..

where -l identifies the ONS daemon local (or listener) port, -r indicates the ONS daemon remote (or write) port, and -t specifies a list of host:port pairs of remote hosts that are part of the ONS network but are not part of the Oracle Clusterware cluster. The local port is used for communication between the ONS process and the ONS clients on the same node. A remote port is defined in the OCR that is used for communication between the ONS process and other ONS processes on other cluster nodes or middle-tier nodes.The syntax to start and stop the ONS is as follows:srvctl start|stop nodeapps [-f]

To display configuration information for ONS (and all other nodeapps), run the following command:srvctl config nodeapps

To get the environment variables for ONS (and all other nodeapps), run the following command:srvctl getenv nodeapps


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Quiz

When Oracle Clusterware is installed, the GENERIC and FREEserver pools are created automatically.1. True2. False

Answer: 1The statement is true.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Quiz

Oracle Database 11g Release 2 offers two methods for managing resources. Resources can be managed:1. Administration Based Management2. Threshold Based Management3. Policy Based Management

Answer: 1,3Administration based management and policy based management are the correct answers.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Summary

In this lesson, you should have learned how to:• Describe the high availability components of Oracle

Clusterware• Contrast policy-managed and administration-managed

databases• Describe the functionality of server pools• Describe application-placement policies• Create an application Virtual IP (VIP)• Manage application resources


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Practice 5 Overview

This practice covers protecting the Apache application.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Troubleshooting Oracle Clusterware

Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Objectives

After completing this lesson, you should be able to:• Locate Oracle Clusterware log files• Gather all log files using diagcollection.pl• Enable resource debugging• Enable component-level debugging• Enable tracing for Java-based tools• Troubleshoot the Oracle Cluster Registry (OCR) file


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Golden Rule in Debugging Oracle Clusterware

• Always make sure that your nodes have exactly the same system time to:– Facilitate log information analysis– Ensure accurate results when reading GV$ views for Oracle

Real Application Clusters (RAC) database instances– Avoid untimely instance evictions

• The best recommendation is to synchronize nodes using Network Time Protocol (NTP).– Modify the NTP initialization file to set the -x flag, which

prevents time from being adjusted backward.

• If NTP is not used, Clusterware will automatically configure Cluster Time Synchronization Service (CTSS).

# vi /etc/sysconfig/ntpOPTIONS="-x -u ntp:ntp -p /var/run/ntpd.pid"

Golden Rule in Debugging Oracle ClusterwareIt is recommended that you set up Network Time Protocol (NTP) on all cluster nodes before you install Oracle Clusterware. This synchronizes the clocks among all nodes and facilitates the analysis of tracing information based on time stamps as well as results from queries issued on GV$ views when using an Oracle RAC database instance. These views are used by the database administrator to view consolidated database information from each node in the cluster. For Oracle Enterprise Linux, NTP is configured using the /etc/ntp.conf file. Edit the file and add the following entries:server name01.example.com #Server with atomic clockserver name02.example.com #Server with less accuracyrestrict name01.example.com mask 255.255.255.255 nomodify notrap noqueryrestrict name02.example.com mask 255.255.255.255 nomodify notrap noquery

You can start the NTP service with the service ntpd start command. Enable NTP to start at each boot with the chkconfig ntpd on command. The ntpq utility can be used to check the performance of the NTP servers.If you are using NTP, and you prefer to use it instead of CTSS, then you need to modify the NTP initialization file to set the -x flag, which prevents time from being adjusted backward. Restart the network time protocol daemon after you complete this task.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Golden Rule in Debugging Oracle Clusterware (continued)To do this, on Oracle Enterprise Linux, Red Hat Linux, and Asianux systems, edit the /etc/sysconfig/ntpd file to add the -x flag, as in the following example:OPTIONS="-x -u ntp:ntp -p /var/run/ntpd.pid"

If NTP is not used, Clusterware will automatically configure Cluster Time Synchronization Service (CTSS) and start the octcssd.bin daemon on the cluster nodes.Note: Adjusting clocks by more than five minutes can cause instance evictions. It is strongly advised to shut down all instances before date/time adjustments.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Oracle Clusterware Main Log Files

<Grid_Home>

<hostname>

crsd cssd evmd

log

alert<nodename>.log

racg

gpnpd mdnsd ohasd

clientadmin ctssdagent

diskmon gipcd gnsd gnsd

Oracle Clusterware Main Log FilesOracle Clusterware uses a unified log directory structure to consolidate the Oracle Clusterware component log files. This consolidated structure simplifies diagnostic information collection and assists during data retrieval and problem analysis.The slide shows you the main directories used by Oracle Clusterware to store its log files:

• Cluster Ready Service (CRS) logs are in <Grid_Home>/log/<hostname>/crsd/. The crsd.log file is archived every 10 MB (crsd.l01, crsd.l02, …).

• Cluster Synchronization Service (CSS) logs are in <Grid_Home>/log/<hostname>/cssd/. The cssd.log file is archived every 20 MB (cssd.l01, cssd.l02, …).

• Event Manager (EVM) logs are in <Grid_Home>/log/<hostname>/evmd.• SRVM (srvctl) and OCR (ocrdump, ocrconfig, ocrcheck) logs are in

<Grid_Home>/log/<hostname>/client/ and $ORACLE_HOME/log/<hostname>/client/.

• Important Oracle Clusterware alerts can be found in alert<nodename>.log in the <Grid_Home>/log/<hostname> directory.

• Oracle Cluster Registry tools (ocrdump, ocrcheck, ocrconfig) logs can be found in <Grid_Home>/log/<hostname>/client.

In addition, important Automatic Storage Management (ASM)–related trace and alert information can be found in the <Grid_Base>/diag/asm/+asm/+ASMn directory, specifically the log and trace directories.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Diagnostics Collection Script

• The <Grid_Home>/bin/diagcollection.pl script is used to collect important log files and must be run as root.

• It generates the following files in the local directory:– crsData_<hostname_date_time>.tar.gz

– ocrData_<hostname_date_time>.tar.gz

– coreData_<hostname_date_time>.tar.gz

– osData_<hostname_date_time>.tar.gz

# /u01/app/11.2.0/grid/bin/diagcollection.pl --collectProduction Copyright 2004, 2008, Oracle. All rights reservedCluster Ready Services (CRS) diagnostic collection toolThe following diagnostic archives will be created in the local directory.crsData_host01_20090729_1013.tar.gz -> logs,traces and cores from CRS home. Note: core files will be packaged only with the --core option.ocrData_host01_20090729_1013.tar.gz -> ocrdump, ocrcheck etccoreData_host01_20090729_1013.tar.gz -> contents of CRS core files osData_host01_20090729_1013.tar.gz -> logs from Operating System...

Diagnostics Collection ScriptUse the diagcollection.pl script to collect diagnostic information from an Oracle Grid Infrastructure installation. The diagnostics provide additional information so that Oracle Support can resolve problems. This script is located in <Grid_Home>/bin. Before executing the script, you must be logged in as root. The example in the slide shows you how to invoke the script to collect diagnostic information. When invoked with the –-collect option, the script generates, in the local directory, the four files mentioned in the slide. • coreData...tar.gz contains core files and core file analysis extracted to text files.• crsData...tar.gz contains log files from <Grid_Home>/log/<hostname>.• ocrData...tar.gz files contain the results of an ocrdump and ocrcheck, and the

list of OCR backups.• osData...tar.gz contains /var/log/messages and the associated archived files.

You can also invoke the script with the -–clean option to clean out the files generated from a previous run in your local directory. Alternatively, you can invoke the script to capture just a subset of the log files. You can do so by adding extra options after the –-collect option: –-crs for collecting Oracle Clusterware logs, –-core for collecting core files, or –-all for collecting all logs. The –-all option is the default.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Cluster Verify: Overview

• Use the Cluster Verification Utility (CVU) to verify that you have a well-formed cluster for Oracle Grid Infrastructure and RAC:– Installation– Configuration– Operation

• You can perform a full stack verification.• It uses a nonintrusive verification.• Diagnostic mode seeks to establish a reason for the failure

of any verification task.• You can generate fixup scripts with some CVU commands

by using the -fixup flag.

Cluster Verify: OverviewCluster Verification Utility (CVU) is provided with Oracle Clusterware and Oracle Database 10g Release 2 (10.2) and later with the RAC option. The purpose of CVU is to enable you to verify during setup and configuration that all components required for a successful installation of Oracle Grid Infrastructure or Oracle Grid Infrastructure and a RAC database are installed and configured correctly, and to provide you with ongoing assistance any time that you need to make changes to your cluster or RAC database.There are two types of CVU commands:

• Stage commands are CVU commands used to test system setup and readiness for successful software installation, database creation, or configuration change steps. These commands are also used to validate successful completion of specific configuration steps.

• Component commands are CVU commands used to check individual cluster components and determine their state.

In addition, you can use CVU to verify a particular component while the stack is running or to isolate a cluster subsystem for diagnosis. During the diagnostic mode of operation, CVU tries to establish a reason for the failure of any verification task to help diagnose a problem. CVU can generate fixup scripts that perform system configuration needed for a successful installation in addition to identifying system issues that can cause installation failures. CVU provides additional checks to address installation, configuration, and operational issues.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Cluster Verify Components

• An individual subsystem or a module of the RAC cluster is known as a component in CVU.

• The availability and integrity of a cluster component can be verified.

• Various components—some simple such as a specific storage device, others complex such as the Oracle Clusterware stack—include:– Space availability– Shared storage

accessibility– Node connectivity– Cluster File System integrity– Oracle Clusterware integrity $ cluvfy comp -list

– Cluster integrity– Administrative

privileges– Peer compatibility– System requirements

Cluster Verify ComponentsCVU supports the notion of component verification. The verifications in this category are not associated with any specific stage. A component can range from basic, such as free disk space, to complex (spanning over multiple subcomponents), such as the Oracle Clusterware stack. Availability, integrity, or any other specific behavior of a cluster component can be verified. You can list verifiable CVU components with the cluvfy comp -list command:nodereach Checks node reachability peer Compares properties with peersnodecon Checks node connectivity ha Checks HA integritycfs Checks CFS integrity asm Checks ASM integrityssa Checks shared storage acfs Checks ACFS integrityspace Checks space availability olr Checks OLR integritysys Checks minimum requirements gpnp Checks GPnP integrityclu Checks cluster integrity gns Checks GNS integrityclumgr Checks cluster manager integrity scan Checks SCAN configurationocr Checks OCR integrity ohasd Checks OHASD integrityadmprv Checks administrative privileges crs Checks CRS integritysoftware Checks software distribution vdisk Checks Voting Disk Udev settingsclocksync Checks clock synchronizationnodeapp Checks node applications’ existence


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Cluster Verify Locations

• Download it from OTN:– Create a local directory.– Copy and extract cvu_<OS>.zip.– Use the most recent version.

• Oracle software DVD:– Disk1 directory– runcluvfy.sh

• Grid Infrastructure home:– <Grid_Home>/bin/cluvfy

• Oracle home:– $ORACLE_HOME/bin/cluvfy

Cluster Verify LocationsCluster Verification Utility (CVU) was first released in Oracle Clusterware Release 10.2.0.1.0. CVU supports 11gR2, 11gR1, 10gR2, as well as 10gR1 for Oracle Clusterware and RAC products. CVU is available in three different forms:

• Available on Oracle Technology Network (OTN) at: http://www.oracle.com/technology/products/database/clustering/cvu/cvu_download_homepage.html From there, you need to download the package and unzip it to a local directory (<cvhome>). You can use the cluvfy command from <cvhome>/bin. Optionally, you can set the CV_DESTLOC environment variable. This should point to a writable area on all nodes. CVU attempts to copy the necessary bits as required to this location. If this variable is not set, CVU uses /tmp as the default.

• Available in 11.2 Oracle software DVD as a packaged version. Make use of runcluvfy.sh, which is needed when nothing is installed. You can find it in Disk1.

• Installed in both 11.2 Oracle Clusterware and RAC homes. Make use of cluvfy if the CRS software stack is installed. If the CRS software is installed, you can find cluvfyunder <Grid_Home>/bin.

Note: For manual installation, you need to install CVU on only one node. CVU deploys itself on remote nodes during executions that require access to remote nodes.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Cluster Verify Configuration File

$ cat cvu_config# Configuration file for Cluster Verification Utility(CVU)# Version: 011405##If CRS home is not installed, this list will be#picked up when -n all is mentioned in the commandline argument.#CV_NODE_ALL=

#if enabled, cvuqdisk rpm is required on all nodesCV_RAW_CHECK_ENABLED=TRUE

# Fallback to this distribution idCV_ASSUME_DISTID=OEL4

# Whether X-Windows check should be performed for SSH user equivalence #CV_XCHK_FOR_SSH_ENABLED=TRUE

# To override SSH location#ORACLE_SRVM_REMOTESHELL=/usr/bin/ssh

# To override SCP location#ORACLE_SRVM_REMOTECOPY=/usr/bin/scp

Cluster Verify Configuration FileYou can use the CVU’s configuration file to define specific inputs for the execution of CVU. The path for the configuration file is $CV_HOME/cv/admin/cvu_config. The following is the list of keys supported in cvu_config:• CV_ORACLE_RELEASE: This is the version of the software to check against.• CV_NODE_ALL: If set, this specifies the list of nodes that should be picked up when

Oracle Clusterware is not installed and the -n all option has been used in the command line.

• CV_RAW_CHECK_ENABLED: If set to TRUE, this enables the check for accessibility of shared SCSI disks on Red Hat Release 3.0 and later. This shared disk accessibility check requires that you install a cvuqdisk rpm on all the nodes. By default, this key is set to TRUE and shared disk check is enabled.

• CV_ASSUME_DISTID: This specifies the distribution ID that CVU uses. For example, to make CVU working with SuSE 9 ES, set it to Pensacola.

• CV_XCHK_FOR_SSH_ENABLED: If set to TRUE, this enables the X-Windows check for verifying user equivalence with ssh. By default, this entry is commented out and X-Windows check is disabled.

• CV_TRACELOC: To choose the location in which CVU generates the trace files, set this environment variable to the absolute path of the desired trace directory.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Cluster Verify Configuration File (continued)• ORACLE_SRVM_REMOTESHELL: If set, this specifies the location for the ssh/rsh

command to override CVU’s default value. By default, this entry is commented out and the tool uses /usr/sbin/ssh and /usr/sbin/rsh.

Note: If CVU does not find a key entry defined in the configuration file, it searches for the environment variable that matches the name of the key; otherwise, it uses a default.• ORACLE_SRVM_REMOTECOPY: If set, this specifies the location for the scp or rcp

command to override the CVU default value. By default, this entry is commented out and CVU uses /usr/bin/scp and /usr/sbin/rcp.

If CVU does not find a key entry defined in the configuration file, it searches for the environment variable that matches the name of the key. If the environment variable is set, CVU uses its value. Otherwise, it uses a default value for that entity.To provide CVU with a list of all the nodes of a cluster, you can use the -n all option while executing a command.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Cluster Verify Output: Example

$ cluvfy comp crs -n all -verbose

Verifying CRS integrity

Checking CRS integrity...The Oracle clusterware is healthy on node "host03"The Oracle clusterware is healthy on node "host02"The Oracle clusterware is healthy on node "host01"

CRS integrity check passed

Verification of CRS integrity was successful.

Cluster Verify Output: ExampleThe slide shows you the output of the cluvfy comp crs –n all –verbosecommand. This command checks the complete Oracle Clusterware stack.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Enabling Resource Debugging

• Change the USR_ORA_DEBUG resource attribute to 1 for specific resources:

• After you capture all trace information, change the debug attribute back to 0:

• You can use an initialization file to configure debugging.• The initialization file name includes the name of the

process that you are debugging (process_name.ini). • The file is located in the

<Grid_Home>/log/host_name/admin/ directory.

# crsctl set log res "ora.host01.vip:0"

# crsctl set log res "ora.host01.vip:1"

Enabling Resource DebuggingOracle Support may request that you enable tracing to capture additional information for problem resolution with Oracle Clusterware resources. Because the procedures described here may affect performance, perform these activities only with the assistance of Oracle Support.You can enable debugging for Oracle Clusterware resources by issuing crsctl set logand crsctl set trace commands, using the following syntax:crsctl set {log | trace} resource “resource_name=debugging_level”

Run the crsctl set command as the root user, and supply the following information:• resource_name: The name of the resource to debug.• debugging_level: A number from 1 to 5 to indicate the level of detail you want the

debug command to return, where 1 is the least amount of debugging output and 5 provides the most detailed debugging output.

You can dynamically change the debugging level in the crsctl command or you can configure an initialization file for changing the debugging levelThe example in the slide enables debugging for the ora.host01.vip resource. After you capture all trace information, do not forget to execute the corresponding crsctl set log res "<resource name>:0" commands.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Enabling Resource Debugging (Continued)Another method of enabling resource debugging is the use of an initialization file. This debugging information is stored for use during the next startup. For each process to debug, you can create an initialization file that contains the debugging level.The initialization file name includes the name of the process that you are debugging (process_name.ini). The file is located in the <Grid_Home>/log/host_name/admin/directory. For example, the name for the CLSCFG debugging initialization file on node1 would be: <Grid_Home>/log/node1/admin/clscfg.ini


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Dynamic Debugging

crsctl

lsmodules debug

log trace statedump

csscrs evm

# crsctl set log crs "CRSEVT=1","CRSCOMM=2","CRSTIMER=5"

$ crsctl lsmodules cssThe following are the Cluster Synchronization Services modules:

CSSD COMCRS COMMNS CLSF SKGFD

# crsctl set log res "myResource1=1"

crsctl set log {crs | css | evm} "component_name=debugging_level,[...]"

Dynamic DebuggingYou can use crsctl set log commands as the root user to enable dynamic debugging for Cluster Ready Services (CRS), Cluster Synchronization Services (CSS), and the Event Manager (EVM), and Clusterware subcomponents. Debugging information remains in the Oracle Cluster Registry (OCR) for use during the next startup. You can also enable debugging for resources. The crsctl lsmodules crs| css|evm commands are used to list the module’s components that can be used for debugging. The example in the slide lists the ones for CSS. Nondefault logging levels are best interpreted with access to the source code and are designed for Oracle Support needs. When asked by Oracle Support, you can use commands such as the following to enable additional logging:• crsctl set log <module name> "<component>=<debugging level>",

where <module name> is the name of the module, CRS, EVM, or CSS; <component name> is the name of the corresponding component obtained using the crsctl lsmodules command, and <debugging level> is a level from 0 to 5

• crsctl debug statedump crs|css|evm, which dumps state information for crs, css, or evm modules

• crsctl set log res "<res_name>=<debugging_level>“, which sets logging for the specified resource

The example in the slide shows you how to dynamically enable additional logging (level 5) for the following CRS components: CRSEVT, CRSAPP, CRSTIMER, and CRSRES.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Dynamic Debugging

Component Name DescriptionCRSUI User interface moduleCRSCOMM Communication moduleCRSRTI Resource management moduleCRSMAIN Main module/driverCRSPLACE CRS placement moduleCRSAPP CRS applicationCRSRES CRS resourcesCRSOCR Oracle Cluster Registry interfaceCRSTIMER Various timers related to CRSCRSEVT CRS EVM/event interface moduleCRSD CRS daemon

Oracle Clusterware (CRS) Components

Dynamic Debugging (Continued)The table above describes the functions performed by each CRS component.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Enabling Tracing for Java-Based Tools

To enable tracing for cluvfy, netca, and srvctl, set SRVM_TRACE to TRUE:

$ export SRVM_TRACE=TRUE$ srvctl config database –d orcl > /tmp/srvctl.trc$ cat /tmp/srvctl.trc...[main] [ 2009-09-16 00:58:53.197 EDT ] [CRSNativeResult.addRIAttr:139] addRIAttr: name 'ora.orcl.db 3 1', 'USR_ORA_INST_NAME@SERVERNAME(host01)':'orcl1'[main] [ 2009-09-16 00:58:53.197 EDT ] [CRSNativeResult.addRIAttr:139] addRIAttr: name 'ora.orcl.db 3 1', 'USR_ORA_INST_NAME@SERVERNAME(host02)':'orcl2'[main] [ 2009-09-16 00:58:53.198 EDT ] [CRSNativeResult.addRIAttr:139] addRIAttr: name 'ora.orcl.db 3 1', 'USR_ORA_INST_NAME@SERVERNAME(host03)':'orcl3'[main] [ 2009-09-16 00:58:53.198 EDT ] [CRSNative.searchEntities:857] found 3 ntitie...

Enabling Tracing for Java-Based ToolsAll Java-based tools and utilities that are available in RAC are invoked by executing scripts of the same name as the tool or utility. This includes the Cluster Verification Utility (cluvfy), the Database Configuration Assistant (dbca), the Database Upgrade Assistant (dbua), the Net Configuration Assistant (netca), and Server Control (srvctl). For example, to run the Database Configuration Assistant, enter the dbca command.By default, Oracle enables traces for dbca and dbua. The resulting log files are written to $ORACLE_HOME/cfgtoollogs/dbca/ and $ORACLE_HOME/cfgtoollogs/dbua, respectively. For cluvfy and srvctl, you can set the SRVM_TRACE environment variable to TRUE to make the system generate traces. Traces are written to either log files or standard output. For example, the system writes traces to log files in <Grid_Home>/cv/log/ for cluvfy. However, as shown in the slide, it writes traces directly to the standard output for srvctl.To disable tracing for Java-based tools, unset the SRVM_TRACE variable:export SRVM_TRACE=


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Preserving Log Files Before Wrapping

• Write a shell script to copy log files before they wrap.

• Execute the script created in the background.

# Script to archive log files before wrapping occurs

# Written for CSS logs. Modify for other log file types.

CSSLOGDIR=/u01/app/11.2.0/grid/log/host01/cssd

while [ 1 –ne 0 ]; do

CSSFILE=/tmp/css_`date +%m%d%y"_"%H%M`.tar

tar -cf $CSSFILE $CSSLOGDIR/*

sleep 300

done

exit

# chmod 755 archscript.sh; nohup ./archscript.sh &

Preserving Log Files Before WrappingDepending on the level of debugging that has been enabled, many of the log files are capable of wrapping, which causes only a limited amount of time-stamp information to be available for debugging. To prevent the loss of information, create a script as illustrated in the slide that will copy all the log files before wrapping can occur. Execute the script in the background to capture the logs while debugging is being used. When it is no longer necessary to capture log files, kill the archiving script that is running. This technique can be applied to any log file for which it is needed. The logs for CSS are shown only as an example. Oracle Clusterware logs already have a log rotation mechanism to save logs, but with excessive debug levels turned on, they could still wrap and lose information.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Process Roles for Node Reboots

The following processes can evict nodes from the cluster or cause a node reboot:• oclskd: Is used by CSS to reboot a node based on

requests from other nodes in the cluster• cssdagent and cssmonitor: Monitor node hands and

vendor clusterware• ocssd: Monitors the internode’s health status• hangcheck-timer: Monitors for machine hangs and

pauses– Not required for Oracle Clusterware 11g Release 2

Process Roles for Node RebootsOracle Clusterware is designed to perform a node eviction by removing one or more nodes from the cluster if some critical problem is detected. A critical problem could be a node not responding via a network heartbeat, a node not responding via a disk heartbeat, a hung or severely degraded machine, or a hung ocssd.bin process. The purpose of this node eviction is to maintain the overall health of the cluster by removing suspect members.CSS uses the Oracle Clusterware Kill Daemon (oclskd) to stop processes associated with CSS group members for which stop requests have come.The cssdagent process is spawned by ohasd and is responsible for spawning the ocssdprocess, monitoring for node hangs, and monitoring the ocssd process for hangs, and monitoring vendor clusterware (via vmon functionality). This is a multi-threaded process that runs at an elevated priority and runs as the root user.The ocssd process is spawned by the cssdagent process. It runs in both vendor clusterware and non-vendor clusterware environments. OCSSD's primary job is inter-node health monitoring and RDBMS instance endpoint discovery. The health monitoring includes a network heartbeat and a disk heartbeat (to the voting files). The ocssd process can also evict a node after escalation of a member kill from a client (such as a database LMON process). This is a multi-threaded process that runs at an elevated priority and runs as the Oracle user.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Process Roles for Node Reboots (Continued)The cssdmonitor daemon monitors the ocssd daemon for hangs or scheduling issues and can reboot a node if there is a perceived hang. If the ocssd daemon is lost, the node will be rebooted.The hangcheck-timer module is a kernel module on Linux systems that monitors for hangs and pauses where the system resumes after some time, but the clock has not noticed the system’s lost time. If a threshold of lost time is exceeded, the module will reboot the node. Note: While the hangcheck-timer module is still required for Oracle Database 11g Release 1RAC databases, it is no longer needed for Oracle Database 11g Release 2 RAC.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Determining Which Process Caused Reboot

Log File Locations for Processes Causing Reboots.• oclskd

– <Grid_Home>/log/<hostname>/client/oclskd.log

• ocssd

– /var/log/messages

– <Grid_Home>/log/<hostname>/cssd/ocssd.log

• cssdagent

– <Grid_Home>/log/<hostname>/agent/ohasd/oracssdagent_root

• cssdmonitor

– <Grid_Home>/log/<hostname>/agent/ohasd/oracssdmonitor_root

• hangcheck-timer

– /var/log/messages

Determining Which Process Caused RebootFirst, determine the time of the node reboot by using the uptime command and subtracting the up time from the current system time. The reboot time will be used when examining log files. When the OCSSD daemon is responsible for rebooting a node, a message similar to “Oracle CSSD failure. Rebooting for cluster integrity” is written into the system messages log at /var/log/messages. The cssd daemon log file that is located at <Grid_Home>/log/<hostname>/cssd/ocssd.log may also contain messages similar to "Begin Dump" or "End Dump" just before the reboot.If hangcheck-timer is being used, it will provide message logging to the system messages log when a node restart is initiated by the module. To verify whether this process was responsible for the node reboot, examine the /var/log/messages file and look for an error message similar to: "Hangcheck: hangcheck is restarting the machine."Other useful log files include the Clusterware alert log in <Grid_home>/log/<hostname>and the lastgasp log in /etc/oracle/lastgasp or /var/opt/oracle/lastgasp.If no indication of which process caused the reboot can be determined from these files,

additional debugging and tracing may need to be enabled.Note: The oclsomon and the oprocd background processes have been eliminated in Oracle Database 11g Release 2.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Using ocrdump to View Logical Contents of the OCR

• To dump the OCR contents into a text file for reading:

• To dump the OCR contents for a specific key:

• To dump the OCR contents to stdout in XML format:

• To dump the contents of an OCR backup file:

[root]# ocrdump filename_with_full_results.txt

[grid]$ ocrdump filename_with_limited_results.txt

# ocrdump –keyname SYSTEM.language

# ocrdump –stdout -xml

# ocrdump –backupfile week.ocr

Using ocrdump to View Logical Contents of the OCRThe ocrdump utility can be used to view the OCR content for troubleshooting. The ocrdump utility enables you to view logical information by writing the contents to a file or displaying the contents to stdout in a readable format. If the ocrdump command is issued without any options, the default file name of OCRDUMPFILE will be written to the current directory, provided that the directory is writable. The information contained within the OCR is organized by keys that are associated with privileges. Therefore, the root user will not see the same results as the clusterware owner. Consider the following:As root, ocrdump –stdout | wc –l results in 3355 lines on a test system, andas grid, ocrdump –stdout | wc –l results in 521 lines on the same system.The number of lines of information in the OCR depends on many factors such as number of nodes in the cluster and number of resources registered in the OCR. Your numbers are expected to be different.To determine all the changes that have occurred in the OCR over the previous week, locate the automatic backup from the previous week and compare it to a dump of the current OCR as follows:# ocrdump

# ocrdump –stdout –backupfile week.ocr | diff - OCRDUMPFILE


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Checking the Integrity of the OCR

Use the ocrcheck command to check OCR integrity.

# ocrcheckStatus of Oracle Cluster Registry is as follows :

Version : 2Total space (kbytes) : 275980Used space (kbytes) : 2824Available space (kbytes) : 273156ID : 1274772838Device/File Name : +DATA1

Device/File integrity check succeededDevice/File Name : +DATA2

Device/File integrity check succeededCluster registry integrity check succeededLogical corruption check succeeded

Checking the Integrity of the OCRThe ocrcheck utility displays the version of OCR’s block format, total space available and used space, OCRID, and the OCR locations that you have configured. The ocrcheck utility performs a block-by-block checksum operation for all the blocks in all the OCRs that you have configured. It also returns an individual status for each file as well as a result for the overall OCR integrity check.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


OCR-Related Tools for Debugging

• OCR tools:– ocrdump

– ocrconfig

– ocrcheck

– srvctl

• Logs are generated in the following directory: <Grid_Home>/log/<hostname>/client/

• Debugging is controlled through the following file: <Grid_Home>/srvm/admin/ocrlog.ini

mesg_logging_level = 5comploglvl="OCRAPI:5 ; OCRSRV:5; OCRCAC:5; OCRMAS:5; OCRCONF:5; OCRRAW:5"comptrclvl="OCRAPI:5 ; OCRSRV:5; OCRCAC:5; OCRMAS:5; OCRCONF:5; OCRRAW:5"

OCR-Related Tools for DebuggingAs you have seen, you can use various tools to manipulate the OCR: ocrdump, ocrconfig, ocrcheck, and srvctl.These utilities create log files in <Grid_Home>/log/<hostname>/client/. To change the amount of logging, edit the <Grid_Home>/srvm/admin/ocrlog.ini file. The default logging level is 0, which basically means minimum logging. When mesg_logging_level is set to 0, which is its default value, only error conditions are logged. You can change this setting to 3 or 5 for detailed logging information.If that is not enough, you can also change the logging and trace levels for each of the components used to manipulate the OCR. To do that, edit the entries containing comploglvl and comptrclvl in ocrlog.ini. The slide shows you the three lines you could add to ocrlog.ini to turn on additional debugging information for some OCR components. A typical example where you may have to change the ocrlog.ini file is in a situation where you get errors while using either the ocrdump or ocrconfig tool.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


OCR-Related Tools for Debugging

Component Name DescriptionOCRAPI OCR abstraction componentOCRCLI OCR client componentOCRSRV OCR server componentOCRMAS OCR master thread componentOCRMSG OCR message componentOCRCAC OCR cache componentOCRRAW OCR raw device componentOCRUTL OCR util componentOCROSD OCR operating system dependent (OSD) layerOCRASM OCR ASM component

Oracle Cluster Registry (OCR) Components

OCR-Related Tools for Debugging (Continued)Using an initialization file or the crsctl set log crs command, you can debug the OCR components listed in the table above. The components listed can also be used for the Oracle Local Registry (OLR) except for OCRMAS and OCRASM. You can also use them for the OCR and OLR clients, except for OCRMAS and OCRSRV. Some of the OCR and OLR clients are ocrconfig, ocrdump, ocrcheck, and so on.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Browsing My Oracle Support Knowledge Articles

Browsing My Oracle Support Knowledge ArticlesMy Oracle Support is the next generation of Oracle’s award-winning support platform integrated with Software Configuration Manager for embedding configuration management capabilities directly into the support processes. Simply by installing the configuration manager and uploading your configuration information, you can enable Oracle’s personalized and proactive support capabilities, driven by Oracle’s vast knowledge base. This proactive advice includes customized security and product health checks that identify problems and recommend fixes before they impact your system performance. One individual in your organization should be designated as a Oracle support administrator that can create additional access accounts for other members in your organization. After signing in at http://support.oracle.com, knowledge articles may be accessed to assist troubleshooting Oracle Clusterware.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Quiz

Configuration of the hangcheck-timer module is required for Grid Infrastructure 11g Release 2 clusters.1. True2. False

Answer: 2False. The hangcheck-timer module is no longer needed in Oracle Grid Infrastructure 11g Release 2 clusters.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Quiz

Which of the following statements regarding cluvfy are true?1. You use it to perform a full stack verification.2. It uses nonintrusive verification methodology.3. It works only on clusters employing ASM for shared

storage.4. You can generate fixup scripts with some CVU commands

by using the -fixup flag.

Answer: 1,2,4Statements 1, 2, and 4 are correct.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Summary

In this lesson, you should have learned how to:• Locate Oracle Clusterware log files• Gather all log files using diagcollection.pl• Enable resource debugging• Enable component-level debugging• Enable tracing for Java-based tools• Troubleshoot the Oracle Cluster Registry (OCR) file


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Practice 6 Overview

This practice covers the following topics:• Working with log files• Working with ocrdump• Working with cluvfy


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Administering ASM Instances

Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Objectives

After completing this lesson, you should be able to:• Explain and apply Automatic Storage Management (ASM)

initialization parameters• Manage ASM instances and associated processes• Monitor ASM using the V$ASM dynamic performance views


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


ASM Initialization Parameters

• ASM initialization parameters can be set:– After installation, by using ALTER SYSTEM or ALTER

SESSION SQL statements.• INSTANCE_TYPE=ASM is the only mandatory parameter

setting.• There are a number of ASM-specific parameters.

– These have names starting with ASM_.• Some database parameters are valid for ASM.

– For example, MEMORY_TARGET• You can use a PFILE or SPFILE to manage parameters.

ASM Initialization ParametersYou use Enterprise Manager or SQL ALTER SYSTEM or ALTER SESSION statements to change the initialization parameters for an ASM instance after installation.There are three groupings of parameters for an ASM instance: mandatory, ASM-only, and parameters that are also valid for the database. INSTANCE_TYPE is the only mandatory parameter for ASM. ASM-only parameters have names that are prefixed with ASM_. These have suitable defaults for most environments. Some database initialization parameters are also valid for ASM (for example, MEMORY_TARGET). In general, ASM uses appropriate defaults for relevant database parameters. Oracle strongly recommends that you use a server parameter file (SPFILE) as the ASM instance parameter file. The SPFILE is maintained by the instance, so it must be in a shared location. In a clustered ASM environment, SPFILE is placed by default in an ASM disk group; it could also be located on a cluster file system. You can use a text initialization parameter file (PFILE) with PFILEs on each cluster node, but these must be maintained manually and kept synchronized.The rules for file name, default location, and search order that apply to database initialization parameter files also apply to ASM initialization parameter files. The search order is: the location set in the Grid Plug and Play profile, then for an SPIFILE in the Oracle home, finally for a PFILE in the Oracle home. The default location for a PFILE for the +ASM1 instance is:

$ORACLE_HOME/dbs/spfile+ASM1.ora


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


ASM_DISKGROUPS

ASM_DISKGROUPS specifies a list of disk group names that ASM automatically mounts at instance startup.• Uses the default value of NULL• Is ignored in certain circumstances:

– If ASM is started with the NOMOUNT option– If all disk groups are explicitly mounted

• Can be set dynamically using ALTER SYSTEM• Is automatically modified when disk groups are added,

deleted, mounted, or unmounted if using an SPFILE• Must be manually adjusted if using a PFILE

– Except when ASMCA is used to create a new disk group

ASM_DISKGROUPS

The ASM_DISKGROUPS initialization parameter specifies a list of disk group names that ASM automatically mounts at instance startup. The default value of the ASM_DISKGROUPSparameter is a NULL string. ASM_DISKGROUPS is ignored when you specify the NOMOUNT option at instance startup or when you issue the ALTER DISKGROUP ALL MOUNT statement.The ASM_DISKGROUPS parameter is dynamic. The following is an example of setting the ASM_DISKGROUPS parameter dynamically:

ALTER SYSTEM SET ASM_DISKGROUPS = 'DATA, LOG, STANDBY'

When an SPFILE is used, ASM_DISKGROUPS is modified when a disk group is mounted, dismounted, created, or dropped. When you use ASMCMD or SQLPLUS to mount a disk, the disk is mounted only on the local node and the ASM_DISKGROUPS is modified only for the local instance. When you use ASMCA to mount a disk group, you have a choice to mount locally or on all nodes. ASM_DISKGROUPS is modified in accordance with your choice. With Enterprise Manager, you are given a choice of which nodes to mount and dismount. EM will operate only on the nodes where there is an agent present.When using a PFILE, you must edit the initialization parameter file to add or remove disk groups names. The following is an example of the ASM_DISKGROUPS parameter in a PFILE:

ASM_DISKGROUPS = 'DATA', 'LOG', 'STANDBY'


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Disk Groups Mounted at Startup

At startup, the Oracle ASM instance attempts to mount the following disk groups:• Disk groups specified in the ASM_DISKGROUPS

initialization parameter• Disk group used by Cluster Synchronization Services

(CSS) for voting files• Disk groups used by Oracle Clusterware for the Oracle

Cluster Registry (OCR)• Disk group used by the Oracle ASM instance to store the

ASM server parameter file (SPFILE)

Disk Groups Mounted at StartupAt startup, the Oracle ASM instance attempts to mount the following disk groups:

• Disk groups specified in the ASM_DISKGROUPS initialization parameter• Disk group used by Cluster Synchronization Services (CSS) for voting files• Disk groups used by Oracle Clusterware for the Oracle Cluster Registry (OCR)• Disk group used by the Oracle ASM instance to store the ASM server parameter file

(SPFILE)If no disk groups are found in the previous list, the Oracle ASM instance does not mount any disk groups at startup.In your practice cluster, the DATA disk group is mounted because it has the OCR, voting files, and SPFILE for the ASM instance in it. The FRA and ACFS disk groups are mounted at startup only when they are named in the ASM_DISKGROUPS initialization parameter.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


ASM_DISKSTRING

ASM_DISKSTRING specifies a list of strings that limits the set of disks that an ASM instance discovers.• Uses the default value of NULL

– Meaning ASM searches a default path looking for disks that it has read-and-write access to

— Default search path is platform specific.— Default search path includes Oracle ASMLib disks.

• Can use * and ? as wildcards• Can be set dynamically using ALTER SYSTEM

– However, a change will be rejected if the proposed value cannot be used to discover all the disks belonging to the currently mounted disk groups

ASM_DISKSTRING

The ASM_DISKSTRING initialization parameter specifies a comma-delimited list of strings that limits the set of disks that an ASM instance discovers. Only disks that match one of the strings are discovered.The discovery strings can include wildcard characters. The * character is the wildcard for zero or more characters. The ? character, when used as the first character of a path, expands to the ORACLE_HOME directory. Depending on the operating system, when you use the ? character elsewhere in the path, it is a wildcard for one character. On Linux and UNIX-based systems, there are additional wildcard character patterns that can be used. The discovery string format depends on the ASM library and the operating system that are in use. For example, on a Linux server that does not use ASMLib, to limit the discovery process to include only disks that match the /dev/sd* pattern, set ASM_DISKSTRING = /dev/sd*

To limit the discovery process to include only disks that have a name that ends in 3 or 4, set ASM_DISKSTRING = /dev/sd*3, /dev/sd*4.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

ASM_DISKSTRING (continued)The default value of the ASM_DISKSTRING parameter is a NULL string. A NULL value causes ASM to search a default path for all disks in the system to which the ASM instance has read-and-write access. The default search path is platform specific. The following list shows the default search path for a selection of common platforms:Linux: /dev/raw/*AIX: /dev/rhdisk* HP-UX: /dev/rdisk/* Solaris: /dev/rdsk/* Windows: \\.\ORCLDISKnNote that the default value of ASM_DISKSTRING might not find all disks in all situations. If your installation uses multipathing software, the software might place pseudodevices in a path that is different from the operating system default. If your site is using a third-party vendor ASMLib, the vendor might have discovery string conventions that you must use for ASM_DISKSTRING. Note: Because ASMLib is open source, the possibility exists that there will be vendors other than Oracle that provides ASMLib. If Oracle ASMLib is in use, the default search patch also includes all disks defined to ASMLib. The default path that specifies all Oracle ASMLib disks is ORCL:*.If ASM is used in conjunction with an Oracle Exadata Storage Server, add the global search path o/*/* to the ASM_DISKSTRING parameter to ensure discovery of the Exadata Grid Disks. Another examples of disk discovery with Oracle Exadata Storage Server is: o/192.168.234.*/data*, which would find all grid disks beginning with data on a range of IP addresses. The ASM_DISKSTRING parameter is dynamic and can be set using the ALTER SYSTEMcommand. An attempt to dynamically modify ASM_DISKSTRING will be rejected and the old value retained if the new value cannot be used to discover a disk that is in a disk group that is already mounted. Note: ASM cannot use a disk unless all the ASM instances in the cluster can discover the disk through one of their own discovery strings. The names do not need to be the same on every node, but all disks must be discoverable by all the nodes in the cluster. This may require dynamically changing the initialization parameter to enable adding new storage. Dynamically changing this parameter requires the SQL command:To change the disk discovery string on all nodes of the cluster:ALTER SYSTEM SET asm_diskstring=value sid=*

To change the disk discovery string on the +ASM1 instance only:ALTER SYSTEM SET asm_diskstring=value sid=+ASM1


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


ASM_POWER_LIMIT

The ASM_POWER_LIMIT initialization parameter specifies the default power for disk rebalancing.• Default is 1.

– Meaning ASM conducts rebalancing operations using minimal system resources

• Allowable range is 0 to 11.– 0 disables rebalancing operations.– Lower values use fewer system resources but result in

slower rebalancing operations.– Higher values use more system resources to achieve faster

rebalancing operations.• It can be set dynamically using ALTER SYSTEM

or ALTER SESSION.

ASM_POWER_LIMIT

The ASM_POWER_LIMIT initialization parameter specifies the default power for disk rebalancing. The default value is 1 and the range of allowable values is 0 to 11. A value of 0disables rebalancing. Higher numeric values enable the rebalancing operation to complete more quickly, but at the cost of greater system resource usage and I/O overhead. Each instance can have a different value for ASM_POWER_LIMIT.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


INSTANCE_TYPE

INSTANCE_TYPE specifies whether the instance is a database instance or an ASM instance.• Set INSTANCE_TYPE = ASM for an ASM instance.• This is the only mandatory parameter setting for ASM.

INSTANCE_TYPE

The INSTANCE_TYPE initialization parameter must be set to ASM for an ASM instance. This is a required parameter and cannot be modified. The following is an example of the INSTANCE_TYPE parameter in the initialization file:

INSTANCE_TYPE = ASM


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


CLUSTER_DATABASE

CLUSTER_DATABASE specifies whether or not storage clustering is enabled.• Set CLUSTER_DATABASE = TRUE for multiple ASM

instances to access the same ASM disks concurrently.• Clustered ASM can support clustered (RAC) and single-

instance (non-RAC) databases.

CLUSTER_DATABASE

When referring to ASM, the CLUSTER_DATABASE instance parameter specifies whether or not storage clustering is enabled. You must set CLUSTER_DATABASE = TRUE if you want multiple clustered ASM instances to access the same ASM disks concurrently.Note that you can use a clustered ASM installation to support both Real Application Clusters (RAC) and single-instance databases.Note: All the instances in a cluster must have the same setting.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


MEMORY_TARGET

MEMORY_TARGET specifies the total memory used by an ASM instance.• Oracle strongly recommends that you use automatic

memory management for ASM.• All other memory-related instance parameters are

automatically adjusted based on MEMORY_TARGET.• The default value of 272 MB is suitable for most

environments.• The minimum value for MEMORY_TARGET is 256 MB.

– Values smaller than 256 MB are ignored.• It can be increased dynamically using ALTER SYSTEM.

MEMORY_TARGET

Oracle strongly recommends that you use automatic memory management (AMM) for ASM. Automatic memory management automatically manages the memory-related parameters for ASM instances with the MEMORY_TARGET parameter. AMM is enabled by default on ASM instances, even when the MEMORY_TARGET parameter is not explicitly set. The default value used for MEMORY_TARGET (272 MB) is acceptable for most environments. This is the only parameter that you need to set for complete ASM memory management. You can also increase MEMORY_TARGET dynamically, up to the value of the MEMORY_MAX_TARGET parameter, just as you can for a database instance. Note: For Linux environments, automatic memory management will not work if /dev/shmis not available or is sized smaller than MEMORY_TARGET. For Enterprise Linux Release 5, /dev/shm is configured to be half the size of the system memory by default. You can adjust this by adding a size option to the entry for /dev/shm in /etc/fstab. For more details, see the man page for the mount command.Note: The minimum MEMORY_TARGET for ASM is 256 MB in the SPFILE. If you set MEMORY_TARGET to a lower value, Oracle Database increases the value to 256 MBautomatically.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Adjusting ASM Instance Parameters in SPFILEs

• The server parameter file (SPFILE) is a binary file that cannot be edited using a text editor.

• Use Oracle Enterprise Manager or the ALTER SYSTEM SQL command to adjust ASM instance parameter settings in an SPFILE.

• In a clustered ASM environment, SPFILEs should reside in ASM or a cluster file system.

SQL> ALTER SYSTEM SET ASM_DISKSTRING='ORCL:*'

2> SID='*' SCOPE=SPFILE;

Adjusting ASM Instance Parameters in SPFILEsYou can use a server parameter file (SPFILE) as the ASM instance parameter file. The server parameter file is a binary file that cannot be edited using a text editor. You can use Oracle Enterprise Manager or the ALTER SYSTEM SQL command to adjust ASM instance parameter settings in an SPFILE. For example, to adjust your SPFILE so that your ASM environment discovers only Oracle ASMLib disks, you could execute:

ALTER SYSTEM SET ASM_DISKSTRING='ORCL:*' SID='*' SCOPE=SPFILE;

Using SCOPE=SPFILE changes the parameter setting stored in the SPFILE only. It does not alter the current parameter setting for the running ASM instance. If the parameter can be altered dynamically, you can use SCOPE=MEMORY to adjust a parameter for a running instance or use SCOPE=BOTH (or omit the SCOPE clause) to dynamically adjust a parameter and save the change in the SPFILE. You can add an optional SID clause to specify that the setting is instance specific or use the default SID='*' to explicitly state that the parameter applies to all instances. For example, to adjust your SPFILE so that the +ASM1 instance uses a specific power-limit setting, you could execute:

ALTER SYSTEM SET ASM_POWER_LIMIT=5 SCOPE=SPFILE SID='+ASM1';

If you use an SPFILE in a clustered ASM environment, you should place the SPFILE in ASM, a shared Network-Attached Files (NFS) system, or on a cluster file system.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Starting and Stopping ASM Instancesby Using srvctl

The Sever Control utility (srvctl) can be used to start and stop ASM instances.• One node at a time:

• All nodes simultaneously:

$ srvctl start asm -n host01$ srvctl start asm -n host02$ srvctl status asm -n host01ASM is running on host01.$ srvctl status asm -n host02ASM is running on host02.

$ srvctl stop asm$ srvctl status asm -n host01ASM is not running on host01.$ srvctl status asmASM is not running on host01,host02.$

Starting and Stopping ASM Instances by Using srvctlThe Sever Control utility (srvctl) can be used to start and stop ASM instances along with other resources managed by Oracle Clusterware. srvctl can be found under the Grid_home/bin location established during the Oracle Clusterware installation. One of the major advantages of srvctl is that it allows you to easily start or stop all the ASM instances in your cluster from any node in the cluster. srvctl can be used to control ASM in the following ways:

• Start an ASM instance.srvctl start asm [-n <node>] [-o <start_option>]<node> is the name of the cluster node hosting the ASM instance (optional); if omitted, the

ASM instance will be started on all nodes.<asm_inst> is the name of the specific ASM instance being acted upon (optional).<start_option> is one of the valid instance startup options (FORCE, MOUNT, OPEN,

NOMOUNT, or RESTRICT) (optional).• Stop an ASM instance.

srvctl stop asm [-n <node>] [-o <stop_option>]<stop_option> is one of the valid instance shutdown options

(NORMAL, IMMEDIATE, TRANSACTIONAL, or ABORT) (optional).• Report the status of an ASM instance.

srvctl status asm [-n <node>]

Note: srvctl can also be used to control a nonclustered ASM instance.Oracle 11g: RAC and Grid Infrastructure Administration Accelerated 7 - 13

Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Starting and Stopping ASM Instances by Using SQL*Plus

Starting and stopping ASM instances using SQL*Plus is similar to the way in which you start and stop database instances.$ export ORACLE_SID=+ASM1$ export ORACLE_HOME=/u01/app/11.2.0/grid$ $ORACLE_HOME/bin/sqlplus / AS SYSASM

Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - ProductionWith the Real Application Clusters and Automatic Storage Management

options

SQL> startup ASM instance started

Total System Global Area 284565504 bytesFixed Size 1312896 bytesVariable Size 258086784 bytesASM Cache 25165824 bytesASM diskgroups mounted

SQL>

Starting and Stopping ASM Instances by Using SQL*PlusWith SQL*Plus, you start an ASM instance by using the STARTUP command, similar to the way in which you start an Oracle Database instance. When starting an ASM instance, note the following:

• To connect to an ASM instance with SQL*Plus, set the ORACLE_SID environment variable to the ASM SID. The default ASM SID for a single instance database is +ASMand the default SID for ASM for an Oracle RAC node is +ASMnode_number, where node_number is the number of the node.

• Set the ORACLE_HOME environment variable to the ASM installation location.• The initialization parameter file must contain the following entry:

INSTANCE_TYPE = ASM

This parameter indicates that an ASM instance, not a database instance, is starting.• When you run the STARTUP command, rather than trying to mount and open a database,

this command attempts to mount the disk groups specified by the initialization parameter ASM_DISKGROUPS and the rules for mounting disk groups. If no disk groups meet the rules, the ASM instance starts and ASM displays a message that no disk groups were mounted. You can later mount disk groups with the ALTER DISKGROUP...MOUNTcommand.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Starting and Stopping ASM Instances by Using SQL*Plus (continued)The following list describes the STARTUP command parameters relevant to ASM.• FORCE: Issues a SHUTDOWN ABORT to the ASM instance before restarting it• MOUNT or OPEN: Mounts the disk groups specified in the ASM_DISKGROUPS

initialization parameter. In an ASM instance, MOUNT and OPEN are synonymous. OPENis the default if no command parameter is specified.

• NOMOUNT: Starts up the ASM instance without mounting any disk groups• RESTRICT: Starts up an instance in restricted mode. The RESTRICT clause can be

used in combination with the MOUNT, NOMOUNT, and OPEN clauses.In restricted mode, database instances cannot use the disk groups. That is, databases cannot open files that are in that disk group. Also, if a disk group is mounted by an instance in restricted mode, that disk group cannot be mounted by any other instance in the cluster. Restricted mode enables you to perform maintenance tasks on a disk group without interference from clients. Rebalance operations that occur while a disk group is in restricted mode eliminate the lock and unlock extent map messaging that occurs between ASM instances in a clustered environment. This improves the overall rebalance throughput. At the end of a maintenance period, you must explicitly dismount the disk group and remount it in normal mode.

The ASM shutdown process is initiated when you run the SHUTDOWN command in SQL*Plus. Before you run this command, ensure that the ORACLE_SID and ORACLE_HOMEenvironment variables are set so that you can connect to the ASM instance.Oracle strongly recommends that you shut down all database instances that use the ASM instance before attempting to shut down the ASM instance.The following list describes the SHUTDOWN command parameters relevant to ASM.• NORMAL: ASM waits for any in-progress SQL to complete before dismounting all the

disk groups and shutting down the ASM instance. Before the instance is shut down, ASM waits for all the currently connected users to disconnect from the instance. If any database instances are connected to the ASM instance, the SHUTDOWN command returns an error and leaves the ASM instance running. NORMAL is the default shutdown mode.

• IMMEDIATE or TRANSACTIONAL: ASM waits for any in-progress SQL to complete before dismounting all the disk groups and shutting down the ASM instance. ASM does not wait for users currently connected to the instance to disconnect. If any database instances are connected to the ASM instance, the SHUTDOWN command returns an error and leaves the ASM instance running.

• ABORT: The ASM instance immediately shuts down without the orderly dismount of disk groups. This causes recovery to occur upon the next ASM startup. If any database instance is connected to the ASM instance, the database instance aborts.If any Oracle Automatic Storage Management Cluster File System (Oracle ACFS) file systems are currently mounted on Oracle ASM Dynamic Volume Manager (ADVM) volumes, those file systems should first be dismounted. Otherwise, applications will encounter I/O errors and Oracle ACFS user data and metadata may not be flushed to storage before the Oracle ASM storage is fenced.If there are connected ASM clients, the ASM instance will shut down only if the abortoption is used.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Starting and Stopping ASM Instances by Using ASMCA and ASMCMD

Starting and Stopping ASM Instances by Using ASMCA and ASMCMDThe ASMCA utility shown in the slide allows you to start and stop an ASM instance. The ASMCMD utility also includes the ability to start and start ASM instances with the following commands:

$ asmcmdASMCMD [+] > shutdown

ASMCMD [+] > shutdown --immediate

ASMCMD [+] > shutdown --abort

ASMCMD> startup --nomount --pfile asm_init.ora

ASMCMD> startup –-mount


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Starting and Stopping ASM Instances Containing Cluster Files

ASM instances containing the OCR and voting disks:• Will be automatically restarted by the high availability

services daemon• Use crsctl stop crs

Starting and Stopping ASM Instances Containing Cluster FilesASM instances that contain the OCR files or voting disk files will be automatically restarted by the high availability services daemon. ASM instances with connected clients cannot be shut down except with the abort option. Oracle Clusterware is a client of ASM when the OCR files and voting files are in ASM disk groups. Stopping the Oracle Clusterware services includes stopping ASM.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Starting and Stopping the Node Listener

• Using the lsnrctl utility:

• Using the srvctl utility:$ srvctl start listener -n host01$

$ lsnrctl start listenerLSNRCTL for Linux: Version 11.2.0.1.0 - Production on 08-OCT-2009

22:44:22Copyright (c) 1991, 2009, Oracle. All rights reserved.Starting /u01/app/11.2.0/grid/bin/tnslsnr: please wait...... Intermediate output removed ... The command completed successfully$

Starting and Stopping the Node ListenerA standard clustered ASM installation configures an Oracle network listener under the Grid home directory. This listener can be manually started and stopped using the lsnrctl utility installed as part of the ASM installation:

$ lsnrctl start listener$ lsnrctl stop listener

You can alternatively use the Server Control utility (srvctl) to start and stop the ASM listener as follows:

$ srvctl start listener –n <node>$ srvctl stop listener –n <node>

The lsnrctl and srvctl utilities exist in both the grid_home and RDBMS home directories; which one you use depends on where the listener configuration files reside. The Grid Infrastructure installation will start a listener with the configuration files in the grid_home. By default, the database installation will use that listener. In this case, set the ORACLE_HOME and PATH environment variables to use the grid_home and then run the utilities.If you create a new listener with configuration files in the RDBMS home, set the ORACLE_HOME and PATH environment variables to use the RDBMS home and then run the utilities.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


ASM Dynamic Performance Views

The ASM instance hosts memory-based metadata tables presented as dynamic performance views.• Is accessed by ASM utilities to retrieve metadata-only

information using the SQL language• Contains many dedicated ASM-related views such as:

The V$ASM_* views exist in both ASM and database instances. The rows returned will vary.

V$ASM_ALIAS V$ASM_ATTRIBUTE V$ASM_CLIENT

V$ASM_DISK V$ASM_DISK_IOSTAT V$ASM_DISK_STAT

V$ASM_DISKGROUP V$ASM_DISKGROUP_STAT V$ASM_FILE

V$ASM_OPERATION V$ASM_TEMPLATE V$ASM_ACFSVOLUME

V$ASM_FILESYSTEM

ASM Dynamic Performance ViewsAn ASM instance does not have a data dictionary. Instead, information relating to its configuration and operation is available by querying a set of dynamic performance views. The ASM dynamic performance views have names having the prefix V$ASM. They can be queried using SQL in the same manner as any database dynamic performance view. The slide lists the most common dynamic performance views that contain ASM-related metadata. The V$ASM views exist inside both ASM instances and database instances. If you query a V$ASM view while connected to an ASM instance, information relating to the ASM instance is returned. If you query a V$ASM view while connected to a database instance, some views return no information, whereas others return a subset of information relating to the use of ASM in that database instance.In a clustered environment, in addition to the V$ASM views, there is a corresponding set of GV$ASM views. The GV$ASM views provide a consolidated view of information across all thecurrently running clustered ASM instances. The structure of each GV$ASM view is the same as for its corresponding V$ASM view except for an additional column, INST_ID, that reports the instance identifier that relates to each record.Note: ASM dynamic performance views are referenced later in this course in the context of performing various administrative tasks. For a complete definition of all the columns in each view, refer to the Oracle Database Reference 11g Release 2 in the Oracle Database Documentation Library.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


ASM Dynamic Performance Views Diagram

V$ASM_DISKGROUP

V$ASM_ATTRIBUTE

V$ASM_DISK

V$ASM_ALIASV$ASM_FILE

V$ASM_TEMPLATE

V$ASM_OPERATION

V$ASM_CLIENTV$ASM_DISK_IOSTAT

Screen reader hint: This slide contains a diagram that is explained in the speaker

notes.

ASM Dynamic Performance Views DiagramThe diagram shows the relationships between the ASM dynamic performance views. It is noteworthy that all the views in some way relate back to V$ASM_DISKGROUP through the GROUP_NUMBER column.The relationships shown in the diagram may be expressed using the following statements:

• A disk group listed in V$ASM_DISKGROUP may contain numerous files listed in V$ASM_FILE.

• A disk group listed in V$ASM_DISKGROUP may contain numerous aliases listed in V$ASM_ALIAS.

• A file record listed in V$ASM_FILE will have a file name and may have an alias listed in V$ASM_ALIAS.

• A file name listed in V$ASM_ALIAS will relate to a file record contained in V$ASM_FILE.

• A file alias listed in V$ASM_ALIAS will relate to a file record contained in V$ASM_FILE.

• A directory alias listed in V$ASM_ALIAS will not have a corresponding record contained in V$ASM_FILE.

• A directory alias listed in V$ASM_ALIAS may be the parent of numerous file names and file aliases listed in V$ASM_ALIAS.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

ASM Dynamic Performance Views Diagram (continued)• A disk group listed in V$ASM_DISKGROUP will have attributes associated with it listed

in V$ASM_ATTRIBUTE if the disk group attribute COMPATIBLE.ASM is set to 11.1 or higher.

• A disk group listed in V$ASM_DISKGROUP may be the subject of a long-running operation listed in V$ASM_OPERATION.

• A disk group listed in V$ASM_DISKGROUP will contain numerous templates listed in V$ASM_TEMPLATE.

• A disk group listed in V$ASM_DISKGROUP might be currently used by numerous database instances listed in V$ASM_CLIENT.

• A disk group listed in V$ASM_DISKGROUP will contain numerous disks listed in V$ASM_DISK.

• A database client listed in V$ASM_CLIENT will have numerous performance records listed in V$ASM_DISK_IOSTAT, each of which corresponds to a disk listed in V$ASM_DISK.

Note: The V$ASM_DISKGROUP_STAT and V$ASM_DISK_STAT views are not shown here. These views mirror V$ASM_DISKGROUP and V$ASM_DISK, respectively. These views are described in more detail in the Oracle Database Reference 11g Release 2.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Quiz

The recommended configuration for ASM instance initialization is:1. Store the SPFILE on a shared raw device.2. Use a server parameter file (SPFILE).3. Store the SPFILE on separate disks.4. Store the SPFILE in an ASM disk group.5. Use a text initialization parameter file (PFILE).6. Use a PFILE that references an SPFILE.

Answer: 2,4


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Quiz

Automatic memory management is enabled by default on ASMinstances.1. True2. False

Answer: 1Oracle strongly recommends that you use automatic memory management for ASM.The default value used for MEMORY_TARGET (272 MB) is acceptable for most environments. The MEMORY_TARGET value is set during installation and may be adjusted later.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Summary

In this lesson, you should have learned how to:• Explain and apply ASM initialization parameters• Manage ASM instances and associated processes• Monitor ASM using the V$ASM dynamic performance views


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Practice 7 Overview: Administering ASM Instances

This practice covers the following topics:• Adjusting initialization parameters• Stopping and starting instances• Starting Enterprise Manager• Monitoring the status of instances


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Administering ASM Disk Groups

Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Objectives

After completing this lesson, you should be able to:• Create and delete Automatic Storage Management (ASM)

disk groups• Set the attributes of an existing ASM disk group• Perform ongoing maintenance tasks on ASM disk groups• Explain key performance and scalability considerations for

ASM disk groups


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Disk Group Overview

Disk 1 Disk 2 Disk 3 Disk 4 Disk 5 Disk 6

File 1 File 2

File 3 File 4


notes.

Disk Group OverviewA disk group is a grouping of one or more disks that ASM manages as a unit of storage. Each disk group contains the metadata required for the management of space in that disk group. An ASM disk group is conceptually similar to a logical volume (LV) in a typical Storage Area Network (SAN).Files are allocated from the space inside a disk group. The content of files that are stored in a disk group are evenly distributed, or striped, across the disks in the disk group to eliminate hot spots and to provide uniform performance across the disks. ASM striping balances disk usage such that all the disks in a disk group will be used evenly in percentage terms. When a disk group is made up of uniform-sized disks, the amount of data on each disk is approximately the same. When a disk group contains different-sized disks, the larger disks will contain more data than the smaller disks. A comparatively large disk may present an I/O bottleneck within a disk group, if the bandwidth to all disks in the disk group is the same. By default, ASM automatically rebalances storage whenever the storage configuration of a disk group changes such as when a disk is added.Each ASM file is completely contained within a single disk group. However, a disk group can contain files belonging to several databases and a single database can use different files from multiple disk groups. For most installations, you need only a small number of disk groups.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Creating a New Disk Group

The CREATE DISKGROUP command creates new ASM disk groups.

• Example:

CREATE DISKGROUP diskgroup_name[ { HIGH | NORMAL | EXTERNAL } REDUNDANCY ]{ [ FAILGROUP failgroup_name ]DISK qualified_disk_clause [, qualified_disk_clause]...

} ...[ ATTRIBUTE { 'attribute_name' = 'attribute_value' }... ]

;

qualified_disk_clause ::= search_string [ NAME disk_name ][ SIZE size_clause ][ FORCE | NOFORCE ]

CREATE DISKGROUP FRA NORMAL REDUNDANCY DISK 'ORCL:SDD11' NAME 'FRA_DISK1' SIZE 977 M,

'ORCL:SDD12' NAME 'FRA_DISK2' SIZE 977 M;

Creating a New Disk Group The CREATE DISKGROUP statement creates a disk group, assigns one or more disks to the disk group, and mounts the disk group for the first time. If you want ASM to automatically mount the disk group when an ASM instance starts, you must add the disk group name to the value of the ASM_DISKGROUPS initialization parameter in your parameter files (PFILEs). If you use a server parameter file (SPFILE), the disk group is added to the initialization parameter automatically.The CREATE DISKGROUP command can also be run using database management tools such as the ASM Configuration Assistant (ASMCA), Oracle Enterprise Manager, and ASM Command-Line utility (ASMCMD).The CREATE DISKGROUP statement has the following clauses:

• REDUNDANCY clauseThe REDUNDANCY clause lets you specify the redundancy level of the disk group.NORMAL REDUNDANCY requires the existence of at least two failure groups. By default, NORMAL REDUNDANCY provides a two-way mirror of all ASM files except for control files, which are mirrored three ways. NORMAL REDUNDANCY disk groups can tolerate the loss of one failure group.HIGH REDUNDANCY requires the existence of at least three failure groups. ASM fixes mirroring at three-way mirroring, with each file getting two mirrored copies. HIGH REDUNDANCY disk groups can tolerate the loss of two failure groups.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Creating a New Disk Group (continued)EXTERNAL REDUNDANCY indicates that ASM does not provide any redundancy for the diskgroup. The disks within the disk group must provide redundancy (for example, using a storage array), or you must be willing to tolerate the loss of the disk group if a disk fails. You cannot specify the FAILGROUP clause if you specify EXTERNAL REDUNDANCY.

• FAILGROUP clauseUse this clause to specify a name for one or more failure groups. If you omit this clause, and you have specified NORMAL or HIGH REDUNDANCY, then ASM automatically adds each disk in the disk group to its own failure group. The implicit name of the failure group is the same as the name in the NAME clause.

• DISK clauseUse this clause to specify one or more disks for each failure group.For each disk that you are adding to the disk group, specify the operating system–dependent search string that ASM will use to find the disk. The search_string must point to a subset of the disks returned by discovery using the strings in the ASM_DISKSTRING initialization parameter. If search_string does not point to any disks to which the ASM user has read/write access, then ASM returns an error. If it points to one or more disks that have already been assigned to a different disk group, then Oracle Database returns an error unless you also specify FORCE. For each valid candidate disk, ASM formats the disk header to indicate that it is a member of the new disk group.The optional NAME subclause is valid only if the search_string points to a single disk. It specifies an operating system–independent name for the disk. The name can be up to 30 alphanumeric characters. The first character must be alphabetic. If you omit this clause and you assigned a label to a disk through ASMLib, then that label is used as the disk name. If you are not using ASMLib, then ASM creates a default name of the form diskgroup_name_nnnn, where nnnn is the disk number. You can use this name to refer to the disk in subsequent ASM operations.Use the optional SIZE subclause to specify the size of the disk. If you specify a size greater than the capacity of the disk, then ASM returns an error. If you specify a size less than the capacity of the disk, you limit the disk space ASM will use. If you omit this clause, ASM attempts to determine the size of the disk programmatically.You can specify FORCE or NOFORCE for each disk.Specify FORCE if you want ASM to add the disk to the disk group even if the disk is already a member of a different disk group. Exercise caution because using FORCE in this way may destroy existing disk groups. For this clause to be valid, the disk must already be a member of a disk group and the disk cannot be part of a mounted disk group.Specify NOFORCE if you want ASM to return an error if the disk is already a member of a different disk group. NOFORCE is the default.

• ATTRIBUTE clauseUse this clause to set attribute values for the disk group. ASM disk group attributes are described later in this lesson.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Creating a New Disk Group with ASMCMD

ASMCMD allows an XML configuration file to create and change the disk group.• Sample XML used with the mkdg command:

<dg name=“DATA” redundancy=“normal”>

<fg name=“fg1”>

<dsk string=“/dev/sda1” />

<dsk string=“/dev/sdb1” />

</fg>

<fg name=“fg2”>

<dsk string=“/dev/sdc1” />

<dsk string=“/dev/sdd1” />

</fg>

<a name=“compatible.asm” value=“11.2”/>

<a name=“compatible.rdbms” value=“11.2”/>

</dg>

Creating a New Disk Group with ASMCMDASMCMD has added the ability to use an XML configuration file to either create a disk group or change a disk group configuration.The XML file for the mkdg command specifies the name of the disk group, redundancy, attributes, and paths of the disks that form the disk group. Redundancy is an optional parameter; the default is normal redundancy. For some types of redundancy, disks are required to be gathered into failure groups. In the case that failure groups are not specified, every disk will be in its own failure group.It is possible to set the disk group attribute values during disk group creation. Some attributes, such as AU_SIZE and SECTOR_SIZE, can be set only during disk group creation.The following is an example of an inline XML configuration for chdg. This XML alters the disk group named DATA. The FG1 failure group is dropped and the DATA_0001 disk is also dropped. The /dev/disk5 disk is added to the FG2 failure group. The rebalance power level is set to 3.ASMCMD> chdg '<chdg> <dg name="DATA" power="3"> <drop> <fgname="FG1"> </fg> <dsk name="DATA_0001" /> </drop> <add> <fgname="FG2"> <dsk string="/dev/disk5"/> </fg> </add> </chdg>'


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Disk Group Attributes

The following attributes can be set for a disk group:Attribute Description Valid Values Default

ValueAU_SIZE Specifies the allocation unit (AU)

size. This attribute can be set only during disk group creation.

1M, 2M, 4M, 8M, 16M, 32M, 64M (M = Megabyte)

1M

DISK_REPAIR_TIME Specifies the amount of time ASM will wait from when a disk goes offline until ASM drops it and rebalances the disk group

0 to 136 years specified in minutes (M) or hours (H)

3.6H

COMPATIBLE.RDBMS Specifies the minimum software version required for a database instance to use files in this disk group

At least the first two digits of a valid Oracle Database release number

10.1

COMPATIBLE.ASM Specifies the minimum software version required for an ASM instance to mount this disk group

At least the first two digits of a valid Oracle Database release number

10.1

COMPATIBLE.ADVM Allows the creation of ASM volumes

>=11.2 NONE

Disk Group AttributesFirst introduced in Oracle Database 11g, disk group attributes govern various aspects of how a disk group functions.AU_SIZE specifies the allocation unit size, which is the fundamental unit of space within a disk group. It is also the stripe size for coarse-grained striping. This attribute can be set only during disk group creation.DISK_REPAIR_TIME specifies the amount of time ASM will wait from when a disk goes offline until ASM drops it and rebalances the disk group. This attribute enables the fast mirrorresync feature, whereby ASM keeps track of pending changes on an offline disk during an outage and the extents are automatically resynced when the disk is brought back online. There are three compatibility attributes. COMPATIBLE.RDBMS specifies the minimum software version required for a database instance to use files in this disk group, whereas COMPATIBLE.ASM specifies the minimum software version required for an ASM instance to mount this disk group. The compatibility attributes are discussed in greater detail later in this lesson. The value for the disk group COMPATIBLE.ADVM attribute determines whether the disk group can contain Oracle ASM volumes. The COMPATIBLE.ASM must be set to 11.2 first, and the ASM Dynamic Volume Manager (ADVM) drivers must be loaded.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Disk Group Attributes

The following attributes can be set for a disk group:Attribute Description Valid

ValuesDefault Value

ACCESS_CONTROL.ENABLED Allows the creation of access control lists (ACL)

TRUE, FALSE FALSE

ACCESS_CONTROL.UMASK Sets the default permissions to be set for files created in the disk group

0,2,6 in a triple for owner, group, and others

None

SECTOR_SIZE Is the optional parameter used with CREATE DISKGROUP to specify the disk sector size

512,4096,4K 512

CELL.SMART_SCAN_CAPABLE Enables smart scan predicate offload processing if all disks in the disk group are Exadata Grid Disks

TRUE, FALSE FALSE

Disk Group Attributes (continued)ACCESS_CONTROL.ENABLED and ACCESS_CONTROL.UMASK are discussed in more detail in the lesson tilted “Administering ASM Files, Directories, and Templates.” Both attributes can be set after the disk group is created.SECTOR_SIZE is used to support 4KB sector disk drives. From Oracle Database 11g version 11.1.0.7, there is a new attribute, CELL.SMART_SCAN_CAPABLE, that enables smart scan predicate offload processing if all disks in the disk group are Exadata Grid Disks.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


V$ASM_ATTRIBUTE

• In an ASM instance, V$ASM_ATTRIBUTE displays one row for each attribute defined.

• In addition to attributes specified by CREATE DISKGROUPand ALTER DISKGROUP statements, the view may show other attributes that are created automatically.

• Disk group attributes are listed in V$ASM_ATTRIBUTE only if the disk group attribute COMPATIBLE.ASM is set to 11.1 or higher.

• In a database instance, V$ASM_ATTRIBUTE displays no rows.

• This is the information displayed by asmcmd lsattr –lm –G <diskGroup>.

V$ASM_ATTRIBUTE

V$ASM_ATTRIBUTE lists the ASM disk group attributes if the disk group attributeCOMPATIBLE.ASM is set to 11.1 or higher. If COMPATIBLE.ASM is not set to 11.1 or higher, then V$ASM_ATTRIBUTE will have no rows corresponding to the disk group.Following is a typical example of a query that retrieves attributes for a particular disk group:

SQL> SELECT G.NAME DISK_GROUP, A.NAME ATTRIBUTE, A.VALUE2 FROM V$ASM_ATTRIBUTE A, V$ASM_DISKGROUP G3 WHERE A.GROUP_NUMBER = G.GROUP_NUMBER4 AND G.NAME = 'DATA';

The ASMCMD lsattr –lm –G command will show the same information for a disk group.

$ asmcmd lsattr -lm -G DATAGroup_Name Name Value RO Sys DATA access_control.enabled FALSE N Y DATA access_control.umask 066 N Y DATA au_size 1048576 Y Y DATA cell.smart_scan_capable FALSE N N DATA compatible.asm 11.2.0.0.0 N Y DATA compatible.rdbms 10.1.0.0.0 N Y DATA disk_repair_time 3.6h N Y DATA sector_size 512 Y Y


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Compatibility Attributes

• Disk group compatibility attributes can be set using CREATE DISKGROUP or ALTER DISKGROUP.

• Values can only be advanced and the setting is irreversible.• COMPATIBLE.RDBMS must be less than or equal to

COMPATIBLE.ASM.• Some valid attribute combinations:

COMPATIBLE

ASM

COMPATIBLE

RDBMS

COMPATIBLE

ADVM

ASM Instance Version

COMPATIBLESetting for RDBMS Instance

10.1 10.1 n/a >=10.1 >=10.1

11.1 10.1 n/a >=11.1 >=10.1

11.2 11.1 11.2 >=11.2 >=11.1

11.2 11.2 11.2 >=11.2 >=11.2

Compatibility AttributesYou can set disk group compatibility with the CREATE DISKGROUP or ALTER DISKGROUP SQL statement. For example:

ALTER DISKGROUP FRA SET ATTRIBUTE 'compatible.asm' = '11.2';

When setting the values for the COMPATIBLE.RDBMS and COMPATIBLE.ASM attributes, specify at least the first two digits of a valid Oracle Database release number. For example, you can specify compatibility as 10.1 or 11.1. ASM assumes that any missing version number digits are zeros.Note that although it is possible to alter these attributes, they can only be advanced (for example, from 10.1 to 11.1). Advancing the values for on-disk compatibility attributes is an irreversible operation. To revert to the previous value, you must create a new disk group with the old compatibility attributes and then restore the database files that were in the disk group.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Features Enabled by Disk Group Compatibility Attributes

Disk Group Features Enabled COMPATIBLE ASM

COMPATIBLE RDBMS

COMPATIBLE ADVM

Different AU sizes (1, 2, 4, 8, 16, 32, or 64 MB) >=11.1 >=11.1 n/aAttributes displayed in the V$ASM_ATTRIBUTE view >=11.1 >=11.1 n/aFast mirror resync (DISK_REPAIR_TIME attribute) >=11.1 >=11.1

Exadata storage >= 11.1.0.7 >= 11.1.0.7 n/a

Variable size extents 11.1 11.1 n/a

Preferred read failure groups 11.1 11.1 n/a

Intelligent Data Placement >=11.2 >=11.2 n/a

OCR and voting disks in a disk group >=11.2 n/a n/a

Sector size set to nondefault value >=11.2 >=11.2 n/a

Oracle ASM SPFILE in a disk group >=11.2 n/a n/a

Oracle ASM File Access Control >=11.2 >11.2 n/a

Volumes in disk groups >=11.2 n/a >=11.2

Features Enabled by Disk Group Compatibility AttributesSetting the disk group compatibility attributes is essentially a balance between flexibility and functionality. At one extreme setting, COMPATIBLE.RDBMS and COMPATIBLE.ASM to 10.1 will allow the greatest flexibility regarding the ASM and database software versions that can use a disk group. However, such a setting limits access to various Oracle Database 11g ASM features such as large (>8 MB) allocation units, fast mirror resync, variable sized extents, and preferred read failure groups.That is, setting COMPATIBLE.RDBMS and COMPATIBLE.ASM to 11.1 will provide access to the Oracle Database 11g ASM features but require that the Oracle Database 11g software be installed and in use across your environment.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Support for 4KB Sector Disk Drives

Oracle supports 4KB sector disk drives.• In emulation mode:

– No changes but there are performance penalties• In native mode, there is no performance penalty with:

– Oracle database files– ASM files, at the disk group level

ACFS/ADVM does not support 4KB sector disk drives.

Support for 4KB Sector Disk DrivesThe disk drive industry decided several years ago that to continue to develop larger capacity and more reliable hard disks, a change to the sector size was needed. The sector size has been 512 bytes and has not changed in 25 years. This sector size has been hard-coded into operating systems and file system code. The disk manufacturers have announced a plan to start changing over to the “big-sector” disks in 2011. In a few years, they expect to be producing only 4KB sector disk drives. Most operating systems can already query the physical disk device to determine the sector size setting. Because Oracle software—specifically, the Database Writer, Log Writer, and ASM processes—can write directly to the disk without going through the operating system, Oracle is now supporting 4KB sector disk drives for ASM and database files on a file system. The performance penalty, in emulation mode, is caused when the 512-byte sector writes are translated onto 4K sector disks. On a physical disk, the sector reads and writes are atomic, and the disk drive reads the 4K byte sector into disk cache memory, changes a 512-byte section, and writes the entire 4K sector.ASM disk group commands have changed to allow you to specify the sector size.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Supporting 4 KB Sector Disks

• Emulation mode:

LBA0 LBA1 LBA2 LBA3 LBA4 LBA5 LBA6 LBA7

4096-byte physical sector

• Native mode:LBA0

4096-byte physical sector

Logical sector

Physical sector

Supporting 4 KB Sector Disks4 KB sector disks have physical sectors (shown in gray) and logical sectors (shown in blue). There are two types of 4 KB sector disks: emulation mode and native mode.

• 4 KB sector disks in emulation mode have eight logical sectors per physical sector (as shown in the slide). They maintain a 512-byte interface to their 4 KB physical sectors—that is, the logical block address (LBA) references 512 bytes on disk.

• 4 KB sector disks in native mode have one logical sector per physical sector (as shown in the slide). So, there is only the 4 KB interface. In other words, the LBA references 4096 bytes on disk.

Emulation mode can hurt performance because the disk drive reads the 4 KB sector into disk cache memory, changes the 512-byte section, and then writes the entire 4 KB sector back to disk. For example, when redo is being written to disk in 512 chunks, each write requires a read of the 4KB sector, an update of the 512-byte section, and then a write. With native mode, the 4 KB sector is written without requiring a read and update.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


ASM Support for 4KB Sector Disks

• ASM commands:– CREATE DISKGROUP…ATTRIBUTE SECTOR_SIZE

• ASMCA support for the new commands• Enterprise Manager support• All the disks in a disk group must have the same sector

size.• Restrictions on redo log files:

– The BLOCKSIZE redo log file and the SECTOR_SIZE disk group

ASM Support for 4KB Sector DisksASM sector size can be managed from three different tools: SQL command line, the ASM Command-Line utility (ASMCMD), and Enterprise Manager. Because disk groups must have a compatible sector size on all the disks in a disk group, the sector size is set as an attribute of the disk group when the disk group is created. The ALTER DISKGROUP ADD DISK command requires that the candidate disk to be added has the same sector size as the existing disks in the group. (This should be handled by checking the disk properties at the time the ADD DISKcommand is executed.) Best Practice: For data files and temp files, avoid using data block size of less than 4K bytes with 4KB sector disk drives. The default data block size is 8K bytes.For redo log files and archive log files, the block size specified for the log file must match the sector size of the disk group. The syntax for specifying a new log file in the +DATA disk group of 100 MB with the redo block size of 4 KB is:SQL> ALTER DATABASE ADD LOGFILE +DATA SIZE 100M BLOCKSIZE 4K;


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Using the SECTOR_SIZE Clause

Creating a disk group (in ASM) with a 4 KB sector size:

CREATE DISKGROUP mydgroup1 NORMAL REDUNDANCY

FAILGROUP mycontroller1 DISK

'/devices/diska1',

'/devices/diska2',

'/devices/diska3',

'/devices/diska4'

FAILGROUP mycontroller2 DISK

'/devices/diskb1',

'/devices/diskb2',

'/devices/diskb3',

'/devices/diskb4'ATTRIBUTE 'sector_size'='4096';

Using the SECTOR_SIZE Clause You can manage ASM with three different tools: the SQL command line, the ASMCMD utility, and Enterprise Manager.What you already know: Use the CREATE DISKGROUP clause to name a group of disks and specify that the Oracle database should manage the group for you. The Oracle database manages a disk group as a logical unit and evenly spreads each file across the disks to balance I/O. It also automatically distributes database files across all available disks in disk groups and rebalances storage automatically whenever the storage configuration changes. What is new: Use the optional SECTOR_SIZE clause in the CREATE DISKGROUP command to explicitly specify the sector size value for the disk group. If you do not use this clause, ASM detects the hardware sector size and uses it. The SECTOR_SIZE disk group attribute can be set only during disk group creation. The values for the SECTOR_SIZE can be 512, 1024, 4096, or 4K. ASM provides support for 4 KB sector disk drives without negatively affecting performance.You can determine the sector size value that has either been assumed or explicitly set for a successful disk group creation by querying the V$ASM_ATTRIBUTE view. You can also query the SECTOR_SIZE column in the V$ASM_DISKGROUP view. The asmcmd lsattr command also displays the sector size of a disk group.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Using the SECTOR_SIZE Clause (continued)For details about how ASM verifies the sector size (and when this operation fails), see the Oracle Database Storage Administrator’s Guide 11g Release 2 (11.2). All disks in the disk group must have a sector size equal to the attribute value specified. When processing the CREATE DISKGROUP command, ASM queries the operating system for the sector size of every disk specified in this command before the disk is added to the disk group. It ensures that a disk group is made up of disks with an identical sector size. If a disk has a different sector size, the command fails. Similar checks are performed when mounting the disk group. If you omit this attribute in the CREATE DISKGROUP command, ASM proceeds with the create operation as long as all specified disks have an identical sector size and this value is used as the disk group sector size.When you add new disks to an existing disk group with the ALTER DISKGROUP ADD DISKcommand, the new disks must also have the same sector size value. ASM verifies this. The ALTER DISKGROUP command fails if any of the disks have a different sector size.By setting a value for this attribute, you can establish the sector size for the disk group, rather than let ASM assume a value for all the disks in the disk group. You can query the SECTOR_SIZE column of the V$ASM_ATTRIBUTE view to determine the sector size before attempting to add a new disk to the disk group.Note: The Oracle Automatic Storage Management Cluster File System (ACFS) does not support 4 KB sector disks.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Viewing ASM Disk Groups

• In an ASM instance, V$ASM_DISKGROUP displays one row for every ASM disk group discovered by the ASM instance.

• In a database instance, V$ASM_DISKGROUP displays one row for every ASM disk group available to the database instance.

• Other ASM dynamic performance views relate to V$ASM_DISKGROUP through the GROUP_NUMBER column.

• ASMCMD lsdg command provides the same information.• V$ASM_DISKGROUP_STAT displays performance statistics

in the same way that V$ASM_DISKGROUP does, but without performing discovery of new disks.

Viewing ASM Disk GroupsV$ASM_DISKGROUP contains configuration and status information for ASM disk groups. It is commonly queried by itself to display information about disk groups but is also commonly joined with other ASM dynamic performance views as shown in many of the examples in this lesson.The following example shows a typical V$ASM_DISKGROUP query that summarizes the current space utilization for the ASM disk groups:

SQL> SELECT NAME, TYPE, TOTAL_MB, FREE_MB, 2 REQUIRED_MIRROR_FREE_MB RMFM, USABLE_FILE_MB3 FROM V$ASM_DISKGROUP;

NAME TYPE TOTAL_MB FREE_MB RMFM USABLE_FILE_MB---- ------ -------- ------- ----- --------------DATA NORMAL 9998 4248 1449 1399 ACFS EXTERN 9996 3706 0 3706FRA EXTERN 7497 7356 0 7356


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Viewing ASM Disk Groups (continued) The asmcmd lsdg command is a preformatted view of the information fromV$ASM_DISKGROUP.$ asmcmd lsdgState Type Rebal Sector Block AU Total_MB Free_MB

Req_mir_free_MB Usable_file_MB Offline_disks Voting_files NameMOUNTED EXTERN N 512 4096 1048576 9996 3706

0 3706 0 N ACFS/MOUNTED NORMAL N 512 4096 1048576 9998 4248

1449 1399 0 N DATA/MOUNTED EXTERN N 512 4096 1048576 7497 7356

0 7356 0 N FRA/

Note the following when considering the space utilization statistics provided by V$ASM_DISKGROUP or asmcmd lsdg.

• The TOTAL_MB column is the total capacity of the disk group in megabytes, not taking mirroring into account. In the preceding example, both disk groups are NORMALredundancy disk groups, so by default most files contained in them will be two-way mirrored. In practical terms, that means the total usable size of both disk groups is approximately half of the number reported.

• The FREE_MB column is the total unused capacity of the disk group in megabytes, not taking mirroring into account. In the preceding example, both disk groups are NORMALredundancy disk groups, so in practice the total free space in both disk groups is approximately half of the number reported.

• The REQUIRED_MIRROR_FREE_MB column shows the amount of space that is required to be available in a given disk group in order to restore redundancy after the worst failure that can be tolerated by the disk group. In the case of the DATA disk group in the preceding example, 977 MB is the size of eachof the six failure groups that currently make up that disk group. The worst failure that can be tolerated by this disk group is the loss of one failure group because the loss of any more would mean the loss of any data that was spread across the lost failure groups. So in essence, as long as the DATA disk group has 977 MB free, the loss of any one failure group can be tolerated without compromising mirroring.In the case of the FRA disk group, zero is reported because that disk group consists of only two disks. Because of that, the loss of either disk will compromise mirroring regardless of the amount of space that is free.

• USABLE_FILE_MB is computed by subtracting REQUIRED_MIRROR_FREE_MB from the total free space in the disk group and then adjusting for mirroring. It is supposed to show the amount of free space that can be safely utilized taking mirroring into account and yet be able to restore redundancy after a disk failure. In the case of the DATA disk group, the reported negative value shows that mirroring would be compromised by a failure although there is space available in the disk group.In the case of the FRA disk group, the value has a different meaning. It is already known that the disk group cannot tolerate failure without compromising mirroring. So in this case, the computed value of 630 MB simply refers to the amount of available free space after mirroring is factored in.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Viewing ASM Disk Information

• In an ASM instance, V$ASM_DISK displays one row for every disk discovered by the ASM instance, including disks that are not part of any disk group.

• In a database instance, V$ASM_DISK displays rows only for disks in disk groups in use by the database instance.

• V$ASM_DISK_STAT has the same columns but does not discover new disks (may not have the latest information).

• ASMCMD lsdsk has several options and two modes.– Modes:

— Connected shows information from dynamic performance views.— Nonconnected shows information from disk headers.

– Options: -t, -k, and -p provide different preformatted views.

Viewing ASM Disk InformationFollowing is a typical example of a query that retrieves information about disks in a particular disk group:

SQL> SELECT G.NAME DISK_GROUP, D.NAME, D.STATE, D.TOTAL_MB, 2 D.FREE_MB3 FROM V$ASM_DISK D, V$ASM_DISKGROUP G4 WHERE D.GROUP_NUMBER = G.GROUP_NUMBER5 AND G.NAME = 'DATA';

DISK_GROUP NAME STATE TOTAL_MB FREE_MB---------- ------ -------- ---------- ----------DATA SDE5 NORMAL 977 136DATA SDE6 NORMAL 977 134DATA SDE7 NORMAL 977 135DATA SDE10 NORMAL 977 150DATA SDE8 NORMAL 977 150DATA SDE9 NORMAL 977 150


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Viewing ASM Disk Information (continued) Following is another example that uses V$ASM_DISK to display disks that do not belong to any disk group:

SQL> SELECT PATH, MOUNT_STATUS, HEADER_STATUS, OS_MB2 FROM V$ASM_DISK3 WHERE GROUP_NUMBER = 0;

PATH MOUNT_S HEADER OS_MB--------------- ------- ------ ----------ORCL:SDD10 CLOSED FORMER 977ORCL:SDD8 CLOSED FORMER 977ORCL:SDD9 CLOSED FORMER 977ORCL:SDD11 CLOSED FORMER 977

V$ASM_DISK_STAT displays performance statistics in the same way that V$ASM_DISKdoes, but without performing discovery of new disks. This results in a less expensive operation. However, because discovery is not performed, the output of this view does not include any data about disks that are new to the system.The columns for V$ASM_DISK_STAT are the same as those for V$ASM_DISK.The ASMCMD lsdsk command provides a way to see the V$ASM_DISK information in preformatted columns without having to write SQL statements. The ASMCMD lsdskcommand can be used in connected mode where it retrieves the data from V$ASM_DISK, ornonconnected mode where the data is collected from the ASM disk headers. The -g, and --discovery options control whether V$ASM_DISK or V$ASM_DISK_STAT are used. The ASMCMD lsdsk –k command connects to the ASM instance shows the following:

$ asmcmd lsdsk -kTotal_MB Free_MB OS_MB Name Failgroup Library Label UDID Product Redund Path

1953 654 1953 DATA1 DATA1 ASM Library - Generic Linux, version 2.0.4 (KABI_V2) DATA1 UNKNOWN ORCL:DATA1





Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Extending an Existing Disk Group

• The ALTER DISKGROUP command enables you to extend an existing disk group by adding disks to it.

– Example:

• ASMCMD, ASMCA, and Enterprise Manager also provide interfaces to extend the existing disk groups.

ALTER DISKGROUP diskgroup_name { ADD{ [ FAILGROUP failgroup_name ]DISK qualified_disk_clause [, qualified_disk_clause ]...

}...}[ REBALANCE [POWER integer] [WAIT | NOWAIT] ]

;

ALTER DISKGROUP FRA ADD DISK 'ORCL:SDE5' NAME 'FRA_DISK3','ORCL:SDE6' NAME 'FRA_DISK4';

Extending an Existing Disk GroupYou can use the ADD clause of the ALTER DISKGROUP statement to add a disk or a failure group to a disk group. The same syntax that you use to add a disk or failure group with the CREATE DISKGROUP statement can be used with the ALTER DISKGROUP statement.ASM automatically rebalances the disk group when disks are added. By default, the ALTER DISKGROUP statement returns immediately after the disks have been added while the rebalance operation continues to run asynchronously. You can query the V$ASM_OPERATION view to monitor the status of the rebalance operation.You can optionally use the REBALANCE clause to manually control the rebalance process. In the POWER clause, you can specify a value from 0 to 11. A value of 0 disables rebalancing for this statement. A value of 1 causes the rebalance to take place with minimal resources allocated to it, whereas a value of 11 permits ASM to execute the rebalance as quickly as possible. If you do not specify a value, the POWER clause defaults to the value of the ASM_POWER_LIMIT initialization parameter.You can specify WAIT or NOWAIT to determine whether the ALTER DISKGROUP statement waits for the rebalance to conclude before returning or not. NOWAIT is the default.You can also extend a disk group using the Add Disks button in the ASM Disk Groups window of the Database Configuration Assistant (DBCA). Enterprise Manager provides an Add button in each specific Disk Group window.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Dropping Disks from an Existing Disk Group

• The ALTER DISKGROUP command enables you to remove disks or failure groups from an existing disk group.

– Example:

ALTER DISKGROUP diskgroup_name { DROP{ DISK

disk_name [ FORCE | NOFORCE ][, disk_name [ FORCE | NOFORCE ] ]...

| DISKS IN FAILGROUPfailgroup_name [ FORCE | NOFORCE ][, failgroup_name [ FORCE | NOFORCE ] ]...

}}[ REBALANCE [POWER integer] [WAIT | NOWAIT] ]

;

ALTER DISKGROUP FRA DROP DISK FRA_DISK1, FRA_DISK4;

Dropping Disks from an Existing Disk GroupTo drop disks from a disk group, use the DROP DISK clause of the ALTER DISKGROUPstatement. A dropped disk can be reused by adding it to a disk group or by using it in the creation of a new disk group. A dropped disk can be added back to the disk group it was removed from.When a disk is dropped, the disk group is rebalanced by moving all the file extents from the dropped disk to other disks in the disk group. A drop disk operation will fail if not enough space is available on the other disks. Data will not be lost by dropping a disk.By default, the ALTER DISKGROUP...DROP DISK statement returns before the drop and rebalance operations are complete. Do not reuse, remove, or disconnect the dropped disk until the HEADER_STATUS column for this disk in the V$ASM_DISK view changes to FORMER. You can query the V$ASM_OPERATION view to determine the amount of time remaining for the drop/rebalance operation to complete. If you specify the FORCE clause for the drop operation, the disk is dropped even if ASM cannot read from or write to the disk. You cannot use the FORCE flag when dropping a disk from an external redundancy disk group. A DROP FORCE operation leaves data at reduced redundancy for as long as it takes for the subsequent rebalance operation to complete. Take great care because this increases your exposure to data loss if there is a subsequent disk failure during rebalancing. The REBALANCE clause works in the same way as when extending a disk group. You can also use Enterprise Manager, ASMCA, or ASMCMD interfaces to drop disks from a disk group.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


REBALANCE POWER 0

• You can effectively disable rebalancing when adding disks to or removing disks from a disk group by specifying REBALANCE POWER 0 in the ALTER DISKGROUPstatement.

• When adding, the disks become immediately available to the disk group; however, existing data is not moved to the new disks.– New data may be written to the new disks.

• When removing, the statement executes and the disks are marked with a status of DROPPING. However, because the operation to move the data to another disk is effectively disabled, the disk will remain in this state indefinitely until another operation causes the disk group to be rebalanced.

REBALANCE POWER 0

When adding disks to or removing disks from a disk group, you can effectively disable rebalancing by specifying REBALANCE POWER 0 in the ALTER DISKGROUP statement.When adding disks to a disk group in conjunction with REBALANCE POWER 0, the disks become immediately available to the disk group; however, existing data is not moved to the new disks. New data may be written to the new disks. This is useful in situations where you need to add disks quickly to a running system and you want to defer the rebalance operation until a later time such as a scheduled maintenance period.When removing disks from a disk group in conjunction with REBALANCE POWER 0, the statement executes and the disks are marked with a status of DROPPING. However, because the operation to move the data to another disk is effectively disabled, the disk will remain in this state indefinitely until another add or drop operation causes the disk group to be rebalanced or the disk group is manually rebalanced using the ALTER DISKGROUP <diskgroup_name> REBALANCE statement.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


V$ASM_OPERATION

• In an ASM instance, V$ASM_OPERATION displays one row for every active long-running operation executing in the ASM instance.

• In a database instance, V$ASM_OPERATION displays no rows.

• The ASMCMD lsop command shows ASM operations:ASMCMD [+] > lsop

Group_Name Dsk_Num State Power

FRA REBAL REAP 3

V$ASM_OPERATION

V$ASM_OPERATION provides information about long-running operations conducted by ASM. A long-running operation is one of the following:

• A rebalance operation that results from adding disks to or removing disks from a disk group

• A mirror resync operation that results from a disk being brought back online after an outage

The following example shows a typical query listing all the current operations: SQL> SELECT G.NAME DISK_GROUP, O.*

2 FROM V$ASM_OPERATION O, V$ASM_DISKGROUP G3 WHERE O.GROUP_NUMBER = G.GROUP_NUMBER;

DISK_GROUP GROUP_NUMBER OPERATION STATE POWER ACTUAL---------- ------------ --------- ----- ----- ------SOFAR EST_WORK EST_RATE EST_MINUTES ERROR_CODE----- -------- -------- ----------- ----------DATA 1 REBAL RUN 2 2

644 1237 565 1

In this example, a rebalance operation is currently running. The operation is estimated to require the movement of 1,237 allocation units. So far, 644 allocation units have been moved at a rate of 565 per minute. It is estimated that the operation will complete in 1 minute. No errors have been reported.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Adding and Dropping in the Same Command

You can add and drop disks in the same ALTER DISKGROUPcommand.

• Is useful when replacing disks• Avoids multiple rebalancing operations

Example:

ALTER DISKGROUP diskgroup_name { ADD ... }{ DROP ... }[ REBALANCE [POWER integer] [WAIT | NOWAIT] ];

ALTER DISKGROUP FRA ADD DISK 'ORCL:SDE7' NAME 'FRA_DISK5' SIZE 977 M ,

'ORCL:SDE8' NAME 'FRA_DISK6' SIZE 977 MDROP DISK FRA_DISK1, FRA_DISK2;

Adding and Dropping in the Same CommandIt is possible to combine both the add and drop operations in the same ALTER DISKGROUPstatement. This approach is recommended in preference to separate operations wherever possible. Adding and dropping in the same command has the benefit of rebalancing data extents once and can provide greater assurance that there is enough space for the rebalance operation to succeed. Adding disks to a disk group and dropping disks from a disk group, whether in the same command or not, is an effective way of migrating ASM from an existing storage platform to a new one. For example, when a new disk subsystem is configured for discovery by ASM, it is possible to migrate your data from its current storage to the new storage without down time by using a single command for each disk group.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Adding and Dropping Failure Groups

Examples:

ALTER DISKGROUP FRA ADD FAILGROUP FAIL2 DISK 'ORCL:SDE11' SIZE 977 M ,'ORCL:SDE12' SIZE 977 MDROP DISKS IN FAILGROUP FAIL1REBALANCE POWER 11 WAIT;

ALTER DISKGROUP FRA DROP DISKS IN FAILGROUP FAIL2REBALANCE POWER 1;

ALTER DISKGROUP FRAADD FAILGROUP FAIL1DISK 'ORCL:SDE9' SIZE 977 M ,'ORCL:SDE10' SIZE 977 MREBALANCE POWER 0;

Adding and Dropping Failure GroupsAdding and dropping failure groups is essentially the same as adding and dropping individual disks. The only noticeable difference from a user’s perspective is the variation in the ALTER DISKGROUP statement to include explicit references to failure group names.In the slide’s examples, the first statement will add a failure group with two disks. However, no data will be moved into the failure group because rebalancing is disabled for this statement.If the second statement is executed immediately after the first, the FAIL1 failure group and its disks will be removed and effectively replaced by the FAIL2 failure group and the two disks it contains. This statement will trigger a rebalancing operation to use FAIL2 and it will not return until the rebalance is complete. The rebalance operation will execute as quickly as possible but may consume considerable system resources.The final statement will remove the FAIL2 failure group and return the disk group to the same disk configuration as it had before this series of statements. This statement will return quickly. The rebalance operation triggered by it will proceed asynchronously and will be allocated minimal system resources.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Undropping Disks in Disk Groups

Use ALTER DISKGROUP ... UNDROP DISKS to cancel all pending disk removal operations within disk groups.

Examples:

ALTER DISKGROUP{ diskgroup_name [, diskgroup_name ] ...| ALL}UNDROP DISKS

;

ALTER DISKGROUP DATA UNDROP DISKS;

ALTER DISKGROUP DATA2, DATA3 UNDROP DISKS;

ALTER DISKGROUP ALL UNDROP DISKS;

Undropping Disks in Disk GroupsThe UNDROP DISKS clause of the ALTER DISKGROUP statement enables you to cancel all pending disk removal operations within disk groups. Pending disk removal operations include those currently in progress.This statement cannot be used to restore disks that are being dropped as the result of a DROP DISKGROUP statement, or for disks that are being dropped using the FORCE clause.If a drop disk operation has already completed, this statement cannot be used to restore it. In this case, you can simply add the dropped disk back to the disk group to achieve the same outcome.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Mounting and Dismounting Disk Groups

The ALTER DISKGROUP command enables you to manually mount and dismount disk groups.

Examples:

ALTER DISKGROUP{ diskgroup_name [, diskgroup_name ] ...| ALL}{ MOUNT [ RESTRICTED | NORMAL ]

[ FORCE | NOFORCE ]| DISMOUNT [ FORCE | NOFORCE ]}

;

ALTER DISKGROUP DATA MOUNT;

ALTER DISKGROUP DATA2, DATA3 MOUNT;

ALTER DISKGROUP ALL DISMOUNT;

Mounting and Dismounting Disk GroupsDisk groups must be mounted for database instances to access the files they contain. To mount or dismount disk groups, use the MOUNT or DISMOUNT clauses of the ALTER DISKGROUPstatement. You can mount or dismount disk groups by name, or specify ALL. Specify ALL MOUNT to mount all the disk groups specified in the ASM_DISKGROUPS initialization parameter. Specify ALL DISMOUNT to dismount all currently mounted disk groups.If a disk group is mounted using the RESTRICTED option, no other ASM instance in the same cluster can mount that disk group and the disk group is not usable by any database instances. RESTRICTED mode can be used to improve rebalancing performance on a cluster by eliminating messaging and coordination with other ASM instances and database instances.For normal and high redundancy disk groups, use the FORCE option of the MOUNT clause to mount a disk group if there are sufficient ASM disks available. The mount succeeds if ASM finds at least one complete set of extents in a disk group. MOUNT FORCE is used in situations where one or more disks are not available. When MOUNT FORCE is used, ASM flags the unavailable disks as offline and drops the disks after DISK_REPAIR_TIME expires.If you try to dismount a disk group that contains open files, the statement will fail, unless you also specify the FORCE clause. If you DISMOUNT FORCE, the files in the disk group become immediately inaccessible to your databases, and any operations involving those files will fail. In extreme cases, your database instance will abort if a disk group that it uses is forcibly dismounted.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Viewing Connected Clients

To view ASM clients:• In an ASM instance, V$ASM_CLIENT displays one row for

each open ASM disk group used by each client database instance.

• In a database instance, V$ASM_CLIENT displays one row for each open ASM disk group used by the database instance.

• Connect to an ASM instance with the asmcmd lsctcommand.

Viewing Connected ClientsFollowing is a typical example of a query that retrieves information about databases that are current clients of a particular disk group using V$ASM_CLIENT from the ASM instance:

SQL> SELECT G.NAME DISK_GROUP, C.*2 FROM V$ASM_CLIENT C, V$ASM_DISKGROUP G3 WHERE C.GROUP_NUMBER = G.GROUP_NUMBER4 AND G.NAME = 'DATA';

SOFTWARE COMPATIBLEDISK_GROUP GROUP# INSTANCE_NAME DB_NAME STATUS VERSION VERSION---------- ------ ------------- -------- --------- ---------- ----------DATA 1 racdb2 racdb CONNECTED 11.2.0.1.0 11.2.0.0.0DATA 1 +ASM2 +ASM CONNECTED 11.2.0.1.0 11.2.0.1.0

The asmcmd lsct provides similar information:$ asmcmd lsct

DB_Name Status Software_Version Compatible_version Instance_Name Disk_Group

+ASM CONNECTED 11.2.0.1.0 11.2.0.1.0 +ASM2 DATA

racdb CONNECTED 11.2.0.1.0 11.2.0.0.0 racdb2 DATA


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Dropping Disk Groups

The DROP DISKGROUP statement enables you to delete an ASM disk group and, optionally, all its files.

Examples:

DROP DISKGROUP diskgroup_name[ FORCE INCLUDING CONTENTS| { INCLUDING | EXCLUDING } CONTENTS]

;

DROP DISKGROUP DATA;

DROP DISKGROUP DATA2 INCLUDING CONTENTS;

DROP DISKGROUP DATA3 FORCE INCLUDING CONTENTS;

Dropping Disk GroupsThe DROP DISKGROUP statement enables you to delete an ASM disk group and, optionally, all its files. You can specify the INCLUDING CONTENTS clause if you also want to delete any files that might be contained in the disk group. The default is EXCLUDING CONTENTS, which prevents you from dropping the disk group if it has any contents.In order for the DROP DISKGROUP statement to succeed, the disk group must be mounted by the current ASM instance only and have none of the disk group files open. When you drop a disk group, ASM dismounts the disk group and removes the disk group name from the ASM_DISKGROUPS initialization parameter if an SPFILE file is being used. If a text initialization parameter file (PFILE) is being used, you must manually adjust the ASM_DISKGROUPS initialization parameter to make sure that ASM does not attempt to mount the dropped disk group the next time the instance starts.If you cannot mount a disk group but need to drop it, you can use the FORCE INCLUDING CONTENTS option. This enables you to remove the headers on disks that belong to a disk group that cannot be mounted by any ASM instances. When you use the FORCE INCLUDING CONTENTS option, the ASM instance does not attempt to verify that the disk group is being used by another ASM instance. The FORCE INCLUDING CONTENTS option can be useful in various unusual situations but should be used with extreme caution.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Checking the Consistency of Disk Group Metadata

You can check the internal consistency of disk group metadata using the ALTER DISKGROUP statement.

• This is conceptually the same as fsck in Linux or UNIX.• An error summary is returned by the statement.• Error details are written to the ASM instance alert log.• Use the REPAIR option to resolve inconsistencies.

– Similar to fsck -y in Linux or UNIX• Additional specific CHECK clauses have been deprecated.

ALTER DISKGROUP diskgroup_name CHECK [ REPAIR | NOREPAIR ];

Checking the Consistency of Disk Group MetadataYou can check the internal consistency of disk group metadata using the ALTER DISKGROUP statement with the CHECK keyword. The disk group must be mounted to perform these checks. The statement displays summary errors and details about the errors are written to the alert log. The CHECK keyword performs the following operations:

• Verifies the consistency of the disk• Cross-checks all the file extent maps and allocation tables for consistency• Checks that the alias metadata directory and file directory are linked correctly• Verifies that the alias directory tree is linked correctly• Checks that ASM metadata directories do not have unreachable allocated blocks

Note that these checks do not validate any stored database content such as index structures and table data.The REPAIR clause specifies that ASM should attempt to repair errors that are found during the check. Use the NOREPAIR clause to receive alerts about inconsistencies without ASM resolving the errors automatically. The default is NOREPAIR. In earlier releases, you could specify CHECK for ALL, DISK, DISKS IN FAILGROUP, or FILE. These additional clauses have been deprecated. Oracle recommends that you do not introduce these clauses into your new code because they are scheduled for desupport.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


ASM Fast Mirror Resync

Enabled when COMPATIBLE.RDBMS >= 11.1

ASM redundancy used

P P PS SS

Disk access failure(disk offline)

P P PS SS

Failure time < DISK_REPAIR_TIME

P--

P PS S--

S

Disk becomes accessible.Modified extents synchronized

P--

P--

PS--

S--

S

1 2

34

Primary extentSecondary extent


notes.

ASM Fast Mirror ResyncWhenever ASM is unable to write an extent, ASM takes the associated disk offline. If the corresponding disk group uses ASM mirroring (NORMAL or HIGH redundancy), at least one mirror copy of the same extent exists on another disk in the disk group.Before Oracle Database 11g, ASM assumed that an offline disk contains only stale data and no longer reads from such disks. Shortly after a disk is put offline, ASM drops it from the disk group by re-creating the extents allocated to the disk on the remaining disks in the disk group using mirrored extent copies. This process is quite resource intensive and can take hours to complete. If the disk is replaced or the failure is repaired, the disk must be added again and another rebalance operation must take place. ASM fast mirror resync significantly reduces the time required to resynchronize a transient failure of a disk. When a disk goes offline following a transient failure, ASM tracks the extents that are modified during the outage. When the transient failure is repaired, ASM can quickly resynchronize only the ASM disk extents that have been affected during the outage.Using ASM fast mirror resync, the failed disk is taken offline but not dropped if you have set the DISK_REPAIR_TIME attribute for the corresponding disk group. The setting for this attribute determines the duration of disk outage that ASM will tolerate while still being able to resynchronize after you complete the repair. Note that the tracking mechanism uses one bit for each modified extent and is very efficient.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Preferred Read Failure Groups

In a multisite cluster, with separate storage at each site, preferred read failure groups allow every node in the cluster toread from its local disks resulting in better performance.

Site B +ASM2Site A +ASM1

F3F1 F2F4

= DG1

= DG2

+ASM1.ASM_PREFERRED_READ_FAILURE_GROUPS = DG1.F1, DG2.F3+ASM2.ASM_PREFERRED_READ_FAILURE_GROUPS = DG1.F2, DG2.F4

Disk groups

Screen reader hint: This slide contains a diagram with an example of how ASM_PREFERRED_READ_FAILURE_GROUPS might be set. In this example, two disk groups,

DG1 and DG2, span two sites, Site A and Site B. Site A has ASM instance +ASM1 and houses failure groups DG1.F1 and DG2.F3. Site B has ASM instance +ASM2 and houses failure groups

DG1.F2 and DG2.F4. The suggested parameter setting for this scenario is:+ASM1.ASM_PREFERRED_READ_FAILURE_GROUPS = DG1.F1, DG2.F3+ASM2.ASM_PREFERRED_READ_FAILURE_GROUPS = DG1.F2, DG2.F4

Preferred Read Failure GroupsBefore Oracle Database 11g, ASM always read the primary copy of a mirrored extent. It may be more efficient for a node to read from a failure group extent that is closest to the node, even if it is a secondary extent. This is especially true in extended (multisite) cluster configurations where reading from a local copy of an extent provides improved performance. With Oracle Database 11g, you can do this by configuring the preferred mirror read using the new initialization parameter, ASM_PREFERRED_READ_FAILURE_GROUPS. The disks in the identified failure groups become the preferred read disks. Thus, every node can be configured to read from its local extent copy. This results in higher efficiency and performance, and reduced network traffic. The setting for this parameter is instance specific.If there is more than one mirrored copy and you have set a value for the ASM_PREFERRED_READ_FAILURE_GROUPS parameter, ASM first reads the copy that resides on a preferred read disk. If that read fails, ASM attempts to read from another mirrored copy that might not be on a local preferred read disk. You can use the PREFERRED_READ column in the V$ASM_DISK view to determine whether a particular disk belongs to a preferred read failure group.To identify performance issues with the ASM preferred read failure groups, use the V$ASM_DISK_IOSTAT view. This view displays input/output statistics for each ASM client.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Preferred Read Failure GroupsBest Practice

Two sites: Normal redundancy

Two failure groups: One for each site

P S

Three sites: High redundancy

Three failure groups: One for each site

S P S

Two sites: High redundancy

Four failure groups: Two for each site

P SSS PS


notes.

Site A Site B Site A Site B

Site A Site B Site C

Preferred Read Failure Groups Best PracticeIn practice, there are only a limited number of sensible disk group configurations in an extended cluster. A good configuration takes into account both performance and availability of a disk group in an extended cluster. Here are some possible examples:For a two-site extended cluster, a normal redundancy disk group should have only two failure groups; all disks local to each site should belong to the same failure group. Also, no more than one failure group should be specified as a preferred read failure group by each instance. If there are more than two failure groups, ASM may not mirror extents across both sites and you will not be protected against possible site failure. If the disk group is to be created as a high-redundancy disk group, at most two failure groups should be created on each site. This guarantees that at least one extent copy is located at each site. Both local failure groups should be specified as preferred read failure groups for the local instance.For a three-site extended cluster, a high-redundancy disk group with three failure groups should be used. In this way, ASM can guarantee that each extent has a mirror copy local to each site and that the disk group is protected against a catastrophic disaster on any of the three sites.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Viewing ASM Disk Statistics

• V$ASM_DISK_IOSTAT displays information about disk I/O statistics for each ASM client.

• If V$ASM_DISK_IOSTAT is queried from a database instance, only the rows relating to that instance are shown.

• ASMCMD lsdsk --statistics shows just the disk statistics.

• ASMCMD iostat shows a subset of the statistics depending on the option.

Viewing ASM Disk StatisticsV$ASM_DISK_IOSTAT relates information about disk I/O statistics to each database instance that is a client of ASM. The following example shows the use of V$ASM_DISK_IOSTAT to summarize I/O statistics for each instance and disk group combination:

SQL> SELECT INSTNAME, G.NAME DISKGROUP, SUM(READS) READS, 2 SUM(BYTES_READ) BYTES_READ, SUM(WRITES) WRITES, 3 SUM(BYTES_WRITTEN) BYTES_WRITTEN4 FROM V$ASM_DISK_IOSTAT I, V$ASM_DISKGROUP G5 WHERE I.GROUP_NUMBER = G.GROUP_NUMBER6 GROUP BY INSTNAME, G.NAME;

INSTNAME DISKGROUP READS BYTES_READ WRITES BYTES_WRITTEN-------- --------- ------- ----------- ------- -------------ORA11G1 FRA 730 12373504 20303 382233088ORA11G1 DATA 73619 1272131584 53549 953606656


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Viewing ASM Disk Statistics (continued) The following query quickly shows the presence of any I/O errors reported by ASM. If this query returns a value other than zero, further investigation may be undertaken:

SQL> SELECT SUM(READ_ERRS)+SUM(WRITE_ERRS) ERRORS2 FROM V$ASM_DISK;

ERRORS----------

0

The ASMCMD lsdsk --statistics command shows just the statistics columns.$ asmcmd lsdsk --statisticsReads Write Read_Errs Write_Errs Read_time Write_Time

Bytes_Read Bytes_Written Voting_File Path6033 42139 0 0 54.332 597.764

68358144 604267008 Y ORCL:DATA16278 11890 0 0 71.596 163.1

81133568 131769344 Y ORCL:DATA234902 39797 0 0 58.86 602.488

545398784 580783616 Y ORCL:DATA3191934 44743 0 0 162.28 620.704 3112785920 623890432 N ORCL:DATA4

The ASMCMD iostat command has more ways to see the statistics including errors. $ asmcmd iostat -eGroup_Name Dsk_Name Reads Writes Read_Err Write_Err DATA DATA1 69439488 612191744 0 0 DATA DATA2 81936384 132383744 0 0 DATA DATA3 553680896 588099072 0 0 DATA DATA4 3160528896 631962624 0 0


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Performance, Scalability, and Manageability Considerations for Disk Groups

• Create separate disk groups for database files and fast recovery area.

• Disks in a disk group should have the same size and performance characteristics.– Allows the disk group to deliver consistent performance– Allows ASM to use disk space most effectively– Allows operations with different storage requirements to be

matched with different disk groups effectively• Using separate disk groups for each database as opposed

to having multiple databases in a disk group has various benefits and drawbacks.

Performance, Scalability, and Manageability Considerations for Disk GroupsThere is no ideal number of disk groups. However, there are a few guiding principles that can help you decide how many disk groups you should create.

• Create separate disk groups for your database files and fast recovery area for backup files. This configuration allows fast recovery in case of a disk group failure.

• Disks in a disk group should have the same size and performance characteristics. If you have several different types of disks in terms of size and performance, create disk groups that contain disks with similar characteristics. ASM load-balances the file activity by uniformly distributing file extents across all the disks in a disk group. For this technique to be effective, it is important that disks in a disk group have similar performance characteristics. For example, the newest and fastest disks might reside in a disk group reserved for the database work area, and slower drives could reside in a disk group reserved for the fast recovery area.

• There are benefits and drawbacks associated with housing multiple databases in the same disk group as opposed to maintaining each database in a separate disk group.Housing multiple databases in a single disk group affords the most efficient use of space. However, any faults or maintenance that affects the disk group may affect many databases. Separate disk groups provide greater isolation from the effects of a fault or maintenance operation. However, to achieve this may consume more disk space and may require more disk group maintenance to balance disk resources.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Quiz

If you create a disk group and do not specify the REDUNDANCYclause, the default setting will be:1. EXTERNAL

2. NORMAL

3. HIGH

4. The default varies depending on the number of failure groups that are defined.

Answer: 2However, if only one failure group is specified, the CREATE DISKGROUP statement will fail. In any event, it is recommended that the REDUNDANCY clause should be specified explicitly because the setting cannot be changed after the disk group has been created.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Quiz

Adding a disk to a disk group will always cause a rebalance operation to occur.1. True2. False

Answer: 2If the ASM_POWER_LIMIT initialization parameter is set to zero or REBALANCE POWER 0is specified in the ALTER DISKGROUP statement, then rebalancing will be disabled.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Summary

In this lesson, you should have learned how to:• Create and delete ASM disk groups• Set the attributes of an existing ASM disk group• Perform ongoing maintenance tasks on ASM disk groups• Explain key performance and scalability considerations for

ASM disk groups


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Practice 8 Overview: Administering ASM Disk Groups

This practice covers the following topics:• Configuring disk groups • Adding and removing disks• Controlling rebalance operations• Monitoring disk and disk group I/O statistics


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Administering ASM Files, Directories, and Templates

Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Objectives

After completing this lesson, you should be able to:• Use different client tools to access Automatic Storage

Management (ASM) files• Describe the format of a fully qualified ASM file name• Explain how ASM files, directories, and aliases are created

and managed• Describe and manage disk group templates


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


ASM Clients

ASM

DBMS_FILE_TRANSFER

RMAN

ASMCMD

ASMCMD

>

SQL

SQL>

Database instance

XML DB repository

ftp>

http:/

/ftp>

http:/

/

SQL

SQL>


notes.

ACFS

ASM ClientsOracle 11g ASM is a volume manager and a file system for Oracle database files. This slide introduces various ASM clients. Many of these will be examined in greater detail later in this lesson. Clients that connect directly to ASM include:

• Oracle Database: Oracle Database is the most fundamental ASM client. It makes requests of ASM relating to numerous types of activities such as ASM File creation.

• ASM Cluster File System: This depends on ASM to provide the ASM volumes.• ASM Clusterware: If the Oracle Cluster Registry (OCR), voting files, or ASM server

parameter file (SPFILE) is stored on an ASM disk group, the ASM instance is a client of itself.

• ASMCA: ASM Configuration Manager is a graphical interface that allows you to manage ASM instances, disk groups, and volumes.

• Enterprise Manager: This allows you to manage and monitor the ASM instance directly, and indirectly through the database.

• Clusterware: Clusterware uses ASM to store the OCR and voting disk files by default.• SQL clients (such as SQL*Plus): A series of ASM-specific SQL commands (such as

CREATE DISKGROUP) provide the most fundamental management client for ASM.• ASMCMD: ASM command-line interface is used to interrogate and manage ASM. It

includes many UNIX-like commands that can be used to manage the files and directories in an ASM system.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

ASM Clients (continued)In addition to clients that connect to ASM directly, there are a series of interfaces provided by Oracle Database that can be used to manipulate ASM in various different ways. These include:

• SQL: When an Oracle Database is managed under ASM, activities that create and delete ASM files (such as CREATE DATABASE and DROP TABLESPACE) will implicitly interact with the underlying ASM instance. A database instance will also directly read and write ASM files as a result of SQL data manipulation language (DML) commands (INSERT, UPDATE, and DELETE). However, operations that involve space management (such as extending a data file) will also require interaction with the ASM instance.

• Oracle Recovery Manager (RMAN): RMAN is Oracle’s recommended backup and recovery utility. RMAN is well integrated with ASM and can be used to migrate non-ASM databases into ASM.

• XML DB: ASM files and directories can be accessed through a virtual folder in the XML DB repository. XML DB provides a means to access and manipulate the ASM files and directories with programmatic APIs, such as the DBMS_XDB package, and with XML DB protocol services such as FTP and HTTP/WebDAV.

• DBMS_FILE_TRANSFER: The DBMS_FILE_TRANSFER package provides procedures to copy ASM files within a database or to transfer binary files between a local ASM instance and a remote database file. DBMS_FILE_TRANSFER.COPY_FILE supports all transfer combinations involving ASM and/or your local file system, namely:

- Local file system to local file system- Local file system to ASM- ASM to local file system- ASM to ASM


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Interaction Between Database Instances and ASM

Database instance

ASM storage

ASM instance

Create file1

Initialize

4

Extent map3

Commit5

Allocation

2

Close6


notes.

Interaction Between Database Instances and ASMThe file creation process provides an illustration of the interactions that take place between database instances and ASM. The file creation process occurs as follows:

1. The database requests file creation. 2. An ASM foreground process creates a Continuing Operation Directory (COD) entry and

allocates space for the new file across the disk group. 3. A background database process (ASMB) receives an extent map for the new file.4. The file is now open and the database process initializes the file directly.5. After initialization, the database process requests that the file creation be committed.

This causes the ASM foreground process to clear the COD entry and mark the file as created.

6. Acknowledgement of the file commit implicitly closes the file. The database instance will need to reopen the file for future I/O.

There are two important points to consider from this example:• The database instance and ASM instance work together in a coordinated fashion. A

database instance must interact with ASM to map database files to ASM extents. It also receives a stream of messages relating to ASM operations (such as disk group rebalancing) that may lock or move ASM extents.

• Database I/O is not channeled through the ASM instance. In fact, the database conducts I/O operations directly against ASM files, as illustrated in step 4 in the slide.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Accessing ASM Files by Using RMAN

• RMAN should be used to back up ASM files.– RMAN is needed to unmap data file extent locations.– Third-party Oracle Database backup and recovery managers

also use RMAN to access ASM.• ASM can be used as a storage area for RMAN backups.• Depending on the context, RMAN can reference individual

ASM files or entire disk groups.

• RMAN can be used to migrate databases into ASM.– Migrating a database to ASM can be done one file at a time,

or all at once.

RMAN> BACKUP AS COPYDATAFILE "+DATA/rdbms/datafile/tbs_5.256.565313879"FORMAT "+DATA2";

Accessing ASM Files by Using RMANThe Oracle Recovery Manager (RMAN) is the preferred method for backup and recovery of databases contained in ASM. RMAN is very well integrated with ASM and is also used by third-party Oracle Database backup and recovery managers.RMAN can back up from and recover to ASM. It can also use ASM as a store for backup sets.Depending on the context, RMAN can reference individual ASM files or entire disk groups. For example, the following RMAN BACKUP command refers to an individual ASM file on the second line and also refers to an ASM disk group on the third line.

RMAN> BACKUP AS COPYDATAFILE "+DATA/rdbms/datafile/tbs_5.256.565313879"FORMAT "+DATA2";

RMAN can also be used to migrate existing databases into ASM. This can be done one file at a time or a complete database can be migrated into ASM in the same operation.The following provides an example of the procedure you can use to relocate your entire database to an ASM disk group (assuming the use of a server parameter file):

1. Obtain the file names of the current control files and online redo logs using V$CONTROLFILE and V$LOGFILE.

2. Shut down the database consistently.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Accessing ASM Files by Using RMAN (continued)3. Modify the server parameter file of your database as follows:

- Start the database with the NOMOUNT option.- Set the DB_CREATE_FILE_DEST parameter to the desired ASM disk group.- Remove the CONTROL_FILES parameter. It will be re-created automatically.

4. Edit to replace the placeholder file and disk group references with your actual locations, and then run the following RMAN command file. This backs up the database, switches the current data files to the backups, renames the online redo logs, and re-creates the temporary tablespaces.

RESTORE CONTROLFILE FROM '/u1/c1.ctl';ALTER DATABASE MOUNT;BACKUP AS COPY DATABASE FORMAT '+YOURDG';SWITCH DATABASE TO COPY;# Repeat command for all online redo log membersSQL "ALTER DATABASE RENAME '/u1/log1' TO '+YOURDG' ";ALTER DATABASE OPEN RESETLOGS;# Repeat commands for all temporary tablespacesSQL "ALTER TABLESPACE temp ADD TEMPFILE";SQL "ALTER DATABASE TEMPFILE '/u1/temp1' DROP";

5. Delete the old database files.Note: This example illustrates the procedure for migrating a database into ASM. You may want to use other options and settings to migrate your specific databases into ASM. For a complete discussion of this topic, refer to the Oracle Database Backup and Recovery User’s Guide 11g Release 2.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Accessing ASM Files by Using XML DB

• ASM files and directories can be accessed through the /sys/asm virtual folder in the XML DB repository. – Access is via PL/SQL APIs, FTP, and HTTP/WebDAV.

• The diagram shows the hierarchy under /sys/asm.


notes.

Accessing ASM Files by Using XML DBASM files and directories can be accessed through a virtual folder in the Oracle Database XML DB repository. The repository path to the virtual folder is /sys/asm. The folder is virtual because its contents do not actually reside in the repository; they exist as normal ASM files and directories. /sys/asm provides a means to access and manipulate the ASM files and directories with programmatic APIs, such as the DBMS_XDB package, and with XML DB protocols such as FTP and HTTP/WebDAV. You must log in to XML DB as a user other than SYS and that user must be granted the DBA role to access /sys/asm with XML DB protocols.A typical use for this capability might be to view /sys/asm as a Web folder in Windows Explorer, and then copy a Data Pump dumpset from an ASM disk group to an operating system file system by dragging and dropping.Under /sys/asm, the folder hierarchy is defined by the structure of an ASM fully qualified file name. That is, the /sys/asm virtual folder contains one subfolder for every mounted disk group, and each disk group folder contains one subfolder for each database that uses the disk group. In addition, a disk group folder might contain files and folders corresponding to aliases created by the administrator. Continuing the hierarchy, the database folders contain ASM file type folders, which contain the ASM files. Note: XML DB must be configured to enable FTP and HTTP/WebDAV. Use Enterprise Manager or the script provided at $ORACLE_HOME/rdbms/admin/catxdbdbca.sqlto do this.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Accessing ASM Files by Using DBMS_FILE_TRANSFER

• DBMS_FILE_TRANSFER provides procedures to:– Copy ASM files within a database– Transfer binary files in either direction between a local ASM

instance and a remote database file• DBMS_FILE_TRANSFER.COPY_FILE supports all transfer

combinations between ASM and local files.• Example:

– Copy a local database file into ASM.SQL> CREATE DIRECTORY dgb AS '+DATA/dbfiles'; SQL> CREATE DIRECTORY loc AS '/u01/app/oracle/oradata/db'; SQL> BEGIN

2 DBMS_FILE_TRANSFER.COPY_FILE('loc','tmp.dbf','dgb','tmp.dbf');3 END;4 /

Accessing ASM Files by Using DBMS_FILE_TRANSFERThe DBMS_FILE_TRANSFER package provides procedures to copy ASM files within a database or transfer binary files between databases that use ASM. The DBMS_FILE_TRANSFER package has the following procedures:• COPY_FILE: Reads a file from a source directory and creates a copy of the file in a

destination directory. The source and destination directories can both be in a local file system or in an ASM disk group. You can also use this procedure to copy between a local file system and an ASM disk group; the copy operation is valid in either direction.

• GET_FILE: Contacts a remote database to read a remote file and then creates a copy of the file in the local file system or ASM disk group

• PUT_FILE: Reads a local file or ASM disk group and contacts a remote database to create a copy of the file in the remote file system

When using DBMS_FILE_TRANSFER, note the following:• The size of the copied file must be a multiple of 512 bytes.• The size of the copied file must be less than or equal to two terabytes.• Transferring a file is not transactional. To guarantee consistency, bring files offline when

the database is in use.• The copied file is treated as a binary file, and no character set conversion is performed.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Accessing ASM Files by Using ASMCMD

• ASMCMD provides a command-line interface to interrogate and manage ASM.

• It includes many UNIX-like commands that can be used to manage the files and directories in an ASM system.

• Example AMSCMD session:[grid@host01 ~]$ asmcmdASMCMD> cd +DATA/racdb/DATAFILEASMCMD> ls -l SYS*Type Redund Striped Time Sys NameDATAFILE MIRROR COARSE AUG 03 16:00:00 Y SYSAUX.257.692926339

DATAFILE MIRROR COARSE AUG 03 16:00:00 Y SYSTEM.256.692926339ASMCMD> cd ..ASMCMD> pwd+DATA/racdb/DATAFILEASMCMD> exit[grid@host01 ~]$

Accessing ASM Files by Using ASMCMDASMCMD is a command-line utility that you can use to view and manipulate files and directories within ASM disk groups. ASMCMD can list the contents of disk groups, perform searches, create and remove directories and aliases, display space utilization, and more. When you run ASMCMD, you connect to an ASM instance and you are provided a command line where you can execute any of the ASMCMD commands. The ASMCMD commands include many commands that perform the same basic functions as their UNIX/Linux counterparts. These include: cd, cp, du, find, ls, mkdir, pwd, and rm. There are additional commands that perform ASM-specific functions.ASMCMD commands are discussed in parallel with the other utilities for performing ASM administration tasks.When you invoke the ASMCMD utility from the Grid software $ORACLE_HOME, the utility attempts to connect AS SYSASM. The ASMCMD utility attempts to connect AS SYSASM, so the user that invokes ASMCMD must be a member of the OSASM group. When connecting from a $ORACLE_HOME database, ASMCMD attempts to connect AS SYSDBA. The OS user must be a member of the OSDBA group for ASM. Set the ORACLE_SID environment variable to the local name of the database instance. Any OS user who is a member of the appropriate OS group may connect with ASMCMD to a local instance, and specify the connect role with the -a option.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Fully Qualified ASM File Names

• Every file created in ASM gets a system-generated file name, known as the fully qualified file name. – You cannot set the fully qualified file name.– ASM guarantees uniqueness within the ASM environment.– The fully qualified file name is used in database views that

display Oracle Database file names.• Format:

—

• Examples:+DATA/ORA11G/DATAFILE/SYSTEM.262.676172197+DATA/ORA11G/PARAMETERFILE/spfile.268.676172697+DATA/ORA11G/CONTROLFILE/Current.257.676172363+DATA/ORA11G/ONLINELOG/group_1.256.676172371+FRA/ORA11G/CONTROLFILE/Backup.275.676172942

<+group>/<dbname>/<file_type>/<file_type_tag>.<file>.<incarnation>

Fully Qualified ASM File NamesEvery file created in ASM gets a system-generated file name, known as the fully qualified file name. You cannot set the fully qualified file name. The fully qualified file name represents a complete path name in the ASM file system. An example of a fully qualified file name is:

+DATA/racdb/DATAFILE/SYSTEM.262.676172197

You can use the fully qualified file name to reference (read or retrieve) an ASM file. You can also use other abbreviated file name formats, such as an alias ASM file name (discussed later in this lesson) to reference an ASM file. You can find the fully qualified file name in database views displaying Oracle Database file names, such as V$DATAFILE and V$LOGFILE. A fully qualified file name has the following form:


where:• +group is the disk group name preceded by a plus sign. You can think of the plus sign

(+) as the root directory of the ASM file system, similar to the slash (/) on UNIX or Linux file systems.

• dbname is the DB_UNIQUE_NAME of the database to which the file belongs.• file_type is the Oracle Database file type.• file_type_tag is type-specific information about the file.• file.incarnation is the file/incarnation pair, used to ensure uniqueness.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Fully Qualified ASM File Names (continued)The following table shows possible values for the file_type and file_type_tagcomponents of a fully qualified file name:

file_type file_type_tag Description

CONTROLFILE Current or Backup Control files and backup control files

DATAFILE <tsname> Data file and data file copies. <tsname>is the tablespace containing the data file.

ONLINELOG group_<group#> Online logs

ARCHIVELOG thread_<thread#>_seq_<sequence#>

Archive logs

TEMPFILE <tsname> Temporary tablespace file. <tsname> is the tablespace containing the data file.

BACKUPSET <sp>_<timestamp> Data file backup, data file incremental backup, or archive log backup pieces. <sp> has the value s when the backup set includes the spfile; n indicates that the backup set does not include the spfile.

PARAMETERFILE spfile Server parameter files

DATAGUARDCONFIG <dbname> Data Guard configuration file. <dbname>is the value of the DB_UNIQUE_NAMEdatabase initialization parameter.

FLASHBACK log_<log#> Flashback logs

CHANGETRACKING ctf Block change tracking file, used in incremental backups

DUMPSET <user>_<job#>_<file#>

Data Pump dumpset. Dumpset files encode the username, the job number that created the dump set, and the file number as part of the tag.

XTRANSPORT <tsname> Cross-platform transportable tablespace data files. <tsname> is the tablespace containing the data file.

AUTOBACKUP <sp>_<timestamp> Automatically generated control file backup. <sp> has the value s when the backup set includes the spfile; nindicates that the backup set does not include the spfile.

ASMPARAMETERFILE

registry Name of the Oracle ASM SPFILE

OCRFILE registry Name of the OCR files


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Other ASM File Names

• Numeric ASM file name

• Alias ASM file name

• Alias ASM file name with template

• Incomplete ASM file name

• Incomplete ASM file name with template

Format: <+group>.<file>.<incarnation>Example: +DATA.262.676172197

Format: <+group>/<alias>Example: +DATA/my_dir/my_other_dir/my_file_name.dbf

Format: <+group>(<template>)/<alias>Example: +DATA(my_template)/my_dir/my_other_file_name.dbf

Format: <+group>

Format: <+group>(<template>)Example: +DATA(my_template)

Other ASM File NamesIn addition to the fully qualified file name, the following other ASM file names can be used in various different situations:Numeric ASM File NameThe numeric ASM file name can be used for referencing existing files. It is derived from the fully qualified ASM file name and takes the form:

+group.file.incarnation

Alias ASM File NameAlias ASM file names, or simply aliases, can be used both for referencing existing ASM files and for creating new ASM files. Alias names start with the disk group name preceded by a plus sign, after which you specify a name string of your choosing. Alias file names are implemented using a hierarchical directory structure, with the slash (/) or backslash (\) character separating name components. You can create an alias in any system-generated or user-created ASM directory already in existence except for the root (+) level.Alias ASM file names are distinguished from fully qualified file names or numeric file names because they do not end in a dotted pair of numbers. It is an error to attempt to create an alias that ends in a dotted pair of numbers.You can think of aliases as achieving the same function as symbolic links in a UNIX or Linux environment.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Other ASM File Names (continued)Alias ASM File Name with TemplateAn alias ASM file name with template is essentially the same as an alias ASM file name. The only difference is that a file created using this type of file name receives the mirroring and striping attributes specified by the named template. The template must belong to the disk group that the file is being created in. An alias ASM file name with template is used only for ASM file creation operations. Incomplete ASM File NameIncomplete ASM file names are used only for file creation operations. An incomplete ASM file name is just a reference to a disk group name. When incomplete ASM file names are used, ASM automatically creates fully qualified file names under the specified disk group.Incomplete ASM File Name with TemplateIncomplete ASM file names with templates are also used only for file creation operations. They consist of the disk group name followed by the template name in parentheses. This explicit template reference causes ASM to use the specified template to determine mirroring and striping attributes for the file, instead of the default template for that file type.Incomplete ASM file names, with and without templates, can be used in conjunction with the Oracle Managed Files (OMF) feature of Oracle Database. If the DB_CREATE_FILE_DESTinitialization parameter for the database is set to an incomplete ASM file name, it is possible to create a tablespace without any further reference to a file name.For example, assume the following initialization parameter setting:

DB_CREATE_FILE_DEST = '+DATA'

The following statement creates the tspace1 tablespace.CREATE TABLESPACE tspace1;

ASM automatically creates and manages a data file for tspace1 on ASM disks in the DATAdisk group. File extents are stored using the attributes defined by the default template for a data file.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Valid Contexts for the ASM File Name Forms

File Name Form Valid to Reference Existing Files

Valid for Single File Creation

Valid for Multiple File Creation

Fully Qualified Yes No No

Numeric Yes No No

Alias Yes if defined Yes No

Alias with Template No Yes No

Incomplete No Yes Yes

Incomplete with Template No Yes Yes

Valid Contexts for the ASM File Name FormsThe table in the slide specifies the valid contexts for each file name form.A single file creation is an operation that creates a single file, such as a data file or a control file. You can use an alias or incomplete file name, where a file name is called for in a statement to create a single file, such as CREATE TABLESPACE or CREATE CONTROLFILE.A multiple file creation is an operation that can create multiple ASM files. An example of such an operation would be a single ALTER DATABASE statement that adds a new online redo log group with multiple members.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Single File Creation Examples

• Alias ASM file name

• Alias file name with template

• Incomplete file name

SQL> ALTER TABLESPACE myspace ADD2 DATAFILE '+DATA/mydir/myspace02.dbf' SIZE 50M;

SQL> ALTER TABLESPACE myspace ADD2 DATAFILE '+DATA(mytemplate)/mydir/myspace03.dbf';

SQL> ALTER TABLESPACE myspace ADD2 DATAFILE '+DATA' SIZE 50M;

Single File Creation ExamplesIn a single file creation operation, you specify one of the following ASM file name forms instead of a local file name. As previously mentioned, the valid forms for single file creation operations are:

• Alias ASM file name• Alias ASM file name with template• Incomplete ASM file name• Incomplete ASM file name with template

Note: When the SIZE parameter is not given, the SIZE defaults to 100M.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Multiple File Creation Example

• Create an online redo log group with two members.– Set the following database initialization parameters:

– Then create the log group without any file references:

• The resulting fully qualified file names are:

SQL> ALTER DATABASE ADD LOGFILE;

+DATA/ORA11G/ONLINELOG/group_5.269.699798885+FRA/ORA11G/ONLINELOG/group_5.256.699799169

DB_CREATE_ONLINE_LOG_DEST_1 = '+DATA'DB_CREATE_ONLINE_LOG_DEST_2 = '+FRA'

Multiple File Creation ExampleMultiple file creation operations work only in conjunction with incomplete ASM file names. In such operations, the incomplete file name reference is implied through the use of the following database initialization parameters:

• DB_CREATE_FILE_DEST• DB_CREATE_ONLINE_LOG_DEST_<n> (where <n> = 1, 2, 3, 4 or 5)

The slide example illustrates how an online redo log group containing two members is created without any explicit file references.Although it is recommended that you use the Database Configuration Assistant (DBCA) to create databases, it is possible to set DB_CREATE_FILE_DEST to an ASM disk group and create a complete database using the following simple statement:

CREATE DATABASE sample;

This statement creates a database with at least the following ASM files:• A SYSTEM tablespace data file in the disk group specified in

DB_CREATE_FILE_DEST• A SYSAUX tablespace data file in the disk group specified in

DB_CREATE_FILE_DEST• A multiplexed online redo log with two log groups. Each log group will have one

member in the disk group specified in DB_CREATE_FILE_DEST and a second member in the disk group specified in DB_RECOVERY_FILE_DEST.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


View ASM Aliases, Files, and Directories

• In an ASM instance, V$ASM_ALIAS displays one row for each alias, system-generated file name, and directory present in every disk group mounted by the ASM instance.

• V$ASM_ALIAS contains the full hierarchy of aliases, files, and directories.– Use a hierarchical query to show this.

• In a database instance, V$ASM_ALIAS displays no rows.• The asmcmd ls command behaves much like the UNIX

ls command.

View ASM Aliases, Files, and DirectoriesV$ASM_ALIAS contains a record for each system-generated file name, user-defined alias, and directory for every currently mounted disk group. Use a hierarchical query on V$ASM_ALIAS to reconstruct the hierarchy of files, aliases, and directories in ASM. Join V$ASM_ALIAS with V$ASM_FILE to display additional file information, such as the file type. V$ASM_ALIAS is also commonly joined with V$ASM_DISKGROUP to display the disk group name.The asmcmd ls -l command lists the contents of a directory:ASMCMD> ls -l +DATA/RACDB/DATAFILE

Type Redund Striped Time Sys Name

DATAFILE MIRROR COARSE JUL 31 14:00:00 Y EXAMPLE.264.692926563

DATAFILE MIRROR COARSE JUL 31 14:00:00 Y SYSAUX.257.692926339

DATAFILE MIRROR COARSE JUL 31 14:00:00 Y SYSTEM.256.692926339

DATAFILE MIRROR COARSE JUL 31 14:00:00 Y UNDOTBS1.258.692926341

DATAFILE MIRROR COARSE JUL 31 14:00:00 Y UNDOTBS2.265.692926841

DATAFILE MIRROR COARSE JUL 31 14:00:00 Y USERS.259.692926341

The following query shows an example of how to reconstruct the ASM file hierarchy using V$ASM_ALIAS in combination with other V$ASM views:


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

View ASM Aliases, Files, and Directories (continued)

SQL> SELECT CONCAT('+'||GNAME, SYS_CONNECT_BY_PATH(ANAME, '/')) 2 FULL_PATH, SYSTEM_CREATED, ALIAS_DIRECTORY, FILE_TYPE3 FROM ( SELECT B.NAME GNAME, A.PARENT_INDEX PINDEX, 4 A.NAME ANAME, A.REFERENCE_INDEX RINDEX, 5 A.SYSTEM_CREATED, A.ALIAS_DIRECTORY,6 C.TYPE FILE_TYPE7 FROM V$ASM_ALIAS A, V$ASM_DISKGROUP B, V$ASM_FILE C8 WHERE A.GROUP_NUMBER = B.GROUP_NUMBER9 AND A.GROUP_NUMBER = C.GROUP_NUMBER(+)

10 AND A.FILE_NUMBER = C.FILE_NUMBER(+)11 AND A.FILE_INCARNATION = C.INCARNATION(+)12 )13 START WITH (MOD(PINDEX, POWER(2, 24))) = 014 CONNECT BY PRIOR RINDEX = PINDEX;

FULL_PATH S A FILE_TYPE------------------------------------------------------- - - --------------

--+DATA/cluster03 Y Y+DATA/cluster03/ASMPARAMETERFILE Y Y+DATA/cluster03/ASMPARAMETERFILE/REGISTRY.253.692923731 Y N

ASMPARAMETERFILE+DATA/cluster03/OCRFILE Y Y+DATA/cluster03/OCRFILE/REGISTRY.255.692923735 Y N OCRFILE+DATA/RACDB Y Y+DATA/RACDB/DATAFILE Y Y+DATA/RACDB/DATAFILE/UNDOTBS1.258.692926341 Y N DATAFILE+DATA/RACDB/DATAFILE/EXAMPLE.264.692926563 Y N DATAFILE+DATA/RACDB/DATAFILE/UNDOTBS2.265.692926841 Y N DATAFILE+DATA/RACDB/DATAFILE/USERS.259.692926341 Y N DATAFILE+DATA/RACDB/DATAFILE/SYSTEM.256.692926339 Y N DATAFILE+DATA/RACDB/DATAFILE/SYSAUX.257.692926339 Y N DATAFILE+DATA/RACDB/CONTROLFILE Y Y+DATA/RACDB/CONTROLFILE/Current.260.692926519 Y N CONTROLFILE+DATA/RACDB/ONLINELOG Y Y+DATA/RACDB/ONLINELOG/group_1.261.692926525 Y N ONLINELOG+DATA/RACDB/ONLINELOG/group_2.262.692926529 Y N ONLINELOG+DATA/RACDB/ONLINELOG/group_4.267.692926921 Y N ONLINELOG+DATA/RACDB/ONLINELOG/group_3.266.692926919 Y N ONLINELOG+DATA/RACDB/TEMPFILE Y Y+DATA/RACDB/TEMPFILE/TEMP.263.692926547 Y N TEMPFILE+DATA/RACDB/PARAMETERFILE Y Y+DATA/RACDB/PARAMETERFILE/spfile.268.692926927 Y N PARAMETERFILE+DATA/RACDB/spfileracdb.ora N N PARAMETERFILE

25 rows selected.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Viewing ASM Files

• In an ASM instance, V$ASM_FILE displays one row for every ASM file in every disk group mounted by the ASM instance.

• In a database instance, V$ASM_FILE displays no rows.

Viewing ASM FilesV$ASM_FILE contains information about the physical attributes of ASM files. This includes settings for striping, redundancy, and block size, along with information about the file size, type, redundancy status, creation date, and date of last modification. Because it does not contain directory path information, V$ASM_FILE is often joined with V$ASM_ALIAS and V$ASM_DISKGROUP to construct fully qualified file names and full alias paths. See the description of V$ASM_ALIAS earlier in this lesson for an example.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


ASM Directories

• ASM disk groups contain the following system-generated hierarchical directory structure for storing ASM files.

• You can create your own directories within this hierarchy to store aliases that you create.

ASM DirectoriesASM disk groups contain a system-generated hierarchical directory structure for storing ASM files. The system-generated file name that ASM assigns to each file represents a path in this directory hierarchy. The following is an example of a system-generated file name:

+DATA/racdb/DATAFILE/SYSTEM.262.676172197

The plus sign represents the root of the ASM file system. The DATA directory is the parent directory for all files in the DATA disk group. The racdb directory is the parent directory for all files in the racdb database, and the DATAFILE directory contains all data files for the racdb database.You can create your own directories within this hierarchy to store aliases that you create. Thus, in addition to having user-friendly alias names for ASM files, you can have user-friendly paths to those names. For example, the following user-defined directory might be used to store a collection of alias file names:

+DATA/racdb/my_directory

User-defined directories may be created at any level below the disk group directories. That is, you cannot create a user-defined directory at the root (+) level.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Managing ASM Directories

Use the ALTER DISKGROUP statement to create, rename, and drop ASM directories.

Examples:

ALTER DISKGROUP diskgroup_name{ ADD DIRECTORY 'dir_name' [, 'dir_name' ]...| DROP DIRECTORY

'dir_name' [ FORCE | NOFORCE ] [, 'dir_name' [ FORCE | NOFORCE ]]...

| RENAME DIRECTORY'old_dir_name' TO 'new_dir_name'[, 'old_dir_name' TO 'new_dir_name' ]...

};

SQL> ALTER DISKGROUP DATA ADD DIRECTORY '+DATA/mydir';

SQL> ALTER DISKGROUP DATA 2 RENAME DIRECTORY '+DATA/mydir' TO '+DATA/myotherdir';

Managing ASM DirectoriesYou can use the ALTER DISKGROUP statement to create, rename, and drop user-defined directories. You cannot rename or drop system-created directories.When using the CREATE DIRECTORY clause, you cannot create a nested directory unless the parent directory already exists. For example, the following statement fails unless first_diralready exists:

ALTER DISKGROUP DATAADD DIRECTORY '+DATA/first_dir/second_dir';

When using the DROP DIRECTORY clause, you cannot drop a directory containing alias names unless you also specify the FORCE clause. If you specify DROP DIRECTORY ... FORCE, ASM will recursively remove all the subdirectories and aliases that exist below the specified directory.The ASM Command-Line utility (ASMCMD) can also be used to manage ASM directories. The ASMCMD commands:• mkdir: Makes directory• mv: Renames a file or directory• rm: Removes a file or directory and its aliases• rmalias: Removes the alias only

You can also perform all these operations from Enterprise Manager.Oracle 11g: RAC and Grid Infrastructure Administration Accelerated 9 - 22

Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Managing Alias File Names

Use the ALTER DISKGROUP statement to create, rename, and drop ASM aliases.

Examples:

ALTER DISKGROUP diskgroup_name{ ADD ALIAS 'alias_name' FOR 'filename'

[, 'alias_name' FOR 'filename' ]...| DROP ALIAS 'alias_name' [, 'alias_name' ]...| RENAME ALIAS 'old_alias_name' TO 'new_alias_name'

[, 'old_alias_name' TO 'new_alias_name' ]...}

;

SQL> ALTER DISKGROUP DATA ADD ALIAS '+DATA/mydir/system.dbf'2 FOR '+DATA/sample/datafile/system.262.676172197';

SQL> ALTER DISKGROUP DATA RENAME ALIAS '+DATA/mydir/datafile.dbf'2 TO '+DATA/payroll/compensation.dbf';

Managing Alias File Names ASM alias file names, or aliases, provide a more user-friendly means of referring to ASM files, rather than the system-generated file names. For example, the following ALTER DATABASE statement uses an alias to the data file that is being taken offline:

ALTER DATABASE DATAFILE '+DATA/mydir/myfile.dbf' OFFLINE;

You can create an alias for a file when you create it in the database. For example:CREATE TABLESPACE myts

DATAFILE '+DATA/mydir/myts.dbf' SIZE 50M ONLINE;

The alias name is created automatically, but the command fails if the directory alias does not already exist. Note: A file created in this manner is not an OMF file.You can add an alias to an existing file by using the ADD ALIAS clause of the ALTER DISKGROUP statement. You can create an alias in any system-generated or user-created ASM directory already in existence. However, you cannot create an alias at the root level (+). You cannot create multiple aliases for the same ASM file.The ALTER DISKGROUP statement also provides clauses to RENAME and DROP existing aliases. Dropping an alias does not drop the referenced file. If you use the ALTER DISKGROUP ... DROP FILE statement to drop an ASM file, the alias associated with the file is also removed.ASMCMD can also be used to manage ASM aliases.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


ASM Intelligent Data Placement

ASM Intelligent Data Placement (IDP):• Places files in hot or cold regions based on:

– File attribute– Template

• Improves performance by:– Placing frequently accessed files in the hot region

• Improves disk usage by:– Placing infrequently used files in the cold region

IDP attribute changes to existing data will be implemented bythe next rebalance operation.

ASM Intelligent Data PlacementIt is known that access to the data furthest from the spindle of a disk can be as much as 50% faster than for the data closest to the spindle. So placing frequently accessed data in the disk region furthest from the spindle will increase performance. This region is called the hot region and the region closest to the spindle, the cold region. Some customers would never use the cold region of the disk nearest the spindle due to the increased latency. With Intelligent Data Placement, you can assign an ASM file to a disk region for best performance and use the lower performance region for less frequently accessed files. Using the disk region settings, you can ensure that frequently accessed data is placed on the outermost tracks (hot region). In addition, when files with similar access patterns are located physically close together, latency is reduced. Intelligent Data Placement also allows the placement of primary and mirror extents in different hot or cold regions.Intelligent Data Placement settings can be specified for a file or in disk group templates. The disk region settings for a file can be modified after the file has been created. The disk region setting can improve I/O performance by placing more frequently accessed data in regions furthest from the spindle, while reducing your cost by increasing the usable space on a disk.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Guidelines for Intelligent Data Placement

Intelligent Data Placement works best for: • Data files that are accessed at different rates• Disk groups more than 50% full • Disks with the geometry exposed to ASM (JBOD)

Intelligent Data Placement requires disk group attributes.• COMPATIBLE.ASM = 11.2.0 or higher• COMPATIBLE.RDBMS = 11.2.0 or higher

Guidelines for Intelligent Data PlacementIntelligent Data Placement works best for the following:

• Databases with data files that are accessed at different rates. A database that accesses all data files in the same way is unlikely to benefit from Intelligent Data Placement.

• Disk groups that are more than 50% full. Disk groups less than 50% full may see some performance benefit. For disk groups less than 50% full, the management overhead is likely to cancel any benefit. By default, the disk fills from the outer (hot) sectors, so the first 25% to 50% of the disk is in the hot region and enjoying the faster access. The cold region extents follow the hot region extents, but if all the extents are in the outer sectors of the disk, all would be accessed as though they were marked hot. As the disk fills, the performance difference between files marked hot and cold should become more pronounced. The actual performance benefit will depend on the file access pattern.

• Disks that have better performance at the beginning of the media relative to the end. Because Intelligent Data Placement leverages the geometry of the disk, it is well suited to just a bunch of disks (JBOD). In contrast, a storage array with logical units (LUNs)composed of concatenated volumes masks the geometry from Oracle ASM. When LUNs are used, Intelligent Data Placement will use the logical sector numbering, which may have no relation to the physical disk layout.

The COMPATIBLE.ASM and COMPATIBLE.RDBMS disk group attributes must be set to 11.2 or higher to use Intelligent Data Placement.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Assigning Files to Disk Regions

• Create a file template:ALTER DISKGROUP data ADD TEMPLATE datafile_hot

ATTRIBUTES ( HOT MIRRORHOT);

• Alter the file attributes:ALTER DISKGROUP data MODIFY FILE

'+data/orcl/datafile/users.259.679156903'

ATTRIBUTES (COLD MIRRORCOLD);

Assigning Files to Disk RegionsThere are two methods for assigning a file to a region. The first is with a template. Set the primary region and mirror region attributes to the desired regions, and then use the template to create the ASM file.The second method is to change the region attributes of the file. You can change these attributes either through SQL*Plus with the command shown in the slide or using Enterprise Manager as shown in a following slide. Changing the assigned region of the disk for a file does not trigger a rebalance operation. If enough space exists in the proper region, any new extents are created in the proper region. The region specification sets a preferred policy, not a commitment. The next rebalance operation, after any IDP changes that require existing data to be moved, will attempt to move extents into the specified region.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Assigning Files to Disk Regions with Enterprise Manager

Assigning Files to Disk Regions with Enterprise ManagerAn ASM file can be assigned to a disk region through Enterprise Manager. On the Edit File page, you can assign the primary and mirror extents of a file to either the hot or cold region. An online log file—an example is shown in the slide—with a high volume of writes should be assigned to the hot region for both the primary and mirror extents. The default settings of cold region are shown in the slide’s example.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Monitoring Intelligent Data Placement

Performance benefit is dependent on: • File access pattern• Correct file access evaluation

Use performance views to select the candidate files:• V$ASM_FILE

• V$ASM_DISK

• V$ASM_DISK_STAT

Performance benefit can be verified by testing.• Review Automatic Workload Repository (AWR) reports

before and after making a change.• Use V$FILEMETRIC and V$FILEMETRIC_HISTORY.

Monitoring Intelligent Data PlacementIt is possible that inappropriate file policy specification can degrade performance. Measuring the frequency of accesses to each file will be an important tool for diagnoses. Columns have been added to ASM-fixed tables that will provide this information.V$ASM_FILE has the following columns for each file: HOT_READS, HOT_WRITES, HOT_BYTES_READ, HOT_BYTES_WRITTEN, COLD_READS, COLD_WRITES, COLD_BYTES_READ, and COLD_BYTES_WRITTEN. These columns will help you to select the files that should be moved to another region and verify that the selection is correct. The V$ASM_DISK and V$ASM_DISK_STAT views have the same statistics as V$ASM_FILE, but aggregated at the disk level. These statistics can be viewed with the ASMCMD command: iostat -–region.To verify that the change of regions is actually changing the performance, record the AVERAGE_READ_TIME, AVERAGE_WRITE_TIME, PHYSICAL_READS, and PHYSICAL_WRITES from V$FILEMETRIC and V$FILEMETRIC_HISTORY before and after the change, or use AWR reports (especially the File I/O statistics in the I/O Stats section). The ASMCMD command iostat --region -t will list these statistics.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Disk Group Templates

• Templates are used to set striping, redundancy, and region attributes for ASM files.

• Striping attribute options are:– FINE: 128 KB stripe size– COARSE: 1 allocation unit (AU_SIZE attribute) stripe size

• Redundancy attribute options are:– MIRROR: Two-way mirroring– HIGH: Three-way mirroring– UNPROTECTED: No ASM mirroring

• A set of default templates is created for each disk group.– Default template settings depend on disk group redundancy.

• The redundancy attribute applies only to NORMALredundancy disk groups.

Disk Group TemplatesTemplates are used to set redundancy (mirroring) and striping attributes of each individual file created in an ASM disk group. When a file is created, redundancy and striping attributes are set for that file based on an explicitly named template or the system template that is the default template for the file type.When a disk group is created, ASM creates a set of default templates for that disk group. The set consists of one template for each file type (data file, control file, redo log file, and so on) that is supported by ASM. For example, a template named ONLINELOG provides the default file redundancy and striping attributes for all redo log files written to that disk group. Default template settings for redundancy depend on the disk group type. The default template for data files for a normal redundancy disk group sets two-way mirroring, whereas the corresponding default template in a high redundancy disk group sets three-way mirroring. You can modify the default templates.The striping attribute of templates applies to all disk group types (normal redundancy, high redundancy, and external redundancy). In practice, the redundancy attribute of templates applies only to normal redundancy disk groups. It is effectively ignored for high-redundancy disk groups in which every file is always three-way mirrored. It is also effectively ignored for external redundancy disk groups in which no files are mirrored by ASM. Nevertheless, each type of disk group gets a full set of templates, and the redundancy value in each template is always set to the proper default for the disk group type.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Disk Group Templates (continued)The following table lists the default templates and the attributes that are associated with different ASM file types. As the table shows, the initial redundancy value of each default template depends on the type of disk group that the template belongs to. The region values for all the system templates are set to COLD and MIRRORCOLD.

Template Name (File Type)

Striping Mirroring, Normal Redundancy Disk Group

Mirroring, High Redundancy Disk Group

Mirroring, External Redundancy Disk Group

CONTROLFILE FINE HIGH HIGH UNPROTECTED

DATAFILE COARSE MIRROR HIGH UNPROTECTED

ONLINELOG COARSE MIRROR HIGH UNPROTECTED

ARCHIVELOG COARSE MIRROR HIGH UNPROTECTED

TEMPFILE COARSE MIRROR HIGH UNPROTECTED

BACKUPSET COARSE MIRROR HIGH UNPROTECTED

PARAMETERFILE COARSE MIRROR HIGH UNPROTECTED

DATAGUARDCONFIG COARSE MIRROR HIGH UNPROTECTED

FLASHBACK COARSE MIRROR HIGH UNPROTECTED

CHANGETRACKING COARSE MIRROR HIGH UNPROTECTED

DUMPSET COARSE MIRROR HIGH UNPROTECTED

XTRANSPORT COARSE MIRROR HIGH UNPROTECTED

AUTOBACKUP COARSE MIRROR HIGH UNPROTECTED

OCRFILE COARSE MIRROR HIGH UNPROTECTED

ASMPARAMETERFILE

COARSE MIRROR HIGH UNPROTECTED


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


• In either an ASM or database instance, V$ASM_TEMPLATEdisplays one row for every template present in every disk group mounted by the ASM instance.

• The ASMCMD lstmpl command shows all the templates for a given disk group.

$ asmcmd lstmpl -l -G data

Group Group

Name Num Name Stripe Sys Redund PriReg MirrReg

DATA 1 ARCHIVELOG COARSE Y MIRROR COLD COLD

DATA 1 ASMPARAMETERFILE COARSE Y MIRROR COLD COLD

DATA 1 CONTROLFILE FINE Y HIGH COLD COLD

DATA 1 DATAFILE COARSE Y MIRROR COLD COLD

…

Viewing Templates

Viewing TemplatesThe V$ASM_TEMPLATE view displays disk group templates and their attribute settings. The following query lists all the templates associated with the DATA disk group. The SYSTEMcolumn returns Y for system-generated templates and N for user-defined templates.

SQL> SELECT T.NAME, T.REDUNDANCY, T.STRIPE, T.SYSTEM2 FROM V$ASM_TEMPLATE T, V$ASM_DISKGROUP G3 WHERE T.GROUP_NUMBER = G.GROUP_NUMBER4 AND G.NAME = 'DATA';

NAME REDUNDANCY STRIPE SYSTEM------------------------- ---------- ------ ------PARAMETERFILE MIRROR COARSE YDUMPSET MIRROR COARSE YNOMIRROR UNPROT COARSE NDATAFILE MIRROR COARSE YONLINELOG MIRROR FINE YARCHIVELOG MIRROR COARSE YCONTROLFILE HIGH FINE Y...

Note: The output from this example has been truncated for page-formatting purposes.Oracle 11g: RAC and Grid Infrastructure Administration Accelerated 9 - 31

Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Managing Disk Group Templates

Use the ALTER DISKGROUP statement to add, modify, and drop disk group templates.

Examples:

ALTER DISKGROUP diskgroup_name{ { ADD | MODIFY } TEMPLATE

qualified_template_clause [, qualified_template_clause ]...| DROP TEMPLATE template_name [, template_name ]...}

;

qualified_template_clause ::= template_name ATTRIBUTES([ MIRROR | HIGH | UNPROTECTED ] [ FINE | COARSE ])

SQL> ALTER DISKGROUP DATA ADD TEMPLATE unprot ATTRIBUTES (UNPROTECTED);

SQL> ALTER DISKGROUP DATA ADD TEMPLATE reliable ATTRIBUTES (HIGH FINE);

SQL> ALTER DISKGROUP DATA MODIFY TEMPLATE DATAFILE ATTRIBUTES (FINE);

Managing Disk Group TemplatesCreating your own templates allows you to set the right combination of attributes to meet your requirements. It also allows you to specify an intuitive name for the attribute set. For example, in a normal or high redundancy disk group, you might use a template with a redundancy setting of UNPROTECTED to create transient files without the overhead of mirroring. This can be a useful way of creating short-term databases for testing and development purposes.Using the ALTER DISKGROUP statement, you can add new templates to a disk group, modify existing ones, or drop templates.When adding a new template, you can omit either the redundancy or the stripe attribute setting. If you omit an attribute setting, it will default as follows:

• For the redundancy attribute, the value defaults to MIRROR for a normal redundancy disk group, HIGH for a high redundancy disk group, and UNPROTECTED for an external redundancy disk group.

• For the stripe attribute, the value defaults to COARSE.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Managing Disk Group Templateswith ASMCMD

• Use the mktmpl statement to add disk group templates.

• Use the chtmpl command to modify disk group templates.

• Use the rmtmpl command to drop disk group templates.

• Use the lstmpl command to view disk group templates.

ASMCMD> mktmpl -G DATA -–redundancy unprotected unprotect;

ASMCMD> chtmpl –G DATA -–striping fine DATAFILE;

ASMCMD> rmtmpl –G DATA unprotect;

ASMCMD> lstmpl –G DATA

Managing Disk Group Templates (continued)You can modify both user-defined and system-generated default templates. Modifying a template does not change the attributes of any existing files. The attributes of a modified template will only be applied to new files that reference the modified template, either explicitly or implicitly. You can only drop user-defined templates, not system-default templates.The V$ASM_TEMPLATE view lists all of the templates known to the ASM instance.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Using Disk Group Templates

You can apply disk group templates to newly created files in thefollowing ways:• Alias file name with template

• Incomplete file name with template

• DB_CREATE_FILE_DEST database initialization parameter

SQL> ALTER TABLESPACE myspace ADD2 DATAFILE '+DATA(mytemplate)/mydir/myspace02.dbf' SIZE 50M;

SQL> ALTER TABLESPACE myspace ADD2 DATAFILE '+DATA(mytemplate)' SIZE 50M;

SQL> ALTER SYSTEM SET DB_CREATE_FILE_DEST = '+DATA(mytemplate)';

SQL> CREATE TABLESPACE yourspace;

Using Disk Group TemplatesYou can reference a template name when creating a file by using either an alias or an incomplete file name. This allows you to assign desired attributes based on an individual file rather than on the file type. You can also use an incomplete file name with template in the DB_CREATE_FILE_DESTdatabase initialization parameter. Using this setting will apply the named template to files created by statements such as CREATE TABLESPACE and ALTER TABLESPACE.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


ASM Access Control Lists

ASM Access Control Lists (ACLs)• Set permissions at the ASM file level.• Set grants to groups or users.

– Users are Database software owners.– Users are identified by the OS user ID.

• They are available only on Linux and UNIX operating systems.

ASM Access Control ListsASM access control lists (ACLs) provide an optional protection for the ASM files. The objective of the ASM access control list is not security but separation of duties to prevent accidental file damage. Without ACLs, any user with SYSDBA privilege may access ASM files in the mounted disk group, up to and including removing them. To set up Oracle ASM File Access Control, you must create separate operating system groups for the OSASM, OSDBA for ASM, and OSDBA for database groups. The OSDBA group for the database must be different for each database instance using the same ASM instance.Each ASM file is created by a DBUSER. This DBUSER is usually an owner of a database instance. The ASM files created for that database are owned by that DBUSER. The operating system user ID of the database software owner identifies the DBUSER to the ASM instance. Access is limited by the operating system effective user ID NUMBER of the DBUSER. The operating system user of a running database instance is automatically added to a disk group when the database instance accesses that disk group and creates files. Each DBUSER can create access control lists. The ASM ACL includes a user group, a list of DBUSERS, but not any passwords. ASM trusts the OS authentication mechanisms. A DBUSER is a member of the OSDBA group for ASM.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


ASM ACL Prerequisites

Access control lists for ASM files require:• Linux or UNIX operating system• Job role separation at the OS level• Diskgroup attributes:

– COMPATIBLE.ASM to 11.2 or higher– COMPATIBLE.RDBMS to 11.2 or higher– ACCESS_CONTROL.ENABLED to TRUE– ACCESS_CONTROL.UMASK to a mask value

• Permissions: none (0), read (4), or read-write(6)• Mask values: 6 (removes all), 2 (removes write), 0

(removes nothing)

ASM ACL PrerequisitesAccess control lists for ASM are available only on Linux and UNIX operating systems. Job role separation must be configured. Each database and ASM instance must have different owners.Disk group attributes must be set. The COMPATIBLE.ASM and COMPATIBLE.RDBMS must be set to 11.2 or higher. For each disk group that is using ACLs, set ACCESS_CONTROL.ENABLED to TRUE and set ACCESS_CONTROL.UMASK to a mask value. ACCESS_CONTROL.ENABLED must be set to TRUE before ACCESS_CONTROL.UMASK can be set. The umask value removes permissions from full access of read-write for owner, group, and others. In permissions, 6 indicates read-write, 4 indicates read, and 0 indicates none. A umask value of 0 removes nothing; 2 removes write privilege; 6 removes read-write privilege. Concatenating the values gives the permissions for a user, group, or others. For example a umask of 026 will cause files to have permissions of 640, which is read-write for the owner, read for the group, and no access for all other users. ALTER DISKGROUP DATA2 SET ATTRIBUTE 'access_control.enabled' = 'true';

ALTER DISKGROUP DATA2 SET ATTRIBUTE 'access_control.umask' = '026';

Equivalent ASMCMD commands:ASMCMD> setattr -G data2 access_control_enabled true

ASMCMD> setattr -G data2 access_control_umask 026

Attributes of the disk group can also be set with the asmca tool.Oracle 11g: RAC and Grid Infrastructure Administration Accelerated 9 - 36

Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Managing ASM ACL with SQL Commands

The ACL for ASM can be managed with SQL commands.ALTER DISKGROUP ADD USERGROUP ... WITH MEMBER

ALTER DISKGROUP DROP USERGROUP

ALTER DISKGROUP MODIFY USERGROUP ADD MEMBER

ALTER DISKGROUP MODIFY USERGROUP DROP MEMBER

ALTER DISKGROUP ADD USER

ALTER DISKGROUP DROP USER

ALTER DISKGROUP SET PERMISSION

ALTER DISKGROUP SET OWNERSHIP

SELECT * FROM V$ASM_USER

SELECT * from V$ASM_USERGROUP

Managing ASM ACL with SQL CommandsAccess control lists in ASM can be managed with the SQL commands shown. To create a user group ACL, use the following command: ALTER DISKGROUP ADD USERGROUP groupname WITH MEMBER user [, user]

Each user must already exist. Any user with the SYSDBA or SYSASM privilege may create groups. Only the group owner or an ASM administrator may change or drop groups. Only an ASM administrator may add or drop users. Only the owner of a file or an ASM administrator may change the ownership of a file.Users of files are usually the database owners, and users are added automatically as files are created in a disk group. Adding users to a disk group should seldom be needed. An OS user that is not a database owner could be added to a disk group and a user group. The SET PERMISSION clause modifies permissions of an Oracle ASM file. Note that setting the read-only permission to a file that has read-write permission revokes the write permission. Only the file owner or the Oracle ASM administrator can change the permissions of a file. You cannot change the permissions on an open file.You cannot change the ownership on an open file.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Managing ASM ACL with ASMCMD Commands

The ACL for ASM can be managed with ASMCMD commandschgrp usergroup list_of_files

chmod mode list_of_files

chown user[:usergroup ] list_of_files

groups diskgroup user

grpmod { --add | --delete } diskgroup usergroup user1 [user2]…

lsgrp [-Ha] [ -G diskgroup ] [ pattern_expr ]

lsusr [-Ha] [ -G diskgroup ] [ pattern_expr ]

mkgrp diskgroup usergroup [user1] [user2]...

mkusr diskgroup user

passwd user

rmgrp diskgroup usergroup

rmusr diskgroup user

Managing ASM ACL with ASMCMD CommandsThe ASMCMD commands allow the storage administrator to perform the same operations as were shown with SQL commands but in a more familiar format. The asmcmd commands are patterned after the UNIX commands. The asmcmd commands have the same restrictions as the SQL commands.• chmod: Modifies permissions of an Oracle ASM file or list of files. Note that setting the

read-only permission to a file that has the read-write permission revokes the write permission. Only the file owner or the Oracle ASM administrator can change the permissions of a file. You cannot change the permissions on an open file.

• chown: You cannot change the ownership on an open file.In ASMCMD, the user can connect using the –a option to set privileges, either SYSASM or SYSDBA, but the OS user invoking the command must be a member of the OS group having that privilege. SYSASM is the default. To connect with the SYSDBA privilege, use the following syntax:

asmcmd –a SYSDBA


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Managing ASM ACL with Enterprise Manager

Managing ASM ACL with Enterprise ManagerThe ASM access control lists can be viewed and managed through Enterprise Manager. This slide shows the attributes and permissions associated with each file.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Managing ASM ACL with Enterprise Manager

Managing ASM ACL with Enterprise Manager (continued)On the Edit File page, permissions and ownership of an ASM file can be modified. The ownership and permissions cannot be changed on an open file.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


ASM ACL Guidelines

• Use ACLs to prevent accidental file damage.• Be careful when enabling access control if it blocks a

user’s access to a critical file.

ASM ACL GuidelinesASM access control lists are intended as a safety feature rather than a security feature. Without access control lists, any user with SYSDBA privileges can manipulate any file in the disk group, possibly unintentionally dropping or damaging files belonging to other databases. This becomes more important with storage consolidation where multiple databases share a single ASM installation, either on a single server or clustered.To implement ACLs, each database must have a different OS owner and group, and the ASM instance must have yet another owner as shown in the lesson titled “Oracle Grid Infrastructure Architecture.” Then even if the databases share disk groups, ACLs can prevent a user with SYSDBA privilege in ASM from damaging a file belonging to another database.When configuring ACL permissions and changing ownership, be careful that the ACL does not block access to a critical file or disk group. For example, make sure the database can access the disk group that contains the control file.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Quiz

RMAN is the only way to back up an Oracle Database that resides in ASM.1. True2. False

Answer: 2It is possible to back up an Oracle Database by using a custom PL/SQL routine with DBMS_FILE_TRANSFER or by using the XML DB repository. RMAN is, however, the preferred method of backing up Oracle Databases that reside in ASM. RMAN is very well integrated with ASM and provides many advanced backup and recovery features to aid administrators.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Quiz

Which of the following ASM file name forms will always be a valid way of referencing an existing ASM file?1. Fully Qualified2. Numeric3. Alias4. Alias with Template5. Incomplete6. Incomplete with Template

Answer: 1,2Option three is not a correct answer because there might not be an alias defined for some ASM files.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Summary

In this lesson, you should have learned how to:• Use different client tools to access ASM files• Describe the format of a fully qualified ASM file name• Explain how ASM files, directories, and aliases are created

and managed• Describe and manage disk group templates


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Administering ASM Cluster File Systems

Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Objectives

After completing this lesson, you should be able to:• Administer ASM Dynamic Volume Manager• Manage ASM volumes • Implement ASM Cluster File System• Manage ASM Cluster File System (ACFS)• Use ACFS Snapshots• Use command-line tools to manage ACFS


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


ASM Files and Volumes

Dynamic VolumesASMDisk

Group

ASM Files

ACFS Third-PartyFS

DB filesDB files

DB filesDB files

DB filesDB files

DB filesDB files

Dynamic Volume Manager

ASM Cluster File System (ACFS)ACFS Snapshots

ASM Instanceuser

Kernel

ASM Device File: /dev/asm/dgbvol1

OS

ASM Files and VolumesThe Automatic Storage Management (ASM) feature of Oracle database has been extended in Oracle Grid Infrastructure 11g Release 2 to include support for a general purpose cluster file system, the ASM Cluster File System (ACFS). To understand the operation of this feature, some terminology needs to be defined and explained.At the operating system (OS) level, the ASM instance provides the disk group, which is a logical container for physical disk space. The disk group can hold ASM database files and ASM dynamic volume files. The ASM Dynamic Volume Manager (ADVM) presents the volume device file to the operating system as a block device. The mkfs utility can be used to create an ASM file system in the volume device file. Four OS kernel modules loaded in the OS provide the data service. On Linux, they are:oracleasm, the ASM module; oracleadvm, the ASM dynamic volume manager module;oracleoks, the kernel services module; and oracleacfs, the ASM file system module. These modules provide the ASM Cluster File System, ACFS snapshots, the ADVM, and cluster services. The ASM volumes are presented to the OS as a device file at /dev/asm/<volume name>-<number>. The volume device file appears as another ASM file to the ASM Instance and asmcmd utility. The ASM layers are transparent to the OS file system commands. Only the files and directories created in ACFS and the ACFS snapshots are visible to the OS file system commands. Other file system types such as ext3 and NTFS may be created in an ADMV volume using the mkfscommand on Linux and advmutil commands on Windows.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


ACFS and ADVM Architecture Overview

ASM disk

ASM disk groupASM view

ASMvolume

file

OS view

ASM dynamic volume (Block device)

/dev/asm/volname-id

\\.\asm-volume_name-nnn

File systemsor

disk-basedapplications

ASMcluster

filesystem

SQL, ASMCA,EM, ASMCA

VDBG VBG# VMB

mkfs, mount,…

ACFS and ADVM Architecture OverviewIn 10g, Automatic Storage Management (ASM) is both a file system and volume manager built specifically for Oracle database files. Oracle 11g Release 2 includes the ASM dynamic volume manager (ADVM) and Oracle ASM cluster file system (ACFS). ADVM provides volume management services and a standard disk device driver interface to clients. Clients, such as file systems and other disk-based applications, issue I/O requests to ADVM volume devices as they would to other storage devices on a vendor operating system. ADVM extends ASM by providing a disk driver interface to storage backed by an ASM file. The administrator can use the ADVM to create volumes that contain file systems. These file systems can be used to support files beyond Oracle database files such as executables, report files, trace files, alert logs, and other application data files. With the addition of ADVM and ACFS, ASM becomes a complete storage solution of user data for both database and non-database file needs. ACFS is intended as a general file system accessible by the standard OS utilities. ACFS can be used in either a single server or a cluster environment. Note: Oracle ACFS file systems cannot be used for an Oracle base directory or an Oracle grid infrastructure home that contains the software for Oracle Clusterware, ASM, Oracle ACFS, and Oracle ADVM components. Oracle ACFS file systems cannot be used for an OS root directory or boot directory.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

ACFS and ADVM Architecture Overview (continued)ASM volumes serve as containers for storage presented as a block device accessed through ADVM. File systems or user processes can do I/O on this “ASM volume device” just as they would on any other device. To accomplish this, ADVM is configured into the operating system. A volume device is constructed from an ASM file. ASM file extents map the ASM volume file to logical blocks located on specific physical devices. Additional processes are started as part of the ASM instance and serve as intermediaries between the ASM instance and ADVM. To use the ADVM driver, an ASM instance must exist with at least one disk group mounted that can be used to contain an ASM volume file. An ASM volume is an ASM file. It inherits the properties of the ASM disk group and behaves similar to any other ASM file. ASM volume storage is automatically rebalanced whenever a storage configuration change occurs. This reconfiguration can be performed while an ASM volume is in use. Because ASM uses direct I/O, ASM volumes offer performance equivalent to raw disks. An OS device file is created automatically when an ASM volume is created using either asmcmd, SQL*Plus, ASMCA, or the Enterprise Manager graphical interfaces. On Linux, this device file is created in the /dev/asm directory. You can configure both disk group mount and volume enable operations to occur automatically upon ASM instance startup. The volume device file names are unique clusterwide and persistent across all nodes in the cluster that have an ASM instance running with the disk group mounted and volumes enabled.Upon Linux system startup, the Oracle clusterware startup will load the drivers (oraclesacfs,oracleoks, and oracleadvm). The ASM instance is started by the ASM cluster registry service (CRS) agent, which will also mount the appropriate ASM disk groups and enable volumes. The CRS agent will then mount any ACFS file systems in the Oracle Cluster Registry (OCR). Similar actions are performed on Windows.ACFS file systems are accessed through OS file system tools and APIs on UNIX and Linux systems, and accessed through Windows file system tools and APIs on Windows systems. Remote access is supported using standard NAS file access protocols such as network file systems (NFS) and common internet file system (CIFS) in support of heterogeneous file data sharing. The ACFS File System and ADVM components are installed onto each host along with the other ASM components into the Grid Infrastructure home location. The ACFS components consist of drivers that are dynamically loadable OS modules, several command-line tools, and a set of processes that execute within the ASM instance. However, loading the ACFS drivers requires rootprivileges on UNIX/Linux and Administrator privileges on Windows. So, the configuration and loading of the ACFS drivers is performed by the root scripts associated with the Oracle Grid Infrastructure installation. ACFS file systems are generally mounted on all cluster synchronization service (CSS) cluster members. In the event of a member failure, another cluster member will recover any outstanding metadata transactions on behalf of the failed member. In addition, any lock tokens held by the failed cluster members will be recovered and the failed member will be I/O fenced from the active CSS cluster. Following recovery, access by other active cluster members and any remote client systems may resume.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


ADVM Processes

New ASM instance background processes:• Volume Driver Background (VDBG) • Volume Background (VBGn) • Volume Membership Background (VMB)

ASM disk

ASM disk group

ASM dynamic volume (block device)

/dev/asm/volname-id

\\.\asm-volume_name-nnn

VDBG VBG# VMB

ADVM ProcessesA small number of processes will be added to the ASM instance when a volume is enabled. These processes are not started when there are no volumes configured.

• The Volume Driver Background (VDBG) process forwards ASM requests to lock or unlock an extent for rebalancing, resize the volume, offline the disk, add or drop a disk, and force and dismount a disk group to the dynamic volume manager driver. The VDBG is a fatal background process so the unplanned death of this process brings down the ASM instance.

• Volume Background (VBGn) processes wait for requests from the dynamic volume manager driver that need to be coordinated with the ASM instance. An example of such a request would be opening or closing an ASM volume file when the dynamic volume manager driver receives an open for a volume (possibly due to a file system mount request) or close for an open volume (possibly due to a file system unmount request). The unplanned death of any of these processes does not have an effect on the ASM instance.

• Volume Membership Background (VMB) coordinates cluster membership with the ASM instance.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


ADVM Restrictions

• Partitioning of dynamic volumes (using fdisk or similar) is not supported.

• Do not use raw to map ADVM volume block devices into raw volume devices.

• Do not create multipath devices over ADVM devices.• Do not create ASMLIB devices over ADVM devices.• ADVM supports all ASM supported storage solutions

except NFS and Exadata.• ADVM volumes cannot be used as a boot device or a root

file system.

ADVM RestrictionsAlthough ADVM provides a standard disk device interface for dynamic volumes, the following restrictions should be noted:

• Device partitions are not supported on Oracle ADVM dynamic volumes. Dynamic volumes supersede traditional device partitioning. Each volume is individually named and may be configured for a single file system. Oracle ADVM volumes may be created on demand from ASM disk group storage and dynamically resized as required. These attributes make Oracle ADVM volumes far more flexible than physical devices and associated partitioning schemes.

• On Linux platforms, Oracle ADVM volume devices are created as block devices regardless of the configuration of the storage underpinning the ASM disk group. Do not use the rawcommand to map Oracle ADVM volume block devices into raw volume devices.

• You should not create multipath devices over Oracle ADVM volume devices. Multipathingshould be applied over the disk devices that are initially consumed by ASM to construct the disk group underpinning an ADVM volume.

• You should not use ASMLIB over an ADVM volume device. You cannot layer ASM over ASM in a recursive fashion as this serves no useful purpose and is not supported.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


ASM Cluster File System

• General purpose scalable file system– Journaling, extent based– Single node and cluster

• Multi-OS platform (Linux and Windows at initial release)– POSIX, X/OPEN file system solution for UNIX/Linux

– Windows file system solution for Windows platforms• Accessible through NAS protocols (NFS, CIFS)• Integrated with Oracle Clusterware for cluster support• Integrated with Oracle system management tools

– Oracle installation and configuration– Enterprise Manager and ASM Storage mgt tools– Native OS File System Management tools

ASM Cluster File System The ASM cluster file system (ACFS) extends Automatic Storage Management (ASM) by providing a robust, modern, general-purpose, extent-based, and journaling file system. ACFS provides support for files such as application binaries, report files, trace files, alert logs, and other application data files. With the addition of the ACFS, ASM becomes a complete storage management solution for both Oracle database and non-database files. ACFS scales from small files to very large files(Exabyte) and supports a large numbers of nodes in a cluster. ACFS is an extent-based file system to provide high performance and utilizes a log-based metadata transaction engine for file system integrity and fast recovery. The ACFS on-disk structure supports endian neutral metadata. ACFS file systems can be exported to remote clients through industry standard protocols such as NFS and CIFS. ACFS eliminates the need for third-party cluster file system solutions and simplifies management for all file types in single node and grid computing environments. ACFS supports dynamic file system expansion and contraction without any downtime. ACFS makes use of the ASM mirroring and striping features in addition to hardware RAID functionality. Note: ACFS on a single server does not use cluster features, and thus is not a cluster file system.ACFS can be managed with tools familiar to the DBA: Enterpriser Manager, SQL*Plus, and ASMCMD. ACFS is integrated with the native OS file management tools, so the file system is transparent to the users. ASMCMD has UNIX-like syntax and command names that should be familiar to the system administrator and storage administrator. A new graphical tool (asmca) has been introduced to manage ASM and ACFS.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


ADVM Space Allocation

A volume allocates space in Volume Allocation Units (VAU) at creation and resize.• VAU is stripe column multiplied by volume extent size.• Volume extent is statically assigned based on the disk

group AU.– Volume extent is 64 MB when the AU is 1 MB.– Each extent is allocated round-robin on different disks.

• Stripe column is the number of stripes used inside a volume.– Stripe column can range from 1 to 8. – Stripe column default is 4.

ADVM Space AllocationADVM allocates space from an ASM disk group for a volume. The volume allocation unit is the smallest allocation when a volume is created and smallest amount of space that can be added to a volume. The VAU is based on the volume extent and the value of the stripe column. The volume extent is assigned based on the disk group allocation unit (AU). The AU can vary from 1 MB to 64 MB in powers of 2 (2,4,8,…,64). The volume extent is assigned based on the AU. The default AU of 1 MB has a volume extent size of 64 MB. The value of the stripe column is the number of volume extents that are allocated in each VAU. Thus VAU is the stripe column multiplied by the volume extent. The volume extents are allocated round-robin (each on the next disk in the disk group). If the stripe column is set to 1, volume striping is turned off. Example: If the stripe column is 4 and the AU is 1 MB, the volume extent is 64 MB and the VAU will be 256 MB. When the volume is created, multiples of the VAU will be allocated. If you asked for a volume of 300 MB in size, you would get a volume of 512 MB, assuming the space is available in the disk group. If you resized the volume, the space would be added or dropped in 256 MB chunks.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Striping Inside the Volume

Inside a volume, space is allocated based on:• Stripe column – the number of stripes (default 4)• Stripe width – the size of each stripe (default 128 KB)

Striping Inside the VolumeStripe column and stripe width are properties that are set when you create the volume. The stripe column is used both for allocating the initial space for the volume but also for the way the space inside the volume is used. The volume will be formatted for a file system. As the file system allocates space for files, the files will be stored in pieces the size of stripe width. If defaults are used, they are stored in 128 KB pieces. Each piece goes into the next volume extent. Since the volume extents are placed on disks in the disk group in the same manner, as the pieces are placed in the extents, the pieces are spread across the disks in the disk group.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Volume Striping: Example

Example:• Stripe column is 8, stripe width is 16 KB• First 200 KB of a file inside the file system is written as 13

chunks of 16 KB stripes across eight allocation units.

Volume Striping: ExampleFor this example, assume the default AU size of 1 MB. This means that the volume extent is 64 MB. For simplicity, assume that there are eight disks in the disk group. Since the stripe column is 8, eight volume extents will be created in every volume allocation unit of 512 MB. Eight volume extents are shown, one on each of the eight disks. Note: Each VAU will be filled before the next VAU is used. The diagram on this page shows how ACFS volume striping works. The strip width is 16 KB. A new file of 200 KB will be written, occupying 13 chunks of 16 KB spread round-robin across the eight volume extents. Consequently, reads and writes to this file are spread across eight disks instead of one.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Volume Striping: Example

Example:• Disk group with eight disks and external redundancy• Default AU size of 1 MB in use• Another ACFS file of 90 KB written as six chunks of 16 KB

stripes across the same eight allocation units

Volume Striping: Example (continued) Continuing the previous example, the next file of 90 KB is spread across each of the same volume extents in 16 KB chunks. This pattern continues until the first set of volume allocation units is filled and another set is allocated.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Creating an ACFS Volume

Create the volume:$ asmcmd volcreate -G DATA -s 100M testvol

View the volume information:$ asmcmd volinfo –G DATA testvol

Make a mount point directory:$ mkdir /u01/app/oracle/acfsdata/testvol

Make the file system (as root):# mkfs -t acfs /dev/asm/testvol-403

Mount the file system to the mount point:# mount -t acfs /dev/asm/testvol-403 \

/u01/app/oracle/acfsdata/testvol

Creating an ACFS VolumeThe COMPATIBLE.ADVM attribute of the disk group must be 11.2.0 or higher before you can create an ADVM volume in the disk group. You can set the attribute with the command:

asmcmd setattr -G DATA compatibility.advm 11.2.0.0.0

To create and mount an ACFS volume, use the procedure shown above.As a user with SYSDBA privilege in the ASM instance, perform the following:

1. Create the volume by using the volcreate command.2. View the volume information to determine the volume name.3. Make a mount point directory. This is where the volume will be mounted. It should have

appropriate ownership and permissions.As the root user:

1. Make the file system. This command formats the volume for an ACFS file system.2. Mount the volume to the mount point.

Note: The mkfs command can be used to create other file systems on an ADVM volume. Only one file system is allowed per volume. Only the ACFS file system is supported as a clustered file system.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Create an ASM Dynamic Volumewith Enterprise Manager

Create an ASM Dynamic Volume with Enterprise Manager1. On the Enterprise Manager home page, scroll down to the Instances section. Click the name of

one of the ASM instances.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED



Create an ASM Dynamic Volume with Enterprise Manager (continued)2. On the Automatic Storage Management page, click the ASM Cluster File System tab. 3. On the ASM Cluster File System tab, click Create. 4. On the Create ASM Cluster File System page, click Create ASM Volume.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED



Create an ASM Dynamic Volume with Enterprise Manager (continued)5. On the Create ASM Volume page, enter a volume name, and size. Click Ok. In this example, a

volume named testvol with a size of 1 GB is created.If you click Show SQL, you will see the SQL command:ALTER DISKGROUP DATA ADD VOLUME TESTVOL SIZE 1G;

Click Ok to create the volume.Optionally, you can specify a region of the physical disk in the Disk group where the extents should be placed. The ASMCMD command to add a volume is:

ASMCMD> volcreate -d DATA -s 1G testvol

ASMCMD and SQL have commands to view the volume informationASMCMD> volinfo –a SQL> SELECT * FROM V$ASM_VOLUME


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Managing ADVM Dynamic Volumes

Managing ADVM Dynamic VolumesEnterprise Manager has pages that allow you to create, delete, and manage ASM volumes.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Create an ASM Cluster File Systemwith Enterprise Manager

Create an ASM Cluster File System with Enterprise ManagerCreate an ASM Cluster File System on the ASM volume testvol that was created previously. Note that a disk group number is added to the volume name. This number is system assigned.

1. Assign a volume label. In this example it is labeled test.2. Assign a mount point. The mount point must be a exiting directory. Best practice is to place the

ACFS mount points in a common directory below the ORACLE_BASE directory. In this example the ORACLE_BASE is /u01/app/oracle the common directory is acfsdata and the mount point is /u01/app/oracle/acfsdata/test. The mount point directory must be created on all the nodes of the cluster.

If you click Show Command, the commands to make the file system on the ASM volume and to register the volume and mount point are shown. For Linux, the commands are:

$ /sbin/mkfs -t acfs -b 4k /dev/asm/testvol-93 -n "test" $ /sbin/acfsutil registry -f -a /dev/asm/testvol-93 \

/u01/app/oracle/acfsdata/test

3. Click Ok to perform the operations.You will be asked for the OS credentials of a user that has SYSASM privileges in the ASM instance.The file system will be created on the ASM volume and the volume will be registered in the cluster. The file system is then mounted on all the nodes of the cluster.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Manage Dynamic Volumes with SQL*PLUS

Command examples:SQL> ALTER DISKGROUP DGROUPA

ADD VOLUME asmvol1 SIZE 10G;

SQL> ALTER DISKGROUP DGROUPA

RESIZE VOLUME asmvol1 SIZE 15G;


DROP VOLUME asmvol1;


ENABLE VOLUME asmvol1;

SQL> ALTER DISKGROUP ALL

DISABLE VOLUME ALL;


MODIFY VOLUME asmvol1 USAGE 'acfs';

Manage Dynamic Volumes with SQL*PLUSSQL*Plus has added clauses to the ALTER DISKGROUP command that allow you to manage ASM volumes. Notice that each volume is managed as a part of a disk group. SQL*Plus allows you to create volumes with the add command, resize the volume, drop, enable, and disable the volume. The modify volume command allows you to set the intelligent data placement option[HOT|COLD], the mount point, and the usage name of the volume.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


As the grid owner, register the volume:$ /sbin/acfsutil registry -a -f \

/dev/asm/volume1-167 \

/u01/app/oracle/acfsdata/volume1

Registering an ACFS Volume

Registering an ACFS VolumeYou may register an ACFS volume. Registering the volume enables the cluster-ready services daemon to mount the volume automatically at startup. The –a option says add this volume to the registry. The –f option is used with the add option to allow replacement of an existing registry entry. You may register the volume before or after you mount the volume.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Creating an ACFS Volume with ASMCA

Creating an ACFS Volume with ASMCAIn ASMCA, click the Volume tab, and then, on the Volumes page, click Create.An alternative way is to click the ASM Cluster File System tab, and then click Create. On the Create ASM Cluster File System page (shown in the slide, upper left) click the Volume field and select Create Volume. Both methods bring you to the Create Volume page, where you specify the name, disk group, and size of the volume. Optionally, with the Advanced options you can specify the mirroring and striping attributes. Click OK to create the ACFS volume.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Creating the ACFS File System with ASMCA

Creating the ACFS File System with ASMCAOnce the volume is created, on the Create ASM Cluster File System page, select the volume, the type of file system, and mount point. ASMCA creates the file system on the volume and registers the volume, but the mount must be done by the root or administrator user.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Mounting the ACFS File System with ASMCA

Mounting the ACFS File System with ASMCABecause it requires root privileges to mount a file system, ASMCA does not mount the file system, but generates the command for you to use.Starting on the ASMCA ASM Cluster File Systems tab, right-click the new Dismounted volume and select Show Mount Command. On the Mount ACFS Command page, verify the mount point and click Generate Command.On the Mount ACFS Command page, follow the instructions. Execute the command shown on each of the nodes in the cluster as the root user. The command in this example is:/bin/mount -t acfs /dev/asm/images-389 /u01/app/oracle/acfsdata/images


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Managing ACFS with EM

Enterprise Manager provides the same functionality as ASMCA.

Managing ACFS with EMThe DBA, System Administrator, or Storage Administrator can use OS tools, ASMCMD, ASMCA, or Enterprise Manager to manage an ASM Cluster File System. Each Administrator can use the tools that they prefer. ACFS provides extensions to existing tools on Linux and UNIX systems and new commands on Windows for use by System Administrators.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Extending ASMCMD for Dynamic Volumes

asmcmd volcreate dgname volname -s size

[--redundancy {high|mirror|unprotected}][--width stripe_width]

[--column stripe_columns] [--primary [hot|cold]]

[--secondary [hot|cold]]

asmcmd volresize -G dgname -s size [-f] volname

asmcmd voldelete -G dgname volname

volenable { -a | -G diskgroup -a | -G diskgroup volume }

voldisable { -a | -G diskgroup -a | -G diskgroup volume }

volset -G diskgroup [ --usagestring string]

[--mountpath mount_path ]

[--primary {hot|cold}] [--secondary {hot|cold}] volume

volinfo { -a | -G diskgroup -a | -G diskgroup volume }

volinfo [--show_diskgroup |--show_volume] volumedevice }]

volstat [-G diskgroup] [volume]

Extending ASMCMD for Dynamic VolumesThe commands shown in the slide have been added to ASMCMD to allow the ASM volumes to be managed from ASMCMD. The ASMCMD commands allow you to have the same level of control through ASMCMD as you do through the SQL*Plus commands or Enterprise Manager.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Linux-UNIX File System APIs

ACFS supports:• Standard POSIX and X/OPEN file system APIs and

commands• User commands: access, ar, cat, chattr, chgrp,

chmod, chown, cksum, compress, cp, cpio, dd, df, diff, dir, dirname, du, file, find, fuser, grep, gunzip, gzip, link, ln, lockfile, ls, mkdir, mkfifo, mknod, mktemp, more, mv, od, pwd, rcp, rename, rm, rmdir, size, stat, string, sync, tail, tar, unlink, …

• Administrator commands: fsck, mkfs, mount, and umount

Linux-UNIX File System APIsThe ACFS files system is integrated with the POSIX and X/OPEN file system APIs and commands. This allows users and administrators to use ACFS transparently. As shown in the slide, the file system access commands are supported with ACFS transparently.Even the administration commands such as mkfs, fsck, mount, and umount are used to manage the ACFS file system.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Linux-UNIX Extensions• Create an ACFS file system:

mkfs [-vf] -t acfs [-b blksz] [-n name ] device [blocks]$ mkfs -t acfs /dev/asm/vol1-nnn

• Mount an ACFS file system:mount [-v] -t acfs [-o options] device dir$ mount -t acfs /dev/asm/vol1-nnn \

/oracle/cluster1/myacfs

• Unmount an ACFS file system:umount [-v] device|dir

• $ umount /oracle/cluster1/myacfs

• Check and repair an ACFS file system:fsck [-avnf] -t acfs [info] device$ fsck -t acfs /dev/asm/vol1-nnn

Linux-UNIX ExtensionsThe administration commands have added some extension to the common commands to handle the differences between ACFS and other file systems. Most of these extensions use the familiar file system type and file system–specific option parameters just as other file systems do. The example commands in the grey boxes show the most common usage. Each command is typed on one line. The device file is /dev/asm/volume-nnn, where volume name is appended with a disk group identifier represented by –nnn in the slide.Note: The backslash character shown as part of the mount command in the example above is a continuation character and used for formatting purposes only.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Command Function

acfsutil info Display new ACFS file and file system features (ACFS, snapshots, registry, size) and information

acfsutil snap Create and delete ACFS snapshots

acfsutil registry Register an ACFS file system with the ACFS mount registry (that is, cluster-wide fstab)

acfsutil rmfs Remove an ACFS file system

acfsutil size Resize an ACFS file system

acfsutil tune View or modify ACFS tune-ables

ACFS Platform-Independent Commands

New ACFS commands for Linux/UNIX and Windows (not natively supported)

ACFS Platform-Independent CommandsThe slide shows a set of commands that are the same on both Windows and Linux/UNIX platforms. The commands listed allow you to display and manipulate ACFS file systems.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


ACFS Snapshots

ACFS snapshots:• Are space efficient. Snapshots store:

– Metadata– Original versions of changed blocks– Using sparse files

• Can be used to:– Revert to a version in the snapshots– Recover a deleted file– Back up a consistent data set

• Are limited to 63 snapshot views per volume

ACFS SnapshotsAn Oracle ACFS snapshot is an online, read-only, point-in-time copy of an Oracle ACFS file system. The snapshot copy is initially sparse and merely references the storage allocation information maintained by the file system. Before an Oracle ACFS file extent is modified or deleted, its current value is copied to the snapshot using a copy-on-write (COW) technique to maintain the snapshot’s point-in-time view of the file system. ACFS snapshots provide a low cost way to preserve the state of a volume at a point in time. When a snapshot is created, a set of metadata about the volume is created in the same volume. This metadata includes the directory structure and the names of all the files. The snapshot consists of pointers to blocks in the volume that have not changed since the snapshot was created and the actual blocks if the file has changed. After the snapshot is created, data blocks are written to the snapshot as files change, but only the original version of changed blocks are written. The snapshot allows you recover deleted files, revert files to the version at the time of the snapshot, and back up a consistent file set by taking a backup of the snapshot.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Managing ACFS Snapshots

Manage ACFS snapshots with acfsutil commands.• Create snapshots:

$ acfsutil snap create snapshot_2 /u01/app/oracle/acfsdata/testvol

• Delete snapshots:$ acfsutil snap delete snapshot_2

/u01/app/oracle/acfsdata/testvol

• View file system information, including snapshots:$ acfsutil info fs <mount point> ls –l

<mount_point>/.ACFS/snaps

Managing ACFS SnapshotsACFS snapshots can be managed with the acfsutil snap commands to create and delete snapshots. You can view the contents of the snapshots by using standard OS commands to search and view the contents of a directory.An example of creating a snapshot on the ACFS volume mounted at /u01/app/oracle/acfsdata/testvol with a snapshot name of snapshot_2 is:

acfsutil snap create snapshot_2 /u01/app/oracle/acfsdata/testvol

This create command will create a directory named:/u01/app/oracle/acfsdata/testvol/.ACFS/snaps/snapshot_2

The snapshot is immediately available for use.You can view the file system information including the number of snapshots with:

acfsutil info fs /u01/app/oracle/acfsdata/testvol


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Managing ACFS Snapshots (continued)/u01/app/oracle/acfsdata/testvol

ACFS Version: 11.2.0.1.0.0flags: MountPoint,Availablemount time: Wed Jul 1 12:00:10 2009volumes: 1total size: 268435456total free: 163360768primary volume: /dev/asm/testvol-403

label: flags: Primary,Available,ADVM

…ADVM stripe width: 131072

number of snapshots: 1snapshot space usage: 32768

You can see the names of the snapshots with the OS command:ls /u01/app/oracle/acfsdata/testvol/.ACFS/snaps


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Managing ACFS Snapshotswith Enterprise Manager

Managing ACFS Snapshots with Enterprise ManagerEnterprise Manager allows you to create, manage, and search snapshots. Select the volume and click Create Snapshots to create a snapshot on that volume. To view snapshots for a particular volume, click the snapshots link for that volume.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Creating ACFS Snapshots

Creating ACFS SnapshotsEnterprise Manager allows you to create an ACFS snapshot. You can also open telnet session from Enterprise Manager and use the acfsutil snap create command.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Managing Snapshots

Managing SnapshotsOn the ASM Cluster File System page of Enterprise Manager, you can Create new snapshots, delete existing snapshots, search multiple snapshots. To search or browse a single snapshot, click the snapshot name link.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Viewing Snapshots

Viewing SnapshotsUsing Enterprise Manager you can view the contents of a snapshot and search for files or directories. In this example that orahome directory in snapshot_3 is being viewed.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


ACFS Backups

• An ACFS file system may be backed up using:– Standard OS file system backup tools– Oracle Secure Backup– Third-party backup tools

• ACFS snapshots present a stable point-in-time view.

ACFS Backups ACFS file systems accommodate and enable a variety of backup and restore solutions. Oracle Secure Backup and other enterprise backup tools supports full and incremental backups of file systems by reading file system files using the native OS file system interface and storing the results on a choice of storage media (tape, disk). If the file system is active, backup results will reflect the dynamic nature of the file modifications in the file system. ACFS file system snapshots can be employed to present a stable point-in-time view of the file system to a backup tool. If the storage subsystem supports atomic copy domains, for example: Netapp or EMC, a system administrator will be able to leverage a storage array snapshot for backing up ACFS file systems.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


ACFS Performance

ACFS performance benefits from:• Distribution and load balancing of ASM file segments• ACFS file extents distributed across ASM file segments• User and metadata caching• In-memory updates of transaction logs

ACFS PerformanceThe ACFS storage allocation and I/O access schemes benefit from the distribution and load balancing of ASM file segments across the ASM disks in a disk group. ACFS storage allocation further distributes individual ACFS file extents across ASM file segments for increased distribution of individual file data and file I/O parallelism. ACFS user and metadata caching techniques accommodate a variety of non-database application workload I/O access patterns with high cache hit rates that result in reduced I/O traffic to the underlying ASM file and deferred updates of modified file cache data. The ACFS transaction logs are first updated in memory, deferring the actual transfer of log and file data through periodic updates to the associated ASM file.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Using ACFS Volumes After Reboot

In cluster configuration, ACFS drivers are automatically reloaded after reboot.In stand-alone server configuration:• Oracle Restart cannot load ACFS drivers • Root or administrator must load drivers

Using ACFS Volumes After RebootThe ACFS drivers are loaded when the Grid Infrastructure is installed, but they are not automatically reloaded by Oracle Restart in a single-server environment. Note: Oracle cluster-ready services reloads these drivers in a RAC environment. The drivers must be loaded by the root user on Linux or administrator on Windows.On Linux, load the drivers with the command

# /<GridHome>/bin/acfsload start –s

Enable volume in ASM instanceALTER ... ENABLE VOLUME <vol_name>;

Mount volume as root:# /bin/mount -t acfs -rw <volume_name> <mount_point> # /bin/mount -t acfs -rw /dev/asm/asm01-480 /u01/app/oracle/acfs01

View file systems as root:# acfsutil info fs


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


ACFS Views

• Available in SQL*Plus connected to the ASM instance– V$ASM_FILESYSTEM contains one row for every mounted

file system.– V$ASM_ACFSVOLUMES contains one row for every ACFS

device file that is a member of a mounted ACFS file system.• ASMCMD volinfo –a provides similar volume

information– Includes unmounted file systems – Does not include freespace or snapshot space usage

• The ASMCMD acfsutil info fs command gives detail information on each file system.

ACFS ViewsThe information and availability of ASM dynamic performance views vary depending on the instance that queries the view. The ASM dynamic performance views for ACFS are available only from the ASM instance. The V$ASM_FILESYSTEM view contains one row for every mounted file system. This view includes space usage, and snapshot space usage. The V$ASM_ACFSVOLUME view contains one row for every ACFS device file that is a member of a mounted ACFS file system.Similar information can be obtained from the ASMCMD volinfo command. The volinfo output includes unmounted volumes. The space usage information is not provided from the volinfocommand.The command acfsutil info fs gives detail information about each mounted file system, including space usage, snapshot space used, volume name, and stripe configuration. The acfsutilcommand is available to the root or administrator user.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Quiz

The ADVM volume can only be used for ACFS file systems.1. True2. False

Answer: 2Other file systems can be created on ADVM volumes but they are not clustered file systems just because they are on an ADVM volume.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Summary

In this lesson, you should have learned how to:• Administer ASM dynamic volume manager• Manage ASM volumes • Implement ASM cluster file system (ACFS)• Manage ACFS with various tools• Use ACFS snapshots


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Practice 10 Overview: Managing ACFS

This practice covers the following topics:• Managing an ACFS file system

– Create – Register– Mount

• Managing ACFS Snapshots


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Real Application Clusters Database Installation

Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Objectives

After completing this lesson, you should be able to:• Install the Oracle database software• Create a cluster database • Perform post-database-creation tasks• Perform a single instance to RAC conversion


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Installing the Oracle Database Software

$ /stage/database/Disk1/runInstaller

Installing the Oracle Database SoftwareThe Oracle Universal Installer (OUI) is used to install the Oracle Database 11g Release 2 (11.2) software. Start the OUI by executing the runInstaller command from the rootdirectory of the Oracle Database 11g Release 2 CD-ROM or from the software staging location. You can use the Specify Oracle Direct Connect Identification window to specify an email address to receive security updates directly from Oracle Support as they occur. Alternatively, you can elect to opt out of these alerts. If you want to receive them, supply your email address and your Oracle Support password, and click Next.The “Select installation option” window enables you to create and configure a database, install database software only, or upgrade an existing database. Select the “Install database software only” option and click Next.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED



Installing the Oracle Database Software (continued)In the Grid Installation Options window, select “Real Application Clusters database installation” and select all nodes in your cluster on which the software should be installed. Click Next to continue. In the “Select product languages” window, select your desired languages from the Available Languages list and click the right arrow to promote the selected languages to the Selected Languages list. Click Next to continue. In the “Select database edition” window (not shown in the slide above), you select whether to install the Enterprise Edition or the Standard Edition. Select the Enterprise Edition option and click Next to continue.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED



Installing the Oracle Database Software (continued)In the Specify Installation Location window, provide a value for ORACLE_BASE if you have not yet already done so. The default ORACLE_BASE location is /u01/app/oracle, provided the RDBMS software is being installed by the oracle account. The Software Location section of the window enables you to specify a value for the ORACLE_HOMElocation. The default ORACLE_HOME location is /u01/app/oracle/product/11.2.0/dbhome_1. Accept the suggested path or enter your own location. After entering the information, review it for accuracy, and click the Nextbutton to continue. In the “Privileged operating system groups” window, select the operating system group that will act as the OSDBA group. Next, select the group that will act as the OSOPER group. Click Next to continue.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED



Installing the Oracle Database Software (continued)The “Perform prerequisite checks” window verifies the operating system requirements that must be met for the installation to be successful. These requirements include:

• Certified operating system check• Kernel parameters as required by the Oracle Database software• Required operating system packages and correct revisions

After each successful check, the Status for that check will indicate Succeeded. Any tests that fail are also reported here. If any tests fail, click the “Fix & Check Again” button. The Installer will generate fix-up scripts to correct the system deficiencies if possible. Execute the scripts as directed by the Installer. The tests will be run again after completing the script executions. When all tests have succeeded, click the Next button to continue. In the Summary window, review the Global settings and click Finish. At the end of the installation, the OUI will display another window, prompting you to run the root.sh scripts on the nodes you chose for the installation. Follow the instructions to run the scripts. When finished, click the OK button to close the Execute Configuration Scripts window and return to the Finish screen. Click Close to complete the installation and close the OUI.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Creating the Cluster Database

$ cd /u01/app/oracle/product/11.2.0/dbhome_1/bin$ ./dbca

Creating the Cluster DatabaseTo create the cluster database, change directory to $ORACLE_HOME/bin on the installing node and execute the database configuration assistant (DBCA) utility as shown below:

$ cd /u01/app/oracle/product/11.2.0/dbhome_1/bin$ ./dbca

The Welcome window appears first. You must select the type of database that you want to install. Select the “Oracle Real Application Clusters database” option, and then click Next. The Operations window appears. For a first-time installation, you have only two choices: the first option enables you to create a database and the other option enables you to manage database creation templates. Select the “Create a Database” option, and then click Next to continue.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Select Database Type

Select Database TypeThe Database Templates window appears next. The DBCA tool provides several predefined database types to choose from, depending on your needs. The templates include:

• General Purpose or Transaction Processing• Custom Database• Data Warehouse

In the example in the slide above, the “General Purpose or Transaction Processing” option is chosen. Click the Next button to continue.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Database Identification

Database IdentificationIn the Database Identification window, you must choose between an administrator-managed and a policy-managed cluster database. Administrator-managed RAC databases specify a list of cluster nodes where RAC instances will run. Services may also be specified and associated with preferred and alternative nodes. There is an explicit association between database services, instances, and cluster nodes.Policy-based management, a new feature in this release, breaks the explicit associationbetween services, instances, and cluster nodes. Policy-based management introduces the concept of server pools, which are logical divisions of a cluster that are dynamically allocated based on relative importance. Database services are associated with server pools and RAC instances are automatically started to satisfy the service to server pool associations. You specify in which server pool the database resource will run and the number of instances needed (cardinality). Oracle Clusterware is responsible for placing the database resource on a server. Server pools are logical divisions of a cluster into pools of servers which are allocated to host databases or other applications. Server pools are managed using crsctl and srvctlcommands. Names must be unique within the resources defined for the cluster.You must also choose the global database name and the nodes on which to create the cluster database. The global database name can be up to 30 characters in length and must begin with an alphabetical character. When you have finished, click Next to continue.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Cluster Database Management Options

Cluster Database Management OptionsThe Management Options window is displayed. For small cluster environments, you may choose to manage your cluster with Enterprise Manager Database Control. To do this, select the “Configure Enterprise Manager” check box. If you have Grid Control installed somewhere on your network, you can select the “Use Grid Control for Database Management” option. If you select Enterprise Manager with the Grid Control option and the DBCA discovers agents running on the local node, you can select the preferred agent from a list. Grid Control can simplify database management in large, enterprise deployments. You can also configure Database Control to send email notifications when alerts occur. If you want to configure this, you must supply a Simple Mail Transfer Protocol (SMTP) or outgoing mail server and an email address. You can also enable daily backups here. You must supply a backup start time as well as operating system user credentials for this option. If you want to use Grid Control to manage your database, but have not yet installed and configured a Grid Control server, do not click either of the management methods. When you have made your choices, click the Next button to continue.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Passwords for Database Schema Owners

Passwords for Database Schema OwnersThe Database Credentials window appears next. You must supply passwords for the user accounts created by the DBCA when configuring your database. You can use the same password for all of these privileged accounts by selecting the “Use the Same Administrative Password for All Accounts” option. Enter your password in the Password field, and then enter it again in the Confirm Password field.Alternatively, you may choose to set different passwords for the privileged users. To do this, select the “Use Different Administrative Passwords” option, enter your password in the Password field, and then enter it again in the Confirm Password field. Repeat this for each user listed in the User Name column. Click the Next button to continue.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Database File Locations

Database File LocationsIn the Database File Locations window, you must indicate where the database files are to be stored. You can choose your storage type from the drop-down list. Detected (and supported) shared storage types are available here. You can choose to use a standard template for file locations, one common location, or Oracle Managed Files (OMF). This cluster database uses ASM and Oracle Managed Files. Therefore, select the Use Oracle-Managed Files option, and enter the disk group name in the Database Area field. Alternatively, you can click the Browse button to indicate the location where the database files are to be created. When you have made your choices, click the Next button to continue.Note: Step numbers in the title bar can be different, depending on the choices made during the installation.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Recovery Configuration

Recovery ConfigurationIn the Recovery Configuration window, you can select redo log archiving by selecting Enable Archiving. If you are using ASM or cluster file system storages, you can also select the Flash Recovery Area size in the Recovery Configuration window. The size of the area defaults to 2048 megabytes, but you can change this figure if it is not suitable for your requirements. If you are using ASM and a single disk group, the flash recovery area defaults to the ASM Disk Group. If more than one disk group has been created, you can specify it here. If you use a cluster file system, the flash recovery area defaults to $ORACLE_BASE/flash_recovery_area. You may also define your own variables for the file locations if you plan to use the Database Storage window to define individual file locations. Select the Enable Archiving check box to enable archiving immediately for the new cluster database.Note: Flash Recovery Area has been renamed Fast Recovery Area in Oracle Database 11gRelease 2 (11.2); however, some of the tools and utilities continue to use the previous name.When you have completed your entries, click Next, and the Database Content window is displayed.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Database Content

Database ContentIn the Database Content window, you can choose to install the Sample Schemas included with the database distribution. On the Custom Scripts tabbed page, you can choose to run your own scripts as part of the database creation process. When you have finished, click the Next button to continue to the next window.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Initialization Parameters

Initialization ParametersIn the Initialization Parameters window, you can set important database parameters. The parameters are grouped under four tabs:

• Memory• Sizing• Character Sets• Connection Mode

On the Memory tabbed page, you can set parameters that deal with memory allocation, including shared pool, buffer cache, Java pool, large pool, and PGA size. Automatic Memory Management is the preferred memory management method and can be selected here. On the Sizing tabbed page, you can adjust the database block size. Note that the default is 8 kilobytes. In addition, you can set the number of processes that can connect simultaneously to the database.By clicking the Character Sets tab, you can change the database character set. You can also select the default language and the date format. On the Connection Mode tabbed page, you can choose the connection type that clients use to connect to the database. The default type is Dedicated Server Mode. If you want to use Oracle Shared Server, click the Shared Server Mode button. If you want to review the parameters that are not found on the four tabs, click the All Initialization Parameters button. Click the Use Automatic Memory Management button and click the Next button to continue.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Database Storage Options

Database Storage OptionsThe Database Storage window provides full control over all aspects of database storage, including tablespaces, data files, and log members. Size, location, and all aspects of extent management are under your control here.When you have finished, click the Next button to continue to the next page.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Create the Database

Create the DatabaseThe Creation Options window appears. You can choose to create the database, or save your DBCA session as a database creation script by selecting the corresponding check box. Select the Create Database check box, and then click the Finish button. The DBCA displays the Summary screen, giving you a last chance to review all options, parameters, and so on that have been chosen for your database creation. Review the summary data. When you are ready to proceed, close the Summary window by clicking the OK button.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Monitor Progress

Monitor ProgressThe Progress Monitor window appears next. In addition to informing you about how fast the database creation is taking place, it also informs you about the specific tasks being performed by the DBCA in real time. When the database creation progress reaches 100 percent, the DBCA displays a dialog box announcing the completion of the creation process. It also directs you to the installation log file location, parameter file location, and Enterprise Manager URL. By clicking the Password Management button, you can manage the database accounts created by the DBCA.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Postinstallation Tasks

• Back up the root.sh script.• Download and install the required patch updates.• Verify the cluster database configuration.$ srvctl config database -d orcl

Database unique name: ORCLDatabase name: ORCLOracle home: /u01/app/oracle/product/11.2.0/dbhome_1Oracle user: oracleSpfile: +DATA/orcl/spfileorcl.oraDomain: example.comStart options: openStop options: immediateDatabase role: PRIMARYManagement policy: AUTOMATICServer pools: orclDatabase instances: orcl1,orcl2, orcl3Disk Groups: DATA,FRAServices:Database is administrator managed

Postinstallation TasksAfter the cluster database has been successfully created, run the following command to verify the Oracle Cluster Registry configuration in your newly installed RAC environment:

$ srvctl config database -d db_name

It is also recommended that you back up the root.sh script after you complete an installation. $ cd $ORACLE_HOME

$ cp root.sh root.sh.bak

If you install other products in the same Oracle Home directory, the OUI updates the contents of the existing root.sh script during the installation. If you require information contained in the original root.sh script, you can recover it from the root.sh file copy.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Check Managed Targetshttps://host01.example.com:1158/em

Check Managed TargetsAnother postinstallation task that you should perform if you are using Enterprise Manager Database Control or Grid Control is to check whether all the managed nodes and their managed resources are properly registered and available. Open a browser and enter the address for your Database Control console. Click the Targets tab to verify that all the targets appear here.Note: If Enterprise Manager Database Control has not been started on your node, you can start it with the following commands logged in as the oracle user, or the account that owns the database software installation:

$ export ORACLE_UNQNAME=orcl$ emctl start dbconsole


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Background Processes Specific to Oracle RAC

• ACMS: Atomic Control file to Memory Service• GTX[0-j]: Global Transaction Process• LMON: Global Enqueue Service Monitor• LMD: Global Enqueue Service Daemon• LMS: Global Cache Service Process• LCK0: Instance Enqueue Process• RMSn: Oracle RAC Management Processes• RSMN: Remote Slave Monitor

Background Processes Specific to Oracle RAC An Oracle RAC database has the same processes and memory structures as a single-instance Oracle database and additional process and memory structures that are specific to Oracle RAC. The global cache service and global enqueue service processes, and the global resource directory (GRD) collaborate to enable cache fusion. The Oracle RAC processes and their identifiers are as follows:

• Atomic Control file to Memory Service (ACMS): In a RAC environment, the ACMSper-instance process is an agent that contributes to ensuring a distributed SGA memory update is either globally committed on success or globally aborted if a failure occurs.

• Global Transaction Process (GTX[0-j]): The GTX[0-j] processes provide transparent support for XA global transactions in a RAC environment. The database automatically tunes the number of these processes based on the workload of XA global transactions.

• Global Enqueue Service Monitor (LMON): The LMON process monitors global enqueues and resources across the cluster and performs global enqueue recovery operations.

• Global Enqueue Service Daemon (LMD): The LMD process manages incoming remote resource requests within each instance.

• Global Cache Service Process (LMS): The LMS process maintains records of the data file statuses and each cached block by recording information in the GRD.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Background Processes Specific to Oracle RAC (continued)The LMS process also controls the flow of messages to remote instances and manages global data block access and transmits block images between the buffer caches of different instances. This processing is part of the cache fusion feature.

• LCK0: Instance Enqueue Process The LCK0 process manages noncache fusion resource requests such as library and row cache requests.

• RMSn: Oracle RAC Management Processes The RMSn processes perform manageability tasks for Oracle RAC. Tasks accomplished by an RMSn process include creation of resources related to Oracle RAC when new instances are added to the clusters.

• RSMN: Remote Slave Monitor The RSMN process manages background slave process creation and communication on remote instances. These background slave processes perform tasks on behalf of a coordinating process running in another instance.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Single Instance to RAC Conversion

• Single-instance databases can be converted to RAC using:– DBCA– Enterprise Manager– RCONFIG utility

• DBCA automates most of the conversion tasks.

• Before conversion, ensure that:– Your hardware and operating

system are supported– Your cluster nodes have access to

shared storage

Single-instance database

RAC database

Single Instance to RAC ConversionYou can use the Database Configuration Assistant (DBCA) to convert single-instance Oracle databases to RAC. The DBCA automates the configuration of the control file attributes, creates the undo tablespaces and the redo logs, and makes the initialization parameter file entries for cluster-enabled environments. It also configures Oracle Net Services, Oracle Clusterware resources, and the configuration for RAC database management for use by Oracle Enterprise Manager or the srvctl utility.Before you use the DBCA to convert a single-instance database to a RAC database, ensure that your system meets the conditions:

• It is a supported hardware and operating system configuration.• It has shared storage. A supported Cluster File System, NFS mount, or ASM is available

and accessible from all nodes.• Your applications have no design characteristics that preclude their use with cluster

database processing.You can also use Enterprise Manager and the rconfig utility to perform the single instance to RAC conversion.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Issues for Converting Single Instance Databases to Oracle RAC

• Backup procedures should be available before conversion takes place.

• Archiving in Oracle RAC environments requires a thread number in the archive file format.

• The archived logs from all instances of an Oracle RAC database are required for media recovery.

• By default, all database files are migrated to Oracle Managed Files (OMF).

Issues for Converting Single Instance Databases to Oracle RACWhatever method you choose to use for the conversion, note the following administrative considerations before converting single-instance databases to Oracle RAC:

• Backup procedures should be available before converting from a single-instance Oracle Database to Oracle RAC. For archiving with Oracle RAC environments, the archive file format requires a thread number.

• The archived logs from all instances of an Oracle RAC database are required for media recovery. Because of this, if you archive to a file and you do not use a cluster file system, or some other means to provide shared file systems, then you require a method of accessing the archive logs from all nodes on which the cluster database has instances.

• By default, all database files are migrated to Oracle Managed Files (OMF). This feature simplifies table space creation, ensures data file location consistency and compliance with OFA rules, and reduces human error with data file management.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Single-Instance Conversion Using the DBCA

Conversion steps for a single-instance database onnonclustered hardware:1. Back up the original single-instance database.2. Complete the Oracle Clusterware installation.3. Validate the cluster.4. Copy the preconfigured database image.5. Install the Oracle Database 11g software with RAC.

Single-Instance Conversion Using DBCATo convert from a single-instance Oracle database that is on a noncluster computer to a RAC database, perform the following steps:

1. Back up the original single-instance database.2. Complete the Oracle Clusterware installation.3. Validate the cluster.4. Copy the preconfigured database image.5. Install the Oracle Database 11g software with RAC.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Conversion Steps1. Back up the original single-instance database.

Conversion Steps1. Back Up the Original Single-Instance Database.Use the DBCA to create a preconfigured image of your single-instance database by using the following procedure:

1. Navigate to the bin directory in $ORACLE_HOME, and start the DBCA.2. In the Welcome window, click Next.3. In the Operations window, select Manage Templates, and click Next.4. In the Template Management window, select “Create a database” template and “From an

existing database (structure as well as data),” and click Next. 5. In the Source Database window, enter the database name in the Database instance field,

and click Next.6. In the Template Properties window, enter a template name in the Name field. By default,

the template files are generated in $ORACLE_HOME/assistants/dbca/templates. Enter a description of the file in the Description field, and change the template file location in the Template data file field if you want. When you have finished, click Next.

7. In the Location of Database Related Files window, select “Maintain the file locations,”so that you can restore the database to the current directory structure, and click Finish. The DBCA generates two files: a database structure file (template_name.ctl) and a database preconfigured image file (template_name.dfb).


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Conversion Steps

2. Perform the preinstallation steps.– Tasks include kernel parameter configuration, hardware

setup, network configuration, and shared storage setup 3. Set up and validate the cluster.

– Create a cluster with the required number of nodes according to your hardware vendor’s documentation.

– Validate cluster components before installation.– Install Oracle Clusterware. – Validate the completed cluster installation by using cluvfy.

4. Copy the preconfigured database image.— The database structure *.ctl file — The preconfigured database image *.dfb file

5. Install the Oracle Database 11g Release 2 software with RAC.

Conversion Steps (continued)2. Perform the Preinstallation Steps.Complete the Oracle Clusterware installation, as described in the lesson titled “Grid Infrastructure Installation” or refer to the Oracle Grid Infrastructure Installation Guide for your platform.3. Set Up and Validate the Cluster.Form a cluster with the required number of nodes according to your business needs. When you have configured all the nodes in your cluster, validate cluster components by using the Cluster Verification Utility, and then install Oracle Clusterware. When the clusterware is installed, validate the completed cluster installation and configuration using the Cluster Verification Utility.4. Copy the Preconfigured Database Image.This includes copying the database structure *.ctl file and the database preconfigured image *.dfb file that the DBCA created in step one (Back Up the Original Single-Instance Database) to a temporary location on the node in the cluster from which you plan to run the DBCA.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Conversion Steps (continued)5. Install the Oracle Database 11g Release 2 software with RAC.

1. Run the OUI to perform an Oracle database installation with RAC. Select Cluster Installation Mode and select the nodes to include in your RAC database.

2. In the OUI Database Configuration Types window, select “Advanced install.” After installing the software, the OUI runs postinstallation tools such as NETCA, DBCA, and so on.

3. In the DBCA Template Selection window, use the template that you copied to a temporary location in the “Copy the Preconfigured Database Image” step. Use the browse option to select the template location.

4. After creating the RAC database, the DBCA displays the Password Management window in which you must change the passwords for database privileged users. When the DBCA exits, the conversion is complete.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Single-Instance Conversion Using rconfig

1. Locate the appropriate xml file located in the $ORACLE_HOME/assistants/rconfig/sampleXMLsdirectory.

2. Modify the ConvertToRAC_AdminManaged.xml orConvertToRAC_PolicyManaged.xml file as required for your system.

3. Save the file under a different name.

$ cd $ORACLE_HOME/assistants/rconfig/sampleXMLs$ vi ConvertToRAC_PolicyManaged.xml... Saved as my_rac_conversion.xml

$ rconfig my_rac_conversion.xml

Single-Instance Conversion Using rconfigYou can use the rconfig command-line utility to convert a single-instance database to RAC. To use this feature, perform the following steps:

1. Go to the $ORACLE_HOME/assistants/rconfig/sampleXMLs directory as the oracle user and open the ConvertToRAC_AdminManaged.xml or the ConvertToRAC_PolicyManaged.xml file (depending on your desired management style) using a text editor, such as vi.

2. Review the XML file, and modify the parameters as required for your system. The XML sample file contains comment lines that provide instructions about how to configure the file. When you have completed making changes, save the file with the syntax filename.xml. Make a note of the name you select.

3. Assuming that you save your XML file as my_rac_conversion.xml, navigate to the $ORACLE_HOME/bin directory, and use the following syntax to run the rconfigcommand:$ ./rconfig my_rac_conversion.xml

Note: The Convert verify option in the xml file has three options:• Convert verify="YES": rconfig performs checks to ensure that the

prerequisites for single-instance to RAC conversion have been met before it starts conversion.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

Single-Instance Conversion Using rconfig (continued)• Convert verify="NO": rconfig does not perform prerequisite checks, and starts

conversion.• Convert verify="ONLY": rconfig performs only prerequisite checks; it does

not start conversion after completing prerequisite checks.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Quiz

The RAC database software installation is initiated by executingrunInstaller from the root directory of the Oracle Database 11g Release 2 CD-ROM or from the software staging location.1. True2. False

Answer: 1The statement is true.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Quiz

A single instance database can be converted to a RACdatabase by using (choose the correct options):1. rconfig

2. netca

3. dbca

Answer: 1,3Choices a and c are correct.


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Summary

In this lesson, you should have learned how to:• Install the Oracle database software• Create a cluster database • Perform post-database-creation tasks• Perform a single instance to RAC conversion


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED


Practice 11 Overview

This practice covers the following topics:• Installing the Oracle database software• Creating a RAC database


Ora

cle

Uni

vers

ity a

nd E

gabi

Sol

utio

ns u

se o

nly

THES

E eK

IT M

ATER

IALS

AR

E FO

R Y

OU

R U

SE IN

TH

IS C

LASS

RO

OM

ON

LY.

CO

PYIN

G e

KIT

MAT

ERIA

LS F

RO

M T

HIS

CO

MPU

TER

IS S

TRIC

TLY

PRO

HIB

ITED

d60488gc11_sg1

Documents

oracle andor

oracle parkway

oracle clusterware networking

oracle clusterware startup

oracle clusterware concepts

oracle training course

fair use

disclaimerthis document