dell emc ready solutions for oracle: design for dell emc ... · oracle database 12cr2 to oracle...
TRANSCRIPT
Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage
With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1
February 2019
H17577
Reference Architecture Guide
Abstract
This reference architecture guide shows that customers can migrate from Oracle
Database 12c to Oracle Database 18c running on the Dell EMC Unity 650F All Flash
storage array with no performance impact to production workloads.
Dell EMC Solutions
Copyright
2 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Databases 12cR2 and 18cR1 Reference Architecture Guide
The information in this publication is provided as is. Dell Inc. makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose.
Use, copying, and distribution of any software described in this publication requires an applicable software license.
Copyright © 2019 Dell Inc. or its subsidiaries. All Rights Reserved. Dell Technologies, Dell, EMC, Dell EMC and other trademarks are trademarks of Dell Inc. or its subsidiaries. Intel, the Intel logo, the Intel Inside logo and Xeon are trademarks of Intel Corporation in the U.S. and/or other countries. Other trademarks may be trademarks of their respective owners. Published in the USA February 2019 Reference Architecture Guide H17577.
Dell Inc. believes the information in this document is accurate as of its publication date. The information is subject to change without notice.
Contents
3 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1
Reference Architecture Guide
Contents
Chapter 1 Executive Summary 5
Ready Solutions for Oracle ................................................................................... 6
Scope.................................................................................................................... 7
Audience ............................................................................................................... 7
We value your feedback ........................................................................................ 7
Chapter 2 Architecture and Design Considerations 8
Unity 650F storage design .................................................................................. 12
Compute and network design .............................................................................. 14
FC fabric connectivity and zoning ........................................................................ 14
Virtual network design ......................................................................................... 16
VM configuration ................................................................................................. 17
Guest operating system configuration ................................................................. 18
Chapter 3 Oracle Database 12cR2 Performance on Unity 650F 20
Test objectives .................................................................................................... 21
Use cases, test methods, and test results ........................................................... 22
Chapter 4 Oracle Database 18cR1 Performance on Unity 650F 32
Test objectives .................................................................................................... 33
Upgrading Oracle Database 12cR2 to 18cR1...................................................... 34
Use cases, test methods, and test results ........................................................... 36
Chapter 5 Summary 49
Summary ............................................................................................................ 50
Chapter 6 References 52
Dell EMC documentation..................................................................................... 53
VMware documentation ...................................................................................... 53
Oracle documentation ......................................................................................... 53
SLOB documentation .......................................................................................... 53
Appendix A Test Tools, and Database and SLOB Configuration 54
Testing and performance collection tools ............................................................ 55
SLOB dataset customization ............................................................................... 55
Database parameter configuration ...................................................................... 56
SLOB parameter settings (slob.conf)................................................................... 58
Contents
4 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1 Reference Architecture Guide
Appendix B Equipment List 59
Hardware components ........................................................................................ 60
Software components ......................................................................................... 62
Chapter 1: Executive Summary
5 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1
Reference Architecture Guide
Chapter 1 Executive Summary
This chapter presents the following topics:
Ready Solutions for Oracle ................................................................................ 6
Scope ................................................................................................................... 7
Audience .............................................................................................................. 7
We value your feedback ..................................................................................... 7
Chapter 1: Executive Summary
6 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1 Reference Architecture Guide
Ready Solutions for Oracle
Today’s database world faces many challenges including performance stabilization, data
consolidation, rapid creation of production copies, cost of organizational infrastructure
(space and storage), and other factors contributing to TCO. The challenges are becoming
more complex, critical, and intense as organizations are pushed to upgrade their
databases to the latest versions (for example, upgrading Oracle Database 12cR2 to
18cR1) because of business, technical, or infrastructure issues. Because of the
challenging circumstances, it is extremely important to offer and support a reference
architecture that is not only integrated, tested, and validated, but also addresses all the
customers’ pain points.
To address these challenges, Dell EMC Ready Solutions for Oracle provides a reference
architecture that features operational agility, efficiency, stability, resiliency, and storage
savings. Testing and validation of the architecture prove that customers can migrate from
Oracle Database 12cR2 to Oracle Database 18cR1 running on a Dell EMC Unity 650 All
Flash (Unity 650F) storage array with no adverse performance impact to production
workloads. This Dell EMC reference architecture for Oracle yields a faster time-to-value
along with superior performance, significant cost savings, and future-ready scalability.
Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Storage is a reference
architecture consisting of:
Unity 650F storage
Dell EMC PowerEdge R840 and R640 servers
Dell EMC Networking and Dell EMC Connectrix switches
VMware vSphere virtualization (version 6.5)
Oracle Databases 12cR2 and 18cR1
By eliminating the time-consuming and complex process of designing a system, this
tested and validated reference architecture streamlines the purchase and update cycles
for the IT organization and accelerates delivery times of complex mission-critical Oracle
Databases 12cR2 and 18cR1 and related applications. Features of this reference
architecture for Oracle include:
Significant data compression and storage savings, and reduced TCO
Support for simplified upgrading of Oracle Database 12cR2 to Oracle Database
18cR1 with no degradation of performance, as demonstrated by the performance
metrics in this guide
Consistent database performance while running an 80/20 read/write Silly Little
Oracle Benchmark (SLOB) stress test and creating two snapshots (both in 12cR2
and 18cR1 databases) hosted on the Unity 650F storage array
Overview
Design for Unity
reference
architecture
Chapter 1: Executive Summary
7 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1
Reference Architecture Guide
Scope
This reference architecture guide describes the architecture design and the process and
outcome of upgrading an Oracle Database 12cR2 to an Oracle Database 18cR1. The
guide describes how we tested and validated the architecture with multiple datasets and
workloads, with and without database snapshots, to ensure maximum flexibility and to
prove that upgrading the database has no negative impact on data integrity or
performance. This guide describes how to upgrade the database and discusses the
methodology and results of the testing that we conducted on the architecture.
Audience
This guide is for IT administrators, storage administrators, virtualization administrators,
system administrators, IT managers, and personnel who evaluate, acquire, manage,
maintain, or operate Oracle database environments.
We value your feedback
Dell EMC and the authors of this document welcome your feedback. Contact the Dell
EMC Solutions team by email or provide your comments by completing our
documentation survey.
Authors: Oracle Ready Solutions Engineering team, Indranil Chakrabarti, Reed Tucker
Note: The following page of the Oracle space on the Dell EMC Communities website provides
links to additional documentation for Dell EMC solutions for Oracle: Oracle Info Hub for Ready
Solutions.
Chapter 2: Architecture and Design Considerations
8 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1 Reference Architecture Guide
Chapter 2 Architecture and Design Considerations
This chapter presents the following topics:
Unity 650F storage design ................................................................................ 12
Compute and network design .......................................................................... 14
FC fabric connectivity and zoning ................................................................... 14
Virtual network design ...................................................................................... 16
VM configuration ............................................................................................... 17
Guest operating system configuration ............................................................ 18
Chapter 2: Architecture and Design Considerations
9 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1
Reference Architecture Guide
Architecture
This section provides an overview of the physical and logical architecture of the database
environment.
The following figure shows the major hardware components of the reference architecture.
Figure 1. Physical architecture
As shown in the physical architecture diagram, the architecture consists of a server layer,
network layer, and storage layer.
Server layer
The server layer consists of:
PowerEdge R840 virtual database server—One virtual OLTP database–
Oracle12cR2, which is later upgraded to Oracle 18cR1—is deployed in a single VM
running Red Hat Enterprise Linux (RHEL) 7.4 as the guest operating system. The
VM runs on a single PowerEdge R840 server with the VMware ESXi 6.5 hypervisor
installed. The server includes the following network components:
Two dual-port 10 GbE network interface controllers (NICs)—For Oracle
public traffic
Two dual-port 16 Gbps host bus adapters (HBAs)—For SAN traffic
Physical
architecture
Chapter 2: Architecture and Design Considerations
10 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1 Reference Architecture Guide
A minimum of one 1 GbE management remote Network Daughter Card
(rNDC) or LAN on motherboard (LOM) port—For in-band management of the
server from within the operating system
Dedicated 1 GbE iDRAC Ethernet port—For out-of-band management of the
server
PowerEdge R640 management server—The management server runs the
VMware ESXi 6.5 hypervisor. One 1 Gb rNDC port is used for management traffic
and two 10 GbE ports are used for the workload generation (SLOB I/O toolkit)
traffic.
Network layer
The network layer consists of:
Two 10 GbE network switches—Connect to two 10 Gb ports on the database
server to route the Oracle public traffic
Two 16 Gbps Fibre Channel (FC) fabric switches—Route SAN traffic between
the R840 database/ESXi host and the Unity 650F storage array
One 1 GbE network switch—Routes all management traffic between the
components—ESXi host, management server, switches, and Unity 650F storage
Storage layer
The storage layer consists of one Unity 650F storage array as the FC SAN storage to host
Oracle Database 12c and 18c. The Unity array tested in this reference architecture
consists of:
One disk processor enclosure (DPE) with two storage processor (SP) controllers
Four 16 Gbps front-end FC ports per SP
Usable storage capacity of 48.6 TB
The LAN and SAN design features redundant components and connectivity at every level
to ensure that there is no single point of failure. The design enables the application server
to reach the database server and the database server to reach the storage array even if
any of the following components fail:
One or more NICs or HBA ports
One LAN or FC switch
One or more Unity front-end ports
One Unity SP
For details about SAN zoning best practices and configuration, see FC fabric connectivity
and zoning in Chapter 2.
Chapter 2: Architecture and Design Considerations
11 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1
Reference Architecture Guide
The following figure shows the logical architecture overview of the database environment.
It shows the multiple layers of infrastructure components in the reference architecture,
along with a high-level overview of the software components that are deployed on the
individual hardware components.
Figure 2. Logical architecture
We tested and validated the following types of Oracle databases in the reference
architecture:
A single-node Oracle Database 12cR2 in a virtual environment—Oracle 12cR2
Grid infrastructure (GI) and a standalone Oracle Database 12cR2 run on one virtual
machine (VM).
A single-node Oracle Database 18cR1 in a virtual environment—Oracle 18cR1
GI and a standalone Oracle Database 18cR1 on one VM. This Oracle Database
18cR1 stack is upgraded from the Oracle Database 12cR2 stack.
As shown in Figure 2, one PowerEdge R840 server is the ESXi host, with ESXi 6.5 U2
hosting a single VM. The VM hosts Oracle Database 12cR2, which we later upgraded to
Oracle Database 18cR1. The R840 server has four Intel 18C CPUs and 1,536 GB RAM.
The VM uses RHEL 7.4 as the guest operating system that runs the Oracle 12cR2
Logical
architecture
Chapter 2: Architecture and Design Considerations
12 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1 Reference Architecture Guide
software stack, which includes Oracle 12cR2 GI and a standalone Oracle Database
12cR2.
After completing the Oracle Database 12cR2 validation and performance studies, we
upgraded the Oracle 12cR2 software stack to Oracle 18cR1; that is, we upgraded Oracle
12c GI to Oracle 18cR1 GI and Oracle Database 12cR2 to Oracle Database 18cR1.
A management server runs the VMware vCenter Server Appliance (VCSA) and the SLOB
benchmarking tool, which are deployed on two separate VMs.
The Unity 650F storage array hosts the storage volumes of Oracle Database 12c and 18c
as well the VM operating system disks. The Unity 650F storage array has two pools:
VM pool (VM_pool) —Stores the VM operating system LUN, three Oracle GI
clusterware storage LUNs for Oracle Cluster Registry (OCR), the voting disk, and
the GI Management Repository (GIMR).
Oracle Database pool (DB_Pool) —Stores all the database LUNs and the
snapshots of these database LUNs.
This architecture includes the following networks:
LAN/Public network—Provides the public network connection between the
database server and applications. For our test environment, this network connects
the SLOB benchmark server and the database server. This network is based on
10 GbE physical network components, as described in Physical architecture.
SAN network—Provides storage I/O communication between the database server
and the Unity storage array. The 16 Gbps Fibre Channel (FC) network is used for
this SAN network.
Management network—Manages the ESXi host, Unity storage array, and network
switches. This network is based on a 1 GbE physical network, as described in
Physical architecture.
Unity 650F storage design
Dell EMC Unity storage systems support two types of storage pools on all-flash storage
arrays: traditional pools and dynamic pools1. Dynamic pools provide many benefits over
traditional pools. The new pool structure eliminates the need to add drives in multiples of
RAID stripe-widths. Data space and replacement space are spread across the drives
within the pool. For greater flexibility in managing and expanding the pool, better drive
utilization, and improved application I/O, we recommend that you create a small number
of dynamic pools that include large numbers of drives of the same type. When
determining the number of pools, consider that there might be different types of
workloads, and dedicate resources to meet specific performance goals. For additional
Unity storage best practices for Oracle Database, see Dell EMC Unity Storage with Oracle
Databases.
1 Dynamic pools are supported in Unity OS version 4.4.x and higher. Refer to Appendix B for
storage array details
Chapter 2: Architecture and Design Considerations
13 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1
Reference Architecture Guide
The following table details the storage pools that we created for this reference
architecture.
Table 1. Unity storage array pool design
The following table details the database LUNs that we created in the storage pools. In
additional to these database LUNs, we created two snapshots of the database volumes to
study the impact of snapshot creation on database performance and storage capacity.
Table 2. Oracle Database LUN design on Unity storage
LUN Purpose VM_POOL DB_POOL Thin LUN Data reduction enabled?
VM OS VM operating system volume
1 x 500 GB Yes Yes
OCR/VD Clusterware and virtual disk (VD) storage
3 x 50 GB Yes Yes
DATA Database files
4 x 500 GB Yes Yes
REDO REDO logs 4 x 25 GB Yes Yes
TEMP TEMP tablespace
1 x 500 GB Yes Yes
FRA Flash Recovery Area for archive logs
2 x 100 GB Yes Yes
Pool name RAID configuration
Capacity Purpose Number of LUNs
VM_Pool RAID 5 2.8TB VM operating system LUN and Oracle Clusterware LUNs
4
DB_Pool RAID 5 45.8 TB Database LUNs and their snapshots
11 original database LUNs plus additional snapshots upon creation
Chapter 2: Architecture and Design Considerations
14 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1 Reference Architecture Guide
Compute and network design
We configured the PowerEdge R840 database server as follows:
Installed ESXi 6.5 by using the Dell EMC customized ISO image (Dell Version: A04,
Build# 5310538), which is available on Dell EMC Online Support at VMware ESXi
6.5.
Zoned two dual-port 16 Gb/s HBAs, four initiators in total, and configured them with
the Unity 650F front-end FC ports for high-bandwidth, load-balanced, and highly
available SAN traffic. For the recommended FC connectivity and zoning best
practices, see FC fabric connectivity and zoning. For optimal performance, the two
HBA cards are populated in slots 2 and 5 of the R840 server.
Configured one 1 GbE rNDC or LOM port for the management traffic and two 10
GbE ports for the Oracle public traffic. For more details on the virtual networking
design, see Virtual network design. For optimal performance, we installed the two
10 GbE network adapters in slots 3 and 6 of the R840 server.
Created a single VM with RHEL 7.4 as the guest operating system for the virtual
Oracle standalone databases. For more details about the VM properties and best
practices, see VM configuration.
We configured, monitored, and maintained the ESXi host, virtual networking, and the VM
by using VMware vSphere Web Client and VCSA, which is deployed as a VM on the
management server.
Multipath configuration
We configured multipathing on the ESXi 6.5 host according to the following best practices:
Use vSphere Native Multipathing (NMP) as the multipathing software.
Retained the default selection of round-robin for the native path selection policy
(PSP) on the Unity volumes that are presented to the ESXi host.
Change the NMP round-robin path switching frequency from the default value
(1,000 I/O packets) to 1. For information about how to set this parameter, see Dell
EMC Unity Storage with VMware vSphere.
FC fabric connectivity and zoning
The following figure shows the recommended FC connectivity between the HBAs and the
FC switches, and the connectivity between the FC switches and the Unity storage array.
As shown in the figure, each port in each HBA connects to two separate FC switches, and
two front-end ports from each of the Unity SPs connect to two separate FC switches,
forming two FC fabrics.
ESXi host
configuration
Chapter 2: Architecture and Design Considerations
15 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1
Reference Architecture Guide
Figure 3. FC fabric connectivity design
Dell EMC recommends single-initiator zoning when creating zone sets on the FC
switches. For high availability, bandwidth, and load balancing, each initiator or HBA port
on the ESXi host is zoned with four front-end Unity storage ports that are spread across
the two storage controllers or SPs, as shown in the following logical representation of
zone sets.
Figure 4. FC zoning logical representation
Chapter 2: Architecture and Design Considerations
16 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1 Reference Architecture Guide
Virtual network design
The following diagram shows a high-level overview of the virtual network design that we
implemented in the ESXi host. It also shows the mapping between the virtual switch
network and the physical switch network.
Figure 5. Virtual network design in the ESXi host
The virtual network design supports:
VM and management traffic—The VM network and management traffic uses the
default standard virtual switch (vSwitch), which contains two default standard ports
groups. The Management Network port group provides the VMkernel port vmk0 to
manage the ESXi host from VCSA. The VM Network port group provides the 1 GbE
virtual interfaces for in-band management of the database VM. All management
traffic is routed through the 1 GbE physical rNDC or LOM port on the ESXi server
that is connected to the external 1 GbE management switch virtual or network.
Public traffic—The Oracle public traffic uses an additional dedicated standard
vSwitch to which we assigned two physical 10 GbE uplink ports. For high
availability and load balancing, the two 10 GbE uplink ports reside on two separate
physical network adapters and are connected to two separate 10 GbE physical
network switches. Within this vSwitch, we created a standard port group that
provides the virtual network interface for the Oracle public traffic within the
database VM.
Chapter 2: Architecture and Design Considerations
17 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1
Reference Architecture Guide
VM configuration
We used the following design principles and best practices to create the database VM:
SCSI controllers—We created multiple SCSI controllers to optimize and balance
the I/O for the different database disks, as shown in the following table. We chose
the controller type VMware Paravirtual for optimal performance.
Table 3. SCSI controller properties in the database VM
Controller Purpose SCSI bus sharing Type
SCSI 0 Guest operating system disk None VMware Paravirtual
SCSI 1 Oracle DATA disks Physical VMware Paravirtual
SCSI 2 Oracle REDO disks Physical VMware Paravirtual
SCSI 3 Oracle OCR, GIMR, FRA, TEMP
Physical VMware Paravirtual
Hard disk drives—We assigned the following properties to all database-related
virtual disks (for example, DATA, REDO, FRA, OCR/VD, and TEMP):
Raw Device Mapping (RDM)—For optimal performance and management
simplicity, all Oracle related disks presented to the ESXi host from the Unity
storage array are mapped directly as raw devices to the database VM.
Virtual Device Node—For load balancing and optimal performance, the SCSI
controllers are assigned as noted in Table 3.
VM vCPU and vMem—The following table lists the amount of virtual CPU (vCPU)
and virtual memory (vMem) that we assigned to the database VM during the testing
of the reference architecture.
Table 4. VM configuration: vCPU and vMem details
vCPUs vMem
Number of vCPUs
Limit (MB) Reservation (GB)
Total (GB) Limit (MB)
18 Unlimited 192 256 Unlimited
Network adapters—We added two network adapters, one for in-band VM or
guest operating system management and one for Oracle public traffic, to the
database VM. We configured the two adapters with the recommended type setting
of VMXNet 3.
Enable disk UUID—In each of the VM options, we added the configuration
disk.enableUUID parameter and set it to TRUE. This setting ensures that the
VMDK always presents a consistent disk UUID to the VM.
Chapter 2: Architecture and Design Considerations
18 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1 Reference Architecture Guide
Guest operating system configuration
In this reference architecture, we use the following best practices to deploy and configure
RHEL 7.4 as the guest operating system in the VM running the Oracle standalone
database:
Install and configure the operating system, network, storage disks, Oracle 12cR2
Grid, and standalone Oracle Database 12cR2 within the VM, as instructed in the
following Dell EMC knowledge base article: How to deploy Oracle 12c Release 2
Standalone Database on RHEL 7.x
Set up the Oracle Grid and database software prerequisites (required operating
system RPMs, users, groups, kernel parameters, and so on) by using the
information and deployment package in the following Dell EMC knowledge base
article: Dell Oracle Deployment RPMs for Oracle 12cR2 on RHEL7.x
Important best practices include:
For each Oracle virtual disk, create a single partition that spans the entire disk
and has a starting offset of 2,048 sectors.
Ownerships and permissions on the Oracle disks within the VM are established
using UDEV rules. The following example shows a UDEV rule set for one of the
Oracle disks (REDO disk) within the custom /etc/udev/rules/60-oracle-
asmdevices.rules UDEV rules file:
KERNEL=="sd[a-z]*[1-9]", SUBSYSTEM=="block",
PROGRAM=="/usr/lib/udev/scsi_id -g -u -d /dev/$parent",
RESULT=="3600601600f004300accaed5bd9741db5",
SYMLINK+="oracleasm/disks/ora-redo1", OWNER="grid",
GROUP="asmadmin", MODE="0660"
Note: For this reference architecture, we tested the standalone Oracle Database 18c by
performing an in-place upgrade of the deployed 12cR2 database. For an overview of the upgrade
process, see Upgrading Oracle Database 12cR2 to 18cR1.
As described in Unity 650F storage design, we mapped all Oracle related LUNs that are
presented to the ESXi host from the Unity 650F storage array directly as raw devices to
the database VM through RDM. In compliance with the UDEV rules, we assigned the
ownership of the raw devices to the grid user who is the owner of the Oracle GI and
Oracle Automatic Storage Management (ASM). The device link for these Oracle related
raw devices is /dev/oracleasm/disks/ora-XXX. For example,
/dev/oracleasm/disks/ora-redo1 is the device link for REDO1 LUN/raw device.
The following table shows the Oracle disk groups that are created based on these
LUNs/raw devices. Except for the OCR disk group that uses the normal redundancy (with
triple mirroring), all other disk groups use the external redundancy setting. The coarse
striping setting is also used for DATA, FRA, and OCR disk groups, and the fine-grain
striping setting is used for REDO1, REDO2, and TEMP disk groups.
Chapter 2: Architecture and Design Considerations
19 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1
Reference Architecture Guide
Table 5. ASM disk group design
ASM disk group
Purpose Redundancy ASM striping ASM disk group size (GB)
LUN LUN size (GB)
DATA Data files, control files, undo tablespace
External redundancy
Coarse 2,000 DATA00 500
DATA01 500
DATA02 500
DATA03 500
FRA Archive log files
External redundancy
Coarse 200 FRA0 100
FRA1 100
REDO1 Online redo logs
External redundancy
Fine-grain 50 REDO0 25
REDO1 25
REDO2
Online redo logs
External redundancy
Fine-grain
50 REDO2 25
REDO3 25
TEMP
Temp files External redundancy
Fine-grain
500 TEMP
500
OCR OCR, voting disk, GIMR
Normal redundancy
Coarse
50 OCR0 50
OCR1 50
OCR3 50
Oracle ASM has a feature to move the data to higher performance tracks of the spinning
disks in the compact phase at the end of ASM disk rebalancing. This feature has no
benefit for Dell EMC Unity storage when the physical storage is virtualized and the flash
devices are used. You can disable the rebalancing feature by running the alter
diskgroup command for all the disk groups. The following example shows the command
for the DATA disk group:
SQL> alter diskgroup DATA set attribute '_rebalance_compact' =
'FALSE';
For more information about the ASM disk group guidelines, see Dell EMC Unity Storage
with Oracle Databases. For more information about ASM compact rebalancing, see
Oracle Support note 1902001.1.
Chapter 3: Oracle Database 12cR2 Performance on Unity 650F
20 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1 Reference Architecture Guide
Chapter 3 Oracle Database 12cR2 Performance on Unity 650F
This chapter presents the following topics:
Test objectives .................................................................................................. 21
Use cases, test methods, and test results ...................................................... 22
Chapter 3: Oracle Database 12cR2 Performance on Unity 650F
21 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1
Reference Architecture Guide
Test objectives
In a typical IT environment, databases might be required for testing, development,
reporting, or online analytics. Usually, these additional databases must be based on
copies of the production databases because the new features or hardware cannot be
tested directly on the production systems themselves.
The Dell EMC Unity 650F storage array’s snapshot feature enables you to create multiple
copies of any database. The objective of the tests described in this chapter is to simulate
a typical customer environment in which we create a baseline Oracle production
database, measure its performance, and then create multiple snapshot copies of it to
measure the impact of creating snapshots. Another objective is to study the data savings
feature of the Unity storage as applied to the Oracle databases and analyze how it can
help customers with data compression and space savings. These savings will ultimately
translate into storage-cost and TCO savings.
In our use cases, we analyze the impact of upgrading Oracle Database 12c to the 18c
version on the related performance and data savings numbers. This data benefits the
DBAs and test/dev engineers who frequently must spend hours managing database
creation and refreshing the environments, often while limited by capacity, performance,
and number of database copies.
The following use cases demonstrate the performance and capacity savings of Oracle
Database 12cR2 running on the Unity 650F storage array, as well as the performance
impact of creating Unity snapshots of Oracle Database 12c:
Use case 1: Deduplication and compression of Oracle Database 12cR2
Use case 2: Oracle Database 12cR2 baseline performance
Use case 3: Unity storage snapshot-creation impact on Oracle Database 12cR2
performance
Use case 4: Deduplication and compression of Oracle Database 12cR2 with data
changes
Our use-case testing included stress and compression testing to produce the performance
numbers that are shown in this reference architecture guide. The data was extracted from
Oracle Automatic Workload Repository (AWR) reports. Compression and performance
results documented in this guide are provided as a reference. The actual numbers you
achieve might vary with your environment.
Note: During all testing, the Unity inline data reduction feature was enabled, as noted in Unity
650F storage design. This feature uses some storage CPU and memory cycles.
Chapter 3: Oracle Database 12cR2 Performance on Unity 650F
22 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1 Reference Architecture Guide
Use cases, test methods, and test results
In this use case, we loaded data into Oracle Database 12cR2 on the Unity 650F storage
array to understand the impact of data compression and deduplication, also referred to as
data reduction. We tested this reference architecture by loading approximately 1.2 TB of
data into the Oracle 12cR2 test database using the SLOB multiple schema model. The
native SLOB schema contains highly redundant data and, therefore, is highly compressible.
To remove redundancy and to showcase the Unity array’s data compression capabilities
in the worst-case scenario, we used a custom PL/SQL script to insert randomized and
unique data into the SLOB schema (for details, see Appendix A).
After loading approximately 1.2 TB of data into the Oracle 12cR2 test database, a space
savings of 29.12 percent was realized from the data reduction features in the storage
layer, as shown in the following figure.
Figure 6. Space savings for Oracle Database 12cR2 running on Unity 650F storage
As shown in the following figure, the CLI (sqlplus) interface reflects the schema capacity
as seen by the Oracle database. The database sees the entire 1,224.91 GB of the data
loaded in the IOPS tablespace, which validates that all the reduced physical capacity as
seen on the storage is due to the Unity array's data reduction feature and is transparent to
the database.
Use case 1:
Deduplication
and compression
of Oracle
Database 12cR2
Chapter 3: Oracle Database 12cR2 Performance on Unity 650F
23 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1
Reference Architecture Guide
Figure 7. CLI interface (sqlplus) showing schema capacity as seen by the database
In this test, we loaded 100 percent unique data that was generated by the PL/SQL
program. Through the Unity 650F storage compression capabilities, we reduced the
amount of storage space required by 29 percent on a loaded data volume of 1,225 GB.
The actual data reduction numbers might vary, depending on the workload, quality of
data, or other factors that are unique to any typical organization.
Note: The compression numbers that are shown in this guide were generated by Dell EMC
engineers on in-house equipment and are for reference purposes only.
The compression feature of the Dell EMC Unity 650F storage array generates storage
cost savings (CAPEX) and TCO savings for a typical organization. For a similar
demonstration of storage data reduction in Oracle Database 18c, see Use Case 1:
Deduplication and compression of Oracle Database 18c in Chapter 4.
In this use case, we created a standalone Oracle Database 12cR2 on a Dell EMC
PowerEdge R840 server, as shown in the following figure. We ran the performance test
for 30 minutes using SLOB on an OLTP workload featuring an 80/20 read/write mixture.
This database features an 8 KB block size with Automatic Storage Management (ASM) in
a coarse-striped and externally redundant configuration.
Figure 8. Performance testing on 12cR2 DB running on Unity 650F (Test 1)
Use case 2:
Oracle Database
12cR2 baseline
performance
Chapter 3: Oracle Database 12cR2 Performance on Unity 650F
24 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1 Reference Architecture Guide
During the stress test, we collected performance data from the AWR report that was
generated by the Oracle database. For database and SLOB parameter settings used
during all test cases, see Appendix A. The following table shows the performance metrics
that we captured from AWR for Test 1. We used these values as the baseline numbers for
comparison in Use Cases 3 and 4.
Table 6. Test 1 performance results
Performance metric Value
IOPS 101,727
Database server CPU utilization (%) 25
Database bandwidth (MB/s) 805
Database response time (milliseconds) 0.32
Transactions per second (TPS) 6,869
The performance metrics in Table 6 show that the IOPS value was over 100,000 with an
average database server CPU utilization of 25 percent and a database response time of
0.32 milliseconds (ms). The database server had plenty of capacity for performing other
tasks while running this Oracle Database 12c.
The database bandwidth was healthy (805 MB/s) and the average response time for
queries was quite fast at 0.32 ms. Also, the database performed 6,869 TPS, which means
the commits and rollbacks were happening very quickly. We use the Test 1 results as a
baseline to later compare these results to those that we obtained while creating snapshots
in Use Case 3.
This use case shows the performance efficiency of Oracle Database 12cR2 on the Unity
650F storage array. In Chapter 4, we compare the results in Table 6 to the results we
obtained when running the upgraded Oracle Database 18cR1 on the Unity 650F array.
The goal of this use case is to study the performance impact of creating Unity snapshots
on Oracle Database 12cR2. This use case involves two tests—Test 2 and Test 3:
In Test 2, we performed a 30-minute stress test using SLOB and, at the same time,
created snapshots to measure performance impact, similar to the testing in Use
Case 2.
In Test 3, we created two snapshots and then performed SLOB stress testing to
observe and understand the resulting change in performance numbers.
Note: Before taking the snapshots, we created a consistency group on the Unity storage array and
added all Oracle database volumes to it. Dell EMC recommends taking snapshots of database
volumes at the consistency group level rather than at the individual database-volume level to
guarantee that the Oracle database snapshots can mount and restart successfully on the
database host.
Use case 3:
Unity storage
snapshot-creation
impact on Oracle
Database 12cR2
performance
Chapter 3: Oracle Database 12cR2 Performance on Unity 650F
25 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1
Reference Architecture Guide
The following figures show the methodology for Tests 2 and 3.
Figure 9. Test 2 methodology
Figure 10. Test 3 methodology
The goals of Tests 1, 2, and 3 were to:
Capture the baseline performance of Oracle Database 12cR2 (Test 1)
Capture the performance impact on the baseline database during its snapshot
creation (Test 2)
Capture the performance impact on the baseline database after its snapshot is
created (Test 3)
Chapter 3: Oracle Database 12cR2 Performance on Unity 650F
26 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1 Reference Architecture Guide
Compare the following performance metrics from the three tests:
IOPS
TPS
Database server CPU utilization (%)
DB bandwidth
DB response time
IOPS results
The following figure shows the comparison of IOPS among Test 1 (Use Case 2), and
Tests 2 and 3 (Use Case 3) on Oracle Database 12c.
Figure 11. Total IOPS for Test 1, Test 2, and Test 3
As shown in Figure 11, with the snapshots created in Test 2 and Test 3, including the
SLOB stress testing in Test 3, the IOPS dropped less than 0.7 percent in comparison to
Test 1 (the baseline test). The IOPS numbers from these three tests prove that, despite
requiring more system resources like drive I/O to handle metadata writes, this reference
architecture generates impressive IOPS numbers.
Chapter 3: Oracle Database 12cR2 Performance on Unity 650F
27 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1
Reference Architecture Guide
Database server CPU utilization results
The following figure shows the database server CPU utilization as captured by the AWR
report for Tests 1, 2, and 3.
Figure 12. Database server CPU utilization (%) comparison for Tests 1, 2, and 3
Creating two snapshots caused virtually no change in CPU utilization in Test 2 and Test 3.
In Test 3, which included stress testing with the SLOB tool after creating snapshots, CPU
utilization decreased very slightly as compared to Test 2. Tests 2 and 3 prove that
snapshot creation and stress testing do not have a major impact on the Oracle Database
12cR2 server running on the Unity 650F storage array.
Any data that is written to either the baseline database or to the snapshot database is
redirected to a new write location in the same storage pool. The Unity storage array uses
metadata to track data blocks belonging to the base objects, and snapshots of metadata
consume more storage system resources such as CPU and memory to handle metadata
updates. Even considering the metadata updates, there was no increase in database CPU
utilization (Test 3) as compared to Test 1 (the baseline test).
Chapter 3: Oracle Database 12cR2 Performance on Unity 650F
28 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1 Reference Architecture Guide
Database bandwidth results
The following figure compares database bandwidth in terms of MB/s for Oracle Database
12cR2 for Tests 1, 2, and 3.
Figure 13. Comparative analysis of database bandwidth for Test 1, Test 2, and Test 3
Figure 13 demonstrates that creating two snapshots and performing OLTP operations
such as SLOB data loading do not adversely impact the bandwidth. Therefore, there is no
impact on the performance of Oracle Database 12cR2 running on the Unity 650F storage
array. On the contrary, the database bandwidth increased by 3 to 4 percent during
performance stress testing and snapshot creation.
Chapter 3: Oracle Database 12cR2 Performance on Unity 650F
29 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1
Reference Architecture Guide
Database response time results
The following figure shows that the database response remained the same while
snapshots were created during stress testing (Test 2) and when snapshots were created
before the stress testing (Test 3) when compared with the baseline number (Test 1).
Therefore, creating snapshots had no impact on the latency performance of the baseline
Oracle Database 12cR2 running on the Unity 650F storage array, which is notable
performance considering that there was also virtually no increase in the CPU utilization.
Figure 14. Database response time during Test 1, Test 2, and Test 3
Chapter 3: Oracle Database 12cR2 Performance on Unity 650F
30 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1 Reference Architecture Guide
Transactions per second results
Transactions per second (TPS) is also known as transaction throughput. The following figure
shows the TPS results in Tests 1, 2, and 3. The figure shows a minimal drop in TPS of less
than 1.3 percent in Test 2 and Test 3 compared with the baseline number from Test 1.
Figure 15. TPS during Test 1, Test 2, and Test 3
The performance metrics from Tests 1, 2, and 3 show that there was minimal impact from
creating snapshots and applying stress testing, and there was no performance impact
when running an Oracle Database 12cR2 on the Unity 650F storage array. This ability to
maintain performance is helpful when you have to create multiple copies of the production
database while the production OLTP workloads are running in parallel.
Chapter 3: Oracle Database 12cR2 Performance on Unity 650F
31 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1
Reference Architecture Guide
Test methodology
To further determine the storage capacity savings that is realized through the use of the
deduplication and compression feature of the Unity 650F storage array during a data
change, we increased the data that was loaded during Use Case 1 by 5 percent and
captured the savings in storage capacity. The following diagram shows the comparison of
storage capacity before and after the 5 percent data increase.
Figure 16. Data reduction achieved in 12c database after 5% data insertion
Test results
We performed Use Case 4 to find the data reduction percentage after inserting data in six
new SLOB schemas with 100 percent randomized and unique data generated by a
PL/SQL program. After loading the data in six new schemas, we achieved a space
savings of 26.15 percent, as shown in Figure 16. As described in Chapter 4, we achieved
similar savings by running the same tests on an Oracle Database 18cR1.
Use Case 4:
Deduplication
and compression
of Oracle
Database 12cR2
with data changes
Chapter 4: Oracle Database 18cR1 Performance on Unity 650F
32 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1 Reference Architecture Guide
Chapter 4 Oracle Database 18cR1 Performance on Unity 650F
This chapter presents the following topics:
Test objectives .................................................................................................. 33
Upgrading Oracle Database 12cR2 to 18cR1 .................................................. 34
Use cases, test methods, and test results ...................................................... 36
Chapter 4: Oracle Database 18cR1 Performance on Unity 650F
33 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1
Reference Architecture Guide
Test objectives
The preceding chapter describes the performance and capacity saving studies of Oracle
Database 12cR2 on the Unity 650F storage array. This chapter focuses on similar studies
of Oracle Database 18cR1. We upgraded Oracle Database 12cR2 to 18cR1, conducted
the same performance and capacity-saving studies, and compared the results. Because
we upgraded the previously used Oracle Database 12cR2, both the 12c and 18c test
databases shared the same data and the same configuration of the entire stack, from the
storage to the ESXi host/guest VM operating system and the Oracle database. Through
the same set of use cases as described in Chapter 3, we can compare the performance
and capacity savings on the Unity 650F storage array before and after upgrading the 12c
database to 18c.
To achieve these objectives, we first upgraded the 12cR2 database to 18cR1 as shown in
the following figure.
Figure 17. Upgrading Oracle Database 12c to 18c
Chapter 4: Oracle Database 18cR1 Performance on Unity 650F
34 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1 Reference Architecture Guide
Upgrading Oracle Database 12cR2 to 18cR1
We upgraded the database by using Oracle Data Pump technology and the upgradable
tablespace method. Upgrading from Oracle Database 12cR2 to 18cR1 involves the
following steps:
1. Upgrade the GI from 12cR2 to 18cR1.
2. Upgrade the Oracle Database from 12cR2 to 18cR1.
This section provides a high-level overview of the process. For detailed information about
the Oracle 18cR1 upgrade, see the following Oracle documentation:
Oracle Database Upgrade Guide 18c
Oracle Support MOS note (Doc ID 2418576.1): Oracle 18c - Complete Checklist for
Upgrading to Oracle Database 18c (18.x) using DBUA
To upgrade Oracle 12cR2 GI to 18cR1:
1. Stage the software.
Download the 18c GI binary LINUX.X64_180000_grid_home.zip and unzip
the files to the new 18c GI home: /u01/app/18.3.0/grid
2. Ensure that the prerequisites for upgrading to Oracle 18c GI are met.
Check the version and status of the current clusterware with crsctl commands
and run the clusterware verification utility runcluvfy.sh as shown in this
command:
$ /u01/app/18.3.0/grid/runcluvfy.sh stage -pre hacfg
3. Apply the pre-upgrade GI 27006180 patch on the Oracle 12cR2 (12.2.0.1.0) GI
home (see Doc ID 2414935.1).
Download and unzip this patch (p27006180_122010_Linux-x86-64.zip) to
the /home/grid/patches directory, and then use the opatchauto utility to
apply the patch to 12.2.1.0 GI home as the grid user:
[grid@]$/u01/app/12.2.0/grid/OPatch/opatchauto apply
/home/grid/patches/27006180
–h /u01/app/12.2.0/grid
4. Run the following command to validate that the GI patch 27006180 is applied
successfully on the 12cR2 GI home:
$/u01/app/12.2.0/grid/OPatch/opatch lsinventory
5. Before upgrading the 12cR2 GI, back up the clusterware configuration including
OCR.
6. Shut down the 12cR2 database and run the Oracle GI 18c installer setup.sh,
selecting upgrade the Oracle Grid Infrastructure to upgrade the 12cR2 GI to
18c.
Upgrade 12cR2
Grid Infrastructure
to 18cR1
Chapter 4: Oracle Database 18cR1 Performance on Unity 650F
35 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1
Reference Architecture Guide
To upgrade Oracle Database 12cR2 to 18cR1:
1. Install Oracle Database 18c software.
Download and unzip the Oracle Database 18c software
LINUX.X64_180000_db_home.zip, and then run the installer to install the
software to the new 18c Oracle home:
/u01/app/oracle/product/18.3.0/dbhome_1
During the installation, select the Set up Software Only configuration option.
Also, during database installation, select the Single Instance database
installation option and select Enterprise Edition for the database edition.
2. Restart the 12cR2 database, and then run the pre-upgrade information tool
(preupgrade.jar) command:
java -jar
/u01/app/oracle/product/18.3.0/dbhome_1/rdbms/admin/preupgra
de.jar TERMINAL
This command checks the current Oracle Database 12cR2 and identifies any
required pre-upgrade actions. The output of this command includes the pre-
upgrade actions and post-upgrade actions.
3. Perform the 18cR1 upgrade with the dbua upgrade utility from the 18c Oracle
Database home page.
The dbua upgrade utility prompts the database to upgrade. To speed up the
upgrade process, select Enable parallel upgrade and Recompile invalid
objects during post upgrade. Once the upgrade is complete, upgrade results
are displayed, as shown in the following figure.
Figure 18. Upgrade results
Upgrade Oracle
Database 12cR2
to 18cR1
Chapter 4: Oracle Database 18cR1 Performance on Unity 650F
36 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1 Reference Architecture Guide
Use cases, test methods, and test results
The following use cases for Oracle Database 18cR1, which mirror our 12cR2 use cases,
demonstrate the performance and capacity savings of Oracle Database 18cR1 running on
the Unity 650F storage array, as well as the performance impact of creating Unity
snapshots of Oracle Database 18cR1:
Use case 1: Deduplication and compression of Oracle Database 18cR1
Use case 2: Oracle Database 18cR1 baseline performance
Use case 3: Unity storage snapshot-creation impact on Oracle Database 18cR1
performance
Use case 4: Deduplication and compression of Oracle Database 18cR1 with data
changes
To establish the comparison with the 12cR2 database, we used the same test methods
and test configuration and ensured that the 18cR1 database contained the same data as
the 12cR2 database.
To establish the comparison of the deduplication and compression (data reduction)
savings between the 12c database and the 18c database, we reloaded the 1.2 TB of test
data that was used for Test 1 of the 12c database, as described in Chapter 3. We
observed that the size of the database was 1,223 GB, as shown in the following figure.
We inserted 100 percent randomized and unique data (generated by the PL/SQL
program) stored in 128 SLOB schemas. These schemas are initially created by a SLOB
data load and then the data from those schemas is truncated and repopulated with
randomized data. As shown in the following figure, we achieved a data reduction rate of
29.75 percent.
Figure 19. Data reduction in 18c database
Use Case 1:
Deduplication
and compression
of Oracle
Database 18cR1
Chapter 4: Oracle Database 18cR1 Performance on Unity 650F
37 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1
Reference Architecture Guide
When we compare the space savings of Oracle Database 12cR2 with Oracle Database
18cR1, we observe a slight improvement, as shown in the following figure.
Figure 20. Data reduction observed between 12c and 18c databases for Use Case 1
In Use Case 2, we ran the performance test for 30 minutes using SLOB to generate an
OLTP workload with an 80/20 read/write mixture on the PowerEdge R840 server, as
shown in the following figure. We used the same database configuration for the 18cR1
database as we used for the 12cR2 database.
Figure 21. Performance testing on 18cR1 database running on Unity 650F (Test 1)
Use case 2:
Oracle Database
18cR1 baseline
performance
Chapter 4: Oracle Database 18cR1 Performance on Unity 650F
38 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1 Reference Architecture Guide
For database and SLOB parameter settings that were used during all test cases, see
Appendix A. During the stress test, we collected performance data from the AWR report
generated by the Oracle database. The following table shows the performance metrics
that we captured from AWR for Test 1. We used these values as the baseline numbers for
comparison with Test 2 and Test 3 in Use Case 3.
Table 7. Test 1 performance results
Performance metric Value
IOPS 104,067
Database server CPU utilization (%) 26
Database bandwidth (MB/s) 823
Database response time (milliseconds) 0.31
Transactions per second (TPS) 7,023
The performance metrics in Table 7 show that the IOPS value was over 100,000 with an
average database server CPU utilization of 26 percent and a database response time of
0.31 milliseconds (ms). The database server had plenty of capacity for performing other
tasks while running this Oracle 18c database.
The database bandwidth was healthy (823 MB/s) and the response time for queries was
quite fast at 0.31 ms. Also, the database performed 7,023 TPS, which means that the
commits and rollbacks were happening very quickly. We use the Test 1 results as a
baseline to later compare these results to those that we obtained while creating snapshots
in Use Case 3.
The goal of Use Case 3 is to study the performance impact of creating Unity snapshots on
the Oracle 18cR1 OLTP database. This use case involves two tests—Test 2 and Test 3:
In Test 2, we performed a 30-minute stress test using SLOB and, at the same time,
created snapshots to measure performance impact, similar to the testing in Use
Case 2.
In Test 3, we created two snapshots and then performed SLOB stress testing to
observe and understand the resulting change in performance numbers.
Note: Before taking the snapshots, we created a consistency group on the Unity storage array and
added all Oracle database volumes to it. Dell EMC recommends taking snapshots of database
volumes at the consistency group level rather than at the individual database-volume level to
guarantee that the Oracle database snapshots can mount and restart successfully on the
database host.
The following figures show the methodology for Test 2 and Test 3.
Use case 3:
Unity storage
snapshot-creation
impact on Oracle
Database 18cR1
performance
Chapter 4: Oracle Database 18cR1 Performance on Unity 650F
39 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1
Reference Architecture Guide
Figure 22. Test 2 methodology
Figure 23. Test 3 methodology
As shown in Figure 24, with the snapshots created in Test 2 and Test 3 including the
SLOB stress testing in Test 3, the IOPS dropped less than 0.7 percent in comparison to
Test 1 (the baseline test). The IOPS numbers from these three tests prove that, despite
requiring more system resources like drive I/O to handle metadata writes, this reference
architecture generates impressive IOPS numbers.
Tests 2 and 3 let us study the performance impact after running stress testing on an
Oracle Database 18cR1 with or without snapshots by comparing the following benchmark
parameters:
IOPS
TPS
Chapter 4: Oracle Database 18cR1 Performance on Unity 650F
40 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1 Reference Architecture Guide
Database server CPU utilization (%)
DB bandwidth
DB response time
IOPS results
The following figure shows the comparison of IOPS among Test 1 (Use Case 2) and Tests
2 and 3 (Use Case 3) on Oracle Database 18cR1.
Figure 24. Total IOPS numbers for Test 1, Test 2, and Test 3
As shown in Figure 24, with the snapshots created in Test 2 and Test 3, including the
SLOB stress testing in Test 3, the IOPS dropped less than 1 percent in comparison to
Test 1 (the baseline test). The IOPS numbers from these three tests prove that, despite
requiring more system resources like drive I/O to handle metadata writes, this reference
architecture generates impressive IOPS numbers.
The following figure compares the number of IOPS generated by the 12c and 18c
databases during the three tests.
Chapter 4: Oracle Database 18cR1 Performance on Unity 650F
41 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1
Reference Architecture Guide
Figure 25. Comparative analysis of total IOPS numbers for Test 1, Test 2, and Test 3
As shown in Figure 25, all three tests generated slightly more IOPS with the 18c database
than with the 12c database. In Tests 2 and 3, despite snapshots being created during
stress testing (Test 2) and before stress testing (Test 3), the reduction in the number of
IOPS is minimal. Also, the IOPS numbers achieved by the 18c database are consistently
higher than those of the 12c database.
Database server CPU utilization results
The following figure shows the database server CPU utilization as captured by the AWR
report for Tests 1, 2, and 3.
Figure 26. Database server CPU utilization (%) comparison for Tests 1, 2, and 3
Creating two snapshots in Test 2 caused database server CPU utilization to increase very
little in Test 2 and Test 3. In Test 3, which included stress testing with the SLOB tool after
Chapter 4: Oracle Database 18cR1 Performance on Unity 650F
42 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1 Reference Architecture Guide
creation of snapshots, CPU utilization actually decreased by 1 percent in comparison to
Test 2. Tests 2 and 3 prove that snapshot creation and stress testing have very little
impact on the Oracle Database 18cR1 server running on the Unity 650F storage array.
Any data written to either the baseline database or to the snapshot database is redirected
to a new write location in the same storage pool. The Unity storage array uses metadata
to track data blocks that belong to the base objects, and all snapshots consume more
storage system resources such as CPU and memory to handle metadata updates. Even
considering the metadata updates, there was no significant increase in the database
server CPU utilization percentage. Therefore, there is no significant performance impact
on Test 2 and Test 3 in comparison to Test 1 (the baseline test).
Comparing database server CPU utilization during the tests running on the Oracle 12c
database with those on the 18cR1 database, we find that database server CPU utilization
decreases with the 18cR1 database, as shown in the following figure.
Figure 27. Comparative analysis of database server CPU utilization (%) during Test 1, Test 2 and Test 3
This leaves plenty of unused CPU resources available for other activities.
Database bandwidth results
The following figure compares database bandwidth in terms of MB/s for Oracle Database
18cR1 for Tests 1, 2, and 3.
Chapter 4: Oracle Database 18cR1 Performance on Unity 650F
43 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1
Reference Architecture Guide
Figure 28. Comparative analysis of DB bandwidth during Test 1, Test 2, and Test 3
Figure 28 demonstrates that creating two snapshots and performing OLTP operations
such as SLOB data loading do not adversely affect the bandwidth, so there is no impact
on the performance of the Oracle Database 18cR1 running on the Unity 650F storage
array. On the contrary, the database performance (bandwidth) increases by about 2
percent as more workloads are applied along with the creation of snapshots during Test 2.
The following figure compares the bandwidth results for the Oracle 12cR2 and 18cR1
databases during the three tests.
Figure 29. Comparative analysis of database bandwidth during Test 1, Test 2, and Test 3
As shown, the bandwidth numbers that were recorded during 12c and 18c database
testing remained the same or improved slightly despite snapshots being created during
Chapter 4: Oracle Database 18cR1 Performance on Unity 650F
44 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1 Reference Architecture Guide
stress testing (Test 2) and before stress testing (Test 3) compared with the benchmarking
test (Test 1).
Database response time results
The following figure shows database performance (response time) as measured in Tests
1, 2, and 3. As shown, the database maintained the same response level during the
creation of snapshots during stress testing (Test 2) and before the stress testing (Test 3)
when compared with the baseline number (Test 1). Creating snapshots and doing stress
testing did not significantly impact the database response time and had no impact on the
latency performance of Oracle Database 18cR1 running on the Unity 650F storage array.
Figure 30. Database response time during Test 1, Test 2, and Test 3
The following figure compares the 12cR2 and 18cR1 database response times during our
testing.
Chapter 4: Oracle Database 18cR1 Performance on Unity 650F
45 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1
Reference Architecture Guide
Figure 31. Comparative analysis of database response time during Test 1, Test 2, and Test 3
As shown in Figure 31, the database response time during testing of the 12c and 18c
databases remained nearly the same despite snapshots being created during stress
testing (Test 2) and before stress testing (Test 3) compared with the benchmarking test
(Test 1).
Transactions per second results
The following figure shows the transaction throughput that was achieved in Tests 1, 2, and 3
during the testing of the Oracle Database 18c. The figure shows a minimal drop in TPS of less
than 1 percent in Test 2 and Test 3 compared with the baseline number from Test 1.
Figure 32. TPS during Test 1, Test 2, and Test 3
Chapter 4: Oracle Database 18cR1 Performance on Unity 650F
46 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1 Reference Architecture Guide
The following figure compares the 12c and 18c database TPS results.
Figure 33. Comparative analysis of TPS during Test 1, Test 2, and Test 3
A comparison of the 12c and 18c database test results shows a minimal reduction of 2
percent TPS from the creation of two snapshots. Thus, we can conclude that Oracle 12c
and 18c databases running on this reference architecture maintain stable transaction
throughput.
The performance metrics from Tests 1, 2, and 3 show that there was minimal impact from
creating snapshots and applying stress testing, and there was no performance impact
when running an Oracle Database 18cR1 on the Unity 650F storage array. This capability
of minimizing impact on the server performance is helpful when you have to create
multiple copies of the production database for nonproduction purposes.
Test methodology
To further determine the storage capacity savings that is realized through the use of the
deduplication and compression feature of the Unity 650F storage array during a data
change, we increased the data that was loaded during Use Case 1 by 5 percent and
captured the savings in storage capacity. The following diagram shows the comparison of
storage capacity before and after the 5 percent data increase.
Use case 4:
Deduplication
and compression
of Oracle
Database 18cR1
with data changes
Chapter 4: Oracle Database 18cR1 Performance on Unity 650F
47 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1
Reference Architecture Guide
Figure 34. Data reduction achieved in 18c database after 5% data insertion
The figure above shows that Unity’s data reduction capabilities yield a space savings of
25.97 percent inside the Unity storage array after adding five percent new data. This
space savings helps the customer to consolidate their data which reduces the cost of
storage and the total cost of ownership. (TCO).
Chapter 4: Oracle Database 18cR1 Performance on Unity 650F
48 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1 Reference Architecture Guide
Test Results
We performed the testing in Use Case 4 to find the data reduction percentage after
inserting data in six new SLOB schemas with 100% randomized and unique data
generated by a PL/SQL program. After loading the data in six new schemas, we achieved
a space savings of 25.97 percent, as shown in Figure 34. We achieved similar savings by
running the same tests on an Oracle Database 12c.
The following figure compares the data compression rates that were achieved during our
testing with the 12cR2 and 18cR1 databases.
Figure 35. Comparative analysis of data reduction (%) for Use Case 4 between 12c and 18c databases
Figure 35 shows that the data reduction percentage between Oracle Database 12cR2 and
18cR1 on the Unity 650F storage array is quite similar.
Conclusion
The tests described in Chapters 3 and 4 prove that Dell EMC Ready Solutions for Oracle
designed using the Unity 650F storage array and other Dell EMC hardware for networking
and servers creates a reliable reference architecture that will support the upgrade from
Oracle Database 12cR2 to Oracle Database 18cR1. Snapshot creation results in very little
impact to the performance of these databases. This solution exhibits a simplified upgrade
from 12c R2 to 18cR1. Customers can enjoy the many advantages of Dell EMC product
portfolios along with the advanced features of the Oracle Database 18cR1.
Chapter 5: Summary
49 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1
Reference Architecture Guide
Chapter 5 Summary
This chapter presents the following topics:
Summary ........................................................................................................... 50
Chapter 5: Summary
50 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1 Reference Architecture Guide
Summary
This Ready Solutions for Oracle Design for Unity All-Flash Unified Storage provides
simple, easy-to-use management by allowing storage administrators to provision storage
with very little setup or planning. This architecture provides predictable and consistent
performance and data savings for every environment in a typical enterprise application.
This validated and tested reference architecture enables the upgrading of Oracle
Database 12cR2 to 18cR1 without performance degradation, providing faster time-to-
value, operational agility, and efficiency.
An examination of AWR reports of the 18cR1 database and the 12cR2 database from
which it was upgraded reveals that there was minimal performance impact after adding
snapshots and running stress testing. Further, the deduplication and compression
features of the Unity 650 All Flash storage array provided significant storage capacity
savings and reduced TCO.
In this reference architecture guide, we have demonstrated the ease of upgrading the
Oracle Database 12cR2 to the Oracle Database 18cR1. The IOPS numbers exceeded
100,000 in all the use cases, with better results in the testing of the 18c database. We
noted a consistent pattern in database server CPU utilization, which ranged between 26
and 28 percent for all the use cases. Database response time was consistently at
approximately 0.31 ms. TPS and bandwidth were improved in the 18cR1 database test
results as compared to the 12cR2 database test results. Finally, running either Oracle
Database 12cR2 or 18cR1 on the Unity 650F array provided notable data compression
rates.
If customers elect to upgrade their Oracle Database 12cR2 to Oracle Database 18cR1 on
Dell EMC products like the PowerEdge Server R840, Unity 650F All-Flash Storage, and
Dell EMC networking products, there will be no adverse impact on database performance,
and performance may actually improve. In summary, here are the key results from our
testing of this Ready Solutions for Oracle:
Support for streamlined upgrades from Oracle Database 12cR2 to 18cR1 with no
performance degradation, based on performance metrics that were extracted from
the Oracle AWR report including:
Total IOPS
TPS
Database server CPU utilization
Database bandwidth
Database response time
Minimal or no performance degradation in 12cR2 and 18cR1 databases when
database snapshots were added during SLOB stress testing or before SLOB stress
testing. Thus, multiple copies of the production database can be created without
significant impact, even while the production database is performing its OLTP tasks.
Data compression rates of more than 25 percent after the insertion of unique data
into both 12cR2 and 18cR1 databases. Favorable data compression results were
Chapter 5: Summary
51 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1
Reference Architecture Guide
also observed when 5 percent new data was added to the existing data of both
12cR2and 18cR1 databases.
Chapter 6: References
52 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1 Reference Architecture Guide
Chapter 6 References
This chapter presents the following topics:
Dell EMC documentation .................................................................................. 53
VMware documentation .................................................................................... 53
Oracle documentation ...................................................................................... 53
SLOB documentation ....................................................................................... 53
Chapter 6: References
53 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1
Reference Architecture Guide
Dell EMC documentation
The following Dell EMC documentation provides additional and relevant information.
Access to these documents depends on your login credentials. If you do not have access
to a document, contact your Dell EMC representative.
Dell EMC Unity 650F All-Flash Storage
Dell EMC Unity Storage with Oracle Databases
How to deploy Oracle 12c Release 2 Standalone Database on RHEL 7.x
Dell EMC Ready Solutions for Oracle
For additional information about Dell EMC Ready Solutions for Oracle, see the Oracle Info
Hub for Ready Solutions on the Dell EMC Community Network.
VMware documentation
The following VMware documentation provides additional and relevant information:
VMware vSphere 6.5, ESXi 6.5, vCenter Server 6.5 Installation and setup
Oracle Databases on VMware Best Practices Guide
Dell EMC Host Connectivity Guide for VMware ESX Server
Oracle documentation
The following Oracle documentation provides additional and relevant information:
Installing Oracle 18c Standalone Database
Upgrading Oracle Grid Infrastructure from Oracle 12c to Oracle 18c
SLOB documentation
The following SLOB documentation provides additional and relevant information:
SLOB Resources
(Blog) EMC Unity Storage Performance testing with Oracle ASM and SLOB
Appendix A: Test Tools, and Database and SLOB Configuration
54 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1 Reference Architecture Guide
Appendix A Test Tools, and Database and SLOB Configuration
This appendix presents the following topics:
Testing and performance collection tools ....................................................... 55
SLOB dataset customization ............................................................................ 55
Database configuration .................................................................................... 56
SLOB parameter settings (slob.conf) .............................................................. 58
Appendix A: Test Tools, and Database and SLOB Configuration
55 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1
Reference Architecture Guide
Testing and performance collection tools
To simulate Oracle database workloads, we used SLOB version 2.4, which is a simple Oracle
database schema generation and I/O testing toolkit. We used the SLOB multiple schema model
to generate a 1.2 TB dataset (128 schemas x 9,600 MB each). We simulated an OLTP workload
by configuring SLOB to generate a distribution of 80 percent reads and 20 percent write I/Os.
For details on the SLOB parameter settings, see SLOB parameter settings (slob.conf).
The default schema that is generated by the SLOB tool contains a lot of redundant and
repetitive data and, therefore, is highly compressible. To get more realistic results,
especially for the deduplication and compression study, we modified the original SLOB
dataset and replaced it with a unique dataset using a customized PL/SQL script that uses
Oracle’s dbms_random.string function. For details on the custom script, see SLOB
dataset customization.
We collected the database performance data through the Oracle AWR report. During each
test, we configured AWR to capture data every 3 minutes. From the report, we extracted read,
write, and total IOPS; average I/O latency (in ms); and CPU utilization percentage metrics.
SLOB dataset customization
We replaced the default dataset that was loaded by SLOB with a custom and unique
dataset using the following PL/SQL script:
/* Truncate/remove original rows in tables for all users/schemas
(128 schemas in our case) */
for i in {1..128}
do
sqlplus / as sysdba << EOF
truncate table user${i}.cf1;
exit;
EOF
done
/* Start code for inserting random/unique data */
for i in {1..128}
do
sqlplus / as sysdba <<EOF
declare
type cf_table is table of user1.cf1%rowtype
index by binary_integer;
cfa cf_table;
begin
for i in 1..2450000 loop -- Loop through each row to generate
random and unique data for each column
cfa(i).custid:=i;
cfa(i).c2:=dbms_random.string('p',128);
cfa(i).c3:=dbms_random.string('p',128);
SLOB workload
testing toolkit
Oracle Automatic
Workload
Repository report
Appendix A: Test Tools, and Database and SLOB Configuration
56 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1 Reference Architecture Guide
cfa(i).c4:=dbms_random.string('p',128);
cfa(i).c5:=dbms_random.string('p',128);
cfa(i).c6:=dbms_random.string('p',128);
cfa(i).c7:=dbms_random.string('p',128);
cfa(i).c8:=dbms_random.string('p',128);
cfa(i).c9:=dbms_random.string('p',128);
cfa(i).c10:=dbms_random.string('p',128);
cfa(i).c11:=dbms_random.string('p',128);
cfa(i).c12:=dbms_random.string('p',128);
cfa(i).c13:=dbms_random.string('p',128);
cfa(i).c14:=dbms_random.string('p',128);
cfa(i).c15:=dbms_random.string('p',128);
cfa(i).c16:=dbms_random.string('p',128);
cfa(i).c17:=dbms_random.string('p',128);
cfa(i).c18:=dbms_random.string('p',128);
cfa(i).c19:=dbms_random.string('p',128);
cfa(i).c20:=dbms_random.string('p',128);
end loop;
for all i in 1..cfa.count
insert into user${i}.cf1 values cfa(i); -- Insert the above newly
generated random+unique data into each column of each row
end;
/
exit;
EOF
done
Database parameter configuration
The following table provides the database configuration and parameter settings used to
conduct all Oracle Database 12cR2 tests:
Table 8. Oracle 12cR2 standalone database configuration
Category Specification/setting Configuration
Operating system VM guest OS RHEL 7.4
Grid configuration Grid version 12c R2 Grid Infrastructure
Database configuration Database version 12c R2 Standalone Database
Database size 1.2 TB
Database parameter configuration
db_block_size 8 KB
db_file_multiblock_read_count 4
sga_max_size 96 GB
pga_aggregate_target 25 GB
12cR2
configuration
Appendix A: Test Tools, and Database and SLOB Configuration
57 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1
Reference Architecture Guide
The following table provides the database configuration and parameter settings used to
conduct all Oracle Database 18cR1 tests:
Table 9. Oracle 18cR1 standalone database configuration
Category Specification/setting Configuration
Operating system VM guest OS RHEL 7.4
Grid configuration Grid version 18c R1 Grid Infrastructure
Database configuration Database version 18c R1 Standalone
Database
Database size 1.2 TB
Database parameter settings
db_block_size 8 KB
db_file_multiblock_read_count 4
sga_max_size 96 GB
pga_aggregate_target 25 GB
Note: 18cR1 Grid and Standalone Database were upgraded from existing 12cR2 Grid and
Database, respectively.
18cR1
configuration
Appendix A: Test Tools, and Database and SLOB Configuration
58 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1 Reference Architecture Guide
SLOB parameter settings (slob.conf)
The following table provides the slob parameter settings we used to conduct all Oracle
Database 12cR2 and 18cR1 tests:
Table 10. Slob parameter settings
Parameter Setting
UPDATE_PCT 20
SCAN_PCT 0
RUN_TIME 1800
WORK_LOOP 0
SCALE 9600M
SCAN_TABLE_SZ 1M
WORK_UNIT 6
REDO_STRESS LITE
LOAD_PARALLEL_DEGREE 1
THREADS_PER_SCHEMA 1
Appendix B: Equipment List
59 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1
Reference Architecture Guide
Appendix B Equipment List
This appendix presents the following topics:
Hardware components...................................................................................... 60
Software components ....................................................................................... 62
Appendix B: Equipment List
60 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1 Reference Architecture Guide
Hardware components
The reference architecture includes the following primary hardware components:
1 x Dell EMC 4S PowerEdge R840 server
2 x Dell EMC Networking 10 GbE switches
2 x Dell EMC Connectrix 16 Gb/s FC switches
1 x Dell EMC Networking 1 GbE switch
1 x Dell EMC 2S PowerEdge R640 server
Unity 650F storage array
The following tables list the details of the hardware, firmware, and driver components of
the ESXi host server that we used in the tested configuration of this reference
architecture.
Note: Newer and updated BIOS and firmware versions are supported, if available. For the latest
version, go to Dell EMC Online Support.
Table 11. ESXi host server components
Component Description
Server 1 x Dell EMC PowerEdge R840
Chassis2 24 x 2.5 in. chassis with up to 12 SAS/SATA bays + 12 NVME/SAS/SATA bays
Processor 4 x Intel Xeon Gold 6154 18c 3.0 GHz
Memory 1,536 GB (24 x 64 GB QR DDR4 2,666 MT/s LRDIMMs)
Local disks in server 3 x 1.2 TB 10 K SAS 12 Gb/s 2.5 in. HDDs (includes one hot spare)
RAID controller PERC H740P adapter
iDRAC iDRAC9 Enterprise
rNDC or LOM Broadcom 5720 2P 1 Gb Base-T + 2P 10G SFP rNDC
Add-on NICs 2 x Intel 10 GbE 2P X710 adapter
HBAs 2 x QLogic QLE2692 DP 16 Gb/s FC HBAs
Power supplies 2 x 2,000 W
2 Other chassis configurations are supported.
ESXi host server
Appendix B: Equipment List
61 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1
Reference Architecture Guide
Table 12. PowerEdge R840 component firmware and drivers
Component Firmware Driver3
BIOS 1.1.1 N/A
Lifecycle Controller and iDRAC9 Enterprise
3.21.21.21 N/A
QLogic QLE2692 16 Gb/s FC HBAs
09.00.06 2.1.50.0 (qlnativefc)
Intel X710 DP 10 GbE SFP+ adapters
20.06.04.03; 20.6.34.0-1OEM.650.0.0.4598673
2.0.6 (i40en)
Broadcom 5720 2P 1 Gb Base-T + BCM57412 2P 10G SFP rNDC
20.6.16 (BCM5720); 20.06.04.03 (BCM57412)
20.6.34.0 (bnxtnet – 10 GbE)
Dell EMC PERC H740P adapter
50.0.3-0962 7.700.50.00 (lsi-mr3)
Delta 2,000 W power supplies
00.1C.53 N/A
The following table lists the hardware details of the storage array that we used in the
tested configuration of this reference architecture.
Table 13. Unity 650F storage array details
Storage array component Details
Operating system version 4.4.0.1534750794
Number of disk processor enclosures (DPEs)
1
Number of storage processor (SP) controllers
2
Number and type of front-end FC ports 4 x 16 Gb/s SFP+ FC ports per SP
Size per SSD 400 GB
Usable capacity 48.6 TB
RAID type RAID 5
3 Dell EMC customized ISO image (Dell Version: A04, Build# 5310538) was used to deploy the
ESXi hypervisor. Inbox drivers in RHEL 7.4 were used within the guest operating system.
Storage array
Appendix B: Equipment List
62 Dell EMC Ready Solutions for Oracle: Design for Dell EMC Unity All Flash Unified Storage With Dell EMC PowerEdge R840 and R640, RHEL 7.4, ESXi 6.5, and Oracle Database 12cR2 and 18cR1 Reference Architecture Guide
In this reference architecture for Oracle, we recommend that you use a dedicated
management server to install VCSA, Oracle Enterprise Manager, Dell EMC OpenManage
Essentials, or other centralized management software applications.
The following table lists the management server and its components that were used in the
tested configuration of this reference architecture.
Table 14. Management server and components
Category Components
Server 1 x Dell EMC PowerEdge R640 server
Chassis 2.5 in. with up to 8 HDDs and 3 PCIe slots
Processor 2 x Intel Xeon Gold 6136 12c 3 GHz
Memory 192 GB (12 x 16 GB DR DDR4 2,667 MT/s RDIMMs)
Local disks 3 x 1.2 TB 10k rpm SAS 12 Gb/s 2.5 in. HDDs (includes 1 hot spare)
RAID controller Dell EMC PERC H740P/H730P
iDRAC Dell EMC iDRAC9 Enterprise
rNDC Broadcom 5720 2P 1 Gb Base-T + BCM57412 2P 10G SFP rNDC
Software components
The following table specifies the versions of the software components of this reference
architecture as deployed in the tested configuration.
Table 15. Software versions
Software Version
VMware ESXi 6.5 [Dell EMC customized ISO image (Dell Version: A04, Build# 5310538)]
VMware vCenter Server Appliance 6.5.0.5500 Build Number 5318154
RHEL (guest operating system) 7.4, kernel version 3.10.0-693.el7.x86_64
Management
server