using hypermirror of huawei oceanstor v3 converged …...using hypermirror of huawei oceanstor v3...

Using HyperMirror of HUAWEI OceanStor V3

Converged Storage Systems to Harden

Mission-Critical Oracle Database's Reliability

The fast growth of enterprises has created an ever-increasing amount of data. This massive amount of data is

playing an important role in enterprise development because it not only records daily enterprise operation, but also

provides necessary information for decision making. How to ensure data reliability then becomes a great challenge.

In addition, with the wide use of heterogeneous virtualization, a new technology different from traditional RAID is

needed to ensure reliability of third-party LUN takeover. The HyperMirror feature provided by HUAWEI OceanStor

V3 converged storage systems enables data to be stored in an OceanStor V3 converged storage system or in a

heterogeneous storage environment in a 1:1 mirroring manner in different disk domains, thereby continuously

protecting mission-critical data.

This document describes two typical scenarios: using HyperMirror to continuously protect mission-critical Oracle

Database; using HyperMirror to perform online migration of mission-critical Oracle Database.

Li Yong

Storage Solutions, IT, Huawei Enterprise BG

2015-03-17 V1.0

Huawei Technologies Co., Ltd.

mailto:[email protected]

Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 2

Using HyperMirror of HUAWEI OceanStor V3 Converged Storage Systems to Harden


Contents

1 Overview ......................................................................................................................................... 4

1.1 Introduction .................................................................................................................................................................. 4

1.2 Purpose ......................................................................................................................................................................... 4

1.3 Intended Audience ........................................................................................................................................................ 5

1.4 Business Scenario ......................................................................................................................................................... 5

1.5 Customer Benefits ........................................................................................................................................................ 5

1.6 Workload Model ........................................................................................................................................................... 6

2 Products and Technologies ......................................................................................................... 7

2.1 OceanStor V3 Converged Storage Systems .................................................................................................................. 7

2.1.1 HyperMirror ............................................................................................................................................................... 8

2.1.2 SmartVirtualization .................................................................................................................................................. 11

2.2 Oracle Database and Cluster ....................................................................................................................................... 11

2.2.1 Oracle RAC and ASM ............................................................................................................................................. 12

2.2.2 Oracle System Architecture ..................................................................................................................................... 14

2.2.3 Oracle Application Types ......................................................................................................................................... 16

2.2.4 SwingBench ............................................................................................................................................................. 17

3 Using HyperMirror to Ensure Mission-Critical Databases' Reliability ........................... 19

3.1 Business Scenario ....................................................................................................................................................... 19

3.2 Huawei Solution ......................................................................................................................................................... 20

3.2.1 Solution Architecture ............................................................................................................................................... 20

3.3 Solution Configuration ............................................................................................................................................... 20

3.4 Verification Procedure ................................................................................................................................................ 22

3.4.1 Preparing the Environment ...................................................................................................................................... 22

3.4.2 Testing Basic Performance ...................................................................................................................................... 23

3.4.3 Testing Local Reliability Protection Implemented by HyperMirror ........................................................................ 23

3.4.4 Testing Heterogeneous Reliability Protection Implemented by HyperMirror ......................................................... 26

4 Verification Results .................................................................................................................... 29

4.1 Reliability Protection .................................................................................................................................................. 29

4.2 Data Synchronization Speed ....................................................................................................................................... 30

5 Appendix ...................................................................................................................................... 31

5.1 Reference Documents ................................................................................................................................................. 31




5.2 Glossary ...................................................................................................................................................................... 31




1 Overview

1.1 Introduction

The fast growth of enterprises has created an ever-increasing amount of data. This massive

amount of data is playing an important role in enterprise development because it not only

records daily enterprise operation, but also provides necessary information for decision

making. However, enterprises face various storage risks such as storage system failure, disk

failure, and irrecoverable damage to data. How to ensure data reliability is a great challenge.

In addition, with the wide use of heterogeneous virtualization, a new technology different

from traditional RAID is needed to ensure reliability of third-party LUN takeover.

HyperMirror, a volume mirroring feature provided by HUAWEI OceanStor V3 converged

storage systems, can store multiple physical copies in different disk domains to continuously

protect mission-critical data. Furthermore, HyperMirror can be used with SmartVirtualization

to store multiple physical copies in an OceanStor V3 converged storage system and a storage

system from another vendor and/or of another system architecture, thereby improving data

reliability.

This document describes how to implement continuous data protection and online data

migration based on HyperMirror in the Oracle Database 12c scenario. This document includes

the following contents:

Overview of HyperMirror

Working principle of HyperMirror

Using HyperMirror to continuously protect mission-critical data of Oracle Database

Using HyperMirror to implement online migration of Oracle Database

Best practices for using HyperMirror in Oracle Database scenarios

1.2 Purpose

This document aims to verify the feasibility and performance indicators of using HyperMirror

of OceanStor V3 converged storage systems in Oracle Database scenarios. It also provides

reference about IT system solutions for Huawei partners and customers, in order to help them

improve data reliability of IT systems.




1.3 Intended Audience

This document is intended for Huawei employees as well as IT engineers from partners and

customers. It provides the usage and best practices of HyperMirror for IT partners, storage

architects, database architects, and IT system administrators who want to implement Oracle

database solutions based on HUAWEI OceanStor V3 converged storage systems.

It is assumed that the readers are familiar with the following products and technologies:

Oracle Database

Linux operating systems

Storage systems

1.4 Business Scenario

As a typical business scenario, databases play an important role in enterprise applications. To

continuously protect data and prevent applications and data from being affected by storage

space faults are the most urgent needs of customers. HyerMirror, a type of volume mirroring

technology, is developed to address such needs. HyperMirror enables two identical copies of

data to be stored in different storage spaces. If one storage space becomes unavailable, the

continuity of data and applications is not affected, improving business reliability. HyperMirror

is implemented in storage systems without the need to purchase and manage additional

hardware, thereby reducing the total cost of procurement.

Therefore, HyperMirror can easily address the following scenarios:

If mission-critical applications require robust business continuity, HyperMirror can be

used to enable the local storage system to continuously protect data, preventing a

physical fault in storage space from causing data loss.

If mission-critical applications require robust business continuity and reliability,

HyperMirror can be used with SmartVirtualization to continuously protect data in a

heterogeneous storage environment, preventing the failure of a storage system from

causing data loss.

If HyperMirror is used in a heterogeneous storage environment, SmartVirtualization must be used. For

details about the scenario, refer to the Using SmartVirtualization of OceanStor V3 Converged Storage

Systems in Oracle Database for Unified Management of Heterogeneous Storage Systems, Data

Migration, and Data Protection.

1.5 Customer Benefits

HyperMirror can be used to continuously protect customers' mission-critical databases based

on local mirroring, thereby improving data reliability. Furthermore, HyperMirror can be used

with SmartVirtualization to create mirror copies in different storage systems, without the need

to purchase and manage additional hardware, thereby reducing the total cost of procurement.




1.6 Workload Model

This document uses the mainstream OLTP test model SwingBench Order Entry 2.0 to test the

use of SmartMigration for application SLA improvement, application migration, and LUN

type adjustment. This model defines a type of online order business and simulates a scenario

where a large number of users are querying products, placing orders, processing orders, and

viewing orders online. Those operations are the most common ones in transaction systems. In

this workload model, there are two main performance indicators: transactions per minute

(TPM) and average transaction response time. The TPM indicates the number of transactions

processed per minute. A higher TPM indicates higher productivity. The average transaction

response time directly affects the speed of user operations. A shorter response time indicates

better user experience.

Order Entry 2.0 defines 10 tables, storing information about products, customers, orders,

warehouses, and login. During the workload tests, 50% of operations are SELECT, 30%

INSERT, 20% UPDATE, and no DELETE operations. From the perspective of I/O layer, the

workload model is the most typical OLTP workload model, where small data blocks are

accessed at random and the ratio between reads and writes is 6:4.




2 Products and Technologies

2.1 OceanStor V3 Converged Storage Systems

HUAWEI OceanStor V3 converged storage systems are next-generation unified storage

products designed for enterprise-class applications. Leveraging a storage operating system

oriented to cloud architecture, a powerful next-generation hardware platform, and a full range

of intelligent management software, OceanStor V3 converged storage systems deliver

industry-leading functionality, performance, efficiency, reliability, and ease of use. They

provide data storage for applications such as large-scale database OLTP/OLAP, file sharing,

and cloud computing, and can be used in industries ranging from government, finance,

telecommunications, energy, to media and entertainment (M&E). Meanwhile, OceanStor V3

converged storage systems can provide a wide range of efficient and flexible backup and

disaster recovery solutions to ensure business continuity and data security, delivering

excellent storage services.

For details about HUAWEI OceanStor V3 converged storage systems, click the following

link:

http://e.huawei.com/en/products/cloud-computing-dc/storage/unified-storage/mid-range

http://e.huawei.com/en/products/cloud-computing-dc/storage/unified-storage/mid-range




2.1.1 HyperMirror

HyperMirror is a volume mirroring function provided by OceanStor V3 converged storage

systems. HyperMirror creates two available mirror copies of a LUN. If one mirror copy is

unavailable, the host can still access the LUN and services running on the host are not

affected. After the faulty mirror copy is recovered, it automatically synchronizes data from the

mirror LUN to ensure data consistency and availability.

HyperMirror provides the following functions:

Creates mirror copies for a thick LUN using local or external third-party LUNs.

Creates mirror copies for an eDevLUN using local or external third-party LUNs.

Non-disruptively converts a thick LUN that contains data into a mirror LUN.

Non-disruptively creates, splits, and deletes mirror copies.

Supports initial full synchronization upon the creation of a mirror copy and incremental

synchronization after a mirror copy is split.

HyperMirror does not support thin LUNs.

Continuous Data Protection Implemented by HyperMirror

HyperMirror creates a mirror LUN and multiple mirror copies for a LUN (called original

LUN) to prevent the failure of a single LUN from causing service interruption or data loss,

thereby improving service reliability.

HyperMirror requires that mirror copies do not reside in the same disk domain as the original

LUN. If the disk domain where the original LUN resides is faulty, the disk domains where

mirror copies reside are not affected. The normal mirror copy takes over all services from the

original LUN, ensuring host service continuity and data integrity.

Figure 2-1 Continuous protection of a local LUN

HyperMirror implementation is divided into three phases: creating a mirror LUN, synchronization, and splitting.

HyperMirrorMirror LUN

Local LUN

Mirror copy A Mirror copy B

Oce

an

Sto

r V

3 C

on

ve

rge

d S

tora

ge

Syste

m




Creating a mirror LUN

A mirror LUN is created for the original LUN (which is a local or external LUN). The

mirror LUN inherits the storage space, basic properties, and services from the original

LUN. During the creation of the mirror LUN, host services are not interrupted, and local

mirror copy A is automatically generated. The original LUN is changed into the mirror

LUN, and storage space of the original LUN is handed over to mirror copy A. Then,

mirror copy B is created for the mirror LUN, and data is synchronized from mirror copy

A to mirror copy B. In doing so, the LUN with mirror copies A and B has the space

mirroring function.

I/O requests sent by a host to a mirror LUN are processed as follows:

− When a host sends a read request to a mirror LUN, the storage system reads data

from the mirror copies in round-robin mode through the mirror LUN. If one mirror

copy fails, host services are not affected.

− When a host sends a write request to a mirror LUN, the storage system writes data to

the mirror copies in dual-write mode.

Figure 2-2 Mirror LUN creation

Mirror

Mirror

copy

Mirror

copy

Synchronization

The following figure illustrates the synchronization principle. After a mirror copy

recovers from a fault or when data on a mirror copy is being made complete, incremental

data is synchronized from a complete mirror copy to that mirror copy, making data

between mirror copies identical.




Figure 2-3 Mirror LUN synchronization

Synchronization

Mirror LUN Mirror copy

Initial synchronization: Full copy is performed in the initial synchronization.


Incremental synchronization: Incremental copy is performed when a mirror copy recovers from a splited or interrupted state.

Dual Write After the Synchronization

Host

Write data d to replace data a.


When the host sends a write request, the same data is written to both the mirror LUN and mirror copy (dual write).

Data already stored

Data to be stored

Internal signal flow in the storage system

Host signal flow

Splitting

Splitting is implemented to isolate a mirror copy from its mirroring relationship with the

mirror LUN. After the mirror copy is isolated, the mirror LUN cannot perform mirroring

on the mirror copy. Subsequent data changes between the mirror copy and the mirror

LUN are recorded by the Data Change Log (DCL). When the mirroring relationship is

restored, incremental data synchronization is performed based on the DCL.

Figure 2-4 Mirror LUN splitting

Splitting a Pair

Mirror Copy After the Splitting

Mirror LUN

Mirror copy

Mirror copy

After a pair is split, data synchronization between the mirror LUN and mirror copy A is no longer implemented.

Host

Mirror copy

Mirror copy

Mirror LUN

After the split, data is no longer written to the mirror LUN and mirror copy A in dual-write mode. The data host of mirror copy A becomes unavailable, but mirror copy B can still work normally. Therefore, host services are not interrupted.

Subsequent data changes made between mirror copy A and the mirror LUN are recorded by the DCL for incremental data synchronization.

Data already stored

Internal signal flow in the storage system Host signal flow




2.1.2 SmartVirtualization

SmartVirtualization uses LUNs mapped from heterogeneous storage arrays to the local

storage array as logical disks (LDs) that can provide storage space for the local storage array

and creates eDevLUNs that can be mapped to the host on LDs. LDs provide data storage

space for data volumes, and the local storage array provides storage space for meta volumes

of eDevLUNs. SmartVirtualization ensures data integrity of external LUNs. eDevLUNs and

LUNs of heterogeneous storage arrays have different World Wide Names (WWNs).

The primary advantages of SmartVirtualization are as follows:

Wide compatibility

As storage arrays from different vendors have different understanding and compliance

for Small Computer System Interface (SCSI), incompatibility among storage arrays in

takeover becomes a big challenge. SmartVirtualization is enhanced for different storage

arrays in terms of interoperability and can identify and process incompatibility issues of

heterogeneous storage arrays. For example, it can identify and process LUN path faults

of heterogeneous storage arrays to ensure reliability and interoperability of

heterogeneous storage arrays. Heterogeneous storage arrays supported by

SmartVirtualization have been certified by Huawei Interoperability Lab.

Back-end multipathing

SmartVirtualization leverages back-end multipathing software to enable heterogeneous

storage arrays to be connected by redundant paths. In this way, if a physical path

between two heterogeneous storage arrays fails, services are not interrupted. As the

back-end multipathing software is enhanced for a heterogeneous storage array in terms

of interoperability, it can identify heterogeneous LUN paths, such as preferred paths,

non-preferred paths, or standby paths, and select the most appropriate path to deliver

I/Os. Meanwhile, it provides a range of path selection algorithms for back-end load

balancing.

Less storage space consumption

The eDevLUN is not a complete physical data mirror of the LUN in the third-party

storage system. Therefore, it occupies only a small amount of storage space.

Excellent function extension

Value-added features like remote replication and snapshot are available for the

eDevLUN, meeting higher data security and reliability requirements.

eDevLUNs and local LUNs have the same properties. Therefore, HyperMirror can be used to

provide mirror protection for heterogeneous LUNs. In addition, heterogeneous LUNs can be

used as mirror copies to improve service reliability.

For more information about the SmartMigration feature, refer to the following document:

HUAWEI OceanStor V3 Converged Storage Systems SmartVirtualization Technical White

Paper

2.2 Oracle Database and Cluster

Oracle Database is one of the most widely used relational databases. This section briefly

introduces Oracle Database 12c and focuses on Multitenant-related components and features,

including RAC, ASM, Multitenant, data files, database instance architecture, and application

types.




2.2.1 Oracle RAC and ASM

As shown in Figure 2-5, an Oracle 12c RAC contains two types of nodes: Hub nodes and Leaf

nodes. Hub nodes have direct access to shared storage, whereas Leaf nodes access shared

storage through Hub nodes. When a database is deployed on an Oracle RAC, the nodes can be

grouped into multiple server pools. Each database is deployed in a server pool, and every

node in a server pool runs a database instance. Application servers access the virtual IP

addresses (VIPs) of nodes to store data. If a node fails, its VIP network is restored on another

node of Oracle RAC. Application servers reconnect to the Oracle database through a

reconnection mechanism. Setting connection character strings on application servers can

enable multiple modes of accessing Oracle RAC nodes, including load balancing and failover

modes. In these modes, a multi-node Oracle cluster is presented as a single database to

application servers.

The shared storage of the Oracle RAC Hub nodes includes Oracle Cluster Registry (OCR),

voting disks, and database. OCR records information about node statuses, voting disks

synchronize data between nodes, and the database is a set of files.

Figure 2-5 Oracle Flex Cluster 12c

Oracle ASM provides a simple storage management interface for database administrators to

manage servers and storage across different platforms. As a built-in file system and volume

manager, Oracle ASM is exclusive to Oracle database files. ASM simplifies file system

management, provides asynchronous I/O performance tuning, saves management time for

administrators, and offers a flexible, efficient database environment.

ASM can consolidate LUNs into a disk group and use Allocation Units (AUs) to allocate

storage space from the disk group. ASM supports three types of disk groups.

External: Data is not mirrored between LUNs, and the storage system provides data

protection.

Normal: A normal disk group consists of two failure groups between which data is

mirrored.

High: A high disk group consists of three failure groups among which data is mirrored.

When an OceanStor V3 storage system is used to create ASM disk groups, it is recommended

that controllers A and B evenly own the LUNs in the disk groups before external or normal

disk groups are created.




Oracle Flex ASM is a new Oracle ASM deployment model that increases database instance

availability and reduces Oracle ASM related resource consumption. Oracle Flex ASM

facilitates cluster based database consolidation, as it ensures that Oracle Database 12c

instances running on a particular server will continue to operate, should the Oracle Flex ASM

instance on that server fail.

Figure 2-6 ASM before Oracle Database 12c

Figure 2-7 Oracle 12c Flex ASM

Oracle RAC provides the following key characteristics, essential for HA data management:

Reliability — Oracle Database is known for its reliability. Oracle RAC takes this step

further by removing the database server as a single point of failure. If an instance fails,

the remaining instances in the cluster remain open and active. Oracle Clusterware

monitors all Oracle processes and immediate restarts any failed component.

Error detection — Oracle Clusterware automatically monitors all Oracle RAC databases

as well as other Oracle processes (Oracle ASM, instances, Listeners, etc.) and provides

fast detection of problems in the environment. It also automatically recovers from

failures often before users notice that a failure has occurred.

Recoverability — The Oracle Database includes many features that make it easy to

recover from various types of failures. If an instance fails in an Oracle RAC database, it

is recognized by another instance in the cluster and recovery will start automatically. Fast

Application Notification (FAN) and Fast Connection Failover (FCF) and especially the




Oracle RAC 12c Application Continuity feature make it easy to mask any component

failure from the user.

Continuous Operations — Oracle RAC provides continuous service for both planned and

unplanned outages. If a server (or an instance) fails, the database remains open and

applications continue to be able to access data, allowing for business critical workloads

to finish, mostly without a delay in service delivery.

For more information about Oracle 12c RAC and ASM, refer to the following documents:

White Paper: Oracle Real Application Clusters (RAC)

A Technical Overview of New Features for Automatic Storage Management in Oracle

Database 12c

2.2.2 Oracle System Architecture

Figure 2-8 Oracle system architecture

As shown in the preceding figure, System Global Area (SGA) and Program Global Area

(PGA) of Oracle databases consume memory. SGA stores system information and page cache

information, and PGA stores session information. SGA contains the following parts:

Buffer Cache: buffers data blocks.

Redo Log Buffer: buffers log records as a recycle data group.

Share Pool: buffers data dictionaries and shared SQL information.

Oracle files are categorized as follows:

Control file: records the database structure, parameters, and locations of other data fails.

Data file: stores user data and temporary data.

http://www.oracle.com/technetwork/database/options/clustering/rac-wp-12c-1896129.pdf

http://www.oracle.com/technetwork/products/cloud-storage/oracle-12c-asm-overview-1965430.pdf

http://www.oracle.com/technetwork/products/cloud-storage/oracle-12c-asm-overview-1965430.pdf




Online log: record changes to data blocks and consists of several log groups. Files in the

log groups are mirrored to each other. After a log group is used up, data in written to the

next log group. After the first log group is used up, data is written to the first log group

again.

Archive log: In archive mode, databases copy fully written log groups to the archive area

for data restoration when anomalies occur.

Among Oracle processes, the Listener process monitors client connections. Clients are

connected in two modes:

In shared mode, the listener redirects client requests to the dispatcher process, which

places the request in the request queue of the large pool. Then the shared server obtains

and processes the request in the request queue and puts the processing results in the

response queue of the large pool. At last, the dispatcher returns the processing results to

the client.

In dedicated mode, a dedicated server process serves each client connection. After

receiving the request from a client, the Oracle server process looks for the data block in

the buffer cache. If the data block is found, data is read, computed, and changed in the

buffer cache. If the data block does not exist in the buffer cache, the Oracle server

process writes the data block from data files to the buffer cache, and then computes and

changes it.

Oracle uses the LRU algorithm to eliminate outdated data in the buffer cache so that the

released storage space can be used by new data blocks. Data that has been changed in the

buffer cache is called "dirty data", which is written by the DB writer (DBW) process to data

files.

To ensure data integrity and reliability, relational databases use "transaction" to indicate an

atomic operation. When processing a transaction, the Oracle server process records changed

data and the change time in the log buffer. When the transaction is delivered, the log writer

(LGWR) synchronizes the log data in the log buffer to online log files. The log buffer is a

memory area where data can be written in a circular manner. When the log buffer is one-third

full, the LGWR synchronizes the log records to the log file, regardless of whether the

transaction is committed. In addition, Oracle databases synchronize logs automatically every

three minutes.

By default, Oracle databases perform a checkpoint every 30 minutes. When the checkpoint is

performed, the Checkpoint (CKPT) process synchronizes the control file and triggers the

DBW to write dirty data to data files.

Online logs of Oracle are divided into several groups, each of which contains one or multiple

log files. When multiple log files exist, the files are mirrored to each other. Oracle databases

write logs to the log groups in sequence. When the last log group is filled, the databases write

to the first log group, restarting the cycle. Before a log group switch, Oracle checks whether

the dirty data recorded in the next log group is completely written to data files. If not, Oracle

waits until the DBW process writes all the dirty data to the data files before it starts the log

group switch.

When an Oracle database is in archive mode, the Archive (ARC) process copies filled logs to

the archive area. If data anomalies occur, the archived logs are used for precise data recovery.

Oracle Database 12c also supports a Multitenant that allows multiple PDBs to run in one

CDB. Figure 2-9 shows a CDB with four containers: the root, seed, and two PDBs (hrpdb and

salespdb). Each PDB has its own dedicated application. A different PDB administrator

manages each PDB. User SYS can manage the root and every PDB. At the physical level, this

CDB has a database instance and database files. The Multitenant feature brings better serviceability to the Oracle Database.




The seed PDB is a system-supplied template that the CDB can use to create new PDBs. The seed PDB is

named PDB$SEED. You cannot add or modify objects in PDB$SEED.

Figure 2-9 Oracle Multitenant architecture

For more information about Oracle Multitenant Architecture, refer to the following document:

White Paper: Oracle Multitenant

2.2.3 Oracle Application Types

Data transactions are categorized as two types: OLTP and OLAP.

OLTP: A number of online users perform transactions.

OLAP: A small number of users perform long-term complex statistical queries.

OLTP applications have the following I/O characteristics:

From the perspective of database

The reading, writing, and changing of each transaction involve a small amount of data.

Database data must be up-to-date. Therefore, OLTP applications require a high database

availability.

Many users connect to and use the database concurrently.

The database must be highly responsive and able to complete a transaction within

seconds.

http://www.oracle.com/technetwork/database/multitenant-wp-12c-1949736.pdf




From the perspective of storage

Every I/O is small-sized, ranging from 2 KB to 8 KB.

Disk data is randomly accessed.

At least 30% of data is generated by random writing operations.

Redo logs are written frequently.

2.2.4 SwingBench

In verification tests described in this document, SwingBench is used to stress test Oracle

Database. SwingBench is a free database benchmark tool provided by Oracle. The tool

provides four workload models, of which Order Entry is the most used one.

Order Entry is an online transactional workload model. This model defines a type of online

order business, simulating online product searching, placing of orders, managing of orders,

reviewing of orders, and other common system operations by a number of users. Order Entry

defines nine tables, storing information about products, customers, orders, warehouses, and

login. During the workload tests, 50% of operations are SELECT, 30% INSERT, 20%

UPDATE, and no DELETE operations. From the perspective of storage layer, Order Entry

represents the most typical OLTP model, where small data blocks are accessed at random and

the ratio between reads and writes is 6:4. This model uses two main performance indicators:

transactions per minute (TPM) and average transaction response time. The TPM indicates the

number of transactions processed per minute. A higher TPM indicates higher productivity.

The average transaction response time directly affects the speed of user operations. A shorter

response time indicates better user experience.

Loading Test Data

SwingBench provides two data loading modes for the Order Entry test: oewizard and

DataGenerator. In oewizard mode, all user data is stored in one tablespace, and table partitions

and indexes cannot be customized. This mode is suitable for testing less than 100 GB of data.

In DataGenerator mode, you can customize data loading scripts to distribute data optimally.

The scripts and configuration files used for loading data in the tests described in this

document can be found in the attachments.

Multi-client testing

Multiple SwingBench clients can jointly test an Oracle RAC. The coordinator coordinates

these clients.

Each SwingBench client can perform a test in the following two modes: invoking the

SwingBench program to perform tests in visual mode or invoking the CharBench program to

perform tests in CLI mode. The following table lists the key configuration items of the

swingconfig.xml file.

Table 2-1 Key configuration items of swingconfig.xml

Configuration Item Description

UserName/Password User name and password of a database

ConnectString Database connection string

NumberOfUsers Number of users

MinDelay/MaxDelay Interval between two transactions




3 Using HyperMirror to Ensure Mission-Critical Databases' Reliability

3.1 Business Scenario

As a typical business scenario, databases play an important role in enterprise applications. To

continuously protect data and prevent applications and data from being affected by storage

space faults are the most urgent needs of customers. HyerMirror, a type of volume mirroring

technology, is developed to address such needs. HyperMirror enables two identical copies of

data to be stored in different storage spaces. If one storage space becomes unavailable, the

continuity of data and applications is not affected, improving business reliability. HyperMirror

is implemented in storage systems without the need to purchase and manage additional

hardware, thereby reducing the total cost of procurement.

Therefore, HyperMirror can easily address the following scenarios:

If mission-critical applications require robust business continuity, HyperMirror can be

used to enable the local storage system to continuously protect data, preventing a

physical fault in storage space from causing data loss.

If mission-critical applications require robust business continuity and reliability,

HyperMirror can be used with SmartVirtualization to continuously protect data in a

heterogeneous storage environment, preventing the failure of a storage system from

causing data loss.




3.2 Huawei Solution

3.2.1 Solution Architecture

Figure 3-1 Solution architecture

As shown in the preceding figure, Oracle Database runs on an OceanStor 5500 V3 converged

storage system. HyperMirror uses different storage spaces provided by the OceanStor 5500

V3 converged storage system to continuously protect Oracle Database and improve data

reliability.

SmartVirtualization is used to enable the OceanStor 5500 V3 converged storage system to

take over the S5500T. HyperMirror protects Oracle Database and improves data reliability

across the two storage systems.

3.3 Solution Configuration

Table 3-1 Hardware configuration

Hardware Component Quantity

Server: RH2288 V2 2

Intel® Xeon

® E5620 @ 2.40 GHz 1 x 8

Memory 8 GB x 6

QLogic 8 Gbit/s dual-port Fibre Channel

HBA 1 x 2

Oracle 12c RAC

2 x RH2288 V2

OceanStor 5500 V3

50 x 600 GB 10k rpm SAS disks

4 x

8G

bit/s

FC

Cluster Private Interconnection

S6700 10GE switch

S5500T

24 x 600 GB 10k rpm SAS disks




Hardware Component Quantity

Storage system: OceanStor

5500 V300R001C10

1

600 GB 10k rpm SAS disk 50

Storage system: OceanStor

S5500T V200R002C20

1

600 GB 10k rpm SAS disk 24

Fibre Channel switch:

SNS2124

2

Table 3-2 Software configuration

Hardware Software Quantity

RH2288 V2 2

Operating system: Red Hat Enterprise Linux 6.5 1 x 2

Multipathing software: UltraPath for Linux

8.01.031

1 x 2

Database cluster: Oracle Grid 12.1.0.2 1 x 2

Database software: Oracle Database 12.1.0.2 1 x 2

Java operating environment: Oracle SUN JRE

7U64

1 x 2

Data loading tool: SwingBench 2.5 1 x 1

OceanStor 5500

V300R001C10

1

HyperMirror license 1 x 1

SmartVirtualization license 1 x 1

Table 3-3 Storage plan for the OceanStor 5500 V3 converged storage system

Function LUN Storage Pool Disk Domain

Data volume 7 x 300 GB StoragePool1 - RAID 6

(8D+2P)

DiskDomain1 (25 SAS

disks)

Data volume 6 x 500 GB StoragePool2 - RAID 6

(4D+2P)

DiskDomain2 (25 SAS

disks)

Online log volume 1 x 300 GB StoragePool1 - RAID 6

(8D+2P)

DiskDomain1 (25 SAS

disks)





Archive log volume 2 x 300 GB StoragePool1- RAID 6

(8D+2P)

DiskDomain1 (25 SAS

disks)

Cluster voting volume 3 x 5 GB GridPool - RAID6

(4D+2P)

DiskDomain1 (25 SAS

disks)

Create two disk domains, each of which consists of 25 x 600 GB SAS disks. In DiskDomain1, create

GridPool and StoragePool1. GridPool is used to create cluster voting volumes for Oracle RAC.

StoragePool1 is used to create data volumes, online log volumes, and archive log volumes related to the

database. In DiskDomain2, create StoragePool2 that is used to create data volumes related to the

database. The remaining space is used to create mirror copies for data volumes in the storage pools.

Table 3-4 S5500T storage plan


Data Volume 7 x 300 GB StoragePool1 - RAID 6

(8D+2P)

DiskDomain1 (24 SAS

disks)

3.4 Verification Procedure

3.4.1 Preparing the Environment

Step 1 Prepare a physical environment.

Deploy a test environment based on the solution architecture shown in Figure 3-1.

Step 2 Install and configure the operating system on each host.

Install Red Hat Enterprise Linux 6.5 x86_64 on each host and configure the operating system

based on Oracle Database's requirements.

Step 3 Prepare storage devices.

Based on the plan, configure a SAN switch, create an alias, and set zones.

Based on the plan, configure storage, create host groups and hosts, and map LUNs.

Step 4 Install UltraPath.

Upload the UltraPath software package, which is used with the OceanStor 5500 V3 converged

storage system, to database nodes. Then, install UltraPath.

Step 5 Configure a UDEV policy.

On database nodes, configure a UDEV policy for LUNs.

Step 6 Install Oracle Database.

Deploy an Oracle cluster and install Oracle Database in the OceanStor 5500 V3 storage

system. Then, use SwingBench to generate 1 TB of test data.




----End

3.4.2 Testing Basic Performance

In the Oracle environment deployed based on the OceanStor 5500 V3 storage system, use

SwingBench to perform a benchmark workload test (namely, the database I/O latency is about

10 ms). Perform the test for 30 minutes. Record the TPM, IOPS, and latency as reference

values of basic performance. Based on the reference values, assess the impact of HyperMirror

on Oracle Database.

3.4.3 Testing Local Reliability Protection Implemented by HyperMirror

Step 1 Creating a HyperMirror task.

1. Open a web browser and enter https://management IP address:8088.

The login page of DeviceManager of the OceanStor 5500 V3 storage system is

displayed.

2. Enter the user name and password.

3. Select the Data Protection tab page.

4. Click HyperMirror.

5. Create a HyperMirror task, start initial synchronization, and record the synchronization

time.

If a HyperMirror task is created for a LUN (called mirror LUN in the HyperMirror feature),

mirror copy A is generated in the pool where the mirror LUN resides.

During the creation of a HyperMirror task, mirror copy B must be generated for the mirror

LUN too. In the local mirroring scenario, mirror copy B and the mirror LUN do not reside in

the same disk domain, mirror copy B is a LUN automatically created by the storage system,

and data is automatically synchronized from mirror copy A to mirror copy B.

After a HyperMirror task is created, a mirror volume contains three LUNs: a mirror LUN,

mirror copy A, and mirror copy B. The two mirror copies reside in two disk domains. The

mirror LUN is mapped to the host as a data LUN that stores data generated by Oracle

Database.




Wait for the initial synchronization to complete.

The synchronization speed can be low, medium, high, or highest. After creating a HyperMirror task, you

can change the speed in the properties of a mirror copy.

One HyperMirror task can contain only two mirror copies.

The two databases are running properly. As shown in the following figure, database

HWPDB1 is protected by HyperMirror, and HWPDB2 is not.

The following figure shows the HyperMirror configuration.




Step 2 Simulate a fault.

After the synchronization is complete, remove multiple disks to simulate a disk failure.

Removing multiple disks will destroy the RAID property of the storage pool. Then, LUNs in

the storage pool will stop providing services.

Step 3 Check the database service.

All the LUNs in StoragePool2 are faulty. All the LUNs that serve HWPDB2 are faulty. Mirror

copy B of the data LUN that serves HWPDB1 is faulty, but mirror copy A is still available.

HWPDB1 protected by HyperMirror is not affected, ensuring business continuity. HWPDB2

that is not protected by HyperMirror cannot be opened.




Step 4 Reinsert disks.

After the removed disks are inserted, the disk failure is resolved, and the damaged RAID

property is also recovered. Then, LUNs in the storage pool become available and can provide

service again.

----End

3.4.4 Testing Heterogeneous Reliability Protection Implemented by HyperMirror

Step 1 Set up a relationship between the remote storage system and the local storage system.

In the remote storage system, create a host and identify the local storage system as a physical

host.

Step 2 In the local storage system, discover the remote storage system.

The local storage system automatically discovers the remote storage system.

Step 3 In the remote storage system, create LUNs that have the same size as the mirror LUN, and

map the LUNs to the local storage system. Then, create eDevLUNs that reside in a different

disk domain from the mirror LUN.




Step 4 Create a HyperMirror task.

Create a HyperMirror task with LUN Type set to eDevLUN, start initial synchronization, and

record the synchronization time.

Step 5 Simulate a fault.

Remove multiple disks that belong to the disk domain where the mirror LUN resides to

simulate a LUN failure.

Step 6 Check the database service.

Heterogeneous LUN mirroring protection is provided for the database. Therefore, the local

LUN failure does not cause all mirror copies to fail. The database still runs properly.




----End




4 Verification Results

4.1 Reliability Protection

The verification described in section 3.4.3 "Testing Local Reliability Protection Implemented

by HyperMirror" and section 3.4.4 "Testing Heterogeneous Reliability Protection

Implemented by HyperMirror" proves that HyperMirror can protect mirror volumes of Oracle

Database. If a volume fails as a result of damage to storage media, the database can still work

properly.

The mirror copies in a HyperMirror task reside in different disk domains. Therefore, before

storage controllers become a performance bottleneck, the use of HyperMirror does not affect

performance. The splitting of a mirror copy from a HyperMirror task has only a negligible

impact on performance. The test result shown in the following figure proves the preceding

conclusion.

Figure 4-1 Performance impact of HyperMirror

The preceding figure shows the maximum IOPS of the storage system in the scenario where the I/O

latency of the OLTP database is less than 10 ms. In the preceding figure, three cases are compared:

without using HyperMirror, using a HyperMirror task in normal state, and using a HyperMirror task in

split state.

HyperMirror not

used

HyperMirror in

normal state

HyperMirror in

split state

Max. IOPS




If the mirror copies of a HyperMirror task are normal, the system performance does not change. If a

mirror copy of a HyperMirror task is split, the system performance decreases by 10%.

4.2 Data Synchronization Speed

HyperMirror provides four synchronization speeds. The following table lists the

synchronization time and average bandwidth that each speed option can provide for a

HyperMirror task created for a 300 GB LUN when there is no workload. In actual use, select

a synchronization speed based on the service impact and migration time that you can accept.

Table 4-1 Synchronization time and average bandwidth provided by each speed option

Speed Synchronization Time (Min) Average Bandwidth (MB/s)

Low 2366 2

Medium 340 15

High 108 48

Highest 36 142




5 Appendix

5.1 Reference Documents HUAWEI OceanStor V3 Converged Storage Systems HyperMirror Technical White

Paper

HUAWEI OceanStor V3 Converged Storage Systems SmartVirtualization Technical

White Paper

5.2 Glossary

Table 5-1 Glossary

Term Description

OLTP Online Transaction Processing

OLAP Online Analytical Processing

TPM Transactions Per Minute

UDEV Device manger of Linux kernel 2.6

RAC Oracle Real Application Clusters

ASM Oracle Automatic Storage Management

UltraPath Huawei storage multipathing management software




Huawei Technologies Co., Ltd.

Address: Huawei Industrial Base

Bantian, Longgang

Shenzhen 518129

People's Republic of China

Website: http://www.huawei.com

Email: [email protected]

Copyright © Huawei Technologies Co., Ltd. 2015. All rights reserved.

No part of this document may be reproduced or transmitted in any form or by any means without prior written

consent of Huawei Technologies Co., Ltd.

Trademarks and Permissions

and other Huawei trademarks are trademarks of Huawei Technologies Co., Ltd.

All other trademarks and trade names mentioned in this document are the property of their respective holders.

Notice

The purchased products, services and features are stipulated by the contract made between Huawei and the

customer. All or part of the products, services and features described in this document may not be within the

purchase scope or the usage scope. Unless otherwise specified in the contract, all statements, information,

and recommendations in this document are provided "AS IS" without warranties, guarantees or

representations of any kind, either express or implied.

The information in this document is subject to change without notice. Every effort has been made in the

preparation of this document to ensure accuracy of the contents, but all statements, information, and

recommendations in this document do not constitute a warranty of any kind, express or implied.

http://www.huawei.com/

mailto:[email protected]