federation enterprise hybrid cloud 3.1 - data protection ... · pdf filefederation enterprise...

This solution guide describes the Continuous Availability Modular add-on to the

Federation Enterprise Hybrid Cloud Foundation solution for SAP. This guide

introduces the architecture and features that ensure continuous availability of

SAP services and operations in the cloud.

October 2015

2

Copyright © 2015 EMC Corporation. All rights reserved. Published in the USA.

Published October 2015

EMC believes the information in this publication is accurate as of its publication date. The

information is subject to change without notice.

THE INFORMATION IN THIS PUBLICATION IS PROVIDED AS IS. EMC CORPORATION MAKES

NO REPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE

INFORMATION IN THIS PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED

WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Use,

copying, and distribution of any EMC software described in this publication requires an

applicable software license.

EMC2, EMC, VMAX, VNX, XtremIO, VPLEX, ViPR, PowerPath, Data Domain, Avamar, and the

EMC logo are registered trademarks or trademarks of EMC Corporation in the United States

and other countries. All other trademarks used herein are the property of their respective

owners.

For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on

EMC.com.

Federation Enterprise Hybrid Cloud 3.1 - Data Protection for SAP: Continuous

Availability Solution Guide

Part Number H14266

http://www.emc.com/legal/emc-corporation-trademarks.htm

Contents

3

Chapter 1 Executive Summary ............................................................. 5

Document purpose .............................................................................................. 6

Audience ............................................................................................................ 6

Essential reading ................................................................................................ 6

Solution purpose ................................................................................................. 6

Business challenge .............................................................................................. 7

Technology solution............................................................................................. 7

Terminology ....................................................................................................... 7

Chapter 2 Continuous Availability Solution Overview ............................... 9

Overview .......................................................................................................... 10

Solution architecture .......................................................................................... 10

Logical architecture ........................................................................................ 10

Physical architecture with VPLEX host Cross-Connect ......................................... 12

Software resources ............................................................................................ 12

Chapter 3 Design Considerations ........................................................ 14

Overview .......................................................................................................... 15

SAP system architecture ..................................................................................... 15

SAP instances outage impact ............................................................................... 16

SAP database instance outage ......................................................................... 16

SAP ASCS instance outage (enqueue and message server services) ..................... 16

SAP Global File Systems ................................................................................. 16

Considerations and best practices ........................................................................ 17

Auto start of SAP instances ............................................................................. 17

vSphere HA and DRS ..................................................................................... 18

Site affinity for virtual machines ...................................................................... 19

Chapter 4 Protecting SAP Applications with Continuous Availability ......... 21

Overview .......................................................................................................... 22

Implementing Continuous Availability for SAP applications ...................................... 22

Use case 1: Composing SAP provisioning service on CA-enabled infrastructure ...... 22

Use case 2: Setting costs for CA-enabled storage .............................................. 25

Validation.......................................................................................................... 26

Use case 3: Single ESXi host failure ................................................................. 28

Use case 4: One leg of VPLEX cluster failure ..................................................... 29

Use case 5: Single data center failure .............................................................. 30

Use case 6: Planned host maintenance ............................................................. 31

Use case 7: Compute resource balance ............................................................ 33

Chapter 5 Conclusion ........................................................................ 34

Conclusion ........................................................................................................ 35

Contents

4

Appendix A References ........................................................................ 36

References ........................................................................................................ 37

EMC Solution Guides ...................................................................................... 37

Product documentation .................................................................................. 37

Other documentation ..................................................................................... 37

Figures

Figure 1. Logical architecture ......................................................................... 11

Figure 2. Physical architecture ....................................................................... 12

Figure 3. SAP system architecture .................................................................. 15

Figure 4. Dependency among SAP instances .................................................... 17

Figure 5. Enable auto start of SAP instances during provisioning ........................ 18

Figure 6. Deploying SAP applications with site affinity ....................................... 20

Figure 7. Sample view of Site Affinity DRS Rule configuration ............................ 20

Figure 8. Adding a cloud template from vRealize Automation blueprints .............. 23

Figure 9. Adding cloud template mapping in the logical template ....................... 23

Figure 10. Creating new deployment profiles to use the infrastructure enabled for

continuous availability ..................................................................... 24

Figure 11. Mapping cloud templates enabled for continuous availability ................ 25

Figure 12. Creating a virtual machine storage profile .......................................... 26

Figure 13. Setting the cost for CA-enabled storage............................................. 26

Figure 14. VPLEX status after data center A VPLEX isolation ................................ 29

Figure 15. PowerPath status after VPLEX isolation .............................................. 30

Figure 16. No interruption of SAP CA2 after VPLEX cluster-2 failure on data center

B .................................................................................................. 30

Figure 17. Non-disruptive host maintenance ...................................................... 32

Tables

Terminology .................................................................................... 7 Table 1.

Solution software resources ............................................................. 12 Table 2.

Impact of SAP instances outages ...................................................... 16 Table 3.

VM restart priority for SAP instances ................................................. 18 Table 4.

List of ESXi hosts ............................................................................ 27 Table 5.

CA1 SAP system details ................................................................... 27 Table 6.

CA2 SAP system details ................................................................... 27 Table 7.

List of vSphere DRS rules ................................................................ 27 Table 8.

Use case 1: Reference metrics for ESXi failure .................................... 28 Table 9.

Use case 5: Reference metrics for single data center failure ................. 31 Table 10.

ESXi host resource balancing ........................................................... 33 Table 11.

Chapter 1: Executive Summary

5

This chapter presents the following topics:

Document purpose .............................................................................................. 6

Audience ............................................................................................................ 6

Essential reading ................................................................................................ 6

Solution purpose ................................................................................................. 6

Business challenge .............................................................................................. 7

Technology solution............................................................................................. 7

Terminology ....................................................................................................... 7


6

This solution guide describes a Continuous Availability (CA) add-on solution for the

Federation Enterprise Hybrid Cloud for SAP. This document:

Introduces the technical components needed to implement and operate the solution

Describes EMC’s testing and validation of the solution’s functionality

Evaluates the technical and business value of the solution in the context of a

Federation Enterprise Hybrid Cloud for SAP

This document is a companion to Federation Enterprise Hybrid Cloud 3.1: Foundation for

SAP Solution Guide, which provides critical reference information for planning and designing

a data protection solution for the hybrid cloud.

This solution guide is intended for executives, managers, SAP architects, cloud

administrators, and technical administrators of IT environments who want to implement CA

for a hybrid cloud infrastructure-as-a-service (IaaS) platform managing SAP landscapes. You

should be familiar with the VMware® vRealize® Suite, storage technologies, and general IT

functions and requirements, and how they fit into a hybrid cloud architecture.

To understand the concepts, architecture, and functionalities of the Federation Enterprise

Hybrid Cloud and how to operate SAP on top of it, you are advised to read the following

documents:

Federation Enterprise Hybrid Cloud 3.1: Foundation for SAP Solution Guide

Federation Enterprise Hybrid Cloud 3.1: Foundation Infrastructure Reference

Architecture Guide

Federation Enterprise Hybrid Cloud 3.1: Concepts and Architecture Solution Guide

Federation Enterprise Hybrid Cloud 3.1: Operations Solution Guide

Federation Enterprise Hybrid Cloud 3.1: Security Management Solution Guide

Organizations for which SAP is a major part of their core business operations must have a

business continuity management plan to manage risks in an IT environment and prepare for

recovery in the event of a disaster. Specifically, the plan should include measures to:

Safeguard the continuity of their business operations and protect revenue

Recover systems to at least a minimum level of operation if an outage occurs

A business continuity management plan has three components: backup, continuous

availability, and disaster recovery.

This guide focuses on the CA component. It describes a cloud computing solution that

increases the availability of an SAP landscape and provides rapid recovery after a severe,

isolated technical failure in one or more infrastructure or application components.


7

IT leaders are increasingly transferring mission-critical SAP applications to cloud

infrastructures to gain considerable cost savings, simplified management, and data center

efficiencies.

High availability and application mobility are essential for SAP applications. They provide

non-disruptive and live workload migrations within or across data centers, and implement

fully automated restarts when outages happen.

In traditional infrastructure designs, workloads across physical data centers are broken

when any of the infrastructure components fail, causing application downtime. Also,

traditional solutions are usually limited to specific combinations of operating

system/database software and hardware. These combinations make the solutions complex,

inflexible, costly, and difficult to implement and maintain.

To address these challenges for SAP applications, the Federation Enterprise Hybrid Cloud

offers:

High availability, which helps to manage risks in an IT environment and plans for

recovery when a technical failure occurs

Application mobility, which moves running workloads without disruption between

servers to avoid any downtime resulting from planned maintenance or a foreseeable

outage

This continuous availability solution uses EMC and VMware technologies, in particular EMC

VPLEX® and VMware vSphere® High Availability (HA), to enable continuous availability and

application mobility as a service within the Federation Enterprise Hybrid Cloud to provide:

Automated non-disruptive virtual machine movement within a cloud data center,

stretched across two separated data centers running SAP mission-critical systems

Simple automated infrastructure HA protection for complete SAP on-premises cloud

data center

Minimized RTO and near zero RPO

Additional benefits include:

Automatic restart of SAP instances in the event of server failures

Increased utilization of hardware and software assets:

Automatic load balancing between data centers

Zero downtime on maintenance

Table 1 defines the key terms relevant to this solution that are used in this document.

Terminology Table 1.

Term Definition

ABAP SAP central service (ASCS)

ABAP SAP Central Services. An instance in a distributed SAP system that hosts message server and enqueuer server

AccessAnywhere A technology of VPLEX distributed virtual storage to enable data access over distance within and between data centers.


8

Term Definition

Additional application server (AAS)

An SAP application server instance installed in a distributed SAP system

Application blueprint Logical topology of an application for deployment in vRealize Automation Application Services. An application blueprint captures the structure of an application with logical nodes, their corresponding services and operating systems, dependencies, default configurations, and network and storage topology requirements. The blueprint is published as a catalog item in the common service catalog.

High availability (HA) A system or component continuously operational for a desirably long length of time. Availability can be measured relative to "100% operational" or "never failing."

Logical template A predefined virtual machine definition in vRealize Application Services that can be mapped to a cloud template (and supporting services) in the cloud catalog

enabling an application blueprint to remain cloud-agnostic.

Primary application server

(PAS)

The first application server instance installed in a

distributed SAP system

VMware vSphere Metro

Storage Cluster (VMware vMSC)

VMware vMSC is a configuration option that allows for the

use of stretched clusters, meaning that servers within a cluster are spread across geographical locations. This configuration allows organizations to perform load balancing and nondisruptive live migrations between active data centers.

Chapter 2: Continuous Availability Solution Overview

9


Overview .......................................................................................................... 10

Solution architecture .......................................................................................... 10

Software resources ............................................................................................ 12


10

This solution is built on the proven design of the VMware vSphere and EMC VPLEX Metro™

storage cluster. Using the features and functionality of Metro, the solution offers:

Workload mobility

Disaster avoidance

Location-optimized storage

Increased uptime and utilization

Storage provisioning automation

Storage for the solution is provided by XtremIO, VMAX and VNX arrays located at both of

the data center sites that host the infrastructure. The storage is presented to the VPLEX

arrays, also at both sites, forming distributed storage kept synchronized and presented to

the vSphere cluster as active/active LUNs.

VMware vCenter Orchestrator™ is central to all the customizations and operations that are

used in this solution. It manages operations across several EMC and VMware products,

including:

VMware vRealize™ Automation

VMware vCenter™

EMC ViPR®

This solution focuses on the implementation, management, and automation of the

continuous availability for SAP applications on the Federation Enterprise Hybrid Cloud.

Figure 1 shows the overall logical architecture of the Continuous Availability solution for

Federation Enterprise Hybrid Cloud. Logical

architecture


11

Figure 1. Logical architecture

In this configuration, each pod is stretched across both sites in active/active fashion. The

underlying VPLEX distributed storage allows all the management components and workloads

to either proactively move before a known event using vMotion or reactively restart using

vSphere High Availability if an unpredicted failure event occurs.

Core Pod

The Core Pod is used to host a core set of resources that must exist before the remainder of

the cloud can be deployed. These core resources include vCenter Server, Microsoft SQL

Server 2012, and the vCloud Networking and Security or NSX Manager. The hardware that

hosts this pod need not managed by cloud components, but the virtual machines it hosts are

a critical foundation of the cloud.

Automation Pod

The Automation Pod hosts the virtual machines that automate and manage the cloud

infrastructure that supports the workloads consumed by the clouds tenants. The Automation

Pod supports the components responsible for functions such as the user portal and

automated provisioning, monitoring, metering, and reporting.


12

NEI Pod

The NEI Pod hosts the NSX Edge appliances and NSX Controllers or the vCloud Networking

and Security components, and becomes the convergence point at which the physical and

virtual networks connect.

Workload Pods

The Workload Pods are configured and assigned in vRealize Automation as shared resources,

to host SAP virtual machines deployed by the different business groups in the hybrid cloud

environment. These Workload Pods are deployed as VMware vSphere clusters in VMware

vCenter endpoints.

EMC GeoSynchrony® supports the concept of a VPLEX Metro cluster with Cross-Connect.

This configuration provides a perfect platform for a uniform vSphere stretched-cluster

deployment.

ESXi hosts can access a distributed volume on the local VPLEX cluster and on the remote

cluster in the event of a failure. When this configuration is used with VPLEX Witness, ESXi

hosts are able to survive through multiple types of failure scenarios. For example, in the

event of a VPLEX cluster or back-end storage array failure, the ESXi hosts can still access

the second VPLEX cluster with no disruption in service.

Figure 2 shows the physical architecture of uniform host access configuration with VPLEX

host Cross-Connect:

Figure 2. Physical architecture

Table 2 details the software resources that are used in the solution.

Solution software resources Table 2.

Software Version Purpose

Federation Enterprise Hybrid

Cloud, Foundation module

3.1 Customization package for storage as a

service (STaaS) and foundation workflows

VMware vRealize Automation Application Services

6.2.1 Application provisioning

Physical

architecture with

VPLEX host

Cross-Connect


13

Software Version Purpose

SAP provisioning module 3.1.0 Customization package for provisioning SAP systems

For a complete, up-to-date list of all software requirements for Federation Enterprise Hybrid

Cloud, refer to the EMC Simple Support Matrix (ESSM).

Chapter 3: Design Considerations

14


Overview .......................................................................................................... 15

SAP system architecture ..................................................................................... 15

SAP instances outage impact ............................................................................... 16

Considerations and best practices ........................................................................ 17


15

This solution is designed to eliminate single-points-of-failure (SPOFs) at the infrastructure

level, and to provide fast and automatic service recovery for SAP applications in case of

hardware failure. This chapter describes the SAP system architecture and analyzes the

impact of SAP instances outage. It also details the design considerations necessary to

minimize the duration and impact of service outages, and to automatically restore SAP

services.

This solution focuses on the SAP-specific design considerations, including:

Auto start of SAP instances

vSphere HA

vSphere Distributed Resource Scheduling (DRS) rule

For the general best practices of designing the CA solution on Federation Enterprise Hybrid

Cloud, refer to Federation Enterprise Hybrid Cloud 3.1: Concepts and Architecture Solution

Guide.

A typical SAP ERP 6.0 system consists of:

ABAP SAP Central Services (ASCS)

Application server instances

The first application server instance installed in an SAP system is called the Primary

Application Server instance (PAS). Additional Application Server (AAS) instances can

be installed later as needed to expand the compute resource capability.

Database (DB) instance

Also, a Global File System is required to provide shared access to /sapmnt and

/usr/sap/trans. In this solution, we enable the network file system (NFS) server on ASCS to

serve as the SAP Global File System. Figure 3 shows the SAP system architecture.

Figure 3. SAP system architecture


16

SAP PAS and AAS instances are installed to provide work processes such as dialog (DIA),

background (BGD), update (UPD), spool (SPO), and gateway. This enables workload balance

and protection against vSphere ESXi host or virtual machine failures.

The SAP ASCS, SAP DB, and SAP Global File Systems (NFS) server are SPOF components in

an SAP distributed architecture without additional operating system , DB-specific, or

complex-clustering technologies. Those application-level high-availability technologies can

be implemented with additional efforts, but their implementation is beyond the scope of this

document.

As shown in Table 3, this section analyzes the impact of SAP instances outage and

addresses these SPOFs.

Impact of SAP instances outages Table 3.

SPOF component Technical impact Business impact

Database server SAP system is suspended SAP service will become unavailable

Enqueue server Application transaction locks

are lost

SAP will become unavailable

Message server New connection or requests

cannot be executed

Users cannot login via

message server

RFC connections in other SAP systems via message server will be unavailable

Global File system /sapmnt/<SID>

Cannot start new application servers

Cannot write background job logs

Scalability of application servers will be affected

Tractability of background jobs will be affected

Global File system

/usr/sap/trans

Cannot use change and transport system

Development and customization will be affected

The whole SAP system is suspended if the SAP database instance component fails. When the

database becomes available again, the SAP work processes in the SAP application servers

reconnect automatically and users can resume their work. All in-flight transactions are rolled

back for consistency.

If the SAP ASCS instance component fails, both the enqueue and the message server

services become unavailable.

If the enqueue service fails, the SAP application transaction locks are lost, and the SAP

system is unavailable. If this happens, the SAP functional team must analyze the business

process impact after the service is restarted.

If the message server stops working, existing connections are not affected. New requests

cannot be executed for the dialog, update, and enqueue server.

If an SAP Global File Systems (NFS server) component fails, the NFS shares /sapmnt/<SID>

and the /usr/sap/trans are not available, with the following consequences:

NFS shares /sapmnt/<SID> not available: Additional SAP application servers are

prevented from starting. Active SAP application servers are not affected. The

background job will no longer be able to write logs under /sapmnt/<SID>/global.

SAP database

instance outage

SAP ASCS

instance outage

(enqueue and

message server

services)

SAP Global File

Systems


17

/usr/sap/trans transport directory is not available: The change and transport system

cannot be used.

In a large-scale cloud environment, a tenant may run hundreds of SAP systems. During

hardware failure, some SAP virtual machines are restarted by vSphere HA. To start a

distributed SAP system:

1. ASCS with Global file system

2. DB

3. PAS/AAS

Figure 4 shows the dependency among SAP instances.

Figure 4. Dependency among SAP instances

On the Linux operating system, however, the SAP and Oracle database, services are not

restarted by default until they are manually triggered. To eliminate manual intervention and

reduce recovery time, we developed the scripts to start each type of SAP instance

automatically on startup of the operating system.

For NFS services, the NFS client waits until the NFS server is available.

For the DB service, we developed shell scripts to monitor the availability of the

database in the background during boot up. Once the database is online, PAS/AAS is

started automatically.

The auto start of SAP instances can be enabled during provisioning by choosing Yes for the

ENABLE_AUTO_START parameter, as shown in Figure 5.

Auto start of SAP

instances


18

Figure 5. Enable auto start of SAP instances during provisioning

Admission control

Admission Control must be enabled. EMC recommends that you set the Admission Control

Policy to Percentage of Cluster Resources Reserved as failover spare capacity and

reserve 50% of CPU and memory for the DRS clusters where SAP production systems run.

This configuration reserves cluster compute resources to guarantee the restart of all virtual

machines to another surviving data center in the event of failure of a single data center.

VM restart priority

Override the VM Restart Priority according to the rules listed in Table 4.

VM restart priority for SAP instances Table 4.

SAP instance VM restart priority

ASCS High

DB Medium

PAS/AAS Low

If an ESXi host fails, vSphere HA powers on the SAP virtual machines according to the

virtual machine restart priority, from high to low. Although this setting does not guarantee

the proper start order of SAP services, it ensures that virtual machines with higher priority

are powered on first, avoiding the PAS or AAS instances being powered on before ASCS or

DB.

VM monitoring

When VM monitoring is enabled, vSphere HA restarts a virtual machine when both the

following conditions are met:

Heartbeats sent from VMware Tools are not received in a set period

The virtual machine is not generating storage or network IO for 120 seconds by

default. (You can change the default interval by using the das.iostatsInterval

advanced cluster level setting.)

vSphere HA and

DRS


19

VM monitoring is disabled by default. If VM monitoring is required, we recommend setting

the Monitor Sensitivity to Low.

vSphere DRS rule

We recommend creating vSphere DRS rules manually to separate SAP PAS and AAS

instances on different ESXi hosts. If the ESXi host running PAS or AAS instances fails, the

surviving PAS or AAS instances can provide redundancy of SAP work processes.

We also recommend creating vSphere DRS rules manually to separate SAP ASCS and DB on

different ESXi hosts in order to minimize the duration of service outage.

Federation Enterprise Hybrid Cloud 3.1 introduces a new feature called Site Affinity, which

uses VMware host Distributed Resource Scheduler (DRS) groups to subdivide the ESXi hosts

in each workload and management cluster into groupings of hosts corresponding to their

respective sites. The site affinity feature allows better network bandwidth utilization for a

distributed SAP system within one data center, avoiding additional network overhead across

two data centers. It does this by defining two groups in the format SiteName_Hosts where

the site names of both sites are defined during the installation of the Federation Enterprise

Hybrid Cloud foundation packaging.

VMware virtual machine DRS groups are also created in the format Sitename_VMs during

the preparation of the ESXi cluster for continuous availability.

Storage reservation polices created by the Federation Enterprise Hybrid Cloud storage as

service workflows are named to indicate the site in which that storage type is preferred to

run.

During composing of SAP provisioning services, the administrator will choose from a list of

storage reservation policies. Federation Enterprise Hybrid Cloud custom workflows use this

information to place the virtual machine on an ESXi cluster with access to the required

storage type as well as placing the virtual machine into the appropriate virtual machine DRS

group.

Virtual machine to host DRS rules are then used to bind virtual machines to the preferred

site by configuring the respective SiteName_VMs virtual machine DRS group to a value of

‘should run’ on SiteName_Hosts host DRS group. This ensures virtual machines run on the

required site, while allowing them the flexibility of failing over if the infrastructure on that

site becomes unavailable.

Figure 6 shows a simple example of two scenarios where SAP applications are deployed to a

VMware vMSC and how the logic operates to place those virtual machines on their preferred

sites.

Site affinity for

virtual machines


20

Figure 6. Deploying SAP applications with site affinity

Figure 7 shows how the virtual machine DRS groups and affinity rules might look in a

sample configuration.

Figure 7. Sample view of Site Affinity DRS Rule configuration

Chapter 4: Protecting SAP Applications with Continuous Availability

21


Overview .......................................................................................................... 22

Implementing Continuous Availability for SAP applications ...................................... 22

Validation.......................................................................................................... 26


22

This chapter demonstrates use cases to configure provisioning of an SAP system on

continuous availability (CA)-enabled infrastructure, including:

Composing SAP provisioning service on the CA-enabled infrastructure

Setting the cost for CA-enabled storage

This chapter also validates how to protect SAP applications from different failure scenarios

and demonstrates the flexibility of planned non-disruptive host maintenance. The following

scenarios are described:

Single ESXi host failure

One leg of VPLEX cluster failure

Single data center failure

Planned host maintenance

Compute resource balance

Before creating SAP provisioning services on CA-enabled infrastructure, follow the

Federation Enterprise Hybrid Cloud 3.1: Operations Solution Guide to configure CA services:

1. Provision CA-enabled storage on both data centers (Use case 2).

2. Create Reservation Policies (Use case 3).

3. Create a new Reservation to reserve CA-enabled storage (Use case 2).

4. Copy an existing SAP blueprint and specify the reservation policy and storage

reservation policy which connects to the CA-enabled infrastructure (Use case 4 and

6).

A CA environment presents new considerations for auto-provisioning SAP systems. After

configuring a reservation and blueprint in the vRealize Automation Center portal, you must

compose the SAP provisioning service on vRealize Automation Application Services to deploy

an SAP application on the CA-enabled infrastructure. All configurations in this section are

modifications and enhancements based on the existing components that were created as

described in Federation Enterprise Hybrid Cloud 3.1: Foundation for SAP Solution Guide You

must create and properly configure all of the components, including the cloud provider,

logical template, service, and application, for SAP provisioning.

To enable SAP provisioning service on the CA-enabled infrastructure:

1. Log in to the vRealize Automation application services portal as a service

architect.

2. Navigate to Cloud Providers and edit the cloud provider for HR business

group. Add the blueprints from vRealize Automation that use CA-enabled

infrastructure, as shown in Figure 8.

Use case 1:

Composing SAP

provisioning

service on CA-

enabled

infrastructure


23

Figure 8. Adding a cloud template from vRealize Automation blueprints

3. Navigate to Logical Templates. Edit the logical template SLES11 SP3 for SAP

Application and add the two cloud template mappings (see Figure 9) created in

step 2.

Figure 9. Adding cloud template mapping in the logical template

4. Navigate to Applications. Edit the SAP Distributed installation applications.

Navigate to a proper application version. For example, we navigated to SAP Distributed installation and clicked version 6.0.7. Create new deployment profiles to use the CA-enabled infrastructure.


24

As shown in Figure 10, we created a new deployment profile named “ERP 6.07 SR2

CA-Enabled” under the HR business group.

Figure 10. Creating new deployment profiles to use the infrastructure enabled for continuous availability

5. From the Deployment Profile Creation Wizard, map each node to the CA-

enabled cloud template that was added in step 3. For storage, select the CA-

enabled storage reservation policy created automatically by the cloud storage

provisioning service in vRealize Automation portal.

As shown in Figure 11, we mapped the cloud template SLES 11 SP3 for SAP

CA-Enabled to the ASCS and PAS instances, and mapped SLES 11 SP3 for SAP

DB instance CA-Enabled to the DB node. We also selected the S1-CA-Silver

CA-Enabled storage reservation policy for additional hard disks in each node.


25

Figure 11. Mapping cloud templates enabled for continuous availability

6. Finish the wizard with appropriate values for vCPUs, memory, and disk size for each

node.

7. Publish the deployment profile to the vRealize Automation portal.

8. Repeat steps 4 to 7 to create additional deployment profiles as needed. For example,

you may need to provision SAP systems on other storage tiers on different sites,

larger or smaller disk size.

9. Repeat steps 4 to 8 to configure SAP Standard Installation and SAP AAS instances.

10. Log in to the vRealize Automation portal as tenant administrator. Assign the

published SAP provisioning application to services and assign entitlements in

vRealize Automation portal.

End users can now request an SAP system, which will be provisioned on the CA-enabled

infrastructure.

This solution uses VMware vRealize Business, with its integration with vCenter and vRealize

Automation, to provide chargeback information on the storage provisioned to and consumed

in the hybrid cloud.

The ViPR storage provider running in vCenter automatically captures the underlying

capabilities of storage provisioned by ViPR. The storage administrator creates storage

profiles based on the storage capabilities of the ViPR storage, which is configured to provide

multiple storage service offerings. This integration enables vRealize Business to

automatically discover and group datastores based on predefined service levels of storage.

In this solution, we created separate virtual machine storage profiles for each of the storage

capabilities that were automatically captured by the ViPR storage provider, as shown in

Figure 12.

Use case 2:

Setting costs for

CA-enabled

storage


26

Figure 12. Creating a virtual machine storage profile

After the ViPR storage provider has automatically configured the datastores with the

appropriate storage capability, these datastores can be grouped and managed in vRealize

Business, according to their storage profile. Figure 13 shows that the cost profiles created in

vCenter are discovered by vRealize Business.

Figure 13. Setting the cost for CA-enabled storage

This enables the business management administrator to group tiered datastores provisioned

with ViPR and set the monthly cost per GB as needed to differentiate the cost of various

storage service levels. For example, to differentiate CA-enabled storages from other

storages, we set the monthly cost per GB of CA-enabled storage ”S1-CA-Silver” to $0.4 to

reflect the additional cost for VPLEX and other hardware required to build the solution.

In this section, we demonstrate how the CA-enabled infrastructure protects SAP systems

from different types of hardware failure. We simulated the following three hardware failure

scenarios:

Single ESXi host failure

Single leg of VPLEX and storage array failure

Single data center failure

We recorded the behavior of the SAP virtual machines during the hardware failures and

captured the duration of the SAP service restoration. We also simulated a planned host


27

maintenance, which allows non-disruptive host maintenance while keeping the SAP system

service availability.

In the test environment, six ESXi hosts were configured for a tenant resource cluster. Three

of the hosts came from data center A, and three from data center B. Table 5 shows the host

details.

List of ESXi hosts Table 5.

Data center ESXi hosts

A s1vccomp145, s1vccomp146, s1vccomp147

B s2vccomp45, s2vccomp46, s2vccomp47

The two data centers were configured according to the solution architecture, as shown in

Figure 1 and Figure 2. In order to simulate the worst connection between two data centers,

we configured a round trip latency of 10 ms for both the SAN and IP networks using a WAN

simulator.

We provisioned a 1 TB data center A preferred CA-enabled storage and a 1 TB data center B

preferred CA-enabled storage for the HR business group. We deployed two distributed SAP

systems with CA1 on data center A and CA2 on data center B. Table 6 and Table 7 list the

details of the SAP systems.

CA1 SAP system details Table 6.

Instance Virtual machine name

ASCS sapca1ascs

DB sapca1db

PAS sapca1pas

AAS sapca1aas

CA2 SAP system details Table 7.

Instance Virtual machine name

ASCS sapca2ascs

DB sapca2db

PAS sapca2pas

AAS sapca2aas

vSphere DRS rules were created to prevent the SAP DB and ASCS instances from running in

the same host. This is because both instances are SPOFs and must be protected from ESXi

host failure. Additional rules were created to keep SAP Application Servers running on

different ESXi hosts to provide redundancy for the SAP work processes. Table 8 lists details

about the vSphere DRS rules.

List of vSphere DRS rules Table 8.

DRS rule name Members Rule Type

SAP Application Server CA1 sapca1pas, sapca1aas Separate

SAP DB and ASCS CA1 sapca1db, sapca1ascs Separate

SAP Application Server CA2 sapca2pas, sapca2aas Separate


28

DRS rule name Members Rule Type

SAP DB and ASCS CA2 sapca2db, sapca2ascs Separate

S1_VM_to_Host_DRS_rule Virtual machines:

sapca1pas, sapca1aas, sapca1db, sapca1ascs

Hosts:

s1vccomp145, s1vccomp146, s1vccomp147

Should Run

S2_VM_to_Host_DRS_rule Virtual machines:

sapca2pas, sapca2aas, sapca2db, sapca2ascs

Hosts:

s2vccomp45, s2vccomp46, s2vccomp47

Should Run

Test scenario

This test scenario validates how the SAP virtual machines on a failed host can be restarted

on the surviving ESXi hosts when an unexpected ESXi host failure occurs in a tenant

resource cluster. To simulate this hardware failure scenario, we powered off two of the ESXi

hosts.

Objectives

Verify that the virtual machines running on the failed ESXi hosts are restarted by

vSphere HA on the surviving ESXi hosts, with the proper start order.

Verify that the configured DRS affinity rules are followed when placing the virtual

machines after the failure.

Procedure

1. Migrate the PAS and DB instances of SAP system CA1 to the same ESXi host

s1vccomp147.

2. Migrate the ASCS and AAS instances of SAP system CA2 to the same ESXi host

s2vccomp47.

3. Power off ESXi host s1vccomp147 and s2vccomp47.

4. Verify that vSphere HA automatically detects that the ESXi hosts have failed and

that VSphere HA initiates a restart of the SAP virtual machines on other ESXi hosts.

5. Check Oracle database startup log in /home/ca1adm/startdb.log

6. Check the log file /usr/sap/<SID>/DVEBMGS00/work/available.log to determine the

duration of SAP service unavailability.

Results and Analysis

Table 9 shows the observed behaviors of the system and reference metrics in minutes and

seconds (mm:ss) when ESXi hosts failed.

Use case 1: Reference metrics for ESXi failure Table 9.

Virtual

machine

ESXi host

after failure

VMHA restart

duration

(mm:ss)1

Service outage

duration

(mm:ss)2

DRS rules

compliance

CA1 DB s1vccomp145 0:44 2:30 Compliant

CA1 PAS s1vccomp146 0:47 3:28 Compliant

CA2 ASCS s2vccomp45 0:47 1:37 Compliant

1 VMHA restart duration measures the time from ESXi host failure to successful trigger of SAP virtual machine

power on. 2 Service outage duration measures the time from ESXi host failure to successful restart of services on SAP

instances.

Use case 3:

Single ESXi host

failure


29

Virtual machine

ESXi host after failure

VMHA restart duration

(mm:ss)1

Service outage duration

(mm:ss)2

DRS rules compliance

CA2 AAS s2vccomp45 0:49 3:18 Compliant

This use case demonstrates that vSphere HA automatically reacts to the vSphere ESXi host

failure incident and restarts the affected virtual machines on the available vSphere ESXi

host. vSphere DRS, with the configured affinity and anti-affinity rules, immediately,

automatically, and non-disruptively adjusts the placement of the restarted virtual machines

in the surviving ESXi hosts in the respective cluster, restoring the previous high-availability

service level provided.

Test scenario

This test scenario validates that, in the event of an unexpected VPLEX cluster isolation, the

SAP systems continue services without interruption. To simulate this isolation scenario, we

powered off VPLEX in data center B. With VPLEX powered off, we can assume the storage

that connects to the failed VPLEX is also unavailable.

Objective

Verify that there is no service interruption for the SAP systems running on data center

A.

Procedure

1. Power off VPLEX cluster-2 and storage in data center B.

2. Check the path status on datastore.

3. Verify the VPLEX cluster-1 functionality in data center A.

4. Verify the status of SAP Load Generator and check the log file

/usr/sap/CA2/D00/work/available.log to verify that there was no service

interruption during the outage of VPLEX.

Results and analysis

When VPLEX cluster-2 in data center B are powered off at 10:51:35, the VPLEX Witness

detects failure of VPLEX cluster-2.The storage served by VPLEX cluster-1 on data center A

remains available, as shown in Figure 14.

Figure 14. VPLEX status after data center A VPLEX isolation

Due to the VPLEX Cross-Cluster Connect configuration, EMC PowerPath®/VE sets the Cross-

Cluster paths on hosts at data center B to auto standby proxy before the isolation. When the

isolation event occurs, PowerPath/VE can detect the isolation of the VPLEX at data center B

and claims the paths are unresponsive. When the cluster is recovered, PowerPath/VE

automatically recovers the dead paths. Figure 15 shows the behavior of PowerPath/VE

during the isolation event.

Use case 4: One

leg of VPLEX

cluster failure


30

Figure 15. PowerPath status after VPLEX isolation

Since the ESXi hosts are connected to both VPLEX clusters, PowerPath/VE simply reroutes

the I/O to the surviving path.

During one leg of VPLEX cluster failure, the SAP CA2 system remains unaffected. No SAPGUI

drop is observed. Figure 16 shows that the SAP CA2 system remains available after

10:51:35, the time we powered off the VPLEX cluster-2 in data center B.

Figure 16. No interruption of SAP CA2 after VPLEX cluster-2 failure on data center B

This use case demonstrates that:

Even after one leg of VPLEX cluster failure, the SAP workload is not interrupted during

the transparent failover of the storage services from the VPLEX in data center B to

data center A.

PowerPath/VE identifies the failed paths and redirects the Fibre Channel (FC) traffic

through the active paths, which are connected to the VPLEX storage on data center A.

This makes the VPLEX failure transparent to the SAP application running inside the

SAP virtual machines on data center B, as well as avoiding the need to restart virtual

machines on other ESXi hosts, which would cause downtime.

Test scenario

This test scenario validates that, in the event of a complete data center failure, all of the

SAP virtual machines running on that data center are restarted on the remote data center.

To simulate this hardware failure scenario, we powered off all ESXi hosts and the VPLEX in

data center B.

Objective

Verify that the virtual machines running on the failed ESXi hosts in data center B are

restarted by vSphere HA on the hosts in data center A in the proper start order.

Verify that the configured DRS affinity rules are followed when placing the virtual

machines on the remote data center after the failure.

Use case 5:

Single data

center failure


31

Procedure

1. Power off the ESXi hosts and the VPLEX cluster-2 in data center B.

2. Verify that the ESXi hosts failure is detected by vSphere HA. vSphere HA

automatically detects that the servers have failed and then initiates a restart of the

SAP CA2 virtual machines.

3. Verify that the SAP CA2 instances are restarted in the proper order.

4. Check Oracle database startup log in /home/ca1adm/startdb.log

5. After automatic startup of SAP CA2, check the log file

/usr/sap/CA2/DVEBMGS00/work/available.log to determine the duration of the SAP

service unavailability.

6. After VPLEX cluster-2 in data center B is online, verify that SAP CA2 virtual

machines are automatically migrated to ESXi hosts in data center A according to

site affinity rule.


Table 10 shows the observed behaviors of the system and reference metrics in minutes and

seconds (mm:ss), when the ESXi hosts failed.

Use case 5: Reference metrics for single data center failure Table 10.

Virtual

machine

ESXi host

after failure

ESXi host

after VPLEX cluster-2 is restored

VMHA

Restart duration

(mm:ss)

Service

outage duration

(mm:ss)

DRS rules

Compliance

CA2 ASCS s1vccomp147 s2vccomp46 0:56 2:23 Compliant

CA2 DB s1vccomp145 s2vccomp47 0:52 4:03 Compliant

CA2 PAS s1vccomp147 s2vccomp47 0:57 5:13 Compliant

CA2 AAS s1vccomp145 s2vccomp46 1:10 4:57 Compliant

This use case demonstrates that vSphere HA automatically reacts to the vSphere ESXi host

failure incidents and restarts the virtual machines on the available vSphere ESXi host. With

VPLEX, the SAP virtual machines are restarted on the other data center. vSphere DRS, with

the configured affinity and anti-affinity rules, immediately, automatically, and non-

disruptively adjusts the placement of the restarted virtual machines in the surviving ESXi

hosts in the respective cluster, restoring the previous high-availability service level provided.

Test scenario

Planned maintenance includes hardware maintenance, hypervisor maintenance, installation

of patches, and upgrades of the server BIOS, drivers, or the hypervisor itself. All these

activities can cause downtime in your environment. This solution provides the benefit of

allowing SAP or other workloads to be non-disruptively moved across data centers when

planned downtime results from an external event, such as a power upgrade planned by the

utility company.

This test scenario validates that no SAP service disruption occurs during planned downtime

of ESXi hosts. To simulate this scenario, we put all of the ESXi servers in data center B into

maintenance mode.

Objectives

Verify that the SAP virtual machines, which run under an SAP workload on the ESXi

host set on maintenance mode, are non-disruptively migrated across data centers.

Verify that the vSphere DRS affinity and anti-affinity rules are enforced by vCenter

during the migration of the SAP virtual machines.

Use case 6:

Planned host

maintenance


32

Procedure

1. Log in to SAP CA2 using SAPGUI. Run SAP Load Generator (t-code sgen) to

generate workload.

2. Temporarily disable Admission Control in the vSphere HA settings to make sure

virtual machines can be migrated to ESXi hosts in data center A.

3. Select all ESXi hosts in data center B and put them into maintenance mode.

4. Verify that SAP CA2 virtual machines are migrated data center A. Verify that SAP

CA2 is not interrupted. Verify that DRS rules are followed.

5. Select all ESXi hosts in data center B and exit maintenance mode.

6. Verify that SAP CA2 virtual machines are migrated back to data center B. Verify that

SAP CA2 is not interrupted. Verify that DRS rules are followed.


All SAP virtual machines are migrated to data center A in less than one minute without any

service interruption, and DRS anti-affinity rules are complied with, as shown in Figure 17.

Figure 17. Non-disruptive host maintenance

This use case demonstrates that:

A data-center-wide planned host maintenance can be performed non-disruptively and

on demand without affecting the planned downtime commitments on your current

service level agreements (SLAs).

vSphere DRS automatically enforces the affinity rules configured to keep the SAP

virtual machines’ ASCS, DB, AAS, and PAS instances in separate physical vSphere ESXi

hosts, maintaining the same protection level that exists before the hosts are put into

maintenance mode.

An administrator can proactively avoid the impact of a foreseeable outage, to preserve

the availability of the business-critical SAP applications. Business-critical SAP

applications can be moved live and within seconds to a remote data center, outside

the area to be affected by the foreseeable outage.


33

Test scenario

This test scenario validates that, in the event of compute resource contention on ESXi hosts,

vSphere DRS can balance host resource utilization by migrating SAP virtual machines to

other ESXi hosts. To simulate this scenario, we stressed ESXi host CPU resources by running

SAP Load Generator (t-code sgen).

Objectives

Verify that when the resource utilization of a tenant resource cluster becomes

imbalanced, vSphere DRS can trigger non-disruptive virtual machine migration to

rebalance resource utilization.

Verify that the vSphere DRS affinity and anti-affinity rules are enforced by vCenter

during the non-disruptive migration of the SAP virtual machines

Procedure

1. Migrate DB and PAS instances of SAP CA2 to one ESXi host (s2vccomp45).

2. Verify Automation Level is set to Fully Automated in vSphere DRS settings for the tenant resource cluster.

3. Log in to SAP CA2. Run SAP Load Generator (t-code sgen) to generate workload on both SAP systems.

4. Verify that the resource utilization of the tenant resource cluster becomes imbalanced.

5. Verify that vSphere DRS can trigger non-disruptive virtual machine migration to balance resource utilization.


During the process of compute resource balancing, there was no service interruption of

either SAP system, and the vSphere DRS rules were followed. Table 11 shows the observed

behaviors during ESXi host resource balancing.

ESXi host resource balancing Table 11.

SAP virtual

machine

ESXi host before

migration

ESXi host after

migration

DRS rule

compliance

CA2 DB s2vccomp45 s2vccomp45 Compliant

CA2 PAS s2vccomp45 s2vccomp46 Compliant

This use case demonstrates that vSphere DRS can balance compute resource utilization

while maintaining SAP service availability.

Use case 7:

Compute

resource balance

Appendix A: References

34


Conclusion ........................................................................................................ 35


35

The Federation Enterprise Hybrid Cloud for SAP integrates the best of EMC and VMware

products and enables customers to continuously protect SAP system landscapes from events

ranging from single host failure to single data center outage. vSphere HA can restart SAP

instances within a few minutes of a hardware failure.

With the integration of EMC ViPR, vCO, and vRealize Automation, cloud administrators can

request CA-enabled storage through the vRealize Automation self-service portal. End users

can provision SAP systems on CA-enabled infrastructures with just a few clicks.

VPLEX Cross-Connect enables high availability of SAP systems in case of VPLEX and storage

failure in one of the data centers.

With the VMware metro storage cluster powered by EMC VPLEX and vSphere, Federation

Enterprise Hybrid Cloud can provide non-disruptive planned host maintenance and resource

balance across data centers.

This solution provides these benefits for protecting SAP systems:

Eliminates single points of failure across all infrastructure layers

Eliminates recovery time in case of a single VPLEX or storage outage by using VPLEX

Metro with Cross-Cluster Connect

Minimizes downtime, with less than 6 minutes RTO and zero RPO, and provides high

availability across datacenters

Enables mission-critical business continuity for SAP applications running across local

and remote data centers


36

This appendix presents the following topics:

References ........................................................................................................ 37


37

These documents are available on www.emc.com. Access to Online Support depends on

your login credentials. If you do not have access to a document, contact your Federation

representative.

Federation Enterprise Hybrid Cloud 3.1: Foundation Infrastructure Reference

Architecture Guide

Federation Enterprise Hybrid Cloud 3.1: Concepts and Architecture Solution Guide

Federation Enterprise Hybrid Cloud 3.1: Operations Solution Guide

Federation Enterprise Hybrid Cloud 3.1: Security Management Solution Guide

Federation Enterprise Hybrid Cloud 3.1: Foundation for SAP Solution Guide

EMC Cloud-Enabled Infrastructure for SAP: HA and Application Mobility Bundle

These documents are available from www.emc.com or EMC Online Support. Access to online

support depends on your login credentials. If you do not have access to a document, contact

your EMC representative.

Implementation and Planning Best Practices for EMC VPLEX Technical Notes

EMC VPLEX Metro Witness Technology and High Availability

Using VPLEX Metro with VMWare High Availability and Fault Tolerance for Ultimate

Availability

VMware KB Topic 2007545: Implementing vSphere Metro Storage Cluster (vMSC)

using EMC VPLEX

For additional information on SAP HA, refer to the following documents, available on the SAP

support portal.

High Availability Solutions for SAP on VMware

SAP Note 1380654 - SAP support in public cloud environments

SAP Note 1492000 - General Support Statement for Virtual Environments

SAP Note 1122388 - Linux: VMware vSphere configuration guidelines

SAP Note 98051 - Database Reconnect: Architecture and function

SAP Note 24806 - Database Reconnect: technical details and settings

SAP Note 1122387 - Linux: SAP Support in virtualized environments

SAP Note 1122388 - Linux: VMware vSphere configuration guidelines

SAP Note 0171356 - SAP software on Linux: Essential information

SAP Note 1310037 - SUSE Linux Enterprise Server 11: Installation notes

EMC Solution

Guides

Product

documentation

Other

documentation

http://www.emc.com/

http://www.emc.com/

http://support.emc.com/

https://service.sap.com/sap/support/notes/1380654








federation enterprise hybrid cloud 3.1 - data protection ... · pdf filefederation enterprise...

Documents